TDD and Embedded Programming

Hi Michael,

I have always used TDD and am very happy with the results.

I have noticed that a lot of other developers underestimate the time and effort needed to test and verify the software.

By leaving it to the end, they are always hard pressed to complete proper testing.

The lack of testing is very apparent when they start getting bug reports from the users.

So plan for testing early and often.  It will save you a lot of time in the end and make your products better.

DAB

Do you mean this thread?  We're looking for Top Members who want to help STEM educators!

I haven't officially used TDD yet, although with a brief review of the wikipedia page, it appears that I somewhat follow the idea.  I always define the baby steps needed to accomplish a big task, and write code to accomplish each of those steps.  Some of the old time coders around here don't like the way I do things, as there is a very short cycle involved.  I'm constantly starting and stopping things, making sure one thing at a time works before moving to the next.  Essentially, my view is that I'm changing the minimum number of variables.  I think this is especially important when working in an interpreted language such as Python, as it's far too easy to add a bunch of control structures that don't get tested until you've deployed the thing and someone does something you don't expect and didn't test for, resulting in a runtime error.  Doesn't look good...

I find the Grenning blog entry #458 "Engineers and Programmers, Stop Writing So Much Firmware" fascinating.  It's always a balancing act.  As we say here at the day job, we might not have time to do it right the first time before delivery, but we always have the time to redo it.  This is driven by the customer who wants things and wants them now.  I know some people are able to get on the Agile high horse, and stick to their principles, but I don't think that's applicable to most of the real world, where lowest bidder wins.  Also, depending upon the market/client/customer, one might not wish to produce something open and easily maintainable.  Occasionally, we get paid to build something that is explicitly open and maintainable, but naive idealism aside, the reality is that it's easier to make something that ensures job security.  Just sayin'...  LOL

A thought though, used properly, couldn't firmware really be the place to write the code that would keep compatibility between hardware and software?

@DAB, one of the thoughts that I keep running into is that design takes about half of a projects resources, and testing/debug takes about a third, leaving a really small window for the actual coding.  I think Agile-related ideas are good at getting the mingle started early in the process.  Seems the same with TDD, right?

Yes, Doug, that was the thread I was referring to.

Mike

Actually, coding is only about 10% of the job, so you should scale your estimates accordingly.

The less time you spend doing the up front test planning results in x time multipliers in wasted testing and retesting and retesting time.

DAB

OK, so adding things up:

design 50%

coding 10%

test/debug 33%

Is the last 7% for feeding trolls on Element14?

OK, TDD sounded interesting from the above comments so I took a look at Wikipedia.

First impression: Yikes!

Second impression: Sounds like a hacker's dream methodology -- write code that works for the test cases, and who gives a flying bit whether it works for anything else.  [Update: By "hacker", I mean someone who writes code by "hacking away" in a non-professional way until it seems to work.  IIRC this was the most common use back when I was a student, e.g., "he's not a computer scientist -- he's just a hacker."  I don't know if this meaning is still in use.  In any case, since "hacker" means so many different things nowadays it's IMO always necessary to define it.]

Third impression: This reminds me of my assembly-language programming class back in Summer 1974.  We used Don Knuth's MIX assembly language.  Our last assignment was to write a subroutine to do a specified function.  The instructor had the main program that tested the subroutine, and you didn't know what it would do.  So you had to write a subroutine to do the specified function correctly (including all the special cases), do your own testing, but then hand the subroutine in as a deck of cards which the instructor would run to get the final grade.  Concentrates the mind wonderfully, as they say.

The third impression is like real life.  Your code goes out to users, who will discover all sorts of things you didn't test.  If you write code to make a handful of tests work instead of thinking through the complete problem domain, you can expect a disaster when you ship IMO.

Fourth impression: Maybe I'm missing something?

Fifth impression (update): Does Jive use this methodology?  That would explain a lot

JMO/YMMV

Hi John,

I definitely think you missed something. Realize the design/development part of this is at the unit level. You still have system level integration tests and all the other things that are important to ship code with the fewest amount of bugs possible.

As i mentioned in the earlier thread, we adopted tdd about 8 (maybe 9 by now) years ago and we have had 5 releases over that period of time. The people that track these things tell me this project has the lowest defect rate of all previous releases. Our defects tend to be more along the lines of missed requirements or improperly specified requirements as opposed to broken software. Although I don't believe this is a valuable metric other than relative size, we are over 2 million lines of code. One of the interesting things, is with each new release, even with added functionality, our code size doesn't grow very much. We are able to reuse code that we have written over time, perhaps with a slight adaptation to allow for new functionality, without worry, because we have the tests to prove that it still works as it did when first written.

One of the things that I have found during this period of development is that in order for things to be testable, they have to be decoupled. This allows our modules to be well defined, and specific.

I have heard many of the same comments about hackers paradise whenever people first hear about this methodology, I think I felt this way at first, but ultimately it takes quite a bit of discipline to work in this manner.

It's not for everyone, and there is no magic bullet in software development, but this is a technique that has worked for me and my team. It gives us the confidence to make changes to the code, to make it clearer and more readable. To make adjustments to facilitate performance, and to make changes when new requirements are introduced. We have a large collection of tests that run in a few minutes, 17000+ test methods, and who knows how many assertions inside those methods.

as you say YMMV  ,

Mike

Hi All,

IMHO,

TDD and Agile are just descriptions of a two SDLC methodologies (that are not necessarily poised against each other).

I had a look at the TDD Wiki article and had a laugh because I found it incomplete and quite corny. (e.g. Who tests the tests? - Does the test dictate the desired result?)

I also had a look at an officially released excerpt of Test-Driven Development for Embedded C and within that excerpt I found errors in it.

The excerpt describes many of the principles that I have used since the 1970's that are still just as relevant right now!.

Many of the new methodologies such as TDD and Agile are not new at all and have their genesis from techniques from 1957 and before.

People (including egos or lack of appropriate skills) and business drivers (including $$$ & time to market) are diametrically opposed to the methodology principles and as such can taint or corrupt SDLCs and reduce the quality of their output.  On the other hand, you can impose too many bureaucratic measures and stifle progress.

Irrespective of what is used the bottom line is that one needs the appropriate communication and comprehension thereof, controls, skills, standards, enthusiasm and discipline in place to produce a reliable and resilient product.

"It is far to easy to get stuck in such irrelevant fluff!" - Very Compact

"None of my inventions came by accident. I see a worthwhile need to be met and I make trial after trial until it comes. What it boils down to is one per cent inspiration and ninety-nine per cent perspiration." - Thomas Edison

"Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it."- Brian W. Kernighan and P.H Plauger

"Short Cuts can only be found at the Butcher Shop." - Tommy Emmanuel

Very Compact

Irrespective of what is used the bottom line is that one needs the appropriate communication and comprehension thereof, controls, skills, standards, enthusiasm and discipline in place to produce a reliable and resilient product.

 

 

 

I agree, that is the most important take away from this. You can get bad code or a bad product from any methodology if the principles you mention are ignored.

I also love the Kernighan quote.

Mike

I've not tried TDD with embedded projects yet but I have just started using it for work projects (so I'm no expert)

I find it helps in two key ways:

1. It makes you think more about what you are trying to achieve
2. It allows you to make quite bold changes without having to be scared you are going to break stuff.

We had a guy in the office the other year who liked to talk about "Technical Debt" and I think TDD really helps elimitating that.