Question of the Month: How important is test automation to engineers?

Test automation in production is crucial. Test automation in R&D and product development is seldom worth the effort.

That's an interesting take Doug.  

By "in production" do you mean a product that's about to be released to production manufacturing, or a product actually deployed in production which you automate health checking for to ensure it's working properly?  

Test automation in R&D and development is absolutely and always worth the effort - we would live and die by the quality of, and automation of, testing: it's such a boring, repetitive activity that not automating would reduce the testing quality and efficacy significantly.  Not only that, automation reduces the actual testing time, directly impacting resource costs.  Not all tests can be automated of course and those are the ones that need careful monitoring.  I did use to have the argument with less experienced colleagues along the lines of "whilst we're creating these tests, we could just be getting one with it" but not for very long.  I also used to have an internal philosophical dialog along the lines of "who's testing the tests and do we need to create tests to test the tests" but then I went insane and other voices took over  

That's an interesting take Doug.  

By "in production" do you mean a product that's about to be released to production manufacturing, or a product actually deployed in production which you automate health checking for to ensure it's working properly?  

Test automation in R&D and development is absolutely and always worth the effort - we would live and die by the quality of, and automation of, testing: it's such a boring, repetitive activity that not automating would reduce the testing quality and efficacy significantly.  Not only that, automation reduces the actual testing time, directly impacting resource costs.  Not all tests can be automated of course and those are the ones that need careful monitoring.  I did use to have the argument with less experienced colleagues along the lines of "whilst we're creating these tests, we could just be getting one with it" but not for very long.  I also used to have an internal philosophical dialog along the lines of "who's testing the tests and do we need to create tests to test the tests" but then I went insane and other voices took over  

I also think that if you wait with test automation until production time, you are only harvesting half the apple. You should think about executable test options as soon as you start to spec. What's the point of letting a highly skilled and experienced engineer do repetitive tasks?

Absolutely DFT (design for test) of both prototypes and production should be part of the design process from the start, but that doesn't mean automated test must be used in development. Designing the automated test systems to be used in manufacturing a product is a project in itself that can be started in parallel with product development, but it is costly to keep redesigning the automated test systems as the product evolves through development. For example I've seen a designer revise his complex PCB over 900 times before it was released for manufacturing. The card was designed to be tested with an automated bed-of-nails system, but it would have been prohibitively expensive to design hundreds of beds-of nails and associated automation software for each revision and then use the system for a few minutes before it becomes obsolete. Meanwhile perfecting each bed-of-nails system would massively delay the main project when an hour or two of manual probing tells you everything you need to know. When it takes longer to design and perfect an automated test system than it takes to run all the tests you will ever run, it is usually not worth it. When you can design and perfect an automated test that saves on total test time, is more consistent and reduces errors, then of course it is worth considering it.