RoadTest Review Writing: What Happens with the RoadTest Doesn't Work Out As Planned?

In my opinion, road testers are not obliged to create a working product demonstration. They are there to test the ins and outs. Sure, it is generally the intention to develop a project of sorts as I find that is the best way to learn - i.e. when you have a problem to solve.

So, in my opinion, the emphasis of approach is to document the difference between your user assumptions and intentions made at time of first reading about the product, and the intended solution, and the actual user experience getting to know a product, when trying to solve the particular problem. Where did you come unstuck. Was it project documentation, was it a hardware fault or firmware bug etc. or was it simply too complex (they over complicated things).

So, it is the development journey that matters for engineers and developers (as this helps determine development risk) and not simply a working product demo, which merely satisfies the marketing crowd and twitter feeds.

Just wanted to chip in my 2c, as usual:

In my experience, the truth is that most RoadTests never quite go to plan. Just like life . But the show must go on ...

Whenever I write my proposal, I think of reasonable things to try with a given product, features that I want to test and more. But there's always a disclaimer - because often times, things might be different when the device lands in your hands. It could arrive damaged, you might find you need some particular "part" to make it work, you may learn of problems with functionality (software/hardware), limitations that you weren't aware of, or come up with even better ideas and find more interesting things to test.

The important thing to do is to keep going and try your best not to let a slight detour get the whole RoadTest derailed. You have a strict two month deadline - not forgetting you'll need some of that time to prepare visual assets and write the final review. We're all counting on you!

I've done a fair share of RoadTests and I can honestly say, I've come across a lot of situations which have required a bit of a detour. This might mean a combination of the following:

• Contacting rscasny, danzima and explaining the situation, documenting the problems and complying with procedures (e.g. my package was damaged in the Rohde & Schwarz HMP4040.04 RoadTest).
• Doing some research on your own and finding out the best way forward (e.g. I need to buy an an accessory, and I don't mind spending the money, so I'll go and buy it - but read the documentation first! e.g. in my Molex Antennas RoadTest).
• Asking your peers for help - you can leave messages on the RoadTest product page, or perhaps even message other fellow RoadTesters directly via private message (assuming you have the privileges). As a RoadTester, it might be useful to help your fellow peers out, by verifying issues (e.g. during the Harting MICA RoadTest, the help of Jan Cumps and stevesmythe was invaluable in troubleshooting an issue that only I had).
• Trying something else in the meantime - don't waste the time, maybe there's other tests you can do without a particular functionality/part/etc. Once you overcome the vital blockage, you can circle back and try your idea again.
• Face the problem head-on - perhaps it's not worth waiting for the manufacturer to provide everything (e.g. a sample application), so perhaps it's worth taking your time to write one yourself and help everyone out (as I have done for the presently-in-review B&K Precision DAS240-BAT Multi-Channel Recorder where I implemented my own Modbus client using pymodbus as their example was not relevant).
• Contact the representative, or failing that, the manufacturer - sometimes there are some problems which are worth feeding back to the company for product improvement. Of course, consult with Randall/Dan first - but I've found a number of instances where I had to go directly to the manufacturer and sometimes it can be quite rewarding - e.g. in the case of my Rohde & Schwarz RTM3004 RoadTest, several new firmware revisions resulted.
• Omitting the experiment entirely or substituting something else of "equal" value - perhaps ultimately the experiment has to be abandoned, to be revisited later in a blog (e.g. a teardown and attempted repair of the damaged HMP4040.04). But you should probably look at doing something else that could be just as interesting.
• Negotiating a delay in review - sometimes the problems are so big in scope that a delay in review delivery is warranted (e.g. working with manufacturer), so as long as all parties agree, perhaps the review delivery could be delayed (e.g. my RoadTest of the Tektronix PA1000 Power Analyzer).

At the end of the day, the review doesn't have to be "all good things" - the best reviews have a mixture of strengths, weaknesses, frustrations, areas for improvements and highlight key applications for given products and differentiating factors compared to their competitors. It also doesn't have to be "all unique" - if someone else has already put up their review which covered something you were intending to do, it doesn't mean you shouldn't report your own findings as well. It would be good to put up something you can stand by as your experience of a given product, that would inform new prospective purchasers about the product and is a fair exchange for the product and two months of work. After all, your future RoadTest eligibility and reputation in the community is riding on your report, so make it a good one.

- Gough

Hi Gough,

thanks, your 2c are entirely in accordance with my own experiences.

Gerrit