I read that the question in my title was posed by a reader in my discussion on: Do You Know Enough or Are You Good Enough To Be a RoadTester? I think this question has been answered various times in the past by experienced roadtesters themselves. But the question came up again, and I think it somewhat relates to Road Test Fears and influences whether someone applies or doesn't apply, or completes or doesn't complete their review. So, I'm going to take a shot at answering it from my point of view as the RoadTest Program Manager.
Prior to my assuming the role of RoadTest Program Manager, I noticed that the roadtest application was kind of sparse. It just asked, why do you want to roadtest this product? I think that can work for some products, like the Raspberry Pi, which as an extensive following on element14 and people see the roadtest as a de facto project giveaway. But there are a lot of new products being launched where the knowledge level varies. My concern was the very question I am posing in the title: you get a product for a roadtest that you know nothing about: what do you do? The way I resolved this issue was to ask a series of pointed questions in the application that I believe help a roadtester clarify his or her thoughts so they never enter panic mode. This is where the question about the testing procedure came from. If you provide a testing procedure you have a starting point or a roadmap for the review.
So, if you draw a mental blank once you get the roadtest product, go back to your application. Re-read your testing procedure. If something isn't right, modify the procedure. Ask yourself, what about your testing procedure is not working for you? Think about ways you can break up the testing procedure into mini-tests. Then, methodically try to complete these mini tests one by one. (If it isn't obvious on how much I rely on your testing procedure in selecting you, then this paragraph demonstrates it. An application with a weak testing procedure has less value than a detailed one. I do it to help the actual roadtesters so they don't draw a blank.)
Ask somebody for help. It could be a colleague, a professor, another element14 member. Hashing out a problem with someone else usually helps people clear up your thoughts (and doubts) and gives you the fortitude to progress, no matter what the endeavor. Part 2 of this question is somewhat of a risk on my part, but I will say it anyway and let the members respond: if you see someone who is an experienced roadtester, try to message or contact them. Introduce yourself. Tell them you have seen their reviews and you were impressed and tell them the reason you are writing is you need some advice about a roadtest. Explain your concern. Be honest, sincere and professional. I think there is a good chance that an experienced roadtester will respond to your request for advice.
Ask me. That's part of my job. Message at rscasny
Get back to the basics. You are conducting a technical review of the product. So, every roadtest whether explicitly or implicitly answers three questions: Does it work? Is the product easy to use? And is the documentation correct? Those three questions tell a lot. A sponsor obviously want to see Yes to all these questions. But if there is a No, and you can support a No in your roadtest, they would your review to be useful and valuable, as well.
The last thing I would say is to ask yourself, is my roadtest complex or simple. I think complex roadtests are probably hard to pull off unless you are an experienced professional and experienced roadtester. If it is too complex, simplify it.
Randall Scasny
RoadTest Program Manager
