RoadTests: Project vs Unboxing vs Experiment vs Other

Hopefully nobody wants you to track every hour you work on the review. Bu an understanding of how much effort goes into a test, and what comes out of it (and how good the original estimation was) can be helpful to select testers being able to pull of what the propose. Even a question like "I spend more / less / exactly the planned amount of time" can be helpful.

I was thinking more of a case of getting some metrics so people who've not yet done a roadtest might have an indication of how much time is required.

I find one of the benefits of experience is being able to better estimate how long things take. Although even I get it wrong still.

Merit != Ambition. A project proposal does not have merit, only the person has it

Sorry but I'm not sure I agree with that.

We've seen some Roadtests where the applicant used the device for something outside of it's normal use.

I used the Insulation Resistance tester to prove it was a very easy method of testing Line Protection devices used on Krone style Distribution Frames.

This probably was a very left field application, but for our field technicians it could be a 5 min test, which was difficult to do and certainly not in the field.

I saw another Roadtest where the writer was checking rocks ..... which probably wasn't the manufacturers first thoughts.

So I think there is merit in some ideas.

Whether the applicant can pull it off is another conversation altogether.!

Mark

As a reader of a road test, the questions I want answered are "Is this product any good?" and "Is it worth buying one if I need something similar?". The usefulness of the product can be demonstrated in a number of ways, but will really depend on what it is. Personally, I find unboxing style road tests entirely useless - just an excuse to type something and pretend you've done something useful. In my opinion, they're no benefit to the reader at all.

David Murphy wrote:

 

Personally, I find unboxing style road tests entirely useless

I agree. The unboxing reviews are only really a review of the packaging.

Some of my recent purchases have been influenced by watching certain you-tubers using test equipment and seeing the results.

Hendrik,

You make some great points.

I agree that a project is a great way to really evaluate the product. At the same time, one may not have enough time to complete it given the time we give you for the roadtest review. But that's okay. The deliverable of the roadtest is the review, not the completed project. I think you could do a good review without completing the whole project. My concern is that there have been a few people who've told me that they are taking longer to complete the project and that's what was holding up the review -- or they just gave up.

Remember, we need your help to review the product not just complete a new prototype. So, do the best given the finite period of time you're given. Explain why you think you could not complete it: too big of a concept or technical difficulties or a lack of documentation, et al.

Randall Scasny

RoadTest Program Manager

Douglas,

I surely don't want it to be a lottery. I've contact our legal department to re do the terms and conditions such that it is not defined as a contest. I don't want someone to think it is a game and the best and longest application is the winner.

The winner is someone who provides a useful test plan that would be helpful to someone who is interested in using the product. A good paragraph can do the job on the application as well as a long app.

Andy,

Great point. We don't have metrics on how one long one actually takes. I'm sure there is a lot of variability.

How long do you think it takes you to do a Roadtest. Say, when you did the Bitscope Blade?

Randall

David,

As you always do, you make good points.

Perhaps I need to give more clarity on unboxing in a RoadTest.

But first, your points are at the heart of the RoadTest: is the product any good? Would you the roadtester recommend buying it? I would like to see these questions in some form addressed in the review. I don't want to dictate the words you use. But we would like to know if you think it's worth it.

For unboxings...

Before I took over RoadTests, I went through reading a lot of roadtests. Unboxings seem to be the basic fodder at the beginning of the review. Nothin particularly wrong with it. But surely the unboxing is NOT the total review. I would not say they are worthless at all. It's a chance for everyone to see the parts (assuming visuals are posted) and how to do the basic hook up (probably for newbies). I read someone say he didn't have enough USB ports so he had to modify things -- I think he added a hub, so an unboxing / basic hookup could show that. Again, most helpful to newbies.

But a review needs to be "a review" and answer the basic questions of the product.

Thanks.

Randall

The bitscope blade was quite an involved mini project. It was written up in 6 blog posts over several weeks. The first 3 probably were only an hour each but the Nodes and Controllers were bigger, perhaps 3 hrs each and the same for the last. So total you are looking around 12hrs.

However the roadtest report was completed about half way throught that so to do the roadtest really only took 6hrs.

For the two lots of capacitive sensors I've tested those took around 8hrs each, again spread over several weeks.