Please Read: An Important RoadTest Program Change

I only found out about the RoadTest group recently, so these old posts are all new to me!

I sometimes do contract work, and when I do I feel a large burden of responsibility to complete the job the client is paying for. A RoadTest should be no different: a professional contract "please do a review and we pay you in kind with the item being reviewed". Unfortunately, as Randall has seen there seems to be a lot of people who don't have the high standards we expect, especially when being "paid in advance" as it were.

My company also manufactures equipment that we send out for review to professionals (i.e., established press) and bloggers (of various popularity). Similarly, we don't always get reviews done in exchange for the products we send, but it's probably more than 90% success, rather than 13% you've seen). This is probably because we are very careful to choose where we sent the products and make sure we have some good communication before the process starts.

I'm wondering if a deposit scheme would filter out those that don't intend to do a review. Essentially, before the item is sent to you, a deposit (card payment) is taken from you, and this is repaid as soon as the review is posted. Maybe this is only for high-ticket items, or maybe the deposit amount is proportional to the item list price.

For a high-ticket item, I'd have no problem paying a refundable deposit to show that I was serious is fulfilling the contract to which I have agreed.

Once you've done a few RoadTests and are shown to be reliable, perhaps the deposit is no longer required.

James,

I see merit in your idea of requiring a deposit on high value items, or better yet, a deposit proportional to the item value ('cause some road test items have been very expensive).

This approach should drastically increase the fulfillment rate for reviews, but it may not impact the quality of reviews, which is another concern.

I have read many Road Test reviews structured as undisciplined streams of consciousness.  Others come across as minimalist scribbles barely meeting the definition of a review.  Then there are reviews that are beautifully structured, well written publication grade pieces with obvious inclusion of important detail and full awareness of the needs of the reader.  The very best reviews often include high quality labelled illustrations, useful external hyperlinks, and well lit, tightly edited and clearly voiced video clips.  I prefer reading reviews written by experienced professionals that can offer expert opinions, or by hobbyists or students that are excited to share their learning experience in an engaging and thoughtful way.  Not every member has the skill to craft a well written and well structured review.  The range of skill runs from barely competent to professional grade.

In short, there are ways to increase review fulfillment, but what are the ways to increase the consistency of review caliber?

Mark A

Mark,

I agree that it's very important to maintain a high standard of reviews. All I can suggest is that RoadTest applicants include links to examples of what they are capable of. This doesn't necessarily need to be other reviews, but should at least show their level of competency in photo and/or video and how well they can write and structure their thoughts.

James

In my opinion, I am not sure a deposit for all items will work, as it is has to be compared against the person's own opportunity cost for their time. In others words a $30 item is probably only equivalent, in value terms, to about 30 minutes of someones time. The rest of the time spent on reviewing an item is then essentially goodwill. This can then be countered by using some form of reputation evaluation or score. This is often how accreditation works. You then use this process to gain road test reputation points as a way to qualify yourself etc.

Then in terms of road test reports, I think it is really important to have flexibility in style and structure, as I find when I am grappling with a problem or learning something new, in particular with software development and /or MCU based development platforms, I need to read different perspectives to understand how to do it. This, in my opinion, is why I find open source works better for me compared to corporate communication / documentation as it is much richer and varied in structure and style.

In my opinion, I am not sure a deposit for all items will work, as it is has to be compared against the person's own opportunity cost for their time. In others words a $30 item is probably only equivalent, in value terms, to about 30 minutes of someones time. The rest of the time spent on reviewing an item is then essentially goodwill. This can then be countered by using some form of reputation evaluation or score. This is often how accreditation works. You then use this process to gain road test reputation points as a way to qualify yourself etc.

Then in terms of road test reports, I think it is really important to have flexibility in style and structure, as I find when I am grappling with a problem or learning something new, in particular with software development and /or MCU based development platforms, I need to read different perspectives to understand how to do it. This, in my opinion, is why I find open source works better for me compared to corporate communication / documentation as it is much richer and varied in structure and style.

I think over 90% of road test items would fall into the good will category, and to try to put a value on the time it takes to perform a review (My Kind of review) would far outweigh the value of the item, My first ever review was a 30$ battery booster pack from TI and E14. it resulted in over a weeks worth of work with research, video editing and schematic updates to get the review complete. even an item costing a few thousand $$ would be a week of effort to review correctly, now I will be honest and say that not all my reviews are so extensive, but in general they tend to be several 30 minute or longer videos, many pictures with words etc.

The deposit I think is simply to inhibit the grab and run folks who have no interest in providing a quality review or even a review at all, they just want a freebie and run, if you have to put up even a small deposit until after review is posted then I for sure would be OK with that as I know I would always get it back. Also the minute we start to put a value on the time is the moment the customs and excise folks start to get in the way, and that is a tricky think for E14 to deal with even now. Again, I dont mind the duty on a product that is useful to me but on a product were it has no use beyond the review (The integrated mains live tester review just closed for instance) then it would be hard to justify.

I like your comments. It does indeed take up a fair amount of time, but I find this is well worth the effort as it helps build up ones own knowledge base and offers an opportunity to contribute back to the community by helping others through the review.

Maybe I'm being misunderstood. I was not trying find a way to value time as too problematic (as you rightly highlighted). What I was proposing was a non-financial method, by way of visible reputation points, as a way of accrediting or qualifying testers. In my opinion, this would be a simple way of achieving the objective of improving response rates. Those who do a runner would get negative points and thus are essentially user-named and shamed in the visible online reputation list (I know I proposed this earlier - would also use physical postal address as a further check to ensure new users names are not being created etc.)

OK, that seems like a good idea but there already is a points system in place, not specifically for Road Tests but for general community participation, it has been proposed on numerous occasions to use this as a "Threshold" system to determine relative qualification for participation. Low value items would still be open to anyone of any level but perhaps need 1000 points or something, perhaps they would at least have to be a member for a certain time and earned some reputation in the community (We have had many instances were someone signs up for the purpose of a road test and are never seen again till the next road test). then more expensive items are allocated to higher level members who have proved their commitment to the community over a long period of time.

I also agree, to me, this is all about giving back to the community, sharing what I have spent most of my life learning, this is why I also run a full time YouTube channel so I can share even beyond E14.

Agreed, flexibility in style and structure is welcomed as it helps keep content fresh and interesting.

As long as the writing style and structure is high quality, that is, the style and structure keep the needs of the audience in mind, then I'm all for flexibility.