Actions
Forum Thread Details
  • Replies 22 replies
  • Subscribers 2561 subscribers
  • Views 3274 views
  • Users 0 members are here
Related

When Is A RoadTest Review Too Much or Too Little?

rscasny

I have been spending more time reading (including proofreading) roadtest reviews. Some of our suppliers are reading your reviews and providing me feedback on them as well. One of the main reasons why a sponsor participates in the RoadTest program is to get feedback about the product, documentation, design tools, and so on. So, your reviews could be read by a variety of stakeholders within the sponsor's organization (engineers, business managers, marketing, etc.). Your reviews are highly visible. (So, make sure you do a spell check before you publish them, please!) While I have always said you don't be a superior writer to be a roadtester, roadtesters should try to spot basic errors and correct them. If you need help, please contact me.

One thing I have noticed recently as I read reviews is that some reviews cover a lot of things. Some have indexes to separate blogs that function as tutorials. I have also noticed that some are fairly short and don't "tell the story behind the review" visually to make a reader think "Wow" this is cool, incredible, etc."

So, when is a review too much or too little?

I do not desire to establish stringent max-word-count guidelines. I never will. After I choose you as an official roadtester, I need you to "take ownership" of the roadtest. I fully understand that people will write reviews differently. In fact, that is something I ponder when I go through the roadtest applications. I want to obtain different kinds of reviews, which in my mind represents different kinds of engineering minds and, yes, different kinds of customers.

But back to my core question: When is a review too much or too little?

There are some experienced roadtest reviews who can juggle gads and gobs of information and, like a conductor at a symphony, produce a multi-faceted, "perfect" review. These are great productions.

But there are also some reviewers who can easily perform the testing but when one reads the review it doesn't flow so smoothly and makes it a bit harder to read.

I personally think: don't forget the basics.

A review should (a) explain what you are going to do and why, (b) provide a background on the product because not all readers will be familiar with it, (c) conduct and describe the results of your tests (this can include an unboxing), (d) go through how you experienced the product and (e) draw some conclusions.

For some roadtesters, too much is when there are side information that bogs down the basics, so they never get to the basics. But for other roadtesters, they can add the side information to explain things that perhaps are not explicitly stated in the sponsor's documentation; these are skilled roadtesters. 

When is a roadtest review too little? The comments are a guide. If our members ask the roadtester a lot of questions, then maybe your review lacks something.

People learn a lot visually. Take a cartesian grid. One can learn more from a graph of a sensor temperature vs Time than just a table of temp data. When a review has no visuals that support what is being stated in the text of your narrative, it is probably too little. Finally, a just-the-right-length review is persuasive. Read the comments of a bunch of reviews. When you see comments like "nice review," you know you have hit a homerun.

Thank you for your time.

Randall Scasny

RoadTest Program Manager

  • in reply to Gough Lui

    I'd forgotten about that! : ) There was always inevitably one random reader who would feel the review was totally not for them*, and would rate it 1 star..  and then others would vote at 5 stars to try to compensate for that one anomaly vote, but it would never get back to normal..

    * You can't please everyone no matter what you do!

  • in reply to genebren

    Hi Gene,

    Thanks for the RA4M1 feedback! Incidentally that content was going to be even shorter: the content contains information on several peripherals, and I was going to publish that, but it felt incomplete without one more peripheral, namely USB. So from my perspective, it would have been a review that was too short without that peripheral.

    I figured that USB Serial was essential for usability/config/troubleshooting, and readers might feel cheated if it wasn't there, so I didn't publish it until I had tested USB and written that up too. That's why USB is the last example peripheral in that blog post, simply because it was written last. It read fine as the last peripheral example, so I didn't rearrange the text.

  • in reply to wolfgangfriedrich

    I would prefer that all communication is public, in the community.

    I try to write for users of the device, not for the sponsors. I do try to review all things e14 or sponsor suggest before the test starts, but the audience is the potential user.

  • I totally agree with you and against myself I maybe talking.  Sometimes I take the long road to explain something simple. I just want to make sure the reader understands it well, but sometimes I complicate a bit to much.

    I've seen Reviews that are extensive tutorials on the use of the hardware, and I think it makes it to extensive or even boring - to much to read. Some people might like it, others don't.

    In my opinion, a Road Test should explain a bit how it works and some steps taken (a tutorial) to reach the destination we set out to reach, so that the reader can reproduce it without any problem. Nothing too fancy.

    At least, it should lead the reader to further discovery and investigation if he or she caught an interest on the hardware being reviewed.

    That's it.

  • in reply to Jan Cumps
    Jan Cumps said:
    I would prefer that all communication is public, in the community.

    There are certainly companies that do not want that to be the case, for the sake of their own public relations and reputation. A lot of their approach is still 'old school' in this regard, rather than what I consider transparency and accountability.

    Jan Cumps said:
    I try to write for users of the device, not for the sponsors. I do try to review all things e14 or sponsor suggest before the test starts, but the audience is the potential user

    This is definitely the way to write that has the most value.Companies already have their own opinion of their hardware :D 

  • in reply to feiticeir0

    yes, and maybe help the reader to understand if it would be a fit in their design / lab. If we can show how well (or not so well) the product performs its advertised functions, it will help possible customers.

  • in reply to cstanton
    cstanton said:
    A lot of their approach is still 'old school' in this regard, rather than what I consider transparency and accountability.

    That is true. And there is a risk when sending out a product, without control on what the feedback will be.

    It takes a brave manufacturer to do that, and to embrace it.

  • in reply to Jan Cumps
    Jan Cumps said:
    It takes a brave manufacturer to do that, and to embrace it

    The phrase:

    "Well, if they don't like the review, maybe they should improve and make a better product?" has certainly passed element14 Community staff's lips before when it's been unanimously bad Smiley

  • It's always a tricky one. I try to target my road tests towards what I think a potential purchaser might want to know.

    For a microcontroller / FPGA boards that's roughly "does this do what I need and is it easy to get started coding with it". Here I'll tend to focus on the toolchain, documentation and usability as to be honest most devices are more than capable for most projects.

    For bigger stuff like a scope, I've tried to target a couple of groups - a serious puchasers with a detailed review and most E14 members who are thinking "that's cool, I wish I could afford one" with a video summary.

    The content length tends to depend on what I want to say, but obviously is a little influenced by the "size" (i.e. cost mostly). For bigger ticket items it's hard to say that a review could be too long. I've definitely seen the odd one in the past that have been a bit short.

  • in reply to Fred27

    Agree. I was thinking of writing something similar but thankfully the community got to read your good concise answer instead of my wordy waffle.

    I would add that for microcontroller dev boards a key question I try and address in the review, as it's on my mind when buying something new, is whether you can get up and running in a reasonable time period (based on experience with other boards) without getting bogged down in the learning process. Often with new boards there is very little to work with other than your own experience and the manufacturers documentation. So I try and highlight the pitfalls and the gaps you had to fill along the road test journey.

    Sometimes a short review can be seen as a good thing, especially if everything works very well.  I leave content filling to news reporters and thus I don't expect engineers to follow that wordy approach, just for the sake of it.