Actions
Forum Thread Details
  • Replies 22 replies
  • Subscribers 2561 subscribers
  • Views 3268 views
  • Users 0 members are here
Related

When Is A RoadTest Review Too Much or Too Little?

rscasny

I have been spending more time reading (including proofreading) roadtest reviews. Some of our suppliers are reading your reviews and providing me feedback on them as well. One of the main reasons why a sponsor participates in the RoadTest program is to get feedback about the product, documentation, design tools, and so on. So, your reviews could be read by a variety of stakeholders within the sponsor's organization (engineers, business managers, marketing, etc.). Your reviews are highly visible. (So, make sure you do a spell check before you publish them, please!) While I have always said you don't be a superior writer to be a roadtester, roadtesters should try to spot basic errors and correct them. If you need help, please contact me.

One thing I have noticed recently as I read reviews is that some reviews cover a lot of things. Some have indexes to separate blogs that function as tutorials. I have also noticed that some are fairly short and don't "tell the story behind the review" visually to make a reader think "Wow" this is cool, incredible, etc."

So, when is a review too much or too little?

I do not desire to establish stringent max-word-count guidelines. I never will. After I choose you as an official roadtester, I need you to "take ownership" of the roadtest. I fully understand that people will write reviews differently. In fact, that is something I ponder when I go through the roadtest applications. I want to obtain different kinds of reviews, which in my mind represents different kinds of engineering minds and, yes, different kinds of customers.

But back to my core question: When is a review too much or too little?

There are some experienced roadtest reviews who can juggle gads and gobs of information and, like a conductor at a symphony, produce a multi-faceted, "perfect" review. These are great productions.

But there are also some reviewers who can easily perform the testing but when one reads the review it doesn't flow so smoothly and makes it a bit harder to read.

I personally think: don't forget the basics.

A review should (a) explain what you are going to do and why, (b) provide a background on the product because not all readers will be familiar with it, (c) conduct and describe the results of your tests (this can include an unboxing), (d) go through how you experienced the product and (e) draw some conclusions.

For some roadtesters, too much is when there are side information that bogs down the basics, so they never get to the basics. But for other roadtesters, they can add the side information to explain things that perhaps are not explicitly stated in the sponsor's documentation; these are skilled roadtesters. 

When is a roadtest review too little? The comments are a guide. If our members ask the roadtester a lot of questions, then maybe your review lacks something.

People learn a lot visually. Take a cartesian grid. One can learn more from a graph of a sensor temperature vs Time than just a table of temp data. When a review has no visuals that support what is being stated in the text of your narrative, it is probably too little. Finally, a just-the-right-length review is persuasive. Read the comments of a bunch of reviews. When you see comments like "nice review," you know you have hit a homerun.

Thank you for your time.

Randall Scasny

RoadTest Program Manager

  • Great explanation, and quite true, it's always a challenge to be concise without skipping important aspects.

  • Good guideline Randall.

    When I read a review I use a similar process.

    When looking at a new technology, I really like it when the review gives the reader some background and takes the time to explain the finer details.

    When I see a review that is too high level, I try to ask pointed questions to see if I can get the reviewer to respond with more detail.

  • As always, the devil is in the details.  Not enough information - then no value added.  Too much information - it will be really difficult to absorb or process the data.

    An example of a great review (not a roadtest), was a recent review written by ,  Working with the Renesas RA4M1 Microcontroller .  This was very concise and it did a great walk through of a new processor, working up to a real practical example.  This would have been a great roadtest review, given the value of the evaluation board.  It might have been a little shy of a worthy roadtest review of a $4000 Oscilloscope.

    I think the value of the product sets the scope for the appropriate amount of content that needs to be in the review.  The more features, the more content.  That being said, I really like a problem or project specific type of a review (like shabaz's example above), as this really drives home how easy or difficult it might be to use this device.  Also, a good summary, with pros and cons is a really valuable addition to a review.

  • Maybe the reviews themselves could have a star rating to provide some feedback to the reviewer....

  • in reply to dougw

    I remember Jive had star ratings for reviews, but then people seemed to use them disparagingly and rarely, so it only took one "bad" opinion to tank the score which could be very demotivational.

    Perhaps if the stars were assigned by element14 staff as a form of feedback, that could be better, but I'm not sure if that's viable.

    - Gough

  • in reply to Gough Lui

    Maybe the reviews could be performed by a selected group of Top Members to attempt to unify the value of the ratings.

  • in reply to dougw
    dougw said:
    Maybe the reviews themselves could have a star rating to provide some feedback to the reviewer

    Hey Doug,

    If we could have that so it was only visible to the creator, then maybe there would be value to that, if we could enforce "thanks for leaving a star rating, why did you do that?" and have something easy to select from to low the barrier for that.

    Anyone doing a review is always welcome to feedback from staff on a one to one basis, and we as staff should be doing a better job with comments on reviews to communicate support of how the work is going (if we don't already). I'm sure that those who run the reviews already reach out to the person doing it and encourage them to go further when needed (or they should).

    It's a good suggestion, and it's not one that I would have back in the public eye as I feel that isn't constructive enough.

  • rscasny said:
    So, your reviews could be read by a variety of stakeholders within the sponsor's organization (engineers, business managers, marketing, etc.). Your reviews are highly visible.

    I think it would be motivating for every road-tester to have this a bi-directional process. The sponsor gets a ton of value out of the program. It does not have to be public, but feedback from the sponsor to the road-tester, would be a big motivator to improve and know better what is important next time. Yes, the tester gets to keep the device, but communication should not be a diode or black hole.

  • The fact that reviewers are able to write in their own way without a template or word count etc., is brilliant. It's good to read a review and then read the next ones and it always feels different, and I like seeing the things the individual reviewers felt were important to bring out, and how they used the product.

    The basics make a lot of sense and resonate tremendously. Especially with a new product, the reader will not know much about it, so the background info regarding what the product is and what it does is important. I often only vaguely know about a product and rely on the intro to give the background to understand the rest of the review.

    However, excessive background/intro or side stuff detracts a lot, especially if, after scrolling past it all, the review tails off without addressing the exciting things.

    Regarding if it is too short or too long, one subjective litmus test for reviewers could simply be, by the end, does the review leave readers feeling informed or not? If the answer seems to be "no" then it's possible the review is too short or inadequate in some other way.

    I also like plain language like "What is it?" and "What does it do?" etc. Maybe phrased more nicely, but kept simple to understand.