element14 Community
element14 Community
    Register Log In
  • Site
  • Search
  • Log In Register
  • Community Hub
    Community Hub
    • What's New on element14
    • Feedback and Support
    • Benefits of Membership
    • Personal Blogs
    • Members Area
    • Achievement Levels
  • Learn
    Learn
    • Ask an Expert
    • eBooks
    • element14 presents
    • Learning Center
    • Tech Spotlight
    • STEM Academy
    • Webinars, Training and Events
    • Learning Groups
  • Technologies
    Technologies
    • 3D Printing
    • FPGA
    • Industrial Automation
    • Internet of Things
    • Power & Energy
    • Sensors
    • Technology Groups
  • Challenges & Projects
    Challenges & Projects
    • Design Challenges
    • element14 presents Projects
    • Project14
    • Arduino Projects
    • Raspberry Pi Projects
    • Project Groups
  • Products
    Products
    • Arduino
    • Avnet & Tria Boards Community
    • Dev Tools
    • Manufacturers
    • Multicomp Pro
    • Product Groups
    • Raspberry Pi
    • RoadTests & Reviews
  • About Us
  • Store
    Store
    • Visit Your Store
    • Choose another store...
      • Europe
      •  Austria (German)
      •  Belgium (Dutch, French)
      •  Bulgaria (Bulgarian)
      •  Czech Republic (Czech)
      •  Denmark (Danish)
      •  Estonia (Estonian)
      •  Finland (Finnish)
      •  France (French)
      •  Germany (German)
      •  Hungary (Hungarian)
      •  Ireland
      •  Israel
      •  Italy (Italian)
      •  Latvia (Latvian)
      •  
      •  Lithuania (Lithuanian)
      •  Netherlands (Dutch)
      •  Norway (Norwegian)
      •  Poland (Polish)
      •  Portugal (Portuguese)
      •  Romania (Romanian)
      •  Russia (Russian)
      •  Slovakia (Slovak)
      •  Slovenia (Slovenian)
      •  Spain (Spanish)
      •  Sweden (Swedish)
      •  Switzerland(German, French)
      •  Turkey (Turkish)
      •  United Kingdom
      • Asia Pacific
      •  Australia
      •  China
      •  Hong Kong
      •  India
      •  Korea (Korean)
      •  Malaysia
      •  New Zealand
      •  Philippines
      •  Singapore
      •  Taiwan
      •  Thailand (Thai)
      • Americas
      •  Brazil (Portuguese)
      •  Canada
      •  Mexico (Spanish)
      •  United States
      Can't find the country/region you're looking for? Visit our export site or find a local distributor.
  • Translate
  • Profile
  • Settings
RoadTests & Reviews
  • Products
  • More
RoadTests & Reviews
RoadTest Forum A Conversation about RoadTests, Missing Reviews, and the Future
  • Blogs
  • RoadTest Forum
  • Documents
  • RoadTests
  • Reviews
  • Polls
  • Files
  • Members
  • Mentions
  • Sub-Groups
  • Tags
  • More
  • Cancel
  • New
Join RoadTests & Reviews to participate - click to join for free!
Actions
  • Share
  • More
  • Cancel
Forum Thread Details
  • Replies 124 replies
  • Subscribers 2570 subscribers
  • Views 10866 views
  • Users 0 members are here
  • RoadTest
  • scasny2
  • hardware testing
  • road test
Related

A Conversation about RoadTests, Missing Reviews, and the Future

rscasny
rscasny over 8 years ago

Several years ago when I interviewed with Newark element14, I visited element14 website as part of the preparation for my interview. Most recently, I had been a technical writer for an industrial connector company that sold a special kind of "switch-rated" connector used in industrial facilities, oil fields and ocean platforms, mines, generators, wastewater treatment plants, among others for high horsepower, 3-phase induction motors that could interrupt currents at full load and eliminate the risk associated with arc flash accidents. So, element14.com was quite a different kind of company for me! I told the interviewer that I was "wowed" by the depth of content, its variety, and the endless amount of participation of the element14 community members.

 

I thought back then, as I do now, that the RoadTest section of the community was remarkable. The reviews, including vivid images, scope shots, and videos were just so special. Even now, I enjoy reading through them; the authors of RoadTest reviews exhibit a passion for their subject and a depth of knowledge that is unmatched.

 

I didn't initially work in the Community when I joined the company. I was assigned to the Video team where I produced product video scripts for our transactional sites, Newark and Farnell. With recent staff changes, I ended up taking over the management of the RoadTest program. It is a lot of work, but it is satisfying when am complimented on the RoadTest area. Of course, these compliments are your compliments-- the community of RoadTesters.

 

There's much more work involved in managing the program than may appear to the casual observer. I interact with suppliers, select products to review, order & ship parts, create the RoadTest pages (with the help of my colleagues tariq.ahmad and pchan).Since I am the main contact for the suppliers who want to participate in the RoadTest program, I hear what's on their minds: their concerns and their problems.

 

On the topic of problems and concerns, last December a supplier contacted me regarding a RoadTest Review he wanted to read; he couldn't find it. I just thought he had problems navigating our huge website so I looked for it myself; I couldn't find it, either.  I contacted the RoadTester regarding the review I could not find several times; I received no response. The supplier contacted the RoadTester as well. (The RoadTester had logged onto the site after being contacted by us.) Ultimately, in frustration, the supplier's National Sales Manager contacted the RoadTester asking for the product to be returned. As a result of this experience, I spent a day auditing the RoadTest Reviews. I stopped when I saw 85 were missing over the last year. I realized then I had a problem that I needed to address.

 

Understanding the Problem of Unwritten Reviews Has Been a Problem for Me

 

Prior to writing this piece, I met with other members of our team to discuss the problem. Was it a problem, really?  I think that's a valid question. After all, our RoadTesters are human beings. You get sick. You have family problems. Your day jobs can overload your time. I guess if I were an official RoadTester and a family problem was preventing me from submitting a review by the deadline, I would send a courtesy email telling me (or Kelly or Diane) of the situation. I rarely receive status updates by the RoadTesters, which I found a bit odd. (When I do, it usually concerns shipping issues.) Maybe is an age thing: I am much older than many of our members. I was in the military, trained to "go the extra mile." Who knows!

 

I think why I am having a difficult time understanding this problem is when I began following up with the RoadTesters. I contacted an official RoadTester about his review, which was due 4 months prior to my message, and he messaged me back with the following: "I don't have the resources to just pick it up now. Sorry." I guess I just got blown off.

 

I contacted another official RoadTester asking for the status of his review. He responded with the following: "Well, first I would like to reaffirm that I WILL BLOG HERE and honor what was asked of me." It's been about 75 days since he received the product and I haven't received the review. I guess he is not someone who stands by his word. I messaged him today that without the review I would have to suspend his participation in the program. He shot me a message and saying that's okay by him. He still has the product and I don't have the review. And I have a supplier to answer to.

 

Another person even blamed me for asking for reviews because I was dissing all RoadTesters (presumably for even asking.) That's not worthy of a response.

 

I guess I am perplexed. After all, a RoadTester is given a free product and in return all I am asking for is a posted review. You don't have to write a book. Just provide your objective opinion.

 

So, how will this problem we solved. Well, let me start with a self-critique.

 

Solving the Problem of Missing RoadTest Reviews

 

I could make excuses and complain about my heavy workload, but that won't solve anything. For starters, I think I could do a better job at communicating with the official RoadTesters. I have communicated, but it has been later in the process; I have to do a better job at communicating earlier. I have enlisted my colleague e14megan, our Community Manager, to help me. We will try harder.  But open communication is a two-way street; I feel I should not have to always initiate communication on the status of your RoadTests.

 

I also feel I have to do a better job about ordering parts earlier in the process. I have been managing the program for a few months and have realized that many of the products being roadtested are new product introductions, which often don't have a lot of inventory; therefore, I have to order them earlier. Moving forward, I will not create any new RoadTests until I have possession of the products.

 

I think I have to do a better job at screening RoadTest applicants. I review all applications and make recommendations before sending the whole package off to the supplier who makes the final decision on official RoadTesters. I think our current RoadTest application form is too general. Therefore, I am in the process of improving the form for better screening of applicants, regarding their qualifications, knowledge and (most importantly) their participation in the element14 Community. People who participate a lot by commenting, writing blogs, and reporting on project builds deserve to RoadTest that $3,000 piece of test equipment.

 

Finally, I have noticed a lot of the applications that do not communicate clearly what they are going to do for a RoadTest. Some applications are so long that I wonder if the project can be accomplished in 60 days. I've pondered this most of all, asking myself what is a RoadTest? This reflection made me list 5 different RoadTest types:

 

  • Unboxing (14 days to complete): I think an unboxing can be a useful way to RoadTest a product despite its apparent simplicity. It surely tests if the product works out of the box and all the parts and documentation are provided. But this type of RoadTest does not require 60 days to complete. I think it can be done in a few hours. For this reason, if a RoadTester chooses to perform this type of RoadTest, he/she will need to post the official review in 14 days from delivery of the product.

 

  • Test a Product Claim (21 days to complete): I think testing a product claim, albeit more limited than an project-type, is another way to RoadTest a product. For example, some manufacturers claim you can get the product up and running in 30 minutes. Well, go ahead and test it! And report what happens. I give this 21 days to complete.

 

  • Testing to a Procedure or Standard (30 days to complete): For some products, companies will provide a detailed procedure for pre-compliance testing. (There are also testing standards by regular orgs; these may be more difficult to replicate the procedure.) In any event, testing a D.C. power Supply to an established Procedure would be an example of this type of RoadTest. Since it requires a bit of research on the RoadTester's part, I give it 30 days to complete.

 

  • Simple Project (45 days to complete): Projects always take a lot more time and creativity to produce. Plus, the review itself is usually longer. A simple project would be something like taking a Raspberry Pi, adding a sensor on the input and a motor/actuator on the output to perform some type of action. I give something like this 45 days to complete.

 

  • Complex Prototype Build (60 days to complete): Complex projects are the pinnacle of RoadTests. They take the product and use it to build a product prototype. This type of RoadTest would be similar to a Design Challenge project. As such, it would require more time and a more involved review. Hence, it merits more time. I give it 60 days to complete; however, I could be swayed to 75 days if it were really something unique.

 

The Solution: Not Only Me But the Community

 

While it is my job to manage the RoadTest program, and offer my ideas, I think the solution will be forged by the community of RoadTesters as a whole. Perhaps some ideas I have offered in this piece, resonate with you-- tell the community why. Perhaps you think I am off base or wrong; that's fine, tell the community why (but anticipate that I will engage you). Perhaps you have another idea; by all means enter it in the comments section below. I will be rolling out some improvements to the RoadTest Program, so stay tuned. Thank you for participating in our Community.

  • Sign in to reply
  • Cancel

Top Replies

  • mcb1
    mcb1 over 8 years ago +14
    rscasny I think you've identified the same issues we observed some time back. Without 'inner knowledge' it was simply an observation, but we did question the value of providing high value equipment to…
  • jc2048
    jc2048 over 8 years ago +11
    I haven't participated in any roadtests, so my comments are just those of an interested observer. Like you, I am dismayed at people treating the tests as a free giveaway. I originally thought that I couldn…
  • DAB
    DAB over 8 years ago +11
    Hi Randall, Last year I proposed a simple solution called creating trusted road testers. You could do it as an extension of your badges. Use the inexpensive gear to test new road testers. Those that comply…
Parents
  • balearicdynamics
    balearicdynamics over 8 years ago

    Just a consideration that probably is not "aligned" with the vision of the other but I think it is the worth to think about.

    Some drastic solutions suggested - seems attracting - like the restitution of the material in my opinion are not reliable, at least for two reasons:

    • Probably the process to organise the restitution of the test product (especially with the cheaper ones) is more complex and expensive than the risk to lose the test product.
    • This procedure involves a bi-lateral form of contract that implies legal problems, logistics and more.

    In my - maybe too much - optimistic view I ask: are we sure that the vanishing of some roadtester has some different reason than just the desire to take the stuff? I tend to answer yes, and the reason maybe some kind of difficult, especially in the case when it is the first time one should try this kind of "challenge" (what for many of us maybe easy for other can be extremely difficult).

    I have not checked the statistics but I am almost sure that the vanishing roadtester phenomenon hat its hight pitch with the first roadtest experience of the users. Base on this assertion ths suggestion (and here the TMs minions band maybe very helpful) is to associate a sort of tutor to every user that is selected to road test a product for the first time. The main role of this supporter should be to help the author, give suggestions, answer questions, support him and be helpful to maintain as much as possible the road test deadlines. I think that with this method we can have more happy road testers and make a good educational support to all involved.

     

    Enrico

    • Cancel
    • Vote Up +6 Vote Down
    • Sign in to reply
    • Cancel
  • shabaz
    shabaz over 8 years ago in reply to balearicdynamics

    Hi Enrico,

     

    I think you've hit the nail on the head, I think Jan also thought the same thing too in an earlier post, and so did I but wasn't sure how to resolve it so went away thinking : ) I think it is way too easy for some to be confident about something but when it tends to be new, with perhaps not a lot of documentation, and generally complex, it soon becomes too difficult to feel confident in writing a review probably. The initial intent is unlikely to be be to deliberately avoid writing the report. I always remember a quote from a (large : ) friend, who said (about anything!) to pretend it is like a massive burger, but as long as one keeps taking bites they can complete it, it can only get smaller : )

    • Cancel
    • Vote Up +4 Vote Down
    • Sign in to reply
    • Cancel
  • mcb1
    mcb1 over 8 years ago in reply to shabaz

    I guess only Randall can answer that theory.

     

    Considering that contact was made and more promises made, then nothing, I can't imagine 'stage fright' is the reason.

     

    Randall has explained that he is making changes to the application process.

    With what has been suggested, and some vigilance at the other end, hopefully these problems will go away.

     

    Mark

    • Cancel
    • Vote Up +3 Vote Down
    • Sign in to reply
    • Cancel
  • rscasny
    rscasny over 8 years ago in reply to shabaz

    Shabaz,

     

    I didn't mention it in my original post, but I noticed a couple of reviews where I felt the RoadTester produced a review that wasn't ready for prime time. I have pondered that some of them thought they were ready to do a roadtest but found it more difficult than they had thought it would be and gave up. I was thinking about creating something on the page, maybe a new tab that said "tech support." But all they need to do is post a comment and most people, like yourself, respond.

    • Cancel
    • Vote Up +3 Vote Down
    • Sign in to reply
    • Cancel
  • shabaz
    shabaz over 8 years ago in reply to rscasny

    Hi Randall,

    I think it really could be that for some, i.e. it gets too overwhelming. It would be so awesome if the reviewers mentioned that in the comments that they want some pointers, because the readers would have ideas.

     

    Maybe, when it becomes apparent that a review is really brief (there have been some that are just a few sentences) readers like us should suggest in the comments some simpler things to see, to extract value. That might persuade them to do the things that are not overwhelming to continue with the review.

     

    As an example of things that could be simpler:

    * Unboxings  (I rarely do unboxings because I know others will do that, and to be honest I get more joy from more technical matters. But the unboxings could be useful to some, since some people want to see everything)

    * URLs to documentation (It really is useful to have URLs to all the documentation, but I don't always remember to have those since I figure people can locate that stuff with not too much difficulty, and URLs can change. But ease of documentation accessibility for the reader is important.

    * Teardowns (i.e. disassembly photos).

    * Detailed screenshots of connectors/the underside etc : ) Or a video showing it powering up (that could persuade someone to begin creating videos)

     

    I'm not suggesting that reviewers ever get selected to actually do such topics, these they are really not ideal things for a full review, but they could be last-resort suggestions if a user wants to contribute but is having trouble with more complex topics.

    (Or maybe it is a bad idea. I'm just brainstorming but not very well, I had a long start to the week already ; )

     

    But anyway, I think these 'easier' sections may be less immediately impactful, but occasionally could be helpful.

    For example I recently did a personal teardown of an Amazon Echo Dot (I did it just for personal curiosity but I will upload the photos in a blog post sometime), just to see how Amazon manage the clever voice positioning detection to better detect where the user is located. I figured it was well worth the £50 to see how it is put together. I was surprised, there are about 8 layers of "stuff" packed inside the little puck : ) Very difficult to put it back together in functioning state, I failed, I broke a flat flex connector : )

    • Cancel
    • Vote Up +6 Vote Down
    • Sign in to reply
    • Cancel
  • balearicdynamics
    balearicdynamics over 8 years ago in reply to shabaz

    Hi Shabaz,

     

    IMHO everyone of the suggested ways maybe be criticised as none of these we are sure it is 100% perfect; but in a democratic vision and giving at least the "reasonable doubt" option maybe this is the worth. Not last but if we associate - if they agree etc. etc. - a TM to the new reviewers this can be also useful for other reasons that you can imagine, in-line with the philosophy of this community. New reviewers maybe good new contributors, but sometimes needs some more support to be conscious of their potential. If you remember just a case that I followed and saw great results was with Dale and his daughter during 2015 challenge.

    Maybe the worth to test a period then see what happens...

     

    Enrico

    • Cancel
    • Vote Up +4 Vote Down
    • Sign in to reply
    • Cancel
  • DAB
    DAB over 8 years ago in reply to shabaz

    Good point Shabaz,

     

    Road Test is all about the experience and the product.

    If a reviewer encounters issues of any type, then both the community and the manufacturer need to know what they are.

     

    Before applying for a road test, you need to assess your time, your own testing capability and equipment and make a commitment to assess the product.

     

    The only way the process works is if everyone is honest and open.

    If you cannot test the product because of a lack of information or ability to understand the product, those issues need to be posted.

     

    If you encounter a personal issue, and I have in the past, let the community know that you will be delayed and why.  It is also helpful if you can outline a recovery plan.

     

    Given the past performance, Element 14 needs to TRUST, but VERIFY road testers.

     

    DAB

    • Cancel
    • Vote Up +3 Vote Down
    • Sign in to reply
    • Cancel
Reply
  • DAB
    DAB over 8 years ago in reply to shabaz

    Good point Shabaz,

     

    Road Test is all about the experience and the product.

    If a reviewer encounters issues of any type, then both the community and the manufacturer need to know what they are.

     

    Before applying for a road test, you need to assess your time, your own testing capability and equipment and make a commitment to assess the product.

     

    The only way the process works is if everyone is honest and open.

    If you cannot test the product because of a lack of information or ability to understand the product, those issues need to be posted.

     

    If you encounter a personal issue, and I have in the past, let the community know that you will be delayed and why.  It is also helpful if you can outline a recovery plan.

     

    Given the past performance, Element 14 needs to TRUST, but VERIFY road testers.

     

    DAB

    • Cancel
    • Vote Up +3 Vote Down
    • Sign in to reply
    • Cancel
Children
  • Jan Cumps
    Jan Cumps over 8 years ago in reply to DAB

    DAB wrote:

     

    Good point Shabaz,

    ...

     

    Before applying for a road test, you need to assess your time, your own testing capability and equipment and make a commitment to assess the product.

     

    ...

     

    DAB

    All valid points, Don. And communicating if anything happens is a basic matter of being polite and considerate.

    On the time remark above, many people that have a profession can't predict/assess what the future holds.

    Flexibility on the job has priority. My company asked me to travel abroad 4 days per week for the next 2 years just today - starting in two weeks. It's a reality that we have to deal with.

    Lucky for me it's to Barcelona. Could be worse image

    • Cancel
    • Vote Up +2 Vote Down
    • Sign in to reply
    • Cancel
  • DAB
    DAB over 8 years ago in reply to Jan Cumps

    I hear the Tapa's there are very good.

     

    DAB

    • Cancel
    • Vote Up +2 Vote Down
    • Sign in to reply
    • Cancel
element14 Community

element14 is the first online community specifically for engineers. Connect with your peers and get expert answers to your questions.

  • Members
  • Learn
  • Technologies
  • Challenges & Projects
  • Products
  • Store
  • About Us
  • Feedback & Support
  • FAQs
  • Terms of Use
  • Privacy Policy
  • Legal and Copyright Notices
  • Sitemap
  • Cookies

An Avnet Company © 2025 Premier Farnell Limited. All Rights Reserved.

Premier Farnell Ltd, registered in England and Wales (no 00876412), registered office: Farnell House, Forge Lane, Leeds LS12 2NE.

ICP 备案号 10220084.

Follow element14

  • X
  • Facebook
  • linkedin
  • YouTube