element14 Community
element14 Community
    Register Log In
  • Site
  • Search
  • Log In Register
  • Community Hub
    Community Hub
    • What's New on element14
    • Feedback and Support
    • Benefits of Membership
    • Personal Blogs
    • Members Area
    • Achievement Levels
  • Learn
    Learn
    • Ask an Expert
    • eBooks
    • element14 presents
    • Learning Center
    • Tech Spotlight
    • STEM Academy
    • Webinars, Training and Events
    • Learning Groups
  • Technologies
    Technologies
    • 3D Printing
    • FPGA
    • Industrial Automation
    • Internet of Things
    • Power & Energy
    • Sensors
    • Technology Groups
  • Challenges & Projects
    Challenges & Projects
    • Design Challenges
    • element14 presents Projects
    • Project14
    • Arduino Projects
    • Raspberry Pi Projects
    • Project Groups
  • Products
    Products
    • Arduino
    • Avnet & Tria Boards Community
    • Dev Tools
    • Manufacturers
    • Multicomp Pro
    • Product Groups
    • Raspberry Pi
    • RoadTests & Reviews
  • About Us
  • Store
    Store
    • Visit Your Store
    • Choose another store...
      • Europe
      •  Austria (German)
      •  Belgium (Dutch, French)
      •  Bulgaria (Bulgarian)
      •  Czech Republic (Czech)
      •  Denmark (Danish)
      •  Estonia (Estonian)
      •  Finland (Finnish)
      •  France (French)
      •  Germany (German)
      •  Hungary (Hungarian)
      •  Ireland
      •  Israel
      •  Italy (Italian)
      •  Latvia (Latvian)
      •  
      •  Lithuania (Lithuanian)
      •  Netherlands (Dutch)
      •  Norway (Norwegian)
      •  Poland (Polish)
      •  Portugal (Portuguese)
      •  Romania (Romanian)
      •  Russia (Russian)
      •  Slovakia (Slovak)
      •  Slovenia (Slovenian)
      •  Spain (Spanish)
      •  Sweden (Swedish)
      •  Switzerland(German, French)
      •  Turkey (Turkish)
      •  United Kingdom
      • Asia Pacific
      •  Australia
      •  China
      •  Hong Kong
      •  India
      •  Korea (Korean)
      •  Malaysia
      •  New Zealand
      •  Philippines
      •  Singapore
      •  Taiwan
      •  Thailand (Thai)
      • Americas
      •  Brazil (Portuguese)
      •  Canada
      •  Mexico (Spanish)
      •  United States
      Can't find the country/region you're looking for? Visit our export site or find a local distributor.
  • Translate
  • Profile
  • Settings
RoadTests & Reviews
  • Products
  • More
RoadTests & Reviews
RoadTest Forum A Conversation about RoadTests, Missing Reviews, and the Future
  • Blogs
  • RoadTest Forum
  • Documents
  • RoadTests
  • Reviews
  • Polls
  • Files
  • Members
  • Mentions
  • Sub-Groups
  • Tags
  • More
  • Cancel
  • New
Join RoadTests & Reviews to participate - click to join for free!
Actions
  • Share
  • More
  • Cancel
Forum Thread Details
  • Replies 124 replies
  • Subscribers 2571 subscribers
  • Views 10890 views
  • Users 0 members are here
  • RoadTest
  • scasny2
  • hardware testing
  • road test
Related

A Conversation about RoadTests, Missing Reviews, and the Future

rscasny
rscasny over 8 years ago

Several years ago when I interviewed with Newark element14, I visited element14 website as part of the preparation for my interview. Most recently, I had been a technical writer for an industrial connector company that sold a special kind of "switch-rated" connector used in industrial facilities, oil fields and ocean platforms, mines, generators, wastewater treatment plants, among others for high horsepower, 3-phase induction motors that could interrupt currents at full load and eliminate the risk associated with arc flash accidents. So, element14.com was quite a different kind of company for me! I told the interviewer that I was "wowed" by the depth of content, its variety, and the endless amount of participation of the element14 community members.

 

I thought back then, as I do now, that the RoadTest section of the community was remarkable. The reviews, including vivid images, scope shots, and videos were just so special. Even now, I enjoy reading through them; the authors of RoadTest reviews exhibit a passion for their subject and a depth of knowledge that is unmatched.

 

I didn't initially work in the Community when I joined the company. I was assigned to the Video team where I produced product video scripts for our transactional sites, Newark and Farnell. With recent staff changes, I ended up taking over the management of the RoadTest program. It is a lot of work, but it is satisfying when am complimented on the RoadTest area. Of course, these compliments are your compliments-- the community of RoadTesters.

 

There's much more work involved in managing the program than may appear to the casual observer. I interact with suppliers, select products to review, order & ship parts, create the RoadTest pages (with the help of my colleagues tariq.ahmad and pchan).Since I am the main contact for the suppliers who want to participate in the RoadTest program, I hear what's on their minds: their concerns and their problems.

 

On the topic of problems and concerns, last December a supplier contacted me regarding a RoadTest Review he wanted to read; he couldn't find it. I just thought he had problems navigating our huge website so I looked for it myself; I couldn't find it, either.  I contacted the RoadTester regarding the review I could not find several times; I received no response. The supplier contacted the RoadTester as well. (The RoadTester had logged onto the site after being contacted by us.) Ultimately, in frustration, the supplier's National Sales Manager contacted the RoadTester asking for the product to be returned. As a result of this experience, I spent a day auditing the RoadTest Reviews. I stopped when I saw 85 were missing over the last year. I realized then I had a problem that I needed to address.

 

Understanding the Problem of Unwritten Reviews Has Been a Problem for Me

 

Prior to writing this piece, I met with other members of our team to discuss the problem. Was it a problem, really?  I think that's a valid question. After all, our RoadTesters are human beings. You get sick. You have family problems. Your day jobs can overload your time. I guess if I were an official RoadTester and a family problem was preventing me from submitting a review by the deadline, I would send a courtesy email telling me (or Kelly or Diane) of the situation. I rarely receive status updates by the RoadTesters, which I found a bit odd. (When I do, it usually concerns shipping issues.) Maybe is an age thing: I am much older than many of our members. I was in the military, trained to "go the extra mile." Who knows!

 

I think why I am having a difficult time understanding this problem is when I began following up with the RoadTesters. I contacted an official RoadTester about his review, which was due 4 months prior to my message, and he messaged me back with the following: "I don't have the resources to just pick it up now. Sorry." I guess I just got blown off.

 

I contacted another official RoadTester asking for the status of his review. He responded with the following: "Well, first I would like to reaffirm that I WILL BLOG HERE and honor what was asked of me." It's been about 75 days since he received the product and I haven't received the review. I guess he is not someone who stands by his word. I messaged him today that without the review I would have to suspend his participation in the program. He shot me a message and saying that's okay by him. He still has the product and I don't have the review. And I have a supplier to answer to.

 

Another person even blamed me for asking for reviews because I was dissing all RoadTesters (presumably for even asking.) That's not worthy of a response.

 

I guess I am perplexed. After all, a RoadTester is given a free product and in return all I am asking for is a posted review. You don't have to write a book. Just provide your objective opinion.

 

So, how will this problem we solved. Well, let me start with a self-critique.

 

Solving the Problem of Missing RoadTest Reviews

 

I could make excuses and complain about my heavy workload, but that won't solve anything. For starters, I think I could do a better job at communicating with the official RoadTesters. I have communicated, but it has been later in the process; I have to do a better job at communicating earlier. I have enlisted my colleague e14megan, our Community Manager, to help me. We will try harder.  But open communication is a two-way street; I feel I should not have to always initiate communication on the status of your RoadTests.

 

I also feel I have to do a better job about ordering parts earlier in the process. I have been managing the program for a few months and have realized that many of the products being roadtested are new product introductions, which often don't have a lot of inventory; therefore, I have to order them earlier. Moving forward, I will not create any new RoadTests until I have possession of the products.

 

I think I have to do a better job at screening RoadTest applicants. I review all applications and make recommendations before sending the whole package off to the supplier who makes the final decision on official RoadTesters. I think our current RoadTest application form is too general. Therefore, I am in the process of improving the form for better screening of applicants, regarding their qualifications, knowledge and (most importantly) their participation in the element14 Community. People who participate a lot by commenting, writing blogs, and reporting on project builds deserve to RoadTest that $3,000 piece of test equipment.

 

Finally, I have noticed a lot of the applications that do not communicate clearly what they are going to do for a RoadTest. Some applications are so long that I wonder if the project can be accomplished in 60 days. I've pondered this most of all, asking myself what is a RoadTest? This reflection made me list 5 different RoadTest types:

 

  • Unboxing (14 days to complete): I think an unboxing can be a useful way to RoadTest a product despite its apparent simplicity. It surely tests if the product works out of the box and all the parts and documentation are provided. But this type of RoadTest does not require 60 days to complete. I think it can be done in a few hours. For this reason, if a RoadTester chooses to perform this type of RoadTest, he/she will need to post the official review in 14 days from delivery of the product.

 

  • Test a Product Claim (21 days to complete): I think testing a product claim, albeit more limited than an project-type, is another way to RoadTest a product. For example, some manufacturers claim you can get the product up and running in 30 minutes. Well, go ahead and test it! And report what happens. I give this 21 days to complete.

 

  • Testing to a Procedure or Standard (30 days to complete): For some products, companies will provide a detailed procedure for pre-compliance testing. (There are also testing standards by regular orgs; these may be more difficult to replicate the procedure.) In any event, testing a D.C. power Supply to an established Procedure would be an example of this type of RoadTest. Since it requires a bit of research on the RoadTester's part, I give it 30 days to complete.

 

  • Simple Project (45 days to complete): Projects always take a lot more time and creativity to produce. Plus, the review itself is usually longer. A simple project would be something like taking a Raspberry Pi, adding a sensor on the input and a motor/actuator on the output to perform some type of action. I give something like this 45 days to complete.

 

  • Complex Prototype Build (60 days to complete): Complex projects are the pinnacle of RoadTests. They take the product and use it to build a product prototype. This type of RoadTest would be similar to a Design Challenge project. As such, it would require more time and a more involved review. Hence, it merits more time. I give it 60 days to complete; however, I could be swayed to 75 days if it were really something unique.

 

The Solution: Not Only Me But the Community

 

While it is my job to manage the RoadTest program, and offer my ideas, I think the solution will be forged by the community of RoadTesters as a whole. Perhaps some ideas I have offered in this piece, resonate with you-- tell the community why. Perhaps you think I am off base or wrong; that's fine, tell the community why (but anticipate that I will engage you). Perhaps you have another idea; by all means enter it in the comments section below. I will be rolling out some improvements to the RoadTest Program, so stay tuned. Thank you for participating in our Community.

  • Sign in to reply
  • Cancel

Top Replies

  • mcb1
    mcb1 over 8 years ago +14
    rscasny I think you've identified the same issues we observed some time back. Without 'inner knowledge' it was simply an observation, but we did question the value of providing high value equipment to…
  • jc2048
    jc2048 over 8 years ago +11
    I haven't participated in any roadtests, so my comments are just those of an interested observer. Like you, I am dismayed at people treating the tests as a free giveaway. I originally thought that I couldn…
  • DAB
    DAB over 8 years ago +11
    Hi Randall, Last year I proposed a simple solution called creating trusted road testers. You could do it as an extension of your badges. Use the inexpensive gear to test new road testers. Those that comply…
Parents
  • ciorga
    ciorga over 7 years ago

    I think things have changed to worse on E14 roadtest system recently and I am getting closer and closer to giving up and quitting.  It was a time when I put a lot of passion and I contributed with many blog posts on interesting projects that I was passionate about like a few examples here: "brainwaves appliance controller", "musical EKG", "fundamental concepts tutorial series", and winning probably the highest price value design contest so far "sudden impact health monitor".

     

    I put a lot of effort in my projects and in writing applications for road tests and design contests.  But what can I do when after putting so much effort I see that the person selected is far behind with the quality of activities for example for the FPC 1500 Spectrum analyzer I see that one of the selected persons had only one roadtest review done over a 6 year period and that one was the lowest quality review that I have ever seen on Element 14 "click here to see it yourself".  And that person gets selected for the FPC1500 and he is not even able to post a review at all within the required time. As today he has not posted a review and we are way past the due date.  Well, I am not surprised based on the past history of this member but I am surprised of the E14 road test reviewers selection system.  What was so attractive to pick this applicant instead of other applicants with proven history of high quality posts and reviews?

     

    Where is the quality and standards of Element 14 that we all believed in for so many years?

     

    What is going on in this community?

     

    I decided to write these thoughts with the risk of never been selected for any road test but with the hope that my message will help the E14 community bring back the standards to what we have been used to.

     

    Best Wishes to all of you,

    Cosmin

    • Cancel
    • Vote Up +6 Vote Down
    • Sign in to reply
    • Cancel
Reply
  • ciorga
    ciorga over 7 years ago

    I think things have changed to worse on E14 roadtest system recently and I am getting closer and closer to giving up and quitting.  It was a time when I put a lot of passion and I contributed with many blog posts on interesting projects that I was passionate about like a few examples here: "brainwaves appliance controller", "musical EKG", "fundamental concepts tutorial series", and winning probably the highest price value design contest so far "sudden impact health monitor".

     

    I put a lot of effort in my projects and in writing applications for road tests and design contests.  But what can I do when after putting so much effort I see that the person selected is far behind with the quality of activities for example for the FPC 1500 Spectrum analyzer I see that one of the selected persons had only one roadtest review done over a 6 year period and that one was the lowest quality review that I have ever seen on Element 14 "click here to see it yourself".  And that person gets selected for the FPC1500 and he is not even able to post a review at all within the required time. As today he has not posted a review and we are way past the due date.  Well, I am not surprised based on the past history of this member but I am surprised of the E14 road test reviewers selection system.  What was so attractive to pick this applicant instead of other applicants with proven history of high quality posts and reviews?

     

    Where is the quality and standards of Element 14 that we all believed in for so many years?

     

    What is going on in this community?

     

    I decided to write these thoughts with the risk of never been selected for any road test but with the hope that my message will help the E14 community bring back the standards to what we have been used to.

     

    Best Wishes to all of you,

    Cosmin

    • Cancel
    • Vote Up +6 Vote Down
    • Sign in to reply
    • Cancel
Children
  • thamizharasan
    thamizharasan over 7 years ago in reply to ciorga

    Cosmin, people like yourself should continue contributing, motivating & inspiring others.

    I've read many of your Tutorials & Road-Test Reviews & you are a true professional.


    I'm herewith 20+ years of hands-on experience with Electronics & IT background & passionately into IoT R&D, but recently came to know about this community & joined recently to contribute by sharing knowledge.

     

    Lets be part of this like-minded people's community & there could be some people who do not understand / support the purpose of this community & we can ignore them & they themselves would face the consequences if they don't contribute for over a period of time (till then we believe they are into some bad situation & still need time to showcase their skills if they have one).

     

    Happy Knowledge Sharing !

    • Cancel
    • Vote Up +4 Vote Down
    • Sign in to reply
    • Cancel
  • kk99
    kk99 over 7 years ago in reply to ciorga

    Hi Cosmin,
    I agree with you that quality of projects, blogs and knowledge sharing should be one of important factors during selecting reviewers. Additionally it should be checked if given person is able to test in 100 % given equipment, because it sometimes requires additional devices e.g. signal generator etc.

    • Cancel
    • Vote Up +2 Vote Down
    • Sign in to reply
    • Cancel
  • sally314
    sally314 over 7 years ago in reply to kk99

    It is somewhat disconcerting that there seems to be a lack of technical knowledge among some of the

    key staff here.  It's must be difficult for non-technical people to manage the production of high quality

    technical content and reviews.  For example, https://www.element14.com/community/message/228663/l/re-many-thanks#228663

    says:

    Mindi and I are so sorry about the mistake with the voltages. She and I are both not technical people, and we just didn't think about it!

    There is also no shortage of occurrences of staff making promises that they don't follow through on.

    For example, Help Us Improve the Community!  ends with:

     

    I will return next week with an exciting intro post.

    and that was 6 months ago.  Did I miss this exciting intro post?

     

    Certainly there are numerous bug reports with a response such as:
    https://www.element14.com/community/message/210784/l/re-an-unexpected-error-has-occurred#210784

    Yes, that's not a great error... I'll make a note of it!

    where either no note was actually made, or subsequently got lost, or in any case

    was never addressed.

    • Cancel
    • Vote Up 0 Vote Down
    • Sign in to reply
    • Cancel
  • rscasny
    rscasny over 7 years ago in reply to sally314

    Interesting comment, not sure I agree. People make mistakes. Even technical people make mistakes.

     

    But we make an honest effort to correct them when they do occur.

     

    Randall Scasny

    RoadTest Program Manager

    • Cancel
    • Vote Up +2 Vote Down
    • Sign in to reply
    • Cancel
  • sally314
    sally314 over 7 years ago in reply to rscasny

    I'm not sure what part you disagree with.  Maybe you can clarify.

    Non-technical people can evaluate content by counting click-thru's

    to the transactional websites, but would have no clue what ciorga

    is referring to regarding the low quality review he cites.

    I don't think the admitted lack of technical competency is just

    some sort of non-serious banter.

    • Cancel
    • Vote Up 0 Vote Down
    • Sign in to reply
    • Cancel
  • rscasny
    rscasny over 7 years ago in reply to sally314

    I'm referring to your reference to one of our assistants in our office.

     

    As far as roadtest reviews go, some reviews are fantastic, some are interesting, some are sub-standard and a couple over the last two years were lemons. Just like anything, it's a mixed bag. One would like all reviews to be fantastic, but that's probably expecting too much.

     

    I would urge you to read through the reviews -- lots of them. Most are very good. They are interesting reading. I haven't seen a drop off in quality overall.

     

    If anything, I would love the applications to be roadtest reviewers to improve. And they have to a large degree.

     

    Randall Scasny

    RoadTest Program Manager

    • Cancel
    • Vote Up +8 Vote Down
    • Sign in to reply
    • Cancel
  • neuromodulator
    neuromodulator over 7 years ago in reply to ciorga

    I completely agree with that Cosmin.

     

    I've seen a similar pattern in other users, they fail to deliver the roadtest (or leave it incomplete) and keep enrolling and winning (sometimes very expensive equipment). Evaluating previous roadtests should be enough to quickly spotting bad roadtesters and avoiding this issue. Moreover, to avoid huge losses, and refrain the greedy from applying to expensive stuff, maybe it should be required that the users enrolling for expensive equipment have previously posted a high quality roadtests on more inexpensive equipment. Imho a high quality previous roadtest is a better indicator of how serious a roadtester is, than an impressive application (that could be filled with lies). Those that have not delivered or delivered poor quality roadtests should be last in the priority list when it comes to selecting new roadtesters...

    • Cancel
    • Vote Up 0 Vote Down
    • Sign in to reply
    • Cancel
  • sally314
    sally314 over 7 years ago in reply to rscasny
    Interesting comment, not sure I agree.

    I'm referring to your reference to one of our assistants in our office.

    You're being really vague.  If one of the assistants in your office has

    undisclosed technical qualifications, please enlighten us.  I don't want

    to be seen as disparaging someone by quoting them if it turns out they

    weren't being serious in lumping themselves in the same non-technical

    category as their boss.

     

    In the past, element14 has employed staff such as Drew Fustini and

    Spanner Spencer with no question as to their technical competence.

    • Cancel
    • Vote Up 0 Vote Down
    • Sign in to reply
    • Cancel
  • rscasny
    rscasny over 7 years ago in reply to sally314

    I'm referring to the conversation you linked to in a previous comment:

    https://www.element14.com/community/message/228663/l/re-many-thanks#228663

     

    Apparently, a roadtester was shipped a soldering iron with a 120v plug to the UK which uses 220V. The roadtester notified Dan, our shipping clerk, of the error. Dan, the consummate professional that he is, apologized for the inconvenience. He's a shipping clerk and is not expected to know about different power supply standards in different parts of the world. I tell him what to ship and he submits the order to the shipper. I'm sure he was not aware of what come in the box anyway. And he's not required to.

     

    As far as your comments and Cosmin's comments regarding the quality of roadtests, well, I'm glad you are participating in the conversation.  I appreciate your comments and I think the discussion can only improve the quality of the roadtest reviews. This discussion has increased and changed our screening of roadtest applicants. But, your conversation tells me we need to continue improving the screening and selection system. But some of it cannot be helped. The sponsor of roadtests make the final decision. Not element14.

     

    Finally, Spanner Spencer was not been with the company for over a year.

     

    Randall Scasny

    RoadTest Program Manager

    • Cancel
    • Vote Up +5 Vote Down
    • Sign in to reply
    • Cancel
  • Fred27
    Fred27 over 7 years ago in reply to ciorga

    ciorga  wrote:

    I think things have changed to worse on E14 roadtest system recently and I am getting closer and closer to giving up and quitting.

    I'm not sure I agree with you that things here on E14 are getting worse. I'd say that over the last year or so things have generally improved. The number of "missing" or "token gesture" road tests seems to have decreased.

    ciorga  wrote:

    I put a lot of effort in my projects and in writing applications for road tests and design contests.  But what can I do when after putting so much effort I see that the person selected is far behind with the quality of activities for example for the FPC 1500 Spectrum analyzer I see that one of the selected persons had only one roadtest review done over a 6 year period and that one was the lowest quality review that I have ever seen on Element 14 Keysight 34470A and Texas Instruments DAC8734EVM - Review .  And that person gets selected for the FPC1500 and he is not even able to post a review at all within the required time. As today he has not posted a review and we are way past the due date.  Well, I am not surprised based on the past history of this member but I am surprised of the E14 road test reviewers selection system.  What was so attractive to pick this applicant instead of other applicants with proven history of high quality posts and reviews?

    However, I do agree with you on this instance. You may have noticed I gave him some gentle prodding myself! I completely agree with Randall's rule that if you don't complete a previous road test then you don't get to do any more but it does seem to be difficult to police. His earlier road test definitely shouldn't have counted as "completed" in my opinion. I can imagine it is impractical for someone to validate previous road tests for all applicants. Would it be possible for a company with 3 items to offer to pick say 5 testers and for these to then be manually vetted?

     

    It would be good to see someone who doesn't complete a review to a minimum standard actually be billed for or return a product, but I can imagine that's easier said than done.

     

    Most importantly, is there anything that we as a community can do to help with this? Maybe some senior members with the ability to flag a road test as completed to a minimum standard or not? Any other way we could make things work better without too much load falling on Randall and his team?

     

    It must be very difficult to manage this sort of community and balance the requirements of E14, manufacturers and community members - and particularly balancing the encouragement of new members with the trust of more well known and trusted ones. My gut feel would be to have potential road testers prove themselves on a more mundane road test before getting the chance to do something bigger. Could we have different road test levels? Would the current  reputation levels (Coulomb, Volta, Ampere, Ohm, etc.) do?

     

    On the downside, this may risk putting off new road testers but should at least give some guarantee of quality. And sometimes it can seem that the same reviewers do all the cool stuff. Despite the high quality of your own road tests Cosmin, I remember when you were selected for the RTB2004 thinking "Damn! This guy just road tested a 'scope. Give someone else a chance." Of course, now I've been given the opportunity to test the RTM3004 I see things differently! You can't please everyone but I suppose the people you really need to please are the manufacturers who are supplying the products to be tested.

    • Cancel
    • Vote Up +5 Vote Down
    • Sign in to reply
    • Cancel
element14 Community

element14 is the first online community specifically for engineers. Connect with your peers and get expert answers to your questions.

  • Members
  • Learn
  • Technologies
  • Challenges & Projects
  • Products
  • Store
  • About Us
  • Feedback & Support
  • FAQs
  • Terms of Use
  • Privacy Policy
  • Legal and Copyright Notices
  • Sitemap
  • Cookies

An Avnet Company © 2025 Premier Farnell Limited. All Rights Reserved.

Premier Farnell Ltd, registered in England and Wales (no 00876412), registered office: Farnell House, Forge Lane, Leeds LS12 2NE.

ICP 备案号 10220084.

Follow element14

  • X
  • Facebook
  • linkedin
  • YouTube