Actions
Forum Thread Details
  • Replies 124 replies
  • Subscribers 2570 subscribers
  • Views 10878 views
  • Users 0 members are here
Related

A Conversation about RoadTests, Missing Reviews, and the Future

rscasny

Several years ago when I interviewed with Newark element14, I visited element14 website as part of the preparation for my interview. Most recently, I had been a technical writer for an industrial connector company that sold a special kind of "switch-rated" connector used in industrial facilities, oil fields and ocean platforms, mines, generators, wastewater treatment plants, among others for high horsepower, 3-phase induction motors that could interrupt currents at full load and eliminate the risk associated with arc flash accidents. So, element14.com was quite a different kind of company for me! I told the interviewer that I was "wowed" by the depth of content, its variety, and the endless amount of participation of the element14 community members.

 

I thought back then, as I do now, that the RoadTest section of the community was remarkable. The reviews, including vivid images, scope shots, and videos were just so special. Even now, I enjoy reading through them; the authors of RoadTest reviews exhibit a passion for their subject and a depth of knowledge that is unmatched.

 

I didn't initially work in the Community when I joined the company. I was assigned to the Video team where I produced product video scripts for our transactional sites, Newark and Farnell. With recent staff changes, I ended up taking over the management of the RoadTest program. It is a lot of work, but it is satisfying when am complimented on the RoadTest area. Of course, these compliments are your compliments-- the community of RoadTesters.

 

There's much more work involved in managing the program than may appear to the casual observer. I interact with suppliers, select products to review, order & ship parts, create the RoadTest pages (with the help of my colleagues and ).Since I am the main contact for the suppliers who want to participate in the RoadTest program, I hear what's on their minds: their concerns and their problems.

 

On the topic of problems and concerns, last December a supplier contacted me regarding a RoadTest Review he wanted to read; he couldn't find it. I just thought he had problems navigating our huge website so I looked for it myself; I couldn't find it, either.  I contacted the RoadTester regarding the review I could not find several times; I received no response. The supplier contacted the RoadTester as well. (The RoadTester had logged onto the site after being contacted by us.) Ultimately, in frustration, the supplier's National Sales Manager contacted the RoadTester asking for the product to be returned. As a result of this experience, I spent a day auditing the RoadTest Reviews. I stopped when I saw 85 were missing over the last year. I realized then I had a problem that I needed to address.

 

Understanding the Problem of Unwritten Reviews Has Been a Problem for Me

 

Prior to writing this piece, I met with other members of our team to discuss the problem. Was it a problem, really?  I think that's a valid question. After all, our RoadTesters are human beings. You get sick. You have family problems. Your day jobs can overload your time. I guess if I were an official RoadTester and a family problem was preventing me from submitting a review by the deadline, I would send a courtesy email telling me (or Kelly or Diane) of the situation. I rarely receive status updates by the RoadTesters, which I found a bit odd. (When I do, it usually concerns shipping issues.) Maybe is an age thing: I am much older than many of our members. I was in the military, trained to "go the extra mile." Who knows!

 

I think why I am having a difficult time understanding this problem is when I began following up with the RoadTesters. I contacted an official RoadTester about his review, which was due 4 months prior to my message, and he messaged me back with the following: "I don't have the resources to just pick it up now. Sorry." I guess I just got blown off.

 

I contacted another official RoadTester asking for the status of his review. He responded with the following: "Well, first I would like to reaffirm that I WILL BLOG HERE and honor what was asked of me." It's been about 75 days since he received the product and I haven't received the review. I guess he is not someone who stands by his word. I messaged him today that without the review I would have to suspend his participation in the program. He shot me a message and saying that's okay by him. He still has the product and I don't have the review. And I have a supplier to answer to.

 

Another person even blamed me for asking for reviews because I was dissing all RoadTesters (presumably for even asking.) That's not worthy of a response.

 

I guess I am perplexed. After all, a RoadTester is given a free product and in return all I am asking for is a posted review. You don't have to write a book. Just provide your objective opinion.

 

So, how will this problem we solved. Well, let me start with a self-critique.

 

Solving the Problem of Missing RoadTest Reviews

 

I could make excuses and complain about my heavy workload, but that won't solve anything. For starters, I think I could do a better job at communicating with the official RoadTesters. I have communicated, but it has been later in the process; I have to do a better job at communicating earlier. I have enlisted my colleague , our Community Manager, to help me. We will try harder.  But open communication is a two-way street; I feel I should not have to always initiate communication on the status of your RoadTests.

 

I also feel I have to do a better job about ordering parts earlier in the process. I have been managing the program for a few months and have realized that many of the products being roadtested are new product introductions, which often don't have a lot of inventory; therefore, I have to order them earlier. Moving forward, I will not create any new RoadTests until I have possession of the products.

 

I think I have to do a better job at screening RoadTest applicants. I review all applications and make recommendations before sending the whole package off to the supplier who makes the final decision on official RoadTesters. I think our current RoadTest application form is too general. Therefore, I am in the process of improving the form for better screening of applicants, regarding their qualifications, knowledge and (most importantly) their participation in the element14 Community. People who participate a lot by commenting, writing blogs, and reporting on project builds deserve to RoadTest that $3,000 piece of test equipment.

 

Finally, I have noticed a lot of the applications that do not communicate clearly what they are going to do for a RoadTest. Some applications are so long that I wonder if the project can be accomplished in 60 days. I've pondered this most of all, asking myself what is a RoadTest? This reflection made me list 5 different RoadTest types:

 

  • Unboxing (14 days to complete): I think an unboxing can be a useful way to RoadTest a product despite its apparent simplicity. It surely tests if the product works out of the box and all the parts and documentation are provided. But this type of RoadTest does not require 60 days to complete. I think it can be done in a few hours. For this reason, if a RoadTester chooses to perform this type of RoadTest, he/she will need to post the official review in 14 days from delivery of the product.

 

  • Test a Product Claim (21 days to complete): I think testing a product claim, albeit more limited than an project-type, is another way to RoadTest a product. For example, some manufacturers claim you can get the product up and running in 30 minutes. Well, go ahead and test it! And report what happens. I give this 21 days to complete.

 

  • Testing to a Procedure or Standard (30 days to complete): For some products, companies will provide a detailed procedure for pre-compliance testing. (There are also testing standards by regular orgs; these may be more difficult to replicate the procedure.) In any event, testing a D.C. power Supply to an established Procedure would be an example of this type of RoadTest. Since it requires a bit of research on the RoadTester's part, I give it 30 days to complete.

 

  • Simple Project (45 days to complete): Projects always take a lot more time and creativity to produce. Plus, the review itself is usually longer. A simple project would be something like taking a Raspberry Pi, adding a sensor on the input and a motor/actuator on the output to perform some type of action. I give something like this 45 days to complete.

 

  • Complex Prototype Build (60 days to complete): Complex projects are the pinnacle of RoadTests. They take the product and use it to build a product prototype. This type of RoadTest would be similar to a Design Challenge project. As such, it would require more time and a more involved review. Hence, it merits more time. I give it 60 days to complete; however, I could be swayed to 75 days if it were really something unique.

 

The Solution: Not Only Me But the Community

 

While it is my job to manage the RoadTest program, and offer my ideas, I think the solution will be forged by the community of RoadTesters as a whole. Perhaps some ideas I have offered in this piece, resonate with you-- tell the community why. Perhaps you think I am off base or wrong; that's fine, tell the community why (but anticipate that I will engage you). Perhaps you have another idea; by all means enter it in the comments section below. I will be rolling out some improvements to the RoadTest Program, so stay tuned. Thank you for participating in our Community.

Parents

  • I see a lot of ideas thrown around here but I don't think all the details of the problem have been properly laid out. The way I see it, there are several causes for what's going on and they're all linked.

     

    First and foremost, the obvious issue of the never-planning-to-deliver roadtesters, whose existence is predicated on the fact that whatever they're doing, works, and who are responsible for driving away and/or displacing legitimate roadtesters. They have to be completely eliminated, because they won't go away until it completely stops working or until they destroy this program. If there's the slightest chance that they can get something out of it, they'll find a way to exploit it to the fullest and you'll be back at square 1 in no time.

     

    Secondly, the community is stacked with very new hobbyists. They're here to learn and for the free stuff, which is great, and they're fit for plenty of the RoadTests, including entry level scopes and other goodies that are right on their alley, but there are some that they simply can't handle, like some evaluation boards or the more specialized T&M items. Many will probably enroll in all the RoadTests, because why not? But it's not their fault when they're selected and can't complete the assignment. A better screening process will benefit everyone.

     

    Thirdly, while there are skilled individuals in the community, they're not being engaged, which is what is needed for this community to grow in the right direction (assuming I have the right idea about what E14 is trying to do with all of this). The discussions are simply not enticing enough and those that are, are far apart or so old it's pointless to join them anymore.

     

     

    I don't know how to solve the first issue (actually I have some ideas, but I don't think they'd be feasible), but for the engagement problem, it's simple: Content created specifically for that category of users. Maybe some fun design challenges with clear goals, that offer NO kits, just prizes, and require actual electronic engineering skills. Basically something along the lines of "Build something that does this. The best performing devices win." - the less "innovation" needed the better, because if you end up inventing something marketable, you don't want to give the idea away for a soldering station.

  • in reply to jadew

    I like the idea of having some (not all) design challenges with no sponsorship or kit supplied, just prizes for completion for the top x participants. It would be tricky (hard) to come up with a topic that had enough appeal to motivate wide participation, but if a suitable topic is discovered.....

    On some road tests the kit should not ship until the first satisfactory blog is published.

  • in reply to dougw

    These are both valid arguements, but the kit shipping is where delays can occur.

    Randall has decided that he'll wait until Newark have the product in hand which may fix some problems, but have a look at one challenger where through no fault of his the kit went to Germany, America , Canada and now is heading back to Turkey (hopefully).

    In his case there are significant delays.

    Getting some stuff through to me in NZ has been a battle with the NZ customs adding delays due to incorrect understanding of the USA terms v NZ terms for who pays the fees.

    That is a long wait between blog1 and being able to advance your design.

     

     

     

    If you look back at a number of the Raod Tests they have catered for the newer members.

    Your no prozes solution is worthy but if you look at the T&C of the later challenges, all IP remains with you, so regardless of a prize you don't give up the marketability.

     

    Mark

Reply
  • in reply to dougw

    These are both valid arguements, but the kit shipping is where delays can occur.

    Randall has decided that he'll wait until Newark have the product in hand which may fix some problems, but have a look at one challenger where through no fault of his the kit went to Germany, America , Canada and now is heading back to Turkey (hopefully).

    In his case there are significant delays.

    Getting some stuff through to me in NZ has been a battle with the NZ customs adding delays due to incorrect understanding of the USA terms v NZ terms for who pays the fees.

    That is a long wait between blog1 and being able to advance your design.

     

     

     

    If you look back at a number of the Raod Tests they have catered for the newer members.

    Your no prozes solution is worthy but if you look at the T&C of the later challenges, all IP remains with you, so regardless of a prize you don't give up the marketability.

     

    Mark

Children
  • in reply to mcb1

    I'll have to confess that I have had to go through a bit of a learning curve in selecting RoadTest winners. (Design Challnegers are selected by a panel of judges; for RoadTests, I do the selection recommendations). I had emphasized creativity. I guess I assumed everyone who applied was an adult and would follow up with a review accordingly. But I wasn't really aware of the extent of the problem when I assumed my role. I thought it was a very minor problem. Not until suppliers started asking for missing reviews did I conduct a detailed audit and find it was a much bigger problem.

     

    I hope I did not leave the impression I was focusing on new members only. Of course, we want new members to participate. But as the future moves on, most roadtests will be a combination of Top members, experienced roadtesters and new members. But everyone will need to demonstrate they have the time and desire to write the review. One does not need to write a manifesto of a review that includes videos, although those are great, but a solid review that is informative.

     

    Randall

  • in reply to rscasny

    It was interesting to see the 'behind the scenes' as everyone probably assummed various things, so well done for bringing it our attention.

     

    The whole subject has had discussions in the past, but the changes proposed will help ensure the future of the program, and more importantly keep the suppliers happy.

     

    John Lydgate

    “You can please some of the people all of the time, you can please all of the people some of the time, but you can’t please all of the people all of the time”.”

     

    The nature of these means you'll be lucky to please some of the people some of the time ..... image

     

     

    Mark

  • in reply to mcb1

    Yes, but you need to keep the minions happy in any event.

     

    Bored minions are very dangerous.

     

    DAB

  • in reply to rscasny

    I guess the upshot of this all is there are a number of concerned parties.

    1. The very generous sponsors who want to see a return on the investment. To them this is not a free exercise.

    2. Element14 who have a reputation to uphold as a classy act in the supply world and the ability to provide an interface to between tester and the sponsors.

    3. The committed road testers that see the opportunity to have a dabble with some interesting gadgets and tech.

     

    Without all 3 cohesively working together this program may very well fail to the detriment of all parties.

     

    Unfortunately in any program where there is the possibility of getting something for nothing there will always be someone ready to rort the system.

    All Randall can do is to try and screen these time and resource wasting testers out of the system.  for one do not envy his position as there are people joining the program with no credentials that may well make excellent road testers.

     

    I for one would be very sad to see worthy candidates miss out, or this program collapse because of some that are not so worthy.

    Tough problem.

    Sorry I cannot help in solving this one!

     

    Keep up the good work.

    Warren.