Actions
Forum Thread Details
  • Replies 91 replies
  • Subscribers 2566 subscribers
  • Views 6777 views
  • Users 0 members are here
Related

Please Read: An Important RoadTest Program Change

rscasny

This is the third virtual "fireside chat" that I'm writing regarding the RoadTest Program. For this one, I'd like to roll out a RoadTest Program change and explain why the change is occurring.

 

To start, I'd like to share a snippet of an email I received yesterday from a vendor who participates in the RoadTest Program:

 

"We received only 13% reviews back from the RoadTests we sponsored in 2016. I would like to know the status of each of these roadtests. Are the missing reviews still being pursued? I would also like to know if the members that failed to post a review are now being flagged so we do not chose them again."

 

It was not pleasant to respond to. I had to eat "humble pie," as we say in the US, when I responded to this vendor.

 

To solve the problem, I decided I don't want to go down the road of suspending members from the RoadTest program. Suspensions sound too much like the hard rules of the military. I was in the Navy for 10 years and I didn't like tough rules back then, but I knew they were necessary given the reality of uncomfortable shipboard conditions and potentially dangerous operations we were participating in.

 

But this is a community. It should be a place to have some fun, meet new people, network professionally, be a mentor, or get mentored. Suspending people just is the wrong road to follow.

 

The way I've chosen to resolve the problem is the program change.

 

RoadTest application reports will soon include whether a specific applicant has completed all of his/her previous RoadTest reviews. I'm told by our developers this is very easy to do. We will provide this information to the vendor with a RoadTester's application. The supplier will make the decision where to select the RoadTester or not. Given the above comment by the supplier, I think the answer is self-evident.

 

Now, this change will probably cause me more work. If someone were flagged, I will have to check to see if that person is on a current RoadTest and it's before the due date of the official review. I surely don't want to penalize a conscientious RoadTester. I will also have to ensure there are not any extenuating circumstances that have caused the person not to write the review, i.e., sickness, family issues.

 

I am ambivalent about this whole thing. There are many RoadTesters who are doing a fantastic job. But a 13% success rate is not successful. I really have no choice but to implement this change.

 

Sincerely,

 

Randall Scasny

RoadTest Program Manager

  • in reply to jkutzsch

    the ones who truly want to participate

    That is the best comment I've seen.

     

    Writers block, too busy, are all just excuses.

    More worryingly is that if that is the ethic they have (can't be bothered complying or communicating), then I pity the Engineering that they produce.

     

    Mark

  • in reply to jkutzsch

    I also like how gave example timelines in a previous post he put out.  https://www.element14.com/community/message/217461/l/a-conversation-about-roadtests-missing-reviews-and-the-future#21746…   Giving a good solid framework with timeline is very helpful.  I suggest adding in there something along the line of at the 1/2 or 3/4 point if delays have occurred please contact xxxxxx with an explanation. 

     

    Quite honestly even the basic sharing of such a low percentage of finished Reviews is a great piece of information for our Members to be aware of and makes me embarrassed that I am behind on my current own review.

     

    Regardless of "real life slowdowns" I believe and hope that the majority of participants value Element 14 and their awesome offerings to members.  I have heard of some members not being impressed with design challenge kits and would assume that would follow over for RoadTests as well, but personally I place a high value on every item I have received from Element 14 whether it be a little wind up blue dude, a cool Element 14 sticker, an Arduino UNO or even a pricey piece of test equipment.  They don't have to send me a single item, they don't owe me anything and I will do my best to support them in their endeavors, especially as they continue to provide so much to the members, new and old.

  • in reply to kas.lewis

    I suspected duplicate accounts too. People who enroll just to get their hands on the item, with no intention of giving back a review, would certainly have no moral issues with creating as many accounts as they can and enrolling with all of them.

     

    I've often wondered what these people could possibly say in their applications that they're sometimes chosen almost exclusively over actually qualified and willing roadtesters.

  • in reply to ntewinkel

    And being biased against that might reduce the chances of people who actually have 10 hours/week to work with a product.

  • in reply to jadew

    Interesting thoughts on multiple accounts.  Not knowing the logistics that Element 14 would have to go through, but perhaps a shipping address flagged list?  Of course this could be bypassed by multiple P.O. Box use, etc...  But depending on programming resources available I would think cross referencing accounts, shipping address and even IP might allow an interesting source to provide basic tracking.  Obviously having Randall monitor individually could turn quite brutal for his time, but if the bits could be captured and placed into a tool E14 can use, then perhaps that would be of assistance.

  • in reply to geralds

    Gerald Schwarz wrote:

     

    ... "Kauderwelsch".

     

     

    ---

    Hey, we call that "koeterwaals" around here...

  • in reply to rachaelp

    rachaelp  wrote:

     

    it seems it's not a few, for this particular supplier it's 87%

     

    Err ... without knowing the sample size, we don't know whether 87% is actually a large number of people.

     

    So it may be just a small number of people, and this supplier has been unlucky that they made up a large percentage of their testers ... ?

     

    Not that this excuses anything, of course - but we don't want to get things unnecessarily out of proportion.

     

     

     

     

     

     

  • in reply to awneil

    Andy Neil wrote:

     

    rachaelp wrote:

     

    it seems it's not a few, for this particular supplier it's 87%

     

    Err ... without knowing the sample size, we don't know whether 87% is actually a large number of people.

     

    So it may be just a small number of people, and this supplier has been unlucky that they made up a large percentage of their testers ... ?

     

    Not that this excuses anything, of course - but we don't want to get things unnecessarily out of proportion.

     

     

     

     

     

     

     

    Even if this supplier is unlucky and their percentage failure rate is higher, I'm sure it's a problem across the board otherwise I suspect the focus would be why that supplier had such a low success rate and try to fix that rather than put changes in across the entire road test program to combat the issue so we can probably assume that, whilst it might not be as bad for all suppliers, it's a common problem.

     

    To your point, yes we don't know the sample size but we know at least 7 people didn't submit a review for this one supplier. If I'd been giving product out for review and I only got results from 1 in 8 people I gave things to I would be quite annoyed.

     

    Best Regards,


    Rachael

  • in reply to awneil

    It's a large sample I didn't want to spell it out because the actual numbers are dreadful. Simplifying it to a percentage is bad enough. But things are improving immensely. People are getting their reviews in or notifying about upcoming review postings, which was not happening 2 weeks ago. Thanks.

  • in reply to rscasny

    Randall Scasny wrote:

    things are improving immensely. People are getting their reviews in or notifying about upcoming review postings, which was not happening 2 weeks ago. Thanks.

     

    Well - that's a result, then!

     

    image