Actions
Forum Thread Details
  • Replies 91 replies
  • Subscribers 2568 subscribers
  • Views 6794 views
  • Users 0 members are here
Related

Please Read: An Important RoadTest Program Change

rscasny

This is the third virtual "fireside chat" that I'm writing regarding the RoadTest Program. For this one, I'd like to roll out a RoadTest Program change and explain why the change is occurring.

 

To start, I'd like to share a snippet of an email I received yesterday from a vendor who participates in the RoadTest Program:

 

"We received only 13% reviews back from the RoadTests we sponsored in 2016. I would like to know the status of each of these roadtests. Are the missing reviews still being pursued? I would also like to know if the members that failed to post a review are now being flagged so we do not chose them again."

 

It was not pleasant to respond to. I had to eat "humble pie," as we say in the US, when I responded to this vendor.

 

To solve the problem, I decided I don't want to go down the road of suspending members from the RoadTest program. Suspensions sound too much like the hard rules of the military. I was in the Navy for 10 years and I didn't like tough rules back then, but I knew they were necessary given the reality of uncomfortable shipboard conditions and potentially dangerous operations we were participating in.

 

But this is a community. It should be a place to have some fun, meet new people, network professionally, be a mentor, or get mentored. Suspending people just is the wrong road to follow.

 

The way I've chosen to resolve the problem is the program change.

 

RoadTest application reports will soon include whether a specific applicant has completed all of his/her previous RoadTest reviews. I'm told by our developers this is very easy to do. We will provide this information to the vendor with a RoadTester's application. The supplier will make the decision where to select the RoadTester or not. Given the above comment by the supplier, I think the answer is self-evident.

 

Now, this change will probably cause me more work. If someone were flagged, I will have to check to see if that person is on a current RoadTest and it's before the due date of the official review. I surely don't want to penalize a conscientious RoadTester. I will also have to ensure there are not any extenuating circumstances that have caused the person not to write the review, i.e., sickness, family issues.

 

I am ambivalent about this whole thing. There are many RoadTesters who are doing a fantastic job. But a 13% success rate is not successful. I really have no choice but to implement this change.

 

Sincerely,

 

Randall Scasny

RoadTest Program Manager

  • in reply to mcb1

    Mark wrote: ...here is the banner...

    Well that's quite misleading actually... no wonder people don't bother to do a review when it's "free" and a "giveaway". I guess the fine print in the roadtest T&C's mention it, but many might not bother to read that much detail.

     

    I kinda always thought that the "free giveaway" roadtests were meant to attract people to the forum, but I guess the suppliers/sponsors don't feel that way.

     

    -Nico

  • in reply to ntewinkel

    I kinda always thought that the "free giveaway" roadtests were meant to attract people to the forum,

    If that's the incentive to join the community, then it's not right.

     

    People should be here to learn, share and intereact, not just for freebies. IMO

  • in reply to mcb1

    People should be here to learn, share and intereact, not just for freebies. IMO

     

    Totally agree. But the banner seems to tell another story.

     

    Maybe that's why the community numbers at something like half a million people, but only a few hundred have much activity.

  • in reply to mcb1

    Thanks Mark and everyone who has commented on the text in our banner.

     

    I don't know how I missed it. I guess you always need two sets of eyes or more to catch this stuff. I had it updated:

     

    RoadTests & Reviews

  • in reply to rscasny

    Hi Randall,

     

    That's much better although if it were me I would revise it slightly further to be absolutely clear.

     

    From:

    If selected, you'll receive a new product for free in exchange for your official RoadTest review.

     

    To:

    If selected, you'll receive a new product free of charge in exchange for your official RoadTest review.

     

    Best Regards,

     

    Rachael

  • Randall,

     

    I have been pondering the issue of dead beat road testers.  It would be great if the Road Test program could be structured around game theory principles so that a Nash Equilibrium could be achieved where all "players" take into consideration the strategies of all other "players" to reach a probability of optimal outcome.  As I see it, the Road Test program is not structured as a game that can be driven by Nash Equilibrium concepts. Here is why:

     

    Applicants take no risk to participate in the game and any strategy they employ in deciding their moves is not influenced by the strategies employed by others in the game.  Player strategies are effectively decoupled from each other. The participation cost is the effort required to compose and submit a road test application.  The benefit of winning is free stuff desired by the applicant.  The cost of failure to win is low, about the same as going fishing on a sunny afternoon and not catching anything.  Still had a nice afternoon outside.  Failure to submit a review after winning bears little, if any, cost to the applicant.  If an applicant is not morally invested in the community then any disapprobation by the community for failure to deliver on promises is inconsequential and therefore not a deterrent to grab and go strategies.

     

    As pointed out by , it is possible to masquerade as a morally invested member by exploiting point bonanzas to rapidly level up.  So, there will always be a non-zero risk in selecting unknown members for Road Test prizes.  Even your reasonable proposal to conduct due diligence screens based on past performance is not assured to prevent Road Test hit and runs. Even though, as most investment advisors will tell you, past performance is not an indicator of future performance, I think past performance is a fair indicator of likely future performance (even with stocks), so I fully support your Road Test program change because I think it will improve response rates.

     

    Mark A.

  • in reply to rscasny

    The one on the homepage is still as I posted before:

     

    image

     

    Still suggests that I can enroll to win this product for free...

     

    Kas

  • in reply to rscasny

    Cheers.

    Sometimes you can look at something many times over and still not see it.

  • Wouldn't a tiered system of promotion rather than punishment be easier to manage and more difficult to game with multiple accounts, etc.? By which I mean that if you complete a good review on a less popular piece of equipment you then become eligible for the more popular items. This should increase both the quality and return rate of reviews. It's frustrating for community members to miss out on a road test you think you could do a good job on and see either no reviews appear or pointless ones. When someone is supposed to review an oscilloscope and decides pictures of their dog are relevant, you know it's just typing for the sake of typing.

     

    Personally, I only apply for testing something that I believe I can review well and that I have a particular interest in. I imagine most people are the same, but I'm sure there are people who just think "ooh. free stuff."

  • Hi Randall,

     

    I was lucky to be selected for some roadtests in the past. What I noticed, at least it is my impression, that roadtests became more and more big projects and would take too much time for me to finish. It also became a contest of who would make the most genious and biggest project out of it. (again, my impression of it) So I stopped applying because it is a hobby for me, and not my daily job.

     

    Will you take this into consideration too?

     

    I always liked the idea of the roadtest ...

     

    Kind regards,

     

    Reinoud