Actions
Forum Thread Details
  • Replies 91 replies
  • Subscribers 2568 subscribers
  • Views 6808 views
  • Users 0 members are here
Related

Please Read: An Important RoadTest Program Change

rscasny

This is the third virtual "fireside chat" that I'm writing regarding the RoadTest Program. For this one, I'd like to roll out a RoadTest Program change and explain why the change is occurring.

 

To start, I'd like to share a snippet of an email I received yesterday from a vendor who participates in the RoadTest Program:

 

"We received only 13% reviews back from the RoadTests we sponsored in 2016. I would like to know the status of each of these roadtests. Are the missing reviews still being pursued? I would also like to know if the members that failed to post a review are now being flagged so we do not chose them again."

 

It was not pleasant to respond to. I had to eat "humble pie," as we say in the US, when I responded to this vendor.

 

To solve the problem, I decided I don't want to go down the road of suspending members from the RoadTest program. Suspensions sound too much like the hard rules of the military. I was in the Navy for 10 years and I didn't like tough rules back then, but I knew they were necessary given the reality of uncomfortable shipboard conditions and potentially dangerous operations we were participating in.

 

But this is a community. It should be a place to have some fun, meet new people, network professionally, be a mentor, or get mentored. Suspending people just is the wrong road to follow.

 

The way I've chosen to resolve the problem is the program change.

 

RoadTest application reports will soon include whether a specific applicant has completed all of his/her previous RoadTest reviews. I'm told by our developers this is very easy to do. We will provide this information to the vendor with a RoadTester's application. The supplier will make the decision where to select the RoadTester or not. Given the above comment by the supplier, I think the answer is self-evident.

 

Now, this change will probably cause me more work. If someone were flagged, I will have to check to see if that person is on a current RoadTest and it's before the due date of the official review. I surely don't want to penalize a conscientious RoadTester. I will also have to ensure there are not any extenuating circumstances that have caused the person not to write the review, i.e., sickness, family issues.

 

I am ambivalent about this whole thing. There are many RoadTesters who are doing a fantastic job. But a 13% success rate is not successful. I really have no choice but to implement this change.

 

Sincerely,

 

Randall Scasny

RoadTest Program Manager

Parents

  • Randall,

     

    I have been pondering the issue of dead beat road testers.  It would be great if the Road Test program could be structured around game theory principles so that a Nash Equilibrium could be achieved where all "players" take into consideration the strategies of all other "players" to reach a probability of optimal outcome.  As I see it, the Road Test program is not structured as a game that can be driven by Nash Equilibrium concepts. Here is why:

     

    Applicants take no risk to participate in the game and any strategy they employ in deciding their moves is not influenced by the strategies employed by others in the game.  Player strategies are effectively decoupled from each other. The participation cost is the effort required to compose and submit a road test application.  The benefit of winning is free stuff desired by the applicant.  The cost of failure to win is low, about the same as going fishing on a sunny afternoon and not catching anything.  Still had a nice afternoon outside.  Failure to submit a review after winning bears little, if any, cost to the applicant.  If an applicant is not morally invested in the community then any disapprobation by the community for failure to deliver on promises is inconsequential and therefore not a deterrent to grab and go strategies.

     

    As pointed out by , it is possible to masquerade as a morally invested member by exploiting point bonanzas to rapidly level up.  So, there will always be a non-zero risk in selecting unknown members for Road Test prizes.  Even your reasonable proposal to conduct due diligence screens based on past performance is not assured to prevent Road Test hit and runs. Even though, as most investment advisors will tell you, past performance is not an indicator of future performance, I think past performance is a fair indicator of likely future performance (even with stocks), so I fully support your Road Test program change because I think it will improve response rates.

     

    Mark A.

Reply

  • Randall,

     

    I have been pondering the issue of dead beat road testers.  It would be great if the Road Test program could be structured around game theory principles so that a Nash Equilibrium could be achieved where all "players" take into consideration the strategies of all other "players" to reach a probability of optimal outcome.  As I see it, the Road Test program is not structured as a game that can be driven by Nash Equilibrium concepts. Here is why:

     

    Applicants take no risk to participate in the game and any strategy they employ in deciding their moves is not influenced by the strategies employed by others in the game.  Player strategies are effectively decoupled from each other. The participation cost is the effort required to compose and submit a road test application.  The benefit of winning is free stuff desired by the applicant.  The cost of failure to win is low, about the same as going fishing on a sunny afternoon and not catching anything.  Still had a nice afternoon outside.  Failure to submit a review after winning bears little, if any, cost to the applicant.  If an applicant is not morally invested in the community then any disapprobation by the community for failure to deliver on promises is inconsequential and therefore not a deterrent to grab and go strategies.

     

    As pointed out by , it is possible to masquerade as a morally invested member by exploiting point bonanzas to rapidly level up.  So, there will always be a non-zero risk in selecting unknown members for Road Test prizes.  Even your reasonable proposal to conduct due diligence screens based on past performance is not assured to prevent Road Test hit and runs. Even though, as most investment advisors will tell you, past performance is not an indicator of future performance, I think past performance is a fair indicator of likely future performance (even with stocks), so I fully support your Road Test program change because I think it will improve response rates.

     

    Mark A.

Children
No Data