Please Read: An Important RoadTest Program Change

I just saw today's updated message and thought to respond, only to find there is a rather large chain of older responses linked to this message. So apologies if I'm repeating what someone else has said.

My suggestion is that you follow the StackOverflow approach and be rigorous, and maybe a little draconian, by creating points and/or level thresholds for each road test which you then use to compare against a member's reputation points and level. The Element14 system already has a points and a level tracking system so why not put this into practice. So for example, if a new user applies for a road test and has never once contributed to a blog or helped someone, then you can treat them as an unknown entity as they have zero points and level 0 (so a high-risk candidate). Then the more road testing and contributions the more points you get / higher your level and therefore it will be easier to do the next road test etc. as they are lower risk.

Now for this to work you need to be upfront in the road test description and state the minimum thresholds. Also suggest that for high-cost road test items, you raise the threshold etc. for someone to do the road test etc. This then gives you a good way to manage risk etc. of not getting a response. You could also provide "yellow cards" or even points / level deductions for those not completing the road test.

BigG, the thing with using the point systems is that the same people (me) would get selected all the time.

It's a hard thing to manage. If you give someone a yellow card, there's no stopping that person to create a new account.

The way it's done now - selecting on the merit of the application and filtering when one hasn't logged on since submitting the application - seems to work reasonably well.

Hi Jan,

My suggestion is to use the points only as a minimum threshold, to help from a risk management perspective, rather than having it where the person with the highest number of points gets the product. Thus as suggested those more expensive products have higher thresholds or this is left to the manufacturer or yourself to determine. As said it is then displayed in the description... only those people with a minimum of xx points can apply.

Deductions should apply IMHO as setting up a new account puts the person back to zero anyway which is probably much worse.

The problem faced is that the points system scores community involvement not suitability for roadtest.

The 600v roadtest is a good example where a non member with niche skills might actually be the best candidate and would sign up just to complete the roadtest. They could add value for the manufacturer and community but may not convert to a regular contributor if there is lack of stuff here to interest them.

Statistics for the blog posts are able to calculate the different kind of accesses to a page; on thing is the number of views including all the people that for some reason opened a page. Another thing is the number of viewers; it is the number of people that - I suppose - really read the article or comment or interact more in depth with the post. Maybe interesting to evaluate the points giving a different weight when a user just click-and-go instead of being a viewer. Ordered by blog posts (articles, documents etc) and filtered by user name. If this can be an automated query a better design of the member can be depicted based on his participation to the community.

Enrico

Yes that is true. But even though you may need a niche skill set for certain road tests, that person is still an unknown, with regards to the likelihood of completing a review, if they have never contributed before, and their claim to having the skills and promising to do a review is unverified at the time of application. Hence the need for some evaluation / verification tool and the use of points is at least some sort of risk evaluation. It may not be perfect but better than nothing. Then it is really up to the manufacturer, in those situations, to accept that person's application at face value, knowing that the likelihood of non compliance is quite high because of the applicant's lack of points / prior contributions. Thus you have now pushed that risk back to them.

Remember with road tests you are required to provide one extra piece of data which can also be used for cross checking (in case of duplicate accounts) which is the physical delivery address. As teleporting has yet to materialise :-) goods have to be shipped somewhere. This probably involves more work but it does add checks and balances if warranted.

Yes that is true. But even though you may need a niche skill set for certain road tests, that person is still an unknown, with regards to the likelihood of completing a review, if they have never contributed before, and their claim to having the skills and promising to do a review is unverified at the time of application. Hence the need for some evaluation / verification tool and the use of points is at least some sort of risk evaluation. It may not be perfect but better than nothing. Then it is really up to the manufacturer, in those situations, to accept that person's application at face value, knowing that the likelihood of non compliance is quite high because of the applicant's lack of points / prior contributions. Thus you have now pushed that risk back to them.

Remember with road tests you are required to provide one extra piece of data which can also be used for cross checking (in case of duplicate accounts) which is the physical delivery address. As teleporting has yet to materialise :-) goods have to be shipped somewhere. This probably involves more work but it does add checks and balances if warranted.