Actions
Forum Thread Details
  • Replies 70 replies
  • Subscribers 2567 subscribers
  • Views 5783 views
  • Users 0 members are here
Related

Roadtesting stats

neuromodulator

Today out of boredom and curiosity, motivated by other posts on site stats I decided to compute some stats of the published roadtests. There are a couple of interesting things that can be computed and would be interesting to know, specially in terms of correlations. For instance, do member points increase the chances of winning expensive items? What about number of posts? There are many questions I was wondering about to better understand the dynamics of the system. One of the first questions (and easiest to answer with my data) was how many users have won and published different amount of roadtests. And the results I obtained were these:

 

1: 359

2: 68

3: 32

4: 11

5: 9

6: 10

7: 7

8: 2

9: 0

10: 2

11: 3

12: 1

13: 1

14: 2

15: 0

16: 0

17: 1

18: 1

 

That is, 356 users have won and published 1 roadtest, and 1 user has won and published 18 roadtests. Ideally I would like to count won roadtests (not just published ones), having such data would also be useful to calculate the number of users that win roadtests and never publish anything. Other interesting stats could be to analyse trends, for instance, have the number of words in roadtests reviews increased? As the competition has gotten harder, I would expect to see some visible trends since the beginning of the roadtesting program.

  • This is very interesting to see. Thank you for doing this and sharing! I agree that I would like to see different stats for these as well. Seeing the percentage of people who win compared to those who actually publish would be interesting. I wonder if Roadtests have a higher completion/participation rate than Design Challenges. But again thank you for sharing this work and providing a great conversation topic.

  • in reply to Jan Cumps

    Applied for 6 and selected for 2 for me - I was surprised it’s many as 6 over the years, I’m pretty selective and also my work keeps me busy... good job I love it, I’m lucky in that respect for sure! image

  • As many enjoyed the post I'll probably post a few more analyses. As of now, my data only contains the items that got roadtested, the name of the testers and the review uploading date. To perform more interesting analyses I'll have to scrape more data, which may take some time, unless Dudley can give me a hand and provide me an easier access to the public data. image

  • I don't know if there is a formula for success buried in here but for those that are interested:

    I have applied for 25 road tests.

    I have been successful on 15 applications. The 12 listed on the Road Tests Authored page only go back to 2015.

    I was only successful on 1 of the first 4 applications.

    I was successful on 4 out of the first 10 applications. Only one of these was expensive - a Cel Robox 3D printer.

    I had been road testing for 4 years (20 applications) before I got to road test an R&S oscilloscope.

    I have been successful on 9 of the last 10 applications.

    I know my applications have improved over time although the road tests themselves may only be slightly improved - it depends on the device.

    My credibility has definitely improved over time. Putting a big effort into some minor road tests helped a lot. Always putting forth a good effort was crucial.

    Increased general participation on the forum also helped significantly.

  • in reply to dougw

    Thanks for the tips Douglas Wong! I won my first roadtest after the 4th attempt too. But I've also seen some low rank, low participation members scoring multiple thousands dollar equipment, so I guess its still possible to get expensive stuff right from the start.

  • in reply to dougw

    You've inspired me to check in much the same way ...

    • RoadTests Won - 14 [1 in progress, 2 awarded awaiting commencement]
    • RoadTests Published - 11
    • RoadTests Applied - 26
    • Number of RoadTests applied to before I got awarded my first - 3
    • Successes in first 10 applications - 5
    • Successes in latest 10 applications - 7
    • Time to R&S Oscilloscope - 22 applications, 10 delivered reviews, 7 years since the first application.

     

    I definitely think that I've become slightly more selective but also become more proficient at writing applications leading to a marginally greater chance of success. But overall, there were quite a few products I've applied to and didn't get - but that's not a bad rate overall, all things considered. I think I've actually improved quite a bit and the "snowball" effect is starting to take effect - namely that those with previously awarded test equipment are the ones with the equipment to test the new equipment and compare them, making it more likely to lead to an interesting and informative review, and increasing my chances of being chosen.

     

    - Gough

  • in reply to Gough Lui

    Those are great stats (not just personally) but showing the progression and why you may have more than others.

     

    Thanks for sharing

    Cheers

    Mark

  • in reply to dougw

    For me its more than 50 applications over nearly 6 years now (although that also includes the proposals  for the design challenges). Looking back at the first ones I wrote makes me wondering how I ever got one accepted... It took more than a year and 10 applications (although I got accepted for a design challenge in the meantime - maybe that helped).

    May acceptance rate for development kits / boards is quite good, but for test equipment its quite bad. Seems I need to work on my skills in that area... (sometimes I wish I could get some feedback on why I was not accepted, but I know its way too much work to do that)

  • in reply to hlipka

    sometimes I wish I could get some feedback on why I was not accepted, but I know its way too much work to do that

    That is one of the reasons I've been advocating the publishing of the selected testers applications.