Actions
Forum Thread Details
  • Replies 24 replies
  • Subscribers 2564 subscribers
  • Views 1880 views
  • Users 0 members are here
Related

RoadTests: Project vs Unboxing vs Experiment vs Other

rscasny

Over the past few months, I have implemented some changes to the RoadTest program. Some of these changes had to do with things that needed to change in order to have the program work for all stakeholders. Of course, some of them have to do with things that would help me manage the program better (e.g., asking applicants to use their profile email address on applications or my asking supplemental questions.)

 

But one comment caught my attention last week. To paraphrase it, the commenter thought that RoadTests were more or less morphing into prototyping "big" projects. This has led some individuals not to apply because of a lack of time.

 

While creating a big project would be nice, it's definitely not required. After all, we give you 60 days to complete testing and write a review. Not a whole lot of time to create something totally new.

 

A RoadTest can be a combination of performing an unboxing (start up/installation), conducting some experiments or tests (e.g., verify a power supply's load regulation is in specs), building a prototype, or doing something else.

 

When I review the applications -- all of which get sent to the supplier -- I do provide some recommendations. Sometimes they go with my recommendations, sometimes they do not.

 

But I've observed the following things regarding applications:

 

  • We are looking for a concise plan of testing.
  • Is the project do-able? Some projects aren't in 60 days.
  • Does the RoadTester appear to be able to pull off the testing and write the review.
  • The plan isn't being re-hashed over multiple RoadTests.
  • The application sounds like the applicant took the time to study the product; we have had some RoadTesters that submitted applications that were irrelevant to the product being tested.
  • Long applications may get looked at, but the winners are a combination of long or short applications. The application should be only as long as need be to explain your testing plan. But 1 sentence applications are rarely taken seriously.

 

I encourage everyone who is interested to keep on applying for RoadTests. Since we are given a limited number of units, we can't get everyone to test all the time. But a good plan will rise to the top.

 

Randall Scasny

RoadTest Program Manager

Parents

  • I think this is a really really tough issue, but it is great to see efforts to tackle it.

    For me, the fact that the judging criteria have an element of merit is absolutely crucial.

    Even though I have never been selected to road test a multi-thousand dollar instrument, I still prefer the proposals to be judged on merit - hopefully I will eventually figure out how to be successful on those proposals. So far I am learning a lot about what doesn't work.

    The closer the judging process becomes to a lottery, the less likely it is that significant effort will be expended on either the proposal or the project. Why waste your time on a big effort if it doesn't improve your chances?

    If you want good projects, there needs to be a motive to do a good project, and that motive is: if you don't do a good job, you won't be selected.

    As soon as merit is a factor, the more ambitious plans will generally have a better chance of winning. Even if there is a distribution of winners where some less meritorious projects are selected along with some highly meritorious projects, the best chance of winning is still by submitting an ambitious project. (Because the less meritorious projects are much like a lottery with a low probability of winning and no control over whether you win or not)

    However, I do like the idea of encouraging less experienced members to participate, it is just hard for sponsors to accept the lower quality that might result. They need to understand that their client base (and this forum) can shrink if they only cater to a small segment.

    So I do like your method of selecting a distribution of winners....The hard part is to figure out what distribution will yield best results across both the membership and the sponsor base.

  • in reply to dougw

    Merit != Ambition. A project proposal does not have merit, only the person has it. I think merit means that you have proven to finish what you promised, on time. It doesn't mean to propose big and fancy projects. OK, if you do propose and manage to finish them on time, then thats good for future applications. But someone proposing smaller things and finishes them on time and writes a good review probably has higher merits than someone coming up with fancy ideas and does not manage to finish them (and then probably writes a bad review).

    I have written ambitious proposals in the past, and got rejected. Maybe with more merit and proving that I can fulfil on the road tests I would have selected. Maybe even then they would have been deemed too ambitious, who knows?

Reply
  • in reply to dougw

    Merit != Ambition. A project proposal does not have merit, only the person has it. I think merit means that you have proven to finish what you promised, on time. It doesn't mean to propose big and fancy projects. OK, if you do propose and manage to finish them on time, then thats good for future applications. But someone proposing smaller things and finishes them on time and writes a good review probably has higher merits than someone coming up with fancy ideas and does not manage to finish them (and then probably writes a bad review).

    I have written ambitious proposals in the past, and got rejected. Maybe with more merit and proving that I can fulfil on the road tests I would have selected. Maybe even then they would have been deemed too ambitious, who knows?

Children
  • in reply to hlipka

    Merit != Ambition. A project proposal does not have merit, only the person has it

    Sorry but I'm not sure I agree with that.

     

    We've seen some Roadtests where the applicant used the device for something outside of it's normal use.

    I used the Insulation Resistance tester to prove it was a very easy method of testing Line Protection devices used on Krone style Distribution Frames.

    This probably was a very left field application, but for our field technicians it could be a 5 min test, which was difficult to do and certainly not in the field.

     

    I saw another Roadtest where the writer was checking rocks ..... which probably wasn't the manufacturers first thoughts.

     

     

    So I think there is merit in some ideas.

    Whether the applicant can pull it off is another conversation altogether.!

     

    Mark