Actions
Forum Thread Details
  • Replies 13 replies
  • Subscribers 2470 subscribers
  • Views 1139 views
  • Users 0 members are here
Related

Do You Have Enough Time to be a RoadTester?

rscasny

Let me tell you a story: I had someone apply to a roadtest and he planned on placing the development board (I don't remember which one) in a weather balloon his college was going to launch, and he was going to collect environmental data with its onboard sensors. At face value, it sounded like an interesting idea that people would be interested in learning about. Fast forward 60 days, the standard length of a roadtest, and I received no review and no emails by the roadtester explaining why he hadn't posted the review. So, as usual, my colleague, , followed up with the roadtester. We ultimately learned that the weather balloon launch was postponed until 2 months after the roadtest's deadline. In the end, I never got the review.

 

So, the question I posed in my title is a common one: Do you have enough time for a roadtest. My simple answer is...

 

(drum roll again...)

 

NO, IF YOU HAVE TO WAIT FOR A WEATHER BALLOON TO TAKE OFF!!!!!!!!!

 

I'm making light of the question, but it is a practical one that I am sure comes up with a lot of people who are considering to apply for a roadtest, but decide not to because they (a) believe (and I think wrongly) that all roadtests require a lot of time or (b) element14 and its sponsors are rigid about deadlines that they will not compromise when there are extenuating circumstances (again, is not quite right).

 

We give 60 days to complete a roadtest from the date you receive the product. In the past, I had entertained the possibility of extending it to 90 days, but I chose not to because some roadtesters I asked said it would not improve the roadtest completion overall. I also looked back at roadtests where I gave a longer time: the result was mixed. Sometimes people did their reviews and sometimes they did not.

 

It  is my observation that anyone who applies to a roadtest has the time to be a roadtester because the time it takes to complete the roadtest is SOLELY UP TO the roadtester. The time it takes depends on what you plan to do in the roadtest. That's one of the reasons I ask the question in the application: What's your testing procedure? (Be as specific as possible.) I have on occasion emailed an applicant who had an interesting procedure and asked him if he really could complete what he is proposing in less than 60 days. I say less than 60 days because every roadtest requires some research, some learning, and some organization such that a real roadtest is probably somewhere between 40 to 50 days.

 

In other words, the length of a roadtest is a causal relationship. The more complex your roadtest is, the longer it is going to take. There are some roadtesters who can manage a complex roadtest. And there are other roadtesters who have big ideas but when they receive the kit and now they to do something with it and write about it, they tend to freeze up. And I don't learn about it until we follow-up and by then the roadtest is down to less than 30 days and it is crunch time and they may not be in a position to do the roadtest.

 

Take a look at what you are proposing to do in the roadtest. Also, take a look your school and/or work schedule. If you are maxed out for time, then you may need to either (a) reassess what testing you can get done or (b) have a contingency roadtest, that is, a Plan B: if I don't have enough time to do Big Plan A, then I will do the SMALLER Plan B roadtest.

 

Your ability to organize, coordinate and manage a roadtest project, especially for students and early-career roadtesters, is an important skill you need to acquire to be successful as you progress in higher education or your business career. A roadtest can be good practice in developing these organizational and coordination skills.

 

The last thing I want to mention is Dan and myself are flexible to some degree. But this flexibility usually is extended to those roadtesters who communicate with us. If we just get 59 days of silence and you don't respond to our follow ups, you have put us in the uncomfortable position of reporting to the sponsor that the roadtest review is not going to happen. If we have to write off the review, we won't be able to use you again as a roadtester.

 

However, if you are communicating with us throughout the roadtester (or at the very least to 's email) telling us your obstacles or additional testing you need to conduct that will delay you publishing the review "x" number of days from then, we can report back to the sponsor when we expect it. This is another skill you can develop as a roadtester if it doesn't come natural to you: project communication.

 

In the end, most people have the time for the roadtest if they have a Plan B and they communicate with me what is going on. You control the complexity or the simplicity of the roadtest. You are in the driver's seat of your roadtest.

 

Randall Scasny

RoadTest Program Manager

  • About your point about Plan A and B, should you add that in your application, will it affect your application in any way, or something else?

  • in reply to hugohu

    Hugo,

     

    You can add it to your application.

     

    Since I realy on the testing procedure when selecting roadtest applicants, if you put a Plan B on the application, I would probably see that as a positive, although I would focus on the Plan A.

     

    Randall

  • in reply to rscasny

    Thanks.

    I'll always try to get Plan A but you make a really good point where it's good to have an alt plan.

    Now, let me edit my applications...

  •   Thank you for the recent series on Roadtesting.  I think the clarification will help alleviate some reservations about applying.

     

    I'd like to share my opinion that doing a Project14 Challenge first is a great way to prepare for a Roadtest.  Both are announced ahead of time to allow people to prepare and plan the project.  Both have deadlines.  By doing a Project14 Challenge first, I was reminded that my supply chain isn't exactly Wile E. Coyote's Acme Corporation.  For one project, I waited a month for parts and the order got cancelled.  What's my Plan B?  More importantly, when do I initiate Plan B?  How much time and effort can I really devote to the project?  Am I allowing project creep?  The Project14 Challenges have forced me to have a very real conversation with the guy in the mirror on what I can and can't get done in a set amount of time.

    "Realistic expectations" is the phrase we use at work.     My two cents.

  • There are schedule risks, technical risks, procurement risks, cost risks and then there is Murphy's law which involves unpredictable risks.

    It is often more than just a plan A and plan B as several aspects of a project can all go off the rails.

    When planning a project, each perceived risk could be assessed to decide if a contingency plan might be wise and thought through.

    Or just assume you will have to adjust as needed all the way through, but at least be ready for the extra time involved.

  • If there's one thing my previous roadtest taught me, is that procrastination doesn't pay.. ^_^

     

    It helps if you not only have a plan, but also stick to it. I ended up increasingly frustrating myself because the closer I got to the deadline, the more annoying it was that I didn't make as much progress as I wanted. And I knew where the lack of preogress came from. I learned a lot from that, and have plans to approach this differently in future projects.

     

    IMO: All in all, 60 days for a roadtest should be enough time. Some of the items I've seen passing by are impressive and feature-rich, but 60 days is a good amount of time. At least to get a review in.     

  • Thanks for the insight RS. I concur with your decision on 60 days for a RoadTest. My own approach is to respect the time allotted and ask for the extension if circumstance warrant it.  What surprises me is a lack of communication with your office.

     

    I'm reading, The Last Lecture by Randy Pausch a Carnegie Mellon professor of Computer Science and he tries to address the same problem with his students by using pier scoring them. How could a team of students work together and believe it is acceptable to not communicate.

     

    I start a project with a Plan A but know sooner than later, realize if a Plan B is needed. Most importantly when I need to communicate the change. I learned early in my career, that if I allow the clock to tick down closer to the deadline before engaging, I have greatly increased the chances of poor success. "If you knew that why didn't you tell us or why didn't you know earlier that was a problem." were statements I hoped to avoid in my career.

     

    Here is spit balling this teachable moment. I wish there was a water cooler so we could chat and I wouldn't have to write it down. Who am I kidding, the writing part is fun!

     

    The suggestion may increase your workload managing the RoadTest program exponentially or maybe not? The application must include a project plan. Provide the headings that are expected to be in the plan. The plan doesn't have to be a 40 page tomb put together by a project management office but should contain some idea of what can be expected over the course of a RoadTest.

    • Scope Management: (what are you going to do)
    • Requirements Management: (what are you going to need)
    • Schedule Management: (what is the timeline for doing it)
    • Communication Management: (what are milestones and how are they communicated)
    • Risk Management: (what are the risks and what are the plans to mitigate)

     

    I found these bullets from a project plan post on Wikipedia. The italicise entry is my minimization of the content for each heading. That process question on the application would include the bullets. Your experience RS, no doubt is looking to extract this information from what the author has submitted today. I think the bullets added to the form would help you find what is needed and help the applicant know what is expected. Maybe they become more aware that, hey crap this is work.

     

    I tend to error on the side of, they didn't know instead of they should have known. The RoadTest program in attracting new candidates is going to have an abundance of the "I didn't know." Once they know, whether they would do it or not is another story. I think one of the intangibles of the RoadTest is the candidate learns. They may not like the lesson but it is learning.

  • in reply to colporteur

    Some good points Sean.

    One note on the risks - it is the $64K question.

    It appears the standard risks you find in a risk management register are generally manageable - as all the successful road tests prove.

    The unsuccessful road tests generally do not fail due to a risk that would be identified.

    For example a road tester may have no intention of completing their road test - this is not going to show up in their risk management plan.

    The road tester may realize at some point they cannot do what they intended - they just underestimated the knowledge and work required and don't publish because they will look incompetent. This is a potential risk in lots of road tests but if the applicant could quantify this risk, which is unlikely, they still would not want to state it accurately up front because it would hurt their chances of winning the road test.

    It is sort of up to Randall to read between the lines and assess this risk based on the rest of the application. A tough job to be sure, as a sales pitch can always be convincing while at the same time it can be insincere.

    He cannot just select high probability of success road testers who have a good track record, because it would devolve into a cabal of trusted cronies with no possibility of outsiders joining in.

    Randall has made great strides to maximize participation (through extraordinary diligence and careful analysis) but the dilemma of the road test manager cannot be completely solved.

  • in reply to dougw

    RS has a number of struggles with running a successful RoadTest program. If I gave the impression my suggestion is the fix for all the woes that have besot the program that was not my intent.

     

    I agree with whole heartily with you on two points. Randall's efforts have greatly improved the program. Risk management will do little to mitigate free scavengers, those that have no intention of completing a RoadTest.

     

    I do believe the project list will make the hill to free stuff a little harder to climb by increasing the effort from those folks. Those truly interested in the RoadTest program are on a journey to learn. The list is just knowledge. My goal is through knowledge, understanding is developed. 

     

    Randall's post premise is time requirements. If you complete all the bullet headings before preparing a schedule, you have accumulated sufficient knowledge to develop an understand of time.

     

    The risk management part was to explore the need for alternate plans if your original plan hits a snag.

     

    I know I am being altruistic with the RoadTest program. I think it can be great way to develop a skill for some people. A skill that can be included on a resume.

  • Sometimes, you can craft a Plan A that is more bulletproof and can turn into a Plan B if necessary.

     

    For example -

    • Break down your review into separate chapters focusing on different elements. The less "overlap" and "dependencies" you have between them, the more likely a bump isn't going to disrupt the whole review.
    • Break down each chapter into subtasks or separate experiments. This way if you can't do something (e.g. I couldn't make a ripple-and-noise measurement accurate enough to publish), you can move on without jepoardising your whole review.

     

    This is perhaps more difficult for a review that looks at performing a project with a product as there is a "chain" of dependencies, but just as a baby cannot go from lying down to walking in an instant, perhaps small intermediate steps should be budgeted for in the plan - e.g. get a "Hello World" example working first, then make it a "Goodbye World" example, then make it beep as well, then make it detect a servo, then make it drive the servo one step, then make it move the servo a programmed number of steps ... etc. Think of things you can demonstrate along the journey and deliver that as a review assuming you got far enough. If not, ask for help on the forums, or with fellow RoadTesters.

     

    Even I, a seasoned RoadTester, have asked other fellow RoadTesters for help, to confirm certain findings, to determine if we have similar issues, etc. Most of us are happy to communicate if you are willing to communicate clearly, precisely and co-operate with answering questions we might have. This doesn't have to be done publicly as well (especially if you feel it is a sensitive question) - you might send a private message for example.

     

    - Gough