• Author Author: pettitda
  • Date Created: Date Created
  • Views 2279 views
  • Likes 6 likes
  • Comments 17 comments
Related
Recommended

How To Deal with RoadTest Goals That Were Unachievable

pettitda

Shortly after the Raspberry Pi 3 was released, Element14 announced a new road test entitled RoadTest Review a Raspberry Pi 3 Model B !  I was instantly excited.  I had followed the Raspberry Pi project for several years and from all the specifications, this looked to be a fantastic new member of the RPI family.  So, I went to work crafting a project idea to use the capabilities of the RPi 3 to the fullest.  I had recently lost my home and half of it's contents to a flood and most of what I didn't lose was in storage while my family and I lived with my mother in law.  So, doing a full-on electronics hardware project was out of the question.  I eventually came up with what I thought was the perfect project considering the circumstances; one which was almost totally software work.  This would be an excellent way to keep my mind occupied while I waited for my house to be rebuilt.

 

I had been following the Numenta Hierachical Temporal Memory (HTM) project for several years.  I originally read about it in an issue of the IEEE Spectrum.  I knew the project was based upon a open source code base which aimed to simulate how the brains works.  I also knew that the code was successfully learning patterns of data, predicting the next data point in a series, and flagging anomalies in a data set.  What I didn't know at the time was how exactly the code went about learning patterns or what it's limitations were.  Nevertheless, I went forward to describe a project with what I would eventually come to realize were very aggressive goals.  I proposed to build a raspberry pi based computer which would recognize speech and trigger actions based upon that speech pattern. 

 

In a short amount of time, I learned that I had been selected to road test the RPi 3 and a few days after that I received the Pi.  I jumped in with both feet, ordered a USB sound adapter, microphone, case, power supply, and SD card, then started working on the software. Porting the Numenta project to run on the Pi was slow work, but I found the user group on Github to be friendly and willing to advise me on getting it to build and run.  After two or three weeks the project was ported, built, and running on the Pi.  From that point I spent a couple more weeks getting the project updates for building on RPi up to the project standards for inclusion in the main line code.  While not strictly necessary, I figured this was the least I could do considering how helpful the group had been to me. 

 

From this point, I jumped into the examples online to look for a starting point from which to build my project.  I found examples of predicting the energy usage in a gym and similar sets of data which were long arrays of fairly simple data (i.e. a watt-hour reading for energy usage taken every hour for several weeks).  However, what I didn't find was examples of processing dense, complex data such as a person's voice.  Eventually, I came to the realization that the project goals that I had given myself were more appropriate for a PhD student doing original research for a doctoral thesis than for an engineer tinkering with the software on his off hours.  I was seriously disheartened.

 

If you find yourself in a similar circumstance, here's what you should do.  First notify the road test manager of your problem.  Seriously, these people are friendly and understanding and they want to see you succeed.  Second consider how you could refocus the project in order to make it successful.  Talk it over with the road test manager and get their input.  Post a message on the road test area and get feedback from the other road testers.  Perhaps there's an alternative way of looking at the problem that could get you going.  Third, if you are totally up against a brick wall, you may have to abandon the original project and come up with an alternative.  Again the road test manager can be extremely helpful.  Just because your original project proposal is not achievable doesn't mean that you can't complete a perfectly acceptable review of the product.  In the end, the goal is a review of the product which is insightful and helpful to others considering using the product in one of their projects.  Finally, blog about your failure.  The experience you learned in failure may be extremely valuable to others considering similar projects. 

 

I hope my humbling story will encourage you to not simply give up but make the most of your situation.  Happy road testing!

  • in reply to dougw

    In the past there were some explicit comments that road tester were not chosen because their intended projects were deemed to complex and not achievable in the given time frame. So its not that you should come up with a spectacular project - doing something simpler which can result in the same amount of feedback might be deemed to be of higher quality (because you can focus on the quality of the review).

  • in reply to Instructorman

    I think a road test is not a design challenge. The (completed) project is a means to do the review, but not the goal. A road test is about providing feedback to the manufacturer, and to other users / potential buyer which might be interested. And apart from amplifiers, there is no negative feedback: even when the road test ends with "for this intended purpose its completely useless because X" its useful. At least as long "X" is understandable, and comprehensible - because then the manufacturer at least knows how to either improve the product or change its marketing to avoid this target group.

    I had some reviews where the stated goal from the beginning was to verify the claims of the manufacturer, and just a large number of measurements and observations. There were some where I intended to use the item in a project, but for most of them I did not finish the project when I wrote the review. Sometimes there just wasn't time, sometimes I found out that the product could not be used for the intended purpose, or that there were some major road blocks which need to be solved first.

    When you find that the project most likely will not be finished in time for the road test review, its time to pivot. Not with the project, but with what you do for the road test. In one project (not on E14) I found out that, for a board worth $50, I would need to purchase a programmer for $200, and a IAR license (more more than $1000 probably) because the software library I would need to use doesn't run with the free tools. I stopped the project there, and reviewed all the other aspects of the board as good as I could (and in the end, came up with a small project which didn't need reprogramming). I failed completely with my original goal, but the review itself was fine (and the library in question is about to be made available for gcc now...).

  • in reply to awneil

    I agree with everyone - blogging about the work that was done in this case would be more than adequate for a road test review. However, if you want to improve your chances of participating it would seem that promising to do something spectacular is a positive factor. I don't see much of a way to get around this, impressive projects need to be favoured. It is just not an easy task to decide whether the contestant will be able to do a good job on an impressive proposal/project.

    As an example of needing an impressive proposal, I think I have applied for every Pi3 contest, road test and challenge I've come across without ever managing to propose something impressive enough to be selected. Actually not quite accurate - I did win one once as a prize, but it never arrived.

  • in reply to Instructorman

    I wonder to what extent this perspective is shared by other members, and officially, if Element 14 supports this perspective.

    That would be a good question for . 

  • in reply to awneil

    Andy,

    I agree with your perspective on the purpose of road tests.

     

    I wonder to what extent this perspective is shared by other members, and officially, if Element 14 supports this perspective.

     

    And David, thank you for sharing your harrowing experience.  I think your efforts are worthy of sharing and quite remarkable, in spite of not achieving your original goals, and your advice to others in similar situations is thoughtful and appreciated.

     

    Mark A