Disappointed - My opinion

No I guess Andy Clark (Workshopshed ). I had a thought of doing it, let me see if I can collate some useful stats for different design challenges.

Hello All

As the person that does select challengers and runs Design Challenges, I am very invested in this thread. I appreciate you thinking about this on my behalf. You all have echoed conversations we have had within the e14 Team and within the Top Member circle.

To address some of your recommendations and questions, I can confirm that when checking applications we first check them for originality, and we do select the best applications based on a few sets criteria.

I personally do get disheartened when some people ghost on us... even more when their proposed application was fantastic.

I also understand that some people have better technical and practical skills than written English skills and try to keep an open mind on this when selecting challengers. Every challenger I accept represents a project I would like to see brought into the world and is most likely off the back of a convincing application.

We do track blog progress against the timeline and check in on stragglers, this is often when we spot the ones that have ghosted within a 3 weeks of launch. That is when we reach out to them, via what ever methods they have added to their applications. This 3 week period is where you could consider new challengers, but in challenges where we have a limited amount of kits, this is not an option.

We sanity check new challengers and new users but do also take a few chances, and will continue to do so, for example, 3 or 4 of the most recent design challenges have had new members as winners and many others as finisher prize winners.

These success stories are the reason I continue to take risks on new members as well as a mix of trustworthy members.

We find a lot of people are daunted by the size of design challenges so try to keep the barriers to entry a low as possible and also promote project 14 as something with easier access.

Now... lets discuss how we can weight these a little bit more to reduce drop numbers.

Originally a high quality application was seen to be enough of a deterrent to weed out most of the people testing their luck.

This no longer seems to be enough.

As for people building up reputation before being invited to join a challenge, my stance is that I will continue to take applications from new and old members, but I think there is a need for a skype call with applicants that could be classed as higher risk.

I will update the application page to help be more transparent in regards to what is required of a challenger.

I want to confirm my stance of keeping the barrier to entry a low as possible to new members but also confirm the intention of those joining.

Thank you all for your input and suggestions, please keep it coming!

-Phil Hutchinson

Hi Phil,

From my perspective there is a lot of interesting Design Challenge output, there is always something to read! (I just wish each participant could bear in mind that a reader may not have the benefit of reading all previous posts when a blog post is written each week - it is hard to recall what each project is about when there are many challengers and blog posts. Some people are great at this, e.g. dixonselvan content has an overview or convenient links on every blog post, and others provide an introduction too, which is great.

I think your current screening approach/balance is therefore very good judging by the quality and volume of output, and can only get better with time as it is refined in the manner you say, e.g. Skype calls and other 1:1 methods to get confidence in participants where needed.

It might be helpful to reach out _before the challenge_ to new applicants (ones without track record). While this will not weed out the ones that really want to free ride, but it still might help (it similar to Randall reaching out to road tester after the selection was made - and it resulted in some people not replying any more).

OTOH when applicants for a challenge do bail on the actual project, it mostly rises the costs. So when looking at the "Design for a cause" challenge, 16 board for 30EUR each were handed out. When half of these board disappear without project, its about the same as when you would run a challenge with 8 boards costing 60EUR each. So the question is: would you run the same challenge under this new assumption? If your answer is 'yes', then half of the applicants dropping out would not be a problem. If your answer is 'no', then more scrutiny is needed (and maybe you select more of the seasoned applicants).

I personally dropped out of one design challenge (not an E14 one). Back then I was approached by the manufacturer running it (because I had participated in a previous one and was somewhat successful), told him I could have a look but probably would lack the time for a real project. They did send the stuff nonetheless, and in the end I did not have enough time. I considered that fair, but afterwards I carefully considered each challenge based on how much fun it would be (which includes and interesting topic and interesting stuff to work with) and how much effort (vs. the time at hand).

What I would like to know: is there a difference in drop-out rates depending on the price value? Do people try to finish more when there is something valuable to win?

A separate topic but perhaps all that linking stuff could be made easier for people doing a blog series such as a design challenge. I do remember it being a pain to have to manually add all the links.

This is something that we have put some thought into.  Allowing people move from post to post.

We have some project shakeups coming soon.

e14phil  wrote:

 

This is something that we have put some thought into.  Allowing people move from post to post.

We have some project shakeups coming soon.

Interested! Many of my posts are series.

Also I think some stuff needs to be done by authors, to aid the reader into understanding a blog post in isolation.

I hate to use an example, I'm not putting them down, but see here (this was just on the list of recent updates):

Week 8(?) - Putting It All Together

It is hard to know what this project is about : ( and even if there were links to previous parts, I'm not sure anything on the page would motivate some to click and see (say) Week 1, in the hope it describes it more. And yet if they did, perhaps this week 8 blog post would make total sense to many more people - clearly a lot of work has gone into it, judging by the photo on that blog post. It's just a shame it's not been conveyed to potential readers. In fact some may never click on that page at all, because the blog title doesn't hint at what it's about, except that something got put together in week 8 : ( To be clear, the project looks super interesting from the photo, and it looks neatly assembled, just not many people may see it : (

I feel that people tend to over-promise when they haven't done enough research on what they plan on doing. Some might try to propose a project that requires that they learn something new (which is good, because you're more likely to go through and learn it since you've made a commitment), but if the learning curve is too steep, they tend to abandon the project entirely.

The same seems to be happening for road tests occasionally: you tend to see some one/two paragraph reviews where the applicant simply states that 'the board was difficult to work with because of the lack of documentation', which if I recall correctly seems to be happening a lot with MCU dev boards, since people who have worked with Arduinos tends to try to jump in and try complicated ARM boards.

I'm guessing that they did have very good project proposals considering that e14 selected them, and the proposal format usually requires that an applicant includes data on their experience, skills & commitment.

One possible solution would be that besides simply including the 'idea' (i.e. what you want to build), e14 makes it compulsory to include:

• details like how an applicant plans on doing it: to check how well the applicant has researched the finer details of implementation (eg: number of ADCs/SPIs available for sensors on a particular MCU).
• for roadtests: has the applicant looked at the documentation of the product or tried out the software in demo mode?
• how familiar he/she is with the parts: a person who's only used an Arduino will probably not be able to build a complex FPGA project.
• what are the expected difficulties/why the project might not work - this will tell the reviewer whether the applicant understands the complexities of the project & understands how easy it will be to implement in practice (eg complex control systems  which require a lot of tuning).

Yes, shabaz. Its sometimes difficult to see a design contest post in its context. Maybe e14 could help with making a contest series of blogs more consistent. Or versed bloggers could step in to help contestants to put a line in the story?