RE: Seeking Feedback on New Scoring System for Experimenting Challenges

The new scoring system sounds good. I have several minor notes:

• Blogs count minimum you need at least for finisher prizes. You can express it using score but I think it will still define minimum requirement on blog count. I recommend do not removing minimum of two blog posts. If you remove this limit, then you can face situation that someone post the only one (summary) blog in last week. In extreme case he/she can win with 200 points, but I think in this case single blog entry was not very useful contribution to the competition.
• Score must distinguish contestants. I recommend defining rule that in every category two contestants never receive exactly the same number of points. For example look 7 Ways to Leave Your Spartan-6: Contestant Standings. Look at First Project column. All has 0 points (did not complete project) or 50 points (completed project). If you take two projects from this table which received 50 points, then it means that both projects/blogs are totally equal in quality but in reality this is never true. There is always difference (at least very small difference) and scoring system should highlight it. You can for example give one project 45 and second project 44 points. Both contestant get very similar points for very similar quality of projects/blog but there is an order. The contestant with one point less can look at it and decide that he will post one additional intermediary blog for beating the originally slightly better contestant. Idea of this paragraph is that every time there must be the order. Giving all contestant 75 points for introductory blog does not make sense because all (except contestant who post nothing) will get them and there is no need to make any complex strategy in this case.
• Mentioned system rate blogs and project quality but do not count other contestant activities. Very frequently happen that someone ask on forum, someone answer question and help the other contestant to continue its journey or something like it. I think this should be graded and this kind of cooperation should be awarded by points, I think.
• In addition to previous point: forum activity should count. Both question and answers should count. In 7 ways to leave spartan there were category for this but in promoted system it is not. Contestant sometimes get into troubles and asking on forum is very effective way because at the same time there is another 9 contestant who have exactly the same hardware in the hands and together, they can resolve even very complicated issues.
• Carefully take in account amount of work needed to do on element14 side every week. We frequently see delays in selecting challengers, waiting for judges selection, postponing shipping dates and so on. I recommend changing “score updated on weekly basis” to “score updated every second week” and also remember to dedicate time slot every second Friday (or other day) in your calendar. If you will not have time for evaluating blogs every week, than space for complex strategies on contestant side decreases significantly. For allowing making strategies evaluation timing should be strictly followed.

Interesting points. For what it's worth, I think the "forum activity should count. Both question and answers should count." would be hard to deal with, because then judges need to go through all the comments as well as the blog content. Also, it could be open to abuse, by people just replying many times with not as much help as required (e.g. mundane comments instead of well thought out responses). i think the onus on getting answers to challenge issues should be on the contestants, they are best placed to do it, and if they cannot find the answer on the forum then they can ask a question anwhere, e.g. on their blog, on someone elses blog or other sites. Collecting up questions and answers in discussions seems (to me) difficult (unless I've misunderstood it). If people are not responding to questions asked of them, then it becomes clear they are not a team player, and you wouldn't get much of an answer from them even if there were (say) 10 points for a response.

Regarding ranking, sometimes content is just as good, so it would be nice if there was still a way to recognize that sometimes people do equally well in different ways. But I can see it would make it hard for contestants during the contest, who might prefer ranking (unless they are going to come second or third when their content may be equally as good! :) 

Interesting points. For what it's worth, I think the "forum activity should count. Both question and answers should count." would be hard to deal with, because then judges need to go through all the comments as well as the blog content. Also, it could be open to abuse, by people just replying many times with not as much help as required (e.g. mundane comments instead of well thought out responses). i think the onus on getting answers to challenge issues should be on the contestants, they are best placed to do it, and if they cannot find the answer on the forum then they can ask a question anwhere, e.g. on their blog, on someone elses blog or other sites. Collecting up questions and answers in discussions seems (to me) difficult (unless I've misunderstood it). If people are not responding to questions asked of them, then it becomes clear they are not a team player, and you wouldn't get much of an answer from them even if there were (say) 10 points for a response.

Regarding ranking, sometimes content is just as good, so it would be nice if there was still a way to recognize that sometimes people do equally well in different ways. But I can see it would make it hard for contestants during the contest, who might prefer ranking (unless they are going to come second or third when their content may be equally as good! :) 

I did not take abuses in account. Of course if some start flooding too much questions, then he/she will receive 0 or very few points for it. Similarly you can abuse intermediary blogs. Some can split his text in a ways one blog -> one paragraf and then he will easily have tons of intermediate blogs but I gues it will not receive lot of points for it. Even worse, you can post 50 blogs of lorem ipsum with idea that 50 blogs can bring you up to 500 points! In this cases the magic words "up to" should apply and zero points still satisfy rule up to 10 points.

This needs clarification. Let's say for example Week 3 starts on Jan 22 and Week 4 starts on Jan 29th and week 5 starts on Feb 5th. In other words, three weeks. Each week you will get awarded a maximum of ten points for supporting blogs. Hence, for 3 weeks, that's 30 points. If people understand that, they are not going to write 50 blogs nor expect 500 points. Thus, I need to work on clarifying these instructions. 

I'd suggest that the participants work and posts should weight an order of magnitude higher than the downstream forum activity. To keep the contests focusing on engineering versus popularity.

" I think the "forum activity should count. Both question and answers should count." would be hard to deal with, because then judges need to go through all the comments as well as the blog content."

As we tend to do Design Challenges / these competitions in a consolidated group - then the leaderboards which keep track of participation points can be a good steer for these.

We should also be able to award say, a gamification achievement badge for those who have their questions marked as a suggested/verified answer in a certain group.

So ultimately we could automate some amount of this that does checks and results in an achievement as a signifier of accomplishing requirements in a given group.

Then it's down to peer review and a reduction of administrative overhead.

I've worked on some aspect(s) of this already, it'd simply need picking up/awareness by the rest of the team...