Actions
Forum Thread Details
  • Replies 34 replies
  • Subscribers 12 subscribers
  • Views 4723 views
  • Users 0 members are here
Related

RE: Seeking Feedback on New Scoring System for Experimenting Challenges

rscasny

Note: this discussion covers only the Experimenting Design Challenge Competitions

Greetings to All,

I would like to introduce a blog scoring system in addition to the judges I use to determine the top prize winners of the experimenting with components challenges (e.g, capacitors, inductors, vibration sensors, etc.).

I would like to get feedback on this proposal by members of element14.

Before I detail the change, let me explain why I am suggesting it? There are a couple of reasons:

  • Transparency: The blog will be scored on a weekly basis and the score posted in a table similar to what I used in another one of my programs. Here is an example of the table I had used in that program. Click here to see.
  • Strategy: With participants seeing the point totals of all participants, they can plan a strategy for achieving a point total to win.
  • Ensuring Participation Throughout the Program Period: Sometimes participants post their blogs at the end of the 8 to 10 week period, not giving me the opportunity to promote the blogs through the program period. In addition, giving participants a reason to post supporting blogs throughout the active period allows them to pick up points along the way.
  • Making Judging More Convenient: Our judges will receive the final summary blog to judge, which I think will make it easier for them to manage. Since the final summary blog can earn the most points, the judges' decision will have the most weight in the final selections.

What is the Judging/Scoring System In Place Now?

There is a two-blog minimum required to successfully finish the competition. There is no scoring system in place now. Currently, at the conclusion of the competition (usually 8 to 10 weeks), I forward all the blogs of each participant to the judges. The judges rank the blogs and select the winners of the top prizes. At the end of the judging, I notify the winners and write a winner's announcement.

What is the Judging/Scoring System That I Would Like to Introduce?

There will be no blog minimum, but each participant needs to blog to accrue points to win the top prizes. There will be three scoring types, as follows:

  • Introductory Blog: participant describes in detail what he/she plans on doing. Worth up to 75 points.
  • Final Summary Report Blog: participant describes in details what he/she did and proposed in the introductory blog. What experiments were conducted. What results were obtained? What were the takeaways, that is, what was learned. Worth up to 200 points.
  • Intermediary Supporting Blogs: these are blogs that support their experimenting program but are not the Introductory or Final Summary Blogs. Each weekly blog is worth up to 10 points.

The overall point totals will be posted on the table on a weekly basis.

The typical program timeline will be:

  • The up-to-75-point Introductory blog: Needs to be posted in Week 1 and Week 2
  • The up-to-10-point Intermediary Supporting Blogs: Need to be posted each week in Weeks 3 through 9.
  • The up-to-200-point Final Summary Blog: Needs to be posted in week 10.

Element14 will assign points for the Introductory and Supporting Blogs.

The judges will assign points for the Final Summary blog.

The members with the most points at the end of the program period will win.

What do you think?

Randall
--element14 Team; Experimenting Challenge Program Manager

Parents

  • The new scoring system sounds good. I have several minor notes:

    • Blogs count minimum you need at least for finisher prizes. You can express it using score but I think it will still define minimum requirement on blog count. I recommend do not removing minimum of two blog posts. If you remove this limit, then you can face situation that someone post the only one (summary) blog in last week. In extreme case he/she can win with 200 points, but I think in this case single blog entry was not very useful contribution to the competition.
    • Score must distinguish contestants. I recommend defining rule that in every category two contestants never receive exactly the same number of points. For example look 7 Ways to Leave Your Spartan-6: Contestant Standings. Look at First Project column. All has 0 points (did not complete project) or 50 points (completed project). If you take two projects from this table which received 50 points, then it means that both projects/blogs are totally equal in quality but in reality this is never true. There is always difference (at least very small difference) and scoring system should highlight it. You can for example give one project 45 and second project 44 points. Both contestant get very similar points for very similar quality of projects/blog but there is an order. The contestant with one point less can look at it and decide that he will post one additional intermediary blog for beating the originally slightly better contestant. Idea of this paragraph is that every time there must be the order. Giving all contestant 75 points for introductory blog does not make sense because all (except contestant who post nothing) will get them and there is no need to make any complex strategy in this case.
    • Mentioned system rate blogs and project quality but do not count other contestant activities. Very frequently happen that someone ask on forum, someone answer question and help the other contestant to continue its journey or something like it. I think this should be graded and this kind of cooperation should be awarded by points, I think.
    • In addition to previous point: forum activity should count. Both question and answers should count. In 7 ways to leave spartan there were category for this but in promoted system it is not. Contestant sometimes get into troubles and asking on forum is very effective way because at the same time there is another 9 contestant who have exactly the same hardware in the hands and together, they can resolve even very complicated issues.
    • Carefully take in account amount of work needed to do on element14 side every week. We frequently see delays in selecting challengers, waiting for judges selection, postponing shipping dates and so on. I recommend changing “score updated on weekly basis” to “score updated every second week” and also remember to dedicate time slot every second Friday (or other day) in your calendar. If you will not have time for evaluating blogs every week, than space for complex strategies on contestant side decreases significantly. For allowing making strategies evaluation timing should be strictly followed.
  • in reply to misaz

    I like the idea that every score should be different, but that adds a lot of judging work. Maybe someone can come up with an easy way to achieve accurate differentiated scores without a huge amount of judging work.

  • in reply to dougw

    I agree that it is lot of work and maybe this rule can be relaxed slightly. I mainly highlighted the situation when all received full points which eficiently decraded score to true/false ranking.

Reply Children
No Data