Actions
Forum Thread Details
  • Replies 34 replies
  • Subscribers 12 subscribers
  • Views 5053 views
  • Users 0 members are here
Related

RE: Seeking Feedback on New Scoring System for Experimenting Challenges

rscasny

Note: this discussion covers only the Experimenting Design Challenge Competitions

Greetings to All,

I would like to introduce a blog scoring system in addition to the judges I use to determine the top prize winners of the experimenting with components challenges (e.g, capacitors, inductors, vibration sensors, etc.).

I would like to get feedback on this proposal by members of element14.

Before I detail the change, let me explain why I am suggesting it? There are a couple of reasons:

  • Transparency: The blog will be scored on a weekly basis and the score posted in a table similar to what I used in another one of my programs. Here is an example of the table I had used in that program. Click here to see.
  • Strategy: With participants seeing the point totals of all participants, they can plan a strategy for achieving a point total to win.
  • Ensuring Participation Throughout the Program Period: Sometimes participants post their blogs at the end of the 8 to 10 week period, not giving me the opportunity to promote the blogs through the program period. In addition, giving participants a reason to post supporting blogs throughout the active period allows them to pick up points along the way.
  • Making Judging More Convenient: Our judges will receive the final summary blog to judge, which I think will make it easier for them to manage. Since the final summary blog can earn the most points, the judges' decision will have the most weight in the final selections.

What is the Judging/Scoring System In Place Now?

There is a two-blog minimum required to successfully finish the competition. There is no scoring system in place now. Currently, at the conclusion of the competition (usually 8 to 10 weeks), I forward all the blogs of each participant to the judges. The judges rank the blogs and select the winners of the top prizes. At the end of the judging, I notify the winners and write a winner's announcement.

What is the Judging/Scoring System That I Would Like to Introduce?

There will be no blog minimum, but each participant needs to blog to accrue points to win the top prizes. There will be three scoring types, as follows:

  • Introductory Blog: participant describes in detail what he/she plans on doing. Worth up to 75 points.
  • Final Summary Report Blog: participant describes in details what he/she did and proposed in the introductory blog. What experiments were conducted. What results were obtained? What were the takeaways, that is, what was learned. Worth up to 200 points.
  • Intermediary Supporting Blogs: these are blogs that support their experimenting program but are not the Introductory or Final Summary Blogs. Each weekly blog is worth up to 10 points.

The overall point totals will be posted on the table on a weekly basis.

The typical program timeline will be:

  • The up-to-75-point Introductory blog: Needs to be posted in Week 1 and Week 2
  • The up-to-10-point Intermediary Supporting Blogs: Need to be posted each week in Weeks 3 through 9.
  • The up-to-200-point Final Summary Blog: Needs to be posted in week 10.

Element14 will assign points for the Introductory and Supporting Blogs.

The judges will assign points for the Final Summary blog.

The members with the most points at the end of the program period will win.

What do you think?

Randall
--element14 Team; Experimenting Challenge Program Manager

  • in reply to dougw

    Yes, there is more :"administrative overhead" with the table. As I told shabaz, 7 ways had 25 participants, while my upcoming Experimenting Challenge will have 7, due primarily due to budgeting limitations. So, it's less work. I'd also add the updating of the table into our project management tool. You ask some really great questions. I felt that the 7 Ways approach got more participation and it was a checklist for the participants. Would it alter the results? I can't be certain, but for one or two of them, it could have. I've really soul-searched about how much a burden I should be placing on the judges. I know some of them have full time jobs. I have been concerned about sending the judges 30 or 40 individual blogs to judge and then requiring them to rank the participants. I want the final decision to rest with the judges,yet at the same time foster participation throughout the program's timeline. 

  • in reply to misaz

    You mention a lot of things. Thanks for taking the time to join this conversation. I'll address two of them now. 

    Supporting blogs should not get all the same points. Yes, for the 7 ways program, most of the activities but not all of them I awarded with the same points. But this was different from Experimenting. If I asked them to write the differences between Spartan 6 and Spartan 7, the blogs would be similar since they were based on the same source info from the sponsor. Hence, similar point awards. For experimenting challenge, the supporting blogs would inherently be different. But to answer your question, yes they could be score differently. I am thinking of a standard such as the following:

    If the supporting blog is value adding (better explains something, helps some understand better, is very relevant, etc.), it merits the full ten points

    If the supporting blog is only an unboxing that adds images and repeats into in data sheets, it merits 5 points.

    If the supporting blog is a blog just for the sake of writing a blog (ie., telling me he received the kit and phots of the kit and nothing more, it merits only 2 points. 

    Your other suggestions concerned quantifying forum assistance. Very hard to track that in not only a democratic manner but also would not take much time. I would have to do that. I would not burden the judges with that task. 

    Perhaps I could create a poll at the end of the program asking the community which participant do you feel has provided the most support? The top vote get would receive, say, a 20-point bonus. Just thinking out loud. 

  • in reply to misaz

    This needs clarification. Let's say for example Week 3 starts on Jan 22 and Week 4 starts on Jan 29th and week 5 starts on Feb 5th. In other words, three weeks. Each week you will get awarded a maximum of ten points for supporting blogs. Hence, for 3 weeks, that's 30 points. If people understand that, they are not going to write 50 blogs nor expect 500 points. Thus, I need to work on clarifying these instructions. 

  • in reply to dougw

    This is my first thought on this system:

    --If the supporting blog is value adding (better explains something, helps some understand better, is very relevant, etc.), it merits the full ten points

    --If the supporting blog is only an unboxing that adds images and repeats info in data sheets, it merits 5 points.

    --If the supporting blog is a blog just for the sake of writing a blog (ie., telling me he received the kit and photos of the kit and nothing more, it merits only 2 points

  • in reply to Andrew J

    The introductory blog (worth 75 points) should spell out the theory of the experiment

  • in reply to dougw

    From my point of view these are very important for the introductory and the final summary blogs.  For the supporting blogs, they can be important. But a supporting blog is only worth 10 points, so I was looking for a simplified point-award structure. I got to think about this some more. Thanks for the input.

  • in reply to rscasny

    I'd suggest that the participants work and posts should weight an order of magnitude higher than the downstream forum activity. To keep the contests focusing on engineering versus popularity.

  • If you look at the point awards, the introductory and final blogs get the most points, thus, participants are compelled into completing those successfully and spending the most time with them. Supporting blogs are not required; they are optional to my mind. Because one could actually win if they score the full 275 points with the first and last blogs. 

  • May I ask one more question to everyone?

    We normally have a Grand Prize, a Runner Up prize and a complimentary finisher's prize for those people who wrote a minimum of two blogs.

    I am thinking about dropping the finishers prizes and replacing it with a Third Prize of about the same value as all the finishers' prizes.

    What do you feel about me doing this change?

  • in reply to rscasny

    I think finisher prize for all who spend some time with competition is better than one more valuable prize because other finishers can feel like it was waste of time and do not apply anymore. If you receive something, even it is very cheap device, then it motivates you to try it again in next competition.