Some Changes in How Blogs are Scored in Experimenting Challenges

SCORING SYSTEM UPDATE

I appreciate everyone who has offered their ideas. I like the point system and will continue to use it. While no scoring system is perfect, I think that a sweet spot can be found where it has a consensus for its fairness.

Before I posted this update to my discussion, I went back and read through every blog in Experimenting with Supercapacitors. I saw the blogs covered the following things:

• An introduction, which often was a combination of an unboxing and kit overview
• Unboxing
• Kit Overview 
• Deep dive into the kit and components
• Background technical information related to the components
• A report on a specific experiment that answered a question: How does high temperature affect a supercapacitor?
• A blog that tests a component through an application or prototype
• A tutorial--the blog is meant to teach the reader something

My feeling after reading all the blogs was that this is a component experimenting challenge, so blogs with a report on a experiment should score higher. A tutorial should also score high because it should be teaching something specific. An experiment using the component in a device or application should also score high. The others would have less points.

BUT ONCE I READ THROUGH ALL YOUR IDEAS, as a group, you want the scoring to reflect the quality as well.

I was attracted to Shabaz's suggestion (he has been a judge for a number of my experimenting challenges). I modified the ASSIGNED points to reflect quality.

• A blog that contains a decent unboxing or detailed photographs will get up to 20 points.
• A blog that contains a deep-dive into the components will get up to 20 points.
• A blog that contain a fair amount of information towards an actual experiment will get up to 20 points.
• A blog that significantly helps newcomers to work with the technology will get up to 20 points.
• A blog that contains a good quality video will get up to 20 points.
• A blog that uses components in an application or prototype to test or experiment with them will get up to 20 points.

The actual scoring will be judged based on quality, with a 20-point perfect score based on a very high level of quality and thoroughness. A blog with more than one of the above characteristics would get an additive score, based on quality.

By dropping the Introduction blog and forum participation, gives more time to judge the 5 blogs.

The final project blog will stay the same, up to 300 points.

I'm leaning towards this scoring system. I'll let it "age" for a day or so before finalizing it. Your feedback is always appreciated.

Randall

SCORING SYSTEM UPDATE

I appreciate everyone who has offered their ideas. I like the point system and will continue to use it. While no scoring system is perfect, I think that a sweet spot can be found where it has a consensus for its fairness.

Before I posted this update to my discussion, I went back and read through every blog in Experimenting with Supercapacitors. I saw the blogs covered the following things:

• An introduction, which often was a combination of an unboxing and kit overview
• Unboxing
• Kit Overview 
• Deep dive into the kit and components
• Background technical information related to the components
• A report on a specific experiment that answered a question: How does high temperature affect a supercapacitor?
• A blog that tests a component through an application or prototype
• A tutorial--the blog is meant to teach the reader something

My feeling after reading all the blogs was that this is a component experimenting challenge, so blogs with a report on a experiment should score higher. A tutorial should also score high because it should be teaching something specific. An experiment using the component in a device or application should also score high. The others would have less points.

BUT ONCE I READ THROUGH ALL YOUR IDEAS, as a group, you want the scoring to reflect the quality as well.

I was attracted to Shabaz's suggestion (he has been a judge for a number of my experimenting challenges). I modified the ASSIGNED points to reflect quality.

• A blog that contains a decent unboxing or detailed photographs will get up to 20 points.
• A blog that contains a deep-dive into the components will get up to 20 points.
• A blog that contain a fair amount of information towards an actual experiment will get up to 20 points.
• A blog that significantly helps newcomers to work with the technology will get up to 20 points.
• A blog that contains a good quality video will get up to 20 points.
• A blog that uses components in an application or prototype to test or experiment with them will get up to 20 points.

The actual scoring will be judged based on quality, with a 20-point perfect score based on a very high level of quality and thoroughness. A blog with more than one of the above characteristics would get an additive score, based on quality.

By dropping the Introduction blog and forum participation, gives more time to judge the 5 blogs.

The final project blog will stay the same, up to 300 points.

I'm leaning towards this scoring system. I'll let it "age" for a day or so before finalizing it. Your feedback is always appreciated.

Randall

I think this will definitely give contestants a lot of freedom, and pick up deserved points for good photos/experimenting/informative blogs/training content or video etc..

Also, an extra benefit, it might be the push for those who were previously writing minimal content for the five blogs (not always for negative reasons, perhaps they just didn't realize that it was a useful thing to do for themselves). Because, if they can get through a certain amount of experimenting etc early on, they may naturally get the drive once they start blogging, and then hopefully the rest of the challenge will become easier for them once they start seeing their own progress.

Starting from the point that personally, I don't care much about scoring systems. I believe it's important to have a fair scoring system for those solely competing for prizes. It's a competition, and it's logical that some people compete solely to win.

Shabaz's proposal seems fair to me, and I think it will also encourage the creation of higher-quality intermediate content for those who, in an engineering sense, choose to publish the worst acceptable content. Evaluating the worst quality content must still be of good quality.

Assessing all intermediate blogs based on their quality would take a lot of time for a single person, who may not necessarily be knowledgeable in the field.

What if we established a peer-to-peer review system where challengers assess the quality of the intermediate blogs based on a limited set of criteria? I believe this would boost the challengers' participation and engagement. Participation in peer-to-peer reviews would be necessary to compete for major prizes. This way, judges would only need to review the final projects.

Personally, I don't mind this - I think the added freedom to allocate points where necessary is good, as it's less prescriptive and constraining.

But at the same point in time, I don't want to see the same thing happen as before, where full marks are awarded to nearly everything, thus the score "saturates" at 100%. As long as your 20 points are sensibly allocated in such a way that an excellent blog scores 20, and a satisfactory blog scores less - something like 10-15, then it makes sense. It has to be a scale ...

If everything is 20 points or 0 points ... then I feel that we're back to square one ...

- Gough

Agreed.  I also refer back to an earlier point: entries shouldn’t even make it to scoring if it’s obvious the effort and quality is poor (I.e. lip service is being paid).  I don’t think Randall should worry about disqualifying entries.

I think that was the problem last time. They were scored sort of in a pass fail approach. If you did it, you got all points. I realize that each blog needs individual scoring based on its inherent value. 

Randall