I don't know if I am just in the wrong view or not but as far as I know the judging process has always been the following:

tariq.ahmad announces that the judging is open. All those users that are available to judge, just contact him by message so he has clear who are the judges and will wait for all before deciding the winners.

Then, the judging process is a private message to Tariq expressing just the note for the grand prize winner project (if any) and the three winners. Not a report of your personal impressions on the project.

When the judging process closes, Tariq announces the winners. That's all.

Frankly, I never saw a public report over all the  presented projects, for this kind of considerations there are the comments to the posts, maybe the list of the participant projects of directly in the project author's blog post(s)

In my opinion (this is my personal view) this is not a so correct judging process.

Enrico

Hi balearicdynamics ,

I will set up a list of projects that are currently submitted for the contest about a week in advance.   After the contest closes I will ask for volunteers to participate for judging.   I will then tabulate the votes.   The way that this is being done currently is a Grand Prize vote is given 2 points and a First Place vote is given 1 point.   To make sure that each vote is valid we ask that each judge adds 1-2 lines explaining the reasoning behind their decision.

As far as anything added late, this hasn't been a problem and any time there were adjustments made after the deadline submission most members have added these in the form of a separate blog. This has never been a real issue but as these themes have gotten a little tougher,  I could see the need to look at adjustments to ensure fairness.  This was a tough competition so there were a lot of entries at the last minute.   Would welcome any suggestions on how to improve the judging process if you have any.   As much as possible I've tried to stay out of the judging and let the votes from the members that volunteer to judge be the sole factor.  

Hello Tariq,

I think that the judging process works fine. IMHO also this phase of the challenging episode after episode become always more attractive for the users. I think there is nothing secret in the judging process but I think that judging is better if kept not public to leave the single evaluation clean and polite without the involuntary influence of the other judging. Then, personally, I love to send you my judging, as well as the other participating members and waiting for the surprise of the final result.

A possible improvement/change – but I am not sure it is is worth – maybe revealing the judges of every challenge when the winners are announced. Does this idea make sense?

Enrico

Hi Tariq,

I believe the judges should send private messages to you rather than post here, but other than that have no problems with the judging process as it stands.  I think it would be helpful to clarify what the rules are on resubmitting previous entries, including variations on previous entries whether they were on element14 or another site.  I don't think it is a problem as long as the person making the entry notes it as such so the judges are aware.  Benefit should be given to new and original entries.

Frank

I understand.  The reason for keeping the feedback anonymous was that when we began the program,  some members had reservations about revealing their feedback and preferred to judge as long as their vote was anonymous.  At the same time,  feedback is important with anything,  if for any other reason to validate the effort put in the project and encourage others to learn about what impressed other members with the project.  From what I was told from other members, the idea of community judges is something that did not work very well on the community before Project14 so we made effort to learn from others experiences with that,  and tried to make sure people could give honest opinion on projects without having it become too personal.  One thought I've had before, but abandoned, was having a form where you can choose to make your vote anonymous or not.   This was something that I did not follow through on as what we were doing seemed to working.    Also,  its very hard to get these winners announcements out in a timely manner whenever you add another layer to the process.   One thing I am planning on doing is adding more transparency via the "weekly content journal" update series that just got started.  For instance, when badges are awarded, its a manual process where I send a list of names to the community manager.   What I am planning to do this week is do a badge announcement to let people know who won badges this round.   If you recall, when we first started doing winners announcement, we threw some badge announcements at the end.  It became very hard to do this with the winners announcements so we stopped but with the "weekly content journal" this might be worth revisiting.  You always worry about things getting stale and will be looking to freshen things up and look for ways we can add more transparency to this process and these could lead to more discussions on how to freshen things up with this program.

Hi fmilburn ,

This is a very good suggestion.   One of the things we would like to do is add some more documentation such as "what is the ideal project post" and clarify the rules for judging something that was previously submitted.   The guidelines we have given its ok as long as you are honest and mention that this is something that you've been working on.   Members have told me that the program is great for taking a project that you put on the shelf and encouraging you to finish it.   I also want to give more points, and the judges have done a great job thus far, to projects that were started and then completed during the competition.  The site in general could use a refresh and some of the documentation was done way back when the program got started and we didn't really know where this was going.  I think now that we've done a lot of these,  it will make sense to roll up the sleeves, take a look at what we have as documentation, and clarify what qualifies as a submission.   Ultimately,  we want to bring in new people to participate, and keeping the rules simple but working on making them more clear would be the way to approach it.

Hi fmilburn ,

This is a very good suggestion.   One of the things we would like to do is add some more documentation such as "what is the ideal project post" and clarify the rules for judging something that was previously submitted.   The guidelines we have given its ok as long as you are honest and mention that this is something that you've been working on.   Members have told me that the program is great for taking a project that you put on the shelf and encouraging you to finish it.   I also want to give more points, and the judges have done a great job thus far, to projects that were started and then completed during the competition.  The site in general could use a refresh and some of the documentation was done way back when the program got started and we didn't really know where this was going.  I think now that we've done a lot of these,  it will make sense to roll up the sleeves, take a look at what we have as documentation, and clarify what qualifies as a submission.   Ultimately,  we want to bring in new people to participate, and keeping the rules simple but working on making them more clear would be the way to approach it.

Hi Tariq,

Sorry for the misunderstanding, this was not my intension, I was offering my own perspective and did not realise it was a closed discussion (Note for future self).

I was so amazed by all the work done by the contestants that I rushed in not thinking of the consequences. I hope that my views were constructive and helped the contestants to push harder and help them in future projects..

Best regards,

Mike Clapham.

Hey Mike, don't worry, no one is complaining anyway and I think that we all think to your intervention as with a constructive proposal. Good you got us the opportunity for some better clarification and discussing on this important topic.

Enrico

Hi mike-0rc ,

Like balearicdynamics said,  your feedback was constructive and is appreciated by everyone.   This was a challenging competition.   With this program we are always trying to encourage participation and there have been plenty of times that members have continued working on a project long after it has been closed.   Also, with the Programmable Logic there were a few instances of members dusting off projects that they have been working on and the competition has encouraged them to continue working and learning.   At the end of the day,  we're all here to learn and the members are appreciative of feedback and the input coming from the collective mind of all the community members. 

Tariq

tariq.ahmad  wrote:

 

  Also, with the Programmable Logic  there were a few instances of members dusting off projects that they have been working on and the competition has encouraged them to continue working and learning.  

 

Tariq

I have mixed feelings regarding this.  However, I would hate to discourage anyone from participating and personally am grateful to have an opportunity to be part of any of these events. 

Considering the difficult task you have to juggle these competitions, along with the other tasks you are involved with with the site and behind the scenes, I am amazed that you are able to keep these competitions flowing as you do.  Keep up the great work.

You're not alone.   Not sure if you saw but  dougw did a poll and there seemed to be a strong consensus that this was alright as long as people were upfront and/or made an improvement:

Re-Entering Old Projects In Project14 Contests

End of the day,  as long as people are upfront the volunteer community judges will have the final say and when a project was started will be part of what the judges considered.  There are so many of these competitions and the learning that comes from sharing projects makes even these older projects worthy of inclusion especially if there is a way to improve/modify as some have done with some of their design challenge projects.

Thank you,   jomoenginer for the kind words.   I'm always amazed by the work that you produce.   (as a fellow fan of robots!)    This program has been really fun and its truly been one of the coolest things I have been fortunate to be a part of.   Witnessing these projects and all the smart creativity involved has been an incredible experience.

Although I wasn't a judge on this competition, I too just automatically add it to the consideration, if a project is older than the date the competition started, so that people who had less time to work on the project, are not disadvantaged from my perspective (I prefer seeing newer projects though, or something significantly changed on an old project - others may rate that differently). Sometimes it's really hard judging, in which case I just speak up that I cannot pick (say) 3, and can only pick 4 or 5 because that many are deserving.

Personally I like seeing well-documented projects (including an overview that I can understand, and source code and schematics)... but will also mention to the author if they can add a better description or schematics etc.. since some people unintentionally forget to add such things. I remember the xmas project14 being particularly hard to judge! Lots of fun projects..

I agree with you Shabaz, but by the other side – at least by my personal point of view – if I participate in a challenge I always present a project that maybe I started it before but sure it is the first time the project is published. Then I can relaunch it everywhere if I have time like Hackster, Instructables, and others.

Enrico