Actions
Forum Thread Details
  • Replies 20 replies
  • Subscribers 206 subscribers
  • Views 2734 views
  • Users 0 members are here
Related

What's Your Ideal Way of Handling Judging and Mentoring?

tariq.ahmad

One of the hardest things to be asked in the community is to be asked to judge a competition.  For any given competition there might be five or six really impressive entries that are each worthy winners. You look really closely at each entry and narrow down your list as much as possible and go with the entry you believe is the strongest. Once a winner has been decided, unless there is one that clearly blows away everything else you see, you go with the candidate you believe embodies the spirit of the competition and hope you make the right decision. Often times, you second guess yourself but reassure yourself that you made a conscientious decision, and that any of the candidates you had to choose from, would be worthy winners.  At the same time, you want to reach out to the people that made your decision hard, as its their ideas or projects that made your decision hard. Having multiple prizes each month makes things easier but if this program is as successful as we hope than picking winners is going to be a very difficult job!

 

In an ideal world, its the community members, who the community is for, who would decide.  In reality, there are a lot of practical and logistical things you have to consider if you were to go that route. Like I mentioned, you don't want to undermine the integrity of the program by having the decision based on factors other than the work itself.  At the end of the day, whatever route we go with judging would have to respect the integrity of the program.

 

Several of you expressed concern, precisely over this. Project14 hopes to be a community member-centric program that's not about us but about you.  To ensure the integrity of the program the judging will be in part a responsibility on our end.  However, a Jury of Peers and Mentoring is a core feature of the program that makes it unique.

 

In the comments below tell us what your ideal way handling Judging and Mentoring for our new program!

Parents

  • Tariq, In my opinion the unicity of Judging + mentoring is really great and it is also the way for a honest judging. Most of the times I was part of a judging team the authors was unknown. In this case if a serious judge also seriously mentor it has the opportunity to collect all the aspect of the challenge process for every challenger not only supporting him but also giving an evaluation of his job in the most correct way.

     

    Enrico

  • in reply to balearicdynamics

    My worry is that a panel of judges with community members would make this feel like a job if the same community members are volunteering.  Then it makes sense to rotate a different panel every month. What if you reach a situation where you're stuck finding volunteers in a competition that is always happening? Could advertising the judges create a bad blood in the community if people think they deserved to win but didn't?  At the same time I do like the idea of a serious judge + mentor but it seems like the best way to do this would be having their names be anonymous. I guess one route is to only allow judging by top members but then we don't want people to feel excluded as one of the core tenants of the program is to make it as inclusive for everyone.  Want to keep the focus on fun and the learning and not the prize you get for winning.   These are good ideas and I see what you thinking but tying this too closely to people's names may add a personal element which I worry would change the good mojo we have running in the community.

  • in reply to tariq.ahmad

    Tariq,

    as far as I know, including the previous Element14 challenges, judges are never anonymous and in this case it is even more difficult as the judges are also - at least someone of them - mentor. Also keeping the information public I think that it is not a shame to involve only the TM for first then making some reports and discussions this mechanism will set by itself.

     

    About involving community members the judge role is just an invitation, IMHO, like the users to participate to a challenge. A judge is a volunteer, involved in the challenge as well as the challenger. Do you know the game Eleusis, illustrated around the mid of '60s (past century image ) by Martin Gardner on Scientific American? It is just our case.

    To say this in few words:

    • judges should mentor and the judges panel decide the winner and the challengers ranking based on the duality of the project results, and project conduction
    • Challengers will report their feel with judges and the judges panel will become a circulating group of volunteers that have their own ranking (and can be excluded with a too low value) based on the challengers reporting.
    • The judges are a fixed number every challenge but the subjects rotates (not necessarily all should change in two different challenges) depending on the number of volunteers judges respect the nudges panel (fixed number of persons)

    I think that with these rules what is granted is a self-balancing system.

     

    Enrico

Reply
  • in reply to tariq.ahmad

    Tariq,

    as far as I know, including the previous Element14 challenges, judges are never anonymous and in this case it is even more difficult as the judges are also - at least someone of them - mentor. Also keeping the information public I think that it is not a shame to involve only the TM for first then making some reports and discussions this mechanism will set by itself.

     

    About involving community members the judge role is just an invitation, IMHO, like the users to participate to a challenge. A judge is a volunteer, involved in the challenge as well as the challenger. Do you know the game Eleusis, illustrated around the mid of '60s (past century image ) by Martin Gardner on Scientific American? It is just our case.

    To say this in few words:

    • judges should mentor and the judges panel decide the winner and the challengers ranking based on the duality of the project results, and project conduction
    • Challengers will report their feel with judges and the judges panel will become a circulating group of volunteers that have their own ranking (and can be excluded with a too low value) based on the challengers reporting.
    • The judges are a fixed number every challenge but the subjects rotates (not necessarily all should change in two different challenges) depending on the number of volunteers judges respect the nudges panel (fixed number of persons)

    I think that with these rules what is granted is a self-balancing system.

     

    Enrico

Children
  • in reply to balearicdynamics

    Hey ,

     

    One idea that was discussed internally was a jury of peers and a system for signing up.

     

    Don't think we're ready for that as there are still some concerns.

     

    A concern would be that this may over complicate the process and become burdensome if the unexpected happens and there isn't a lot of interest around a particular theme, or there's a time of the year where people are occupied with family, vacation, etc. like over the winter.

     

    Don't want the judging to feel like work or get too bureaucratic.

     

    An honor system with top members, at least initially, is very sensible and should work.

     

    We will be reaching out further to top members for help with the judging and those that volunteered will be contacted once projects start coming in.

  • in reply to tariq.ahmad

    Thank you for your details Tariq. Despite the suggested rules I mentioned (maybe the list makes things similar to a sort of "must") the idea around the Eleusis games was just to create a balanced system between judges and challengers. In a free and friendly way. I perfectly agree that the entire process by both sides (judging and challenging) should never feel like a sort of job (also a volunteering job) but should feel mostly a nice activity, a cooperation oriented to knowledge exchange.

     

    I have considered TMs as the starting point, then it is needed to assume the risk of the first months to experiment and calibrate the best methodology.