Free 186 More Best Homemade Tools eBook:  
Get 2,000+ tool plans, full site access, and more.

User Tag List

Results 1 to 7 of 7

Thread: Homemade Tool of the Week voting process expansion

  1. #1
    Jon
    Jon is online now Jon has agreed the Seller's Terms of Service
    Administrator
    Supporting Member
    Jon's Avatar
    Join Date
    Jan 2012
    Location
    Colorado, USA
    Posts
    25,451
    Thanks
    7,928
    Thanked 38,621 Times in 11,284 Posts

    Homemade Tool of the Week voting process expansion

    I'd like to start a discussion about expanding voting rights to allow members to assist in choosing the Homemade Tool of the Week winners, and the Best Documented Build winners.

    Typically, Ken and I have a weekly discussion as to who should win the Homemade Tool of the Week award, with a similar monthly discussion for the Best Documented Build winner. We discuss it back and forth, and decide on a winner.

    Initially, the pool of potential winners was small, and the choice was fairly easy. Recently, we've started with a list of a bunch of potential winners, tried to hone it down, and found great difficulty in choosing a single winner. With so many contenders, it makes sense to expand the system beyond two judges.

    We could have a weekly discussion with a poll to select the winner (monthly for Best Documented Build), and we could limit participation in that poll by certain factors (number of posts, time on forum, winners of previous contests, etc.). Votes could be private by default, but if the voter wanted, he could make a post in the thread mentioning his vote, or just generally discussing the contenders without revealing his vote.


    ADVANTAGES OF CURRENT SYSTEM
    -Likelihood of collusion is minimal. Since Ken and I don't compete, we can't, either intentionally or unknowingly, trade votes or participate in quid pro quo. Nobody is perfectly impartial, but this helps protect our impartiality.

    -Verifiability of identity. We know and trust each other, and our identities are easily verified. If our voting pool was comprised of forum regulars, I think it would be very reliable. However, if we opened up the poll to the unknown public, it could turn into a contest that rewards the person most capable of performing online vote manipulation.

    -No public drama. The judges are removed from the community, and the process is somewhat opaque. Drama is time-consuming and detrimental to most forums, and we've thus far avoided it. In a forum focused on drama, the loudest and most persistent voices are often victorious, and the spotlight is removed from creation and sharing of new ideas, and placed on personal viewpoints and conflicts.


    DISADVANTAGES OF CURRENT SYSTEM
    -Subconscious cogitive bias elimination is problematic among two people. You could be highly ethical, and make every effort to produce an unbiased decision, when, in fact, such a thing is not possible. If we spread voting rights across more people, we can't eliminate cognitive bias, but we can balance it. Voter A might like heavy equipment, and be more likely to vote for it. Voter B might prefer simple hand tools with high-quality craftsmanship. Voter C might prefer metalworking tools with CAD files attached. Working in concert, those biases can even each other out and produce a more fair overall vote.

    -No two judges, between them, can have the vast skillset necessary to properly judge such a wide array of entries. The most fair decision comes from a board of people with deep and varied expertise, and we have that among our members.

    -We've come up against some weeks where two or three entrants could easily have won. Although we're making a conscious decision, sometimes it feels like we could have just flipped a coin.


    OPTIONS
    -Keep things how they are. It's not perfect, but the system has been widely respected and successful thus far. We can always change it later if we want.

    -Expand the voting pool to include all previous Tool of the Week and Best Documented Build winners.

    -Expand the voting pool to include all forum members with more than a certain number of posts, and/or a certain period of time being registered on the forum.



    What are your ideas? Do you favor one of the above options? Or do you have other ideas or constructive criticism?

    186 More Best Homemade Tools eBook

  2. The Following 2 Users Say Thank You to Jon For This Useful Post:

    Paul Jones (Feb 12, 2016), scorch (Feb 12, 2016)

  3. #2
    Supporting Member machiningfool's Avatar
    Join Date
    Jul 2015
    Location
    Seffner Florida
    Posts
    265
    Thanks
    0
    Thanked 271 Times in 146 Posts

    machiningfool's Tools
    I think you should keep it as it is. I am speaking for myself, but I think the way it is would be more appreciated, coming from the bosses and less confusing for you. Bob.

    2000 Tool Plans

  4. #3
    Supporting Member mklotz's Avatar
    Join Date
    Aug 2015
    Location
    LA, CA, USA
    Posts
    3,434
    Thanks
    357
    Thanked 6,380 Times in 2,117 Posts

    mklotz's Tools
    If you have a jury including or composed of frequent submitters of tool ideas, they will often be voting on contestants that include their own submissions. That has the potential to lead to all sorts of problems.

    There will be weeks without good contestants and weeks with several, difficult to pick among, contestants. Maybe a longer time period is an answer. Have the moderators, who are currently doing a good job of selecting, pick tools they want included in the monthly "tools of the month" awards. The number picked wouldn't be fixed but would correspond roughly with one tool picked for each week in the month; say somewhere between 2 and 6 tools.
    ---
    Regards, Marv

    Home Shop Freeware
    https://www.myvirtualnetwork.com/mklotz

  5. #4
    Supporting Member astroracer's Avatar
    Join Date
    May 2012
    Location
    Byron, Michigan
    Posts
    310
    Thanks
    20
    Thanked 558 Times in 173 Posts

    astroracer's Tools
    I agree with the opinions to leave things be. Throwing more "judges" at the problem just muddies the waters and makes the process more drawn out. "Maybe" pick a few contributors to use as sounding boards or tie breakers when/if you guys get stuck but, other then that I think you have been doing a good job.
    Did you ever think the site would get this big when you started it?
    Mark

  6. #5
    Supporting Member C-Bag's Avatar
    Join Date
    Aug 2015
    Location
    California, central coast
    Posts
    720
    Thanks
    689
    Thanked 876 Times in 471 Posts

    C-Bag's Tools
    Quote Originally Posted by astroracer View Post
    "Maybe" pick a few contributors to use as sounding boards or tie breakers when/if you guys get stuck but, other then that I think you have been doing a good job.
    Mark
    I agree and was thinking this exactly. From past experience the more complicated you make things the more likely it will crash. You guys have found a balance that works. It ain't broke, so be careful about fixing.

    Even though I get an pang of jealousy when I see all the guys who have all the awards under their avatars I also don't want to fall into that competitive trap that will only lead resentments on my part. My feeble efforts are my babies so I'm torn between opening them up to criticism. But here at Homemadetools is a civil and supportive forum for this form of creative outlet. I'd hate to see it get messed up. Just my 2c.

  7. #6
    Supporting Member Paul Jones's Avatar
    Join Date
    May 2014
    Location
    Del Mar, California
    Posts
    1,231
    Thanks
    5,810
    Thanked 1,440 Times in 655 Posts

    Paul Jones's Tools
    Jon, let's keep the selection process simple and I think keeping the voting the way it is today is okay. Thanks for asking, Paul

  8. #7
    Jon
    Jon is online now Jon has agreed the Seller's Terms of Service
    Administrator
    Supporting Member
    Jon's Avatar
    Join Date
    Jan 2012
    Location
    Colorado, USA
    Posts
    25,451
    Thanks
    7,928
    Thanked 38,621 Times in 11,284 Posts
    Lots of good feedback here. If the most popular opinion is to keep things as they are (at least for now), then that's exactly what we should do. We've managed to collectively cultivate this forum as a refuge from web drama, even with multiple contests, and that environment has proven to be relaxing, productive, and merit-based.

    There are some good ideas here for involving members in the judging process, for either the Tool of the Week contest or others, and that's something to keep in mind, now and in the future.



    186 More Best Homemade Tools eBook

  9. The Following User Says Thank You to Jon For This Useful Post:

    Paul Jones (Feb 16, 2016)

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •