Had a lengthy Horse Armor meeting the other day. We discussed the submission form on the website. We originally wanted people to rate from 0-100 with 100 being the most “aggressive” score. It was suggested to avoid negative connotations. We decided to reverse the score and change the scale to 1-10.
There are many studies on the value of using different scales in ratings. Apparently, most studies find that 5 or 7 are the best with the latter being the most accurate. However, the difference can be nominal. Accuracy of a rating can drop with more data points to choose from. You see diminishing returns after 11.
Also, the more questions there are, the less the scale tends to matter. That and the fact that with a big enough dataset, most inaccuracies in statistics often correct themselves.
Our form doesn’t need that much thought. We just need it to be 1) easy and 2) make immediate sense to the user.
We also removed a question from the form. We have two new members of the “team” that were added as consultants. We are running this all by them to make sure there isn’t anything we may be missing because one thing you don’t want to do is change the data after the fact. That could dirty up your data.
Overall, it was a very productive meeting. I didn’t do any coding. Not sure if that’s a good or a bad thing yet 😂.