Originally Posted by MississippiRed
There are several comments/questions that come to my mind regarding this thread.
1. Social scientists have used training programs to establish inter-rater reliability among data collectors for years. If baseball is serious about these stats, they should do something similar. It really isn't that hard to find some good examples (not the obvious ones, but the ones people might initially disagree on) and reach group concensus based on expert analysis. Of course, getting experts to agree on the criteria may be difficult, but getting the raters to agree can be done fairly simply.
Yes. I do this all the time.