Turn Off Ads?
Page 1 of 3 123 LastLast
Results 1 to 15 of 34

Thread: Olney's comments on defensive ratings......

  1. #1
    Member jimbo's Avatar
    Join Date
    May 2006
    Location
    Troy, OH
    Posts
    2,614

    Olney's comments on defensive ratings......

    I read this comment by Buster Olney on defensive ratings in his ESPN chat and thought some here might consider it interesting because I know several here put a lot of stock into them. I know some will not like his opinion and will downplay it simply because it's Olney, but I tend to agree with it.

    Bill (Hartford): "In this era of stats worship, which stat do you think is the most overused, overanalyzed, and generally overrated predictor of actual performance?"

    SportsNation Buster Olney: Bill: "A lot of the defensive stats are a joke and have an inherent flaw that has never really been addressed -- typically, these systems employ an evaluator stationed in each park to determine defensive positioning. That means there are 30 different evaluators, meaning 30 different standards (in practice), 30 different perspectives, 30 different sets of eyes. The evaluator in one town might be much more attentive than the evaluator in another city, to things like how infielders lean a step (or more) if they know a breaking ball is being thrown. And you can't tell me one evaluator can accurately track the movement of seven fielders (those besides the pitcher and the catcher). That's why, when you see something about defensive efficiency -- 'These are the balls he should've caught' -- then you should be very, very, very skeptical. It's ironic that a lot of the statistical evaluators are very skeptical of scouts' evaluations, and yet when it comes to defensive numbers, they are, in effect, basing their results on a common denominator -- one evaluator making subjective judgments."

  2. Turn Off Ads?
  3. #2
    Stat Wanker Hodiernus RedsManRick's Avatar
    Join Date
    Dec 2004
    Location
    Chicago, IL
    Posts
    15,739

    Re: Olney's comments on defensive ratings......

    I love the "stats aren't perfect, therefore they are useless" argument. I hope Buster doesn't trust the strikes and balls data he gets back. Each game uses a different umpire, with a different set of eyes, to interpret the pitches. Therefore, any inferences about a pitchers ability to throw strikes based on a human umpire's decisions cannot be judged with any objectivity.

    Yes, Buster, it's not perfect. Yes, there's subjectivity involved. But what is the alternative? I don't know of any statistician, or sabermatrician, who suggests that you should ignore the stating reports, or that scouts don't provide valuable insight. But how is this any different than the scout himself, who has his own set of biases and limitations?
    Games are won on run differential -- scoring more than your opponent. Runs are runs, scored or prevented they all count the same. Worry about scoring more and allowing fewer, not which positions contribute to which side of the equation or how "consistent" you are at your current level of performance.

  4. #3
    He has the Evil Eye! flyer85's Avatar
    Join Date
    Jul 2004
    Location
    south of the border
    Posts
    23,858

    Re: Olney's comments on defensive ratings......

    which is why you don't rely on just one inidicator. However, when you've have a number of them all agreeing about a player it is probably a good idea to assume where there is smoke there is fire.
    What are you, people? On dope? - Mr Hand

  5. #4
    Posting in Dynarama M2's Avatar
    Join Date
    Sep 2000
    Location
    Boston
    Posts
    27,926

    Re: Olney's comments on defensive ratings......

    Well, there's some hyperbole in there, but I generally agree with Olney. First off, a lot of those systems use a grid, so it doesn't particularly matter where fielder X is standing. What's being measured is where the ball goes on the grid and whether the fielder reaches it.

    Mind you, Olney's hit upon the problem with his "30 different perspectives" point. Not everyone perceives the grid the same way. A fielder with a lot of range might go off-grid an almost make a play, earning a demerit because he's made a ball look like it's in his grid while another guy may not get anywhere close to a ball that a rangier guy would offer at (Juan Castro leaps to mind here), suffering no penalty when he should get it.

    Now, work has been done to whittle down that perceptual problem and teams pay big money for defensive data with the subjectivity better removed from the results. I've always thought that until you measure distance covered over time you're not going to get at the core of what constitutes defense. As far as I know, there's no public data on that.

    Though Olney should know better than to tar "defensive efficiency" as it's wholly different from anything he's talking about. Team defensive efficiency measures how well a team fields balls in play. It might be the defensive metric ever as it involves no subjectivity and it uses big sets of data.
    Baseball isn't a magic trick ... it doesn't get spoiled if you figure out how it works. - gonelong

    I'm witchcrafting everybody.

  6. #5
    Member Cedric's Avatar
    Join Date
    Aug 2002
    Location
    Monroe
    Posts
    6,343

    Re: Olney's comments on defensive ratings......

    Quote Originally Posted by RedsManRick View Post
    I love the "stats aren't perfect, therefore they are useless" argument. I hope Buster doesn't trust the strikes and balls data he gets back. Each game uses a different umpire, with a different set of eyes, to interpret the pitches. Therefore, any inferences about a pitchers ability to throw strikes based on a human umpire's decisions cannot be judged with any objectivity.

    Yes, Buster, it's not perfect. Yes, there's subjectivity involved. But what is the alternative? I don't know of any statistician, or sabermatrician, who suggests that you should ignore the stating reports, or that scouts don't provide valuable insight. But how is this any different than the scout himself, who has his own set of biases and limitations?
    I don't think he is saying that a scout himself is perfect. The problem with ratings and numbers is they become official and there is no going back. I think that's what his point is.
    This is the time. The real Reds organization is back.

  7. #6
    You know his story Redsland's Avatar
    Join Date
    Nov 2000
    Location
    Norfolk, VA
    Posts
    7,714

    Re: Olney's comments on defensive ratings......

    Quote Originally Posted by RedsManRick View Post
    Each game uses a different umpire, with a different set of eyes, to interpret the pitches.
    And a different official scorer.
    Quote Originally Posted by Buster Olney
    That means there are 30 different evaluators, meaning 30 different standards (in practice), 30 different perspectives, 30 different sets of eyes. The evaluator in one town might be much more attentive than the evaluator in another city.... That's why, when you see something about defensive efficiency -- 'These are the balls he should've caught' -- then you should be very, very, very skeptical.
    Hit? Error? It's all very subjective. It's a good thing widely quoted stats like "fielding percentage" aren't affected by such bias. Why, such a thing could taint the sanctity of the Gold Glove.
    Makes all the routine posts.

  8. #7
    Member
    Join Date
    Aug 2002
    Posts
    3,736

    Re: Olney's comments on defensive ratings......

    Agreed!

    I've been saying defensive stats are pretty much useless for a while now

  9. #8
    Stat Wanker Hodiernus RedsManRick's Avatar
    Join Date
    Dec 2004
    Location
    Chicago, IL
    Posts
    15,739

    Re: Olney's comments on defensive ratings......

    My point is that there's very little in the game which isn't subjective on some level. I'm just a wary of the subjectivity of stats which tell me Derek Jeter isn't a great defensive SS based on his inability to go laterally as I am the people who claim he is great based on the dozens of plays they've seen him make. If the difference between a great fielder an a poor one, using errors is 30 players per 1000, I'm guessing that a sample of plays your average commentator sees isn't a great sample.
    Games are won on run differential -- scoring more than your opponent. Runs are runs, scored or prevented they all count the same. Worry about scoring more and allowing fewer, not which positions contribute to which side of the equation or how "consistent" you are at your current level of performance.

  10. #9
    Member SteelSD's Avatar
    Join Date
    Mar 2002
    Posts
    9,276

    Re: Olney's comments on defensive ratings......

    SportsNation Buster Olney: Bill: "A lot of the defensive stats are a joke and have an inherent flaw that has never really been addressed -- typically, these systems employ an evaluator stationed in each park to determine defensive positioning. That means there are 30 different evaluators, meaning 30 different standards (in practice), 30 different perspectives, 30 different sets of eyes. The evaluator in one town might be much more attentive than the evaluator in another city, to things like how infielders lean a step (or more) if they know a breaking ball is being thrown. And you can't tell me one evaluator can accurately track the movement of seven fielders (those besides the pitcher and the catcher). That's why, when you see something about defensive efficiency -- 'These are the balls he should've caught' -- then you should be very, very, very skeptical.
    It'd be helpful to Olney if he actually understood how things like Zone Rating are tracked. If he knew, he'd actually sound coherant and could possibly produce a reasonable argument against the more objective publicly published defensive metrics.

    First, something like Zone Rating doesn't result from someone's objective opinion of "should have been caught". ZR only cares about areas of responsibility. Is there a potential issue with determining balls that fall in a player's area of responsibility? Sure. Slight. But nothing about the "movement of seven fielders" Olney notes. ZR doesn't freakin' care about movement. It only cares about areas of responsibility. Now, I'm not saying that Olney was talking about Zone Rating, mind you. Frankly, I'd be surprised if he had the first clue about it (or anything resembling objective analysis).

    The ZR metric, while not perfect, can only be dramatically skewed by the following:

    1. The person tracking the ball is a moron.
    2. The positioning of the fielder is always bad (which is either his problem or the coaching staff's problem).
    3. The player didn't catch a ball hit into his zone of responsibility.

    Sorry, but I'm going to cite Occam's Razor on that one.

    It's ironic that a lot of the statistical evaluators are very skeptical of scouts' evaluations, and yet when it comes to defensive numbers, they are, in effect, basing their results on a common denominator -- one evaluator making subjective judgments."
    Yeah, Buster? Well, the Reds just finished a season in which Ken Griffey Jr. played Center Field and in which Juan Castro and Royce Clayton were considered solid defensive options at Shortstop. If that's what relying on scouts gets ya', I'll take the alternative every seven days a week.
    "The problem with strikeouts isn't that they hurt your team, it's that they hurt your feelings..." --Rob Neyer

    "The single most important thing for a hitter is to get a good pitch to hit. A good hitter can hit a pitch that’s over the plate three times better than a great hitter with a ball in a tough spot.”
    --Ted Williams

  11. #10
    Professor of Cardinalogy
    Join Date
    Jun 2004
    Posts
    89

    Re: Olney's comments on defensive ratings......

    What's sad is that Buster Olney is a young guy. The same nonsense Bill James was fighting 20 years ago...from a twenty-something. Sigh. D.GOOCH

  12. #11
    Member Ron Madden's Avatar
    Join Date
    Nov 2003
    Posts
    5,815

    Re: Olney's comments on defensive ratings......

    I know a lot of fans who believe in statisical evaluation. Not one of them has ever claimed thier studies to be 100% fool proof. Each and everyone of them will tell us we need a mixture of both, Scouts and Sabermetric Tools.

    Most fans I know discredit all stats other than a hitters BA, HR and RBI. Or a pitchers W-L record and ERA. This is understandable, since we all grew up believing these to be the only numbers that really matter.

    Baseball like no other sport is linked to tradition.

    Tradition=the handing down of customs, practices, and beliefs that are valued by a particular culture.

    This particukar culture is made up of players, coaches, managers, scouts, gm's owners, broadcasters, writers and baseball fans.

    New ideas are seldom embraced.

  13. #12
    Will post for food BuckeyeRedleg's Avatar
    Join Date
    Oct 2001
    Location
    Dublin, OH
    Posts
    5,325

    Re: Olney's comments on defensive ratings......

    I know what would solve the subjective defensive measurement issue.

    Implant a chip into each player and track their movements by GPS. That with the help of video coverage from each game would pretty much remove any subjectivity from the equation.


  14. #13
    I hate the Cubs LoganBuck's Avatar
    Join Date
    Jul 2003
    Posts
    6,047

    Re: Olney's comments on defensive ratings......

    Quote Originally Posted by BuckeyeRedleg View Post
    I know what would solve the subjective defensive measurement issue.

    Implant a chip into each player and track their movements by GPS. That with the help of video coverage from each game would pretty much remove any subjectivity from the equation.

    Not necessarily a bad idea though! You could put a radio frequency ID chip in the glove, without affecting the play of the game. Actually a darn good idea.
    The Sox traded Bullfrog the only player they've got for Shottenhoffen. Four-eyes Shottenhoffen a utility infielder. They've got a whole team of utility infielders.

  15. #14
    Churlish Johnny Footstool's Avatar
    Join Date
    Sep 2001
    Location
    Olathe, KS
    Posts
    13,773

    Re: Olney's comments on defensive ratings......

    Quote Originally Posted by BuckeyeRedleg View Post
    I know what would solve the subjective defensive measurement issue.

    Implant a chip into each player and track their movements by GPS. That with the help of video coverage from each game would pretty much remove any subjectivity from the equation.

    GPS is only accurate up to a few feet, though. Use a laser grid instead. Just make sure the players wear eye protection to avoid getting their retinas burned.

    Buster Olney has a legitimate complaint about that particular system of evaluation. Of course, no decent team would rely on a system like that, and no decent stathead would put any more stock into it than, say, a scout's evaluation.

    There are a number of other systems that involve significantly less subjectivity -- The Fielding Bible uses a pretty good system, and their evaluators watch every play of every game. Olney should educate himself before making a misleading statement like the one he just made.

    Of course he won't, though. He's trying to make a name for himself as an "always goes against the grain" journalist.
    "I prefer books and movies where the conflict isn't of the extreme cannibal apocalypse variety I guess." Redsfaithful

  16. #15
    Member
    Join Date
    Dec 2004
    Posts
    3,072

    Re: Olney's comments on defensive ratings......

    As far as fielding goes, I'd take what a scout says 9.9 times out of 10 over some statiscal analysis.
    I hope it's never sunny in Philly again.


Turn Off Ads?

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

Board Moderators may, at their discretion and judgment, delete and/or edit any messages that violate any of the following guidelines: 1. Explicit references to alleged illegal or unlawful acts. 2. Graphic sexual descriptions. 3. Racial or ethnic slurs. 4. Use of edgy language (including masked profanity). 5. Direct personal attacks, flames, fights, trolling, baiting, name-calling, general nuisance, excessive player criticism or anything along those lines. 6. Posting spam. 7. Each person may have only one user account. It is fine to be critical here - that's what this board is for. But let's not beat a subject or a player to death, please.

Thank you, and most importantly, enjoy yourselves!


RedsZone.com is a privately owned website and is not affiliated with the Cincinnati Reds or Major League Baseball


Contact us: Boss | GIK | BCubb2003 | dabvu2498 | Gallen5862 | LexRedsFan | Plus Plus | RedlegJake | redsfan1995 | The Operator | Tommyjohn25