PDA

View Full Version : UZR Tidbits



nate
05-18-2009, 08:48 AM
From Fangraphs (http://www.fangraphs.com/blogs/index.php/uzrmay17th):


Almost a fourth of the way into the season, let’s take some position-by-position glances at the best and worst defenders.
1B
Best: Chris Davis (3.9), Ryan Howard (http://www.fangraphs.com/statss.aspx?playerid=2154&position=1B) (3.2), Lyle Overbay (http://www.fangraphs.com/statss.aspx?playerid=1617&position=1B) (2.9)
Worst: Jason Giambi (http://www.fangraphs.com/statss.aspx?playerid=818&position=1B/DH) (-3.9), Nick Johnson (http://www.fangraphs.com/statss.aspx?playerid=828&position=1B) (-3.4), Joey Votto (http://www.fangraphs.com/statss.aspx?playerid=4314&position=1B) (-3.3)

2B
Best: Ian Kinsler (http://www.fangraphs.com/statss.aspx?playerid=6195&position=2B) (5.4), Rickie Weeks (http://www.fangraphs.com/statss.aspx?playerid=1849&position=2B) (5.4), Brandon Phillips (http://www.fangraphs.com/statss.aspx?playerid=791&position=2B) (5.3)
Worst: Skip Schumaker (http://www.fangraphs.com/statss.aspx?playerid=3704&position=OF) (-7), Dan Uggla (http://www.fangraphs.com/statss.aspx?playerid=3442&position=2B) (-5.7), Chris Getz (http://www.fangraphs.com/statss.aspx?playerid=3388&position=2B) (-5)

3B
Best: Ryan Zimmerman (6.7), Joe Crede (http://www.fangraphs.com/statss.aspx?playerid=227&position=3B) (6.4), Evan Longoria (http://www.fangraphs.com/statss.aspx?playerid=9368&position=3B) (5.6)
Worst: Michael Young (http://www.fangraphs.com/statss.aspx?playerid=1286&position=2B/SS) (-7), Josh Fields (http://www.fangraphs.com/statss.aspx?playerid=7490&position=3B) (-4.1), Chipper Jones (http://www.fangraphs.com/statss.aspx?playerid=97&position=3B) (-3.8)

SS
Best: Marco Scutaro (http://www.fangraphs.com/statss.aspx?playerid=1555&position=2B/SS) (4.9), Elvis Andrus (http://www.fangraphs.com/statss.aspx?playerid=8709&position=SS) (4.2), Ryan Theriot (http://www.fangraphs.com/statss.aspx?playerid=3811&position=SS) (3.6)
Worst: Yuniesky Betancourt (http://www.fangraphs.com/statss.aspx?playerid=8585&position=SS) (-8.2), Khalil Greene (http://www.fangraphs.com/statss.aspx?playerid=1826&position=SS) (-4.4), Miguel Tejada (-4.3)

COF
Best: Nyjer Morgan (http://www.fangraphs.com/statss.aspx?playerid=4885&position=OF) (9.9), Jay Bruce (http://www.fangraphs.com/statss.aspx?playerid=9892&position=OF) (8), Brandon Moss (http://www.fangraphs.com/statss.aspx?playerid=4467&position=OF) (7.7)
Worst: Jason Bay (http://www.fangraphs.com/statss.aspx?playerid=1717&position=OF) (-9), Andre Ethier (http://www.fangraphs.com/statss.aspx?playerid=6265&position=OF) (-8.4), Adam Dunn (http://www.fangraphs.com/statss.aspx?playerid=319&position=OF) (-8.2)

CF
Best: Mike Cameron (http://www.fangraphs.com/statss.aspx?playerid=1070&position=OF) (7.8), Matt Kemp (http://www.fangraphs.com/statss.aspx?playerid=5631&position=OF) (7.3), Franklin Gutierrez (http://www.fangraphs.com/statss.aspx?playerid=3255&position=OF) (5.7)
Worst: Shane Victorino (http://www.fangraphs.com/statss.aspx?playerid=1677&position=OF) (-7.5), Vernon Wells (http://www.fangraphs.com/statss.aspx?playerid=1326&position=OF) (-5.8), Elijah Dukes (http://www.fangraphs.com/statss.aspx?playerid=4946&position=OF) (-3.8)

A few other tidbits:
Proof that this amount of UZR data is pretty useless in predictive value, Carlos Beltran (http://www.fangraphs.com/statss.aspx?playerid=589&position=OF) ranks as the fourth worst center fielder in the entire league at -3.5 runs. Over the last three years, Beltran’s UZR have been 8.8, 1.2, and 5.3. Even if you think his skills have declined, he’s unlikely to keep up this pace, which would have him at nearly -12 over 150 games.

The Pirates are your new team UZR leaders at 17.5, ahead of the Rays (16.3), Reds (15.2), Rangers (15.2), and Brewers (14.5). Meanwhile, on the other side of the spectrum, the Nationals (-18.3), Mets (-14.4), White Sox (-14.4), Red Sox (-12), and Orioles (-9) rate as the worst set of gloves in the league.



Interesting summary. Whether or not you think UZR is a good defensive metric, it should indicate that the Reds are improved with their gloves.

redsmetz
05-18-2009, 09:38 AM
From Fangraphs (http://www.fangraphs.com/blogs/index.php/uzrmay17th):



Interesting summary. Whether or not you think UZR is a good defensive metric, it should indicate that the Reds are improved with their gloves.

Can you explain it a little bit. I'm curious as to what gives Votto his number as I thought folks indicated they thought he has improved at 1st base.

nate
05-18-2009, 09:49 AM
Can you explain it a little bit. I'm curious as to what gives Votto his number as I thought folks indicated they thought he has improved at 1st base.

I'm not a real whiz at UZR (Ultimate Zone Ratings) but from what I understand, it cuts the field into little pie slices and measures how many balls in play are converted to outs in each slice. Here's part 1 (http://www.baseballthinkfactory.org/files/primate_studies/discussion/lichtman_2003-03-14_0/) and part 2 (http://www.baseballthinkfactory.org/files/primate_studies/discussion/lichtman_2003-03-21_0/) of an article explaining the system in eye-watering detail.

jojo
05-18-2009, 10:13 AM
Here's an attempt to put UZR into "layman's" terms:

http://www.redszone.com/forums/showpost.php?p=1399640&postcount


There is a treasure trove of information on the web about the nuts and bolts of play by play (PBP)-based defensive metrics just a google search away. I know some of it can be dry reading but if you have the time, it might change how you view defense so the effort would almost surely be rewarded.

I'll specifically focus on UZR in an effort ot answer your question because it's currently considered the gold standard. Basically, UZR uses play by play (PBP) data to determine the probability a ball in play (BIP) will be converted into an out based upon batted ball type (grounder, flyball, line drive, fliner, etc), BIP velocity (i.e. how hard it was hit), and zone into which it was hit. The field is divided into 78 zones as recorded on baseball reference website. UZR uses 64 of them. Each position has specific zones of responsibility (though some zones will overlap between two positions in which case balls within a zone are divided up proportionally i.e. if 80% of balls in a zone overlapping first and second base were caught by the second baseman, the first baseman would only be expected to field 20% of such balls).

In a nutshell a player’s UZR rating will be calculated as follows:

Player A’s “caught ball value” is determined for his specific position by using PBP data to tabulate how many put outs (PO) and hits were recorded in each zone of responsibility associated with his position while he was on the field. This value is then compared to the total number of PO and hits recorded in those zones by all players over the course of the season. It should be noted that a player could have a negative caught ball value for some zones of responsibility but a positive one for others depending upon his unique abilities (i.e. Felipe Lopez is weak to his right but actually very good to his left). If player A was below average in a zone, he’ll have a negative caught ball value for that zone compared to the league (average is zero or “neutral”). In contrast, if player A was above average in a zone his caught ball value will be a positive number.

Now to specifically address your question, Player A’s caught ball value for a zone can be fairly easily converted to a run value by multiplying it by the average run value of a hit in that zone (which has been determined by traditional linear weights hit values). Run values are totaled for each zone of responsibility for player A’s position to yield his final UZR value while playing his position.

UZR attempts to account for everything that could impact a fielder’s chances of recording an out in a zone by adjusting for a plethora of factors including environment, situation (number and position of runners and number of outs), batter’s handedness, and the pitcher’s BIP tendencies (i.e. GO/FO).

UZR is the best way I’ve seen defensive value tackled. Importantly, a majority of major league clubs are now using either this system or similar permutations of it.

Here’s some links that explain the system in much greater detail:

http://www.baseballthinkfactory.org/files/primate_studies/discussion/lichtman_2003-03-14_0/

http://www.baseballthinkfactory.org/files/primate_studies/discussion/lichtman_2003-03-21_0/

M2
05-18-2009, 10:49 AM
The galloping problem with fielding stats is that they're all over the place. CF seems to be a particular problem with UZR. Beltran and Victorino were among the best CF's in baseball last season. According to UZR, Aaron Rowand and Torii Hunter fell off of a cliff prior to 2008 and Vernon Wells is now as bad as the late model Ken Griffey Jr. in CF. It's got players instantly losing it out there, and then in many case rediscovering it (Willy Taveras leaps to mind).

Fielding skills ought to be consistent. You've got the range, arm, hands, reaction time you've got. That doesn't come and go.

Either we've discovered defense is a vastly more chaotic landscape than we ever suspected or the defensive measurements we're using are employing false norms and/or they are measuring the wrong things. There's way too much noise in these numbers.

jojo
05-18-2009, 10:59 AM
The galloping problem with fielding stats is that they're all over the place. CF seems to be a particular problem with UZR. Beltran and Victorino were among the best CF's in baseball last season. According to UZR, Aaron Rowand and Torii Hunter fell off of a cliff prior to 2008 and Vernon Wells is now as bad as the late model Ken Griffey Jr. in CF. It's got players instantly losing it out there, and then in many case rediscovering it (Willy Taveras leaps to mind).

Fielding skills ought to be consistent. You've got the range, arm, hands, reaction time you've got. That doesn't come and go.

Either we've discovered defense is a vastly more chaotic landscape than we ever suspected or the defensive measurements we're using are employing false norms and/or they are measuring the wrong things. There's way too much noise in these numbers.

It's mostly sample size issues. In the case of the CFers, UZR is looking at less than 350 defensive innings. That's not nearly enough to make meaningful conclusions.

M2
05-18-2009, 11:24 AM
It's mostly sample size issues. In the case of the CFers, UZR is looking at less than 350 defensive innings. That's not nearly enough to make meaningful conclusions.

First off, we see these variances occur in full season measurements. Players are constantly up and down. Second, I don't buy into small sample size on defensive numbers. We're not talking about a precise skill like hitting or pitching. Fielders aren't dealing with the constraints of the strikezone. Either you can reach a line drive 30 feet to your left or you can't. If there's one skill in baseball that shouldn't run hot and cold, it's fielding. The fielder can't control the number of balls that come his way, but that's not what we're supposed to be measuring.

The only way you've got a small sample size problem with a defensive measurement, at least one to this extent, is if you're using a false norm and/or measuring the wrong thing.

jojo
05-18-2009, 11:36 AM
First off, we see these variances occur in full season measurements. Players are constantly up and down. Second, I don't buy into small sample size on defensive numbers. We're not talking about a precise skill like hitting or pitching. Fielders aren't dealing with the constraints of the strikezone. Either you can reach a line drive 30 feet to your left or you can't. If there's one skill in baseball that shouldn't run hot and cold, it's fielding. The fielder can't control the number of balls that come his way, but that's not what we're supposed to be measuring.

The only way you've got a small sample size problem with a defensive measurement, at least one to this extent, is if you're using a false norm and/or measuring the wrong thing.

Or if the data is just inherently noisy because it takes a lot of defensive innings to get enough similar balls to control for randomness.

M2
05-18-2009, 11:54 AM
Or if the data is just inherently noisy because it takes a lot of defensive innings to get enough similar balls to control for randomness.

Given what we know about full season defensive numbers, randomness in small sample sizes isn't the problem here. Plus, it's a lazy excuse that doesn't deal with the reality that a player's ability to make a defensive play isn't random. A truly accurate defensive measurement would tell me who's got range in a fairly small sample size.

jojo
05-18-2009, 12:05 PM
Given what we know about full season defensive numbers, randomness in small sample sizes isn't the problem here.

Considering that several seasons of defensive data is the ideal, I'm not sure what you're arguing we know regarding a full season of data and sample size issues.


Plus, it's a lazy excuse that doesn't deal with the reality that a player's ability to make a defensive play isn't random.

It's not the player that's random. It's the opportunities.


A truly accurate defensive measurement would tell me who's got range in a fairly small sample size.

Sure at the extremes.

M2
05-18-2009, 12:17 PM
Considering that several seasons of defensive data is the ideal, I'm not sure what you're arguing we know regarding a full season of data and sample size issues.

How many seasons of inconsistent numbers should it take before people recognize the inherent flaws in the system?


It's not the player that's random. It's the opportunities.

Then you're measuring the wrong thing.


Sure at the extremes.

No, we should be able to determine true range rather quickly. You can either cover x distance over y time or you can't.

jojo
05-18-2009, 12:37 PM
How.......................................can't.

I guess the world awaits UZR/PMR/Plus-minus version 2.0. For now we'll have to settle for the systems that have allowed huge advances in evaluating and valuing defense.

CaiGuy
05-18-2009, 12:42 PM
The galloping problem with fielding stats is that they're all over the place. CF seems to be a particular problem with UZR. Beltran and Victorino were among the best CF's in baseball last season. According to UZR, Aaron Rowand and Torii Hunter fell off of a cliff prior to 2008 and Vernon Wells is now as bad as the late model Ken Griffey Jr. in CF. It's got players instantly losing it out there, and then in many case rediscovering it (Willy Taveras leaps to mind).

Fielding skills ought to be consistent. You've got the range, arm, hands, reaction time you've got. That doesn't come and go.

Either we've discovered defense is a vastly more chaotic landscape than we ever suspected or the defensive measurements we're using are employing false norms and/or they are measuring the wrong things. There's way too much noise in these numbers.

Wells' hitting stats have been up and down during that last several years. Like you said the physical tools stay the same, but the performance over certain sample sizes can vary. Wouldn't that apply to defense as well?

blumj
05-18-2009, 01:00 PM
First off, we see these variances occur in full season measurements. Players are constantly up and down. Second, I don't buy into small sample size on defensive numbers. We're not talking about a precise skill like hitting or pitching. Fielders aren't dealing with the constraints of the strikezone. Either you can reach a line drive 30 feet to your left or you can't. If there's one skill in baseball that shouldn't run hot and cold, it's fielding. The fielder can't control the number of balls that come his way, but that's not what we're supposed to be measuring.

The only way you've got a small sample size problem with a defensive measurement, at least one to this extent, is if you're using a false norm and/or measuring the wrong thing.
Why would you think players' fielding wouldn't run hot and cold? They're not robots, and they're not always in perfect health or at peak focus or playing with the same level of confidence.

RedsManRick
05-18-2009, 01:34 PM
The galloping problem with fielding stats is that they're all over the place. CF seems to be a particular problem with UZR. Beltran and Victorino were among the best CF's in baseball last season. According to UZR, Aaron Rowand and Torii Hunter fell off of a cliff prior to 2008 and Vernon Wells is now as bad as the late model Ken Griffey Jr. in CF. It's got players instantly losing it out there, and then in many case rediscovering it (Willy Taveras leaps to mind).

Fielding skills ought to be consistent. You've got the range, arm, hands, reaction time you've got. That doesn't come and go.

Either we've discovered defense is a vastly more chaotic landscape than we ever suspected or the defensive measurements we're using are employing false norms and/or they are measuring the wrong things. There's way too much noise in these numbers.

UZR doesn't measure skills; it measures performance. It's extremely important to adhere to this difference. Skills change gradually. Beltran's speed, reaction time, etc. likely haven't changed much since last year. But his performance, his utilization of those skills in a relatively small sample of opportunities, has been worse. We shouldn't confuse these two statements; just as we wouldn't suggest that Jimmy Rollins is suddenly Mario Mendoza or that Aaron Hill is suddenly ARod.

People say that fielding doesn't slump; maybe they're wrong. Offensive performance can swing pretty wildly from year to year (just ask Paul Konerko) -- why would assume that fielding is different? I would feel comfortable saying that defense should vary less, because there are fewer variables involved, but I see nothing wrong with guys having good and bad years -- let alone good/bad 40 games.

It's quite difficult to measure skills directly, so we use performance as a proxy. Because performance is affected by more than just the skill of the player, and because the application of skill is not completely consistent from one event to the next, performance varies. Even the best methods of offensive performance are completely unreliable in just 1/4 of a season. I don't see why fielding should be much different.

lollipopcurve
05-18-2009, 01:39 PM
Why would you think players' fielding wouldn't run hot and cold? They're not robots, and they're not always in perfect health or at peak focus or playing with the same level of confidence.

A player's defensive performance should be much more consistent than his offensive performance. There is not nearly the randomness in a fielding chance as there is in a batted ball.

One of the things to keep in mind about defensive play is that it often involves teamwork. Keystone infielders take throws from teammates before turning a double play. First basemen scoop balls in the dirt. Players must communicate on balls hit into zones that others can cover. So, I like the DER percentages, since they're team-based. The individual metrics, compared season to season and system to system, just confuse me.

jojo
05-18-2009, 01:48 PM
UZR doesn't measure skills; it measures performance. It's extremely important to adhere to this difference. Skills change gradually. Beltran's speed, reaction time, etc. likely haven't changed much since last year. But his performance, his utilization of those skills in a relatively small sample of opportunities, has been worse. We shouldn't confuse these two statements; just as we wouldn't suggest that Jimmy Rollins is suddenly Mario Mendoza.

People say that fielding doesn't slump; maybe they're wrong.

Another point to keep in mind is that PBP-based metrics measure defense relative to position so noise is added because the relative pool of players is always changing. Also randomness in opportunities isn't a trivial issue-and this isn't referring to total chances but rather the character of those chances.

M2
05-18-2009, 01:57 PM
Why would you think players' fielding wouldn't run hot and cold? They're not robots, and they're not always in perfect health or at peak focus or playing with the same level of confidence.


Wells' hitting stats have been up and down during that last several years. Like you said the physical tools stay the same, but the performance over certain sample sizes can vary. Wouldn't that apply to defense as well?

Defense is a different activity. Hitting involves a constrained activity (swinging a bat) through a loosely defined area in order to put a round bat on a moving round ball, inside of a highly compressed reaction time. The hitter controls very little of what's going on when he's hitting.

Fielding is completely different. While the fielder has no control over where the ball is heading and in what fashion it's heading there, the fielder has absolute control over covering x distance in y time. The ball is going to be where it's going to be and the fielder has to get there. The event is random, but we claim we're interested in the player's ability to respond to the event.

Professional players spend years before they reach the majors honing their reaction times. They are somewhat robotic in this sense. Their reaction times are fairly standard, down into sub-second decimals in terms of reaction time variance. The ones who don't get consistent reads or reactions (e.g. Felipe Lopez) aren't very good fielders and they're pretty easy to spot.

Generally speaking, you're not a lot faster one day to the next. You're not running a 4.5 40 on Monday and a 5.0 on Tuesday. You're also not throwing a lot better one day vs. the next. These are your raw physical abilities - run, jump. throw. It's got nothing to do with confidence and if you're so injured you can't do those basic things, you probably shouldn't be playing because that's a serious injury.

There are some mechanics involved in getting the glove on the ball, transferring the ball for a throw and making a good throw. Yet we figured out how to capture most of that long ago with fielding percentage, which tells you how a player performs when he gets to a ball (or the ball gets to him). Yet that's only the mechanical complement to range. Hitting is all about the mechanics.

As a hitter, technically you can hit anything in or near the strikezone, but you know that you will only get a hit out of a small percentage of those balls. The factors at play are beyond your control. The best hitters are the ones who do a slightly better job of controlling overwhelming circumstances.

As a fielder, you know your range. As soon as you pick up the ball you've got it sorted into three categories - I will get this, I've got a shot at this, I'm not getting this. That's because you've got an innate sense of how much distance you can cover in the time you have to get there. You know your abilities and they don't vary that much. Getting there is in your control.

M2
05-18-2009, 02:03 PM
UZR doesn't measure skills; it measures performance.

My knock on UZR is it measures neither. It takes performance (you made x number of plays) and then compares that to an assumed skills overlay (but you could have made y number of plays) and then spits out an inconsistent number that may not reflect the fielder in question. All it may be telling us is that certain players have had easier chances than others ... and who bloody cares about that?


It's quite difficult to measure skills directly, so we use performance as a proxy.

That's my complaint in a nutshell. It's like trying to determine someone's eye color by looking at their shadow.

jojo
05-18-2009, 02:15 PM
My knock on UZR is it measures neither. It takes performance (you made x number of plays) and then compares that to an assumed skills overlay (but you could have made y number of plays) and then spits out an inconsistent number that may not reflect the fielder in question. All it may be telling us is that certain players have had easier chances than others ... and who bloody cares about that?

What UZR is measuring is whether or not a player made a play on a ball that his peers converted given a similar location, speed, and type of the ball (GB/LD/FB etc) while also considering the base/out sate, environment, batter handedness, and pitcher's ball-in-play tendencies.


It's like trying to determine someone's eye color by looking at their shadow.

It's like trying to determine a person's eye color by comparing it to a palette containing the range of eye colors expressed in a given human population all the while knowing the frequency at which each shade is expressed in that population.

M2
05-18-2009, 02:16 PM
Another point to keep in mind is that PBP-based metrics measure defense relative to position so noise is added because the relative pool of players is always changing.

Noisy statistics = bad statistics. I'm not terribly moved by the notion that these are the best cacophonous statistics we've got. I use these numbers because it's all I've got too, but it doesn't mean I have to like it.

FWIW, I don't think we've made so much as one ounce of headway in terms of defensive evaluation at the player level. Go back in time 30 years and coaches generally had it sorted out as to who could play defense and who couldn't. GMs could do a quick thumbnail on whether a player would add enough defense to offset what he doesn't deliver on offense. They weren't always right, but I don't see that they're right any more often these days. To my eyes, it all comes across as a fancier, more expensive way of making the same mistakes.

M2
05-18-2009, 02:20 PM
What UZR is measuring is whether or not a player made a play on a ball that his peers converted given a similar location, speed, and type of the ball (GB/LD/FB etc) while also considering the base/out sate, environment, batter handedness, and pitcher's ball-in-play tendencies.

And if I happen to get a lot of easy chances, I'm a brilliant fielder. Once again, who bloody cares? It's a false norm and the output isn't really telling you anything. I mean, maybe it is, but there's a good chance it isn't. That's a lousy stat.


It's like trying to determine a person's eye color by comparing it to a palette containing the range of eye colors expressed in a given human population all the while knowing the frequency at which each shade is expressed in that population.

Well, whatever you do, don't look at the person's eyes. That would be crazy.

jojo
05-18-2009, 02:21 PM
Noisy statistics = bad statistics. I'm not terribly moved by the notion that these are the best cacophonous statistics we've got. I use these numbers because it's all I've got too, but it doesn't mean I have to like it.

Noise is an inherent property of every population. That's why sample size is an issue with every metric.

jojo
05-18-2009, 02:27 PM
And if I happen to get a lot of easy chances, I'm a brilliant fielder. Once again, who bloody cares? It's a false norm and the output isn't really telling you anything. I mean, maybe it is, but there's a good chance it isn't. That's a lousy stat.

If you get a lot of easy chances, then much is expected of you because your peers have converted them too.


Well, whatever you do, don't look at the person's eyes. That would be crazy.

It's the ultimate irony....in order to be able to differentiate subtleties in defensive skill, one has to WATCH all of the plays....

Who knew that would be considered outlandish, crazy talk?

Scouts should be rejoicing because PBP data is the ultimate complete eye-derived data set.

LvJ
05-18-2009, 03:00 PM
:beerme: Nashville, eh?

M2
05-18-2009, 03:09 PM
If you get a lot of easy chances, then much is expected of you because your peers have converted them too.

If 85% of Player A's chances are easy and 70% of Player B's chances are easy then UZR is going to tell me Player A is a better fielder regardless of whether he is.


It's the ultimate irony....in order to be able to differentiate subtleties in defensive skill, one has to WATCH all of the plays....

Who knew that would be considered outlandish, crazy talk?

Scouts should be rejoicing because PBP data is the ultimate complete eye-derived data set.

The problem is they don't watch the plays and measure them. They watch the ball and measure fictional zones that consistently churn out bad numbers. They substitute perceptual flaws and assumptions for actual measurement, all while indulging a fixation on the ball.

To borrow a phrase from the female population, "My eyes are up here buster."


Noise is an inherent property of every population. That's why sample size is an issue with every metric.

And fielding stats are noisier despite the fact that they attempt to measure what should be a more consistent skill set. Also, I know how to adjust for small sample noise in hitting and pitching stats because I understand the general and specific player norms. There are no general and specific player norms in fielding stats (which creates a mathematical nightmare that proponents of these numbers blithely ignore).

I am able to take hitting and pitching stats and determine ability across the entire population of players. Fielding stats are measuring happenstance as much as ability.

Scrap Irony
05-18-2009, 03:23 PM
Are you saying, M2, that a player's defensive ability doesn't change much from year to year? If so, I think I would disagree with that, as numerous nicks and little injuries plus changes in depth perception and vision plus the vagaries of aging all affect defensive ability (far more, I would think, than offensive ability).

Can I prove any of that? Nope. Not a whit. But I suspect it to be true. Anyone else agree with that or am I (pardon the pun) completely out in left field?

blumj
05-18-2009, 03:36 PM
There is not nearly the randomness in a fielding chance as there is in a batted ball.
How can one be more random than the other when they're the exact same thing, a fielding chance IS a batted ball?

nate
05-18-2009, 03:40 PM
How can one be more random than the other when they're the exact same thing, a fielding chance IS a batted ball?

I think the difference is, a fielding chance is a play that a defender can make where a batted ball is just a ball in play. In other words, a ball hit to LF is a batted ball as far as the 1B is concerned and a fielding chance for the LF.

Just a guess.

TRF
05-18-2009, 03:52 PM
I've been reading threads on defensive stats on Redszone for years. One thing I can't recall seeing discussed was health. A player healthy enough to hit, say Barry Bonds in 2006 might not be healthy enough to field adequately. (didn't want to mention Dunn here as that has the effect of napalming the thread, but he did deal with a torn meniscus). Often we equate health with hitting and pitching only, but almost never defensively. Could not the wide range attributed to Carlos Beltran have anything to do with his overall health?


do'h... Scrap beat me to it.

M2
05-18-2009, 03:56 PM
Are you saying, M2, that a player's defensive ability doesn't change much from year to year? If so, I think I would disagree with that, as numerous nicks and little injuries plus changes in depth perception and vision plus the vagaries of aging all affect defensive ability (far more, I would think, than offensive ability).

Can I prove any of that? Nope. Not a whit. But I suspect it to be true. Anyone else agree with that or am I (pardon the pun) completely out in left field?

Players always have nagging injuries. That's true in every sport. That said, you don't speed up or slow down appreciably from year to year without suffering a major injury (e.g. Ken Griffey Jr. in 2002 when he lost multiple steps in the OF and never got them back). Your eyes also don't go bad overnight. We're talking about players in their physical primes here and in no sport do their raw physical abilities vary drastically when they are deemed healthy enough to play.

M2
05-18-2009, 04:03 PM
How can one be more random than the other when they're the exact same thing, a fielding chance IS a batted ball?

I think you misread that. lollipop started that thought with "A player's defensive performance should be much more consistent than his offensive performance."

The reference to "batted ball" referred to what a hitter can control while he's at the plate (e.g. not very much). At least that's how I read it. A fielder can see exactly where the ball in play is headed and only has to get there, not easily done in all cases, but not a process with any significant built-in randomness. The fielder also is able to get his hands on the ball to make a play rather than try to push it in the right direction with a blunt instrument.

jojo
05-18-2009, 04:08 PM
If 85% of Player A's chances are easy and 70% of Player B's chances are easy then UZR is going to tell me Player A is a better fielder regardless of whether he is.

A player gets rewarded the most for making the exceptional play. That said, the issue of chance is controlled by sample size. Every metric is a slave to sample size when being used as a measure of true skill. PBP-based metrics are data intensive. If you find that needing more than 150 chances is unpalatable to you, that's your choice but it's not a fatal flaw. I'd also wonder what metric you think is better.


They watch the ball and measure fictional zones that consistently churn out bad numbers. They substitute perceptual flaws and assumptions for actual measurement, all while indulging a fixation on the ball.

Could you demonstrate this, hopefully in a systemic fashion? You've concluded it and have said it several times now but base upon what?


And fielding stats are noisier despite the fact that they attempt to measure what should be a more consistent skill set.

I think it's begging the question to state that getting good reads, getting good jumps, taking proper routes to balls, and making split decisions about which base to throw to etc is an inherently less variable skill than judging the strikezone/plate discipline.


I am able to take hitting and pitching stats and determine ability across the entire population of players. Fielding stats are measuring happenstance as much as ability.

Fielding stats are measuring how a player handled a play and comparing it to the performance of all of his peers that had a similar play. Getting a lazy flyball hit right at you isn't really any more happenstance that one guy getting to hit Ryan Franklin's fastball while another guy is stuck hitting Volquez's on any given night.

M2
05-18-2009, 05:08 PM
A player gets rewarded the most for making the exceptional play. That said, the issue of chance is controlled by sample size. Every metric is a slave to sample size when being used as a measure of true skill. PBP-based metrics are data intensive. If you find that needing more than 150 chances is unpalatable to you, that's your choice but it's not a fatal flaw. I'd also wonder what metric you think is better.

Not sure how much clearer I can make that I don't think a good fielding metric exists, or that the current sample size are only signposts of the larger problems and not the problem in and of themselves.

I just don't allow for the lazy man's excuse that the only problem here is small sample size when it's readily apparent the same problems exist with large sample sizes as well.


Could you demonstrate this, hopefully in a systemic fashion? You've concluded it and have said it several times now but base upon what?

That's how they do ZR analysis. They watch the play and determine if a ball is in zone or not. That they're watching the wrong thing - the ball as opposed to the fielder - is just the failure to apply common sense. The absurdity of how the zones are determined is just a bonus.


I think it's begging the question to state that getting good reads, getting good jumps, taking proper routes to balls, and making split decisions about which base to throw to etc is an inherently less variable skill than judging the strikezone/plate discipline.

I think it's oblivious to the actual playing of the game to speculate that it isn't. Aside from time compression, which alone makes batting vastly more variable than fielding, running across a two dimensional surface in pursuit of a ball in play is inherently less variable than swinging a blunt object through a three dimensional space in the hopes of making contact with a small, hurtling sphere.


Fielding stats are measuring how a player handled a play and comparing it to the performance of all of his peers that had a similar play. Getting a lazy flyball hit right at you isn't really any more happenstance that one guy getting to hit Ryan Franklin's fastball while another guy is stuck hitting Volquez's on any given night.

Last I looked, Franklin and Volquez still have to pitch to the same strikezone.

I've already got quantitative fielding statistics - totals chances, fielding percentage. UZR purports to be a qualitative statistic, but it fails basic sniff tests by churning out remarkably inconsistent numbers (both in large and small sample sizes).

Given that, here's your options -

1) You're measuring the wrong thing and you need to start measuring something else.

2) You've created a false norm and you need to perform a whole lot more rigor in establishing that norm. Comparing something to the average is only meaningful when the average itself has meaning. The wiggle in these numbers is just nature's way of saying, "Your zones are silly and you don't understand the norm yet."

3) Fielding really is that random, based mostly on the quality of the ball in play and not the fielder, which means you're grossly overstating the run value of the defenders in question.

4) All of the above.

Honestly, all UZR is doing is yet again formalizing the age-old process of stating "I think he should have gotten to that one." It's nothing new under the sun and it's provided no revelations. I didn't need a qualitative stat to tell me Ken Griffey Jr. was an awful OF. Why? Because too often I found myself saying "he should have gotten to that one."

I know, but it gives me the exact run value of the plays made and not made. Sure, except for the other BIP fielding stats that spit out different run values for the same events. So it doesn't give us that. It's yet another thumbnail for something we had thumbnailed for decades before it.

TRF
05-18-2009, 05:22 PM
What exactly does it measure?

Does it take into account positioning by the bench?

pitching staff FB/GB tendencies?

Park?

Weather?

Day/Night games?

What about intangibles like health?

The great thing about hitting stats is like M2 stated, there is but one strikezone. It gets interpreted differently by different umpires, but the variance is pretty small. To measure Player A's defense in CF vs Player B's in two different divisions with two completely different pitching staffs AND the relative defensive prowess of the LF, RF, and to a lesser degree SS and 2B, and the other factors i listed above seems a bit hard to qualify/quantify.

OUReds
05-18-2009, 05:30 PM
3) Fielding really is that random, based mostly on the quality of the ball in play and not the fielder, which means you're grossly overstating the run value of the defenders in question.



I agree with a lot of your points, especially the one above. Attempting to quantify how well a ball is hit introduces a very troublesome set of variables.

I don't agree with your conclusion though. If I think "he should have gotten to that one", I need to know how often others get to that ball to make that judgment fairly. UZR is that best tool so far to gauge that answer, because I'm just not interested enough in non-Reds baseball to see enough other players to make my own judegement. I'd say that goes for 99.9% of fans as well.

Besides, UZR does a reasonably good job on passing my sniff test. Players who I see enough to think are good fielders generally fair well. Those who I think are bad generally don't. I just don't see enough crazy variation from year to year to get excited about it.

jojo
05-18-2009, 05:45 PM
At this point we'll just have to agree to disagree.

All I'll add is that statements like below need to be demonstrated rather than simply thrown out as conclusions.


I've already got quantitative fielding statistics - totals chances, fielding percentage. UZR purports to be a qualitative statistic, but it fails basic sniff tests by churning out remarkably inconsistent numbers (both in large and small sample sizes).


Beltran was used as an example of wild variation. Below is his OPS and defense as measured by UZR and Dewan's +/-. I don't see wild fluctuations in his defense either across years or between metrics with the exception of 2005 which really reflects his year long battle with a quadriceps issue. Certainly his glove wasn't fluctuating much greater than his bat over the period covered below.



OPS UZR U rank Dewan D rank
2002 0.847 9 5 X X
2003 0.911 10 6 X X
2004 0.915 4 7 6 7
2005 0.744 -7 14 2 15
2006 0.982 6 11 6 8
2007 0.878 1 9 14 2
2008 0.876 9 5 13 2

jojo
05-18-2009, 05:48 PM
The great thing about hitting stats is like M2 stated, there is but one strikezone.

I'd argue that there are about 80 different strikezones and the differences aren't necessarily trivial (just ask David Weathers).

blumj
05-18-2009, 06:42 PM
Honestly, all UZR is doing is yet again formalizing the age-old process of stating "I think he should have gotten to that one." It's nothing new under the sun and it's provided no revelations. I didn't need a qualitative stat to tell me Ken Griffey Jr. was an awful OF. Why? Because too often I found myself saying "he should have gotten to that one."
Sure, because he was on the team that you watch every day. I want the stats to tell me about the players I'm not watching every day, I don't really need them for the ones I do. I don't need to look up Jason Bay's UZR or his OPS to know how he's playing because I watch him play every day. I might look up Carlos Quentin's UZR or OPS because I don't, and both of those numbers will likely give me an impression of something that I haven't seen for myself, and that's really all I want them to do.

M2
05-18-2009, 07:45 PM
All I'll add is that statements like below need to be demonstrated rather than simply thrown out as conclusions.

We've been over that territory multiple times and I see no reason to repeat past conversations. I've got two such inconsistent cases listed below and I know you've spent enough time on Fan Graphs to have seen plenty of similar ones.


Beltran was used as an example of wild variation. Below is his OPS and defense as measured by UZR and Dewan's +/-. I don't see wild fluctuations in his defense either across years or between metrics with the exception of 2005 which really reflects his year long battle with a quadriceps issue. Certainly his glove wasn't fluctuating much greater than his bat over the period covered below.



OPS UZR U rank Dewan D rank
2002 0.847 9 5 X X
2003 0.911 10 6 X X
2004 0.915 4 7 6 7
2005 0.744 -7 14 2 15
2006 0.982 6 11 6 8
2007 0.878 1 9 14 2
2008 0.876 9 5 13 2


Apparently that wasn't all you'll add.

So the difference in being an average fielder (1 UZR) and a good one (9-10 UZR) isn't significant? Again, I EXPECT fluctuations in hitting numbers given the nature of the task. The fluctuations I see in the fielding numbers strike me as the inevitable result of following the ball instead of the fielder.

And you don't have to look too hard to find wild fluctuations inside of UZR. Mike Cameron (8.2 last season, -8.0 the season before) is a perfect example. Taveras supposedly dropped off by 22.7 runs (actual, and 29.7 runs per 150 games) when he went from Houston to Colorado at age 25.

Let's not even get into the disparities between UZR and Dewan who are looking at the same events. They can't even agree on what a make-able or difficult play is. Is Carlos Beltran the the best CF alive (which is how Dewan rates him from 2006-8) or is he in the bottom half of the top 10? The difference matters. It boils the whole mess down to a question of whose perception do you trust more? And that's not a question we should be asking when it comes to a statistic.

lollipopcurve
05-18-2009, 07:50 PM
I think you misread that. lollipop started that thought with "A player's defensive performance should be much more consistent than his offensive performance."

The reference to "batted ball" referred to what a hitter can control while he's at the plate (e.g. not very much). At least that's how I read it. A fielder can see exactly where the ball in play is headed and only has to get there, not easily done in all cases, but not a process with any significant built-in randomness.

Yes, M2, thank you.

To put it another way, what a batter does with average waist high fastballs on the outer half will vary a lot more than what a shortstop will do with the groundballs that batter hits into the SS zone on those pitches. The SS is going to make the play most of the time, but the batter is not going to hit grounders to to the SS's zone most of the time.

M2
05-18-2009, 07:53 PM
Sure, because he was on the team that you watch every day. I want the stats to tell me about the players I'm not watching every day, I don't really need them for the ones I do. I don't need to look up Jason Bay's UZR or his OPS to know how he's playing because I watch him play every day. I might look up Carlos Quentin's UZR or OPS because I don't, and both of those numbers will likely give me an impression of something that I haven't seen for myself, and that's really all I want them to do.

Fair point, though I'm not sure you wouldn't get a more useful and consistent representation of fielding ability by polling advance scouts from each team to give a letter grade to each fielder and then averaging it.

The revolutionary part of hitting statistics is they revealed more about hitters than we knew from empty proclamations like "he handles the bat really well." They've demonstrated that players were being misplaced in various lineups and that baseball men were often making nonsensical decisions. So far with fielding statistics it seems like the baseball men more or less had it right. It's comforting in some ways, but it's not much of a revolution.

Rojo
05-19-2009, 04:12 PM
When I first came across zone ratings I got to the dividing up the field part and stopped reading. If someone's going to observe and judge, I just as soon it was Cam Bonifay as Bill James.

Call me old-fashioned but I want my green eye shade types in the back room crunching numbers, digesting digital inputs for digital outputs.

M2
05-19-2009, 04:27 PM
If someone's going to observe and judge, I just as soon it was Cam Bonifay as Bill James.

Same here, though if the stats guys want to measure the reaction times and ground covered by the fielders I'm cool with that.

lollipopcurve
05-19-2009, 05:12 PM
though if the stats guys want to measure the reaction times and ground covered by the fielders I'm cool with that.

Yep. Get the trajectory/flight path, the spin and hang time, too. Wind readings also appreciated. We're talking serious surveillance of the terrain and airspace. Then maybe they'll have something trustworthy.