Is the WAR war over?

[jojo]

Do you see the same sized swings in offensive players from year to year w/o an obvious type of reason? Look at Bourn, he goes from #4 in the majors for OFers in 2010 to #50 in 2011 to #1 in 2012. Kemp from 2009-2011, 22nd to dead last (62nd) to 37th. Crawford is God's gift until he gets to Fenway and then all of a sudden is a liability. Nyjer Morgan goes from an UZR/150 of 34.7 in 2009 (and a WAR of 5.2, #7 in all of baseball) to an UZR/150 of 2.3 in 2010, knocking his WAR down to 1.0 the following season.

And, even in those instances where someone may have an off year at the plate, it is much easier to see that and prove that compared to an abstract metric like UZR. If there were a way to illustrate that these material swings in defensive performance were legitimate, I would obviously feel better about using UZR and related metrics.

And maybe the answer to your question about getting your head around overall value is that the industry isn't there yet. So you can obviously use what you wish, but excuse some if WAR isn't used as a legitimate measurement tool.

Lets put intuition to the test:

Bourn's bat:           Bourn's glove
'08: -24.5 runs          3
'09: 2.0 runs            9.9
'10: -5.2 runs           19.4
'11: 5.9 runs            -6
'12: 6.6 runs            22.4

Bourn has had a single season as a starter where he wasn't graded as a plus defender (2011). Importantly other metrics such as Dewan's mirror Bourn's defensive estimates almost perfectly. Meanwhile the value of his bat has been all over the place with a 31 run difference between his best and worse seasons (a bigger spread than his defensive values).

Concerning Crawford, first, in 2011 he transitioned to an environment where it's well known that LF defense is difficult to accurately measure and he really didn't play defense in 2012 (less than 250 defensive innings). So it is correct that he graded much worse in 2011 than 2010 as he went from a UZR of 18.2 to one of -2.2. But even if you ignore the Fenway issue and conclude that a change of 20 runs is too volatile to believe, then you have to also throw offense out with the bathwater too... Why? Crawford's bat went from being worth 28.8 runs in 2010 to -9 in 2011-a swing of almost 40 runs!!!!!!!!! So much for offense not being volatile.

Here's look at Morgan:

Morgan's bat:           Morgan's glove
'08: -.6 runs              4.4
'09: 4.6 runs              27.6
'10: -15.2 runs            1.1
'11: 9.5 runs              9.2
'12: -10.4 runs            3.8

Intiution needs to be checked with the numbers. I really think youre imagining a very consistent offensive side of things and an especially volatile defensive side of things. At least the players you've keyed on as quintessential examples to support your conclusion, really undercut your overarching argument as based upon components of WAR, they've been far more consistent defensively then they have offensively.

The three guys above really don't point to a flaw in UZR that would invalidate WAR as a legitamate evaluation tool.

[dougdirt]

To be fair, we know that Fenway and left field will certainly screw up UZR because of the Green Monster.

[RedsManRick]

Do you see the same sized swings in offensive players from year to year w/o an obvious type of reason? Look at Bourn, he goes from #4 in the majors for OFers in 2010 to #50 in 2011 to #1 in 2012. Kemp from 2009-2011, 22nd to dead last (62nd) to 37th. Crawford is God's gift until he gets to Fenway and then all of a sudden is a liability. Nyjer Morgan goes from an UZR/150 of 34.7 in 2009 (and a WAR of 5.2, #7 in all of baseball) to an UZR/150 of 2.3 in 2010, knocking his WAR down to 1.0 the following season.

Well, I noticed you ignored my Ben Zobrist example. I guess I'll have to search for more anecdotes.

Regarding Fenway, it depends on what you're trying to measure. Did playing in Fenway make Carl Crawford a worse defensive player? No. Did playing in Fenway drastically reduce the number of opportunities he had to prevent runs? Absolutely. Why would we not account for that? Red Sox LFs have less opportunities to create runs than their LF peers. That's a reality.

To take another example, if a player were to get dropped from 3rd to 8th in the lineup, we wouldn't try to normalize the number of plate appearances or RBI opportunities in assessing his sum production.

I think part of the problem here is when treated as an estimate of talent, UZR has some pretty big flaws due to the impact of context. And many people, myself included, are probably overly inclined to use UZR as a talent estimator without sufficient adjustment.

But to get back to the question of what the data really looks like, I pulled some. I looked at all batters with at least 500 PA in each 2011 & 2012 and examined the difference in the # of runs they created (Runs Above Average -- wOBA based). I then looked at all qualified fielders and compared the difference in in their UZRs. Below are the range of those differences.

2012-2011	Runs Produced (Runs Above Average)				Runs Prevented (Ultimate Zone Rating)
Run Difference	Players	%ofTot	Extreme 3					Players %ofTot	Extreme 3	
 40 or more 	 2	2%	Alex Rios, Aaron Hill, Ian Desmond		0	0%	
 30 to 39	 3	3%							0	0%	
 20 to 29	 2	2%							1	1%	Michael Bourn 
 10 to 19	14	15%							9	11%	Mark Reynolds, Ichiro Suzuki
 0 to 9		19	21%							34	43%	
-1 to -9	24	26%							29	37%	
-10 to -19	14	15%							5	6%	Rickie Weeks, Ian Kinsler
-20 to -29	 9	10%							1	1%	Alcides Escobar
-30 to -39	 5	5%	Michael Young, Casey Kotchman, Adrian Gonzalez	0	0%	
-40 or less	 0	0%							0	0%	
		92								79

Suffice it to say that among players who were "regulars" in both 2011 & 2012, there was a broader spread of changes in offensive performance than defensive production.
- For RAA, 53% of the players experienced a variation of at least 10 runs.
- For UZR, only 20% of the players experienced a variation of at least 10 runs.

What's not captured here, among other things, is the positional adjustment associated UZR that go in to WAR -- if a guy moved up or down the defensive spectrum, that will affect his WAR by a handful. But presumably, if he's shifted, the new adjustment will be somewhat offset by better or worse performance in his new position.

I think the basic observation still stands. Yes, there are some pretty dramatic year-to-year variations in UZR. However, broadly speaking, the range of variation in UZR is smaller than the variation in offensive performance.

[RedsManRick]

To be fair, we know that Fenway and left field will certainly screw up UZR because of the Green Monster.

What do we mean by screw it up? Sure it's not a reliable measure of player defensive talent, but is it an inaccurate reflection of the production that player provided compared to his positional peers?

Isn't it true that LFers in Fenway will have a lot fewer opportunities to make plays, especially OOZ plays, thus limiting their ability to contribute to run prevention relative to their positional peers? It would be wrong for faulting the player, claiming he played poorly or had less ability. But it is not wrong to suggest that he had less defensive contribution.

I can see how it messes with certain applications of UZR. But I don't think it's evidence that the stat is fundamentally broken in some way.

[dougdirt]

What do we mean by screw it up? Sure it's not a reliable measure of player defensive talent, but is it an inaccurate reflection of the production that player provided compared to his positional peers?

Isn't it true that LFers in Fenway will have a lot fewer opportunities to make plays, especially OOZ plays, thus limiting their ability to contribute to run prevention relative to their positional peers? It would be wrong for faulting the player, claiming he played poorly or had less ability. But it is not wrong to suggest that he had less defensive contribution.

I can see how it messes with certain applications of UZR. But I don't think it's evidence that the stat is fundamentally broken in some way.

Basically what you said. Firstly, the "zone" in Fenway is smaller because of the wall, giving them fewer chances to make "in zone plays", which makes each play not made "hurt" more. Then it also limits how many out of zone plays a guy can make.

I don't think the stat is broken because of Fenway, but I also think we need to take Fenway defensive UZR rating with a grain of salt. It is why I would love to see UZR ratings for home/road to see how much parks can come into play.

[edabbs44]

Interesting chart RMR. Can you run for 09-10 and 10-11 as well?

[RedsManRick]

Interesting chart RMR. Can you run for 09-10 and 10-11 as well?

Definitely. The plan is to expand it to 3 years of matched pair data.

[AtomicDumpling]

Some ideas for why a fielder's UZR could fluctuate:

• He was playing hurt for awhile. An outfielder with a pulled groin or a sore heel would struggle to reach as many balls as when he was healthy. An infielder with a sore arm could make weaker throws.
  
• Vision. A new pair of contacts could improve a fielder's jumps and rapid judgement of the ball in flight.
  
• Equipment change. Different glove, shoes, knee brace etc could affect performance.
  
• Defensive positioning. Moving your standing position in or out or left or right (possibly in response to scouting reports or spray charts) could result in more balls reached.
  
• Change in ballpark. Different fields have different dimensions, wind patterns, lighting, and shadows that could affect a player's fielding.
  
• Change in pitching staff. Results in different types of balls hit to you or the amount of balls hit to your zone.
  
• Different fielders in neighboring positions. Having a great fielder next to you could reduce the number of balls you can field. Having a weak fielder next to you can allow you to get some balls the other guy couldn't get.
  
• Conditioning or size fluctuations. If a player gains weight he loses speed and quickness. When he is in great shape he fields better.
  
• Body mechanics and footwork. If a player improves his mechanics when fielding a ball (possibly due to improved coaching) he can make fewer errors and stronger throws. If he gets lackadaisical and sloppy his performance could suffer.
  
• Concentration. If a player is focused, motivated and concentrating on defense he can play better than at times when he is distracted or doesn't care.

Just a quick list of reasons why a player's defensive performance and hence his defensive metrics could change over time without calling the defensive metric itself into question. I am not a huge believer in UZR and +/- and DRS yet, but I think they are strong steps in the right direction and should not be belittled just because they don't give you the same result every time.

[RedsManRick]

Ok, here's the data I promised. I changed things a little to save time. I'm looking at the Bat and Fld components of WAR for all qualified batters 2009-2012. If they're qualified on offense, they're in this sample. That means I do capture guys who were primary DHs. I should have pulled them, but I didn't Doing that would remove some of the small Fld values, but otherwise have no effect.




The takeaway is basically the same:

- For batting, 45% of the annual variances were within +/- 10 runs, with an average variance of 14 runs per season and a standard deviation of 17.3 runs.
- For fielding, 75% the annual variances were within +/- 10 runs, with an average variance of 7 runs and a standard deviation of 9.8 runs.

There's roughly twice as much variance in year-to-year batting runs than in year-to-year fielding runs.

Now, if you want to take out DH and C, which I wouldn't argue, the fielding variance would come up a little. But it certainly wouldn't jump up to the level of offense.

I know it feels weird to recognize that defense might possibly actually have the kind of significant yearly variance we've observed. But we should recognize, the yearly variances we've actually observed aren't actually all that big compared to what we see in offense. For every Jay Bruce outlier defensive season, there's an Aubrey Huff and a Aaron Hill who the same thing, but bigger, offensively

[edabbs44]

Great chart Rick, thanks. Definitely different than I expected. I think removing catchers and DHs would definitely change things but unsure how much. Are you able to do that or at least do you have a count of how many appear out of the 289 who may have limited innings in the field?

After looking at this more, I noticed something that maybe distorted my view but still looks to be pretty interesting.

Using the same time frame, here are the worst single year fielding metrics along with the following year:

Adam Dunn 2009-10: -37.1, -3.1
Mark Reynolds 2011-2012: -28.2, -8.3
Matt Kemp 2010-2011: -25.7, -4.6
Carlos Quentin 2010-2011: -24.3, 0.6
Jermaine Dye 2009: retired after 2009, 2008 was -19
Brad Hawpe 2009-10: -20, -3.3

Hawpe is really interesting as his 2010 year had limited innings in the field, however check out his UZR/150s for 2007-2010: -23.8, -44.7, -23.9, -12.2

Ibanez 2011-12: -18.9, 1.8
Granderson 2012: -18.2, no following year. 2011 was -5.3

Seems like most to all on the extreme side do not follow up with a similar season.

Here is the best single season performances with following year in the same time span:

2009-10 Franklin Gutierrez: 30.9, 6.6
2009-10 Nyjer Morgan: 27.6, 1.1
2010-11 Brett Gardner: 24.9, 25.8
2009-10 Ben Zobrist: 22.4, 11.2
2010-2012 Michael Bourn: 19.4, -6.4, 22.4 (widened to 3 years since he had multiple "best" seasons)
2010-2011 Andres Torres: 22, 9.4

What is your take on the extreme cases? Only one that we see on both sides did not follow up with less than a 10 run difference in the other direction, and in many cases experienced a more than 20 run difference.

Now looking at the offense side of things, here are the names of those who posted the best offensive seasons: Pujols, Hamilton, Bautista, Mauer, Votto, Cabrera, Braun, Fielder, etc. The worst? Pretty much what we expect with the exception of Alex Rios.

So while that chart shows more consistency generally on the defensive side, in extreme cases it seems like offense is more reliable.

What are your thoughts on that view?

[Brutus]

It should also be noted another reason a player's UZR could fluctuate... the greatest flaw in the metric: the fact that the "zones" are subjective.

It's one thing to draw out zones and pinpoint which ones players are expected to make plays based on observation, but it's another thing to watch a game and subjectively decide whether the ball is within that zone out there in a vast outfield with no real markers to guide the spotter.

Is UZR the best available defensive metric we have? Indeed. Does that mean it's accurate for quantifying actual runs scored/saved? Debatable.

[RedsManRick]

Great chart Rick, thanks. Definitely different than I expected. I think removing catchers and DHs would definitely change things but unsure how much. Are you able to do that or at least do you have a count of how many appear out of the 289 who may have limited innings in the field?

Yeah, I want to spend a little more time with it. Another thing to remember is that 1 outlier season creates two outlier differentials: the season before the outlier and the season after.
After looking at this more, I noticed something that maybe distorted my view but still looks to be pretty interesting.

Using the same time frame, here are the worst fielding metrics along with the following year:

Adam Dunn 2009-10: -37.1, -3.1
Mark Reynolds 2011-2012: -28.2, -8.3
Matt Kemp 2010-2011: -25.7, -4.6
Carlos Quentin 2010-2011: -24.3, 0.6
Jermaine Dye 2009: retired after 2009, 2008 was -19
Brad Hawpe 2009-10: -20, -3.3

Hawpe is really interesting as his 2010 year had limited innings in the field, however check out his UZR/150s for 2007-2010: -23.8, -44.7, -23.9, -12.2

Ibanez 2011-12: -18.9, 1.8
Granderson 2012: -18.2, no following year. 2011 was -5.3

Seems like most to all on the extreme side do not follow up with a similar season.

Here is the best single season performances with following year in the same time span:

2009-10 Franklin Gutierrez: 30.9, 6.6
2009-10 Nyjer Morgan: 27.6, 1.1
2010-11 Brett Gardner: 24.9, 25.8
2009-10 Ben Zobrist: 22.4, 11.2
2010-2012 Michael Bourn: 19.4, -6.4, 22.4 (widened to 3 years since he had multiple "best" seasons)
2010-2011 Andres Torres: 22, 9.4

What is your take on the extreme cases? Only one that we see on both sides did not follow up with less than a 10 run difference in the other direction, and in many cases experienced a more than 20 run difference.

Now looking at the offense side of things, here are the names of those who posted the best offensive seasons: Pujols, Hamilton, Bautista, Mauer, Votto, Cabrera, Braun, Fielder, etc. The worst? Pretty much what we expect with the exception of Alex Rios.

So while that chart shows more consistency generally on the offense side, in extreme cases it seems like offense is more reliable.

What are your thoughts on that view?

A few thoughts:

1) Because players have a smaller number of defensive opportunities than offensive one, "luck" plays a greater role in defense variance. That takes a number of forms including the actual composition and thus difficulty of opportunities -- not all OOZ plays are created equal. There is simply less time for the quality of opportunities to even out. This also means that if a player has a few good or bad plays, it has a larger proportionate impact on his season-end stats.

2) Compared to hitting, defense is much more physical ability than skill. This is evident in that guys peak defensively in their early-mid 20s. For hitting, players don't peak until later. This is because of three things:

1. Hitting is a more refined skill and has a much larger mental component, limiting the negative impact of minor physical injuries

2. Players gain power and wisdom as they age, which offsets loss of contact ability and fleet footedness. Because of the outsized importance of agility/speed for defense, the gain of wisdom has a much smaller benefit. And the gain of power that helps hitters often drives further decreases in speed.

3. Players are basically in a constant state of physical decline as soon as they hit the majors while in a constant state of mental improvement.

Consider the case of Votto last year after he came back from his knee surgery. He was able to offset his loss of power at the plate by drawing more walks. There was nothing he could do defensively to offset his loss of range.

It's also likely that positioning has a significant influence, somewhat akin to approach at the plate. John Dewan showed that one year the Pirates kept having Nate McClouth play further in in an attempt to offset his lessening range. However, McClouth was particularly bad about going back on the ball and the trade-off of fewer singles for more extra base hits was not worth it. Small strategies like that which might change from one season to the next could be more common for defense. I imagine those types of things are more common than a retool of a player's approach at the plate.

Another issue is this: you have much more ability adjust a defender's opportunities to take advantage of his skills. You can't do that for hitters. You can move a poor defender down the defensive spectrum where he may go from being below average to above average, more than offsetting the positional adjustment. This is particularly true if you're dealing with a guy who had a particular weakness you could protect, like moving an immobile LFer with good hands to 1B or a SS with good reactions and hands but poor range to 3B. And not only does the position require less/different physical ability as you move down the spectrum, but you also see fewer opportunities, further minimizing (or maximizing the impact of the switch).

You can't do this on offense. Despite Dusty's protestations, batting a guy 2nd doesn't take particular advantage of his skills compared to batting him 4th or 8th. Perhaps it does on the very margins as a guy tweaks his approach, but basically the challenge the batter faces is the same. If you have power, you'll hit for power no matter where you are in the lineup. And vice versa. You can tweak the order a bit to maximize the team-level outcomes that result from the interaction effects, but it doesn't change the production that any one player can provide. Votto won't hit more HR because Choo is at the top of the lineup. (yes, having men on base does have a small effect, not but nearly as big as people think).

Lastly, I think teams are much more hesitant to move a guy up the spectrum than down it. If your LFer has a great year defensively, it's unlikely you're going to move him to CF the next season (thus hurting his production when his performance regresses). However, if he has a bad year, it's more likely he'll get moved down the spectrum, providing him not just cover but the opportunity to provide more positive value when his performance regresses.

As for offense being more reliable, I'm not quite sure I understand your point.

[edabbs44]

As for offense being more reliable, I'm not quite sure I understand your point.

My point was that the best and worst offensive seasons are generally put up by people who are the best and worst offensive producers. In the period we are looking at, we don't really see someone going from the best offensive producer in the league to league average or worse YOY barring injury or other extenuating circumstances, of course.

[jojo]

My point was that the best and worst offensive seasons are generally put up by people who are the best and worst offensive producers. In the period we are looking at, we don't really see someone going from the best offensive producer in the league to league average or worse YOY barring injury or other extenuating circumstances, of course.

Using the same time frame, here are the worst fielding metrics along with the following year:

Adam Dunn 2009-10: -37.1, -3.1
Mark Reynolds 2011-2012: -28.2, -8.3
Matt Kemp 2010-2011: -25.7, -4.6
Carlos Quentin 2010-2011: -24.3, 0.6
Jermaine Dye 2009: retired after 2009, 2008 was -19
Brad Hawpe 2009-10: -20, -3.3

Hawpe is really interesting as his 2010 year had limited innings in the field, however check out his UZR/150s for 2007-2010: -23.8, -44.7, -23.9, -12.2

Ibanez 2011-12: -18.9, 1.8
Granderson 2012: -18.2, no following year. 2011 was -5.3

Seems like most to all on the extreme side do not follow up with a similar season.

Here is the best single season performances with following year in the same time span:

2009-10 Franklin Gutierrez: 30.9, 6.6
2009-10 Nyjer Morgan: 27.6, 1.1
2010-11 Brett Gardner: 24.9, 25.8
2009-10 Ben Zobrist: 22.4, 11.2
2010-2012 Michael Bourn: 19.4, -6.4, 22.4 (widened to 3 years since he had multiple "best" seasons)
2010-2011 Andres Torres: 22, 9.4

Gutierrez's 2009 season might be the best defensive centerfield season that I have ever witnessed and I've been blessed with having several outstanding defensive CFer's play for my favorite teams over the years. His health struggles over the last three seasons have been pretty well documented yet he still rates as a plus defender. Gardner, Torrez and Zobrist all followed up awesome years with great years. Bourne has had one inexplicable season in his career as a plus defender (the one you're picking on) and you're pointing to a season where Morgan stunk at everything to do with baseball.

In 2009-2010, Dunn went from splitting time between being really bad at an old position and really bad at a new, easier position to just being bad at a new easier position. Likewise Mark Reynolds made a similar transition to a less demanding position (from 3b to 1b) in the years you quote for him. Quentin, never a good defender, spent 2010 largely fighting through various injuries. In 2010, Hawpe was basically on his way out the door and barely played 500 defensive innings that season. Possibly sample size issues or just shear randomness helped him. Concerning Kemp, 2010 was a slumpy year for him all around and he seemed to be on ESPN every week making another silly baserunning goof. The guy wasn't right in the head that season. Did it effect his defense? Well he's not really a good defensive CFer to begin with and he couldn't seem to do much right in 2010.

Given your lists and looking at the devils in the details for each player, I don't really see a result that would make one say, "I just can't believe that"....

[RedsManRick]

My point was that the best and worst offensive seasons are generally put up by people who are the best and worst offensive producers. In the period we are looking at, we don't really see someone going from the best offensive producer in the league to league average or worse YOY barring injury or other extenuating circumstances, of course.

Sure. A big a part of that is sample size. In a half season of offense, which defense is somewhat comparable to, you see mediocre guys hit like stars and vice versa. That happens much less frequently over the course of a full season.

The other issue is that the talent curves are very different. Defensive ability is driven largely by simple athleticism. That's not to downplay the hard work and craft involved, but the pool of people who can play major league caliber defense is massive. I would argue that a majority of minor league position players can/have the potential to play major league caliber defense. And because there are a lot of awesome athletes and because a lot of defensive value comes from simply not screwing up, the spread of defensive ability between the very top end of the pool and possible replacements is much, much smaller than for offense.

Think about it this way. If you were trying to fill DH, replacing Jay Bruce. what's the pool of players who can possibly do what he does over 600 PA? 50 guys? Now, imagine you're trying to fill a DRF (Designated Right Fielder), replacing, Jay Bruce. The guy never has to hit. What's the size of that pool? 200? Remember to consider every minor leaguer and guys who couldn't hit enough to make the minors. It could be 5,000 or more.

The guys who are truly "elite" defenders simply aren't as much better as their possible replacements as is the case with hitters. Guys make the majors first and foremost because they can hit at a certain level. Even Brendan Ryan is a better hitter than most minor-leaguers. And the level of athleticism it takes to be a professional baseball hitter is such that of the guys who can hit, most are able to play major league defense.