UZR: One Year IS Enough

[RedsManRick]

I've tried (and largely failed) to articulate this position before, so I'm very pleased to see FanGraphs themselves do it much better than I could.

http://www.fangraphs.com/blogs/index...sition-on-uzr/

And here's the ESPN article showing how PBP defensive metrics work in the context of Curtis Granderson: http://espn.go.com/blog/new-york/yan...sily-evaluated

That’s right — in fact, it’s so right I’ll write again, embolden’d even: One year of UZR data is enough.

The proper question, however, is, “Enough for what?”

A single season of UZR is a point along the spectrum of true talent, but the error range for that single point is so large as to mean almost nothing. So yes, one season of UZR is not enough to determine a players true talent level.

Bronzing a player’s glove after a single season would be as foolish as enshrining a hitter after just 400 plate appearances.

HOWEVER! That is only when we are talking about true talent levels. Many people who use UZR suppose incorrectly that since a single season does not accurately report true talent levels we can effectively ignore that season until there’s a wealth of data.

But just because a single season does not tell us a player’s true talent, it does not mean that year’s UZR tells us nothing. To the contrary, the data shows us the story of the season, it describes the season precisely.

But definitely read the whole article.

Really, it's pretty intuitive. If a guy hits .400 for 200 PA, we don't want to call him a ".400 hitter". But he really did get hits in 40% of his at bats. That's not to say UZR is perfect, obviously. But we should be considering our question before dismissing it due to small sample size. Small samples are a problem if you want to use them to estimate the whole population -- but as measures of that specific thing, they can be perfectly fine.

To be fair, the author probably overstates the case slightly. UZR is still an estimation, a translation of what happened, not a direct measurement -- unlike, say, batting average, which just counts events and does not place differential estimated value on them.

Lastly, I would just say, be thoughtful about your own perceptions. Human memory is a funny thing. What we choose (subconsciously) to flag as memorable is based in large degree on our existing beliefs and understandings. And when we do reflect, we choose a narrative first and then create the images. In short, our memories aren't good at systematically recalling and summarizing series' of events over time. So when we try to judge a guy's defense, in the absence of data we immerse ourselves in that shapes the narrative (like we do with OPS), we're likely defaulting back to some big-picture summary of ability rather than actual performance over the last year. Useful for getting a reasonable estimate of true-talent, not so useful for a given year. If the data conflicts with our perception, we should consider it's our perceptions that are unreliable.

[jojo]

I've been beating this drum for a while...

If one really wants to be comfortable estimating a player's true talent level, roughly 3000 defensive innings (or approx 3 yrs) seems to be the appropriate threshold.

That's a different issue than simply asking what defensive value was provided over a season (i.e. given the distribution of chances, how did a player perform relative to his peers), which may or may not be repeatable oe reasonable to expect going forward.

[pahster]

Lastly, I would just say, be thoughtful about your own perceptions. Human memory is a funny thing. What we choose (subconsciously) to flag as memorable is based in large degree on our existing beliefs and understandings. And when we do reflect, we choose a narrative first and then create the images. In short, our memories aren't good at systematically recalling and summarizing series' of events over time. So when we try to judge a guy's defense, in the absence of data we immerse ourselves in that shapes the narrative (like we do with OPS), we're likely to defaulting back to some big-picture summary of ability rather than actual performance over the last year.

Very true. We're all biased information processors.

[jojo]

Lastly, I would just say, be thoughtful about your own perceptions. Human memory is a funny thing. What we choose (subconsciously) to flag as memorable is based in large degree on our existing beliefs and understandings. And when we do reflect, we choose a narrative first and then create the images. In short, our memories aren't good at systematically recalling and summarizing series' of events over time. So when we try to judge a guy's defense, in the absence of data we immerse ourselves in that shapes the narrative (like we do with OPS), we're likely to defaulting back to some big-picture summary of ability rather than actual performance over the last year.

Determining defensive value of a guy using just our eyes and memory? Ha! We're not even good enough to reliably pick the guy out of a lineup.

[Captain Hook]

I've been beating this drum for a while...

Three years seems like a long time.I would think injuries and age could really throw the accuracy off if your looking to project what you could expect from a player.I admit that I don't know a lot about UZR so maybe it takes these things into account somehow but I like the one year idea better.I can't imagine too many people disagreeing if after a year, UZR says someone is a great defended and what they've actually seen with their eyes suggest the player is a great defender.So I would agree that if you were going by stats alone then maybe the one year idea isn't the best way to go but who really makes a decision based on stats alone?

BTW the link to the article isn't working for me.

[lollipopcurve]

This sounds to me like a year of UZR is enough to tell you what someone's UZR for the year was.

Not sure I see much value here. Until they start measuring ball flight and speed, and defender's routes and speed (plus throw times), count me a major skeptic on the defensive metrics for individuals.

[blumj]

That fangraphs article seems to have vanished, the espn one was really interesting, though, thanks for the link.

[RedsManRick]

This sounds to me like a year of UZR is enough to tell you what someone's UZR for the year was.

Not sure I see much value here. Until they start measuring ball flight and speed, and defender's routes and speed (plus throw times), count me a major skeptic on the defensive metrics for individuals.

UZR is pretty simple. How many times did a guy make an out compared to how often his peers did in a similar situation. Is the definition of those situations perfect? Of course not.

But what's your alternative? Ignoring defense entirely? If not, you're doing something to account for it -- what is that process and how is it better than UZR?

[lollipopcurve]

But what's your alternative? Ignoring defense entirely? If not, you're doing something to account for it -- what is that process and how is it better than UZR?

I'll take scouting info in this case. UZR throws out some wild numbers for single years -- too easy to get bamboozled by it.

[RedsManRick]

I'll take scouting info in this case. UZR throws out some wild numbers for single years -- too easy to get bamboozled by it.

Sure, UZR throws out wild numbers for single years. But maybe that's because over a single year, players can have pretty extreme defensive performances -- just like a hitter can have pretty extreme offensive performances over a few hundred PA. Again, I think we're conflating performance measurement with true talent estimation. Just like a .900 OPS hitter can put up a 1.200 OPS over a few hundred PA, a +7 runs fielder could put up a +20 over a season. It would be interesting if we took samples of 200 PA, converted them in to runs above replacement and looked at the distribution. Imagine Jay Bruce's. He'd swing from 0 to 20 and then we could question the validity of that too.

Just curious, but how do you use the scouting info? Whose scouting info? Is that scout talking about a player's overall ability or his performance in the last year?

For all we know, a given scout is likely to be more off-base about a given player in a given year than UZR. We just don't have a convenient way to sort through his assessments and judge them. Just because the biases may not be as obvious doesn't mean they don't exist.

We treat scouting data like it's a safe default, but particularly when it comes to measuring performance over a discrete time frame and not assessing overall ability, how do we know that scouts are better than the stats? I'm not saying the stats are better, but I think it's a fair question.

I would also add, we have to remember the scale differences. UZR, as it's presented, is relative to average at that position, not total contribution. THis is really different than the way were used to looking at stats. We're used to making adjustments in our head -- not having to make them is disorienting, especially when it result in counterintuitive findings such as "A SS with negative UZR can still be a better defender than a RF with positive UZR".

It's just really challenging to accept that on offense, average is notable above replacement level whereas defense, average is at replacement level, and once you adjust for position, can even be below replacement. That is to say, the average LF is a worse defender than a generic replacement level defender freely available in the minors or FA. That doesn't seem too odd when it's written out, but when it's in number form, it just looks wrong.

[lollipopcurve]

We treat scouting data like it's a safe default, but particularly when it comes to measuring performance over a discrete time frame and not assessing overall ability, how do we know that scouts are better than the stats?

I'm not saying the stats are better, but I think it's a fair question.

Give me a seasoned eye who watches a guy play 20 games in the OF, combined with whatever video they can compile, and that's plenty to assess whether a guy is above average, average or below average. Not rocket science.

[dougdirt]

Give me a seasoned eye who watches a guy play 20 games in the OF, combined with whatever video they can compile, and that's plenty to assess whether a guy is above average, average or below average. Not rocket science.

The Baseball Info Solutions guys watch every play of the year for The Fielding Bible awards/ +/- system for Dewan's book. They then categorize each play with ball trajectory and landing location. The +/- system usually agrees with UZR. Yes, there are some faults in there, but there are also going to be faults/differences among scouts too. The numbers aren't perfect because no defensive play is created equally. But that is the same way for offense. Joey Votto doesn't get the same pitches to hit as Edgar Renteria does. But we treat their 600 PA's in a season in the exact same light in terms of hits/outs/bases don't we?

[lollipopcurve]

But that is the same way for offense. Joey Votto doesn't get the same pitches to hit as Edgar Renteria does. But we treat their 600 PA's in a season in the exact same light in terms of hits/outs/bases don't we?

The number of pitches a hitter sees over the course of a season is way higher than the number of chances a defender gets. That's why a single season of data for hitters is more trustworthy than a single season of defensive data. What we're debating here is whether or not a single season of UZR data is worth all that much, and I'm saying it's not. It's not without value -- it's just got too much volatility year to year for my liking.

Keep in mind that the pitches a Votto chooses to swing at and the pitches a lesser hitter like Renteria chooses to swing at are quite a bit more similar than the two entire cohorts of pitches seen. In general, those guys are swinging at "good pitches to hit."

[IslandRed]

My take... The advanced metrics are, or will be, important for parsing the distinctions between fielders at a position and helping teams evaluate the players accordingly.

On the other hand, for over 100 years, the unaided eye was sufficient to get players into their proper slot on the defensive spectrum.

I think part of it is just a matter of the probabilities involved. It's difficult to distinguish a .250 hitter from a .300 hitter without paying attention to the stats over a large sample. But a defender is supposed to make the play nearly 100% of the time, so a truly deficient defender is not that hard to identify. It's at the margins -- calculating the difference in value between two shortstops, or deciding if a borderline shortstop needs to be moved across to second base -- where the stats can be really helpful.

[Scrap Irony]

If the Reds buy this, it's probably good news for Chris Heisey as a CF. Both he and Drew Stubbs grade out as 4.5-ish defensive players, meaning the loss of Stubbs would be negligible from a glove standpoint.