PDA

View Full Version : Do we really consider defense?



RedsManRick
09-28-2007, 12:11 PM
As the end of year award balloting heats up, it's become increasingly clear to me that the lack of a standard method of defensive valuation is really skewing our notion of value.

Ryan Braun has been a monster at the plate... and disastrous in the field. You can virtually take your pick of stats and he's the worst 3B in baseball defensively.

Prince Fielder is destroying the ball and yet is one of the worst 1B in baseball.

So, as people go about evaluating players, how they account for this? Do they come up with some basic offensive value and some defensive value, add the two together, and then see where people stand? I would suggest they do not.

Rather, I think that most people look at the offensive numbers, decide where everybody stands, and then move people up or down a little bit based on a qualitative estimation of defensive ability. A little extra credit here, a tie-breaker there. In doing so, I think people are severely discounting the effect of defense.

(Big props to Justin Inaz and his superb Reds blog: http://jinaz-reds.blogspot.com)

I won't go through the whole exercise, because Justin did so well, but here's the most blatant example, using THTs +/- Runs metric. (yes, no defensive metric is perfect, and occasionally they even disagree -- this is one I've found to be extremely consistent with scouting and other accepted metrics and is logically sound)

NL MVP Race:
3B Wright: +27.6 runs
1B Fielder: -9.0 runs
2B Utley: +20.5 runs
SS Rolllins: -2.4 runs
SS Ramirez: -22.2 runns
RF Holliday: +8.2 runs

Now, even if we were to assume that these guys were equal offensively (which they are not) and have generated 90 runs each with their bats, look at the effect of defense. There's a net 50 run swing between Wright and H-Ram. It's not even close. Wright gets a 30% boost to his value, whereas Ramirez cuts his by 25%. Wright becomes arguably the best player in baseball and Hanley goes from the 2nd most productive player in baseball, to just pretty good. Crazy huh?

Look at the NL ROY:
3B Braun: -30.6
SS Tulowitski: +27.9
CF Pence: +4.7

If you take a wholistic view of offense and defensive truly added together on the same scale, Bruan loses half his value. Now, if Braun had hit .270/.350/.475, 18 HR with decent defense, would anybody be talking about him as ROY? By Justin's methodology, Braun becomes just marginally more productive than fellow rookie Kevin Kouzmanoff, who is on nobody's radar. Add in some park factors and he might even be worse.

I know the defensive metric debate continues and by no means do I think we have "the" answer. However, I think we should be careful about giving defense it's due. Not having a perfect metric is no reason for not trying to incorporate defense in to our valuation process in some quantitative fashion.

At the risk of starting another "Dunn thread", if Dunner hit just .250/.340/.475 and played a solid LF, would you still want to build the team around him? Some guys who are like that? Ryan Church and Hideki Matsui. Raul Ibanez fits the offensive profile. Using THTs +/- runs, Eric Byrnes was worth 40 runs more than Dunn defensively this year. Even taking away his 50 SB, he might have been more valuable. I think we have a tendency to give lip service to defense, without really considering it's impact. We talk up Dunn's .900+ OPS, but don't think of how much his defense drags that down in terms of overall value.

By no means am I'm saying give me a team of John McDonalds and Doug Mientkiewiczs. However, let's honestly ask ourselves, how much of our run prevention problem is really a function of pitching, and how many runs are we giving away by fielding sub-par defenders? It might be more than you think.

BoydsOfSummer
09-28-2007, 12:19 PM
Wow, is Hanley Ramirez really that bad? I haven't seen him play enough to notice, but that surprises me.

dougdirt
09-28-2007, 12:31 PM
I have gone through Justins defensive stuff, and while I have an untrained eye, it seems to be fairly accurate to me. I know of some others who simply think its insane to suggest what his numbers actually suggest. I had an argument of Curtis Granderson versus Grady Sizemore and who was more valuable. I quoted Justins numbers and Granderson is a +32.2 runs defender, while Sizemore is -19.9, which is a difference in over 50 runs on defensive value. Is the gap truly that big? I don't know, but even if we cut that number in half, its pretty much making a HUGE difference in the value of a player.

As far as Hanley Ramirez on defense, only Ramirez and Jeter are worse than -19 runs at shortstop in baseball.

jojo
09-28-2007, 12:31 PM
As the end of year award balloting heats up, it's become increasingly clear to me that the lack of a standard method of defensive valuation is really skewing our notion of value.

Ryan Braun has been a monster at the plate... and disastrous in the field. You can virtually take your pick of stats and he's the worst 3B in baseball defensively.

Prince Fielder is destroying the ball and yet is one of the worst 1B in baseball.

So, as people go about evaluating players, how they account for this? Do they come up with some basic offensive value and some defensive value, add the two together, and then see where people stand? I would suggest they do not.

Rather, I think that most people look at the offensive numbers, decide where everybody stands, and then move people up or down a little bit based on a qualitative estimation. In doing so, I think people are severely discounting the effect of defense. Let's take that extreme case of defense.

(Big Props to Justin Inaz and his superb Reds blog: http://jinaz-reds.blogspot.com)

I won't go through the whole exercise, because Justin did so well, but here's the most blatant example, using THTs +/- Runs metric. (yes, no defensive metric is perfect, and occasionally they even disagree -- this in one I've found to be extremely consistent with scouting and other accepted metrics)

NL MVP Race:
3B Wright: +27.6 runs
1B Fielder: -9.0 runs
2B Utley: +20.5 runs
SS Rolllins: -2.4 runs
SS Ramirez: -22.2
RF Holliday: +8.2 runs

Now, even if we were to assume that these guys were equal offensively (which they are not) and have generated 90 runs each with their bats, look at the effect of defense. There's a net 50 run swing between Wright and H-Ram. It's not even close. Wright gets a 30% boost to his value, whereas Ramirez cuts his by 25. Wright becomes arguably the best player in baseball and Hanley becomes mediocre. Crazy huh?

Look at the NL ROY:
3B Braun: -30.6
SS Tulowitski: +27.9
CF Pence: +4.7

If you take a wholistic view of offense and defensive truly added together on the same scale, Bruan loses half his value. Now, if Braun had hit .270/.350/.475, 18 HR with decent defense, would anybody be talking about him as ROY?

I know the defensive metric debate continues and by no means do I think we have "the" answer. However, I think we should be careful about giving defense it's due. Not having a perfect metric is no reason for not trying to incorporate defense in to our valuation process in some quantitative fashion.

At the risk of starting another "Dunn thread", if Dunner hit just .250/.340/.475 and played a solid LF, would you still want to build the team around him? Some guys who are like that? Ryan Church and Hideki Matsui. Raul Ibanez fits the offensive profile. Using THTs +/- runs, Eric Byrnes was worth 40 runs more than Dunn defensively this year. Even taking away his 50 SB, he might have been more valuable. I think we have a tendency to give lip service to defense, without really considering it's impact. We talk up Dunn's .900+ OPS, but don't think of how much his defense drags that down in terms of overall value.

By no means am I'm saying give me a team of John McDonalds and Doug Mientkiewiczs. However, let's honestly ask ourselves, how much of our run prevention problem is really a function of pitching, and how many runs are we giving away by fielding sub-par defenders? It might be more than you think.

This post is very http://www.freesmileys.org/emo/sport017.gif (http://www.freesmileys.org)

One day I'm goiing to buy you a beer.

Chip R
09-28-2007, 12:34 PM
The better you do offensively, the more your defense is overlooked and vice versa.

dougdirt
09-28-2007, 12:45 PM
The better you do offensively, the more your defense is overlooked and vice versa.

Right, but I think what RMR is trying to say is that it shouldn't be that way. Despite how good some players are offensively at their position (Derek Jeter for example), their defense negates so much of it that its incredible and makes the player almost barely above replacement level.

Chip R
09-28-2007, 12:51 PM
Right, but I think what RMR is trying to say is that it shouldn't be that way. Despite how good some players are offensively at their position (Derek Jeter for example), their defense negates so much of it that its incredible and makes the player almost barely above replacement level.


I agree but that's the way it is. Dunn's a perfect example. I'm not sure if his defense is any better than it was last year but he's hitting for a higher average and striking out less so people are not talking as negatively about his defense as they did last year or earlier in the season.

TeamBoone
09-28-2007, 12:58 PM
But Dunn's defense has improved; in fact it's the best it's been in several seasons... .976 with six errors. He actually isn't as bad as he's made out to be, while others are made out to be better than they actually are.

RedsManRick
09-28-2007, 01:01 PM
Right, but I think what RMR is trying to say is that it shouldn't be that way. Despite how good some players are offensively at their position (Derek Jeter for example), their defense negates so much of it that its incredible and makes the player almost barely above replacement level.


I agree but that's the way it is. Dunn's a perfect example. I'm not sure if his defense is any better than it was last year but he's hitting for a higher average and striking out less so people are not talking as negatively about his defense as they did last year or earlier in the season.

The thing is, you are both right. We'll take that league average offensive player, and rightly recognize that his poor defense basically turns him in to a replacement level contributor. However, when it comes to the star level players, we often don't go through the same calculus. We aren't' willing to take that step that says Derek Jeter was actually about as productive as Alex Gonzalez in 2007...

The problem is one of both reality (failure of teams to act on this) and perception, as evidenced in award balloting and discussion.

IslandRed
09-28-2007, 01:03 PM
I have gone through Justins defensive stuff, and while I have an untrained eye, it seems to be fairly accurate to me. I know of some others who simply think its insane to suggest what his numbers actually suggest. I had an argument of Curtis Granderson versus Grady Sizemore and who was more valuable. I quoted Justins numbers and Granderson is a +32.2 runs defender, while Sizemore is -19.9, which is a difference in over 50 runs on defensive value. Is the gap truly that big? I don't know, but even if we cut that number in half, its pretty much making a HUGE difference in the value of a player.

I wonder if some metrics overstate the case a bit. Let's take David Wright and Ryan Braun. According to those numbers above, Wright is nearly 60 runs better defensively than Braun. Think about that for a second. For a league-median team that allows about 750 runs over a season, they're implying that Braun vs. Wright could explain 8% of a team's runs allowed. Out of every 12.5 runs that would cross the dish, one of them could be laid at the feet of the lousy third baseman. That just doesn't intuitively compute.

I'm sure the state of the art has advanced, but back in Moneyball, the A's derivative-based system that did everything possible to strip out luck decided that 15 runs separated Johnny Damon, a pretty good center fielder at the time, from Terrence Long, a truly execrable one. For what it's worth, I think that's too low. But systems that routinely produce 50-60 run spreads between the best and worst are probably too high, in my opinion. I don't think there are that many outliers in this day and age.

Chip R
09-28-2007, 01:14 PM
But Dunn's defense has improved; in fact it's the best it's been in several seasons... .976 with six errors. He actually isn't as bad as he's made out to be, while others are made out to be better than they actually are.


He's not going to make anyone forget Willie Mays out there - or Willie Mays Hayes, for that matter - but while he may be a little better than he was last year, he's still below average.

But that's the point we're making. Just like the Gold Glove is not always awarded to the best defensive player, guys who hit very well get a break on their defense. There may be a few fans in BOS who want to get rid of Manny Ramirez because his OF defense is not good but most take a whimsical approach to it because he hits very, very well. If Manny were an average or just an above average hitter he would receive more flack for his defense than he does now. It's why Juan Castro gets a pass on being a poor offensive player. He has a reputation as being an outstanding defensive player that his lack of contribution offensively is overlooked.

It isn't right but it's human nature.

RedsManRick
09-28-2007, 01:14 PM
But Dunn's defense has improved; in fact it's the best it's been in several seasons... .976 with six errors. He actually isn't as bad as he's made out to be, while others are made out to be better than they actually are.

It depends on how you measure it. Using the aforementioned +/- Runs system, Dunn clocked in at -18.1 runs. Of the 45 guys with 300+ innings in LF, only Pat Burrell and Manny Ramirez were worse. Better than he was in the past? Maybe. Good? Hardly.

When it comes to OF, errors are far less important than range. You can drop a flyball and it turns a would-be out in to a double or you can come up a step short and it turns a would-be out in to a double. Guess which one is more frequent? Guess which one counts as an error? Heck, how many times does a ball drop 5 feet ahead of him that a better fielder would've been camped under. That sort of play barely touches our radar, but it adds up.

FLD% is like batting average. It counts the freuqency of some thing happen. Unfortunately, that thing is not inclusive of all the important events and is in no way weighted for the impact.

FWIW, Griffey was 2nd worst in RF and Hamilton was merely below average in CF, whereas Hopper was average, and Freel above average.

And IR, I agree, the scale of that metric seems a bit stretched out to me too. But even if you cut it in half, there are some very real adjustments that need to be made to our perception of overall value. And Chip is really refinforcing the point that I most wanted to make; defense is simply ignored way too often. We have this ambivalent approach where we sometime count defense and sometimes don't.

You'll often get this argument "Player X is a superb offensive player whose offensive value is somewhat offset by his poor defense. But, all told, he's still a great player". In reality it should be, "Player X is a superb offensive player whose offensive value is somewhat offset by his poor defense. Therefore, we shouldn't consider him amongst the best players."

It's:
Great Offense + Poor Defense = Great Player

When it should be:
Great Offense + Poor Defense = Pretty Good Player

Good luck finding somebody who will say "All things considered Derek Jeter is an average SS" or "Manny Ramirez is an average LF".

dougdirt
09-28-2007, 01:16 PM
I wonder if some metrics overstate the case a bit. Let's take David Wright and Ryan Braun. According to those numbers above, Wright is nearly 60 runs better defensively than Braun. Think about that for a second. For a league-median team that allows about 750 runs over a season, they're implying that Braun vs. Wright could explain 8% of a team's runs allowed. Out of every 12.5 runs that would cross the dish, one of them could be laid at the feet of the lousy third baseman. That just doesn't intuitively compute.

I'm sure the state of the art has advanced, but back in Moneyball, the A's derivative-based system that did everything possible to strip out luck decided that 15 runs separated Johnny Damon, a pretty good center fielder at the time, from Terrence Long, a truly execrable one. For what it's worth, I think that's too low. But systems that routinely produce 50-60 run spreads between the best and worst are probably too high, in my opinion. I don't think there are that many outliers in this day and age.

I think its all a matter of position. What percentage of baseballs in a third basemans zone are going for doubles down the line? Same thing with outfielders.... how many balls that they don't get to are turning into extra base hits. It doesn't take many XBH to turn into a few runs. For shortstops and second baseman, balls they don't get to seem to turn into singles almost every time, and then likewise, the plays they don't make, in theory anyways, are not hurting as bad... of course, they get so many more chances than the other positions, it all evens out probably.

While I am not sure there is a 60 run difference between Wright and Braun, even if we divide that in half, its still a huge difference in value between the two players.

dougdirt
09-28-2007, 01:19 PM
Hamilton was merely below average in CF, whereas Hopper was average, and Freel above average.
Funny you bring that up, I noticed that there were three CFers over 30, then the next closest was around 15 (going off the top of my head here). I found that to be very interesting that 3 guys were just so much better than the rest of the guys in baseball that they almost doubled the 4th best player at such an important position.

SteelSD
09-28-2007, 01:23 PM
I won't go through the whole exercise, because Justin did so well, but here's the most blatant example, using THTs +/- Runs metric. (yes, no defensive metric is perfect, and occasionally they even disagree -- this is one I've found to be extremely consistent with scouting and other accepted metrics and is logically sound)

Except that the +/- Runs when applied to defense isn't actually logically sound. There's no way on Earth that Ryan Braun is worth negative 30 Runs defensively. Here's why:

The expected Plays/BIZ for a third baseman is .706. According to THT's numbers (which I do not trust), Braun has seen 219 Balls in Zone this season and has made 123 "Plays". That means Braun is 32 Plays short of the .706 standard. Unless every one of those Plays projects to be an inside the park Home Run, there simply isn't enough opportunity for Braun to be worth -30+ Runs defensively; and that's even if we factor in that he appears to reach fewer OOZ balls than your average 3B (@-12 OOZ Plays).

Even if we trust THT's wonky BIZ/OOZ data, we're looking at a total of 44 missing plays by Braun versus expected results. And those plays are allegedly good for around minus 30 defensive Runs? Sorry, but that's quite literally impossible.

RedsManRick
09-28-2007, 01:29 PM
Except that the +/- Runs when applied to defense isn't actually logically sound. There's no way on Earth that Ryan Braun is worth negative 30 Runs defensively. Here's why:

The expected Plays/BIZ for a third baseman is .706. According to THT's numbers (which I do not trust), Braun has seen 219 Balls in Zone this season and has made 123 "Plays". That means Braun is 32 Plays short of the .706 standard. Unless every one of those Plays projects to be an inside the park Home Run, there simply isn't enough opportunity for Braun to be worth -30+ Runs defensively; and that's even if we factor in that he appears to reach fewer OOZ balls than your average 3B (@-12 OOZ Plays).

Even if we trust THT's wonky BIZ/OOZ data, we're looking at a total of 44 missing plays by Braun versus expected results. And those plays are allegedly good for around minus 30 defensive Runs? Sorry, but that's quite literally impossible.

What's the run value of a missed play by a 3B? I'd be very curious to go through the exercise of each missed play and look at the sum run expectancy effect.

I agree Steel. I do think the +/- Runs number is too big. But how big? I absolutely think it needs to be refined. But until I see a metric I can use with greater confidence, I'm going to run with it -- all caveats on the table.

Simply shooting things down because they aren't perfect doesn't get us anywhere.

Just a horribly rough estimate for illustrative purposes only, let's use the following values.

Singles are worth .25 runs, Doubles .50, and triples .75. Missed plays by 3B turn in to singles 60% of the time, doubles 30% of the time, and triples 10% of the time.

Braun missed 44 plays: 27 became singles, 13 doubles, and 4 triples.

24*.25 + 13*.50 + 4*.75 = 16 runs allowed. That's about half of what the +/- Runs system suggests.

There are all kinds of variables, specifically including base/out situations. But in any case, even if the difference between Wright and Braun is 25 runs, that's a pretty darn big chunk of production for 1 guy at 1 position.

flyer85
09-28-2007, 01:33 PM
defense is just a part of the equation. A player at a position who is a big plus both with the bat and the glove is a rare commodity.

pedro
09-28-2007, 01:33 PM
My beef is that someone who makes every single play, isn't average, they are a robot, and it seems to me that the +/- numbers at the periphery are more likely to be the differential between the best and the worst rather than the the differential between either end and "average".

PuffyPig
09-28-2007, 01:37 PM
But Dunn's defense has improved; in fact it's the best it's been in several seasons... .976 with six errors.

He looks better than before, but citing someone's fielding average of proof he's better just doesn't cut it.

SteelSD
09-28-2007, 01:40 PM
What's the run value of a missed play by a 3B? I'd be very curious to go through the exercise of each missed play and look at the sum run expectancy effect.

I agree Steel. I do think the +/- Runs number is too big. But how big? I absolutely think it needs to be refined. But until I see a metric I can use with greater confidence, I'm going to run with it -- all caveats on the table.

Simply shooting things down because they aren't perfect doesn't get us anywhere.

It's not that +/- isn't "perfect", Rick. I'm not shooting it down because of a small flaw. I'm telling you that it's completely and utterly unreliable as a tool to guage actual run value. It may have some use in showing us who might be good or bad in an abstract way (similar to Zone Rating), but the Run values assigned to Ryan Braun can't happen. There just isn't a way that 32 missed BIZ and 12 missed OOZ plays can turn into 30 Runs.

flyer85
09-28-2007, 01:40 PM
But systems that routinely produce 50-60 run spreads between the best and worst are probably too high, in my opinion. .especially at a position like 3b where a defender gets 2-3 balls a game. The fact that Wright, who has never been considered a top notch defensive 3b, is somehow 60 runs better than Braun(who is bad) just doesn't pass the smell test. I really don't like the defensive metrics that try to translate it into runs, it seems like a futile exercise unless you use PBP data to see what actually happened after a play was not made that should have been made, or vice-versa.

M2
09-28-2007, 01:42 PM
Except that the +/- Runs when applied to defense isn't actually logically sound. There's no way on Earth that Ryan Braun is worth negative 30 Runs defensively. Here's why:

The expected Plays/BIZ for a third baseman is .706. According to THT's numbers (which I do not trust), Braun has seen 219 Balls in Zone this season and has made 123 "Plays". That means Braun is 32 Plays short of the .706 standard. Unless every one of those Plays projects to be an inside the park Home Run, there simply isn't enough opportunity for Braun to be worth -30+ Runs defensively; and that's even if we factor in that he appears to reach fewer OOZ balls than your average 3B (@-12 OOZ Plays).

Even if we trust THT's wonky BIZ/OOZ data, we're looking at a total of 44 missing plays by Braun versus expected results. And those plays are allegedly good for around minus 30 defensive Runs? Sorry, but that's quite literally impossible.

Exactly.

I noticed something else in the ratings. Vernon Wells, Grady Sizemore, Melky Cabrera, Torii Hunter, David DeJesus and Dave Roberts rank down at the bottom of the CF list. Wells is supposedly as bad a CF as Ken Griffey Jr. was in recent years. My question is can the baseball industry be that wrong about player skills? We're talking about six well to highly regarded CFs here. This isn't a case of the industry overvaluing a skill (like contact) and undervaluing another (like OB). This would be a case of the industry being completely wrong about a skill - seeing it where it isn't. Or is the +/- system missing something fundamental in its ratings?

The other thing is the wiggle in these ratings is tremendous. Wright goes from stooge to genius. Wells goes from genius to stooge. Did that really happen? Could that really happen? Just thinking out loud here, but maybe what these systems is showing us is that defense is more random than we thought. If so, then we're measuring the randomness and not the skill. Also, if it's more random than we thought, what's causing the randomness? Sheer physics? Weather/field conditions? Pitching quality? Hitting quality?

PuffyPig
09-28-2007, 01:43 PM
Except that the +/- Runs when applied to defense isn't actually logically sound. There's no way on Earth that Ryan Braun is worth negative 30 Runs defensively. Here's why:

The expected Plays/BIZ for a third baseman is .706. According to THT's numbers (which I do not trust), Braun has seen 219 Balls in Zone this season and has made 123 "Plays". That means Braun is 32 Plays short of the .706 standard. Unless every one of those Plays projects to be an inside the park Home Run, there simply isn't enough opportunity for Braun to be worth -30+ Runs defensively; and that's even if we factor in that he appears to reach fewer OOZ balls than your average 3B (@-12 OOZ Plays).



Even if we trust THT's wonky BIZ/OOZ data, we're looking at a total of 44 missing plays by Braun versus expected results. And those plays are allegedly good for around minus 30 defensive Runs? Sorry, but that's quite literally impossible.

It's not impossible if you consider that a single which prolongs an inning (as opposed to it being the third out) could result in 5 runs being scored. It's not just the run that the hitter scores, it's all the resulting runs too.

flyer85
09-28-2007, 01:46 PM
It's not impossible if you consider that a single which prolongs an inning (as opposed to it being the third out) could result in 5 runs being scored. It's not just the run that the hitter scores, it's all the resulting runs too.what if not making a play doesn't result in any runs being scored?

RedsManRick
09-28-2007, 01:53 PM
It's not that +/- isn't "perfect", Rick. I'm not shooting it down because of a small flaw. I'm telling you that it's completely and utterly unreliable as a tool to guage actual run value. It may have some use in showing us who might be good or bad in an abstract way (similar to Zone Rating), but the Run values assigned to Ryan Braun can't happen. There just isn't a way that 32 missed BIZ and 12 missed OOZ plays can turn into 30 Runs.

And I'm agreeing with you that the assigned run values are out of whack. I'm not beholden to any given metric. Please show me a publicly available metric with run values that are more reasonable and I'll happily switch. It may very well be out there, I just haven't come across one that's easily accessible.

And, even so, I don't think my general point is invalidated.

flyer85
09-28-2007, 01:57 PM
And, even so, I don't think my general point is invalidated.I think the only thing that can be established with any certitude is that Ryan Braun is not a major league 3b.

pedro
09-28-2007, 01:57 PM
And I'm agreeing with you that the assigned run values are out of whack. I'm not beholden to any given metric. Please show me a publicly available metric with run values that are more reasonable and I'll happily switch. It may very well be out there, I just haven't come across one that's easily accessible.

And, even so, I don't think my general point is invalidated.

I have a problem with the defensive run values when they are then added to the offensive run value, which are known to be much more accurate, and then used as a method of evaluating a players total value in a ranking. If the defensive run values are not accurate, then they shouldn't be used this way b/c it just creates misinformation IMO.

Johnny Footstool
09-28-2007, 02:00 PM
Except that the +/- Runs when applied to defense isn't actually logically sound. There's no way on Earth that Ryan Braun is worth negative 30 Runs defensively. Here's why:

The expected Plays/BIZ for a third baseman is .706. According to THT's numbers (which I do not trust), Braun has seen 219 Balls in Zone this season and has made 123 "Plays". That means Braun is 32 Plays short of the .706 standard. Unless every one of those Plays projects to be an inside the park Home Run, there simply isn't enough opportunity for Braun to be worth -30+ Runs defensively; and that's even if we factor in that he appears to reach fewer OOZ balls than your average 3B (@-12 OOZ Plays).

Even if we trust THT's wonky BIZ/OOZ data, we're looking at a total of 44 missing plays by Braun versus expected results. And those plays are allegedly good for around minus 30 defensive Runs? Sorry, but that's quite literally impossible.

I agree that the run value assigned to missed plays is skewed. Even if we assume the *extremely simplified* view that every hit is worth .52 of a run (based on the MLB average of Runs/Hits), the 44 missed plays assigned to Braun only generates 23 runs.

If we remove HR from the equation (since Braun's missed plays didn't result in HRs) and assign non-HR hits a value of about .46 runs (MLB average of Runs-HR/Hits-HR), 44 missed plays only generates 20 runs.

I assume a more scientific study of the missed plays would most likely result in a reduced Run/Hit ratio and fewer negative runs assigned to poor fielding.

flyer85
09-28-2007, 02:08 PM
I would never use the defensive metrics currently to draw anything other than general conclusions ... such as Zimmerman-good and Braun-bad. The point anyway is what do you do with the information? I would think you will see the Brewers move Braun to the OF because his bat will play at any position. The Reds defense needs to get better and I for one think the continued maturation of EE and BP will help. Having Votto at 1B is an upgrade over Hatty. If there is a position you trade defense for offense it is SS ... and yet the Reds didn't have a problem with playing Keppinger there. If you want to be a good defensive team you need quality up the middle, there is no way around it.

RedsManRick
09-28-2007, 02:09 PM
The other thing is the wiggle in these ratings is tremendous. Wright goes from stooge to genius. Wells goes from genius to stooge. Did that really happen? Could that really happen? Just thinking out loud here, but maybe what these systems is showing us is that defense is more random than we thought. If so, then we're measuring the randomness and not the skill. Also, if it's more random than we thought, what's causing the randomness? Sheer physics? Weather/field conditions? Pitching quality? Hitting quality?

M2, I think this is perhaps the great insight. While defensive ability is likely somewhat constant from year to year, the expression of that ability on the field as actual plays various substantially from year to year. Why? I think we see it in the distribution itself.

The plays made distribution is fairly tight and the opportunities fairly consistent on balls in the zone. The average CF gets about 350 balls hit in to his zone each year. I think the real differences are in the ability to get balls outside of the zone. The

What really makes a guy like Granderson and Ichiro amazing are the amount of balls they get to that are out of their zone. I imagine that the run values on those balls, particularly in CF, is quite high. Essentially, every ball that they get to in the gap that somebody else doesn't get to is a double or triple. If they do that 30 or 40 times more than other guys, that's huge. The deviation of plays made out of zone probably is a larger portion of defensive run variance from player to player than is balls in the zone, despite the small number of opportunities.

As for the turn-around of David Wright, I'd love to dive in to the metric and figure out where the shift occurred.

RedsManRick
09-28-2007, 02:12 PM
I have a problem with the defensive run values when they are then added to the offensive run value, which are known to be much more accurate, and then used as a method of evaluating a players total value in a ranking. If the defensive run values are not accurate, then they shouldn't be used this way b/c it just creates misinformation IMO.

I wouldn't base an arbitration argument on it, but again, give me a better option and I'll take it.

I think simply ignoring defense, or conjuring up an adjustment on gut feel is not more accurate than using run values we aren't really comfortable with.

I absolutely want to be intellectually honest. But let's also try to keep the discussion constructive. We could sit here all day and bash the various metrics, point out inconsistencies, etc. I have no problems with anybody taking issue with a flawed metric. But let's also figure out how we should go about making a fair overall valuation. If this isn't it, what is?

pedro
09-28-2007, 02:16 PM
I wouldn't base an arbitration argument on it, but again, give me a better option and I'll take it.

I think simply ignoring defense, or conjuring up an adjustment on gut feel is not more accurate than using run values we aren't really comfortable with.

I absolutely want to be intellectually honest. But let's also try to keep the discussion constructive. We could sit here all day and bash the various metrics, point out inconsistencies, etc. I have no problems with anybody taking issue with a flawed metric. But let's also figure out how we should go about making a fair overall valuation. If this isn't it, what is?

Honestly I have no idea.

M2
09-28-2007, 02:19 PM
I have a problem with the defensive run values when they are then added to the offensive run value, which are known to be much more accurate, and then used as a method of evaluating a players total value in a ranking. If the defensive run values are not accurate, then they shouldn't be used this way b/c it just creates misinformation IMO.

Amen.

Though I do agree with RMR's premise that defensive ability is overlooked at award time. I'd still vote for Braun for ROY because his bat is off the charts and I'd prioritize the ferocious hitting display over the net positional run value for that award.

My take is a little different for MVP. I put more stock in guys who have superior seasons in up-the-middle positions because they affect the game in ways that pure mashers can't. Carlos Beltran was a GG CF in 2006, he led baseball in OF assists, he ran the bases well and he beat the snot out of the ball. Despite the gaudy numbers Ryan Howard posted, I am positive that Beltran was the better and more valuable player (Beltran might also be the best player alive this season, but he missed some time to injury).

In trying to approach those questions systematically, I think it's important not to assume that marginal hitting runs and marginal fielding runs share a one-to-one equivalency (which is the point pedro just made). Mixing and matching the two without some investigation for how the two should be combined can net you some cock-eyed results. One of the reasons I like Win Shares is that James went through the rigor of sorting out that very issue. Feel free to take issue with the methodology for how he reaches his defensive values, but on the question of how defensive performance relates to offensive performance he's miles ahead.

RedsManRick
09-28-2007, 02:20 PM
Honestly I have no idea.

Well, I do find it interested that the two rough passes on find the real run value of Braun's 44 missed opportunities were -16 and -20 runs. Would people feel more comfortable if we used 50% of those published values? I'd still suggest that a 25 run swing between Braun and Tulowitzki would significantly affect their chances of winning ROY -- a greater effect than an adjustment made by gut feel.

pedro
09-28-2007, 02:24 PM
Well, I do find it interested that the two rough passes on find the real run value of Braun's 44 missed opportunities were -16 and -20 runs. Would people feel more comfortable if we used 50% of those published values? I'd still suggest that a 25 run swing between Braun and Tulowitzki would significantly affect their chances of winning ROY -- a greater effect than an adjustment made by gut feel.

Probably. Either way, I don't disagree with your general premise.

flyer85
09-28-2007, 02:26 PM
In the end its just data. The question is where does it impact the decision making process and how much value is assigned to the numbers.

Rojo
09-28-2007, 02:42 PM
Singles are worth .25 runs, Doubles .50, and triples .75. Missed plays by 3B turn in to singles 60% of the time, doubles 30% of the time, and triples 10% of the time.

Braun missed 44 plays: 27 became singles, 13 doubles, and 4 triples.

24*.25 + 13*.50 + 4*.75 = 16 runs allowed. That's about half of what the +/- Runs system suggests.

And you're still missing the value of extra outs.

jojo
09-28-2007, 02:43 PM
The better you do offensively, the more your defense is overlooked and vice versa.

I would say it the another way.....the more above average your offense, the less drag your defense is on your value.

When Dunn's bat is worth 5 wins he's still big plus (basically a $12-13M player) even if his glove gives two wins back. While his defense is still a drag (i.e. people could lament, "just think how good he would be if....), for practical purposes, three win players make impacts.

RedsManRick
09-28-2007, 02:48 PM
I would say it the another way.....the more above average your offense, the less drag your defense is on your value.

When Dunn's bat is worth 5 wins he's still big plus (basically a $12-13M player) even if his glove gives two wins back. While his defense is still a drag (i.e. people could lament, "just think how good he would be if....), for practical purposes, three win players make impacts.

Except that's not true. It's not that the impact of your defense is any less. That's the misconception. Just because Dunn is a great offensive player, doesn't mean we should give less weight to his defense. It's that because your offensive value is so high, your overall value is still high, even once defensive value is properly factored in. It's the Derek Jeter issue.

The question is, if Dunn, as a "5-2" 3 win player is worth 13M, then shouldn't a "2+1" 3 win player also be worth 13M? That's the player who I think we're perennially underrating. We're undervaluing the defensive shortcomings for great offensive players and not properly assigning defensive credit for mediocre offensive players. It was the argument for Austin Kearns' value and I think it's still relevant. Average offense and above average defense is an above average player; just as great offense and crap defense is.

jojo
09-28-2007, 03:12 PM
Except that's not true. It's not that the impact of your defense is any less. That's the misconception. Just because Dunn is a great offensive player, doesn't mean we should give less weight to his defense. It's that because your offensive value is so high, your overall value is still high, even once defensive value is properly factored in. It's the Derek Jeter issue.

I wasn't suggesting that the value of his defense changes but rather the impact of the defense does become diluted by increasing the value of a player's offense. For instance a -20 run glove diminishes the overall value of player with a 3 win bat by 66%. That same -20 run glove only reduces the overall value of a 5 win bat by 40%. Really, the impact of the poor defense is more tolerable as the offensive production increases especially when talking about bats that are in the 5 win range because the player's overall worth would be climbing into a value range that is more difficult to acquire despite an atrocious -20 defensive rating (i.e. his bat carries his glove like Manny's). The converse is true as well concerning the effect of defense though it appears the upper limit of offense is much higher than the upper limit of defense (it is probably unrealistic for a player to be more than a 3 win glove). Sure it would be better to have a left fielder who is a five win bat and a league average defender (or even a plus defender). But really, there aren't many of those. Just off of the top of my head, I'm having a hard time remembering one that I've seen in person not named Bonds (and then only in his prime).

My issue is when a player's defensive value is ignored altogether when arguing his total worth (i.e. when a team pays a player like he can hit AND catch).



The question is, if Dunn, as a "5-2" 3 win player is worth 13M, then shouldn't a "2+1" 3 win player also be worth 13M?

ABSOLUTELY. That's been a central theme of my rants on defense since, well, my very first post on redszone. I've argued many times that Dunn's defense mitigates his total value so even though he might be a 4-5 win bat, he has somewhere around a 3 win impact in reality. Five wins is for all intents and purposes irreplaceable. Three wins is within a range that a talented FO could deal with as there are many different ways to get to three wins. Really a 2 win bat who plays league average defense gets you 2/3 of the way there.

Falls City Beer
09-28-2007, 04:06 PM
Except that the +/- Runs when applied to defense isn't actually logically sound. There's no way on Earth that Ryan Braun is worth negative 30 Runs defensively. Here's why:

The expected Plays/BIZ for a third baseman is .706. According to THT's numbers (which I do not trust), Braun has seen 219 Balls in Zone this season and has made 123 "Plays". That means Braun is 32 Plays short of the .706 standard. Unless every one of those Plays projects to be an inside the park Home Run, there simply isn't enough opportunity for Braun to be worth -30+ Runs defensively; and that's even if we factor in that he appears to reach fewer OOZ balls than your average 3B (@-12 OOZ Plays).

Even if we trust THT's wonky BIZ/OOZ data, we're looking at a total of 44 missing plays by Braun versus expected results. And those plays are allegedly good for around minus 30 defensive Runs? Sorry, but that's quite literally impossible.

Great post. I think some folks want to stake the next great claim in metrics, overstating the case FOR defense so they can mine the popularity that might emerge from "discovering" something about defense that heretofore had eluded people.

By the way, that's not a shot at RMR, but at THT's methods--as well as others'.

Johnny Footstool
09-28-2007, 04:23 PM
And you're still missing the value of extra outs.

Does the out get extra value, though? You reach base, and that has a positive value towards scoring. You've created part of a run. You don't, and it has zero value. You haven't created a negative run.

Highlifeman21
09-28-2007, 04:27 PM
I think the only thing that can be established with any certitude is that Ryan Braun is not a major league 3b.

Defensively, no.

Offensively, I'd definitely say he is.

Ryan Braun is only a 3B for the Brewers b/c Prince Fielder plays 1B and they are in the NL.

jojo
09-28-2007, 04:40 PM
Great post. I think some folks want to stake the next great claim in metrics, overstating the case FOR defense so they can mine the popularity that might emerge from "discovering" something about defense that heretofore had eluded people.

By the way, that's not a shot at RMR, but at THT's methods--as well as others'.

THT's defensive metrics aren't good (and shouldn't be weighted as heavily as the gold-standard PBP metrics) so Justin's run conversions really are more talking points that have the advantage of being updated and accessible throughout the season. That said, RMR is arguing for a philosophy that I think is dead on.

Concerning quantifying defense, being able to better evaluate defensive value would allow significant advances in how both position players and pitchers are evaluated. I think it's fair to suggest that it's a significant enough problem that a great many of the minds attracted to seeking it's solution are motivated to do so by noble intentions.

M2
09-28-2007, 05:37 PM
Concerning quantifying defense, being able to better evaluate defensive value would allow significant advances in how both position players and pitchers are evaluated. I think it's fair to suggest that it's a significant enough problem that a great many of the minds attracted to seeking it's solution are motivated to do so by noble intentions.

Totally agreed on the potential value of better defensive evaluation and that folks trying to do so have their hearts in the right place. I just think they're a little quick to proclaim that they've had "eureka" moments.

pedro
09-28-2007, 05:39 PM
I also have a bit of an issue with this "gold standard" stuff. It makes it sound like the metrics are actually more accurate than they are.

M2
09-28-2007, 05:44 PM
I also have a bit of an issue with this "gold standard" stuff. It makes it sound like the metrics are actually more accurate than they are.

Good point, when it comes to defense we're usually talking about pyrite variants.

RedsManRick
09-28-2007, 05:44 PM
So, realizing of course that the existing defensive metrics are flawed significantly, are we better off using none at all, and just using our best guess at how to adjust our valuations, or using a flawed metric?

Chip R
09-28-2007, 05:49 PM
So, realizing of course that the existing defensive metrics are flawed significantly, are we better off using none at all, and just using our best guess at how to adjust our valuations, or using a flawed metric?


It's a good question. However, if we just use conventional wisdom, you have players like Juan Castro having more value than they should because they have a good rep.

pedro
09-28-2007, 05:51 PM
So, realizing of course that the existing defensive metrics are flawed significantly, are we better off using none at all, and just using our best guess at how to adjust our valuations, or using a flawed metric?

I think they have value with 3 caveats.

1. It should be noted that the numbers on the periphery are skewed to the point of virtual impossibility.

2. They should never be used in conjunction with offensive run value to arrive at a composite run value for a player.

3. Being that the run values appear to be highly volatile from year to year without a clear explanation as to why, it shouldn't be assumed that these PBP metrics are necessarily really that much more accurate then the more commonly used defensive benchmarks such as RF and ZR.

That's my opinion anyway.

Falls City Beer
09-28-2007, 05:55 PM
So, realizing of course that the existing defensive metrics are flawed significantly, are we better off using none at all, and just using our best guess at how to adjust our valuations, or using a flawed metric?

I think some healthy agnosticism is in order, that's all. That doesn't mean you throw everything out--just know that you are dealing with serious, serious limitations.

My biggest gripe with presumed certainty of defensive metrics (either singular or combined) is the leap taken from performance in the field to RA--that's where a huge bulk of my agnosticism lies. Clear that up with some degree of certainty and repeatability and you're probably moving towards an answer to the value of defense.

RedsManRick
09-28-2007, 06:09 PM
I think they have value with 3 caveats.

1. It should be noted that the numbers on the periphery are skewed to the point of virtual impossibility.

2. They should never be used in conjunction with offensive run value to arrive at a composite run value for a player.

3. Being that the run values appear to be highly volatile from year to year without a clear explanation as to why, it shouldn't be assumed that these PBP metrics are necessarily really that much more accurate then the more commonly used defensive benchmarks such as RF and ZR.

That's my opinion anyway.

1. Duly Noted. In the original post, I was using the number more for illustrative purposes.

2. Ok.... So now what? I have a great offensive guy who sucks at defense and a great defender who's just ok offensively. Who do I play? Who do I sign in FA? For how much? These are decisions that still have to get made. How do we make them?

3. Agreed to a point. This is based on the (likely correct) assumption that a player's actual defensive ability doesn't vary significantly from year to year. However, that doesn't mean defensive performance doesn't vary. Dave Ross isn't a worse hitter than he was last year, and yet his OPS dropped 28%. Does that make OPS a bad stat? Perhaps we just need to aggregate the pbp data to get bigger samples.

RedsManRick
09-28-2007, 06:36 PM
Justin doesn't have access to ORG, and I asked that I post this on his behalf.


Comments after skimming this thread:

* [The runs/play at third base is estimated at 0.8r/p. This is calculated used to convert the difference in plays made in zone and the difference in plays made out of zone compared to expected into runs. You can find the rest of his runs/play translations in this post by Chris Dial: http://www.baseballthinkfactory.org/...cussion/33941/

They are roughly equivalent to the difference between making a play and not making a play. Based on linear weights, an out (play made) is worth ~-0.3 runs compared to the average play, while a single is worth ~+0.4 runs compared to the average play. This puts the difference between making a play and allowing a single at ~0.7 runs. The addition 0.1 run is because some of those missed plays become doubles, some of the "missed plays" become throwing errors that advance the runner an extra base, etc.

Yes, it's just an estimate. But it's probably not too far off.

* Braun is hit particularly badly on balls in zone, which I think reflects the number of throwing errors he's had (14!). Maybe, then, he can still improve with better footwork?

* Center field is the position where I'm the least confident with the THT measures. The reason is that a) there is no adjustment for the handedness of the batter, which makes a huge difference, and b) they differ drastically with UZR at times. For example, as I mentioned in my Cleveland Indians profile, Sizemore is rated as awful with THT's data, but is rated as excellent in UZR. I tend to favor UZR in this case because it agrees with Grady's reputation, and because UZR does include both park effects and an adjustment for batter handedness.

* It is fair to critique the fielding methods we have. For example, a recent study published at Hardball times identified that there are, at times, substantial differences in the raw data that STATS Inc and BIS report, which inevitably will lead to differences in the metrics we extract from those data.

The safest thing to do in fielding evaluation -- and the best way to decrease uncertainty -- is to use UZR and THT's data together, perhaps along with other systems like Tom Tango's fan scouting report and David Pinto's PMR. Since all of those can be reported or converted to +-runs saved, I'd be most comfortable simply taking the mean of all of these measures. Unfortunately, THT's data is all we currently have for this season. We'll have more in the offseason. But as I've shown in past work on my blog, there's enough agreement with THT and other metrics to be reasonably confident with those numbers. If there's a player who seems way off the mark, however, you do have to use your head.

Once you're comfortable with your estimate of fielding, however, I think that a player's total value has to be estimated as the sum of his runs created on offense and runs saved on defense, ideally with reference to a replacement player at the same position. The tendency of analysts--both in and out of the game--to largely ignore defense or use it as a tiebreaker is a rather naive way to evaluate players.

* Here's Dunn's recent fielding history (UZR for '03-'06, THT for '07):
2003: -10 runs
2004: -6 runs
2005: -6 runs
2006: -20 runs(!!!)
2007 (THT data): -18 runs

My interpretation of that is that Dunn's defense was below average but tolerable until last year, when it fell off the cliff. And he is not noticeably better this season. Fortunately, he actually hit this season, which puts him at ~30 runs over replacement overall--still quite valuable. Last season, defense was a bigger issue because he contributed less on offense.
-Justin

IslandRed
09-28-2007, 06:40 PM
2. Ok.... So now what? I have a great offensive guy who sucks at defense and a great defender who's just ok offensively. Who do I play? Who do I sign in FA? For how much? These are decisions that still have to get made. How do we make them?

Good point. If a team is analytical at all, and attempts to quantify a player's offensive run value, they're going to be making judgments about a player's defensive run value also. They might not be taking any one particular metric's word for it. They may not even come up with a specific number. But at some level, they're making judgments, even if implicit or subconscious. It's probably helpful to go through the exercise of trying to quantify it, even if they can't, just to clarify their thinking on a player.

pedro
09-28-2007, 06:45 PM
2. Ok.... So now what? I have a great offensive guy who sucks at defense and a great defender who's just ok offensively. Who do I play? Who do I sign in FA? For how much? These are decisions that still have to get made. How do we make them?



Well, if I'm a GM for a major league club I get the video of every ball hit to a guy during the past two seasons and make a determination for myself or task someone to do it for me. I'd also look at ALL the competing metrics and try and come up with an aggregated estimate.

Major league clubs have access to a lot of information that we don't. Leaning only on publicly available info would probably be a bad idea for someone who does this for a living.

dougdirt
09-28-2007, 07:17 PM
Well, if I'm a GM for a major league club I get the video of every ball hit to a guy during the past two seasons and make a determination for myself or task someone to do it for me. I'd also look at ALL the competing metrics and try and come up with an aggregated estimate.

Major league clubs have access to a lot of information that we don't. Leaning only on publicly available info would probably be a bad idea for someone who does this for a living.

Baseball Info Solutions does this, for every ball put in play all year. It started with the fielding bible, but they decided to only publish the info once. I believe that they now sell this information. They however don't just make a decision on whether they think someone should have gotten it, they chart everything about the play and use all of the data to come up with much stronger information on who would and wouldn't have made plays over the course of a season.

jojo
09-28-2007, 07:19 PM
I also have a bit of an issue with this "gold standard" stuff. It makes it sound like the metrics are actually more accurate than they are.

Gold standard implies the best currently available (at least to the public) which certainly applies to a metric like UZR or Dewan's +/- system.

jojo
09-28-2007, 07:39 PM
So, realizing of course that the existing defensive metrics are flawed significantly, are we better off using none at all, and just using our best guess at how to adjust our valuations, or using a flawed metric?

I agree 100% that often a lot of the argument against advanced metrics boils down to "they're not perfect so we should discard them" despite their improved ability to capture a player's skill versus older, more comfortable metrics.

That said, i'd submit it has yet to be argued convincingly that UZR and Dewan's +/- system are flawed significantly especially to the point of not informing a player's skillset. A survey of metrics like UZR, Dewan's +/-, and PMR in the very least can allow a range (i.e. analogous to a confidence interval) to be attributed to a players glove. For instance, it's pretty clear a survey of those metrics support the notion that Dunn is a -25 to -15 defender. Therefore if he has a VORP of 50, his true value is somewhere on the order of 2.5 to 3 wins over replacement. That is a huge advance in player evaluation and criticisms that PBP-based metrics don't yield an exact number miss the point and ignore their usefulness IMHO. Combine a system like UZR with detailed scouting reports (an approach most major league FOs now use) and a player's true defensive skill can really be determined in a tangible sense that informs personnel decisions.

SteelSD
09-28-2007, 08:14 PM
Singles are worth .25 runs, Doubles .50, and triples .75. Missed plays by 3B turn in to singles 60% of the time, doubles 30% of the time, and triples 10% of the time.

Braun missed 44 plays: 27 became singles, 13 doubles, and 4 triples.

24*.25 + 13*.50 + 4*.75 = 16 runs allowed. That's about half of what the +/- Runs system suggests.

That's a more reasonable drill-down IMHO, but we need to remember that a portion of those 44 plays represent OOZ opportunities. That's where we hit a snag, because I'd suggest that not all "missing" OOZ opportunities turn into the equivalent of base hits. Supporting fielders may be scooping up some of those opportunities and turning them into Outs and, if those OOZ "missing" plays result from balls dropping foul or into the stands, we again don't necessarily have a runner reaching base.

While I agree that there's a value in trying to understand the actual run value assigned to fielding, none of the current efforts have gone nearly far enough from a validity standpoint. As it stands, they simply assign the same kind of abstract values to defense that we get with things like Zone Rating. While speaking to proficiency is fine, I've yet to see anything able to provide a reasonable reflection of actual run value.

jojo
09-28-2007, 08:20 PM
I think they have value with 3 caveats.

1. It should be noted that the numbers on the periphery are skewed to the point of virtual impossibility.

Each PBP-based metric has a slightly different approach to the problem and while one may have a blind spot for an aspect of a certain player's defense, a survey of them will invariably sort out the blind spots. If all three think a player is minus infinity then simply say he's atrocious and cut him off at -25 or -20 if that makes you more comfortable while acknowledging his true worth may be somewhat of an overestimate as a result. That said, like any other metrics, sample size and special circumstances need to be considered too. For instance, Manny isn't a -50 defender since the Green Monster skews measures of his ability. Jeremy Reed probably wasn't a three win defensive center fielder in '05 either (though the Fielding Bible suggested he was) because Safeco causes overestimates of left and center field defense. That said, Manny is bad every year and Reed was very,very good that year.


2. They should never be used in conjunction with offensive run value to arrive at a composite run value for a player.

Why not? Replacement level defense is essentially league average so defensive runs from PBP metrics are congruent with both a metric like RC or a positional measure like VORP or RC over average. In order to argue this caveat, it has to be argued that defensive runs are wholly unreliable. That isn't a caveat really, it's a rejection of the metrics outright.


3. Being that the run values appear to be highly volatile from year to year without a clear explanation as to why, it shouldn't be assumed that these PBP metrics are necessarily really that much more accurate then the more commonly used defensive benchmarks such as RF and ZR.

I definitely understand where people are coming from with this concern. As with all metrics, context and sample size are extremely important considerations for PBP-based metrics. Sample size has a huge impact on the reliability of run values these metrics spit out. HUGE impact. Age, injury, change in environment etc all play important roles too. Simple random variation also makes a difference. Justin's comments highlighted what really looks like an age effect with Dunn. He's past the peak age for defense and while he was never a good defender, his skills have seemingly started to erode further.

As with all metrics (offensive or defensive), the more data the better when trying to precisely estimate a player's true skill level. Multiple season's worth of data are much better than a single season's. A half season's worth of defensive data really is only a talking point due to sample size.


That's my opinion anyway.

:beerme:

TeamBoone
09-28-2007, 08:32 PM
He looks better than before, but citing someone's fielding average of proof he's better just doesn't cut it.

Sorry, I didn't mean to ruffle your feathers.

I don't know much about defensive stats, but I can compare them. I was merely looking at the defensive stats that were readily available to me so that I could compare them quickly to years previous. I don't know what's meaningful and what's not. Sorry it made you mad.

Falls City Beer
09-28-2007, 08:44 PM
Gold standard implies the best currently available (at least to the public) which certainly applies to a metric like UZR or Dewan's +/- system.

This is circular argumentation. The term "gold standard" applies to things that establish a clear effect--like in the efficacy of medications, for instance, where the alleviation of symptoms is clearly measureable against a control group. How can they be considered "gold standard" when they have no controls against which to be measured? Again, where's the reliability? the validity?

Why is there such massive disagreement between and among systems?

You blame most of us for incuriosity, but that doesn't bug you? the fact that each system seems incurably unreliable?

jojo
09-28-2007, 08:57 PM
That's a more reasonable drill-down IMHO, but we need to remember that a portion of those 44 plays represent OOZ opportunities. That's where we hit a snag, because I'd suggest that not all "missing" OOZ opportunities turn into the equivalent of base hits. Supporting fielders may be scooping up some of those opportunities and turning them into Outs and, if those OOZ "missing" plays result from balls dropping foul or into the stands, we again don't necessarily have a runner reaching base.

Really though the baseline is for performance relative to other players at the same position. So if the shortstop makes the play on a ball that an average third baseman would make, then the third baseman that failed to make the play should be penalized for his lack of range. This points to a range-related problem with zone-rating based metrics though...."the ball-hog effect" where a player poaches a ball out of another player's zone (*cough* Ichiro *cough). UZR doesn't treat balls dropped into stands/in foul territory the same as ones missed in zones that are in fair play when calculating run values (I don't think PMR or Dewan's +/- do either).

Concerning THT defense metrics, I'm biased against them and can't speak to how Justin manages the issue. He's got a great write up on his website that is well worth the read for a glimpses into how the problem can be approached. I just don't remember details concerning the issue you raised.


As it stands, they simply assign the same kind of abstract values to defense that we get with things like Zone Rating.

Maybe I'm misunderstanding your point, but things like RC, VORP, LWTS, winshares etc basically assign abstract values to offensive events (in the sense that they aren't measuring actual runs) and few people other than edabbs (:beerme:) criticizes such metrics for that reason.

jojo
09-28-2007, 09:05 PM
This is circular argumentation. The term "gold standard" applies to things that establish a clear effect--like in the efficacy of medications, for instance, where the alleviation of symptoms is clearly measureable against a control group. How can they be considered "gold standard" when they have no controls against which to be measured? Again, where's the reliability? the validity?

Why is there such massive disagreement between and among systems?

You blame most of us for incuriosity, but that doesn't bug you? the fact that each system seems incurably unreliable?

I'd suggest you're just being argumentative. I simply explained what I meant by MY use of the term gold standard..... dock me style points if you think I've turned the phrase into hideous slang.

I've never intentionally blamed any group on the ORG for incuriosity. I've been described as being incurious recently though. I'm thinking it must be the new buzz phrase...maybe it will replace verboten?.

IslandRed
09-28-2007, 09:12 PM
While I agree that there's a value in trying to understand the actual run value assigned to fielding, none of the current efforts have gone nearly far enough from a validity standpoint. As it stands, they simply assign the same kind of abstract values to defense that we get with things like Zone Rating. While speaking to proficiency is fine, I've yet to see anything able to provide a reasonable reflection of actual run value.

Agreed on the lack of being able to take any current metric's word as gospel.

But let's say we're looking at two potential center fielders. We believe to the best of our analytical ability that Player A is about 15 runs better offensively than Player B. We also have the belief, however derived, that Player B is a better defensive center fielder than Player A. The two players will cost about the same to acquire.

In the end, like it or not, we're going to make an assumption -- quantified or implicit -- on whether B is 15 runs better in the field than A.

So how to make that determination, then? What do you consider the best available methodology in the absence of a trustable metric? I'm not trying to put you on the spot. It's a honest question to which I don't have a good answer.

RedsManRick
09-28-2007, 09:15 PM
Agreed on the lack of being able to take any current metric's word as gospel.

But let's say we're looking at two potential center fielders. We believe to the best of our analytical ability that Player A is about 15 runs better offensively than Player B. We also have the belief, however derived, that Player B is a better defensive center fielder than Player A. The two players will cost about the same to acquire.

In the end, like it or not, we're going to make an assumption -- quantified or implicit -- on whether B is 15 runs better in the field than A.

So how to make that determination, then? What do you consider the best available methodology in the absence of a trustable metric? I'm not trying to put you on the spot. It's a honest question to which I don't have a good answer.

Exactly IR. Thanks for reiterating my question. Agree with the methodology or not, decisions are made that require some sort of valuation of defense as a part of the whole picture. If this sort of quantitative method isn't it, what is? Steel, given your level of rigor in particular, I'd like to understand how make the above decision.

Falls City Beer
09-29-2007, 12:03 AM
Agreed on the lack of being able to take any current metric's word as gospel.

But let's say we're looking at two potential center fielders. We believe to the best of our analytical ability that Player A is about 15 runs better offensively than Player B. We also have the belief, however derived, that Player B is a better defensive center fielder than Player A. The two players will cost about the same to acquire.

In the end, like it or not, we're going to make an assumption -- quantified or implicit -- on whether B is 15 runs better in the field than A.

So how to make that determination, then? What do you consider the best available methodology in the absence of a trustable metric? I'm not trying to put you on the spot. It's a honest question to which I don't have a good answer.

How? Question how one arrives at a methodology that translates defensive events into runs. That would be the place to start. That's a gaping chasm, and a bunch of really smart people with really interesting self-verifying abstract systems are going to throw a bunch of ideas into that chasm trying to fill it in.

I'd say scouting is still "the gold standard" for determining players' defensive worth. And it's probably not all that close. I actually think you'd find far more agreement (validity) among the best scouts about a player's defensive efficacy than you would even among the most 24-karat self-verifying systems.

Falls City Beer
09-29-2007, 12:18 AM
I'd suggest you're just being argumentative. I simply explained what I meant by MY use of the term gold standard..... dock me style points if you think I've turned the phrase into hideous slang.

I've never intentionally blamed any group on the ORG for incuriosity. I've been described as being incurious recently though. I'm thinking it must be the new buzz phrase...maybe it will replace verboten?.

I"m not being argumentative for the sake of being argumentative. At all. No style points.

You just arrogantly repeat again and again these appeals to authority that end up NOT explaining at all your position except to post many links to sites; however, when asked to put your argument in simple laymen's terms, you are perfectly incapable. You answer any logical query I might have with an imperious wave of the hand, as if it doesn't apply, as if it were a matter of "style" instead of the absolute crux of the matter, which it is. You are completely eschewing scientific rigor, yet you are protesting for it. You just proclaim over and over that "this system works" without ever doing anything but ultimately cutting and pasting the arguments of others into your text box and claiming superiority. Explain how you can reliably sift run values from defensive events with the same reliability as you can pitching and offensive events (though admittedly pitching and defense are tied together). Put your argument in simple, textbook terms, all acronyms explained and no copying and pasting.

SteelSD
09-29-2007, 12:24 AM
Really though the baseline is for performance relative to other players at the same position. So if the shortstop makes the play on a ball that an average third baseman would make, then the third baseman that failed to make the play should be penalized for his lack of range.

I see your point, jojo, but what I'm talking about is an assumption that an out-of-zone play not made by the third baseman automatically turns into a base hit. For example, Ryan Braun may appear to have an OOZ disadvantage vs. his expected OOZ Play%, but that doesn't mean that the balls he's not getting to aren't turned into Outs by the Shortstop. I'd suggest a 100% correlation doesn't exist between OOZ "missed" chances and base hits.


This points to a range-related problem with zone-rating based metrics though...."the ball-hog effect" where a player poaches a ball out of another player's zone (*cough* Ichiro *cough).

That's a good point. But it's also important to remember that teammates work in unison to track down balls in play. Should a team have a rangey Shortstop, and should such team's 3B have a pretty clear understanding of said SS's range, it's possible that the third sacker could actually echew a potential OOZ play knowing that the SS is likely to produce the Out by converting an easier chance.


UZR doesn't treat balls dropped into stands/in foul territory the same as ones missed in zones that are in fair play when calculating run values (I don't think PMR or Dewan's +/- do either).

I'd have to look it up to be absolutely certain, but I believe that foul balls producing Outs don't count as "BIZ" chances, yet do count as "OOZ" Outs. If that's true in the case of how THT's metrics are compiled, fields with more foul ground would offer an advantage to corner OF and IF versus fields that do not. If foul ground does come into play as OOZ "Plays" on caught balls, I'd expect a skewing of the data based on foul ground dimension.


Concerning THT defense metrics, I'm biased against them and can't speak to how Justin manages the issue. He's got a great write up on his website that is well worth the read for a glimpses into how the problem can be approached. I just don't remember details concerning the issue you raised.

Hey, feel free to link me, my man. :)


Maybe I'm misunderstanding your point, but things like RC, VORP, LWTS, winshares etc basically assign abstract values to offensive events (in the sense that they aren't measuring actual runs) and few people other than edabbs (:beerme:) criticizes such metrics for that reason.

The main offensive statistic you'll see me use is Runs Created. The reason? It's probably the least "abstract" creation out there to measure offensive contribution because of it's correlation with a firm baseline (i.e. actual team offensive Run Scoring output). Of course, it's not wholly predictive, but it's an excellent reflection of how much a player was actually worth to his team. Runs Above Position is a very solid comparative tool as long as one understands it's nuances (i.e. the "same number of Outs" clause) due to having a starting point (EQR) that is highly correlated with actual offensive production.

If we were talking about abstraction on a scale of 1 (low) to 10 (high), RC would be a "2". It's actually a "1" for me, but that's because I've studied it inside and out. RAP would rate a "3" due to the viewer's need to equalize for Outs. Win Shares would be about a "5" because you need to start squinting at that point from a team-to-team compartive perspective.

Right now, assigning 30+ negative Runs to a fielder based on a total of 32 "missing" BIZ and 12 "missing" OOZ plays is an abstraction that's off-the-scale high and it starts with the fact that we've got no established baseline for correlation analysis. Add in impossible results like 30+ negative Runs given the limited "missing" plays and we've got an abstraction of the highest order. It's a result that isn't valid because it can't be valid unless baseballs are breaking the laws of physics. While it may speak to relative proficiency, it certainly doesn't speak to actual Run value.

At best, the +/- stuff is specious. At worst, it's the poisoned fruit of folks who really really want to be the breakthrough artist in the field of defensive analysis.

Spitball
09-29-2007, 01:34 AM
...if it's more random than we thought, what's causing the randomness? Sheer physics? Weather/field conditions? Pitching quality? Hitting quality?

Very well said. Defense has a boatload of variables. Hometown score keepers have to be thrown into that mix ,also. I watched a Cardinals' game the other night in which Miguel Cairo fielded a routine grounder, tripped over an unseen clump of something, and the hometown scorekeeper gave the batter a hit. Geesh! It happens too regularly.

M2
09-29-2007, 02:45 AM
One thing that always jumps into my mind when the subject of defense comes up is that those seeking to come up with a good universal measurement always focus on the wrong things. They either put their faith in a manipulation of the raw fielding stats or they track the ball or they segment the field. Seems to me that if you're trying to understand the value of the defender you'd do well to measure the defender and not the atmospherics around him.

Rojo
09-29-2007, 04:45 AM
One thing that always jumps into my mind when the subject of defense comes up is that those seeking to come up with a good universal measurement always focus on the wrong things. They either put their faith in a manipulation of the raw fielding stats or they track the ball or they segment the field. Seems to me that if you're trying to understand the value of the defender you'd do well to measure the defender and not the atmospherics around him.

You always lose something when you try to digitize the analog.

GAC
09-29-2007, 05:14 AM
Since this discussion has mainly centered around 3B (Braun).... what are some of the feelings about the Nat's Zimmerman (+21.1)?

jojo
09-29-2007, 06:04 AM
I"m not being argumentative for the sake of being argumentative. At all. No style points.

You just arrogantly repeat again and again these appeals to authority that end up NOT explaining at all your position except to post many links to sites; however, when asked to put your argument in simple laymen's terms, you are perfectly incapable. You answer any logical query I might have with an imperious wave of the hand, as if it doesn't apply, as if it were a matter of "style" instead of the absolute crux of the matter, which it is. You are completely eschewing scientific rigor, yet you are protesting for it. You just proclaim over and over that "this system works" without ever doing anything but ultimately cutting and pasting the arguments of others into your text box and claiming superiority. Explain how you can reliably sift run values from defensive events with the same reliability as you can pitching and offensive events (though admittedly pitching and defense are tied together). Put your argument in simple, textbook terms, all acronyms explained and no copying and pasting.

Wading through all of the noise and hand waving in your post, I think I found a question, though really, I doubt most would bother to answer you. But anyway. It's very simple. Defensive run values are derived the same way many very sound offensive ones are-using run expectancy and linear weights.

BTW, here's once again a link to an explanation of UZR per a previous conversation we've had (http://www.redszone.com/forums/showpost.php?p=1399640&postcount=123). It was the first of many times that I've answered this very question from you. If it's overly technical, flawed by an over reliance on acronyms, while both appealing to authority and representing an example of blatant plagiarism, then I guess i'll have to try harder next time.

jojo
09-29-2007, 06:42 AM
I see your point, jojo, but what I'm talking about is an assumption that an out-of-zone play not made by the third baseman automatically turns into a base hit. For example, Ryan Braun may appear to have an OOZ disadvantage vs. his expected OOZ Play%, but that doesn't mean that the balls he's not getting to aren't turned into Outs by the Shortstop. I'd suggest a 100% correlation doesn't exist between OOZ "missed" chances and base hits.



That's a good point. But it's also important to remember that teammates work in unison to track down balls in play. Should a team have a rangey Shortstop, and should such team's 3B have a pretty clear understanding of said SS's range, it's possible that the third sacker could actually echew a potential OOZ play knowing that the SS is likely to produce the Out by converting an easier chance.



I'd have to look it up to be absolutely certain, but I believe that foul balls producing Outs don't count as "BIZ" chances, yet do count as "OOZ" Outs. If that's true in the case of how THT's metrics are compiled, fields with more foul ground would offer an advantage to corner OF and IF versus fields that do not. If foul ground does come into play as OOZ "Plays" on caught balls, I'd expect a skewing of the data based on foul ground dimension.

I'm the wrong person to defend RZR because I pretty much tend to ignore it as a system. Really your concerns are related to how zones are handled and I too think that strikes at the heart of many of it's flaws.


Hey, feel free to link me, my man. :)

My apologies, I meant to provide a link.

Anyway, at the risk of being called incurious or incapable of forming an argument, here you go (:cool:): On Baseball and the Reds (http://jinaz-reds.blogspot.com/)

I think Justin provides a direct link to his methodology in his entry for 9/26/07 were he reports his latest defensive run values data....


The main offensive statistic you'll see me use is Runs Created. The reason? It's probably the least "abstract" creation out there to measure offensive contribution because of it's correlation with a firm baseline (i.e. actual team offensive Run Scoring output). Of course, it's not wholly predictive, but it's an excellent reflection of how much a player was actually worth to his team. Runs Above Position is a very solid comparative tool as long as one understands it's nuances (i.e. the "same number of Outs" clause) due to having a starting point (EQR) that is highly correlated with actual offensive production.

If we were talking about abstraction on a scale of 1 (low) to 10 (high), RC would be a "2". It's actually a "1" for me, but that's because I've studied it inside and out. RAP would rate a "3" due to the viewer's need to equalize for Outs. Win Shares would be about a "5" because you need to start squinting at that point from a team-to-team compartive perspective.

Right now, assigning 30+ negative Runs to a fielder based on a total of 32 "missing" BIZ and 12 "missing" OOZ plays is an abstraction that's off-the-scale high and it starts with the fact that we've got no established baseline for correlation analysis. Add in impossible results like 30+ negative Runs given the limited "missing" plays and we've got an abstraction of the highest order. It's a result that isn't valid because it can't be valid unless baseballs are breaking the laws of physics. While it may speak to relative proficiency, it certainly doesn't speak to actual Run value.

At best, the +/- stuff is specious. At worst, it's the poisoned fruit of folks who really really want to be the breakthrough artist in the field of defensive analysis.

I adore discussions like this.

Personally, I've been won over by run expectancy and how the concept facilitates new avenues of understanding. That said, I agree wholeheartedly with you that runs created is a useful metric too. Truth be told, I've recently been pondering whether league average or replacement level is an appropriate baseline (which could ultimately result in an even greater reliance on RC as a metric by me in the future).

jojo
09-29-2007, 07:47 AM
I"m.......pasting.

This is a great thread that has stimulated alot of thoughtul debate even though it tackles an often contentious issue (and kudos to RMR-he's authored several great threads recently). So all snark aside, there is one appeal I would like to make- lets not ruin this with needless sniping. It takes two to tango so I'll own up to my contribution to the fire.

Falls City Beer
09-29-2007, 09:29 AM
This is a great thread that has stimulated alot of thoughtul debate even though it tackles an often contentious issue (and kudos to RMR-he's authored several great threads recently). So all snark aside, there is one appeal I would like to make- lets not ruin this with needless sniping. It takes two to tango so I'll own up to my contribution to the fire.

It is a contentious issue--but both you and RMR were the ones to get very defensive with pedro and me when we dared poke holes in the *certainty* of current PBP metrics that attempt to show a one to one relationship between defensive events and runs.

jojo
09-29-2007, 10:11 AM
It is a contentious issue--but both you and RMR were the ones to get very defensive with pedro and me when we dared poke holes in the *certainty* of current PBP metrics that attempt to show a one to one relationship between defensive events and runs.

I've just reread this thread and while I can't speak for RMR, aside from being put off by an ad hominem attack (where I still answered your question for the sake of discussion), I just don't see where you're coming from with that statement.

I think as a matter of discourse there has to be an acceptance of a standard that suggests there is a HUGE difference between disagreeing with and rebutting an opinion as part of a dialog and *getting very defensive*. If not, I have no idea how discussions can even be entertained.

I'd also suggest that there has yet to be "serious holes poked" in the way play-by-play metrics such as UZR calculate defensive runs.

BTW, MGL (the creator of UZR) is a paid consultant for major baseball teams (most recently being on the payroll of the St. Louis Cardinals). His system is either used by or forms the basis for the systems many mlb teams use as part of the way they evaluate defense. He has coauthored a book with Tom Tango, also a paid consultant to major league baseball teams as well as creator of FIP and really a pioneer in building upon the linear weights model with his research. Speaking of linear weights, alot of Tango's work builds upon that of Pete Palmer and John Thorn who, among other contributions, have coauthored several books such as "The Hidden Game of Baseball", which is considered a definitive work in sabermetrics and "Total Baseball: The Official Encyclopedia of Major League", a work that to anyone who has read it, is a definitive resource on all things baseball. John Dewan, former president and CEO of Stats Inc now owns Baseball Info Solutions, a company from which every major league club purchases information. His Fielding Bible is really a stripped down for the public version of his scouting report that all major league baseball teams purchase. His +/- system has been heartily endorsed by Bill James. James, whose body of work speaks for itself, is a consultant with Baseball Info Solutions and as well as senior baseball operations advisor for the Boston Red Sox.

By what objective standard would referencing the bodies of work of these people, especially in regards to play-by-play based defensive metrics be considered an "appeal to authority (http://www.nizkor.org/features/fallacies/appeal-to-authority.html)"?

Falls City Beer
09-29-2007, 10:24 AM
I've just reread this thread and while I can't speak for RMR, aside from being put off by an ad hominem attack (where I still answered your question for the sake of discussion), I just don't see where you're coming from with that statement.

I think as a matter of discourse there has to be an acceptance of a standard that suggests there is a HUGE difference between disagreeing with and rebutting an opinion as part of a dialog and *getting very defensive*. If not, I have no idea how discussions can even be entertained.

I'd also suggest that there has yet to be "serious holes poked" in the way play-by-play metrics such as UZR calculate defensive runs.

BTW, MGL (the creator of UZR) is a paid consultant for major baseball teams (most recently being on the payroll of the St. Louis Cardinals). His system is either used by or forms the basis for the systems many mlb teams use as part of the way they evaluate defense. He has coauthored a book with Tom Tango, also a paid consultant to major league baseball teams as well as creator of FIP and really a pioneer in building upon the linear weights model with his research. Speaking of linear weights, alot of Tango's work builds upon that of Pete Palmer and John Thorn who, among other contributions, have coauthored several books such as "The Hidden Game of Baseball", which is considered a definitive work in sabermetrics and "Total Baseball: The Official Encyclopedia of Major League", a work that to anyone who has read it, is a definitive resource on all things baseball. John Dewan, former president and CEO of Stats Inc now owns Baseball Info Solutions, a company from which every major league club purchases information. His Fielding Bible is really a stripped down for the public version of his scouting report that all major league baseball teams purchase. His +/- system has been heartily endorsed by Bill James. James, whose body of work speaks for itself, is a consultant with Baseball Info Solutions and as well as senior baseball operations advisor for the Boston Red Sox.

By what objective standard would referencing the bodies of work of these people, especially in regards to play-by-play based defensive metrics be considered an "appeal to authority"?

If these are the same men that say in no uncertain terms that Ryan Braun is worth -30 runs defensively, then yes, their opinions need to have holes shot in them.

But those men *aren't* saying that. Bill James is not THT. He would unequivocally shake his head at a methodology that arrived at that conclusion.

That's the problem: you're trotting out the heavy hitters, guys who are incredibly learned, schooled people in these matters and throwing them side-by-side with bloggers. Bill James is one of the most sober-minded thinkers in the game, despite thrusting some of the most revolutionary thinking on the sport over the last 30 years; that's the thinking I enjoy and trust.

In general, I feel like many tools, like UZR, in the wrong hands, become mildly absurd--the tool's not ultimately the problem, I suppose, but the conclusions that are arrived at from their usage (Braun = -30 runs defensively) aren't being appropriately tested for validity.

RedsManRick
09-29-2007, 10:36 AM
It is a contentious issue--but both you and RMR were the ones to get very defensive with pedro and me when we dared poke holes in the *certainty* of current PBP metrics that attempt to show a one to one relationship between defensive events and runs.

Let's not confuse defensive with back and forth discussion. I think people on all sides, particularly on a message board, often equate disagreement with personal affront. I personally will take a stand for the sake of argument or discussion, rather than because I passionately agree.

That said, I hope that in any post I make clear all the assumptions I'm making. Particularly when it comes to defensive metrics, I never meant to imply certainty. Rather, for the sake of constructive conversation, I used a specific example of the +/- runs system to illustrate a point I was trying to make.

When it comes to defensive metrics, we spend a lot of time and energy tearing them apart and talking them down. However, the purpose of this thread was to question, in light of that conversation, how do we go about integrating defensive measurement (using any system) in to the decision making process.

I posited that because there was no system of defensive performance measurement on which people can agree, or even systems which strongly agreed with each other, that defense goes either unaccounted for, or is overweighted. I was hoping to get a discussion of how we go about creating a whole player measurement given this fact.

Falls City Beer
09-29-2007, 10:45 AM
I posited that because there was no system of defensive performance measurement on which people can agree, or even systems which strongly agreed with each other, that defense goes either unaccounted for, or is overweighted. I was hoping to get a discussion of how we go about creating a whole player measurement given this fact.

All fair points. And your questions throughout have been good, important, and fair.

But I think before we can even begin to tackle the question of "how we account for defense when assigning value to the total player" there has to be some kind of agreement on the premises that guide determing defensive run valuation. I would argue that we aren't really even close to agreeing on appropriate premises, while others (you and jojo) seem to think we are quite close. Don't take that last sentence as an insult; it's not--I gather from our conversations that you two believe that the current defensive metrics are giving us a very clear picture of players' defensive value. Correct me if I'm wrong.

RedsManRick
09-29-2007, 10:57 AM
In general, I feel like many tools, like UZR, in the wrong hands, become mildly absurd--the tool's not ultimately the problem, I suppose, but the conclusions that are arrived at from their usage (Braun = -30 runs defensively) aren't being appropriately tested for validity.

I think it was Steel that pointed it out earlier, the challenge is that while -30 does seem obviously incorrect, how does one test for validity? What baseline do we use? What do we correlate it too? That's really the challenge. Not only do we not know what is right, we don't know how to figure it out and haven't agreed upon the system for measurement.

jojo
09-29-2007, 11:12 AM
If these are the same men that say in no uncertain terms that Ryan Braun is worth -30 runs defensively, then yes, their opinions need to have holes shot in them.

The by all means, start whenever you're ready. :cool:


But those men *aren't* saying that. Bill James is not THT. He would unequivocally shake his head at a methodology that arrived at that conclusion.

That's the problem: you're trotting out the heavy hitters, guys who are incredibly learned, schooled people in these matters and throwing them side-by-side with bloggers.

Dewan is the brainchild of the RZR that THT publishes BTW and it's difficult to separate Dewan from James in many regards.

Is it safe to assume your use of "blogger" isn't referring to paid consultants of major league baseball teams? If so, concerning the derivation of defensive runs, to be accurate, I've argued many times that RZR and by extension defensive runs derived from the data are inferior to advanced play-by-play based metrics such as UZR, Dewan's +/- and PMR.

To the best of my knowledge, I have never quoted Justin's defensive runs when evaluating a player's defense so really I don't think the above criticism is valid.

That said, Justin is an extremely bright fellow and knowledgeable baseball enthusiast. I doubt he'd ever equate himself to Bill James, but Justin has earned a great reputation for his efforts and simply discounting him as a "blogger" seems unfair. He's a true resource that Reds fans are lucky to have. Not reading his blog and talking baseball with him is a missed opportunity IMHO.


In general, I feel like many tools, like UZR, in the wrong hands, become mildly absurd--the tool's not ultimately the problem, I suppose, but the conclusions that are arrived at from their usage (Braun = -30 runs defensively) aren't being appropriately tested for validity.

BTW, neither UZR, Dewans' +/- system nor PMR has indicated Braun's true skill level is that of a -30 defender and MGL would suggest a partial season of data isn't enough to inform definitively on the subject.

jojo
09-29-2007, 11:25 AM
All fair points. And your questions throughout have been good, important, and fair.

But I think before we can even begin to tackle the question of "how we account for defense when assigning value to the total player" there has to be some kind of agreement on the premises that guide determing defensive run valuation. I would argue that we aren't really even close to agreeing on appropriate premises, while others (you and jojo) seem to think we are quite close. Don't take that last sentence as an insult; it's not--I gather from our conversations that you two believe that the current defensive metrics are giving us a very clear picture of players' defensive value. Correct me if I'm wrong.

I've continually argued for a spectrum approach with current defensive metrics and really advocate only trusting the most rigourous systems currently available. Even then, I usually give a range when positing a player's true defensive skill.

A "clear picture" can mean different things to each individual. The language i'd use is this-the current defensive metrics such as UZR, Dewan's +/- etc represent significant advances such that these metrics are good enough to inform defensive evaluation in a tangible, meaningful, and useful way. As an example, I think suggesting Dunn is between a -25 and a -15 defender captures his true defensive value pretty accurately (i.e. his true skill level is contained somewhere within those boundries). That should, I hope better define my thoughts on the precision of the current PBP-based defensive metrics.

SteelSD
09-29-2007, 12:25 PM
I adore discussions like this.

Personally, I've been won over by run expectancy and how the concept facilitates new avenues of understanding. That said, I agree wholeheartedly with you that runs created is a useful metric too. Truth be told, I've recently been pondering whether league average or replacement level is an appropriate baseline (which could ultimately result in an even greater reliance on RC as a metric by me in the future).

I like these discussions as well. Good stuff.

As for a "baseline", we actually could use the league average data and apply it to the RC formula. Then take the "missing" plays, derive base events; resulting in the probable additional bases allowed. As I don't believe that "missing" OOZ Plays are obvious Outs, I'm going to count them as .50 "missing" Plays at 3B. Yeah, that's an arbitrary number, but I'm just trying to work through some of the data noise.

Per the RC formula, each additional Base is worth .302 RC. Based on 2007 NL averages, the result of adding 55 Total Bases would be a RC adjustment of +16.56 Runs Allowed for Braun at the hot corner.

Now let's check out Troy Tulowitzki versus expected BIZ and OOZ Play rates:

BIZ Play variance: +20
OOZ Play variance: +26

And yes, I feel dirty just playing around with THT's data. ;)

And here's where we run into a problem from a positional standpoint. A Shortstop is less likely to have OOZ chances affected by park or other fielders, so it's possible (if not likely) that a higher percentage of his OOZ Plays would be Singles had he not gobbled them up. Secondly, I need a little help as to the number of bases each "missed" BIZ and OOZ play. Certainly, we'd see far fewer Doubles and virtually no Triples at this position. So I'm going to assume that 90% of these "missed" plays turn into Singles and 10% into the equivalent of Doubles (including overthrows from Throwing Errors). I'll certainly adjust if someone can provide me the actual data. And I'll apply a .80 OOZ modifier rather than the .50 I used for 3B.

The result is -15.66 Runs Allowed via the RC formula.

I won't begin to think that I'm anywhere near a finished product at this point, but it might just be a decent starting point.

RedsManRick
09-29-2007, 12:54 PM
I gather from our conversations that you two believe that the current defensive metrics are giving us a very clear picture of players' defensive value. Correct me if I'm wrong.

Sort of. I think that the existing metrics are all we have, and better than nothing. They can and should be improved, but I don't want to wait for the perfect metric before trying to incorporate defense in to holistic player valuation. It will absolutely be flawed. But flawed is better than nothing at all.

RedsManRick
09-29-2007, 12:56 PM
The result is -15.66 Runs Allowed via the RC formula.

I won't begin to think that I'm anywhere near a finished product at this point, but it might just be a decent starting point.

It's interesting, because again that sort of squares with the -16 and -20 estimates from earlier. We still lack a good way to test the metric, but if we can find something that we're all at least sort of comfortable with, it provides the ground for that discussion of:

Adam Dunn vs. Other OF who's worse offensive and better defensively.

Falls City Beer
09-29-2007, 12:58 PM
The by all means, start whenever you're ready. :cool:


Steel's taken an elephant gun to them.

I've yet to see you countervail his take. I'm not arguing that the data aren't worthwhile; I'm talking about conclusions.

As flyer85 pointed out some time ago in this thread--these current defensive metrics, in general, help us arrive at such conclusions as "Braun bad defense" and "Wright good defense."

This is probably "useful," as you put it, in making a determination of player "value" to a team. But it doesn't answer how useful. It attempts to, but when even a premise so fundamental and necessary as a baseline can't be established with anything like authority, don't you think a bit more time should be spent on establishing those primary premises before enormous leaps are made to conclusions built upon shaky premises?

Falls City Beer
09-29-2007, 01:01 PM
Sort of. I think that the existing metrics are all we have, and better than nothing. They can and should be improved, but I don't want to wait for the perfect metric before trying to incorporate defense in to holistic player valuation. It will absolutely be flawed. But flawed is better than nothing at all.

I've never not agreed with this position. Data are better than guesses based on nothing.

jojo
09-29-2007, 02:01 PM
Steel's taken an elephant gun to them.

I've yet to see you countervail his take. I'm not arguing that the data aren't worthwhile; I'm talking about conclusions.

As flyer85 pointed out some time ago in this thread--these current defensive metrics, in general, help us arrive at such conclusions as "Braun bad defense" and "Wright good defense."

This is probably "useful," as you put it, in making a determination of player "value" to a team. But it doesn't answer how useful. It attempts to, but when even a premise so fundamental and necessary as a baseline can't be established with anything like authority, don't you think a bit more time should be spent on establishing those primary premises before enormous leaps are made to conclusions built upon shaky premises?

I truthfully, don't agree with a majority of the way you've characterized this issue.

For instance, concerning a baseline for defense, it's crystal clear. Replacement level and league average are pretty much identical.

Also, why frame the debate about run values using -30 runs as a total? Since UZR has been published (2003) there have only been two players out of 564 who played over 60% of their team's games at a position that met/exceed +30 (Adam Everett; Carlos Beltran) and only three players who have met/exceeded -30 (Michael Young, Jr, Manny). With the exception of Manny, all of these guys play premium defensive positions with lots of chances. As alluded to earlier, with Manny, we know his numbers are skewed because of his environment.

Also, being confident that a player's true defensive skill is somewhere between 0 and -10 for instance, is much more than simply saying...he bad-other guy good. It's gives an informed basis for understanding how bad or how good which allows defense to be factored into the RS/RA equation during roster formulation to a degree that actually can be leveraged.

Anyway, that's my view from the cheap seats.

Falls City Beer
09-29-2007, 02:08 PM
I truthfully, don't agree with a majority of the way you've characterized this issue.

For instance, concerning a baseline for defense, it's crystal clear. Replacement level and league average are pretty much identical.

Also, why frame the debate about run values using -30 runs as a total? Since UZR has been published (2003) there have only been two players out of 564 who played over 60% of their team's games at a position that met/exceed +30 (Adam Everett; Carlos Beltran) and only three players who have met/exceeded -30 (Michael Young, Jr, Manny). With the exception of Manny, all of these guys play premium defensive positions with lots of chances. As alluded to earlier, with Manny, we know his numbers are skewed because of his environment.

Also, being confident that a player's true defensive skill is somewhere between 0 and -10 for instance, is much more than simply saying...he bad-other guy good. It's gives an informed basis for understanding how bad or how good which allows defense to be factored into the RS/RA equation during roster formulation to a degree that actually can be leveraged.

Anyway, that's my view from the cheap seats.

So it gives a range. How reliable is it from year to year? And what accounts for fluctuations in their range (besides park changes)?

Falls City Beer
09-30-2007, 12:28 AM
I'd also like to say that as far as defensive positions and their importance, catcher, shortstop, and CF are the most crucial positions, and it just so happens that the Reds are anywhere from moderately weak to downright awful in each of those areas. I think an upgrade at just two of those positions would make a marked improvement in the overall RA picture--and for relatively cheap.

I think one of the reasons the Cards will finish ahead of the Reds in the standings (aside from their bullpen) is the presence of a guy like Yadier Molina behind the plate instead of Ross or Valentin. I'd say at this juncture Molina's defense is more valuable than even Rolen's.

GAC
09-30-2007, 09:03 AM
I'd also like to say that as far as defensive positions and their importance, catcher, shortstop, and CF are the most crucial positions, and it just so happens that the Reds are anywhere from moderately weak to downright awful in each of those areas. I think an upgrade at just two of those positions would make a marked improvement in the overall RA picture--and for relatively cheap.

I think one of the reasons the Cards will finish ahead of the Reds in the standings (aside from their bullpen) is the presence of a guy like Yadier Molina behind the plate instead of Ross or Valentin. I'd say at this juncture Molina's defense is more valuable than even Rolen's.

Looking at it from solely a defensive perspective, I don't see any great disparity between the two. In fact, lookng at PO's, Ross has 80+ over Molina in less innings.

If you think Ross' defense is a liability or terrible, then you certainly can't put Molina's on the level of a Scott Rolen. Not comparing the numbers.


Inn PO A TE FE FPct DPS DPT
Molina 861 582 63 5 1 .991 7 3
Ross 837 662 50 7 0 .993 8 1

Now from a Win Shares perspective, Molina gets the nod....


Batting Fielding ExpWS WSP WSAB Total WS
Molina 10.0 5.3 13 .559 6 15
Ross 0.5 5.1 9 .296 -1 5

But what is dragging Ross' Total Win Shares down? His bat, not his defense.

RedsManRick
09-30-2007, 11:46 AM
I think what really drove me to consider this question is wanting a better sense of how much blame our ERA problem is a result of poor defense. We rail on our pitching over and over, and give lip service to the defensive issues. Of course, that's partly because you cannot separate defensive performance from offensive performance. Improving the defense means significantly altering the offense as well.

So the Reds are about 70 runs short of scoring as many runs as they'll allow. The basic question is, how do they bridge that gap? Obviously, the 2 basic answers are score more runs and/or allow fewer.

Let's say for a moment that we know Junior is going to get 450 PA and hit .270/.340/.475 in 2008. A wide variety of defensive measurements show that he's somewhere between average and quite poor in the field.

So, if we were able to replace that offensive production, how many runs would we prevent in the process? I think that's a very important question to answer. While I can appreciate all the problems highlighted so far, I'm extremely curious how those of you whose only contributions to this thread have been to discuss the uselessness of a given metric go about answer a question like this. What stat/metric/method do you use in your player valuation calculus?

Falls City Beer
09-30-2007, 12:22 PM
I think what really drove me to consider this question is wanting a better sense of how much blame our ERA problem is a result of poor defense. We rail on our pitching over and over, and give lip service to the defensive issues. Of course, that's partly because you cannot separate defensive performance from offensive performance. Improving the defense means significantly altering the offense as well.

So the Reds are about 70 runs short of scoring as many runs as they'll allow. The basic question is, how do they bridge that gap? Obviously, the 2 basic answers are score more runs and/or allow fewer.

Let's say for a moment that we know Junior is going to get 450 PA and hit .270/.340/.475 in 2008. A wide variety of defensive measurements show that he's somewhere between average and quite poor in the field.

So, if we were able to replace that offensive production, how many runs would we prevent in the process? I think that's a very important question to answer. While I can appreciate all the problems highlighted so far, I'm extremely curious how those of you whose only contributions to this thread have been to discuss the uselessness of a given metric go about answer a question like this. What stat/metric/method do you use in your player valuation calculus?

I would use the numbers to verify or strengthen the evaluations made by a consensus of independent scouting reports. Not the other way around. I suspect that's not the answer you're looking for, but I just don't know where to go with defensive metrics--there are just way too many variables to pin down accurately what a player does on defense and how that *specifically* leads to other teams putting runs on the board.

Steel, however, has raised a couple of interesting possibilities in this thread. I don't think the "critics" have been without contributions--go back and read the thread.

RedsManRick
09-30-2007, 01:46 PM
I would use the numbers to verify or strengthen the evaluations made by a consensus of independent scouting reports. Not the other way around. I suspect that's not the answer you're looking for, but I just don't know where to go with defensive metrics--there are just way too many variables to pin down accurately what a player does on defense and how that *specifically* leads to other teams putting runs on the board.

Steel, however, has raised a couple of interesting possibilities in this thread. I don't think the "critics" have been without contributions--go back and read the thread.

Without contributions? Of course not. Answering the question that was posed as the basis for the thread? Hardly.

Again, so you have a scouting report which says one player is good defensively and one is poor. The poor defensive player is a measurably more productive offensive player. Who do you want? How do you decide? We can, and frequently do say Adam Dunn is X more productive offensively than player Y. But as we've discussed, there's more to proper player valuation than just offensive performance. How do you incorporate defensive evaluation in to your overall player valuation?

Falls City Beer
09-30-2007, 02:12 PM
Without contributions? Of course not. Answering the question that was posed as the basis for the thread? Hardly.

Again, so you have a scouting report which says one player is good defensively and one is poor. The poor defensive player is a measurably more productive offensive player. Who do you want? How do you decide? We can, and frequently do say Adam Dunn is X more productive offensively than player Y. But as we've discussed, there's more to proper player valuation than just offensive performance. How do you incorporate defensive evaluation in to your overall player valuation?

As I said, I first consider as big a battery of scouting reports as I can find on a player, see how they compare with statistical measurements like UZR--then make a determination. But really, the question you're asking is to some extent operating in the realm of ideals--one player's defense/offense are inextricable/inseparable, and often you're talking about *choosing* between two players, not among 50 players. I don't think you need superadvanced metrics to make a determination whether to play Hopper or Dunn in left. And if you have to choose between an Ichiro and a Dunn, it's not a choice: you're going to make space for BOTH of them, come hell or high water.

I think your concern really only applies to the players that a GM is going to go out and acquire; 99 times out of 100, a simple consensus will tell you shouldn't even HAVE to choose between a Dunn and an Ichiro. Or a Dunn and a healthy Hamilton. Or a Hamilton and a Junior.

RedsManRick
09-30-2007, 08:49 PM
But it's not just at the highest levels of roster construction. It's Juan Castro (or the imagined Juan Castro) vs. Jeff Keppinger too. And it's not necessarily on the same level of two equivalent (?) players. On his own, is Dunn really worth 13M? If he just a 3 win player as Jojo asserts (+5 offensive, -2 defensive), then I would assert he isn't. A 4 win (+3 offensive, +1 defensive) would be more valuable. It really comes in to every aspect of player decisions, not just FA signings. Arbitration, trades, etc. You have to factor in the value defense in terms of dollars and cents somehow.

jojo
09-30-2007, 09:13 PM
But it's not just at the highest levels of roster construction. It's Juan Castro (or the imagined Juan Castro) vs. Jeff Keppinger too. And it's not necessarily on the same level of two equivalent (?) players. On his own, is Dunn really worth 13M? If he just a 3 win player as Jojo asserts (+5 offensive, -2 defensive), then I would assert he isn't. A 4 win (+3 offensive, +1 defensive) would be more valuable. It really comes in to every aspect of player decisions, not just FA signings. Arbitration, trades, etc. You have to factor in the value defense in terms of dollars and cents somehow.

I think $13M for Dunn's option year is fair market as wins are likely to go for $4M this off season and his option year isn't risk adverse as it's reasonable to expect him to be a 4.5 win bat next season. Since I think he's a -15 to -25 glove, he's likely to be near a 3 win player again next season. There is always a reasonable chance he'd be a 5.5 win bat next season too in which case he'd be underpaid again (in my estimation it was only in '06 where he was overpaid as a Red and he'd been dramatically underpaid earlier).

What I wouldn't do is pay him market value for a long term deal based upon this argument- his defense at best will remain static and he's right at the cusp of offense that can carry his bat at a market salary. Unfortunately, projection systems suggest '08-'09 may be the last season before an offensive decline begins for him. If that comes to pass, he could be overpaid as early as '09.

Obviously, he could be a three outcome guy that doesn't age early. It's just that given the massive size of his next contract, there's more risk then I'd be willing to take given a payroll that looks like $70M will be the high side. I'd take the comp picks after '08.

jojo
09-30-2007, 09:42 PM
But it's not just at the highest levels of roster construction. It's Juan Castro (or the imagined Juan Castro) vs. Jeff Keppinger too. And it's not necessarily on the same level of two equivalent (?) players. On his own, is Dunn really worth 13M? If he just a 3 win player as Jojo asserts (+5 offensive, -2 defensive), then I would assert he isn't. A 4 win (+3 offensive, +1 defensive) would be more valuable. It really comes in to every aspect of player decisions, not just FA signings. Arbitration, trades, etc. You have to factor in the value defense in terms of dollars and cents somehow.

I'm just not seeing too many 4 win players in the FA market. Bonds, Hunter, and AROD look like the only guys that are *safe bets* for '08 (though all bets would be off for Hunter and Bonds in '09). Bonds could probably be had for a single year but I'm doubting any of the three could be signed for less than $4M a win.

Then consider, the infield really is *set* (Votto, Phillips, Gonzo, EE) and two outfield spots should be spoken for (Hamilton, Bruce). Really if Dunn is not in the mix the Reds would be paying more for their third outfielder hoping to squeeze an extra win out of them.

There are ALOT of ways to get to 3-4 wins. That said, the Reds already have something close to that in Dunn at roughly fair market money in '08 and he only requires a one year commitment. In my mind, that's the definition of managing risk.

Besides, Dunn's second half has forced the Red's hand. I suspect the conversations in the FO are closer to arguments about the merits of an extension then they are about the best way to get out from under his contract.

LoganBuck
09-30-2007, 11:21 PM
Defensively, no.

Offensively, I'd definitely say he is.

Ryan Braun is only a 3B for the Brewers b/c Prince Fielder plays 1B and they are in the NL.

And they drafted Matt LaPorta!

paulrichjr
10-01-2007, 09:49 AM
Say what? Is this true? This team tied a record for fewest errors????


http://news.enquirer.com/apps/pbcs.dll/article?AID=/20071001/SPT04/710010341/1071

IT'S A TIE: Edwin Encarnacion's error in the seventh inning Sunday gave the Reds 95 on the year. That tied the club record for the fewest errors, set in 1977.

westofyou
10-01-2007, 11:38 AM
Say what? Is this true? This team tied a record for fewest errors????


http://news.enquirer.com/apps/pbcs.dll/article?AID=/20071001/SPT04/710010341/1071

IT'S A TIE: Edwin Encarnacion's error in the seventh inning Sunday gave the Reds 95 on the year. That tied the club record for the fewest errors, set in 1977.



The best player in a nine is the player that makes the most good plays in a match, not the one who commits the fewest errors.

Henry Chadwick 1870

I'd say that can apply across the board to the Reds team defense.

RedsManRick
10-01-2007, 11:44 AM
Perfect supporting evidence for the case that errors are a poor, poor way to measure defense.

paulrichjr
10-01-2007, 01:04 PM
Perfect supporting evidence for the case that errors are a poor, poor way to measure defense.

I completely 100% agree with what you just said but I also think that it is amazing that this team could tie the record for fewest errors ever by a Reds team. I'm not saying that it was a great team in the defense department. I am wondering though if they are as bad as I would have thought at the beginning of the season.

Look at some of the positions:

Phillips - As good as any 2nd baseman on defense as there is
Encarnacion - After his trip to Louisville he was very very good in my opinion.
Agon - When he was out there he made his share of errors but he still played above average defense.
Hatte - Did a good job on defense at 1B
Ross - Another fairly good player on defense ie above average.
CF (Hopper, Hamilton, Freel, etc) - They all did average or better
LF and RF - Weak on range but Dunn improved dramatically...Griffey was much better in RF than in CF.

If you had asked me two days ago, "Are the Reds an average defensive team?" I would have responded with a no. In fact I would have thought that they were a really bad defensive team. Now after thinking about it...I am not sure.

RedsManRick
10-01-2007, 01:25 PM
The thing is though Paul, if you look at a simple measure, team defensive efficiency (ie. outs per ball in play), the Reds clock in at 28th of 30 (.675 vs an MLB average of .689 and NL average of .691). If they aren't making many errors, which can loosely be defined as screwing up a sure out, then why aren't they creating outs? For starters, we can see easily that the Reds are among the worst at allowing line drives, which far and away turn in to hits at the greatest rate. Though I am curious, does anybody know if you can find a DER broken out by hit type? I imagine the Reds do just fine on grounders and got killed on fly balls.

Here's an exercise I'd love to go through when I have the time -- though unfortunately I won't for a while. A few of the criticisms of defensive metrics that hav been brought up in this thread include:

1.) Year-to-year variability
2.) Metric to metric variability
3.) Metric vs. Scouting differences
4.) Specious Metric definitions

A lot of people have taken a boil the ocean approach, trying to create new metrics and look at correlation coefficients, team performance, etc. We can agree, I think, that the spread of variables in play and the difficulty of defining standard measures makes that quite difficult.

So, I propose we collectively take up the case of a single player and examine these 4 critiques in depth. How did he do on the various metrics? What did he do measurably better/worse this year? Did the scouting report on his defensive ability change? Do the stats match up with the scouting report? Can we isolate his abilities from his opportunities? How much inherent "noise" or variability is there due to simple sample size issues? Let's pick this sucker apart and see what we can tease out.

I'll propose that we use BP. Anybody interested?

Falls City Beer
10-01-2007, 01:31 PM
The thing is though Paul, if you look at a simple measure, team defensive efficiency (ie. outs per ball in play), the Reds clock in at 28th of 30 (.675 vs an MLB average of .689 and NL average of .691). If they aren't making many errors, which can loosely be defined as screwing up a sure out, then why aren't they creating outs?

Here's an exercise I'd love to go through when I have the time -- though unfortunately I won't for a while. A few of the criticisms of defensive metrics that hav been brought up in this thread include:

1.) Year-to-year variability
2.) Metric to metric variability
3.) Metric vs. Scouting differences
4.) Specious Metric definitions

A lot of people have taken a boil the ocean approach, trying to create new metrics and look at correlation coefficients, team performance, etc. We can agree, I think, that the spread of variables in play and the difficulty of defining standard measures makes that quite difficult.

So, I propose we collective take up the case of a single player and examine these 4 critiques in depth. How did he do on the various metrics? What did he do measurably better/worse this year? Did the scouting report on his defensive ability change? Do the stats match up with the scouting report? Can we isolate his abilities from his opportunities? How much inherent "noise" or variability is there due to simple sample size issues? Let's pick this sucker apart and see what we can tease out.

I'll propose that we use BP. Anybody interested?

Sounds interesting. I might make one recommendation, though: choose a player with a longer MLB resume than BP. Or maybe you deliberately chose BP because of his short resume.

RedsManRick
10-01-2007, 01:35 PM
Sounds interesting. I might make one recommendation, though: choose a player with a longer MLB resume than BP. Or maybe you deliberately chose BP because of his short resume.

Yeah, I'd actually like to avoid the situation of actual ability improvements or decline clouding the measurement issues. Though I wouldn't begrudge another choice, especially if somebody was willing to do some of the number crunching.

Cooper
10-03-2007, 12:40 AM
Steel: did you factor in added plate appearances by a team when the player doesn't get to a ball?...that hit would be worth about a good bit (.40-.50 runs) Plus you would then add the value of the extra out a now has (.25 runs or there abouts). I could see how it adds up and thus makes a differnce in players value (i.e. Ryan Braun).

Johnny Footstool
10-03-2007, 11:50 AM
Steel: did you factor in added plate appearances by a team when the player doesn't get to a ball?...that hit would be worth about a good bit (.40-.50 runs) Plus you would then add the value of the extra out a now has (.25 runs or there abouts). I could see how it adds up and thus makes a differnce in players value (i.e. Ryan Braun).

I'm still missing the reason why extra plate appearances are assigned additional value because of a missed out. This seems like a fallacy to me.

Reaching base creates part of a run, regardless of how you do it. Making an out creates 0 runs, regardless of how you do it.

Now, if someone can show proof that reaching base on an error actually increases your chance of scoring more than hitting a single, I might change my tune. But statistically speaking, shouldn't each plate appearance be considered independently of all other plate appearances? So even if three consecutive batters reach base on 1-base errors, if the next batter hits a single, he's still only created .25 (or so) of a run.

RedsManRick
10-03-2007, 01:19 PM
I'm still missing the reason why extra plate appearances are assigned additional value because of a missed out. This seems like a fallacy to me.

Reaching base creates part of a run, regardless of how you do it. Making an out creates 0 runs, regardless of how you do it.

Now, if someone can show proof that reaching base on an error actually increases your chance of scoring more than hitting a single, I might change my tune. But statistically speaking, shouldn't each plate appearance be considered independently of all other plate appearances? So even if three consecutive batters reach base on 1-base errors, if the next batter hits a single, he's still only created .25 (or so) of a run.

A. Bases empty, No Outs
B. Bases empty, One Out
C. Man on first, No Outs

The thing is, by definition the error is not just a hit, but it is a lost out. A normal hit also is not an out, but it is not judged against the outcome of having been out, whereas the error is.

The standard hit is considered a swing of run expectancy from A, the starting state, to C, the final state. The error is considered a swing of run expectancy from B, the should-have-been state, to C, the final state.

It's this extra value lost that makes an error an error. If there wasn't additional value lost, then we wouldn't have errors at all, because we'd be working from a starting assumption that all at bats should be outs.

Another way to look at it is that all batted balls are pulled from two buckets -- balls which should become outs and balls which should become hits. We can count on about 70% of batted balls to come from the former bucket. Once the ball is batted, it gains the value associated with being from one of these buckets. The outcome then has be judged against the bucket from which that ball was drawn.

So, let's say that your pitchers strike out 6 batters in a game. There are 21 outs remaining which must come from balls in play and we know that, in general, 30% of balls in play become hits. This is based on an understanding of the distribution of balls that are hit and the likelihood by which they turn in to outs. This means, we will need 30 balls in play to get 21 outs.

If we go by your logic, and simply equate the error to a hit, that 30% of balls which become hits has just increased because our underlying understanding of what the expected hit rate on balls in play has changed. If that's the case, we can expect the other team to get more value from their balls in play, which leads to more runs than we previously expected.

Another way to look at it is that the game won't be over until we've picked 21 balls from the out bucket. You pick 3 hits, then 7 outs, 3 hits, 7 outs, 3 hits, 7 outs, game over. When you make an error, you picked a ball from the out bucket, but it ended up being a hit ball. So instead it looks like this: 3 hits, 7 outs, 3 hits, 1 error, 7 outs, 3 hits, 7 outs. Instead of it taking 30 balls to get those 21 outs, it took 31. The error functioned like a hit but it didn't come out of the hit bucket. Therefore, it had additional cost beyond just allowing another hit.

Cooper
10-03-2007, 02:07 PM
Good post RMR.

Falls City Beer
10-03-2007, 02:43 PM
A. Bases empty, No Outs
B. Bases empty, One Out
C. Man on first, No Outs

The thing is, by definition the error is not just a hit, but it is a lost out. A normal hit also is not an out, but it is not judged against the outcome of having been out, whereas the error is.

The standard hit is considered a swing of run expectancy from A, the starting state, to C, the final state. The error is considered a swing of run expectancy from B, the should-have-been state, to C, the final state.

It's this extra value lost that makes an error an error. If there wasn't additional value lost, then we wouldn't have errors at all, because we'd be working from a starting assumption that all at bats should be outs.

Another way to look at it is that all batted balls are pulled from two buckets -- balls which should become outs and balls which should become hits. We can count on about 70% of batted balls to come from the former bucket. Once the ball is batted, it gains the value associated with being from one of these buckets. The outcome then has be judged against the bucket from which that ball was drawn.

So, let's say that your pitchers strike out 6 batters in a game. There are 21 outs remaining which must come from balls in play and we know that, in general, 30% of balls in play become hits. This is based on an understanding of the distribution of balls that are hit and the likelihood by which they turn in to outs. This means, we will need 30 balls in play to get 21 outs.

If we go by your logic, and simply equate the error to a hit, that 30% of balls which become hits has just increased because our underlying understanding of what the expected hit rate on balls in play has changed. If that's the case, we can expect the other team to get more value from their balls in play, which leads to more runs than we previously expected.

Another way to look at it is that the game won't be over until we've picked 21 balls from the out bucket. You pick 3 hits, then 7 outs, 3 hits, 7 outs, 3 hits, 7 outs, game over. When you make an error, you picked a ball from the out bucket, but it ended up being a hit ball. So instead it looks like this: 3 hits, 7 outs, 3 hits, 1 error, 7 outs, 3 hits, 7 outs. Instead of it taking 30 balls to get those 21 outs, it took 31. The error functioned like a hit but it didn't come out of the hit bucket. Therefore, it had additional cost beyond just allowing another hit.

I see what you're saying, but that seems like an abstraction of a pretty high order. Then add in the capriciousness of what an error "is" and what an error "isn't" (based on the scorekeeper's definition), and you've got a fairly dicey proposition.

Maybe it all evens out over time--that is to say, maybe over time the vast majority of errors should be "outs," but I don't know....

I pretty well dismiss the "error" ruling out of hand. There's no consistency in its interpretation. Perhaps you should label them "balls that should have become outs."

Johnny Footstool
10-03-2007, 04:40 PM
Another way to look at it is that all batted balls are pulled from two buckets -- balls which should become outs and balls which should become hits. We can count on about 70% of batted balls to come from the former bucket. Once the ball is batted, it gains the value associated with being from one of these buckets. The outcome then has be judged against the bucket from which that ball was drawn.

That seems like the old "Gambler's Fallacy" to me -- like if you flip a coin 100 times, 50 of those would be heads and 50 tails. We know that isn't true.

Even if we assume that every ball in play has a 70% chance of becoming an out, we can't assume that because one of those batted balls was an error, then the out is somehow "used up."

As for the run expectancy matrix, I can sort of believe in using the difference between the values of having a runner on vs. not having a runner on, but those values are not constant based on the situation -- in other words, committing a one-base error with one out is a lot less damaging than committing that same error with no outs.

Another problem with relying on the run expectancy matrix is that it does not consider each plate appearance as a discrete event. Instead, it shows what happens from a given point through the rest of the inning. A myriad of other events can affect run scoring during that time frame -- events that can make the error or missed play moot, but still result in a run, thus inflating the supposed value of the error.

Because there are so many factors involved in reaching the supposed value of a missed play/extra out, I think we need to view each ball in play as a discrete event, and calculate run values accordingly.

RedsManRick
10-03-2007, 05:51 PM
That seems like the old "Gambler's Fallacy" to me -- like if you flip a coin 100 times, 50 of those would be heads and 50 tails. We know that isn't true.

Even if we assume that every ball in play has a 70% chance of becoming an out, we can't assume that because one of those batted balls was an error, then the out is somehow "used up."


I addressed that point, albeit awkwardly. It's not a gambler's fallacy because there actually isn't a fixed percentage outcome with every ball hit in to play. Balls that become errors are balls that otherwise had a 100% chance of becoming an out. So, all the hits that otherwise would have happened still happened, but now you add another to the pile that you otherwise would've had counted as an out. This requires making another out to make up for the one that you turned in to a hit.

The assumption of course is that the distribution of balls that should turn in to hits/outs is somewhat constant around 30%. So by making an error, you've changed that math, making "hits" actually more frequent.

Going back to your heads/tails comparison, making an error is like having the coin turn up heads, but you accidentally writing down a tails. After a million throws, where you should've had 500k heads and 500 tails, you now have 499,999 heads and 500,001 tails. You're right, the probability of throwing a heads or tails doesn't change because you made that error, but you've changed one of the outcomes that 50% system would have produced, so your end outcome is going to be slightly off from a "pure" 50/50.

Let's simplify it using an example of 10 coin flips. The average number of heads will be 50%*10=5. If we guarantee that one of the flips is a head (the error), the average number of heads is 50%*9+100%*1=5.5.

However, back to my earlier point, unlike coinflips, not all balls in play are created with equal hit likelihoods. Errors only occur on balls that had a 100% (or close) likelihood of being outs. So you're not just creating another hit but you're guaranteeing a not-out.

So you need 21 outs to end the game. On average, that will take 30 balls in play. If know absolutely that there will be an error, that becomes 31 balls in play.

21 outs = 0% chance of out * 1 BIP +70% chance of out * 30 BIP.

So the value of that error isn't just the hit which it becomes. It's the out which it did not become.

As for your last point, this is true. However, unless you think that errors are not normally distributed across the various base/out scenarios, then it all comes out in the wash. Sometimes they'll be more costly, sometimes less, but the averages still line up.

In terms of measuring the effect of a single player's errors, I think you're absolutely right. It's quite possible that one guy made 10 errors which led to 2 runs and another guy made 5 errors which led to 10 runs. However, as the player (arguably) does not pick and choose when he makes his errors, when we are trying to measure his ability, we don't want to give extra or less credit based on scenario beyond the player's influence.

D-Man
10-04-2007, 01:21 AM
A few additional thoughts about a really nice thread. . .

1.) What is needed are a collection of *approaches* to evaluating defense, rather than metrics. (I make a distinction here because evalution *approaches* and evaluation *metrics* are materially different concepts.)

When I evaluate a player (or team), here is the general framework I would like to use:

(a) How many plays twere made
(b) Which plays were not made, i.e., where (and how many) hits fell and their associated cost
(c) How the player handled position-specific responsibilities (e.g., fielding bunts for corner infielders, fielding and relaying double plays for middle infielders, throwing out baserunners for OFers, digging out infield throws for 1B, etc.)
(d) What are the interactive effects among players on the diamond, and what is the value of the components of these effects. For instance a groundball out isn't just a play, it is a *process*, with several contributors to that process. Pitcher pitches --> hitter hits ball to shortstop --> shortstop fields --> throws to first. We shouldn't judge the defensive player, independent of the process, unless we better understand the out creation that goes into each step of the process.

Unfortunately, there is no silver bullet for all of these needs. Most approaches or metrics cover how many plays a player makes, and even those discussions get squirrelly with all sorts of wild adjustments, such as handedness of pitching staff, groundball/flyball tendencies, etc. It appears that the discussion usually begins here and ends pretty quickly.

Some systems captures some of (b), but John Dewan's approach is the only one that quantifies (c). I think very little, if any, research has been done on (d). It's largely uncharted territory, which is unfortunate.

2.) What's missing with defensive statistics is the power of description. When I look at a player's offensive numbers, I have a good sense of what kind of player he is, what skills he has, and how he approaches a plate appearance. In contrast, we can infer very little, in a descriptive sense, from defensive statistics. How deep or shallow does he generally play? Does he have a strong arm? Is it accurate? Is he good making plays while diving plays to his left? How quick is his first step? I see very few statistics that help me to *understand* a given player's defense and defensive skills. That's a shame.

3.) With regard to the outliers (e.g., -30 runs for Braun), I wouldn't want to throw the baby out with the bathwater. A much better approach would be to triangulate the result with other metrics and approaches, understand how the systems are different, and determine why different systems value certain players in extremely different way.

4.) In regard to RMR's persistent question in the thread (which is a good one, BTW), I think that we are slowly moving toward evaluating the run value of defensive statistics. As we should.

However, will we ever get to the point of being able to evaluate relative tradeoffs among various contributors at different positions? Say you have the opportunity to trade a +20 LF for a +20 SS (assume all else equal). Which player would have a higher value, or would they be the same??

I see two more issues at play that are not considered in this defensive value framework. One, the LF will have fewer opportunties than the SS, so the LF will make a bigger impact per opportuntity in the field. In other words, the LF has more of a superstar defensive quality than the shortstop, which should have some inherent value. The second issue is relative scarcity. Finding an average SS defender is moderately easy. Finding a +20 defensive SS is extremely difficult, and much harder than finding a +20 LF because good defenders can move easily from a harder position to a less demanding defensive position, like LF. Therefore, a +20 SS is in extremely short supply, as he is likely one of the handful of best defenders on the planet.

How do we account for these factors with the runs scoring framework?? How do you value the superstar factor, by standard deviation? Relative scarcity isn't fixed, it changes over time. Do you track these changes?

Johnny Footstool
10-04-2007, 10:50 AM
In terms of measuring the effect of a single player's errors, I think you're absolutely right. It's quite possible that one guy made 10 errors which led to 2 runs and another guy made 5 errors which led to 10 runs. However, as the player (arguably) does not pick and choose when he makes his errors, when we are trying to measure his ability, we don't want to give extra or less credit based on scenario beyond the player's influence.

The system using Run Expectancy does just that. The numbers are based on a scenario (an entire inning) that is beyond the player's influence.

RedsManRick
10-04-2007, 11:27 AM
How do we account for these factors with the runs scoring framework?? How do you value the superstar factor, by standard deviation? Relative scarcity isn't fixed, it changes over time. Do you track these changes?

I would argue that scarcity doesn't enter the equation until the valuation stage, and we're stuck on measurement.

We can agree that Dunn's and Jorge Posada's .900+ OPSs are worth about the same from an offensive standpoint. However Posada's defensive superiority and scarcity make him a more valuable player.

The problem isn't comparing to plus fielders, but comparing to different players. Who is really more valuable strictly from a wins and losses perspective; the .900 OPS LF who comes in at -10 or the .775 OPS SS who comes in at +15? Throw scarcity in to the pool, add agents and market conditions, and now you have a pricing schema.

Essentially this is what the Moneyball approach is. Figure out what combination of skills gives you the greatest overall production value for the lowest cost. In 2000, OBP wasn't being properly valued. That was comparatively easy to say because we have robust models showing the role of OBP in run production. Getting to the same place with defense is going to be much much tougher.

SteelSD
10-04-2007, 01:24 PM
The system using Run Expectancy does just that. The numbers are based on a scenario (an entire inning) that is beyond the player's influence.

Yep. One cannot use the Run Expectancy matrix to determine "missed" play value. You're certainly right in noting that the remainder of events in an inning are beyond the player's control- including the number of Outs when a play is "missed".

We can say that the Inning is likely to produce between 1 and 2 Runs should a hitter double with none Out (RA = 1.1385). The double is a contributing factor. However, we cannot say that a no-Out double is an event worth 1.1385 Runs.

RedsManRick
10-04-2007, 04:55 PM
Yep. One cannot use the Run Expectancy matrix to determine "missed" play value. You're certainly right in noting that the remainder of events in an inning are beyond the player's control- including the number of Outs when a play is "missed".

We can say that the Inning is likely to produce between 1 and 2 Runs should a hitter double with none Out (RA = 1.1385). The double is a contributing factor. However, we cannot say that a no-Out double is an event worth 1.1385 Runs.

Good point Steel. However, could you use the difference between the existing state and the new state to assign a run value, and go from there?

So, for the leadoff double, if the run expectancy for the inning is .5165 and the double makes it 1.1385, then the double was worth .622 runs. If the next guy makes an out, the RE drops to .6911. So the out was worth .4474 expected runs prevented.

Would this work? (and I think this is essentially being done with WPA for a win based metric). Using the RE structure can we assign the RE change at the end of each play to a player(s) based on their responsibility for the outcome? So if a pitcher strikes somebody out and RE drops from 0.5165 to 0.2796, we credit the pitcher with .2369 runs prevented. If the pitcher allowed a single which changed the RE from 0.5165 to 0.8968, he gets credit for .3803 runs allowed.

So, in our error situation, A = out, B = error.

Starting state: Man on 2B, none out. RE = 1.1385
Resulting state A: Man on 2B, one out. RE = 0.6911 (-0.4474)
Resulting state B: 1 run scored, man on first, none out. RE 1.8968 (+.7583)

Thus, if the fielder had made the out, we could have given him .4474 runs prevented. However, since he (and not the pitcher) was responsible for the outcome of the play, he gets credit for .7583 runs allowed.

I can see where you'd need to adjust for the assist versus the putout (apply some portion to both the fielder and the 1B -- though I can see a variable weighting based on in zone or out of zone coming in to play here...). You can work off the assumption that the ball "should" be put in play every AB, so that the pitcher simply is getting full credit for all Ks and BBs. In this way, every single run accrued by the opposing team will have been credited to various players.

So, you allowed 5 runs versus an expected 4.65. There's a total of -.35 ERD (expected run difference) to be distributed among the fielders, including the pitcher.

Maybe this has already been done, but it seems like it could be hard coded pretty easily, since WPA does that. What I don't like about WPA is it gives too much credit for context. In my opinion, a solo shot in the first which gives you a 1 run lead should not be more than a solo shot walk off. Going on the run basis wouldn't remove this context dependency entire, but it does seem like it would minimize it substantially.

I have a sneaking worry about not properly controlling for specific player opportunity functioning as a multiplier. Perhaps you can bring range in to play by multiplying their final total by the ratio of ball in zone to expected balls in zone....

What about a hit type adjustment to account for the fact that some pitchers allow less dangerous balls in play?

Am I crazy? Is there something wrong in my logic? Am I just avoiding my real work..............?

Johnny Footstool
10-04-2007, 05:05 PM
Good point Steel. However, could you use the difference between the existing state and the new state to assign a run value.

You're still attaching run value to the state of the game instead of to the player, though.

If SS a makes 3 errors in a game, but all of them result in a runner on first with 2 outs, he'd have allowed 0.351 runs (0.117 x 3). If another SS makes only one error in a game, but it happened to allow the leadoff hitter to reach base, he'd have allowed .555 runs.

The timing of the error should have no bearing on our evaluation of the fielder.

RedsManRick
10-04-2007, 05:20 PM
The timing of the error should have no bearing on our evaluation of the fielder.

I think that's up for discussion. There is a philosophical discussion of whether you should measure the performance of a player within the context of the game or just measure raw ability based on in game performance.

Using a model such as mine, I guess I figure it all comes out in the wash. If you have a 3B on a team where his pitchers get 26 Ks per game and then allow a bip to 3B with 2 outs in the 9th, he never has much of a chance to show himself as very good or very bad.

Conversely a guy whose pitchers get lit up and frequently has guys on base and gets the ball hit to him a lot will have a lot of opportunity to prove himself bad or good.

In either case, it's what he does in those circumstances that will create our evaluation of him.

Outside of account for specific situations, I'm not sure how you can possibly have any measurement of defense which is measured in runs.

Unless you want do something like creating average run values for balls hit to every area of the field, just count balls fielded or not fielded in every area and do the math. Pretty sure that metric already exists though and that lots of people don't like it.

jojo
10-04-2007, 07:40 PM
The way linear weights handles the issue is pretty straightforward..... basically take the weighted averages of events to determine run values so it all works out. Is it a perfect reflection of that shortstop for the inning in question? Nope. But over the course of a season, it works well.

pedro
10-04-2007, 07:54 PM
I have a question. If you took a team and estimated the number of runs that team would have allowed over the course a whole season in a park and defense neutral environment (such stats exists correct?) and then aggregated the defense plus/minus numbers for every player who played for them over the course of the season, the differential between the number of runs that the pitching staff surrendered and the number of runs that they theoretically should have surrendered should match the aggregated defensive plus/minus numbers. Shouldn't they?

westofyou
10-04-2007, 08:05 PM
I have a question. If you took a team and estimated the number of runs that team would have allowed over the course a whole season in a park and defense neutral environment (such stats exists correct?) and then aggregated the defense plus/minus numbers for every player who played for them over the course of the season, the differential between the number of runs that the pitching staff surrendered and the number of runs that they theoretically should have surrendered should match the aggregated defensive plus/minus numbers. Shouldn't they?


http://www.metroactive.com/papers/metro/02.13.97/gifs/slingblade-9707.jpg

RFS62
10-04-2007, 08:07 PM
http://www.metroactive.com/papers/metro/02.13.97/gifs/slingblade-9707.jpg

What's he asking GAC for?

pedro
10-04-2007, 08:12 PM
What's he asking GAC for?

:lol:

SteelSD
10-05-2007, 01:30 AM
Good point Steel. However, could you use the difference between the existing state and the new state to assign a run value, and go from there?

Any valuation of defense has to be "state-neutral", because the defender has absolutely no control over it. I understand what you're saying about Inning Out state. And I've read your posts beyond the one I'm quoting. But I'd suggest that things don't necessarily "even out" over time- particularly because we've got eight other defense players who affect Out state. Especially on a bad defensive team, other defenders may compound any fielder's numbers if we use a Run Expectancy analysis. Let's take a look at one of your prior examples:


So, for the leadoff double, if the run expectancy for the inning is .5165 and the double makes it 1.1385, then the double was worth .622 runs. If the next guy makes an out, the RE drops to .6911. So the out was worth .4474 expected runs prevented.

Now, let's say that a bad Outfielder...we'll call him "Ken Griffey Jr."... allows a leadoff single by not catching a ball in front of him; resulting in a "missed" BIZ play with none Out. The RE gain for the opposition, assuming a no-Out none-On "neutral" state of .5165, would be +.3803. At this point, RE is .8968. Next, Edwin Encarnacion double-clutches on a bunt and fails to throw the runner out at 1B. Now we have a RE of 1.4693 if the lead runner only advances to 2B. The RE gain for the opposing team is now +.5725 for that play versus the previous state (Runner on 1B/none Out). Simply put, the 3B in that scenario had zero control over the previous play and may have only slight control over base advancement if the lead runner advances past 2B, but is being assigned an additional .1922 negative Runs at minimum (if the lead runner stops at 2B) if we use a RE differential methodology.

And don't even get me started on situation-based WPA. It's the ultimate warping of metrics based on game state outside of player control.


Am I crazy? Is there something wrong in my logic? Am I just avoiding my real work..............?

Hey, I prep myself for real work and wind down from real work by doing this kind of stuff. I'm weird that way. ;)

Wheelhouse
10-05-2007, 01:51 AM
Don't you'd ever suggest that offensive numbers don't obliterate everything else!

jojo
10-05-2007, 07:56 AM
But I'd suggest that things don't necessarily "even out" over time- particularly because we've got eight other defense players who affect Out state. Especially on a bad defensive team, other defenders may compound any fielder's numbers if we use a Run Expectancy analysis.

With UZR, the quality of the defense surrounding a player does not affect his rating (although there may be a rare case where a player is next to a ball hog but those situations are easily, easily spotted). A player is only responsible for the BIP within his zones and he's compared only to other players who play his position. Run values are derived using run expectancy data, but the run values for each event are determined using league wide data across multiple years. Data are also park adjusted.



Let's take a look at one of your prior examples:

Now, let's say that a bad Outfielder...we'll call him "Ken Griffey Jr."... allows a leadoff single by not catching a ball in front of him; resulting in a "missed" BIZ play with none Out. The RE gain for the opposition, assuming a no-Out none-On "neutral" state of .5165, would be +.3803. At this point, RE is .8968. Next, Edwin Encarnacion double-clutches on a bunt and fails to throw the runner out at 1B. Now we have a RE of 1.4693 if the lead runner only advances to 2B. The RE gain for the opposing team is now +.5725 for that play versus the previous state (Runner on 1B/none Out). Simply put, the 3B in that scenario had zero control over the previous play and may have only slight control over base advancement if the lead runner advances past 2B, but is being assigned an additional .1922 negative Runs at minimum (if the lead runner stops at 2B) if we use a RE differential methodology.

With UZR (or any metric that derives run values from lwts), that hypothetical inning would be one data point in a sample consisting of about 10,000.

I understand that some might find it distasteful to apply weighted average run values to a metric. But really, attributing a run value to a fielding event because on average over the last X number of seasons the associated change in the base out state has resulted in Y runs isn't any more abstract than using a metric like RC to measure offense. If lwts-based metrics bother people, then win shares really should drive those same people batty....


Hey, I prep myself for real work and wind down from real work by doing this kind of stuff. I'm weird that way.

I prefer to wind down by reading a good book while driving (just kidding)... :cool:


:beerme:

RedsManRick
10-05-2007, 09:38 AM
Now, let's say that a bad Outfielder...we'll call him "Ken Griffey Jr."... allows a leadoff single by not catching a ball in front of him; resulting in a "missed" BIZ play with none Out. The RE gain for the opposition, assuming a no-Out none-On "neutral" state of .5165, would be +.3803. At this point, RE is .8968. Next, Edwin Encarnacion double-clutches on a bunt and fails to throw the runner out at 1B. Now we have a RE of 1.4693 if the lead runner only advances to 2B. The RE gain for the opposing team is now +.5725 for that play versus the previous state (Runner on 1B/none Out).

Ah, but if EE does make the play, he would've been credited with .8968 - .5487 = +3481. If he had made the out with the bases empty, it would have only been worth .2189. So yes, because of the hit, the value of the next ball hit to him has been changed. So while he's been placed in a higher leverage situation, he still has the opportunity to either benefit or be harmed by what he does next.

That's what I was talking about in terms of it being a scale modifier. Given two defenders who make the exact same number of plays and never miss one, the one who makes his play in higher leverage situations (because of worse teammates) will look like a better defender.[/quote]

I think it's clear that any metric trying to estimate run contributions will be skewed by specific opportunity context, and thus need have the performance events treated in a context independent manner.

So it becomes an issue of categorizing the set of possible defensive events, assigning run values to each event, and then doing the math. Isn't this what UZR does? I don't see a way around this.

Johnny Footstool
10-05-2007, 10:35 AM
Run values are derived using run expectancy data, but the run values for each event are determined using league wide data across multiple years. Data are also park adjusted.

...which means that the number are not state-neutral. Which means they reflect situations and results as opposed to ability.

It's kind of like judging a hitter based on Runs Scored and RBI instead of OBP and SLG.

jojo
10-05-2007, 11:22 AM
...which means that the number are not state-neutral. Which means they reflect situations and results as opposed to ability.

I don't really see this as an issue.

Just so everyone is on the same page (I know you understand this because if I interpret your comment correctly this is the heart of your objection), basically, run values in the lwts system are simply the weighted average values for an event over all base-out states (determined using PBP data over many seasons). So, on average, event X has Y impact on run scoring.

UZR, basically tabulates the outcomes of a player's defensive performance on a play-by play basis and calculates his totals relative to his peers.

These totals are then converted to a run value using the average value of each event.

So in reality defensive runs in UZR directly reflect a player's ability because they are absolutely dependent upon that player's caught ball value (the total number/type of balls the player converted/failed to convert relative to his peers within designated zones).

Given sufficient sample size for a player, the fact that defensive runs are derived from estimated run values as opposed to actual ones really isn't a problem.


It's kind of like judging a hitter based on Runs Scored and RBI instead of OBP and SLG.

I guess I'd argue it's more like judging a hitter by taking OBP and SLG information another step further.....

Johnny Footstool
10-05-2007, 12:01 PM
These totals are then converted to a run value using the average value of each event.


I guess I'd argue it's more like judging a hitter by taking OBP and SLG information another step further.....

I guess the bigger question is, does the Runs Created formula for offense use the same run values for base-acquisition events as UZR?

RedsManRick
10-05-2007, 12:14 PM
It's kind of like judging a hitter based on Runs Scored and RBI instead of OBP and SLG.

Exactly, so the real point here is that you can create a truly context netural run based metric. We can pretend like the events were in an "average" base/out situation, but we cannot remove them entirely and maintain the run as the unit. UZR seems to have this down with it's remaining issues being zone definitions and the handling of out of zone plays and player interaction (the ball hog effect).

We have to evaluate fielders outside of the run context in terms of only the things which they control -- ie. something like PiZ% with a PooZ kicker.

Falls City Beer
10-05-2007, 12:27 PM
PooZ kicker.


Watch your language. :)

jojo
10-05-2007, 04:47 PM
I guess the bigger question is, does the Runs Created formula for offense use the same run values for base-acquisition events as UZR?

The short answer is no (though it's readily apparent that James uses linear weights in his formula). Here (http://www.redszone.com/forums/showpost.php?p=1268045&postcount=380) is a quick and dirty glimpse into how RC are calculated. RC has issues with the way it treats OBP and SLG and really uses a team context. Without getting swamped in the math, the practical thing to remember is that RC is unfair to guys with low OBP/SLG and overly generous to those with high OBP/SLG. Those aren't really issues with the way defensive runs are calculated by UZR.

Johnny Footstool
10-07-2007, 11:11 PM
The short answer is no (though it's readily apparent that James uses linear weights in his formula). Here (http://www.redszone.com/forums/showpost.php?p=1268045&postcount=380) is a quick and dirty glimpse into how RC are calculated. RC has issues with the way it treats OBP and SLG and really uses a team context. Without getting swamped in the math, the practical thing to remember is that RC is unfair to guys with low OBP/SLG and overly generous to those with high OBP/SLG. Those aren't really issues with the way defensive runs are calculated by UZR.

Then where are we getting the numbers we use to compare a player's offensive value to defensive value? If we're using different criteria to come up with run values, then the comparison really isn't valid.

jojo
10-08-2007, 09:26 AM
Then where are we getting the numbers we use to compare a player's offensive value to defensive value? If we're using different criteria to come up with run values, then the comparison really isn't valid.

Runs are runs.

Johnny Footstool
10-08-2007, 10:30 AM
Runs are runs.

On the field, yes.

In a sabremetric equation, no, not unless they are figured using equivalent methodology.

If the defensive system charges a fielder .55 runs for giving up a base, and the offensive system doesn't give you .55 runs for acquiring a base, the system is fundamentally flawed.

westofyou
10-18-2007, 11:45 AM
Ryan Braunís Defense
by Nate Silver

http://www.baseballprospectus.com/unfiltered/?p=609



Iíve made several snide remarks about Ryan Braunís third base defense in recent weeks, most recently in my review of the 50 most valuable players in baseball. I donít have anything against Ryan Braun; it would be hard for someone with the last name Silver to root against the Hebrew Hammer. Nor is this based on any sort of personal observation. Iíd probably have to see a player at least 10 times in person or 25-30 times on television before Iíd be comfortable contributing any kind of scouting take on his defense, and the only teams I see play that often during the season are the two local clubs, and maybe the two that advance to the World Series. But thereís no doubt about it: Ryan Braunís defense is bad.

These were the trailers in Clay Davenportís Fielding Runs Above Average (FRAA) last season:


-22 Kouzmanoff, 3B, SD
-20 Braun, 3B, MIL
-19 Durham, 2B, SFG
-18 Burrell, LF, PHI
-18 Harris, INF, TB
-17 Willingham, LF, FLA
-17 Dunn, LF, CIN
-17 Atkins, 3B, COL
-16 Fielder, 1B, MIL
-16 Hall, CF, MIL
-16 Lugo, SS, BOS
-15 Eckstein, SS, STL
-15 Sanchez, 2B, PIT


Braun doesnít actually have the lowest score on this list ó he has the second lowest behind Kevin Kouzmanoff. But FRAA is a counting stat rather than a rate stat, and Braun played only about two-thirds of the season, whereas most everybody else had the full year to accumulate their low scores. On a rate basis, Braun was a little worse than Kouzmanoff, and Braun and Kouzmanoff were quite a bit worse than any other regular or quasi-regular in the game. By the way, notice how many of these players are notoriously old, slow, fat, or were playing out of position; these are exactly the people youíd expect to see on a list like this one.

We can also look at a second-generation, play-by-play based metric like the Hardball Timesí Revised Zone Rating. According to that statistic, there were five regular defensive players who made at least 10% fewer plays than an average player at their position. Ryan Braun is one of them; in fact he rates at the second-worst defender in baseball relative to his position, ahead of only Manny Ramirez.


-19.1% Ramirez, LF, BOS
-16.4% Braun, 3B, MIL
-12.0% Encarnacion, 3B, CIN
-13.4% Young, 1B, WAS
-10.7% Sexson, 1B, SEA
-9.5% Fielder, 1B, MIL
-9.3% Bautista, 3B, PIT
-9.2% Atkins, 3B, COL


And so we have agreement from two systems, with radically different approaches, that Braun was one of the very worst defensive players in baseball in 2007.

I donít have Braunís rating in my favorite third-gen metric, UZR, but hereís what Mitchel Lichtman had to say about him:

Braun would certainly be the worst [third baseman], if he qualified. Even using zero UZR to make up the games he ďmissedĒ he is still probably the worst. That is a shame since he is such a good hitter, and his poor defense takes most (75% or so) of his hitting value away.

Making a few inferences from Mitchelís statements, this would imply that Braunís UZR falls somewhere between Miguel Cabreraís -28, and 75% of his 2007 VORP, which works out to -43. So here again, Braun ranks as one of the worst four or five defensive players in baseball, challenged only by Pat Burrellís -34, Manny Ramirezí -33, and Raul IbanezĎ -30. By consensus view, Braun was probably the very worst defensive player in baseball in 2007; the only real competitors for the title might be Ramirez, and perhaps Burrell.

But the question is not so much whether Braun was bad in 2007 ó he was awful ó but how heís likely to play going forward. Before the season, Braun was regarded anywhere from a somewhat below-average defender at third base to a problematically bad one. Hereís what Kevin Goldstein wrote about him last December.

Despite the tools, Braun is still inadequate at the hot corner. His footwork is bad, and while he has plus arm strength, his throws lack accuracy. His bat is nearly major league ready, so if the glove doesnít catch up fast enough, he could be moved to right field. The bat will play anywhere.

This view is largely confirmed by Tom Tangoís Fans Scouting Report:


Speed 60
Arm Strength 57
First Step 52
Instincts 50
Hands 25
Release 9
Arm Accuracy 0


Braun grades out as roughly average in several categories, including the range-based metrics of speed and first step. Unfortunately, his hands are bad, and his release and throwing accuracy are off-the-charts bad. And double unfortunately, these would tend to be the categories that are most important for a third baseman; speed doesnít matter much at all, for instance.

Are these things that Braun can work on? Perhaps: Iíd certainly hold out more hope that a coach could improve a playerís accuracy and release than his arm strength. But the thing is, once a player reaches the major leagues for a competitive club, there isnít much room for experimentation. Whatís the upside to that experiment? Braun goes from being a way below-average defender at third base to a somewhat below-average one. And then you have to move him in a couple years anyway once he bulks up and slows down. Whatís the downside? Braun canít or wonít improve, or he takes a long time to do so, costing the Brewers runs in the field each week while the experiment in progress. Even worse, you could trigger a crisis of confidence, or detract from his work on his hitting approach, which requires more improvement than you might think. And all of this for what benefit? National League third basemen hit .280/.348/.456 last year, which is actually a bit better than the .275/.344/.442 it got out of its right fielders, and only slightly worse than the .278/.358/.478 from the left field position. The gap in the defensive spectrum between third base and corner outfield is small as compared with the gap between third base and center field or second base.

Braun, in spite have never played the position professionally, is probably much closer to being an adequate defensive corner outfielder than an adequate defensive third baseman. He runs reasonably well, and while his arm is not accurate, it is probably strong enough to deter baserunners from advancing on him Johnny Damon style. And the good news is that there are any number of permutations that would allow the Brewers to improve their defense without really losing any ground at the plate. For example:

* Move Braun to LF, sign Mike Lowell at 3B, sit on Matt LaPorta for a year

* Move Bill Hall to 3B, Braun to RF, and Corey Hart to CF; LaPorta gets the opportunity to make the big league club out of spring training.
* Trade Bill Hall and Claudio Vargas for Joe Crede and Faustino de los Santos, move Hart to CF, Braun to RF.

You get the idea Ö there are a million ways the Brewers can play this without incurring any real transaction costs.

If Braun does move to the outfield, does he become one of the most valuable commodities in baseball? Weíll have to see what PECOTA does with him, since thereís room for concern about his plate discipline numbers too. But the worst case would seem to be that he becomes another version of Jeff Francoeur, and thatís a pretty good worst-case scenario.