View Full Version : Baseball analysis is dead

Chip R
09-04-2007, 05:27 PM
So says Gary Huckabay, founder of Baseball Prospectus.


09-04-2007, 06:14 PM
I'll have to analyze what he is saying and get back with you. ;)

09-04-2007, 06:20 PM
. Those companies (and others) are happy to let a self-directed, competent, and uncompensated gaggle of fragrant, bearded unix gurus take time out from watching Mystery Science Theatre 3000 to develop a fantastic piece of software for the masses, then adopt it as their own without having to spend a huge amount of their own resources on the project.

Maybe folks in the front offices did read Moneyball afterall ? :laugh:

09-04-2007, 06:26 PM
So says Gary Huckabay, founder of Baseball Prospectus.


Nonsense. One only needs to look at the rate of return for the draft and see there is lots of advantage to be gained.

There are also plenty of teams (Red Sox & A's come to mind) that have been working diligently on their own defensive ratings.

I know many thought Kulman was a nut with his brain-typeing, but its regularly accepted that a scout is looking for "it" from a player. Why not brain-typing? Has it been disproven?

The more data that is available, the more meaningful the research becomes. I think baseball is headed towards more analysis, not less.


09-04-2007, 06:39 PM
The article very clearly makes the point that the definition of analysis he's using does not include things like biomechanics. It's about outcome valuation.

There is more to be gained by effectively communicating the already developed base of knowledge than there is exploring increased efficiency on the margins of the unknown.
Sure, there's still a cutting edge of research. But the biggest gains, as Gary correctly points out, have already been uncovered.

Baseball is a fixed system of rules and limited human ability. It's possible that we're reaching a point at which statistical analysis has squeezed 99.5% of the juice from the orange.

The fact is that the skills and attitudes that allow people to become top level management are different from the skills and attitudes that make one pursue heavy data driven analysis. Analysts need to stop limiting their scope of responsibility to crunching the numbers and include the necessity to communicate their gained knowledge effectively. Sure, somebody has to crunch the numbers. But all the numbers in the world don't matter if they are put in to action. Sure, there will be greater inclusion as time passes and the guys who grow up on Bill James get jobs. However, there will always be a rift between the people who can manage people and the people who can manage numbers. Bridging the gap between them is the third leg of the stool.

09-04-2007, 09:05 PM
My subscription to BP ran out Friday. I didn't renew it. I don't think I'm going to; anyone else of the same mindset?

09-05-2007, 07:38 AM
My subscription to BP ran out Friday. I didn't renew it. I don't think I'm going to; anyone else of the same mindset?

I quit getting my gas at BP years ago. http://www.freesmileys.org/emo/whacky097.gif (http://www.freesmileys.org)

Roy Tucker
09-05-2007, 10:26 AM
I understand what Huckabay is saying, but I can't help but think this is what people said about HRs, RBIs, and batting average. That this is all you need to know, its the true gauge of a player, and its just getting people to understand now.

I've been to enough rodeos now to know that it's a dangerous thing when I think I know everything and everything worth discovering has been discovered. Innovation always continues and if you don't believe that, you will soon find yourself in the tar pits with the mastodons saying "what happened?".

Johnny Footstool
09-05-2007, 10:35 AM
I get the feeling people have declared baseball analysis "dead" in the past. Yet here we are.

09-05-2007, 11:35 AM
I think these are the key sentences in the article:

When I’m talking about ‘baseball analysis’ here, I’m talking about the rigorous review of player performance data. I’m not talking about the inclusion of pitch velocity and location data that’s now coming available, and I’m not talking about the integration of scouting data with performance data. I’m strictly talking about activities like developing value metrics, forecasting, and all the other stuff we do with the massive yarn-ball of data we’ve all put together over the years.

This calls to mind the comment DePodesta made in Moneyball about what's-their-names' derivatives system, about how it was more than just "churning conventional statistics in unconventional ways." I think that's kind of Huckabay's point here; slicing and dicing of traditional performance statistics and having it tell us something we didn't kind of already know is reaching the point of diminishing returns. The next big gains in player evaluation will come from systems that work with data coming from outside the box score.

I like that he mentioned the pitch velocity/location data. A lot of the assumptions we make on pitchers are based on aggregates and averages and a definition of "luck" that presumes that anything outside the normal range evens out over time. Being able to define the quality of the pitch itself is a more granular level of skill evaluation. Teams can evaluate the process and not just the outcome.

I think the next great frontier is biomechanics and injury "prehab," to steal a phrase from Will Carroll. Teams that can keep their best players on the field more, particularly when it comes to pitchers, will gain a serious advantage.

cincinnati chili
09-05-2007, 12:28 PM
Huckabay is the guy at BP who I enjoy the most, and I wish he had time to write more.

This one was way too general for my taste. It almost makes me wonder if Billy Beane wasn't taking his suggestions while Huckabay was consulting for the A's, and he got pissed off. Or perhaps he's voicing frustration on behalf of one of his former-BP colleagues who got brought in house and then ignored.

09-05-2007, 12:33 PM
The Reds are still a bit tone deaf when it comes to OBP and protecting their starters.

09-05-2007, 12:37 PM
In a way, it kind of read like a sequel to his article in the back of BP '06.

09-05-2007, 01:55 PM
I wasn't aware that OB and pitch counts were settled matters.

Certainly how to leverage OB in lineup construction seems to be a lingering issue. And it doesn't seem to me much of anything has changed in terms of pitcher injuries/usage. IMO, all that's happened is that hitters have improved to the point where pitchers have seen the number of effective pitches they can throw in a season diminish. Perhaps they could throw a 120 pitches somewhat frequently and work effectively with the associated fatigue if they were facing 1970s hitters.

Meanwhile you've got defense and pitching development begging for objective discovery.

09-05-2007, 02:29 PM
I don't think he's saying that there's nothing yet to learn, but rather that there is nothing "big" left. If you read something like "The BOOK" by Tom Tango, you can see that at best we're gaining tenths of a percent with new knowledge. It's important to distinguish between obtained knowledge and actual practice.

When he says it's dead, he's talking about the cutting edge. In terms of actually utilizing the knowledge obtained over the last 30 years, there's still a lot of ground to gain.

09-05-2007, 05:29 PM
I wonder if there is any way to use statistical anaysis, combined with perhaps body-typing, gentics, family mapping, etc to predict the durablity of a pitchers arm?

Since pitching is such a valuable commidity and high-ticket item, even a tenth of a point improvement on the field would be worth the investment.

09-06-2007, 03:08 PM
Hardball Times responds. An interesting read.

Long live baseball analysis
by Dave Studeman

Is Baseball analysis dead?

Gary Huckaby, in a recent article for Baseball Prospectus, announced that "baseball analysis is dead." Gary is the founder of Baseball Prospectus, a site that prominently features a lot of baseball analysis. In fact, we analyze baseball here at The Hardball Times, too. So what's going on? Should we all hang up our figurative spikes and move onto other causes, like world peace or something? Is baseball analysis truly dead?


09-06-2007, 05:30 PM
When you can run the numbers, pick the champion and repeat it every time, then...it's dead.

cincinnati chili
09-06-2007, 05:56 PM
When you can run the numbers, pick the champion and repeat it every time, then...it's dead.

That's a good point. Baseball would be boring if it was ALL science, rather than a combination of arts/science. The problem is that 90% of MLB organizations are frustratingly unscientific. Just compaare the lack of data-driven decisions to the NFL. NFL runs numbers but football analysis is far from dead.

09-06-2007, 06:15 PM
When you can run the numbers, pick the champion and repeat it every time, then...it's dead.

Or here's another one: When no matter what numbers you run, you can never pick the champion, then it's dead. That's the point being made.

We're essentially past the point where running the numbers (in any conventional sense) is going to tell us anything that helps us win ballgames. Actually using the numbers that are being run is another point entirely...

09-15-2007, 11:46 AM

Lies, Damned Lies
New Life on Different Fields

by Nate Silver

This piece was originally intended as a response to Gary Huckabay’s column of last week, the idea being to contradict his assertion that baseball analysis is dead by counting down 10 points of decision that at least a significant minority of baseball franchises get wrong. But after reading through my article—I generally write my introductions last—as well as re-reading Gary’s piece, I am not so sure it is orthogonal to it at all. I agree with Gary that there is relatively little to be gained from what he describes as “the rigorous review of player performance data.” Relatively little does not mean “nothing,” however, and I have isolated some of the exceptions below. Most of the items on my list, however, have to do with questions that run outside the scope of the GM or the field manager. They have more to do with the guy sitting in the owner’s box, and those places on a baseball team’s org chart where the names stop becoming familiar.

10. Inappropriate Leadoff Hitters: We start with one of those minor exceptions that takes place at the field level. There are very few in-game strategic decisions that amount to a hill of beans, and lineup order—within reasonable boundaries of sanity—is no exception. Most teams, save perhaps for three or four outliers like the Dodgers, have no trouble understanding the value of on-base percentage.

Still, for fully one-third of the teams in baseball, the most frequently-used leadoff hitter this season has a lower OBP than that of the team in general. The ten culprits are these:

Team Hitter Hitter OBP Team OBP
BOS Lugo .296 .362
NYY Damon .350 .364
CHW Owens .307 .317
PHI Rollins .346 .356
WAS Lopez .307 .321
CHC Soriano .329 .332
CIN Freel .308 .336
HOU Biggio .283 .326
ARI Young .293 .316
SDP M. Giles .306 .321

By contrast, 29 of the 30 primary leadoff hitters are on pace to finish with double-digit stolen base totals; the lone exception is Craig Biggio, which can readily be explained by the fact that the Astros came into the season apparently thinking they were getting the 1997 version of the former star. Generally speaking, the conceit of using a Brian Downing or Wade Boggs type as a leadoff hitter never really caught on.

What’s funny is that there are several analysis-friendly teams on the list, like the Red Sox, Padres, and Diamondbacks. It would appear that this is a battle that our SABRCat Superfriend GMs have concluded is not worth fighting; there is just too much inertial momentum going the other way, and too little marginal gain to be had.

9. Underaccounting for Injury Risk: I have only anecdotal evidence to present here: Randy Wolf. Jason Schmidt. Nomar Garciaparra. Wait, all of those guys are Dodgers? There is ample room for improving analysis of injury risk and its concomitant effects on performance, feeding off the head start that Will Carroll and Sig Mejdal have given us. But for the time being, there are a number of clubs that could stand to develop a respect for the Inertial Law of Injuries: what’s healthy tends to stay healthy, and what’s injured tends to stay injured.

8. Focusing Too Much on Year N-1, and other small sample sizes: Now we start to get into the more serious sins. My analysis of the free agent market indicates that the player’s performance in his most recent season generally accounts for about 65 percent of the salary package he eventually receives, when statistically speaking the proper fraction for a mid-career player is closer to 50 percent. This is especially problematic if, as Dayn Perry found in Baseball Between the Numbers, players tend to perform especially well in contract drive seasons (and even more so if my further speculation is correct that baseball players are more likely to use PEDs when they’re angling for a new contract).

7. Failure to Understand Pitcher Peripherals, Especially for Relief Pitchers: Stop me if you’ve heard this one before. Baseball seems to have experienced some improvement in downplaying the importance of W-L record, but it still treats ERA as the holy grail. Yet ERA is not much more reliable; the year-over-year correlation for ERA is just .38 for starting pitchers, and a fair bit lower than that for relievers. Key peripheral statistics like walk rate, strikeout rate, and groundball percentage, on the other hand, all check in with correlations between .68 and .81. The failure to heed the importance of peripheral statistics can lead to disasters like the White Sox middle relief corps of the past two seasons.

A related adage is the old piece of Bill James wisdom about power pitchers holding up better than finesse pitchers over the longer run. But after seeing the difference in the value of the recent contract extensions signed by Mark Buehrle and Carlos Zambrano, I am not so sure that teams haven’t started to price this into the market.

6. Failure to Adjust Strategy Based on Position on the Wins Curve: This is a tricky one, because it requires both a solid understanding of baseball economics and a capacity for intellectual honesty. Because of the substantial increase in marginal revenues associated with making the playoffs, proper strategy differs rather radically based on where a team falls on the wins curve; a free agent that might be a good buy for an 88-win club could be an awful investment for a 73-win club. The problem is that a baseball team may not be honest with itself about just where it stands. Most people that establish a high degree of authority in a people business like baseball tend to be optimists, and optimists tend to exaggerate their lot in life. If you surveyed the 30 major league GMs about how many games they expected their team to win next season (and were somehow able to get an honest answer) I would guess that the average would come out to something like 88. This fact alone begets substantial irrationality in the free agent market…

5. Lack of Objective Analysis of Marginal Returns: …and this issue facilitates those problems. Most teams don’t seem to be conducting objective analysis of how much extra wins or extra championships will actually improve revenues in their market. Rather, teams tend to spend the money they have, and market prices are dictated by a monkey-see, monkey-do approach. How else to explain how 27 of 30 teams lost money on their marginal spending on payroll last year? Or that in spite of that, prices for free agent talent in this winter’s market increased by nearly 50 percent because of an influx of cash from sources like MLBAM and national TV rights, neither of which are responsive to marginal changes in team quality?

4. Lack of Coordination between Baseball Ops and Ownership: I was talking recently with Aaron Schatz about the differences in ownership structures between baseball and football. In football, for whatever reason there are a large number of franchises owned by families; by my quick count 11 of the 32 current NFL owners fall under the category of legacy hires. In contrast, this is relatively rare in baseball; baseball teams tend to be owned by corporations, investment conglomerates, or Very Rich Dudes. You would think that this breakdown would tend to be favorable to baseball, since family legacy teams tend to make nepotistic hires, leading to occasional disasters like the pre-Carson Palmer Cincinnati Bengals.

In fact, however, Aaron theorizes that all this nepotism tended to breed less tension into the NFL’s culture of ownership-management relations. In the NFL, the owner was some rich guy wearing argyle socks, and the general manager was probably also some rich guy wearing argyle socks, who happened to share half the owner’s DNA. In baseball, on the other hand, the owner was some guy wearing argyle socks, and the GM was some ex-jock who had come up in a different culture.

Sure, the argyle-sock guys might authorize the trade of John Elway for an offensive lineman and a six-pack of Schlitz, but at least they knew where one another was coming from. Baseball, on the other hand, seems to be in a perpetual state of ownership-management tension, damned if they do and damned if they don’t. On the one hand, you have the Drayton McLanes of the universe, who interfere with their GM’s ability to do his job. On the other hand, you have cases like the Cubs, where ownership changes its payroll requirements willy-nilly from year to year based on balancing the corporate bottom line, rather than trying to maximize the profits from the baseball team itself. The few franchises where ownership and management seem to see eye-to-eye, like the Red Sox and the Angels, stick out like a sore thumb, and they almost always do well in the standings column.

3. Misuse of the Closer: Gary touched on this one—optimizing the use of the bullpen is perhaps the one in-game managerial decision that has the most impact on a team’s ultimate place in the standings. Unfortunately, it is also the one that teams routinely get wrong. We are not talking about using a closer by committee; most teams will have an alpha dog in their bullpen, and he deserves to get the most important opportunities. But those opportunities are not particularly strongly related to the definition of the save.

This is a cultural thing, not an analysis thing. The theory behind it is not particularly hard to understand, and yet almost every team is giving away a couple of wins a season by succumbing to the groupthink that the save rule has engendered. With the notable exception of NFL coaches’ irrational disdain for going for it on fourth down, this might be the most reliably botched strategic protocol in American sports.

2. Failure to Appreciate the Value of Draft Picks and Pre-Free Agent Players: I am not necessarily talking about the signing bonuses paid to draft picks. Although teams are generally rewarded when they go over slot for a special player, there are also some incentives toward collective action that keeps prices down overall. Nevertheless, baseball teams fail to appreciate that essentially the only reliable way to make money at the margins in the industry is to employ quality talent at below-market prices. The expected savings on a premium prospect like Jarrod Saltalamacchia can approach $50 million before he hits his first cycle of free agency. There is almost no reasonable combination of veteran talent, and no reasonable discount rate, that could justify trading away such an asset.

But at least the guy the Braves got for Saltalamacchia is a good player. How to explain a trade like Scott Moore and Rocky Cherry for Steve Trachsel? Nobody noticed this one, but it might be the worst trade in several seasons, and it required a violation of at least half of the ten principles that you see in this column to come to fruition. Trachsel is a liability rather than an asset; he is not worth his prorated salary. In fact, provided that you understand #7 above, he is worth less than nothing. His QERA at the time the Cubs acquired him from the Orioles was 6.47, largely because he had walked 50 percent more batters than he’d struck out. Nevertheless, the Cubs gave away a prospect in Moore whom PECOTA thinks could produce about $30 million in value before he becomes a free agent, and a decent reliever who would cost at least a million or two each season in the free agent market to replace. It makes absolutely no sense, nor does the rationalization that the Cubs traded for Trachsel to keep him away from the Brewers and Cardinals; the Cubs should actively seek to have Trachsel pitching for their rivals. You can’t tell me that the arbitrage opportunities in baseball are anywhere near exhausted when you see a trade like that, which barely got a shrug from the analysis community.

1. Too-Long Contracts (and Too-Short Time Horizons): How many contracts of at least four years in length turn out to produce a positive return on investment for the club? I’ve given an off-the-cuff estimate of between 20 and 30 percent when asked this question. People like David Regan who have studied this issue more systematically would tend to support that conclusion. By the third year of his contract, a free agent hitter has generally lost about 40 percent of his value from his contract-year season, and a free agent pitcher has generally lost 60 percent of his value. Yet teams give out extra years on their contracts like they’re Halloween candy. It’s very common to hear of a team that is considering signing a particular player to a three-year, $30 million deal, and before you know it, that contract has become four years and $40 million (or worse still, the extra year might be a player option).

Now, overall, a great number of these problems—certainly #1 and #2, but also to some extent #4, #5, #6, and #8—stem from the fact that teams are exceptionally focused on the near-term. Vince Gennaro in Diamond Dollars suggests that the discount rate applied by baseball teams approaches 35 percent—you’ll trade 1.35 wins next season for one win today. Such a discount rate would be unconscionable in any other mature industry, but it should not be entirely surprising, given the principal-agent problems that baseball teams face. The median tenure of current major league managers is between two and three years, GMs between four and five years, and principal team owners last about seven years. Why should the Cubs care about giving too much money to Alfonso Soriano if neither Lou Piniella nor Jim Hendry (and certainly not the Tribune Corp.) is going to be around when that deal expires?

Of course, the Tribune should care about Soriano’s contract if the potential new ownership groups are smart enough to recognize that it represents a liability, but they have some reasonable hope that it will be written off as rounding error by whoever purchases the franchise. The market for baseball franchises is not particularly liquid, nor are the prospective owners necessarily evaluating the investment as rigorously; they might understand finance, but not baseball economics. The owners may be willing to pay substantially more than the price that the P&L justifies because they think they’ll be able to pass the franchise off to a greater fool.

The next revolution in baseball will not take place at the field level, and it will probably not take place at the general manager level; it will take place at the ownership level, and it will not be led by statheads, but by investment bankers. Like a lot of revolutions, it might be precipitated by hardship. I believe there is at least a 60:40 chance that that franchise valuations for baseball are presently in a bubble, and that the bubble will burst at some point within the next decade. If the credit crunch has legs, such that leveraged buys of baseball teams become harder to execute, the collapse could come toward the beginning of that cycle.

If baseball’s existing owners want to protect their investments, they ought to change their practices for vetting potential new owners, focusing not so much on those owners that are likely to perpetuate the collective groupthink, but rather on those who come with business plans in hand, and have a demonstrated ability to shepherd long-term investments. In contrast to the perception that everything is just ducky, I believe that the industry is in a fairly dangerous place. That’s what I have to conclude when I see discount rates of 35 percent, or the market price for free agent talent jumping by 50 percent with little underlying economic rationale, or the Cubs throwing away $30 million in future value to trade for a player like Steve Trachsel.

09-15-2007, 01:36 PM
WOY -- THat was a great read. Nice find.

The constant MLB ownership/GM tension and higher rate of turnover (NFL teams almost never change hands these days -- no one wants to sell a money printing press) and the failure to properly value pre-free agent players has always baffled me. That's why stealing guys like Hamilton/Phillips/Burton and maybe even solid contributors like Keppinger/Cantu is critical to payroll flexibility. You'd think teams would be ringing the phone off the hook to try and grab guys like EdE and Votto.

09-15-2007, 01:38 PM