The most predictive way to look at stats

[Hoosier Red]

The recent Aaron Harang discussions have left me trying to figure out a way to best weigh recent results versus year to date overall stats versus career norms.

In my mind, statistics are most useful and interesting when we take the numbers and attempt to project for the next start, rest of the season or next season etc...

For instance; here are Harang's numbers/start;

Career; 6.17 IP/Start, 4.30 ERA, 1.334 WHIP, 7.49 K/9
Year to Date: 5.91 IP/Start, 5.02 ERA, 1.455 WHIP, 6.75 K/9
Last 2 months: 6.02 IP/Start, 4.21 era, 1.413 WHIP, 6.73 K/9
Last 1 month: 5.7 IP/Start, 4.15 ERA, 1.52 WHIP, 6.05 K/9
Last Start: 6.1 IP, 3 ER,(4.26 ERA), 1.475 WHIP, 5 Ks(7.11 K/9)

Essentially I'm asking for the best way to weigh recent results versus the sample size, and at what point do recent results reach a large enough sample that they are more relevant than say a larger sample size?

My question is not specific to Aaron Harang, you could make this about any number of less controversial subjects like Drew Stubbs or Nick Masset or Edwin Encarnacion.

[Marc D]

The recent Aaron Harang discussions have left me trying to figure out a way to best weigh recent results versus year to date overall stats versus career norms.

In my mind, statistics are most useful and interesting when we take the numbers and attempt to project for the next start, rest of the season or next season etc...

For instance; here are Harang's numbers/start;

Career; 6.17 IP/Start, 4.30 ERA, 1.334 WHIP, 7.49 K/9
Year to Date: 5.91 IP/Start, 5.02 ERA, 1.455 WHIP, 6.75 K/9
Last 2 months: 6.02 IP/Start, 4.21 era, 1.413 WHIP, 6.73 K/9
Last 1 month: 5.7 IP/Start, 4.15 ERA, 1.52 WHIP, 6.05 K/9
Last Start: 6.1 IP, 3 ER,(4.26 ERA), 1.475 WHIP, 5 Ks(7.11 K/9)

Essentially I'm asking for the best way to weigh recent results versus the sample size, and at what point do recent results reach a large enough sample that they are more relevant than say a larger sample size?

My question is not specific to Aaron Harang, you could make this about any number of less controversial subjects like Drew Stubbs or Nick Masset or Edwin Encarnacion.

For a hitter I'd probably look at a 10 WMA. For a pitcher maybe the last 3-5 starts. It would just be a lot of trial and error to see which time frame had the best "predictive" powers but I think what you'd find is its very hard to accurately predict game to game performance relying solely on past data.

[RedsManRick]

Firstly, a basic definitional point. There are projections and predicitions. Projections are a number of possible outcomes based on a set of assumptions. A prediction is selecting a single projection as the one you believe will (or is most likely to) occur. For example, PECOTA is a projection system which produces like 7 different possible paths. It's weighted mean projection takes each of those possibilities and weights them to come up with a single "prediction". It's semantics, but it does matter.

Secondly, about the nature of projections. As your sample sizes of the thing you're trying to predict grows, your set of projections narrows. In 1 game, it's completely reasonable to think Harang might allow 0 runs or 6 runs. The chance of those things happening might each be 10%. But the most likely scenario, 3 or 4 runs, might be 60%. For a season, predicted ERAs of 0.00 or 9.00 are obviously ridiculous and nowhere near 10%. But an ERA between 3.50 and 5.00 could be at 80%. The set of projections is very different. If you picture the possible outcomes as a bell-shaped curve, with ERA on the x-axis and likelihood of occurring on the Y-axis, the bigger the thing you're trying to predict, the skinnier that bell gets. However, if you had to make a single prediction, you might guess a 4.50 ERA in both cases. That is, the peak of the bell is in the same place regardless of how fat or skinny it is.

However, this is only true if you are only using your knowledge of the pitcher, and not some external characteristics of next game or month (park, matchups, etc.). If you try to account for each game differently, obviously your predictions will change. But if not, your projection for the next game is basically the exact same as the game following, the month following, etc.

More directly to your question, the way you predict is your recent data and regress it back towards a known baseline. The amount by which you regress it depends on how much recent data you have. The less recent data you, the more you should regress to (i.e. weight towards) the known baseline. So if coming in to the season you had him as a 4.00 ERA guy, but to date he's been a 5.00 ERA guy, you're going to regress that 5 heavily back to 4. You need a few hundred IP to get a stable ERA figure which has good predicitive value.

Beyond that, I'd say just go fangraphs at look at the ZIPS projections on the player page. They do just this.

[Hoosier Red]

Thanks guys. Those are excellent specific solutions.
I guess my question is more big picture though.

If Nick Masset has better stats since May 1 than his season stats, how would you balance that versus his overall stats.

I never like the "If you throw out starts X and Y" as an argument, but if I have a one run lead today I feel comfortable with Masset coming in. Even if his year long stats don't show it.

The reason why I incorrectly interchanged projections and predictions is that I'm not looking for a specific number perse(I don't have money riding on tonights start or anything.) But if I'm looking for a good guess as to what say Harang's ERA will be from now until the end of the season, how would I best predict that?

[Hoosier Red]

For a hitter I'd probably look at a 10 WMA. For a pitcher maybe the last 3-5 starts. It would just be a lot of trial and error to see which time frame had the best "predictive" powers but I think what you'd find is its very hard to accurately predict game to game performance relying solely on past data.

WMA? Sorry I'm not familiar with that Acronym.

I've played around with weighted averages and things like that for some marketing projects in the past and though I would try to apply my theory on how much weight to give each to something useful like baseball statistics.

Essentially my trial and error is going to be something along the lines of;

Last 5 starts-15%
Last 10 starts-30%
Year to Date: 30%
Career Numbers: 15%
Last two years: 10%

And then try to come up with something that would give me a good prediction over a one month or two month period.
I have a feeling this will be a lot more work than I bargained for.

[nate]

But if I'm looking for a good guess as to what say Harang's ERA will be from now until the end of the season, how would I best predict that?

I'd recommend looking at FIP, xFIP, SIERA and other defense independent stats that correlate better to ERA than ERA does.

Currently, Harang's FIP is 4.55 and xFIP 4.37. His career FIP is 4.13 and xFIP 4.40. Based on that, I would think it's safe to say he's more of a low to mid 4-ish ERA guy than the 5.02 guy he currently is.

[nate]

WMA? Sorry I'm not familiar with that Acronym.

I've played around with weighted averages and things like that for some marketing projects in the past and though I would try to apply my theory on how much weight to give each to something useful like baseball statistics.

Essentially my trial and error is going to be something along the lines of;

Last 5 starts-15%
Last 10 starts-30%
Year to Date: 30%
Career Numbers: 15%
Last two years: 10%

And then try to come up with something that would give me a good prediction over a one month or two month period.
I have a feeling this will be a lot more work than I bargained for.

I think I'd give most weight to the career numbers followed closely by the last three years. Maybe an older player would weight the last three years more heavily than the career numbers (for example, Griffey Jr.)

[Marc D]

WMA? Sorry I'm not familiar with that Acronym.

I've played around with weighted averages and things like that for some marketing projects in the past and though I would try to apply my theory on how much weight to give each to something useful like baseball statistics.

Essentially my trial and error is going to be something along the lines of;

Last 5 starts-15%
Last 10 starts-30%
Year to Date: 30%
Career Numbers: 15%
Last two years: 10%

And then try to come up with something that would give me a good prediction over a one month or two month period.
I have a feeling this will be a lot more work than I bargained for.

WMA= weighted moving average.

I think you are on it when you say its going to be a lot of work to find out your covering the same ground that's already been explored. RMR said it all better than I could but he is spot on.

[RedsManRick]

As others have said, there's really no need to reinvent the wheel here. Taking a 3 year weighted average and then regressing current season performance gets you basically as close as you're going to get. Any differences in predictive accuracy would be marginal at best.

Here's a good overview of projection systems: http://msn.foxsports.com/mlb/story/p...systems-021610

I would a caution about modeling: over-fitting.

Basically, this means you can try to make your projection too perfect by making it fit your data exactly. If you had every piece of data in the world, this wouldn't be a problem. But we never do. So we have to recognize that our data is a sample of the possibilities and treat it as such. If we try to be too perfect, we end up with worse projections. From an actual projections standpoint, this shows up in comparisons of different systems. Year after year, Marcel, the simple regressed weighted average system does quite well. In any given year, some other system will beat it. But other more complex systems will see patterns where they don't really exist. And even then, the differences between the systems, on balance, is pretty minor.

The use of monthly splits is a perfect example. We might look at a guy who has the following splits of:

April: 3.25 ERA
May: 4.00 ERA
June: 3.50 ERA
July: 4.25 ERA
August: 4.25 ERA
September: 3.25 ERA

1st Half: 3.58 ERA
2nd Half: 3.92 ERA
Season: 3.75 ERA

Let's say it's July 1 and our guy has a 3.50 ERA. What should we expect for July? For the 2nd half? We're going to be tempted to say, well, he's put up a 4.25 ERA during July for his career, so that's our best guess. Or we might say that he's a 3.92 2nd half guy, so that's our best guess.

But let's pretend he's really a 3.75 guy throughout the season, would we really expect him to have a 3.75 ERA each month? Each half season? Of course not. Sometimes it will be higher, sometimes lower. As we've analyzed ERA over the years, we've learned that these types of variation simply don't have predictive value. Most players will display some sort of seasonal trend, but those trends are largely random. Pitcher's skill levels simply don't fluctuate significantly during the course of the season. Sure, their ability changes slowly over time, over the arc of their career. Or it may change suddenly in the event of an injury, changed mechanics, or a new pitch. But time is not the driver.

Our best guess for July and for the 2nd half isn't based just on the performances from past Julys and 2nd Halves. It's from our best estimation of how talented a pitcher the guy is -- and that's done most cleanly using a weighted three year average. Attempts to get more granular than that are basically reading tea leaves.

[Far East]

...Beyond that, I'd say just go fangraphs at look at the ZIPS projections on the player page. They do just this.

This was my first visit to the fangraphs site. They had an article about rookies, which included this about Heisey:

"Despite playing in just 39 games this season, Heisey has put up 0.9 WAR for the year, which would be worth 3.5 WAR over 150 games. At twenty-five, Heisey is a little older than some of the more hyped-up rookies, and his Triple-A numbers this year (.241/.307/.430) weren't too pretty. He started 0 for his first 7 big league at-bats, striking out three times. However, Heisey has hit .271/.371/.542 for the entire campaign, good for a .386 wOBA. He's also done well in the field with 3.1 runs saved (UZR) while playing all three outfield positions."

What is WAR, and how does Heisey's 0.9 WAR compare to other Reds?

[RedsManRick]

This was my first visit to the fangraphs site. They had an article about rookies, which included this about Heisey:

"Despite playing in just 39 games this season, Heisey has put up 0.9 WAR for the year, which would be worth 3.5 WAR over 150 games. At twenty-five, Heisey is a little older than some of the more hyped-up rookies, and his Triple-A numbers this year (.241/.307/.430) weren't too pretty. He started 0 for his first 7 big league at-bats, striking out three times. However, Heisey has hit .271/.371/.542 for the entire campaign, good for a .386 wOBA. He's also done well in the field with 3.1 runs saved (UZR) while playing all three outfield positions."

What is WAR, and how does Heisey's 0.9 WAR compare to other Reds?

http://saberlibrary.com/misc/war/

If you had to pick one statistic – and only one statistic – to use in evaluating players’ value to their teams, Wins Above Replacement (WAR) should be it, end of story. You should always use more than one metric at a time when evaluating players, but that doesn’t change the fact that WAR is pretty darn all-inclusive. WAR basically looks at a player and asks the question, “If this player got injured and their team had to replace them with a minor leaguer or someone from their bench, how much value would the team be losing?” This value is expressed in a wins format, so we could say that player x is worth 6.3 wins to their team while player y is only worth 3.5 wins.

To get to WAR for the Reds, go to Fangraphs and click on the Team Tab. Then click on the Reds in the data table. When the list of Reds shows up, click the Value tab on the far right to get to the WAR data. I believe a league average starter puts up something like 2.5 WAR (Jojo would know...). ~1 WAR in his limited playing time is pretty darn good.

A good summary:
Votto is at ~4 WAR
Rolen is at ~3 WAR
Bruce is at ~2 WAR
Stubbs is at ~1 WAR

1 WAR is about the same as 10 runs above replacement (RAR). It might be easier to think of the players in terms of runs rather than wins. So the Reds have a +51 run differential right now. If you took away Votto and Stubbs and replaced them with replacement level players, the Reds would be about a .500 team (0 run differential).

A general rule of thumb is that (for a full season), 0 is replacement, 2-4 is average, 4-6 is all-star territory, and 6+ is MVP (vote recipient) range. You get above 8 and you're usually leading the league.