Turn Off Ads?
Results 1 to 6 of 6

Thread: Measuring starting pitching

  1. #1
    Stat Wanker Hodiernus RedsManRick's Avatar
    Join Date
    Dec 2004
    Location
    Chicago, IL
    Posts
    15,929

    Measuring starting pitching

    I've been increasingly frustrated by metrics used to measure starting pitcher quality. Let's look at the 3 Reds starters in 2006 who started 25 games or more:

    Harang: 35 GS
    Arroyo: 34 GS
    Milton: 27 GS

    ERA is surely decent, but it ignores the very real fact that innings and runs are accrued in bunches (35 games) and that their true value is contained within that given start. A pitcher could give you 34 CGS and 1 outing with 153 ER and 0 outs. His ERA is 4.50 and, had we not known that distribution, we would've thunk him a league average starter, despite having the best season ever. Is he really worse than the guy who gives you 35 9 IP, 4 ER starts? Of course not. So ERA could be better...

    By ERA:
    Arroyo: 3.29
    Harang: 3.78
    Milton: 5.19

    So how do measure the individual games? Well, the easiest metric is W/L, but let's not go down that road. We want to measure pitchers only, not their offensive support. So how about QS? Well, it's better, but quite flawed as there are "better" starts which don't get the QS tag. Eric Milton had a higher percentage of quality starts than Aaron Harang last year, which seems quite counter-intuitive. Aaron Harang was penalized for going longer outings and giving up an extra run. So my first thought is, rather than simply throwing out QS, let's adjust it. The idea remains the same. Namely, how often did the pitcher give his team a decent chance to win?

    The 3 Reds starters made a combined 8 starts of less than 5 IP and each time allowed at least as many runs as IP. So we can confidently say that there is no such thing (for the 2006 Reds) as a "good" sub 5 IP start. So let's put the first threshold there. You gotta throw 5 IP. Next, rather than look at raw ER, let's account for the per inning run allowance so that 3 more innings and 1 more run is a GOOD thing. So, rather than looking for 3 or fewer ER, we're looking for a "Start ERA" or SERA of 4.50 or lower. Let's see how that changes things (AQS: Adjusted Quality Start):

    Arroyo: QS 22/34 (65&#37, AQS 22/34 (65%)
    Milton: QS 15/27 (56%), AQS 17/29 (59%)
    Harang: QS 17/35 (49%), AQS 19/35 (54%)

    Hmm... Harang still isn't looking so good. So why is his ERA so low compared to Milton? Was he really a better pitcher? And herein is the crux of the problem. While Milton had more "quality" starts, Milton toed the line. While Harang had some real dominant starts, dragging down his ERA, Milton just made it over the ASQ threshold. Ok, well, does that matter or not?

    I say it does. If your team scores 4.5 runs per game, a AQS basically means you gave your team a 50% chance to win. But if you go 8 IP and 2 ER, you've done much more than that. Likewise, a 7 IP 4 ER start isn't "quality", but it gives your team a much better shot than does a 5 IP, 6 ER start. So I want to account for that, while maintaining the "by game" measurement style. Instead of a simply binomial on the 50% line, let's put starts in 5 groups, sticking the with basic AQS methodology (all the sub 5 IP starts are all very poor SERA's, so we'll toss out that requirement). The idea is to capture what kind of opportunity that start afford the Reds to win the game. The pitcher can't control runs scored by the Reds, so we have to level the playing field and assume an average of 4.50 runs scored by the Reds. There's also a correlation of -.6 between ER and IP, saying basically that the fewer runs you allow the more innings you pitch. So we can roughly say that lower SERAs in general have longer innings attached to them, capturing that ideal of longer outings as well.

    (I think we often inappropriately weight IP in our off the cuff measurements of start quality. Given a replacement level bullpen with a 5.50 ERA, a 5 IP, 1 ER outing IS better than a 7 IP, 3 ER outing. Is your bullpen going to give up 2 runs in 2 innings? Most of the time, they won't. It's why I get bugged when starters who are great for 80 pitches, like Javier Vazquez, routinely get forced to throw those last 30 pitches where they get rocked. If insanity is doing the same thing but expecting a different outcome, we've got a lot of insane managers out there. Give me a Kirk Sarloos or Matt Belisle in the pen and let him throw 2-3 innings twice a week when the starter is losing it early -- but get them in there BEFORE the walk-single-3 run homer)

    It's important to remember that while the average of SERA accross a season (ERA) has a pretty narrow distribution (I'd estimate 90% of SP between 3.50 and 5.50), the SERA of a given game is a fairly wide spread. That is, even a guy with a 3 ERA will give up 0 runs sometimes and 7 runs other times and rarely gives up exactly 3.

    SERA <2.00 = Excellent chance to win
    SERA >2.00 & <3.50 = Good chance to win
    SERA >3.50 & <5.00 = Mediocre chance to win
    SERA >5.00 & <6.50 = Poor chance to win
    SERA >6.50 = Terrible chance to win

    Now we're looking at a distribution of starts. The chart/table below will be the percentage of starts in each group: %ES/%GS/%MS/%PS/%TS

    That should tell you all you need to know. Milton, Arroyo, and Harang were all Poor or Terrible at about the same frequency (35-40% of their starts). However, when they were mediocre or better, they were much different pitchers. Arroyo was Good or Excellent in 56% of his starts, compared to 43% for Harang, and just 34% for Milton.

    When discussing the performance of a pitcher over time, it's important to consider the distribution of the runs he's allowed, not just the average rate at which he allows them. As bad as Eric Milton is, he doesn't "lose" the Reds any more games than Harang or Arroyo. To me, this should be the #1 take away from this whole post. Read that last sentence again. Milton doesn't lose more games. However, he doesn't win very many games either. What made Arroyo so good last year is that he "won" over 50% of his starts. That's why he was our ace.

    As a follow up, I considered looking at Bill James' Game Scores. They consider IP a little more, and also consider Hits, BBs, Ks, and unearned Runs, getting a little more closely at the true effectiveness of the pitcher. You could group Game Scores as I've done with SERA and see similar distributions.

    Lastly, I want to make the point about the use of averages. Averages are somewhat useful in comparing the overall effectiveness of something over large samples. However, we need to be clear that averages also represent a middle point that isn't itself a reality. Aaron Harang didn't have 35 9 inning starts in each of which he gave up 3.78 runs. In fact, less than 20% of his starts were even within .5 ER of that. In baseball, games are what counts. Every game results in a win or a loss and after that game, it doesn't matter if you lost by 10 or by 2. If you shut 'em out next time, you're gonna be 1-1, regardless of that ERA. We should be careful to account for the way in which stats are accrued as we value aggregated performances.
    Last edited by RedsManRick; 04-08-2007 at 02:41 PM.
    Games are won on run differential -- scoring more than your opponent. Runs are runs, scored or prevented they all count the same. Worry about scoring more and allowing fewer, not which positions contribute to which side of the equation or how "consistent" you are at your current level of performance.

  2. Turn Off Ads?
  3. #2
    The Big Dog mth123's Avatar
    Join Date
    Jul 2006
    Posts
    14,728

    Re: Measuring starting pitching

    Excellent post. This post goes deeper with statistics into understanding actual on field performance in the context of winning games (which is still the point) than most I have seen.

    I have grown to be quite a numbers fan, but everyone needs to understand that most of these statistics are a result of applying overly simplified linear equations (and reality is not linear). Depth and context can not be dismissed when interpreting any stat. I think the popularity of fantasy games (I enjoy those too) has made that a little blurry for many.

    Are these stats avaialble for all pitchers somewhere or did you do the painful compilation by going through the logs game by game?
    "All I can tell them is pick a good one and sock it." --BABE RUTH

    Having better players makes "the right time" or "the big hit" happen a lot more often. PLUS PLUS

  4. #3
    Stat Wanker Hodiernus RedsManRick's Avatar
    Join Date
    Dec 2004
    Location
    Chicago, IL
    Posts
    15,929

    Re: Measuring starting pitching

    Quote Originally Posted by mth123 View Post
    Are these stats avaialble for all pitchers somewhere or did you do the painful compilation by going through the logs game by game?
    I imported a seasons worth of game logs for these 3 guys in to excel (after a little formatting in notepad) and then did the number crunching there. Just a copy paste from a standard game log page for each of them and then some clean up before pulling in the tab-delimited file in to excel. My day job is a lot of simple database manipulation in excel, access, etc. -- so it was old hat :-P
    Games are won on run differential -- scoring more than your opponent. Runs are runs, scored or prevented they all count the same. Worry about scoring more and allowing fewer, not which positions contribute to which side of the equation or how "consistent" you are at your current level of performance.

  5. #4
    Member
    Join Date
    Dec 2004
    Posts
    3,072

    Re: Measuring starting pitching

    I can believe. And while I don't use W/l as gauge of how good a pitcher is, I give it a tad more credence than most here.

    Excellent post.
    I hope it's never sunny in Philly again.

  6. #5
    Member Spitball's Avatar
    Join Date
    Apr 2000
    Posts
    5,620

    Re: Measuring starting pitching

    A very interesting read, Rick. Thanks!
    "I am your child from the future. I'm sorry I didn't tell you this earlier." - Dylan Easton

  7. #6
    Senor Votto Degenerate39's Avatar
    Join Date
    Sep 2006
    Posts
    7,914

    Re: Measuring starting pitching

    Good post Rick
    Most Vottomatic Player


Turn Off Ads?

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

Board Moderators may, at their discretion and judgment, delete and/or edit any messages that violate any of the following guidelines: 1. Explicit references to alleged illegal or unlawful acts. 2. Graphic sexual descriptions. 3. Racial or ethnic slurs. 4. Use of edgy language (including masked profanity). 5. Direct personal attacks, flames, fights, trolling, baiting, name-calling, general nuisance, excessive player criticism or anything along those lines. 6. Posting spam. 7. Each person may have only one user account. It is fine to be critical here - that's what this board is for. But let's not beat a subject or a player to death, please.

Thank you, and most importantly, enjoy yourselves!


RedsZone.com is a privately owned website and is not affiliated with the Cincinnati Reds or Major League Baseball


Contact us: Boss | GIK | BCubb2003 | dabvu2498 | Gallen5862 | LexRedsFan | Plus Plus | RedlegJake | redsfan1995 | The Operator | Tommyjohn25