More on regression, xFIP, HR/FB, BABIP, and the likeAugust 18th, 2010 by Derek Carty in Player Discussion, Prediction, Theoretical
Yesterday, Chris Liss penned a post at RotoWire's RotoSynthesis blog that mentioned my luck, randomness, and Dan Haren article. This stemmed a little debate in the comments section, which I wanted to respond to here since I'm incapable of writing up a succinct response that's appropriate for a comments section.
One commenter was Yahoo!'s Scott Pianowski, who said:
I love xFIP. It's a way for otherwise intelligent people to convince themselves that bad pitchers are really not that bad and great pitchers aren't so special. In other words, let's grade on a scale and find a way to bridge the gap between Aaron Harang and Adam Wainwright.
The best pitchers in baseball consistently beat the league average in HR/FB. Go look it up for yourself.
As Scott suggested, I looked up the HR/FB numbers, and here's what I found:
From 2004 to 2009, there were 43 pitchers (who spent at least half of their games starting – relievers are a different beast) who posted HR/OF rates below 10% (league average is 11%-ish) in at least 200 IP (one full season-ish). If we require 400 IP, we get 29 pitchers. At 600 IP, we get 22 pitchers. At 800 IP, we’re down to 13. Of course, there's a little bias here, but I think it's a pretty decent argument in favor of regression (if the endless, more rigorous studies aren’t enough). To phrase these results differently, the fewer innings he's pitched, the easier it is for a pitcher to beat a league average HR/FB – luck! The more he pitches, the more he regresses and falls off our list. At the 3-year mark, we only have 5 pitchers below 9% (Cain, Kelvim Escobar, Clemens, Wainwright, and Wang). At the 4-year mark, it's only Cain.
Regression is real, but that's not to say that xFIP or LIPS or any other ERA estimator is the end all, because it's not. It's a shortcut, a quick way of seeing what a pitcher's peripherals tell us about him for that particular year. It’s not meant to be a forecast. It uses one year of data and implies 100% regression to the mean for BABIP, HR/FB, and LOB%, which is incorrect (but not terribly so for the vast majority of cases, which is why these things are usable if we know what we’re looking at).
Regression is real, but I think a lot of analysts give us the wrong impression of it. They either (out of laziness or ignorance) assume that every BABIP, HR/FB, LOB% should be league average at all times. That’s not what regression is! Regression means that the player’s numbers should move in the direction of a league or group average – how far towards that number depends on a number of factors (in some cases it may not move much at all from the player’s actual performance).
For a guy like Aaron Harang, he has a .310 career BABIP and a .329 or so BABIP over the past three years. For a guy like this, with this much data, it would be foolish to assume he’ll post a .300 BABIP going forward. But it’s also foolish to assume that he’ll post a .329 BABIP going forward as well (in the absence of some other data that says he deserves a high BABIP). If we know something about Aaron Harang from scouting or other means, like I said about Haren, we can say that it’s best to regress Harang to, say a .320 BABIP. But if we don’t have these things, the best we can do is regress to league average (or some group average).
This doesn’t, however, mean that we assume Harang will have a league average BABIP. That’s not what regression is. It just means our estimate will move some distance toward that number. We take all the data we have on him, and based on that sample size and the league-wide variance in BABIP, we can come up with a good estimation of his BABIP going forward. This will be far more accurate than simply saying, “For three years in a row, Harang has had a high BABIP and an ERA higher than his xFIP, so xFIP is useless (not just for Harang, but for all players) and we should just use Harang’s ERA or our gut impression of him.”
So to answer Chris’s question, yes, I would make the exact same case for Harang or Masterson. That is, yes, it’s possible that these guys truly deserve high BABIPs (or HR/FBs or whatever), but unless you have some sort of information that shows me that they should, I’m simply going to take the data I have, and regress the proper amount (and, ideally, treat vs. LHP and vs. RHP separately (but not independently), especially for Masterson). We’re just looking at a different magnitude here. For Haren’s BABIP, it’s one year and will be nearly completely erased when we account for previous seasons and regression. For Harang, it’s several years and will show up somewhat in our projection.
Just because there are these guys that look like they are “exceptions” doesn’t mean that they don’t follow the rules of regression. They do – they just regress less the more data we have on them. And if we have some scouting or other information, they regress to a number other than league average. Everyone regresses, but a lot of people assume that everyone regresses to league average, when in fact they don’t. In fact, very few players regress to league average. Everyone, truly, regresses to their own absolute true talent level (which is unknown), so we do the best we can to estimate that. League average is the bare minimum acceptable guess we can make, but once we know some things about the player, we can regress him to a group of players similar to him (for example, small-framed lefties with underwhelming stuff and a fastball-slider-change repertoire) or to some unique number that better suits him than league average.