Does Prior Experience Matter for a Closer?July 28th, 2011 by Derek Carty in General Guidance, Theoretical, Uncategorized
This season, I’ve been conducting a series of studies in an attempt to better understand which closers will accumulate the most saves—the primary reason we draft them in the first place. Over the past couple of months, I’ve found that closers who begin the season with a tenuous hold on the closer’s chair tend to wind up with far fewer saves than those who begin the year with the job all to themselves. I’ve also found that relievers with poor skills who “aren’t good enough to be closers” tend to accumulate nearly as many saves as relievers with elite skills. Today, I thought I’d add a third variable to the mix: experience.
Managers love to talk about how a closer has experience saving games, and it sure seems like the Ryan Franklins of the world get more chances than the Matt Thorntons. Let’s see whether this is actually the case of if it’s merely a couple of bad beats that tend to stick in our heads (insert Rounders reference here).
The first way I’m going to measure “closing experience” is to look at all relievers from 2001-2010 who began the year closing games broken down by the number of times over the previous six seasons they finished with at least 30 save opportunities. That is, for every season that a reliever finishes with at least 30 save opps, he is credited with a year of closing experience. Let’s check it out:
Well this is interesting. I came into this study fully expecting to see a very large gap between the guys like Mariano Rivera and Francisco Rodriguez who have closed for years and the guys like Thornton and Kyle Farnsworth who were first-time closers. The logic behind that hypothesis was that a team would be more willing to look past small sample size struggles of a proven closer than one who was new to the job, chalking it up to the new guy not being closer material or not being able to handle the pressure.
Looking at the chart above, we see roughly three levels. The first level belongs solely to the first time closers, who are expected to save 25 games, absent other information. Once a closer has at least one year of experience logged, however, he’s expected to save roughly 29 games per year up until the time he manages to get his fifth year of experience under his belt. Once he has at least five years of experience, his saves expectation jumps up to 34-35 per year.
That group is home to the absolute elite; only 10 closers—total—have made the list over the past ten years: Mariano Rivera (9 times), Trevor Hoffman (9), Troy Percival (3), Billy Wagner (3), Jason Isringhausen (3), Robb Nen (2), Francisco Cordero (2), Francisco Rodriguez (1), Joe Nathan (1), and Roberto Hernandez (1).
Wait a second…if these guys are so good, then how come there aren’t more repeaters? If these guys are saving 35 games per year, that should carry over as another year of experience to the next year, shouldn’t it? Excellent question observant hypothetical reader. If I remove Rivera and Hoffman’s numbers, closers with 5 or 6 years of experience only average 31 saves per year—right in line with all other closers with at least one year of experience.
Even if a closer manages to reach this elite level, and even if we include Rivera and Hoffman, we’re only looking at a gain of 10 saves on his expected performance. That’s certainly significant, but it’s not enormous, especially when Mariano Rivera goes for $20 and a guy like Farnsworth goes for $6. Maybe you can justify giving that to Mariano, but otherwise, you may only be looking at a gain of 5-6 saves for Jonathan Papelbon (who went for $18) over Farnsworth.
In statistical terms, we get just an 0.04 r-squared when running a linear regression between closing experience and saves. That means that years of closing experience explains just 4 percent of the difference between closer save totals.
To be thorough and to avoid the subjectivity in defining what a “year of closing experience” is, I decided it would also be a good idea to look at how many saves the closer has accrued in his career, coming into a given season.
The gap here is even smaller than before. The difference between a player with little-to-no closing experience and one with lots of experience is just 8 saves.
Additionally, our r-squared drops to 0.02 here.
That about wraps it up for this time. Through three parts of this series, I think we've learned quite a lot about closers and how to approach drafting them. While I think the last article really challenged conventional wisdom, this one was equally surprising, at least for me.
I was really expecting to see a much weaker showing for first year closers and a much stronger showing for fourth year closers. The difference between them, however, is just four saves over the course of the entire season. I was fully expecting to write a section of this article talking about how ill-conceived my confident drafting of Frank Francisco (last year and this year) and Matt Thornton (this year) looked in retrospect, but that doesn't appear to be the case. It's looking more and more like I simply got unlucky, but I'll continue looking next time.