October 28, 2009 at 8:00 am by Capitol Avenue Club under Atlanta Braves
Pythagorean Winning Percentage theory states that a team’s ability to score and prevent runs is a better representation of their fundamental ability to win baseball games than their actual record. (See Steven J. Miller’s derivation of the formula) The theory seems intuitive to me. The ability to score runs and prevent them is a team attribute, where as winning the games seems to be a function of deviation from the averages. Whether or not that’s an actual attribute is up for debate, but Pythagorean Winning Percentage theory doesn’t believe it is. I mostly tend to agree.
However, a debate has surfaced as to whether or not teams can take measures to outperform their Pythagorean Winning Percentage. Is the ability to consistently over preform really an ability or is it just luck?
One prominent belief is that a good bullpen, specifically the back end thereof, allows a team to over achieve. The logical argument is that a good back end of the bullpen allows a team to win a lot of close games, but it doesn’t factor into blowouts. Example – 2007 Diamondbacks. Their Pythagorean record was 79-83. They overachieved this mark by 11 wins and won the NL West behind a 90-72 record, despite having only the 4th best Pythagorean Winning Percentage in their division. Proponents of the argument claim that their strong bullpen–featuring Jose Valverde, Tony Pena, Juan Cruz, Doug Slaten, and Brandon Lyon–is responsible for the over achievement. Those who don’t buy the argument say the 11 game over achievement can be explained entirely by luck.
My belief is that the truth is probably somewhere in the middle. Back to the Bill James principle (the principle of questioning every belief. Always ask the question, “Is that true?”). Is this true? Does a strong bullpen, or a strong closer, really allow a team to over achieve? I’ve designed a study that attempts to answer this question.
For 2009, I’ve listed every team, their 3rd order winning percentage, their actual wins, their 3rd order wins, and the difference between their 3rd order wins and their actual wins (dubbed “luck”) in the chart below:
If you accept the systematic assumptions that the Baseball Prospectus crew makes when they calculate their Pct3, this should be a fairly good indicator of how lucky a team was.
I’ve chosen WXRL as my statistic to model how good the back end of each bullpen performed. WXRL answers the question “how much more cumulative win probability has this reliever added to his team over what a replacement level reliever would, adjusted for the quality of the opposing batters”. I’ve then plotted the “Luck” column on the X-axis versus the WXRL of the team’s top WXRL reliever (usually, but not necessarily, the team’s closer). If strong closer performances are, indeed, responsible for Pythagorean over performance, we should see a roughly linear pattern, trending upward.
There isn’t much convincing evidence here that closers, or a team’s most effective reliever, can vastly influence Pythagorean over achievement. An R value of 0.374410 isn’t particularly strong, a p-value of 0.041750 indicates our results aren’t too robust in the first place, and a R^2 value of 0.140183 suggests that around 14% of Pythagorean over achievement can be explained by an effective closer, best case scenario.
But what about closer and set-up man?
The abThe above figure is that of a scatter plot of the sum of the WXRL of the team’s top two relievers versus their Pythagorean luck. The results are starting to become more meaningful. A correlation coefficient of 0.479670 is much better than the 0.374410 from the previous plot. A p-value of 0.007267 suggests our results are more significant, and a coefficient of determination of 0.230083 suggests as much as 23% of the Pythagorean over achievement in 2009 can be explained by strong performances from the closer and set-up man.
Just for form, let’s take it one step further, top 3 relievers:
As alAs always, sample size is an issue, but it’s pretty clear to me that having a strong back end of the bullpen has some influence on Pythagorean over performance. In this graph, we have a correlation coefficient of 0.554878, a coefficient of determination of 0.307890, and a p-value of 0.001456. What does this mean? Well, the two things do correlate, 31% of the change in Pythagorean over performance in 2009 can be explained by a strong trio of relievers, and our results are statistically significant.
I’m not out to shock the baseball world, I’m sure a lot of people in front offices have done studies similar to this one (or have reached similar conclusions through other methods), but I do think we (outsiders to the game) oversimplify the value of relievers. When analyzing the game economically, most people conclude that a pitcher pitching only 72 innings can only have so much impact and that the bullpen should be the last area that a team attempts to address. Component-wise, yes, this is true. But I think there’s sufficient evidence that a good bullpen allows you to out-preform your components, which is valuable in itself.
Final take-away: I think this also runs parallel to my thoughts on bullpen construction theory in a different way, that diversification is the better play. You’ll see that the performance of the top 3 displays a much stronger correlation with Pythagorean over achievement than the performance of the most valuable reliever, alone. Of course, every case should be treated individually, but the strength of the bullpen (or the back end thereof) as a whole is more important than the upside of any singluar relief ace. Case study: 2009 New York Mets. Spend $36 million on the FA market’s top closer, only to a) have him be out-performed (on the Mets) by Pedro Feliciano (and by a rather substantial margain) in addition to many other moderately priced relievers on other teams (including Mike MacDougal, Mr. 31-to-31 strikeout-to-walk ratio. *Embarassing!* Good move, Omar) and b) post the 4th lowest cumulative WXRL of their top-3 relievers in baseball.