That’s why they play the games
One of Baseball Prospectus’s signature projects is its annual PECOTA (Player Empirical Comparison and Optimization Test Algorithm) analysis, which evaluates players and forecasts team success through a formula that includes past performance in major statistical categories.
Although PECOTA (named after former major league infielder Bill Pecota) has become a staple in the baseball community, it doesn’t mean BP gets it right all the time — not by a long shot.
Take, for instance, BP’s White Sox projections since 2005. As you see in the chart below, the projection has gotten it wrong and short-changed the Sox in six of the eight seasons, including twice by double digits (in the ’05 world championship season and in 2008 when the Sox won the thrilling tiebreaker to win the A.L. Central). In all, BP shorted the Sox an average of seven-plus wins, including the ’08 tiebreaker victory.
So, how about the upcoming season? Baseball Prospectus predicts the White Sox will finish 77-85 and third in the A.L. Central behind the Tigers and Indians. Based on BP’s past evaluations, it’s logical to assume that Sox fans can expect good things to happen on the South Side in 2013.
Year Prediction Actual Difference in Wins
2005 80-82 99-63 +19
2006 82-80 90-72 + 8
2007 73-89 72-90 - 1
2008 77-85 *89-74 +12
2009 73-89 79-83 + 6
2010 79-83 88-74 + 9
2011 82-80 79-83 - 3
2012 78-84 85-77 + 7
*actual 2008 record includes the tiebreaker game

Unless there is some fundamental flaw in the PECOTA system, it is actually likely that the prediction will probably relatively accurate this year. To use a statistical turn of phrase, there should be “regression toward the mean.” To put it in practical terms, if both of your parents are very tall (taller than the other relatives in the family), it is likely that you will be shorter than your mom and dad. Like your very tall parents, the PECOTA system has produced a set of “outliers” in predicting the White Sox performance. Chances are that this year won’t be another outlier.
I think PECOTA had a few seasons of forecasting prior to 2005. Can you share those comps?