This is the 4th and final part of an article discussing the relative importance of factors in winning NFL games. Part 1 is here, part 2 is here, and part 3 is here.
ANALYSIS
A good passing game is far more important than a good running game in the NFL. It’s at least twice as important, and probably even more so. If we include interceptions as part of the passing game, passing efficiency and interception rates dwarf the importance of running efficiency by a factor of 4 to 1.
An alternate way of looking at interceptions is that they are a threat to the passing game, so their importance should be subtracted, not added, to passing efficiency to properly compare running and passing. Although it's a valid consideration, you cannot win without passing, and so the risk of interception is not optional. The bottom line is that if a team would rather be good at passing or good at running, it should choose passing.
Part of the conventional wisdom about the running game in the NFL is that it “sets up” the passing game. It keeps the defense off balance and unable to focus exclusively on defending against the pass. This may be true up to a point, but it appears that the balance between defending against the run and the pass is far out of equilibrium.
If a good running game gives a team an advantage in passing then we would see a positive and significant correlation between offensive running and passing efficiency. In fact, the correlation is 0.13, which is very weak and not statistically significant. Running well does not prevent interceptions either. The correlation between running efficiency and interception rates is 0.12. Further, including an interaction variable (running * passing) in the regression model results in an insignificant coefficient and a marginally weaker model. We have to conclude that running and passing are fundamentally independent of one another.
It appears that offensive passing is more important than defensive passing, and that offensive running is more important than defensive running. We might conclude that offense is more important than defense, but it may not be that simple. For starters, that violates the symmetry of the sport. Points allowed are equally as important as points scored.
One explanation becomes clear when you add the coefficients of offensive passing and offensive interceptions (1.14 + 0.45 = 1.59). Compare that with the sum of defensive passing and defensive interceptions (0.92 + 0.76 = 1.68). We see that the balance between offensive passing and defensive passing starts to equalize.
Comparing the sum of all offensive weights with the sum of all defensive weights yields a very balanced result. Offensive variables add up to a total weight of 2.38 vs. a total weight of defensive variables of 2.34. Although the defensive variables appear slightly stronger, the relative sums of the weights are within 1.7%--remarkably balanced.
Passing is indeed far more important than running, and although offense appears more important than defense, they're equally important.
What Makes Teams Win? 4
What Makes Teams Win? 3
This is the third part of a four-part article discussing the relative importance of factors in winning NFL games. Part 1 is here and Part 2 is here.
CORRELATION SUMMARY
So far we’ve analyzed each phase of the game and its statistical connection with regular season team wins. Below is a table that lists the relevant statistics and their correlations. The table is sorted in order of absolute strength of correlation.
| Stat | Win Correlation |
| Off Pass Yds/Att | 0.61 |
| Def Pass Yds/Att | -0.47 |
| Off Fumble Rate | -0.46 |
| Off Int Rate | -0.45 |
| Def FFumble Rate | -0.41 |
| Def Int Rate | 0.39 |
| Off Pen Rate | -0.37 |
| Off Run Yds/Att | 0.18 |
| Def Run Yds/Att | -0.04 |
The table is presented graphically below. Negative coefficients, such as defense pass efficiency, are shown as positive values to make it easier to compare each variable's relative importance.
The relative importance of each aspect of the game begins to come into focus. Passing is most important, followed by turnovers, then penalties and running. For every aspect, the correlation on the offensive side of the ball is stronger than on the defensive side.
But this isn’t the final word. Correlation coefficients by themselves do not take into account the other factors. In other words, they ignore the effect of the other stats when calculating the correlation.
REGRESSION
To take all facets of the game into account simultaneously and produce a valid model of winning NFL games, we can use linear regression to estimate coefficients for each stat. The relative value of the coefficients will reveal the relative importance of each phase of the game, holding all other variables constant. This will yield estimates that are more pure and accurate than simple correlations.
The dependent variable of the regression model is regular season wins. The independent variables are the efficiency stats I’ve previously outlined. The data set continues to be all 32 teams over the past 5 seasons for a total of 160 observations. The results of the regression are detailed below.
| VARIABLE | COEFFICIENT |
| const | 5.31 |
| O Pass Eff | 1.43 |
| D Pass Eff | -1.65 |
| O Int Rate | -53.50 |
| D Int Rate | 81.70 |
| O Fum Rate | -49.10 |
| D FF Rate | 70.90 |
| O Run Eff | 1.00 |
| D Run Eff | -0.55 |
| Pen Rate | -2.73 |
| R-squared | 0.802 |
Each of the independent variables are statistically significant at the 0.05 level or better, except defensive run efficiency, which is significant at 0.06. The R-squared value indicates an extremely good overall fit for the model. 80% of the variance in team wins can be explained by the included variables. The remaining 20% could be due to any number of factors, but we have to accept that outcomes in any sport are partly due to luck.
Using the regression results we can estimate a team’s expected wins using a linear equation. Here is what the equation would look like:
Wins = 5.31 + (1.43 * O Pass Eff) + (- 1.65 * D Pass Eff) + …
The regression coefficients are stated in terms of wins per unit of the variable. For example, the coefficient for offensive pass efficiency (yds/att) is 1.43. So for every 1 yard improvement in pass efficiency a team can expect 1.43 additional wins. When coefficients are stated this way, it makes it very easy to estimate the effect on the dependent variable (wins) given a change in one of the independent variables. But it makes it very difficult to get a sense of the relative importance of each variable. Defensive forced fumble rates are certainly not 70 times more important than offensive run efficiency.
To reveal the true relative importance of each factor, we need to standardize each variable by calculating the number of standard deviations from its average value. In statistics, these are known as “normalized” or "standardized" variables, noted by the prefix “z.”
Here are the regression results again, this time calculated with standardized coefficients. The significance of each variable, and the overall fit of the model remain the same since only the units of the variables have changed.
| VARIABLE | COEFFICIENT |
| constant | 8.06 |
| Z O Pass Eff | 1.14 |
| Z D Pass Eff | -0.92 |
| Z O Int Rate | -0.45 |
| Z D Int Rate | 0.76 |
| Z O Fum Rate | -0.33 |
| Z D FF Rate | 0.42 |
| Z O Run Eff | 0.46 |
| Z D Run Eff | -0.24 |
| Z Pen Rate | -0.39 |
It may seem like we’ve gone through a tortured process to arrive at these coefficients. But they are merely the mathematical weight we would need to give each factor to have the best estimate of actual team wins. These are based on real-world data from every team’s season between 2002 and 2006.
Here is a graph representing each variable’s relative weight. Negative coefficients, are shown as positive values.
Probably the simplest way to interpret the chart is this way. If my team is average in absolutely everything, I'd expect to win 8 games. But if my team is average in everything except offensive pass efficiency, in which we're one standard deviation above average, I'd expect to win 9.14 games (8 + 1*1.14).
So if my team was the league's best at running the ball, say 2.5 standard deviations above average, but average at everything else, we'd expect to win 9.15 games (8 + 2.5*0.46). Compare that to passing--if my team were average at everything but best in the league in passing, we'd expect to win 10.85 games (8+2.5*1.14).
Continue reading the fourth and final part of this article.
What Makes Teams Win? 2
This is a contiuation of an article discussing the relative importance of factors in winning NFL games. Part 1 is here.
TURNOVERS
Turnovers can be analyzed in the same manner. If we want to know how good a defense is at intercepting passes, we should look at interceptions per pass and not total interceptions. Offenses may shy away from passing the ball if a secondary is known to be excellent at generating interceptions. Their ‘total interceptions’ stat would suffer as a result of teams leaning towards the run, but their interception efficiency would tell the real story.
Fumbles are slightly different. Once the ball is on the ground, who recovers it is a very random thing. Fumble recoveries are almost purely the result of unplanned conditions, such as where players happen to be and who happens to be facing the direction of the ball. (Note: Skills like running, passing, or generating interceptions are repeatable and enduring skills which tend to carry over from year to year. Fumble recovery percentage does not.) The best measure of the fumble, therefore, is the forced fumble, which does carry over from year to year. Defenses can be good at jarring the ball loose, but who recovers the ball is random. The more often the forced fumble, the more opportunities the defense has of recovering. Forced fumble rates correlate better with wins, which bolsters its case as the stronger factor in winning.
| Stat | Win Correlation |
| Def Fumbles | 0.33 |
| Def Forced Fumbles | 0.41 |
On the offensive side of the fumble, the opposite is true. Unforced fumbles are the repeatable, enduring skill--or I should say avoiding the unforced fumble is the skill. Therefore, the fumble per play stat is the optimum stat to measure an offense’s tendency to fumble. Keep in mind that fumbles happen on both pass and run plays.
SPECIAL TEAMS
So far I’ve ignored special teams. There’s a reason for that—they don’t really matter on a consistent basis. Of course, we can remember a critical play here, a miracle play there--they can change the outcome of a game on rare occasions. We can remember these spectacular plays because they are so exceptional. But no matter how we measure special teams in a meaningful way, the correlation coefficients are not statistically significant. That’s not to say that teams can ignore special teams. Every advantage a team can find is important, but in the long run, the importance of kickoffs, punts, return yards, and field goal percentages are all dwarfed in relation to things like running, passing, and turnovers.
There are several valid explanations for the relative unimportance of special teams. First, and perhaps most obvious, is that special teams plays are far less frequent than pass or run plays. Second, the difference between the best special teams and the worst special teams is small in comparison to the spread between the best and worst offenses and defenses. Third, special teams stats are very hard to evaluate. For example, average net punt yards negatively correlate with winning at -0.17. This means the better the team is at punting, the fewer wins it can expect. But I believe this is misleading because teams with good offenses would be able to kick more “inside the 20 yard line” punts than teams that often kick from their own 30 yard line.
One last point about special teams: Big special teams plays, such as kickoff returns for touchdowns, are rare. In the past five seasons, there has been an average of 13.6 kickoffs returned for touchdowns--by all 32 teams combined. The best test of skill vs. luck I have come across asks the question, “does it tend to be repeatable?” Kickoff or punt returns for touchdowns, or 60+ yard field goals, unfortunately, do not tend to be repeatable.
PENALTIES
Penalty rates, however, do tend to be repeatable. Teams that have committed lots of penalties tend to continue to commit more, and vice versa. Experience, discipline, preparation, and playbook complexity probably all contribute to penalty rates. Other factors probably come into play. For example, it might be expected that weak secondaries would tend to commit more pass interference penalties in desperation to prevent deep passes.
Penalties and penalty yards correlate negatively with winning, as we’d expect. (Penalty rate is defined as penalty yards per play.)
| Stat | Win Correlation |
| Team Penalties | -0.21 |
| Team Pen Yds | -0.18 |
| Team Pen Rate | -0.37 |
The penalty rate stats have the stronger correlations with wins than either penalties or penalty yards.
Continue reading part 3 of this article.
What Makes Teams Win? 1
This is the first part of a four part article summarizing the relative importance of each phase of the game in winning.
INTRODUCTION
Passing is much more important than running, and offense appears to be more important than defense. Turnovers matter a great deal, interceptions more than fumbles. Penalties matter too, but not like you’d think.
We've been told by NFL analysts for years that defenses win championships and that a solid running game is the key to winning. I'd agree that things appear that way. Teams with lots of passing yards aren't always winners, but teams with lots of running yards almost always are. But, as you're about to find out, appearances can be deceiving.
One way to measure the relative importance of passing vs. running, or offense vs. defense, is to measure their statistical correlations with season wins. For example, the correlation coefficient of a team's total rushing yards correlates with its number of wins in a season is 0.45. A correlation of 1.0 would be perfect correlation, while a correlation of 0.0 would indicate no relationship. So 0.45 implies that total rushing yards is a moderately strong indication of how many games a team would win.
In comparison, total passing yards correlates with wins at 0.31. Compared with total rushing yards, total passing yards is less important in terms of winning games. This is what most fans and analysts notice when watching games or glancing at team stats. But does this mean that passing is less important than running in the NFL?
Before we settle on an answer, we need to consider the number of attempts of runs and passes. In baseball, the statisticians understood this 150 years ago when they created the batting average. If one player has 300 hits and another player has 350 hits, who is the better batter? The answer is we don't know until we divide the number of hits by their at-bats.
RUN AND PASS EFFICIENCY
How often have you heard an NFL commentator say, "When running back X gets at least Y carries, his team wins?" The clear implication is that the team in question should feed a steady diet of carries to the running back, and this will cause his team to win. What if we stated the same observation this way, "When his team is winning, running back X receives at least Y carries?" And by the way, why don't we ever hear, "When QB X passes at least Y times, his team wins?" Here's why:
| Stat | Win Correlation |
| Rush Attempts | 0.58 |
| Pass Attempts | -0.17 |
The negative correlation for pass attempts means that the more often a team passes, the less likely it is to win. The correlation of rush attempts with wins (0.58) is even stronger than that for total rushing yards (0.45). This is a curious result, and it’s where conventional NFL analysis begins to crumble.
When we see two things that appear correlated, it is natural for us to say that one causes the other. The runs come during the game, and the win comes at its conclusion. Therefore most fans and analysts assume the running causes the winning. The problem is, it usually doesn't. It's the winning that causes the running. Teams that are ahead, and likely to win, run the ball to take time off the clock and to minimize the risk of a turnover. Teams that are behind, and likely to lose, abandon the run in favor of the pass. Statistics can measure the correlation, but it can't determine the direction of causation.
The critical question then becomes: how can we truly measure a team's passing and running abilities and their respective contributions to winning? The answer is football's equivalent to the batting average--efficiency stats. If we want to know how good a team is at running, the best way is to know how many yards it tends to gain each time it runs the ball--yards per rush attempt. The table below lists basic passing statistics and their correlation with season wins.
| Stat | Win Correlation |
| Pass Yards | 0.31 |
| Pass Attempts | -0.17 |
| Pass Yds/Att | 0.61 |
Yards per pass attempt is merely pass yards divided by pass attempts. So we have a relatively weak statistic (0.31) divided by an even weaker one with a negative correlation with winning (-0.17). We would expect to have a fairly meaningless result, but we don't. Passing efficiency turns out to be strongly correlated with winning (0.61). And unless having a lead in a game ‘causes’ a team’s passes to be more successful, we can safely say that passing efficiency leads to winning.
Because sacks are an important factor in the passing game, I include plays that result in sacks as pass attempts for the purpose of calculating efficiency. Likewise, I also subtract sack yards from total passing yards. I call this true pass efficiency and have found it correlates better with both offensive points scored and wins.
Continue reading part 2 of the article.
How Good Will NE Be in ’07?
The Patriots are the consensus pre-season favorites to win the Super Bowl next February. The going odds that they will take home the Lombardi Trophy are, astonishingly, almost even at 9 to 5. The optimism in New England is due to the acquisition of three star players—wide receivers Randy Moss and Dante Stallworth, and linebacker Adalius Thomas.
MOSS AND STALLWORTH
The wide receiver position was seen as one of the Patriots' weaknesses last year, so the off-season upgrades are expected to dramatically improve their passing game. Unspectacular players such as Reche Caldwell and Jabar Gaffney became Tom Brady’s primary targets by the end of last season. The combination of a quarterback as good as Brady and receivers as talented as Moss and Stallworth could make New England’s passing attack dominant. But just how good can we expect them to become?
By replacing the performance of Patriots’ top two receivers of ’06 with the appropriate stats of Moss and Stallworth, we can estimate the most important measure of a team’s passing game—passing yards per attempt.
First, some assumptions: Tight ends and running backs will have the same performance as last year. Tom Brady will have his usual brilliant season. He’ll throw as few interceptions per attempt as he did last year, and he’ll be highly accurate with a 61.9% completion rate, which happens to be both his career average and last year’s rate. (Brady has been remarkably consistent. His best year was 2001 with a 63.9% completion rate, and his worst was 2003 with a 60.2% rate.) We’ll also assume both new receivers will remain healthy all year.
First, let’s look at Randy Moss. His stats over his last two seasons at Oakland are listed below. ADJ YDS is receiving yards adjusted for playing a full 16 game season.
YR G REC YDS AVG REC/G ADJ YDSMoss’s performance in 2006 was obviously hurt by playing with a sub-par team. Raiders quarterback Andrew Walter cannot be compared to Brady. Moss’s performance in 2005 was actually better than his final year in Minnesota, when he wasn’t even the top receiver for the Vikings. We’ll use his ’05 stats.
2005 16 60 1005 16.8 3.8 1005.0
2006 13 42 553 13.2 3.2 682.3
Despite all the attention on Moss, Dante Stallworth may prove to be the better acquisition. His stats over the last two years are below. He was in Philadelphia in ’06, and New Orleans in ’05.
YR G REC YDS AVG REC/G ADJ YDSStallworth had impressive stats both years. After projecting his numbers for playing a full 16 game season in ’05, we’ll use the average of his stats for the last two years.
2005 12 38 725 19.1 3.2 967.7
2006 16 70 945 13.5 4.4 945.0
Last year, New England’s passing offense was mediocre. They ranked 11th in total yards, but more importantly, only 15th in passing efficiency with 6.81 yards per attempt. To see how much that stat can improve we’ll start by looking at the ’06 receiving corps. WRs with fewer than 3 catches are excluded to save space.
NAME G REC YDS AVG REC/G ADJ YDSNow let’s replace the Patriot’s top two receivers with Moss and Stallworth. We’ll assume the #3 receiver remains Troy Brown, or at least will have stats very similar to his. We’ll also leave the 4th and 5th receivers as they are. Those receptions may go to Caldwell, Gaffney or Jackson, or to second tier acquisions Wes Welker and Kelley Washington, depending on who is healthy and active. But the resulting yardage should be similar for the #3 receivers and below on the depth chart. Below is the Patriot’s projected receiving corps for ’07.
Reche Caldwell 16 61 760 12.5 3.8 762.5
Doug Gabriel 12 25 344 13.8 2.1 460.0
Troy Brown 16 43 384 8.9 2.7 382.7
Jabar Gaffney 10 11 142 12.9 1.1 227.0
Chad Jackson 12 13 152 11.7 1.1 202.8
NAME G REC YDS AVG REC/G ADJ YDSWhere last year the Pat’s WRs contributed 1798 yards, next year’s squad should produce about 2706 yards, a 50% improvement. Add a steady 170 catches for 1792 yards from TEs and RBs for a total of 362 receptions. To achieve that number of receptions given a 62% completion rate would require 584 attempts.
Randy Moss 16 61 1021 16.8 3.8 1021.0
Dante Stallworth 16 61 991 16.3 3.8 991.0
Troy Brown 16 43 384 8.9 2.7 382.7
(Jabar Gaffney) 10 11 142 12.9 1.1 227.0
(Chad Jackson) 12 13 152 11.7 1.1 202.8
The result is 362 catches for 4498 yards on 584 attempts, improving the Patriot’s pass efficiency from 6.81 to 7.70 yds/att. This would rank New England’s passing offense third in 2006 and 21st out of 160 in the last five seasons. But what does this mean in terms of wins?
By using a regression model to estimate regular season wins based on efficiency stats, we can compare how many wins the Patriot’s should expect given both last year’s and this year’s receivers.
The regression model uses each team’s offensive and defensive running and passing efficiency stats, plus turnover rates and penalty yards, to calculate a probability of the outcome of any given game. After calculating each game’s probability for the 2007 season, we can then sum the probability of each possible season outcome for the Patriots to estimate how many wins a team will have.
For the Patriots, we’ll run the process twice, first using the pass efficiency from ’06, and then with the projected ’07 pass efficiency. Here is how the ’07 season would look using the old stats:
Vis. Home P(V) P(H)Using the law of total probability, we can estimate the Patriots would win approximately 9 games this year, given their exceptionally difficult schedule. Here is the actual distribution:
NE NYJ 0.49 0.51
SD NE 0.57 0.43
BUF NE 0.24 0.76
NE CIN 0.47 0.53
CLE NE 0.12 0.88
NE DAL 0.41 0.59
NE MIA 0.50 0.50
WAS NE 0.19 0.81
NE IND 0.27 0.73
NE BUF 0.60 0.40
PHI NE 0.57 0.43
NE BAL 0.34 0.66
PIT NE 0.41 0.59
NYJ NE 0.32 0.68
MIA NE 0.32 0.68
NE NYG 0.56 0.44
But with their new receivers, New England can expect better probabilities. Here is their revised 2007 season and the associated total probabilities of season wins:
Vis. Home P(V) P(H)
NE NYJ 0.58 0.42
SD NE 0.48 0.52
BUF NE 0.18 0.82
NE CIN 0.56 0.44
CLE NE 0.09 0.91
NE DAL 0.49 0.51
NE MIA 0.59 0.41
WAS NE 0.14 0.86
NE IND 0.34 0.66
NE BUF 0.68 0.32
PHI NE 0.49 0.51
NE BAL 0.42 0.58
PIT NE 0.33 0.67
NYJ NE 0.25 0.75
MIA NE 0.25 0.75
NE NYG 0.64 0.36
The Patriots are now projected to win 10 games instead of 9. The addition of Moss and Stallworth are worth 1 additional win. I expected a much larger difference, particularly because offensive pass efficiency is the factor with the most weight in the regression model. But another way to look at the improvement is to compare how the game-by-game probabilities changed. Notice that the addition of Moss and Stallworth are worth about a 9 point improvement in the probability to win each game. That’s an 18 point swing when considering the opponent’s win probability is equally reduced.
ADALIUS THOMAS
Projecting the improvement to the Patriot’s defense due to Adulius Thomas’s addition is tougher than for the WRs because it is much harder to quantify Thomas’s contribution to team defense. He is extremely versatile and athletic enough to impact every part of defensive football, from sacks to stopping the run to defending passes.
I propose a simpler but less exact approach. We will compute the share of Thomas’s contribution to his former team’s defense. The Ravens’ defense was historically good in 2006, ranking first or second in nearly every statistical category. We will improve the Patriot’s defensive stats towards those of the Ravens in proportion to Thomas’s share of overall team defense. In other words, if Thomas counted for half the tackles, assists, and sacks of the Ravens, then we would credit the Patriots with an improvement of 50% of the difference between Baltimore and New England’s team stats.
In 2006, The Ravens’ and Thomas’s stats were:
TKL AST SCK INT PD FF FRThomas recorded 9% of the Ravens’ tackles and 18% of their sacks. Excluding the forced fumble stat, the average share of Thomas’s contribution was 9%. To be generous, and to account for moving to a team with fewer stars where he can make a bigger impact, we’ll give Thomas credit for 10%.
Ravens 707 200 60 28 111 12 11
Thomas 64 19 11 1 7 0 1
Thomas(%) 9% 10% 18% 4% 6% 0% 9%
The adjustment to New England’s defensive efficiency stats is calculated below. YPA is passing yards per attempt, YPR is yards per run, and INTPA is interceptions per pass attempt.
YPA YPR INTPAHere is how NE’s season looks with their new WRs and Adalius Thomas.
NE ’06 5.70 3.90 0.042
BAL ’06 5.29 3.30 0.055
Adjustment -0.04 -0.06 +0.002
NE ’07 5.66 3.84 0.044
Vis. Home P(V) P(H)
NE NYJ 0.59 0.41
SD NE 0.47 0.53
BUF NE 0.17 0.83
NE CIN 0.57 0.43
CLE NE 0.09 0.91
NE DAL 0.50 0.50
NE MIA 0.60 0.40
WAS NE 0.14 0.86
NE IND 0.35 0.65
NE BUF 0.69 0.31
PHI NE 0.47 0.53
NE BAL 0.43 0.57
PIT NE 0.31 0.69
NYJ NE 0.24 0.76
MIA NE 0.24 0.76
NE NYG 0.65 0.35
Thomas’s contribution is significantly less dramatic than that of Moss and Stallworth. He counts for about 1 additional point of probability per game, for a 2 point swing, and adds far less than one win to the Patriot’s projections.
Rating Gameday Head Coaches
The head coach in the NFL must be a strategist, a leader, a talent evaluator, a manager, and perhaps most importantly--a field marshall. By field marshall, I mean the person who sets gameday tactics, who sets play call doctrine, who manages the clock, and who decides whether to go for it or kick on 4th down.
At kickoff, a head coach is given a certain ability level in his team. During the game it is his job to squeeze every possible advantage from his team's capabilities, while minimizing its weaknesses. Part of his duties are to accurately assess the risks and rewards of every situation. His team is ahead 14-10 and it's 3rd down and 7 late in the 4th quarter. Should he decide to run and likely have to punt, but chew up some time and force his opponent to use a time out? Should he pass and go for the 1st down but risk stopping the clock, or even worse, an interception? The answers depend on how accurate his perception is of his offense's passing ability and his defense's ability to stop the opposition.
So given those abilities, how can we judge a head coach on his ability to make the best in-game decisions? Here is one way.
In the last post, I charted teams' expected wins vs. their actual wins. The model that estimated the expected wins had an r-squared of 0.85. This mean 85% of the variance in team wins could be accounted for by the model's variables. (If you're not a statistician, that means it's really good.) Some teams had steady expected wins from year to year, but had somewhat erratic actual wins. Some teams had expected and actual wins that matched up every year. But a couple teams stood out as unique.
In recent years, the Lions won fewer games than would be expected given their performance on the field. I admit part of that could be due to luck. However, such a consistent underperformance leads me to suspect something systematic is at work. Could it be that the Lions' coaching staff is not effectively maximizing its team's abilities? Is it inaccurately assessing the risks and rewards of in-game situations? What else would explain such consistent underperformance? Except for an abberation in 2003 (when they only broke even), Detroit annually won about 2 games less than expected.
In stark contrast to the Lions are the Patriots. New England has consistently won about 2 more games than we'd expect given their on-field performance. To me, this appears to be evidence of Bill Belichick's coaching skills, or at least his knack for in-game decision-making. He is known for some unconventional practices, such as going for it on 4th down in more situations than other coaches.
To truly assess a head coach's abilities, we need to follow the coach from year to year and from team to team. Using data from pro-football-reference.com, I collected modern head coach's wins and efficiency stats for each year since the 1982 strike. I chose 1983 as a starting point because it followed the birth of the "west coast offense" and significant changes in NFL rules governing the passing game (which actually began in 1978 but took serveral years for coaches to adapt.) I only considered coaches with 4 or more years of experience. In some cases I included some noteworthy coaches with shorter tenures such as Lovie Smith or Jim Mora Jr. I also included Spurrier and Saban (2 years each) because I was interested in how "big-name" college coaches fared.
I constructed a regression model similar to my previous efforts to estimate expected wins given each team's passing and running efficiencies on both sides of the ball. I propose that the difference between a coach's actual wins and expected wins are, to some degree, "extra" wins that a coach can squeeze from his team given a particular performance level. It is an estimate of his strategic and gameday coaching abilities. Each coach's "extra wins" are averaged and sorted from best to worst.
COACH "EXTRA" WINS/YR
-----------------------------
Dungy 1.73
Smith 1.44
Shottenheimer 1.38
Gibbs 1.32
Billick 0.93
Reeves 0.91
Fisher 0.84
Lewis 0.82
Berry 0.81
Reid 0.79
Shula Don 0.73
Ditka 0.67
Martz 0.60
Belichick 0.59
Pardee 0.50
Mariucci 0.49
Holmgren 0.44
Ross 0.41
Shanahan 0.41
Parcells 0.39
Fassel 0.36
Wannstedt 0.36
Cowher 0.29
Haslett 0.24
Mora 0.21
Green 0.17
Saban 0.17
Johnson 0.17
Noll 0.16
Levy 0.15
Knox 0.05
Sherman 0.04
Robinson -0.08
Walsh -0.08
Coughlin -0.10
Seifert -0.12
Switzer -0.26
Del Rio -0.29
Phillips -0.36
Jauron -0.39
Glanville -0.41
Fox -0.51
Gruden -0.58
Kotite -0.62
Capers -0.66
Edwards -0.69
Erickson -0.69
Shell -0.73
Tice -0.85
Fontes -0.88
Flores -0.89
Vermeil -0.96
Rhodes -1.00
Infante -1.15
Carroll -1.22
Wyche -1.23
Mora, Jr. -1.24
Marchibroda -1.34
Shula David -1.40
Henning -1.42
Spurrier -1.54
Turner -1.63
Bugel -1.77
Riley -1.89
Coslet -2.15
Davis -2.32
Campo -3.20
The list ranks many of the more successful coaches at the top, and some of the least successful at the bottom. Shottenheimer, Gibbs, Reeves, Ditka, Shula are some of the legends at the top. The two coaches from last year's Super Bowl, Dungy and Smith, are at the top as well. Coaches with short and brutally unsuccessful tenures are at the bottom including Dave Campo, Butch Davis, Bruce Coslet, and Steve Spurrier. Some longer tenured coaches such as Ted Marchibroda and Norv Turner don't fare well either. I'm surprised to see top coaches such as Walsh and Cowher lumped in the middle.
But what happened to Belichick? He's down the list behind several other current coaches such as Billick, Reid, Fisher, and Lewis. It may be fair to break Belichick's head coaching career into two separate parts--at CLE and at NE. By his own admission he changed his approach following his time with the Browns, and applied what he learned since then into his current stint in NE. Belichick averaged -1.84 "extra" wins per season at CLE, but earned +2.33 "extra" wins per season at NE. The NE Belichick would be #1 on the list and the CLE Belichick would be third from the bottom. I might attribute such a difference to the small sample size (3 years) at CLE, but such a stark difference speaks for itself.
To be fair to other coaches with distinct tenures, I split the averages of some other notable coaches apart too. Redskins fans may not like what they see.
Gibbs ('83-'92) 2.11
('04-'06) -1.31Ditka CHI 1.37
NO -1.19
Shottenhmr CLE -0.11
KC 1.97
WAS 0.83
SD 0.90
Parcells NYG -0.36
NE 1.97
NYJ 1.16
DAL -0.30
Reeves DEN 1.85
NYG -0.26
ATL 0.11
Note: Also, since this post is appearing on many other sites recently, I've posted a response to many of the excellent comments and criticisms over the past several months.
Next Year's Wins
With a valid and predictive model of team wins (r-squared =0.80), we can estimate how many wins a team "should" have earned, known as expected wins. Based on a regression model using (offensive and defensive) passing, running, and turnover efficiency, plus penalties, we can compare expected wins with actual wins and make some observations.
The first thing I noticed was that expected win totals were much steadier from year to year than actual wins. This supports the notion that the efficiency model is actually a better evaluation of how good a team truly is than a team's actual wins. A team may be "X wins good," but might get lucky or unlucky by a game or two.
Further, if we want to predict the record of a team before the season starts, we'd be much better off by starting from their previous year's expected wins rather than their actual wins. By simply using prior expected wins to predict the following year's wins, the standard error is only about 1.7. That is, the estimate is usually wrong by less than a game or two. By using prior actual wins the standard error is about 3.0--not a very accurate predictor.
Take the recent records of the Jaguars for example. Their actual wins appear to be fluctuating around a relatively steady expected win trend. This gives the appearance that they are an inconsistent team, but in reality they are performing very steadily judging by their passing and running efficiencies, turnover rates, and penalties.
In another example, we see that the Falcons' season in 2004 was something of a fluke. They came back down to Earth after posting a win total over 3 wins higher than what their stats told us to expect. Sure enough, they returned to predictability in 2005. The likely reasons for such a deviation are an easy schedule and some lucky bounces of the ball (and perhaps the upreparedness of league defenses for Vick's running ability).
The Ravens illustrate another observation. Their mediocre performance through 2005 suddenly improved in 2006. This is due to the increase in offensive pass efficiency from 5.1 to 6.4 yards per attempt. The offensive line was the same, and they had the same receivers and tight ends. They only real difference was the quarterback--Steve McNair replaced Kyle Boller. The biggest changes in team fortunes are due to quarterback changes. We can also note that although Baltimore won 13 games, they had about 10 expected wins. This tells us not to hold our breath for another 13 win season in Baltimore, but still count on a solidly above-average team.
Both Baltimore's and Atlanta's graphs illustrate the importance of scheduling. Teams tend to overperform their stats when they have a low number of wins the previous year. Because of the NFL's scheduling system, and possibly somewhat due to draft placement, teams benefit following an "off" year.
Some teams are amazingly consistent and predictable. Notice how flat the Colts' trend line is, and how small the deviation between expected and actual wins.
The Jets are the counter-example. They've been on a roller coaster ride the past 5 years, in terms of actual wins. But their performance as a team has been very steady. It appears to be slightly above average. To Jets fans, it might appear that they had a miraculous turnaround from 4 wins in 2005 to 10 wins and a playoff berth in 2006. But to me, they underperformed in '05, benefitted from an easier schedule, and got a little lucky in '06 to scrape together 10 wins. This is a good measure of "are they for real?" The Jets are, unfortunately, not for real.
The biggest surprise "storybook" season in 2006 belonged to the Saints. So were they for real, or just lucky? Remember that true changes in team fortunes start with changes behind center, and Drew Brees made all the difference. The Saints' chart shows they are for real. They'll be a playoff contender again in 2007.
We can apply the reverse analysis to the "for real" logic. There are some teams that underperformed their stats, and can anticipate improving their win total next year, even without any fundamental improvement in team performance. Look for TEN, STL, MIA, PHI, PIT, MIN, ARI, and JAX to probably improve on their win total from last year.
Math vs. Vegas
I thought it might be interesting to compare my efficiency model's predictions with the consensus predictions. Probably the best estimates of the "consensus" predictions are the Las Vegas betting lines. Although I'm am not interested in gambling, betting odds and point spreads are useful as benchmarks for evaluating the usefulness of all this number crunching.
Note: The over/under lines are often in .5 increments. In these cases I rounded to the win total that is costlier to bet on, and therefore considered more likely. For example, if the line for a team is 9.5 wins and to bet on the 'over' it takes $120 to win $100, and to bet on the 'under' it takes $115 to win $100, I rounded to 10 wins. When the over and under were equally costly I left the prediction at a .5 increment.