A few weeks back I wrote about how the distribution of team offensive production is measurably wider than team defensive production. Although I've written about the phenomenon a few times over the years, it never hurts to apply newer and better analytic tools to the question.
I had produced this histogram to illustrate the comparison between offense and defense, but the format doesn't mesh very well at NYT. For those not familiar, a histogram plots the frequency of occurrence of various levels of a variable. In this case it's a plot of team total Expected Points Added (EPA) for the 2000-2012 regular seasons. For example, there were 62 defenses (the lighter plot) that totaled between 10 and 35 EPA for a season. And there were 43 offenses that totaled the same amount.
Great Offenses > Great Defenses Visualized
Deadspin/Slate Roundtable: Passing in 2011
Tables vs. Graphs
The concept of time is one example. We think and talk of time, a concept virtually without its own terminology, in terms of space and motion: Time goes by...Our best days are ahead of us...I'm looking forward to next season. The blathering talking heads on CNBC can't go 20 seconds without convulsively saying the phrase going forward whenever referring to the future.
Abstract sports concepts like win probability are no exception. We would all call 2-yard run on 4th and 1 a big play, even though it was anything but literally big. How would we characterize a 38-35 game? As a high-scoring game, of course. We are universally comfortable speaking about abstract concepts in terms of the metaphors of physical position, size and motion, and it's a window into how we think. That's why I'll take a graph over a table of numbers any day.
Top Offenses > Top Defenses
Correlations between Team Offense and Defense
In a recent post about team balance, I noted that we should expect a correlation between a team's offensive performance and its defensive performance--at least, not in the same way we would expect to see a team's passing and running performance correlate. Jared commented that although we shouldn't expect a positive correlation, perhaps we should expect a negative one.
A negative correlation between team offense and defense would mean that the better the offense, the worse the defense, and vice-versa. This would make sense in the salary cap era because the more resources a team invests on one side of the ball, the fewer resources it can invest on the other side. If you are throwing $15 million/yr at a star QB, that's $15 million less available to shore up a weak secondary. And that star left tackle drafted in the first round is a top defensive end that's going to some other team.
On the other hand, it could be that good GMs and well-run organizations would simultaneously produce teams composed of both above-average offenses and defenses. Conversely, dysfunctional organizations would produce poor squads on both sides of the ball.
Team Balance
It's no wonder the odds-makers put the Steelers as 3-point favorites over the Ravens, the exact amount of points traditionally given to home teams. They appear to be identical teams in so many ways.
We can see why in the graph below. Like I did recently with running and passing Success Rate (SR) charts, I've plotted each team's total offense and total defense. This time I used opponent-adjusted SR, which accounts for the SRs of each team's opponents.
Unlike passing and running SR, which correlate to a fair degree, there is no reason to expect team offense and team defense to correlate. We shouldn't expect to see a nice tight diagonal relationship like we did when looking at passing and running. But we can still see team balance.
Year of the Run Defense?
Before the playoffs started last year, I did a post on which facets of team strength are most decisive in the playoffs. I looked at passing offense and defense, running offense and defense, turnovers and penalties. For each game, I tabulated how often the team with the better season-long performance in each stat won the game. For example, in the playoffs, the team with the better season-long offensive interception rate won the game 58% of the time.
I also looked at playoff-caliber match-ups in the regular season. Playoff-caliber was defined as teams that would finish with 10 or more wins. I wanted to see if there was something special about "playoff football" beyond the fact that there are (usually) only good teams on the field.
The most intriguing result was the sudden importance of run defense. Teams with the better defensive run efficiency won regular season playoff-caliber games 48% of the time. But in the playoffs, teams with the better run defense won 67% of the time.
This year, the four remaining teams in the playoffs feature the #2, #4, and #5 best run defenses in the league. The team with the better defensive run efficiency has won every single playoff game but one so far this year. That's 7 for 8.
But I'm not sure this means anything. My analysis only used 5 years of data--55 playoff games. This year obviously supports the notion that run defense somehow takes on special importance in the playoffs. But 2007 showed the opposite. Only 2 of 9 games were won by the team with the superior run defense. Two more games were pushes.
Still, that's 64% overall for the 2002 through 2008 seasons. You'd expect a team stronger in any category to win more often, but 65% is the highest of all the core abilities including offensive passing, offensive running, defensive passing, and even turnovers. It's particularly remarkable because run defense seems relatively insignificant in the regular season.
I'm not sure what causes the effect. It could be just random variation and small sample size. But the effect might be real and it could be due to weather or even conservative gameplans. (We could call that the Schottenheimer Effect.)
Single-Point-Failure Model of the Passing Game
Baseball has long been considered the easiest of professional sports to model and analyze mathematically. It’s certainly far simpler than football. One reason baseball is easier to model is that the sport isn’t really a team sport, at least in the most mechanical sense. It’s an orderly series of one-on-one match-ups between pitchers and hitters. Fielding and base-running certainly matter at the margins, but it’s the pitcher-batter interaction that dominates most outcomes.
In contrast, every football play seems like a desperate, chaotic scramble of 22 players. Where baseball is a series system, football is more of a parallel one. In a very simplified way, much of a football play can be modeled as several simultaneous one-on-one match-ups. Take a simple pass play. Each pass blocker matches-up with a pass rusher, a back picks up a blitz or dog, and each receiver matches-up with a pass defender. (At least this would be the case with a man-on man pass defense. Zone defenses can be thought of in a very similar way as I’ll describe below.)
This kind of system is similar to a chain. If any one link fails, the entire system fails. No matter how well the other offensive lineman are blocking, if one lineman misses his block there’s probably going to be a sack. And if one pass defender blows his assignment, either by being beat in man-to-man or being in the wrong place in a zone, there’s a good chance for a big pass completion. This is why a football play can be thought of as a “point-failure” system.
Just like each player has a batting average, each offensive lineman could have a core probability of allowing a pass rusher to beat him and either pressure or sack the quarterback. Likewise, each pass rusher has a core probability of beating a blocker and getting to the QB. These baseline probabilities could be very low, but because it only takes 1 of the 5 pass rushers to be successful on any given play, the resulting chance of a hurry or sack grows considerably.
This is why having a world-class, Hall-of-Fame worthy tackle might not mean that much for a team’s overall pass protection, especially if there are weak blockers elsewhere on the same line. The math works out so that it’s better to have a line full of average blockers rather than a line of one all-pro and four slightly below-average colleagues.
For simplicity’s sake, say each pass rusher has a 5% chance of beating his blocker (within the likely time period before the throw). With 5 pass rushers on a pass play, the chance of any one of them getting to the QB would be 1 – (1-0.05)5), which is 0.23. So in this very simple model, the chance of any 1 of the 5 pass rushers hurrying, hitting, or sacking the QB would be 23%.
The receiver-defender match-ups would work similarly. Say there is a 5% chance a pass defender will either be beaten man-on-man or blow his zone assignment. It only takes one blown assignment for a failure to occur. No matter how well the other members of the secondary are doing, a single failure can lead to a big pass. With four defensive backs in coverage, this would put the overall chance of a wide open receiver at 1-(1-0.05)4) = 0.19.
So, in a very simple way, a passing play is like two chains under strain. One chain is the pass protection, and the other is the pass defense. Each link is a player vs. player match-up, and it has its own probability of breaking based on the abilities of the respective players. The first chain to break loses.
Can you imagine a football team with a starting player who is a point-failure in nearly every play? He'd be a lineman who always gets beat by a pass-rusher or a defensive back who always gets beat by a receiver. It would be ugly. Can you imagine any sport where this could be the case every game? Consider the National League, where pitchers are nearly always an easy out. In baseball, the failure of a pitcher at the plate is confined to his at bat.
So far, I’ve left out the most important player. The quarterback has to see open receivers and throw accurately to make big plays. He has maneuver in the pocket, and scramble from pass rushers. The QB is a big wildcard in my chain analogy.
I imagine this is how football video games like Madden are modeled, at least at the core. The game designers need to know what probabilities of allowing a pass rusher to beat a blocker should be to yield a realistic sack rate. Just looking at sacks alone, we can estimate a ballpark individual “sack allowed” rate is for individual linemen. Overall, the NFL sack rate is about 6.5%, so to solve for the baseline individual rate we can say:
-whole bunch of algebra-
x= 1.1%
Remember that’s an extremely rough figure because there are lots of other factors to consider, such as overload blitzes that linemen can’t handle or don’t control, or quick out passes that allow almost no chance of a sack. Plus we’re only counting sacks, not hits or hurries. So I’m only demonstrating a process, not declaring an answer, or even claiming there is a worthwhile answer. With such a low baseline rate and the NFL’s small sample sizes, it would be difficult in the extreme to grade a lineman purely statistically.
I'm only offering this analysis as a way of thinking about the sport. The only conclusion I’ll draw is a simple one. Ask yourself which is stronger, a chain with 10 links, or a chain of 20 links? It’s the shorter chain. If each link has a certain chance of breaking, you’d want the one with the fewest links.
Offensive passing systems that are heavy on multiple-receiver sets have a mathematical advantage. The more pass defense match-ups and the fewer the pass-rush match-ups an offense can create, the better. An offense would generally want the pass-rush match-ups to be the like the chain with fewer links, and the pass-defense match-ups to be the chain with more links. This way, there is a greater chance of a single point failure in the secondary and a lesser chance of one in the offensive line.
Again, I'm not proposing any sort of statistic to grade individual players. I'm just stepping back and examining why football is sometimes called the ultimate team sport.
Visualizing Team Stats
Click the play button and watch the churn of team records from year to year. Team wins are on the vertical axis and offensive pass efficiency is on the horizontal axis. Notice how strong the linear correlation is between passing and winning. Change the horizontal axis to offensive run efficiency and note how the correlation isn't nearly as strong.
In fact, play around with any of the parameters, including colors, circle sizes, etc. I've set up this motion chart with several "core" stats, including passing and running efficiency, and turnover rates. Click on a team (or several) to highlight it and follow it through the past 6 seasons.
A larger version is here.
Signal vs. Noise in Football Stats
In 2007, the Detroit Lion defense began the first half of the season with 13 interceptions, the most in the NFL. The next best teams had 11. It's reasonable to expect that the Lions would tend to continue to generate high numbers of interceptions through the rest of the season, notwithstanding calamitous injuries.
I wouldn't expect them to necessarily continue to be #1 in the league, but I'd expect them to be near the top. And I'd be wrong. It turns out they only had 4 interceptions in their final 8 games, ranking dead last. So halfway through the season, if I were trying to estimate how good the Lions are in terms of how likely they are to win future games, I might be better off ignoring defensive interceptions.
Although turnovers are critical in explaining the outcomes of NFL games, defensive interceptions are nearly all noise and no signal. Over the past two years, defensive interceptions from the first half to the second half of a season correlate at only 0.08. In comparison, offensive interceptions correlate at 0.27. As important as interceptions are in winning, a prediction model should actually ignore a team's past record of defensive interceptions.
You might say that if defensive interception stats are adjusted for opponents' interceptions thrown, then the correlation would be slightly higher. I'd agree--but that's the point. Interceptions have everything to do with who is throwing, and almost nothing to do with the defense.
This may be important for a couple reasons. First, our estimations of how good a defense is should no longer rest on how many interceptions they generate. Second, interception stats are probably overvalued when rating pass defenders, both free-agents and draft prospects.
I've made this point about interceptions before when I looked at intra-season auto-correlations of various team stats. That's a fancy way of saying how consistent is a stat with itself during the course of a season. The more consistent a stat is, the more likely it is due to a repeatable skill or ability. The less consistent it is, the more likely the stat is due to unique circumstances or merely random luck.
The table below lists various team stats and their self-correlation, i.e. how well they correlate between the first half and second half of a season. The higher the correlation, the more consistent the stat and the more it is a repeatable skill useful for predicting future performance. The lower the correlation, the more it is due to randomness.
| Variable | Correlation |
| D Int Rate | 0.08 |
| D Pass | 0.29 |
| D Run | 0.44 |
| D Sack Rate | 0.24 |
| O 3D Rate | 0.43 |
| O Fumble Rate | 0.48 |
| O Int Rate | 0.27 |
| O Pass | 0.58 |
| O Run | 0.56 |
| O Sack Rate | 0.26 |
| Penalty Rate | 0.58 |
In a related post, I made the case that although 3rd down percentage tended to be consistent during a season (0.43 auto-correlation), other stats such as offensive pass efficiency and sack rate were even more predictive of 3rd down percentage. In other words, first-half-season pass efficiency predicted second-half-season 3rd down percentage better than first-half-season 3rd down percentage itself.
But what about other stats? Are there other examples where another stat is more predictive of of something than that something itself? Below is a table of various team stats from the second half of a season and how well they are predicted by other stats from the first half of a season.
For example, take offensive interception rates (O Int). Offensive sack rates (O Sack) from the first 8 games of a season actually predict offensive interception rates from the following 8 games slightly better than offensive interception rates (0.28 vs. 0.27).
| Predicting | With | Correlation |
| D Fum | D Fum | 0.33 |
| D Fum | D Sack | 0.15 |
| D Fum | D Run | 0.12 |
| D Int | D Sack | 0.08 |
| D Int | D Int | 0.08 |
| D Int | D Pass | 0.01 |
| D Pass | D Pass | 0.28 |
| D Pass | D Sack | 0.26 |
| D Run | D Run | 0.44 |
| D Sack | D Sack | 0.24 |
| D Sack | D Pass | -0.07 |
| O 3D Pct | O Sack | -0.53 |
| O 3D Pct | O 3D Pct | 0.43 |
| O 3D Pct | O Int | -0.42 |
| O 3D Pct | O Pass | 0.42 |
| O 3D Pct | O Run | 0.08 |
| O Fum | O Fum | 0.48 |
| O Fum | O Sack | 0.24 |
| O Int | O Sack | 0.28 |
| O Int | O Int | 0.27 |
| O Int | O Run | 0.06 |
| O Int | O Pass | -0.37 |
| O Pass | O Pass | 0.49 |
| O Pass | O Sack | -0.33 |
| O Pass | O Run | -0.10 |
| O Run | O Run | 0.56 |
| O Run | O Pass | 0.00 |
| O Sack | O Pass | -0.40 |
| O Sack | O Sack | 0.26 |
| O Sack | O Run | 0.03 |
| Pen | Pen | 0.58 |
| Pen | D Pass | -0.23 |
| Pen | O Sack | -0.08 |
There are a thousand observations from this table. I still see new and interesting implications whenever I look it over.
- Having a potent running game does not prevent sacks.
- The pass rush predicts defensive pass efficiency as well as defensive pass efficiency itself.
- Running does not "set up" the pass, and passing does not "set up" the run. They are likely independent abilities.
- Offensive sack rates are much better predicted by offensive passing ability than previous sack rates.
- Defensive sack rate predicts defensive passing efficiency, but defensive passing efficiency does not predict sack rate.
The implications of these auto-correlations are numerous. Team "power" rankings and game predictions (both straight-up and against the spread) rely on a very simple premise--past performance predicts future performance. We now know that's not necessarily true for some aspects of football.
Lions head coach Rod Marinelli might be banging his head against the wall trying to understand how his defense was able to grab 13 interceptions through game 8, but only 4 more for the rest of the season. He's wasting his time. The answer is that in the first half of the season, the Lions played against QBs Josh McCown (2 Ints), Tavaris Jackson (4 Ints), and Brian Griese twice (4, 3 Ints).
Run Defense Dominates in the Playoffs
In the last post I indirectly analyzed playoff games by looking at regular season games that only featured opponents who both went on to win at least 10 games. We saw that teams with the better running games actually won less than 50% of the time.
In this post, I'll look at actual playoff games directly. Compared with the 114 "good vs. good" regular season games I looked at yesterday, there were 50 playoff games, plus 5 Super Bowls, in the 2002 through 2006 seasons.
The table below lists the winning percentage of the team with the superior season-long performance in each stat. In other words, the team with the better [stat] won [x] percent of the time. The regular season good vs. good match-ups are also listed for comparison. (The home win percentage excludes the five Super Bowls during the period.)
| Stat | Good vs Good | Playoffs |
| Home | 59.6 | 64.0 |
| O Run | 45.6 | 45.5 |
| D Run** | 48.2 | 67.3 |
| O Pass* | 52.6 | 63.6 |
| D Pass | 51.8 | 56.3 |
| O Int Rate | 50.9 | 58.1 |
| D Int Rate | 55.3 | 58.1 |
| O Fum Rate** | 55.3 | 40.0 |
| D FFum Rate | 54.4 | 54.5 |
| Pen Rate | 47.3 | 52.7 |
* = good vs. good / playoff difference is significant at the p=0.10 level
The home team won more often in the playoffs than in good vs good regular season match-ups, which is expected because higher seeded teams host the playoff games.
The team with the better run efficiency won only 45.5% of the 55 playoff games during the '02-'06 period. Keep in mind the small sample size could make these results misleading. 45.5% is only 2.5 games below 50%. But the result echoes the same result for the regular season good vs. good match-ups.
Defensive run efficiency is a different story. Although the team with superior run stopping ability won only 48.7% of the regular season good vs. good match-ups, it won 67.3% of playoff games. This is a striking difference to say the least, especially considering how unimportant run defense is based on regression models of regular season games.
The passing game stands out as well. Both offensive and defensive passing stats appear to be more important in the playoffs. Interception rates also appear very important.
Another striking result is that of offensive fumble rates. The team with the lower fumble rate wins only 40% of playoff games. As one of the more random stats, it's not too surprising to see a spurious result for fumble rate, but 40% is fairly low, even for such a small sample size.
I suspect that coaches may become too conservative in the playoffs, relying on the run. This might explain why having a good running game doesn't help teams win and why the ability to stop the run suddenly becomes very important in the playoffs.
My own gut feeling is that coaches don't coach to win. They coach to avoid a loss. It sounds inane, but there is a difference. Maybe the play-calling in the playoffs becomes even more timid. But then again, perhaps the January weather has something to do with it. Previous research has established the importance of climate, especially when dome teams play outdoors. Weather may explain the 67% win percentage of teams with the better run defense. Defense may win championships after all, particularly run defense.
Maybe Defense Does Win Championships...
...but the running game probably doesn't help at all.
I previously thought that the "defense wins championships" theory was conventional wisdom bunk. But after doing a new analysis, I think it might be true. It's the importance of the running game, however, that really surprised me.
In a recent post, I illustrated the distribution of offenses and defenses in terms of total efficiency (yards per play). The distribution for offensive efficiency was wider than for defensive efficiency. This indicated that "good" offenses were better than the equivalent "good" defense. In other words, the best offenses in the league tend to get more yards per play above average than the best defenses in the league give up below average.
"Having a good running game is not only unimportant, it actually seems counter- productive."
As an indirect way to infer tendencies about post-season competition, I analyzed regular season games that featured only opponents that would go on to win at least 10 games. I think this criteria best reflects the level of competition usually found in the playoffs. Although 9-win or even 8-win teams occasionally make the playoffs, many do not. Plus, 9 wins is only 1 win above a .500 win percentage, and a 9-win team has never won a championship.
First, I looked at how important various team stats were in determining the winner of match ups between 10+ win teams. I looked at offensive running and passing efficiencies, turnover efficiencies, penalty rates, and home field advantage. The data is from the 2002-2006 seasons, and there were 114 such games between "good" teams. (The stats used here are year-long stats, not stats only within that particular game.)
Instead of an advanced regression analysis, I started by looking at how often a team with an advantage in each particular stat won. The table below lists various team efficiency stats along with the win percentage of the team superior in that stat. For example, the team with home field advantage won 59.6% of the match-ups between 10+ win teams. And the team with the better offensive pass efficiency won 52.6% of the match-ups. The winning percentage for all regular season games is included for comparison. Significant differences in winning percentages between good vs. good games and all games are noted.
| Stat | Good vs. Good | All Reg Season |
| Home | 59.6 | 57.4 |
| O Run** | 45.6 | 55.0 |
| D Run | 48.2 | 50.0 |
| O Pass** | 52.6 | 63.8 |
| D Pass** | 51.8 | 59.8 |
| O Int Rate** | 50.9 | 59.5 |
| D Int Rate | 55.3 | 59.4 |
| O Fum Rate | 55.3 | 60.8 |
| D FFum Rate | 54.4 | 58.0 |
| Pen Rate* | 47.3 | 54.1 |
* = difference is significant at the p=0.10 level
What immediately strikes me is that being good in the running game, both on offense and defense, appears to be no help in beating other good teams. Teams with the better offensive running efficiency won only 45.6% of the games, and teams with the better defensive running efficiency won only 48.2% of the games.
Teams with superior passing, fumbles, defensive interception rate, and penalties win slightly more than 50%-55% of the games. I'm surprised passing efficiencies don't appear to be more important. The stats that tend to be more random, such as fumbles and interceptions, appear to make the biggest difference. This result may be due to the fact that when good teams play each other stats like passing efficiency and offensive interception rates are very good for both teams, and the difference is in the more random stats.
"When teams very close in ability meet, the more important other factors such as randomness and home field advantage become."
Home field advantage also appears more important than is typical in the NFL. Home teams usually won 57.4% of all regular season games in the period studied. In the good team vs. good team match-ups, home field advantage appears slightly stronger. Again, the closer the teams are in ability, the more important other factors become.
But having a good running game is not only unimportant, it actually seems counter-productive. How can this be? (First, I should note this is not a regression tested for significance, but with 114 observations, and the fact that both offensive and defensive running abilities appear unhelpful, the results are likely somewhat meaningful.) If true, my theory is that winning teams that count on the running game to win might overuse the run against better opponents. Leaning on the running game wouldn't help, and may actually hurt.
Running too frequently would do harm because the pass does have a higher expected return per attempt (link requires registration), even accounting for the possibility of an interception. Every run attempt precludes a pass attempt, reducing ultimate effectiveness.
To get a better context of the results in the table above, I also calculated the winning percentage of teams with superior stats for other types of match-ups. I analyzed "bad vs. bad" match-ups which featured both opponents that ultimately earned 9 or less regular season wins. Also analyzed were "good vs. bad" match-ups which featured a 10+ win team against a 9- win team. (I realize 9 wins is not "bad," but it's a lot shorter than "other than good.")
The winning percentage of teams with the better stat are listed for each type of match-up in the table below.
| Stat | Good vs Good | Bad vs Bad | Good vs Bad |
| Home | 59.6 | 57.0 | 56.0 |
| O Run | 45.6 | 53.7 | 55.8 |
| D Run | 48.2 | 51.4 | 67.9 |
| O Pass | 52.6 | 63.6 | 65.4 |
| D Pass | 51.8 | 59.8 | 63.7 |
| O Int Rate | 50.9 | 59.5 | 53.8 |
| D Int Rate | 55.3 | 59.4 | 57.3 |
| O Fum Rate | 55.3 | 60.5 | 63.7 |
| D FFum Rate | 54.4 | 58.0 | 62.3 |
| Pen Rate | 47.3 | 53.8 | 71.9 |
The results for the other types of match-ups seem to make sense. Being superior in any of the stats does not appear to be unhelpful (as with running in the good vs. good match-up). The bigger the difference in team record, the larger we would expect the difference in each team stat. Accordingly, the winning percentages are higher for the bad vs. bad and good vs bad match-up types than the good vs. good match-up type.
I could go on and on with observations. Penalty rates appear critical in good vs bad match-ups, offensive passing efficiency appears most important in the bad vs. bad match-ups, etc. I'll leave it to others to draw their own inferences.
Ultimately, when teams very close in ability meet, the more important other factors such as randomness and home field advantage become. Playoff teams are by definition relatively similar in ability, so home field and randomness become critically important. Turnovers are the most random of the stats, especially defensive turnover efficiency. Perhaps then it is randomness that wins championships. And because defensive performance trends are more random than offensive trends, perhaps that's why we see defense as more important come January.
Does Defense Win Championships?
Well, of course. And so does offense. But the conventional wisdom that "defense wins championships" implies that defense is particularly more important than offense in the playoffs and the Super Bowl. This post will begin to look at whether defense really does matter more than offense in the NFL by comparing the right tails of the performance distributions of offenses and defenses--where playoff teams come from.
This time of year we are helped to the standard slew of articles declaring that defense is more important. Here is the latest example from ESPN.com. It's a good example because it suffers from some fatal flaws (itemized by Phil Birnbaum here). Typically, these articles look at past examples of NFL champions and comparing the offensive and defensive rankings of each team. I don't think this kind of analysis is necessarily very valid--
- They are anecdotal
- The sample size is usually very small, and results are probably not statistically significant--just due to chance
- If the sample size is large, it covers very distinct periods of NFL passing and blocking rules, confounding any results
- If the sample size is limited to one period of NFL rules, it can be dominated by one or two particular teams would skew the results (PIT in the 70s or NE currently, for example)
- Often, the analysis shows that defense is indeed important, but not more important than offense
- The rankings of each squad is almost always based on points scored or total yards, which are more often than not deceiving about the true performance of a squad
There is also an assumption that playoff football is somehow systematically different than regular season football. I'm not comfortable with that assumption--the rules are the same, the refs are the same, the field is still 100 yds long, and a touchdown is still 7 points. There are some unique things about the playoffs, but for now I'm going to set them aside and begin to look at how offenses and defenses compare in general by using regular season stats. (Using post-season stats is possible, but very problematic. About half of all playoff teams only play a single game before being eliminated, yielding very erratic and extreme team averages).
To best compare offensive and defensive performance and ability, squad efficiency--yards per play--should be used. Offensive and defensive efficiency are independent of each other. They aren't dependent on special teams or field position considerations. And perhaps most importantly, they aren't confused by the direction of causation. For example, total rushing yards correlates strongly with winning, but it's actually the winning that allows teams to inflate their rushing yards by eating the clock at the end of a winning game.
Efficiency stats are usually divided into passing and running efficiencies here at NFL Stats. But in this article we're interested in offenses and defenses as a package and not pass/run balance, so we'll use offensive efficiency (yards gained per offensive play) and defensive efficiency (yards allowed per offensive play). Data is from the 2002-2006 NFL seasons (N=160).
The graph below illustrates the actual distribution of team offensive and defensive efficiencies. The vertical axis represents the number of squads with the noted efficiency level (horizontal axis).
With enough observations, we would expect the distributions to smooth out, resembling more of a typical bell-shaped normal distribution. With the given averages and standard deviations, the theoretical efficiencies are approximated in the figure below.
So if a great offense usually trumps a great defense, where does the perception that "defense wins championships" come from? Truly dominant defenses such as the 2000 Ravens, 2002 Buccaneers, or 1985 Bears are relatively rare, and are therefore more memorable. Also, defense has traditionally been overlooked, at least by the mainstream hype-laden media. Even football insiders seem to focus on offense, demonstrated by who is inducted in the Hall of Fame, or who the MVPs tend to be. So the phrase "defense wins championships" may really mean "defense helps win championships more than most people think they do."
One thing that this analysis does not do is focus on the specific case of great offensive vs. great defense. It considers team efficiencies as a whole. While this analysis indicates the likely outcomes of strong offenses vs. strong defenses, it is an indirect inference. A case by case study could look at playoff-type match-ups of good teams only, and tell us more.
Admittedly, there are some special qualities about the playoffs. The outdoor weather in northern cities can be extreme, and the home team is more often the better team. Weather may indeed affect the balance of offense and defense, but it likely affects the balance between running and passing games more. And weather affects both opponents in a game, so it's not clear if it really matters. Playoff weather could also be analyzed in further research.
So when looking at the NFL as a whole, offense and defense balances symmetrically. But when focusing on the right tails of performance, where playoff teams come from, we see that great offenses out-pace equally great defenses.