Maybe Defense Does Win Championships...

...but the running game probably doesn't help at all.

I previously thought that the "defense wins championships" theory was conventional wisdom bunk. But after doing a new analysis, I think it might be true. It's the importance of the running game, however, that really surprised me.

In a recent post, I illustrated the distribution of offenses and defenses in terms of total efficiency (yards per play). The distribution for offensive efficiency was wider than for defensive efficiency. This indicated that "good" offenses were better than the equivalent "good" defense. In other words, the best offenses in the league tend to get more yards per play above average than the best defenses in the league give up below average.

"Having a good running game is not only unimportant, it actually seems counter- productive."

However, that analysis was for regular season games. Post season games comprise a smaller sample size, usually too small for very meaningful analysis. They're also biased in certain ways. For example, the home team is usually the better team, and it would therefore be hard to separate the advantage in team strength from home field advantage. But the biggest difference between the regular and post seasons is the level of competition.

As an indirect way to infer tendencies about post-season competition, I analyzed regular season games that featured only opponents that would go on to win at least 10 games. I think this criteria best reflects the level of competition usually found in the playoffs. Although 9-win or even 8-win teams occasionally make the playoffs, many do not. Plus, 9 wins is only 1 win above a .500 win percentage, and a 9-win team has never won a championship.

First, I looked at how important various team stats were in determining the winner of match ups between 10+ win teams. I looked at offensive running and passing efficiencies, turnover efficiencies, penalty rates, and home field advantage. The data is from the 2002-2006 seasons, and there were 114 such games between "good" teams. (The stats used here are year-long stats, not stats only within that particular game.)

Instead of an advanced regression analysis, I started by looking at how often a team with an advantage in each particular stat won. The table below lists various team efficiency stats along with the win percentage of the team superior in that stat. For example, the team with home field advantage won 59.6% of the match-ups between 10+ win teams. And the team with the better offensive pass efficiency won 52.6% of the match-ups. The winning percentage for all regular season games is included for comparison. Significant differences in winning percentages between good vs. good games and all games are noted.
















StatGood vs. Good
All Reg Season
Home59.657.4
O Run**45.655.0
D Run48.250.0
O Pass**52.663.8
D Pass**51.859.8
O Int Rate**50.959.5
D Int Rate55.359.4
O Fum Rate55.360.8
D FFum Rate54.458.0
Pen Rate*47.354.1

** = difference is significant at the p=0.05 level
* = difference is significant at the p=0.10 level

What immediately strikes me is that being good in the running game, both on offense and defense, appears to be no help in beating other good teams. Teams with the better offensive running efficiency won only 45.6% of the games, and teams with the better defensive running efficiency won only 48.2% of the games.

Teams with superior passing, fumbles, defensive interception rate, and penalties win slightly more than 50%-55% of the games. I'm surprised passing efficiencies don't appear to be more important. The stats that tend to be more random, such as fumbles and interceptions, appear to make the biggest difference. This result may be due to the fact that when good teams play each other stats like passing efficiency and offensive interception rates are very good for both teams, and the difference is in the more random stats.

"When teams very close in ability meet, the more important other factors such as randomness and home field advantage become."



Home field advantage also appears more important than is typical in the NFL. Home teams usually won 57.4% of all regular season games in the period studied. In the good team vs. good team match-ups, home field advantage appears slightly stronger. Again, the closer the teams are in ability, the more important other factors become.

But having a good running game is not only unimportant, it actually seems counter-productive. How can this be? (First, I should note this is not a regression tested for significance, but with 114 observations, and the fact that both offensive and defensive running abilities appear unhelpful, the results are likely somewhat meaningful.) If true, my theory is that winning teams that count on the running game to win might overuse the run against better opponents. Leaning on the running game wouldn't help, and may actually hurt.

Running too frequently would do harm because the pass does have a higher expected return per attempt (link requires registration), even accounting for the possibility of an interception. Every run attempt precludes a pass attempt, reducing ultimate effectiveness.

To get a better context of the results in the table above, I also calculated the winning percentage of teams with superior stats for other types of match-ups. I analyzed "bad vs. bad" match-ups which featured both opponents that ultimately earned 9 or less regular season wins. Also analyzed were "good vs. bad" match-ups which featured a 10+ win team against a 9- win team. (I realize 9 wins is not "bad," but it's a lot shorter than "other than good.")

The winning percentage of teams with the better stat are listed for each type of match-up in the table below.















StatGood vs GoodBad vs BadGood vs Bad
Home59.657.056.0
O Run45.653.755.8
D Run48.251.467.9
O Pass52.663.665.4
D Pass51.859.863.7
O Int Rate50.959.553.8
D Int Rate55.359.457.3
O Fum Rate55.360.563.7
D FFum Rate54.458.062.3
Pen Rate47.353.871.9

The results for the other types of match-ups seem to make sense. Being superior in any of the stats does not appear to be unhelpful (as with running in the good vs. good match-up). The bigger the difference in team record, the larger we would expect the difference in each team stat. Accordingly, the winning percentages are higher for the bad vs. bad and good vs bad match-up types than the good vs. good match-up type.

I could go on and on with observations. Penalty rates appear critical in good vs bad match-ups, offensive passing efficiency appears most important in the bad vs. bad match-ups, etc. I'll leave it to others to draw their own inferences.

Ultimately, when teams very close in ability meet, the more important other factors such as randomness and home field advantage become. Playoff teams are by definition relatively similar in ability, so home field and randomness become critically important. Turnovers are the most random of the stats, especially defensive turnover efficiency. Perhaps then it is randomness that wins championships. And because defensive performance trends are more random than offensive trends, perhaps that's why we see defense as more important come January.

  • Spread The Love
  • Digg This Post
  • Tweet This Post
  • Stumble This Post
  • Submit This Post To Delicious
  • Submit This Post To Reddit
  • Submit This Post To Mixx

8 Responses to “Maybe Defense Does Win Championships...”

  1. JG says:

    Hi. While I certainly agree that in the NFL teams win with passing, measuring "running efficiency" by average yards per carry (I believe that's what you use?)seems pretty misleading.

    E.g. Lombardi's '66 Packers, NFL champs (12-2) and SB champs, were an historic run-first team (fewest pass attempts in the league) that averaged only 3.5 yards a carry, 13th in a 15-team league.

    I don't think anybody ever thought of Lombardi's Packers as being a "very inefficient" running team. But if you rate efficiency as Y/C and consider the likes of them, then yes, having a more efficient running game will contribute to losing.

    Running efficiency might be better measured by median yards per carry, or even by the mode. I.e., by how reliably it accomplishes an objective at which running is more effective than passing, like converting short-yardage first downs.

    Then a steady 3 yds (1st down) 4 yds (1st down), 3 yds (1st down), for a Packer-like 3.5 average, is a lot more efficient than -1 (punt), no gain (punt), +16, for a 5.0 average.

    (And it's *a lot* more efficient than running for -1, 0, -2, 0, -1... until you are 20 points down in junk time, when facing a prevent defense you start running draw plays for 12, 15, 20 yards to inflate your average. Which was how my Jetsies ran this year. ;-( )

    A theoretical "3.5 yards on each and every carry run offense" will be an unstoppable scoring machine that never punts once. While a "5.0 average carry offense" that gets that 5.0 by going the likes of 0, -1, 0, +1, +25, will be punting frequently, rarely go the length of the field, and get clobbered by the former in a rushing contest.

    It's hard to believe that really being better at any major part of the game will systematically cause teams *to lose*. It's a lot easier to beleive that if the measure used for "better" is off, one will get that apparent result -- and make the value of defending against that part of the game look misleadingly inconsequential as well.

    Again, I'm not denying that passing is more important. Lombardi's Packers were famously efficient when they passed -- heck, even Shula's 17-0 Fins with Old Earl Morrall at QB, the NFL team that came closest to running like Nebraska, passed with tremendous efficiency.

    I'm just saying, well ... you know what I'm saying. I already beat it to death.

    Thanks for the blog. Lots of fresh insights here. I'm glad I found it, and will be coming back as a regular.

  2. Brian Burke says:

    Right, a team that gets 3.5 yds per carry every attempt would be below average but literally unstoppable.

    I would tend to agree with you. There is probably a better measure of a team's central tendency of running ability than pure average yards per carry. I've suggested median yds per carry, but there are probably other ideas too. FO's "line yards" seems like a pretty good start.

    http://www.bbnflstats.com/2007/08/median-rushing-yards.html

    However, there are a couple reasons that suggest avg yd per carry is adequate.

    First, although a couple 70 yd TD runs skews the average, they're still a good thing to have. Average may not be optimum, but more is better and less should be worse.

    Which it is...in all cases except when good teams (playoff teams or 10+win teams) face off. Rushing average positively correlates with winning except when good teams play each other.

    I think that's pretty interesting, but I only have a few guesses why that would be.

  3. JG says:

    " .... there are a couple reasons that suggest avg yd per carry is adequate. First, although a couple 70 yd TD runs skews the average, they're still a good thing to have. Average may not be optimum, but more is better and less should be worse."

    That's certainly true if you are trying to win a game, but I don't think this really implies that average yards per carry (AYC)correlates well with real rushing efficiency overall.

    A couple more thoughts about this -- not arguing, just continuing a conversation.

    First, AYC seems loaded with noise. Look at the annual numbers for some All-Pro rushers year-by-year. Tomlinson: 3.6, 4.5, 5.3, 3.9, 4.3, 5.2... Curtis Martin: 3.6, 4.2, 3.5, 4.5, 3.8 4.6., etc.

    Those are some pretty big swings for LT from "below average efficiency" to "highly efficient" (3.x to 5.x) by the same guy on the same team in adjacent years. Appears like there's a major random component to me. Almost like a random walk.

    And that's easy for me to understand: 70-yard runs are rare, and rare events occur with a high degree of randomness in a moderate sample size, yet they have a big effect on AYC. Just a few more or less from year-to-year can be over 1 yd/carry.

    And compounding things, just the chance to have them is random. The line opens a gaping hole and LT has a clear shot to go the field, a rare event to start with -- but is he 10 yards away from the goal or 90?

    So AYC seems not only misleading in principle (high AYC with a low median is *bad*) it's also very polluted with noise.

    Relatedly, there seems good evidence to me that a good running game *does* improve the passing game, and thus helps winning by passing.

    To begin with we have all the coaches' testimony and persuasive anecdotes from Earl Morrall being picked up off waivers for $100 to QB a 17-0 season throwing for 9.1 yards per attempt (Brady this year was 8.3) behind 200 yards of rushing a game, to Roethlisberger getting to the SB as a rookie with the league's heaviest rushing team.

    But we also do have game-by-game data. Elias tracks "play action" passing rating, and it can be a lot higher that overall rating. E.g: this year Brady, 147 v 117; Garcia, 132 v 95; Schaub 117 v 87; etc. When the running game isn't respected there's no play action, so that seems pretty good evidence of running helping passing, and thus helping winning.

    If the effect of running efficiency on producing the likes Earl's 9.1 and those play action ratings can't be readily detected, I'd suspect there's a problem with the measure of it being used.

    Not that I have a better one available for ready use, because I certainly don't. It's just that football can be a very complex game to measure. So I'd be slow to assume that absence of evidence in a regression or correlation is evidence of absence.

    "Rushing average positively correlates with winning except when good teams play each other. I think that's pretty interesting, but I only have a few guesses why that would be."

    My first guess would be that the average difference in performance in games between two good teams is a good deal smaller than generally, and that if AYC "undermeasures" this performance generally (as I think it does for the reasons given) then the measured effect of it when the difference is small could disappear entirely.

    One opinion, FWIW. I shall say no more on this subject ever again.

    Having a place to explore such issues is great. Thanks.

  4. Brian Burke says:

    JG-Great stuff. I generally agree.

    My own theory is that running now in the NFL needs only to be good enough to do what you suggest--enable the play-action and keep defenses guessing.

    With today's passing rules, maybe a team wouldn't have to be that great at running the ball, just good enough to keep the defense off balance and occasionally get the first down on 3rd and short. It functions like a jab in boxing.

    A couple counter-points:

    Yes, there are some big inter-year variances in all-pro runners like LT and Curtin Martin. But we'd expect larger inter-year variance than intra-year variance, which tells us more about a team's ability.

    In another recent post, the self-correlation of team running efficiency (AYC) is relatively high compared to other stats. It would probably be even higher if adjusted for opponent strength.

    I would love to have good median yds per carry data. Unfortunately, it requires a large play-by-play database which is hard to come by.

    Great comments. By the way, where did you get the Elias play-action data?

  5. Brian Burke says:

    JG - You might enjoy this paper about run/pass balance.

    http://www.bepress.com/cgi/viewcontent.cgi?article=1051&context=jqas

  6. Anonymous says:

    Perhaps good running efficiency does not help a team win because the team with the lead runs more often to use up the clock, and reduce risk of turnovers, they are less concerned with yards gained. The team that is behind knows that running plays are more likely and gears its defense to stop the run, thereby reducing the opponents running efficiency.

    It’s the same reason you hear announcers say “when running back X gets Y carries, his team wins.” The running back gets the carries BECAUSE his team is winning. Same goes for running efficiency, it drops down to a lower value BECAUSE the team is winning.

    To test this you could look at the same games between “Good” teams. Then keep only the games that were close throughout (so it is likely that neither team went into “run out the clock” mode). You’d have to define what “close throughout” means- how about games that neither team held more than a 6 point advantage at any time. Then repeat your analysis.

    I believe the above method would give a more significant value to the importance of running efficiency.

  7. Tomb says:

    Nate Silver (FiveThirtyEight.com, Baseball Prospectus) made a similar observation about baseball. Certain offensive statistics seem to be most important in getting teams to the playoffs, whereas certain defensive statistics seem to be most important in winning in the playoffs. Unlike in football, there's no clock to bias any particular offensive statistic against the winning team. Nor do batters risk injury as footballers do. Granted, a team that has the lead might be more satisfied with a sacrifice fly relative to a home run, for example, but might there be another explanation? My theory is that defense is more important in the playoffs BECAUSE offense is more important generally. Defense is less important because it varies less. Because offense is more important, teams with poor offenses tend more to be eliminated from playoff contention than teams with poor defenses. This results in a playoff offensive performance spectrum that's much narrower, perhaps narrower than the defensive or at least narrow enough to be more mistaken to be of greater importance.

  8. Anonymous says:

    So does defense then win championships?

    With the predominance of run-defense in the post-season, does that answer the question?

    Also, what does the fact that the best defenses (and best defensive coaches) seem to pair themselves with run oriented offensive approaches?

    I can't think of a great run defense being paired with a great passing offense that plays up-tempo in order to lengthen the game and maximize possesions in recent memory.

    True, if an opponent were to face a lightning passing attack and a great run defense the opponent probably would pass themselves in order to keep up and to avoid the run defense. But then...

    Logically, there seems to be a "correct" approach out there. But it seems to get very muddled.

    Also, is there a corellation between teams with a great run defense also having a great pass defense?

    Perhaps the answer is in mentality. Does a team need to believe in its ability to be more physical than its opponent in order to win, even if being more physical doesn't really lead to victories (physical defined as great run defense and offensive run ability)?

Leave a Reply

Note: Only a member of this blog may post a comment.