Before the playoffs started last year, I did a post on which facets of team strength are most decisive in the playoffs. I looked at passing offense and defense, running offense and defense, turnovers and penalties. For each game, I tabulated how often the team with the better season-long performance in each stat won the game. For example, in the playoffs, the team with the better season-long offensive interception rate won the game 58% of the time.
I also looked at playoff-caliber match-ups in the regular season. Playoff-caliber was defined as teams that would finish with 10 or more wins. I wanted to see if there was something special about "playoff football" beyond the fact that there are (usually) only good teams on the field.
The most intriguing result was the sudden importance of run defense. Teams with the better defensive run efficiency won regular season playoff-caliber games 48% of the time. But in the playoffs, teams with the better run defense won 67% of the time.
This year, the four remaining teams in the playoffs feature the #2, #4, and #5 best run defenses in the league. The team with the better defensive run efficiency has won every single playoff game but one so far this year. That's 7 for 8.
But I'm not sure this means anything. My analysis only used 5 years of data--55 playoff games. This year obviously supports the notion that run defense somehow takes on special importance in the playoffs. But 2007 showed the opposite. Only 2 of 9 games were won by the team with the superior run defense. Two more games were pushes.
Still, that's 64% overall for the 2002 through 2008 seasons. You'd expect a team stronger in any category to win more often, but 65% is the highest of all the core abilities including offensive passing, offensive running, defensive passing, and even turnovers. It's particularly remarkable because run defense seems relatively insignificant in the regular season.
I'm not sure what causes the effect. It could be just random variation and small sample size. But the effect might be real and it could be due to weather or even conservative gameplans. (We could call that the Schottenheimer Effect.)
Year of the Run Defense?
By
Brian Burke
published on 1/12/2009
in
offense vs defense,
playoffs,
research
Subscribe to:
Post Comments (Atom)
Your analysis highlights what is so maddening about the statistical analysis of football - especially the analysis of PLAYOFF football. The sample sizes are so small that you must constantly second guess your own conclusions. If you continually expand your data set by going further back in time, you then run the risk of invalidating your own conclusions because, after a certain period of time, the very nature of the game itself evolves and the data is no longer applicable.
This problem has been outlined for me when doing regression analysis on NBA games. For example, there are 1,230 games in the NBA regular season so you might think that ANY statistical trend that is consistent throughout the length of an entire NBA season is in fact a valid trend, right? Wrong. I have found certain algorithms that yield stunning accuracy for the length of an entire NBA season, or for two consecutive seasons, but then crash entirely in subsequent seasons.
And if seemingly-valid trends materialize and then evaporate over the course of thousands of NBA games, then it must be much more difficult to accurately identify trends that materialize over the course of merely dozens of NFL playoff games.
How can this year be 100% playoff win percentage when Minn has the best DRUN rate?
It's possible that because the average playoff game takes place in worse weather, the ability to stop the run has a greater impact. Aside from that, I'd guess it's statistical noise.
Part of the problem when analyzing statistics over a long period is changes in the way statistics are measured.
This doesn't have as big of an effect in football but in basketball, who gets credited with assists, steals, and some rebounds are depending on the scoring table and changes in how they are perceived.
The other major thing I'd point out is the *absolute* strength of a team's run defense. Some years the best run defense in the league might be (if it were transported through time) only the fifth or sixth best in the league now (or vice versa). So maybe what matters is that your run defense is "good enough" rather than simply "top ten". The first term is an measure of absolute skill, the second a relative term.
@bytebodger
I had the similar problem with my NBA model. I trained my model on the last 5 years of data. Then for the current year, I split the set into a validation set and a prediction set. In my model I remove any games between teams that have not played 15 games. The validation set is all the games between teams that played at least 15 games but less than 30 games. So with the validation set I can see if my model will perform well or not. It is not 100% sure but at least it gives you an idea of your model future performance.
Here are a few other questions: during the season, are you using a teams run defense rank from the end of the season (after looking at their whole body of work) or updating the rank after each game as we get more data? It might clear up the information a bit if you use end-of-season rankings as those are more accurate.
Also, consider that players playing the playoffs are trying harder than the regular season. Think about it, intensity and physicality go up in the playoffs, the refs are less inclined to alter a game with borderline penalties, players are typically more experienced, more veteran, better able to get away with physical play.
Coaches don't want to "get fancy" and risk losing their job because of poor play calling. Remember: conservatism is rewarded in the NFL by owners and fans alike. Its much easier to blame the offensive line if you get stuffed on 4th and 2 than to blame them if you try a halfback pass on 4th and 2 and the fan base calls for you to be fired because of a single bad outcome in a special play.
Thank you, Anonymous. I missed one. MIN lost to PHI but had the better def run eff. I've corrected the article to say 7 out of 8.
Odysseus your dead bang on your comment maybe what matters is that your run defense is "good enough" rather than simply "top ten". As you advanced towards the superbowl, balance is critical so you also have to have a good offense to take the pressure off even the best of defenses.
Odysseus-Good question. I used end of year rankings, not 'to-date' stats to compare teams.
I hear people talk about the intensity of the playoffs a lot, and the stakes are certainly very high. But I wonder how that really changes the game. Do players really hit harder, etc.? Is that even possible? Players might claim that, but are those just words given the high stakes? Is there objective evidence of increased physicality or changes in the refs' dispositions?
I agree conservative play is a plausible theory for what we observe here in comparisons of playoff winners. But I don't have any direct evidence.
bytedodger-Small sample size is always a problem with playoffs. That's the reason I looked at 'playoff-caliber' regular season games. But it turns out there are differences between those classes of games too. I've got a couple more years of data (22 more playoff games) since the original study, so when I have time I'll compile the data and see how the effects hold up.
how about comparing the stat (run D) during the regular season restricted to playoff teams playing other playoff teams.
It may be that run D is more important playing against a good opponent (or a good oponent with a good run game), whereas it is not important when you are playing a lousy team.
-bob
If the rate of injury is stat. significantly higher during the playoffs than typical late-season games that would offer evidence of increased "intensity"...i.e. players hitting harder, yada, yada. Restrict it to warm weather and dome games to eliminate the snow (haha now lets talk about small sample)...............Can't wait for the Boss half-time show.
I wonder if officiating might play into this.
It is my feeling (with absolutely no evidence) that in the playoffs the refs are more likely to Let'em play.
This means fewer holding calls (advantage for both run and pass, not sure wich is greater) and less pass interference calls clearly a disadvantage for pass offense. So if you are a good run defense already your pass defense might get better automatically in the playoffs due to officiating. Just thinking off the top of my head.
Also, and you may have allready done this, it would be interesting to see how playoff game look statistically compared to non playoff games.
Number of plays, number of runs vs pass, average yards, scoring etc. That might give you a feel for how playoff games develop compared to regular season games
I can tell you with assuredness that the speed of the game DOES go up a notch, simply because there are absolutely no loafs by a player. During the regular season every player is graded for various aspects of their position. One of the details that is graded on every player is a "loaf", and it means just that. They didn't go "all out" so to speak. During the playoffs, I can assure you that there are NO "loafs" on anyones sheet the next day.
Couple that with the fact that teams tend to run the ball more in the playoffs due to weather and wind factors, not to mention that a team absolutely has to be two dimensional in their offense to be effective in the playoffs. It is why the AZ team is all of a sudden successful in the playoffs. They have run the ball more to offset what was basically a pass only offense, and they have definitely been able to stop the run better than they did during the regular season.
I would suggest that while a Good rush D is essential ingredient to contributing to a playoff win, you really will find that the real contributing factor to winning in the playoffs is not turning the ball over.
Officiating is also a factor and they do tend to let them play a little by not calling a lot of holding penalties, but as far as letting them go in the passing game.....ain't happening. They will call pass interference and defensive holding on pass plays.
I believe we can find the same statistic that you found on Rush D 7/8 winners is exactly the same # as it pertains to turnovers.
Interesting point about the "loafs."
Yes, of course turnovers are crucial in any particular game, playoffs or not. That's obvious in retrospect. I'm trying to figure out what stats are predictive.
I'd wager that teams run the ball quite a lot more in the playoffs, partly due to weather, and partly due to job security.
I doubt any coach wants to have his team employ a high variance strategy - and fail - in a huge game. We already see this kind of effect with the whole 4th down issue, but it's magnified in the playoffs. So in turn you have more emphasis on "old school conservative football", which means run offense and run defense are of incredibly high importance.
Oh, never mind, I noticed someone said exactly the same thing above.