In earlier posts I claimed that special teams could be neglected in a prediction model of wins because big special teams plays tended to be chaotic--largely random and non-repeatable. I think I may be wrong.
I can defintely say now that special teams (ST) retrospectively explain a good deal of variance in team records. But I'm not so sure it can help predict future ST performance, and therefore future wins.
The biggest challenge in analyzing ST stats is that they are difficult to measure. For example, consider a team that is frequently punting from midfield or the opponent's 40 yard line. They would likely have poor net punt yardage compared to teams that punt from their own territory more often. All of football is dependent on the situation, but special teams are in particular.
The website Football Outsiders has a potential solution. They grade each play according to the situation and compare each team's performance against the league average in the same situation. They call this measure Value Over Average (VOA). They go a step further and factor in a correction for opponent strength which results in Defense-adjusted VOA (DVOA). To be honest, I'm not sold on D/VOA as the best measure of team performance, but it does take situation into consideration on a play-by-play basis, which is espeically handy for special teams.
I was able to gather the D/VOA stats for special teams from the 2003-2006 regular seasons (n=128). I then added it into the team efficiency model which estimates team wins based on each team's efficiency stats of running, passing, turnovers, and penalties. I normalized each variable in the model, so that their regression coefficients could be directly compared to one another.
The baseline efficiency model, without ST data, results in an r-squared of 0.73. Including either ST DVOA or ST VOA in the model increases the r-squared to 0.77. The regression results are shown in the table below.
Variable | Std.Coeff. | P-Value |
O PASS | 1.27 | 0.00 |
D PASS | -0.80 | 0.00 |
O RUN | 0.46 | 0.00 |
D RUN | -0.44 | 0.00 |
O INT | -0.53 | 0.00 |
D INT | 0.57 | 0.00 |
O FUM | -0.40 | 0.02 |
D FFUM | 0.44 | 0.01 |
PEN | -0.39 | 0.01 |
ST DVOA | 0.68 | 0.00 |
r-squared | 0.77 |
The results above can be interpreted as follows. Each coefficient indicates how many additional regular season wins a team can expect, on average, per standard deviation above average. I was surprised by how strong a variable ST DVOA turned out to be. It's standarized coefficient is 0.68, which is third only to offensive and defensive passing efficiency. If true, that would mean that the best ST squad in the league (about 2 standard deviations above the mean) is worth about 1.4 additional wins, on average.
I thought that a lot of that strength may be due to field goal kicking, which has the most direct impact on the score. So I reran the regression with each component of special teams broken out: FG/XP kicking, kick offs, kick returns, punts, and punt returns.
Variable | Std.Coeff. | P-Value |
O PASS | 1.26 | 0.00 |
D PASS | -0.81 | 0.00 |
O RUN | 0.47 | 0.00 |
D RUN | -0.47 | 0.00 |
O INT | -0.63 | 0.00 |
D INT | 0.60 | 0.00 |
O FUM | -0.33 | 0.07 |
D FFUM | 0.36 | 0.02 |
PEN | -0.41 | 0.01 |
FG/XP | 0.34 | 0.00 |
KICK | 0.44 | 0.00 |
K RET | 0.07 | 0.64 |
PUNT | 0.23 | 0.12 |
P RET | 0.27 | 0.10 |
r-squared | 0.79 |
The standardized coefficients show that kick-off and kick coverage performance (KICK) is the most important (in winning) of all the components of ST. But kick return (KRET) is the only non-significant variable, which is puzzling. Why would kick-offs be so important if kick returns don't matter? This might indicate a shortcoming of the DVOA system.
Punt and punt return performance are marginally significant, but the signs of the coefficients make sense and they are relatively symmetric. So we can be confident they are relevant but their true coefficients may not precisely as indicated by this data set.
To simplify the results, I computed the relative strength of each of the standardized coefficients in terms of percent of the total strength of all variables.
Variable | Importance |
O PASS | 19% |
D PASS | 12% |
O RUN | 7% |
D RUN | 7% |
O INT | 9% |
D INT | 9% |
O FUM | 5% |
D FFUM | 5% |
PEN | 6% |
FG/XP | 5% |
KICK | 7% |
K RET | 1% |
PUNT | 3% |
P RET | 4% |
So according to this analysis, special teams accounts for about 20% of the game in terms of winning and losing. In a way, this is disproportionately strong. ST plays comprise far fewer than 1 in every 5 plays on the field. GMs may want to take another look at how much they're paying their punters, kickers, and returners. Perhaps the league is noticing the importance of ST evidenced by the recent attention return specialists have received in the draft.
ST is a considerable part of the game. It retrospectively helps explain why teams won. The question remains, however, if prior ST performance indicates future ST performance, and if it is predictive of future wins.
I think one of the things football outsiders proved was that other than kickoff distance not much was repeatable with special teams.
I'd probably agree with that assessment, but I haven't run the numbers yet.
In general, I think Football Outsiders is great for data collection, but their analysis is relatively poor. They tend to run a couple of correlations and stop there.
Brudog,
No, that was not really what they proved. What they've showed is that, roughly, the following things are predictive from year-to-year (that is, teams that are good or bad tend to remain good or bad):
- kickoff distance (or more precisely, this correlates well with individual kickers)
- coverage and return teams (average performance)
While the following things were not really predictive of anything:
- missed and made field goals
- blocked FG/extra points
- ST touchdowns
BB,
As you say, there's a big difference between whether something is important, and whether it is predictive. Obviously, making a really high percentage of field goals one year should improve your win total. But that doesn't necessarily mean that that same kicker should be expected to make a great percentage of FGs the next year.
Why are you not sold on D/VOA as the best measure of team performance?
DVOA is circular. It basically says a team is successful because it was successful. It's also somewhat subjective, incredibly complex, and requires armies of data collectors every Sunday.
Don't get me wrong. I don't claim to have the "best" measure (unlike FO). In fact, I don't believe there is any one best stat. And I do think DVOA has it's place, especially in special teams evaluation. FO does a wonderful job at data collection.
But personally, their analysis is very rudimentary, mathematically shallow, and in many cases, plain wrong. FO seems like a kid who just found the CORREL() function in Excel. Or better, they're like a computer with a 200 GB harddrive but a 1 kHz processor from a 1982 IBM PC.
They could really use someone with a real applied background in statistical analysis. One thing is certain--they have accumulated the smartest community of football fans anywhere.
I couldn't agree more, particularly with your last two points: "They could really use someone with a real applied background in statistical analysis. One thing is certain--they have accumulated the smartest community of football fans anywhere"
As far as DVOA, it's really hard to refute what is essentially a black box process. There is no peaking under the hood to look at coefficients, etc. Personally, I think they've data mined the crap out of their projection model and it will get them in trouble. I'm sure they never constructed in sample and out of sample periods or followed standard statistical techniques. Schatz certainly is a very savvy marketer and self-promoter. I will say that the guys at FO are knowledge, offers unique insights, and are great writers - they are talented sport journalists, but statisticians they are not. That said, it's the second site I hit the morning after PFT.
Special Teams do come to close to 1 of 5 plays. All together they come to about 18.5%.
in the Pro football Historical Abtract, the author translates many special teams stats into yards. Although they are presented in total yards, they can be translated into rates. I would be interested to seeif these adjusted yards correlate the same as the DVOA numbers