Rating Gameday Head Coaches

The head coach in the NFL must be a strategist, a leader, a talent evaluator, a manager, and perhaps most importantly--a field marshall. By field marshall, I mean the person who sets gameday tactics, who sets play call doctrine, who manages the clock, and who decides whether to go for it or kick on 4th down.

At kickoff, a head coach is given a certain ability level in his team. During the game it is his job to squeeze every possible advantage from his team's capabilities, while minimizing its weaknesses. Part of his duties are to accurately assess the risks and rewards of every situation. His team is ahead 14-10 and it's 3rd down and 7 late in the 4th quarter. Should he decide to run and likely have to punt, but chew up some time and force his opponent to use a time out? Should he pass and go for the 1st down but risk stopping the clock, or even worse, an interception? The answers depend on how accurate his perception is of his offense's passing ability and his defense's ability to stop the opposition.

So given those abilities, how can we judge a head coach on his ability to make the best in-game decisions? Here is one way.

In the last post, I charted teams' expected wins vs. their actual wins. The model that estimated the expected wins had an r-squared of 0.85. This mean 85% of the variance in team wins could be accounted for by the model's variables. (If you're not a statistician, that means it's really good.) Some teams had steady expected wins from year to year, but had somewhat erratic actual wins. Some teams had expected and actual wins that matched up every year. But a couple teams stood out as unique.

In recent years, the Lions won fewer games than would be expected given their performance on the field. I admit part of that could be due to luck. However, such a consistent underperformance leads me to suspect something systematic is at work. Could it be that the Lions' coaching staff is not effectively maximizing its team's abilities? Is it inaccurately assessing the risks and rewards of in-game situations? What else would explain such consistent underperformance? Except for an abberation in 2003 (when they only broke even), Detroit annually won about 2 games less than expected.

In stark contrast to the Lions are the Patriots. New England has consistently won about 2 more games than we'd expect given their on-field performance. To me, this appears to be evidence of Bill Belichick's coaching skills, or at least his knack for in-game decision-making. He is known for some unconventional practices, such as going for it on 4th down in more situations than other coaches.

To truly assess a head coach's abilities, we need to follow the coach from year to year and from team to team. Using data from pro-football-reference.com, I collected modern head coach's wins and efficiency stats for each year since the 1982 strike. I chose 1983 as a starting point because it followed the birth of the "west coast offense" and significant changes in NFL rules governing the passing game (which actually began in 1978 but took serveral years for coaches to adapt.) I only considered coaches with 4 or more years of experience. In some cases I included some noteworthy coaches with shorter tenures such as Lovie Smith or Jim Mora Jr. I also included Spurrier and Saban (2 years each) because I was interested in how "big-name" college coaches fared.

I constructed a regression model similar to my previous efforts to estimate expected wins given each team's passing and running efficiencies on both sides of the ball. I propose that the difference between a coach's actual wins and expected wins are, to some degree, "extra" wins that a coach can squeeze from his team given a particular performance level. It is an estimate of his strategic and gameday coaching abilities. Each coach's "extra wins" are averaged and sorted from best to worst.

Dungy 1.73
Smith 1.44
Shottenheimer 1.38
Gibbs 1.32
Billick 0.93
Reeves 0.91
Fisher 0.84
Lewis 0.82
Berry 0.81
Reid 0.79
Shula Don 0.73
Ditka 0.67
Martz 0.60
Belichick 0.59
Pardee 0.50
Mariucci 0.49
Holmgren 0.44
Ross 0.41
Shanahan 0.41
Parcells 0.39
Fassel 0.36
Wannstedt 0.36
Cowher 0.29
Haslett 0.24
Mora 0.21
Green 0.17
Saban 0.17
Johnson 0.17
Noll 0.16
Levy 0.15
Knox 0.05
Sherman 0.04
Robinson -0.08
Walsh -0.08
Coughlin -0.10
Seifert -0.12
Switzer -0.26
Del Rio -0.29
Phillips -0.36
Jauron -0.39
Glanville -0.41
Fox -0.51
Gruden -0.58
Kotite -0.62
Capers -0.66
Edwards -0.69
Erickson -0.69
Shell -0.73
Tice -0.85
Fontes -0.88
Flores -0.89
Vermeil -0.96
Rhodes -1.00
Infante -1.15
Carroll -1.22
Wyche -1.23
Mora, Jr. -1.24
Marchibroda -1.34
Shula David -1.40
Henning -1.42
Spurrier -1.54
Turner -1.63
Bugel -1.77
Riley -1.89
Coslet -2.15
Davis -2.32
Campo -3.20

The list ranks many of the more successful coaches at the top, and some of the least successful at the bottom. Shottenheimer, Gibbs, Reeves, Ditka, Shula are some of the legends at the top. The two coaches from last year's Super Bowl, Dungy and Smith, are at the top as well. Coaches with short and brutally unsuccessful tenures are at the bottom including Dave Campo, Butch Davis, Bruce Coslet, and Steve Spurrier. Some longer tenured coaches such as Ted Marchibroda and Norv Turner don't fare well either. I'm surprised to see top coaches such as Walsh and Cowher lumped in the middle.

But what happened to Belichick? He's down the list behind several other current coaches such as Billick, Reid, Fisher, and Lewis. It may be fair to break Belichick's head coaching career into two separate parts--at CLE and at NE. By his own admission he changed his approach following his time with the Browns, and applied what he learned since then into his current stint in NE. Belichick averaged -1.84 "extra" wins per season at CLE, but earned +2.33 "extra" wins per season at NE. The NE Belichick would be #1 on the list and the CLE Belichick would be third from the bottom. I might attribute such a difference to the small sample size (3 years) at CLE, but such a stark difference speaks for itself.

To be fair to other coaches with distinct tenures, I split the averages of some other notable coaches apart too. Redskins fans may not like what they see.

Gibbs ('83-'92)  2.11
('04-'06) -1.31

Ditka      CHI   1.37
NO -1.19

Shottenhmr CLE  -0.11
KC 1.97
WAS 0.83
SD 0.90

Parcells   NYG  -0.36
NE 1.97
NYJ 1.16
DAL -0.30

Reeves     DEN   1.85
NYG -0.26
ATL 0.11

Note: Also, since this post is appearing on many other sites recently, I've posted a response to many of the excellent comments and criticisms over the past several months.

Next Year's Wins

With a valid and predictive model of team wins (r-squared =0.80), we can estimate how many wins a team "should" have earned, known as expected wins. Based on a regression model using (offensive and defensive) passing, running, and turnover efficiency, plus penalties, we can compare expected wins with actual wins and make some observations.

The first thing I noticed was that expected win totals were much steadier from year to year than actual wins. This supports the notion that the efficiency model is actually a better evaluation of how good a team truly is than a team's actual wins. A team may be "X wins good," but might get lucky or unlucky by a game or two.

Further, if we want to predict the record of a team before the season starts, we'd be much better off by starting from their previous year's expected wins rather than their actual wins. By simply using prior expected wins to predict the following year's wins, the standard error is only about 1.7. That is, the estimate is usually wrong by less than a game or two. By using prior actual wins the standard error is about 3.0--not a very accurate predictor.

Take the recent records of the Jaguars for example. Their actual wins appear to be fluctuating around a relatively steady expected win trend. This gives the appearance that they are an inconsistent team, but in reality they are performing very steadily judging by their passing and running efficiencies, turnover rates, and penalties.

In another example, we see that the Falcons' season in 2004 was something of a fluke. They came back down to Earth after posting a win total over 3 wins higher than what their stats told us to expect. Sure enough, they returned to predictability in 2005. The likely reasons for such a deviation are an easy schedule and some lucky bounces of the ball (and perhaps the upreparedness of league defenses for Vick's running ability).

The Ravens illustrate another observation. Their mediocre performance through 2005 suddenly improved in 2006. This is due to the increase in offensive pass efficiency from 5.1 to 6.4 yards per attempt. The offensive line was the same, and they had the same receivers and tight ends. They only real difference was the quarterback--Steve McNair replaced Kyle Boller. The biggest changes in team fortunes are due to quarterback changes. We can also note that although Baltimore won 13 games, they had about 10 expected wins. This tells us not to hold our breath for another 13 win season in Baltimore, but still count on a solidly above-average team.

Both Baltimore's and Atlanta's graphs illustrate the importance of scheduling. Teams tend to overperform their stats when they have a low number of wins the previous year. Because of the NFL's scheduling system, and possibly somewhat due to draft placement, teams benefit following an "off" year.

Some teams are amazingly consistent and predictable. Notice how flat the Colts' trend line is, and how small the deviation between expected and actual wins.

The Jets are the counter-example. They've been on a roller coaster ride the past 5 years, in terms of actual wins. But their performance as a team has been very steady. It appears to be slightly above average. To Jets fans, it might appear that they had a miraculous turnaround from 4 wins in 2005 to 10 wins and a playoff berth in 2006. But to me, they underperformed in '05, benefitted from an easier schedule, and got a little lucky in '06 to scrape together 10 wins. This is a good measure of "are they for real?" The Jets are, unfortunately, not for real.

The biggest surprise "storybook" season in 2006 belonged to the Saints. So were they for real, or just lucky? Remember that true changes in team fortunes start with changes behind center, and Drew Brees made all the difference. The Saints' chart shows they are for real. They'll be a playoff contender again in 2007.

We can apply the reverse analysis to the "for real" logic. There are some teams that underperformed their stats, and can anticipate improving their win total next year, even without any fundamental improvement in team performance. Look for TEN, STL, MIA, PHI, PIT, MIN, ARI, and JAX to probably improve on their win total from last year.

Math vs. Vegas

I thought it might be interesting to compare my efficiency model's predictions with the consensus predictions. Probably the best estimates of the "consensus" predictions are the Las Vegas betting lines. Although I'm am not interested in gambling, betting odds and point spreads are useful as benchmarks for evaluating the usefulness of all this number crunching.

In this post I'll compare my 2007 predictions with the current over/under lines for season win totals.

The first thing that stands out is that my predictions are more aggressive. That is, my model goes further out on a limb by predicting wins further from the average (8 wins) than do the consensus predictions. In the AFC my predictions range from 4 to 12 (2 teams at 12) wins, compared to 5 to 12 (1 team at 12) wins for Vegas. In the NFC my predictions range from 5 to 12 wins, compared to 6 to 10 wins for Vegas.

6 to 10 wins is amazingly narrow. But this is a natural result of unconfident models. The less confident the model, the tighter the spread around the average dependent variable. Accordingly, we'd expect to see increasingly large differences in the predicted wins towards the top and bottom of each conference--which is exactly what we see.

The teams that show a big difference between the model and the consensus are of more interest. Notice that MIA, DEN, and NE have differences of at least 2 wins. The model is saying MIA is underrated, and DEN and NE are overrated. It's easy to understand the difference in NE's predictions considering their free agent acquisitions this year.

The notable differences in the NFC lie in the West. The model says SS is probably going to reliquish the division crown back to STL this year.

Note: The over/under lines are often in .5 increments. In these cases I rounded to the win total that is costlier to bet on, and therefore considered more likely. For example, if the line for a team is 9.5 wins and to bet on the 'over' it takes $120 to win $100, and to bet on the 'under' it takes $115 to win $100, I rounded to 10 wins. When the over and under were equally costly I left the prediction at a .5 increment.

NFC Predictions based on 2nd Half of '06

Here are the 2007 NFC projections for win totals using only efficiency stats from weeks 9-17 of the 2006 season. In parentheses are each team's difference from projected wins based on stats from the entire 2006 season.

CHI 10 (-2)
GB 9 (+1)
MIN 7 (0)
DET 6 (+1)

NO 10 (-1)
CAR 10 (+1)
TB 6 (+1)
ATL 3 (-4)

PHI 10 (-2)
DAL 9 (0)
NYG 8 (0)
WAS 5 (-1)

ARI 10 (+3)
STL 9 (-1)
SS 8 (+1)
SF 7 (0)

The Cardinals could be a big surprise if they continue to play the way they did the second half of 2006. It also looks like the collapse in Atlanta last year was for real, and not just due to luck.

AFC Predictions based on 2nd Half of '06

Instead of using team efficiency stats from all of the 2006 NFL season for the basis for 2007 predictions, I tried the same projections using only the stats from the 2nd half of the 2006 season.

Ignoring the first half of the season would account for several things. First, important personnel replacements occur partway through the season, such as quarterbacks Cutler, Leinart, and Campbell.

It also accounts for player development. Vince Young appeared to blossom in the second half of the year. Steve McNair turned it up a notch once he felt comfortable in the Ravens offensive system.

Lastly, teams as a whole fundamentally improve or get worse throughout a season. Using only data from the 2nd half of the season better captures those trends.

However, there are drawbacks to using a smaller data set. The sample size is smaller and the variance from luck is noisier. It also might capture effects from critical injuries that aren't certain to repeat this year. Donovan McNabb's and Bob Sanders' injuries come to mind.

So here are the 2007 projections for win totals using only efficiency stats from weeks 9-17 of the 2006 season. I'm not sure how much weight to give these projections, except that they give some deeper insight into the projections from the full 2006 season. In parentheses are each team's difference from projected wins based on stats from the entire 2006 season.

BAL 13 (+2)
PIT 8 (-2)
CIN 8 (-1)
CLE 2 (-2)

JAX 14 (+4)
IND 10 (-1)
TEN 7 (+1)
HOU 6 (+1)

NYJ 10 (+2)
NE 9 (0)
BUF 9 (+3)
MIA 8 (-1)

SD 11 (-1)
KC 8 (-1)
DEN 6 (-2)
OAK 5 (+1)

2007 Team Win Predictions

Here is an organized list of each divisions predicted wins for 2007. But before you believe anyone's pre-season predictions, read this.

AFC North
AFC South
AFC East
AFC West

NFC North
NFC South
NFC East
NFC West

2007 Prediction Summary

AFC Predictions (based on 2nd half of '06 only)
NFC Predictions (based on 2nd half of '06 only)

Advanced Football Analytics Overview

Who has the longest interception return ever? Which running back ran for the most yards in a single game? What team scored the most fourth quarter touchdowns in a season?

Answer: I don't care.

That's trivia, not statistics. What I'm interested in is analysis. What makes a winning team? Is it better to go for it or punt on 4th and short at the 50 yard line? Which teams are most likely to make the playoffs? How much does luck play a part in any game?

Advanced Football Analytics tries to answer those questions and many others. I call it "Advanced" because I use real statistical methods--regression, tests of significance, game theory, simulation and other advanced techniques. This isn't the kind of analysis you get on TV, and it's not for everyone.

This site is where you'll find some of the most innovative and interesting research in professional football. Almost everything here is completely original, but I also take a critical look at other analysis out there, debunking myths and fallacies where I find them. Being innovative and original can be risky, however. I can sometimes make mistakes, but I'm always open to constructive criticism and better ideas.

Thanks for stopping by.

2007 Prediction Summary

Below are the predicted wins for each NFL team based on a projection of last year's efficiency stats and this year's schedule.

BAL 11
PIT 10

IND 11
JAX 10

NE 9

SD 12
KC 9

CHI 12
GB 8

NO 11
TB 5

PHI 12

STL 10
SF 7

With these predictions, it looks like the playoff teams in the AFC woud be SD, BAL, PIT, IND, NE, and JAX. From the NFC, the playoff teams would be CHI, NO, PHI, STL, DAL, and CAR.

Note 1: I'm not actually predicting those exact win totals. The model is saying that each team's win probility distribution is centered at the specified win total. I realize that sounds rather weasel-like, but that's the nature of the sport. The standard deviation is just over +/-1 game and the 90% confidence interval is very close to +/-2 games. In other words, I'm about 65% sure STL will win between 9-11 games, and 90% sure that STL will win between 8 and 12 games.

Note 2: I've actually predicted 260 wins when there are really only 256 games. This is due to rounding.

Note 3: Historically, teams that have significant differences between their actual and expected win totals are teams that have had quarterback changes, due to either injuries or acquisitions. Look for teams such as MIA, KC, TB, and MIN to deviate from their projected wins.

Note 4: A couple teams have demonstrated a trend of winning consistently more or consistently fewer games than their efficiency stats would predict. NE, in particular, has won 2 more games than projected each of the past 5 years. In contrast is DET, which has consistently lost more games than their stats would predict. Perhaps this is evidence of good or bad coaching, especially in-game play calling and clock management. But maybe it's just evidence of lucky kicking.

NFC West 2007 Predictions

Here are the predictions for the NFC West. Methodology is described here.

AFC West 2007 Predictions

Here are the predictions for the AFC West. Methodology is described here.

NFC East 2007 Predictions

Here are the predictions for the NFC East. Methodology is described here.

Introducing "Air Yards"

If I've proven anything over the past year of analyzing NFL stats, it's that passing is by far the most important part of the game. So let's look at the pass and passing stats in more detail than usual.

The football pass is typically judged by how many yards it produces. Quarterbacks and receivers are judged by the their yardage compiled per game or per season. But there are two distinct components of the pass.


One of the components that is often discussed is Yards After Catch (YAC). Although it is not an official NFL stat, it is kept by Stats, Inc., and has become a buzzword when talking about the west coast offense. YAC is the yardage gained by the receiver after catching the ball.

The other component of the pass, the complement of YAC, has never been mentioned to my knowledge. It has no name...until now. The distance forward of the line of scrimmage that the passer throws a complete pass through the air is hereby called Air Yards. It can be expressed in a simple equation:

Pass Yds = Air Yds + Yds after Catch

From 2002 to 2006, NFL teams accumulated an average of 3533 yds per season. Of those passing yards, 2008 yds (57%) came through the air and 1533 (43%) yds came on the ground as YAC.


Digging a little deeper, I ran a couple very simple regressions. I estimated Passing Yards per Completion using Air Yds/Completion and then using YAC/Completion. (I couldn't run a single regression using both components because multivariate regression doesn't work when the predictor variables comprise the entirety of the dependent variable).

Air Yds/Completion accounts for 56% of the variance in Yds/Attempt, while YAC/Completion accounts for 20% of the variance. As a proportion of the total variance explained by both variables, Air Yds accounts for 74%, and YAC accounts for the other 26%. Put simply, Air Yards are 3 times as important than YAC in producing total passing efficiency.

Here is one way to interpret these results. Although Air Yds yards comprise 57% of total passing yards, it is far more critical than YAC. YAC is less variable among teams. Air Yards, therefore, is much more of a "difference maker" in determining total passing yards.

In future posts, I'll examine how much QBs contribute to YAC. Air yards are a far more appropriate and accurate measure of a QB's ability than total yards or yards per attempt. We'll also see how QBs vary greatly in their ability to produce air yards. Ultimately, these results will contribute to a better QB rating formula.

NFC North 2007 Predictions

Here are the predictions for the 2007 season in the NFC North. Methodology is described here. Note the predictions do not account for personnel changes such as free-agent acquisitions, draft picks, retirements, or otherwise. It is still the best purely statistical prediction available.

NFC South 2007 Predictions

Here are the predictions for the 2007 season in the NFC South. Methodology is described here. Note the predictions do not account for personnel changes such as free-agent acquisitions, draft picks, retirements, or otherwise. It is still the best prediction available.

AFC East 2007 Predictions

Here are the predictions for the 2007 season in the AFC East. Methodology is described here. Note the predictions do not account for personnel changes such as free-agent acquisitions, draft picks, retirements, or otherwise. It is still the best prediction available.

AFC South 2007 Predictions

Here are the season predictions for the AFC South. The solid lines are the probabilities (on left of graph) that each team will finish with that particular amount of wins. The curved line is the probability each team will finish with at least that many wins (on right). Methodology of estimating these probabilities is explained here.

AFC North 2007 Predictions

Based on the assumptions described previously, here are the predictions for the AFC North division. The graphs represent season wins for each team. The solid bars represent the probability that the team will achieve exactly that number of wins in 2007. The curving black line represents the probability that win at least that many wins.

Notice how that for each team the distribution of win probability is almost perfectly normal. The "at least" win distribution is sigmoidal, i.e. an S-curve. It is a CDF--a cumulative distribution function.

Note that the model does not predict CLE will have have 4 wins. It predicts that it's win-probability-distribution is centered at 4 wins. The Browns could have up to 9 wins, or could go 0-16. Although BAL has the best chance to win the division, PIT and CIN have solid chances as well.

This does not take into account personnel changes--draft picks, free agent singings, or retirements (or suspensions in the Bengals case). It merely extrapolates last season's performance stats onto this year's schedule, then calculates the total probabilities of every possible outcome.

2007 Season Win Predictions

I've assembled a new and larger database and tested the most predictive variables for wins. Now it's time to apply the data to the upcoming 2007 NFL season.

Using data from the 2002-2007 seasons, I've run a logit regression on the efficiency stats most predictive of winning for every game played. The result is a model that correctly predicts the winner of 68.9% of games during the 5 most recent seasons.

For every game, the model produces a probability that each team would win. We can apply this model to future games as well, assuming we can estimate the values of each team's efficiency statistics. As any shrink will tell you, the best predictor of future behavior is past behavior. We'll use 2006 stats as our baseline for 2007. I realize this is highly imperfect, but it is the best predictor available. Although this method does not account for personnel changes, efficiency stats are relatively very steady from year to year--much steadier than actual win totals. Previous year 'expected win' totals are singificantly better predictors of the following year's record than are previous year actual wins. Plus, this is all just for fun.

I now have win probabilities for all 256 upcoming regular season games and I've sorted them by team. The probabilily of every possible sequence of game outcomes for a team (there are 2^16 of them--65,536) was computed. Then each sequence of outcomes that result in a certain number of total wins is summed. (There is 1 possible sequence for 16 wins, 16 sequences for 15 wins, 256 sequences for 14 wins,...) Now I have estimated probabilities for every win total for each team.

For example, the Ravens have a breakdown of win probabilities that centers on 11 wins. Although they had 13 wins last year, they appeared to squeak by on a couple wins on luck, and they had a relatively easy schedule. This year, they'll play a much tougher schedule. The table below lists the probabilities for each game on their '07 schedule based on last year's efficiency stats.

BAL CIN 0.54 0.46
NYJ BAL 0.26 0.74
ARI BAL 0.17 0.83
BAL CLE 0.82 0.18
BAL SF 0.69 0.31
STL BAL 0.32 0.68
BAL BUF 0.67 0.33
BAL PIT 0.47 0.53
CIN BAL 0.28 0.72
CLE BAL 0.09 0.91
BAL SD 0.32 0.68
NE BAL 0.34 0.66
IND BAL 0.49 0.51
BAL MIA 0.57 0.43
BAL SS 0.69 0.31
PIT BAL 0.34 0.66
The next table lists the probability of winning each possible number of games. The 'cumulative' column lists the probability of the Ravens winning at least that number of games.

16 0.00 0.00
15 0.01 0.01
14 0.03 0.04
13 0.09 0.14
12 0.17 0.30
11 0.21 0.52
10 0.20 0.72
9 0.15 0.87
8 0.08 0.95
7 0.03 0.99
6 0.01 1.00
5 0.00 1.00
4 0.00 1.00
3 0.00 1.00
2 0.00 1.00
1 0.00 1.00
0 0.00 1.00
The stats say 11 wins is the most probable of all win totals for Baltimore (21% probability). Keep in mind however, they're far more likely to finish with some other total (79%). The cumulative probabilities indicate they have a 50/50 chance to finish with at least 11 wins (52%).

Year-to-Year Team Wins

My recent analysis of draft picks and their relative value was complicated by the problem of every team’s tendency to regress to mean. Good teams appear to get worse, and bad ones appear to improve. Because draft picks are allotted according to team record, it is very difficult to tell whether the change team in record was due to the draft picks.

One factor dominates the equation when trying to predict the following year’s record with draft values—the previous year’s record. Teams with high numbers of wins tend to have fewer wins the following year, and teams with low win totals tend to have more wins in the next season. This tendency is strong.

In fact, if you look at a graph of every team’s change in wins from year to year (DELTAWINS) vs. the team’s win total from last year (LASTWINS), you see a very strong trend towards mediocrity, i.e. regression to the mean. The graph includes all teams from 2002-2006.

In fact, between ‘02-’06 no team that had 13 or more wins improved their record the next season, and no team with 3 or fewer wins got worse. Only 1 team with 4 wins earned fewer wins, and only 2 teams with 5 wins got worse.

The reasons for teams to tendency to regress to the mean are likely the obvious ones. First, there is the scheduling system. Each team is given 2 strength of schedule games in which they play the teams that finished in the same order in their respective divisions. First-place teams play the other first place teams, second place teams play each other, and so on. Additionally, there is the draft which allocates draft position in reverse order of win-loss records. Lastly, there may be effects of salary cap boom/bust cycles in which individual teams load up on talented and costly players by amortizing signing bonuses into out-years. This causes teams to 'purge' their rosters of those players, and others, to allow cap room for the dead weight of past signing bonuses.

NFL Stats Overview

This post is intended to summarize the statistical methods commonly used throughout this site. The statistical model uses each team's efficiency statistics in running and passing, on both offense and defense, to predict team wins. Turnover rates and penalties are also included. Multivariate linear regression was used to estimate total season wins, and multivariate logit (non-linear logistic) regression was used to estimate individual game probabilities.

Efficiency stats are defined by yards per play. Passing efficiency is defined as the average yards gained (or lost) per passing play. I included yards lost in sacks in both offensive and defensive pass efficiency. Likewise, run efficiency is defined as yards per run. Turnovers are also defined as efficiencies. Interception efficiency is defined as interceptions per pass play. Fumbles are defined as fumbles per any play--both run and pass plays can result in fumbles.

Efficiency stats were found to be correlated with wins much better than total yardage stats. Teams with poor defenses that find themselves trailing (and likely to lose) tend to accumulate large amounts of gross passing yards, but only because of frequent attempts. Teams that are ahead (and likely to win) tend to accumulate large amounts of gross running yards. In both cases it's often the winning or losing that lead to the yardage, not the other way around. For this reason, efficiency stats are almost always a better measure of a team's proficiency than gross yardage stats.

Once I established a workable, logical, predictive, and statistically significant model of winning and losing, there were any number of interesting applications. I could now predict individual games. Summing the total probabilities of all 256 games in the season estimates the probabilities of a total number of wins for each team. Toward the end of the season, the game-by-game model can also be used to compare likelihoods of selected teams earning a playoff spot, or will capture a wildcard berth. I could also create very accurate power (or efficiency) rankings. Another application was that I could estimate how lucky each team was, and how much luck played a part in determining game and season outcomes.

I'm not just trying to predict games. I'm trying to understand the game itself. For example, which is more important--running or passing? Offense or defense? Interceptions or fumbles? Do special teams matter? But answering those questions accurately depends on how accurate the model is, which can only be judged by how well it predicts wins. We each may have intuitive answers to these questions, but statistics is one of the best ways to test our hunches. We can even quantify them.

2006 was the first season in which I did all of this number crunching. I needed some minimum amount of data to use for the efficiency stats, so I started predictions after week 4. As the season went on, I had a larger data set. Ulitmately, here is how my predictions fared:

At first glance, 63% correct may not appear too impressive, especially for all that work I did. But almost 2 out of 3 games correct isn't too bad, especially when compared to the national consensus. Each week, I recorded not only my own results, but those of the Las Vegas odds makers as well. (Although my purposes here are not related to gambling, I'll use the betting lines as a benchmark.) The Vegas line was only accurate in picking winners 58.2% of the time for the entire '06 season. Vegas' record was slightly worse from week 5 onward, at 57.1%.

By end of the 2006 regular season, my model was correct significantly more often than the Vegas line, by 63% to 57%. On the surface, it may not sound as impressive as it really is. But think of game prediction this way: A monkey will guess winners correctly 50% of the time. So the real question is: how much better than 50% is the model? In this case, the efficiency model added almost twice the predictive power as the Vegas line (the consensus favorite).

The 2007 season was more predictable. The model was 70.8% correct compared to just 66.7% for the Vegas consensus. Over the past two years it has been the most accurate prediction system published.

Also consider that no model could be 100% correct. Upsets happen for many reasons. Some games are very evenly matched, so the favorite has very little advantage to begin with. Additionally, luck plays a large role in determining outcomes. It's hard to say exactly how well the theoretically best possible model could do, but from my experience it seems it would be something just under an 80% correct rate.

Ultimately, we're not talking about 71% vs. 67%. We're asking how far from 50% and how close to 80% can a prediction model get.