Instability Compensation

To compensate for early-season over-confidence in model predictions, the model now includes a mechanism for extremes in team stat variation.

To explain how the compensation works, consider the recent SD at DEN game. The model had DEN at 0.97 favorites to win, but the Chargers crushed the Broncos. (Some of the following is taken from my comments to the week 5 predictions.) The current model basically says this:

1. Assumes SD's and DEN's to-date efficiency stats represent their true full-season talent,skill, and performance.
2. If a team with season-long stats such as SD's plays at a team with season-long stats such as DEN's, the home team with a stat advantage that enormous would win 97 times out of 100 games.

I think #1 is the problem, but I think #2 is true. I think the overconfidence in the model at this point in the season comes from only a few data points for each statistic so far. Outlier/unrepresentative performances have a large effect on each team's aggregate efficiency stats early in a season. In other words, Denver may have played their best 4 games in terms of team efficiencies already, and from here out their stats will regress to the mean. Until their efficiency stats stabilize, the model would overweight Denver's odds of winning, and vice versa for San Diego.

[Incidentally, this would be true of all statistical models this early in the season. But I think a pure efficiency model might be less susceptable. For example, including 3rd down conversion rates or red zone rates would exaggerate the effect of unrepresentative performances to an even greater degree.]

Ultimately, it's the unstable input variables which may not represent each team's true ultimate efficiency averages. I had already been working on a simple method of Bayesian compensation for early-season unstable team stats. It's similar to what IMDB or other similar sites do with their movie-rating system. IMDB doesn't want one voter to give 5 stars to his favorite Pauly Shore movie, resulting in a perfect 5.00 rating for "Encino Man" until someone else takes time to rate it a zero. So they assign every movie a certain number of baseline votes, so one early voter doesn't move the aggregate too far. Eventually, as the real votes accumulate, the baseline phantom votes are pushed out of the calculation. Throughout the process, the aggregate rating remains stable.

The model's input stats will include a number of "phantom" games at league-average stats for every team. This mitigates the exaggerated confidence levels early in the season. (DEN's stats wouldn't have looked so good, and SD's wouldn't have looked so bad.) The question is, how many pure-average phantom games should be used? We need to look at how quickly each team's stats stabilize. In other words, how many games of data are required before the stats are within an acceptable margin of their final season average?

The answer is more complicated than expected. I plotted each team's 2006 stats by week. The graph for defensive pass efficiency is posted below as an example. The y-axis is yds allowed per attempt and the x-axis is week number. Although hard to read, it's easy to see that the 'funnel' stabilizes shortly before halfway through the season.



Each stat has it's own rate of stabilization. For example, offensive run efficiency stabilizes very quickly, within 4 or 5 games. But defensive pass efficiency doesn't stabilize until about week 8. Turnover rates don't stabilize until week 9.

The compensation is reduced as each week goes by in the early season. Each week, one fewer phantom game is included in the team's averages until the each stat stabilized. For example, by week 5 the compensation for offensive run efficiency has run its course. And by week 8, defensive pass efficiency no longer contains any compensation. At this point in the season, between week 5 and 6, the resulting effect is to mitigate the importance of defensive pass efficiency and turnover rates.

Season Win Projections Week 5

Season win totals and division standing projections are listed below. Projections are based on each team's opponent-adjusted generic win probability (Adj GWP). Total wins account for current and projected wins. Methodology can be found here.

Team	Adj GWP	Current	Proj.
AFC E
NE	0.92	5	15.2
BUF	0.43	1	5.7
MIA	0.47	0	5.2
NYJ	0.28	1	4.1
AFC N
PIT	0.70	4	11.7
CIN	0.60	1	8.2
BAL	0.37	3	7.1
CLE	0.34	2	5.7
AFC S
IND	0.90	5	14.9
JAX	0.76	3	12.2
TEN	0.69	3	11.3
HOU	0.61	3	9.7
AFC W
DEN	0.70	2	9.7
SD	0.47	2	7.2
KC	0.43	2	6.8
OAK	0.34	2	6.0
NFC E
DAL	0.82	5	14.0
WAS	0.81	3	12.7
NYG	0.82	3	12.1
PHI	0.70	1	9.5
NFC N
GB	0.60	4	10.6
MIN	0.40	1	5.8
DET	0.25	3	5.8
CHI	0.16	2	3.8
NFC S
TB	0.73	3	11.0
CAR	0.36	3	7.0
ATL	0.40	1	5.4
NO	0.23	0	2.8
NFC W
SEA	0.73	3	11.0
ARI	0.62	3	9.8
SF	0.26	2	4.9
STL	0.24	0	2.7

Week 5 Efficiency Rankings

Team efficiency rankings are listed below in terms of generic winning probability. The GWP is the probability a team would beat the league average team at a neutral site. Each team's opponent's average GWP is also listed, which can be considered strength of schedule. The adjusted GWP (Adj GWP) modifies the generic win probability to reflect the strength of to-date opponents. Rankings are based on data through week 5. A full explanation of the methodology can be found here.

Team	Rank	Last Wk	Adj GWP	Opp GWP
NE	1	4	0.92	0.36
IND	2	1	0.90	0.47
NYG	3	8	0.82	0.72
DAL	4	5	0.82	0.34
WAS	5	12	0.81	0.56
JAX	6	7	0.76	0.54
SEA	7	3	0.73	0.53
TB	8	6	0.73	0.33
PHI	9	9	0.70	0.58
PIT	10	11	0.70	0.34
DEN	11	2	0.70	0.58
TEN	12	10	0.69	0.57
ARI	13	13	0.62	0.55
HOU	14	15	0.61	0.55
GB	15	16	0.60	0.54
CIN	16	14	0.60	0.60
MIA	17	18	0.47	0.59
SD	18	29	0.47	0.55
KC	19	17	0.43	0.39
BUF	20	22	0.43	0.67
MIN	21	19	0.40	0.45
ATL	22	25	0.40	0.53
BAL	23	24	0.37	0.41
CAR	24	27	0.36	0.49
OAK	25	21	0.34	0.41
CLE	26	23	0.34	0.55
NYJ	27	28	0.28	0.52
SF	28	30	0.26	0.57
DET	29	20	0.25	0.41
STL	30	31	0.24	0.58
NO	31	26	0.23	0.79
CHI	32	32	0.16	0.55

Opponent Strength

To get a feel for which teams are really doing well, and which have just been beating up on weaker teams, we should look at strength of schedule. One of the interesting applications of accounting for opponent strength using a regression model is that offensive and defensive opponent strength can be analyzed separately. We can see which squads have faced the toughest and easiest opposing units so far this year.

The table below lists each team and their to-date opponents' strength ranked from toughest (1) to weakest (32). The table includes overall opponent strength, opponent offensive strength, and defensive oppoenent strength.

Team	Opp Str	Opp O Str	Opp D Str
NO	1	1	1
BUF	2	2	5
NYG	3	4	2
CIN	4	3	12
JAX	5	5	10
STL	6	8	6
CHI	7	20	4
MIA	8	10	11
SF	9	14	18
PHI	10	21	8
SD	11	24	3
DEN	12	9	21
SEA	13	18	14
HOU	14	6	20
ARI	15	22	9
TEN	16	7	27
OAK	17	16	16
WAS	18	12	19
CLE	19	19	15
IND	20	25	23
MIN	21	23	13
ATL	22	17	24
NYJ	23	11	25
CAR	24	13	26
GB	25	26	17
BAL	26	15	30
DET	27	31	7
PIT	28	29	29
KC	29	32	22
TB	30	28	32
NE	31	27	31
DAL	32	30	28

The defensive units that have faced the toughest offenses so far are NO, BUF, and CIN. Look for those defenses to improve if only because they're not likely to face as difficult a schedule as they have so far. Likewise, the teams that have faced the weakest defenses so far are KC, DET, and DAL. They might appear better than they really are.

The offensive units that have faced the toughest defenses so far are NO, NYG, SD, and CHI. The Giants haven't been too bad on offense so far this year, so perhaps they'll put up some bigger numbers when their schedule eases up. NO has started with the most brutal schedule of all 32 teams. The teams that have faced the weakest offenses so far are TB, NE, BAL, PIT, and TEN. Those defenses might be overrated at this point.

Game Predictions Week 5

Game probabilities for week 5 are listed below. The probabilities are based on an efficiency win model explained here and here. The model considers offensive and defensive efficiency stats including running, passing, sacks, turnover rates, and penalty rates. Team stats are adjusted for previous opponent strength.

V Prob	VISITOR	HOME	H Prob
0.85	ARI	STL	0.15
0.13	ATL	TEN	0.87
0.62	CAR	NO	0.38
0.04	CLE	NE	0.96
0.22	DET	WAS	0.78
0.73	JAX	KC	0.27
0.33	MIA	HOU	0.67
0.10	NYJ	NYG	0.90
0.69	SS	PIT	0.31
0.19	TB	IND	0.81
0.49	BAL	SF	0.51
0.03	SD	DEN	0.97
0.09	CHI	GB	0.91
0.90	DAL	BUF	0.10

Season Win Projections Week 4

Season win totals and division standing projections are listed below. Projections are based on each team's opponent-adjusted generic win probability (Adj GWP). Total wins account for current and projected wins. Methodology can be found here.

Team	Adj GWP	Current	Projected
AFC E
NE	0.79	4	13.5
MIA	0.43	0	5.2
BUF	0.32	1	4.8
NYJ	0.26	1	4.1
AFC N
PIT	0.61	3	10.3
CIN	0.55	1	7.6
CLE	0.32	2	5.8
BAL	0.30	2	5.6
AFC S
IND	0.92	4	15.0
JAX	0.70	2	11.1
TEN	0.64	2	10.3
HOU	0.53	2	8.4
AFC W
DEN	0.86	2	12.3
KC	0.44	2	7.3
OAK	0.41	2	6.9
SD	0.23	1	3.8
NFC E
DAL	0.78	4	13.3
WAS	0.61	2	9.9
NYG	0.65	2	9.8
PHI	0.65	1	8.7
NFC N
GB	0.50	4	10.0
DET	0.42	3	8.0
MIN	0.42	1	6.0
CHI	0.14	1	2.6
NFC S
TB	0.75	3	12.0
CAR	0.26	2	5.1
ATL	0.29	1	4.5
NO	0.26	0	3.4
NFC W
SS	0.82	3	12.8
ARI	0.59	2	9.0
SF	0.23	2	4.7
STL	0.18	0	2.1

For now, the projections don't account for future strength of schedule. At this point only 4 games into the season, you should pay more attention to the relative ranking of each team than the projected win total. Further into the season, the projections will account for future opponent strength.

2006 Season Win Projections

Here are the projections from the same timeframe in 2006. Each column indicates the season win projections of a very similar model following weeks 2, 3, and 4. The actual total numbers of wins aren't terribly great, but the relative position of teams within each division is pretty good. I can't emphasize that point enough.

Seven of eight division winners were correctly projected. The only exception was Atlanta in the NFC South, which suffered an epic collapse at the end of the '06 season. The model actually had NO winning the division in week 3, but gave it to ATL by week 4.

Team	Week 2	Week 3	Week 4	Actual
AFC E
06New Engl	9.2	6.6	8.7	12
06Buffalo	5.7	7.0	8.1	7
06NY Jets	7.5	6.9	7.0	10
06Miami	5.6	6.1	5.6	6
AFC N
06Baltimor	9.8	10.6	10.9	13
06Cincinna	10.5	10.9	9.5	8
06Pittsbur	7.4	6.9	6.3	8
06Clevelan	5.4	5.5	4.6	4
AFC S
06Indianap	7.1	9.4	10.4	12
06Jacksonv	10.9	9.5	8.2	8
06Houston	3.8	3.6	4.3	3
06Tennesse	3.9	3.2	2.1	8
AFC W
06San Dieg	12.6	14.2	13.0	14
06Denver	6.0	10.2	10.1	9
06Kansas C	7.2	7.4	9.4	9
06Oakland	5.7	4.4	4.3	2
NFC E
06Philadel	10.4	12.1	13.1	10
06Dallas	10.5	9.7	12.1	9
06NY Giant	11.4	11.5	11.5	8
06Washingt	7.5	9.3	10.9	5
NFC N
06Chicago	11.3	11.3	12.2	13
06Minnesot	8.8	10.2	9.3	6
06Green Ba	6.7	6.0	4.9	8
06Detroit	6.1	5.2	4.4	3
NFC S
06Atlanta	10.3	9.4	11.1	7
06New Orle	9.2	11.1	10.1	10
06Carolina	4.5	4.1	5.8	8
06Tampa Ba	4.4	1.6	1.6	4
NFC W
06Seattle	9.5	10.6	8.6	9
06St. Loui	6.3	6.5	7.8	8
06San Fran	10.7	8.3	6.5	7
06Arizona	7.2	5.0	3.1	5

The biggest misses were the Jets and Redskins. The Jets have returned to mediocrity and were most likely very luckly last year. The model was very wrong about the Redskins, and I have no idea why they took the downturn they did last year.

One more note: Although these predictions were based on a similar model to the current one, the coefficients were derived solely from 2005 season data. 2005 turned out to be uniquely favorable to running offenses and defenses, which skewed the projections. I redid the model using data from 2002-2006 and the results were much better.

Why do the Chargers Stink?

The San Diego Chargers are 1-3 so far in 2007 and are on pace to win between only 3 to 4 games this year according to their efficiency stats. Their sole win has come against perhaps the only team considered more disappointing this year, the Chicago Bears. Media attention focuses on star running back LaDanian Tomlinson and his woes, but what are the real reasons for San Diego's slow start?

By comparing San Diego's stats in 2006 and 2007 through an efficiency model, we can see how the difference in each stat impacts the number of expected wins. The table below lists each efficiency stat in the model for both years. The column "+/-Wins" lists the estimated number of wins lost or gained due to the difference in each stat. Also listed is the Generic Win Probability (GWP) for the Chargers for 2006 and 2007. The GWP is the probability a team would beat the league-average team at a neutral site. The Adjusted GWP accounts for opponent strength.

STAT	SD '06	SD '07	+/-Wins
OPass	6.60	5.54	-1.6
ORun	4.90	3.10	-1.6
OIntRate	0.019	0.043	-1.2
OFumRate	0.016	0.029	-1.0
DPass	5.36	7.39	-3.4
DRun	4.20	3.96	+0.1
DIntRate	0.030	0.031	0.0
PenRate	0.40	0.38	+0.1
GWP	0.77	0.20
Adj GWP	0.77	0.23
Exp. Wins	12.3	3.7	-8.6

First, note that the Chargers were expected to win 12.3 wins last year, 1.7 wins short of their actual total. This indicates they were a solidly talented team, but there was some luck involved in getting to a 14-2 record. This year they are on pace to win 3.7 wins, a difference of 8.6 wins.

LT has seen his yards per rush attempt fall from 5.2 in 2006 to 3.4 in 2007, certainly a significant difference. But San Diego's running game only accounts for 1.6 of the 8.6 win difference between '06 and '07. Their passing game shares and equal amount of blame with a 1.6 win difference.

San Diego's opponents so far have an average GWP of 0.54, slightly above average. But this would only account for 0.03 in their own GWP so far. This equates to only 0.1 wins.

The Chargers' pass defense is the prime culprit, accounting for 3.4 wins of the 8.6 difference in expected wins. Last year their defense gave up only 5.36 yards per pass attempt, but this year they are allowing 7.39 yards per attempt (including sack yards). Although they've faced a ferocious New England passing attack and a resurgent Brett Favre, they've also played against below average QBs Rex Grossman and Damon Huard.

But in total, it's the Charger offense that's responsible for the bulk of their difficulties. Added together, their passing game and running game including turnovers are costing them 5.4 expected wins over the course of a season.

Are the Chargers a 4-win team? Maybe not. But for them to turn their season around, their pass defense needs to improve and not necessarily just their running game.