The team rankings below are in terms of generic win probability. The GWP is the probability a team would beat the league average team at a neutral site. Each team's opponent's average GWP is also listed, which can be considered to-date strength of schedule, and all ratings include adjustments for opponent strength.
Offensive rank (ORANK) is offensive generic win probability, which is based on each team's offensive efficiency stats only. In other words, it's the team's GWP assuming it had a league-average defense. DRANK is is a team's generic win probability rank assuming it had a league-average offense.
GWP is based on a logistic regression model applied to current team stats. The model includes offensive and defensive passing and running efficiency, offensive turnover rates, defensive interception rates, and team penalty rates. If you're scratching your head wondering why a team is ranked where it is, just scroll down to the second table to see the stats of all 32 teams.
Click on the table headers to sort.
| RANK | TEAM | LAST WK | GWP | Opp GWP | O RANK | D RANK |
| 1 |  SD | 1 | 0.81 | 0.44 | 2 | 2 |
| 2 |  PIT | 3 | 0.78 | 0.51 | 5 | 1 |
| 3 |  GB | 5 | 0.75 | 0.52 | 4 | 6 |
| 4 |  NE | 2 | 0.75 | 0.58 | 1 | 19 |
| 5 |  NYG | 4 | 0.68 | 0.49 | 6 | 8 |
| 6 |  BAL | 7 | 0.63 | 0.52 | 12 | 5 |
| 7 |  PHI | 6 | 0.62 | 0.52 | 7 | 10 |
| 8 |  NYJ | 11 | 0.62 | 0.57 | 20 | 3 |
| 9 |  IND | 9 | 0.57 | 0.52 | 13 | 12 |
| 10 |  HOU | 14 | 0.56 | 0.54 | 3 | 30 |
| 11 |  MIA | 8 | 0.55 | 0.56 | 24 | 14 |
| 12 |  TEN | 10 | 0.54 | 0.53 | 19 | 4 |
| 13 |  KC | 12 | 0.51 | 0.45 | 22 | 9 |
| 14 |  CHI | 13 | 0.51 | 0.52 | 29 | 7 |
| 15 |  MIN | 16 | 0.51 | 0.55 | 27 | 11 |
| 16 |  DAL | 15 | 0.50 | 0.51 | 9 | 25 |
| 17 |  TB | 23 | 0.50 | 0.41 | 8 | 26 |
| 18 |  CIN | 24 | 0.48 | 0.56 | 16 | 21 |
| 19 |  BUF | 17 | 0.47 | 0.57 | 25 | 22 |
| 20 |  NO | 18 | 0.46 | 0.42 | 11 | 24 |
| 21 |  ATL | 20 | 0.46 | 0.47 | 17 | 20 |
| 22 |  DET | 26 | 0.43 | 0.56 | 23 | 17 |
| 23 |  OAK | 22 | 0.43 | 0.49 | 15 | 13 |
| 24 |  SF | 25 | 0.43 | 0.43 | 14 | 16 |
| 25 |  CLE | 19 | 0.43 | 0.52 | 26 | 23 |
| 26 |  WAS | 21 | 0.40 | 0.53 | 21 | 29 |
| 27 |  JAC | 27 | 0.33 | 0.53 | 18 | 31 |
| 28 |  DEN | 28 | 0.33 | 0.47 | 10 | 32 |
| 29 |  SEA | 29 | 0.29 | 0.43 | 28 | 27 |
| 30 |  STL | 30 | 0.27 | 0.40 | 30 | 18 |
| 31 |  CAR | 31 | 0.21 | 0.47 | 32 | 15 |
| 32 |  ARI | 32 | 0.19 | 0.42 | 31 | 28 |
And here are each team's efficiency stats.
| TEAM | OPASS | ORUN | OINT% | OFUM% | DPASS | DRUN | DINT% | PENRATE |
| ARI | 4.8 | 4.3 | 3.4 | 1.1 | 6.5 | 4.4 | 3.2 | 0.43 |
| ATL | 5.9 | 3.8 | 1.6 | 0.5 | 6.1 | 4.6 | 3.9 | 0.29 |
| BAL | 6.3 | 3.8 | 2.0 | 0.7 | 5.8 | 3.9 | 3.2 | 0.33 |
| BUF | 5.7 | 4.3 | 4.0 | 1.5 | 6.1 | 4.8 | 2.3 | 0.31 |
| CAR | 4.3 | 4.3 | 4.3 | 2.2 | 6.1 | 3.9 | 3.2 | 0.45 |
| CHI | 5.8 | 3.9 | 4.5 | 0.1 | 5.8 | 3.7 | 3.6 | 0.40 |
| CIN | 6.1 | 3.6 | 3.4 | 1.3 | 6.4 | 4.4 | 3.1 | 0.36 |
| CLE | 5.8 | 4.0 | 3.8 | 1.7 | 6.6 | 4.1 | 3.7 | 0.34 |
| DAL | 6.7 | 4.2 | 3.3 | 0.6 | 6.8 | 4.3 | 3.7 | 0.43 |
| DEN | 6.5 | 3.9 | 2.1 | 1.6 | 7.2 | 4.7 | 2.0 | 0.46 |
| DET | 5.8 | 4.0 | 2.5 | 0.7 | 6.2 | 4.5 | 2.7 | 0.49 |
| GB | 7.1 | 3.8 | 2.4 | 0.5 | 5.4 | 4.7 | 4.6 | 0.31 |
| HOU | 6.8 | 4.8 | 2.1 | 0.6 | 7.4 | 4.0 | 2.4 | 0.34 |
| IND | 6.6 | 3.8 | 2.5 | 0.6 | 6.1 | 4.6 | 1.9 | 0.34 |
| JAC | 6.0 | 4.7 | 4.5 | 1.0 | 7.5 | 4.7 | 2.6 | 0.34 |
| KC | 5.9 | 4.7 | 1.7 | 0.8 | 5.7 | 4.3 | 2.4 | 0.37 |
| MIA | 5.9 | 3.7 | 3.8 | 1.5 | 6.2 | 3.6 | 2.2 | 0.29 |
| MIN | 5.7 | 4.4 | 5.1 | 0.9 | 6.0 | 3.9 | 2.8 | 0.39 |
| NE | 7.2 | 4.3 | 1.0 | 0.2 | 6.4 | 4.2 | 4.1 | 0.38 |
| NO | 6.5 | 4.0 | 3.3 | 0.8 | 5.9 | 4.3 | 1.8 | 0.44 |
| NYG | 7.0 | 4.6 | 4.6 | 1.4 | 5.7 | 4.2 | 3.0 | 0.39 |
| NYJ | 5.9 | 4.4 | 2.7 | 1.3 | 5.6 | 3.6 | 2.3 | 0.45 |
| OAK | 5.9 | 4.9 | 3.3 | 1.1 | 5.9 | 4.5 | 2.6 | 0.63 |
| PHI | 6.4 | 5.5 | 2.3 | 0.8 | 6.0 | 4.2 | 4.3 | 0.54 |
| PIT | 6.9 | 4.1 | 1.9 | 1.1 | 5.3 | 3.0 | 3.5 | 0.46 |
| SD | 7.8 | 4.0 | 2.4 | 1.5 | 5.3 | 3.7 | 3.3 | 0.34 |
| SF | 6.2 | 4.1 | 3.0 | 0.8 | 6.3 | 3.5 | 2.7 | 0.47 |
| SEA | 5.8 | 3.7 | 3.7 | 0.4 | 6.4 | 4.2 | 2.1 | 0.44 |
| STL | 5.2 | 3.7 | 2.5 | 0.2 | 5.8 | 4.5 | 2.5 | 0.42 |
| TB | 6.4 | 4.6 | 1.2 | 0.9 | 5.9 | 4.7 | 3.6 | 0.43 |
| TEN | 6.2 | 4.3 | 3.2 | 1.0 | 6.1 | 3.9 | 2.7 | 0.51 |
| WAS | 6.0 | 4.2 | 3.1 | 0.8 | 6.9 | 4.6 | 2.4 | 0.32 |
| Avg | 6.2 | 4.2 | 3.0 | 0.9 | 6.2 | 4.2 | 2.9 | 0.40 |
I know you get this a lot, but when two of your top five and five of your top 12 fail to make the playoffs, you have to explain why your system is a good one.
Not saying the system is bad, but you really have to introduce an explanatory variable and discuss it to build credibility in your product. That explanatory variable may well be turnovers, but you have to address it.
Also, I think your system might be improved by giving more weight to recent performance. For example, New England's defense is probably low ranked because of its performance early. My impression is that it has gotten better over the last few weeks. Maybe someone who works with times series a lot has ideas.
Sample size plays a large role. There's a much larger sample size for offensive/defensive plays than there are for games, so it follows that measuring offensive/defensive is more accurate than games played.
Jim -> it's not a measure of wins, it's a measure of efficiency. Efficiency can lead to wins, but in football, there's a lot of random chance. Sometimes the ball bounces funny on a Browns D Lineman rumbles across the field for a touchdown. Twice, IIRC.
That's a big deal as far as win/loss goes, but it's not really a measure of efficiency.
It's a good measure of "yeah but" factor as I call it. Sure, Seattle is in the playoffs. Yeah but they suck, as these stats illustrate. Houston didn't make the playoffs. Yeah, but they aren't a total train wreck, and, once again, there should be hope for the team next year.
This whole site is an explanation for why the system is good. You think he really needs to say it every single week?
I would love to see some study that discusses the results of his weekly game probabilities predictions, yes. Brian seems to avoid discussing this topic
a very basic measure that should be relatively easy to quantify: how do his game probability predictions compare to Vegas moneylines?
It's beat the money lines 3 out of the last 3 years, independently verified. This year, however, I seriously doubt it has.
I do avoid over-analyzing model results every single week, as I should. We'll wait until the season is over and take a look.
Brian,
fair enough. would love to see something when the season is over...also have you posted before about the last 3 years results you mentioned? have any links handy?
Sure. Most recent one was posted in a comment earlier this season in one of the previous weekly ranking posts. Not sure which one.
Brian,
Maybe you should make wins the only variable to your formula that way the rankings will look more like the rankings that some of the random posters want :)
On a more serious note have you considered some type of dummy variable for a team that has been eliminated from the playoffs? I'm not exactly sure how that would look but I am going to assume that teams like Miami and Tennessee were not giving nearly the same effort at the end of the year as they were in the begining of the year when they accumulated their great efficiency rankings.
I really wish SDG would have made it into the playoffs. I would have been interested to see how that played out.
hmm I went back and looked at each week and couldn't find anything in the comments that looked at the models results. any other readers have any links?
You must not have looked very hard homer. There are a couple. You commented in the most recent one.
when two of your top five and five of your top 12 fail to make the playoffs ... you really have to introduce an explanatory variable
I think there are two explanatory points that near cover it all: (1) The NFL has rules that put a 7-9 team in the playoffs and keep a 10-6 team, the #5 one, out -- amid a general system that divides playoff spots between conferences, so if one is stronger than the other some of the top-12 teams league-wide will be out and some below-12 teams in, pretty much definitionally. Considering that, 7 of the top 9 being in seems OK to me. (2) This. In a season of only 16 games, it makes a far bigger difference in the W-L standings than most appreciate.
I think the original poster is absolutely right. There's a lot more explaining to do if Brian wants more credibility.
Does anyone here really believe the 4-12 Bengals are even close to being a better team than the 13-3 Falcons? Or that the 6-10 Texans are ten teams better than the 11-5 Saints? I don't see how randomness and/or divisional alignments explain that.
I've suggested this before, but if you really want the predictions to be independently verified, they should be submitted to thepredictiontracker.com; very few computer systems beat the Vegas lines each year and none exists that can do it consistently.
To Jim - consider a simple example of two people, Andy and Bob, rolling a die each. Andy gets a win if he rolls a 3+, Bob gets a win on a 4+. They each roll 16 times. There's around a 1 in 8 chance that the Bob will have a better record than Andy.
Back to the NFL, if you go the the week 8 efficiency rankings you can see that Brian was saying that the then 2-5 Chargers had a 0.81 GWP. Since then, they went 7-2. Did anyone else have San Diego to finish that strong after their start?
On looking again I don't really see anything that has to be explained regarding the rankings versus who made the playoffs. The rankings here correctly ID 9 of the top 9 teams as being playoff-deserving strength.
The Giants are out at 10-6 because NFL rules put Seattle in at 7-9. That's hardly on Brian. SD is the one big outlier, but everybody agrees it is -- FOers, PFR.com, they all agree it's a playoff strength team that's out because ... stuff happens. Credit it to Norv, special teams calamities, real bad luck, whatever. By DVOA, SRS, etc., SD should be in the playoffs. So that's not on Brian either -- that makes all the top 9 accounted for.
Then Seattle is in because of the NFL rules again -- does anyone argue the rankings above are wrong because Seattle's not in the top 12? :-) And that covers 10 of the 12 teams.
The remaining ones are down in the middle of the pack where teams are clumped close together, and the fact that the two conferences don't have an equal distribution of teams by strength forces breaks in the count -- it happens every year.
So what's to explain at all? Well, maybe just that Atlanta looks very low compared to its W-L, but it's played a weak schedule, and teams like them with big W-L numbers via a lot of close wins in the playoffs do very poorly. I thought earlier in the year that the rankings here had Mia way too high -- but the rankings proved right. As to Atlanta we shall see. For the rest, I really see no problem at all.
Simple Y/PP-Ranking;
"I" have 9 out of the best 12 teams in the Playoffs. It always works :-)
Y/PP-Difference Final-Rankings for PO-Teams:
2 GB +1,71
3 Pitt + 1,56
5 NE + 0,84
6 NO + 0,57
7 Indy + 0,55
9 Balt. + 0,50
10 Phil. + 0,36
11 NYJ + 0,24
12 KC + 0,17
------------
15 Chic. - 0,04
18 Atl. - 0,18
26 Sea. - 0,65
Yes, SD is No. 1. (+ 2,44), but Norv´s team fails in every model. Yes, ATL is ranked low, but that is also happening to every other model.
As always: Passing-Efficiency rules.
AFC-Final: NE vs. Pitt.
NFC-Final: GB vs. NO
Cheers, Karl from Germany
Re Jim:
While I agree in general, that Brians ratings do just fine, especially if you consider that he is taking a pain, to include only predictive stats, so its not surprising, that the other 50% of football determine soo much of the real standings.
Still I think it is a legit question, how a 13-3 team and a 11-5 team are considered below average by his ratings.
From my take I would say the answer to this one is probably opponent Adjustment:
Atlanta played 5 games against the bottom 4 of the league plus another 5 games against the rest of the NFCS and SF. An average team should go 7-3 against this schedule. The rest of their schedule were quite hard, but included some known self destructors like GB. So an 9-7 record could have ben expected with 10-6 not being a surprise. You can add another 2 wins due to excellent ST which Brian believes not to be predictive and you land only one game off.
OTOH Cin played NE, NYJ, PT, Bal, SD and IND plus the NFC S + Cle, Mia and BUF.
That should combine 6-10 for an slightly below average schedule. Another win lost to bad ST, although not as bad as ATL is good and you have 2 teams of otherwise equal strength going 5-11 vs 12-4.
NO is even easier to explain as they drew lucky and got DAL and MIN as their SOS opponents so they should be expected to go 10.5-5.5 as an slightly subaverage team so I would say Brian is almost dead on here.
While it is not the explaination of everything, people would be surprised how much ofa swing in W/L it can make during a single season.
Anon ,
I didn't see anything that linked to "beating the moneyline" 3/3 years.
This is something I found:
http://www.advancednflstats.com/2010/10/how-accurate-is-prediction-model.html
It shows that his percentages are actually pretty dead on.
The numbers (the facts) make the model. I don't make the model.
This site isn't here to try to buy credibility or approval with anyone.
There will always be teams that buck the general trends. Always. Learn it. Accept it. Love it.
I can easily, in 5 minutes, make a model with efficiency, turnovers, and penalties that capture every little up and down in a game and spit out rankings that accord to the W-L loss records of all 32 teams and every moron's intuitive gut 'power rankings'. That tells us absolutely nothing we don't already know. If you want rankings like that, there are about 1,000 other sites you can visit. Or, you can simply look at W-L records.
This model is open. There are step-by-step instructions for recreating it. All the data you need is freely available. I neither taut nor sell anything. I'm here to learn and share what I learn.
Let me be perfectly clear. DO NOT lecture me on credibility. I do not take kindly to implications that I lack it. If you can't understand the difference between prediction and explanation, that's your own deficiency.
This season, I count that the model went 130-78 in straight up wins, for a percentage of 62.5% (sorry Brian, but I counted the few toss-up 50/50 match ups as automatic losses for comparison reasons). According to thepredictiontracker.com the opening line went 169-86 for a percentage of 66.275.
The ANS model doesn't predict the first three weeks of the season which explains the total game difference (but im not sure why the opening line only predicted 255 games instead of 256).
Assuming that opening line predicted the first three weeks at the same rate as the rest of the season, the difference between ANS and the opening line represents 7.852 games. I think that difference is not to bad for a down year, but it is what it is.
Andy-Thanks.
To me, 8 games off is not too shabby. It's a miracle the model beat the lines at all in previous seasons. Keep in mind it does not know that Whitehurst is starting in place of Hasselbeck, or that Tony Romo was injured halfway through the season, or Antonio Gates is out, etc, etc.
"I know you get this a lot, but when two of your top five and five of your top 12 fail to make the playoffs, you have to explain why your system is a good one."
Two things to say about this.
Number 1, and most obviously, is that you're subjectively picking particular points in the rankings where you can make your point strongest. The obvious retorts in favor of the model are that "3 of the top 4 made the playoffs" and "7 of the top 9 made the playoffs." You haven't even come close to proving anything by doing what every lawyer in the world does - try and convince everyone by subjectively making your argument look better.
Number 2, and more importantly, is that arguments like this tend to come from people who have very little knowledge/understanding of statistics and probability, and more generally, randomness. I know that probably comes off sounding rude and insulting, but it's not meant that way. This is not to say that Brian's model is perfect - but to argue that it's wrong or that he needs to "explain why it's a good one" based on a few (expected) outliers is worthless.
I just don't understand why some people have such a difficult time applying basic concepts of probability, such as sampling error, to a game like football. We can do it with coins, dice, and cards, but some people seem to hit a wall when you apply it to a game that, although has elements of control, is for all intents and purposes functionally equivalent to dice, etc. (assuming both teams try to win).
No one seems to have any problem accepting the very real possibility of flipping, say, 13 heads in 16 tries. Unlikely, but try it enough and it'll happen. Go on youtube and watch Derren Brown flip heads 10 times in a row. Presumably, this took him hours and hours to do. But he did it. Similarly, if we roll a die 18 times, we expect to see three 6's. It's actually more likely we'll see a frequency other than three, so we shouldn't be surprised when there's error.
The only difficulty in applying this to football is that we don't know the odds, but surely odds DO exist. If you could simulate a game between Team A and Team B a hundred million times, we'd find that Team A wins x percent of the time and Team B wins 1-x percent. Play just 16 of these games, and the odds may differ wildly from X. When you have 32 teams, you're almost guaranteed some wild fluctuation from a team's true talent for a few teams.
Of course, the argument is that the odds predicted for Atlanta and a few other teams are incorrect. But unless you can point to something in the model that accounts for this rather than simple sampling error and normal bell-curve probability characteristics, I'm not sure the argument has any credibility. I don't have the numbers, but I imagine that the error from the model isn't far off from expected error.
"I know you get this a lot, but when two of your top five and five of your top 12 fail to make the playoffs, you have to explain why your system is a good one."
Two things to say about this.
Number 1, and most obviously, is that you're subjectively picking particular points in the rankings where you can make your point strongest. The obvious retorts in favor of the model are that "3 of the top 4 made the playoffs" and "7 of the top 9 made the playoffs." You haven't even come close to proving anything by doing what every lawyer in the world does - try and convince everyone by subjectively making your argument look better.
Number 2, and more importantly, is that arguments like this tend to come from people who have very little knowledge/understanding of statistics and probability, and more generally, randomness. I know that probably comes off sounding rude and insulting, but it's not meant that way. This is not to say that Brian's model is perfect - but to argue that it's wrong or that he needs to "explain why it's a good one" based on a few (expected) outliers is worthless.
I just don't understand why some people have such a difficult time applying basic concepts of probability, such as sampling error, to a game like football. We can do it with coins, dice, and cards, but some people seem to hit a wall when you apply it to a game that, although has elements of control, is for all intents and purposes functionally equivalent to dice, etc. (assuming both teams try to win).
No one seems to have any problem accepting the very real possibility of flipping, say, 13 heads in 16 tries. Unlikely, but try it enough and it'll happen. Go on youtube and watch Derren Brown flip heads 10 times in a row. Presumably, this took him hours and hours to do. But he did it. Similarly, if we roll a die 18 times, we expect to see three 6's. It's actually more likely we'll see a frequency other than three, so we shouldn't be surprised when there's error.
The only difficulty in applying this to football is that we don't know the odds, but surely odds DO exist. If you could simulate a game between Team A and Team B a hundred million times, we'd find that Team A wins x percent of the time and Team B wins 1-x percent. Play just 16 of these games, and the odds may differ wildly from X. When you have 32 teams, you're almost guaranteed some wild fluctuation from some teams' true talent.
Of course, the argument is that the odds predicted for Atlanta and a few other teams are incorrect. But unless you can point to something in the model that accounts for this rather than simple sampling error and normal bell-curve probability characteristics, I'm not sure the argument has any credibility. I don't have the numbers, but I imagine that the error from the model isn't far off from expected error.
Brian,
How do you come up with your offense fumble %? Is it total fumbles or fumbles lost?
"I know you get this a lot, but when two of your top five and five of your top 12 fail to make the playoffs, you have to explain why your system is a good one."
Two things to say about this.
Number 1, and most obviously, is that you're subjectively picking particular points in the rankings where you can make your point strongest. The obvious retorts in favor of the model are that "3 of the top 4 made the playoffs" and "7 of the top 9 made the playoffs." You haven't even come close to proving anything by doing what every lawyer in the world does - try and convince everyone by subjectively making your argument look better.
Number 2, and more importantly, is that arguments like this tend to come from people who have very little knowledge/understanding of statistics and probability, and more generally, randomness. I know that probably comes off sounding rude and insulting, but it's not meant that way. This is not to say that Brian's model is perfect - but to argue that it's wrong or that he needs to "explain why it's a good one" based on a few (expected) outliers is worthless.
I just don't understand why some people have such a difficult time applying basic concepts of probability, such as sampling error, to a game like football. We can do it with coins, dice, and cards, but some people seem to hit a wall when you apply it to a game that, although has elements of control, is for all intents and purposes functionally equivalent to dice, etc. (assuming both teams try to win).
No one seems to have any problem accepting the very real possibility of flipping, say, 13 heads in 16 tries. Unlikely, but try it enough and it'll happen. Go on youtube and watch Derren Brown flip heads 10 times in a row. Presumably, this took him hours and hours to do. But he did it. Similarly, if we roll a die 18 times, we expect to see three 6's. It's actually more likely we'll see a frequency other than three, so we shouldn't be surprised when there's error.
The only difficulty in applying this to football is that we don't know the odds, but surely odds DO exist. If you could simulate a game between Team A and Team B a hundred million times, we'd find that Team A wins x percent of the time and Team B wins 1-x percent. Play just 16 of these games, and the odds may differ wildly from X. When you have 32 teams, you're almost guaranteed some wild fluctuation from some teams' true talent.
Of course, the argument is that the odds predicted for Atlanta and a few other teams are incorrect. But unless you can point to something in the model that accounts for this rather than simple sampling error and normal bell-curve probability characteristics, I'm not sure the argument has any credibility. I don't have the numbers, but I imagine that the error from the model isn't far off from expected error.
Brian,
I understand you aren't selling anything.
I think some of my questioning comes from the fact that you seem to delight in poking fun at FO/DVOA/some of their disastrous predictions from past years. and that's fine and good and fair.
however, you ALSO publish weekly game predictions on the NYT, with wp%, so when you do this I think it's completely fair to ask "ok, is ANFLS better at predicting stuff than the other folks?"
If you are, awesome, good for you, and good for me as someone who is always looking for the best data.
With that being said, FO seems to be much more forthcoming about their results and their records over the last 3 years. it's there for me to see, both against the spread and straight up.
I'm just saying I would love to see this type of backwards looking study from you, particularly since you like pointing out when the other guys are wrong
"I know you get this a lot, but when two of your top five and five of your top 12 fail to make the playoffs, you have to explain why your system is a good one."
Two things to say about this.
Number 1, and most obviously, is that you're subjectively picking particular points in the rankings where you can make your point strongest. The obvious retorts in favor of the model are that "3 of the top 4 made the playoffs" and "7 of the top 9 made the playoffs." You haven't even come close to proving anything by doing what every lawyer in the world does - try and convince everyone by subjectively making your argument look better.
Number 2, and more importantly, is that arguments like this tend to come from people who have very little knowledge/understanding of statistics and probability, and more generally, randomness. I know that probably comes off sounding rude and insulting, but it's not meant that way. This is not to say that Brian's model is perfect - but to argue that it's wrong or that he needs to "explain why it's a good one" based on a few (expected) outliers is worthless.
(continued from above)
I just don't understand why some people have such a difficult time applying basic concepts of probability, such as sampling error, to a game like football. We can do it with coins, dice, and cards, but some people seem to hit a wall when you apply it to a game that, although has elements of control, is for all intents and purposes functionally equivalent to dice, etc. (assuming both teams try to win).
No one seems to have any problem accepting the very real possibility of flipping, say, 13 heads in 16 tries. Unlikely, but try it enough and it'll happen. Go on youtube and watch Derren Brown flip heads 10 times in a row. Presumably, this took him hours and hours to do. But he did it. Similarly, if we roll a die 18 times, we expect to see three 6's. It's actually more likely we'll see a frequency other than three, so we shouldn't be surprised when there's error.
The only difficulty in applying this to football is that we don't know the odds, but surely odds DO exist. If you could simulate a game between Team A and Team B a hundred million times, we'd find that Team A wins x percent of the time and Team B wins 1-x percent. Play just 16 of these games, and the odds may differ wildly from X. When you have 32 teams, you're almost guaranteed some wild fluctuation from some teams' true talent.
Of course, the argument is that the odds predicted for Atlanta and a few other teams are incorrect. But unless you can point to something in the model that accounts for this rather than simple sampling error and normal bell-curve probability characteristics, I'm not sure the argument has any credibility. I don't have the numbers, but I imagine that the error from the model isn't far off from expected error.
I find the notion that FO is "more forthcoming" laughable. FO is a for-profit business running a proprietary block box behind a pay wall. They are presenting *their* interpretation of results.
How did they count ties? How did they count 50/50 games? Which spreads are they comparing against, opening closing, which site's? Are they choosing the most favorable to them or least favorable to them. Or worse, are they picking which comparison on a case by case basis? Why do they only go back 3 years?
See the point...
FO counts pushes as pushes, I don't know what a 50/50 game is, they take a stance on every game.
they also list the spreads they are using at the time they make picks. I look at them every week, they are generally right in line with what is widely available, though there are indeed lots of sources for lines. either way they are there for you to see.
they only go back 3 years b/c that is when they started doing this, I think.
I still contest that this isn't a very accurate system, and Brian can do much better. The big problem may be cumulative rankings, and many aspects of NFL teams change greatly throughout the season. Examples: Romo hurt, NYJ's momentum after the NE loss, NE getting the NFL's best Guard back half way through the season, Eagles going from Kolb to Vick, etc.. Team play is not consistent one week to the next.
A few months ago, I compared this system to Vegas lines for the past three years and found that the accuracy was just slightly better than a coin flip. Only the most divergent games were used (Ex: Brian says the Redskins have a 52% chance of winning the game, Vegas has them at +6.5). The total of these extreme cases came out to be 81-72, a 53% overall rating, but that has gone down to very near 50/50 over the past few weeks.
Based on what has been posted on this great site, the prediction model could absolutely be improved, but Brian has to first realize that in its current state the model is not a good predictor, and go from there.
Here are some suggestions on variables for a model:
Take into account offensive line play vs opposing teams ability to break through, since this seem to be a very over looked aspect of NFL games and is really the foundation for every play in a game.
Weigh recent performance more than past performance. The Week 17 Cowboys or Eagles are much different teams than they were Week 1.
The real point of "is this model any good" is really asking if all of this analysis is any better than an extremely simple "model", such as 1) The Home Team Always Wins, or 2) 50/50, or my personal favourite 3) difference of defense's opposing qb rating to your offense's qb rating.
and also as significant, is there any year to year consistency in the model. if not, then we are just doing really complicated coin flips, and it means nothing.
@Chuck
Trying to adjust the model for position changes is a very difficult task, especially considering that empiricial data drives the model and backup players may have little to no data to work with. Furthermore, the model as it stands is about as objective as you can get. You either a) introduce extreme subjectivity and bias by adjusting for, as an example, a replacement quarterback, or b) detract from the quality of the sample size by not using past data because a backup QB is in.
It's unfortunate that that's the case, but it simply is. How do we know for sure how an offense plays without its star quarterback? Do the Patriots decline as much as the Seahawks do if Hoyer and Whitehurst play rather than Brady and Hasselbeck? It introduces a myriad of subjective variables that are nearly impossible to incorporate into an objectively driven model.
So what can we take from all this? The numbers and predictions the model creates are a great guideline to work with. Even taking into consideration that it doesn't factor in injury...it still outpaces many other models. Introduce subjectivity if you need to after the fact, but not before the numbers are spit out. Those numbers are based on a whole hell of a lot of sound theory and analysis, AND have a few years of good data to back it up.
And as for offensive line numbers being included...it's very difficult to track exactly what the offensive line is responsible for, both positively and negatively. There are a number of confounding factors. Fortunatley, much of how good an O-Line is is often captured with the offensive run and pass efficiency stats.
As for weighting recent games more heavily...I believe the model does. Not by as much as many other models, but it still does. There's a trade off...perhaps more recent games are more reflective of a team's true talent because of injury or other factors, but you run into sample size issues when you weight games less or discount them altogether.
Chuck, what is your definition of a good predictor? Can you cite some other models that have consistently performed better? Im sure you could find a bunch for just this year, but if you take the past 4 years into account....id be interested to see what there is.
Also, I read your comment that you mentioned back when you posted it and I was a little confused (and still am), are you saying that in games that have large odds differences Brian is actually beating vegas but by a statistically insignificant amount?
If thats the case i would say this: Brian isn't taking the "vegas model" and tweaking it to try and improve upon it. We have no idea what goes into a vegas line and how much of it is stats and how much is intuition. Brian has created a brand new model. If it comes close to vegas (which for all practical purposes is the best consistent predictor of wins) then i would be impressed. To have no idea what goes into a black box, but then be able to replicate the results is impressive to me. Brian never said you can make money on these picks, the point was to see what parts of the game are important, which we cant get from a vegas line.
The model didn't do GREAT this year, but didn't do bad. I estimated it was about 8 SUW behind opening vegas lines in a span of 208 games.
The vegas opening lines from 2003-2010 according to thepredictiontracker.com are:
67, 62, 69, 58, 67, 65, 67, 66 (i dropped the decimals). Seasons vary wildly, for Brians model to drop to 62 does not mean the model needs any changes, at least i dont think so.
It might be fun to test out some other things next year (run SR?) but I would want to see way more than 1 season to say that the model has actually stopped being predictive.
Just to check that I understand the calculation right (I've read around a bit in the site, and just want to check that I'm reading correctly): each efficiency stat (opass, orun, etc.) is given a weight based on a regression, then you enter each team's weighted stats into a logistic equation to come up with a win probability, then you adjust for schedule strength, factor that in, and you have your final number?
Is that about right? I remember you mentioned somewhere that opass was the single best predictor of future success, and I wonder if that has something to do with why San Diego ranks so highly in your system. They have a very good (almost absurdly good) passing number, but a very weak running number, and if passing is weighted more heavily than running, that would explain their high rank.
That isn't a critique, just trying to understand the situation. If passing efficiency is a historically good predictor of success, so be it. My only question is whether a team like San Diego, that is extremely biased toward efficiency in one area, is historically typical. It seems like most teams settle into an equilibrium where their run and pass games are sort of close to one another in efficiency, because opponents try to take away whichever thing the team is best at. So, historically, if a team had an absurdly high pass efficiency, that would indicate their run efficiency was not far behind, and that they had an absurdly great offense overall. San Diego could have bucked that trend, sort of like Indianapolis this year (and other years).
The variables are weighted according to their predictive ability (correlation to wins) as well as their correlation to themselves (i.e., how reproducible the variable is). There is a post a while back with all of the details.
Yes, part of the reason San Diego is ranked so high is their extremely high OPass efficiency. But their ranking at number 1 is still an unbiased product of all the other factors of the model. Sure, OPass is weighted heavily, but San Diego still outperformed everyone when all of the factors were considered with their relative weights.
San Diego did exceedingly well at the reproducible variables that are most highly correlated to winning. Unfortunately, there are a number of other variables that are notoriously difficult to predict (and largely not reproducible by teams) that San Diego was(insert "unfortunate," "unlucky," "cursed," or whatever you want here) to have done so poorly in - namely, special teams.
Sometimes, the cards just don't fall in your favor.
And to add a final comment to that...
Unfortunately, again, for the Chargers, things like unpredictable special teams and inopportune turnovers can carry more influence over a game than great efficiency stats.
the thing is, Special Teams aren't *JUST* variance and unpredictable. the Chicago Bears have great special teams, and this is a skill. it was true this year, and last year, and the year before that, and the year before that, and the year before that.
by making a decision to ignore them, right off the bat you are ignoring some important factors.
True, special teams is not entirely unpredictable, but it's signal-to-noise ratio is way higher than any of the other variables. If there is at least SOME correlation from year to year, I wonder why it's not included in the model with a very, very small weight. I'm sure Brian can answer that better than I can. My guess is that you reach a point where the addition of such low-correlated stats, even when given very little weight, may tend to detract predictive value from the model rather than add anything. But maybe I'm wrong about that.
More importantly, saying the Bears have had good special teams for a few years in a row (implying this proves special teams is predictive) is about as scientific as saying you flipped coins for a few minutes and got a few heads in a row.
"More importantly, saying the Bears have had good special teams for a few years in a row (implying this proves special teams is predictive) is about as scientific as saying you flipped coins for a few minutes and got a few heads in a row."
No, no, no, and this is the type of analogy that gives stats people a real bad name. there is a huge difference between 5 or 10 coin flips and 80 football games.
I'm a poker player and I understand the difference between results and process, and understand sample size.
there is a huge difference between 5 or 10 coin flips and 80 football games.
What team plays 80 football games in a season? If NFL games outcomes are 50% luck, how huge is the difference between 8 coin flips and 8 games, which is half the season?
16 games in a season * 5 elite seasons of ST = 80 games
SD is the outlier. And "everybody" insults Brian. But it´s not his fault. It´s Norv´s fault. Even with bad ST, Norv always found a way to loose. It was studied in one of the FO-Books (2007 as far as i remember) and in one of Brians studies.
So the only critic Brians model should get is, that he don´t bring a "Coach-Factor" into it. No more, no less.
Karl, Germany.
"Even with bad ST" should mean "without" of course.
Karl
how many other teams would you say have had 5 straight good seasons of ST?
the Colts have been below average at ST for as long as the Bears have been great
the Bengals and Redskins also have 4 years in a row of bad special teams. that's at least 4 teams with a clear pattern over multiple years and I've only looked at a handful.
By chance alone we should expect to see 1 team have 5 straight years of above average Special Teams. Again, we can go back to coin flipping. The odds of getting 5 straight heads are 1/32.
I know that there are probably 3 or 4 teams that have had above average special teams in the last 5 year span, but the fact that by chance alone we should expect 1 and not be surprised to see 1 or 2 more is very powerful. The fact that we probably see slightly above what's expected by chance is attributable to the small amount of reproducibility that you have with special teams.
"By chance alone we should expect to see 1 team have 5 straight years of above average Special Teams. Again, we can go back to coin flipping. The odds of getting 5 straight heads are 1/32."
unless you think Special Teams are 100% luck, then no you are wrong about this. a season is comprised of 16 games, I don't know why you are treating 1 season as 1 coin flip
and the Bears have not had 5 years in a row of above average ST. they have been ELITE 5 years in a row.
even assuming special teams is 100% luck:
% chance that a team is top 5, 5 years in a row =
0.0093%
How did you reach .0093? Just curious.
Probability can be deceiving. In a 100% "luck" league, what do you think the chances are that a team finishes in the top 5 twice in a row?
(5/32) ^ 5 = .0093%
and I did read the 50% article. to begin with it's overstated for some of the reasons Brian later mentioned in the comments
I know that probabilities can be deceiving. I also know that handwaving away skill based results in the name of variance is a large failing among many sabr types. even the most advanced baseball projection systems consistently under project Jeter's AVG, etc
Special teams outcomes account for 12% of the win probability 'movement' in games. So even if we had perfect clairvoyance in every game about which kicks would be missed, made or be returned how far, and when, and in what game situation, we'd only have 12% of the answer.
And even if, after the fact, we can see that some special teams were better than others, that's a far, far cry from the clairvoyance we'd need to get to 12%.
Put another way: Even if prior ST performance correlated perfectly with future ST performance, which it absolutely does not, it still only impacts a small part of game outcomes.
Further, ST are not one single factor. There are 6 distinct skills involved, punt, kick, and FG offense and defense. Each one is susceptible to random (sample) error in different ways.
Of course, there are exceptions. There always are. This year the Chargers are a good candidate as a team truly bad on ST in several of the 6 departments. But that doesn't mean the model should be skewed to account for the one or two possible exceptions. That's exactly the wrong way to do it. If you think it is, you get an F.
Incidentally, the Chargers, which had the undisputed worst ST this year were favored by Vegas in 15 of their 16 games this season, including vs. NE. My model favored them in all of the games predicted (weeks 4 through 17). Special teams is a very, very small part of the reason the model didn't out-predict Vegas this season.
And lastly, as Andy pointed out above (independently I might add), even if you count every toss-up as a loss for the model, it comes within 8 games of Vegas. How many toss ups were there, 4, 5 maybe? I'm not sure, but there were a few. Throw them out as any fair arbiter would, and the model is within 4 games of Vegas.
That's *4* or so games behind a system that knows which QBs are injured, which teams have nothing to play for in week 17, which teams are coming off byes, which ST are "good," and so on. And that's for the model's very worst year, its only one behind Vegas out of 4 seasons.
@ anon
~57% that any team ends up in top 5 two years in a row
I actually think the model is a pretty good one. I still would like to see STs even at a very small weight. But, overall the model does what it says it does, generic win prob and team efficiency. The Bears point does bring up an interesting idea. Perhaps the coefficients can be adjusted for each team. If a team’s offense depends heavily on run efficiency to set up the pass versus the average team, then the coefficient for run eff would increase in the model. I understand small 10 game sample size may cause the coefficients to be regression to average, but I don't think it has even been looked at.
makewayhomer
Could you give us an actual stat to measure special teams by please? This is 'Advanced NFL Stats' not 'Rambling NFL Conjecture'. If the Bears ST are so good, and the Colts so bad, there must be some numbers we can use to back that up with. You can't include "well I think they're pretty good" in a prediction model.
DVOA Special teams
makewayhomer
Ok, as we're doing this from a stats point of view I took a look at DVOA ST. To start with, I agree that ST DVOA does correlate with winning, although you would expect that seeing as it's essentially a modified EPA.
To etst the predictivity of it, I've looked at the week 6 DVOA stats (they were the first that came up in a search) versus the final ones. Week 6 ST DVOA correlates with Final ST DVOA with an r of 0.6. Sounds good, but remember that the final stats also include those first 6 weeks.
To compare, if you take the average of 6 random numbers and compare it against the average of those same 6 random numbers added to another 10 (to make your 16 game season), they also correlate at 0.6. So the overall predictivity of ST DVOA is no better than random numbers.
It's a good stat for measuring special teams play, but I don't see anything in it that tells me how future special teams will go.
Lots of good points above. For one I at least think Brian does a great job of explaining a lot of the things in the model and think the model is awesome.
It would be nice to see a weighted ( most recent 8 game?) probability even if it doesn't improve the model.
Some things that MIGHT improve the model or or at least worth looking at.
1. A weather factor. This old pro-football-reference post had some interesting data:
http://www.pro-football-reference.com/blog/?p=157
2. Aren't some aspects of special teams repeatable? At least within the season? For example I thought footballoutsiders said that kickoff distance was a pretty repeatable skill? Or punt distance given the average field position?
3. Using offensive and defensive rushing success rather than rushing efficiency.
Brian, I guess you already sort of looked at special teams way back here:
http://www.advancednflstats.com/2007/08/importance-of-special-teams.html
I still think average kick coverage/distance is worth revisiting along with my other two points.
Does anyone here really believe the 4-12 Bengals are even close to being a better team than the 13-3 Falcons? [etc...] I don't see how randomness and/or divisional alignments explain that.
For context on how much W-L record and playoff seeding vary from true team strength on average, *and* at not so uncommon extremes, take a look at the series of posts PFR.com did on 10,000 simulated seasons. "the best team in football missed the playoffs 10.47% of the time..." (And, yes, the #32 team can win it all.)
Quickly glancing at NFL.com the Bears kickoff and punting teams seemed very average this year
Bears kicking and punting teams do seem about average. however, each of their return teams are elite. said another way, Devin Hester
A lot of debate about the Falcons, but they are actually not the biggest statistical anomaly in this bunch. At 0.47, their WP is very close to 50%; as such, they have a much higher standard deviation than the San Diego Chargers:
Atlanta expected/actual wins: 7.36/13
Difference of 5.64 wins. St. dev = 1.99
Atlanta is 2.83 standard deviations away from the mean
San Diego expected/actual wins: 12.96/8
Difference of 4.96 wins. St. dev = 1.57
San Diego is 3.16 standard deviations from the mean.
While the sum of Atlanta's good luck outweighs the sum of San Diego's bad luck, San Diego's run is somewhat less probable. Interesting how many people have pointed to the Atlanta number as evidence that the model is off, when in fact the San Diego number would be the easier of the two to attack.
Re: Chicago Bears
I think the best way to prove that their special teams are truly above average is to point towards opposing teams' punts. Rumor has is that teams always kick away from Devin Hester; ergo, the punter has less leeway to boom a long kick. Something like that could cost the punting team a few yards' worth of field position per kick.
Intuitively, I'd say they have an advantage. But that's just my guy speaking, and I'm a marathon runner, so I don't have much of a gut to rely upon.
Seriously, people. Predictive vs. Explanatory. It's not that complicated. Look at the lines all season long for SD and ATL. I, for one, have not (hehe), but I'll bet they aren't much different from Brian's predictions.
Special teams are incredibly explanatory. So are win-loss records (!). But it's been shown over and over by Brian and others that gutting out close wins (ATL) and getting consistently dominated on special teams (SD) don't imply future success.
So no, Brian won't change his model to account for special teams. Geez, you people and your demands.
Also, I would be extremely interested in seeing any football pundits or media types that predict ATL winning the super bowl. If anyone has links, put them up.
You'd think the team with the 2nd best record (and pretty consistently ranked 2nd in "power rankings") would have at least a few believers. But I'm imagining many more pundits are picking pittsburgh or green bay over ATL. And if SD was in the playoffs, you'd better believe those hypocritical pundits would rank ATL 2nd and SD 15th, while AT THE SAME TIME saying "Watch out for SD! They're underrated!!!"
I agree with some of the other posters on the potential importance of teams changing over a season.
I'd really like to see an article based on how predictive a team's performance over the first few weeks are on the team's performance over the last few weeks or playoffs.
I'd also like to see an article exploring how opponent strength adjustments on team statistics are made/could be improved.
@Adam. the question isn't whether Atlanta will win the super bowl, it's whether they are one of the top 5-10 teams in the NFL or are they 21st best, as ranked here.
I find it interesting that many of the teams "overrated" here made the playoffs last year (dal, min, sd, cin). Makes me wonder if something in the model isn't adapting quickly enough to changes in team strength.
@ Adam, not quite
"But I'm imagining many more pundits are picking pittsburgh or green bay over ATL."
current odds to win SB on pinnacle:
WILL THE PATRIOTS WIN THE SUPER BOWL?
4103 Yes 178
WILL THE STEELERS WIN THE SUPER BOWL?
4107 Yes 575
WILL THE FALCONS WIN THE SUPER BOWL?
4119 Yes 590
WILL THE SAINTS WIN THE SUPER BOWL?
4125 Yes 850
WILL THE BEARS WIN THE SUPER BOWL?
4127 Yes 1000
WILL THE EAGLES WIN THE SUPER BOWL?
4121 Yes 1179
WILL THE RAVENS WIN THE SUPER BOWL?
4109 Yes 1600
WILL THE PACKERS WIN THE SUPER BOWL?
4129 Yes 1640
WILL THE COLTS WIN THE SUPER BOWL?
4115 Yes 2213
WILL THE JETS WIN THE SUPER BOWL?
4105 Yes 2850
WILL THE CHIEFS WIN THE SUPER BOWL?
4113 Yes 6300
WILL THE SEAHAWKS WIN THE SUPER BOWL?
4133 Yes 20968
also @ Adam
at least some parts of Special Teams are both explanatory and predictable. Brian certainly agrees with the former, as his Billy Cundiff post says, and other studies have already shown that certain factors like kickoff distance are predictable.
I'm A little shocked how low Green Bay's Defensive rank is. Their D has been better IMO than their O.
@makewayhomer
actually, the odds of of any one team finishing in the top 5 in ST for 5 years in a row in a 100% "luck" league is closer to .00218%
Even this number is probably slightly off, but it's much closer to the real probability than (5/32)^5, which is actually .000093, not .0093.
All you need to do is calculate the odds that at least 1 of the top 5 teams from any season are in the top 5 for the next 4 seasons. So the odds for season 1 is 100%...you will always have 5 teams in the top 5. The odds for at least 1 of those teams in the top 5 next season is roughly 57%, which is 1-(27/32)^5
Then to eliminate the difficulty of calculation, assume only 1 team has made it in the top 5 for two seasons. Now you can just multiply the (non-rounded) 57% by (5/32)^3, which gets you roughly .00218
The actual odds are probably slightly higher since I eliminated the possibility of having 2 or more of the 5 teams in the top 5 in year 2 and thus thereafter. Anyone care to take a shot at the full calculation?
Orrrr you can take the (5/32)^5 caluclation (which is the odds one specific team will do it throughout 5 years), and subtract that from 1 for the odds a team WON'T do it. In other words, 1-.000093....
Then, raise that to the 32nd power since there are 32 teams. Then, subtract that from 1 for the odds that a team DOES do it 5 years in a row. I end up with .0029759, fairly close to the .00218 figure
Anon - I think it's actually around 0.3%. I ran a large simulation (100,000 sims) and the answer came to around that. It also makes sense because the probability of any one team being in the top 5 five seasons in a row is 0.0093%, so the probability of at least one of the 32 teams being in the top 5 five seasons in a row is 32 * (5/32)^5 = 0.298%
I think you mean .00298? In which case it's the number I arrived at in the second post.
Other Anon re: Green Bay. It could still be, it's only the ranks. Nothing saying they're on the same scale, so a 6th placed defense could well be better than a 4th placed offense. After all, the PFR SRS has the Packers with the best Defensive SRS.
Yes, indeed - 0.298% = 0.00298 (I did realise after I posted that you'd got to the same number first).
ah, sorry, missed the percent sign in your post!
:) no problem, it was actually lazy stats from me. Proper statisticians report probabilities from 0 to 1 rather than 0% to 100%.
Just curious, what program did you use to run the simulation?
Excel. I just set up a 32 x 5 grid (to represent each team and season), filled that with randoms and checked to see whether any of them were in the top 5 five seasons in a row. Then it's just a case of writing a macro that recalcualtes random numbers and totals the number of 'successes'.
I just ran a simulation of 50,000 seasons - 145 times a team was top 5 for five years in a row, or .0029
And just as a fun excercise in variance, Team "19" was WAY better than Teams "4", "10", and "13" in my simulation. In fact, they're 11 times better since they were in the top 5 for five straight years a whopping ELEVEN times, and the other three were only there once.
80+ comments is too exhausting so this a general post.
Considering that 7 of the top 10 teams are in the playoffs, that randomness and luck is rampant in the NFL, and that points, wins, media attention etc. have nothing to do with this rating system, it's pretty f'n good.
Whoops, it also looked like you already tried running a success rate regression and it was worse than the efficiency model.
I need to improve my post searching before making recommendations.
http://sports.espn.go.com/nfl/playoffs/2010/news/story?id=5988263
It's not a Super Bowl prediction but it's perhaps even more relevant to the quality of the teams and these rankings. When Scouts Inc. rated all the playoff teams on each position the Falcons came out as the best overall.
Of course I realize this is one exercise that is to some extent pointless, but it does raise a point I wanted to make. It's not like the Falcons are a team that you look at on the field and wonder how they're winning. If this was the 2008 Falcons that came out of nowhere and had several glaring holes then the #21 ranking would make more sense, but it's not. This is a team that many were picking before the season to win the division over the Super Bowl champion, and their 13-3 record comes as a shock to noone other than some commenters on this website.
All that said, the model seems to work very well overall. Brian, you record against Vegas shows that on the whole the model holds strong predictive value. I want to say though that the model can be good overall but still be very wrong about some teams. Brian has had a post and I have talked at length in other comments about reasons Atlanta might be underrated through this model. I suspect San Diego has underperformed for years because of the "luck" elements that could also be called "coaching". What's true for most as far as predictive value might not be true for all.
Some of the people on here, and I am only talking to some, should stop trying to defend the model by saying statistical anomalies happen and start recognizing that it doesn't have to be perfect to be good. It just might be wrong about the Chargers and Falcons.
Also, Adam, you weren't lying when you said you haven't looked at the lines all season long for the Falcons. They have actually been favored in 14 of their 16 games.
I don't recall anyone saying the model is perfect. Anyone who does think that is clearly deluded. It's probably true that Atlanta is better than the 21st most efficient team in the NFL. It's also probably true (and much more likely) that Atlanta is not the 2nd best team in the NFL, as their record would indicate.
That being said, the fact is that Atlanta has been involved in a lot of close games this year. As great as it sounds to talk about a team with a knack for "winning" close games, there's no empirical or scientific basis that conclusively proves teams are particularly good at winning close games. All the historical evidence has pointed to the fact that teams who win close games were fortunate to come out on top, and should be expected to regress to the mean. If one or two highly variable, highly unpredictable, highly fortunate things didn't happen for the Falcons, then you're looking at an 11-5 record, Saints win the division, and almost NO ONE is talking about the Falcons being the best team in the NFC.
Look, they're a good team. They're probably top 6 in the NFC. They could easily be 11-5 or 10-6 though.
And as for statistical anomalies, it would be more of an anomaly if there were NO statistical anomalies at the end of the season. Expect the unexpected.
David-Very interesting. I'm going to link to this in the Roundup. Thanks.
But, the overall talent ranking seems to follow team W-L records too closely for my taste. A bit too 'ex-post-facto,' in other words, which is what I think you were getting at yourself.
Just for the sake of debate, imagine NO had not missed that OT chip shot FG against ATL early in the season. Theoretically, all else being equal, this does not change ATL's performance one bit. There was nothing they did to make that FG go wide. ATL would now be a wildcard behind NO, and all the pundits and experts would have them 'power-ranked' much lower...all for absolutely no other reason than outcome bias. Instead of being thought of as #2 in the league by everyone, they'd be ranked 6th or 8th or possibly lower.
And in defense of those of us who would doubt Atlanta's top ranking, I'd say that it's not surprising for a perfectly average team to end up 13-3, particularly if they don't have a tough strength of schedule. Statistical anomalies do happen, and I'll make a very strong case below that ATL qualifies as one.
No one here (at least me) is saying ATL is the 21st best team in the league. They are, however, apparently the 21st most efficient team accounting for opponent. And that suggests their 13-3 record is a reflection of quite a bit of good fortune.
I'll give everyone 5 numbers. Ready? 5.9, 3.8, 6.1, and 4.6. That's their offensive passing efficiency, offensive rushing eff, defensive pass eff, and defensive rushing eff. Only one of those three is better than average, and only by a hair. (Until they just played horrid CAR in week 17, they were below average in def pass eff all year.) The fifth number is 24. That's their rank in terms of strength of schedule. They are below average in (virtually) all four core efficiency stats against a weak slate of opponents.
It's no mystery (to me) how ATL is winning. #1 really good turnover differential, which is quite random (according to research by many others), and only partially predictive. #2 is clutch QB play, where Ryan has over-performed his overall average play when he happens to be in very high-leverage situations. #3 Very, fortunate penalty calls against opponents in critical situations, one miraculous fumble recovery off an interception, and a missed chip shot FG. #4 And lastly, a slightly weak strength of schedule.
So the defense of ATL's low rankings is not to shrug our shoulders and say "Sometimes anomalies happen." The defense is, "Anomalies do happen. And there are very, very good reasons to believe ATL is one of them this season."
I agree with you completely that they could easily be 11-5 or 10-6. Obviously that's true.
However, it's not "probably" true that Atlanta is better than the 21st best team in the NFL. It's definitely true. The close games thing isn't an argument against that. The Falcons have blown out plenty of teams.
Did you go to the link I posted? The Falcons have a great football team. Just ask any football analyst, coach, scout, player, or fan in the world who doesn't use this site.
I'm fine if you say the answer probably lies in the middle, as most of your post did say, but everybody needs to stop pretending it might be true that the Bills and Bengals are better than the Falcons.
Excluding teams ranked 25th or lower in efficiency...they didn't win a game by more than 7 points.
Brian, if you added your subjective judgment to the wholly statistical efficiency rankings, where would you rank the Falcons?
So what? This model wouldn't expect them to beat the teams ranked 25th or lower in efficiency by more than 7 points. It also would expect them to get blown out by Green Bay, Pittsburgh, and Baltimore.
"It would expect them to get blown out by Green Bay, Pittsburgh, and Baltimore."
Say it ain't so, David, say it ain't so.
In preparation for fantasy, I had the bright idea of plotting each of the top RB's weekly points against the defensive efficiency stat for the defense they faced that week. I did this for all of 2010 in the hope that if the relationship was strong, it could lead to a more effective matchup strategy in which I focused more on getting the best decent back against the worst run defense efficiency stat I could. Unfortunately, the R^2 was only 0.066, so it wont work. First, does anyone have any ideas (beyond "get a life nerd")? Second, I feel a little dejected, but possibly a similar analysis with QBs will be more fruitful. I'll let you know. I think its just more fuel to the whole "its all luck anyway dude" argument.