Team Efficiency Rankings - Week 14

San Diego's strange season continues, but they lose their #1 spots on offense and defense. NE now owns the offensive honors while Pittsburgh owns the defensive title.

The team rankings below are in terms of generic win probability. The GWP is the probability a team would beat the league average team at a neutral site. Each team's opponent's average GWP is also listed, which can be considered to-date strength of schedule, and all ratings include adjustments for opponent strength.

Offensive rank (ORANK) is offensive generic win probability, which is based on each team's offensive efficiency stats only. In other words, it's the team's GWP assuming it had a league-average defense. DRANK is is a team's generic win probability rank assuming it had a league-average offense.

GWP is based on a logistic regression model applied to current team stats. The model includes offensive and defensive passing and running efficiency, offensive turnover rates, defensive interception rates, and team penalty rates. If you're scratching your head wondering why a team is ranked where it is, just scroll down to the second table to see the stats of all 32 teams.

Click on the table headers to sort.


RANKTEAMLAST WKGWPOpp GWPO RANKD RANK
1 SD10.800.4322
2 GB20.750.5138
3 PIT30.720.5191
4 NYG40.710.4786
5 NE70.690.56123
6 PHI60.660.51511
7 MIA50.650.56149
8 BAL90.630.511110
9 IND130.600.55187
10 MIN120.590.541612
11 KC100.580.441213
12 NYJ80.570.55245
13 CHI140.560.50274
14 HOU150.560.56430
15 NO160.550.40618
16 TEN110.540.54263
17 DAL170.530.56726
18 CLE180.480.512016
19 BUF190.450.572520
20 ATL200.430.481324
21 WAS210.430.561727
22 TB230.390.411525
23 CIN220.380.532921
24 DEN240.380.501032
25 SF250.370.422217
26 JAC270.360.571931
27 OAK260.360.512115
28 DET280.330.562822
29 SEA290.310.432328
30 STL300.240.363019
31 CAR310.230.463214
32 ARI320.180.433129

And here are each team's efficiency stats.

TEAMOPASSORUNOINT%OFUM%DPASSDRUNDINT%PENRATE
ARI4.94.33.91.06.84.42.90.46
ATL6.14.01.60.16.74.33.80.34
BAL6.63.62.00.85.74.02.70.36
BUF5.74.43.21.46.44.72.20.32
CAR4.54.04.72.46.04.13.90.43
CHI6.03.94.30.25.53.73.60.40
CIN5.73.73.21.36.54.63.50.35
CLE6.04.03.11.46.44.14.40.38
DAL6.93.93.40.57.14.23.20.44
DEN6.63.81.31.47.34.31.60.46
DET5.73.72.61.06.64.62.40.54
GB7.24.02.20.55.54.53.90.35
HOU6.84.82.00.57.63.92.30.33
IND6.73.52.80.66.04.82.40.33
JAC6.04.64.60.97.74.32.90.33
KC6.44.91.10.65.94.22.00.38
MIA6.33.84.11.35.93.72.40.28
MIN6.34.55.50.76.23.62.60.37
NE7.24.21.30.26.74.33.80.36
NO6.94.13.30.85.84.11.80.41
NYG7.04.74.21.65.54.03.40.41
NYJ6.04.52.81.45.93.41.70.52
OAK5.74.73.61.16.24.41.90.64
PHI6.85.21.40.86.14.14.90.60
PIT6.84.02.61.25.73.03.00.48
SD7.84.02.41.45.43.73.40.38
SF6.04.23.40.96.43.62.30.50
SEA6.03.63.40.46.64.22.20.42
STL5.43.72.30.25.94.52.30.45
TB6.04.41.60.76.14.64.70.46
TEN6.14.33.61.05.84.13.30.53
WAS6.14.13.41.16.65.02.40.34
Avg6.34.23.00.96.34.22.90.42

  • Spread The Love
  • Digg This Post
  • Tweet This Post
  • Stumble This Post
  • Submit This Post To Delicious
  • Submit This Post To Reddit
  • Submit This Post To Mixx

55 Responses to “Team Efficiency Rankings - Week 14”

  1. Mark White says:

    This is the most screwed up model I have seen. San Diego Chargers # 1? Miami and Minnesota in the top 10? The Falcons have the best record in the NFL and they are #20! Your model needs plenty of work. Does it take into account the abilty to patch together long drives or 3rd down conversion %? I love WPA analysis because it shows how well teams and players perform when the game is on the line. Does this model take WPA into account. I would much rather have Matt Ryan leading my team when we are down in the fourth quarter than Rivers, Henne or Farve. I don't think anyone in their right mind would say the Miami Dolphins have a 70% chance of beating the Atlanta Falcons at a neutral site. Good models should always take into account what is actually observed. Sorry but you can do better.

  2. Anonymous says:

    "I love WPA analysis because it shows how well teams and players perform when the game is on the line."

    This implies that you believe there is a repeatable ability, and statistically significant effect, that some players perform better during "clutch" situations than they do in others. I think you'll be hard-pressed to find data that corroborates this. I'll take Rivers over Ryan down 1 score late in the 4th any day.

    While WPA analysis is very interesting, it tells you less about the quality of a player than EPA. WPA is sensitive to late game heroics, and while late game heroics correlate with being a good player, there is a lot of "error" due to small sample size.

    I'd rather have Rivers' 7.8 YPA rather than Ryan's 6.1 YPA in the 4th quarter. Maybe it's a consistency issue? I'd still rather have Rivers' 49% success rate than Ryan's 48%.

    Just because Ryan's KQ beat Rivers' AJ on the river (no pun intended?) doesn't mean I'm going to prefer the KQ in the future.

  3. mustardboy3141 says:

    Is Min doing better this year in terms of your model yet worse in terms of W-L record?

  4. dfan says:

    Also note that adding up the WPA of a team's members is equivalent to just using its W-L record.

  5. James W says:

    The ability to play in the clutch may well be repeatable if there are a set of skills specific to those situations. We make the allusion to clutch being pressure-related performance but it may just be highly situational.
    To my mind, it is unlikely there exists the skill of clutch hitting in baseball because the interaction between batter and pitcher is little different to other situations.
    But there could be such a thing as clutch quarterback play, because endgame strategy in football is demonstrably different to other game-states, both from an offensive and defensive perspective.

  6. Anonymous says:

    First sentence:

    "San Diego's strange season continues, but they *lose their #1 spots on offense and defense."

  7. Anonymous says:

    i like to think they loosened their grip on them...

  8. Ty Will says:

    Brian,

    I have a foolproof suggestion to improve your "screwed up" system. First, begin by subjectively ranking the teams and their units according to what you believe would square with popular opinion. THEN, work backward with your formula and gerrymander the numbers so they retroactively endorse the result you wanted! Huh, huh? That would be so much... ah, less... interesting.

    Ty

  9. Anonymous says:

    @ James W.

    I understand there is a situational component that needs to be considered, and that you may see "clutch" performance come out in the form of the context the player is put in. For example, you actually do tend to see very powerful lefty hitters in baseball do worse in LIPS (late-inning pressure situations). This is almost entirely attributable to the fact that opposing managers bring in a lefty reliever to face the hitter in these exact situations. The tendency for lefty pitchers to do better against lefty hitters and for lefty hitters to do worse against lefty pitchers creates a sort of anti-clutch performance - like you said, because of situation.

    I'd be cautious in extending this to the present example of Rivers v. Ryan, or more generally Chargers v. Falcons. Your goal is simply to move the ball as far down the field as you can and usually as quickly as possible. Two QB stats that are very important/predictive of this success are YPA and SR. Why would you ever choose the quarterback that is inferior in both of these categories? (Not meant to be a shot at Ryan...he just simply isn't as efficcient as Rivers)

    I know your answer was that it's perhaps due to situational factors - that Ryan is in fact better than Rivers when it comes down to one last drive to either tie or put your team ahead, for example. But don't you think random variation around a true mean for each quarterback is a more likely explanation? That is, Ryan has simply been "fortunate" enough to do better than his average when it counted most? And Rivers, perhaps not? It's very normal to see a gaussian ditribution of players over/underperforming their "norms" given a specific situation. A quarterback who overperforms as well as Ryan one year is almost certain to regress to the mean. His regression to the mean would almost certainly mean less wins and more losses for the Falcons. I bet you could find that some quarterbacks are performing way above their average in the first quarter, and others well below their average...is it really due to the situation of the first quarter?

    Unless you can provide meaningful data that Ryan thrives in a last possession drive because of situational factors (perhaps the defense always drops their safeties, always plays zone, etc.) and that he thrives in these same situations outside of the last possession drive...I think variance is a better explanation.

  10. Mark White says:

    "This implies that you believe there is a repeatable ability, and statistically significant effect, that some players perform better during "clutch" situations than they do in others. I think you'll be hard-pressed to find data that corroborates this." I understand the whole concept of clutch shooting and how the "hot hand" has been been proven to be a myth. That applies more to streaks as opposed to how some perform under pressure.

    I think you can show in statistically significant way that some players perform better when the game is on the line versus others that have a tendency to "choke". If WPA is due more to "error" then you would expect different WPA leaders every year.

    The ability to run a two minute drill when the game is on the line is very different than a QB running a one dimensional passing offense that has impressive QB numbers but can not produce wins. John Elway (79.9 QB rating) had an average passer rating but produced wins in the clutch. Elway had 46 GWD over 231 GS (20 %). He was a guy you could depend on when the game was on the line.

    I think Matty Ice will be known as a clutch QB that you want when the game is on the line. WPA is an awesome stat that separates the winning QB's that don't choke from the losers. It should always be a part of Team Efficiency Ratings if the team is lead by clutch players that get the job done when it counts the most. And yes, I would rather have Ryan and his 13 GWD/ 42 GS (31%) over Rivers 15 GWD/76 GS (19%).

  11. Anonymous says:

    You should expect the WPA leaders to correlate heavily with players that have the highest EPA on the teams that have the most wins. But it's just that - a correlation. I expect you're more likely to see high EPA players from quality teams towards the top of WPA leaderboards each year. If there is a player that way outperforms his average - that is, has a very high ratio of WPA/EPA relative to others - you can expect him to regress to the mean, and thus drop from the leaderboard.

    Brian linked to an excellent article discussing John Elway and GWD's in one of his previous posts.

    Last year's stats for Rivers and Ryan:

    Rivers: 5.83 WPA 181.3 EPA .032 WPA/EPA
    Ryan: 2.77 WPA 78.8 EPA .035 WPA/EPA

    Rivers stats include his one playoff game as well, since I can't seem to access the regular season WPA right now, only a grand total of season + playoffs.

  12. Ian Simcox says:

    @Mark White - I'd rather have a QB who put me so far ahead early in the game that he never needed to lead a last minute game-winning drive, rather than one who needed a 4th quarter kickoff return TD to even give himself a chance to win the game, but maybe that's just me :)

  13. Alex says:

    I took a quick look at the WPA/clutch issue if anyone is curious: http://sportskeptic.wordpress.com/2010/11/27/quarterbacks-are-still-inconsistent/ . WPA correlated worst from season to season out of WPA, EPA, and success rate for quarterbacks in the past few years. Even if it had correlated well, I don't know how you'd attribute it to the quarterback as opposed to his line or wide receivers, which generally overlap quite a bit from year to year.

  14. johnnyjohnnywu says:

    Back to the discussion about the Atlanta Falcons. From the table above, we see that the league avg OPASS and DPASS is 6.3 and the league avg ORUN AND DRUN is 4.2.

    Falcons are BELOW AVERAGE in all categories:
    OPASS = 6.1
    ORUN = 4.0
    DPASS = 6.7
    DRUN = 4.3

    Are these stats suggesting that Falcons are waaaay overrated? As of December 07, both ESPN and CBS Sports rank Atlanta Falcons as the #1 team. Does these analyses suggest that so-called experts should value empirical statistics more so than just relying on subjective 'expert' opinion?

  15. Brian Burke says:

    I would say that this system underrates them, but everyone else is overrating them. The truth is likely somewhere in between. Keep in mind they've been feasting on a weak NFC West and their own division isn't the strongest.

  16. Dave B says:

    Atlanta has been at least a little "lucky" luck if you b They are +10 in turnover margin (9 of that plus 10 is from the interception dept).

    Their penalty rate is low which is also helpful.

    Also another giant luck aspect is that opponents have only made 11 of 19 FG's vs the Falcons for a rate of 57.9%. Not sure what the average distance is but I doubt its enough to justify that rate.

    I'm not sure why their opponent game winning probability is so low since pretty much every other analytical measure out there has their schedule as around average or harder than average. Maybe Brian can explain?

  17. Dave B says:

    Sorry I take that back...the simple rating method on pro-football reference has ATL's schedule right around where Brian's puts them.

  18. Jim Glass says:

    I think you can show in statistically significant way that some players perform better when the game is on the line versus others that have a tendency to "choke".

    If you can, do it and make a name for yourself, because nobody else has been able to do it.

    The first problem is *defining* "clutch, when the game is on the line." E.g., Joe Montana threw that famously clutch pass for that come-from-behind win in the NFC championship game. But before then he threw three picks to put his team behind. Was the game not on the line when he threw the three picks? What part of a close Championship Game is not clutch?

  19. Probable Picks says:

    BB: "I would say that this system underrates them, but everyone else is overrating them."

    The Falcons are 8-4 against the spread. Seems like the market has been underrating them too, although I suppose with a sample size that small, it's possible they are just lucky.

  20. Anonymous says:

    I think certain players MAY be more "clutch" only if you define clutch as "not choking."

    By that I mean maybe certain players maintain their standard level of performance in pressure situations more often than the league average. Or perhaps there are really anti-clutch guys that consistently do worse than expected in pressure situations.

  21. Anonymous says:

    football outsiders did a study of comeback drives and came up with a stat called "ACE" or adjusted comeback efficiency.

  22. Brian Burke says:

    One of the first things we'd need to do to measure clutch would be to make a good leverage index for football. LI identifies high leverage or 'clutch' situations.

    One other approach is to measure the ratio between a player's EPA (performance ignoring time & score) and WPA (which accounts for time & score). Because EPA and WPA are zero-neutral stats, we'd also need to establish a replacement level for each stat.

    WPA per EPA might be a good measure for clutch.

    But I don't know if it would tell us much more than we already know. I would bet the mortgage that we wouldn't find any statistically significant talent for clutch play (over and above baseline performance levels) that persists in players.

  23. Jim Glass says:

    I think certain players MAY be more "clutch" only if you define clutch as "not choking."

    This seems plausible to me -- more plausible than some players magically improving their game at key moments. James says that in baseball there is no evidence of any managers/teams being better than any others at winning close games, but there may be some evidence of a few being more adept at losing them.

    IMHO, one of the reasons for the popular belief in clutch play is that among us ordinary people it is true, we experience it. In amateur leagues when the big game comes some people will concentrate harder and play better, and others will have their nerves do them in. But the pros aren't like us. They have to play their 100% best all the time or they get cut -- "raising one's game in the clutch" indicates one plays less than his best the rest of the time -- and any who don't learn to master their nerves are culled from the pool.

    Netting it out, logically I just don't see how anyone can play better than his best at selected moments, but can see how maybe some new players getting a chance might "clutch out" on the downside when they get to the top level. However I'd expect them to either get better soon or be culled from the pool quickly, so their failure would be only marginally detectible in the numbers, which is pretty much what James found for MLB.

  24. Brian Burke says:

    I doubt very many professional athletes are chokers, if any. They weren't born as NFL or MLB players. They played in high school, college, on the practice fields of the pros, and somehow made it where they are today. They only were able to get to where they are by performing well under pressure.

  25. Anonymous says:

    Brian,

    On your comment about the Falcons feasting on the NFC West and "their own division which isn't the strongest", in their division the Falcons have only played @NO, @TB, and home to TB, which is hardly a feast by any ratings system other than yours.

    Also, it's easy to say that your system is underrating them while other systems overrate them, but the in between is very clearly closer to other systems.

    To Ty, you shouldn't gerrymander the model to get it closer to the results you want, but you should be open to adjusting the model somewhat.

    Brian, if you agree that if the Falcons win the rest of their games through the Super Bowl you will adjust the model in an attempt to recognize the important strengths they have that are not currently accounted for, you can perhaps keep incensed Falcons fans at bay. That shouldn't be too much of a concern for you because the Falcons going 17-2 all the way to a Super Bowl title should be almost statistically impossible based on your numbers.

  26. James W says:

    @ Anonymous
    Thanks for your response. Some interesting points, and I do agree with everything you are saying.
    What troubles me is a function of Ryan's efficiency measured by drive statistics. In his career to date, the Atlanta offense has rated 95h-5th-4th annually in FO's DSR stat, and 6th-10th-1st in 3rd Down %.
    While Atlanta's running game is certainly a factor in this, these figures greatly outstrip his efficiency measured by YPA and EPA stats.
    I cannot help but perceive this as being part evidence of the perception that Ryan knows how to manage risk appropriately through an NFL game.
    Personally, I believe Atlanta are not a great team but that Ryan is an elite decision-maker now he has reached his 3rd year. That, in my opinion, gives them a great chance of winning it all.

  27. Kulko says:

    Well your arguement actually gives some possibility to the theory of certain players being chokers.

    Obviously most NFL players cant be chokers, the same as they cant be brittle or slow, becuase they just wouldnt be in the NFL if they were.

    But the really high profile guys would still make it, even if they performed worse in clutch than everywhere else, so these ranks could include some chokers.

  28. Brian Burke says:

    The model does not single out uniquely successful teams and base the rest of the league around it. That wouldn't make any sense. The model is based on the statistics and records of all the teams over several past seasons. Chasing the occasional outlier around is the worst way to devise a model.

    Why do you say the in between is closer to other systems? What "other systems?" Do you mean ones that are calibrated to past events, nearly guaranteed to match the W-L records. We could do that here too, and ATL would be right near the top.

  29. Mark White says:

    BB- I love your website even though I do not agree with this model. I think the missing ingredient is variance of OPASS and ORUN. For example:
    Team 1 offensive pass plays: +30, 0, 0, 0, punt: OPASS = 7.5 result is punt:
    Team 2 offensive pass plays: 5, 5, 0, 10, 5, 5: OPASS = 5.0 result is sustained drive.
    Atlanta's whole key to their success is the ability to sustain long drives with short play action passes and runs. They do not get penalized nor do they turn over the ball. When it is 3rd and short they still convert with regularity. This results in their ability to control the clock. The defense of Atlanta is good enough to keep Atlanta close. In the 4th quarter, Atlanta still controls the clock and this results in facing a tired defense. Therefore they can execute their game plan when the game is on the line.

    I like the idea of a leverage index. I think if you can show some players consistently (year in and year out) perform better in high WPA situations than others then you are on to something. Malcolm Gladwell wrote an article about "choking" in sports:

    http://www.gladwell.com/2000/2000_08_21_a_choking.html

    It is more about how in high pressure situations some elite athletes fail to perform because they start thinking too much. Maybe it's not that some players perform better when the game is on the line but it is that other "choke" because the game is on the line. Do field goal kickers as a whole perform worse when they are kicking the game winning kick? I don't know the answer to that but I would think they do. Do QB's passing efficiency change when the game is on the line? I think your proposed leverage index may reveal the answer.

    Thanks for your work.

  30. Mark White says:

    By the way, The Falcons have not played Carolina yet making your argument that the Falcons have it easy by feasting on the NFC South a moot point. If anything, the Falcons so far have faced one of the hardest SOS in the league.

  31. Anonymous says:

    Brian - the results do seem a bit surprising, but obviously could just be statistical abberations, and the model is still basically valid. I am curious though whether the results here are causing any question in your mind about factors not taken into account in the model, or perhaps the relative weighting of factors? I'm not making an argument here - I'm genuinely curious whether you have some hunches about where the model might need improvement, and whether the results here are surprising to you.

  32. Anonymous says:

    @ Mark White

    What you're describing in reference to Atlanta's ability to sustain long drives is what Brian has labeled Success Rate, which is similar to what Football Outsiders has labeled as DVOA (Value Over Average).

    Success Rate (SR) – The proportion of plays in which a player was directly involved that would typically be considered successful. Specifically, SR is the percentage of plays resulting in positive Expected Points Added (EPA).

    Atlanta currently ranks 12th in pass SR, 13th in run SR, and 13th in overall offensive SR. This arguably means that 12 teams in the NFL are better at sustaining drives than Atlanta is. In addition, most of those teams are netting more yards per pass attempt, as well as being more "successful" on each play.

    And it's not Atlanta's defense that's got them to 10-2 either. They are tied for 2nd to last in Defensive Pass SR.

    They are by no means a bad team - they are definitely above average. As I said earlier, it's very possible that the present model is missing something that's: a) quantifiable, b) repeatable, and c) something that "causes" winning. However, I think it's much, much more likely that Atlanta has been fortunate in many games that could have just as easily gone the other way. It's not meant to be a shot at Atlanta. The Jets are in the same boat - they're not as good as their 9-3 record suggests (and I think we can all agree that even before the Pats game, they were not as good as their 9-2 record suggested).

    The fact is that W-L record, especially over such a short period of time, is highly susceptible to large error from the mean. Whether you want to believe it or not, it's almost certain that if you were to replay the first 12 weeks of the season, Atlanta would very, very likely be worse than 10-2. Generally, the best teams will do worse, and the worst teams will do better. It's regression to the mean, and it happens whether you're flipping coins or you're a professional sports team.

    Let's say Atlanta's "true" mean through 11 games is 7-4 (obviously we can never know this, but we can estimate it based on a large number of factors). It's much more likely that they finish 11 games with a record OTHER than 7-4. 7-4 may be the most likely record, but the probability of all othe records combined far outweighs the probability of just 7-4. If they are 9-2, we would expect them to regress to the mean and have a lower win percentage than 9-2 for their remaining games. If they were 5-6, we would expect them to regress to the mean as well, in this case, have a higher win percentage than 5-6 for their remaining games

  33. Joseph says:

    To those who have asked Brian some variation of "What's wrong with your model?"
    The answer is SPECIAL TEAMS. Especially in SD's case. Their special teams was SOOOOO bad at the start of the year, they have gotten better by just being bad, instead of craptacularly horrible. IIRC, they have had 5 punts and 1 FG blocked, and have allowed 3 KR/PR TD's--including 2 in one game to Leon Washington of SEA. Those are real points allowed, which count on the W-L record. The ST "improving" to poor/mediocre elevated the rest of the team, resulting in their 4 game win streak (before the loss to OAK this past Sun.)

  34. Anonymous says:

    Not sure if the above poster is advocating the addition of special teams into the model, or if he is just noting that special teams accounts for a lot of the variance between actual and expected W-L record.

    Anyway, I think it's been shown that special teams performance has very, very little correlation with itself from season to season, first half to second half, odd and even games, or however you split it up. It's simply not good at predicting wins - only good at retrospectively explaining wins.

  35. Anonymous says:

    Can we improve the model by subtracting from defensive yards allowed the following
    1-offensive penalty.
    2-interception return yards.
    3-all drive ending yards that result in a turnover.

    And subtract from defensive points allowed defensive points scored.

    Hope these will capture the game changing nature
    of turnovers and penalty's in the yards allowed measurement.

  36. Brian Burke says:

    "...special teams performance has very, very little correlation with itself from season to season, first half to second half, odd and even games, or however you split it up. It's simply not good at predicting wins - only good at retrospectively explaining wins. "-- Winner, winner, chicken dinner!

    Based on the question above, there seems to be some misunderstanding about what the model does. The model relies on basic efficiency stats that correlate from week to week and that are predictive of game outcomes: offensive and defensive pass efficiency, running efficiency, own-team penalty yards per snap, and turnover rates. Each individual stat is regressed (diluted) according to how 'consistent' it is, in other words how much can we rely on it to represent a team's 'true' ability in each facet of the game. Home field advantage is thrown in, and to-date opponent strength is also accounted for.

    Each stat is weighted according to a logistic regression, based on all games from the past several regular seasons 2002-2009.

    I can't help it if one or two teams every few years with winning records are below average in passing and running efficiency on both sides of the ball. Unless the sport of football has somehow fundamentally changed between the last several seasons and this one, the model does not need to be monkeyed with.

    One of the unspoken assumptions of modeling is that the fundamental nature of the system being modeled applies to all participants. No one is immune from the laws football.

    That said, the model is just a tool to learn things. It's worthwhile to see a team that's 10-2 rated so lowly and ask why. Let's figure out what's causing the difference.

    In Atlanta's case it's 2 things: clutch play (which you can clearly see by their difference in EPA & SR and in WPA) and turnovers. These are 2 things that are highly explanatory but weakly predictive. Unless you understand that, you won't understand the model and why Atlanta is ranked so low.

    Could there occasionally be the rare team that does possess a consistent talent for turnovers or maybe even for clutch play? I can't prove that there's not. Perhaps Atlanta is one of those. But how do you tell those rare teams apart from the ones that are simply lucky? Why is Atlanta 'good' at those things this year and not last? I can almost guarantee they won't be nearly as 'good' at turnovers and clutch play next year, even if their roster remains 100% intact.

    It took me 2 years to finally grasp the difference between predictive effects and explanatory effects, so I don't expect everyone to get it.

  37. Brian Burke says:

    One last point on Atlanta. For the sake of discussion, assume all NFL teams are perfectly average and all games are in essence coin flips.

    According to the binomial distribution, any single team has a 1.9% chance of seeing a W-L as at least as good 10-2.

    With 32 teams to observe each season, it's not unexpected at all to see teams over- or under-perform their 'true' ability to that degree.

  38. Anonymous says:

    "I can't help it if one or two teams every few years with winning records are below average in passing and running efficiency on both sides of the ball."

    I'd go as far to say that you should expect to see this from a few (random) teams each season...to not see this effect at all is more unlikely than observing it in a few teams.

  39. Anonymous says:

    Brian, in your coin flip example, each team would considered average, right? So with Atlanta actually being BELOW average in every phase of the game, that 1.9% looks to be on the high end. I haven't followed your rankings before this season so I don't know about previous seasons, but if Atlanta is overperforming as much as this model says, that's gotta rank pretty high on all-time flukes.

    I'm not sure how exactly your model differs from Football Outsiders', but I find it interest that their top 3 in rush D is 1. PIT, 2.NYJ, 3. ATL while your's has 1. PIT, 2. NYJ, 19. ATL. In total offensive rank, both your model and FO have the same teams in the top 5, but FO has ATL as #6 and you have them as #13. I'm not going to put out every difference but for models that produce seemingly similar results, why do they differ so colossally on ATL?

  40. Anonymous says:

    @ Anon poster above

    To clarify Brian's example of the coin flipping, if the league was full of average teams all with an equal chance of beating each other, each individual team had a 1.9% chance of doing that well. The odds one team did that well are obviously not (32)*(1.9), but it's still a number that's far greater than 1.9 (if someone feels like dipping into their high school math textbooks, feel free).

    And if 1.9% was only the odds a specific team would end up 3 wins above expected...you can double that percentage for ending up 3 wins BELOW expected OR above. Factoring in a large swing from a true mean in EITHER direction yields an even greater probability.

    What's my point? The point is that some people might be assessing the Falcon's overperformance based on the fact that, WOW, it was/is very unlikely. True, it is very unlikely to happen. But with SO MANY possibilities of unlikely things (i.e., teams with W-L records that are way better or way worse than expected), it's nearly inevitable we end up with certain teams that seem to have beaten the odds.

    When we hear someone won the lottery, are we surprised? The odds were 1 in 10 million!!! Well, with millions of people playing, it's not that surprising!

  41. Brian Burke says:

    To answer the question above, the efficiency model has ATL at 0.43 GWP. Making ATL a 0.50 GWP allows for a reasonable amount of error. I would say that a 10-2 record indicates that whatever error there is underrates them.

    As for the difference between here and FO, I can't speak for their system. From what I can tell they do not regress nearly enough for turnovers. Eliminating fumble recovery luck is not nearly enough. Fumbles themselves are highly random. I think their system is primarily backwards-looking (explanatory).

    But the defensive run ranking is interesting. I've got ATL at 7th in def run SR. That may explain a lot of the difference. But overall, their def SR is 25th because their pass SR is 31st.

  42. Jim Glass says:

    When we hear someone won the lottery, are we surprised? The odds were 1 in 10 million!!! Well, with millions of people playing, it's not that surprising!

    Or consider winning the lottery *twice*. Prospectively, for any one individual going forward, the odds of doing it are as close to zero one can imagine. But retrospectively, considering the number of people who've already won lotteries, the chance that some will win again is a sure thing. And there's a nice little club of people who have. (They're the greatest "clutch" lottery players of all!)

  43. Anonymous says:

    Some people have won the lottery twice? The lottery model is broken!

    The Falcons only have a .43 GWP? The prediction model is broken!

  44. Unknown says:

    Brian - I was the somewhat annoying poster earlier that asked you to agree to change the model if the Falcons win out, and I apologize for being somewhat annoying. I'm taking away the shield of anonymity now.

    Smartass comments aside, I was trying to make a real point that if you stretched out your binomial calculations to going 17-2, the odds would be I believe .036%. Once you factor in that your GWP for the Falcons is actually .43 and they would have to beat well above average teams to win the Super Bowl, the probability becomes microscopic.

    The other factor at play of course is that I fully believe the Falcons can win out, which is clearly where we differ. Not saying it will happen, just that it could, and your model is essentially saying otherwise. This is because for all of the statistics talk, when you watch the play on the field, yes, a couple of games could have gone the other way but by and large the Falcons have earned their record.

    Anyway, we're not going to resolve that point. I basically just wanted to apologize for my earlier comment and make this more constructive. I felt like there was somewhat of a stubborn resistance to change in the model, but I understand that it has a very solid grounding and you can't just change it based on the case of one team over twelve weeks. I also appreciate your efforts to explain the model and have now had a chance to read more of the great stuff on the site, so I can recognize that you would of course think about the implications for the model if the Falcons won out.

    I look forward to hopefully picking up this conversation in 8 weeks.

  45. Anonymous says:

    Brian, I think you should publish two models for ranking teams, one predictive and one explanatory (or retrodictive) and clearly label them as such. Using a vague term like "efficiency" is always going to invite confusion.

  46. Anonymous says:

    David -

    Just curious, if the playoffs were to start today (and Atlanta were to receive a first round bye), what percent chance would you put them at winning the Super Bowl?

  47. Anonymous says:

    Have you considered using Margin of Victory instead of winning pct for the regression model. I think you are losing some information trying to compare per play to final outcome and skipping over points score/allowed

  48. Joseph says:

    Replying to the anonymous commenter who posted right below my first post: I am saying that--#1 Brian's model is not "broken" or "wrong"--just that the difference in rankings here and others (e.g. DVOA of Football Outsiders) is special teams. #2--I understand that many special team events are lucky or random, not as consistent or predictable as offense or defense. However, they are part of the football game. And in the outlier case of this year's Chargers, it does account for almost all the difference in their "perceived" strength and their "real" strength. They do make a real difference in the actual strength of the team. For at least THIS YEAR's Chargers' ST--if they were to make the playoffs (doubtful at this point), they would not make the SB because a costly error on ST would lead to a "lucky" TD for the other team.
    There comes a point where, when a "lucky" event happens multiple time to the same team, you have to admit that there is an unquantifiable element that is causing it.
    #3--I think that Brian could improve his model by adding in some elements of special teams, even though they might not have much weight. IMO, his model is akin to football coaches in the 50's who (because of roster limits) ignored kicking specialists. Then, when a couple of teams began using them, and the improvement was noticeable and tangible, everyone followed suit. It's now to the point where some teams temporarily carry a "kickoff specialist" plus a kicker, punter, and snapper. Some carry a full-time returner, and many have basically a full-time gunner or other special-teams player(s). From what I read, ST ability is what causes some players to be active on game day, and others not to be.
    I don't know what the ideal weight would be, but FO gives offense 3/7, defense 3/7 & ST 1/7. I don't know how they figured out those weights, but I'm guessing it's based upon the average number or ratio of plays per game.

  49. Jim Glass says:

    Re how good the Falcons really are, for what it's worth: Last week FOers' "playoff report" had them as the #1 favorite to win the Super Bowl, a bit odd since they were 7th by DVOA ... Have you considered using Margin of Victory instead of winning pct for the regression model...?, FWIW #2: The PFR.com Simple Rating System, which is s-o-s adjusted average MoV, ranks the Falcs =5th-6th, which is quite good but hardly dominating ... FWIW #3: The Falcs have won 8 close games (by 10 or less) so far this season. Over the last 15 years nine teams have gotten to the playoffs after winning 9 or more close games during the regular season -- and their combined record in the playoffs has been 8-9 (as I've noted on the Community site). So the Falcs had better hope they don't win that 9th!

  50. James says:

    David, Brian's point isn't about Atlanta specifically.

    While there's only a 1.9% chance that a perfectly average team goes 10-2, there is a much bigger chance that one of 32 perfectly average teams will be 10-2. In fact, it's somewhere around a 50% chance!

    Last year during their 6-0 start all the Broncos fans complained about no one respecting them in the preseason, then they finished 8-8. In 2008 the Jets were 8-3 but only had a 0.51 GWP, they finished 9-7. This year the Falcons seem to be that one lucky team that wins more than it should, and the Bucs could be another.

    Granted, Atlanta's record is even better than those above and is nearly a lock for the playoffs. They should be at home for at least one playoff game and might even make the conference championship game... just like last year's 14th ranked Vikings, and look how they regressed to the mean.

  51. Anonymous says:

    what about an efficincy model that predits margin of victory and not winning pct.

  52. Unknown says:

    Something like 70%-60%-50%, so around 20% overall, maybe a little less.

  53. Unknown says:

    Statistically speaking there's a good chance that one of 32 average teams would be 10-2. My point is that I don't think the Falcons are a statistical anomaly, but again that's not something that we're going to be able to resolve through this conversation.

  54. Anonymous says:

    They are by definition of the model a statistical anomaly, or outlier. The only other explanation is that the model is deficient.

    While the present model is obviously not perfect (what model is?), understand that by changing it to accomodate the Falcons, you'd probably be creating new outliers in the form of other teams.

    It should really only be changed if you can find an a priori reason as to why the model is inefficient, and not as predictive as it could be. Looking at the 4 or 5 outliers that are naturally expected to occur (and are completely unpredictable before the season begins) and adjusting the model based on that is the wrong way to go about improving the predictivity of the model.

  55. jason says:

    @ Jim Glass and also David

    Atlanta has a great chance to go to the Super Bowl simply because of home field advantage not because they are better than the Eagles, Giants or Packers.

    Once in the Super Bowl however, they have unlikely odds to beat either the Patriots or the Steelers who are the most likely AFC representatives.

Leave a Reply

Note: Only a member of this blog may post a comment.