tag:blogger.com,1999:blog-38600807.post6695630588592183590..comments2023-01-29T03:55:02.336-05:00Comments on Advanced Football Analytics (formerly Advanced NFL Stats): Team Efficiency Rankings - Week 14Unknownnoreply@blogger.comBlogger55125tag:blogger.com,1999:blog-38600807.post-65769678852922319402010-12-11T20:06:16.936-05:002010-12-11T20:06:16.936-05:00@ Jim Glass and also David
Atlanta has a great ch...@ Jim Glass and also David<br /><br />Atlanta has a great chance to go to the Super Bowl simply because of home field advantage not because they are better than the Eagles, Giants or Packers.<br /><br />Once in the Super Bowl however, they have unlikely odds to beat either the Patriots or the Steelers who are the most likely AFC representatives.jasonnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-65029238769364708782010-12-10T13:58:16.927-05:002010-12-10T13:58:16.927-05:00They are by definition of the model a statistical ...They are by definition of the model a statistical anomaly, or outlier. The only other explanation is that the model is deficient.<br /><br />While the present model is obviously not perfect (what model is?), understand that by changing it to accomodate the Falcons, you'd probably be creating new outliers in the form of other teams.<br /><br />It should really only be changed if you can find an a priori reason as to why the model is inefficient, and not as predictive as it could be. Looking at the 4 or 5 outliers that are naturally expected to occur (and are completely unpredictable before the season begins) and adjusting the model based on that is the wrong way to go about improving the predictivity of the model.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-47727204304773477592010-12-10T02:08:28.605-05:002010-12-10T02:08:28.605-05:00Statistically speaking there's a good chance t...Statistically speaking there's a good chance that one of 32 average teams would be 10-2. My point is that I don't think the Falcons are a statistical anomaly, but again that's not something that we're going to be able to resolve through this conversation.Unknownhttps://www.blogger.com/profile/16811989812393513991noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-10269622714423795492010-12-10T02:00:42.084-05:002010-12-10T02:00:42.084-05:00Something like 70%-60%-50%, so around 20% overall,...Something like 70%-60%-50%, so around 20% overall, maybe a little less.Unknownhttps://www.blogger.com/profile/16811989812393513991noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-11351880123684418642010-12-09T19:38:40.244-05:002010-12-09T19:38:40.244-05:00what about an efficincy model that predits margin ...what about an efficincy model that predits margin of victory and not winning pct.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-63207376094711466922010-12-09T15:27:17.831-05:002010-12-09T15:27:17.831-05:00David, Brian's point isn't about Atlanta s...David, Brian's point isn't about Atlanta specifically. <br /><br />While there's only a 1.9% chance that a perfectly average team goes 10-2, there is a much bigger chance that one of 32 perfectly average teams will be 10-2. In fact, it's somewhere around a 50% chance!<br /><br />Last year during their 6-0 start all the Broncos fans complained about no one respecting them in the preseason, then they finished 8-8. In 2008 the Jets were 8-3 but only had a 0.51 GWP, they finished 9-7. This year the Falcons seem to be that one lucky team that wins more than it should, and the Bucs could be another.<br /><br />Granted, Atlanta's record is even better than those above and is nearly a lock for the playoffs. They should be at home for at least one playoff game and might even make the conference championship game... just like last year's 14th ranked Vikings, and look how they regressed to the mean.Jameshttps://www.blogger.com/profile/01838293735141324662noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-17533693315967744122010-12-09T14:54:04.703-05:002010-12-09T14:54:04.703-05:00Re how good the Falcons really are, for what it...Re how good the Falcons really are, for what it's worth: Last week FOers' "playoff report" had them as the #1 favorite to win the Super Bowl, a bit odd since they were 7th by DVOA ... <em>Have you considered using Margin of Victory instead of winning pct for the regression model...?</em>, FWIW #2: The PFR.com Simple Rating System, which is s-o-s adjusted average MoV, ranks the Falcs =5th-6th, which is quite good but hardly dominating ... FWIW #3: The Falcs have won 8 close games (by 10 or less) so far this season. Over the last 15 years nine teams have gotten to the playoffs after winning 9 or more close games during the regular season -- and their combined record in the playoffs has been 8-9 (as I've noted on the Community site). So the Falcs had better hope they don't win that 9th!Jim Glassnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-48627887103930882182010-12-09T13:17:13.765-05:002010-12-09T13:17:13.765-05:00Replying to the anonymous commenter who posted rig...Replying to the anonymous commenter who posted right below my first post: I am saying that--#1 Brian's model is not "broken" or "wrong"--just that the difference in rankings here and others (e.g. DVOA of Football Outsiders) is special teams. #2--I understand that many special team events are lucky or random, not as consistent or predictable as offense or defense. However, they are part of the football game. And in the outlier case of this year's Chargers, it does account for almost all the difference in their "perceived" strength and their "real" strength. They do make a real difference in the actual strength of the team. For at least THIS YEAR's Chargers' ST--if they were to make the playoffs (doubtful at this point), they would not make the SB because a costly error on ST would lead to a "lucky" TD for the other team. <br />There comes a point where, when a "lucky" event happens multiple time to the same team, you have to admit that there is an unquantifiable element that is causing it. <br />#3--I think that Brian could improve his model by adding in some elements of special teams, even though they might not have much weight. IMO, his model is akin to football coaches in the 50's who (because of roster limits) ignored kicking specialists. Then, when a couple of teams began using them, and the improvement was noticeable and tangible, everyone followed suit. It's now to the point where some teams temporarily carry a "kickoff specialist" plus a kicker, punter, and snapper. Some carry a full-time returner, and many have basically a full-time gunner or other special-teams player(s). From what I read, ST ability is what causes some players to be active on game day, and others not to be.<br />I don't know what the ideal weight would be, but FO gives offense 3/7, defense 3/7 & ST 1/7. I don't know how they figured out those weights, but I'm guessing it's based upon the average number or ratio of plays per game.Josephnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-71329548498916788482010-12-09T12:24:49.705-05:002010-12-09T12:24:49.705-05:00Have you considered using Margin of Victory instea...Have you considered using Margin of Victory instead of winning pct for the regression model. I think you are losing some information trying to compare per play to final outcome and skipping over points score/allowedAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-32290404449118194692010-12-09T11:50:47.066-05:002010-12-09T11:50:47.066-05:00David -
Just curious, if the playoffs were to sta...David -<br /><br />Just curious, if the playoffs were to start today (and Atlanta were to receive a first round bye), what percent chance would you put them at winning the Super Bowl?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-15344342831717956822010-12-09T11:02:49.229-05:002010-12-09T11:02:49.229-05:00Brian, I think you should publish two models for r...Brian, I think you should publish two models for ranking teams, one predictive and one explanatory (or retrodictive) and clearly label them as such. Using a vague term like "efficiency" is always going to invite confusion.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-19669105080731780212010-12-09T10:25:16.835-05:002010-12-09T10:25:16.835-05:00Brian - I was the somewhat annoying poster earlier...Brian - I was the somewhat annoying poster earlier that asked you to agree to change the model if the Falcons win out, and I apologize for being somewhat annoying. I'm taking away the shield of anonymity now. <br /><br />Smartass comments aside, I was trying to make a real point that if you stretched out your binomial calculations to going 17-2, the odds would be I believe .036%. Once you factor in that your GWP for the Falcons is actually .43 and they would have to beat well above average teams to win the Super Bowl, the probability becomes microscopic. <br /><br />The other factor at play of course is that I fully believe the Falcons can win out, which is clearly where we differ. Not saying it will happen, just that it could, and your model is essentially saying otherwise. This is because for all of the statistics talk, when you watch the play on the field, yes, a couple of games could have gone the other way but by and large the Falcons have earned their record. <br /><br />Anyway, we're not going to resolve that point. I basically just wanted to apologize for my earlier comment and make this more constructive. I felt like there was somewhat of a stubborn resistance to change in the model, but I understand that it has a very solid grounding and you can't just change it based on the case of one team over twelve weeks. I also appreciate your efforts to explain the model and have now had a chance to read more of the great stuff on the site, so I can recognize that you would of course think about the implications for the model if the Falcons won out. <br /><br />I look forward to hopefully picking up this conversation in 8 weeks.Unknownhttps://www.blogger.com/profile/16811989812393513991noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-62588694927554351382010-12-08T20:34:02.807-05:002010-12-08T20:34:02.807-05:00Some people have won the lottery twice? The lotter...Some people have won the lottery twice? The lottery model is broken!<br /><br />The Falcons only have a .43 GWP? The prediction model is broken!Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-71536969406734914572010-12-08T20:23:09.917-05:002010-12-08T20:23:09.917-05:00When we hear someone won the lottery, are we surpr...<em>When we hear someone won the lottery, are we surprised? The odds were 1 in 10 million!!! Well, with millions of people playing, it's not that surprising! </em><br /><br />Or consider winning the lottery *twice*. Prospectively, for any one individual going forward, the odds of doing it are as close to zero one can imagine. But retrospectively, considering the number of people who've already won lotteries, the chance that some will win again is a sure thing. And there's a nice little club of people who have. (They're the greatest "clutch" lottery players of all!)Jim Glassnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-77362859704410433892010-12-08T19:56:43.159-05:002010-12-08T19:56:43.159-05:00To answer the question above, the efficiency model...To answer the question above, the efficiency model has ATL at 0.43 GWP. Making ATL a 0.50 GWP allows for a reasonable amount of error. I would say that a 10-2 record indicates that whatever error there is underrates them.<br /><br />As for the difference between here and FO, I can't speak for their system. From what I can tell they do not regress nearly enough for turnovers. Eliminating fumble recovery luck is not nearly enough. Fumbles themselves are highly random. I think their system is primarily backwards-looking (explanatory).<br /><br />But the defensive run ranking is interesting. I've got ATL at 7th in def run SR. That may explain a lot of the difference. But overall, their def SR is 25th because their pass SR is 31st.Brian Burkehttps://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-32160773561353372282010-12-08T18:54:21.127-05:002010-12-08T18:54:21.127-05:00@ Anon poster above
To clarify Brian's exampl...@ Anon poster above<br /><br />To clarify Brian's example of the coin flipping, if the league was full of average teams all with an equal chance of beating each other, each individual team had a 1.9% chance of doing that well. The odds one team did that well are obviously not (32)*(1.9), but it's still a number that's far greater than 1.9 (if someone feels like dipping into their high school math textbooks, feel free).<br /><br />And if 1.9% was only the odds a specific team would end up 3 wins above expected...you can double that percentage for ending up 3 wins BELOW expected OR above. Factoring in a large swing from a true mean in EITHER direction yields an even greater probability.<br /><br />What's my point? The point is that some people might be assessing the Falcon's overperformance based on the fact that, WOW, it was/is very unlikely. True, it is very unlikely to happen. But with SO MANY possibilities of unlikely things (i.e., teams with W-L records that are way better or way worse than expected), it's nearly inevitable we end up with certain teams that seem to have beaten the odds.<br /><br />When we hear someone won the lottery, are we surprised? The odds were 1 in 10 million!!! Well, with millions of people playing, it's not that surprising!Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-29851802132421040712010-12-08T18:25:35.354-05:002010-12-08T18:25:35.354-05:00Brian, in your coin flip example, each team would ...Brian, in your coin flip example, each team would considered average, right? So with Atlanta actually being BELOW average in every phase of the game, that 1.9% looks to be on the high end. I haven't followed your rankings before this season so I don't know about previous seasons, but if Atlanta is overperforming as much as this model says, that's gotta rank pretty high on all-time flukes. <br /><br />I'm not sure how exactly your model differs from Football Outsiders', but I find it interest that their top 3 in rush D is 1. PIT, 2.NYJ, 3. ATL while your's has 1. PIT, 2. NYJ, 19. ATL. In total offensive rank, both your model and FO have the same teams in the top 5, but FO has ATL as #6 and you have them as #13. I'm not going to put out every difference but for models that produce seemingly similar results, why do they differ so colossally on ATL?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-26013367449013502632010-12-08T17:07:13.777-05:002010-12-08T17:07:13.777-05:00"I can't help it if one or two teams ever..."I can't help it if one or two teams every few years with winning records are below average in passing and running efficiency on both sides of the ball."<br /><br />I'd go as far to say that you should expect to see this from a few (random) teams each season...to not see this effect at all is more unlikely than observing it in a few teams.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-48035087481551187902010-12-08T16:29:17.137-05:002010-12-08T16:29:17.137-05:00One last point on Atlanta. For the sake of discuss...One last point on Atlanta. For the sake of discussion, assume all NFL teams are perfectly average and all games are in essence coin flips.<br /><br />According to the binomial distribution, any single team has a 1.9% chance of seeing a W-L as at least as good 10-2. <br /><br />With 32 teams to observe each season, it's not unexpected at all to see teams over- or under-perform their 'true' ability to that degree.Brian Burkehttps://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-60610747445147569432010-12-08T16:10:43.327-05:002010-12-08T16:10:43.327-05:00"...special teams performance has very, very ..."...special teams performance has very, very little correlation with itself from season to season, first half to second half, odd and even games, or however you split it up. It's simply not good at predicting wins - only good at retrospectively explaining wins. "-- Winner, winner, chicken dinner!<br /><br />Based on the question above, there seems to be some misunderstanding about what the model does. The model relies on basic efficiency stats that correlate from week to week and that are predictive of game outcomes: offensive and defensive pass efficiency, running efficiency, own-team penalty yards per snap, and turnover rates. Each individual stat is regressed (diluted) according to how 'consistent' it is, in other words how much can we rely on it to represent a team's 'true' ability in each facet of the game. Home field advantage is thrown in, and to-date opponent strength is also accounted for. <br /><br />Each stat is weighted according to a logistic regression, based on all games from the past several regular seasons 2002-2009.<br /><br />I can't help it if one or two teams every few years with winning records are below average in passing and running efficiency on both sides of the ball. Unless the sport of football has somehow fundamentally changed between the last several seasons and this one, the model does not need to be monkeyed with.<br /><br />One of the unspoken assumptions of modeling is that the fundamental nature of the system being modeled applies to all participants. No one is immune from the laws football. <br /><br />That said, the model is just a tool to learn things. It's worthwhile to see a team that's 10-2 rated so lowly and ask why. Let's figure out what's causing the difference. <br /><br />In Atlanta's case it's 2 things: clutch play (which you can clearly see by their difference in EPA & SR and in WPA) and turnovers. These are 2 things that are highly explanatory but weakly predictive. Unless you understand that, you won't understand the model and why Atlanta is ranked so low. <br /><br />Could there occasionally be the rare team that does possess a consistent talent for turnovers or maybe even for clutch play? I can't prove that there's not. Perhaps Atlanta is one of those. But how do you tell those rare teams apart from the ones that are simply lucky? Why is Atlanta 'good' at those things this year and not last? I can almost guarantee they won't be nearly as 'good' at turnovers and clutch play next year, even if their roster remains 100% intact.<br /><br />It took me 2 years to finally grasp the difference between predictive effects and explanatory effects, so I don't expect everyone to get it.Brian Burkehttps://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-26967038465998112172010-12-08T15:30:34.616-05:002010-12-08T15:30:34.616-05:00Can we improve the model by subtracting from defen...Can we improve the model by subtracting from defensive yards allowed the following<br />1-offensive penalty.<br />2-interception return yards.<br />3-all drive ending yards that result in a turnover.<br /><br />And subtract from defensive points allowed defensive points scored.<br /><br />Hope these will capture the game changing nature<br />of turnovers and penalty's in the yards allowed measurement.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-29771233414839314152010-12-08T14:01:50.739-05:002010-12-08T14:01:50.739-05:00Not sure if the above poster is advocating the add...Not sure if the above poster is advocating the addition of special teams into the model, or if he is just noting that special teams accounts for a lot of the variance between actual and expected W-L record.<br /><br />Anyway, I think it's been shown that special teams performance has very, very little correlation with itself from season to season, first half to second half, odd and even games, or however you split it up. It's simply not good at predicting wins - only good at retrospectively explaining wins.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-69239602118984824202010-12-08T11:48:23.495-05:002010-12-08T11:48:23.495-05:00To those who have asked Brian some variation of &q...To those who have asked Brian some variation of "What's wrong with your model?"<br />The answer is SPECIAL TEAMS. Especially in SD's case. Their special teams was SOOOOO bad at the start of the year, they have gotten better by just being bad, instead of craptacularly horrible. IIRC, they have had 5 punts and 1 FG blocked, and have allowed 3 KR/PR TD's--including 2 in one game to Leon Washington of SEA. Those are real points allowed, which count on the W-L record. The ST "improving" to poor/mediocre elevated the rest of the team, resulting in their 4 game win streak (before the loss to OAK this past Sun.)Josephnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-72965123066059621322010-12-08T10:20:16.487-05:002010-12-08T10:20:16.487-05:00@ Mark White
What you're describing in refere...@ Mark White<br /><br />What you're describing in reference to Atlanta's ability to sustain long drives is what Brian has labeled Success Rate, which is similar to what Football Outsiders has labeled as DVOA (Value Over Average).<br /><br />Success Rate (SR) – The proportion of plays in which a player was directly involved that would typically be considered successful. Specifically, SR is the percentage of plays resulting in positive Expected Points Added (EPA).<br /><br />Atlanta currently ranks 12th in pass SR, 13th in run SR, and 13th in overall offensive SR. This arguably means that 12 teams in the NFL are better at sustaining drives than Atlanta is. In addition, most of those teams are netting more yards per pass attempt, as well as being more "successful" on each play.<br /><br />And it's not Atlanta's defense that's got them to 10-2 either. They are tied for 2nd to last in Defensive Pass SR.<br /><br />They are by no means a bad team - they are definitely above average. As I said earlier, it's very possible that the present model is missing something that's: a) quantifiable, b) repeatable, and c) something that "causes" winning. However, I think it's much, much more likely that Atlanta has been fortunate in many games that could have just as easily gone the other way. It's not meant to be a shot at Atlanta. The Jets are in the same boat - they're not as good as their 9-3 record suggests (and I think we can all agree that even before the Pats game, they were not as good as their 9-2 record suggested).<br /><br />The fact is that W-L record, especially over such a short period of time, is highly susceptible to large error from the mean. Whether you want to believe it or not, it's almost certain that if you were to replay the first 12 weeks of the season, Atlanta would very, very likely be worse than 10-2. Generally, the best teams will do worse, and the worst teams will do better. It's regression to the mean, and it happens whether you're flipping coins or you're a professional sports team.<br /><br />Let's say Atlanta's "true" mean through 11 games is 7-4 (obviously we can never know this, but we can estimate it based on a large number of factors). It's much more likely that they finish 11 games with a record OTHER than 7-4. 7-4 may be the most likely record, but the probability of all othe records combined far outweighs the probability of just 7-4. If they are 9-2, we would expect them to regress to the mean and have a lower win percentage than 9-2 for their remaining games. If they were 5-6, we would expect them to regress to the mean as well, in this case, have a higher win percentage than 5-6 for their remaining gamesAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-86829888057472661352010-12-08T09:46:18.074-05:002010-12-08T09:46:18.074-05:00Brian - the results do seem a bit surprising, but ...Brian - the results do seem a bit surprising, but obviously could just be statistical abberations, and the model is still basically valid. I am curious though whether the results here are causing any question in your mind about factors not taken into account in the model, or perhaps the relative weighting of factors? I'm not making an argument here - I'm genuinely curious whether you have some hunches about where the model might need improvement, and whether the results here are surprising to you.Anonymousnoreply@blogger.com