tag:blogger.com,1999:blog-38600807.post3666895546838015432..comments2024-10-28T16:59:39.881-04:00Comments on Advanced Football Analytics (formerly Advanced NFL Stats): Playoff ClarityUnknownnoreply@blogger.comBlogger10125tag:blogger.com,1999:blog-38600807.post-25952351894717916442013-11-10T10:28:03.051-05:002013-11-10T10:28:03.051-05:00@Chris
I think Denver's GWP had a much larger...@Chris<br /><br />I think Denver's GWP had a much larger variance than that based on Brian's model.<br /><br />Anyway, rare things are supposed to happen rarely, not occasionally.<br /><br />nfl-forecast predicted 110 team's fates were "almost sealed" (<10% or >90% odds as described in the blog) from 2007-2012 in week 8*. Based on the model's own probabilities, it should have only been wrong about only 2 or 3 of those locks.<br /><br />But it was wrong 7 times which is enough to reject the model (p=0.078) despite the small sample size.<br /><br />* - and week 9 for 2008 when I couldn't find week 8Stevehttps://www.blogger.com/profile/10230344931186858123noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-52356788016045289032013-11-09T20:31:34.788-05:002013-11-09T20:31:34.788-05:00@Steve -- Adding variance to the GWP has less effe...@Steve -- Adding variance to the GWP has less effect than one would think because of the uncertainty of the GWP of the opponents.<br /><br />For example after 6 weeks, the expected number of wins for the 2009 Broncos out of their 10 remaining games was 7.4 wins with a standard deviation of 1.2 games.<br /><br />If you re-do the analysis allowing for a SD of 0.10 in GWP for each team, Denver's SD of games won only increases to about 1.5 games. I doubt it would have changed Denver's playoff odds by more than 1%<br /><br />Occasionally, rare things happen.Chrishttp://nfl-forecast.comnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-38550125389937261942013-11-08T19:19:57.402-05:002013-11-08T19:19:57.402-05:00> Perhaps the two above are simply using fancie...> Perhaps the two above are simply using fancier terms that I am, but why <br />> not just add up the percentages of the top 6 teams in each conference?<br /><br />Not so much:<br />http://en.wikipedia.org/wiki/Shannon_entropy<br />http://en.wikipedia.org/wiki/Gini_coefficient<br /><br />The reason that I suggested Shannon entropy is that it measures how much unknown 'information' is left to completely determine the playoffs -- it's pretty much exactly what you'd use to describe how 'unclear' the playoff picture is.NateTGnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-84921123149689789402013-11-08T16:02:00.957-05:002013-11-08T16:02:00.957-05:00There is still a possibility that they are off--ei...There is still a possibility that they are off--either too high or too low--even if they are right on average.<br /><br />The easiest way to see why this matter is if you try to simulate the playoffs ignoring the error terms. The probability of winning the Super Bowl for a team with a bye is P(win divisional AND win championship AND win SB). If there is no uncertainty about team quality then those are all independent so you can simulate them and multiply.<br /><br />But if they aren't independent then you can't. One reason they wouldn't be independent is if Baltimore is actually better than we think, so their GWP is higher for the div round, the CC, and the SB, or worse (so they're more likely to be one and done or win and then crash and burn).<br /><br />You can see some evidence of the overconfidence if you plot the actual outcomes (smooth) vs the predictions from nfl-forecast like here but the small sample size caveat applies:<br /><br />https://dl.dropboxusercontent.com/u/2174585/proj_vs_realized.jpgStevehttps://www.blogger.com/profile/10230344931186858123noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-54648174511372614222013-11-08T15:36:45.698-05:002013-11-08T15:36:45.698-05:00Steve, since the GWPs are already regressed for th...Steve, since the GWPs are already regressed for their predictiveness for the rest of the season shouldn't that mitigate the misevaluation error?Jameshttps://www.blogger.com/profile/01838293735141324662noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-38825079358664985002013-11-08T15:18:33.001-05:002013-11-08T15:18:33.001-05:00I think this is interesting but nfl-forecast exagg...I think this is interesting but nfl-forecast exaggerates the odds of good teams making the playoffs and underestimates the odds of bad teams making the playoffs.<br /><br />The 2009 teams are a good illustration of the overconfidence of the model. Indy and NO of course made the playoffs but Denver crashed and burned despite that 99% chance of making it.<br /><br />The NBA playoff forecaster on ESPN tries to correct for this kind of overconfidence with the anti-Dennis Green postulate: "Maybe they AREN'T who we think they are" and you have to take that into account.<br /><br />Technical explanation of the above:<br /><br />nfl-forecast (as I understand) makes forecasts by using the GWP formula to calculate a probability of each team winning for each game. Then it does a Monte carlo simulations of the season based on random draws for each game and sees who makes the playoffs and repeats 10,000 times (or so). The projected odds of making the playoff for each team is (# of times made playoffs)/10,000 simulations.<br /><br />But the GWPs aren't exact. They are based on the team's estimated quality level which is measured with error. So when you do a monte carlo simulation you should not only draw errors for the individual games, all the teams should get errors drawn for their quality. This adds a lot more variation in who makes the playoffs because intuitively there are now TWO ways low probability teams can sneak in: (1) as before they can get lucky in a string of simulated games or (2) they are better than their GWP suggests and they win games based on their underlying strength. So when you simulate it out you'll see more teams with low generic GWP/quality get in and more teams with high generic GWP/quality miss out.Stevehttps://www.blogger.com/profile/10230344931186858123noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-45673669583204552102013-11-08T14:49:15.979-05:002013-11-08T14:49:15.979-05:00Perhaps the two above are simply using fancier ter...Perhaps the two above are simply using fancier terms that I am, but why not just add up the percentages of the top 6 teams in each conference? Jon Greimanhttps://www.blogger.com/profile/12788609333000216699noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-12845459994243983602013-11-08T11:41:48.457-05:002013-11-08T11:41:48.457-05:00Building on NateTG's comment, I wonder if you ...Building on NateTG's comment, I wonder if you could use the Gini coefficient as a way to measure clarity. It's usually calculated in the context of income inequality (how much wealth is concentrated in the hands of the few). But I would think it would work similarly for playoff probabilities. You basically have 1200% of playoff probability (100% each for the 12 teams). How equally is that playoff probability distributed amongst the 32 teams?Michael Beuoyhttps://www.blogger.com/profile/03960600491528993233noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-66932452479098751062013-11-08T10:36:24.482-05:002013-11-08T10:36:24.482-05:00That is definitely a neat thing to be thinking abo...That is definitely a neat thing to be thinking about, but it seems like the Shannon Entropy of the projection would be a much more appropriate measure for the clarity of the playoff picture than taking the standard deviation of teams' chances to make the playoffs.NateTGnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-63961786110541318092013-11-08T10:31:17.660-05:002013-11-08T10:31:17.660-05:00Very cool analysis!Very cool analysis!Chrishttp://nfl-forecast.comnoreply@blogger.com