tag:blogger.com,1999:blog-38600807.post478555591584080399..comments2023-09-09T10:36:20.114-04:00Comments on Advanced Football Analytics (formerly Advanced NFL Stats): Game Probabilities - Conference ChampionshipsUnknownnoreply@blogger.comBlogger29125tag:blogger.com,1999:blog-38600807.post-13597797114274134752010-01-25T00:48:00.425-05:002010-01-25T00:48:00.425-05:002-0 for this week not too bad.2-0 for this week not too bad.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-56530611682565542992010-01-23T17:21:51.445-05:002010-01-23T17:21:51.445-05:00No. I factor that in inside my own head, but the m...No. I factor that in inside my own head, but the model doesn't automatically consider it.Brian Burkehttps://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-4043856762176738902010-01-23T17:19:25.785-05:002010-01-23T17:19:25.785-05:00Does your HFA coefficient adjust for dome vs. Cold...Does your HFA coefficient adjust for dome vs. Cold games. Say for example if the Colts were going to the Jets, would the Jets HFA be higher in your model because it's a cold vs. Dome game?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-76313205211970407092010-01-23T17:04:43.481-05:002010-01-23T17:04:43.481-05:00Thanks for doing this by the way.
I'm really ...Thanks for doing this by the way.<br /><br />I'm really surprised by that. I thought it would be the opposite. My HFA coefficient is the highest it's been this year. It gets up over 9% (59 vs 41) when teams are evenly matched.<br /><br />Maybe I need to interact HFA with the other variables. Perhaps increase it just for home underdogs?<br /><br />Home field can vary pretty wildly from year to year. About three years ago home teams only won 53 or 54%. Then another year it was up well over 60%. Maybe 2009 was just a very high year. I can't see increasing the HFA coefficient any higher than it already is.Brian Burkehttps://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-20875368457218967872010-01-23T16:48:29.816-05:002010-01-23T16:48:29.816-05:00By the way I love the site -- I realize I just jum...By the way I love the site -- I realize I just jumped in and starting critiquing the model, but that is just because I think it is really interesting and fun to play with (and I don't feel like doing real work right now!). I hope you keep it up for a long time! :-)Tim Bradyhttp://web.mit.edu/tfbradynoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-31352039948750568702010-01-23T16:47:18.968-05:002010-01-23T16:47:18.968-05:00The overconfidence interacts with home field advan...The overconfidence interacts with home field advantage pretty severely. When you predict the home team will win, you do very well (in fact, the model is quite underconfident). However, when you predict the away team will win you are pretty much randomly guessing. Probably means you are underweighting home field in some situations.<br /><br />E.g., http://web.mit.edu/tfbrady/winAndHomefield.pngTim Bradyhttp://web.mit.edu/tfbradynoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-30366978975279372302010-01-23T16:34:16.645-05:002010-01-23T16:34:16.645-05:00Each week I only really pay attention to the games...Each week I only really pay attention to the games where I disagree with the spread. If my favorite is different than the opening line's (I usually look at USA Today's lines), I'll note if the model was correct or not. I think I'm up at least 2, maybe 3 games this year--not enough to make money if you're a gambler. I only count weeks 4 through 16 each season.Brian Burkehttps://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-62990710758841789312010-01-23T16:32:05.280-05:002010-01-23T16:32:05.280-05:00Yup. I agree. It's about an average of 5 % poi...Yup. I agree. It's about an average of 5 % points overconfident this season. I'm a little surprised because this year I increased the strength of the regression of the early season team stats. If anything I thought it would be underconfident.Brian Burkehttps://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-55233840945478966022010-01-23T16:26:54.920-05:002010-01-23T16:26:54.920-05:00So I just spent an hour or so analyzing your model...So I just spent an hour or so analyzing your model and I think it is pretty clear that the model was overconfident this year. I analyzed the 192 games you predicted between week 4 and week 16. <br /><br />My first pass graph looked pretty good for the model: http://web.mit.edu/tfbrady/football/modelCheck.png plots the model predictions (win%) and the actual win% of the teams, binned into 5% units. No obvious trend for the actual win% to be shallower than predicted by the model, although certainly a hint of that.<br /><br />But if you look at it more precisely, using the log likelihood of the data under the model, then it is clear that making the model predictions less extreme improves the model. Taking the sum of the log likelihoods for each game (sum(log(winProbabilityOfWinningTeam))), closer to 0 is better (if you predicted all games with certainty 1 and were always right you'd have a 0), and closer to -Inf is worse (if you predicted all games with a certainty of 1 and were always wrong you'd have -Inf). If you predicted 0.5 for each game, you'd have a score of -133.1. The model as is has a log likelihood of -116.1. So the model is doing far better than just saying all games are toss ups.<br /><br />But you can also look at what the log probability of the model is if you make it uniformly more or less conservative. The way I did this is to take the model's predictions and move them a percentage closer to or further from 0.5. The results are plotted here: http://web.mit.edu/tfbrady/football/modelTooExtreme.png As you can see, if you make all of the model predictions 1.2 times less extreme (e.g., make 75% into 70.1%, 90% into 83%, etc), the model achieves its best log probability (closest to 0). So I think that it is pretty clear that in general the model is overconfident in its predictions (they should be 20% closer to 0.50).<br /><br />I'm curious what you make of this.Tim Bradyhttp://web.mit.edu/tfbradynoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-79911628723369802392010-01-23T15:54:00.323-05:002010-01-23T15:54:00.323-05:00Brian, you wrote that you have beaten Vegas predic...Brian, you wrote that you have beaten Vegas predictions. Can you give a summary on why you think you have?Unknownhttps://www.blogger.com/profile/01128551772469314869noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-54927610671370006432010-01-23T15:39:47.749-05:002010-01-23T15:39:47.749-05:00At Pro Football Reference.com they have a thread r...At Pro Football Reference.com they have a thread running on why the #1 seeds get to the Super Bowl so infrequently.<br /><br />One of my favorite articles that you did here was the one showing that more than 50% of NFL game outcomes are determined <a href="http://www.advancednflstats.com/2007/08/luck-and-nfl-outcomes-3.html" rel="nofollow">by random chance</a>.<br /><br />I'd expect that this percentage is higher in the playoffs than during the regular season, as a during the regular season there are a lot of "good team v bad team" matchups, while in the playoffs the teams are all withing a much narrower quality band.<br /><br />Would you have a back-of-the-envelope way of estimating that?<br /><br />I mean, commentators <em>never</em> mention this concept -- if anything, they "hero-ize" the top teams to make it seem like they should be <em>more</em> likely to win in the playoffs. Championship character revealing itself through "clutch play" and all.<br /><br />If you have another article coming up in the Times, this could be a good angle: "If your team wins the Super Bowl what does it mean? That you were lucky or good? Mostly <em>lucky</em>!"<br /><br />That would get some conversation going in the comments over there, I bet.Jim Glasshttp://www.scrivener.netnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-20167324894656207332010-01-23T07:20:16.954-05:002010-01-23T07:20:16.954-05:00Brian, as a math major myself, I enjoy your posts ...Brian, as a math major myself, I enjoy your posts (probabilities), but do think you have made a mistake on Minnesota a little bit. I think the 80% chance given to the Saints should call into question something regarding your model, though I know none will be perfect. I think you acknowledged that your ratings will be high. I suspect (perhaps you have the data) but what about the teams you give a 80-90% chance of winning? it is probably in the 70's. I see our friends at footballoutsiders (sorry for the mention) have the saints with just a 51% chance of winning Of course, we will never know what is the THEORETICAL probability, but it almost certainly has to be between (though I lean towards your number a little more) perhaps 70%. <br /><br />I suppose another way to look at things is that an 80% favorite translate to about a 10 point favorite, which means the oddsmakers are off by 55 points. I have followed spots and betting for the better part of 40 years and while that is very possible during the regular season, I have found the playoff lines quite tight (again, nobody can ever verify the 'true' probability)conquistadorhttp://www.twominutewarning.com/childhome.htmnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-44121458646329601462010-01-21T22:28:15.843-05:002010-01-21T22:28:15.843-05:00Brian,
I just looked over all your previous HFA...Brian,<br /> I just looked over all your previous HFA Posts. You suggest that the HFA for dome at cold is .88. My question is as follows, does this impact the input for the Effiecency stats. Should the stats from such games be somehow normlized to take that extra 38% back out?<br /><br />i.e. 2 fictional exactly equal dome teams, Team A has 3 dome at cold games, Team B does not. Team A will have presumable lower efficiency stats for having the Dome at Cold games (The extreme HFA against would likely result in worse production), but it would have no effect on a Team A / Team B. Is this acounted for in the model, or is it a miniscule enough occurance not to matter.<br /><br />Bottom line does the number of Dome at cold games a team has affect their WP in the model for non Dome at cold games. Should it? Does it matter?<br /><br />Thanks,<br /><br />ErikAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-63660956231887990552010-01-21T21:34:30.765-05:002010-01-21T21:34:30.765-05:00Brian,
I appreciate all that you do.
Look forward...Brian,<br /><br />I appreciate all that you do.<br />Look forward to reading your<br />insights each week. How these<br />fans can get so emotional over <br />probabilities is beyond me. <br /><br />Zeke TomberAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-38456249948643097102010-01-21T17:21:58.057-05:002010-01-21T17:21:58.057-05:00I would include the accuracy in you NYTimes postin...I would include the accuracy in you NYTimes posting for now on and include it on the main page.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-50544460852059199632010-01-21T17:17:52.781-05:002010-01-21T17:17:52.781-05:00About 71 or 72%. If you count week 17 it's clo...About 71 or 72%. If you count week 17 it's closer to 70%. It's better than all the other models, in fact, including the Vegas lines. It has been now for 4 years running, although my first year these were unpublished, so I have to really say 3. Also, to be fair, it only predicts games starting in week 4, because it needs some data to crunch. I'm also 1 game behind the consensus in the playoffs so far. <br /><br />I realize it's unpleasant to toot my own horn like this, but considering some of the criticism lately, my hand is forced!Brian Burkehttps://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-18305389946322802652010-01-21T17:06:11.454-05:002010-01-21T17:06:11.454-05:00If I'm not mistaken the model is about 70% acc...If I'm not mistaken the model is about 70% accurate (excluding week 17) this year. That is better than probably any media expert has been this year. Its even better than most models at thepredictiontracker.comAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-33369505860584211202010-01-21T16:38:42.815-05:002010-01-21T16:38:42.815-05:00Sorry Brian,
Didn't mean to typo your name. ...Sorry Brian,<br /><br /> Didn't mean to typo your name. Really enjoy the site.<br /><br />ErikAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-63969650785175460552010-01-21T16:37:44.690-05:002010-01-21T16:37:44.690-05:00Thanks Brina (re:Dome Teams Playing outdoors). I ...Thanks Brina (re:Dome Teams Playing outdoors). I guess my question is that Leaguewide it may be a pretty small effect, but would there be a way to determine if some teams are more affected by it than others. i.e Team a--playing on grass away makes them 15-% more likely to lose than on turf away, whereas Team B is equally unlikely to lose away whether on grass or on turf. I know these numbers are likely way to high, just wondering how evenly distributed effects like this are.<br /><br />Thanks,<br /><br />ErikAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-75302346113603430182010-01-21T16:26:54.874-05:002010-01-21T16:26:54.874-05:00James-They're largely Jets fans, so they have ...James-They're largely Jets fans, so they have emotional attachments. They are essentially searching for validation for their hopes, and a post like mine is not what they want to read. (I confess I do the same thing for my own team.)<br /><br />I was really nervous this year about the predictions because they're featured in the Times. I thought the model would finally have an off year and I'd be humiliated. But luckily it had its best year so far in terms of accuracy. I'm not sure what else would convince skeptics.<br /><br />I have to say I do respect their skepticism. They're basically saying, "What makes you so smart, mister?" That's a good thing. Also, keep in mind that there is a bias in comments on sites like mine. People are far more compelled to speak out when they disagree than when they agree.<br /><br />I try not to respond in the comments at the Times. I'll answer questions, but only rarely defend against criticism. I figure I've had my say. (On my own site I will engage the morons who make misleading or ad hominem attacks. It's kind of a sport to me.)Brian Burkehttps://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-21222926135350306052010-01-21T16:08:36.486-05:002010-01-21T16:08:36.486-05:00Good point. It's true that dome teams playing ...Good point. It's true that dome teams playing outdoors in the cold are at a severe disadvantage. A small part of the league-wide HFA comes from that effect alone. <br /><br />In this game the Vikings are in a dome, so I don't expect it to be a very big factor, maybe a percentage point or two less than league-average. In other words, HFA is worth +6% points rather than +7 or 8% points.Brian Burkehttps://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-65463038063167073732010-01-21T16:06:22.634-05:002010-01-21T16:06:22.634-05:00The 5th down readers just don't get it, do the...The 5th down readers just don't get it, do they? "The Vikings and Jets won, so everything you say must be wrong! And yes, we only look up when you were wrong and harp on that, and completely ignore when you were right!"Jameshttps://www.blogger.com/profile/01838293735141324662noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-28788045413792607842010-01-21T16:04:37.382-05:002010-01-21T16:04:37.382-05:00I wouldn't be so fast to say that. Cardinals f...I wouldn't be so fast to say that. Cardinals fans were all over me last week for the same thing. But one game doesn't prove me right, and this week's Vikings game won't either. <br /><br />This model is very well calibrated, although this year it seems to be a few points overconfident. Case in point, in the 42 games where the favorite is predicted 71%-80%, it was right 69% of the time. So 20% might be too low. It might be more like 25 or 26%. Laughably bad result? <br /><br />But even 20% is not zero. There is a fair chance we'll see the Vikings win.Brian Burkehttps://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-66335853442092860372010-01-21T16:02:20.736-05:002010-01-21T16:02:20.736-05:00Brian,
I know you factor in home away in your s...Brian,<br /> I know you factor in home away in your statistical model, but I am curious is there any way to see how much of a difference field type/surface makes for different teams. For instance, the Vikings have looked (seemed) like a much different team on natural grass surfaces than on the carpet. They seem to be built for speed and lose some of that advantage on natural grass, especially grass in poor condition. I know the Vikings this season are likely to small a sample size for any significant correlations to be made, but it would be interesting to see on a larger sample size if there are teams who do better with certain field types than others. Or if any such discrepancies would fall within normal random distributions.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-82598607081084197832010-01-21T15:47:04.049-05:002010-01-21T15:47:04.049-05:00If your numbers show the Vikings having a 20% chan...If your numbers show the Vikings having a 20% chance of winning the game in New Orleans, I'm not sure you should be comfortable with them. That's one of those results that really should make you question your methodology, because it's just a laughably bad result.<br /><br />Don't get me wrong, I'd probably pick the Saints to win. But the Vikings are just not a 4:1 dog in this game.Anonymousnoreply@blogger.com