tag:blogger.com,1999:blog-38600807.post3661850035813663949..comments2023-11-05T04:16:44.937-05:00Comments on Advanced Football Analytics (formerly Advanced NFL Stats): The Randomness of Win-Loss RecordsUnknownnoreply@blogger.comBlogger25125tag:blogger.com,1999:blog-38600807.post-4789167149803787222014-07-29T10:55:31.927-04:002014-07-29T10:55:31.927-04:00Hi Brian,
I'm trying to apply your analysis h...Hi Brian,<br /><br />I'm trying to apply your analysis here to National Rugby League (NRL) games in Australia. In the NRL there are 16 teams that play 24 games over each season.<br /><br />What would the theoretical binomial variance be here for this competition?<br /><br />I used a binomial variance calculator using 24 trials with p=0.5, which gave me a variance of 6.0. I take that as equaling 0.25 (6/24). Is this correct?<br /><br />MarkAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-21370694119634469652013-12-03T20:52:18.415-05:002013-12-03T20:52:18.415-05:0025% would be closer to the truth, not 24%.25% would be closer to the truth, not 24%.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-78931960836487217512013-10-08T16:48:15.748-04:002013-10-08T16:48:15.748-04:00Hey Brian,
Pats fan here so take this for what it...Hey Brian,<br /><br />Pats fan here so take this for what its worth, but I don't believe the pats with Brady and Belichick have the variance you describe with regard to wins from one season to the next. Neither do Peyton Manning led teams, would you say that 58% is a bit low in some instances?<br /><br /><br /><br />Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-17660078988259930742012-11-15T11:58:09.248-05:002012-11-15T11:58:09.248-05:00Hi, Brian. I wrote something that disputes a few o...Hi, Brian. I wrote something that disputes a few of the things you are saying here. Thought you might be interested to see it:<br /><br />http://www.reddit.com/r/NFLstatheads/comments/138skh/original_research_luck_predictability_and_the/kloverrnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-35858147758529260162011-12-14T18:14:07.277-05:002011-12-14T18:14:07.277-05:00Comments that don't match reality aren't a...Comments that don't match reality aren't any good either. <br /><br />First of all, there is no predictor who can reliably and consistently predict NFL winners > 70% of the time. Unsupported claim.<br /><br />Second, a "favorite" team will win by luck 50% of the time. In other words, the starting point for any model is 50% accuracy, not 0%. So a predictor that is 70% accurate is only adding 20 percentage points of knowledge, not 70.Brian Burkehttps://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-86692244811120046452011-12-14T14:38:16.515-05:002011-12-14T14:38:16.515-05:00Math models aren't very good if they don't...Math models aren't very good if they don't match reality. Those who predict outcomes in NFL games can routinely be correct over 70% of the time (season after season, i.e. with larger sample size). So randomness cannot account for as much as this article indicates.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-17285937670076155382010-11-30T11:44:33.445-05:002010-11-30T11:44:33.445-05:00Brian--
I just applied some of the concepts here ...Brian--<br /><br />I just applied some of the concepts here to the Lions' current season, if you're interested.<br /><br />http://www.thelionsinwinter.com/2010/11/detroit-lions-nfl-and-luck.html<br /><br />Peace<br />TyTy Schalterhttps://www.blogger.com/profile/07228238666916939672noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-88117648732788903862010-11-16T15:33:56.714-05:002010-11-16T15:33:56.714-05:00I think a great deal. The better team actually win...I think a great deal. The better team actually wins more frequently in football than in baseball. It's just that baseball has 10 times as many games for all the breaks to even out.Brian Burkehttps://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-8836522101573366282010-11-16T00:11:13.011-05:002010-11-16T00:11:13.011-05:00Brian, dunno if you're monitoring these older ...Brian, dunno if you're monitoring these older threads, but how much do you think the apparent "parity" of the NFL is due to this effect -- for example, as indicated by postseason success -- given the randomness of a single-elimination tournament format?bmoore_uclanoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-44974008122946464942010-11-11T07:28:14.645-05:002010-11-11T07:28:14.645-05:00Randomness and Luck vs Free Will
I like the overa...Randomness and Luck vs Free Will<br /><br />I like the overall concept ANC approach here and in the prior, linked posts. <br /><br />The comments on transitivity and ranking issues got me thinking. The use of a coin flip as an example obscures the reality that these aren't "start them, stand back and let them happen events." <br /><br />How much of the variance is the result of coaching? (I don't like military analogies but...) smart bomb vs World War 2 era bomb.JMMnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-25577666605642445902010-11-10T14:47:13.027-05:002010-11-10T14:47:13.027-05:00Adam-very true. A game is the sum of smaller parti...Adam-very true. A game is the sum of smaller partially random event. And those events are the sum of even smaller partially random interactions, and so on.<br /><br />Have you seen one of those pachinko-machine exhibits at a science center or in school that demonstrates the normal distribution? (It's actually the binomial distribution, by the way). In them, a series of balls are dropped from the top and bounce off of several evenly spaced pegs on their way to the bottom where the balls collect in bins. The number of balls in each bin will conform to a binomial distribution because they are the sum of several semi-random interactions.<br /><br />Anyway, each layer of pegs can be thought of as another layer of a binomial process--game, play, player, etc.Brian Burkehttps://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-83663171482499701642010-11-10T14:40:40.309-05:002010-11-10T14:40:40.309-05:00weisteinium-41% of the variance in NFL season-long...weisteinium-41% of the variance in NFL season-long w-l records can be accounted for by sample error. That's across the league as a whole.<br /><br />Variance is a somewhat abstract concept. That's why I put things in terms of a correlation coefficient, r, which came to 0.75 for team ability.<br /><br />Hope that helps.<br /><br />So individual teams may have a far larger chance of winning a selected game depending on the opponent and whether it is a home game.<br /><br />What this suggests in terms of prediction models is that even a perfect model could not exceed 75% accuracy by very much.Brian Burkehttps://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-11039342603444906942010-11-10T14:25:12.238-05:002010-11-10T14:25:12.238-05:00Everything I learned about statistics comes from r...Everything I learned about statistics comes from reading football sites on the web <br /><br />What does this mean for game predictions? If 41% of a team's record is due to randomness than wouldn't you expect that you couldn't predict the winner of any given game very accurately? <br /><br />How can your model say that one team has a 90% chance of winning when 41% of the result is due to randomness (and thus can't be predicted)?weinsteiniumnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-4393590297990645692010-11-10T13:57:22.358-05:002010-11-10T13:57:22.358-05:00Sorry I made a small error with my calculation. In...Sorry I made a small error with my calculation. In fact, team A needs only a 50.9% chance of winning each coin flip to have a 57% overall chance of winning the game. And with just a 53.3% chance of winning each flip, team A would have a 75% chance of winning the game.Davenoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-87483207467310370262010-11-10T13:34:52.962-05:002010-11-10T13:34:52.962-05:00Adam, I think your idea is interesting. I think y...Adam, I think your idea is interesting. I think you could probably come up with a crude model that works like this: treat a football game as 101 (to avoid ties) weighted coin flips, where team A has the same probability X of winning each coinflip. Whichever team wins more coin flips wins the game. Suppose that, based on Brian's models, team A has a 57% chance of winning the game (perhaps the teams are completely even and team A is at home). That means that team A's chance of winning each individual flip (X) is only 51.9%, much closer to 50% than their chances of winning the game.<br /><br />So think of each play as a coinflip that the favorite has just slightly better than a 50% chance of winning- over time those odds add up.<br /><br />For instance, using this model, a team with a 75% chance of winning the game has just a 54% chance of winning each play or coinflip.Davenoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-15592182155462581942010-11-10T11:56:12.100-05:002010-11-10T11:56:12.100-05:00I was thinking about this a bit more and I have a ...I was thinking about this a bit more and I have a question about probability that I've had a hard time sorting through in my mind. I understand very well the problems of small sample sizes. And I know that if you flip a coin 16 times, more often than not, you will not get a perfect 8-8 record. <br /><br />But 16 games are not analogous to 16 coin flips. They are more analogous to 16 buckets of ~150 coin flips. In other words, a game is the sum result of many different plays. I know that it's not a simple equation - just because you "win" more plays than you lose doesn't necessarily mean that you will win the game. But wouldn't the deterministic value of 16 games (or even 10 games) be worth much more than the deterministic value of 16 coin flips?Adam Davishttp://blog.insoftusa.comnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-5710039891695275152010-11-09T23:26:21.864-05:002010-11-09T23:26:21.864-05:00Actually, I don't know if the constraint is im...Actually, I don't know if the constraint is important--if it's at the sqrt(256/255) end, then it isn't. I'm reasonably but not completely sure that the effect on the league variance depends on the network structure of which teams play each other. But you have done precisely the simulation I had in mind, including that network structure, so you could numerically work it out--just calculate the variance you see in that simulation (I didn't see the number given anywhere in the posts) and compare to 0.125^2 .<br /><br />Though you phrase the question here in terms of the relative level of perfect luck and perfect skill, personally I would phrase the question a little differently, and in two parts. The first part is--measured in some assumed true-and-constant underlying "power-ranking" type units like points, what is the variation among NFL teams in those units? The second part (and the somewhat trickier concept) is, suppose the true power ranking for a team were even known, what is the expected variation in point-spread outcome if it were to play the same (league-average) team over and over? The "excess variance" of wins-and-losses can be backed out via simulation with these two numbers, essentially it will work out to be a (nonlinear) function of the ratio of those two numbers. <br /><br />FWIW I peg these numbers (the standard deviations) at about 5 points and 14 points, respectively.Andrew Folandhttp://nuclearmangos.blogspot.comnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-51273728172170588112010-11-09T22:18:20.985-05:002010-11-09T22:18:20.985-05:00Like borrowing interceptions from the future? :)
...Like borrowing interceptions from the future? :)<br /><br />The constraint you mentioned, which I hadn't considered and I'd agree should be important, doesn't appear to have a large effect. I did the simulation you suggested a few years ago (linked to in the post above) and got very similar results (r of .75). But I'd like to think about it more...Brian Burkehttps://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-83807167805865741512010-11-09T22:05:07.884-05:002010-11-09T22:05:07.884-05:00The variance for 16 random 50/50 wins is indeed as...The variance for 16 random 50/50 wins is indeed as calculated above. However, this is really the variance for a single average team out of the league, over the course of many years playing other average teams.<br /><br />However, the number you take from the data is over the whole league (averaged over 10 seasons.) Over the whole league (of average teams with no HFA) in a given season, there is an additional nontrivial constraint--namely that every game has exactly one winner and one loser. This is a little different from 16 independent coin tosses for each team, as 16 independent coin tosses for each team effectively allows games with two winners and two losers.<br /><br />The full league will have various strong correlations among sums and this will change the variance. One could probably calculate in closed form what the effect is, though it would probably be simpler to simulate (in Excel for instance.) For what it's worth, the effect on the variance is surely between a factor of sqrt(2) and sqrt(256/255) but without putting actual effort into it I personally can't nail it down any more tightly--though maybe one of your other readers can. <br /><br />For the QB study you plan to do next, I think you're going to find a very surprising result. As far as I have ever seen, at least over single seasons there is generally no discernible effect outside the binomial variance. If you're not careful, some seasons you might even find imaginary standard deviations :)Andrew Folandhttp://nuclearmangos.blogspot.comnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-48438467988261322262010-11-09T17:52:36.154-05:002010-11-09T17:52:36.154-05:00Sorry, HFA=.54, not .50.Sorry, HFA=.54, not .50.Temonoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-49660273883045739872010-11-09T17:51:22.830-05:002010-11-09T17:51:22.830-05:00Anon- The variance of wins in a binomial process w...Anon- The variance of wins in a binomial process with n=162 and HFA=.5 is: 162*.54*.46= 40.24 wins, so seeing a team win 95 games is definitely possible.<br /><br />When I do the same thing as above for baseball, I get 29.1%, and the observed variance of win totals is quite large (138.14 over the past 10 years).Sunparnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-90853452013834271992010-11-09T17:20:25.145-05:002010-11-09T17:20:25.145-05:00Good point above. Accounting for the effect of hom...Good point above. Accounting for the effect of home field advantage (HFA), the answer is 41% randomness, 59% ability.<br /><br />In the NFL, HFA makes the home team a 0.57 favorite. <br /><br />The variance of any binomial process is np(1-p). So the random variance of a 16 game season should be 16*0.57*(1-0.57) = 3.94. <br /><br />The actual variance in win totals since 2000 is 9.60. <br /><br />3.94/9.60 = 41%Brian Burkehttps://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-53638379132883746572010-11-09T17:03:01.874-05:002010-11-09T17:03:01.874-05:00A famously remember a simulation where 162 game ba...A famously remember a simulation where 162 game baseball seasons were played out with 30 teams that had no statistical advantages (ie, each game should have a 50-50 outcome). You still had successful teams (95 wins) and losing teams.<br /><br />Surprising.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-52541763976106396232010-11-09T16:40:06.406-05:002010-11-09T16:40:06.406-05:00I love the overall calculation that points to 42% ...I love the overall calculation that points to 42% of a team's W/L record being attributed to randomness. But I think it's important to point out that that this 42% is a blanket figure that applies to the league in aggregate. I would argue that only applies to each individual team to the extent that each team has played competition of similar skill. In other words, if a team stunk very, very bad (imagine a high school team being forced to play in the NFL) their record would be 0-16 and that would have almost nothing to do with randomness. The *point spread* in any given game might be more subject to randomness, but the fact that they lost every single game had everything to do with their overall lack of talent. <br /><br />The degree to which randomness determines a specific game outcome should be a function of the relative talent/skill difference between the two teams. So if two teams are perfect mirror images of each other (regardless of whether they are *good* or *bad* teams) and they are playing in a neutral environment, the result of the matchup would be entirely predicated upon randomness. But if one team is extremely good and the other is extremely bad, the subsequent butt-kicking has very little to do with randomness or "luck".Adam Davishttp://blog.insoftusa.comnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-3795469717081737582010-11-09T16:32:33.526-05:002010-11-09T16:32:33.526-05:00I can think of two other assumptions that will mak...I can think of two other assumptions that will make randomness appear to be a larger factor than it likely is in reality.<br /><br />One is home field advantage. Continuing with the assumption that team strength is transitive, if you listed out each team with some sort of relative handicap (<a href="http://www.pro-football-reference.com/blog/?p=7710" rel="nofollow">like here for example</a>), you'll see that for just about every team, there are several teams within a few points. Estimates of home field advantage are typically between 2.5 to 3 points. So assuming that home field advantage is roughly constant across time and stadiums, a team that is estimated to be 1 point better than its opponent would be at a disadvantage on the road. If the better team loses by a point or two on the road in this case, it would count as randomness, even though it could plausibly be explained by home field advantage.<br /><br />The other assumption is that a team's ability stays constant over the course of a season. But our observational experience of injuries and mid-season collapses, suggest that this is not the case. Perhaps it was just random that the Broncos were successful early on, and struggled late in 2009, but the reality is that it's an adaptive league. Once weaknesses of a team are exposed, its opponents can exploit them in copycat fashion. Having information about how to beat the Broncos in late 2009 actually made the Broncos a worse team than they were in early 2009, without significant injuries or changes in game planning/coaching. Granted, it becomes difficult to recognize a mid-season collapse ahead of time, and sometimes even during it.<br /><br />I think randomness plays a large role in deciding NFL winners, but less than is suggested here. Difficult to measure though.Probable Pickshttp://probablepicks.comnoreply@blogger.com