Weekly game probabilities are available now at the nytimes.com Fifth Down. This week I look at the Jets-Patriots match up and discuss why great offenses tend to beat great defenses.
- Home
- Unlabelled
- Weekly Game Probabilities
Subscribe to:
Post Comments (Atom)
I'm positive this has been asked before, but I'm hoping you looked into it during the offseason. Have you considered investigating to see if the interaction of offense and defense have a better predictive power than just GWP? The obvious ones are OffGWP vs DefGWP and the more specific OPass vs DPass.
Looking ahead a bit I'm waiting for the Pats #1 offense vs the Cowboys #6 defense, and the Cowboys #2 offense vs the Pats #28 defense. But it occurs to me that the Cowboys' offensive advantage may not be as significant as those rankings indicate, as defenses are grouped closer together.
#9 Pitt has a 57% win probability over #5 Tenn....
I'm not sure I understand.
Home field advantage for Pittsburgh
HFA came out slightly stronger in the new coefficients, by the way.
Both teams are hurt, but I have to think the Kenny Britt injury is extremely costly for TEN. In his 1st 3 games, he accounted for over half the offense's EPA. Nate Washington seems to be pulling some weight though.
It is good to know that SF has A. Smith backing up Alex Smith. If Alex Smith is hurt there will be no drop off with A. Smith.
I guess this is my way of saying that for whatever reason you have two entrees for the same person. Check out this page for example: http://wp.advancednflstats.com/matchUp.php?vteam=TB&hteam=SF
In addition to QBs disproportionately impacting games, I wonder if the nature of offenses inherently are more likely to dictate games. For example, perhaps the tackling rates between good and bad defenders have a smaller standard deviation than the ‘juke’ rates of running backs. I suspect some kind of standard deviation analysis could be done.
I am not espousing that this is true, just speculating that it could be the case in football, if not other sports.
do you have playoff odds posted somewhere? where does that 60% figure come from?
It's difficult to accept the numbers/regressions you've run, without knowing their performance over the first few weeks.
As an aside, because football has fewer star players that dominate outside the QB position, I would bet top dollar that an injury to a player like Kenny Britt would cost Tennessee less than your model suggests.
It seems that the best prediction market in the world--consensus NFL money lines agrees with me (as it has Tennessee as a bigger dog than your model implies, at +153 *with vig* vs. your +132).
We shall see, but I'll be examining the results your biggest disparities for this week (compared to NFL MLs), and seeing how your predictions fare.
For your model's sake, Go:
Panthers +248 (+144)
Buffalo +127 (-212)
Colts -128 (-177)
San Fran -143 (-233)
Jets +365 (+185)
Denver +171 (-138)
Also, the biggest test of whether a model is predictive, is whether opening lines move in the same direction as the model suggests.
In this case, for reasonable disparities in your predictions, only Tennessee and San Francisco seem to indicate a strong model.
Of your other largest margins of difference, the Panthers, Bills, Colts, Cardinals, Jets, Broncos, and Lions seem to not be moving or moving against you.
I would make a large bet that your model is not predictive based on that information. Let me know if you'd take me up on that.
I'll take you up on that. How do opening lines give any indication to the validity of a completely independent model? Opening line movements reflect the bettor's opinion on the starting line. More specifically, they reflect the bookie's desire to make money.
The only way I can see them interacting is if every bettor saw the model's predictions and they placed their bets thereafter (bets influenced by the numbers the model spit out). Even then, it has absolutely no relevance to how predictive the model is.
Opening lines indicate a baseline. Almost everyone would agree closing lines are sharper than opening lines, and I don't know anyone who thinks it's easy to beat closing lines in the NFL. There are probably only a handful of people in the world who do.
If a model indicates that a line should be a good value, then it would validate the model to have sharp money push the opening line toward the model's line by game time.
In this case, the lines are way off, and not moving toward the model's estimated lines. That indicates to me that the model likely has no predictive value.
In fact, if Mr Burke would like to wager, I propose this:
Whenever Mr Burke's model would indicate on closing lines to bet on a side (I describe some such scenarios above), I bet $100 that backing those sides over the course of the rest of the season would be unprofitable at any single bookmaker's lines (my preference would be to BetCris or Pinnacle).
Chris is only partially correct. Lines move during the week, and closing lines are more accurate than opening lines. But this is primarily due to injury information, not because they are converging on some sort of cosmic truth.
To be fair to any purely statistical model, you should compare it only to the opening odds. Because at that point, neither system has access to the upcoming week's injury information.
"Primarily" is vague. Injury reports certainly move lines, but all lines are also moved by sharp action.
Nonetheless, I would take the same bet against CRIS's openers, despite the fact that both of us know that openers can be beaten.
That said, your lines are so far off of openers/closers etc. that you know your model is unlikely to be predictive.
The proof of the pudding is in the eating, and if you want to prove your model works, we can make the bet. I suspect that you know enough to decline.
That said, your blog is great, and the look at statistics is relevant to teams, managers, coaches, etc. However, it is certainly not relevant to predicting actual game outcomes beyond in general, at least in comparison to betting lines.
Looks like I'm going to have to come up w/ a new moniker, because I'm not nearly so aggressive or antagonistic.
Frankly, by definition, Brian's model works for what it is intended to do. If you've paid attention to anything he's ever said, he's not made the claims you seem to be placing on him.
So, yeah, anyway, just wanted to clear thing up on which Chris was speaking.
"If a model indicates that a line should be a good value, then it would validate the model to have sharp money push the opening line toward the model's line by game time."
I'm not sure why you insist on comparing the model to closing lines. Predictive models for NFL football games try to predict the probability of a binary outcome of the game. Closing lines for an independent method of establishing a probability are entirely irrelevant. It doesn't matter if closing lines are better, worse, or exactly the same as the opening lines. There is no linearity between opening lines, closing lines, this model's probabilities or any other model that allows you to rank them in an ordinal fashion. Maybe one week this model is exactly the same as the opening line. Maybe another week it's exactly the same as the closing line. Maybe another week it's unlike either of them. Yet, it can still perform better or worse than the closing line in the aggregate.
Also, how in the world does it "validate the model" if the opening lines move towards closing lines? That would validate a model that predicts closing lines.
Furthermore, if Brian were to accept the bet, his winning or losing the bet also does not validate or invalidate the model. Technically, it can never be validated or invalidated because its output is not binary - by definition, probabilistic models can never be proved true or false. You can only mess around with p values to see the probability of a random sample that looks as extreme or more extreme than the model's ultimately subjective "success rate." Look at the model's history and make your own assessment of how well it has done.
I'll switch to Chris H. to avoid confusion with the Chris who came first.
"if Brian were to accept the bet, his winning or losing the bet also does not validate or invalidate the model"
Maybe so, but if the model is worse than one published in any newspaper, what's the point?
The same thing is true with money managers--if a guy earns 5% in the market, but consistently gets beaten by the S&P 500, then he should save his time and put his money there.
Also, when you say that "by definition, probabilistic models can never be proved true or false", that's wrong.
You can look at how likely results would come in like they actually do on Sundays if a model were correct in picking the true odds. You know that. P values are a quality tool.
In the same way, you could compare that likelihood to another model, say opening/closing lines.
Also, as for the other Chris, you say "he's not made the claims you seem to be placing on him". While he's never made those specific claims, he is publishing game probabilities in the New York Times, which are worse than commonly available game probabilities (betting lines).
Maybe his goal is just to show a different model, but there's just a table there. What exactly does he intend people do with it?
Just as I'm not impressed by a money manager not beating the market average, I'm not going to be impressed by a sports statistician not beating betting lines. Quite simply, this blog is not good at predicting game outcomes, and that's not what it should be. Unfortunately, that's what the NYT wants, and because there is a dearth of sports stats guys, he gets the nod, despite this particular field not being his forte.
Chris H-"he is publishing game probabilities in the New York Times, which are worse than commonly available game probabilities (betting lines)."
"Quite simply, this blog is not good at predicting game outcomes..."
These are false statements. In fact, your entire comment is full of completely false statements.
By my scoring, the model I have published has never been beaten by opening lines, until last year and only very slightly. So that makes 3 years for the model and 1 for the lines.
Even if you assume the most unfavorable scoring method, counting ties or 50/50 games as losses, etc., the model comes very close to betting lines, without knowing a thing about injuries to critical players or other locally relevant information.
The NYT approached me specifically because of the model's accuracy.
Your comments are patently absurd. You are welcome to your own opinion, but you are not welcome to your own facts.
Ok. Now for some facts. This is from an independent source, not my scoring. This uses a relatively unfavorable scoring method, and it includes week 17 junk games (see NYJ-IND '09), in which the betting lines have an obvious advantage. This is for picking straight-up winners.
Vegas opening lines (according to predictiontracker.com) : 66.5%
ANS efficiency model : 67.3%
Admittedly, last year was not the model's best year. Here are the results for 2010 alone, including playoffs:
opening lines (which were actually more accurate than closing lines for 2010!):62.2%
ANS efficiency model: 62.6% (beat it by 1)
"Maybe so, but if the model is worse than one published in any newspaper, what's the point?"
You have done nothing to prove the model is worse. Pointing to the fact that opening lines are not moving towards the ANS predictions is not proof that the model is better or worse than any other model.
"Also, when you say that 'by definition, probabilistic models can never be proved true or false', that's wrong."
It's not wrong. Probabilistic models can never be falsified. They can also never be validated. That is a fact. Call it scientific, mathematical, or philosophical - it's a fact on number of levels.
The question of the accuracy of Brian's model vs. betting odds seems to come up regularly here. It seems like submitting it to PredictionTracker or some other independent model tracking site would put these arguments to rest so we wouldn't have to rely on self-reporting.
Perhaps a complication is that Brian's model doesn't attempt the first several weeks of the season. This would probably give it an unfair advantage (percentage-wise) over other models that predict all 17 weeks. Another comment in recent thread offered suggestions for seeding the model with data from the previous season. That would seem to be a relatively simple way to improve the model's accuracy early in the season.
Publishing the probabilities in the NY Times certainly makes criticism expected and Brian should be prepared to defend them. However, perhaps having the most accurate predictions isn't Brian's top priority with his research. They may be "for entertainment purposes only" and what they teach us about why and how teams win is far more useful and rewarding.
Have you tried adding an effect for teams coming off a bye week?
In the data-set I looked at, if memory serves 1993-2008, road teams coming off a bye won an unusually high 48% which was statistically significant compared to their 41% non-bye win rate. Home teams coming off a bye didn't receive any advantage over their normal 59% win rate from HFA.