I’ve begun using some new stats here lately, and I want to take the opportunity to explain them clearly. Readership has been increasing rapidly too, and some readers aren't as familiar with the terms we use around here. This post will serve as a reference each time I use the new stats in the future.
Although it’s been used in baseball sabermetrics for several years now, Win Probability Added (WPA) is a new, or at least rediscovered, concept for football stats. It measures each play in terms of how much it increased or decreased a team’s chances of winning the game.
WPA starts with a Win Probability (WP) model of the game of football. Every situation in a game gives each opponent a particular chance of winning, and a WP model estimates those chances. The model created here at Advanced NFL Stats uses score, time, down, distance, and field position to estimate how likely each team will go on to win the game. For example, at the start of the 2nd quarter, a team down by 7 points with a 2nd down and 5 from their own 25 will win about 36% of the time--in other words a 0.36 WP.
On that 2nd down and 5, let’s say there is a 30-yard pass, setting up a 1st down and 10 on the opponent’s 45. Now that team has gone from a 0.36 to a 0.39 WP. The WPA for that play would be +0.03.
If instead the quarterback throws an interception returned back to the line of scrimmage, the opponent now has the ball at the 25, giving the trailing team a 0.28 WP. The WPA for the interception would be -0.08.
WPA is very sensitive to the context of the game. That same interception that cost -0.08 when a team was down by 7 points in the 2nd quarter would cost much more if it the offense was leading by a point late in the 4th quarter. Putting your opponent in immediate field goal range would be nearly fatal.
Stats are tools, and each tool has its own purpose. WPA is what I call a narrative stat. Its purpose is not to be predictive of future play or to measure the true ability of a player or team. It simply measures the impact of each play toward winning and losing.
WPA has a number of applications. For starters, we can tell which plays were truly critical in each game. From a fan’s perspective, we can call a play the ‘play of the week’ or the ‘play of the year.’ And although we still can't separate an individual player's performance from that of his teammates', we add up the total WPA for plays in which individual players took part. This can help us see who really made the difference when it matters most. It can help tell us who is, or at least appears to be, “clutch.” It can also help inform us who really deserves the player of the week award, the selection to the Pro Bowl, or even induction into the Hall of Fame.
As interesting as that might be to us as fans, WPA might be even more useful to coaches and strategists. We can measure whether a kicker’s high field goal accuracy was worth a trade-off in short kickoffs. And for the first time, we can measure the effectiveness of clock management strategies—is it better to run three times and punt, or pass and go for the first down to end the game?
Even better, WPA can measure the risk-reward balance of a given situation. For example, when is it best to take a sack, and when is it best to take your chances throwing into traffic? Or for defenses, when is it best to roll the dice on a big blitz and risk a long completion due to a big hole in the secondary? We can measure which interceptions were the most costly, and which quarterback tends to throw the costliest ones. WPA may actually be able to measure decision-making ability on the field and on the sideline.
WPA is not ‘the one stat to rule them all,’ and it does have its disadvantages. It can certainly be improved in several respects, and I’m already working on several ideas. As time goes on, we’ll find even more new and interesting applications for it, and this is only the beginning. More than any other sport, football is about strategy, risk, and reward, and WPA is well suited to capturing much of what makes football so compelling.
- Home
- win probability
- Win Probability Added (WPA) Explained
Win Probability Added (WPA) Explained
By
Brian Burke
published on 1/27/2010
in
research,
win probability
Subscribe to:
Post Comments (Atom)
nitpicking but second to last paragraph wHOLE
Hi Brian - I've often wondered why you don't include time outs remaining in the WP model, since they seem like they would be very important to 4th quarter scenarios. Is it a correlation issue because teams almost always have all 3 in the first and third quarters, or is it a sparseness issue where there aren't enough 4th quarter data points?
It's sparseness. I'm working on some methods now. One idea would be to equate a timeout to a certain amount of time added or subtracted if on defense or on offense.
I'm not so sure about it being able to tell us about the individual player. Chris Johnson appears to be a great running back but if he were on the Seahawks there's no way his WPA numbers would be anywhere close to as high. I suppose you could compare players on the same team but even this runs into the problem of game theory (ie. Johnson is hurt so the Titans opponent defends more against the pass which in turn makes the backup RB look better than he actually is.)
First off I would like to say I am a big fan of this website. Keep up the good work Brian.
This may be a little off topic but, I have noticed in many of your posts you use WP to calculate how crucial a play was for team X to win or whether a coach Y made the right call. On these posts you often have to qualify your assessment by saying WP is based on league average stats. This is also a point many of your critics often use to discredit your work. I have very little education in statistics, but I have always wondered if it is possible for you to develop a "matchup coefficient" that I guess would be some kind of ratio of the strength of one team's offense vs the other team's defense on any given play type. In other words the next time there is a controversial 4th down between a high ranked rushing team vs a poor rush defense you could factor that in to your equation to make going for the 1st more preferable.
If such a variable were possible you could make your predictions more specific to each situation and I hope it will quiet some of your detractors who feel your calculations are not in line with their gut feeling. That kind of commentary was especially bad on the Belichek 4th and 2 topic last November.
The problem with the 4th and 2 detractors is that nearly all of the situational adjustments (i.e. Manning is awesome, the Pats defense was gassed, Brady had been pretty efficient) point MORE toward going for it.
Anyway,
Brian, I agree with you entirely when you say "WPA is what I call a narrative stat. Its purpose is not to be predictive of future play or to measure the true ability of a player or team. It simply measures the impact of each play toward winning and losing." As long as it is used and treated as such, it's a meaningful stat.
I suppose in the most literal reading, this makes using WPA for MVP a pretty meaningful thing to do. However, this is why I think that, for example, the EPA measurements of punters were far more meaningful than the WPA measurements. The punters don't really have much control over the game context, so the relatively context-free EPA tells us more about their value.
Derek
Or alternatively show that no matter what offense/defense combination is on the field, the chances of converting e.g. 4th and 2 are pretty steady across the league.
One shouldn't change the facts to match the theory (see Climategate). Just because people may have a gut instinct about these things, the stats may well show something completely different (e.g. the WP model shows that statistically teams should go for 4th and 1 pretty much anywhere on the field, in marked contrast to commentators who 'know' that you never go for 4th down in your own half and are even dubious about 4th and 1 inside the 'maroon zone').
That said, it would be interesting if team-specific adjustment could be made, if not for WP then perhaps just for 3rd and X conversion rates.
Very interesting stuff. I was looking through the wpa and epa for players last year, and there's a piece of data missing that I'd very much be interested in seeing (and hopefully you'll be interested enough to do the work for me!).
I figured that for running backs epa is probably more important than wpa because running backs tend to run more at times when wpa won't be affected (eg at the end of a blowout).
For running backs especially (but for the other skill positions as well), I'm interested in knowing what their epa/play are depending on the particular win probability of the game. I was thinking if you made a graph of all running backs and had epa/play on one axis and win probability of the game on the other axis, what that graph would look like. Since running backs are often given the ball to let the clock run down when the team's win probability is extremely high, I wonder if that will change the stats of a running back on a good team vs a bad team. But in any case, I was hoping you could not only put up epa stats, but epa/play stats within ranges of win probability (for instance, epa/play when win probability is between 30 and 70 percent...ie when it's still a game...the specific limits could be determined by looking at the curve of the graph mentioned above).
I think that would provide novel information for the stat.
Also, an adjustment for the quality of opponents would be interesting.
Also, for quarterback stats, I was curious to know what counted as a "play." I divided epa by epa per play and couldn't figure out what that number of plays referred to. I would assume for quarterback it's all plays that aren't handoffs? Does it include penalites (which I think would actually be a good idea)?
Thanks for a fantastic site, hope you find my comments worthwhile.
Sorry, one last thing. For receivers, I think it would be a good idea to have another stat that's epa/play where the "play" isn't a pass thrown to them, but rather any passing down in which they were on the field. Since getting open so that you're passed to is just as important as making a catch.
PS: I also can't figure out what counts as a play for a running back. I added up carries and number of times thrown at and it's still short by a few dozen plays. Penalties?
Sorry, one last thing. I think for receivers it would be worthwhile to have another stat that's epa/play where play is any passing down in which the receiver participated. Since getting open is just as important as making the catch for a receiver.
PS: how did you calculate plays for running backs. I added carries and targets for receptions and it doesn't add up. Penalties?
Anonymous said...
nitpicking but second to last paragraph wHOLE
Either it was fixed or you are one of the dumnest people of all time
"Dumnest...?"
Yeah, you tell 'em, Einstein :P
There's been a lot of speculation about the statistical benefits of always going for it on 4th down. Where does WPA come out on that?
Greetings, Brian
I wish to congratulate you for all the great work you have done in developing WPA. Undoubtedly, WPA is an immensely valuable tool in analyzing the game’s situational strategies. Its value in this respect should not be underestimated. Again, I congratulate you for your great success in this regard.
When WPA is applied to evaluating a player’s value, however, I believe WPA offers very little utility unless the time-sensitive feature of the model is omitted. I submit that the nature of the game is such that an individual player’s true value cannot be truly discerned by any quantitative method that seeks to add the parts of a game into a quantifiable whole. While I concede that the value of a player’s contribution may loom especially large AT EXACTLY THE MOMENT IT OCCURS in a game, the story is altogether different when the game is over. When the game is over, it is no longer possible to break it into parts to determine the relative significance of each play. Instead, the game can only be taken as a whole. Plays that happen to occur early in the game are just as determinant to the outcome as plays occurring at the finish. That is the unfortunate reality.
In that case, we have EPA.
Hey Brian,
I'm a new reader and I really love the site. I recently read Moneyball (the book about sabermetrics in baseball) and I'm a huge fan of the application to football. I have a quick question that sort of relates to what Scott said above about putting Chris Johnson on the Seahawks. How does WPA interact with offensive lines? Under statistics on the website you have WPA and EPA for offensive lines, but how do you know when to attribute WPA to offensive lines as opposed to RBs (or QBs or WRs on pass plays, for that matter)?
Please respond, I'm very interested in how it works. Thanks so much!
I too am appreciative of the effort and thought that goes into this site, and look forward to advances in these techniques. Like some others i too am a bit skeptical about the ability to translate WPA from team results to an individual. The example given to explain WPA is the impact of a 30 yard pass under certain conditions. Certainly there is an impact, but can you use that to "rate" the play of the qb? Some teams virtually never throw a 30 yard pass. And that is not always due to the quarterback's abilities. Sometimes it is a matter of offensive philosophy. Alternatively the team might want to throw long passes but know that their tackle tandem can rarely hold off a rush long enough to set up a long pass. So the ability to complete a long pass reflects the whole team's capabilities as well as even those of the coaches. In the end differences in WPA for different qb's may say nothing about their relative abilities but more about the relative abilities of the teams on which they play.
Other articles on this site indicate that more passing equates to more wins. It would seem then that teams that pass more would increase the WPA of their qbs. Comparing the WPA of such a qb to a qb on a team that emphasizes running more would tend to make the qb on the "running team" look less good, even though the difference might reflect differences in the extent to which those teams coaches understand the relative value of running vs passing. No? I think this is the similar to the comment made by the third post in this thread.
I agree. But there is a distinction in the purpose of various statistics. A 30-yd pass has many components, influences, some of which are random. However, no matter how you look at it, QB Steely McGunner actually threw that pass, and WR Speedy O'Grabby actually caught it and ran it for 30 yds. In the immortal words of Ricky Bobby, "That just happened."
In other words, WPA captures the narrative of what happened and puts it into a number. It does not divide credit or blame among players. It does not pretend to predict future performance.
We say 'so-and-so made a big play' all the time. WPA just puts a number on 'big'.
But it's more useful than that. Were Barry Sanders' frequent tackles for losses worth his explosive gains? WPA would tell us. Were Kurt Warner's turnovers in STL worth his big arm? WPA tells us. Are Adrian Peterson's fumbles worth his big runs? WPA tells us. We no longer have to conjecture about these things.
Do these points get distributed evenly - i.e. on a pass play that increased WP by .03, does the QB and WR get .03 points?
How about a DB, if he knocks away a pass that would have increased the offense's WP by .05, does he get the .05? I'm guessing no because that doesn't show up as a stat in game logs.
How about something like a hurry on a QB by a DE or LB that results in an incomplete pass - do the defensive players get those "avoided" WPs?
There's no perfect way to do it. In my implementation, I give full credit for the play to all those directly involved. For example, say there is a play in which a QB passes to a WR, and that play takes the team from a 0.40 to a 0.50 win probability. The play as a whole is worth 0.10 WPA. Both the QB and the WR get credited for the full 0.10 WPA (just like both get credited for the yards gained in conventional stats).
"Were Barry Sanders' frequent tackles for losses worth his explosive gains? ... WPA tells us. We no longer have to conjecture about these things."
If WPA is calculated on league averages, would it tend to overstate the pain of Barry Sanders' frequent tackles for losses? The reason I think it would is that it's not so bad to get tackled for a loss if you have Barry Sanders on your team, who is likely to follow that up with a long run for a first down. The WPA is going to tell you how much it would hurt an average team to have a two yard loss in this situation, but this particular team might be better able to absorb that two yard loss than another team. Is that a reasonable assessment, or am I missing something?
Separately, I was interested to know if you publish the details of the WPA model somewhere, or is that kept proprietary?
Is there a card/chart like the "when to go for it on 4th down" chart that tells you the expected points earned on offense on average when in certain down and distances for first, 2nd 3rd and 4th down?
Like so you have 3 chart for each down showing the number of expected points for each down.
1st and 15/1st and 10/1st and 5 on each yardline (5 yard false start or 5 yard encroachment penalty). I.e. 1st and 10 on 20 is worth 1.6 points, 1st and 10 on the other 20 is worth 5.3 points (or whatever) but there would be a table or chart showing the expected point values for 1st on 10 on each yardline. Then 1st and 5 for each yardline and first and 15 on 3each yardline.
For other downs it would be averaged to less specific.
2nd and long (8+ yards), 2nd and medium (5-7yards), 2nd and short (1-4 yards), 2nd less than 1
3rd and long,medium,short,inches
4th and long,medium,short,inches
The short yardage would average point value of 2nd and 1, 2nd and 2, 2nd and 3, and 2nd and 4, or else consider the average points scored or whatever. and what is the average probbility of turnover on each yardline on a given play?
If I had this information I could come up with a ton of great hypotheticals regarding football strategy. i.e. Is a west coast offense better than a vertical passing attack? What must the completion percentages and chance of turnover be to match a 70% short yardage passer with a 3% chance of interception?
I feel there are a lot of conditional probabilities that would be more accurate if they took into consideration "variable change" I.e. Although an average team should gain a greater probability of winning by making a certain less aggressive decision, a team with a weak offense, but powerful defense against a very weak offense that probably isn't going to score on a long drive anyways and doesn't have a great punter might be better off throwing deep hail marries on ever play and even if they are intercepted it is perhaps just as good or better than running 3 plays and punting anyways. There may be alternatives such as throwing screen passes, medium passes and trick plays. Additionally, there may be the opposite situation where an INT is much more costly because your offense is very likely to move into scoring range anyways, but the defense won't stop anyone so a turnover will almost surely cost you more points than most teams. Teams like the '98 Vikings couldn't stop anyone on defense, but their offense was so good they would outscore them and they would be able to go vertical anyways, so they wouldn't need all WRs in the same spot on a jump ball and would be much better off having 4 vertical routes spread out with one breaking it off and reading the safties and which side they cheat towards and going the other direction.
Additionally, one major variable change that you may not consier, is although running the ball may not be as effective or advisable as passing as moften, if the defense only has 5 guys in the box to your single back formation, it would be much more favorable to run the ball. That is an issue that has to be considered, and down and distance and the other metrics such as win probabilities that may suggest running or passing the ball may not be correct against a certain presnap look. Because of conditional probabilities I think calling plays at the line is much more effective, but what win probability could show us is looking at a given situation against a given number of men in the box or given presnap look (1 single free safety deep or 2 or perhaps 1 and corners back 9 yards, or 2 and corners back deep) is how it might change based upon the defense. I bet if defense shows a cover 4 look with 4 secondary memers back deep, that runs would be more effective and short passes would be while deep passes would be less effective. In cover 2 or cover 3 deep passes would be more effective and short passes would be less effective. Football could be turned into more of a chess match if people had this kind of in depth information and actually had the probabilities and used advanced statistics, but I think we could take it much farther based upon isolated situations and there probably is enough data available to learn it, and also to learn how much more points a situation might be worth with the best offense against the worst defense so you can keep your estimations and calculations in check. A statistician could probably come up with a much better game plan but the head coaches still have intuitive sense for what coaches might do and be better at coming up with contingency plans, but they would need the advance stats to optimize it, and game theory to vary their play.
Hi Brian, on one of your posts explaining how you derived the WP model you stated that:
"... I use chunks of data, so depending on the situation, I'll average a block of 20 yards of field position and up to 5 minutes of time. Then I'll interpolate between "chunks" for the particular win %. There's a lot of sophisticated modeling and smoothing going on underneath the raw win%."
Would you be able to share how you divided the chunks of yardage (e.g., 0-20 yards from own goal, 21-40 yards from own goal, etc.) and how you divided the minutes remaining (e.g., you said up to five minutes but I suspect you used smaller chunks of time at the end of the game)?
I'd really like to understand better how you structured the data. Thanks!