I've built a tool for calculating the Win Probability for a given state of a game, and it's now available on-line. I originally built it for myself to streamline analyses of kick/go-for-it type decisions. But I thought if I made the interface user friendly enough other people might find it useful or interesting too.
You just enter the score difference, time remaining, field position, and down and to go distance. The applet returns the win probability for the team with possession along with some other handy stats.
Here is an example of how I'd use it. Say a team is up by 3 points with 2 minutes remaining in the 4th quarter. They're pinned down at their own 2 yard line and are facing 4th and 9. Should they take the intentional safety and give the ball (on average) to the other team at their own 44? Or should they punt, saving the 2 points but handing the ball (on average) to the other team at your own 42, just outside of field goal range?
I'd look at it from the opponent's point of view since they'll have the ball. I'd enter -3 for score difference, 2:00 in the 4th for time remaining, opponent's 42 for field position, and 1st and 10. The resulting WP is 0.37 for the punt.
For the safety, I'd enter -1 for score difference, 2:00 remaining in the 4th, own 44 for field position, and 1st down and 10. The resulting WP is 0.38 for the safety.
Since we'd want the WP for the opponent to be as low as possible, the 0.37 for the punt is the better option, but just barely. The options aren't that far apart. So if the punter gets a bad snap or feels they're a good chance the punt would be blocked, he should just fall on the ball or run out of the endzone. Far better to take the safety in that situation than risk a blocked kick and an easy touchdown.
Another way to look at it uses the probabilities of scoring from each position. Let's make one assumption--With 2:00 remaining, there is plenty of time for the opponent to score from midfield, and if he does, there's not enough time left for you to answer.
If you chose to give up the intentional safety, your opponent has a 0.23 probability of scoring a TD, and a 0.16 of scoring a FG. Since the opponent only needs a FG, chances are he'd stop short of the TD once the FG is relatively assured, so the total chance of scoring is 0.39. Any score will beat you, so the 0.39 WP is very close to the 0.38 we got using the WP calculator directly.
The punt analysis tells a different story. From your 42 yard line, the opponent has a 0.32 probability of scoring a TD, which will beat you, and a 0.22 probability of scoring a FG, which will tie the game. The net WP for your opponent is therefore 0.32 + (0.5 * 0.22) = 0.43. This is a little off from the 0.37 WP we calculated directly. What causes the difference?
The direct WP calculations are based on actual game situations and results--that is, what did coaches really do, and what were the real outcomes? But the scoring probabilities are general and not specific to the particular game circumstances. The difference in the estimated WP suggests that coaches are too timid when behind by 3 at the end of the game. Once in FG position, they'll become very conservative and play for the tie rather than risk a turnover going for the TD and the win. It sounds reasonable...at first.
But a tie only gives you a 50/50 shot at winning, and turnovers--which would always cause a loss--occur far less than 50% of the time. A turnover of any type only occurs less that 12% of the time inside FG range (the 35). Coaches should press for the win as long as time and downs permit.
Ultimately, the safety might be the better option (0.39 vs 0.43) if coaches could actually be expected to play to win. But because coaches can usually be counted on to play for the tie, the punt is the slightly better option.
- Home
- win probability
- Win Probability Calculator
Win Probability Calculator
By
Brian Burke
published on 1/17/2009
in
win probability
Subscribe to:
Post Comments (Atom)
Awesome!
If you don't mind, I wrote something on my website about two-point conversions using this calculator. You can find it here.
Zack--Very cool. What do you think causes the 'shelf' on both curves? I'd guess it has to do with the rhythm of possession. Between 4:30 and 6 min remaining, I'd figure the team that just scored would have one more possession to score, but after 4:30 the chances of getting another scoring drive drop dramatically.
I also wonder how this would compare to how coaches actually decide when to go for 2.
With 4:30-5:00 left, the team with the ball on their own 20 isn't even favored to win if the score is tied--their WP is 44 and 45% in each situation.
If a team is down one, their WP is the same at 4:30 and 5:00, but at 4:00, their WP lowers by 5%, which could mean that the chance of getting the ball back one last time is the highest at around 4:30, and after that the chance of getting one more possession is significantly lowered.
is it possible to show what type of play would be optimal in a particular situation?
I was thinking about two unrelated topics:
1) Is it possible to determine which coaches deviate from optimal decision making the most?
2) Is it possible to determine which players caused the most change in probability of the games during the season?
^ Great ideas! I would love to see those stats.
Does the in-game win prob. page account for home field? Or are the odds for two equal teams playing at a neutral site?
Neutral site.
Is it possible in the offseason that you add Homefield advantage and timeouts/challenges in your model?
Also, is it possible to have the model adjust for good offenses/defense. Like say you enter that you are facing a good defense (top 10%) or a terrible offense (bottom 25%)?
"So if the punter gets a bad snap or feels they're a good chance the punt would be blocked, he should just fall on the ball or run out of the endzone. Far better to take the safety in that situation than risk a blocked kick and an easy touchdown."
I'll play devil's advocate here (big shock, right?) - are you saying that coaches should essentially program their players with these sorts of probabilities? Like in your example, before the punter goes out to the field, does a coach grab him and tell him "hey man, if the play blows up it's ok to take the safety"? What if the snap is high and the punter just automatically runs out the back of the end zone, but never thinks to look in front of him to see a huge hole open up that he could have run through for a first down?
I think there is value in coming up with probabilities like this, but should they be such a significant part of the play calling process?
Anon-Timeouts are possible, but challenges are really impractical. I'd like to rework the model for the final 2-4 minutes of the game, and timeouts would be a big part of that.
HFA is very complex, but could be roughly factored in. My model is almost purely empirical, meaning I base the WPs on actual past games in similar situations, so factoring in HFA requires I divide already thin data in half. I'd probably have to do some kind of rough estimated adjustment. Same goes for good/ bad offense/defense.
Showing what kind of play is optimal would be tricky. If you're saying kick/go-for-it, I could do that. But run/pass/screen/draw choices are never optimal themselves. There is a game theory component that requires a mix of strategies in varying proportions.
Doug-That's exactly what I'd suggest. A ST coach could tell his punter its ok to take the safety. I don't think you want a punter running up the middle on 4th and long from the goal line under any circumstances. I don't trust punters to know how big holes are or aren't.
Brian - what I'm getting at is this - it sounds like you're suggesting that coaches essentially remove the players' ability to improvise. How about a different example.
A QB receives a play in from the sideline. The play is based on probabilities such as the ones proposed above...it's a "programmed" play. The QB gets up to the line of scrimmage and notices something in the defense that he believes he can exploit by calling an audible. In his opinion, the reward outweighs the risk.
Does the QB:
A - call the audible and gamble that his gut feeling is right,
or
B - go with the "programmed" play that offers less reward but lower risk is involved.
If I'm his coach, I want him to go with his gut. He's the one on the field. Your thoughts?
I'm just suggesting that coaches can help their players realize when they don't need to panic, and that its ok to take the sack or safety or fair catch or whatever. It would help players use their intuition better.
I'm not a coach and haven't played since high school, so I have to admit to the limits of my expertise. I guess it's up to the coach how much to trust his players with audibles and improvisation. But he should do so knowing all the facts.
Sorry. My home server looks off-line at the moment. I'll get it back up and running this evening.
Back on-line.
I noticed that the cards had a 100% win probability some time in the 3rd quarter. How is that possible.
On a related note, could you (perhaps as an option) show the win probabilities on a log scale (as the difference from 0.5)? Because the play that drops your win percentage from 20% to 10% is a very different play than the one that drops you from 50% to 40%, but on the linear scale they look the same.
Anon-I noticed that too. And then the Eagles actually grabbed > .50 WP when they took the lead.
The explanation is that the system relies on comparative historical examples. So it's saying no team has ever won being down by that much at that point in the game.
I have a smoothing algorithm that compensates for situations with low sample sizes. Down by 18 in the 3rd qtr isn't a common score at all, and no team has ever come back. In some specific cases--down and distance combinations--the smoothing correction get skipped. It's a bug I need to fix. Even with all the smoothing, there should have been only a 1% chance.
bmoore-You're right about the log scale. Doing the math should be very easy, but the hard part is implementing another graph option. There are some other improvements I'd probably do first, though. But your suggestion will go on the but I'll definitely keep it in mind.
"I guess it's up to the coach how much to trust his players with audibles and improvisation. But he should do so knowing all the facts."
What facts are we dealing with here exactly? Is it a fact that the probabilities in the post above are .39 and .43? I don't think you can call those facts. I think all you can say is that in the past, here's what has happened next after a similar series of events. What actually does happen next is a completely different beast, IMHO. I'm not trying to shoot holes in your work, I'm just having some trouble grasping what the end result of this work will/should really be.
"I don't think you can call those facts. I think all you can say is that in the past, here's what has happened..."
That's what I'd call facts.
I'm not sure I could possibly think of more useful facts than what actually happened in actual games. The database of games I use is larger than several lifetimes of what a coach could possibly experience first hand, much less remember accurately.
The example I chose yielded very close win probabilities for both possible strategies, but that won't always be the case. In many situations, there will be a clearly better strategy decision--for example:
Punt, FG, or go for the first down? Go for 2 or kick the XP? Run, pass, or mixed strategies? Play for the tie or the win? The applications are limitless.
What happens next is everything, isn't it? But coaches and players can't control what happens next. They can only influence it by making good choices in the present. Those choices can be informed ones or uninformed ones. I vote for informed.
Well Doug, wouldn't one desirable end result be for coaches (or at least the coaches of our respective favorite teams) to go for it on 4th down more frequently? If they could be convinced that a more aggressive situational decisionmaking would actually result in more wins over time -- as various analysts have shown, from The Hidden Game Of Football to Brian here -- then might not coaches start to think about tinkering with NFL conventional wisdom a little bit?
It's probably unlikely, given that any individual trailblazing coach would inevitably face an enormous barrage of media/fan/organizational criticism, and hence be more likely to lose his job. But still, wouldn't it be sweet if one coach could convince one owner to roll with him while he went for it on 4th and 5 in his own territory, etc etc, on a consistent basis?
How big is the database? How many years does it stretch? Does it include past playoff games?
So, what you'd like to see is a coach using a win probability estimator to determine the next play or plays that will produce the greatest likelihood of success...that's fine, but where is the unpredictability? What if an opposing coach has the same estimator...he could plug in the same situational stats and figure out his opponent's next move, couldn't he?
Unpredictability is very important--sometimes. In the intentional safety example, though, it doesn't matter. You can't stop me from taking the safety if that's what I want to do.
In game theory terms, there are games with "perfect information" and "imperfect information". Chess is a game of perfect information, meaning both players can see the pieces of the other, and both know the history of the moves to get to the current state. Battleship is an example of imperfect information. Neither player knows where the others' ships are.
Football plays are a mix between perfect and imperfect information. If it's 4th and 1, and an offense lines up in its short yardage formation with the QB under center, you know they're not going to punt. Predictability is irrelevant, at least regarding the go-for-it/kick decision.
But in some cases, game theory prescribes a mix of strategies, and unpredictability is key. Take 3rd and 2. A run can get 2 yards, but so could a pass. The optimum mix depends on the likelihoods of conversion and what the defense does. You don't know their strategy (run blitz, run gap, pass d, etc.) and they don't know your play call.
All of this modeling can tell us an optimum proportion of running and passing to maximize the overall likelihood of success. The answer isn't "always run" or "always pass." It's, say, run 40% and pass 60%. As long as the offense is unpredictable within the confines of that overall mix, it's unpredictable enough to make the defense respect either option.
So the short answer is: sometimes unpredictability doesn't matter, and sometimes it matters a lot but the answer is a random mix of strategies in an optimized proportion.
The database goes from 2000-2007, and soon I'll add '08. That will bring it to about 2300 games. I have playoff games from those years, but I'm not using them currently.
Actually I saw a QB punt in a 4th and short this season. They were just outside field goal range. It was a pretty short punt but of course no return.
In my opinion a much better choice than lining up and hoping for an offsides and then calling a timeout.
Would the QB have to be in the shotgun to do that? In any case, it's a rarity.
I think that it was Rothlisberger and yeah he was in a shotgun. They stood around for a bit hoping to draw the offsides and then he walked up to the line, audibled and then dropped into a deep shotgun and punted.
You haven't reached back to 1998 and 1999? Or, is the size of the database sufficient going back to 2000?
I don't have any data prior to 2000.
Besides, I'm not sure how far back is too far back. The sport has evolved and continues to do so.