I already have a post this season called Lovie's Blunder, but I guess I'm just not terribly imaginative, much like many coaches in the NFL.
In a tight game against the Redskins, Smith decided not to challenge Jay Cutler’s fumble on first and goal from the one. Most accounts claim the replay clearly showed the ball crossed the plane of the end zone prior to the fumble. Had the play been a touchdown, the Bears would have been up 11 points early in the third quarter, giving them an 89% chance of winning. Instead, the fumble put the Bears’ chances at 70%, a difference of 19%. To put that in perspective, only two of the game's plays were more significant, and they were the two interception returns for touchdowns.
Smith had lost a challenge on the prior play, and you have to think that must have affected his judgment.
Just for the sake of argument, let's assume that the Bears eventually found themselves down by 3 points inside the two-minute warning. (They were up by 4 at the time of the play.) If we value a timeout as a full 40 seconds between plays, the difference in Win Probability (WP) between having 2:00 left to play and 1:20 left to play at their 30 yard line would have been 0.04 WP (0.24 WP compared to 0.20 WP). The difference between having 60 and 20 seconds to play is 0.07 WP. The biggest difference in the value of a timeout is between 40 seconds and zero seconds (a certain loss), which is 0.10 WP. Assuming the challenge had a reasonable probability of being upheld, he should have thrown the flag.
One of the cool things about a WP model is that it's a linear utility function, so we can evaluate gambles like the decision to challenge or not. For now let's set aside the value of consuming his second and final coach's challenge. Assuming the largest value of the timeout (0.10 WP), the break-even probability that the challenge would be upheld (x) would need to be:
So Smith would only need a 34% chance of being upheld for the challenge to be worth it, even assuming the worst-case scenario that the Bears would be down by a single score at the end of the game. You don't need algebra to figure out that play should have been challenged. There aren't going to be many reviewable plays all year as big as that--it's either a TD or a turnover.
Brian - I like the idea of trying to quantify the worth of a timeout, but I disagree with this analysis. The data from which you pull the WPs by time left (i.e. 2:00 left = .24 WP, etc) is not broken down by the number of timeouts the team has or whether the clock is running or not. Consider a case where a team has the ball on the opponents 15 yard line with 5 seconds left, clock running, down by 2 points. A team with a timeout obviously has a larger propbablity of winning than a team that doesn't. However, your data treats both those situations as the same. The historical data should not only collate by field position, time left, and score differntial, but also by # of timeouts left and whether the clock was stopped.
Ahh Lovie Smith...
I'm sure you're on this already but I was wondering what the numbers would be for the end of the Dallas game when those terrible announcers were calling for the FG when they went for the TD instead. Oh and I just noticed it on your twitter so I'm sure the write-up is coming soon.
Parsing the data by timeouts remaining chops it up into tiny chunks to small for reliable estimates. That's why it's necessary to use other means to estimate a the value of a timeout. Keep in mind there are 16 different possible timeout states 3,3; 3,2;...3,0 etc. In this exercise, I'm only worried about estimating the highest possible delta in WP that a timeout could give me rather than the absolute WP of any given timeout.
Timeouts are a really interesting study because frankly there just isn't enough data to work with.
Omar is right. There are some narrowly defined situations when having an extra timeout is worth about 100% in WP. But I think the problem with this sort of thinking is encapsulated in the example he gave. Exactly, what is the likelihood that you are going to end up in such a narrowly defined situation.
I mean it should really be emphasized that the goal is ultimately to win the game and not to save the timeouts. Most of the time, the timeouts end up being fairly uselese in the end. Even in the times when they are useful, you'd almost certainly rather be the team trying to run out the clock than the team trying to preserve it.
Nearly every time a team doesn't have all timeouts, they are referred to as having "wasted" them. There is a difference between spending and wasting. If you see a key play or challenge early, getting the right result is almost certainly going to be worth more than having an extra TO.
Another factor is that the Bears would be out of challenges. Still should have challenged, since it is such a big play, but win or lose it would be their last challenge.
Right. As time expires, you always wish you had another timeout if you're losing. The idea is to work backwards as far as reasonable to estimate its realistic marginal value. Going to the beginning of a possible final possession and adding 40 seconds is seemingly a reasonable upper end of the value. The next step would be estimating how likely you are to find yourself in a state where you would need that timeout. Already, the math explodes into a nearly infinite range of possible outcomes.
So to assess the value of a timeout at any given state in the game is an incredibly difficult thing to do. Bill Krasker attempted to use his backward induction model to do it, but it makes a lot of assumptions along the way. It's probably the best method currently available though.
Part of the problem is that the challenge on the previous play (the one he lost) was highly questionable: the result of the play was first down, Chicago, on the Washington 1. Even given Chicago's struggles running the ball, particularly in the red zone (struggles discussed in detail on this site), I don't think this is a play you should bother to challenge, if only because the challenge itself gains you so little.
Obviously, the next play made it seem as though Lovie was right to challenge, but fumbling from the one is pretty rare, I would guess. Most of the time, you would either score or be in position to kick a field goal ... and typically there are plays later in the game with greater swings where a challenge would benefit you more (like on the fumble).
On a tangent.....
When is it a better decision to give up the five yards for delay of game and not call the timeout? Is the convential wisdom to take the timeout a statement on the value of timeouts?
Interesting article. The idea of quantifying a timeout is certainly a complex and valuable concept. JMM, I have often wondered about why coaches don't occasionally just take the 5 yard penalty. On numerous occasions it would seem better than taking a timeout or running a hurried, ill-advised play. Decisions of this sense are part of what makes football a great sport. For more on what makes sports like football great please check out www.teenandinbetween.com
JMM - the only times I can think of are on a short 'chip shot' field goal or on a short field punt (from ~the 40 on 4th and long). The FG because the extra 5 yards add very little to the difficulty of the kick and the punt because supposedly it gives the punter more room to work with (not that I've seen any evidence supporting that idea - punters seem to just boom the ball through the endzone regardless and if I was a coffin corner punter I'd rather be closer to the corner I was aiming at).
In any case where you're still trying to get a first down the 5 yards is quite costly - based on EP across the field, the average difference between say 1st and 10 and 1st and 15 is about 0.6 EP (A quick check shows the difference between 3rd and 5 and 3rd and 10 is also around 0.6 EP).
So I guess it's a case of whether you think the timeout will hold more value later in the game than 0.6 EP does now. Complicated question.
So is it fair to say Ian's first estimate of 0.6 EP is the high limit for a vue for timeouts?
The low limit would be zero since teams routinely don't call all three Per half.
The actual value is, as mentioned above, function of score, time in game, down, distance and field position.
As zlionsfan commented the bigger error was challenging on the prior play. The odds you'd need to make that a good challenge are greater than 100%.
Also the relevant resource might not be a timeout, but a challenge. Regardless of right or wrong, if they challenged, the Bears could not initiate any more challenges in the game. Just makes the prior challenge look even worse if it causes the next play to not be challenged to save a timeout
5 yds might be a better comparison than a timeout. Most teams would trade the timeout to avoid delays or 12 men. How much is 5 yards worth at that juncture?)
JMM - that depends what the timeout is buying you I guess. Sometimes they're used for preventing delay of game penalties, sometimes because you don't like the look of the defense and sometimes to stop the clock at the end of the game (or there's icing the kicker, but no-ones shown any good evidence hat buys you anything anyway).
In terms of what you're risking when you challenge, it really depends what use you intend to have for the timeout. I would argue that a 'stop the clock' type timeout actually buys you more than 40 seconds - it buys you the middle of the field to pass to as well. Lots of stuff that's hard to quantify.
Oh and something random I found looking at EP - if you find yourself running out of play clock time but don't want to use a timeout, you're actually better off snapping the ball and spiking it than you are taking the 5 yard penalty, apart from between about the OPP 35-45, where it's about a dead heat. 2nd and 10 is better than a backed up 1st and 15.
From the explanation of WP you use score, time, down, distance, and field position for the model. To account for the value of timeouts, you adjust the "time" and then calculate WP. Would a metric such as "likely offensive/defensive plays remaining" be able to replace time in your baseline correlation? Then the "LOPR" could be a function of the timeout states and time remaining, maybe also things like score differential. Maybe this has been done and shown not to work, but if no one has tried it do you see any obvious reason it wouldn't improve the model?
jiji
Another way to look for mo would be to look at techniques used by analysts tracking stocks and stock index movements.
It is a similiar problem, lots of imput factors summarized by one number, WP or stock price.