In this part, I'll apply a more rigorous analysis and get a better approximation. We'll also be able to repeat the methodology and build a generalized model of timeout values for any combination of score, time, and field position.
Methodology
For my purposes here, I used a logit regression. (Do not try to build a general WP model using logit regression. It won't work. The sport is too complex to capture the interactions properly.) Logit regression is suitable in this exercise because we're only going to look at regions of the game with fairly linear WP curves. I'm also only interested in the coefficient of the timeout variables, the relative values of timeout states, and not the full prediction of the model.
I specified the model with winning {0,1} as the outcome variable, and with yard line, score difference, time remaining, and timeouts for the offense and defense as predictors. The sample was restricted to 1st downs in the 3rd quarter near midfield, with the offense ahead by 0 to 7 points.
Results
The regression results produced a coefficient of -0.111 for the variable tolDef, which represented the timeouts remaining for the defense, and +0.141 for the variable tolOff, which represented the timeouts remaining for the offense. Both variables were significant to some degree,--the p-value of tolDef was .07 and the p-value for tolOff was <.001. (In other specifications where I held tolOff = 3, tolDef was significant at <.001.)
The coefficients are unintuitive until you apply them in a prediction of the model using the logit transformation. If we call the additive effects of the coefficients and variables "x":
WP = e^x/(1 + e^x)
A selected example clarifies things. If we look at a situation at midfield with 7 minutes to play in the 3rd quarter, while the offense leads by 3 points and the defense has all 3 timeouts remaining, the offense's WP is .702. When the defense has just 2 timeouts in the same situation, the offense's WP increases to .729. This suggests the value of a timeout in this situation is .027.
Analysis
This is how the model estimates the WP in such a situation (offense up by 3, midfield) across the 3rd quarter according to how many timeouts the defense has remaining:
The focus is on the unit value of the timeout, so here is the same chart with just the 3-timeout and 2-timeout states included, along with their standard errors.
Isolating the offensive timeout variable, the effect size is .034 WP, somewhat larger than for the defense. If true, that's not what I expected. Because the defense is trailing in this subsample, I would think they would be more likely to need all their timeouts. Other specifications (with other time and score subsamples) tended to have symmetrical effect sizes for offense and defense. And when I hold one side's timeouts left to 3, the effect sizes are about even, including for this subsample. I'm inclined to think the true effect sizes are likely symmetrical for the offense and defense, and this particular result includes some error.
Another approach would be to replace both timeout variables with a net TOL variable, which would be TOL(off) - TOL(def).
Ultimately, if we add symmetry to the list of assumptions, which is not completely unreasonable, we get an effect size of about .031 WP for a timeout in the subsample. That's more modest than the first approximation of about .05. My gut was that .05 was a bit high, because the difference between 0 TOL timeouts and 3 TOL would be about .15 WP. That's roughly the equivalent of a turnover in a tie game in the 3rd period. The regression estimate of about .03 is likely closer to the true value because the method accounted for variance due to the other factors in a much more sound way.
In Practice
Back to the situation where a team has a 3rd and 1 at midfield as the play clock winds down. Should they call the timeout to prevent the 5-yard penalty, or should they save the timeout and accept the 3rd and 6? Let's say the offense is up by 3 points, and there is 7 minutes to play in the 3rd quarter. A 3rd and 1 in that situation is worth .70 WP according to the timeout-unaware model. A 3rd and 6, 5 yards back is worth .68 WP. The difference of .02 WP is smaller than any of the estimates for the timeout value, so it's probably a good idea to save the timeout and accept the penalty.
Admittedly, these estimates are within a couple standard deviations of error, but accepting the null hypothesis is not an option. The coach (or QB) has to make one decision or the other. He might as well make the one that numbers (very probably) back up.
A more nuanced analysis can take local factors into consideration. If a team is defense-oriented and can't move the ball very easily, the difference between a 3rd and 1 and 3rd and 6 might be quite large. And if you're banking on winning with defense, you may not be banking on getting the ball back for a final drive. In that case you may want to use the timeout. But if you're an offense-oriented team, you may be better able to convert the 3rd and 6, and you'll want those timeouts to set up your offense for a final drive. In that case, you may want to save the timeout and take the penalty.
Generalizing the Approach
To get a complete model of the value of a timeout, the data can be sliced up into various subsamples with relatively linear WP curves, and the regression can be repeated. The 4th quarter is obviously critical, but it may make the estimates easier. Unlike the in 3rd quarter, there are many more examples of situations where teams have 2,1, or no timeouts.
Can you compute the average increase in win probability by the use of each team's third timeout? So, on average, how much does a team's WP jump when it uses its last timeout? I understand this may not be easy, but I'd be curious on the chance that it is pretty easy.
Sorry, I couldn't find the link to Part 1: Presumable the WP isn't static, i.e. if one team has a huge lead in the 4th quarter, and both teams had all their time outs, the WP is negligible as managing the clock still won't make a win significantly more probable?
Presumably the WP is dynamic and more amplified the closer the scores are later in the game (when WP swings drastically anyway)?
Go easy on me, poker's all the probability I managed to understand...
Very interesting work.
One thing to keep in mind is that this isn't necessarily the "true" or "optimal" value of a timeout in the following sense: you are estimating the value of timeouts with respect to how coaches currently use them (using the historic data). That is not necessarily the optimal way to use them, and so this could be an underestimate of their true value. I'm not sure there's anything you can do about that, though.
As always, keep up the good work.
How about this.
Take all the games and put them into bins based on the home team's WP at the start of the 4th quarter. And then compare the overall WP (which should match the average for that bin) to the WP further divided by # of timeouts.
IE, out of 3,000+ games, at the start of the 4th quarter, there were 400 of them where the home take had a WP between 0.5 and 0.6. Out of those 400, 224 of them went on to win, for an average WP of 0.56. Then you look at it further and find that of the teams with 3 timeouts, 62% of them won, but only 53% of the ones with 1 timeout.
Then you don't have to worry about offense/defense since we're only looking at home team.
Cardlinger-You're correct. The value of a timeout in this situation isn't x WP. It's e^x/(1+e^x). The closer the game gets to a blowout, the less things like TOL matter.
Brian-Excellent point. We could do some kind of backward induction to find the true value, but that would assume the opponent is as optimized as our model.
Jon-I will try that.
The biggest WP component of a timeout I see comes down to an end game scenario where one is trailing without the ball, and can keep an extra 40s on the clock when one gets the ball back. An upper limit on the value of this component would seem to be the percentage of games that have a lead change in the final 40s.
I am getting some very odd results looking at this myself.
I'm looking at offenses that require a touchdown to win (so down 6,5 or 4) at the two minute warning and it looks like the more timeouts an offense has, the more likely they are to lose.
Can't decide if it's just something weird in the data, or if there's a case that playing hurry up offense is more of a disadvantage to the defense. It could be that if you have plenty of timeouts it gives the defense a chance to catch their breath and call a play, whereas they may go to a default 'prevent' style when you hurry up.
Something I think I'll have to dig into more.
Ian, that is an odd result. Did you account for field position in your comparisons? If not...maybe teams that are closer to scoring are also more likely to have used timeout(s) in the process of getting into that position?
And, in perhaps another example of suboptimal coaching corrupting the data, maybe if the team doesn't have timeouts, they are more likely to run the clock down to almost nothing in the process of scoring, while a team that has timeouts will use them, and then leave time for the other team to come back.
J.D. - field position was accounted for, but it was the only thing. The question I asked gretl was "given a team is down 4, 5 or 6 and has the ball on this yard line, with this many timeouts and two minutes left, what's the probability they will win?"
I didn't account for down or distance yet but unless there's some really unlucky bias in the data I can't believe that will affect it so much. Back to Excel anyway.
I've found that in the late end-game offense TOL doesn't matter if they're behind. You'd think it would, especially for the final timeout, but I suppose offenses can stop the clock easily enough in other ways.
Don't you get more time with a timeout than you do with a delay-of-game? If you accept the penalty, you better be confident that you can get the play off the second time. The Saints couldn't, for example, in their game against the Seahawks; they took a delay of game and THEN had to take a timeout.
You do also get to talk more about play selection and such with a timeout, and you can substitute. This effect is obviously not terribly feasible to measure, but it might contribute something to the probability to convert.
Ian, maybe you should try it at the 1:00 mark instead. Seems like 2:00 is plenty of time to score, even without any timeouts. A team in that position usually has just one chance. If they have some timeouts, they might use them to scheme, and in the process give the other team more time to score in response.