First downs are easier to analyze because they almost always begin with 10 yards to go. Unfortunately, 2nd downs aren't so cooperative. It's amazing how thin he data gets sliced up. Most downs aren't losses, even fewer have holding penalties, and rarely are they declined. Still, there are enough cases for a solid analysis using 1st-down conversion probability as the bottom line.
Put simply, a defense would prefer to decline a penalty on a 2nd down play whenever the resulting 3rd down situation leads to a conversion less often than the 2nd down plus the 10 yards.
The chart below plots conversion probability for 2nd and 3rd down situations. The red line illustrates the conversion probability of 3rd down and X to go situations. For example, 3rd down and 7 situations are converted about 40% of the time.
The green line illustrates 2nd down situations, but slightly differently. It plots conversion probabilities for 2nd down and X plus 10 yards. For example, 2nd and 13 (i.e. 3 + 10 yds) situations are converted 45% of the the time. The black line is the smoothed line fitted to the 3rd down conversion rates. I plotted things this way because it's the actual comparison we're interested in, given a gain of zero yards.
We can see that the conversion probabilities of a 3rd and 7 and a 2nd and 15 are roughly equal--about 40%. That means that a defense should be indifferent to accepting or declining a penalty following a loss of 2 yards on 2nd down.
We can repeat this process for all 2nd down distances to go, establishing a break-even point of indifference for loss amounts. On 2nd and 1, a defense would want a loss of 4 or more yards before declining a 10-yard penalty. On 2nd and 3, they would want a loss of 3 or more yards. And so on.
Reading the graph above isn't going to be practical for a coach on the sidelines. So I thought back to my flying days and thought a rule of thumb would be handy. When doing math in a F-18 (you'd be amazed how often you have to--how much gas will I have left when I get back to the carrier?), it pays to simplify things because there aren't that many brain cells left over after all the flying stuff is worried about. Rules of thumb may be inexact, but the trade-off between precision and speed is well worth it.
In this case, the 'loss of indifference' yardage fits neatly to the following:
4 - 1/3 * original distance to go
For example, from a 2nd and 6, a defense would want (4 - 6/3 =) a 2 yard loss or greater before it declined a 10-yard penalty.
How well do coaches approximate this rule of thumb? Surprisingly well, just like they do for 1st down penalties.
The plot below shows the rule of thumb in red and the actual loss of indifference for coaches in blue. I established the actual point of indifference by noting which loss amount resulted in equal likelihoods that a coach would decline or accept.
Coaches do really well, but it appears they look for greater losses than required before declining. In other words, they should be declining slightly more often on smaller losses. It's not a big difference, and certainly nothing that will swing any one game. But every little bit helps.
There are additional considerations as well, such as game variables like time and score. For instance, if trailing late in a game a defense would prefer never to hand the offense an additional down with which it could burn 45 more seconds. (DET declined a 10-yd holding penalty on their way to their improbable comeback over DAL last Sunday.) One big consideration is forcing the offense outside of field goal range. But as we saw in the 1st down analysis, that's usually secondary to minimizing the chance of conversion.
Does anything interesting happen with respect to the indifferent yardage or the coaches' behavior in no-man's land?
> 4 - 1/3 * original distance to go
Makes me wonder if coaches ever decline the penalty after a 1 yard gain or incompletion on 2nd and 15.
This looks like an editing error:
> We can see that the conversion probabilities of a 3rd and 5 and a 2nd
> and 17 are roughly equal. That means that a defense should be
> indifferent to accepting or declining a penalty following a loss of 2
> yards on 2nd down.
Shouldn't that be a choice between 3rd and 7 or 2nd and 15?
Yup--3rd and 7 vs 2nd and 15. Fixed. Thanks Nate.
Regarding no man's land--the data gets sliced pretty thin on 2nd down to say anything definitive. I can aggregate things relative to the point of indifference to see how things change in that region.
Quick question, maybe I'm misunderstanding or just missing something. Why are you limiting your dataset to only plays that are losses and have a holding penalty and are declined? I would think you wouldn't need to do any of those, you simply have a choice between result of play and result of penalty. I would think you could just use EP or 1Dp for those scenarios.
@ Nate: That's a good point. Personally I think I'd rather take 3rd and 15 over 2nd and 25, unless I've got a good defense and I plan on playing agressively on both 2nd and third down (leaving aside field position for the moment). It seems to me that coaches sometimes take the yards and then go into a less agressive nickle defense, and then the offense responds with a 10-15 yard gain on a draw or a screen. That's purely anecdotal evidence on my part, and even when coaches do this it generally works out in the long run becuase even with a long gain on 2nd down the offense is still facing 3rd and 10+. But the objective of calling a conservative defense on 2nd and 25 would be to get to 3rd and long with very little chance of allowing a long pass completion, at the cost of giving the offense some easy yards on the ground. In my opinion declining the penalty acheives the same purpose with less risk.
On the other hand, if you're going to play agressively on 2nd and 25 (which I personally would prefer), then taking the penalty might be preferable, becuase it's likely that you'll be able to hold on to those 10 yards in the form of field position to your offense after forcing a punt.
> I would think you could just use EP or 1Dp for those scenarios.
That's what he's doing. The idea is that, instead of memorizing the entire EP or 1Dp table, you can use a rule of thumb.
> ...Personally I think I'd rather take 3rd and 15 over 2nd and 25, unless
Yeah, I primarily brought it up because that's such a counterintuitive results. (Even if there must obviously be some point where the down is worth more than the yardage.)
Rule of thumb highlights how game management decisions in real time are not as easy to navigate as post game. Does the NFL allow teams to use tablets or laptops in the booth? If so, do you know any teams that use the calculator or a proprietary version in real time and relay the results to the HC? Seems like a must have for any team that cares about analytics in any way.
Thomas, IIRC the rules do not allow the coaching staff to use computer or calculators or have contact with people who are using such. Thats why they all have clipboards, 'Denny's Menus', and playbooks. I think that is in the same section that the Patriots violated in their so-called 'spygate' thing.
Cheers,
J
How does momentum play into the stats? If, as a coach, I have a defense that was being aggressive and kicking @ss, wouldn't it kill their morale to decline a penalty that would turn the next play into a long distance?
Or what about the fact that you're making a team more one-dimensional by putting them into a longer distance situation? For teams that have better runners than passers, this might also impact the coaches' decisions.
J-
Any thoughts on how teams that employ advanced stats work around not having access to real time data? Create 'Denny's Menus' of potential game management situations etc? Thanks
Thomas,
I know Brian Burke has mentioned 'decision making drills' somewhere or other on the page. Like the Government or military does for nuclear war scenarios or crises where very complex descisions need to be made quite quickly. I think having a good understanding of the basic WP and EP of various crazy situations and then 'gaming' it out in practice would give most coaches a good 'eye' for the correct call when the time comes in an actual game. I think during the practice week it would be easy enough to design a computer program that generated 'tough call situations' that could then be set up on the field and run. Maybe one of those little laminated wheels (I recall using things like this in math class way back when) where you could rotate the key to find down and distance and the WP considerations. Also rules of thumb like what Brian Burke has suggested in other articles. Really, I think that there is probably already too much information on the sidelines as it is and adding more playbooks, flashcards, menus, etc would probably end up having a negative effect - too much information paralyzing decision. It would seem the best bet is to get yourself used to the decisions and ideas beforehand and maybe take a card or two with general notes/ideas in the game. Prior preperation prevents piss-poor performance!
Cheers,
J
Dave: Momentum does not play into these stats or into anything else. Bill Barnwell explains:
http://www.grantland.com/story/_/id/9580775/bill-barnwell-theory-momentum-football
http://www.grantland.com/blog/the-triangle/post/_/id/71970/nomentum-in-sports-part-2