When Should the Defense Decline a Penalty After a Loss? Part 1

Let's say there's a sack or other tackle that results in a several-yard loss. And to compound the offense's woes a flag for holding is thrown, potentially setting up a 1st and 20 situation. Should the defense accept or decline the penalty and force a 2nd and X? We can evaluate this question in a few ways. We'll use a simple method and a more complex method to find out when a defense should normally decline a penalty on first down.

Before you read on, what do you think the break-even yardage is? What do you think most coaches think it is?

The first criteria we'll use is simple first down probability. Offenses go on to convert 1st and 20 situations 46% of the time. For to go distances that the defense can expect to allow a conversion on 2nd down less often than 46% of the time, they should normally decline the penalty. According to this criteria 2nd down and 13 situations are generally converted 44% of the time, meaning it's probably a good idea to decline whenever the 1st down loss is 3 or more yards.

We can also evaluate the question using a more advanced tool--Expected Points (EP). EP could be useful because it considers field position and factors in things like the likelihood of turnovers. For instance, a 1st and 20 may force the offense to pass more often, giving the defense and enhanced opportunity for a turnover.

We'll need to translate (shift) the EP curve to account for the difference in field position. A 1st and 20 at the 50, and a 2nd and 13 at the 50, are not our choices. It would have to be a choice between 1st and 20 at the 50, and a 2nd and 13 at the defense's 43 to be consistent. The chart below slides the 2nd & 13 line to account for the 7-yard difference. In this comparison of EP values across all field positions, the 1st & 20 situation has the same or higher EP value than for the 2nd and 13, meaning the defense should usually prefer the 2nd and 13. This includes the outer field goal range area, when a defense would most prefer to set the offense back in terms of field position.

So how do coaches fare in reality?  I suspected that coaches would overvalue yards and undervalue downs (like they typically do), meaning they would accept the penalty taking the yards but giving back the down to the offense. It turns out they do pretty well. The chart below shows the proportion of offensive holding calls declined according to the yardage lost on the play. The data includes 874 plays outside field goal range (the 35) which did or would have resulted in a 10-yard enforcement from the line of scrimmage. At the 3-yards-lost mark, coaches declined the penalty exactly 50% of the time.

Next, we'll look at 2nd downs.

12 Responses to “When Should the Defense Decline a Penalty After a Loss? Part 1”

1. James says:

2. Anonymous says:

Are there public tools available I can use to evaluate specific decisions? I am curious about the Packers decision to decline a 10 yard penalty against the Ravens that would have made it 3rd and goal from the 11. McCarthy declined opting for 4th and goal at the 1. This seems like quite a poor decision, but since I haven't seen is discussed anywhere I'd like to test my intuition if possible.

3. Brian Burke says:

That's exactly the play that got me motivated to start looking at the defensive side of penalty accept/decline decisions. I thought I'd start simple and get to more complex situations like the GB-BAL one on the goal line. That's particularly difficult because there is the added uncertainty of what BAL would do on 4th down. Given most coach's tendencies, it was probably a good gamble to decline, and it was probably a good gamble for BAL to go for it (their horrible running game notwithstanding).

4. Brett Barkelar says:

Can you evaluate decisions to decline or accept penalties near the goal line. The Packers have had 2 decisions this year. Versus SF they took a 5 yard penalty to make it 3&5 at approx the 7. Against BAL, McCarthy declined a holding penalty to make it 4th and goal at the 1 yard line.

5. Sampo says:

@Anon:

Yes there are!

The Win Probability Calculator (http://wp.advancednflstats.com/winprobcalc1.php) is just the tool you need. It is pretty straightforward to use. Just input the score difference, time left, field position, down and how much to go. Make two calculations: first the 3rd&11 and then the 4th&1 and compare the Expected Points and Win Probability!

I'd do it, but I'm too lazy to find out the situation when it happened.

6. Anonymous says:

For the WinProb calculator, should it return a first down probability in a goal to go situation? Right now on 3rd and 11 from the 11, the first down probability is 0.28 and the TD Prob is 0.32.

7. Anonymous says:

I was thinking the same thing as everyone else. It appears McCarthy was being results oriented with his decision against Baltimore based on what happened against SF despite the fact that the Baltimore holding call was 10 yards and if declined left the ball at the 1 while the SF penalty was 5 yards and if declined left the ball at the 2. The only possible defense of McCarthy and how he happened to both decline the Baltimore penalty and accept the SF penalty is that Jim is known as more aggressive than John and might be more likely to go for it on 4th down.

8. Brian Burke says:

I don't recall, but I don't think the calculator is going to work for goal to go situations >10 yds. Actually, the 1st down = TD prob in that case, so I'd suggest going with 1st down prob. The other thing I should mention about the calculator is that it stops caring >15 yds to go. IOW, if the situation is >15 yds to go, it assumes it's 15 yds to go. The probability curves get really flat (but not completely) beyond 15 yds, so it was a way to reduce complexity when I first did the model.

9. Anonymous says:

@ Brian's first comment:

I agree that if we have a very high confidence that John H. auto-kicks the FG the correct decision is to decline. However I think it is unlikely that confidence can be very high in this situation. Perhaps one question you can look at in your next post is what weighting of FG/go makes McCarthy indifferent to accepting the penalty.

@Sampo: Thanks. I tried using the calculator but I don't think it/my use of it is working correctly. With inputs: score -3, 6:00 to go in the 2nd quarter, FP opponents 1, 4th and 1 to go I get:

Win Probability: 0.48
Expected Points: +2.29
First Down Prob: 0.25
TD Prob: 0.15
FG Prob: 0.69

I don't see how this situation can yield so few expected points (2.29) since such a high percentage FG is pushing 3, and conservative NFL coaches means that going for it should result in a sample of above average offenses making at least 3EP when they don't kick. Also I'm not sure what I'm looking at re: FG and TD probability. Is this the success rate of all plays so that .69FG+.15TD means .16 of the time teams either miss a FG or go for it and get stopped?

10. Sampo says:

Those numbers do seem odd. But the thing to remember with EP and scoring is that successful field goals are not exactly worth three points as you are giving away the possession of the ball after the kick off. Having the ball at your own 22 yard line is worth about +0.42 points. So successful field goal is worth around 3 - 0.42 = 2.58 EP. Same of course with touchdowns (7 - 0.42 = 6.58).

11. Anonymous says:

Good point about FG EP, I should have thought of that. Yet the EP still seems a bit low. Also, I am still unsure what the variable "TD Prob", "FG Prob", and FIrst Down Prob" correspond to in the scenario I calculated above.

12. SlackerInc says:

I played along at home as you suggested, and my gut sense was that the lost down was worth about seven or eight yards, so pretty close. I had the opposite hunch as Brian and thought the coaches would prefer to decline the penalty unless they got something like ten yards or more in exchange (though this could vary in situations near the end of a game where they are desperate to keep a team out of FG range). Funny that we had opposite expectations of how the coaches would be wrong, and it turned out they were pretty much in the middle, right on the money.