## Analyzing the Patriots' 4th Down Calls

Kurt Bullard is a freshman at Harvard and a first year member of the Harvard Sports Analysis Collective. He intends to major in either Economics or Statistics. Go 'Cuse.

This Sunday, the Patriots found themselves down six points to the Saints with only 1:13 left in the game. In that span, Tom Brady was able to lead a drive down the field, connecting on a 17-yard pass to Kenbrell Thompkins in the left corner of the end zone to complete one of Boston’s two major come-from-behind wins of the day.

Hidden by the final drive were two controversial fourth down calls in the fourth quarter that happened at key moments in the game. The first came with 8:34 remaining in the game when the Patriots held a 20-17 lead over the Saints. Faced with a 4th-and-goal at the five yard line, New England opted to kick a field goal rather than try to score a touchdown to go up two scores.

The win probability and expected point value each suggest different optimal decisions. The expected point formula suggests that the Patriots ought to have gone for it, while the win probability calculator says otherwise, albeit by a slight margin.

With a field goal almost a guarantee, the three points would have put the Patriots up six points and produced a win probability of 82 percent. Yet the Saints still could have erased their lead with a touchdown. If the Patriots had gone for it, their win probability would have been 79 percent, slightly lower than opting for a field goal. The reward for the risk was greater than the field goal -- if the Patriots had converted, they would have gone up two scores, which would have increased the probability of winning to 92 percent. Even if they had failed, the Saints would have only been able to tie the game with a field goal take the lead with a touchdown, just like the field goal situation. At this point, Belichick had no clear choice since the win probabilities were so similar. He opted to play conservatively and trusted his defense and offense to protect a six-point lead. Whatever Belichick had decided to do, it would have been defensible.

The second decision came later in the game. Down, 24-23, with 2:46 left in the contest and all three timeouts remaining, Belichick decided to go for it on 4th-and-6 on his team’s own 24-yard line. This time, Belichick adopted the more aggressive strategy, going for the first down deep in his own territory. The Patriots attempt was unsuccessful, as Brady’s pass fell incomplete to Aaron Dobson.

This model also suggests two different courses based on the two statistical methods employed here. Going for it would have increased the win probability, but it also would have led to fewer expected points.

A Patriots’ punt would have put the Saints, on average, on their own 37-yard line. The win probability for the Patriots – to force a Saints punt and either score a field goal or touchdown – was 14 percent. In contrast, the win probability that they achieved by actually going for it was 27 percent. Since the chance of conversion is relatively high, and there is no guarantee that the Patriots defense could get the ball back for the offense, going for it makes sense. Even after converting, however, they still would have only had a 54 percent chance of winning, as they still would have had to move the ball significantly downfield to even get into Gostkowski’s field goal range. According to win probabilities, the Patriots were more than twice as likely to win if they had attempted to go for it, which they did. In the case they failed on 4th down, there was still a four percent chance of victory, which they ultimately managed to beat the odds to achieve. Belichick would have needed to believe that the chance of conversion was less than 20% in order to justify a punt.

Such a difference in win probabilities may be attributed to situational defense. When the Saints had forced a turnover on downs, they got the ball at the NE 24. The chances that they would be able to kick a field goal here are relatively high, at 74 percent for a 41-yard field goal. However, it was on the edge of field goal range. A six-yard sack could have pushed New Orleans to the NE 30 for a 47-yard field goal, which would have reduced the probability of success to 62 percent. This may be an overestimate for Hartley too, whose career long is only 53 yards. With that being said, the Saints were more likely to play conservatively in this situation, since any negative play would have a significant effect on the probability of a made field goal from Hartley. Belichick trusted that, if they failed to convert on fourth down, his defense could force a three-and-out, with the Saints most likely being very careful with the ball. This outcome would give Brady a chance to lead a two-minute touchdown drive to victory -- odds that Belichick probably thought were in his favor.

From this conversation stems another one: Should teams follow the expected points values of a decision or the win probabilities. As was demonstrated in both situations above, the two do not always coincide with each other. A team should not necessarily be looking to always maximize its score – doing so depends on the situation of the game. For example, when faced with a fourth-and-goal at the five yard line up 14 points in the fourth quarter, it would make sense to kick a field goal and go up by three scores, instead of taking the risk of going for it despite the expected points value being higher. Conversely, if a team were instead down 14 points, it would make more sense to go for it, since a team would still be down two scores after a successful field goal. Expected points only account for an isolated situation; win probabilities, on the other hand, account for the context of the situation.

So, while going for the fourth-and-goal and punting in the Patriots’ two respective situations would have maximized expected points, win probabilities said otherwise. The extra four points from scoring a touchdown instead of kicking a field goal were not valuable enough to risk going for it. In contrast, risking turning the ball over on downs, even though punting would have led to more expected points, was a good decision because those points were valuable enough given the time crunch.

Belichick’s change of mind from conservative to aggressive is, by the statistics, justified. While either choice in the first situation was defensible, it was clear that the right choice on the last fourth down call was to entrust the game to Brady and company.

### 12 Responses to “Analyzing the Patriots' 4th Down Calls”

1. Anonymous says:

I love that you analyzed this, and it's a shame that no one else will. Because the Patriots won, people forget about or ignore Belichick's correct decision to go for the 4th down from deep in his own territory. Had they lost, the media would be all over Belichick for not punting to "give his defense a chance" or whatever BS spin they'd put on it.

While Brady's final drive was impressive, it's a shame that the media will focus on it instead of focusing on Belichick's correct decision - one that very few coaches in the NFL would have ever even considered.

2. Anonymous says:

3. The Wizard says:

It seems to me that expected points is completely irrelevant. Shouldn't it all come down to probability? As an extreme example, assume you are at the one inch line with 1 second left in the game, and you are down 2 points. Assume that you have a 75% chance of converting. ( I am not sure what it is). Do you go for a TD or field goal? The decision is clear.
Sure, not every decision is an end game decision, but probabilities are ultimate measure, not points.

4. Matthias says:

I was thinking what "The Wizard" above was thinking. Obviously each game's situations that pop up deviate slightly from the model since the two teams have different strength and are not "average." But in theory, if a true win probability was known, that should be what is followed. "You play to win the game!"

5. Anonymous says:

Payton given almost the same choices as Belichick made the incorrect decision by taking the bait given to him by Belichick and went for the useless field goal when he should have went for the TD and the win. Also made the ultimate mistake by not being more aggressive in getting the first down by using vanilla offense.

6. Anonymous says:

obviously one would maximize win probability.

(though, one only has a crude estimate of win probability)

7. Anonymous says:

Hi Kurt,

Thanks for posting this. I would love to see a similar analysis on how some first half situations like this impact a game as well. For example, when the Bills went for it on 4th & goal at the one and failed while down 3 with 13:47 left in the 2nd Quarter, or when the Ravens went for it on 4th & goal at the one and failed while down 3 with 5:22 left in the 2nd Quarter. While I think we can all agree maximizing expected points it is correct to go for it in these situations, but did they maximize win probability? Have any posts analyzed situations like this before?

8. Anonymous says:

Sorry I also meant to include in those posts, it was also interesting that both of those games were decided by 3 points or less as well.

9. TPM says:

I keep waiting to get tired of these "4th-down-did-the-coach-make-the-right-call" posts, but I never do. Great read Kurt!

10. Andy says:

to the wizard and company, obviously win probability is the only thing that really matters, and in the extreme example at the last seconds of the game maximizing EP is totally irrelevant. But it is at least interesting to football statistics fans to see where expected points and WP diverge when you get deeper into the game. And I actually think that one of the reasons why Bellicheck is statistically the best coach is because he has a better grasp of the idea that you should keep maximizing points longer than the typical NFL coach thinks. In other words most coaches think that EP and WP diverge greatly way earlier than the numbers say, and they fall for the obligatory prevent defense and conservative offense.

11. Wizard says:

I don't mean to sarcastic, but I just think that if it is irrelevant, why put it? is it of interest? ehhh.. maybe to some. I guess my problem is in the way it article was presented. "The win probability and expected point value each suggest different optimal decisions"

well, not really. expected point value should NOT IN ANY WAY (whether it is of interest or not) suggest a decision, because it is irrelevant.

I will probably be vilified for this, but if an 18 year old is going to post something, he should be held to the same standard as anybody else, and I would have made the same comment if Albert Einstein had written this.

12. LamKram says:

Excellent analysis. This brings up a question I've been thinking about for a while: Is it better to be ahead by 3 or fewer points or by 4 to 8 points late in the game when your opponent has the ball? Obviously, if every team and coach made optimal decisions, then the bigger the lead the better. But it seems to me that in the real world of football, if a team can tie or take the lead with a FG, they will make sub-optimal decisions to "settle" for the FG, even if trying for the TD would maximize their WP. But if they are down by more than three, the decision is easy: they clearly have to go for the TD, which is the optimal decision. Thoughts?