## Atlanta's 4th Down Decision Vs. Tampa

The Falcons trailed the Buccaneers by 6 with 4:11 left in the 4th. They faced a seemingly impossible 4th and goal from the Tampa 15. Head coach Mike Smith elected for the field goal attempt. Was this the right call?

4th and goals from the 15 might seem impossible, but they're not. The success rate is low but not zero. In fact it's about 14%. That's not a welcome prospect, but neither is handing the ball to an opponent who's up by 3 points with 4 minutes to play. And even if the conversion attempt fails, it's far from fatal. The calculus merely changes from needing a FG to tie, to needing a TD to win. Notice I said needing a FG to tie. Sure, TDs are harder to come by than FGs, but tying is not the object of the game. Put simply, TDs are more than half as likely as FGs in that situation, which makes the gamble for a tying FG a sucker's bet.

FG attempts are successful 88% of the time from the 15. A successful FG gives the Bucs the ball near their own 20 with a 3-point lead, worth 0.19 Win Probability (WP). A missed attempt hands the ball to the Bucs at their own 22, worth 0.17 WP. (If you're paying attention, you noticed that making the field goal is only worth +0.02 WPA.) The total WP for the FG attempt is:

0.88 * 0.19 + (1-0.88) * 0.17 = 0.19 WP

As mentioned, going for it in this situation will be successful about 14% of the time. A TD takes a 1-point lead, handing the ball to Tampa near their own 20, worth 0.51 WP. A failed conversion attempt is worth  0.22 WP--more than a made FG! In total, the WP for the conversion attempt is:

0.14 * 0.51 + (1-0.14) * 0.22 = 0.26 WP

So going for the TD, then and there, is the better decision, all other things being equal. Admittedly, 4th and goal from the 15 is very unusual. There just aren't many examples to use as a foundation for the 14% conversion rate, but 14% is not unreasonable at all. It's mostly based on extrapolated and regressed 3rd down data. To make the FG attempt worthwhile, you'd have to drive the conversion probability down to a break-even point of...well, there is no break-even point. It would never be a good idea to try the field goal, because the way typical coaches tend to call games, it would be better to simply take a knee on the 15 than try the field goal.

As crazy as it sounds, this odd situation can be explained by three heuristic errors. First, FGs are not automatic. Coaches need to get in the habit of saying "send in the FG attempt unit." Second, Atlanta head coach Mike Smith failed to consider that the consequence of a conversion failure is not fatal. In fact, it's better than making a FG. Teams with 6-point leads play more conservative than those with 3-point leads. And teams down by 6 play more aggressively than teams down by 3. Lastly, I think Mike Smith fell to the conjunctive fallacy, a logical error to which Buccaneers coach Raheem Morris has fallen victim in the past. By trying the FG, Smith was counting on:

1. Making the kick
2. Preventing a TB score and getting the ball back
3. Driving back in FG range
4. Making another FG
5. Scoring first in OT, which requires:
a. (Half the time) Making another stop
b. Driving into FG range again, and
c. Making a third FG, or scoring  a TD.

That's a lot of things to go right, all in a row.

The difference between the FG miss WP of 0.17 and the failed conversion WP of 0.22 is 0.05, a considerable amount for merely 7 yds of field position. But those 7 yards are critical in terms of making a prospective FG. Assuming you get the ball back, those yards might mean the difference between getting within the mythical 'FG range' or not, or assuming you do, the difference between a 50% kick and a 63% kick.

The conventional thought process is "Let's just try to play for the tie here, and we'll roll the dice in OT." But the right idea is "Let's roll the dice here to take the lead, but if we fail, we've still got about the same chance to win as if we tried a FG."

As it turns out, Atlanta made the FG, but never saw the ball again, falling to the Bucs 16-13.

### 17 Responses to “Atlanta's 4th Down Decision Vs. Tampa”

1. Anonymous says:

I'm not sure that this fully passes the smell test: if a team is up 6 with 4:00 left on its own 15, a 7-yard field position improvement (no change in down/dx) increases its chance of winning from 78% to 83%?

The other team still has to score a TD to win. Obviously, if the data is sufficient, I'll believe it. But it doesn't seem that logical.

2. Ian Simcox says:

Always interests me to look at the play calls in these situations. The sack on 1st-5 hurt, but from 2nd-15 it's not all lost, assuming you realise that you've still got three more plays to get it into the end zone.

Incomplete, incomplete, field goal shows that Atlanta had no idea that the field goal added nothing to their WP.

3. Brian Burke says:

Looking at the data, it appears that there is a severe drop off in WP behind a team's own 20. In other words, when the team ahead by 6 is behind its 20, they play differently than if they were outside their 20.

4. Dan Whitney says:

First of all I just wanted to say that this is really a great site, and that almost all of the analysis is extremely fascinating and not too difficult to follow. Second I was just wondering what yard line you were calculating the WP for on the failed 4th down conversion. It seems to me like a stop on the 5 would lead to a much different WP then an incomplete pass that gives the Bucs the ball on the 15.
Also your model doesn't consider the option of making the field goal and then attempting an onside kick, I was wondering what your thoughts on that idea was/ what the WP for that is. Once again great analysis and I look forward to reading your posts this season.

5. Brian Burke says:

Dan-Good points. When I do these analyses, I always say that the failed attempt will leave the ball at the original line of scrimmage. From the perspective of 'going for it', it's a near-worst case scenario that allows us to say, 'even with conservative assumptions, it's still worthwhile to go for it.'

An onside kick is factored in as a possibility, at least to the degree that teams have tried that strategy in the past. With 4 min to play, I suspect that's not very often. You have to think that TB would be on guard for an onside kick, and the recovery probability would be very low. Only if ATL had no more TOs should that be considered. It's easier to get a stop than recover an expected onside kick.

6. Dan Whitney says:

Brian, Thanks a lot for the quick response
Also just wanted to point out that in the last section you talk about the probability of making a perspective field goal when Atlanta would be down by 6 points. I imagine that the extra probability of getting a touchdown would be kind of similar, but I imagine it might be enough to make a difference. Thanks again, Dan

7. LarryinLA says:

Love this site.

Isn't an INT touchback a strong possibility in this situation, so the 15-yd line isn't the worst case likely scenario for a failed conversion?

Also, given the assumptions of everyone involved, and the low realtive downside of failure, this seems like the ideal scenario for a fake FG.

8. Joseph says:

Larry--didn't see the actual game, so I can't say for sure--but your thought process is great. Line up for the FG--and give the holder the chance to call for the fake, or to actually kick it if TB keeps in its normal defense and only half-heartedly lines up to block the kick.

Brian--IMO, your model underestimates the WP for ATL with that much time remaining. For example, for this week's Texans/Saints game, I was worried that the Saints left too much time on the clock for the Texans' explosive offense. But in a 16-13 game, with ~4 min left, Smith must have thought that TB needed 2 1st downs, and prob. 3, to ice the game. He assumes that his defense, which has not allowed TB to move up and down the field at will, can get the stop and give him a chance to get that tying FG or go-ahead TD. He doesn't like his chances of 4th and goal from the 15 as much.
I also suspect that your model SEVERELY overstates the 4th and goal from the 15. The defense is defending a much smaller area than 4th and 15 from midfield, for example. Including 3rd down plays only makes it worse. The offense has the legitimate option, esp. toward the end of the game, to try to make a 10 yd play to set up 4th & 5--which a defense must try to defend. In fact, even if it isn't the end of the game, a team backed up in its own end will do that to provide space for the punter.
Now, I understand that true percentages on 4th & goal from the 15 suffer from small sample size. What are the real percentages for 4th and goal from 12+ yds? Or even 10+? I'll bet that with 4th & goal from X yd line, the probability decreases as X increases. So take all 4th and goal attempts, plot the line as best you can, and then give the estimate.
I do agree with your analysis in this, though: if ATL gets a positive, but non-scoring play (esp. of 5+ yds), their chances of stopping TB and getting the ball back to score the game-winning TD are better, because TB is likely to be more conservative to avoid a turnover while running time off the clock.

9. Jeff Clarke says:

Brian,

I'm curious about how you arrived at the probability for estimating 4th and 15. I agree that it normally makes sense to use 3rd down conversions as a proxy for fourth down conversions. The thing is that on 3rd and goal from the 15, I believe a lot of coaches don't make a realistic effort to get a touchdown. They call it safe and run a draw play, calculating that they don't want to ruin a legitimate opportunity at a field goal. In my opinion this is overconservative and it isn't logical but it is definitely the m.o. of NFL coaches.

I don't know how you could do it (passes only??) but I think 3rd and 15 stats probably underestimate the probability of actually converting when you actually make a full fledged attempt to score a TD in that situation.

Or was that already accounted for in the extrapolation and regression?

10. Brian Burke says:

Joseph-You hit the nail on the head in your last paragraph. The ultimate point is that no matter how small the chances are of converting from the 15, it makes no sense to go from 6 points down to 3 points down.

Also, the number of actual 4th and goal from the 15 conversion attempts is very, very tiny. Far too small for any conclusions. The conversion probabilities start with 3rd down probabilities and use actual 4th down results to verify there is no systematic bias (due to the super-safe option of running on 3rd, for example). The numbers do take into the compression effects of the shortened field in the red zone. There are 3 sets of numbers I use--outside the 20, from the 20 to the 10, and inside the 10. Each set of numbers follows a consistent curve with a certain shape. By the time you get >10 yds to go, the probabilities 'flatten', meaning the chances of conversion for 4th and 15 aren't much smaller than 4th and 11.

In the end, it's just a baseline estimate. If you prefer different input numbers, that's ok. I'm mostly wedded to the process, and people are free to use whatever numbers they like.

That's one reason why I favor calculating the break-even success probability and letting the coach make the decision about whether or not the chances of conversion are higher or lower. He should also be aware of the league baseline, to help anchor his estimation.

Jeff-The 3rd down numbers used only come from 'normal' situations, i.e. within 10 points in the 1st and 3rd period. That way, end-game hyper-conservatism won't be a factor. It also weeds out the desperation plays of poor passing teams throughout trash time.

There are many, many considerations, and each has its own trade off in terms of sample size, relevance, assumptions required, etc. There's no perfect answer. 14% seems about right (1 in 7) for a single 15-yd play, and it's consistent with the broader numbers.

It was a good point above about an INT in the EZ. An INT isn't nearly as fatal as in other situations, which would allow greater risk acceptance on ATL's part if they went for it. And on the other hand, an INT that ends in a touchback puts the ball at the 20, not the 15. That's true, but would have a negligible effect on the final analysis. Further, it's balanced by the possibility of a play that ends short of the EZ but inside the 15.

Great points, all.

11. Jeff Clarke says:

Thanks

Here is the thing though: I think that coaches are hyper-conservative during "normal" situations. I've seen a lot of coaches call running plays straight up the middle in close games during the 3rd quarter on 3rd and 15. I know that its not because they believe it gives them the best opportunity at scoring.

I'm curious how many times people went for it on 4th and 10-20 yards.

12. Ian Simcox says:

Another thing I just realised about not going for it on 4th and goal. If they had failed and turned the ball over inside the 5 yard line, Tampa Bay might well have had to look at the possibility of taking the intentional safety - which would have opened up even more fun "what's the right decision?" posts.

So not only does Mike Smith make the wrong call, he also frustrates the advanced stats community by denying us a chance to look at a rare event. Shocking behaviour.

13. Brian Burke says:

That would have been so cool.

On the 4th and 15 numbers:
There have been 69 actual 4th and 15 attempts of all kinds at all spots on the field. 68 passes and 1 run. In total, they're successful 26% of the time. That includes all trash time plays.

For 3rd and goal from the actual 15. They're successful 14% of the time(result in either a 1st down or TD. Passes are successful 17.5 % of the time, runs 0% of the time. There were 40 passes, and 10 runs.

That's way too many runs to make sense in game theory context, so Jeff is probably correct about the hyper-conservativism on the play calls. This consideration suggests the actual success probability for ATL was probably higher than the 14% overall rate I used.

14. Anonymous says:

"A successful FG gives the Bucs the ball near their own 20 with a 3-point lead, worth 0.19 Win Probability (WP). A missed attempt hands the ball to the Bucs at their own 22, worth 0.17 WP."

Can someone explain how WP is calculated?

15. Brian Burke says:

Sure. This is a bit outdated as the model has been improved since, but it's a good primer on how it was created.

16. Pete says:

Generally, I strongly agree with the go for the TD strategy. But in this case, calling a field goal a sucker's bet might over-state the case. 14% is unlikely, but it's reasonable to assume that the 14% of teams that converted in that case had better offenses than the 86% that didn't. I bet Aaron Rodgers makes a TD in that situation more often than Tavaris Jackson. So let's assume that the Falcon's staff quant put the WP stats in front of Mike Smith when he is making the call. He might respond, "yeah, but we've scored a grand total of one touchdown in the other 56 minutes of the game and our offense is one of the worst in the NFL this year. Tampa also has only one touchdown today and Josh Freeman ain't John Elway, so I like my chances of stopping them quickly and getting the ball back." It still might be the wrong call, but I'm not going to beat up Mike Smith too much for going for a 0.19 vs. a 0.26 WP in that situation even if he didn't know what he was doing.

17. Pete says:

Generally, I strongly agree with the go for the TD strategy. But in this case, calling a field goal a sucker's bet might over-state the case. 14% is unlikely, but it's reasonable to assume that the 14% of teams that converted in that case had better offenses than the 86% that didn't. I bet Aaron Rodgers makes a TD in that situation more often than Tavaris Jackson. So let's assume that the Falcon's staff quant put the WP stats in front of Mike Smith when he is making the call. He might respond, "yeah, but we've scored a grand total of one touchdown in the other 56 minutes of the game and our offense is one of the worst in the NFL this year. Tampa also has only one touchdown today and Josh Freeman ain't John Elway, so I like my chances of stopping them quickly and getting the ball back." It still might be the wrong call, but I'm not going to beat up Mike Smith too much for going for a 0.19 vs. a 0.26 WP in that situation even if he didn't know what he was doing.

### Leave a Reply

Note: Only a member of this blog may post a comment.