This is the third part of my look at momentum in the NFL. The first part examined whether several momentum-swinging types of events caused any increase in a team's chances of scoring on the subsequent possession. The second part compared the expected and observed Win Probability (WP) following momentum-swinging events to find out whether those events increased a team's chances of winning beyond what we would otherwise expect.

This installment cuts to the chase. From a strategic perspective, we want to understand how momentum may or may not affect the game so that coaches can make better decisions. Often, momentum is cited as a consideration to forgo strategically optimal choices for fear of losing the emotional and psychological edge thought to comprise momentum.

Here's the thinking: If a team tries to convert on 4th down but fails or unsuccessfully tries for a two-point conversion, it gives up the momentum to the other team. The implication is that failing on 4th down means that winning is now less probable than the resulting situation indicates, beyond what the numbers say. Therefore, the WP and Expected Points (EP) models used to estimate the values of the options no longer apply. In a nutshell, the analytic models underestimate the cost of failing.

[By the same token, the reverse argument should be just as valid. Wouldn't succeeding in a momentum-swinging play mean the chances of winning are even higher than the numbers indicate? For now, I'll set the 'upside' argument aside and examine only the 'downside' claim.]

Methodology

As I did in Part 2, I plotted the observed (actual) winning percentage from all drives from 1999-2013 (week 8) against the expected WP based on the game state. The plots are broken out according to whether the offense gained possession following a punt or following a turnover on downs.

The WP model is agnostic to how possession is obtained, so comparing its estimates to the actual winning percentages tells us whether teams win more or less often than expected given a particular game state. As I wrote in the previous article:
Because the WP model is ignorant of past events and only the present state, true momentum-swinging events should cause the model to break--if momentum is real, that is. In other words, teams with positive momentum should win more often than the WP model predicts, and teams without momentum should win less often than the WP model predicts.

The blue dashed line represents perfect calibration of the WP model. For example, if the model is tuned correctly, teams that have a 0.80 WP will go on to win 80% of the time. The green line represents game outcomes for offenses that obtained possession following a punt. The red line represents game outcomes for offenses that obtained possession following a turnover on downs.

I realize this isn't the most intuitive way to present this information. For clarity's sake, if the line is above the diagonal, it means that teams gaining possession go on to win more often than expected. If the line is below the diagonal, it means that teams gaining possession win less often than expected. [In retrospect, I wish I had reversed the chart's presentation, making it from the perspective of the team making the decision to either punt or go for it, and not from the perspective of the team gaining possession. This would make 'up' represent winning more often for the team with the choice.]

Results

Let's look specifically at the red line representing turnovers on downs. It is primarily and consistently below the diagonal, indicating that offenses that gain possession following a failed 4th down conversion attempt usually win less often than otherwise expected. This result suggests that there is no true momentum loss following a turnover on downs. The opposite may be true.

And just to be clear: These results do not say that failing on 4th down is better than punting in any particular circumstance. It's saying: assuming that you've handed possession over to the other team in comparable sets of circumstances, teams that failed on 4th down win slightly more often than teams that punted.

Unfortunately, we can't stop there because the results could be biased. It's possible that teams that go for it on 4th down and win anyway just happen to be very good teams. Perhaps they have great defenses that can get the ball back quickly, or maybe their offense is potent enough to more than make up for turning the ball over on downs. Maybe that's why coaches of those teams feel that they can take bigger risks. If so, this tendency would mask the loss of momentum. For example, if NE (.710 win%) and IND (.660 win%) are the teams with all the failed 4th down attempts, they would naturally win more often just because they were good teams.

Bias?

To check for such a bias, I broke out each team's winning percentage in each year in the data set. Next, I weighted all the team win% by how often each team failed to convert a 4th down in each season. Lastly I recomputed the average weighted win%. I did this twice, once for all the failed 4th down attempts in which teams went on to win, and once for all the failed 4th down attempts in which teams went on to lose.

The average weighted win% for teams that failed on 4th down but went on to win was .577, which is considerable but not unexpected. After all, we're spotting ourselves at least one win and eliminating virtually all the teams with very few wins in a given season. This may indicate there is bias which would mask the presence of momentum loss after a turnover on downs. But the bias is stronger in the other direction. The average weighted win% for teams that failed on 4th down but went on to lose was .415, slightly further from .500 than the .577 teams that went on to win. Although there is some bias that may mask momentum loss, on net the bias points slightly in the other direction. If anything, we would be seeing momentum when it's not really there.

This makes some intuitive sense. Losing teams that fail to convert on 4th downs, well, they're likely to go on losing. So it's easy to see how the perception that turning the ball over on downs can create negative momentum gets reinforced.

Conclusion

The bottom line of this analysis is that turning the ball over on downs does not cause a team to lose more often than would otherwise be expected, given the same game state after the change of possession.  Coaches should not be fearful of losing momentum.