The Passing Paradox Part 3

This is a continuation of an analysis of run/pass balance in the NFL. In part 1 of this article, I discussed the potential application of financial portfolio theory in football strategy. In part 2, I critiqued a recent study that make a great stride toward applying economic and financial math to football. Here in the final part of this article, I present an alternative way of understanding risk and reward in the run/pass balance question.

To recap, two research papers came to opposite conclusions about the run/pass balance in the NFL. The Alamar paper "The Passing Premium" found that the expected gain for a pass is higher than the expected gain for a run, accounting for interceptions. He concluded that teams should pass more often. But the Rockerbie paper "Passing Premium Revisited" found the opposite, that teams pass too much. He applied an economic utility equation that accounts for risk and concluded that running more often helps teams win.

10 Yards for a First Down

Both papers simplify football into a yardage optimization game. Unlike financial investing where the goal is to maximize total return for certain acceptable levels of risk, football requires a minimum gain every 4 downs to maintain possession. At the end of each year no one takes most of your money away if your mutual funds don't earn at least 10%. If they did, and you hadn't made your 10% by November, your risk tolerance would dramatically increase for the final 2 months of the year.

And I think that's how we should model football. Every down and distance situation requires its own risk equation. On first and second down, teams can chose a balanced run/pass attack. But on 3rd down, risk tolerance needs to increase. The net effect would be to bias the offense towards the pass. Although not mathematically optimum in terms of total yardage gain, passing may be optimum when considering the added risk of having to punt.

Take a situation such as 3rd down and 5 yards to go. The table below is the cumulative distribution of yardage gained by running and passing. It lists the cumulative percent of each play that results in at least x yards gained. For example, a run play yields 5 yards or more 24.4% of the time, and a pass play yields 5 yards or more 45.7% of the time.

Yards Gained	Running	Passing
<0	87.1	93.2
0	79.4	59.2
1	69.2	57.6
2	54.6	55.8
3	41.9	53.5
4	31.9	49.9
5	24.4	45.7
6	19.2	41.2
7	15.3	37.1
8	12.2	33.3
9	9.0	28.7
10	7.9	27.3

So given the distribution above for a situation such as 3rd and 5, which type of play should be called? The chance of converting a first down by calling a pass is almost double than that for calling a run. The run is the better choice only in situations requiring gains less than 2 yards.

A coach can call plays with pure yardage optimization balance in mind until third down, commonly considered the do-or-die, make-or-break down. Then, he has to consider the risk of being forced to punt. The coach's decision is reduced to that single play and not an overall strategy. Because most 3rd down situations require more than 2 yards, the run/pass balance is biased toward the pass.

This is why play selection is a paradox. The worse an offense is at passing, the more often it needs to pass, and the higher its risk tolerance needs to be. Incomplete passes on either 1st or 2nd down typically lead to 3rd and long situations, requiring a pass. Teams with poor passing games would also tend to be behind towards the end of a game, which requires even more passing. Teams that don't pass well are therefore forced to play to their weakness. Thus, the passing paradox.

The inverse is also true. The better a team is at passing, the less often they need to do it. They would find themselves ahead in most games, allowing for a lower risk tolerance. Burning time off the clock by running the ball would be to their advantage.

Risk Aversion and Tolerance

In the "Revisited" paper, the author guessed at a perfect risk aversion coefficient for the NFL as a whole. He used the risk aversion (α) for the Chargers, because they had the best record in the year studied. Then he calculated what each teams' run/pass ratio should be based on that league-wide perfect α.

I explained the reasons why this was a bad idea in my last post, notably that poor teams (that tend to be behind) must increase their risk tolerance if they hope to overcome a significant deficit in a game, particularly towards the end of the game. Further, the analysis in the "Revisited" paper failed to recognize that it's winning that often leads to running, rather than the other way around.

So instead of choosing a perfect α based on a single team, then apply it to the entire NFL, why not calculate what each team's actual α was based on their actual run/pass balance? If I'm right about how winning leads to running, teams with a lot of wins should have a risk-averse portfolio, and teams with a lot of losses should have a risk-tolerant portfolio.

So that's what I did. The equation below solves for risk aversion in the maximized utility equation, instead of run/pass ratio as the author of "Revisited" did.

where:
α = risk aversion (negative values are risk tolerant, zero is neutral)
γ = % of plays that are runs
μ_R = mean (expected) gain of runs
μ_P = mean (expected) gain of passes
σ_R = standard deviation of run gains
σ_P = standard deviation of pass gains

The table below lists each team's running and passing stats (borrowed from "Revisited"), their actual play selection balance, their number of wins, and their calculated risk level according to the equation above. Keep in mind that positive α means risk aversion and negative α indicates risk tolerance. The list is sorted from most risk averse (conservative) offenses at top to the most risk tolerant (aggressive) at bottom. Click on the table headers to sort as desired.

Team	R Avg (μR)	P Avg (μP)	R SD (σR)	P SD (σP)	Actual (γ)	Wins	Risk (α)
NE	4.0	6.0	8.2	11.6	0.46	12	0.039
NO	3.1	7.3	9.0	15.7	0.39	10	0.030
SD	5.7	6.5	9.4	11.5	0.49	14	0.029
IND	4.5	6.6	5.9	12.4	0.45	12	0.026
BAL	3.8	5.2	6.2	12.2	0.45	13	0.020
DAL	4.5	6.3	5.8	13.7	0.49	9	0.019
CIN	4.1	6.5	6.0	15.0	0.43	8	0.019
STL	4.1	5.5	7.1	12.0	0.38	8	0.018
KC	4.5	5.6	7.0	13.5	0.52	9	0.016
NYJ	3.6	4.9	5.9	13.8	0.50	10	0.016
ARI	2.7	4.1	6.5	14.2	0.38	5	0.013
MIA	3.6	4.3	8.9	12.0	0.37	6	0.011
HOU	3.6	4.1	7.7	10.9	0.40	6	0.011
WAS	4.8	5.5	6.2	12.8	0.49	5	0.011
PHI	4.9	6.1	9.5	15.6	0.39	10	0.011
DET	3.9	4.8	9.1	13.0	0.32	3	0.009
JAX	4.8	5.2	6.9	11.9	0.51	8	0.009
MIN	3.9	4.4	8.9	12.7	0.42	6	0.008
CHI	3.9	4.8	5.3	15.1	0.48	13	0.008
CLE	2.7	3.4	8.0	13.8	0.40	4	0.008
CAR	4.2	4.7	7.6	13.1	0.42	8	0.007
GB	4.2	4.8	8.1	13.9	0.38	8	0.006
BUF	3.8	4.2	6.0	15.1	0.46	7	0.003
NYG	4.9	5.0	8.0	12.3	0.45	8	0.002
SEA	3.8	4.0	6.7	13.9	0.43	9	0.002
SF	4.4	4.5	10.9	14.2	0.48	7	0.002
TB	3.6	3.6	7.7	11.9	0.39	4	0.000
DEN	4.8	4.7	8.4	13.5	0.44	9	-0.001
PIT	4.7	3.6	9.5	14.7	0.36	8	-0.011
TEN	4.3	3.7	8.6	12.7	0.48	8	-0.013
OAK	3.6	2.5	8.4	13.7	0.44	2	-0.016
ATL	5.7	4.2	10.5	12.4	0.54	7	-0.438

Notice that most teams are very close to neutral risk (α = 0) but with one very large exception. Michael Vick's Falcons appear to be the biggest risk takers by far, with an eye-popping α = -0.438. But I think that result is due to the unique nature of Vick's offense. His runs were very boom and bust, with either a big gain or deep sack. Those were often called pass plays in which Vick scrambled. Plus, his running ability on the outside often opened up running holes for conventional run plays on the inside.

Most other teams, however, tilted slightly positive, meaning they were slightly risk averse. The teams with the most wins tended to be the teams that were most risk averse. Teams such as NE, NO, SD, IND, and BAL top the list of the most conservative offenses. They were also the best teams of 2006, with one team missing.

The NFC champion Bears managed 13 wins with a relatively risky offensive balance. This is due to their boom and bust passing game (μ_P = 4.8, σ_P=15.1). This result suggests that in 2006 CHI rolled the dice often with deep pass plays and got lucky. 2007 wasn't so kind to them.

One interesting application of this kind of risk analysis would be to repeat these calculations for multiple years to see which coaches and/or coordinators really are the most conservative and who are the biggest gamblers. I was surprised to see Belichick as coach of the most risk averse team. He does have a reputation for running on 3rd and short more often than other teams, so perhaps that explains NE's placement on top of the list.

Below is a graph of risk aversion vs. team wins. We can see that teams with a lot of wins generally are the teams that can afford to be conservative.


One possible application of this graph is to measure the vertical distance between the best-fit line and each team's risk aversion score. This distance is the regression "residual" accounting for wins and losses. It basically says how risk averse/tolerant a team was accounting for its wins. A multi-variate regression would be even better, accounting for both team defensive ability and wins. And instead of wins, we could use "4th quarter leads," which would be what really drives deviations from optimum risk tolerance. This analysis has the potential to be a good measure of how well a coach understands the game and his team--not just their running ability and passing ability, but their defensive ability as well.

There is tremendous potential for the application of portfolio theory in football. The "10 yards in 4 downs rule" complicates the analysis, however. Accordingly, each type of down and distance situation may require its own analysis. Plus, play-calling is not a simple pass or run binary decision. There are draws, screens, outs, hitches, flares, and all sorts of other unique plays. The risks and benefits of each type of play also require their own analysis.

To me, this is exciting because football may finally have a way of matching the depth of mathematical analysis pioneered by our sabremetrician friends in baseball. Baseball is simpler in many ways--run production is generally linearly additive and there are very few options for a team or player to increase or decrease risk as the situation requires. Unlike most other sports, risk in football is dynamic. Perhaps that's what makes it so exciting.