The Passing Paradox Part 3

This is a continuation of an analysis of run/pass balance in the NFL. In part 1 of this article, I discussed the potential application of financial portfolio theory in football strategy. In part 2, I critiqued a recent study that make a great stride toward applying economic and financial math to football. Here in the final part of this article, I present an alternative way of understanding risk and reward in the run/pass balance question.

To recap, two research papers came to opposite conclusions about the run/pass balance in the NFL. The Alamar paper "The Passing Premium" found that the expected gain for a pass is higher than the expected gain for a run, accounting for interceptions. He concluded that teams should pass more often. But the Rockerbie paper "Passing Premium Revisited" found the opposite, that teams pass too much. He applied an economic utility equation that accounts for risk and concluded that running more often helps teams win.

10 Yards for a First Down

Both papers simplify football into a yardage optimization game. Unlike financial investing where the goal is to maximize total return for certain acceptable levels of risk, football requires a minimum gain every 4 downs to maintain possession. At the end of each year no one takes most of your money away if your mutual funds don't earn at least 10%. If they did, and you hadn't made your 10% by November, your risk tolerance would dramatically increase for the final 2 months of the year.

And I think that's how we should model football. Every down and distance situation requires its own risk equation. On first and second down, teams can chose a balanced run/pass attack. But on 3rd down, risk tolerance needs to increase. The net effect would be to bias the offense towards the pass. Although not mathematically optimum in terms of total yardage gain, passing may be optimum when considering the added risk of having to punt.

Take a situation such as 3rd down and 5 yards to go. The table below is the cumulative distribution of yardage gained by running and passing. It lists the cumulative percent of each play that results in at least x yards gained. For example, a run play yields 5 yards or more 24.4% of the time, and a pass play yields 5 yards or more 45.7% of the time.

















Yards GainedRunningPassing
<087.193.2
079.459.2
169.257.6
254.655.8
341.953.5
431.949.9
524.445.7
619.241.2
715.337.1
812.233.3
99.028.7
107.927.3


So given the distribution above for a situation such as 3rd and 5, which type of play should be called? The chance of converting a first down by calling a pass is almost double than that for calling a run. The run is the better choice only in situations requiring gains less than 2 yards.

A coach can call plays with pure yardage optimization balance in mind until third down, commonly considered the do-or-die, make-or-break down. Then, he has to consider the risk of being forced to punt. The coach's decision is reduced to that single play and not an overall strategy. Because most 3rd down situations require more than 2 yards, the run/pass balance is biased toward the pass.

This is why play selection is a paradox. The worse an offense is at passing, the more often it needs to pass, and the higher its risk tolerance needs to be. Incomplete passes on either 1st or 2nd down typically lead to 3rd and long situations, requiring a pass. Teams with poor passing games would also tend to be behind towards the end of a game, which requires even more passing. Teams that don't pass well are therefore forced to play to their weakness. Thus, the passing paradox.

The inverse is also true. The better a team is at passing, the less often they need to do it. They would find themselves ahead in most games, allowing for a lower risk tolerance. Burning time off the clock by running the ball would be to their advantage.

Risk Aversion and Tolerance

In the "Revisited" paper, the author guessed at a perfect risk aversion coefficient for the NFL as a whole. He used the risk aversion (α) for the Chargers, because they had the best record in the year studied. Then he calculated what each teams' run/pass ratio should be based on that league-wide perfect α.

I explained the reasons why this was a bad idea in my last post, notably that poor teams (that tend to be behind) must increase their risk tolerance if they hope to overcome a significant deficit in a game, particularly towards the end of the game. Further, the analysis in the "Revisited" paper failed to recognize that it's winning that often leads to running, rather than the other way around.

So instead of choosing a perfect α based on a single team, then apply it to the entire NFL, why not calculate what each team's actual α was based on their actual run/pass balance? If I'm right about how winning leads to running, teams with a lot of wins should have a risk-averse portfolio, and teams with a lot of losses should have a risk-tolerant portfolio.

So that's what I did. The equation below solves for risk aversion in the maximized utility equation, instead of run/pass ratio as the author of "Revisited" did.

where:
α = risk aversion (negative values are risk tolerant, zero is neutral)
γ = % of plays that are runs
μR = mean (expected) gain of runs
μP = mean (expected) gain of passes
σR = standard deviation of run gains
σP = standard deviation of pass gains

The table below lists each team's running and passing stats (borrowed from "Revisited"), their actual play selection balance, their number of wins, and their calculated risk level according to the equation above. Keep in mind that positive α means risk aversion and negative α indicates risk tolerance. The list is sorted from most risk averse (conservative) offenses at top to the most risk tolerant (aggressive) at bottom. Click on the table headers to sort as desired.







































TeamR Avg (μR)P Avg (μP)R SD (σR)
P SD (σP)Actual (γ)WinsRisk (α)
NE4.06.08.211.60.46120.039
NO3.17.39.015.70.39100.030
SD5.76.59.411.50.49140.029
IND4.56.65.912.40.45120.026
BAL3.85.26.212.20.45130.020
DAL4.56.35.813.70.4990.019
CIN4.16.56.015.00.4380.019
STL4.15.57.112.00.3880.018
KC4.55.67.013.50.5290.016
NYJ3.64.95.913.80.50100.016
ARI2.74.16.514.20.3850.013
MIA3.64.38.912.00.3760.011
HOU3.64.17.710.90.4060.011
WAS4.85.56.212.80.4950.011
PHI4.96.19.515.60.39100.011
DET3.94.89.113.00.3230.009
JAX4.85.26.911.90.5180.009
MIN3.94.48.912.70.4260.008
CHI3.94.85.315.10.48130.008
CLE2.73.48.013.80.4040.008
CAR4.24.77.613.10.4280.007
GB4.24.88.113.90.3880.006
BUF3.84.26.015.10.4670.003
NYG4.95.08.012.30.4580.002
SEA3.84.06.713.90.4390.002
SF4.44.510.914.20.4870.002
TB3.63.67.711.90.3940.000
DEN4.84.78.413.50.449-0.001
PIT4.73.69.514.70.368-0.011
TEN4.33.78.612.70.488-0.013
OAK3.62.58.413.70.442-0.016
ATL5.74.210.512.40.547-0.438


Notice that most teams are very close to neutral risk (α = 0) but with one very large exception. Michael Vick's Falcons appear to be the biggest risk takers by far, with an eye-popping α = -0.438. But I think that result is due to the unique nature of Vick's offense. His runs were very boom and bust, with either a big gain or deep sack. Those were often called pass plays in which Vick scrambled. Plus, his running ability on the outside often opened up running holes for conventional run plays on the inside.

Most other teams, however, tilted slightly positive, meaning they were slightly risk averse. The teams with the most wins tended to be the teams that were most risk averse. Teams such as NE, NO, SD, IND, and BAL top the list of the most conservative offenses. They were also the best teams of 2006, with one team missing.

The NFC champion Bears managed 13 wins with a relatively risky offensive balance. This is due to their boom and bust passing game (μP = 4.8, σP=15.1). This result suggests that in 2006 CHI rolled the dice often with deep pass plays and got lucky. 2007 wasn't so kind to them.

One interesting application of this kind of risk analysis would be to repeat these calculations for multiple years to see which coaches and/or coordinators really are the most conservative and who are the biggest gamblers. I was surprised to see Belichick as coach of the most risk averse team. He does have a reputation for running on 3rd and short more often than other teams, so perhaps that explains NE's placement on top of the list.

Below is a graph of risk aversion vs. team wins. We can see that teams with a lot of wins generally are the teams that can afford to be conservative.


One possible application of this graph is to measure the vertical distance between the best-fit line and each team's risk aversion score. This distance is the regression "residual" accounting for wins and losses. It basically says how risk averse/tolerant a team was accounting for its wins. A multi-variate regression would be even better, accounting for both team defensive ability and wins. And instead of wins, we could use "4th quarter leads," which would be what really drives deviations from optimum risk tolerance. This analysis has the potential to be a good measure of how well a coach understands the game and his team--not just their running ability and passing ability, but their defensive ability as well.

There is tremendous potential for the application of portfolio theory in football. The "10 yards in 4 downs rule" complicates the analysis, however. Accordingly, each type of down and distance situation may require its own analysis. Plus, play-calling is not a simple pass or run binary decision. There are draws, screens, outs, hitches, flares, and all sorts of other unique plays. The risks and benefits of each type of play also require their own analysis.

To me, this is exciting because football may finally have a way of matching the depth of mathematical analysis pioneered by our sabremetrician friends in baseball. Baseball is simpler in many ways--run production is generally linearly additive and there are very few options for a team or player to increase or decrease risk as the situation requires. Unlike most other sports, risk in football is dynamic. Perhaps that's what makes it so exciting.

  • Spread The Love
  • Digg This Post
  • Tweet This Post
  • Stumble This Post
  • Submit This Post To Delicious
  • Submit This Post To Reddit
  • Submit This Post To Mixx

6 Responses to “The Passing Paradox Part 3”

  1. Unknown says:

    Brian,

    Wow, fascinating stuff. I'm not much of a stats whiz. Are yards gained distributed with a standard Gaussian distribution? It seems that there would be a fat tail on the right (positive yardage) side. Not sure if that changes the analysis at all.

  2. Tarr says:

    Great stuff. It's funny, and unsurprising, to see the Denny Green Cardinals as the most risk-averse losing team. Of course we all remember the infamous Bears game, where the Cardinals built a big lead, and then decided the best way to maintain it was to go double tight end I formation, run Edgerrin James into the line three times, and punt to Devin Hester.

    I disagree with one thing you say:

    A coach can call plays with pure yardage optimization balance in mind until third down, commonly considered the do-or-die, make-or-break down. Then, he has to consider the risk of being forced to punt.

    In reality, breaking down the analysis by down and distance opens up further complexities. What you should do on earlier downs is dictated not merely by yardage optimization, but by how good your team is at getting out of a hole. Your risk-reward analysis will still lead you to yardage optimization on 2nd-and-10 if you have a great passing attack that can rescue you on 3rd-and-10. But if you don't, then the fear of 3rd-and-10 leads you to more conservative play calls.

  3. Brian Burke says:

    Yes, the distributions would be heavily skewed right. I'm not sure how it changes things either (yet).

    Good point about "getting out of a hole." A good passing team might have more flexibility on 2nd down. I wonder though.

    What if my team is a poor passing team? I run on 1st down. It's now 2nd and 8. Would I go for a run to make 3rd down more manageable, say 3rd and 4. Or would I want two pass attempts at making the 8 yards?

  4. Anonymous says:

    Isn't the Football Outsider's approach similar (ie, rate each play as success/fail depending on the down and distance (well the percent distance to go)?

  5. Brian Burke says:

    True, but the Football Outsiders' approach is similar only in that they grade each team on a play-by-play basis. But DVOA is an ad hoc retrospective scoring system for ranking teams, not a risk/reward analysis tool.

    What I'm proposing is using the risk/reward math to identify future optimum strategies for play selection and other decisions. I'm not proposing this as a prediction model or as a system for some sort of power ranking.

  6. Scott Segal says:

    Wow. I stumbled upon this site looking for some stats helping in my fantasy football rankings and I come up with this potentially game changing research. Wonderful work. Im not a math whiz so the formulas are a touch geeky to me but you do a brilliant job simplifying it.

    One thing I wanted to add however.

    1- Field position should factor as a team that is 3rd and 5 say inside the 40 yd line may run the ball on a draw thinking they would pound it on 4th and 1 if the element of surprise didnt work. Dallas did this a lot the past couple of years with Barber running shotgun draws.

    2- A teams defensive capabilities may affect the offensive thought process as a confident coach who is defensive minded would want to score explosively (with more risk) via the pass to put his defense at an advantage forcing the opponent to pass.

    Just my 2 cents.

Leave a Reply

Note: Only a member of this blog may post a comment.