The Passing Paradox Part 2

This is a continuation of an analysis of run/pass balance in the NFL. In part 1 of this article, I discussed the potential application of financial portfolio theory in football strategy. In part 2 of this article, I critique a recent study that made a great stride in this effort.

Commenter JG referred us to a very interesting research paper by economist Duane Rockerbie called "The Passing Premium Revisited." The author applies portfolio theory to re-examine the run-pass balance in the NFL. He finds that teams pass too often. I think his approach is brilliant, but unfortunately his methodology has flaws similar to Alamar's original Passing Premium paper and his conclusions misinterpret his results.

In "Revisited" the author applies a version of the utility function (below) to find the optimal run/pass selection of all 32 NFL teams for the 2006 season. The optimal run/pass ratio is found by taking the derivative of the utility function and setting it equal to zero, thereby finding the curve's maximum. Each team's optimum run/pass ratio is based on the relative strength and variance of their running and passing games.
(The equation basically says that utility of a strategy (X) is a diminishing function of risk aversion/tolerance (alpha) and the expected return of the strategy (v).)

The author finds that most teams do not run as much as they should, and calls the difference between the optimum and actual run/pass ratio "run inefficiency." Run inefficiency is found to be linearly and convincingly correlated with losing. In other words, teams that run as much as they should won more than teams that passed too often. This would be very strong evidence that the run is underused in the NFL, and that the author has discovered a method for instructing coaches how often to run.

First, the computation of expected run yards and expected pass yards leave out some considerations. Like Alamar, the author assigns a -45 ard assessment for each interception. But also like Alamar, he appears to leave out sacks. Sack yards should count against the average pass, and each sack should count as a pass attempt--although the ball was not thrown, a pass play was called. The effect would be bias in favor of the pass. It's also not clear if he factored in additional yardage bonuses on touchdown plays. He doesn't mention it, so I would think not. Since more TDs are from passes than runs, the effect would be bias against the pass.

Also, quarterback scrambles should count as pass yards, not as run yards. They are the result of pass plays, just as sacks are not negative run plays. This might have a large effect on the stats of teams such as Michael Vick's Falcons or Vince Young's Titans in 2006.

But the author goes a step further, and better, than Alamar by excluding kneel downs and clock-stopping spikes from the data. He also factors in penalty yards, which may be important. If passes tended to result in interference calls against the defense, that would make passing look more attractive.

"But the most important factor in a team's risk tolerance may be its defense. With a very strong defense, a team's risk tolerance should be low."

The data is used to calculate the expected (average) result and standard deviation of the run and pass for all 32 teams. From this, he uses the utility function above to estimate each team's optimum ratio of running and passing. But first, the author needs to select the optimum risk tolerance (alpha in the equation above). The optimum risk tolerance is chosen by stipulating that the Chargers' run/pass balance is optimum because they had the best record at 14-2 in 2006, the year from which the data was taken.

I believe this is an error. Recall that the Chargers went 14-2 largely on the back of LaDanian Tomlinson, who led San Diego to an epic 5.7 yds per carry average, and on their #2 ranked defense led by rookie sack leader Shawne Merriman. The author seems unaware of the "running causes winning" fallacy in which teams appear to win because the chose to run more often. In reality, teams that are ahead late in a game, and already very likely to win, chose to run almost exclusively because it is less risky and it burns time off the clock.

The Chargers won 14 of 16 games, presumably leading in almost all of them when they could feed the ball to their talented running back in the 4th quarter. By selecting a 14-win team as the "perfect alpha" team, the author guarantees that any other team that runs less often (accounting for relative strengths of their running and passing abilities) will appear to run less often than they "should."

In fact, I don't believe there is a single uniform alpha for the entire NFL. It changes from down to down and situation to situation. If my team is down by 4 with 2 minutes remaining my alpha would be very negative (very high risk tolerance).

But the most important factor in a team's risk tolerance may be its defense. With a very strong defense, a team's risk tolerance should be low. With a weak defense, a team will likely to need to take additional risks to keep up with its opponent's easy scoring. This leads to my final point.

The author admits that although the selection of the optimum risk tolerance is arbitrary, the correlation of running a lot (accounting for relative strengths) and winning is still strong evidence supporting his utility-maximization analysis. The graph below shows all 32 teams' win totals for the 2006 regular season vs. run inefficiency. The lower the run inefficiency, the more wins a team tends to have.



What we see is that teams that run more often than their risk-reward utility indicate are teams that win. I suspect the direction of causation is that winning allows the running, not the reverse as the author implies.

The next chart is offensive points scored vs run inefficiency. There is a moderate and significant correlation, similar to team wins.


The final chart is points allowed vs. run inefficiency. It's clear that defensive ability explains much of each team's run inefficiency, i.e. run-pass imbalance.


Every team appears to have its own baseline alpha (risk tolerance) based on its defensive strength. Then, based on their respective running and passing abilities, they have an optimum run/pass ratio. But as game situations change in terms of leads, time remaining, and other situational variables, a team's risk tolerance should deviate from its baseline.

There isn't one optimum alpha for the NFL, and if it did it certainly should not be based on the 2006 Chargers. What we see in the charts is that most NFL teams roughly run and pass about as often as they should, given their respective defensive, running, and passing strengths. But some teams do not.

But there's one more wrinkle--football isn't a simple game of yardage optimization. It's complicated by its first down rules. In the final part of this article, I'll examine how this requirement affects play selection, and why I call this concept the "passing paradox."

Continue reading part 3 of The Passing Paradox.

  • Spread The Love
  • Digg This Post
  • Tweet This Post
  • Stumble This Post
  • Submit This Post To Delicious
  • Submit This Post To Reddit
  • Submit This Post To Mixx

3 Responses to “The Passing Paradox Part 2”

  1. JG says:

    This is really, really good. You're ahead of both Rockerbie and Alamar now. I wish I coulda done this. Kudos.

  2. JG says:

    BTW, I hope you're planning to tell us who those outlier teams on your charts are.

  3. JC says:

    Brian, this discussion of financial modelling (risk tolerance/sharpe ratio/etc) projected onto football (and I realize that it was posted a year and a half ago when I say this...) is a waste of time. Investment decisions are made with the goal of growing your bankroll as quickly as possible, taking into account the fact that you will have gains and losses. In a zero-slippage/infinite-liquidity environment a trader would want to bet 10x as big on stock X (perhaps 800 shares instead of 80) when he has $100000 than he would when he has $10000. Risk tolerance is a function of your bankroll, plain and simple.

    Football does not have anything resembling the notion of a bankroll. A team with 35 points does not have 5 times the risk tolerance as a team with 7 points, and a team with a 21 point lead does not have 3 times the risk tolerance of a team with a 7 point lead. These concepts just don't mean anything, and applying formulas that are meaningful in portfolio optimization just looks silly in this context. For that matter, finance doesn't have any meaningful equivalent to going four-and-out.

    One technique that IS relevant in both finance and football analysis is the use of monte-carlo simulations, which I'm sure you already do and are capable of coding up. This enables you to account for the probabilities of various yardage gains, turnovers, etc without losing sight of the fact that this is football, not portfolio optimization.

Leave a Reply

Note: Only a member of this blog may post a comment.