In previous posts, I've referred to a concept called the passing premium. Specifically, I point to Benjamin Alamar's paper which found that the expected yards per play is higher for passing than for running. The difference accounts for incompletions and the risk of interception.
I've discovered a flaw in the author's analysis, however. He does not appear to account for sacks.
His analysis finds that for every passing play in the 2005 NFL season, the expected gain is 5.8 yards per attempt. Interceptions are factored in by assigning them a -45 yard value. (40-50 yards as an equivalent for interceptions is a commonly accepted value. It also makes intuitive sense because an interception differs from an incompletion by precluding the possibility of a punt, which usually nets about 40 yards.) Touchdown passes also get an adjustment because the goal line truncates the pass. For every touchdown pass, an extra 10 yds is added.
The true expected gain from a pass play should be:
(Pass Yds Gained - Sack Yds - Int Adjustment + TD Adjustment) / (Pass Att + Sacks)
The author leaves out the sacks both in the numerator and denominator, which makes a difference.
The average run yields 4.1 yards. The difference is an unexplained premium for passing, suggesting that play selection is not rationally balanced in the NFL.
Realizing that football is more complex than a binary run or pass decision, and that averages are not always the truest measure of performance in all situations, the difference of 1.7 yards per play remains considerable. So despite those limitations, perhaps coaches should be calling more passes and fewer runs.
I performed my own analysis, repeating Alamar's methodology for 2005 data, and then expanding it to data from the 2002-2006 seasons. By adding 10 yds per touchdown pass and subtracting 45 yards per interception, I also calculated 5.8 yds per attempt. But when I subtracted sack yards, the expected yield for a pass attempt becomes 5.0 yards per attempt.
The passing premium now becomes 5.0 - 4.1 = 0.9 yards per play, a smaller difference than the author found.
I also calculated running yards per attempt when the same touchdown adjustment is applied. Aren't many running touchdowns truncated by the end zone too? It does seem generous to add 10 yards because some touchdown runs are goal line dives, but many are not. Some touchdown passes would not automatically yield an extra 10 yards either. Adding the 10 yard touchdown bonus makes the expected gain for running 4.5 yards per attempt.
The passing premium would now become only 5.0 - 4.5 = 0.5 yards per play.
Running in some situations, however, has value in addition to yards gained. Towards the end of a game, the leading team can use more clock time by running, denying additional opportunities for the trailing team to score. In short yardage situations, running for a short gain can be more beneficial than the chance to have a longer gain with a pass. Goal line runs can sometimes require 2 or 3 attempts before a gain of the single yard that yields the touchdown, but that single yard is worth the possibility of no gain on previous plays. (In a way, the generous 10 yard bonus for a touchdown run seems more appropriate considering the frequent stuffs on the goal line due to the high predictability of running in that situation.
All things considered, perhaps the run and pass are nearly balanced in the NFL. Balance is important because it suggests maximization. If a team runs too much, a defense can concentrate their efforts on stopping run plays, reducing the expected gain for a run but a greater expected gain for pass plays. Every team would have their own optimum balance, but over the league as a whole the optimum run-pass mix would yield about the same expected gain every play.
- Home
- run-pass balance
- The Passing Premium
The Passing Premium
By
Brian Burke
published on 1/30/2008
in
research,
run-pass balance
Subscribe to:
Post Comments (Atom)
Good stuff! One question: does the 45 yard adjustment for interceptions take post-interception returns into account? I assume it does, since if a punt nets 40 yards, that means after the return.
First, I haven't been able to get through the subscription wall to the Alamar article, so if my comments here are stupidly ignorant in light of it, please forgive.
"Every team would have their own optimum balance, but over the league as a whole the optimum run-pass mix would yield about the same expected gain every play."
I don't see why anyone would assume this. Say teams tend to run for one purpose and pass for another. The two tactics could be equally effective at achieving their purposes with a very different average yards per play.
Take the real world example (sports being escape from reality for so many of us) of investing. One can invest in a T-bill, long bond or equities, with different objectives. Nobody would ever expect the three options to produce the same average return. Except on a risk-adjusted basis, with the difference in average returns being the famous risk premium.
And that requires that in the generic, middle-case situation the high-risk strategy produce a higher average return then the low-risk strategy, in order for the choice between them to be indifferent.
If that's not intuitive, in football again consider a team that gets 3.5 yards each and every play (no risk) versus one that averages the same 3.5 yards by getting a random 0 to 7 yards on every play (riskily). The former team is unstoppable, the latter is punting all the time. The former is going to kill the latter. The same 3.5 yard average that each gets is not equivalent, risk adjusted. With higher variation the latter team needs a much higher average to make the score closer (and will never catch a true "no risk" 3.5 average team).
The main purpose of passing is to rip off a lot of yards, accepting risk of a significant loss. (Like investing in equities.) The main purpose of rushing is *not* to get a big gain but to keep a drive going, minimizing risk of loss. (Like using T-bills or bonds.)
In the generic situation, I'd be really surprised if passing *didn't* have a signifcant risk premium in average yards per play compared to rushing. It doesn't have to be explained away any more than different average returns to various investing strategies. After all, GMs and coaches are literally investing in their teams to win, using multiple strategies.
(I'd take things further to say yards per play isn't a very good measure of rushing effectiveness at all, thus explaining the '66 Packers and similar rush-heavy, low yards per carry top winning teams that otherwise seemed intentionally coached to reduce their effectiveness and increase their likelihood of being upset.)
One might simply accept the difference in average yards-per-play as the risk premium to passing over rushing. Or, following the thought example of the two 3.5 per teams above, one could have more fun testing my speculations, to make me look bad.
E.g., pick the real yardage results of say 1,000 random passing and rushing plays from real NFL games, gainers and losers (not their "average"), then string them together randomly in a whole bunch of simulations to see which is most successful at putting together say 80-yard drives, all-rushing or all-passing. Then fiddle with the play distributions of each to see what yards-per-play average for each produces equal success.
I wish I had the time to do it. Maybe I'm all wrong!
Hello again.
Looking around for an available copy of the Alamar paper I found a paper written as an answer to it: "The Passing Premium Puzzle Revisited", by Duane Rockerbie, a Canadian economist.
He says the Alamar doesn't consider the cost of "risk" due to variance in play results (which I mentioned). He adds it for both rushing and passing and starts with a result along my own line of thinking:
"First, as the variance to running increases, the optimal share of running plays decreases"
Then gets to a surprise:
"The evidence suggests that the majority of teams choose more passing plays and fewer running plays than the portfolio model predicts"
NFL teams pass too much! With data for each team for 2006.
http://people.uleth.ca/~rockerbie/Passing_premium_puzzle_revisited.pdf
Phil-Yes. Interception returns are symmetrical with respect to the line of scrimmage. Some interceptions are deep-thrown balls, so the net effect is not so bad for the offense. That is balanced by longer interception returns where the defense returns an interception past the original line of scrimmage. (Unfortuately the data is polluted slightly with Hail Mary passes.)
Here's a good link with a histogram of interception returns:
http://www.sportsquant.com/turnovers.htm
I think the concept first came from Palmer et al's "Hidden Game of Football." Football Outsiders is based primarily on that book.
Even fantasy football is somewhat balanced this way. In most leagues, every 20 yds gained is worth a point for RBs and WRs, and an interception is worth -2 points for a QB. So in effect an interception is -40 yds in fantasy.
JG-Very interesting. My own thinking is that neither the run or pass is "best" but they need each other to be successful. I think of plays as punches--the run as a jab and the pass as a cross. No one ever wins a boxing match by just jabbing, unless the boxer is lopsidedly superior.
I get what you're saying about risk, but the risk is already considered in the premium. But there are much better/more sophisticed ways to measure risk. I saw another site that proposed the Sharpe Ratio, which accounts for the variability of results. I'll have to do some more research on that.
Using your equities/bonds analogy, wouldn't it be wise to almost always pass when the game is "young", and then gradually convert to the run as you build a lead as the games "ages," ultimately using the run exclusively at "retirement?" That's sort of what teams do in reality except for the 'almost always pass part.' I know a lot about risk mitigation, but almost nothing about risk/reward considerations other than the above investing strategy. That's more of a business concept. I'll have to get smart on that.
I'll also have to check out the other article you cited. Looks interesting. Thanks!
Using your equities/bonds analogy, wouldn't it be wise to almost always pass when the game is "young", and then gradually convert to the run...
In general, sure, (assuming you are safely ahead on your investment at the end). It's better to pursue a higher-risk, higher-average-return strategy early on when you have more time to recover if the risk hits you in the head. If you invested in stocks in 1929 at the market high just before the Crash and Depression hit, and retired on that investment in 1969, you retired rich and happy. If Brady's first pass this Sunday is deflected, picked and run back for a TD, there'll still be plenty of time for him to throw for 400 yards and beat the Gints by 30. While late in the proverbial or literal game, depending on the situation, identical bad luck can be a disaster. But there's more to it than that.
I get what you're saying about risk, but the risk is already considered in the premium.
I don't see where. Rockerbie says the Alamar paper doesn't consider risk at all, except as yards lost to sacks and picks, etc., which is not the risk we (his paper fully makes my point for me, I'll ride his coattails) are talking about.
I take it for granted we agree the risk-reward relationship requires higher average return to the strategy with higher variation of result -- so, since passing has higher variance of result than running, passing must have a higher average yards per play than rushing. QED. No mystery there.
Where I think we're not connecting is how reducing passing yards per play by 45-yards per pick, subtracting sack yards, etc, to create a lower average yard per play for passing, does not fully capture the risk that requires the higher return.
The true "risk" to one using a high variance strategy (equities, passing) is not that one will have bad events sprinkled into one's results, reducing them in the short run while getting more in the long run, compared to a low variance strategy. The real risk is that several bad results will occur in short order and kill you. There is no later, no long run, because you are dead.
Passing (like equities) provides a higher return in the long run -- but the long run in football is longer than a game. It's a season. In one game a QB as good as Peyton Manning can throw six picks, and a team that skillfully runs the run-and-shoot offense all season to dominate all the league's passing stats can blow a 32-point lead in the second half of a playoff game to lose. That's "high variance" in action, in the small sample size of a game. Averages don't capture this. They omit the risk of multiple bad events occuring in short order to cause game ending calamity.
In investing, the goal is not highest average return, but highest return relative to risk taken to produce the best final result. Diversifying a portfolio partially out of stocks can improve expected return by eliminating risk of calamity in the game period that lasts to one's investment horizon. The Rockerbie paper's point is that the same is true in NFL football games, with numbers and data to show it.
My own thinking is that neither the run or pass is "best" but they need each other to be successful. I think of plays as punches--the run as a jab and the pass as a cross.
Very true, but there's more. A low-variance running game can have the same scoring capability as a high-variance passing game with a significantly higher yards-per-play (as my 3.5 yard per play teams illustrate), *and* diversifying into it can reduce risk of short-term calamity, to increase expected return by the end of the game, in terms of W-L, if not yards.
Words may fail to convey the point of "portfolio analysis" for football play selection. Math and data are better. I just refer you to the Rockerbie paper:
"The results suggest that most NFL teams pass the ball too often and that a running premium exists."
'Nuff said by me. Way too much, in fact, probably. This is just something for your consideration.
JG-Thanks again for the reference. I'm going over the 'revisited' paper now. It's really interesting, but the math is dense, so I need to get smarter on portfolio theory.
This is right in line with the Sharpe ratio examination I mentioned earlier. One possible flaw is that most of these methods consider upside volatility as well as downside. I.e. that some passes yield +40 yards instead of just +20 shouldn't count against the pass. But there are other methods that account for upside/downside differences. I think portfolio theory has a lot of potential in the run/pass balance issue.
I am very happy to see this healthy debate on an issue that I find so interesting. JQAS will be publishing Prof Rockerbie's paper in April and I am preparing an analysis of that paper in response.
As one of you mentioned difficulty in getting at JQAS papers, you should be able to get any paper by registering which is free. If you have trouble with getting access to a paper, please let me know.
Ben Alamar