Deep vs. Short Passes

Over the past couple years, we've learned that passing well is  more important than running well in terms of winning games. We've learned that passing has become more and more lucrative over the years. And we've learned that offenses should pass more often, particularly outside the red zone and on 1st down. But 'passing' is a large category, encompassing everything from a screen pass 2 yards behind the line of scrimmage to a 50 yard bomb. 

Beginning in 2006, the NFL classified every pass attempt as either 'short' or 'deep,' where deep means anything past 15 yards. About 19% of pass attempts are classified as deep. Unfortunately, that's all we get, so we can't tell a screen from a 14-yard down-field pass attempt. Still, it allows us to begin to pull apart different types of passes and examine them in one more layer of detail.

In normal football situations, in which the clock is not yet a factor and the score is relatively close, pass plays, including sacks, yield an average of +0.08 Expected Points Added (EPA), while run plays yield an average of +0.01 EPA. Which type of pass is more responsible for that advantage, risky deep passes or safer short passes?

Let's compare the value of short and deep pass attempts, limited to normal situations (1st and 3rd quarters, the score within 10 points) and outside the 15-yard line. (It's difficult to throw a deep pass inside the 15.)


Pass DepthAvg EPA Success Rate
Short

-0.0147%
Deep+0.4543%

Deep pass attempts average +0.45 EPA, and short pass attempts average -0.01 EPA. However, short pass attempts are more likely to be 'successful,' defined as more likely to result in a positive EPA. Short passes in the sample are successful on 47% of attempts, and long passes are successful on 43% of attempts.

It's important to note some serious limitations of a direct comparison between deep and short passes. First, any pass attempt means that several things have already gone well. The pass protection held up long enough to allow a throw, and a receiver and a passing lane was judged to be open. Deeper pass attempts naturally require more time to develop, so the pass protection must last longer.

Deeper pass attempts have a higher risk of interception. Deep attempts are intercepted 6.4% of the time, compared to only 1.8% for short attempts. Deep attempts certainly are riskier, but the risk of interception is already baked into the respective EPA values of deep and short attempts.

It's the additional risk of a sack that makes the comparison tricky. About 6% of pass plays result in sacks. Unfortunately, there is no way to tell whether a play that resulted in a sack was drawn up as a deep pass play or not. We can't directly measure the increased risks of deep attempts due to sacks, but we can attack the question from a different direction.

In the data sample (normal situations from 2006 through 2009), there were 4,609 deep attempts worth a total of +2,084 EPA. There were also 1,464 sacks (including sack-fumbles) costing a total of -1,985 EPA. So even if every single sack were due to a play drawn up with a deep route as the primary target and we count it against the deep category, the deep attempt still comes out as a net positive. 

The deep pass is probably an underused tactic by current NFL offenses, but this does not mean that every team should immediately start heaving the ball down the field. Obviously, given the choice between a deep and short pass, teams would prefer deeper passes. The problem is that not all teams have that option, either due to pass protection or receiver abilities. There is a large bias in the results above, because the offenses that do throw deep often are likely to be the offenses that throw deep well. Additionally, many short passes are 'check-downs' to a safety-valve receiver, typically a back, when the deeper routes are covered. This is a murky topic without clear answers.

Still, I think there is something to be learned here. All things considered, short pass attempts yield a net negative average EPA. Run plays average a better EPA than short attempts. It's not simply that passing has generally superior payoffs than running. It's that the entire advantage of modern passing comes from deep passes. Teams should probably be passing deep more often, and running and passing short less often.

I suspect the culprit isn't the 8- to 15-yard 'short' passes, it's the really short stuff. I wonder if all those quick-out WR screens are just wastes of downs. I realize they are often intended as 'constraint' plays designed to keep the defense honest. But they rarely seem to be very successful. We'll need a lot more detailed data to really understand what types of pass attempts are most and least productive.

Sure, every team would like to throw down-field more often, but only some are able. The lesson for GMs and coaches is: become one of those teams. 'Easier said than done' would be an understatement, but it's well past the time when teams should be building predominantly around the run or thinking that high percentage short passes can replace a real down-field threat. Teams should invest in draft picks and free-agents who can consistently get large chunks of yards, and invest in the practice time needed to perfect deep routes. And if you already are one of those teams who can throw deep, you should probably do it more often.

You can try to march 70 yards down the field-- 3, 5, or 7 yards at a time--allowing the defense ample opportunities to force a punt, all the while hoping not to commit a turnover. Or, you can go for large chunks of yards 15, 20, or more at a time. Sure, it's riskier on any single given play, but the numbers suggest that's where the advantage is.

  • Spread The Love
  • Digg This Post
  • Tweet This Post
  • Stumble This Post
  • Submit This Post To Delicious
  • Submit This Post To Reddit
  • Submit This Post To Mixx

36 Responses to “Deep vs. Short Passes”

  1. JMM says:

    First impressions:

    Vindication for Al Davis!?!!

    Are 5 yards throws + 11 yards run after the catch short or long?

    Also vindication for Dick LeBeau who's philosophy is to stop the run and eliminate big plays.

    Why do the Pats win (specifically over the Steelers?) The Pats O vs the Steelers D is the counter example. I guess great execution overcomes "the numbers."

  2. Sander says:

    This year the NFL has started tracking what it calls 'Pass Yards' and 'Yards after Catch', and they've started to categorize 'short' and 'deep' by Pass Yards, and not Total Yards.

    That muddles this sample too, if you're using data from this year.

  3. Adam Davis says:

    Actually, I think that the NFL's new system of short/deep categorization finally starts to *clear* the waters. On top of all of the mitigating factors listed by Brian above, I believe that the NFL's previous play-by-play data only showed "deep" passes based on the total yards gained. This means that if I threw a check-down pass to the RB, 2 yards forward in the air, and he broke it for a 30-yard gain, it was classified as a "deep" pass.

  4. Brian Burke says:

    In their play-by-play system, the NFL classifies deep and short based on 'air' yards, not total yards. From last night: "(9:35) 7-M.Vick pass short left to 25-L.McCoy to HST 29 for 40 yards (30-J.Allen)."

  5. Anonymous says:

    Brian,
    one thing you could do to get some sense of the bias involved by passing teams passing more is to simply count the instances for each team-year. Plot the number of pass attempts sorted from highest to lowest.
    I can't begin to tell you how to quantify that graph but its a starting point..
    Andy

  6. Adam Davis says:

    "In their play-by-play system, the NFL classifies deep and short based on 'air' yards, not total yards."

    Right - but is that a new convention, or have they always been classify them that way?

  7. Sunpar says:

    It does justify certain things that NFL teams already do that currently are used as evidence of inefficient thinking-- namely the use of 40 yard dash times and evaluations of QB arm strength.

    Every year around combine time you'll see many reporters auto-complete the same stories, citing historical examples of WRs who were not perceived to be fast enough (Jerry Rice) and QBs who were perceived to not have the required arm strength (Tom Brady) and slipped through the cracks.

    Now we have a plausible reason why looking at these attributes may be efficient after all (even if they don't tell the whole story, and can still result in missing some amateur talent).

  8. Joseph says:

    Brian, I know that you're not a (big) fan of Football Outsiders, but for a while (at least since their 2007 Prospectus, but it appears they have these numbers for 2005 & 2006), they have been classifying passes as "Short", "Middle", "Deep," and "Bomb". As far as I know, this is based upon "Air Yards." Now, this is only the percentage of passes thrown--but I bet if you asked nice, they might could give you an exact breakdown in a database of some sort.
    Another idea--breaking these numbers down by QB, team offense, and esp. TEAM DEFENSE. Fans in ATL, NO, & CAR are saddened by the release of Sabby Piscatelli, and I'll bet that there are other teams who are similarly bad at defending deep/bomb passes. In other words, I am betting that the positive EPA is more likely due to BAD DEFENSES than the Colts, Saints, and other teams who throw deep frequently and effectively.

  9. ChrisZ says:

    Maybe it's like the Blitzkrieg strategy. Deep passes are like the big concentration of force, but the big concentration of force doesn't break through unless you have the holding fire all along the line to keep the defenders occupied.

    Similarly, the deep passes are where you get your points, but they only work if you have the basically neutral holding fire of runs and short passes to otherwise keep the defense occupied.

    There, I've successfully Godwined a discussion on NFL stats.

  10. Anonymous says:

    Im having trouble getting the EPAs to add up:
    assuming EPA/Pass play is .08
    2084 EPA for long pass attempts
    -1985 EPA for sacks assumed to happen with long pass attempts
    this comes to .0163 EPA/play for long pass attempts and -.01 EPA/play for short pass attempts. I dont know the ratios of long to short but there is no way that can ever add to .08, right?
    Also, was this all on first down play or all plays?
    Thanks!
    Andy

  11. Jeff Clarke says:

    I have a little bit of Andy's concern. I guess I might need to look at the EPA article closer but I thought that one of the basic tenets was that EPAs for all plays averaged to zero. This makes sense because it is in essence saying that if both teams play completely average games from here on out, neither team will outscore the other.

    If pass plays have an average of +0.08 and running plays have an average of +0.01, they can't all average to zero. It sort of sounds like saying the "average play has an above average return".

    I feel like there is something obvious I'm missing here...

  12. Brian Burke says:

    Net pass EPA will add up to be positive and run EPA is net negative. There are also plays that are classified as neither run nor pass, mostly 'aborted' plays, which are typically negative.

  13. Anonymous says:

    Actually, I think the reason that the run and pass EPA don't add to zero is simply because of whats not included, aborted plays (penalties) and fumbles. Those are huge negative EPA plays and, when all is said and done, should bring everything to zero.
    I am more wondering about is specifically the passing .08 number; if our data set has all plays that turned out to be passes included, shouldn't we be able to add all the EPAs for those plays up and get back to .08?
    Thats why i was wondering if maybe this is for all downs instead of just first down (because i think .08 coincides with the 1st down passing number you came up with a while ago).
    Thanks again!
    Andy

  14. Chris says:

    Hey Brian, did you include plays with PI penalties to your analysis? If not, that would further skew the advantage of going deep.

    Great study by the way.

  15. Brian Burke says:

    Yes, this includes all downs.

    Chris-Yes, penalties are included in the analysis.

  16. John Morgan says:

    Have you looked into whether success on deep passing plays increases the rate/extent of success of run plays and short passing plays?

    Deep passes often force "off" coverage and safeties out of the box. That might mean that not only are deep passes more effective, but successful deep passes improve the offense overall.

  17. JJB says:

    Speaking of PI penalties.. I recall a quick and dirty study of drives on which defensive PI penalties occurred, which found that drives on which a PI penalty occurs are MUCH more likely to end in a score, and disproportionately, in a touchdown. IIRC the avg NFL drive results in a score just under 40% of the time and in a TD about 20% of the time. Throw in the defensive PI penalty and those numbers double (actually the TD likelihood goes to just under 50%, more than double).
    I guess my question is, if you factor OUT the defensive PI penalties, how much EPA does the "long pass" strategy lose ?

  18. Dean Jackson says:

    Mike Wallace as Steelers MVP comes to mind on this one.

  19. Vince says:

    Could you look separately by down? I'd guess that short passes on 1st and 2nd down have positive EPA; it's just the 3rd down short passes that have negative EPA because of all the 3rd & long dumpoffs.

  20. Brian Burke says:

    That makes sense. I'll do that.

  21. Jeff Clarke says:

    The aborted plays makes sense. Thanks for the explanation.

  22. Guy says:

    Brian: If you assume the sacks all count against deep attempts, then 95% of the positive value of deep attempts is gone. Yes, it's still a net positive, but a pretty small advantage. Why do you feel so confident that long attempts have a much higher payoff?

    In fact, it's easy to imagine that the remaining advantage is explained by plays where the intent was to go long but the QB didn't see an opportunity and so settled for a short attempt (or short run) instead. It seems possible, maybe even likely, that if we knew the intent of every play before the snap, long attempts would have no advantage at all. On the flip side, some short attempts are probably a response to a blitz when the intent was a long attempt.

  23. Anonymous says:

    Why is the long/short pass divide at 15 yards? Would it make more sense to group the quick-out WR screens with runs instead of passes? To me, defensing a 5 yd pass would have more in common with defensing a run that gets past the line of scrimmage than defensing against a 10 yd pass. Based on your comments, I'd like to see WPA/EPA that categorizes a play as in front of or behind the linebackers, regardless of whether it is a pass or run.
    I would also surmise that a defense's success against the long pass is more an element of random luck (as long as the QB has time to throw)than success against the run or short pass. There is simply too much field to defend.

  24. Brian Burke says:

    Certainly short pass plays have much greater than zero percent of the burden of sack risk. Whatever that burden is, it subtracts utility from the short category and adds utility to the long category.

  25. Guy says:

    Fair enough: maybe 20% or more of sacks were short plays. But against that you likely have a huge number of intended long passes where the QB couldn't find a man open and threw short instead. And what about the many passes where the QB, unable to find an open receiver, deliberately throws it away? Won't almost all of those be recorded as a short attempt?

    The fact that the completion% is basically the same tells the story: it can't possibly be true that completing long attempts is as easy as completing short attempts. So many, many intended long plays are clearly not visible to you here. (And given the number of plays where the QB probably has both a long and short option, maybe this isn't even possible to analyze in these terms.) I'm not sure this data is telling us anything more than "throwing for long yardage is better than throwing for short yardage."

  26. Brian Burke says:

    I don't disagree. However, the result could have been the reverse, but wasn't. The data could say that deep passing was less efficient and the added complications of deep attempts would probably give them net negative value.

    Also, the data show that short passing is not much more effective than running. The main conclusion of the article is the finding that the known EP advantage of passing over running is from the deep attempts, not all passes. That's substantially more than "throwing for long yardage is better than throwing for short yardage.

    Additionally, this analysis includes penalties, such as holding or PI on both sides. It also accounts for down/dist/yl situations. I think one of the most underused tactic on 3rd and long (or very long) is to throw it up for grabs and let your WR make a play for it. A very deep int is not much worse than an incomplete pass, a sack, or a checkdown that has little chance of success. Plus, there is the opportunity for a PI call.

  27. Anonymous says:

    I'm curious to see what happens if third downs are removed. Like a poster above said I imagine a lot of negative EPA for short passes comes from dumpoffs on third down.

    But maybe EPA is also proportionately negative for long passes that occur on third down. It would be interesting to know.

    Also I think the success rate of long passing would take a fairly big hit if the majority of the sacks were attributed to long passing.

    I agree that throwing it up deep on 3rd and longs might be slightly underutilized but the defense is also defending against this fairly often.

  28. Guy says:

    "I think one of the most underused tactic on 3rd and long (or very long) is to throw it up for grabs and let your WR make a play for it."

    Really interesting idea. We're conditioned to think "interception = bad," but in some circumstances not that different than an incomplete.

  29. Anonymous says:

    Brian,

    why don´t you invest 200 $ to buy the play by play charts from FO? They started this game charting in 2006 i think. They have all passes with length of throw and YAC.

    BTW, i think deep passing sacks can be calculated to a certain point. If you look at stats when teams are behind in the 2nd Half, they throw more deep because they are sacked way more often. Or you look at 3rd and Longs, where sacks are also more up.

    Greetings from Germany, Karl.

  30. TheGreatFatMan says:

    I wonder if theres a way to control for throws on plays that are meant to be deep throws, but are covered and then the QB has to get out of the tackle box and throw it away to avoid the intentional grounding, which should show up as an short incomplete pass. Doubt it would have too much of an effect, but the current system essentially takes that from an unsuccessful deep pass to an unsuccessful short pass, making the deep play look better than it really should be while making short plays look worse. Again, this doesn't happen enough to really explain the discrepancy, but if this happens a few times a game, could explain why it may be logical for teams to use the mix they currently do, especially since that should improve the SR of short passes relative to long.

  31. Brian Burke says:

    There may be a way. On a pass play, the intended target receiver is listed in the play-by-play. If there is no target, it's generally a thrown-away ball, or sometimes a tipped pass. The 'untargeted' passes could be excluded.

  32. Bigmouth says:

    I really think the sack data -- and attendant injury risk -- is a big question mark in this analysis. There's a reason Mike Martz's offense, with its ubiquitous seven-step drops and long routes, has fallen out of favor. It's hell on the quarterback.

  33. Tarr says:

    I know I'm late to the party here, but a couple comments:

    1) Frankly, I don't think the analysis here tells us anything meaningful at all about whether teams should pass deep more often or not. As you mention, "if a team is passing deep, several things have already gone right". You need pass protection to hold up, and someone has to at least seem to be open downfield. Many short passing plays are simply check downs that result from aborted attempts to pass deep. These factors will heavily bias the result unless we are able to control for them. Including sacks in the deep pass data is far from sufficient, in my opinion.

    2) Is there anywhere where QB's "Air Yards" are compiled? The only place I can even find YAC data for receivers in a public spot is on Yahoo sports, and that's only active players.

    Since all the evidence we have suggests that the large majority of the credit for YAC should go to the receiver, not the QB, it seems like we should be able to use this as a basis for dividing EPA/WPA shares between the QB and the receiver. (Sorry, O-line, you're still out of luck.)

  34. Bigmouth says:

    Tarr, I'm unfamiliar with that evidence re YAC. Can you (or anyone) point me in the right direction? Thanks.

  35. Tarr says:

    Search "Air Yards" on this very blog, and you get lots of material on this subject. Here's a good post on this:

    http://www.advancednflstats.com/2007/08/who-gets-credit-for-yac.html

  36. Anonymous says:

    There are other factors not accounted for and the term "deep pass" is a broad one. I believe that fades, 9 routes and deep corner routes are "safer" than seam (skiiny posts) and posts. The deep side line routes I mentioned at the outset of this post are safe routes in that, provided the proficiency of the passer (and we would certainly hope that most NFL starters are proficient) they can use the sidelines as oppossed to throwing deep or even intermediate middle where there is a lot of clutter

Leave a Reply

Note: Only a member of this blog may post a comment.