website statistics

The relative merit of passing and running has been an ongoing debate for generations. Early attempts to quantify the risks and rewards of the two general tactics did what they could using yards as a measure of effectiveness, but they could never fully account for the all the situational factors and outcome variables. How to account for down, distance, field position, sacks, turnovers, penalties, first down conversions, and so on made the problem too complex. And that’s before we even start to think about score and time.

Since 2008, EPA and WPA has allowed us to account for all the primary situational factors while boiling all the outcomes into a single measure. We can also measure variance, an important consideration in game strategy. This analysis produced a complex result in which it appears offenses run too often in most situations but pass too often in others.

But those models can only answer questions within the box of what teams already do. They can’t consider what would happen when a team operates outside the boundaries of what has already happened in the past, at least not directly. For example, we could never estimate how often a hypothetical team would win by always passing and never running, which is probably something a team like the 2014 San Diego Chargers should at least consider.

The Chargers

Going into week 9 the Chargers average 0.33 EPA per pass (3rd in the league) but -0.09 EPA per run (29th). That’s an enormous difference—roughly a half point per play. In effect, every time the Chargers chose to run rather than pass they cost themselves nearly half a point. For twenty carries in a game, that’s a disadvantage of just over 8 points.

Those numbers may be unrepresentative of the Chargers’ true average expected production due to sample error or uncaptured situational variables. However, although the Chargers have only played 8 games this season, those EPA averages are from several hundred plays, so perhaps they are more representative than we might think. But even if we say those numbers are extreme, a regressed 4 or 5 point disadvantage would still be very large. That’s the equivalent of forfeiting over a 10% chance of winning a game.

So what would happen if a team were to never run the ball?

I created a version of the WOPR game simulator to find out. Normally the sim pitches two perfectly equal, league-average teams against each other. Both teams share identical, highly conventional tactics and strategies. But for this excursion, I programmed one team to always pass. How often that team wins, compared to its normal 50% win rate, might tell us the value of only passing.

If only it were that simple. The sim can tell the offense to always pass, but it can’t tell the defense to know to always expect a pass. Passing outcomes can be very different when a defense knows what’s coming. However, we can use some cleverness to trick the sim into telling the defense to expect a pass. We can do this by telling it that it’s always 3rd and long for the purposes of generating play results, regardless of the actual down-distance situation. The play outcomes in the simulation would be drawn from the distribution of 3rd and long plays rather than the actual down-distance distribution. There would be many more sacks, strip-sacks, interceptions, and fewer completions than if defenses had to respect the threat of the run. The only exception to this rule would be on the goal line, when there are no 3rd and long outcomes to draw upon. The sim would still pass, but it would accurately know the distance to gain.

I think this may penalize our pass-happy offense too much. After all, many situations would be less severe than 3rd and long, allowing them to make better risk-reward trade-offs. On the other hand, the real-world 3rd and longs still have at least a trace of probability of a run which defenses would need to respect. Additionally, 3rd and longs can result in relatively easy short yardage check down completions that end short of the line to gain.

But on net, I’d bet a coach would say that a team transparently acting as if were 3rd and long on every play would have almost no chance of winning. That’s all they ever seem to talk about before games—take away the run early, force them to be one-dimensional, make them to beat us with [quarterback X]’s arm, and so on.

This essentially turns the game into something more like the way people play flag or touch football. No one tries to inch their way toward a first down, attempting to set up ‘manageable’ third downs. They see a series as three tries to get all 10 yards (or past the tree or mailbox or whatever) on a single play. The best distance to go on 3rd down isn’t 2 yards or 1 yard or even inches to go. It’s no yards to go, because you already converted on 1st or 2nd down.

Results

So how does our pass-happy team do? With a typical run/pass balance, they win a perfectly even 50% of the time. The pass-only strategy wins 54.0% of the time, turning a 50/50 contest into a 54/46 contest.

What if we allowed some wise exceptions to our pass-only rule? If we allow them to run only inside the opponent’s 10 yard line, they won 54.8% of the time. I bet if we could add some other common sense exceptions the pass-happy team would win even more often.

As you might expect, the effect is even stronger if our pass-happy team finds itself behind its conventional opponent. Starting the second quarter down by a TD, a conventional team wins 29.9% of the time while our pass happy team wins 34.7% of the time. At first that appears to be roughly the same effect size as for a game that begins tied, but remember a quarter of the game has already passed.

[Note: The sim was run 100,000 times for each variation, yielding a 95% confidence interval of 0.003.]

In case you think this is just a marginal improvement in win probability, consider that the odds go from 1:1 for the conventional run/pass balance to better than 1.2:1 for the passing strategy. This is with the same exact team—same players, same talent levels, just a different mindset. I'm not claiming a typical team would win exactly 54.3452345% of the time, but I would submit that a typical team would win significantly more often if they reserved most of their runs for particular situations.

But San Diego is not a typical team. Because they are a much better passing team than running team, their benefit of forgoing the run entirely might be considerably better than 54/46. Our earlier back-of-the-envelope analysis conservatively estimated their running costs them 10% win probability per game, so a pass-only policy might buy that 10% back while adding more than the 4% benefit a typical team would enjoy. 

Leave your comments

Post comment as a guest

0

Comments (10)

  • Guest - Johnny Hirschauer

    Why not utilize WPA rather than EPA? EPA ignores the time benefit of running (from what I understand) while WPA would allow you to see that element more clearly. In addition, I reason to venture that you cannot boil the effect of full knowledge of pass into the numbers by simply substituting in third and long situations. Part of what makes the pass so successful is exploiting the resources the defense puts in the game to stop the run- namely linebackers, and obviously the play action which you have proven assists in the pass game. I just think the run, even in a passing league, has a place.

  • I use both. WPA is very sensitive to context and can be quite noisy. EPA is more stable and more predictive of future performance.

  • Guest - Jay Stokes

    As a Charger fan, I have wondered why they try to "establish the run" in some games, especially since they seem to use play action so infrequently. Why not "establish the pass", making your runs, when you do them, more effective and surprising?

    Also, so many of the short slants are quasi-runs. The risk of interception or sack are very low, so really the run vs. pass nomenclature is probably lower variance, lower return plays vs. higher variance, higher return plays. You can do both in the passing game. SD has a great QB and a decent receiving corps and an OL that can pass block OK. They have a decimated RB corps and an OL that can run block poorly. Seems like they should not really be trying to run.

    Instead I would love to see SD use the run highly selectively, out of pass formations and use a higher mix of the lower variance, lower return pass plays.

  • Guest - Tom

    @Jay Stokes - as far as "establishing the pass" to set up the run, I completely agree, and I think this is why the Saints have such a great run game. They're #2 in yards per attempt and #3 in EP/play. But in attempts per game, they rank #23 (#1 being the most attempts per game, which, of course, is Dallas). They pass, pass, pass and then let Ingram loose for a 13-yard pickup. It's not perfect, but when it works (like against Green Bay the other night), it's really cool to watch.

  • Guest - Michael choe

    Reich and McCoy have been hurting the team by trying to establish the run.

  • Guest - SlackerInc

    This is so interesting. I do feel that (unlike going for it on fourth down) this is such uncharted territory that it is pretty hard to estimate, even with the clever tricks you used here. I also suspect it would be more effective at first and then defensive coordinators would come up with some ways to blunt its effectiveness somewhat (what precisely, I don't know; but that's why they get the big bucks).

    I do think though that it's absolutely worth a try. I would love to see it happen!

  • Guest - Robert Giambatista

    I'm not a game theory expert, but unless I misunderstand your analysis, it does not seem to take into account how radically defensive strategy would be altered by a pass-only offense. Under such a scheme, passing efficiency would be lower than it would by having an optimal mix that features enough of a run threat to force some variability into defensive game planning. Because of this, the inputs used for your analysis may perhaps be biased.
    A very simple 2*2 game theory matrix with expected yards per run and pass vs. defensive expectations of run and pass will generate what are called "oddments" - the optimum ratio of runs to passes (and of defensive run/pass expectation). Just playing with some reasonable numbers in such a matrix by adding/subtracting a yard or two per play as a function of crossing the defense, it can be shown that it is not generally plausible for strategy to be optimized with an all-pass offense. Further, such a simplified matrix doesn't take into account variability - pass play results are far more variable than run play results, in a way that generally (but not always) favors the run. Pass play variability is relatively desirable to an offense only when a team is well behind or running generates a very low average per attempt.
    It is probable that the pass/run mix we see in the NFL is reasonably close to an "efficient market", though clearly varying by teams as a function of skills.

  • Guest - Brian Burke

    Robert- You are correct. You do misunderstand the analysis.

  • Guest - intrepidrook

    Great post. I always thought an interesting idea would be for a team to completely abandon the run game until the goal line. This team would have to have a better than average quarterback and 3 great receivers every down or two great WR and a dominant TE. Also the RB should be quick and great hands for catching. Do away with the FB and extra RBs to save cash for use on the WR. Defensively teams would have to decide if its worth investing in talented CBs for one team or take their chances with an ordinary defense. I don't think pass heavy team would win the SB, but posting winning records year after year wouldn't be far fetched.

  • Guest - Josh

    I think the Patriots put up some valuable data points in the second half of their playoff game against the Ravens. All pass and no run makes Brady an interesting boy.

    I think that an almost-all-passing offense would be so much fun to watch and you could probably construct it in such a way as to make the sparse runs even more effective. Opposing teams would know your extreme passing tendencies and would likely have their 'dime' defense in as their base defense for most of the game. If you ran a no-huddle, no-substitution offense from a 12 man package (one RB, two TE) you would have two likely mis-matches (the TEs against CB or S) on your frequent passing plays, plus a blocking advantage when you switch over to run plays around the goal-line (TEs against CB/S). With the two TEs and one RB you'd also have various pass-protection options against any blitz packages the defense might try to bring.

    Off the top of my head, the Pats and the Chargers are the two teams with the personnel capable of this type of play. Gronk and Wright are both capable receiving TEs as well as adaquate run-blocking/pass-pro TEs (Wright is obviously the better blocker, Gronk the better receiver) and Vereen is a good pass-catcher at RB and also capable in picking up blitzes. Sand Diego also has two good receiving TEs (I have no idea how good they are at blocking) as well as good receiving/blocking RBs. Both teams also have excellent QBs who are perfectly capable of handling large passing workloads while running no-huddle offenses.

    Really interesting stuff. As always, thank you very much for all the great work you all do on the site. We are all of us forever in your debt!

    Cheers,
    J