Belichick's Belichick

That's what author David Halberstam called Ernie Adams. A longtime friend of the Patriots head coach, Adams is Belichick's stat guru in the shadows. A great article from ESPN Magazine spotlights one of the most enigmatic men in pro football. Adams, who is described as a brilliant but quiet analyst, could be the real brains behind the Patriots' dynasty. The spirit of the piece is captured by Patriots tackle Matt Light, "I'm not sure what Ernie does, but I'm sure whatever it is, he's good at it."

A similar article on Adams from the Boston Globe here.

The Passing Paradox Part 3

This is a continuation of an analysis of run/pass balance in the NFL. In part 1 of this article, I discussed the potential application of financial portfolio theory in football strategy. In part 2, I critiqued a recent study that make a great stride toward applying economic and financial math to football. Here in the final part of this article, I present an alternative way of understanding risk and reward in the run/pass balance question.

To recap, two research papers came to opposite conclusions about the run/pass balance in the NFL. The Alamar paper "The Passing Premium" found that the expected gain for a pass is higher than the expected gain for a run, accounting for interceptions. He concluded that teams should pass more often. But the Rockerbie paper "Passing Premium Revisited" found the opposite, that teams pass too much. He applied an economic utility equation that accounts for risk and concluded that running more often helps teams win.

10 Yards for a First Down

Both papers simplify football into a yardage optimization game. Unlike financial investing where the goal is to maximize total return for certain acceptable levels of risk, football requires a minimum gain every 4 downs to maintain possession. At the end of each year no one takes most of your money away if your mutual funds don't earn at least 10%. If they did, and you hadn't made your 10% by November, your risk tolerance would dramatically increase for the final 2 months of the year.

And I think that's how we should model football. Every down and distance situation requires its own risk equation. On first and second down, teams can chose a balanced run/pass attack. But on 3rd down, risk tolerance needs to increase. The net effect would be to bias the offense towards the pass. Although not mathematically optimum in terms of total yardage gain, passing may be optimum when considering the added risk of having to punt.

Take a situation such as 3rd down and 5 yards to go. The table below is the cumulative distribution of yardage gained by running and passing. It lists the cumulative percent of each play that results in at least x yards gained. For example, a run play yields 5 yards or more 24.4% of the time, and a pass play yields 5 yards or more 45.7% of the time.

Yards Gained	Running	Passing
<0	87.1	93.2
0	79.4	59.2
1	69.2	57.6
2	54.6	55.8
3	41.9	53.5
4	31.9	49.9
5	24.4	45.7
6	19.2	41.2
7	15.3	37.1
8	12.2	33.3
9	9.0	28.7
10	7.9	27.3

So given the distribution above for a situation such as 3rd and 5, which type of play should be called? The chance of converting a first down by calling a pass is almost double than that for calling a run. The run is the better choice only in situations requiring gains less than 2 yards.

A coach can call plays with pure yardage optimization balance in mind until third down, commonly considered the do-or-die, make-or-break down. Then, he has to consider the risk of being forced to punt. The coach's decision is reduced to that single play and not an overall strategy. Because most 3rd down situations require more than 2 yards, the run/pass balance is biased toward the pass.

This is why play selection is a paradox. The worse an offense is at passing, the more often it needs to pass, and the higher its risk tolerance needs to be. Incomplete passes on either 1st or 2nd down typically lead to 3rd and long situations, requiring a pass. Teams with poor passing games would also tend to be behind towards the end of a game, which requires even more passing. Teams that don't pass well are therefore forced to play to their weakness. Thus, the passing paradox.

The inverse is also true. The better a team is at passing, the less often they need to do it. They would find themselves ahead in most games, allowing for a lower risk tolerance. Burning time off the clock by running the ball would be to their advantage.

Risk Aversion and Tolerance

In the "Revisited" paper, the author guessed at a perfect risk aversion coefficient for the NFL as a whole. He used the risk aversion (α) for the Chargers, because they had the best record in the year studied. Then he calculated what each teams' run/pass ratio should be based on that league-wide perfect α.

I explained the reasons why this was a bad idea in my last post, notably that poor teams (that tend to be behind) must increase their risk tolerance if they hope to overcome a significant deficit in a game, particularly towards the end of the game. Further, the analysis in the "Revisited" paper failed to recognize that it's winning that often leads to running, rather than the other way around.

So instead of choosing a perfect α based on a single team, then apply it to the entire NFL, why not calculate what each team's actual α was based on their actual run/pass balance? If I'm right about how winning leads to running, teams with a lot of wins should have a risk-averse portfolio, and teams with a lot of losses should have a risk-tolerant portfolio.

So that's what I did. The equation below solves for risk aversion in the maximized utility equation, instead of run/pass ratio as the author of "Revisited" did.

where:
α = risk aversion (negative values are risk tolerant, zero is neutral)
γ = % of plays that are runs
μ_R = mean (expected) gain of runs
μ_P = mean (expected) gain of passes
σ_R = standard deviation of run gains
σ_P = standard deviation of pass gains

The table below lists each team's running and passing stats (borrowed from "Revisited"), their actual play selection balance, their number of wins, and their calculated risk level according to the equation above. Keep in mind that positive α means risk aversion and negative α indicates risk tolerance. The list is sorted from most risk averse (conservative) offenses at top to the most risk tolerant (aggressive) at bottom. Click on the table headers to sort as desired.

Team	R Avg (μR)	P Avg (μP)	R SD (σR)	P SD (σP)	Actual (γ)	Wins	Risk (α)
NE	4.0	6.0	8.2	11.6	0.46	12	0.039
NO	3.1	7.3	9.0	15.7	0.39	10	0.030
SD	5.7	6.5	9.4	11.5	0.49	14	0.029
IND	4.5	6.6	5.9	12.4	0.45	12	0.026
BAL	3.8	5.2	6.2	12.2	0.45	13	0.020
DAL	4.5	6.3	5.8	13.7	0.49	9	0.019
CIN	4.1	6.5	6.0	15.0	0.43	8	0.019
STL	4.1	5.5	7.1	12.0	0.38	8	0.018
KC	4.5	5.6	7.0	13.5	0.52	9	0.016
NYJ	3.6	4.9	5.9	13.8	0.50	10	0.016
ARI	2.7	4.1	6.5	14.2	0.38	5	0.013
MIA	3.6	4.3	8.9	12.0	0.37	6	0.011
HOU	3.6	4.1	7.7	10.9	0.40	6	0.011
WAS	4.8	5.5	6.2	12.8	0.49	5	0.011
PHI	4.9	6.1	9.5	15.6	0.39	10	0.011
DET	3.9	4.8	9.1	13.0	0.32	3	0.009
JAX	4.8	5.2	6.9	11.9	0.51	8	0.009
MIN	3.9	4.4	8.9	12.7	0.42	6	0.008
CHI	3.9	4.8	5.3	15.1	0.48	13	0.008
CLE	2.7	3.4	8.0	13.8	0.40	4	0.008
CAR	4.2	4.7	7.6	13.1	0.42	8	0.007
GB	4.2	4.8	8.1	13.9	0.38	8	0.006
BUF	3.8	4.2	6.0	15.1	0.46	7	0.003
NYG	4.9	5.0	8.0	12.3	0.45	8	0.002
SEA	3.8	4.0	6.7	13.9	0.43	9	0.002
SF	4.4	4.5	10.9	14.2	0.48	7	0.002
TB	3.6	3.6	7.7	11.9	0.39	4	0.000
DEN	4.8	4.7	8.4	13.5	0.44	9	-0.001
PIT	4.7	3.6	9.5	14.7	0.36	8	-0.011
TEN	4.3	3.7	8.6	12.7	0.48	8	-0.013
OAK	3.6	2.5	8.4	13.7	0.44	2	-0.016
ATL	5.7	4.2	10.5	12.4	0.54	7	-0.438

Notice that most teams are very close to neutral risk (α = 0) but with one very large exception. Michael Vick's Falcons appear to be the biggest risk takers by far, with an eye-popping α = -0.438. But I think that result is due to the unique nature of Vick's offense. His runs were very boom and bust, with either a big gain or deep sack. Those were often called pass plays in which Vick scrambled. Plus, his running ability on the outside often opened up running holes for conventional run plays on the inside.

Most other teams, however, tilted slightly positive, meaning they were slightly risk averse. The teams with the most wins tended to be the teams that were most risk averse. Teams such as NE, NO, SD, IND, and BAL top the list of the most conservative offenses. They were also the best teams of 2006, with one team missing.

The NFC champion Bears managed 13 wins with a relatively risky offensive balance. This is due to their boom and bust passing game (μ_P = 4.8, σ_P=15.1). This result suggests that in 2006 CHI rolled the dice often with deep pass plays and got lucky. 2007 wasn't so kind to them.

One interesting application of this kind of risk analysis would be to repeat these calculations for multiple years to see which coaches and/or coordinators really are the most conservative and who are the biggest gamblers. I was surprised to see Belichick as coach of the most risk averse team. He does have a reputation for running on 3rd and short more often than other teams, so perhaps that explains NE's placement on top of the list.

Below is a graph of risk aversion vs. team wins. We can see that teams with a lot of wins generally are the teams that can afford to be conservative.


One possible application of this graph is to measure the vertical distance between the best-fit line and each team's risk aversion score. This distance is the regression "residual" accounting for wins and losses. It basically says how risk averse/tolerant a team was accounting for its wins. A multi-variate regression would be even better, accounting for both team defensive ability and wins. And instead of wins, we could use "4th quarter leads," which would be what really drives deviations from optimum risk tolerance. This analysis has the potential to be a good measure of how well a coach understands the game and his team--not just their running ability and passing ability, but their defensive ability as well.

There is tremendous potential for the application of portfolio theory in football. The "10 yards in 4 downs rule" complicates the analysis, however. Accordingly, each type of down and distance situation may require its own analysis. Plus, play-calling is not a simple pass or run binary decision. There are draws, screens, outs, hitches, flares, and all sorts of other unique plays. The risks and benefits of each type of play also require their own analysis.

To me, this is exciting because football may finally have a way of matching the depth of mathematical analysis pioneered by our sabremetrician friends in baseball. Baseball is simpler in many ways--run production is generally linearly additive and there are very few options for a team or player to increase or decrease risk as the situation requires. Unlike most other sports, risk in football is dynamic. Perhaps that's what makes it so exciting.

The Passing Paradox Part 2

This is a continuation of an analysis of run/pass balance in the NFL. In part 1 of this article, I discussed the potential application of financial portfolio theory in football strategy. In part 2 of this article, I critique a recent study that made a great stride in this effort.

Commenter JG referred us to a very interesting research paper by economist Duane Rockerbie called "The Passing Premium Revisited." The author applies portfolio theory to re-examine the run-pass balance in the NFL. He finds that teams pass too often. I think his approach is brilliant, but unfortunately his methodology has flaws similar to Alamar's original Passing Premium paper and his conclusions misinterpret his results.

In "Revisited" the author applies a version of the utility function (below) to find the optimal run/pass selection of all 32 NFL teams for the 2006 season. The optimal run/pass ratio is found by taking the derivative of the utility function and setting it equal to zero, thereby finding the curve's maximum. Each team's optimum run/pass ratio is based on the relative strength and variance of their running and passing games.
(The equation basically says that utility of a strategy (X) is a diminishing function of risk aversion/tolerance (alpha) and the expected return of the strategy (v).)

The author finds that most teams do not run as much as they should, and calls the difference between the optimum and actual run/pass ratio "run inefficiency." Run inefficiency is found to be linearly and convincingly correlated with losing. In other words, teams that run as much as they should won more than teams that passed too often. This would be very strong evidence that the run is underused in the NFL, and that the author has discovered a method for instructing coaches how often to run.

First, the computation of expected run yards and expected pass yards leave out some considerations. Like Alamar, the author assigns a -45 ard assessment for each interception. But also like Alamar, he appears to leave out sacks. Sack yards should count against the average pass, and each sack should count as a pass attempt--although the ball was not thrown, a pass play was called. The effect would be bias in favor of the pass. It's also not clear if he factored in additional yardage bonuses on touchdown plays. He doesn't mention it, so I would think not. Since more TDs are from passes than runs, the effect would be bias against the pass.

Also, quarterback scrambles should count as pass yards, not as run yards. They are the result of pass plays, just as sacks are not negative run plays. This might have a large effect on the stats of teams such as Michael Vick's Falcons or Vince Young's Titans in 2006.

But the author goes a step further, and better, than Alamar by excluding kneel downs and clock-stopping spikes from the data. He also factors in penalty yards, which may be important. If passes tended to result in interference calls against the defense, that would make passing look more attractive.

"But the most important factor in a team's risk tolerance may be its defense. With a very strong defense, a team's risk tolerance should be low."

The data is used to calculate the expected (average) result and standard deviation of the run and pass for all 32 teams. From this, he uses the utility function above to estimate each team's optimum ratio of running and passing. But first, the author needs to select the optimum risk tolerance (alpha in the equation above). The optimum risk tolerance is chosen by stipulating that the Chargers' run/pass balance is optimum because they had the best record at 14-2 in 2006, the year from which the data was taken.

I believe this is an error. Recall that the Chargers went 14-2 largely on the back of LaDanian Tomlinson, who led San Diego to an epic 5.7 yds per carry average, and on their #2 ranked defense led by rookie sack leader Shawne Merriman. The author seems unaware of the "running causes winning" fallacy in which teams appear to win because the chose to run more often. In reality, teams that are ahead late in a game, and already very likely to win, chose to run almost exclusively because it is less risky and it burns time off the clock.

The Chargers won 14 of 16 games, presumably leading in almost all of them when they could feed the ball to their talented running back in the 4th quarter. By selecting a 14-win team as the "perfect alpha" team, the author guarantees that any other team that runs less often (accounting for relative strengths of their running and passing abilities) will appear to run less often than they "should."

In fact, I don't believe there is a single uniform alpha for the entire NFL. It changes from down to down and situation to situation. If my team is down by 4 with 2 minutes remaining my alpha would be very negative (very high risk tolerance).

But the most important factor in a team's risk tolerance may be its defense. With a very strong defense, a team's risk tolerance should be low. With a weak defense, a team will likely to need to take additional risks to keep up with its opponent's easy scoring. This leads to my final point.

The author admits that although the selection of the optimum risk tolerance is arbitrary, the correlation of running a lot (accounting for relative strengths) and winning is still strong evidence supporting his utility-maximization analysis. The graph below shows all 32 teams' win totals for the 2006 regular season vs. run inefficiency. The lower the run inefficiency, the more wins a team tends to have.



What we see is that teams that run more often than their risk-reward utility indicate are teams that win. I suspect the direction of causation is that winning allows the running, not the reverse as the author implies.

The next chart is offensive points scored vs run inefficiency. There is a moderate and significant correlation, similar to team wins.


The final chart is points allowed vs. run inefficiency. It's clear that defensive ability explains much of each team's run inefficiency, i.e. run-pass imbalance.


Every team appears to have its own baseline alpha (risk tolerance) based on its defensive strength. Then, based on their respective running and passing abilities, they have an optimum run/pass ratio. But as game situations change in terms of leads, time remaining, and other situational variables, a team's risk tolerance should deviate from its baseline.

There isn't one optimum alpha for the NFL, and if it did it certainly should not be based on the 2006 Chargers. What we see in the charts is that most NFL teams roughly run and pass about as often as they should, given their respective defensive, running, and passing strengths. But some teams do not.

But there's one more wrinkle--football isn't a simple game of yardage optimization. It's complicated by its first down rules. In the final part of this article, I'll examine how this requirement affects play selection, and why I call this concept the "passing paradox."

Continue reading part 3 of The Passing Paradox.

The Passing Paradox Part 1

Recently, I've been examining the "passing premium," the difference in expected gain between a pass play and a run play. After extending the research of this paper, it appears that passing yields a better average gain than running, even after accounting for incompletions, sacks, and interceptions. This would suggest that NFL teams should pass more than they currently do because balance indicates optimization.

"We can think of passing and running as two investments, each with its own expected payoff and volatility. "

Unfortunately, the optimum run and pass mix is more complicated than comparing average expected gains. Commenter "JG" pointed out that passing's comparatively high variance (they are often incomplete, or result in sacks or turnovers) means that passing should have a higher expected payoff. It would not be worthwhile if it didn't. If a team could get the same expected gain by only running, why would it ever risk a pass?

This kind of analysis is based on financial portfolio theory, a branch of math that analyzes and weighs risks and rewards. We can think of passing and running as two investments, each with its own expected payoff and volatility. When a team calls a running play it invests in a run at the price of 1 down, hoping for a payoff in yards. Running would be like buying a share of GE. Passing would be more like buying a share of a tech startup. There is more upside for rapid gain, but there is also a decent chance you'll lose the kids' college fund.

The author of this site proposes several possible applications of the Sharpe Ratio in football. The Sharpe Ratio is a financial measure of expected returns per unit of variability. Specifically, it is the ratio of average returns of an investment over a risk-free alternative to the standard deviation of the investment's value.

By comparing the Sharpe Ratio of running and passing, we can see if there is a premium of one tactic over the other accounting for each tactic's risk. We could also compare two different passing strategies, a high risk/high reward passing offense or a high percentage "dink and dunk" offense.

Consider a simple fictitious example below. Team A is the high-risk/reward passing team and Team B is the higher percentage passing team. The table lists the results of several pass attempts of each team (order is not important in the Sharpe Ratio). Both teams average the same number of yards per attempt. Team A had more incompletions and sacks, yet had more yards per completion. For the zero-risk alternative, I'll use a zero-yard "QB flop" play. Each team had one interception, a -45 yard equivalent.

Pass	Team A	Team B
Pass 1	40	27
Pass 2	22	18
Pass 3	20	18
Pass 4	15	13
Pass 5	15	13
Pass 6	10	13
Pass 7	3	10
Pass 8	0	5
Pass 9	0	3
Pass 10	0	0
Pass 11	0	0
Pass 12	-5	0
Pass 13	-5	-5
Pass 14	-10	-10
Pass 15	-45	-45
Avg YPA	4.00	4.00
Std Dev	18.86	16.75
Sharpe Ratio	0.11	0.12

In this example, the Sharpe Ratio is higher for Team B's high percentage offense, suggesting its rewards are more worth its risks. We would get similar results for any comparison of higher-risk tactics vs. low risk tactics, assuming the average net gain is equal.

The potential for the application of the Sharpe Ratio and all of Portfolio Theory in football strategy is vast. We might finally answer the question of whether a boom/bust running back like Barry Sanders is better than a straight-ahead pounder like Jamal Lewis. We could analyze the merits of Mike Martz's high risk/reward passing doctrine. I'm sure I'll be pursuing such applications in future research. In the meantime, however, the next post will critique a very interesting research paper that makes great strides in applying portfolio theory directly to the passing premium issue.

Continue reading part 2 of The Passing Paradox.

Super Bowl XLII and Team Possessions

First of all, that was an amazing game, possibly the most entertaining Super Bowl ever. Eli Manning played a great game, but the big story to me is how the Giants defense was able to hold the Patriots to only 14 points.

New York's pass rush obviously had a lot to do with their success. Their secondary played amazingly well too. But one of the biggest factors will probably go unmentioned in all the conventional post-game analysis--the clock.

The research from this article suggested that a heavy underdog could see its chance of winning significantly increase when the number of possessions for each team is reduced. The more possessions for each team, the more likely the better team will eventually come out on top. The fewer the possessions, the more likely that luck or other factors can conspire to create opportunities for the underdog to win. Perhaps the easiest way to think of it is that fewer possessions probably means a lower score, and a single drive or play can cause an upset.

In Super Bowl XLII, each team only had 8 full possessions. (This does not count the Giants 10 sec possession at the end of the 2nd quarter and their 1 sec possession at the end of the game.) Most games feature 10 to 13, the average being 11.5 full drives per game. The Patriot's final possession began with 35 seconds remaining, allowing time for only 3 desperation throws and a sack.

The table below illustrates each team's chance of winning a game with the given number of possessions based on a simulation of each team's historic scoring per possession rates. My original table did not even consider the possibility of 8, but I include it here.

Possessions	NE Wins	NYG Wins	Overtime
8	69.8	24.1	6.1
9	71.4	23.5	5.1
10	72.7	22.4	4.9
11	74.0	21.5	4.5
12	75.5	20.7	3.8
13	76.5	20.0	3.5

How did the game yield so few possessions? Long drives with high 3rd down conversion percentages appears to be the biggest reason. The Giants started the game with an amazingly long 10 minute drive culminating in a field goal. The Patriots didn't even finish their first drive until the second quarter. The Patriots started the third quarter with a drive of over 8 minutes resulting in a turnover on downs.

If each team had 2 or 3 more possessions, New England may well have been able to overcome their 3-point deficit. It's hard to say that with any certainty because the Giants slightly outplayed the Patriots almost all night. Congratulations to the Giants and their fans.

Super Bowl XLII Prediction

The game probability for Super Bowl XLII is listed below. The calculations consider only the last 11 weeks of football. The first half of the season is excluded and recent playoff performance is included, accounting for regular season and playoff strength of schedule. The probability estimate also accounts for the game's neutral site.

The probabilities are based on an efficiency win model explained here and here. The model considers offensive and defensive efficiency stats including running, passing, sacks, turnover rates, and penalty rates. Team stats are adjusted for previous opponent strength. Click here for a sortable comparison of playoff team stats from the regular season.

NE Prob	Super Bowl XLII	NYG Prob
0.76	NE vs. NYG	0.24

More Patriots Cheating Allegations

Yesterday (the Saturday before Super Bowl XLII) the Boston Herald reported that a source alleged "a member of the [Patriots] video department filmed the Rams’ final walkthrough" before the 2002 Super Bowl in which the Patriots upset the heavily favored Rams. ESPN reported that the Rams stated the walkthrough primarily focused on plays they intended to run in the red zone.

Asked about the allegations yesterday, commissioner Robert Goodell answered, "I’m not aware of that.” NFL Spokesman Greg Aiello added, "We have no information on that."

Then later, on the same or very next day, spokesman Aiello told the AP, "We were aware of the rumor months ago and looked into it. There was no evidence of it on the tapes or in the notes produced by the Patriots, and the Patriots told us it was not true." (emphasis mine)

Well, that clears that up. Imagine if the movie Untouchables ended this way: Elliot Ness--"Your honor, there is no evidence of tax evasion in the documents provided by Mr. Capone, and he has told us the charges are not true." Judge--"Case dismissed."

Previous Research

I don't normally add my own opinions here at this site, but I'll make an exception here because of my previous look into possible statistical evidence that the Patriots benefited from unfair advantages. Specifically, the Patriots had won about 2 more games per year, every year, from 2002-2006 than their on-field performance would statistically indicate. In other words, other teams with similar performance stats win 2 fewer games in a season than Belichick's Patriots did. If the Patriots used knowledge of their opponents' play calls in primarily high-leverage situations (3rd downs or critical 4th quarter plays) we would see this kind of result.

My own interest in statistics began when I did my masters thesis, a research paper on midshipmen at the U.S. Naval Academy who violated its Honor Concept. It was basically research on cheaters at an elite institution in highly competitive and stressful environment. Although not the primary focus of my research, along the way I learned that cheaters are recidivists. Once they are able to rationalize their behavior, they will continue to cheat. Additionally, those who are caught are rarely nabbed on their first attempt, most likely because they select methods and opportunities hard to detect. They go out of their way to hide their cheating activity. No surprise there.

(Coincidentally, it was at Annapolis where Belichick learned his football under his dad, an assistant coach for Navy.)

So when we saw first hand how the Patriots violated league rules, I surmised it was highly unlikely that their activities were limited to taping defensive signals in isolated games. I think it would be naive to believe otherwise. Several very smart commenters (with very good points) accused me of making "assumptions" about the Patriots' cheating. So if the recent allegations have any merit, I'd feel somewhat vindicated.

Super Bowl XXXVI

Then, just as I was thinking of writing this post this afternoon, I channel-surfed onto the NFL Films highlights of the Patriots-Rams 2002 Super Bowl on ESPN2. Immediately prior to the drive in which Ty Law jumped a quick out route to intercept a Kurt Warner pass and return it for a touchdown, there was a sideline shot of three Patriot defenders discussing signals. (My thanks to TiVo, by the way.)

In the shot, Safety Lawyer Malloy runs up to cornerbacks Terrel Buckley and Terrance Shaw and says,
"Listen! Listen!
We got 'Sloop.' (makes a hand signal)
We got 'Move'...you know the move signal. (makes a different signal)
We got 'Marine 5'...'Marine.' (makes signal)
We got 'Seagull.' (another signal)
We got, this is...this is 'Double Out' right here. (making signal)"
Buckley and Shaw mimic the signals and nod each time.

My guess is these would not be their own signals--they would know them already and Milloy's words "we got" and "you know" plus the names for each signal suggest the signals are somewhat but not entirely new. Additionally, "double out" sounds like an offensive call. They appear to be rehearsing the Rams' signals, although possibly Warner's QB signals and not sideline signals. But the main point is that knowledge of some of the Rams' offensive signals was widespread on the Patriots defense, it was a priority to them, and it apparently didn't hurt New England's performance.

On the other hand, I'd guess that all teams try to read QB signals, but if they could do it reliably well, offenses wouldn't use them. Offenses would also be able to use countermeasures or easily spoof a defense. The Patriots may just play this part of the game better than other teams. If so, we'd see the same statistical results I found in my earlier post. It also underscores that outsiders like myself really don't have a any idea of what really goes on inside the film rooms and coordinator booths in the NFL.

Belichick's Focus

Perhaps Belichick had an intense focus, within the rules, on exploiting opponent's signals and deceiving them with his own. In the military we call this 'SigInt' for signals intelligence, a critically important part of modern warfare. The advantage from signal exploitation may have encouraged the Patriots to pursue it beyond permitted means. Jets coach and former Belichick assistant Eric Mangini would have been aware of the importance of this part of the game to the Patriots, so It's no surprise he was the one to blow the whistle.

A Real Investigation

The other point is that if the NFL really wanted to investigate these things, there is ample evidence in the NFL Films archive. There are probably hours upon hours of sideline film from just the Patriots' Super Bowls alone, not to mention playoff games or regular season games. An honest investigation would have taken weeks, not the couple of days the NFL took before destroying the evidence.

Don't get me wrong. I'm not a Belichick hater. I appreciate his cerebral approach to the game. I like how he goes for it on 4th down and focuses intensely on details, and I don't find him arrogant at all. But I do have a strong intolerance for cheating, and I believe these things deserve to be investigated.

The Passing Premium

In previous posts, I've referred to a concept called the passing premium. Specifically, I point to Benjamin Alamar's paper which found that the expected yards per play is higher for passing than for running. The difference accounts for incompletions and the risk of interception.

I've discovered a flaw in the author's analysis, however. He does not appear to account for sacks.

His analysis finds that for every passing play in the 2005 NFL season, the expected gain is 5.8 yards per attempt. Interceptions are factored in by assigning them a -45 yard value. (40-50 yards as an equivalent for interceptions is a commonly accepted value. It also makes intuitive sense because an interception differs from an incompletion by precluding the possibility of a punt, which usually nets about 40 yards.) Touchdown passes also get an adjustment because the goal line truncates the pass. For every touchdown pass, an extra 10 yds is added.

The true expected gain from a pass play should be:

(Pass Yds Gained - Sack Yds - Int Adjustment + TD Adjustment) / (Pass Att + Sacks)

The author leaves out the sacks both in the numerator and denominator, which makes a difference.

The average run yields 4.1 yards. The difference is an unexplained premium for passing, suggesting that play selection is not rationally balanced in the NFL.

Realizing that football is more complex than a binary run or pass decision, and that averages are not always the truest measure of performance in all situations, the difference of 1.7 yards per play remains considerable. So despite those limitations, perhaps coaches should be calling more passes and fewer runs.

I performed my own analysis, repeating Alamar's methodology for 2005 data, and then expanding it to data from the 2002-2006 seasons. By adding 10 yds per touchdown pass and subtracting 45 yards per interception, I also calculated 5.8 yds per attempt. But when I subtracted sack yards, the expected yield for a pass attempt becomes 5.0 yards per attempt.

The passing premium now becomes 5.0 - 4.1 = 0.9 yards per play, a smaller difference than the author found.

I also calculated running yards per attempt when the same touchdown adjustment is applied. Aren't many running touchdowns truncated by the end zone too? It does seem generous to add 10 yards because some touchdown runs are goal line dives, but many are not. Some touchdown passes would not automatically yield an extra 10 yards either. Adding the 10 yard touchdown bonus makes the expected gain for running 4.5 yards per attempt.

The passing premium would now become only 5.0 - 4.5 = 0.5 yards per play.

Running in some situations, however, has value in addition to yards gained. Towards the end of a game, the leading team can use more clock time by running, denying additional opportunities for the trailing team to score. In short yardage situations, running for a short gain can be more beneficial than the chance to have a longer gain with a pass. Goal line runs can sometimes require 2 or 3 attempts before a gain of the single yard that yields the touchdown, but that single yard is worth the possibility of no gain on previous plays. (In a way, the generous 10 yard bonus for a touchdown run seems more appropriate considering the frequent stuffs on the goal line due to the high predictability of running in that situation.

All things considered, perhaps the run and pass are nearly balanced in the NFL. Balance is important because it suggests maximization. If a team runs too much, a defense can concentrate their efforts on stopping run plays, reducing the expected gain for a run but a greater expected gain for pass plays. Every team would have their own optimum balance, but over the league as a whole the optimum run-pass mix would yield about the same expected gain every play.