More Patriots Cheating Allegations

Yesterday (the Saturday before Super Bowl XLII) the Boston Herald reported that a source alleged "a member of the [Patriots] video department filmed the Rams’ final walkthrough" before the 2002 Super Bowl in which the Patriots upset the heavily favored Rams. ESPN reported that the Rams stated the walkthrough primarily focused on plays they intended to run in the red zone.

Asked about the allegations yesterday, commissioner Robert Goodell answered, "I’m not aware of that.” NFL Spokesman Greg Aiello added, "We have no information on that."

Then later, on the same or very next day, spokesman Aiello told the AP, "We were aware of the rumor months ago and looked into it. There was no evidence of it on the tapes or in the notes produced by the Patriots, and the Patriots told us it was not true." (emphasis mine)

Well, that clears that up. Imagine if the movie Untouchables ended this way: Elliot Ness--"Your honor, there is no evidence of tax evasion in the documents provided by Mr. Capone, and he has told us the charges are not true." Judge--"Case dismissed."

Previous Research

I don't normally add my own opinions here at this site, but I'll make an exception here because of my previous look into possible statistical evidence that the Patriots benefited from unfair advantages. Specifically, the Patriots had won about 2 more games per year, every year, from 2002-2006 than their on-field performance would statistically indicate. In other words, other teams with similar performance stats win 2 fewer games in a season than Belichick's Patriots did. If the Patriots used knowledge of their opponents' play calls in primarily high-leverage situations (3rd downs or critical 4th quarter plays) we would see this kind of result.

My own interest in statistics began when I did my masters thesis, a research paper on midshipmen at the U.S. Naval Academy who violated its Honor Concept. It was basically research on cheaters at an elite institution in highly competitive and stressful environment. Although not the primary focus of my research, along the way I learned that cheaters are recidivists. Once they are able to rationalize their behavior, they will continue to cheat. Additionally, those who are caught are rarely nabbed on their first attempt, most likely because they select methods and opportunities hard to detect. They go out of their way to hide their cheating activity. No surprise there.

(Coincidentally, it was at Annapolis where Belichick learned his football under his dad, an assistant coach for Navy.)

So when we saw first hand how the Patriots violated league rules, I surmised it was highly unlikely that their activities were limited to taping defensive signals in isolated games. I think it would be naive to believe otherwise. Several very smart commenters (with very good points) accused me of making "assumptions" about the Patriots' cheating. So if the recent allegations have any merit, I'd feel somewhat vindicated.

Super Bowl XXXVI

Then, just as I was thinking of writing this post this afternoon, I channel-surfed onto the NFL Films highlights of the Patriots-Rams 2002 Super Bowl on ESPN2. Immediately prior to the drive in which Ty Law jumped a quick out route to intercept a Kurt Warner pass and return it for a touchdown, there was a sideline shot of three Patriot defenders discussing signals. (My thanks to TiVo, by the way.)

In the shot, Safety Lawyer Malloy runs up to cornerbacks Terrel Buckley and Terrance Shaw and says,
"Listen! Listen!
We got 'Sloop.' (makes a hand signal)
We got 'Move'...you know the move signal. (makes a different signal)
We got 'Marine 5'...'Marine.' (makes signal)
We got 'Seagull.' (another signal)
We got, this is...this is 'Double Out' right here. (making signal)"
Buckley and Shaw mimic the signals and nod each time.

My guess is these would not be their own signals--they would know them already and Milloy's words "we got" and "you know" plus the names for each signal suggest the signals are somewhat but not entirely new. Additionally, "double out" sounds like an offensive call. They appear to be rehearsing the Rams' signals, although possibly Warner's QB signals and not sideline signals. But the main point is that knowledge of some of the Rams' offensive signals was widespread on the Patriots defense, it was a priority to them, and it apparently didn't hurt New England's performance.

On the other hand, I'd guess that all teams try to read QB signals, but if they could do it reliably well, offenses wouldn't use them. Offenses would also be able to use countermeasures or easily spoof a defense. The Patriots may just play this part of the game better than other teams. If so, we'd see the same statistical results I found in my earlier post. It also underscores that outsiders like myself really don't have a any idea of what really goes on inside the film rooms and coordinator booths in the NFL.

Belichick's Focus

Perhaps Belichick had an intense focus, within the rules, on exploiting opponent's signals and deceiving them with his own. In the military we call this 'SigInt' for signals intelligence, a critically important part of modern warfare. The advantage from signal exploitation may have encouraged the Patriots to pursue it beyond permitted means. Jets coach and former Belichick assistant Eric Mangini would have been aware of the importance of this part of the game to the Patriots, so It's no surprise he was the one to blow the whistle.

A Real Investigation

The other point is that if the NFL really wanted to investigate these things, there is ample evidence in the NFL Films archive. There are probably hours upon hours of sideline film from just the Patriots' Super Bowls alone, not to mention playoff games or regular season games. An honest investigation would have taken weeks, not the couple of days the NFL took before destroying the evidence.

Don't get me wrong. I'm not a Belichick hater. I appreciate his cerebral approach to the game. I like how he goes for it on 4th down and focuses intensely on details, and I don't find him arrogant at all. But I do have a strong intolerance for cheating, and I believe these things deserve to be investigated.

Response to Luck and Belichick Article Comments

The traffic to this site has increased quite a bit lately. A lot of it is likely due to the interest in the playoffs, but much of it is from direct links from other sites. Most direct links are to my articles about Rating 'Gameday' Coaches and about Belichick Cheating Evidence. They have appeared on many message boards across the football world, and many of the comments and criticisms are outstanding. Allow me to address some of the comments here. The foundation of both articles deals with luck. Because there are so many new readers here, I'd like to clarify what I mean by luck, and how I use this definition when I apply statistics to make observations about coaching.

Luck

To me, a good example of luck is the "bunching" of successful events. In football, first downs are nice, but consecutive first downs are what allow touchdown drives. The number of first downs and the yardage gained in each should be ascribed to skill. Those are things the teams on the field control. But whether those first downs come in bunches or are interspersed is something different.

Perhaps a baseball analogy illustrates my point best. One single per inning gets a team zero runs after nine innings. But nine singles in one inning, followed by zero hits in eight innings would usually yield about six runs. In football, think of a drive as an inning--a team usually needs consecutive successes to score. If players could control when successful plays occurred, sports would be very different. Batters would save their hits for when runners are on base, or when the game is on the line. Receivers would save their dropped passes for the 4th quarter of blowouts.

Here is a football example I've used before: Let's say both PIT and CLE each get 12 1st downs in a game against each other. PIT's 1st downs come as 6 separate bunches of 2 consecutive 1st downs followed by a punt. CLE's 1st downs come as 2 bunches of 6 consecutive 1st downs resulting in 2 TDs. CLE's remaining drives are all 3-and-outs followed by a solid punt. Each team performed equally well-same yards, first downs, turnovers, kicking etc. But the random "bunching" of successful events gave CLE a 14-0 shutout.

That's what I mean by luck. There are several other factors that could be considered random, but my theory is that the bunching effect could explain the bulk of the observed differences in in-game performance and ultimate outcomes.

Luck and the Model of Team Wins

So when I rank teams by luck, I am using my prediction model to estimate how many games a team would be expected to win given their on-field performance. If they win more than expected, I say they're lucky. If they win less, they're unlucky.

Admittedly, the model can't possibly account for every possible consideration on the field of play. There are too many moving parts and inter-dependencies in football to just say, "whatever I don't account for must be luck." There are other factors, such as weather or coaching tactics, some of which are unmeasurable.

But we do know how accurate the model is. We know it accounts for 80% of the variance in team win totals. And there are sound techniques showing that luck, or randomness, accounts for a very large part of the 20% that's left over. That's why I call the model's residual (the difference between estimated and actual wins) luck. Or least the bulk of it is.

Coaching Tactics

Coaching tactics on gameday, such as clock management and whether to kick or go for a first down, are one of the things the model does not account for. It is a small part of the residual. When I ranked coaches on their gameday tactics I used the residual of the model. But a critic would rightfully point out the obvious contradiction--How can the residual be considered luck in one case and 'coaching tactics' in the other?

The answer is that luck is random by definition. It does not correlate with anything. So if you average out enough years of performance, the luck part of the residual tends to cancel itself out, and what's left over is non-random considerations, including coaching tactics. The more years you have in the data, the more likely it is that the luck cancels out. When there is only one year of data, the residual will still contain the luck. This isn't my own personal theory, but one of the central tenets of inferential statistics.

Further, when you have enough years of data and divide up and analyze the data by coaches, and not by something else, you get a good estimate of that coach's gameday contribution to his teams' win totals. Essentially, I'm saying other coaches, given the same on-field capability of their players, would win X many games. Coach so-and-so won on average X+1.2 games per year, so he is credited with a +1.2 "wins added wins per year" score.

A coach that takes a fantastically talented football team to a 10-6 record would not score high. But a coach that can consistently take an average team to a 10-6 record would be ranked at the top.

By the way, I call it 'gameday,' because the preparation and practice part of the coaching job would be reflected in the on-field performance stats and not in the residual. It's the 4th down decisions and such that aren't captured in the efficiency data I use.

Belichick and Cheating

After ranking all the coaches, I had expected to see Belichick at or near the top of the list. He was actually near the middle. So I split his ratings for his tenures at Cleveland and New England. His 'wins added' score was literally off the charts. It was 3 standard deviations beyond any other coach, and he never had any single year that wasn't off-the chart itself. That would make him not only a once-in-a-lifetime type of tactician, but a once-in-a-millennium super-genius.

At first I thought, wow, he really is something special. But then the cheating revelations hit, and I thought this could be due to more than just genius. In fact, it makes a lot of sense given that we already know he is willing to break rules for a competitive edge. There were many other reports of cheating by the Patriots, beyond taping signals, such as exploiting QB helmet radio communications in various ways.

I'm not saying the Patriots aren't a great team or even that Belichick isn't a great coach. They obviously are. But both things can be true. They can be both great and cheating.

A solid criticism of my approach would be that I can't just chalk up the one team that breaks my model to cheating. I'm not. I was scratching my head wondering why this one team defies the statistical tendencies of the 31 other teams. Then several weeks later, it was revealed that that same team had been cheating.

By no means do I claim that my analysis is iron-tight. I think it's useful and interesting. Feel free to disagree.

Belichick Cheating Evidence?

One of the more apparent signs that a spy or corrupt official is cheating is that he is living a lifestyle beyond his means.

If Belichick's Patriots exploited unfair advantages in stealing signs from opposing sidelines we would expect to see some sort of evidence that they won games "beyond their means." By means I am referring to the Patriots' passing and running performance on offense and defense.

By successfully exploiting stolen signs, we might expect the Patriots to choose to use that advantage on critical plays--3rd downs in the 4th quarter for example. These critical plays would heavily "leverage" performance on the field to be converted into wins. In other words, the Patriots would win more games than their on field stats would indicate.

This is exactly what we see in the data. Year-in and year-out, Belichick's Patriots have won about 2 more games than expected given their offensive and defensive efficiencies, including turnovers and penalties. No other modern team has even come close to the Patriots in consistently winning more games than their stats indicate. Could those extra wins be due to cheating?


For a comparison of other teams' actual/expected wins charts, see this article. When I first discovered this pattern, I believed it was evidence of Belichick's in-game "genius." He was known for some unconventional tactics, such as going for it on 4th downs more often than his counterparts. (I'd go for it on 4th down too if I knew the play the defense would run.)

In a previous article, I ranked every head coach since 1983 in terms of how many excess wins they had above their expected wins based on team efficiency. Belichick's career, as a whole, was rather average due to his poor results in Cleveland. But by isolating his tenure with New England, his excess wins per season would rank him as the best tactical coach ever, with an extra +2.33 wins per season. The rest of the pack is far behind with the 2nd best coach at +1.83 wins per season. Belichick is a true outlier, at almost 3 standard deviations above the mean.

This is only circumstantial evidence of cheating, but it is evidence. And although hardly damning, we can be sure of one thing about Belichick--he is willing to cheat. If someone has crossed the line by breaking one rule, what makes you think he's not willing to break others?

On some other sites I've read some emotional comments rationalizing the Patriots' cheating. Whether other teams have attempted this or not is not relevant. Patriots fans and Belichick fans must accept the possibility that much of their success has been due to cheating. What I've done here has added weight to that possibility and quantified the potential scope of this scandal. At the very least, we would be justified in continuing to investigate the Patriots' methods.

Note: Some other examples of teams' expected vs. actual wins can be found here. Also, since this post is appearing on many other sites recently, I've posted a response to many of the excellent comments and criticisms over the past several months.

Patriots Sign Stealing

Is there evidence that the New England Patriots benefitted on the field from videotaping their opposition's defensive signals? One indication would be their success in rematch games. If the Patriots have been exploiting signal stealing regularly in past years, we would expect them to have an advantage against teams they play more than once in a season. The Patriots would be expected to score more points in the second game against a given opponent.

Here are the games in which the Patriots played the same opponent twice in a season for the past four years. Each year they play their three division rivals twice, plus they happened to play one of their playoff opponents during the regular season. The points scored by the Patriots in the first meeting and the rematch are listed.

2006	Points Scored
Opponent	1st Game	2nd Game
NYJ	24	14
MIA	20	0
IND	20	34
BUF	19	28
Total	83	76

2005	Points Scored
Opponent	1st Game	2nd Game
BUF	21	35
DEN	20	23
MIA	23	26
NYJ	16	31
Total	80	115

2004	Points Scored
Opponent	1st Game	2nd Game
IND	27	20
BUF	31	29
MIA	24	28
NYJ	13	23
PIT	20	27
Total	115	127

2003	Points Scored
Opponent	1st Game	2nd Game
BUF	0	31
NYJ	23	21
IND	38	17
MIA	19	12
Total	80	81

Only 2005 shows any real difference in points scored between the first and second games against the same opponent. Even so, we'd have to show that the 8.3 point/game difference in 2005 is significantly outside the normal variation between first and second games between all teams during the same period.

This result does not indicate the Patriots were not cheating by videotaping signals, but it does show they did not benefit signifcantly in points scored during rematch games in recent years.