How to Lie with Statistics

Stochastic Football

 

Statistics make me cringe. Not the upstanding work done by Brian but too much analysis done by too much of the greater sportswriting community. That word, “analysis,” it doesn't help matters. Like many backwards things Western Civilization, modern usage derives from Aristotle. Real analysis, that's a marvel. What I encounter throughout the web would be better labeled: statistical rhetoric: the use of statistics to forward a previously held opinion. There's this great quote by Wayne C. Booth about critical theory—specifically the idea of showing versus telling in fiction writing—and how it disseminated from scholarly critics, down to commercial critics … well, I'll just share it:

“[T]he legitimate defense of the new soon froze into dogma. … [W]hen such rule-making descended further into the hands of unabashed commercial critics, it was simplified to the point of caricature.”

This is the progression: from new to accepted to—in some skeletal, bastardized form—the mother truckin law. Wanna be taken seriously? Gotta speak the language. For the modern sportswriter statistics are jargon, argot and shibboleth all in one. No wonder an old hat striving for relevance rushed to create an eponymous, um, effect? despite its obvious bogusness.

Brian attracted me to Advanced NFL Stats through his work exposing the phoniness of the so-called Curse of 370. His simple, clearly worded argument of why said curse was cooked up, and either indicative of blithe error or chicanery, challenged me to be careful and inquiring instead of gullible. So in honor of Mr. Burke and his fine and reputable site, I now intend the exact opposite. Let us together learn how to lie with statistics.

 

Compare Differently Sized Samples

Suppose you're a fan of a team—we'll land on the New York Giants because then I get to write G Men. New York has started the season 3-5, and by some miracle, Washington, Dallas and Philadelphia are 7-1, 6-1 and 5-2, respectively. It seems clear that the Giants will need to finish 11-5 or better, both to have a good shot at winning the Wild Card and a realistic shot at winning the division. How do I frame this task so it seems impossible?

Well I might say that New York needs to play 1.000 football the rest of the season. And to underscore just how improbable this is, and further prey on people's pessimism regarding the team, I might point out that that means New York will need to play like the 2007 Patriots to have a realistic shot at the playoffs. But they don't, of course, because eight is not 16 and winning eight straight is nothing like winning 16 straight. In fact, some team has won eight or more to finish the season every season in the last five years, and 23 of the 32 NFL franchises have won eight straight sometime in their franchise history.

This doesn't happen too-too much in analysis of football, where win totals involve small enough numbers to not induce the glazing over of the eye so favored by those that wish to bludgeon with statistics, but it's fairly common in baseball and basketball. Obviously there is no such thing as X.XX football or baseball or basketball. If you shoot like Charles Barkley, sometimes you're going to shoot like Artis Gilmore and sometimes like Allen Iverson. Barkley's not Gilmore when he's shooting well and he's not Iverson when he's chucking it. Data clots like mismatched blood. Don't kill a kidney seeking explanations.

On June 21, the Los Angeles Dodgers were 30-42. If we set the target for potential playoff contention at 90 games, LA would have to finish 60-30, or play .667 baseball. That means LA, this rudderless bunch of individuals, constructed by-uh-bunch-uh suits that think they can buy LEADERSHIP and CHEMISTRY, would have to be better than the 1928, Ruth-Gehrig-Lazzeri New York Yankees, etc.

And that's never gonna happen.

How to apply: Tell your spouse you cook on weekends more often than … um, her. Nevermind you only cook on weekends. Use data to justify not cooking this Sunday. `Cuz woman's gotta lotta work to equal your six—er, five Saturday streak of microwaving Manwiches. Ha, sexism.

 

Irrelevant Groupings

An irrelevant grouping is achieved by picking certain information, say: attempts, receptions and percentage of team attempts. And then showing that by this cooked-up collection of data points, Shonn Greene is analogous to Jim Brown. This often involves arbitrary end points.

Television broadcasts are drunk with irrelevant groupings. Someone in the dark recesses of the booth is finding out, just now, that Arian Foster is the fastest player in NFL history to touchdown runs of ten or more yards against divisional opponents while playing in a meteorologically determined “drizzle.”  Mine enough data and you'll find a lode that extends from Joe Namath to Mark Sanchez, and it won't mean a thing.

How to apply: Henry Miller was a drunk, a womanizer and a failure into his thirties. Are you too on the path to literary immortality?

 

Making a Case

This team is a pretender among contenders. It eked its way into the playoffs on a technicality. The defensive line is starting twice as many street free agents as the offensive line is starting Pro Bowlers. And it shows: the rush defense is ranked 28^th in yards per carry, and the starting quarterback has been sacked 36 times and pressured or hit countless more. He was held out of a vital Week 14 game against a divisional rival because of a concussion. The run game is a joke. They've started three separate backs, this latest the third line in a war of attrition, and he has averaged just 3.5 yards per carry. Winning in the playoffs is about establishing the run and defense. This team accomplishes neither.

The team is the 2010 Green Bay Packers. Ta. Da. Ain't I clever.

This manipulation is so common becoming sensitized to it is nigh-destabilizing, but in this case I think it's important to point out the lie is not in the method but the conclusion. It's not lying to compile information to support a preexisting assumption. That's no more than kiddie he-said she-said argumentation. It becomes a lie when said compiled information is presented as fair and balanced argumentation.

Making a Case is symptomatic of what I think of as the dangers of plausibility. We have access to so much information, so much data to be chopped in so many ways, it is possible to weave a plausible argument for fad diets, fringe political beliefs, out-there conspiracy theories, risque revisionist history, and about anything else. Take a stroll through the non-fiction section of a large bookstore. There is no end to narratives one can weave from that unbounded data set: reality.

Ideally, any argument should do its best to weigh as many sides as possible, and present those sides weighted to their credibility, so that the reader or viewer may determine what is true by what is most soundly reasoned. Instead, rhetoric typically begins with a presumptive conclusion, marshals facts to support that conclusion, maybe offers a straw-man counterargument to be whooped silly, and then concludes with bombast, forewarning, promises—that infernal/eternal set: fear + desire[ ... death, failure, humiliation … success, happiness, satisfaction …]

How to apply: Write critical analysis of Shakespeare. Har Har. But really, do. Interpret Djimon Hounsou's portrayal of Caliban through emerging digit ratio science, and discuss whether The Bard intended* for his mooncalf to show indications of low fetal testosterone levels.

*intentional fallacy, blah blah

 

Shock & Awe

The validity of an idea is not determined by its simplicity, or as James Burke said, “who said genius is simple?” Nevertheless, I've yet to encounter statistical analysis of sport that cannot be fairly easily explained. The data, almost universally, come from box scores—something with which sports fans are all too comfortable. The methods are probably not more sophisticated than can be found here or here or here—all respected analytics, all explained at a level I can understand. (An aside, I am no daunting -mancer of statistics. I am frankly a layman with interest and some knowledge.)

Sometimes people write opaquely because of the curse of knowledge, or a simple lack of skill as a writer. Sometimes people write opaquely to obscure the faultiness of their methods or the ambiguity of their results. Sadly, with statistical analysis so buzzed about, it's often enough to appear to know what you're doing, and be confusing enough to be intimidating. The saying goes: better to hold your tongue and be thought a fool than open your mouth and remove all doubt. A hearty fear of being thought stupid is the flimflam man's best friend. But, shucks, someone's got to be brave and shout out that the emperor is trotting round stark naked.

How to apply: Any incomprehensible graph will do. Make sure to not explain it.

 

Number the Subjective

Statistical analysis has flourished in baseball and basketball, because of the long seasons and resulting sample sizes, but also because of the accuracy and absoluteness of their respective stats. Aside from occasional trickeration, a shot at the basket is a shot at the basket, and opposing defensive ability not withstanding, a hit is a hit.

Football produces some absolute statistics. A pass attempt is a pass attempt. A rush attempt is a rush attempt. If a defender tackles a player in the process of trying to throw, well that's a sack. But what's a pressure? A dropped pass? How catch-able does a pass need to be before we can be sure the receiver dropped it?

In baseball where data is rich, copious and clearly defined, there are still arguments about, for instance, how much we should regress BABIP for hitters and pitchers. That is because though we have a broad understanding of why pitchers are better judged by component stats like strikeouts and walks than runs allowed, DIPS theory basically, we know too that pitchers influence some control over how hard a ball is struck. The information is solid but the information is incomplete.

The problem with so many proprietary stats is that many are dependent on someone, somewhere reducing a subjectively defined action into an objective data point. And, not surprisingly, the upshot of that is the resulting objective data often confirms preexisting biases. It does so because the “scorer” is biased, and it does so because emerging stats are often judged by outsiders by the so-called “smell test.” Which is nothing more than determining if it conforms with, you guessed it, their preexisting biases.

How to apply: Tell your boss you deserve a raise because your Will Or Rather Tenacity & Heart is in the 97% among coworkers you've scored.

 

The Wondrous Black Box

The other problem with so many proprietary stats is in order to protect the recipe for the secret sauce, the proprietor hides the process. It is impossible to independently audit whether that process is any good, and you're left with no more assurance than faith in your fellow man.

How to apply: According to my proprietary algorithm, that I'm calling C.A.M. G.I.G.A.N.D.E.T., Ben Tate will outrush Arian Foster in 2013. Subscribe to my premium service for only $9.95 and dominate your fantasy league.

 

Tiny Boxes Made of Asterisk

This:

1. Tom Brady

Current ESPN Live Draft ADP: 21.5
Current ESPN Live Draft positional rank: No. 4
The Football Scientist (TFS) positional rank: No. 8

Why is Brady being drafted so early? He ended the 2012 season in a slump. His 10.1 vertical YPA (VYPA, a measure of productivity on passes thrown 11 or more yards downfield) ranked 28th in the league last year and his stretch vertical YPA (SVYPA, production on passes thrown 20 or more yards) placed 23rd.

Geez, his SVYPA is down? That spells disaster.

Apart from hiding within a wondrous black box, the above also falls victim to categories so fitted, so particular, as to defy any attempt to determine if they actually mean anything. And, implicit within Joyner's comment, is that they don't mean anything. After all, his data is from 2012, when Brady was the third most valuable quarterback by VBD, and within a short hair of being the most valuable.

Now this is an eye-grabbing blurb before paywall truncation, so maybe beyond the orange 'in' is something more substantial, some chart that shows the correlation between declining SVYPA and overall performance, but I sort of doubt it. And I sort of doubt it because of Brady's 637 pass attempts, 68 were thrown 20 or more yards. A statistical sample effectively equal bubkes.

How to apply: Write a book, brand yourself the Football Scientist, laugh at pissy snotbags like me.

I could go on, but I've surpassed 2,000 words and all this sitting's killing my heart. Statistics and American Football make strange bedfellows. The one: chaotic, indistinct, poor of quantitative information about most of its players. The other: depending on simplicity, distinctness and a robust data set. Yet it's so tempting … stats are trendy, people are attaining celebrity through their mastery, Moneyball and all that; and, more nobly speaking, stats hold old saws, canards and truisms to the candlelight of fact. Statistics have done wonders to raise the level of conversation about sport. It has become okay to be smart and a sports fan. But smart is more endeavor than quality. Smart people are those that seek intelligence rather than those that believe themselves to be intelligent. And as such, with smart comes circumspection for stupid. Stupid is a rust that feeds on idle minds.