## Stats Measure the Intangibles

It’s a common criticism of statistics: Stats don’t measure the intangibles. You know—the drive, character, discipline, leadership, teamwork, and all the other desirable qualities in athletes. Except that they do. Stats measure the effects of what we call intangibles, and always have.

Although there is no event at the Combine that can directly gauge a player's will to win, it still manifests in his performance on the field. And as long as the stats reflect what actually transpires between the sidelines, they capture the effects of the competitors’ intangible qualities. And if the intangibles don’t show up in the numbers, then they didn’t matter in the first place.

What statistics can’t do is separate the physical-material tangible qualities, such as speed, strength and skill, from the intangible ones. They are captured together. But that's no different than trying to separate the effect of speed from the effect of strength. All qualities, whether tangible or intangible, are inseparable using game statistics alone.

If a great player is also a great leader who inspires his teammates to play better, then that will be reflected in his teams’ statistics. It may not be captured in his personal stats, but that has always been true of the effects of teammates on each other’s performance, whether tangible or intangible.

There are exceptions, usually temporary. If a team is one win from the playoffs, and their opponents have been eliminated from contention for weeks, then the team stats to-date won’t fully reflect the relative motivation of each team. But once the game is done, the game stats will reflect the difference in motivation, assuming it matters.

Don’t get me wrong. I’m in favor of good character and the will to win as much as the next guy. But to the extent these qualities influence play on the field, the numbers will capture their effect. So the next time your buddy says stats don’t measure the intangibles, you can say, ‘Sure they do. And besides, where have these magical qualities been hiding all season? And why haven’t they shown up on the field until now anyway?’

### 27 Responses to “Stats Measure the Intangibles”

Brian, have you ever considered looking into which college statistics correlate well with professional success? Scouting seems like such a crapshoot, and while college statistics are notoriously inflated, maybe there's something like net passing YPA above conference average that is a decent predictive stat?

2. Anonymous says:

TEBOW IS A WINNER.

Especially when he plays winless teams. :/

3. James says:

Adam, someone has looked into this and I don't remember the final results, but Completion Percentage was one of the most predictive/carry-over stats (although that's not saying much).

As I recall the formula they came up with had Colt McCoy with a very high ranking.

4. Anonymous says:

One of Football Outsiders' collaborators found that college completion percentage and games started were the two stats that correlated most highly with pro success, but there's no consensus over whether it's held up in recent years - for example, Tebow grades highly on both.

5. Anonymous says:

I totally agree. Like Peyton Manning. The intangible that he sucks in the playoffs is well captured by statistics like:

Sub-500 record
Terrible record when his team is the favorite
Lower QB rating than Rex Grossman in 2006 playoffs

6. David Myers says:

I suspect comments about intangibles are holdovers from the day when all anyone had to work with was a box score, perhaps incomplete.

The better the data set, the more completely the athlete's performance is captured. I'm sure notions of this kind are what powered Daryl Morey's comments in his recent HBR article.

David.

7. Michael Beuoy says:

I can't tell whether Anonymous is joking, but I'll take the bait:

Peyton Manning in the playoffs: 18 games, 0.21 WPA/G, 0.17 EPA/P

Tom Brady in the playoffs: 19 games, 0.16 WPA/G, 0.08 EPA/P

8. Whispers says:

Well, tangible means "capable of being perceived (esp. by touch)" so I would have thought that whatever stats are measuring, they are tangible properties. If the quote were about observables (or unobservables) instead of intangibles, it would be practically tautological.

9. Anonymous says:

The seeming paradox lies in the fact that "intangibles" don't really exist - they're something that fans and the media invent to build up a player who is beloved but not very good. If they're doing something that actually helps their team, it's tangible and its effects can be measured.

10. Anonymous says:

Mike B: that's really nice for Peyton. Yet he trails in the only stat that matters in this arena...

11. Jim Glass says:

One of Football Outsiders' collaborators found that college completion percentage and games started were the two stats that correlated most highly with pro success, but there's no consensus over whether it's held up in recent years

They came up with a projection method that had the common failing of backtesing very well but predicting not so well. What fell apart was the "years started in college" correlation. Now they have come up with a Version 2 that backtests even better. As to how well it will predict .... ? (If it predicts really well then they've made the dread mistake of Voros McCraken, coming up with an idea worth millions of dollars and giving it away for free on the Internet.)

You can read the whole story here:
http://www.footballoutsiders.com/stat-analysis/2011/introducing-lewin-career-forecast-v20

The completion percentage correlation is pretty dubious in my mind too. Backtested on a small sample of QBs who were actually drafted it may show some results, but the challenge is to look at the entire population of college QBs and decide to who draft going forward. Right now about a dozen Div 1 QBs are hitting over 70%. How many of them will be good NFL QBs?

Tom Brady hit only 61% in college. His completion % went *up* in the NFL. Colt McCoy hit 77% (the all-time record) and 71% his last two years in college, so if that measure rules he should be a lock ... but is he?

The college game is so different from the pro game by scheme, physical level of competition and mental requirements that how a college QB's skills will "scale up" in the pros is the deca-million dollar question. Brady and some scale up great, even after being less than stellar in college. Others not at all. How to tell who will be which still is pretty much a mystery.

12. Jim Glass says:

As to Peyton playing so poorly in the playoffs, here's PFR.com list of top QB single game performances since 1970:

http://www.pro-football-reference.com/blog/?p=7234

Check out #3, #4, and #5.

"I can't tell whether Anonymous is joking, but I'll take the bait: Peyton Manning in the playoffs, 18 games, 0.21 WPA/G, 0.17 EPA/P ...Tom Brady in the playoffs: 19 games, 0.16 WPA/G, 0.08 EPA/P".

that's really nice for Peyton. Yet he trails in the only stat that matters in this arena...

No, he doesn't. If you are referring to Super Bowl victories, teams win those, *not* QBs.

If you don't believe it then tell us: If Brady had been drafted if the fifth round by Matt Millen for the Lions, where would he rank today in "the only stat that matters", eh?

Peyton has solidly better numbers than Brady in the playoffs. Deal with it.

13. Jonathan says:

"If Brady had been drafted if the fifth round by Matt Millen for the Lions"

Or the Colts...good luck to anyone not named Peyton Manning in getting that 2009 team to the Super Bowl. Shoddy defense, no running game, and apparently a 0-7 team two years later without Manning.

14. Michael Beuoy says:

Thanks Jim. Apparently, Peyton's missing intangibles are his poor special teams play and sloppy open field tackling.

Until he fixes their issues with kickoff coverage, Peyton will never be a truly elite QB.

15. Anonymous says:

Which stats measure heart?

16. Anonymous says:

Pulse, blood pressure, and, during an autopsy, weight.

17. JMM says:

Consider what is measured. Things like "heart," 40 yard dash times, leadership, strength, "will to win" are inputs to a process called a football game. Things like WPA, completion %, DVOA, passer rating, rushing yards per attempt are measures of different performance aspects of the game which are proposed to, in some way make the game more predictable by making it more understandable. Some of these have been shown to be somewhat correlated with winning games. However, since there are a large number of decisions being made, many at the same time, during the game by independent actors none of the measures of inputs nor of performance aspects will ever be a substitute for the actual playing of the game.

Hence the axiom: "That's why they play the game."

18. Ryan Wanger says:

I like hating on Manning also, but the Colts are 49.4% against the spread as favorites since 1998 (178 games). If you're just counting playoffs, its 46.7% (16 games).

"Terrible record when his team is the favorite" as stated above is most certainly not true.

19. Jonathan says:

"Which stats measure heart?"

Here's a clue: It's not RINGZ or WINZ.

20. James says:

I don't see why comparing someone to the spread is helpful at all when determining how good they are. The spread takes that into account! If the Colts started a bad QB then the spread would be lower, if they starting a guaranteed Hall of Famer (say... Manning) then the spread goes up.

The only thing comparing someone to the spread does is tell us the public's perception of the player (and by player I really mean the team and it's opposition). If the public and/or Vegas inherently underrates someone, then the team will consistently beat the spread regardless of actual ability, and vice versa. Tell me how a team has performed compared to the spread tells me nothing about the actual ability of the team!

Imagine Team A is a 10-point underdog every game, but they end up losing all their games by 3-7 points. Meanwhile Team B plays the same schedule but is favored by 10-points, but only wins each game by 3-7 points. Team A is 10-0 against the spread, and Team B is 0-10, but Team B is clearly the better team! The record vs the spread tells you nothing!

21. Jim Glass says:

With NO scoring 62 via 236 yards rushing and 321 passing on 31 of 35, either (1) the Colts are sorely missing Peyton's invaluable defensive contributions or (2) he picked the right year to be injured.

As to the notion that this year could be a "natural experiment" that quantifies his value to the Colts, forget about it.

22. James says:

Let's not forget that Peyton could have SOME effect on the defense. If nothing else, Peyton would have extended the Colts' offensive drives, improved field position, and was less likely to turn the ball over and reduce the total number of points scored.

Whether or not he would have had any moral effect on the Colts defense or improved their efficiency is certainly not clear, but he WOULD have impacted the game. For all we know we missed out on a repeat of the Packers-Cardinals shootout from the playoffs two years ago.

23. Jim Glass says:

Let's not forget that Peyton could have SOME effect on the defense.

Sure, just like defenses have *some* effect on QB performance by affecting whether they are playing from behind, the conditions in which they get the ball, etc.

For all we know we missed out on a repeat of the Packers-Cardinals shootout from the playoffs two years ago

OTOH, if you seriously believe Peyton by himself is worth enough to turn a 62-7 game into a close shootout -- and, correspondlingly, move the Colts #32 defense up to league-average or better -- then, well, we have a difference of opinion about the scale of that effect.

24. Jeff Fogle says:

R. Wanger, do you have Peyton's ATS record as a favorite since the Dungy era began in 2002? And, his overall ATS record during that time? Thanks in advance if you do...

25. takeitdown says:

Intangibles get measured, of course, in the game, as you say Brian. However, intangibles seems to be used to describe overall game stats that normally wouldn't be attributed to a single player.

I'm not a big fan of the intangibles argument, but I don't think people claim they can't be measured at all...they argue about whether they should be applied to a guy.

We who use advanced stats would normally say a QB with a 6.5ypa, 3/2 TD/INT ratio, and 200 ypg is well below average, even if his team is 12-4. People who argue for intangibles would argue that he isn't a well below avg QB...look, he leads his team to more victories than this "great stats" QB who only got 8 wins.

Obviously, usually this is hogwash. But, if you believe in the intangibles, maybe that 6.5 ypa 200ypg QB IS better than the 7.5 ypa 280 ypg QB. I think that's where the argument is normally had (especially if neither have dominant D's)

26. Jeff Fogle says:

Didn't hear back from R.W. Did some by hand over at covers. Looks like about 57% in the regular season overall since 2003, 55% as favorites for Manning within the Dungy/Caldwell era in terms of besting Vegas expectations. Mileage may vary depending on openers/widely available/closers/etc...

27. Jim Glass says:

Obviously, usually this is hogwash.

Right, because if the QB has "intangibles" that win games why is the team passing offense below-par? What won the games? Probably the defense, special teams, luck.

I've seen people say: "Tebow won Sunday's game with clutch play. They were behind, he led them from behind in the last two minutes, he won the game. Fact. Deal with it."

He won the game by leading the offense to zero points in 57 minutes against an 0-6 team ... having special teams recover an on-side kick ... having the opposing QB fumble the ball to set up his team's winning FG ... all after the other team missed the extra point that would have secured the win in regulation.

Intangibles in action! How he won the game!