Trying to measure individual performance in football is nearly impossible. Perhaps kickers are the only players whose performance we can isolate from the rest of his team. The position in a distant second might be the quarterback. But measuring QB performance with statistics is still very problematic. Consider this tale of two QBs.
One quarterback led his team to a 12-4 regular season record after starting slow at 3-3. He won the Associated Press's MVP award. Another quarterback was replaced due to ineffectiveness early in the season by an aging journeyman best known for headbutting a stadium wall. Which one would you want on your team? You wouldn't be able to tell from their official stats.
Although one didn't play the entire season, both passers had nearly identical "per attempt stats." The first QB threw for 7.2 yards per attempt and the second QB threw for 7.1. They both had a 95 NFL passer rating. Actually, Tavaris Jackson's was 95.4 and Peyton Manning's was 95. But then again, I might be able to approach a 95 rating if I were throwing dump-offs to Adrian Peterson.
If football were a brand new invention, and we had to decide how to credit the various amounts of yards gained to various players, how would we do it? If I said, "There's this kind of play called a pass, in which a thrower passes the ball to a another player who then runs with it as far as he can. I say we credit all the yards run by the receiver to the thrower," you'd say I was nuts.
I'd say, "Well, it takes a special kind of talent for a passer to get a lot of yardage after the catch (YAC). I won't be able to prove it, in fact, I won't have any evidence for that statement at all, but I still think our primary measure of a passer should include all those yards." I'd be laughed at.
Here are the QBs from 2007 who led the league in percent of their passing yardage as YAC: Croyle, Testaverde, Greise, Harrington, Favre, McCown, Losman, and Lemon. The 2006 list includes Brunell, Carr, Favre, (Rob) Johnson, and (Alex) Smith. There's isn't a single guy on that list who we can call a legitimate starter.
The 2008 season's list of leaders in %YAC include Cassel, O'Sullivan, Campbell, Favre (again), Losman, and Wallace. But Matt Cassel is good, right? Maybe not. Keep in mind how good the team around him was. He was handed the keys to a Ferrari. If a QB racks up his passing yards with YAC, he's either throwing lots of short check-downs and screens, or he has spectacular receivers--or both. Neither is necessarily an indication of a particularly skilled passer.
If we throw away all the YAC and look underneath, what do we have left? I call it Air Yards (AY). It's the distance forward of the line of scrimmage a pass travels. Although it's not a perfect measure of a passer, I think it makes a lot more sense than crediting Donovan McNabb with 71 yards and a touchdown for a 1-yard screen pass to Brian Westbrook.
To be clear, I'm not claiming that a QB has absolutely zero contribution to YAC. The QB has to complete the pass for there to be any YAC in the first place. It's just that the majority of credit assignable between the QB and receiver is due to the receiver. (Much of it can be attributed to the defense and to random variation). Plus, there are better ways of crediting the QB for a completion. Looking at Air Yards at least tells us a lot about a QB that we wouldn't otherwise know. I might be throwing a little of the signal out with the bathwater, but the remaining signal-to-noise ratio is hopefully much better.
Here is how the 2008 regular season Air Yard stats break out. Click on the table headers to sort.
Rank | Name | Team | Yds | YAC | YAC% | AY/Att |
1 | Delhomme | 3288 | 1269 | 39 | 4.9 | |
2 | Ryan | 3440 | 1404 | 41 | 4.7 | |
3 | Rivers | 4009 | 1840 | 46 | 4.5 | |
4 | Rodgers | 4038 | 1652 | 41 | 4.5 | |
5 | Pennington | 3653 | 1546 | 42 | 4.4 | |
6 | Rosenfels | 1431 | 664 | 46 | 4.4 | |
7 | Cutler | 4526 | 1881 | 42 | 4.3 | |
8 | Manning P | 4002 | 1627 | 41 | 4.3 | |
9 | Manning E | 3238 | 1220 | 38 | 4.2 | |
10 | Brees | 5069 | 2398 | 47 | 4.2 | |
11 | Romo | 3448 | 1578 | 46 | 4.2 | |
12 | Schaub | 3043 | 1470 | 48 | 4.1 | |
13 | Roethlisberger | 3301 | 1368 | 41 | 4.1 | |
14 | Warner | 4583 | 2173 | 47 | 4.0 | |
15 | Hill | 2046 | 895 | 44 | 4.0 | |
16 | Garrard | 3620 | 1494 | 41 | 4.0 | |
17 | Garcia | 2712 | 1248 | 46 | 3.9 | |
18 | Edwards | 2699 | 1266 | 47 | 3.8 | |
19 | Orlovsky | 1616 | 652 | 40 | 3.8 | |
20 | Frerotte | 2157 | 1023 | 47 | 3.8 | |
21 | Jackson | 1056 | 502 | 48 | 3.7 | |
22 | McNabb | 3916 | 1805 | 46 | 3.7 | |
23 | Hasselbeck | 1216 | 451 | 37 | 3.7 | |
24 | O'Sullivan | 1678 | 887 | 53 | 3.6 | |
25 | Flacco | 2971 | 1433 | 48 | 3.6 | |
26 | Thigpen | 2608 | 1101 | 42 | 3.6 | |
27 | Russell | 2423 | 1143 | 47 | 3.5 | |
28 | Collins | 2676 | 1292 | 48 | 3.3 | |
29 | Orton | 2972 | 1450 | 49 | 3.3 | |
30 | Favre | 3472 | 1779 | 51 | 3.2 | |
31 | Quinn | 518 | 230 | 44 | 3.2 | |
32 | Wallace | 1532 | 755 | 49 | 3.2 | |
33 | Griese | 1073 | 488 | 45 | 3.2 | |
34 | Anderson | 1615 | 724 | 45 | 3.1 | |
35 | Bulger | 2720 | 1336 | 49 | 3.1 | |
36 | Campbell | 3245 | 1686 | 52 | 3.1 | |
37 | Cassel | 3693 | 2116 | 57 | 3.1 | |
38 | Palmer | 731 | 358 | 49 | 2.9 | |
39 | Fitzpatrick | 1905 | 848 | 45 | 2.8 | |
40 | Losman | 584 | 294 | 50 | 2.8 |
I think this is an interesting way to look at QB performance. Quick thoughts...
This includes screen passes, which have nothing to do with the QBs throwing ability, and probably accounts for a substantial part of YAC. Cassel, for example, is really low on this list and i'm not sure he deserves that. Note how his YAC% is 57! thats much higher than almost everyone else. The patriots throw a lot of screens. Maybe find a way to exclude passes of 1 yd and shorter? That doesn't seem likely given available statistics, but I thought I'd point it out.
Just to clarify, rather than saying cassel and favre aren't as good as their numbers, we should say their case is more complicated. Simply assuming high YAC% means overrated isn't true. We don't know how many TRUE AY/attempt a QB like Cassel has. While screens artificially increase his passing yards, as you point out, they also artificially decrease his AY/attempt.
There is also a difference between a 10 yard come back route and a 10 yard slant. The slant has a much better chance of YAC as the receiver has the chance to catch the ball in stride and continue the play with YAC. Both require the QB to throw 8-10 yards with accuracy and timing. The reasons one route may be thrown more than the other can include considerations including QB and receiver abilities as well as the type of defense being played.
Like all football stats, an interesting piece of the puzzle, but still just a piece of the puzzle.
Very interesting stuff.
I do agree with mrmacdude, though. A lot of screen passes are completed behind the line (I think). You don't want to give them more credit for throwing an incompletion than a completion.
Here's an idea that might be interesting: take every completed pass a QB threw, and credit him with that distance plus the league average YAC on completions of that distance.
So if Cassel completes a pass 2 yard behind the line, he doesn't get a -2, he gets the league-wide average of all passes that were completed 2 yards behind the line. I'm guessing that'd be around 4. That might not change the numbers much, but I'd be curious to see.
My name is Martin from Dundee - love your website, Brian! Really interesting article - have you considered looking at the other side of the equation by calculating the "ground yards" of receivers based on the amount of yardage they actually complete themselves from the point at which they have made the reception?
Maybe a better measure would be passes attempted at 10 or more Air Yards or beyond the first-down marker, whichever is greater, because the defense often sets up their zone near the first-down marker and throwing underneath the zone is only slightly more difficult than throwing a screen. Unfortunately the NFL statistics are not conveniently broken down this way, so this type of analysis can be time consuming and tedious.
Martin from Dundee - the Washington Post has a partial listing of receivers' yards gained after catch here:
http://stats.washingtonpost.com/fb/leaders.asp?type=Receiving&range=NFL&rank=231
(update) Yahoo Sports has an even better listing of YAC:
http://sports.yahoo.com/nfl/stats/bycategory?cat=Receiving&conference=NFL&year=season_2008&sort=28&timeframe=All
Interesting, but why is air yards per attempt a better measure of skill than completion percentage within a certain range?
your measure penalizes a guy whose coach called alot of screen passes (which I don't see how that negatively reflects on the QB (unless he's doesn't complete them))
seems to me that if QB A completes 55% of passes traveling through the air 10-15 yards (or whatever, pick the measurement that makes sense) and QB B completes 50% of passes traveling through the 10-15 yards, that would be a better indicator that A>B than the air yards per attempt
this measure would reward the overly conservative passer that checked down to safe passes instead throwing riskier passes downfield (though your measure rewards the opposite rewarding risky downfield throws instead of safe dumpoffs (which are rewarded equally to an incompeletion essentially))
while its kind of 6 of one half dozen of the other I think viewing the dumpoff as a walk (not really indicative of anything) makes more sense, to me at least
on a side note I'd love to see where Tebow ranks to other guys in college on the benefiting from YAC
While I agree with some of the confounding considerations others have brought up, fundamentally, I agree with you that this is such an obvious way to accurately measure QB yardage that it seems absurd to me now that we include YAC. I feel like we're going to eventually look back on this the way that we will eventually look back on measuring a hitter by RBIs: i.e., WTF?
If you just asked me to sort my personal assessments of QBs in 2008 from best to worst, it would look much more like your AY/A list then this list (sorted by Y/A):
http://www.pro-football-reference.com/years/2008/passing.htm
Your list bumps Peyton Manning and demotes Kurt Warner, to list just two examples, which certainly makes sense to me.
Also, not to bring the level of discussion down a notch, but LOL Favre. If you were to publish an Adjusted AY/A list (I think the formula would be air yards + 10 yards per passing TD - 45 yards per interception) then there wouldn't be three QBs worse with at least 100 attempts.
Brian, excellent work as usual. As a Ravens fan, I can't tell you how many Kyle Boller passes I've seen where his WR would have to make a huge adjustment to just come down with a catch, therefore allowing no chance for the WR to even get YAC. If Boller hit his WRs in stride, YAC would be possible, but Boller wasn't accurate enough for that.
My point is that while Air Yards are a better indicator of a QB's effectiveness, YAC is also good. Boller didn't even set his WRs up for YAC.
Decided to chip in and do it myself. Top 5 in Adj AY/A (descending) are Rivers, Pennington, Delhomme, Ryan, and Rodgers. So not much change there.
The QBs with the most to gain by including TDs and INTs (as Adj yds per attempt does) are Wallace, Jackson, Campbell, Cassel, and Collins, some of whom were names that initially jumped out at me as not being quite as poor overall QBs as their AY/A suggested.
Those who lose the most by considering Adj AY/A instead of AY/A are Rosenfels, O'Sullivan, Frerotte, Hasselbeck, and a tie between Roethlisberger and Orlovsky. That seems fair too. And I was close, but wrong: Favre is 30/40 on your AY/A list, but 35/40 on the Adj AY/A list. Losman was unmovable at the bottom.
Yes, air yards would punish QBs asked to through a lot of screens. I'm not sure how much though.
Part of the issue is that many in the sports stats world tend to focus on (or sometimes obsess over) creating the ONE TRUE STAT that encompasses all skill and ability.
I have to admit I sometimes fall prey to that instinct, but I'm recovering. The NFL passer rating is one good example. Football Outsiders thinks DVOA is the end-all be-all, and baseball SABRmetrics has lots of competing "win shares" and similar scores.
No single number is perfect. But we can have stats that are more useful than others. For QBs, it's necessary to look at more than one stat to get the whole picture. I think air yards could be a very useful part of that picture, and a more useful stat than many of those that include receiver YAC.
Isn't the whole goal to come truer stats that encompass as much skill and ability as we can?
(respect to all the work put into this blog, which is far more work than I (and probably anyone else commenting) have put into said goal))
"They both had a 95 NFL passer rating. Actually, Tavaris Jackson's was 95.4 and Peyton Manning's was 95. But then again, I might be able to approach a 95 rating if I were throwing dump-offs to Adrian Peterson...."
I bet you could! It's hard to overstate how bad the NFL passer rating is as a stat.
Basically it resolves to a yards-per-attempt formula with a 20-yard bonus for each completion, plus 80 yards for each TD, minus 100 yards for each INT.
It so overweights completion percentage that if a QB completes 100% of his passes he gets a rating of 79 even if every completion loses yardage -- no matter how many yards they lose!!
E.g:. Attempts 20, Completions 20, TDs 0, Ints 0, yards from -10 to -1000: rating 79.
Give this same QB who loses 1,000 yards on 20 attempts, -50 per attempt, just one single TD and his rating goes up to 95.8 ... at the top of the league!
Here's a calculator to use to come up with your own fun examples:
http://brucey.net/nflab/statistics/qb_rating.html
In the search for the "one true statistic" this is the evil Anti-Stat.
(You can imagine how greatly this biases the NFL's official annual and All-Time passer rankings in favor of short-throwers to the cost of long-throwers, making it look like every great QB who ever lived played after 1980 in a West Coast Offense. Johnny Unitas ... who was he?)
This stat, just like passer rating, rewards a QB who takes a sack and penalizes the QB who avoids a sack by throwing the ball away.
Pro football is a game which focuses on pass protection more than anything else. With solid pass pro, ordinary QBs look like all-pros. Without it, all-pros look like chumps. That's why offensive coordinators and defensive coordinators focus so much time on that part of the war. It's key.
If you want to revolutionize statistical analysis, compute the quality of pass protection and analyze stats as a function thereof.
Anon-Agreed. You should click on the links to the 2006 and 2007 articles above. There you'll see a much more comprehensive rating.
O-lines, receivers, coaches--they all affect a QB's performance. One improvement at a time.
I think that eveyrone agrees Air Yards will rank a QB that is asked a lot of screens lower than one that throws the ball deep a lot. Screens are often called for strategic reasons, such as slowing down an over-agressive pass rush or because the opposing defense has better CB's than LB's or something like that.
So how about a stat that tells us how well a QB does when asked to do really hard things like through deeper down field. Presumably most QB's good enough to be drafted in the NFL can throw screens and swing passes to RB. So drop them out of the equation and only report on longer attempts. What does the QB contribute that other QB's may not be able to do.
And definitely that should include interceptions and TD's (even short TD's are hard to throw because the field is so compressed and the defense is actively defending the goal line instead of giving up the easy 4 yard pass).
And as for QB being part of YAC, look at Manning E. I swear that every throw I've seen him throw to Burress was a come back route where he through it about 10 feet high and Burress went up and got it with a DB all over his back. Absolutely 0 chance for YAC by both play design and execution.
And there is Manning E with the second lowest YAC in the NFL.
Are you ever going to resurrect your quarterback rating that was based on (IIRC) air yards, sacks, INTs, and rushing yards?
This topic is one that I've argued for years, and I wish there was YAC data that went back many years to allow a more meaningful discussion of "greatest ever" QB. I have almost been excommunicated from my gaggle of Niner friends by daring to suggest that Dan Fouts was a better QB than Joe Montana. I think Montana benefitted significantly from the fact that a) the West Coast offense was new at the time and defenses hadn't figured out the best way to defend it and b) the WC offense relied a lot on short slants and quick dump-offs and high YAC. A monkey could be the QB in that system, with those receivers, and generate a decent QB rating. Not that Montana wasn't really good within that offense, but the fact that Fouts was throwing long passes more often and still racked up great stats puts him higher in my book if you're just evaluating physical QB skills. But I wish I had the data to prove this out. How far back does YAC data go? I haven't found anything older than 2002.
I think this stat overcompensates quarterbacks with great jump ball receivers.
This is a very interesting statistic. However YAC as stated above also is influenced by the accuracy of a QB. Cassel has a high YAC % largely beacause he threw many screens and little deep balls early in the season, but also beacause when he threw short and intermediate routes he was surprisingly accurate for a first year started and his receivers were lead well by the throw creating opportunities to run
This is bull. A major part of being a good QB is passing accuracy. And thats NFL passing accuracy not high school where if you hit the WR its good enough. Great QBs lead the receiver towards the running lane in the defense. In many most routes the receiver cant even see the defense when he makes the catch. Its up to the QB to direct them to the opening. If Young or Montana threw the ball behind Rice it was because they wanted rice to run that way. And he did and racked up one hell of a lot of YAC in doing so. Steve young did a bit on NFL accuracy and leading the receiver towards the running lane a few years ago. I'll look for it. I suggest you guys do too.
So you're saying that the Mannings, Pennington, Roethlisberger, Hasselbeck, Ryan, and Rodgers aren't accurate passers? They are among the lowest in %YAC.
QB's with high accuracy, judged by either completion % or low Int rates, do not tend to generate a lot of YAC.
Admittedly, this is not 100% conclusive, but it is good evidence. There is more to those stats than just accuracy. But we would, at a minimum, expect some correlation.
It depends on the offense Brian. You mention Hasselbeck but that is precisely the reason Hasselbeck is not a great QB in the west coast system. He gets the ball to the receiver but he doesn't lead the receiver like Montana, Young and Farve. Because of this Hasselbekc has always been average and unable to get the full effect out of the offense. Its a question of system. Some offenses like Indy's don't require much form the QB in regards to leading the receiver. But to say that becasue one team airs it out deep with semi accurate long balls that the receiver goes and gets and another throws the underneath game with extremely accurate passes that lead the receiver to the opening is comparing apples and oranges.
If throwing out YAC is a good idea, might it not make sense to also throw out the attempt occasionally? The hard part lies in determining when an attempt isn't a true pass attempt much like YAC isn't true passing yards.
Maybe McNabb shouldn't even have the one yard and the one attempt. I'd find an examination of air yards distributional breakout interesting to see if there's any clear data division into a bimodal form.
I completely agree that YAC isn't qb related on pseudorunning plays, but is measuring AY and counting attempts meaningful? I'm still debating if YAC on passing plays is meaningful (Wayne's long td from the wc game seems like it should count in full as he would have remained open due to the defensive lapse, and recognizing and throwing to the more open downfield receiver might be a qb skill).
Do a YAC for receivers only, that eliminates most screen passes that usually goes to running backs.