QB Passer Rating

Building on my previous efforts to devise a better passer rating, and on my analysis of Air Yards, I've created a more complete passer rating formula.

Most fans are familiar with the NFL passer rating, and all its flaws. The formula itself is almost too complex to type out, so I won't bother, but you can find it here. Its shortcomings are also almost too long to list. But here are a few I've mentioned before:

It is incomplete because it does not consider sacks.

It is redundant because it includes both completion percentage and yards per attempt. Because yards per attempt is strongly dependent on completion percentage, it is double counted in the formula.

It is arbitrary because each of the four components are not weighted in any meaningful way. The components of the formula are based on multipliers and constants selected to give the rating a nice scale rather than based on their importance to scoring or winning. They also use arbitrary maximums.

Lastly, it includes touchdown passes. They should not be included in a passer rating because they are the result of many other factors in addition to QB passing proficiency. Factors such as a great defense that produces turnovers in an opponent's territory, a solid running offense that sustains drives, or spectacular recievers who generate large amounts of YAC are strong contributors to passing TDs. Further, TD passes are the culmination of all the other attributes of the passer including accuracy, avoiding interceptions, and avoiding sacks.

This improved QB rating system is primarily based on Air Yards per Attempt. Air Yards is what I call the yardage gained by the pass through the air. It's essentially total passing yards minus YAC. I've learned that YAC is largely independent of passer ability and it should be credited to the receiver more than to the QB.

Sack yards per pass attempt are also included. Although partially dependent on pass protection, sacks are also dependent on a QB's pocket presence, ability to read open receivers, decision-making, and mobility. Pass attempts are defined as throws and sacks.

Interceptions are obviously crucial to a passer's performance as well. I include interception rate (INTs/Att) in the formula.

The components of the new passer rating are weighted according to how important they are in terms of team wins. The formula is based on a multivariate regression model of team wins. Using data from the past five NFL regular seasons, the regression model estimates team wins based on the efficiency stats of each team including passing, running, turnovers, and penalties. Regression models can hold all other factors equal, so by only adjusting the factors of interest (passing performance) we can calculate the effect on the estimate of season wins. Arbitrary weighting is not necessary.

1. Is not arbitrary. Each component is weighted exactly as much as their relative importance to winning games. These weights are derived from a regression model using data from all teams since the 2002 expansion.

2. The result is stated in units of team wins over a 16-game regular season. The regression model allows the passer rating model's component weights to translate directly into how many additional wins a QB's performance would yield, on average, over 16 full games.

3. Is not redundant. The components do not double count passing stats.

4. Includes only the passing stats primarily controlled by the QB. Factors such as passing yards after catch are not included.

After some quick algebra to simplify the equation, the resulting formula of the improved new passer rating is:

QB Wins Added = [(Air Yds - Sack Yds) * 1.56 - INTs * 50.5 ] / Pass Attempts - 3

Basically, every additional yard of passing/sack efficiency yields an additional 1.56 wins on average. That is, assuming an average running game, and an average defense, a team whose passing efficiency is 1 yard/att above average will win 9.56 games.

The average interception rate is about 0.03 interceptions per attempt. So if a QB throws 0.04 INTs/Att, he'll cost his team 0.01 * 50.5 = 0.5 wins, all other factors being equal.

I subtract 3 because 3.01 was the average score of a QB in 2006. Better than average QBs have positive wins added, and vice versa.

The table below ranks the QBs of 2006 in Wins Added:





































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































PlayerCmp PctYdsAYYACYPAAY/ CmpYAC/ CmpAY/ AttTAY/ AttInt%SkYd RateYAC%+Wins
Manning P654397288915087.98.04.25.24.91.60.15343.88
Romo652903185010538.68.44.85.54.83.90.35362.61
Huard61187810458337.77.15.64.33.60.40.41442.46
Palmer624035250315327.87.74.74.84.12.50.42382.14
Brees644418233020888.06.55.94.23.92.00.18472.10
Garcia6213097096007.06.15.23.83.41.10.21461.88
Rivers623388195314357.46.95.14.23.72.00.30421.84
Bulger634301235519467.36.45.34.03.11.40.57451.21
McNabb572647125013978.46.97.84.03.31.90.42531.21
McNair633050172213286.55.84.53.73.42.60.17441.03
Leinart57254715539946.87.34.64.13.53.20.40390.89
Warner6413777156628.26.66.14.33.42.90.57480.80
Brady623529180117286.85.65.43.53.02.30.32490.53
Pennington653352186714856.96.04.73.83.33.30.33440.49
Hasselbeck57244216298136.67.83.94.43.54.00.57330.38
Grossman553193187713166.77.25.03.93.54.20.28410.33
Delhomme612805144513606.55.55.23.42.82.60.37480.13
Manning E583244186213826.26.24.63.63.13.40.34430.06
Brunell62178973810516.94.66.52.82.41.50.3459-0.05
Vick53247415719036.47.74.44.02.93.40.7036-0.10
Losman623051170513467.16.45.04.02.93.30.7044-0.13
Kitna624208237718317.16.44.94.03.03.70.5944-0.13
Garrard6017359028337.26.25.73.73.03.70.4648-0.18
Young52219912379626.26.75.23.52.93.60.3444-0.29
Roethlisberger603513197415397.57.15.54.23.34.90.5444-0.32
Leftwich5911595326276.34.95.82.92.52.70.2554-0.39
Favre563885176421216.35.16.22.92.62.90.2155-0.45
Harrington57223612509865.85.64.43.22.83.90.2944-0.54
Green6113427815616.86.54.63.92.94.30.5742-0.58
Cutler5910014835187.36.06.43.52.73.50.5741-0.62
Plummer55199410589366.36.05.33.32.84.10.3347-0.64
Smith582890140514856.55.55.83.22.53.60.4251-0.87
Johnson622750133114196.34.95.33.02.43.40.4352-0.94
Carr682767119815696.34.05.22.72.02.70.5057-1.26
Frye642454127511796.35.14.73.32.34.30.6048-1.55
Campbell5312974058926.33.78.12.01.62.80.2645-1.86

AY = Air Yards
YAC = Yards after Catch
YPA = Yards per Attempt (sacks not included)
TAY/ATT = 'True' Air Yards per Att (Includes sack data)
SkYdRate = Sack Yards per Attempt (Attempts include sacks)
YAC% = Percentage of passing yards obtained from YAC
+Wins = Wins Added by QB over a 16-game season


Manning tops the list, which should be a surprise to no one. But Romo's numbers are very impressive. Manning beats him by virtue of his low sack and interception rates.

Campbell's numbers are bad news for Redskins fans. Not only is his completion percentage very low, he's also throwing very short passes. His average completion only goes 3.7 yds down field.

Notice how much better Huard fared than Green. They had the same team around them, but Huard distinguished himself thanks to his very low interception rate and his above average down field passing ability.

Perhaps Vick's passing ability will be missed in Atlanta after all. When he does hit his receiver, it's usually for a good chunck of yards.

Rivers appears to be genuinely talented and was not relying on the talent around him to pound out YAC. He was successfully throwing deep in 2006.

There are many more observations that can be made, but if you're like me you look right for where your favorite QB ranks, and then decide if you buy the formula. Next, I'll expand it further to include fumbles and rushing yards.

  • Spread The Love
  • Digg This Post
  • Tweet This Post
  • Stumble This Post
  • Submit This Post To Delicious
  • Submit This Post To Reddit
  • Submit This Post To Mixx

11 Responses to “QB Passer Rating”

  1. Tarr says:

    I don't think you should report that number as wins, as this is a "rate stat" as oppose to a "counting stat". After all, a guy who throws only a few passes isn't worth multiple wins, even if those passes are all completed bombs.

    Even converting it to a winning percentage implies that the QB is making a league-average number of attempts per game. I'm pretty sure that Brees and Palmer were worth more wins to their teams than Romo and Huard were.

    That said, I really like the idea that (Yards - YAC - Sack Yards)/Attempt, along with picks, can give us a pretty solid picture of what a QB is doing.

  2. Brian Burke says:

    Tarr-You're right about the rate stat/cumulative stat issue, but I disagree about the number of attempts.

    It might be better named +Wins/16 or Wins per 16. In other words, its the number of wins above average a QB means to his team with his skills and abilities.

    Regarding attempts: Losing teams tend to have very large numbers of attempts, and gobs of passing yards. It's very misleading to see someone like Kitna with huge fantasy numbers, because he's not that good. He's just on a bad team with a poor defense.

    It's a bit counter-intuitive. The better the quarterback's skills, the fewer pass attempts need to be made.

    The stat not really converted to a winning percentage. It's based on a regression of team efficiency factors. It's just that the resulting units are in team wins.

    So I'm not going after QB production, but QB ability, separated from the game situations and from the offensive scheme he was in. If I were a GM and picking a real QB for my team, and not a fantasy QB, who would I want?

    Thanks for reading and all your great ideas.

  3. Tarr says:

    Wow, was Kitna really third in attempts last year? Crazy.

    That said, all that multiplying his numbers by attemps does is make his bad production look even worse. Guys like Garrard and Young (who were worse on a per-attempt basis) actually did much less damage to their teams through below-average passing than Kitna did, because they didn't drop back as often.

    As a GM, I'd rather have Palmer or Brees for a full season, than Romo or Huard for half. (Of course, Romo and Huard weren't really in control of only playing half of a season, but that's not really my point. The point is...) All those guys are producing at a high level, and as a team I want to get as much of that high production as is possible. A full season of Brees beats the snot out of a rotation of Huard/Green. (Incidentally, where does Bledsoe fit on the list?)

    Just to be clear - I agree that there's absolutely nothing wrong with looking at Air Y/A, Sack Y/A, and INT/A, and forming a stat based around that. And I agree that doing a regression that ties your weights to winning is a really clever approach, and is far less arbitrary than the NFL's passing efficiency stat.

    But takingh that stat and calling it "winning percentage added" seems to assume that that QB gets a fairly league-average number of attempts in those games. The regression that produces those weights implicitly takes into account the average amount of passes that get thrown in games.

    My instinct is that a "wins added" stat along these lines would weight FOUR factors: air yards, sack yards, INTs, and incompletions.

  4. Tarr says:

    Funny side note: the QBs of my two favorite teams bookend this list. Should I laugh, or cry?

  5. Brian Burke says:

    Campbell looked good Saturday until he got hurt. It looked scary, just like Palmer a couple years ago. But he should be ok.

    The more I dissect QB stats, the better Manning looks. I hate the guy, but he's probably the best ever. And that's coming from a kid who went to mass with John Unitas every week. I wish Peyton would get involved in a dog fighting ring or beat up a stripper or something.

  6. D Str8s says:

    I feel to truly get a pure quarterback passer rating the rating should take into account receiver error. The passes dropped by a receiver should factor out of the quarterbacks incompletion numbers. This would make a more even picture on how the quarterback is performing versus how the receivers are performing. I mean if you have a great receiving core that halls in everything you throw there way then that is due to great receivers, not how well the quarterback is performing.

    For example the 92 washington redskins. Mark Rypien looked like a god, but it was his receiving core that shined not him. They snaged some throws that more resembled a water fowl than a football. You could actually see the wings.

    I digres... if this little change would be implimented this would change the outcome tremendously and really show which quarterbacks really do have the right stuff and which ones are just lucky enough to be on the right team.

  7. Brian Burke says:

    I agree that receiver catching ability should be included in a perfect QB rating.

    A couple problems though. First, I'd imagine we'd use reciever drop rates to represent catching ability. Drops are published by receiver, not by thrower. So it would take quite a bit of work to match drops to who threw the ball for any team that did not have 1 QB throw 100% of the passes in a season. Drops are also (sometimes) subjective.

    Second, one of my goals with the formula is for it to be simple. The more we account for, the more complex it gets.

    But like I said, I do agree with you. We just have to balance the perfection of the formula with practical considerations and accept that any formula will have its flaws. Hopefully there aren't too many Rypiens out there to expose its weaknesses.

  8. KiranR says:

    Brian,

    An absolutely great post! Also, you have a truly remarkable site, with a lot of rich content. Truly great stuff!

    I am undertaking a very ambitious project (for me, anyway) - I am writing a book. The topic: The best passer in NFL History.

    Problem: How to define what a great passer is. Obviously, I don't think that the current QB Rating system is a good measure.

    I have done some research, and formulated my own ideas, and, it just so happens, that your ideas re: the current QB Rating system are very much aligned with my own thinking - i.e that it is incomplete, redundant, and arbitrary. I could not have said it better myself.

    You seem to have introduced a new concept - "Air Yards". I will not be using this concept, simply because, when looking back in history, this data isn't available. My preliminary work indicates that by using Completions/Attempt, I can eliminate the need for a concept such as "Air Yards". This isn't to suggest that "Air Yards" is not a good concept. It is merely to take into account that this data is unavailable over the course of history.

    Also, a priori, I am not a big fan of tying a QB Rating to wins, as you have done. This is simply because there are many factors that contribute to wins and losses (as you have pointed out on numerous occasions on this site), and the quarterback isn't in control of a lot of those situations/variables. Unlike baseball, or basketball, the same players do not play both offense and defense.

    I agree with your assessment that INTs are important.

    That all being said, however, your site is very rich in content, I plan to read it from beginning to end over the next several weeks! You may see me sending you a few comments.

    I just launched a blog of my own - newqbrating.blogspot.com

    For now, it only has a few tables. Over the coming weeks/months, I will be adding to it regularly, and introducing lots of data re: the current QB rating system, and, eventually introducing a new QB Rating system.

    My own background is in mathematics/statistics (I have a BA in Applied Math from UC Berkeley), and I spent several years in the actuarial profession. Sports has always been my true love, and I am finally willing to dedicate some time to marry my love of sports and passion for #s.

    My goal (with my blog first, and book later) is to create a QB Rating system that is meaningful, easily explainable, easily understood, simple, and that can be used to compare QBs over time.

    I am hopeful that over the next several months, that you and I will start a great dialogue regarding this subject.

    Best Regards,
    Kiran

  9. Brian Burke says:

    Kiran-Glad you found the site interesting. I recommend Hidden Game of Football if you haven't already read it. You can find an old used copy on Amazon. Get the original edition.

    I'll be certain to check out your site.

  10. KiranR says:

    Brian,

    I just bought Hidden Game of Football and Hidden Game of Baseball. Thanks for the recommendation though - it confirms what I thought was a good decision!

    Cheers,
    Kiran

  11. Anonymous says:

    Great work Brian, I love the site.
    But I have a few comments. I think that YAC is not completely independent of passer ability. The passer must decide who he thinks will get him the most YAC every play, not just throw it to the first guy he sees open. Plus, the better the pass, the better chance the receiver has to gain some YAC.
    I am a Steelers fan so I watch Roethlisberger more than any other QB and he has had problems with the offensive line. A lot of the sacks he gives up aren't his fault. Yeah, I know he holds on to the ball longer but he still has the worst O-Line in the league.
    I definitely agree with taking out the TDs from the equation. But what about game winning drives? I know that would be tough to incorporate but Roethlisberger had about eight game winning drives this past season. I would take Manning or Brady during the first three quarters, but Roethlisberger is the man in the clutch.

Leave a Reply

Note: Only a member of this blog may post a comment.