Kurt Warner announced his retirement Friday, and his Hall of Fame credentials have been a topic of conversation. Everyone is familiar with his career: three Super Bowl appearances, one ring, two MVP awards, and a compelling personal story that inspires grocery stock clerks around the country.
According to Pro-Football-Reference, his career Adjusted Net Yards Per Pass Attempt (net passing efficiency minus 45 yards for each interception and with a 20-yard bonus for each TD pass) is 6.7 ANY/A. That's very, very good, and it puts him in league with the era's best passers. But it's not called the Hall of Efficiency; it's the Hall of Fame.
With that in mind, we turn to Win Probability Added (WPA), a narrative stat uniquely suited for measuring performance in terms of how each play helps a team win or lose. Fame is about a lot of things, but it's primarily about wins, so let's see what WPA has to say.
I'll compare Warner to the other top quarterbacks of his era in terms of both WPA and, for perspective, Expected Points Added (EPA). I chose Peyton Manning, Tom Brady, Brett Favre, and Donovan McNabb. The first three are locks for the Hall, but McNabb is probably considered a notch or two below.
Per-play or per-game stats are most appropriate for predicting future performance, but that's not what this discussion is about. I think using career totals is most useful for Hall of Fame discussions, so I'll start with those. Playoff games and Super Bowls are included.
Unfortunately, digital play-by-play data is not available prior to the 2000 season, and perhaps Warner's best year was his first as a starter, in 1999. The other QBs to whom I'm comparing him played in years prior to 2000 as well, so although this analysis is not complete, I still think it's illuminating. Favre, in particular, enjoyed his best seasons prior to 2000. But in a way, Favre's latter career is a very appropriate comparison to Warner's entire career. In essence, Warner only had a second half of his career. His first half was spent in the Arena League, NFL Europe, or down at the Save Mart.
Here are the WPA totals by year (* denotes MVP year):
WPA | K.Warner | P.Manning | T.Brady | B.Favre | D.McNabb |
2000 | 2.1 | 3.6 | 0.0 | -0.2 | 0.5 |
2001 | 3.8* | 0.8 | -0.8 | 1.3 | 0.3 |
2002 | 0.0 | 0.2 | 0.9 | 1.6 | 0.2 |
2003 | -0.7 | 2.4* | 1.6 | 1.1 | 1.1 |
2004 | 0.1 | 6.9* | 3.1 | 4.0 | 2.1 |
2005 | 0.1 | 3.6 | 4.8 | -1.5 | -0.6 |
2006 | -0.5 | 6.9 | 1.4 | -1.2 | 1.4 |
2007 | 1.1 | 4.3 | 6.1* | 2.6 | 0.9 |
2008 | 3.0 | 3.2* | -0.2 | 0.1 | -0.8 |
2009 | 1.8 | 7.6* | 1.8 | 3.0 | 1.4 |
Total | 10.8 | 39.3 | 18.8 | 10.9 | 6.6 |
Warner slots-in below Manning and Brady, but well above McNabb. His career is similar to Favre's performance in the second half of his career. Add in about another 3 or 4 WPA to cover Warner's 1999 career-year, and he's definitely in the conversation for the Hall.
And here is each QB's EPA by year:
EPA | K.Warner | P.Manning | T.Brady | B.Favre | D.McNabb |
2000 | 76.9 | 121.6 | -1.8 | 7.5 | 44.1 |
2001 | 93.8* | 17.2 | 6.1 | 48.2 | -2.7 |
2002 | 0.0 | 71.8 | 1.4 | 40.3 | 16.9 |
2003 | -24.0 | 105.8* | 6.5 | 48.6 | -1.7 |
2004 | 3.3 | 194.3* | 70.6 | 123.3 | 87.9 |
2005 | 27.7 | 138.0 | 126.3 | -9.6 | 15.2 |
2006 | 11.6 | 171.9 | 52.1 | -34.9 | 51.9 |
2007 | 30.7 | 147.5 | 257.7* | 79.5 | 41.2 |
2008 | 82.8 | 135.7* | -7.0 | 34.7 | 51.6 |
2009 | 53.6 | 148.0* | 127.2 | 136.7 | 66.9 |
Total | 356.5 | 1,251.7 | 639.2 | 474.2 | 371.3 |
When we look at both WPA and EPA, we can see some interesting things. For example, Warner and McNabb's EPA numbers, which are less context-dependent than WPA, are about equal. But Warner's WPA numbers are higher, indicating that Warner performed better when it mattered. It's not McNabb's fault he played on teams with great defenses, which is likely why his WPA numbers aren't higher. But then again, shouldn't Warner get credit for carrying his team when they needed it?
Warner had some injury problems, and held the clipboard for a good share of time during his stints in New York and Arizona. If we want to look at per-game numbers for the past decade, here is how Warner compares:
Player | WPA/Game | EPA/Game |
K.Warner | 0.10 | 3.2 |
P.Manning | 0.22 | 7.1 |
T.Brady | 0.13 | 4.3 |
B.Favre | 0.06 | 2.8 |
D.McNabb | 0.04 | 2.4 |
Again, if we could add in 1999, Warner's per-game numbers would improve. In terms of per-game performance, he looks like a stronger candidate for the Hall.
In the end, Warner enjoyed a phenomenal career and is widely admired for his dignity and class. After all, the stats, whether simple yardage totals or WPA, are only part of the picture.
Great analysis. Warner is a Hall of Famer in my opinion. Pretty good career numbers, amazing peak numbers, plus the grocery store to MVP story makes him one of the NFL's best success stories.
In the WPA/EPA numbers, I like how you can see that Brady's 2007 statistics were partly caused by Belichick's running up the score in blowouts. Brady only has the third highest WPA, trailing Manning's 2005 and 2009 by a substantial amount, even though Brady's EPA for 2007 is 60 points higher than the next best season. That's a lot of garbage time touchdowns, which fits pretty well with my recollection of the 2007 Patriots.
And I know it's beating a dead horse, but boy that Peyton Manning guy is pretty good, huh?
Whoops, a mistake: Brady's WPA is fourth highest, trailing Manning's 2004, 2006, and 2009 seasons.
I also bet Warner's 1999 season would be competitive with the above 4 for best season.
Brian, i don't if this is too much to ask, but can you show the regular season and postseason WPA/EPA for these guys. I want to see how they performed in the playoffs. Warner's biggest part of his Hall campaign is his postseason performance. How does that compare to the other guys?
Brian, what is the baseline for WPA and EPA? Does 0 correspond to an average player or a replacement player?
Zero would be average.
This is a case for Warner's HOF credentials, but what it really shows is that the Manning vs Brady debate is pretty much over, as Peyton has doubled Brady's WPA and EPA.
No probowl live graph?
-Pro Bowl graph finally got going.
-I can break out playoff games vs regular season. I was thinking of concocting some formula for over-weighting playoff and Super Bowls in the career WPA totals, but we already know that Warner played lights-out in the playoffs.
A few quick points...
-Every single player to win at least 2 NFL MVP awards is a HOFer.
-Every QB that has played in at least 3 Super Bowls is a HOFer.
-Take a quick look at http://en.wikipedia.org/wiki/NFL_starting_quarterback_playoff_records at glance at the company around Warner for games started and wins.
-Warner holds all the records for yards in Super Bowls... the top 3 single game passing yards and total yards.
-I think he is top 5 all time for career playoff TD's and a bunch of other key metrics for the playoffs.
Also take a look at http://en.wikipedia.org/wiki/Most_career_300-yard_passing_games_by_NFL_quarterbacks
Has an excellent 208 TD to 128 INT ratio ... 1.625. For HOF QB's from 1980 on that's better than Favre, Kelly, Moon, Elway, and Aikman.
He has the 2nd highest completion % of all time.
http://www.pro-football-reference.com/leaders/pass_cmp_perc_career.htm
2nd all time passing yards / game
http://www.pro-football-reference.com/leaders/pass_yds_per_g_career.htm
7th all time yards / attempt
http://www.pro-football-reference.com/leaders/pass_yds_per_att_career.htm
Compare him to Troy Aikman or Jim Kelly.
Kurt Warner is a lock for the Hall of Fame.
Kurt Warner Playoff Stats
13 Games (13th all time)
9 Wins (tied for 8th all time)
4 Losses
QB Rating 102.8 (2nd all time)
Completions 307 / Attempts 462
Completion Percentage 66.5% (1st all time)
Yards per Attempt 8.55 (1st all time)
Passing Yards 3,952 (7th all time)
Touchdowns 31 TDs (4th all time)
Interceptions 14 INTs
I haven't looked back before this time, but his 2.21 TD/INT ratio is the best among all HOF QBs post 1970.
A defense of Brady (and Belichick)...
Comparing talent with WPA across multiple teams seems imperfect. Consider that superman (who scores a touchdown on 100% of his plays) would only be +4.0 on the season if his defense was also full of superheros.
Don't forget that NE's defense was better and their receivers were worse until 2007. Both of those factors contribute to the Manning-Brady disparity in favor of Manning.
To address the 2007 "running up the score" issue I looked into scoring differential by quarter. NE outscored opponents by 196 in the first half compared with 119 in the second. The difference was even stronger if you remove the 5 games decided by fewer than 17 points. +194 for the first half vs. +98 in the second. The 4th quarter was the closest quarter. In my opinion, the idea that Brady was padding numbers against surrendered teams is silly and unsupported by evidence.
I broke it down further and found that of the 64 quarters played in 2007, 22 began with a Patriots lead of 20+ points and 42 began with a closer score.
In the "blowout" quarters (20+ lead), the average subsequent differential was +1.0 pts.
In the "close" quarters (<20 lead), the average subsequent differential was +7.0 pts.
You could make a strong case for the charitable and sympathetic coaching of Belichick. I conclude that the "running up the score" myth is part of the vast conspiracy to hate on the Patriots.
You make a good case. This is just my personal feeling, but I don't believe there is such a thing as running up the score in the NFL. If Notre Dame is playing Army and keeping its starters in the game trying to make a 45-3 game into a 52-3 game, that's running up the score.
In the NFL, you can't dress 100 guys on the sidelines. Second string guys play regularly in tight games, plus have special teams duties. NFL teams are much, much closer than college teams in terms of talent.
I think Belichick took a lot of grief because he was going for it on 4th down in FG range with big leads. His thinking was that he was giving the other team a chance to stop them. I think it's a fair point. Going for the FG in that situation might be more of a case of running up the score than going for it. (Isn't that what teams do when they really are trying to maximize their score?)
There's also a philosophical point. I think there is Platonic value in striving for excellence on every single play. My son's basketball team doesn't have many wins this year, so I tell him, "make the next play your championship. Be the champion of this play, this quarter, this day." Ignore the score. Ignore the records. Just do your best on every opportunity. Championships will follow.
I don't really have a problem with teams running up the score, I just think it makes certain statistics hard to compare when some teams run up the score and others don't. I also don't recall claiming that the 2007 Patriots didn't slow down some late in blowouts, just that they did it less than most people expected. Again, I'm fine with this for professional football players, but we need to down weight Brady's stats because of this.
Another way to look at this is to consider how often 2007 Brady sat at the end of the games compared to, say, 2004 Manning (the year he broke the TD record). Going to the game logs would be the best way to figure this out, but a cursory glance at Pro Football Reference shows that Jim Sorgi attempted 4 times as many passes in 2004 than Matt Cassel attempted in 2007. This fits with what everyone's eyes were telling them: Belichick was leaving Brady in long after most coaches would pull their QB during blowouts. This padded his stats (and was probably the point). This is morally fine for them to do, but we can try to adjust for this in various ways when we do analysis.
One way to control for this to look at QB's effect on game outcome - that's the approach Brian is going for with the WPA numbers. Another is to look at rate stats, for instance using yards per attempt, not just total yards. Looking at PFR's adjusted net yards per attempt, we see that Brady's 2007 was out of this world good, but still almost a WHOLE YARD less than Manning's 2004 year.
Good point, Andy.
It is also worth noting when comparing Manning's 2004 to Brady's 2007, that Manning basically accomplished his stats in 15 games while Brady played the full 16.
Brady's a pretty good QB, but not in Manning's league. I think your advanced stats show this.
I'd tend to agree that Manning is superior, even without looking at the stats. But in Brady's defense, his WPA for 2007 (and several other seasons) would have been higher had the Pats' defense been worse. I think the '07 Pats' pass offense would have carried them to 14 wins that year with an average or even slightly below-average defense. This would have the effect of making games closer, and making Brady's WPA higher.
I will concede that Brady can't stand up to Manning in terms of career numbers, but I will still push for the argument that Brady '07 beats Manning '04 for the greatest passing offense of all time.
My argument here is that WPA is not a good metric for this discussion because it is heavily influenced by team defense (among other things), which we would like to remove from the discussion.
I think that EPA is the best measurement. The most common argument against this is the garbage time padding, which I addressed earlier. I think that even if there exists situations where a teams give up late in blowouts, the Pats were not taking advantage of them.
There's a massive gap of 63 points in favor of Brady's EPA, and even when looking at efficiency, I have Brady at .446 EP/play and Manning at .390EP/play.
Andy made the point I was about to make.
Showing that a team outscored its opponents by more in the first quarter than they did in the fourth doesn't mean they didn't run up the score. Its obvious that any dominant team will score more in the first than the fourth. The question is the extent that they slow down in the fourth. How often do they not score when they have the opportunity to?
One way to look at this would be run/pass balance when WP> 98%. If a team is still throwing the ball when they have a 21 point lead in the fourth, they are running up the score. If they are running it and using 30 seconds on every play, they aren't.
I'm not sure that I agree philosophically that is immoral to run up the score. These are professionals and they should just get over it if they lose by 30 when they would have lost by 20.
I do think that if you look at, you will probably see that there was considerable evidence that the Patriots did run up the score.
I don't like this "running up the score" argument for two reasons. First, we all agree that there's nothing wrong with it. Second, it's definition is vague. I consider it to be any case where WP maximization is sacrificed in exchange for maximizing points. For example, I don't consider passing on 3rd and long to ever be an example of running up the score.
Back to the more fun argument...
The basis of this discussion was the belief that Brady padded his stats in a way that Manning was not allowed.
But the evidence shows that the Patiots (I don't have specific passing data) performed dramatically, and statistically significantly poorer once the lead exceeded 20. This would indicate that the common belief that NE was running over teams that had already checked-out is false. Look over the 4th quarter scores in some of those really bad games. You'll find defensive TDs, 3rd string QB rushing TDs, and TDs by some 4th string RB named Kyle Eckel.
You make a valid point that Brady stayed in longer than Manning would have in similar situations. I agree with this. It likely contributed to Brady's larger number of attempts and likely made the difference in the TD record. But Brady's EP per play was still higher. In other words, he was MORE efficient over a LARGER dataset.
It's telling that the comments in a post about Kurt Warner have focused on Manning vs. Brady, (my own comments included). I guess that's just the reality of playing QB in this era. Manning and Brady are going to overshadow everyone else. There's no way around it.
I think the stats you show make me less inclined to think that Warner is a HOF candidate. If you look at his career rate statistics (ANY/A or QB rating), he is between Brady and Manning. I think 8 years of quartbacking (which is about what Warners career amounted to)at that level would be Hall of Fame worthy.
But your analysis shows that he is actually a notch below elite and played at the "very good" level for the equivalent of 8 years. I think that is too short of a time at that level to be inducted into the Hall of Fame. If he played 4 or 5 more years I would say he deserved to be inducted.
You definitely should compare players in their era. But I feel it would be a mistake to conclude that Warner is not a HOFer just by saying he doesn't stack up to Manning or Brady. They will probably go down among the greatest ever.
Here are the HOF QB's since Joe Montana. I go back that far cause it kind of groups nicely into the same era. No one has been inducted since.
Joe Montana 1979-1994
Dan Marino 1983-1999
John Elway 1983-1998
Warren Moon 1984-2000
Steve Young 1985-1999
Jim Kelly 1986-1996
Troy Aikman 1989-2000
In my opinion, the next crop of HOFers is as follows...
Brett Favre
Kurt Warner
Donovan McNabb
Peyton Manning
Tom Brady
Drew Brees
Ben Roethlisberger
Seriously, for the most part, being a HOF QB comes down to being a great QB and being all up in the mix in the playoffs battling against the other greats of your era.
Just look at the proof. http://en.wikipedia.org/wiki/NFL_starting_quarterback_playoff_records
Sort by playoff games played and playoff games won and look at the HOFers. It's pretty clear.
Again, he was a 2 time MVP. Led his team to 3 Super Bowls and was the Super Bowl MVP of one of them. Those are HUGE accomplishments for the HOF.
Look at his playoff numbers that I posted earlier.
It is remarkable what he accomplished in a somewhat short career.
For the record, a QB on a team of superheroes would end up +8 WPA, because he would take each game from .5 expected WP to 1, and 16*.5 = 8.
Manning's 2004 was better than Brady's 2007 by most rate stats, from simple passer rating to more advanced stuff. I remember when Manning threw 5 TDs against Detroit, he sat after two series in the second half. Contrast that to the Pats/Redskins game in 2007, where Brady threw the TD to make the score 45-0 in the 4th quarter.
I'm not saying this is somehow immoral. As others have said, they're all professionals, and unless the losing team starts running into the line to burn clock, the winning team is under no obligation to step off the gas. But there's little doubt that Belichick was less likely to pull Brady than Dungy was to pull Manning. That's just the empirical reality.
And of course, the rest of Brady's career really doesn't compare at all. The idea that Brady is on Manning's level is really fairly silly, and is only propagated because people want to measure players by rings only - as if defense and special teams (or other offensive players, for that matter) do not exist.
Brady was pretty clearly the second best QB of the era, which is nothing to sniff at. His reputation was pretty overblown at first, but his performance caught up.
Oh yeah... what were we talking about? Warner? Yeah, he's a HoFer. His rate stats, peak performance, and postseason numbers are so exceptional that we can forgive the shorter career and the lost few seasons in the middle.
I have to say that Warner's career needs to be considered in a different context than the other QBs. I don't think he should be "penalized" for having his career start so late. Comparably he shines as potentially the best "yardage" passer in NFL History. No one can even come close to comparing to Warner's 300 yard game percentage. I think the closest is Dan Marino whose percentage is about 20% lower. That is a major statement. Also, compared to the major QBs he also has one of the best ratings and the 2nd best completion percentage in NFL History.
Considering when he did all this (in the years when many QBs start to falter), and the fact that he is arguably the best postseason QB in history (both of the Super Bowls he lost were really faults of the defense), he has to be a Hall of Famer. When comparing what he did with the time he had and how the overall career stats (that don't depend on how many years you've been in, like TDs and yardage) compare, I think there is no question, he will be in the Hall of Fame. I'm sure his character will add a few bonus points too!
Anon,
Just to be clear, he's not the best yardage passer in NFL history. He's #2 in yards per game; Peyton Manning is #1. He's been a bit higher variance, hence the higher percentage of 300 yard games.
#2 is still pretty good; I'm not trying to trash the guy.
Tarr,
The superman argument was meant to illustrate the importance of defense in WPA, a factor which should be irrelevant to this discussion. If Superman played QB on a team with superhero defense, he would split the +8.0 WPA with the defense, so, yes, he would only have +4.0 WPA on the season. This helps illustrate why Brady's 2007 WPA was only +6.1, there simply wasn't anything left to gain because the NE defense took the rest.
Again, I concede that Manning beats Brady with career numbers(most reasonable people do), and by a substantial margin. I was only trying to compare the 2007 Brady to 2004 Manning only.
Main point: Brady had a substantially higher EPA/play over Manning, and did it over a larger number of plays.
Oh, and Kurt Warner is pretty good. I guess.
No, he wouldn't split the WPA with the defense. He would get at least 90% of it. Each TD significantly increases the expected win percentage, but each stop only slightly increases it. Going up 7-0 or 14-0 radically changes the probabilities in the game, while getting a single stop doesn't move things that much.
A superhero defense scores a defensive touchdown on every play.
But, you miss the point. An offense of a 16-0 team will have a +8.0 WPA if the defense is average, and a +4.0 WPA if the defense is equally excellent. Those are two very different numbers for the exact same production.
EPA/play is a more specific measurement. True, it ignores time and score differential. But it does not introduce any unneccessary variance. It's just pure, cold, hard efficiency.
Wrong again. If a superhero defense scored a TD on every play, then they would get a lot more than 50% of the WPA. The offense would get no more than one posession in the entire game, as the rest of the game would just be an unending stream of defensive TDs and touchbacks. The defense would get probably +7 or so WPA.
You are basically right about the "non-superhero" assertion, though: Since there is +8 WPA as the total pie to split, an even team should split things roughly evenly.
However, the conclusion you draw - that these are "two very different numbers for the exact same production", is wrong. If the defense is constantly giving up the lead and forcing the offense to regain it and win out again and again, it actually takes a lot more positive production out of the offense to win the game. This is accurately reflected in the higher WPA.
At the far extreme, there's really no hard limit on the amount of WPA an offense could have. If the offense always scores, and the defense also gives up a TD every time, you could have a game where the offense has a WPA of 3 or 4 for just one game, and the defense is negative 3 or 4.
I agree that EPA's less context-driven numbers are a bit more reliable, all told.
What if it was a team of mini-Ditkas?
Try not to focus too much on the superhero scenario. You're reading too much into it. It was never meant to be an exact calculation. I was using it to help people understand how an offense's WPA is not independent of their defense, particularly for teams that deviate significantly from the mean. I can see that you understand this as well, and it seems you also agree that EPA or EPA/play is a superior metric for this discussion. This is the basis of my argument that Brady 07 beats manning 04.
Brian, a team of mini-Ditkas would actually be able to bend the laws of probability and exceed +8 WPA on a 16-0 season.
Anon, it's very close between Brady '07 and Manning '04. Some rate stats point one way while others point the other. I get .4 EPA/play for Brady and .36 EPA/play for Manning. Pro Football Reference's advanced stats like Net Yards per pass attempt and Adjusted Net Yards per pass attempt come out in favor of Manning. Passer rating comes out in favor of Manning.
Footballoutsiders' comparison is illustrative: DVOA (the rate stat) is higher for Manning, but Effective Yards (the counting stat) is higher for Brady.
At any rate (HA!), it's close.
I found it stunning when last year (when speculations about KW's retirement were high after the SB loss to the Steelers), one of my favorite NFL writers, Peter King, wrote how Warner is not a lock for the Pro-Football HOF, essentially providing no argument other than relatively short career. With one additional year, which has not fundamentally altered his life-long stats, one wonders what will HOF voters do (shades of Florida 2000?). I would have thought that, from the number-crunching standpoint, KW is a lock for HOF, precisely because of his short career - b/c few (if any) have accomplished what he did in essentially less than a decade in NFL (effectively about 7 playing seasons). I guess I am trying to say that the way HOF voting works does not go in step with the sophistication of the times we happen to live in - can anyone imagine statistical analysis savy person born in 1940s, 1950s who chose sports writing as their profession? Basic stats, maybe (mostly baseball driven in US), but analysis of the kind BB nd NS do... no way...
Prior to wk 16 both QBs sat for virtually the same amount of time. The only reason why Brady saw more action was because NE tried to cap off the 16-0 season – which is perfectly reasonable – and the Colts had nothing to play for in their last game. If you compare the other 15 games by the drive chart you’ll see what I’m talking about.
Manning sat: 2 full quarters and 8:50 mins before sitting the last game:
Wk 10 vs. CHI Sorgi came in with 12:42 in the 4th
Wk 11 vs. DET Sorgi came in right 1:15 left in the 3rd
Wk 12 vs. TEN Sorgi came in at 9:53
Brady sat: 2 full quarters and 8:29 mins in before playing the last game:
Wk 2 vs. SD - Gutierrez came in with 1:18 in the 4th
Wk 3 vs. BUF - Cassel came in with 8:15 in the 4th
Wk 4 vs. CIN - Cassel came in with 2:38 in the 4th
Wk 7 vs. MIA - Cassel came in with 11:28 left, Brady returned at 10:22, Gutierrez in at 3:45 in the 4th
Wk 8 vs. WAS - Cassel came in with 8:30 in the 4th
Wk 10 vs. BUF - Cassel came in with 10:57 in the 4th
Wk 15 vs. MIA - Gutierrez came in with 2:00 in the 4th
http://www.coldhardfootballfacts.com/Articles/11_3529_Debate_is_over:_Brady_the_best_in_all_seasons.html
i'm starting to wonder if maybe the "anon" poster so ferociously arguing for Brady above is Giselle? Even supermodels are getting into advanced statistics!
there is no way you can come up with any argument that should keep warner out of the hof.
his overall numbers (despite his shortened career caused by injuries & stupid coaches)....his playoff performancesd....his fantastic play in 3 superbowls.....& the fact that he took at second division club to the superbowl.
add to that his character would be a welcome addition to a hof full of players with checkered pasts. this is an opportunity for the nfl to do some hof upgrading!!