Clearly, Tom Brady is the NFL's biggest choke artist.
That last example is just one way real and armchair analysts alike can place selective focus on certain facts to create a skewed perception. Perhaps no player has had more damage done to his reputation in this manner than Tony Romo. Romo played the game of his life against the Broncos last Sunday before a single ill-timed mistake reignited accusations of "choking." As Grantland's Bill Barnwell put it, Romo did not have a perfect game, but rather the "perfect Tony Romo game."
But what does that even mean? Most people know that Romo is not really as bad as his choker reputation implies. More sensible fans may even understand that Romo receives far too much blame for the organizational and team-wide failures of the Cowboys.
However, do people know that Tony Romo may actually be the most clutch quarterback in the league, or at least very close to it? To answer this, let's delve into the murky depths of "clutchness" through a couple different lens.
Clutch By Metrics
In defense of Romo this week, many have floated out the fact that he led the league in fourth-quarter comebacks last season, with six. However, that stat is far too circumstantial to truly justify calling Romo a "clutch" quarterback. As a competitive but sub-elite team, the Cowboys are more likely than most to play close games against a wide range of teams, giving Romo more chances to orchestrate comebacks than if he had played with a stronger supporting cast.
One stat we can use to measure situational performance is Win Probability Added (WPA). The league's best quarterbacks will always top the WPA charts because they consistently produce positive plays; however, because WPA is so context-dependent, it gives some insight into who performs better in the most important circumstances. Since Romo became a regular starter in 2006, here's how he compares with some notable quarterbacks in WPA:
No huge surprises yet. Romo mostly falls into the middle of the pack here, below the highest standards of Peyton and Brees but in the tier below them with Roethlisberger, Rivers, etc. (just as a side note, you can't see Big Ben's name on the graph, but his is the dark gray line between Romo and Eli). However, instead of simply examining WPA in a vacuum, it's more helpful to juxtapose the stat with Expected Points Added (EPA). EPA measures total production rather than context-dependent production, accounting for how much a given play adds to a team's chances of scoring on their drive.
Put in layman's terms, a 20-yard pass on 1st-and-10 from midfield affects EPA the same in both the first and fourth quarter, but it would add more WPA in the fourth quarter because of context. So if we're defining clutch based on a WPA vs. EPA graph, we would expect the most clutch quarterbacks to fall above the regression line. That is, those above the line put up a WPA greater than expected for the EPA they compiled over the course of the season. I can feel your eyes starting to gloss over, so take a look at this convenient graph from 2012:
Yes, that star behind Andrew Luck indicates that Romo and the Colts rookie both exceeded their expected WPA more than any other quarterback in the league. This allows us to validate those much-ballyhooed fourth-quarter comebacks, as it illustrates how Romo excelled in late game situations throughout the season.
That is not a one-year fluke either. In fact, Romo has consistently met or exceeded his expected WPA, reflecting a career of solid performance in tight situations:
Excluding a 2010 season in which he played only six games before a broken clavicle ended his year, Romo has failed to meet the WPA threshhold just twice. And his two most recent seasons, both of which ended in the Cowboys being eliminated in the last game of the year, have actually been his two most "clutch."
Clutch By Situation
OK, but those numbers still don't provide raw stats in so-called clutch circumstances, but rather a macro-scale overview of Romo's performance across all four quarters of many games. Against Denver, for instance, Romo's 0.81 WPA was the third-highest mark of the week, but critics can still point to his interception as evidence of late-game failure.
Thus, it may be instructive to dive into Romo's statistics under various late-game parameters. In this manner, we can see whether Romo's stats actually decline in tight situations, or if fans selectively magnify a tiny percentage of failures to forge a consensus. All stats in this section are courtesy of Pro-Football-Reference's invaluable Game Play Finder.
Well, in the fourth quarter of one-possession games since 2006, Romo's 99.3 quarterback rating is third-highest among quarterbacks with at least 100 passes in those situations, trailing only Peyton Manning and Aaron Rodgers. Romo has also thrown 534 passes in these situations, a massive sample size that roughly amounts to an entire season. For sake of comparison, let's take Romo's clutch-time stats and place them side-by-side a couple of other notable seasons:
Player A, as you might have guessed, was Clutch-Time Romo. Player B was 2009 Peyton Manning, who threw 571 passes, and Player C was 1995 Brett Favre, who threw 570 passes. Both players won the MVP in their respective seasons. With the exception of '09 Manning completing a higher percentage of passes and '95 Favre throwing a few more touchdowns, there is evidence that Romo has essentially compiled an MVP-caliber resume (!) in one-possession fourth-quarter situations.
If that's not enough, let's examine Romo's performance when tied or trailing by one possession in the fourth quarter. Theoretically, Romo should be pressing more in these situations in attempting to gain a lead or erase a deficit. Perhaps this is when his turnover bugaboo appears.
Nope, try again. Romo has completed 62.9 percent of his passes with a 7.8 yards/attempt and 93.0 quarterback rating in these situations. He has also thrown 20 touchdowns to 10 interceptions under those guidelines. Those are slight but not enormously significant dips from his overall one-possession fourth-quarter stats.
To elucidate why Romo does draw so much wrath and ridicule, let's put on our "First Take" hats and magnify the times where he does perform somewhat poorly:
Player A illustrates Romo's one-possession fourth-quarter stats of games the Cowboys ended up losing. Those are certainly sub-par numbers, but nowhere near as bad as Players B and C in similar circumstances. In case you were wondering, B is Ben Roethlisberger, and C is Drew Brees.
Putting aside the advanced notion that quarterbacks tend to play worse in games their teams lose, there is one overriding commonality with several of Romo's losses: the timing.
Bottom Line
Playing on a team annually on the fringes of playoff contention, Romo plays more crucial regular-season games than nearly any other quarterback in the league. And the sad truth is, many of his worst moments have come at the worst times possible. Think about the Romo career lowlights:
- The botched snap against the Seahawks (not even a quarterback play!)
- Interception to seal a loss to the Giants in the '08 Divisional Round
- Three turnovers in a Week 17 eliminating loss to the Eagles
- Two interceptions in a Week 17 eliminating loss to the Giants
- A late-game pick in a Week 17 eliminating loss (sensing a theme here?) to the Redskins
Those last three are the most recent, and are all crucial errors in do-or-die regular season games. The last two were the annual NBC Sunday Night finales, meaning that Romo's mistakes were on national TV for millions to make snapshot judgments. Here are stat lines and game-costing errors a different quarterback made in the course of just a single season:
- 20-of-39 with two interceptions, including a pick-six
- Two picks, including one that leads to a touchdown three plays later
- 25-of-50, no touchdowns as his team gets steamrolled 44-17
Those are all performances by Peyton Manning from three Colts losses during the 2006 season. But no matter, because those were inconsequential mid-season games in a season where Indy would eventually win the Super Bowl. No one remembers those select games where an off day from Manning contributed to a loss.
But unfortunately for Romo, the Cowboys have had very little room for error during his tenure. Consequently, rare critical mistakes by Romo often mark the difference between the postseason and going home early. Manning's mistakes simply meant the Colts lost a first-round bye and had to play an extra game on their way to a championship.
It may seem counterintuitive, but Romo actually produces more wins than all but a select few quarterbacks in the league. Check out this graph comparing Romo's career WPA to that of his contemporaries:
This debunks the notion that Romo is the reason for Dallas' constant mediocrity, and that most other quarterbacks would have propelled the Cowboys over the top. Yes, perhaps Peyton or Brady would have, but those are also two of the top-5 to 10 quarterbacks of all time.
No one should sweep Romo's aforementioned gaffes under the rug, because they were undeniably crushing errors in big situations. But he is far from the only quarterback to commit those errors, and the evidence shows that not only is Romo an excellent quarterback overall, he is also generally excellent in "clutch" circumstances. With an aging roster and a tight cap situation, the Cowboys' window of contention may be closed. Just don't blame Tony Romo for the entire Dallas organization's underachievement.
It may be an oversimplification, but I really believe that most of Romo's public (mis)perception can be traced back to that single botched snap against Seattle. Up to that point, I think the public verdict on Romo was rather neutral. But after that snap, "analysts" and everyday fans have seized upon every one of his missteps, no matter how many brilliant plays they are couched in.
Romo definitely gets a bad rap. Reminds me of the Jerry character from Parks & Rec. Here are a couple ways to visualize DAL's woes.
1- Here is the Def and Off Team EPA chart for DAL. Notice in the Romo era, DAL is always in the right-bottom quadrant, meaning good offense, poor defense.
2-And here is the QB career viz Sterling included above. Compare Romo to anyone on the career WPA page or the Nth Best Season page.
Using Peyton Manning as a standard is not necessarily helpful since the rap on P. Manning is that he also doesn't play well in the most critical games (i.e. the playoffs).
You're a choker until you aren't.
Anyone remember what they used to say about Lebron James?
Where can I find play-by-play WPA data?
The notion that Peyton is unclutch in the playoffs is also wildly false, even more so than Romo. Barnwell went on a rant about this last year, and the stats say that Manning is actually exceeds Brady in the playoffs in pretty much everything besides wins. Also foolish to make sweeping judgments of player ability based on total team results.
It also doesn't help Romo that he just plays on a team with an inordinate amount of games in primetime matchups with gigantic viewing audiences, so that every generic Week 7 non-division matchup in which he throws an interception is declared a "big game" simply because it's on Sunday Night announced by Al Michaels rather than the 11 am Sunday kickoff announced by Kevin Harlan and Solomon Wilcots
Great article. I had read somewhere that Romo had the 3rd most INTs that resulted in at least a 20% decline in WPA from that single play. Is that true and might it be a stat that would quantify the perception that Romo has made mistakes that directly costs games more so than other QBs?
I think Romo has been amazing, but this statistic gave me some pause.
Certainly interesting and sophisticated data, but as a lifelong Cowboys fan (since late 60's), I could have told you that Tony Romo isn't the Cowboys' problem. They have been on a roller coaster since Jimmy Johnson left, mostly down, but just simply not championship caliber. There are multiple reasons, but simply put, we suffer from the Good Twin/Evil Twin syndrome that is Jerry Jones/Owner, and Jerry Jones/G.M. There is really is no one I'd rather have as an owner than JJ, but there is no one worse than G.M. Jones. As many have noted, Jerry would have fired himself many times over for the job he's done as GM, but to me, it goes deeper than that. He has allowed a culture of "mediocrity is okay" to permeate the atmosphere, and until he realizes the smell is coming from nowhere else but him, the Cowboys will remain mired in that netherworld of NFL hell: Not good enough to make it to the Big Show, not bad enough to rebuild through the draft. It's so predictable, it (almost) makes me want to start playing golf again on Sundays, just to lower my frustration level.
Idea:
can we simply control EPA for how competitive the game is?
distance from 0.5 WPA? That way, if a player has a large EPA play when the game is at 50% win probability, he will get full credit for the EPA. if WPA is at 0 or 1, he gets zero credit. if its at 75% or 25%, he gets half credit.
Perhaps, an average of WPA before and WPA after the play?
THIS SHOULD WORK
The best example of America's Tony Romo Derangement Syndrome is Rodney Harrison claiming he had scouting reports dating back to when he played with the Chargers that said Tony Romo melts down late in games. Which was an interesting claim, considering Tony Romo was playing at Eastern Illinois when Harrison was last playing for the Chargers.
Confirmation bias.
People notice and remember events that reinforce their existing beliefs.
That is what Romo is the victim of.
The handful of games that people remember and point to are a very small sample size and probably have more to do with pure randomness than anything else, which plays a much larger role in the NFL than most fans recognize.
@scottmaui - you're exactly right. After hearing all the talk about Romo, etc., after the game, I did what Sterling did and checked out PFR's Game Finder. I searched for late 4th quarter situations when the QB's team is down by a TD or less, passing plays only...I'm looking at the results thinking, "Why don't I remember Drew Brees or Eli Manning throwing all these game ending picks?" Must be the rings...
"Certainly interesting and sophisticated data, but as a lifelong Cowboys fan (since late 60's), I could have told you that Tony Romo isn't the Cowboys' problem."
I could also tell you from watching every play in his career -- and most of the plays in every other QBs' career in that time -- that he is not the problem, too.
But a lot of fans don't pay constant attention to every play, they just pay attention to the things they remember. Or they only see highlights and results, or listen to media memes, and buy in to how that fuels their perception. Which is poorly, btw. And when we tell them otherwise, it's just our word against theirs.
That's why data is useful. Thanks a ton for this article, ANS.
Their problem is Jerry Jones playing GM.
and if you judge a QB because of what he did in the past especially one of his first years with Dallas (Seahawk Snap) then you are delusional Romo is a great QB, he has had a little bad luck in the past, but with some certain individuals Defense and Offense, mainly defense they need to step it up and not blow Romos lead all the time to where he has all the preeure to score last minute all the time, defense doesn't get pressure at the end cause ultimately it comes down to the offense to score the game winner when defense lets it get that close, so In reality the defense blows the game and its up to Romo and the Offense to carry the wjole team to the end and when you do that to a QB for 11 years shit is going to happen..... this year is different though..... Go Cowboys!!!!!
Statistics don't make a QB or any other player in the NFL. A team is a team and looking at individuals doesn't really give you a fair or realistic view of that player. It's like the A student who can't write a decent paper or successfully play a game of Jeopardy. Sure, their grades show that their smart, but are they really?
I have always been a Cowboys fan and always will be. As the old saying goes, "People love to hate the Cowboys." They don't need a reason. They just do what everyone else is doing like helpless and mindless sheep. Is Romo the best QB? No. Is Romo the worst QB? No. Statistics show that he normally plays great games consistently, but people always remember that one bad play. People only do those things to him. If you say bad things about Brees, Brady, or the Mannings, fans will rage like 5 year olds. Does anyone remember the Saints-Falcons game last year where Brees threw 5 interceptions and 0 touchdowns? No, because "He just had a bad game" or "Everyone makes mistakes" or You're just hating on the Saints." If you are going to accuse one QB of botching a game but not another for doing an even worse job, then shame on you. The facts are that Romo is a great QB, and he makes mistakes. It just so happens that when he makes them, the team just kinda gives up. I agree with
@0d4bd760-32a8-11e3-a5e0-000f20980440 that Jerry Jones is a horrible GM. Great owner, but cannot run a football team right. And it goes into to so many other problems with the organization as a whole. I think what people want is another Troy Aikman or Roger Staubach. But statistically, Romo is the best Cowboys QB ever. But then again, what matters in the end is whether or not you make it to the big dance. Like @Doctorjorts said: You're a choker until you aren't. I wouldn't say a choker, but people are going to bash Romo until the Cowboys finally make it to the Superbowl. Until then, the bias and ignorance will not end. Sorry for the rant, but had to get that out there.
Romo is NOT the problem. In fact he is the best player on the team and usually carries the team on his back. He is the reason they are ever in positions to win games. He's a baller with eyes in the back of his head... if we had a defense that could protect our lead, we would have a couple rings with Romo. I would take Romo over ever QB not named Brady or Brees or Peyton. Nobody has more heart than Romo; playing through injuries that most would go on IR on...