Today I've got another post up at the Washington Post's Redskins Insider site. This time I look at DeAngelo Hall's single-handed performance last Sunday against the Bears. Some other Redskins nuggets include:
-Just how hard did the Redskins offense try to lose Sunday's game?
-Lovie Smith's big replay blunder
-Albert Haynesworth's impact
-Former Redskins making waves in the league
-Why the Redskins have played in so many close games this season
-How badly the loss of Tony Romo hurts Dallas
- Home
- washington post
- Washington Post: Can One Player Single-Handedly Win a Game?
Washington Post: Can One Player Single-Handedly Win a Game?
By
Brian Burke
published on 10/27/2010
in
team analysis,
washington post
Subscribe to:
Post Comments (Atom)
Didn't know where else to post this...it's really off topic, but...
Cliff Lee's postseason stats are regressing to the mean. :)
Those people at the site don't show any respect...
"Losing quarterback Tony Romo to a broken collarbone is going to replace a passer with a career 7.4 net YPA with one with 5.0 net YPA. We can't say for certain how Jon Kitna will perform in relief, but that difference of 2.4 YPA would take a team that is theoretically average in every other category from an 11-win team to a 5-win team over the course of a full season. Dallas is already buried in last place in the NFC East at 1-5, and things aren't looking good for Cowboys fans."
This is the most ridiculous thing I've ever read. I once put together a spreadsheet of all all the seasons over the last 30 years where the starting QB missed at least 3 games. Then I compared the win pct of the backups to the starters. I didn't use just any starters - I used a list of the top 50 of all time from some site.
The result was that the starters had a winning pct that would result in about 1 more win a year.
Look it up for yourself - you will see that starters don't come even close to adding 2 games in a season let alone 6.
THAT'S the most ridiculous thing I've ever read. Look at the difference between a healthy Favre and a hurt Favre. Look at the difference between Brady (16-0, all-time records) and Cassel (11-5). How are the Cardinals doing without Warner? You don't think the Colts or the Saints would drop a ton of games without Manning or Brees?
I bet your problem is you self-biased by picking quarterbacks with good teams instead of choosing the best quarterbacks. For instance, there's a big difference between the Cardinals with and without Warner compared to the Steelers with and without Roethlisberger. I bet Terry Bradshaw made that list, and biasing it for different reasons I bet Montana/Young did as well.
Did all the starters in your spreadsheet have a career 7.3 net YPA?
I once put a spreadsheet together of all the dumbest comments on this site. Now I'm going to have to add another one.
Firstly let me apologize for being so rude.
I simply chose any qb in the last 30 years or so who was listed in the top 50 or 100 of all time (I can't remember the list - I think it was from pro football reference)
the qbs included Steve Young, Joe Montana, John Elway, Donavon McNabb,
Rest assured they were all star levels QB's.
in some cases the backups replacing them were very good QB's like Jeff Garcia but it was littered with names like Elvis Grbac.
I honestly can't find the spreadsheet - which means I'll have to re create it. But go ahead and do it yourself if you like, you will find the same thing.
I am confident that there are many people better than me to remove bias from such a study. But I am also confident that any such study will show that QB's effect on winning games is not nearly as much as people think.
Certainly no where near 6 games in a 16 game season.
Just created a quick list - used top 100 qbs from Pro Football Reference. Used qb's that played 1980s on.
Used seasons where starting qb's missed at least 3 games. Took out 1987 because of the scab teams
Starters 237-205..pct .537 (8.59 wins/16)
Backups 151-156..pct .498 (7.97 wins)
Starters:
Montana, Young, Elway, McNabb, Hasselbeck, Aikman, Marino, Warner, Garcia, Esiason, Green, Brees, Moon, Kelly, McNair
Backups
Kemp, Moroski, Young, Krieg, Grbac, Stenstrom, Karcher, Maddox, Brister, Feeley, Detmer, McMahon, Garcia, Dilfer, Wallace, Frye, Buerline, Garrett, Cunningham, Wright, Deberg, Mitchell, Huard, Bulger, Martin, Covington, Manning, McCown, Holcomb, Harrington, Griese, Graham, Blake, Huard, Lemon, Beck, Flutie, Carlson, B.Johnson, Kitna, Friesz, O'Donnell, Volek, Boller, T.Smith, Reich
Very good. Except in this case, we know the records of the relevant QBs.
Brian,
Probably not a good idea to insult your readers. You're not the only football stats site on the internet.
RE: I once put a spreadsheet together of all the dumbest comments on this site. Now I'm going to have to add another one.
in fairness I did insult him first. Which is not really called for.
jim maron
To anonymous who posted several times:
#1--In the comment posted above, you left out Brady/Cassel, & Romo/Brad Johnson. (Unless you are only using backups that appear in the top 100.--But then how would some of the nobody's appear in your study?)
#2--I think Brian's point is this: to show the dropoff for the Cowboys, you need to ONLY use those that have a severe dropoff in YPA--say at least 1.5 to include more players in your sample. I mean, Steve Young is a HOFer--using him pollutes your study SEVERELY! Kitna is nowhere near that level.
#3--Referencing Romo to Brad Johnson 2 years ago, you want to SERIOUSLY suggest that he wasn't horrible?? They barely edged Tampa, and got whipped by the RAMS of all teams, and then the Giants! If Romo is there and they go 2-1 instead of 1-2, that's 1 game out of 3, or 5.3 out of 16. That sounds seriously close to 6.
#4--In their 2009 Almanac, Football Outsiders compared backups DVOA to their starters, where the backups threw at least 75 passes. This study, going back to '94, showed that Johnson to Romo was the 3rd largest dropoff ever! Now, is Kitna better or worse than Johnson? I personally think that Kitna is slightly better, but I wouldn't argue if somebody said otherwise. (Their ages are also comparable--40 for Johnson 2 yrs. ago, and Kitna just turned 38 last month.) I think Romo has probably improved in these last two years, so any improvement from Johnson to Kitna is washed out by Romo's own improvement.
#5--Comparing WP of the QB's is quick and easy, but if Big Ben goes 9-3 in his 12 games, when you compare it with Batch/Dixon's 3-1, you get the same .750 WP. Do you want to say that Batch/Dixon are just as good as Big Ben? (Or worse, that they're BETTER than BR because he goes 8-4?)
To summarize--you're wrong, Brian's right. You both need a timeout to fix your attitudes, but the numbers you used have some SERIOUS problems & biases.
Here's the deal with hostile comments. I do not screen, edit, or delete comments. However, I do enforce a policy of respect around here. My rule is, if you wouldn't walk into my office in person and say it to my face, then prepare to get humiliated. That goes double for people with agendas. Making fun of rude comments is how I enforce the policy. (And it's also good, fun sport.)
As the saying goes, "Never pick a fight with a man who buys ink by the barrel."
Joseph -
I didn't get through entire list. I was throwing something together quickly. I simply started at the top and went down until I got what I thought would be enough data to show the point. I may keep going - but it will result in roughly the same result.
I don't know how there is bias on my part - I simply looked at the top QB's from a list generated from a reputable site and cmpared the records of starters and backups.
I have no idea what wpa is or how Mr. Burke comes to such a number. But I think when one looks at actual instances of star level QBs records compared to backups filling in for them in the same season and see's such a small difference in actual record over 100s of games it should make one question the value of such a stat.
As for anecdotal evidence such as look what happened to the Cardinals when Warner left. Yeah and look what happened to the Rams when Warner got hurt - they played better. Or pretty much every time McNabb got hurt the backups did as well or better. Of when Bubby Brister went 4-0 in 98 when Elway went 10-2. Or when Frank Reich went 3-0 filling in for Jim Kelly after Kelly went 6-7. Each instance in and of itself tells us nothing. Comparing the records of backups and star starters over many instances gives us a rough estimate.
Numbers like wpa or dvoa try to assign a precise value to the contributions of individuals players to wins. This is a ridiculously difficult thing to do - if not impossible. But I think simple studies like the one I just threw together suggest strongly that a 6 games swing based on one player is probably way out.
As Warren Buffet once said I would rather be roughly right than precisely wrong. Sorry Mr. Burke I think you are precisely wrong.
Brian Burke wrote:
"Did all the starters in your spreadsheet have a career 7.3 net YPA?"
No - they in fact averaged 7.2733333 per attempt
what's your point - are you arguing that because Romo has a career 7.3ypa and Kitna has averaged 6.6 that amounts to 6 more wins for a Romo lead team?
Did Romo and Kitna play in the same environment on the same teams. Yards per attempt is far more a product of the team you play for than the skills you possess. That's why as a raw rookie Matt Cassel averages 7.2/att for NE but drops to 5.9 when he goes to KC - despite having more experience.
QB's are not all the same - Tony Romo is almost certainly better than John Kitna. But show me any concrete evidence that these top QB's actually deliver anything like 6 extra wins.
Further to the ypa averages
the starters I from the list I used had a 7.27 yds/att average
the backups had 6.51
So Romo at 7.3 and Kitna at 6.6 are almost dead on the averages of the two groups. those two groups averaged .61 more wins per 16 games than the backups.
I once made a spreadsheet.
It was exhausting.
Now my ass't does it.
I think some of the anonymous poster(s) above are missing the point. We already know that Romo is a very good quarterback. AND we know that his backup is a very bad quarterback. Average quarterbacks replaced by average backups don't tell us anything here. AND good quarterbacks replaced by average backups don't either. So we would want to look at the database of only well above average QBs replaced by well below average backups to see that huge drop in wins.
haha, i noticed one of the WP commentators quoted Raheem Morris as saying:
"Stats are for losers, so you keep looking at stats and we'll keep looking at wins."
...so far he is 7 and 15 in 2009 and this year.
super small sample size, but really? Who quotes a guy saying "we'll keep looking at wins" who is .318?
ironically enough what do you think WPA stands for?
also...people view "stats" as some voodoo magic, most of what is done that pertains to sports is barely more than counting. I wonder if people realized that what is happening, most of the time, is just counting. We're allowed to count, right?
dear sunrise089
What evidence is there that Kitna is a well below average backup? What is a well below average QB?
Good QB's don't move around much - but when they do as far as I can see they move towards the level of the team.
Young plays in TB and he's awful. Goes to SF and he's a Hall of Famer.
Cutler plays for a good offensive system in Denver he's just fine. Traded to Chic - where they struggle offensively and he's awful.
Jeff Garcia in SF - just dandy. Over the terrible squads in Clev and Det - he's terrible. Gets two mores chances in Philly and Tampa with good coaches and he's just fine.
Cassell as a lowly regarded rookie puts up fine numbers in NE. Goes to a bad KC team and his stats are awful.
Now M. Vick is putting up passing numbers like never before. Granted it's only two games but you want to be Andy Reid's system makes it all work for whichever one of those guys play.
"the starters I from the list I used had a 7.27 yds/att average
the backups had 6.51
So Romo at 7.3 and Kitna at 6.6 are almost dead on the averages of the two groups"
Brian's article says that Kitna's career net Y/A (which includes sacks, and maybe TDs/INTs?) is 5.0, and Romo's is 7.4. That difference is 2.4 Y/A. The QBs in your sample had a difference of .76 Y/A. That's a three-fold difference.
BTW--a Y/A difference of 0.76 definitely shows your sample is biased. In 2009, the difference between the #5 QB and #12 QB in terms of Y/A was 0.90 yds, and in 2008 0.80 yards was the difference between the #5 and #16 QB. Backup QBs should not perform above the 50th percentile! And, Kitna is NOT one of the 25 best QBs in the league, much less top 15.
Look - I simply took a list of the top 100 qb's of all time from Pro Football Reference - I started at the top and for any QB that played in the 80,90s and 00s I checked his career record and recorded his record in any season he missed more than 3 games. Then I recorded the records of any backups that had a win/loss record for that same team. I added up the total wins and losses for both groups and that resulted in a very small difference in winning pct.
If the Tony Romo's of the world were really capable of a 6 game improvement over a backup qb than you would obviously see some drastic decline in performance for teams that lost such great qb's to injury.
You don't need any advanced statistics. It would simply jump out at you whenever a star went down to injury. There would example after example of staggering drops in performance whenever such players fell to injury. Oh there are some examples - but as I've shown there are lots of counter examples.
Perhaps the backup list is biased somewhat. Perhaps the backups are somewhat better than your average backup. But the difference over roughly 30 separate seasons and 45 or so different qbs picked in a random fashion was .61 games per 16 game season. Mr Burke is suggesting Romo to Kitna is 6 games. That is a massive difference.
How's this - how about you folks show me some stats that support the notion that losing a guy like Romo costs you 6 games....show some actual evidence not some individual stat that Brian Burke or Football Outsiders made up - but some actual win loss results. Best of luck to you.
Is there not a bias in these 'starter/backup' tests around why the backup is playing?
As far as I can think, your backup could be in because a) your starter is injured or b) your starter was benched.
If your starter was benched then you shouldn't see a dropoff in performance because the coach has made a call that the backup will be at least a good as the guy he's replacing.
I don't know what proportion of starting QBs are benched compared to injured, but any decent amount will skew this data heavily - giving the impression that backups are better than they really are.
It really boils down to splitting the QBs by whether the coach wanted them to play or not.
Ian - I have no way of knowing if the QB's were benched or not but I would guess that the list of 15 QB's I mentioned were very unlikely to be benched. However, even if you could ascertain which were benched and which weren't - do you really think it would make enough difference to make up such a huge spread in performance (.61 vs 6 games)?
jmaron - I don't know, but I'm open to seeing the stats and being convinced either way. It does sound a lot, but then again losing your starting QB is a huge thing.
But I'm not arguing whether the 6 games is right, I'm just pointing out where the alternate method people are talking about could fall down.
I don't think the effect of having an identical supporting cast is taken into account in Brian's methodology; I would guess that this significantly lessens the gap in Romo and Kitna's respective true talent levels as Dallas QB.
Ok, I've done a bit more a look into this, specifically into Jon Kitna.
I've taken the data from 2003-2010 and compared Kitna to other QBs on teams he plays on. Controlling for year-on-year differences in teams (thanks to Brian for the idea of the 3-4 defense post), Kitna's AY/A (as the PFR formula of +20 per TD, -45 per INT) is on average 0.749 lower than other QBs on the same team (that is, the IsKitna dummy variable has a coefficient of -0.749).
A quick check reveals that to be equivalent to 1.4 games over a 16 game season. So, Dallas' GWP of 0.52 should come down to 0.43 with Kitna under center. Of course, it's all just guesswork and it could turn out that Kitna plays brilliantly. Past data, though, suggests Dallas have gone from a 21% chance of going 8-8 or better to only 8% and what little outside hope they had of a playoff spot after a 1-5 start has pretty much disappeared now.
Anonymous could've saved himself the trouble of redoing the analysis using PFR.com data because PFR.com already did it, in a three-parter. To quote:
~~~~~~
What's a Starting QB Worth?
In Parts 1 and 2, I attempted to figure out what a starting QB --- an actual regular starting QB, not a placeholder like say Chris Redman --- is worth to a team.
I came up with an empirical answer of 2.3 points per game ... an average drop of .038 in winning percentage, which comes out to 0.6 wins per season...
I then decided to make absolutely sure I wasn't biasing the results toward the backups by intentionally trying to bias them toward the starters.
In particular, I threw out all the teams whose starting QBs had a below-.500 record. So I was looking at all teams since 1990 whose game one starting QB started at least eight games and whose passing stats were at least league average and whose record was at least .500. That sample includes 31 teams.
Using those teams, the average difference in winning percentage between starter and backups was .108, which would imply 1.7 wins per year...
~~~~~
People grossly over-hero worship the value of the QB. As I said in a comment under the "Game Probabilities" post, even if the QB's value to winning matched that of five other staters combined -- way more than what NFL payroll data tell us GMs believe -- he'd still contribute only about 15% of the team's ability to win. About as much as the third-best player on a basketball team.
Of course the starting QB *is* by far the most important single player on a team -- but when a team has 53 players and about 35 serious contributors to each game, there's no contradiction in saying he's by far the most important single player on the team, and also a whole lot less important than most fans think.
He's just one guy out of 11 on the field at all times for the offense. They all have to work together to make things work. It should be obvious that the other 10 together have more influence on the QB's stats than he has on theirs. (Unless the QB is really terribly *bad*.)
Anyone who doubts this common sense and doesn't believe the PFR.com numbers should go look at Dave Berri's data at Wages of Wins.
No matter how often everybody says it and puts it in stat lines, no QB has a yards-per-attempt number, the team has one.
Jim Glass - I'm the guy who did the spreadsheet. Would have been nice if you had been in the conversation before I dug up another list to prove a point.
But at least I've made Burke's list of most ridiculous emails.
I really should keep a list. There are some real weirdos out there, and most readers never see them because they show up on months-old posts. JMaron- your comment wasn't bad at all, just the first 'ridiculous' part.
In fact, the substance of your point is very well taken. The PFR post is solid. I see 2 concepts in play. What's the general drop-off when a starter goes down? And second, What's the potential drop-off in this specific case when Tony Romo goes down and is replaced by Jon Kitna?
The first concept informs us, and the second concept also informs us. The truth will be in the middle.
There have been 31 comments discussing the issue here, taking up about 5 pages of debate. I have about 1 inch to make people think in the WP articles. It's a mass audience, generally not familiar with stats, so I have to be simple and straight forward.
I think I was just reacting to the excessive credit QB's, pitchers and goalies get in the media.
It sure shouldn't be targeted at someone like yourself who is making great efforts to find the truth and move away from the nonsense in the mainstream sports media.
I once criticized PFR.com's Chase Stuart for writing "I don't care about the Jon Kitna's of the world". I sort of had to laugh at that comment because my own area of research had always been comeback wins and Jon Kitna, among major QBs, had the highest ratio of comeback wins vs. career victories of any of the QBs I examined---38%---19 of his 50 career wins had been by comebacks. To my way of thinking, Kitna had to work harder than any other QB to attain wins. Maybe we shouldn't just dismiss Kitna so quickly.