If a team is in the market for a young quarterback, should it trade up and grab someone a spot or two higher on their draft board? Or should they stand pat and take the best QB that falls to their own pick? There are many considerations to weigh, including:
- What will they have to trade to move up in the draft?
- How much cap space will they have?
- How likely is it that a higher rated QB will turn out to be better than another further down the draft board?
Perhaps the last question is the hardest to quantify. As fans, we're frequently reminded by commentators how Tom Brady was a 6th round pick and how Ryan Leaf was rated as highly as Peyton Manning. Ok, but what about the dozens of low round quarterbacks who stunk up the league, or the many first round picks who are now in the Hall of Fame?
Quantifying career-long performance is more difficult than I expected. For starters, QBs playing over different NFL eras need an adjustment to their stats. Second, how do we measure a QB's total performance? Career totals? Yards per attempt and interception rates? Both methods give different results. If we use career totals, a slightly above average QB with a long career might outshine a great QB who retired sooner. But then again, longevity should count for something. If we use rate stats, there are a number of QBs with outlier performance stats simply due to small numbers of pass attempts.
We'd need to use a minimum number of qualifying attempts. But then that defeats the purpose of the analysis. QBs who aren't even good enough to win a starting job should be counted when we're comparing draft outcomes. If they're excluded we'd have a severe selection-bias in the data because only the "diamonds in the rough" of later rounds will remain, obscuring the true worth of a late-round QB.
I'll address those issues in a later post, but for now I'm taking a cheap and easy solution--I'll rate QB draft picks by how many Pro Bowls they've been selected to. This method avoids the problems of normalizing stats across eras and figuring out how to handle QBs without a significant number of pass attempts.
With data from Pro-Football-Reference.com (which has a great draft database) I compiled a list of all QB draft picks from 1980-2000. I chose those years to limit the QBs to the modern era with the post-1978 passing rules, and to give enough time for the class of 2000 to develop and be assessed. I counted how many times each QB had been to at least one Pro Bowl (PB), at least two PBs, and at least three PBs. One PB might indicate a flash-in-the-pan guy, such as Gus Frerotte, but two PBs probably indicates sustained excellence. Three or more PBs would make any GM happy.
The table and graph below show the likelihood that a QB drafted in each round will pan-out and be selected to at least 1 Pro Bowl (1+ PB), at least 2 PBs (+2 PB), and at least 3 PBs (+3 PB). The data set is not large (only 193 QBs total), so the graph contains smoothed lines for a more realistic estimate of future expected performance from each round.
Quantifying career-long performance is more difficult than I expected. For starters, QBs playing over different NFL eras need an adjustment to their stats. Second, how do we measure a QB's total performance? Career totals? Yards per attempt and interception rates? Both methods give different results. If we use career totals, a slightly above average QB with a long career might outshine a great QB who retired sooner. But then again, longevity should count for something. If we use rate stats, there are a number of QBs with outlier performance stats simply due to small numbers of pass attempts.
We'd need to use a minimum number of qualifying attempts. But then that defeats the purpose of the analysis. QBs who aren't even good enough to win a starting job should be counted when we're comparing draft outcomes. If they're excluded we'd have a severe selection-bias in the data because only the "diamonds in the rough" of later rounds will remain, obscuring the true worth of a late-round QB.
I'll address those issues in a later post, but for now I'm taking a cheap and easy solution--I'll rate QB draft picks by how many Pro Bowls they've been selected to. This method avoids the problems of normalizing stats across eras and figuring out how to handle QBs without a significant number of pass attempts.
With data from Pro-Football-Reference.com (which has a great draft database) I compiled a list of all QB draft picks from 1980-2000. I chose those years to limit the QBs to the modern era with the post-1978 passing rules, and to give enough time for the class of 2000 to develop and be assessed. I counted how many times each QB had been to at least one Pro Bowl (PB), at least two PBs, and at least three PBs. One PB might indicate a flash-in-the-pan guy, such as Gus Frerotte, but two PBs probably indicates sustained excellence. Three or more PBs would make any GM happy.
The table and graph below show the likelihood that a QB drafted in each round will pan-out and be selected to at least 1 Pro Bowl (1+ PB), at least 2 PBs (+2 PB), and at least 3 PBs (+3 PB). The data set is not large (only 193 QBs total), so the graph contains smoothed lines for a more realistic estimate of future expected performance from each round.
Round | 1+ PB | 2+ PB | 3+ PB |
1 | 0.38 | 0.28 | 0.23 |
2 | 0.32 | 0.21 | 0.16 |
3 | 0.13 | 0.04 | 0.00 |
4 | 0.06 | 0.03 | 0.03 |
5 | 0.10 | 0.10 | 0.10 |
6 | 0.14 | 0.11 | 0.06 |
7 | 0.03 | 0.00 | 0.00 |
Historically, moving up from the second round to the first round for a QB buys you a 16% to 23% improvement in the chance he'll go to at least three PBs. It would also buy you a reduction from a 68% to 62% chance he'll never go to a single PB.
Looking at draft picks by round may not be the best approach. What round someone is selected in may have more to do with team needs and other factors aside from how much potential a prospective QB has. Looking at QBs in terms of which QB pick they were in their draft class might be helpful. For example, although Tom Brady was taken in the 6th round of 1999 and Brock Huard was taken in the 3rd round of 2000, they were both the 7th QB taken in their respective years.
The table and graph below list the same information as above--how often a drafted QB is selected to a PB. This time the QBs are broken out by which QB pick they were in their draft year. Again, smoothed lines are added.
Looking at draft picks by round may not be the best approach. What round someone is selected in may have more to do with team needs and other factors aside from how much potential a prospective QB has. Looking at QBs in terms of which QB pick they were in their draft class might be helpful. For example, although Tom Brady was taken in the 6th round of 1999 and Brock Huard was taken in the 3rd round of 2000, they were both the 7th QB taken in their respective years.
The table and graph below list the same information as above--how often a drafted QB is selected to a PB. This time the QBs are broken out by which QB pick they were in their draft year. Again, smoothed lines are added.
QB Pick | 1+ PB | 2+ PB | 3+ PB |
1 | 0.38 | 0.38 | 0.33 |
2 | 0.29 | 0.14 | 0.05 |
3 | 0.14 | 0.10 | 0.10 |
4 | 0.19 | 0.10 | 0.10 |
5 | 0.19 | 0.10 | 0.00 |
6 | 0.11 | 0.06 | 0.06 |
7 | 0.17 | 0.17 | 0.17 |
8 | 0.19 | 0.06 | 0.00 |
9 | 0 | 0 | 0 |
10 | 0 | 0 | 0 |
11 | 0 | 0 | 0 |
12 | 0 | 0 | 0 |
13 | 0 | 0 | 0 |
14 | 0 | 0 | 0 |
There appears to be a very large drop-off in expected performance from the first QB taken to the second. A third of the first QBs taken went to three or more PBs, but only 5%-10% of the second through sixth QBs taken went to three or more.
I interpret these results as an indication that unless a team can get the very first QB or at least the second taken in a given year, it shouldn't expect to find a franchise player. It does happen, but it's very rare. From the 3rd QB taken through the 7th, there's no apparent difference. If a team doesn't get one of the first two, it might as well wait until later rounds and take a chance on a later pick or set its sights on a free agent. Meanwhile, it can fill other needs.
Of course, Pro Bowl selection is not the best measure of a QB's performance--it's subjective, somewhat arbitrary, and it's an all or nothing measure. But it does help answer a the question all GMs ask when drafting a QB--how likely is it this guy going to pan out? Next, I'll attempt to quantify just how much better top picks are in terms of team wins.
I interpret these results as an indication that unless a team can get the very first QB or at least the second taken in a given year, it shouldn't expect to find a franchise player. It does happen, but it's very rare. From the 3rd QB taken through the 7th, there's no apparent difference. If a team doesn't get one of the first two, it might as well wait until later rounds and take a chance on a later pick or set its sights on a free agent. Meanwhile, it can fill other needs.
Of course, Pro Bowl selection is not the best measure of a QB's performance--it's subjective, somewhat arbitrary, and it's an all or nothing measure. But it does help answer a the question all GMs ask when drafting a QB--how likely is it this guy going to pan out? Next, I'll attempt to quantify just how much better top picks are in terms of team wins.
This is a really great article - nice work. As you mentioned, Pro Bowl appearances can be an imperfect measure of how successful a QB has been, because the PB is often a popularity contest, and it's likely that a first-round QB will be more well-known than a later-round QB early in his career. Do you by any chance have the same data by draft round, but using number of QB starts as the measuring stick? Would be curious what that shows. Thanks again.
DB-I have a follow-up forthcoming that's based on actual passing stats instead of Pro Bowl appearances.
I have "seasons as primary starter" by draft round and pick, but "games started" isn't handy.
What about the likelihood of getting to the Super Bowl vice the PB? Or perhaps, the playoffs?
I know you have little enough data to work with as it is, but to get a more accurate trend line, wouldn't it be better to discount anomalies by dropping the highest- and lowest-ranked QB's from each categoty?
That's a good idea. Hadn't thought of that.
What about the fact that there is only so many pro bowl slots? The 1980 guys had to compete with the previous years draft classes to earn their first pro bowl slots. What is to say the guys drafted 1975-79 was unnecessarily stronger/weaker than other years?
Nice work!