Should Rookie QBs Start?

Wages of Wins author and loyal Detroit Lions fan Dave Berri recently asked me about research on whether rookie quarterbacks are better off standing on the sideline all season. One of the big questions in Detroit this year will certainly be whether overall number one pick Matthew Stafford should start at QB. The central issue is whether starting a rookie QB somehow harms his long-term development. Does a year holding a clipboard allow rookies to adapt to the NFL and boost their prospects for a successful career?

Names like Boller, Harrington, Couch, Shuler, and Carr underscore the danger of starting rookie passers. But there are also names such as Manning, Roethlisberger, Marino, Elway, and Aikman that indicate that starting as a rookie may not be so damaging to a QB's development.

This is a question I get often but haven’t looked into it because of complications associated with the issue. First, there aren’t that many top QBs to analyze—the sample size is fairly small. Any inference we make needs to keep in mind the small sample. Second, there is a problem of bias in the data. The better QBs would be the ones to earn starting jobs their rookie year, and would also likely tend to be the ones to enjoy successful careers.

The trick would be to properly account for underlying QB potential, which would be quite a trick. If we knew that, that’s all we’d ever really need to know about a QB. There's no perfect way to measure that, but in the end, I think the most reasonable variable to indicate potential is overall draft pick number. It’s something that is established prior to any decision to start or not as a rookie. Also, although it is often an unreliable predictor of career performance for individual QBs, it correlates very well for QBs as a group. In other words, the number one QB might not pan out or even be better than the second QB taken in any particular draft year. But as a whole, the top picks reliably tend to outperform subsequent picks.

Data

I looked at first and second round QBs drafted between 1980 and 2004. I chose 1980 because it is roughly the dawn of the modern NFL passing rules. I chose 2004 to allow at least four years to asses each QB’s performance. Data are from Pro-Football-Reference.com. To measure career success, I used Adjusted Yards per Attempt (Adj YPA). This is passing yards minus 45 yards for every interception, per pass attempt. For QBs with very low pass attempts and spurious Adj YPA stats, I replaced their actual Adj YPA with 3.0 YPA, generally the floor for QBs with a reasonable sample size of attempts. (There is no reason to expect 1989 Chief’s 2nd-round pick Mike Elkins to lose 20 yards for every pass based on only 2 attempts.)

Methodology

The first step is to account for potential using overall draft pick number. The graph below plots career Adj YPA by pick number. The scatterplot is pretty random for any individual QB, but as a whole there is a predictable trend. Top draft picks tend to end up as NFL top passers. I estimated the expected Adj YPA based on pick number. A QB from the top of the first round would be expected to average 5.0 Adj YPA, while a QB from the bottom of the second round would be expected to average 4.1 Adj YPA.



We can see a clear trend. Unsurprisingly, top picks would be expected to end up with better career passing stats than later picks. A linear regression estimates what the baseline expectation should be for each slot in the draft.

Next I calculated the 'Adj YPA above expected' for each QB. Now we can compare QBs who started their first year to those that didn’t, while holding “potential” equal.

But how do we define “started their first year?” How many starts qualifies--4, 5, 11? I don’t know, so let’s start by looking at the whole picture.

Results

We can use that baseline to compare each QB's career Adj YPA. Some QBs did better than expected given their draft slots and some did worse. We can now test if there is a connection between better than expected performance and the number of rookie year starts.



As it turns out, it appears that QBs with more rookie starts tend to enjoy greater career success, even accounting for draft order.

Some of you might have noticed that what I've really done here is a crude multivariate regression. Holding for draft order, I estimated the effect of games started. What if we just do the regression directly?

As expected we get a small negative effect with draft order. (The higher the pick number, the worse the expected stats.) The Games Started variable is positive and significant at p=0.03. The model as a whole has an r-squared of 0.15--small in absolute terms, but considerable given the highly random individual variance in QB careers.

But this is only one way of looking at whether a QB was a starter or not. What if we draw a line at say, 5 rookies starts--below 5 starts he's not a rookie starter and above it he is. The group of QBs with 5 or less starts averages -0.01 Adj YPA above expected, and the group with 6 or more starts averages +0.3 Adj YPA above expected. If we define it at zero starts, those with no rookie starts averaged -0.03 Adj YPA above expected, while those with at least one start averaged +0.02 starts above expected. In fact, no matter where I chose the endpoints of the groups, from 3 to 13 starts, the group with more starts outperforms the group with fewer starts by about 0.4 Adj YPA.

Conclusion

Does this mean teams should rush their rookies out to face the onslaught of NFL defenses to somehow make them better? I really doubt it. If I had to bet, I'd say that we simply haven't fully accounted for QB "potential" using draft order alone. I think the better QBs, those with the best chances of career success, often gain and maintain starting positions earlier.

But at the very least, we can say this: Given this analysis, there is no reason for a coach to arbitrarily keep a rookie QB on the bench. He should start his best QB, rookie or not, and not worry about incubating him under a ballcap on the sidelines. In the end, it should be the coach's qualitative judgment on the readiness of the player.

Here are the QBs and their stats I used for this article. (It's interesting just to see who the QBs are who exceeded expectations. There are some surprising names--Pennington and Batch at the top, for example. And Is Eli Manning really worse than David Carr? Wow.) You can sort the table by clicking on the column headers.

Year	Rnd	Pick	Name	Team	Adj YPA	Exp Adj YPA	Yr1 GS	AdjYPA Abv Exp
2001	2	32	Drew Brees		6.0	4.6	0	1.4
1981	2	33	Neil Lomax		5.9	4.6	7	1.4
1998	1	1	Peyton Manning		6.4	5.0	16	1.4
1998	2	60	Charlie Batch		5.5	4.1	12	1.4
2004	1	4	Philip Rivers		6.4	5.0	0	1.4
2004	1	11	Ben Roethlisberger		6.2	4.9	13	1.3
1983	1	27	Dan Marino		6.0	4.6	9	1.3
2000	1	18	Chad Pennington		6.1	4.8	0	1.3
1999	1	11	Daunte Culpepper		6.1	4.9	0	1.3
1984	2	38	Boomer Esiason		5.7	4.5	4	1.2
1985	2	37	Randall Cunningham		5.6	4.5	4	1.1
1983	1	24	Ken O'Brien		5.7	4.7	0	1.1
1995	2	45	Todd Collins		5.3	4.4	1	1.0
1991	2	33	Brett Favre		5.5	4.6	0	1.0
1983	1	14	Jim Kelly		5.8	4.8	0	0.9
1999	1	2	Donovan McNabb		5.9	5.0	6	0.9
1983	1	15	Tony Eason		5.7	4.8	4	0.8
2003	1	1	Carson Palmer		5.8	5.0	0	0.8
1987	1	26	Jim Harbaugh		5.4	4.7	0	0.7
1995	1	3	Steve McNair		5.7	5.0	2	0.7
1996	2	42	Tony Banks		5.1	4.4	13	0.7
1983	1	1	John Elway		5.7	5.0	10	0.7
1990	1	1	Jeff George		5.7	5.0	12	0.6
1997	2	42	Jake Plummer		5.1	4.4	9	0.6
2001	2	53	Quincy Carter		4.9	4.3	8	0.6
1989	1	1	Troy Aikman		5.6	5.0	11	0.6
2003	1	7	Byron Leftwich		5.5	4.9	13	0.6
1982	1	5	Jim McMahon		5.5	5.0	7	0.5
1995	2	60	Kordell Stewart		4.7	4.1	2	0.5
1986	1	3	Jim Everett		5.5	5.0	5	0.5
2002	1	32	Patrick Ramsey		5.0	4.6	5	0.4
1999	2	50	Shaun King		4.7	4.3	5	0.4
1989	2	51	Billy Joe Tolliver		4.6	4.3	5	0.3
2001	1	1	Michael Vick		5.3	5.0	2	0.3
2004	1	22	J.P. Losman		5.0	4.7	0	0.3
1987	1	13	Chris Miller		5.1	4.9	2	0.2
1993	1	1	Drew Bledsoe		5.3	5.0	12	0.2
1995	1	5	Kerry Collins		5.2	5.0	13	0.2
1987	1	1	Vinny Testaverde		5.1	5.0	4	0.1
2003	1	22	Rex Grossman		4.8	4.7	3	0.1
1992	1	25	Tommy Maddox		4.7	4.7	4	0.0
1986	2	47	Jack Trudeau		4.3	4.3	11	0.0
2002	1	1	David Carr		5.0	5.0	16	-0.1
2004	1	1	Eli Manning		4.9	5.0	7	-0.1
1991	1	24	Todd Marinovich		4.6	4.7	1	-0.1
1980	1	15	Marc Wilson		4.7	4.8	0	-0.1
1990	1	7	Andre Ware		4.7	4.9	1	-0.3
2003	1	19	Kyle Boller		4.5	4.8	9	-0.3
1999	1	1	Tim Couch		4.7	5.0	14	-0.3
1994	1	6	Trent Dilfer		4.6	5.0	2	-0.3
1999	1	12	Cade McNown		4.4	4.9	6	-0.5
1982	2	44	Oliver Luck		3.9	4.4	0	-0.5
1980	1	28	Mark Malone		4.0	4.6	0	-0.7
2002	1	3	Joey Harrington		4.3	5.0	12	-0.7
1986	1	12	Chuck Long		4.1	4.9	2	-0.8
1992	1	6	David Klingler		4.2	5.0	4	-0.8
1993	1	2	Rick Mirer		4.2	5.0	16	-0.8
1983	1	7	Todd Blackledge		4.1	4.9	0	-0.9
1991	2	34	Browning Nagle		3.6	4.5	0	-0.9
1982	2	48	Matt Kofler		3.3	4.3	0	-1.1
1994	1	3	Heath Shuler		3.7	5.0	8	-1.3
1992	2	46	Tony Sacca		3.0	4.4	0	-1.4
1992	2	40	Matt Blundin		3.0	4.4	0	-1.4
1999	1	3	Akili Smith		3.5	5.0	4	-1.5
1980	2	37	Gene Bradley		3.0	4.5		-1.5
2001	2	59	Marques Tuiasosopo		2.7	4.2	0	-1.5
1989	2	32	Mike Elkins		3.0	4.6	0	-1.6
1987	1	6	Kelly Stouffer		3.4	5.0	0	-1.6
1991	1	16	Dan McGwire		3.2	4.8	1	-1.6
1997	1	26	Jim Druckenmiller		3.0	4.7	1	-1.7
1998	1	2	Ryan Leaf		3.1	5.0	9	-1.9
1981	1	6	Rich Campbell		3.0	5.0	0	-2.0
1982	1	4	Art Schlichter		3.0	5.0	0	-2.0