QB Wins Added 3

Recently, I began constructing a passer rating system that, to the greatest degree possible, includes performance variables controlled by the quarterback alone and avoids redundant use of variables that measure the same attribute. This improved system includes completion percentage, interception rate, and sack yards per attempt. The previous two posts discussed completion percentage and interception rate. This post will examine sacks. Each of the stats used in the rating system are computed in terms of how many wins they add, on average, to a QB's team over the course of a season. The wins added are based on the previously established regression model based on team efficieny stats.

More and more, I'm coming to realize that sacks are far more dependent on the quarterback than on his pass protection. A quarterback's mobility, release, decision-making, and pocket presence all are critically important. I'm using sack yardage rate instead of total absolute sacks for two reasons. First, the number of pass attempts must be accounted for. We would expect a QB with more pass attempts to be sacked more often than a QB who passes less often, given equal abilities to prevent sacks. Second, sack yards yield more information about the QB than sacks alone. Some QBs have fewer sacks, but because they often gamble by scrambling deep in the backfield, they give up large numbers of sack yards. Does the QB tend to step up in the pocket or drift backward? As you'll see, some of the QBs considered the most mobile actually give up some of the most sack yards.

The table below lists each quarterback with at least 100 pass attempts in 2006, their sack yards, and their sack yardage rates (sack yards per pass attempt). The column labeled "Sk Yd Rate > Avg" is the sack yardage rate above (or below) league average. The next column, "+Sk Yd Wins" lists the wins added based on sack yards for each quarterback. The list is sorted in order of best to worst.













































































































































































































































































































































PlayerSk YdsSk Yd

Rate
SkYdRate > Avg+Sk Wins
Peyton Manning860.15-0.280.44
Tim Rattay180.17-0.260.41
Steve McNair840.17-0.260.41
Drew Brees1050.18-0.250.39
Jeff Garcia400.21-0.230.36
Brett Favre1340.21-0.220.35
Byron Leftwich480.25-0.180.29
Jason Campbell550.26-0.180.28
Rex Grossman1420.28-0.150.24
Joey Harrington1160.29-0.150.23
Chris Simms320.29-0.140.22
Philip Rivers1440.30-0.140.22
Tom Brady1750.32-0.110.17
Jake Plummer1110.33-0.100.16
Chad Pennington1720.33-0.100.16
Vince Young1290.34-0.100.15
Mark Brunell920.34-0.100.15
Eli Manning1860.34-0.090.15
Tony Romo1240.35-0.090.14
Jake Delhomme1670.37-0.070.10
Matt Leinart1580.40-0.040.06
Damon Huard1060.41-0.030.04
Bruce Gradkowski1460.41-0.020.03
Donovan McNabb1400.42-0.020.03
Carson Palmer2330.42-0.010.02
Alex Smith2020.42-0.010.02
Brad Johnson2000.43-0.010.01
David Garrard1190.460.02-0.03
David Carr2400.500.06-0.10
Derek Anderson660.530.09-0.15
Seneca Wallace830.540.10-0.16
Ben Roethlisberger2800.540.11-0.17
Matt Hasselbeck2290.570.13-0.21
Jay Cutler850.570.13-0.21
Kurt Warner1040.570.14-0.22
Trent Green1270.570.14-0.22
Marc Bulger3660.570.14-0.22
Drew Bledsoe1070.580.14-0.23
Jon Kitna3880.590.15-0.24
Charlie Frye2620.600.17-0.26
J.P. Losman3320.700.26-0.41
Michael Vick3030.700.27-0.42
Andrew Walter2560.800.36-0.57
Aaron Brooks1740.800.36-0.57
Daunte Culpepper1500.970.53-0.84
Average155.90.43

QB Wins Added 2

In the last post, I began constructing a passer rating system that, to the greatest degree possible, includes performance variables controlled by the quarterback alone and avoids redundant use of variables that measure the same attribute.

The new system includes completion percentage, interception rate, and sack yards per attempt. The previous post discussed completion percentage, and this one will cover interception rate. Each QB's stats are computed in terms of how many wins they add, on average, to his team over the course of a season.

While most fans and analysts consider how many inteptions a QB has thrown, they ignore the fact that various QBs have a wide range of passing attempts, even among QBs with the same number of games played. Interceptions per pass attempt is a better indicator of a QB's true tendency to throw interceptions.

The table below ranks QB's interception rates in 2006. Only QBs with over 100 pass attempts are included. Theh column labled "Int Rate > Avg" is the amount above (or below) the average NFL interception rate. Negative numbers are better here because they mean fewer interceptions per attempt. The column labled "Int Rate Wins" are the wins added (or lost) due to the interception rate of the listed QB. Postive numbers are (obviously) better here. The added wins are computed by the efficiency win model Ipreviously established.



















































PlayerInt RateInt Rate >

Avg
Int Rate Wins
Damon Huard0.004-0.0271.385
Jeff Garcia0.011-0.0211.060
Marc Bulger0.013-0.0180.913
Mark Brunell0.015-0.0160.826
Peyton Manning0.016-0.0160.788
Donovan McNabb0.019-0.0130.650
Philip Rivers0.019-0.0120.622
Tim Rattay0.019-0.0120.611
Drew Brees0.019-0.0120.608
Daunte Culpepper0.022-0.0100.485
Tom Brady0.023-0.0090.443
Carson Palmer0.024-0.0070.360
Jake Delhomme0.025-0.0070.334
Steve McNair0.025-0.0070.329
David Carr0.026-0.0050.256
Byron Leftwich0.027-0.0050.248
Bruce Gradkowski0.027-0.0050.243
Jason Campbell0.028-0.0030.169
Brett Favre0.029-0.0030.151
Kurt Warner0.029-0.0030.132
Matt Leinart0.031-0.0010.033
J.P. Losman0.0320.000-0.005
Chad Pennington0.0320.000-0.022
Michael Vick0.0320.001-0.046
Brad Johnson0.0330.002-0.077
Eli Manning0.0330.002-0.092
Alex Smith0.0350.003-0.173
Vince Young0.0350.004-0.183
Jay Cutler0.0350.004-0.187
Jon Kitna0.0360.004-0.207
David Garrard0.0360.004-0.227
Tony Romo0.0370.006-0.284
Joey Harrington0.0370.006-0.288
Matt Hasselbeck0.0390.007-0.371
Jake Plummer0.0390.008-0.398
Rex Grossman0.0400.008-0.429
Aaron Brooks0.0400.008-0.429
Charlie Frye0.0420.010-0.508
Trent Green0.0430.012-0.604
Andrew Walter0.0450.013-0.680
Drew Bledsoe0.0450.014-0.691
Ben Roethlisberger0.0470.015-0.770
Seneca Wallace0.0470.016-0.797
Chris Simms0.0620.030-1.537
Derek Anderson0.0640.032-1.641
Average0.032



In the next post, I'll look at sacks. Then finally, I'll pull it all together for a comprehensive QB win-rating.

QB Wins Added 1

In this series of posts I'm going to build a better quarterback rating system. The system will be try to isolate the passing stats that the QB alone is primarily accountable for. The stats will be efficiency stats in that they are in terms of attempts. The system will also attempt to exclude redundancy stats.

The official Passer Rating system includes co-dependent factors such as TDs, interceptions, and completion percentage. But an accurate QB will tend to have both a high completion rate and high TDs. The passer rating, therefore, double counts accuracy. Further, a QB's passing TDs may be due to his defense's ability to get the ball turned over to him in his oppenents' territory, or due to an effective running game, as much as due to his own ability. The goal is to isolate what the QB is truly responsible for and filter out what the QB does not control.

The system will include completion percentage, interceptions per attempt, and sack yards per pass attempt. Although interceptions and sacks are not solely attributable to the QB, the stats imply that the QB is overwhelmingly responsible for them. Take the case of McNair in Baltimore. After replacing Kyle Boller, Baltimore went from one of the league's worst sack rates to one of the best. The rest of the offense, including the offensive line, was completely identical.

Each stat in the system will be translated into team wins by the efficiency win model previously established. In the first post we'll rank quarterbacks in terms of their 2006 regulare season completion percentage. We'll compute how many percentage points above or below average each QB was last year. All other factors being equal, every additional point of completion percentage brings an extra 0.18 wins to a team on average. Therefore we can compute how many wins each QB contributed to his team by his completion percentage alone.

(Only QBs with more than 100 attempts are considered.)


















































PlayerComp PctPct > AvgWins Added
David Carr68.38.891.60
Tony Romo65.35.891.06
Peyton Manning65.05.591.01
Chad Pennington64.55.090.92
Charlie Frye64.34.890.88
Kurt Warner64.34.890.88
Drew Brees64.34.890.88
Steve McNair63.03.590.65
Marc Bulger62.93.490.63
J.P. Losman62.53.090.56
Jon Kitna62.42.990.54
Carson Palmer62.32.890.52
Mark Brunell62.32.890.52
Tom Brady61.82.390.43
Philip Rivers61.72.290.41
Jeff Garcia61.72.290.41
Brad Johnson61.52.090.38
Trent Green61.11.690.30
Jake Delhomme61.01.590.29
Damon Huard60.71.290.23
Daunte Culpepper60.40.990.18
Tim Rattay60.40.990.18
David Garrard60.20.790.14
Ben Roethlisberger59.70.290.05
Jay Cutler59.1-0.31-0.06
Byron Leftwich59.0-0.41-0.07
Seneca Wallace58.2-1.21-0.22
Alex Smith58.1-1.31-0.24
Eli Manning57.7-1.71-0.31
Joey Harrington57.5-1.91-0.34
Aaron Brooks57.3-2.11-0.38
Donovan McNabb57.0-2.41-0.43
Matt Leinart56.8-2.61-0.47
Matt Hasselbeck56.6-2.81-0.51
Derek Anderson56.4-3.01-0.54
Brett Favre56.0-3.41-0.61
Jake Plummer55.2-4.21-0.76
Chris Simms54.7-4.71-0.85
Rex Grossman54.6-4.81-0.87
Bruce Gradkowski54.0-5.41-0.97
Andrew Walter53.3-6.11-1.10
Drew Bledsoe53.3-6.11-1.10
Jason Campbell53.1-6.31-1.14
Michael Vick52.6-6.81-1.23
Vince Young51.5-7.91-1.42
Average59.4

In the next post, I examine interception rate. Then in the third part of this this article, I'll look at sack yardage rate. Finally, I'll sum the wins added for each QB to produce a more valid passer rating system.

How Many Wins is Manning Worth?

In a recent comment at another site, someone mentioned that "Peyton Manning was good for at least 10 wins a year." I thought that sounded about right.

Using the QB rule-of-thumb I made the other day, I'll estimate how many wins Manning is good for on the basis of his accuracy alone. By accuracy I mean his completion percentage and interceptions.

Manning's completion percentage over the last 4 years hovers around 65.0% while the league average is 59.4%. His interceptions average at 9.8 per year while the NFL average is 16.4.

Based on the coefficients from the efficiency-win model, we can do some quick math.

Cmp%: 65.0 - 59.4 = 5.6% * 0.18 wins/cmp% point = 1.32 wins
Int: 16.4 - 9.8 = 6.6 Interceptions * 0.11 wins/int = 0.62 wins
1.32 + 0.62 = +1.94 wins

So about 10 wins, compared to league, average is right. There are also other unquantifiable factors, such has his leadership or audible calling that probably contribute to his success. But all other things being equal, Peyton Manning is worth (at least) 10 wins, right out of the gate.

McGahee in Baltimore

Jamal Lewis has been on a slide since his monster year in 2003 when he threatened Dickerson’s single season rushing record. Since then Lewis has declined from 5.3 yds per rush to 3.6. Even his receiving has declined steadily, except for 2004 when his average was distorted by a very small number of catches (10).

Year  G  Att  Yds   Avg  Rec Yds   YPC
----------------------------------------
2000 16 309 1364 4.4 27 296 11.0
2002 16 308 1327 4.3 47 442 9.4
2003 16 387 2066 5.3 26 205 7.9
2004 12 235 1006 4.3 10 116 11.6
2005 15 269 906 3.4 32 191 6.0
2006 16 314 1132 3.6 18 115 6.4


Willis McGahee’s performance in Buffalo over the past three seasons has been solid but unspectacular, with an average of 3.9 yds per rush.

Year  G   Att   Yds  Avg  Rec Yds  YPC
----------------------------------------
2004 16 284 1128 4.0 22 169 7.7
2005 16 325 1247 3.8 28 178 6.4
2006 14 259 990 3.8 18 156 8.7

Assuming McGahee gets the same number of carries and receptions as Lewis did in 2006, we can estimate the Ravens' new team rushing and passing efficiency. Although that assumption is unlikely because Baltimore will likely use spread sets, it gives us a feel for the difference between the two running backs. McGahee gives Baltimore 0.3 extra yds per rush and 1.2 extra yds per catch.

314 carries x 0.3 yds/rush = 51.4 added rushing yards
18 catches x 1.2 yds/catch = 21.6 added passing yards

+51.4 yds / 476 team rush attempts = +0.11 yds/rush team rushing efficiency
+21.6 yds / 525 team pass attempts = +0.04 yds/att team passing efficiency

If we run the new efficiencies through the efficiency win model, we see that McGahee could be worth half a win.


I realize it’s a little abstract to say a player can add 0.42 wins to a team’s record. What the math says is: given a similar number of carries and receptions, and given McGahee’s slightly better stats, the Ravens could expect to average 0.42 more wins if the season were played out a million times.

Compared to what we saw with quarterbacks McNair and Brees, who were each estimated to bring an extra 2 or more wins to their new teams, 0.42 added wins seems relatively modest. For a running back to make a very large impact on a team’s record, he would have to be a 5 yd/carry type of back. LaDanian Tomlinson, for example, averaged 5.2 yds/rush in 2006.

Personally, I think these estimates are conservative. Lewis’s strength was in the straight-ahead runs, and not much else. From all the reports, Baltimore intends to use McGahee in a variety of formations making their offensive more versatile and less predictable.

Defense Penalties Puzzle Solved

In an earlier post, I found that penalties committed by a team's defense correlated postively with team wins. In other words, the more penalties committed on defense, the more games a team could expect to win. I was confused by this result and re-checked my math several times, but it always came out the same.

I finally realized the cause of the result. What the stat services call defensive penalties are actually penalties committed by opposing teams on offense and defense. And what they call offensive penalties are penalties committed by one's own team on offense and defense. All the stat sites that I've used for data use the same nomenclature: nfl.com, sportsline.com, and espn.com. I don't know why it's done like that.

It seems all the big websites use data from Stats, Inc., so I bet that's how they do it there. Unfortunately, NFL gamebooks, the official box scores of NFL games, do not breakout a team's penalties between offense and defense.

As an explanatory variable, it's valid--opposition penalties help explain what makes teams win. But as a predictive variable for season wins, it's useless and should be removed. This means some revisions are in order for my models. The good news is that it doesn't change the models' predictions in a significant way because the solution is to remove the variable. And fortunately, a team's opposition's penalty stats tends to average out throughout the season, so removing a variable that is very average doesn't effect the dependent variable (team wins) appreciably.

Trent Green in Miami

How many added wins can Trent Green bring to Miami? First, let’s look at his recent performance in KC. I'm going to include his most recent year despite his head injury early in the season because I believe it's representative of what the Dolphins may expect from Green. His average stats for the last four seasons are:


         CMP%      YPA     YD/CMP    INTRATE    SKRATE
-------------------------------------------------------
GREEN 63.3 7.7 12.1 0.027 0.43

Green’s projected stats and, for comparison, Miami’s 2006 passing stats are:


         CMP%      YPA     YD/CMP    INTRATE    SKRATE
-------------------------------------------------------
PROJ 63.3 7.0 11.1 0.027 0.34
MIA 06 58.0 6.1 10.5 0.032 0.27
(Interception rate is defined as interceptions per pass attempt. Sack rate is defined as sack yards lost per pass attempt. )

Green’s biggest problem in KC last year was avoiding the sack. After returning from his head injury, he was sacked on 12% of his pass plays, more than double his rate in any other year. If the Dolphin’s line can protect Green, he should be a solid upgrade to a woeful passing attack.

All other factors being equal, we can expect Trent Green to bring about +2.0 added wins to Miami. Assuming the other parts of the Dolphins team perform as well as last year, they would likely end the season with 10 wins.

New QB Added Wins

In this article, I’ll explain how I estimated how many additional wins an established quarterback can bring to new team by forecasting his likely effect on team passing stats.

First, we need an idea of which stats the quarterback on an NFL team is primarily responsible for. Team passing performance is dependent on every member of the offense, so it is very difficult to parse what can attributed to the quarterback and what can be attributed to other players. We’ll need to make some assumptions from a qualitative look at the NFL records.

These assumptions are only rough estimations of the stats we need to estimate wins. At this point, it would be impossible to declare exactly how much responsibility a QB has for certain team stats, and even if we could it probably varies from team to team. I’m only going to use them to get a back-of-the-envelope estimation for the effect of acquiring a veteran QB. Assuming all other things stay the same, would he add 5 wins, 2 wins, or none?

PROJECTIONS

Completion percentage appears to be entirely owned by the QB. Year-to-year deviations in completion % tend to be very narrow. QB careers are typically marked by one low percentage rookie year, followed by a steady completion rate in subsequent years. Team changes do not affect completion % any more than the small year-to-year variations while playing for the same team. We’ll say that the QB owns completion percentage.

Yards per catch varies significantly from year-to-year. It seems to be far more dependent on the QB’s supporting cast than completion %. My rough estimate is that the QB is responsible for 1/3 of the yards per catch stat. The other 2/3 is most likely due to receiver performance, and yards-after-catch in particular. Another factor might be the type of offensive scheme used by his offense. We’ll say that the QB owns 1/3 of yards per catch and the other 2/3 is owned by the team’s receivers or other factors.

(Yards per catch)*(completion %) = yards per attempt, the primary pass stat used in the win estimation model. So we can already approximate it.

We also want to consider sacks. Sack yards are generally due to a combination of three things: QB immobility/poor decision, pass protection failure, or a failure of the receivers to get open/run the correct route. QB sacks do appear to vary significantly from year-to-year, suggesting a quarterback’s share of responsibility for his sacks is no more than 50%. We’ll say that 1/2 of a QB’s sack rate is owned by the QB himself.

The last stat we need to estimate is interception rate. Total interceptions vary from year to year, but interceptions per pass attempt is much steadier. And although the cause of an interception can be contributed to by a receiver breaking the wrong way or a QB being hurried by a pass rusher, it is the QB’s ultimate decision to throw the ball. There are also freak interceptions, such as bobbled passes, but we’ll assume those types of plays tend to even out among QBs over time. Ultimately, the decision to throw and the location of the pass rests on his ability alone. We’ll say the QB completely owns his interception rate.

We can use two recent cases of veteran QBs coming to new teams to see if the assumptions and projections make sense. Both Drew Brees and Steve McNair had significant impacts on their new teams.

DREW BREES IN NEW ORLEANS

Averaging Brees’s last 4 (healthy) seasons at San Diego as a baseline, his typical performance yields the following stats. (Notes: YPA does not yet factor in sack data. Sack rate is defined as sack yards per pass attempt.)

       CMP%      YPA     YD/CMP    INTRATE    SKRATE
-----------------------------------------------------
BREES 62.6 7.0 11.1 0.030 0.40

New Orleans’s 2005 passing stats were:

       CMP%      YPA     YD/CMP    INTRATE    SKRATE
-----------------------------------------------------
NO 05 55.6 6.5 11.6 0.043 0.47

Replacing NO’s 2005 passing stats with the forecast stats due to Brees arrival we get the following projection. Below the projection are the actual 2006 stats for NO.

       CMP%      YPA     YD/CMP    INTRATE    SKRATE
-----------------------------------------------------
PROJ 62.6 7.2 11.6 0.030 0.44
NO 06 63.0 8.0 12.4 0.020 0.22

Brees had a career year, and our projections are sure to undervalue his contribution to NO’s improved record. By using a win model based on efficiency, we can compare the projected wins “added” by Brees and the actual wins added. (Sack data is now factored in to pass efficiency.)



How to read the chart: The first data column lists the actual efficiency stats of NO in 2005. The expected wins given their stats in ’05 was 4.7 wins, although they only won 3. The second column lists the efficiency stats we would project with the assumptions discussed above. The third column lists the actual stats for NO in ’06. The next column, titled +QB wins, indicates how many added wins our projections would attribute to the arrival of Drew Brees in NO. The final column lists the actual added wins attributed to the actual performance of the entire Saints team in 2006.

In the case of Brees, the assumptions would have underestimated his impact. Brees had a career year in ’06. The projection would have estimated +2.0 added wins attributed to him. In actuality, his contribution was almost double the projection at +3.9 added wins. Other team improvements added another +2.7 wins.

I think the primary reason for the disparity is the arrival of receivers Hopper and Colston, and an amazing 23.3 yards/catch from 2nd year receiver Henderson. I’m sure Brees had a lot to do with their success, but his new receivers’ success would have been impossible to predict. At the very least, our projections can say how many wins a quarterback could add to a new team, given that he has a career year--a best case scenario.

STEVE MCNAIR IN BALTIMORE

Averaging McNair’s last four (healthy) seasons in Tennessee, we get this baseline:


         CMP%      YPA     YD/CMP    INTRATE    SKRATE
-------------------------------------------------------
MCNAIR 61.6 7.1 11.6 0.028 0.33

Baltimore’s passing stats in 2005, split between Kyle Boller and Anthony Wright, were:


         CMP%      YPA     YD/CMP    INTRATE    SKRATE
-------------------------------------------------------
BAL 05 59.4 6.0 10.1 0.037 0.49

Replacing BAL’s passing stats with the projected stats due to McNair’s arrival, and the actual stats from 2006 are listed below.


         CMP%      YPA     YD/CMP    INTRATE    SKRATE
-------------------------------------------------------
PROJ 61.6 6.5 10.6 0.028 0.33
BAL 06 63.0 6.5 11.3 0.025 0.21

McNair was kind enough to match our projections much better than Brees. Unlike Brees, McNair was working with a receiving corps in Baltimore that was unchanged from ’05 to ’06. The biggest difference between the projections and the actual stats was in the sack rates.

We would have predicted that McNair would bring +1.8 added wins to the Ravens in ’06. He actually performed well enough to contribute +2.3 added wins, but “about 2 wins” is good enough for the back of the envelope. The biggest difference for the Ravens between ’05 and ’06 was in the defense’s interception rate which added +2.7 wins to the bottom line.

Although the projections are well in line with the actual results in ’06, we still may be discounting McNair’s presence. The difference between the 10.2 projected wins and the 13 actual wins might be due to his three game-winning 4th quarter drives against SD, CLE, and TEN. All three games appeared out of reach to Baltimore fans who were accustomed to the pre-McNair passing offense.

RULES OF THUMB

When I was a fighter pilot, I loved rules of thumb. At the speed of sound I didn't have time for a lot of math, so quick, simple, but accurate estimates helped prevent me from running out of gas. Here are a couple quick and simple rules for estimating a quarterback's statistical impact on his team's wins.

Completion percentage directly affects our model's most important statistic, pass yards per attempt. Using the league average yard per completion average of 11.5, every point of completion percentage adds 0.115 yds/att. (11.5 * 0.01 = 0.115). Every 0.115 yds/att added to pass efficiency adds 0.18 expected wins. About a 5 point increase would be needed for an extra win based on completion percentage alone.

Interceptions are also critical. Based on the league average of 519 pass attempts, every interception increases the interception rate by 1/519 = 0.18%. In turn, every 0.18% increase in interception rate reduces expected team wins by 0.11 wins. About every 10 interceptions throughout the season results in 1 fewer expected win.

In the next post I will apply the projection model onto the Miami Dolphins' new quarterback, Trent Green.