## More on the Cost of Interceptions

In a recent post I looked at the cost of interceptions in terms of equivalent yards and expected points. In this post, I'll look at them in terms of win probability added (WPA).

In a comment on my Fifth Down post regarding the context of Jay Cutler's 2008 interceptions, Will wrote:

"I know you can calculate a change in WP for a given play; this is how you came up with the best plays of the year for last season. Can you also calculate an average change in WP for a type of play? For example, can you find the average change in WP for a Cutler interception vs. a Favre interception vs. the league average, to see who throws more bad picks? I've long felt that many of Favre's interceptions equate to punts, as he throws it up deep on a late-down, long-yardage situation. By the same token, it might be good to know which passers have the highest delta-WP per attempt, or which rushers most change their team's fortunes per rush."

You can read my response in the original post, but I'll expand on it here. Will was getting a little ahead of me because I'm planning on publishing some neat stuff on individual player WPA (win probability added) later this season.

To make things a little easier on myself, I'll cite Bronco and Jet passing game numbers from 2008, not necessarily Cutler and Favre, but I think they're identical for practical purposes. I'll also calculate the league average.

Denver's 18 INTs cost a total of -1.56 WPA (or, in a sense, lost 1.56 games ). That averages to -.087 WPA/INT.

New York's 23 INTs cost a total of -2.49 WPA. That averages to -.108 WPA/INT. You could say that the Jets' interceptions were about 20% more costly than the Broncos' last year.

For reference, there were 465 INTs in the league in 2008, costing a total of -46.98 WPA. That averages to -.101 WPA/INT. So on average, an interception costs a team a 10% chance of winning. An interception equates to 3.8 points, 60 yards, or 10% WP lost.

Interceptions are not typically similar to punts. I've read elsewhere (I think at footballcommentary.com) that interceptions are on average returned to about the line of scrimmage. My data shows something slightly different. Interceptions are on average returned to within 8.1 yds of the original line of scrimmage. Removing the 2nd and 4th quarters from the data to account for Hail Mary interceptions, it's 7.2 yds. So, I suppose you could consider a typical interception like an incomplete pass and then a really, really short punt.

But WPA for any particular interception is dependent on a number of factors. Field position and score are obviously critical, so here is a graph of interception WPA by field position, broken out by selected score differences. Despite the noise in the graph, there are some points to be made.

It's interesting how the WPA drops the steepest for when a team is down by 3 points (the red line). Throwing a pick deep in one's own territory when up by 3 points is nearly equally as costly. The bigger the difference in score, whether ahead by a lot or down by lot, the smaller the impact of the interception. The graph makes sense (at least to me)--it's what I'd intuitively expect.

Time is also critical. So here is the same graph, except limited to only 4th quarter interceptions. (WPA is a function of many things--score, time, fld position, down, to go distance--that I can't show everything on a single graph.) It's a little noisier, so I grouped the field position by 20-yard chunks instead of 10.

Again, we see what we'd expect. The tighter the score, the more costly the interception. Tied or down by 3 in opponent territory is where they're the costliest.

### 6 Responses to “More on the Cost of Interceptions”

1. Dr Obvious says:

Interesting. I've read elsewhere that many of Cutlers INTs where on 3rd down last year. Did that play into these calculations? (sorry if this is a double post. I think some angry tube monster ate my first one.

2. Brian Burke says:

Yes. It factors down, dist, time, score difference, and field position.

I'm pretty sure most QB's ints are going to be disproportionately on 3rd downs. The offense is usually a little more desperate for a conversion compared to 2nd or 1st down, plus the defense is expecting a pass.

3. John Candido says:

You're really entering into interesting territory here. My curiousity would be to see whether you can calculate some sort of a "clutch" variable based on the cost of Individual QB's interceptions, as above hinted at, and whether or not such a variable would have a significant coefficient of determination related to your logistic regression model for game prediction. My guess is you will find significance there. Very interested to see how much of a variance it would explain.

4. Brian Burke says:

John-Yes. One idea I'm developing is a leverage index, based loosely on the same concept in baseball WP. (Created by Tom Tango.) The LI would be a great indicator of "clutch."

However, keep in mind that while WP and WPA are great "narrative" stats, they aren't necessarily good predictive stats. They capture the story of what did happen perfectly well, and they're ideal for decision-making analysis, but they aren't the end-all-be-all.

5. John Candido says:

The leverage index sounds like a great measure for the phenomena of "clutch". I'll look forward to when you come out with those individual measures. Given that most games are decided in the first two quarters in a battle to see who can average the best passing yards per attempt, wouldn't a measure of average QB influence over WP be an even more statistically precise measure of the same phenomena? And wouldn't a career or season average of this measure for individual QB's have some predictive value? I would think that this would best be applied to QB's specifically, since they are the most independent position in football and have the largest sole impact on WP.

6. QuantCoach says:

Very interesting. It looks like you are onto something. Love your work.