I keep seeing 49ers linebacker Patrick Willis' name listed at the top of defensive player statistics the last few years. He led the league in tackles in 2009 and 2007, and was second in 2008, but does this mean that Willis is really a top player?
Most fans understand that the tackle statistic is not a very good way to measure a defender. Weaker defenses tend to give up longer drives, giving players more opportunities to make tackles. So in a perverse way, more tackles can be a bad thing. If a defensive back has a lot of tackles, it may be because he's being thrown on successfully. Plus, certain positions get more tackles by the nature of team defense. Middle and inside linebackers will naturally have the most tackles by virtue of their role and where they are at the snap. If you scan down the list of the season leaders in tackles, you're likely to see a simple list of each team's central linebacker, assuming he was healthy most of the year. So how can we tell if Patrick Willis is really that good using just tackle information?
An Idea from Baseball
Baseball faced similar problems with defensive statistics. Until recent years, fielding skill was measured solely by the Fielding Percentage stat, which is a player’s number of put-outs and assists divided by his total of put-outs, assists, and errors. It’s basically a player’s “non-error rate.” This is a flawed way of looking at fielding for many reasons. For one thing, you can't make an error if you can't get to the ball.
In 1977 Bill James revolutionized fielding stats with the invention of Range Factor (RF). Say that for the major leagues as a whole, the shortstop position typically accounts for 20% of its team's putouts and assists. Assuming a relatively even distribution of fielding opportunities, a shortstop who creates significantly more than 20% of his team's outs could be considered to have better than average range and skill. And a shortstop who has significantly fewer than 20% could be considered to have below average range and skill. It’s elegantly simple and compellingly useful.
Tackle Factor
(I bet you know where I'm going with this.) What if we looked at the proportion of all 49ers tackles for which Patrick Willis was given credit? San Francisco logged a total of 832 tackles in the 2009 regular season, and Willis got credit for 114, a proportion of 13.7%. Willis is an ILB in a 3-4 scheme, and in 2009 the ILB position in all the NFL’s 3-4 schemes accounted for 21.5% of a team's tackle total. Because there are two ILBs on the field at once, a single ILB could be expected to average half that, or 10.7% of a team's total.
Willis' 13.7% compares very well with his position’s expected tackle rate. His ratio of tackle percentage compared to the expected percentage for his position is 13.7/10.7, or 1.23. In other words, Patrick Willis has a 'Tackle Factor' of 1.23; he makes 23% more tackles than you'd expect from his position, which tells us a lot about his ability to shed blocks, get to a ball carrier, and make a tackle.
To compare Willis to other players we can follow the same process. Redskins MLB London Fletcher notched 95 of Washington's 804 tackles in 15 games last season. Over a full season we could estimate he would have 16/15 * 95 = 101 'season-adjusted' tackles. Fletcher's adjusted share of the Redskin's tackles would be 101/804, or 12.6%. The MLB position in a 4-3 defense averages 11.9% of a team's tackles, making Fletcher's Tackle Factor 1.06.
Shortcomings
There are a number of shortcomings with TF. For starters, it tells us something very different about defensive backs than for linemen and linebackers. Just like total tackles, a weak pass defense would increase the proportion of tackles in the secondary. It still may tell us something about safeties, however. If a safety is making a very high proportion of his team’s tackles it may mean he’s a standout in an otherwise weak defense. We could also modify the stat to count only run plays, which might be even more illuminating.
TF penalizes players who are not every-down defenders. For now, it is adjusted for games played, but not for snaps on the field. Ideally, if we knew how many snaps each player was on the field we’d get a more reliable stat. Because so many players are not every-down defenders, the average TF is not 1.0. But on the other hand, if a player is not worthy of playing every down, that alone tells us something about his ability.
Baseball’s Range Factor suffers from many of the same issues, but it was nevertheless considered a quantum leap forward in defensive statistics. I think TF could also be a step forward despite its flaws. Defensive baseball stats have evolved significantly in the generation since RF was invented, and the concepts Bill James set forth underlie each new development.
Applications
So what can TF tell us now? The other night I came across a post at Pro Football Talk that ranked the free agent LBs available this year. Here are the available players and their 2009 TF numbers:
Free Agent | TF |
Karlos Dansby | 1.01 |
Gary Brackett | 0.84 |
Jason Taylor | 0.68 |
Keith Bulluck | 1.12 |
Scott Fujita | 1.07 |
Joey Porter | 0.82 |
Tully Banta-Cain | 0.70 |
Larry Foote | 0.86 |
Antonio Pierce | 0.74 |
In this group, Keith Bulluck looks like the better tackler according to TF, followed by Karlos Dansby. Jason Taylor is not an every-down guy anymore, and that’s reflected in his tackle numbers.
Tackle Factor: the ratio of a player’s proportion of his team’s tackles compared to what is expected at his position. Pretty simple. We can improve it by putting it on a per-snap basis and possibly by limiting it to run plays. One obvious improvement I’ve already made is to count assists as half a tackle. (The free agent TF numbers above include assists.) We can do much more too, such as giving extra credit when a tackle was for a loss or whether the play was a “success” (defined by whether it resulted in a positive or negative change in Expected Points (EP). There could be opponent adjustments or adjustments for a defenses' overall strength. This would reward the best players on the best defenses and not penalize good players surrounded by better players. It's really just a matter of the getting good data.
For reference, here are the proportions of tackles (plus one half for every assist) that each defensive position garnered in 2009.
Position | 3-4 DEF | 4-3 DEF |
CB | 9.3% | 9.5% |
DE | 6.0% | 6.6% |
DT | - | 5.6% |
ILB | 11.0% | - |
MLB | - | 11.9% |
NT | 6.4% | - |
OLB | 7.0% | 9.9% |
S | 10.5% | 10.0% |
Interesting as always.
Seems like tackle factor is great but only tells part of the story. If we could add TF to the passing element (something like passes defensed/passes targeted) we'd have a more complete picture of each defender.
This is a good start - I am currently working on trying to calculate for 2009 how far from the line of scrimmage each defender's average run & pass tackle is - I hope that will show which guys are stopping plays vs. allowing the offense to succeed.
www.IDPAuthority.com
I love this metric, not necessarily because it is so descriptive but because it is so simple. I agree that adding an additional piece about passing defense (e.g. fraction of total team passes defensed) would give a more complete picture of a defender. There are many nuances that a metric like this misses, but at least the errors are all omissions rather than commissions, and I like that it rewards players who stay on the field. Taking a trend of a player's score in this type of metric over the past several seasons might give an idea of who is a rising star and who is wearing down, and therefore who should be pursued as a free agent. You could also find the ratio of this factor to salary to find deals. Clearly not all defenses operate the same way (4-3 MLB's appear to make more tackles than both 3-4 ILB's combined; a SS on one team may act more like a LB while on another he may play only on the hash), yet for Occam's Razor value, this is a great start.
Should the percentages of tackles in Table 2 above add up to 100%? They seem to only get to 95% or so... Maybe I'm not understanding that table - if so, disregard my comment about 3-4 vs. 4-3 linebackers.
Larry Foote might not look good on that list. But everything is relative. Relative to the guy that manned his position in 2008 - Paris Lenon - Foote was just fine.
Will-That's due to tackles on special teams and offensive turnovers. At this point, I'm not sure which tackles are which, so I have accept the discrepancy for now. (The data is not from the play-by-play yet, but from nfl.com defensive player stats.) When I do implement TF using p-b-p data, it will take care of the discrepancy.
You've got to be careful with the original data though. I read something a few years ago about how poorly stats were kept in the NFL and as an example it pointed out that the Colts defense had more solo tackles than they had plays.
Brian,
Great work, as usual. Two things:
Do you know what constitutes a tackle when it comes to forcing a ball carrier out of bounds? Does it depend on the amount of pressure (i.e, a nudge vs. a shove)?
Also, I can see how this stat could be misleading, for the same reason you can't just look at INTs and passes defended for CBs. If you're a dominant outside linebacker, the opponent's running game plan might be to constantly attack the opposite side of the field, severely limiting that player's tackle %, but not necessarily his impact on the game. Not sure if there'd be any way to split up strong vs. weak side or left vs. right using your play-by-play data, but it might help. Nevertheless, it's a great idea.
Unfortunately, an out is an out, but a tackle is not a tackle, and range is a very limited portrait of a player. A good linebacker might work in unison with his teammates and instead of bowing out and attempting a tackle, instead penetrate and string the run wide, or engage the lead blocker and allow another player to tackle. And I think that's just the tip of the iceberg of potential critiques. Interesting, but I doubt it will tell us much about a player's actual ability.
True, not all tackles are equal. However,
1. every single tackle is better than no tackle, no matter where or how it happens. And,
2. Every single tackle is the very first tackle a defense can make, or else the play would have already been over.
An out is not always an out. There are fielder's choices, getting the lead runner or trail runner, catching laser line drives or a soft pop up in foul territory. Each play is equal in the eyes of range factor. It seems tackle factor is no more flawed.
I'm not sure a standard deviation wouldn't be a good thing to know, just for some relativity on the numbers. Of course, 12% is always going to be better than 10%, but how much better is that extra 2% (for each position).
I think this is a great way to start some better analysis on defensive players besides absolute numbers (like, 80 tackles, because, like you said, depending on the situation that could be a good or bad thing).
Is the "Interception Factor" coming next?
Seeing Jason Taylor so low, I wondered what would happen if you counted a sack as two or three tackles.
I think I'm a little more concern with the distortion of an above average player on a below average defense. I don't watch baseball at all, but my understanding is that this stat wouldn't be influenced by the rest of the team, right?
SF was ranked #10 in your metric last year, so Willis probably isn't getting any particular help there, but imagine what his TF would be on Cle, StL, or Det?
I don't know any way to control for that, other than to say "Player X has a TF of Y on a Z ranked Defense." That logic makes Foote look pretty bad.
I wonder what the TF distribution looks like for bad teams as opposed to good ones? Do safety and FS have higher TFs? Is there an ideal, instead of league average, TF distribution among the good teams?
1. every single tackle is better than no tackle, no matter where or how it happens.
An incomplete pass or pass not attempted is better than a tackle after a completed pass. A tackle from one player that comes after that players has abandoned his assignment is also not as valuable as a tackle not recorded from one player, but recorded by another because the first player did his job.
2. Every single tackle is the very first tackle a defense can make,
Only within the definition of the play as it happened. If the person who ultimately made the tackle did something else, like contain the backside, the tackle could have been accomplished earlier.
Seattle had a player named David Hawthorne who would record a ton of tackles, and he did because he was poor in coverage, took long angles to the ball carrier, and was not assignment correct. He recorded tackles, but his tackles came at the expense of better outcomes.
Does this number adjust for pass/run ratio? For example: A DE has a lot more tackles on run plays than pass plays. You have to adjust because some teams face a lot more run plays than pass plays and vice versa.
Defensive stats can be perverse.
As James has pointed out (since he's been mentioned) superior counts of good plays can actually indicate bad play -- for instance, a league-top number of double plays generally indicates poor infield players rather than good ones, because poor infields have more runners on base; first basemen with superior assist numbers generally are immobile fielders who can't catch the ball and get over to the base themselves so they have to lob to the pitcher, etc. (a couple of James' examples).
And baseball is the simplest sport to analyze because its stats record individual actions that occur in sequence, the number of plays is limited and consistent, and defenses are pretty uniform.
Football has 11 (really 22) players all interacting simultaneously, applies different systems of offense and defense that affect players' numbers, etc., and so is a whole lot more complex to analyze by D numbers.
This is not a criticism of TF. It's just saying that analyzing D stats will be a challenge. Good luck!
I think you're on to something and this definitely has some value, although I think it has less value than it does in baseball. In baseball, the distribution of fielding opportunities is almost random. You can argue that a strong pitcher will give his infield more fielding opportunities (on ground balls) and a poor pitcher will give his outfield more fielding opportunities (on fly balls), but other than that, the actual position receiving the fielding opportunity is so random.
This is quite different on a football field. In rare cases, a defensive line can be so strong as to take tackling opportunities away from the defenders behind them. More frequently, I see defensive lines and linebacker corps that are so WEAK that they actually give more tackling opportunities to the secondary. This effect is particularly acute in run defense. There can also be secondaries that are so strong that they give tackling opportunities to the DLs and LBs (because the opposing offense is reluctant to throw against them).
I agree with Jim Glass above that analysing football is incredibly more challenging than baseball, because of all the range of interaction between all players.
It is in my opinion a mistake to use statistical analysis for purposes of evaluation of individual players, both in defence or in offence. Credit of a play cannot be attributed to one single player (a QB or a RB, typically), not even easily distributed among players.
Concerning Tackle stats, I think they must at some point be associated with the number of yards allowed in the given play.
I love this stat!
Brian,
Might not a player playing on a good defensive team get "cheated" of some tackles as his team mates "steal" his tackles. Likewise, playing on a poor defensive team, a player may get to make tackles his team mates couldn't make.
What impact does playing on a good/bad defense have on the Tackle Factor, and might you be able to account for this? Like a scaling factor based on the quality of the rest of the defenders around you.
Also - if Dansby has just a 1.01 Tack Factor, doesn't that imply that Bill Parcells over paid? Or maybe Parcells sees something he likes, and doesn't value TF highly.
Yes. Other players can 'steal' tackles from another guy, say a really good DT can eat some tackles that would otherwise go to the LB and vice versa.
They have the same problem in baseball, where one fielder with a lot of range eats up the put outs that would typically go to an adjacent fielder. They have adjustment systems for this, but I'm not an expert on them. They also have ways of adjusting for park factors (that thankfully we don't have to worry about in football.)
Oh, also regarding Dansby... 1.01 turns out to be solidly above average (as it's now formulated). Most guys do not play every down, and the numbers above do not filter out special teams tackles (or offense tackles following turnovers) yet. Right now I've got him at 27th in TF out of 134 LBs who played in 6 or more games.
Brian,
Thank you for responding.
It would be interesting to compare "full time (every down)" players.
Is there any way to get a "# of downs played percentatge"?
Not in the data I have, unfortunately. The only place I've found # snaps publicly available is at a website called profootballfocus.com. But I'm not sure how reliable their numbers are. Ultimately, that's my intent with TF-->make it a per snap stat.
Hey Brian, as usual an interesting read.
Just wanted to add that sacks and forced fumbles are (last I checked) not counted as tackles. I think it makes sense to include these in any "basic" version of this stat, and not just in a special weighted expanded version. Both, like tackles, involve stopping offensive action and should be counted accordingly. After all, does it make sense to penalize a 100-T 10-S 5-FF MLB who stops exactly as many plays as a 115-T 0-S 0-FF MLB?
(Unfortunately strip-sacks are recorded as both a sack and a FF, so adding up T+S+FF will double-count these stops... but in the absence of play-by-play data, this discrepancy should be pretty small.)
I suppose, if you're using full tackle data and not just run plays only, the same argument can be made to also include INTs and PDs. PDs in particular are unfortunate since they seem to be assigned arbitrarily, but I think it does still add to the picture.
Hmmm, I guess offensive penalties forced would also then meet that bar. Those I think you can safely ignore without play-by-play data -- they number < 10 per team a game, of which only 1/3 - 1/2 are offensive, of which only a fraction are forced. ~50 "stops" a season by penalty per team isn't insignificant, but won't affect many players' numbers.
What do you think?
One note on TF by snap: not all snaps are created equal, as I expect the tackles-by-position distribution to vary in say dime packages vs goal-line defense. A part-time player thus won't be measured fairly even going by snap if he's primarily a sub-package player. For that matter, if a part-time player is a 1st/2nd-down or a 3rd-down only player, I'd suspect the tackle distribution to vary quite a bit there too. However, I'd guess a snap-adjusted TF probably does work for older or rotation players who split snaps non-situationally (e.g. by series).
Strong work, Brian. Just caught a link to this post today.
You'll want to strongly consider using some form of stadium factor in your next iteration of these stats.
Despite the good work that the NFL's statistical coordinator has done since 2007 in trying to bring uniformity to the solo and assisted tackle statistic, there are still very significant differences in how each stadium's stat crew tallies solos and assists.
Some crews (eg STL, ATL, JAX) are extremely stingy with assisted tackles. Others (eg NE, CIN, CLE) will give them out in bunches, often choosing to give two assisted tackles instead of one solo and one assisted.
We've been tracking percentage of team solo tackles and tackle opportunity at FBG for some time now and would really love to see better tracking of tackle numbers so that metrics like what you've attempted here can be all that they could be.
And getting reliable snap data to allow per snap metrics would also lots of improvement to defensive stats.
We may never be able to fully drill down through the team nature of defensive football to get metrics that don't have to be considered with a healthy dose of context, but here's hoping efforts like yours will continue to get us closer to that goal.
What about a stat for tackle efficiency? Like contact rate in baseball, shouldn't there be a stat showing how effective a player is at tackling?
I remember reading an article about defensive backs on the issue a few years ago. Basically, it said that while many defensive backs might look like stars on the surface because of great INT stats and tackle totals, they might actually be underperformers (and vice verce for underrated guys). They used a stat like tackle success rate, and found that Jim Leonhard was one of the NFL's most sure tacklers, yet got very little credit as a solid player, while Antonio Cromartie-- lauded as an elite corner-- whiffed on many of his attempted take-downs.
Great article, but the percentage table doesn't seem to add up? For a 3-4 defence it totals 94.0% for a 4-3 it totals 95.10% - why is it not 100%? (Special teams tackles included, perhaps?)
Bear in mind that this is hugely affected by the Defensive Line. If you have a monster inside DL, teams are not going to be ploughing up the middle at you, is that going to make your ILB or MLB look weak? and vice versa. I like the idea but I'm not sure it can be so easily broken down into a single value, much like a similar pass defence factor for DBs since teams actively throw away from premium defenders. It's all a bit hard to decouple