Ray Rice, Matt Forte, Payton Hillis, Marshawn Lynch, and Arian Foster were all due to become free agents this off-season. Rice and Forte have already been franchise-tagged, and Foster is a restricted free agent. Still, the question of what they're worth in terms of salary and cap space remains.
A couple months ago, I took a look at the safety position and how free agents are paid. And more recently I took a look at how QBs are paid with the intent to eventually establish a rational framework for how they should be paid according to their actual production in terms of wins and points. Like the prior analyses, my primary goal for now is to get a feel for the market.
As in all individual player analysis, it's worth stating up front that football is always a team sport. Like all other positions, a RB's stats are not his alone, as they depend heavily on offensive lines, opponents, scheme, situational variables, passing game strength, and other considerations. We can account for the situational variables with the EPA and WPA models, but we must rely on the tendency for factors beyond the control of the individual player to even out in the long run. They may not even out for any individual RB, but in aggregate, the idea is that some RBs will have above average lines, some will have poor lines, and collectively we can roughly estimate the financial value that a RB brings his team based on production.
First, let's take a look at how RB salaries are actually determined. There are two graphs below. The first plots career total rushing yards vs. career total cap hit. The second plots career Yards Per Carry (YPC) vs. career cap hit. The data include years 2000-2009 and is limited to RB seasons with greater than 6 game appearances and greater than 50 carries. Cap hit is adjusted for NFL inflation. Only players who had a cap hit greater than $1M in year 2000 dollars were considered. This limitation is so that we sample only high-priced RBs, and is necessary to see any relationship at all between performance and pay.
It appears RBs are paid more for total rushing yardage than per play yardage. They're paid about $2,887 per rushing yard, or if we want to throw receiving yards into the mix (which only increases the correlation from 0.89 to 0.91), we get $2,289 per total yard. This would mean a player that could reliably be expected to produce 2,000 total yards would be worth about $4.5M per year in 2000 cap dollars, which is equivalent to $9M in 2011 dollars. That's about the top of the current market for RBs.
But the method I used above aggregates career total salary and yardage, which would tend to reward past performance with future pay and obscure what's truly important--which is current performance based on current pay. So here are the same two graphs, but this time each player's season is broken out as an individual data point.
The relationship isn't nearly as strong, as the correlation drops from 0.9 to 0.3. This is what we'd expect with smaller sub-samples of attempts (divied by year), and the overall slope of the relationship drops as well. Instead of $2,887 per career rushing yard, it becomes $945 per yard. But now there is an intercept to consider. Among high-priced RBs, the relationship is $1.7M + $945*rush yd. This would suggest a RB who could be expected to produce 1,500 rushing yds would be worth only $3M compared to $4.3M in 2000 cap dollars using the aggregate model above. If we want to look at our 2,000 total-yard RB from above, the relationship would be $1.8M + $696*tot yds, for a total of $3.1M instead of $4.5M.
But as I've harped on previously, total yardage is a not a good way to measure RB effectiveness, even in the aggregate. Total yardage is primarily due to attempts, which is beyond the control of the player and usually a function of game situation. So next let's look at how advanced stats relate to salary.
The first graph plots Expected Points Added (EPA) per Play by adjusted cap hit. The second graph plots Win Probability Added (WPA) per Game by adjusted cap hit. And the third graph plots Success Rate (SR) by adjusted cap hit.
You can immediately sense that the relationship between current production, either in terms of points or in winning, is not strongly related to current salary, at least in terms of current year cap hit. (In fact, when you aggregate career stats and pay, as I did in the QB analysis and as I did above for yardage production, the relationship makes even less sense. The more a player is paid over his career, the worse his performance.)
I thought that perhaps the randomness of lost fumbles was contributing to the apparent disconnect between salary and performance as measured by EPA and WPA. If coaches and GMs understood how random fumbles are, perhaps they are properly discounting that when paying RBs. But after removing all fumble plays from the data, the relationship between pay and performance barely budged.
If I were reading this analysis, the first thing I would think is that these "EPA" and "WPA" stats must be bunk. GMs know what they're doing and they pay for total yardage, not some kooky made-up stat like EPA. The second thing I would think is that RB performance is so dependent on teammate performance, namely that of the offensive line, and on other situational factors, that it's not fair to even have "individual" stats for RBs.
I'd counter that the EPA and WPA stats fit just fine with other positions, including QBs and safeties. Further, although I agree RB performance is highly dependent on things like line play, so is passing performance. In fact, the case can be made that QB stats would be more dependent on teammates than RB stats. QBs rely on both their line and their receivers, while RBs depend (almost) exclusively on their line. The criticism that stats don't capture a RB's pass-blocking ability would be valid, but I don't think it explains the disconnect. (Does anyone believe Cleveland gave a 29-year old Jamal Lewis $10 million guaranteed over three years in 2008 because he was such a great pass blocker? Or was it because he had gobs of career rushing yards?)
I'd also say that if running outcomes depend so much more on other factors beside RB ability, then why pay high salaries to any RB at all, regardless of how good he is thought to be? Wouldn't a GM prefer to figure out what those factors are and put his limited resources there?
Either the running game just isn't as important as coaches believe or its outcomes are much more random than the present salary scale suggests. Among "qualified" players (the ones that have enough carries that their stats aren't excessively noisy), the difference between the 25th and 75th RBs in career EPA/G is 1.3 points per game. But among QBs, the difference between the 25th and 75th percentile players is 4.7 points per game.
So what kind of inferences can we make? Keep in mind nothing here is solid; it's all first cut stuff. But I think these numbers support several ideas.
First, GMs pay RBs based on rushing and total yardage, which is driven predominantly by the number of carries or touches. These are beyond the control of the RB. It could be that coaches and GMs evaluate RBs poorly, or at least not as well as they do QBs. Despite the claims that insiders don't put much weight in stats, they apparently do. They just use the wrong ones, like total yards. I hear a lot of nonsense about how so-and-so has good "vision," "instincts," and "ball skills," but ultimately pay comes down to past yardage totals.
Second, the very top RBs are generally overpaid. Whether you believe RBs have much control over their individual statistics or not, the present relationship between pay and performance is either barely existent (in the former case) or unwarranted (in the latter case). The point here is that GMs and coaches how important it is to have have a top RB on the roster. RB outcomes tend to be dominated by situational and team factors. Compared to QBs, top RBs are relatively inconsistent from year-to-year in terms of true production (as opposed to total yardage), and the importance of the running game in general is less than generally believed.
As it stands today, the league's very top RBs consume nearly 10% of their team's cap space. I believe this is too high, perhaps by a factor of 50%. More work would need to be done for a solid estimate, and it ultimately may be impossible because we'd have to untangle the relationship between the RB and his line. But based on the relative importance of the running and passing games in terms of WP, and based on the relative inconsistency of year-to-year RB performance, 50% intuitively seems like a good first cut. Instead of commanding say $9M per year, they should more rationally get about $6M per year.
Subscribe to:
Post Comments (Atom)
Could the pay/performance correlation be due to running backs wearing down fairly quickly? Meaning, by the time their rookie deals are up and they hit their first open-market FA contract, their bodies are so beaten down by the constant pounding that is taken at that position, they are almost guaranteed to under-perform compared to their past numbers (which is usually what their FA contract is based on).
First, I think RB performance is extremely variable, in large part due to injuries and team performance. So with that said, I'd be very interested in seeing these plots but only looking at a RB's performance in the season before he signed a new contract and compare that to the cap hit. This will drastically reduce the number of points, and more closely matches the data from which a GM is making a decision.
An interesting situation to analyze would be the Denver Broncos trade of Clinton Portis to Washington for Champ Bailey. You traded a top notch (supposedly) running back for a shut down corner. Quantitatively, did the trade work out for the team giving up the top notch RB?
Great article. My only comment: in the big picture, how well a RB plays only affects one facet of how much he is actually paid. How that running back is percieved by the lucrative fan base-how he helps sell tickets/the brand of the franchise-directly impacts his pay. Although he's certainly not worth the money (as you've proven) from a yardage point-of-view, the RB is worth the paycheck in terms of how he impacts the team from a much broader financial angle. After all, football is a business. period.
The "superstars sell tickets" angle has been looked at in other sports, like baseball, and mostly debunked. I don't know if it's the same with football, but I'm pretty confident in saying that if there is an effect, it's swamped by the financial benefits of simply winning football games.
I agree that RB's are generally overpaid, and that they have a weakening role in the NFL. But I do think you are discounting GM's and market forces a bit:
- While teams don't require top tier RB talent, having a talented RB does help a team, and the market cost makes it hard to find a RB that can improve a team without paying. Similarly to a corner back, while not required to make a good team, they make a good team better, but finding talented corners is expensive.
- You give GM's too little credit, looking at history of GM choices and interviews, saying they only look at total yards is just incorrect. That's like saying teams only look at a QB's total yards or a linebackers total tackles. To say they only look at these things is underselling them. To believe that mentality is just incorrect, they obviously look at the film, yards per carry, yards after contact, dropped passes, blocking ability, and ability to escape pressure. Basically from the research and interviews I've done, I see there is little evidence total yards are the main category GM's look at. GM's aren't quite stupid enough to only rely on basic stats, no matter how much we think they do. So while the graphs show a higher correlation with total yards, take into account that the top 10 in yards per carry also only feature two RB's who top 15 carries a game, where as teams want a RB who can handle 15+ carrier a game, but because when you top 15 carries, your YPC drops. So obviously those with a higher YPC will have less carries, to expect those who get more carries to have the same YPC as those who have less carries misses a huge fact. When it comes down to it, GM's want RB's who can both handle a lot of carries and have a good YPC. This is why they want a MJD and Fred Jackson who can average 15+ carries a game and 4.5+ YPC over a RB who can only handle 10 carries a game and can't be trusted to run a large number of plays if that is what the game plan calls for.
- You mentioned this, but I would like to expand, the RB is one of the more intertwined positions in the NFL, especially on offense. Relying on far more thing than most positions. Whereas the QB can chose who to throw to, can audible, and while relying on an offensive line, a quick release or decision making can compensate for that. A wide receiver can't chose his targets but he has control over what he catches. A running back can't chose when there is a run play, he doesn't chose the blocking assignments, the direction of the run, and has no control over the lineman themselves getting down field to block. Really only offensive lineman is tied so closely to other positions when it comes to the play production, because he can block perfectly and all the other lineman can fail, leading to a play failing, yet he fullfilled his assignment, similarly to a RB getting hit right when he gets the ball, or there being no gaps in the line when get takes the hand off. I do believe there are statistics that can analysis RB's individually, but I do believe YPC and total yards are tied closely to more factors than other positions, but are still good metrics when in use with others.
- Also I would like to point out your graphs would look similar to what you'd see from QB's with the top paid QB's leading the league in yards, while guys like Rodgers, Schaub and Romo near the top of the league in yards per attempt. Same could be said for WR's, with guys like Johnny Knox, Malcom Floyd, and Jordy Nelson near the top in YPR but not near there in payment.
Overall good article, you posted this same type of article when Adrian Peterson got his contract, but the contracts this season likely won't be that big, as Foster's and Lynch's contracts have shown. So I agree with your premise, RB's are overpaid, but I do think there is more to it than "just pay them less." Once again, nice article, glad to see Advanced NFL Stats hasn't shut down for the off-season.
I would expect, although I'll have a look at it sometime, a kind of Laffer Curve effect on paying running backs v team wins. My guess is we ought to be seeing a peak cap hit for rb salary where any increase in salary then hurts the rest of the team.
I think Denver certainly got the better end of the deal. Bailey is still playing at a high level while Portis was effectively done in 2009
Some teams that have gotten by in recent years without "star"* running backs: New England, Green Bay, Indianapolis, both New York teams, Pittsburgh and Baltimore. All succesful franchises with respected front offices - have these forward-thinking organizations figured out the same thing?
* - by "star" I mean a running back who is both an elite player and a highly paid one. Obviously Ray Rice is an elite player, but Baltimore has so far been able to avoid paying him as one...
Regarding your third paragraph:
If a Back's YPC, and more importantly SR starts dropping as a game progresses, wouldn't game theory dictate that it is time to stop calling run plays at that point to try to maximize overall yards per play?
And if game theory dictates that you need fewer running plays, is it really that vital to have a running back who can handle more that 15-20 carries/game?
I don't think you can really count Pittsburgh or Baltimore. Ray Rice and Rashard Mendenhall would certainly be considered "Elite" at this point, but they aren't getting paid as such because they were still in their rookie contracts through the 2011-2012 season.
Ray Rice is likely going to get his star pay this season. They've already Franchised him, which is worth about 7m... double his previous contract's total value.
That is one area I'd disagree on. Mostly because it totally depends on the situation. Just saying Game Theory says you should stop calling run plays is a gross misuse of the term since it should be applied situationally. The best example may have been the Denver Broncos this season, where in most games the running game had nearly the same yards per attempt as the passing game but had a higher Success Rate. Obviously when looking at this, and when applying situational topics like Game Theory, it's best to look at things in smaller bits than speaking with broad strokes, otherwise Game Theory would suggest you pass on every down, all game, but reality would then begin to beat down that team unless they had a top tier group of offensive players, otherwise it would just cause the team to likely loose.
Mr. Burke's pricing does not consider the pounding the RB's take. Their handed the ball and expected to get thru nearly 2000 pounds of defenders, and thats just the front 7. That's where the extra money is due them!
GMs could be paying running backs for something other than on-field performance. Other than quarterback, running back is the top star position for fans and media attention. If a running back attracts sufficient attention in terms of season tickets sales, souvenir sales, or sufficient buzz at the owner's country club, then maybe there is value other than on the field. Maybe the real stat you should look at is how many GM's get fired within 2 years of signing a big-name running back.
Do you know where I could find an article on the "superstars sell tickets" angle? Google is pretty unhelpful, and I'm not very familiar with many baseball analysis sites.
Ditka wedded his fate to R.Williams. Worked out pretty well, didn't it?
IMO RBs are overrated because of tradition (they used to mean a lot) and they still represent "what football is all about", getting hit often, getting dirty, etc. It's hard for people to let go of that.
Does anyone know if the "superstars sell tickets" argument as actually mostly debunked?? Looking for an article, or something other than an anonymous poster posting something.
One question: Doesn't RB yards per carry decrease the more carries a running back gets in a game? Is it possible that higher paid RB's get more carries than lesser paid ones thus decreasing their yards per carry while still being better than the replacement level backup that takes their place? Made up example:
Adrian Peterson: 7M salary for 300 carries at 4.1 YPC
vs.
Average Starter: 3M salary for 200 carries at 4.2 YPC
Average Backup: 1M salary for 100 carries at 3.0 YPC
If this were the case, Peterson would look overpaid relative to the average starter according to YPC, but only because you are failing to account for the extra volume that Peterson covers the backups for playing. In a case like this it would follow that if Peterson's carries were decreased to 200, his YPC would rise to say 5, showing his superiority.
Just my play at devil's advocate.
Thoughts on the slew of $10-$20m/yr WR signings with $15-$60m guaranteed? Curious as to what sort of WR value analysis you've done in the past akin to what you've done here - Joe Landers
If you're talking about retaining a star who sells many jersey's, then maybe this factors a small amount. But fame comes quickly with performance. Stars are born each season, sometimes within a few games. Fans are fickle and their attention shifts just as quickly.
Based on this analysis, it seems most efficient to invest all of the offensive budget on QB and OL, hire unproven running backs and receivers, hope for breakout seasons, trade them for champ baileys in the off season, and repeat. XD
Would you say these findings support the idea that in order to get the best value, the Eagles should move quickly to re-sign LeSean McCoy to an extension before the season since he only has around 4,300 career yards at present? McCoy should surpass 6,000 yards by the end of the season and would appear to substantially increase his pay-grade once he reaches that threshold. McCoy is a young, dynamic, all-around workhorse back with high yards per carry and high yards per touch. No NFL RB has accumulated more EPA since 2010.
I wonder if you could do the same comparison using the price tag for a RB plus the offensive line. That would give you offensive ground game production (in EPA and WPA) as a function of the whole investment in the running game.
Sports players are just overpaid nowadays, and some just don't do well.
This would probably provide some insight. However, if you look at a team like Denver last year, the option threat of Tim Tebow is going to heavily affect the overall running game's production, even discarding his direct rushing yardage.
Also, say Tom Brady is injured midway through the season and they have to start using Caleb Hainie as a starter (hypothetically, of course). I have no doubt in my mind that, even though you have not directly affected the O-line or RB options, their production will drop rather significantly, since teams no longer have to worry about defending the pass.
My point is that we can twist the numbers and evaluate any combination of players we want, but the bottom line is that there are just too many variables to isolate any one aspect of the game and assign it a value that has much significant meaning.
I completely agree. This is exactly why Ryan Grant had probably his best WPA/EPA year.
I've read that in baseball, playoff contention/winning sells tickets. Superstars, notsomuch. But that was just chatter on a message forum and nothing I can verify.
Brian-
Might check out
http://www.bloggingtheboys.com/2012/4/6/2910447/dallas-cowboys-key-to-2012-success-passer-rating-differential Similar vein to some of your older posts.
miles
A RB is paid by potential, and they keep getting drafted at lower and lower spots.
all the good player are getting drafted not at there right place can someone please tell me whats going on here.
Angie davis
http://www.balltribe.com/toolbar/24256
If I were a GM, I'd ask my RB if he were willing to forego $1-3mm so I could use it to get him better linemen, which would improve his overall performance, and give him a chance to make a portion up in some kind of performance based bonus.
Assuming the RB goes along with this scenario, he has a chance to not only make the same or more, but the chance to enhance revenue opportunities outside of the team (endorsements, etc.). Overall, it would be a net "win" for RB, Team, and Linemen.....assuming this model weren't utilized by every single team, thus negating its potential value.
It is refreshing to see someone looking at stats to try and garner information about various ideas. I have been trying to do it myself. Would you agree a top five running back is worth over 7 millions dollars?
I have no idea how anyone can say that a running backs total yardage is just because of how many carries they get. That is basically saying that if you put any bum on the field full time and gave him the looks/touches of say a Ray Rice, he would put up 1800-2100 total yards just like Rice does very year.
Maybe I am missing what the author was trying to say, but that is ridiculous.
And while RB may be "overpaid" according to your metrics, every GM for...ever has continued to pay the stars at that position on a second contract. Every single time a star running back comes up for a deal after their rookie contract is up, they get paid. It was true in 1965 and it is still true in 2012.
I am guessing every GM out there as far back as you can go knew somewhat what they are doing in that regard.
Only exception was Marshall Faulk, and we saw how that worked out for Indy. And for St Louis. Here is a good article recapping that deal.
http://sports.yahoo.com/nfl/news?slug=jc-faulkpoliantrade020511
Notice the comments from Warner about Faulk's role in a "high flying passing attack", which some use today as an argument to devalue the running back position. All running back's are not created equal, especially the ones who can catch the ball out of the backfield and stay on the field for all 3 downs.
Elite runners who get paid as such should be able to do it all. Most can't.
This is awesome. Now what is each draft choice worth on average? Although that may not be best measurement because you can replace a draft pick that misses by cutting him and starting over and teams may be willing to miss on 3 draft choices at a crucial position, particularly if they already have a starter. And even if they hit they made trade that player away for draft picks and start again like the packers did many times with Favre's backup... but it would be interesting if there was some type of analysis that could be done to determine how many draft picks it is worth giving up to trade for a given player. Like the most underpaid player. Seems like with a lot of number crunching and top tier analysis like you do here you could estimate who got the better of a particular trade, and what a particular player is worth in terms of draft picks to go and get someone already on a contract. But longevity also would have to play a role as a HB/QB who is 32/42 just can't really be worth trading for as much as that draft pick and selecting a rookie QB in terms of overall value that it adds (sum of the value per play times the amount of plays) as the rookie could potentially give you 10 years where as the old player may only get you 1 or 2 and may be regressing.
A runnigback may succeed in spite of or because of his Oline and effectiveness in passing game.
A QB may success in spite of or because of his Oline
A QB may success in spite of or because of his Receivers
Very hard to separate some things based on stats alone, although stats can at least conclusively give you some details worth considering. stats like those at this site and at footballoutsiders.com do well to separate stats vs opponents in similar situations. But it still doesn't always work due to the great amount of variables and varience. Game tape and giving grades based upon various minor details certainly come into play.
Additionally, you can't expect a completely efficient market since many players agents might not do this type of analysis, and you need a RB to make play action effective in passing game, and the synergy that comes when you have both. Teams will pay less if possible, but may wish to use draft picks and resources such as dollars elsewhere, and game plans and how RBs are used vary greatly.
you flip a coin 100 times and it comes up 60 heads. Is that an anomoly? Were the 10 extra heads the result of "skill", "luck", the "coin flipper" or the "type of coin" What if 1000 people are flipping different types of coins with different weights, wind speed, in different locations with different rules on whether or not you can flip the coin. You can always draw upon correlations, but it does not imply causality. a player is on a winning team, but did he cause the win? Statistics will always be an inexact science. With that being said, in a game where everyone else plays by the same rules, statistics if often an excellent tool to get more value more often then not by making calculated decisions.
Yeah it's crazy. RBs aren't worth it. They don't do it! 3year average lifespan
Miles-Thanks.
New England has proven its a dime a dozen position.
The 3 stats that really matter for running backs are:
1. The percentage of rushes the resulted in either a first down or a touchdown
2. The percentage of rushes that extended beyond three yards when there was more than 3 yards to go to get a first down or a touchdown even if the result was not a touchdown or first down.
3. The percentage of rushes that initially put you in field goal range even if the field goal was not attempted or was missed.
You could also throw 15+ and 20+ gains in there too. I just hate seeing the yards per carry benchmarks used to judge RB effectiveness. That stat completely disregards the situations that RBs are put in throughout the course of a game/season/career.