Cade Massey on Flipping Coins and the NFL Draft

Readers of this site will recall the name Case Massey. Along with fellow noted economist Richard Thaler, he co-authored the Massey-Thaler draft study titled The Loser's Curse. The paper found that, under the previous CBA, "surplus" draft value peaked with picks in the late first round and early second round. Surplus value was defined as the expected performance value above which a team could expect by spending an equivalent amount on a veteran free agent.

Massey has continued research into the draft. His presentation at the 2012 MIT Sloan Sports Analytics Conference outlines his recent findings. (I recommend using IE to view the presentation. Chrome didn't play nice with the video.) The slides from the brief can be viewed here.

If I understand things correctly, Massey has found that:

1. The draft requires skill, as demonstrated by virtue of the superior performance of earlier picks to later picks.
2. Teams are not significantly better than others at the draft, largely due to the fact that:
3. Randomness dominates the process. It is an exceptionally uncertain task to predict human performance years into the future.
4. Therefore, teams are (statistically) roughly equal in ability to draft talent.

This makes sense in a lot of ways. All teams have equal access to information. Personnel talent is diversified and highly mobile across the league. Draft techniques and strategies can be observed and copied readily. Yes, it's true that if a clown were made GM of a team the draft would not appear random. But because team personnel staffs are not clowns (perhaps except a couple), the draft is effectively randomized in terms of successfully selecting quality players.

Massey goes on to point out specific strategies that are good bets in any highly uncertain environment, including the draft.

1. In a random environment, the best thing you can do is increase your opportunities, i.e. number of picks. (Stop trying to milk a 1 in 100 chance into a 1 in 90 chance. Go get yourself two 1 in 100 chances instead.)
2. Minimize the costs of choosing. (Don't pay for the right to think you know more than everyone else.)
3. Don't trust any one opinion. Trust the consensus.
4. Success is largely luck. Failure is largely luck. Success and failure should be evaluated in terms of the soundness of the process rather than outcome. Don't allow outcomes to be attributed to individuals.

I generally agree with the results and their implications, but I have two comments about the Massey-Thaler methodology.

First, a lot of the performance value of each player is highly dependent on opportunity. That is, players drafted highly get more starts, more Pro-Bowl opportunities and other performance achievements merely due to the opportunities granted to them by virtue of when they were selected. It's a causation-direction problem. Are young players successful solely because they are good, or also because their teams have sunk varying levels of investment in them? I believe performance achievements in the Massey-Thaler methodology are mostly due to real player talent, but some degree of sunk investment is involved.

Second, what about the Gladiator effect? High premiums on the very top picks might be justified by the fundamental rules of the sport. An All-Pro QB can't be replaced on the field by two mediocre ones. There are limited numbers of roster spots available, so having multiple opportunities at the expense of top picks might be a mistake. I'd suggest that there's a happy medium, maximizing the number of picks near the peak surplus value at the end of the first round, or wherever the new peak surplus is based on the current CBA.

  • Spread The Love
  • Digg This Post
  • Tweet This Post
  • Stumble This Post
  • Submit This Post To Delicious
  • Submit This Post To Reddit
  • Submit This Post To Mixx

14 Responses to “Cade Massey on Flipping Coins and the NFL Draft”

  1. Anonymous says:

    This is a good, current, article on the topic:

    http://www.footballperspective.com/are-certain-teams-better-at-drafting-than-others/

  2. Anonymous says:

    "Yes, it's true that if a clown were made GM of a team the draft would not appear random" Fortunately, this experiment was already performed on the Detroit Lions from 2001-2008.

  3. Nate says:

    I've got to wonder how strong the 'starts' heuristic is - ostensibly teams with more draft picks are going to have fewer draft picks per start. And, earlier picks will have higher perceived value thanks to higher perceived cost.

    Similarly, it seems like the value of a pick is - roughly - e^-kn so earlier picks are, indeed, more valuable. The 'picks per year' chart in the talk makes me wonder what it would look like if draft position were accounted for.

  4. Trent says:

    I did my econ senior thesis on the Massey-Thaler paper, and as part of it I updated their findings in light of the new rookie wage scale. The CBA is still too new to be able to replicate their methodology completely, but enough time had passed to be able to show that the wage scale eliminated "the loser's curse" (ie made it so the first pick is no longer the least "valuable" pick in the first round, as M&T originally found). This seems to contradict the executive overconfidence narrative that they were trying to push. As Brian indicated above, the real problem with the M&T paper is in how they treat the concept of value, They try to fit drafting football players in the traditional economics framework of marginal cost = marginal benefit. The problem is that in football, relative value is what matters to decision makers, not absolute value. This is why they treat the salary cap as a team's primary constraint, when in reality its the number of players allowed on the field at one time.

  5. Toby says:

    As an avid NFL fan and someone with a passion for economic theory, I check your site regularly because of articles like these. This is great stuff Brian - thanks!

  6. James Kingsbery says:

    As you say, NFL GMs are uniformly pretty good at their jobs. I'd be curious if the findings are true for a slightly different problem - how good college coaches are recruiting out of high school. Since there are many more college coaching jobs, I'd expect less uniformity in judging talent. Any idea if anyone's ever looked at that?

  7. Brian Burke says:

    Trent--Any interest in publishing your results here? Is there a link to your paper?

  8. Steve says:

    Daryl Morey's article (http://www.economist.com/blogs/gametheory/2012/04/predicting-performance-basketball) from The Economist defending the team's release of Jeremy Lin (before they signed him as a free agent last summer) has a great expansion on the Bricklayer vs. Gladiator theme:

    "The structure of professional basketball makes it impossible for teams to give a chance to every prospect who shows some potential.

    For example, Major League Baseball (MLB) drafts some 1,500 players a year. Once selected, they must work their way up four levels of minor-league teams before joining the parent club. This gives MLB organisations the incentives of salmon, which spawn scores of young far upriver in hopes that a handful make the treacherous journey all the way back to sea. Each team can invest in hundreds of prospects and see which pan out, needing only a few winners that “hit it big” to make up for all the failures.

    In contrast, NBA teams cannot hold the rights to anyone beyond the 15 players on their active roster. That makes them more like elephant mothers, who give birth to very few babies and have to gestate them for almost two years. With limited investment opportunities, teams are forced to choose only the players with the greatest likelihood of success, and then give them a long-term contract and a potential path to significant playing time. And no one could have called Mr Lin a high-probability prospect. In the end, he only got his break after he had polished his game for a year in the NBA Development League (“D-League”), basketball’s minor league, and when a series of injuries on the Knicks created an opening at his position."

  9. jarhead says:

    Agreed that there is a lot of luck and randomness in NFL draft picks, and it may be possible to lower (not eliminate) some of the luck and randomness. And scouting is an art, and some scouts are better than others. Two things not discussed are 1) picking the best player available vs picking for need, and 2) picking big offensive and defensive linemen as opposed to picking "skill" players. My feeling is that if a team does not have one of the first ten picks, it will be better off picking the best available linemen as opposed to trying to fill the team's needs with skill players.

  10. jarhead says:

    Another view of where to find value in the draft

    http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2060317

    Criminal Records and the Labor Market for Professional Athletes: The Case of the National Football League

    Kendall Weir

    Hamilton College

    Stephen Wu

    Hamilton College - Economics Department

    May 15, 2012


    Abstract:
    This is the first paper to formally analyze character discrimination in the specific labor market for professional athletes. We observe all 1,273 players drafted into the National Football League between 2005-2009 to determine what effects character concerns, namely arrest records and team suspensions, have on draft status and performance in the NFL. Prospects that have a history of formal criminal charges or are suspended for team or university violations fall between 16-22 spots in the draft. The impacts of character concerns on performance depend on the nature of the issue. Players that have a history of suspension (non-criminal related) start and play in fewer games per season, but having an encounter with law enforcement does not negatively predict performance. This suggests that having a problem with coaches or fellow players is an indicator of future problems, but having a run in with the law does not adversely affect on the field performance. We also find some evidence that conditional on where players are picked in the draft, teams undervalue those with criminal records and overvalue those with other types of suspensions.

  11. Anonymous says:

    it is not so much that high draft picks get more opportunity because of the investment put into them, but rather that high draft picks go to lousy teams. These high draft will necessarily get more starts because there is no in-team competition.

    and of course drafting takes "skill" in that the "better" players tend to get drafted earlier, but what is that skill? one can simply list running backs in order of yards gained that season. It is not surprising that the rb with 2000 yards is better than the rb with 800 yards.

    As for all teams being equal at draft skill, that clearly is not what happened. look at oakland and their record of 1st round busts - 9 years with no probowlers drafted. Look at the lions under millen with his bewildering picks culminating in a 0-16 season. There is also the fact that some teams develop players better than others, thus giving them better draft results.

  12. Unknown says:

    Do your accounting homework, accounting assignments with our accounting homework solution and accounting tutoring help. Our homework assistance helps you to do your accounting homework and accounting assignments.
    Please visit once www.homeworkhelpindia.com or email me on prashant8778@yahoo.co.in

  13. Unknown says:

    One potential flaw in the "starts" metric: the "need factor". The starts metric tends to assume that all players and positions are equal, and that teams are picking and starting based on BPA. They are not. Complete teams are. Incomplete teams pick (and start) based on need. If a team selects a safety in the second round, there is a really good chance its because they NEED a safety and are really hoping they can find a starting safety. Result, that safety is probably going to end up starting that year, because that team had a big gap at that position, but the same safety might not have won a starting role on most other teams.

    Not saying the start metric is totally broken, just slanted.

  14. Anonymous says:

    Good stuff. One aspect I believe was overlooked in the linked video presentation is related to the negative auto-correlation he mentions, which he subsequently decides to call zero. The reality is that if you are measuring success via starts, than a team that has succeeded in drafting historically will have fewer starts to allocate, because they have good players. I think one could make the parallel to the memory of a black jack deck. If you drafted well and picked up Frank Gore in the 3rd round, you're not going to have "starts" to allocate at the RB for a decade, which may explain the negative correlation.

Leave a Reply

Note: Only a member of this blog may post a comment.