Decision Theory in Football

In Decision Theory, there are generally two kinds of analysis. Descriptive analysis is what people actually do, and prescriptive analysis is what people should do. Rarely are the two things the same. For example, when I use the win probability model to evaluate 4th down decisions, I'm doing prescriptive analysis. Trying to explain whatever the heck coaches are actually doing would be descriptive analysis.

To be fair, coaches are not computers. They are subject to all the imperfections of human decision making. In this post, I'll examine some of the ways that coaches may be making decisions, including minimax, minimax-regret, prospect theory, and expected utility. I'll also discuss the potential for how much of a difference a pure prescriptive analysis can make when applied in real games.

NFL Orthodoxy

NFL football has evolved as extremely conservative game. By that I mean that coaches adhere to the wisdom passed down from previous generations and are reluctant to deviate from the established orthodoxy. In the real world, away from sports, this approach usually makes sense. Unlike sports, the world is not bounded by sidelines, end zones, and 15-minute quarters. It is highly uncertain and far less predictable than we'd like to think. It makes sense to adhere to what is known to work rather than try to engineer an optimized outcome in a highly uncertain environment.

But in football, we have the stats. We know the probabilities. And we know the possible consequences. 'Conservative,' as I defined it, is therefore often not the best approach. I think the reason that so many coaches adhere to the same orthodoxy, whether in terms of playbooks or 4th down doctrine, is because they aren't conscious of the level of certainty available to them.


One of the more conservative approaches is the minimax criterion. Minimax says pick the option that assures you the highest minimum utility. Let's say you have the choice between going on a picnic and going bowling. You'd really rather go on the picnic, but it might rain. Your payoff matrix would look like this:

Payoff Matrix

No Rain

If it doesn't rain, the picnic pays off, but if it rains you've lost the afternoon. Bowling is not as much fun as the picnic, but it wouldn't matter if it rains. Minimax says go bowling because 1 is its minimum payoff while 0 is the minimum payoff for the picnic.


Another decision method is known as the minimax-regret criterion. This method seeks to minimize potential regrets. Imagine coming out of the bowling alley and being greeted by a sunny blue sky. 'Darn. Should have gone on the picnic.' In this case, if you go bowling and it doesn't rain, you've gained 1 unit of utility but lost out on 4 units, for a net regret of 3. If you go on the picnic and it does rain, you've gained 0 utility but lost out on 1 unit, for a net regret of 1. If you want to minimize your regret, you'd choose the picnic.

Notice that I haven't mentioned the weather forecast yet. These methods are best relied upon when there is a very high level of uncertainty in the "states of nature" that will determine the payoffs.

Now consider a football example. Say a coach has three plays that make sense for a given situation, and the opposing defense can call one of three kinds of defenses. An example payoff matrix might look something like this:

Hypothetical Football Payoff Matrix

Def X
Def Y
Def Z
Play A
Play B
Play C

Note that this is not game theory. We're not looking for a Nash equilibrium. The offensive coordinator is thinking of the defense as a "state of nature." It's something he has no control over and is difficult to predict.

In this case, both Plays A and B have the possibility of negative payoffs. Play C guarantees at least a payoff of 1, and therefore would be the minimax decision.

The regret method says something different. Assume the defense had called Def X. The best payoff possible given Def X would be 3 with Play C, so had we called Play C there would be no regret. But had we called Play B, we would have earned a -2 payoff, which equates to a regret of -5. In other words, we could have had 3, but we got -2. And had we called Play A, we would have earned a -4, which is a regret of -7.

If we repeat the regret calculation for each possible defense, we get a whole new regret matrix:

Regret Matrix

Def X
Def Y
Def Z
Play A

Play B

Play C

Given this regret matrix, the minimax-regret criterion would look for the choice that assures us of the best worst-case scenario. For Play A, the worst regret is -7. For Play B, it is -5. And for Play C, it's -11. Therefore, we'd pick Play B because it is the least costly in terms of maximum possible regret.

Of course, coaches or anyone else would never actually draw up a matrix and do the math to make a decision. But just like in the picnic-bowling example, our brains are attempting poor analog versions of these kinds of decision criteria, and emotions play a large role.

Expected Utility

What if we reduce the uncertainty in the defense? We can't predict exactly which one we'll see, but we can estimate the probabilities that we can expect each defense. The expected utility of a choice is the weighted average of the possible payoffs. For simplicity, say each defense is equally likely with a 1 in 3 chance. Now we can estimate the expected utility for each play choice. In the example above, the expected utility for Play A is (1/3)(-4) + (1/3)(4) + (1/3)(12) = 4. The expected utility for Play B is 3, and for Play C it's 2. The expected utility method therefore says Play A is the best choice.

The three methods each call for a different decision. Each method is logical and consistent in its own way, but there is only one truly correct method in football, only one prescriptive analysis. Remember, in football we can know the probabilities and the payoffs, or at least have a solid league-wide baseline for them. The expected utility method is the only correct method.

The math behind expect utility analysis couldn't be any easier. It's 5th grade arithmetic. The challenge is knowing the utility function. Yards, and even points, don't equate to utility. A 7-yard gain is usually good, but it's relatively useless on 3rd and 8. And a 3-point field goal doesn't help late in the 4th quarter when down by 7.

Fortunately, there is win probability (WP). WP is the one and only correct utility function for any game, including football. Winning is all that matters, whether by 1 point or 100 points. WP is also perfectly linear, which is essential to valid expected utility analysis. A 0.40 WP is exactly twice as good as a 0.20 WP, and 0.80 WP is twice as good as 0.40 WP.

Prospect Theory

But even if coaches were to somehow use expected WP analysis when making decisions (say by using 'quick reference' cards like they sometimes do for 2-point conversion decisions), it's likely they still wouldn't be very rational.

Prospect theory says that people fear losses more than they value equivalent gains. Humans evolved with a tendency to try to avoid loss. We're usually more upset with ourselves when we misplace a $20 bill than we are happy when one falls out of the laundry. This tendency has been borne out time and time again in clinical experiments and other studies.

In football, this means that decisions are warped because coaches would fear a loss in WP more than an equivalent gain in WP. The chart below illustrates this concept. According to prospect theory, the "joy" from a 0.05 gain in WP is less than the "pain" from a 0.05 loss in WP.

This asymmetry would affect tactical decisions in many ways, but the most obvious may be 4th down doctrine. Say a team finds itself in a situation where punting would result in a 0.50 WP, but the expected utility analysis says going for the conversion would result in a net 0.55 WP. If the goal is to win the game, the correct decision in this case is to go for it. Period.

The analysis isn't so straightforward for the coach (even if he could do all the math on the spot). Say the failed conversion results in a 0.45 WP and the successful conversion results in a 0.65 WP. A 50% chance at successful 4th down conversion therefore results in a net 0.55 WP.

But the coach sees the 0.45 WP as a possible loss of 0.05 WP, and he sees the 0.65 as a gain of 0.15 WP. Because he fears the loss far more than he values the potential gain, even one 3 times as large, he'll prefer the sure-thing option and punt.

Further, it's possible to actually measure the risk aversion of coaches by comparing the WP advantages in situations where they went for the conversion to the WP advatanges in situations where they forego the conversion attempt.

An Advantage

The coach who can resist this human tendency and make decisions based purely on expected utility will have an advantage. Just how big an advantage, no one can ever know. Actually, that's not true--I'll tell you right now. Just by following a pure expected utility analysis on 4th down, a coach would win an average of an extra 1.4 games per year.

I calculated this based on a play-by-play database from the past 9 seaons. For each 4th down in which a team kicked either a FG attempt or punt, I calculated the difference between going for it and kicking. Wherever the difference was positive, I summed the increase in WP for going for it. The grand total for nearly 2400 games was +203.1 WP, which equates to an increase of 0.17 WP for every game. But since there are always two teams competing in every game, this means that we need to halve that, which is 0.086. The bottom line is that a pure expected utility approach to 4th down decisions would increase a team's chances of winning a game from 0.50 WP to about 0.59 WP. This is equivalent to an extra 1.4 wins per season (0.086*16).

That's a bold claim, I realize. But if you trust my WP model, which is really nothing more than a smoothed empirical observation of how often teams actually won in given game situations in real NFL games, then the claim is not so bold. It's not a perfect model, but the errors are unbiased, meaning it overestimates as much as it underestimates.

Still, if a coach only followed the expected utility recommendations when the WP for going for it was greater than 0.05 more than the WP for kicking, his team would still benefit by an extra 0.8 wins per season. That's nothing to sneeze at in a 16-game season.

Recent Contributions

It can be an interesting exercise to answer questions for reporters and bloggers. Sometimes their questions get me thinking about things in ways I hadn't thought of, or questioning my own assumptions.

Here are a few recent articles from around the web that I've answered questions for: asked me about injury predictions and why I'm so skeptical about them.

The Wall Street Journal did short blurb on how many dollars per wins Eli Manning's new contract might be worth.

Here is a really neat article by Sam Arbesman in the Boston Globe about the concept of clock management in football. As a thought exercise, Sam proposes the addition of a "Time In," which is essentially an anti-time out. It gets you thinking about how important every few seconds are in a close game. I'm kind of a fan of some of Sam's other articles too. My favorite is this one, a suggestion of how to gain immortality in the name of a mathematical constant.

Last month, I did a radio interview with a Chicago station about my Jay Cutler articles.

Koko Fantasy Rankings - Defense

Last but not least are Koko the Monkey's team defense rankings. In case you aren't familiar with the Koko rankings, they are the simplest projections possible based on previous-year performance. These are intended to serve as the baseline for the bare-minimum accuracy we should expect from all other fantasy projections. A full explanation can be found in the write-up for the QB rankings.

Defense point totals can vary widely depending on your league's scoring rules. For these rankings, I used turnovers, sacks, defensive TDs, and points allowed. Year-to-year correlations for turnovers and defensive TDs are extremely weak, but sacks and points allowed can be projected relatively well. I ignored special teams-based scoring, which should not have any effect on the rankings, as special teams stats are notoriously random and unpredictable.

Here are how sacks and points allowed regress in case anyone is curious.

Here are the final rankings. Regarding fantasy scoring for points allowed, every league is different. I didn't do the math required to estimate how many times a team with a given point average would fall into the various scoring "bins." Instead, I looked at my own league and saw they (very) roughly assign half a point for every point allowed below league-average, which is about 20 pts. So that's what I did here.

RankDefenseSks/GInts/GFum/GPts Allwd/GTDs/GPts/GTotal

Koko Fantasy Rankings - Kickers

The rankings everyone's been waiting for--kickers. Koko the fantasy football monkey is making his way through each position. In case you aren't familiar with the Koko rankings, they are the simplest projections possible based on previous-year performance. The rankings are intended to serve as the baseline for the bare-minimum accuracy we should expect from all other fantasy projections. A full explanation can be found in the write-up for the QB rankings.

Kickers are highly unpredictable, both in real terms and in fantasy terms. In fact, there is no year-to-year correlation in the fantasy points gained by field goal kicking. But there is some consistency in extra points, which has nothing to do with kicking skill and everything to do with the rest of the kicker's team.

With that in mind, Koko has ranked his kickers based on XPs alone. The bottom line is that kickers are generally interchangeable. Treat the kicker as a position you'll want to swap out later in the year once you get an idea of who's kicking a lot of FGs. For example, look at which teams have good defenses but can't put the ball in the end zone.

One result of relying only on XPs is that we're only interested in team TDs. Therefore, if a kicker is replaced on a team, Koko's projection remains for new kicker on that team. For example, Steve Houshka is replacing Matt Stover in Baltimore. His projection is what Stover's would have been had he remained on the team. To be honest, I'm not closely watching kicker news, so if there are other examples, or if there are guys on here who will be on the IR or out of a job, let me know.

PlayerXP/GFG Pts/GPts/GProj Pts
John Kasay
Mason Crosby
Nate Kaeding
David Akers
Neil Rackers
Adam Vinatieri
Jason Elam
Nick Folk
Jay Feely
Steven Houshka2.45.27.6113.5
Robbie Gould
Garrett Hartley2.45.27.6113.4
Dan Carpenter
Rob Bironas
Ryan Longwell
Stephen Gostkowski
Matt Prater
Kris Brown
Jeff Reed
Matt Bryant
Joe Nedney
Rian Lindell
Josh Scobee
Olindo Mare
Jason Hanson
Sebastian Janikowski
Shaun Suisham
Josh Brown
Phil Dawson
Shayne Graham

Koko Fantasy Rankings - Tight Ends

Tight ends are relatively easy to project. In addition to their steadily increasing prominence, the notable thing about TEs is how steep the fall-off is among the top several players. In case you aren't familiar with the Koko rankings, they are the simplest projections possible based on previous-year performance. The rankings are intended to serve as the baseline for the bare-minimum accuracy we should expect from all other fantasy projections. A full explanation can be found in the write-up for the QB rankings.

I'll cut right to the chase. Here are how tight end TDs per game and receiving yards per game regress. As you'd expect, yards are more consistent and predictable than TDs.

Fumbles are unpredictable, and hover at 0.2 fumbles per game. Top TEs tend to be relatively durable, and will play 14.5 games per year. Based on standard scoring of 6 points per TD, 1 point for 20 receiving yards, and -2 points for each fumble, here is how the projections stack up:

RankPlayerTD/GYds/GFum/GPts/GTotal Pts
1 Tony Gonzalez 0.4057.60.24.970.7
2 Dallas Clark 0.3050.50.23.957.0
3 Antonio Gates 0.3441.20.23.754.1
4 Jason Witten 0.2452.70.23.652.9
5 Visanthe Shiancoe 0.3236.20.23.348.1
6 Tony Scheffler 0.2345.
7 Kellen Winslow 0.2640.
8 Owen Daniels 0.1848.
9 John Carlson 0.2637.
10 Anthony Fasano 0.3229.
11 Greg Olsen 0.2635.20.22.942.5
12 Chris Cooley 0.1548.00.22.942.3
13 Kevin Boss 0.3027.50.22.840.3
14 Zach Miller 0.1544.60.22.739.8
15 Heath Miller 0.2235.80.22.739.2
16 Dustin Keller 0.2133.30.22.536.4
17 Bo Scaife 0.1834.60.22.434.9
18 Billy Miller 0.1537.20.22.434.6
19 Daniel Graham 0.2426.60.22.333.9
20 David Martin 0.2129.40.22.333.6
21 Donald Lee 0.2622.50.22.333.4
22 Jeremy Shockey 0.1338.40.22.333.0
23 Marcedes Lewis 0.1831.
24 L.J. Smith 0.2325.
25 Jerramy Stevens 0.1929.
26 Todd Heap 0.2127.
27 Martellus Bennett 0.2421.
28 Alex Smith 0.2221.
29 Vernon Davis 0.1825.10.21.928.1
30 Robert Royal 0.1525.90.21.826.4
31 Desmond Clark 0.1525.50.21.826.0
32 Benjamin Watson 0.1919.60.21.724.7

Koko Fantasy Rankings - Running Backs

Next up in Koko the monkey's fantasy rankings are the running backs. In case you aren't familiar with the Koko rankings, they are the simplest projections possible based on previous-year performance. The rankings are intended to serve as the baseline for the bare-minimum accuracy we should expect from all other fantasy projections. A full explanation can be found in the write-up for the QB rankings.

RBs are notoriously difficult to predict. While a starting QB can be expected to have the vast majority of his team's snaps, starting RBs often share snaps with backups. The dreaded RB-by-committee makes it difficult for a simple quantitative system to account for the varying expected number of carries each RB can get.

Still, the regression plots show us we can still project RB per-game production fairly well. I tried the projections two different ways. One way was to estimate production per run--yds per rush and TDs per rush--and then project the number of rushing attempts for each RB. The second method simply projected per game production--yds per game and TDs per game--without regard for the total number of carries. The two methods were equally accurate, so I chose the second, simpler method.

Ascent of the Tight End

The Tight End position appears to be increasing in prominence in the passing game. Since 2001, TEs have been accumulating an increasing number of receiving yards, both in terms of total yards and in terms of the share of the overall receiving yards by all position types.

When I was going over the Koko fantasy projections for TEs, I noticed the trend illustrated in the graph below. TEs have been getting more receiving yards each year since 2001. Not a single year saw a decline, and the increase has been over 40% since the beginning of the nine-year period.

The increase from 2001-2002 can be partially explained by the expansion of the league by the addition of the Texans. However, the increase in that year exceeds the 1/31st increase we'd expect, and the increasing trend continues to be steady.

The increase is not simply due to an increase in overall passing yards as a whole. Receiving yards by wide receivers have held very steady and receiving yards by running backs have slightly declined during that span. The graph below shows the relative change in receiving yards by position compared to the 2001 baseline.

The decline in RB receiving yards is small and does not account for the increase in TE yards. In other words TEs are not robbing passes that used to go to RBs. The increase in TE yards are "new" yards. TEs eclipsed RBs in receiving yards for the first time in 2004, and have continued to hold a an edge ever since.

This isn't too surprising given the abundance of pass catching TEs around the league. Anyone following the game for the past 20 years has witnessed the evolution of the TE position. (And I suspect I may be retreading ground others have plowed.) Forty percent over nine years, however, is a stark increase. Could you imagine an equivalent change in baseball, say a 40% increase in hits by second basemen over nine years? It would be stunning.

The increase is far too large to be explained by a handful of outliers like Gonzalez, Heap, Clark, or Gates. It's due to a change in the league as a whole. I'm not sure when the trend began as my data only goes back to 2001. The PFR guys have data back to the dawn of time, and might be able to shed some light.

Koko WR Fantasy Projections

Here is the next installment of the Koko The Monkey's fantasy football projections. Wide receivers are ranked based on a regression from last year's stats.

These projections are intended to be establish the baseline minimum accuracy of projections as the most reasonably naive predictions. The general explanation of the system can be found in the post ranking quarterbacks.

WR fantasy performance is far simpler than QB fantasy performance. Receiving yards and touchdowns are the only driving factors. Rushing or kick returning yards are ignored in these WR projections. Receiving yards and TDs appear to regress at similar rates from year to year, so these projected rankings will likely be a straight regurgitation of last year's end-of-year rankings.

I included WRs with at least 20 receptions in a season in the analysis. Depending on how high the cutoff, or which stat you use (yards, games, receptions), the regression rate is different. In the end however, the overall rankings aren't significantly affected, and the projected points are only slightly different.

Wide Receivers don't appear to have any consistency in terms of fumbles or injuries. They average 0.05 fumbles per game, with guys who get a lot of receptions obviously having more opportunities. WRs play in an average of 14.3 games in a season regardless of how many they played the year before.

I've removed Burress and Harrison, but I didn't spend much time looking through the list to find guys who may have retired or who may already be on the IR.

RankNameTDsYdsFumPts/GProj Pts
1Anquan Boldin 0.5471.10.076.695.0
2Larry Fitzgerald 0.4772.80.056.491.3
3Calvin Johnson 0.4769.
4Andre Johnson 0.3878.
5Steve Smith CAR0.3579.
6Greg Jennings 0.4067.80.055.781.7
7Roddy White 0.3671.10.055.680.0
8Randy Moss 0.4557.40.065.578.1
9Brandon Marshall 0.3469.80.065.477.5
10Terrell Owens 0.4359.00.055.477.4
11Antonio Bryant 0.3666.20.055.376.5
12Lance Moore 0.4354.
13Marques Colston 0.3661.
14Vincent Jackson 0.3660.
15Reggie Wayne 0.3362.
16Hines Ward 0.3658.
17Dwayne Bowe 0.3657.90.054.970.7
18Kevin Walter 0.3853.40.054.969.5
19Bernard Berrian 0.3655.80.054.869.2
20Santana Moss 0.3358.70.054.869.2
21Justin Gage 0.3852.30.054.868.7
22Eddie Royal 0.3258.70.054.767.8
23Deion Branch 0.3850.70.054.767.5
24Derrick Mason 0.3158.50.054.766.8
25Donald Driver 0.3157.60.054.666.4
26Wes Welker 0.2663.10.054.666.2
27Laveranues Coles 0.3651.60.054.666.1
28Isaac Bruce 0.3651.10.054.665.8
29Muhsin Muhammad 0.3154.30.054.563.9
30T.J. Houshmandzadeh 0.2955.80.054.563.7
31Santonio Holmes 0.3252.60.054.463.3
32Lee Evans 0.2657.80.054.462.3
33Steve Breaston 0.2657.30.054.462.2
34Jerricho Cotchery 0.3151.90.054.362.1
35Matt Jones 0.2557.60.054.361.6
36Greg Camarillo 0.2653.
37Braylon Edwards 0.2652.
38DeSean Jackson 0.2453.
39Chad Ochocinco 0.3144.
40Torry Holt 0.2649.
41Devery Henderson 0.2649.
42Michael Jenkins 0.2649.00.053.956.2
43Anthony Gonzalez 0.2944.80.053.955.3
44Chris Chambers 0.3339.90.053.955.2
45Kevin Curtis 0.2845.90.053.955.2
46Donnie Avery 0.2746.80.053.955.1
47Malcom Floyd 0.3141.50.053.854.8
48Devin Hester 0.2746.50.053.854.7
49Ted Ginn Jr. 0.2449.50.063.854.2
50Mark Clayton 0.2646.00.053.854.1
51Antwaan Randle El 0.2942.30.053.753.5
52Amani Toomer 0.2941.80.053.753.0
53Nate Washington 0.2643.60.053.752.4
54Patrick Crayton 0.2940.70.053.752.2
55Josh Reed 0.2247.40.053.651.6
56Mark Bradley 0.2939.10.053.651.2
57Brandon Stokley 0.2741.20.053.651.1
58Dennis Northcutt 0.2543.30.053.550.6
59Bobby Wade 0.2444.10.053.550.5
60Bryant Johnson 0.2640.50.053.550.2
61Jerheme Urban 0.2937.00.053.549.7
62Brandon Lloyd 0.2639.90.053.549.6
63Domenik Hixon 0.2442.40.053.549.5
64Koren Robinson 0.2540.10.053.449.2
65Josh Morgan 0.2936.10.053.449.1
66Ike Hilliard 0.2936.10.053.448.9
67Hank Baskett 0.2737.70.053.448.5
68Roy Williams 0.2737.40.053.448.3
69Johnnie Lee Higgins 0.2934.00.053.347.5
70Steve Smith NYG0.2241.30.053.347.0
71Davone Bess 0.2240.
72Chansi Stuckey 0.2734.
73Jabar Gaffney 0.2437.
74Rashied Davis 0.2436.
75Reggie Williams 0.2633.
76Bobby Engram 0.1942.
77Michael Clayton 0.2239.
78Jason Avant 0.2435.
79Arnaz Battle 0.1941.
80James Jones 0.2336.
81Shaun McDonald 0.2236.
82Brandon Jones 0.2237.
83Jordy Nelson 0.2434.
84Dane Looker 0.2532.
85Jason Hill 0.2432.20.052.942.1
86Justin McCareins 0.1937.80.062.941.9
87Harry Douglas 0.2232.30.052.840.3
88Roscoe Parrish 0.2231.00.052.839.8
89Antonio Chatman 0.1931.90.052.637.8
90Brian Finneran 0.2226.80.052.536.2