The 4th Down Study - Part 1

If there's one topic where quantitative analysis can change the way football is played, it's 4th down decision-making. Many articles here have chronicled the conservative nature play-calling on 4th down in the modern NFL. In this post I'll explain, as clearly and simply as possible, why the evidence points to a more aggressive attack on 4th down.

Previous studies on 4th down decision-making include Carroll, Palmer, and Thorn's book Hidden Game of Football (1988, 1998) and Professor David Romer's Do Firms Maximize? (2005). The first serious study of the concepts used in these studies was by former NFL quarterback Virgil Carter, who co-authored an operations research paper examining the value of field position using data from the first 56 games of the 1969 season.

My own analysis published in this post largely repeats the methods used in previous studies. But I think I can add a good deal to the topic. First, this analysis is based on a much larger data set compared to previous research. Second, this analysis offers possible confirmation of previous results. Third, I think I can explain a complex, abstract subject such as this in a straightforward manner, which is essential if the 4th down revolution is going to make any headway. Frankly, it doesn't matter how strong the analysis is if it can't be communicated clearly and convincingly.

This is how the study goes: At each yard line, I'll calculate and compare the expected point value, based on recent historical averages, of each of the three 4th down options--punt, field goal, or go for it. The option with the highest value is the recommended choice.

I've divided the article into four parts. In this the first part, I review the concept of Expected Points for those who aren't familiar. I also review the concept of expected utility. Both ideas are critical for understanding the thrust of the study. The second part of the article will detail the kicking game. The third part will explore the value of 4th down conversion attempts. The final part of the article will put all the concepts together to ultimately produce a chart of recommended decisions for 4th downs at every combination of field position and distance to go.

'Expected Points'

Speaking of abstract subjects, this entire analysis rests upon the foundation of the Expected Points (EP) concept. Most readers here may already be familiar with EP, but I'm going to summarize it now before I move on to the meat of the study.

EP is the average potential points a team can expect given a certain situation. The most common example is the potential point value of a 1st down at each yard line on the field. EP is the average of all 'next' score values at any given yard line. It's not necessary the average points scored on the current possession because possession could be exchanged several times before the 'next' score. EP is positive when the offense will usually score next, and negative if the defense will usually score next.

Here is the EP graph for a 1st down at each field position. These EP values are based on data from 2,400 NFL games from the 2000-2008 seasons. I used only data from the 1st and 3rd quarters to exclude situations hurried by an expiring clock and by desperate teams or teams with large leads playing differently late in games.

I'll be referring to the graph several times, so be sure you understand how to read it. A 1st down on an opponent's 20 is worth 3.7 EP. But a 1st down on an offense's own 5 yd line (95 yards to the end zone) is worth -0.5 EP. The team on defense is actually more likely to eventually score next.

Note that a 1st down at an offense's own 27 yd line is worth 0.7 EP. This is critical to explain an important twist in the EP concept. Every score requires a subsequent kickoff, and this has value to the receiving team. So to understand the real value of a the score, we need to subtract the value of the kickoff. For example, field goals aren't really worth 3 points. In the long run, they're worth 3 - 0.7 = 2.3 EP. And touchdowns are really worth 6.3 EP.

This concept is especially apparent when considering safeties. After a safety, the scoring team gets the ball back, on average at its own 40, which is equivalent to 1.3 EP. A safety is therefore really worth 2 + 1.3 = 3.3 EP. These score values and the resulting EP values are used throughout the rest of this analysis.

Expected Utility

Before I go any further, I'm going to take a step back and explain the concept of 'expected utility' in general. Say I can choose one of two routes home from work each day. One route is through the side roads, and the other is via the highway. If I take the side roads my commute always takes 20 minutes.

But the highway is more dicey. Sometimes it's clogged with traffic and can take 25 minutes to get home, but most other times traffic is clear and it only takes 15 minutes to get home. The highway is backed up 40% of the time and clear 60% of the time. Which route should I normally choose?

My 'expected commute time' can be calculated using simple proportions. The side roads take 20 minutes 100% of the time, so that's easy--the expected time for that route is 20 minutes. But assuming each minute of my day is equally valuable to me, the highway's expected time would be:

(25 min * 0.40) + (15 min * 0.60) = 19 min

So over the long run, the freeway is the better option. Unfortunately, life isn't that simple. We don't have such clear-cut options and we don't know the probabilities and payoff values of our decisions. But in football, we do.

Continue reading part 2 of this article.

  • Spread The Love
  • Digg This Post
  • Tweet This Post
  • Stumble This Post
  • Submit This Post To Delicious
  • Submit This Post To Reddit
  • Submit This Post To Mixx

21 Responses to “The 4th Down Study - Part 1”

  1. Jason says:

    Unfortunately, even if a coach knows the odds, going for it on 4th down is high-risk/low-reward from a social standpoint -- i.e., what will fans/owners/players/media think if we make it (or punt) vs. what will they think if we miss it? If you make it or punt, nobody says anything. If you miss it, why did you go for it there, you idiot? While that shouldn't be the sole deciding factor for a coach, it can have an effect on his job security and will therefore almost certainly figure into his decision-making process.

    I tend to think of editing the same way. Nobody notices and rewards good editing, but if I let a few sutble errors slip by in whats supposed to be a profesional text, I'm labeled as a poor editor.

    (You probably looked at my first paragraph and didn't think there was anything special about it, but you looked at my second and probably figured I was an idiot. Obviously, there's no "risk" in editing properly, but it's the same general issue of perception. Nobody notices if you do the right thing but they all notice if you fail.)

  2. James says:

    I'm excited to see where this is going!

  3. Kulko says:

    Jason, of course you are probably nailing the reason why coaches behave as they do. But I think you are missing a few things here.

    IF a more aggressive 4th Down Approach has a higher expected utility, it will lead to more wins, which help your job security a lot more then a nice perception.

    Also if you read media articles of the last fw years, you will notice that the climate has definitely changed. Coaches get more ofen critized for being to timid, and aggressive failures have ample defenders in the press by now. SO I think even from a social risk/reward standpoint the times are changingand th caches should too.

  4. Happy says:

    Ultimately you can't argue with success. The coach may take some grief in the short run for an unconventional decision, but if their decisions have a firm basis in expectation value and increase the team's likliehood of winning the team will win more games.

    That's when the coach gets to say "that's why I'm the coach and you're the fans." Not every coach has this level of confidence, but some do.

    I'm curious if there are any teams with greater than 50% on 2-point conversions. Those teams should never kick an extra point in the first three quarters of a game.

  5. Anonymous says:

    Happy - there's tons of info on the net about the TPC situation

    Part of the reason it can be successful is that teams can try in unexpected scenarios

    But there is another school of thought that suggests TPC should be reserved (+ or -) for the last six minutes. A team trailing by 14 on scoring a TD should go for two because if successful the game can be won on the tying TD with the PAT. If it fails, the game can still be tied with a TPC. This route gives the best chance of winning without the dreaded OT

  6. Eugene Carpet says:

    Great trin of thought. I have never seen such a complete analysis of points. What really struck me is the expected points from a safety. Really cool.

  7. Anonymous says:

    Yes. Yes. Yes.

    -Belichick fan

  8. Anonymous says:

    How does this breakdown in the case of last nights game? The model should include:

    1. Home vs away team
    2. Lead or trailing at the time and by how many points
    3. Current field position
    4. Time left in the game
    What does the model look like and tell us then? You should also weight the QB rating for the offense, and in the case you don't make the 1st down, the other teams QB coming back at you to score. That likelihood would go up the deeper you are in your own territory of course.

  9. Anonymous says:

    Here is the problem with this argument. The odds of making the first down on fourth and one or fourth or two are statistically good. That is great and seems to tell us that "going for it" is a good decision. However it is not a good decision when you are in your own territory at all. Even if you team has better fourth down efficiency than the average. Here is why. Let's say your team makes 75% of their 4th and 2s. A very high percentage. The other side of that is that there is a 25% chance of not making it or a TURNOVER. What are the odds on a turnover on any other play? A run with a decent running back? I believe that the average is somewhere around 1 fumble for every 40 carries. How about intercepts per pass attempt? For a good QB it is around 1 in 30. For a bad QB it is lower. Now I am a coach in the NFL and I have the ball on the 30 yard line and I am going to run a play that has a 25 to 45% chance of turning the ball over? Which is the bad decision? Punting the ball and giving the ball to the other team in their territory when the best team in the NFL has around a 30% chance of scoring a Touchdown in that situation or risking an extremely high turnover play and giving the same team a 40 to 50% chance of scoring a TD and a much higher chance of kicking a field goal. I think we can see why 90% of the time or higher NFL coaches make the desicion they make.

    All that being said. I am not questioning Bill Belichek's call. He has a better feel for what his team is doing on the field than I have. He would win the game with a first down. How his line is blocking. How beat up the other D line is. How his backs are running. However in most cases coaches should and do opt to punt in this situation. The odds of a TURNOVER are too high.

  10. Unknown says: is the problem w/ your argument. A punt is a guaranteed turnover. Its throwing the ball 40 yards downfield and a 100% chance of interception. If you go for it on 4th down you are limiting your chances to a turnover to 25% (according to your math) so what's your preference on any given play, a 100% chance of a turnover or a 25% chance of a turnover.

  11. Bill says:

    An issue of some interest might be play selection in go-for-it mode. It appears that most coaches want to dominate the other team, thus multiple straight-ahead runs is a common choice (and commonly leads to failure). The Broncos used to have good results from variations off an Elway bootleg. At the very least, it adds some deception for the next crack at the center of the line!

  12. Anonymous says:

    This is an interesting discussion no doubt, but it seems a little too analytical and not so much practical. The previous anonymous poster hit it right that the Pats coach has a feel for whats going on on the field better than anyone else. Analyzing data can be a useful tool when calculating percent success over all teams over decades of football. It does not, however necessarily relate to the given game being played as all teams are of different skill level from talent to play calling and certain teams are just going to flat out be better at converting than others. Some may be successful some of the time while others may never squeeze a win or points after a decision to go for it. So many more variables would affect a given decision to go or punt that it would be impossible for one to accurately calculate the chances of winning a game based on this generic quantitative analysis.

  13. Ben Rushing says:

    What most people don't know, but should is that football is like a coin that has two sides. Everyone knows the universal truth of the first side of the coin that says that a team that scores more points than the opponent always wins. What people don't know is how a team lost a game without factoring the score. The other side of the coin would say that the team with the most real turnovers always losses the game. Statisticians do not count all events that turn the ball over without first scoring atleast 6 points. If they did they would see that the team with the higher number always losses. Try this with any game and you will see that it works:

    Complete Turnovers: Interceptions, fumbles lost, missed fg's, punts, 4th down fails, kickoffs lost, safeties, and turnovers due to time (given if the team in possesion at the end of the half failed to score and has to kick to start the second half. Not counted if another turnover half or whole occurs at the same time. Also when the game ends and the team trailing fails to score before the time expiresn not counted against the team with the lead. also nullified if another turnover takes place.

    Half Turnovers: Failed extra point or 2pat, Successful field goal, 2point conversion allowed.

  14. Anonymous says:

    Nomination for worst 4th down decision of the year so far goes to Sean Payton for punting on 4th and 2 with 5 mins left in the game down by 10 lol

  15. FourthandFortyTwo says:

    So I've done my own number-crunching, based more on the probability that the receiving team takes it back to the original punting yard of scrimmage: . Basically, what I found out is it all starts to break down right around mid-field, and really, for anything less than 2 yards to go.

  16. Gary says:

    Brian, excellent stuff. I've always intuitively thought it makes sense to go for it in a lot of 4th down situations, but never seen any analysis before yours. In fact I thought teams would be more efficient overall if punting wasn't an option. Looking at your final chart, simplified recommendations would be always go for it within 2 yards, and within 5 in your opponents half of the field. I guess it's a little like basic strategy for blackjack, but easier. I think part of the problem has been an indoctrination into thinking of football as a 3 down game rather than 4, consciously or otherwise. Do you think the new kick-off position will alter your 0.7 point valuation much?

  17. Michael Carlson says:

    I was familiar with the statistical arguments, and in broad agreement, but having just read the original post I am surprised no one has made more of the 'expected utility' concept in understanding why coaches do not go for it more often.

    The key to the 'driving to work' analysis is not the time, it is 'work'. Obviously the difference between 19 and 20 minutes isnt enough to be worth the risk. But even assuming the average were weighted more toward the highway, the point is that if your boss is going to pay far more attention to the time you are late by 5 or more minutes, and far less to all the times you are on time or indeed early. In this sense, a football coach has to worry not only about his 'boss' but all those who mediate his boss' reactions (other owners, execs, coaches, and media) and also the indirect 'boss' of fan reaction, again mediated, but of significance especially when it leads to empty seats.

    Were there a way to quantify a scale of effect of negative impact versus job security it might help commuters, if not coaches.

  18. Mike says:

    There are also some hidden benefits sometimes from some "bad decisions". For example, situations when game theory might tell you to go for it on. 4th down may have unintended consequences. Punting on 3rd down may have hidden benefits. Throughout the course of the game, you may learn more about an opponents tendencies by punting even if going for a long fieldgoal has more EP. There may be opportunity costs later on. If you for example were to let opponent to gain .7ep knowing you would be the last Team to have the ball with good field position. You may willingly take one step back to take two forward. That is second level thinking and stats still has basis in it, but requires you to make estimates of other situations and calculations

  19. Anonymous says:

    very interesting but a major factor that doesnt seem to be considered is the element of surprise when going for it via fake punt or fake fg? this would seem to drive up the success rate quite a bit particularly in areas of the field where they are more uncommon to occur (thus larger element of surprise). the same factor lies within the onside kick too.
    Oregon U manages their playcalling based on all these odds and factors.

  20. Jared Doom says:

    Quick question: Have you considered a possible selection bias in looking at the success rate of the teams that opt to go for it? It seems possible that the teams opting to go for it on 4th down are better-than-average offensive teams.

  21. Unknown says:

    Something I think is completely wrong about your study:

    Excluding data from the 2nd and 4th Quarter of games. Those minutes count towards the outcomes of the game, therefore they count in any comprehensive statistical study.

    As you mentioned, teams tend to play differently with time expiring, whether they are in hurry up mode and passing only (down by a single score), or if they have a huge lead (and are playing a prevent defense). But this doesn't mean the data is invalid and should be excluded. It just means you need a better way to account for it. If I were a coach, and hired you to sit in the booth upstairs and give me stats over the headset on whether to go for it or not, I would make sure the graph you're looking at is for the quarter that the game is currently in (if we're in the 4th quarter, I don't want your graph to reflect 1st and 3rd quarter data, I want it to reflect 4th quarter data, since that's the nearest neighbor of data points to the current game state), or better yet, that the graph is not static, represented on an App that updates with the amount of time remaining in the game.

    Similarly, Expected Utility is going to change depending on the amount of time left on the clock. A made field goal with 2 seconds left in the game is closer to 3 Expected Points than it is 2.3 Expected Points.

    A good starting point, but leaves something to be desired in terms of being comprehensive.

Leave a Reply

Note: Only a member of this blog may post a comment.