Leaving Free WP All Over the Field

If you were a coach, would you voluntarily give up a down at some point in the game, just to be sporting? Ehh, let's just make it 3rd and 5 instead of 2nd and 5. Of course not. For a random play in the 2nd quarter, that would cost you about 0.02 WP (2% chance of winning) for no reason.

So why do NFL coaches voluntarily leave WP out on the field?

Take yesterday's DEN-SEA game as an example. SEA was ahead 17-12 in the 4th quarter, and had the ball deep in their own territory with about 9 minutes to play. With the game clock running, they snapped the ball with: 8, 5, 5, 8, and 10 seconds left on the play clock. That's a total of 36 seconds. Plus, there was a play in which the receiver could have just as easily remained in bounds. Because there was more than 5 minutes left in the game and the clock restarts after the ball is set, that may have only cost 10-15 seconds of play clock rather than up to 40 seconds. To be fair let's say there was a total of 46 seconds SEA could have burned off the clock during their second to last drive with almost no effort or risk.

Texans Try Once, Fail Twice

Down 14-0 at the start of the second half to the New York Giants,
the Houston Texans faced a 4th-and-1 on their own 46-yard line. At this point, with just a 9.0% chance to win, Bill O'Brien made the correct call to go for it. A successful conversion means a 12.9% win probability, while a punt means about an 8.6% chance to win. The break-even point going for it is far below an estimated 65% conversion rate on 4th-and-1. Alfred Blue ran off right tackle and was stuffed, turning the ball over on downs. The Giants would kick a field goal to go up 17-0.

On the very next drive, Ryan Fitzpatrick led the Texans downfield to the Giants 9-yard line where they faced another 4th-and-1. With 6:13 left in the third, down 17, Bill O'Brien elected to take the chip-shot field goal. Even the commentators suggested he should be going for it. Obviously, the prior failure on fourth down should not have an affect on the Texans' decision this time. If that were the case, O'Brien would be judging his previous decision on the outcome, rather than the process. The only other logic could be that he figured they would need a field goal at some point, down 17 - common faulty logic in the NFL as coaches should be doing whatever they can to maximize their chances of winning.

Two-Point Conversion in the KC-DEN game

With 7:15 left in the 4th quarter against DEN, KC's Knile Davis ran for a 4-yard TD, narrowing the Broncos' lead 21-16 pending the extra point or two-point conversion. Andy Reid elected for the extra point, and following the kick the Chiefs trailed by 4 points rather than 3 or 5 points resulting from a two-point try.

NFL coaches typically adhere to what's known as the Vermeil Chart for making two-point decisions. The chart was created by Dick Vermeil when he was offensive coordinator for UCLA over 40 years ago. It's a very simple chart that simply looks at score difference prior to any conversion attempt and does not consider time remaining, with one caveat. It applies only when the coach expects there to be three or fewer (meaningful) possessions left in the game.

With just over 7 minutes to play, there could be three possessions at most left, especially considering that at least one of those possessions would need to be a KC scoring drive for any of this to matter. (In actuality, there were only two possessions left, one for each team.) Even the tried-and-true Vermeil chart says go for two when trailing by 5. But it's not the 1970s any more and this isn't college ball, so let's apply the numbers and create a better way of analyzing go-for-two decisions.

Except for rare exceptions I've resisted analyzing two-point conversion decisions with the Win Probability model because, as will become apparent, the analysis is particularly susceptible to noise. Now that we've got the new model, noise is extremely low, and I'm confident the model is more than up to the task.

First, let's walk through the possibilities for KC intuitively. If KC fails to score again or DEN gets a TD, none of this matters. Otherwise:

Chiefs Crawling Drive, Come Away With Nothing

The Chiefs lost to the Broncos 24-17 on Sunday and had a chance to at least tie the game at the very end. Kansas City kept Peyton Manning off the field for an enormous chunk of the second half. The Broncos offense had only two drives after halftime (not including the final kneel down), one for a punt, one for a field goal, totaling just 8:51 in possession. The longest drive came from the Chiefs at the very start of the second half, where they ran 23 plays, taking 10 minutes off the clock... and ultimately missed a field goal. This got me thinking, how does drive length (in minutes) affect the probability of a team scoring?

First, here's a look at the ridiculous drive using our Markov model:

Analyzing Replay Challenges

The new WP model allows some nifty new applications. One of the more notable improvements is the consideration of timeouts. That, together with enhanced accuracy and precision allow us to analyze replay challenge decisions. Here at AFA, we've tinkered with replay analysis before, and we've estimated the implicit value of a timeout based on how and when coaches challenge plays. But without a way to directly measure the value of a timeout the analysis was only an exercise.

Most challenges are now replay assistant challenges--the automatic reviews for all scores and turnovers, plus particular plays inside two minutes of each half. Still, there are plenty of opportunities for coaches to challenge a call each week.

The cost of a challenge is two-fold. First, the coach (probably) loses one of his two challenges for the game. (He can recover one if he wins both challenges in a game.) Second, an unsuccessful challenge results in a charged timeout. The value of the first cost would be very hard to estimate, but thankfully the event that a coach runs out of challenges AND needs to use a third is exceptionally rare. I can't find even a single example since the automatic replay rules went into effect.

So I'm going to set that consideration aside for now. In the future, I may try to put a value on it, particularly if a coach had already used one challenge. But even then it would be very small and would diminish to zero as the game progresses toward its final 2 minutes. In any case, all the coaches challenges from this week were first challenges, and none represented the final team timeout, so we're in safe waters for now.

Every replay situation is unique. We can't quantify the probability that a particular play will be overturned statistically, but we can determine the breakeven probability of success for a challenge to be worthwhile for any situation. If a coach believes the chance of overturning the call is above the breakeven level, he should challenge. Below the breakeven level, he should hold onto his red flag.

Weekly Game Probabilities: A New Home

This season the weekly game probabilities will be featured at Sports on Earth. Each game will have the probability, a score prediction, and a couple notes on why the numbers are what they are. In the early weeks of the season, the numbers are at least partially based on the same preseason estimates of team strength I used for the season projections. But as we get a few weeks of data, those preseason ratings will fade out.

For now the score predictions are simply maximum-plausibility estimates. (Yes, I just made that term up.) Predicting an actual score for each game is statistically boring. With few exceptions, a statistically sound estimate would be 24-20 or 27-21 for every game, so I've added some of the human element to the score predictions. The bottom line is that readers should focus on the probabilities and don't bet the mortgage on the scores.

The game probabilities will be matched up against the picks of Will Lietch, one of the cornerstone writers at SOE. The idea is to create a friendly competition between man and machine.

The game probabilities had a great run at the New York Times--5 years. But there are only so many thought-provoking or counter-intuitive lessons on probabilities and predictions that can be squeezed out of a week of NFL games. But AFA will continue working with the Times on various projects as the season unfolds.

Here's the link to the probabilities for week one. For those keeping score at home, I had the Seahawks at 66% to win last night.

The 4th Down Bot Returns

The 4th Down Bot is returning to the New York Times this season. You might recall we booted him up late last season, but this year he'll be around starting week one. At its heart, the bot is a fun application of the 4th Down Calculator feature here at AFA. It uses both the Expected Points model and the Win Probability model to estimate the best option for every 4th down as a game is in progress.

As I mentioned last year, although the 4th down issue is growing mold with smarter fans, it remains the lowest hanging fruit on the football analytics tree. So it's nice to be able to automate things and not have to do the analysis myself. But on the other hand, we can add 'football analyst' to the list of jobs being taken over by robots.

The Bot will be faster, more accurate, and come with some new features this season. Here is a brief introduction. Here's are a few notes on how it works. And here is his Twitter feed.

Super Bowl XVLXVLVLIIICDMVXXXIII Analysis

First of all, I'm getting tired of the Roman numerals. It was cool up until maybe Super Bowl XXV, but now it just hurts my brain.

Secondly, although my numbers pointed to a SEA edge I did not see that coming. The game notched a 1.5 on the Excitement Index, the lowest of any SB in the data (since '99). The next lowest were the TB-OAK 2002 game and the BAL-NYG 2000 game, each at 2.7. There weren't many decisions to analyze because the game got out of hand so quickly, but I'll go over the little we can learn from last night.

Overall, the game hinged on the fundamentals. SEA's defense was faster, bigger, stronger. Even a layman like myself could tell SEA won because of lots and lots of individual matchup victories. They made tackles at first contact. Guys shook their blocks lightning fast. They swarmed to the screens, caved the pocket, and covered the receivers in stride. There weren't many blitzes or scheming contrivances. Instead it was plain old physical football. The only wrinkle I noticed was that SEA played more cover/man 2 than we expected, but that's not exactly something Manning shouldn't normally be able to handle.

The Challenges

Advanced Stat Breakdown for Super Bowl 48

Instead of reading a bunch of words about the Super Bowl matchup, where each phase is trying its hardest to express some sort of numerical evaluation, wouldn't you prefer to see the numbers themselves collected into one giant eye chart? Well, if that appeals to you, you'll enjoy the SEA-DEN Matchup page.

NFCCG SF-SEA Observations

As expected, this was a real defensive slugfest. The winning QB had -3.4 EPA. Kaepernick posted -0.28 WPA and 2.2 AYPA. Both offensive lines were beaten soundly. SF's notched -5.4 EPA and SEA's had -2.6 EPA.

Unlike the AFC game, this one was all about 4th downs. HUGE leverage throughout the game. I know I can be a broken record on this stuff, but this game really hinged on some very interesting strategic decisions.

-SF 4th and 2 on the SEA 7, 1st qtr. They punted. Probably should have gone for it.

-SF 4th and goal on the SEA 1. They went for it. Great call.

-SF 4th and 6 on the SEA 46, 1st qtr. They punted. Probably should have gone for it.

-SEA 4th and 6 on SF 38, 26 sec in 2nd qtr. They went for it, converted then kicked a FG to end the half.

-SEA 4th and 7 on the SF 35, 4th qtr 14 min to play, down by 4. They went for it. Great call except SEA burned a timeout that they were reasonably likely to need in order to think things over. Here's the thing: Timeouts are very valuable. If you can't decide between going for it or kicking or punting, you're probably very close to the point of indifference anyway. You may be better off making any quick decision and saving the timeout than you are making an optimum decision but wasting a timeout.

-SEA 4th and goal from the 1, 4th qtr 8:39 to play, up by 1. The went for it. Great call. Why? First, because they'll probably make it and virtually put the game away. And if they don't they're likely to leave the ball on the SF 1-yd line. That's not exactly a good place to be for an offense. I heard someone say that despite the math you can't take a chance like that against the SF defense. But as I noted last week, over the past 2 seasons SF has faced 15 (now 17) plays from the 1-yd line and allowed TDs on 10 of them. That's worse than league average. Don't get me wrong. I'm not saying that the SF defense is below average. Instead, the point is that good and bad teams aren't that different on any one given play. It's just that good or bad teams show up that way after accumulating very small advantages over several dozen individual plays in a game.

-Here's a weird one. SEA 4th and 11 on the SF 29, 4th qtr 3:43 to play, up by 3. My numbers say...punt? Yes, punt. Here's why:

AFCCG NE-DEN Observations

I thought the big story of the game wasn't how easily DEN moved the ball. We all expected that. The big story was DEN's defense, which held NE to just 3 points in the 1st half and 10 points through 55 minutes. Brady was held to -0.02 WPA. He did notch +8.3 EPA, but a lot of that was after the game was mostly decided.

NE was going to need some fluky things to go their way to win--turnovers, a special teams play, or some terrible call by the refs. It never came.

Manning and the DEN passing game did have a fantastic day. Manning: +.48 WPA, +17.9 EPA, 9.3 AYPA, no turnovers, no sacks.

I, and the NYT 4th Down Bot--(funny how you never see the two of us together at the same time), agreed with every 4th down call during the game. Belichick knows what he's doing. I was disappointed to see DEN burn a timeout just prior to NE's 4th down conversion attempt. Teams should be better prepared for a 4th down attempt, particularly in situations like this: a 4th and short in or near the red zone. In a high-leverage situation like that, it's ok for a team with a significant lead to use a timeout, but in a closer game, it would be much more costly. (I'm working on a project to value timeouts in terms of WP now, and without any spoilers--they are very precious in the 2nd half.)

Seahawks Should Not Have Scored

I'm not going to touch Sean Payton's late field goal attempt, down eight, into the wind, with only one timeout remaining. After the miss, on the ensuing Seahawks drive, Marshawn Lynch tore down the left sideline and it appeared he was let into the end zone by the Saints. The question is, did the Saints make the right decision letting him score once the first down was achieved - and should Lynch have gone down at the 1-yard line? The answer to both of these will always be the same, and in this case, it's yes.

Lynch scored with 2:48 remaining in the game. Up by 15, kicking off to the Saints, the Seahawks had about a 99% chance of winning. The Saints would need to score, recover an onside kick, and score again just to tie.

Update: Lynch actually scored with 2:40 remaining, not 2:48 as originally listed in the play-by-play. That means that the Seahawks could have milked the entire clock as listed at the very bottom of this article.

But, what if Lynch had gone down at the one? Using the time calculator, we can see how much time the Saints would have had following a chip shot (99.7%) field goal from the 1-yard line. The Saints had no timeouts left, so we are looking at the blue line.

Would Auburn Have Been Better Off Not Scoring a Touchdown?

Slate asked me to take a look at the possibility that Auburn would have a higher win probability had they not scored on Tre Mason's go-ahead touchdown run, and instead taken a knee at the 1-yard line with 1:19 to play. It was a difficult analysis, and required some unsatisfying assumptions, but in the end the results confidently pointed toward one conclusion.

...Nevertheless, Auburn had about an 81 percent chance of winning after Mason’s score, as teams in Florida State’s position are able to score a touchdown about 19 percent of the time. (These numbers are based on analogous situations in the NFL, though I’ve made slight adjustments for the differences in pace between college and the pros, and to include the chance of a kick return for a touchdown.)

So, the question we’re evaluating is whether having a first down at the 1-yard line would have given Auburn more than an 81 percent chance of winning. It’s a tricky question because it needs to be analyzed backwards...

(What I didn't explain in the article is that it's easiest to work backwards because the Auburn WP on 3rd down is depending on the results of a potential 4th down. And the WP on 2nd down are dependent on the potential results of 3rd down, which in turn depend on 4th down. And so on.)

For the Slate articles, I can't get away as much math and equations as I like, so here's a table of the relevant probabilities I used. It was complicated because the deeper into the goal-line series Auburn went, the lower Auburn's chances of getting the TD went BUT the lower Florida State's chances of responding went too. This edition assumes Auburn goes for it on the 4th down on what would be a single make-or-break play for championship.

What if KC Had Just Kneeled Out the 2nd Half?

With 13:39 left in the 3rd quarter, KC led IND 38-10. Obviously, 3rd grade math tells us IND needed at least 4 touchdowns just to tie the game at this point. A little more arithmetic might illustrate just how badly KC bungled this game.

There were a total of 1,519 seconds left in the game. KC can burn 40 seconds between plays and 6 seconds during a typical play just by calling a super safe run (that stays in bounds) or even a kneel. Even if KC doesn't try to convert a single first down, they can burn 144 seconds on a series. However, IND can use its 3 timeouts to make one series only take 24 seconds off the game clock.

Because IND is due to receive the kickoff at 13:39 in the 3rd, KC was guaranteed to have at least three possessions--one between each theoretical IND TD. That means that just by kneeling, KC can burn a total of 456 seconds (7:36) off the game clock, which leaves a total of 1,207 seconds of game time (20:07) and no timeouts for IND to score 4 touchdowns.

Chargers Courageous Call & Playoff-Clinching Drive

Despite the controversy surrounding an illegal defense on the Chiefs' missed field goal at the end of regulation, the San Diego Chargers defied odds and clinched a postseason berth on Sunday. In overtime, Philip Rivers orchestrated a 17-play, nine-and-a-half minute field goal drive to start the extra quarter that ultimately sealed their win. The length of the drive, in this case, is just as important as the outcome as San Diego could advance with either a win or a tie.

Using our Markov model, let's take a look at the drive. Keep in mind, the model is best used for a standard drive when time and score differential would not greatly affect decision-making or play-calling. Since this was the opening drive of overtime, those standards will predominantly hold true, although not perfectly given the leverage of the situation.

PIT Should Not Have Scored the TD

With the score tied and 1:51 to play, PIT had a 1st and 10 on the GB 17 yard line. In many circumstances a team can run down the clock and kick a short FG to win the game. PIT was near the 'Field Goal Choke Hold' zone, when it's better for the offense not to score a TD and better for the defense to allow a TD. But fortunately for GB, they had all 3 of their timeouts, and could be assured of getting the ball back with 1:27 to play if they made a stop and forced the FG. So with 3 timeouts remaining, the numbers say it never makes sense for a defense to intentionally allow the TD.

But GB jumped offside on the FG attempt, and gave PIT a 1st down and goal from the 5 with 1:35 to play. Now GB had only 1 timeout left, and it would have certainly made sense for PIT to refrain from scoring the TD, burn time off the clock, and kick an easy FG for the win.

The chart below illustrates when a defense would prefer to allow a TD. The black diamond represents the state of the game at the 1st and goal mark assuming PIT does not score a TD. The black line shows the win probability of the defense if they allow the TD.

Did Shanahan Make the Right Decision to Go for Two?

Following a TD to pull within an extra point of tying the game with just seconds left to play, WAS head coach Mike Shanahan elected to attempt a two-point conversion to win rather than enter overtime. It’s not fashionable to defend Shanahan around DC these days, but I think this was the right decision for a couple reasons.

Philbin Shows Up Phil

In a huge division game with significant playoff implications, the Dolphins trailed the Patriots 20-17 with a little over three minutes remaining and three timeouts in their back pocket. Facing a 4th-and-5 from their own 45-yard line, Jim Nantz asked Phil Simms whether he would punt in that situation, to which Simms replied "Absolutely, punt it!" The logic here is that with three timeouts remaining, the Dolphins could stop the Patriots and force a three-and-out, getting the ball back with another chance to drive down and tie or win the game.

Let's look at the baseline win probability numbers:

Payton Was Right to Decline

At least according to Expected Points, he was.

Here's the situation: At the beginning of the 3rd quarter against CAR, NO had a 1st and 10 at their own 16-yard line. They threw for a 7-yard gain, setting up a 2nd and 3 from their 23. But CAR was flagged for defensive holding, which would have given NO 5 yards and an automatic first down at their 21. NO head coach Sean Payton declined the penalty to the bafflement of many including the tv announcers.

The game did not hinge on this decision by any stretch. But it's worth taking a look at. The EP model is probably the right tool for the job in this case because it gives a much finer level of precision to down/distance/yd-line situations than the WP model or other approaches.

Using the hand-dandy WP calculator tool (which as a bonus is also an EP calculator), here are the relevant numbers:

Seahawks Stumble, Should Have Allowed TD

In one of the most anticipated games of the week, the San Francisco 49ers took over down 17-16 to the Seattle Seahawks with 6:20 remaining. After a huge Frank Gore 51-yard run, the Niners lined up for a 1st-and-Goal from the 7-yard line with 2:39 remaining. Seattle had no timeouts remaining. Should the Seahawks have tried to intentionally allow the Niners to score a touchdown? Let's look at Brian's graph for this situation in his intentional TD study: