Simulating the Saints-Falcons Endgame

I was asked yesterday about the end of regulation of the Saints-Falcons game. With about a minute and a half remaining, NO was down by 4 but had a 1st & goal at the 1. With 2 timeouts left, should ATL have allowed the touchdown intentionally?

I previously examined intentional touchdown scenarios, but only considered situations when the offense was within 3 points. In this case NO needed a TD, which--needless to say--makes a big difference. Yet, because NO was on the 1, perhaps the go-ahead score was so likely that ATL would be better off down 3 with the ball than up 4 backed-up against their goal line.

This is a really, really hard analysis. There's a lot of what-ifs: What if NO scores on 1st down anyway? What if they don't score on 1st but on 2nd down? On 3rd down? On 4th down? Or what if they throw the ball? What if they stop the clock somehow, or commit a penalty? How likely is a turnover on each successive down? You can see that the situation quickly becomes an almost intractable problem without excessive assumptions.

That's where the WOPR comes in. The WOPR is the new game simulation model created this past off-season, designed and calibrated specifically for in-game analytics. It simulates a game from any starting point, play by play, yard by yard, and second by second. Play outcomes are randomly drawn from empirical distributions of actual plays that occurred in similar circumstances.

If you're not familiar with how simulation models work, you're probably wondering So what? Dude, I can put my Madden on auto-play and do the same thing. Who cares who wins a dumb make-believe game? 

Analyzing Replay Challenges

The new WP model allows some nifty new applications. One of the more notable improvements is the consideration of timeouts. That, together with enhanced accuracy and precision allow us to analyze replay challenge decisions. Here at AFA, we've tinkered with replay analysis before, and we've estimated the implicit value of a timeout based on how and when coaches challenge plays. But without a way to directly measure the value of a timeout the analysis was only an exercise.

Most challenges are now replay assistant challenges--the automatic reviews for all scores and turnovers, plus particular plays inside two minutes of each half. Still, there are plenty of opportunities for coaches to challenge a call each week.

The cost of a challenge is two-fold. First, the coach (probably) loses one of his two challenges for the game. (He can recover one if he wins both challenges in a game.) Second, an unsuccessful challenge results in a charged timeout. The value of the first cost would be very hard to estimate, but thankfully the event that a coach runs out of challenges AND needs to use a third is exceptionally rare. I can't find even a single example since the automatic replay rules went into effect.

So I'm going to set that consideration aside for now. In the future, I may try to put a value on it, particularly if a coach had already used one challenge. But even then it would be very small and would diminish to zero as the game progresses toward its final 2 minutes. In any case, all the coaches challenges from this week were first challenges, and none represented the final team timeout, so we're in safe waters for now.

Every replay situation is unique. We can't quantify the probability that a particular play will be overturned statistically, but we can determine the breakeven probability of success for a challenge to be worthwhile for any situation. If a coach believes the chance of overturning the call is above the breakeven level, he should challenge. Below the breakeven level, he should hold onto his red flag.

Eagles Escape Embarrassment

Let my bias not be unknown, I am an Eagles fan. Watching Nick Foles fumble twice, throw an interception, and Chad Henne connect with rookie Allen Hurns twice for touchdowns -- all in the first half -- was one of the more frustrating ways to start the season. The Eagles were lucky to only be down 17-0 at halftime. On the opening drive of the second half, Philly converted twice and Nick Foles connected with Darren Sproles for eight yards on 3rd-and-9, bringing up a 4th-and-1 at the Jaguars 49-yard line.

Chip Kelly is known for his progressive thinking and he didn't hesitate -- calling for a hurry-up, one of the first times the Eagles really played up-tempo in the game. The Jaguars safeties got crossed and a huge gap opened up as Darren Sproles ran untouched for a 49-yard score. The Eagles would not look back and ultimately went on to win "handily," 34-17. Had the Eagles not converted, though, Kelly would likely have been ridiculed for his call as it could have effectively ended the game (dropping Philly's win probability to 5.4%).

Let's look at the fourth down call, taking into consideration the relative strength of the two teams.

Weekly Game Probabilities: A New Home

This season the weekly game probabilities will be featured at Sports on Earth. Each game will have the probability, a score prediction, and a couple notes on why the numbers are what they are. In the early weeks of the season, the numbers are at least partially based on the same preseason estimates of team strength I used for the season projections. But as we get a few weeks of data, those preseason ratings will fade out.

For now the score predictions are simply maximum-plausibility estimates. (Yes, I just made that term up.) Predicting an actual score for each game is statistically boring. With few exceptions, a statistically sound estimate would be 24-20 or 27-21 for every game, so I've added some of the human element to the score predictions. The bottom line is that readers should focus on the probabilities and don't bet the mortgage on the scores.

The game probabilities will be matched up against the picks of Will Lietch, one of the cornerstone writers at SOE. The idea is to create a friendly competition between man and machine.

The game probabilities had a great run at the New York Times--5 years. But there are only so many thought-provoking or counter-intuitive lessons on probabilities and predictions that can be squeezed out of a week of NFL games. But AFA will continue working with the Times on various projects as the season unfolds.

Here's the link to the probabilities for week one. For those keeping score at home, I had the Seahawks at 66% to win last night.

The 4th Down Bot Returns

The 4th Down Bot is returning to the New York Times this season. You might recall we booted him up late last season, but this year he'll be around starting week one. At its heart, the bot is a fun application of the 4th Down Calculator feature here at AFA. It uses both the Expected Points model and the Win Probability model to estimate the best option for every 4th down as a game is in progress.

As I mentioned last year, although the 4th down issue is growing mold with smarter fans, it remains the lowest hanging fruit on the football analytics tree. So it's nice to be able to automate things and not have to do the analysis myself. But on the other hand, we can add 'football analyst' to the list of jobs being taken over by robots.

The Bot will be faster, more accurate, and come with some new features this season. Here is a brief introduction. Here's are a few notes on how it works. And here is his Twitter feed.

Season Projections Visualization

I put together a season projection visualization that illustrates the playoff probabilities and win totals for each team. The numbers are based on the results of the season prediction project I did for ESPN The Magazine.

The method used to create the projections is explained here.

The viz is intended one-stop shopping for the season outlook. The top window shows the probabilities each team will make the playoffs. Dark green indicates a playoff berth by winning the division, lighter green indicates a wildcard berth.

The three windows below are team-specific. Hover the cursor over (or tap) a team's column in the top chart to see its details below. The window on the left is a chart of win totals. The bars represent the probability the selected team will finish with a corresponding number of wins. The second window shows the same information presented in a different way. It's the cumulative probability of each win total. In other words, it's the probability the selected team will win at least that many games. The third window is a pie chart. (Yes, I know pie charts are the unloved orphans of the chart world.) It illustrates the probability each team will win its division.

2014 Season Predictions for ESPN the Magazine

ESPN asked me to predict the 2014  season for their NFL Preview edition of their magazine. I was very hesitant because predicting the season to any degree is extremely difficult. I'm even on record as proclaiming that all pre-season predictions are "worthless." (More on that below). "You want me to predict which teams make the playoffs?" "Yes," they said, "in fact, we want you to predict the winner of all 267 games."

Then it got worse. "We want you to predict every score of every game."

I started doing some math in my head. There's 267 games in the season, including the playoffs, which means there's 2^267 different possible combinations of game outcomes in the season. While that might sound like a lot of different possibilities, it's even more than a human being could possibly fathom. Physicists and astronomers estimate there are about 10^80 atoms in the universe (that's 100 quinvigintillion to you and me). And the NFL season's 2^267 possible outcomes comes to 2.4x10^80, or about 240 quinvigintillion. Put simply, there more than twice as many possible outcomes to the NFL season than there are atoms in the universe. And that just refers to wins and losses, and doesn't even consider scores.

So how hard could it be?

Podcast Episode 27 - Mike Sando

It's an episode full of pivot tables and preseason predictions as Mike Sando, ESPN Insider, joins the show to discuss coaching tiers, predict breakout candidates and preview the upcoming NFL season. Mike shares his process for ranking and evaluating players and coaches, makes some predictions for the upcoming season and reveals his secret analytical weapon - a massive excel spreadsheet he uses to track every conceivable type of player data.

Subscribe on iTunes and Stitcher