Is Sabermetrics Helping to Ruin Baseball?

Caution: A lot of opinion to follow. And please understand I love baseball and I obviously am a believer in analytics.

Imagine a world in which baseball teams had no idea how good or bad a player could be expected to perform from year to year. All players would be total blank-slate mysteries. In this world, it wouldn't matter at all how high or low a team’s payroll is. A team that spends $1M on payroll would be just as likely to win a championship as a team that spent $200M. The correlation between payroll and wins would be zero.

Now imagine the opposite scenario in which teams had perfect crystal-ball foresight about exactly how well a player could be expected to perform. Now the team with the highest payroll would always have the most wins because there would be a 1:1 direct linkage between performance and pay. This would make the correlation between payroll and wins a perfect 1. We wouldn't even need to play out the season. Just mail the trophy to the team with the highest payroll and save everyone the trouble.

In reality we’re about halfway between the first case and the second case. We neither have no idea about expected performance, nor do we have perfect predictive knowledge.  MLB's correlation between team payroll and wins has been trending at about r = 0.4. It varies from season to season, but its recent central tendency appears to be right round 0.4.

The problem that analytics causes is this: The better the sabermetrics (or the more uniformly adopted it becomes), the better the crystal ball becomes and the closer we move toward a correlation of 1...which, by the way, is a bad thing. Of course, we'd never see a correlation of 1 because of injuries and simple randomness, but it could get much higher as it did in earlier eras. The correlation has been higher and lower in years past, varying for reasons that have nothing to do with analytics. But the point is that sabermetrics, to the degree it is effective, makes the correlation higher than it otherwise would be, and this gives an ever greater advantage to big-payroll teams.

Game Probabilities - Week 9

Game probabilities for week 9 are up at the New York Times. This week I take a quick look at much things have changed in New England and Pittsburgh.

Pittsburgh's defense ranks 26th in the league in yards allowed per play, and its offense is struggling to run the ball. The Steelers rank 29th in the league in running Success Rate--the proportion of running plays that result in an improved scoring potential. Only the Giants, Jaguars, and Ravens rank behind the Steelers in running success...

Podcast Episode 7 - Virgil Carter

Joining the show this week is Virgil Carter, the man many consider to be the founding father of advanced football statistics. Dave, Brian and Virgil look back at Virgil's playing days when he was suiting up at quarterback for the Bears while studying for his MBA at Northwestern in the off-season. It was there at Northwestern that he published his first paper, Operations Research on Football. That paper introduced the idea of expected point value based on game situation, an idea that is still at the core of advanced football analysis. Virgil also talks about what it was like to play under head coach Paul Brown, and why if it weren't for him, Bill Walsh might have never needed to create his "West Coast" passing offense. 

If you want to learn more about Virgil and his fascinating career both on and off the field, check out the following links:


-Sports Illustrated article from October, 1972: "Handy Pair of Brainy Bengals
-Virgil's graduate research paper: Operations Research on Football
-Pro-Football-Reference Player Page: Virgil Carter

Subscribe on iTunes and Stitcher

Playoff Projections - Week 8

Just like past years, all of the numbers below come from Chris Cox at NFL-forecast.com. His app uses the win probabilities from the ANS team efficiency model to run a Monte Carlo simulation of the remaining NFL games thousands of times. Based on current records, our estimates of team strength, and knowledge of the NFL's tie breaking procedures we can come up with some pretty interesting predictions of how each team will fare come the end of the season. If you want to use a different model or just fiddle with the numbers by hand, go ahead and download the app yourself.

What is the deal with these numbers?

This is our first look at playoff projections this year so it seems like a good time to talk about how good these predictions really are. Like all models, this one has limitations. Here are some of the issues that will come up.

Team Efficiency Rankings: Week 8

If you've read this column before, you know these rankings are all about cancelling out the noise of past perception and evaluating every team's performance based solely on their data from this season.  That does have some limitations, such as not being able to account for injuries, but once the sample size of games gets large enough, it's a fairly accurate predictor of future success.

Halfway through the season, there are no more huge swings waiting to happen.  That does not mean teams cannot rise or fall—last year, the 3-5 Bengals and Redskins made the playoffs while the 7-1 Bears and 6-2 Giants missed the dance—but the teams near the top are quite likely to stay there.

If we accept that, then perhaps we should start taking our new number team a bit more seriously.

Slate: Punt From the Opponent's 26?

I make my first appearance this season at Slate to propose that DAL may have been better off punting from the DET 26 than attempting a 44-yd FG. If DAL could have pinned DET at or inside their own 10, the numbers suggest punting might have been preferable to going for it or trying the FG. It may have even been preferable to making the FG. I also discuss DAL's holding penalty that was the start of the critical path toward an improbable comeback.

...But there’s an extra wrinkle. Strangely, Dallas would have preferred to keep Detroit within 3 points rather than extend its lead to 6. When desperate teams like the Lions with no timeouts remaining get into the outer rim of field goal range, they send in the field goal unit for a long-range attempt. This is an irrational decision, one I discovered the very first time I began looking at win probability numbers. Rather than try to win the game, teams in this situation settle for a tie—or rather, an attempted tie. Even if the field goal attempt is good, it only buys a 50–50 shot at the win in overtime...

When the Defense Should Decline a Penalty After a Loss Part 2 (2nd Downs)

I recently looked at when it made sense for the defense to decline a 10-yard holding penalty following a 1st down play for no gain or a loss. It turned out that defenses should generally prefer to decline after a loss of 3 or more yards.

First downs are easier to analyze because they almost always begin with 10 yards to go. Unfortunately, 2nd downs aren't so cooperative. It's amazing how thin he data gets sliced up. Most downs aren't losses, even fewer have holding penalties, and rarely are they declined. Still, there are enough cases for a solid analysis using 1st-down conversion probability as the bottom line.

Put simply, a defense would prefer to decline a penalty on a 2nd down play whenever the resulting 3rd down situation leads to a conversion less often than the 2nd down plus the 10 yards.

The chart below plots conversion probability for 2nd and 3rd down situations. The red line illustrates the conversion probability of 3rd down and X to go situations. For example, 3rd down and 7 situations are converted about 40% of the time.

The green line illustrates 2nd down situations, but slightly differently. It plots conversion probabilities for 2nd down and X plus 10 yards. For example, 2nd and 13 (i.e. 3 + 10 yds) situations are converted 45% of the the time. The black line is the smoothed line fitted to the 3rd down conversion rates. I plotted things this way because it's the actual comparison we're interested in, given a gain of zero yards.

Packers' Perfect Third Quarter

After a grown-man run from Adrian Peterson to end the first half, the Green Bay Packers opened up the second half up only a touchdown to the dismal Vikings, 24-17. Aaron Rodgers led the Pack on a 16-play, 80-yard touchdown drive that lasted over eight minutes. During the march, Green Bay converted on three third downs and a fourth down. Let's look at the progression of the drive using our Markov model:

Sunday's Numbers Have Been Crunched

Sunday's numbers are now available, including advanced stat box scores, top players of the week, team stats, and season leader boards.

Advanced stat box scores
Top QBs of the week
Top RBs of the week
Top WRs of the week
Top TEs of the week
Top Defenders of the week
Advanced team stats
Offensive player season leaders
Defender season leaders
Team Viz
Position Leaders Viz
QB Viz
RB Viz

Good News for Hipsters and Grandmothers

If you're a hipster or an elderly person, you probably rely on some kind of Apple product. And that's bad news, because you're missing out on the live Win Probability Graphs. Since 2009 they've been run using the Adobe Flash browser plug-in, which was the bane of Steve Jobs.

You've probably noticed that on your iPad the ANS front page presented you with a big white box with nothing but a line of text. It told you that you don't have Flash, and here's a link to a non-Flash version of the graph. And the link did nothing but take you to another page that told you the same thing and gave you the same link, which finally took you to the old pre-Flash version (which was optimized for circa-2008 Blackberrys.) Embarrassing.

But now things are looking up for you. Unfortunately I can't do anything about the hipster thing--you'll grow out of it eventually. But I can make it a lot easier to see live WP updates without Flash. Starting this weekend I've gotten off my butt and replaced the ugly 'You don't have Flash' error with the Apple-friendly version of the graph. This version isn't interactive and doesn't provide enhanced details, so it won't exactly knock your fedora off, but it's better than a big white box with an error message.

Roundup 10/26/13

Which NFL pundit makes the worst predictions? Umm. Wow. If you just pick the home team you're right 58% of the time.

 Cool visualizations of MLB boxscores.

How do Dez Bryant and Calvin Johnson compare? You might be surprised.

Rivers does a great job breaking down goal line success by play type.

ESPN's NFL Live on-air analysts say TAKE THE POINTS and kick the FG. Unfortunately they're LIVING IN THE 70s. Nice take-down by Jason.

The Myth of Playoff Peyton

Let's play a quick word association game.  I say "Peyton Manning in the playoffs," and you tell me what images your mind conjures up.

For someone who may be the greatest quarterback in NFL history, the picture isn't quite befitting. You probably imagined something like Peyton hanging his head in the snowy Foxboro winter, or slumped over following his pick-six against the Saints in the Super Bowl.  Or perhaps just a general expression of chagrin, like the hilariously petulant "Manning Face."

Yes, Manning does have a losing record in the playoffs for his career (though only eight have had more wins, but who has time to split hairs?).  The next time Manning loses a playoff game will give him the record for most career playoff losses, and fairly or not, that will always be a part of his legacy.

Most rational fans realize that Peyton has only thrown up a handful of postseason clunkers, and that such a small sample size should not significantly affect his standing as an all-time great.  They might defend by saying, "Yeah, Peyton's been a bit worse in the playoffs, but his regular-season numbers are so great that it doesn't matter."

Actually, even that statement would be false.  What people fail to realize is that Peyton has not been any worse in the postseason.  In fact, one could make a fairly convincing argument that he's been one of the two or three best playoff quarterbacks of this generation.

To illustrate this point, let's take off our "Embrace Debate" hats and let the numbers tell the story.

Game Probabilities Week 8

Game probabilities for week 8 are up at the New York Times. This week I take a look at how lopsided this weekend's matchups are.

In terms of probabilities for the favorite and underdog, a game is typically about a 62-38 percentage affair. When the underdog wins almost 4 out of 10 games, it can make for an exciting Sunday afternoon. But this Sunday probably won’t be one of them...

On the other hand, when there are so many mismatches, there is bound to be a big upset. Statistically, it’s very unlikely that all those favorites will win...

Podcast Episode 6 - Ben Alamar

Ben Alamar, author of Sports Analytics: A Guide for Coaches, Managers and Other Decision Makers is on the show this week to discuss his new book. Ben is a professor of sports management at Menlo College, and has worked as a consultant to a variety of NBA and NFL teams.

The episode begins with Ben describing his background in economics and why he saw a need to found the Journal of Quantitative Analysis in Sports, the only peer-reviewed journal solely for papers on sports economics. They then discuss his work with Michael Lewis for the book Moneyball, and how he approached the question of how to analyze the success of offensive linemen.

Ben then discusses his new book, and breaks down some of the experiences he’s had working with analytics programs of professional sports organizations. Finally, the episode ends with his thoughts on the recent NFL season by continuing the now weekly tradition of praising Peyton Manning for his otherworldly statistical year.

A quick note to listeners, I'll be announcing each week's guest on twitter a few days before recording, so tweet at me if you've got questions you'd like the guest to answer on the upcoming episode.

Subscribe on iTunes

Team Efficiency Rankings: Week 7

For the first time all year, the top 10 teams from last week remained the same.  In fact, no team moved more than five places in the efficiency rankings, the least movement we've seen so far.  This does not means things won't change, however, especially considering the spate of critical injuries we've seen the past couple of weeks.  Something tells me the Bears won't remain the ninth-most efficient passing offense with Josh McCown at the helm for the next month (#analysis). 

The top was not entirely constant, as only the top-ranked Seahawks, second-ranked Broncos and ninth-ranked Chargers remained in their spots.  The Bengals sitting at fourth looks a bit curious, but the majority of the top-10 seems fairly reasonable.  In fact, apart from the offensively-uberefficient Eagles, these rankings seem to agree with the mainstream perception.

However, there is one particularly curious disagreement, as the team at the top of the league standings is nowhere near the top of these rankings.

Was BAL's Onside Kick Attempt Smart?

Trailing with 13 minutes left in the 4th quarter vs. PIT, BAL kicked a FG to make the score 13-9. BAL then attempted a surprise onside kick attempt, which was unsuccessful. What do the numbers say?

Surprise onside kick attempts are generally worth the gamble. Based on the Expected Points model, a normal kickoff is typically worth -0.42 EP, as the opponent has a 1st down at about their own 22. A recovered onside kick (1st down at one's own 45) is worth 1.77 EP. A failed onside kick (opponent's 1st down a one's own 45) is worth -2.36 EP. The break-even recovery probability would therefore be:

Sunday's Numbers Have Been Crunched

Sunday's numbers are now available, including advanced stat box scores, top players of the week, team stats, and season leader boards.

Advanced stat box scores
Top QBs of the week
Top RBs of the week
Top WRs of the week
Top TEs of the week
Top Defenders of the week
Advanced team stats
Offensive player season leaders
Defender season leaders
Team Viz
Position Leaders Viz
QB Viz
RB Viz

Jets 'Push' Their Luck

After forcing overtime, the Jets stopped the Patriots on their first drive, reverting to the old OT format - sudden death. Geno Smith and the Jets moved downfield before being stopped for a 4th-and-7 from the New England 38. Rex Ryan had three viable options here, keeping in mind that the next score wins: Kick a low-probability (40% league-wide) 55-yard field goal, attempt to convert a low-probability (42% league-wide) 4th-and-7, or punt the ball deep and risk Tom Brady leading a game-winning drive.

The Jets elected to attempt the field goal. Nick Folk missed wide left, but in a crazy turn of events, New England was penalized 15-yards for an unsportsmanlike conduct "pushing" penalty. Before we get to the penalty, let's talk about the decision. While I almost always advocate going for it in no-man's land, in this situation, I was leaning toward the punt.

For this analysis, I used a combination of my Markov probabilities as well as Brian's overtime win probabilities.

Dolphins Should Have Intentionally Allowed the TD

Leading by a point, the MIA defense was on its heals as BUF drove into FG range with less than 3 minutes to play. With all 3 timeouts, MIA must have been feeling pretty good about its chances of getting the ball back with some time to retake the lead.

BUF ran the ball, forcing MIA to use its timeouts. With one timeout remaining and 2:37 to play, BUF faced a 3rd and 4 at the MIA 28. BUF ran the ball for a 10-yard gain, earning a fresh set of downs and forcing MIA to use its last timeout.

At this point, MIA would have preferred to allow BUF to score a TD. The odds favored scoring a TD of their own in response. Accordingly, BUF should have preferred to take a knee rather than score then.

Here is the chart, built using this research. The red dot is where MIA found itself. The dotted black line is the Win Probability curve for allowing the TD, and for reference, the teal line is the 20 yd line.

We're Gonna Party...

Like it's 1999. I've got the 1999 season added to the database now, thanks to the league's efforts to digitize that season's numbers. That means we now have advanced team stats, offensive player stats, defensive player statsteam stat visualizations, player stat visualizations, and WP graphs for that notable season.

It's great to have the season included for a few reasons. Now we get to see the offensive output of the '99 Rams--The Greatest Show on Turf. I was actually surprised that their EPA totals weren't higher. It was also the first season of the reconstituted Browns franchise, and 1999 was Peyton Manning's first winning season, leading the Colts to a 13-3 record in his second year.

We also get to see the full careers of guys like:

Roundup 10/19/13

A Visualization of NFL betting lines. Cool viz, but it makes me wonder how many people out there are, say, betting on the Cowboys because they're so good at beating the spread this year. I'd think a betting system that just counter-bets stupid analysis would be a winner. But then again, there are probably already thousands of sharks who do that.

How have defenses adapted as the fullback disappears from NFL offenses?

Little technicalities like this can drive guys like me crazy when the numbers don't add up. Neil has the answer.

Peyton Manning is obviously on steroids.*

Did the Toronto Maple Leafs prove advanced hockey metrics wrongPhil's take.

Never miss an installment of Bill's Thank You for Not Coaching series.

Hear Me "Analyze All Football" at the Fangraphs Podcast

Carson Cistulli, a name familiar to longtime ANS readers, invited me on the Fangraphs Podcast. Hear us break down every conceivable topic in football.

On the Effect of Coaching

There are exceptions, like that guy on the left, but my hunch is that NFL coaches are mostly interchangeable.

I think at the NFL level, all coaches employ the same best practices. There is no secret sauce that one coach has over another in terms of instruction, motivation, strategy, etc. This is because of the highly mobile, fluid market for coaches and the large size of their staffs. There are very strong constraints on deviation from league norms in any dimension.

Also, from statistical analysis, we can measure the variance in team performance attributable to randomness (sample error due to a short 16-game season) and player impacts (the addition or subtraction of a player's impact on team production, player interaction effects). There is very little variance left that can be attributed to other causes, including coaching. In other words NFL outcomes are overwhelmingly driven by player talent and luck, and there's not much room left for coaching to make a big impact.

We can observe this intuitively, as the very same coaches can have wildly different records from year to year. How much effect can they really have?

Game Probabilities Week 7

Game probabilities for week 7 are up at the New York Times. This week I take a look at the Patriots Jets matchup and some surprising numbers.

A purely statistical approach is (deliberately) ignorant of things like the identity of the quarterbacks. It doesn’t know about Super Bowl rings or supermodel wives. It doesn’t know who is a rookie and who is a certainty for the Hall of Fame. It doesn’t know that Patriots linebacker Jerod Mayo is out for the regular season with a chest injury.

Podcast Episode 5: Brian Burke

Brian Burke is back for another episode of the Advanced NFL Stats podcast. This week, Dave dives into Brian’s most recent articles. First up is a study analyzing when defenses should decline penalties after a loss. This piece uses expected points to determine the break even yardage required for a defensive coach to opt for a loss of down over a loss of yardage.

The second part of the episode is dedicated to the Baltimore Ravens and their mediocre awful rushing attack. Inspired by his favorite team’s recent offensive woes, Brian looked at which teams might be better off if they abandoned the run completely as part of their offensive game plan.

The show concludes with a discussion based on a comment from the Brian’s previous podcast appearance. Dave asks Brian to justify his claim that most NFL coaches are relatively interchangeable, and that at the NFL level, there is little discernible difference among head coaches.

Subscribe on itunes


Update to the Top Game Finder

I got the same unusual request from two separate people 10 minutes apart last night. Each one wrote that they and their wife watch NFL Game Rewind during the week, and they use the Top Game Finder to see which games are worth watching. (Yes, I asked to make sure it wasn't the same guy writing twice.) The problem is that the Finder results include each team's score, spoiling the game. One of the requests noted that he and his wife take turns using the Finder so the other one won't know the outcomes of the games they watch.

The request was to build a version of the tool that would not reveal the game outcome. That was easy enough, so if that's something others would be interested in, here is the link. (Note the address simply appends NoScore to the usual filename.)

http://wp.advancednflstats.com/greatgamesNoScore.php

As a bonus I thought it would be useful to include a Week field in the search bar. Unless one of the games during the recent week pops up as a game with an exceptionally high Excitement Index, you'd have to search team by team within a season to identify the best games of the week. Now you can enter the most recent week in the dropdown menu and get a list of that week's games sorted according to EI or Comeback Factor. And if you'd like to know the most exciting game of Week 6 of the 2004 season, it's at your fingertips. (STL 28 TB 21.)  I've added this to both the normal tool and the score-free version.

Analyzing the Patriots' 4th Down Calls

Kurt Bullard is a freshman at Harvard and a first year member of the Harvard Sports Analysis Collective. He intends to major in either Economics or Statistics. Go 'Cuse.

This Sunday, the Patriots found themselves down six points to the Saints with only 1:13 left in the game. In that span, Tom Brady was able to lead a drive down the field, connecting on a 17-yard pass to Kenbrell Thompkins in the left corner of the end zone to complete one of Boston’s two major come-from-behind wins of the day.

Hidden by the final drive were two controversial fourth down calls in the fourth quarter that happened at key moments in the game. The first came with 8:34 remaining in the game when the Patriots held a 20-17 lead over the Saints. Faced with a 4th-and-goal at the five yard line, New England opted to kick a field goal rather than try to score a touchdown to go up two scores.

The win probability and expected point value each suggest different optimal decisions. The expected point formula suggests that the Patriots ought to have gone for it, while the win probability calculator says otherwise, albeit by a slight margin.

Team Efficiency Rankings: Week 6

Not much movement in the rankings this week, at least compared to the first three installations.  As I alluded to last week, the GWP gaps should start to solidify as the sample size becomes larger and teams separate themselves more definitively.

We can see that immediately in our top 10, where only the Lions fell out.  This does not necessarily mean a team outside the top-10 cannot become a legitimate contender of course—at this time last year, the Super Bowl champion Ravens were 11th, and eventual playoff participants Washington and Indianapolis were 22nd and 27th, respectively.  Four of the top 10 teams also finished outside the postseason.

To figure out who might fall into that ignominious category in 2013, let's take a look at two of the most likely teams to miss the playoffs, despite what their current ranking might suggest.

When Should the Defense Decline a Penalty After a Loss? Part 1

Let's say there's a sack or other tackle that results in a several-yard loss. And to compound the offense's woes a flag for holding is thrown, potentially setting up a 1st and 20 situation. Should the defense accept or decline the penalty and force a 2nd and X? We can evaluate this question in a few ways. We'll use a simple method and a more complex method to find out when a defense should normally decline a penalty on first down.

Before you read on, what do you think the break-even yardage is? What do you think most coaches think it is?

Sunday's Numbers Have Been Crunched

Sunday's numbers are now available, including advanced stat box scores, top players of the week, team stats, and season leader boards.

Bengals Punt Away Regulation

Heading into the fourth quarter, the Cincinnati Bengals led the Buffalo Bills handily, 24-10. With just over 10 minutes left, the Bills drove down the field and found themselves in a 4th-and-8 from the 22-yard line. With that much time left on the clock, it is safe to assume that most NFL coaches would kick the field goal. In fact, with backup Thad Lewis under center, the probability of coaches kicking the field goal would increase. The Bills made a gutsy call and went for it, hitting Scott Chandler for a 22-yard strike and scoring. The numbers say it’s a toss up:

Which Teams Should Abandon the Run?

Yeah, yeah, yeah. It's a passing league. We got it. And still, according to the numbers, teams aren't passing enough. In the cases of some teams, it's painfully obvious that they should be passing more and running less. As a Ravens fan, I watched another game where nearly every run was simply a wasted down. Most of their paltry positive rushing yards seem to come from trash draw plays on long distances to gain, intended to mitigate very poor field position prior to a punt. It's like they're playing with two or three downs when everyone else gets four.

I wonder if, at some point, when an offense is so much better at passing than running, should it abandon the run almost altogether. On top of the general imbalance in the league, some teams are just throwing away downs when calling conventional run plays. Of course, running and passing generally play off of each other in a game-theory sense. To be successful, passing needs the threat of running, and vice versa. But sometimes, the cost of running is so high for some offenses, that it would be worth the trade-off to forfeit the unpredictability and just pass nearly every down.

It sounds crazy, but take a look at the Expected Points Added per play so far this season (through the 1pm games on Sunday 10/13). The right-most column is the pass-run split. The bigger that number, the greater the imbalance. Pay particular attention to the teams highlighted in red:

Return of the Roundup 10/12/13

The Roundup is back. We've dusted it off to provide some of the best smart sports research links in one place. Wasting your time on a lazy Saturday since 2008.

Cool visualization of NFL salaries by position.

Going for it on 4th down at the end of a half is a unique animal because there's no chance of either the opponent scoring or getting the ball back in great field position. Jason Lisk uses the WP Calculator to crunch the numbers.

How the Packers use individual player stats.

Where have all the fullbacks gone?

People Consume 10x More Music Than Sports, So Why Does Sports Make 6x the Cash? via Marginal Revolution

The Truth Is Tony Romo Isn't a Choker

Last week, a premier quarterback threw a devastating fourth-quarter interception that effectively sealed his team's fate in a back-and-forth game.  This critical gaffe perpetuated what has become an increasingly disturbing trend of failures to "come through in the clutch," as critics like to deride.  Despite gaudy statistics, he has a recent track record of playing his worst in big games, raising questions as to whether or not his team's so-called championship window remains open.

Clearly, Tom Brady is the NFL's biggest choke artist.

That last example is just one way real and armchair analysts alike can place selective focus on certain facts to create a skewed perception.  Perhaps no player has had more damage done to his reputation in this manner than Tony Romo.  Romo played the game of his life against the Broncos last Sunday before a single ill-timed mistake reignited accusations of "choking."  As Grantland's Bill Barnwell put it, Romo did not have a perfect game, but rather the "perfect Tony Romo game."

But what does that even mean?  Most people know that Romo is not really as bad as his choker reputation implies.  More sensible fans may even understand that Romo receives far too much blame for the organizational and team-wide failures of the Cowboys.

However, do people know that Tony Romo may actually be the most clutch quarterback in the league, or at least very close to it?  To answer this, let's delve into the murky depths of "clutchness" through a couple different lens.

Play-by-Play Data Update

I finally got around to updating the play-by-play data with last year's postseason. Plus, I've added weeks 1 - 4 of 2013. It's available here, or via the main drop-down menu under the About tab. Nearly every NFL play since 2002 is included for your research purposes. Happy crunching.

Examining the Value of Coaches' Challenges

Kevin Meers is the Co-President of the Harvard Sports Analysis Collective. He is a senior majoring in economics with a statistics minor, and has spent the past two years or so as an analytics intern in the NFL. He is currently writing his thesis on game theory in the NFL, and probably puts too much thought into how the perfect fantasy football league would be structured.

The coach’s challenge is an important yet poorly understood part of the NFL. We know challenges are an asset, but past that, we do not have a good understanding of what makes a good challenge or if coaches are actually skilled at challenging plays. This post takes a step towards better understanding those questions by examining the value of the possible game states that stem from challenged plays.

To value challenges, we must understand how challenges change the game’s current state. When a play is challenged, the current game state must transition into one of two new game states: one where the challenged play is reversed, the other where it is upheld. These potential game states are the key to valuing challenges.

Let’s look at a concrete example from last season. With two minutes and two seconds left in the fourth quarter in their week ten matchup, Atlanta had first and goal on New Orleans’ ten-yard line. Matt Ryan completed a pass to Harry Douglas, who was ruled down at the Saints’ one-yard line… only Douglas appeared to fumble as he went to the ground, with the Saints recovering the ball for a potential touchback. When New Orleans challenged the ruling on the field, the game could have transitioned into two possible game states: Atlanta’s ball with second and goal on the one, or New Orleans’ ball with first and ten on their own 20 yard line. If the Saints lost the challenge, they would have a Win Probability (WP) of 0.28, but if they won, their WP would jump to 0.88. This potential WP added, which I refer to as “leverage,” is key to valuing challenges. Mathematically, I define leverage as:

Game Probabilities Week 6

Game probabilities for week 5 are up at the New York Times. This week I discuss the how teams with great offenses can mask the true strength of their defense.

Since the 2000 season, the 32 teams with the best offenses allowed 2.1 more Expect Points per game than the average team over the same period — a significant difference. This translates to about a 4 percent chance of winning a game when matched against a roughly equal opponent, and slightly less when matched against a lesser opponent.
By the way, I gooned up the table on the post, leaving out Monday night's game. It's .57 - .43 IND over SD.

Podcast Episode 4: Dean Oliver

Dean Oliver, Director of Analytics at ESPN Stats and Info, joins the show to discuss ESPN’s quarterback measurement tool, QBR. Dean led the development of QBR, and has shepherded its development since its release in 2011.
Dave and Dean begin the show with a trip back in time to Dean’s childhood in Hawaii, where he explains how he first became interested in statistics and developed his own baseball simulation dice games to play with friends.
They then discuss QBR in depth, looking specifically at the types of questions it is designed to answer. Dean explains how QBR’s component parts work together to describe a Quarterback’s contributions to his team’s success. He also discusses why certain QBs may surprise people with their ratings, and why we’ll be talking about Andrew Luck for years to come. Finally, they end the episode by covering how the stat has changed and improved since its inception, and look at how the college version of QBR differs from its professional counterpart.

Subscribe on itunes

Team Efficiency Rankings: Week 5

Take a look at this week's rankings and you might notice happening: as we move away from the small sample size caveat, teams are beginning to stratify themselves by tiers.  Not only does this allow ESPN talking heads to scream about whether or not teams are "elite," but it also begins to validate some early-season surprises.

Most tellingly, we can see the separation in the top 10 teams, and not simply because 10 is a nice round number highlight shows like to use.  Removing the ridiculous outlier that is the Denver Broncos, and all the teams from second-ranked Seattle to 10th-ranked Cincinnati have a GWP somewhere in between 0.67-0.59.  The 11th-ranked Jets immediately drop off to 0.55, providing a convenient mark-off point.

Examining the top 10, there's one team in particular that few believed in before the season, but now looks like a viable contender.

A Plug for the Community Site

Let's face it. The world of football analytics has changed greatly over the years since Advanced NFL Stats broke ground on its multi-million dollar research complex in 2006. These days, a small portion of analytics is being done outside of the paradise-like garden of forward-thinking that is ANS. I've been told that up to one, possibly two percent of football research is being done elsewhere. Hard to believe, but it's true.

That's where Advanced NFL Stats Community comes in. If you'd like to publish your own research, you are free to do so at ANS Community. We haven't seen a submission since last season, and loyal editor Ed Anthony hasn't had much to do over the past few months. (I'm beginning to rethink his guaranteed salary of $6.5 million...no, wait. That's Matt Flynn, not Ed. I only pay Ed like $80,000/yr, but I'm not worried because it's in Canadian pesos or whatever they have up there. So that's what, $50 bucks or something in actual money, right?)

I know there's a lot of research out there waiting for the bright lights of ANS Community. I must have sent data to and answered questions for over (honestly) 100 students, from high school students to PhD candidates to tenured professors. And there are lots of armchair statisticians out there like myself, with their own ideas about football stats. Every post at the Community site has been read by thousands of readers, and the very best will find their way onto ANS proper.

I was going through the posts at the Community site and I was surprised to see there are 133 posts over the 5-year life of ANS-C, a substantial amount--an amazing 200 articles per year! (Or about that. I'm not good with numbers.)

Some of the posts were a bit out-there, but some of them are real diamonds. A few of my favorites were Bryan Davies' explanation of why a team should always go for two when down 14 and Michael Beuoy's team rankings based on multiple simultaneous equations of point spreads. (You can always check Michael's up-to-date rankings at his site Inpredictable.com.) Longtime reader Adam Tarr put together this examination of QB YAC, which broke the site's record for TL;DR flame-war comments. I liked this one by 'Tunesmith' on the Kelly Criterion, a financial market concept. And here's Ed's own (puzzling) contribution on kicking or receiving to start the half.

Here are Ed's favorite posts through the years:

Bengals' Big Time Drive

A week after losing to the Cleveland Browns, the Cincinnati Bengals were tasked with taking down the undefeated New England Patriots in Week 5. The game was not enjoyable to watch and had a baseball score of 6-3 heading into the fourth quarter. Needing to separate themselves from the Patriots, the Bengals put together a 15-play, 93-yard, nearly eight minute drive, ultimately scoring a touchdown to go up two scores, 13-3.

Using our Markov model, we can look at the evolution of the drive:

When Should DAL Have Allowed DEN to Score a TD?

Once Demaryius Thomas crossed the line to gain at the DAL 14 yard line with 1:50 to play, the DAL defense should have intentionally allowed the TD. With 2 timeouts and 1:40+ to play, they would have had a better chance of winning than allowing DEN to choke the life out of them and kick an easy FG for the win.

That's based on this analysis of when teams should intentionally allow a TD when tied.

Aften Thomas gained a 1st down, DAL correctly called a timeout setting up a 1st down and 10 at the DAL 11 with 1:49 to play. This chart illustrates DAL's chances of winning based on field position and time remaining with 1 timeout left.

Denver over Dallas: Instant Analysis

Sunday's game was a hard fought contest of wills, but in the end Denver pulled out the much-needed victory over Dallas. Question marks remain for both teams, however. Denver was finally able to find their identity and stick with what works. Although it's not time to panic yet in Dallas, they're going to need to find a way to get it done before it's too late.

Peyton Manning silenced his critics with a solid, although imperfect, outing. "For me it's not about the individual records," he said after the game. "It's all about the team. I'm just glad we were able to go out there and get the win today," he added. Peyton Manning needs to be careful not to look past next week's opponent. Their upcoming match-up will be nationally televised in prime time, a classic trap game.

Tony Romo looked confident at times, but eventually let the game slip out of his grasp. He said after the game, "We didn't bring our A-game today. We didn't step up when we needed to and make the plays we needed to make." Tony Romo showed a lot of courage in the face of adversity on Sunday. Now he needs to put it all together for Dallas to get back among the top teams in the league.

Dallas just can't continue to make the kinds of mistakes they made at this level of the sport. Dallas fans had higher hopes for the defense this season, but if they keep playing at this level significant changes will be needed in the offseason. The offense's performance was also less than inspiring, stalling numerous times in opponent territory.

Dallas lost that game more than Denver won it. Everyone knows football is a game of emotion, and clearly Denver showed they had the passion to come out on top. You could feel the electricity in the stadium from kickoff to the final whistle. It was a game for the ages.

Don't like this column? Generate your own with the ANS Sports Column Generator!

Sunday's Numbers Have Been Crunched

Sunday's numbers are now available, including advanced stat box scores, top players of the week, team stats, and season leader boards.

Introducing ANY/A Differential

The favorite passing stat of many in the advanced stat world is adjusted net yards per attempt, or ANY/A.

Despite this, there does not exist on the internet (to my knowledge) any database of single game ANY/A, or, more interestingly, ANY/A differential and its correlation with winning football games.  You can get yards per attempt differential correlated with wins, or passer rating differential -- but not ANY/A differential.

I took both challenges on in my spare time this past week to bring you such data today.  And as the season rolls forward, I will continue looking back into seasons past to both establish a database of single game ANY/As and also see what use ANY/A might serve as a predictive tool.

Rest vs. Rust After Thursday Night Football

Andrew Mooney is the Co-President of the Harvard Sports Analysis Collective. He is a senior majoring in Social Studies, which is another way of saying he's an economics major. Andrew has worked as an analytics intern in the NFL for about two years, and previously wrote for the Stats Driven blog at Boston.com. He's a big fan of all Detroit sports, and he'll throw an octopus on your ice if you're not watching.

In struggling out of my bed at the witching hour of 8:00 am this morning, I had to wonder how much more equipped to tackle the day’s challenges I would be with an hour more of sleep. I then noticed that I wasn’t missing the tip of my finger, nor had I sustained a concussion the day before, incidents from which I would need significantly more than a week to recover. In this state of empathy, I couldn’t think of a more welcome time of the NFL season for a player than an extra day or two off.

Though I’m sure it provides players some much needed rest, it is not immediately clear what effect this time off has on performance. The qualitative cases for each side are pretty straightforward, and your grandfather used each of them liberally in instructing you in the wonders of sporting conventional wisdom. “Ah, they had an extra week to prepare AND get healthy,” he said knowingly after Washington’s 31-6 thrashing of the Eagles last season. “They just got rusty,” he told you after the Vikings fell to Chicago, 28-10, the following week. “What in the Sam Hill…” he muttered after the 49ers and Rams battled to a 24-24 tie.

Podcast Episode 3: Chase Stuart

Chase Stuart, founder of Football Perspective and a regular contributor to the New York Times, calls in to the show from his home in New York City. Dave and Chase begin by discussing two point conversions, and how expected value and variance come into play when coaches make the call to go for two. Chase goes on to discuss his own metric for looking at in-game point differential, called Game Scripts. He explains how Game Scripts can be used in conjunction with run/pass ratio to contextualize a team's general tendencies over the course of a season. The episode ends with a check-in on season win projections for the five remaining undefeated teams, along with some well-deserved gushing about Peyton Manning's unbelievable season.

Make sure to subscribe on iTunes so you don't miss an episode!

Game Probabilities Week 5

Game probabilities for week 5 are up at the New York Times. This week I discuss the preeminence of offense over defense.


Team Efficiency Rankings: Week 4

At the quarter-pole of the season, we know about the extremes with a fair degree of certainty.  The Broncos offense is quite literally off the charts, and the Jaguars everything could be historically bad.

In the middle is where the room for significant variation still lies.  Eight teams moved at least six places in the rankings (though interestingly, only the Giants at 17 stayed in the middle third).  And yet, even with all this movement, five of the top six and five of the bottom six remained at their respective opposites of the totem pole.

Even though the small sample size caveat still applies, it's safe to assume the top and bottom are fairly fleshed out.  So now the goal is to find which lesser-heralded teams are contenders who could steal the Super Bowl, just as the Ravens and Giants have done the past two seasons.  Let's start in the top 10, where we find two teams who few would consider serious threats.