The Excitement Index (explained here) lets us measure how riveting a game was. Here are the average EIs for the division round games by year since 2001.
Season | Avg. EI |
2006 | 5.83 |
2012 | 4.90 |
2003 | 4.73 |
2011 | 4.35 |
2007 | 4.23 |
2002 | 4.05 |
2005 | 3.83 |
2008 | 3.73 |
2004 | 3.48 |
2001 | 3.40 |
2010 | 3.30 |
2013 | 2.78 |
2009 | 2.50 |
This year ranks second only to 2009, when NO beat ARI 45 to 14, IND beat BAL 20-3, MIN beat DAL 34-3, and NYJ beat SD 17-14. The NYJ-SD game was was decent 4.2 on the EI scale, but the other games were dogs.
This year, the most exciting game was SF-CAR with an EI of 3.1. For comparison, Last year's BAL-DEN heart-attackathon was an 8.5, which moved the Win Probability needle more than 3 of the 4 games of 2013 combined.
2006 leads the pack as the season with the most exciting division round. That year IND beat BAL 15-6 in what was the snoozer of the weekend (3.8 EI). NO squeaked by PHI 27-24 (5.3 EI). CHI edged SEA by the same score (7.5) EI. And NE beat SD 24-21 (6.7 EI).
I think the EI should be revised. The current formula only looks at the movement in WP, and not the WP itself. A hypothetical game where the WP graph stayed at flat at 50-50 until the last play of the game would be considered "boring" by this measure, even though the outcome was in the maximum possible doubt throughout the game.
JD-Actually, EI is most sensitive when the WP is near .50, just the way you describe. Please read the link in the main post to learn more.
I did read the link. EI being most sensitive near .50 means that the WP has the most potential to move at that point. And that's what makes the game exciting: the **potential** for WP to move, not the movement itself. Which is my point. The EI measures the results, while really the anticipation of the results is what's exciting. A game where the WP curve stayed flat around 0.5 until the last play would probably be a very exciting game. While a game where the WP swung wildly for a few plays at the end might only be exciting for those few plays, but boring for the previous 3 hours. EI would proclaim the latter game as being leaps and bounds more exciting than the former game, while in reality the excitement was much more sustained in the former.
JD, how would it even be possible for the WP to stay near 0.50 for an entire game? That would mean no turnovers, no drives into opposing territory, no big plays... it would be a lot of short gains followed by long punts. Not exactly an exciting game.
@JD: The potential for WP movement at any point is captured by EI because the distribution of WP change probabilities is dependent upon the same parameters as the current WP value. It's not displayed as a simple, in-your-face scalar value but it is being measured.
Drew--same parameters, but different realizations, and the realizations are what counts.
James, you have a good point. The flat-0.5 game would be unexciting in terms of the nature of the action. But...it would very exciting in terms of uncertainty of the ultimate outcome. Those are two different definitions of excitement, both valid. EI measures yet another definition of excitement, namely the *volatility* of the expectation of the ultimate outcome. I don't see a compelling case for why the EI definition of excitement completely trumps the other definitions. EI focuses on one aspect of excitement and neglects others.
@ Drew I was thinking the same, but I think it doesn't hold up to closer scrutinity. A game with a big comeback like last weeks KC vs Indy Bout gains an "automatic" 1 WP due to the comeback effect. This is hard to catch up for a more normal game.
Take Indy vs KC 2013 (EI 3.8)
OTOH Indy vs KC 2003 a great score for score game which had a slow rise of Indys GP throughout the game manages only and E.I. of 3.4
So maybe the tuning really need to e better between this scenarios.
EI picks the 'uncertainty' effect up very well. The theoretical permanent .50/.50 WP game is implausible in a number of ways. It would require both teams to permanently punt back and forth to each other's 15 yd-line.
Plus, even a game like that would set off the Richter scale toward the end. Every little yard gained or lost would cause very large swings in WP. Eventually someone would have to gain the upper hand, and in that process there would be large fluctuations in WP. Look at any game that eventually went into OT. The end of the 4th quarter sees the WP go nuts.
It seems to me that in a boring game, one team comes out to a big lead and never relinquishes it. In this case, the WP graph would look like a monotonic curve toward one extreme. (I think it would look something like an exponential.) One could define excitement as being any game situation where the outcome is more uncertain (closer to the midline) than that. For example, you could take the total length of WP curve falling between the two opposing "boring blowout" curves.
True, such a metric would weigh tight low-scoring games more than sloppy back-and-forth games, but I would consider that a positive feature. After all, the CBF already makes a decent measure of last-second rapid swings of WP and surprise come-from-behind victories. This way, the "excitement" metric would measure something a bit more distinct.
Alternatively, what about a metric that measured the "interestingness" of a game? One would weight each play by the inverse of the probability of its outcome. For example, a random passing play has a low probability of ending in an interception, so an interception is interesting. So is a 50-yard gain. Making it on 3rd-and-27: interesting. Getting stuffed on 4th-and-inches: interesting. A 20-yard field goal attempt has low probability of missing, so that's interesting. One might want to weight by WP movement as well, since a team that's up 20 missing a chip-shot still wouldn't be very interesting.
There's an easy fix, and all you need to do is weigh WPA for its distance from 0.5. As a result, a WP Change from .45 to .55 would have far more value than .9 to .8.
ESPN QBR does this on a 0.3 to 3.0 scale
No fix is required. WP (as a concept) is *perfectly* scaled according to the uncertainty of the game. It is a "linear utility function," meaning that moving the WP from .90 to .89 is precisely equal in terms of outcome uncertainty as moving it from .50 to .51.
A yard gained or lost when the WP is close to .50 moves the WP needle far more than that same yard gained or lost moves the needle when the WP is near .90. And it does so is exactly in proportion to the yard's effect on the outcome of the game.
That's the beauty of WP. It already does what people above are proposing fixes and adjustments to do. Over-weighting an arbitrary range is not a good idea.
(PS QBR does not scale WP, it scales EPA according to a WP cutoff to discount trash time.)
Ultimately, I think the EI formula is simply incomplete because in practice, it becomes mostly a measure of late-game volatility. A game rated as very exciting will probably have almost all of its excitement concentrated in a few plays in OT or in the last few minutes. And that just feels wrong, because I relish the entire game watching experience, not just the end of it. If you are served a delicious pizza, is it just as satisfying to gulp it all down in one minute, or to savor it one small bite at a time?
I thought the Seahawks-Saints game got pretty exciting at the end.
We are not talking about a fix on WP which should be what you say it is, but on the question whether EI catches Excitement as it feels during the games.
But after checking further games in the DB I withdraw my objection.
Games like KC@ IND with EI of 3.8 are actually quite average from an EI point of view, and all NE games of the last few years are actually pretty accurately ranked.
Makes sense; but wouldn't QBR be using something more than a WP cutoff? I understand cutting off trash time with a clutch index of 0.3, but what of 3.0? wouldn't that imply its when the game is on the line?
Another angle of what JD is saying is that sometimes the excitement is derived from the plays themselves.
Imagine a team runs for exactly 4 yards per carry. They go 6/6 on 3rd downs, and finally punch it in after consuming almost 13 minutes of clock.
Imagine the same team instead returns the kickoff 108 yards for a TD.
EI would actually rate the first team as having the more exciting drive because the total change in WP would be higher.
Perhaps another metric could be to look at the sum of the absolute slopes.
Or the number of "Exciting plays" where the change in EP (or WP) was over a certain threshold.
What were the most exciting games of 2013 by this index?
Go to Tools -> Top Games Finder and select whatever Criteria you like.
Thanks. Ha, my Vikings were in five of the top eight games, at least there's that.
Brian,
How did this game get a comeback factor of 100? http://wp.advancednflstats.com/nflarchive.php?gameid=56060
It looks like the lowest win probability was 10%. So 1/0.1= 10.
I checked the play by play and it doesn't look like the graph is missing a play where the probaiblity would have been down to 1%.
I had trouble buying into EI as a measure at first. Some of the concerns were about whether its appeal was based too much on the elegance of its simplicity (often simple models end up not catching the nuances that can explain things). Some were akin to those JD Krull mentions--would a "close tight game with few swings in WP" be exciting but end up with a low EI?
Since, I've converted and I do think that the responses that Brian and some others posted answer JD's objections well. There are a couple things that I'd add to this which may help others and perhaps JD understand why those responses are so compelling.
I spent time on the WP calculator trying to engineer this "exciting" game with a low EI that would help disprove EI as an effective metric to use for judging excitement. It's pretty difficult and also a nicely illustrative lesson about what makes the WP scale shake (try it yourself!). It also helped me understand another reason I love football, which is that it is always changing. You can hold WP between 45 and 55 for a while, and there is even some variability in how it stays there. You can have long field goals--as long as the team doesn't get a first down very far into opposition territory. It helps if the team who kicked off first also scores first. But really this game we're creating isn't very exciting. It mostly involves running and slightly above average play by linebackers, lots of long punts, relatively few first downs, and no kick or punt returns of any sizable length. Long completions are out, as are turnovers.
You learn pretty fast why we watch and track 3rd downs: they are big swing plays. In this game of low EI, 3rd down is almost always with 1 or 2 or maybe 3 yards to go and few of them are converted. It turns out that this game isn't exciting at all. And when I did the WP calculator equivalent of fast-forwarding on my DVR, I found that at the end of the game it is downright IMPOSSIBLE to have the WP stay that balanced: every change in field position changes things a lot, changes in down with no yards gained are more costly for the team in possession, changes in possession are huge. All this squares with intuition but I didn't internally process just how hard it would be for a game to be close yet boring by EI. It is hard to believe lots of people would watch games where great punting and mediocre linebacker play are the stars of the show and very few passes are attempted, most are short completions.
JD is right that there are different types of exciting. Some games are big leads in which there is a giant late comeback: KC blowing the lead to the Colts on wild-card weekend and the Buffalo comeback against the Houston Oilers are two examples. These games are great comebacks but for most of the game it is a snoozer and more to the point it LOOKS like it will be a snoozer. On the other hand, there are games like this year's Balt-Minn game in which 5 TDs were scored in the last 2-3 minutes, and all changed the lead. The Music City miracle game, similar but with higher stakes as it was playoff game, wasn't just exciting because of Wycheck's pass—the whole 4th Q was back-and-forth. I'd argue that these types of games are more exciting than the big comeback games and I think EI backs me up on that.
Lastly there is the "are we measuring the finish or the whole game" question. I agree with JD that the whole game being exciting is better than just an exciting end, but I bet that (nearly) all football fans would have trouble with the concept of a really exciting game with a dud ending. The ending is the MOST IMPORTANT PART of the most exciting game and EI values that well. Ask any fan the most exciting game that had a let-down ending and they might look at you quizzically.
Tyson
Tyson, the takeaway from what you wrote is that in practice (as opposed to theory), EI does indeed identify the most exciting games, and therefore it's a good measure despite any conceptual imperfections.
Fair enough. But we're not talking categorically here. Nobody is claiming that the games with the highest EI's are boring, or those with the lowest EI's are exciting. It's like arguing that passer rating is a good measure of QB performance, even if it is rife with obvious conceptual flaws, because you have to be a really good QB to lead the league in it, so why bother coming up with other metrics like QBR?
I think the bottom line is that EI suffers from the fatal flaw of trying to turn something subjective (excitement) into something objective (a calculated number). When you try to turn an emotional concept into a mathematical concept, a lot will get lost in translation.
So maybe trying to come up with an alternate metric is doomed as well. Maybe my real gripe is the labeling...calling this useful statistic an excitement index is too ambitious and presumptuous. Let the stats be stats, and emotions be emotions.
Brian, have you ever plotted the slope of the WP graph? It's a similar method to EI, but might show the "excitement" as the game goes on and it would peak when the most influential plays occur.