If only...

Another post at the Slate/Deadspin rountable. 

Any of these five plays could easily have gone the other way. In total they represent 0.73 WP—nearly the entire difference between winning and losing. Win probability is obviously an abstract concept, but it helps put a concrete number on what we already intuitively understand. Numbers like this underscore how razor-thin the difference between winning and losing is, especially when the two opponents are evenly matched. This year's Super Bowl was like most other NFL games—it hinged on a handful of critical and unusual events. The conference championship games were no different: The Giants won thanks to two bungled punt returns by the 49ers, and the Patriots won in large part thanks to a missed 32-yard field goal attempt by the Ravens.

If you missed my first one from Sunday night, here's the link. It covered the end-game strategy decisions of allowing the final TD.

The smartest play of all would've been for Belichick to have allowed the touchdown even earlier. The Patriots certainly could have done so on the play prior to Bradshaw's touchdown run, when he was stopped for a one-yard gain, forcing New England to burn its second timeout. In fact, they probably should have allowed a touchdown as early as the two-minute warning. That’s the point at which the Win Probability of receiving a kickoff down by four or six points (0.23) exceeds the Win Probability of trying to stop the Giants from bleeding the clock dry (0.2). The Patriots would have had almost two minutes, two timeouts, and all four downs available to get a touchdown and steal the win.

  • Spread The Love
  • Digg This Post
  • Tweet This Post
  • Stumble This Post
  • Submit This Post To Delicious
  • Submit This Post To Reddit
  • Submit This Post To Mixx

23 Responses to “If only...”

  1. JG says:

    Brian - great work as usual; one question - as a commenter on Deadspin pointed out, you didn't include the Bradshaw fumble deep in the Giants' own end in the 4th quarter (after the Blackburn pick) that the Giants recovered. What are the numbers on that one?

  2. Brian Burke says:

    I guess I had that merged in my head with one of the other fumbles.

    That would have been an enormous swing, obviously. I'll crunch it and report back.

  3. Brian Burke says:

    Without the fumble, it would have been 3rd and 5 on their own 13, worth 0.37 WP. Had NE recovered, it would have been 0.19 WP--an 0.18 WPA swing.

  4. bmcosas says:

    clear bellichek should have let giants score at 2 minute warning - but should giants have fallen down 3 times and kicked fg? - figure there would have been about a minute left and pats w/o timeouts but only needing FG to win

  5. Jim Glass says:

    "Numbers like this underscore how razor-thin the difference between winning and losing is"

    Whenever a game is decided by 1, 2, 3 points, you can go back though it and find a score of plays that were or coulda been worth that much or more. Almost always the commentariat and fans remember only the last one, or maybe two. (If they remember even them, instead of concluding the decision was a decisive one that proved winner's superior character.)

    Through 14 games the media and fans calling in to sports radio here in NYC were full of "Coughlin has to go. He was kind of a good coach once, but he's old and out of touch, the game clearly has passed him by." Now the very same people and newspapers are going: "Tom just locked up his well deserved spot in the Hall of Fame."

    I'd like to see a list of all the plays going back to game 15 that coulda swung that.

  6. Anonymous says:

    I'm familiar with baseball WP, but not so much football. Although I am very familiar with football strategy & stuff. So I have a question... why wouldn't the best time to allow a TD have been once the Giants crossed into FG range? Say, the 35 yard line or maybe the 30?

    I ask 'cause I think that should've worked out to be the ideal time. Before that point, a score (FG or TD) was very preventable via fumble, interception, stopped on 3rd down, or a crazy penalty. Once they were in FG range though, they could wear down the clock and not worry much about getting closer 'cause it's a pretty solid chance the kicker would make the 3 pts needed.

    I realize there'd have been more time on the clock at the 2 minute mark than when they crossed the 35, but... with a 2pt lead & 2 minutes to go & the Giants not in scoring range... wouldn't it have been smarter for New England to keep defending to prevent a score than to allow a TD then than it would've been to allow a score at that point in time?

    Obviously, what you're saying - says it wouldn't have been. So, my question is... why?

  7. James says:

    Devon, the reason is because the kicker actually does NOT have a pretty solid chance of making the field goal. There is a critical range of field goal distances where closer field goals are easy and longer field goals are nearly impossible. This is the area generally misleadingly described as "field goal range" even though within it each yard marks a significant change in field goal success rate.

    To use your example, when the Giants reached the 35 that corresponds to a 52 yard field goal (once you include the 10 yard end zone and 7 yard snap). 52 yard field goals are made only 58% of the time. 5 yards closer makes a significant difference - 47 yd FGs are good 67% of the time. 5 more yards results in 76% made field goals.

    Also, once you consider how much easier it would be for the Patriots to respond with a field goal of their own rather than a touchdown, holding the Giants to a field goal is much more preferable. Once the 2 minute warning is reached and the Giants could sit on the ball until the game was essentially over is when the tables turn to letting them score.

  8. Jim Curren says:


    Thanks this is wonderful. Before I started reading your analysis today I began thinking about decision-tree analysis which lead me back to your site. Clearly, there were about 3 plays at 2 mins and in that would have been better to let the Giants score. I suspect that the probability of a successful FG from the 18 is over 90% - Can you give the WP for the Giants for each of the 3 or 4 plays had the Pats given up a TD in each instance. Additionally, had Bradshaw successfully squatted on the one, I have questions about what Coughlin would do on 3rd and goal from the one. After a Pats TO, Coughlin could try for the TD and score(leaving 53 secs), not score or take a knee, kicking a FG on 4th down (leaving a few seconds). Or, he could attempt the FG on 3rd down - ensuring that a botched snap/hold he would have 4th down but leaving over 50 secs. There are a lot of possible outcomes - can you give the Giants WP for each.
    Very interesting stuff - with so many intangibles. Although the math says one thing, I really think the Giants taking into the end zone against a cold Brady (after spectacular Brady in the 2nd and 3rd qtrs) was the correct way to go.

  9. Adam says:

    Great stuff as always, but it neglects that not every somewhat random-outcome huge moment swung the Giants way. What about that kind of questionable holding call on Boothe in the 2nd quarter that turned a successful 3rd and 1 conversion into 3rd and 11 that ultimately resulted in a punt? That cost the Giants .09 WPA, How about the non-call on DPI on Manningham on the Giants next to last possession? That was a whopping .23. Not saying either was a terrible call, but both could have clearly gone either way, certainly the 2nd quarter holding which was just purely random.

  10. Adam says:

    And 1 other thing, not much written about Belichek challenging the MAnningham catch. Its tough to gauge the WPA risk of blowing a timeout at that very moment, whereas the reward was huge, it was a .33 WPA play. So I'm no way saying it wasn't worth the risk of a challenge at the time. BUT...as the game played out, not having (and using) that extra timeout really did cost them 40 seconds they desparately needed. It would have added at least .10 WPA, and I say at least because that extra TO would have also changed the strategy of both teams down the stretch. That number would be more like .20 if the Giants had correctly knelt on the ball and attempted the short FG. And as it played out, that was potentially the last .20 WPA the Pats had.

    So let's realistically say the risk of that challenge was somewhere between .05 and .10. That implies Belichek only needed to win that challenge about 1 in 5 times for it to make sense. Sounds great....except he's standing right there and pretty much has to see a good catch, and obviously his guys in the booth couldn't have (or shouldn't have) seen anything amiss, since there was nothing amiss.

    Now there's no way to assess the probability of a replay overturning a call, and he really had no choice in the moment. So not sure my point. I guess that its sort of a higher beta version of blowing a TO to avoid a delay of game penalty. The cost of losing a TO can ultimately prove so enormous that the smarter coaches may underestimate the risk side of it.

  11. Anonymous says:

    Adam, that catch was about as close as possible to not being good, as he had control of the ball with both feet on the ground for like 2-3 frames (so under 0.1 seconds). Furthermore, the Giants clearly wern't positive either as they were rushing their next play.

    With the amount of time he had, I don't think he got confirmation from the booth, and his close proximity to the play surely resulted in him being unsure (I was certainly unsure until the slow - mo replays). So just like a FG attempt is a vast cloud of uncertainty, so was Belichek's decision to challenge. With the high +WPA to getting the call overturned, and the hard to quantify -WPA to losing the timeout, I think it was the correct decision to challenge.

    Also, you are basing your analysis based on the actual -WPA lost by losing the timeout (which I agree is about as big of a loss of WPA possible from a single timeout), and not the overall situation at the time of the challenge (if the NYG turned the ball over, or kicked a FG with 2:30 left, or whatever, the extra NE timeout has a WPA of essentially 0). So if we say that NE actually lost 0.1WPA with the events that actually unfolded, at the time of the decision, it has to be less than that, and probably more in the 0.03-0.05 range.

    Finally, your basic idea of how to analyze the decision to challenge is correct late in a game (so that we don't need to care about wasting a challenge and being out for a future +WPA challenge opportunity). Take the WPA of the play, multiply it by the probability of a reversal, and then compare it to the WPA loss from losing a TO and multiply it by the probability of losing the challenge. Doing this analysis, I think it is clear due to the high WPA of the play, and the at the time, reasonably high probability of success, I think the challenge was a no-brainer. The only improvement would be to have a better, more streamlined process for reviewing this sort of play quicker.

  12. Jim Glass says:

    When discussing these events everyone focuses on the last moments of the game, and WPA naturally emphasizes late plays -- but "luck events" can turn an entire game early on.

    I just watched the game again, and IMHO the big random chance luck event that changed its entire course was on that early Giants punt, the ball coming down point first and bouncing away from the end zone, instead of into it, leaving the Pats backed up on their own goal line.

    From there Brady gets called for grounding while throwing the ball 40 yards downfield (how often do you see *that*?) for a safety ... which gives the Giants not only 2 points but a possession on a short field that they turn into a TD.

    That's 9 of the Giants' 21 points coming from a ball landing point first and taking an odd bounce in the 1st quarter. There's no way for WPA to measure it, yet but for that odd bounce of the ball in the first quarter it is an entirely different game.

    WPA measures late-in-the-game big plays and scores as being more important/valuable than such plays early in the game -- in close games. But big plays early in the game are just as important as late in the game, as they can lead to the game being not close at the end (in which case WPA gives the late big play only minimal value, as it is not enough to make a difference in the outcome of the non-close game).

    This is a case where that early bounce of the ball was *huge*, because it led to Giants getting enough of a lead to still be close after the Pats ran off 17 straight points -- so the Giants' last big plays could make a difference.

  13. Anonymous says:

    That bounce of the ball was worth about 1 EPA for the Giants, they just happened to get more of a reward than that, but we can quantify the effect of said bounce.

  14. Anonymous says:

    it would have not been the smartest play at all. It would in fact have made it more likely for the offense to stop at the 6 inch line, and take knees. That statement assumes that the giants take the TD. Then they would have lost, and hoody would have looked like a moran.

  15. Adam says:

    I do realize that .10 or .20 number is too high, its just the max WPA that the TO cost, so i guess the issue to figure the risk side is what's the probability the lost TO faces a situation where its actually worth, say, .15 WPA. And then what's the probability the call gets reversed, which is obviously completely unknowable in a flash. Pretty sure neither team could assess it that fast, like you say, not too clear in fast motion, Giants simply doing the sensible move and not giving the Pats enough time to know for sure. In this particular case though, Belichek has as good a view as anyone of the play. And in fact he may even challenge if he thought Manningham might be in, thinking the risk/reward is still there. The refs aren't exactly perfect even with replay, maybe there's a bobble somewhere, maybe he stepped out somewhere earlier, et. al.

    Was thinking also that the result of the challenge and the chances the TO actually comes into play are hugely intertwined. With the Giants 2nd and 10 on their own 12 with 3:45 remaining, the clock's still not a gigantic factor, they have enough time to score, its more about the results of the plays. But they're certainly in poor position, so the Pats essentially control the clock and the TO has little to no value. With the ball at the 50 and 1st and 10, the Giants are now somewhat in control of the clock and on the verge of near complete control, and thus its way more likely the lost TO will matter. So the challenge itself compounds the Pats situation.

    Belichek really has no choice though, he can't risk not challenging. I guess all I'm saying is the risk side of challenging there is higher than meets the eye. A

  16. Mike M says:

    If only................... the Pats had lost in the conference championship like they should have after being hugely outplayed by the Ravens.

    If you haven't seen the two stats of teams outplaying their opponents like the Ravens outplayed the Pats, it's 110-1 and 115-1 in NFL playoff history.

    The Pats were very, very fortunate to be in this game to begin with.

    Ravens outplayed the Pats by..................
    ave per pass, +.9
    ave per rush, +.6
    TO +2
    total yds, +68
    rush yds, +20
    QB passer rating, +37.9 pts

    When a teams wins these stats by this amount they will win the game 98% of the time for a ave of 18 pts.

    The Giants outplayed the Pats by.......................
    ave per pass, +.4
    ave per pass, -.3, Pats actually won something
    TO +1
    total yds, +47
    rush yds, +31
    QB passer rating, +12.7 pts

    A team winning these stats will win the game 84% of the time by a ave of 12 pts.

    It would take a minor miracle for the Pats to win both those games back to back, THEY WERE FORTUNATE TO WIN ONE, I doubt any team in the history of SB's has been outplayed has much as the Pats were in 2 games and despite that go on and win the SB.

    As I suggested, the Pats chances to win the SB did not increase because they played a 14 pt dog in the opening round, they could not outplay a single good team in the postseason.

  17. Jim Glass says:

    That bounce of the ball was worth about 1 EPA for the Giants, they just happened to get more of a reward than that, but we can quantify the effect of said bounce.

    Don't confuse the probable effect estimated looking forward with the actual consequences seen looking backward.

  18. Anonymous says:

    "As I suggested, the Pats chances to win the SB did not increase because they played a 14 pt dog in the opening round..."
    Except it did. Can't you see that being more likely to win the first game helps them have a better shot at winning the whole thing? After all, if you don't win the first game, you have no shot.

  19. NFL Stadiums says:

    There was a lot of fluctuation in this game. What foes a dropped pass in the 4th quarter by Welker and Hernandez do to the graph?

  20. Anonymous says:

    I read a lot of cup half empty rather cup half full logic into this debate. Patently in any close game; not just football, to make the go-ahead score as late as possible to minimise of the opposition responding in kind but there has to be a time in open competition to play the game fairly and positively to the rules.

    Obviously there is a weakness in professional football that once the LofS is inside the Def 38 there is little incentive to run the risk to try for first downs and then run the clock out by taking a knee or keep a TO back for the game winning FG at 0:02

    No-one likes to see any team give up a soft score or worse still deliberately not score too soon.

    One approach may be to review the width of the posts but maintain the separation of the hash marks. Patently late in Q4 the 7-8yard LofS penalty for missing a FG outside the red zone is a non-factor, so perhaps an adjustment needs to be made to bring the probability of a FG down to around the parento 80% mark for FGA from the limit of the red zone.

    This would mean that a team getting inside the Def38 would still have to gain at least one more 1st Down to be assured a reasonably easy FG.

    Also not to be forgotten is the very occasional misuse of concession of a safety late in a game to ensure better field position late in a game, but I am not sure how to work round this one...


  21. James says:

    Field goal attempts from the 38 yardline are 55 yard FGs, which are made only 51% of the time. You need to be inside the 23 yardline just for a greater than 80% chance to convert a field goal.

    If being 60% more likely to make the field goal and win the game isn't a significant incentive to go for the first down and more, I don't know what is.

  22. Anonymous says:

    James...I think you need to look more closely at the stats for shorter kicks; for longer kicks the stats are going to be influenced by the fact that failing on anything but a kick late in the half guarantees excellent field position to the opposition.


  23. Anonymous says:

    The graph you made of the Super Bowl is so awesome. How do you make something like that? Anyway, the Giants were going to win either way no matter what, but its cool to see how they won. When they were at the lowest in the game and when they were at the highest.

Leave a Reply

Note: Only a member of this blog may post a comment.