New England coach Bill Belichick is taking a lot of heat for his decision to attempt a 4th down conversion late in the game against the Colts. Indianapolis came back to win in dramatic fashion. Was the decision a good one?
With 2:00 left and the Colts with only one timeout, a successful conversion wins the game for all practical purposes. A 4th and 2 conversion would be successful 60% of the time. Historically, in a situation with 2:00 left and needing a TD to either win or tie, teams get the TD 53% of the time from that field position. The total WP for the 4th down conversion attempt would be:
(0.60 * 1) + (0.40 * (1-0.53)) = 0.79 WP
A punt from the 28 typically nets 38 yards, starting the Colts at their own 34. Teams historically get the TD 30% of the time in that situation. So the punt gives the Pats about a 0.70 WP.
Statistically, the better decision would be to go for it, and by a good amount. However, these numbers are baselines for the league as a whole. You'd have to expect the Colts had a better than a 30% chance of scoring from their 34, and an accordingly higher chance to score from the Pats' 28. But any adjustment in their likelihood of scoring from either field position increases the advantage of going for it. You can play with the numbers any way you like, but it's pretty hard to come up with a realistic combination of numbers that make punting the better option. At best, you could make it a wash.
- Home
- team analysis
- Belichick's 4th Down Decision vs the Colts
Belichick's 4th Down Decision vs the Colts
By
Brian Burke
published on 11/16/2009
in
research,
team analysis
Subscribe to:
Post Comments (Atom)
Yes! I just reached a similar conclusion, with a closer margin, by looking up a bunch of estimated stats at various sites, but it looks like you have the more accurate numbers. Thanks for the confirmation.
Listening to Dungy say "he should have gone with the percentages" just aggrevated me. I knew he had no clue about the numbers, and was going with his gut.
I'll have to check out the rest of your site, it looks great. -Jim
Stupid.
Ever heard of the 3rd variable effect? Like Peyton Manning?
Dummy-The better you consider Peyton Manning, the better going for it is.
I love how Trent Dilfer's holding onto his argument that going for it is dumb.
So much backpedaling on the way.
I was looking up the numbers, and came across your site. Great stuff. I figured you'd post something tonight.
One other point: if you go for it, you can tell your team that you gave BOTH your offense AND your defense the chance to win the game. He showed he believed in the team, and was willing to take personal heat
I went to LSU and I'm a big Les Miles fan. Les got his team to believe in themselves, and one way was the 4th down.
Yeah, I felt the same thing about Dungy.. if he'd said, "I'd have kicked it" it would have been fine." But he said, "the stats say." You'd think with all the bucks & technology in the NFL, statisitics would be common.
Great site. -Brian
Jackson Sneed does not make good music
That was very interesting, maybe someone should send some of the ESPN folks a link to this. Cool website!
I am not sure most of the ESPN writers could understand something like this. I thought they were all sub-3rd grade reading level? Even the ones who supposedly "love advanced stats" (looking at you Eric Karabell, etc.) only ever mention them when they support their point, and ignore or actively flout them the rest of the time.
Remember, if sports were all defined by nothing but numbers, what would sports journalists have to pontificate on? Poetry in journalism works better with a story. Numbers are the truth but fiction is more entertaining than the truth (for most). I personally enjoy both, and shake my head but appreciate that those who refuse to see the math, also play poker at my table.
One quick question about the 4th and 2 being successful 60% of the time: Does that take into account the game situation? I'd imagine its harder to convert a 4th down when the defense knows that a 1st down is a certain loss, versus situations where the defense has to guard against the possibility of a big play.
In other words, in your average 4th down situation, a defense is primarily worried about giving up the 1st down, but also doesn't want the offense to completely beat their man and score a 70-yard TD. Here, the defense need not worry about that, since the Pats win regardless, which should allow them to play their men tighter and drive down the conversion percentage a bit.
I'd bet that's still not enough to make Bill's call bad, but worth considering ...
That last factor of the defense defending harder against the deep threat is likely counteracted completely by the fact that the Pats offense was matching up very well against the Colts' D and would be able to gain 2 yards more than an average team anyway.
League-average 2 point conversions are successful about 45% of the time, and I think that is a better baseline than league-average 4th and 2 conversions, considering that the Colts aren't going to seriously respect a deep route.
Even adjusted for the Pats offense, a 60% conversion rate seems a little high.
With that said, going for it is still correct.
As a Pats fan who happened to have a significant amount bet on the game on the moneyline (plus some more Pats+2.5 so I didn't lose in the end), it was painful indeed... but even as it all unrolled, deep down it felt correct to me. I had no idea what the stats were, but the chances of converting that play and effectively ending the game must be greater than the sum of all that can go wrong. So as gut-wrenching as it was to watch, it felt like the right thing. Thanks for confirming it. Perhaps I'll sleep better, but likely not by much.
Belichick may have made the right decision on the 4th-and-2 play, but he screwed it up later once the Colts got down to the 1-yard line by not allowing the Colts to score on first down.
In fact, the Colts should be thanking whichever Patriot defender stopped Addai on the 1-yard line. It allowed the Colts to run another minute off the clock, eliminating any chance the Patriots could drive back for a game-winning FG.
A lot of interesting stuff here but the game situation component needs to be addressed better. Considering the time left in the game and the field position, Belichick's decision was horrible by any measure. If we were talking midfield or on Colts' side, I'd be in agreement. This was a bad, bad call by a usually astute head coach. The coach's job is to put his team in the best position to win the game. In this game, Billy put his team in position to lose -- and that's exactly what ended up happening.
The game doesn't always go according to stats, if it did computers would be calling plays, not humans. Belichick's there to assess the situation and plan accordingly.
Fact of the matter was, New England's offense had been lackluster in the 2nd half while the Colts put together two 80 yard TD drives in the 4th. With the game in the balance as it were, why risk giving Manning a short field? Even if these stats are accurate would you really want to look at a 40% chance of handing Manning the game?
Belichick should have factored in the opposition, not the numbers, in his decision.
What's funny is that I immediately thought of this website when Belichick went for it. Belichick's decision was a very critical one in the history of 4th down conversion attempts, and the way it played out will only lead to more conservatism in the future, +WPA or not.
"A Punt from the 28 typically nets 38 yards, starting the Colts at their own 34. Teams historically get the TD 30% of the time in that situation."
This doesn't sound like it takes into account time taken to get the TD. Indy would have had less than 2 minutes left which can only make those chances go down.
As a Colts fan I wanted the Pats to punt. Basically, after the 3rd down stop I thought the Colts were right where they needed to be. Two minutes left and Peyton has the ball. When the Pats went for it I thought it was the worst thing for the Colts. I trust Manning going 70 yards more then the Colts D stopping two yards.
Good point, Brad. I'm also curious if that 60% chance of converting on 4th and 2 goes up with Tom Brady passing and/or Bill Belichick calling the play.
I'm also curious if the probablity of the Colts scoring after receiving a punt is higher because it's Peyton Manning at QB. How would it be different if JaMarcus Russell were at the helm?
In my opinion, it's not fair to assume the success of the conversion in the analysis. Instead, I reason this way: the decision to punt gives a 70% chance to win and the decision to go for it (not actually converting but DECIDING to go for it) gives a 60% chance of winning.
It would've been a 'safer' decision to punt, but not by much. Going for it was a LOT closer than most people realized. My guess is we'll be hearing a lot of criticism on ESPN from the 'experts'.
What's the probablity these stats make it on to Sportscenter?
"Belichick should have factored in the opposition"
OK, everyone has to stop saying this. Factoring in the opposition is a reason for GOING FOR IT. It's the only chance you have to keep the ball out of the oppositions hands.
The blog boys (and girls) have mixed opinions on Belichick's decision.
I tend to think he should have punted myself.
One of the funny blog sites I came across was this one because of the Norv Turner reference. Cracked me up because I hate Norv Turner and don't understand how he could land THREE head coaching positions.
http://mrsportsblog.wordpress.com/2009/11/15/belichick-becomes-a-buffoon/
I like the 2-point conversion suggestion (convenient that its 2 yards). Taking your word on the 45% stats and re-running the numbers.
(.45 * 1) + (.55 (1-.53)) = .7085
Thats about the same .70 for punting, so its basically a wash.
You're wrong. You don't take into account that indy had only 1 time out and a punt would have taken the game past the 2 min warning. Your statistics are averages which don't take into account many situational things. Playing prevent defenses with that ammount of time left and forcing throws into the middle make it almost impossible to score with that much time left.
"This doesn't sound like it takes into account time taken to get the TD. Indy would have had less than 2 minutes left which can only make those chances go down."
I'm pretty sure that his percentages are drawn from similar situations, including time remaining.
"Belichick should have factored in the opposition, not the numbers, in his decision."
I'm pretty sure he was. The other team had Manning at QB, not Russell.
"League-average 2 point conversions are successful about 45% of the time, and I think that is a better baseline than league-average 4th and 2 conversions, considering that the Colts aren't going to seriously respect a deep route."
But the 2-point conversion does actually have the back of the end zone as an extra defender. While a long gain on the 4th down conversion is available, such as throwing over the top after the Colts jump on a pump fake. Still, the general point is a good one, I suspect we should split the difference and consider it less than a typical 4th and 2 but greater than a 2 point conversion. I'm sure Brian will tell us what the break even conversion rate to justify going for it would be. If I'm adding correctly, it looks like it would be around 42% chance of converting.
Oh, and Anonymous, you need to go to bed. And stop contradicting yourself.
One thing to take into consideration why you can't just use the 2pt conversion is because you are on a compressed field with 2 pt conversions. Here the Pats had pretty much the whole field open to them plus in actuality they had an element of surprise going for them as well since most people (im guessing including the colts) were surprised they were going for it so they may not have been quite as set for the play.
Like Dave, I immediately thought of this site and though I was rooting for the Colts in the game overall, I was disappointed to immediately see a good Belichek decision on the fourth and two go down in infamy such that it may ripple across the NFL and prevent evidence-based practices from being implemented for the foreseeable future.
And I agree with Anonymous that the Pats should have let the Colts score a play earlier. The percentages on making a full goal line stand that close aren't good--better to go with Brady and Moss and work the sidelines.
By the way, according to the NFLN the Patriots have converted 78% of all 4th downs in recent years so going for it was probably made an even better decision by that.
The call being called stupid is more frustrating because the only reason its not being called "gutsy" and brilliant is it failed, and the only reason it failed was a bad spot by the referee and an inability to challenge the call
I love how Trent Dilfer's holding onto his argument that going for it is dumb.
Going for it is dumb. If the opposing quarterback is Trent Dilfer.
As other have noted, 60% is clearly far too high - 45% for the 2 pt conversion is much more reasonable. The fact that the end zone is only 10 yds deep doesn't change things much, because the Pats strategy will be to go for the highest percentage 2+ yd play that the Colts give them. Thus the Colts only have to make the weakest possible effort to protect against the 12+ yd play because even a very weak effort will make that lower percentage than a 2-3 yd play.
At 45% the odds become pretty much even for go vs punt (0.45 * 1) + (0.55 * (1-0.53)) = 0.71 WP.
Furthermore, it's definitely not a 1.00 probability that the Pats win with a 1st down. IIRC, Indy had 1 TO and the 2 minute warning left. If you go for all runs, you give the Colts back the ball with a high probability (50%?) and 60 seconds left. Suppose the Colts chances drop to 35% in that case...you're still knocking .45*.50*.35 = 0.08 off the WP, which dramatically changes things. If you go for a mix of runs and passes, the odds of giving back the ball go down (let's say 30%), but you likely give them back the ball with 90 seconds or more. Suppose the Colts chances jump to 45% in that case. Then you're knocking .45*.30*.45 = 0.06 off the WP.
Bottom line is that you should be knocking off at least another 0.05 off the WP of going for it. At which point the balance shifts to punting.
According to the NFLN the Patriots have converted 78% of all 4th downs in recent years so going for it was probably made an even better decision by that.
Most of those are probably 4th and inches scenarios, which this is not.
"Furthermore, it's definitely not a 1.00 probability that the Pats win with a 1st down. IIRC, Indy had 1 TO and the 2 minute warning left. If you go for all runs, you give the Colts back the ball with a high probability (50%?) and 60 seconds left."
No, the 4th down play ran the clock under two minutes (1:57). With one timeout, on three runs/kneeldowns that leaves the Colts with about 65-70 yards to go in about 20 seconds with no timeouts. Still not a 1.00 WP, but pretty darn close.
"At 45% the odds become pretty much even for go vs punt (0.45 * 1) + (0.55 * (1-0.53)) = 0.71 WP."
I think, when it comes down to it, the odds are pretty darn close to 50-50 in a vaccuum. When you add in the fact that it's Brady going for it on 4th and 2 and Manning going against a gassed defense after a punt, I think it makes BB's decision a good one.
Now, as for tackling Addai on the 2 yard line...
I'm very interested to see the aftermath of this play. Some commenters here have suggested it will cause a regression to more conservative play-calling, and that may be right, but on the other hand, has there ever been such a high-profile decision, in such a crucial moment, that so blatantly defied conventional wisdom? Belichick being so highly respected, once the "what the hell was he thinking!" crowd has their say, people may actually start thinking about what he may have been thinking.
In his post-game interview, Peyton Manning (sort of, vaguely) agreed with the decision. Presumably, somebody on Around the Horn will defend Belichick, because sports chatter shows require conflict. I'm sure there will be others.
My point is, if it was Jim Zorn, he'd just be fired and the whole thing would be quickly forgotten. But since it's Belichick, everyone will be talking, and a few might even change their minds. Of course, it's too bad he undermined his genius reputation by mismanaging his timeouts so badly.
the football would have punted itself
from ESPN:
"ESPN Stats & Information dug through the archives and found that entering Sunday's game, the Patriots had converted a fourth-and-2 or shorter 33 out of 52 times (63.5 percent) since 2001, Brady's first year as a starter. "
Re: the comments suggesting we should use the 2 pt conversion percentage rather than the 4th and 2 percentage - if the Colts aren't going to respect a deep route, then the patriots should be throwing a deep ball.
I've got no problem with the decision. As anon says above me, ESPN have dug out that Brady converts 63.5% of 4th and 2, well within the confidence interval to assume the conversion had a league average probability of 60%.
I do hope this doesn't reverse the trend of teams going for 4th down. A coach saw that his team needed two yards to win the game and went for it. If he made it people would be saying what a great decision it was. Hindsight is a wonderful thing.
Things I’m already sick of hearing:
1. You have to play the percentages and punt the ball
As discussed above, what percentages? Seriously, are these numbers out there? Even if the analysis is faulty, which I have to assume it would be (though I try to remain open-minded), I’d like to see a link.
2. Belichick apparently didn’t have confidence in his defense
On the contrary, it requires a lot of confidence in your defense to risk giving up the ball at your own 30 instead of your opponent’s 30. Both of those statements are oversimplifications, but to me they seem contradictory and equally valid, so doesn’t the whole argument cancel out?
3. He’ll regret this decision because it cost his team a chance to win
This is (more or less) what Trent Dilfer said, and several problems come to mind. First, it indicates a clear misunderstanding of how decision-making works—obviously he regrets that they didn’t get the first down, but (hopefully) he doesn’t regret the decision itself. Second, after the fourth down attempt, they still had a chance to win. Third, just saying it “cost his team a chance to win” establishes nothing. Every play costs your team a chance to win if it doesn’t work.
No matter what others say, this is just the best game of the season for me. Too much excitement and unpredictable moves. Loved it. Thanks for sharing. Anyways, it's always cool to be a great sports fan because one good company is giving away free coupons.
Maybe if brady hadn't burned a timeout atthe beginning of the crive. Maybe if they were so deep in their own territory. Maybe if they hadn't burned their second timeout. Maybe if Indy did not already have a ton of momentum. Maybe against any except Manning. Maybe. Maybe they could have pulled it off. There's also a decent percentage that monkey will, in fat, fly out of my butt. There's a lot of maybes, but the one fact that remains is, the Pats blew a 17 point lead snatching defeat from the jaws of victory.
Well, he went for it and it didn't work. Then his team lost a game it was winning by six points with two minutes left.
We don't need any more proof then that to know it was a dumb decision, no matter what any stat geeks claim.
This isn't calculus calls. This is the NFL.
This site is an amazing resource. Thanks so much for doing out the math - I had a feeling that going for it at such a short distance was actually the statistically better option. I linked to this page and cited your stats in an article I wrote (link above). Let me know what you guys think.
"Well, he went for it and it didn't work. Then his team lost a game it was winning by six points with two minutes left.
We don't need any more proof then that to know it was a dumb decision, no matter what any stat geeks claim."
Oy Vey!!!
Its scary that people are out there that are this reactionary.
The really scary thing is that I think this guy would have been posting "should have gone for it" if the punt was run back for a touchdown.
It was totally the right move.
"League-average 2 point conversions are successful about 45% of the time, and I think that is a better baseline than league-average 4th and 2 conversions, considering that the Colts aren't going to seriously respect a deep route."
In today's Cincinnati game, the Bengals lined up for an extra point. The snap got bobbled so there wasn't going to be a kick. The holder got up and tried to make a desperation throw. It didn't work.
That attempt was for all intents and purposes a failed 1 point conversion. The NFL disagrees. Nobody actually kicked a ball. The holder tried to throw the ball. So it goes into the record books as a failed 2 point try. Don't believe me. Look up the play-by-play on the Bengals game.
This is sort of bullshit. Plays like this are the reason why 1 point conversions are 99% and 2 point conversions are 44%. If you treated failed snap/desperation twos as missed extra points and not 2 point attempts, you would see that the actual 2 point conversion rate is well over 50%.
Simply, you do not get statistics.
You write, "A 4th and 2 conversion would be successful 60% of the time."
Yet, your claim is false.
You would be right if you said:
In all recorded past, for those teams that attempt converting on a 4th and 2 have been successful 60% of the time.
You have no way of knowing what the true odds are of being successful in any future in an open system like football.
No one can predict what the percentage of being successful "would be".
Ketch,
Way to be pedantic and useless.
"A punt from the 28 typically nets 38 yards, starting the Colts at their own 34. Teams historically get the TD 30% of the time in that situation. So the punt gives the Pats about a 0.70 WP"
I thank the poster who emphasized this point. An accurate assessment of Bellicheck's decision must factor in time and score. Is it accurate that teams starting drives from their own 34 yard line with a shade under 2 minutes left in the game and needing a touchdown to win succeed 30$ of the time? That simply doesn't compute; if the previous were true Bellicheck's decision wouldn't on its face seem so bizarre. If the 30 percent figure factored in time and score, however, I will stand corrected. Can anyone clarify this?
Your stats are fine. No issue with them. However, I think Bellichek saw that Indy scored twice in the fourth quarter on long drives and that his Defense was not performing. And rather than give the ball back to Indy to let them win he would rather trust his offense to get the two yards. Wouldn't you rather control your fate by performing the one play (which usually is simple to do and you've been doing successfully all game) rather than have to perform very many plays (defending) which you've proven to not be able to do (especially over the last 13 minutes). I would have gone for it too. I just watched Phillip Rivers slice up the Giants last week. The come from behind win happens all the time by letting the other team try win it away from you. So it's not like the conservative play guarantees a win. I'm glad the Pats lost, but I don't blame Bellichek for the loss. BTW, this was the PERFECT bookend game to cap off the 2003 Indy vs. Pats game. Thanks NFL Network for reairing it for us a few days back.
Another factor that yet hasn't been mentioned is location. It's unlikely the 4th down conversion rate of road teams is as high as the league wide average of 60%. Based on the above win probability calculator if road teams convert 4th and 2 even at 55% Bellicheck's "gamble" isn't worth the risk. More still, is the postulated 30% figure used to calculate the win probability of punting time and situationally specific? It doesn't on its face seem that teams driving in that specific circumstance, with less than 2 minutes left and needing a touchdown to win succeed that often.
Is anybody even going to mention that b#llsh!t spot on the 4th down conversion?? How do you catch it ON the 30 yard line (after a touchback) and NOT have a first down???
What I don't understand is why the Pats didn't sell out and play very aggressive defense. Which would be better for the Pats, (a) up by 6 with Manning with the ball like that, or (b) down by 1 with Brady on the field?
It definitely was a time to sell out with blitzing and jumping routes, and "missing" tackles. Whoever called only a 4 man rush on the pass to Wayne (the first one, not the TD), and whoever tackled Addai at the 1 (both times) deserve game balls from the Colts.
my understanding of his win expectancy numbers is that they cover ALL game situations (as in, 60% of the time, teams succeed at 4th down and 2). but these are averages that don't consider game situation (location, time left, score, etc) correct?
i think baseball win probability covers the exact situation that exists - probably because it is a much simpler game to break down -- there are only 3 positions a runner can be in, and a finite number of 'pitch count' situations. bottom of the 7th, 2 on, 1 out? there have been a significant number of game situations from which one can make a conclusion about the likely outcome, after considering years worth of data.
here, the author seems to be using a general, catchall average of win probability but NOT considering the specific game situation to predict an outcome. going for it on 4th and 2 in the 1st quarter, and winning the game, is completely different than going for it on 4th and 2 in the 4th quarter w/2 minutes left. i think in theory it is a good idea to try and break it down like this, but i don't think it is as simple as throwing out a win probability number out there based on what i understand of the original post.
i could be wrong, but just something that i was thinking about.
Just to clear up any confusion, the TD rates I cited for the Colts account for the time/score situation. I'll have a full post on the 2-minute drill soon.
To me the biggest surprise are the 53% and 30% numbers. Are teams really that likely to score a TD from their own 30? The normal scoring probability graphs on this site don't say that (they say more like 20%). So I'd really like to see the source of these particular figures.
"If you treated failed snap/desperation twos as missed extra points and not 2 point attempts, you would see that the actual 2 point conversion rate is well over 50%."
Jeff Clarke: I have done research on 2-point conversions, removing desperation attempts from failed extra points. There are usually one or two of these per season in the NFL. Since 1998, the correct 2-point conversion success rate is 47%. Since 2004 the success rate is 51%. Last season it was 45%. There are about 70 attempts per season and the success rate fluctuates quite a bit, so it's hard to say for sure what the "true" conversion rate is.
"Baseline numbers for the league as a whole" as Brian puts it have nothing whatsoever to do with the Belichicks decision, Dungy or Dilfer's comments, or most anybody else's comments. Neither team is exactly "league average" and one could assume that applies to their defenses as well. The Colts had 2 TD's 2 Int's and one punt in the 2nd half prior to the 4th down play. So one could say "the numbers" say they score a TD last night 2 out of 5 times and that's starting at the 19 which was their average for the 2nd half. Had the 2nd half played more like the first (2 Colt TD's 6 punts) I imagine Belichick woulda punted. Basing decisions on stats is fun, but not not applicable to many game time decisions.
Belichick is an offensive coach and Dungy is a defensive coach, one goes for it another punts. I refuse to say one is right and another is wrong based on "baseline numbers for the league as a whole", that is absolute nonsense.
The offense may have a slight advantage in today’s NFL, but teams can still win with defense. Dungy probably got his numbers from Brian where he says the team that loses the flip in OT gets the ball on offense 70% of the time. Granted this was not OT, the Colts woulda had 2 whole minutes and needed a TD. You can't pull a MJD when you need a TD, so Indy may have left time on the clock. 70% of the time the defense stops an offense in OT which only needs a FG. Yea, I know, probably only proves offenses are too conservative in OT, but the "book/stats" say punt for that case. See my point, baseline numbers are useless to NFL coach’s in unique situations.
KenYonLV
Indy had two fourth quarter TDs prior to the critical call. They had moved 79 yards on both drives. The first took 2:05. The second took 1:49. Neither drive used a timeout.
Going for it was the right call.
I haven't read all the comments yet, so forgive me if this has been said. By my poor man's math, the the Pats would need to convert that 4th down the same % of the time they would have won if they punted. For instance, if they make it they win, if they lose they don't (peyton factor...), so:
1 * x + 0 * (1-x) = x
x = x
If they had a 50% chance to win if they punted, they needed a 50% conversion rate to make it worthwhile.
Also, what would the Pats W% have been if they let Addai score with about 1:10 left?
I always love these posts and I ran to this site as soon as I heard all of the criticism of Belichick. (I guess I'm a pretty slow runner given that mine is the 55th comment).
A few things:
1) I once hear (or read) Ross Tucker say that Belichick is a data junkie. If there's a dataset available, he has people run model after model on it. I wouldn't be surprised to hear he reads this site.
2) Belichick has the luxury of job security. I feel very strongly that more coaches would be more aggressive on 4th down if they weren't scared of the fan/owner/GM backlash in case of failure )or, een success sometimes)
3) Not to jump to the defense of the "world leader", but I have seen Brian/advancednflstats.com referenced a handful of times there. Also, I'm pretty sure Erik Kuselias is a member of MENSA (although I just read back, and the joke was about Eric Karabell).
Great work Brian....especially on such short notice.
But I think some of the math in this situation is very skewed as you calculate it, and not in favor of going for it. If the Colts do not stop NE, Colts lose. Thus, the Colts can run an all-out blitz, goal-line defense there, with press coverage on the recievers, and that all-out defense decreases the odds of NE making the first down. Yes, it increases the odds of the NE receiver breaking a tackle and going the distance for a TD, but for the Colts, that is not a real risk, because ANY successful play by NE loses Indy the game. Thus, I think the odds of NE converting the 4th and 2 are smaller than normal.
For example, if that play happens at the 50 yard line during the 3rd quarter, Indy has to defend the whole field; they can't sell out with a blitz and press coverage, as they did last night. Blitzing in press coverage makes the damages to a failed defensive play much more damaging in most situations, so most defenses don't do that; but here, the Colts have nothing to lose -- a 70-yard catch and run is no worse for them than a 2-yard completion; either one is fatal. Hence, the Colts get to significantly lower the chance of NE getting the 2 yards by pressuring the QB and receivers.
In addition, the odds of Indy scoring from the 30 are much higher than you present, I would suggest, because with one timeout, 2 minutes, and only 30 yards to cover (actually, 29 1/2), every play in the book is available. The Colts can run or pass. They can draw, or go deep over the middle, etc. The fact that they have so little ground to cover makes the clock almost a non-factor, and you saw the Colts run the draw to good effect (12 yards or so). Even on the last play, when Wayne caught the TD, NE had to defend the run, because the Colts had a TO. They could have run the ball, been stopped, called a TO, and had at least one more try into the end zone via pass.
Is your conclusion slightly skewed? I mean, the Patriots' likelihood of scoring wasn't a factor in the decision to go for it. They didn't need to score. The Colts absolutely HAD to score. Since the objective was to run the clock out and not score, I look at the actual probabilities as more along the lines of 60% success rate of holding on to the win versus 70% if you were to punt it.
No has yet to point out that the Colts previous two TDs were executed in 2:04 and 1:49 and were 79 yards each. The expectation that the Pats D would step up for a 66 yard, time running out, hurry up O, with PEYTON MANNING AT THE HELM! is a
s t r e t c h.
Sorry DZ, just saw your post.
This is about the only credible argument against going for it:
"the Colts have nothing to lose -- a 70-yard catch and run is no worse for them than a 2-yard completion; either one is fatal."
WSJevon: Actually a few of us made that same point already. You ain't the first.
At the very least, this was a defensible decision, which is not what we're hearing from the dumbasses out there.
Another minor point: I have to assume that 4th down conversions are higher off of a timeout than not, because they can mull over plays, their QB has more time to analyze the defense and maybe audible, etc.
Given how easily Peyton moved the ball on the Pats in the following drive, I'm not sure why anyone thinks the outcome would have been different if NE had punted. He got 30 yards in maybe the first 30 seconds they had the ball. That's basically the distance of the punt right there.
Add in the fact that the refs seemed to be favoring the Colts at this point in the game (phantom PI penalties and unsportsmanlike conduct penalties) and I think the percentage likelihood of Peyton making that TD in the final 2 minutes with a TO had to be over 50%.
I'm glad I came over here today. I expected it to me a much bigger difference in IND taking over from the NE 28 as opposed to their own 38. I love the analysis on this site.
I do agree with the other commentors that NE WP is someone short of the full 1.00 if they convert the 4th and 2 there. You would probably have to mitigate that with the probabilty of NE having to punt on the next series (if they don't making one more first down), or the chance IND would get the ball on their own 30ish with about :40 left. I imagine it's still close.
And yes, it would be nice if there could be "team specific" stats, but I'm afraid there will never be a sample size large enough to be realistic. Is it fair to apply Patriots results from this season to two seasons ago? What about the Vikings this year to last? Or the Bengals? Or the Saints? At most you could probably realistically go back 16 games, but I think the "league baseline" this site offers over the past 9 years where most NFL teams have gone way up and down is more useful than doing team specific WP's with a small sample size. Just bear in mind you can take some guesses as to how your specific teams' (and the opposition) may effect the curve.
I came on here this morning thinking it was a blowout that Belichek made a mistake, but I think this analysis, even if you believe it's flawed, proves the decision is closer than most of the sportswriters think...
Joe, the one problem that I have with your analysis is that it would justify the same decision if the Patriots were on their own nine yard line facing a fourth and two.
I just heard Dale on WEEI talking about this article. He said: "You can give me all the statistics you want. I don't care about any of that. Belichick screwed up."
Yes folks, you can actually have pudding for a brain, and be a sports analyst.
It was clearly the right decision. The only noteworthy thing about the call was that someone actually had the guts to make it.
Genius. I love it.
I sided the 'Chekian way too but your numbers are amazing. 4SportBoston.com
Forget the baselines stats. If you want to know the real numbers, what is the Pat's success rate on 4th and short versus the Colt defense's success rate on 4th and short? The base stats say punters typically net 38 yards when punting from the 28, but according the Box Score, Hanson averaged 44 yards a punt last night with no returns. That would have put the Colt's on their own 28. What are the percetanges of the Colts scoring a TD from their own 28 versus the Pats' 28?
The base stats need to be adjusted to the situation before one can make a conclusion about whether the odds favored Belichick or not.
:O Mr. Burke is getting some serious traffic...
Fantastic job...and I just got done saying how stupid I thought the call was!
I think you are waaaay wrong. First of all the odds of converting 4th and 2 are not 60%. And some of that is motivation and some could be sample size. Often times teams that go for it need to make it to win. Often times, it is on the other end of the field so they have established momentum. Anyway the odds of converting a 3rd and 2 over the last 3 years is only 52%. The colts are a good defensive team and the Pats good counteract. I think the Colts D had a psychological edge given what had happened in the 4th quarter. Anyway, the real expectation of 4th down was (IMO):
(0.52 * .9) + (0.48 * (1-0.65)) = 0.636 WP
And maybe the .65 is too low. The psychology of not converting is a huge momentum swing. Everyone KNEW at that point that Peyton would score from 29 yards out. They even had the luxury of running the ball a couple times to run clock so Brady would have no time to answer.
As for this:
A punt from the 28 typically nets 38 yards, starting the Colts at their own 34. Teams historically get the TD 30% of the time in that situation. So the punt gives the Pats about a 0.70 WP. I think this is too low. Mannings Third Down Efficiency for the game was 5/12 - 41%. Going 70 yards is a lot harder than going 29. And with one time out, the playbook is more limited which is an advantage to the defense. Belicheck could be faulted not only for the 4th down call, but why not run the ball on 3rd and 2 (and then he burned a TO which meant he could not challenge). Anyway, I would say Manning has about 25% chance of going 70 yards. So WP of punting was 75%, WP of going for it 63% or less.
The percentages say punt.
Data on 3rd and 2: www.nfl.com/news/story?id=09000d5d80ae2bf7&template=with-video&confirm=true
Your analysis assumes the success of the punt. What is the probability of a muffed punt (decreasing the Patriots' win%) vs. the odds of a fumble on the return (increasing the Patriots' win%)?
When you make the 53% success claim from the NE 28, what is the sample size (roughly)?
The only complaint i have with the original analysis is that it only accounts for the probability that Indy scores a touchdown (53%) if NE doesn't convert the 4th down, it doesn't account for the probabability that NE scores a subsequent last second field goal after the touchdown. I understand that you can't measure this WP without knowing haw much time would be remaining. Regardless, any additional WP helps the case for BB's call.
Can somebody run the numbers to tell me whether the Pats made the right decision when they tackled Addai at the 1-yard line with ~1 minute on the clock? My guess is that the tackle was the right decision, but only by a small WP.
Considering the Patriots Offense was the #1 DVOA offense through week 9, going for it was DEFINETELY the right decision.
Brian's analysis is the baseline NFL average. Both Tom Brady and Peyton manning are anything but.
That skews it towards going for it on 4th down even more.
And another point: the Pats punter was averaging 44 yards a punt with no return and a 55 yard long. Back the Colts up another 10 yards and how does that change the formula?
Going for it was dumb IMO, but not nearly as dumb as the wasting of timeouts and play calling (pass attempts on all four downs) done on that possession. THAT is what Belichick (and Brady, who took the timeout to start the series) should be killed for, not so much for the gutsy call.
Another point in favor of going for it. Imagine two scenarios where the Colts score a TD. One after a punt, likely no time left, Colts win. Two after failed 2 pt, time left for the Pats to come back and win w/ a field goal. In fact that may have been a high pct. chance of a Pats win as many have pointed out if they just let Addai score. I'm guessing at least 30-40% chance Pats score a FG w/ under 2:00 and no timeouts after the kickoff.
People say that Brady and Manning are better than average and that tilts the stats to going for it. I disagree. Why? Because the Colts and NE defenses are also better than average. What is the 3rd/4th down conversion rate versus the Colts D or how often do teams drive 70 or 80 yards against the Pats D for a game winning score?
You can't just have it one way and say the offenses are above average and not take into account who they are facing.
The Colts D is ranked #1 in points allowed per game at 15.8 p/g. The Patriots D is ranked #3 at 16.7 p/g. Put that in your pipe and smoke it. Going 70 or 80 yards is a lot harder than going 29 yards. Peyton would have to air it out and take some risk going 70 or 80 yards. Might well have thrown another INT like earlier in the 4th.
Belichek did not play the percentages. He gambled and he lost. He should have more confidence in his defense. His play calling and time outs and going for it on 4th were all bone headed. Pats choked away a 17 point 4th quarter lead.
PW going for it on 4th and 2 = .636
Prob Win if punting = .75
Belichik blunder.
In reference to one of the posts above, I don't think it mattered if a Patriots defender stopped Addai. He could have just gone down with it on his own like Jones-Drew did against the Jets yesterday.
He could have gone down, but he didn't. He was trying to score.
The way he was running, there's no way Addai could have stopped himself short of the goalline.
I know those stats and I am here to say they are Bull. The reason being they do not take into account in high risk critical situations. They only look at the aggregate of all the short yardage situations on 4th down. I bet if someone actually complied all under 2 minutes short yardage 4th downs in a second quarter but probaly more specific ,a fourth quarter, the percentages will come out closer to 50 / 50 or less. So enough of Belichik and his vaunted genius with stats use. Geniuses also are arrogant and place to much faith in faulty facts.
Think about it this way, if you were the head coach of the Colts, would you have rather:
a) Pats punted the ball in that situation, giving the ball back to you with the best clutch QB in the game today with ~2 min left to play?
or
b) Pats gone for it, with 2 yards needed to win. But with a chance to get the ball back at the 28-30 yard line if they were unsuccessful.
The opposite choice of what you just selected would have been the right call in the situation.
To the poster who made the point about the indy and pats defense:
Per DVOA, both defenses were below their respective offensive DVOA.
Also, new england started with 2 downlinemen to start the game, effectively playing a base 2-5-4 defense. They had NO depth.
ALSO, the defense was torched in the 4th quarter already and was gassed.
The decision to go for it was right.
If you argue that "time and distance" reduce the chance of Manning scoring after a NE punt, clearly you were asleep for the first 13 minutes of 4th quarter. Indy's 3 previous 4th quarter possessions:
Start on Ind 21, TD in 5 plays, 2:04 elapsed (No timeouts used on drive)
Start on Ind 18, INT on first play
Start on Ind 29, TD in 7 plays, 1:49 elapsed (No timeouts used on drive)
Manning had gone 70+ yards in 2 minutes without using a time out twice in the last 15 minutes.
Some posters have argued that converting the 4th down doesn't guarantee a W for NE. That's true, but failing to convert doesn't guarantee a W for Ind either. There's a chance that Indy scores too fast, and the Pats get the ball back, down 1, with some amount of time.
Going for it on 4th and 2 may, statistically, be better than punting it in that situation, but running the ball back for a Safety would be a better decision than punting, since the Colts would still need a touchdown to win it, and with a a possible ~80 yards to drive and less than 2 minutes on the clock, I think a Safety would have been an even safer decision than going for it on 4th and 2.
Think about it this way, if you were the head coach of the Colts, would you have rather:
"a) Pats punted the ball in that situation, giving the ball back to you with the best clutch QB in the game today with ~2 min left to play?
or
b) Pats gone for it, with 2 yards needed to win. But with a chance to get the ball back at the 28-30 yard line if they were unsuccessful."
B doesn't look so favorable if you factor in the key thing you left out.
c) Pats go for it, convert the two yards and win 98% of the time.
Great post, I completely agree. Especially with the additional P. Manning variable!
I can appreciate the argument for supporting Belichick's call, and in a way I admire his going for the jugular. But I don't care what these calculations say, you just can't risk giving the ball to the other team at your 29 yard line with the game on the line. Especially when it's Peyton Manning who will be leading the opponent's drive.
The Patriots lost, by the way, because their defense stunk in the fourth quarter. Also, they didn't come up with a stop in the second quarter with the Pats up 24-7 when it would have really been timley. They had two costly pass interference penalties, and of course Maroney's fumble came at the worst possible time, which is something he seems to specialize in. Robert Mathis made some huge plays late in the game. Belichick's call is what everyone is focusing on, but the fact is the Colts just played better than the Pats.
Adam Schefter just tweeted this
"Since 2000, including postseason, Bill Belichick had gone for it 16 times on fourth down against the Colts -- and converted 12 (75 percent)."
I think your analysis is right on, and I think it was a good decision by Belichick even if ultimately incorrect.
I also agree with the comments about Dungy. I think it is a great man and a brilliant football coach, but was dissappointed with his comments about 'going withthe numbers'.
I humbly submit that if, as a coach, I were given the option to start every drive on my opponents 29 yard line, and all I had to do was stop a 4th and 2 attempt, I would take that deal and win by the most amazingly lopsided margins. Too many factors are being left out of your equations for them to be taken seriously.
Schefter's tweet is useless. So many things have changed over that time that it doesn't tell you anything.
Digression43... would still take that deal if you lost the game everytime you allowed your opponent to convert the 4th down?
Wait a sec Anon 2 posts up;
Even though there are no gaurantees you point to the fact that Indy scored 2 tds in the previous 3 drives as an indicator of likely scoring again even if they punted, yet in the same post say that if Indy stoppped NE after getting a first down, it wouldn't have been likely for them to get the TD? Exceptionalism is so blatant here.
I don't have a problem with them going for it, but I do have a problem with the play they called plus the fact that they burned their last timeout to call it, eliminating any chance of challenging the spot of the ball.
That being said, Belichick didn't lose the game for them, the combination of Belichick's call plus Brady's interception in the end zone plus Maroney's fumble in the end zone lost the game for them.
There's another variable that comes into play for
(.60*1) + (0.40*(1-.53)) = .79 WP and it nearly happened.
If Addai doesn't get tacked at the 1 on the second play, and instead gets the TD with over a minute on the clock, now you've got to add in the chances of Brady getting a FG or TD with all of that time left. You have to add in the win percentage from the Colts scoring too quickly and giving the ball back to the Pats.
You don't lose the game EVERYtime you allow your opponent to convert, just most of the time. ;)
At least with the 4th down call, there can be a reasonable debate as to whether it was optimal to punt or go for it.
What was truly idiotic and inexcusable was going to a prevent defense with 4 or 5 minutes still left in the game and only a 13 point lead. One way or another, dumb coaching lost this game for the Pats.
Going for it on 4th down didn't cost the Pats the game. They likely would have lost even if they had punted. The reason they lost the game was that they couldn't stop the Colts in the 4th quarter. The bottom line here (which none of the so-called experts on TV will figure out) is that there is too much punting in the NFL.
NE was going to lose if Manning got the ball with over 2 minutes and time outs left - no matter where they started. Why not try to win it by getting two yards. Not as bad a decision as people are making it out to be.
You link to your 4th down study (which is fantastic by the way!!!) where it says a 4th and 2 ouside the 20 is converted 60% of the time. I'm guessing that's outside the opponents 20....is that the case?
If so, what's the data tell you on 4th and 2 inside your own 30? Is it different?
Why was the primary reciever from an empty-backfield set running a two-yard route? It takes time for defenders to be drawn downfield. There was an acre-and-a-half of field in which to run recievers.
The decision seems reasonable, given how poorly the Pats had been defending in the 4th quarter, but the play call was weak.
I don't know if the stats take into consideration the magnitude of the decision if it backfires and where the game is being played, in this case a domed stadium with a huge crowd. Also the assumption is the kick only nets 38 yards. Taking all of this into account, it would have been close to a 50/50% decision, so Belichick is getting a lot of undeserved criticism because he defied conventional wisdom. He should have been given more credit so going against the grain. What is the biggest gripe against coaches is that they are too conservative in their decision making especially on fourth down plays. It sure made for some entertaining football and some great second guessing on Monday morning.
Use whatever statistics you want. These numbers are simply numbers. League averages mean ABSOLUTELY nothing in a game like this. So many things have to happen in order for a team to drive 70 yards down the field. Why pin your hopes on one play? Bottom line is....it was the wrong call. Disagree? Look at history, not math.
given the information i've gleaned from this discussion, at the very least you can say that the decision to go for it wasn't boneheaded
One likely flaw in the original equation is the assumption that the difference in field of position of some 38 yds is only equivalent to a 23% difference in scoring the TD. This seems way off. Think about it...
THe problem is trying to figure out how to relate field position to probability of scoring a TD. I dont think we have this specific data but we do have a couple of references, one from ths website:
http://www.advancednflstats.com/2009/09/4th-down-study-part-1.html
Where he comes up with expected pt. versus field position. If you look at the slope of this graph it seems to be about 20 yards per pt. in your half the field with steeper slope as you get into enemy territory (say from mid field to the 30), maybe 1.5 pts. per 20 yds in that sweet spot just into enemy territory.
You cant necessarily equate expected pts with TD probability, however, given the situation and the colts having to go for it, it is probably not far off to assume about 15% greater TD probability for every 20 yds most of the time, and probably more like 25% per 20 yards in that sweet spot just inside enemy territory.
THink about it, just in theoretical terms. the field is 100 yds long, is 15% per 20 yards a bad assumption (given the only outcome is TD or game over).
Another link (again not having to do with exact TD probability study but similar study) at
http://www.dynamist.com/articles-speeches/nyt/football.html
comes up with 18 yards = 1 pt. Again this is not the same as saying that 18 yds is 15% more likely of a TD (because of FG and stuff like ensuing kickoff) but it might be a reasonable starting pt for taking a guess in this situation.
Seeing as how Manning and the colts have to go for the TD, the FG is out of the decision making process and so this might be a reasonable assumption. (18 yds = about 15% more likely TD). Again note: that with time running out this graph should only get steeper and thus helps my argument.
Okay...Assuming say a 40 yard differential in field position on a punt (assuming no penalties, fumbles, fumbled snaps etc.) and assuming the above articles are reasonable estimates (to me it seems hard to argue/conceived of other than a 15% TD probability per 20 yards). It is more like a 40% different probability of Manning scoring from his own 30 as opposed to the Patriot 30.
Doesn’t this seem more reasonable? Also given the time remaining, this really seems to bolster my argument as with less than 2 minutes it is a lot more comfortable in enemy territory than your own. Not a quantitative measure but again isn’t it reasonable to suggest
Colts at own 30…25% chance of winning
Colts at New England 30….65% chance of winning.
Seems far more realistic, having read the above articles there seems no other way to estimate TD as a function of field position.
Also 60% chance of NEng making the first down seems incredibly generous. Colts have to sell out there. 50% is more reasonable. N Eng is only 45% on the year at making 4th downs (albeit 75%, 59% last couple years)
So the equation becomes 0.5 + 0.5 times .35 = 0.675
As opposed to prolly 0.75 if they punt; maybe .70 if we assume manning is super manning.
This seems more natural to me, a wrong decision although not very bad.
QUOTE: "Use whatever statistics you want. These numbers are simply numbers. League averages mean ABSOLUTELY nothing in a game like this. .. Bottom line is....it was the wrong call. Disagree? Look at history, not math."
THe problem with this argument is that history suffers from the exact same thing you are complaining about with statistics. They are both static, and dont account for ever changing situations; they are based on large data sets that maybe less relevant to the specific situation at hand.
Your argument is just as inane as those who use statistics..
Great analysis Brian. This is Mark Kamal from Protrade (I sometimes post comments and love the site).
We ran the WP using our WP model. For league average offenses, we uses a 59% chance to convert, WP=99% if get it and 61% if fail to get it. That gives a 83.4 WP to go for it, versus 86.5% to punt. (our main difference is the IND chances after a punt: in a 2-minute drill with no timeouts, our model says this is less than 30% for a league-avg offense).
Now adjusting for NE/IND offenses, the decision basically becomes a wash: both are about 84%.
A bigger strategic tactic would have been to let Addai score on his 13 yard run with 1:20 left (the inverse of MJD sitting at the 1). Our anaylsis shows that NE's WP by tackling Addai at the 1 was 15.1%, versus 22% if he scores there. New England would have gotten the ball back with 1:20 left, only needing the FG for the win.
Good stuff...keep it up!
-Mark
Although it is not enough data points, could we not draw some conclucsion from the fact that the Pats could not get two yards on two consecutive plays? If it seems to have been a wash with 70% success for the Pats and they DIDN'T get that success we can think they were "unlucky" or that the assumptiosn we are making are wrong. Which on eis it? Dunno.
The only problem I have with Mr. Kamal's analysis above is assuming a 59% chance of converting. THis seems wrong, the data set this is composed of has to deal with teams that have to cover long passes/long gains. The colts are in no such position they have to sell out entirely to stop the first down. Hell isnt that what they were doing?
If you adjust the conversion downward to a more reasonable 50% ala 2 pt conversions, this should show that the decision was wrong but not terribly wrong. (This is a follow up to my post with the links to dynamist and 4th down study)
I bet the odds of Peyton Manning leading the Colts to a winning touchdown starting from anywhere from the field with about two minutes left to play is greater than 75%. Furthermore, whether they got 28 yards or 66 yards to go, they'd find a way to score the touchdown with about 10 seconds left to play in the game. They are just that good. Perhaps, the optimum strategy for the Patriots would have been for Tom Brady to run into the endzone, set the ball down on the turf and let the Colts recover for the touchdown. Then there would have been probably a better than even chance that the Patriots would have won the game on a last second field goal.
Faulty data. No way does an NFL team have a 60% chance of making 2 yards on fourth down. That figure has been known to be 43% for decades, going back to research by former NFL QB and engineer Virgil Carter. When you factor in game situation, that number figures to be smaller. Belichick's chance of success was not zero, but neither was it anything approaching 78%. Punt was the play here.
pats made the first down should have been reviewed they wanted colts/peyton unbeaten sham
Something else occurred to me about this - if NE wins, they are 7-2 and Indy is 8-1. If they finish with the same record, the winner of this game would get home-field advantage. Also, NE would be 3 games ahead in their division.
It wouldn't surprise me that Belicheck knew that winning the game with Indy could determine home-field advantage, and factored that into his decision (which was the right one).
That review would have had to come from the PAts since the 2-minute warning had not happened before the play started. Since they had no TO's, they couldn't challenge (I don't quite think it was a conspiracy theory).
As for the chances to get 4th and 2 - we are using the most recent decade's worth of data. Is it possible the older analysis could be dated now? 2-point conversions are converted around 48% of the time - so given there is more room for the offense to work with, instinct supports the data in saying the 4th-and-2 conversion would be hire than that, no?
That 60% chance of conversion is a high, especially realizing that the Pats of have stuggled on 3rd and short this year. Also realizing that there overall 4th down conversion rate is around the league median (40-50%) for this particular Pats team, I'd put their chances of conversion at 45-50%, if not lower. Additionally, Peyton Manning and the Colts with all the momentum and a discouraged Pats team will score from their own 28 way higher then 53% of the time. I'm thinking more 85-90%. Also, with a drive going 68 yards, considering Peyton has to throw much longer, I don't think the 30% chance of scoring goes up much higher. I'll say 35-40%.
Doing this math its more like 53-58% vs. 60-65%, which most likely says punt depending on your risk aversion.
Even outside of this math, you have the lead! Why take a chance like that and literally give the game away?
[Quote]One quick question about the 4th and 2 being successful 60% of the time: Does that take into account the game situation? I'd imagine its harder to convert a 4th down when the defense knows that a 1st down is a certain loss, versus situations where the defense has to guard against the possibility of a big play.
In other words, in your average 4th down situation, a defense is primarily worried about giving up the 1st down, but also doesn't want the offense to completely beat their man and score a 70-yard TD. Here, the defense need not worry about that, since the Pats win regardless, which should allow them to play their men tighter and drive down the conversion percentage a bit. [Quote]
I have to agree on Brad with the above statement. In the Final mins of a football all the stats get thrown out. The game changes immensely. Going for it on the 30 is a real bad decision with a 60% of making it. That is really close to leaving the game up to a coin toss. Then if you miss the opposing team has great field position. With a starting field position like that the Lions could even score a game winning TD.
Great analysis, as always. It kills me how many "experts" commenting on this topic do not have a clue, and present absolutely nothing to support the "conventional" wisdom. Moreover, as you say, the fact you are facing Manning only makes it a better decision. The higher the probability the opposing QB will score at relative locations on the field, the more valuable your 60% chance of winning by going for it becomes. In other words, if you are facing the Raiders and Russell, maybe you punt.
Great article. Factoring in the power of the two offenses in question -- Pats are much more likely to convert, and Colts are much more likely to score -- only skews the results even more in favor of trying the 4th down conversion.
Journalists on tv and in print should reserve their laughter and ridicule for times when it's obvious that a mistake has been made -- LET ALONE times when the correct decision was made.
Speaking of momentum shift, I lookup the game book and see how both QB perform in 4rd quarter:
Manning: 12 Attempts: 9 Completed, 1 as defensive passing interference, 1 incomplete, 1 Intercepted;
83.3%
Net Gain 150 yards
In contrast,
Brady: 11 Attempts: 8 Completed, 3 incomplete
72.7%
Net Gain 54 yards
And as pervious mention, Manning deliver two 79 yards TD drive, 1 took 1:49, and 1 took 2:04, both without using timeout.
I guess this might help some of you thinking of this decision?
A lot of comments saying that NE's conversion probability is less than 60%, beause Indianapolis does not have to worry about the deep ball. I could not disagree more.
1) The 2-point conversion rate in the NFL is 47% on average--this is after you factor out botched PAT snaps which misleadingly end up in the 2-point conversion stats. That is 47% with the back of the end zone to work with, meaning you have >12 yards with which to work. That makes a difference--think about how common it is for a catch to be ruled incomplete because the pass went beyond the end zone.
My point here is that 47% is the lower limit for the possibility of NE converting on fourth and 2. We should not even consider giving them a 46% chance or less.
2) On fourth-and-2, the defense is normally focusing on preventing the first down regardless of game situation. They may pay a little bit of attention to the deep threat if the ball is at midfield, but not much. It's somewhat unusual to go deep on fourth-and-short.
3) You HAVE to pay attention to the deep threat against the Pats, especially in that 5 receiver set. Tom Brady has a 63.3% lifetime completion percentage and he's 66.2% this year. So both numbers are above 60% and yield a rather favorable probability of conversion if Brady throws downfield. And seriously, if Randy Moss is 30 yards downfield and unaccounted for, Brady is completing it to Moss a lot more than 60% of the time. Indy can NOT sell out in this fourth-and-2 situation.
4) And as I alluded to already, both teams have great offenses. They have very good defenses, but they are even better on offense. Also factor in that Indy's secondary has been ravaged by injuries lately. This all works in favor of New England converting on fourth down.
I figure NE's chances of converting were right around 60%, maybe even 65%.
"A punt from the 28 typically nets 38 yards, starting the Colts at their own 34."
Useless stat unless it only considers domes.
For those who favor punting the ball away to Manning:
After Manning picked apart the Pats' defense from the 29, are you still convinced that the Pats' defense would have held up if Manning had to drive 70? New England's defense appeared depleted late in the fourth.
The mistake Belichick made, I believe, was NOT bringing the house against Manning in the final two minutes. If you rush eight, you hope to force a turnover or a four and out. If the blitz backfires, Indy scores quickly and gives the ball back to Brady with over a minute to try to set up a field goal.
I apologize. I linked to your site in the ESPN column and I'm sure I brought some of the uninformed comments on your site.
At the time, I was relatively confident the numbers worked out in favor of going for it.
"3. He’ll regret this decision because it cost his team a chance to win"
Thanks James for posting this. So funny. Going for it on 4th down WAS A CHANCE TO WIN THE GAME ON THAT PLAY. Punting eliminates that chance, thus it is punting that "costs" a chance to win. It's one (expected) thing for ESPN to have poor analysis. But using cliches that are exactly the opposite of the truth, well, that's beyond idiotic.
In response to Bart's comment about Virgil Carter's research:
If the 4th Down Conversion Percentage has been known to be 43% for "decades", does that figure take into account anything that has happened over those last decades? I would think that thousands of NFL games might shift the statistic a bit.
And thanks Bart, for granting that the Success Rate was not zero. I know that we were ALL thinking it was zero.
It's funny that the formula doesn't take into affect the area on the field, which is the only thing that really matters. FAIL.
I love the hypocrisy amongst the cliche-dependent masses. Last week Tom Coughlin plays it conservative and depends on his defense to prevent a touchdown, and most people were saying "They had a change to win the game right there! Coughlin was too conservative, he was playing not to lose!"
Then Belichek plays to win, doesn't play it conservative at all, and it's like "oh nonononono we didn't say to go for the win, we said you shouldn't give Manning a short field."
I hope next week the Vikings are trailing by six points, and some team decided to kick it deep instead of squibbing it with 90 seconds to go. And then if they return that kick for a touchdown and win the game, people will say "WAIT...nononono, we didn't say that you have to give Farve a long field to work with. We said you can't llose the game on secial teams and that you have to trust your defense."
Maybe BB actually made the wrong decision yesterday. Maybe Coughlin made the wrong choice a week ago. It's one thing to side against the coaches. But it's another thing to use silly football cliches as your only argument.
A lot of folks have pointed out flaws in the analysis. The biggest one, for me, is that it's based on league averages, whereas Belichick, assuming he considered all this, has to factor in the personnel on the field: not just Manning, but the punter, Faulk, the guy covering Faulk, etc.
But assuming you could give him an airtight analysis of the odds, there is to me a significant factor that isn't subject to stats: the long-term impact of effectively telling your defense "I don't trust you to stop the opponents after a punt." Is that worth an extra 9 chances out of 100 of victory, or whatever the analysis would show?
Stupid argument.. they took basic numbers leaving out field position and the QB of the other team. Not enough variables in your math!!!!
Spin it anyway you want it was a horrible decision. Hopefully it will cost and the arrogant jerk of a coach the season.
Rollo said:
"But I don't care what these calculations say, you just can't risk giving the ball to the other team at your 29 yard line with the game on the line. Especially when it's Peyton Manning who will be leading the opponent's drive."
"The Patriots lost, by the way, because their defense stunk in the fourth quarter. "
Can't you see your contradiction? You say the defense stunk. So your optimal solution (for BB) is to force them to win the game, rather than your offense? 29 yards is impossible to defend, but somehow 70-75 yards is a lock for the D? Against Manning who just led 2 long TD drives?
So we have 9 percentage point difference when discussing NFL averages, which includes such power houses as the Lions. What about just using the Colts? Just looking through the first 3 games of this year and the colts scored a TD 6 out 9 times they crossed their opponents' 28. I think that about changes the formula, no?
You are trusting your defense to stop them, but from your 30. That is a lot more trust than punting it away and saying I am giving you 70 yards to stop them. Even with the best defense in the world you still go for it. In fact, the better your defense the better decision it is to go for it.
"Historically, in a situation with 2:00 left and needing a TD to either win or tie, teams get the TD 53% of the time from that field position."
The Colts chances of scoring after a failed 4th down are irrelevant - a complete red herring. At the time the decision was made, all that we could have possibly known about the 40% of the time the conversion failed is just that - the conversion failed. Bill didn't know where the Colts would get the ball. Brady could have been sacked on the 1. Or they could have been stopped on the 28 and the Colts may have been asses an unsportsmanlike conduct backing them up to the 43. There's simply no way of knowing what situation the Colts would have been in if the 40% failure came to pass. So really, the decision was go for the first (60% chance) or punt (%70% chance).
BB has guts. (He also has job security.) While he probably didn't know the exact stats, I think he assumed that the WP was slightly in his favor if he went for it instead of punting. In this situation, almost any other coach in the country would take the easy way out and punt the ball for fear that he will be lambasted if his team didn't covert. I know the coach of my team wouldn't go for it.
The only other coach that I think would make the same call in this situation is Pete Carroll (not this year, of course). He is one of the few coaches who tries to convert on 4th down when he thinks it is worth the risk, fans and critics be darned.
The better the Colts are inside the 30, the better they are from their own 30. They are good. And they better they are, the better decision it is to go for it. Think about it, if they colts have a 100% chance of scoring inside the 30 and 90% from their own 30, your only real chance of winning is to go for it. The better the Colts, the higher the chance of loss by punting.
"Stupid argument.. they took basic numbers leaving out field position and the QB of the other team. Not enough variables in your math!!!!"
I don't see how you can say they left field position out of the argument, unless you're not referring to the original article. The thesis is that the average in that position for the games logged since 2000 points to a .78 WP for going for it there, or a .70 WP for punting. Both of those figures factor in the resulting field position and the potential for converting the first down.
The grey area in this debate is which way does the the fact these are two teams considerably above average skew these chances?
In any event I don't think you can say either decision is horrible, it's certainly a closer call than anyone in the idiot sports media thinks...
Great dissection of Rollo's comment. This seems to be the trend of the Punt Supporters.
On one end, they say "You don't trust your defense to stop Peyton from 70 yards away?"
On the other end, they say, "You're going to give the ball to Peyton on your own 29??"
Extremely contradictory. You criticize Belichick for not trusting his defense when your premise involves not trusting the Patriots' defense.
If you're going to be a proponent of the punt, you must decide that you have confidence in the Patriots' defense. And after you decide that, consider how THAT confidence matches up with this scenario: (Confidence that Brady will convert on 4th and 2) + (Confidence that the defense will keep Peyton out of the end zone from 29 yards out)
Then reach an educated conclusion.
Wasn't the real mistake Bellicheck made throwing it on 3rd and 2? If you know you're going to go for it on 4th and 2, why not run it on third, take it to the 2 minute warning and use that time to come up with your 4th down play instead of burning your last timeout? Even if everything still played out the same way, the Pats at least get 1 extra play to potentially have a 60 yard field goal to win.
I also think they should have told the defense that if Indy gets a first down, just let them score. Then, they'd be down 1 with a timeout and a minute or so left.
Either or both of these decisions increase their WP in combination with going for it on 4th.
I like this debate--beats jumping off the Tobin. It appears that going for it was the percentage play. Anyone have any statistical thoughts on the next series: now that Indy actually has the ball at the Pats 30, what strategy gives the Pats the best chance to win? Some have suggested that letting Indy score outright, or not making the tackle at the one, gives the ball to Brady with anywhere from 1 to 2 minutes to play. As outrageous as those ideas sound, what does the math say (if anything)?
"Wasn't the real mistake Bellicheck made throwing it on 3rd and 2? If you know you're going to go for it on 4th and 2, why not run it on third, take it to the 2 minute warning and use that time to come up with your 4th down play instead of burning your last timeout?"
That's an EXCELLENT POINT! Bellicheck certainly is guilty of at least not thinking 2 moves ahead.
What those statistics show is a 60% chance to win if you go for it, but a 70% chance to win if you punt... might just be me, but, 70% > 60% no? lol.
The post above me is dead on. They threw it on 3rd down and 4th down, all they needed was 2 measely yards... Not to mention the fact, burning their timeouts like nothing and for no reason. It's like they wanted to throw that game away.
A fair point. That is why they had to take the last timeout. He wasn't sure what he was going to do on 4th down, the punt team started to go out, etc. If he knew he was going on 2nd down, you probably run twice to get the clock impact as well. Hard to know whether they would have made it, but he clock would have been impacted.
No... if you go for it, the chances of winning is above 60% because: There is a 60% chance of CONVERTING on 4th. Even if you don't convert, you still have a chance to hold Indy to less than 29 yards.
Throwing on 3rd and 2, throwing it on all four downs, calling time out prior to the 4th down, calling timeout at the start of the possession... there were many mistakes that added up. Most people are over looking those and focusing soley on the 4th down play.
A game never truly comes down to one play, there are always a handful of plays that led to that one play being so important. That whole possession was a poor coaching job.
This is one which doesn't include everything, or anything close to it, to make the results even remotely reliable. Lets throw in a few other simple variables.
Like when facing the #1 scoring D in the NFL, what's the percentage?
Or what's Peyton's percentage of scoring when needing a TD from 28 yards out?
For the original results to work in the real world, you have to make the assumption that there's no difference from Jamarcus Russel going 60 yards against Pittsburgh, or Brees going 60 yards against Detroit. THIS IS A HUGE DIFFERENCE!
I'll do a quick adjustment using football outsiders rankings of "above average" for these players and D's. They aren't perfect, but they are clearly better than saying Peyton is no better than Brady Quinn which is the foundation the original formula uses.
Teams and coaches use this kind of thinking. Without it there would only be one game plan that would never change.
For example, QB's scramble for first downs x (we will call it 2 for arguments sake) percent of a time. Thus when facing Vick a few years ago or Vince Young, using a spy defense is completely worthless, as it will only help 2% of the time. Clearly common sense shows the fallability of this argument, and why it doesn't work.
Colts pass D is 11.4% better than average for the throw to Faulk, Peyton is rated at 44% better than average overall, NE pass D 1.3% worse than average, negligible...
so .88 is indy's d count, and .69 is the Peyton modifier
(.6*.88)+ (.4*(.69-.53)= 53.3 win percentage for NE if they go for it
now the flip side is the same modifier in a positive, 1.44 for Peyton
(.3*1.44)= 66.8 win percentage if they punt it
Thus the opponent here clearly makes a massive difference. Giving NE a much better winning chance if they just punt the ball.
Unfortunately, the flaw in your argument is that you are saying "Peyton is a great quarterback, so give him the ball instead of going for the win and THEN giving him the ball"
"Indy had two fourth quarter TDs prior to the critical call. They had moved 79 yards on both drives. The first took 2:05. The second took 1:49. Neither drive used a timeout."
Here's a twist:
What are the odds that Manning can engineer not two, but three perfect drives down a long field in less than 2 minutes with no timeouts? He's a great QB and their logo is a horseshoe, but the odds are that the third one would fall short as the probability axe falls.
Interesting how you don't count for New England's offense being better than average in that.
Also,Indy does no t have an 11.4% better pass d...they don't give up many tds, but that is irrelevant in this situation where you don't need a td.
They give up one of the highest completion %, but one of the lowest YPC and are average in PYA. In a situation where you only need 2 yards, not sure that Indy is a bad d to go against.
General stats don't work when comparing high quality teams - why put in stats from the Rams and Lions,.... Why not pull out Pats 4th down % vs Colts Defense, and manning 2:00 % vs the 2 min D of the Pats. Use useful stats, not general stats! It's like those meaningless stats "Every 2nd Thursday of November the teams wearing black in an away game wins, but only when it's raining"
What are the odds that Manning can engineer not two, but three perfect drives down a long field in less than 2 minutes with no timeouts? He's a great QB and their logo is a horseshoe, but the odds are that the third one would fall short as the probability axe falls.
------------
Flipping heads twice doesn't change the probability of flipping a third heads.
I wish everyone would stop trying to modify the percentages by +1%, -2% there. The fact is, while there are assumptions being made, this is a good ballpark estimate. The conclusion is that Belichick is not a complete moron for making this call.
And don't forget that Indy has injuries to Bob Sanders, Marlin Jackson, Kelvin Hayden. They may have been a great pass defense thru week seven, but not anymore. That's three key members of their secondary.
Injured Colts -----> Better chance of a Patriots fourth down conversion.
This is complete bs. You have to look at where the momentum of the game was going. First, the Pats defense already had 2 picks on Manning. If you punt it (and btw the Pats were averaging way more than 38 yards net on punts) Manning is probably going to have to air it out at least once or twice giving your defense the chance to make a big play. Also, it's a huge momentum swing giving the ball up at the 30 vs. making the colts drive the length of the field. But hey, we all called Bill a genius for taking an intentional safety in that win over the Broncos and we all would've said how gutsy he was if the Pats converted... he just didn't get it and now looks like an idiot
What everyone is missing is: We don't have to ask WHAT WOULD HAVE happened if the Pats' defense got the chance to stop Peyton. THEY HAD THE CHANCE TO! And I'm not sure if everyone had already turned off the game, but Peyton was 2/2 for around 15 yards and a touchdown. No reason to play what-if when you actually HAVE the benefit of heinsight.
THis was my pt. about trying to use a TD probability as a function of field position: It doesnt really matter what you think the Colts exact chances are from their own 30 or the enemy 30. It only matters how much DIFFERENT is the probability from the (hopefully) 40 yard approx. difference on the punt.
WHether you are playing the Steel Curtain Steelers or the Detroit Cream puffs. Whether Peyton Manning at the controls or Joe Pizarcik, there has to be something in the order of 40% difference on a 40 yard change of position; it's not going to vary that much team by team . That's the beauty of trying to calculate it that way. YOu can try plugging in other numbers but it still seems somewhere in that neighborhood.
Take this quote:
QUOTE "What about just using the Colts? Just looking through the first 3 games of this year and the colts scored a TD 6 out 9 times they crossed their opponents' 28. I think that about changes the formula, no?
Right assume they are 67% from the NE 30, their still has to be a lot chance from their own 30. One does not need the exact probability from the 30, one only needs the differential, assuming a 40 yard difference on change of possession.
This was my pt. when citing the articles in expected pt. vs field position.
You cannot create a perfect set of statistics to govern the situation. The two teams only play eachother once a year, and data back more than a year or two is worthless as the players and coaches change
The best that you can do is what was done here...show that more often than not a team in NE's situation should go for it.
QUOTE: "Indy had two fourth quarter TDs prior to the critical call. They had moved 79 yards on both drives. The first took 2:05. The second took 1:49. Neither drive used a timeout."
I guess this quote is used to support the idea that the clock doesnt matter to the Indy juggernaut. But if they took 2.05 for the final drive, they would have run out of time as they got the ball back with less than 2 minutes....
SO this argument/comment seems weak or at least it is not understood what it is trying to prove.
"Flipping heads twice doesn't change the probability of flipping a third heads."
True with coin-flipping, etc, but if that applied to long football drives against a top pro defense playing to preserve a win after being burned, then winning game scores near 70-75 points in pro football would not be uncommon at all.
It simply cannot be done on a repeated consistent basis.
QUOTE: "General stats don't work when comparing high quality teams - why put in stats from the Rams and Lions,.... Why not pull out Pats 4th down % vs Colts Defense, and manning 2:00 % vs the 2 min D of the Pats. Use useful stats, not general stats! It's like those meaningless stats "Every 2nd Thursday of November the teams wearing black in an away game wins, but only when it's raining"
THe problem with your first set of stats is the lack of a meaningful sample size. YOu really need about 20 or so data pts. before any of this begins to have "confidence."
THe problem with your second set of quotes, is that this is the sort of stat (when its raining) that doesnt have much predictive value; that something may or may not occur when teams where black may have been true but there is very little reason to think it is the REASON stuff is happening.
There's also a matter of consistency. If you could play the end of the game 20 times, you would win it more often by going for it But you only get to play it once. What you're giving up is a CONSISTENT 70% of winning by punting... based on the outcome of ONE PLAY you go to either 100% or 46%. That's a lot to risk on one play. A coach had better be very confident that the SPECIFIC play he's calling against this specific defense has a 60+% chance of working.
I immediately thought of this site when it became apparent the Patriots were going for it. I knew from previous work Brian has done that the numbers were probably slightly in favor of going for it but as a Colts fan who has watched Peyton for years I thought the decision was incorrect for several reasons.
Peyton does not have the strongest arm or prefect spiral throws and as a result can not hit the 65-70 yard bomb that Brady could. Going for it and failing left Peyton in a position to reach the end zone on any given play. His first touchdown of the fourth quarter was a pass to Garcon from almost the exact same distance. Punting probably makes him execute a series of at least 3-5 and more likely 5-7 plays or more against a defense that had already picked him off twice.
From the spot where the Colts got the ball Peyton basically had the entire playbook at his disposal and the option to waste plays if he didn't like the setup. Add an extra 40 yards and the need to execute several more plays and his margin of error diminishes greatly. In fact, at least two of the successful plays on the final drive (the touchdown slant and the first Addai run) either would not have been options or would have wasted a lot of time.
Statistically, Brian's work has shown that the theoretical ceiling for game prediction models is 76%. Given that randomness has such a large role in game outcomes and extending that to play outcomes, I would personally need to see a much larger discrepancy in favor of an unconventional approach before I would use it in such an important situation.
A net punt of 38 yards seems a little low. Hadn't the Pats punter been booming them all night?
Yes, the Pats punter had an average of 44 with no return on the night, but on the season he averages only a net of 35. 38 seems a bit low for and average, but probably accounts for blocks as well.
There are two types of naysayers on this board.
First, there are those who accept that this is a straightforward question of probabilities and basic math. They may challenge the assumptions, but they accept reason as the guiding principle. Fair enough.
The second type is incredibly irksome. They reject the math as being the domain of "geeks" and instead offer cliches and well-trodden conventional wisdom. They seem to be saying that logical analysis has no role in decision making; every decision should be made by feel, from the gut instinct. But then how can you argue w/Belichick's gut instinct?
"Belichick has done so many things right during his Patriots coaching career that it's a stunner when he does something so wrong. But this was a total miscalculation of the percentages. In the pressure of the moment, he forgot how to do the football math."
http://sports.espn.go.com/espn/columns/story?columnist=wojciechowski_gene&id=4660392&sportCat=nfl
Wrong, Gene Wojciechowski.
Something else worth mentioning: .79 to .70 WP does not look like a large discrepancy, but it's actually quite significant. Looking at the flip side of the equation the Colts have a 30% shot in one case and only a 21% shot in the other. Big difference.
I had the Colts at 35% to score if NE punted and 60% to score if they did not make it. This is a 76% win chance for not punting and a 65% win chance if you would punt. To account for that 35% chance to score, I'd have to put the Colts at 95% chance to score a TD and New England to have no time to kick a FG. Forty two percent above the league average with no adjustment to the 35% (basically the best offense in NFL history by a lot). I am guessing Belicheck hopes his collegues like Tony Dungy think he was not playing the percentages, so he can continue to beat them by actually playing the percentages and not his gut or fear.
Finally, someone who is willing to look at this problem in the correct context ...instead of reverting to conventional football dogma: "You can't go for it in this situation ..it is unprecented"
Several associates and I were engaged in this debate earlier (prior to my reading this), and this is what is crystal clear:
This is simply a probability tree (as also mentioned by you). The question is what probabilities do you assign:
1) Making 4th and 2 in that situation
2) The Colts scoring a TD from that position
3) The Colts scoring a Td following a punt
In order to intelligently entertain a "punt or go for it" discussion, these are the factors that must be quantified.
I'd disagree with your %s ...but that is a matter of opinion. My #s:
4th Down success: 50% ... since there is little fear from Colts of Patriots going deep and scoring a superfluous TD, they were able to crowd the line of scrimmage and jump the short routes, thereby reducing what would be the normal conversion rate by the Pats.
% chance of Colts scoring from that spot: 70% ... we are talking about the Colts and Manning here in critical situation ..plus the Pats had no answers for Colts in the prior drive.
% chance of Colts going the whole field to score: 45% (maybe closer to 50%)
My #s suggest punt success is 50% to 55% versus going for it which is 65%.
Either way, there is no clear cut answer to this, and any analyst who suggests after the fact that this was a tragically bad decision after the fact without quantifying the probabilities is full of ****.
What does seem unwise is:
If you know you are going for it on 4th down, why not run it on 3rd down since:
1) The Colts are going to see this as a 3 down situation and are going to defend pass first given the tendancies of the Pats.
2) Even if you don't get the 1st down, it may be a 4th and inches situation which would raise the 4th down probability.
Also, the timeout fiasco was less than optimal.
Clearly not Bill's best day, but an unthinkably bad mistake going for it on 4th down? No!
Posted by: Clock Murphy
Obama is a turd. He would have punted.
Great post on the numbers behind this. Pats success rate on getting 2 yards has to be higher than the league average as well, making it even more obvious to go for it.
And I don't buy that making the Colts go 65 yards is THAT much better than making them go 30 with 2 minutes left. Your defense has to step it up no matter what.
More teams should embrace this to win more games. I love the Houston Rockets philosophy where they have broken down where the most efficient shots are for each player, and build their team strategy around that. No wonder they always push teams to the brink with sub-standard talent.
http://fifthdown.blogs.nytimes.com/2009/11/16/zeus-computer-program-supports-belichicks-fourth-down-bid/
For those saying using league averages is wrong, the ZEUS simulation program also supports Brian's conclusions USING CUSTOM FEATURES of the Pats and Colts, saying that the Pats had a 78.6% WP going for it and 76.4% punting it.
1. Commenters are overselling the similarities between this particular 4th down and making a two-point conversion. Considering two point conversions are more successful on running plays than passing plays, this is clearly a distance where small yardage changes make a large difference (as indeed was the case here). This was a shorter conversion than a two-point and thus more likely to occur. There's also a large difference between not being able to use the field past ten yards and simply being unlikely to. The Pats threw quickly, which encourages one to think of this play as only taking place in the ten yard window, but many two-point conversions are slow-developing passing plays where the QB only throws the ball away hopelessly after a long period of time. Clearly in those situations, the extra field would help. The Patriots had the option of calling such a play (and protection scheme) and the Colts had to take it into consideration.
2. Many commenters want what has happened earlier in the game to be taken into consideration, both the percentage of drives where the Colts had scored before and the average net distance of Patriot punts. Obviously the coaches are aware of these things, but Belichick will also understand sample size and will understand that what has happened in a small sample is not the same thing as what is most likely to happen.
3. Many commenters arguing for a larger difference in the chances of the Colts scoring after a punt versus after the failed 4th down do not seem to be taking 4 downs into the equation. When punting is no longer an option, the chance of scoring on any particular drive is much, much higher. This is why you see so many late game scores. It is also the reason some people argue against punting as a general rule. This effect would seem to be exaggerated by the presence of an elite QB like Manning. There is probably a stronger argument to be made in favor of an even smaller margin of difference. The most likely place for the Patriots to have stopped the Colts after the punt is exactly where they gave them the ball anyway (due to field compression issues).
Why are people so stupid? Over half the people on an ESPN poll thin that there was no excuse for Belicheck's call, but half of the people also think that the Colts would have won anyway.
So, that means that there is a group of people out there who think that Belicheck should have gone with a strategy that was likely to lose just because it is conventional wisdom.
In the greatest 4th quarter comebacks in NFL history, such as the Jets' Monday Night Miracle, did any winning team drive the full length of the field thrice in approx 2 mins each time to pull it off or did they need more time and/or field position assistance from the losing team?
Funny, someone above asked what the odds were of this making SportsCenter. I just saw them quote Brian's stats. Congrats on the publicity.
The NFL is changing with the MJD taking a knee play etc, you really want to be the last team to score and leave no time. Somebody said "Perhaps, the optimum strategy for the Patriots would have been for Tom Brady to run into the endzone, set the ball down on the turf and let the Colts recover for the touchdown."
What happens if the Colts refuse to recover the ball in the endzone, would it be a safety or does the clock just continue to run? Imagine that game of chicken ... eventually somebody would have to recover it. LOL
It was clearly the right decision. The only thing I can add is that Belichick may be a genius but he doesn't go for it all the time. I can't read Belichick's mind, but I think he went for it precisely because of the situation. The situation was that Peyton Manning, who has more game winning drives than Tom Brady in his career, would get the ball back on a punt.
The fact that its was still a smart play even if the Colts only had an average qb should put to rest the argument that going for it was a mistake.
What I find hilarious is the "stat-heads" relying on studies about 4th & 2 and 2-pt conversions, which include a good bit of data, but then asserting that TWO little drives by Manning in the fourth quarter are somehow conclusive that the Colts WOULD DEFINITELY have scored if NE had punted.
Talk about small sample size and misuse of statistics.
Going for it or not is somewhat debateable in that the analysis above deals with long run league wide averages. However, the problem I have with Belichicks decision is that he didn't allow Indianapolis to score on the first play in order to immediately get the ball back. The probability of scoring decreases as the distance to score increases and as the amount of time left decreases. What's the likelihood of a team being able to drive 50 yards in 1:30 to set up a FG with no time outs vs. driving 50 yards in 6 seconds. Of course I'm assuming that Indy's receiver (Wayne?) doesn't pull a Jones-Drew and fall down at the 1 yard line to take time off the clock. Oh would the league office have a conniption over such a strategy.
For a more accurate calculation, one should really look at punt averages by the given punter and also at the given stadium (a dome). It doesn't really make sense to factor in dozens of punts made in the last couple of years at Cleveland into the teeth of wind howling in off Lake Erie.
Also, I have a very, very difficult time believing the stated stat that *average* teams score a TD in under 2 minutes when starting at their own 34. That seems ludicrous and notably without any link or support.
Meant to say "score a TD *30% of the time* in under 2 minutes. when starting at their own 34"
thats overthinking it.
Digression said:
"I humbly submit that if, as a coach, I were given the option to start every drive on my opponents 29 yard line, and all I had to do was stop a 4th and 2 attempt, I would take that deal and win by the most amazingly lopsided margins. Too many factors are being left out of your equations for them to be taken seriously."
Your assuming every one of your opponents drives ends up in a 4th down on their 28 yard line......ummmm yeah, obviously you'd win in that situation. The punting/going decision isn't really the deciding factor there.
Change the decision and force the opponent to go or not go on 4th down from every field position they end up with 4th and short and you wouldn't win lopsided. You'd lose more often than not.
There are countless tweaks you can make, but the question is does that make up 9 points in the model?
And you're going to have to better than tired ESPN cliche's to make this point.
@Clock Murphy
I appreciate that you get there need to be numbers assigned to this, and you're not relying on homespun wisdom.
Still I can't help but think your numbers are coming from an arbitrary place, as opposed to Brian's model which is 9 years of game data.
That said, I think there are minor flaws in the model. I think it's a fair point to tweak the punt distance a bit farther, I think it's fair to not assign a 1 WP to NE converting, they would still have to make a subsequent first down or leave :30 or so for the Colts.
But I don't think it's that drastic.
What isn't addressed here is the difference between running and passing on 4th and 2. Even if you believe that going for it was the right choice, the play call didn't play the percentages.
Football outsiders ran a study in 2003 and determined that on 4th and 2 or less, teams running the ball succeeded 69% of the time while passing succeeded just 43% of the time.
And if forethought had been made, the Patriots should have been running on 3rd and 2 in the first place...if they felt it was 4 down territory, a run on third down has a better chance of getting a 4th and 1 or less, not to mention picking up the first down on third down.
It also needs to be noted that the draw the Patriots were running from shotgun had pretty much worked all night long.
"For a more accurate calculation, one should really look at punt averages by the given punter and also at the given stadium"
Punting in a dome helps, but Chris Hanson is one of the worst punters in the league.
ESPN Mention! Heyo!
"There's also a matter of consistency. If you could play the end of the game 20 times, you would win it more often by going for it But you only get to play it once. What you're giving up is a CONSISTENT 70% of winning by punting... based on the outcome of ONE PLAY you go to either 100% or 46%. That's a lot of risk on one play. A coach had better be very confident that the SPECIFIC play he's calling against this specific defense has a 60+% chance of working."
This is another classic cognitive fallacy. Deferring the Patriots' agency in determining the outcome of the game over 2 minutes rather than over one play may feel "safe", but that doesn't improve its chances at all.
ESPN mentioned the numbers on their MNF countdown, but used their own numbers from the WP calculator ( http://espn.go.com/boston/columns/patriots/blog/_/post/4660828 ) instead of the numbers in this post. Their %'s understate Belichick's right decision, too.
We'll see the Pats again in post-season, so one important thing is how does Belichick comes off tom his team. Does he say, "the math was right, so trust that I'm a genius?" Does he say further that he put it in the hands of BOTH his offense (I gambled on you) and his defense (I trust you on a short field.) In effect, he put his reputation on the line for his team. If either unit had performed, Belichick would be hailed a genius.
Beyond statistics (which are important) there is an element of leadership here that I'll bet was a big part of Belichick thinking. I mean, don't you want to play for a guy like that?
"What you're giving up is a CONSISTENT 70% of winning by punting... based on the outcome of ONE PLAY you go to either 100% or 46%. That's a lot of risk on one play. A coach had better be very confident that the SPECIFIC play he's calling against this specific defense has a 60+% chance of working"
Yes...because losing over the course of two minutes is so much better than losing over the course of one play.
Are you freaking serious?!?!?!
Great site, and what a great day for interesting (statistically-speaking)decisions in football.
I like your analysis, but I have to take issue with using the average probability of success for a 4th and 2. I know it would probably require a pretty rigourous analysis, but there are too many factors that might bias the average 4th and 2 numbers for this situation.
I'm not going to explain my reasoning, but I think the following factors could be significant enough that the average 4th and 2 conversion rate is not applicable here: the timeouts that were taken, the differences in fatigue that defenses and offenses encounter (it's definitely not the same, thus skewing 4th and 2 averages early v. later in the game), and the "game situation" here (I know averages are meant to eliminate things like this, but Belichick clearly did not have the world of play calling available that a coach going for it on 4th and 2 in a more "traditional" situation---e.g., outside of a makeable field goal but inside a meaningful punt--- in the first quarter of a game, and the defensive scheme would reflect this narrowed world of play calling choices, thus affecting the probabilities).
I like this site, though. Thanks!
While the numbers may be accurate (granted they are league averages being generally applied here), the interpretation is incorrect. The 0.79 WP, describes the total probability that the Patriots would win PRIOR to running a play. Once the decision is made to "go for it", the function bifurcates to two independent probabilties - (1) a 60% probability of winning based on successful conversion; given the assumption that a first down would win the game -which I'll concede, and (2) a 47% probability if the conversion fails. Thus, the correct interpretive reasoning is "if we go for it, we have a 60% chance of winning now based on conversion, and would still have a 47% chance of winning even of the conversion fails". If instead the decision is made to punt, the probability of winning is 70%. The reality is that punting would have given the greatest probability of winning.
You're using 60% as the probability that NE makes the 1st down. Yet in your Win Probability article, you state that for 2007, there was a 30% probability of making 4th and 2. Now this is 2009 instead of 2007, but the probabilities couldn't have changed THAT much.
anonymous your numbers are wrong
its about 98% or whatever if they convert, or 47% if they dont. your overthinking things
its not 60% to win based on the conversion, its a 60% chance to convert (or 50%, whatever number u wanna use)
Two posts above...I am afraid your own analysis is flawed. If they don't make it, it's a 47% chance of winning. But if he does make it, he has a 99% chance of winning.
Than means that 60% of the time, he'll have a 99% chance of winning. The other 40% of the time, he'll have a 47% chance of winning.
.6*.99=.594
.40*.47=.188
Add the two together to get .782 (the number is .788 if you assume a 100% shot at winning in the even of a successful conversion)