Let's look at the FG decision more closely. I won't use the WP model, but instead apply some math and logic. There were three options for Alabama:
1. Kneel
2. Hail Mary
3. Attempt the FG
Let's make some assumptions. First, OT is a 50/50 proposition. Alabama was favored in this game, but Auburn was playing strong. Plus, OT is a bit of a dice roll to begin with. Second, Hail Marys (Maries, Mary's?) from that range are probably no more successful in college than they are in the pros, which is around 5%. Lastly, for the sake of the argument, let's say there is zero chance of a defensive TD return on the Hail Mary.
We don't really know the probability of a successful FG attempt or the probability of a successful return or block & return from a range like that, especially in college ball from a kicker without many attempts. But let's set that aside for a moment.
Kneeling would be worth 0.50 WP.
A Hail Mary would be worth:
0.05 * 1.0 + (1-0.05) * 0.50 = 0.525 WP
Now let's ask: What Field goal probability (FG) and TD return probability (R) would we need for the long FG attempt to make sense compared to the Hail Mary? We need to solve for the break-even FG% and Return% which gives us a WP equivalent to 0.525.
A good FG is worth a WP of 1.0. A miss and a return are worth 0.0. A miss and no return are worth .50 WP.
We get an equation for Alabama's WP for a field goal attempt like the following:
FG * 1.0 + (1-FG)(1-R)(.5) = .525
FG + (1-FG)(1-R)(.5) = .525
This equation produces a hyperbolic curve on a 2-dimensional plane as plotted below, where the x-axis is R (the probability of a return TD) and the y-axis is FG (the probability of a successful field goal). We can ignore the negative quadrants and only worry about the space between 0,0 and 1,1. Remember the line is the break-even, so above it the FG decision was smart, and below it the decision was not so smart.
To wrap our heads around this, let's look at an example. If we posit the chance of making the FG is 40%, the chance of a return, given a miss, would need to be less than 58% for the FG to make sense. And if we assume the chance of making the FG is 20%, the attempt would be worthwhile as long as the chance of a return is less than 19%, as shown below.
We can also look at things from the other direction. Suppose that given a miss, Auburn would return the ball for a TD 100% of the time. Alabama would need a 52.5% chance at the FG to make its attempt worthwhile. This makes intuitive sense because it's the same as the WP for the Hail Mary. This is one way of saying that the higher the FG probability, the less the chance of a return even matters.
The question becomes, which side of the curve are we on? Above the curve, the FG attempt makes sense. Below it, the attempt is too risky. To be honest, at the time I thought this was a slam-dunk good decision. But now I'm not so sure. The pros hit 57-yd attempts about 20% of the time, so that's a solid upper bound on the FG chance for Alabama. If the FG probability is 15%, the chance of return can be no more than 10%. If the FG probability is 10%, the chance of return can be no more than 6%.
We can also test how sensitive the answer is to our assumptions. For example, if we say there is no chance for the Hail Mary and that being the stronger team, Alabama's chance in OT was 55% instead of 50%, we get different answers. Under these assumptions, if we posit a 10% FG chance, the chance of a return can be no greater than 9%. And if we assume an 80% chance for Alabama in OT, the chance of return can be no greater than 3%.
One last example: If we keep the Hail Mary at 5% and set the OT chance for Alabama at 55%, for a 10% FG chance, the maximum return chance would be 4%.
My gut tells me the play was pretty close to the FG=10% / Return=6% point. The Alabama kicker maybe had about half the chance of a pro, and the chance of return was significant because of the kick's long range and Alabama's personnel. If my gut is right, we're on the borderline depending on the assumptions.
One thing this exercise shows is that all the media analysis of the decision is hot air. This is complicated stuff, relying on many variables without much in the way of empirical stats to go on. None of the opinions are based on a serious analysis or comparison of relevant past events. I heard one sports writer on the radio today say it was a bad decision because "they run those out in the CFL all the time." Ok. Define all the time. And tell me what the FG probability needs to be to trump all the time.
I don't expect people to do math like this, but let's at least spell out what the considerations are and admit how complicated it is.
Great stuff. Obviously don't have the data, but I would think the return probability is still below 6% -- although being woefully unprepared makes that more questionable.
I also don't think you can dismiss hail mary interceptions for a touchdown in this analysis. Granted, it is a very low probability event (more so than the FG return), but both are similarly low probability events.
Brian, do you have any data on FG missed returns in the NFL? How often are long FG's caught and brought out of the endzone?
Piggybacking on Keith's comment, another potential bad outcome from a hail mary attempt would be a fumble returned for a TD. Granted Auburn would have been in their Hail Mary/prevent defense and probably only rushing 3 guys, but the need for the QB to wait for his receivers to get downfield might increase the risk of a sack/fumble and return for a TD if they do manage to get pressure.
Again not a high-probability event, but not necessarily negligible compared to the probability of a FG returned for a TD.
Nice piggyback. What's the chance of a TD return on a Hail Mary, 1 in 1,000? --> Negligible.
I think the return for tocuhdown probability is probably somewhere between close to or above the INT returned for a TD rate, which is like 15% (right?). Offenses are terrible at making a tackle in the middle of chaos and the failed Fg returner gets a nice head of steam and time to find a running lane unlike in most INT cases.
Steve, you are making the assumption that the missed field goal is returnable. If you use 15% as a baseline you would have figure out the probability of the fg failing short enough to return vs missing right, left or just under but still out of reach.
I don't know, I think the probability of a return man outrunning a bunch of fat guys is a lot, lot higher than 4%.
A solid lower bound would be the probability that a non-fair caught punt would be returned for a touchdown.
I think the greatest probability on the hail Mary isn't the interception return for a TD but the Sack/FF/Recover/Return, think Kurt Warner in the Super Bowl against the Steelers, but with the game tied.
Is there data on how often field goals get blocked according to distance? I have read that's a big concern with the longer fg tries and if so the block plus chance of Td return should be considered
Yes. I did a post on block rate by dist a couple years back.
Found it: http://www.advancednflstats.com/2010/10/its-blocked.html
Thanks for this. I'm curious what to make of the negative values. For example, using the formula above, if the chances of making the field goal was 4%, it would be smart to kick the field goal if the chance of a return, given a miss, was less than -17.7 percent. What should we make of that? What does that mean?
Since the chance of a return cannot be less than -17.7, FG is always a bad decision in that example.
Am I the only one who keeps reading the first word of the title as "Satan"?
(This isn't even me dissing on the Alabama coach. "Satan's hyperbola" just SOUNDS like a concept a mathematician would come up with.)