I'm never quite sure what decision-making trends to expect out of the NFL playoffs. It seems any decision can be justified by the playoffs. "It's the playoffs," one of the exalted keepers of the true knowledge can say. "You have to leave it all on the line," he says as the coach keeps the offense on the field for a fourth and goal.
But, he could just as easily say, "You don't have a choice here. You have to live to fight another day." The field goal team trots out for a 20-yard chip shot instead.
The field goal teams were out in force for Wild Card Weekend. Presented with 27 fourth downs inside the opponent's 40 yard line, teams kicked 17 field goals, punted twice, and went for it just eight times. Of the 17 field goal attempts, only eight were the optimal win expectancy choice according to the 4th down calculator. All told, coaches left 0.24 of win expectancy and 6.3 expected points on the table with these decisions.
Overall, teams saw 66 fourth down plays and made the optimal decision 49 times. Only one of the 11 decisions to go for it was suboptimal (by Washington when they were already down by 10 late in the game) and seven of the 33 punts were as well. The biggest whiffs were typically in field goal situations, but to the coaches' credit, the kickers were sharp: they combined to convert 16 of the 17 field goals on the week.
Still, there were a few calls worth questioning even given the true kicks. After making a borderline call to go for it on fourth-and-5 from the 34 -- probably the right call given the unreliability of Mason Crosby this season -- and succeeding, the Packers kicked on a fourth-and-goal from the one yard line with 3:25 to go in the 2nd quarter. The Seahawks, up by seven against the Redskins, chose to kick on fourth-and-goal from the 4 with 5:32 to go in the game.
Despite the automatic nature of field goals from such short distances, the Packers were leaving 2.1 points on the board (they "led" the week with 4.3 EPA and 0.10 WPA below optimal decisions). For the Seahawks, leaving Washington pinned inside the five down by seven was a roughly equivalent position to leading by 10 and giving them the ball on a kickoff; leading by 14 would have all but assured overtime as the worst possible case.
Fourth down decisions made a boring game in Houston between the Texans and Bengals into an infuriating one. Gary Kubiak's Texans chose to kick from inside the 10-yard line three times and again on a fourth-and-1 from the 30; he also punted on a fourth-and-1 from the 50 up by six in the third quarter. All told, he left 3.3 points and five percentage points of win expectancy on the table.
Marvin Lewis had fewer tough decisions to make, but he was somehow even more passive. He twice chose to kick field goals down by nine, first on a fourth-and-2 from the 16 in the third quarter -- the one miss of the weekend -- and again on a fourth-and-2 from the 29 in the fourth quarter. Lewis left a full point on the table with these decisions and, more importantly, seven percentage points of win expectancy.
Of course, solid play and good coaching in other areas can more than overcome these (usually) small tactical errors. The Packers and Seahawks won despite worse decisions (by WPA) than the Vikings and Washington. The Bengals and Colts were losers in the same situation.
A breakdown by team and situation can be seen below:
- Home
- game analysis
- Coaches Bring Passiveness To Wild Card Weekend
Coaches Bring Passiveness To Wild Card Weekend
By
Jack Moore
published on 1/07/2013
in
analysis,
game analysis
Subscribe to:
Post Comments (Atom)
Really enjoyed this. Encore next week?
*In the Marvin Lewis paragraph you said Cinci missed the lone FG this week, but it was Indy.
I am all for aggressive decisions, but even I was mildly surprised when reading "For the Seahawks, leaving Washington pinned inside the five down by seven was a roughly equivalent position to leading by 10 and giving them the ball on a kickoff."
Occasionally, the announcers will speak of an impending tough decision, but in this case, they did not even consider the merits of going for it.
I would love to see info like this for the entire season. I think I remember either this site or Football Outsiders publishing an article last year that ranked all 32 coaches by their aggressiveness. As a Lions fan it was frustrating to see Jim Schwartz be so passive in several situations this season. It was one of their many problems.
How about the Vikings decision to pooch kick instead of onsides kick down 2 TD's with 3 minutes to go?
I've been giving coaches a Coach's Tolerance Constant of about 0.5 EPA on these decisions.
Yeah, there's something wonky with the 4th down decision that Luke Kim referenced. According to the calculator, punting is a better decision than kicking a chip shot field goal. o_O
I was palming my face because they bothered to pass the ball. Two things: one, the clock becomes a factor much earlier in a two-score game. With four minutes on the clock, you probably need to hit an onside kick to score twice. Two, there's no benefit to passing--you're just as likely to score on two running plays, if not moreso.
Brian, I apologize if you have answered this question in a past post and I did not get a chance to check for your response:
Isn't there a selection bias in the past data you use to evaluate 4th down probabilities? That is, the teams that, in the past, went for it in these 4th down situations are likely to have been teams with much-better-than-average offense, or teams facing an inferior defense, or some combination. This would probably be most true in situations where teams didn't "have" to score a TD (as opposed to FG attempt) or "have" to score at least a FG (as opposed to a punt) on that drive.
If that was the case, wouldn't these probabilities be suited only for teams having offenses significantly better than the defenses faced?
Sorry, I just noticed Jack posted this article, my question above is directed at both/either of you.
I agree Jared. Additionally, the use of a 'league average' is not proper in determining an optimatal result, as I have mentioned before.
Just look at the regular season spread on going for it on 4th downs - it ranges from 87% (Colts 7 of 8) to 25% (oakland, 4 of 16) and is pretty evenly spread throughout that range of 60%.
The proper conclusion is more like the success rate of going for it would be 55% +- 30%, so if optimal point falls in that range, then it is not conclusively an optimal decision. Even if you look at a single team like the colts, do they really have a 87.5% chance of making the next 4th down? no. that is merely an estimate of a mean, of an extremely complicated situation with very few samples. The colts could very easily have been 8 for 8, or 6 for 8, giving a range of 25%. A range of much more than that is probably more appropriate.
All the charts above show the WPA of less than 0.1, then it seems to be in the noise.
There is an additional point of whether one should try to maximize expectation value. The coach most certainly will not get enough realizations to make that an effective strategy - especially when making a high variance choice exposes you to a significant chance of losing. Coaches clearly choose to eliminate losing situations rather than maximize expectation value.
that GB situation illustrates the point pretty well (FG from the 1 in 2nd quarter).
In looking at the WP, since the coaches goal is to win, we see that coach has a choice of either a certain 73% chance to win, or a gamble on 84% or 65%.
There is only one game, one situation, the coach needs to make that dip in win percentage impossible. It is even more profound if that dip would drop below WP to below 50%.
The over all expectation value of WP is only 5% so if you play an entire season, it is unlikely that the strategy produces an additional win for you (even if you face it each game). However, you will have 5 or 6 blown opportunities (32% failure rate - probably more when teams know you always go for it), some of which may be losses (22%*16 = 3.5 losses) and that will get you fired.
Love this, Jack. Great stuff
Criticisms of the model are certainly apropos - but stuff like this demonstrates misunderstanding:
> The coach most certainly will not get enough realizations to make
> that an effective strategy - especially when making a high
> variance choice exposes you to a significant chance of losing.
Do you know how many hands of blackjack do you have to play, before you get enough realizations to make splitting eights an effective strategy? The answer is not even one -- splitting eights is *always* an effective strategy.
On some level, it's mind-blowing that NFL teams don't either have coaches who are capable of making good decisions or, alternatively, hire or train someone to do so. The teams certainly have enough money for that (and I expect they could even get qualified volunteers.)
> ...Coaches clearly choose to eliminate losing situations rather
> than maximize expectation value...
In this context, maximized expectation value and minimized losing situation mean the same thing. Talking about 'football capital' doesn't excuse the poor decision making.
"splitting eights is *always* an effective strategy."
not relevant. The choice is different and does not apply to the scenario I discussed.
A better example would be would you want a certainty of $10 million or a 50/50 chance at $100 million? The "effective strategy" would be the 50/50 chance, but only a fool would take it.
When you only have one realization, a difference in estimated expectation values is meaningless compared to the certainty of a minimum loss level (at least in some cases - like the coach if fails to get the TD 3 or 4 times in a season, loses a few of those games, and gets fired because of it).
Now compound that by the fact that these success rates are merely estimates and have a huge range (huge). With 4th down percentages, fully half of the teams in the NFL are either below 40% or above 60% at being successful. How could you possibly believe you have a correct prediction for a future play based on that?
thanks for the discussion.
> A better example would be would you want a certainty of $10
> million or a 50/50 chance at $100 million? The "effective
> strategy" would be the 50/50 chance, but only a fool would
> take it.
There are some other issues with this analogy, but what, if anything, is at stake, other than the outcome of the game? We're talking about, say, 60/40 chance at a win, or a 80/20 chance at a win.
> ...these success rates are merely estimates and have a huge
> range (huge)...
The methods for estimating the conversion rate and the values of the payoffs are things that can be criticized. I'm not certain about these particular scenarios, but in a larger context, a number of sophisticated, smart, and careful people have all come to the conclusion that coaches are too timid.
> ...like the coach if fails to get the TD 3 or 4 times in a
> season, loses a few of those games, and gets fired because
> of it).
Sure, we can say that the coach's first job is keeping the owners happy, and maybe, by that measure, keeping Rex Ryan's decision to keep Sanchez in as starter is great coaching, and many people have heard the stories about Al Davis calling plays as an owner.
It's worth pointing out that 4th down decisions are a well-known issue at this point. I can't help but think that a well-prepared coach should be able to make them confidently and correctly, and have little trouble justifying them to the ownership.
FWIW, I recently read that for a coach who wants to retain his job, the winning strategy is to have a terrible first year - lowering expectations during the honeymoon period - and then gradually improve the team's record.
I guess to further try to explain my point on the subject, is that this analysis looks at the starting case (4th down), and the ending case (win probability after the play), and when looking at expectation values, one assumes a large number of iterations.
It does not examine every single path to get to those results. It is like gambling, and tripling your bet when you lose. In theory, you will eventually win a bet large enough to cover all previous losses, so you will always win. However, in practice, you will eventually have a losing streak and go broke - or hit house limits on the bet amount. in practice, tripling your bet when you lose fails because you could not access the entirety of state space.
Similarly with 4th down. You need to be able to access all states, to make the expectation value a correct representation. But you will never have a coach that fails on 4th down 18 straight times, without having the owner, the GM, the assistant coaches, the fans, reporters forming a mob and lynching him. It is not merely a 'looking good' thing, it is an actual period of failure that teams will move to correct. Changes will happen. And while 18 straight failures is "unlikely", it is not impossible, and it is necessary to be a part of the ensemble in calculating the expected value, on which the decision is based.
"Similarly with 4th down. You need to be able to access all states, to make the expectation value a correct representation. But you will never have a coach that fails on 4th down 18 straight times, without having the owner, the GM, the assistant coaches, the fans, reporters forming a mob and lynching him. It is not merely a 'looking good' thing, it is an actual period of failure that teams will move to correct. Changes will happen. And while 18 straight failures is "unlikely", it is not impossible, and it is necessary to be a part of the ensemble in calculating the expected value, on which the decision is based."
you can't argue with this logic because there is none...
seriously, you are all over the place. I think you tried too hard to make your case, and highly distorted I think I see what you are trying to say. you made one credible statement in that blanket results are given for all teams, no matter the offense or defense. to that point, why not index? wouldnt that be better? so, if 4th and 1 in some situation is x%, a below average offense should have something below x%. even if it is not completely more accurate whatever method you use, it is likely more accurate than using x%.
The more important thing to account for than the relative strength of the offense and defense contesting a 4th down is the relative strength of the teams contesting the game. Overwhelming favorites should be in the business of reducing variance.
Given the state of Minnesota's offense, a great way for Green Bay to blow that game was to pass on guaranteed points in the search of higher expected points or to turn the ball over on downs in its own territory instead of making a hapless team drive 80 yards.
I decided to take a look at how 3rd and 1 and 4th and 1 conversion rates vary by team strength, where I use the point spread as a proxy for team strength.
Link: 4th and 1 Conversion Rate and Team Strength
When stratified by point spread, conversion rates show variation, but only weakly. A 10 point underdog would only expect a 4% lower conversion rate on 4th and 1. So, the use of league average numbers when analyzing fourth down decisions is most likely still appropriate, even in cases where a team is significantly overmatched (or undermatched).