Maybe June should be "other sport month" here at Advanced NFL Stats. Except for Albert Haynesworth's principled, valiant stand against being forced to play defensive tackle a foot and a half to the right from where he is accustomed, there's not much going on in the NFL. Fortunately, there's plenty of other sports going on, including the world's biggest events in soccer and tennis.
Soccer and tennis offer two of the best examples of simple two-strategy zero-sum game theory. Soccer offers us the penalty kick, when a player kicks to the left or right extreme side of the goal so hard that the goalkeeper must simultaneously guess a direction to lunge. Tennis gives us the serve, where the server aims for either the extreme forehand or backhand side of his opponent's service box. Both examples give us the opportunity to examine how well experts are able to approximate the optimum strategy mix.
In any two-player game with two strategy choices, and there is no obvious "dominant" strategy that is always preferred, there is an optimum mix of strategies that will guarantee a minimum long-term payoff. This is known as the minimax solution, and in this case it's also a Nash equilibrium. If both players are playing their minimax strategy mix, neither player has any incentive to change their mix. At this equilibrium, the average payoffs for each strategy will be equal.
In cases like modern NFL football, there is clearly an imbalance in payoffs between two strategies. Passing, in most situations, has a significantly higher payoff than running. But in cases like soccer and tennis, where the payoffs are much easier for the players to measure, the actual strategy mixes differ from the theoretically optimum mixes by very small margins. Players either win the contest at hand (the kick or the point) or not. By comparison, football payoffs are very complex. They're a function of down, to-go distance, yards gained or lost, and original line of scrimmage, not to mention the possibility of a turnover.
In 1998, Mark Wooder and John Walters examined serve strategy in men's tennis matches at Wimbledon. Servers target either the extreme forehand or extreme backhand side of their opponent’s service box. Returners strategies are hard to discern. They can position themselves at any point between the extremes, either splitting the difference or guessing to one side or the other.
The payoffs were whether the server ultimately won the point or not. Although a lot can happen between the serve and the end of a point in tennis, the game can be “collapsed.” In essence, the payoff becomes the probability of winning the ensuing point. If tennis players are playing at the minimax, the average payoffs of serving to both the forehand and backhand side of the courts should be approximately equal.
Wooder and Walters (defined a “game” as an entire match divided into deuce-court serves and ad-court serves. They found that the of the 40 “games” they analysed, the vast majority were not significantly different from the minimax. Using the Pearson statistic and the Kolmogorov test (statistical methods for estimating how much a data set differs from the expected), they concluded that tennis players are able to intuitively approximate the minimax equilibrium.
In a 2003 study, economist Ignacio Palacios-Huerta studied penalty kicks from the top English, Spanish, and Italian professional soccer leagues. He found that there was a difference in strategy choices according to the dominant foot of a kicker. For example, a left-footed kicker is stronger kicking to his left, and kicks more frequently to his dominant side. Palacios-Huerta divided the sample of kicks into dominant side and non-dominant side kicks. Penalty kicks are more informative than tennis serves because the defender’s (the goalkeeper’s) strategy choice can be directly observed. He calculated the optimum equilibrium mix between left and right (for both kickers and goalies). Using similar statistical tests to Wooder and Walters, Palacios-Huerta also found that soccer players also play minimax.
In 2004, researchers Shih-Hsun Hsu, Chen-Ying Huang, and Cheng-Tao Tang studied a larger data set of tennis serves than Walker and Wooders, including men’s, women’s, and juniors’ matches. Their findings were more mixed. Although they found about the same minor deviations from the minimax strategy mixes, they used more stringent statistical tests and concluded that the players deviated from the minimax more often than Walker and Wooders’ study. Further, they found that certain strategies that followed a “rule of thumb” matched the players' actual strategies better than a pure minimax solution. They were saying that experts don’t really try to play minimax strategy mixes, but instead they follow simple rules that can approximate the theoretical minimax.
Ofer Azar and Michael Bar-Eli studied penalty kicks from the Israeli soccer league and found that players' strategies do not deviate significantly from the minimax. Their study is notable because it accounts for the Neeskens effect, a neat story in itself.
Traditionally, penalty kicks are kicked to either extreme side of the goal, forcing the goalkeeper to guess which direction to lunge to defend the goal. Then, in the 1974 World Cup final between Germany and the Netherlands, Johan Neeskens shocked the soccer world (and the opposing goalkeeper) by kicking a penalty kick straight into the middle of the net. No matter which way the goalie lunged, his kick would likely score. Within two years following Neesken’s kick, penalty kick success rates in international play increased 11%.
Azar and Bar-Eli modeled the penalty kick as a 3x3 game instead of a 2x2 game, taking into account the “center” strategy. They compare the Nash Equilibrium to other strategy options, such as “probability matching.” They conclude that the proposition that people can find the optimum solution to such a complex game is strong support of experts' ability to play at the optimum strategy mix.
There are additional soccer studies. Chiappori, Levitt (of Freakonomics fame), and Groseclose conducted a study of penalty kicks in 2002 and found that players did not deviate from the minimax significantly. (Levitt also co-authored a similar study on MLB baseball and NFL football, and found significant deviations. However, the study of such sports is drastically more complex than left/right and goal/no goal, and his study failed to capture the utilities of the outcomes properly.)
Another notable soccer study was done by GianCarlo Moschini in 2004. His study looked at goals in regular play rather than from penalty kicks. He also found support for the notion that players can play at the minimax equilibrium, but he also admits his study has “low power” to discern equilibrium play from non-equilibrium play. He suggests goalies might be shading too far toward the near side, for example.
Keep in mind what these studies are trying to find out. The question isn’t whether or not the minimax is the optimum. That is a mathematically proven truth—there’s no way around it. The question is whether or not humans can actually zero-in on the optimum, or if we’re even trying to.
Personally, I think this debate is silly. No one thinks the human brain can explicitly solve the math needed. And it should be no surprise that people might use simple rules to approximate the optimum. The mental costs involved in even attempting to derive mathematical perfection probably outweigh the added benefit above simple approximation. To know the optimum strategy mix in a simple 2x2 game, much less actually execute it, a player would first need to have perfect recall of lots of data. Then, he would need to solve for the intersection of two simultaneous linear equations. Take the typical football run-pass “game” below:
yPASS D = 4 - 7x
To find the minimax strategy mix, a player would need to solve those two equations and then find the x value of the intercept. Does anyone really expect someone to be able to do that in his head? Of course, not. But humans have an uncanny ability to estimate the answer using shortcuts. I used to do it all the time, traveling near the speed of sound.
Let’s say I’m flying in my F-18 and I see a MiG-29 I want to intercept. I want to close in on him as quickly as possible and either chase him off or gun him full of holes. I don’t know the MiG’s x or y (or z) coordinates, or his velocity or anything else. I can just see him with my eyes traveling across my canopy (windscreen). What I need to do is plot an intercept of two linear paths, a task no different in mathematical terms than solving for a mixed strategy Nash Equilibrium.
I was never very good at math under G forces, so what I’d do is maneuver to point the nose of my own aircraft out in front of the MiG’s flight path. This is known as ‘lead’ pursuit. As I close in, the MiG will either appear to me to drift forward or drift backward on my canopy. If it’s drifting forward, it means I haven’t pointed my aircraft far enough out in front of the MiG’s flight path. If it’s drifting aft, it means I'm pointed too far out in front of his nose. I can make a correction, note the how the MiG’s apparent drift changes, and re-correct until the drift stops and the MiG just appears to get bigger and bigger. At that point, I have zeroed-in on the solution to a complex set of simultaneous equations--intuitively, without any math.
In fact, the same calculations are required on the football field. A receiver maneuvering to catch a deep pass will adjust his route until the apparent drift of the ball is canceled out. A defender pursuing a ball carrier in the open field will make the same mental calculations to take the best angle.
For decades, psychologists have been puzzled by how baseball (or cricket) outfielders know where to go to catch a fly balls. Experiments have shown that they perform the same intuitive math that fighter pilots use. If you have outfielders watch a rising hit and ask him to guess where on the field it will land, they do very poorly. But if you allow the outfielder to maneuver, he’ll zero-in on the ball easily, and arrive just in time to make the catch. They perform what researchers call “optical acceleration cancellation."
You don’t have to be chasing MiG-29s or running backs or fly balls to do intuitive algebra. Chasing your runaway toddler at the playground or merging into traffic on the freeway require the same mechanics. I mention these examples just to suggest that humans are indeed capable of doing intuitive math similar to that needed to play at the minimax. Our ability, however, depends on the richness of the feedback available.
In a dogfight or on the athletic field, the feedback is immediate and evident. Real-world deviation from the minimax strategy mix can be due to a lack of robust feedback. Perhaps the deviation isn’t due to a flaw in the human intuition process but with the availability and relative complexity of the information needed to make optimum decisions.
In soccer penalty kicks, the feedback is very simple—either you score a goal or you don’t. In football, the feedback is very complex—a function of gain, down, distance, field position, turnovers etc. The simpler the feedback, the closer to the optimum we can expect to be. But when the utility function is excessively complex, people will (often wisely) fall back on tradition and convention. My point isn’t that chasing things and strategy mixes are identical problems, and the human brain can solve them in identical ways. My point is, given rich enough feedback, the human brain is capable of amazing feats, things we take for granted every day in sports and beyond.
re: the cricket ball, here's Douglas Adams's take on it:
"Compare that to somebody who tosses a cricket ball at you. You can sit and watch it and say, 'It's going at 17 degrees'; start to work it out on paper, do some calculus, etc. and about a week after the ball's whizzed past you, you may have figured out where it's going to be and how to catch it. On the other hand, you can simply put your hand out and let the ball drop into it, because we have all kinds of faculties built into us, just below the conscious level, able to do all kinds of complex integrations of all kinds of complex phenomena which therefore enables us to say, 'Oh look, there's a ball coming; catch it!'"
http://www.biota.org/people/douglasadams/
I'm not sure why you say the debate is silly. The question isn't really if people can do the math in their head; as you say, it's pretty obvious they don't. The question is actually, can people perform optimally, or close to it, if we know what optimum is? And that seems worthwhile, because if the answer is no we can start asking why not and how can we fix it. Isn't that what your fourth down work has been all about?
Alex-Fair point. I would say that in all the studies people perform close amazingly close enough to it, given proper information. You have to read the papers and how they cite each other to get a feel for the, "Yes they do," "No they don't," "Yes, they do too, Nah nah nah nah nah!" flavor. Players are so close to the NE that the cost of additional refinement probably exceeds the benefit. It's not worth the trade-off in time and mental energy. None of the papers even mentions that consideration.
These research papers (and the debate at their core) are not really about the sports they analyze. They're about the human capacity for rational decisions.
I'm (obviously) all for finding an edge with advanced analysis, but my real complaint is that economists are extrapolating findings from arbitrary games (like soccer, football, tennis, and others) into real economic interactions in the real world. The real world contains infinitely greater uncertainty, and infinitely more complex utility functions that vary from individual to individual.
"The question isn’t whether or not the minimax is the optimum. That is a mathematically proven truth—there’s no way around it."
... Kindof. It's only an optimum if both you and your opponent are on equal information footing - that is, neither of you has any idea what the other will do - because that, of course, is the foundation of the game anyway.
There's always an interesting question as to how long should a player remain at a minimax strategy even in the face of evidence that the opponent is an idiot.
"Personally, I think this debate is silly. No one thinks the human brain can explicitly solve the math needed."
I think it's even sillier than that - as you've pointed out often, at minimax, when you're really playing an equal-information zero-sum game, the two options result in equal success. That, I think, is what humans are zeroing in on: if a goalkeeper starts to lunge to his right a bit more often, he'll notice that he's getting beat to his left more often, and correct, slowly.
The fact that you can get to a minimax solution by simply applying "do what works more often" suggests that all you really need is someone trying to maximize their winnings, and enough time for statistics to win out.
In fact, blind adherence to minimax won't do as well as "do what works more often" in a repeated game environment if many of the opponents play clearly inferior strategies.
"In soccer penalty kicks, the feedback is very simple—either you score a goal or you don’t. "
You're being a bit too simplistic here. The feedback isn't just the result - it's the result plus the input. If the inputs don't span the full phase space available, the players could easily happily sit at some point other than the minimax, because they don't know there's an option other than the ones they've experienced.
The Neeskens story is a good example of that; it simply wasn't done before, so the previous solution wasn't the minimax of the 'new' game - the players likely took time to adapt to how often to attempt the new strategy, because the other players may or may not know about it.
Isnt an interesting point the fact that given the small sample size for each individual goalkeeper or penalty taker the game theroy optimum emerges. In the English football league each team takes only 5.5 penalty kicks in a 38 game season. So there is very little feedback to adjust your technique. Of course you can practice but not against your opponent. I doubt any goalkeeper face the same penalty taker more than twice in their career although I think both goalies and penalty takers study their opponents.
In the world cup when penalty shootouts take place you have people taking penalties who never take them normally. While the success rate drops as you go through a penalty shootout is this because of lack of skill of the penalty taker or because they become too predictable or are they almost the same thing?
There was a book published a few years ago called "how to take a penalty" which looked at various mathematical things to do with sport sincluding the penalty game theory issue. They suggested that you work out before hand that you will kick left x% of the time and right the rest of the time and then create some randomisation element such as the clock on the scoreboard or the number of the opponent nearest you as you walk over to take the penalty and use that to decide which way to kick.
James
More recently than Neeskens a further penalty strategy has been developed by the taker of waiting for the goalkeeper reveal their strategy before choosing which side to shoot.
This is done by hesitating in the run up to the kick. It is technically demanding but the approach has been successful enough to cause FIFA to outlaw such a run up (although not the strategy itself).
EdBed
The penalty taking rules do evolve slightly over the years. Once upon a time (when I was growing up and playing soccer competitively), the goalie was not allowed to move before the ball was kicked. So after he had picked his position, that was it.
Fifa then changed the rules so that the goaltender could move around. As a result, if the goalie saw something in the penalty taker's run up, then he could take advantage of it.
A few penalty takers then ran up and then stopped, thus sending the goalie diving and then just kicking it in wherever the goalie wasn't. FIFA has outlawed this.
This has evolved further. On my blog yesterday, I wrote about Japan v Paraguay's penalties. Against Japan, Paraguay's penalty takers saw that Japan's goalie was committing, so they began running up slower and slower (seriously, watch the penalties) and eventually Japan's goalie would commit. They were probably helped by the fact that Japan missed their 3rd penalty, which added pressure on the goalie to save one.
Even so, the approach was so slow from the 5th Paraguayan penalty taker that if the Japanese goalie hadn't committed, there wouldn't have been much momentum to help with the force of the shot.
Good post overall Brian. It's nice to see other non-academics who spend time on the SSRN. It's a shame more pro sports owners don't hire guys like us.
btw, you linked to me on Twitter a few days ago but linked directly to a file, not to the post.
kinda...harsh?
Football Polemics, I noticed Paraguay's run before the kicks slowed down as well. The last player practically walked up to the ball before he kicked it.
Sorry, didn't mean to offend. It's very hard to fit everything you want to say/link on Twitter.
For those interested in some great soccer analysis, I heartily recommend Football Polemics.
thanks Brian, that's kinda of you.
I really should spellcheck.
I couldn't help but notice that ESPN now has something called the "Soccer Power Index" by which they are making binary logistic type predictions for the outcome of World Cup games. I'm not sure if the methodology is exactly like the ones used for the NFL on the New York Times blog, but I thought the application appeared very similar. The guy who runs it is named Nate Silver, who is popular for his development of baseball and political statistics. I figured that readers of this site would find it as interesting as I did given the MIT Sloan debate and the ever increasing openness of the public towards the consumption of advanced statistical analysis in sports.
Interesting article. But there are far more options with tennis serves than just serving it to the forehand or backhand. The server can put different kinds of spin on the ball, different speeds, or even serve it straight at the returner to hand cuff them on their return (the first article looked at that but the 2nd didn't). There also is usually a change in strategy on the 2nd serve if the server misses their 1st serve beyond the "I served it left last time so I should serve it right this time."
Those 2 groups got their point across, but really dumbed down the situation to do it. It would be like evaluating hockey shoot-outs on only the criteria if the player shot left or right.
I love this blog. I admit I hadn't read it much since football ended, but I love these articles.
Excellent comment!
I have to agree with the small sample size comment above. Unless you are going to lump all players together -- treat them as identically skilled at all types of kicks or all types of saves -- it is hard to imagine getting anything like a good sample size.