Vote for the GOAT.
Crowdsourcing might be the future of science. It has already been a fixture of fantasy sports and in sabermetrics, but it's usually just a collective vote or estimate about a player's future performance. What if there were a distributed data collection and analysis network for rich, robust football analytics?
When I did my Washington Post column on 'rebuilding' through the draft, I thought about testing the correlation between the number of picks and team wins. But then I thought, I bet someone's already done this. They did. The correlation found was 0.4. Of all the things that determine wins, including random luck, that's nothing to sneeze at.
Michael Lewis and Billy Beane talk Moneyball.
"It's a lot of things." Sure is.
"I present to you a bit of a look into the black box that is DVOA. I can't show you how I figure each baseline, or what the specific equation is to figure play value for each play." Huh? Why not? Is someone going to steal the magic formula?
Has the lockout caused an increase in injuries?
This about sums it up on the Tebow thing.
Does the pre-snap chess match forfeit the offense's first-move advantage?
Who's getting beat for sacks?
The Mike Smith 4th down call was the big news of the week. Here's the deal. Everyone agrees in a tight situation like that the coach should play the percentages. But almost no one knows what the percentages really are. It amazes me that not only are so-called experts willfully ignorant of the percentages, but when told them, dismiss them as irrelevant.
Gets it.
Doesn't get it.
Gets it.
Wants to get it. But doesn't.
Gets it.
Sorta' gets it.
Gets it.
Really doesn't get it!
Gets it.
Got it a long time ago.
We're making progress. A couple years ago, this would have been 99 to 1:
- Home
- Unlabelled
- Roundup 11/18/11
Roundup 11/18/11
By
Brian Burke
published on 11/19/2011
Subscribe to:
Post Comments (Atom)
Although you're calling it crowdsourcing here, Eric Raymond talked about it in "The Cathedral and the Bazaar" as Linus's law. The notion is if you open up data/code/whatever, that to someone across the collective Internet the problem is easy.
I even have an example from my blog. A reader named maqi just took a look at all SRS matrices, and noticed that the sum of any row in a SRS matrix is zero. The matrix is symmetric, so the columns sum to zero as well. That means there is a reduced matrix with a row of all zeros, and a matrix with a row of all zeros is singular. Hence, all SRS matrices are singular.
It's an easy observation, but someone has to make it. Give a problem to enough eyeballs, and solutions will come faster. That's what crowd sourcing was about, what Eric Raymond's seminal essay was about, and what I'm seeing happening in my modest steps into football analytics.
David, of Code and Football.
Scott Kacsmar gets close to making an important point. There is nothing precise about the WP model. It's close to the best stab in the dark that can be made, but without pairwise analysis of each defense and offense in the league, without including an accurate measure of relative team strengths, it doesn't represent the truth, and a good approximation. As such, I'd see a 5% change according to the WP model as questionable. That is, if one option is just 5% better than the other, then I'm not sure the resolution of the model is good enough to say, with confidence, that either decision is in fact superior.
A few points. First, the argument about the pairwise comparison of the two teams is a strawman. That's implicit in any quantitative football analysis. And never in my analysis did I claim that 0.47 > 0.42 therefore it was a good call.
I wrote, "The go for it option is worth, on net, a 0.47 WP. That's better than the 0.42 WP of the punt option, at least according to league-average percentages. " And after a brief discussion of the particulars, I called it a "smart call."
Plus, the two teams had played to a draw by that point, suggesting neither was over-matched that day.
Second, no one here has ever claimed that because one particular WP model says 'go' that's the pure absolute truth. Everything is an estimate, and as you say, it's just the best thing we've got at the moment.
5% is actually pretty big as these things go. Not questionable at all. But I understand and sympathize with your skepticism. It's well outside the uncertainty given the OT data I have. There are situations with less certainty--big leads of uncommon amounts, for example. But in OT, the score is always tied and field position is the only critical parameter. Consequently, the resolution is much better than 5%, at least when we're talking about the league baseline.
Additionally, the analysis I provided also calculated the break-even success probability for the conversion. Coaches, fans, whoever, are all free to guess whether or not ATL had a better than 64% shot to convert on 4th and inches.
And the final point is that, even if it were a low-resolution estimate, the decision was just as likely a 10% decision as it was a wash.
I thought that Tebow article was about the least fair thing written about him this week. Chock full of exaggerations and misrepresentations.
I think that it implies a good question, though. If there really isn't such thing as clutch, then why does Tebow appear so much worse earlier in the games? You don't want to discount the bad play, but you also don't want to discount the good.
My own working theory - not backed up with statistics - is that in his successful stretches, Denver's offensive style tends to change to a style that suits Tebow better. I think the staff is still learning about him. Plus, I think Fox believes in keeping it close even when the Broncos are ahead, which makes three-and-outs more common even as it burns the clock.
Still, the Jets game was a perfect example of critics' common complaints about Tebow. His last couple games weren't quite so exaggerated, and you also have to give the Jets' defense credit for this but still - if you look at WPA by drive, every single Denver offensive drive before the last drive had negative WPA, except for a +0.01 drive that got field position but ended in a punt. And then the last drive was worth +0.59 .
The ESPN one is hilarious! I originally thought it was a good decision. Then it didn't work and I changed mind. Wow! Really?!
I thought that Tebow article was about the least fair thing written about him this week ... If there really isn't such thing as clutch, then why does Tebow appear so much worse earlier in the games? ... the Jets game was a perfect example ... if you look at WPA by drive, every single Denver offensive drive before the last drive had negative WPA, except for a +0.01 drive that got field position but ended in a punt. And then the last drive was worth +0.59.
Suppose the Jets' on their drive before that had run the length of the field, burning the clock, ending with an extra sealing score as time ran out. What would Tebow's WPA on his "last, clutch drive" have been then? Negative.
This is one of the many logical problems with believing in clutch play. Nobody knows what the clutch plays are until *afterward* looking backward. If a player *during* the game doesn't know what plays will and won't be deemed "clutch" in the story of the game as told afterward, then how can he rise to be clutch in just those plays?
When Tebow got his last possession before that one by the Jets, he *had* to believe it was "clutch, vital" -- because if he had negative WPA *again*, giving the ball back to the Jets once more, they'd be in a position to seal the game right there.
Yet believing the situation was vital and clutch, he went three-and-out and gave the ball back to the Jets right at mid-field, in perfect position to seal the win.
How clutch was that? How was that not clutch *failure*?? The only way would be if he had a time machine telling him that the Jets would so screw up in turn. But without a time machine he couldn't know that until *afterward*.
It's the exact same thing with the most famous "clutch play" of all -- Joe Montana throwing "the catch" in the last seconds of the NFC Championship, immortalizing him as "Mr Clutch" forever. But everyone forgets he threw three picks before then, which is why they were behind in the last minute. Why wasn't throwing three picks in the Championship Game being "Mr Choke"?
If the Cowboys in their last possession had gotten one more first down they'd have won right there, and that would have been the story about Montana for that game: "Mr. Choke".
Montana *had* to have thought his last drive before that was probably going to be his last one. So if he was the world's greatest clutch player why did it end in failure -- as a follow-up to his three picks?
why does Tebow appear so much worse earlier in the games
Tebow's started all of, what, four games? He has to look better in either the early part or later part of each. The odds that he will mostly in one end or the other are *short*. ("The law of small numbers.")
"Why NFL Players Love A Coach Who Goes For It On Fourth Down, And Why They Hate Eric Mangini"
Because Mangini drills and quizes the players on all the probabilities and percentage tendencies of every possible situation, and they hate it.
Bill James, now working for the Red Sox, says knowledge of all this percentage stuff is for the coaches, GM, office people, etc., when building teams, designing strategies and calling plays, but never to be inflicted on the players (except in small easy-to-accept spoonfuls as may be useful).
At best, pro athletes as a group don't have the aptitude to understand probabilities; at worst, e.g., if the players start to get a message that games are largely determined by random chance rather than them giving 110% effort, all kinds of bad things can happen.
So true. The actual players have to live in a world where randomness doesn't exist in their minds. If they let themselves acknowledge how random their world was, they'd behave very differently.
I see a line on DVOA you there...does anyone really understand it? I know I don't.
Zirondelle
Crowd sourcing in the NFL = the closing betting line.
I have to say, I'm a little surprised at the love for Nate Jackson's piece, which read to me like a standard polemic against statistical analysis. While I agree it's probably wrong to drill players on all but the simplest concepts, his point seemed to be that statistical analysis generally is no substitute for trusting your gut:
But is the fourthdownulator a joke? No, no, it is not a joke. I just checked. It's very real. So real that our weekly roundtable consults it, or other similar -ulators, to discuss the sanity of a coaching decision. But statistics are dirty, lying liars, and trusting them to explain the reasons why teams win is a dangerous game.
But that won't stop us from trying. And arbitrary statistical analyses are not reserved for media and fans. Teams have detailed studies, too.
***
Because if Smith ever did consult a probability chart, he might get beat by an opposing coach who trusted his gut. Like the guy sitting at third base on a blackjack table and hitting his 12 with the dealer showing a 6. Idiot! But then he catches a 9, the dealer flips a 6 to reveal a 12, then pulls a king and busts. Nice hit, man!
The head coach must trust his instincts and hit against the probabilities from time to time if he wants to leave his mark on the game. Conservative, predictable coaches are a dime a dozen.
I'm particularly baffled by the last claim. Coaches who decide according to the probabilities are common in the NFL? Seriously?
Bigmouth, I too agree that as far as Jackson's article reads he appears not to understand what any of this stuff means. To actively use your gut instead of probabilities to make decisions is akin to a card-counter keeping count but then deciding that he fancies a shot at luck, but since those that can count cards understand why it works, they don't suffer from such stupid reasoning.
WTF? Nate Jackson is missing a few brain cells.
"Because if Smith ever did consult a probability chart, he might get beat by an opposing coach who trusted his gut. Like the guy sitting at third base on a blackjack table and hitting his 12 with the dealer showing a 6. Idiot! But then he catches a 9, the dealer flips a 6 to reveal a 12, then pulls a king and busts. Nice hit, man!
The head coach must trust his instincts and hit against the probabilities from time to time if he wants to leave his mark on the game. Conservative, predictable coaches are a dime a dozen."
Regarding the last hyperlink and, towards the end of the article, the author speaking about no-huddle/rapid-fire offenses at the NFL level being less effective than at the lower levels:
Obviously, a less talented offensive team would benefit less than a very talented offensive team by going to this style of play-calling. However, in this era of pre-snap confusion, I think the number of options in this pre-snap cat-and-mouse guessing game gets greatly reduced more so for defense than for the offense by going no-huddle/rapid-fire.
Which, to me, comes back to a "what does the data say?" question...so...
Brian: Are there numbers out there to see how defenses across the NFL over time have done when facing a "no-huddle" or rapid-snap concept (per the very last hyperlink) as compared to how they have done in other situations?
I realize that there are multiple factors to consider when weighing those numbers (i.e. a team up 14 at the end of a game is a lot more willing to give up short gains than a team up by 3) but I have to think that there are distinct advantages that tip the balance of strategic power, even at the pro level, to the offense if such a play-calling system is employed more than is typical.
Is this even a remotely answerable question with all of the noise inherent in this or is there a way to find "something" out in this realm of "does pace matter to an offense"?
This has been on my mind for some time now, especially after I read Bill Walsh's "Finding The Winning Edge" which, while "dated", had him predicting that no-huddle offenses would be used more as a full-time system in the future than they were at the time of the book's publishing.
One problem with analyzing the no-huddle offense would be that most of the data comes from situations where the defense is in prevent mode.