tag:blogger.com,1999:blog-38600807.post6204472722211783811..comments2024-10-28T16:59:39.881-04:00Comments on Advanced Football Analytics (formerly Advanced NFL Stats): Point / Counterpoint on Rodgers' ExtensionUnknownnoreply@blogger.comBlogger32125tag:blogger.com,1999:blog-38600807.post-23788533688019634152013-07-29T22:20:48.684-04:002013-07-29T22:20:48.684-04:00Brian is correct. Extending Rodgers @ $11 m a yr....Brian is correct. Extending Rodgers @ $11 m a yr. Is completely unrealistic. Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-54441936141101690232013-05-17T22:54:38.186-04:002013-05-17T22:54:38.186-04:00I upvote Steve's statistical explanation (Phil...I upvote Steve's statistical explanation (Phil's intuitive example is good as well). Although I'm not sure that "RBs are paid a lot more per unit of value (say EPA/G) than QBs" necessarily implies an inefficient market. Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-88241852385030981662013-05-16T12:33:39.713-04:002013-05-16T12:33:39.713-04:00Linear regression assumes that the X variable is k...Linear regression assumes that the X variable is known with certainty and the variance (aka error term, random fluctuations, measurement error, "shit happens") is all associated with the Y variable.<br /><br />In this case, we know his salary with certainty; he's going to get paid $21MM. His expected performance is something like 8 EPA/G based on what he will be paid, with uncertainty associated with the performance. The uncertainty is random variance in performance as well as teams or agents making mistakes regarding performance and paying a player the "wrong" amount of money (but salary is still locked in).<br /><br />On the flipside, the alternative is to use his performance (with no error) to predict what you should pay him. In this case, 11 EPA/G is worth about $10MM. The problem with this approach is that it removes the risk of performance variations. If you know exactly what you're getting then you don't have to pay extra to get it. There's also the implication that salary has a random variance term that it really doesn't.<br /><br />Linear regression also falls on its face a bit in cases like this since performance variation isn't normally distributed, and the variation is more likely to be something like lognormal (small chance of a great season, greater chance of something closer to expected).Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-49967936408496001072013-05-16T11:54:15.541-04:002013-05-16T11:54:15.541-04:00You're forgetting that all regression inferenc...You're forgetting that all regression inference is conditioned upon X (not Y). Thus, your statement: "According to this analysis, Rodgers would be worth something like $25M or more per season." is simply wrong. What that regression tells you is that "given an elite quarterback has a cap hit of $12M, we expect them to have about 5 EPA/G".<br /><br />Moral of the story, never make inference for X conditional on a value of Y!Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-76415682191507149112013-05-15T09:25:20.188-04:002013-05-15T09:25:20.188-04:00Just occurred to me. Maybe try repeated this exer...Just occurred to me. Maybe try repeated this exercise, but with a much smaller sample size: say, 20% of all the QB plays (take every fifth snap, for instance, and ignore the four in between). That should make it clear with interpretation is correct. One of them will be so wildly off, I'm guessing, that it will become obvious.Phil Birnbaumhttps://www.blogger.com/profile/03800617749001032996noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-14000631036948029582013-05-14T16:59:48.552-04:002013-05-14T16:59:48.552-04:00fellas,
both graphs are wrong. both LS fits are ...fellas,<br /><br />both graphs are wrong. both LS fits are wrong. you can always calculate a LS fit, but sometimes you shouldn't. when one performs statistical analysis there are assumptions made when certain techniques are applied. these assumptions cannot be ignored - doing so leads one to this apparent "problem".<br /><br />It is an ill-posed question. Please do not "vote" on which fit you "feel" is best.<br />Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-11078573274473681962013-05-13T14:49:28.048-04:002013-05-13T14:49:28.048-04:00Math version: The slope for the first graph is the...Math version: The slope for the first graph is the sample version of cov(EPA/G,caphit)/var(caphit). The slope in the second graph is cov(EPA/G,caphit)/v(caphit).<br /><br />If you put the two best-fit lines on the same graph, say graph 1 then you have these two slopes (the second one is the reciprocal since you flipped the axes):<br /><br />Brian 1's slope: cov(EPA/G,caphit)/var(caphit)<br />Brian 2's slope: var(caphit)/cov(EPA/G,caphit)<br /><br />If you do some algebra you can see that <br /><br />Brian 1's slope / Brian 2' slope = cov(EPA/G,caphit)^2 / (var(caphit)var(EPA/G))^2 = R^2 < 1<br /><br />So Brian 2 is always going to have a steeper slope. So what we see is that Brian 1's you should expect 0.33 EPA/G per $1 mil while Brain 2 thinks you should get a much higher return of about double that (because 1/R^2 ~= 2).<br /><br />Phil Birnbaum explained the intuition for this really well. But I think he's under the impression the graphs show the average EPA/G on the previous contract and the new contract cap hit but I'm under the impression is EPA/G on a given contract and avg cap hit under that contract. That flips the reasoning around from "graph 2 makes more sense" to "graph 1 makes more sense."<br /><br />You can think of graph 1 which predict EPA/G based on expectations about EPA/G (reflected in willingness to pay). Graph 2 predicts yours past expectation about EPA/G based on realized EPA/G which is about as uninteresting/hard to interpret as it sounds. Graph 1 corresponds to predicting how much you'll win in the lottery based on # of tickets and graph 2 predicts how many tickets you bought based on how much you won.<br /><br />Anyway, I wouldn't use either of these as a way of assessing if Rodgers is "worth it." The assumption embedded in graph 1 is that the QB market is efficient. The implicit model is that teams offer contracts of x $/year based on expectations about EPA/G that are unbiased but have error. But if that is true then then natural interpretation is that Rodger's $21 mil/year doesn't mean he is a good value at 11 EPA/G for just $21 million, it means that they don't expect him to continue to be an 11 EPA/G guy and are expecting something more like 8 EPA/G.<br /><br />In practice we know the NFL labor market is far from efficient. If it were efficient and we plotted value against caphit for running backs and for QBs we'd see the same slope. But RBs are paid a lot more per unit of value (say EPA/G) than QBs (right?). So we need another model of how contracts are drawn up and shouldn't assume salaries reflect unbiased expectations about performance.Stevehttps://www.blogger.com/profile/10230344931186858123noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-81652070210878693922013-05-12T17:51:26.412-04:002013-05-12T17:51:26.412-04:00For the anon that said WPA/G is a better measure o...For the anon that said WPA/G is a better measure of QB talent, I bet that EPA/G is a more reliable predictor of future WPA/G than past WPA/G, much like a team's offense outside of the red zone is a more reliable predictor of future red zone performance than past red zone performance.Jameshttps://www.blogger.com/profile/01838293735141324662noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-77457869778929863442013-05-11T21:43:37.426-04:002013-05-11T21:43:37.426-04:00Brian,
one can fit data where there are errors in ...Brian,<br />one can fit data where there are errors in x and y. Or one can simply look at the correlation and see that epa/g and salary do not have much to do with each other (it was below .5)<br /><br />you are wrong in thinking that salary has no error, it most certainly does (see your mark sanchez example, for example)<br /><br />As to the fitting, BIP is actually close to the answer, an approximation to that is indeed simply averaging the two fits.<br /><br />but like i said before, the biggest problem is using EPA/g.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-7619421225517081232013-05-11T16:30:12.844-04:002013-05-11T16:30:12.844-04:00I think the "correct" value is somewhere...I think the "correct" value is somewhere between the two ideas. It's worth remembering not all the QBs in those graphs are free agents today. Many are locked in at salaries below what they'd be paid if they could void their contracts and renegotiate them.BIPnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-67158308850866798672013-05-10T11:54:09.143-04:002013-05-10T11:54:09.143-04:00I normally hate these large contracts, but Rodgers...I normally hate these large contracts, but Rodgers is probably the one player where I wouldn't complain. It's certainly more sensible than Calvin's megadeal, simply because of positional value. Either way, I think both Brians would agree that Rodgers' contract make this Flacco contract look even worse. <br /><br />I really wonder if we have finally hit the peak, for how much teams are willing to pay players, or if Rodgers did indeed give the team a break. lumberjackhttp://occamskangaroo.blogspot.com/noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-57227669865085198562013-05-10T10:51:57.873-04:002013-05-10T10:51:57.873-04:00I'm thinking that, unless you know something a...I'm thinking that, unless you know something about front offices of teams using statistical analysis in their decision-making processes that I don't(you certainly do), that invalidates this, that QB's are primarily paid based on perceived market value and nothing else. I doubt if it'd matter if he was getting way overpaid based on relative value per EPA/G. If perceived market value doesn't match Value per EPA/G, a QB would be correct in passing up on a contract extension that was in line with perceived value per <br />EPA/G, and waiting for a better offer to come his way.<br /><br />How do your numbers take into account the value of denying opponents a good quality QB? In other terms, the EPA/G a QB contributes to his team denies that EPA/G to a potential opponent that might sign him. Do either of the above charts account for that somehow? I think it might be reasonable to add a little to the value of keeping someone like Aaron Rogers just off of that alone.<br /><br />Thanks for your excellent analysis on this.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-2592389022784847342013-05-09T17:18:23.558-04:002013-05-09T17:18:23.558-04:00Phil, Peter, X, and all the commenters---thanks. E...Phil, Peter, X, and all the commenters---thanks. Excellent insight. The truth is I saw this apparent paradox and it confused the heck out of me. I had some similar insights as in your comments above, and quant-extraordinaire Eugene Shen helped clarify things for me. But in all honesty, I don't know the 'right' answer, and was hoping smart folks like you guys would do all the hard thinking for me. It worked!<br /><br />Here are my thoughts:<br /><br />Another consideration is the normality of the data. Ideally the data comprise a Guassian distribution. EPA/G is very normal, but salary is not. It's very power law-ish. Just a few rich guys and lots of poor guys.<br /><br />OLS works for Guassian distributions because it minimizes the square of the errors. The square function is not chosen arbitrarily, but is derived directly from the Guassian function. So when the y (dependent) variable is non normal, OLS fits lose their special meaning and are not sacrosanct. <br /><br />There are error-minimization functions other than OLS that could be applied. For example, least absolute error produces a symmetrical fit, so that you get the same results regardless of how the axes are configured. Peter mentioned RMA (regressing to both axes) above.<br /><br />The cause/effect consideration is hard to untangle. I think it really is a matter of perspective... For example, from the player's perspective, if he reliably performs around 11 EPA/G (independent x), how much money can he expect in return on the FA market (dependent y). But from the team's perspective, if they buy $21M worth of QB on the FA market (independent x), how much performance can they expect (dependent y)?<br /><br />You might say (as I think someone above did), arbitrarily paying a person a lot of money does not "cause" him to play well at QB, as the Jets proved with Mark Sanchez (zing!). Case in point--if you paid me $20M to be an NFL QB, I'd average -100 EPA/G.<br /><br />...BUT I've left an important systematic linkage out of the discussion: The Market. Paying someone $20M to play QB doesn't cause someone to be skilled, but purchasing a $20M asset in a competitively priced market provides a systematic linkage from pay to performance. Like buying a race car...all other things being equal, paying $100k for a car rather than $50k for a car in a competitive market means I should expect a faster car. Money does "cause" performance, but only indirectly via the market process.<br /><br />So, my hunch right now is that Brian 1's analysis is the useful/meaningful one. Here's why: We know cost as a certainty with no error, but EPA/G is variable with a Guassian distribution. If we accept my argument on causation above, then performance should be the y (dependent) axis and pay should be the x (independent).<br /><br />Therefore, I'm thinking Rodgers is a bargain at $21M--assuming he continues as an 11 EPA/G guy. But perhaps GB is smartly regressing that a bit, saying he's a "true" 8 EPA/G guy going forward, which would make his value lie right on the regression line. <br /><br />Just my current opinion. Not 100%...Brian Burkehttps://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-22512715302514956552013-05-09T00:29:04.747-04:002013-05-09T00:29:04.747-04:00What caliber targets and protection does a $25+ mi...What caliber targets and protection does a $25+ million QB get under the current cap? tmknoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-61423497878914082492013-05-08T18:05:02.900-04:002013-05-08T18:05:02.900-04:00they are both wrong, because they use EPA/$ as the...they are both wrong, because they use EPA/$ as the criteria. :)<br /><br /><br />Clearly, the first one is minimizing the deviations in epa/g, whereas the second is minimizing the deviations in salary. These are not the same. <br /><br />Typically in data analysis your x axis is a "known" quantity like a timestamp, and you fit your measurements on the y axis to minimize the error of the ordinate.<br /><br /><br />Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-50565221669824954472013-05-08T14:16:51.356-04:002013-05-08T14:16:51.356-04:00Brian, you're eventually gonna tell us the ans...Brian, you're eventually gonna tell us the answer, right?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-81702055372601881862013-05-08T11:34:58.426-04:002013-05-08T11:34:58.426-04:00Both Brian 1 and Brian 2 are ignorant sluts! Actu...Both Brian 1 and Brian 2 are ignorant sluts! Actually, I'm not a skilled statistician, but I question the use of EPA as the measure of production. Teams care about wins more than they care about points. Points are a means to the end, but it seems to me that EPA doesn't fully account for a player's "clutch" potential, which is the ability to make a positive play in a high-leverage situation.<br /><br />My subjective view is that Rodgers is still incredibly valuable in this regard and therefore WPA/G would be the better measure of production.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-41364148137157027472013-05-08T01:50:26.449-04:002013-05-08T01:50:26.449-04:00OK, thought of an easier argument.
A regression s...OK, thought of an easier argument.<br /><br />A regression shows how a change in X implies a certain change in Y. NOT the other way around. For instance, buying a Chevrolet is associated with having one extra car in your driveway. But having one extra car in your driveway is NOT associated with one extra Chevrolet. (It's associated with, maybe, 0.1 extra Chevrolets, because there are other kinds of cars too.)<br /><br />Brian 1 says, "A team choosing to pay $21MM is associated with 11 extra points per game." But that doesn't work the other way -- it does NOT mean a player associated 11 extra points per game is associated with $21MM. So, throwing that extra "Rodgers" point on the graph is invalid. Only when a team chooses to pay $21MM for Rodgers can you do that. And that hasn't happened.<br /><br />Brian 2 says, "A player performing at the rate of 11 extra points per game is associated with the team having paid $11MM for him." That one is OK, because, yes, Rodgers does qualify as having performed at 11 extra points per game. (Technically, you can only say that's associated with *having been paid* $11MM, but you can argue further from there what that should mean for his future.)<br /><br />So, Brian 1 loses, under the "If X implies Y, it doesn't follow that Y implies X" rule.<br /><br />Phil Birnbaumhttps://www.blogger.com/profile/03800617749001032996noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-40220605144708230452013-05-08T00:21:31.317-04:002013-05-08T00:21:31.317-04:00BTW, I found a season's worth of baseball team...BTW, I found a season's worth of baseball team salary data (I don't know which year). Same kind of situation happens.<br /><br />For every $6.1 million a team spent, it won an extra game. But for every extra win a team had, it spent only $1.75 million extra.<br /><br />Same idea as the lottery example. <br /><br />Phil Birnbaumhttps://www.blogger.com/profile/03800617749001032996noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-72236666040580630992013-05-08T00:04:14.379-04:002013-05-08T00:04:14.379-04:00OK, here's what I think.
Brian 1's chart ...OK, here's what I think.<br /><br />Brian 1's chart asks the following question: if you know a team decided to pay a player $X, what does that tell us about the player's eventual performance? The chart shows that, for instance, when a player was paid $10 million, he returned about 5 EPA/g for his team.<br /><br />Brian 2's chart asks the following question: if you know a player returned X EPA/g, what does that tell you about what the team paid the player? The chart shows that, for instance, when a player returned 5 EPA/g, he was paid about $7 million.<br /><br />Those two numbers are different -- in one case, $10 million, and in the other case, $7 million. This is normal, because they're asking two different questions. <br /><br />---<br /><br />It's easier to see that they SHOULD be different with a more obvious example. Say, lottery tickets.<br /><br />Suppose there's a $1 Powerball-type lottery with a 50% payout rate. There are a bunch of different prizes, from $5 to $5 million. People buy as many tickets as they like.<br /><br />Brian 1 is answering: if you know a person bought 200,000 tickets, how much do you expect them to win? The answer: $100,000.<br /><br />Brian 2 is answering: if you know a person won $100,000, how many tickets do you expect they bought? The answer: I dunno, maybe, 10 or 20? Because, hardly anyone buys 200,000 tickets, but *someone* has to win the big prizes.<br /><br />So, Brian 1 finds that $100K is associated with 200K tickets. Brian 2 finds that $100K is associated with 20 tickets.<br /><br />They're both right, for their respective questions.<br /><br />The question we really want to ask is: given that Joe Blow won $100K with his portfolio of winning tickets ... how much should we pay for that portfolio of tickets for the next lottery? The answer is NOT $200,000 (Brian 1). The answer is $10 (Brian 2). <br /><br />---<br /><br />Now, what if teams KNOW that Aaron Rodgers is going to stay at 11 EPA/g? In that case, maybe they SHOULD pay him $21 million. But they don't know that. How do we know they don't know that? Because, look how far they were off on all the other QBs. They thought Matt Hasselbeck was as good as Tony Romo! They thought Tom Brady was about the same as Jay Cutler! Clearly, QB performance is unpredictable (probably mostly from luck). That means you have to regress Rodgers' past performance back to the mean, just like you have to regress lottery tickets back to the mean.<br /><br />-----<br /><br />Another way of putting it: Brian 1 is asking, how much would a team have to spend on a QB and *expect* 11 EPA/g? The answer to that one is, indeed, $21 million. But Brian 2 is asking, how much should a team spend for a player who *previously produced* 11 EPA/g? The answer to that one is, around $10 million, because he's probably not truly an 11 EPA/g player.<br /><br />Brian 2 is the question we actually want answered. <br /><br /><br /><br /><br /><br />Phil Birnbaumhttps://www.blogger.com/profile/03800617749001032996noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-54930471742987243402013-05-07T18:02:58.152-04:002013-05-07T18:02:58.152-04:00X, I'm not convinced by my own analysis either...X, I'm not convinced by my own analysis either...if nothing else I wanted to get my thoughts down so people were on the same page as to why the regression slope changed, if nothing else.<br /><br />I agree that the key to this is thinking about the uncertainties in the data, and what exactly we want to know at the end of the analysis.<br /><br />I guess what we could say is: "What is the empirical relationship between the 'value/production' of a QB and his 'cost', and how does Rodger's new contract compare?" Then we can make the assumptions that (a) EPA/G gives an estimate of a QB's 'value' with gaussian error and (b) cap hit gives an estimate of a QB's 'cost', either without error or with gaussian error (we have to assume gaussian errors to perform the standard fits since the sample size is small). If we assume cap hit gives 'cost' with no error, thats the analysis from Brian 1. If we are interested in the relationship of 'value/production' with some more nebulous 'cost' (that I can't really articulate at the moment) that cap hit is merely an estimate of, then we'd need to perform the fit minimizing the sum of squares error in both variables.<br /><br />If you are trying to wrap your head around these different regression slopes, think about it this way geometrically, looking at the first plot (where EPA/G on the y-axis): <br /><br />You get the Brian 1 result by minimizing the squared distance in the y direction from the regression line to each point. (Error only in EPA/G)<br /><br />You get the Brian 2 result by minimizing the squared distance in the x direction from the regression line to each point. (Error only in cap hit)<br /><br />You get a third result (which will fall between the other two, and is called the reduced major axis) by minimizing the squared distance perpendicular to the regression line from the regression line to each point. (Error in both variables.)<br /><br />Just my thoughts between meetings....<br /><br />-PeterPeternoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-35053612790749210492013-05-07T17:24:05.490-04:002013-05-07T17:24:05.490-04:00I'm not sure I agree with Peter's analysis...I'm not sure I agree with Peter's analysis. Both EPA/G and $/yr are known (subject to model uncertainty converting exactly known play results into EPA), but the "true" EPA/G and $/yr are both unknown. If we run imaginary seasons in our NFL simulator, players are not going to negotiate the same contracts each time. I don't really see a reason to treat the two variables differently in the fit.<br /><br />I do think we should try to think more carefully about what the uncertainties are for each of these data. Generally, any time you see data without errorbars, you should hear alarm bells in the back of your mind.Xnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-360775142949130052013-05-07T16:44:51.405-04:002013-05-07T16:44:51.405-04:00Don't get hung up on the 'above replacemen...Don't get hung up on the 'above replacement' concept. That's addressed by the intercept (constant term) in each plot.Brian Burkehttps://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-9144131915128239982013-05-07T16:28:03.159-04:002013-05-07T16:28:03.159-04:00"...The only difference from your chart above..."...The only difference from your chart above is that I swapped the vertical and horizontal axes..." Changing the dependent variable in the regression is really quite a bit more than that.<br /><br />"...which analytic approach is right?"<br />Can I say neither?<br /><br />One thing that they should be looking at is performance over replacement and cost over replacement. An eyeball average is 7.5M salary and 3EPA, so the Ravens are spending 13.5M of cap for 8EPA.<br /><br />The salient question is whether the Ravens could get more expected EPA for that 13.5M in salary cap elsewhere. (My gut says this is a good move for the Ravens. Would you give up Rodgers in exchange for David Garrard and Calvin Johnson?)<br /><br />Assuming, for the sake of discussion, that it makes sense to rehire Rodgers, another question to ask is whether they could have retained him for less, and it's pretty clear that Rodgers was in a strong negotiating position as a free agent coming off the Super Bowl win.<br /><br />Natenoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-24706476485313383492013-05-07T15:26:05.504-04:002013-05-07T15:26:05.504-04:00Brian 2's chart is correct, (as is said above,...Brian 2's chart is correct, (as is said above, $$ is a result of talent), but Brian 1's argument is correct. Rodgers is worth $30m.Elvisnoreply@blogger.com