Koko's 2009 Fantasy Report Card

Last August Advanced NFL Stats ventured into the fantasy world with its own player projections. But these projections weren’t what you might expect. Instead of trying to develop a complex and advanced projection system, the Koko system projected fantasy performance using the simplest rules reasonably possible.

The Koko projections, named for George Costanza’s simian nickname from his tenure at Kruger Industrial Smoothing, are what a monkey might guess given the typical regression rates from one season to the next. Koko is a (bad) rip-off of the Marcel baseball projections created by Tom Tango. Instead of competing with other projections, the intention is to establish a minimum baseline of predictive power against which other projections can be measured. Further, Koko tests whether other systems are really worth all the additional analysis, subjective and objective, that goes into them.

Koko looks at the separate regression rates for the various aspects of fantasy performance, such as touchdowns, yards, and turnovers. It estimates future performance based on those rates, on a per-game basis, then sums them for a total projection for each player. For example, TDs regress quicker than yards, so a player that did well because of gobs of TDs is likely to be overrated the following year. And a player who did well because of lots of yards but few TDs may be underrated. The 2009 projections can be found here: QB, RB, WR, TE, K, DEF. Scoring is based on 6 points for a TD, 1 pt for 20 rushing yds, 1 pt for 50 passing yds, -2 for a turnover, and -1 for a sack.

So how did Koko do? Are other systems, with their expert assessments, really any better than a simple regression? I compared Koko to the Yahoo fantasy projections to find out. I chose Yahoo because it’s the most popular, it’s free, and I happen to have its 2009 projections on hand.

Below are the Koko, Yahoo, and actual rankings for QBs in 2009.

PlayerAct RankKoko RankYahoo Rank
Drew Brees 113
Peyton Manning 235
Aaron Rodgers 372
Tom Brady 481
Matt Schaub 5127
Philip Rivers 668
Tony Romo 7212
Ben Roethlisberger 82014
Donovan McNabb 996
Kurt Warner 1044
Eli Manning 111318
Matt Ryan 122113
Jay Cutler 1359
Carson Palmer 141115
Kyle Orton 151724
Joe Flacco 162420
Jason Campbell 172319
David Garrard 181511
Matt Hasselbeck 192610
Kerry Collins 202825
Matt Cassel 211422
Shaun Hill 221027
Marc Bulger 233028
Trent Edwards 242516
Jake Delhomme 252226
Derek Anderson 261642
Chad Pennington 271817
JaMarcus Russell 282932

I measured the accuracy of each system in two different ways. I measured rank order accuracy by using the Spearman rank correlation coefficient for per-game performance. Additionally, I measured the total season accuracy by the correlation of raw total points.

The Spearman rank correlation for Koko was 0.72 and for Yahoo was 0.82.

The season point total correlation for Koko was 0.73 and for Yahoo was 0.79.

So Yahoo edged out Koko, at least for QBs. It’s about what I’d expect. In Koko’s defense, it does not know who switched teams, who added a free-agent WR, or who lost his all-pro starting left tackle to injury or free agency.

For running backs, Koko did not fare so well, but neither did Yahoo. It appears much more difficult to predict RB performance, and the difficulty appears to be related to projecting the number of carries a RB will get, because of sharing the backfield, injuries, rookie performance or other reasons. For RBs, the Spearman rank correlation for Koko was 0.20 and for Yahoo was 0.41.

The season point total correlation for Koko was 0.26 and for Yahoo was 0.56. That’s a big difference. Simpler isn’t always better.

So is there any point to Koko rankings? Sure. For starters, they provide a benchmark for comparing other projections. Second, if there is a big difference between what Koko projects and what another system projects, it’s worth examining why there is such a big difference. Over the next few days, I’ll roll out the 2010 projections.

Notes: I excluded the Vikings' QBs because the 2009 projections were made prior to Favre's return. I did not do comparisons of other positions because the process is labor intensive, and I doubt the results would be much different for receivers. 

  • Spread The Love
  • Digg This Post
  • Tweet This Post
  • Stumble This Post
  • Submit This Post To Delicious
  • Submit This Post To Reddit
  • Submit This Post To Mixx

9 Responses to “Koko's 2009 Fantasy Report Card”

  1. Anonymous says:

    I was waiting for these! I almost expected Koko to come through, sort of like Easterbrook's generic "team with better record wins" system.

    I don't suppose you have tons of free time to find the raw point correlations for yahoo WRs and TEs, as well as for the other major fantasy sites? :)

  2. Anonymous says:

    I'm kind of bummed Koko didn't do better :-(
    He/She is a cute little bugger.

    In all seriousness, I would love to see a simple system almost as effective as the complex ones these sites try to peddle.

  3. Anonymous says:

    Brian,
    This is off topic (sorry) but with the Darelle Revis holdout I have a question - can you please do some articles on probability of injury (and severity) by position and age?

    The Jets offered Revis a variety of contracts, one of them was (I think) "show up this year and we will give you an extra 5 million).

    What I would like to know is - what is the chance that a corner back gets injured, especially a long term / serious injury, as opposed to a jammed toe that causes him to miss a couple of games.

    - Jets Fan

  4. Andy says:

    Brian,

    Wouldn't it be better to compare the actual fantasy point totals instead of just the ranks? Two reasons:

    1. You're throwing away information by focusing on ranks, especially since the distribution is skewed. That is, identifying someone as 20th who is actually 30th doesn't really matter because the talent level is typically very flat between 20 to 30. So messing up that ranking is not that big a deal, at least compared to messing up the rankings of the top 10 QBs.

    2. A related point: the key to good fantasy drafting is value above replacement, not just having the right ranking. To truly compare the usefulness of systems in drafting, you need to know how the systems vary in what we actually care about, ie, VORP.

  5. Andy says:

    Follow-up: Apparently, I have reading comprehension problems, since you clearly said you used total fantasy points in calculating the correlations. Nevermind.

    Although it would be interesting to calculate the correlation for the 18 or so typically drafted fantasy QBs. It doesn't really matter if you miss big on the guys no one drafts and no one picks up.

  6. Anonymous says:

    Andy, VORP becomes more valuable if you switch to Auction Drafting. Let the FREE MARKET live!!

    Auction drafts involve so much more strategy and gamesmanship. And if you find Brian's work interesting it will inspire you to do your own VORP calculations.

  7. Anonymous says:

    Why is Brett Favre not on the list?

  8. Brian Burke says:

    He was retired and not on a roster at the time the projections were published.

  9. Unknown says:

    It would be nice if your Koko system could be improved with factors such as:

    Rank of opponent(s)
    Weather predictions factoring scores that week (tom brady in the snow sucks for FF).
    etc..

Leave a Reply

Note: Only a member of this blog may post a comment.