Sneak Peek at WP 2.0

I've just completed the development, validation, and testing of the next-generation Win Probability model. It took the better part of the past 6 months. Despite many heartaches and frustrating turns, I'm really thrilled with the results. But as excited as I am to have this new tool, I'm also somewhat humbled by how inadequate the original model is in some regards.

As a quick refresher the WP model tells us the chance that a team will win a game in progress as a function of the game state--score, time, down, distance...etc. Although it's certainly interesting to have a good idea of how likely your favorite team is to win, the model's usefulness goes far beyond that.

WP is the ultimate measure of utility in football. As Herm once reminded us all, You play to win the game! Hello!, and WP measures how close or far you are from that single-minded goal. Its elegance lies in its perfectly linear proportions. Having a 40% chance at winning is exactly twice as good as having a 20% chance at winning, and an 80% chance is twice as good as 40%. You get the idea.

That feature allows analysts to use the model as a decision support tool. Simply put, any decision can be assessed on the following basis: Do the thing that gives you the best chance of winning. That's hardly controversial. The tough part is figuring out what the relevant chances of winning are for the decision-maker's various options, and that's what the WP model does. Thankfully, once the model is created, only fifth grade arithmetic is required for some very practical applications of interest to team decision-makers and to fans alike.


The new edition of the model is superior in several important ways. It's more relevant to recent seasons. It's more accurate, especially in the endgame when good decisions matter most. It accounts for a larger number of variables. It offers greater precision. It's more honest in that it produces its own measure of uncertainty. It's based on fewer assumptions. It also preserves the best quality of the original model, which is that it is based on empirical observations rather than abstract models. In other words, it's a refection of what actually happens in actual NFL games played by real players led by real coaches. (An exception is the overtime model, because there have not yet been enough games under the new format for reliable estimates.)

From my point of view, it makes my life easier because it's constructed with a modular design. For example, when a new season of data is ready to be included I can update the core analysis output with a simple copy and paste of a single file. The code that sorts through the relevant data points is better written so it's easier to maintain and improve.

To show off my shiny new toy, I'll make this visualization tool available, at least for a limited time. It's one of three different graphical tools I created for testing and validation. The tools allowed me to view large swaths of output at a glance to detect any discontinuities or anomalies. It illustrates the model's internal consistency and coherence. I think it's also a compelling way to get a feel for how the concept of WP works and, assuming the model is largely valid, learn some things about the sport itself.

The visualization tool works in a similar way to the WP Calculator, but rather than produce one WP estimate for a given set of inputs, it displays a range of outputs across the entire field for all four downs. The overtime states are especially fun to play with. One last note--This demo is not the current version, so it has a few minor things that have since been tweaked.

  • Spread The Love
  • Digg This Post
  • Tweet This Post
  • Stumble This Post
  • Submit This Post To Delicious
  • Submit This Post To Reddit
  • Submit This Post To Mixx

7 Responses to “Sneak Peek at WP 2.0”

  1. Anonymous says:

    Fantastic. As always, thanks for all your hard work.

    Cheers,
    J

  2. Anonymous says:

    Brian

    Great -- can you give us a sense of what tools you used to develop the model. Have you coded it in Python or some other language?

    Best
    John

  3. Topher Doll says:

    Sweet! Can't wait to dig into this.

  4. Anonymous says:

    To echo previous comments, this is awesome.

    Any chance you might have a visualization with time remaining as the x-axis instead of field position?

  5. Brian Burke says:

    I've actually built 3 versions for testing:

    1. x=yard line, colors=downs
    2. x=time remaining, colors=score differences
    3. x=time remaining, colors=yard lines

    (y=WP for all 3)

  6. Anonymous says:

    What would be interesting if it could tell us the suggested play type (pass, run, punt, field goal) so I could get pissed off at my lousy coach when he does something different.

  7. Hugh Nightingale says:

    As it stands alone the graph of YTG vs WP for Downs is not terribly helpful. 1st Down is almost linear whilst successive Downs have an increasing hump around the 38 as you move within scoring range. To Anon above, I suggest you start by reading Brian's articles on 4th Down options if you have not already done so.

Leave a Reply