The Office Pool 3

You might be wondering why I'm interested in NFL pick 'em pools in the middle of March Madness. Well, there are already plenty of statistical analyses on the NCAA basketball tournament. Here are a couple of sites to get your bracket filled out scientifically.

But for now, I've got the luxury of time before the NFL season starts, or even draft season, when I can think through these things. In the last post I looked at using the point spreads as a baseline for picking winners. I looked at the accuracy with which the point spread correctly favored straight-up winners. Over the past six seasons, the spread was accurate about 67% of the time, and no single week showed any statistically significant deviation from the overall average. In other words, the spread is no more or less accurate in early or late weeks than throughout the season.

The reason I analyzed spread accuracy by week was because when you're behind in a pick 'em pool, you'll probably have to gamble on some upsets in order to catch up. I wanted to know if it was to your benefit to go against spread favorites in any particular week. The answer is no.

But what about spread amounts? It certainly makes sense that games with +1 or -1 spreads will be less predictable than games with +14 or -14 spreads. But how much less predictable? Is there a point of inflexion when it never makes sense to go against the spread? Are there situations when it's basically a toss up and the spread is no more accurate the flip of a coin?

Below are the answers. The graph shows the accuracy of the spread in terms of predicting the straight-up winner for each spread amount. Data is from all regular season games from 2002-2007.


As expected, the spread predicts winners more accurately with increasing spread amounts. Note the fan-shaped dispersion of the data points. Some spread amounts are far less common than others. For example an 8-point spread is less common than a 7-point spread. At the least common spread amounts, there are fewer cases and therefore a wider range of accuracies.

Also notice how games with spreads less than 3 points are no better than 50% accurate. I wouldn't expect much better than 50% - 55%, but less than 50% is surprising.

The 'home underdog' phenomenon has been established in previous research. This may be due to many observers underestimating the home field advantage due to weather conditions late in the season. But whatever the reason, the home underdog effect clearly exists. The graph below breaks up the spreads into home underdogs and home favorites.


Notice the 40% accuracy of low-spread home underdog games. When the spread is below +2.5 points, the home underdog is not only likely to cover, but will probably win.

It makes sense to pick upsets in low-spread home underdog games. However, there have only been 70 such cases in the past 6 years, averaging about 11 games per year. If you go for the underdog in all 11, on average that gives you a 2-game edge over someone picking all favorites. It's not nearly enough of an advantage to guarantee you bragging rights around the office, but every little bit helps.

And if you need to catch up in the later weeks, games with low spreads and home favorites aren't much better than 50%. Going with upsets in such games would make sense because it's not a bad gamble and your leading opponent might be playing it safe by picking all favorites.

You'll want to be very careful picking against the favorite in games with spreads more than 3.5 points, for either home underdog or home favorite games. The odds against you climb rapidly beyond that point.

The Office Pool 2

When picking winners in an office pool, I'd guess that most people start with the point spreads, or at least look at the records of each opponent, when making their predictions. Most people have some sort of baseline even if it's not Sagarin, DVOA, or the regression-based predictions on this site.

So I thought it would be interesting to look at the spreads, and how often they're correct in identifying winners. If someone needs to correctly pick a few upsets to win a pool, it might be good to know that some weeks are less predictable than others. You'd ideally want to pick upsets in weeks where the spread is less accurate.

In this installment I'll look at how well the spread does in picking winners by week. My theory was that the spread would be relatively less accurate in the early weeks of the the season when there is less information about team performance. There may also be a high degree of bias towards teams expected to be strong in the pre-season. Week 17 may be inaccurate too, due to the uncertainty of some playoff-bound teams resting their better players. Additionally, late season weather may also contribute to higher uncertainty and less predictability.

Using point spread data from the 2002-2007 seasons obtained here, I analyzed how often the spread was correct. Overall, the point spread favorites win 66.2% of the time. Weekly accuracy ranges from 59.0% in weeks 4 and 9 to 72.6% in week 12. The graph and table below list the weekly averages.

Week	Accuracy
1	63.8%
2	62.5%
3	69.8%
4	59.0%
5	71.3%
6	69.5%
7	61.4%
8	64.6%
9	59.0%
10	61.6%
11	76.3%
12	72.6%
13	62.8%
14	71.6%
15	69.5%
16	63.5%
17	65.3%
Total	66.2%

Although there appears to be substantial differences between some weeks, they are most likely random. The only statistically significant difference between any one week and the season average of 66.2% was week 12 with p=0.04. However, there are 17 weeks, so we should not be surprised to see a week or even two appear significant when there really is no systematic connection (a type I error).

The bottom line here is that no week can be viewed as particularly favorable for picking upsets. If you're behind in your office pool toward the end of the season and need to pick some upsets to make up ground, one week is as good as any other to start getting aggressive.

Next, I'll look at point spreads from a different angle and see how accurate they are at picking winners by the size of the spread. I'll also break it down into two types of games, home-underdogs and home-favorites, to see if there are any inefficiencies in how the spread accounts for home field advantage.

Beating the Season Over-Under Follow-Up

Before the 2007 season began, I proposed a system that appeared to be able to systematically beat Las Vegas over-under lines on team wins. For the 2007 season, the results are in. The system's record would have been 8 correct, 3 incorrect, with 1 push (73% correct).

The system itself consists of two rules:

1. For teams predicted to win 9.5 games or more, bet on the under.
2. For teams predicted to win 6.5 games or less, bet on the over.

In other words, bet on mediocrity.

Historically, the system would have been 70% correct over the previous two years, and slightly over 58% correct over a 10 year period between 1996 and 2005. By betting 'over' on teams predicted to win 6 or less, instead of 6.5 or less, the overall rate would have improved to 61%.

Here is how the system fared this year. Over-under lines were taken from bodog.com on 6/30/07.

Unders	Line	Actual	Result
SD	11	11	-
BAL	9.5	5	W
CHI	10.5	7	W
IND	11	12	L
NE	11.5	16	L
DEN	9.5	7	W
PHI	9.5	8	W
CIN	9.5	7	W
Overs	Line	Actual	Result
DET	6	7	W
CLE	6	10	W
HOU	6.5	8	W
BUF	6.5	7	W
OAK	5	4	L

One thing I learned watching the results develop over the season is that the system is more successful the earlier in the year that the over-under lines are taken. Early in the year, long before training camps open, there is the least amount of information and the more likely it is that over-under lines are set based on the previous season's results. This is when uncertainty would be greatest, and perhaps when overconfidence is accordingly great. When looking for records of the over-under lines, I noticed that the earlier the line, the more confident it was--i.e. the further from 8 wins teams were predicted to win.

As player movements, retirements, injuries, team schedules, and pre-season games come into focus, over-under lines move to reflect the new information. Uncertainty is reduced, and the overconfidence would be mitigated. It might therefore better to place bets earlier rather than later in the pre-season, capitalizing on maximum uncertainty.

Why the System Works

The system is based on three principles:

1. The NFL season is extremely difficult to predict.
2. Regression to the mean is very strong in the NFL.
3. People are overconfident in their ability to predict team wins.

Humans, including NFL fans and gamblers, are susceptible to cognitive biases, and I believe these biases affect prediction markets. Cognitive (or heuristic) biases are systematic errors in judgment made in certain situations. These biases exist probably because they benefited early humans in their efforts to adapt and survive. For example, the "overconfidence effect" may bias people toward action rather than passivity in the face of a challenge.

But in a prediction situation, overconfidence is counterproductive. People who believe their prediction abilities are better than they truly are would take excessive risks. In one survey 80% of respondents claimed they were in the top 30% in driving skills. Similarly, it's likely that most gamblers also believe that they have some special ability or intuition to predict outcomes--otherwise they wouldn't be gambling.

Hindsight bias, also known as the "I knew it all along" effect, may also be a factor in fooling people into forgetting how poor their predictions really are. Ask yourself how well you thought New England would do this year. Be honest. According to the betting lines, half of all people thought they'd win 11 games or fewer. Remember how Randy Moss was supposed to be a "cancer?" How about the Bears? Half of bettors believed Chicago would win at least 11 games this year. But as the Bears floundered this year, many people forgot just how good they were expected to be--just on their defense alone. Most fans, including tv commentators, forget how uncertain things were before the first snap of the season. They say "it was obvious the Patriots would be unbeatable. They have Randy Moss!" or "obviously the Bears fell flat, Rex Grossman is terrible." I know that's my first instinct.

People also tend to remember correct predictions and forget incorrect ones--both by themselves and others. Today I noticed the cover page of an obscure football "prospectus" book I bought in August. It trumpeted, "Last year, we correctly predicted player X would have a breakout year! We said team Y would return to the playoffs!" That a book of over 300 pages of NFL predictions made four or five correct guesses is not an accomplishment. But it's the correct ones that are remembered.

Many people are also unable to grasp the concepts of randomness or luck. They discount regression to the mean because they expect extreme performances to continue. In the NFL, a team that finishes with a 14-2 record was probably a "fundamentally" 12-4 team that got lucky in a couple games. Fans and bettors may expect much of that 14-2 performance to carry over into the next season, thinking that 11 or 12 wins is a safe bet. But in reality, the team was a "12-win" team the previous season, so the chance of repeating such a successful year would be lower than expected. This effect is compounded when the same phenomenon affects a team's opponents. For example, part of the reason the Ravens and Bengals did not reach their over-under expectations is because division rival Cleveland improved from extremely poor performance.

This field of heuristics and decision-making fascinates me. I'm sure there is a lot of money to be made in not just betting markets, but equity markets. The one thing about sports though, is that it's relatively easy to analyze statistically. And no. I'm sorry to report I did not put my money where my mouth is.

How to Beat the Over-Under

Note: A follow-up to the results of the 2007 season can be found here.

Just in time for the beginning of the season, I think we've cracked the code on beating the Vegas over-under lines. Prompted by a recent comment from Tarr, I did some additional analysis. Although this is not a gambling site, I realize a lot of people are interested in it, and many people would at least be interested in how to beat conventional wisdom.

I only had the last two seasons of data available, and although I'd prefer to have more, the results are convincing.

In my past several months of researching NFL win-loss records, I've noted two overwhelming phenomena:

1. The NFL is impossible to predict before the season starts. And,
2. Regression to the mean rules the day.

In practical terms, expert predictions, including the consensus Las Vegas over-under predictions, are bad primarily because they underestimate the annual tendency for bad teams to improve their records and good teams to worsen their records. Expert predictions stink--that's the good news.

The bad news is our own predictions usually stink worse. Sometimes people get lucky and outguess Vegas or the experts, but over time, luck will catch up to you. So the trick is to take advantage of Vegas' flaws by applying the two lessons above, all the while ignoring our own predictions. Here's how:

1. Take the under on teams predicted to finish with 9.5 wins or more.
2. Take the over on teams predicted to finish with 6.5 wins or less.

And here's why:

1. Of the 18 teams predicted to have 9.5 wins or more, 13 finished the season under, and 5 finished over.
2. Of the 15 teams predicted to have 6.5 wins or fewer, 11 finished the season over, and 5 finished under.

Using these two rules, the '9.5 or more' teams would have yielded a net of 13-5=8 winning bets, and the '6.5 wins or fewer' teams would have yielded 11-4=7 winning bets.

I am not a gambler at all, so forgive me if I mess this up. In the last 2 years, if you bet $110 on each game according to these rules, you'd have placed 18+15=33 bets for a total of $3300. For each loss, you'd receive nothing, but for each of the 24 wins, you'd get back $210 ($110 + $100) for a total of $5040. Your net winnings would be $1740. That's a 152% return on your "investment" over two years.

This analysis is based on only two years of data. However, of the 33 observations included, the chance of being correct by chance 24 or more times is p=0.0068. To statisticians, that means it's significant. To gamblers, it means it's a safe bet. But if anyone has over-under data from years prior to 2005, I can include it in the analysis to increase the confidence level.

The lesson here is if you're going to gamble, don't bet on your own dumb guesses. Bet against the dumb guesses of everyone else. Take the over on stupidity.

The table below lists each team's actual wins, pre-season over-under lines, and the error of the over-under predictions. The system's correct predictions are in green and its incorrect guesses are in red.

Year	Team	Wins	Over-Under	O-U Error
2005	IND	14	11.5	-3
2005	PHI	6	11.5	6
2006	IND	12	11.5	-1
2005	NE	10	10.5	1
2006	NE	12	10.5	-2
2006	SS	9	10.5	2
2006	DEN	9	10	1
2005	ATL	8	9.5	2
2005	BAL	6	9.5	4
2005	CAR	11	9.5	-2
2005	MIN	9	9.5	1
2005	NYJ	4	9.5	6
2005	PIT	11	9.5	-2
2006	CAR	8	9.5	2
2006	DAL	9	9.5	1
2006	JAX	8	9.5	2
2006	KC	9	9.5	1
2006	NYG	8	9.5	2
2006	PIT	8	9.5	2
2006	CHI	13	9	-4
2006	CIN	8	9	1
2006	MIA	6	9	3
2006	SD	14	9	-5
2006	WAS	5	9	4
2005	BUF	5	8.5	4
2005	DAL	9	8.5	-1
2005	DEN	13	8.5	-5
2005	DET	5	8.5	4
2005	JAX	12	8.5	-4
2005	KC	10	8.5	-2
2005	SS	13	8.5	-5
2005	STL	6	8.5	3
2006	ARI	5	8.5	4
2006	MIN	6	8.5	3
2006	PHI	10	8.5	-2
2006	TB	4	8.5	5
2005	SD	9	8	-1
2006	ATL	7	8	1
2005	ARI	5	7.5	3
2005	CIN	11	7.5	-4
2005	GB	4	7.5	4
2005	HOU	2	7.5	6
2005	NO	3	7.5	5
2005	OAK	4	7.5	4
2005	WAS	10	7.5	-3
2006	BAL	13	7.5	-6
2006	NO	10	7	-3
2006	STL	8	7	-1
2005	CHI	11	6.5	-5
2005	NYG	11	6.5	-5
2005	TB	11	6.5	-5
2005	TEN	4	6.5	3
2006	BUF	7	6.5	-1
2006	CLE	4	6.5	3
2006	DET	3	6.5	4
2006	GB	8	6	-2
2006	OAK	2	6	4
2005	MIA	9	5.5	-4
2006	HOU	6	5.5	-1
2006	NYJ	10	5.5	-5
2006	TEN	8	5.5	-3
2006	SF	7	5	-2
2005	CLE	6	4.5	-2
2005	SF	4	4.5	1