Testing my NFL Model

[mebaran]

I may be a little late to the party on this one, as the NFL season has already started, but I am trying to play catch up before week 6 (the week in which one of my models starts to generate plays).

My question is about testing past model results in the NFL. I know some people choose to use beating the closing line as a metric. I have chosen to focus on average line error compared to the market's average line error when predicting spreads.

First of all, what would the NFL model's average line error have to be to consider it a good model? What about a great model? Does it not necessarily have to beat the market's average line error?

Second (more of a philosophical question), how low could you hypothetically get your model's average line error? The NFL, and any sport for that matter, possess a lot of white noise. So even with a "perfect" model (and I use the term perfect loosely), you would still see a fairly significant avg. line error, no? The market, as a whole, has an average line error of about 10-11 (correct me if I'm wrong). I guess my question is "what could the best modeler in the world, using all known mathematical/theoretical approaches at his disposal get his NFL avg. line error down to?"

Any input would be appreciated.

[mebaran]

Also, as an aside...If my model spits out that the home team line is -4.5319681 pt., should I round it to -4.5 when computing my average line error since the market only has whole numbers and half decimals? No team will obviously win by a decimal amount of points, so in essence, my line, as a decimal, will NEVER be absolutely correct. How do I adjust for this, if at all?

[infamousbacardi]

Also, as an aside...If my model spits out that the home team line is -4.5319681 pt., should I round it to -4.5 when computing my average line error since the market only has whole numbers and half decimals? No team will obviously win by a decimal amount of points, so in essence, my line, as a decimal, will NEVER be absolutely correct. How do I adjust for this, if at all?

In this instance, I would round to the standard mathematical rounding rules...4.5319, I would round down to 4.5...I'd probably round 4.749 UP. This shouldn't mess w/ your model to an extent where a play isn't or is now a play...standard variances considered.

[Juret]

You want to have the largest win rate % for games where your model and the market's prediction differs the most.

[Wrecktangle]

I always use closing line as a model metric. If you can beat that consistently, you have a good model. Matter of fact, if you are inside the line an average of .5 point in NFL, you will do well; inside 1 pt you'll be riding in a limo pretty quickly.

In the late 90s and early 00s I was inside of 10 on avg dif score on occasion. Those were the days when my win % ATS for sides was about 58-59%. Needless to say, those days are over as the NFL simply changes to much year-to-year today, and as far as beating the line is concerned, syndicates have sharpened the line, especially since 2005. I'd be happy to achieve 55% in the NFL consistently today. Its the big reason I've swapped over to CFB.

I believe entropy accounts for approx 25% in the NFL game; i.e. a perfect model can account for approx 75% of the variation in a game. Almost everyone thinks I'm full of sh*t. But I've seen evidence to this, and at least one other good football statistician agrees with this number. His article is somewhere on http://www.advancednflstats.com/

[mebaran]

I always use closing line as a model metric. If you can beat that consistently, you have a good model. Matter of fact, if you are inside the line an average of .5 point in NFL, you will do well; inside 1 pt you'll be riding in a limo pretty quickly.

In the late 90s and early 00s I was inside of 10 on avg dif score on occasion. Those were the days when my win % ATS for sides was about 58-59%. Needless to say, those days are over as the NFL simply changes to much year-to-year today, and as far as beating the line is concerned, syndicates have sharpened the line, especially since 2005. I'd be happy to achieve 55% in the NFL consistently today. Its the big reason I've swapped over to CFB.

I believe entropy accounts for approx 25% in the NFL game; i.e. a perfect model can account for approx 75% of the variation in a game. Almost everyone thinks I'm full of sh*t. But I've seen evidence to this, and at least one other good football statistician agrees with this number. His article is somewhere on http://www.advancednflstats.com/

Doesn't CFB have even more entropy, as it is a somewhat "sloppier" and lesser skilled football game? I guess it wouldn't matter much if you are beating the closer by a bigger number than you would NFL. So then what would a "good" CFB model have to beat the closer by on average? A great one?

The amount of games per week in CFB deters a lot of modeling efforts, I'm sure. I guess the trick to CFB modeling, more than anything, is staying on top of data collection and organization, and account for things in college games that matter more like momentum, rivalries, etc.

[Wrecktangle]

Doesn't CFB have even more entropy, as it is a somewhat "sloppier" and lesser skilled football game? I guess it wouldn't matter much if you are beating the closer by a bigger number than you would NFL. So then what would a "good" CFB model have to beat the closer by on average? A great one?

The amount of games per week in CFB deters a lot of modeling efforts, I'm sure. I guess the trick to CFB modeling, more than anything, is staying on top of data collection and organization, and account for things in college games that matter more like momentum, rivalries, etc.

I don't have many years in CFB yet, but so far I think it's perhaps a lttle more predictable than NFL simply because of all the NFL rule changes (CFB rarely changes), lockout sh*t (have you wondered why all these QBs are throwing for all these high yards? I'm from Auburn, and followed Cam pretty closely; rest assured if it were a normal year, he'd have normal numbers.), coaching flavor of the month, etc. A few years back we had 11 NFL head coaching changes in one year.

I do know this, though: with a 2nd year model, I posted a 56.8% ATS win record here in the SBR CFB weekly contests last year (9 weeks, 15 games each week, 76-58-1, +20.35 units, beat the line an avg 3.3 pts) and I haven't done that well in that many games in the NFL since 2000.

[Wrecktangle]

One other thing. If the amount of data to be collected intimidates you then you've got to realize that you're probably not cut out to be a modeler.

But as I've mentioned in other posts, the skills you can learn in data mining, stat, model building is easily transferable to skills in the job market. If you contend that you don't need such skills you'll live off handicapping; good luck, that will last another few years.

[mebaran]

One other thing. If the amount of data to be collected intimidates you then you've got to realize that you're probably not cut out to be a modeler.

But as I've mentioned in other posts, the skills you can learn in data mining, stat, model building is easily transferable to skills in the job market. If you contend that you don't need such skills you'll live off handicapping; good luck, that will last another few years.

Granted there is a lot more CFB data than NFL, it's still more readily available than some European croquet league...so no, there isn't any intimidation here, I actually look forward to retrieving the data. Going through each process I get better every time. The fact that these skills can be used in a variety of settings is also appealing. So instead of begging for someone to give me their database, I actually prefer, and enjoy, getting the data.

Appreciate your input, Wreck.

[hoop22]

good luck