I may be a little late to the party on this one, as the NFL season has already started, but I am trying to play catch up before week 6 (the week in which one of my models starts to generate plays).
My question is about testing past model results in the NFL. I know some people choose to use beating the closing line as a metric. I have chosen to focus on average line error compared to the market's average line error when predicting spreads.
First of all, what would the NFL model's average line error have to be to consider it a good model? What about a great model? Does it not necessarily have to beat the market's average line error?
Second (more of a philosophical question), how low could you hypothetically get your model's average line error? The NFL, and any sport for that matter, possess a lot of white noise. So even with a "perfect" model (and I use the term perfect loosely), you would still see a fairly significant avg. line error, no? The market, as a whole, has an average line error of about 10-11 (correct me if I'm wrong). I guess my question is "what could the best modeler in the world, using all known mathematical/theoretical approaches at his disposal get his NFL avg. line error down to?"
Any input would be appreciated.