Backtesting period

[neila]

A question that arises every time I build a model is the backtesting period. How far do you look back? Using horses for example, how far back in the past performances do you go? I use to maintain an extensive database on the horses. I could pull up a horse's data for as long as he had been racing. But, at some point, he ain't that horse any more.
Then there's the stock market. I've been running a model on the market for about 5 years now. And, I have data on some stocks in my database as far back as 1970. But that old adage that "It's not your daddy's stock market," kicks in at some point.
Now I'm building a model on the NBA. Although, it's not my first rodeo, I consider professional sports to be my greatest challenge to date. And, I would never ask any other modelers for specifics. What would be the fun in that? But, in general, what do you consider to be a good look back period for the NBA. It's always a challenge to get a large enough sample that is still relevant. Any thoughts appreciated.

[sharpcat]

In sports it is a good idea to pay attention to changes in rules which may have a drastic side effect on what you may be focusing on, some rule changes may benefit the offense or some may benefit the defense or some may effect the total score of the event. Obviously the larger the sample the better but if you are deriving numbers from a sample prior to an important rule change such as say the introduction of the 2 point conversion in the NFL and your focus is game totals this may have a drastic impact on your numbers and you may decide to not use games prior to this.

[Justin7]

You have some good questions.

First, as sharpcat said, I would pay attention to any real big rule changes. In NFL, I might discard anything before the change in the PAT rules back in the 90's.

Second, realize that the market is always getting tighter. If you observe a good win rate for most of the back-testing, but the last year or two is significantly worse, this is because more professionals are sharpening these lines. Don't expect to kill sports today like you could 5-10 years ago with simple models.