Backtesting question

**rsigley** · 01-16-12, 07:12 PM

data splitting (2 or 3 depending on size) then k fold cross validation

**Miz** · 01-17-12, 10:15 PM

I do lookups to another database with known opens and closes all based on pinnacle. It is a pain but it beats not knowing where the lines come from. There are errors occasionally but they can sorted out. I just concatenate the date, and home team and use that as the lookup criterion, then save the output as its own database and use it going forward. It eliminates one more source of error, and allows for easy testing of a model against openers or closers. Most models do better against openers than closers, and I like to test that so it is nice to have a consistent source. I don't have any tennis data though. Best of luck.

**CanuckG** · 01-17-12, 10:44 PM

Has this been beneficial for you since you've been doing this?

**Blax0r** · 01-18-12, 05:20 AM

Originally posted by rsigley

data splitting (2 or 3 depending on size) then k fold cross validation

I'll try this out during the davis cup week after the AO; thanks for the suggestion.

Miz, I like idea of testing against openers/closers (data permitting); it more or less takes the "when-to-bet" factor out of the equation.

CanuckG, I'm somewhat new to sports betting, and very new to implementing backtesters and models in conjunction with my betting (previously I was a *very* small-time steam-chaser). This calendar YTD, I am down a substantial amount despite getting outstanding results from my backtesting (no bugs as far as I'm aware - each betting decision doesn't incorporate future information).

Has anyone else been in a situation where backtesting looks great, but in practice, the strategy is a loser?

**FourLengthsClear** · 01-18-12, 06:17 AM

Originally posted by Blax0r

I'll try this out during the davis cup week after the AO; thanks for the suggestion.

Miz, I like idea of testing against openers/closers (data permitting); it more or less takes the "when-to-bet" factor out of the equation.

CanuckG, I'm somewhat new to sports betting, and very new to implementing backtesters and models in conjunction with my betting (previously I was a *very* small-time steam-chaser). This calendar YTD, I am down a substantial amount despite getting outstanding results from my backtesting (no bugs as far as I'm aware - each betting decision doesn't incorporate future information).

Has anyone else been in a situation where backtesting looks great, but in practice, the strategy is a loser?

Pretty much every year in MLB, LOL.

**Blax0r** · 01-18-12, 05:54 PM

Or, as I just found out today from a first-hand source, the odds source posted incorrect historical odds.

That would explain a lot.

**Dom177** · 01-18-12, 06:18 PM

How are you getting outstanding results but down a substantial amount? I've done a little bit of back testing but realized I don't have the time or patience to do it right now. I'll probably start up something before baseball season though so I'm trying to gather up as much info/ideas as possible.

**Blax0r** · 01-18-12, 06:59 PM

Originally posted by Dom177

How are you getting outstanding results but down a substantial amount? I've done a little bit of back testing but realized I don't have the time or patience to do it right now. I'll probably start up something before baseball season though so I'm trying to gather up as much info/ideas as possible.

From my experience, this means the backtesting process itself is somehow using future information in each betting decision.

For example, I arrange my data into a "winner" and "loser" column that is fed into my backtester. I had logic that would "default" to picking the player in the winner column in certain situations, and that obviously inflated my numbers. This is a blatantly bad methodology that I spotted after doing some paper betting. There are much more subtle things, however, that could leak future information in a betting decision.