[podonne]

Greetings,

I have a model I've built that works pretty well for NHL, +50 to +100 units a season (never less than +52) for the last four years. I've been very careful in evaluating it, literaly processing the database, each bet, and the box scores\results day by day, so I'm confident that the results aren't reflecting any overfitting.

The only remaining "concern" I have is that the odds I used in the historical database are from Covers.com past results. A valid criticism might be that my model assumes that I can get Covers.com odds (or better) on every bet. (Popular concensus is that the odds come from Pinnacle). I don't have any way of directly answering this question. The model doesn't look that closely at odds when it bets (it looks at whether the team is a fav\dog and whether is is a strong one of those).

I reason that I may not always get the exact odds, but if its random (assuming that Pinnacle does not ALWAYS have the best odds) then I should get some better odds and some worse odds, and it will even out. Secondly, with 50-100 units I can afford to give up a few.

My question to the forum is, how much would this issue of "can I get the right odds" concern you? Assuming the model is steadily profitable taking every other concern out, am I worrying too much about this?

Thanks.

[roasthawg]

It probably should worry me more than it does... I just sit down and place all of my bets at 5dimes at the same time for the day.

[Tackleberry]

I put in 50-60 hours a week at work where I don't have access to the internet. So I don't have always have the ability to make sure I get the best possible price. As long as I get a price, that as I see it has value, I'm happy. Plus I am small time so a few cents either way doesn't add up to a significant amount.

[podonne]

I put in 50-60 hours a week at work where I don't have access to the internet. So I don't have always have the ability to make sure I get the best possible price. As long as I get a price, that as I see it has value, I'm happy. Plus I am small time so a few cents either way doesn't add up to a significant amount.

The model does make a lot of bets, so a few cents might make a difference over a long time. What if I give it a bit of a handicap? Say rewarding 5% less for every win? I'm worried that might destroy the model.

[skrtelfan]

Find a second source of odds and compare the two sets for any major discrepancies.

[That Foreign Guy]

If the model produces a fair line then you'll know if you're getting a good price or not.

[pedro803]

I am a little confused (yes I realize this is normal for me so please refrain from pouncing think tankers), are you talking about the price you are getting when you place the bet? Or are you talking about the odds as a component of your model?

I realize you could be talking about both at the same time, odds as a predictive factor in the model obviously -- but you mentioned your backtesting method -- so it could be that you are worrying about whether the model would have actually triggered those plays at the time -- or it would have indicated the plays but you would have gotten a worse line and lost so your back test is compromised.

on re-reading the thread for the second time, I see that this must have been what you were asking just as the answers the others have given have indicated -- sorry for the intrusion

[suicidekings]

Has this model ever been used? Or is it a new construct that's basing its success on past season data? The above questions sound like it's a new model.

[illfuuptn]

Shouldn't matter much at all.

*I'm certainly not an expert on this but are you using data from last year to backtest last year's season? If so, that's bad(this has been said trillions of times in the think tank).

On another side note I'm about to send you a pm asking you where the **** to begin when handicapping the NHL. I'm deciding between that and wnba as a second sport.

[That Foreign Guy]

Goalies IMO.

[podonne]

If the model produces a fair line then you'll know if you're getting a good price or not.

I struggled for a long time with models where I would build a great one that produced an accurate fair line, but it always fell apart running on new data.

Eventually I came to the realization that when the book odds and my fair line disagreed, either the book odds were missing some information that I had (my line was better), or my line was missing information that the book had (the book line was better). I think most people automatically assume that the former is the case, but I often found that the latter was more accurate. The only way to know for sure is to run the model on new data and see if its profitable, but once you include the book odds at the end of the model, you lose the argument for excluding it at the beginning of the model, and you also run into the same issue I raised above, choosing the right source for your odds. So I gave up on producing a "fair line" and just included the odds as a factor in the model, just like runs scored or goals.

My point, in short, is that just saying "produce a fair line and you'll know when to bet" doesn't work well for me, hence the need to find a realistic source for odds. I'd rather not rely on getting the best price for profitability, I'd like that more to be icing on the cake.

[podonne]

Has this model ever been used? Or is it a new construct that's basing its success on past season data? The above questions sound like it's a new model.

The database starts with 2005-2007 data, builds a model based on that, then walks day by day through the next four seasons updating every year or so. So it should be a good, validated model without bias from backtesting.

When I say day by day, I mean quite literally day by day. Starting with 2005-2007 games loaded, the program treats each day as a single entity, loading the games that day into the database (but not the results), reading the odds, adjusting the model, then making the bets. Then the box scores for the games are loaded into the database and the bets are resolved. This ensures that there are no mistakes anywhere that could introduce backtesting errors, since the results of the game being bet on are not in the database when the bet is made. Its more like a simulation that a backtest.

[suicidekings]

The simple solution is that not every matchup of Team A @ Team B is equivalent. In a sport like the NHL, momentum means a lot, and there are numerous factors that are much harder to quantify accurately.

Team defensive breakdowns leading to excessive screened shots, elite scorers on hot streaks, teams experimenting by shuffling of offensive lines from game to game and therefore disrupting chemistry, nagging injuries of the starting goalie on a team with a weak backup, a couple of dirty goals early in the game throwing the team out of rhythm, specific matchup issues related to the teams' differences in size/speed, etc.

Depending on the data you're inputting, you probably are missing key information, even in the games where you don't notice the discrepancy in the model output. Maybe a 7-2 loss started off as a 4-0 deficit a few minutes into the game, but the gameplay over the remaining 50+ minutes was much closer than the final score indicates.

Does your model attempt to assess a team's current form (ie: last 5/10 games)? Or simply its overall rating at a given point?

[podonne]

The simple solution is that not every matchup of Team A @ Team B is equivalent. In a sport like the NHL, momentum means a lot, and there are numerous factors that are much harder to quantify accurately.

Team defensive breakdowns leading to excessive screened shots, elite scorers on hot streaks, teams experimenting by shuffling of offensive lines from game to game and therefore disrupting chemistry, nagging injuries of the starting goalie on a team with a weak backup, a couple of dirty goals early in the game throwing the team out of rhythm, specific matchup issues related to the teams' differences in size/speed, etc.

Depending on the data you're inputting, you probably are missing key information, even in the games where you don't notice the discrepancy in the model output. Maybe a 7-2 loss started off as a 4-0 deficit a few minutes into the game, but the gameplay over the remaining 50+ minutes was much closer than the final score indicates.

Does your model attempt to assess a team's current form (ie: last 5/10 games)? Or simply its overall rating at a given point?

I use averages for all factors and ratings that cover the season-to-date and seperately for the last 60 days. 60 days in the NHL covers, what, 25-30 games?

I'm interested to hear if anyone has been burned by this. Made a great model that got great results backtesting\simulating, but in practice failed because you couldn't get as good of odds as you got during the simulation.

Note: This is not the same as backtesting a model and then it failed on new data because of overfitting, that happens to everybody.

[suicidekings]

I use averages for all factors and ratings that cover the season-to-date and seperately for the last 60 days. 60 days in the NHL covers, what, 25-30 games?

I would argue that 60 days is too long of a period to really accurately describe a team's current form.

[podonne]

I would argue that 60 days is too long of a period to really accurately describe a team's current form.

Does 30 days seem better? If you get too small, the sample size drops rapidly, so there's a better chance that a "run" is a product of chance rather than a truly well-performing team. If 60 days includes 30 games, 30 is a pretty good sample size for these purposes.

[ebemiss]

I've done something similar with NBA/WNBA/International Basketball and I've had more success than anything I've created in the past. The success I've had with this is using the closing number when creating a "new" ranking for team. So to answer your question, I think you will be fine using covers numbers. As long as you are loading data into your model for that current day. It should work fine.

I try to make sure each team is beating or close to the closing line for about a 5 game back test based on my CURRENT ranking for that team. Teams will change slightly over that small period so back testing a line or total much beyond that, using your current "ranking" or number for that team, is useless, in my opinion. Unless you keep track of each team's rankings/pace scenario/total daily, using basketball terms it wouldn't make sense to use current number to back test much further.

Looks like you have something solid. good luck with it. Hope I answered the question as I didn't mean to confuse.

[louis.ana]

Getting the right odds for me is to not play a favorite of more than -125, EV to +points is better.
A 31 day sample size is adequate, you have to break up the season over its months.. and look at how the teams in the league play differently as the season progresses, changes are made, some teams get better, some give up, some are overrated.

I've noticed this a lot in MLB and NBA, where my best months would be towards the middle of the season (June/July for MLB), (Nov/Feb for NBA) the beginning and end of those seasons its either breaking even or losing. But those 2 winning months make up for much more of my loss that it ends up a winning season.

You already have one winning model, if you can find a second or third model to also play, like sets of plays.. you can earn more over the long run.

[podonne]

You already have one winning model, if you can find a second or third model to also play, like sets of plays.. you can earn more over the long run.

The challenge I've always had with multiple models is what to do when both models pick different sides of the same game. "Pick the one with the highest EV" never seemed like the right answer, since I never trust the degree of EV represented by a model, so much as the fact that the model picked a team and the model is +EV. I think in that case I would not bet on the game at all, since betting both teams is guaranteed -EV. :-)

[suicidekings]

Does 30 days seem better? If you get too small, the sample size drops rapidly, so there's a better chance that a "run" is a product of chance rather than a truly well-performing team. If 60 days includes 30 games, 30 is a pretty good sample size for these purposes.

Therein lies the problem. Hockey is such a streaky sport that when you get into discussion about sample size, you're assessing a period of 30 games which might include winning and losing streaks that greatly diverge from the expected long term win rate. I randomly selected a team and pulled their first 30 games of the season.

Columbus Blue Jackets First 30 games [2010-11] W-L (Goal differential)
Oct 8 - Oct 30: 6W-4L (-5)
Nov 2 - Nov 24: 8W-2L (+17)
Nov 26 - Dec 18: 2W-8L (-20)

At the end of those 30 games the team was 16-14. They then went 7-11 from Dec 21 to Feb 1, 8-2 from Feb 4-25 before closing out the season going 3-19 in their last 22 games. Final record was 34-35-5-8.

If your model is predicting a team that in the long run is a little below .500, then you'd be correct. However, it's easy to see that there are times when the team's true odds of winning diverge greatly from their expected averages produced by the model. Sample size is not the answer to predicting when a team will get hot, and recognizing when i team is hitting a high or low period is critical to capping NHL.

[vyomguy]

If you can pick winners..odds doesnt matter.

[wiffle]

getting the right odds is the only thing that matters

[subs]

ridiculously important - unless of course u can pick winners. lolz

good luck with that... we work so hard to find an edge and the piss half of that away because we can't b bothered to do the really easy part.

guess it is pretty boring tho.

[louis.ana]

The challenge I've always had with multiple models is what to do when both models pick different sides of the same game. "Pick the one with the highest EV" never seemed like the right answer, since I never trust the degree of EV represented by a model, so much as the fact that the model picked a team and the model is +EV. I think in that case I would not bet on the game at all, since betting both teams is guaranteed -EV. :-)

I play both sides. I follow each model as is, viewing all of it's plays as a set or atomic. You should focus on the net units from your models on a weekly basis, not the daily plays that cancel each other... shoot for a win rate of 55%+

Unless in the extreme case that you have 2 models and all games from both cancel each other for the daily, which is unlikely to happen, you should trust your model(s).

[TheLock]

If you can pick winners..odds doesnt matter.

[ronibrown]

It always worries me more than the outcome itself

[podonne]

I play both sides. I follow each model as is, viewing all of it's plays as a set or atomic. You should focus on the net units from your models on a weekly basis, not the daily plays that cancel each other... shoot for a win rate of 55%+

Unless in the extreme case that you have 2 models and all games from both cancel each other for the daily, which is unlikely to happen, you should trust your model(s).

Interesting. I wasn't expecting to see a defender of playing both, as opposed to picking one or betting neither.

Have you ever testing your models to see if it improves by not playing both sides of the same game?

[TheCentaur]

There are a few things that worry me about this. First, you said you have worked and tried several models to find a successful one. That sounds like overfitting right there, especially with a sample size of 2005-2007. Not a large sample at all. Second, I share your concern about the line inaccuracies. Maybe run the same model with a different site's archives, like here at sbr. Run your model for '09 and '10 through two different archives and see if they are similar.

[lakerboy]

If you can pick winners..odds doesnt matter.

okay bro.

[podonne]

There are a few things that worry me about this. First, you said you have worked and tried several models to find a successful one. That sounds like overfitting right there, especially with a sample size of 2005-2007. Not a large sample at all. Second, I share your concern about the line inaccuracies. Maybe run the same model with a different site's archives, like here at sbr. Run your model for '09 and '10 through two different archives and see if they are similar.

Always a valid concern, and I'm very careful to avoid anything that might be overfitting. For instance, I set approaches by looking at past performances, but I don't "fine tune" parameters. I never manually set parameters based on past performances, only on rules of thumb. I never evaluate the 2010 season, I will save that for when I think I have a model that I'm ready to start betting with. I use 2005-2007 as a configuration but I've tested on 2007-2009.

I even go so far as to run the model by building the database day by day without no pre-set parameters to eliminate "future information leak". The model walks day-by-day, reading the lines, making bets, and only then reading the box scores and results and evaluating the bet. Its very careful.

Admititedly its more art than science, but I'm probably the most careful person in that regard. I'm not a gambler. :-)