Statistical significance of betting system

[SD07]

Hello guys, I'm new here. I had an account a long time ago but I don't know what happened to it. Anyways, I was wondering if anyone could help me out on finding statistical significance of a betting system?

I tested a theory and it's come true 25 out of 37 times during back-testing. That's not statistically significant is it? Also, there's another one which is like 19 or 20 - 1. That one I'm sure has statistical significance? How can I be sure that a system is statistically significant if it doesn't have a very high win rate? I don't know too much about statistics. Thank you.

[Barrakuda]

The main issue is that if you are hunting for systems with great records, you will always find them by chance. Significance testing assumes that the record was sampled at random, so if you discarded five systems to find one that went 20-1, it really doesn't mean anything.

That said, for testing the significance of random samples, try the spreadsheets here (right hand margin):

http://www.football-data.co.uk/

[gojetsgomoxies]

looking at your 20-1 ATS record, i think i really depends on how long that was for. if it was a month or two in one sport, it's potentially awesome (i'd have to see your parameters, and it needs some sort of logical basis etc.)

if it's over 4 seasons so that you are picking 5 games a year with it, it is meaningless.

and of course, all scenarios in-between.

[SD07]

The main issue is that if you are hunting for systems with great records, you will always find them by chance. Significance testing assumes that the record was sampled at random, so if you discarded five systems to find one that went 20-1, it really doesn't mean anything.

That said, for testing the significance of random samples, try the spreadsheets here (right hand margin):

http://www.football-data.co.uk/

Hey thanks. Which spreadsheet exactly? Sorry for the late reply, just been researching this whole time.

[SD07]

looking at your 20-1 ATS record, i think i really depends on how long that was for. if it was a month or two in one sport, it's potentially awesome (i'd have to see your parameters, and it needs some sort of logical basis etc.)

if it's over 4 seasons so that you are picking 5 games a year with it, it is meaningless.

and of course, all scenarios in-between.

It was over the course of many years. The bet doesn't happen much, so yah it's pretty meaningless. There's no way that is random chance though. I suspect it's even over 20-1. But yah, it doesn't happen much, if ever.

There's another system I got, which has gone 36-11 so far. Can anyone tell me if this is statistically significant or not? Seems like it is, going by this source https://www.sportsinsights.com/sport...-significance/

[gojetsgomoxies]

there's a bunch of steps to the process,

1) easiest step.............. google "binomial calculator" and use one. and yes, 36-11 is very highly significant.

2) there's steps are endless. how did this come about? if you try many different parameters you will come up with many systems which are very high winning rate over small number of games over 7 years. but that's mostly a matter of trying so many things .......... one possible problem with systems that pick a few games a year is simply data error. i.e. bad lines........... basically google "data mining"

if you really want to get deeply into this i think campbell harvey has done an academic paper on this phenomenon

[gojetsgomoxies]

The main issue is that if you are hunting for systems with great records, you will always find them by chance. Significance testing assumes that the record was sampled at random, so if you discarded five systems to find one that went 20-1, it really doesn't mean anything.http://www.football-data.co.uk/

very well said................ and i think everyone cuts off seasons and optimizes parameters to turn a 54% WP into a 59% WP system.

and i find there needs to be a logical basis for the system...... and don't get wrong, that's quite subjective. i'm fine with favorite-underdog bias being different in each sport.

there are so many systems out there based on 7 conditions too.

[SD07]

there's a bunch of steps to the process,

1) easiest step.............. google "binomial calculator" and use one. and yes, 36-11 is very highly significant.

2) there's steps are endless. how did this come about? if you try many different parameters you will come up with many systems which are very high winning rate over small number of games over 7 years. but that's mostly a matter of trying so many things .......... one possible problem with systems that pick a few games a year is simply data error. i.e. bad lines........... basically google "data mining"

if you really want to get deeply into this i think campbell harvey has done an academic paper on this phenomenon

Thanks but that calculator doesn't address statistical significance. Does it?

And I'm not sure what it could have been. I have tried some systems, but not that many. It could have been data error, but what about the 36-11 system? You think I'm on to something with this one?

Edit: it's now at 38-13

[danshan11]

here you go

19-5 (1.25, 79.2%) avg line: -146.2 / 134.1 on / against: +$1,185 / -$1,257 ROI: +33.8% / -52.3% 5-11 (-1.06, 31.2%) avg line: 146.2 / -162.8 on / against: -$416 / +$344 ROI: -25.8% / +13.2% 9-15-0 (-0.08, 37.5%) avg total: 9.3 over / under: -$750 / +$490 ROI: -28.6% / +18.4%

and here you go

243-141 (1.03, 63.3%) avg line: -155.0 / 141.2 on / against: +$3,139 / -$4,943 ROI: +5.3% / -12.9% 134-160 (-0.48, 45.6%) avg line: 140.4 / -155.6 on / against: +$2,754 / -$4,725 ROI: +9.1% / -10.2% 177-188-18 (0.44, 48.5%) avg total: 8.9 over / under: -$2,968 / -$600 ROI: -7.0% / -1.4%

as you can see these have awesome records and anyone can make these all day, you start with one parameter and keep changing the number in that parameter until it shows max ROI and then do the next and keep going, you would be amazed at the 20-30% ROI over 200-1000s of games "systems" there are

THESE ARE 100% NOISE! they mean absolutely nothing and have no value.
you can argue a team won the last game and the total here is low and they win when the total is low and blah blah blah but NOISE, its only noise.

[gojetsgomoxies]

Thanks but that calculator doesn't address statistical significance. Does it?

https://stattrek.com/online-calculator/binomial.aspx ............. yes, this calculator shows it's signficant to 1%. i think that 2 standard deviations. not sure, been awhile.

the bigger thing is ripping the system/idea apart.

why not just share what it is? maybe generally....... people openly share winning ideas on here all the time

[gojetsgomoxies]

you have shared much but my sense is you don't have enough games per season....... i wouldn't talk anything to seriously that doesn't give 70 plays per season (NHL, NBA, MLB), maybe even 100 plays minimum per season.

[peacebyinches]

There's a bunch of possible interpretations in this situation. Since these were ATS bets would be reasonable to say that if your model was totally random it would follow a binomial distribution. You can find some information on the probability mass function for this type of distribution and related probability distributions that might be more suitable depending on the specifics of your model here:

http://www.math.wm.edu/~leemis/chart/UDR/UDR.html

Since the sampling here seems to be very sparse, is there any way to relax the constraints so that you can increase your sample? Try optimizing your model (if possible) so that you can find a happy medium with more samples but reduced win %. Even if you come across model with a 90% win rate, if you only can bet 3 games each season you're ROI is going to be very small, and/or extremely volatile. Let us know how it goes.

[SD07]

here you go

19-5 (1.25, 79.2%) avg line: -146.2 / 134.1 on / against: +$1,185 / -$1,257 ROI: +33.8% / -52.3% 5-11 (-1.06, 31.2%) avg line: 146.2 / -162.8 on / against: -$416 / +$344 ROI: -25.8% / +13.2% 9-15-0 (-0.08, 37.5%) avg total: 9.3 over / under: -$750 / +$490 ROI: -28.6% / +18.4%

and here you go

243-141 (1.03, 63.3%) avg line: -155.0 / 141.2 on / against: +$3,139 / -$4,943 ROI: +5.3% / -12.9% 134-160 (-0.48, 45.6%) avg line: 140.4 / -155.6 on / against: +$2,754 / -$4,725 ROI: +9.1% / -10.2% 177-188-18 (0.44, 48.5%) avg total: 8.9 over / under: -$2,968 / -$600 ROI: -7.0% / -1.4%

as you can see these have awesome records and anyone can make these all day, you start with one parameter and keep changing the number in that parameter until it shows max ROI and then do the next and keep going, you would be amazed at the 20-30% ROI over 200-1000s of games "systems" there are

THESE ARE 100% NOISE! they mean absolutely nothing and have no value.
you can argue a team won the last game and the total here is low and they win when the total is low and blah blah blah but NOISE, its only noise.

I'm not understanding what you're trying to say here.

[danshan11]

I'm not understanding what you're trying to say here.

systems or trends or piles of stats to derive a +ROI are a dime a dozen. they have no value and are purely a consequence of mixing tons of variables, its just noise.

I dont know what you are doing but when you say a "system" is 37-10 or whatever that is usually a system type thing, contrary to popular belief past events have very little effect on current games and any past event that does have value it is in the current games line.

[HeeeHAWWWW]

..... contrary to popular belief past events have very little effect on current games and any past event that does have value it is in the current games line.

The first is clearly not true. How do you think anyone comes up with a prediction on anything?

The second has some truth to it.

[danshan11]

really elaborate, so I can understand it better please

[danshan11]

simple, yes or no answer, if a team won yesterday does that give them a better chance of winning today everything else equal?

my opinion maybe very slightly it does but unfortunately, that tiny edge is adjusted in today's line and because the line adjusts that gives the edge no value today

[SD07]

https://stattrek.com/online-calculator/binomial.aspx ............. yes, this calculator shows it's signficant to 1%. i think that 2 standard deviations. not sure, been awhile.

the bigger thing is ripping the system/idea apart.

why not just share what it is? maybe generally....... people openly share winning ideas on here all the time

you have shared much but my sense is you don't have enough games per season....... i wouldn't talk anything to seriously that doesn't give 70 plays per season (NHL, NBA, MLB), maybe even 100 plays minimum per season.

It's an MMA betting strategy which I don't want to completely reveal, but essentially, it's about momentum. It's 50-30 now. So I'm not sure how good that is anymore. Guessing not very good.

[danshan11]

its 50-30 and what is the avg line on the 80 games?

[gojetsgomoxies]

It's an MMA betting strategy which I don't want to completely reveal, but essentially, it's about momentum. It's 50-30 now. So I'm not sure how good that is anymore. Guessing not very good.

thx,

MMA probably good for a momentum system...... i'm guessing you can find inefficiency in MMA. and you can probably fundamentally handicap it too. it's very very time consuming to do full-handicapping on the major USA sports other than NFL. too many games.

i would note for anyone to take it remotely seriously, it needs to be a set rule, no judgement involved.

[SD07]

its 50-30 and what is the avg line on the 80 games?

I checked around 40-50 fights and it was like 1.80ish or so

[SD07]

Sorry, it was 30 lines and it was about 1.85 on avg

[SD07]

Alright, I just completed 100 backtests. Results were 67 wins and 33 losses. That's statistically significant, right?

[SD07]

I just did all the winning lines and got 1.747761194 average. Is this good?

[danshan11]

so 57% of the time they should win according to the avg line including margin I dont know how big the margin is in the mma

and they are winning 66% of the time
sounds like you have either NOISE or the best system ever!

[SD07]

so 57% of the time they should win according to the avg line including margin I dont know how big the margin is in the mma

and they are winning 66% of the time
sounds like you have either NOISE or the best system ever!

A lot of the lines are in the 1.40-50's but the odd time there are lines that are like 1.80 to even 2.80 and even as high as 3.50+.

So 67 out of 100 wins is noise? Are you sure? Assuming that it wasn't noise, how would one use this system?

I can't play all the lines on every card, so do I just randomly pick 1 or 2 fights on a card and place my bet? Also, those were opening odds, so let's say there were 2 fights on a card that this system said to play, what would I do? Pick the one that starts earlier and then bet the other one afterward?

I'm guessing that might not work since the second fight will be near closing odds and not profitable, right?

[SD07]

thx,

MMA probably good for a momentum system...... i'm guessing you can find inefficiency in MMA. and you can probably fundamentally handicap it too. it's very very time consuming to do full-handicapping on the major USA sports other than NFL. too many games.

i would note for anyone to take it remotely seriously, it needs to be a set rule, no judgement involved.

Yah np, I've been watching MMA for a long time now, so it's easier for me to follow along. It seems like the books use a similar momentum algo, because a lot of the lines are efficient. And this strategy is rule based for sure.

[danshan11]

A lot of the lines are in the 1.40-50's but the odd time there are lines that are like 1.80 to even 2.80 and even as high as 3.50+.

So 67 out of 100 wins is noise? Are you sure? Assuming that it wasn't noise, how would one use this system?

I can't play all the lines on every card, so do I just randomly pick 1 or 2 fights on a card and place my bet? Also, those were opening odds, so let's say there were 2 fights on a card that this system said to play, what would I do? Pick the one that starts earlier and then bet the other one afterward?

I'm guessing that might not work since the second fight will be near closing odds and not profitable, right?

well, you threw me off when you said you are giving me opening lines, it would be wiser to use the closing lines, they tend to be more efficient.

the number of results unless its a huge sample size does not mean much, more importantly is how your bets are doing from the price you pay compared to the closing price

[Gaze73]

If you're confident the system has a logical basis there's a good chance it will work. E.g. I noticed a certain pattern in soccer where draws would occur around 40% of the time, and after I started betting on it it kept hitting at 40% to this day after another 50 games.

[gojetsgomoxies]

i didn't even think about MMA and money line bets........ if you are betting 1.7x on average then your win rate should 70%ish (just did this in my head. hope it's not wrong). but you are miles away from 50/50.

the binomial calculator will let you put in 70% for odd. and then you put in 26-11. and 70% of 37 = 26... so you are right on the expected random number

hope i interpreted your comments correctly

[gojetsgomoxies]

If you're confident the system has a logical basis there's a good chance it will work. E.g. I noticed a certain pattern in soccer where draws would occur around 40% of the time, and after I started betting on it it kept hitting at 40% to this day after another 50 games.

i am very interested in betting soccer ties...... i've done some work but not alot....

[danshan11]

pretty interesting about that stuff

http://www.football-data.co.uk/Contrarian.pdf

[peacebyinches]

Thank you for sharing that article, THAT is the kind of information we need more of in the discussions in The Handicapper Think Tank.

I’ve had similar schemes in the past to the one illustrated in that article, though never implemented long enough (and in retrospect used a poor parameter estimate for determining bet sizes that long story short did nothing but increase variance in the end). I do see the potential for such a ‘contrarian’ approach since it is not terribly difficult to work with and is founded in what I would consider a decent (perhaps sufficient?) logical framework. That article got me thinking again about some objective quality checks I do want to follow up on... anyways great post.

[danshan11]

I still believe in the stink test hypothesis. Lost 5 in a row, got shutout last game, 2-8 last 10, 0-5 last 5 ats. Stuff like that where no one would bet on them

[bettingman6]

MMA depends largely on who you're betting on. Even if your long term win % is 62.5%, you will lose money long term if your average win is on a favorites of +167 or better. Note that it really your average win that matters rather than your average bet. (If you lose a bet it doesn't really matter if it was on a -900 favorite or +900 underdog.)

Much like with baseball, soccer, and hockey, what matters in MMA and boxing is your units won, not your win %. Win % only matters in basketball and football, and even in those sports it only matters if you do exclusively spread bets and don't do moneyline.