Statistics Question

[lid73]

Suppose you have a system where there are two possible outcomes, 0 or 1. Each of them is 50% likely to occur in any given trial. When a result such as 1 occurring 6 times out of 10 occurs, how can you tell if this is statistically significant? At what point does this occur? For example, if the 1 occurs 30 times out of 50, is this significant?

I would like to know the mathematics and reasoning behind this. Also, is there a way of figuring standard deviation of the results?

Obviously, I want to know how many games you must pick above 50% in order to consider that your long term win percentage.

[maritime]

in general, chi-square analysis with p value < 0.05 would demonstrate that the results were not due to random chance.

In your example of 30 times out of 50 the p value would be 0.15 so the results could have been from chance.
32 out of 50 puts the p value less than 0.05 so would be considered statistically significant (as would 18 out of 50).

Check out http://www.graphpad.com/quickcalcs/chisquared1.cfm

Put in win for category 1, loss for category 2
then put in observed wins and losses in appropriate boxes
then put in expected wins and losses (50% equivalents)
click calculate to get p value

[lid73]

Thanks Maritime.

But in order to say that (for example) I am a 60% winner, how could I calculate the confidence interval?

In your example, 32/50 is only due to chance 5% of the time. So, we know it is significant. However, you can't say that your win rate is 64% there, can you? Or, is it 95% likely that your win rate is 64%? That doesn't seem right.

Can you say after 50 trials that your win percentage should be between 60% and 68% with 95% confidence? That is what I hope to do.

[Ganchrow]

Suppose you have a system where there are two possible outcomes, 0 or 1. Each of them is 50% likely to occur in any given trial. When a result such as 1 occurring 6 times out of 10 occurs, how can you tell if this is statistically significant? At what point does this occur? For example, if the 1 occurs 30 times out of 50, is this significant?

I would like to know the mathematics and reasoning behind this. Also, is there a way of figuring standard deviation of the results?

Obviously, I want to know how many games you must pick above 50% in order to consider that your long term win percentage.

In this type of situation it makes sense to use the use the binomial distribution.

Using your example of 6 successes out of 10, we see that the likelihood of 6 or more successes occurring assuming an actual probability of 50% would be 37.7%. We can get this from Excel using the following formula:

=1-BINOMDIST(5,10,0.5,1)

In general, after n trials, the probability of s or more successes occurring strictly by chance assuming a probability of p would be given by:

1) =1-BINOMDIST(s-1, n , p, 1)

So looking at another specific example, let's say that we observe 32 succeesses out of 50 trials, how confident can we be that the actual success rate is greater than 52.38%? (Putting the question another way, if the "true" success rate were 52.38% with what probability would we expect to see at least 32 successes out of 50?)

Plugging in the values in to the Excel formula above, we get

=1-BINOMDIST(32-1, 50, .5238, 1) = 6.6%

which tells us that 6.6% of the time we'd expect to see 32 or more successes, strictly by chance. given an actual probability of 52.38%. Therefore, we would not be able to say at a p-value of 0.05 that the system in question would be profitable at -110 odds.
<hr>
Looking at this from a different angle, let's say that say that you perform n trials of a particular event, and you want to know how many successes you would need in order to determine that the probability of that event occuring is greater than a given probability p, for a confidence level of &alpha;.

So again using Excel:

2) =1+CRITBINOM(n, p, &alpha

Now using a specific example, let's say that we have 1,000 trial and we want to be at least 99.9% confident that the true success rate of the system is greater than 52.38%. Plugging into formula 2:

=1+CRITBINOM(1000, 0.5238, 0.999)

we get a result of 574, meaning that we'd need to see 574 successes out of 1,000 to be at least 99.9% sure that our system picks with greater than 52.38% accuracy.


I refer the reader to Chapter 7 of Stanford Wong's Sharp Sports Betting, "Testing W-L Records for Significance".

[tacomax]

I was just about to post what Ganchrow posted but he got in there just before me.

[onlooker]

I was just about to post what Ganchrow posted but he got in there just before me.

I know how you feel. He does that to me all the time.

[lid73]

Ganchrow:

You are the man. Thank you. Now, I plan on figuring out my 99% confident win percentage, and determining unit size based on a risk of ruin equation. I have 3 NFL seasons worth of bets, which is close to 200 trials. I was unsure how to use the binomial distribution. Thanks again.