Intermediate-level probability question -- Accounting for Type I & II Errors

[Ganchrow]

My impetus for posting this problem was a discussion I had had earlier with Crazyl regarding how one might go about determining which posters are truly +EV. I owe some of the generalized phrasing of the first two questions to a take home quiz shown to me by BudddyBear.

All interested are free to post their answers. No need for anyone to wait.

You want to test a few hypothesis regarding a given bettor.

1. Over the course of the following season the player will place 500 uncorrelated unit-bet no-push picks at odds of -110. Of those 500 picks, at least how many would he need to win in order for us to be 95% certain he's not a -EV player. (In other words for there to only be a 5% chance that we would mislabel him not -EV when in fact he were actually -EV. (This is know a "false positive" or a "Type I" error.)
2. If we knew a priori that the player were in fact a 54% picker, then based on the test above what would be his probability of being mislabeled a -EV player? (This is known as a "false negative" or a "Type II" error.)
3. If we wanted the player's probability of being mislabeled -EV to be no higher than 25% (say because we had bet at -290 that the player would be labeled +EV), and if the player needed to place all 500 of his bets at one and only one odds level, what would be the lowest magnitude odds at which the player would need to place all his bets in order for the test's Type I error rate to be no more than 5%, as above.
   
   You can assume that the player's real edge in this question #3 is always the same as in #2 and doesn't vary with payout odds.

[Mudcat]

This is a trick question. The real answer is, anyone placing bets at -110 obviously doesn't know what they are doing and can't possibly be +ev.

Seriously, I'm not sure if this is really smart stuff or over-thinking. Are we trying to mathematically determine frauds on the forum?

Maybe it's just supposed to be a fun math exercise. People have different definitions of fun.

My background in university calculus and statistics does not enable me to answer that. Of course, that was only 1st year credits and it happened in the distant past, around the Renaissance (at least that's how it feels right about now.)

I am just extremely glad that it is not necessary to be able to solve this problem to be a +ev player.

[Justin7]

The test is two-fold (IMO). 1. Identify sharps. I suspect anyone making 100k+ a year from sports can do this test without looking anything up. 2. Give people a reality check. It's very hard to tell if someone is a winning player from a short segment (e.g. under 1000) of picks.

[Mudcat]

I suspect anyone making 100k+ a year from sports can do this test without looking anything up.

I can state unequivocally that this is not the case.

I am not sure why you think these things would be so connected.

To me that is like saying, anyone who can make 100K a year from sports should be able to be a grand master at chess. There are some dimensions that overlap somewhat, but to assume such a rigid connection is pretty far-fetched IMO.

[Chi_archie]

interesting

[Boner_18]

Wow. All that stats material from college in '04-'05 is really lost on me. Had to take it because I was a science major... Wish I had retained enough to do problems like this.

[SBR Lou]

My impetus for posting this problem was a discussion I had had earlier with Crazyl

One small step for a man, one giant leap for +EV.

[etothep]

It is way too early for me to even read that problem, much less think about it.

Where's my adderall, coffee & mountain dew cocktail?

[Peep]

The real answer is, anyone placing bets at -110 obviously doesn't know what they are doing and can't possibly be +ev.

I don't think that is at all a given. A lot of prop players, for example, would kill for -110 lines.

I do agree with you about Justin's statement about "anyone making 100K a year from betting sports" knows the answer without looking any thing up being presumptuous.

There are usually several ways to skin a cat.

[dcbt]

#1. I come up with 281 winning bets out of the 500 placed.
#2. I get 23.43% chance...

#3 gives me tiredhead, but i might play around with it anyway.

[Justin7]

I don't think that is at all a given. A lot of prop players, for example, would kill for -110 lines.

I do agree with you about Justin's statement about "anyone making 100K a year from betting sports" knows the answer without looking any thing up being presumptuous.

There are usually several ways to skin a cat.

I didn't say they know it... just they know how to solve it. They still need a calculator. I know about 10 people who fit, and every one of them can do this.

[Peep]

I didn't say they know it... just they know how to solve it. They still need a calculator. I know about 10 people who fit, and every one of them can do this.

You may be right. Ten out of ten is a good percentage for sure. Or it could just be "like hangs around like", you know some good numbers guys.

I know two who fit, and I don't think either could solve or be bothered solving this equation.

What can I say, I still believe in idiot savants and single factor handicapping.....

[Ganchrow]

The test is two-fold (IMO). 1. Identify sharps. I suspect anyone making 100k+ a year from sports can do this test without looking anything up.

So what are you waiting for, Chief?... Post the answers.

Let's get this game moving already.

[Panic]

If someone could change this into the form of a bar tab...I could figure it out.

[Mudcat]

I didn't say they know it... just they know how to solve it. They still need a calculator. I know about 10 people who fit, and every one of them can do this.

If we throw out the 100K figure which was arbitrary and just talk about +ev players . . .

My experience is exactly the opposite. I know quite a few of those and I am pretty sure that not a single one would have a clue how to solve this.

Psychology is also an extremely important part of knowing where to make money in betting but it doesn't mean you have to be able to write a second-year paper on Freudian Analysis of Oedipus Complex in Linesmakers.

As I say, it may be a fun math problem if you like math problems, but I wouldn't make any more of it than that as pertains to gambling.

[tomcowley]

Done when first waking up, so probably wrong somewhere.

1) I get 281 winning bets to differentiate from a 0 EV player

2) I get an 82.7% chance that the 54% guy won't meet the test (although also 77.7% chance that he actually makes money by winning 262+ bets over the sample).

3) I get -1058.

[Ganchrow]

Done when first waking up, so probably wrong somewhere.

1) I get 281 winning bets to differentiate from a 0 EV player

2) I get an 82.7% chance that the 54% guy won't meet the test (although also 77.7% chance that he actually makes money by winning 262+ bets over the sample).

3) I get -1058.

Looks good. Well done.

Everyone else can down their pens.

I'll post the explanations sometime after lunch.

[Thremp]

The test is two-fold (IMO). 1. Identify sharps. I suspect anyone making 100k+ a year from sports can do this test without looking anything up. 2. Give people a reality check. It's very hard to tell if someone is a winning player from a short segment (e.g. under 1000) of picks.

You'd be very very wrong

[Ganchrow]

Looks like dcbt was the first to come up with the correct answer to the first subquestion back in post 10.

Well done on that.

[dcbt]

Looks like dcbt was the first to come up with the correct answer to the first subquestion back in post 10.

Well done on that.

Thx. now i'm curious how i screwed up #2 so badly... back to my spreadsheet..

[Data]

1. Over the course of the following season the player will place 500 uncorrelated unit-bet no-push picks at odds of -110. Of those 500 picks, at least how many would he need to win in order for us to be 95% certain he's not a -EV player. (In other words for there to only be a 5% chance that we would mislabel him not -EV when in fact he were actually -EV. (This is know a "false positive" or a "Type I" error.)

I think I have a very basic understanding of this but I will go out on a limb and say that there is no way to answer this question unless we know the percentage of +EV players in the population this player was drawn from and that ignoring the population aspect will produce wrong results considering the nature of the issue that is being researched.

[tomcowley]

Agreed. I almost put (assuming no data-mining issues) in #1. Enough coinflipping monkeys will generate some positive test results even though they're P=0 to be +EV.

[Ganchrow]

1. Over the course of the following season the player will place 500 uncorrelated unit-bet no-push picks at odds of -110. Of those 500 picks, at least how many would he need to win in order for us to be 95% certain he's not a -EV player. (In other words for there to only be a 5% chance that we would mislabel him not -EV when in fact he were actually -EV. (This is know a "false positive" or a "Type I" error.)
2. If we knew a priori that the player were in fact a 54% picker, then based on the test above what would be his probability of being mislabeled a -EV player? (This is known as a "false negative" or a "Type II" error.)
3. If we wanted the player's probability of being mislabeled -EV to be no higher than 25% (say because we had bet at -290 that the player would be labeled +EV), and if the player needed to place all 500 of his bets at one and only one odds level, what would be the lowest magnitude odds at which the player would need to place all his bets in order for the test's Type I error rate to be no more than 5%, as above.
   
   You can assume that the player's real edge in this question #3 is always the same as in #2 and doesn't vary with payout odds.

1. We're looking now to determine the critical number of picks such that a precisely 0 EV player (which in this case implies p = 110 210 ≈ 52.381%) would have no greater than a 5% probability of correctly selecting that number of picks.
   
   One rather tiresome of way of doing this in Excel would be to use the =BINOMDIST() function, changing the number of successes to the lowest integer level that gives a result ≤5%. To wit:
   
   =1-BINOMDIST(successes-1, 500, 110/210, 1) ≈ 4.777%
   
   We see that this giives the highest result ≤5% for successes-1 = 280, implying successes = 281.
   
   A quicker way of obtaining the same answer would be using the inverted binomial function provided by Excel, CRITBINOM(). THis function returns the smallest value for which the cumulative binomial distribution is greater than or equal to the given criterion value. So:
   
   = 1+CRITBINOM(500,110/210,95%) = 281
2. Here, we're looking to determine the probability that a player who really did have a "true" pick rate of 54% would fail the test set up above (in other words, he woulkd pick 280 or fewer correctly). In Excel:
   
   =BINOMDIST(280,500,54%,1) ≈ 82.69%
3. In this case the player can select the odds at which he's allowed to bet. Becuase we're told that his edge on every bet is the same as in #2 above, he bets at an edge of 54%* 21 11 ≈ 3.091%. Hence, if he selects a bet at decimal odds of d, it will have a win probability of p = 103.091% d .
   
   So our goal is then to find the smallest value of p, such that:
   =BINOMDIST(CRITBINOM(500,p/1.030909,0.95),500,p,1) ≤ 25%
   .
   
   We can possibly find a solution using a non linear optimizer such as Excle Solver. Caution should be taken, however, as the problem is nonconvex and the objective function not everywhere differentiable.
   
   Hence, numerous solution that appear to satisfy the problem only do so locally.
   
   TomCowley's solution of -1058 is one such local optimum. (To be fair, I had initially determined the same local solution as he.)
   
   Best I can find (and I may well have missed something better) is p ≈ 93.6207%, implying, at an edge of 3.091%, US-style odds of roughly -988.58,. This give us a type I error of 4.906% ≤ 5% and a type II error of 25%.

Anyway, the point of all this is that there's always going to be a trade-off between the rate of false negative and and the rate of false positives. The more you try to reduce the chances of a non-advantage player erroneously passing a given statistical test (by reducing the cut-off alpha), the more likely it is that your testing you will erroneously exclude real advantage players.

For a given acceptable false positive rate (Type I error or "alpha" error), a lower false negative rate (Type II error or "beta" error) is known as greater test "power". When hypothesis testing, the practioner should always attempt to structure his tests so as they present the highest power possible. One way of increasing test power is by increasing the number of trials.

In sports betting if the number of trials is fixed, higher test power can still be obtained by focusing on shorter odds bets (generally speaking). This will in general allow a tester to keep his false negative rate as low as possible (i.e., the test power as high as possible) for a given alpha level.

[Ganchrow]

I think I have a very basic understanding of this but I will go out on a limb and say that there is no way to answer this question unless we know the percentage of +EV players in the population this player was drawn from and that ignoring the population aspect will produce wrong results considering the nature of the issue that is being researched.

The question implicitly assumes we're looking only at a single bettor, all of the observation of whom are made out-of-sample.

As such, within this (admittedly contrived) framework, data mining is a non-issue.

[Data]

data mining is a non-issue.

Imposter alert!

[Ganchrow]

Imposter alert!

We're testing the out-of-sample record of a single bettor, where "success" is clearly defined ex-ante.

Data mining simply doesn't come in to play within the context of this toy problem.

[Dark Horse]

Just curious. Does anyone actually enjoy solving pointless math problems?

Reminds me of my grandfather. Great guy, but always came up with some science question I didn't care about. He liked giving me money, when I was a little kid. So one day in the garden he tells me he's going to throw these balls through some holes with numbers above them, and get a certain number of points. I had played that game all summer, so I told him "no you won't". He looked at me, measuring up this 8-year old because of what he was about to suggest, and said "I'll bet you 5 bucks." Without losing a beat I said "I'll raise you to 25". You should have seen the surprise on his face... lol

Not only did he end up 25 bucks poorer, but he had to bear the scorn of my father when he found out. For giving me that kind of money. I remember him trying to explain: "It was a fair bet. And he raised." But that only made things worse. He winked at me while my dad went on about it.

I forgot all about this until years later, when I was deep into sports betting. In any case, it doesn't take a college degree in math to recognize an edge. Just a kind grandfather.

[tomcowley]

What's wrong? Afraid time spent learning math will mean less time spent in hot zones?

[Justin7]

Just curious. Does anyone actually enjoy solving pointless math problems?

Enjoy? No. But we understand it, because it makes winning at sports much easier.

[Rain Man]

I'm a bit curious DH. Why are you pissed or seemingly pissed at this. Seems like a diversion that can be enjoyed by those who choose.....and more importantly........easily and harmlessly ignored by all else.

[turnip]

Just curious. Does anyone actually enjoy solving pointless math problems?

aye

[Dark Horse]

aye

Thank you.

All I wanted to know.

I'm a bit curious DH. Why are you pissed or seemingly pissed at this. Seems like a diversion that can be enjoyed by those who choose.....and more importantly........easily and harmlessly ignored by all else.

Let's just say that there is a strong disparity in this think tank between the emphasis that is placed on superior math skills and the field of pattern recognition.

There is a clear tendency in this think tank among math minds to claim that it's their way or the highway. In this particular thread that idea is, subtly, fortified by Justin, who suddenly pulls out a 100K number to support this claim. (really, where did that come from?)

I have come across plenty of good gamblers who could not solve such problems, but who are strong in pattern recognition. And, so far, I have no reason to believe that excellent mathematicians would make good gamblers. So my words were more of an effort to keep it real. The arrogance on display in this think tank, in certain cases, has not failed to keep up with some of the brilliance.

There's someone in the NFL forum - NEP1293- who's developed a system that's 57-35 ATS for the season. That's real. That takes skill. I'm not suggesting that completed systems should be on display, but the manner in which they are developed, the skill and creativity that goes into them, would be of interest to a wider public than this one-sided emphasis on math.

[donjuan]

There's someone in the NFL forum - NEP1293- who's developed a system that's 57-35 ATS for the season. That's real. That takes skill.

LOL. Way to completely miss the entire point of this thread.

[marcoforte]

Thank you.

All I wanted to know.



Let's just say that there is a strong disparity in this think tank between the emphasis that is placed on superior math skills and the field of pattern recognition.

There is a clear tendency in this think tank among math minds to claim that it's their way or the highway. In this particular thread that idea is, subtly, fortified by Justin, who suddenly pulls out a 100K number to support this claim. (really, where did that come from?)

I have come across plenty of good gamblers who could not solve such problems, but who are strong in pattern recognition. And, so far, I have no reason to believe that excellent mathematicians would make good gamblers. So my words were more of an effort to keep it real. The arrogance on display in this think tank, in certain cases, has not failed to keep up with some of the brilliance.

There's someone in the NFL forum - NEP1293- who's developed a system that's 57-35 ATS for the season. That's real. That takes skill. I'm not suggesting that completed systems should be on display, but the manner in which they are developed, the skill and creativity that goes into them, would be of interest to a wider public than this one-sided emphasis on math.

As a trained pathologist, pattern recognition is my trade. I approach sports betting with the same bent using a computer. I appreciate this forum for refreshing the math I've forgotten and pushing me to learn more. Math allows you to verify the significance of the perception of a seen pattern. They work hand in glove, both are needed just like the pink and purple stains which complement each other in my day job.

[Dark Horse]

I agree. It's not one or the other, and I suppose the balance is slightly different for each individual. For me it's mostly pattern recognition and some math to rule out false positives and determine correct bet size. I did learn much here as well, mostly from Ganch.