Estimated Edge Error: A simulation of the Kelly conundrum

[MonkeyF0cker]

There have been countless posts in the Think Tank regarding the virtues of Kelly staking versus flat betting. Most naysayers of the Kelly Criterion are convinced that inaccurate edge estimation creates a dangerous environment for Kelly staking. Because of this, they insist that flat betting is a more suitable alternative for sports betting staking. To address these concerns, along with a personal aversion to the constant rehashing of this topic, I've provided an Excel simulator that allows you to see the differences yourself.



The simulator randomly chooses a line between the maximum line and minimum line settings (max = highest implied win probability). It then applies a randomly chosen edge between the maximum and minimum edge percentages. This is simply a representation of your estimated edge for the wager. Your estimated win percentage is then derived from the line you would be betting and your estimated edge for the wager. An error in your estimated edge is introduced (-10% > edge error > 10%). This is also randomly chosen between the maximum and minimum edge estimation error settings. The error is then subtracted from your edge and an actual win percentage is derived. The simulator then determines whether the play is a win or loss based on the actual win percentage.

Some other settings:

Plays: Self-explanatory I hope. (The number of plays that the simulator runs.)

Kelly Multiplier: The fraction of Kelly that you choose to simulate. This can range from anywhere between 0 and 8.

Flat Betting Percentage: The percent of your bankroll that is flat bet. This is the amount bet TO WIN.

Minimum Bet: The minimum bet that can be placed. If the wager that Kelly or flat betting attempts to place is smaller than this, the bet will be set to zero. Keep this in mind when setting your edges, Kelly multiplier, and starting bankroll. Small edges may not be bet with Kelly where they might with flat betting in certain scenarios.

The simulator also lists each individual flat/Kelly wager and the respective running bankrolls.

What you'll likely find is that when there is an equal chance of your estimated edge being either higher or lower than the actual edge, it tends to even out over time and Kelly remains far superior to flat betting. Only when flat betting approaches the average Kelly wager, does it come close to competing.

When the sample underperforms against the REAL win percentage, Kelly does moderately more poorly than flat betting. However, this is made up and then some when the win percentage regresses to the mean.

And finally, when you think you have an edge on the majority of your wagers but you don't, full Kelly does moderately poorer than flat betting. It's certainly NOT as treacherous as many would have you think. However, half Kelly and other variants show similar results to flat betting. But let's face it, if you THINK you have an edge but really don't, no staking system is going to make you money over the long haul.


In any case, you can come to your own conclusions with this simulator. And any time that the question of flat betting versus the Kelly Criterion is posed, hopefully people will be pointed to this simulator rather than rehashing the topic ad-nauseum once again.

[SportsMushroom]

good job sir

it seems to me that kelly increases variance, makes the highs higher and the lows lower

[thom321]

Great stuff. The big take away for me is that it is extremely easy to confuse variance (randomness) with actual edge, which becomes apparent when running this on a large number of plays.

Monkey, I adjusted the code to run faster on large number of plays so 100,000 plays are completed in < 1 sec. If you care, I can attach it in another post but I won't do so unless you approve.

[FourLengthsClear]

Thanks Monkey.

I did something similar myself a few years back.

[FourLengthsClear]

good job sir

it seems to me that kelly increases variance, makes the highs higher and the lows lower

That is always going to be the case as long as the average wager size is bigger. If you were to increase the flat betting percentage to the 2.5-3.0% range you would see similar peaks/troughs.

[HeeeHAWWWW]

Great stuff. The big take away for me is that it is extremely easy to confuse variance (randomness) with actual edge, which becomes apparent when running this on a large number of plays.

Monkey, I adjusted the code to run faster on large number of plays so 100,000 plays are completed in < 1 sec. If you care, I can attach it in another post but I won't do so unless you approve.

That would be great, I was running these big sims a few times.

Might be interesting to automate multiple runs instead, to explore variance in bankroll growth over a more realistic number of plays (eg run 100 sims of 1k plays, rather than 1 x 100k).

[Salmon Steak]

Very smart guy. I would probably blow my brains out after that big drop in the middle though.

[MonkeyF0cker]

Great stuff. The big take away for me is that it is extremely easy to confuse variance (randomness) with actual edge, which becomes apparent when running this on a large number of plays.

Monkey, I adjusted the code to run faster on large number of plays so 100,000 plays are completed in < 1 sec. If you care, I can attach it in another post but I won't do so unless you approve.

Sure. Go ahead.

I was going to optimize it but I got lazy.

[MonkeyF0cker]

That would be great, I was running these big sims a few times.

Might be interesting to automate multiple runs instead, to explore variance in bankroll growth over a more realistic number of plays (eg run 100 sims of 1k plays, rather than 1 x 100k).

I figured people could use the base code and add a loop for multiple sims or track it in another spreadsheet to analyze separately. It certainly wouldn't be hard to do and perhaps when I get some time, I'll add a charted feature like that myself.

[thom321]

Slightly adjusted simulation workbook

In the version I attached, I store the data in arrays and write the arrays to the worksheet when the simulation is done, rather than write the data to the worksheet as the simulation is running. I also disabled screen updating and automatic calculation when the simulation is running.

A simulation of 100,000 plays now take less than a second on my computer.

I also disabled the message box and instead write the message to a cell in the sheet. Everything else is the way Monkey did it.

[mathdotcom]

What if you give naysayers the ultimate credit by assuming there might be a correlation between your model predicting a huge edge and the possibility that there's some information you're unaware of?

i.e. if your model predicts a 20% edge when normally it's 1-2%, your reaction as a modeler is usually "I must have ****** up", not "grandma I need to borrow 100k to bet the Pirates"

[MonkeyF0cker]

Okay. Here's my final version.

It includes multiple iteration simulations as requested. It also includes the optimized code that Thom provided.

[MonkeyF0cker]

What if you give naysayers the ultimate credit by assuming there might be a correlation between your model predicting a huge edge and the possibility that there's some information you're unaware of?

i.e. if your model predicts a 20% edge when normally it's 1-2%, your reaction as a modeler is usually "I must have ****** up", not "grandma I need to borrow 100k to bet the Pirates"

Garbage in, garbage out. Like I said in my original post, if you're miscalculating your edge to the point of negative expectation (or grossly overbetting) with regularity, you won't be successful with whatever staking strategy that you use. If your model is puking on you then you shouldn't be betting it regardless. If you were flat betting would you place a wager on that game? You probably shouldn't be.

But that also assumes that you're a modeler. You don't need to model games in order to use Kelly.

[HeeeHAWWWW]

Appreciated guys, very impressive. I was going to give it a crack tomorrow, but usually my attempts at programming/scripting result in things so unoptimized you need a supercomputer just to run them.

[jolmscheid]

Very interesting..so if one is going to use half kelly, is that about the same as flat Betting? It seems like it anyways..

[mathdotcom]

I just said modeling so that you can precisely calculate your edge given your model. A handicapper could in theory do this, I guess...

So their claim is that if you estimate your edge as x% but this estimate is actually x% + error, where the error is uniform or normal or whatever, that flat betting is superior?

If that's true I'm surprised you went to the bother of simulation to refute that claim.

[MonkeyF0cker]

I just said modeling so that you can precisely calculate your edge given your model. A handicapper could in theory do this, I guess...

So their claim is that if you estimate your edge as x% but this estimate is actually x% + error, where the error is uniform or normal or whatever, that flat betting is superior?

If that's true I'm surprised you went to the bother of simulation to refute that claim.

I wouldn't have if it weren't for years of persistence.

[jolmscheid]

So can someone help with
My question about half kelly vs full kelly? If one does half kelly, is it basically close to flat betting?

It seems like if one does not use full kelly, then one is I have essence not reaping the benefits and is closer to flat betting I have the end

[MonkeyF0cker]

Can't you run the sim? That's what it's there for.

The short answer is: it depends.

There are many factors that go into it. Flat betting %, average line, range of edges, etc.

If you're betting mostly -110's and the flat betting % is near the half Kelly stake then they will have almost identical results.

[jolmscheid]

Thanks Monkey...appreciate it

[FourLengthsClear]

So can someone help with
My question about half kelly vs full kelly? If one does half kelly, is it basically close to flat betting?

No, not necessarily.

It seems like if one does not use full kelly, then one is I have essence not reaping the benefits and is closer to flat betting I have the end

Would be true only if edge was known. What these simulations do is help to put some perspective on the margin of error in edge calculations that can be 'absorbed' before Kelly/Fractional Kelly are not viable.

[MonkeyF0cker]

For a standard comparison of flat betting versus Kelly without introducing error, just set the min and max edge estimation error to zero.

[SportsMushroom]

That is always going to be the case as long as the average wager size is bigger. If you were to increase the flat betting percentage to the 2.5-3.0% range you would see similar peaks/troughs.

I disagree

kelly increases variance by a lot more


first off you have variance regarding your handicapping, which evens out in both kelly and flat betting


but in kelly you have additional variance occuring, variance related to the miscalculation of edge, which would lead to overbetting when you overestimate your edge, and underbetting when you underestimate your edge, which as the OP said will even out in the long run, but it is still a factor that increases your variance by as much or as little as your miscalculation of edge

[FourLengthsClear]

I disagree

kelly increases variance by a lot more


first off you have variance regarding your handicapping, which evens out in both kelly and flat betting


but in kelly you have additional variance occuring, variance related to the miscalculation of edge, which would lead to overbetting when you overestimate your edge, and underbetting when you underestimate your edge, which as the OP said will even out in the long run, but it is still a factor that increases your variance by as much or as little as your miscalculation of edge

All of which is also true with flat betting.
Try it out, enter a flat bet stake of 5% in any of the sims posted above and see what happens in terms of the severity of run ups and drawdowns! Make it 10% and it would approach certainty that you go broke.

In my first post to you I said 2.5%-3.0% because that is equivalent to the baseline min/max Kelly edge range that Monkey had entered.

[jolmscheid]

All of which is also true with flat betting.
Try it out, enter a flat bet stake of 5% in any of the sims posted above and see what happens in terms of the severity of run ups and drawdowns! Make it 10% and it would approach certainty that you go broke.

In my first post to you I said 2.5%-3.0% because that is equivalent to the baseline min/max Kelly edge range that Monkey had entered.

Good insight 4LC...even though Kelly is the ultimate way to bet, why not RISK 2% of your roll on each bet that you KNOW you have an edge and re-calc. daily....perhaps that would take the ups and downs of mis-calculating the edge more? But then again, one could then be sacrificing growth....so to me, it ultimately comes down to kelly = maximized growth with more ups and downs perhaps....and flat betting = lesser growth but hopefully more even keeled.....thoughts?

[SportsMushroom]

All of which is also true with flat betting. Try it out, enter a flat bet stake of 5% in any of the sims posted above and see what happens in terms of the severity of run ups and drawdowns! Make it 10% and it would approach certainty that you go broke. In my first post to you I said 2.5%-3.0% because that is equivalent to the baseline min/max Kelly edge range that Monkey had entered.

Good insight 4LC...even though Kelly is the ultimate way to bet, why not RISK 2% of your roll on each bet that you KNOW you have an edge and re-calc. daily....perhaps that would take the ups and downs of mis-calculating the edge more? But then again, one could then be sacrificing growth....so to me, it ultimately comes down to kelly = maximized growth with more ups and downs perhaps....and flat betting = lesser growth but hopefully more even keeled.....thoughts?

how is that good insight? all he did was state the obvious, if you flat bet at 5% or 10% then your balance will reach zero at some point, no shit shirlock

you will go broke with kelly as well if you bet that big a percentage of your bankroll, all you did was state a statistical inevitability and present is as a fact that proves that kelly is superior

what I said was that due to the subjective nature of the edge calculation, kelly incorporates a variable that increases variance, a variable that is not existent is flat betting since flat betting does not take edge into consideration, what you stated does not refute this fact


to all the kelly die hards here, stop taking the criticisms of kelly so personally, yes kelly is in theory optimal to flat betting, but only so far as the calculation of edge is accurate, if you sit opposite me and tell me that a methodology that uses subjective and possibly misleading figures is better than one that doesnt then I will not continue this discussion with you as you are obviously biased

[HeeeHAWWWW]

.....if you sit opposite me and tell me that a methodology that uses subjective and possibly misleading figures is better than one that doesnt then I will not continue this discussion with you as you are obviously biased

Your choice of what %roll to flatbet is just as subjective.

At least Kelly stakesizing is based on something.

[SportsMushroom]

Your choice of what %roll to flatbet is just as subjective.

At least Kelly stakesizing is based on something.

how does that change the fact that by incorporating edge into your calculation, you increase variance?

in flat betting, variance manifests itself only by convergence to the mean in regards to win percentage

by using kelly, you still experience variance regarding win percentage, but this time you also have to face variance that occurs from misscalculating your edge, were you will overbet when overestimating your edge and underbetting when you underestimate your edge

having to deal with both variances is bad enough, but imagine when these two variances overlap, and you face a period when you are on a losing streak while at the same time overbetting because you miscalculated your edge

[jgilmartin]

If you flat bet, for example, 2% per game, you are still (inadvertently) incorporating your edge, you are just doing it using an arbitrary figure.

[SportsMushroom]

If you flat bet, for example, 2% per game, you are still (inadvertently) incorporating your edge, you are just doing it using an arbitrary figure.

yes, because you dont know what that figure really is


trying to calculate an incalculable number is something that boggles my mind

although I am sure that people that do use kelly believe that they are calculating their edge right, but I just dont see how that is possible

to calculate your edge accurately you need to know that

1. the odds setters price is not accurate
2. that you have the ability to calculate a more accurate price than the odds setter
3. that the odds setters have done such a horrible job that your edge is enough to cover the vig and produce a profit
4. to do all this I have to assume that the odds setter has incomplete information, and that you have more information than the odds setter
5. yeah right

[FourLengthsClear]

yes, because you dont know what that figure really is


trying to calculate an incalculable number is something that boggles my mind

although I am sure that people that do use kelly believe that they are calculating their edge right, but I just dont see how that is possible

to calculate your edge accurately you need to know that

1. the odds setters price is not accurate
2. that you have the ability to calculate a more accurate price than the odds setter
3. that the odds setters have done such a horrible job that your edge is enough to cover the vig and produce a profit
4. to do all this I have to assume that the odds setter has incomplete information, and that you have more information than the odds setter
5. yeah right

Again, all of this is also true with flat betting.

For the third time the severity of run ups and drawdowns experienced as a result of variance are almost wholly dependant on the size of the bets made NOT the method used to arrive at that bet size.

The basis of making a bet utilising (fractional) Kelly is "I have an edge which I can quantify to within an acceptable margin of error". The basis of flat betting is "I believe I have an edge but I cannot quantify it". How can one of those "boggle your mind" and yet you find the other rational?

Monkey and Thom have provided a tool which gives some insight on the 'margin of error' issue, nothing more than that, it demonstrates to those of us that do use Kelly what we already believed.

[SportsMushroom]

The basis of making a bet utilising (fractional) Kelly is "I have an edge which I can quantify to within an acceptable margin of error". The basis of flat betting is "I believe I have an edge but I cannot quantify it". How can one of those "boggle your mind" and yet you find the other rational?

yes I cannot quantify my edge accurately, and that makes me a pragmatist


does believing you can quantify your edge make you right? no it does not, only when someone provides a method of accurately calculating edge I will choose kelly over flat betting, flat betting protects me from the downside of making a wrong calculation


anyhow I am not sure of this but didnt you state on this forum that you are an arbitrator? if so, how does an arbitrator advocate kelly? thats like a scientist advocating religion

[FourLengthsClear]

yes I cannot quantify my edge accurately, and that makes me a pragmatist

does believing you can quantify your edge make you right? no it does not, only when someone provides a method of accurately calculating edge I will choose kelly over flat betting

anyhow I am not sure of this but didnt you state on this forum that you are an arbitrator? if so, how does an arbitrator advocate kelly? thats like a scientist advocating religion

That is completely your choice but does not change the fact that what you said starting in post #2 was incorrect.

I have never been an arbitageur. I do engage in trading in addition to regular betting.

[SportsMushroom]

That is completely your choice but does not change the fact that what you said starting in post #2 was incorrect.

I have never been an arbitageur. I do engage in trading in addition to regular betting.

no it is correct, using subjective estimates in place of what should be a precise figure increases variance, its common sense, even the simulations the op run indicate that


but I will never convince you otherwise and neither will you convince me, so lets just leave it at the fact that we have differing opinions


let me just make a distinction here, when I say it increases variance I do not mean in the traditional sense of wins and losses, but rather, variance in the fluctuations of the bank roll

[Kolotoure]

How can you be sure you have an edge yet be unable to quantify it?