Estimated Edge Error: A simulation of the Kelly conundrum

[FourLengthsClear]

no it is correct, using subjective estimates in place of what should be a precise figure increases variance, its common sense, even the simulations the op run indicate that


but I will never convince you otherwise and neither will you convince me, so lets just leave it at the fact that we have differing opinions


let me just make a distinction here, when I say it increases variance I do not mean in the traditional sense of wins and losses, but rather, variance in the fluctuations of the bank roll

Just so I have your position straight, what you are saying is that a staking plan

a) Based on Kelly which results in an average wager size of 3% or BR

will produce significantly steeper run ups and drawdowns than staking plan

b) Based on a fixed wager size of 3% of BR.


Is that really what you are saying?

[SportsMushroom]

Just so I have your position straight, what you are saying is that a staking plan

a) Based on Kelly which results in an average wager size of 3% or BR

will produce significantly steeper run ups and drawdowns than staking plan

b) Based on a fixed wager size of 3% of BR.


Is that really what you are saying?

no, I am saying that using inaccurate calculations of edge will do that

you just said it yourself, kelly uses AVERAGE, AVERAGE , AVERAGE, which means that you could be betting max on losing streaks and minimum on winning streaks which will minimize wins and maximize loses. you could also miscalculate your edge and be betting max on games with a lower edge and low on games with a higher edge

in the end it should all even out, but that is not the point, my point is there is greater variance with kelly, and if you dont understand then its not my fault

[SportsMushroom]

How can you be sure you have an edge yet be unable to quantify it?

that is not exactly what I am saying

all I am saying is that varying betsize according to edge will increase variance because ultimately your calculations cannot not be accurate

[subs]

if u run the sim for 20 minutes u might be surprised or maybe not...

[SportsMushroom]

people in here are supposed to be smart but I guess I was wrong, all I did was point out the obvious, but people choose to disregard it as non sense because it messes up their world view

I hate to go all wiki on you but I am too tired to search for a better source or run simulations of my own:

A natural assumption is that taking more risk increases the probability of both very good and very bad outcomes. One of the most important ideas in Kelly is that betting more than the Kelly amount decreases the probability of very good results, while still increasing the probability of very bad results. Since in reality we seldom know the precise probabilities and payoffs, and since overbetting is worse than underbetting, it makes sense to err on the side of caution and bet less than the Kelly amount.
Kelly assumes sequential bets that are independent (later work generalizes to bets that have sufficient independence). That may be a good model for some gambling games, but generally does not apply in investing and other forms of risk-taking.
The Kelly property appears "in the long run" (that is, it is an asymptotic property). To a person, it matters whether the property emerges over a small number or a large number of bets. It makes sense to consider not just the long run, but where losing a bet might leave one in the short and medium term as well. A related point is that Kelly assumes the only important thing is long-term wealth. Most people also care about the path to get there. Kelly betting leads to highly volatile short-term outcomes which many people find unpleasant, even if they believe they will do well in the end.




kelly is volatile fact, and for more than one reason, because of that it is more likely to go broke with kelly than flat betting


also in the long run, if the average wager for kelly and flat betting is the same then you will get equal results and with flat betting you dont get the increased variance (volatility)


in the begginning it was just friendly conversation, but it seems that some people have a problem with accepting different opinions, I guess the same people that are in players talk are in here as well

I cant believe that people in here pretend to be math wizes but cant understand the basic concept of variance in input = variance in results

[subs]

sportsmushroom, wow dude...

i like money so i bet fractional kelly, variance is fine... not busto yet.

i don't see how any1 is giving u a hard time in here. i accept ur POV, that's all fine....

i get better results, maybe u don't.

[MonkeyF0cker]

kelly is volatile fact, and for more than one reason, because of that it is more likely to go broke with kelly than flat betting


also in the long run, if the average wager for kelly and flat betting is the same then you will get equal results and with flat betting you dont get the increased variance (volatility)


in the begginning it was just friendly conversation, but it seems that some people have a problem with accepting different opinions, I guess the same people that are in players talk are in here as well

I cant believe that people in here pretend to be math wizes but cant understand the basic concept of variance in input = variance in results

I think you should try running the multiple iteration sim. I just ran a 1000 iteration sim 10 times with FULL Kelly versus 2% flat betting, 1000 plays, between -200 and +200, 0.5%-5% edge, and -1% to -5% error (meaning that I'm overbetting EVERY play) and Kelly outperformed flat betting every time. While that's just a small sample, surely you could track it yourself over a larger sample of runs and perhaps come to a different conclusion.

[MonkeyF0cker]

You might also want to try smaller fractions of Kelly as well.

[MonkeyF0cker]

If your goal is to minimize bankroll fluctuation, your bet size should be zero. If your goal is to maximize bankroll growth, well... some variant of Kelly.

[FourLengthsClear]

no, I am saying that using inaccurate calculations of edge will do that

a) you just said it yourself, kelly uses AVERAGE, AVERAGE , AVERAGE, which means that you could be betting max on losing streaks and minimum on winning streaks which will minimize wins and maximize loses. you could also miscalculate your edge and be betting max on games with a lower edge and low on games with a higher edge

in the end it should all even out, but that is not the point, b) my point is there is greater variance with kelly, and if you dont understand then its not my fault

a) I said no such thing.

b) You can repeat this another 50 times and it still will not be true. Volatility is a function of the amounts wagered and variance, that's all.

If I am offered +115 on 1000 coinflips, a scenario where edge is known, I am inevitably going to experience substantial volatility based on a full Kelly wager of 6.52%. I trust you would agree that this is still the optimal wager amount despite that prospect.

In terms of sportsbetting and the issue of miscalculation of edge there are two things.

i) I doubt you will find anyone here who applies full Kelly. A sensible Kelly multiplier that allows a margin of error to avoid overbetting is a must.

ii) The flat bettor is also prone to the same issue. Despite not having made an edge calculation he is still making a wager based on a perceived edge which correlates with a specific Kelly edge. You could argue that the flat bettor has a narrower range of potential error but that would be a tough case to make not least because of the issue highlighted by mathdotcom in post #11.

[HeeeHAWWWW]

Trying to dream up worst case scenarios for Kelly.

Low RoI: set edge at 1 to 5%
Wild error range: -15 to +11%
Aggressive Kelly: 0.5

Others: 1000 plays, 1% flat, line -500 to +500


That's going to be an average real edge of 1%, which means the optimal bet size is also 1%. I've cheated and set flat to that to contine the worst case scenario.

Kelly comes out basically the same (about +2.5%) on average via a 32k multi-sim.

The fun thing: if you calculate the cumulative variance, 0.5 Kelly actually has lower with this setup, typically by a factor of 2x to 5x. It's because of the 1/x staking element I assume, although too hungover to actually think about it.


To exclude that, I did the same run with fixed odds (-105 to -105), and Kelly at 0.33. That means, on average, the Kelly stake will be the same % of your roll (1%). Result? Average bankroll is again basically the same between flat and Kelly, but this time so is variance ..... despite the crazy edge estimation errors.

[thom321]

Kelly vs Flat, results from multiple simulations

Not sure if this will settle anything or add more fuel to the fire.

Since Monkey made it very easy to run multiple simulations, I decided to add some additional output variables and run batches of simulations to see if some obvious pattern would emerge.


I left the settings the way they were as far as edge, error etc. I adjusted the code to allow for running repeated simulations e.g. running 100 consecutive simulations where each simulation was e.g. 100 iterations of 1,000 plays. One “batch” as I refer to them is hence a simulation using x iterations of y number of plays.
I then calculated, for each batch, the average, median, max and min ending bank roll and also the % of time where the ending Kelly bankroll was larger than the ending Flat bankroll.


I ran this for
100 batches of 100 iterations times 1,000 plays
100 batches of 100 iterations times 10,000 plays
10 batches of 100 iterations times 100,000 plays


It was taking too much time for the 100,000 plays but when you see the output you will see that it doesn’t seem to matter. The results for each batch when running the simulations on 100,000 plays was very similar. This is either how the math works or it has something to do with the Excel random number generator not producing random enough numbers
.
The conclusion is that for simulations of 100 x 1,000 plays, Kelly produces a slightly higher average bankroll but only produces a higher average bankroll about 50% of the time. Kelly does generate a higher average max and a lower min.


On simulations of 100 x 10,000 plays, Kelly has a higher average ending bankroll but consistently has a lower median ending bankroll and has a lower ending bankroll that flat betting about 70% of the time.


In the 10 x 100,000 plays the average and median ending bankroll for Kelly is always lower than Flat and appears to generate a lower ending bankroll over 100,000 plays every time. Keep in mind this is using the default settings provided by Monkey as far as edge and error and could also be a function of some flaw in Excels random number generator. I don’t know enough about that or Kelly math to have a view.


I have included the worksheet that has the results from each simulation. Cell A1 has the settings that were used each time. Nothing changed between the different simulations other than the number of plays used.


If anyone cares, I can upload the adjusted workbook I used to generate these results.

[MonkeyF0cker]

Not sure if this will settle anything or add more fuel to the fire.

Since Monkey made it very easy to run multiple simulations, I decided to add some additional output variables and run batches of simulations to see if some obvious pattern would emerge.


I left the settings the way they were as far as edge, error etc. I adjusted the code to allow for running repeated simulations e.g. running 100 consecutive simulations where each simulation was e.g. 100 iterations of 1,000 plays. One “batch” as I refer to them is hence a simulation using x iterations of y number of plays.
I then calculated, for each batch, the average, median, max and min ending bank roll and also the % of time where the ending Kelly bankroll was larger than the ending Flat bankroll.


I ran this for
100 batches of 100 iterations times 1,000 plays
100 batches of 100 iterations times 10,000 plays
10 batches of 100 iterations times 100,000 plays


It was taking too much time for the 100,000 plays but when you see the output you will see that it doesn’t seem to matter. The results for each batch when running the simulations on 100,000 plays was very similar. This is either how the math works or it has something to do with the Excel random number generator not producing random enough numbers
.
The conclusion is that for simulations of 100 x 1,000 plays, Kelly produces a slightly higher average bankroll but only produces a higher average bankroll about 50% of the time. Kelly does generate a higher average max and a lower min.


On simulations of 100 x 10,000 plays, Kelly has a higher average ending bankroll but consistently has a lower median ending bankroll and has a lower ending bankroll that flat betting about 70% of the time.


In the 10 x 100,000 plays the average and median ending bankroll for Kelly is always lower than Flat and appears to generate a lower ending bankroll over 100,000 plays every time. Keep in mind this is using the default settings provided by Monkey as far as edge and error and could also be a function of some flaw in Excels random number generator. I don’t know enough about that or Kelly math to have a view.


I have included the worksheet that has the results from each simulation. Cell A1 has the settings that were used each time. Nothing changed between the different simulations other than the number of plays used.


If anyone cares, I can upload the adjusted workbook I used to generate these results.

The problem with this is that it's a losing strategy. The median bankroll for both flat betting and Kelly after 10,000 plays is almost identical - pretty much broke. The difference is simply that the minimum wager limit is met with the flat betting stake at a set amount. Kelly will still bet a wager with a large perceived edge at the same bankroll amount. At the same time, Kelly's average is higher 97 out of the 100 simulations.

In the 1000 play sim, Kelly's average is higher 98 out of the 100 times and median bankroll is higher 58 out of the 100 sims.

The 100,000 play sim is far too small of a sample to deduce anything even remotely conclusive. However, you can assume that you'd be broke with either flat betting or Kelly.

What you should take from this is that Kelly will not bankrupt you any faster than flat betting wagers where you overestimate your edge. In either case, you'll need to recognize this and adjust because you'll likely go broke with whichever staking system you use. However, Kelly actually gives you an equal or better chance of surviving such a scenario over a fairly lengthy stretch rather than a worse chance.

[MonkeyF0cker]

For a better representation, try starting with a bankroll much higher than $1000. Therein lies the issue that you're running into with those sims.

[thom321]

Monkey,
I was going to ask but you partially beat me to it. Rather than starting with a higher bankroll, which might lead to an Excel overflow error on large simulations (that might happen anyway), starting with a lower minimum bet should accomplish the same thing right? Other than that, what are reasonable settings to use to get a good representation of Kelly vs Flat?

[MonkeyF0cker]

You could do both. I'd probably just set the starting bankroll to $10k-$20k and compare results. You shouldn't have to worry about overflowing the double data type with 10,000 plays.

You could potentially run into the issue with 100,000 plays, however. But that's not realistic anyway. There is no doubt you'll go broke the vast majority of the time after grossly overestimating your edge for that length of time with either staking strategy.

[thom321]

Simulation results from 1 billion plays (times 3)

I decided to spend/waste some more time on running simulations. In my opinion, to compare a flat bet % with Kelly, one has to control for bet size (as % of bankroll). Otherwise, comparisons are meaningless. If your average bet is very different of course the return distribution would be as well. I played around with the numbers a bit and using a Kelly multiplier of 0.94 (in combination with the line and edge numbers below) results in an average bet size as a % of bankroll that is very similar to a flat bet (to win) of 2%.

The only variables I changed between my simulation runs are the minimum and maximum edge estimation error.

There are the settings I used for all simulations:

Maximum Line: -200
Minimum Line: 200
Maximum Estimated Edge %: 5%
Minimum Estimated Edge %: 0.5%
Number of Plays: 10000
Kelly Multiplier: 0.94
Flat Betting %: 2%
Starting Bankroll: 1000
Minimum Bet: 0.05
Number of Simulations: 1000
Number of batches: 100

A “batch” in this context is a simulation using x plays, repeated y times. So in my simulations 1 batch = 1,000 simulations of 10,000 plays.

In Monkey’s simulation sheet, the ending bankroll using flat betting and using Kelly is calculated for each simulation. So if I run 1,000 simulations, I will end up with a list of 1,000 ending bankroll numbers for each method. As an additional variable, I calculated the % of time that the ending Kelly bankroll was higher than the Flat bankroll. This is stored as an additional output variable for each “batch” I ran. I also calculated the max, min, average, and median ending bank roll for each simulation run.

In the tables below are the results from running 100 batches of 1,000 simulations of 10,000 plays. So each table is created from 1 billion plays.
Above each table I have specified the maximum and minimum edge estimation error settings used, which were the only variables that changed between the three sets of batches.

Please note that the average bet, as a percentage of bankroll was very consistent between Flat and Kelly for each of the batches.

Sorry about the look of the table but I couldn't figure out how to copy it from Excel and keep the table structure.

Table 1: Max edge estimation error -1%, Min edge estimation error -5%
Avg Flat Bank 890
Avg Kelly Bank 1,818
Median Flat Bank 68
Median Kelly Bank 109
Max Flat Bank 253,830
Max Kelly Bank 221,015
Min Flat Bank 1
Min Kelly Bank 1
Ending Kelly>Flat 64%
Avg Flat Wager, % of bank 2.18%
Avg Kelly Wager, % of bank 2.17%

Table 2: Max edge estimation error -1%, Min edge estimation error -3%
Avg Flat Bank 4,275
Avg Kelly Bank 10,386
Median Flat Bank 370
Median Kelly Bank 652
Max Flat Bank 1,027,238
Max Kelly Bank 1,284,960
Min Flat Bank 1
Min Kelly Bank 1
Ending Kelly>Flat 67%
Avg Flat Wager, % of bank 2.19%
Avg Kelly Wager, % of bank 2.18%

Table 3: Max edge estimation error 0%, Min edge estimation error 0%
Avg Flat Bank 119,715
Avg Kelly Bank 401,510
Median Flat Bank 11,266
Median Kelly Bank 23,836
Max Flat Bank 25,796,194
Max Kelly Bank 76,749,434
Min Flat Bank 19
Min Kelly Bank 12
Ending Kelly>Flat 73%
Avg Flat Wager, % of bank 2.19%
Avg Kelly Wager, % of bank 2.19%

I am not a statistician, nor do I have a vested interest in proving that Kelly does/doesn't work. It would be great if it does since then "all" I have to do is to find a method that provides me with a consistent and sustainable long term edge and a bookmaker that won't limit me. That part should be easy....

However, given the results above, I am interested in knowing where I went "wrong" since Kelly does produce better simulation results across the board. Which simulation variable is it that creates an unrealistic situation that favors Kelly over flat, assuming that Kelly isn't actually a better method than flat bet? Or are the differences in my simulations over 1 billion plays too small to say that Kelly did "significantly" better?

[CrimsonQueen]

This sim is incredible. I love it. Well done.

[Inkwell77]

Good thread here. This sim as well as backtesting on covers consensus made me a huge believer in betting less. Look at the guys with over 10,000 plays, it is mind boggling.

[Wilbo86]

Great thread and great work on the simulator and vairants, has helped me get my mind around the topic. Not sure how old the thread is or if anyone is still paying attention to it, but I have a few queries.

I'm trying to ascertain if I'm currently using the correct betting size strategy. For context I have been betting combat sports with a medium bankroll using fractional kelly (usually 1/8) although I don't stick to this rigidly and when i think I have a significant edge in a given fight I'll use a multiple. I'm happy with my results so far but I want to make sure I'm maximizing my potential returns.

My take away from the simulator is that I would derive higher BR growth from using a kelly multiplier of 1 or higher (instead of 1/8th). Given that my fractional kelly bets generally result in a fairly standard/small bet (around 2-3% bankroll), does my current strategy basically equate using flat betting and using kelly when my edge is obvious and quantifiable?

[lamichaeljames]

Sim is great.

[jasi]

This is an older thread, but I was wondering if someone can explain to me how to use the Run Sim. When I click Run Sim it does nothing, as if its not even a click option.

Thanks in advance.