Estimated Edge Error: A simulation of the Kelly conundrum

Trying to dream up worst case scenarios for Kelly.

Low RoI: set edge at 1 to 5%
Wild error range: -15 to +11%
Aggressive Kelly: 0.5

Others: 1000 plays, 1% flat, line -500 to +500


That's going to be an average real edge of 1%, which means the optimal bet size is also 1%. I've cheated and set flat to that to contine the worst case scenario.

Kelly comes out basically the same (about +2.5%) on average via a 32k multi-sim.

The fun thing: if you calculate the cumulative variance, 0.5 Kelly actually has lower with this setup, typically by a factor of 2x to 5x. It's because of the 1/x staking element I assume, although too hungover to actually think about it.


To exclude that, I did the same run with fixed odds (-105 to -105), and Kelly at 0.33. That means, on average, the Kelly stake will be the same % of your roll (1%). Result? Average bankroll is again basically the same between flat and Kelly, but this time so is variance ..... despite the crazy edge estimation errors.

Not sure if this will settle anything or add more fuel to the fire.

Since Monkey made it very easy to run multiple simulations, I decided to add some additional output variables and run batches of simulations to see if some obvious pattern would emerge.


I left the settings the way they were as far as edge, error etc. I adjusted the code to allow for running repeated simulations e.g. running 100 consecutive simulations where each simulation was e.g. 100 iterations of 1,000 plays. One “batch” as I refer to them is hence a simulation using x iterations of y number of plays.
I then calculated, for each batch, the average, median, max and min ending bank roll and also the % of time where the ending Kelly bankroll was larger than the ending Flat bankroll.


I ran this for
100 batches of 100 iterations times 1,000 plays
100 batches of 100 iterations times 10,000 plays
10 batches of 100 iterations times 100,000 plays


It was taking too much time for the 100,000 plays but when you see the output you will see that it doesn’t seem to matter. The results for each batch when running the simulations on 100,000 plays was very similar. This is either how the math works or it has something to do with the Excel random number generator not producing random enough numbers
.
The conclusion is that for simulations of 100 x 1,000 plays, Kelly produces a slightly higher average bankroll but only produces a higher average bankroll about 50% of the time. Kelly does generate a higher average max and a lower min.


On simulations of 100 x 10,000 plays, Kelly has a higher average ending bankroll but consistently has a lower median ending bankroll and has a lower ending bankroll that flat betting about 70% of the time.


In the 10 x 100,000 plays the average and median ending bankroll for Kelly is always lower than Flat and appears to generate a lower ending bankroll over 100,000 plays every time. Keep in mind this is using the default settings provided by Monkey as far as edge and error and could also be a function of some flaw in Excels random number generator. I don’t know enough about that or Kelly math to have a view.


I have included the worksheet that has the results from each simulation. Cell A1 has the settings that were used each time. Nothing changed between the different simulations other than the number of plays used.


If anyone cares, I can upload the adjusted workbook I used to generate these results.

Originally Posted by thom321

Not sure if this will settle anything or add more fuel to the fire.

Since Monkey made it very easy to run multiple simulations, I decided to add some additional output variables and run batches of simulations to see if some obvious pattern would emerge.


I left the settings the way they were as far as edge, error etc. I adjusted the code to allow for running repeated simulations e.g. running 100 consecutive simulations where each simulation was e.g. 100 iterations of 1,000 plays. One “batch” as I refer to them is hence a simulation using x iterations of y number of plays.
I then calculated, for each batch, the average, median, max and min ending bank roll and also the % of time where the ending Kelly bankroll was larger than the ending Flat bankroll.


I ran this for
100 batches of 100 iterations times 1,000 plays
100 batches of 100 iterations times 10,000 plays
10 batches of 100 iterations times 100,000 plays


It was taking too much time for the 100,000 plays but when you see the output you will see that it doesn’t seem to matter. The results for each batch when running the simulations on 100,000 plays was very similar. This is either how the math works or it has something to do with the Excel random number generator not producing random enough numbers
.
The conclusion is that for simulations of 100 x 1,000 plays, Kelly produces a slightly higher average bankroll but only produces a higher average bankroll about 50% of the time. Kelly does generate a higher average max and a lower min.


On simulations of 100 x 10,000 plays, Kelly has a higher average ending bankroll but consistently has a lower median ending bankroll and has a lower ending bankroll that flat betting about 70% of the time.


In the 10 x 100,000 plays the average and median ending bankroll for Kelly is always lower than Flat and appears to generate a lower ending bankroll over 100,000 plays every time. Keep in mind this is using the default settings provided by Monkey as far as edge and error and could also be a function of some flaw in Excels random number generator. I don’t know enough about that or Kelly math to have a view.


I have included the worksheet that has the results from each simulation. Cell A1 has the settings that were used each time. Nothing changed between the different simulations other than the number of plays used.


If anyone cares, I can upload the adjusted workbook I used to generate these results.

The problem with this is that it's a losing strategy. The median bankroll for both flat betting and Kelly after 10,000 plays is almost identical - pretty much broke. The difference is simply that the minimum wager limit is met with the flat betting stake at a set amount. Kelly will still bet a wager with a large perceived edge at the same bankroll amount. At the same time, Kelly's average is higher 97 out of the 100 simulations.

In the 1000 play sim, Kelly's average is higher 98 out of the 100 times and median bankroll is higher 58 out of the 100 sims.

The 100,000 play sim is far too small of a sample to deduce anything even remotely conclusive. However, you can assume that you'd be broke with either flat betting or Kelly.

What you should take from this is that Kelly will not bankrupt you any faster than flat betting wagers where you overestimate your edge. In either case, you'll need to recognize this and adjust because you'll likely go broke with whichever staking system you use. However, Kelly actually gives you an equal or better chance of surviving such a scenario over a fairly lengthy stretch rather than a worse chance.

For a better representation, try starting with a bankroll much higher than $1000. Therein lies the issue that you're running into with those sims.

Monkey,
I was going to ask but you partially beat me to it. Rather than starting with a higher bankroll, which might lead to an Excel overflow error on large simulations (that might happen anyway), starting with a lower minimum bet should accomplish the same thing right? Other than that, what are reasonable settings to use to get a good representation of Kelly vs Flat?

You could do both. I'd probably just set the starting bankroll to $10k-$20k and compare results. You shouldn't have to worry about overflowing the double data type with 10,000 plays.

You could potentially run into the issue with 100,000 plays, however. But that's not realistic anyway. There is no doubt you'll go broke the vast majority of the time after grossly overestimating your edge for that length of time with either staking strategy.

I decided to spend/waste some more time on running simulations. In my opinion, to compare a flat bet % with Kelly, one has to control for bet size (as % of bankroll). Otherwise, comparisons are meaningless. If your average bet is very different of course the return distribution would be as well. I played around with the numbers a bit and using a Kelly multiplier of 0.94 (in combination with the line and edge numbers below) results in an average bet size as a % of bankroll that is very similar to a flat bet (to win) of 2%.

The only variables I changed between my simulation runs are the minimum and maximum edge estimation error.

There are the settings I used for all simulations:

Maximum Line: -200
Minimum Line: 200
Maximum Estimated Edge %: 5%
Minimum Estimated Edge %: 0.5%
Number of Plays: 10000
Kelly Multiplier: 0.94
Flat Betting %: 2%
Starting Bankroll: 1000
Minimum Bet: 0.05
Number of Simulations: 1000
Number of batches: 100

A “batch” in this context is a simulation using x plays, repeated y times. So in my simulations 1 batch = 1,000 simulations of 10,000 plays.

In Monkey’s simulation sheet, the ending bankroll using flat betting and using Kelly is calculated for each simulation. So if I run 1,000 simulations, I will end up with a list of 1,000 ending bankroll numbers for each method. As an additional variable, I calculated the % of time that the ending Kelly bankroll was higher than the Flat bankroll. This is stored as an additional output variable for each “batch” I ran. I also calculated the max, min, average, and median ending bank roll for each simulation run.

In the tables below are the results from running 100 batches of 1,000 simulations of 10,000 plays. So each table is created from 1 billion plays.
Above each table I have specified the maximum and minimum edge estimation error settings used, which were the only variables that changed between the three sets of batches.

Please note that the average bet, as a percentage of bankroll was very consistent between Flat and Kelly for each of the batches.

Sorry about the look of the table but I couldn't figure out how to copy it from Excel and keep the table structure.

Table 1: Max edge estimation error -1%, Min edge estimation error -5%
Avg Flat Bank 890
Avg Kelly Bank 1,818
Median Flat Bank 68
Median Kelly Bank 109
Max Flat Bank 253,830
Max Kelly Bank 221,015
Min Flat Bank 1
Min Kelly Bank 1
Ending Kelly>Flat 64%
Avg Flat Wager, % of bank 2.18%
Avg Kelly Wager, % of bank 2.17%

Table 2: Max edge estimation error -1%, Min edge estimation error -3%
Avg Flat Bank 4,275
Avg Kelly Bank 10,386
Median Flat Bank 370
Median Kelly Bank 652
Max Flat Bank 1,027,238
Max Kelly Bank 1,284,960
Min Flat Bank 1
Min Kelly Bank 1
Ending Kelly>Flat 67%
Avg Flat Wager, % of bank 2.19%
Avg Kelly Wager, % of bank 2.18%

Table 3: Max edge estimation error 0%, Min edge estimation error 0%
Avg Flat Bank 119,715
Avg Kelly Bank 401,510
Median Flat Bank 11,266
Median Kelly Bank 23,836
Max Flat Bank 25,796,194
Max Kelly Bank 76,749,434
Min Flat Bank 19
Min Kelly Bank 12
Ending Kelly>Flat 73%
Avg Flat Wager, % of bank 2.19%
Avg Kelly Wager, % of bank 2.19%

I am not a statistician, nor do I have a vested interest in proving that Kelly does/doesn't work. It would be great if it does since then "all" I have to do is to find a method that provides me with a consistent and sustainable long term edge and a bookmaker that won't limit me. That part should be easy....

However, given the results above, I am interested in knowing where I went "wrong" since Kelly does produce better simulation results across the board. Which simulation variable is it that creates an unrealistic situation that favors Kelly over flat, assuming that Kelly isn't actually a better method than flat bet? Or are the differences in my simulations over 1 billion plays too small to say that Kelly did "significantly" better?

This sim is incredible. I love it. Well done.

Good thread here. This sim as well as backtesting on covers consensus made me a huge believer in betting less. Look at the guys with over 10,000 plays, it is mind boggling.

Great thread and great work on the simulator and vairants, has helped me get my mind around the topic. Not sure how old the thread is or if anyone is still paying attention to it, but I have a few queries.

I'm trying to ascertain if I'm currently using the correct betting size strategy. For context I have been betting combat sports with a medium bankroll using fractional kelly (usually 1/8) although I don't stick to this rigidly and when i think I have a significant edge in a given fight I'll use a multiple. I'm happy with my results so far but I want to make sure I'm maximizing my potential returns.

My take away from the simulator is that I would derive higher BR growth from using a kelly multiplier of 1 or higher (instead of 1/8th). Given that my fractional kelly bets generally result in a fairly standard/small bet (around 2-3% bankroll), does my current strategy basically equate using flat betting and using kelly when my edge is obvious and quantifiable?

Sim is great.

This is an older thread, but I was wondering if someone can explain to me how to use the Run Sim. When I click Run Sim it does nothing, as if its not even a click option.

Thanks in advance.