When does a "small sample size" stop being a "small sample size"?

[TheLock]

I was having this discussion with a friend today in relation to a baseball "system".

When is a small sample size no longer small? Obviously, if you're trying out a new system and you're doing really well/getting pounded after 30 games, it's a very small sample size.


At the risk of sounding stupid, I would say after 250 plays on a "system", the results are no longer a "small sample size". I'm not saying 250 is a "big sample size" but..........


Thoughts on this are appreciated and thanks in advance.

[donjuan]

250 plays is still too small, especially if you bet underdogs and/or your plays are correlated in any way.

Also, the problem with systems/angles is that even if they were +EV in the past, there is no guarantee they will continue to be in the future as markets tend to adjust.

[accuscoresucks]

600 or so i would guess if criteria is 7-21 weekly pick avg

[reno cool]

I would say that after 250 approx even money bets if you're showing a return of over 10% better than vig something other than luck is involved. So its dependent on how high a return your showing.

[Mudcat]

I usually think in terms of about 200. If I am testing an angle and I find it showing a decent profit after ~200 live tests, that's when I'll start backing it with real money.

I start with smaller units and build up. Sometimes I find that the second I start backing it with real money, the angle suddenly doesn't work. So I abandon it. Obviously 200 is not always a sufficient sample size. But that's why I start real-money betting with small units.

I don't doubt someone could come in here and tell me how this is all wrong in theory but it is a methodology that has worked for me for a long time. It's relevant to the discussion that the angles I test are seldom random. There is almost always some established betting theory behind them, usually relating to known biases in bettor psychology that have already been proven to throw lines out-of-whack. So I know before I collect a single sample that there is an above average chance it will work.

If I was just testing some random theory that didn't make any particular sense at the outset, I would probably want a bigger sample than just 200 before I would back it with real $$$.

donjuan is correct though. Constant evaluation is required. Sometimes an angle will work for years over 1000+ bets, then it will stop working. Sometimes there is an obvious reason (like when the introduction of the BCS to NCAAF encouraged favorites to run up the score and cover more lop-sided spreads). But then sometimes I have no idea why the angle stops working.

[TheLock]

great feedback so far, appreciate it

[marcoforte]

I'm an nfl only player. i use n=60 with a z value greater than 3.