Very true.
I figured when I created the thread that questions would come up regarding how we judge that a system or scenario is worth adding to the collection and wagering hard earned money on, and the short answer is...everyone is different.
For me, I prefer to play far fewer scenarios and am constantly whittling them away to keep only what I feel are the very best possible plays. I also prefer to only play league-wide systems versus the more popular team trends, because team trends are volatile and can change rapidly. That doesn't mean team trends aren't valid, it's just what I prefer as a 'lazy' bettor.
So I can only tell you how I personally do it, and what my own philosophy is. I am not recommending you follow these requirements per se, they're just here to serve as an example of how I try to filter out the noise from the actual signal. I apply the following standard to every NBA query I find, create, or tinker with:
NBA SYSTEM CREATION PARAMETERS - (I force myself to stick to all these, hurts to lose a query when it's SO close but that's the discipline)
1. SYSTEM MUST BE BASED ON A MINIMUM AVERAGE OF 6-7 PLAYS PER SEASON FROM 2006+ (50 minimum)
Any results below that level of frequency make me suspicious, always remember that the smaller the sample size the more likely the results are just random. This doesn't mean that smaller trends aren't valid, far from it, but just make sure you have other validation techniques in place like the ones I'm about to list to compensate for the smaller snapshot.
2. SYSTEM MUST HAVE AN ATS WIN RATE OF 45%+ WITH -5/+5 POINT PLEASER HIT APPLIED
To do this, I take the ATS win % and apply a -5 or +5 point 'pleaser' to it in a negative way, effectively making the line 5 points 'worse' than it actually was, then see how it impacts the wins/losses/net. If a system still 'wins' at 45% or more after taking a 5 point line hit, it tells me that the scenario might be stronger at keeping bad beats away, along with the randomness of small ats margin wins. Lately I've moved this figure up to 50%, meaning the system still breaks even minus the vig despite a massive 5 point line change away from it.
3. SYSTEM MUST HAVE MINIMUM WIN RATES OF 62.5% FOR BOTH 2006-2009 AND FROM 2010 TO CURRENT
This one is simple as to the 'why', it's what we all do every time we load up a query...to evaluate the ATS win rate. Win rate isn't everything, as we'll see the volume of plays combined with the win rate is what is truly critical, but it's the starting point. If the win rate isn't high enough for your base query the tendency is to filter it like crazy until it gets the 'right' number...which can be dangerous as it leads to overfitted results (explained in #6 below).
4. SYSTEM MUST HAVE AN AVERAGE ATS WIN MARGIN OF 3.0 POINTS OR HIGHER
I take a snapshot of whatever system I'm working on from 2006 to 2013, and not only must the results fit within the win rates of #3 above, but they must an average ATS margin of 3.0+ or higher with those win rates. Again, this is like #2, where I'm trying to see through the systems that just 'got lucky' the past few years, barely covering the spread through randomness, versus systems where there is actual EV of +3 points or more versus the line for whatever hopefully-logical reason/circumstance.
5. SYSTEM MUST HAVE A Z-SCORE OF 3.00+ FOR 2013-2006 PERIOD
Z-Scores or Z-Values are another useful filter that can be applied to help you see whether a system is just lucky or a true consistent play. To calculate a Z-Score, first take the total of wins+losses for the system and get the square root of that total number. Now subtract losses from wins and divide that number by the square root you determined a step earlier. You should get a number between 1.00 and 5.00, or close to it for the average NBA query that wins at 60%+. I choose to only play systems that achieve a 3.00 Z-Score from 2006-2013, because it seems to be a nice break point. If you get confused by this just Google 'Z-score' or 'Z-value' and there are plenty of auto-calculators that pop up to help.
6. SYSTEM MUST HAVE A LOGICAL DEFINITION TO EXPLAIN/JUSITFY RESULTS
One other extremely misleading and dangerous problem that the SDQL lends itself to is called "Overfitting". You can Google 'overfitting' or the other term that is sometimes used in its place ("data mining"), but the end result is that you're plugging in variables and filters to get a desired win/loss rate for the database and assuming that because it provided a "75-25 win/loss rate" that it "must" be real...when there isn't a logical reason behind the results. It's got to "make sense" in other words, like say two teams having 0 days rest playing each other that have poor defenses with average points given up of 100+ on the season which could make the Over bet a solid one.
IT'S ALL STILL A CRAPSHOOT
All of these examples are ways I try to 'protect' myself from getting too excited and following a query that looks like 24KT but winds up as fools' gold from the point I discover or build it onward. As time goes on I'll likely continue to add to the barriers based on experience and trying to figure out 'why' a system goes bad or doesn't work as it has in the past. But in reality anything can change overnight, and a bulletproof system that paid off for multiple seasons can be obsolete after a rule change or schedule change or any other number of variables.
Be smart, try to filter out the junk, and refine your process constantly in order to achieve better results over time.
Originally Posted by Wojo
This is something I always have difficulties with. Do you follow a short-term trend that's working well or do you stay away if it hasn't been that good in the past?
tonywayne gave NBACover 1 Betpoint(s) for this post.
njb5572 gave Mako-SBR 2 Betpoint(s) for this post.
