Fading your own model?

[HUY]

If you had a model that properly (that is, when properly using training and test data sets) exhibited a ROI of -10%, would you fade its suggestions to receive a profit?

[Justin7]

No. Find out what in your model is predictably wrong. There might be a gem there, but fading it is another disaster unless you know why it is failing.

[mathdotcom]

J7 says in his book this happened to him a lot

[Justin7]

J7 says in his book this happened to him a lot

Strange, I don't remember printing that in my book.

[NSN21]

It has to be something that can be logically explained as to why it works. Otherwise, it's really nothing more than just data mining.

[TomG]

fudge factors though

[mathdotcom]

Strange, I don't remember printing that in my book.

"In an earlier attempt to model MLB, I made some assumptions that were worse than bad, and the model was a disaster of epic proportions ... After testing it for a year, it looked like it was a winning model. My team then bet Fezzik 15k that my model would win at least 5 units in its first 250 plays. When our contest started, my model went 17-47, losing 31.94 units, before I conceded defeat to Fezzik." pg.170

Sounds like this model was 'predictably wrong'. And I believe in forum posts you have mentioned that in your career you have built many failing models.

[mathdotcom]

fudge factors though

The truly hilarious part of the book was the FUDGE FACTOR discussion

[HUY]

"In an earlier attempt to model MLB, I made some assumptions that were worse than bad, and the model was a disaster of epic proportions ... After testing it for a year, it looked like it was a winning model. My team then bet Fezzik 15k that my model would win at least 5 units in its first 250 plays. When our contest started, my model went 17-47, losing 31.94 units, before I conceded defeat to Fezzik." pg.170

Sounds like this model was 'predictably wrong'.

He says that the model was winning in the training set and then it lost in the actual betting. That's not really predictably wrong.

[mathdotcom]

He says that the model was winning in the training set and then it lost in the actual betting. That's not really predictably wrong.

He gave up after 64 plays ...

[Dark Horse]

If you had a model that properly (that is, when properly using training and test data sets) exhibited a ROI of -10%, would you fade its suggestions to receive a profit?

You can't tweak a wrong hypothesis into a right one. Throw it out and start from scratch.

[Justin7]

The truly hilarious part of the book was the FUDGE FACTOR discussion

Whatever you want to call the "market adjustment", every single model I have developed is "fudged" to market. And yes, this includes the better winning ones.

If you can identify where a market "should" price a game, and see where a game is mispriced relative to that, you will gain an edge no matter what you call the method.

[Emily_Haines]

You can back test these bastards to yield shit that hits 70% but going forward they never seem to lead you to that pot of gold.

[hutennis]

Fading yours or someone else's "model" is losing proposition no matter what.

For the most part your "model" inevitably belongs to a "50/50 minus commission" grave yard b/c no matter what you do you take market prices that are 50/50 minus commission. And that is true for both sides. Your losses beyond expected are simply result of being unlucky. As soon as you switch sides your "model" will start getting back to its mean and you'll lose ungodly more than you should have theoretically due to bad timing.

[mathdotcom]

Whatever you want to call the "market adjustment", every single model I have developed is "fudged" to market. And yes, this includes the better winning ones.

If you can identify where a market "should" price a game, and see where a game is mispriced relative to that, you will gain an edge no matter what you call the method.

I've never created a model that had to be 'fudged'. How does this not sound off alarm bells when you're creating your estimates?

[hutennis]

If you can identify where a market "should" price a game, and see where a game is mispriced relative to that, you will gain an edge no matter what you call the method.

The above statement is identical to

"If you can be smarter than the rest of the world, you will gain an edge no matter what."

And yet, it is amazing.
Tell them they have to outsmart the world and they'll go to the movie.
Tell them they have to identify mispricing and the'll lose the farm trying.

[doesnotcompute]

J7 says in his book this happened to him a lot

Why would you even read that book?

[mathdotcom]

Why would you even read that book?

Basically to be a dick to him on the forum And curiosity about how other people approach the same problems as I do.

J7 is a good guy and looks good in a suit with that graphical bookcase thing behind him, but I worry about his regression work

[mathdotcom]

Why would you even read that book?

Basically to be a dick to him on the forum And curiosity about how other people approach the same problems as I do.

J7 is a good guy and looks good in a suit with that graphical bookcase thing behind him, but I worry about his regression work

Also, at the end of the day he makes money so may as well listen to what he has to say

[MonkeyF0cker]

Whatever you want to call the "market adjustment", every single model I have developed is "fudged" to market. And yes, this includes the better winning ones.

If you can identify where a market "should" price a game, and see where a game is mispriced relative to that, you will gain an edge no matter what you call the method.

Must be why you're still here handing out infractions to me whenever I respond to you posting nonsense like this.

[HUY]

Can anyone explain what this "fudge factor" is?

[Justin7]

Can anyone explain what this "fudge factor" is?

When you develop a model, you can never include every single factor that affects a score. If 1. The market is at least quasi-efficient, and 2. your model is accurate, you should have roughly equal numbers of overs and unders as potential bets. Similarly, the average total of your model should be similar to the average of the market.

What if omitted factors cause a 5% discrepancy? So your totals are consistently higher than the market? Or lower? You measure the difference, and mark to market. If over the course of thousands of games, your average total was 5% less than the the average market total, you'd introduce a fudge factor.... Multiply your projected total to get you in line with the market.

[Justin7]

Must be why you're still here handing out infractions to me whenever I respond to you posting nonsense like this.

Nahhh. If I hand out enough infractions, I get certain databases in return.

[HUY]

When you develop a model, you can never include every single factor that affects a score. If 1. The market is at least quasi-efficient, and 2. your model is accurate, you should have roughly equal numbers of overs and unders as potential bets. Similarly, the average total of your model should be similar to the average of the market.

What if omitted factors cause a 5% discrepancy? So your totals are consistently higher than the market? Or lower? You measure the difference, and mark to market. If over the course of thousands of games, your average total was 5% less than the the average market total, you'd introduce a fudge factor.... Multiply your projected total to get you in line with the market.

That's a good idea as far as I'm concerned. I see no reason to leave out the market opinion from a model.

[mathdotcom]

Using a "fudge factor" takes a very naive view of regression analysis, and I've said this before, you should really take a course on what's going on in the background. If you're consistently predicting higher than normal totals, you can't just subtract x from the predicted values and call it a day. It means that at least one of your co-efficients is too high (other than the constant), which means your model is overestimating the marginal effect of a certain variable, probably because you have omitted something seriously relevant. If the underlined part of that statement doesn't bother you, then, well, carry on...

[MonkeyF0cker]

Nahhh. If I hand out enough infractions, I get certain databases in return.

That says it all. You need others to do your work for you.

[MonkeyF0cker]

Using a "fudge factor" takes a very naive view of regression analysis, and I've said this before, you should really take a course on what's going on in the background. If you're consistently predicting higher than normal totals, you can't just subtract x from the predicted values and call it a day. It means that at least one of your co-efficients is too high (other than the constant), which means your model is overestimating the marginal effect of a certain variable, probably because you have omitted something seriously relevant. If the underlined part of that statement doesn't bother you, then, well, carry on...

Exactly.

[MonkeyF0cker]

When you develop a model, you can never include every single factor that affects a score.

But the market can? How exactly does the market become increasingly efficient if NOBODY can price it properly? By copying it?

A circle of nonsense. Brilliant.

[Justin7]

But the market can? How exactly does the market become increasingly efficient if NOBODY can price it properly? By copying it?

A circle of nonsense. Brilliant.

Perhaps I should clarify. You (alone), a single person, or even a small group cannot include everything. These may define the earlier markets, but not the later markets. As the market size grows, it becomes more lucrative to larger groups, even with the price gradually becoming more efficient.

The larger groups can include a lot more information, and are the ones that define the closing price.

[Justin7]

That says it all. You need others to do your work for you.

A lot of it, yes.

[MonkeyF0cker]

Perhaps I should clarify. You (alone), a single person, or even a small group cannot include everything. These may define the earlier markets, but not the later markets. As the market size grows, it becomes more lucrative to larger groups, even with the price gradually becoming more efficient.

The larger groups can include a lot more information, and are the ones that define the closing price.

Not following. What information can larger groups include that a small group or even a single person cannot?

How is group size positively correlated to the information available to them?

[MonkeyF0cker]

And how could you possibly IMPROVE upon the pricing of these "larger groups" that "can include a lot more information and are the ones that define the closing price" when you say that you have less information than them anyway?

[mathdotcom]

J7 making bizarre statements here

Created all my models alone in my basement with the lights off, cellphone off, etc.

The minimum requirements are some form of data collection ability, moderate knowledge of the sport, and a background in statistics.

The Billy Walters method is definitely the exception rather than the norm

[allin1]

Not following. What information can larger groups include that a small group or even a single person cannot?

First thing that comes to mind is inside info that hasn't got out yet...

[MonkeyF0cker]

First thing that comes to mind is inside info that hasn't got out yet...

Why couldn't a single person or small group have access to inside information?