'Enforcing' a series of exclusive (but complementary) curve fitted formulas

**HUY** · 07-31-12, 12:38 PM

Originally posted by brettd

Just wondering whether anyone has any ideas on this:

I have a matrix filled with data. Every row of the matrix has has a curve fitted formula that best fits the data located in that row.

Every row (and it's accompanying curve fitted formula) has a relation to the the curve fitted formulas to the rows adjacent.

Now, due to data scarcity and the inherent 'noise' from the data in each row, I have curve fitted formulas that sometimes wildly overlap one another. However in theory, these related curve fitting formulas should be exclusive but complementary of each other (no 'overlap').

So in essence, I need to create a series of curve fitted formulas that are exclusive of each other, yet 'snugly' fit above and below each other with limited and noisy data.

I hope I've described this well enough to envision what I am trying to say here.

Does anyone have any thoughts on this? Is there an 'easy' way to create a stable set of exclusive but complementary curve fitted formulas?

Thanks in advance.

Edit: I feel like I need to 'curve fit the curve fits' somehow.

Give examples.

**mathdotcom** · 07-31-12, 05:12 PM

Originally posted by brettd

Just wondering whether anyone has any ideas on this:

I have a matrix filled with data. Every row of the matrix has has a curve fitted formula that best fits the data located in that row.

Every row (and it's accompanying curve fitted formula) has a relation to the the curve fitted formulas to the rows adjacent.

Now, due to data scarcity and the inherent 'noise' from the data in each row, I have curve fitted formulas that sometimes wildly overlap one another. However in theory, these related curve fitting formulas should be exclusive but complementary of each other (no 'overlap').

So in essence, I need to create a series of curve fitted formulas that are exclusive of each other, yet 'snugly' fit above and below each other with limited and noisy data.

I hope I've described this well enough to envision what I am trying to say here.

Does anyone have any thoughts on this? Is there an 'easy' way to create a stable set of exclusive but complementary curve fitted formulas?

Thanks in advance.

Edit: I feel like I need to 'curve fit the curve fits' somehow.

English please

**sapidoc** · 07-31-12, 05:36 PM

are your curves periodic?

I'm trying to understand your question more clearly...
if you take y=sin(x) as one of your 'best fit' curves then would you consider the curve y = sin(x-pi) to be complementary to this? Is that what you're looking for?

**buby74** · 07-31-12, 06:05 PM

Dont you need a single formula that predicts the content of each cell given the x and y values in your matrix. I would use solver in excel to adjust the coefficients of a formula based on x and y in order to minimize the differences between the actual and predicted values. The hard part is choosing the right type of formula in first place.

**Juret** · 08-01-12, 12:46 AM

Assuming this is for sports betting purposes, isn't this just making things more complicated than they need to be? Correct me if I'm wrong.

**brettd** · 08-02-12, 09:46 AM

I got around curve fitting altogether. I smoothed the matrix enough to do a straightforward retrieval of an output from any cell co-ordinate. It's close enough to what a curve fit would provide as an output, yet it allows the capture of any 'nuances' in the data that would be missed through curve fitting.

Yes this is for sports modelling purposes.

**mathdotcom** · 08-02-12, 10:52 AM

Because there's only one way to fit a curve right?

How about giving us the function if you want help

**yak merchant** · 08-02-12, 06:09 PM

I'm not as smart as most around here, but a lot of this makes zero sense to me.

Originally posted by brettd

Every row (and it's accompanying curve fitted formula) has a relation to the the curve fitted formulas to the rows adjacent.

So if each row is relation to the next row are we talking about time series events?

Originally posted by brettd

However in theory, these related curve fitting formulas should be exclusive but complementary of each other (no 'overlap').

Please explain the "theory" as any sports statistics I have ever seen should have overlap?

Originally posted by brettd

Does anyone have any thoughts on this? Is there an 'easy' way to create a stable set of exclusive but complementary curve fitted formulas?

And what are going to do with all these curves? It sounds like you are trying some sort of ensemble learning (i.e bagging/boosting/stacking) but the whole premise of that is overlap so I really don't understand what the purpose of all the "unique" curves. Sounds like you are trying to build a model that wins the prize for "overfitting" to me.

**chunk** · 08-03-12, 03:23 AM

Good luck with anyone making sense in a meaningful way.

**uva3021** · 08-03-12, 01:38 PM

so each row of data has something like

[theta1, theta2, ...theta_n, x1, x2, ...x_n]

is this right?

then the curve is sum(theta_i * x_i), or if its non-linear, then for some number r, x_i^r

this is what I am visualizing

why does it matter if the curves overlap, since you are fitting the curve to the data in that row, which would mean you are fitting a curve to a set of data that is multidimensional but of sample size 1

the way you described it it doesn't sound like you have set up your data/matrices in a reasonable fashion

**TomG** · 08-03-12, 01:55 PM

OP is using historical info so his rows will look something like this...

spread pr(winning)
-8 70%
-7 65%
-6 66%
-5 58%

-6 is screwed up because a 6 pt fav shouldn't win more than 7 pt fav. he isn't using win% though he's probably doing something like push% for each number.

thus

Every row (and it's accompanying curve fitted formula) has a relation to the the curve fitted formulas to the rows adjacent.

Now, due to data scarcity and the inherent 'noise' from the data in each row, I have curve fitted formulas that sometimes wildly overlap one another. However in theory, these related curve fitting formulas should be exclusive but complementary of each other (no 'overlap').

there are plenty of smooth functions out there you're more likely to figure it out on your own than asking here

**brettd** · 08-05-12, 09:19 AM

Originally posted by TomG

OP is using historical info so his rows will look something like this...

spread pr(winning)
-8 70%
-7 65%
-6 66%
-5 58%

-6 is screwed up because a 6 pt fav shouldn't win more than 7 pt fav. he isn't using win% though he's probably doing something like push% for each number.

thus

there are plenty of smooth functions out there you're more likely to figure it out on your own than asking here

You're pretty much spot on the money there. There's cells in the matrix that are 'out of place' in relation to the cells around them. However i'm not creating a push chart.

I've pretty much alleviated this problem using an 'area' smoothing technique. In my case, a 25x25 cell smoother area.