Propagation of Error in a Non-Linear Equation

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • tweek
    SBR Hustler
    • 02-17-09
    • 60

    #1
    Propagation of Error in a Non-Linear Equation
    Say I have a non-linear equation, take y = a^2 + b^2 as an example.

    If I know the variance on a and b, what is the variance on y?

    I know that generally, sigma_y^2 = (dy/da)^2*sigma_a^2 + (dy/db)^2*sigma_b^2. But, what I don't understand is with this nonlinear equation, my partial derivatives are still a function of a and b respectively. So, if I have:

    sigma_y^2 = 4a^2*sigma_a^2 + 4b^2*sigma_b^2 (assuming covariance is 0),

    what do I use for "a" and "b" when all I have are the variances?
    Last edited by tweek; 06-11-09, 04:02 PM.
  • tweek
    SBR Hustler
    • 02-17-09
    • 60

    #2
    Ok, so after a little bit more research, it looks like you use the mean of either "a" or "b". And, while this doesn't really apply to sports handicapping, it begs the question of propagation of error for zero-mean random variables. If a and b are both zero mean RV's with non-zero variance, clearly y will be zero mean but have non-zero variance. So, how is the variance of y calculated in this case?
    Comment
    • Ganchrow
      SBR Hall of Famer
      • 08-28-05
      • 5011

      #3
      Originally posted by tweek
      If a and b are both zero mean RV's with non-zero variance, clearly y will be zero mean
      In the example you gave in the first post y would clearly not have zero mean (for real A & B, non-zero variance for either A or B).

      In general there is no closed form solution solution to your original question. In the simple example you gave with random variable a, however:

      Let M = E(a)
      Let V = Var(a)
      Let X = a2

      We know that for any random variable X:
      Var(X) = E(X2) - E(X)2
      = E(a4) - E(a2)2

      But E(a2) = V + M2

      So we have:
      Var(X) = E(a4) - (V + M2)2

      Where E(a4) is the 4th moment about the origin, a concept related to kurtosis.

      So if we assume M = 0, then:
      Var(X) = (Kurt(a)+3)*V2 - V2
      = (2+Kurt(a)) * V2

      where Kurt(a) refers to the excess kurtosis of the random (population) variable a.

      So for Y = a2 + b2, with a = b = 0, and Cov(a,b) = 0 then:

      Var(Y) = (2+Kurt(a)) * Var(a)2 + (2+Kurt(b)) * Var(b)2
      Comment
      • Ganchrow
        SBR Hall of Famer
        • 08-28-05
        • 5011

        #4
        Originally posted by Ganchrow
        So for Y = a2 + b2, with a = b = 0, and Cov(a,b) = 0 then:

        Var(Y) = (2+Kurt(a)) * Var(a)2 + (2+Kurt(b)) * Var(b)2
        In case anyone's tried this in Excel over a given and is wondering why the values don't match, recall that while we here are using population kurtosis the Excel KURT() functions calculates sample kurtosis defined as:
        [ATTACH]6954[/ATTACH]
        where s represent the sample deviation.

        By the way ... any guesses on the over/under for how many posters including both myself and the OP will actually read all this in it entirety? I'd go with 1.5.
        Comment
        • tweek
          SBR Hustler
          • 02-17-09
          • 60

          #5
          Originally posted by Ganchrow
          In the example you gave in the first post y would clearly not have zero mean (for real A & B, non-zero variance for either A or B).
          Whoops... you're right.

          Originally posted by Ganchrow
          In general there is no closed form solution solution to your original question. In the simple example you gave with random variable a, however:

          Let M = E(a)
          Let V = Var(a)
          Let X = a2

          We know that for any random variable X:
          Var(X) = E(X2) - E(X)2
          = E(a4) - E(a2)2
          But E(a2) = V + M2

          So we have:
          Var(X) = E(a4) - (V + M2)2
          Where E(a4) is the 4th moment about the origin, a concept related to kurtosis.

          So if we assume M = 0, then:
          Var(X) = (Kurt(a)+3)*V2 - V2
          = (2+Kurt(a)) * V2

          where Kurt(a) refers to the excess kurtosis of the random (population) variable a.
          So for Y = a2 + b2, with a = b = 0, and Cov(a,b) = 0 then:

          Var(Y) = (2+Kurt(a)) * Var(a)2 + (2+Kurt(b)) * Var(b)2
          Ok... I think I follow that analysis. I guess what I'm really after is the propogation of standard deviation for the Pythagorean equation: Y = a2 / (a2 + b2). In practice, a and b will not be zero mean. I should be able to work through the analysis... I'll give it a shot.

          Thank for pointing me in the right direction.
          Comment
          • tweek
            SBR Hustler
            • 02-17-09
            • 60

            #6
            Quick question - while the variance of a sum of RV's is the sum of the variance of the RV's, how does the variance combine for multiplication and/or division?

            If we assume they're independent, for multiplication we get:

            Var(XY) = E[(XY)^2] - E[XY]^2
            = E[X^2] E[Y^2] - E[X]^2 E[Y]^2
            = E[X^2] E[Y^2] - E[X]^2 E[Y^2] + E[X]^2 E[Y^2] - E[X]^2 E[Y]^2
            = Var(X) E[Y^2] + E[X]^2 Var(Y)
            = Var(X) Var(Y) + Var(X) E[Y]^2 + E[X]^2 Var(Y)

            Is this it? What about division?
            Last edited by tweek; 06-11-09, 06:11 PM.
            Comment
            • Ganchrow
              SBR Hall of Famer
              • 08-28-05
              • 5011

              #7
              Originally posted by tweek
              Quick question - while the variance of a sum of RV's is the sum of the variance of the RV's, how does the variance combine for multiplication and/or division?

              If we assume they're independent, for multiplication we get:

              Var(XY) = E[(XY)^2] - E[XY]^2
              = E[X^2] E[Y^2] - E[X]^2 E[Y]^2
              = E[X^2] E[Y^2] - E[X]^2 E[Y^2] + E[X]^2 E[Y^2] - E[X]^2 E[Y]^2
              = Var(X) E[Y^2] + E[X]^2 Var(Y)
              = Var(X) Var(Y) + Var(X) E[Y]^2 + E[X]^2 Var(Y)

              Is this it? What about division?
              Bingo.

              And FWIW, independence, while sufficient for the above isn't necessary. That the variables are uncorrelated is the only necessary condition for the above to hold.

              To my knowledge there's no similar equality for the quotient of two random variables.
              Comment
              • tweek
                SBR Hustler
                • 02-17-09
                • 60

                #8
                Originally posted by Ganchrow
                Bingo.

                And FWIW, independence, while sufficient for the above isn't necessary. That the variables are uncorrelated is the only necessary condition for the above to hold.

                To my knowledge there's no similar equality for the quotient of two random variables.
                Yeah... just tried to work through it without a lot of luck.

                So, that being the case, do you have any insight as to working through the calculation of

                Var(X2 / (X2 + Y2))?
                Comment
                • Wrecktangle
                  SBR MVP
                  • 03-01-09
                  • 1524

                  #9
                  This is the stuff we need to see to scare off the room-temp IQ folks that Monkey was railing about...
                  Comment
                  • Ganchrow
                    SBR Hall of Famer
                    • 08-28-05
                    • 5011

                    #10
                    Originally posted by tweek
                    Yeah... just tried to work through it without a lot of luck.

                    So, that being the case, do you have any insight as to working through the calculation of

                    Var(X2 / (X2 + Y2))?
                    All that immediately springs to mind is that if X and Y represent points scored and are both of sufficiently large magnitude then they can nevertheless be very roughly approximated with the Poisson (to varying degrees of accuracy) with λX and λY equal to X and Y, respectively.

                    These in turn could be approximated by two Gaussian distributions of each with mean and variance of the appropriate λ.

                    Standardize each and you'd have a χ2 distribution with 1 d.o.f. for your numerator and 2 d.o.f. for your denominator, the quotient of which should follow an F-distribution.

                    So another words, my immediate reaction would be that that the Pythagorean expectation of the square of the standard scores of each of our two pint distributions should follow a roughly F(1,2) distribution.

                    How (if at all) this might help with your question I couldn't readily say. Perhaps, however, it'll get you thinking along the right lines.
                    Comment
                    • MrLuckyPants
                      SBR Hustler
                      • 02-25-09
                      • 54

                      #11
                      Originally posted by Wrecktangle
                      This is the stuff we need to see to scare off the room-temp IQ folks that Monkey was railing about...
                      Runs for the hills!
                      Comment
                      SBR Contests
                      Collapse
                      Top-Rated US Sportsbooks
                      Collapse
                      Working...