In probability theory, it is possible to approximate the moments, such as the mean and variance , of a function f of a random variable X using Taylor expansions, provided that f is sufficiently differentiable and that the moments of X are finite.
First moment A Taylor expansion of the expected value of can be found via:Haym Benaroya, Seon Mi Han, and Mark Nagurka. Probability Models in Engineering and Science. CRC Press, 2005, p166.
Since the second term vanishes. Also, is . Therefore,
.
So, the output expected value is seen to depend on the input variance.
Second moment Similarly, the variance is obtained as:
It improves on the typical first-order approximation, which will be a poor approximation if is highly non-linear (see uncertainty propagation):
A third order approximation, when X follows a normal distribution, is:
Random vectors If X is a random vector, the approximations for the mean and variance of are given by
Here and denote the gradient and the Hessian matrix respectively, and is the covariance matrix of X.
Multiple variables
It is possible to generalize this to functions of more than one variable using multivariate Taylor expansions. For example (see ratio distribution),
With , we get . The variance is then computed using the formula .
An example is,
First product moment To find a second-order approximation for the covariance of functions of two random variables (with the same function applied to both), one can proceed as follows. First, note that . Since a second-order expansion for has already been derived above, it only remains to find . Treating as a two-variable function, the second-order Taylor expansion is as follows:
Taking expectation of the above and simplifying—making use of the identities and —leads to . Hence,
See also Propagation of uncertainty WKB approximation Delta method Monte Carlo simulation
Notes
Further reading