Moment Generating Functions

DefinitionMoment Generating Function

The moment generating function (MGF) of a random variable $X$ is a function that provides a way to compute all moments of the distribution.

$M_X(t) = \mathbb{E}[e^{tX}]$

For discrete random variables: $M_X(t) = \sum_{x} e^{tx} \cdot p_X(x)$

For continuous random variables: $M_X(t) = \int_{-\infty}^{\infty} e^{tx} \cdot f_X(x) dx$

Properties

Moment Generation

The $n$-th moment of $X$ can be obtained by differentiating the MGF $n$ times and evaluating at $t=0$:

$\mathbb{E}[X^n] = M_X^{(n)}(0)$

Key Properties

1. Uniqueness: If two random variables have the same MGF, they have the same distribution
2. Linear Transformations: For $Y = aX + b$, $M_Y(t) = e^{bt} M_X(at)$
3. Independence: If $X$ and $Y$ are independent, $M_{X+Y}(t) = M_X(t) \cdot M_Y(t)$

Common MGFs

Distribution	MGF	Domain
Bernoulli(p)	$1 - p + pe^t$	All $t$
Binomial(n,p)	$(1 - p + pe^t)^n$	All $t$
Poisson(λ)	$e^{\lambda(e^t - 1)}$	All $t$
Normal(μ,σ²)	$e^{\mu t + \frac{1}{2}\sigma^2 t^2}$	All $t$
Exponential(λ)	$\frac{\lambda}{\lambda - t}$	$t < \lambda$
Uniform(a,b)	$\frac{e^{tb} - e^{ta}}{t(b-a)}$	$t \neq 0$

Applications

1. Finding moments: Easy computation of expected values, variances, and higher moments
2. Characterizing distributions: Unique identification of probability distributions
3. Sum of random variables: Simplifying analysis of sums of independent variables
4. Limit theorems: Proving convergence in distribution

For more details on expectation and variance, see Expectation and Variance.