Moment Generating Functions

A moment generating function packages expectations into one function, but its useful differentiation and uniqueness properties require a domain condition.

DefinitionMoment generating function

For a real random variable $X$, define $M_X(t)=E[e^{tX}]$ wherever this expectation is finite. In the discrete and density cases,

$$M_X(t)=\sum_x e^{tx}p_X(x),\qquad M_X(t)=\int_{\mathbb R}e^{tx}f_X(x)\,dx,$$

respectively. Always $M_X(0)=1$. We say the MGF exists near zero if it is finite for every $t$ in some interval $(-a,a)$ with $a>0$.

Having all polynomial moments does not by itself justify an MGF near zero. For example, if $Z$ is standard normal and $X=e^Z$, completing the square gives $E[X^n]=e^{n^2/2}$ for each nonnegative integer $n$. Yet $E[e^{tX}]=\infty$ for every $t>0$: in the defining normal integral, $te^z-z^2/2\to\infty$, so the integrand eventually exceeds a positive constant.

Moment generation

TheoremDifferentiate to obtain moments

If $M_X$ is finite on $(-a,a)$, then every absolute moment is finite, and

$$M_X^{(n)}(t)=E[X^ne^{tX}],\qquad |t|<a.$$

In particular $M_X^{(n)}(0)=E[X^n]$.

Proof

Choose $|t|<b<c<a$. Since $e^{c|X|}\le e^{cX}+e^{-cX}$, its expectation is finite. The function $u^n e^{-(c-b)u}$ is bounded for $u\ge0$, so for some finite $C_n$,

$$|X|^n e^{b|X|}\le C_n e^{c|X|}.$$

The same bound for $n+1$ dominates difference quotients of the $n$-th derivative in a small interval around $t$, by the mean value theorem. Dominated convergence therefore passes each derivative through the expectation. Repeating establishes all orders. At $t=0$, the bound also gives finiteness of every absolute moment. This uses dominated convergence as an integration prerequisite; its general proof remains outside this note.

Affine transformations and independent sums

For $Y=aX+b$, the definition gives

$$M_Y(t)=E[e^{t(aX+b)}]=e^{bt}M_X(at),$$

whenever $M_X(at)$ is finite. If $X,Y$ are independent, their joint law factors, giving

$$M_{X+Y}(t)=E[e^{tX}e^{tY}]=M_X(t)M_Y(t)$$

on the common finite domain. In a discrete model this follows by factoring the double sum; for densities it follows by factoring the double integral. Nonnegative integrands justify the interchange, and finiteness makes the product finite. Iteration proves the rule for a finite mutually independent family. Pairwise independence alone does not justify the many-variable factorization.

What uniqueness requires

TheoremMGF uniqueness with a neighborhood condition

If $M_X(t)=M_Y(t)<\infty$ throughout an open interval containing zero, then $X$ and $Y$ have the same distribution.

For a common finite support $\{x_1,\ldots,x_m\}$ of distinct values, here is an elementary proof. Equal derivatives at zero give equal expectations of all polynomials. The Lagrange polynomial

$$\ell_j(x)=\prod_{i\ne j}\frac{x-x_i}{x_j-x_i}$$

is one at $x_j$ and zero at every other support point. Therefore $P(X=x_j)=E[\ell_j(X)]=E[\ell_j(Y)]=P(Y=x_j)$ for every $j$.

For general real distributions, the standard proof extends $E[e^{zX}]$ analytically to a vertical strip, uses the identity theorem to obtain equality on the imaginary axis, and applies uniqueness of characteristic functions. Those two analytic uniqueness theorems have not yet been proved in this collection, so the general theorem is an explicit prerequisite rather than a claimed elementary proof here. MIT’s probability notes state the neighborhood condition and discuss transform uniqueness [1][1] M. I. of Technology, “Fundamentals of Probability: Lecture 13, Moment Generating Functions,” 2018. https://ocw.mit.edu/courses/6-436j-fundamentals-of-probability-fall-2018/1a592ed184fb4c444547f67c9bcdd8ec_MIT6_436JF18_lec13.pdf. Equality only at $t=0$ gives no information: every probability distribution has that value one.

Common MGFs and their derivations

Assume $0\le p\le1$, integer $n\ge0$, $\lambda>0$, $\sigma>0$, and $a<b$.

Distribution	MGF	Finite domain
Bernoulli	$1-p+pe^t$	All real $t$
Binomial	$(1-p+pe^t)^n$	All real $t$
Poisson	$\exp(\lambda(e^t-1))$	All real $t$
Normal	$\exp(\mu t+\sigma^2t^2/2)$	All real $t$
Exponential	$\lambda/(\lambda-t)$	$t<\lambda$
Uniform	$(e^{tb}-e^{ta})/(t(b-a))$ if $t\ne0$; $1$ at $t=0$	All real $t$

For Bernoulli, sum the two outcomes. A binomial variable is the sum of $n$ independent Bernoulli variables, so the product rule gives its MGF. For Poisson, the exponential series gives

$$M_X(t)=e^{-\lambda}\sum_{k=0}^{\infty}\frac{(\lambda e^t)^k}{k!} =e^{-\lambda}e^{\lambda e^t}.$$

For a normal density, complete the square:

$$tx-\frac{(x-\mu)^2}{2\sigma^2} =-\frac{(x-\mu-\sigma^2t)^2}{2\sigma^2} +\mu t+\frac{\sigma^2t^2}{2}.$$

The shifted normal density integrates to one, leaving the stated factor. For exponential variables integrate $\lambda e^{-(\lambda-t)x}$ on $[0,\infty)$, which is finite exactly when $t<\lambda$. For uniform variables integrate $e^{tx}/(b-a)$ on $[a,b]$. At zero the integrand is constant, so the apparent singularity in the quotient is removable.

A complete moment calculation

For a Poisson variable,

$$M'_X(t)=\lambda e^tM_X(t),\qquad M''_X(t)=(\lambda e^t+\lambda^2e^{2t})M_X(t).$$

Thus $E[X]=\lambda$, $E[X^2]=\lambda+\lambda^2$, and $\operatorname{Var}(X)=\lambda$. The expectation and variance note proves the second-moment identity used in the final step.

References

1. [1] M. I. of Technology, “Fundamentals of Probability: Lecture 13, Moment Generating Functions,” 2018. https://ocw.mit.edu/courses/6-436j-fundamentals-of-probability-fall-2018/1a592ed184fb4c444547f67c9bcdd8ec_MIT6_436JF18_lec13.pdf ↩