Eigenvalues, Invariant Subspaces, and Diagonalization

A general matrix changes both length and direction. Eigenvectors identify exceptional directions that remain on the same line and are only scaled or reversed.

An eigenvector turns a matrix action into scaling

A nonzero vector $\mathbf v$ is an eigenvector of $A$ when

$$A\mathbf v=\lambda\mathbf v.$$

The scalar $\lambda$ is its eigenvalue. The zero vector is excluded because it satisfies the equation for every scalar and identifies no direction. Rearranging gives

$$(A-\lambda I)\mathbf v=\mathbf0.$$

A nonzero solution exists exactly when $A-\lambda I$ is singular, so

$$\det(A-\lambda I)=0.$$

This is the characteristic equation. Finding a root is only the first step; solving the corresponding homogeneous system gives the eigenvectors.

We use the monic convention $p_A(t)=\det(tI-A)$. Since $\det(A-tI)=(-1)^np_A(t)$, either characteristic equation has the same roots and multiplicities.

An eigenspace is a kernel

For fixed $\lambda$, include zero with all corresponding eigenvectors:

$$E_\lambda=\ker(A-\lambda I).$$

This eigenspace is a subspace. Its dimension is the geometric multiplicity. The root multiplicity of $\lambda$ in the characteristic polynomial is its algebraic multiplicity, and

$$1\le \dim E_\lambda \le \operatorname{mult}_{\lambda}(p_A).$$

Eigenvectors belonging to distinct eigenvalues are independent. This permits several one-dimensional invariant directions to form a basis.

Why distinct eigenvalues give independent directions

Proceed by induction. Suppose the first $k-1$ eigenvectors are independent and a combination of all $k$ vanishes. Apply $A-\lambda_k I$ to obtain

$$\sum_{i=1}^{k-1}c_i(\lambda_i-\lambda_k)\mathbf v_i=0.$$

Independence and distinct eigenvalues force the first $k-1$ coefficients to vanish; substitution gives $c_k=0$. The proof uses eigenvalue information, not merely pairwise nonparallelism.

Why geometric multiplicity cannot exceed algebraic multiplicity

Let $g=\dim E_\lambda$. Extend a basis of the eigenspace to a basis of the entire space. In these coordinates,

$$\widetilde A=\begin{pmatrix}\lambda I_g&C\\0&B\end{pmatrix}.$$

The lower-left block vanishes because the first $g$ basis vectors map to their own $\lambda$ multiples. The block-triangular determinant gives

$$\det(tI-\widetilde A)=(t-\lambda)^g\det(tI-B).$$

Hence the root multiplicity is at least $g$. Distinct eigenspaces have a direct sum, by the same elimination argument used for independent eigenvectors, applied to nonzero combinations in each eigenspace. When the characteristic polynomial splits, their dimensions sum to $n$ exactly when every geometric multiplicity equals its algebraic multiplicity, equivalently when an eigenvector basis exists.

Invariant subspaces are more general

A subspace $W$ is invariant under $A$ when $A(W)\subseteq W$. Every eigenvector spans an invariant line, but an entire rotation plane can be invariant without containing a real eigenvector. A basis adapted to invariant subspaces gives a block matrix and can split a large problem into smaller ones.

Diagonalization means finding an eigenvector basis

If $A$ has $n$ independent eigenvectors, put them in the columns of $S$ and their eigenvalues on the diagonal of $D$. Then

$$AS=SD, \qquad A=SDS^{-1}.$$

Conversely, the columns of $S$ in such a factorization are eigenvectors. A square matrix is therefore diagonalizable exactly when the space has an eigenvector basis.

Distinct eigenvalues guarantee enough independent eigenvectors. Repeated eigenvalues require checking eigenspace dimensions. For example,

$$\begin{pmatrix}1&1\\0&1\end{pmatrix}$$

has only a one-dimensional eigenspace and cannot supply a basis of two eigenvectors.

Construct a complete diagonalization

For $A=\begin{pmatrix}2&1\\0&3\end{pmatrix}$, choose eigenvectors $(1,0)^{\mathsf T}$ and $(1,1)^{\mathsf T}$. Then

$$S=\begin{pmatrix}1&1\\0&1\end{pmatrix}, \quad D=\begin{pmatrix}2&0\\0&3\end{pmatrix}, \quad S^{-1}=\begin{pmatrix}1&-1\\0&1\end{pmatrix}.$$

An input $(x,y)$ has new coordinates $(x-y,y)$. Scale by $2,3$ and convert back to obtain $(2x+y,3y)$, exactly the original action. This explains the order of $S^{-1}$, $D$, and $S$.

Diagonalizability depends on the scalar field. A ninety-degree rotation is diagonalizable over the complex numbers but not the reals. In general the characteristic polynomial must split over the chosen field and every geometric multiplicity must equal its algebraic multiplicity.

Real matrices may need complex scalars

The ninety-degree rotation

$$R=\begin{pmatrix}0&-1\\1&0\end{pmatrix}$$

has characteristic equation $\lambda^2+1=0$. No real direction remains on its original line. Over the complex numbers, the eigenvalues are $i$ and $-i$. The existing complex-numbers chapter reviews $i$, conjugation, and polar form.

For complex vectors, use the conjugate inner product $\langle\mathbf x,\mathbf y\rangle=\mathbf x^*\mathbf y$, where $^*$ is conjugate transpose. Ordinary transpose can fail to produce a nonnegative squared norm.

Similarity preserves spectral information

A change of basis replaces $A$ by $S^{-1}AS$. Similar matrices have the same characteristic polynomial, eigenvalues, determinant, and trace. Their entries change while the scaling behavior of the underlying map does not.

Why similarity preserves characteristic polynomial and trace

Use $tI-S^{-1}AS=S^{-1}(tI-A)S$ and determinant multiplicativity to obtain equal characteristic polynomials. Trace is the sum of diagonal entries. Interchanging finite sums gives

$$\operatorname{tr}(XY)=\sum_i\sum_j x_{ij}y_{ji} =\operatorname{tr}(YX).$$

Consequently $\operatorname{tr}(S^{-1}AS)=\operatorname{tr}(ASS^{-1})=\operatorname{tr}(A)$.

Exercises

ExerciseFrom eigenvalue to eigenspace

Find the eigenvalues of $A=\begin{pmatrix}2&1\\0&3\end{pmatrix}$ and an eigenvector basis.

Solution

The eigenvalues are $2$ and $3$. Corresponding vectors $(1,0)^{\mathsf T}$ and $(1,1)^{\mathsf T}$ are independent, so $A$ is diagonalizable.

ExerciseA stationary state is an eigenvector

Explain why $P\boldsymbol\pi=\boldsymbol\pi$ makes a stationary state an eigenvector for eigenvalue $1$.

Solution

The equation is exactly $P\boldsymbol\pi=1\boldsymbol\pi$. Probability normalization selects from this eigenspace a vector whose entries sum to one.

ExerciseOne distinct root can still suffice

Explain why the identity matrix is diagonalizable although it has only one distinct eigenvalue.

Solution

Its eigenspace for eigenvalue $1$ is all of $\mathbb R^n$, so it contains a basis of $n$ eigenvectors.