Eigenvalues and Eigenvectors

The special directions a linear transformation merely scales — central to PCA, differential equations, and Google's PageRank.

Eigenvectors only get scaled — other vectors also rotate

u angle: 30°

Eigenvectors (green, purple) keep their direction under M — only their length changes

Definition

An eigenvector of a square matrix $A$ is a nonzero vector $\mathbf{v}$ that only gets scaled (not rotated) when multiplied by $A$ :

$A\mathbf{v} = \lambda \mathbf{v}$

The scalar $\lambda$ is the corresponding eigenvalue. The word comes from German — eigen means "own" or "characteristic."

Geometrically: most vectors change direction when multiplied by $A$ . Eigenvectors are special — they stay on the same line, just stretched or flipped.

Simple 2×2 example

$A = \begin{pmatrix} 2 & 1 \\ 0 & 3 \end{pmatrix}$

$A\begin{pmatrix}1\\0\end{pmatrix} = \begin{pmatrix}2\\0\end{pmatrix} = 2\begin{pmatrix}1\\0\end{pmatrix}$ — so $(1,0)^T$ is an eigenvector with $\lambda = 2$ .

$A\begin{pmatrix}1\\1\end{pmatrix} = \begin{pmatrix}3\\3\end{pmatrix} = 3\begin{pmatrix}1\\1\end{pmatrix}$ — so $(1,1)^T$ is an eigenvector with $\lambda = 3$ .

Try it

Is the zero vector an eigenvector? Why or why not? What about a scalar multiple of an eigenvector?

Solution

No: eigenvectors are defined to be nonzero — otherwise $A\mathbf{0} = \lambda\mathbf{0}$ is trivially satisfied for any $\lambda$ with no information content.

A scalar multiple $c\mathbf{v}$ (with $c \neq 0$ ) is also an eigenvector with the same eigenvalue: $A(c\mathbf{v}) = cA\mathbf{v} = c\lambda\mathbf{v} = \lambda(c\mathbf{v})$ . Eigenvectors define directions, not specific vectors.

Related concepts

Needs first

Determinants Linear Transformations Matrix Multiplication

Markov Chains Principal Component Analysis

Uses this

Linear Discriminant Analysis

View in full concept graph →