Short Definition
PCA is a dimensionality-reduction method that finds directions of maximum variance in data.
Intuition
PCA rotates the coordinate system so the first few axes capture as much variation as possible.
Technical Definition
Principal Component Analysis projects data onto orthogonal components derived from eigenvectors or singular vectors of a covariance-related matrix.
Example
PCA can reduce a high-dimensional feature matrix to two or three dimensions for inspection.
Common Misunderstandings
PCA is linear, so it can miss nonlinear structure.
High variance directions are not always the most predictive directions.