I've seen all three used for dimensionality reduction. What's the difference, and when should I use each?
How they differ
PCA is a linear method. It finds straight-line directions of maximum variance. It's fast, deterministic, interpretable (each component is a weighted sum of features), and can transform new data. It can't capture curved or non-linear structure.
t-SNE is a non-linear method designed for visualization. It's very good at showing local groups and clusters. It's slow on large datasets, can't transform new data, and distorts global distances and cluster sizes.
UMAP is also non-linear. It usually produces visualizations similar to t-SNE, while being faster, preserving more global structure, and supporting transform for new data. Its axes still have no direct meaning, and it has several parameters that affect the result.
Answer 1: A practical decision guide
Use PCA as preprocessing for models, for noise reduction, for compression, and whenever you need to explain the components. It's also a good first step before t-SNE or UMAP.
Use t-SNE when you want a 2D visualization of clusters in small to medium datasets and don't need to embed new points.
Use UMAP for visualization of large datasets, for clustering in a reduced space, or when you need to embed new data with a non-linear method.
