My first principal component is almost entirely one feature (salary). Is PCA broken?
Why this happens
PCA finds the directions of greatest variance. Variance depends on units: salary in rupees varies in the hundreds of thousands, while age varies in tens. Without scaling, the salary column has by far the largest variance, so it dominates the first component. PCA is working correctly; it's just measuring the units rather than the structure you care about.
Answer 1: Standardize with StandardScaler before PCA (the usual choice)
StandardScaler gives every feature a mean of 0 and a standard deviation of 1, so each feature starts on equal footing.
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
pipe = Pipeline([
("scale", StandardScaler()),
("pca", PCA(n_components=0.95)),
])
X_reduced = pipe.fit_transform(X)
Putting both steps in a Pipeline ensures the same scaling is applied to new data, and prevents leakage during cross-validation (see Q4).
