My dataset has categorical columns like 'city' and 'product type'. Can I use PCA on them?
Why this is tricky
PCA is designed for continuous numeric data. It relies on variance and linear relationships, which don't have a natural meaning for categories. You can still reduce categorical data, but you need to encode it first, or use a method designed for categories.
Answer 1: One-hot encode, then apply PCA
This is the most common practical approach. It works, but be aware of the drawbacks: categories with many levels create many sparse columns, and rare categories can have outsized influence after scaling.
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder
prep = ColumnTransformer([
("cat", OneHotEncoder(handle_unknown="ignore",
sparse_output=False), cat_cols),
("num", StandardScaler(), num_cols),
])
pipe = Pipeline([("prep", prep), ("pca", PCA(n_components=0.9))])
X_reduced = pipe.fit_transform(X)
sparse_output was called sparse before scikit-learn 1.2. If the one-hot output is very large, keep it sparse and use TruncatedSVD instead (see Q6).
