f_classif prints warnings about constant features and some scores are NaN. Other times I get an error about NaN in the input.
Why this happens
These are two different problems.
The first is constant features. If a column has the same value in every row, it has zero variance. The F-test divides by the variance, so it ends up dividing by zero and the score becomes NaN. SelectKBest treats NaN scores as the lowest possible, so constant features won't be selected, but you still get the warnings.
The second is missing values. Scikit-learn's selectors don't accept NaN in the input, so they raise an error.
Answer 1: Remove constant features first with VarianceThreshold
VarianceThreshold drops columns with zero (or very low) variance. Put it before the selector:
from sklearn.feature_selection import VarianceThreshold
pipe = Pipeline([
("var", VarianceThreshold(threshold=0.0)),
("select", SelectKBest(f_classif, k=10)),
])
Constant features carry no information, so removing them is always safe. You can raise the threshold to also drop near-constant features. Choose the value with care, because variance depends on the feature's scale.
