I have a few thousand features and RFECV has been running for hours.
Why this happens
With the default step=1, RFE removes one feature at a time and re-fits the model after each removal. With 2,000 features, that's nearly 2,000 model fits. RFECV repeats this for every fold, so 5-fold CV multiplies the work by five.
Answer 1: Increase step
step controls how many features are removed per iteration. An integer removes that many; a float between 0 and 1 removes that fraction of the remaining features.
RFE(estimator, n_features_to_select=20, step=0.1) # drop 10% each round
RFECV(estimator, step=50, cv=5) # drop 50 at a time
Removing a fraction is especially effective. It takes big steps while there are many features and small steps near the end, where precision matters most.
