I'm using RFE with SVC(kernel='rbf'), KNeighborsClassifier, or MLPClassifier and I get this error.
Why this happens
At each step, RFE must decide which feature is the weakest. It does this by reading the fitted model's coef_ (for linear models) or feature_importances_ (for tree-based models). Models such as an RBF-kernel SVM, k-nearest neighbors, or a neural network don't provide a per-feature weight, so RFE has nothing to rank by.
Answer 1: Use a model that provides feature weights
If you're not tied to a specific model, use one that RFE supports. A linear-kernel SVM, a linear or logistic regression, or any tree-based model works.
from sklearn.svm import SVC
from sklearn.ensemble import RandomForestClassifier
RFE(SVC(kernel="linear"), n_features_to_select=5)
RFE(RandomForestClassifier(random_state=0),
n_features_to_select=5)
You can select features with one model and then train a different final model on them. Selecting with a random forest and then training your KNN on the selected features is a valid approach.
