Recursive feature elimination with Python

Recursive feature elimination (RFE) is the process of selecting features sequentially, in which features are removed one at a time, or a few at a time, iteration after iteration.

Given a machine learning model, the goal of recursive feature elimination is to select features by recursively considering smaller and smaller sets of features.

In RFE, first an estimator is trained using all features, and then the importance of each variable is obtained. If using Scikit-learn, these would be obtained either from the coefficients of a linear regression model ( coef_) or the importance derived from decision trees ( feature_importances_). Then, the least important feature or group of features would be removed, and a new machine learning model would be trained using the remaining features.

RFE initial steps:

1. Train a machine learning model
2. Derive feature importance
3. Remove least important feature(s)
4. Re-train the machine learning model on the remaining features