Random Forest on Credit Card Approval Classification

This is an excerpt from my Kaggle notebook where I used a Random Forest classifier on credit card data (https://www.kaggle.com/datasets/rohitudageri/credit-card-details/data).

Random Forest was extremely easy to use and offer great insights into the data relatively quickly.

Exploring the Data

The CSV had a total 1548 rows, the table below shows the top 10 rows and with an additional column "Approved" that is 1 or 0

Modes

The table below shows the modes for all columns

Number of Approved vs Declined

Note the data is heavily skewed to "Declined", this will be important later when fitting the model.

Education Distribution

Mostly high school and junior college.

Occupation Type Distribution

Income Type Distribution

Marital Status Distribution

Employed Days Distribution

This looks odd, but, it's the start day of the job backwards from the current day (0). A positive number means the person is currently unemployed (currently at the time of collection). Mostly around 5-7 years employment

Housing Type Distribution

Preprocess Data

We'll first separate the categorical and continuous fields.

• Age is calculated in years from the birthday_count field.
• Two new fields are added that count the number of employed days and unemployed days for each person

Random Forest Classifier

• Given how skewed the classes are, over sampling is needed.
• Given how small the dataset is, undersampling won't be used.

X, y = df[cats + conts].copy(), df[dep]

X_over, y_over = RandomOverSampler().fit_resample(X, y)

X_train, X_val, y_train, y_val = train_test_split(X_over, y_over, test_size=0.25)

X_train[cats] = X_train[cats].apply(lambda x: x.cat.codes)
X_val[cats] = X_val[cats].apply(lambda x: x.cat.codes)

rf = RandomForestClassifier(100, oob_score=True)
rf.fit(X_train, y_train);

Feature Importance

And now my favorite part! This plot shows the most influential fields in the data. These are the ones that have the highest split ratio. No surprise length of employment and age are the 2 dominant factors. Annual income a close 3rd. So much data analysis can be condensed into the feature importance plot!

Confusion

There were no cases where the model predicted a "Declined" result where the actual status was "Approved". So the model is very good and accurately declining a credit approval. But it's a little too lenient where it predicted 8 would get approved when they were actually declined.

confusion = confusion_matrix(y_val, preds)

It follows that the ROC curve would almost be perfect, with AUC 0.988

Results

This was a non-starter without oversampling. Without oversampling, accuracy was deceptively pretty good around 91% but when looking at the confusion matrix and abismal F1 score it was obviously aweful.

I split employed days column into "unemployed" and "employed". It's not surprising looking at the feature importance plot that unemployed days, age and income are the top contributors.

Final Results: