随机森林模型调参实战

这是一道kaggle上的题目,通过信用卡交易记录数据对欺诈行为进行预测,信用卡欺诈检测文件记录了2013年9月欧洲信用卡持有者所发生的交易。在284807条交易记录中共包含492条欺诈记录。 

数据集下载地址：请在公众号后台回复[56] 

需要说明的是，本文重点是RF模型调参，所以不涉及数据预处理、特征工程和模型融合的内容，这些我会在本栏目未来的章节中再做介绍。所以最终结果可能会不理想，这里我们只关注通过调参给模型带来的性能提升和加深对重要参数的理解即可。

1、导入模块

import numpy as np
import pandas as pd
from sklearn.model_selection import GridSearchCV,train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import roc_auc_score

2、导入数据

df = pd.read_csv("creditcard.csv")
data=df.iloc[:,1:31]

284807条交易记录中只有492条欺诈记录，样本严重不平衡，这里我们需要使用下采样策略(减少多数类使其数量与少数类相同)

X = data.loc[:, data.columns != 'Class']
y = data.loc[:, data.columns == 'Class']

number_records_fraud = len(data[data.Class == 1]) # class=1的样本函数
fraud_indices = np.array(data[data.Class == 1].index) # 样本等于1的索引值
normal_indices = data[data.Class == 0].index # 样本等于0的索引值
random_normal_indices = np.random.choice(normal_indices,number_records_fraud,replace = False)
random_normal_indices = np.array(random_normal_indices)
under_sample_indices = np.concatenate([fraud_indices,random_normal_indices]) # Appending the 2 indices
under_sample_data = data.iloc[under_sample_indices,:] # Under sample dataset
X_undersample = under_sample_data.loc[:,under_sample_data.columns != 'Class']
y_undersample = under_sample_data.loc[:,under_sample_data.columns == 'Class']
X_train, X_test, y_train, y_test = train_test_split(X_undersample,y_undersample,test_size = 0.3, random_state = 0)

3、先用默认参数训练RF

rf0 = RandomForestClassifier(oob_score=True, random_state=666)
rf0.fit(X_train,y_train)
print(rf0.oob_score_)
y_predprob = rf0.predict_proba(X_test)[:,1]
print("AUC Score (Train): %f" % roc_auc_score(y_test, y_predprob))

0.9244186046511628

 AUC Score (Train): 0.967082 

除oob_score将默认的False改为True, 我们重点优化n_estimators、max_depth、min_samples_leaf 这三个参数。为简单起见，模型评价指标，我们选择AUC值。

模型调优我们采用网格搜索调优参数(grid search)，通过构建参数候选集合，然后网格搜索会穷举各种参数组合，根据设定评定的评分机制找到最好的那一组设置。

先优化n_estimators

param_test1 = {'n_estimators':range(10,101,10)}
gsearch1 = GridSearchCV(estimator = RandomForestClassifier(oob_score=True, random_state=666,n_jobs=2),
                       param_grid = param_test1, scoring='roc_auc',cv=5)
gsearch1.fit(X_train,y_train)
gsearch1.cv_results_, gsearch1.best_params_, gsearch1.best_score_

{'n_estimators': 50}, 0.9799524239675649

n_estimators=50，优化max_features

param_test2 = {'max_depth':range(2,12,2)}
gsearch2 = GridSearchCV(estimator = RandomForestClassifier(n_estimators= 50,oob_score=True, random_state=666,n_jobs=2),
   param_grid = param_test2, scoring='roc_auc',cv=5)
gsearch2.fit(X_train,y_train)
gsearch2.cv_results_, gsearch2.best_params_, gsearch2.best_score_

{'max_depth': 6}, 0.9809897227343921

n_estimators=50，max_features=6，优化max_depth

param_test2 = {'min_samples_split':range(2,8,1)}
gsearch2 = GridSearchCV(estimator = RandomForestClassifier(n_estimators= 50,max_depth=6,
                                  oob_score=True, random_state=666,n_jobs=2),
   param_grid = param_test2, scoring='roc_auc',cv=5)
gsearch2.fit(X_train,y_train)
gsearch2.cv_results_, gsearch2.best_params_, gsearch2.best_score_

{'min_samples_split': 5}, 0.9819618127837587

最后我们将优化后的参数带入模型

rf1 = RandomForestClassifier(n_estimators= 50,max_depth=6,min_samples_split=5,oob_score=True, random_state=666,n_jobs=2)
rf1.fit(X_train,y_train)
print(rf1.oob_score_)
y_predprob1 = rf1.predict_proba(X_test)[:,1]
print("AUC Score (Train): %f" % roc_auc_score(y_test, y_predprob1))

0.9331395348837209 AUC Score (Train): 0.977811 

最终结果比默认参数时的模型袋外估计准确率得分、测试集上AUC值均有所提升。