C5.0 Classification Models

Classification Trees

To fit a simple classification tree model, we can start with the non-formula method:

library(C50) tree_mod <- C5.0(x = train_data[, vars], y = train_data$Status) tree_mod ## ## Call: ## C5.0.default(x = train_data[, vars], y = train_data$Status) ## ## Classification Tree ## Number of samples: 3000 ## Number of predictors: 2 ## ## Tree size: 4 ## ## Non-standard options: attempt to group attributes

To understand the model, the summary method can be used to get the default C5.0 command-line output:

summary(tree_mod) ## ## Call: ## C5.0.default(x = train_data[, vars], y = train_data$Status) ## ## ## C5.0 [Release 2.07 GPL Edition] Wed Apr 2 20:24:11 2025 ## ------------------------------- ## ## Class specified by attribute `outcome' ## ## Read 3000 cases (3 attributes) from undefined.data ## ## Decision tree: ## ## Home in {owner,parents}: good (1946.3/421.6) ## Home in {ignore,other,priv,rent}: ## :...Seniority > 5: good (447.4/100.4) ## Seniority <= 5: ## :...Seniority <= 0: bad (148/46) ## Seniority > 0: good (458.4/204) ## ## ## Evaluation on training data (3000 cases): ## ## Decision Tree ## ---------------- ## Size Errors ## ## 4 772(25.7%) << ## ## ## (a) (b) <-classified as ## ---- ---- ## 102 726 (a): class bad ## 46 2126 (b): class good ## ## ## Attribute usage: ## ## 99.93% Home ## 35.17% Seniority ## ## ## Time: 0.0 secs

A graphical method for examining the model can be generated by the plot method:

plot(tree_mod)

A variety of options are outlines in the documentation for C5.0Control function. Another option that can be used is the trials argument which enables a boosting procedure. This method is model similar to AdaBoost than to more statistical approaches such as stochastic gradient boosting.

For example, using three iterations of boosting:

tree_boost <- C5.0(x = train_data[, vars], y = train_data$Status, trials = 3) summary(tree_boost) ## ## Call: ## C5.0.default(x = train_data[, vars], y = train_data$Status, trials = 3) ## ## ## C5.0 [Release 2.07 GPL Edition] Wed Apr 2 20:24:11 2025 ## ------------------------------- ## ## Class specified by attribute `outcome' ## ## Read 3000 cases (3 attributes) from undefined.data ## ## ----- Trial 0: ----- ## ## Decision tree: ## ## Home in {owner,parents}: good (1946.3/421.6) ## Home in {ignore,other,priv,rent}: ## :...Seniority > 5: good (447.4/100.4) ## Seniority <= 5: ## :...Seniority <= 0: bad (148/46) ## Seniority > 0: good (458.4/204) ## ## ----- Trial 1: ----- ## ## Decision tree: ## ## Seniority > 5: good (1330.8/319.3) ## Seniority <= 5: ## :...Home in {ignore,other,priv,rent}: bad (666/280.5) ## Home in {owner,parents}: good (1003.2/448.8) ## ## ----- Trial 2: ----- ## ## Decision tree: ## ## Home in {owner,parents}: good (1113.9) ## Home in {ignore,other,priv,rent}: ## :...Seniority <= 0: bad (74.5) ## Seniority > 0: good (1243.5/262.2) ## ## ## Evaluation on training data (3000 cases): ## ## Trial Decision Tree ## ----- ---------------- ## Size Errors ## ## 0 4 772(25.7%) ## 1 3 822(27.4%) ## 2 3 772(25.7%) ## boost 772(25.7%) << ## ## ## (a) (b) <-classified as ## ---- ---- ## 102 726 (a): class bad ## 46 2126 (b): class good ## ## ## Attribute usage: ## ## 100.00% Seniority ## 99.93% Home ## ## ## Time: 0.0 secs

Note that the counting is zero-based. The plot method can also show a specific tree in the ensemble using the trial option.