Introduction to Backward Deletion Method
With many predictor variables, one can create the most statistically significant model from the data. There are two main choices: forward stepwise regression and backward deletion method.
In Forward Stepwise Regression: Start with the single best variable and add more variables to build your model into a more complex form.
In Backward Deletion (Backward Selection) Regression: put all the variables in the model and reduce the model by removing variables until you are left with only significant terms.
Table of Contents
Backward Deletion method (Step by Step Procedure)
Let’s start with a big model and trim it until you get the best (most statistically significant) regression model. This drop1() command can examine a linear model and determine the effect of removing each one from the existing model. Complete the following steps to perform a backward deletion. Note that the model has different R packages for the Backward and Forward Selection of predictors.
Step 1: (Full Model)
Step 1: To start, create a “full” model (all variables at once in the model). It would be tedious to enter all the variables in the model, one can use the shortcut, the dot notation.
mod <- lm(mpg ~., data = mtcars)
Step 2: Formula Function
Step 2: Let’s use the formula() function to see the response and predictor variables used in Step 1.
formula(mod)
Step 3: Drop1 Function
Step 3: Let’s use the drop1() function to see which term (predictor) should be deleted from the model
drop1(mod)
Step 4: Remove the Term
Step 4: Look to remove the term with the lowest AIC value. Re-form the model without the variable that is non-significant or has the lowest AIC value. The simplest way to do this is to copy the model formula in the clipboard, paste it into a new command, and edit out the term you do not want
mod1 <- lm(mpg ~ ., data = mtcars)
Step 5: Examine the Effect
Step 5: Examine the effect of dropping another term by running the drop1() command once more:
drop1(mod1)
If you see any variable having the lowest AIC value, if found, remove the variable and carry out this process repeatedly until you have a model that you are happy with.