mctest R Package for Detection of Collinearity

In this post, I will discuss the existence and detection of collinearity among regressors using the mctest R Package.

The problem of multicollinearity plagues the numerical stability of regression estimates. It also causes some serious problems in the validation and interpretation of the regression model. Consider the usual multiple linear regression model,

$$y = X \beta+u$$

where $y$ is an $n\times 1$ vector of observation on dependent variable, $X$ is known design matrix of order $\times p$, having full-column rank $p$, $\beta$ is $p \times 1$ vector of unknown parameters and $u$ is an $n\times 1$ vector of random errors with mean zero and variance $\sigma^2 I_n$, where $I_n$ is an identity matrix of order $n$.

Collinearity Among Regressors

The existence of linear dependence (relationship) between regressors can affect the regression model’s ability to estimate the model’s parameters. Therefore, multicollinearity is a lack of independence or the presence of interdependence signified by usually high inter-correlations within a set of regressors (predictors).

In case of severe multicollinearity (mathematically when the matrix is ill-conditioned) the $X’X$ matrix cannot be inverted, implausible signs of coefficients, low t-ratios, high R-squared values, inflated standard errors, wider confidence intervals, very large condition number (CN) and non-significant and/or magnitude of regression coefficient estimates are some of the possible issues.

There are many diagnostic methods are available to check the existence of collinearity among regressors, such as variance inflation Factor (VIF), values of pair-wise correlation among regressors, eigenvalues, CN, Farrar and Glauber tests, Theil’s measure, and Klein’s rule, etc.

Our recently developed R package mctest computes several collinearity diagnostics measures to test the existence of collinearity among regressors. We classified these measures as individual collinearity diagnostics and overall collinearity diagnostics. Overall collinearity diagnostic includes determinant of $X’X$ matrix, red indicator, Farrar Chi-Square test, Theil indicator, CN, and the sum of lambda inverse values. Individual collinearity diagnostics include VIF/ TOL, Farrar and Glauber Wi test, the relationship between $R^2$ and F-test, corrected VIF (CVIF), and Klein’s rule.

How to use the mctest R Package

You must have installed and loaded the mctest R Package to start with the testing/ detection of collinearity among regressors. As an example, we used Hald data which is already bundled in the mctest R Package.

mctest R package have 4 functions namely, mctest(), omcdiag(), imcdiag() and mc.plot() functions. The mctest() function can be used to have overall and/or individual collinearity diagnostic. The mc.plot() is used to draw the graph of VIF and eigenvalues to judge collinearity among regressors.

mctest illustrative Example

Arguments of mctest with syntax are

mctest(x, y, type = c("o", "I", "b"), na.rm = TRUE, Inter = TRUE, method = NULL, corr = FALSE, detr = 0.01, red = 0.5, theil = 0.5, cn = 30, vif = 10, tol = 0.1, conf = 0.95, cvif = 10, leamer = 0.1, all = all)

See the detail of each argument and see the mctest package documentation. Following are a few commands that can be used to get different collinearity diagnostics.

x <- Hald[ ,-1]     # X variables from Hald data
> y <- Hald[ ,1]      # y variable from Hald data

mctest(x, y)        # default collinearity diagnostics
mctest(x, y, type = "i")  # individual collinearity diagnostics
mctest(x, y, type = "o") # overall collinearity diagnostics

Overall Collinearity Diagnostics in R

For overall collinearity diagnostics, eigenvalues, and condition numbers are also produced whether the intercept term is included or not. The syntax of omcdiag() function is

omcdiag(x, y, na.rm = TRUE, Inter = True, detr = 0.01, red = 0.5, conf = 0.95, theil = 0.5, cn = 30, …)

Determinant of the correlation matrix, Farrar test of Chi-square, Red indicator, the sum of lambda inverse values, Theils’ indicator, and CN.

omcdiag(x, y, Inter=FALSE)
omcdiag(x, y)[1]

omcidag(x,y, detr = 0.001, conf = 0.99)

For the last command (with the threshold for determinant and confidence interval for the Farrar and Glauber test). The output is,

mctest r package overall collinearity diagnostics

Individual Collinearity Diagnostics in R

The general syntax of individual collinearity Diagnostics in R Language through mctest packages is:

imcdiag(x, y, method = NULL, na.rm = TRUE, corr = FALSE, vif = 10, tol = 0.1, conf = 0.95, cvif = 10, leamer = 0.1, all = all)

Function imcdiag() detects the existence of multicollinearity due to certain X-variable. This includes VIF, TOL, Klein’s rule, CVIF, F&G test of Chi-square and F-test.

imcdiag(x = x, y)
imcdiag(x = x, y, corr = TRUE) # correlation matrix
imcdiag(x = x, y, vif = 5, leamer = 0.05)  # with threshold of VIF and leamer method
mctest r package individual collinearity diagnostics
imcdiag(x = x, y, all = True)
imcdiag(x = x, y, all = TRUE, vif = 5, leamer = 0.2, cvif = 5)
mctest: individual collinearity diagnostics

Graphical Representation of VIF and Eigenvalues

mc.plot(x, y, Inter = FALSE, vif = 10, ev = 0.01)
mc.plot(x, y)
mc.plot(x, y, vif = 5, ev = 0.2)
mctest: collinearity diagnostic measures

For further details about collinearity diagnostic see

R Package Questions and Answers

The post is about some important R package questions and answers. The R Package Questions and Answers are about how to load, install, and remove an R package.

R Package Questions and Answers

Question: What is an R Package?
Answer: The r package is a collection of objects that the R Language can use. A package contains functions, data sets, and documentation (which helps how to use the package) or other objects such as dynamically loaded libraries of already compiled code.

Question: How do I see which packages I have available?
Answer: To see which packages you have to use the command at the R prompt

library()

Question: Which packages do I already have?
Answer: To see what packages are installed one can use the installed.packages() command an R prompt. The output will show the packages installed.

installed.packages()
installed.packages()[1:5,]

Loading R Packages

Question: How one can load a Package in R language?
Answer: Basic packages are already loaded. If you want to load a downloaded version of packages use the command

library("package name")
library("car")

where the package name is the name of the package you want to load. Here in the example, we used the “car”, which means the “car” package will be loaded.

Getting Help in R Language

Question: How one can see the documentation of a particular package?
Answer: To see the documentation of a particular package use the command

library(help="package name")
help(package="package name")
help(package="car")
library(help="car")

for more information about getting help follow the link: Getting Help in R Language

Question: How do I see the help for a specific function?
Answer: To get help with a function in R use the command

help("function name")
? function name
?Manova
help("Manova")

Question: What functions and datasets are available in a package?
Answer: To check what functions and datasets are in a package using the help command at the R prompt. This will provide package information giving a list of functions and datasets.

help(package = "MASS")

Note that once a package is loaded, the help command can also be used with all available functions and datasets.

Installing and Removing R Packages

Question: How can one add or delete a package?
Answer: A package can be installed using the command

install.packages("package name")

and a package can be removed or deleted using the command

remove.packages("package name")
R Packages Questions and Answers R Faqs

Computer MCQs Test Online

SPSS Data Analysis

R Packages: An Introduction

The post is about R Packages in the form of Questions and Answers.

Question: What version of R do I run on my computer or laptop?

Answer: To get the information about the version of R, use the following command at the R prompt.

# get a version of R
R.version.string

You will get a result like

[1] "R version 3.2.1 (2015-06-18)"

Note that a package in R language is a collection of objects that R Language can use. A package contains functions, data sets, and documentation (which helps how to use the package) or other objects such as dynamically loaded libraries of already compiled code.

Installing R Packages

Question: How to check what packages are already installed?

Answer: To get a list of installed packages, write “library()” without quotation marks at the prompt. You will see the list of all of the packages installed in the local R directory of your computer system and then it will list all packages installed globally on your computer system.

# list all packages installed
library( )

You would get results like (note that results below are given as an example only, it’s not a complete list)

in library ‘C:/Users/abcd/Documents/R/win-library/3.2’:
combinat     Combinatorics utilities
proftools      Output Processing Tools for R
rgl                3D visualization device system (OpenGL)

Packages in library ‘C:/Program Files/R/R-3.2.1/library’:
KernSmooth      Functions for kernel smoothing for Wand & Jones (1995)
MASS                Support Functions and Datasets for Venables and Ripley’s MASS
Matrix               Sparse and Dense Matrix Classes and Methods
methods           Formal Methods and Classes
mgcv                Mixed GAM Computation Vehicle with Automatic Smoothness Estimation

Following is a very short list of packages installed in the local library.

Packages in library ‘C:/Users/imdad/Documents/R/win-library/3.5’:

abind               Combine Multidimensional Arrays
AlgDesign      Algorithmic Experimental Design
askpass          Safe Password Entry for R, Git, and SSH
assertthat      Easy Pre and Post Assertions
tibble               Simple Data Frames
plyr                  Tools for Splitting, Applying and Combining Data

Available R Packeges in Local and Global Directory

R packages are essentially a combination of reusable code, documentation, and code that extend the power and capabilities of the R programming language. R packages are designed to be easily installed and used. The R packages are a major reason why R is so popular in data science. There are tens of thousands of R packages available on CRAN and other repositories, covering a wide range of tasks, from data manipulation and analysis to visualization and modeling.

For further details on R Packages, see the link Packages in R Language.

Learn Basic Statistics