ASSESSING THE PERFORMANCE OF PENALIZED REGRESSION METHODS AND THE CLASSICAL LEAST SQUARES METHOD
ASSESSING THE PERFORMANCE OF PENALIZED REGRESSION METHODS AND THE CLASSICAL LEAST SQUARES METHOD
No Thumbnail Available
Date
2015-07
Authors
MATTHEW, Pascalis Kadaro
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Regression is one of the most useful statistical methods for data analysis. Multicollinearity is a problem that, pose a challenge to regression analysis by increasing the standard error of the estimators, making the model to be less predictive and difficult for interpretation. Penalized regression which is a variable selection techniquehave been developed specifically to eliminate the problem of multicollinearity and also reduce the flaws inherent in the prediction accuracy of the ordinary least squares (OLS) regression technique. In this thesis, the focus is on the numerical study of these three penalized methods, namely: least absolute shrinkage selection operator (LASSO), elastic net and the newly introduced correlation adjusted elastic net (CAEN). A diabetes dataset which was shown to possess the qualities of multicollinearitywas obtained from previous literature to compare these well-known techniques. 10-fold cross validation (CV) within glmnet package was used to entirely search for the optimal λ.The whole path of results (in λ ) for the LASSO, Elastic Net and CAEN models were calculated using the path wise Cyclic Coordinate Descent (CCD) algorithms– in glmnet package in R,a computationally effective technique for finding out these convex optimization solutions. A regularized profile plot of the coefficient paths for the three methods, were also shown. Predictive accuracy was also assessed using the mean squared error (MSE) and the penalized regression models were able to produce feasible and efficient models capable of capturing the linearity in the data than the ordinary least squares model.It was observed that correlation adjusted elastic net generates a less complex model with a minimum mean square error (MSE).
Description
A THESIS SUBMITTED TO THE SCHOOL OF POSTGRADUATE STUDIES, AHMADU BELLO UNIVERSITY, ZARIA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE AWARD OF A MASTER DEGREE IN STATISTICS. DEPARTMENT OF MATHEMATICS, FACULTY OF SCIENCE AHMADU BELLO UNIVERSITY, ZARIA NIGERIA
Keywords
ASSESSING,, PERFORMANCE,, PENALIZED,, REGRESSION METHODS,, CLASSICAL LEAST,, SQUARES METHOD,