Robust Lasso Estimator for the Liu-Type regression model and its applications
Robust Lasso Estimator for the Liu-Type regression
Keywords:
Multicollinearity, outliers, Liu-Type regression, LAD-Lasso estimator
Abstract
In this paper, we propose a new estimator for Liu-Type regression model, called the LAD-Lasso-Liu estimator, which addresses the issues of multicollinearity, outliers and it performs the variable selection. By combining the LAD Lasso and Liu-Type estimators, our proposed estimator achieves double shrinkage for the parameters and at the same time it has the robust properties. We thoroughly discuss the properties of the new estimator and conduct a simulation study to demonstrate its superiority over the LAD, Lasso, and LAD-Lasso estimators. We used the Median(MSE) as a criteria to compare between the estimators at a different factors. The simulation results showed that the proposed estimator has superiority over the other estimators especially when the correlation coefficient between the explanatory variables increases and when the error variance decreases. In addition, the proposed estimator has better correct selection of the number of zeros coefficients than other penalized estimators. To demonstrate the work of the estimator presented under empirical data, we apply the proposed estimator to prostate cancer data. Our results for the empirical data indicate that the proposed estimator outperforms the other estimators and can provide accurate results in challenging scenarios with multicollinearity and outliers.References
1. Ana M., Graciela B. and Gonzalo C., (2019)," Penalized robust estimators in logistic regression with applications to sparse models", arXiv: 1911.00554 [stat.ME], Vol.1.
2. Arslan O., (2012)," Weighted LAD-LASSO method for robust parameter estimation and variable selection in regression", Computational Statistics and Data Analysis”, Vol.56 (6).
3. Avi G., Jeffrey S., Bhaskar S., (2006), " Robust weighted LAD regression", Computational Statistics and Data Analysis, Vol.50, 3124-3140.
4. Chi, E., Scott D., (2014)," Robust parametric classification and variable selection by a minimum distance criterion", Journal of Computational and Graphical Statistics, Vol 23, 111-128.
5. Efron, B., Hastie, T., Johnston, I. and Tibshirani, R., (2004)," Least angle regression", The Annals of Statistics, Vol.32 (2).
6. Fela O., (2018)," Evaluation estimation performances of Liu Type and Two parameter using Monte Carlo experiments", IISES Annual Conference, Seville.
7. Giloni A., Simonoff J. and Sengupta B. (2006a)," Robust weighted LAD regression", Computational Statistics & Data Analysis, Vol.50 (11).
8. Golub, G., Wahba, G. and Heat, C. (1979), "Generalized Cross Validation as a Method for Choosing a Good Ridge Parameter", Technimetrics, Vol.21, 215–223.
9. Hoerl A. and Kennard, R., (1970a), "Ridge Regression: Biased Estimation for Non-Orthogonal Problems", Tech., Vol. 12.
10. Hssan S., (2019)," weighted Lasso subsampling for high dimensional regression", Electronic journal of applied statistical analysis, Vol.12 (1).
11. Hu Y., Ning L., (2018)," WLAD-LASSO method for robust estimation and variable selection in partially linear models", Communications in Statistics - Theory and Methods, Vol. 47(20), 4958-4976.
12. Jinyu L., Wei L., Shuyuan H., (2011), "Empirical likelihood for LAD estimators in infinite variance ARMA models", Statistics & Probability Letters, Vol.81 (2), 212-219.
13. Kang j., (2011)," weighted least absolute deviation Lasso estimator", Communication of Korean statistical society Vol.18 (6), 733-739.
14. Kang-Mo J., (2011) "Weighted Least Absolute Deviation Lasso Estimator", Communications of the Korean Statistical Society, Vol. 18(6), 733–739.
15. Lie W., (2013), "The L1 penalized LAD estimator for high dimensional linear regression", Journal of Multivariate Analysis, Vol. 120, 135-151.
16. Liu, K. (2003)," Using Liu-type estimator to combat collinearity ", Communications in Statistics - Theory and Methods. Vol. 32(5), 1009-1020.
17. Lukman, A. F., Ayinde, K., Binuomote, S. and Onate, A. C., (2019). Modified ridge‐type estimator to combat multicollinearity: Application to chemical data. Journal of Chemometrics, e3125. https://doi.org/10.1002/cem.3125
18. Murat G., (2021), "A new double-regularized regression using Liu and lasso regularization", Computational Statistics, https://doi.org/10.1007/s00180-021-01120-4.
19. Olcay A., (2012), "weighted LAD-LASSO method for robust parameter estimation and variable selection regression", computational statistics and data analysis, Vol.56, 1952-1965.
20. Phillips P., (1991), "A Shortcut to LAD Estimator Asymptotics", Economic theory, Vol.7, 450-463.
21. Qiang Li., Liming W., (2017), "Robust change point detection method via adaptive LAD-LASSO", Stat Papers, Vol. 61(1), 109-121.
22. Sakallioglu S., Kaçiranlar S., (2008), "A new biased estimator based on ridge estimation" Statistic Paper, Vol. 49,669–689.
23. Stamey T., Kabalin J., (1989), "Prostate specific antigen in the diagnosis and treatment of adenocarcinoma of the prostate", Official journal of the American Urological Association, Vol.141, 1070-1075.
24. Tibshirani, R. (1996)," Regression shrinkage and selection via the Lasso", Journal of the Royal Statistical Society, Series B, 267-288.
25. Wang H., Li G., Jiang G., (2007)," Robust regression shrinkage and consistent variable selection through the LAD-Lasso", Journal of Business & Economic Statistics, Vol. 25.
26. Yadolah D., (1997), "LAD Regression for Detecting Outliers in Response and Explanatory Variables", Journal of multivariate analysis, Vol. 61, 144-158.
27. Yue S., Zhiguo F., Cedric Y., (2017), " A descent method for least absolute deviation lasso Problems", Optimization Letters, Vol. 13, 543–559.
28. Genc M., (2022) “A new double-regularized regression using Liu and lasso regularization”, Computational Statistics, Vol. 37, No.2, 159-227.
29. Norouzirad M., Hossain S., Arashi M.,(2018), “ Shrinkage and penalized estimators in weighted least absolute deviations regression models”, Journal of Statistical Computation and Simulation, Vol.88, No.8, 1557-1575.
30. Omara T., (2019), “ Modifying Two-Parameter Ridge Liu Estimator Based on Ridge Estimation”, Pakistan Journal of Statistics and Operation, Vol. 15, No. 4, 881-890.
31. Liu K., (1993), “ A New Class of Biased Estimate in Linear Regression”, Communications in Statistics—Theory and Methods, Vo.22, 393-402.
32. Omara T., (2022), “ Almost unbiased modified ridge-type estimator: an application to tourism sector data in Egypt”, Heliyon, Vol.8 No. 9 e10684.
2. Arslan O., (2012)," Weighted LAD-LASSO method for robust parameter estimation and variable selection in regression", Computational Statistics and Data Analysis”, Vol.56 (6).
3. Avi G., Jeffrey S., Bhaskar S., (2006), " Robust weighted LAD regression", Computational Statistics and Data Analysis, Vol.50, 3124-3140.
4. Chi, E., Scott D., (2014)," Robust parametric classification and variable selection by a minimum distance criterion", Journal of Computational and Graphical Statistics, Vol 23, 111-128.
5. Efron, B., Hastie, T., Johnston, I. and Tibshirani, R., (2004)," Least angle regression", The Annals of Statistics, Vol.32 (2).
6. Fela O., (2018)," Evaluation estimation performances of Liu Type and Two parameter using Monte Carlo experiments", IISES Annual Conference, Seville.
7. Giloni A., Simonoff J. and Sengupta B. (2006a)," Robust weighted LAD regression", Computational Statistics & Data Analysis, Vol.50 (11).
8. Golub, G., Wahba, G. and Heat, C. (1979), "Generalized Cross Validation as a Method for Choosing a Good Ridge Parameter", Technimetrics, Vol.21, 215–223.
9. Hoerl A. and Kennard, R., (1970a), "Ridge Regression: Biased Estimation for Non-Orthogonal Problems", Tech., Vol. 12.
10. Hssan S., (2019)," weighted Lasso subsampling for high dimensional regression", Electronic journal of applied statistical analysis, Vol.12 (1).
11. Hu Y., Ning L., (2018)," WLAD-LASSO method for robust estimation and variable selection in partially linear models", Communications in Statistics - Theory and Methods, Vol. 47(20), 4958-4976.
12. Jinyu L., Wei L., Shuyuan H., (2011), "Empirical likelihood for LAD estimators in infinite variance ARMA models", Statistics & Probability Letters, Vol.81 (2), 212-219.
13. Kang j., (2011)," weighted least absolute deviation Lasso estimator", Communication of Korean statistical society Vol.18 (6), 733-739.
14. Kang-Mo J., (2011) "Weighted Least Absolute Deviation Lasso Estimator", Communications of the Korean Statistical Society, Vol. 18(6), 733–739.
15. Lie W., (2013), "The L1 penalized LAD estimator for high dimensional linear regression", Journal of Multivariate Analysis, Vol. 120, 135-151.
16. Liu, K. (2003)," Using Liu-type estimator to combat collinearity ", Communications in Statistics - Theory and Methods. Vol. 32(5), 1009-1020.
17. Lukman, A. F., Ayinde, K., Binuomote, S. and Onate, A. C., (2019). Modified ridge‐type estimator to combat multicollinearity: Application to chemical data. Journal of Chemometrics, e3125. https://doi.org/10.1002/cem.3125
18. Murat G., (2021), "A new double-regularized regression using Liu and lasso regularization", Computational Statistics, https://doi.org/10.1007/s00180-021-01120-4.
19. Olcay A., (2012), "weighted LAD-LASSO method for robust parameter estimation and variable selection regression", computational statistics and data analysis, Vol.56, 1952-1965.
20. Phillips P., (1991), "A Shortcut to LAD Estimator Asymptotics", Economic theory, Vol.7, 450-463.
21. Qiang Li., Liming W., (2017), "Robust change point detection method via adaptive LAD-LASSO", Stat Papers, Vol. 61(1), 109-121.
22. Sakallioglu S., Kaçiranlar S., (2008), "A new biased estimator based on ridge estimation" Statistic Paper, Vol. 49,669–689.
23. Stamey T., Kabalin J., (1989), "Prostate specific antigen in the diagnosis and treatment of adenocarcinoma of the prostate", Official journal of the American Urological Association, Vol.141, 1070-1075.
24. Tibshirani, R. (1996)," Regression shrinkage and selection via the Lasso", Journal of the Royal Statistical Society, Series B, 267-288.
25. Wang H., Li G., Jiang G., (2007)," Robust regression shrinkage and consistent variable selection through the LAD-Lasso", Journal of Business & Economic Statistics, Vol. 25.
26. Yadolah D., (1997), "LAD Regression for Detecting Outliers in Response and Explanatory Variables", Journal of multivariate analysis, Vol. 61, 144-158.
27. Yue S., Zhiguo F., Cedric Y., (2017), " A descent method for least absolute deviation lasso Problems", Optimization Letters, Vol. 13, 543–559.
28. Genc M., (2022) “A new double-regularized regression using Liu and lasso regularization”, Computational Statistics, Vol. 37, No.2, 159-227.
29. Norouzirad M., Hossain S., Arashi M.,(2018), “ Shrinkage and penalized estimators in weighted least absolute deviations regression models”, Journal of Statistical Computation and Simulation, Vol.88, No.8, 1557-1575.
30. Omara T., (2019), “ Modifying Two-Parameter Ridge Liu Estimator Based on Ridge Estimation”, Pakistan Journal of Statistics and Operation, Vol. 15, No. 4, 881-890.
31. Liu K., (1993), “ A New Class of Biased Estimate in Linear Regression”, Communications in Statistics—Theory and Methods, Vo.22, 393-402.
32. Omara T., (2022), “ Almost unbiased modified ridge-type estimator: an application to tourism sector data in Egypt”, Heliyon, Vol.8 No. 9 e10684.
Published
2025-02-15
How to Cite
Omara, T. (2025). Robust Lasso Estimator for the Liu-Type regression model and its applications . Statistics, Optimization & Information Computing. https://doi.org/10.19139/soic-2310-5070-2106
Issue
Section
Research Articles
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
