Generalized Ridge Regression Estimator in High Dimensional Sparse Regression Models
Abstract
Modern statistical analysis often encounters linear models with the number of explanatory variables much larger than the sample size. Estimation in these high-dimensional problems needs some regularization methods to be employed due to rank deficiency of the design matrix. In this paper, the ridge estimators are considered and their restricted regression counterparts are proposed when the errors are dependent under a multicollinearity and high-dimensionality setting. The asymptotic distributions of the proposed estimators are exactly derived. Incorporating the information contained in the restricted estimator, a shrinkage type ridge estimator is also exhibited and its asymptotic risk is analyzed under some special cases. To evaluate the efficiency of the proposed estimators, a Monte-Carlo simulation along with a real example are considered.References
F. Akdenız, and G. Tabakan, Restricted ridge estimators of the parameters in semiparametric regression model, Communication in Statistics-Theory and Methods, vol. 38, no. 11, pp. 1852–1869, 2009.
E. Akdenız Duran, W.K. Hardle, M. Osipenko, Difference based ridge and Liu type estimators in semiparametric regression models, Journal of Multivariate Analysis, vol. 105, no. 1, pp. 164–175, 2012.
F. Akdenız, and M. Roozbeh, Generalized difference-based weighted mixed almost unbiased ridge estimator in partially linear models, Statistical Papers, DOI: 10.1007/s00362-017-0893-9, 2017.
M. Amini, and M. Roozbeh, Optimal partial ridge estimation in restricted semiparametric regression models, Journal of Multivariate Analysis, vol. 136, no. C, pp. 26–40, 2015.
P. Buhlmann, and S. van de Geer, Statistics for High-dimensional Data: Methods, Theory and Applications, Springer, Heidelberg, 2011.
P. Buhlmann, M. Kalisch, and L. Meier, High dimensional statistics with a view towards applications in biology, Annual Review of Statistics and Its Application, vol. 1, pp. 255–278, 2014.
N. El Karoui, Asymptotic behavior of unregularized and ridge-regularized high-dimensional robust regression estimators: rigorous results, Arxiv: 1311.2445, 2016.
D.G. Gibbons, A simulation study of some ridge estimators, Journal of the American Statistical Association, vol. 76, no. 373, pp. 131–139, 1981.
T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, second edition, Springer, New York, 2009.
W.J. Hemmerle, and T.F. Brantle, Explicit and Constrained Generalized Ridge Estimation, Technometrics, vol. 20, pp, 109–119, 1978.
W.J. Hemmerle, and M.B. Carey, Some Properties of Generalized Ridge Estimators, Communications in Statistics-Simulation and Computation, vol. 12, pp, 239–253, 1983.
A.E. Hoerl, and R.W. Kennard, Ridge regression: biased estimation for non-orthogonal problems, Technometrics, vol. 12, no. 1, pp. 55–67, 1970.
R.R. Hocking, F.M., Speed, and M.J. Lynn, A Class of Biased Estimators in Linear Regression, Technometrics, vol. 18, pp. 425–437, 1976.
B.M.G. Kibria, (2003). Performance of some new ridge regression estimators, Communications in Statistics-Simulation and Computation, vol. 32, pp.419–435.
B.M.G. Kibria, and S. Banik, Some Ridge Regression Estimators and Their Performances, Journal of Modern Applied Statistical Methods, vol. 15, no. 1, pp. 206–238, 2016.
B.M.G. Kibria, and A.K.Md.E. Saleh, Improving the Estimators of the Parameters of a Probit Regression Model: A Ridge Regression Approach, Journal of Statistical Planning and Inference, vol. 142, no. 6, pp. 1421–1435, 2011.
K. Knight, and W. Fu, Asymptotic for lasso-type estimators, Annals of Statistics, vol. 28, no. 5, pp. 1356–1378, 2000.
G.C. McDonald, and D.I. Galarneau, A monte carlo evaluation of some ridge-type estimators, Journal of the American Statistical Association, vol.70, no. 350, pp. 407–416, 1975.
M. Roozbeh, (2015). Shrinkage ridge estimators in semiparametric regression models, Journal of Multivariate Analysis, vol. 136, no. C, pp. 56–74, 2015.
A.K.Md.E. Saleh, Theory of Preliminary Test and Stein-type Estimation with Applications, John Wiley, New York, 2006.
A.N. Tikhonov, A. N. (1963). Solution of incorrectly formulated problems and the regularization method, Soviet Mathematics vol. 4, no. 4, pp. 1035–1038, 1963.
B. Yuzbashi, and S.E. Ahmed, Shrinkage ridge regression estimators in high-dimensional linear models, Proceedings of the Ninth International Conference on Management Science and Engineering Management, pp. 793-807, 2015.
A. Zellner, An efficient method of estimating seemingly unrelated regressions and tests for aggregation bias, Journal of the American Statistical Association, vol. 57, no. 298, pp. 348–368, 1962.
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
