Estimation Procedure for Reduced Rank Regression, PLSSVD
Abstract
This paper presents a procedure for coefficient estimation in a multivariate regression model of reduced rank in the presence of multicollinearity. The procedure permits the prediction of the dependent variables taking advantage of both Partial Least Squares (PLS) and Singular Value Decomposition (SVD) methods, which is denoted by PLSSVD. Global variability indices and prediction error sums are used to compare the results obtained by classical regression with reduced rank (OLSSVD) and the PLSSVD procedure when applied to examples with different grades of multicollinearity (severe, moderate and low). In addition, simulations to compare the methods were performed with different sample sizes under four scenarios. The new PLSSVD method is shown to be more effective when the multicollinearity is severe and especially for small sample sizes.References
Álvarez W. and Cárdenas O., 2010. Un enfoque basado en la descomposición en valores singulares generalizada para el biplot de coeficientes de regresión. Ingenier Industrial.Actualidad y Nuevas Tendencias. A III. vol. II, N4. ISSN: 1856-8327, p. 65-74.
Anderson T., (2002). Specification and Misspecification in Reduced Rank Regression. The Indian Journal of Statistics, 64,193-205.
Anderson T., (1999). Asymptotic Distibution of the Reduced Rank Regression estimator under general conditions. The Annals of Statistcs, 27, 1141-1154.
Anderson T., (1951). Estimating Linear Restrictions on Regression Coefficients for Multivariate Normal Distributions. Ann. Math. Statist, 22, 327-351.
Davies, P.and Tso M., (1982). Procedures for reduced rank regression. Applied Statistic 31, 244-255.
Eckart, C.and Young, G, (1936). The Approximation de One Matrix for Another of Lower Rank. Psychometrika 1, 211-218.
Gabriel, K., (1971). The Biplot-graphic display of matrices with applications to principal component analysis. Biometrika, 58, 453-467.
Graybill, F.A. (1969). Introducction to Matrices with Statistical Applications. Belmont, California. Wadsworth.
Heinen A. and Rengifo E., (2004). Multivariate Reduced Rank Regression in non-Gaussian Contexts, Using Copulas. Center for Operations Research and Econometrics Catholic University of Louvain Belgium. Roma.
Hervé Abdi, (2003). Partial Least Squares (PLS) Regression. Eds. Encyclopedia of Social Sciencies Research Methods.
Höskuldsson, A., (1988). PLS regression methods. Journal of Chemometrics, 2, 211- 228.
Izenman, A. J., (1975). Reduced rank regression for the multivariate linear model. Journal of Multivaiate Analysis 5, 248-264.
Johnson, R.and Wichern, (1998). Applied Multivariate Statistical Analysis. Prentice Hall, Upper Saddle River, New Jersey.
Jollife I.T. (1986) Principal Component Analysis. Springer - Verlag.
Mardia K.V., Kent J.T. and Bibby J.M. (1979). Multivariate Analysis, London Academic Press.
Martens H. and NæT. (1989). Multivariante Calibration. John Wiley and Sons.
Montgomery D. and Peck E. (1992). Introduction to Linear Regression Analysis. Second Edition. John Wiley and Sons.
Pena, D., (2002). Análisis de Datos Multivariantes. Mc Graw Hill.
Rencher A. (2002). Methods of Multivariate Analysis. Wiley-Interscience Publication.
Tenenhaus, M. (1998). La régression pls, théorie et pratique.Éditions Technip. Paris.
Ter Braak, C. and Looman, C. (1994). Biplots in reduced rank regression. Biometrical Journal 8, 983-1003.
TerBraak, C. (1990).Interpreting canonical correlationanalysis through biplotsof structuralcorrelations and weights.Psychometrika, 55, 519-531.
Tso, M. K-S (1981). Reduced-rank regression and canonical analysis. J.R. Statist. Soc. B, 43, 183-189.
Valencia J., D´ıaz-Llanos F. and Sainz-Calleja (2004). Métodos de Predicción en Situaciones L´ımite. Editorial La Muralla, S.A. Madrid Espa.
Van Der Wollenberg, A.L. (1977). Redundancy Analysis. An Alternative for Canonical Correlation Analysis. Psychometrika, 42:207-219.
Vásquez, M. (1995). Aportaciones al Análisis Biplot: Un enfoque algebraico. Thesis Doctoral. Universidad de Salamanca.
Velu Raja P., Reinsel Gregory C. and Wichern Dean W. (1986). Reduced rank models for multiple time series. Biometrika Vol. 73.1, pp.105-18.
Wold H. (1966). Estimation of principal components and related models by iterative least squares. In Multivariate analysis (ed. P.R. Krishnaiah), Academic Press, New York.
Wold S., Martens H. and Wold H. (1983), in Lecture Notes in Mathematics: Matrix Pencils, ed. by A. Ruhe and B. Kagström, pp. 286-293. Springer, Heidelberg
Qing-Song Xu, Yi-Zeng Liang and Hai- Lin Shen (2001). Generalized PLS regression. Journal of Chemometric, Vol. 15, 135-148.
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
