Generalized ridge estimator shrinkage estimation based on particle swarm optimization algorithm.

Section: Research Paper
Issue
Vol. 17 No. 2 (2020): Volume 17 Issue 2
Published
Jun 25, 2025
Pages
26-35

Abstract

It is well-known that in the presence of multicollinearity, the ridge estimator is an alternative to the ordinary least square (OLS) estimator. Generalized ridge estimator (GRE) is an generalization of the ridge estimator. However, the efficiency of GRE depends on appropriately choosing the shrinkage parameter matrix which is involved in the GRE. In this paper, a particle swarm optimization method, which is a metaheuristic continuous algorithm, is proposed to estimate the shrinkage parameter matrix. The simulation study and real application results show the superior performance of the proposed method in terms of prediction error.

References

  1. Al-Hassan YM. Performance of a new ridge regression estimator. Journal of the Association of Arab Universities for Basic and Applied Sciences. 2010; 9: 23-26.
  2. Algamal ZY. Shrinkage parameter selection via modified cross-validation approach for ridge regression model. Communications in Statistics - Simulation and Computation. 2018: 1-9.
  3. Alheety M and Kibria BG. A generalized stochastic restricted ridge regression estimator. Communications in Statistics-Theory and Methods. 2014; 43: 4415-4427.
  4. Alkhamisi M,Khalaf G and Shukur G. Some modifications for choosing ridge parameters. Communications in Statistics - Theory and Methods. 2006; 35: 2005-2020.
  5. Alkhamisi MA and Shukur G. A Monte Carlo study of recent ridge parameters. Communications in StatisticsSimulation and Computation. 2007; 36: 535-547.
  6. Allen DM. The relationship between variable selection and data agumentation and a method for prediction. technometrics. 1974; 16: 125-127.
  7. Asar Y and Gen A. New shrinkage parameters for the Liu-type logistic estimators. Communications in Statistics - Simulation and Computation. 2015; 45: 1094-1103.
  8. Asar Y,Karaibrahimolu A and Gen A. Modified ridge regression parameters: A comparative Monte Carlo study. Hacettepe Journal of Mathematics and Statistics. 2014; 43: 827-841.
  9. Batah FSM,Ramanathan TV and Gore SD. The efficiency of modified jackknife and ridge type regression estimators: a comparison. Surveys in Mathematics & its Applications. 2008; 3.
  10. Bhat S and Raju V. A class of generalized ridge estimators. Communications in Statistics-Simulation and Computation. 2017; 46: 5105-5112.
  11. Bhat S,Vidya R and Parameshwar VP. Maximum Likelihood Estimation of Parameters in a Mixture Model. Communications in Statistics-Simulation and Computation. 2016; 45: 1776-1784.
  12. Bhat SS. A comparative study on the performance of new ridge estimators. Pakistan Journal of Statistics and Operation Research. 2016; 12: 317-325.
  13. Cervantes J,Garcia-Lamont F,Rodriguez L,Lpez A,Castilla JR and Trueba A. PSO-based method for SVM classification on skewed data sets. Neurocomputing. 2017; 228: 187-197.
  14. Chen A-C and Emura T. A modified Liu-type estimator with an intercept term under mixture experiments. Communications in Statistics-Theory and Methods. 2017; 46: 6645-6667.
  15. Chen K-H,Wang K-J,Wang K-M and Angelia M-A. Applying particle swarm optimization-based decision tree classifier for cancer classification on gene expression data. Applied Soft Computing. 2014; 24: 773-780.
  16. Dorugade A and Kashid D. Alternative method for choosing ridge parameter for regression. Applied Mathematical Sciences. 2010; 4: 447-456.
  17. Dorugade A. New ridge parameters for ridge regression. Journal of the Association of Arab Universities for Basic and Applied Sciences. 2014; 15: 94-99.
  18. Firinguetti L. A generalized ridge regression estimator and its finite sample properties: A generalized ridge regression estimator. Communications in Statistics-Theory and Methods. 1999; 28: 1217-1229.
  19. Hamed R,Hefnawy AEL and Farag A. Selection of the ridge parameter using mathematical programming. Communications in Statistics - Simulation and Computation. 2013; 42: 1409-1432.
  20. Hefnawy AE and Farag A. A combined nonlinear programming model and Kibria method for choosing ridge parameter regression. Communications in Statistics - Simulation and Computation. 2014; 43: 1442-1470.
  21. Hocking RR,Speed F and Lynn M. A class of biased estimators in linear regression. Technometrics. 1976; 18: 425-437.
  22. Hocking RR. A Biometrics invited paper. The analysis and selection of variables in linear regression. Biometrics. 1976; 32: 1-49.
  23. Hoerl AE and Kennard RW. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics. 1970; 12: 55-67.
  24. Kennedy J and Eberhart RC. Particle swarm optimization. Proceedings of IEEE Conference on Neural Network. 1995; 4: 19421948.
  25. Khalaf G and Shukur G. Choosing ridge parameter for regression problems. Communications in Statistics - Theory and Methods. 2005; 34: 1177-1182.
  26. Kibria BMG. Performance of some new ridge regression estimators. Communications in Statistics - Simulation and Computation. 2003; 32: 419-435.
  27. Kiran MS. Particle swarm optimization with a new update mechanism. Applied Soft Computing. 2017; 60: 670-678.
  28. Lai C-M,Yeh W-C and Chang C-Y. Gene selection using information gain and improved simplified swarm optimization. Neurocomputing. 2016; 218: 331-338.
  29. Lin S-W,Ying K-C,Chen S-C and Lee Z-J. Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Systems with Applications. 2008; 35: 1817-1824.
  30. Loesgen K. A generalization and Bayesian interpretation of ridge-type estimators with good prior means. Statistical Papers. 1990; 31: 147.
  31. Lu Y,Wang S,Li S and Zhou C. Particle swarm optimizer for variable weighting inclustering high-dimensional data. Machine Learning. 2009; 82: 43-70.
  32. Mnsson K,Shukur G and Golam Kibria B. A simulation study of some ridge regression estimators under different distributional assumptions. Communications in Statistics-Simulation and Computation. 2010; 39: 1639-1670.
  33. McDonald GC and Galarneau DI. A Monte Carlo evaluation of some ridge-type estimators. Journal of the American Statistical Association. 1975; 70: 407-416.
  34. Mirjalili S and Lewis A. S-shaped versus V-shaped transfer functions for binary Particle Swarm Optimization. Swarm and Evolutionary Computation. 2013; 9: 1-14.
  35. Muniz G and Kibria BMG. On some ridge regression estimators: An empirical comparisons. Communications in Statistics - Simulation and Computation. 2009; 38: 621-630.
  36. Nomura M. On the almost unbiased ridge regression estimator. Communications in Statistics-Simulation and Computation. 1988; 17: 729-743.
  37. Ohtani K. Generalized ridge regression estimators under the LINEX loss function. Statistical Papers. 1995; 36: 99-110.
  38. References
  39. Trenkler G and Toutenburg H. Mean squared error matrix comparisons between biased estimatorsAn overview of recent results. Statistical Papers. 1990; 31: 165.
  40. Troskie C and Chalton D. Multidimensional statistical analysis and theory of random matrices, Proceedings of the Sixth Lukacs Symposium, eds. Gupta, AK and VL Girko1996, pp 273-292.
  41. Wen JH,Zhong KJ,Tang LJ,Jiang JH,Wu HL,Shen GL and Yu RQ. Adaptive variable-weighted support vector machine as optimized by particle swarm optimization algorithm with application of QSAR studies. Talanta. 2011; 84: 13-18.
  42. Woods H,Steinour HH and Starke HR. Effect of composition of Portland cement on heat evolved during hardening. Industrial & Engineering Chemistry. 1932; 24: 1207-1214.
  43. Yang S-P and Emura T. A Bayesian approach with generalized ridge estimation for high-dimensional regression and testing. Communications in Statistics-Simulation and Computation. 2017; 46: 6083-6105.
  44. Zhou W and Dickerson JA. A novel class dependent feature selection method for cancer biomarker discovery. Computers in Biology and Medicine. 2014; 47: 66-75.

Authors

Zakariya Yahya Algamal

Dept. of Statistics and Informatics/ college of computer sciences and mathematics/ University of Mosul, Mosul, iraq

ORCID
Qamar Abdul kareem

Department of Statistics and Informatics, College of Computer science and Mathematics, University of Mosul, Mosul, Iraq

Identifiers

Download this PDF file
PDF

Statistics

How to Cite

Yahya Algamal, Z., زکریا, Abdul kareem, Q., & قمر. (2025). Generalized ridge estimator shrinkage estimation based on particle swarm optimization algorithm. IRAQI JOURNAL OF STATISTICAL SCIENCES, 17(2), 26–35. Retrieved from https://rjps.uomosul.edu.iq/index.php/stats/article/view/20766