A New Family of Hybrid Conjugate Gradient Methods for Unconstrained Optimization
Abstract
The conjugate gradient method is a very efficient iterative technique for solving large-scale unconstrainedoptimization problems. Motivated by recent modifications of some variants of the method and construction of hybrid methods, this study proposed four hybrid methods that are globally convergent as well as computationally efficient. The approach adopted for constructing the hybrid methods entails projecting ten recently modified conjugate gradient methods. Each of the hybrid methods is shown to satisfy the descent property independent of any line search technique and globally convergent under the influence of strong Wolfe line search. Results obtained from numerical implementation of these methods and performance profiling show that the methods are very competitive with well-known traditional methods.References
O. J. Adeleke, and I. A. Osinuga, A five-term hybrid conjugate gradient method with global convergence and descent properties for unconstrained optimization problems, Asian Journal of Scientific Research, vol. 11, pp. 185–194, 2018.
A. O Adewumi, and O. J. Adeleke, A survey of recent advances in vehicle routing problems, International Journal of System Assurance Engineering and Management, vol. 9, no. 1, pp. 155–172, 2018.
M. Al-Baali, Descent property and global convergence of the Fletcher-Reeves method with inexact line search, IMA Journal of Numerical Analysis, vol. 5, no. 1, pp. 121–124, 1985.
N. Andrei, An unconstrained optimization test functions collection, Adv. Mod. and Optim., vol. 10, no. 1, pp. 147–161, 2008.
I. Bongartz, A. R. Conn, N. I. M. Gould, and P. L. Toint, CUTE: constrained and unconstrained testing environments, ACM Trans. Math. Software, vol. 21, pp. 123–160, 1995.
Y. H. Dai, and Y. Yuan, Convergence properties of the Fletcher-Reeves method, IMA Journal of Numerical Analysis, vol. 16, no. 2, pp. 155–164, 1996.
Y. H. Dai, and Y. Yuan, A nonlinear conjugate gradient method with a strong global convergence property, SIAM J. Optim., vol. 10, pp. 177–182, 1999.
Y. H. Dai, and Y. Yuan, An efficient hybrid conjugate gradient method for unconstrained optimization, Annals of Operations Research, vol. 103, pp. 33–47, 2001.
E. D. Dolan, and J. J. Mor´e, Benchmarking optimization software with performance profiles, Math. Program., vol. 91, pp. 201–213, 2002.
X. Du, P. Zhang, and W. Ma, Some modified conjugate gradient methods for unconstrained optimization, Journal of Computational and Applied Mathematics, vol. 305, pp. 92–114, 2016.
A. E. Ezugwu, O. J. Adeleke, and S. Viriri, Symbiotic organisms search algorithm for the unrelated parallel machines scheduling with sequence-dependent setup times, PloS one, vol. 13, no. 7, 2018.
A. E. Ezugwu, O. J. Adeleke, A. A. Akinyelu, and S. Viriri, A conceptual comparison of several metaheuristic algorithms on continuous optimisation problems, Neural Computing and Applications, vol. 2019, pp. 1–45, 2019.
R. Fletcher, and C. Reeves, Function minimization by conjugate gradients, Comput. J., vol. 7, pp. 149–154, 1964.
W.W. Hager, and H. Zhang, A new conjugate gradient method with guaranteed descent and an efficient line search, SIAM J. Optim., vol. 16, no. 1, pp. 170–192, 2005.
W. W. Hager, and H. Zhang, A survey of nonlinear conjugate gradient methods, Pacific Journal of Optimization, vol. 2, pp. 35–58, 2006.
M. R. Hestenes, and E. L. Stiefel, Method of conjugate gradients for solving linear systems, J. Res. Nat. Bur. Standards, vol. 49, pp. 409–436, 1952.
H. Huang, Z.Wei, and Y. Shengwei, The proof of the sufficient descent condition of the WeiYaoLiu conjugate gradient method under the strong WolfePowell line search, Applied Mathematics and Computation, vol. 189, no. 2, pp. 1241–1245, 2007.
H. D. Huang, Y. J. Li, and Z. W. Wei, Global convergence of a modified PRP conjugate gradient method, J. Math. Res. Exposition, vol. 30, no. 1, pp. 141–148, 2010.
J. Jian, L. Han, and X. Jiang, A hybrid conjugate gradient method with descent property for unconstrained optimization, Applied Mathematical Modelling, vol. 39, pp. 1281–1290, 2015.
X. Jiang, and J. Jian, Improved FletcherReeves and DaiYuan conjugate gradient methods with the strong Wolfe line search, Journal of Computational and Applied Mathematics, vol. 348, pp. 525–534, 2019.
Q. Li, A modified Fletcher-Reeves-type method for nonsmooth convex minimization, Statistics, Optimization & Information Computing, vol. 2, no. 3, pp. 200–210, 2014.
X. Li, W. Zhang, and X. Dong, A class of modified FR conjugate gradient method and applications to non-negative matrix factorization, Computers & Mathematics with Applications, vol. 73, no. 2, pp. 270–276, 2017.
X. Li, J. Shi J, X. Dong, and J. Yu, A new conjugate gradient method based on Quasi-Newton equation for unconstrained optimization, Journal of Computational and Applied Mathematics, vol. 350, pp. 372–379, 2019.
Y. Liu, and C. Storey, Efficient generalized conjugate gradient algorithms, Part 1: Theory, J. Optim. Theory Appl., vol. 69, pp. 129–137, 1991.
P. Mtagulwa, and P. Kaelo, An efficient modified PRP-FR hybrid conjugate gradient method for solving unconstrained optimization problems, Applied Numerical Mathematics, vol. 145, pp. 111–120, 2019.
D. A. Oladepo, O. J. Adeleke, and C. T. Ako, A mixed hybrid conjugate gradient method for unconstrained engineering optimization problems, In Computer Science On-line Conference 2018, pp. 423–431. Springer, Cham.
E. Polak, and G. Ribier´e, Note sur la convergence de directions conjuges, Rev. Francaise Informat Recherche Operationelle, 3e Anne, vol. 16, pp. 35–43, 1969.
B. T. Polyak, The conjugate gradient method in extreme problems, USSR Comp. Math. Math. Phys., vol. 9, pp. 94–112, 1969.
Y. Shengwei, Z.Wei, and H. Huang, A note about WYLs conjugate gradient method and its applications, Appl. Math. Comput., vol. 191, no. 2, pp. 381–388, 2007.
Z. Wei, S. Yao, and L. Liu, The convergence properties of some new conjugate gradient methods, Appl. Math. Comput., vol. 183, pp. 1341–1350, 2006.
Z. X. Wei, H. D. Huang and Y. R. Tao, A modified hestenes stiefel conjugate gradient method and its convergence, J. Math. Res. Exposition, vol. 30, no. 2, pp. 297–308, 2010.
W. Xue, and W. Zhang, Learning unknown structure in CRFs via adaptive gradient projection method, Statistics, Optimization & Information Computing, vol. 4, no. 3, pp. 243–251, 2016.
D. Xuewu, and X. Chengxian, Global convergence properties of two classes of optimal algorithms constrained by the FR conjugate gradient method, Numer. Math. J. of Chinese Univ., vol. 22, no. 4, pp. 311–318, 2000, (in Chinese).
L. Zhang, An improved WeiYaoLiu nonlinear conjugate gradient method for optimization computation Applied Mathematics and computation, vol. 215, no. 6, pp. 2269–2274, 2009.
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
