Wei-Yao-Liu Conjugate Gradient Algorithm for Nonsmooth Convex Optimization Problems
Abstract
This paper presents a Wei-Yao-Liu conjugate gradient algorithm for nonsmooth convex optimization problem. The proposed algorithm makes use of approximate function and gradient values of the Moreau-Yosida regularization function instead of the corresponding exact values. Under suitable conditions, the global convergence property could be established for the proposed conjugate gradient method. Finally, some numerical results are reported to show the efficiency of our algorithm.References
J.R. Birge, L. Qi and Z. Wei, Convergence analysis of some methods for minimizing a nonsmooth convex function, Journal of Optimization Theory and Applications, vol. 97, pp. 357-383, 1998.
J.R. Birge, L. Qi and Z. Wei, A general approach to convergence properties of some methods for nonsmooth convex optimization, Applied Mathematics and Optimization, vol. 38, pp. 141-158, 1998.
J.F. Bonnans, J.C. Gilbert, C. Lemar´echal and C.A. Sagastiz´abal, A family of veriable metric proximal methods, Mathematical Programming, vol. 68, pp. 15-47, 1995.
F.H. Clark, Optimization and Nonsmooth Analysis, New York, Wiley, 1983.
A.R. Conn, N.I.M. Gould and P.L. Toint, Trust-Region Methods, SIAM, Philadelphia, 2000.
R. Correa and C. Lemar´echal, Convergence of some algorithms for convex minimization, Mathematical Programming, vol. 62, pp. 261-273, 1993.
M. Fukushima, A descent algorithm for nonsmooth convex optimization, Mathematical Programming, vol, 30, pp. 163-175, 1984.
M. Fukushima and L. Qi, A global and superlinearly convergent algorithm for nonsmooth convex minimization, SIAM Journal on Optimization, vol. 6, pp. 1106-1120, 1996.
J.C. Gilbert and J. Nocedal, Global convergence properties of conjugate gradient methods for optimization, SIAM Journal of Optimization, vol. 2, pp. 21-42, 1992.
J.B. Hiriart-Urruty and C. Lemmar´echal, Convex Analysis and Minimization Algorithms, Berlin, Springer, 1993.
H. Huang, Z. Wei and S. Yao, The proof of the sufficient descent condition of the Wei-Yao-Liu congugate gradient method under the strong Wolfe-Powell line search, Applied Mathematics and Computation, vol. 189, pp. 1241-1245, 2007.
Y. Hu and Z. Wei, Wei-Yao-Liu conjugate gradient projection algorithm for nonlinear monotone equations with convex constraints, International Journal of Computer Mathematics, vol. 92, pp. 2261-2272, 2015.
S. Lu, Z. Wei and L. Li, A trust region algorithm with adaptive cubic regularization methods for nonsmooth convex minimization, Computational Optimization and Applications, vol. 51, pp. 551-573, 2012.
L. Lukˇsan and J. Vlˇcek, Test Problems for Nonsmooth Unconstrained and Linearly Constrained Optimization, Technical Report No.798, Institute of Computer Science, Academy of Sciences of the Czech Republic, 2000.
L. Lukˇsan and J. Vlˇcek, A bundle-Newton method for nonsmooth unconstrained minimization, Mathematical Programming, vol. 83, pp. 373-391, 1998.
Q. Li, Conjugate gradient type methods for the nondifferentiable convex minimization, Optimization Letters, vol. 7, pp. 533-545, 2013.
Q. Li, A Modified Fletcher-Reeves-Type Method for Nonsmooth Convex Minimization, Statistics, Optimization & Information Computing, vol. 2, no. 3, pp. 200-210, 2014.
S. Lu, Z.Wei and L. Mo, Some global convergence properties of the Wei-Yao-Liu conjugate gradient method with inexact line search, Applied Mathematics and Optimization, vol. 217, pp. 7132-7137, 2011.
N. H. Monjezi, S. Nobakhtian, A new infeasible proximal bundle algorithm for nonsmooth nonconvex constrained optimization, Computational Optimization and Applications, vol. 74, pp. 443-480, 2019.
L. Qi, Convergence analysis of some algorithms for solving nonsmooth equations, Mathematics of Operations Research, vol. 18, pp. 227-244, 1993.
L. Qi and J. Sun, A nonsmooth version of Newton´s method, Mathematical Programming, vol. 58, pp. 353-367, 1993.
A.I. Rauf and M. Fukushima, Globally convergent BFGS method for nonsmooth convex optimization, Journal of Optimization Theory and Applications, vol. 104, pp. 539-558, 2000.
Z. Wei and L. Qi, Convergence analysis of a proximal Newton method, Numerical Functional Analysis and Optimization, vol. 17, pp. 463-472, 1996.
Z. Wei, L. Qi and J.R. Birge, A new methods for nonsmooth convex optimization, Journal of Inequalities and Applications, vol. 2, pp. 157-179, 1998.
Z. Wei, S. Yao and L. Liu, The convergence properties of some new conjugate gradient methods, Applied Mathematics and Computation, vol. 183, pp. 1341-1350, 2006.
S. Yao, Z Wei and H. Huang, A note about WYLs conjugate gradient method and its applications, Applied Mathematics and Computation, vol. 191, pp. 381-388, 2007.
Z. Sheng and G. Yuan, An effective adaptive trust region algorithm for nonsmooth minimization, Computational Optimization and Applications, vol. 71, pp. 251-271, 2018.
G. Yuan, Z. Meng and Y. Li, A modified Hestenes and Stiefel conjugate gradient algorithm for large-scale nonsmooth minimizations and nonlinear equations, Journal of Optimization Theory and Applications, vol. 168, pp. 129-152, 2016.
G. Yuan, Z.Wei and G. Li, A modified Polak-Ribi`ere-Polyak conjugate gradient algorithm for nonsmooth convex programs, Journal of Computational and Applied Mathematics, vol. 255, pp. 86-96, 2014.
G. Yuan, Z. Wei and Z. Wang, Gradient trust region algorithm with limited memory BFGS update for nonsmooth convex minimization, Computational Optimization and Applications, vol. 54, pp. 45-64, 2013.
G. Yuan and M. Zhang, A modified Hestens-Stiefel conjugate gradient algorithm for large-scale optimization, Numerical Functional Analysis and Optimization, vol. 34, pp. 914-937, 2013.
G. Yuan and Z.Wei, The Barzilai and Borwein Gradient Method with Nonmonotone Line Search for Nonsmooth Convex Optimization Problems, Mathematical Modelling and Analysis, vol. 17, pp. 203-216, 2012.
L. Zhang, A new trust region algorithm for nonsmooth convex minimization, Applied Mathematics and Computation, vol. 193, pp. 135-142, 2007.
H. Zhang and W. Hager, A nonmonotone line search technique and its application to unconstrained optimization, SIAM Journal on Optimization, vol. 14, pp. 1043-1056, 2004.
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
