A dai-liao hybrid hestenes-stiefel and fletcher-revees methods for unconstrained optimization

Authors

  • Nasiru Salihu Department of Mathematics, School of Physical Sciences, Modibbo Adama University of Technology, Yola, Nigeria.
  • Mathew Remilekun Odekunle Department of Mathematics, School of Physical Science, Moddibo Adama University of Technology, Yola, Nigeria
  • Also Mohammed Saleh Department of Mathematics, School of Physical Science, Moddibo Adama University of Technology, Yola, Nigeria
  • Suraj Salihu Department of Computer Science, Faculty of Science,Gombe State University, Gombe, Nigeria

DOI:

https://doi.org/10.12928/ijio.v2i1.3054

Keywords:

Unconstrained optimization, Descent property, Conjugacy condition, Global convergence, Conjugate gradient algorithm

Abstract

Some problems have no analytical solution or too difficult to solve by scientists, engineers, and mathematicians, so the development of numerical methods to obtain approximate solutions became necessary. Gradient methods are more efficient when the function to be minimized continuously in its first derivative. Therefore, this article presents a new hybrid Conjugate Gradient (CG) method to solve unconstrained optimization problems. The method requires the first-order derivatives but overcomes the steepest descent method’s shortcoming of slow convergence and needs not to save or compute the second-order derivatives needed by the Newton method. The CG update parameter is suggested from the Dai-Liao conjugacy condition as a convex combination of Hestenes-Stiefel and Fletcher-Revees algorithms by employing an optimal modulating choice parameterto avoid matrix storage. Numerical computation adopts an inexact line search to obtain the step-size that generates a decent property, showing that the algorithm is robust and efficient. The scheme converges globally under Wolfe line search, and it’s like is suitable in compressive sensing problems and M-tensor systems.

References

Abubakar, A. B., Kumam, P., Mohammad, H., Awwal., A. M. & Sitthithakerngkiet, K.(2019). A modified fletcher–reeves conjugate gradient method for monotone nonlinear equations with some applications, MPDI, mathematics, 7, 745 doi:10.3390/math7080745.

Al-Baali, M. (1985). Descent property and global convergence of the Fletcher Reeves method with inexact line search. IMA Journal of Numerical Analysis, 5, 121-124.

Al-Namat, F.N. and Al-Naemi, G.M. (2020). Global convergence property with inexact line search for a new hybrid conjugate gradient method. Open Access Library Journal, 7: e6048. https://doi.org/10.4236/oalib.1106048

Andrei, N. (2008). An unconstrained optimization test function. Advanced Modeling and Optimization. An Electronic International Journal,10, 172-182.

Andrei, N. (2017). A Dai-Liao nonlinear conjugate gradient algorithm. Numer. Algor., DOI 10.1007/s11075-017-0362-5.

Babaie-Kafaki, S. (2011). A modified BFGS algorithm based on a hybrid secant equation. Science China Mathematics,58, 315-331.

Babaie-Kafaki, S. (2013). A hybrid conjugate gradient method based on quadratic relaxation of Dai-Yuan hybrid conjugate gradient parameter. Journal of Mathematical Programming and Operation Research, 62(7), 929-941.

Babaie-Kafaki, S. & Mahdavi-Amiri, N. (2013). Two hybrid conjugate gradient methods based on hybrid secant equation. Mathematical Modeling and Analysis,18(1):32-52.

Babaie-Kafaki, S. & Ghanbari, R. (2014c). Two hybrid nonlinear conjugate gradient methods based on a modified secant equation. Journal of Mathematical Programming and Operation Research,63(7), 1027-1042.

Babaie-Kafaki, S., & Ghanbari, R. (2014a). The Dai-Liao nonlinear conjugate gradient method with optimal parameter choices. Eur. J. Oper. Res. 234(3), 625-630.

Babaie-Kafaki, S., & Ghanbari, R. (2014b). A descent family of Dai-Liao conjugate gradient methods. Optim. Methods Softw., 29(3): 583-591.

Babaie-Kafaki, S., & Ghanbari, R. (2015). Two optimal Dai-Liao conjugate gradient methods. Optimization, 64(1): 2277-2287.

Babaie-Kafaki, S. (2015). On optimality of two adaptive choices for the parameter of Dai-Liao method. Optim. Lett., DOI 10.1007/s11590-015-0965-5.

Babaie-Kafaki, S., & Ghanbari, R. (2017). Two adaptive Dai-Liao nonlinear conjugate gradient methods. Iran J. Sci. Technol. Trans. Sci., DOI 10.1007/s40995-017- 027 1-4.

Dai, Y.H. & Yuan, Y. (1999). A nonlinear conjugate gradient method with a strong global convergence property. SIAM J. Optim.,10, 177-182.

Dai, Y.H & Liao, L.Z. (2001). New conjugacy conditions and related nonlinear conjugate gradient methods, Appl. Math. Optim., 43, 87-101.

Dai, Y.H. & Yuan, Y. (2001). An efficient hybrid conjugates gradient method for unconstrained optimization. Annals of Operations Research, 103, 33-47.

Djordjevic, S.S. (2016). New hybrid conjugate gradient method as a convex combination of FR and PRP methods. Published by faculty of science and mathematics. University of Nis, Serbia, Filomat, 31, 3083-3100. https://doi.org/10.2298/FIL16083D

Djordjevic, S.S. (2017). New hybrid conjugate gradient methods as a convex combination of LS and CD methods. Published by faculty of science and mathematics. University of Nis, Serbia,31(6), 1813-1825.

Djordjevic, S.S. (2018). New hybrid conjugate gradient method as a convex combination of HS and FR conjugate gradient methods. Journal of Applied Mathematics and Computation, 2, 366-378. https://doi.org/10.26855/jamc.2018.09.002

Djordjevic, S.S. (2019). New hybrid conjugate gradient method as a convex combination of LS and FR conjugate gradient methods. Acta Mathematica Scientia, 39, 214-228.

Dolan, E.D. & More´, J.J. (2002). Benchmarking optimization software with performance profiles. Journal of Math. Program, 91(2), 201-213.

Esmaeili H., Rostami M., & Kimiaei M. (2018). Extended Dai–Yuan conjugate gradient strategy for large-scale unconstrained optimization with applications to compressive sensing. Published by faculty of sciences and mathematics, University of Nis, Serbia, Filomat, 32(6), 2173–2191. Available at: http://www.pmf.ni.ac.rs/filomat

Fletcher, R., & Reeves, C. (1964). Function minimization by conjugate gradients. Computational Journal,7, 149-154.

Fletcher, R. (1987). Practical methods of optimization, (vol. 1), Unconstrained optimization. New York: John Wiley & Sons.

Gould, N.I.M., Orban, D. & Toint, P.L. (2003). CUTEr: a constrained and unconstrained testing environment, revisited. ACM Transactions on Mathematical Software,29(4), 373-394.

Guo, J., & Wan, Z. (2019). A modified spectral PRP conjugate gradient projection method for solving large-scale monotone equations and its application in compressed sensing. Hindawi Mathematical Problems in Engineering. Volume 2019, Article ID 5261830, 17 pages. https://doi.org/10.1155/2019/5261830

Hager, W.W. & Zhang, H. (2006). A survey of nonlinear conjugate gradient methods. Pacific J. Optim., 2, 35-58.

Hestenes, M.R., & Stiefel, E.L. (1952). Methods of conjugate gradients for solving linear systems. Journal Res. Nat. Bur. Stand., 49, 409-436

Ibrahim, A.H., Kumam, P., Abubakar, A. B., Jirakitpuwapat, W., & Abubakar, J. (2020). A hybrid conjugate gradient algorithm for constrained monotone equations with application in compressive sensing. Heliyon.e03466. https://doi.org/10.1016/j.heliyon.2020.e03466

Li, D.H., & Fukushima, A. M. (2001). modified BFGS method and its global convergence in nonconvex minimization, J. Comput. Appl. Math. 129(1) 15-35.

Liu, K., & Du, S. (2019). Modified three-term conjugate gradient method and its applications. Hindawi Mathematical Problems in Engineering. Volume 2019, Article ID 5976595, 9 pages. https://doi.org/10.1155/2019/5976595

Liu, J., Du, S., & Chen, Y. (2020). A sufficient descent nonlinear conjugate gradient method for solving M-tensor equations. Journal of Computational and Applied Mathematics, 371112709

Liu, Y., & Storey, C. (1991). Efficient generalized conjugate gradient algorithms, part 1: Theory. Journal of optimization theory and applications. 69, 129-137.

Lotfi, M., & Hosseini, S.M. (2019). An efficient Dai–Liao type conjugate gradient method by reformulating the CG parameter in the search direction equation. Journal of Computational and Applied Mathematics, 371. 112708. https://doi.org/10.1016/j.cam.2019.112708

Mohammed, N. S., Mustafa M., Mohd R. & Shazlyn M. S. (2020). A new hybrid coefficient of conjugate gradient method. Indonesian Journal of Electrical Engineering and Computer Science, Vol. 18, No. 3, June 2020, pp. 1454-1463.

Perry, A. (1978). A modified conjugate gradient algorithm. Oper. Res. Tech. Notes,26(6): 1073-1078.

Polak, B.T. (1969). The conjugate gradient method in extreme problems. USSR Comput. Math. Math. Phys. 4, 94-112.
Polyak, B.T. (1967). A general method of solving extremal problems. Soviet Math. Doklady, 8, 14-29.

Powell, M.J.D. (1984). Nonconvex minimization calculations and the conjugate gradient method, Numerical Analysis (Dundee, 1983), D.F. Griffiths, ed., Lecture Notes in Mathematics, Vol. 1066, Springer, Berlin. 122-141.

Salihu, N., Odekunle, M., Waziri, M. & Halilu, A. (2020). A new hybrid conjugate gradient method based on secant equation for solving large scale unconstrained optimization problems. Iranian Journal of Optimization, 12(1): 33-44.

Waziri, M. Y., Ahmed, K., & Sabi’u, J. (2019). A Dai-Liao conjugate gradient method via modified secant equation for system of nonlinear equations. Arab. J. Math. https://doi.org/10.1007/s40065-019-0264-6.

Xue, W., Ren, J., Zheng, X., Liu, Z., & Liang, Y. (2018). A New DY conjugate gradient method and applications to image denoising. IEICE TRANS. INF. & SYST., VOL.E101–D, NO.1 2, December 2018. DOI: 10.1587/transinf.2018EDP7210

Yao, S., Feng, Q., Li, L., & Xu, J. (2019). A class of globally convergent three-term Dai-Liao conjugate gradient methods. Applied Numerical Mathematics, 151 354-366. https://doi.org/10.1016/j.apnum.2019.12.026.

Zoutendijk, G. (1970). Nonlinear programming computational methods, in integer and nonlinear programming. J. Abadie ed., North-Holland, Amsterdam 37-86.

Downloads

Published

2021-02-24

How to Cite

Salihu, N., Odekunle, M. R., Saleh, A. M., & Salihu, S. (2021). A dai-liao hybrid hestenes-stiefel and fletcher-revees methods for unconstrained optimization. International Journal of Industrial Optimization, 2(1), 33–50. https://doi.org/10.12928/ijio.v2i1.3054

Issue

Section

Articles