UNIGE document Scientific Article
previous document  unige:151152  next document
add to browser collection
Title

Explicit stabilised gradient descent for faster strongly convex optimisation

Authors
Eftekhari, Armin
Zygalakis, Konstantinos C.
Published in BIT Numerical Mathematics. 2021, vol. 61, no. 1, p. 119-139
Abstract We introduce the explicit stabilised gradient descent method (ESGD) for strongly convex optimisation problems. This new algorithm is based on explicit stabilised integrators for stiff differential equations, a powerful class of numerical schemes to avoid the severe step size restriction faced by standard explicit integrators. For optimising quadratic and strongly convex functions, we prove that ESGD nearly achieves the optimal convergence rate of the conjugate gradient algorithm, and the suboptimality of ESGD diminishes as the condition number of the quadratic function worsens. We show that this optimal rate is obtained also for a partitioned variant of ESGD applied to perturbations of quadratic functions. In addition, numerical experiments on general strongly convex problems show that ESGD outperforms Nesterov's accelerated gradient descent.
Identifiers
Full text
Structures
Research group Analyse numérique
Citation
(ISO format)
EFTEKHARI, Armin et al. Explicit stabilised gradient descent for faster strongly convex optimisation. In: BIT Numerical Mathematics, 2021, vol. 61, n° 1, p. 119-139. doi: 10.1007/s10543-020-00819-y https://archive-ouverte.unige.ch/unige:151152

42 hits

8 downloads

Update

Deposited on : 2021-04-21

Export document
Format :
Citation style :