Generalization of Quasi-Newton Methods: Application to Robust Symmetric Multisecant Updates

Damien Scieur, Lewis Liu, Thomas Pumir, Nicolas Boumal
Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, PMLR 130:550-558, 2021.

Abstract

Quasi-Newton (qN) techniques approximate the Newton step by estimating the Hessian using the so-called secant equations. Some of these methods compute the Hessian using several secant equations but produce non-symmetric updates. Other quasi-Newton schemes, such as BFGS, enforce symmetry but cannot satisfy more than one secant equation. We propose a new type of quasi-Newton symmetric update using several secant equations in a least-squares sense. Our approach generalizes and unifies the design of quasi-Newton updates and satisfies provable robustness guarantees.

Cite this Paper

BibTeX
@InProceedings{pmlr-v130-scieur21a, title = { Generalization of Quasi-Newton Methods: Application to Robust Symmetric Multisecant Updates }, author = {Scieur, Damien and Liu, Lewis and Pumir, Thomas and Boumal, Nicolas}, booktitle = {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics}, pages = {550--558}, year = {2021}, editor = {Banerjee, Arindam and Fukumizu, Kenji}, volume = {130}, series = {Proceedings of Machine Learning Research}, month = {13--15 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v130/scieur21a/scieur21a.pdf}, url = {https://proceedings.mlr.press/v130/scieur21a.html}, abstract = { Quasi-Newton (qN) techniques approximate the Newton step by estimating the Hessian using the so-called secant equations. Some of these methods compute the Hessian using several secant equations but produce non-symmetric updates. Other quasi-Newton schemes, such as BFGS, enforce symmetry but cannot satisfy more than one secant equation. We propose a new type of quasi-Newton symmetric update using several secant equations in a least-squares sense. Our approach generalizes and unifies the design of quasi-Newton updates and satisfies provable robustness guarantees. } }
Endnote
%0 Conference Paper %T Generalization of Quasi-Newton Methods: Application to Robust Symmetric Multisecant Updates %A Damien Scieur %A Lewis Liu %A Thomas Pumir %A Nicolas Boumal %B Proceedings of The 24th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2021 %E Arindam Banerjee %E Kenji Fukumizu %F pmlr-v130-scieur21a %I PMLR %P 550--558 %U https://proceedings.mlr.press/v130/scieur21a.html %V 130 %X Quasi-Newton (qN) techniques approximate the Newton step by estimating the Hessian using the so-called secant equations. Some of these methods compute the Hessian using several secant equations but produce non-symmetric updates. Other quasi-Newton schemes, such as BFGS, enforce symmetry but cannot satisfy more than one secant equation. We propose a new type of quasi-Newton symmetric update using several secant equations in a least-squares sense. Our approach generalizes and unifies the design of quasi-Newton updates and satisfies provable robustness guarantees.
APA
Scieur, D., Liu, L., Pumir, T. & Boumal, N.. (2021). Generalization of Quasi-Newton Methods: Application to Robust Symmetric Multisecant Updates . Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 130:550-558 Available from https://proceedings.mlr.press/v130/scieur21a.html.

Related Material