Principled Acceleration of Iterative Numerical Methods Using Machine Learning

Sohei Arisaka, Qianxiao Li
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:1041-1059, 2023.

Abstract

Iterative methods are ubiquitous in large-scale scientific computing applications, and a number of approaches based on meta-learning have been recently proposed to accelerate them. However, a systematic study of these approaches and how they differ from meta-learning is lacking. In this paper, we propose a framework to analyze such learning-based acceleration approaches, where one can immediately identify a departure from classical meta-learning. We theoretically show that this departure may lead to arbitrary deterioration of model performance, and at the same time, we identify a methodology to ameliorate it by modifying the loss objective, leading to a novel training method for learning-based acceleration of iterative algorithms. We demonstrate the significant advantage and versatility of the proposed approach through various numerical applications.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-arisaka23a, title = {Principled Acceleration of Iterative Numerical Methods Using Machine Learning}, author = {Arisaka, Sohei and Li, Qianxiao}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {1041--1059}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/arisaka23a/arisaka23a.pdf}, url = {https://proceedings.mlr.press/v202/arisaka23a.html}, abstract = {Iterative methods are ubiquitous in large-scale scientific computing applications, and a number of approaches based on meta-learning have been recently proposed to accelerate them. However, a systematic study of these approaches and how they differ from meta-learning is lacking. In this paper, we propose a framework to analyze such learning-based acceleration approaches, where one can immediately identify a departure from classical meta-learning. We theoretically show that this departure may lead to arbitrary deterioration of model performance, and at the same time, we identify a methodology to ameliorate it by modifying the loss objective, leading to a novel training method for learning-based acceleration of iterative algorithms. We demonstrate the significant advantage and versatility of the proposed approach through various numerical applications.} }
Endnote
%0 Conference Paper %T Principled Acceleration of Iterative Numerical Methods Using Machine Learning %A Sohei Arisaka %A Qianxiao Li %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-arisaka23a %I PMLR %P 1041--1059 %U https://proceedings.mlr.press/v202/arisaka23a.html %V 202 %X Iterative methods are ubiquitous in large-scale scientific computing applications, and a number of approaches based on meta-learning have been recently proposed to accelerate them. However, a systematic study of these approaches and how they differ from meta-learning is lacking. In this paper, we propose a framework to analyze such learning-based acceleration approaches, where one can immediately identify a departure from classical meta-learning. We theoretically show that this departure may lead to arbitrary deterioration of model performance, and at the same time, we identify a methodology to ameliorate it by modifying the loss objective, leading to a novel training method for learning-based acceleration of iterative algorithms. We demonstrate the significant advantage and versatility of the proposed approach through various numerical applications.
APA
Arisaka, S. & Li, Q.. (2023). Principled Acceleration of Iterative Numerical Methods Using Machine Learning. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:1041-1059 Available from https://proceedings.mlr.press/v202/arisaka23a.html.

Related Material