A Deep Conjugate Direction Method for Iteratively Solving Linear Systems

Ayano Kaneda, Osman Akar, Jingyu Chen, Victoria Alicia Trevino Kala, David Hyde, Joseph Teran
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:15720-15736, 2023.

Abstract

We present a novel deep learning approach to approximate the solution of large, sparse, symmetric, positive-definite linear systems of equations. Motivated by the conjugate gradients algorithm that iteratively selects search directions for minimizing the matrix norm of the approximation error, we design an approach that utilizes a deep neural network to accelerate convergence via data-driven improvement of the search direction at each iteration. Our method leverages a carefully chosen convolutional network to approximate the action of the inverse of the linear operator up to an arbitrary constant. We demonstrate the efficacy of our approach on spatially discretized Poisson equations, which arise in computational fluid dynamics applications, with millions of degrees of freedom. Unlike state-of-the-art learning approaches, our algorithm is capable of reducing the linear system residual to a given tolerance in a small number of iterations, independent of the problem size. Moreover, our method generalizes effectively to various systems beyond those encountered during training.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-kaneda23a, title = {A Deep Conjugate Direction Method for Iteratively Solving Linear Systems}, author = {Kaneda, Ayano and Akar, Osman and Chen, Jingyu and Kala, Victoria Alicia Trevino and Hyde, David and Teran, Joseph}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {15720--15736}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/kaneda23a/kaneda23a.pdf}, url = {https://proceedings.mlr.press/v202/kaneda23a.html}, abstract = {We present a novel deep learning approach to approximate the solution of large, sparse, symmetric, positive-definite linear systems of equations. Motivated by the conjugate gradients algorithm that iteratively selects search directions for minimizing the matrix norm of the approximation error, we design an approach that utilizes a deep neural network to accelerate convergence via data-driven improvement of the search direction at each iteration. Our method leverages a carefully chosen convolutional network to approximate the action of the inverse of the linear operator up to an arbitrary constant. We demonstrate the efficacy of our approach on spatially discretized Poisson equations, which arise in computational fluid dynamics applications, with millions of degrees of freedom. Unlike state-of-the-art learning approaches, our algorithm is capable of reducing the linear system residual to a given tolerance in a small number of iterations, independent of the problem size. Moreover, our method generalizes effectively to various systems beyond those encountered during training.} }
Endnote
%0 Conference Paper %T A Deep Conjugate Direction Method for Iteratively Solving Linear Systems %A Ayano Kaneda %A Osman Akar %A Jingyu Chen %A Victoria Alicia Trevino Kala %A David Hyde %A Joseph Teran %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-kaneda23a %I PMLR %P 15720--15736 %U https://proceedings.mlr.press/v202/kaneda23a.html %V 202 %X We present a novel deep learning approach to approximate the solution of large, sparse, symmetric, positive-definite linear systems of equations. Motivated by the conjugate gradients algorithm that iteratively selects search directions for minimizing the matrix norm of the approximation error, we design an approach that utilizes a deep neural network to accelerate convergence via data-driven improvement of the search direction at each iteration. Our method leverages a carefully chosen convolutional network to approximate the action of the inverse of the linear operator up to an arbitrary constant. We demonstrate the efficacy of our approach on spatially discretized Poisson equations, which arise in computational fluid dynamics applications, with millions of degrees of freedom. Unlike state-of-the-art learning approaches, our algorithm is capable of reducing the linear system residual to a given tolerance in a small number of iterations, independent of the problem size. Moreover, our method generalizes effectively to various systems beyond those encountered during training.
APA
Kaneda, A., Akar, O., Chen, J., Kala, V.A.T., Hyde, D. & Teran, J.. (2023). A Deep Conjugate Direction Method for Iteratively Solving Linear Systems. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:15720-15736 Available from https://proceedings.mlr.press/v202/kaneda23a.html.

Related Material