Neural operators meet conjugate gradients: The FCG-NO method for efficient PDE solving

Alexander Rudikov, Vladimir Fanaskov, Ekaterina Muravleva, Yuri M. Laevsky, Ivan Oseledets
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:42766-42782, 2024.

Abstract

Deep learning solvers for partial differential equations typically have limited accuracy. We propose to overcome this problem by using them as preconditioners. More specifically, we apply discretization-invariant neural operators to learn preconditioners for the flexible conjugate gradient method (FCG). Architecture paired with novel loss function and training scheme allows for learning efficient preconditioners that can be used across different resolutions. On the theoretical side, FCG theory allows us to safely use nonlinear preconditioners that can be applied in $O(N)$ operations without constraining the form of the preconditioners matrix. To justify learning scheme components (the loss function and the way training data is collected) we perform several ablation studies. Numerical results indicate that our approach favorably compares with classical preconditioners and allows to reuse of preconditioners learned for lower resolution to the higher resolution data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-rudikov24a, title = {Neural operators meet conjugate gradients: The {FCG}-{NO} method for efficient {PDE} solving}, author = {Rudikov, Alexander and Fanaskov, Vladimir and Muravleva, Ekaterina and Laevsky, Yuri M. and Oseledets, Ivan}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {42766--42782}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/rudikov24a/rudikov24a.pdf}, url = {https://proceedings.mlr.press/v235/rudikov24a.html}, abstract = {Deep learning solvers for partial differential equations typically have limited accuracy. We propose to overcome this problem by using them as preconditioners. More specifically, we apply discretization-invariant neural operators to learn preconditioners for the flexible conjugate gradient method (FCG). Architecture paired with novel loss function and training scheme allows for learning efficient preconditioners that can be used across different resolutions. On the theoretical side, FCG theory allows us to safely use nonlinear preconditioners that can be applied in $O(N)$ operations without constraining the form of the preconditioners matrix. To justify learning scheme components (the loss function and the way training data is collected) we perform several ablation studies. Numerical results indicate that our approach favorably compares with classical preconditioners and allows to reuse of preconditioners learned for lower resolution to the higher resolution data.} }
Endnote
%0 Conference Paper %T Neural operators meet conjugate gradients: The FCG-NO method for efficient PDE solving %A Alexander Rudikov %A Vladimir Fanaskov %A Ekaterina Muravleva %A Yuri M. Laevsky %A Ivan Oseledets %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-rudikov24a %I PMLR %P 42766--42782 %U https://proceedings.mlr.press/v235/rudikov24a.html %V 235 %X Deep learning solvers for partial differential equations typically have limited accuracy. We propose to overcome this problem by using them as preconditioners. More specifically, we apply discretization-invariant neural operators to learn preconditioners for the flexible conjugate gradient method (FCG). Architecture paired with novel loss function and training scheme allows for learning efficient preconditioners that can be used across different resolutions. On the theoretical side, FCG theory allows us to safely use nonlinear preconditioners that can be applied in $O(N)$ operations without constraining the form of the preconditioners matrix. To justify learning scheme components (the loss function and the way training data is collected) we perform several ablation studies. Numerical results indicate that our approach favorably compares with classical preconditioners and allows to reuse of preconditioners learned for lower resolution to the higher resolution data.
APA
Rudikov, A., Fanaskov, V., Muravleva, E., Laevsky, Y.M. & Oseledets, I.. (2024). Neural operators meet conjugate gradients: The FCG-NO method for efficient PDE solving. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:42766-42782 Available from https://proceedings.mlr.press/v235/rudikov24a.html.

Related Material