Schwarz–Schur Involution: Lightspeed Differentiable Sparse Linear Solvers

Yu Wang, Mazdak Abulnaga, Yaël Balbastre, Bruce Fischl
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:62172-62221, 2025.

Abstract

Sparse linear solvers or generalized deconvolution are fundamental to science and engineering, applied in partial differential equations (PDEs), scientific computing, computer vision, and beyond. Indirect solvers possess characteristics that make them undesirable as stable differentiable modules; existing direct solvers, though reliable, are too expensive to be adopted in neural architectures. We substantially accelerate direct sparse solvers by up to three orders of magnitude, violating common assumptions that direct solvers are too slow. We "condense" a sparse Laplacian matrix into a dense tensor, a compact data structure that batch-wise stores the Dirichlet-to-Neumann matrices, reducing the sparse solving to recursively merging pairs of dense matrices that are much smaller. The batched small dense systems are sliced and inverted in parallel to take advantage of dense GPU BLAS kernels, highly optimized in the era of deep learning. Our method is efficient, qualified as a strong zero-shot baseline for AI-based PDE solving, and a reliable differentiable module integrable into machine learning pipelines.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-wang25a, title = {Schwarz–Schur Involution: Lightspeed Differentiable Sparse Linear Solvers}, author = {Wang, Yu and Abulnaga, Mazdak and Balbastre, Ya\"{e}l and Fischl, Bruce}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {62172--62221}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/wang25a/wang25a.pdf}, url = {https://proceedings.mlr.press/v267/wang25a.html}, abstract = {Sparse linear solvers or generalized deconvolution are fundamental to science and engineering, applied in partial differential equations (PDEs), scientific computing, computer vision, and beyond. Indirect solvers possess characteristics that make them undesirable as stable differentiable modules; existing direct solvers, though reliable, are too expensive to be adopted in neural architectures. We substantially accelerate direct sparse solvers by up to three orders of magnitude, violating common assumptions that direct solvers are too slow. We "condense" a sparse Laplacian matrix into a dense tensor, a compact data structure that batch-wise stores the Dirichlet-to-Neumann matrices, reducing the sparse solving to recursively merging pairs of dense matrices that are much smaller. The batched small dense systems are sliced and inverted in parallel to take advantage of dense GPU BLAS kernels, highly optimized in the era of deep learning. Our method is efficient, qualified as a strong zero-shot baseline for AI-based PDE solving, and a reliable differentiable module integrable into machine learning pipelines.} }
Endnote
%0 Conference Paper %T Schwarz–Schur Involution: Lightspeed Differentiable Sparse Linear Solvers %A Yu Wang %A Mazdak Abulnaga %A Yaël Balbastre %A Bruce Fischl %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-wang25a %I PMLR %P 62172--62221 %U https://proceedings.mlr.press/v267/wang25a.html %V 267 %X Sparse linear solvers or generalized deconvolution are fundamental to science and engineering, applied in partial differential equations (PDEs), scientific computing, computer vision, and beyond. Indirect solvers possess characteristics that make them undesirable as stable differentiable modules; existing direct solvers, though reliable, are too expensive to be adopted in neural architectures. We substantially accelerate direct sparse solvers by up to three orders of magnitude, violating common assumptions that direct solvers are too slow. We "condense" a sparse Laplacian matrix into a dense tensor, a compact data structure that batch-wise stores the Dirichlet-to-Neumann matrices, reducing the sparse solving to recursively merging pairs of dense matrices that are much smaller. The batched small dense systems are sliced and inverted in parallel to take advantage of dense GPU BLAS kernels, highly optimized in the era of deep learning. Our method is efficient, qualified as a strong zero-shot baseline for AI-based PDE solving, and a reliable differentiable module integrable into machine learning pipelines.
APA
Wang, Y., Abulnaga, M., Balbastre, Y. & Fischl, B.. (2025). Schwarz–Schur Involution: Lightspeed Differentiable Sparse Linear Solvers. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:62172-62221 Available from https://proceedings.mlr.press/v267/wang25a.html.

Related Material