Learning to Solve Related Linear Systems

Disha Hegde, Jon Cockayne
Proceedings of the First International Conference on Probabilistic Numerics, PMLR 271:103-121, 2025.

Abstract

Solving multiple parametrised related systems is an essential component of many numerical tasks, and learning from the already solved systems will make this process faster. In this work, we propose a novel probabilistic linear solver over the parameter space. This leverages information from the solved linear systems in a regression setting to provide an efficient posterior mean and covariance. We advocate using this as companion regression model for the preconditioned conjugate gradient method, and discuss the favourable properties of the posterior mean and covariance as the initial guess and preconditioner. We also provide several design choices for this companion solver. Numerical experiments showcase the benefits of using our novel solver in a hyperparameter optimisation problem.

Cite this Paper

BibTeX
@InProceedings{pmlr-v271-hegde25a, title = {Learning to Solve Related Linear Systems}, author = {Hegde, Disha and Cockayne, Jon}, booktitle = {Proceedings of the First International Conference on Probabilistic Numerics}, pages = {103--121}, year = {2025}, editor = {Kanagawa, Motonobu and Cockayne, Jon and Gessner, Alexandra and Hennig, Philipp}, volume = {271}, series = {Proceedings of Machine Learning Research}, month = {01--03 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v271/main/assets/hegde25a/hegde25a.pdf}, url = {https://proceedings.mlr.press/v271/hegde25a.html}, abstract = {Solving multiple parametrised related systems is an essential component of many numerical tasks, and learning from the already solved systems will make this process faster. In this work, we propose a novel probabilistic linear solver over the parameter space. This leverages information from the solved linear systems in a regression setting to provide an efficient posterior mean and covariance. We advocate using this as companion regression model for the preconditioned conjugate gradient method, and discuss the favourable properties of the posterior mean and covariance as the initial guess and preconditioner. We also provide several design choices for this companion solver. Numerical experiments showcase the benefits of using our novel solver in a hyperparameter optimisation problem.} }
Endnote
%0 Conference Paper %T Learning to Solve Related Linear Systems %A Disha Hegde %A Jon Cockayne %B Proceedings of the First International Conference on Probabilistic Numerics %C Proceedings of Machine Learning Research %D 2025 %E Motonobu Kanagawa %E Jon Cockayne %E Alexandra Gessner %E Philipp Hennig %F pmlr-v271-hegde25a %I PMLR %P 103--121 %U https://proceedings.mlr.press/v271/hegde25a.html %V 271 %X Solving multiple parametrised related systems is an essential component of many numerical tasks, and learning from the already solved systems will make this process faster. In this work, we propose a novel probabilistic linear solver over the parameter space. This leverages information from the solved linear systems in a regression setting to provide an efficient posterior mean and covariance. We advocate using this as companion regression model for the preconditioned conjugate gradient method, and discuss the favourable properties of the posterior mean and covariance as the initial guess and preconditioner. We also provide several design choices for this companion solver. Numerical experiments showcase the benefits of using our novel solver in a hyperparameter optimisation problem.
APA
Hegde, D. & Cockayne, J.. (2025). Learning to Solve Related Linear Systems. Proceedings of the First International Conference on Probabilistic Numerics, in Proceedings of Machine Learning Research 271:103-121 Available from https://proceedings.mlr.press/v271/hegde25a.html.

Related Material