Equivariant Learning of Stochastic Fields: Gaussian Processes and Steerable Conditional Neural Processes

Peter Holderrieth, Michael J Hutchinson, Yee Whye Teh
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:4297-4307, 2021.

Abstract

Motivated by objects such as electric fields or fluid streams, we study the problem of learning stochastic fields, i.e. stochastic processes whose samples are fields like those occurring in physics and engineering. Considering general transformations such as rotations and reflections, we show that spatial invariance of stochastic fields requires an inference model to be equivariant. Leveraging recent advances from the equivariance literature, we study equivariance in two classes of models. Firstly, we fully characterise equivariant Gaussian processes. Secondly, we introduce Steerable Conditional Neural Processes (SteerCNPs), a new, fully equivariant member of the Neural Process family. In experiments with Gaussian process vector fields, images, and real-world weather data, we observe that SteerCNPs significantly improve the performance of previous models and equivariance leads to improvements in transfer learning tasks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-holderrieth21a, title = {Equivariant Learning of Stochastic Fields: Gaussian Processes and Steerable Conditional Neural Processes}, author = {Holderrieth, Peter and Hutchinson, Michael J and Teh, Yee Whye}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {4297--4307}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/holderrieth21a/holderrieth21a.pdf}, url = {https://proceedings.mlr.press/v139/holderrieth21a.html}, abstract = {Motivated by objects such as electric fields or fluid streams, we study the problem of learning stochastic fields, i.e. stochastic processes whose samples are fields like those occurring in physics and engineering. Considering general transformations such as rotations and reflections, we show that spatial invariance of stochastic fields requires an inference model to be equivariant. Leveraging recent advances from the equivariance literature, we study equivariance in two classes of models. Firstly, we fully characterise equivariant Gaussian processes. Secondly, we introduce Steerable Conditional Neural Processes (SteerCNPs), a new, fully equivariant member of the Neural Process family. In experiments with Gaussian process vector fields, images, and real-world weather data, we observe that SteerCNPs significantly improve the performance of previous models and equivariance leads to improvements in transfer learning tasks.} }
Endnote
%0 Conference Paper %T Equivariant Learning of Stochastic Fields: Gaussian Processes and Steerable Conditional Neural Processes %A Peter Holderrieth %A Michael J Hutchinson %A Yee Whye Teh %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-holderrieth21a %I PMLR %P 4297--4307 %U https://proceedings.mlr.press/v139/holderrieth21a.html %V 139 %X Motivated by objects such as electric fields or fluid streams, we study the problem of learning stochastic fields, i.e. stochastic processes whose samples are fields like those occurring in physics and engineering. Considering general transformations such as rotations and reflections, we show that spatial invariance of stochastic fields requires an inference model to be equivariant. Leveraging recent advances from the equivariance literature, we study equivariance in two classes of models. Firstly, we fully characterise equivariant Gaussian processes. Secondly, we introduce Steerable Conditional Neural Processes (SteerCNPs), a new, fully equivariant member of the Neural Process family. In experiments with Gaussian process vector fields, images, and real-world weather data, we observe that SteerCNPs significantly improve the performance of previous models and equivariance leads to improvements in transfer learning tasks.
APA
Holderrieth, P., Hutchinson, M.J. & Teh, Y.W.. (2021). Equivariant Learning of Stochastic Fields: Gaussian Processes and Steerable Conditional Neural Processes. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:4297-4307 Available from https://proceedings.mlr.press/v139/holderrieth21a.html.

Related Material