Learning Partially Known Stochastic Dynamics with Empirical PAC Bayes

Manuel Haußmann, Sebastian Gerwinn, Andreas Look, Barbara Rakitsch, Melih Kandemir
Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, PMLR 130:478-486, 2021.

Abstract

Neural Stochastic Differential Equations model a dynamical environment with neural nets assigned to their drift and diffusion terms. The high expressive power of their nonlinearity comes at the expense of instability in the identification of the large set of free parameters. This paper presents a recipe to improve the prediction accuracy of such models in three steps: i) accounting for epistemic uncertainty by assuming probabilistic weights, ii) incorporation of partial knowledge on the state dynamics, and iii) training the resultant hybrid model by an objective derived from a PAC-Bayesian generalization bound. We observe in our experiments that this recipe effectively translates partial and noisy prior knowledge into an improved model fit.

Cite this Paper

BibTeX
@InProceedings{pmlr-v130-haussmann21a, title = { Learning Partially Known Stochastic Dynamics with Empirical PAC Bayes }, author = {Hau{\ss}mann, Manuel and Gerwinn, Sebastian and Look, Andreas and Rakitsch, Barbara and Kandemir, Melih}, booktitle = {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics}, pages = {478--486}, year = {2021}, editor = {Banerjee, Arindam and Fukumizu, Kenji}, volume = {130}, series = {Proceedings of Machine Learning Research}, month = {13--15 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v130/haussmann21a/haussmann21a.pdf}, url = {https://proceedings.mlr.press/v130/haussmann21a.html}, abstract = { Neural Stochastic Differential Equations model a dynamical environment with neural nets assigned to their drift and diffusion terms. The high expressive power of their nonlinearity comes at the expense of instability in the identification of the large set of free parameters. This paper presents a recipe to improve the prediction accuracy of such models in three steps: i) accounting for epistemic uncertainty by assuming probabilistic weights, ii) incorporation of partial knowledge on the state dynamics, and iii) training the resultant hybrid model by an objective derived from a PAC-Bayesian generalization bound. We observe in our experiments that this recipe effectively translates partial and noisy prior knowledge into an improved model fit. } }
Endnote
%0 Conference Paper %T Learning Partially Known Stochastic Dynamics with Empirical PAC Bayes %A Manuel Haußmann %A Sebastian Gerwinn %A Andreas Look %A Barbara Rakitsch %A Melih Kandemir %B Proceedings of The 24th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2021 %E Arindam Banerjee %E Kenji Fukumizu %F pmlr-v130-haussmann21a %I PMLR %P 478--486 %U https://proceedings.mlr.press/v130/haussmann21a.html %V 130 %X Neural Stochastic Differential Equations model a dynamical environment with neural nets assigned to their drift and diffusion terms. The high expressive power of their nonlinearity comes at the expense of instability in the identification of the large set of free parameters. This paper presents a recipe to improve the prediction accuracy of such models in three steps: i) accounting for epistemic uncertainty by assuming probabilistic weights, ii) incorporation of partial knowledge on the state dynamics, and iii) training the resultant hybrid model by an objective derived from a PAC-Bayesian generalization bound. We observe in our experiments that this recipe effectively translates partial and noisy prior knowledge into an improved model fit.
APA
Haußmann, M., Gerwinn, S., Look, A., Rakitsch, B. & Kandemir, M.. (2021). Learning Partially Known Stochastic Dynamics with Empirical PAC Bayes . Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 130:478-486 Available from https://proceedings.mlr.press/v130/haussmann21a.html.

Related Material