Is Learning Summary Statistics Necessary for Likelihood-free Inference?

Yanzhi Chen, Michael U. Gutmann, Adrian Weller
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:4529-4544, 2023.

Abstract

Likelihood-free inference (LFI) is a set of techniques for inference in implicit statistical models. A longstanding question in LFI has been how to design or learn good summary statistics of data, but this might now seem unnecessary due to the advent of recent end-to-end (i.e. neural network-based) LFI methods. In this work, we rethink this question with a new method for learning summary statistics. We show that learning sufficient statistics may be easier than direct posterior inference, as the former problem can be reduced to a set of low-dimensional, easy-to-solve learning problems. This suggests us to explicitly decouple summary statistics learning from posterior inference in LFI. Experiments on diverse inference tasks with different data types validate our hypothesis.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-chen23h, title = {Is Learning Summary Statistics Necessary for Likelihood-free Inference?}, author = {Chen, Yanzhi and Gutmann, Michael U. and Weller, Adrian}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {4529--4544}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/chen23h/chen23h.pdf}, url = {https://proceedings.mlr.press/v202/chen23h.html}, abstract = {Likelihood-free inference (LFI) is a set of techniques for inference in implicit statistical models. A longstanding question in LFI has been how to design or learn good summary statistics of data, but this might now seem unnecessary due to the advent of recent end-to-end (i.e. neural network-based) LFI methods. In this work, we rethink this question with a new method for learning summary statistics. We show that learning sufficient statistics may be easier than direct posterior inference, as the former problem can be reduced to a set of low-dimensional, easy-to-solve learning problems. This suggests us to explicitly decouple summary statistics learning from posterior inference in LFI. Experiments on diverse inference tasks with different data types validate our hypothesis.} }
Endnote
%0 Conference Paper %T Is Learning Summary Statistics Necessary for Likelihood-free Inference? %A Yanzhi Chen %A Michael U. Gutmann %A Adrian Weller %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-chen23h %I PMLR %P 4529--4544 %U https://proceedings.mlr.press/v202/chen23h.html %V 202 %X Likelihood-free inference (LFI) is a set of techniques for inference in implicit statistical models. A longstanding question in LFI has been how to design or learn good summary statistics of data, but this might now seem unnecessary due to the advent of recent end-to-end (i.e. neural network-based) LFI methods. In this work, we rethink this question with a new method for learning summary statistics. We show that learning sufficient statistics may be easier than direct posterior inference, as the former problem can be reduced to a set of low-dimensional, easy-to-solve learning problems. This suggests us to explicitly decouple summary statistics learning from posterior inference in LFI. Experiments on diverse inference tasks with different data types validate our hypothesis.
APA
Chen, Y., Gutmann, M.U. & Weller, A.. (2023). Is Learning Summary Statistics Necessary for Likelihood-free Inference?. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:4529-4544 Available from https://proceedings.mlr.press/v202/chen23h.html.

Related Material