Predictive Coding beyond Correlations

Tommaso Salvatori, Luca Pinchetti, Amine M’Charrak, Beren Millidge, Thomas Lukasiewicz
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:43142-43179, 2024.

Abstract

Biologically plausible learning algorithms offer a promising alternative to traditional deep learning techniques, especially in overcoming the limitations of backpropagation in fast and low-energy neuromorphic implementations. To this end, there has been extensive research in understanding what their capabilities are. In this work, we show how one of such algorithms, called predictive coding, is able to perform causal inference tasks. First, we show how a simple change in the inference process of predictive coding enables to compute interventions without the need to mutilate or redefine a causal graph. Then, we explore applications in cases where the graph is unknown, and has to be inferred from observational data. Empirically, we show how such findings can be used to improve the performance of predictive coding in image classification tasks, and conclude that such models are naturally able to perform causal inference tasks using a biologically plausible kind of message passing.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-salvatori24a, title = {Predictive Coding beyond Correlations}, author = {Salvatori, Tommaso and Pinchetti, Luca and M'Charrak, Amine and Millidge, Beren and Lukasiewicz, Thomas}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {43142--43179}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/salvatori24a/salvatori24a.pdf}, url = {https://proceedings.mlr.press/v235/salvatori24a.html}, abstract = {Biologically plausible learning algorithms offer a promising alternative to traditional deep learning techniques, especially in overcoming the limitations of backpropagation in fast and low-energy neuromorphic implementations. To this end, there has been extensive research in understanding what their capabilities are. In this work, we show how one of such algorithms, called predictive coding, is able to perform causal inference tasks. First, we show how a simple change in the inference process of predictive coding enables to compute interventions without the need to mutilate or redefine a causal graph. Then, we explore applications in cases where the graph is unknown, and has to be inferred from observational data. Empirically, we show how such findings can be used to improve the performance of predictive coding in image classification tasks, and conclude that such models are naturally able to perform causal inference tasks using a biologically plausible kind of message passing.} }
Endnote
%0 Conference Paper %T Predictive Coding beyond Correlations %A Tommaso Salvatori %A Luca Pinchetti %A Amine M’Charrak %A Beren Millidge %A Thomas Lukasiewicz %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-salvatori24a %I PMLR %P 43142--43179 %U https://proceedings.mlr.press/v235/salvatori24a.html %V 235 %X Biologically plausible learning algorithms offer a promising alternative to traditional deep learning techniques, especially in overcoming the limitations of backpropagation in fast and low-energy neuromorphic implementations. To this end, there has been extensive research in understanding what their capabilities are. In this work, we show how one of such algorithms, called predictive coding, is able to perform causal inference tasks. First, we show how a simple change in the inference process of predictive coding enables to compute interventions without the need to mutilate or redefine a causal graph. Then, we explore applications in cases where the graph is unknown, and has to be inferred from observational data. Empirically, we show how such findings can be used to improve the performance of predictive coding in image classification tasks, and conclude that such models are naturally able to perform causal inference tasks using a biologically plausible kind of message passing.
APA
Salvatori, T., Pinchetti, L., M’Charrak, A., Millidge, B. & Lukasiewicz, T.. (2024). Predictive Coding beyond Correlations. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:43142-43179 Available from https://proceedings.mlr.press/v235/salvatori24a.html.

Related Material