On the Independence Assumption in Neurosymbolic Learning

Emile Van Krieken, Pasquale Minervini, Edoardo Ponti, Antonio Vergari
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:49078-49097, 2024.

Abstract

State-of-the-art neurosymbolic learning systems use probabilistic reasoning to guide neural networks towards predictions that conform to logical constraints. Many such systems assume that the probabilities of the considered symbols are conditionally independent given the input to simplify learning and reasoning. We study and criticise this assumption, highlighting how it can hinder optimisation and prevent uncertainty quantification. We prove that loss functions bias conditionally independent neural networks to become overconfident in their predictions. As a result, they are unable to represent uncertainty over multiple valid options. Furthermore, we prove that the minima of such loss functions are usually highly disconnected and non-convex, and thus difficult to optimise. Our theoretical analysis gives the foundation for replacing the conditional independence assumption and designing more expressive neurosymbolic probabilistic models.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-van-krieken24a, title = {On the Independence Assumption in Neurosymbolic Learning}, author = {Van Krieken, Emile and Minervini, Pasquale and Ponti, Edoardo and Vergari, Antonio}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {49078--49097}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/van-krieken24a/van-krieken24a.pdf}, url = {https://proceedings.mlr.press/v235/van-krieken24a.html}, abstract = {State-of-the-art neurosymbolic learning systems use probabilistic reasoning to guide neural networks towards predictions that conform to logical constraints. Many such systems assume that the probabilities of the considered symbols are conditionally independent given the input to simplify learning and reasoning. We study and criticise this assumption, highlighting how it can hinder optimisation and prevent uncertainty quantification. We prove that loss functions bias conditionally independent neural networks to become overconfident in their predictions. As a result, they are unable to represent uncertainty over multiple valid options. Furthermore, we prove that the minima of such loss functions are usually highly disconnected and non-convex, and thus difficult to optimise. Our theoretical analysis gives the foundation for replacing the conditional independence assumption and designing more expressive neurosymbolic probabilistic models.} }
Endnote
%0 Conference Paper %T On the Independence Assumption in Neurosymbolic Learning %A Emile Van Krieken %A Pasquale Minervini %A Edoardo Ponti %A Antonio Vergari %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-van-krieken24a %I PMLR %P 49078--49097 %U https://proceedings.mlr.press/v235/van-krieken24a.html %V 235 %X State-of-the-art neurosymbolic learning systems use probabilistic reasoning to guide neural networks towards predictions that conform to logical constraints. Many such systems assume that the probabilities of the considered symbols are conditionally independent given the input to simplify learning and reasoning. We study and criticise this assumption, highlighting how it can hinder optimisation and prevent uncertainty quantification. We prove that loss functions bias conditionally independent neural networks to become overconfident in their predictions. As a result, they are unable to represent uncertainty over multiple valid options. Furthermore, we prove that the minima of such loss functions are usually highly disconnected and non-convex, and thus difficult to optimise. Our theoretical analysis gives the foundation for replacing the conditional independence assumption and designing more expressive neurosymbolic probabilistic models.
APA
Van Krieken, E., Minervini, P., Ponti, E. & Vergari, A.. (2024). On the Independence Assumption in Neurosymbolic Learning. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:49078-49097 Available from https://proceedings.mlr.press/v235/van-krieken24a.html.

Related Material