Combining Knowledge and Reasoning through Probabilistic Soft Logic for Image Puzzle Solving

Somak Aditya, Yezhou Yang, Chitta Baral, Yiannis Aloimonos
Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, PMLR R16:237-247, 2018.

Abstract

The uncertainty associated with human per- ception is often reduced by one’s extensive prior experience and knowledge. Current datasets and systems do not emphasize the ne- cessity and benefit of using such knowledge. In this work, we propose the task of solving a genre of image-puzzles (“image riddles”) that require both capabilities involving visual de- tection (including object, activity recognition) and, knowledge-based or commonsense rea- soning. Each puzzle involves a set of images and the question “what word connects these images?”. We compile a dataset of over 3k riddles where each riddle consists of 4 im- ages and a groundtruth answer. The annota- tions are validated using crowd-sourced eval- uation. We also define an automatic evalua- tion metric to track future progress. Our task bears similarity with the commonly known IQ tasks such as analogy solving, sequence fill- ing that are often used to test intelligence. We develop a Probabilistic Reasoning-based ap- proach that utilizes commonsense knowledge about words and phrases to answer these rid- dles with a reasonable accuracy. Our approach achieves some promising results for these rid- dles and provides a strong baseline for future attempts. We make the entire dataset and re- lated materials publicly available to the com- munity (bit.ly/22f9Ala).

Cite this Paper

BibTeX
@InProceedings{pmlr-vR16-aditya18a, title = {Combining Knowledge and Reasoning through Probabilistic Soft Logic for Image Puzzle Solving}, author = {Aditya, Somak and Yang, Yezhou and Baral, Chitta and Aloimonos, Yiannis}, booktitle = {Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence}, pages = {237--247}, year = {2018}, editor = {Globerson, Amir and Silva, Ricardo}, volume = {R16}, series = {Proceedings of Machine Learning Research}, month = {06--10 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r16/main/assets/aditya18a/aditya18a.pdf}, url = {https://proceedings.mlr.press/r16/aditya18a.html}, abstract = {The uncertainty associated with human per- ception is often reduced by one’s extensive prior experience and knowledge. Current datasets and systems do not emphasize the ne- cessity and benefit of using such knowledge. In this work, we propose the task of solving a genre of image-puzzles (“image riddles”) that require both capabilities involving visual de- tection (including object, activity recognition) and, knowledge-based or commonsense rea- soning. Each puzzle involves a set of images and the question “what word connects these images?”. We compile a dataset of over 3k riddles where each riddle consists of 4 im- ages and a groundtruth answer. The annota- tions are validated using crowd-sourced eval- uation. We also define an automatic evalua- tion metric to track future progress. Our task bears similarity with the commonly known IQ tasks such as analogy solving, sequence fill- ing that are often used to test intelligence. We develop a Probabilistic Reasoning-based ap- proach that utilizes commonsense knowledge about words and phrases to answer these rid- dles with a reasonable accuracy. Our approach achieves some promising results for these rid- dles and provides a strong baseline for future attempts. We make the entire dataset and re- lated materials publicly available to the com- munity (bit.ly/22f9Ala).}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Combining Knowledge and Reasoning through Probabilistic Soft Logic for Image Puzzle Solving %A Somak Aditya %A Yezhou Yang %A Chitta Baral %A Yiannis Aloimonos %B Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2018 %E Amir Globerson %E Ricardo Silva %F pmlr-vR16-aditya18a %I PMLR %P 237--247 %U https://proceedings.mlr.press/r16/aditya18a.html %V R16 %X The uncertainty associated with human per- ception is often reduced by one’s extensive prior experience and knowledge. Current datasets and systems do not emphasize the ne- cessity and benefit of using such knowledge. In this work, we propose the task of solving a genre of image-puzzles (“image riddles”) that require both capabilities involving visual de- tection (including object, activity recognition) and, knowledge-based or commonsense rea- soning. Each puzzle involves a set of images and the question “what word connects these images?”. We compile a dataset of over 3k riddles where each riddle consists of 4 im- ages and a groundtruth answer. The annota- tions are validated using crowd-sourced eval- uation. We also define an automatic evalua- tion metric to track future progress. Our task bears similarity with the commonly known IQ tasks such as analogy solving, sequence fill- ing that are often used to test intelligence. We develop a Probabilistic Reasoning-based ap- proach that utilizes commonsense knowledge about words and phrases to answer these rid- dles with a reasonable accuracy. Our approach achieves some promising results for these rid- dles and provides a strong baseline for future attempts. We make the entire dataset and re- lated materials publicly available to the com- munity (bit.ly/22f9Ala). %Z Reissued by PMLR on 04 October 2026.
APA
Aditya, S., Yang, Y., Baral, C. & Aloimonos, Y.. (2018). Combining Knowledge and Reasoning through Probabilistic Soft Logic for Image Puzzle Solving. Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R16:237-247 Available from https://proceedings.mlr.press/r16/aditya18a.html. Reissued by PMLR on 04 October 2026.

Related Material