Intrinsically Motivated Graph Exploration Using Network Theories of Human Curiosity

Shubhankar Prashant Patankar, Mathieu Ouellet, Juan Cervino, Alejandro Ribeiro, Kieran A. Murphy, Danielle Bassett
Proceedings of the Second Learning on Graphs Conference, PMLR 231:23:1-23:15, 2024.

Abstract

Intrinsically motivated exploration has proven useful for reinforcement learning, even without additional extrinsic rewards. When the environment is naturally represented as a graph, how to guide exploration best remains an open question. In this work, we propose a novel approach for exploring graph-structured data motivated by two theories of human curiosity: the information gap theory and the compression progress theory. The theories view curiosity as an intrinsic motivation to optimize for topological features of subgraphs induced by nodes visited in the environment. We use these proposed features as rewards for graph neural-network-based reinforcement learning. On multiple classes of synthetically generated graphs, we find that trained agents generalize to longer exploratory walks and larger environments than are seen during training. Our method computes more efficiently than the greedy evaluation of the relevant topological properties. The proposed intrinsic motivations bear particular relevance for recommender systems. We demonstrate that next-node recommendations considering curiosity are more predictive of human choices than PageRank centrality in several real-world graph environments.

Cite this Paper

BibTeX
@InProceedings{pmlr-v231-patankar24a, title = {Intrinsically Motivated Graph Exploration Using Network Theories of Human Curiosity}, author = {Patankar, Shubhankar Prashant and Ouellet, Mathieu and Cervino, Juan and Ribeiro, Alejandro and Murphy, Kieran A. and Bassett, Danielle}, booktitle = {Proceedings of the Second Learning on Graphs Conference}, pages = {23:1--23:15}, year = {2024}, editor = {Villar, Soledad and Chamberlain, Benjamin}, volume = {231}, series = {Proceedings of Machine Learning Research}, month = {27--30 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v231/patankar24a/patankar24a.pdf}, url = {https://proceedings.mlr.press/v231/patankar24a.html}, abstract = {Intrinsically motivated exploration has proven useful for reinforcement learning, even without additional extrinsic rewards. When the environment is naturally represented as a graph, how to guide exploration best remains an open question. In this work, we propose a novel approach for exploring graph-structured data motivated by two theories of human curiosity: the information gap theory and the compression progress theory. The theories view curiosity as an intrinsic motivation to optimize for topological features of subgraphs induced by nodes visited in the environment. We use these proposed features as rewards for graph neural-network-based reinforcement learning. On multiple classes of synthetically generated graphs, we find that trained agents generalize to longer exploratory walks and larger environments than are seen during training. Our method computes more efficiently than the greedy evaluation of the relevant topological properties. The proposed intrinsic motivations bear particular relevance for recommender systems. We demonstrate that next-node recommendations considering curiosity are more predictive of human choices than PageRank centrality in several real-world graph environments.} }
Endnote
%0 Conference Paper %T Intrinsically Motivated Graph Exploration Using Network Theories of Human Curiosity %A Shubhankar Prashant Patankar %A Mathieu Ouellet %A Juan Cervino %A Alejandro Ribeiro %A Kieran A. Murphy %A Danielle Bassett %B Proceedings of the Second Learning on Graphs Conference %C Proceedings of Machine Learning Research %D 2024 %E Soledad Villar %E Benjamin Chamberlain %F pmlr-v231-patankar24a %I PMLR %P 23:1--23:15 %U https://proceedings.mlr.press/v231/patankar24a.html %V 231 %X Intrinsically motivated exploration has proven useful for reinforcement learning, even without additional extrinsic rewards. When the environment is naturally represented as a graph, how to guide exploration best remains an open question. In this work, we propose a novel approach for exploring graph-structured data motivated by two theories of human curiosity: the information gap theory and the compression progress theory. The theories view curiosity as an intrinsic motivation to optimize for topological features of subgraphs induced by nodes visited in the environment. We use these proposed features as rewards for graph neural-network-based reinforcement learning. On multiple classes of synthetically generated graphs, we find that trained agents generalize to longer exploratory walks and larger environments than are seen during training. Our method computes more efficiently than the greedy evaluation of the relevant topological properties. The proposed intrinsic motivations bear particular relevance for recommender systems. We demonstrate that next-node recommendations considering curiosity are more predictive of human choices than PageRank centrality in several real-world graph environments.
APA
Patankar, S.P., Ouellet, M., Cervino, J., Ribeiro, A., Murphy, K.A. & Bassett, D.. (2024). Intrinsically Motivated Graph Exploration Using Network Theories of Human Curiosity. Proceedings of the Second Learning on Graphs Conference, in Proceedings of Machine Learning Research 231:23:1-23:15 Available from https://proceedings.mlr.press/v231/patankar24a.html.

Related Material