Social learning spontaneously emerges by searching optimal heuristics with deep reinforcement learning

Seungwoong Ha, Hawoong Jeong
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:12319-12338, 2023.

Abstract

How have individuals of social animals in nature evolved to learn from each other, and what would be the optimal strategy for such learning in a specific environment? Here, we address both problems by employing a deep reinforcement learning model to optimize the social learning strategies (SLSs) of agents in a cooperative game in a multi-dimensional landscape. Throughout the training for maximizing the overall payoff, we find that the agent spontaneously learns various concepts of social learning, such as copying, focusing on frequent and well-performing neighbors, self-comparison, long-term cooperation between agents, and the importance of balancing between individual and social learning, without any explicit guidance or prior knowledge about the system. The SLS from a fully trained agent outperforms all of the traditional, baseline SLSs in terms of mean payoff. We demonstrate the superior performance of the reinforcement learning agent in various environments, including temporally changing environments and real social networks, which also verifies the adaptability of our framework to different social settings.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-ha23a, title = {Social learning spontaneously emerges by searching optimal heuristics with deep reinforcement learning}, author = {Ha, Seungwoong and Jeong, Hawoong}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {12319--12338}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/ha23a/ha23a.pdf}, url = {https://proceedings.mlr.press/v202/ha23a.html}, abstract = {How have individuals of social animals in nature evolved to learn from each other, and what would be the optimal strategy for such learning in a specific environment? Here, we address both problems by employing a deep reinforcement learning model to optimize the social learning strategies (SLSs) of agents in a cooperative game in a multi-dimensional landscape. Throughout the training for maximizing the overall payoff, we find that the agent spontaneously learns various concepts of social learning, such as copying, focusing on frequent and well-performing neighbors, self-comparison, long-term cooperation between agents, and the importance of balancing between individual and social learning, without any explicit guidance or prior knowledge about the system. The SLS from a fully trained agent outperforms all of the traditional, baseline SLSs in terms of mean payoff. We demonstrate the superior performance of the reinforcement learning agent in various environments, including temporally changing environments and real social networks, which also verifies the adaptability of our framework to different social settings.} }
Endnote
%0 Conference Paper %T Social learning spontaneously emerges by searching optimal heuristics with deep reinforcement learning %A Seungwoong Ha %A Hawoong Jeong %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-ha23a %I PMLR %P 12319--12338 %U https://proceedings.mlr.press/v202/ha23a.html %V 202 %X How have individuals of social animals in nature evolved to learn from each other, and what would be the optimal strategy for such learning in a specific environment? Here, we address both problems by employing a deep reinforcement learning model to optimize the social learning strategies (SLSs) of agents in a cooperative game in a multi-dimensional landscape. Throughout the training for maximizing the overall payoff, we find that the agent spontaneously learns various concepts of social learning, such as copying, focusing on frequent and well-performing neighbors, self-comparison, long-term cooperation between agents, and the importance of balancing between individual and social learning, without any explicit guidance or prior knowledge about the system. The SLS from a fully trained agent outperforms all of the traditional, baseline SLSs in terms of mean payoff. We demonstrate the superior performance of the reinforcement learning agent in various environments, including temporally changing environments and real social networks, which also verifies the adaptability of our framework to different social settings.
APA
Ha, S. & Jeong, H.. (2023). Social learning spontaneously emerges by searching optimal heuristics with deep reinforcement learning. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:12319-12338 Available from https://proceedings.mlr.press/v202/ha23a.html.

Related Material