Learning Latent Representations to Influence Multi-Agent Interaction

Annie Xie, Dylan Losey, Ryan Tolsma, Chelsea Finn, Dorsa Sadigh
Proceedings of the 2020 Conference on Robot Learning, PMLR 155:575-588, 2021.

Abstract

Seamlessly interacting with humans or robots is hard because these agents are non-stationary. They update their policy in response to the ego agent’s behavior, and the ego agent must anticipate these changes to co-adapt. Inspired by humans, we recognize that robots do not need to explicitly model every low-level action another agent will make; instead, we can capture the latent strategy of other agents through high-level representations. We propose a reinforcement learning-based framework for learning latent representations of an agent’s policy, where the ego agent identifies the relationship between its behavior and the other agent’s future strategy. The ego agent then leverages these latent dynamics to influence the other agent, purposely guiding them towards policies suitable for co-adaptation. Across several simulated domains and a real-world air hockey game, our approach outperforms the alternatives and learns to influence the other agent.

Cite this Paper

BibTeX
@InProceedings{pmlr-v155-xie21a, title = {Learning Latent Representations to Influence Multi-Agent Interaction}, author = {Xie, Annie and Losey, Dylan and Tolsma, Ryan and Finn, Chelsea and Sadigh, Dorsa}, booktitle = {Proceedings of the 2020 Conference on Robot Learning}, pages = {575--588}, year = {2021}, editor = {Kober, Jens and Ramos, Fabio and Tomlin, Claire}, volume = {155}, series = {Proceedings of Machine Learning Research}, month = {16--18 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v155/xie21a/xie21a.pdf}, url = {https://proceedings.mlr.press/v155/xie21a.html}, abstract = {Seamlessly interacting with humans or robots is hard because these agents are non-stationary. They update their policy in response to the ego agent’s behavior, and the ego agent must anticipate these changes to co-adapt. Inspired by humans, we recognize that robots do not need to explicitly model every low-level action another agent will make; instead, we can capture the latent strategy of other agents through high-level representations. We propose a reinforcement learning-based framework for learning latent representations of an agent’s policy, where the ego agent identifies the relationship between its behavior and the other agent’s future strategy. The ego agent then leverages these latent dynamics to influence the other agent, purposely guiding them towards policies suitable for co-adaptation. Across several simulated domains and a real-world air hockey game, our approach outperforms the alternatives and learns to influence the other agent.} }
Endnote
%0 Conference Paper %T Learning Latent Representations to Influence Multi-Agent Interaction %A Annie Xie %A Dylan Losey %A Ryan Tolsma %A Chelsea Finn %A Dorsa Sadigh %B Proceedings of the 2020 Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2021 %E Jens Kober %E Fabio Ramos %E Claire Tomlin %F pmlr-v155-xie21a %I PMLR %P 575--588 %U https://proceedings.mlr.press/v155/xie21a.html %V 155 %X Seamlessly interacting with humans or robots is hard because these agents are non-stationary. They update their policy in response to the ego agent’s behavior, and the ego agent must anticipate these changes to co-adapt. Inspired by humans, we recognize that robots do not need to explicitly model every low-level action another agent will make; instead, we can capture the latent strategy of other agents through high-level representations. We propose a reinforcement learning-based framework for learning latent representations of an agent’s policy, where the ego agent identifies the relationship between its behavior and the other agent’s future strategy. The ego agent then leverages these latent dynamics to influence the other agent, purposely guiding them towards policies suitable for co-adaptation. Across several simulated domains and a real-world air hockey game, our approach outperforms the alternatives and learns to influence the other agent.
APA
Xie, A., Losey, D., Tolsma, R., Finn, C. & Sadigh, D.. (2021). Learning Latent Representations to Influence Multi-Agent Interaction. Proceedings of the 2020 Conference on Robot Learning, in Proceedings of Machine Learning Research 155:575-588 Available from https://proceedings.mlr.press/v155/xie21a.html.

Related Material