A Regret Minimization Approach to Multi-Agent Control

Udaya Ghai, Udari Madhushani, Naomi Leonard, Elad Hazan
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:7422-7434, 2022.

Abstract

We study the problem of multi-agent control of a dynamical system with known dynamics and adversarial disturbances. Our study focuses on optimal control without centralized precomputed policies, but rather with adaptive control policies for the different agents that are only equipped with a stabilizing controller. We give a reduction from any (standard) regret minimizing control method to a distributed algorithm. The reduction guarantees that the resulting distributed algorithm has low regret relative to the optimal precomputed joint policy. Our methodology involves generalizing online convex optimization to a multi-agent setting and applying recent tools from nonstochastic control derived for a single agent. We empirically evaluate our method on a model of an overactuated aircraft. We show that the distributed method is robust to failure and to adversarial perturbations in the dynamics.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-ghai22a, title = {A Regret Minimization Approach to Multi-Agent Control}, author = {Ghai, Udaya and Madhushani, Udari and Leonard, Naomi and Hazan, Elad}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {7422--7434}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/ghai22a/ghai22a.pdf}, url = {https://proceedings.mlr.press/v162/ghai22a.html}, abstract = {We study the problem of multi-agent control of a dynamical system with known dynamics and adversarial disturbances. Our study focuses on optimal control without centralized precomputed policies, but rather with adaptive control policies for the different agents that are only equipped with a stabilizing controller. We give a reduction from any (standard) regret minimizing control method to a distributed algorithm. The reduction guarantees that the resulting distributed algorithm has low regret relative to the optimal precomputed joint policy. Our methodology involves generalizing online convex optimization to a multi-agent setting and applying recent tools from nonstochastic control derived for a single agent. We empirically evaluate our method on a model of an overactuated aircraft. We show that the distributed method is robust to failure and to adversarial perturbations in the dynamics.} }
Endnote
%0 Conference Paper %T A Regret Minimization Approach to Multi-Agent Control %A Udaya Ghai %A Udari Madhushani %A Naomi Leonard %A Elad Hazan %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-ghai22a %I PMLR %P 7422--7434 %U https://proceedings.mlr.press/v162/ghai22a.html %V 162 %X We study the problem of multi-agent control of a dynamical system with known dynamics and adversarial disturbances. Our study focuses on optimal control without centralized precomputed policies, but rather with adaptive control policies for the different agents that are only equipped with a stabilizing controller. We give a reduction from any (standard) regret minimizing control method to a distributed algorithm. The reduction guarantees that the resulting distributed algorithm has low regret relative to the optimal precomputed joint policy. Our methodology involves generalizing online convex optimization to a multi-agent setting and applying recent tools from nonstochastic control derived for a single agent. We empirically evaluate our method on a model of an overactuated aircraft. We show that the distributed method is robust to failure and to adversarial perturbations in the dynamics.
APA
Ghai, U., Madhushani, U., Leonard, N. & Hazan, E.. (2022). A Regret Minimization Approach to Multi-Agent Control. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:7422-7434 Available from https://proceedings.mlr.press/v162/ghai22a.html.

Related Material