Universal Agent Mixtures and the Geometry of Intelligence

Samuel Allen Alexander, David Quarel, Len Du, Marcus Hutter
Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR 206:4231-4246, 2023.

Abstract

Inspired by recent progress in multi-agent Reinforcement Learning (RL), in this work we examine the collective intelligent behaviour of theoretical universal agents by introducing a weighted mixture operation. Given a weighted set of agents, their weighted mixture is a new agent whose expected total reward in any environment is the corresponding weighted average of the original agents’ expected total rewards in that environment. Thus, if RL agent intelligence is quantified in terms of performance across environments, the weighted mixture’s intelligence is the weighted average of the original agents’ intelligence. This operation enables various interesting new theorems that shed light on the geometry of RL agent intelligence, namely: results about symmetries, convex agent-sets, and local extrema. We also show that any RL agent intelligence measure based on average performance across environments, subject to certain weak technical conditions, is identical (up to a constant factor) to performance within a single environment dependent on said intelligence measure.

Cite this Paper

BibTeX
@InProceedings{pmlr-v206-alexander23a, title = {Universal Agent Mixtures and the Geometry of Intelligence}, author = {Alexander, Samuel Allen and Quarel, David and Du, Len and Hutter, Marcus}, booktitle = {Proceedings of The 26th International Conference on Artificial Intelligence and Statistics}, pages = {4231--4246}, year = {2023}, editor = {Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem}, volume = {206}, series = {Proceedings of Machine Learning Research}, month = {25--27 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v206/alexander23a/alexander23a.pdf}, url = {https://proceedings.mlr.press/v206/alexander23a.html}, abstract = {Inspired by recent progress in multi-agent Reinforcement Learning (RL), in this work we examine the collective intelligent behaviour of theoretical universal agents by introducing a weighted mixture operation. Given a weighted set of agents, their weighted mixture is a new agent whose expected total reward in any environment is the corresponding weighted average of the original agents’ expected total rewards in that environment. Thus, if RL agent intelligence is quantified in terms of performance across environments, the weighted mixture’s intelligence is the weighted average of the original agents’ intelligence. This operation enables various interesting new theorems that shed light on the geometry of RL agent intelligence, namely: results about symmetries, convex agent-sets, and local extrema. We also show that any RL agent intelligence measure based on average performance across environments, subject to certain weak technical conditions, is identical (up to a constant factor) to performance within a single environment dependent on said intelligence measure.} }
Endnote
%0 Conference Paper %T Universal Agent Mixtures and the Geometry of Intelligence %A Samuel Allen Alexander %A David Quarel %A Len Du %A Marcus Hutter %B Proceedings of The 26th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2023 %E Francisco Ruiz %E Jennifer Dy %E Jan-Willem van de Meent %F pmlr-v206-alexander23a %I PMLR %P 4231--4246 %U https://proceedings.mlr.press/v206/alexander23a.html %V 206 %X Inspired by recent progress in multi-agent Reinforcement Learning (RL), in this work we examine the collective intelligent behaviour of theoretical universal agents by introducing a weighted mixture operation. Given a weighted set of agents, their weighted mixture is a new agent whose expected total reward in any environment is the corresponding weighted average of the original agents’ expected total rewards in that environment. Thus, if RL agent intelligence is quantified in terms of performance across environments, the weighted mixture’s intelligence is the weighted average of the original agents’ intelligence. This operation enables various interesting new theorems that shed light on the geometry of RL agent intelligence, namely: results about symmetries, convex agent-sets, and local extrema. We also show that any RL agent intelligence measure based on average performance across environments, subject to certain weak technical conditions, is identical (up to a constant factor) to performance within a single environment dependent on said intelligence measure.
APA
Alexander, S.A., Quarel, D., Du, L. & Hutter, M.. (2023). Universal Agent Mixtures and the Geometry of Intelligence. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 206:4231-4246 Available from https://proceedings.mlr.press/v206/alexander23a.html.

Related Material