Scalable and Provable Decision-Making for Large-Population Multi-Agent Systems in Complex Domains
Author(s)
Guan, Yue Scott
Advisor(s)
Editor(s)
Collections
Supplementary to:
Permanent Link
Abstract
Decision-making algorithms with increased autonomy are critical for enabling future multi-agent systems. Unlike single-agent settings, multi-agent systems face severe scalability challenges due to inter-agent interactions. In particular, the joint policy and state spaces grow exponentially with the number of agents and the environment’s complexity, rendering classical approaches inadequate. This dissertation develops new formulations, frameworks, and algorithms to improve the efficiency and robustness of decision-making in large-scale multi-agent systems. The contributions address scalability along two key dimensions: the number of agents and the size of the state space.
For challenges associated with large-population teams, we employ a mean-field approximation, which reduces interactions among agents to those between a representative agent and the aggregate population distribution. We extend this framework to discrete state and action spaces by introducing entropy regularization to address the lack of regularity. To capture adversarial team interactions, we propose a zero-sum mean-field team game and provide a theoretical justification for the identical strategy simplification on the opponent team, enabling tractable analysis in competitive multi-team settings. Building on these foundations, we develop a scalable algorithm to learn competitive strategies for systems with hundreds of agents.
The study of large-population adversarial interactions led to the dynamic Defender Attacker Blotto game formulation, which generalizes static Blotto models to dynamic resource allocation over graphs. Through reachability analysis, we removed the classical assumption of instantaneous relocation and derived necessary and sufficient conditions for successful defense. These results bridge abstract allocation models with realistic robotic defense scenarios, where mobility constraints and network topology critically shape outcomes.
To address large state spaces, we develop a hierarchical decomposition framework for stochastic games. By leveraging the options framework, we construct a meta-game over coarse state representations, thereby reducing the complexity of computing Nash equilibria. We further enhance this approach with an automatic state aggregation method based on multi-timescale learning and tree-based abstractions, which adaptively partitions the state space to balance approximation accuracy with tractability, eliminating the need for hand-crafted abstractions.
Finally, we investigate decision-making under asymmetric information in hierarchical games. We introduce the notion of abstraction concealment, wherein agents strategically obscure their internal environment representations from adversaries. Formulated as a Bayesian game, the problem is solved using a bilinear program that computes perfect Bayesian equilibria. This represents one of the first principled treatments of deception through abstraction concealment, revealing how reduced representations shape equilibrium behaviors in adversarial settings.
Together, these contributions advance the scalability and robustness of multi-agent decision-making, bridging theory and algorithm design for hierarchical, large-population, and information-asymmetric settings.
Sponsor
Date
2025-12
Extent
Resource Type
Text
Resource Subtype
Dissertation (PhD)