Egocentric Team AI: Enabling Tactical Reasoning from the Operator’s View

Abstract

The future of human–AI integration in high-stakes environments—ranging from defense operations to emergency response—requires AI systems that function as intuitive teammates rather than isolated tools. Yet a persistent perspective gap remains: video understanding models rely on third-person, broadcast-style views, while multi-agent reinforcement learning (MARL) systems operate on egocentric inputs but often depend on centralized critics during training, reducing the need for decentralized policies to internalize team-centric structure. As a result, agents may optimize behavior without learning policy-level representations of the tactical picture grounded in their own observations. To address this gap, we propose Egocentric Team AI, a research direction centered on learning implicit, distributed Common Operating Pictures from first-person views. Building on our prior work with the X-Ego-CS dataset and Cross-Ego Contrastive Learning (CECL), we outline how cross-egocentric representation alignment can be extended from passive video understanding to active multi-agent control. Specifically, we propose integrating cross-ego contrastive objectives into MARL within a multi-agent Doom-based environment as a testbed for decentralized, team-aware policy learning. By positioning cross-egocentric alignment as an inductive bias for decentralized coordination, this work charts a path toward embodied systems capable of adaptive cooperation without explicit communication.