Hierarchical Adversarially-Resilient Multi-Agent Reinforcement Learning for Cyber-Physical Systems Security

Abstract

Cyber-Physical Systems are integral to modern critical infrastructure, including manufacturing, energy grids, and autonomous systems, but their increasing interconnectivity exposes them to sophisticated cyber threats. Traditional security measures, such as rule-based intrusion detection and single-agent learning, often fail against adaptive and zero-day attacks. To address this challenge, we propose a Hierarchical Adversarially-Resilient Multi-Agent Reinforcement Learning (HAMARL) framework, integrating adversarial training into a multi-agent security system. HAMARL leverages a hierarchical control structure where local agents manage subsystem security, and a global coordinator optimizes system-wide defense strategies. Additionally, an adversarially-aware learning loop simulates evolving cyber threats, allowing defenders to preemptively adapt to sophisticated attacks. Evaluations on a simulated industrial IoT testbed demonstrate that HAMARL significantly enhances attack detection, reduces response time, and maintains operational continuity compared to traditional MARL approaches. Our findings suggest that hierarchical MARL, combined with adversarial training, presents a promising advancement for securing next-generation CPS.