Coordination Between Individual Agents in Multi-Agent Reinforcement Learning

Authors

  • Yang Zhang Faculty of Electronics and Information, Xi’an Jiaotong University, Xi’an, China
  • Qingyu Yang Faculty of Electronics and Information, Xi’an Jiaotong University, Xi’an, China; State Key Laboratory for Manufacturing System Engineering (SKLMSE), Xi’an Jiaotong University, Xi’an, China; Ministry of Education (MOE) Key Laboratory for Intelligent Networks and Network Security, Xi’an Jiaotong University, Xi’an, China
  • Dou An Faculty of Electronics and Information, Xi’an Jiaotong University, Xi’an, China; State Key Laboratory for Manufacturing System Engineering (SKLMSE), Xi’an Jiaotong University, Xi’an, China; Ministry of Education (MOE) Key Laboratory for Intelligent Networks and Network Security, Xi’an Jiaotong University, Xi’an, China
  • Chengwei Zhang Dalian Maritime University, Dalian, China

Keywords:

Coordination and Collaboration, Reinforcement Learning

Abstract

The existing multi-agent reinforcement learning methods (MARL) for determining the coordination between agents focus on either global-level or neighborhood-level coordination between agents. However the problem of coordination between individual agents is remain to be solved. It is crucial for learning an optimal coordinated policy in unknown multi-agent environments to analyze the agent's roles and the correlation between individual agents. To this end, in this paper we propose an agent-level coordination based MARL method. Specifically, it includes two parts in our method. The first is correlation analysis between individual agents based on the Pearson, Spearman, and Kendall correlation coefficients; And the second is an agent-level coordinated training framework where the communication message between weakly correlated agents is dropped out, and a correlation based reward function is built. The proposed method is verified in four mixed cooperative-competitive environments. The experimental results show that the proposed method outperforms the state-of-the-art MARL methods and can measure the correlation between individual agents accurately.

Downloads

Published

2021-05-18

How to Cite

Zhang, Y., Yang, Q., An, D., & Zhang, C. (2021). Coordination Between Individual Agents in Multi-Agent Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 35(13), 11387-11394. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17357

Issue

Section

AAAI Technical Track on Multiagent Systems