Policies of Multiple Skill Levels for Better Strength Estimation in Games

Abstract

In many online competitive games, matchmaking is typically based on ratings systems, such as Elo or Glicko-2. However, these rating systems generally require sufficient matches to achieve accurate estimates, resulting in mismatches with opponents of different skill levels, especially during the early stages. To address this issue, methods for estimating strength from a small number of matches are essential. We use "strength" to refer to a player's overall ability, which can be quantified as a skill level such as a rating tier. In a previous state-of-the-art study, researchers estimated player's skill levels using strength scores learned from human match data. In this paper, we further incorporate policies (i.e., probability distributions of moves) of different skill levels from neural networks trained to imitate human players' gameplay. Namely we combine features from policies and the strength scores to estimate skill levels. We targeted Go and chess, where abundant data is available. Experiments in Go show that our method achieved 80% accuracy in strength estimation when given 10 matches, increasing to 92% when given 20 matches. Compared to the previous state-of-the-art method's 71% and 84%, our approach yields 8% improvements. Similar improvements were observed in chess. Since our method requires no game-specific features, it can be applied to other games or real-world problems.