Word Autobots: Using Transformers for Word Association in the Game Codenames

Abstract

Winning the social game Codenames involves combining cooperative and language understanding capabilities. We developed six cooperative bots designed to play the Codemaster and Guesser roles in the Codenames AI Competition and tested them using the provided framework and a round-robin tournament set. The bots are based on term frequency - inverse document frequency (TF-IDF), Naive-Bayes and GPT-2 Transformer word embedding. Additionally, Transformer-based bots were assessed and compared with the concatenation of word2vec and GloVe baseline bot developed by Codenames AI Competition creators. Results from this Transformer implementation rivals the concatenated bot in terms of win rates and guess precision and outperforms it in terms of minimum and average turns taken to win the game and training data load time. Additionally, in an initial evaluation performed with 10 human players, the Transformer agent performed slightly better than the baseline as Codemaster, but worse as a Guesser.