Opponent Hand Estimation in Gin Rummy Using Deep Neural Networks and Heuristic Strategies

Abstract

A vital part of any good strategy for most imperfect-information games is making predictions about the information that is unavailable. For example, in card games like Poker and Gin Rummy, predicting the kinds of cards the opponent is holding is necessary for playing well. Specifically, it is useful for agents to be able to map the partial game states that are made available to them to the probabilities of each of the possible complete game states, given that they are playing against other rational player(s). Finding this relationship, however, is difficult, as it requires knowledge of how a rational player would play, which is the problem this relationship is being used to solve. In this paper, we attempt to find this relationship in the context of the card game Gin Rummy, though instead of predicting the complete game state, we focus on what is most useful to a player: the opponent's hand. We do this by using heuristic utility functions to create an agent that approximates how a rational player would play, and then using the resulting game data to train a Deep Neural Network mapping known information to predictions about the opponent's hand. This model is used to improve the existing agent and, in turn, to produce more data to create better models.