Game Career Guide is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


Get the latest Education e-news
 
  • Poker AI: A Starting Point

    [09.30.10]
    - Simon Tomlinson
  •  Poker is a fascinating game, both in itself and as a challenging AI development project. Indeed I believe it makes a very useful student training project as it illustrates the thought processes used to build a larger application, yet is tractable on reasonable time scales. This article will look at a compact poker AI solution, originally created for mobile phone platforms with potential for further expansion.

    Learning the Game

    It is very important in this type of project, for any AI project really, to fully understand the rules of the game and the tactics typically used to win. So when I worked on poker I spent a fair amount of time reading the best books and playing the game, both against existing AI and real humans of various levels of experience.

    Luckily, I also had access to a real life professional player, which helped enormously. The primary thing to understand about poker is the game is not about how you manipulate the cards in front of you -- it is about when you bet, and how much - without realistic betting the game completely changes in character. Understanding poker is a bit like peeling an onion. You remove one layer and there is inevitably another layer beneath. But this is good -- it means you can get something simple working quickly and add to it layer by layer.

    There are several variants of poker which use different betting and card dealing patterns, but for the purposes of this article I will focus on No Limit Texas Hold'Em. Each player is initially dealt two cards face down which only he/she will view -- the "hole" cards. There is then a round of betting based on these cards. Each player can fold (leave the current game), call (match the current highest amount bet by previous players) or raise. When players raise they can choose a small amount to increase the bet by (e.g. double the initial bet) right up to their entire "stack" of chips, which is known as "all-in" -- there is no limit to the bet. All bets accumulate in the pot, which is won by the best hand at the end of the game, or the sole surviving player if all others fold. There are up to three passes around the table of betting in any round, and to continue to the next round all players must have called and leveled up to the highest raise in the round.

    There are no "antes" (a fee to enter the game) but two players to the left of the dealer are obliged to make compulsory bets called "blinds"; this ensures that the players cannot simply wait for a perfect hand at no cost. Three cards are then dealt face up in the table centre -- "the flop". All players can use any of these "community" cards in combination with their own hole cards to make up the best five card poker hand they can. So the players are sharing the majority of cards, only their two hole cards are unique to them. Poker hands progress in a hierarchy of values, and the numeric value of the cards is also important. The lowest hand is five unrelated cards, then one pair (two cards of the same face value), two pairs, three of a kind, run (e.g. 2,3,4,5,6 of any suit), flush (five cards the same suit), full house (a pair and a three), four of a kind and then a running flush (e.g. 2,3,4,5,6 of the same suit).

    If players have similar hands numeric value wins -- jacks beat tens, queens beat jacks, kings beat queens and aces beat kings. Aces can also be used "low" in a run with a value of 1. So a pair of kings beats a pair of eights, and a run ending in a queen beats a run ending in a ten. Where hands match such as two players with one pair of the same value, the highest value singleton card (the kicker) wins the hand. If two or more hands are exactly equal, the pot of money is split. It should now be obvious that sharing the community cards has interesting tactical implications. For example if the flop is 3 threes, everyone has that hand as a minimum; it is what you can add with your own two hole cards that will win the game.

    Following the flop there is another round of betting, followed by another card "the Turn" and more betting and a final card "the River", and more betting. From the flop onwards the first player to bet (left of the dealer) has the option to "check" -- that is, bet nothing. If all players check the game moves on, but if one bets then all other players must at least "call". It is also possible for the large blind in the pre-flop round to check if nobody has raised above the blind bet. After the final betting round players reveal their two hole cards and the pot is claimed by the winner. Various other special rules apply -- especially to do with "all-in" players, but I won't describe those here. Once the game is complete the "dealer button" moves on to the next player to the left so that the blinds move around the table, as does the advantage of being last to play.

    There are a number of basic tactics to consider. A player can win a game by having the best hand, by having a hand that has the potential to become the best hand as the community cards are dealt, or by forcing all the other players to fold using tactical betting. Since a player must match a raise to continue, raising by a large amount forces the opponent to weigh his chances of winning the poker hand against the investment he must make -- more on this later. A typical Texas Hold'Em table will have 8-12 players, more than half usually folding on the first round. In some games a large raise by one player in strength will force all the others to fold before all the cards are dealt. Of course this also opens up the possibility of bluffing -- raising large to fool an opponent into folding when in fact you only have a weak hand (there are also other forms of bluffing). Probably only a minority of games will go to the River and actually reveal the hands, and this will usually be only two or three players.

    I have described these playing patterns because it is important to understand how the game will play out in real life. Indeed these patterns can be used as a metric. In testing I set up AI only tables playing thousands of games and collected statistics on how many games went to the river, fold rates and so on to help balance the game and verify that the AI performed in a similar way to humans. For the same reason it is also useful to know that the average win rate per game of even an expert player is usually quite small compared to the typical pot size: poker players will win some pots and loose some, but expect to make profit on average over time rather than on individual hands. It's also worth saying though that tournament play differs from general continuous play -- here your wins are secondary to the elimination of opponents, and forcing players to go all-in quickly is a common tactic for an aggressive player.

Comments

comments powered by Disqus