AI BEATS HUMAN POKER CHAMPIONS
By R. Colin Johnson
EE Times
July 7, 2008

Original Link

PORTLAND, OREGON - Humanity was dealt a decisive blow by a poker-playing artificial intelligence program called Polaris during the Man-Machine Poker Competition in Las Vegas.

Poker champs fought the AI system to a draw, then won in the first two of four rounds (each round had Polaris playing 500 hands against two humans, whose points were averaged.) But in the final two rounds of the match, Polaris beat both human teams, two wins out of four, with one loss and one draw.

IBM's Deep Blue beat chess champion Gary Kasparov in 1997. A year later, the University of Alberta's Computer Poker Research Group began winning hands with early prototypes that eventually became Polaris. A decade later, Polaris 2.0 added poker to the list of machine triumphs.

The key to Polaris' poker prowess last weekend was a tactical shift in midstream designed to prevent human's from exploiting perceived weaknesses. Add to that, Polaris learned from experience.

"There are two really big changes in Polaris over last year," said professor Michael Bowling, who supervised graduate students who programmed Polaris. "First of all, our poker model is much expanded over last year -- its much harder for humans to exploit weaknesses. And secondly, we have added an element of learning, where Polaris identifies which common poker stratagy a human is using and switches its own strategy to counter. This complicated the human players ability to compare notes, since Polaris chose a different strategy to use against each of the humans it played," Bowling said.

Before the Las Vegas match, this newest version of Polaris had only played two matches against champion poker players, resulting in one loss and one victory. Polaris repeated the pattern of improving as it learned, falling to humans in the first two rounds, but defeating them in rounds three and four. "Repeatedly, I heard players exclaim that they had never seen a human do that before," said Bowling. "Switching strategies really threw the humans for a loop."

Polaris played against Nick "Stoxtrader" Grudzien -- a $1 million poker contest winner and founder of a Web site which provides poker-coaching and online play with world champions. Other human champions were coaches on Grudzien's site.

In the first Man-Machine Poker Competition, two human champions beat Polaris in its last two matches, but Polaris won and played to a draw in the first two. The older version of Polaris did not learn, but the humans did, beating Polaris 1.0 in three of four rounds by exploiting weaknesses.

Polaris 2.0 had learning built into its programming, thereby countering the learning ability of the humans by switching strategies whenever they did.

Even though Polaris beat the humans in Las Vegas, the University of Alberta group said it expects to be asked for rematches by the vanquished pros as well as by other poker experts who will claim the win by Polaris was a fluke. "Even after Deep Blue beat Kasparov, there were still some skeptics, and I think the same is true here," said Bowling. "Over the next year or so there are going to have to be several rematches before everyone is convinced that humans have been surpassed by machines in poker."

Meanwhile, Bowling's group plans to expand Polaris beyond its current limitations, enabling it to play more complicated poker games than its current heads-up, hold-em version. They also plan to expand efforts to apply the poker-playing algorithms to useful applications.

"The techniques we are devising have broad applications outside of poker," said Bowling. "For instance, wireless sensor networks are exploring one of our poker-like algorithms to lay out sensors in buildings in a way that yields better understanding of how heat flow patterns affect efficiency."

One algorithm, called counter-factual regret, monitored the outcome of hands lost by Polaris and what could have been done to change the outcome. Polaris could then watch for similar circumstances and adjust more effectively.

BioTools Inc. (Edmonton, Alberta) has built previous versions of Polaris into a downloadable poker coach called the Poker Academy.