It’s time for computers to get their heads in the game. On Thursday artificial intelligence researchers at the University of Alberta published evidence in Science that an algorithm they developed had solved a type of poker called heads-up limit Texas hold ’em. It’s the first time an imperfect-information game that’s actually played competitively by humans has been solved. (An imperfect-information game is one in which the players all know different things, like in poker, instead of all seeing the same things, like in chess.)
The researchers say that the algorithm has “weakly solved” the game—it can’t win every hand, but over a large number of hands it will either break even or come out ahead. In heads-up limit, players can only bet set amounts of money at particular times. This version of the game is less popular—and less complex from an experimental perspective—than heads-up no-limit, which allows players to bet as much of their money as they want. Heads-up games are always between only two players. Adding more players would make the scenario significantly more difficult to parse.
If you think you’re a pretty good poker player, you can try competing against the algorithm, known as Cepheus. Your confidence will be shattered pretty quickly. It’s awesome.
Solving the game isn’t just about creating robotic dogs that can all sit around a table and play perfect poker, though. “We are a computer poker research group, but we exist because we’re really interested in advancing artificial intelligence techniques,” said group lead Michael Bowling.
Julian Togelius, a researcher at IT University of Copenhagen who works on artificial intelligence and computer games, supports the University of Alberta researchers’ approach. “Games have more and more come to be used as benchmarks for artificial intelligence research,” he said. “This is partly because the other benchmarks are slow and expensive, like robots, or unrealistic and a bit pointless, like simple mathematical functions.”
Bowling explains that poker is a particularly appealing development and testing ground for artificial intelligence algorithms because it presents complicated but somewhat controllable scenarios in which the computer can develop a strategy through trial and error. “There are many different forms of uncertainty that we as humans are able to deal with every day,” he said. “[But] computer programs often can’t cope with these types of uncertainties. Poker embodies all of that in a very pure way so we can test our techniques … and really measure our progress.”
Togelius notes that games are valuable for AI research because they can be iterative for testing. This is related to the idea that algorithms can’t function if they face overwhelming uncertainty. “It’s easy to make many variations of a game, which is important in order to test AI properly: To be generally intelligent, you must be good at not only one task, but many tasks,” Togelius said.
The goal of the hold ’em research is to use poker puzzles as experimental stand-ins for real-world problems. Bowling says the findings in this new research are especially valuable because they give us a hint at the scale of problems AI can solve. Algorithms are already used to optimize solutions in numerous areas, like elevator control and security (for example, air marshal scheduling and coast guard patrolling). But the bigger a problem gets, the less certain people are that they can trust it to an algorithm. Solving a game as complex as heads-up limit Texas hold ’em could mean a breakthrough in our conception of how big is too big. “With our results we’ve been able to show that we can solve … enormous problems on a much larger scale than has been done before,” Bowling said. For example, problems involving vast transportations systems—take a network of airport checkpoints—seem increasingly applicable to new AI methods.
Bowling has also brought some of his group’s AI findings to the University of Alberta hospitals where he, another computer scientist, and two diabetologists are working on creating diabetes management software that can make treatment recommendations—which is tricky because patients’ situations vary widely, and they aren’t always compliant. Bowling acknowledges that the connection between poker and diabetes isn’t obvious, but he says, “It turns out that one of the things a doctor does so well is come up with robust [recommendations] … And that’s what our poker programs have to do, they have to be robust to ‘what are the cards my opponent has, and how does my opponent play?’ ”
OK, so is it time to turn all of our decisions over to computers yet? Bowling says that that process will continue incrementally, the way it has for decades. And there are still significant limits on what AI can achieve. By and large, AI still can’t generalize across things it has learned to extrapolate theories about new uncertainties. “An excellent heads-up limit player can walk into heads-up no-limit and they don’t start from scratch,” Bowling says. “Whereas if we were to try to take our program, which plays heads-up limit at a near-perfect level, and play heads-up no-limit, it couldn’t.” But this is the type of problem that Bowling plans to work on with his group in future research. Hopefully he’s not bluffing about that.
