8

Following up on an article I read in the NY Times I've learned that the best limit hold'em bots are better than the best humans. My understanding is that for each game state they reach these bots must be computing a set of weights, summing to one, for the set of available actions, a subset of { check, call, bet, raise, fold }. My question is, is there a decently-strong computer program available that I can play poker against and be able to query it for what multistrategy it would play if it were in my shoes? I'm emboldened by the NY Times article saying that the bot they're writing about is a winning video poker player even without adapting to the tendencies of its opponent.

There's good information here but its targeted to people looking to write their own bots, not learn from them.

1
  • See my answer below - the Alberta effort is light years behind the fact that there are machines in casinos that you can play today with an AI that most humans can't beat Commented Jan 14, 2015 at 22:16

4 Answers 4

12

I'm Mike Johanson, one of the authors on the recent "HULHE is solved" paper, and I wrote a fair chunk of the code for Cepheus.

kuzzooroo's link was the right one. At the link he gave (not reproduced here - new account, I have no reputation to post > 2 links), you can query any part of Cepheus' strategy, or play against it -- at least, you will be able to once the traffic lightens up, we've had to disable the play-Cepheus page for now. But the query service is running fine.

If you'd like to read the Science paper, you can find a link to it on my personal webpage. My earlier papers there will also give more detail on the algorithm it's based on, called CFR.

If you want the source code, that's available too - BSD license. link

As for other programs and groups, and the history of our effort... The University of Alberta's Computer Poker Research Group started in 1994, and we've had a considerable focus on heads-up limit hold'em since 2001. In 2003 we created the first decent game theoretic strategy for the game, called PsOpti (vlzvl mentioned Sparbot - it's the same strategy, renamed for Poker Academy). In 2007 and 2008, our program Polaris competed against human pros, narrowly losing in 2007 and narrowly winning in 2008, marking the first time that a computer has ever defeated human poker pros in a meaningful match. In 2011, we developed the first tractable algorithm for HULHE that lets us compute an optimal counter-strategy to defeat a given strategy, and measure how much that strategy loses for against the counter-strategy. That lets us measure how close to unbeatable play we are. Ever since, we've been driving closer and closer to perfect play (i.e., towards a worst-case loss of 0), and have been publishing our progress over the years in open, publicly accessible research papers.

In this paper, after 13 years of focus on HULHE from our first decent game theoretic strategy to this one, we're announcing that we have finally essentially solved the game. Cepheus is maximally beatable for under 0.000986 big blinds per game, or under 0.05 big bets / 100 if you prefer those units. It'd take more than 60 million hands of using the perfect counter-strategy to have 95% confidence of winning against Cepheus, which is why we say it's essentially solved: it'd take more than a human lifetime of play to statistically discern the difference between Cepheus and an exact optimal strategy. We go into much more detail on this in the paper.

So as far as Jim Beam's statement that we're claiming credit for something others have already done... I disagree. We've been in this space for 20 years, were the first to achieve the earlier big milestones in HULHE (first not-terrible strat, first program to beat humans, first measurement of worst-case loss), and have pushed the science by fully publishing our work at every step of the way. Essentially solving the game is the final step in a long line of research on this game.

Not much is known about PokerSnowie and the casino game - they don't release stats on how close to optimal they are, or any useful details of what algorithms they use or how their programs work. PokerSnowie claims to be close to equilibrium, but offer no proof at all, and from the discussions on 2+2 where they also claim to play optimal 6-player and no-limit, it's clear that they either don't know what "game theoretic optimal" play means, or they're exaggerating: no-limit is unfathomably larger than heads-up limit and it's nonsense to claim optimal play there, and an "optimal" strategy wouldn't have any theoretically useful properties in a 6-player game. Actually, in the 2+2 threads, they've admitted to this and removed most of the advertising from their webpage that mentions "optimal play", and have refocussed their marketing.

Cepheus is an essentially optimal strategy for heads-up limit hold'em, and it can play against either of them, or any human or other program, and be guaranteed to not lose. We're @PolarisPoker on twitter - we mostly use that for posting links to new papers that we publish.

2

Heads-up limit hold'em is now solved!. The AI has a great website that will, among other things, tell you the optimal multistrategy at http://poker.srv.ualberta.ca/.

2
  • Nice one. I've updated my answer to point people at yours. Commented Jan 14, 2015 at 22:21
  • I don't know about that An 'essentially unbeatable' algorithm (perhaps it is) but Poker Academy Pro certainly have a very powerful bot specialized in heads-up, called Sparbot. The good thing about it is that it doesn't tries to learn the opponent, thus is easier to emulate it. Didn't tried it though.
    – user1165
    Commented Jan 14, 2015 at 23:58
0

The NY Times article was fascinating to me because they completely ignored the fact that Poker Snowie already exists. Google them and read up on them. They have been around a while and they are generally accepted as playing very, very good poker. I'm not sure if they've "solved" poker (whatever that means!) but they have been around a long time.

In fact, most people don't even realize that there are already LHE machines in casinos that you can play against that also play very, very good poker (based on the work by Poker Snowie). You can play them heads up, various stakes. Again, maybe they didn't succeed in "solving" poker, but do you think they offered in casinos because they are losing money on them? Of course not. So all you have to do is find a casino in your area that offers this and you can try yourself against a very good poker AI

4
  • Do you work for Poker Snowie? Commented Jan 14, 2015 at 22:19
  • @WillCalderwood No, no of course not. In fact, I don't necessarily even like all of the Snowie tools - I just like the replayer. I just don't think it's fair to see Alberta claiming credit for LHE when there's already machines in casinos. That's like me coming out and saying, "hey guys, I invented something I call a 'car'! Can you believe it?!?!". Commented Jan 15, 2015 at 3:01
  • 1
    Point 1. I assume the casino machines have rake, so they don't need a perfect, or even near perfect strategy to win. They just need to play at an ok level. Point 2, you can't query them. Commented Jan 15, 2015 at 11:48
  • @WillCalderwood making less not more assumptions here would help you :/ Have you researched anything I've put out there??? No the machines do not have a rake and you can query them. Poker Snowie is basically the only tool out there that lets you enter hands into the replayer and it will let you query EV and action moves along the way. When you're playing against it, you can also get real-time feedback on right/wrong moves, also with EV. Commented Jan 15, 2015 at 16:32
0

There are no publicly available AIs to practice against and train with at the level I'm afraid. If you're looking for an AI to play against you can look at Poker Academy, their site has been up and down in recent years but that considered the best limit AI publicly available. Their no limit AIs are rubbish however.

Update. As kuzzooroo pointed out, the above answer is now incorrect. Please see kuzzooroo's answer instead, here.

7
  • Unfortunately it looks like Poker Academy's web page is currently in the down state. However, it links to Poker Genius which does claim to provide "action frequencies."
    – kuzzooroo
    Commented Feb 7, 2014 at 4:57
  • Poker Academy is "down" in the sense that the company is out of business. Poker Genius claims to be the de facto replacement, but I haven't tried it.
    – elixenide
    Commented Feb 13, 2014 at 8:22
  • This is just plain wrong, you CAN play. Please see my answer Commented Jan 14, 2015 at 22:10
  • @JimBeam At the time this answer was posted it was correct. The answer the kuzzooroo posted a few hours ago is more correct than your answer. Commented Jan 14, 2015 at 22:18
  • @WillCalderwood No way that's correct. The machines have been in casinos for more than 18 months - your answer was less than a year ago. Commented Jan 15, 2015 at 3:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.