4

In real games, it seems an NLH heads up bot is now at the point of (or close to) being able to beat the best human players. http://www.cs.cmu.edu/news/cmu-ai-tough-poker-player

For heads up limit, the Nash equilibrium is close to known http://arstechnica.com/science/2015/01/computers-used-to-solve-two-person-limit-texas-holdem/

Logically, if each situation, defined by having the exact same cards on the table and the exact same preceding action, has only one bet sizing, NLH is no larger than limit holdem. For example, if there's a bet and a call preflop and the flop comes 842 all hearts, the continuation bet (assuming there is one) could always be 3/4 pot. I assume no mixing between bet sizes would mean that bet sizes are independent of the hole cards, but I could be wrong.

In addition, if there's no mixing between different actions, no 80% bet / 20% check, but always 100% one or the other, the game becomes significantly smaller.

Of course, a strategy without mixing is significantly worse than the true Nash equilibrium, but it would still be interesting to see. Is such a strategy known? Is there any publicly available software that calculates such a strategy?

  • I'm asking about the perfect Nash equilibrium strategy for heads up NLHE, with the restriction that there is no mixing, nothing probabilistic; with a given range of hole cards the player will always do the same thing in a given situation. – JollyJoker Jan 22 '17 at 17:21
1

I am probably one of the few people with significant experience in all three fields (mathematics, Machine Learning (and AI), poker).

To answer is it known, the answer is probably not yet. There are two approaches to beating an imperfect information game with AI. The first has really nothing to do with Nash Equilibrium. The first is basically in a way, a brute force method. With significant training data, the computer can learn what works without actually having the logic to understand why it works. It can train on previous games for 5 million hands and realize "holy cow, if I raise with AA preflop, I seem to win more." This is Machine Learning and it's counter-intuitive to what people would generally consider AI (if they don't actually work with AI).

The second method of beating a game of poker doesn't ACTUALLY work, that would be based on Nash Equilibrium. Most game theory assume perfectly intelligent players making optimum choices, poker isn't that. Most poker game involves reading a table and beating the field (not individual players). So if you start playing the $1/$2 cash game table with a table full of drunken bachelor party guests, you do not employ any optimized decision making game theory strategy. Nash Equilibrium by definition requires perfect actors and in poker, the players are imperfect. This is even more true when you tell the AI to make deterministic decisions instead of a "mixing strategy". In fact, an "optimized" AI that's not learning and changing strategy will be easier to beat than a human due to the fact that it DOES NOT vary its strategy.

So regardless of how the AI obtains its base strategy (via some programmer given logic or via machine learning), you need to make it adapt. Which means reevaluation of strategy and rescoring of decisions. It would be possible to make an AI that's smart enough to start off with good base strategy and adapt enough to win, but honestly, it will never be considered "beating" poker simply because... The same strategy that works on Tom Dwan will not work on your drunken uncle, they might play EXACTLY the same the first few hands, but the AI would not be able to predict their strategy changes because it simply doesn't know the player. It cannot 100% beat a game like poker, but it can 100% beat Chess (due to perfect information).

1

The solution is actually known. I can solve it in 2-3 days using a server I rent for that exact purpose (studying poker) and some specialized software.

To the answer from Ying Li:

You misunderstand Nash Equilibrium in poker. If you played a perfect GTO strategy, you essentially would be unbeatable. There is no way your strategy would be beat by drunk players or even top players. Literally, by definition, the optimal counter strategy vs a player playing perfect a GTO strategy is to also play a perfect GTO strategy.

Your confusing game theory optimal strategies with maximally exploitative strategies. A maximally exploitative strategy will make more $$ vs a drunk idiot but your max exploit strat will be open to also being exploited. So if the player adjusts or your assumptions about his leaks are wrong you open yourself up to being exploited.

While playing a GTO strategy, guarantee's people will not be able to get an edge against you and lets you profit from all of their mistakes without really worrying about how they play. B/c your strategy is perfect, your worst case scenario is facing another GTO player in which case you both will breakeven against eachother. If you face a player playing worse than GTO you will profit (not counting rake).

Extreme example:

Imagine a player that only plays AA heads up and folds everything else.

Max Exploit: Steal 100%, folding everything other than AA when the player enters the hand. Your going to make a ton of money if the opponent doesn't adjust, but your strategy can very easily be exploited. If the player decides to raise 72o your going to be folding KK.

GTO Strat: Play a strategy that cant be exploited. That means your not folding KK or QQ or AK preflop. You leaving a lot of money on the table but by not exploiting this player (YOU STILL ARE MAKING A TON OF MONEY JUST NOT THE MAXIMUM AMOUNT VS HIS TERRIBLE STRAT) but at the same time if he changes his strategy the best he will be able to do is to b/e vs you.

  • The issue here is that the perfect GTO strategy cannot exist in poker. It's not the same as other game that can be beaten. There's no Nash equilibrium strategy for poker simply because beating the game is opponent dependent (not skill level dependent). The reason is that most decisions in Poker are pretty much binary (except the ones that involve bet sizing). A very bad player can make the same moves as a very good play at one street and then do completely different things on another round; similarly, the same player can make different moves based on his/her mood/condition. – Ying Li Jun 23 '18 at 22:53
  • What can be mathematically perfected is a BASE strategy against "the field", but never for a particular poker match. The best BASE strategy is designed to win against maximum amount of average players for maximum amount of profit. The rough best BASE strategy is pretty much known (don't play shit hand, bet instead of call if you are going to call anyways, aggressive > passivity... etc... ) This general strategy coded in perfectly to a poker AI will probably make a shit load of money on Pokerstar, but it would not be able to "be unbeatable", that's literally impossible in poker. – Ying Li Jun 23 '18 at 22:56
  • I will demonstrate why it's "impossible". Let's imagine the perfect unbeatable strategy is able to calculate the perfect reaction when you have middle set vs a pot size bet against a flop with a flush draw. Let's imagine that perfect move is to reraise. This move will NEVER be unbeatable because of imperfect information and humans with imperfect information. Your reraise might make sense because you will win the max against draws and overpairs, but you didn't realize that drunken Bob only bets pot size into that flop with top set and never calls a reraise. – Ying Li Jun 23 '18 at 22:59
  • Which essentially means, that "perfect strategy" will be beaten "randomly" by someone using random retarded strategies. Poker has too few moves, it's too binary, really shitty players can make really good move because he doesn't understand and just made that move randomly. It's literally impossible for an AI to consider that UNLESS the AI is a mind reader or understands the opponent well (with machine learning data on that opponent). But that's cheating, that's called prep. If you have an AI specifically trained to beat A, it can definitely beat A, but the strategy will be uniquely set for A. – Ying Li Jun 23 '18 at 23:02
  • @lessharm Could you do this and make the result available somewhere? It would be interesting to see. – JollyJoker Jun 25 '18 at 7:07
-1

It should also be noted that the Nash equilibrium won't even be CLOSE to the most profitable poker strategy. They are just unrelated concepts. That's because poker is a human con game more than a straight strategy game. It's a fascinating question, and it illustrates the human nature of the game that can't really be articulated in the rules alone. For instance, in real life poker, you might decide to tip the floor person to call you when a sucker comes to town. This is a real life poker strategy that the pure Nash equilibrium won't consider. You can say all you want that's not part of the game, but that's how I keep score! Very soon, human sized robots will move around in the world. I guarantee when that happens, the Bellagio will ban them from the poker room [except to maybe serve drinks].

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.