Is the heads up NLH no mixing Nash equilibrium strategy known?

Question

In real games, it seems an NLH heads up bot is now at the point of (or close to) being able to beat the best human players. See https://www.cs.cmu.edu/news/cmu-ai-tough-poker-player

For heads up limit, the Nash equilibrium is close to known. See: https://arstechnica.com/science/2015/01/computers-used-to-solve-two-person-limit-texas-holdem/

Logically, if each situation, defined by having the exact same cards on the table and the exact same preceding action, has only one bet sizing, NLH is no larger than limit holdem. For example, if there's a bet and a call preflop and the flop comes 842 all hearts, the continuation bet (assuming there is one) could always be 3/4 pot. I assume no mixing between bet sizes would mean that bet sizes are independent of the hole cards, but I could be wrong.

In addition, if there's no mixing between different actions, no 80% bet / 20% check, but always 100% one or the other, the game becomes significantly smaller.

Of course, a strategy without mixing is significantly worse than the true Nash equilibrium, but it would still be interesting to see. Is such a strategy known? Is there any publicly available software that calculates such a strategy?

Answer 1

The solution is actually known. I can solve it in 2-3 days using a server I rent for that exact purpose (studying poker) and some specialized software.

To the answer from Ying Li:

You misunderstand Nash Equilibrium in poker. If you played a perfect GTO strategy, you essentially would be unbeatable. There is no way your strategy would be beat by drunk players or even top players. Literally, by definition, the optimal counter strategy vs a player playing perfect a GTO strategy is to also play a perfect GTO strategy.

Your confusing game theory optimal strategies with maximally exploitative strategies. A maximally exploitative strategy will make more $$ vs a drunk idiot but your max exploit strat will be open to also being exploited. So if the player adjusts or your assumptions about his leaks are wrong you open yourself up to being exploited.

While playing a GTO strategy, guarantee's people will not be able to get an edge against you and lets you profit from all of their mistakes without really worrying about how they play. B/c your strategy is perfect, your worst case scenario is facing another GTO player in which case you both will breakeven against eachother. If you face a player playing worse than GTO you will profit (not counting rake).

Extreme example:

Imagine a player that only plays AA heads up and folds everything else.

Max Exploit: Steal 100%, folding everything other than AA when the player enters the hand. Your going to make a ton of money if the opponent doesn't adjust, but your strategy can very easily be exploited. If the player decides to raise 72o your going to be folding KK.

GTO Strat: Play a strategy that cant be exploited. That means your not folding KK or QQ or AK preflop. You leaving a lot of money on the table but by not exploiting this player (YOU STILL ARE MAKING A TON OF MONEY JUST NOT THE MAXIMUM AMOUNT VS HIS TERRIBLE STRAT) but at the same time if he changes his strategy the best he will be able to do is to b/e vs you.

Answer 2

I am probably one of the few people with significant experience in all three fields (mathematics, Machine Learning (and AI), poker).

To answer is it known, the answer is probably not yet. There are two approaches to beating an imperfect information game with AI. The first has really nothing to do with Nash Equilibrium. The first is basically in a way, a brute force method. With significant training data, the computer can learn what works without actually having the logic to understand why it works. It can train on previous games for 5 million hands and realize "holy cow, if I raise with AA preflop, I seem to win more." This is Machine Learning and it's counter-intuitive to what people would generally consider AI (if they don't actually work with AI).

The second method of beating a game of poker doesn't ACTUALLY work, that would be based on Nash Equilibrium. Most game theory assume perfectly intelligent players making optimum choices, poker isn't that. Most poker game involves reading a table and beating the field (not individual players). So if you start playing the $1/$2 cash game table with a table full of drunken bachelor party guests, you do not employ any optimized decision making game theory strategy. Nash Equilibrium by definition requires perfect actors and in poker, the players are imperfect. This is even more true when you tell the AI to make deterministic decisions instead of a "mixing strategy". In fact, an "optimized" AI that's not learning and changing strategy will be easier to beat than a human due to the fact that it DOES NOT vary its strategy.

So regardless of how the AI obtains its base strategy (via some programmer given logic or via machine learning), you need to make it adapt. Which means reevaluation of strategy and rescoring of decisions. It would be possible to make an AI that's smart enough to start off with good base strategy and adapt enough to win, but honestly, it will never be considered "beating" poker simply because... The same strategy that works on Tom Dwan will not work on your drunken uncle, they might play EXACTLY the same the first few hands, but the AI would not be able to predict their strategy changes because it simply doesn't know the player. It cannot 100% beat a game like poker, but it can 100% beat Chess (due to perfect information).

Answer 3

It should also be noted that the Nash equilibrium won't even be CLOSE to the most profitable poker strategy. They are just unrelated concepts. That's because poker is a human con game more than a straight strategy game. It's a fascinating question, and it illustrates the human nature of the game that can't really be articulated in the rules alone. For instance, in real life poker, you might decide to tip the floor person to call you when a sucker comes to town. This is a real life poker strategy that the pure Nash equilibrium won't consider. You can say all you want that's not part of the game, but that's how I keep score! Very soon, human sized robots will move around in the world. I guarantee when that happens, the Bellagio will ban them from the poker room [except to maybe serve drinks].