Is NLHE "solved"?

Question

Recently an AI poker program built by Facebook and Carnegie Mellon University had a profitable 12 day run against poker pros. The program was earning 10BB an hour at $50/100 against the likes of Jimmy Chou, Seth Davies, Michael Gagliano, Anthony Gregg, Dong Kim, Jason Les, Linus Loeliger, Daniel McAulay, Greg Merson, Nicholas Petrangelo, Sean Ruane, Trevor Savage, and Jacob Toole.

Artificial intelligence has been conquering the masters of games such as Go and Chess over the years, so is poker next?

Answer 1

I saw this and thought it was pretty cool, but I do not think poker is "solved". This AI works by playing a large number of hands against players, mathematically analyzing the decisions it made in those hands, and applying those analytics to future decisions in future hands.

I personally do not think that this means that the AI has solved poker for a couple of reasons:

1. This AI has only been able to master 6-max cash games against a small player pool. (Other AI's have been developed that can reliably beat top pros in heads-up play)

2. It has achieved its level of play by learning, similar to how humans get better. This means that it will have a skill cap based on who it plays against because it will only be able to get good enough to beat them consistently. it will not get any better than that.

3. If this AI plays against itself, I predict that it will reach a nash equilibrium fairly quickly because it will solely focus on playing perfectly optimally against itself, preventing the AI from being able to exploit itself in any way.

These are just my thoughts, and they probably have some kinks or exceptions involved. There is no doubt that this AI is very good at NLHE and it can most likely beat any player out there. I think that it has not yet solved the game though. In my opinion, the breadth of NLHE is much wider than chess or go because of all of the factors that must be taken into account when making a decision (of which there are very many). Things like exploitative play vs. GTO play, tournament strategies including ICM considerations, adjustments in larger games like 9-max games where players are constantly leaving and joining the table, and even information associated with live poker.

Answer 2

I second Clarko's opinion. For no-limit holdem to be solved, you basically need a mathematical way or an algorithm that can be mathematically proven that you win or turn a profit all the time.

An AI being able to beat pros cannot be mathematically proven that it has the algorithm that guarantees it will always win or make money. It could be the best algorithm, but you will never know it is because it is trained to optimize some utility function.

I think there could be multiple version of texas holdem to be considered solved. It really first gets down to defining what is considered to be solved.

One way to define "solved" is that we come up with some algorithm that will guarantee you to turn a profit under certain environment.

Another way to define "solved" is that you come up with some algorithm that will guarantee you will have the most chip at the end of the game. My mathematical instinct tells me that this problem is going to be more difficult than the earlier one.

To conclude, we cannot considered AI beating human in poker as the problem being solved mathematically because you cannot mathematically prove what the AI is doing is indeed the correct algorithm.

Answer 3

The researchers behind Pluribus were not even trying to solve 6max poker. Just prove that they can be less exploitable than some of the best players in the world. The did prove that.

Heads up no limit is not solved either, solving requiring as many decision points as the number of atoms in the universe.

Heads up limit is solved though.

Empirically though, what we know is that with reasonable computing power, doesn't take much to be less exploitable than humans, so we did get to a point where machines will beat humans in the long run forever now.

Answer 4

1- there is one or more Nash equilibriums to poker NLHE 6max and full ring. This is proven by Nash theorem and is explained here: https://www.cs.ubc.ca/~jiang/papers/NashReport.pdf

2- A Nash equilibrium is precisely a point where if both players know exactly the strategy of the other player, there is no way to exploit, ie deviate and benefit from it.

3- This has been found and solved for heads-up limit poker.

4- It has been proven to be unsolvable with computing power as we know it in our known universe for no limit due to the number of decision points. Same as Go which we can't solve.

5- Both Libratus and Cepheus have algorithms converging to Nash equilibrium for heads up nlhe. And have had trials with humans that they beat.

6- Pluribus has not been proven to converge to Nash equilibrium, it does reduce exploitability with each "learning cycle" though. And did reach algos where it statistically beats real top pros.

7- Pluribus does not attempt to reduce variance and as such assumes infinite bankroll. IE in real life it would go bankrupt pretty often before winning big money. In the highly publicised match again,st pros, it did lose 70K$ actually.

Answer 5

No, it's not solved. I think a better question would be - is it even solvable?

The problem with solving no limit poker is that there is no limit to the depth of player stacks. Sure, computers might be able to calculate the optimal strategy for 100 big blinds. But you can have much deeper stacks, such as 1000 big blinds, where there are far more decision points than 100 bbs. Then you can have even deeper stacks: a billion bbs, a google bbs, 3↑↑↑3 bbs, a graham's number amount of bbs, etc etc... going on indefinitely. Optimal post-flop play would look different at every depth, and there would be far more 'options' for bet-sizing.

That being said, I think it still may be solvable, even despite this seemingly insurmountable obstacle of stack depth limitlessness. For example, let's consider heads up play at a million big blinds deep. Even though we are able to raise up to a million big blinds, it is in fact a mistake to ever raise more than ~300 big blinds as the opening raise. This is because an optimal response is simply to fold every hand except AA, and go allin with AA. The odds of getting AA are about 1 in 225, so on average every 225 hands against an opponent making such a raise size, we would be losing 1 blind 224 times while winning 300 blinds one time (assuming our opponent folds to our allin, which he obviously should unless he also has AA himself). This example shows that despite stacks being much deeper than 300, any raise beyond 300 times the original bet would be considered a foolish raise unless it is with AA (pre-flop). This is of course much more complex as we get into post-flop, where there are far more possibilities considering board textures, range asymmetry, etc, but perhaps similar concepts can be applied to eliminate certain bet-sizings that are beyond reason and put us in a more finite realm of possibilities.

All in all it's interesting to contemplate whether no-limit poker is actually solvable, but I highly doubt we will see it within our life times if it is.