Does the OTG strategy for heads-up limit Hold'em give a positive reward against a non-optimal opponent?

Question

Consider the game of Rock-Paper-Scissors (1v1), +1 when we win, -1 when we loose. An OTG strategy is to select each sign uniformly at random. Unfortunately, this strategy will give you an expected reward of 0, even if the opponent plays terribly (e.g. shows paper all the time).

In the case of heads-up limit Hold'em, will the OTG strategy (a near-perfect approximation of it is used by the Cepheus bot) give a positive reward if an opponent plays non-optimally?

Answer 1

Will the OTG strategy give a positive reward if an opponent plays non-optimally?

I am 99.99999999% sure that the answer is yes. I don't have a mathematical proof, but here are two intuitive arguments:

1. Play many hands against the Cepheus bot. You will loose money in the long run no matter what strategy you choose.

2. For the following toy poker game below I will prove that the OTG strategy gives a positive reward against a non-optimal opponent. Based on this example you can then guess that the OTG strategy for the limit Hold'em also gives a positive reward against a non-optimal opponent:

Toy example:

Each of the two players independently uniformly at random get either a J or a Q. 
They always pay 1 chip ante to play the game. 

The first player can either bet 1 chip or call for an immediate showdown. If the 
first player bets, the second player can either fold or call for a showdown.

Several hand play examples (p1 = player 1, p2 = player 2):

p1 hand | p2 hand | p1 action | p2 action | reward for p1
Q       | J       | raise     | call      | +2 chips (1 ante + 1 bet)
J       | J       | raise     | fold      | +1 chip
J       | Q       | raise     | call      | -2 chips
J       | J       | raise     | call      |  0 chips (pot split)

The strategy of player 1 can be summarized using these variables:

q - the probability to raise having a queen.
j - the probability to raise having a jack.
(just subtract these values from 1 to get probabilities of other actions).

The strategy of player 2 can be summarized using a single variable f denoting the probability to fold when he has a jack (if he has a queen there is never a reason to fold).

If you do the maths, you will arrive at the following expected reward equation for player 1:

(q - j)(1 - f)

What does this equation tell us about the OTG play? Note that if q=1, j=0, the expected reward for the first player is always maximized. If you set f=1 for the second player, his expected reward is always 0, thus he can never be exploited.

Thus the Nash-equilibrium strategy pair (i.e. the OTG strategy) for players 1 and 2 is

([q, j], [f]) = ([1, 0], 1), giving the expected reward of 0 for both players.

But what happens if a player 2 plays non-optimally and does not fold his J all the time? Then the expected reward for player 1 is strictly positive and thus the OTG strategy of player 1 also exploits the weakness of player 2.