Suphx / Naga Haipai Efficiency - Part 2

If you haven't read it yet, check out Part 1 of the series.

Inefficient But Low Risk Yaku

Chanta, Honitsu, Chiitoitsu, and Toitoi hands can be some of the most challenging to play. Often times, we want to go for one of these yaku while keeping standard hands available. These two goals are often incompatible - middle tiles are the most efficient tiles, but cannot be used for chanta, do not help for honitsu of a different suit, and are bad pair candidates for chiitoi or toitoi. At what point do we force a yaku and toss the efficient middle tile?

These hands are also the most flexible in terms of dealin risk management. We often keep many honor tiles when going for these hands, making them low risk, but also low winrate hands. The control we have over our dealin risk is a blessing and a curse. If we overextend our hand by calling or cutting honors, we may be stuck fighting a riichi with a low winrate hand and not enough safe tiles. On the other hand, we always have the option of sacrificing winrate to reduce dealin rate by passing calls and keeping safe tiles.

Let's take a look at how the two AI approach these challenging yaku. I encourage readers to mentally choose the tile they would discard before scrolling down and looking at the AI decisions.

Chanta

We have 3 blocks for chanta and 3 honors - this is a pretty good candidate for a chanta hand. It's a low winrate hand, so going for a low risk yaku and keeping our honors is a good choice. 6s is our least efficient middle tile, having overlapping acceptance with 3s and the 89s shape.

Suphx and Naga : 6s

Notice the decisions that were made last turn. We passed the 9p call with 3 chanta blocks, keeping our hand flexible and safe. We then drew our yakuhai pair forming a 4th block, and chose to cut 3s over 4p, which connects to dora and keeps honitsu open.

At this point, we consider cutting 1m and 4p. The yakuhai pair draw increased our winrate significantly, so I would cut 1m. This gives up some chanta efficiency for standard hand efficiency. Chanta is still possible with the two once cut yakuhai in our hand, but is no longer a priority. Our path for honitsu has also closed with the 7s draw - it's hard to imagine a sequence of draws where we cut the whole 789s block.

Suphx: 4p

Naga: 1m

Comment: Playing efficiently with a yakuhai pair is a common Naga pattern.

In the last turn, we cut 4 from 12(4) with 4 chanta blocks and a possible 123 sanshoku. We saw the same decision in the first example. This is a pretty common pattern for chanta hands.

We just drew our 5th chanta / junchan block, and are deciding between 5s or a yakuhai. If we want to open our 2-shanten hand for junchan + ssk + dora 1, we would end up cutting 5s anyways. Keeping the yakuhai is safer, and when paired up, is more likely to be useful if we call and replace the 89m penchan.

Suphx and Naga : 5s

We have 4 chanta / junchan blocks and consider the following candidate discards: 9m, 3s, 5p, and West. 5p forces chanta (and dora) but gives up the 4p draw, sacrificing winrate. 9m is the most efficient discard but gives up the valuable junchan yaku. West is similar to 9m but sacrifices yakuhai potential. 3s is a middle tile that can form a 5th block for junchan, and possibly sanshoku. For the ability to open the hand and add value, 9m and 3s are too valuable to discard. Between 5p and West, I am not sure what the correct discard is. Let's revisit this hand after some more analysis...

Suphx: West

Naga: 5p (~60% confidence v.s. West)

Tangent - Naga Simulations

The above hand is a pretty complicated, with many potential discards trading off value, winrate, and risk. To supplement my brief analysis of the hand, I ran a Naga simulation to quantify the pros and cons of discarding 5p v.s. West. For the simulation, the hand in the screenshot serves as the starting hand for the dealer. In each simulation, every other player is a Naga bot which gets dealt random tiles and plays the best Naga move until the end of the round. I ran 10K simulations with the round set to East 1 and all scores set to 25K. Because simulations require the player to be dealer, I replaced the West Yakuhai with the White Dragon. Here is a summary of the important metrics:

Because Mahjong is a game of high variance, we can't definitively tell which discard is better, even with 10K simulations - all of the confidence intervals for important metrics overlap. However, we do see that the round EV, win rate, and dealin rates are all similar. Cutting 5p increases call rate, chanta rate, and junchan rate considerably compared to cutting the yakuhai.

So let's say we cut West in the last round. Should we call 8s to open our hand? With a big lead, I think it makes more sense to keep the hand closed and reduce risk.

Suphx: Pass

Naga: Call (~80% score)

Comment: I find it interesting that Naga makes the safer 5p discard and then the riskier call, whereas Suphx makes the riskier west discard but keeps the hand closed. From the simulations, the Naga winrate of this hand is ~19% while the dealin rate is ~15%. If we play the hand closed like Suphx, we would expect to see a lower winrate and dealin rate than these numbers. Rethinking the initial discard, if we do not plan on opening the hand for chanta / junchan, we should keep the 5p to maximize our closed winrate.

Comment 2: Notice how in the above simulation, we observe a 35% call rate when cutting the yakuhai and a 45% call rate when cutting 5p. Since Suphx does not open this hand, even with an infinite amount of simulations, we would not be able to evaluate the yakuhai cut => closed hand line. In reinforcement learning, exploration vs exploitation is a common tradeoff, where the AI can choose to make decisions it currently thinks are the best, or explore decisions that it would not normally make. While Naga is certainly trained with exploration in mind, this simulation strictly follows Naga's current strategy.

Honitsu

We have 3 blocks for honitsu with a double wind pair, a completed sequence, and a ryanmen. If we draw a second dora, we should go for a fast mangan with double wind yakuhai + dora 2, so let's keep the off-suit middle tiles for now. It is best to cut 9p or 9s, keeping the guest wind for the chance of making a 4th honitsu block.

Suphx and Naga : 9s

We now have 4 honitsu blocks with great shape. We consider cutting the guest wind versus a middle off-suit tile. Since our shape is so good and the honitsu path is worth 4x more than the standard hand, we should sacrifice efficiency for value and cut 4p or 4s. Cutting 4p is marginally better since 4s is half suji against one player, making it slightly safer.

Suphx and Naga : 4p

With 4 honitsu blocks including a yakuhai pair, forcing honitsu is a standard play. We should cut 8s, keeping our 12 total tiles of acceptance for a 5th block among the honor tiles.

Suphx and Naga : 8s

A minor decision - we can cut the more dangerous 8s, or instead cut 1s to hide our honitsu hand and increase our chance of a call.

Suphx: 1s

Naga: 8s

We have 3 good honitsu blocks including a yakuhai pair. We also have 3 middle tiles and a dora in the other suits. We can cut South to keep a normal yakuhai hand open, or cut 4m to increase our honitsu chance (4m has overlapping acceptance with 7m, making it worse than 5p). Keeping the middle tiles for a fast hand may be more favorable since the scores are close.

Suphx: South

Naga: 4m (~60% confidence)

Naga Simulation (East 1, 25K pts, South replaced with Green Dragon, West replaced with White Dragon):

According to the simulation, cutting the yakuhai is higher winrate, but very slightly lower expected value. Cutting the middle tile is lower winrate, but higher expected value, likely due to the increase in honitsu proability from 13% to 48%.

Chiitoitsu / Toitoi

We have 3 pairs and 3 blocks for chanta. It is also possible to make the rare triple 9 triplet yaku. With a yakuhai draw, we speed up our hand drastically and build towards chanta, chiitoi, toitoi, or yakuhai. We should definitely cut a middle tile for this bad shape hand. The 4m has overlapping acceptance with 7m, but the 6s has overlapping acceptance with the 9s pair. I think cutting the 6s is better because drawing 7s or 8s is still useful for chanta / ssk.

Suphx: 4m

Naga: 6s

We wouldn't usually consider calling from 3 pairs / 3 chanta blocks, but the triple 9 triplet yaku adds value and makes the 35p kanchan potentially useful. With this lead, I think it makes more sense to pass though.

Suphx: pass

Naga: call, cut 8p (~70% score)

Comment: For call decisions, Naga assigns a value from 0 to 100. If the value is under 50, it passes, otherwise it calls.

We have a 4 pair hand, so we need to keep in mind chiitoi and toitoi outcomes. We also have a dora floater - a standard NAGA pattern is to maintain 4 pairs whenever a dora floater is involved, because any pair draw will reach chiitoi 1-shanten with at least 4 han. Out of the middle tiles, we can cut 7p, which loses acceptance due to the 8p triple, or 6m, which is more dangerous and harder to call if it pairs up. As players make more discards in the upcoming turns, 7p is more likely to come out due to the 8p in our hand blocking many shapes involving 7p from forming, giving us more information on how live this tile is in upcoming turns. It is also slightly safer. Therefore, I would rather keep 7p as a better chiitoi / toitoi floater.

Suphx: 6m

Naga: 7p (call 3p ~55% score, pass 8s ~45% score)

Comment: Check out the toitoi dash article on mahjong.guide for guidelines on when to call for toitoi. I would cut 9m in the second hand in the article though.

Comment 2: I would expect Suphx to pass all calls for this 4 pair hand to reduce risk. Naga calling only 3p makes sense because it is the hardest pair to call out of the 4 pairs. I think the dora tanki open toitoi is too risky for this position though.

This 4 pair hand has pretty bad pair quality, with 4p and 7s being middle tile pairs and 1s being once cut. This hand should be played closed for chiitoi, though with 3 honitsu blocks, opening for honitsu may be possible with the right draws. We should keep the honors, which are better for honitsu and chiitoi. Between 8p and 8m, we should keep 8m which connects to the dora.

Suphx and Naga : 8p

We still have 4 pairs, but with more kanchan shapes that allow for a normal hand to form. We can cut 6p or one of the two yakuhai. Note that cutting 6p still maintains 5 blocks. With how slow the hand is, I would rather hedge towards chiitoi and reduce risk by cutting 6p.

Suphx: 6p

Naga: Green

Comment: I ran a Naga simulation for this hand, but the outcomes were greatly affected by the turn 1 assumption that is required for simulations. Naga would call the yakuhai pair and go for honitsu, which it does not do at this board state.

As discussed above, this hand is too weak and slow to call. On turn 1 with more honitsu ukeire, Naga does make this call.

Suphx: pass

Naga: pass (~10% score)

Summary

In summary, we looked at a variety of starting hands where chanta, honitsu, and chiitoi / toitoi were potential yaku. In general, we observe that Suphx is more likely to pass calls and play closed, whereas Naga is more willing to open these hands. Both AI tended to prioritize floating yakuhai over middle tiles in low winrate hands with these yaku available.

In the next and final post of this series, we will be looking at decisions between overlapping middle tiles, 0, once, and twice cut honors.