Reviewing My First 50 Houou Games With Naga
My Naga Stats (50 games) Naga Stats Don't Matter? When you feed a game log into the Naga AI, it gives you 3 summary metrics: match%, Naga rating, and bad move%. The match% is how many of Naga's moves you match exactly. The Naga rating gives you 100% credit for matched moves, and partial credit for moves that it considers but aren't the top choice. The bad move% penalizes every move that Naga does not consider making. Average Match Rate/Bad Move Rate by Tenhou Rank (2020 data) Good metrics are correlated with mahjong performance, but they can be misleading. For example, in its last 100 games, the 9-10 Dan AI Suphx had a match% of 74.4, an average Naga Rating of 86.3, and a bad move rate of 7.1%, stats comparable to the average 7 dan in 2020. Tencent's new AI LuckyJ hit 10 Dan with bad move rates of >10% in many games. The main reason Naga rating is imperfect is because Naga assigns probabilities to moves, when we really