-
-
Notifications
You must be signed in to change notification settings - Fork 195
Playing strength
Fairy-Stockfish outperforms most chess variant engines, except for variants where many highly specialized engines exist (e.g., Shogi), and is the strongest engine in a variety of variants such as Janggi, Sittuyin, Capablanca chess, etc. For some variants NNUE networks are available that can further improve playing strength, see the patreon page and this spreadsheet. All statements below however refer to usage with the built-in classical handcrafted evaluation, i.e., without NNUE.
For standard chess, functionality is almost identical with official or multi-variant Stockfish, but the slowdown (>2x) due to overhead for fairy pieces and variants leads to >100 Elo weaker performance. When using NNUE the performance difference is lower, since the variant code has much less impact on NNUE than on classical evaluation. Actually, NNUE evaluation even is faster than classical, which is why Fairy-Stockfish uses pure NNUE instead of hybrid evaluation.
For the variants supported by lichess and multi-variant Stockfish (Crazyhouse, antichess, 3check, etc.), playing strength is on par (+-200 Elo) with multi-variant Stockfish, and above other variant engines. Strong NNUE networks are available for king of the hill, atomic, 3check, and racing kings.
For chess variants with fairy pieces (e.g., shatranj, makruk, sittuyin, etc.), playing strength is above all other known multi-variant engines (e.g., Sjaak II, FairyMax, Nebiyu) and at least on par with the best (non-Stockfish) engines in the respective variant, e.g., Bilis (makruk) or Shokidoki (minishogi), except for Shogi and Xiangqi, see below.
For Janggi the playing strength of Fairy-Stockfish seems to be at least on par with professional players as well as strong engines (such as Janggidosa), see e.g. https://www.youtube.com/playlist?list=PLGE6rVolMRXZwprT9CQYz3CAvU2Y5zNnj.
Playing strength in Xiangqi is slightly above the level of Elephant Eye and Cyclone 0.55, at least on master level.
When using the same NNUE evaluation as the top engine(s), Fairy-Stockfish plays on superhuman level, only a few hundred Elo below the top engines, corresponding to around 4100 in the shogi engine rating list (old). When using the handcrafted/classical evaluation, playing strength is more than 1000 Elo lower, estimated around 2700. Compared to human players this should be on top amateur to professional level.
In shogi variants (minishogi, euroshogi, etc.), it seems to be the strongest available engine, e.g., being 2x winner of the UEC cup for Minishogi. Furthermore, it is able to practically solve small-sized shogi variants such as dobutsu, micro, and kyoto shogi.
When no statistical uncertainty is specified, the estimate is derived from very few games (<100).
For variants marked with * a stronger NNUE network exists, see the variant NNUE overview, but the given test results always are for classical/handcrafted evaluation.
| variant | relative elo | reference engine |
|---|---|---|
| chess | -109.26 +-9.7 * | MV-SF |
| crazyhouse | -83.94 +-15.5 | MV-SF |
| giveaway | +76.06 +-14.2 | MV-SF |
| atomic | -268.47 +-17.3 * | MV-SF |
| 3check | +278.24 +-20.3 * | MV-SF |
| king of the hill | -87.63 +-15.0 * | MV-SF |
| racing kings | -32.05 +-12.1 * | MV-SF |
| horde | -132.54 +-16.3 | MV-SF |
| losers | +53.58 +-21.0 * | MV-SF |
| extinction | +600.90 +-44.5 | MV-SF |
| placement | +748.80 +-68.8 | MV-SF |
| seirawan | -109.83 +-19.4 | Seirawan-Stockfish |
| shatranj | -0.69 +-12.4 | Shatranj-Stockfish |
| makruk | +230.16 +-16.7 | Makruk-Stockfish |
| makruk | +300 +-100 | Bilis v1.0 |
| makruk | +400 +-100 | NebiyuChess 1.45 |
| makruk | >+400 | Sjaak II |
| shatranj | >+400 | Tiyaga v1.0 |
| shatranj | >+400 | Sjaak II |
| shatranj | >+400 | NebiyuChess 1.45 |
| ASEAN | >+400 | Sjaak II |
| sittuyin | >+400 | Sjaak II |
| shatar | >+400 | Sjaak II |
| losalamos | +600 | Sjaak II |
| losalamos | +700 | NebiyuAlien 1.45 |
| minishogi | >+400 | Sjaak II |
| minishogi | +400 | Crazywa |
| minishogi | +250 | Shokidoki ICGA15 |
| minishogi | +250 | Lima v2-00 |
| minishogi | +200 | Lima v4 |
| euroshogi | >+400 | Sjaak II |
| euroshogi | +400 | Shokidoki ICGA15 |
| breakthrough | >+400 | GameMaster 2.0 |
For variants that are also supported by the normal version (i.e., variants on boards <= 8x8), using the version for large boards decreases playing strength by 50-200 Elo due to lower speed and a few functional differences. Playing strength estimates for variants with large boards are given below.
| variant | relative elo | reference engine |
|---|---|---|
| xiangqi | +100 | Cyclone 0.55 |
| xiangqi | +100 | Elephant Eye 3.31 |
| xiangqi | +300 | Sjaak II |
| xiangqi | >+500 | MaxQi |
| shogi | -400 * | apery wcsc26 |
| shogi | 0 * | Gikou 2 D9 |
| shogi | +300 * | apery (no eval file) |
| shogi | +300 * | Shokidoki ICGA15 |
| shogi | >+400 * | Crazywa |
| shogi | >+400 * | Sjaak II |
| capablanca | +700 | NebiyuChess 1.45 |
| capablanca | +700 | Sjaak II |
Below is a list of mostly rough estimates about the playing strength compared to human level. Feel free to update this table if you have some information about a variant.
| variant | level |
|---|---|
| chess | superhuman |
| crazyhouse | superhuman |
| janggi | superhuman |
| xiangqi | master/grandmaster |
| shogi | master/professional |
| minishogi | superhuman |
| makruk | grandmaster/superhuman |