# Checkmating One, by Using Many:
Combining Mixture of Experts with MCTS to Improve in Chess

Felix Helfenstein1 Jannis Blüml1,2,∗ Johannes Czech1, &Kristian
Kersting1,2,3,4 1 Computer Science Department, Technical University Darmstadt,
Germany
2 Hessian Center for Artificial Intelligence (hessian.AI), Darmstadt, Germany
3 Centre for Cognitive Science, TU Darmstadt, Germany
4 German Research Center for Artificial Intelligence (DFKI), Darmstadt,
Germany
<EMAIL_ADDRESS>
{blueml, johannes.czech<EMAIL_ADDRESS>These authors
contributed equally.

###### Abstract

This paper presents a new approach that integrates deep learning with
computational chess, using both the Mixture of Experts (MoE) method and Monte-
Carlo Tree Search (MCTS). Our methodology employs a suite of specialized
models, each designed to respond to specific changes in the game’s input data.
This results in a framework with sparsely activated models, which provides
significant computational benefits. Our framework combines the MoE method with
MCTS, in order to align it with the strategic phases of chess, thus departing
from the conventional “one-for-all” model. Instead, we utilize distinct game
phase definitions to effectively distribute computational tasks across
multiple expert neural networks. Our empirical research shows a substantial
improvement in playing strength, surpassing the traditional single-model
framework. This validates the efficacy of our integrated approach and
highlights the potential of incorporating expert knowledge and strategic
principles into neural network design. The fusion of MoE and MCTS offers a
promising avenue for advancing machine learning architectures.

Figure 1: Incorporating MoE into MCTS based on game phase selection. First, a
new sample is allocated in the MCTS selection phase. Next, a new leaf node of
the search tree is expanded. In the simulation phase the game state is
assigned to its designated phase. At last, the appropriate model is chosen to
evaluate the sample and to backpropagate the evaluation result in the search
tree.

##  1 Introduction

Neural Networks (NNs) are crucial to the progress of machine learning and
artificial intelligence. However, they face a significant challenge when
applied to diverse and dynamic tasks. The traditional one-for-all network
approach, while versatile, often fails to effectively address the complexities
of varied applications (Riquelme et al., 2021; Artetxe et al., 2021).

This raises the question: should we continue to rely on monolithic network
architectures, or is it time to pivot to more modular and adaptable
structures? Recent explorations (Agarwala et al., 2021) suggest that the
latter may hold greater promise. The Mixture of Experts (MoE) paradigm,
introduced in 1991 by Jacobs et al. (1991), has demonstrated remarkable
versatility across multiple domains of machine learning. Its significant role
in shaping the development of Large Language Models (LLMs) (Fedus et al.,
2022; Du et al., 2022) is particularly noteworthy. Building on this success,
we propose reintegrating MoE into the domain of chess and other board games.
MoE provide a strategic approach to breaking down complex tasks into smaller,
more manageable segments, allowing a team of specialized experts to
collaboratively tackle these challenges.

In the field of chess, a game with dynamic and strategic nuances, this
approach takes on special significance. Traditional engines like Stockfish
utilized phase-specific heuristics, but contemporary models such as AlphaZero
(Silver et al., 2018), MuZero (Schrittwieser et al., 2020), and NNUE-enhanced
Stockfish (Nasu, 2018) have moved towards a unified network approach for all
game phases. We argue that taking a nuanced approach, which recognizes the
distinct characteristics of each phase of chess (opening, middlegame, and
endgame), can result in significant performance improvements.

Our goal is to create a chess agent that can adapt its strategy to the
changing phases of the game, similar to skilled human players. We investigate
the combination of MoE with Monte-Carlo Tree Search (MCTS) in modern chess
engines, as shown in Figure 1.

Empirically, we demonstrate that our phase-specific
architecture111https://anonymous.4open.science/r/CrazyAra-3F10 outperforms
traditional single-model approaches in the game of chess, achieving a
substantial increase in playing strength by approximately 120 Elo points.

Our contributions can be summarized as follows:

  * •

We investigate various training strategies for expert models and establish
that a straightforward, separated learning regime is as effective, if not more
so, than complex alternatives.

  * •

We underscore the necessity and effectiveness of a MoE approach in MCTS for
chess and other classical board games.

  * •

Through rigorous testing, we confirm that incorporating a small number of out-
of-distribution samples in batch-wise inference does not diminish an expert’s
performance.

We start by explaining our game-phase-specific MoE architecture. Subsequent
sections present an extensive evaluation, including its performance against
traditional single-network models. Before we conclude we discuss related work
and limitations.

##  2 Combining MCTS and MoE

The MoE paradigm, an innovative approach in AI, consists of $n$ specialized
“expert networks” $m_{1},m_{2},\dots,m_{n}$ complemented by a strategic
“gating network” $G$. This network outputs a sparse $n$-dimensional vector
that effectively acts as a classifier. Each expert network is a distinct NN,
with its own parameters and architecture, yet designed to have a uniform input
and output shape across all networks.

The power of MoE lies in its ability to strategically synthesize the outputs
of these experts. Several combination strategies are explored in Yang and
Browne (2004), with averaging and the max operator being the most common:

$\displaystyle y=\sum_{i}^{n}G_{i}(x)m_{i}(x)\quad\textrm{or}\quad
y=\max_{G_{i}(x)}(m_{i}(x))$ (1)

In our work, we adopt the max operator for its clarity in selecting a single,
relevant phase, thus eliminating the possibility of ambiguous phase
assignments. This choice also brings significant computational efficiencies,
as it requires inference from only one expert model $m_{i}$ at a time.

Integrating MCTS with NNs typically involves using the model as an evaluator
within MCTS, replacing the simulation phase with a forward pass. We innovate
this process in MCTS by integrating both a gating mechanism $G$ and a curated
set of expert models. This approach, illustrated in Figure 1, not only
preserves the core structure of MCTS, but enriches it with phase-specific
evaluation capabilities.

###  2.1 Utilizing the Game Phase Definitions of Chess

Chess, a game known for its complexity and strategic depth, is traditionally
divided into three central phases: the opening, the middlegame, and the
endgame. Each phase is distinct: the opening sets the strategic foundation of
the game, where players compete for control of the board; the middlegame,
characterized by intense piece interaction and tactical play, requires keen
strategic insight; and the endgame focuses on exploiting earlier advantages to
secure victory with fewer pieces in play. These phases are not just segments
of a game, but represent critical strategic shifts that determine the course
of the game.

In our approach, we adopt the Lichess game phase definitions (cf. Appendix B)
as gating mechanism. Lichess222https://lichess.org/, an esteemed open-source
chess server, provides a well-rounded and intuitive set of criteria for phase
identification. This definition includes various factors such as board
mixedness, back-rank sparseness, and piece count, offering a holistic view of
the game’s progression. By applying these phase conditions to any given game
state, we can independently determine its phase without relying on previous
positions, thus preserving the Markov property. This intuitive method allows
us to return to previous phases if necessary, based on the current board
position. For an in-depth exploration of Lichess phase definitions, we refer
the reader to Appendix B. The distribution over the phases, using the
mentioned phase definition is visualized in Figure 2.

Figure 2: A common game phase definition in chess is to split states into
opening, middlegame and endgame. This figure presents a detailed distribution
of positions across the three game phases based on the Lichess phase
definition, aggregated by move number. The average starting moves of the
middlegame (11.89, std: 2.966) and the endgame (29.15, std: 7.441) are marked
by vertical lines.

###  2.2 Training Multiple Experts

In this paper, we explore the effectiveness of integrating MoE within MCTS
through three innovative training approaches:

#### Separated Learning

This approach involves training each of the three expert networks exclusively
on positions corresponding to their specific game phase. The objective is to
simplify the learning process by narrowing the input space to a more
manageable subspace. This focus enables each network to develop specialized
expertise, enhancing the overall parameter count without compromising
inference time. Implementation is straightforward and does not increase the
training duration. In fact, the training process can be parallelized across
experts and as such reduce training time significantly. While this method
ensures depth in phase-specific knowledge, its limitation lies in a potential
lack of understanding of transition nuances between phases. However, this is
offset by the efficiency and specificity of the learning process, ensuring
that training time remains on par with single network approaches.

#### Staged Learning

In this method, a network is initially trained on one phase, then its weights
are used as a foundation for subsequent training on the next phase. This cycle
repeats through all defined phases, with the final phase defining the
network’s expertise. This approach aligns with transfer learning principles,
allowing knowledge transfer between phases while focusing on a single phase
ultimately. Each training cycle involves resetting the optimizer state while
preserving the network weights, ensuring consistent learning dynamics. The
primary advantage is the imparted understanding of all phases, albeit with a
focus on the final phase. However, this approach requires more training time
and may encounter challenges like catastrophic forgetting.

#### Weighted Learning

This strategy involves applying different weights to the loss function for
training samples based on their game phase. It allows each expert to learn
from the entire dataset while emphasizing its designated phase.

$\displaystyle L_{\text{weighted}}=\frac{1}{B}\sum_{n=1}^{B}w_{n}L(x_{n})$ (2)

Here, $B$ is the batch size, $w_{n}$ the weight of a specific sample, and
$L(x_{n})$ the regular sample loss. To prioritize the main phase samples, we
introduce the parameter $a\geq 1.0$, which balances the weights of the main
phase samples ($w_{\text{main}}$) and the ones of all other samples
($w_{\text{other}}$): $w_{\text{main}}=a\cdot w_{\text{other}}$. The approach
ensures that the loss magnitude remains consistent across training and
evaluation by normalizing the sample weights to keep the average sample weight
close to 1.0. The test set is unweighted for fair performance comparison.
While this method requires triple the GPU time, it offers comprehensive
learning from the entire dataset, distinguished by phase-specific emphases.

Each of these approaches is designed to enhance the MoE model’s performance in
MCTS. The detailed configurations and outcomes of these training strategies
are further elaborated in Section 3.

Figure 3: MCTS in combination with MoE can be used for batch sizes greater
than 1. Each sample is assigned to its phase while allocating a new mini-batch
during the MCTS search. When a mini-batch is complete, majority vote selection
is used to select the model that contains the most samples of its associated
phase. This model is then used to evaluate all samples from this mini-batch
ensuring efficient and phase-focused processing.

###  2.3 Training MoE on Chess

In this study, we train individual “experts” to master 3 distinct sets of
tasks in chess, corresponding to its well-defined game phases. This enables
dynamic resource allocation and using specialized knowledge for each phase of
a chess game. Implementing this concept, we use three distinct expert models,
each fine-tuned to excel in one of the chess game phases: opening, middlegame,
and endgame.

While traditional MoE systems typically use a gating network to select an
expert for a given input, our approach differs by adopting a more direct
method. Given the robustness and widespread acceptance of the game phase
definitions used in this research (detailed in Section 2.1), we choose to use
a handcrafted criterion for expert selection, bypassing the need for the
additional complexity inherent in an additional gating network. This
predefined input segmentation simplifies the learning process by ensuring that
each expert is focused only on its respective phase. However, it assumes that
the game phases are accurately identified and classifiable, an aspect that is
not extensively explored in this paper. Also, for this work, we decided to
assume that each state belongs to one exact phase, not a mixture of different
phases.

By integrating this MoE into the MCTS within an AlphaZero-like approach, we
aim to improve the performance of chess AI by exploiting phase-specific
strategies to achieve a level of mastery unattainable with conventional
methods.

###  2.4 Training in Batches

While the ideal scenario is to query only the appropriate network aligned with
the current phase of the MCTS position, this approach is not applicable to
MCTS batch sizes larger than 1, as it is common to improve GPU utilization.
Larger batch sizes therefore require buffering of multiple positions prior to
neural network inference, thereby reducing the frequency of node statistics
updates.

To address this, we have developed a strategic process for handling larger
batches. When encountering a batch size larger than 1, we perform an analysis
of all positions within the batch to identify the dominant game phase. The
network of experts corresponding to this dominant phase is then applied to the
entire batch. A detailed illustration of this batch processing method can be
found in Figure 3. While this adaptation introduces the possibility of
processing some instances with a non-ideal expert network, it is important to
note that such instances are expected mainly during phase transitions. In
these scenarios, the incorrectly processed positions are still close to their
appropriate phase, ensuring minimal compromise in prediction accuracy. This
modification is essential to maximize GPU efficiency and effectively handle
larger batch sizes.

##  3 Experiments

During the entire training phase, the model is subjected to an evaluation
every 500 iterations. In addition to evaluating the currently trained expert
on the dedicated evaluation dataset, assessments are extended to all four test
sets (opening, middlegame, endgame, and no-phases). Checkpoints of the model
state are saved if the evaluation loss surpasses the best recorded loss, and
the final model is derived from the last saved checkpoint, representing
optimal performance on the evaluation set.

The backbone of our NN architecture is RISEv3.3, an evolution of the RISEv2
architecture presented in the work by Czech et al. (2020), called CrazyAra.
This architecture, a convolutional NN with residual blocks and dual output
heads (value and policy), is further improved by incorporating $5\times 5$
convolutions and the Efficient Channel Attention module (Wang et al., 2020).
For an in-depth understanding of our model’s architecture, we refer readers to
the CrazyAra wiki333https://github.com/QueensGambit/CrazyAra/wiki/. The input
and output representation of our model is based on the work of Czech et al.
(2023). The training and test datasets are curated from the Kingbase Lite 2019
database444https://archive.org/details/KingBaseLite2019, which contains over
one million games played by individuals rated at least 2200 Elo. We filter out
games shorter than five moves to ensure the reliability of the data, as these
are often either pre-arranged draws or database errors.

To increase the stability and reliability of the model, we have implemented a
loss spike recovery mechanism. If there is a significant increase in the
evaluation set loss from one cycle to the next, the model is reset to a
previous state. This is controlled by the condition $L_{\text{prev}}\cdot
s<L_{\text{curr}}$, where $s=1.5$, $L_{\text{prev}}$ is the previous
evaluation loss, and $L_{\text{curr}}$ is the current loss.

Our comprehensive evaluation includes a match of 1000 games that combines the
trained model with MCTS, similar to AlphaZero’s methodology. To objectively
measure the strength of the individual approaches, we use the Elo rating
system (detailed in Appendix C). In addition, we calculate 95% confidence
limits for Elo differences based on the number of games played and their
results. This calculation closely follows the method used in the Cutechess
repository555https://github.com/cutechess/cutechess, ensuring both accuracy
and consistency in our ratings. More information on hyperparameters, datasets,
and other technical details is provided in the supplementary materials.

###  3.1 Using MCTS with MoE

Figure 4: In staged learning, the endgame expert converges to a lower test loss compared to no-phases training. Shown is the endgame expert’s loss over the course of the training process for the staged learning approach. The metrics are evaluated on the opening (blue), middlegame (orange), endgame (green) and no-phases (black) test set. The dashed lines describe the final loss of the no phases training setup and the vertical lines indicate the points at which the training on a new data set started (initialized with the parameters of the last model checkpoint of the previous training stage). Figure 5: MCTS with MoE significantly outperforms standard MCTS. The MoE framework, utilizing the separated learning strategy outlined in Section 2.2, achieves up to 150 Elo points higher than a single ’one-for-all’ model. This elevation in performance was consistently observed across a range of experiments, involving various batch sizes from 1 to 64, and different tree complexities, measured in terms of nodes per tree.  Table 1: MoE reduces overall loss in chess using game phase specific experts. This table provides a comprehensive overview of the loss values for each expert’s final model across different approaches. Performance is evaluated over four different test sets: opening, middlegame, endgame, and no-phases. In addition to the individual expert losses, we include the loss for the resulting mixture model that includes all three experts. This mixture model determines the appropriate expert based on the phase of the current position in the test set. The weighted learning approach was analyzed with different weighting factors to emphasize the main phase over others (see Section 2.2). The table uses color coding for each column, ranging from high loss (red) to low loss (blue). Bold text highlights the most effective model within each learning process for each test set, while the best performing model in the no-phases test set is underlined, highlighting its overall superiority. Approach | Expert | Opening | Middlegame | Endgame | No-Phases
---|---|---|---|---|---
Regular Learning | - | 0.9269 | 1.5043 | 1.3523 |  missing1.2897
Separated Learning | Opening | 0.9169 | 2.4030 | 3.2433 | 2.2587
| Middlegame | 1.6064 | 1.4371 | 1.5494 | 1.5260
| Endgame | 2.0097 | 1.6605 | 1.2594 | 1.6293
| Mixture | 0.9169 | 1.4371 | 1.2594 |  missing1.2245
Staged Learning | Opening | 0.8786 | 1.8775 | 2.5501 | 1.8153
| Middlegame | 1.5966 | 1.4437 | 1.5293 | 1.5203
| Endgame | 2.0281 | 1.6748 | 1.2774 | 1.6458
| Mixture | 0.8786 | 1.4437 | 1.2774 |  missing1.2218
Weighted Learning ($a=4$) | Opening | 0.8636 | 1.5125 | 1.3780 | 1.2820
| Middlegame | 0.9661 | 1.5055 | 1.4207 | 1.3248
| Endgame | 0.9339 | 1.5041 | 1.2864 | 1.2695
| Mixture | 0.8636 | 1.5055 | 1.2864 |  missing1.2433
Weighted Learning ($a=10$) | Opening | 0.8887 | 1.6600 | 1.5389 | 1.3995
| Middlegame | 0.9955 | 1.4834 | 1.4470 | 1.3346
| Endgame | 1.0334 | 1.5547 | 1.3036 | 1.3241
| Mixture | 0.8887 | 1.4834 | 1.3036 |  missing1.2483
Table 2: MoE methods outperform our baseline “one-for-all” approach in direct comparison. This table presents an in-depth analysis of the relative Elo gains achieved by various MoE approaches over our baseline model across different batch sizes. Each value represents the average Elo gain across all configurations of nodes and move times for the specified batch size. The data is visually coded for clarity, with the best performances in blue and the least effective in red. The rightmost column aggregates the average Elo gain across all 55 experiments conducted for each method. Bold text highlights the most successful model within each approach. | Batch Size
---|---
Approach | 1 | 8 | 16 | 32 | 64 | Average
Separated Learning | 106.89 | 123.45 | 126.81 | 124.36 | 129.49 | 122.20
Staged Learning | 111.68 | 120.81 | 122.49 | 125.00 | 125.55 | 121.11
Weighted Learning ($a=4$) | 25.61 | 23.20 | 23.52 | 22.28 | 21.31 | 23.18
Weighted Learning ($a=10$) | 50.41 | 52.42 | 56.33 | 63.55 | 56.51 | 55.84

In our study, we exhaustively trained an expert model for each of the three
game phases (cf. Section 2.1) using different training methodologies described
in Section 2.2. We then evaluated the performance of these expert models, both
individually and as part of the MoE ensemble, against our extensive test set.
Our results, detailed in Table 1, reveal that the MoE approaches consistently
outperform the baseline model — CrazyAra using RISEv3.3 (Czech et al., 2020) —
in terms of combined losses. The baseline model was trained under similar
conditions, but without phase-specific distinctions.

The separated learning and staged learning approaches show an average
improvement of about 0.06 over our regular learning approach, while the
weighted learning regime shows an improvement of about 0.04. This improvement
translates into a significant increase in playing strength, ranging from 55 to
122 Elo points (cf. Table 2 and Figure 26) against the baseline. An exemplary
training process for staged learning is shown in Figure 22.

The separated and staged learning methods significantly outperform our
weighted learning approach. Interestingly, a higher factor ($a=10$) in our
weighted learning strategy resulted in more pronounced Elo gains than a lower
factor ($a=4$), despite similar overall loss values for both settings. It
suggests that further increasing the specialization factor could potentially
yield even more competitive results, similar to or better than other
approaches. However, it is important to note that using all samples for
training in the weighted and staged learning approach triples the training
time compared to separated learning, where each expert is trained on only a
third of the dataset.

In summary, our MoE methods clearly outperform the baseline model, not only in
terms of training loss, but also in terms of actual playing strength, as
evidenced by a substantial relative Elo gain of 122.20. To our knowledge, this
is the first instance where such a combination has been successfully
implemented and empirically validated.

###  3.2 Comparing different Mixture Models

Figure 6: Expert models are especially helpful in middle- and endgame. This
figure shows the relative Elo gains achieved by selectively deploying
individual phase experts within Monte-Carlo Tree Search (MCTS), compared to
using a baseline network for all phases. The experts, derived from our
separated learning approach, exhibit notable strengths in both the middlegame
and endgame phases of chess. The experiments were conducted using a consistent
batch size of 64 for all matches.

Our analysis, detailed in Table 1, indicates that the middlegame and endgame
models contribute significantly to the overall effectiveness of our MoE
framework. To quantify the individual impact of each expert model on the Elo
gain of MoE, we organized a series of tournaments. In these tournaments, a
single expert model was used for its respective phase, while the baseline
model was used for the other phases. The results, illustrated in Figure 6,
reveal that the middlegame and endgame networks contribute comparably to the
overall performance improvement. However, the use of the opening net either
slightly hinders performance or shows no significant improvement over the
baseline model. Several factors may contribute to this observed behavior.
First, the opening play in our expert dataset may lack sufficient variability,
potentially leading the expert model to overfit specific lines rather than
gaining a broad understanding of opening strategies. This narrow focus may
limit the ability of the opening model to generalize across the wide range of
opening scenarios encountered in play. Secondly, the opening expert’s relative
underperformance might stem from its limited exposure to terminal game states
during training. This limited perspective could hinder its ability to evaluate
opening positions in the broader context of the game’s possible trajectories.

Finally, there is a notable difference between the opening suites used in our
validation tournaments and those in the training set. Our test tournament used
a more randomized suite of openings generated by a computer algorithm, as
opposed to the expert gameplay-based common openings in the training set. This
mismatch between training and testing conditions may explain the reduced
effectiveness of the opening network in real game scenarios. These findings
not only highlight the nuanced contributions of each expert model to the MoE
framework, but also underscore the importance of dataset diversity and
training methodology in developing robust, stage-specific AI strategies for
chess.

###  3.3 Influence of the Game Phase Definition

Figure 7:  Game phase definitions strongly influence the MoE performance. Our
analysis shows that simplistic approaches to phase classification, such as
relying only on move counts, lead to suboptimal phase categorization and
consequently to inferior performance. This effect increases with the number of
experts. In contrast, adopting the Lichess phase definition leads to a
significant performance improvement.

In our research, we initially adopted a traditional view of chess, dividing
the game into three distinct phases with based on classical rules defining the
game phases. This raises intriguing questions like how does varying the phase
subdivision, as with that the number of experts, impact the overall result and
can we simplify the phase definition using only the move counter. To explore
these, we divided our dataset into 2, 3, 4, and 5 segments purely based on the
move counter, generating similar sized phase datasets. The Elo comparison for
different numbers of partitions is shown in Figure 7. It becomes apparent that
using more phases does not guarantee better results and that the number of
experts, i.e., phases, has to be adapted to the specific problem definition.
Further, a logical semantic phase selection criterion is far superior to
relying on a simple feature, like the move counter, when comparing the 3
phases Lichess agent with the 3 phases move counter agent. To conclude, simple
features, like the move counter in chess, alone are not a sufficient game
phase definition. To define a good gating mechanism, a clustering over
concepts is necessary.

##  4 Limitations

In this study, we have advanced the use of the MoE framework in chess AI and
demonstrated its significant potential. However, like any pioneering approach,
it comes with its own set of challenges and future prospects.

A primary limitation of MoE is its susceptibility to overfitting, especially
in scenarios with smaller datasets. This inherent risk requires careful data
management and model training strategies. In addition, MoE has traditionally
struggled to efficiently handle larger batch sizes due to the selective
activation of experts. However, our framework proves resilient to this problem
through the use of a majority voting mechanism for all samples in a mini-
batch. By strategically mixing a few outlier samples from different phases, we
have achieved a model that performs robustly across different batch sizes. In
particular, our model maintained consistent performance with batch sizes
ranging from 1 to 64, but it is unclear how well this strategy works for batch
sizes beyond 512. Exploring more sophisticated methods, such as dividing the
batch into multiple intermediate segments or even merging the insights of
multiple experts, could potentially refine our decision-making process.

The requirement for increased VRAM, due to the necessity of loading all
experts into memory, is a technical consideration with MoE. However, in the
context of MCTS and with a limited number of experts, this has not posed a
significant constraint in our experiments.

##  5 Related Work

In advancing the application of MoE to chess, our work builds upon ongoing
developments in both MCTS and RL techniques, particularly in the context of
AlphaZero inspired methods. The integration of MoE into deep learning, as
demonstrated by Shazeer et al. (2017), has shown promising results, especially
in the context of LLMs and transformer-based architectures, driven by the
growing complexity of modern datasets (Fedus et al., 2022; Du et al., 2022;
Jiang et al., 2024). However, the combination of MoE with MCTS, in a manner
similar to AlphaZero (Silver et al., 2018), represents a novel approach in the
field.

Our inspiration to adapt MoE to chess is further strengthened by the work of
Dobre and Lascarides (2017) and McIlroy-Young et al. (2022). The former
demonstrates the utility of MoE in complex games such as Settlers of Catan,
using expert models trained on diverse datasets. The latter demonstrates the
potential of MoE to develop specialized chess models that can be fine-tuned to
individual player behavior. These works underscore the diverse applicability
of chess as a research domain, given its rich history, extensive data corpus,
and active community engagement.

The shift from heuristic-based models to NN approaches, as exemplified by
Silver et al. (2018); Nasu (2018), has significantly improved performance, but
at the cost of interpretability and adherence to human-centered chess theories
(Pálsson and Björnsson, 2023; McGrath et al., 2022). The use of game phase
definitions in evaluation, a concept familiar to chess players and heuristic-
based engines, found its place in early RL explorations in chess, such as the
work of Block et al. (2008). We also took inspiration to use game phase-
specific NNs for chess by the work of
dkappe666https://github.com/dkappe/leela-chess-weights, showing this can have
improved performance in endgame scenarios. His work has also been integrated
into Daniel Shawul’s NN based chess engine
Scorpio777https://github.com/dshawul/Scorpio/releases/tag/3.0, demonstrating
the practical applicability of such approaches. In contrast to our work, their
implementation falls short in delivering a comprehensive experimental
evaluation. Additionally, their approach relies on a rather simplified
framework, employing material-based criteria to ascertain the currently active
phase. Based on the experiments in Section 3.3 we believe that improving the
game phase definition will also improve this approach even further. Our
research aims to bridge this gap by reintegrating phase-specific insights with
advanced NN methods to enrich the strategic depth and performance of chess AI.

##  6 Conclusion

Our exploration of modern chess AI with the MoE framework has resulted in
significant progress, as evidenced by an impressive gain of approximately 120
Elo points. This achievement underscores the efficacy of employing phase-
specific models tailored to the unique strategic requirements of each phase of
a chess game. The MoE framework we developed and implemented is characterized
by its ease of implementation and remarkable impact on performance. By
separating the training process into distinct subsets, we have successfully
cultivated specialized expertise within each expert model, thereby enhancing
the overall performance of the system. In addition, our framework has
demonstrated considerable robustness, effectively handling a range of
minibatch sizes from 1 to 64 during the search process. This is achieved
through our reliable majority voting mechanism, which selects the most
appropriate expert for each game phase.

Looking ahead, there are exciting opportunities to further refine and develop
our MoE framework. One particularly promising avenue is the development of a
learnable gating network. This dynamic approach could replace our current
method of using fixed, hand-crafted phase definitions, providing greater
adaptability and precision in expert selection.

## Acknowledgment

The authors would like to thank Cedric Derstroff and Florian Busch for their
diligent proofreading, which contributed significantly to the refinement of
this manuscript. The authors would also like to acknowledge the use of GPT
4.0, whose advanced language processing capabilities were instrumental in
improving the linguistic quality of this paper. Finally, we are grateful for
the unique dataset used in this research, which consists of games played by
human grandmasters, a factor that has been crucial in underpinning the
validity and depth of our analysis.

## Ethics Statement

In conducting this research, it is important to us to emphasize the ethical
considerations associated with data collection, model development, and
experimentation. Throughout the study, ethical guidelines were strictly
adhered, ensuring the respectful use of data of any entities involved, like
the usage of the Kingbase chess dataset, a collection of chess games played by
human players, for the purpose of training and evaluating artificial
intelligence models. We acknowledge the potential for biases in the dataset,
such as variations in time periods, regions, or player demographics, however
were not able to discover any in our research. We provide algorithm and models
to make our results reproducible and transparent. We also encourage other
researchers to use our framework in their work.

## References

  * Agarwala et al. [2021] Atish Agarwala, Abhimanyu Das, Brendan Juba, Rina Panigrahy, Vatsal Sharan, Xin Wang, and Qiuyi Zhang.  One network fits all? modular versus monolithic task formulations in neural networks.  In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
  * Artetxe et al. [2021] Mikel Artetxe, Shruti Bhosale, Naman Goyal, Todor Mihaylov, Myle Ott, Sam Shleifer, Xi Victoria Lin, Jingfei Du, Srini Iyer, Ramakanth Pasunuru, Giridhar Anantharaman, Xian Li, Shuohui Chen, Halil Akın, Mandeep Baines, Louis Martin, Xing Zhou, Punit Singh Koura, Brian O’Horo, Jeff Wang, Luke Zettlemoyer, Mona T. Diab, Zornitsa Kozareva, and Ves Stoyanov.  Efficient large scale language modeling with mixtures of experts.  In Conference on Empirical Methods in Natural Language Processing, 2021.
  * Block et al. [2008] Marco Block, Maro Bader, Ernesto Tapia, Marte Ramírez, Ketill Gunnarsson, Erik Cuevas, Daniel Zaldivar, and Ral Rojas.  Using reinforcement learning in chess engines.  Research in Computing Science, 35:31–40, 2008.
  * Coulom [2006] Rémi Coulom.  Efficient selectivity and backup operators in monte-carlo tree search.  volume 4630, 05 2006.
  * Czech et al. [2020] Johannes Czech, Moritz Willig, Alena Beyer, Kristian Kersting, and Johannes Fürnkranz.  Learning to play the chess variant crazyhouse above world champion level with deep neural networks and human data.  Frontiers Artif. Intell., 2020.
  * Czech et al. [2023] Johannes Czech, Jannis Blüml, and Kristian Kersting.  Representation matters: The game of chess poses a challenge to vision transformers, 2023.
  * Dobre and Lascarides [2017] Mihai Sorin Dobre and Alex Lascarides.  Combining a mixture of experts with transfer learning in complex games.  In AAAI Spring Symposia, 2017.
  * Du et al. [2022] Nan Du, Yanping Huang, Andrew M. Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, Barret Zoph, Liam Fedus, Maarten P. Bosma, Zongwei Zhou, Tao Wang, Yu Emma Wang, Kellie Webster, Marie Pellat, Kevin Robinson, Kathleen S. Meier-Hellstern, Toju Duke, Lucas Dixon, Kun Zhang, Quoc V. Le, Yonghui Wu, Zhifeng Chen, and Claire Cui.  Glam: Efficient scaling of language models with mixture-of-experts.  In ICML, volume 162 of Proceedings of Machine Learning Research, pages 5547–5569. PMLR, 2022.
  * Fedus et al. [2022] William Fedus, Barret Zoph, and Noam Shazeer.  Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.  J. Mach. Learn. Res., 23:120:1–120:39, 2022.
  * Jacobs et al. [1991] Robert A. Jacobs, Michael I. Jordan, Steven J. Nowlan, and Geoffrey E. Hinton.  Adaptive mixtures of local experts.  Neural Comput., 3(1):79–87, 1991.
  * Jiang et al. [2024] Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed.  Mixtral of experts, 2024.
  * Kocsis and Szepesvári [2006] Levente Kocsis and Csaba Szepesvári.  Bandit based monte-carlo planning.  In ECML, volume 4212 of Lecture Notes in Computer Science, pages 282–293. Springer, 2006.
  * McGrath et al. [2022] Thomas McGrath, Andrei Kapishnikov, Nenad Tomašev, Adam Pearce, Martin Wattenberg, Demis Hassabis, Been Kim, Ulrich Paquet, and Vladimir Kramnik.  Acquisition of chess knowledge in alphazero.  Proceedings of the National Academy of Sciences, 119, 2022.
  * McIlroy-Young et al. [2022] Reid McIlroy-Young, Russell Wang, Siddhartha Sen, Jon M. Kleinberg, and Ashton Anderson.  Learning models of individual behavior in chess.  In Aidong Zhang and Huzefa Rangwala, editors, KDD ’22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, August 14 - 18, 2022, pages 1253–1263, 2022.
  * Nasu [2018] Yu Nasu.  Efficiently updatable neural-network-based evaluation functions for computer shogi.  The 28th World Computer Shogi Championship Appeal Document, 185, 2018.
  * Pálsson and Björnsson [2023] Adhalsteinn Pálsson and Yngvi Björnsson.  Unveiling concepts learned by a world-class chess-playing agent.  In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, pages 4864–4872, 2023.
  * Riquelme et al. [2021] Carlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, André Susano Pinto, Daniel Keysers, and Neil Houlsby.  Scaling vision with sparse mixture of experts.  In Marc’Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pages 8583–8595, 2021.
  * Schrittwieser et al. [2020] Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, Timothy P. Lillicrap, and David Silver.  Mastering atari, go, chess and shogi by planning with a learned model.  Nat., 2020.
  * Shazeer et al. [2017] Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc V. Le, Geoffrey E. Hinton, and Jeff Dean.  Outrageously large neural networks: The sparsely-gated mixture-of-experts layer.  In ICLR (Poster). OpenReview.net, 2017.
  * Silver et al. [2018] David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, and Demis Hassabis.  A general reinforcement learning algorithm that masters chess, shogi, and go through self-play.  Science, 362(6419), 2018.
  * Wang et al. [2020] Qilong Wang, Banggu Wu, Pengfei Zhu, Peihua Li, Wangmeng Zuo, and Qinghua Hu.  Eca-net: Efficient channel attention for deep convolutional neural networks.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11534–11542, 2020.
  * Yang and Browne [2004] Shuang Yang and Antony Browne.  Neural network ensembles: combining multiple models for enhanced performance using a multistage approach.  Expert Syst. J. Knowl. Eng., 2004.

##  Appendix A Data Availability Statement

All data and code, needed to reproduce our experiments can be found online.
The Kingbase Lite Dataset can be downloaded from
{https://archive.org/details/KingBaseLite2019}. To recreate our dataset split,
use the code provided by us in
https://anonymous.4open.science/r/CrazyAra-3F10, following the instructions in
the readMe file. To test our approach, we provide some of the models used in
this paper
anonymized888https://drive.google.com/drive/folders/1d8CoQBieNqeEomhbYfI_TifZUbCFyS3O?usp=sharing.
These models will be added to the official github repository after acceptance.

##  Appendix B Lichess game phase definitions

Lichess is an open-source chess server for online play, study, and analysis.
State-of-the-art chess engines power their analysis section, which is why
players of all skill levels, from beginner to master, use them on a daily
basis. Submitting a game for engine analysis results in getting a report of
the game development, including a separation of the game in the three typical
phases to teach people where they went wrong and how they could improve. The
site uses a more sophisticated system to determine game phases by
incorporating several transition criteria.

Endgame definition. According to the Lichess implementation, a position
belongs to the endgame if the total count of major and minor pieces (queens,
rooks, bishops, and knights) is less than or equal to 6. Note that while this
resembles a material count criterion, it does not use different relative
values for the pieces and instead values them all equally as 1. A potential
reason for this approach could be that the complexity of a position is not
tied to the relative value of its pieces but rather to their total amount.
Furthermore, the pawn count is irrelevant to the decision.

Middlegame definition. A position counts towards the middlegame if it does not
qualify as an endgame position and if one of the following three criteria is
fulfilled. The number of major and minor pieces is less than or equal to 10,
the backrank of at least one player is sparse, or the total mixedness score of
the position is bigger than 150. Here, backrank sparseness is defined as
having less than four total pieces on rank 1 (for white) or 8 (for black),
including the king. The mixedness score describes how close black and white
pieces are to each other. It is calculated by going through all two by two
squares of the chess board, starting from the square a1, b1, a2, b2 and ending
at the square g7, h7, g8, h8. For each of those two by two squares, we count
the number of black and white pieces inside it and assign a score based on the
result and the square’s location. We then sum up all square scores to get the
final mixedness score of a position. The exact implementation can be found in
the Lichess repository999https://github.com/lichess-
org/scalachess/blob/master/src/main/scala/Divider.scala.

Opening definition. All remaining positions are classified as opening
positions.

Using the previously described phase definitions may lead to transitions to
previous phases (e.g., going back to the opening because the mixedness score
has increased again). Therefore, to do a strict separation into three
sections, Lichess forbids such transitions and only allows transitions to
later phases.

###  B.1 Resulting Dataset Statistics

After defining the game phases as in Section B, we examine the distributions
of input and output in the resulting datasets for each phase.

Figure 12 shows the distribution of game outcomes in our training set. Every
game contributes to the opening phase data, with 34.42% games won by White,
40.36% draws, and 25.22% Black wins. Notably, games extending into the
middlegame and endgame phases have a lower draw rate (38.51% and 36.09%,
respectively) and a higher rate of decisive outcomes.

Figure 8: The mixedness scores (according to the definition in Section B) of
chess game aggregated by move number. The data is based on all positions of
the chess test set. Figure 9: The average number of major and minor pieces
left aggregated by move number. The data is based on all positions of the
chess test set. Figure 10: The percentage of positions with a sparse backrank
by move number. The data is based on all positions of the chess test set.
Figure 11: The distribution of starting moves of each phase aggregated by move
number. The average starting moves of the middlegame (11.89 with an std of
2.966) and the endgame (29.15 with an std of 7.441) are added as vertical
lines. The data is based on all games of the chess test set. Figure 12: Game
result distributions in the chess training set, calculated for games reaching
each phase.

Figure 13 illustrates the distribution of game phases. While 96.81% of the
games reach the middlegame, only 67.85% reach the endgame. Nevertheless, the
number of endgame positions (31.63%) exceeds that of openings (28.67%),
because endgames tend to last longer.

(a) Games by phase

(b) Positions by phase

Figure 13: Distribution of games and positions across phases in the chess
training set.

Table 3 gives a comprehensive overview of our datasets, the full dataset
contains 1,112,647 games and 91,413,951 positions. Table 4 summarizes the
number of games contributing to each phase, highlighting the variance in phase
coverage.

Dataset/Phase | No-Phases | Opening | Midgame | Endgame
---|---|---|---|---
Train Chess | 91,413,951 | 26,212,273 | 36,287,651 | 28,914,027
Val Chess | 79,042 | 24,566 | 29,333 | 25,143
Test Chess | 85,114 | 24,938 | 31,364 | 28,812
Table 3: Overview of dataset sizes by phase, showing the number of positions. Dataset/Phase | No-Phases | Opening | Midgame | Endgame
---|---|---|---|---
Train Chess | 1,112,647 | 1,112,647 | 1,077,136 | 754,899
Table 4: Number of games contributing to each phase dataset, indicating phase
coverage.

##  Appendix C Elo Ratings

The Elo rating system, invented by Arpad Elo, is used to measure relative
skill differences between players of a game. Everyone starts with the same
arbitrary starting value, and the ratings are adjusted from that point on
based on the outcome of finished games. Winning games increases the Elo
rating, while losing decreases it. Using the rating of two players $A$ and
$B$, it is possible to calculate the expected score of player $A$ (and player
$B$ by replacing $R_{\mathrm{B}}-R_{\mathrm{A}}$ with
$R_{\mathrm{A}}-R_{\mathrm{B}}$):

$E_{\mathrm{A}}=\frac{1}{1+10^{\left(R_{\mathrm{B}}-R_{\mathrm{A}}\right)/400}}.$
(3)

##  Appendix D Sample weights for the Weighted Learning Approach

We carry out experiments for two different a values ($a\in\\{4,10\\}$) to see
whether a big weight difference (more expert specialization) or a small weight
difference (more expert similarity) leads to better overall performance. The
values of the resulting normalized sample weights can be found in Table 5.

Table 5: Sample weights for the Weighted Learning Approach a | expert main phase | $w_{\text{main}}$ | $w_{\text{other}}$ | $w_{\text{opening}}$ | $w_{\text{midgame}}$ | $w_{\text{endgame}}$
---|---|---|---|---|---|---
4 | opening | 2 | 0.5 | 2 | 0.5 | 0.5
midgame | 2 | 0.5 | 0.5 | 2 | 0.5
endgame | 2 | 0.5 | 0.5 | 0.5 | 2
10 | opening | 2.5 | 0.25 | 2.5 | 0.25 | 0.25
midgame | 2.5 | 0.25 | 0.25 | 2.5 | 0.25
endgame | 2.5 | 0.25 | 0.25 | 0.25 | 2.5

##  Appendix E Extended Experimental Setup

###  E.1 Datasets

The KingBase dataset is a comprehensive collection of chess games, renowned
for featuring high-quality matches played by grandmasters and skilled players.
Typically provided in Portable Game Notation (PGN) format, it encompasses
detailed information on moves, players, dates, and tournament specifics.
Widely employed in computer chess research, machine learning, and artificial
intelligence, the dataset is valued for studying advanced chess strategies,
training and testing chess engines, and analyzing player performance.

Our chess datasets are built based on the Kingbase Lite 2019 database. This
database includes over one million chess games from players rated at least
2200 Elo. We filter out all games shorter than five moves in total as these
games are either quickly arranged draws or errors in the database and,
therefore, unreliable sources. In order to create the training and evaluation
data for our experiments, we build four datasets. The first dataset, which we
will refer to as ”no-phases”, contains all positions left after filtering the
database as described above. For the remaining three datasets (”opening”,
”midgame” and ”endgame”), we strictly split the database into one part for
each of the three phases. This splitting is carried out by going through every
game in the dataset and removing all board positions that do not belong to the
phase for which we are currently preparing the dataset. This procedure leads
to some games being completely excluded for a specific part if they contain no
position of the particular phase. For example, about 32 percent of games ended
before the endgame began, therefore contributing no position for the endgame
dataset.

###  E.2 Neural Network Architecture and Input Representation

Table 6: Input representation, taken from Czech et al. [2023]. Features are encoded as binary maps, and specific features are indicated with $\ast$ as single values applied across the entire $8\times 8$ plane. $*$ represent scalar planes only saving one value over all 64 bits of the plane. The historical context is captured as a trajectory spanning the last eight moves. Overall the input representation consists of 52 planes.  Feature | Planes | Type | Comment
---|---|---|---
P1 pieces | 6 | bool | order: {PAWN, KNIGHT, BISHOP, ROOK, QUEEN, KING}
P2 pieces | 6 | bool | order: {PAWN, KNIGHT, BISHOP, ROOK, QUEEN, KING}
Repetitions* | 2 | bool | how often the board positions has occurred
En-passant square | 1 | bool | the square where en-passant capture is possible
P1 castling* | 2 | bool | binary plane, order: {KING_SIDE, QUEEN_SIDE}
P2 castling* | 2 | bool | binary plane, order: {KING_SIDE, QUEEN_SIDE}
No-progress count* | 1 | int | sets the no progress counter (FEN halfmove clock)
Last Moves | 16 | bool | origin and target squares of the last eight moves
is960* | 1 | bool | if the 960 variant is active
P1 pieces | 1 | bool | grouped mask of all P1 pieces
P2 pieces | 1 | bool | grouped mask of all P2 pieces
Checkerboard | 1 | bool | chess board pattern
P1 Material difference* | 5 | int | order: {PAWN, KNIGHT, BISHOP, ROOK, QUEEN}
Opposite color bishops* | 1 | bool | if they are only two bishops of opposite color
Checkers | 1 | bool | all pieces giving check
P1 material count* | 5 | int | order: {PAWN, KNIGHT, BISHOP, ROOK, QUEEN}
Total | 52 |  |

Since our approach is based on and compared to the CrazyAra approach, by Czech
et al. [2020], we use the same model architecture called RISE in its current
version (RISEv3.3). This architecture is in turn an improved ResNet
archtiecture how they were used in AlphaZero. More information about the
archtiecture can be found under
https://github.com/QueensGambit/CrazyAra/wiki/Model-architecture. As loss
function we used

$\small\ell=-\alpha(\mathbf{WDL}^{\top}_{\text{t}}\log\mathbf{WDL}_{\text{p}})-\boldsymbol{\pi}^{\top}\log\boldsymbol{p}+\beta(ply_{\text{t}}-ply_{\text{p}})^{2}+c\|\theta\|_{2}^{2}$
(4)

introduced in the work of Czech et al. [2023]. Within the approach our MCTS
implementation follows the work by Kocsis and Szepesvári [2006]; Coulom [2006]
with the adaptations, done by Silver et al. [2018], e.g., using the PUCT
formula instead of UCT.

Similar to AlphaZero [Silver et al., 2018] or CrazyAra [Czech et al., 2020],
we represent the game state in the form of a stack of so-called levels or
planes. The complete stack of planes can be found in Table 6 and is taken from
Czech et al. [2023]. Each layer or plane represents a channel describing one
of the input features in the current state. Each plane is encoded as a map
with $8\times 8$ bits. We hereby distinguish between two types of planes, bool
and int. In a bool plane each bit is representing a different field of a
chessboard, e.g., if a pawn is placed on this field. In some cases, like
repetitions, a single information is stored in a plane, in these cases all
bits of the plane show the same information. In Table 6 this is marked with
$*$. Integer or int planes have the same functionality but instead of 0 and 1,
they store integer, like how many pawns are left on the board.

The output of our network, also follows Silver et al. [2018] and is described
as the expected utility of the game position, represented by a numeric value
in the range of $[-1,1]$, often called value, and a distribution over all
possible actions, called policy $\boldsymbol{\pi}$.

###  E.3 Reproducibility and Hyperparameters

Table 7: Hyperparameter Configuration for Experimental Settings. This table provides a comprehensive overview of the essential hyperparameters utilised in our experimental design.  Hyperparameter | Value | Hyperparameter | Value
---|---|---|---
max learning rate | 0.14 | value loss factor | 0.01
min learning rate | 0.00001 | policy loss factor | 0.988
batch size | 2048 | wdl loss factor | 0.01
max momentum | 0.95 | plys to end loss factor | 0.002
min momentum | 0.8 | stochastic depth probability | 0.05
epochs | 7 | pytorch version | 1.12.0
optimizer | NAG | spike thresh | 1.5
weight decay (wd) | 0.0001 | dropout rate | 0.0
seed | 9 |  |

(a) Learning rate schedule

(b) Momentum schedule

Figure 14: Learning rate and momentum schedules over the course of a training run of a single expert model. Table 8: Hard- and software configuration for our experimental section.  Hardware/Software | Description
---|---
GPU | 8 $\times$ NVIDIA® Tesla V100
NGC Container | pytorch:22.05
GPU-Driver | CUDA 11.4
CPU | Dual Intel Xeon Platinum 8168
Operating System | Ubuntu 20.04 LTS
CrazyAra | Release 1.0.4
Backend | TensorRT-8.4.1

Our hyperparameter setup is very similar to the settings proposed in Czech et
al. [2020]. The list of hyperparameters can be seen in Table 7 and the
schedules for learning rate and momentum are depicted in Figure 14. An
exemplary command for running cutechess101010For detailed information about
the cutchess-cli
https://manpages.ubuntu.com/manpages/xenial/en/man6/cutechess-cli.6.html
matches with all used parameters can be found in Figure 15. In order to
provide a diversified playing ground, we make use of an opening
suite111111https://raw.githubusercontent.com/ianfab/books/master/chess.epd
featuring positions appearing after the first several moves have already been
played. These positions are intentionally chosen in a way that each position
is imbalanced, with a slight edge for either side to reduce the amount of
resulting draws. For each match, we randomly sample 500 opening positions from
this opening suite and let our agents play against each other starting from
there. In order to take the imbalances of the positions into account, we use
each starting position twice so that each agent is playing once as white and
once as black.

    ./cutechess-cli -variant standard -openings file=chess.epd format=epd order=random
    -pgnout /data/cutechess_res.pgn -resign movecount=5 score=600 -draw movenumber=30
    movecount=4 score=20 -concurrency 1
    -engine name=ClassicAra_correct_phases cmd=./ClassicAradir=~/CrazyAra/engine/build
    option.Model_Directory=/data/model/ClassicAra/chess/correct_phases proto=uci
    -engine name=ClassicAra_no_phases cmd=./ClassicAra dir=~/CrazyAra/engine/build
    option.Model_Directory=/data/model/ClassicAra/chess/no_phases proto=uci
    -each option.First_Device_ID=0 option.Batch_Size=64 option.Fixed_Movetime=0
    tc=0/6000+0.1 option.Nodes=100 option.Simulations=200 option.Search_Type=mcts
    -games 2 -rounds 500 -repeat

Figure 15: Exemplary command for running a cutechess match between two
approaches. The Batch_Size is set to 1, 8, 16, 32, and 64. The Nodes parameter
(values of 0, 100, 200, 400, 800, 1600, 3200) limits the number of nodes that
the MCTS can visit per move during its search. The Simulations parameter
describes the number of simulations per move and is set to double the Nodes
value. When the Nodes parameter is 0, we instead limit the extent of the
search by a fixed move time in milliseconds (Fixed_Movetime values of 100,
200, 400, 800, 1600).

Ensuring reproducibility is paramount in scientific research, as it
establishes the foundation for the reliability and credibility of study
findings. For this, we provide an anonymous version of our code, with a short
manual how to run the code in the ReadMe as well as our trained models used in
the experimental section. We used seeds where it was applicable, e.g., in the
training process and rerun our experiments to test the robustness of our
results. The experiments were run on a setup, described in Table 8, using the
NVIDIA GPU Cloud (NGC) docker container for
pytorch121212https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch. As
stated before, all needed data is openly available.

##  Appendix F Additional Results

In this section, we present supplementary findings and extended analyses that
complement and enrich the core results discussed in the paper, providing a
more comprehensive understanding of the investigated approach.

#### Training Process

From Figure 16 till Figure 25 we show the training process of our three
learning approaches for all three experts on all four test sets (opening,
midgame, endgame and no phases). It can be seen that the experts outperform
other models in their designated phase.

#### Elo Development over Training

Figures 26, 27 and 28 show the result of test matches between the model in
training and a baseline model for different restricting factors in the search
and evaluation of game states. On the left we used the number of iterations,
i.e., the number of explored nodes within MCTS as restricting factor. On the
right, instead of restricting the search by the number of iterations, we chose
to restrict it, using a time limit, a common pratice in, e.g., computational
chess. As baseline model in all these figures the regular learning approach,
i.e., the single model approach, was taken.

#### Extension to Figure 6

Since we did all our experiments using two restricting factors, i.e., using
the number of nodes and the move time as restrictions, we also evaluated
Figure 6 twice. The evaluation using time as the restricting factor, can be
found in Figure 29.

(a) Loss

(b) Policy Accuracy

Figure 16: The loss (Fig. 16(a)) and policy accuracy (Fig. 16(b)) over the
course of the training process for the Regular Learning approach. The metrics
were evaluated on the opening (blue), midgame (orange), endgame (green) and
no-phases (black) test set. The evaluation on the train set is shown in the
background in grey. The values on the x-axis represent iterations, where one
iteration is defined as doing backpropagation on one batch of training data.

(a) Loss

(b) Policy Accuracy

Figure 17: The opening expert’s loss (Fig. 17(a)) and policy accuracy (Fig.
17(b)) over the course of the training process for the Separated Learning
approach. The metrics were evaluated on the opening (blue), midgame (orange),
endgame (green) and no-phases (black) test set. The evaluation on the train
set is shown in the background in grey. The values on the x-axis represent
iterations, where one iteration is defined as doing backpropagation on one
batch of training data. The metric values of the final model checkpoint of the
Regular Learning approach are added as dashed reference lines (same colors
represent the same test set).

(a) Loss

(b) Policy Accuracy

Figure 18: The midgame expert’s loss (Fig. 18(a)) and policy accuracy (Fig.
18(b)) over the course of the training process for the Separated Learning
approach. The metrics were evaluated on the opening (blue), midgame (orange),
endgame (green) and no-phases (black) test set. The evaluation on the train
set is shown in the background in grey. The values on the x-axis represent
iterations, where one iteration is defined as doing backpropagation on one
batch of training data. The metric values of the final model checkpoint of the
Regular Learning approach are added as dashed reference lines (same colors
represent the same test set).

(a) Loss

(b) Policy Accuracy

Figure 19: The endgame expert’s loss (Fig. 19(a)) and policy accuracy (Fig.
19(b)) over the course of the training process for the Separated Learning
approach. The metrics were evaluated on the opening (blue), midgame (orange),
endgame (green) and no-phases (black) test set. The evaluation on the train
set is shown in the background in grey. The values on the x-axis represent
iterations, where one iteration is defined as doing backpropagation on one
batch of training data. The metric values of the final model checkpoint of the
Regular Learning approach are added as dashed reference lines (same colors
represent the same test set).

(a) Loss

(b) Policy Accuracy

Figure 20: The opening expert’s loss (Fig. 21(a)) and policy accuracy (Fig.
21(b)) over the course of the training process for the Staged Learning
approach. The metrics were evaluated on the opening (blue), midgame (orange),
endgame (green) and no-phases (black) test set. The evaluation on the train
set is shown in the background. The values on the x-axis represent iterations,
where one iteration is defined as doing backpropagation on one batch of
training data. The vertical lines indicate the points at which the training on
a new dataset started (initialized with the parameters of the last model
checkpoint of the previous training stage).

(a) Loss

(b) Policy Accuracy

Figure 21: The opening expert’s loss (Fig. 21(a)) and policy accuracy (Fig.
21(b)) over the course of the training process for the Staged Learning
approach. The metrics were evaluated on the opening (blue), midgame (orange),
endgame (green) and no-phases (black) test set. The evaluation on the train
set is shown in the background. The values on the x-axis represent iterations,
where one iteration is defined as doing backpropagation on one batch of
training data. The vertical lines indicate the points at which the training on
a new dataset started (initialized with the parameters of the last model
checkpoint of the previous training stage).

(a) Loss

(b) Policy Accuracy

Figure 22: The endgame expert’s loss (Fig. 22(a)) and policy accuracy (Fig.
22(b)) over the course of the training process for the Staged Learning
approach. The metrics were evaluated on the opening (blue), midgame (orange),
endgame (green) and no-phases (black) test set. The evaluation on the train
set is shown in the background. The values on the x-axis represent iterations,
where one iteration is defined as doing backpropagation on one batch of
training data. The vertical lines indicate the points at which the training on
a new dataset started (initialized with the parameters of the last model
checkpoint of the previous training stage).

(a) Loss ($a=4$)

(b) Policy Accuracy ($a=4$)

(c) Loss ($a=10$)

(d) Policy Accuracy ($a=10$)

Figure 23: The opening expert’s loss (Fig. 23(a) and Fig. 23(c)) and policy
accuracy (Fig. 23(b) and Fig. 23(d)) over the course of the training process
for the Weighted Learning approach with different $a$ values. The metrics were
evaluated on the unweighted opening (blue), midgame (orange), endgame (green)
and no-phases (black) test set. The evaluation on the train set (weighted) is
shown in the background. The values on the x-axis represent iterations, where
one iteration is defined as doing backpropagation on one batch of training
data. The metric values of the final model checkpoint of the Regular Learning
approach are added as dashed reference lines (same colors represent the same
test set).

(a) Loss ($a=4$)

(b) Policy Accuracy ($a=4$)

(c) Loss ($a=10$)

(d) Policy Accuracy ($a=10$)

Figure 24: The midgame expert’s loss (Fig. 24(a) and Fig. 24(c)) and policy
accuracy (Fig. 24(b) and Fig. 24(d)) over the course of the training process
for the Weighted Learning approach with different $a$ values. The metrics were
evaluated on the unweighted opening (blue), midgame (orange), endgame (green)
and no-phases (black) test set. The evaluation on the train set (weighted) is
shown in the background. The values on the x-axis represent iterations, where
one iteration is defined as doing backpropagation on one batch of training
data. The metric values of the final model checkpoint of the Regular Learning
approach are added as dashed reference lines (same colors represent the same
test set).

(a) Loss ($a=4$)

(b) Policy Accuracy ($a=4$)

(c) Loss ($a=10$)

(d) Policy Accuracy ($a=10$)

Figure 25: The endgame expert’s loss (Fig. 25(a) and Fig. 25(c)) and policy
accuracy (Fig. 25(b) and 25(d)) over the course of the training process for
the Weighted Learning approach with different $a$ values. The metrics were
evaluated on the unweighted opening (blue), midgame (orange), endgame (green)
and no-phases (black) test set. The evaluation on the train set (weighted) is
shown in the background. The values on the x-axis represent iterations, where
one iteration is defined as doing backpropagation on one batch of training
data. The metric values of the final model checkpoint of the Regular Learning
approach are added as dashed reference lines (same colors represent the same
test set).

(a) Elo gain by Nodes

(b) Elo gain by move time

Figure 26: Relative Elo gain of the Separated Learning approach for different
batch sizes. Node values of 100, 200, 400, 800, 1600 and 3200 in Fig. 26(a).
Move time values of 100, 200, 400, 800 and 1600 in Fig. 26(b)

(a) Elo gain by Nodes

(b) Elo gain by move time

Figure 27: Relative Elo gain of the Staged Learning approach for different
batch sizes. Node values of 100, 200, 400, 800, 1600 and 3200 in Fig. 27(a).
Move time values of 100, 200, 400, 800 and 1600 in Fig. 27(b)

(a) Elo gain by Nodes ($a=4$)

(b) Elo gain by move time ($a=4$)

(c) Elo gain by Nodes ($a=10$)

(d) Elo gain by move time ($a=10$)

Figure 28: Relative Elo gain of the Weighted Learning approach for different
batch sizes and $a$ values of 4 and 10. Node values of 100, 200, 400, 800,
1600 and 3200 in Fig. 28(a) and Fig. 28(c). Move time values of 100, 200, 400,
800 and 1600 in Fig. 28(b) and 28(d) Figure 29: Relative Elo gain of using
only one specific phase expert in the MCTS and using the baseline network for
all remaining predictions. The used experts are taken from the separated
learning approach for chess and the batch size was set to 64 for all matches.