# Design and Implementation of TAG:
A Tabletop Games Framework

Raluca D. Gaina*, Martin Balla*, Alexander Dockhorn*,
Raúl Montoliu†, Diego Perez-Liebana*
*School of Electronic Engineering and Computer Science, Queen Mary University of London, UK
†Insitute of New Imaging Technologies, Jaume I University, Castellon, Spain

###### Abstract

This document describes the design and implementation of the Tabletop Games
framework (TAG), a Java-based benchmark for developing modern board games for
AI research. TAG provides a common skeleton for implementing tabletop games
based on a common API for AI agents, a set of components and classes to easily
add new games and an import module for defining data in JSON format. At
present, this platform includes the implementation of seven different tabletop
games that can also be used as an example for further developments.
Additionally, TAG also incorporates logging functionality that allows the user
to perform a detailed analysis of the game, in terms of action space,
branching factor, hidden information, and other measures of interest for Game
AI research. The objective of this document is to serve as a central point
where the framework can be described at length. TAG can be downloaded at:

https://github.com/GAIGResearch/TabletopGames

_K_ eywords Tabletop Games $\cdot$ General Game AI $\cdot$ TAG $\cdot$
Statistical Forward Planning

###### Contents

  1. 1 How to use this Document?
  2. 2 TAG and Tabletop Games
  3. 3 Implementing Games in TAG
    1. 3.1 Game State
    2. 3.2 Forward Model
    3. 3.3 Actions
    4. 3.4 Other modules
  4. 4 Games
    1. 4.1 Tic-Tac-Toe
    2. 4.2 Love Letter [1]
    3. 4.3 Colt Express [2]
    4. 4.4 Uno [3]
    5. 4.5 Virus! [4]
    6. 4.6 Exploding Kittens [5]
    7. 4.7 Pandemic [6]
  5. 5 Game Analysis
  6. 6 Agents and Algorithms
    1. 6.1 Human Players
    2. 6.2 Random
    3. 6.3 One Step Look Ahead (OSLA)
    4. 6.4 Rolling Horizon Evolutionary Algorithm (RHEA)
    5. 6.5 Monte Carlo Tree Search (MCTS)
  7. 7 Running the framework
    1. 7.1 Running a game
    2. 7.2 Running multiple games
    3. 7.3 Running tournaments
  8. 8 Future Directions
  9. A A Game Example: Love Letter
    1. A.1 Game State
    2. A.2 Forward Model
    3. A.3 Actions in Love Letter

##  1 How to use this Document?

This paper serves the purpose of a living documentation for TAG - a Tabletop
Games framework for studying computational intelligence in this type of games.
This document will be continuously updated with the new developments of the
framework. When citing this paper please make sure to specify its version
since the paper will be updated over time.

##  2 TAG and Tabletop Games

TAG was designed to capture most of the complexity that modern tabletop games
provide, with a few games implemented already and more in progress. Our
framework includes handy definitions for various concepts and components
common across many tabletop games [7].

We define an action as an independent unit of game logic that modifies a given
game state towards a specific effect (e.g. player draws a card; player moves
their pawn). These actions are executed by the game players and are subject to
certain rules: units of game logic, part of a hierarchical structure (a game
flow graph). Rules dictate how a given game state is modified and control the
flow through the game graph (for instance, checking the end of game conditions
and the turn order). This turn order defines which player is due to play at
each time, possibly handling player reactions forced by actions or rules. At a
higher level, games can be structured in phases, which are time frames where
specific rules apply and/or different actions are available for the players.

All tabletop games use components (game objects sharing certain properties),
whose state is modified by actions and rules during the game. TAG includes
several predefined components to ease the development of new games, such as
tokens (a game piece of a particular type), dice (with N sides), cards (with
text, images or numbers), counters (with a numerical value), grid and graph
boards. Components can also be grouped into collections: an area groups
components in a map structure in order to provide access to them using their
unique IDs, while a deck is an ordered collection (list) with specific
interactions available (e.g. shuffle, draw, etc.). Both areas and decks are
considered components themselves. A summary of the components follows:

  * •

Token: A game piece of a particular type, usually with a physical position
associated with it.

  * •

Die: A die has a number of sides N associated with it and can be rolled to
obtain a value between 1 and N (inclusive).

  * •

Card: A card usually has text, images or numbers associated with any of its 2
sides, and is the most common type of component used in decks.

  * •

Counter: An abstract concept used to keep track of a particular variable
numerical value; usually represented on a board with tokens used to mark the
current value, but recognized as a separate object in this framework. It has a
minimum, maximum and current value associated with it, where the current value
can vary between the minimum (inclusive) and maximum (inclusive).

  * •

Graph board: A graph representation for a board, as a collection of several
board nodes connected between each other.

  * •

Board node: A node in a graph board which keeps track of its neighbours (or
connections) in the board.

  * •

Grid board: A 2D grid representation of a board, with a width and height
associated with it. It can hold elements of any type.

The structure of TAG consists of several packages:

  * •

core: All core framework functionality, including all abstract classes to be
extended by game implementations.

  * •

evaluation: Classes for running tournaments and evaluations of games or AI
players.

  * •

games: Specific implementations of abstract classes for each game, each
grouped in its own package.

  * •

gui: Generic Graphical User Interface (GUI) helper classes, including the
PrototypeGUI class which can be used with any in-progress game. Each game can
extend this to implement customized functionality in their GUIs, for better
player interaction and thus a better player experience.

  * •

players: All players (human and AI) available.

  * •

utilities: Various utility classes and generic functionality shortcuts.

Additionally, the folder data (outside of the source files) stores assets for
the games, each in their own folders. Most of these refer only to images used
in the interfaces, but may include JSON files and other data as well (e.g.
Pandemic stores board and card information here as well).

The TAG framework brings together all of the concepts and components described
previously and allows quick implementation and prototyping of new games. To
this end, a flexible API is provided for all functionality needed to define a
game, with multiple abstract classes that can be extended for specific
implementations. The framework already provides some generic functionality:
ready-made components, rules, actions, turn orders and game phases. These are
all represented through several game objects found in the core package, which
can be instantiated in game implementations for immediate use. Additionally,
TAG includes a fully functional game loop and a prototyping GUI. The GUI
allows users to start interacting with the games as soon as they have the two
main classes required set up: a Game State (GS) class and a Forward Model (FM)
class.

GS is a container class, including all variables and game components which
would allow one to describe one specific moment in time. It defines access
methods in the game state to retrieve existing game components, make custom
and partially observable copies of the state, and define an evaluation
function that can be used by the playing agents. We strongly recommend that
game components (i.e. extending the Component class) are used to describe a
game state as much as possible (e.g. using Counters instead of integer
variables to keep track of numbers in the game), in order for general AI
players to have an easy common access to understand game states.

FM encompasses the logic of the game: performs the game setup (by setting the
GS components and variables to their initial state and/or reading from files
and creating components); defines what actions players can take in a
particular game state; applies the effect of player actions (received one at a
time; if a simultaneous-action game is to be implemented, we recommend waiting
to receive all before modifying the game state) and any other game rules
applicable; uses a turn order to decide which player is due to play next, and
checks for any end of game conditions (and setting the game status and player
results in the game state appropriately if the game is over). The FM is
available to AI players for game simulations.

For each game, users can further implement specific actions, rules, turn
orders, game parameters (for easy modification of game mechanics), a GUI and
provision of game data. The last is useful when the game requires large
amounts of data such as tile patterns, cards and board node connections, and
it is provided via JSON files. A full guide on using the framework and
implementing new games is available in the wiki provided with the
code111https://github.com/GAIGResearch/TabletopGames/wiki.

TAG’s game-loop is presented in Algorithm 1. Given a list of agents and
parameters of the game to be played, the framework performs an initial setup
of the game state ($s_{0}$) and the game’s forward model (FM). While the game
state is not terminal, the turn order selects the next player to act. To
ensure partial observability, we generate an observation object for the
current agent which hides the state of unobserved components. Hence, a list of
available actions is generated and the agent is queried to provide the next
action to be executed ($a_{t}$). Finally, the forward model is used to modify
the game state given the action (producing $s_{t+1}$), and the graphical user
interface is updated accordingly.

Algorithm 1 Overview Game Loop

0: list of agents, game parameters $gp$

0: win rate statistics

$s_{0}$, FM = setupGame($gp$)

while not isTerminal($s_{t}$) do

$agent$ $\leftarrow$ getCurrentPlayer($s_{t}$)

observation $\leftarrow$ getObservation($s_{t}$, $agent$)

actions $\leftarrow$ FM.getAvailableActions($s_{t}$, $agent$)

$a_{t}$ $\leftarrow$ $agent$.getAction(observation, actions)

$s_{t+1}\leftarrow$ FM.next($s_{t}$, $a_{t}$)

GUI.update()

end while

##  3 Implementing Games in TAG

This section dives deeper into the structure of the framework. TAG can be used
to implement new games, for which it provides a set of abstract classes and
interfaces to ease the development. Although there is no hard requirement to
implement games in a particular way, the recommended set of classes to
implement is described here.

In order to implement a new game in TAG, the following core modules can be
implemented:

###  3.1 Game State

This core class includes all the information about the components that form
the state of the game. New game states can extend the class
core.AbstractGameState, implementing the following methods:

  * •

_getAllComponents(): returns a list of all the components of the game state.
The method is called after game setup, so you may assume all components have
already been created. Decks and Areas have all of their nested components
automatically added.

  * •

_copy(int playerId): defines a reduced, player-specific, copy of your game
state. This includes only those components (or parts of the components) which
the player with the given ID can currently see. For example, some decks may be
face down and unobservable to the player. All of the components in the
observation should be copies of those in the game state, as the player is not
prohibited from modifying the state it receives (copies ensures the real
objects in the game are not affected in any way).

  * •

_reset(): resets any variables of the game state to their state before FM
initialisation.

  * •

_getScore(int playerId): implements a heuristic function that returns a
numerical value for the current game state, given a specific player - the
bigger this value, the better the state.

Additionally, game state classes can implement the
core.interfaces.IFeatureRepresentation interface, which provides an interface
for methods that can be used to extract further generic information about the
game state, in terms of abstract features. Game state classes can also
implement the core.interfaces.IVectorObservation interface, which allows AI
players to receive a vector observation from the game state.

Appendix A.1 shows an example of a Game State implementation for the game Love
Letter.

###  3.2 Forward Model

The forward model class is in charge of setting up the initial game state and
advancing the game state when provided with an action to be executed by one of
the players. In order to implement a forward model, extend the class
core.AbstractForwardModel and implement the following methods:

  * •

_setup(AbstractGameState firstState): perform the initial game setup according
to the game rules, initialising all components and variables in the given game
state (e.g. give each player their starting hand, place tokens on the board
etc.).

  * •

_next(AbstractGameState currentState, AbstractAction action): apply the given
action to the given game state, execute any other non-action-dependent game
rules (e.g. related to game phase changes), check for game end conditions
(setting the game status and player results in the given state) and move to
the next player if required.

  * •

_computeAvailableActions(AbstractGameState gameState): return a list with all
actions available for the current player, in the context of the game state
object.

  * •

_copy(): return a new instance of the Forward Model object with any necessary
variables copied. Use a new random seed (e.g. System.currentTimeMillis() for
initialisation of the copy object).

  * •

endGame(): override this function if the game requires any extra end of game
computation (e.g. to print end of game results).

Appendix A.2 shows an example of a Forward Model implementation for the game
Love Letter.

Note: Forward model classes can instead extend from the abstract class
core.rules.AbstractRuleBasedForwardModel, if they wish to use the rule-based
system instead; this class provides basic functionality and documentation for
using rules, and an already implemented _next() function.

###  3.3 Actions

TAG provides a series of simple actions that are common for many games, such
as shuffling decks (or, more generally, shuffling groups of components),
drawing components from one deck to another, replacing components, re-
arranging decks or placing tokens in grid boards. The games currently included
in the framework use these actions to manipulate game components such as decks
and counters in the game.

It is also possible to define game specific actions for new games. For this, a
new action class needs to be created that extends from
core.actions.AbstractAction. Two main methods can be overriden:

  * •

execute(AbstractGameState gs): this method executes the action the class
represents, applying its effect to the given game state. Within this method,
any component that has been registered with a unique ID can be retrieved from
the game state with the function AbstractGameState.getComponentById(int id).

  * •

copy(): this method returns an exact copy of this action.

Important: Actions should not hold any reference to other objects to avoid
cross-referencing in the game and prevent issues with deep copies. Unique
component IDs should be passed instead into action classes constructors.

Examples of game-related actions for Love Letter can be seen in Appendix A.3

###  3.4 Other modules

Additionally, some other classes should also be implemented or added to,
following the framework’s structure:

  * •

Game Type: TAG allows to categorize its games into different internal
collections of games of the similar type. This allows the user, for instance,
to retrieve games with certain mechanics or number of players within the
framework. The class games.GameType includes several methods that can be added
to so that a new game is known and search-able within the framework. In order
to incorporate a game to the framework in this way, you should add the
following:

    * –

Add a new enum value for your game, defining the minimum and maximum number of
players, a list of categories the game belongs to, and a list of mechanics the
game uses (both categories and mechanics can be already existing ones or newly
defined options).

    * –

In the method stringToGameType(String game), add a new case for your game so
that the string name can be converted to the correct game type.

    * –

In the method createGameInstance(int nPlayers, long seed), add a new case for
your game, creating the appropriate instances of the game state and forward
model needed to run, given the number of players taking part in the game and
the random seed.

    * –

In the method createGUI(AbstractGameState gameState, ActionController ac), add
a new case for your game (if GUI is implemented, the PrototypeGUI may be used
otherwise), creating the appropriate instance of GUI, given an initial game
state and the action controller managing user interactions within the GUI.

Important: doing this step is necessary for using the main methods of running
games in the framework in the core.Game class and game analysis.

  * •

GUI: The abstract class core.AbstractGUI can be extended to indicate not only
how the game should be displayed in a graphical user interface, but also to
specify what bits should be occluded to respect partial observability of the
game state for human players. The class gui.PrototypeGUI and the other GUI
helper classes in package gui can be used as a simple template to be
customized depending on necessary visualisations and interactions.

  * •

Game Parameters: These allow tweaking the game behaviour and rules when these
depend on numerical or discrete values. For instance, the number of cards in a
hand of Uno or the number of outbreaks allowed before losing a game of
Pandemic can be specified this way. This provides a centralized point for the
parameters of a game that define its behaviour and allow play-testing
different versions of the same game easily. In order to implement game
parameters, a parameters class for a specific game must extend
core.GameParameters or implement core.interfaces.ITunableParameters (which
allows game parameters to be randomized or tuned).

  * •

Turn Order: The order in which players make moves in the game can be specified
by extending the abstract class core.turnorders.TurnOrder, which provides
functionality to establish the starting and current player and advance the
turn to the next player to move. Default turn orders such as alternate (one
after another until all players have played) and reactive (which adds the
possibility of a player to act out of turn) are included in the framework and
can be reused for new games.

  * •

Constants: For efficiency reasons, you may declare constants of a given game
for quick access through the framework. The class pandemic.PandemicConstants
shows an example of how to specify game-related constants for a game, and how
to use the framework’s Hash utility to provide direct access to certain game
components.

The following classes can further be implemented, but they are not required
for essential game functionality, and may not be needed at all depending on
the game implemented:

  * •

Game (Optional): Extending core.Game, allows to specify the types of
parameters, game state, GUI and forward model for a specific game. This class
can also be used to add the entry point to run the game with some AI agents or
humans.

  * •

Game Data (Optional): Some tabletop games need extra data information, which
may come in the form of specific card sections (costs, effects, categories,
etc.), tile patterns, etc. This data can be specified in json format files for
each component type and read with the json parser incorporated in the
framework. Alternatively, it is possible to add own format and parser code.
The abstract class core.AbstractGameData can be extended to incorporate data
reading for the particular game.

  * •

Game Phase (Optional): Some games can be structured in phases, we different
rules and available actions apply. The core.AbstractGameState class includes a
definition for some basic default game phases (main, player reaction, end),
which could be expanded in order to provide specific phases for new games.

##  4 Games

There are currently 7 games implemented in the framework, varying from very
simple test games (Tic-Tac-Toe) to strategy games (Pandemic [6]), as well as
diverse challenges for AI players. A few games are currently in active
development (Descent [8], Carcassonne [9] and Settlers of Catan [10]), and
many more are in the project’s backlog, including games from other frameworks
to allow for easy comparison. Further development plans also include adding
easy to use functionality for wrapping external games, so that a direct
comparison can be carried out under the same conditions without the need to
re-implement games under our framework’s full restrictions.

All games implemented can be found in the games package, each registered in
the games.GameType class; this class allows specifying properties for each
game, to allow for automatic listing for experiments (e.g. a list of all games
with the “cooperative” tag). We highlight next some particularities of the
games currently implemented in the framework.

###  4.1 Tic-Tac-Toe

$2$ players alternate placing their symbol in a $N\times N$ grid until one
player completes a line, column or diagonal and wins the game; if all cells in
the grid get filled up without a winner, the game is a draw. This is the
simplest game included in the framework, meant to be used as a quick reference
for the minimum requirements to get a game up and running. Its implementation
makes use of mostly default concepts and components, but it implements a
scoring heuristic and a custom GUI for an easier interaction given the
specific game mechanics.

###  4.2 Love Letter [1]

$2$ to $4$ players start the game with one card each, representing a
character, a value and a unique effect. A second card is drawn at the start of
each turn, one of which must be played afterwards. After the last card of the
deck is drawn, the player with the highest valued card wins the current round.
A player wins the game after winning 5 rounds. Love Letter features partial
observability, asymmetric and changing player roles and a point system over
several rounds. Figure 1 shows an example game state.

Figure 1: Screenshot of Love Letter in TAG.

###  4.3 Colt Express [2]

$2$ to $6$ players control a bandit each, with a unique special ability. Their
goal is to collect the most money while traversing the two-level compartments
in a train and avoiding the sheriff (a non-player character moved by players
and round card events). The game consists of several rounds, each with a
planning (players play action cards) and an execution (cards are executed in
the same order) phase. During the former, players play action cards, which are
executed in the same order in the latter phase. This processing scheme forces
players to adapt their strategy according to all the moves already played, in
an interesting case of partial observability and non-determinism: the
opponents’ type of action may be known (sometimes completely hidden in a
round), but not how it will be executed. Additionally, the overall strategy
should be adapted to a bandit’s unique abilities. Those actions can move the
player’s character along the compartments, interact with neighbouring players,
or collect money. The game ends after all rounds are played, the player with
the most money being the winner. Figure 2 shows an example game state.

Figure 2: Screenshot of Colt Express in TAG.

###  4.4 Uno [3]

The game consists of coloured cards with actions or numbers. Numbered cards
can only be played in case either the colour or the number matches the newest
card on the discard pile. Action cards let $2$ to $10$ players draw additional
cards, choose the next colour to be played or reverse the turn order. A player
wins after gaining a number of points over several rounds (computed as the sum
of all other players’ card values). Uno features stochasticity, partial
observability and a dynamically changing turn order. This game has the
potential of being the longest game in the framework since players need to
draw new cards in case they cannot play any. Figure 3 shows an example game
state.

Figure 3: Screenshot of Uno in TAG.

###  4.5 Virus! [4]

$2$ to $6$ players have a body each that consists of four organs, which can
be: infected (by an opponent playing a virus card), vaccinated (by a medicine
card), immunised (by 2 medicine cards) or destroyed (by opponents playing 2
consecutive virus cards). The winner is the first player who forms a healthy
and complete body. Virus! features stochasticity and partial observability,
with the draw pile and opponents’ cards being hidden.

###  4.6 Exploding Kittens [5]

$2$ to $5$ players try to avoid drawing an exploding kitten card while
collecting other useful cards. Each card gives a player access to unique
actions to modify the game-state, e.g. selecting the player taking a turn next
and shuffling the deck. This game features stochasticity, partial
observability and a dynamic turn order with out-of-turn actions: in contrast
to previous games, Exploding Kittens keeps an action stack so that players
have the chance to react to cards played by others using a Nope card. A Nope
card cancels the most recent effect, but can itself be cancelled by another
Nope card. The turn order and action stack are implemented by extending the
base-classes of the framework. Figure 4 shows an example game state.

Figure 4: Screenshot of Exploding Kittens in TAG.

###  4.7 Pandemic [6]

Pandemic is a cooperative board game for $2$ to $4$ players. The board
represents a world map, with major cities connected by a graph. Four diseases
break out and the objective of the players is to cure them all. Diseases keep
spreading after each player’s turn, sometimes leading to outbreaks. Each
player is assigned a unique role with special abilities and is given cards
that can be used for travelling between cities, building research stations or
curing diseases. Additionally, they have access to special event cards, which
can be played anytime (also out-of-turn). All players lose if they run out of
cards in the draw deck, if too many outbreaks occur or if the diseases spread
too much. Pandemic features partial observability with face-down decks of
cards and asymmetric player roles. It employs a reaction system to handle
event cards and is the only game currently using the graph-based rule system.

In each turn, the player can play up to $4$ consecutive actions, with a
changing action space (e.g. moving to a new city may result in new actions).
To handle event cards and actions that require consent or reactions from other
players, a reaction system is used, so that a player can be asked to return an
action whenever needed. Pandemic also features partial observability, as the
draw deck and the infection deck are not visible. Figure 5 shows an example
game state.

Figure 5: Screenshot of Pandemic in TAG.

##  5 Game Analysis

All games in the framework can be analysed to illustrate the challenge they
provide for AI players. Here, we present measurements currently implemented
and taken from the existing games described in Section 4. We measure the
following averages, observed in our experiments:

  * $\mu_{1}$ Action space size: the number of actions available for a player on their turn.

  * $\mu_{2}$ Branching factor: the number of distinct game states reached through player actions from a given state.

  * $\mu_{3}$ State size: the number of components in a state.

  * $\mu_{4}$ Amount of hidden information: the percentage of components hidden from players on their turn.

  * $\mu_{5}$ Game speed: the execution speed of 4 key functions (in number of calls per second): setup, next, available action computation and state copy.

  * $\mu_{6}$ Game length: measured as the number of decisions taken by AI players, the total number of game loop iterations (or ticks), the number of rounds in the game and the number of actions per turn (APT) for a player.

  * $\mu_{7}$ Reward sparsity: granularity of the heuristic functions provided by the game, measured by minimum, maximum and standard deviation of rewards observed by the players.

Table 1: Analysis of games, played 1000 times for each possible number of players on each game, using random agents: action space size, branching factor, state size, hidden information and game speed. | $\mu_{1}$ | $\mu_{2}$ | $\mu_{3}$ | $\mu_{4}$ | $\mu_{5}$
---|---|---|---|---|---
Setup | Next | Actions | Copy
Tic-Tac-Toe | 5.69 | 6.79 | 1 | 0% | $10^{5}$ | $10^{6}$ | $10^{5}$ | $10^{6}$
Love Letter | 4.74 | 10.78 | 24.00 | 62.96% | $10^{5}$ | $10^{6}$ | $10^{5}$ | $10^{6}$
Uno | 1.88 | 4.32 | 116.00 | 92.3% | $10^{4}$ | $10^{6}$ | $10^{4}$ | $10^{6}$
Virus! | 9.70 | 11.03 | 78.00 | 89.1% | $10^{4}$ | $10^{6}$ | $10^{5}$ | $10^{5}$
Exploding Kittens | 3.17 | 4.99 | 60.00 | 84.5% | $10^{5}$ | $10^{6}$ | $10^{5}$ | $10^{6}$
Colt Express | 2.91 | 5.46 | 87.60 | 87.67% | $10^{4}$ | $10^{6}$ | $10^{4}$ | $10^{6}$
Pandemic | 11.15 | 17.97 | 138.00 | 64.97% | $10^{2}$ | $10^{5}$ | $10^{4}$ | $10^{5}$
Table 2: Analysis of games, played 1000 times for each possible number of players on each game, using random agents: game length and reward sparsity | $\mu_{6}$ | $\mu_{7}$
---|---|---
#decisions | #ticks | #rounds | #APT
Tic-Tac-Toe | 7.24 | 7.61 | 3.17 | 1 | [-0.000, | 0.103] | sd=0.02
Love Letter | 53.22 | 109.48 | 6.89 | 1.96 | [-0.000, | 0.970] | sd=0.20
Uno | 193.51 | 540.78 | 6.07 | 1 | [-0.150, | 0.270] | sd=0.05
Virus! | 317.09 | 319.56 | 75.75 | 1 | [-0.000, | 0.800] | sd=0.17
Exploding Kittens | 51.86 | 73.23 | 8.98 | 1.07 | [-0.500, | 0.750] | sd=0.11
Colt Express | 91.71 | 176.13 | 5.00 | 1.00 | [-0.500, | 0.500] | sd=0.14
Pandemic | 108.62 | 173.94 | 8.25 | 5.58 | [-1.000, | 0.420] | sd=0.12

Results are presented in Tables 1 and 2 (Virus! games were limited to 100
rounds, as random play can lead to infinite games). The first thing to note is
that all games are very fast to execute: most games can execute over 1 million
calls per second to the (usually) most expensive functions (next and copy).
The games vary in length, with only $7.61$ ticks for the simplest game, Tic-
Tac-Toe, but $540.78$ for Uno. We further see variations in the state size,
with Pandemic showing most complex, while Uno includes the most hidden
information. Love Letter shows its strategic complexity through the higher
branching factor, while Exploding Kittens boasts one of the largest spread of
rewards. More complete information and graphical visualisations can be
obtained by running the evaluation.GameReport class included with the
framework.

Many of the metrics reported do not paint a complete picture if only their
average is given: action spaces, for example, vary widely during play for most
of the games presented. An example which highlights this is Uno, where the
action space is dependent on the number of cards in a player’s hand,
increasing on average as the game goes on (see Figure 6). It is further
interesting to note here that games get shorter with more players (as players
would earn more points per round if they have more opponents, and thus more
cards to total the sum of) - an example of insights which can be obtained
through analysis of the games themselves, readily available for any newly
implemented games in the framework.

Figure 6: Action space size in Uno with all player number versions; 1000 runs
per version played by random players.

##  6 Agents and Algorithms

All implemented players follow a simple interface, only requiring one method
to be implemented: getAction. This receives a game state object reduced to the
specific player’s observation of the current state of the game. How this
reduced game state is built is game-dependent, usually randomising unknown
information. This method expects an action to be returned out of those
available and is called whenever it is the player’s turn and they have more
than 1 action available (i.e. the player actually has a decision to make). If
no decision is required, the agent can choose to still receive and process the
information on the game state (in the registerUpdatedObservation function) but
an action is not requested. They may also choose to implement the
initializePlayer and finalizePlayer functions which are called at the
beginning and end of the game, respectively.

Each player has a player ID assigned by the game engine, and they receive the
forward model of the game currently being played. The FM can then be used to
advance game states given actions, compute actions available, or reset a game
to its initial state. The rest of this section defines the sample players
implemented in the framework. These agents use the game’s score to evaluate
game states (as implemented on the game side and accessible through the
observation.getScore(this.getPlayerID()) method call), but their heuristic
functions may be swapped with a different object implementing the
IStateHeuristic interface.

The rest of this section describes the different agents implemented for TAG.
Performance of the AI agents described here can be found at [11]

###  6.1 Human Players

Two types of human interaction are available, both of which interrupt the game
loop to wait for human actions on their turn. Console allows human play using
the console. It outputs the game state and available actions in the console
and the player inputs the index of the action they choose to play. GUI allows
human play with a Graphical User Interface, which is game-specific. It uses an
ActionController object to register player action requests, which are then
executed in the game.

###  6.2 Random

The simplest automatic player chooses random actions out of those available on
its turn. Its implementation is as follows:

⬇

1public class RandomPlayer extends AbstractPlayer {

2 private final Random rnd;

3

4 public RandomPlayer()

5 {

6 this(new Random());

7 }

8

9 @Override

10 public AbstractAction getAction(AbstractGameState observation) {

11 int randomAction = rnd.nextInt(observation.getActions().size());

12 List<AbstractAction> actions = observation.getActions();

13 return actions.get(randomAction);

14 }

15}

###  6.3 One Step Look Ahead (OSLA)

A greedy exhaustive search algorithm, it evaluates all actions from a given
game state and picks that which leads to the highest valued state.

###  6.4 Rolling Horizon Evolutionary Algorithm (RHEA)

RHEA [12] evolves a sequence of $L=10$ actions over several generations,
choosing the first action of the best sequence found to play in the game. The
algorithm is randomly initialised with a sequence of actions. At each
generation it creates a mutation of the current best solution, keeping the
best solution of the two. This process repeats until the given budget is
exhausted.

Given the variable action spaces and that actions available are highly
dependent on the current game state, the mutation operator chooses a gene in
the individual (i.e. position in the action sequence) and changes all actions
from that point until the end of the individual to new random valid actions.
The game’s forward model is therefore used in both mutation (to advance the
game state given the last action, in order to find the available actions for
the given position in the sequence) and evaluation (game states reached
through the sequence of actions are evaluated using the game’s heuristic,
added up for a discounted total with discount factor $\gamma=0.9$, and this
total becomes the fitness of the individual). It is important to note that
RHEA evolves only its own actions and opponents are given a random model (with
intermediate states after opponent actions ignored in fitness evaluations).

###  6.5 Monte Carlo Tree Search (MCTS)

MCTS [13] incrementally builds an asymmetric game tree balanced towards to
most promising parts of the game state space. It uses multiple iterations of
four steps: first, it navigates through the tree, using a tree policy, until
reaching a node which is not yet fully expanded; next, it adds a new random
child of this node to the tree; it then performs a Monte Carlo rollout from
the new child (randomly sampling actions until the end of the game or a
predetermined depth $L=10$); the state reached at the end of the rollout is
evaluated with a heuristic function, and this score is backed up through all
the nodes visited during the iteration. The process is repeated until the
budget is exhausted, and the most visited child of the root is chosen as the
action to play.

The version implemented in the framework is closed-loop: it stores game states
in each of the nodes. Further, the rollout step was removed after initial
experiments showing an increased performance without it; therefore, the
forward model of the game is only used when expanding a leaf node. The
resulting node is immediately evaluated using the heuristic and its value is
backed up through the tree.

##  7 Running the framework

This section describes how to run the games/AI in the framework and the
different modes available.

###  7.1 Running a game

The main method in the core.Game class allows for running any game registered
in the framework (i.e. those available in the games.GameType enum).

#### 1\. Choose on/off visuals:

Set the ActionController ac variable to null if running without visuals, or
leave as-is to enable visuals (if the game has a GUI implemented).

#### 2\. Setting the random seed for the game

Set the seed variable to the specific seed you wish to run the game with (or
leave to System.currentTimeMillis() for a new random value):
long seed = System.currentTimeMillis();

#### 3\. Choose players for the game

Add to the players array instances of the agents you wish to run the game
with. HumanConsolePlayer can be used to play as a human interacting with the
game via the console. HumanGUIPlayer can be used to play as a human
interacting with the game via the GUI (if implemented).

⬇

1 ArrayList<AbstractPlayer> players = new ArrayList<>();

2 players.add(new RandomPlayer(new Random()));

3 players.add(new RandomPlayer(new Random()));

4 players.add(new RandomPlayer(new Random()));

5 players.add(new OSLA());

#### 4\. Run!

Choose final parameters for the method call:

⬇

1private static Game runOne(GameType gameToPlay, List<AbstractPlayer> players,
long seed, ActionController ac, boolean randomizeParameters);

  * •

gameToPlay: the type of the game to play

  * •

players: previously defined players array

  * •

seed: previously defined seed variable

  * •

ac: previously defined ac variable

  * •

randomizeParameters: true if game parameters should be randomized (if
implemented for the game), false otherwise.

Adjust settings for the following variables in class core.CoreConstants if
needed:

⬇

1 public final static boolean VERBOSE = true;

2 public final static boolean PARTIAL_OBSERVABLE = false;

3 public final static boolean DISQUALIFY_PLAYER_ON_ILLEGAL_ACTION_PLAYED =
false;

Finally, run the core.Game class.

###  7.2 Running multiple games

To run multiple games, choose one of the following methods, depending on
whether you wish to run with a fixed random seed for all games (1, passing
desired seed to the method), with a new random seed for each run (1, passing
null to the seed parameter in the method) or with fixed random seed for each
repetition of each game (2):

⬇

11. private static void runMany(List<GameType> gamesToPlay, List<AbstractPlayer> players, Long seed, int nRepetitions, ActionController ac, boolean randomizeParameters)

22. private static void runMany(List<GameType> gamesToPlay, List<AbstractPlayer> players, int nRepetitions, long[] seeds, ActionController ac, boolean randomizeParameters)

Use the chosen function and parameters in the core.Game.main function as
before, and run the Game class.

Note: Games within a specific category can be obtained by using the
corresponding Category method, e.g. GameType.Category.Strategy.getAllGames()
would return a list of all games in the “Strategy" category. Games using a
specific mechanic can be obtained by using the corresponding Mechanic method,
e.g. GameType.Mechanic.Cooperative.getAllGames() would return a list of all
cooperative games.

###  7.3 Running tournaments

Tournaments can be run using classes available in the evaluation package. At
the time of writing a round-robin tournament is available, which requires the
same setup as running single or multiple games in the core.Game class, but
will pit all the specified players against all others and run several
instances of the given games.

##  8 Future Directions

The presented framework opens up several directions of research and proposes a
variety of challenges for AI players, be it search/planning or learning
algorithms. Its main focus is to promote research into General Game AI that is
able to play many tabletop games at, or surpassing, human level. Relatedly,
the agents should be able to handle both competitive (most common testbeds in
literature), cooperative and even mixed games. For instance, a future planned
development is the inclusion of the game Betrayal at House on the Hill [14],
in which the players start off playing cooperatively to later split into teams
mid-way through the game, from which point on they are competing instead with
newly given team win conditions and rules. Most tabletop games include some
degree of hidden information (e.g. face-down decks of cards) and many more
players compared to traditional video-game AI testbeds, introducing higher
levels of uncertainty. However, such games often make use of similar
mechanics, even if in different forms: thus knowledge transfer would be a
fruitful area to explore, so that AI players can pick up new game rules more
easily based on previous experiences, similar to how humans approach the
problem. Some tabletop games further feature changing rules (e.g. Fluxx [15])
which would require highly adaptive AI players, able to handle changes in the
game engine itself, not only the game state. Many others rely on large amounts
of content and components, for which the process of creating new content or
modifying the current one for balance, improved synergies etc. could be
improved with the help of Procedural Content Generation methods (e.g. cards
for the game Magic the Gathering [16] were previously generated in a mixed-
initiative method by [17]).

Specific types of games can also be targeted by research, an option
highlighted by TAG’s categorisation and labelling of games and their
mechanics. Thus AI players could learn to specialise in games using certain
mechanics or in areas not yet explored, such as Role-Playing or Campaign games
(i.e. games played over several linked and progressive sessions). These games
often feature asymmetric player roles, with a special one highlighted (the
dungeon master) whose aim is to control the enemies in the game in order to
not necessarily win, but give the rest of the players the best experience
possible and the right level of challenge. Strategy AI research could see
important applications in this domain, as many tabletop games include elements
of strategic planning. Role-playing games focused more on the story created by
players (e.g. Dungeons and Dragons) rather than combat mechanics (e.g.
Gloomhaven) would also be a very engaging and difficult to approach topic for
AI players, where Natural Language Processing research could take an important
role.

The framework enables research into parameter optimisation: all parameter
classes for games, AI players or heuristics can implement the
ITunableParameters interface; parameters can then be automatically randomised,
or more intelligently tuned by any optimisation algorithm. This allows for
quick and easy exploration of various instances of a problem, a potential
increase in AI player performance, or adaptation of AI player behaviour to
user preference.

We have mentioned previously that the games implemented offer reduced
observations of the game state to the AI players, based on what they can
currently observe. These hidden information states (usually) do not keep a
history of what was previously revealed to a player. Instead, the AI players
should learn to memorise relevant information and build belief systems, as
humans would in a real-world context - a very interesting direction of
research encouraged by TAG.

Lastly, the framework includes the possibility for games to define their
states in terms of either vector observations (IVectorObservation), which
enables learning algorithms to be easily integrated with the framework; or
feature-based observations (IFeatureRepresentation), which allows for more
complex algorithms which can perform a search in the feature space of a game,
rather than the usual game state space approached.

## Acknowledgements

This work was partly funded by the EPSRC CDT in Intelligent Games and Game
Intelligence (IGGI) EP/L015846/1 and EPSRC research grant EP/T008962/1.

## References

  * [1] Seiji Kanai.  Love Letter.  Alderac Entertainment Group, 2012.
  * [2] Christophe Raimbault.  Colt Express.  Ludonaute, 2014.
  * [3] Merle Robbins.  Uno.  AMIGO, 1971.
  * [4] Domingo Cabrero et al.  Virus! El Dragón Azul, 2015.
  * [5] Matthew Inman et al.  Exploding Kittens.  Ad Magic, Inc., 2015.
  * [6] Matt Leacock.  Pandemic.  Z-Man Games, Inc., 2008.
  * [7] G. Engelstein and I. Shalev.  Building Blocks of Tabletop Game Design: An Encyclopedia of Mechanisms.  CRC Press LLC, 2019.
  * [8] Fantasy Flight Publishing, Inc.  Descent: Journeys in the Dark 2nd Edition.  Diamond Comic Distributors, 2012.
  * [9] Klaus-Jürgen Wrede.  Carcassonne.  Hans im Glück, 2000.
  * [10] Klaus Teuber.  The Settlers of Catan.  Mayfair Games, 1995.
  * [11] Raluca D. Gaina, Martin Balla, Alexander Dockhorn, Raul Montoliu, and Diego Perez-Liebana.  TAG: A Tabletop Games Framework.  In Proceedings of the AIIDE workshop on Experimental AI in Games, 2020.
  * [12] Diego Perez-Liebana, Spyridon Samothrakis, Simon M. Lucas, and Philipp Rolfshagen.  Rolling Horizon Evolution versus Tree Search for Navigation in Single-Player Real-Time Games.  In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), pages 351–358, 2013.
  * [13] Cameron B Browne, Edward Powley, Daniel Whitehouse, Simon M Lucas, Peter I Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton.  A Survey of Monte Carlo Tree Search Methods.  IEEE Transactions on Computational Intelligence and AI in games, 4(1):1–43, 2012.
  * [14] Bruce Glassco et al.  Betrayal at House on the Hill.  Avalon Hill Games, Inc., 2004.
  * [15] Andrew Looney and Kristin Looney.  Fluxx.  Looney Labs, 1997.
  * [16] Richard Garfield.  Magic: The Gathering.  Wizards of the Coast, 1993.
  * [17] Adam James Summerville and Michael Mateas.  Mystical Tutor: A Magic: The Gathering Design Assistant Via Denoising Sequence-To-Sequence Learning.  In Twelfth artificial intelligence and interactive digital entertainment conference, 2016.

Appendices

##  Appendix A A Game Example: Love Letter

This section shows some important code snippets from the implementation of
Love Letter in TAG.

###  A.1 Game State: LoveLetterGameState.java

Love Letter adds one game phase on top of default phases:

⬇

1public enum LoveLetterGamePhase implements IGamePhase {

2 Draw

3}

The game state class defines several different components:

⬇

1// List of cards in player hands

2List<PartialObservableDeck<LoveLetterCard>> playerHandCards;

3

4// Discarded cards

5List<Deck<LoveLetterCard>> playerDiscardCards;

6

7// Cards in draw pile

8PartialObservableDeck<LoveLetterCard> drawPile;

9

10// Cards in the reserve

11PartialObservableDeck<LoveLetterCard> reserveCards;

12

13// If true: player cannot be effected by any card effects

14boolean[] effectProtection;

15

16// Affection tokens per player

17int[] affectionTokens;

and a constructor that initializes the parameters, turn order and game phase:

⬇

1public LoveLetterGameState(AbstractParameters gameParameters, int nPlayers) {

2 super(gameParameters, new LoveLetterTurnOrder(nPlayers));

3 gamePhase = Draw;

4}

As indicated in Section 3.1, the following methods are implemented to retrieve
all components:

⬇

1@Override

2protected List<Component> _getAllComponents() {

3 List<Component> components = new ArrayList<>();

4 components.addAll(playerHandCards);

5 components.addAll(playerDiscardCards);

6 components.add(drawPile);

7 components.add(reserveCards);

8 return components;

9}

copy a game state for a particular player (defined by a playerId variable):

⬇

1@Override

2protected AbstractGameState _copy(int playerId) {

3 LoveLetterGameState llgs = new LoveLetterGameState(gameParameters.copy(),
getNPlayers());

4 llgs.drawPile = drawPile.copy();

5 llgs.reserveCards = reserveCards.copy();

6 llgs.playerHandCards = new ArrayList<>();

7 llgs.playerDiscardCards = new ArrayList<>();

8 for (int i = 0; i < getNPlayers(); i++) {

9 llgs.playerHandCards.add(playerHandCards.get(i).copy());

10 llgs.playerDiscardCards.add(playerDiscardCards.get(i).copy());

11 }

12 llgs.effectProtection = effectProtection.clone();

13 llgs.affectionTokens = affectionTokens.clone();

14

15 if (PARTIAL_OBSERVABLE && playerId != -1) {

16 // Draw pile, some reserve cards and other player’s hand is possibly
hidden. Mix all together and draw randoms

17 for (int i = 0; i < getNPlayers(); i++) {

18 if (i != playerId &&
llgs.playerHandCards.get(i).getDeckVisibility()[playerId]) {

19 // Hide!

20 llgs.drawPile.add(llgs.playerHandCards.get(i));

21 llgs.playerHandCards.get(i).clear();

22 }

23 }

24 for (int i = 0; i < llgs.reserveCards.getSize(); i++) {

25 if (!llgs.reserveCards.isComponentVisible(i, playerId)) {

26 // Hide!

27 llgs.drawPile.add(llgs.reserveCards.get(i));

28 }

29 }

30 Random r = new Random(llgs.getGameParameters().getRandomSeed());

31 llgs.drawPile.shuffle(r);

32 for (int i = 0; i < getNPlayers(); i++) {

33 if (i != playerId &&
llgs.playerHandCards.get(i).getDeckVisibility()[playerId]) {

34 // New random cards

35 for (int j = 0; j < playerHandCards.get(i).getSize(); j++) {

36 llgs.playerHandCards.get(i).add(llgs.drawPile.draw());

37 }

38 }

39 }

40 for (int i = 0; i < llgs.reserveCards.getSize(); i++) {

41 if (!llgs.reserveCards.isComponentVisible(i, playerId)) {

42 // New random card

43 llgs.reserveCards.setComponent(i, llgs.drawPile.draw());

44 }

45 }

46 }

47 return llgs;

48}

The function reset() sets the data structures of the game state to a
functional but empty state:

⬇

1@Override

2protected void _reset() {

3 gamePhase = Draw;

4 playerHandCards = new ArrayList<>();

5 playerDiscardCards = new ArrayList<>();

6 drawPile = null;

7 reserveCards = null;

8 effectProtection = new boolean[getNPlayers()];

9}

and the getScore() function uses a helper method to determine the heuristic
value of this state:

⬇

1@Override

2protected double _getScore(int playerId) {

3 return new LoveLetterHeuristic().evaluateState(this, playerId);

4}

This heuristic method is defined as follows in
games.loveletter.LoveLetterHeuristic.java:

⬇

1public double evaluateState(AbstractGameState gs, int playerId) {

2 LoveLetterGameState llgs = (LoveLetterGameState) gs;

3 LoveLetterParameters llp = (LoveLetterParameters) gs.getGameParameters();

4 Utils.GameResult playerResult = gs.getPlayerResults()[playerId];

5

6 if (playerResult == Utils.GameResult.LOSE)

7 return -1;

8 if (playerResult == Utils.GameResult.WIN)

9 return 1;

10

11 double cardValues = 0;

12

13 Random r = new Random(llgs.getGameParameters().getRandomSeed());

14 for (LoveLetterCard card:
llgs.getPlayerHandCards().get(playerId).getComponents()) {

15 if (card.cardType == LoveLetterCard.CardType.Countess) {

16 if (r.nextDouble() > COUNTESS_PLAY_THRESHOLD) {

17 cardValues += LoveLetterCard.CardType.Countess.getValue();

18 }

19 } else {

20 cardValues += card.cardType.getValue();

21 }

22 }

23

24 double maxCardValue = 1+llgs.getPlayerHandCards().get(playerId).getSize() *
LoveLetterCard.CardType.getMaxCardValue();

25 double nRequiredTokens = (llgs.getNPlayers()-1 < llp.nTokensWin.length ?
llp.nTokensWin[llgs.getNPlayers()-1] :

26 llp.nTokensWin[llp.nTokensWin.length-1]);

27 if (nRequiredTokens < llgs.affectionTokens[playerId]) nRequiredTokens =
llgs.affectionTokens[playerId];

28

29 return FACTOR_CARDS * (cardValues/maxCardValue) + FACTOR_AFFECTION *
(llgs.affectionTokens[playerId]/nRequiredTokens);

30}

###  A.2 Forward Model: LoveLetterForwardModel.java

The first game state is initialised in the _setup() method:

⬇

1@Override

2protected void _setup(AbstractGameState firstState) {

3 LoveLetterGameState llgs = (LoveLetterGameState)firstState;

4

5 // Set up all variables

6 llgs.drawPile = new PartialObservableDeck<>("drawPile", llgs.getNPlayers());

7 llgs.reserveCards = new PartialObservableDeck<>("reserveCards",
llgs.getNPlayers());

8 llgs.affectionTokens = new int[llgs.getNPlayers()];

9 llgs.playerHandCards = new ArrayList<>(llgs.getNPlayers());

10 llgs.playerDiscardCards = new ArrayList<>(llgs.getNPlayers());

11

12 // Set up first round

13 setupRound(llgs, null);

14}

This setup counts on the auxiliary method setupRound(). This method is
separate as it is called whenever a new round starts in the game, and it sets
up a round for the game, including the draw pile, reserve pile and starting
player hands:

⬇

1private void setupRound(LoveLetterGameState llgs, HashSet<Integer>
previousWinners) {

2 LoveLetterParameters llp = (LoveLetterParameters) llgs.getGameParameters();

3

4 // No protection this round

5 llgs.effectProtection = new boolean[llgs.getNPlayers()];

6

7 // Reset player status

8 for (int i = 0; i < llgs.getNPlayers(); i++) {

9 llgs.setPlayerResult(Utils.GameResult.GAME_ONGOING, i);

10 }

11

12 // Add all cards to the draw pile

13 llgs.drawPile.clear();

14 for (HashMap.Entry<LoveLetterCard.CardType, Integer> entry :
llp.cardCounts.entrySet()) {

15 for (int i = 0; i < entry.getValue(); i++) {

16 LoveLetterCard card = new LoveLetterCard(entry.getKey());

17 llgs.drawPile.add(card);

18 }

19 }

20

21 // Put one card to the side, such that player’s won’t know all cards in the
game

22 Random r = new Random(llgs.getGameParameters().getRandomSeed() +
llgs.getTurnOrder().getRoundCounter());

23 llgs.drawPile.shuffle(r);

24 llgs.reserveCards.clear();

25 llgs.reserveCards.add(llgs.drawPile.draw());

26

27 // In min-player game, N more cards are on the side, but visible to all
players at all times

28 if (llgs.getNPlayers() == GameType.LoveLetter.getMinPlayers()) {

29 boolean[] fullVisibility = new boolean[llgs.getNPlayers()];

30 Arrays.fill(fullVisibility, true);

31 for (int i = 0; i < llp.nCardsVisibleReserve; i++) {

32 llgs.reserveCards.add(llgs.drawPile.draw(), fullVisibility);

33 }

34 }

35

36 // Set up player hands and discards

37 llgs.playerHandCards.clear();

38 llgs.playerDiscardCards.clear();

39 for (int i = 0; i < llgs.getNPlayers(); i++) {

40 boolean[] visible = new boolean[llgs.getNPlayers()];

41 if (PARTIAL_OBSERVABLE) {

42 visible[i] = true;

43 } else {

44 Arrays.fill(visible, true);

45 }

46

47 // add random cards to the player’s hand

48 PartialObservableDeck<LoveLetterCard> playerCards = new
PartialObservableDeck<>("playerHand" + i, i, visible);

49 for (int j = 0; j < llp.nCardsPerPlayer; j++) {

50 playerCards.add(llgs.drawPile.draw());

51 }

52 llgs.playerHandCards.add(playerCards);

53

54 // create a player’s discard pile, which is visible to all players

55 Deck<LoveLetterCard> discardCards = new Deck<>("discardPlayer" + i, i);

56 llgs.playerDiscardCards.add(discardCards);

57 }

58

59 // Game starts with drawing cards

60 llgs.setGamePhase(Draw);

61

62 if (previousWinners != null) {

63 // Random winner starts next round

64 int nextPlayer = r.nextInt(previousWinners.size());

65 int n = -1;

66 for (int i: previousWinners) {

67 n++;

68 if (n == nextPlayer) {

69 llgs.getTurnOrder().setTurnOwner(i);

70 }

71 }

72 }

73

74 // Update components in the game state

75 llgs.updateComponents();

76}

The method that rolls the state forward with a given action, next(), is
implemented as follows:

⬇

1 @Override

2protected void _next(AbstractGameState gameState, AbstractAction action) {

3 // each turn begins with the player drawing a card after which one card will
be played

4 // switch the phase after each executed action

5 LoveLetterGameState llgs = (LoveLetterGameState) gameState;

6 action.execute(gameState);

7

8 IGamePhase gamePhase = llgs.getGamePhase();

9 if (gamePhase == Draw) {

10 llgs.setGamePhase(AbstractGameState.DefaultGamePhase.Main);

11 } else if (gamePhase == AbstractGameState.DefaultGamePhase.Main) {

12 llgs.setGamePhase(Draw);

13 llgs.getTurnOrder().endPlayerTurn(gameState);

14 checkEndOfRound(llgs);

15 } else {

16 throw new IllegalArgumentException("The game phase " + llgs.getGamePhase()
+

17 " is not know by LoveLetterForwardModel");

18 }

19}

Another important method in Love Letter’s forward model is the one in charge
of computing the available actions for a player. The snippet for that method
is shown below:

⬇

1@Override

2protected List<AbstractAction> _computeAvailableActions(AbstractGameState
gameState) {

3 LoveLetterGameState llgs = (LoveLetterGameState)gameState;

4 ArrayList<AbstractAction> actions;

5 int player = gameState.getTurnOrder().getCurrentPlayer(gameState);

6 if
(gameState.getGamePhase().equals(AbstractGameState.DefaultGamePhase.Main)) {

7 actions = playerActions(llgs, player);

8 } else if
(gameState.getGamePhase().equals(LoveLetterGameState.LoveLetterGamePhase.Draw))
{

9 // In draw phase, the players can only draw cards.

10 actions = new ArrayList<>();

11 actions.add(new DrawCard(llgs.drawPile.getComponentID(),
llgs.playerHandCards.get(player).getComponentID(), 0));

12 } else {

13 throw new IllegalArgumentException(gameState.getGamePhase().toString() + "
is unknown to LoveLetterGameState");

14 }

15

16 return actions;

17}

In Love Letter, the actions available depend on the card the player holds.
Here’s an extract of the method playerActions() that
_computeAvailableActions() uses:

⬇

1switch (playerDeck.getComponents().get(card).cardType) {

2 case Priest:

3 for (int targetPlayer = 0; targetPlayer < llgs.getNPlayers();
targetPlayer++) {

4 if (targetPlayer == playerID || llgs.getPlayerResults()[targetPlayer] ==
Utils.GameResult.LOSE)

5 continue;

6 actions.add(new PriestAction(playerDeck.getComponentID(),

7 playerDiscardPile.getComponentID(), card, targetPlayer));

8 }

9 break;

10

11 case Guard:

12 for (int targetPlayer = 0; targetPlayer < llgs.getNPlayers();
targetPlayer++) {

13 if (targetPlayer == playerID || llgs.getPlayerResults()[targetPlayer] ==
Utils.GameResult.LOSE)

14 continue;

15 for (LoveLetterCard.CardType type : LoveLetterCard.CardType.values())

16 actions.add(new GuardAction(playerDeck.getComponentID(),

17 playerDiscardPile.getComponentID(), card, targetPlayer, type));

18 }

19 break;

20

21 case Baron:

22 for (int targetPlayer = 0; targetPlayer < llgs.getNPlayers();
targetPlayer++) {

23 if (targetPlayer == playerID || llgs.getPlayerResults()[targetPlayer] ==
Utils.GameResult.LOSE)

24 continue;

25 actions.add(new BaronAction(playerDeck.getComponentID(),

26 playerDiscardPile.getComponentID(), card, targetPlayer));

27 }

28 break;

29

30 case Handmaid:

31 actions.add(new HandmaidAction(playerDeck.getComponentID(),

32 playerDiscardPile.getComponentID(), card));

33 break;

34

35 case Prince:

36 for (int targetPlayer = 0; targetPlayer < llgs.getNPlayers();
targetPlayer++) {

37 if (targetPlayer == playerID || llgs.getPlayerResults()[targetPlayer] ==
Utils.GameResult.LOSE)

38 continue;

39 actions.add(new PrinceAction(playerDeck.getComponentID(),

40 playerDiscardPile.getComponentID(), card, targetPlayer));

41 }

42 break;

43

44 case King:

45 for (int targetPlayer = 0; targetPlayer < llgs.getNPlayers();
targetPlayer++) {

46 if (targetPlayer == playerID || llgs.getPlayerResults()[targetPlayer] ==
Utils.GameResult.LOSE)

47 continue;

48 actions.add(new KingAction(playerDeck.getComponentID(),

49 playerDiscardPile.getComponentID(), card, targetPlayer));

50 }

51 break;

52

53 case Countess:

54 actions.add(new CountessAction(playerDeck.getComponentID(),

55 playerDiscardPile.getComponentID(), card));

56 break;

57

58 case Princess:

59 actions.add(new PrincessAction(playerDeck.getComponentID(),

60 playerDiscardPile.getComponentID(), card));

61 break;

###  A.3 Actions in Love Letter

Here are some actions that are exclusive to the game Love Letter. Actions can
extend other actions provided within TAG, enhancing their functionality. For
instance, the following DrawCard action modifies the execution of a regular
drawing card move to set certain values in the game state.

⬇

1public class DrawCard extends core.actions.DrawCard implements IPrintable {

2

3 public DrawCard(int deckFrom, int deckTo, int fromIndex) {

4 super(deckFrom, deckTo, fromIndex);

5 }

6

7 @Override

8 public boolean execute(AbstractGameState gs) {

9
((LoveLetterGameState)gs).setProtection(gs.getTurnOrder().getCurrentPlayer(gs),
false);

10 return super.execute(gs);

11 }

12

13 @Override

14 public String getString(AbstractGameState gameState) {

15 return "Draw a card and remove protection status.";

16 }

17

18 @Override

19 public String toString() {

20 return "Draw a card and remove protection status.";

21 }

22

23 @Override

24 public void printToConsole(AbstractGameState gameState) {

25 System.out.println(toString());

26 }

27

28 @Override

29 public AbstractAction copy() {

30 return new DrawCard(deckFrom, deckTo, fromIndex);

31 }

32}

This DrawCard action is further extended to provide particular effects for
each action in the game. For instance, in Love Letter, the execute() function
for playing a Baron card is implemented as follows:

⬇

1@Override

2public boolean execute(AbstractGameState gs) {

3 LoveLetterGameState llgs = (LoveLetterGameState)gs;

4 int playerID = gs.getTurnOrder().getCurrentPlayer(gs);

5 Deck<LoveLetterCard> playerDeck = llgs.getPlayerHandCards().get(playerID);

6 Deck<LoveLetterCard> opponentDeck =
llgs.getPlayerHandCards().get(opponentID);

7

8 // compares the value of the player’s hand card with another player’s hand
card

9 // the player with the lesser valued card will be removed from the game

10 if (llgs.isNotProtected(opponentID) && gs.getPlayerResults()[playerID] !=
Utils.GameResult.LOSE){

11 LoveLetterCard opponentCard = opponentDeck.peek();

12 LoveLetterCard playerCard = playerDeck.peek();

13 if (opponentCard != null && playerCard != null) {

14 if (opponentCard.cardType.getValue() < playerCard.cardType.getValue())

15 llgs.killPlayer(opponentID);

16 else if (playerCard.cardType.getValue() < opponentCard.cardType.getValue())

17 llgs.killPlayer(playerID);

18 } else {

19 throw new IllegalArgumentException("player with ID " + opponentID + " was
targeted using a Baron card" +

20 " but one of the players has now cards left.");

21 }

22 }

23

24 return super.execute(gs);

25}

Other actions are simpler, such as playing a Guard card:

⬇

1@Override

2public boolean execute(AbstractGameState gs) {

3 LoveLetterGameState llgs = (LoveLetterGameState)gs;

4 Deck<LoveLetterCard> opponentDeck =
llgs.getPlayerHandCards().get(opponentID);

5

6 // guess the opponent’s card and remove the opponent from play if the guess
was correct

7 if (llgs.isNotProtected(opponentID)){

8 LoveLetterCard card = opponentDeck.peek();

9 if (card.cardType == this.cardType) {

10 llgs.killPlayer(opponentID);

11 }

12 }

13 return super.execute(gs);

14}

or a Priest card:

⬇

1@Override

2public boolean execute(AbstractGameState gs) {

3 LoveLetterGameState llgs = (LoveLetterGameState)gs;

4 int playerID = gs.getTurnOrder().getCurrentPlayer(gs);

5 PartialObservableDeck<LoveLetterCard> opponentDeck =
llgs.getPlayerHandCards().get(opponentID);

6

7 // Set all cards to be visible by the current player

8 if (((LoveLetterGameState) gs).isNotProtected(opponentID)){

9 for (int i = 0; i < opponentDeck.getComponents().size(); i++)

10 opponentDeck.setVisibilityOfComponent(i, playerID, true);

11 }

12

13 return super.execute(gs);

14}