# Artificial Intelligence and Algorithmic Price Collusion in Two-sided
Markets††thanks: This work was partially supported by NSF award DMS-2124913.

Cristian Chica, Yinglong Guo, and Gilad Lerman School of Mathematics,
University of Minnesota. Email addresses<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Algorithmic price collusion facilitated by artificial intelligence (AI)
algorithms raises significant concerns. We examine how AI agents using
$Q$-learning engage in tacit collusion in two-sided markets. Our experiments
reveal that AI-driven platforms achieve higher collusion levels compared to
Bertrand competition. Increased network externalities significantly enhance
collusion, suggesting AI algorithms exploit them to maximize profits. Higher
user heterogeneity and greater utility from outside options generally reduce
collusion, while higher discount rates increase it. Tacit collusion remains
feasible even at low discount rates. To mitigate collusive behavior and inform
potential regulatory measures, we propose incorporating a penalty term in the
$Q$-learning algorithm.

Keywords: Artificial Intelligence (AI), $Q$-learning, Price Collusion, Network
Externalities, Two-sided Markets

JEL Codes: D43, D83, L13

##  1 Introduction

Algorithmic price collusion occurs when economic agents set prices using
artificial intelligence (AI) algorithms. Through repeated interactions, these
agents learn that tacit collusion is optimal, as noted by Calvano et al.,
2020a .111Tacit collusion happens when firms coordinate behavior without
explicit communication. Du and Tanriverdi, (2023); Bertomeu et al., (2021)
have documented evidence of this in the U.S. multihospital system and
automotive industry. Economists and antitrust authorities have expressed
significant concerns about this form of collusion. The OECD, (2017) raised
concerns about pricing algorithms learning to collude via tacit coordination.
Assad et al., (2024) suggested that algorithmic pricing in Germany’s retail
gasoline market increased price margins by approximately 15%. U.S. Senator Amy
Klobuchar introduced the S.3686 - Preventing Algorithmic Collusion Act of 2024
to curb anticompetitive behavior through algorithmic pricing using nonpublic
competitor data.

Recent experiments (Calvano et al., 2020b , Klein, (2021)) have demonstrated
that collusion can be achieved in Bertrand and Stackelberg competition models
by simulated economic agents using $Q$-learning, a benchmark reinforcement
learning algorithm. Building on these findings, this study experimentally
investigates AI-driven platforms using $Q$-learning in a repeated two-sided
platform competition game. We illustrate how these AI agents facilitate
collusion compared to Bertrand competition. Our focus is particularly on the
impact of network externalities on collusion.

In our model of repeated two-sided platform competition, multiple horizontally
differentiated platforms compete to serve buyers and sellers, collectively
referred to as users. These platforms repeatedly interact and independently
choose prices using $Q$-learning, with the last period price as the state
variable. This implies that platforms have bounded memory and employ one-
memory strategies (Barlo et al.,, 2009). In each repetition, users can choose
to join one of the platforms or opt for the outside option. Buyers who join a
platform receive network externality benefits proportional to the number of
buyers (within-side externalities) and sellers (cross-side externalities) on
the same platform. Sellers who join the market receive both types of
externalities as well.

Our experiments show that even with zero network externalities, AI-driven
platforms achieve higher collusion levels compared to those reported by
Calvano et al., 2020b  for Bertrand competition. This is likely due to the
larger action space, which allows more information exchange. Furthermore,
increased network externalities lead to significantly high collusion levels,
suggesting AI-driven platforms can leverage these externalities to boost
profits. In particular, algorithmic pricing can increase collusion in markets
with significant positive within-side externalities (e.g., online/cloud
gaming) and positive cross-side externalities (e.g., video streaming, social
media).

Our findings indicate that higher user heterogeneity or greater utility from
the outside option generally decrease collusion levels, except in certain
local regions. In contrast, collusion levels typically rise with higher
discount rates, especially in the presence of significant network
externalities. Additionally, tacit collusion remains feasible even at very low
discount factors. This contrasts with traditional literature on firms’
collusion without AI agents, which suggests that collusion is feasible only at
high discount factors (Tirole,, 1988; Obara and Zincenko,, 2017).

Finally, we propose reducing collusion through a penalty term in the
$Q$-learning algorithm.

Related Literature. There is a growing literature on algorithmic price
collusion, with a particular emphasis on using numerical simulations to show
that $Q$-learning results in tacit collusion. Waltman and Kaymak, (2008)
showed that firms using $Q$-learning in repeated Cournot oligopoly games
produce lower quantities than the competitive Nash equilibrium. Calvano et
al., 2020b  showed that $Q$-learning firms choose high prices in repeated
Bertrand games and learn strategies consistent with tacit collusion. Similar
work was done by Klein, (2021) for repeated Stackelberg games. Assad et al.,
(2024) is the first work that uses real-life data to show that firms may
increase price margins with the adoption of algorithmic pricing. Our work
extends the numerical understanding of algorithmic pricing, particularly in
two-sided markets with network externalities.

Studies by Johnson et al., (2023) and Brero et al., (2022) on single platforms
with AI-driven sellers show how platform-designed rules can promote
competition and reduce collusion. Nevertheless, this setting does not apply to
ours, where multiple platforms apply AI algorithms.

Our model of repeated two-sided platform competition uses the model in Chica
et al., (2023), which, in turn, builds upon previous models by White and Weyl,
(2016); Tan and Zhou, (2021); Chica et al., (2021). These models analyze
network externality effects on equilibrium outputs. Our simulations use their
insights to study the impact of these externalities on the collusive levels.

Ruhmer, (2010) finds that higher cross-side externalities make collusion
harder to sustain, when following the model of Armstrong, (2006) without AI
agents. This is consistent with our numerical results, even though we consider
algorithmic pricing and follow the model of Chica et al., (2023).

Theoretical work in economics on algorithmic price collusion includes Brown
and MacKay, (2023), which demonstrates that simple pricing algorithms can
elevate price levels. Additionally, Arslantas et al., (2024) illustrates how a
sophisticated agent can exploit another agent using a naive version of
$Q$-learning, provided the former agent knows the algorithm being used.

##  2 Review of Our Economic Framework

We introduce the economics framework used in our experiments. Section 2.1
presents the baseline platform competition game. Section 2.2 extends the
latter model to an infinite repeated game.

###  2.1 The Baseline Platform Competition Game

The baseline platform competition game consists of two stages. In stage I, a
set of horizontally differentiated platforms strategically choose prices to
maximize profits. In stage II, given the prices determined by the platforms,
users on each of the two sides of the market choose whether to participate or
not and if they participate they also choose which platform to join. The
solution concept for the baseline game is backward induction. More
specifically, $N$ platforms provide service options for users on two sides of
a market, buyers and sellers. Users in these two sides of a market are denoted
with $k\in\\{b,s\\}$, where $b$ and $s$ represent buyers and sellers,
respectively. These users have $N+1$ choices, where $N\geq 2$. They can either
opt out of the market by choosing the outside option, or join one of the $N$
horizontally differentiated platforms, each one denoted with
$i\in[N]:=\\{1,\dots,N\\}$. The users on side $k$ opting out of the market
receive a deterministic outside option utility $u^{(0)}_{k}\in\mathbb{R}$. The
users on side $k$ joining platform $i\in[N]$ receive a deterministic utility

$u^{(i)}_{k}:=\phi_{k}(x^{(i)}_{b},x^{(i)}_{s})-p^{(i)}_{k},$

where $p^{(i)}_{k}$ is the price paid by the user to access services provided
by the platform $i$; $x^{(i)}_{k}$ is the total mass of users on side $k$
joining platform $i$; and

$\phi_{k}(x^{(i)}_{b},x^{(i)}_{s}):=\phi_{kb}x^{(i)}_{b}+\phi_{ks}x^{(i)}_{s},\
\text{ with }\ \phi_{kb},\phi_{ks}\in\mathbb{R},$

is the network externality function that captures the network benefits users
enjoy by joining platform $i$. The network externalities are captured by the
following linear transformation

$(\phi_{b}(x_{b}^{(i)},x_{s}^{(i)}),\phi_{s}(x_{b}^{(i)},x_{s}^{(i)}))^{T}=\Phi{\bm{x}}^{(i)},\
\text{ where }\ \Phi=\left[\begin{array}[]{cc}\phi_{bb}&\phi_{bs}\\\
\phi_{sb}&\phi_{ss}\end{array}\right].$ (1)

To save space, we write $\Phi=[\phi_{bb},\phi_{bs};\phi_{sb},\phi_{ss}]$ when
specifying choices of $\Phi$. The endogenous mass of users on each side of the
market subscribed to platform $i$ is denoted by
${\bm{x}}^{(i)}:=(x_{b}^{(i)},x_{s}^{(i)})\in[0,1]^{2}$ and the mass of users
not participating in the market is denoted by
${\bm{x}}^{(0)}:=(x_{b}^{(0)},x_{s}^{(0)})\in[0,1]^{2}$. Assuming all users
have Gumbel-distributed idiosyncratic preferences with parameters
$(\mu_{k},\beta_{k})$, $\mu_{k}\in\mathbb{R}$ and $\beta_{k}>0$,222The Gumbel-
distribution parameter $\beta_{k}$ measures the standard deviation of
$\epsilon_{k}^{i}$ and it captures the degree of heterogeneity in users’
tastes. Unlike $\beta_{k}$, $\mu_{k}$ does not affect the equilibrium output
of the model (see (2)). the quantities $x_{k}^{(i)}$ are determined through a
maximization process conducted by users on side $k$ who solve the following
equations (see Chica et al., (2023)):333We note that the model presented here
is equivalent to a model in which users have Logistic-distributed preferences
for the platforms, and the outside option utility is deterministic.

$x_{k}^{(i)}=1-\left({1+\exp\left(u^{(i)}_{k}/\beta_{k}-\ln\left(\sum\limits_{j=0,1,\cdots,N,j\neq
i}e^{u^{(j)}_{k}/\beta_{k}}\right)\right)}\right)^{-1}\ i\in[N]\cup\\{0\\},\
k\in\\{b,s\\}.$ (2)

The platforms incorporate (2) into their profit maximization problem as
follows, where $\pi^{(i)}$ denotes the profit of platform $i$ and
$\Pi_{\text{tot}}$ denotes the total profits of $N$ colluding platforms:

(i) when competing, they solve

$\max_{\\{p^{(i)}_{b},p_{s}^{(i)}\\}}\text{
}\pi^{(i)}(p_{b}^{(i)},p^{(i)}_{s}),\text{ where
}\pi^{(i)}(p_{b}^{(i)},p^{(i)}_{s}):=x^{(i)}_{b}p^{(i)}_{b}+x^{(i)}_{s}p^{(i)}_{s};$
(3)

(ii) when colluding, they solve

$\max_{\begin{subarray}{c}p_{b},p_{s}\end{subarray}}\
\Pi_{\text{tot}}(p_{b},p_{s}),\text{ where
}\Pi_{\text{tot}}(p_{b},p_{s}):=\sum_{i=1}^{N}\left(x^{(i)}_{b}p_{b}+x^{(i)}_{s}p_{s}\right).$
(4)

The maximizer of (3) is called the competitive Nash Equilibrium (CNE) and the
corresponding equilibrium quantities are denoted by $p^{(i),*}_{k}$ and
$x^{(i),*}_{k}$ for $i\in[N]\cup\\{0\\}$ and $k\in\\{b,s\\}$. The maximizer of
(4) is called the collusive equilibrium (CE) and the corresponding equilibrium
quantities are denoted by $p^{(i),\text{C}}_{k}$ and $x^{(i),\text{C}}_{k}$,
$i\in[N]\cup\\{0\\}$, $k\in\\{b,s\\}$. In the symmetric equilibrium,
$p_{k}^{(i),*}=p_{k}^{*}$, $p_{k}^{(i),\text{C}}=p_{k}^{\text{C}}$,
$x_{k}^{(i),*}=x_{k}^{*}$ and $x_{k}^{(i),\text{C}}=x_{k}^{\text{C}}$ for all
$i\in[N]$. Propositions 3.2 and 3.4 in Chica et al., (2023) provide first-
order conditions for solving $p_{k}^{\ast}$ and $p_{k}^{\text{C}}$. Similarly,
Propositions 3.3 and 3.5 in the same work provides sufficient conditions for
the existence and uniqueness of symmetric CNE and CE equilibria. The symmetric
CNE and CE individual platform profits are respectively defined by

$\pi^{*}:=\pi^{(i)}(p_{b}^{*},p_{s}^{*})\ \textnormal{ and }\
\pi^{\text{C}}:=\frac{\Pi_{tot}(p_{b}^{\text{C}},p_{s}^{\text{C}})}{N}.$ (5)

###  2.2 The Infinite Repeated Game

The infinite repeated game consists of a sequence of games, where at time
$t\in{\mathbb{N}}\cup\\{0\\}$, platforms and users interact following the
rules of the baseline platform competition game, introduced in Section 2.1,
and additional ones. At each time step $t$, we use the same notations as
above, but with a subscript $t$. We assume that users on all sides are myopic,
i.e., they make decisions to maximize the utility at current time $t$ by
solving (2) which depends solely on the current prices observed in the market.
We further assume that platforms compete and act strategically and determine
the charged prices to maximize the total discounted future rewards at every
step $t$ based on the past market states, which we clarify next after
introducing some notation and definitions. Given a discounting rate
$\delta\in(0,1)$, we define the total discounted future rewards at time $t$
for platform $i$ by

$r_{t}^{(i)}:=\sum_{\tau=0}^{\infty}\delta^{\tau}{\mathbb{E}}[\pi_{t+\tau}^{(i)}],\
\textnormal{ where }\
\pi_{t+\tau}^{(i)}=\sum_{k\in\\{b,s\\}}x_{t+\tau,k}^{(i)}p_{t+\tau,k}^{(i)}$
(6)

and $x_{t+\tau,k}^{(i)}$ is the mass of users on side $k$ joining platform $i$
at time $t+\tau$, and $p_{t+\tau,k}^{(i)}$ is the price that platform $i$
charges on side $k$ at time $t+\tau$. Note that from (2), $x_{t+\tau,k}^{(i)}$
is a function of all platforms prices at time $t+\tau$. Furthermore, this
observation and (3) imply that $\pi_{t+\tau}^{(i)}$ can be written as a
function of all platforms prices at time $t+\tau$, that is,

$\pi_{t+\tau}^{(i)}=\pi^{(i)}({\bm{p}}_{t+\tau}^{(1)},{\bm{p}}_{t+\tau}^{(2)},\cdots,{\bm{p}}_{t+\tau}^{(N)}).$
(7)

From the viewpoint of platform $i$, the policies of all other platforms are
unknown, so their present and future prices are considered random
variables.444For $\tau=0$, the market share for platform $i$, $x_{t,k}^{(i)}$,
is a random variable since it depends on all the prices charged by all other
platforms (see (2)). For $\tau>0$, the future policies of platforms are random
variables. Since the future prices depend on the future states, which depend
on the platform policies, they are also random variables.

Each platform needs to strategically charge prices in order to maximize the
expected total discounted future rewards (6). A common method to optimize the
expectation of the total discounted future reward is Q-learning, which we
introduce in § 3.1.

For $t\in{\mathbb{N}}\cup\\{0\\}$, denote by
${\bm{p}}_{t}:=({\bm{p}}_{t}^{(1)},\dots,{\bm{p}}_{t}^{(N)})$ the vector of
prices chosen by the $N$ platforms at time $t$, where
${\bm{p}}_{t}^{(i)}:=(p_{t,b}^{(i)},p_{t,s}^{(i)})$, $i\in[N]$. For $L\geq 1$,
denoting previous time steps, and $t\geq L$, let

${\bm{s}}_{t,L}:=({\bm{p}}_{t-L},{\bm{p}}_{t-L+1},\dots,{\bm{p}}_{t-1})\
\text{ and }\ H_{t,L}=\\{{\bm{s}}_{t,L}\in\mathbb{R}^{2LN}\\},$

where one typically constrains $H_{t,L}$ to be a discrete set (see Section
3.2). The problem for each platform is to identify a policy

$\sigma_{t}^{(i)}:H_{t,L}\longrightarrow{\mathbb{R}}^{2}$

that inputs the current observed state ${\bm{s}}_{t,L}$ and outputs the
charged price ${\bm{p}}_{t}^{(i)}\equiv(p_{t,b}^{(i)},p_{t,s}^{(i)})$. During
this infinitely repeated game, at each time step $t$, each platform $i$
updates the policy $\sigma_{t}^{(i)}$ based on the observed data (the states
and rewards) to refine this policy that aims to maximize the expected total
discounted future reward. Moreover, at each time step $t$, each platform $i$
uses the policy $\sigma_{t}^{(i)}$ to determine the charged prices
${\bm{p}}^{(i)}_{t}$. A particular framework for doing this is discussed in
Section 3. Figure 1 demonstrates the different stages of the infinite repeated
game.

Platform 1 charges $\displaystyle p_{0,k}^{(1)}$ Platform 2 charges
$\displaystyle p_{0,k}^{(2)}$ time $\displaystyle 0$ users
$\displaystyle(k\in\\{b,s\\})$ solve for $\displaystyle x_{0,k}^{(j)}$ (see
(2)) Platforms receive $\displaystyle\pi_{0,k}^{(j)}$ (see (3)) $\displaystyle
t=1,2,\dotsc$time $\displaystyle t+\tau$ users $\displaystyle(k\in\\{b,s\\})$
solve for $\displaystyle x_{t+\tau,k}^{(j)}$ (see (2)) Platform 1 updates
policy $\displaystyle\sigma_{t+\tau}^{(1)}$ and charges $\displaystyle
p_{t+\tau,k}^{(1)}=\sigma_{t}^{(1)}(s_{t}^{(1)})$ Platform 2 updates policy
$\displaystyle\sigma_{t+\tau}^{(2)}$ and charges $\displaystyle
p_{t+\tau,k}^{(2)}=\sigma_{t}^{(2)}(s_{t}^{(2)})$ Platforms receive
$\displaystyle\pi_{t+\tau,k}^{(j)}$ (see (3)) $\displaystyle\infty$ Figure 1:
Demonstration of the infinite repeated game framework

##  3 Simulation Framework

We first review the framework of multi-agent reinforcement learning in Section
3.1. We then detail our simulation setting in Section 3.2, building upon the
framework developed in Section 3.1.

###  3.1 Preliminaries: Multi-agent Reinforcement Learning

Multi-agent reinforcement learning considers $N$ agents interacting in a
dynamic environment. At each time $t\in{\mathbb{N}}$, each agent $i\in[N]$
observes a state $s^{(i)}_{t}\in{\mathcal{S}}$ and takes an action
$a_{t}^{(i)}\in{\mathcal{A}}$, based on this observed state and following a
policy $\sigma^{(i)}_{t}:{\mathcal{S}}\longrightarrow{\mathcal{A}}$, which
could be either deterministic or stochastic. Here, ${\mathcal{S}}$ denotes the
state space and ${\mathcal{A}}$ denotes the action space. Let
$\sigma_{t}=(\sigma^{(1)}_{t},\sigma^{(2)}_{t},\cdots,\sigma^{(N)}_{t})$ and
$A_{t}:=(a_{t}^{(1)},\cdots
a_{t}^{(N)})=(\sigma_{t}^{(1)}(s_{t}^{(1)}),\cdots\sigma_{t}^{(N)}(s_{t}^{(N)}))$
denote all policies and actions, respectively, at time $t$. We denote by
$a_{t}^{(-i)}$, ${\bm{p}}_{t}^{(-i)}$, and $\sigma^{(-i)}_{t}$ the respective
vectors of all actions $a_{t}^{(j)}$, prices ${\bm{p}}_{t}^{(j)}$, and
policies $\sigma^{(j)}_{t}$with $j\neq i$. The agent collects a reward
$\pi_{t}^{(i)}$, which is a random variable conditioned on the state
$s_{t}^{(i)}$ and actions $A_{t}$. The state in the next time,
$s_{t+1}^{(i)}$, is a random variable conditioned on the state $s_{t}^{(i)}$
and the actions $A_{t}$ taken by all the agents in the current time $t$. Given
a discounting rate $\delta\in(0,1)$, at each time, each agent aims to find a
policy in order to maximize the following expectation of the total discounted
future reward given all observed states at time $t$:

$\sum_{\tau=0}^{\infty}{\mathbb{E}}_{\pi,s,\sigma}\left[\delta^{\tau}\pi_{t+\tau}^{(i)}(s_{t+\tau}^{(i)},A_{t+\tau})\right].$
(8)

The expectation is needed due to the randomness in the rewards, the future
states, and the future actions of all the agents.

Q-learning is a classic method for finding the policy that maximizes (8). It
uses the $Q$-function of agent $i$ at state $s$ given an action $a$, which is
defined by

$\begin{split}&Q^{(i)}(s,a,\sigma^{(i)};\sigma^{(-i)}):=\\\
&\sum_{\tau=0}^{\infty}{\mathbb{E}}_{\pi,s,\sigma}\left[\delta^{\tau}\pi_{t+\tau}^{(i)}\big{|}s_{t}^{(i)}={\bm{s}},a_{t}^{(i)}=a,a_{t+u}^{(i)}=\sigma^{(i)}(s_{t+u}^{(i)}),\
u\geq 1,\ a_{t+v}^{(-i)}=\sigma^{(-i)}(s_{t+v}),\ v\geq 0\right].\end{split}$
(9)

Note that (9) differs from (8) by having agent $i$ follow the given action $a$
at time $t$ instead of the policy $\sigma^{(i)}$, whereas in both formulations
all other agents at times $t,t+1,\ldots,$ and agent $i$ at times
$t+1,t+2,\ldots,$ follow their policies.

We denote an optimal policy for agent $i$ by $\sigma^{(i)\ast}$, which is hard
to find. Q-learning overcomes this difficulty by carefully estimating the
solution $Q^{(i)\ast}(s_{t},a_{t};\sigma^{(-i)})$ to the following Bellman
equation

$Q^{(i)\ast}(s_{t},a_{t};\sigma^{(-i)})={\mathbb{E}}_{\pi}\left[\pi(s_{t},A_{t})\right]+\delta\max_{a^{\prime}}{\mathbb{E}}_{s_{t+1}}\left[Q^{(i)\ast}(s_{t+1},a^{\prime};\sigma^{(-i)})|A_{t}\right].$
(10)

It then estimates $\sigma^{(i)\ast}$ using the following relationship between
$\sigma^{(i)\ast}$ and $Q^{(i)\ast}(x,a;\sigma^{(-i)})$:

$\sigma^{(i)\ast}(s)=\operatorname*{arg\,max}_{a}Q^{(i)\ast}(s,a;\sigma^{(-i)}).$
(11)

We detail the methods for estimating the $Q^{\ast}$-function in (10) in the
following section.

###  3.2 The Simulation Setup

We consider a market with two platforms, that is, we set $N=2$.555Since our
model assumes two sides of the market, each of the $N$ platforms must choose
two prices. At each stage, our simulation estimates $2N$ different prices, and
there are $M^{2N}$ possibilities for the vector of prices, where $M$ is the
size of the set of price choices available to each platform. To make our
simulations feasible, we choose $N=2$. At time $t$, each platform
$i\in\\{1,2\\}$ observes the following state
${\bm{s}}_{t}^{(i)}:={\bm{p}}_{t-1}$, which contains prices at the previous
step. Platform $i$ determines its prices ${\bm{p}}^{(i)}_{t}$ based on the
observed state ${\bm{s}}_{t}^{(i)}$. At each time $t$, after all platforms
have chosen prices ${\bm{p}}_{t}^{(i)}$, they receive the reward
$\pi_{t}^{(i)}=\sum_{k\in\\{b,s\\}}x_{t,k}^{(i)}p_{t,k}^{(i)}$, where
$x_{t,k}^{(i)}$ is solved using (2).

To simplify the computation, we allow platforms to choose from a discrete set
of $M$ prices. While it is common to expect that
$p_{k}^{\ast}<p_{k}^{\text{C}}$,666Note that Proposition 4.11 in Chica et al.,
(2023) provides sufficient conditions to guarantee that
$p^{\ast}_{k}<p^{\text{C}}_{k}$. our model also allows the case
$p^{\text{C}}_{k}<p^{*}_{k}$. We further introduce the parameter
$\epsilon=0.1$ so the lowest price is slightly lower than
$\min(p_{k}^{*},p^{\text{C}}_{k})$ and the highest one is slightly higher than
$\max(p_{k}^{*},p^{\text{C}}_{k})$. For each $k\in\\{b,s\\}$, our set of
prices is

$\displaystyle{\mathcal{P}}_{k}:=\Big{\\{}p_{k}^{*}-\epsilon(p_{k}^{\text{C}}-p_{k}^{*})+\frac{j}{M-1}(1+2\epsilon)(p_{k}^{\text{C}}-p_{k}^{*})\Big{|}j=0,\cdots,M-1\Big{\\}}.$
(12)

Note that the cardinality of the price space $|{\mathcal{P}}_{k}|$ is $M$ for
both $k=b$ and $k=s$. The overall state space (for both platforms) and the
action space for each platform are respectively defined by

${\mathcal{S}}:=({\mathcal{P}}_{b}\times{\mathcal{P}}_{s})\times({\mathcal{P}}_{b}\times{\mathcal{P}}_{s})\
\textnormal{ and}\ {\mathcal{A}}:={\mathcal{P}}_{b}\times{\mathcal{P}}_{s}.$
(13)

We note that the size of the state space is $|{\mathcal{S}}|=M^{4}$ and the
size of the action space is $|{\mathcal{A}}|=M^{2}$.

Platform policy: We denote the estimation at time $t$ of
$Q^{(i)\ast}({\bm{s}},{\bm{a}};\sigma^{(-i)})$ by
$Q^{(i)}_{t}({\bm{s}},{\bm{a}})$, where ${\bm{s}}\in{\mathcal{S}}$,
${\bm{a}}\in{\mathcal{A}}$ and $i\in\\{1,2\\}$ indexes the platform.
Q-learning alternately estimates $Q_{t}^{(i)}$ and the stochastic policies at
time $t$. We first assume that $Q_{t}^{(i)}$ is known and show how the
platforms determine the stochastic policy at time $t$. We then explain the
Q-learning estimation of $Q_{t}^{(i)}$. Instead of directly computing the
policy as the maximum value in (11), Q-learning computes a softmax value using
a temperature parameter ${\mathcal{T}}_{t}$. For this purpose, at time $t$ and
given a state ${\bm{s}}_{t}^{(i)}\in{\mathcal{S}}$ and the $Q^{\ast}$-function
estimate, $Q^{(i)}_{t}$, the policy of platform $i$ is the Boltzmann
probability distribution:

$P({\bm{a}}_{t}^{(i)}={\bm{a}}|{\bm{s}}_{t}^{(i)})={\exp\left(Q_{t}^{(i)}({\bm{s}}_{t}^{(i)},{\bm{a}})/{\mathcal{T}}_{t}\right)}/{\sum\limits_{{\bm{a}}^{\prime}\in{\mathcal{A}}}\exp\left(Q_{t}^{(i)}({\bm{s}}_{t}^{(i)},{\bm{a}}^{\prime})/{\mathcal{T}}_{t}\right)}.$
(14)

We remark that all platforms independently determine their prices based on
(14).777As ${\mathcal{T}}_{t}$ decreases, (14) increasingly focuses on the
optimal action based on $Q_{t}^{(i)}$. When ${\mathcal{T}}_{t}\to 0$, the
policy randomly selects between the actions that yield the maximal reward
$Q_{t}^{(t)}$ with uniform probabilities. In the simulation, we set
${\mathcal{T}}_{0}=1000$ to encourage exploration of possible actions,
gradually decreasing it towards $0$ to exploit optimal actions.

Q-learning estimation: At each time step, after determining the price
following (14), platform $i$ collects the reward $\pi_{t}^{(i)}$ defined by
(7). Next, platform $i$ updates the estimated values of the $Q^{*}$-function
at the given state ${\bm{s}}_{t}^{(i)}$ and the selected action
${\bm{a}}_{t}^{(i)}$ with a learning rate $\alpha$ as follows:

$Q^{(i)}_{t+1}({\bm{s}}_{t}^{(i)},{\bm{a}}_{t}^{(i)}):=(1-\alpha)Q^{(i)}_{t}({\bm{s}}_{t}^{(i)},{\bm{a}}_{t}^{(i)})+\alpha\left(\pi_{t}^{(i)}+\delta\max_{{\bm{a}}}Q^{(i)}_{t}({\bm{s}}_{t}^{(i)},{\bm{a}})\right).$
(15)

We remark that (15) is an approximation of (10) (see Watkins and Dayan,
(1992)).

We initialize the $Q^{\ast}$-function at ${\bm{s}}\in{\mathcal{S}}$ and
${\bm{a}}\in{\mathcal{A}}$ assuming that in all future states platform $i$
charges ${\bm{a}}$ and all other platforms charge the prices in ${\bm{s}}$.
Therefore, for platform $i$, state ${\bm{s}}$, a given action ${\bm{a}}$ and
the price vector for platform $j\neq i$, which we denote by ${\bm{p}}^{(j)}$
and it is part of the state ${\bm{s}}$, the $Q^{\ast}$-function for platform
$i$ is initialized by

$Q^{(i)}_{0}({\bm{s}},{\bm{a}})=\sum_{\tau=0}^{\infty}\delta^{\tau}\pi^{(i)}({\bm{a}},{\bm{p}}^{(j)})=\frac{\pi^{(i)}({\bm{a}},{\bm{p}}^{(j)})}{1-\delta}.$
(16)

Parameter setup: We choose exponentially decaying temperature parameter
${\mathcal{T}}_{t}={\mathcal{T}}_{0}\lambda^{t}$ with
${\mathcal{T}}_{0}:=1000/(1-\delta)$ and $\lambda=1-10^{-7}$. This choice
encourages exploration in the early stages and exploits optimality in the
later stages. We choose both the idiosyncratic preference parameters and the
outside option utilities to be the same on both sides of the market. Therefore
we denote $\beta_{k}=\beta_{b}\equiv\beta_{s}$ and
$u_{k}^{(0)}=u_{b}^{(0)}\equiv u_{s}^{(0)}$. We set the learning rate
$\alpha=0.15$, discount rate $\delta=0.05$, idiosyncratic preferences
$\beta_{k}=1$, and outside option utility $u_{k}^{(0)}=-2$ for each
$k\in\\{b,s\\}$. We choose a small value for $\delta$, compared to the choice
of the same parameter in Calvano et al., 2020b ; Klein, (2021), to emphasize
that in our setting collusion is already present with a very small discount
rate.

Reporting metric: We define the collusive level of platform $i$ at time $t$ as

$\Delta^{(i)}_{t}:=\frac{\pi_{t}^{(i)}-\pi^{*}}{\pi^{\text{C}}-\pi^{*}},$ (17)

where we recall (see (5)) $\pi^{*}$ and $\pi^{\text{C}}$ respectively denote
the CNE and CE equilibrium profits of the baseline platform competition game.
When $\Delta_{t}^{(i)}=0$, platform $i$’s reward at time $t$ equals the CNE
level, $\pi^{*}$; whereas when $\Delta_{t}^{(i)}=1$, it equals the CE level,
$\pi^{\text{C}}$. Each simulation runs $T=5\times 10^{8}$ iterations and we
report the overall collusive level in the last $K=1,000$ steps as follows:

$\tilde{\Delta}:=\frac{1}{KN}\sum_{s=0}^{K-1}\sum_{i=1}^{N}\Delta^{(i)}_{T-s}.$
(18)

##  4 Experimental Results

We report extensive numerical experiments using the setup of Section 3.2.
Section 4.1 investigates the general dependence of $\tilde{\Delta}$, defined
in (18), on the externality matrix $\Phi$. Section 4.2 further explores the
latter dependence for concrete and useful choices of $\Phi$. Section 4.3
studies the dependence of $\tilde{\Delta}$ on the degree of heterogeneity in
users’ tastes, the outside option utility, and the discount rate. Lastly,
Section 4.4 explores two interesting scenarios: 1) long-run asymmetric
equilibria outperform the symmetric equilibrium, and 2) competition prices are
larger than collusion prices for one side of the market. A supplementary
analysis in Appendix A.1 implies that our numerical results are consistent
with platforms learning tacit collusion and equilibrium strategies.

###  4.1 Dependence of the Collusive Level on the Network Externalities

We applied an additive model to infer the dependence of the collusive level,
$\tilde{\Delta}$, on the externality matrix, $\Phi$. We ran 2,500 simulations
according to the setting described in Section 3.2. For each simulation, we
randomly sampled the elements of the externality matrix $\Phi$ from
independent normalized Gaussians (that is, $\phi_{kl}\sim N(0,1)$ for
$k,l\in\\{b,s\\}$), and recorded the final collusive level, $\tilde{\Delta}$.
In order to infer the dependence of $\tilde{\Delta}$ on $\Phi$, we assume the
following additive model:

$\begin{split}\tilde{\Delta}(\Phi)&=\Delta_{0}+f_{bb}(\phi_{bb})+f_{ss}(\phi_{ss})+f_{bs}(\phi_{bs})+f_{sb}(\phi_{sb})+f_{bb,ss}(\phi_{bb},\phi_{ss})+f_{sb,bs}(\phi_{sb},\phi_{bs})\\\
&+f_{bb,bs}(\phi_{bb},\phi_{bs})+f_{bb,sb}(\phi_{bb},\phi_{sb})+f_{ss,bs}(\phi_{ss},\phi_{bs})+f_{ss,sb}(\phi_{ss},\phi_{sb})+\epsilon,\end{split}$
(19)

where $\Delta_{0}$ is the sample mean of $\tilde{\Delta}$, the next 4
functions ($f_{bb}$, $f_{ss}$, $f_{bs}$, $f_{sb}$) represent the univariate
effects of the elements of $\Phi$ on $\tilde{\Delta}$, the last 6 functions
($f_{bb,ss}$, $f_{sb,bs}$, $f_{bb,bs}$, $f_{bb,sb}$, $f_{ss,bs}$, $f_{ss,sb}$)
represent the bivariate effects of the elements of $\Phi$ on $\tilde{\Delta}$
and $\epsilon$ is an error term, encompassing higher-order multivariate
effects. Since the equilibrium values $\pi^{*}$ and $\pi^{C}$ depend on $\Phi$
nonlinearly (see Section 2.1), the 10 functions, $f_{bb},\cdots,f_{ss,sb}$,
are nonlinear. We thus sequentially fit these functions using XGBoost (Chen
and Guestrin,, 2016), which is a popular non-parametric, nonlinear fitting
method. To reduce the bias of the fitted functions, we alter the order of both
the first four functions and the next six functions, during the sequential
fitting procedure, and average the collusive level over the different orders.
Appendix A.2 contains more details of implementing XGBoost.

We refer to $\Delta_{0}$ as the baseline collusive level, whereas
$\tilde{\Delta}$ is the collusive level. Our simulations show that
$\Delta_{0}$ is approximately $0.3$. Next, we report our estimates for the
univariate and bivariate effects of the elements of $\Phi$ on
$\tilde{\Delta}$.

Figure 2: Demonstration of the dependence of the four fitted univariate
functions on the externalities. Top left: $f_{bb}(\phi_{bb})$; Top right:
$f_{ss}(\phi_{ss})$; Bottom left: $f_{bs}(\phi_{bs})$; Bottom right:
$f_{sb}(\phi_{sb})$.

Figure 2 illustrates the fitted functions $f_{bb}$, $f_{ss}$, $f_{bs}$ and
$f_{sb}$, which capture the univariate effect of each entry in the externality
matrix. The top two subfigures demonstrate the univariate effect of the
within-side externalities, ($\phi_{bb}$ and $\phi_{ss}$). In this case, the
collusive level is close to zero when these externalities are less than 1,
then increases sharply when these externalities increase from $1$ to $2$, and
it is approximately flat when these externalities are above $2$ with a
possible increase of the collusive level when the absolute values of the
negative externalities increase. We remark that we cannot confidently conclude
the latter increase from the current experimental results, but latter
experiments in Section 4.2 support such an increase, especially when
considering lower values of $\phi_{bb}$ and $\phi_{ss}$. The bottom two
subfigures demonstrate the univariate effect of the cross-side externalities
($\phi_{bs}$ and $\phi_{sb}$). In this case, the dependence of the collusive
level on the externalities is depicted by a J-shape function with a minimum
when the externality is around zero. We thus note that in order to minimize
the level of the algorithmic collusion, we would need to bound the values of
the within-side externalities and the absolute values of the cross-side
externalities. In our particular experimental setting, the desired bound is 1.
In general, we expect there can be two different upper bounds for the within-
side and cross-side externalities and they depend on the chosen parameters, in
particular, $\\{\delta,\beta_{k},u_{k}^{(0)}\\}$.

Figure 3: Heatmaps of the fitted functions $f_{bb,ss}(\phi_{bb},\phi_{ss})$
(left) and $f_{sb,bs}(\phi_{bs},\phi_{sb})$ (right), which capture the
bivariate effect between $\phi_{bb}$ and $\phi_{ss}$, and between $\phi_{bs}$
and $\phi_{sb}$, respectively.

Figure 3 demonstrates the fitted functions $f_{bb,ss}$ and $f_{sb,bs}$, which
capture the bivariate effects on collusion of the main diagonal and off-
diagonal elements in the externality matrix. We present the images of these
functions as heatmaps over their planar domains. For example, in the left-hand
subfigure the domain is described by the within-side externality variables
$\phi_{bb}$ and $\phi_{ss}$ and the collusive level is depicted by a heatmap,
changing from purple (highly positive) to orange (highly negative). The left-
hand subfigure implies that when the within-side externalities, $\phi_{bb}$
and $\phi_{ss}$, are both large, they result in the minimal value of the
bivariate effect, which is negative. In this regime, the collusive level,
resulting from both the univariate and bivariate effects, remains positive
(recall that the univariate effect is demonstrated in the top subfigures of
Figure 2). Similarly, when $\phi_{bb}$ and $\phi_{ss}$ are both sufficiently
negative, their bivariate effect reduces the collusive level, albeit by a
small amount. We remark that both the univariate and bivariate contributions
in this case are rather small and it is hard to predict their combined effect
from this experimental result, but another experiment in Section 4.2 indicates
that they cancel each other. The right-hand side subfigure indicates that the
bivariate component of the cross-side externalities reduces the collusive
level when these externalities are large in absolute values and have opposite
signs. On the other hand, it increases the collusive level when the cross-side
externalities have the same sign and have sufficiently large absolute values.
The rate of increase is larger when they are both positive.

Figure 4: Heatmaps of the fitted functions $f_{bb,sb}(\phi_{bb},\phi_{sb})$
(left) and $f_{ss,bs}(\phi_{ss},\phi_{bs})$ (right), which capture the
bivariate effect between $\phi_{bb}$ and $\phi_{sb}$, and between $\phi_{ss}$
and $\phi_{bs}$, respectively.

Figure 4 demonstrates the bivariate effect on collusion when both buyers and
sellers benefit from population joining the market on either side $b$ or $s$
(but not both at the same time). That is, it demonstrates the bivariate effect
for $\phi_{bb}$ and $\phi_{sb}$ when considering side $b$ (left) and the
bivariate effect for $\phi_{ss}$ and $\phi_{bs}$ when considering side $s$
(right). The two subfigures are very similar and we thus only discuss the left
one, with the variables $\phi_{bb}$ and $\phi_{sb}$. The bottom-right corner
of this subfigure implies that if $\phi_{sb}$ sufficiently large and
$\phi_{bb}$ is sufficiently negative, the bivariate effect on the collusive
level is negative. In this regime, the collusive level, resulting from both
the univariate and bivariate effects, remains positive (recall that the
univariate effect is demonstrated in the top-left and bottom-right subfigures
of Figure 2). On the other hand, the top-right corner in Figure 4 shows that
when $\phi_{bb}$ and $\phi_{sb}$ are both large the bivariate effect on the
collusive level is positive.

Figure 5 illustrates the bivariate effect on collusion when either buyers or
sellers (but not both at the same time) benefit from population joining the
market on sides $b$ or $s$. That is, it demonstrates the bivariate effect for
$\phi_{bb}$ and $\phi_{bs}$ when considering only buyers (left subfigure) and
the bivariate effect for $\phi_{ss}$ and $\phi_{sb}$ when considering only
sellers (right subfigure). The two subfigures are very similar and we thus
only discuss the left one, with the variables $\phi_{bb}$ and $\phi_{bs}$. We
notice that when the $\phi_{bb}$ is sufficiently large and $\phi_{bs}$ is
sufficiently negative, the bivariate effect on the collusion is negative. In
this regime, the collusive level, resulting from both the univariate and
bivariate effects, remains positive (the univariate effect is demonstrated in
the left subfigures of Figure 2). We further notice that when $\phi_{bs}$ is
sufficiently large and $\phi_{bb}$ is sufficiently negative the bivariate
effect on the collusion is also negative, but smaller than the latter one.
Similarly, the collusive level, resulting from both the univariate and
bivariate effects, remains positive. On the other hand, when both $\phi_{bb}$
and $\phi_{bs}$ are sufficiently large, the bivariate effect on the collusive
level is positive.

Figure 5: Heatmaps of the fitted functions $f_{bb,bs}(\phi_{bb},\phi_{bs})$
(left) and $f_{ss,sb}(\phi_{ss},\phi_{sb})$ (right), which capture the
bivariate effect between $\phi_{bb}$ and $\phi_{bs}$, and between $\phi_{ss}$
and $\phi_{sb}$, respectively

###  4.2 A Study of the Collusive Level under Special Network Externalities

We assume special parameterizations of the network externality matrices,
$\Phi$, and explore the dependence of $\tilde{\Delta}$ on any such $\Phi$.
This allows us to track more carefully the dependence of $\tilde{\Delta}$ on
$\Phi$ in some special settings. For each specific $\Phi$, we ran 100
simulations. Our figures present the dependence of the overall collusive level
on the elements of $\Phi$, where their main curves represent the average of
the collusive levels from the 100 runs and their shaded areas represent the
uncertainty level, which was computed using bootstrapping with a 99%
confidence interval.

Figure 6 investigates the dependence of the collusive level on the within-side
externalities in two controlled settings. In the first setting (left panel)
$\Phi=[\phi_{bb},0;0,0]$, and in the second one (right panel)
$\Phi=[\phi_{bb},0;0,\phi_{bb}]$. In both cases $\phi_{bb}\in[-6,2]$.

Figure 6: Collusive level with varying $\phi_{bb}$: $\Phi=[\phi_{bb},0;0,0]$
(left) and $\Phi=[\phi_{bb},0;0,\phi_{bb}]$ (right).

In the left panel, the collusive level forms a J-shape, where it decreases on
$[-6,0]$ and increases sharply on $[0.5,2]$. The minimum value of the
collusive level is achieved when $\phi_{bb}\approx 0.5$ and it is slightly
below the baseline collusive level. Note that this subfigure indicates a
similar behavior of the collusive level to its univariate effect shown in the
top left panel of Figure 2. Indeed, in this case, the collusive level depends
on the single variable $\phi_{bb}$, so the other univariate and bivariate
functions are irrelevant. However, the minimal value of the univariate effect
in the top left panel of Figure 2 is around zero, since it is separate from
the baseline collusive level $\Delta_{0}$. By adding $\Delta_{0}$ to this
univariate effect, we obtain a function similar to the collusive level
described in the left panel of Figure 6. We remark that in the experiments of
Section 4.1, our domain was restricted by the underlying Gaussian model and
thus the domain in Figure 2 is narrower than that of Figure 6.

In the right panel, the collusive level sharply increases when $\phi_{bb}$
exceeds $1$. This behavior can be explained using our previous findings.
Indeed, as shown in Figures 2 and 3, when the within-side externalities,
$\phi_{bb}$ and $\phi_{ss}$, are both large, the univariate effect is more
significant than the negative bivariate effect, resulting in a significant
increase. We also notice that the collusive level remains flat and around
$\Delta_{0}$ when $\phi_{bb}=\phi_{ss}$ falls below 1. This observation also
confirms our findings in the previous section. Indeed, Figures 2 and 3
indicate that when $\phi_{bb}$ and $\phi_{ss}$ are both sufficiently negative,
the (positive) univariate and (negative) bivariate effects cancel each other.

Figure 7 investigates the dependence of the collusive level on the cross-side
externalities in two controlled settings. In the first setting (left panel)
$\Phi=[0,\phi_{bs};0,0]$, and in the second one (right panel)
$\Phi=[0,\phi_{bs};\phi_{bs},0]$. In both cases $\phi_{bs}\in[-2.5,3]$.

Figure 7: Collusive level with varying $\phi_{bs}$: $\Phi=[0,\phi_{bs};0,0]$
(left) and $\Phi=[0,\phi_{bs};\phi_{bs},0]$ (right).

In the left panel, the collusive level increases when the cross-side
externality exceeds $2$, and it slightly decreases when the same externality
falls below $-1$. This observation aligns with our findings in the previous
section. Indeed, as shown by the bottom panels in Figure 2, the collusive
level increases as $\phi_{bs}$ increases in absolute value with values above
$1$.

In the right panel, the collusive level increases when the cross-side
externalities exceed $1$. It has a sharper increase than the one in the left
panel. These observations agree with the findings of the previous section.
Indeed, Figures 2 and 3 show that the univariate and bivariate effects of
$\phi_{bs}$ and $\phi_{sb}$ are both positive when $\phi_{bs}=\phi_{sb}>1$.
Furthermore, Figure 3 shows the positive bivariate effect between $\phi_{bs}$
and $\phi_{sb}$, which explains the sharper increase in the right panel. The
dependence of the collusive level in the right panel on smaller values of
$\phi_{bs}$, which are not shown in this figure, is rather unique and thus
deferred to Section 4.4.

Figure 8 investigates the dependence of the collusive level on the bivariate
effects between the within- and cross-side externalities in two controlled
settings. For simplicity, we fix the cross-side externality and vary the
within-side externality. In the first setting (left panel)
$\Phi=[\phi_{bb},3;0,0]$, and in the second one (right panel)
$\Phi=[\phi_{bb},3;0,\phi_{bb}]$. In both cases $\phi_{bb}\in[-6,2]$.

Figure 8: Collusive level with varying $\phi_{bb}$: $\Phi=[\phi_{bb},3;0,0]$
(left) and $\Phi=[\phi_{bb},3;0,\phi_{bb}]$ (right).

In the left panel, the collusive level is flat when the within-side
externality falls below $-1$ and increases sharply as it exceeds $-1$. This
increase can be explained when $\phi_{bb}$ exceeds $0$ by the univariate
effect for $\phi_{bb}$ shown in Figure 2. On the other hand, the increase in
$[-1,0]$ can be explained by the bivariate effect between $\phi_{bb}$ and
$\phi_{bs}$, shown in Figure 5. Indeed this bivariate effect increases with
respect to $\phi_{bb}$ in $[-1,0]$ when $\phi_{bs}=3.0$.

In the right panel, the collusive level follows a similar pattern as in the
left one. This follows from a similar explanation as above, where one should
also note that when $\phi_{bs}=3$, the bivariate effect between $\phi_{ss}$
and $\phi_{bs}$ increases with respect to $\phi_{ss}$ in $[-1,0]$, as shown in
Figure 4. Additionally, when $\phi_{bb}<0$, the collusive level in the right
panel is lower than that in the left panel (it is easiest to see this for
$\phi_{bb}<-1$). We clarify this observation in view of the findings of
Section 4.1 as follows. We note that according to the right panel of Figure 4
, when $\phi_{bs}=3$, the bivariate effect between $\phi_{ss}$ and $\phi_{bs}$
is negative when $\phi_{ss}<0$, therefore the collusive level in the right
panel is expected to be lower than the collusive level in the left panel when
$\phi_{bb}=\phi_{ss}<0$.

We make some additional remarks comparing Figures 6 and 8. The left panel in
Figure 6 shows that the collusive level decreases with respect to $\phi_{bb}$,
when $\phi_{bb}<0$ and $\phi_{bs}=0$. On the other hand, in the left panel of
Figure 8, the collusive level is flat or increases with respect to $\phi_{bb}$
when $\phi_{bb}<0$ and $\phi_{bs}=3$. This behavior can be explained by the
contribution from the bivariate effect between $\phi_{bb}$ and $\phi_{bs}$
when $\phi_{bb}$ is negative. Indeed, this bivariate effect is almost flat
with respect to $\phi_{bb}$ when $\phi_{bs}=0$, but is increasing with respect
to $\phi_{bb}$ when $\phi_{bs}=3.0$ (see left panel of Figure 5). A similar
comparison can be made for the right panels in Figures 6 and 8, and the
explanation similarly follows from the right panel of Figure 4 and the left
panel of Figure 5.

###  4.3 A Study of the Collusive Level under Special Market Parameters

We explore the dependence of the collusive level on the market parameters
$\beta_{k}$, $u^{(0)}_{k}$ and $\delta$. In each experiment, we fix two of the
latter parameters, using the setup described in Section 3.2, and the matrix
$\Phi$, where its choices change with the experiments, and vary the remaining
parameter. For each experiment, we ran 100 simulations, averaged the collusive
levels among the 100 runs and computed the uncertainty levels using
bootstrapping with a 99% confidence interval. Our figures present the averaged
collusive level as a function of one parameter, where the shaded areas
represent the uncertainty level.

Figure 9 investigates the dependence of the collusive level on the
idiosyncratic preference parameter $\beta_{k}$, while considering two
different choices of the externality matrix $\Phi$: A symmetric one, where
$\Phi=[1,0;0,1]$ (left panel) and an asymmetric one, where $\Phi=[0,1;-1,0]$
(right panel). In both cases, we vary the idiosyncratic preference parameters
and let $\beta_{k}\in[0.2,6]$. In both panels, the collusive level sharply
decreases when $\beta_{k}$ is sufficiently small. In particular, the collusive
level is high only when the degree of heterogeneity in users’ tastes is
sufficiently small. Section 5.1 interprets this behavior. We note that in the
left panel the sharp decrease stops when $\beta_{k}$ exceeds $0.9$, compared
to the right panel where this happens ses once $\beta_{k}$ exceeds $0.5$.
Furthermore, we note that in both subfigures, the collusive levels remains
almost flat, at a value slightly below the collusive level $\Delta_{0}=0.3$,
as the degree of heterogeneity in users’ tastes exceeds $1$. More experiments
varying $\beta_{k}$ with different choices of $\Phi$ are presented in Appendix
A.3.

Figure 9: Collusive level with varying $\beta_{b}=\beta_{s}\in[0.2,6]$ and
different matrices $\Phi$, where $\Phi=[1,0;0,1]$ (left) and $\Phi=[0,1;-1,0]$
(right).

Figure 10 demonstrates the dependence of the collusive level on the outside
option utility $u_{k}^{(0)}$ with the following choices for the externality
matrix: $\Phi=[0,0;0,0]$ (top left panel), $\Phi=[1,0;0,1]$ (top right panel),
$\Phi=[0,1;1,0]$ (bottom left panel), and $\Phi=[1,1;1,1]$ (bottom right
panel). In all four panels, we observe a main trend of decrease of the
collusive level as a function of the outside option utility over a
sufficiently large domain. In all of these examples when $u_{k}^{(0)}$ is
sufficiently small, the collusive level is at least $0.55$, and when
$u_{k}^{(0)}$ is sufficiently large, the collusive level is around $0.3$,
which is near the baseline level. The most significant reduction of the
collusive level happens in a narrow range and the location of this significant
decrease appears to be determined by the externalities as follows. It tends to
move to the left when the network externalities are small and to the right
when they are large. Additional examples in Figure A4 support this conclusion.
In addition, we observe that both left subfigures exhibit another small region
of increase to the baseline level after the region of sharp decrease. This is
not the case for the right subfigures. We also note a similar phenomenon in
Figure A4. It seems that an increase to the baseline level after a sharp
decrease occurs in cases of sufficiently small externalities, where the
threshold on externalities required to guarantee such a short increase is
smaller for within-side externalities than for cross-side externalities. For
example, considering cases of both Figures 10 and A4, this short increase is
observed at $\Phi=[0,1;1,0]$ but not at $\Phi=[0,2;2,0]$, and is observed at
$\Phi=[1,0;0,0]$ but not at $\Phi=[1,0;0,1]$. Lastly, it follows from the
bottom right subfigure of Figure 10 and cases of Figure A4 that for
sufficiently large externalities, the collusive level may slightly increase
before the sharp decrease.

Figure 10: Collusive level with varying $u^{(0)}_{k}$, $k\in\\{b,s\\}$, with
$\Phi=[0,0;0,0]$ (top left), $\Phi=[1,0;0,1]$ (top right), $\Phi=[0,1;1,0]$
(bottom left) and $\Phi=[1,1;1,1]$ (bottom right).

Figure 11: Collusive level with varying $\delta$, where $\Phi=[0,0;0,0]$
(left) and $\Phi=[1,0;0,1]$ (right); in both cases $\delta\in[0.01,0.99]$.

Figure 11 investigates the dependence of the collusive level on the discount
rate $\delta$, where $\delta\in[0.01,0.99]$, with $\Phi=[0,0;0,0]$ (left) and
$\Phi=[1,0;0,1]$ (right). In both panels, the collusive level increases with
the discount rate. Thus, the more patient platforms are about the future, the
higher the collusive level becomes. We note that the collusive level in the
right panel is approximately a horizontal shift of that in the left panel.
Thus, with a non-zero externality matrix the collusive level increases earlier
than with the zero externality matrix. The positive relationship between the
collusive level and the discount rate was first observed by Calvano et al.,
2020b  for Bertrand model and our experiments verify the same relationship for
a multi-sided market.

###  4.4 Discussion of Exceptional Cases

We numerically demonstrate two exceptional and uncommon scenarios: asymmetric
optimal prices and competition prices larger than collusion prices.

Asymmetric collusion. Our metric for the collusive level compares the
platform’s rewards at time $t$ with the symmetric equilibrium quantities
$\pi^{*}$ and $\pi^{C}$, following previous simulations of collusion (see,
e.g., Calvano et al., 2020b  and Klein, (2021)). However, our specific model
may give rise to asymmetric equilibria and we thus study their possibility
more carefully.

To allow asymmetric equilibria, we modify the definitions of the maximum
values of the total profits and the collusive level as follows. The total
profit, $\Pi_{a}$, is

$\Pi_{a}(p_{b}^{(1)},p_{s}^{(1)},p_{b}^{(2)},p_{s}^{(2)}):=\sum_{i=1}^{2}(x^{(i)}_{b}p_{b}^{(i)}+x^{(i)}_{s}p_{s}^{(i)})\equiv\sum_{i=1}^{2}\pi_{t}^{(i)},$
(20)

where unlike (4) it does not assume symmetric prices (that is, it does not
assume that $\pi_{t}^{(1)}=\pi_{t}^{(2)}$) and its subscript $a$ indicates
asymmetry. Similarly, the collusive level $\Delta_{t}$ is averaged among the
two firms as follows

$\Delta_{t}:=(\Delta_{t}^{(1)}+\Delta_{t}^{(2)})/2\equiv\frac{\Pi_{a}-2\pi^{*}}{2(\pi^{\text{C}}-\pi^{*})},$

where $\Delta_{t}^{(i)}$, $i\in\\{1,2\\}$, was defined in (17).

Figure 12 demonstrates the maximum value that $\Delta_{t}$ can achieve in two
controlled settings. In the first setting (left panel)
$\Phi=[\phi_{bb},\phi_{bs};\phi_{bs},\phi_{bb}]$ with $\phi_{bb}\in[-1,1]$ and
$\phi_{bs}\in[-4,4]$, and in the second one (right panel)
$\Phi=[\phi_{bb},\phi_{bs};-\phi_{bs},\phi_{bb}]$ with $\phi_{bb}\in[-2,2]$
and $\phi_{bs}\in[-8,8]$. More precisely, we maximize $\Delta_{t}$ over
${\bm{p}}^{(1)}_{t}\in{\mathcal{A}}$ and ${\bm{p}}^{(2)}_{t}\in{\mathcal{A}}$,
where ${\mathcal{A}}$ was defined in (13), and present the maximal values
using a heatmap, whose values vary from purple ($\Delta_{t}$ greater than $1$)
to orange ($\Delta_{t}$ less than $1$). In both panels, $\beta_{k}=1.0$ and
$u_{k}^{(0)}=-2.0$, $k\in\\{b,s\\}$.

In the left panel, the maximal $\Delta_{t}$ exceeds $1$ when the within-side
externality $\phi_{bb}$ is positive and close to $1$ and the cross-side
externality $\phi_{bs}$ is sufficiently negative. We note that it is much
larger than 1. In the right panel, the maximal $\Delta_{t}$ is larger than 1
when $\phi_{bb}$ is sufficiently large and $\phi_{bs}$ is close to zero. We
note though that it never exceeds the value of $1.0175$. In both panels, for
all other corresponding values of $\phi_{bb}$ and $\phi_{bs}$, the maximum
value of $\Delta_{t}$ is either achieved at a symmetric vector price or at an
asymmetric vector price with corresponding maximum value close to $1$. In the
latter case, of the corresponding maximum value close to $1$, our measure of
collusive level $\tilde{\Delta}$ using the symmetric assumption can still be
used to quantify the level of collusion. That is, one may often follow up our
analysis with the symmetric quantities, except for special cases where the
estimated collusive level exceeds one, where one needs to use the asymmetric
quantities.

Figure 12: Maximal average collusive level with varying $\phi_{bb}$ and
$\phi_{bs}$, where $\Phi=[\phi_{bb},\phi_{bs};\phi_{bs},\phi_{bb}]$ (left) and
$\Phi=[\phi_{bb},\phi_{bs};-\phi_{bs},\phi_{bb}]$ (right).

We remark that for almost all choices of $\beta_{k}$, $u_{k}^{(0)}$, $\delta$
and $\Phi$ in this paper, the asymmetric maximum value of $\Delta_{t}$ does
not exceed the symmetric maximum value. Indeed, our simulations from Sections
4.1 through 4.3 show that $\tilde{\Delta}$ is smaller than or equal to $1$.
Nevertheless, when extending the right panel of Figure 7 to more negative
values a collusive level higher than 1 is noticed, which we depict in Figure
13. In this figure, $\Phi=[0,\phi_{sb};\phi_{sb},0]$ with $\phi_{sb}\in[-4,2]$
and the asymmetric maximum value exceeds the symmetric maximum value when
$\phi_{sb}<-2.5$. We notice that the collusive level reaches values close to
$5$ when the within-side externalities are zero and the cross-side
externalities are equal in magnitude and sufficiently negative. Note that this
scenario agrees with the one depicted in the left panel of Figure 12, where
one can notice a similar value of 5 when $\phi_{bb}=\phi_{ss}=0$ and
$\phi_{bs}=\phi_{sb}=-4$.

Figure 13: Collusive level with varying $\phi_{sb},$ where
$\Phi=[0,\phi_{sb};\phi_{sb},0]$.

Competition prices larger than collusion prices. We recall that both the
simulated action space presented in (12) and the formulated baseline model in
Section 2.1 allow for the competition prices ($p_{k}^{*}$) to be either
smaller or larger than the collusion prices ($p_{k}^{\text{C}}$). We
demonstrate here an uncommon situation where the collusion price can be
smaller than the competition price for one side of the market. In this example
$\Phi=[1,-\phi_{sb};\phi_{sb},-2]$, $\beta_{k}=0.5$ and $u^{(0)}_{k}=-1.0$,
$k\in\\{b,s\\}$. If $\phi_{sb}\in[-5,0.5)$, then $p_{s}^{*}<p_{s}^{\text{C}}$.
If $\phi_{sb}\in(0.5,5]$, then $p_{s}^{*}>p_{s}^{\text{C}}$. The left panel of
Figure 14 demonstrates the competition and collusion prices of both sides of
the market. For side $b$, the collusion price is always higher than the
competition price. However, for side $s$, there are two different regimes
separated by $\tilde{\phi}_{sb}\approx 0.5$. When
$\phi_{sb}<\tilde{\phi}_{sb}$, the collusion price is higher than the
competition price on the seller side, and when $\phi_{sb}>\tilde{\phi}_{sb}$
it is lower.

Figure 14: Demonstration of an uncommon scenario where the collusion prices
can be lower than competition prices. Here,
$\Phi=[1,-\phi_{sb};\phi_{sb},-2]$, where $\phi_{sb}\in[-5,5]$. The left panel
demonstrates collusion and competition prices on both sides of the market. The
right panel demonstrates the collusive level. The black vertical dotted lines
separate the two regimes: $p^{*}_{s}<p_{s}^{\text{C}}$ on the left side of the
black vertical dotted line and $p^{*}_{s}>p_{s}^{\text{C}}$ on the right side
of the black vertical dotted line.

The right panel of Figure 14 shows the collusive level for this example. The
black dotted line is drawn at $\tilde{\phi}_{sb}$ to separate the two regimes.
In the left regime, where $p_{s}^{*}<p_{s}^{\text{C}}$, the collusive level
decreases on $[-5,-3]$ and increases on $[-3,0]$. This behavior is somewhat
similar to the one described in the left panel of Figure 7. During the
transition from $p_{s}^{*}<p_{s}^{\text{C}}$ to $p_{s}^{*}>p_{s}^{\text{C}}$,
that is, in a small interval around $\tilde{\phi}_{sb}$, the collusive level
decreases. Next, the collusive level increases on $[1.5,3]$ and decreases on
$[3,5]$, which is the opposite behavior (in terms of decreasing and
increasing) to the one demonstrated in the other regime.

##  5 Economic and Policy Discussion

We examine the economic implications of the numerical results presented in
Section 4. Specifically, we analyze the impact of network externalities on the
collusive level and discuss the ramifications of these findings on real-life
markets. We then review the Preventing Algorithmic Collusion Act of 2024,
introduced by U.S. Senator Amy Klobuchar, and suggest an additional policy
recommendation to ensure the safer use of $Q$-learning.

###  5.1 Economic Discussion

When $\Phi=0$, our baseline platform competition game reduces to Bertrand
competition games on each side of the market. For $\delta=0.05$, $\beta_{k}=1$
and $u_{k}^{(0)}=-2$, $k\in\\{b,s\\}$, our simulations show that $\Delta_{0}$
in (19) is approximately $0.3$. This means that when $\Phi=0$, the profit gain
relative to competition profits is about 30%. It is interesting to note that
Calvano et al., 2020b  reported a different gain of approximately $20\%$ for
the same value of $\delta$. To understand the difference, we note that in the
latter paper, firms serve only one market and Bertrand competition is the
baseline game. For that reason, the action space used in their simulations is
smaller than the action space used in ours. The larger action space allows the
platforms to algorithmically communicate more information, increasing the
chance of achieving a higher collusive level.

Next, we discuss the impact of $\Phi\neq 0$ on the collusive level compared to
Bertrand competition, where $\Phi=0$. When $\Phi$ has only one non-zero entry,
the dependence of the collusive level on each possible entry is depicted in
Figures 2 and the left panel of 6. We note that if the nonzero externality is
either positive or sufficiently negative, then the collusion is higher in
platform competition than in traditional Bertrand competition. Furthermore,
when $\Phi$ is a diagonal matrix, the dependence of the collusive level on
these entries is depicted in Figures 2 and the left panel of 3. We note that
when these entries are both positive, collusion is higher in platform
competition than in single-sided Bertrand competition. These findings suggest
that in markets such as online (or cloud) gaming, such as in Xbox, mobile and
computer games, where positive within-side externalities are significant,
algorithmic pricing will increase collusion levels above those observed in
baseline Bertrand competition. Finally, when $\Phi$ has only off diagonal non-
zero entries, the dependence on the collusive level on these entries is
depicted in Figures 2 and the right panel of 3. We note that when these
entries are both positive, collusion sharply increases above the baseline
Bertrand level. These findings suggest that in markets such as video streaming
(e.g., Netflix, Hulu, and Amazon) and social media markets (e.g., Instagram
and TikTok), where positive cross-side externalities are significant, high
levels of collusion can be expected if platforms use algorithmic pricing.

Traditionally, platforms use positive network externalities to enhance demand
and profits by subsidizing one side of the market in order to attract
population on the other sides (see, e.g., Armstrong and Wright, (2007), Tan
and Zhou, (2021), and Chica et al., (2021)). Our findings suggest that
algorithmic driven platforms may also learn to use positive network
externalities to significantly increase the profit.

Next we discuss the other scenario, where network externalities result in
relatively small levels of collusion. First, we note that the right panels of
both Figures 6 and 7 indicate examples where either the within-side
externalities (Figure 6) or the cross-side externalities (Figure 7) are both
negative with the same magnitude and in these cases the collusive level
remains flat at the baseline competition level $\Delta_{0}$. On the other
hand, Figure 8 shows a case where the cross-side externality ($\phi_{bs}$) is
large and positive, and the within-side externality is sufficiently small and
negative. In this scenario, the collusive level remains flat at a value
slightly above $\Delta_{0}$. The latter example is relevant to ride-sharing
markets, where drivers compete with each other for riders, while riders
benefit from faster pickup times. In this case, when using algorithmic
pricing, our experiments indicate that collusive levels are close to the
baseline level $\Delta_{0}$.

Our findings reveal some interesting patterns in the dependence of the
collusive level on three different market parameters: the degree of
heterogeneity in users’ tastes $\beta_{k}$, the constant term of the outside
option utility $u^{(0)}_{k}$, and the discount rate $\delta$. As shown by
Figure 9, the collusive level sharply decreases as the degree of heterogeneity
in users’ tastes increases from $0.2$ to $1$. Afterwards, it remains flat
around values lower than the baseline collusive level $\Delta_{0}$. These
findings can also be observed in Appendix A.3 for multiple choices of the
externality matrix $\Phi$. We are not aware of any previous result like this.
A different result states that higher degree of heterogeneity in users’ tastes
leads to inelastic demand and higher price, which in turn leads to higher
individual profits (see, e.g., Perloff and Salop, (1985) and Anderson and De
Palma, (1992)). However, the collusion level generally does not correlate with
individual profit values.

Next, Figure 10 indicates a main trend of decrease of the collusive level as a
function of the outside option utility. This behavior coincides with the
observation of Chica et al., (2023) (see Proposition 4.6) that as the value of
the outside option increases, market power held by the platforms decreases. We
further noticed some very local trends, but they depend on specific choices of
the externalities. This behavior has not been observed before and needs to be
further explored. The main trend of the dependence of the collusive level on
$u_{k}^{(0)}$ has relevant market implications. For instance, we note that for
ride-sharing platforms using AI pricing, our analysis shows that an emphasis
on increasing the outside option utility would decrease price and collusion
levels. Such outside option utility can be increased by enhancing the public
transportation system, and increasing mobility options such as e-bikes.
Instead of investing further in such options and, in particular, in the safety
of public transportation, the city of Minneapolis chose to decrease prices by
passing an ordinance888The ordinance can be found at
https://lims.minneapolismn.gov/Download/FileV2/32072/Transportation-Ride-
Share-Worker-Protection-Ordinance.pdf that would force the two major ride-
sharing platforms in the city, Uber and Lyft, to pay drivers the city’s
minimum hour wage. As a result, Uber and Lyft announced plans to leave the
market. Our analysis indicates that there are other strategies to mitigate the
problem.

Finally, as shown in Figure 11, the collusive level increases as the discount
rate increases. This result coincides with earlier results by Calvano et al.,
2020b . However, the rate at which the collusive level increases w.r.t.
$\delta$ seems to be larger for the case of platform competition compared to
Bertrand competition. In fact, most of our experiments in section 4 use a
value of $\delta=0.05$, which would be considered a low discount rate. Our
sensitivity analysis in Appendix A indicates that this is likely tacit
collusion and not just high prices. As mentioned earlier, for a more specific
case, we believe that the main reason for the higher rate of increase of the
collusive level in the platform competition model is due to a larger action
space.

###  5.2 Policy Discussion

U.S. senator Amy Klobuchar introduced the S.3686 - Preventing Algorithmic
Collusion Act of 2024 in January 2024,999The act can be consulted at
https://www.congress.gov/bill/118th-congress/senate-bill/3686/text, whose
abstract is as follows:

“A bill to prevent anticompetitive conduct through the use of pricing
algorithms by prohibiting the use of pricing algorithms that can facilitate
collusion through the use of nonpublic competitor data, creating an antitrust
law enforcement audit tool, increasing transparency, and enforcing violations
through the Sherman Act and Federal Trade Commission Act, and for other
purposes.”

This bill reflexes the increased concern by congress members and governmental
institutions for the use of algorithmic price collusion. If such legislation
succeeds, it will constitute a major advancement for consumer safety against
the potential threats of AI. However, the above act is rather limited. Indeed,
it only targets AI algorithms trained with nonpublic competitor data and
characterizes them as unlawful:

“ SEC. 4. PREVENTING COLLUSIVE ACTIVITY IN PRICING ALGORITHMS. (a) In
General.—It shall be unlawful for a person to use or distribute any pricing
algorithm that uses, incorporates, or was trained with nonpublic competitor
data. ”

Potentially collusive AI algorithms, such as the ones presented in this work
using $Q$-learning, are left out of the scope of this act. We showed that
these algorithms can learn to sustain high levels of collusion using only
publicly available data, even in cases where agents have limited memory
capacity. While this general observation has been established in previous
works (Waltman and Kaymak, (2008), Calvano et al., 2020b , Klein, (2021) and
Assad et al., (2024)), this work shows that in the presence of positive
network externalities and two-sided markets, algorithmic driven platforms
achieve collusive levels higher than those shown in previous works.

The question of why these algorithms can achieve high levels of collusion and
whether there exists simple conditions to avoid it remains open. One way of
decreasing collusion levels is obtained by increasing the value of the outside
option utility, which has to be done in different ways for different markets
(see e.g., the discussion in Section 5.1 for the ride-sharing platforms).
Nevertheless, we next suggest a preliminary policy recommendation that can
help avoid the risk of collusion by algorithmic driven AI agents in multiple
markets. It is based on penalizing the $Q$-learning rewards.

Policy Recommendation (Q-learning with penalty term). We describe a very basic
approach for reducing the collusive level by a potential intervention method.
It can only be effective if it is enforced by regulators. The method
introduces a penalty coefficient $\rho\geq 0$ that regularizes the
$Q$-learning update formula as follows:

$\displaystyle Q^{(i)}_{t+1}({\bm{s}}_{t}^{(i)},{\bm{a}}_{t}^{(i)})$
$\displaystyle:=(1-\alpha)Q^{(i)}_{t}({\bm{s}}_{t}^{(i)},{\bm{a}}_{t}^{(i)})$
$\displaystyle+\alpha\left({\pi}_{t}^{(i)}+\delta\max_{{\bm{a}}}Q^{(i)}_{t}({\bm{s}}_{t}^{(i)},{\bm{a}})-\rho\left((p^{(i)}_{b}-\bar{p}_{b})_{+}+(p_{s}^{(i)}-\bar{p}_{s})_{+}\right)\right),$

where $(p_{k}-\bar{p}_{k})_{+}=\max\\{p_{k}-\bar{p}_{k},0\\}$ and
$\bar{p}_{k}=\frac{1}{N}\sum_{i=1}^{N}p^{(i)}_{k}$, for $k\in\\{b,s\\}$.
Notice that the penalty term,
$\rho\left((p^{(i)}_{b}-\bar{p}_{b})_{+}+(p_{s}^{(i)}-\bar{p}_{s})_{+}\right)$,
only becomes active, if either of the current prices is larger than the
average price charged by all the platforms in the market at time $t$.

One should fix $\rho$ to ensure a tolerable collusive level. For example, the
strongest requirement of having no collusion fixes $\rho$ such that
$\tilde{\Delta}_{T}=0$. If on the other hand, one accepts the collusion level
with no externalities, i.e., when $\Phi=0$, then one may fix $\rho$ such that
$\tilde{\Delta}_{T}=\Delta_{0}$. We note that any such chosen value of $\rho$
is a function of the parameters of the model
$\\{\beta_{k},\delta,\Phi,u_{k}^{(0)}\\}$. Indeed, our findings suggest that
any policy recommendation aimed to reduce the risk of algorithmic price
collusion needs to be dependent on market parameters.

Figure 15 investigates the dependence of the collusive level on the penalty
coefficient $\rho$ for a setting with $\Phi=[2,0;0,2]$ (left panel) and
$\Phi=[0,2;2,0]$ (right panel). Without penalty, these two cases have shown
significantly high collusive levels (above 0.7) as shown in the right panel of
Figure 6 and the right panel of Figure 7. In both cases, the collusive level
reduces sharply as $\rho$ increases and it reaches $\Delta_{0}$ when $\rho$ is
approximately $0.2$ (left panel) and $0.3$ (right panel). For completeness, we
also show extreme cases where the collusive level can be negative, which are
different from our above recommendations for choosing $\rho$.

Figure 15: Collusive level with varying $\rho\in[0,2]$: $\Phi=[2,0;0,2]$
(left) and $\Phi=[0,2;2,0]$ (right).

##  6 Concluding Remarks

We provided a framework for exploring algorithmic price collusion in a model
of platform competition and numerically studied how the collusive level
depends on the externality matrix.

For the case of zero network externalities, the profit gain relative to the
competition profits is about 30%, higher than the value reported by Calvano et
al., 2020b  for the same value of $\delta$. We attribute this difference to
the larger action space, which allows the platforms to algorithmically
communicate more information, increasing the chances of achieving a high
collusive level.

In common economic scenarios with positive network externalities, particularly
when the positive elements are either single entries, diagonal elements, or
off-diagonal elements of $\Phi$, the collusive level in platform competition
is significantly higher than in traditional Bertrand competition. This
suggests that in markets such as online gaming, video streaming, and social
media, AI-driven platforms may exploit positive network externalities to
significantly increase profits through algorithmic pricing, indicating
potential for high levels of collusion.

Our findings also reveal patterns in how the collusive level depends on user
heterogeneity, discount rates, and outside option utility. Specifically,
greater heterogeneity in user tastes leads to lower collusive levels.
Additionally, collusion increases with higher discount rates, consistent with
findings by Calvano et al., 2020b  for Bertrand competition, though the rate
of increase is higher in platform competition. We provide evidence in the
appendix that this collusion is tacit. Furthermore, higher outside option
utility generally decreases collusion levels within a sufficiently large
domain. This suggests that market regulators can reduce collusion by enhancing
the value of outside options, such as improving public transportation in ride-
sharing markets.

We also proposed a version of $Q$-learning with a penalty term to reduce the
risk of algorithmic price collusion. It can be used for policy recommendations
by considering market parameters.

There are several open future research directions. First, while the platform
competition model in Chica et al., (2023) allows for an unlimited number of
platforms, we only conducted numerical experiments with two platforms due to
the exponential growth of the state space and computational resource
requirements with more platforms. Future research could explore alternative
methods to estimate the $Q$-function to mitigate high computational demands.
Second, replicating our experiments using a model that includes nonlinear
network externality effects and allows for multi-homing, where users join
multiple platforms, would be valuable.

## References

  * Anderson and De Palma, (1992) Anderson, S. P. and De Palma, A. (1992).  The logit as a model of product differentiation.  Oxford Economic Papers, 44(1):51–67.
  * Armstrong, (2006) Armstrong, M. (2006).  Competition in two-sided markets.  The RAND journal of economics, 37(3):668–691.
  * Armstrong and Wright, (2007) Armstrong, M. and Wright, J. (2007).  Two-sided markets, competitive bottlenecks and exclusive contracts.  Economic Theory, 32:353–380.
  * Arslantas et al., (2024) Arslantas, Y., Yuceel, E., and Sayin, M. O. (2024).  Strategizing against q-learners: A control-theoretical approach.  arXiv preprint arXiv:2403.08906.
  * Assad et al., (2024) Assad, S., Clark, R., Ershov, D., and Xu, L. (2024).  Algorithmic pricing and competition: Empirical evidence from the german retail gasoline market.  Journal of Political Economy, 132(3):000–000.
  * Barlo et al., (2009) Barlo, M., Carmona, G., and Sabourian, H. (2009).  Repeated games with one-memory.  Journal of Economic Theory, 144(1):312–336.
  * Barlo et al., (2016) Barlo, M., Carmona, G., and Sabourian, H. (2016).  Bounded memory folk theorem.  Journal of economic theory, 163:728–774.
  * Bertomeu et al., (2021) Bertomeu, J., Evans III, J. H., Feng, M., and Tseng, A. (2021).  Tacit collusion and voluntary disclosure: Theory and evidence from the us automotive industry.  Management science, 67(3):1851–1875.
  * Brero et al., (2022) Brero, G., Mibuari, E., Lepore, N., and Parkes, D. C. (2022).  Learning to mitigate ai collusion on economic platforms.  Advances in Neural Information Processing Systems, 35:37892–37904.
  * Brown and MacKay, (2023) Brown, Z. Y. and MacKay, A. (2023).  Competition in pricing algorithms.  American Economic Journal: Microeconomics, 15(2):109–156.
  * (11) Calvano, E., Calzolari, G., Denicolò, V., Harrington Jr, J. E., and Pastorello, S. (2020a).  Protecting consumers from collusive prices due to ai.  Science, 370(6520):1040–1042.
  * (12) Calvano, E., Calzolari, G., Denicolo, V., and Pastorello, S. (2020b).  Artificial intelligence, algorithmic pricing, and collusion.  American Economic Review, 110(10):3267–97.
  * Chen and Guestrin, (2016) Chen, T. and Guestrin, C. (2016).  Xgboost: A scalable tree boosting system.  In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pages 785–794.
  * Chica et al., (2021) Chica, C., Chuk, K., and Tamayo, J. A. (2021).  Exclusive Dealing and Entry by Competing Two-Sided Platforms.  Harvard Business School.
  * Chica et al., (2023) Chica, C., Guo, Y., and Lerman, G. (2023).  Competition and collusion in two-sided markets with an outside option.  Available at SSRN 4641282.
  * Chica et al., (2024) Chica, C., Guo, Y., and Lerman, G. (2024).  Stochastic games, reinforcement learning and collusion with bounded memory.  In preparation.
  * Du and Tanriverdi, (2023) Du, K. and Tanriverdi, H. (2023).  Does it enable collusion or competition: Examining the effects of it on service pricing in multimarket multihospital systems.  MIS Quarterly, 47(4).
  * Johnson et al., (2023) Johnson, J. P., Rhodes, A., and Wildenbeest, M. (2023).  Platform design when sellers use pricing algorithms.  Econometrica, 91(5):1841–1879.
  * Klein, (2021) Klein, T. (2021).  Autonomous algorithmic collusion: Q-learning under sequential pricing.  The RAND Journal of Economics, 52(3):538–558.
  * Maskin and Tirole, (1988) Maskin, E. and Tirole, J. (1988).  A theory of dynamic oligopoly, ii: Price competition, kinked demand curves, and edgeworth cycles.  Econometrica: Journal of the Econometric Society, pages 571–599.
  * Obara and Zincenko, (2017) Obara, I. and Zincenko, F. (2017).  Collusion and heterogeneity of firms.  The RAND Journal of Economics, 48(1):230–249.
  * OECD, (2017) OECD (2017).  Algorithms and collusion: Competition policy in the digital age.
  * Perloff and Salop, (1985) Perloff, J. M. and Salop, S. C. (1985).  Equilibrium with product differentiation.  The Review of Economic Studies, 52(1):107–120.
  * Ruhmer, (2010) Ruhmer, I. (2010).  Platform collusion in two-sided markets.
  * Tan and Zhou, (2021) Tan, G. and Zhou, J. (2021).  The effects of competition and entry in multi-sided markets.  The Review of Economic Studies, 88(2):1002–1030.
  * Tirole, (1988) Tirole, J. (1988).  The theory of industrial organization.  MIT press.
  * Waltman and Kaymak, (2008) Waltman, L. and Kaymak, U. (2008).  Q-learning agents in a cournot oligopoly model.  Journal of Economic Dynamics and Control, 32(10):3275–3293.
  * Watkins and Dayan, (1992) Watkins, C. J. and Dayan, P. (1992).  Q-learning.  Machine learning, 8:279–292.
  * White and Weyl, (2016) White, A. and Weyl, E. G. (2016).  Insulated platform competition.  Available at SSRN 1694317.

##  Appendix A Appendix

Section A.1 presents sensitivity analysis and exemplifies convergence paths.
Section A.2 provides extra details on how to fit the univariate and bivariate
functions of equation (19). Section A.3 provides additional numerical
experiments on the dependence of the collusive level on $\beta_{k}$ and
$u_{k}^{(0)}$.

###  A.1 Sensitivity Analysis and Examples of Convergence Paths

We assume special parameterizations of the network externality matrix, $\Phi$,
and perform a sensitivity analysis for our simulation framework. For each
specific $\Phi$, we ran 500 simulations and classify the behavior in the last
5000 time steps of each run into one of five categories:

  1. 1.

Symmetric 1-cycle (or 1-Sym), where platforms $1$ and $2$ repeatedly choose
actions $p_{1}$ and $p_{2}$, respectively, and $p_{1}=p_{2}$.

  2. 2.

Asymmetric 1-cycle (or 1-Asym), where platforms $1$ and $2$ repeatedly choose
actions $p_{1}$ and $p_{2}$, respectively, and $p_{1}\neq p_{2}$.

  3. 3.

Cycle of length 2-4 (or C2-4), where platforms $1$ and $2$ repeatedly choose
actions that follow a pattern of length two to four. For example, the actions
in a cycle of length 3 are $\\{(p_{1},p_{2}),(q_{1},q_{2}),(r_{1},r_{2})\\}$.

  4. 4.

Cycle of length 5-8 (or C5-8), where the two platforms repeatedly choose
actions that follow a pattern of length five to eight.

  5. 5.

Cycle of length at least 9 (or C9), where the two platforms repeatedly choose
actions that follow a pattern of length at least nine.

Let $k\in\\{1,\dots,5\\}$ index the five categories described above, and
$C_{k}$ denote the set of simulations within category $k$. For each
$k\in\\{1,\dots,5\\}$ and a chosen $\Phi$, we report the following measures:

  * •

Frequency (Freq.) of the category among the 500 simulations.

  * •

Average (Avg.) and standard deviation (S.D.) of the collusive level
$\tilde{\Delta}$ within the category.

  * •

Frequency of equilibrium, which quantifies the proportion of simulations in
which the AI driven platforms learn the action that achieves maximal total
future reward given that the opponent plays with the final $Q$-function. We
compute this frequency, denoted by $Y_{k}\in[0,1]$ for $k\in\\{1,\ldots,5\\}$,
according to the following three steps:

    1. 1.

Let $Q^{*}_{i}$ be the final $Q$-function of platform $i\in\\{1,2\\}$ in a
given simulation $s\in C_{k}$. It is the numerical approximation to
$Q^{*}_{i}$ defined in (10), as explained in Section 3.2. We use this notation
in defining other frequencies below. We estimate, without loss of generality,
platform’s $1$ best response to $Q^{*}_{2}$ using backward induction. For
$\hat{T}=10$, initiate

$Q^{(\hat{T})}_{1}(p,x)=\pi^{(1)}(p,\operatorname*{arg\,max}_{p^{\prime}}Q^{*}_{2}(p^{\prime},x)),$

where $p$ is a vector of prices and $x$ is the state variable.

For $t\in\\{1,\dots,\hat{T}\\}$, compute via backward induction

$Q^{(t-1)}_{1}(p,x)=\pi^{(1)}(p,\operatorname*{arg\,max}_{p^{\prime}}Q^{*}_{2}(p^{\prime},x))+\delta\max_{p^{\prime\prime}}Q^{(t)}_{1}(p^{\prime\prime},x^{\prime}),$

where
$x^{\prime}=(p,\operatorname*{arg\,max}_{p^{\prime}}Q^{\ast}_{2}(p^{\prime},x))$.
The matrix $Q^{(0)}_{1}$ is used as an approximation to platform’s $1$ best
response to $Q^{*}_{2}$ in the given simulation $s\in C_{k}$.

    2. 2.

Let $P_{s}$ denote the set of states used in the convergence path of
$Q_{i}^{*}$ in the given simulation $s\in C_{k}$. We use this definition in
defining the other frequencies below. For each state $(p_{1},p_{2})\in P_{s}$,
we form an indicator variable $X_{s}(p_{1},p_{2})$ that checks if the action
that platform $1$ takes using $Q^{*}_{1}$ approximates the action that it
would take using $Q^{(0)}_{1}$. Thus, for each $(p_{1},p_{2})\in P_{s}$, let
$p^{0}:=\operatorname*{arg\,max}_{p^{\prime}}Q^{(0)}_{1}(p^{\prime},(p_{1},p_{2}))$
and
$p^{*}:=\operatorname*{arg\,max}_{p^{\prime}}Q^{*}_{1}(p^{\prime},(p_{1},p_{2}))$,
then $X_{s}(p_{1},p_{2})=1$ if and only if either $p^{0}=p^{*}$ or $p^{0}$ and
$p^{*}$ are neighbors with distance at most $1$ in the set of indices of the
$Q$-matrix function.

    3. 3.

Let $Y_{k}=\frac{1}{|C_{k}|}\sum_{s\in
C_{k}}\frac{1}{|P_{s}|}\sum_{(p_{1},p_{2})\in P_{s}}X_{s}(p_{1},p_{2})$.

  * •

Frequency of one-step equilibrium, which quantifies the proportion of
simulations in which the AI driven platforms learn the action that achieves
maximal one-step reward given that the opponent plays with the final
$Q$-function. Note that the underlying learning involves no memory. We compute
this frequency, denoted by $Y_{k}^{(\textnormal{one})}\in[0,1]$ for
$k\in\\{1,\ldots,5\\}$, according to the following three steps:

    1. 1.

Let
$Q^{(\textnormal{one})}_{i}(p,x)=\pi^{(1)}(p,\operatorname*{arg\,max}_{p^{\prime}}Q^{*}_{2}(p^{\prime},x)).$

    2. 2.

Let
$p^{(\textnormal{one})}:=\operatorname*{arg\,max}_{p^{\prime}}Q^{(\textnormal{one})}_{1}(p^{\prime},(p_{1},p_{2}))$.
We define $X_{s}^{(\textnormal{one})}(p_{1},p_{2})=1$ if and only if either
$p^{(\textnormal{one})}=p^{*}$ or $p^{(\textnormal{one})}$ and $p^{*}$ are
neighbors with distance at most $1$ in the set of indices of the $Q$-matrix
function.

    3. 3.

Let $Y_{k}^{(\textnormal{one})}=\frac{1}{|C_{k}|}\sum_{s\in
C_{k}}\frac{1}{|P_{s}|}\sum_{(p_{1},p_{2})\in
P_{s}}X_{s}^{(\textnormal{one})}(p_{1},p_{2})$.

  * •

Frequency of converging back to the limiting action, which quantifies the
proportion of simulations in which after one platform unilaterally deviates to
the one-stage Nash equilibrium price, platforms return back to the limiting
action. We compute this frequency, $Y_{k}^{(\textnormal{b})}\in[0,1]$ for
$k\in\\{1,\ldots,5\\}$, according to the following three steps:

    1. 1.

At time $T+1$, set $p_{T+1}^{(1)}=(p_{b}^{*},p_{s}^{*})$, while
$p_{T+1}^{(2)}\in\operatorname*{arg\,max}_{p}Q_{2}^{*}(p,(p_{T}^{(1)},{p_{T}^{(2)}}))$.
For $\tau\in\\{2,\ldots,101\\}$ and $i\in\\{1,2\\}$, let
$p_{T+\tau}^{(i)}\in\operatorname*{arg\,max}_{p}Q_{i}^{*}(p,(p_{T+\tau-1}^{(1)},{p_{T+\tau-1}^{(2)}}))$.

    2. 2.

Let $p^{(\textnormal{b}),(i)}:=p_{T+101}^{(i)}$. We define
$X_{s}^{(\textnormal{b})}(p^{(\textnormal{b}),(1)},p^{(\textnormal{b}),(2)})=1$
if and only if $(p^{(\textnormal{b}),(1)},p^{(\textnormal{b}),(2)})\in P_{s}$.

    3. 3.

Let $Y_{k}^{(\textnormal{b})}=\frac{1}{|C_{k}|}\sum_{s\in
C_{k}}X_{s}^{(\textnormal{b})}(p^{(\textnormal{b}),(1)},p^{(\textnormal{b}),(2)})$.

  * •

Frequency of converging back to the limiting action for both platforms, which
quantifies the proportion of simulations in which after both platforms deviate
to the one-stage Nash equilibrium price, they return back to the limiting
action. This frequency is computed similarly to the one above, where the only
difference is that $p_{T+1}^{(1)}=p_{T+1}^{(2)}=(p_{b}^{*},p_{s}^{*})$.

  * •

Average and standard deviation of the $Q$-loss w.r.t. $P_{s}$. The $Q$-loss
quantifies how close are the observed rewards on path $P_{s}$,
$\sum_{\tau=0}^{100}\delta^{\tau}\pi_{t+\tau}^{(i)}$, to the optimal rewards
given by the best response $Q_{i}^{(0)}$. We highlight that after convergence,
the observed rewards should be pretty close in value to
$\max_{p\in{\mathcal{A}}}Q^{*}_{i}(p,x)$ for each state $x\in P$. Let
$T_{0}=T-5000$, we compute the $Q$-loss for a given simulation $s$ as

$\frac{1}{|P_{s}|}\sum_{t=T_{0}}^{T_{0}+|P_{s}|}\left|\sum_{\tau=0}^{100}\delta^{\tau}\pi_{t+\tau}^{(i)}-\max_{p\in{\mathcal{A}}}Q^{0}_{i}(p,x_{t})\right|.$

The average and standard deviation are taken w.r.t. all simulations $s\in
C_{K}$ for any $k\in\\{1,\ldots,5\\}$.

  * •

Average and standard deviation of the $Q$-loss w.r.t. all states. The $Q$-loss
w.r.t. all states quantifies in average how close are the maximum values of
$Q^{*}_{i}(\cdot,x)$ and $Q^{0}_{i}(\cdot,x)$, where the maximum is taken over
all actions and the average over all states $x\in{\mathcal{S}}$. We compute
the $Q$-loss w.r.t. all states for a given simulation $s$ as

$\frac{1}{|{\mathcal{S}}|}\sum_{x\in{\mathcal{S}}}\left|\max_{p\in{\mathcal{A}}}Q^{*}_{i}(p,x)-\max_{p\in{\mathcal{A}}}Q^{0}_{i}(p,x)\right|.$

The average and standard deviation are taken with respect to all simulations
$s\in C_{K}$ for any $k\in\\{1,\ldots,5\\}$. When the $Q$-loss w.r.t. all
states equals $0$, platforms’ maximal rewards of the final $Q$-function yield
the same maximal rewards of the best response $Q$-function. We interpret this
as follows: the final $Q$-function exhibits behavior consistent with
equilibrium off-path, also known as subgame perfection.

Tables A1, A2 and A3 show the above measures for the following three
respective choices of $\Phi$: $\Phi_{1}:=[0,0;0,0]$, $\Phi_{2}:=[1,0;0,1]$ and
$\Phi_{3}:=[0,1;1,0]$. The other parameters are set as in the default setting,
i.e., $\delta=0.05$, $\beta_{k}=1$, $u_{k}^{(0)}=-2$.

In all tables, the frequency of symmetric 1-cycles is relatively small, where
its average is 5.8%. Even though this event could be considered rare, this
category is important since within it we can find one-memory stationary
equilibria, which are of great interest (see, e.g., Barlo et al., (2016) and
Chica et al., (2024)). Note that the frequency of asymmetric 1-cycles is very
small, where its average is 2.7%. We will thus omit interpretations of results
for this category. The average of both frequencies of cycles of length 2-4 and
5-8 is 34% and the one for cycles of length at least 9 is 23.3%.

The average collusive level among all tables and categories, excluding 1-Asy,
is approximately 29%. This value is only slightly below the baseline collusive
level $\Delta_{0}$.

For 1-Sym, the frequencies of equilibrium are 46%, 28% and 52% for $\Phi_{1}$,
$\Phi_{2}$ and $\Phi_{3}$, respectively. Thus, approximately half of the
simulations within 1-Sym result in equilibrium behavior for $\Phi_{1}$ and
$\Phi_{3}$, with a smaller value of 28% for $\Phi_{2}$. These numbers suggest
that in cases of zero-externalities or non-zero cross-side externalities,
equilibrium behavior is more likely than in cases of non-zero within-side
externalities. Note that for the frequency of one-step equilibrium in 1-Sym,
we still observe a larger proportion of one-step equilibrium for $\Phi_{1}$
and $\Phi_{3}$ compared to $\Phi_{2}$.

For the categories C2-4, C5-8 and C9, the average frequencies of equilibrium
are 28.8%, 42.9% and 49.4% for $\Phi_{1}$, $\Phi_{2}$ and $\Phi_{3}$,
respectively. Thus, on average 46% of the simulations with cycle lengths
greater than one result in equilibrium behavior for $\Phi_{2}$ and $\Phi_{3}$,
with a smaller value of 28.8% for $\Phi_{1}$. These numerical results suggest
that in cases of non-zero externalities and for cycles of lengths greater than
one, equilibrium behavior is more likely than in the zero-externality case.
The above results for 1-Sym, C2-4, C5-8 and C9 suggest that in a sufficiently
large percentage of cases, AI driven platforms exhibit behavior that is
consistent with Nash equilibrium.

The frequencies of converging back to the limiting action for one platform in
the 1-Sym category are 84%, 81% and 76% for $\Phi_{1}$, $\Phi_{2}$ and
$\Phi_{3}$, respectively. Thus, in at least 76% of the cases, platforms
converge back to the limiting action after one unilateral price change to the
Nash equilibrium price. For the categories C2-4, C5-8 and C9, the frequencies
of converging back to the limiting action for one platform are 90%, 92% and
95% for $\Phi_{1}$, $\Phi_{2}$ and $\Phi_{3}$, respectively. Note that the
smallest frequency of converging back to the limiting action for one platform
in the 1-Sym category is achieved at $\Phi_{3}$, which suggests that non-zero
cross-side externalities make it harder to sustain collusion. A similar result
is shown by Ruhmer, (2010) for a case of platform competition without AI
agents. Nonetheless, this result does not hold for cycles of lengths greater
than one, where the smallest frequency is achieved at $\Phi_{1}$, i.e., at the
zero-externalities case. As a general observation, the above results suggest
that AI driven platforms can learn behavior that is consistent with tacit
collusion. Note that similar behavior is found for the frequencies of
converging back to the limiting action for both platforms.

The average $Q$-losses (on path) for all tables and categories, excluding
1-Asy, are all less than $0.019$. This observation and the definition of the
$Q$-loss imply that the observed rewards on path are sufficiently close to the
optimal rewards given by the best response matrix. These results suggest that
the likelihood of the final action achieving maximal total future reward is
very high. Similarly, the average $Q$-losses w.r.t. all states for all
categories, excluding 1-Asy, are less than $0.043$. Thus, platforms’ maximal
rewards of the final $Q$-function are relatively close to the maximal rewards
of the best response $Q$-function. Furthermore, the $Q$-function exhibits
behavior consistent with the equilibrium off-path or subgame perfection.

Metric | 1-Sym | 1-Asy | C2-4 | C5-8 | C9 | All
---|---|---|---|---|---|---
Freq. | 0.040 | 0.025 | 0.300 | 0.307 | 0.328 | 1.000
Avg. Collusive Level | 0.273 | 0.326 | 0.294 | 0.301 | 0.295 | 0.297
S.D. of Collusive Level | 0.082 | 0.069 | 0.063 | 0.049 | 0.042 | 0.054
Freq. of Eq. | 0.462 | 0.0 | 0.279 | 0.296 | 0.29 | 0.288
Freq. of one-step Eq. | 0.385 | 0.125 | 0.314 | 0.324 | 0.345 | 0.325
Freq. Conv. back (one) | 0.846 | 0.750 | 0.907 | 0.909 | 0.896 | 0.898
Freq. Conv. back (both) | 0.846 | 0.750 | 0.856 | 0.879 | 0.925 | 0.882
Avg. Q loss (on path) | 0.003 | 0.006 | 0.005 | 0.005 | 0.005 | 0.005
S.D. of Q loss (on path) | 0.001 | 0.004 | 0.003 | 0.002 | 0.001 | 0.002
Avg. Q loss (all) | 0.009 | 0.009 | 0.009 | 0.009 | 0.009 | 0.009
S.D. of Q loss (all) | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000
Table A1: Sensitivity analysis with $\Phi=\Phi_{1}:=[0,0;0,0]$. Metric | 1-Sym | 1-Asy | C2-4 | C5-8 | C9 | All
---|---|---|---|---|---|---
Freq. | 0.084 | 0.013 | 0.355 | 0.368 | 0.179 | 1.000
Avg. Collusive Level | 0.297 | 0.433 | 0.298 | 0.301 | 0.322 | 0.305
S.D. of Collusive Level | 0.061 | 0.230 | 0.065 | 0.058 | 0.067 | 0.068
Freq. of Eq. | 0.281 | 0.0 | 0.44 | 0.436 | 0.411 | 0.414
Freq. of one-step Eq. | 0.063 | 0.000 | 0.366 | 0.326 | 0.320 | 0.313
Freq. Conv. back (one) | 0.812 | 0.400 | 0.904 | 0.957 | 0.912 | 0.911
Freq. Conv. back (both) | 0.812 | 0.400 | 0.889 | 0.943 | 0.897 | 0.897
Avg. Q loss (on path) | 0.011 | 0.042 | 0.015 | 0.017 | 0.019 | 0.016
S.D. of Q loss (on path) | 0.005 | 0.030 | 0.011 | 0.010 | 0.011 | 0.011
Avg. Q loss (all) | 0.043 | 0.043 | 0.043 | 0.043 | 0.043 | 0.043
S.D. of Q loss (all) | 0.001 | 0.002 | 0.001 | 0.001 | 0.001 | 0.001
Table A2: Sensitivity analysis with $\Phi=\Phi_{2}:=[1,0;0,1]$. Metric | 1-Sym | 1-Asy | C2-4 | C5-8 | C9 | All
---|---|---|---|---|---|---
Freq. | 0.049 | 0.042 | 0.364 | 0.353 | 0.193 | 1.000
Avg. Collusive Level | 0.264 | 0.282 | 0.279 | 0.265 | 0.287 | 0.275
S.D. of Collusive Level | 0.057 | 0.079 | 0.075 | 0.053 | 0.061 | 0.065
Freq. of Eq. | 0.524 | 0.336 | 0.533 | 0.499 | 0.451 | 0.497
Freq. of one-step Eq. | 0.245 | 0.342 | 0.369 | 0.365 | 0.321 | 0.351
Freq. Conv. back (one) | 0.762 | 0.500 | 0.930 | 0.967 | 0.976 | 0.926
Freq. Conv. back (both) | 0.667 | 0.333 | 0.904 | 0.954 | 0.964 | 0.898
Avg. Q loss (on path) | 0.009 | 0.019 | 0.013 | 0.012 | 0.015 | 0.013
S.D. of Q loss (on path) | 0.004 | 0.010 | 0.009 | 0.006 | 0.009 | 0.008
Avg. Q loss (all) | 0.038 | 0.038 | 0.038 | 0.038 | 0.038 | 0.038
S.D. of Q loss (all) | 0.001 | 0.001 | 0.001 | 0.001 | 0.001 | 0.001
Table A3: Sensitivity analysis with $\Phi=\Phi_{3}:=[0,1;1,0]$.

Figure A1 shows four examples of convergence paths, one from C2-4 (top left
panel), one from C5-8 (top right panel) and two from C9 (bottom left and right
panels), where $\Phi=[0,1;1,0]$. The horizontal blue dashed lines at the
bottom and top portion of each panel represent the Nash and Collusion
equilibrium prices, respectively, i.e., the lines at $p_{b}^{*}=p_{s}^{*}$ and
$p_{b}^{\text{C}}=p_{s}^{\text{C}}$, respectively. Note that buyer and seller
prices are equal due to the symmetric choice of $\Phi$. The vertical blue
dotted lines represent where a cycle ends, while the blue and orange curves
represent the buyer and seller prices, respectively, chosen by platform 1 on
the equilibrium path. The top left panel shows a convergence path of length
four, in which the buyer’s price starts above the Nash equilibrium price.
Then, in two steps, it reaches the Nash equilibrium price, followed by a
sudden increase to the starting price. This price pattern is similar to an
Edgeworth cycle, where prices start well above the Nash equilibrium price,
then slowly converge to the Nash equilibrium, followed by a sudden increase to
the initial high price (see, .e.g., Maskin and Tirole, (1988)). Note that the
seller’s price oscillates between two levels above the Nash equilibrium price.
The top right and bottom left and right subfigures show more intricate
patterns. However, a common feature among them is that prices oscillate
between the Nash equilibrium price and a level higher than the Nash price.
This indicates that platforms learn behavior consistent with punishment and
reward strategies.

Figure A1: Examples of convergence paths from categories C2-4 (top left
panel), C5-8 (top right panel) and C9 (bottom left and right panels), where
$\Phi=[0,1;1,0]$. The horizontal blue dashed lines at the bottom and top of
each panel represent the lines at $p_{b}^{*}=p_{s}^{*}$ and
$p_{b}^{\text{C}}=p_{s}^{\text{C}}$, respectively. The vertical blue dotted
lines indicate where a cycle ends. The blue and orange curves represent the
buyer and seller prices, respectively, chosen by platform 1.

Finally, Figure A2 illustrates four scenarios depicting the convergence path
of platform 1 after both platforms revert to the limiting action following a
deviation to the one-stage Nash equilibrium price by both platforms. The top
left and right panels present an example for $\Phi_{1}$, while the bottom left
and right panels provide examples for $\Phi_{2}$ and $\Phi_{3}$, respectively.
In all four cases, the horizontal blue dashed lines at the bottom and top of
each panel represent the lines at $p_{b}^{*}=p_{s}^{*}$ and
$p_{b}^{\text{C}}=p_{s}^{\text{C}}$, respectively. The vertical blue dotted
lines indicate where each cycle starts and ends. The blue and orange curves
represent the buyer and seller prices, respectively, chosen by platform 1
immediately after the deviation to the Nash equilibrium price. The green and
red curves represent platform 1’s buyer and seller prices once they revert to
the limiting action. In all four panels, we observe that in less than ten
steps, platform 1’s prices revert to the limiting action (the leftmost
vertical dotted line appears before step 10). The top left panel shows that
after one deviation to the Nash equilibrium price by both platforms, platform
1’s prices slowly increase above the Nash equilibrium price, followed by a
small decrease and then a convergence to a price above the Nash equilibrium
price. Similar behavior is observed in the top right panel. The bottom left
and right panels show more intricate patterns. However, as mentioned earlier,
in both figures, prices converge back to the limiting action after 9 steps.
The patterns they follow after reaching the limiting action indicate that
firms oscillate between the Nash equilibrium price and a price higher than the
Nash equilibrium price.

Figure A2: Four examples of convergence paths after both platforms revert to
the limiting action following a deviation to the one-stage Nash equilibrium
price by both platforms. There are two examples for $\Phi_{1}$ (top left and
right panels), one for $\Phi_{2}$ (bottom left), and one for $\Phi_{3}$
(bottom right). The blue and orange curves represent the buyer and seller
prices, respectively, chosen by platform 1 immediately after the deviation to
the Nash equilibrium price. The green and red curves represent platform 1’s
buyer and seller prices once they revert to the limiting action.

###  A.2 Fitting a Sequence of Non-linear Functions

We provide the details of how to fit the univariate and bivariate functions in
(19) in order to describe how the collusive level depends on the externality
matrix $\Phi$ via an additive model. We first let $\Delta_{0}$ equal the
sample mean of $\tilde{\Delta}$. In order to find the four univariate
functions $f_{bb}(\phi_{bb}),f_{ss}(\phi_{ss}),f_{bs}(\phi_{bs})$ and
$f_{sb}(\phi_{sb})$, we fit them sequentially using residuals. We consider all
the twenty-four permutations of 4 elements, represented by $bb$, $ss$, $bs$,
$sb$. Without loss of generality, we describe the fitting procedure for a
given permutation $o:=(bb,ss,bs,sb)$, where the same procedure is followed for
all other permutations. We follow a regression setting with the predictor
$\phi_{bb}$ and the response

$y_{bb}:=\tilde{\Delta}-\Delta_{0},$

and we apply XGBoost, which is a nonlinear regression method, to fit
$\hat{f}^{(o)}_{bb}(\phi_{bb})$ that approximates $y_{bb}$. For the above $o$,
we proceed from $bb$ to $ss$ as follows. We define

$y_{ss}:=y_{bb}-\hat{f}_{bb}^{(o)}(\phi_{bb})$

and consider a regression setting with $y_{ss}$ as response and $\phi_{ss}$ as
predictor. We apply XGBoost to approximate $y_{ss}$ by
$\hat{f}_{ss}^{(o)}(\phi_{ss})$. Similarly, we transition from $ss$ to $bs$
using the residuals

$y_{bs}:=y_{ss}-\hat{f}_{ss}^{(o)}(\phi_{ss})$

and fitting $\hat{f}^{(o)}_{bs}(\phi_{bs})$ to approximate $y_{bs}$ by
XGBoost, and we transition from $bs$ to $sb$ using the residual

$y_{sb}:=y_{bs}-\hat{f}^{(o)}_{bs}(\phi_{ss})$

and fitting $\hat{f}^{(o)}_{sb}(\phi_{sb})$ to approximate $y_{sb}$ by
XGBoost. We repeat this process for the rest of the twenty-three permutations
and thus obtain twenty-four approximation for the functions
$f_{bb}(\phi_{bb})$, $f_{ss}(\phi_{ss})$, $f_{bs}(\phi_{bs})$ and
$f_{sb}(\phi_{sb})$, specified in (19). We average over the twenty-four
approximations to obtain the estimator
$\hat{f}_{bb}(\phi_{bb}),\hat{f}_{ss}(\phi_{ss}),\hat{f}_{bs}(\phi_{bs})$ and
$\hat{f}_{sb}(\phi_{sb})$.

Next, we apply the procedure described above for the six bivariate functions
in (19), such as $f_{bb,ss}(\phi_{bb},\phi_{ss})$ and
$f_{bs,sb}(\phi_{bs},\phi_{sb})$. For a given permutation of the six pairs of
variables, e.g.,

$o=((bb,ss),(bs,sb),(bb,sb),(bb,bs),(ss,sb),(ss,bs)),$

we follow the same procedure introduced above to iteratively fit the bivariate
functions using XGBoost. In the first iteration, the response variable is

$y_{1}:=\tilde{\Delta}-\left(\Delta_{0}+\hat{f}_{bb}(\phi_{bb})+\hat{f}_{ss}(\phi_{ss})+\hat{f}_{bs}(\phi_{bs})+\hat{f}_{sb}(\phi_{sb})\right)$

and for the above permutation $o$ there are two predictors $\phi_{bb}$ and
$\phi_{ss}$. XGBoost then approximates $y_{1}$ by the function
$\hat{f}^{(o)}(\phi_{bb},\phi_{ss})$. In the following iterations, the
response is the corresponding residual and the predictors are the two
corresponding variables. For example, assuming the permutation $o$, in the
second iteration the response variable is

$y_{1}-\hat{f}^{(o)}(\phi_{bb},\phi_{ss})$

and the predictors are $\phi_{bs}$ and $\phi_{sb}$. We average the obtained
approximations over all 720 possible permutations of these six bivariate
functions to approximate each bivariate function.

###  A.3 Other Simulation Results

We provide Figures A3 and A4, which complement Figures 9 and 10, respectively,
with additional choices of the externality matrix.

A common trend in both Figures 9 and A3 is that the collusive level decreases
as $\beta_{k}$ increases.

Figure A3: The collusive level with varying $\beta_{k}\in[0.2,5]$; Top left:
$\Phi=[0,1;1,0]$; Top right: $\Phi=[0,-1;-1,0]$; Bottom left:
$\Phi=[0,1;0,0]$; Bottom right: $\Phi=[1,0;0,-1]$.

Section 4.3 summarizes the main common observation to both Figure 10 and
Figure A4.

Figure A4: The collusive level with varying $u^{(0)}_{k}$, for
$k\in\\{b,s\\}$, where $\Phi=[1,0;0,0]$ (top left), $\Phi=[0,1;0,0]$ (top
right), $\Phi=[0,1;-1,0]$ (middle left), $\Phi=[1,0;0,-1]$ (middle right),
$\Phi=[2,0;0,2]$ (bottom left), and $\Phi=[0,2;2,0]$ (bottom right).