# On semi-restricted Rock, Paper, Scissors

Svante Janson  Department of Mathematics, Uppsala University, PO Box 480,
SE-751 06 Uppsala, Sweden<EMAIL_ADDRESS>
(Date: 22 February, 2024)

###### Abstract.

Spiro, Surya and Zeng (Electron. J. Combin. 2023) recently studied a semi-
restricted variant of the well-known game Rock, Paper, Scissors; in this
variant the game is played for $3n$ rounds, but one of the two players is
restricted and has to use each of the three moves exactly $n$ times. They find
the optimal strategy, and they show that it results in an expected score for
the unrestricted player $\Theta(\sqrt{n})$; they conjecture, based on
numerical evidence, that the expectation is $\approx 1.46\sqrt{n}$.

We analyse the result of the strategy further and show that the average is
$\sim c\sqrt{n}$ with $c=3\sqrt{3}/2\sqrt{\pi}=1.466$, verifying the
conjecture. We also find the asymptotic distribution of the score, and compute
its variance.

###### 2020 Mathematics Subject Classification:

Supported by the Knut and Alice Wallenberg Foundation and the Swedish Research
Council

##  1\. Introduction

A semi-restricted variant of the well-known game Rock, Paper, Scissors (RPS)
was recently studied by Spiro, Surya and Zeng [5]. In the standard version of
RPS, two players simultaneously select one of the three choices _rock_ ,
_paper_ , _scissors_ , where _paper_ beats _rock_ , _scissors_ beats _paper_ ,
and _rock_ beats _scissors_ ; if both select the same, the result is a draw.
The game is symmetric, so there is obviously no advantage to any of the
players. It is easy to see that the optimal strategy for both players is to
choose randomly, with equal probability for each choice (see further Section
2.2).

In the semi-restricted variant in [5], two players R (restricted) and N
(normal) agree to play $3n$ rounds of RPS for some integer $n$, but R is
restricted to choose _rock_ , _paper_ , and _scissors_ exactly $n$ times each,
while N plays without restriction. Clearly, the restriction is a disadvantage
for R. (In particular, N will always win the last round, since R then has only
one choice, and N knows which one.) How large is this disadvantage? More
precisely, let $S_{n}$ be the final score of N, defined as the number of
rounds won by N minus the number lost. We assume (as [5]) that the objective
of both players is the expectation $\operatorname{\mathbb{E}}{}S_{n}$, which N
wants as high as possible, while R wants the opposite. We assume also that
both players play optimally, i.e, they use their optimal randomized strategy.
(Which always exists by the theory of von Neumann [4], since RPS is a two-
player zero-sum game; see further e.g [3, Chapter 2].)

The main result of [5] is that the (unique) optimal strategy for R is to play
greedily, i.e, as if each round were the last; see further Section 2.2. (This
is far from obvious, and rather surprising.) It is also shown in [5] that with
optimal strategies, the expected gain
$\operatorname{\mathbb{E}}{}S_{n}=\Theta(\sqrt{n})$, and it is asked [5,
Question 21] whether $\operatorname{\mathbb{E}}{}S_{n}\sim c\sqrt{n}$ for some
constant $c>0$ as ${n\to\infty}$; [5] says further that numerical calculations
for $n\leqslant 100$ suggest that this might hold with $c\approx 1.46$.

The main purpose of the present note is to verify this conjecture, and to
identify the constant; we also find the asymptotic distribution of the final
score $S_{n}$. The main results are collected in the two theorems below; the
proofs (given in Sections 3 and Section 4) are based on the main result of
[5], and an analysis of the strategy found there.

######  Theorem 1.1.

For semi-restricted RPS played $3n$ rounds, the expected score for N with
optimal plays for both players is, as ${n\to\infty}$,

$\displaystyle\operatorname{\mathbb{E}}{}S_{n}\sim\sqrt{\frac{27n}{4\pi}}=\frac{3\sqrt{3}}{2\sqrt{\pi}}\sqrt{n}.$
(1.1)

The constant $3\sqrt{3}/(2\sqrt{\pi})\doteq 1.4658$, which verifies also the
numerical conjecture in [5].

For the asymptotic distribution, we give several alternative descriptions. See
Sections 4 and 5 for uses of some of them.

######  Theorem 1.2.

As ${n\to\infty}$, we have convergence in distribution, together with all
moments,

$\displaystyle n^{-1/2}S_{n}\overset{\mathrm{d}}{\longrightarrow}S^{*}$ (1.2)

where the limit $S^{*}$ can be described by any of the following equivalent
formulas:

  1. (i)

We have

$\displaystyle S^{*}=W+\max\bigl{\\{}Z_{1},Z_{2},Z_{3}\bigr{\\}}$ (1.3)

where $W,Z_{1},Z_{2},Z_{3}$ are jointly normal with $W$ independent of
$(Z_{1},Z_{2},Z_{3})$ and

$\displaystyle W$ $\displaystyle\in N(0,2),$ (1.4)
$\displaystyle(Z_{1},Z_{2},Z_{3})$ $\displaystyle\in
N\left(0,\begin{pmatrix}\phantom{-}2&-1&-1\\\ -1&\phantom{-}2&-1\\\
-1&-1&\phantom{-}2\end{pmatrix}\right).$ (1.5)

  2. (ii)

We have

$\displaystyle
S^{*}=W^{\prime}+\sqrt{3}\max\bigl{\\{}Z^{\prime}_{1},Z^{\prime}_{2},Z^{\prime}_{3}\bigr{\\}}$
(1.6)

where $W^{\prime},Z^{\prime}_{1},Z^{\prime}_{2},Z^{\prime}_{3}$ are
independent standard normal $N(0,1)$.

  3. (iii)

We have

$\displaystyle
S^{*}=\max\bigl{\\{}Z^{\prime\prime}_{1},Z^{\prime\prime}_{2},Z^{\prime\prime}_{3}\bigr{\\}}$
(1.7)

where $Z_{1}^{\prime\prime},Z^{\prime\prime}_{2},Z^{\prime\prime}_{3}$ are
jointly normal with

$\displaystyle(Z^{\prime\prime}_{1},Z^{\prime\prime}_{2},Z^{\prime\prime}_{3})$
$\displaystyle\in N\left(0,\begin{pmatrix}4&1&1\\\ 1&4&1\\\
1&1&4\end{pmatrix}\right).$ (1.8)

  4. (iv)

We have

$\displaystyle S^{*}=W+R\cos\theta,$ (1.9)

where $W,R,\theta$ are independent with $W\in N(0,2)$ as in (1.4), $R$ has a
Rayleigh distribution with density $\frac{1}{2}re^{-r^{2}/4}$, $r>0$, and
$\theta$ has a uniform distribution $U(0,\pi/3)$.

Note that the normal variables $Z_{1},Z_{2},Z_{3}$ in (1.3) satisfy
$Z_{1}+Z_{2}+Z_{3}=0$ a.s, as a consequence of (1.5); thus
$(Z_{1},Z_{2},Z_{3})$ lives in a 2-dimensional space. See further the proof in
Section 4.

The moment convergence in Theorem 1.2 enables us to find also asymptotics of
higher moments of $S_{n}$, see Section 4.

In Section 5, we discuss the probability that the disadvantaged player R
nevertheless wins the game; we compute it for the strategies used above, but
leave the case of optimal play for this objective as an open problem.

##  2\. Preliminaries

###  2.1. Notation

The three choices _rock_ , _paper_ , _scissors_ will be numbered $1,2,3$; thus
$i+1$ beats $i\pmod{3}$.

The random variable $S(t)$ is the score of N after round $t=1,\dots,3n$, i.e.,
the number of rounds won by N so far minus the number of rounds won by R. As
in the introduction, $S_{n}:=S(3n)$ is the score at the end of the game.
(Except for $S_{n}$, we do not show $n$ explicitly in the notation, although
$S(t)$ and many variables introduced below depend on $n$.)

If $X_{n}$ is a sequence of random variables, and $a_{n}$ a sequence of
(positive) numbers, we write $X_{n}=O_{\mathrm{p}}(a_{n})$ if the family
$\\{X_{n}/a_{n}\\}$ is bounded in probability (also called _tight_), i.e, if
for every $\varepsilon>0$ there exists $C$ such that
$\operatorname{\mathbb{P}{}}(|X_{n}|>Ca_{n})<\varepsilon$ for all $n$.
Furthermore, we write $X_{n}=O_{L^{p}}(a_{n})$ (where $p>0$ is a parameter) if
the family $\\{X_{n}/a_{n}\\}$ is bounded in $L^{p}$, i.e,
$\sup_{n}\operatorname{\mathbb{E}}{}|X_{n}/a_{n}|^{p}<\infty$.

The basis vectors in $\mathbb{R}^{3}$ are denoted $\mathbf{e}_{1}:=(1,0,0)$,
$\mathbf{e}_{2}:=(0,1,0)$, $\mathbf{e}_{3}:=(0,0,1)$.

$C$ denotes unspecified constants that may vary from one occurrence to the
next; we use $C_{p}$ for constants that depends on the parameter $p$.

Unspecified limits are as ${n\to\infty}$.

###  2.2. The greedy strategy

Recall that in any two-person zero-sum game, each player has an optimal
strategy which in general is randomized; the different alternatives are
selected with some probabilities chosen such that they maximize the minimum
over all strategies of the opponent of the expected gain; see [4] and e.g [3].

As said above, it was shown by Spiro, Surya and Zeng [5] that in semi-
restricted RPS, the best strategy of R is to play greedily, i.e, to analyse
each round separately and use the optimal strategy for the expected score in
that round. (This is far from obvious, since the best play in one specific
round may be punished by lower expected score in later rounds; nevertheless,
[5] shows that the expected later gains by any alternative strategy are offset
by the immediate expected loss.) This optimal strategy for a single round is
easy to find (as was done in [5]):

  1. (i)

If R still has all three choices available, then the optimal strategy is
(obviously, by symmetry), to choose one of them randomly, with probability
$1/3$ each. And the best strategy for N is the same. (This game was one of the
examples in the original paper by von Neumann [4].) The outcome for N is $-1$,
$0$, or $+1$ with probability $1/3$ each.

  2. (ii)

If R has only two choices available, say 1 (_rock_) and 2 (_paper_), then the
game is described by the matrix in Figure 1. N should never play 1 (which in
this case can lose but never win). A simple calculation shows [5] that the
best strategy for R is to play 1 with probability $1/3$ and 2 with probability
$2/3$; similarly N plays 2 with probability $2/3$ and 3 with probability
$1/3$. The expected gain for N is $1/3$.

| _rock_ | _paper_ | _scissors_
---|---|---|---
_rock_ | 0 | 1 | $-1$
_paper_ | $-1$ | 0 | 1
Figure 1. Score matrix for N when R is restricted to
$\\{\emph{rock},\emph{paper}\\}$; rows show the move by R; columns the move by
N.

  3. (iii)

If R has only one choice, then R has to play that, and N obviously plays the
next choice $\pmod{3}$ and is sure to win. Gain for N is 1.

##  3\. Analysis of the greedy strategy

Let $N_{t,i}$ be the number of times that R plays $i$ during rounds
$1,\dots,t$. The vector $\mathbf{N}_{t}=(N_{t,i})_{i=1}^{3}$ then evolves as a
random walk which changes character each time some $N_{t,i}$ hits $n$ and R
thus cannot choose $i$ in the future. We let $T_{j}$, $j=1,2,3$, be the first
time that R has used up $j$ of the three choices; in particular, $T_{3}:=3n$,
when the game ends.

Since R uses the greedy strategy described above, $\mathbb{N}_{t}$ evolves as
follows, for $t=0,\dots,n$, starting at $\mathbf{N}_{0}=(0,0,0)$:

  1. I.

A random walk $\mathbf{N}_{0},\dots,\mathbf{N}_{T_{1}}$ with increments that
are independent and uniformly chosen from
$\\{\mathbf{e}_{1},\mathbf{e}_{2},\mathbf{e}_{3}\\}$, until

$\displaystyle T_{1}$ $\displaystyle:=\inf\bigl{\\{}t:N_{t,i}=n\text{ for some
$i\in\\{1,2,3\\}$}\bigr{\\}}.$ (3.1)

  2. II.

A random walk $\mathbf{N}_{T_{1}},\dots,\mathbf{N}_{T_{2}}$ with increments
chosen independently and randomly from the remaining two choices by the
strategy above; for example, if $N_{t,1}$ hits $n$ first, so
$N_{T_{1},1}=n>N_{T_{1},2},N_{T_{1},3}$, then the increments are chosen as
$\mathbf{e}_{2}$ and $\mathbf{e}_{3}$ with probabilities $1/3$ and $2/3$. This
goes on until

$\displaystyle T_{2}$ $\displaystyle:=\inf\bigl{\\{}t:N_{t,i}=n\text{ for at
least two $i\in\\{1,2,3\\}$}\bigr{\\}}.$ (3.2)

  3. III.

A deterministic walk $\mathbf{N}_{T_{2}},\dots,\mathbf{N}_{T_{3}}$ where all
increments are $\mathbf{e}_{i}$ for the only $i$ that still has $N_{t,i}<n$.

We note that the expected gain for N is (assuming optimal play) $0$ for each
step in phase I, $1/3$ for each step in phase II, and 1 for each step in phase
III, so the expected score for N is
$\operatorname{\mathbb{E}}{}[(T_{2}-T_{1})/3+T_{3}-T_{2}]$; we will analyse
this more carefully below and also both bound and asymptotically describe the
random fluctuations. We do this by analysing the constrained random walk
$\mathbf{N}_{t}$ and the stopping times $T_{1}$ and $T_{2}$ in some detail. A
central role is palyed by the (somewhat arbitrary) non-random time

$\displaystyle T_{0}:=3n-3\lceil n^{2/3}\rceil.$ (3.3)

###  3.1. Phase I: until $T_{1}$

Let $(\boldsymbol{\xi}_{t})_{t=1}^{\infty}$ be an i.i.d sequence of random
vectors with the distribution
$\operatorname{\mathbb{P}{}}(\boldsymbol{\xi}_{t}=\mathbf{e}_{i})=1/3$ for
$i=1,2,3$. We may assume that
$\mathbf{N}_{t}-\mathbf{N}_{t-1}=\boldsymbol{\xi}_{t}$ for $1\leqslant
t\leqslant T_{1}$. Let

$\displaystyle\mathbf{N}^{\prime}_{t}=(N^{\prime}_{t,i})_{i=1}^{3}:=\sum_{u=1}^{t}\boldsymbol{\xi}_{u},\qquad
t\geqslant 0;$ (3.4)

thus $\mathbf{N}^{\prime}_{t}=\mathbf{N}_{t}$ for $t\leqslant T_{1}$. (We may
interpret $\boldsymbol{\xi}_{t}$ and $\mathbf{N}^{\prime}_{t}$ as how R would
have played if the restriction had not existed.) In particular, for
$t\leqslant T_{1}$ we have $N^{\prime}_{t,i}=N_{t,i}\leqslant n$ for all $i$,
and for $t\geqslant T_{1}$ we have
$\max_{i}N^{\prime}_{t,i}\geqslant\max_{i}N_{T_{1},i}=n$; thus $T_{1}$ is also
the time that $\max_{i}N^{\prime}_{t,i}$ hits $n$.

At time $T_{0}$, the central limit theorem shows that

$\displaystyle
N^{\prime}_{T_{0},i}=\tfrac{1}{3}T_{0}+O_{\mathrm{p}}(n^{1/2})=n-n^{2/3}+O_{\mathrm{p}}\bigl{(}n^{1/2}\bigr{)}.$
(3.5)

This is w.h.p (with high probability, i.e, with probability $1-o(1)$ as
${n\to\infty}$) $<n$ for each $i$, and thus w.h.p $T_{1}>T_{0}$. More
precisely, the Chernoff inequality (e.g in the version in [2, Remark 2.5])
yields

$\displaystyle\operatorname{\mathbb{P}{}}(T_{1}\leqslant
T_{0})\leqslant\sum_{i=1}^{3}\operatorname{\mathbb{P}{}}(N^{\prime}_{T_{0},i}\geqslant
n)=3\operatorname{\mathbb{P}{}}\bigl{(}N^{\prime}_{T_{0},1}-\tfrac{1}{3}T_{0}\geqslant\lceil
n^{2/3}\rceil\bigr{)}\leqslant e^{-2n^{4/3}/T_{0}}\leqslant e^{-n^{1/3}}.$
(3.6)

Hence, this probability decreases faster than any polynomial, which means that
we can ignore the event $T_{1}\leqslant T_{0}$ also when calculating moments
below (since the random variables we consider all are deterministically
$O(n)$).

Similarly, concentrating on the time after $T_{0}$, define

$\displaystyle M^{\prime}:=\max_{i=1,2,3}\max_{T_{0}\leqslant t\leqslant
3n}\bigl{\lvert}N^{\prime}_{t,i}-N^{\prime}_{T_{0},i}-\tfrac{1}{3}(t-T_{0})\bigr{\rvert}.$
(3.7)

By classical results on moment convergence in the central limit theorem
together with Doob’s inequality (since
$N^{\prime}_{t,i}-N^{\prime}_{T_{0},i}-\tfrac{1}{3}(t-T_{0})$ is a
martingale), see for example [1, Theorem 7.5.1, Corollary 3.8.2, and Theorem
10.9.4], we have, for any $p>1$

$\displaystyle\operatorname{\mathbb{E}}{}(M^{\prime})^{p}$
$\displaystyle\leqslant\sum_{i=1}^{3}\operatorname{\mathbb{E}}{}\max_{T_{0}\leqslant
t\leqslant
3n}\bigl{\lvert}N^{\prime}_{t,i}-N^{\prime}_{T_{0},i}-\tfrac{1}{3}(t-T_{0})\bigr{\rvert}^{p}$
$\displaystyle\leqslant
C_{p}\sum_{i=1}^{3}\operatorname{\mathbb{E}}{}\bigl{\lvert}N^{\prime}_{3n,i}-N^{\prime}_{T_{0},i}-\tfrac{1}{3}(3n-T_{0})\bigr{\rvert}^{p}$
$\displaystyle\leqslant C_{p}(3n-T_{0})^{p/2}\leqslant C_{p}n^{p/3}.$ (3.8)

Consequently,

$\displaystyle M^{\prime}=O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}$ (3.9)

for every $p<\infty$.

We introduce some further notation. Let

$\displaystyle
X_{i}:=N^{\prime}_{T_{0},i}-\operatorname{\mathbb{E}}{}N^{\prime}_{T_{0},i}=N^{\prime}_{T_{0},i}-\tfrac{1}{3}T_{0}.$
(3.10)

Note for later use that

$\displaystyle X_{1}+X_{2}+X_{3}=\sum_{i=1}^{3}N^{\prime}_{T_{0},i}-T_{0}=0.$
(3.11)

Furthermore, let

$\displaystyle X_{\max}:=\max_{i=1,2,3}X_{i}.$ (3.12)

Condition on the event $T_{1}>T_{0}$, which has probability $1-o(1)$. Then,
$N^{\prime}_{T_{0}}=N_{T_{0}}$. Moreover, we may take $t=T_{1}$ in (3.7) and
obtain, using (3.10),

$\displaystyle
N_{T_{1},i}=N^{\prime}_{T_{1},i}=N^{\prime}_{T_{0},i}+\tfrac{1}{3}(T_{1}-T_{0})+O(M^{\prime})=X_{i}+\tfrac{1}{3}T_{1}+O(M^{\prime}).$
(3.13)

Hence, recalling the definitions of $T_{1}$ and $X_{\max}$,

$\displaystyle
n=\max_{i}N_{T_{1},i}=\max_{i}X_{i}+\tfrac{1}{3}T_{1}+O(M^{\prime})=X_{\max}+\tfrac{1}{3}T_{1}+O(M^{\prime}).$
(3.14)

Consequently,

$\displaystyle T_{1}=3n-3X_{\max}+O(M^{\prime})$ (3.15)

and thus, using (3.9),

$\displaystyle T_{1}=3n-3X_{\max}+O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}.$ (3.16)

This was derived conditioned on $T_{1}>T_{0}$, but by (3.6) and the comment
after it, (3.16) holds also unconditionally.

Furthermore, for every $i\in\\{1,2,3\\}$, by (3.13) and (3.9),

$\displaystyle
N_{T_{1},i}-\tfrac{1}{3}T_{1}=X_{i}+O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}$ (3.17)

and thus by (3.16)

$\displaystyle
n-N_{T_{1},i}=n-\tfrac{1}{3}T_{1}-X_{i}+O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}=X_{\max}-X_{i}+O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}.$
(3.18)

Thus, at time $T_{1}$, when R runs out of one of the three choices, she has
approximatively $X_{\max}-X_{i}$ left of each other choice $i$.

To find the score in Phase I, consider first the score at $T_{0}$, and
condition again on $T_{1}>T_{0}$. Then in each round up to $T_{0}$, R plays
normally and thus R and N win with probability $1/3$ each, and draw otherwise;
thus $\Delta S(t):=S(t)-S(t-1)\in\\{\pm 1,0\\}$ with probability $1/3$ each.
Consequently, the central limit theorem shows that, since
$\operatorname{\mathbb{E}}{}\Delta S(t)=0$ and $\operatorname{Var}\Delta
S(t)=2/3$, and $T_{0}\sim 3n$,

$\displaystyle\frac{S(T_{0})}{n^{1/2}}\overset{\mathrm{d}}{\longrightarrow}N(0,2),\qquad\text{as
}{n\to\infty},$ (3.19)

together with all moments. Moreover, since also N is assumed to use the
optimal strategy, which for these $t$ means uniformly randomly, the score in
each round is independent of the choices made by R, and thus of the vectors
$\mathbf{N}_{t}$. Consequently, $S(T_{0})$ is independent of
$(X_{1},X_{2},X_{3})$. We conditioned here on $T_{1}>T_{0}$, but in the
unlikely event $T_{1}\leqslant T_{0}$, we may modify $S(T_{0})$ (similarly as
we defined $\mathbf{N}^{\prime}$ above) and define a sum $S^{\prime}(T_{0})$
that is independent of $(X_{1},X_{2},X_{3})$ and satisfies
$S^{\prime}(T_{0})=S(T_{0})$ whenever $T_{1}>T_{0}$, and thus, by (3.6),
(rather coarsely)

$\displaystyle S(T_{0})=S^{\prime}(T_{0})+O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}.$
(3.20)

For $T_{0}<t\leqslant T_{1}$, we still have the same distribution of $\Delta
S(t)$, and by the same argument as in (3.1), if we condition on $T_{1}>T_{0}$,
then

$\displaystyle S(T_{1})-S(T_{0})=O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}.$ (3.21)

By (3.6), this holds also unconditionally.

###  3.2. Phase II: $T_{1}$ to $T_{2}$

Since the entire game is symmetric under cyclic permutations of the three
choices _rock_ , _paper_ , _scissors_ , we may for the next phase assume that
R first uses up all $n$ _rock_ , i.e., that $N_{T_{1},1}=0$. Note, however,
that the game is not symmetric under odd permutations, so having made this
assumption, choices 2 (_paper_) and 3 (_scissors_) play different roles, since
3 beats 2.

By the discussion of the greedy strategy in Section 2.2, for
$t\in[T_{1},T_{2})$, R should play randomly and choose 2 or 3 with
probabilities 1/3 and 2/3. We argue as in the preceding subsection (and
therefore omit some details); we now let
$(\boldsymbol{\eta}_{t})_{1}^{\infty}$ be an i.i.d sequence of random vectors
with $\operatorname{\mathbb{P}{}}(\boldsymbol{\eta}_{t}=\mathbf{e}_{i})=p_{i}$
for $i=1,2,3$, with $(p_{1},p_{2},p_{3})=(0,\frac{1}{3},\frac{2}{3})$, and we
assume as we may that $\mathbf{N}_{t}-\mathbf{N}_{t-1}=\boldsymbol{\eta}_{t}$
for $T_{1}<t\leqslant T_{2}$. Let

$\displaystyle\mathbf{N}^{\prime\prime}_{t}=(N^{\prime\prime}_{t,i})_{i=1}^{3}:=\mathbf{N}_{T_{1}}+\sum_{u=T_{1}+1}^{t}\boldsymbol{\eta}_{u},\qquad
t\geqslant T_{1}.$ (3.22)

Then $\mathbf{N}^{\prime\prime}_{t}=\mathbf{N}_{t}$ for $T_{1}\leqslant
t\leqslant T_{2}$. Let

$\displaystyle M^{\prime\prime}:=\max_{i=1,2,3}\max_{T_{1}\leqslant t\leqslant
3n}\bigl{\lvert}N^{\prime\prime}_{t,i}-N^{\prime\prime}_{T_{1},i}-p_{i}(t-T_{1})\bigr{\rvert}.$
(3.23)

If we again condition on $T_{1}>T_{0}$, we obtain, by conditioning on $T_{1}$
and arguing as in (3.1) and using $3n-T_{1}<3n-T_{0}=O(n^{2/3})$,

$\displaystyle M^{\prime\prime}=O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}.$ (3.24)

By (3.6) again, this holds also unconditionally. We obtain from (3.23) and
(3.24), taking $t=T_{2}$, for every $i$,

$\displaystyle
N_{T_{2},i}=N^{\prime\prime}_{T_{2},i}=N_{T_{1},i}+p_{i}(T_{2}-T_{1})+O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}$
(3.25)

and thus, by (3.18),

$\displaystyle n-N_{T_{2},i}$
$\displaystyle=n-N_{T_{1},i}-p_{i}(T_{2}-T_{1})+O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}$
$\displaystyle=X_{\max}-X_{i}-p_{i}(T_{2}-T_{1})+O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}.$
(3.26)

We have assumed $N_{T_{1},1}=n$, and then $T_{2}$ is the first $t$ such that
$N_{t,2}=n$ or $N_{t,3}=n$. In particular, (3.2) implies

$\displaystyle
0=\min_{i=2,3}\bigl{(}n-N_{T_{2},i}\bigr{)}=\min_{i=2,3}\bigl{(}X_{\max}-X_{i}-p_{i}(T_{2}-T_{1})\bigr{)}+O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}.$
(3.27)

Consequently,

$\displaystyle\min_{i=2,3}\bigl{(}X_{\max}-X_{i}-p_{i}(T_{2}-T_{1})\bigr{)}=O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}.$
(3.28)

It follows that also

$\displaystyle\min_{i=2,3}p_{i}^{-1}\bigl{(}X_{\max}-X_{i}-p_{i}(T_{2}-T_{1})\bigr{)}=O_{L^{p}}\bigl{(}n^{1/3}\bigr{)},$
(3.29)

which can be written

$\displaystyle\min_{i=2,3}p_{i}^{-1}\bigl{(}X_{\max}-X_{i}\bigr{)}-(T_{2}-T_{1})=O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}.$
(3.30)

Thus

$\displaystyle
T_{2}-T_{1}=\min_{i=2,3}\frac{X_{\max}-X_{i}}{p_{i}}+O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}.$
(3.31)

We repeat that this holds assuming that choice 1 is the first to be used up by
R.

In this phase, the gain $\Delta S(t)$ of N has expectation $1/3$ in each round
(and its absolute value is bounded by 1, so all moments are bounded);
moreover, the gains in different rounds are i.i.d. Hence, similarly to (3.1)
again, the central limit theorem with moment convergence together with Doob’s
inequality yields

$\displaystyle
S(T_{2})-S(T_{1})=\tfrac{1}{3}(T_{2}-T_{1})+O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}.$
(3.32)

###  3.3. Phase III: $T_{2}$ to $T_{3}$

This phase is deterministic, and not very fun to play (at least not for R): R
has only one choice, and N wins every round. The total gain for N in this
phase are thus, using (3.16),

$\displaystyle
S(T_{3})-S(T_{2})=T_{3}-T_{2}=T_{3}-T_{1}-(T_{2}-T_{1})=3X_{\max}-(T_{2}-T_{1}).$
(3.33)

###  3.4. Collecting the gains

By (3.21), (3.32), and (3.33), the final score of N is

$\displaystyle
S_{n}=S(T_{3})=S(T_{0})+3X_{\max}-\tfrac{2}{3}(T_{2}-T_{1})+O_{L^{p}}\bigl{(}n^{1/3}\bigr{)},$
(3.34)

where furthermore $T_{2}-T_{1}$ is given by (3.31) when choice 1 (_rock_) is
the first to be used up by R. We develop (3.34) as follows.

######  Lemma 3.1.

We have

$\displaystyle
S_{n}-S(T_{0})=\max\bigl{\\{}X_{1}+2X_{2},X_{2}+2X_{3},X_{3}+2X_{1}\bigr{\\}}+O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}.$
(3.35)

###### Proof.

We may again, by symmetry, suppose that R first uses up 1. Typically, this is
the case when $X_{\max}=X_{1}$, but it is possible that $X_{1}$ is not the
maximum. (Then $N_{t,1}$ is not the largest at $t=T_{0}$, but $N_{t,1}$
overtakes the other two components and hits $n$ first.) In any case,
$N_{T_{2},1}=N_{T_{1},1}=n$, and thus (3.2) yields, recalling $p_{1}=0$,

$\displaystyle
0=n-N_{T_{2},1}=X_{\max}-X_{1}+O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}.$ (3.36)

Hence,

$\displaystyle X_{\max}=X_{1}+O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}.$ (3.37)

We obtain from (3.34), (3.31) and (3.37), recalling $p_{2}=\frac{1}{3}$ and
$p_{3}=\frac{2}{3}$,

$\displaystyle S_{n}-S(T_{0})$
$\displaystyle=3X_{\max}-\tfrac{2}{3}(T_{2}-T_{1})+O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}$
$\displaystyle=3X_{\max}-\min\bigl{\\{}2(X_{\max}-X_{2}),(X_{\max}-X_{3})\bigr{\\}}+O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}$
$\displaystyle=\max\bigl{\\{}X_{\max}+2X_{2},2X_{\max}+X_{3}\bigr{\\}}+O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}$
$\displaystyle=\max\bigl{\\{}X_{1}+2X_{2},2X_{1}+X_{3}\bigr{\\}}+O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}.$
(3.38)

Furthermore, (3.11) implies that $X_{\max}\geqslant 0$ and that, using also
(3.37),

$\displaystyle 2X_{1}+X_{3}$
$\displaystyle=3X_{1}+X_{2}+2X_{3}=3X_{\max}+X_{2}+2X_{3}+O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}$
$\displaystyle\geqslant X_{2}+2X_{3}+O_{L^{p}}\bigl{(}n^{1/3}\bigr{)}.$ (3.39)

Hence (3.4) yields (3.35) in the case when R first uses up choice 1. By
symmetry (3.35) holds in general. ∎

We may now summarize the analysis in the following limit result.

######  Lemma 3.2.

As ${n\to\infty}$,

$\displaystyle
n^{-1/2}S_{n}\overset{\mathrm{d}}{\longrightarrow}S^{*}:=W+\max\bigl{\\{}V_{1}+2V_{2},V_{2}+2V_{3},V_{3}+2V_{1}\bigr{\\}},$
(3.40)

together with all moments, where $W,V_{1},V_{2},V_{3}$ are jointly normal with
$W$ independent of $(V_{1},V_{2},V_{3})$ and

$\displaystyle W$ $\displaystyle\in N(0,2),$ (3.41)
$\displaystyle(V_{1},V_{2},V_{3})$ $\displaystyle\in
N\left(0,\begin{pmatrix}\phantom{-}\frac{2}{3}&-\frac{1}{3}&-\frac{1}{3}\\\\[3.0pt]
-\frac{1}{3}&\phantom{-}\frac{2}{3}&-\frac{1}{3}\\\\[3.0pt]
-\frac{1}{3}&-\frac{1}{3}&\phantom{-}\frac{2}{3}\end{pmatrix}\right).$ (3.42)

###### Proof.

The random vectors $\boldsymbol{\xi}_{t}$ in (3.4) are i.i.d with
$\operatorname{\mathbb{E}}{}\boldsymbol{\xi}_{t}=0$ and covariance matrix
(regarding $\boldsymbol{\xi}_{t}$ as a column vector)

$\displaystyle\operatorname{\mathbb{E}}{}\boldsymbol{\xi}_{t}^{\mathsf{tr}}\boldsymbol{\xi}_{t}=\Sigma:=\begin{pmatrix}\phantom{-}\frac{2}{9}&-\frac{1}{9}&-\frac{1}{9}\\\\[3.0pt]
-\frac{1}{9}&\phantom{-}\frac{2}{9}&-\frac{1}{9}\\\\[3.0pt]
-\frac{1}{9}&-\frac{1}{9}&\phantom{-}\frac{2}{9}\end{pmatrix}.$ (3.43)

Since $T_{0}\sim 3n$ by (3.3), the central limit theorem yields, recalling
(3.10),

$\displaystyle
n^{-1/2}(X_{1},X_{2},X_{3})\overset{\mathrm{d}}{\longrightarrow}(V_{1},V_{2},V_{3})$
$\displaystyle\in N(0,3\Sigma),$ (3.44)

which agrees with (3.42). Similarly, as noted in (3.19),
$n^{-1/2}S(T_{0})\overset{\mathrm{d}}{\longrightarrow}W$. Furthermore, by
(3.20) we may here replace $S(T_{0})$ be the approximation $S^{\prime}(T_{0})$
which, as noted above, is independent of $(X_{1},X_{2},X_{3})$. Hence,

$\displaystyle
n^{-1/2}\bigl{(}S(T_{0}),X_{1},X_{2},X_{3}\bigr{)}\overset{\mathrm{d}}{\longrightarrow}(W,V_{1},V_{2},V_{3}),$
(3.45)

and thus (3.35) and the continuous mapping theorem yield (3.40). Moreover, all
moments converge in the central limit theorems (3.44) and (3.19) [1, Theorem
7.5.1], and it follows (e.g using uniform integrability) that all moments
converge also in (3.45) and (3.40). ∎

In the following section, we give several alternative expressions for the
limit $S^{*}$.

##  4\. The distribution of the limit

###### Proof of Theorem 1.2.

We have proved the convergence (1.2) in Lemma 3.2, and it remains only to show
that the limit in (3.40) can be expressed as in (i)–(iv). (Note that only the
distribution matters.)

(i): Define $Z_{1}:=V_{1}+2V_{2}$, $Z_{2}:=V_{2}+2V_{3}$, and
$Z_{3}=V_{3}+2V_{1}$. Then Lemma 3.2 shows that (1.2) holds, and a simple
calculation shows that $(Z_{1},Z_{2},Z_{3})$ has the distribution (1.5).

(iii): Define $Z^{\prime\prime}_{i}:=W+Z_{i}$, $i=1,2,3$. Then (1.3) yields
(1.7), and (1.4)–(1.5) yield (1.8).

(ii): We may write $W=W^{\prime}+\widetilde{W}$, where
$W^{\prime},\widetilde{W}\in N(0,1)$, and $W^{\prime}$ and $\widetilde{W}$ are
independent of each other and of $(Z_{1},Z_{2},Z_{3})$. Define
$Z^{\prime}_{i}:=(\widetilde{W}+Z_{i})/\sqrt{3}$, $i=1,2,3$. Then (1.3) yields
(1.6), and it follows from (1.5) that the covariance matrix of
$(Z^{\prime}_{1},Z^{\prime}_{2},Z^{\prime}_{3})$ is the identity matrix; thus
the jointly normal variables
$W^{\prime},Z_{1}^{\prime},Z_{2}^{\prime},Z_{3}^{\prime}$ are independent
$N(0,1)$.

(iv): It follows from (1.5) that $Z_{1}+Z_{2}+Z_{3}$ has variance 0, and thus
$Z_{1}+Z_{2}+Z_{3}=0$ a.s, so $(Z_{1},Z_{2},Z_{3})$ has really a 2-dimensional
normal distribution. In fact, if $\boldsymbol{\zeta}=(\zeta_{1},\zeta_{2})$ is
a centered normal distribution in $\mathbb{R}^{2}$ with
$\operatorname{Var}\zeta_{1}=\operatorname{Var}\zeta_{2}=2$ and
$\operatorname{Cov}(\zeta_{1},\zeta_{2})=0$, then we can construct
$(Z_{1},Z_{2},Z_{3})$ with the desired distribution (1.5) by

$\displaystyle Z_{i}:=\mathbf{f}_{i}\cdot\boldsymbol{\zeta},$ (4.1)

where $\mathbf{f}_{1}:=(1,0)$,
$\mathbf{f}_{2}:=(-\frac{1}{2},\frac{\sqrt{3}}{2})$,
$\mathbf{f}_{3}:=(-\frac{1}{2},-\frac{\sqrt{3}}{2})$. We define
$R:=|\boldsymbol{\zeta}|$ and
$\theta:=\arg(\zeta_{1}+\mathrm{i}\zeta_{2})\in[-\pi,\pi)$; thus

$\displaystyle\boldsymbol{\zeta}=(R\cos\theta,R\sin\theta),$ (4.2)

and it follows from (4.1) by simple calculations (which are made even simpler
by identifying $\mathbb{R}^{2}$ and $\mathbb{C}$ and regarding
$\boldsymbol{\zeta}$ as a complex random variable) that

$\displaystyle Z_{1}=R\cos\theta,\qquad Z_{2}=R\cos(\theta-2\pi/3),\qquad
Z_{3}=R\cos(\theta+2\pi/3).$ (4.3)

The normal distribution of $\boldsymbol{\zeta}$ is rotationally symmetric, and
thus, as is well-known, $R$ and $\theta$ are independent, with $\theta$
uniformly distributed on $[-\pi,\pi)$; furthermore, $R$ has the Rayleigh
distribution stated in the theorem. To find the distribution of
$Z_{\max}:=\max(Z_{1},Z_{2},Z_{3})$, we may by symmetry condition on
$Z_{\max}=Z_{1}$, which by (4.3) is equivalent to $\theta\in[-\pi/3,\pi/3]$,
and since $\cos\theta$ is an even function, we may further restrict to
$\theta\in[0,\pi/3]$. Then $Z_{\max}=Z_{1}=R\cos\theta$, and thus (1.9)
follows from (1.3) ∎

###### Proof of Theorem 1.1.

By the moment convergence in Theorem 1.2, it suffices to find
$\operatorname{\mathbb{E}}{}S^{*}$. For this we use (1.9). We have
$\operatorname{\mathbb{E}}{}W=0$, and by simple calculations

$\displaystyle\operatorname{\mathbb{E}}{}R$
$\displaystyle=\int_{0}^{\infty}\tfrac{1}{2}r^{2}e^{-r^{2}/4}\,\mathrm{d}r=\sqrt{\pi},$
(4.4) $\displaystyle\operatorname{\mathbb{E}}{}\cos\theta$
$\displaystyle=\frac{3}{\pi}\int_{0}^{\pi/3}\cos\vartheta\,\mathrm{d}\vartheta=\frac{3\sqrt{3}}{2\pi}.$
(4.5)

Hence, by the independence,

$\displaystyle\operatorname{\mathbb{E}}{}S^{*}=\operatorname{\mathbb{E}}{}R\cdot\operatorname{\mathbb{E}}{}\cos\theta=\frac{3\sqrt{3}}{2\sqrt{\pi}}\doteq
1.4658075.$ (4.6)

∎

Higher moments of $S^{*}$ can be computed in the same way. For example, we
have

$\displaystyle\operatorname{\mathbb{E}}{}(S^{*})^{2}=\operatorname{\mathbb{E}}{}W^{2}+\operatorname{\mathbb{E}}{}R^{2}\operatorname{\mathbb{E}}{}\cos^{2}\theta=2+4\Bigl{(}\frac{1}{2}+\frac{3\sqrt{3}}{8\pi}\Bigr{)}=4+\frac{3\sqrt{3}}{2\pi}\doteq
4.82699$ (4.7)

and hence

$\displaystyle\operatorname{Var}S^{*}=4+\frac{6\sqrt{3}-27}{4\pi}\doteq
2.67840.$ (4.8)

Hence, we have

$\displaystyle\operatorname{Var}S_{n}\sim\Bigl{(}4+\frac{6\sqrt{3}-27}{4\pi}\Bigr{)}n.$
(4.9)

##  5\. The probability of winning

Finally, we note that we may also calculate the asymptotic probability that R
wins the game, in spite of her restriction, i.e, that the final score
$S_{n}<0$. (Recall that $S_{n}$ is the score for N.)

######  Theorem 5.1.

If both players use the strategies above, then the probability that R wins has
as ${n\to\infty}$ the limit

$\displaystyle\operatorname{\mathbb{P}{}}(S_{n}<0)\to\frac{3\arccos(1/4)-\pi}{4\pi}=\frac{\arccos(11/16)}{4\pi}\doteq
0.064677.$ (5.1)

###### Proof.

By Theorem 1.2, we have
$\operatorname{\mathbb{P}{}}(S_{n}<0)\to\operatorname{\mathbb{P}{}}(S^{*}<0)$
(since $S^{*}$ has a continuous distribution, e.g by (1.3)). We compute this
probability using Theorem 1.2(iii). By (1.7), we have

$\displaystyle S^{*}<0\iff Z^{\prime\prime}_{i}<0\ \forall i.$ (5.2)

We may, similarly to (4.1), construct $Z^{\prime\prime}_{i}$ as

$\displaystyle
Z^{\prime\prime}_{i}:=\widehat{\mathbf{f}}_{i}\cdot\widehat{\boldsymbol{\zeta}},$
(5.3)

where $\widehat{\boldsymbol{\zeta}}$ is a standard normal distribution in
$\mathbb{R}^{3}$, and
$\widehat{\mathbf{f}}_{1},\widehat{\mathbf{f}}_{2},\widehat{\mathbf{f}}_{3}$
are three vectors in $\mathbb{R}^{3}$ such that

$\displaystyle\widehat{\mathbf{f}}_{i}\cdot\widehat{\mathbf{f}}_{j}=\begin{cases}4,&i=j,\\\
-1,&i\neq j.\end{cases}$ (5.4)

By (5.3), the condition (5.2) means that $\widehat{\boldsymbol{\zeta}}$ lies
in the intersection of three open half-spaces $H_{1}$, $H_{2}$, $H_{3}$, which
are bounded by hyperplanes orthogonal to $\widehat{\mathbf{f}}_{1}$,
$\widehat{\mathbf{f}}_{2}$ and $\widehat{\mathbf{f}}_{3}$. The angle between
any two of these vectors is, by (5.4), $\alpha:=\arccos(-1/4)$. Hence, the
interior angle between any of the two hyperplanes is
$\beta:=\pi-\alpha=\arccos(1/4)$, and thus the intersection of the unit sphere
and $H_{1}\cap H_{2}\cap H_{3}$ is a spherical triangle $\Delta$ with all
three angles $\beta$. Consequently, the area $|\Delta|$ of $\Delta$ is
$3\beta-\pi$. The distribution of $\widehat{\boldsymbol{\zeta}}$ is
rotationally symmetric, and thus we may project $\widehat{\boldsymbol{\zeta}}$
onto the unit sphere, and find, recalling that the area of the sphere is
$4\pi$,

$\displaystyle\operatorname{\mathbb{P}{}}(Z_{\max}<0)$
$\displaystyle=\operatorname{\mathbb{P}{}}\bigl{(}\widehat{\boldsymbol{\zeta}}\in
H_{1}\cap H_{2}\cap
H_{3}\bigr{)}=\frac{|\Delta|}{4\pi}=\frac{3\beta-\pi}{4\pi}=\frac{3\arccos(1/4)-\pi}{4\pi}.$
(5.5)

Finally, note that

$\displaystyle\cos(3\beta-\pi)=-4\cos^{3}\beta+3\cos\beta=-4\bigl{(}\tfrac{1}{4}\bigr{)}^{3}+3\cdot\tfrac{1}{4}=\tfrac{11}{16}.$
(5.6)

∎

Theorem 5.1 assumes that the players use their optimal strategies for
maximizing the expected gain; if they instead want to maximize the probability
of winning (but do not care about how much they win or lose), the optimal
strategies are presumably different (see Example 5.3), and most likely much
more complex; hence we do not know whether (5.1) holds or not in that case.

######  Problem 5.2.

Suppose that both players want to maxime
$\operatorname{\mathbb{P}{}}(\text{win})-\operatorname{\mathbb{P}{}}({\text{lose}})$.
What is (asymptotically) the probability that R wins?

It is possible that the asymptotic answer is the same as in Theorem 5.1,
although the probabilities for finite $n$ presumably are different. (See
Example 5.3 for $n=1$.) It might seem likely that a strategy that gives one of
the players a significantly lower expected score will also give a lower
probability that this score is positive. However, Example 5.4 shows that
strategies with the same expectation still might give different distributions
of the score and therefore different probabilities of winning, so it seems
that there is no simple solution to Problem 5.2.

######  Example 5.3.

Here is simple example showing that, at least for $n=1$, the greedy strategy
is not the optimal strategy if the objective is to win, as in Problem 5.2. Let
$n=1$, and suppose that both players choose _scissors_ in the first round.
This is a draw, and then R cannot win the game, since she knows that N will
win the last round. In the second round R plays the game in Figure 1; if she
wants to minimize the probability of losing the entire game, the best strategy
is to play _rock_ or _paper_ with equal probabilities, and not with the
probabilities in Section 2.2 that minimize the expected loss.  $\triangle$

######  Example 5.4.

Suppose that R uses the greedy strategy above, but that N plays only _rock_
until $T_{1}$, and then follows the strategy above. (This is obviously a risky
strategy if R would guess it, but we assume that R is a mathematician and
knows that the greedy strategy is proven to be optimal, and therefore sticks
to it.) Since R plays randomly even when N does not, this leads to exactly the
same behaviour of $\mathbf{N}(t)$ as in the analysis above, and also $S(t)$
behaves as above for $t\leqslant T_{1}$, but now $\mathbf{N}(t)$ and $S(t)$
are dependent. In fact, for $t\leqslant T_{1}$, we have
$S(t)=N_{t,3}-N_{t,2}$. In particular, $S(T_{0})=X_{3}-X_{2}$, and the
argument in the proof of Lemma 3.2 yields

$\displaystyle
n^{-1/2}S_{n}\overset{\mathrm{d}}{\longrightarrow}S^{\prime}:=V_{3}-V_{2}+\max\bigl{\\{}V_{1}+2V_{2},V_{2}+2V_{3},V_{3}+2V_{1}\bigr{\\}}.$
(5.7)

We have $V_{3}-V_{2}=-(V_{1}+2V_{2})$, and thus $S^{\prime}\geqslant 0$ with a
point mass (by symmetry in the maximum in (5.7))
$\operatorname{\mathbb{P}{}}(S^{\prime}=0)=1/3$. In this case, we cannot
immediately find the limit of $\operatorname{\mathbb{P}{}}(S_{n}<0)$, but if
the strategy is perturbed a little, and N plays normally for the first
$\varepsilon_{n}n$ rounds with $\varepsilon_{n}\to 0$ very slowly, it can be
seen that
$\operatorname{\mathbb{P}{}}(S_{n}<0)\to\frac{\operatorname{\mathbb{P}{}}}{(}S^{\prime}=0)=1/6$.

In this case, the new strategy for N is worse for him; it gives the same
expected score but a lower probability that the score is positive (given that
R plays greedily). However, it suggests that there also might be other
strategies that instead increase the probability that N wins.  $\triangle$

## References

  * [1] Allan Gut. _Probability: A Graduate Course_ , 2nd ed. Springer, New York, 2013.
  * Janson, Łuczak and Ruciński [2000] Svante Janson, Tomasz Łuczak & Andrzej Ruciński. _Random Graphs_. Wiley, New York, 2000.
  * [3] Anna R. Karlin & Yuval Peres. _Game Theory, Alive_. American Mathematical Society, Providence, RI, 2017.
  * von Neumann [1928] J. von Neumann. Zur Theorie der Gesellschaftsspiele. _Math. Ann._ 100 (1928), no. 1, 295–320.
  * Spiro, Surya and Zeng [2023] Sam Spiro, Erlang Surya & Ji Zeng. Semi-restricted rock, paper, scissors. _Electron. J. Combin._ 30 (2023), no. 4, Paper No. 4.32, 29 pp.


Current affiliation: ]Nippon Steel Corporation, Steel Research Laboratories.

# Reduced magnetocrystalline anisotropy of $\text{CoFe}_{2}\text{O}_{4}$ thin
films studied by angle-dependent x-ray magnetic circular dichroism

Yosuke Nonaka  [  Department of Physics, the University of Tokyo, Bunkyo-ku,
Tokyo 113-0033, Japan  Yuki K. Wakabayashi  Department of Electrical
Engineering and Information Systems, The University of Tokyo, 7-3-1 Hongo,
Bunkyo-ku, Tokyo 113-8656, Japan  Goro Shibata  Department of Physics, the
University of Tokyo, Bunkyo-ku, Tokyo 113-0033, Japan  Department of Applied
Physics, Tokyo University of Science, Katsushika-ku, Tokyo 125-8585, Japan
Shoya Sakamoto  Department of Physics, the University of Tokyo, Bunkyo-ku,
Tokyo 113-0033, Japan  Keisuke Ikeda  Department of Physics, the University of
Tokyo, Bunkyo-ku, Tokyo 113-0033, Japan  Zhendong Chi  Department of Physics,
the University of Tokyo, Bunkyo-ku, Tokyo 113-0033, Japan  Yuxuan Wan
Department of Physics, the University of Tokyo, Bunkyo-ku, Tokyo 113-0033,
Japan  Masahiro Suzuki  Department of Physics, the University of Tokyo,
Bunkyo-ku, Tokyo 113-0033, Japan  Tsuneharu Koide  Photon Factory, Institute
of Materials Structure Science,High Energy Accelerator Research Organization
(KEK), Tsukuba, Ibaraki 305-0801, Japan  Masaaki Tanaka  Department of
Electrical Engineering and Information Systems, The University of Tokyo, 7-3-1
Hongo, Bunkyo-ku, Tokyo 113-8656, Japan  Center for Spintronics Research
Network, Graduate School of Engineering, The University of Tokyo, 7-3-1 Hongo,
Bunkyo-ku, Tokyo 113-8656, Japan  Ryosho Nakane  Department of Electrical
Engineering and Information Systems, The University of Tokyo, 7-3-1 Hongo,
Bunkyo-ku, Tokyo 113-8656, Japan  Institute for Innovation in International
Engineering Education, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo
113-8656, Japan  Atsushi Fujimori  Department of Physics, the University of
Tokyo, Bunkyo-ku, Tokyo 113-0033, Japan  Department of Applied Physics, Waseda
University, Shinjuku-ku, Tokyo 169-8555, Japan

###### Abstract

Spinel-type $\text{CoFe}_{2}\text{O}_{4}$ is a ferrimagnetic insulator with
the Néel temperature exceeding 790 K, and shows a strong cubic
magnetocrystalline anisotropy (MCA) in bulk materials. However, when a
$\text{CoFe}_{2}\text{O}_{4}$ film is grown on other materials, its magnetic
properties are degraded so that so-called magnetically dead layers are
expected to be formed in the interfacial region. We investigate how the
magnetic anisotropy of $\text{CoFe}_{2}\text{O}_{4}$ is modified at the
interface of $\text{CoFe}_{2}\text{O}_{4}$/$\text{Al}_{2}\text{O}_{3}$
bilayers grown on Si(111) using x-ray magnetic circular dichroism (XMCD). We
find that the thinner $\text{CoFe}_{2}\text{O}_{4}$ films have significantly
smaller MCA values than bulk materials. The reduction of MCA is explained by
the reduced number of Co2+ ions at the $O_{h}$ site reported by a previous
study [Y. K. Wakabayashi et al., Phys. Rev. B 96, 104410 (2017)].

###### pacs:

71.30.+h, 75.25.Dk, 75.47.Lx, 78.70.Dm, 78.20.Ls

##  I Introduction

Spinel-type cobalt ferrite $\text{CoFe}_{2}\text{O}_{4}$ is a classical
ferrimagnetic insulator having the Néel temperature exceeding 790 K and
exhibits a strong cubic magnetocrystalline anisotropy (MCA). The cubic MCA has
been successfully explained by the single-ion anisotropy of the Co2+ ions at
the inequivalent $O_{h}$ sites Tachiki (1960). Recently, heterostructures
incorporating thin $\text{CoFe}_{2}\text{O}_{4}$ layers have attracted much
attention as spintronics devices Moussy (2013) because of the spin-dependent
band gap Szotek  _et al._ (2006) and high Néel temperature of
$\text{CoFe}_{2}\text{O}_{4}$ Sawatzky  _et al._ (1968). For example, a
$\text{CoFe}_{2}\text{O}_{4}$-based tunnel barrier acts as a spin filter
because electrons have spin-dependent tunneling probabilities Chapline and
Wang (2006). However, the experimentally obtained spin-filtering efficiency of
$\text{CoFe}_{2}\text{O}_{4}$-based tunnel barriers still remains lower than
theoretical values of $100\%$ Matzen  _et al._ (2012); Moodera  _et al._
(2007); Moussy (2013). As a possible cause of the low spin-filtering
efficiency, it has been proposed that structural and/or chemical disorder lead
to the formation of impurity states in the spin-dependent gap Ramos  _et al._
(2007); Takahashi  _et al._ (2010); Szotek  _et al._ (2006). In order to
improve the spin filtering efficiency, it is thus essential to understand the
electronic and magnetic phenomena at the interfaces.

In a recent work Wakabayashi  _et al._ (2017), the magnetic properties of
$\text{CoFe}_{2}\text{O}_{4}$(111)/$\text{Al}_{2}\text{O}_{3}$(111)/Si(111)
structures Bachelet  _et al._ (2014) were studied using the element-specific
probe of x-ray absorption spectroscopy (XAS) and x-ray magnetic circular
dichroism (XMCD). A schematic illustration of the stacking structure of the
sample and an example of cross sectional transmission-electron-microscope
(TEM) image are shown in Fig. 1(a).

Figure 1:  Schematic pictures of the sample structure and the method to probe
the interfacial region. (a) Sample structure and cross sectional TEM image of
the
$\text{CoFe}_{2}\text{O}_{4}$(111)/$\text{Al}_{2}\text{O}_{3}$(111)/Si(111)
sample with d = 11 nm, adapted from Ref. Wakabayashi  _et al._ (2017). (b)
Illustration of probing the magnetically dead interfacial layers with XMCD by
changing the thickness of the $\text{CoFe}_{2}\text{O}_{4}$ layer.

Because they employed the total-electron yield (TEY) method to detect the
absorption signals, several nanometers from the surface were preferentially
probed Thole  _et al._ (1985); Frazer  _et al._ (2003). By reducing the film
thicknesses below the probing depth of a few nm, they obtained XAS and XMCD
spectra reflecting the magnetically dead layers of $\sim$1.4 nm thickness and
the cation redistribution near the
$\text{CoFe}_{2}\text{O}_{4}$/$\text{Al}_{2}\text{O}_{3}$ interface as
illustrated in Fig. 1(b). They revealed a high density of Co-Fe antisite
defects and the reduced concentration of the Co2+ ions at the $O_{h}$ sites in
the magnetically dead layers. Since the Co2+ ions at the $O_{h}$ sites are
considered to be the origin of the cubic MCA in bulk
$\text{CoFe}_{2}\text{O}_{4}$, a significant modification of the magnetic
anisotropy is expected in the interfacial region. In the present study, we
investigate how the strong cubic MCA of $\text{CoFe}_{2}\text{O}_{4}$ is
affected at the $\text{CoFe}_{2}\text{O}_{4}$/$\text{Al}_{2}\text{O}_{3}$
interface by angle-dependent XMCD that is a powerful local probe to study
element-specific magnetic anisotropies Shibata  _et al._ (2018); Sakamoto  _et
al._ (2021). The surface-sensitive XMCD method allowed us to investigate the
magnetic properties in the interfacial region with high sensitivity.
Furthermore, by using XMCD, diamagnetic signals from substrates, which prevent
us from the observation of intrinsic paramagnetic signals, can be
automatically excluded.

##  II Experimental Methods

Epitaxial $\text{CoFe}_{2}\text{O}_{4}$(111) thin films with the thicknesses
of d = 1.4, 2.3, 4, and 11 nm were grown on a 2.4 nm-thick
$\gamma$-$\text{Al}_{2}\text{O}_{3}$(111) buffer layer/$n^{+}$-Si(111)
substrate using the pulsed laser deposition method. In order to avoid charging
of the samples during the XAS and XMCD measurements, we used heavily
phosphorus-doped Si(111) wafers with low resistivities of 2 m$\Omega$ cm. For
the epitaxial growth of the $\gamma$-$\text{Al}_{2}\text{O}_{3}$ buffer layers
on the Si substrates, we used solid-phase reaction of Al and $\text{SiO}_{2}$.
A more detailed description of the sample preparation and characterization is
given in Ref. Wakabayashi  _et al._ (2017).

Magnetic field-angle-dependent XAS and XMCD measurements were performed at
room temperature using a superconducting vector-magnet XMCD apparatus Furuse
_et al._ (2013) installed at the undulator beamline BL-16A of Photon Factory,
High Energy Accelerator Research Organization (KEK-PF). The magnetization $M$
of $\text{CoFe}_{2}\text{O}_{4}$ thin film is rotated by the magnetic field
$H$. The magnetic field angle $\theta_{H}$ and magnetization angle
$\theta_{M}$ are defined relative to the surface normal. The absorption
signals were detected in the TEY mode. The XMCD signals were collected by
switching the photon helicity at the rate of 10 Hz Amemiya  _et al._ (2013).
In order to eliminate the saturation effect Nakajima  _et al._ (1999), which
induces an extrinsic angle dependence to spectral line shapes, we fixed the
x-ray incident angle at $45^{\circ}$ and applied magnetic fields of 0.7 T to
the sample along various directions. The geometry of the present measurements
is illustrated in Fig. 2(a).

##  III Results and Discussion

Figure 2:  Angle-dependent XMCD of the 11 nm-thick
$\text{CoFe}_{2}\text{O}_{4}$ film at the Fe $L_{2,3}$ edges. (a) Experimental
geometry of the angle-dependent XMCD measurements. (b) XAS spectrum at
$\theta_{H}=0^{\circ}$ after background subtraction. (c) XMCD spectra for
various $\theta_{H}$’s. (d) $\theta_{H}$ dependence of the relative magnetic
moment projected onto the x-ray incident vector, deduced from the L3-edge XMCD
intensity (see main text). The gray dashed curve (sine curve) shows the
simulation for the magnetically isotropic situation.

Figures 2(b) and (c) show the Fe $L_{2,3}$-edge XAS ($\mu^{+}+\mu^{-}$) and
XMCD ($\mu^{+}-\mu^{-}$) spectra of the 11 nm-thick
$\text{CoFe}_{2}\text{O}_{4}$ thin film. Here, $\mu^{+}$ ($\mu^{-}$) denotes
the absorption coefficient for photons with positive (negative) helicity.
Since the spectral line shape of XAS did not show any appreciable $\theta_{H}$
dependence, only the spectrum at $\theta_{H}=0^{\circ}$ is shown. Figure 2(c)
shows that the XMCD spectrum systematically changes with $\theta_{H}$. Since
the XMCD intensity is proportional to the magnetic moment projected onto the
x-ray incident direction ($M_{\text{proj}}=M\cos(\theta_{M}-45^{\circ})$), its
$\theta_{H}$ dependence reflects the change of the magnetization direction
$\theta_{M}$ under varying $\theta_{H}$. Since the total magnetic moment and
the Fe $L_{2,3}$-edge XMCD spectra of these samples were already obtained in
Ref.Wakabayashi  _et al._ (2017), we deduced $M_{\text{proj}}$ from the
intensity of Fe $L_{2,3}$-edge XMCD under the assumption that the Fe
$L_{2,3}$-edge XMCD intensity is proportional to M. Figure 2(d) shows the
$\theta_{H}$ dependence of $M_{\text{proj}}$. If this film has neither
magnetocrystalline nor magnetic shape anisotropy, M would be fully aligned to
the magnetic field direction ($\theta_{H}=\theta_{M}$) and thus
$M_{\text{proj}}$ should be $M\cos(\theta_{H}-45^{\circ})$, as shown by a gray
dashed curve. The deviation of the experimental data from
$M\cos(\theta_{H}-45^{\circ})$ thus shows the magnetic anisotropy.

In order to analyze the obtained $\theta_{H}$ dependence of $M_{\text{proj}}$,
we use the Stoner-Wohlfarth model Stoner and Wohlfarth (1948); Shibata  _et
al._ (2018). According to the model, the magnetic energy density E of a thin
film is given by:

$E=-\mu_{0}MH\cos\left(\theta_{M}-\theta_{H}\right)+\frac{\mu_{0}}{2}M^{2}\cos^{2}\theta_{M}+E_{\text{MCA}},$
(1)

where $\mu_{0}$ is the vacuum permeability, and the other variables are
defined in Fig. 2(a). In Eq. (1), E is the sum of the Zeeman energy
[$-\mu_{0}MH\cos\left(\theta_{M}-\theta_{H}\right)$] and the magnetic
anisotropy energy. The magnetic anisotropy energy consists of two
contributions. One is the shape anisotropy (SA) energy, which originates from
the demagnetizing field of the film, and the other is the MCA energy, which
has a microscopic origin. Note that the Stoner-Wohlfarth model is applicable
not only to the ferromagnetic state but also to the paramagnetic state if the
actual magnetization M lower than the saturation magnetization is used. Since
$\text{CoFe}_{2}\text{O}_{4}$ has a cubic MCA in addition to the uniaxial
anisotropy and has crystal domains in the present samples Wakabayashi  _et
al._ (2017), the MCA term cannot be fully calculated with the Stoner-Wohlfarth
model. Therefore, we have calculated $M_{\text{proj}}$ by incorporating only
the Zeeman and SA terms in Eq. (1), and attribute the difference between the
measured $M_{\text{proj}}$ and the calculated $M_{\text{proj}}$ to the MCA.

Figure 3:  Experimental and calculated $M_{\text{proj}}$
[$M_{\text{proj}}$defined in Fig. 2(a)] of $\text{CoFe}_{2}\text{O}_{4}$ thin
films. (a)–(d) Experimentally obtained $M_{\text{proj}}$ and calculations for
the films with various thicknesses (d = 11, 4, 2.3, and 1.4 nm). (e)
Difference curves between the experimental and calculated $M_{\text{proj}}$ in
panels (a)–(d). Note that the vertical scale gradually changes from (a) to
(d). (f) Comparison between the difference curves shown in panels (a)–(d) and
the inversion parameter y reported in Ref. Wakabayashi  _et al._ (2017). Note
that the difference curves [shaded area in panel (e)] have been divided by the
magnetization $M$ in each film.

Figures 3(a)–(d) show the measured $M_{\text{proj}}$ and calculated
$M_{\text{proj}}$ for all the samples, where colored dots are measurements and
black solid lines are calculations. One can see a clear difference between the
calculation and the experiment for the 11 nm-thick film as shown in Fig. 3(a),
and the difference decreases with decreasing thickness as shown in Figs. 3(b),
3(c), and 3(d). (The 11 nm-thick film is ferrimagnetic with hysteresis but the
magnetic field of 0.7 T was not sufficient to saturate the magnetization. The
ferrimagnetic behavior is gradually lost with decreasing thickness and the 1.4
nm-thick film is almost paramagnetic Wakabayashi  _et al._ (2017).) For each
film thickness, the difference is the largest at around
$\theta_{H}=0^{\circ}$, as shown in Fig. 3(e),which is consistent with the
cubic magnetic anisotropy of bulk $\text{CoFe}_{2}\text{O}_{4}$ with the [111]
hard axis Shenker (1957). Considering that the relative contribution of the
interface to the XAS and XMCD increases with decreasing film thickness, the
present observation indicates a weakening of the MCA of
$\text{CoFe}_{2}\text{O}_{4}$ in the interfacial region including the
magnetically dead layer. The magnitude of the difference between the
calculation and measurement is plotted as a function of thickness d in Fig.
3(f).

Now we discuss the microscopic origin of the reduction of the MCA near the
interface. The MCA of bulk $\text{CoFe}_{2}\text{O}_{4}$ is well explained by
a single-ion model Tachiki (1960), according to which the exceptionally high
single-ion anisotropy of Co${}^{2+}(O_{h})$ governs the MCA. Therefore, the
distribution of Co${}^{2+}(O_{h})$ is expected have dominant effects on the
MCA. Such modifications of magnetic anisotropy induced by the distribution of
Co${}^{2+}(O_{h})$ are also reported for the $\text{CoFe}_{2}\text{O}_{4}$
nanoparticles Daffé  _et al._ (2018); Moya  _et al._ (2021); Fantauzzi  _et
al._ (2019). The cation distribution in these films has been studied using
XMCD by Wakabayashi et al. Wakabayashi  _et al._ (2017), according to which
the inversion parameter y, defined by the chemical formula
$[\text{Co}_{1-y}\text{Fe}_{y}]_{T_{d}}[\text{Fe}_{2-y}\text{Co}_{y}]_{O_{h}}\\!\text{O}_{4}$,
suddenly decreases in the $d\leq 2.3$ nm region as shown in Fig. 3(f). From
the comparison of y and the normalized MCA (the magnitude of MCA divided by M)
as functions of the film thickness d plotted in Fig. 3(f), one can see that
the reduction of MCA with decreasing film thickness qualitatively follows the
reduction of y. This result implies that the reduction of the normalized MCA
originates from the reduction of the Co${}^{2+}(O_{h})$ ions in the
interfacial region.

##  IV Conclusion

We have investigated the magnetic anisotropy of the epitaxial
$\text{CoFe}_{2}\text{O}_{4}$(111) thin films (thicknesses $d=$11, 4, 2.3, and
1.4 nm) grown on the 2.4 nm-thick $\gamma$-$\text{Al}_{2}\text{O}_{3}$(111)
buffer layer/$n^{+}$-Si(111) substrates using magnetic field-angle-dependent
XAS and XMCD. We have found that the MCA of the interfacial region including
the magnetically dead layer is reduced. We attribute the reduction of MCA to
the thickness-dependent cation redistribution in the interfacial region, that
is, to the abrupt reduction of the Co${}^{2+}(O_{h})$-ion concentration in the
interfacial region.

###### Acknowledgements.

We would like to thank Kenta Amemiya and Masako Suzuki-Sakamaki for technical
support at KEK-PF BL-16. This work was supported by a Grant-in-Aid for
Scientific Research from JSPS (15H02109, 26289086, 15K17696, and 19K03741).
The experiment was done under the approval of the Photon Factory Program
Advisory Committee (Proposal No. 2016S2-005). Y. K. W. and Z. C. acknowledges
financial support from Materials Education Program for the Futures leaders in
Research, Industry and Technology (MERIT). S. S. and Y. W. acknowledges
financial support from Advanced Leading Graduate Course for Photon Science
(ALPS). Y. K. W. and S. S. also acknowledge support from the JSPS Research
Fellowship Program for Young Scientists. A.F. is an adjunct member of Center
for Spintronics Research Network (CSRN), the University of Tokyo, under
Spintronics Research Network of Japan (Spin-RNJ).

## Data Availability Statement

The data that support the findings of this study are available from the
corresponding author upon reasonable request.

## References

  * Tachiki (1960) M. Tachiki, Prog. Theor. Phys. 23, 1055 (1960).
  * Moussy (2013) J.-B. Moussy, J. Phys. D. Appl. Phys. 46, 143001 (2013).
  * Szotek  _et al._ (2006) Z. Szotek, W. M. Temmerman, D. Ködderitzsch, A. Svane, L. Petit, and H. Winter, Phys. Rev. B 74, 174431 (2006), arXiv:0608168 [cond-mat] .
  * Sawatzky  _et al._ (1968) G. A. Sawatzky, F. van der Woude, and A. H. Morrish, J. Appl. Phys. 39, 1204 (1968).
  * Chapline and Wang (2006) M. G. Chapline and S. X. Wang, Phys. Rev. B 74, 14418 (2006).
  * Matzen  _et al._ (2012) S. Matzen, J. B. Moussy, R. Mattana, K. Bouzehouane, C. Deranlot, and F. Petroff, Appl. Phys. Lett. 101, 042409 (2012).
  * Moodera  _et al._ (2007) J. S. Moodera, T. S. Santos, and T. Nagahama, J. Phys.: Condens. Matter 19, 165202 (2007).
  * Ramos  _et al._ (2007) A. V. Ramos, M.-J. Guittet, J.-B. Moussy, R. Mattana, C. Deranlot, F. Petroff, and C. Gatel, Appl. Phys. Lett. 91, 122107 (2007), arXiv:0707.3823 .
  * Takahashi  _et al._ (2010) Y. K. Takahashi, S. Kasai, T. Furubayashi, S. Mitani, K. Inomata, and K. Hono, Appl. Phys. Lett. 96, 072512 (2010).
  * Wakabayashi  _et al._ (2017) Y. K. Wakabayashi, Y. Nonaka, Y. Takeda, S. Sakamoto, K. Ikeda, Z. Chi, G. Shibata, A. Tanaka, Y. Saitoh, H. Yamagami, M. Tanaka, A. Fujimori, and R. Nakane, Phys. Rev. B 96, 104410 (2017).
  * Bachelet  _et al._ (2014) R. Bachelet, P. de Coux, B. Warot-Fonrose, V. Skumryev, G. Niu, B. Vilquin, G. Saint-Girons, and F. Sánchez, CrystEngComm 16, 10741 (2014).
  * Thole  _et al._ (1985) B. T. Thole, G. van der Laan, J. C. Fuggle, G. A. Sawatzky, R. C. Karnatak, and J.-M. Esteva, Phys. Rev. B 32, 5107 (1985).
  * Frazer  _et al._ (2003) B. H. Frazer, B. Gilbert, B. R. Sonderegger, and G. De Stasio, Surf. Sci. 537, 161 (2003).
  * Shibata  _et al._ (2018) G. Shibata, M. Kitamura, M. Minohara, K. Yoshimatsu, T. Kadono, K. Ishigami, T. Harano, Y. Takahashi, S. Sakamoto, Y. Nonaka, K. Ikeda, Z. Chi, M. Furuse, S. Fuchino, M. Okano, J.-i. Fujihira, A. Uchida, K. Watanabe, H. Fujihira, S. Fujihira, A. Tanaka, H. Kumigashira, T. Koide, and A. Fujimori, npj Quantum Mater. 3, 3 (2018), arXiv:arXiv:1706.05183v3 .
  * Sakamoto  _et al._ (2021) S. Sakamoto, G. Zhao, G. Shibata, Z. Deng, K. Zhao, X. Wang, Y. Nonaka, K. Ikeda, Z. Chi, Y. Wan, M. Suzuki, T. Koide, A. Tanaka, S. Maekawa, Y. J. Uemura, C. Jin, and A. Fujimori, ACS Appl. Electron. Mater. 3, 789 (2021).
  * Furuse  _et al._ (2013) M. Furuse, M. Okano, S. Fuchino, A. Uchida, J. Fujihira, and S. Fujihira, IEEE Trans. Appl. Supercond. 23, 4100704 (2013).
  * Amemiya  _et al._ (2013) K. Amemiya, M. Sakamaki, T. Koide, K. Ito, K. Tsuchiya, K. Harada, T. Aoto, T. Shioya, T. Obina, S. Yamamoto, and Y. Kobayashi, J. Phys. Conf. Ser. 425, 152015 (2013).
  * Nakajima  _et al._ (1999) R. Nakajima, J. Stöhr, and Y. Idzerda, Phys. Rev. B 59, 6421 (1999).
  * Stoner and Wohlfarth (1948) E. C. Stoner and E. P. Wohlfarth, Philos. Trans. R. Soc. London. Ser. A 240, 599 (1948).
  * Shenker (1957) H. Shenker, Phys. Rev. 107, 1246 (1957).
  * Daffé  _et al._ (2018) N. Daffé, F. Choueikani, S. Neveu, M.-A. Arrio, A. Juhin, P. Ohresser, V. Dupuis, and P. Sainctavit, J. Magn. Magn. Mater. 460, 243 (2018).
  * Moya  _et al._ (2021) C. Moya, A. Fraile Rodríguez, M. Escoda-Torroella, M. García del Muro, S. R. V. Avula, C. Piamonteze, X. Batlle, and A. Labarta, J. Phys. Chem. C 125, 691 (2021).
  * Fantauzzi  _et al._ (2019) M. Fantauzzi, F. Secci, M. Sanna Angotzi, C. Passiu, C. Cannas, and A. Rossi, RSC Adv. 9, 19171 (2019).