# On the Approximate Purification of Mixed Strategies in Games with Infinite
Action Sets

Yuhki Hosoya
Faculty of Economics, Chuo University
Chaowen Yu
Department of Economics, Keio University Corresponding Author,
E-mail:hosoya(at)tamacc.chuo-u.ac.jp, ORCID ID:0000-0002-8581-4518We are
grateful to Toru Maruyama, Shinsuke Nakamura, Takako Fujiwara-Greve, and Toru
Hokari for their helpful comments and suggestions.

###### Abstract

We consider a game in which the action set of each player is uncountable, and
show that, from weak assumptions on the common prior, any mixed strategy has
an approximately equivalent pure strategy. The assumption of this result can
be further weakened if we consider the purification of a Nash equilibrium.
Combined with the existence theorem for a Nash equilibrium, we derive an
existence theorem for a pure strategy approximated Nash equilibrium under
sufficiently weak assumptions. All of the pure strategies we derive in this
paper can take a finite number of possible actions.

JEL codes: C62, C65, C72, C73.

Keywords: mixed strategy, approximate purification, uncountable action set,
conditionally atomless, Nash equilibrium.

##  1 Introduction

The notion of the mixed strategy was introduced by von Neumann and Morgenstern
(1944) to prove the minimax theorem, and was later used in Nash’s (1950)
existence theorem for a Nash equilibrium. In game theory, there are many cases
in which the existence of a solution can be guaranteed by using this concept.
Thus, the concept of mixed strategies continues to be extensively used today.

Mixed strategies have, however, been criticized for a long time because, if a
player wants to implement a mixed strategy, he/she has to prepare his/her own
randomizing device. From a positive point of view, a solution is interpreted
to represent human behaviors in the real world. However, there is almost no
behavior in the real world that can be interpreted as that of using a
randomizing device. Also, from a normative point of view, the solution of the
game is considered to be a proposal presented by the mediator of the conflict.
However, a proposal that would require each party to use a randomizing device
to resolve a dispute would be difficult for a mediator to submit and, even if
submitted, is unlikely to be accepted by each party. In conclusion, in many
cases, solutions derived using mixed strategies are problematic.

On the other hand, in the context of Bayesian games, a strategy is described
as a function from the space of outside signals into the space of actions.
This strategy is randomized by the signal, and can take probabilistic actions
without the need to use a randomization device, so the problem as described
above is not encountered. Thus, if we can create such a “pure strategy” that
is “equivalent” to a given mixed strategy, the problem of interpretation will
disappear. The research theme of purifying mixed strategies was born from this
idea.

In this research area, a certain independence in the signal structure results
in the existence of a pure strategy that is “exactly equivalent” to a given
mixed strategy (e.g., Radner and Rosenthal (1982), Milgrom and Weber (1985)).
These results have been treated in a unified manner by Khan et al. (2006). In
these studies, the set of possible actions were assumed to be finite. Khan and
Rath (2009) extended this result to the case where the action set is countably
infinite. However, this result cannot be extended to the case where the set of
possible actions is uncountable, as in the case of Cournot games. In this
case, Khan and Zhang (2014) produced the following interesting result. In
games where the set of actions for each player has a continuum cardinality, a
necessary and sufficient condition for there to have to be a pure strategy
Nash equilibrium that is “exactly equivalent” to a given mixed strategy Nash
equilibrium is that the space of signals is saturated.

We view this result of Khan and Zhang (2014) as a kind of impossibility
theorem. To guarantee the existence of a pure strategy that is equivalent to a
mixed strategy, the space of signals must be saturated. This assumption of
saturation implies that the space of signals is “very large”, meaning that an
incredible variety of randomness can be extracted from the signals. However,
in a phenomenon that can be viewed as a game-theoretic situation, it is
unlikely that players are able to receive such many outside signals. Also, in
many applied studies, a Polish space is adopted as the space of signals.
However, any Polish space is not saturated. Therefore, it is impossible to
show the existence of an “exactly equivalent” pure strategy under the usual
assumptions.

In this study, we consider another approach: the existence of a pure strategy
that is “approximately equivalent” to a given mixed strategy. Aumann et al.
(1983) showed that there exists a pure strategy that is “approximately
equivalent” to an arbitrary given mixed strategy when only certain non-atomic
properties of the common prior are assumed. This result has the weakness that
it can only guarantee the existence of an “approximately equivalent” pure
strategy, but has the advantage that it does not require independence in the
signal structure and guarantees the existence of such a strategy in very
general situations. Note that the non-atomic property is also needed for
“exact equivalence” results, and thus the assumption required for this result
is weaker than that required for “exact equivalence” results. Although the
action set is assumed to be finite in Aumann et al. (1983), we extend this
result to the case where the action set is an arbitrary compact metric space
(Theorem 1, Theorem 2). Our assumption is that the space of signals is a
Polish space, and thus it is not assumed to be saturated at all. In this
sense, we consider that this result is a non-trivial generalization of past
studies.

The assumption on the common prior can be further weakened when considering
the purification of a Nash equilibrium. This result was also shown by Aumann
et al. (1983) for the case where each action set is finite, but this study
extends this result to the case where each action set is a compact metric
space (Theorem 3). Balder (1988) proved the existence theorem for a Nash
equilibrium with mixed strategies under certain assumptions. Connecting this
result with ours, we can show the general existence of an approximated Nash
equilibrium with pure strategies (Theorem 4). This result is an extension of
Corollary of Milgrom and Weber (1985).

Note that all pure strategies that come out in our results can be considered
as functions that take values in a finite subset of the action set. This is
one of the good properties of our results, as real people are not always able
to finely change their behavior in response to signals.

In subsection 2.1, we introduce our model and several seminal results. In
subsection 2.2, we introduce the requirements for the common prior used in
this study. In subsection 2.3, we explain our setup of the space of mixed
strategies. Section 3 presents our main results. The proofs of all results are
given in section 4.

##  2 The Model

###  2.1 Basic Notation

A game considered in this study is represented by a tuple

$G=(N,(K_{i})_{i\in N},(\Omega,\mathscr{S},\mu),u,(X_{i},x_{i})_{i\in N}).$

The set $N=\\{1,...,n\\}$ is the set of all players, and $K_{i}$ is the action
set for player $i$. We assume that each $K_{i}$ is a compact metric space. The
probability space $(\Omega,\mathscr{S},\mu)$ is interpreted as the space of
outside signals. The probability measure $\mu$ denotes the common prior. The
function $u:\prod_{i=1}^{n}K_{i}\times\Omega\to\mathbb{R}^{m}$ is the payoff
function. Although it is natural that $n=m$ and $u_{i}$ denotes the payoff of
the $i$-th player, we consider that $m$ may be not equal to $n$ for a
technical reason. If $m=n$, then we say that this game is usual. We assume
that $u(k,\omega)$ is continuous in $k$ and measurable in $\omega$, and that
there exists a positive integrable function $r:\Omega\to\mathbb{R}$ such that
$\|u(k,\omega)\|\leq r(\omega)$ for all $(k,\omega)$.

The set $X_{i}$ is the observation space for player $i$, and the function
$x_{i}:\Omega\to X_{i}$ is the observation function. We assume that, for each
$i$, $X_{i}$ is a Polish space with a continuum cardinality, and
$x_{i}:\Omega\to X_{i}$ is Borel measurable. Note that, by Kuratowski’s
theorem,111See Theorem 3.3.13 of Srivastava (1998). $X_{i}$ is isomorphic to
$[0,1]$ as a measure space. Define a function $\mu^{\prime}$ on
$\prod_{i=1}^{n}X_{i}$ as follows:

$\mu^{\prime}(A)=\int 1_{\\{x\in A\\}}d\mu.$

We can easily confirm that $\mu^{\prime}$ is a Borel probability measure on
$\prod_{i=1}^{n}X_{i}$. By using $\mu^{\prime}$ instead of $\mu$, we can
assume, without loss of generality, that $\Omega=\prod_{i=1}^{n}X_{i}$,
$\mathscr{S}$ is the Borel $\sigma$-algebra of $\Omega$, and $\mu$ is a Borel
probability measure on $\Omega$. Clearly, in this case
$x_{i}(\omega_{1},...,\omega_{n})=\omega_{i}$ for each $i$. In this
connection, we omit the notation $\mathscr{S}$ and $(X_{i},x_{i})$, and simply
write this game $G$ as $(N,(K_{i})_{i\in N},(\Omega,\mu),u)$.

We call a Borel measurable function $f_{i}:X_{i}\to K_{i}$ a pure strategy of
player $i$. Despite the name, the actual action of player $i$ while following
a pure strategy is randomized by the observation of the outside signal
$x_{i}$. Let $\mathscr{M}^{1}_{+}(K_{i})$ be the set of all probability
measures on $K_{i}$. We call a weakly measurable function222The definition of
weak measurability is explained in subsection 2.3.
$f_{i}:X_{i}\to\mathscr{M}^{1}_{+}(K_{i})$ a mixed strategy.333This name may
appear odd. Actually, this function is sometimes called by different names in
this research area. For example, Radner and Rosenthal (1982) called such a
function a ‘behavior strategy’. Milgrom and Weber (1985) called it a
‘behavioral strategy’ and discussed the relationship between this and the
‘distributional strategy’ that they used. Balder (1988) called it a
‘transition probability’. Aumann et al. (1983) called this function a ‘mixed
strategy’. The name here follows that of Khan and Zhang (2014). Because
$k_{i}\in K_{i}$ is identified with the Dirac measure
$\delta_{k_{i}}\in\mathscr{M}^{1}_{+}(K_{i})$, any pure strategy is also a
mixed strategy.444Later, we will show rigorously that our pure strategy is
actually weakly measurable. $f=(f_{1},...,f_{n})$ is called a strategy profile
if each $f_{i}$ is a (pure or mixed) strategy of player $i$. To avoid a
confusion, we use the notation $f_{i,x_{i}}$ instead of $f_{i}(x_{i})$. The
expected payoff function $U$ is defined by

$U(f)=\int ud(f_{1,x_{1}}\otimes...\otimes f_{n,x_{n}})d\mu.$

The following proposition is obviously needed.

Proposition 1. $U$ is a well-defined $\mathbb{R}^{m}$-valued function.

The next proposition asserts the equivalence between our mixed strategy and
the ‘transition probability’ used in Balder (1988). Later, we will use
Balder’s (1988) main theorem to prove our Theorem 4, and thus this proposition
is needed.

Proposition 2. A function $f_{i}:X_{i}\to\mathscr{M}^{1}_{+}(K_{i})$ is a
mixed strategy if and only if for each Borel set $B$ in $K_{i}$, the function
$x_{i}\mapsto f_{i,x_{i}}(B)$ is measurable on $X_{i}$.

Let $f_{i}$ be a mixed strategy of player $i$. A strategy $f_{i}^{\prime}$ is
exactly equivalent to $f_{i}$ if and only if, for every strategy profile
$f_{-i}$ of players other than player $i$ and every $j\in\\{1,...,m\\}$,

$U_{j}(f_{i},f_{-i})=U_{j}(f_{i}^{\prime},f_{-i}).$

Similarly, a strategy $f_{i}^{\prime}$ is $\varepsilon$-equivalent to $f_{i}$
if and only if for every strategy profile $f_{-i}$ of players other than
player $i$,

$\max_{j\in\\{1,...,m\\}}|U_{j}(f_{i},f_{-i})-U_{j}(f_{i}^{\prime},f_{-i})|<\varepsilon.$

A pure strategy $f_{i}^{\prime}$ is called an exact purification (resp.
$\varepsilon$-purification) of the strategy $f_{i}$ if $f_{i}^{\prime}$ is
exactly equivalent (resp. $\varepsilon$-equivalent) to $f_{i}$.

Notes on Basic Notation. In related research, our definition of equivalence is
sometimes called the payoff equivalence. There is another notion of
equivalence, called the distributional equivalence. Two strategies $f_{i}$ and
$f_{i}^{\prime}$ are distributionally equivalent if, for every Borel
measurable set $\Lambda\subset K_{i}$,

$\int 1_{\\{k_{i}\in\Lambda\\}}df_{i,x_{i}}d\mu=\int
1_{\\{k_{i}\in\Lambda\\}}df_{i,x_{i}}^{\prime}d\mu.$

If $K_{i}$ is finite, then the number of possible $\Lambda$ is also finite,
and thus we can easily deduce the distributional equivalence result from the
payoff equivalence result. That is, if the possibility of $\Lambda$ is
$\Lambda_{1},...,\Lambda_{L}$, then for $\Lambda_{\ell}$, define

$u_{m+\ell}(k,x)=\begin{cases}1&\mbox{if }k_{i}\in\Lambda_{\ell},\\\
0&\mbox{otherwise}.\end{cases}$

Then, the payoff equivalence leads to the distributional equivalence. However,
if $K_{i}$ is infinite, then the above method cannot be used, because the
range of $u$ becomes infinite dimensional.

In many previous studies, the finiteness of $K_{i}$ and some independence on
the signal structure were assumed, and then the existence of an exact
purification in both the payoff and distributional senses was proved (see
Milgrom and Weber (1985), Radner and Rosenthal (1982), Khan et al. (2006)).
The finiteness requirement of $K_{i}$ can be replaced with a countability
requirement (see Khan and Rath (2009)). However, if $K_{i}$ is uncountable,
there is an impossibility theorem regarding exact purification. That is, if
$K_{i}$ is uncountable, then there may be no exact purification, even though
the signal structure satisfies the usual independence requirement (see Khan
and Zhang (2014)). If $X_{i}$ is a saturated measurable space, then this
problem can be avoided. However, every Polish space is not saturated, and thus
our games cannot avoid this problem.

Under our setup, Theorem 2 of Khan and Zhang (2014) implies that, for any
given signal structure, we can construct a game in which there is a mixed
strategy that has no exact purification. Meanwhile, Aumann et al. (1983)
showed that if $K_{i}$ is finite, then for every mixed strategy and
$\varepsilon>0$, there exists an $\varepsilon$-purification. Their result does
not need the independence requirement, and only some atomless requirement of
$\mu$ is needed. The purpose of this study is to extend this result to the
case in which $K_{i}$ is not finite.

###  2.2 Conditionally Atomless and Weakly Conditionally Atomless Priors

Recall that each $X_{i}$ is a Polish space, $\Omega=\prod_{i=1}^{n}X_{i}$, and
$\mu$ is a Borel probability measure on $\Omega$. We say that $\mu$ is
conditionally atomless for player $i$ if the conditional probability
$\mu(\cdot|x_{-i})$ is non-atomic almost surely with respect to the marginal
probability $\mu_{X_{-i}}$, where
$x_{-i}=(x_{1},...,x_{i-1},x_{i+1},...,x_{n})$ and $X_{-i}=\prod_{j\in N,j\neq
i}X_{j}$ as usual.555For definitions of marginal probability and conditional
probability, see section 10.2 of Dudley (2002). We also say that $\mu$ is
weakly conditionally atomless for player $i$ if for every $j\in N$ such that
$i\neq j$, $\mu_{ij}$ is conditionally atomless for player $i$, where
$\mu_{ij}(A)=\mu(A\times\prod_{p\in N\setminus\\{i,j\\}}X_{p})$ for every
Borel measurable set $A\subset X_{i}\times X_{j}$.

It is known that every conditionally atomless probability for player $i$ is
weakly conditionally atomless for player $i$, and that the converse is not
true. See Proposition 3 and Example 2 of Yu et al. (2017) for detailed
arguments. Meanwhile, if $\mu$ is not conditionally atomless, then there may
exist a mixed strategy with no $\varepsilon$-purification for sufficiently
small $\varepsilon>0$ even when $K_{i}$ is finite. Therefore, the conditional
atomless requirement is crucial for $\varepsilon$-purification.

###  2.3 Basic Knowledge in the Space of Probability Measures

In this section, we present some basic knowledge regarding the space of
probability measures on a separable and complete metric space. All of the
facts mentioned in this section are proved in ch.11 of Dudley (2002), ch.1 of
Billingsley (1999), or ch.1-2 of Parthasarathy (2014).

Let $(K,d_{K})$ be a separable and complete metric space. Then,
$\mathscr{M}^{1}_{+}(K)$ denotes the set of all probability measures on $K$.
Recall the definition of the Prohorov metric: for a set $A\subset K$ and
$\varepsilon>0$, let $A^{\varepsilon}=\\{x\in K|\exists y\in A\mbox{ s.t.
}d_{K}(x,y)<\varepsilon\\}$. For any $P,Q\in\mathscr{M}^{1}_{+}(K)$, define

$\rho(P,Q)=\inf\\{\varepsilon>0|P(A)\leq Q(A^{\varepsilon})+\varepsilon\mbox{
for each Borel set }A\\}.$

This function $\rho$ is called the Prohorov metric. Under this metric,
$\mathscr{M}^{1}_{+}(K)$ is separable and complete. Moreover, the convergence
of $(P^{\nu})$ to $P$ with respect to the Prohorov metric is equivalent to the
weak* convergence.666Recall that $P^{\nu}$ converges to $P$ with respect to
the weak* topology if and only if $\lim_{\nu\to\infty}\int udP^{\nu}=\int udP$
for every continuous and bounded function $u:K\to\mathbb{R}$. Furthermore,
there exists a countable dense set $\mathscr{P}\subset\mathscr{M}_{1}^{+}(K)$
such that each $P\in\mathscr{P}$ has a finite support, and if $K$ is compact,
$\mathscr{M}_{1}^{+}(K)$ is also compact with respect to the Prohorov metric.

Let $X$ be some measurable space. Then, we call a function
$f:X\to\mathscr{M}^{1}_{+}(K)$ weakly measurable if, for every continuous
function $v:\mathscr{M}^{1}_{+}(K)\to\mathbb{R}$, the composition $v\circ
f:X\to\mathbb{R}$ is measurable. Note that if $g:X\to K$ is measurable, then a
function $f:x\mapsto\delta_{g(x)}$ is weakly measurable, where $\delta_{g(x)}$
is the Dirac measure. Indeed, if we define $h(k)=\delta_{k}$, then
$f(x)=(h\circ g)(x)$. It is easy to show that the function $h$ is continuous,
and thus for every continuous function
$v:\mathscr{M}^{1}_{+}(K)\to\mathbb{R}$, $v\circ f=(v\circ h)\circ g$ is
measurable. Thus, we have that $f$ is weakly measurable. In this connection,
we have that every pure strategy is also a mixed strategy.

If $f(X)$ is finite and $f^{-1}(P)$ is measurable for every
$P\in\mathscr{M}^{1}_{+}(K)$, then we call $f$ a simple function.

We can obtain the following result. This is the basis of this paper.

Lemma 1. Suppose that $K$ is a separable complete metric space, and $X$ is a
measurable space. Let $\mathscr{P}$ be a countable dense set in
$\mathscr{M}^{1}_{+}(K)$ with respect to the Prohorov metric $\rho$. Then, for
every weakly measurable function $f:X\to\mathscr{M}^{1}_{+}(K)$, there exists
a sequence $(f_{\nu})$ of simple functions such that the range of $f_{\nu}$ is
in $\mathscr{P}$ for every $\nu$, and $\rho(f_{\nu}(x),f(x))\to 0$ as
$\nu\to\infty$ for every $x\in X$.

##  3 Results

###  3.1 First Result: the Existence of an Approximate Purification

In this section, we assume that $N=\\{1,2\\}$, and use the following notation:
$X_{1}=X,K_{1}=K,X_{2}=Y,K_{2}=L$. By assumption, $K$ and $L$ are compact
metric spaces, and thus are separable and complete. Recall the definition of
$\varepsilon$-purification. That is, if $f$ is a mixed strategy of player 1,
then a pure strategy $f^{\prime}$ of player 1 is an $\varepsilon$-purification
of $f$ if and only if for every mixed strategy $g$ of player 2,

$\max_{i\in\\{1,...,m\\}}|U_{i}(f,g)-U_{i}(f^{\prime},g)|<\varepsilon.$

Our first main result is as follows.

Theorem 1. Suppose that $N=\\{1,2\\}$ and $\mu$ is conditionally atomless for
player 1. Then, for every $\varepsilon>0$ and every mixed strategy $f$ of
player 1, there exists a finite subset $K^{\prime}$ of $K$ and a
$\varepsilon$-purification $f^{\prime}$ of $f$ such that for every $x$,
$f_{x}^{\prime}$ is included in $K^{\prime}$.

As a corollary, the following result holds. Recall that a game is said to be
usual if and only if the dimension of the range of $U$ is the same as the
number of players. If $N=\\{1,...,n\\}$ and the game is usual, then for a
mixed strategy $f_{i}$ of player $i$, a pure strategy $f_{i}^{\prime}$ of
player $i$ is an $\varepsilon$-purification of $f_{i}$ if and only if, for
every profile $f_{-i}$ of mixed strategies of players other than $i$,

$\max_{j\in\\{1,...,n\\}}|U_{j}(f,f_{-i})-U_{j}(f^{\prime},f_{-i})|<\varepsilon.$

Theorem 2. Suppose that the game is usual and $\mu$ is conditionally atomless
for player $i$. Then, for every $\varepsilon>0$ and every mixed strategy
$f_{i}$ of player $i$, there exists a finite subset $K_{i}^{\prime}$ of
$K_{i}$ and an $\varepsilon$-purification of $f_{i}^{\prime}$ such that for
every $x_{i}$, $f_{i,x_{i}}$ is included in $K_{i}^{\prime}$.

###  3.2 Second Result: the Existence of a Pure Approximated Nash Equilibrium

We should define the notion of approximated Nash equilibria. Suppose that the
game is usual. Then, the function $U_{i}$ can be seen as the payoff of player
$i$. For a given $\varepsilon\geq 0$, the strategy profile $f$ is an
$\varepsilon$-Nash equilibrium if and only if, for every strategy $g_{i}$ of
player $i$,

$U_{i}(g_{i},f_{-i})\leq U_{i}(f)+\varepsilon.$

We call a $0$-Nash equilibrium a Nash equilibrium as usual. If $f$ is a Nash
equilibrium, then $f^{\prime}$ is called an $\varepsilon$-purification of $f$
if the following requirements holds.

  1. 1)

For every $i$, $f_{i}^{\prime}$ is a pure strategy of player $i$.

  2. 2)

If $f^{\prime\prime}$ is another strategy profile such that
$f_{i}^{\prime\prime}$ is either $f_{i}$ or $f_{i}^{\prime}$ for every $i$,
then $f^{\prime\prime}$ is an $\varepsilon$-Nash equilibrium that satisfies

$\max_{i\in\\{1,...,n\\}}|U_{i}(f^{\prime\prime})-U_{i}(f)|<\varepsilon.$

Note that, by 2), $f^{\prime}$ itself is also an $\varepsilon$-Nash
equilibrium.

Then, the following results is obtained.

Theorem 3. If the game is usual and $\mu$ is weakly conditionally atomless for
all players, then for every $\varepsilon>0$ and every Nash equilibrium $f$,
there exists an $\varepsilon$-purification $f^{\prime}$ of $f$ such that, for
each player $i$, the range of $f_{i}^{\prime}$ is finite.

Combining our Theorem 3 and Balder’s (1988) main theorem, we obtain the
following result.

Theorem 4. Suppose that the game is usual, $\mu$ is weakly conditionally
atomless for all players, and $\mu$ is absolutely continuous with respect to
$\mu_{1}\otimes\cdots\otimes\mu_{n}$. Then, for every $\varepsilon>0$, there
exists an $\varepsilon$-Nash equilibrium such that each player $i$ chooses a
pure strategy $f_{i}$ and the range of $f_{i}$ is finite.

##  4 Conclusion

We showed that for only weak atomless assumption on the prior, there exists an
approximate purification for any mixed strategy. This atomless requirement
could be further weakened when considering a purification of a Nash
equilibrium. We did not need any independence assumption on signal structure
to show these results. Using these results, we have succeeded in obtaining an
existence theorem for an $\varepsilon$-Nash equilibrium under sufficiently
weak assumptions.

All of these results were obtained under the assumption that all action sets
are compact. In most of the previous studies, action sets are assumed to be
finite, and in this sense, our results are a generalization of these results.
Since it has been shown that this generalization is not possible in the
context of exact purification, we can say that our result is a non-trivial
generalization.

It is not known whether it is possible to remove compactness of the action
sets. Since we have already used the compactness of the space in Step 1 of the
proof of Theorem 1, it is completely unknown whether such a generalization is
possible.

##  5 Proofs

###  5.1 Proofs of Lemma 1 and Proposition 1

First, we prove Lemma 1. Because $\mathscr{P}$ is countable, we can set
$\mathscr{P}=\\{P_{1},P_{2},...\\}$. Note that, because the function
$P\mapsto\rho(P_{i},P)$ is continuous with respect to $\rho$, we have that
$v_{i}:x\mapsto\rho(P_{i},f(x))$ is measurable. For each $\nu$, define
$I_{\nu}(x)=\\{i\in\\{1,...,\nu\\}|v_{i}(x)=\min_{j\in\\{1,...,\nu\\}}v_{j}(x)\\}$,
and $f_{\nu}(x)=P_{i}$ for $i=\min I_{\nu}(x)$. Then,

$\displaystyle f_{\nu}(x)=P_{i}\Leftrightarrow$
$\displaystyle~{}v_{i}(x)<v_{j}(x)\mbox{ for every }j\in\\{1,...,i-1\\}$
$\displaystyle\mbox{ and }v_{i}(x)\leq v_{j}(x)\mbox{ for every
}j\in\\{i+1,...,\nu\\}.$

Thus, the function $f_{\nu}$ is simple and for every $x$, $f_{\nu}(x)\to f(x)$
as $\nu\to\infty$, which implies that Lemma 1 holds.

Next, we show Proposition 1. Choose any strategy profile
$f=(f_{1},...,f_{n})$. Let $(f_{i}^{\nu})$ be a sequence of simple functions
that converges to $f_{i}$ pointwise with respect to the weak* topology. Then,
$(f_{1}^{\nu}\otimes...\otimes f_{n}^{\nu})$ converges to
$f_{1}\otimes...\otimes f_{n}$ pointwise with respect to the weak*
topology.777See Theorem 2.8 of Billingsley (1999). Now, because

$\|u(k,x)\|\leq r(x),$

we have that a function $u_{x}:k\to u(k,x)$ is bounded and continuous.
Therefore,

$w_{\nu}:x\mapsto\int u_{x}d(f_{1,x_{1}}^{\nu}\otimes...\otimes
f_{n,x_{n}}^{\nu})$

is well-defined and integrable with respect to $\mu$. Moreover, because
$\|w_{\nu}(x)\|\leq r(x)$ for every $x\in X$, we have that

$\int\|w_{\nu}\|d\mu\leq\int rd\mu$

for every $\nu$. Because this function converges to

$w:x\mapsto\int u_{x}d(f_{1,x_{1}}\otimes...\otimes f_{n,x_{n}})$

pointwise, by the dominated convergence theorem, $U(f)=\int wd\mu$ is well-
defined. This completes the proof of Proposition 1. $\blacksquare$

###  5.2 Proof of Proposition 2

Suppose that $f_{i}:X_{i}\to\mathscr{M}^{1}_{+}(K_{i})$ is a function such
that for every Borel set $B\subset K_{i}$, $x_{i}\mapsto f_{i,x_{i}}(B)$ is
measurable. We will show that this function is weakly measurable.

Since $\mathscr{M}_{+}^{1}(K_{i})$ is compact, there exists a countable dense
subset $\mathscr{P}=\\{P_{1},P_{2},...\\}$ of $\mathscr{M}_{+}^{1}(K_{i})$.
For each $\nu$, define

$v_{\nu}(x_{i})=\rho(P_{\nu},f_{i,x_{i}}),$

where $\rho$ is the Prohorov metric of $\mathscr{M}_{+}^{1}(K_{i})$. We show
that each $v_{\nu}$ is measurable. Recall that for $\varepsilon>0$ and
$A\subset K_{i}$,

$A^{\varepsilon}=\\{k_{i}\in K_{i}|\exists k_{i}^{\prime}\in K_{i}\mbox{ s.t.
}d_{i}(k_{i},k_{i}^{\prime})<\varepsilon\\},$

where $d_{i}$ is the metric of $K_{i}$. For each $A\subset K_{i}$, define

$\eta_{A}(x_{i})=\inf\\{\varepsilon>0|f_{i,x_{i}}(A)\leq
P_{\nu}(A^{\varepsilon})+\varepsilon\\}.$

Then, by the definition of the Prohorov metric,

$v_{\nu}(x_{i})=\sup_{C\in\mathscr{C}_{i}}\eta_{C}(x_{i}),$

where $\mathscr{C}_{i}$ denotes the set of all closed subsets of
$K_{i}$.888See section 11.3 of Dudley (2002). Now, for each
$C\in\mathscr{C}_{i}$ and each rational number $q\in[0,1]$, define

$\varphi_{C,q}(x_{i})=\begin{cases}q&\mbox{if }f_{i,x_{i}}(C)\leq
P_{\nu}(C^{q})+q,\\\ 1&\mbox{otherwise}.\end{cases}$

Since $x_{i}\mapsto f_{i,x_{i}}(C)$ is measurable, $\varphi_{C,q}$ is
measurable. Therefore, for each $C\in\mathscr{C}_{i}$,
$\eta_{C}=\inf_{q\in[0,1]\cap\mathbb{Q}}\varphi_{C,q}$ is measurable.

Because $K_{i}$ is compact, it is second countable, and thus it has a
countable basis $\mathscr{V}_{i}=\\{V_{1},V_{2},...\\}$. Define

$\tilde{\mathscr{C}}_{i}=\\{C\in\mathscr{C}_{i}|C=(\cup_{j\in
M}V_{j})^{c}\mbox{ for some finite set }M\subset\mathbb{N}\\}.$

Since $\mathscr{V}_{i}$ is countable, so is $\tilde{\mathscr{C}}_{i}$. Fix
$x_{i}\in X_{i}$ and $C\in\mathscr{C}_{i}$. There exists a subfamily
$\mathscr{V}_{i}^{\prime}$ of $\mathscr{V}_{i}$ such that
$C=(\cup_{V\in\mathscr{V}_{i}^{\prime}}V)^{c}$, and thus, there exists a
decreasing sequence $(A_{j})_{j\in\mathbb{N}}$ of sets in
$\tilde{\mathscr{C}}_{i}$ such that $A_{j}\downarrow C$ as $j\to\infty$. Let
$\varepsilon^{*}=\sup_{j\in\mathbb{N}}\eta_{A_{j}}(x_{i})$ and choose an
$\varepsilon>0$. Then, by the definition of $\varepsilon^{*}$,

$f_{i,x_{i}}(A_{j})\leq
P_{\nu}((A_{j})^{\varepsilon^{*}+\varepsilon})+\varepsilon^{*}+\varepsilon,$

for each $j\in\mathbb{N}$. Therefore, by taking the limit $j\to\infty$, we
have

$f_{i,x_{i}}(C)\leq
P_{\nu}(\cap_{j}(A_{j})^{\varepsilon^{*}+\varepsilon})+\varepsilon^{*}+\varepsilon.$

We show that $\cap_{j=1}^{\infty}(A_{j})^{\varepsilon^{*}+\varepsilon}\subset
C^{\varepsilon^{*}+2\varepsilon}$. Choose any
$k_{i}\in\cap_{j}(A_{j})^{\varepsilon^{*}+\varepsilon}$. Then, for each $j$,
there exists $k_{i}^{j}\in A_{j}$ such that
$d_{i}(k_{i},k_{i}^{j})<\varepsilon^{*}+\varepsilon$. Since $K_{i}$ is
compact, we have that there is a limit point $k_{i}^{*}$ of the sequence
$(k_{i}^{j})$. By the definition of $A_{j}$, we have that $k_{i}^{*}\in C$.
Because
$d_{i}(k_{i},k_{i}^{*})\leq\varepsilon^{*}+\varepsilon<\varepsilon^{*}+2\varepsilon$,
we have that $k_{i}\in C^{\varepsilon^{*}+2\varepsilon}$, as desired.
Therefore,

$f_{i,x_{i}}(C)\leq
P_{\nu}(C^{\varepsilon^{*}+2\varepsilon})+\varepsilon^{*}+2\varepsilon.$

Since $\varepsilon>0$ is arbitrary, it follows that
$\eta_{C}(x_{i})\leq\varepsilon^{*}$. Therefore, for each $x_{i}\in X_{i}$,

$v_{\nu}(x_{i})=\sup_{C\in\mathscr{C}_{i}}\eta_{C}(x_{i})=\sup_{C\in\tilde{\mathscr{C}_{i}}}\eta_{C}(x_{i}),$

and thus $v_{\nu}$ is measurable.

Now, for each $\nu$, let
$I_{\nu}(x_{i})=\\{\ell\in\\{1,...,\nu\\}|v_{\ell}(x_{i})=\min_{j\in\\{1,..,\nu\\}}v_{j}(x_{i})\\}$
for each $x_{i}\in X_{i}$ and define $f_{i,x_{i}}^{\nu}=P_{\ell}$ for
$\ell=\min I_{\nu}(x_{i})$. Then,

$\displaystyle f_{i,x_{i}}^{\nu}=P_{\ell}\Leftrightarrow$
$\displaystyle~{}v_{\ell}(x)<v_{j}(x)\mbox{ for every }j\in\\{1,...,\ell-1\\}$
$\displaystyle\mbox{ and }v_{\ell}(x)\leq v_{j}(x)\mbox{ for every
}j\in\\{i+1,...,\nu\\}.$

Because each $v_{j}$ is measurable, we have that $f_{i}^{\nu}$ is a simple
function. Moreover, $\rho_{i}(f_{i,x_{i}}^{\nu},f_{i,x_{i}})\to 0$ as
$\nu\to\infty$. Let $\tau:\mathscr{M}^{1}_{+}(K_{i})\to\mathbb{R}$ be a
continuous function. Then, for each $\nu$, $\tau\circ f_{i}^{\nu}$ is a simple
function. Moreover, $\tau\circ f_{i}^{\nu}\to\tau\circ f_{i}$ pointwise as
$\nu\to\infty$, and hence $\tau\circ f_{i}$ is measurable. This implies that
$f_{i}$ is weakly measurable, as desired.

Next, we show the opposite direction. Let
$f_{i}:X_{i}\to\mathscr{M}_{+}^{1}(K_{i})$ be a weakly measurable function. It
suffices to show that for every Borel subset $B$ of $K_{i}$, the following
function $\zeta_{B}:x_{i}\mapsto f_{i,x_{i}}(B)$ is measurable. First, for
each continuous function $\psi:K_{i}\to\mathbb{R}$,

$\xi_{\psi}:Q\mapsto\int\psi dQ$

is continuous. Because $f_{i}$ is weakly measurable, for each continuous
function $\psi:K_{i}\to\mathbb{R}$, $\xi_{\psi}\circ f_{i}$ is measurable.
Next, let $\mathscr{L}_{i}$ be a family of Borel sets of $K_{i}$ such that
$B\in\mathscr{L}_{i}$ if and only if $\zeta_{B}$ is measurable on $X_{i}$. Fix
a closed set $C\subset K_{i}$. For each $\nu$, let

$\psi_{\nu}(k_{i})=\max\\{1-\nu\inf_{k_{i}^{\prime}\in
C}d_{i}(k_{i},k_{i}^{\prime}),0\\}.$

Then, $\psi_{\nu}$ is continuous. For each $k_{i}\in K_{i}$,
$\psi_{\nu}(k_{i})\downarrow 1_{C}(k_{i})$ as $\nu\to\infty$, and by the
monotone convergence theorem, we have that for each $x_{i}\in X_{i}$,
$\int\psi_{\nu}df_{i,x_{i}}\to\int 1_{C}df_{i,x_{i}}$. Therefore, the sequence
$(\xi_{\psi_{\nu}}\circ f_{i})$ converges pointwise to the function
$\zeta_{C}$ as $\nu\to\infty$, which implies that $C\in\mathscr{L}_{i}$.

Since $\zeta_{K_{i}\setminus B}=1-\zeta_{B}$, we have that
$B\in\mathscr{L}_{i}$ implies $K_{i}\setminus B\in\mathscr{L}_{i}$. Therefore,
$\mathscr{L}_{i}$ contains all open sets. Moreover, if
$B_{1}\in\mathscr{L}_{i}$ is open and $B_{2}\in\mathscr{L}_{i}$ is closed,
then $B_{1}\cap B_{2}\in\mathscr{L}_{i}$ because $\zeta_{B_{1}\cap
B_{2}}=\zeta_{B_{1}}-\zeta_{B_{1}\setminus B_{2}}$ is measurable. Let
$\mathscr{D}_{i}$ be the set of all Borel sets such that there exists an open
$B_{1}$ and a closed $B_{2}$ such that $B=B_{1}\cap B_{2}$. Then,
$\mathscr{D}_{i}$ is a semiring and
$\mathscr{D}_{i}\subset\mathscr{L}_{i}$.999A family $\mathscr{D}$ of subsets
of $Z$ is called a semiring if 1) $\emptyset\in\mathscr{D}$, 2) for each pair
$A,B\in\mathscr{D}$, $A\cap B\in\mathscr{D}$, and 3) for each pair
$A,B\in\mathscr{D}$, $A\setminus B=\bigcup_{j=1}^{k}C_{j}$ for some $k$ and
disjoint $C_{1},...,C_{k}\in\mathscr{D}$. See section 3.2 of Dudley (2002) for
detailed arguments. Next, define

$\mathscr{A}_{i}=\\{B|B=\cup_{j=1}^{\nu}D_{j}\mbox{ for some }\nu\mbox{ and
disjoint }D_{1},...,D_{\nu}\in\mathscr{D}_{i}\\}.$

Then, $\mathscr{A}_{i}$ is an algebra that includes all open sets.101010See
Proposition 3.2.3 of Dudley (2002). Note that, for each pair
$B_{1},B_{2}\in\mathscr{L}_{i}$ of disjoint sets, we have $B_{1}\cup
B_{2}\in\mathscr{L}_{i}$ because $\zeta_{B_{1}\cup
B_{2}}=\zeta_{B_{1}}+\zeta_{B_{2}}$. This implies that
$\mathscr{A}_{i}\subset\mathscr{L}_{i}$. If $(A_{\nu})$ is an increasing
(resp. decreasing) sequence of sets in $\mathscr{L}_{i}$ with $A_{\nu}\uparrow
A$ (resp. $A_{\nu}\downarrow A$) as $\nu\to\infty$, then by the monotone
convergence theorem, we have that $\zeta_{A_{\nu}}\to\zeta_{A}$ pointwise, and
thus $A\in\mathscr{L}_{i}$. This implies that $\mathscr{L}_{i}$ is a monotone
class that contains $\mathscr{A}_{i}$. By the monotone class lemma,111111See
Theorem 4.4.2 of Dudley (2002). we have that $\mathscr{L}_{i}$ is the same as
the Borel $\sigma$-field of $K_{i}$. Thus, $\zeta_{B}$ is measurable for any
Borel subset $B\subset K_{i}$. This completes the proof. $\blacksquare$

###  5.3 Proof of Theorem 1

Throughout this proof, we treat $\rho$ as the Prohorov metric of either
$\mathscr{M}^{1}_{+}(K)$ or $\mathscr{M}^{1}_{+}(L)$. We consider that the
abbreviation of the notation $K$ or $L$ for Prohorov metrics should not cause
any confusion.

Note that, because $K$ is compact, there exists a countable dense family
$\mathscr{P}=\\{P_{1},P_{2},...\\}$ of $\mathscr{M}^{1}_{+}(K)$ such that each
$P_{i}$ has a finite support. We call a mixed strategy simple if it is a
simple function as a function from $X$ into $\mathscr{M}^{1}_{+}(K)$.

We separate the proof into eight steps.

Step 1. Suppose that $f$ is a mixed strategy of player $1$, and $(f_{\nu})$ is
a sequence of simple mixed strategies of player $1$ that converges to $f$
pointwise as $\nu\to\infty$. Then, for every $(x,y)\in X\times Y$ and
$i\in\\{1,...,m\\}$, the following holds.

$\lim_{\nu\to\infty}\sup_{\ell\in
L}\left|\int_{K}u_{i}(k,\ell,x,y)d(f_{x}-f_{\nu,x})\right|=0.$

Proof of Step 1. Suppose not. By taking a subsequence, we can assume that
there exists $\delta>0$ and a sequence $(\ell_{\nu})$ of $L$ such that

$\left|\int_{K}u_{i}(k,\ell_{\nu},x,y)d(f_{x}-f_{\nu,x})\right|\geq\delta.$

Because $L$ is compact, we can assume without loss of generality that
$\ell_{\nu}\to\ell^{*}\in L$ as $\nu\to\infty$. Moreover, because $K$ and $L$
are compact, the function $(k,\ell)\mapsto u_{i}(k,\ell,x,y)$ is uniformly
continuous, and thus for the metrics $d_{K}$ of $K$ and $d_{L}$ of $L$, there
exists $\delta^{\prime}>0$ such that if
$d_{K}(k,k^{\prime})+d_{L}(\ell,\ell^{\prime})<\delta^{\prime}$, then
$|u_{i}(k,\ell,x,y)-u_{i}(k^{\prime},\ell^{\prime},x,y)|<\frac{\delta}{8}$.
Because $f_{\nu,x}$ converges to $f_{x}$ with respect to the weak* topology,
for sufficiently large $\nu$, $\rho(\ell_{\nu},\ell^{*})<\delta^{\prime}$ and

$\left|\int_{K}u_{i}(k,\ell^{*},x,y)d(f_{\nu,x}-f_{x})\right|<\frac{\delta}{2}.$

Therefore, for any such $\nu$,

$\displaystyle~{}\int_{K}u_{i}(k,\ell_{\nu},x,y)d(f_{\nu,x}-f_{x})$
$\displaystyle\leq$
$\displaystyle~{}\int_{K}\left(u_{i}(k,\ell^{*},x,y)+\frac{\delta}{8}\right)d(f_{\nu,x}-f_{x})^{+}$
$\displaystyle~{}-\int_{K}\left(u_{i}(k,\ell^{*},x,y)-\frac{\delta}{8}\right)d(f_{\nu,x}-f_{x})^{-}$
$\displaystyle\leq$
$\displaystyle~{}\frac{\delta}{2}+\int_{K}u_{i}(k,\ell^{*},x,y)d(f_{\nu,x}-f_{x})<\delta,$

where $(f_{\nu,x}-f_{x})^{+}$ (resp. $(f_{\nu,x}-f_{x})^{-}$) is the positive
(resp. negative) part of the Jordan decomposition of the measure
$f_{\nu,x}-f_{x}$. By symmetric arguments, we can show that

$\int_{K}u_{i}(k,\ell_{\nu},x,y)d(f_{\nu,x}-f_{x})>-\delta,$

and thus,

$\left|\int_{K}u_{i}(k,\ell_{\nu},x,y)d(f_{\nu,x}-f_{x})\right|<\delta,$

which is a contradiction. This completes the proof of Step 1. $\blacksquare$

Fix any $\varepsilon>0$ and any mixed strategy $f$ of player $1$. By Lemma 1,
there exists a sequence $(f_{\nu})$ of simple mixed strategies of player $1$
such that $f_{\nu}\to f$ pointwise as $\nu\to\infty$ and the range of
$f_{\nu}$ is always included in $\mathscr{P}$. By Step 1 and the dominated
convergence theorem, there exists $\nu$ such that

$\int_{X\times Y}\sup_{\ell\in
L}\left|\int_{K}u_{i}(k,\ell,x,y)d(f_{x}-f_{\nu,x})\right|d\mu<\frac{\varepsilon}{2}$

for every $i\in\\{1,...,m\\}$. Set $f^{\prime\prime}=f_{\nu}$, and let
$K^{\prime}$ be the union of the support of $f_{x}^{\prime\prime}$. Then,
$K^{\prime}$ is a finite set. Moreover, for every mixed strategy $g$ of player
2,

$\displaystyle|U_{i}(f^{\prime\prime},g)-U_{i}(f,g)|=$
$\displaystyle~{}\left|\int_{X\times Y}\int_{K\times
L}u_{i}(k,\ell,x,y)d((f_{x}^{\prime\prime}-f_{x})\otimes g_{y})d\mu\right|$
$\displaystyle\leq$ $\displaystyle~{}\int_{X\times
Y}\int_{L}\left|\int_{K}u_{i}(k,\ell,x,y)d(f_{x}^{\prime\prime}-f_{x})\right|dg_{y}d\mu$
$\displaystyle\leq$ $\displaystyle~{}\int_{X\times Y}\sup_{\ell\in
L}\left|\int_{K}u_{i}(k,\ell,x,y)d(f_{x}^{\prime\prime}-f_{x})\right|d\mu$
$\displaystyle<$ $\displaystyle~{}\frac{\varepsilon}{2}.$

Step 2. There exists $\delta>0$ such that for any two strategies $g^{1}$ and
$g^{2}$ of player $2$ and any strategy $f^{\prime}$ of player $1$, if
$\rho(g^{1}_{y},g^{2}_{y})<\delta$ for every $y\in Y$ and the support of
$f_{x}^{\prime}$ is included in $K^{\prime}$ for every $x\in X$, then for
every $i\in\\{1,...,m\\}$,

$|U_{i}(f^{\prime},g^{1})-U_{i}(f^{\prime},g^{2})|<\frac{\varepsilon}{6}.$

Proof of Step 2. Suppose not. Then, for every positive integer $\nu$, we can
choose $i,f_{\nu},g^{1}_{\nu},g^{2}_{\nu}$ such that the support of
$f_{\nu,x}$ is included in $K^{\prime}$ for every $x\in X$,
$\rho(g^{1}_{\nu,y},g^{2}_{\nu,y})<\frac{1}{\nu}$ for every $y\in Y$, and

$U_{i}(f_{\nu},g^{1}_{\nu})-U_{i}(f_{\nu},g^{2}_{\nu})\geq\frac{\varepsilon}{6}.$

Because $\mathscr{M}^{1}_{+}(L)$ is compact and $K^{\prime}$ is finite, we
have that the following mapping

$P\mapsto\max_{k\in K^{\prime}}\int_{L}u_{i}(k,\ell,x,y)dP$

is uniformly continuous with respect to the Prohorov metric $\rho$. Therefore,

$\max_{k\in
K^{\prime}}\int_{L}u_{i}(k,\ell,x,y)d(g^{1}_{\nu,y}-g^{2}_{\nu,y})\to 0$

as $\nu\to\infty$. By the dominated convergence theorem,

$\displaystyle\frac{\varepsilon}{6}\leq$
$\displaystyle~{}U_{i}(f_{\nu},g^{1}_{\nu})-U_{i}(f_{\nu},g^{2}_{\nu})$
$\displaystyle=$ $\displaystyle~{}\int_{X\times Y}\sum_{k\in
K^{\prime}}f_{\nu,x}(\\{k\\})\int_{L}u_{i}(k,\ell,x,y)d(g^{1}_{\nu,y}-g^{2}_{\nu,y})d\mu$
$\displaystyle\leq$ $\displaystyle~{}\int_{X\times Y}\max_{k\in
K^{\prime}}\int_{L}u_{i}(k,\ell,x,y)d(g^{1}_{\nu,y}-g^{2}_{\nu,y})d\mu$
$\displaystyle\to$ $\displaystyle~{}0\mbox{ as }\nu\to\infty,$

which is a contradiction. This completes the proof of Step 2. $\blacksquare$

Step 3. There exist a finite subset $L^{\prime}\subset L$ and
$\\{Q_{1},...,Q_{N}\\}\subset M^{1}_{+}(L)$ such that the support of $Q_{j}$
is included in $L^{\prime}$ for all $j$, and for every mixed strategy $g$ of
player $2$, there exists a simple mixed strategy $g^{\prime}$ of player $2$
such that $g_{y}^{\prime}=Q_{j}$ for some $j$ and
$\rho(g_{y},g_{y}^{\prime})<\delta$ for every $y\in Y$, where $\delta>0$ is
given in Step 2.

Proof of Step 3. Because $L$ is compact, there exists a countable dense subset
$\mathscr{Q}$ of $\mathscr{M}^{1}_{+}(L)$ such that every $Q\in\mathscr{Q}$
has a finite support. Therefore, there exists a finite set $L^{\prime}\subset
L$ and $\\{Q_{1},...,Q_{N}\\}\subset\mathscr{M}^{1}_{+}(L)$ such that the
support of $Q_{j}$ is included in $L^{\prime}$ for all $j$, and for every
$Q\in\mathscr{M}^{1}_{+}(L)$, there exists $j$ such that
$\rho(Q,Q_{j})<\delta$. Choose any mixed strategy $g$ of player $2$, and
define $g_{y}^{\prime}=Q_{j^{*}}$, where
$j^{*}=\min\\{j|\rho(g_{y},Q_{j})<\delta\\}$. Then, $g^{\prime}$ is a simple
mixed strategy such that $g_{y}^{\prime}=Q_{j}$ for some $j$ and
$\rho(g_{y},g_{y}^{\prime})<\delta$ for every $y\in Y$, as desired. This
completes the proof of Step 3. $\blacksquare$

Until the end of Step 7, we assume that $u_{i}$ is nonnegative for all $i$.
Define

$c=\sum_{(i,k,\ell)\in\\{1,...,m\\}\times K^{\prime}\times
L^{\prime}}\int_{X\times Y}u_{i}(k,\ell,x,y)d\mu.$

If $c=0$, then we can add $u_{m+1}(k,l,x,y)\equiv 1$, and thus we can assume
without loss of generality that $c>0$. Define
$Y^{\prime}=Y\times\\{1,...,m\\}\times K^{\prime}\times L^{\prime}$ and a
measure $\nu$ on the set $X\times Y^{\prime}$ such that

$\nu(A\times\\{(i,k,\ell)\\})=c^{-1}\int_{A}u_{i}(k,\ell,x,y)d\mu.$

Define

$c_{i,k,\ell}(y)=\int_{X}u_{i}(k,\ell,x,y)\mu(dx|y),$
$d_{i,k,\ell}(y)=\begin{cases}0&\mbox{if }c_{i,k,\ell}(y)=0,\\\
\frac{1}{c_{i,k,\ell}(y)}&\mbox{otherwise}.\end{cases}$

If $y^{\prime}=(y,i,k,\ell)$, then

$\nu(T|y^{\prime})=d_{i,k,\ell}(y)\int_{T}u_{i}(k,\ell,x,y)\mu(dx|y),$
$\nu_{Y^{\prime}}(S\times\\{(i,k,\ell)\\})=c^{-1}\int_{S}c_{i,k,\ell}(y)d\mu_{Y}.$

In particular, $d_{i,k,\ell}(y)=\frac{1}{c_{i,k,\ell}(y)}$ for almost all
$(y,i,k,\ell)\in Y^{\prime}$ with respect to $\nu_{Y^{\prime}}$, and thus
$\nu$ is conditionally atomless for player $1$.

Step 4. Let $T$ be a Borel subset of $X$. Then, there exists a sequence of the
partitions $(\\{H_{1}^{M},...,H_{M}^{M}\\})$ of $T$ such that

$\lim_{M\to\infty}\int_{Y^{\prime}}\left(\max_{j}\nu(H_{j}^{M}|y^{\prime})\right)d\nu_{Y^{\prime}}=0.$

Proof of Step 4. Because $X$ is Polish, by Kuratowski’s theorem, we can assume
without loss of generality that $X=[0,1]$. First, we show that every Borel
probability measure $P$ on $[0,1]$ is atomless if and only if the cumulative
distribution function $F(x)=P([0,x])$ satisfies $F(0)=0$ and is uniformly
continuous.

Suppose that $P$ is atomless. Then, $P(\\{x\\})=0$ for all $x\in[0,1]$, and
thus $F(0)=0$ and $F$ is left-continuous. Because every cumulative
distribution function is right-continuous, we have that $F$ is continuous.
Since $[0,1]$ is compact, $F$ is uniformly continuous.

Conversely, suppose that $F(0)=0$ and $F$ is uniformly continuous. Then,
$P(\\{x\\})=0$ for all $x\in[0,1]$. For every Borel subset $B\subset[0,1]$,

$\displaystyle~{}\max_{i\in\\{1,...,m\\}}P(B\cap[(i-1)/m,i/m])$
$\displaystyle\leq$
$\displaystyle~{}\max_{i\in\\{1,...,m\\}}[F(i/m)-F((i-1)/m)]\to 0\mbox{ as
}m\to\infty,$

which implies that $B$ is not an atom of $P$. Thus, our claim is correct.

For $M\in\mathbb{N}$, define

$H_{1}^{M}=T\cap[0,M^{-1}],$

and for $j\in\\{2,...,M\\}$,

$H_{j}^{M}=T\cap](j-1)M^{-1},jM^{-1}].$

We show that this sequence $(\\{H_{1}^{M},...,H_{M}^{M}\\})$ of partitions of
$T$ satisfies the requirement of our claim.

Fix any $y^{\prime}\in Y^{\prime}$ such that $\nu(\cdot|y^{\prime})$ is
atomless. By our previous argument, we have that the cumulative distribution
function $x\mapsto\nu([0,x]|y^{\prime})$ is uniformly continuous and
$\nu(\\{0\\}|y^{\prime})=0$, and thus

$\lim_{M\to\infty}\left(\max_{j}\nu(H_{j}^{M}|y^{\prime})\right)=0.$

Because $\nu$ is conditionally atomless for player $1$, we have that
$\nu(\cdot|y^{\prime})$ is atomless for almost all $y^{\prime}$ with respect
to $\nu_{Y^{\prime}}$. Therefore, by the dominated convergence theorem, we
have that

$\lim_{M\to\infty}\int_{Y^{\prime}}\left(\max_{j}\nu(H_{j}^{M}|y^{\prime})\right)d\nu_{Y^{\prime}}=0,$

as desired. This completes the proof of Step 4. $\blacksquare$

Let $\mathbb{R}^{K^{\prime}}$ be the set of all real-valued function on
$K^{\prime}$. For $s\in\mathbb{R}^{K^{\prime}}$, we write $s_{k}$ instead of
$s(k)$, and define $\|s\|=\sqrt{\sum_{k\in K^{\prime}}s_{k}^{2}}$ as usual.
Let $\Delta^{K^{\prime}}$ be the set of all $s\in\mathbb{R}^{K^{\prime}}$ such
that $s_{k}\geq 0$ for all $k\in K^{\prime}$ and $\sum_{k\in
K^{\prime}}s_{k}=1$. Moreover, let $V^{K^{\prime}}$ be the set of all
$s\in\mathbb{R}^{K^{\prime}}$ such that there exists $k\in K^{\prime}$ such
that $s_{k}=1$ and $s_{k^{\prime}}=0$ for all $k^{\prime}\in
K^{\prime}\setminus\\{k\\}$.

Step 5. Suppose that $s\in\Delta^{K^{\prime}}$. Then, for every measurable set
$T$ in $X$ and $\kappa>0$, there exists a measurable function $b:T\to
V^{K^{\prime}}$ such that

$\int_{Y^{\prime}}\left\|\nu(T|y^{\prime})s-\int_{T}b(x)\nu(dx|y^{\prime})\right\|d\nu_{Y^{\prime}}<\kappa.$

Proof of Step 5. Let $\Omega^{\prime}$ be some probability measure space and
$Z^{1},Z^{2},...$ be an independent family of random variables defined on
$\Omega^{\prime}$ such that $Z^{j}(\omega)\in V^{K^{\prime}}$ and
$E(Z^{j})=s$.121212The existence of such $\Omega^{\prime}$ and
$Z^{1},Z^{2},...$ can easily be shown. For example, let $\Omega_{i}$ be a copy
of $K^{\prime}$ with probability $P_{i}$ such that $P_{i}(\\{k\\})=s_{k}$. Let
$\Omega^{\prime}$ be the product probability space
$\prod_{i=1}^{\infty}\Omega_{i}$, and
$Z^{i}_{k}(k_{1},k_{2},...)=\begin{cases}1&\mbox{if }k=k_{i},\\\ 0&\mbox{if
}k\neq k_{i}.\end{cases}$ Then, $Z^{1},Z^{2},...$ satisfies all our
requirements. Note that, by definition, we have that
$\sum_{k}\mbox{Var}(Z_{k}^{j})\leq 1$. Define $H_{1}^{M},...,H_{M}^{M}$ as in
Step 4, and

$b_{M}(x,\omega)=Z^{j}(\omega)\mbox{ if }x\in H_{j}^{M}.$

Then, we have that

$\int_{T}b_{M}(x,\omega)\nu(dx|y^{\prime})=\sum_{j=1}^{M}\nu(H_{j}^{M}|y)Z^{j}(\omega),$

and thus,

$E\left(\int_{T}b_{M}(x,\omega)\nu(dx|y^{\prime})\right)=\nu(T|y^{\prime})s.$

Meanwhile, for almost all $y^{\prime}\in Y^{\prime}$,

$\displaystyle~{}E\left(\left\|\nu(T|y^{\prime})s-\int_{T}b_{M}(x,\omega)\nu(dx|y^{\prime})\right\|^{2}\right)$
$\displaystyle=$
$\displaystyle~{}E\left(\left\|\nu(T|y^{\prime})s-\sum_{j}\nu(H_{j}^{M}|y^{\prime})Z^{j}(\omega)\right\|^{2}\right)$
$\displaystyle=$
$\displaystyle~{}\sum_{k}\mbox{Var}\left(\sum_{j}\nu(H_{j}^{M}|y^{\prime})Z_{k}^{j}\right)$
$\displaystyle=$
$\displaystyle~{}\sum_{j}(\nu(H_{j}^{M}|y^{\prime}))^{2}\sum_{k}\mbox{Var}(Z_{k}^{j})$
$\displaystyle\leq$ $\displaystyle~{}\sum_{j}(\nu(H_{j}^{M}|y^{\prime}))^{2}$
$\displaystyle\leq$ $\displaystyle~{}\max_{j}\nu(H_{j}^{M}|y^{\prime}),$

where the last inequality follows from Hölder’s inequality. Therefore, by Step
4 and Fubini’s theorem, for sufficiently large $M$, we have that

$\displaystyle~{}E\left(\int_{Y^{\prime}}\left\|\nu(T|y^{\prime})s-\int_{T}b_{M}(x,\omega)\nu(dx|y^{\prime})\right\|^{2}d\nu_{Y^{\prime}}\right)$
$\displaystyle=$
$\displaystyle~{}\int_{Y^{\prime}}E\left(\left\|\nu(T|y^{\prime})s-\int_{T}b_{M}(x,\omega)\nu(dx|y^{\prime})\right\|^{2}\right)d\nu_{Y^{\prime}}<\kappa^{2}.$

Hence, there exists $\omega\in\Omega^{\prime}$ such that

$\displaystyle~{}\left(\int_{Y^{\prime}}\left\|\nu(T|y^{\prime})s-\int_{T}b_{M}(x,\omega)\nu(dx|y^{\prime})\right\|d\nu_{Y^{\prime}}\right)^{2}$
$\displaystyle\leq$
$\displaystyle~{}\int_{Y^{\prime}}\left\|\nu(T|y^{\prime})s-\int_{T}b_{M}(x,\omega)\nu(dx|y^{\prime})\right\|^{2}d\nu_{Y^{\prime}}<\kappa^{2},$

where the first inequality is verified by the Cauchy-Schwarz inequality.
Hence, we can set $b(x)=b_{M}(x,\omega)$. This completes the proof of Step 5.
$\blacksquare$

For every bounded measurable function $h:X\to\mathbb{R}^{K^{\prime}}$, define
a seminorm

$\|h\|_{\nu}=\int_{Y^{\prime}}\left\|\int_{X}h(x)\nu(dx|y^{\prime})\right\|d\nu_{Y^{\prime}}.$

Note that, if $h$ is a mixed strategy of player $1$ and the support of $h_{x}$
is always included in $K^{\prime}$, then $h_{x}$ can be seen as an element of
$\Delta^{K^{\prime}}$, and thus $h$ can be treated as a bounded measurable
function from $X$ into $\mathbb{R}^{K^{\prime}}$, and the seminorm
$\|h\|_{\nu}$ can be defined.

Step 6. For every $\kappa>0$ and every mixed strategy $h$ of player $1$ such
that $h_{x}\in\Delta^{K^{\prime}}$ for all $x\in X$, there exists a pure
strategy $h^{\prime}$ of player 1 such that
$h_{x}^{\prime}\in\Delta^{K^{\prime}}$ for all $x\in X$ and

$\|h-h^{\prime}\|_{\nu}<\kappa.$

Proof of Step 6. Clearly, $h:X\to\Delta^{K^{\prime}}$ is measurable in the
usual sense, and thus, by the dominated convergence theorem, there exists a
simple mixed strategy $h^{\prime\prime}:X\to\Delta^{K^{\prime}}$ such that

$\|h-h^{\prime\prime}\|_{\nu}\leq\frac{\kappa}{2}.$

Meanwhile, the range of $h^{\prime\prime}$ is a finite set
$\\{p^{1},...,p^{q}\\}$. Let $T_{j}=(h^{\prime\prime})^{-1}(p^{j})$. Then,
applying Step 5 for $T=T_{j}$ and $s=p^{j}$, we have that there exists a
measurable function $b^{j}:T_{j}\to V^{K^{\prime}}$ such that

$\int_{Y^{\prime}}\left\|\int_{T_{j}}(h_{x}^{\prime\prime}-b^{j}(x))\nu(dx|y^{\prime})\right\|d\nu_{Y^{\prime}}<\frac{\kappa}{2q}.$

Define $h_{x}^{\prime}=b^{j}(x)$ if $x\in T_{j}$. Then, $h^{\prime}$ is a
function from $X$ into $V^{K^{\prime}}$, and thus it is actually a pure
strategy. Moreover,

$\|h^{\prime\prime}-h^{\prime}\|_{\nu}=\int_{Y^{\prime}}\left\|\int_{X}(h_{x}^{\prime\prime}-h_{x}^{\prime})\nu(dx|y^{\prime})\right\|d\nu_{Y^{\prime}}<\frac{\kappa}{2}.$

Thus,

$\|h-h^{\prime}\|_{\nu}\leq\|h-h^{\prime\prime}\|_{\nu}+\|h^{\prime\prime}-h^{\prime}\|_{\nu}<\kappa,$

as desired. This completes the proof of Step 6. $\blacksquare$

Step 7. The claim of this theorem is correct if $u_{i}$ is nonnegative for
each $i$.

Proof of Step 7. Recall that we have already fixed $\varepsilon$ and
$f,f^{\prime\prime}$ before Step 2. By Step 6, there exists a pure strategy
$f^{\prime}$ such that

$\|f^{\prime\prime}-f^{\prime}\|_{\nu}<\frac{\varepsilon}{6c|K^{\prime}||L^{\prime}|}.$

Choose any strategy $g$ of player $2$. By Step 3, there exists a simple mixed
strategy $g^{\prime}$ of player $2$ such that the support of $g_{y}^{\prime}$
is included in $L^{\prime}$ and $\rho(g_{y},g_{y}^{\prime})<\delta$ for every
$y\in Y$. For
$h_{k}(x)=f_{x}^{\prime\prime}(\\{k\\})-f_{x}^{\prime}(\\{k\\})$,

$\displaystyle~{}\left|\int_{Y}g_{y}^{\prime}(\\{\ell\\})\int_{X}h_{k}(x)u_{i}(k,\ell,x,y)\mu(dx|y)d\mu_{Y}\right|$
$\displaystyle\leq$
$\displaystyle~{}\int_{Y}g_{y}^{\prime}(\\{\ell\\})\left|\int_{X}h_{k}(x)u_{i}(k,\ell,x,y)\mu(dx|y)\right|d\mu_{Y}$
$\displaystyle=$
$\displaystyle~{}c\int_{Y\times\\{(i,k,\ell)\\}}g_{y}^{\prime}(\\{\ell\\})d_{i,k,\ell}(y)\left|\int_{X}c_{i,k,\ell}(y)h_{k}(x)\nu(dx|y^{\prime})\right|d\nu_{Y^{\prime}}$
$\displaystyle=$
$\displaystyle~{}c\int_{Y\times\\{(i,k,\ell)\\}}g_{y}^{\prime}(\\{\ell\\})\left|\int_{X}h_{k}(x)\nu(dx|y^{\prime})\right|d\nu_{Y^{\prime}}$
$\displaystyle\leq$
$\displaystyle~{}c\|f^{\prime\prime}-f^{\prime}\|_{\nu}<\frac{\varepsilon}{6|K^{\prime}||L^{\prime}|}.$

Therefore, we have that

$|U_{i}(f^{\prime\prime},g^{\prime})-U_{i}(f^{\prime},g^{\prime})|<\frac{\varepsilon}{6}.$

By Step 2, this implies that

$\displaystyle|U_{i}(f^{\prime\prime},g)-U_{i}(f^{\prime},g)|\leq$
$\displaystyle~{}|U_{i}(f^{\prime\prime},g)-U_{i}(f^{\prime\prime},g^{\prime})|+|U_{i}(f^{\prime\prime},g^{\prime})-U_{i}(f^{\prime},g^{\prime})|$
$\displaystyle~{}+|U_{i}(f^{\prime},g^{\prime})-U_{i}(f^{\prime},g)|$
$\displaystyle<$ $\displaystyle~{}\frac{\varepsilon}{2}.$

Therefore,

$|U_{i}(f,g)-U_{i}(f^{\prime},g)|\leq|U_{i}(f,g)-U_{i}(f^{\prime\prime},g)|+|U_{i}(f^{\prime\prime},g)-U_{i}(f^{\prime},g)|<\varepsilon,$

as desired. This completes the proof of Step 7. $\blacksquare$

Step 8. The claim of this theorem is correct.

Proof of Step 8. Define

$v_{i}(k,l,x,y)=\begin{cases}\max\\{u_{i}(k,l,x,y),0\\}&\mbox{if }1\leq i\leq
m,\\\ \max\\{-u_{i-m}(k,l,x,y),0\\}&\mbox{if }m+1\leq i\leq 2m,\end{cases}.$

Then, applying Step 7 for a nonnegative function $v:K\times L\times X\times
Y\to\mathbb{R}^{2m}$, we obtain the desired result. This completes the proof
of Step 8. $\blacksquare$

###  5.4 Proof of Theorem 2

We construct a modified game of the original game as follows. First, there are
two players, one is $i$ and another is $0$. The observation spaces, the common
prior, and the payoff function are the same as those in the original game. The
action set and the observation function of player $i$ are the same as those of
the original game, whereas the action set and the observation function of
player $0$ are $\prod_{j\neq i}K_{j}$ and
$x_{-i}=(x_{1},...,x_{i-1},x_{i+1},...,x_{n})$, respectively. Because the
prior $\mu$ is conditionally atomless for player $i$ in the original game, it
has the same property in this modified game. Therefore, by Theorem 1, for each
$\varepsilon>0$ and each mixed strategy $f_{i}$ of player $i$, there exists a
pure strategy $f_{i}^{\prime}$ such that for some finite subset
$K_{i}^{\prime}$ of $K_{i}$, $f_{i,x_{i}}^{\prime}\in K_{i}^{\prime}$ for any
$x_{i}\in X_{i}$ and for each mixed strategy $g$ of player $0$ in the modified
game,

$\max_{j\in\\{1,...,n\\}}|U_{j}(f_{i},g)-U_{j}(f_{i}^{\prime},g)|<\varepsilon.$

Now, for each $j\neq i$, choose any mixed strategy $f_{j}$ of player $j$ in
the original game. Then, the profile
$f_{-i}=(f_{1},...,f_{i-1},f_{i+1},...,f_{n})$ is a mixed strategy of player
$0$ in the modified game. Therefore, by the above inequality, we have

$\max_{j\in\\{1,...,n\\}}|U_{j}(f_{i},f_{-i})-U_{j}(f_{i}^{\prime},f_{-i})|<\varepsilon,$

as desired. This completes the proof. $\blacksquare$

###  5.5 Proofs of Theorems 3-4

Theorem 3.1 of Balder (1988) showed that under the assumptions of Theorem 4,
there exists a Nash equilibrium such that every strategy is, in his
terminology, a ‘transition probability’. We showed in Proposition 2 that our
‘mixed strategy’ is equivalent to this ‘transition probability’ of Balder
(1988). Therefore, Theorem 3 implies Theorem 4, and it suffices to show
Theorem 3.

We need a lemma.

Lemma 2. Suppose that $G^{1},...,G^{m}$ are two player games, and for every
game $G^{j}$, player $1$ has the same observation set $X_{0}$ and action set
$K_{0}$. Moreover, suppose that for any $j$, the common prior $\mu^{j}$ in the
game $G^{j}$ is conditionally atomless for player $1$. Then, for every
$\varepsilon>0$ and every mixed strategy $f$ of player $1$, there exists a
pure strategy $f^{\prime}$ of player $1$ such that $f^{\prime}$ is
$\varepsilon$-equivalent to $f$ in every game $G^{j}$, and the range of
$f^{\prime}$ is finite.

Proof. We can assume that the payoff function $u^{j}$ of the game $G^{j}$ is a
real-valued function: if not, then we can replace $G^{j}$ with games
$G^{j}_{1},...,G^{j}_{M}$, where $M$ is the dimension of the range of $u^{j}$,
every $G^{j}_{i}$ has the same observation sets, action sets, and the common
prior as $G^{j}$, and the payoff function of $G^{j}_{i}$ is $u^{j}_{i}$. Let
$Y^{j}$ and $L^{j}$ be the observation set and the action set for player $2$
in $G^{j}$, respectively. We can assume without loss of generality that
$Y^{1},...,Y^{m}$ are disjoint. Define $Y=Y^{1}\cup...\cup Y^{m}$,
$L=L^{1}\times...\times L^{m}$, and

$\mu(B)=\sum_{j=1}^{m}\frac{1}{m}\mu^{j}(B\cap(X_{0}\times Y^{j}))$

for all Borel sets $B\subset X_{0}\times Y$. Moreover, for each $j$, define

$u_{j}(k_{0},\ell,x_{0},y)=\begin{cases}u^{j}(k_{0},\ell^{j},x_{0},y)&\mbox{if
}y\in Y^{j},\\\ 0&\mbox{otherwise},\end{cases}$

where $\ell=(\ell^{1},...,\ell^{m})$. Consider the game
$G=(\\{1,2\\},(K,L),(X_{0}\times Y,\mu),u)$. We can easily check that $\mu$ is
conditionally atomless for player $1$. By Theorem 1, there exists an
$(\varepsilon/m)$-purification $f^{\prime}$ of $f$ in the game $G$ such that
the range of $f^{\prime}$ is finite. For each $j$, choose any mixed strategy
$g^{j}$ of player $2$ in game $G^{j}$ and $y_{j}^{*}\in Y^{j}$. If $y\in
Y^{j}$, define

$g_{y}=g_{y_{1}^{*}}^{1}\otimes...\otimes g_{y_{j-1}^{*}}^{j-1}\otimes
g_{y}^{j}\otimes g_{y_{j+1}^{*}}^{j+1}\otimes...\otimes g_{y_{m}^{*}}^{m}.$

Using Proposition 2, we can easily check that $g$ is a mixed strategy of
player $2$ in game $G$. Therefore,

$\frac{\varepsilon}{m}>|U_{j}(f,g)-U_{j}(f^{\prime},g)|=\frac{1}{m}|U^{j}(f,g^{j})-U^{j}(f^{\prime},g^{j})|,$

which completes the proof of Lemma 2. $\blacksquare$

Choose any Nash equilibrium $f$. We construct pure strategies
$f_{1}^{\prime},...,f_{n}^{\prime}$ recursively. Let $1\leq m\leq n$ and
suppose that for $i<m$, $f_{i}^{\prime}$ is already defined. Define

$H_{m}=\\{h|h_{j}=f_{j}\mbox{ for }j\geq m,\ h_{j}=f_{j}\mbox{ or
}h_{j}=f_{j}^{\prime}\mbox{ for }j<m\\}.$

Note that, this set is finite, and in particular, $H_{1}=\\{f\\}$. For any
$j\neq m$ and $h\in H_{m}$, we construct a two player game
$G^{mjh}=(\\{m,j\\},(K_{m},K_{j}),(X_{m}\times X_{j},\mu_{mj}),u^{mjh})$ by
defining $u^{mjh}(k_{m},k_{j},x_{m},x_{j})$ by the following integral:

$\int_{\prod_{\ell\in N\setminus\\{m,j\\}}(X_{\ell}\times
K_{\ell})}u(k,x)d\left(\bigotimes_{\ell\in
N\setminus\\{m,j\\}}h_{\ell,x_{\ell}}\right)\mu(dx_{-mj}|x_{m},x_{j}),$

where $x_{-mj}=(x_{1},...,x_{m-1},x_{m+1},...,x_{j-1},x_{j+1},...,x_{n})$.
Because $\mu$ is weakly conditionally atomless, the family $(G^{mjh})$
satisfies all requirements of Lemma 2, and thus there exists a pure strategy
$f_{m}^{\prime}$ of player $m$ such that

$\max_{i\in
N}|U^{mjh}_{i}(f_{m},g_{j})-U^{mjh}_{i}(f_{m}^{\prime},g_{j})|<\frac{\varepsilon}{3^{n-m+1}}$

for any $j\neq m$ and mixed strategy $g_{j}$ of player $j$ in game $G^{mjh}$,
and the range of $f_{m}^{\prime}$ is finite.

We use mathematical induction to show that if $h\in H_{m}$, $h$ is an
$(\varepsilon/3^{n-m+1})$-Nash equilibrium, and

$\max_{i\in N}|U_{i}(h)-U_{i}(f)|<\frac{\varepsilon}{3^{n-m+1}}.$

If $m=1$, then $H_{1}=\\{f\\}$, and this result is obvious. Suppose that this
result holds for $m\in\\{1,...,n\\}$, and choose any $h^{\prime}\in H_{m+1}$.
If $h_{m}^{\prime}=f_{m}$, then $h^{\prime}\in H_{m}$, and thus, our claim
automatically holds. Hence, we assume that $h_{m}^{\prime}=f_{m}^{\prime}$.
Let $h$ be a strategy profile such that $h_{j}=h_{j}^{\prime}$ for $j\neq m$
and $h_{m}=f_{m}$. Then, $h\in H_{m}$, and thus, for any $j\neq m$, $u^{mjh}$
is defined and, for a mixed strategy $g_{j}$ of player $j$,

$U^{mjh}(f_{m},g_{j})=U(g_{j},h_{-j}),$
$U^{mjh}(f_{m}^{\prime},g_{j})=U(g_{j},h_{-j}^{\prime}).$

Choose any mixed strategy $g_{i}^{\prime}$ of player $i$ and set
$g_{j}^{\prime}=h_{j}^{\prime}$ for all $j\neq i$. If $i=m$, then
$g_{j}^{\prime}=h_{j}$ for all $j\neq m$, and by the induction hypothesis,

$U_{i}(g^{\prime})-U_{i}(h)\leq\frac{\varepsilon}{3^{n-m+1}}.$

Therefore, for some $j\neq m$,

$\displaystyle U_{i}(g^{\prime})-U_{i}(h^{\prime})=$
$\displaystyle~{}U_{i}(g^{\prime})-U_{i}(h)+U_{i}(h)-U_{i}(h^{\prime})$
$\displaystyle\leq$
$\displaystyle~{}U_{i}(g^{\prime})-U_{i}(h)+|U^{mjh}_{i}(f_{m},h_{j})-U^{mjh}_{i}(f_{m}^{\prime},h_{j})|$
$\displaystyle<$
$\displaystyle~{}\frac{\varepsilon}{3^{n-m+1}}+\frac{\varepsilon}{3^{n-m+1}}$
$\displaystyle<$ $\displaystyle~{}\frac{\varepsilon}{3^{n-(m+1)+1}}.$

Hence, we assume that $m\neq i$. Let $g_{j}=g_{j}^{\prime}$ if $j\neq m$ and
$g_{m}=h_{m}$. Because of the induction hypothesis,

$U_{i}(g)-U_{i}(h)\leq\frac{\varepsilon}{3^{n-m+1}}.$

Meanwhile,

$|U_{i}(g^{\prime})-U_{i}(g)|=|U^{mih}_{i}(f_{m}^{\prime},g_{i})-U^{mih}_{i}(f_{m},g_{i})|<\frac{\varepsilon}{3^{n-m+1}},$
$|U_{i}(h^{\prime})-U_{i}(h)|=|U^{mih}_{i}(f_{m}^{\prime},h_{i})-U^{mih}_{i}(f_{m},h_{i})|<\frac{\varepsilon}{3^{n-m+1}}.$

Therefore,

$\displaystyle U_{i}(g^{\prime})-U_{i}(h^{\prime})=$
$\displaystyle~{}U_{i}(g^{\prime})-U_{i}(g)+U_{i}(g)-U_{i}(h)+U_{i}(h)-U_{i}(h^{\prime})$
$\displaystyle\leq$
$\displaystyle~{}|U_{i}(g^{\prime})-U_{i}(g)|+U_{i}(g)-U_{i}(h)+|U_{i}(h)-U_{i}(h^{\prime})|$
$\displaystyle<$
$\displaystyle~{}\frac{\varepsilon}{3^{n-m+1}}+\frac{\varepsilon}{3^{n-m+1}}+\frac{\varepsilon}{3^{n-m+1}}$
$\displaystyle=$ $\displaystyle~{}\frac{\varepsilon}{3^{n-(m+1)+1}}.$

Thus, $h^{\prime}$ is an $(\varepsilon/3^{n-(m+1)+2})$-Nash equilibrium.
Moreover,

$|U_{i}(h^{\prime})-U_{i}(f)|\leq|U_{i}(h^{\prime})-U_{i}(h)|+|U_{i}(h)-U_{i}(f)|<\frac{\varepsilon}{3^{n-(m+1)+1}}.$

This completes the proof of this induction. In particular, to set $m=n+1$, we
have that $f^{\prime}$ is an $\varepsilon$-purification of the Nash
equilibrium $f$. This completes the proof. $\blacksquare$

## References

[1] Aumann, R. J., Katznelson, Y., Radner, R., Rosenthal, R. W., Weiss, B.
Approximate Purification of Mixed Strategies. Math. Oper. Res. 8, 327-341
(1983).

[2] Balder, E. J. Generalized Equilibrium Results for Games with Incomplete
Information. Math. of Oper. Res. 13, 265-276 (1988).

[3] Billingsley, P. Convergence of Probability Measures. Wiley (1999).

[4] Dudley, R. M. Real Analysis and Probability, 2nd ed. Cambridge University
Press, New York (2002).

[5] Khan, M. A., Rath, K. P. On Games with Incomplete Information and the
Dvoretsky-Wald-Wolfowitz Theorem with Countable Partitions. J. Math. Econ. 45,
830-837 (2009).

[6] Khan, M. A., Rath, K. P., Sun, Y. The Dvoretzky-Wald-Wolfowitz Theorem and
Purification in Atomless Finite-Action Games. Int. J. Game Theory 34, 91-104
(2006).

[7] Khan, M. A., Zhang, Y. On the Existence of Pure-Strategy Equilibria in
Games with Private Information: A Complete Characterization. J. Math. Econ.
50, 197-202 (2014).

[8] Milgrom, P. R., Weber, R. J. Distributional Strategies for Games with
Incomplete Information. Math. Oper. Res. 10, 619-632 (1985).

[9] Nash, J. F. Equilibrium Points in N-person Games. Proc. Nat. Acad. Sci.
USA 36, 48-49 (1950).

[10] von Neumann, J., Morgenstern, O. Theory of Games and Economic Behavior.
Princeton University Press, Princeton (1944).

[11] Parthasarathy, K. R. Probability an Mathematical Statistics. Academic
Press, New York (2014).

[12] Radner, R., Rosenthal, R. W. Private Information and Pure-Strategy
Equilibria. Math. Oper. Res. 7, 401-409 (1982).

[13] Rudin, W. Real and Complex Analysis, 3rd ed. McGraw-Hill, Singapore
(1987).

[14] Srivastava, S. M. A Course on Borel Sets. Springer-Verlag, New York
(1988).

[15] Yu, C., Hosoya, Y., Maruyama, T. On the Purification of Mixed Strategies.
Econ. Bull. 38, 1655-1675 (2018).