# Tuning Rate of Strategy Revision in Population Games

Shinkyu Park This work was supported by funding from King Abdullah University
of Science and Technology (KAUST).Park is with the Electrical and Computer
Engineering, King Abdullah University of Science and Technology (KAUST),
Thuwal 23955, Saudi Arabia<EMAIL_ADDRESS>
###### Abstract

We investigate a multi-agent decision problem in population games where each
agent in a population makes a decision on strategy selection and revision to
engage in repeated games with others. The strategy revision is subject to time
delays which represent the time it takes for an agent revising its strategy
needs to spend before it can adopt a new strategy and return back to the game.
We discuss the effect of the time delays on long-term behavior of the agents’
strategy revision. In particular, when the time delays are large, the strategy
revision would exhibit oscillation and the agents spend substantial time in
“transitioning” between different strategies, which prevents the agents from
attaining the Nash equilibrium of the game. As a main contribution of the
paper, we propose an algorithm that tunes the rate of the agents’ strategy
revision and show such tuning approach ensures convergence to the Nash
equilibrium. We validate our analytical results using simulations.

##  I Introduction

Consider a multi-agent decision problem where each agent selects a strategy to
engage in repeated interactions with other agents. Agents receive payoffs as a
function of their strategy profile – the distribution of their strategy
selection – and revise their strategy selection based on the payoffs. Such
problem setting is prevalent in many engineering applications [1, 2, 3, 4, 5,
6, 7, 8, 9, 10] ranging from demand response in smart grids to task allocation
in multi-robot systems research.

We adopt the population game formalism [11] to model the process of the
agents’ strategy revision and assess long-term behavior of the revision
process. Different from conventional population game problems, in this work,
we investigate the scenario where the strategy revision is subject to time
delays for which every agent revising its strategy needs to spend a fixed
amount of time for transitioning to a new strategy. As a case in point, in
multi-robot task allocation applications [8, 12, 10] where the strategy
revision corresponds to assignment of a new task to a robot, time delays in
the strategy revision reflect the requirement for each robot to travel to a
distant location to take on a new task.

When the agents’ strategy revision is subject to time delays, as we discuss in
the paper, their strategy profile would exhibit oscillation implying that a
portion of the agent population persistently transitions between different
strategies. In the task allocation applications, this means that the robots
would spend substantial time on traveling between distant locations which is a
disadvantage since during the traveling, they cannot carry out assigned tasks.
The authors of [8, 12] concretely discuss such phenomenon observed in their
linear models, and empirical results presented in [10] illustrate the
oscillation of the strategy profile in population game models.

As a main contribution, we propose an algorithm that tunes the rate of the
agents’ strategy revision to eliminate the oscillation of the strategy
profile, and prove that the algorithm ensures convergence of the strategy
profile to the Nash equilibrium. Of relevance to this work are [13],[14,
Section 5.9] and references therein that investigate the scenarios where
payoff mechanisms underlying population games are subject to time delays and
hence the agents revise their strategy selection based on time-delayed
payoffs. Also, [15] examines a relevant problem in which, when revising its
strategy, each agent needs to take on a sequence of sub-tasks (sub-strategies)
which would potentially cause time delays in the agent’s strategy revision.

Our contributions are distinct from existing work in the following aspects:
(i) unlike [13],[14, Section 5.9] and references therein, our problem
formulation considers that the agents’ strategy revision is subject to time
delays, which is different from the delays in payoff mechanisms, and (ii)
different from [8, 12, 15], our proposed approach ensures convergence of the
strategy profile to the Nash equilibrium, which, in our problem formulation,
requires that no agents are in transition or taking on any sub-strategies.
Below we summarize the main results of the paper.

  * •

We model and analyze the effect of time delays in the strategy revision using
the population game formalism and discuss long-term behavior of the agents’
strategy profile. In particular, our analysis explains how the rate of the
agents’ strategy revision affects the convergence of the strategy profile.

  * •

Based on the analysis, we propose an algorithm for tuning the strategy
revision rate. The algorithm judiciously decreases the revision rate and
ensures the convergence of the strategy profile to the Nash equilibrium in
contractive population games. We illustrate our analytical results using
simulations.

The paper is organized as follows. In Section II, we introduce the population
game framework and strategy revision model we adopt in this study. We also
illustrate how the rate of the agents’ strategy revision affects long-term
behavior of their strategy profile. In Section III, we propose an algorithm
that judiciously tunes the strategy revision rate and show that the proposed
algorithm ensures the convergence of the strategy profile to the Nash
equilibrium in the class of contractive population games.111See Definition 1
for its formal definition. We illustrate our analytical results using
simulations with a numerical example in Section IV. We conclude the paper with
discussions and future plans in Section V.

#### Notation

We denote by $\mathbb{R}^{n}$ the set of $n$-dimensional real vectors and by
$\mathbb{R}_{+}^{n}$ the set of (element-wise) non-negative $n$-dimensional
real vectors. Throughout the paper, we adopt the Euclidean norm for matrices
and vectors.

##  II Problem Description

We describe the foundation of the population game formalism [11] and explain
how we adopt it in our multi-agent decision problem to study the effect of
time delays in strategy revision.

###  II-A Population Games and Evolutionary Dynamics

Consider that a population of decision-making agents are engaged in repeated
strategic interactions with one another. Each agent has a finite set
$\\{1,\cdots,n\\}$ of strategies available and can select one strategy at a
time. Based on its strategy selection, the agent receives a payoff determined
by a payoff mechanism underlying the interactions.

Adopting the conventional notation and formalism, we denote the state of the
population by $x=(x_{1},\cdots,x_{n})\in\mathbb{R}_{+}^{n}$, where $x_{i}$
represents the portion of the population selecting strategy $i$ and $x$ sums
up to $1$, i.e., $\sum_{i=1}^{n}x_{i}=1$. A payoff vector
$p=(p_{1},\cdots,p_{n})\in\mathbb{R}^{n}$ is determined by a continuous
function $\mathcal{F}$ as $p=\mathcal{F}(x)$, where $p_{i}$ is the payoff
assigned to the agents selecting strategy $i$. Assuming that there are a large
number of agents in the population, we can define the set of viable population
states as
$\mathbb{X}\\!=\\!\\{z\in\mathbb{R}_{+}^{n}|\sum_{i=1}^{n}z_{i}\\!=\\!1\\}$.

Contractive population games are defined as follows.

######  Definition 1 (Contractive Population Game [16])

A population game $\mathcal{F}$ is called contractive if it holds that

$\displaystyle(w-z)^{T}(\mathcal{F}(w)-\mathcal{F}(z))\leq 0,~{}\forall
w,z\in\mathbb{X}.$ (1)

###### Assumption 1

We assume that the differential map $D\mathcal{F}$ of $\mathcal{F}$ exists and
is bounded: there is a positive constant $B_{D\mathcal{F}}$ satisfying
$\|D\mathcal{F}(z)\|_{2}\leq B_{D\mathcal{F}},~{}\forall z\in\mathbb{X}$.

Note that when $D\mathcal{F}$ exists, the requirement (1) can be cast as

$\displaystyle\tilde{z}^{T}D\mathcal{F}(z)\tilde{z}\leq 0,~{}\forall
z\in\mathbb{X},\tilde{z}\in T\mathbb{X},$ (2)

where $T\mathbb{X}$ is the tangent space of $\mathbb{X}$.

The Nash equilibrium of $\mathcal{F}$ is defined as follows.

######  Definition 2 (Nash Equilibrium)

An element $z^{\footnotesize\text{NE}}$ in $\mathbb{X}$ is called the Nash
equilibrium of a population game $\mathcal{F}$ if it holds that

$\displaystyle(z^{\footnotesize\text{NE}}-z)^{T}\mathcal{F}(z^{\footnotesize\text{NE}})\geq
0,~{}\forall z\in\mathbb{X}.$ (3)

A population game $\mathcal{F}$ can have multiple Nash equilibria. We denote
by $\mathbb{NE}(\mathcal{F})$ the set of Nash equilibria. We adopt the
following example to illustrate our main results.222For a simple presentation
of the paper, we use the RPS game for the illustration purpose.

###### Example 1

Consider the Rock-Paper-Scissors (RPS) game ($n=3$) whose payoff function is
given by

$\displaystyle\mathcal{F}(x)=\begin{pmatrix}-ax_{2}+bx_{3}\\\ bx_{1}-ax_{3}\\\
-ax_{1}+bx_{2}\end{pmatrix},$ (4)

where $a,b$ are positive constants. Note that (4) is a contractive game if
$b\geq a$, and the Nash equilibrium of $\mathcal{F}$ is given by
$(1/3,1/3,1/3)$ for any $a,b>0$.

In our study, we consider that the agents are repeatedly engaged in a same
game and given an opportunity, they can revise their strategy selection based
on the Smith revision protocol, originally presented in [9] and defined as
follows.333Although the analysis we present in this paper can be extended to
other class of strategy revision protocols. However, for a concise
presentation of our main result, we adopt the Smith protocol.

$\displaystyle\rho_{ij}(p)=\varrho[p_{j}-p_{i}]_{+}:=\begin{cases}\varrho(p_{j}-p_{i})&\text{if
}p_{j}\geq p_{i}\\\ 0&\text{otherwise}\end{cases},$ (5)

where $\varrho$ is a positive constant satisfying444Such constant $\varrho$
exists since $\mathcal{F}$ is a continuous function and $\mathbb{X}$ is a
compact subset of $\mathbb{R}^{n}$.

$\displaystyle\textstyle\sum_{j=1}^{n}\varrho[p_{j}-p_{i}]_{+}\leq
1\text{~{}~{}and~{}~{}}\varrho[p_{j}-p_{i}]_{+}\leq 1.$ (6)

According to (5), each agent switches its strategy selection $i$ to $j$ with
probability $\varrho[p_{j}-p_{i}]_{+}$; otherwise, it stays with its current
strategy selection $i$ with probability
$1-\sum_{j=1}^{n}\varrho[p_{j}-p_{i}]_{+}$. Hence, higher the payoff
associated with strategy $j$ more likely the agent selects it. The condition
(6) is required for such probabilistic strategy revision scheme to be well-
defined.

The point in time at which the agents’ strategy revision takes place is
determined by identical and independent Poisson processes with parameter
$\lambda$. In particular, each agent can revise its strategy selection at each
jump time of an associated Poisson process. Since the parameter $\lambda$
determines the rate of the strategy revision, we refer to it as the strategy
revision rate.

Let $x(t)=(x_{1}(t),\cdots,x_{n}(t))$ be the population state at time instant
$t$ driven by the revision protocol (5), the Poisson processes, and payoffs
determined by a population game $\mathcal{F}$. By same discussions presented
in [11, Chapter 10], as the population size tends to infinity, the solution to
the following state equation approximates the population state $x(t)$ with
arbitrary accuracy.

$\displaystyle\dot{x}_{i}(t)$ $\displaystyle=\mathcal{V}_{i}(x(t),p(t))$
$\displaystyle:=\lambda\big{(}\textstyle\sum_{j=1}^{n}x_{j}(t)\varrho\left[p_{i}(t)-p_{j}(t)\right]_{+}$
$\displaystyle\qquad\qquad\qquad-
x_{i}(t)\textstyle\sum_{j=1}^{n}\varrho\left[p_{j}(t)-p_{i}(t)\right]_{+}\big{)}$
(7)

Adopting the same naming convention as in [17], we refer to (II-A) as
Evolutionary Dynamics Model (EDM).

###  II-B $\delta$-Passivity Tools for Convergence Analysis

A focal research theme in population games is in establishing convergence of
the population state $x(t)$, governed by (II-A), to the Nash equilibrium set
$\mathbb{NE}(\mathcal{F})$ of an underlying population game $\mathcal{F}$. An
earlier work [18] in economics literature uses Lyapunov stability theorems to
establish the convergence for a class of EDMs in contractive population games.
Later, the work of [19] introduces the notion of $\delta$-passivity,
originated from dynamical system theory [20], in population games and provides
general tools for studying convergence properties of EDMs. Since then, there
have been refined definitions and applications of $\delta$-passivity proposed
for convergence analysis in population games [17, 21, 22]. Aside from
$\delta$-passivity tools relevant to our work, there are other important works
that present a different notion of passivity as a tool to study convergence in
multi-agent games [23, 24].

We adopt the following definition of $\delta$-passivity for (II-A) from [17].

###### Definition 3

EDM (II-A) is $\delta$-passive if there is a continuously differentiable
function $\mathcal{S}:\mathbb{X}\times\mathbb{R}^{n}\to\mathbb{R}_{+}$ for
which

$\mathcal{S}(x(t),p(t))-\mathcal{S}(x(t_{0}),p(t_{0}))\leq\\\
\int_{t_{0}}^{t}\dot{x}^{T}(\tau)\dot{p}(\tau)\,\mathrm{d}\tau,~{}\forall
t\geq t_{0}\geq 0$ (8)

holds for every payoff vector trajectory $p(t),~{}t\geq 0$. We refer to
$\mathcal{S}$ as the $\delta$-storage function.

According to [19, 17], EDM (II-A), which is constructed using the Smith
revision protocol (5), is $\delta$-passive and its $\delta$-storage function
is given by

$\displaystyle\mathcal{S}(x,p)=\frac{\lambda}{2}\textstyle\sum_{i=1}^{n}\textstyle\sum_{j=1}^{n}x_{i}\varrho[p_{j}-p_{i}]_{+}^{2}.$
(9)

We make the following two observations which are used to establish the main
result of the paper: for all $i,j$ in $\\{1,\cdots,n\\}$, it holds that

$\displaystyle\lambda^{-1}\mathcal{S}(x,p)\geq\frac{1}{2}x_{i}\varrho[p_{j}-p_{i}]_{+}^{2}\geq\frac{1}{2}x_{i}^{2}\varrho[p_{j}-p_{i}]_{+}^{2}\geq
0$

from which we conclude that

$\displaystyle\lambda^{-1}\mathcal{S}(x,p)\to 0\implies
x_{i}\varrho[p_{j}-p_{i}]_{+}\to 0.$ (10)

Also, by similar arguments used in the proof of Theorem 4.5 in [19], we can
verify that

$\displaystyle\nabla_{x}^{T}\mathcal{S}(x,p)\mathcal{V}(x,p)\leq 0,~{}\forall
x\in\mathbb{X},p\in\mathbb{R}^{n}.$ (11)

###  II-C Analyzing the Effect of Time Delays in Strategy Revision

We consider that the agents’ strategy revision is subject to time delays: (i)
an agent revising its strategy is required to spend a constant time period
$d_{ji}$ to conclude its strategy revision from $j$ to $i$, and (ii) during
the strategy revision, the agent cannot participate in the game. As a case in
point, in the task allocation applications, the robots that are in transition
to distant locations to take on newly assigned tasks cannot carry out the
tasks until they arrive at the new locations. To model this, we interpret
$x_{i}(t)$ as the portion of the population that are engaged in the game with
strategy selection $i$ at time $t$, which exclude the agents transitioning to
strategy $i$. Accordingly, the payoff vector $p(t)=\mathcal{F}(x(t))$ depends
only on the agents who are participating in the game.

Note that in this problem setting, the population state $x(t)$ may not sum up
to $1$, i.e., $\sum_{i=1}^{n}x(t)\leq 1$. For this reason, we extend the
definition of the population state space as

$\displaystyle\mathbb{X}^{\footnotesize\text{ext}}=\left\\{z\in\mathbb{R}_{+}^{n}\,\big{|}\,\textstyle\sum_{i=1}^{n}z_{i}\leq
1\right\\}.$ (12)

Accordingly, we extend the domains of the payoff function
$\mathcal{F}:\mathbb{X}^{\footnotesize\text{ext}}\to\mathbb{R}^{n}$, the
vector field
$\mathcal{V}_{i}:\mathbb{X}^{\footnotesize\text{ext}}\times\mathbb{R}^{n}\to\mathbb{R}$,
and the $\delta$-storage function
$\mathcal{S}:\mathbb{X}^{\footnotesize\text{ext}}\times\mathbb{R}^{n}\to\mathbb{R}_{+}$.
Also, we restate the boundedness condition of the map $D\mathcal{F}$ in
Assumption 1 as $\max_{z\in\mathbb{X}^{\text{ext}}}\|D\mathcal{F}(z)\|_{2}\leq
B_{D\mathcal{F}}$.

Denote by $y_{ji}(t)$ the portion of the population that are in transition for
the strategy revision from $j$ to $i$ at time $t$. Given a fixed strategy
revision rate $\lambda$, the following state equation describes how
$y_{ji}(t)$ changes over time.

$\displaystyle\dot{y}_{ji}(t)=\lambda
x_{j}(t)\varrho\left[p_{i}(t)-p_{j}(t)\right]_{+},~{}d_{ji}>t\geq 0$ (13a)
$\displaystyle\dot{y}_{ji}(t)=\lambda\big{(}x_{j}(t)\varrho\left[p_{i}(t)-p_{j}(t)\right]_{+}$
$\displaystyle-\\!x_{j}(t\\!-\\!d_{ji})\varrho\left[p_{i}(t\\!-\\!d_{ji})\\!-\\!p_{j}(t\\!-\\!d_{ji})\right]_{+}\big{)},~{}t\geq
d_{ji}$ (13b)

with $y_{ji}(0)=0$. Note that in (13b), the first term
$x_{j}(t)\varrho[p_{i}(t)\\!-\\!p_{j}(t)]_{+}$ denotes the rate at which the
agents decide to revise their strategy selection from $j$ to $i$ and initiates
transition from $j$ to $i$ at time $t$. Whereas the second term
$x_{j}(t\\!-\\!d_{ji})\varrho\left[p_{i}(t\\!-\\!d_{ji})\\!-\\!p_{j}(t\\!-\\!d_{ji})\right]_{+}$
describes the rate at which the agents complete the transition and return back
to the game after spending the time period $d_{ji}$ in the transition.

(a) Population state $x(t)$ (b) Population in transition $y(t)$

Figure 1: Graphs depicting the trajectories of the (a) population state $x(t)$
and (b) population in transition $y(t)$ derived by EDM (15)
($d_{ij}=|i-j|,\varrho=1/4,\lambda=1$) in the RPS game (4) ($a=1,b=2$).

(a) Population state $x(t)$ (b) Population in transition $y(t)$

Figure 2: Graphs depicting the trajectories of the (a) population state $x(t)$
and (b) population in transition $y(t)$ derived by EDM (15)
($d_{ij}=|i-j|,\varrho=1/4,\lambda=0.5$) in the RPS game (4) ($a=1,b=2$).

Let us define $y_{i}(t)=\sum_{j=1}^{n}y_{ji}(t)$ which denotes the portion of
the population transitioning to strategy $i$. Then, $x_{i}(t)+y_{i}(t)$
quantifies the portion of the population revising their strategies to $i$
including the agents in transition. Using (II-A), we can describe the rate of
change of $x_{i}(t)+y_{i}(t)$ by the following state equation.

$\displaystyle\dot{x}_{i}(t)+\dot{y}_{i}(t)=\mathcal{V}_{i}(x(t),p(t)),$ (14)

where the vector field $\mathcal{V}_{i}$ is defined in (II-A). Hence, from
(13) and (14), we can derive the state equation for $x_{i}(t)$ as

$\dot{x}_{i}(t)=\lambda\big{(}\textstyle\sum_{j=1}^{n}x_{j}(t\\!-\\!d_{ji})\varrho\left[p_{i}(t\\!-\\!d_{ji})-p_{j}(t\\!-\\!d_{ji})\right]_{+}\\\
-x_{i}(t)\textstyle\sum_{j=1}^{n}\varrho\left[p_{j}(t)-p_{i}(t)\right]_{+}\big{)},~{}t\geq
d_{\text{max}},$ (15)

where $d_{\text{max}}=\max_{i,j\in\\{1,\cdots,n\\}}d_{ji}$. Note that
different from EDM (II-A) in its original form, when the agents’ strategy
revision is subject to time delays, the revised model (15) depends not only on
the current information $x(t),p(t)$ but also on the past information
${x(t-d_{ji}),p(t-d_{ji})}$ about the population game.

Figs. 1 and 2 depict the population state trajectories of (15) in the RPS
game, explained in Example 1, with two different choices of the strategy
revision rate ($\lambda=0.5,1$). We observe that in both cases, the
trajectories oscillate around the Nash equilibrium, but with smaller
$\lambda$, both the level of the oscillation and the portion of the population
in transition decrease. Inspired by the observation, we investigate how the
strategy revision rate can be tuned to achieve the convergence of the
population state to the Nash equilibrium set.

We summarize the main problem of this paper as follows.

###### Problem 1

Design an algorithm to tune the strategy revision rate $\lambda$ of (15) and
to achieve the convergence of the population state $x(t)$ to the Nash
equilibrium set $\mathbb{NE}(\mathcal{F})$ of an underlying contractive game
$\mathcal{F}$:

$\displaystyle\lim_{t\to\infty}\inf_{z\in\mathbb{NE}(\mathcal{F})}\left\|x(t)-z\right\|_{2}=0.$
(16)

Note that according to Definition 2, since every Nash equilibrium
$z^{\footnotesize\text{NE}}$ in $\mathbb{NE}(\mathcal{F})$ belongs to
$\mathbb{X}$, (16) requires the portion of the population in transition to
vanish, i.e.,

$\displaystyle\textstyle\lim_{t\to\infty}\textstyle\sum_{i=1}^{n}y_{i}(t)=1-\lim_{t\to\infty}\textstyle\sum_{i=1}^{n}x_{i}(t)=0.$

In this work, we focus on the case where the strategy revision rate $\lambda$
is homogeneous for all the transition between strategies. Our analysis is
rooted in the notion of $\delta$-passivity for the Smith EDM (II-A). As
illustrated in [25], adopting heterogeneous strategy revision rates would not
ensure the $\delta$-passivity of (II-A) and hence our proposed solution may
not guarantee the convergence to the Nash equilibrium set. We leave as a
future plan to further investigate such heterogeneous rate case.

##  III Tuning Strategy Revision Rate

We describe an algorithm that iteratively updates $\lambda$ and in Section
III-A, we explain how a sequence of the revision rates generated by the
algorithm ensures the convergence of the population state to the Nash
equilibrium set.

To explain the algorithm, we need the following lemma. The proof is given in
Appendix of [26].

###### Lemma 1

Given a fixed strategy revision rate $\lambda$, the states $x(t)$ and $y(t)$
satisfy

$\displaystyle\left\|\dot{x}(t)\right\|_{2}\leq
N\lambda,\quad\left\|\dot{y}(t)\right\|_{2}\leq M\lambda^{2},~{}\forall t\geq
d_{\text{max}},$ (17)

where $d_{\text{max}}=\max_{i,j\in\\{1,\cdots,n\\}}d_{ji}$ and $N,M$ are
positive constants.

The inequalities in (17) imply that as $\lambda$ becomes smaller, i.e., the
agents are less frequently revising their strategies, the rate of change of
$y(t)$ decreases faster than that of $x(t)$. Therefore, the implication of
Lemma 1 is that $\lambda$ can be used as a dial to reduce the number of agents
in transition while allowing them to have flexibility in switching between
strategies.

In what follows, we discuss tuning of $\lambda$. Let us define
$\bar{y}(t)=\lambda^{-2}y(t)$,
$\bar{\mathcal{V}}(x(t),p(t))=\lambda^{-1}\mathcal{V}(x(t),p(t))$, and
$\bar{\mathcal{S}}(x(t),p(t))=\lambda^{-1}\mathcal{S}(x(t),p(t))$. Using Lemma
1, we derive the rate of change of $\bar{\mathcal{S}}$ along a solution to
(15) as follows.

$\displaystyle\frac{\mathrm{d}}{\mathrm{d}t}\bar{\mathcal{S}}\left(x(t),p(t)\right)$
$\displaystyle=\lambda\Big{(}\bar{\mathcal{V}}^{T}(x(t),p(t))D\mathcal{F}(x(t))\bar{\mathcal{V}}(x(t),p(t))$
$\displaystyle\qquad\qquad+\nabla_{x}^{T}\bar{\mathcal{S}}(x(t),p(t))\bar{\mathcal{V}}(x(t),p(t))$
$\displaystyle\qquad\qquad-\lambda\dot{\bar{y}}^{T}(t)\big{(}D\mathcal{F}^{T}(x(t))\bar{\mathcal{V}}(x(t),p(t))$
$\displaystyle\qquad\qquad\qquad\qquad\qquad+\nabla_{x}\bar{\mathcal{S}}(x(t),p(t))\big{)}\Big{)}$
$\displaystyle\leq\lambda\Big{(}\nabla_{x}^{T}\bar{\mathcal{S}}(x(t),p(t))\bar{\mathcal{V}}(x(t),p(t))$
$\displaystyle\qquad\qquad+\lambda
M\big{(}B_{D\mathcal{F}}\left\|\bar{\mathcal{V}}(x(t),p(t))\right\|_{2}$
$\displaystyle\qquad\qquad\qquad\qquad+\left\|\nabla_{x}\bar{\mathcal{S}}(x(t),p(t))\right\|_{2}\big{)}\Big{)},$
(18)

where $p(t)=\mathcal{F}(x(t))$ and we use (2), (14), and the fact that
$\nabla_{p}\mathcal{S}(x,p)=\mathcal{V}(x,p)$, derived in [27, Theorem III.3].
For notational convenience, we adopt

$\displaystyle
f(x,p)=M\left(B_{D\mathcal{F}}\left\|\bar{\mathcal{V}}(x,p)\right\|_{2}+\left\|\nabla_{x}\bar{\mathcal{S}}(x,p)\right\|_{2}\right).$
(19)

Note that $f(x,p)$ does not depend on the revision rate $\lambda$.

Given an initial revision rate $\lambda=\lambda_{0}$ at $t=0$, the following
algorithm is used to tune $\lambda$ at each discrete time $t_{k}$ and to
generate a sequence $\\{\lambda_{k}\\}_{k=1}^{\infty}$.

Algorithm $1$:  Suppose the strategy revision rate was updated to
$\lambda_{k}$ at time $t_{k}$. At time instant $t\geq t_{k}+2d_{\text{max}}$,
where $d_{\text{max}}=\max_{i,j\in\\{1,\cdots,n\\}}d_{ji}$, update the
revision rate as

$\displaystyle\lambda=-\frac{1}{2}\frac{\nabla_{x}^{T}\bar{\mathcal{S}}(x(t),p(t))\bar{\mathcal{V}}(x(t),p(t))}{f(x(t),p(t))}$
(20)

if the following two conditions hold:

  1. (C1.

$f(x(t),p(t))>0$

  2. (C2.

$\nabla_{x}^{T}\bar{\mathcal{S}}(x(t),p(t))\bar{\mathcal{V}}(x(t),p(t))+\lambda_{k}\frac{f(x(t),p(t))}{1-\delta}\geq
0$

where $\delta$ is a constant satisfying $0<\delta<1/2$. Then, set $t_{k+1}=t$
and $\lambda_{k+1}=\lambda$.

Note that according to ((C1) and ((C2), the update rule (20) finds $\lambda$
that minimizes the upper bound of
$\frac{\mathrm{d}}{\mathrm{d}t}\bar{\mathcal{S}}(x(t),p(t))$ given as in the
last inequality of (III).

######  Remark 1 (Decreasing strategy revision rates)

Note that if we select $\lambda$ according to (20), by ((C2), it satisfies

$\displaystyle\lambda\\!=\\!-\frac{1}{2}\frac{\nabla_{x}^{T}\bar{\mathcal{S}}(x(t),p(t))\bar{\mathcal{V}}(x(t),p(t))}{f(x(t),p(t))}\\!\leq\\!\frac{1}{2}\frac{\lambda_{k}}{1-\delta}\\!<\\!\lambda_{k},$
(21)

where the last inequality holds since $0<\delta<1/2$. Therefore, Algorithm 1
generates a decreasing sequence $\\{\lambda_{k}\\}_{k=1}^{\infty}$ of the
revision rates satisfying $\lim_{k\to\infty}\lambda_{k}=0$. ∎

######  Remark 2 (Comment on the constraint $t\geq t_{k}+2d_{\text{max}}$)

Under Algorithm 1, since $\lambda$ switches over time, the parameter $\lambda$
becomes time-dependent and the state equation (13) for $y_{ji}(t)$ can be
rewritten as

$\dot{y}_{ji}(t)=\lambda(t)x_{j}(t)\varrho\left[p_{i}(t)-p_{j}(t)\right]_{+}\\\
-\lambda(t\\!-\\!d_{ji})x_{j}(t\\!-\\!d_{ji})\varrho\left[p_{i}(t\\!-\\!d_{ji})\\!-\\!p_{j}(t\\!-\\!d_{ji})\right]_{+}$
(22)

for $t\geq d_{ji}$, where $\lambda(t)$ is the revision rate at time $t$
defined as $\lambda(t)=\lambda_{k}$ if $t\in[t_{k},t_{k+1})$. With the new
representation (22), the derivation of similar inequalities as in (17), which
are key in establishing our main result, would be technically difficult. To
circumvent such difficulty, we require the update time $t$ to satisfy $t\geq
t_{k}+2d_{\text{max}}$ so that $\lambda(t)$ and $\lambda(t-d_{ji})$ in (22)
satisfy $\lambda(t)=\lambda(t-d_{ji})=\lambda_{k}$ for
$t\in[t_{k}+2d_{\text{max}},t_{k+1})$. ∎

######  Remark 3 (Existence of $t$ satisfying ((C1) and ((C2))

There may be no time instant $t\geq t_{k}+2d_{\text{max}}$ satisfying ((C1)
and ((C2) and no further update on $\lambda$, which we consider as the
termination of the algorithm. In what follows, we show that the population
state converges to the Nash equilibrium set even if the algorithm terminates
after a finite number of $\lambda$-updates.

#### Case I

Suppose $f(x(t),p(t))=0,~{}\forall t\geq t_{k}+2d_{\text{max}}$. Then, from
(11) and (III), it holds that

$\displaystyle\frac{\mathrm{d}}{\mathrm{d}t}\bar{\mathcal{S}}\left(x(t),p(t)\right)\leq\lambda_{k}\nabla_{x}^{T}\bar{\mathcal{S}}(x(t),p(t))\bar{\mathcal{V}}(x(t),p(t))\leq
0.$

Noting that

$\nabla_{x}^{T}\bar{\mathcal{S}}(x(t),p(t))\bar{\mathcal{V}}(x(t),p(t))=0\\\
\iff\bar{\mathcal{S}}(x(t),p(t))=0,$

we conclude $\lim_{t\to\infty}\bar{\mathcal{S}}(x(t),p(t))=0$.

#### Case II

Suppose the following inequality holds:

$\nabla_{x}^{T}\bar{\mathcal{S}}(x(t),p(t))\bar{\mathcal{V}}(x(t),p(t))+\lambda_{k}\frac{f(x(t),p(t))}{1-\delta}<0,\\\
~{}\forall t\geq t_{k}+2d_{\text{max}}.$ (23)

Then, from (11) and (III), it holds that

$\displaystyle\frac{\mathrm{d}}{\mathrm{d}t}\bar{\mathcal{S}}\left(x(t),p(t)\right)\leq\delta\lambda_{k}\nabla_{x}^{T}\bar{\mathcal{S}}(x(t),p(t))\bar{\mathcal{V}}(x(t),p(t))\leq
0.$

By the same argument as in (Case I), we have
$\lim_{t\to\infty}\bar{\mathcal{S}}(x(t),p(t))=0$.

For both (Case I) and (Case II), by (10), it holds that
$\lim_{t\to\infty}x_{i}(t)\varrho[p_{j}(t)-p_{i}(t)]_{+}=0,~{}\forall
i,j\in\\{1,\cdots,n\\}$. Then, using Lemma 3 stated in Section III-A, we
conclude
$\lim_{t\to\infty}\inf_{z\in\mathbb{NE}(\mathcal{F})}\|x(t)-z\|_{2}=0$. ∎

###  III-A Convergence Analysis

Recall that by Remark 3, even if Algorithm 1 terminates after a finite number
of $\lambda$-updates, the population state still converges to the Nash
equilibrium set. In this section, we consider the case where Algorithm 1
generates an infinite sequence $\\{\lambda_{k}\\}_{k=1}^{\infty}$ of strategy
revision rates. The following theorem states the main result.

###### Theorem 1

Let $\\{\lambda_{k}\\}_{k=1}^{\infty}$ be a decreasing sequence of strategy
revision rates determined by Algorithm 1. The population state converges to
the Nash equilibrium set, i.e., it holds that

$\displaystyle\lim_{t\to\infty}\inf_{z\in\mathbb{NE}(\mathcal{F})}\left\|x(t)-z\right\|_{2}=0.$
(24)

Using the following two lemmas, we prove the theorem by showing that the
$\delta$-storage function $\mathcal{S}$ decreases at each time instant $t_{k}$
of the $\lambda$-update (Lemma 2) which implies that the population state
converges to the Nash equilibrium set $\mathbb{NE}(\mathcal{F})$ (Lemma 3). We
provide the proofs of the theorem and lemmas in Appendix of [26].

###### Lemma 2

Given $\mathcal{V}=(\mathcal{V}_{1},\cdots,\mathcal{V}_{n})$ and $\mathcal{S}$
as in (II-A) and (9), respectively, define
$\bar{\mathcal{V}}=\lambda^{-1}\mathcal{V}$ and
$\bar{\mathcal{S}}=\lambda^{-1}\mathcal{S}$. Let $K$ be a positive constant
satisfying $K>(1-\delta)^{-1}f(x,p)$ for all $x$ in
$\mathbb{X}^{\footnotesize\text{ext}}$ and $p=\mathcal{F}(x)$, where $\delta$
and $f(x,p)$ are defined in Algorithm 1. There is a decreasing function
$\epsilon:\mathbb{R}_{+}\to\mathbb{R}_{+}$ satisfying

$\displaystyle\nabla_{x}^{T}\bar{\mathcal{S}}(x,p)\bar{\mathcal{V}}(x,p)+\lambda
K>0\implies\bar{\mathcal{S}}(x,p)<\epsilon(\lambda)$ (25)

and $\lim_{\lambda\to 0}\epsilon(\lambda)=0$.

###### Lemma 3

The following statement is true.

$\lim_{t\to\infty}x_{i}(t)\varrho[p_{j}(t)-p_{i}(t)]_{+}=0,~{}\forall
i,j\in\\{1,\cdots,n\\}\\\
\implies\lim_{t\to\infty}\inf_{z\in\mathbb{NE}(\mathcal{F})}\left\|x(t)-z\right\|_{2}=0.$
(26)

##  IV Simulations

(a) Population state $x(t)$ (b) Population in transition $y(t)$

Figure 3: Graphs illustrating the trajectories of the (a) population state
$x(t)$ and (b) population in transition $y(t)$ derived by EDM (15)
($d_{ij}=|i-j|,\varrho=1/4$) in the RPS game (4) ($a=1,b=2$).

Using Example 1, we illustrate our convergence result (Theorem 1) through
simulations. To this end, we adopt the payoff function (4) with $a=1,b=2$ as
follows.

$\displaystyle\mathcal{F}(x)=\begin{pmatrix}-x_{2}+2x_{3}\\\ 2x_{1}-x_{3}\\\
-x_{1}+2x_{2}\end{pmatrix}$ (27)

Note that $\mathcal{F}$ is contractive satisfying (2) and the upper bound of
$D\mathcal{F}$ can be computed as $B_{D\mathcal{F}}\approx 2.65$.

We use Algorithm 1 to compute the sequence $\\{\lambda_{k}\\}_{k=1}^{\infty}$
for EDM (15) with the initial revision rate $\lambda_{0}=1$. Fig. 3 depicts
the trajectories for both $x(t)$ and $y(t)$. Contrast to the fixed revision
rate cases illustrated in Figs. 1 and 2, through tuning the revision rate
using Algorithm 1, we observe that the population state $x(t)$ converges to
the Nash equilibrium $(1/3,1/3,1/3)$ of (27). Also the portion $y(t)$ of the
population in transition vanishes as the revision rate decreases.

##  V Conclusions

We investigate a multi-agent decision problem in population games where the
strategy revision of the agents is subject to time delays, which prevent the
state of the population from converging to the Nash equilibrium set. We
propose an algorithm that judiciously decreases the rate of the agents’
strategy revision and prove that by tuning the revision rate, the population
state converges to the Nash equilibrium set when the underlying population
game is contractive.

As future directions, we plan to improve the proposed algorithm to attain
faster convergence to the Nash equilibrium set. The current version of the
algorithm is designed to minimize the upper bound of the rate of change of the
$\delta$-passive storage function; however this is mainly used to establish
the convergence result and would not guarantee fast convergence. Also,
validating the proposed approach in engineering applications, where the
population would consist of a finite number of decision-making agents, will be
another direction we plan to explore.

###  -A Proof of Lemma 1

Using (15), we can derive a bound on $\dot{x}_{i}(t)$ as follows.

$\displaystyle\left|\dot{x}_{i}(t)\right|$
$\displaystyle\leq\lambda\Big{(}\textstyle\sum_{j=1}^{n}x_{j}(t-d_{ji})\varrho\left[p_{i}(t-d_{ji})-p_{j}(t-d_{ji})\right]_{+}$
$\displaystyle\qquad\qquad+x_{i}(t)\textstyle\sum_{j=1}^{n}\varrho\left[p_{j}(t)-p_{i}(t)\right]_{+}\Big{)}$
$\displaystyle\leq\lambda(n-1+x_{i}(t))\leq n\lambda,$ (28)

where to establish the second inequality, we use (6) and the fact that

$\displaystyle
x_{j}(t-d_{ji})\varrho\left[p_{i}(t-d_{ji})-p_{j}(t-d_{ji})\right]_{+}$
$\displaystyle\leq 1,~{}j\neq i,$ $\displaystyle
x_{j}(t-d_{ji})\varrho\left[p_{i}(t-d_{ji})-p_{j}(t-d_{ji})\right]_{+}$
$\displaystyle=0,~{}j=i.$

Using (-A), we can establish Lipschitz continuity of $x(t)$: For any $d\geq
0$, it holds that

$\displaystyle\left|x_{j}(t)-x_{j}(t-d)\right|\leq\max_{\tau\in[t-d,t]}\left|\dot{x}_{j}(\tau)\right|d\leq
nd\lambda$ (30)

and

$\displaystyle\left\|x(t)-x(t-d)\right\|_{2}\leq n^{1.5}d\lambda.$ (31)

Consequently, using (31), the revision protocol
$\varrho[p_{i}(t)-p_{j}(t)]_{+}$ satisfies

$\displaystyle\left|\varrho\left[p_{i}(t)-p_{j}(t)\right]_{+}-\varrho\left[p_{i}(t-d_{ji})-p_{j}(t-d_{ji})\right]_{+}\right|$
$\displaystyle\leq\varrho\left|p_{i}(t)-p_{j}(t)-p_{i}(t-d_{ji})+p_{j}(t-d_{ji})\right|$
$\displaystyle\leq\varrho\left|p_{i}(t)-p_{i}(t-d_{ji})\right|+\varrho\left|p_{j}(t)-p_{j}(t-d_{ji})\right|$
$\displaystyle\leq\varrho\left(\left\|D\mathcal{F}_{i}\right\|_{2}+\left\|D\mathcal{F}_{j}\right\|_{2}\right)\left\|x(t)-x(t-d_{ji})\right\|_{2}$
$\displaystyle\leq 2\varrho B_{D\mathcal{F}}n^{1.5}d_{ji}\lambda,$ (32)

where $\mathcal{F}_{i}$ is the $i$-th component of
$\mathcal{F}=(\mathcal{F}_{1},\cdots,\mathcal{F}_{n})$,
$\left\|D\mathcal{F}_{i}\right\|_{2}=\max_{x\in\mathbb{X}^{\text{ext}}}\left\|D\mathcal{F}_{i}(x)\right\|_{2}\leq
B_{D\mathcal{F}}$, and we use

$\displaystyle\left|p_{i}(t)-p_{i}(t-d_{ji})\right|$
$\displaystyle=\left|\mathcal{F}_{i}(x(t))-\mathcal{F}_{i}(x(t-d_{ji}))\right|$
$\displaystyle\leq\|D\mathcal{F}_{i}\|_{2}\left\|x(t)-x(t-d_{ji})\right\|_{2}$
$\displaystyle\leq B_{D\mathcal{F}}n^{1.5}d_{ji}\lambda.$ (33)

Therefore, using (30) and (-A), we derive a bound on $\dot{y}(t)$ as follows.

$\displaystyle\left|\dot{y}_{i}(t)\right|$
$\displaystyle\leq\lambda\sum_{j=1}^{n}\Big{|}x_{j}(t)\varrho\left[p_{i}(t)-p_{j}(t)\right]_{+}$
$\displaystyle\qquad\qquad-
x_{j}(t-d_{ji})\varrho\left[p_{i}(t-d_{ji})-p_{j}(t-d_{ji})\right]_{+}\Big{|}$
$\displaystyle\leq\lambda\sum_{j=1}^{n}\Big{|}x_{j}(t)\varrho\left[p_{i}(t)-p_{j}(t)\right]_{+}$
$\displaystyle\qquad\qquad-
x_{j}(t-d_{ji})\varrho\left[p_{i}(t)-p_{j}(t)\right]_{+}\Big{|}$
$\displaystyle\quad+\lambda\sum_{j=1}^{n}\Big{|}x_{j}(t-d_{ji})\varrho\left[p_{i}(t)-p_{j}(t)\right]_{+}$
$\displaystyle\quad\qquad\qquad-
x_{j}(t-d_{ij})\varrho\left[p_{i}(t-d_{ji})-p_{j}(t-d_{ji})\right]_{+}\Big{|}$
$\displaystyle\leq\lambda\sum_{j=1}^{n}nd_{ji}\lambda\varrho\left[p_{i}(t)-p_{j}(t)\right]_{+}+\lambda\sum_{j=1}^{n}2\varrho
B_{D\mathcal{F}}n^{1.5}d_{ji}\lambda$ $\displaystyle\leq(1+2\varrho
B_{D\mathcal{F}}n^{0.5})nd_{i}\lambda^{2}$ (34)

where $d_{i}=\sum_{j=1}^{n}d_{ji}$. Hence, from (-A) and (-A), we conclude
that

$\displaystyle\left\|\dot{x}(t)\right\|_{2}\leq
N\lambda,\quad\left\|\dot{y}(t)\right\|_{2}\leq M\lambda^{2}$

where $N=n^{1.5}$ and $M=(1+2\varrho
B_{D\mathcal{F}}n^{0.5})n\sqrt{\sum_{i=1}^{n}d_{i}^{2}}$. This completes the
proof. ∎

###  -B Proof of Lemma 2

We first show that there is $K$ such that $K>(1-\delta)^{-1}f(x,p)$ holds for
all $x$ in $\mathbb{X}^{\footnotesize\text{ext}}$ and $p=\mathcal{F}(x)$,
where $f(x,p)$ is defined in (19). Note that using (6), we have

$\left|\bar{\mathcal{V}}_{i}(x,p)\right|=\Big{|}\textstyle\sum_{j=1}^{n}x_{j}\varrho\left[p_{i}-p_{j}\right]_{+}\\\
-x_{i}\textstyle\sum_{j=1}^{n}\varrho\left[p_{j}-p_{i}\right]_{+}\Big{|}\leq
1$ (35)

and

$\displaystyle\left|\frac{\partial\bar{\mathcal{S}}}{\partial
x_{k}}(x,p)\right|=\frac{1}{2}\sum_{j=1}^{n}\varrho[p_{j}-p_{k}]_{+}^{2}\leq\frac{1}{2\varrho}.$
(36)

Therefore, using the definition of $f(x,p)$ and (35), (36), we can conclude

$\displaystyle(1-\delta)^{-1}f(x,p)$
$\displaystyle=(1-\delta)^{-1}M\left(B_{D\mathcal{F}}\left\|\bar{\mathcal{V}}(x,p)\right\|_{2}+\left\|\nabla_{x}\bar{\mathcal{S}}(x,p)\right\|_{2}\right)$
$\displaystyle\leq(1-\delta)^{-1}M\left(B_{D\mathcal{F}}+\frac{1}{2\varrho}\right)n^{0.5}<K$
(37)

if we choose
$K>(1-\delta)^{-1}M\left(B_{D\mathcal{F}}+\frac{1}{2\varrho}\right)n^{0.5}$.

To conclude the proof, we prove the contrapositive of (25):

$\displaystyle\bar{\mathcal{S}}(x,p)\geq\epsilon(\lambda)\implies\nabla_{x}^{T}\bar{\mathcal{S}}(x,p)\bar{\mathcal{V}}(x,p)+K\lambda\leq
0$ (38)

Suppose $\bar{\mathcal{S}}(x,p)\geq\epsilon$ then according to (9), it holds
that

$\displaystyle\max_{1\leq i,j\leq
n}\frac{\varrho}{2}x_{i}[p_{j}-p_{i}]_{+}^{2}\geq\frac{\epsilon}{n^{2}}.$ (39)

Otherwise $\bar{\mathcal{S}}(x,p)<\epsilon$ which is a contradiction. Let
$i^{\ast},j^{\ast}$ be indices for which the following holds.

$\displaystyle\max_{1\leq i,j\leq
n}\frac{\varrho}{2}x_{i}[p_{j}-p_{i}]_{+}^{2}$
$\displaystyle=\frac{\varrho}{2}x_{i^{\ast}}[p_{j^{\ast}}-p_{i^{\ast}}]_{+}^{2}$
(40)

from which we obtain

$\displaystyle\frac{\epsilon}{n^{2}}\leq\max_{1\leq i,j\leq
n}\frac{\varrho}{2}x_{i}[p_{j}-p_{i}]_{+}^{2}$
$\displaystyle=\frac{\varrho}{2}x_{i^{\ast}}[p_{j^{\ast}}-p_{i^{\ast}}]_{+}^{2}$
$\displaystyle\leq\frac{1}{2}[p_{j^{\ast}}-p_{i^{\ast}}]_{+},$ (41)

where we use (6) to establish the last inequality. Therefore, we have that

$\displaystyle x_{i^{\ast}}[p_{j^{\ast}}-p_{i^{\ast}}]_{+}^{2}$
$\displaystyle\geq\frac{2\epsilon}{\varrho n^{2}}$ (42a)
$\displaystyle[p_{j^{\ast}}-p_{i^{\ast}}]_{+}$
$\displaystyle\geq\frac{2\epsilon}{n^{2}}.$ (42b)

Also we can derive

$\displaystyle\nabla_{x}^{T}\bar{\mathcal{S}}(x,p)\bar{\mathcal{V}}(x,p)$
$\displaystyle=\sum_{i=1}^{n}\sum_{j=1}^{n}\big{(}x_{j}\varrho\left[p_{i}\\!-\\!p_{j}\right]_{+}\\!-\\!x_{i}\varrho\left[p_{j}\\!-\\!p_{i}\right]_{+}\big{)}\frac{\varrho}{2}\sum_{k=1}^{n}[p_{k}\\!-\\!p_{i}]_{+}^{2}$
$\displaystyle=\frac{\varrho^{2}}{2}\sum_{i=1}^{n}\sum_{j=1}^{n}\Big{(}x_{j}\left[p_{i}\\!-\\!p_{j}\right]_{+}\sum_{k=1}^{n}\left([p_{k}\\!-\\!p_{i}]_{+}^{2}\\!-\\![p_{k}\\!-\\!p_{j}]_{+}^{2}\right)\Big{)}$
$\displaystyle\leq\frac{\varrho^{2}}{2}\Big{(}x_{j}\left[p_{i}-p_{j}\right]_{+}\left([p_{k}-p_{i}]_{+}^{2}-[p_{k}-p_{j}]_{+}^{2}\right)\Big{)}$
(43)

for any $i,j,k\in\\{1,\cdots,n\\}$. If we choose $i=k=j^{\ast},j=i^{\ast}$,
using (42), we can derive

$\displaystyle\nabla_{x}^{T}\bar{\mathcal{S}}(x,p)\bar{\mathcal{V}}(x,p)$
$\displaystyle\leq-\frac{\varrho^{2}}{2}x_{i^{\ast}}\left[p_{j^{\ast}}-p_{i^{\ast}}\right]_{+}\left[p_{j^{\ast}}-p_{i^{\ast}}\right]_{+}^{2}$
$\displaystyle\leq-\frac{2\epsilon^{2}\varrho}{n^{4}}$ (44)

Let us assign $\epsilon(\lambda)=\sqrt{\frac{n^{4}K\lambda}{2\varrho}}$, then
we can conclude

$\displaystyle\nabla_{x}^{T}\bar{\mathcal{S}}(x,p)\bar{\mathcal{V}}(x,p)+K\lambda\leq
0$

and also we can verify that $\epsilon(\lambda)\to 0$ as $\lambda\to 0$. ∎

###  -C Proof of Lemma 3

Suppose $y_{ji}(0)=0,~{}\forall i,j\in\\{1,\cdots,n\\}$, i.e., there is no
agent in transition for the strategy revision at the beginning of the game.
Let $t_{2}$ be the time instant that Algorithm 1 updates the revision rate for
the second time. For $t\geq t_{2}$, a solution $y_{ji}(t)$ to (22) satisfies

$\displaystyle y_{ji}(t)$
$\displaystyle=\int_{0}^{t}\big{(}\lambda(\tau)x_{j}(\tau)\varrho\left[p_{i}(\tau)-p_{j}(\tau)\right]_{+}$
$\displaystyle\qquad\\!-\\!\lambda(\tau\\!-\\!d_{ji})x_{j}(\tau\\!-\\!d_{ji})\varrho\left[p_{i}(\tau\\!-\\!d_{ji})\\!-\\!p_{j}(\tau\\!-\\!d_{ji})\right]_{+}\big{)}\mathrm{d}\tau$
$\displaystyle=\int_{t-d_{ji}}^{t}\lambda(\tau)x_{j}(\tau)\varrho\left[p_{i}(\tau)-p_{j}(\tau)\right]_{+}\,\mathrm{d}\tau$
$\displaystyle\qquad\\!-\\!\int_{-d_{ji}}^{0}\lambda(\tau)x_{j}(\tau)\varrho\left[p_{i}(\tau)-p_{j}(\tau)\right]_{+}\,\mathrm{d}\tau$
$\displaystyle\leq\lambda_{1}\int_{t-d_{ji}}^{t}x_{j}(\tau)\varrho\left[p_{i}(\tau)-p_{j}(\tau)\right]_{+}\,\mathrm{d}\tau,$
(45)

where we use the facts that

$\displaystyle\int_{-d_{ji}}^{0}\lambda(\tau)x_{j}(\tau)\varrho\left[p_{i}(\tau)-p_{j}(\tau)\right]_{+}\,\mathrm{d}\tau\geq
0,$

$\lambda(\tau)=\lambda_{k}\text{ if }\tau\in[t_{k},t_{k+1})$ with $t_{k+1}\geq
t_{k}+2d_{\text{max}}$, and $\\{\lambda_{k}\\}_{k=1}^{\infty}$ is a decreasing
sequence.

Suppose

$\displaystyle\lim_{t\to\infty}x_{i}(t)\varrho[p_{j}(t)-p_{i}(t)]_{+}=0,~{}\forall
i,j\in\\{1,\cdots,n\\}$ (46)

holds as in the statement of the lemma. Then, from (-C), we have that

$\displaystyle\lim_{t\to\infty}y_{ji}(t)=0,~{}\forall i,j\in\\{1,\cdots,n\\}.$
(47)

To complete the proof, using (46) and (47), we establish
$\lim_{t\to\infty}\inf_{z\in\mathbb{NE}(\mathcal{F})}\|x(t)-z\|_{2}=0$. By
contradiction, suppose that there is a converging sequence
$\\{x(t_{k})\\}_{k=1}^{\infty}$ for which its limit point $x^{\ast}$ is not
contained in $\mathbb{NE}(\mathcal{F})$. By (46) and (47), it holds that
$\sum_{i=1}^{n}x_{i}^{\ast}=1$ and

$\displaystyle x_{i}^{\ast}>0\implies\mathcal{F}_{i}(x^{\ast})=\max_{1\leq
j\leq n}\mathcal{F}_{j}(x^{\ast}),~{}\forall i\in\\{1,\cdots,n\\}.$

Hence, for any $z$ in $\mathbb{X}$, we have that

$\displaystyle\left(x^{\ast}-z\right)^{T}\mathcal{F}(x^{\ast})=\max_{1\leq
j\leq n}\mathcal{F}_{j}(x^{\ast})-z^{T}\mathcal{F}(x^{\ast})\geq 0.$

According to Definition 2, we conclude that $x^{\ast}$ is the Nash
equilibrium, which is a contradiction. This completes the proof. ∎

###  -D Proof of Theorem 1

Note that by ((C2) of Algorithm 1, at each update time $t_{k+1}$, it holds
that

$\displaystyle\nabla_{x}^{T}\bar{\mathcal{S}}(x(t_{k+1}),p(t_{k+1}))\bar{\mathcal{V}}(x(t_{k+1}),p(t_{k+1}))+\lambda_{k}K$
$\displaystyle>\nabla_{x}^{T}\bar{\mathcal{S}}(x(t_{k+1}),p(t_{k+1}))\bar{\mathcal{V}}(x(t_{k+1}),p(t_{k+1}))$
$\displaystyle\qquad+\lambda_{k}f(x(t_{k+1}),p(t_{k+1}))\geq 0,$

where $K$ is a constant satisfying $K>(1-\delta)^{-1}f(x,p)\geq f(x,p)$ as in
the statement of Lemma 2. Invoking Lemma 2, there is a decreasing function
$\epsilon:\mathbb{R}_{+}\to\mathbb{R}_{+}$ satisfying
$\bar{\mathcal{S}}(x(t_{k+1}),p(t_{k+1}))<\epsilon(\lambda_{k})$. In what
follows, we show that there is a function
$\bar{\epsilon}:\mathbb{R}_{+}\to\mathbb{R}_{+}$ satisfying

$\displaystyle\bar{\mathcal{S}}(x(t),p(t))<\bar{\epsilon}(\lambda_{k}),~{}\forall
t\in[t_{k+1},t_{k+2})$ (48)

for every positive integer $k$ and
$\lim_{k\to\infty}\bar{\epsilon}(\lambda_{k})=0$.

For $t\in[t_{k+1},t_{k+1}+2d_{\text{max}})$, it holds that

$\displaystyle\bar{\mathcal{S}}(x(t),p(t))-\bar{\mathcal{S}}(x(t_{k+1}),p(t_{k+1}))$
$\displaystyle\leq\max_{x\in\mathbb{X}^{\footnotesize\text{ext}}}\left\|\nabla_{x}\bar{\mathcal{S}}(x,p)|_{p=\mathcal{F}(x)}\right\|_{2}\left\|x(t)-x(t_{k+1})\right\|_{2}$
$\displaystyle\qquad+B_{D\mathcal{F}}\max_{x\in\mathbb{X}^{\footnotesize\text{ext}}}\left\|\nabla_{p}\bar{\mathcal{S}}(x,p)|_{p=\mathcal{F}(x)}\right\|_{2}\left\|x(t)-x(t_{k+1})\right\|_{2}$
$\displaystyle\leq
2d_{\text{max}}L\max_{\tau\in[t_{k+1},t_{k+1}+2d_{\text{max}})}\left\|\dot{x}(\tau)\right\|_{2},$
(49)

where $L$ is a constant satisfying555Note that such constant $L$ exists since
$\mathcal{S}$ is continuously differentiable, $\mathcal{F}$ is continuous, and
$\mathbb{X}^{\footnotesize\text{ext}}$ is compact.

$L=\max_{x\in\mathbb{X}^{\footnotesize\text{ext}}}\left\|\nabla_{x}\bar{\mathcal{S}}(x,p)|_{p=\mathcal{F}(x)}\right\|_{2}\\\
+B_{D\mathcal{F}}\max_{x\in\mathbb{X}^{\footnotesize\text{ext}}}\left\|\nabla_{p}\bar{\mathcal{S}}(x,p)|_{p=\mathcal{F}(x)}\right\|_{2}.$

Note that when the revision rate is repeatedly updated by Algorithm 1,
$\lambda$ becomes a time-dependent parameter and the state equation (13) for
$y_{ji}(t)$ can be rewritten as in (22). Hence, using (14) and (22), it holds
that for $t\in[t_{k+1},t_{k+2})$,

$\displaystyle\left|\dot{x}_{i}(t)\right|$
$\displaystyle\leq\textstyle\sum_{j=1}^{n}\lambda(t\\!-\\!d_{ji})x_{j}(t\\!-\\!d_{ji})\varrho\left[p_{i}(t\\!-\\!d_{ji})\\!-\\!p_{j}(t\\!-\\!d_{ji})\right]_{+}$
$\displaystyle\qquad+\textstyle\sum_{j=1}^{n}\lambda(t)x_{i}(t)\varrho\left[p_{j}(t)-p_{i}(t)\right]_{+}$
$\displaystyle\leq\textstyle\sum_{\begin{subarray}{c}j=1\\\ j\neq
i\end{subarray}}^{n}\lambda(t-d_{ji})+\lambda_{k}$ $\displaystyle\leq
n\lambda_{k},$ (50)

where we use (6), and the facts that $\\{\lambda_{k}\\}_{k=1}^{\infty}$ is a
decreasing sequence and $t_{k+1}\geq t_{k}+2d_{\text{max}}$. Consequently,
from (-D) and (-D), we can derive

$\bar{\mathcal{S}}(x(t),p(t))\leq\bar{\mathcal{S}}(x(t_{k+1}),p(t_{k+1}))+2d_{\text{max}}Ln^{1.5}\lambda_{k},\\\
~{}\forall t\in[t_{k+1},t_{k+1}+2d_{\text{max}}).$ (51)

On the other hand, for $t\in[t_{k+1}+2d_{\text{max}},t_{k+2})$, at least one
of ((C1) and ((C2) of Algorithm 1 is violated and using (11) and (III), we can
derive

$\frac{\mathrm{d}}{\mathrm{d}t}\bar{\mathcal{S}}(x(t),p(t))\leq\lambda_{k+1}\big{(}\nabla_{x}^{T}\bar{\mathcal{S}}(x(t),p(t))\bar{\mathcal{V}}(x(t),p(t))\\\
+\lambda_{k+1}f(x(t),p(t))\big{)}\leq 0.$ (52)

Hence, the function $\bar{\mathcal{S}}(x(t),p(t))$ does not increase over
$[t_{k+1}+2d_{\text{max}},t_{k+2})$. Therefore, we conclude that for all
$t\in[t_{k+1},t_{k+2})$,

$\displaystyle\bar{\mathcal{S}}(x(t),p(t))$
$\displaystyle\leq\bar{\mathcal{S}}(x(t_{k+1}),p(t_{k+1}))+2d_{\text{max}}Ln^{1.5}\lambda_{k}$
$\displaystyle<\epsilon(\lambda_{k})+2d_{\text{max}}Ln^{1.5}\lambda_{k}=\bar{\epsilon}(\lambda_{k}).$
(53)

Since $\lim_{k\to\infty}\lambda_{k}=0$, it holds that
$\lim_{k\to\infty}\bar{\epsilon}(\lambda_{k})=0$ and we have that
$\lim_{t\to\infty}\bar{\mathcal{S}}(x(t),p(t))=0$. By (10), it holds that
$\lim_{t\to\infty}x_{i}(t)\varrho[p_{j}(t)-p_{i}(t)]_{+}=0,~{}\forall
i,j\in\\{1,\cdots,n\\}$, and using Lemma 3, we conclude that
$\lim_{t\to\infty}\inf_{z\in\mathbb{NE}(\mathcal{F})}\|x(t)-z\|_{2}=0$. This
completes the proof. ∎

## References

  * [1] D. Liu, A. Hafid, and L. Khoukhi, “Population game based energy and time aware task offloading for large amounts of competing users,” in _2018 IEEE Global Comm. Conf. (GLOBECOM)_ , 2018, pp. 1–6.
  * [2] S. Moon, H. Kim, and Y. Yi, “Brute: Energy-efficient user association in cellular networks from population game perspective,” _IEEE Trans. Wireless Communications_ , vol. 15, no. 1, pp. 663–675, 2016.
  * [3] D. Lee and D. Kundur, “An evolutionary game approach to predict demand response from real-time pricing,” in _2015 IEEE Electrical Power and Energy Conference (EPEC)_ , 2015, pp. 197–202.
  * [4] H. Tembine, E. Altman, R. El-Azouzi, and Y. Hayel, “Evolutionary games in wireless networks,” _IEEE Trans. Systems, Man, and Cybernetics, Part B (Cybernetics)_ , vol. 40, no. 3, pp. 634–646, 2010.
  * [5] P. Srikantha and D. Kundur, “Resilient distributed real-time demand response via population games,” _IEEE Transactions on Smart Grid_ , vol. 8, no. 6, pp. 2532–2543, 2017.
  * [6] A. Pashaie, L. Pavel, and C. J. Damaren, “A population game approach for dynamic resource allocation problems,” _International Journal of Control_ , vol. 90, no. 9, pp. 1957–1972, 2017.
  * [7] N. Quijano, C. Ocampo-Martinez, J. Barreiro-Gomez, G. Obando, A. Pantoja, and E. Mojica-Nava, “The role of population games and evolutionary dynamics in distributed control systems: The advantages of evolutionary game theory,” _IEEE Control Systems Magazine_ , vol. 37, no. 1, pp. 70–97, 2017.
  * [8] S. Berman, A. Halasz, M. A. Hsieh, and V. Kumar, “Optimized stochastic policies for task allocation in swarms of robots,” _IEEE Transactions on Robotics_ , vol. 25, no. 4, pp. 927–937, 2009.
  * [9] M. J. Smith, “The stability of a dynamic model of traffic assignment—an application of a method of Lyapunov,” _Transportation Science_ , vol. 18, no. 3, pp. 245–252, 1984.
  * [10] S. Park, Y. D. Zhong, and N. E. Leonard, “Multi-robot task allocation games in dynamically changing environments,” in _2021 IEEE International Conference on Robotics and Automation (ICRA)_ , 2021, pp. 8678–8684.
  * [11] W. H. Sandholm, _Population Games and Evolutionary Dynamics_. MIT Press, 2011.
  * [12] T. W. Mather and M. A. Hsieh, “Macroscopic modeling of stochastic deployment policies with time delays for robot ensembles,” _The International Journal of Robotics Research_ , vol. 30, no. 5, pp. 590–600, 2011.
  * [13] S. Park and N. E. Leonard, “KL divergence regularized learning model for multi-agent decision making,” in _2021 American Control Conference (ACC)_ , 2021, pp. 4509–4514.
  * [14] M. J. Fox, “Distributed learning in large populations,” Ph.D. dissertation, Georgia Institute of Technology, Aug. 2012.
  * [15] S. Kara, N. C. Martins, and M. Arcak, “Population games with erlang clocks: Convergence to nash equilibria for pairwise comparison dynamics,” in _2022 IEEE 61st Conference on Decision and Control (CDC)_ , 2022, pp. 7688–7695.
  * [16] W. H. Sandholm, “Chapter 13: Population games and deterministic evolutionary dynamics,” ser. Handbook of Game Theory with Economic Applications, H. P. Young and S. Zamir, Eds. Elsevier, 2015, vol. 4, pp. 703–778.
  * [17] S. Park, N. C. Martins, and J. S. Shamma, “From population games to payoff dynamics models: A passivity-based approach,” in _2019 IEEE 58th Conference on Decision and Control (CDC)_ , 2019, pp. 6584–6601.
  * [18] J. Hofbauer and W. H. Sandholm, “Stable games and their dynamics,” _Journal of Economic Theory_ , vol. 144, no. 4, pp. 1665–1693.e4, 2009.
  * [19] M. J. Fox and J. S. Shamma, “Population games, stable games, and passivity,” _Games_ , vol. 4, pp. 561–583, Oct. 2013.
  * [20] J. C. Willems, “Dissipative dynamical systems part I: General theory,” _Arch. Ration. Mech. Anal._ , vol. 45, no. 5, pp. 321–351, Jan. 1972.
  * [21] S. Park, N. C. Martins, and J. S. Shamma, “Payoff dynamics model and evolutionary dynamics model: Feedback and convergence to equilibria (arxiv:1903.02018),” arXiv.org, March 2019.
  * [22] M. Arcak and N. C. Martins, “Dissipativity tools for convergence to Nash equilibria in population games,” _IEEE Transactions on Control of Network Systems_ , vol. 8, no. 1, pp. 39–50, 2021.
  * [23] B. Gao and L. Pavel, “On passivity, reinforcement learning and higher-order learning in multi-agent finite games,” _IEEE Transactions on Automatic Control_ , pp. 1–1, 2020.
  * [24] L. Pavel, “Dissipativity theory in game theory: On the role of dissipativity and passivity in Nash equilibrium seeking,” _IEEE Control Systems Magazine_ , vol. 42, no. 3, pp. 150–164, 2022.
  * [25] S. Kara and N. C. Martins, “Pairwise comparison evolutionary dynamics with strategy-dependent revision rates: Stability and $\delta$-passivity,” _IEEE Transactions on Control of Network Systems_ , pp. 1–12, 2023.
  * [26] S. Park, “Tuning rate of strategy revision in population games,” 2022. [Online]. Available: https://arxiv.org/abs/2210.05472
  * [27] S. Park, J. S. Shamma, and N. C. Martins, “Passivity and evolutionary game dynamics,” in _2018 IEEE Conference on Decision and Control (CDC)_ , 2018, pp. 3553–3560.


# Distilling Multi-Level X-vector Knowledge for Small-footprint Speaker
Verification

Xuechen Liu  Md Sahidullah  Tomi Kinnunen  National Institute of Informatics,
2-1-2 Hitotsubashi, 101-8430 Tokyo, Japan  University of Eastern Finland,
Länsikatu 15, FI-80110 Joensuu, Finland  Institute for Advancing Intelligence,
TCG CREST, 700091 Kolkata, India

###### Abstract

Even though deep speaker models have demonstrated impressive accuracy in
speaker verification tasks, this often comes at the expense of increased model
size and computation time, presenting challenges for deployment in resource-
constrained environments. Our research focuses on addressing this limitation
through the development of small-footprint deep speaker embedding extraction
using knowledge distillation. While previous work in this domain has
concentrated on speaker embedding extraction at the utterance level, our
approach involves amalgamating embeddings from different levels of the
x-vector model (teacher network) to train a compact student network. The
results highlight the significance of frame-level information, with the
student models exhibiting a remarkable size reduction of 85%-91% compared to
their teacher counterparts, depending on the size of the teacher embeddings.
Notably, by concatenating teacher embeddings, we achieve student networks that
maintain comparable performance to the teacher while enjoying a substantial
75% reduction in model size. These findings and insights extend to other
x-vector variants, underscoring the broad applicability of our approach.

###### keywords:

Speaker recognition , Small foot-print model , Knowledge distillation

††journal: Nuclear Physics B

##  1 Introduction

_Automatic speaker verification_ (ASV) [1] aims at identifying individuals
using their voices. Advancing upon traditional, shallow statistical
comparators [2], _Gaussian mixture models_ (GMM) [3, 4] and _i-vectors_ [5,
6], _deep neural networks_ (DNNs) have emerged as the modern ASV solution [1].
Currently, DNNs are the predominant choice for representing speakers using
_speaker embeddings_ — fixed-dimensional vectors containing speaker-specific
traits.

Meanwhile, proliferating applications of speech processing and biometric
algorithms onto embedded devices [7, 8, 9] with constrained local
computational resources have emerged. Representative examples are smart home
speakers such as Amazon Alexa111https://developer.amazon.com/en-US/alexa and
Google Home222https://home.google.com/, and assistance devices such as
portable voice translator333https://global.iflytek.com/jarvisen/. DNNs have
various constraints to their practical deployment on them, including run-time
(in)efficiency, power consumption, and memory usage due to a large number of
parameters [10]. This is a major shortcoming when the embedded device has
limited memory space or needs to operate with weak or no online access, ruling
out cloud-based solutions. Therefore, lightweight on-device models are
preferable — though this usually comes with a trade-off in recognition
accuracy. Reducing the performance gap between the small and large DNN models
is an important, yet challenging task in speech processing [11, 12, 13].

Therefore, in this paper, we aim at reducing this gap to achieve acceptable
trade-off between the model size and performance. In particular, we propose a
simple and practical approach for creating a small ‘student’ DNN from a larger
‘teacher’ counterpart through _knowledge distillation_ (KD). We review the
relevant related work on small footprint ASV and KD in Sections 2 and 3,
respectively. While our focus is on ASV using _de facto_ off-the-shelf speaker
encoders, the proposed distillation framework can readily be extended to other
networks and tasks (as will be discussed in Section 5.2).

While KD itself is not new to speech processing tasks [14, 15, 16], the
practical implementation details are crucial for achieving desired model
size–accuracy trade-off for a given model and task. We put forward a novel
perspective on _multi-level_ speaker representation suited for small footprint
ASV. Given that the multiple layers in a DNN represent different types of
features—ranging from low-level short-term spectral descriptors to abstract
utterance-level latent speaker traits—a key question concerns the usefulness
of the information represented by the different layers of the teacher network.
While the common approach is to extract a single, utterance-level speaker
embedding, we hypothesize that embeddings extracted from _all_ the layers
could help in reducing the performance gap of the teacher and the student
networks. After formulating our approach in Section 4, we seek to confirm the
above hypothesis experimentally in Section 5. To this end, we investigate the
effect of embedding-level composite (using simple concatenation) and extend
the proposed methods to more advanced x-vector variants including the ones
described in [17, 18, 19]. To the best of our knowledge, this is the first
work that reveals the efficacy of an ‘extended’ set of multi-level speaker
embeddings for developing practical, small footprint ASV systems.

Figure 1: The TS learning framework used for this study. The pre-trained
teacher speaker encoder reads training speech waveforms and outputs speaker
embeddings of them at the first stage (up, divided by the thick dash lines).
The training utterances are then fed into the student speaker encoder and the
learning proceeds by comparing the student prediction and the frame-wise
copied speaker embeddings at the student training stage (middle). At the run-
time stage (bottom), the frame-level enrollment and test embeddings are
extracted and averaged to single embeddings, for decision making and
evaluation.

##  2 Small Footprint Speaker Verification

Modern neural networks for ASV are typically trained on thousands of speakers
and comprise millions of learnable parameters. Despite impressive performance
on common ASV evaluation corpora, the size and complexity of these models pose
deployment challenges on embedded devices with limited memory and
computational power. This challenge is highlighted not only for training but
also, more importantly, during inference, which makes modern ASV systems be
sometimes accessible only through cloud-based services with strong hardware
backbones. In environments with weak or non-existent internet connection, a
preference may arise for more lightweight variants of these systems, thereby
alleviating the constraints on processing power and memory resources.
Meanwhile, there has been a consistently-observed trade-off between model
complexity and performance [20, 13]. Therefore, it becomes important to ensure
that down-scaled models maintain performance levels comparable to their larger
counterparts.

Solutions to this problem can be divided into two broad categories. The first
one, _model down-scaling_ , includes training more compact networks [21, 22],
or quantizing the model [23]. For instance, [24] proposed parameter
binarization for ASV models and [25] reduced the size of self-supervised
speech models via parameter sharing. While achieving promising performance in
particular tasks, these approaches may demand extensive engineering effort and
subtle learning schemes such as model quantization and parameter tuning. The
second category, _knowledge distillation_ (KD), also known as _teacher-
student_ (TS) learning [10, 16, 26], transfers the knowledge from a _teacher_
network (usually a pre-trained DNN) to a new, _student_ network. Compared to
the first category, KD possesses task-specific design of student network and
knowledge [13, 27] and tends to require less hyperparameter tuning —
especially when a pre-trained teacher model is readily available (e.g.
[18]444https://huggingface.co/speechbrain/spkrec-ecapa-voxceleb,
[19]555https://github.com/yuyq96/D-TDNN). Nevertheless, producing effective
knowledge for student learning remains challenging [16], especially when the
student network is expected to be significantly smaller than the teacher.

Having been convinced on the overall positive prospects and simplicity of the
generic KD framework (stemming partly from earlier hands-on experience of the
first author [28]), our work seeks to develop a novel, small footprint ASV
model using KD. When DNNs were initially used for speaker embedding
extraction, an open question was how the embedding should be extracted [29] —
in particular, _which DNN layer(s) would be most useful for speaker
characterization_. Models such as an x-vector with _time-delay neural network_
(TDNN) use utterance-level (referred to as ‘segment-level’ in [30]) speaker
embeddings, which contains abstract high-level latent speaker information.
Meanwhile, the frame-level information at earlier layers can still be useful
and its successful application as bottleneck features has been studied [31,
32], which motivates part of the methodology of this work.

There has been two major approaches towards producing the small footprint
neural networks from their large-scale counterparts, which are detailed below.

###  2.1 Model Quantization

A pragmatic approach to reduce the size of a large DNN to deployable sizes
involves quantizing the model itself. This option is not only practical but
also reasonable, given the redundancies inherent in DNNs during training,
including ineffective connections and void parameters. Recent efforts,
including those focused on state-of-the-art DNN speaker encoders [18], have
delved into visualizing receptive feature maps from various layers to
understand their functions.

One strategy to quantize the network is to prune the parameters. Noteworthy
contributions in this domain, as exemplified by [33, 34], entail the removal
of weights and associated connections through either elementary pruning
followed by subsequent quantization and Huffman encoding, or the application
of weight clustering into discrete groups represented by centroids. The latter
methodology, through the transformation of centroid values, facilitates
resource-efficient weight sharing while concurrently mitigating network
sparsity. In the realm of _convolutional neural networks_ (CNN), the
implementation of pruning across specific dimensions within the convolutional
layers has been explored [35]. Concurrently, studies have been conducted to
investigate error minimization in this context [36]. A recent, promising
approach involves binary (1-bit) quantization of weights [37], applied to
speaker embedding extraction through conventional _time-delayed neural
networks_ (TDNNs) [38]. Alternatively, eliminating redundant parameters can be
done via _low-rank parameter factorization_ [39] achieved by estimating the
importance of weight parameters through matrix decomposition techniques.

Additionally, enhancing the processing efficiency of convolutional filter
kernels to decrease computational complexity is another viable option. This
can be achieved through factorizing the convolution kernel to two separate,
cascaded kernels, with significant size reduction [40, 41]. Another strategy
involves splitting the filters across different channels, followed by
shuffling and re-grouping, a methodology illustrated in works such as [42,
43].

###  2.2 Knowledge Distillation

When implementing knowledge distillation, a certain level of information is
extracted from a large ensemble of models as pseudo data distribution, which
is then utilized to train a DNN model. The distribution can be generated from
unlabeled data using various methods, including random sampling, mixture
modeling, and designated algorithms [10].

Originally applied to speech processing tasks [14, 15], KD served primarily as
a means to effectively transfer the knowledge of a well-trained network to a
new one—but without necessarily prioritizing model size reduction. For
instance, in [14], KD was employed as a domain adaptation technique, where the
training data for the teacher network constituted the source domain, and the
training data for the student network constituted the target domain. This
approach is further explored in [15], where a semi-supervised objective based
on lattice-free maximum mutual information was proposed. Both studies
concentrate on speech recognition, with the training target being the state
estimates of the _Hidden Markov model_ (HMM), typically designed as the
training target for the acoustic model.

##  3 Knowledge Distillation for Speaker Verification

In this section, we outline our methodology of developing and training a
streamlined DNN speaker encoder. We use a pre-trained DNN model as teacher and
implement Knowledge Distillation (KD) via a simple student network. We begin
by providing details about our pre-trained teacher model and then proceed to
discuss the configuration of our learning framework.

###  3.1 X-Vector

Given its efficacy and transparency among existing _deep neural network_ (DNN)
models [1], we consider the widely recognized vanilla x-vector speaker encoder
[30, 44]. This model is founded on the _time-delay_ neural network (TDNN)
architecture [45], as depicted in Fig. 2. Selected for its efficiency in terms
of model parameters, the vanilla x-vector model comprises five frame-level
TDNN layers, a subsequent pooling layer, two utterance-level fully-connected
layers, and an output layer. This choice allows us to explore the nuances of
minimal model parameter compression using a common student network. Following
the common practice, compared to the original vanilla x-vector, we substitute
statistical pooling with _attentive statistics pooling_ [46], and employ
_additive angular margin softmax_ (AAM softmax, as depicted in Fig. 2) [47] as
the loss function, deviating from the conventional softmax cross-entropy.
While we are aware of various data augmentation techniques [48, 49] and more
advanced models [18, 19] (addressed in section 6), we choose not to factor in
the combination of the influence of data, teacher model complexity, and the
effectiveness of the proposed simple learning framework in the main body of
this study.

###  3.2 The Learning Framework of Knowledge Distillation

Originating in [10] and further explored in [50], knowledge distillation has
recently gathered significant attention. This technique harnesses the robust
representation capabilities of a well-trained, large-scale neural network to
impart knowledge to a smaller counterpart. In speech and speaker recognition,
the teacher network is typically trained using standard objectives like cross-
entropy on a massive training corpus. Subsequently, the student network
assimilates knowledge by extracting the “target distribution data” from the
teacher [13, 51, 27], as illustrated in the upper part of Fig. 1. The primary
objective is to align this distribution closely with the task-specific target
distribution. In the case of ASV, the latter involves either speaker posterior
probabilities or the direct use of speaker embedding vectors. We categorize
these as _posterior-_ and _embedding-level_ information, respectively.

We integrate embedding-level KD, following the design outlined in [51] and
[52], with slight adjustments tailored to our systems. The framework is
illustrated in the lower part of Fig. 1. Informed by preliminary experiments,
we opt for cosine similarity [53] as the designated loss function, defined as
follows:

$\displaystyle\ell_{\mathrm{emb}}(\boldsymbol{x}_{t},\boldsymbol{x}_{s})=-\sum_{i=1}^{N}\frac{\boldsymbol{x}_{t}^{i}\cdot\boldsymbol{x}_{s}^{i}}{\lVert\boldsymbol{x}_{t}^{i}\rVert\lVert\boldsymbol{x}_{t}^{i}\rVert}$
(1)

where $\boldsymbol{x}_{t}^{i}$ and $\boldsymbol{x}_{s}^{i}$ are teacher and
student embeddings from the $i$-th sample, respectively. $N$ denotes the
number of samples. The loss is computed in a per-frame manner and the returned
values are summed and averaged:
$\ell_{\mathrm{frame\\_emb}}=\frac{1}{T}\sum_{t=1}^{T}\ell_{\mathrm{emb}}$,
where $t$ denotes the frame index and $T$ denotes the number of total frames.

Figure 2: The x-vector model and the various types of embedding that are
extracted from its layers. The TDNN and fully connected layers, if not marked,
are all 512-dimensional. ‘AAM softmax’ denotes _additive angular margin
softmax_ [47]. The pooling layer on the left-hand side can be either
statistics pooling or learnable dictionary encoding (LDE).

##  4 Embeddings for Knowledge Distillation

While the use of multiple levels of speaker information [54] and KD [27, 26]
have been addressed in prior work, their combination for small footprint ASV
demands a further investigation.

In this section, we present our methodology by examining different types of
speaker embeddings for training student networks. The speaker embeddings are
derived exclusively from the teacher network, as illustrated in Fig. 2.
Notably, these design choices can be seamlessly transposed onto various other
DNN architectures for ASV [1].

###  4.1 Utterance-level Embeddings

The prevalent type of speaker embedding in deep speaker verification systems
is utterance-level embedding [30, 44]. For the x-vector model, we extract
these embeddings, one per utterance, from the output of either the first or
the second fully-connected layer following the statistics pooling layer. As
indicated by [44] and confirmed by our preliminary experiments, there is no
substantial performance difference between extracting from the first or the
second layer. We opt for the former and term these embeddings as _utterance_
in the experimental section.

To synchronize various levels of information, we adhere to the approach
described in [13] through the following steps: 1) During the training of the
student network, we replicate the speaker embedding $T$ times to generate
frame-level target distribution ($T$ is set to 2 seconds for all experiments);
2) When extracting embeddings from the student network, we compute the average
of the frame-level information across the input utterance.

###  4.2 Frame-level Embeddings

In the x-vector, the layers preceding the statistics pooling step encapsulate
various _frame-level_ information, while the subsequent layers aggregate this
information into an utterance-level representation [30]. Despite the
prevalence and popularity of x-vector variants, where utterance-level
embeddings are the norm, the potential of _frame-level_ information at early
layers has been relatively overlooked. Motivated by earlier investigations
into bottleneck (BN) features [31, 32], we explore the efficacy of _frame-
level_ embeddings as the distribution for student network learning.
Specifically, we investigate bottleneck embeddings extracted from the last two
TDNN layers of the teacher network, with the embeddings obtained by averaging
over frame samples. The two types of embeddings are as follows:

  * 1.

The 4-th TDNN layer. We refer to this type of embedding as _narrowBN_. The
dimension of the vector is 512.

  * 2.

The 5-th TDNN layer, which is the final layer before the attentive statistics
pooling. We refer to this type of embedding as _wideBN_. The dimension of the
vector is 1500.

These types of embeddings share similarities with those from models such as
_d-vector_ [31]. The difference lies in the fact that, in this context, since
the teacher model is an x-vector, during teacher training, utterance-level
knowledge is back-propagated to the frame-level layers.

Additionally, for these frame-level embeddings, we explore the impact of mean
normalization [55]. Specifically, we implement windowed mean normalization
with a window size of $s=300$ frames and observe its normalization effect on
the embeddings.

###  4.3 Aggregated Embeddings

Aiming to further exploit the potential of frame-level representations,
drawing inspiration from [54], we adopt a strategy that involves aggregating
the output from multiple frame-level layers to utterance-level. The resulting
embedding is obtained by taking the equal-weighted average of these aggregated
outputs. Denoting $\boldsymbol{x}_{t}$ as the output teacher embedding, this
approach is expressed as follows:

$\displaystyle\boldsymbol{x}_{t}=\frac{1}{K}\sum_{i=1}^{K}\mathbf{\Phi}(\boldsymbol{x}_{i}),$
(2)

where $\mathbf{\Phi}(.)$ denotes the pooling operator and $\boldsymbol{x}_{i}$
is the output from the $i$-th TDNN layer. $K$ is the number of vectors to be
aggregated, which was set to be 4 here since we aggregate the first 4 TDNN
layers (out of 5 in vanilla x-vector).

We benchmark the efficacy of two variants for $\mathbf{\Phi}(.)$. The first
operator is _statistics pooling_ (SP) [30], wherein the mean and standard
deviation of $\boldsymbol{x_{i}}$ are computed and concatenated. The second
operator is _learnable dictionary encoding_ (LDE) [56], involving the
aggregation of frame-level information from the TDNN layers through a
convolutional and a linear encoding layer. The initialization of related
weight values in LDE is accomplished using a normal distribution [57]. The
corresponding embeddings derived from these operators are denoted as _SP Aggr_
and _LDE Aggr_ , respectively.

##  5 Experimental setup

###  5.1 Data

All experiments are conducted on the VoxCeleb dataset [58, 59], consisting of
two subsets: VoxCeleb1 with 1251 speakers and VoxCeleb2 with 6114 speakers.
The training partition of VoxCeleb1, comprising 1211 speakers, and the _dev_
set of VoxCeleb2, with 5994 speakers, are utilized for training our speaker
encoders and backend classifiers. The performance of all systems is evaluated
on the _VoxCeleb1 test set_ [58], a dataset encompassing 40 speakers, 4874
utterances, and 37720 trials. The acoustic features employed are
40-dimensional Mel filterbank coefficients with a sampling rate of 16 kHz.

###  5.2 Experimental Configuration

Baseline teacher model. For the baseline vanilla x-vector, we follow the
configuration outlined in Section 3.1. This x-vector model is trained using
the training set of VoxCeleb1 and the _dev_ set of VoxCeleb2, encompassing a
total of 7205 speakers.

Student models. All student models employ an 8-layer fully-connected Deep
Neural Network (DNN) without pooling layers or residual connectors. Each
fully-connected layer has a dimensionality of 256, except for the final layer,
the dimension of which is determined by the teacher embeddings. Consequently,
the size of the models ranges from 0.54M to 0.81M, as detailed in Table 1.
Compared to the teacher, the single student models reduce the effective size
of the model from 85% to 91%. The teacher embeddings $\boldsymbol{x}_{t}$ in
Eq. (1) are derived from the x-vector using the aforementioned methods. For
each system, the speaker embeddings undergo centering and mean normalization
before being employed to train a _probabilistic LDA_ (PLDA) classifier [60].
This classifier is then utilized to generate log-likelihood scores for
evaluation. The _dev_ set of VoxCeleb2 is used to train the student models.
Both the teacher and student speaker encoders are implemented using PyTorch
[61], and the other modules of the learning framework are implemented using
Kaldi [62].

Composite student model. We first conduct an embedding-level composition by
concatenating the diverse types of speaker embeddings obtained from the
teacher model. Subsequent steps for training and evaluation mirror those
employed for student models trained on individual types of embeddings. The
reported results encompass scenarios where the frame-level embeddings from the
teacher model are both mean-normalized and not. We note this system as
_Composite_ in result analysis. Note that this composite approach still
results in a single student model, by taking advantage of all the embeddings
aforementioned.

Extension to other networks. We extend the aforementioned from the
conventional TDNN-based model onto other x-vector variants, whose performances
are reportedly better than their predecessor on established datasets. Three
backbone architectures are included in this extension: Extended TDNN [17]
(denoted as _ETDNN_), ECAPA-TDNN [18] (_ECAPA_), and Densely-connected TDNN
[19] (_DTDNN_). All three models share a common structure featuring a final
frame-level TDNN layer before the pooling layer and two fully-connected layers
after. Consequently, we extract _utterance_ embeddings (from the first fully-
connected layer) and _wideBN_ embeddings (from the final TDNN layer) from each
of the three networks. We analyze the performance gap of the corresponding
trained student networks. The training data for the teacher networks aligns
with that used for the baseline TDNN x-vector. The dimension of the embeddings
and the architecture of the corresponding student models remain consistent
with earlier sections, and we utilize the pre-trained versions of the three
aforementioned models.

All training steps including training of the teacher x-vector speaker encoder,
have been carried out on a single NVIDIA TITAN V GPU. The evaluation of
student models have been done without the reliance of GPU nodes.

###  5.3 Performance Measures

We report ASV performance through _equal error rate_ (EER) and _minimum
detection cost function_ (minDCF) with target speaker prior
$p_{\mathrm{tar}}=0.01$ and detection costs
$C_{\text{fa}}=C_{\text{miss}}=1.0$. We also show the _detection error trade-
off_ (DET) curve for DNN-based systems for further analysis.

Figure 3: The DET profiles for the teacher x-vector and students, along with the composite system. The composite system here is without mean normalization. Best viewed in color. Figure 4: The relationship between the deepness of teacher models and student ASV performance. Best viewed in color. Table 1: Results on VoxCeleb1 test set, including composite system results via concatenation of teacher embeddings. Number of parameters of the student model is indicated in the brackets. The baseline (teacher) x-vector has 5.90M parameters, and reached 1.88%/0.152 on EER/minDCF. Embedding | mean norm. | EER(%) | minDCF
---|---|---|---
_utterance_ (0.54M) | — | 4.54 | 0.463
_narrowBN_ (0.54M) | no | 4.29 | 0.464
yes | 4.52 | 0.466
_wideBN_ (0.81M) | no | 4.29 | 0.460
yes | 4.50 | 0.469
_SP Aggr_ (0.68M) | — | 4.17 | 0.407
_LDE Aggr_ (0.54M) | — | 5.86 | 0.534
Composite (1.50M) | no | 2.04 | 0.252
yes | 2.26 | 0.280
Table 2: Comparison with previous research on small footprint speaker verification. The cosine backend corresponds to the cosine scoring scheme [53]. Their teacher ECAPA-TDNNs reached 0.99%/0.113 [63] and 1.14%/0.075 [64] on EER/minDCF, respectively. | Large network | Backend | EER(%) | minDCF
---|---|---|---|---
[63] |  ECAPA-TDNN [18] | Cosine | 2.31 | 0.251
[64] |  ECAPA-TDNN [18] | Cosine | 2.62 | 0.252
Ours |  TDNN [44] | PLDA | 2.04 | 0.252
Table 3: Results on _utterance_ and _wideBN_ embeddings from different speaker
encoders. Number of parameters of the teacher model is in the brackets.
_wideBN_ Embeddings covered here do not apply mean normalization.

Teacher | embedding | EER(%) | minDCF
---|---|---|---
_TDNN_ [44](5.90M)  EER=1.88% | _utterance_ | 4.54 | 0.463
_wideBN_ | 4.29 | 0.460
_ETDNN_ [17](13.2M)  EER=1.67% | _utterance_ | 4.14 | 0.462
_wideBN_ | 4.47 | 0.463
_ECAPA_ [18](6.35M)  EER=1.66% | _utterance_ | 4.77 | 0.484
_wideBN_ | 4.74 | 0.469
_DTDNN_ [19](4.43M)  EER=1.74% | _utterance_ | 3.97 | 0.481
_wideBN_ | 3.57 | 0.455

##  6 Results

Results for the baseline teacher and student systems along with their
corresponding model sizes are presented in Table 1. sizes, are presented in
Table 1. The alternatives proposed in this work demonstrate comparable
performance with utterance-level embeddings, with the exception of _LDE Aggr_.
Interestingly, the two frame-level embeddings outperform others when used
without mean normalization. Notably, _wideBN_ without mean normalization
outperforms _utterance_ , while _narrowBN_ without mean normalization yields
equal EER performance with _wideBN_ , despite their different vector
dimensionalities. Among the single embeddings, _SP Aggr_ achieves the lowest
EER and minDCF, while _LDE Aggr_ results in the highest values in both
categories.

Results of the two composite systems are shown at the bottom of Table 1. Both
trained student networks outperform the single systems and significantly
reduce the performance gap between student and teacher on both metrics, while
still utilizing a relatively 75% gap in terms of model size. This further
capitalizes the usefulness of the enlisted embeddings extracted from the
teacher, containing information that may have been missed by utterance-level
embeddings. Meanwhile, deeper architectures with more resorting effort into
engineering may result in better results, but that will lead to a larger size
of the model.

The comparison with previous works on small footprint models is outlined in
Table 2. Although their final models are smaller, it is important to note that
their teacher models are more robust, and their losses [47] are tailored for
cosine similarity. Additionally, these approaches necessitates joint
optimization of the teacher network, and the design of the student network can
be intricate. In contrast, the proposed system, prioritizing simplicity,
achieves competitive performance on both metrics. Meanwhile, it is worth
noting that these performance figures may benefit from PLDA.

Fig. 3 presents the DET profile of the teacher x-vector and the described
student models, including the two composite systems. While the single student
system do not reach the teacher, the composite system demonstrates comparable
performance.

Results of the experiments on the extension to other networks have been
presented in Table 5.3. The best single system performance is achieved by
_DTDNN_ in terms of both metrics, utilizing its _wideBN_ embeddings. This is
interesting as it has the fewest number of parameters among the models listed.
The “deepness” of the model, indicated by the number of frame-level layers,
thus may positively influence the performance of the two types of embeddings.
Fig. 4 depicts such relationships. While the performance relative to the
number of frame-level layers is not monotonic, _DTDNN_ , with a notably deeper
frame-level architecture than others, outperforms all other teachers. Future
work may study on more effective ways to aggregate frame-level information
from neural networks with deep architectures.

##  7 Conclusion

In this work, we have addressed the development of small footprint ASV systems
through knowledge distillation. We have highlighted the potential of various
types of embeddings extracted from the teacher x-vector network, encompassing
both utterance-level and frame-level speaker information. Our results on the
standard VoxCeleb corpus confirm our hypothesis on the usefulness of multiple
levels of speaker representation in small footprint ASV tasks. The student
systems consistently achieve an 85%-91% reduction in model size compared to
the teacher model. Moreover, the composite student learner via concatenation
of various types of speaker embeddings across multiple layers significantly
outperforms individual models, reaching comparable performance with the
teacher (2.04% vs. 1.88%), while still maintaining a substantial 75% relative
size reduction. We have extended our learning framework to incorporate several
other teacher x-vector networks such as ECAPA-TDNN, revealing the
effectiveness of deeper frame-level architectures and generalizability of our
main findings. Future work may investigate on advanced aggregation methods and
explore task-specific speaker embedding vectors, with the involvement of more
advanced learning schemes, along with extension to stronger and more versatile
teacher learning model.

## References

  * [1] Z. Bai, X. Zhang, Speaker recognition based on deep learning: An overview, Neural Networks 140 (2021) 65–99.  doi:https://doi.org/10.1016/j.neunet.2021.03.004.
URL https://www.sciencedirect.com/science/article/pii/S0893608021000848

  * [2] K. Kim, M. Y. Kim, Robust speaker recognition against background noise in an enhanced multi-condition domain, IEEE Transactions on Consumer Electronics 56 (3) (2010) 1684–1688.  doi:10.1109/TCE.2010.5606313.
  * [3] D. Reynolds, R. Rose, Robust text-independent speaker identification using gaussian mixture speaker models, IEEE Transactions on Speech and Audio Processing 3 (1) (1995) 72–83.  doi:10.1109/89.365379.
  * [4] Y. MUKAI, H. NODA, M. NIIMI, T. OSANAI, Text-independent speaker verification using artificially generated gmms for cohorts, IEICE Transactions on Information and Systems E91.D (10) (2008) 2536–2539.  doi:10.1093/ietisy/e91-d.10.2536.
  * [5] N. Dehak, P. J. Kenny, R. Dehak, P. Dumouchel, P. Ouellet, Front-end factor analysis for speaker verification, IEEE Transactions on Audio, Speech, and Language Processing 19 (4) (2011) 788–798.  doi:10.1109/TASL.2010.2064307.
  * [6] S. YAO, R. ZHOU, P. ZHANG, Speaker-phonetic i-vector modeling for text-dependent speaker verification with random digit strings, IEICE Transactions on Information and Systems E102.D (2) (2019) 346–354.  doi:10.1587/transinf.2018EDP7310.
  * [7] T. Perumal, A. R. Ramli, C. Y. Leong, Design and implementation of soap-based residential management for smart home systems, IEEE Transactions on Consumer Electronics 54 (2) (2008) 453–459.  doi:10.1109/TCE.2008.4560114.
  * [8] R. Chatterjee, S. Mazumdar, R. S. Sherratt, R. Halder, T. Maitra, D. Giri, Real-time speech emotion analysis for smart home assistants, IEEE Transactions on Consumer Electronics 67 (1) (2021) 68–76.  doi:10.1109/TCE.2021.3056421.
  * [9] D.-S. Kim, K.-S. Hong, Multimodal biometric authentication using teeth image and voice in mobile environment, IEEE Transactions on Consumer Electronics 54 (4) (2008) 1790–1797.  doi:10.1109/TCE.2008.4711236.
  * [10] C. Bucila, R. Caruana, A. Niculescu-Mizil, Model compression, KDD ’06, Association for Computing Machinery, New York, NY, USA, 2006, p. 535–541.
  * [11] Z. Gao, Y. Yao, S. Zhang, J. Yang, M. Lei, I. McLoughlin, Extremely Low Footprint End-to-End ASR System for Smart Device, in: Proc. Interspeech, 2021, pp. 4548–4552.  doi:10.21437/Interspeech.2021-819.
  * [12] Y. He, T. N. Sainath, R. Prabhavalkar, I. McGraw, R. Alvarez, D. Zhao, D. Rybach, A. Kannan, Y. Wu, R. Pang, Q. Liang, D. Bhatia, Y. Shangguan, B. Li, G. Pundak, K. C. Sim, T. Bagby, S.-y. Chang, K. Rao, A. Gruenstein, Streaming end-to-end speech recognition for mobile devices, in: Proc. ICASSP, 2019, pp. 6381–6385.  doi:10.1109/ICASSP.2019.8682336.
  * [13] R. W. M. Ng, X. Liu, P. Swietojanski, Teacher-student training for text-independent speaker recognition, in: 2018 IEEE Spoken Language Technology Workshop (SLT), 2018, pp. 1044–1051.  doi:10.1109/SLT.2018.8639564.
  * [14] D. Yu, K. Yao, H. Su, G. Li, F. Seide, Kl-divergence regularized deep neural network adaptation for improved large vocabulary speech recognition, in: Proc. ICASSP, 2013, pp. 7893–7897.  doi:10.1109/ICASSP.2013.6639201.
  * [15] V. Manohar, P. Ghahremani, D. Povey, S. Khudanpur, A teacher-student learning approach for unsupervised domain adaptation of sequence-trained asr models, in: 2018 IEEE Spoken Language Technology Workshop (SLT), 2018, pp. 250–257.  doi:10.1109/SLT.2018.8639635.
  * [16] J. Gou, B. Yu, S. Maybank, D. Tao, Knowledge distillation: A survey, Int J Comput Vis 129 (2021) 1789–1819.
  * [17] D. Snyder, D. Garcia-Romero, G. Sell, A. McCree, D. Povey, S. Khudanpur, Speaker recognition for multi-speaker conversations using x-vectors, in: Proc. ICASSP, 2019, pp. 5796–5800.  doi:10.1109/ICASSP.2019.8683760.
  * [18] B. Desplanques, J. Thienpondt, K. Demuynck, ECAPA-TDNN: Emphasized channel attention, propagation and aggregation in TDNN based speaker verification, in: Proc. Interspeech, 2020, pp. 3830–3834.  doi:10.21437/Interspeech.2020-2650.
  * [19] Y. Yu, W. Li, Densely Connected Time Delay Neural Network for Speaker Verification, in: Proc. Interspeech, 2020, pp. 921–925.  doi:10.21437/Interspeech.2020-1275.
  * [20] S. Koppula, J. Glass, A. P. Chandrakasan, Energy-efficient speaker identification with low-precision networks, in: Proc. ICASSP, 2018, pp. 2246–2250.  doi:10.1109/ICASSP.2018.8462498.
  * [21] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, H. Adam, Mobilenets: Efficient convolutional neural networks for mobile vision applications, ArXiv abs/1704.04861 (2017).
  * [22] M. Tan, Q. Le, EfficientNet: Rethinking model scaling for convolutional neural networks, in: K. Chaudhuri, R. Salakhutdinov (Eds.), Proceedings of the 36th International Conference on Machine Learning, Vol. 97, PMLR, 2019, pp. 6105–6114.
  * [23] J. Wu, C. Leng, Y. Wang, Q. Hu, J. Cheng, Quantized convolutional neural networks for mobile devices, in: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Los Alamitos, CA, USA, 2016, pp. 4820–4828.
  * [24] T. Zhu, X. Qin, M. Li, Binary Neural Network for Speaker Verification, in: Proc. Interspeech, 2021, pp. 86–90.  doi:10.21437/Interspeech.2021-600.
  * [25] P. Chi, P. Chung, T. Wu, C. Hsieh, Y. Chen, S. Li, H. Lee, Audio albert: A lite bert for self-supervised learning of audio representation, in: IEEE Spoken Language Technology Workshop (SLT), 2021, pp. 344–350.  doi:10.1109/SLT48900.2021.9383575.
  * [26] B. Liu, H. Wang, Z. Chen, S. Wang, Y. Qian, Self-knowledge distillation via feature enhancement for speaker verification, in: Proc. ICASSP, 2022, pp. 7542–7546.  doi:10.1109/ICASSP43922.2022.9746529.
  * [27] V. Mingote, A. Miguel, D. Ribas, A. Ortega, E. Lleida, Knowledge distillation and random erasing data augmentation for text-dependent speaker verification, in: Proc. ICASSP, 2020, pp. 6824–6828.  doi:10.1109/ICASSP40776.2020.9053153.
  * [28] R. W. M. Ng, X. Liu, P. Swietojanski, Teacher-student training for text-independent speaker recognition, in: 2018 IEEE Spoken Language Technology Workshop (SLT), 2018, pp. 1044–1051.  doi:10.1109/SLT.2018.8639564.
  * [29] D. Snyder, P. Ghahremani, D. Povey, D. Garcia-Romero, Y. Carmiel, S. Khudanpur, Deep neural network-based speaker embeddings for end-to-end speaker verification, in: IEEE Spoken Language Technology Workshop (SLT), 2016, pp. 165–170.  doi:10.1109/SLT.2016.7846260.
  * [30] D. Snyder, D. Garcia-Romero, D. Povey, S. Khudanpur, Deep neural network embeddings for text-independent speaker verification, in: Proc. Interspeech, 2017, pp. 999–1003.  doi:10.21437/Interspeech.2017-620.
  * [31] E. Variani, X. Lei, E. McDermott, I. Moreno, J. Gonzalez-Dominguez, Deep neural networks for small footprint text-dependent speaker verification, in: Proc. ICASSP, 2014.
  * [32] F. Richardson, D. Reynolds, N. Dehak, Deep neural network approaches to speaker and language recognition, IEEE Signal Processing Letters 22 (10) (2015) 1671–1675.  doi:10.1109/LSP.2015.2420092.
  * [33] S. Han, J. Pool, J. Tran, W. J. Dally, Learning both weights and connections for efficient neural networks, in: Proceedings of the 29th International Conference on Neural Information Processing Systems, 2015, p. 1135–1143.
  * [34] S. Han, H. Mao, W. J. Dally, Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding, in: Y. Bengio, Y. LeCun (Eds.), 4th International Conference on Learning Representations, ICLR 2016, 2016.
  * [35] Y. He, X. Zhang, J. Sun, Channel pruning for accelerating very deep neural networks, in: 2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp. 1398–1406.  doi:10.1109/ICCV.2017.155.
  * [36] J. Wu, C. Leng, Y. Wang, Q. Hu, J. Cheng, Quantized convolutional neural networks for mobile devices, in: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society, Los Alamitos, CA, USA, 2016, pp. 4820–4828.  doi:10.1109/CVPR.2016.521.
URL https://doi.ieeecomputersociety.org/10.1109/CVPR.2016.521

  * [37] T. Zhu, X. Qin, M. Li, Binary neural network for speaker verification, in: Proc. Interspeech 2021, 2021, pp. 86–90.  doi:10.21437/Interspeech.2021-600.
  * [38] H. Qin, R. Gong, X. Liu, X. Bai, J. Song, N. Sebe, Binary neural networks: A survey, Pattern Recognition 105 (2020) 107281.  doi:https://doi.org/10.1016/j.patcog.2020.107281.
URL https://www.sciencedirect.com/science/article/pii/S0031320320300856

  * [39] V. Lebedev, Y. Ganin, M. Rakhuba, I. V. Oseledets, V. S. Lempitsky, Speeding-up convolutional neural networks using fine-tuned cp-decomposition, in: Y. Bengio, Y. LeCun (Eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
  * [40] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, L.-C. Chen, MobileNetV2: Inverted residuals and linear bottlenecks, in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 4510–4520.  doi:10.1109/CVPR.2018.00474.
  * [41] J. A. Chagas Nunes, D. Macêdo, C. Zanchettin, Am-mobilenet1d: A portable model for speaker recognition, in: 2020 International Joint Conference on Neural Networks (IJCNN), 2020, pp. 1–8.  doi:10.1109/IJCNN48605.2020.9207519.
  * [42] X. Zhang, X. Zhou, M. Lin, J. Sun, Shufflenet: An extremely efficient convolutional neural network for mobile devices, in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 6848–6856.  doi:10.1109/CVPR.2018.00716.
  * [43] N. Ma, X. Zhang, H.-T. Zheng, J. Sun, Shufflenet v2: Practical guidelines for efficient cnn architecture design, in: V. Ferrari, M. Hebert, C. Sminchisescu, Y. Weiss (Eds.), Computer Vision – ECCV 2018, Springer International Publishing, 2018, pp. 122–138.
  * [44] D. Snyder, D. Garcia-Romero, G. Sell, D. Povey, S. Khudanpur, X-vectors: Robust dnn embeddings for speaker recognition, in: Proc. ICASSP, 2018, pp. 5329–5333.  doi:10.1109/ICASSP.2018.8461375.
  * [45] V. Peddinti, D. Povey, S. Khudanpur, A time delay neural network architecture for efficient modeling of long temporal contexts, in: Proc. Interspeech 2015, 2015, pp. 3214–3218.  doi:10.21437/Interspeech.2015-647.
  * [46] K. Okabe, T. Koshinaka, K. Shinoda, Attentive statistics pooling for deep speaker embedding, in: Proc. Interspeech, 2018, pp. 2252–2256.  doi:10.21437/Interspeech.2018-993.
URL http://dx.doi.org/10.21437/Interspeech.2018-993

  * [47] F. Wang, J. Cheng, W. Liu, H. Liu, Additive margin softmax for face verification, IEEE Signal Processing Letters 25 (7) (2018) 926–930.  doi:10.1109/LSP.2018.2822810.
  * [48] T. Ko, V. Peddinti, D. Povey, M. L. Seltzer, S. Khudanpur, A study on data augmentation of reverberant speech for robust speech recognition, in: Proc. ICASSP, 2017, pp. 5220–5224.  doi:10.1109/ICASSP.2017.7953152.
  * [49] D. Snyder, G. Chen, D. Povey, MUSAN: A music, speech, and noise corpus, CoRR abs/1510.08484 (2015).  arXiv:1510.08484.
URL http://arxiv.org/abs/1510.08484

  * [50] G. Hinton, O. Vinyals, J. Dean, Distilling the knowledge in a neural network, in: NIPS Deep Learning and Representation Learning Workshop, 2015.
  * [51] V. Manohar, P. Ghahremani, D. Povey, S. Khudanpur, A teacher-student learning approach for unsupervised domain adaptation of sequence-trained asr models, in: IEEE Spoken Language Technology Workshop (SLT), 2018, pp. 250–257.  doi:10.1109/SLT.2018.8639635.
  * [52] S. Wang, Y. Yang, T. Wang, Y. Qian, K. Yu, Knowledge distillation for small foot-print deep speaker embedding, in: Proc. ICASSP, 2019, pp. 6021–6025.  doi:10.1109/ICASSP.2019.8683443.
  * [53] A. Singhal, Modern information retrieval: A brief overview., IEEE Data Eng. Bull. 24 (4) (2001) 35–43.
URL http://dblp.uni-trier.de/db/journals/debu/debu24.html#Singhal01

  * [54] Q. Hong, C. Wu, H. Wang, C. Huang, Statistics pooling time delay neural network based on x-vector for speaker verification, in: Proc. ICASSP, 2020, pp. 6849–6853.  doi:10.1109/ICASSP40776.2020.9054350.
  * [55] S. Furui, Cepstral analysis technique for automatic speaker verification, IEEE Transactions on Acoustics, Speech, and Signal Processing 29 (2) (1981) 254–272.  doi:10.1109/TASSP.1981.1163530.
  * [56] W. Cai, Z. Cai, X. Zhang, X. Wang, M. Li, A novel learnable dictionary encoding layer for end-to-end language identification, in: Proc. ICASSP, 2018, pp. 5189–5193.  doi:10.1109/ICASSP.2018.8462025.
  * [57] X. Glorot, Y. Bengio, Understanding the difficulty of training deep feedforward neural networks, in: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, PMLR, 2010, pp. 249–256.
URL http://proceedings.mlr.press/v9/glorot10a.html

  * [58] A. Nagrani, J. Chung, A. Zisserman, Voxceleb: A large-scale speaker identification dataset, in: Proc. Interspeech, 2017, pp. 2616–2620.  doi:10.21437/Interspeech.2017-950.
URL http://dx.doi.org/10.21437/Interspeech.2017-950

  * [59] J. Chung, A. Nagrani, A. Zisserman, VoxCeleb2: Deep speaker recognition, in: Proc. Interspeech, 2018, pp. 1086–1090.  doi:10.21437/Interspeech.2018-1929.
URL http://dx.doi.org/10.21437/Interspeech.2018-1929

  * [60] P.-M. Bousquet, M. Rouvier, On Robustness of Unsupervised Domain Adaptation for Speaker Recognition, in: Proc. Interspeech, 2019, pp. 2958–2962.  doi:10.21437/Interspeech.2019-1524.
  * [61] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, A. Lerer, Automatic differentiation in pytorch, in: 31st Conference on Neural Information Processing Systems, 2017.
  * [62] D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlicek, Y. Qian, P. Schwarz, J. Silovsky, G. Stemmer, K. Vesely, The kaldi speech recognition toolkit, in: Proc. ASRU, IEEE Signal Processing Society, 2011.
  * [63] Q. Li, L. Yang, X. Wang, X. Qin, J. Wang, M. Li, Towards lightweight applications: Asymmetric enroll-verify structure for speaker verification, in: Proc. ICASSP, 2022, pp. 7067–7071.  doi:10.1109/ICASSP43922.2022.9746247.
  * [64] L. Cai, Y. Yang, X. Chen, W. Tu, H. Chen, CS-CTCSCONV1D: Small footprint speaker verification with channel split time-channel-time separable 1-dimensional convolution, in: Proc. Interspeech, 2022, pp. 326–330.  doi:10.21437/Interspeech.2022-913.