# Self-replication of a quantum artificial organism driven by single-photon
pulses

Daniel Valente 1<EMAIL_ADDRESS>1 Instituto de Física, Universidade
Federal de Mato Grosso, Cuiabá 78060-900 Mato Grosso, Brazil

###### Abstract

Artificial organisms are computer programs that self-replicate, mutate,
compete and evolve. How do these lifelike information-processing behaviours
could arise in diverse far-from-equilibrium physical systems remains an open
question. Here, I devise a toy model where the onset of self-replication of a
quantum artificial organism (a chain of lambda systems) is owing to single-
photon pulses added to a zero-temperature environment. The model results in a
replication probability that is proportional to the absorbed work from the
photon, in agreement with the theory of dissipative adaptation. Unexpectedly,
spontaneous mutations are unavoidable in this model, due to rare but finite
absorption of off-resonant photons. These results hint at self-replication as
a possible link between dissipative adaptation and open-ended evolution.

##  I Introduction

The longstanding question concerning the general principles of life still
inspires a large diversity of concepts, methods and viewpoints schrodinger ;
bridge1 ; cleland ; englandbook . Besides its fundamental value, for instance
towards a universal biology nigel , looking for generalities could eventually
enable us to imitate and extend the sophisticated dynamics achieved by living
things. The artificial life approach, for instance, circumvents the
biochemical constraints of living organisms by studying computer programs that
mimic distinctive features of life. Artificial organisms can harness self-
replication so as to mutate, compete and evolve complex features lenski ;
OEE19 . When one is interested in the most essential aspects of artificial
life, quantum mechanics can be instrumental in casting the computation as
operations on a handful of quantum bits unai1 ; unai2 ; unai3 ; lamata .
Still, the finely engineered character of the computers assumed from the
outset leaves open the question of what principles could explain the self-
assembly of lifelike information-processing systems from physical laws.

The nonequilibrium thermodynamic approach, by contrast, looks for the
emergence of lifelike behaviours in driven physical systems, comprising
disordered, self-assembling, and self-replicating ones JCP2013 ;
naturenano2015 ; englandbook ; science2010 ; scirep2013 ; PRE2015 ; natmat2017
; naturephoton2018 ; englandPRE2019 ; naturenano2020 . Remarkably, it has been
recently proposed that nonequilibrium thermodynamics could generalize
Darwinian evolution, even for non-replicating systems prx2016 . The idea is
that exceptional specialization, or fine-tuned adaptation, to an environment
by a fluctuating physical system can be fueled by the irreversible work
consumption along far-from-equilibrium trajectories PNAS2017 ; PRL2017 . This
exceptional self-organized response of a driven system to the patterns of its
environment has been called dissipative adaptation naturenano2015 ;
englandbook ; naturenano2020 . Quantum physics can, once more, help us to
showcase the most elementary aspects of dissipative adaptation dv21 . Yet,
thermodynamic studies of adaptation have been focusing mostly on systems that
stabilize in the out-of-equilibrium states, lacking a more thorough discussion
on the origins of diversification natchem  and open-ended evolution OEE ;
OEE19  akin to that achieved with artificial life, and necessary for
explaining the ever-growing complexity and diversity of biological species.

Can a driven physical system undergoing dissipative adaptation implement an
artificial-life computational operation? This would provide a further step in
the connections between the information-processing underlying self-replication
and the dissipation of energy inherent to metabolism copol ; copolj ; ti ,
ultimately helping us to imitate the transition from inanimate to living
matter bridge1 ; algorithmicorigins  in diverse physical systems. More
broadly, searching for a lifelike self-organized computation could also
contribute to the recently proposed goal of ‘thermodynamic computing’ tc .

Here, I formulate a toy model inspired by this idea of an artificial-life code
with a driven system undergoing dissipative adaptation. Namely, the self-
replication of a minimal quantum artificial organism starts due to single-
photon pulses added to a zero-temperature environment. The model results in a
replication probability that is linearly proportional to the average work
absorbed by the replicating organism, characterizing a quantum dissipative
adaptation. Counterintuitively, spontaneous mutations cannot be completely
suppressed even in the zero-temperature limit, since they are induced by the
driving pulses themselves. Mutations are rare, as they happen far from
resonance. Because mutations may lead to open-endedness, the model thus hints
at a possible connection between dissipative adaptation and open-ended
evolution that has not been discussed so far, to the best of my knowledge.

The significance of this paper is mainly to bridge some aspect of artificial
life and nonequilibrium thermodynamics. The quantum mechanical framework has
been chosen so as to keep things in terms of basic resources such as atoms,
photons, and their interactions. It turns out that the model may be thought of
as illustrative of Walker and Davies viewpoint algorithmicorigins , in that
genetics is a digital form of information processing (simulated here with
quantum bits), to be contrasted with the analogue form of information
processing in metabolism (simulated here with continuous pulses). From a
quantum information perspective, this paper can be seen as the quantum
thermodynamics of a quantum cloning process noclo ; buzek ; gisin ; dv2012
driven by single photons. Even though the words ‘cloning’ and ‘self-
replication’ can be used interchangeably here, the option for the latter is
motivated by the original goal of mimicking lifelike information processing
with an out-of-equilibrium physical system.

##  II Results

Quantum artificial organism. The quantum organism is defined as a chain of
quantum systems, as inspired by the polymeric structure of nucleic acid
strands (DNA or RNA). Each quantum system has three energy levels. We choose a
lambda configuration, so as to guarantee that the two lowest-energy levels are
stable in the zero-temperature limit. The two lowest-energy levels are labeled
here as $|a_{n}\rangle_{1}$ and $|b_{n}\rangle_{1}$, where $n$ refers to the
$n$-th lambda system in the chain, and the index $1$, to the original gene
(the original chain). States $|a_{n}\rangle_{1}$ and $|b_{n}\rangle_{1}$ play
the role of two possible equivalents of nucleotide bases (instead of four, as
in DNA or RNA). The original quantum artificial gene is defined by a
(generally aperiodic) sequence of these lowest-energy states (a string), let
us say

$|\mathrm{gene}\rangle_{1}=|a_{1}\rangle_{1}|b_{2}\rangle_{1}\ ...\
|a_{N}\rangle_{1}.$ (1)

$N$ here is the gene size, also corresponding to the chain size. Quantum
superpositions of bases in $|\mathrm{gene}\rangle_{1}$ are not considered in
this paper, as further explained below.

Self-replication. The idea is to look for a dynamical process $U$ (a global
organism-plus-environment unitary dynamics), so that

$U|\mathrm{gene}\rangle_{1}|\mathrm{bases}\rangle_{2}|\mathrm{source}\rangle\rightarrow|\mathrm{gene}\rangle_{1}|\mathrm{gene}\rangle_{2}|\mathrm{source}^{\prime}\rangle,$
(2)

where $U$ must be independent of the initial state
$|\mathrm{gene}\rangle_{1}$. In (2),
$|\mathrm{bases}\rangle_{2}=\prod_{n=1}^{N}|b_{n}\rangle_{2}$ is the state
representing the available environment bases upon which the organism can act
to compose the copied gene, $|\mathrm{gene}\rangle_{2}$. States
$|b_{n}\rangle_{2}$ here are the fundamental states of the lambda systems
(therefore, their thermal equilibrium states in the limit of zero
temperature). The source state

$|\mathrm{source}\rangle=\prod_{n=1}^{N}|s_{n}\rangle$ (3)

describes the initial state of the environment degrees of freedom that provide
the energy source; by the end of the process, the environment may have been
modified to some final state
$|\mathrm{source}^{\prime}\rangle=\prod_{n}|s_{n}^{\prime}\rangle$.
Importantly, the nucleic-acid analogy guides us to look for a process
$U=U(\left\\{r_{n}\right\\})$, which depends on the free parameters
$\left\\{r_{n}\right\\}$ symbolizing spatial distances between each gene base
(the template lambda system) and its corresponding environment base (the
environment lambda system which undergoes the copying process). In other
words, the ability of the template lambda system to copy itself shall depend
on the distance to the environment base.

From now on, replication is assumed to be modular, that is, the replication of
each gene unit ($|\bullet_{n}\rangle_{1}$) is independent of the other units,
$U(\left\\{r_{n}\right\\})=\prod_{n}U_{n}(r_{n})$, where $[U_{n},U_{m}]=0$ for
all $m,n$. This assumption is motivated by the modular character of the self-
replication of nucleic acids. A perfect replication transition should now read

$U_{n}|a_{n}\rangle_{1}|b_{n}\rangle_{2}|s_{n}\rangle\rightarrow|a_{n}\rangle_{1}|a_{n}\rangle_{2}|s_{n}^{\prime}\rangle,$
(4)

and a perfect dormant transition,

$U_{n}|b_{n}\rangle_{1}|b_{n}\rangle_{2}|s_{n}\rangle\rightarrow|b_{n}\rangle_{1}|b_{n}\rangle_{2}|s_{n}\rangle,$
(5)

for $n=1,...,N$. To guarantee that a gene base does not affect an infinitely
far apart environment base, we look for a unitary that satisfies
$U_{n}(r_{n}\rightarrow\infty)\rightarrow 1$. See Fig.1.

Arbitrary superpositions of $|a_{n}\rangle_{1}$ and $|b_{n}\rangle_{1}$ cannot
be copied with an arbitrarily high fidelity (quality), as states the so called
no-cloning theorem noclo ; buzek ; gisin ; dv2012 . The theorem assumes,
however, that it is perfectly possible in principle to do so for a particular
(preferred) orthonormal basis. This explains our choice in Eq.(1). Here, we
intend to find the best replicator allowed by nature (by quantum mechanics, in
this case), with the goal of verifying whether it does consume a maximal
amount of work. In other words, we look for a model for $U_{n}$, trying to
fulfill the perfect cloning as allowed in principle for the eigenenergies of
the lambda systems.

Figure 1:  Natural versus artificial gene self-replication. (a) Simplified
picture of a self-replicating biological gene. A single nucleic acid strand
forms the original gene with nucleotide bases $A$, $T$ and $A$ . The
complementary environment bases, $T$, $A$ and $T$, form the copy. In the
process, the bare states (dashed gray circles) undergo a transformation to the
filled states (full black circles) dictated by the original gene (curved
arrows). (b) A sequence of states
$|a_{1}\rangle_{1}|b_{2}\rangle_{1}|a_{3}\rangle_{1}$ (full black circles)
specifies the original quantum artificial gene. The environment bases are
initially set to their ground states, $|b_{n}\rangle_{2}$, for $n=1,2,3$
(dashed gray circles), which can be regarded as bare states. In the process,
the bare states undergo a transformation to a filled state (full arrow),
dictated by (and ideally identical to) the original gene state,
$|a_{1}\rangle_{1}|b_{2}\rangle_{1}|a_{3}\rangle_{1}\otimes|b_{1}\rangle_{2}|b_{2}\rangle_{2}|b_{3}\rangle_{2}\rightarrow|a_{1}\rangle_{1}|b_{2}\rangle_{1}|a_{3}\rangle_{1}\otimes|a_{1}\rangle_{2}|b_{2}\rangle_{2}|a_{3}\rangle_{2}$.
This transformation is described by a unitary operator
$U(\left\\{r_{n}\right\\})$ for the global organism-plus-environment dynamics,
including that of the source of energy. The unitary operator
$U(\left\\{r_{n}\right\\})$ depends on the free parameter $r_{n}$,
representing the distance between the $n$-th gene base and its corresponding
environment base. If $r_{n}\rightarrow\infty$, the environment bases are
expected to be left unchanged (dashed arrow).

Organism-plus-environment Hamiltonian. We assume an autonomous dynamics

$U_{n}=\exp{\left(-iHt/\hbar\right)},$ (6)

where the time-independent global Hamiltonian $H$, describing the organism
plus the environment, is given by

$H=H_{S}+H_{I}+H_{E}.$ (7)

$H$ does not depend on the index $n$, meaning that the dynamics of each base
follows the same rules. The system Hamiltonian, $H_{S}$, describes the
original gene base only. $H_{E}$ describes a composite environment,

$H_{E}=H_{B}+H_{F}.$ (8)

$H_{B}$ is the environment base Hamiltonian, and $H_{F}$ is the
electromagnetic field Hamiltonian. Correspondingly, the interaction
Hamiltonian $H_{I}$ is also composite,

$H_{I}=H_{SB}+H_{BF}.$ (9)

$H_{SB}$ describes the interaction of the original gene base with the
environment base. $H_{BF}$ describes the interaction of the environment base
with the electromagnetic field.

The explicit expressions for the Hamiltonians are as follows. First,

$H_{S}=\hbar\omega_{b}^{(0)}|e_{n}\rangle_{1}\langle
e_{n}|_{1}+\hbar\delta_{a}^{(0)}|a_{n}\rangle_{1}\langle a_{n}|_{1},$ (10)

where $|e_{n}\rangle_{1}$ is the excited state of the $n$-th lambda system in
the original artificial organism. Analogously, the environment base
Hamiltonian reads

$H_{B}=\hbar\omega_{b}^{(0)}|e_{n}\rangle_{2}\langle
e_{n}|_{2}+\hbar\delta_{a}^{(0)}|a_{n}\rangle_{2}\langle a_{n}|_{2},$ (11)

showing that all lambda systems have been chosen identical to one another.

The coupling Hamiltonian between the system (the original gene base) and the
environment base is

$H_{SB}=-J\ \sigma_{a1}^{(z)}\sigma_{a2}^{(z)},$ (12)

where $\sigma_{ai}^{(z)}=|e_{n}\rangle_{i}\langle
e_{n}|_{i}-|a_{n}\rangle_{i}\langle a_{n}|_{i}$. $H_{SB}$ allows the gene base
state to act as a switch to the dynamics of the environment base. As will
become clearer below, the gene can selectively induce the environment base to
become energetically resonant with the electromagnetic field, reminding us of
an enzyme-like effect. For the degree of influence to depend on the distance
parameters, we consider $J=J(r_{n})$. Condition
$U_{n}(r_{n}\rightarrow\infty)=1$ suggests that $J(r_{n}\rightarrow\infty)=0$.
Ising-type couplings such as the one used here find widespread applications in
protein physics pnas87 ; dv20 . Also, similar position-dependent dipole-dipole
couplings generalizing the van der Waals forces have been proposed as a means
for constructing biologically-inspired quantum molecular machines, which
actively and autonomously self-protect their quantum data against noise from
pre-existing non-engineered environments tg1 ; tg2 .

The base-field interaction Hamiltonian is given by a dipolar coupling, in the
rotating-wave approximation cohen ; dv21 ; dv2012 ,

$H_{BF}=-i\hbar
g\sum_{\omega}(a_{\omega}\sigma_{a2}^{\dagger}+b_{\omega}\sigma_{b2}^{\dagger}-\mbox{H.c.}).$
(13)

The continuum of frequencies, $\sum_{\omega}\rightarrow\int
d\omega\varrho_{\omega}\approx\varrho\int d\omega$, gives rise to the
dissipation rate $\Gamma=2\pi g^{2}\varrho$, in the Wigner-Weisskopf
approximation. Modes $\left\\{a_{\omega}\right\\}$ and
$\left\\{b_{\omega}\right\\}$ represent orthogonal quantized field modes. The
raising operators read $\sigma_{a2}^{\dagger}=|e_{n}\rangle_{2}\langle
a_{n}|_{2}$ and $\sigma_{b2}^{\dagger}=|e_{n}\rangle_{2}\langle b_{n}|_{2}$.
H.c. is the Hermitian conjugate. The choice for coupling the field only to the
environment base ($|\bullet_{n}\rangle_{2}$) is justified by the fact that all
lambda systems are identical in this model, hence it should ideally make no
difference what base the photon hits each time. Finally, the field Hamiltonian
is

$H_{F}=\sum_{\omega}\hbar\omega(a_{\omega}^{\dagger}a_{\omega}+b_{\omega}^{\dagger}b_{\omega}),$
(14)

also considered in the continuum of frequencies limit.

Single-photon pulses as the energy source. The source of energy for the self-
replication is the initial out-of-equilibrium state of the electromagnetic
environment. To make it the most elementary energy source, we consider in
Eq.(3) single-photon pulses added to a zero-temperature background dv21 , so
that

$|s_{n}\rangle=|1^{b}\rangle\equiv\sum_{\omega}\phi_{\omega}^{b}(0)b_{\omega}^{\dagger}|0\rangle.$
(15)

$|0\rangle=\prod_{\omega}|0_{\omega}^{a}\rangle|0_{\omega}^{b}\rangle$ is the
vacuum state of all the field modes. Modes $\left\\{a_{\omega}\right\\}$ are
not initially populated, so as to maximize the irreversibility of the self-
replication. The single-photon pulse admits a one-dimensional real-space
representation,

$\phi(z,t)\equiv\sum_{\omega}\phi_{\omega}^{b}(t)\exp(ik_{\omega}z),$ (16)

where $k_{\omega}=\omega/c$, and $c$ is the speed of light. As before, the sum
over modes is to be considered in the continuum of frequencies limit. The
pulse can also be decomposed as

$\phi(z,t)=\phi^{\mathrm{e}}(z,t)\exp{[-i\omega_{L}(t-z/c)]},$ (17)

in terms of its central frequency $\omega_{L}$ and its envelope function
$\phi^{\mathrm{e}}(z,t)$.

To keep the spirit of an autonomous scenario, the photon pulse is assumed to
have been spontaneously emitted from an arbitrarily distant source (not
considered in the Hamiltonian). This implies an exponential shaped envelope

$\phi^{\mathrm{e}}(z,0)=\mathcal{N}\Theta(-z)\exp{\left[\Delta
z/(2c)\right]}.$ (18)

$\Delta$ here is the pulse spectral linewidth.
$\mathcal{N}\equiv\sqrt{2\pi\varrho\Delta}$, a normalization constant.
$\Theta(z)$, the Heaviside step function. The transition frequency of this
hypothetical distant emitter corresponds to $\omega_{L}$, in Eq.(17), and its
natural linewidth, to $\Delta$. Most importantly, $\omega_{L}$ shall be
regarded here as a fixed constant, rather than a free-varying parameter. This
is analogous to the idea of a steady peak in the sunlight spectrum. By
contrast, the spectral linewidth $\Delta$ of each photon pulse may vary, so as
to mimic a kind of disorder in the natural linewidths in the hypothetical
ensemble of distant single-photon emitters.

Global organism-plus-environment dynamics. The global dynamics can be
described by the state

$|G_{k}(t)\rangle=\exp(-iHt/\hbar)|G_{k}(0)\rangle,$ (19)

where $H$ is given by Eq.(7), and

$|G_{k}(0)\rangle=|k_{n}\rangle_{1}|b_{n}\rangle_{2}|1^{b}\rangle,$ (20)

for $k=a,b$.

Due to conservation in the number of excitations, the time-dependent state can
be written for $k=a$ as

$\displaystyle|G_{a}(t)\rangle$
$\displaystyle=\sum_{\omega}F_{\omega}(t)|a_{n}\rangle_{1}|b_{n}\rangle_{2}|1_{\omega}^{b}\rangle$
$\displaystyle+R_{e}(t)|a_{n}\rangle_{1}|e_{n}\rangle_{2}|0\rangle$
$\displaystyle+\sum_{\omega}R_{a\omega}(t)|a_{n}\rangle_{1}|a_{n}\rangle_{2}|1_{\omega}^{a}\rangle.$
(21)

$F_{\omega}$ describes a failed replication, leaving a photon at mode
$b_{\omega}$. $R_{e}$ describes a replication transient excitation, leaving
the field in the vacuum state. $R_{a\omega}$ describes a replication
accomplishment, leaving a photon at mode $a_{\omega}$.

Accordingly, the global state can be written for $k=b$ as

$\displaystyle|G_{b}(t)\rangle$
$\displaystyle=\sum_{\omega}D_{\omega}(t)|b_{n}\rangle_{1}|b_{n}\rangle_{2}|1_{\omega}^{b}\rangle$
$\displaystyle+M_{e}(t)|b_{n}\rangle_{1}|e_{n}\rangle_{2}|0\rangle$
$\displaystyle+\sum_{\omega}M_{a\omega}(t)|b_{n}\rangle_{1}|a_{n}\rangle_{2}|1_{\omega}^{a}\rangle.$
(22)

$D_{\omega}$ describes a dormant state, leaving a photon at mode $b_{\omega}$.
$M_{e}$ describes a mutation transient excitation, leaving the field in the
vacuum state. $M_{a\omega}$ describes an (undesirable) mutation
accomplishment, leaving a photon at mode $a_{\omega}$. The initial state of
the field implies that $F_{\omega}(0)=\phi^{b}_{\omega}(0)$ or
$D_{\omega}(0)=\phi^{b}_{\omega}(0)$, as defined in Eq.(15), depending on the
initial state of the original gene base. The equations of motion are shown in
the Methods.

Transition probabilities. The organism replication probability is defined here
as

$p_{a_{1},b_{2}\rightarrow a_{1},a_{2}}(t)\equiv\langle a_{n}|_{1}\langle
a_{n}|_{2}\mbox{tr}_{F}\Big{[}|G_{a}(t)\rangle\langle
G_{a}(t)|\Big{]}|a_{n}\rangle_{1}|a_{n}\rangle_{2},$ (23)

where $\mbox{tr}_{F}[\bullet]$ is the partial trace over the field degrees of
freedom. Similarly, the dormant probability is defined as

$p_{b_{1},b_{2}\rightarrow b_{1},b_{2}}(t)\equiv\langle b_{n}|_{1}\langle
b_{n}|_{2}\mbox{tr}_{F}\Big{[}|G_{b}(t)\rangle\langle
G_{b}(t)|\Big{]}|b_{n}\rangle_{1}|b_{n}\rangle_{2}.$ (24)

By using Eq.(21), we find that

$p_{a_{1},b_{2}\rightarrow a_{1},a_{2}}(t)=\sum_{\omega}|R_{a\omega}(t)|^{2}.$
(25)

With Eq.(22), we also find that

$p_{b_{1},b_{2}\rightarrow b_{1},b_{2}}(t)=\sum_{\omega}|D_{\omega}(t)|^{2}.$
(26)

To obtain explicit expressions, we turn to the one-dimensional real-space
representation of the frequency-dependent amplitudes, namely,
$R_{a}(z,t)\equiv\sum_{\omega}R_{a\omega}(t)\exp(ik_{\omega}z)$ and
$D(z,t)\equiv\sum_{\omega}D_{\omega}(t)\exp(ik_{\omega}z)$, as done in
Eq.(16), and find that

$p_{a_{1},b_{2}\rightarrow a_{1},a_{2}}(t)=\frac{1}{2\pi\varrho
c}\int_{-\infty}^{\infty}|R_{a}(z,t)|^{2}dz,$ (27)

and

$p_{b_{1},b_{2}\rightarrow b_{1},b_{2}}(t)=\frac{1}{2\pi\varrho
c}\int_{-\infty}^{\infty}|D(z,t)|^{2}dz.$ (28)

See the expressions for $R_{a}(z,t)$ and $D(z,t)$ in the methods.

We now assume the spontaneously emitted photon as described by Eq.(18). If the
initial state of the original base is $|a_{n}\rangle_{1}$, we make
$\phi(z,0)=F(z,0)=\sum_{\omega}F_{\omega}(0)\exp(ik_{\omega}z)$; otherwise
(for $|b_{n}\rangle_{1}$), we make $\phi(z,0)=D(z,0)$ (defined in the previous
paragraph). We find that (see Methods)

$p_{a_{1},b_{2}\rightarrow
a_{1},a_{2}}(t\rightarrow\infty)=\mathcal{P}(\Delta,\delta_{L-bJ}),$ (29)

where the detuning is $\delta_{L-bJ}\equiv\omega_{L}-\omega_{bJ}$, with
reference to the perturbed frequency transition
$\omega_{bJ}\equiv\omega_{b}^{(0)}+J$, and

$\displaystyle\mathcal{P}(\Delta,\delta_{L-bJ})$
$\displaystyle\equiv\frac{\Gamma^{2}}{\left(\frac{2\Gamma-\Delta}{2}\right)^{2}+\delta^{2}_{L-bJ}}$
$\displaystyle\times\left[1+\frac{\Delta}{2\Gamma}-\frac{\Delta(2\Gamma+\Delta)}{\left(\frac{2\Gamma+\Delta}{2}\right)^{2}+\delta^{2}_{L-bJ}}\right].$
(30)

We regard only $\Delta$ and $J$ as the truly free parameters in Eq.(30). We
assume, by contrast, that $\omega_{L}$ is fixed by the external world, whereas
$\omega_{b}^{(0)}$ and $\Gamma$ are fixed by the internal world. We are
particularly interested in the regimes where
$\omega_{L}-\omega_{b}^{(0)}\gg\Gamma$ (blue-detuning) and $J>0$, which
clarify the picture.

Optimal replication. The replication probability at long times,
$p_{a_{1},b_{2}\rightarrow a_{1},a_{2}}(\infty)$, is maximized when the
coupling $J=J(r_{n})$ induces the resonance condition

$\delta_{L-bJ}\equiv\omega_{L}-\omega_{bJ}=\omega_{L}-(\omega_{b}^{(0)}+J)=0.$
(31)

This energy-matching mechanism increasing the likelihood of the process,
conditional to the distance $r_{n}$ between the gene base and the environment
base, is reminiscent of an enzyme effect (see Fig.(2)-a). Perfect replication
also requires a monochromatic photon ($\Delta\ll\Gamma$), as we find from
Eq.(30),

$p_{a_{1},b_{2}\rightarrow
a_{1},a_{2}}(\infty)\Big{|}_{\left\\{\delta_{L-bJ}=0,\ \Delta\rightarrow
0\right\\}}=\mathcal{P}(0,0)=1.$ (32)

The dormant transition probability is given by

$p_{b_{1},b_{2}\rightarrow
b_{1},b_{2}}(\infty)=1-\mathcal{P}(\Delta,\delta_{L-b}),$ (33)

where the detuning now refers to the unperturbed frequency,

$\delta_{L-b}\equiv\omega_{L}-\omega_{b}^{(0)},$ (34)

and $\mathcal{P}(\Delta,\delta_{L-b})$ is also defined by Eq.(30). The problem
of maximizing both the replication and the dormant transition probabilities is
discussed in the following.

Figure 2:  Mechanism behind replication and mutation. (a) A gene base at
$|a_{n}\rangle_{1}$ (black dot) induces an energy shift (from the gray to the
black horizontal bars) of $\hbar J$ (smaller arrow), building a resonance with
the photon, $\hbar\omega_{b}^{(0)}+\hbar J=\hbar\omega_{L}$ (longer arrow).
$\hbar\omega_{b}^{(0)}$ is the transition energy of the unperturbed lambda
system (dotted arrow). The environment base thus undergoes a replication
transition $|b_{n}\rangle_{2}\rightarrow|a_{n}\rangle_{2}$ (full curved
arrow). (b) A gene base at $|b_{n}\rangle_{1}$ (black dot) leaves the
frequencies unaltered (black horizontal bars), keeping the environment base
far from resonance, $\omega_{b}^{(0)}\ll\omega_{L}$. However, rare off-
resonant photon absorption may induce the mutation transition
$|b_{n}\rangle_{2}\rightarrow|a_{n}\rangle_{2}$ (dashed curved arrow).

Mutations. Optimizing the replication transition, as described by Eq.(29), and
the dormant transition, Eq.(33), requires a coupling $J$ that makes the
environment base resonant with the photon when the original lambda system is
at $|a_{n}\rangle_{1}$ (i.e., $\omega_{bJ}=\omega_{L}$), whereas keeping it
far from resonance when the original lambda system is at $|b_{n}\rangle_{1}$.
The crucial point here is to notice that, for any finite $J$, an off-resonant
photon with respect to the unperturbed frequency ($\delta_{L-b}=J$, since
$\omega_{bJ}=\omega_{L}$) can eventually be absorbed, with the mutation
probability (see Fig.(2)-b)

$p_{b_{1},b_{2}\rightarrow
b_{1},a_{2}}(\infty)=\sum_{\omega}|M_{a\omega}(\infty)|^{2}=\mathcal{P}(\Delta,\delta_{L-b}),$
(35)

which vanishes only in the $\delta_{L-b}=J\rightarrow\infty$ limit. This
requires un unphysical photon of infinite frequency to fulfill the perfect
replication condition, $\omega_{L}=\omega_{b}^{(0)}+J$. Put differently,
equations $p_{a_{1},b_{2}\rightarrow a_{1},a_{2}}(\infty)=1$ and
$p_{b_{1},b_{2}\rightarrow b_{1},b_{2}}(\infty)=1$ cannot be simultaneously
satisfied for finite couplings, $J<\infty$.

The finite off-resonant photon absorption probability, leading to
$p_{b_{1},b_{2}\rightarrow b_{1},b_{2}}(\infty)<1$, is at the root of the
unexpected mutations in this model. It is unexpected because the model was not
intentionally designed to present mutations. On the contrary, we intended to
find perfect cloning on a preferred basis, as allowed by the no-cloning
theorem noclo . Nevertheless, the idea of a light-induced rare mutation
represents here an appealing analogy with natural mutations in biological
genes. According to ref.nigel , error in replicating information creates the
conditions for the evolvability of life as we know it. This turns an
unfortunate drawback into a powerful resource.

Finally, we note that in the $r_{n}\rightarrow\infty$ limit (i.e.,
$J(\infty)=0$), the incoming photon is off-resonant for both initial states of
the gene base, $\delta_{L-bJ}=\delta_{L-b}\gg\Gamma$, leading to a
predominantly dormant dynamics, $p_{b_{1},b_{2}\rightarrow
b_{1},b_{2}}(\infty)\rightarrow 1$ and $p_{a_{1},b_{2}\rightarrow
a_{1},a_{2}}(\infty)\rightarrow 0$, as expected from the
$U(r_{n}\rightarrow\infty)\rightarrow 1$ condition discussed after Eq.(5).

Dissipative adaptation. We finally investigate how dissipative adaptation
underlies the self-replication in the present model. As stated in the
introduction, dissipative adaptation is a general thermodynamic mechanism
explaining lifelike self-organization in classical far-from-equilibrium
systems. It clarifies how fine-tuned, exceptional behaviours can be
fundamentally related with work consumption. The main picture is that, when a
given fluctuating physical system absorbs work from its environment, it can
reach exceptional dynamical transformations that are selected by the work
source characteristics. If the excess energy provided by this nonequilibrium
work source is dissipated to the environment (in the form of heat), the system
can get irreversibly trapped in those rare configurations; in other words,
adapted to its environment. Mathematically, this idea is best described in
terms of a fluctuation theorem. By using Crooks’ microscopic reversibility
condition for the forward, $p_{i\rightarrow j}(t)$, and backward,
$p^{*}_{j\rightarrow i}(\tau-t)$, classical trajectory probabilities between
the initial $i$ and the final $j$ states crooks , the dissipative adaptation
has been formulated as naturenano2015 ; prx2016

$\frac{p_{i\rightarrow j}(t)}{p_{i\rightarrow k}(t)}=e^{-\beta
E_{kj}}\frac{p^{*}_{j\rightarrow i}(\tau-t)}{p^{*}_{k\rightarrow
i}(\tau-t)}\frac{\langle e^{-\beta W_{\mathrm{abs}}}\rangle_{ik}}{\langle
e^{-\beta W_{\mathrm{abs}}}\rangle_{ij}},$ (36)

where the angle brackets denote a weighted average over all microtrajectories
with fixed start $i$ and end $j,k$ points. $\beta$ is the inverse temperature,
$E_{kj}=E_{j}-E_{k}$ is the energy difference, and $W_{\mathrm{abs}}$ is the
stochastic nonequilibrium work absorbed from a time-dependent drive. Equation
(36) evidences that a higher work absorption in the $i$ to $j$ transition
boosts the probability of state $j$ over the alternative $k$. Dissipative
adaptation can, therefore, describe the self-organization of a physical system
enabling it to become apparently well suited to perform some finely-tuned,
exceptional task: to seek energy PRE2015 ; PRL2017 , to avoid energy PRL2017 ;
bistableDA ; dv21 , or to self-replicate JCP2013 ; naturenano2015 ; prx2016 .

Here, we find quite similar behaviour. An exceptionally high probability of
replication at a zero-temperature environment requires the absorption of a
proportionally high amount of average work, as given by

$p_{a_{1},b_{2}\rightarrow a_{1},a_{2}}(\infty)=\langle
W_{\mathrm{abs}}\rangle_{a_{1},b_{2}}/(2\hbar\omega_{L}).$ (37)

$\langle W_{\mathrm{abs}}\rangle_{a_{1},b_{2}}$ is more precisely defined in
the following paragraph; its index denotes that the average is calculated with
the initial state $|a_{n}\rangle_{1}|b_{n}\rangle_{2}$ for the matter (and
$|1^{b}\rangle$ for the field, as usual). Because Eq.(37) is valid at zero
temperature and has been obtained from a fully-quantized model, we can call it
a quantum dissipative adaptation dv21 . We emphasize that Eq.(37) is
independent of the pulse envelope function (as defined in Eq.(17)), and that
it generalizes the result from ref. dv21 , in being not limited to a resonant
photon, and in having been obtained for the dynamics of coupled lambda
systems.

Work consumption. To obtain our quantum version of the dissipative adaptation,
Eq.(37), we depart from a definition of average incoming work as given by the
Heisenberg picture, following ref.dv21 ,

$\langle
W_{\mathrm{in}}\rangle\equiv\int_{0}^{\infty}\langle\left(\partial_{t}d(t)\right)E_{\mathrm{in}}(t)\rangle
dt,$ (38)

where $d(t)=U_{n}^{\dagger}\ d\ U_{n}$ is the dipole operator and
$E_{\mathrm{in}}(t)$ is the incoming-field operator. Here,
$d=\sum_{i}d_{ei}(\sigma_{i2}^{\dagger}+\mbox{H.c.})$,
$E_{\mathrm{in}}(t)=\sum_{\omega}i\epsilon(a_{\omega}+b_{\omega})e^{-i\omega
t}+\mbox{H.c.}$, and $d_{ei}\epsilon=\hbar g$. Definition (38) is very close
to our classical notion of work OL2018 ; cohen  (to see that, one can think of
an initial coherent, or semiclassical, incoming pulse $|\alpha\rangle$,
fulfilling $a_{\omega}|\alpha\rangle=\alpha_{\omega}|\alpha\rangle$, as
established by Glauber glauber ). In the rotating-wave approximation, and
using the initial global state
$|a_{n}\rangle_{1}|b_{n}\rangle_{2}|1^{b}\rangle$, we find that (see Methods)

$\langle W_{\mathrm{in}}\rangle_{a_{1},b_{2}}=\langle
W_{\mathrm{abs}}\rangle_{a_{1},b_{2}}+\langle
W_{\mathrm{reac}}\rangle_{a_{1},b_{2}},$ (39)

in terms of the absorptive contribution,

$\langle
W_{\mathrm{abs}}\rangle_{a_{1},b_{2}}\equiv\hbar\omega_{L}\int_{0}^{\infty}(-2g)\Re[R_{e}^{*}e^{-i\delta_{a}^{(0)}t}F(-ct,0)]dt,$
(40)

and the reactive (dispersive) contribution OL2018 ,

$\langle W_{\mathrm{reac}}\rangle_{a_{1},b_{2}}\equiv\int_{0}^{\infty}(-2\hbar
g)\Re[iR_{e}^{*}e^{-i\delta_{a}^{(0)}t}(\partial_{t}{F}^{e})e^{-i\omega_{L}t}]dt.$
(41)

$\Re[\bullet]$ stands for the real part. In Eq.(41), we have defined
$F^{e}(t)$ such that $F(-ct,0)\equiv F^{e}(t)\exp(-i\omega_{L}t)$. Eqs.(62)
and (72) in the Methods, combined with (40), give us Eq.(37).

The meaning of the absorptive and reactive work contributions from a single-
photon pulse becomes clearer in the monochromatic regime, where an analogy
with a classical harmonic oscillator takes place. The monochromatic regime is
set for $\Delta\ll\Gamma$, given the exponential pulse used in Eq.(29). We
then find that

$r(t)\approx\chi\ f(t),$ (42)

where the susceptibility is defined as

$\chi\equiv i\Gamma/(\Gamma-i\delta_{L-bJ}).$ (43)

Here, $r(t)\equiv\sqrt{\Gamma}R_{e}(t)e^{i\delta_{a}^{(0)}t}$ and $f(t)\equiv
i\sqrt{\Delta}\exp[-(\Delta/2+i\omega_{L})t]$. This approximately linear
dependence is obtained from Eq.(63), which depends on the entire history of
the photon pulse, revealing the non-Markovianity of the dynamics of the lambda
systems nonmarkov . In the monochromatic regime, however, we find that Eq.(42)
holds at times $t\gg\Gamma^{-1}$. By writing the susceptibility in terms of
its real and imaginary parts,

$\chi=\chi^{\prime}+i\chi^{\prime\prime},$ (44)

we find that

$\langle W_{\mathrm{abs}}\rangle_{a_{1},b_{2}}\approx 2\hbar\omega_{L}\
\chi^{\prime\prime},$ (45)

and

$\langle W_{\mathrm{reac}}\rangle_{a_{1},b_{2}}\approx\hbar\Delta\
\chi^{\prime},$ (46)

in close analogy to what has been discussed in refs.cohen ; OL2018 . Note that
the reactive work, $\langle W_{\mathrm{reac}}\rangle_{a_{1},b_{2}}$, vanishes
both at resonance ($\delta_{L-bJ}=0$), and arbitrarily far from resonance,
$\delta_{L-bJ}\rightarrow\infty$ (as expected in dispersive light-matter
interactions). More importantly, it vanishes in the monochromatic limit
$\Delta\rightarrow 0$, where the self-replication is optimized. The absorptive
work, $\langle W_{\mathrm{abs}}\rangle_{a_{1},b_{2}}$, does not depend on
$\Delta\rightarrow 0$, is maximal at resonance, but also vanishes far from
resonance. This tells us that the far-from-resonance (rare) mutations absorb a
vanishingly small amount of work.

Analogous results arise from considering a fully classical damped harmonic
oscillator with complex position $r_{c}(t)$, driven by the force
$f_{c}(t)=\sqrt{\Delta}\exp[-(\Delta/2+i\omega_{L})t]$, with a slowly-varying
amplitude $\Delta\ll\Gamma$, where $\Gamma$ is the oscillator dissipation
rate. We then have from Newtonian dynamics that
$r_{c}(t)\approx\chi_{c}f_{c}(t)$, so the classical work reads
$W_{\mathrm{cl}}\equiv\int_{0}^{\infty}2\Re[f_{c}^{*}\partial_{t}r_{c}]dt=-\int_{0}^{\infty}2\Re[r_{c}^{*}\partial_{t}f_{c}]dt\approx\Delta\chi_{c}^{\prime}+2\omega_{L}\chi_{c}^{\prime\prime}$,
where $\Delta\chi_{c}^{\prime}$ is the reactive (dispersive) part, and
$2\omega_{L}\chi_{c}^{\prime\prime}$ is the absorptive part (see ref.cohen ).

##  III Discussion

In summary, the toy model devised here shows how the nonequilibrium work
provided by spontaneously emitted single-photon pulses can fuel the self-
replication of an elementary quantum artificial organism formed by a chain of
lambda systems. The guiding intuition was that dissipative adaptation could
result in some kind of self-organized process reminding us of an artificial-
life code. Quantum mechanics allowed us to think in terms of the most basic
resources in nature, namely, atoms, photons, and their interactions.

Mutations were found to be unavoidable, though rare, due to far-from-resonance
photon absorption. Interestingly, because mutations and self-replication may
imply a possible route towards open-ended evolution in biological systems, the
model thus alludes to a theoretical link between dissipative adaptation and
open-endedness that calls for further investigations. Finally, it is worth
emphasizing that the mutations play a central role in justifying a posteriori
why we could assume the existence of an arbitrary state
$|\mathrm{gene}\rangle_{1}$ from the outset. Put differently, how could state
$|a_{n}\rangle_{1}$ first have appeared, in an otherwise zero-temperature
autonomous universe at thermal equilibrium? Without mutations, all the bases
would perpetually remain in their ground states $|b_{n}\rangle_{i}$, implying
a completely dormant, rather trivial universe.

As a perspective, we can search for self-organized artificial-life codes with
some degree of (quantum or classical) complexity nicole . We can think, for
instance, of mimicking the self-organized evolution of an entire artificial
genetic code, going beyond the artificial gene we have considered. To evolve
the natural genetic code, biological organisms have taken great advantage from
the so called horizontal gene transfer (HGT), according to refs.goldenfeld ;
cleland . An artificial HGT can be envisioned by letting the free parameters
$r_{n}$ here (representing the distances between pairs of lambda systems) to
behave as Brownian particles in a common environment (to be more precise, the
center of mass of each lambda system could be considered as a Brownian
particle). Common environments can mediate attractive effective couplings
between pairs of Brownian particles (classical and quantum), as shown in
refs.amir ; oscar ; dv10 . The size $N$ of each artificial gene would then
become a stochastic variable (the artificial gene being slowly split or merged
with others in the environment), simply from the environment-induced Brownian
movements and effective couplings, thus implementing an artificial HGT.

##  IV Methods

Solution of the global dynamics. The Schrödinger equation
$i\hbar\partial_{t}|G_{k}(t)\rangle=H|G_{k}(t)\rangle$ (as defined in Eqs.(7)
and (19)) leads us to

$\displaystyle\partial_{t}D_{\omega}$ $\displaystyle=-i\omega
D_{\omega}+gM_{e},$ (47) $\displaystyle\partial_{t}M_{e}$
$\displaystyle=-i\omega_{b}^{(0)}M_{e}-g\sum_{\omega}(D_{\omega}+M_{a\omega}),$
(48) $\displaystyle\partial_{t}M_{a\omega}$
$\displaystyle=-i(\omega+\delta_{a}^{(0)})M_{a\omega}+gM_{e},$ (49)

for $k=a$, and

$\displaystyle\partial_{t}F_{\omega}$
$\displaystyle=-i(\omega+\delta_{a}^{(0)})F_{\omega}+gR_{e},$ (50)
$\displaystyle\partial_{t}R_{e}$
$\displaystyle=-i(\delta_{a}^{(0)}+\omega_{b}^{(0)}+J)R_{e}-g\sum_{\omega}(F_{\omega}+R_{a\omega}),$
(51) $\displaystyle\partial_{t}R_{a\omega}$
$\displaystyle=-i(\omega+2\delta_{a}^{(0)}-J)R_{a\omega}+gR_{e},$ (52)

for $k=b$.

Formally integrating for $D_{\omega}(t)$, and using $M_{a\omega}(0)=0$, gives,
in the Wigner-Weisskopf approximation, that

$\partial_{t}M_{e}=-(\Gamma+i\omega_{b}^{(0)})M_{e}-gD(-ct,0),$ (53)

where $\Gamma\equiv 2\pi\varrho g^{2}$ (due to the continuum limit,
$\sum_{\omega}\rightarrow\varrho\int d\omega$), and
$D(z,t)\equiv\sum_{\omega}D_{\omega}(t)\exp(ik_{\omega}z)$ (where
$\omega=ck_{\omega}$). Also, we find that

$\displaystyle D(z,t)$ $\displaystyle=D(z-ct,0)$
$\displaystyle+\sqrt{2\pi\varrho\Gamma}\Theta(z)\Theta(t-z/c)M_{e}(t-z/c),$
(54)

and that

$\displaystyle M_{a}(z,t)$
$\displaystyle=\sqrt{2\pi\varrho\Gamma}\Theta(z)\Theta(t-z/c)$
$\displaystyle\times M_{e}(t-z/c)e^{-i\delta_{a}^{(0)}z/c}.$ (55)

The results for $R_{e}(t)$, $F(z,t)$ and $R_{a}(z,t)$ follow quite similarly,

$\partial_{t}R_{e}=-\left(\Gamma+i(\omega_{bJ}+\delta_{a}^{(0)})\right)R_{e}-gF(-ct,0),$
(56)

where $\omega_{bJ}\equiv\omega_{b}^{(0)}+J$,

$\displaystyle F(z,t)$ $\displaystyle=F(z-ct,0)e^{-i\delta_{a}^{(0)}t}$
$\displaystyle+\sqrt{2\pi\varrho\Gamma}\Theta(z)\Theta(t-z/c)R_{e}(t-z/c)e^{-i\delta_{a}^{(0)}z/c},$
(57)

and

$\displaystyle R_{a}(z,t)$
$\displaystyle=\sqrt{2\pi\varrho\Gamma}\Theta(z)\Theta(t-z/c)$
$\displaystyle\times R_{e}(t-z/c)e^{-i(2\delta_{a}^{(0)}-J)z/c}.$ (58)

The mutation probability reads

$p_{b_{1},b_{2}\rightarrow
b_{1},a_{2}}(t)=\sum_{\omega}|M_{a\omega}(t)|^{2}=\frac{1}{2\pi\varrho
c}\int_{-\infty}^{\infty}|M_{a}(z,t)|^{2}.$ (59)

By substituting Eq.(55) in (59), and changing variables, we find that

$p_{b_{1},b_{2}\rightarrow
b_{1},a_{2}}(t)=\Gamma\int_{0}^{t}|M_{e}(t^{\prime})|^{2}dt^{\prime}.$ (60)

Similarly, by substituting Eq.(58) in the replication probability, namely,

$p_{a_{1},b_{2}\rightarrow a_{1},a_{2}}(t)=\frac{1}{2\pi\varrho
c}\int_{-\infty}^{\infty}|R_{a}(z,t)|^{2}dz,$ (61)

we find that

$p_{a_{1},b_{2}\rightarrow
a_{1},a_{2}}(t)=\Gamma\int_{0}^{t}|R_{e}(t^{\prime})|^{2}dt^{\prime}.$ (62)

The integration above is performed with the solution of $R_{e}(t)$, that is,

$R_{e}(t)=-g\int_{0}^{t}F(-ct^{\prime},0)e^{-\left(\Gamma+i(\omega_{bJ}+\delta_{a}^{(0)})\right)(t-t^{\prime})}dt^{\prime}.$
(63)

Eqs.(17) and (18) allow $R_{e}(t)$ to be analytically obtained. Eq.(62) is a
key step for the quantum dissipative adaptation relation, Eq.(37), as
explained below.

Calculating the incoming work. Using integration by parts, we can rewrite
Eq.(38) as $\langle W_{\mathrm{in}}\rangle=-\int_{0}^{\infty}\langle
d(t)\partial_{t}E_{\mathrm{in}}(t)\rangle dt$. In the present model, this
results in

$\displaystyle\langle W_{\mathrm{in}}\rangle=-\int_{0}^{\infty}dt\ $
$\displaystyle(i\hbar
g)\sum_{\omega}(-i\omega)\langle\sigma_{a2}^{\dagger}(t)a_{\omega}\rangle
e^{-i\omega t}$
$\displaystyle+(-i\omega)\langle\sigma_{b2}^{\dagger}(t)b_{\omega}\rangle
e^{-i\omega t}$ $\displaystyle+\mbox{c.c.},$ (64)

where c.c stands for complex conjugate. Choosing $|1^{b}\rangle$ as the
initial state of the field implies that
$\langle\sigma_{a2}^{\dagger}(t)a_{\omega}\rangle=0$. The non-zero correlation
function gives

$\displaystyle\langle\sigma_{b2}^{\dagger}(t)b_{\omega}\rangle$
$\displaystyle=\langle
U^{\dagger}\sigma_{b2}^{\dagger}Ub_{\omega}|a_{n}\rangle_{1}|b_{n}\rangle_{2}|1^{b}\rangle$
(65) $\displaystyle=\langle
U^{\dagger}\sigma_{b2}^{\dagger}UF_{\omega}(0)|a_{n}\rangle_{1}|b_{n}\rangle_{2}|0\rangle$
(66) $\displaystyle=\langle
U^{\dagger}\sigma_{b2}^{\dagger}e^{-i\delta_{a}^{(0)}t}F_{\omega}(0)|a_{n}\rangle_{1}|b_{n}\rangle_{2}|0\rangle$
(67) $\displaystyle=\langle
U^{\dagger}e^{-i\delta_{a}^{(0)}t}F_{\omega}(0)|a_{n}\rangle_{1}|e_{n}\rangle_{2}|0\rangle$
(68) $\displaystyle=\langle
G_{a}(t)|\left(|a_{n}\rangle_{1}|e_{n}\rangle_{2}|0\rangle\right)e^{-i\delta_{a}^{(0)}t}F_{\omega}(0)$
(69) $\displaystyle=R_{e}^{*}(t)e^{-i\delta_{a}^{(0)}t}F_{\omega}(0).$ (70)

Finally,

$\langle W_{\mathrm{in}}\rangle=-\hbar
g\int_{0}^{\infty}R_{e}^{*}(t)e^{-i\delta_{a}^{(0)}t}\left(i\partial_{t}F(-ct,0)\right)+\mbox{c.c.}.$
(71)

We define $F(-ct,0)=F^{e}(-ct,0)\exp(-i\omega_{L}t)$, and obtain Eqs.(40) and
(41). To find Eq.(37), we get from Eq.(56) that

$\partial_{t}|R_{e}(t)|^{2}=-2\Gamma|R_{e}(t)|^{2}-2g\Re[R_{e}^{*}(t)e^{-i\delta_{a}^{(0)}t}F(-ct,0)].$
(72)

We substitute Eq.(72) in Eq.(62), using that $R_{e}(0)=R_{e}(\infty)=0$.
Finally, we identify the real part in Eq.(72) with that in the absorptive
term, Eq.(40), which follows from Eq.(71). This leads to Eq.(37),
independently of the choice for $F(-ct,0)$, that is, the initial photon
wavepacket.

##  V Data availability

Data sharing not applicable to this article as no datasets were generated or
analysed during the current study.

##  VI Code availability

Code availability not applicable to this article as no codes were generated or
analysed during the current study.

##  VII Author contributions

D. V. did the research, prepared the figures and wrote the paper.

##  VIII Competing interests

The author declares no competing interests.

##  IX acknowledgements

I thank Thiago Werlang and Thiago Guerreiro for comments on the manuscript.
This work was supported by the Serrapilheira Institute (Grant No.
Serra-1912-32056) and by the INCT de Informação Quântica, CNPq INCT-IQ
(465469/2014-0), Brazil.

##  X References

## References

  * (1) Schrödinger, E., What is Life? The Physical Aspect of the Living Cell, (Cambridge University Press, 1944).
  * (2) Rasmussen S. et al. Transitions from Nonliving to Living Matter. Science 303, 963-965 (2004).
  * (3) Cleland, C. E., The Quest for a Universal Theory of Life: Searching for Life As We Don’t Know It, (Cambridge University Press, 2019).
  * (4) England, J., Every Life Is on Fire: How Thermodynamics Explains the Origins of Living Things, (Basic Books, 2020).
  * (5) Goldenfeld, N., Biancalani, T. & Jafarpour, F. Universal biology and the statistical mechanics of early life, Phil. Trans. R. Soc. A 375, 20160341 (2017).
  * (6) Lenski, R. E., Ofria, C., Pennock, R. T. & Adami, C. The evolutionary origin of complex features. Nature 423, 139-144 (2003).
  * (7) Packard, N. et al. An Overview of Open-Ended Evolution: Editorial Introduction to the Open-Ended Evolution II Special Issue. Artificial Life 25, 93-103 (2019).
  * (8) Alvarez-Rodriguez, U., Sanz, M., Lamata, L. & Solano, E., Biomimetic Cloning of Quantum Observables. Sci. Rep. 4, 4910; 10.1038/srep04910 (2014).
  * (9) Alvarez-Rodriguez, U., Sanz, M., Lamata, L. & Solano, E., Artificial Life in Quantum Technologies. Sci. Rep. 6, 20956; 10.1038/srep20956 (2016).
  * (10) Alvarez-Rodriguez, U., Sanz, M., Lamata, L. & Solano, E., Quantum Artificial Life in an IBM Quantum Computer. Sci. Rep. 8, 14793; 10.1038/s41598-018-33125-3 (2018).
  * (11) Lamata L., Quantum machine learning and quantum biomimetics: A perspective. Mach. Learn.: Sci. Technol. 1, 033002 (2020).
  * (12) Carnall J. M. A. et al. Mechanosensitive self-replication driven by self-organization. Science 327, 1502-1506 (2010).
  * (13) England, J. L., Statistical physics of self-replication. J. Chem. Phys. 139, 121923 (2013).
  * (14) Ito, S. et al. Selective optical assembly of highly uniform nanoparticles by doughnut-shaped beams. Sci. Rep. 3, 3047; 10.1038/srep03047 (2013).
  * (15) Kondepudi, D., Kay, B. & Dixon, J. End-directed evolution and the emergence of energy-seeking behaviour in a complex system. Phys. Rev. E 91, 050902(R) (2015).
  * (16) England, J., Dissipative adaptation in driven self-assembly. Nat. Nanotech. 10, 919-923 (2015).
  * (17) Bachelard, N. et al. Emergence of an enslaved phononic bandgap in a non-equilibrium pseudo-crystal. Nat. Mater. 16, 808-814 (2017).
  * (18) Ropp, C., Bachelard, N., Barth, D., Wang, Y. & Zhang, X. Dissipative self-organization in optical space. Nat. Photon. 12, 739-743 (2018).
  * (19) Sarkar, S. & England, J. L., Design of conditions for self-replication. Phys. Rev. E 100, 022414 (2019).
  * (20) te Brinke, E. et al. Dissipative adaptation in driven self-assembly leading to self-dividing fibrils. Nat. Nanotech. 13, 849-855 (2020).
  * (21) Perunov, N., Marsland, R. A. & England, J. L. Statistical physics of adaptation. Phys. Rev. X 6, 021036; 10.1103/PhysRevX.6.021036 (2016).
  * (22) Kachman, T., Owen, J. A. & England, J. L. Self-organized resonance during search of a diverse chemical space. Phys. Rev. Lett. 119, 038001 (2017).
  * (23) Horowitz, J. M. and England, J. L. Spontaneous fine-tuning to environment in many-species chemical reaction networks. Proc. Natl. Acad. Sci. USA 114, 7565 (2017).
  * (24) Valente, D., Brito, F. & Werlang, T. Quantum dissipative adaptation. Comm. Phys. 4, 11; 10.1038/s42005-020-00512-0 (2021).
  * (25) Sadownik, J. W., Mattia, E., Nowak, P. & Otto, S. Diversification of self-replicating molecules. Nat. Chem. 8, 264-269 (2016).
  * (26) Adams, A., Zenil, H., Davies, P. C. W. & Walker, S. I. Formal Definitions of Unbounded Evolution and Innovation Reveal Universal Mechanisms for Open-Ended Evolution in Dynamical Systems. Sci. Rep. 7, 997; 10.1038/s41598-017-00810-8 (2017).
  * (27) Andrieux, D. & Gaspard, P. Nonequilibrium generation of information in copolymerization processes. Proc. Natl Acad. Sci. USA 105, 9516-9521 (2008).
  * (28) Jarzynski, C. The thermodynamics of writing a random polymer. Proc. Natl Acad. Sci. USA 105, 9451-9452 (2008).
  * (29) Parrondo, J. M. R., Horowitz, J. M. & Sagawa, T. Thermodynamics of information. Nat. Phys. 11, 131 (2015).
  * (30) Walker S. I. & Davies, P. C. W. The algorithmic origins of life. J. R. Soc. Interface 10: 20120869 (2013).
  * (31) Conte, T. et al. Thermodynamic Computing. Preprint at https://arxiv.org/abs/1911.01968 (2019).
  * (32) Wootters W. K. & Zurek, W. H., A single quantum cannot be cloned. Nature 299, 802 (1982).
  * (33) Buzek, V. and Hillery, M., Quantum copying: Beyond the no-cloning theorem. Phys. Rev. A, 54, 1844 (1996).
  * (34) Gisin, N. and Massar, S., Optimal Quantum Cloning Machines. Phys. Rev. Lett., 79, 2153 (1997).
  * (35) Valente, D. et al. Universal optimal broadband photon cloning and entanglement creation in one-dimensional atoms. Phys. Rev. A 86, 022333 (2012).
  * (36) Bryngelson, J. D. & Wolynes, P. G., Spin glasses and the statistical mechanics of protein folding. Proc. Natl. Acad. Sci. USA 84, 7524-7528 (1987).
  * (37) Valente, D. & Werlang, T., Frustration and inhomogeneous environments in relaxation of open chains with Ising-type interactions. Phys. Rev. E 102, 022114 (2020).
  * (38) Guerreiro, T. Quantum molecular robots. Quantum Sci. Technol. 6, 025006 (2021).
  * (39) Guerreiro, T. Molecular machines for quantum error correction. Preprint at https://arxiv.org/abs/2103.05184 (2021).
  * (40) Cohen-Tannoudji, C., Dupont-Roc, J. & Grynberg, G., Atom-Photon Interactions. (Wiley-VCH, 2004).
  * (41) Crooks, G. E. Entropy production fluctuation theorem and the nonequilibrium work relation for free energy differences. Phys. Rev. E 60, 2721 (1999).
  * (42) Kedia, H., Pan, D., Slotine, J.-J. & England, J. L. Drive-specific adaptation in disordered mechanical networks of bistable springs. Preprint at https://arxiv.org/abs/1908.09332 (2019).
  * (43) Valente, D., Brito, F., Ferreira, R. & Werlang, T. Work on a quantum dipole by a single-photon pulse. Opt. Lett. 43, 2644-2647 (2018).
  * (44) Glauber, R. J. Coherent and Incoherent States of the Radiation Field. Phys. Rev. 131, 2766 (1963).
  * (45) Valente, D., Arruda, M. F. Z. & Werlang, T. Non-Markovianity induced by a single-photon wave packet in a one-dimensional waveguide. Opt. Lett. 41, 3126-3129 (2016).
  * (46) Hillberry, L. E. et al. Entangled quantum cellular automata, physical complexity, and Goldilocks rules. Preprint at https://arxiv.org/abs/2005.01763 (2020).
  * (47) Vetsigian, K., Woese, C. & Goldenfeld, N., Collective evolution and the genetic code. Proc. Natl. Acad. Sci. USA 103, 10696-10701 (2006).
  * (48) Duarte, O. S. & Caldeira, A. O. Effective Coupling between Two Brownian Particles. Phys. Rev. Lett. 97, 250601 (2006).
  * (49) Duarte, O. S. & Caldeira, A. O. Effective quantum dynamics of two Brownian particles. Phys. Rev. A 80, 032110 (2009).
  * (50) Valente, D. M. & Caldeira, A. O. Thermal equilibrium of two quantum Brownian particles. Phys. Rev. A 81, 012117 (2010).


(cvpr) Package cvpr Warning: Package ‘hyperref’ is not loaded, but highly
recommended for camera-ready version

# Enhancing Multi-Camera People Tracking with
Anchor-Guided Clustering and Spatio-Temporal Consistency ID Re-Assignment

Hsiang-Wei Huang1∗ Cheng-Yen Yang1∗ Zhongyu Jiang1 Pyong-Kun Kim2
Kyoungoh Lee2 Kwangju Kim2 Samartha Ramkumar1 Chaitanya Mullapudi1
In-Su Jang2 Chung-I Huang3 Jenq-Neng Hwang1
1 Information Processing Lab, University of Washington, USA
2 Electronics and Telecommunications Research Institute, South Korea
3 National Center for High-Performance Computing, Taiwan

###### Abstract

Multi-camera multiple people tracking has become an increasingly important
area of research due to the growing demand for accurate and efficient indoor
people tracking systems, particularly in settings such as retail, healthcare
centers, and transit hubs. We proposed a novel multi-camera multiple people
tracking method that uses anchor-guided clustering for cross-camera re-
identification and spatio-temporal consistency for geometry-based cross-camera
ID reassigning. Our approach aims to improve the accuracy of tracking by
identifying key features that are unique to every individual and utilizing the
overlap of views between cameras to predict accurate trajectories without
needing the actual camera parameters. The method has demonstrated robustness
and effectiveness in handling both synthetic and real-world data. The proposed
method is evaluated on CVPR AI City Challenge 2023 dataset, achieving IDF1 of
95.36% with the first-place ranking in the challenge. The code is available
at: https://github.com/ipl-uw/AIC23_Track1_UWIPL_ETRI.

##  1 Introduction

Multi-people tracking, which involves detecting and monitoring human movement,
has become an essential tool in various industries. Such tracking utilizes
techniques like sensors, cameras, and deep learning algorithms to track
people’s positions, motions, and directions with time. It plays a critical
role in ensuring security surveillance and bunisness analytics as well as
works with closed-circuit television (CCTV) to prevent accidents.

Figure 1: Illustration of Multi-Camera People Tracking (MCPT). The task
involves detecting and tracking the same individuals across multiple cameras.
The goal is to maintain the identity of each individual and their trajectory
across different views, while dealing with challenges such as occlusion and
camera viewpoint variations.  ††footnotetext: * indicates equal contributions.

The demand for indoor people tracking has increased in recent years. Besides
tracking people’s movements, indoor environments necessitate advanced
technology to detect and monitor people’s activities. In the healthcare
sector, it is vital for tracking patients and staff, equipment and inventory,
and optimizing workflows [47]. Similarly, the retail industry can enhance the
shopping experience by analyzing customer behavior, and security and safety
can leverage this technology to detect and respond to emergencies. Moreover,
the recent COVID-19 pandemic has underscored the need for quarantine measures
such as social distancing [31].

However, due to the privacy issue, the data are limited for researching deep
learning based people tracking methods. Therefore, researchers are exploring
the use of synthetic imagery as an alternative[37, 39, 23]. Synthetic imagery
mimics real-world footage and can be used as training data for machine
learning models to create large training datasets in a cost-effective manner.
This approach also benefits consistency and predictability, enabling
customization for specific user needs, such as class balance considerations.
Moreover, it can enhance the generalization capability of models that are
challenging to train with limited real-world data and address privacy
concerns. However, since synthetic data may not capture the full complexity
and variability of real-world data, models trained on this data could be
biased or inaccurate.

The AI City Challenge recently released data on indoor people tracking using
synthetic videos under mutli-camera settings. The dataset focuses on multi-
camera cross-view scenarios. So, we develop a multi-camera people tracking
method (MCPT) composed of three main components: single-camera tracking,
anchor-guided clustering for multi-camera re-identification, and 3D-based
spatio-temporal consistency ID re-assignment for post-processing. Our proposed
method outperformed all others in AI City Challenge Track 1, resulting in the
best performance. Therefore, we assert three main contributions in this paper:

  * •

We present a robust anchor-guided clustering method for multi-camera people
tracking and re-identification.

  * •

We leverage the spatio-temporal consistency of each track for post-processing
enabled by self-camera calibrations. which can significantly improve the
tracking accuracy of people with similar appearances.

  * •

Achieved the best performance with an IDF1 of $95.36$, in the 2023 AI City
Challenge Track 1 on the public testing set which consists of data from real
and synthetic multi-camera settings.

The subsequent sections of the paper are structured as follows: Section 2
provides an overview of related works. Section 3 describes the proposed
method. Section 4 presents the results of the detailed implementation and
experiment results. Lastly, Section 5 provides the discussion and conclusions
drawn from the study.

##  2 Related Works

###  2.1 Re-Identification

Since the the advent of deep learning technology, CNN features have been used
dominantly [48], studies on person re-identification have been conducted from
three main categories: network structure, loss definition, and sampling
method.

Network Structure. [2] extracted features from each of the two images and
computed relationships between them to train a model. [13] reviewed prior
research on feature drop and localizing different body parts as their proposed
model included a global branch for encoding global salient representations and
a feature-dropping branch for randomly dropping the same region of all input
maps in a batch. [52] proposed a CNN architecture that leverages multiple
scales with different receptive field sizes and dynamically fused them using
channel-wise adaptive aggregation. They demonstrated that their system was
significantly smaller than previous models by utilizing factorized
convolutions in the building blocks. [24] highlighted the tokens generated by
the region level and introduced a region-based feature pooling method for
obtaining more granular areas of interest.

Loss Definition. [33] introduced a new loss called triplet loss, which has
inspired numerous subsequent studies in re-id and metric learning. [17]
demonstrated that a variant of triplet loss outperformed other losses, which
contradicted the prevailing belief that triplet loss was inferior to surrogate
loss functions.

Sampling Method. Early work typically used random sampling[4, 11]. [33]
proposed semi-hard negative mining, which required a large batch size. [34]
utilized hard negative mining in a siamese network. [43] focused on selecting
more informative and stable examples than traditional approaches by employing
a margin-based loss.

Figure 2: The overall pipeline of our Multi-Camera Multi-people Tracking
(MCMT) framework. (1) Single-Camera Tracking is first conducted with a
standard tracking-by-detection scheme, where the extracted frames are fed into
a detector and a feature extractor to get the detections and re-id features
for the preliminary tracking. (2) Then Anchor-Guided Clustering will assign a
global ID as well as fixing the ID switches for each trajectory. (3) Lastly,
Spatio-Temporal Consistency ID Reassignment will utilize the 2D human pose
with camera self-calibration to reproject on the map for final post-
processing.

###  2.2 Monocular Multi-Object Tracking

The field of Monocular Multi-Object Tracking has advanced significantly since
the advent of deep learning technology, with studies typically following the
tracking-by-detection paradigm [3, 8, 22, 49, 6, 42, 1]. This involves
detecting the location of objects to be tracked in each frame and associating
them based on the similarity of their real and predicted locations, which is
calculated using motion information to produce a trajectory. Early studies [7]
explored the use of the Intersection of Union (IoU) metric to improve tracking
speed. Another work [6] focused on improving tracking performance with simple
motion prediction using a Kalman filter to predict object location in the next
frame. While this approach is fast and effective in simple environments, it
has limitations in complex scenarios with a high number of objects to track.
Other studies, such as [42], focused on using appearance information to match
objects, by extracting feature vectors from detected objects and comparing
them across frames.

Many studies [5, 38, 44, 40, 14, 51, 50, 21, 45] have proposed various
approaches to enhance Multi-Object Tracking (MOT) performance with low
computational cost. [5] addressed the issue of the lack of a tracking dataset
compared to the detection dataset. Meanwhile, [38] thought that bounding box
level tracking is saturating and introduced pixel-level tracking as a way to
improve performance. Other studies, such as [44], focused on learning instance
embeddings using 2D point cloud representations to avoid background features.
[40] proposed a real-time Joint Detection and Embedding (JDE) system for MOT,
which outperformed the Separate Detection and Embedding (SDE) system used in
previous studies. [51] treated MOT as a multi-task learning problem of object
detection and association and presented detailed designs to avoid competition
between these tasks. [14] demonstrated that simple designs could perform well
with a few additional tricks. [50] proposed a method that associated almost
every detection box, rather than just the high score ones. [9] computed a
virtual trajectory over the occlusion period based on object observations.
Finally, [1] proposed some bag of tricks to achieve high performance on public
MOT datasets.

###  2.3 Multi-Camera Multi-Object Tracking

Following in the development of single-camera multi-object tracking, Multi-
camera multi-object tracking has been studied actively. Previous studies were
based on the graph-based approaches to associate across frames and cameras
[10, 41, 18, 16]. As is the case with single cameras, the deep feature was
soon introduced in the multi-cameras [32, 19, 35]. [32] proposed an adaptive
weight loss and hard-identity mining scheme for learning better features. [19,
20] proposed the trajectory-based camera link model including deep feature re-
identification. They utilized the TrackletNet Tracker (TNT) to generate the
moving trajectories and the camera link model to constrain the order by the
spatial and temporal information. [46] proposed a unified framework that can
effectively adopt monocular 2D bounding boxes and 2D poses jointly to produce
robust 3D trajectories to track across mutli-camera with overlapping views.
[29] proposed multi-camera multiple object tracking approach based on a
spatial-temporal lifted multicut formulation utilizing 3D geometry projection.

The release of many public datasets has driven progress in this field. [26,
36, 27] released a city-scale traffic camera dataset consisting of more than 3
hours of HD videos. [15] proposed a novel method to construct a large-scale
multi-camera tracking dataset called MMPTrack to alleviate the occlusion
issue. They utilized depth and RGB cameras to build 3D tracking results and
projected them to create 2D tracking results. This helped to build a reliable
benchmark for multi-camera multi-object tracking systems in cluttered and
crowded environments.

Figure 3: The process for anchor-guided multi-camera people tracking involves
periodically sampling frame-based appearance features from each camera, and
performing hierarchical clustering to obtain anchors with corresponding
feature banks. Single camera tracking is then performed to obtain preliminary
trajectories with ID switches denoted in different colors in the same row.
Each detection from the preliminary trajectories are further assigned a global
ID by the anchor using the Hungarian algorithm. Finally, sliding window
majority vote is performed to obtain the final trajectories.

##  3 Methods

###  3.1 Single-Camera Tracking

Single-camera tracking algorithms generally follow the tracking-by-detection
paradigm, which involves using an independent detector on the input image and
then an association algorithm to link bounding boxes across frames. To
incorporate appearance features into the tracking process, we employ the Re-ID
version of BoT-SORT [1] as a single-camera tracking algorithm to obtain
preliminary object trajectories. BoT-SORT is designed to leverage both object
motion and appearance for the association, which has led to achieving state-
of-the-art tracking performance on several multi-object tracking benchmarks.

###  3.2 Anchor-Guided Clustering

Upon completion of single-camera tracking, preliminary trajectories are often
obtained, which may suffer from numerous ID switches due to occlusion,
unaligned object IDs of the same identity between different cameras in the
same scenario, and a lack of a re-entry handling method in the camera field of
view. The absence of such a method makes it difficult to assign a unique
object ID to identities that re-enter the camera field of view after exiting
for a certain time.

To address all three of these problems simultaneously, we propose an effective
method called anchor-guided clustering and global ID assignment. Our method
elegantly enables the assignment of the same unique object ID to re-entering
identities, and it can handle occlusion-induced ID switches and unaligned
object IDs of the same identity between different cameras in the same
scenario.

The proposed method involves periodic sampling of detection appearance
features for a certain number of frames from each camera in the same scene.
After the sampling process is completed, hierarchical clustering is performed
to obtain several anchors, where each anchor contains multiple features that
represent the same identity’s appearance feature under different detection
sizes, lighting conditions, and rotation angles. Each anchor has a unique ID
that will be used as the identity’s global ID in multi-camera tracking.

Subsequently, the detections in the same frame $t$ and the anchors will
perform the Hungarian algorithm with cost described in the following formula:

$\text{cost}(d_{i,t},a_{j})=1-\frac{1}{k}\sum_{l=1}^{k}\frac{d_{i,t}\cdot
a_{j,l}}{\left|d_{i,t}\right|\left|a_{j,l}\right|}$ (1)

We use the cosine distance between the detection’s appearance feature
$d_{i,t}$ (where $t$ is the frame ID of the detection) and each appearance
feature vector $a_{j,k}$ in the anchor $a_{j}$ as the cost for global ID
assignment. Specifically, we compute the average of the cosine distances
between $d_{i,t}$ and all $a_{j,k}$ to obtain the final cost. Note that each
anchor contains multiple appearance feature vectors, denoted as $a_{j,l}$, and
the number of these vectors are denoted as $k$.

After performing the Hungarian algorithm, each single-camera tracking
trajectory obtains a global ID list with the same length as the original
trajectory. To assign the final global ID, we use a sliding window majority
voting approach. This method effectively fixes ID switches in single-camera
tracking, assigns global IDs robustly by considering multiple frames, and
correctly re-identifies individuals who re-enter the camera field of view by
assigning them the same global ID.

###  3.3 Spatio-Temporal Consistency ID Reassignment

Assuming that the multi-view videos are synchronized and overlapped, it is
expected that a person’s trajectories will exhibit both spatial and temporal
consistency in terms of position and motion across all views. Therefore, by
relying on such cross-view consistency, it becomes possible to match the 2D
tracklets under different views to the same person and further re-assigned the
incorrect global IDs in the previous tracking stage either due to similar
appearance or heavy occlusion.

Given the tracking results after the anchor-guided clustering ID assignment,
where $X^{k}_{t,id}\in\mathbb{R}^{4}$ representing the 2D information of each
detection $(x,y,w,h)$ under $k$-th camera-view at frame $t$. The function
$F_{g}$ takes any detection $X^{k}_{t,id}$ as input and outputs 2D coordinates
in image space representing the ground-plane location on which each target is
located whenever both the left and right ankle keypoints $(x_{la},y_{la})$ and
$(x_{ra},y_{ra})$ are available:

$F_{g}(X^{k}_{t,id})=\begin{cases}(\frac{(x_{la}+x_{ra})}{2},\frac{(y_{la}+y_{ra})}{2})&\text{if
}c_{la},c_{ra}\geq\tau_{pose}\\\ (x+w/2,y+h)&\text{otherwise}\end{cases}$ (2)

where $c_{la}$ and $c_{ra}$ are the confidence scores of the keypoints
predicted by the 2D pose estimator. The $\tau_{pose}$ is the thresholding that
controls whether we decided the top-down location of each target by pose-based
analysis or simply compute from the bounding box’s information.

Then, given the homography matrix $H^{k}\in\mathbb{R}^{3\times 3}$ of $k$-th
camera-view obtained via camera self-calibration, we can obtained the top-down
coordinate of any detection $X^{cam}_{t}$ by reprojecting the ground-plane
coordinates of each detections in image space using:

$F_{3D}(X^{k}_{t,id})=H^{k}\cdot F_{g}(X^{k}_{t,id})^{T},$ (3)

we hereby used the notation $\hat{X}^{k}_{t,id}\in\mathbb{R}^{6}$ representing
the 2D information of each detection $(x,y,w,h)$ and 3D information
$(x_{3D},y_{3D})$ under $k$-th camera-view at frame $t$.

The spatial consistency in our work refers to the level of agreement of the
top-down location of each ID from all of the camera views. With
$\hat{X}^{k}_{t,id}$ representing the 3D information $(x_{3D},y_{3D})$ under
$k$-th camera-view at frame $t$, the spatial consistency across multi-view is
defined as:

$D_{spatial}(\hat{X}^{k}_{t,id},t,id)=||\frac{1}{N}\sum_{l\neq
k}\hat{X}^{l}_{t,id}-\hat{X}^{k}_{t,id}||^{2},$ (4)

as it is worth mentioning that we will exclude the outliers identified by the
function $O(\cdot)$ prior to computing the average coordinates. These outliers
are typically detections with similar appearances that are likely to be
misclassified as a different identity in our previous single-camera tracking
or anchor-guided clustering. Finally, we use a self-defined confidence score
as a threshold to determine which detections we will reassign identities to:

$conf_{i\rightarrow
j}(\hat{X}^{k}_{t,i})=1-\frac{D_{spatial}(\hat{X}^{k}_{t,i},t,i)}{D_{spatial}(\hat{X}^{k}_{t,i},t,j)}$
(5)

In addition, if we regard any sudden changes in the locations of tracks as
irregular, we can enhance their consistency over time by employing a
straightforward method of performing a weighted summation of track locations
within a sliding window. This generates a smoothed location estimate that
captures the general movement of the track over time. This method can be
highly beneficial in scenarios where the data contains noise or missing
information that may result in abrupt changes in track locations, which are
not due to actual movements but rather measurement errors. We can establish
temporal consistency by replacing the initial average coordinate calculation
with a weighted variant. This involves assigning weights to the coordinates
based on their relevance to the sliding time window, and computing the
weighted average instead.

##  4 Implementation Details

###  4.1 Dataset

The Multi-camera People Tracking dataset consists of multiple camera feeds
captured in various real-world and synthetic settings. The real-world data
were collected from a warehouse while the large-scale synthetic data were
synthesized using the NVIDIA Omniverse Platform across six different indoor
scenes. The videos are in high-resolution 1080p feeds at 30 frames per second
with tracking annotations across camera views. However, it is important to
note that there are 10 and 5 synthetic sets in the training and validation
data respectively while there are 6 synthetic sets plus an additional real set
in the testing data. Our experiments exclusively used the data from the
dataset and did not incorporate any external data, whether real or synthetic.

Furthermore, for the real data in the testing set, we only employed pre-
trained detector, 2D pose estimator, and re-id feature extractor to
demonstrate the effectiveness and robustness of our proposed multi-camera
people tracking method.

Dataset Split | # of Scenes (Cams) | # of Frames | # of Dets
---|---|---|---
Training (syn) | 10 (58) | 1,065,602 | 4,375,736
Validation (syn) | 5 (28) | 504,252 | 1,950,917
Testing (real) | 1 (7) | 388,671 | -
Testing (syn) | 6 (36) | 648,360 | -
Table 1: Basic information of the Multi-camera People Tracking dataset
presented in 2023 AI City Challenge Track 1. The annotations of the testing
data are and will most likely remain private therefore no accurate number of
total detections can be provided.

###  4.2 Evaluation Metrics

We adopt the mean Average Precision (mAP) for detection-related tasks while
using the Rank-1 accuracy for Re-ID tasks. As for the multi-camera tracking,
we will use the IDF1 score, which measures the ratio of correctly identified
detections over the average number of ground-truth and computed detections.
The challenge submission platform also provided with other MOT-related
evaluation measures, such as IDF1, IDP, IDR, precision (detection), and recall
(detection). Other MOTChallenge evaluation measures, such as MOTA, MOTP, MT,
and FAR will not only be used as self-evaluation metrics.

###  4.3 Camera Calibration

The WILDTRACK and MMP-Track datasets provide camera calibration files with the
pinhole camera model with both extrinsic and intrinsic parameters for each
camera given. However, in Challenge Track 1: Multi-Camera People Tracking
(MCPT), the calibration is not provided with the dataset but the top-down view
map is available for each subset.

We choose correspondences between the camera-view frames and the top-down view
map to compute the homography matrices $H$ for each camera in order to project
the ground-plane location for each target from the 2D image space to the 3D
world space. We adopt the semi-automatic camera calibration based on the
Perspective-n-Point method to compute the homography matrix for each camera.
For each camera view, we manually select 6 to 12 pairs of points as input,
using the approaches including (1) a Least-Squares method using all the
points, (2) a RANSAC-based robust method, (3) a Least-Median-of-Squares method
or (4) a PROSAC-based robust method.

###  4.4 Detector

Synthetic Data. With the high frame rate at 30, the training and validation
sets consist of 1M and 500k high-resolution frames each from 15 sequences with
a total of 6.3M detections. In order to maintain a balance between the long
training elapsed time and the detection performance, we eventually decide to
conduct our first-stage of pretraining under a sampling rate of 20 using all
the scenes and second-stage of fine-tuning under a sampling rate of 15 for
specific scene.

We use YOLOv7 as our backbone and COCO-pretrained weights from [12] for
initialization. Our first-stage pretrained model were trained for 60 epochs
with a batch size of 8 and an initial learning rate of 0.0025. Our second-
stage scene-specific fine-tuned models were trained for 10 epochs with a batch
size of 8 and an initial learning rate of 0.00025.

Real Data. The only real world data in the dataset is sequence S001 from test
split. Although supervised and unsupervised domain adaptation methods on
object detection [25, 30] show promising results on label-scarce target
datasets. The lack of corresponding real data in training or validation split
make it difficult for us to evaluate the performance of cross-domain human
detection. Therefore for the purpose of challenge, we directly employ the
public available pretrained YOLOX_x model from [50] train on CrowdHuman,
MOT17, Cityperson and ETHZ.

This allows us to have a benchmark performance for cross-domain human
detection on the dataset, but further research and experimentation would be
necessary to improve the performance of the models on the dataset. It is also
important to note that the lack of real-world data in the dataset may limit
its applicability to certain real-world use cases, and therefore, collecting
more diverse and representative data would be necessary for the dataset to be
more widely applicable.

2D Pose Estimation. Since there are no 2D pose annotations in the dataset, we
directly impose the top-down human pose estimation method, HigherHRNet, using
pre-trained weights from [15]. The inputs are cropped out based on the
bounding boxes predict from the YOLO detector, then 17 keypoints are estimated
under COCO format.

Figure 4: Top: Self-camera calibration visualizing the estimated ground plane
(red grid lines) for different camera views for the testing sequence S001.
Bottom: Field of view for different cameras projected on the top-down map for
the testing sequence S001 .

###  4.5 Re-ID Model

In our multi-camera people tracking system, we have chosen OSNet as the person
re-identification (ReID) model. The OSNet architecture has shown to be
effective in person ReID tasks using the unified aggregation gate to fuse the
features from different scales and has achieved state-of-the-art performance
on several benchmark datasets.

Synthetic Data. The ReID training data is sampled from the training and
validation set of the 2023 AI City Challenge Track1 dataset. We random sample
each trajectory and divided the samples into training, testing, and query
sets. The ReID dataset used in our training process contains 56,181 training
images, 17,027 testing images, and 2,846 query images from different cameras
and scenes. The OSNet is trained for 60 epochs, Adam optimizer, and 0.0003
learning rate with data augmentation like random flip. The final model
achieves 97.9% Rank-1 accuracy and 97.6% mAP on the sampled testing set.

Real Data. Due to the lack of real-world data in the training set of the 2023
AI City Challenge Track1 dataset, several pre-trained models on other human
Re-ID datasets are used for the multi-camera tracking in the real-world
scenario in the testing set. Three different model architectures are used,
including OSNet, OSNet-IBN, and OSNet-AIN. A total of five models pre-trained
on different dataset combinations are used, the model architectures and the
pre-trained dataset used can be found in the following table. The features
extracted from these models are directly concatenated together for the use of
single-camera tracking and multi-camera tracking.

Model | Re-ID Dataset
---|---
OSNet | Market1501
OSNet | MSMT17
OSNet | Market1501 + CUHK03 + MSMT17
OSNet-IBN | Market1501 + CUHK03 + MSMT17
OSNet-AIN | Market1501 + CUHK03 + MSMT17
Table 2: Pre-trained models used for re-id on the real scene.

###  4.6 Tracking

Single Camera Tracking. For filtering the detection results, the synthetic
scenario tracking has the high score threshold of 0.6 and low score threshold
of 0.1. For the real world scenario, the high score threshold is 0.6 while the
low score is carefully fine-tuned in each camera. All the other parameters
used in the single camera tracking is the default parameters of BoT-SORT.
Since the camera is stationary, the camera motion compensation part is removed
to reduce computational cost.

Figure 5: Visualization of our final tracking results on the testing synthetic
dataset.

Mutlil-Camera Tracking. There are several parameters in the multi-camera
tracking system, including the hierarchical clustering threshold and the
length of majority vote sliding windows. The hierarchical clustering threshold
is carefully fine-tuned to make sure the anchor cluster results are accurate.
The length of sliding windows should be big enough to achieve robustness in
the voting process, while it can not be too big so that the ID switch can not
be fixed immediately after ID switch happened in the single camera tracking.
In our final system, we use the majority vote length of 15 to achieve balance
between ID assignment robustness and the ability for ID correction in single
camera tracking. Finally, linear interpolation is performed to all the
tracking results before submission.

###  4.7 Results on AI City Challenge

Several methods with different models and post-processing are evaluated on the
public test set of the 2023 AI City Challenge Track 1 [28] as shown in Table
3. Our baseline method is the anchor-guided clustering multi-camera people
tracking approach without any spatial-temporal re-assignment, which achieves
an IDF1 score of 89.57%. We then conduct experiments with three variations of
the method: using the Fine-Tuned detector models for each specific scene (FT),
introducing Spatio-Temporal Consistency Re-Assignment (STCRA) into the
framework, and iterative refinement of the latter technique, named iterative
spatio-temporal consistency re-assignment (i-STCRA). For i-STCRA, which is our
final submission, we use a $k=3$ with an ascending confidence score
thresholding and a descending outlier thresholding to ensure that the re-
assignments are stricter after each iteration.

Our experiments demonstrate that each variation leads to improved performance,
with the best result achieved by the method with spatio-temporal consistency
iterative re-assignment, obtaining an IDF1 score of 95.36, IDP score of 95.83,
and IDR score of 94.88 ranking the first-place among 27 teams as shown in
Table 4.

Method | IDF1 | IDP | IDR | Precision | Recall
---|---|---|---|---|---
Baseline | 89.57 | 91.89 | 87.36 | 92.79 | 88.21
\+ FT | 92.98 | 92.01 | 93.97 | 92.83 | 94.81
\+ FT + STCRA | 93.62 | 92.90 | 94.35 | 93.61 | 95.08
\+ FT + i-STCRA | 95.36 | 95.83 | 94.88 | 96.44 | 95.49
Table 3: The experimental results on the public test set of Track 1. Ranking | Team ID | Team Name | IDF1
---|---|---|---
1 | 6 | UWIPL_ETRI (ours) | 95.36
2 | 9 | HCMIU-CVIP | 94.17
3 | 41 | AILab | 93.31
4 | 51 | FraunhoferIOSB | 92.84
5 | 113 | hust432 | 92.07
6 | 133 | ctcore | 91.09
7 | 34 | Team 34 | 91.04
8 | 82 | PersonMatching | 89.81
9 | 151 | AIO2022_VGU | 89.68
10 | 38 | NetsPresso | 86.76
11 | 47 | MCPRL_MOT | 74.47
12 | 24 | Sertis Vision Lab | 71.22
Table 4: Leaderboard of Track 1 in the AICity Challenge 2023: Multi-Camera
People Tracking. Our proposed method obtained an IDF1 score of 95.36 ranking
in the first-place.

##  5 Conclusion

We proposed a multi-camera people tracking framework that assigns global ID
using anchor-based clustering method then calibrates them using spatio-
temporal consistency with the self-calibration of cameras. Our approach can
successfully improve the accuracy of tracking by identifying key features that
are unique to every individual and utilizing the overlap of views between
cameras to predict accurate trajectories without needing the actual camera
parameters. Experiments and results on a multi-camera dataset with various
real and synthetic scenes demonstrated the effectiveness and robustness of our
work. Our proposed method ranked first on the public test set of 2023 AI City
Challenge Track 1 in IDF1.

##  6 Acknowledgement

This work was supported by Electronics and Telecommunications Research
Institute (ETRI) grant funded by the Korean government. (23ZD1120, Regional
Industry IT Convergence Technology Development and Support Project). We also
want to acknowledge and thank National Center for High-Performance Computing
from Taiwan for providing the computing resources.

## References

  * [1] Nir Aharon, Roy Orfaig, and Ben-Zion Bobrovsky.  Bot-sort: Robust associations multi-pedestrian tracking.  arXiv preprint arXiv:2206.14651, 2022.
  * [2] Ejaz Ahmed, Michael Jones, and Tim K Marks.  An improved deep learning architecture for person re-identification.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3908–3916, 2015.
  * [3] Mykhaylo Andriluka, Stefan Roth, and Bernt Schiele.  People-tracking-by-detection and people-detection-by-tracking.  In 2008 IEEE Conference on computer vision and pattern recognition, pages 1–8. IEEE, 2008.
  * [4] Sean Bell and Kavita Bala.  Learning visual similarity for product design with convolutional neural networks.  ACM transactions on graphics (TOG), 34(4):1–10, 2015.
  * [5] Philipp Bergmann, Tim Meinhardt, and Laura Leal-Taixe.  Tracking without bells and whistles.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 941–951, 2019.
  * [6] Alex Bewley, Zongyuan Ge, Lionel Ott, Fabio Ramos, and Ben Upcroft.  Simple online and realtime tracking.  In 2016 IEEE international conference on image processing (ICIP), pages 3464–3468. IEEE, 2016.
  * [7] Erik Bochinski, Volker Eiselein, and Thomas Sikora.  High-speed tracking-by-detection without using image information.  In 2017 14th IEEE international conference on advanced video and signal based surveillance (AVSS), pages 1–6. IEEE, 2017.
  * [8] Michael D Breitenstein, Fabian Reichlin, Bastian Leibe, Esther Koller-Meier, and Luc Van Gool.  Robust tracking-by-detection using a detector confidence particle filter.  In 2009 IEEE 12th International Conference on Computer Vision, pages 1515–1522. IEEE, 2009.
  * [9] Jinkun Cao, Xinshuo Weng, Rawal Khirodkar, Jiangmiao Pang, and Kris Kitani.  Observation-centric sort: Rethinking sort for robust multi-object tracking.  arXiv preprint arXiv:2203.14360, 2022.
  * [10] Weihua Chen, Lijun Cao, Xiaotang Chen, and Kaiqi Huang.  An equalized global graph model-based approach for multicamera object tracking.  IEEE Transactions on Circuits and Systems for Video Technology, 27(11):2367–2381, 2016.
  * [11] Sumit Chopra, Raia Hadsell, and Yann LeCun.  Learning a similarity metric discriminatively, with application to face verification.  In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), volume 1, pages 539–546. IEEE, 2005.
  * [12] MMYOLO Contributors.  MMYOLO: OpenMMLab YOLO series toolbox and benchmark.  https://github.com/open-mmlab/mmyolo, 2022.
  * [13] Zuozhuo Dai, Mingqiang Chen, Xiaodong Gu, Siyu Zhu, and Ping Tan.  Batch dropblock network for person re-identification and beyond.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 3691–3701, 2019.
  * [14] Yunhao Du, Zhicheng Zhao, Yang Song, Yanyun Zhao, Fei Su, Tao Gong, and Hongying Meng.  Strongsort: Make deepsort great again.  IEEE Transactions on Multimedia, 2023.
  * [15] Xiaotian Han, Quanzeng You, Chunyu Wang, Zhizheng Zhang, Peng Chu, Houdong Hu, Jiang Wang, and Zicheng Liu.  Mmptrack: Large-scale densely annotated multi-camera multiple people tracking benchmark.  In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 4860–4869, 2023.
  * [16] Yuhang He, Xing Wei, Xiaopeng Hong, Weiwei Shi, and Yihong Gong.  Multi-target multi-camera tracking by tracklet-to-target assignment.  IEEE Transactions on Image Processing, 29:5191–5205, 2020.
  * [17] Alexander Hermans, Lucas Beyer, and Bastian Leibe.  In defense of the triplet loss for person re-identification.  arXiv preprint arXiv:1703.07737, 2017.
  * [18] Martin Hofmann, Daniel Wolf, and Gerhard Rigoll.  Hypergraphs for joint multi-view reconstruction and multi-object tracking.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3650–3657, 2013.
  * [19] Hung-Min Hsu, Yizhou Wang, and Jenq-Neng Hwang.  Traffic-aware multi-camera tracking of vehicles based on reid and camera link model.  In Proceedings of the 28th ACM International Conference on Multimedia, pages 964–972, 2020.
  * [20] Hsiang-Wei Huang, Cheng-Yen Yang, and Jenq-Neng Hwang.  Multi-target multi-camera vehicle tracking using transformer-based camera link model and spatial-temporal information.  arXiv preprint arXiv:2301.07805, 2023.
  * [21] Hsiang-Wei Huang, Cheng-Yen Yang, Samartha Ramkumar, Chung-I Huang, Jenq-Neng Hwang, Pyong-Kun Kim, Kyoungoh Lee, and Kwangju Kim.  Observation centric and central distance recovery for athlete tracking.  In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 454–460, 2023.
  * [22] Zdenek Kalal, Krystian Mikolajczyk, and Jiri Matas.  Tracking-learning-detection.  IEEE transactions on pattern analysis and machine intelligence, 34(7):1409–1422, 2011.
  * [23] Adam Kortylewski, Andreas Schneider, Thomas Gerig, Bernhard Egger, Andreas Morel-Forster, and Thomas Vetter.  Training deep face recognition systems with synthetic data.  arXiv preprint arXiv:1802.05891, 2018.
  * [24] Kyoungoh Lee, In-Su Jang, Kwang-Ju Kim, and Pyong-Kun Kim.  Reet: Region-enhanced transformer for person re-identification.  In 2022 18th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pages 1–8. IEEE, 2022.
  * [25] Wanyi Li, Fuyu Li, Yongkang Luo, and Peng Wang.  Deep domain adaptive object detection: a survey.  CoRR, abs/2002.06797, 2020.
  * [26] Milind Naphade, Ming-Ching Chang, Anuj Sharma, David C Anastasiu, Vamsi Jagarlamudi, Pranamesh Chakraborty, Tingting Huang, Shuo Wang, Ming-Yu Liu, Rama Chellappa, et al.  The 2018 nvidia ai city challenge.  In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 53–60, 2018.
  * [27] M. Naphade, S. Wang, D. C. Anastasiu, Z. Tang, M. Chang, Y. Yao, L. Zheng, M. Shaiqur Rahman, A. Venkatachalapathy, A. Sharma, Q. Feng, V. Ablavsky, S. Sclaroff, P. Chakraborty, A. Li, S. Li, and R. Chellappa.  The 6th ai city challenge.  In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 3346–3355. IEEE Computer Society, June 2022\.
  * [28] Milind Naphade, Shuo Wang, David C. Anastasiu, Zheng Tang, Ming-Ching Chang, Yue Yao, Liang Zheng, Mohammed Shaiqur Rahman, Meenakshi S. Arya, Anuj Sharma, Qi Feng, Vitaly Ablavsky, Stan Sclaroff, Pranamesh Chakraborty, Sanjita Prajapati, Alice Li, Shangru Li, Krishna Kunadharaju, Shenxin Jiang, and Rama Chellappa.  The 7th AI City Challenge.  In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2023.
  * [29] Duy MH Nguyen, Roberto Henschel, Bodo Rosenhahn, Daniel Sonntag, and Paul Swoboda.  Lmgp: Lifted multicut meets geometry projections for multi-camera multi-object tracking.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8866–8875, 2022.
  * [30] Poojan Oza, Vishwanath A. Sindagi, Vibashan VS, and Vishal M. Patel.  Unsupervised domain adaption of object detectors: A survey.  CoRR, abs/2105.13502, 2021.
  * [31] Narinder Singh Punn, Sanjay Kumar Sonbhadra, Sonali Agarwal, and Gaurav Rai.  Monitoring covid-19 social distancing with person detection and tracking via fine-tuned yolo v3 and deepsort techniques.  arXiv preprint arXiv:2005.01385, 2020.
  * [32] Ergys Ristani and Carlo Tomasi.  Features for multi-target multi-camera tracking and re-identification.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6036–6046, 2018.
  * [33] Florian Schroff, Dmitry Kalenichenko, and James Philbin.  Facenet: A unified embedding for face recognition and clustering.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 815–823, 2015.
  * [34] Edgar Simo-Serra, Eduard Trulls, Luis Ferraz, Iasonas Kokkinos, Pascal Fua, and Francesc Moreno-Noguer.  Discriminative learning of deep convolutional feature point descriptors.  In Proceedings of the IEEE international conference on computer vision, pages 118–126, 2015.
  * [35] Andreas Specker, Daniel Stadler, Lucas Florin, and Jurgen Beyerer.  An occlusion-aware multi-target multi-camera tracking system.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4173–4182, 2021.
  * [36] Zheng Tang, Milind Naphade, Ming-Yu Liu, Xiaodong Yang, Stan Birchfield, Shuo Wang, Ratnesh Kumar, David Anastasiu, and Jenq-Neng Hwang.  Cityflow: A city-scale benchmark for multi-target multi-camera vehicle tracking and re-identification.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8797–8806, 2019.
  * [37] Jonathan Tremblay, Aayush Prakash, David Acuna, Mark Brophy, Varun Jampani, Cem Anil, Thang To, Eric Cameracci, Shaad Boochoon, and Stan Birchfield.  Training deep networks with synthetic data: Bridging the reality gap by domain randomization.  In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 969–977, 2018.
  * [38] Paul Voigtlaender, Michael Krause, Aljosa Osep, Jonathon Luiten, Berin Balachandar Gnana Sekar, Andreas Geiger, and Bastian Leibe.  Mots: Multi-object tracking and segmentation.  In Proceedings of the ieee/cvf conference on computer vision and pattern recognition, pages 7942–7951, 2019.
  * [39] Xintao Wang, Liangbin Xie, Chao Dong, and Ying Shan.  Real-esrgan: Training real-world blind super-resolution with pure synthetic data.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1905–1914, 2021.
  * [40] Zhongdao Wang, Liang Zheng, Yixuan Liu, Yali Li, and Shengjin Wang.  Towards real-time multi-object tracking.  In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI 16, pages 107–122. Springer, 2020.
  * [41] Longyin Wen, Zhen Lei, Ming-Ching Chang, Honggang Qi, and Siwei Lyu.  Multi-camera multi-target tracking with space-time-view hyper-graph.  International Journal of Computer Vision, 122:313–333, 2017.
  * [42] Nicolai Wojke, Alex Bewley, and Dietrich Paulus.  Simple online and realtime tracking with a deep association metric.  In 2017 IEEE international conference on image processing (ICIP), pages 3645–3649. IEEE, 2017.
  * [43] Chao-Yuan Wu, R Manmatha, Alexander J Smola, and Philipp Krahenbuhl.  Sampling matters in deep embedding learning.  In Proceedings of the IEEE international conference on computer vision, pages 2840–2848, 2017.
  * [44] Zhenbo Xu, Wei Zhang, Xiao Tan, Wei Yang, Huan Huang, Shilei Wen, Errui Ding, and Liusheng Huang.  Segment as points for efficient online multi-object tracking and segmentation.  In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I 16, pages 264–281. Springer, 2020.
  * [45] Cheng-Yen Yang, Alan Yu Shyang Tan, Melanie J. Underwood, Charlotte Bodie, Zhongyu Jiang, Steve George, Karl Warr, Jenq-Neng Hwang, and Emma Jones.  Multi-object tracking by iteratively associating detections with uniform appearance for trawl-based fishing bycatch monitoring, 2023.
  * [46] Fan Yang, Shigeyuki Odashima, Sosuke Yamao, Hiroaki Fujimoto, Shoichi Masui, and Shan Jiang.  A unified multi-view multi-person tracking framework, 2023.
  * [47] Wen Yao, Chao-Hsien Chu, and Zang Li.  The adoption and implementation of rfid technologies in healthcare: a literature review.  Journal of medical systems, 36:3507–3525, 2012.
  * [48] Mang Ye, Jianbing Shen, Gaojie Lin, Tao Xiang, Ling Shao, and Steven CH Hoi.  Deep learning for person re-identification: A survey and outlook.  IEEE transactions on pattern analysis and machine intelligence, 44(6):2872–2893, 2021.
  * [49] Fengwei Yu, Wenbo Li, Quanquan Li, Yu Liu, Xiaohua Shi, and Junjie Yan.  Poi: Multiple object tracking with high performance detection and appearance feature.  In Computer Vision–ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part II 14, pages 36–42. Springer, 2016.
  * [50] Yifu Zhang, Peize Sun, Yi Jiang, Dongdong Yu, Fucheng Weng, Zehuan Yuan, Ping Luo, Wenyu Liu, and Xinggang Wang.  Bytetrack: Multi-object tracking by associating every detection box.  2022\.
  * [51] Yifu Zhang, Chunyu Wang, Xinggang Wang, Wenjun Zeng, and Wenyu Liu.  Fairmot: On the fairness of detection and re-identification in multiple object tracking.  International Journal of Computer Vision, 129:3069–3087, 2021.
  * [52] Kaiyang Zhou, Yongxin Yang, Andrea Cavallaro, and Tao Xiang.  Learning generalisable omni-scale representations for person re-identification.  IEEE transactions on pattern analysis and machine intelligence, 44(9):5056–5069, 2021.