# Refined symmetry-resolved Page curve and charged black holes

Pan Li 1,2<EMAIL_ADDRESS>Yi Ling 1,2<EMAIL_ADDRESS>1Institute of High
Energy Physics, Chinese Academy of Sciences, Beijing 100049, China
2 School of Physics, University of Chinese Academy of Sciences, Beijing
100049, China

###### Abstract

The Page curve plotted by the typical random state approximation is not
applicable to a system with conserved quantities, such as the evaporation
process of a charged black hole during which the electric charge does not
radiate out with a uniform rate macroscopically. In this context the symmetry-
resolved entanglement entropy may play a significant role in describing the
entanglement structure of such a system. We attempt to impose constraints on
microscopic quantum states to match with the macroscopic phenomenon of the
charge radiation during black hole evaporation. Specifically, we consider a
simple qubit system with conserved spin/charge serving as a toy model for the
evaporation of charged black holes. We propose refined rules for selecting a
random state with conserved quantities to simulate the distribution of charges
during the different stages of evaporation, and obtain refined Page curves
that exhibit distinct features in contrast to the original Page curve. We find
the refined Page curve may have a different Page time and exhibit asymmetric
behavior on both sides of the Page time. Such refined Page curves may provide
more realistic description for the entanglement between the charged black hole
and radiation during the process of evaporation.

##  I Introduction

Quantum entanglement as one of the most prominent characteristics of a quantum
system has been shown to play an important role in many fields such as quantum
information, quantum computation, condensed matter physics as well as black
hole physicsHorodecki:2009zz ; Eisert:2008ur ; Amico:2007ag ;
Laflorencie:2015eck ; Hawking:1975vcx ; Page:1993df ; Page:1993wv . It
demonstrates the mysterious non-classical correlation between quantum
subsystems. Entanglement entropy as an important measure of quantum
entanglement has been extensively investigated. In particular, for a bipartite
system Page Page:1993df  finds an important feature for the entanglement
entropy between two subsystems, which now sometimes is referred to as “Page’s
theorem”. It states that the average entanglement entropy of the smaller
subsystem over random pure states is very close to its maximal value, which is
constrained by degrees of freedom in the subsystem. It means that typically
the smaller subsystem is almost maximally entangled with the other subsystem.
Inspired by this observation, Page originally noticed that this picture may be
applicable to the famous black hole information loss paradox Hawking:1975vcx
Page:1993wv . The core issue about this paradox is whether the evaporation
process of a black hole due to Hawking radiation 111For a charged black hole,
the Schwinger effect can also contribute to black hole evaporation
Gibbons:1975kk ., which is a semi-classical result in quantum field theory
over curved but classical spacetime, is fundamentally unitary at the complete
quantum mechanical level. Currently a complete quantum theory of gravity has
not been established yet and the microscopic description of a black hole with
quantum states are unknown. Nevertheless, Page suggests that one might choose
random states as the approximation of black hole states during the evaporation
process222Recently some work Shenker:2013pqa ; Cotler:2016fpe ;
Maldacena:2015waa  advocates that a black hole in some aspects behaves as a
chaotic system, implying that the random approximation seems reasonable..
Under the condition that the black hole evaporation process is unitary, it is
believed that the Page theorem is applicable to a system composed of the black
hole and its radiation, and in principle one may find that the typical
entanglement entropy of the radiation subsystem as the function of its size
should follow the so-called Page curve. Such a picture proposed by Page
stimulates a lot of work to further understand the black hole information
paradox by investigating the entanglement between the black hole and
radiation. In particular, since the island paradigm was proposed in the
holographic approachPenington:2019npb ; Almheiri:2019psf ; Almheiri:2019hni ,
reproducing the Page curve for the evaporation process of the black hole by
holography has been becoming a central task in order to argue that the
information would be released in the later half stage of radiation.

Nevertheless, we know the approximation with totally random states sometimes
is not precise enough to describe the evolution of quantum entanglement in
practice, since many physically relevant systems usually have conserved
charges, such as energy, momentum, and electric charge. In a system with
conserved charges, the reduced Hilbert space is composed of quantum states
which are subject to the conservation of charges, thus do not equal to the
tensor product of the Hilbert spaces Casini:2013rba Ma:2015xes , each of which
is defined on individual subsystem separately. However, in this case one can
still compute the entanglement entropy between two subsystems based on the
reduced Hilbert space, which is the so-called “symmetry-resolved (SR)
entanglement entropy” Goldstein:2017bua Xavier:2018kqb , and has been widely
investigated in many theoretical aspects bonsignori2019symmetry ;
belin2013holographic ; caputa2016charged ; dowker2016 ; dowker2017 ;
cornfeld2018imbalance ; barghathi2018renyi ; barghathi2019operationally ;
feldman2019dynamics ; cornfeld2019entanglement ; fraenkel2020symmetry ;
calabrese2020full ; monkman2020operational ; azses2020symmetry ;
azses2020identification ; barghathi2020theory ; tan2020particle ;
murciano2020symmetry ; turkeshi2020entanglement ; kiefer2020bounds ;
murciano2020symmetry_2D ; kiefer2020evidence ; capizzi2020symmetry ;
murciano2020entanglement ; horvath2020symmetry ; bonsignori2020boundary ;
estienne2021finite ; murciano2021symmetry ; chen2021symmetry ;
zhao2021symmetry ; weisenberger2021symmetry ; capizzi2021symmetry ;
hung2021entanglement ; calabrese2021symmetry ; horvath2021u ;
azses2021observation ; kiefer2021slow ; shachar2021entanglement ;
parez2021exact ; parez2021quasiparticle ; ma2022symmetric ;
oblak2022equipartition ; zhao2022symmetry ; ares2022symmetry ;
jones2022symmetry ; horvath2022branch ; chen2022charged ; ghasemi2022universal
; scopa2022exact ; parez2022dynamics ; chen2022negativityboson ; multicharged
; goldstein2022  and even in experiments Lukin_2019
10.21468/SciPostPhys.12.3.106 . It is important to notice that when computing
the average SR entanglement entropy, the original Page’s theorem in general is
not applicable. So it is pretty interesting to investigate the Page curve for
SR entanglement entropy (SR Page curve) and to compare it with the page curve
of systems without conserved charges. Previously some relevant work on this
topic can be found in Murciano:2022lsw ; Lau:2022hvc .

It is well known that a stationary black hole is usually characterized by
three conserved quantities classically, namely the mass $E$, the angular
momentum $J$, and the electric charge $Q$. When considering the evaporation of
such a black hole, it is also quite natural to assume that these three
quantities are conserved during the process of evaporation. Therefore, one may
apply SR entanglement entropy to describe the entanglement between the black
hole and the radiation. In principle, a refined Page curve may be obtained
with a similar method introduced by Page, except that one just considers the
average value of entanglement entropy over the non-factorized reduced Hilbert
space Bianchi:2019stn . However, we point out that for the evaporation of a
charged black hole, such naive calculation based on random states does not
align with the semi-classical calculation of Hawking radiation and Schwinger
effect Gibbons:1975kk . The key point is that the charge does not radiate out
at the same rate as that of the mass Carter:1974yx ; Gibbons:1975kk ;
Page:1976df ; Page:1976ki ; Hiscock:1990ex . Specifically, if we were to
randomly select states in the entire Hilbert space with a fixed global charge
number, the average charge number of the subsystem (radiation part) would
increase linearly with the number of particles (at least in the case of the
usually simple qubit model, see Figure. 1). However, the semi-classical
calculation of Hawking radiation and Schwinger effect reveal that the
evolution of electric charge $Q$, the mass $E$ and the angular momentum $J$
exhibits distinct behavior during the evaporation Carter:1974yx ;
Gibbons:1975kk ; Page:1976df ; Page:1976ki . In general, for a black hole the
rate of losing energy and angular momentum changes relatively slowly in the
entire evaporation process Carter:1974yx , but the radiation rate of electric
charge $Q$ depends on the stage of the black hole which is specified by the
relations of parameters Gibbons:1975kk Hiscock:1990ex . For different
parameter values, the evaporation rate of the charge $Q$ varies dramatically,
which will be described with details in Sec.III. Consequently, the naive
random model, even at a qualitative level, fails to capture the charge
distribution during the course of evaporation for a charged black hole. In
this paper, aiming to simulate the evaporation of a charged black hole, we
analyze the SR entanglement entropy in a qubit model with conserved charges
and obtain various refined Page curves that reflect the different behavior of
the black hole radiating its charge. We will show that this model
qualitatively captures the feature that the charge does not radiate out with a
uniform rate during the evaporation of black holes.

This paper is organized as follows. In Sec.II, we review the calculation of
the average entanglement entropy over totally random states in a system
without conserved charges and the average SR entanglement entropy in a system
with conserved charges. In Sec.III, we introduce a qubit model for the
evaporation of a charged black hole, and propose refined rules for selecting a
random state with conserved quantity to simulate the distribution of charges
during the different stages of evaporation, and then obtain refined SR Page
curves. In Sec.IV, we present some discussions and outlooks on the work in
future.

##  II The average entanglement entropy and Page curve based on random states

In this section, we will review the general consideration about the
entanglement between two subsystems in a bipartite system which is described
by a random pure quantum state. We firstly compute the average value of the
entanglement entropy in a system without conserved charges, and then turn to
the SR entanglement entropy in a system with conserved charges.

###  II.1 The average entanglement entropy and Page curve in a system without
conserved charges

Given a bipartite system $A\cup B$ with the Hilbert space
$\mathcal{H_{AB}}=\mathcal{H_{A}}\otimes\mathcal{H_{B}}$, where
$\mathcal{H_{A}}$ and $\mathcal{H_{B}}$ are the Hilbert space of subsystem $A$
and $B$, respectively. Suppose the total system is described by a pure state
$|\psi\rangle$, then the entanglement entropy of $A$ is defined by

$S_{A}=-\mathrm{Tr}(\rho_{A}\mathrm{ln}\rho_{A}),$ (1)

where $\rho_{A}$ is the reduced density matrix by tracing $B$. Suppose the
dimension of $\mathcal{H_{A}}$ and $\mathcal{H_{B}}$ is
$d(\mathcal{H_{A}})=d_{A}$, $d(\mathcal{H_{B}})=d_{B}$ respectively, then the
dimension of $\mathcal{H_{AB}}$ is $d(\mathcal{H_{AB}})=d_{AB}=d_{A}d_{B}$.
Now we intend to compute the average value of the entanglement entropy
$\langle S_{A}\rangle$ for random states in $\mathcal{H_{AB}}$. We need first
find the uniform measure in $\mathcal{H_{AB}}$. For this purpose we pick out
an orthogonal basis $\\{|n\rangle\\}$ for a random state
$|\psi\rangle=\sum_{n=1}^{d_{AB}}c_{n}|n\rangle$, and then the measure is just
the uniform measure on the unit sphere of $\mathbb{C}^{d_{AB}}$, which is
$d\mu(\psi)=\delta(\sum_{n=1}^{d_{AB}}|c_{n}|^{2}-1)\prod_{n=1}^{d_{AB}}dc_{n}d\bar{c}_{n}$.

As a result, the average value of $S_{A}$ is obtained by integrating all the
quantum states in Hilbert space $\mathcal{H_{AB}}$

$\langle
S_{A}\rangle=\int-\mathrm{Tr}(\rho_{A}\mathrm{ln}\rho_{A})d\mu(\psi).$ (2)

One can also transform the integration variables into eigenvalues of
$\rho_{A}$, where the details can be found in Bianchi:2019stn . The final
result is

$\displaystyle\langle S_{A}\rangle$
$\displaystyle=\Psi(d_{AB}+1)-\Psi(d_{B}+1)-\frac{d_{A}-1}{2d_{B}}$ (3)
$\displaystyle\simeq\ln d_{A}-\frac{d_{A}}{2d_{B}}\text{ for }1\ll d_{A}\leq
d_{B},$ (4)

where $\Psi(x)=\Gamma^{\prime}(x)/\Gamma(x)$ is the so called Digamma
function. The above result indicates that for a bipartite system described by
pure states, the smaller subsystem is almost maximally entangled with the
other subsystem. So the Page curve, which plots the entanglement entropy as a
function of the size of the subsystem, will first increase with the size of
the subsystem up to its maximal value at $d_{A}=d_{B}$, and then decrease with
the size since for a pure system one always has $\langle S_{A}\rangle=\langle
S_{B}\rangle$, which now is constrained by the size of the smaller subsystem
$B$. Obviously, when the Hilbert space of the system is large enough, the Page
time is located at $d_{A}=d_{B}$, and the curve exhibits a symmetric behavior
on both sides of the Page time. We remark that this result is rooted at the
uniform measure over the Hilbert space, thus does not depend on the details of
evolution and in this sense may be treated as a model independent result.

###  II.2 The average SR entanglement entropy and SR Page curve in a system
with conserved charges

In a system with conserved charges, only those quantum states subject to these
constraints are allowable, leading to a reduced Hilbert space which may be
much smaller than the total Hilbert space. For instance, a bipartite system
contains a conserved charge $\hat{Q}$, then the total Hilbert space can be
decomposed into the direct sum of the eigenspace of $\hat{Q}$,

$\mathcal{H_{AB}=\sum_{Q}\mathcal{H_{AB}}}(Q).$ (5)

If the charge number $Q$ is fixed and conserved in a system, then we just need
to care about one sector $\mathcal{H_{AB}}(Q)$. One immediate difference for
$\mathcal{H_{AB}}(Q)$ is that it can not be factorized into the tensor product
of two Hilbert spaces of subsystems any more. Instead, it in general becomes
the direct sum of tensor product of the Hilbert spaces of subsystems with
fixed charges

$\mathcal{H_{AB}}(Q)=\sum_{i=1}^{s}\mathcal{H_{A}}(q_{i})\otimes\mathcal{H_{B}}(Q-q_{i}),$
(6)

where $s$ denotes the number of possible distributions of charges into two
subsystems. In such a system, due to the fact that
$\mathcal{H_{AB}}(Q)\neq\mathcal{H_{A}}\otimes\mathcal{H_{B}}$, we need do
more to figure out the uniform measure over the Hilbert space. By virtue of
the direct sum structure of $\mathcal{H_{AB}}(Q)$, we may write a random state
in $\mathcal{H_{AB}}(Q)$ as
$|\psi\rangle=\sum_{i=1}^{s}\sqrt{p_{i}}|\phi_{i}\rangle$, with $p_{i}\geq 0$
and $\sum_{i=1}^{s}p_{i}=1$, where
$|\phi_{i}\rangle\in\mathcal{H_{A}}(q_{i})\otimes\mathcal{H_{B}}(Q-q_{i})$. As
for each distribution $(q_{i},Q-q_{i})$, the corresponding Hilbert space has
the form of tensor product. Therefore in this situation, the entanglement
entropy of subsystem $A$ can be factorized into two parts

$\displaystyle S_{A}=$
$\displaystyle\sum_{i=1}^{s}p_{i}(q_{i})S_{A}(q_{i})-\sum_{i=1}^{s}p_{i}(q_{i})\ln
p_{i}(q_{i}).\ $ (7)

Here, $S_{A}(q_{i})$ represents the entanglement entropy within the factorized
Hilbert space $\mathcal{H_{A}}(q_{i})\otimes\mathcal{H_{B}}(Q-q_{i})$ for the
state $|\phi_{i}\rangle$, which can be readily computed using the formula
discussed in the previous subsection.

The uniform measure is also factorized into two part Bianchi:2019stn

$\mathrm{d}\mu_{Q}(\psi)=\mathrm{d}\nu\left(p_{1},\ldots,p_{s}\right)\prod_{i=1}^{s}\mathrm{d}\mu\left(\phi_{i}\right)\,,$
(8)

where $\mathrm{d}\nu\left(p_{1},\ldots,p_{s}\right)$ is the multivariate beta
distribution Bianchi:2019stn . After the average integration we obtain the
final result for the average SR entanglement entropy which reads as
Bianchi:2019stn

$\textstyle\langle
S_{A}\rangle_{\\!{}_{Q}}=\sum_{i=1}^{s}\\!\frac{d_{i}}{d_{Q}}\big{(}\langle
S_{A}(q_{i})\rangle+\Psi(d_{Q}+1)-\Psi(d_{i}+1)\big{)},$ (9)

where $d_{i}=d(\mathcal{H_{A}}(q_{i})\otimes\mathcal{H_{B}}(Q-q_{i}))$, and
$d_{Q}=\sum_{i=1}^{s}d_{i}$. Then in a parallel manner as described in the
previous subsection we may obtain the SR Page curve as one changes the size of
$A$ and $B$.

##  III A qubit model for charged black hole evaporation

In this section we consider a simple qubit model with conserved charges to
simulate the evaporation of a charged black hole, with the assumption that the
process of evaporation is unitary. As we mentioned in Sec.I, based on the
analysis of Hawking radiation and Schwinger effect, charged black holes do not
release their charge uniformly during evaporation Hiscock:1990ex  333Here
“uniform” refers to the condition that $\frac{dQ(t)}{dM(t)}$ is approximately
constant, or it does not change dramatically.. As a matter of fact, during the
course of evaporation the black hole may undergo different phases which depend
on the initial value of black hole parameters, which has previously been
revealed in Gibbons:1975kk Hiscock:1990ex . We briefly outline some useful
results here: (i) For black holes with mass $M<\frac{e^{2}}{m}\approx
10^{18}g$, where $e$ and $m$ denote the charge and mass of a single electron
respectively. Due to the presence of a strong electric field at the horizon,
they can not possess even one electron for a reasonable length of time. (ii)
For $M<\frac{e}{m^{2}}\approx 10^{5}M_{\odot}$, where $M_{\odot}$ denotes the
mass of the Sun, if the charge $Q$ is of a magnitude similar to $M$, the
Schwinger effect will be strong, resulting in the rapid discharge of the black
hole. (iii) For $M>2\times 10^{7}M_{\odot}$, then the configuration space
$(M,Q)$ may be divided into two regions according to the characteristics of
evaporation: the “charge dissipation zone” and the “mass dissipation zone”
Hiscock:1990ex . In the “charge dissipation zone” where the charge-to-mass
ratio of the black hole $Q/M$ is close to one, then the black hole loses
charges by Schwinger effect with a rate $dQ(t)/dM(t)>Q(t)/M(t)$ such that the
ratio $Q(t)/M(t)$ becomes smaller as time $t$ goes by. On the other hand, in
the “mass dissipation zone” where the ratio $Q/M$ is relatively small, then
the black hole loses charges with a low rate $dQ(t)/dM(t)<Q(t)/M(t)$. Thus the
ratio $Q(t)/M(t)$ becomes larger with the evaporation. In particular, if a
charged black hole starts to evaporate from certain region within the “mass
dissipation zone”, its electric charge may remain nearly unchanged until more
than half of its mass loses. Only when the mass decreases to an order similar
to the charge (in natural units), significant charge release begins and the
black hole enters “charge dissipation zone”. Therefore, we conclude that the
non-uniform release of charge is a common phenomenon during the evaporation of
charged black holes. Next we apply a micro-level qubit model to simulate the
evaporation of a charged black hole and propose refined rules to describe the
charge release with a non-uniform rate, and investigate how the Page curve in
this scenario differs from that evaluated in the case with completely random
states.

We consider a qubit model which is composed of $N$ qubits to simulate a
charged black hole radiating out particles. We divide the system into two
subsystems $A$ and $B$, corresponding to the radiation and the charged black
hole itself, respectively. For numerical analysis, we set the total number of
qubits to be $N=20$, and require $N_{A}+N_{B}=20$ where $N_{A}$ and $N_{B}$
are the number of qubits in $A$ and $B$, respectively. Thus, the different
partitions with $N_{A}$ from $0$ to $20$ represent the different stages of
evaporation from the initial state to final state. Roughly speaking, the
number $N_{B}$ can be considered as a quantity analogous to the mass value $M$
of the black hole while $N_{A}$ is the energy of radiation. Alternatively,
since the black hole loses its mass and the energy of radiation become larger
during the evaporation, the number of qubits $N_{A}$ may play a role of time
as well. Next we need to introduce a quantity to simulate the charge of a
black hole. Just similar to the notion of spin, we assume each qubit may have
a charge of either $+1$ or $-1$, so there is a 2-dimensional Hilbert space for
each qubit. The total charge $Q$ of the system is defined as the eigenvalue of
the operator $\hat{Q}=\sum_{i=1}^{N=20}{\sigma^{i}_{z}}$.

As the typical pattern of evaporation, we consider the system has a total
charge of $Q=4$, which is assumed to simulate a black hole with a small ratio
of the charge-to-mass. This configuration with $(N=20,Q=4)$ may fall into the
“mass dissipation zone” as described in (iii). In this case we stipulate that
the system does not release any charge until it shrinks to its half size,
namely the half number of the total qubits. This condition implies that when
$N_{A}$ is less than or equal to half of the total particles ($\frac{N}{2}$),
the Hilbert space of radiation is $\mathcal{H}_{A}(q_{i}=0)$, and
correspondingly $d(\mathcal{H_{A}}(q_{i}=0))=\binom{N_{A}}{\frac{N_{A}}{2}}$.
On the other hand, the Hilbert space of black hole is
$\mathcal{H}_{B}(Q-q_{i}=4)$, and correspondingly
$d(\mathcal{H_{B}}(Q-q_{i}=4))=\binom{N-N_{A}}{\frac{N-N_{A}+Q}{2}}$. After
Page time, namely $N_{A}\geq\frac{N}{2}$, we require the charge to evaporate
uniformly from the black hole, therefore the corresponding Hilbert space for
radiation is
$\mathcal{H_{A}}(q_{i}=\frac{Q}{(\frac{N}{2})}(N_{A}-\frac{N}{2}))$ with
$N_{A}$ from $N/2$ to $N$. We remark that for a finite $N$, a technical
problem may arise when $N_{A}$ jumps to $N$ with integer steps. Since for some
$N_{A}$, the corresponding $q_{i}=\frac{Q}{(\frac{N}{2})}(N_{A}-\frac{N}{2})$
possibly is not an integer, then we have to skip this step or demand its
integer part as $q_{i}$. When plotting the figure to illustrate the evolution
of entanglement entropy with small $N$, it may cause visible imprecision. But
we stress that such imprecision do not appear any more in the thermodynamic
limit, which requires $N_{th},Q_{th}\rightarrow\infty$, but keeping the radio
$\frac{Q_{th}}{N_{th}}=\frac{Q}{N}=\frac{4}{20}$. We can just follow the
calculations as above mentioned and then properly normalize the results.

Figure 1: The average charge number $\langle Q_{A}\rangle$ over random states
with particle number $N_{A}$ in subsystems $A$ under different evaporation
patterns. (i) The green dotted line represents the average value over random
states in Hilbert space $\mathcal{H_{AB}}(Q=4)$ ; (ii) The blue dotted line
represents the average value over random states in the refined Hilbert space
$\mathcal{H_{AB}}$; (iii) The purple solid line represents the average value
over the random states in the refined Hilbert space $\mathcal{H_{AB}}$ in the
thermodynamic limit, where the coordinates ($N_{a}$, $\langle Q_{A}\rangle$)
correspond to ($N_{a}$, $\langle Q_{A}\rangle$)$\times\frac{N_{th}}{20}$ as a
result of normalization.  Figure 2: The average entanglement entropy $\langle
S_{A}\rangle$ over random states with particle number $N_{A}$ in subsystems
$A$ under different evaporation patterns. (i) The red solid line represents
the average value over random states in the total Hilbert space
$\mathcal{H_{AB}}=\mathcal{H_{A}}\otimes\mathcal{H_{B}}=\\{|-1\rangle,\ \
|1\rangle\\}^{\otimes 20}$; (ii) The green dotted line represents the average
value over random states in the Hilbert space $\mathcal{H_{AB}}(Q=4)$ ; (iii)
The blue dotted line represents the average value over random states in the
refined Hilbert space $\mathcal{H_{AB}}$; (iv) The purple solid line
represents the average value over random states in the refined Hilbert space
$\mathcal{H_{AB}}$ in the thermodynamic limit, where the coordinates ($N_{a}$,
$\langle S_{A}\rangle$) correspond to ($N_{a}$, $\langle
S_{A}\rangle$)$\times\frac{N_{th}}{20}$ as a result of normalization.

Now we demonstrate our numerical results for $N=20$ and $Q=4$. For comparison
we depict the average charge and average entanglement entropy as the function
of $N_{A}$ for different patterns of evaporation in Figure (1) and Figure (2).
In Figure (1), three different patterns for the charge evaporation are
demonstrated. The blue dotted line plots the numerical results for $N=20$,
where we require no charge is released before $N_{A}\leq\frac{N}{2}$, while
after that moment at $N_{A}\geq\frac{N}{2}$, the charge is released with a
constant rate. The purple solid line plots the expected charge profile in the
thermodynamic limit, with the same rules. It is noticed that for $N=20$, the
deviation from the case of thermodynamic limit with perfect constant rate for
$N_{A}\geq\frac{N}{2}$ is visible, but the deviation would gradually disappear
as $N$ get bigger. As a comparison, the green dotted line illustrates another
pattern where the charge is released uniformly throughout the process of
evaporation, which is obtained by computing the average value in total Hilbert
space $\mathcal{H_{AB}}(Q=4)$. Our main results are demonstrated in Figure
(2), in which various Page curves are plotted for different patterns of
evaporation. Evidently, in comparison with the Page curves with random state
approximation, the refined Pages curves exhibit two prominent features.
Firstly, the refined Page time may shift from the middle point $N_{A}=N/2$.
Secondly, the refined Page curves exhibit asymmetric behavior on both sides of
the Page time. Such features are understandable since in order to match the
non-uniform charge release from the black hole, we apply different rules to
pick out different sectors of the total Hilbert space before and after the
middle point $N_{A}=\frac{N}{2}$, which disrupts the symmetry of the
dimensions of Hilbert space before and after the middle point
$N_{A}=\frac{N}{2}$. In addition, in the plot of the case with $N=20$ (the
blue dotted line), we applied integer approximation to generate more data for
drawing this plot.

In the end of this section, we remark that we have only considered some
typical patterns of charged black hole evaporation with a specific set of
parameters. Definitely one may perform the similar analysis for other patterns
with different discharge behavior as we have listed in (i)(ii)(iii). One need
impose different constraints on original Hilbert space, and then pick up
reduced Hilbert space to match different macroscopic patterns of evaporation.

##  IV Conclusion and Discussion

In this paper we have made the first step to apply symmetry-resolved
entanglement entropy to plot the Page curve for understanding the evaporation
of charged black holes. Due to the non-uniform rate of discharge during
evaporation, we should not simply evaluate the average entanglement entropy
over random states in a single Hilbert space for all time, as this would lead
to a uniform charge release. As a toy model, we have computed the $SR$
entanglement entropy in a qubit system that simulates the charge distribution
at different stages of evaporation for a charged black hole. After imposing
restrictions on the Hilbert space of the system, we have obtained a reduced
Hilbert space for each stage of the evaporation and plotted the refined $SR$
Page curve for this qubit system. We observe that the refined $SR$ Page curve
exhibits two distinctive features compared to the random cases: it has a
different Page time and displays asymmetric behavior on either side of the
Page time.

Although the qubit system as a toy model is too simple to describe the quantum
states of a genuine charged black hole, we stress that considering the
entanglement structure of random states in the qubit model has grasped the
spirit of the unitary evolution of a quantum chaotic system, which exhibits
highly entangled behavior and may be viewed as an analogy of a black hole. The
refined SR Page curve obtained in this paper matches with the macroscopic
phenomenon of the discharge during black hole evaporation, and has shed light
on understanding the release procedure of information from a charged black
holes at a microscopic level. Definitely, many aspects of this model can be
improved in future. Firstly, the analysis presented in this paper for charged
black hole is somewhat at a qualitative level. For a more realistic black
hole, various kinds of particles may evaporate, whose effects can not be
ignored. The evaporation rate for different particles can be derived from
semi-classical calculations of radiation Gibbons:1975kk . Secondly, to
simplify the analysis, we have only considered a linear relationship between
charge and particle number release, while the actual quantitative
relationship(e.g. $\frac{dQ_{A}(t)}{dt}$ and $\frac{dM_{A}(t)}{dt}$) would
also depend on the specific calculations of evaporation detailsPage:1976df ;
Page:1976ki ; Hiscock:1990ex . Thirdly, it is also interesting to consider a
system with multiple conserved charges, such as energy and angular momentum.
We expect the Page curve will differ quantitatively in such situations.

## Acknowledgments

We are very grateful to Kai Li, Wenbin Pan, Zhuoyu Xian, Zhangping Yu and
Hongbao Zhang for helpful discussions. This work is supported in part by the
Natural Science Foundation of China under Grant No. 12035016 and 12275275. It
is also supported by Beijing Natural Science Foundation under Grant No.
1222031, and the Innovative Projects of Science and Technology at IHEP.

## References

  * (1) R. Horodecki, P. Horodecki, M. Horodecki and K. Horodecki, _Quantum entanglement_ , _Rev. Mod. Phys._ 81 (2009) 865 [quant-ph/0702225].
  * (2) J. Eisert, M. Cramer and M.B. Plenio, _Area laws for the entanglement entropy - a review_ , _Rev. Mod. Phys._ 82 (2010) 277 [0808.3773].
  * (3) L. Amico, R. Fazio, A. Osterloh and V. Vedral, _Entanglement in many-body systems_ , _Rev. Mod. Phys._ 80 (2008) 517 [quant-ph/0703044].
  * (4) N. Laflorencie, _Quantum entanglement in condensed matter systems_ , _Phys. Rept._ 646 (2016) 1 [1512.03388].
  * (5) S.W. Hawking, _Particle Creation by Black Holes_ , _Commun. Math. Phys._ 43 (1975) 199.
  * (6) D.N. Page, _Average entropy of a subsystem_ , _Phys. Rev. Lett._ 71 (1993) 1291 [gr-qc/9305007].
  * (7) D.N. Page, _Information in black hole radiation_ , _Phys. Rev. Lett._ 71 (1993) 3743 [hep-th/9306083].
  * (8) G.W. Gibbons, _Vacuum Polarization and the Spontaneous Loss of Charge by Black Holes_ , _Commun. Math. Phys._ 44 (1975) 245.
  * (9) S.H. Shenker and D. Stanford, _Black holes and the butterfly effect_ , _JHEP_ 03 (2014) 067 [1306.0622].
  * (10) J.S. Cotler, G. Gur-Ari, M. Hanada, J. Polchinski, P. Saad, S.H. Shenker et al., _Black Holes and Random Matrices_ , _JHEP_ 05 (2017) 118 [1611.04650].
  * (11) J. Maldacena, S.H. Shenker and D. Stanford, _A bound on chaos_ , _JHEP_ 08 (2016) 106 [1503.01409].
  * (12) G. Penington, _Entanglement Wedge Reconstruction and the Information Paradox_ , _JHEP_ 09 (2020) 002 [1905.08255].
  * (13) A. Almheiri, N. Engelhardt, D. Marolf and H. Maxfield, _The entropy of bulk quantum fields and the entanglement wedge of an evaporating black hole_ , _JHEP_ 12 (2019) 063 [1905.08762].
  * (14) A. Almheiri, R. Mahajan, J. Maldacena and Y. Zhao, _The Page curve of Hawking radiation from semiclassical geometry_ , _JHEP_ 03 (2020) 149 [1908.10996].
  * (15) H. Casini, M. Huerta and J.A. Rosabal, _Remarks on entanglement entropy for gauge fields_ , _Phys. Rev. D_ 89 (2014) 085012 [1312.1183].
  * (16) C.-T. Ma, _Entanglement with Centers_ , _JHEP_ 01 (2016) 070 [1511.02671].
  * (17) M. Goldstein and E. Sela, _Symmetry-resolved entanglement in many-body systems_ , _Phys. Rev. Lett._ 120 (2018) 200602 [1711.09418].
  * (18) J.C. Xavier, F.C. Alcaraz and G. Sierra, _Equipartition of the entanglement entropy_ , _Phys. Rev. B_ 98 (2018) 041106 [1804.06357].
  * (19) R. Bonsignori, P. Ruggiero and P. Calabrese, _Symmetry resolved entanglement in free fermionic systems_ , _J. Phys. A: Math. Theor._ 52 (2019) 475302.
  * (20) A. Belin, L.-Y. Hung, A. Maloney, S. Matsuura, R.C. Myers and T. Sierens, _Holographic charged rényi entropies_ , _JHEP_ 2013 (2013) 59.
  * (21) P. Caputa, M. Nozaki and T. Numasawa, _Charged entanglement entropy of local operators_ , _Phys. Rev. D_ 93 (2016) 105032.
  * (22) J.S. Dowker _J. Phys. A: Math. Theor._ 49 (2016) 145401.
  * (23) J.S. Dowker _J. Phys. A: Math. Theor._ 50 (2017) 165401.
  * (24) E. Cornfeld, M. Goldstein and E. Sela, _Imbalance entanglement: Symmetry decomposition of negativity_ , _Phys. Rev. A_ 98 (2018) 032302.
  * (25) H. Barghathi, C.M. Herdman and A. Del Maestro, _Rényi generalization of the accessible entanglement entropy_ , _Phys. Rev. Lett._ 121 (2018) 150501.
  * (26) H. Barghathi, E. Casiano-Diaz and A. Del Maestro, _Operationally accessible entanglement of one-dimensional spinless fermions_ , _Phys. Rev. A_ 100 (2019) 022324.
  * (27) N. Feldman and M. Goldstein, _Dynamics of charge-resolved entanglement after a local quench_ , _Phys. Rev. B_ 100 (2019) 235146.
  * (28) E. Cornfeld, L.A. Landau, K. Shtengel and E. Sela, _Entanglement spectroscopy of non-abelian anyons: Reading off quantum dimensions of individual anyons_ , _Phys. Rev. B_ 99 (2019) 115429.
  * (29) S. Fraenkel and M. Goldstein, _Symmetry resolved entanglement: exact results in 1d and beyond_ , _J. Stat. Mech._ 2020 (2020) 033106.
  * (30) P. Calabrese, M. Collura, G. Di Giulio and S. Murciano, _Full counting statistics in the gapped xxz spin chain_ , _EPL_ 129 (2020) 60007.
  * (31) K. Monkman and J. Sirker, _Operational entanglement of symmetry-protected topological edge states_ , _Phys. Rev. Research_ 2 (2020) 043191.
  * (32) D. Azses and E. Sela, _Symmetry-resolved entanglement in symmetry-protected topological phases_ , _Phys. Rev. B_ 102 (2020) 235157.
  * (33) D. Azses, R. Haenel, Y. Naveh, R. Raussendorf, E. Sela and E.G. Dalla Torre, _Identification of symmetry-protected topological states on noisy quantum computers_ , _Phys. Rev. Lett._ 125 (2020) 120502.
  * (34) H. Barghathi, J. Yu and A. Del Maestro, _Theory of noninteracting fermions and bosons in the canonical ensemble_ , _Phys. Rev. Research_ 2 (2020) 043206.
  * (35) M.T. Tan and S. Ryu, _Particle number fluctuations, rényi entropy, and symmetry-resolved entanglement entropy in a two-dimensional fermi gas from multidimensional bosonization_ , _Phys. Rev. B_ 101 (2020) 235169.
  * (36) S. Murciano, G.D. Giulio and P. Calabrese, _Symmetry resolved entanglement in gapped integrable systems: a corner transfer matrix approach_ , _SciPost Phys._ 8 (2020) 46.
  * (37) X. Turkeshi, P. Ruggiero, V. Alba and P. Calabrese, _Entanglement equipartition in critical random spin chains_ , _Phys. Rev. B_ 102 (2020) 014455.
  * (38) M. Kiefer-Emmanouilidis, R. Unanyan, J. Sirker and M. Fleischhauer, _Bounds on the entanglement entropy by the number entropy in non-interacting fermionic systems_ , _SciPost Phys._ 8 (2020) 83.
  * (39) S. Murciano, P. Ruggiero and P. Calabrese, _Symmetry resolved entanglement in two-dimensional systems via dimensional reduction_ , _J. Stat. Mech._ 2020 (2020) 083102.
  * (40) M. Kiefer-Emmanouilidis, R. Unanyan, M. Fleischhauer and J. Sirker, _Evidence for unbounded growth of the number entropy in many-body localized phases_ , _Phys. Rev. Lett._ 124 (2020) 243601.
  * (41) L. Capizzi, P. Ruggiero and P. Calabrese, _Symmetry resolved entanglement entropy of excited states in a cft_ , _J. Stat. Mech._ 2020 (2020) 073101.
  * (42) S. Murciano, G. Di Giulio and P. Calabrese, _Entanglement and symmetry resolution in two dimensional free quantum field theories_ , _JHEP_ 2020 (2020) 73.
  * (43) D.X. Horváth and P. Calabrese, _Symmetry resolved entanglement in integrable field theories via form factor bootstrap_ , _JHEP_ 2020 (2020) 131.
  * (44) R. Bonsignori and P. Calabrese, _Boundary effects on symmetry resolved entanglement_ , _J. Phys. A: Math. Theor._ 54 (2020) 015005.
  * (45) B. Estienne, Y. Ikhlef and A. Morin-Duchesne, _Finite-size corrections in critical symmetry-resolved entanglement_ , _SciPost Phys._ 10 (2021) 54.
  * (46) S. Murciano, R. Bonsignori and P. Calabrese, _Symmetry decomposition of negativity of massless free fermions_ , _SciPost Phys._ 10 (2021) 111.
  * (47) H.-H. Chen, _Symmetry decomposition of relative entropies in conformal field theory_ , _JHEP_ 2021 (2021) 84.
  * (48) S. Zhao, C. Northe and R. Meyer, _Symmetry-resolved entanglement in ads3/cft2 coupled to u (1) chern-simons theory_ , _JHEP_ 2021 (2021) 30.
  * (49) K. Weisenberger, S. Zhao, C. Northe and R. Meyer, _Symmetry-resolved entanglement for excited states and two entangling intervals in ads3/cft2_ , _JHEP_ 2021 (2021) 104.
  * (50) L. Capizzi and P. Calabrese, _Symmetry resolved relative entropies and distances in conformal field theory_ , _JHEP_ 2021 (2021) 195.
  * (51) L.Y. Hung and G. Wong, _Entanglement branes and factorization in conformal field theory_ , _Phys. Rev. D_ 104 (2021) 026012.
  * (52) P. Calabrese, J. Dubail and S. Murciano, _Symmetry-resolved entanglement entropy in wess-zumino-witten models_ , _JHEP_ 2021 (2021) 67.
  * (53) D.X. Horváth, L. Capizzi and P. Calabrese, _U (1) symmetry resolved entanglement in free 1+ 1 dimensional field theories via form factor bootstrap_ , _JHEP_ 2021 (2021) 197.
  * (54) D. Azses, E.G. Dalla Torre and E. Sela, _Observing floquet topological order by symmetry resolution_ , _Phys. Rev. B_ 104 (2021) L220301.
  * (55) M. Kiefer-Emmanouilidis, R. Unanyan, M. Fleischhauer and J. Sirker, _Slow delocalization of particles in many-body localized phases_ , _Phys. Rev. B_ 103 (2021) 024203.
  * (56) S. Fraenkel and M. Goldstein, _Entanglement measures in a nonequilibrium steady state: Exact results in one dimension_ , _SciPost Phys._ 11 (2021) 85.
  * (57) G. Parez, R. Bonsignori and P. Calabrese, _Exact quench dynamics of symmetry resolved entanglement in a free fermion chain_ , _J. Stat. Mech._ 2021 (2021) 093102.
  * (58) G. Parez, R. Bonsignori and P. Calabrese, _Quasiparticle dynamics of symmetry-resolved entanglement after a quench: Examples of conformal field theories and free fermions_ , _Phys. Rev. B_ 103 (2021) L041104.
  * (59) Z. Ma, C. Han, Y. Meir and E. Sela, _Symmetric inseparability and number entanglement in charge-conserving mixed states_ , _Phys. Rev. A_ 105 (2022) 042416.
  * (60) B. Oblak, N. Regnault and B. Estienne, _Equipartition of entanglement in quantum hall states_ , _Phys. Rev. B_ 105 (2022) 115131.
  * (61) S. Zhao, C. Northe, K. Weisenberger and R. Meyer, _Charged moments in w3 higher spin holography_ , _JHEP_ 2022 (2022) 1.
  * (62) F. Ares, S. Murciano and P. Calabrese, _Symmetry-resolved entanglement in a long-range free-fermion chain_ , _arXiv:2202.05874_ (2022) .
  * (63) N.G. Jones, _Symmetry-resolved entanglement entropy in critical free-fermion chains_ , _arXiv:2202.11728_ (2022) .
  * (64) D.X. Horvath, P. Calabrese and O.A. Castro-Alvaredo, _Branch point twist field form factors in the sine-gordon model ii: Composite twist fields and symmetry resolved entanglement_ , _SciPost Phys._ 12 (2022) 88.
  * (65) H.-H. Chen, _Charged renyi negativity of massless free bosons_ , _JHEP_ 2022 (2022) 117.
  * (66) M. Ghasemi, _Universal thermal corrections to symmetry-resolved entanglement entropy and full counting statistics_ , _arXiv:2203.06708_ (2022) .
  * (67) S. Scopa and D.X. Horváth, _Exact hydrodynamic description of symmetry-resolved rényi entropies after a quantum quench_ , _arXiv:2205.02924_ (2022) .
  * (68) G. Parez, R. Bonsignori and P. Calabrese, _Dynamics of charge-imbalance-resolved entanglement negativity after a quench in a free-fermion model_ , _J. Stat. Mech._ 2022 (2022) 053103.
  * (69) H.-H. Chen, _Dynamics of charge imbalance resolved negativity after a global quench in free scalar field theory_ , _arXiv:2205.09532_ (2022) .
  * (70) F. Ares, P. Calabrese, G. Di Giulio and S. Murciano _arXiv:2206.01534_ (2022) .
  * (71) S. Fraenkel and M. Goldstein _arXiv:2205.12991_ (2022) .
  * (72) A. Lukin, M. Rispoli, R. Schittko, M.E. Tai, A.M. Kaufman, S. Choi et al., _Probing entanglement in a many-body–localized system_ , _Science_ 364 (2019) 256.
  * (73) V. Vitale, A. Elben, R. Kueng, A. Neven, J. Carrasco, B. Kraus et al., _Symmetry-resolved dynamical purification in synthetic quantum matter_ , _SciPost Phys._ 12 (2022) 106.
  * (74) S. Murciano, P. Calabrese and L. Piroli, _Symmetry-resolved Page curves_ , _Phys. Rev. D_ 106 (2022) 046015 [2206.05083].
  * (75) P.H.C. Lau, T. Noumi, Y. Takii and K. Tamaoka, _Page curve and symmetries_ , _JHEP_ 10 (2022) 015 [2206.09633].
  * (76) E. Bianchi and P. Dona, _Typical entanglement entropy in the presence of a center: Page curve and its variance_ , _Phys. Rev. D_ 100 (2019) 105010 [1904.08370].
  * (77) B. Carter, _Charge and particle conservation in black hole decay_ , _Phys. Rev. Lett._ 33 (1974) 558.
  * (78) D.N. Page, _Particle Emission Rates from a Black Hole: Massless Particles from an Uncharged, Nonrotating Hole_ , _Phys. Rev. D_ 13 (1976) 198.
  * (79) D.N. Page, _Particle Emission Rates from a Black Hole. 2. Massless Particles from a Rotating Hole_ , _Phys. Rev. D_ 14 (1976) 3260.
  * (80) W.A. Hiscock and L.D. Weems, _Evolution of Charged Evaporating Black Holes_ , _Phys. Rev. D_ 41 (1990) 1142.


11institutetext:  Advanced Mining Technology Center & Dept. of E.E.,
Universidad de Chile 22institutetext: Institute of Cognitive Science,
Universität Osnabrück
{22email<EMAIL_ADDRESS>22email<EMAIL_ADDRESS>
22email<EMAIL_ADDRESS>22email<EMAIL_ADDRESS>
# Towards Long-Term Memory for Social Robots: Proposing a New Challenge for
the RoboCup@Home League

Matías Pavez 11 Javier Ruiz del Solar 11 Victoria Amo 22 Felix Meyer zu
Driehausen 22

###### Abstract

Long-term memory is essential to feel like a continuous being, and to be able
to interact/communicate coherently. Social robots need long-term memories in
order to establish long-term relationships with humans and other robots, and
do not act just for the moment. In this paper this challenge is highlighted,
open questions are identified, the need of addressing this challenge in the
RoboCup@Home League with new tests is motivated, and a new test is proposed.

###### Keywords:

long-term-memory, service robot, social robot, RoboCup@Home

##  1 Introduction

Long-term memory allows humans to feel continuous and coherent in his/her
thoughts, i.e., to be a continuous person with a continuous life. Hence, long-
term memory is an essential component of social interaction between people in
daily life, it allows remembering names, events, duties, relationships, etc.
In fact, when long-term memory does not work properly due to illness (e.g.,
Alzheimer disease), the ability to interact with other human beings is
severely damaged.

Therefore, it seems evident that a key aspect in achieving long-term
interaction and social relationship between humans and social robots is the
requirement of a long-term memory system for the latter. Memory constitutes an
important part of a cognitive system implementation; in fact, it is the link
of prior experiences with ongoing and future behavior. As stated by [24], a
nontrivial level of social interaction requires that the robot should be able
to use both semantic and episodic information. Both, semantic and episodic
memories constitute the declarative long-term memory. While semantic memory is
a repository for facts, such as knowing that the capital of Chile is Santiago,
episodic memory is the memory of past experiences, i.e., remembering events
and their associated context (places, persons, emotions, etc.). Both kinds of
memories work together, and both need to be implemented for providing
appropriate social skills for social robots. However, in contrast to
implementing semantic memories, the implementation of episodic memories for
social robots is much less developed, and therefore we focus our work on the
latter.

The amount of information to be processed in a lifetime is vast; therefore,
efficient methods are required for acquiring, filtering, encoding, storing,
deleting and updating a robot’s episodic knowledge of its working environment.
This information can be encoded in symbolic form and held in a storage module
invoking the functionality of an episodic memory system. So far, this
challenge has not been addressed appropriately, and the construction of long-
term memories for social robots, which include both episodic and semantic
components, has not been achieved. In this context, the main goals of this
paper are: (i) to highlight the need of building appropriate long-term memory
for social robots, (ii) to identify open questions that need to be answered in
order to fulfill this first goal, and (iii) to provide basic concepts to
address this challenge in the RoboCup@Home League with new tests. We propose
the structure of such kind of tests, and give an example of a concrete test.
This paper is organized as follows: The basic aspects of human memory are
summarized in Section 2. Relevant open questions are identified in Section 3.
The structure of the proposed tests for the RoboCup@Home League is presented
in Section 4. Finally, some conclusions of this work are outlined in Section
5.

##  2 Human Memory

According to [8], memory is “the record of experience presented in the brain”.
There are multiple memory systems that work in a complementary way. These
systems have different functions, and they are characterized by different
operating characteristics and brain structures in which they are embodied.

A first categorization is related to the persistency of the stored
information. The Sensory Memory is able to store information acquired by our
sensory systems just for the fraction of a second. This information then
enters the so-called Short-Term Memory, which support “brief storage and
immediate recall of substantial detail” [8]. Part of the stored information is
then consolidated into the Long-Term Memory. The term Working Memory is
sometimes used instead of Short-Term Memory, although the working memory
concept includes processes and structures used for the temporal storage and
manipulation of information [22].

The Long-Term Memory is composed by a Declarative or Explicit Memory, which
refers to information that is remembered consciously, and a non-Declarative or
Implicit Memory, which refers to skills, abilities and tasks that can
remembered implicitly (e.g., how to ride a bicycle). The Implicit Memory is
further divided in Procedural Memory, in charge of storing procedures or ways
of doing tasks, and Priming, which refers to the fact that some experiences
are primed or recalled when a given stimuli is received. The Declarative
Memory is divided into two complementary memories, the Episodic Memory and the
Semantic Memory. Episodic memory, first defined by Tulving [21], refers to the
memory of specific events occurring at a specific place and time and enables
human beings to remember past experiences. Semantic memory is basically a
repository for facts. Emotional experiences are also stored in the brain and
they have influence on how other information is stored. The memory about
emotions is stored in the declarative memory, while the emotional memory
belongs to the implicit memory. Detailed explanations about the episodic,
semantic and emotional memories can be found in [4, 7].

The main functionalities provided by the declarative long-term memory are: (i)
the capability of remembering facts, concepts, events, experiences, skills,
tasks, emotions, (ii) the ability to feel like a continuous person, (iii) a
way of linking prior experiences with ongoing and future behavior, and (iv)
the ability to interact with other human beings, i.e. be able to communicate
coherently, and to build long-term relationships.

In this article our analysis will focus in the episodic memory, which as
already mentioned, is related to the conscious remembrance of context-
dependent events that are personally experienced. By context-dependent is it
meant the cognitive state —the temporal, spatial and emotional/affective
context —, as well as the embodied nature of the experience, i.e. the sensory-
perceptual processing of a given experience [1]. For instance, the
experience/event of visiting your mother’s house for having dinner last night
includes a temporal context (last night), a spatial context (your mother’s
house), an emotional context (spending time nicely with your mother), and
sensory-perceptual experiences (how the dinner tasted).

##  3 Open Questions for Implementing Long-Term Memory for Social Robots

During the past decade there have been several approaches that implemented
episodic and semantic long-term memory in artificial systems (e.g., [5, 6, 10,
13, 14, 15, 16, 18, 22]). We believe that these works addressed only partially
some of the main challenges that poses the implementation of long-term memory
for social robots, and that still some of the questions described in the next
paragraphs need to be answered.

How to store events and experiences in the form of episodes in the episodic
memory? It is still not clear how the cognitive state —temporal, spatial and
emotional/affective context —associated to events, as well as the embodied
nature of them, can be stored by a social robot. Naturally, the temporal
context can be easily stored (e.g., using time stamps). The ability to
determine and store spatial context has advanced largely in the last few years
thanks to the deep learning revolution that allows the recognition of places,
objects, and persons more easily. Yet, this has so far not been implemented in
social robots that interact continuously with the changing world. Very few
works have addressed the task of storing the emotional context and the
embodied nature of the experience.

How to give different levels of relevance to the different episodes in terms
of its novelty or the associate emotional state? As in the case of human
beings, the stored episodes have different levels of relevance, which depends
in the associate emotional state, among many other factors. Mechanisms for
determining autonomously this level of relevance need to be developed.

How to store emotional states? In the human brain emotional situations are
stored in the explicit and implicit memory systems, and experiences with a
strong emotive content produce powerful and vivid memories. Mechanisms for
implementing these functionalities need to be developed.

How to consolidate short-term memory into the long-term memory? The update of
the long-term memory is a complex and time-consuming process that involves the
consolidation of short-term memories into long-term ones [2]. In the case of
humans this is carried out during the sleep process, and consumes a large
amount of brain resources [23]. Therefore, the update of episodic information
must be carefully designed and implemented in the robot case. No work has
addressed the challenge of long-term memory consolidation for a social robot
operating continuously in the real-word.

Which mechanisms to use for forgetting and repression? Given that the memory
capacity is limited, forgetting mechanisms need to be implemented on basis of
the relevance of the stored information. In addition, it must be analyzed if,
as in the case of human beings, it is required to implement repression
mechanisms that unconsciously block memories in order to protect the self from
situations/emotions that she/he cannot cope with [9].

How to address the ethical issues related to the management of personal
information of human beings acquired by social robots? A social robot with
long-term memory will store information related with his/her human mates. It
must be analyzed how this personal information will be protected, managed, and
eventually, deleted.

##  4 Long-Term Memory in the RoboCup@Home League

In this Section we propose a new test for the RoboCup@Home league. First, we
will describe the minimum requirements for an EpLTM (Episodic Long-Term
Memory) implementation, how to validate them, and which sources of information
are valid when generating memories. Finally, we present a test proposal for
the competition, focused on EpLTM, Human-Robot Interaction (HRI) and
perception.

###  4.1 Requirements

At present, there is no consensus on the way an EpLTM should be implemented
for service robots, but many approaches can be found [9, 10, 11, 12, 19, 20,
22]. As the RoboCup’s goal is to boost research, we propose to evaluate only
the core requirements an EpLTM must fulfill, avoiding to force an specific
implementation on the teams. The proposed requirements can be separated into 2
categories: I. exclusively episodic, and II. requirements related to
historical and emotional relevances.

Category I is build from the 11 design requirements
$\\{R_{1},\ldots,R_{11}\\}$ presented by Stachowicz [20]. These were created
to match the characteristics every EpLTM system must satisfy and are the
minimum points for validation. The requirements (R1, R2, R4) declare that
every episode must be recollected and stored by its spatio-temporal context:
what, when, and where it happened. Moreover, there are no restrictions on
which information the what field can contain; For the competition there are
useful pieces of data to remember, for example, static information about known
people or objects (name, age), and their dynamic state (last location,
clothes, emotions). On the other hand, (R3, R6) state how the what field can
be accessed and modified, while (R5, R7, R8) give some rules about the episode
system structure (children episodes, anidation and transposition). In this
work, R9 (non intrusiveness), R10 (efficiency), and R11 (scalability) are left
out, because they relate to desirable design requirements and are not
considered as candidates for validation during a test.

Category II adds the concept of relevance to each episode in memory, which is
not covered by category I. Episodic relevance is essential when remembering
interesting events, allowing access to episodes by their importance. On the
one hand, we propose the historic relevance, which is directly related to the
age of an episode; the lower its antiquity is, the higher its importance,
which means a high probability to remember recent events. On the other hand,
we propose the emotional relevance by assigning an emotion and its related
magnitude to each episode; this allows to retrieve older but important events.
It is important to stress that knowing why the episode is relevant and how the
emotion relates to the episode is not required, as this depends on the emotion
engine implementation.

###  4.2 Required information for validation

Next we present a proposal for the minimum data required to be stored for the
competition, and the level of detail needed when validating the requirements
(categories I and II). The concepts to be delimited are the episode
definition, its contents (What, Where, When), and the associated emotions.

Although Stachowicz’s requirements do not impose the exact data to be gathered
for (What, Where, When) fields, a set of verifiable entities should be defined
for the competition. This serves as a way to normalize the validation process,
by clarifying which data and format will be required for validation. It is
important to highlight that by the following constraints we expect not to
impose an implementation to the teams, but just to formalize the minimum
required capabilities to any robot in competition.

####  4.2.1 Episodes:

In order to provide context to any episodic query or to enable a precise
memory description by the robot, we propose at least the following nested
episodic levels:

  1. 1.

Context: RoboCup, Stage X, Test Y, Subtest Z. This let us identify generally
the spatio-temporal context of an episode. Subtest Z only applies to tests
where sub stages are clearly defined, as in: “Stage 1, Test: SPR, Subtest: The
Riddle Game”.

  2. 2.

Tasks: There must exist an episode related to each task or order executed by
the robot. Tasks are defined inside a Context or inside other Task.

  3. 3.

Capabilities: There must exist an episode related to each high-level robot
capability: navigation, manipulation, perception, HRI. Capabilities are
defined inside a Task.

####  4.2.2 When:

Just knowing the sequence of episodes is not enough for validation. On the one
hand, transposed episodes, i.e. episodes that are simultaneous, cannot be
sequenced; on the other hand, the referees need a way to verify that the given
episode description relates to the recorded time. Consequently, at least, the
temporal information of an episode must consider initial and final timestamps.
These can be described in terms of minutes, hours, days, weeks, months or
years.

####  4.2.3 Where:

Location must be described in a simple way. Using coordinates like (x,y,z) is
not allowed. What is allowed:

  * •

The robot can show a map of the arena with drawings marking the interesting
locations.

  * •

The robot can describe locations using semantic information, with room names
and elements of the arena. Some examples: Inside/Outside the arena, rooms
(kitchen, bedroom), furniture (desk, fridge), or by using relative positions
(at the left of, over the). For tests which require Simultaneous Localization
and Mapping (SLAM) the description can be in terms of known areas, as the
“bar” in the Restaurant test.

####  4.2.4 What:

As the information to store is not clearly defined, this highly depends on the
team implementation. However, at least some entities and fields should be
defined by the referees/technical committee, in order to give a normalized
base for all queries and validations during a test. Proposed entities to be
stored are: people, objects and locations. Fields for each of these can be
obtained from capabilities required on previous tests (e.g., age/emotion
recognition, last seen location, and face images). Event description can be
made verbally or by displaying a graph with the related sub-episodes.

  * •

The verbal description is preferred. This should be related as a story, by
saying the associated episode sequence.

  * •

The description should be as specific as possible. E.g.: “I moved” vs. “I
moved towards the door”.

  * •

Transposed episodes should only be considered if they are in context. These
might require more complex verbal explanations to emphasize the concurrence of
the actions.

####  4.2.5 Emotions:

How emotions are generated for a given robot and how they are associated to
any given episode strongly depends on the emotional system used by the
team.Therefore, no matter what emotional model is used, we propose to restrict
the emotions to just 4 groups: Joy/Trust, Sadness/Fear, Surprise/Anticipation,
Anger/Disgust. These are obtained from Plutchik’s theory of emotions [17].
This selection is made to have a simple and verifiable set of emotions for the
competition. Each episode must be related to at least one emotion and its
intensity. Emotion intensity must have at least a resolution of 4 levels:
“normal”, “a little <happy>”, “<happy>”, “very <happy>”.

####  4.2.6 Other limitations:

There are no proposed rules on how the memories are stored into semantic
memory, as it depends on the implementation. There are no rules associated to
other concepts.

###  4.3 Episode Generation

There are many sources from where the robot can generate episodes for the
EpLTM: preparation for the competition, the travel, setup days, time between
tests, non-RoboCup related episodes and the tests in which the robot
participated. However, only the last one is a verifiable source of
information. Then, we propose to limit the episodes to only the ones related
to the tests of the current competition. On the first hand, this lets us
simplify the evaluation and veracity of the memories. On the second hand, by
only considering these episodes, new participants will compete in the same
terms as older teams, this also serves as a regulation between the amount of
memories gathered by teams which compete many times in a year.

As the competition lasts only a few days, the number of learning instances to
gather episodes is small. In order to increase the amount of episodes, we
propose two approaches. First and most important, the EpLTM test must be
postponed as much as possible, ideally as the last test of Stage 2. The second
proposal is to add a “Memory Setup” stage at the end of the “Setup Days”
period, where each robot can generate interesting memories for the queries of
the test. E.g.: By maintaining an informal conversation with someone of the
committee or by an introduction to the people that it will find through the
tests.

Depending on the queries the robot will encounter on the competition, the
organizing committee can find new requirements. For instance for queries about
people participating in 2 or more tests, the same name should be assigned to
them, and they should wear similar but not identical clothes.

Finally, we have proposed to only consider episodes related to the current
RoboCup competition, but it is important to mention that adding other events
and older RoboCup competitions has some advantages. This directly affects the
amount of episodes the robot will recollect. Moreover, this enables us to
consider queries based on episodic inference, so the robot can be confronted
to tasks requiring extrapolation from similar situations (e.g., Joe usually
wants me to clean up the table after a meal). This capability can be added in
later competitions, when the EpLTM test is considered solved.

###  4.4 Validation

The introduction of the memory concept is susceptible to cheating, for
example, with manually written episodes during the competition or through a
random episode generator. To attack this problem, first we consider the Fair
Play concept. On the other hand, a strategy to hinder hardcoding is to
increase the number of episodes and increase the number of available queries.
However, the simple solution is to require evidence of each described episode.

Evidences fulfill two purposes, they hinder the cheating and also simplify the
score assignation by the referees. When describing an episode it is desirable
that the robot displays related evidence. As an example, the robot can provide
a visualization as the one shown on Figure 1., where location, time, and
context are given. Moreover, this can be displayed through the Vizbox [3]
application, so that referees and audience can see it.

  * •

Task is validated by displaying the requirement as text.

  * •

Location is validated by displaying a colored map and images.

  * •

Sub-Episodes are validated with images or video.

Figure 1: Example of episodic visualization for the competition with fields
showing all verifiable information by the referees. In the upper-left panel,
the context and date-time are shown (e.g., “EpLTM Test” - “Wed, July 17, 2019
15:40:15”), followed by a graph with the emotion intensities. In the lower-
left panel, location information is displayed using a map of the environment
and the location name (e.g., “kitchen”). In the right panel, the images and
text fields are meant to display images recorded by the robot, interaction
subtitles, and other useful information as proof for validation.

###  4.5 Test Proposal: Sick and elderly care

Next we present an example of an @Home like test with EpLTM requirements. The
test is based on the methodology proposed in this Section. Particularly, it is
important to postpone the test as part of Stage 2, so that the robots can
collect as much interesting episodes as possible beforehand. We expect that
this proposal can be adjusted as needed for the competition.

####  4.5.1 Focus:

The robot must help a sick or elderly person with reduced mobility (in
bed/wheelchair), by answering questions about the home and recent events at
which he cannot attend. The test is focused on EpLTM, perception and HRI.

####  4.5.2 Setup:

The test takes place on the @Home arena. The operator is waiting in the
bedroom, lying on the bed or sitting on a chair. The arena keeps the same
structure and items as in other tests, but with some small changes. Other
people are located in the house with which the robot can interact as needed.

####  4.5.3 Task:

The robot starts by entering the arena, it moves to the bedroom, approaches
the operator and asks if any assistance is needed. The operator explains he is
sick/tired and cannot move, so it will make some questions to the robot. After
4 queries, the operator tells the robot to leave the bedroom.

####  4.5.4 Considerations:

Queries can be separated into 3 categories, depending on their requirements.
The robot should answer at least one question of each category.

  * •

Cat 1: Queries about memories and emotions.

  * •

Cat 2: Queries which require investigating objects in the arena.

  * •

Cat 3: Queries which require interacting with people in the arena.

  * •

We recommend the use of an episodic queries generator, built to match
requirements of categories I and II.

  * •

The robot must show evidences when answering, e.g., in a screen or using
Vizbox [3], as shown on the official Rulebook.

  * •

Referees must validate the coherence between answers and provided evidence.

  * •

Rooms, people and objects must be set up according to the possible queries.

##  5 Conclusions

The development of EpLTM for social robots is an important challenge for
improving human-robot interaction, and @Home has the means to boost the
progress. For that reason, we have presented a test proposal focused on EpLTM,
its requirements and validation strategies. The proposal is focused on the
minimum requirements any social robot implementing EpLTM should fulfill, but
trying not to impose an specific implementation to the teams.

## References

  * [1] Allen, P.A., Kaut, K.P., Lord, R.R.: Chapter 1.8 emotion and episodic memory. In: Dere, E., Easton, A., Nadel, L., Huston, J.P. (eds.) Handbook of Episodic Memory, Handbook of Behavioral Neuroscience, vol. 18, pp. 115 – 132. Elsevier (2008)
  * [2] Bailey, C.H., Bartsch, D., Kandel, E.R.: Toward a molecular definition of long-term memory storage. Proc. Natl. Acad. Sci. U.S.A. 93(24), 13445–13452 (Nov 1996)
  * [3] van Beek, L., Chen, K., Holz, D., Sanchez, L., Matamoros, M., Nagano, H., Rascon, C., de Souza, J.R., Rudinac, M., Wachsmuth, S.: RoboCup@Home 2017: Rules and Regulations. http://www.robocupathome.org/rules/2017_rulebook.pdf (2017)
  * [4] Dere, E., Easton, A., Nadel, L., Huston, J.P.: Handbook of behavioral neuroscience. In: Handbook of Episodic Memory, Handbook of Behavioral Neuroscience, vol. 18, p. iii. Elsevier (2008)
  * [5] Deutsch, T., Gruber, A., Lang, R., Velik, R.: Episodic memory for autonomous agents. In: 2008 Conference on Human System Interactions. pp. 621–626 (May 2008)
  * [6] Dodd, W., Gutierrez, R.: The role of episodic memory and emotion in a cognitive robot. Proceedings - IEEE International Workshop on Robot and Human Interactive Communication 2005, 692–697 (2005)
  * [7] Eichenbaum, H.: Learning & Memory. W. W. Norton & Company (2008)
  * [8] Eichenbaum, H.: Memory. Scholarpedia 3(3), 1747 (2008)
  * [9] Ho, W.C., Dautenhahn, K., Lim, M.Y., Vargas, P.A., Aylett, R., Enz, S.: An initial memory model for virtual and robot companions supporting migration and long-term interaction. In: RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication. pp. 277–284 (Sept 2009)
  * [10] Jockel, S., Weser, M.: Towards an episodic memory for cognitive robots. European Conference on Artificial Intelligence pp. 68–74 (2008)
  * [11] Kelley, T.D.: Robotic Dreams: A Computational Justification for the Post-Hoc Processing of Episodic Memories. International Journal of Machine Consciousness 06(02), 109–123 (2014)
  * [12] Kim, M.J., Baek, S.H., Cho, S.H., Kim, J.H.: Approach to integrate episodic memory into cogency-based behavior planner for robots. In: 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC). pp. 4188–4193 (oct 2016)
  * [13] Kuppuswamy, N.S., Cho, S.H., Kim, J.H.: A cognitive control architecture for an artificial creature using episodic memory. 2006 SICE-ICASE International Joint Conference pp. 3104–3110 (2006)
  * [14] Laird, J.E., Newell, A., Rosenbloom, P.S.: Soar: An architecture for general intelligence. Artificial Intelligence 33(1), 1 – 64 (1987)
  * [15] Leconte, F., Ferland, F., Michaud, F.: Design and integration of a spatio-temporal memory with emotional influences to categorize and recall the experiences of an autonomous mobile robot. Auton. Robots 40(5), 831–848 (Jun 2016), http://dx.doi.org/10.1007/s10514-015-9496-2
  * [16] Nuxoll, A., Laird, J.E.: A cognitive model of episodic memory integrated with a general cognitive architecture. In: ICCM (2004)
  * [17] Plutchik, R., Kellerman, H.: Emotion: Theory, research, and experience. In: Theories of Emotion, p. ii. Academic Press (1980)
  * [18] Ratanaswasd, P., Gordon, S., Dodd, W.: Cognitive control for robot task execution. In: ROMAN 2005. IEEE International Workshop on Robot and Human Interactive Communication, 2005. pp. 440–445 (Aug 2005)
  * [19] Spexard, T.P., Siepmann, F., Sagerer, G.: A Memory-based Software Integration for Development in Autonomous Robotics. In: International Conference on Intelligent Autonomous Systems. pp. 49–53 (2008)
  * [20] Stachowicz, D., Kruijff, G.J.M.: Episodic-like memory for cognitive robots. IEEE Transactions on Autonomous Mental Development (2012)
  * [21] Tulving, E.: Episodic and semantic memory. Organization of Memory (Tulving E, Donaldson W, eds). pp. 381–403. Academic Press, New York. (1972)
  * [22] Vijayakumar, S.: Long-Term Memory in Cognitive Robots. Ph.D. thesis, Universitaet des Saarlandes (2014)
  * [23] Walker, M.P., Stickgold, R.: Sleep-dependent learning and memory consolidation. Neuron 44(1), 121–133 (Sep 2004)
  * [24] Wood, R., Baxter, P., Belpaeme, T.: A review of long-term memory in natural and synthetic systems. Adaptive Behavior 20(2), 81–103 (2012)