#  Does anti-Unruh effect assist quantum entanglement and coherence?

Shu-Min Wu1111Email<EMAIL_ADDRESS>Xiao-Wei Teng1, Jin-Xuan Li1, Hao-Sheng
Zeng2222Email<EMAIL_ADDRESS>Tonghua
<EMAIL_ADDRESS>(corresponding author) 1 Department of
Physics, Liaoning Normal University, Dalian 116029, China
2 Department of Physics, Hunan Normal University, Changsha 410081, China
3 School of Physics and Optoelectronic Engineering, Yangtze University,
Jingzhou 434023

###### Abstract

In this paper, we use the concepts of quantum entanglement and coherence to
analyze the Unruh and anti-Unruh effects based on the model of Unruh-DeWitt
detector. For the first time, we find that (i) the Unruh effect reduces
quantum entanglement but enhances quantum coherence; (ii) the anti-Unruh
effect enhances quantum entanglement but reduces quantum coherence. This
surprising result refutes the notion that the Unruh effect can only destroy
quantum entanglement and coherence simultaneously, and that the anti-Unruh can
only protect quantum resources. Consequently, it opens up a new source for
discovering experimental evidence supporting the existence of the Unruh and
anti-Unruh effects.

###### pacs:

04.70.Dy, 03.65.Ud,04.62.+v

##  I Introduction

Quantum entanglement plays an important role in quantum information science.
It is a necessary ingredient for various computational tasks, such as quantum
remote control, quantum teleportation and quantum communication L1 ; L2 ; L3 .
A lot of progresses have been made in understanding the behavior of quantum
entanglement in various aspects, such as the sudden death and sudden birth,
the degeneration or enhancement of quantum entanglement L4 ; L5 ; L6 ; L7 ; L8
; L9 ; L10 . On the other hand, as a broader concept, quantum coherence is
also a physical resource in quantum technologies, optical experiments and
biological systems L11 ; L12 ; L13 ; L14 ; L15 . Many works have been done
about how the environment influences quantum coherence and how to protect it
L16 ; L17 ; L18 ; L19 . Quantum entanglement and coherence are closely related
to each other. Generally speaking, quantum coherence is a necessary condition
for quantum entanglement. Despite considerable efforts dedicated to
investigating the relationship between quantum entanglement and coherence L20
; AA2 ; L21 ; L22 ; AA ; AA1 , several challenges still remain unresolved.

The Unruh effect, first proposed by Unruh in 1976 L23 ; L24 , stands as a
crucial prediction of quantum field theory. An inertial observer which
undergoes uniform acceleration in the Minkowski vacuum will detect a thermal
bath of particles of a free quantum field with a temperature proportional to
the acceleration. On the other hand, Hawking discovered that black holes can
emit thermal radiation, a phenomenon subsequently named Hawking radiation L25
. According to the equivalence principle, the investigation of the Unruh
effect holds great significance for studying Hawking radiation and its
associated issues, including thermodynamics and the problem of information
loss L26 ; L27 ; L28 . Generally, both the Unruh effect and Hawking radiation
have been observed to reduce quantum entanglement and coherence L29 ; L30 ;
L31 ; L32 ; L33 ; L34 ; L35 ; L36 ; SMW1 ; SMW2 ; SMW3 ; SMW4 ; SMW5 ; SMW6 ;
SMW7 .

Recent research has suggested the existence of the anti-Unruh effect, wherein,
under specific conditions, the acceleration effect can also cool down a
detector L37 ; L38 ; L39 ; qsc1 ; qsc2 . The concept of anti-Hawking radiation
was also proposed L40 . For the global free models in the full space, the
anti-Unruh effect cannot be detected physically. In order to observe the anti-
Unruh effect, the semiclassical Unruh-DeWitt (UDW) detector model is usually
employed, which consists of a two-level atom interacting locally with the
vacuum field, and avoids the physically unfeasible detection of global models.
The UDW detector model is commonly established in experiments, often within a
finite length of optical cavity. The results have demonstrated that the anti-
Unruh effect enhances quantum entanglement for an initially entangled
bipartite state L38 ; L39 . However, the influence of the anti-Unruh effect on
quantum coherence remains unclear. Additionally, it raises the question of
whether the Unruh and anti-Unruh effects exert similar influences on both
quantum entanglement and coherence within the UDW detector model. In previous
studies L29 ; L30 ; L31 ; L32 ; L33 ; L34 ; L35 ; L36 ; SMW1 ; SMW2 ; SMW3 ;
SMW4 ; SMW5 ; SMW6 ; SMW7 , both the free field model and the UDW model did
not take boundary conditions into account. However, in our paper, we consider
the boundary conditions in the UDW model. Through the investigation of these
models, we may derive some intriguing conclusions, particularly highlighting
how quantum correlations and coherence may exhibit distinctive properties
under the influence of acceleration effects.

In this paper, we study the influence of acceleration effect on quantum
entanglement and coherence. Assume that a spin qubit and a two-level atom is
initially in an entangled pure state. The atom is then injected into a vacuum
cavity with finite length and moves at a uniform acceleration along the length
direction of the cavity. The atom plays the role of a detector which can
detect the thermal radiation due to the acceleration effect. We want to know
how the acceleration effect affects quantum entanglement and coherence. The
underlying motivation is to uncover novel aspects of the Unruh effect and
anti-Unruh effect, contributing to a more comprehensive understanding of these
phenomena.

The paper is organized as follows. In Sec. II, we briefly introduce the UDW
detector model. In Sec. III, we study the influence of acceleration effect on
quantum entanglement and coherence based on the UDW detector model. The last
section is devoted to the brief conclusion.

##  II Unruh-DeWitt model,

Let us first briefly recall the UDW model and the concept of anti-Unruh
effect. The UDW model consists of a two-level atom (detector) interacting
locally with a massless field $\phi(x(\tau))$ along the trajectory $x(\tau)$
with $\tau$ the detector’s proper time L37 . The detector has ground state
$|g\rangle$ and excited state $|e\rangle$, which are separated by an energy
gap $\Omega$. Suppose that the detector moves in a flat static cylinder with a
spatial circumference ($L>0$). This cylinder topology imposes periodic
boundary conditions which is relevant to laboratory systems, such as the
closed optical cavities, superconducting circuits coupled to periodic
microwave guides and optical-fibre loops L41 ; L42 ; L43 .

In the interaction picture, the UDW Hamiltonian that describes the interaction
between the detector and the field $\phi(x(\tau))$ is

$\displaystyle
H_{I}=\lambda\chi(\tau/\sigma)(e^{i\Omega\tau}\sigma^{+}+e^{-i\Omega\tau}\sigma^{-})\phi(x(\tau)),$
(1)

where $\lambda$ is the coupling strength that is assume to be weak,
$\sigma^{\pm}$ denote the ladder operators of detector, and
$\chi(\tau/\sigma)$ is the switching function which controls the duration of
interaction via the parameter $\sigma$. The most natural choice for the
switching function is the Gaussian function

$\displaystyle\chi(\tau/\sigma)=e^{-\tau^{2}/2\sigma^{2}}.$ (2)

For weak coupling, the unitary evolution of the total quantum system is given
by L37

$\displaystyle U$ $\displaystyle=$
$\displaystyle\mathbb{I}+U^{(1)}+\mathcal{O}(\lambda^{2})=\mathbb{I}-i\int
d\tau H_{I}(\tau)+\mathcal{O}(\lambda^{2})$ $\displaystyle=$
$\displaystyle-i\lambda\sum_{m}(I_{+,m}{a}_{m}^{\dagger}\sigma^{+}+I_{-,m}{a}_{m}^{\dagger}\sigma^{-}+\text{H.c.})+\mathcal{O}(\lambda^{2}),$

where $m$ denotes the mode of the scalar field with annihilation and creation
operators $a_{m}|0\rangle=0$ and $a_{m}^{\dagger}|0\rangle=|1_{m}\rangle$. The
sum over $m$ takes discrete values due to the periodic boundary condition
$k=2\pi m/L$, and $I_{\pm,m}$ can be written as

$\displaystyle I_{\pm,m}=\int_{-\infty}^{\infty}\chi(\tau/\sigma)e^{\pm
i\Omega\tau+\frac{2\pi
i}{L}[|m|t(\tau)-mx(\tau)]}\frac{d\tau}{\sqrt{4\pi|m|}}.$ (4)

Within the first-order approximation and in the interaction picture, this
evolution can be expressed as L38 ; L39

$\displaystyle U|g\rangle|0\rangle$ $\displaystyle=$ $\displaystyle
C_{0}(|g\rangle|0\rangle-i\eta_{0}|e\rangle|1_{m}\rangle),$ $\displaystyle
U|e\rangle|0\rangle$ $\displaystyle=$ $\displaystyle
C_{1}(|e\rangle|0\rangle+i\eta_{1}|g\rangle|1_{m}\rangle),$ (5)

where $C_{0}$ and $C_{1}$ are the normalization factors. In this paper, we
assume that the accelerated trajectory of the detector is
$t(\tau)=a^{-1}\sinh(a\tau)$ and $x(\tau)=a^{-1}[\cosh(a\tau)-1]$ with $a$
being the proper acceleration. Denoting $\eta_{0}=\lambda{\sum_{m}I_{+.m}}$
and $\eta_{1}=\lambda{\sum_{m}I_{-.m}}$, Eq.(II) can be rewritten as

$\displaystyle U|g\rangle|0\rangle$ $\displaystyle=$
$\displaystyle\frac{1}{\sqrt{1+|\eta_{0}|^{2}}}(|g\rangle|0\rangle-i\eta_{0}|e\rangle|1_{m}\rangle),$
$\displaystyle U|e\rangle|0\rangle$ $\displaystyle=$
$\displaystyle\frac{1}{\sqrt{1+|\eta_{1}|^{2}}}(|e\rangle|0\rangle+i\eta_{1}|g\rangle|1_{m}\rangle).$

If the detector is initially in its ground state, then the excitation
probability is given by

$\displaystyle P=\sum_{m\neq 0}|\langle
1,e|U^{(1)}|0,g\rangle|^{2}=\lambda^{2}\sum_{m\neq 0}|I_{+,m}|^{2}.$ (6)

From Eq.(6) we can see that the transition probability is dependent on the
concrete parameters, such as the length of cavity $L$, the energy gap
$\Omega$, and the interaction time scale $\sigma$. In particular, the
transition probability may decrease with the growth of acceleration when the
interaction timescale $\sigma$ is much smaller than the reciprocal of the
energy gap $\Omega^{-1}$. In other words, the detector is not be warmed but
cooled down. This counterintuitive effect is called anti-Unruh effect.

##  III Quantum entanglement and coherence for the Unruh-DeWitt model

Quantum entanglement and coherence are two important quantities for describing
quantum states. They are closely related to each other but also have own
characteristics. In the case of two-qubit systems, quantum entanglement can be
effectively described by the concurrence L44 ; L45

$\displaystyle
E(\rho_{AB})=\max\\{0,\sqrt{\lambda_{1}}-\sqrt{\lambda_{2}}-\sqrt{\lambda_{3}}-\sqrt{\lambda_{4}}\\},$
(7)

where $\lambda_{i}$ are the eigenvalues of the matrix
$\rho_{AB}[(\sigma_{y}\otimes\sigma_{y})\rho_{AB}^{*}(\sigma_{y}\otimes\sigma_{y})]$
in decreasing order. On the other hand, quantum entanglement can also be
measured by the logarithmic negativity $N(\rho_{AB})$, which is defined as L29

$\displaystyle N(\rho_{AB})=\log_{2}||\rho_{AB}^{T_{A}}||,$ (8)

where $||\rho_{AB}^{T_{A}}||$ is the sum of the absolute values of the
eigenvalues of the partial transpose of density matrix $\rho_{AB}$ with
respect to subsystem $A$.

There are several methods to describe the coherence of quantum states, in
which the measure of $l_{1}$ norm of coherence is maybe the simple and
intuitive one. In the given reference basis, the $l_{1}$ norm of coherence is
defined as the sum of absolute value of all the off-diagonal elements of the
system density matrix L46

$C_{l_{1}}(\rho_{AB})=\sum_{{i\neq j}}|\rho_{i,j}|.$ (9)

One can also quantify quantum coherence by the relative entropy of coherence
(REC) which is given by

$C_{\rm
REC}\left(\rho_{AB}\right)=S\left({\rho_{\rm{diag}}}\right)-S\left(\rho_{AB}\right),$
(10)

where $S(\rho_{AB})$ donates the von Neumann entropy of quantum state
$\rho_{AB}$, and $\rho_{\rm diag}$ denotes the state obtained from $\rho_{AB}$
by deleting all off-diagonal elements. In this paper, we employ quantum
entanglement and coherence to study the characteristics of both the Unruh
effect and anti-Unruh effect.

Consider a UDW detector, which is initially entangled with a spin qubit with
spin up $|\uparrow\rangle$ and spin down $|\downarrow\rangle$. The detector is
placed in a vacuum cavity with length $L$. The initial state of the whole
system takes the form

$\displaystyle|\psi\rangle_{qDC}=(\alpha|\uparrow_{q}\rangle|g_{D}\rangle+\beta|\downarrow_{q}\rangle|e_{D}\rangle)|0_{C}\rangle,$
(11)

where the real coefficients $\alpha$ and $\beta$ satisfy
$\alpha^{2}+\beta^{2}=1$, and $|0_{C}\rangle$ denotes the vacuum state of the
cavity field. For convenience of description, we use the subscripts $q$, $D$,
and $C$ to denote the qubit, detector, and cavity field, respectively. Now we
let the detector moves with a uniform acceleration $a$ in the cavity.
According to Eq.(II), the state of the whole system becomes

$\displaystyle|\psi\rangle_{q\bar{D}\bar{C}}$ $\displaystyle=$
$\displaystyle\frac{\alpha}{\sqrt{1+|\eta_{0}|^{2}}}|\uparrow_{q}\rangle|g_{D}\rangle|0_{C}\rangle-\frac{i\alpha\eta_{0}}{\sqrt{1+|\eta_{0}|^{2}}}|\uparrow_{q}\rangle|e_{D}\rangle|1_{C}\rangle$
$\displaystyle+$
$\displaystyle\frac{\beta}{\sqrt{1+|\eta_{1}|^{2}}}|\downarrow_{q}\rangle|e_{D}\rangle|0_{C}\rangle+\frac{i\beta\eta_{1}}{\sqrt{1+|\eta_{1}|^{2}}}|\downarrow_{q}\rangle|g_{D}\rangle|1_{C}\rangle.$

Here the symbol “bar” above D and C denotes that the states for the detector
and cavity field are observed in the noninertial frame determined by the
accelerated detector. Eq.(III) implies that the vacuum state in the inertial
frame becomes anti-vacuum observed in the noninertial frame. In other words,
the UDW detector would detect the production of particles in the vacuum
cavity. In the following, we will study the change of quantum entanglement and
coherence induced by the acceleration effect.

Let us first calculate quantum entanglement and coherence between qubit and
detector. Tracing over the cavity field modes in Eq.(III), we obtain the
density operator between qubit and detector as

$\displaystyle\rho_{q\bar{D}}$ $\displaystyle=$
$\displaystyle\frac{\alpha^{2}}{1+|\eta_{0}|^{2}}|\uparrow_{q}g_{D}\rangle\langle\uparrow_{q}g_{D}|+\frac{\alpha^{2}|\eta_{0}|^{2}}{1+|\eta_{0}|^{2}}|\uparrow_{q}e_{D}\rangle\langle\uparrow_{q}e_{D}|$
$\displaystyle+$
$\displaystyle\frac{\beta^{2}}{1+|\eta_{1}|^{2}}|\downarrow_{q}e_{D}\rangle\langle\downarrow_{q}e_{D}|+\frac{\beta^{2}|\eta_{1}|^{2}}{1+|\eta_{1}|^{2}}|\downarrow_{q}g_{D}\rangle\langle\downarrow_{q}g_{D}|$
$\displaystyle+$
$\displaystyle\frac{\alpha\beta}{\sqrt{(1+|\eta_{0}|^{2})(1+|\eta_{1}|^{2})}}(|\uparrow_{q}g_{D}\rangle\langle\downarrow_{q}e_{D}|+|\downarrow_{q}e_{D}\rangle\langle\uparrow_{q}g_{D}|)$
$\displaystyle-$
$\displaystyle\frac{\alpha\beta}{\sqrt{(1+|\eta_{0}|^{2})(1+|\eta_{1}|^{2})}}(\eta_{0}\eta_{1}^{*}|\uparrow_{q}e_{D}\rangle\langle\downarrow_{q}g_{D}|+\eta_{0}^{*}\eta_{1}|\downarrow_{q}g_{D}\rangle\langle\uparrow_{q}e_{D}|).$

Now the system consists of two objects, the inertial qubit and the accelerated
detector. Employing Eq.(7), we obtain the concurrence $E(\rho_{q\bar{D}})$
between qubit and detector as

$\displaystyle
E(\rho_{q\bar{D}})=\max\bigg{\\{}0,\frac{2\alpha\beta(1-|\eta_{0}||\eta_{1}|)}{\sqrt{(1+|\eta_{0}|^{2})(1+|\eta_{1}|^{2})}},\frac{2\alpha\beta(|\eta_{0}||\eta_{1}|-1)}{\sqrt{(1+|\eta_{0}|^{2})(1+|\eta_{1}|^{2})}}\bigg{\\}}.$
(14)

We can also use Eq.(8) to get the logarithmic negativity $N(\rho_{q\bar{D}})$

$\displaystyle N(\rho_{q\bar{D}})$ $\displaystyle=$
$\displaystyle\log_{2}\bigg{[}\sqrt{\bigg{(}\frac{\alpha^{2}|\eta_{0}|^{2}}{1+|\eta_{0}|^{2}}-\frac{\beta^{2}|\eta_{1}|^{2}}{1+|\eta_{1}|^{2}}\bigg{)}^{2}+\frac{4\alpha^{2}\beta^{2}}{(1+|\eta_{0}|^{2})(1+|\eta_{1}|^{2})}}$
$\displaystyle+$
$\displaystyle\frac{\alpha^{2}}{1+|\eta_{0}|^{2}}+\frac{\beta^{2}}{1+|\eta_{1}|^{2}}\bigg{]}.$

It is shown that the concurrence $E(\rho_{q\bar{D}})$ and the logarithmic
negativity $N(\rho_{q\bar{D}})$ depend not only on the initial parameters
$\alpha$ and $\beta$, but also on the acceleration $a$, the length of cavity
$L$, the energy gap $\Omega$ and the interaction time scale $\sigma$, i.e.,
both the acceleration effect and the setup’s parameters can affect quantum
entanglement.

Figure 1:  The transition probability $P$ (left column), concurrence
$E(\rho_{q\bar{D}})$ (middle column), and logarithmic negativity
$N(\rho_{q\bar{D}})$ (right column) as functions of acceleration $a$ for
different energy gaps $\Omega$. The rest parameters are chosen as
$\alpha=\frac{1}{\sqrt{2}}$, $L=200$ and $\sigma=0.2$.

In Fig.1, we plot the transition probability $P$ (left column), concurrence
$E(\rho_{q\bar{D}})$ (middle column), and logarithmic negativity
$N(\rho_{q\bar{D}})$ (right column) as functions of acceleration $a$ for
different energy gaps $\Omega$, with other parameters chosen as
$\alpha=\frac{1}{\sqrt{2}}$, $L=200$, and $\sigma=0.2$. It is shown that for
the smaller energy gap, i.e., $\Omega=0.05$, quantum entanglement increases
and the transition probability of detector decreases, with the growth of
acceleration $a$, meaning that the anti-Unruh effect enhances quantum
entanglement between qubit and detector. On the other hand, for the larger
energy gap, i.e., $\Omega=5$, quantum entanglement decreases and the
transition probability of detector increases with the growth of acceleration,
showing that the Unruh effect reduces quantum entanglement between qubit and
detector. Note that the initial entanglement at zero acceleration ($a=0$) is
not equal to one, which is caused by the interaction between the detector and
the limited vacuum field. The limited cavity space and finite interaction time
$\sigma$ between the cavity field and detector lead to $\eta_{0}\neq 0$ and
$\eta_{1}\neq 0$, so that the initial entanglement between qubit and detector
degrades. As soon as the detector enters the cavity, it is coupled with the
vacuum field in cavity, and the entanglement degradation takes place.

Besides quantum entanglement, we also study the change of quantum coherence
between qubit and detector. According to Eq.(9), the $l_{1}$ norm of coherence
$C(\rho_{q\bar{D}})$ for the system of qubit and detector reads

$\displaystyle
C_{l_{1}}(\rho_{q\bar{D}})=\frac{2\alpha\beta(1+|\eta_{0}||\eta_{1}|)}{\sqrt{(1+|\eta_{0}|^{2})(1+|\eta_{1}|^{2})}}.$
(16)

Next, we study the acceleration effect on the REC $C_{REC}(\rho_{q\bar{D}})$.
For this purpose, we should calculate the eigenvalues of density matrix of
Eq.(III). The density matrix has two non-zero eigenvalues

$\displaystyle\lambda_{1}(\rho_{q\bar{D}})$ $\displaystyle=$
$\displaystyle\frac{\alpha^{2}}{1+|\eta_{0}|^{2}}+\frac{\beta^{2}}{1+|\eta_{1}|^{2}},$
$\displaystyle\lambda_{2}(\rho_{q\bar{D}})$ $\displaystyle=$
$\displaystyle\frac{\alpha^{2}|\eta_{0}|^{2}}{1+|\eta_{0}|^{2}}+\frac{\beta^{2}|\eta_{1}|^{2}}{1+|\eta_{1}|^{2}}.$

Thus, the REC of state $\rho_{q\bar{D}}$ becomes

$\displaystyle
C_{REC}(\rho_{q\bar{D}})=\sum_{i=1}^{2}\lambda_{i}(\rho_{q\bar{D}})\log_{2}\lambda_{i}(\rho_{q\bar{D}})-\sum_{j}\beta_{j}(\rho_{q\bar{D}})\log_{2}\beta_{j}(\rho_{q\bar{D}}),$
(17)

where $\beta_{j}(\rho_{q\bar{D}})$ are the diagonal elements of
$\rho_{q\bar{D}}$ of Eq.(III). Note that Eq.(III) is a X state which has no
monomeric coherence whether for the qubit or for the detector. Thus the
coherence $C_{l_{1}}(\rho_{q\bar{D}})$ and $C_{REC}(\rho_{q\bar{D}})$ is
actually a kind of genuine bipartite coherence between the qubit and the
detector. According to the viewpoint of recent research AA , this genuine
bipartite coherence is actually a kind of quantum correlation between the
relevant subsystems.

Figure 2: The $l_{1}$ norm of coherence $C(\rho_{q\bar{D}})$ (left column) and
the REC $C_{REC}(\rho_{q\bar{D}})$ (right column) as functions of acceleration
$a$ for different energy gaps $\Omega$. The parameters are chosen as
$\alpha=\frac{1}{\sqrt{2}}$, $L=200$, $\sigma=0.2$, $A=0.999999$, and
$B=0.9999988$.

For diffrent energy gaps, we plot the $l_{1}$ norm of coherence
$C(\rho_{q\bar{D}})$ and the REC $C_{REC}(\rho_{q\bar{D}})$ as functions of
acceleration $a$ as in Fig.2, where the parameters are chosen as the same as
in Fig.1. For smaller energy gaps, i.e., $\Omega=0.05$, quantum coherence
$C(\rho_{q\bar{D}})$ changes very slowly with acceleration. It is shown that
quantum coherence $C(\rho_{q\bar{D}})$ decreases with acceleration $a$,
meaning that the anti-Unruh effect reduces quantum coherence between qubit and
detector. For bigger energy gaps, i.e., $\Omega=5$, we see that quantum
coherence $C(\rho_{q\bar{D}})$ increases with acceleration $a$, meaning that
the Unruh effect enhances quantum coherence between qubit and detector.

From above discussions, we see that the Unruh effect and anti-Unruh effect
play completely opposite roles: the Unruh effect reduces quantum entanglement
between qubit and detector but enhances their quantum coherence; while the
anti-Unruh effect enhances quantum entanglement between qubit and detector but
reduces their quantum coherence. This discovery represents a novel outcome. It
was shown in the previous research that both quantum entanglement and
coherence reduce under the influence of the Unruh effect L29 ; L30 ; L31 ; L32
; L33 ; L34 ; L35 ; L36 ; SMW1 ; SMW2 ; SMW3 ; SMW4 ; SMW5 ; SMW6 ; SMW7 ,
which is obviously different from our results. The reason for the difference
is: the previous papers mainly consider free models in the full space and a
Unruh-Dewitt model without boundary conditions, while we consider a Unruh-
Dewitt model in the cavity model with boundary condition. When $a=0$, the
entanglement and coherence are generally less than initial value, owing to the
limited cavity space and finite interaction time between the cavity field and
detector L37 ; L38 ; L39 ; qsc1 ; qsc2 . Physically, we can understand this
phenomenon as a transfer of entanglement and coherence: as the detector enters
the cavity, it interacts with the cavity mode, partially transferring
entanglement and coherence from the detector to the cavity mode, thereby
reducing the entanglement and coherence between detectors. An increase in
acceleration may extract or transfer entanglement and coherence between the
detector and cavity mode to the detectors themselves.

##  IV  Conclutions

In conclusion, we have studied the influence of acceleration effect on quantum
entanglement and coherence between a qubit and a relativistic detector based
on the Unruh-Dewitt model. Depending on the chosen parameters, one can observe
phenomena associated with both the Unruh and anti-Unruh effects. The Unruh
effect reduces the entanglement between qubit and detector but increases their
quantum coherence, challenging the notion that the Unruh effect is uniformly
detrimental to quantum resources. Contrarily, the anti-Unruh effect increases
their entanglement and reduces the coherence, indicating that the anti-Unruh
effect may not always be advantageous for quantum resources. These opposite
changes between quantum entanglement and coherence under the same processes
suggest the difference between them. Furthermore, previous studies have
indicated that the Unruh effect consistently leads to a simultaneous reduction
in both entanglement and coherence L29 ; L30 ; L31 ; L32 ; L33 ; L34 ; L35 ;
L36 ; SMW1 ; SMW2 ; SMW3 ; SMW4 ; SMW5 ; SMW6 ; SMW7 . The reason for the
different results is: the previous articles mainly consider free models in the
full space and a Unruh-Dewitt model without boundary conditions, while we
consider a Unruh-Dewitt model in the cavity model with boundary condition.
These results overturn conventional perceptions of the Unruh and anti-Unruh
effects, simultaneously providing an unexpected source of experimental
evidence for them.

###### Acknowledgements.

The authors would like to thank Wentao Liu for helpful discussions. This work
is supported by the National Natural Science Foundation of China (Grant Nos.
12205133), LJKQZ20222315 and JYTMS20231051.

## References

  * (1) The Physics of Quantum Information, edited by D. Bouwmeester, A. Ekert, A. Zeilinger (Springer-Verlag, Berlin, 2000).
  * (2) C. H. Bennett, G. Brassard, C. Crépeau, R. Jozsa, A. Peres, and W. K. Wootters, Phys. Rev. Lett. 70, 1895 (1993).
  * (3) S. F. Huegla, M. B. Plenio, and J. A. Vaccaro, Phys. Rev. A 65, 042316 (2002).
  * (4) Y. S. Weinstein, Phys. Rev. A 80, 022310 (2009).
  * (5) M. I. Shaukat, E. V. Castro, and H. Terças, Phys. Rev. A 98, 022319 (2018).
  * (6) J. Wang and J. Jing, Phys. Rev. A 82, 032324 (2010).
  * (7) S. M. Wu and H. S. Zeng, Class. Quantum Grav. 37, 115003 (2020).
  * (8) J. Hu and H. Yu, Phys. Rev. A 91, 012327 (2015).
  * (9) L. Mazzola, S. Maniscalco, J. Piilo, K. A. Suominen, and B. M. Garraway, Phys. Rev. A 79, 042302 (2009).
  * (10) B. Zhang, S. You, and M. Lu, Phys. Rev. A 101, 032335 (2020).
  * (11) A. J. Leggett, Prog. Theor. Phys. Suppl. 69, 80 (1980).
  * (12) B. Schumacher, M. D. Westmoreland, Phys. Rev. Lett. 80, 5695 (1998).
  * (13) S. E. Barnes, R. Ballou, B. Barbara, J. Strelen, Phys. Rev. Lett. 79, 289 (1997).
  * (14) A. Streltsov, G. Adesso, M. B. Plenio, Rev. Mod. Phys. 89, 041003 (2017).
  * (15) M. Horodecki and J. Oppenheim, Nat. Commun. 4, 2059 (2013).
  * (16) T. R. Bromley, M. Cianciaruso, and G. Adesso, Phys. Rev. Lett. 114, 210401 (2015).
  * (17) A. Canaguier-Durand and R. Carminati, Phys. Rev. A 93, 033836 (2016).
  * (18) C. M. Kropf, Phys. Rev. Research 2, 033311 (2020).
  * (19) Z. Zhang, H. Fu, and J. Wang, Phys. Rev. B 95, 144306 (2017).
  * (20) K. Chuan Tan, H. Kwon, C. Y. Park, and H. Jeong, Phys. Rev. A 94, 022329 (2016).
  * (21) Y. Dai, Y. Dong, Z. Xu, W. You, C. Zhang, and O. Gühne, Phys. Rev. Appl. 13, 054022 (2020).
  * (22) K. C. Tan and H. Jeong, Phys. Rev. Lett. 121, 220401 (2018).
  * (23) J. X. Hou, S. Y. Liu, X. H. Wang, and W. L. Yang, Phys. Rev. A 96, 042324 (2017).
  * (24) C. Radhakrishnan, M. Parthasarathy, S. Jambulingam, and T. Byrnes, Phys. Rev. Lett. 116, 150504 (2016).
  * (25) S. Kim, L. Li, A. Kumar, and J. Wu, Phys. Rev. A 98, 022306 (2018).
  * (26) W. G. Unruh, Phys. Rev. D 14, 870 (1976).
  * (27) L. C. B. Crispino, A. Higuchi, and G. E. A. Matsas, Rev. Mod. Phys. 80, 787 (2008).
  * (28) S. W. Hawking, Nature (London) 248, 30 (1974).
  * (29) L. Bombelli, R. K. Koul, J. Lee, and R. D. Sorkin, Phys. Rev. D 34, 373 (1986).
  * (30) S.W. Hawking, Commun. Math. Phys. 43, 199 (1975).
  * (31) H. Terashima, Phys. Rev. D 61, 104016 (2000).
  * (32) I. Fuentes-Schuller, and R. B. Mann, Phys. Rev. Lett. 95,120404 (2005).
  * (33) P. M. Alsing, I. Fuentes-Schuller, R. B. Mann, and T. E. Tessier, Phys. Rev. A 74, 032326 (2006).
  * (34) G. Adesso, I. Fuentes-Schuller, and M. Ericsson, Phys. Rev. A 76, 062112 (2007).
  * (35) S. M. Wu, H. S. Zeng, Eur. Phys. J. C 82, 4 (2022).
  * (36) S. Xu, X. k. Song, J. d. Shi, and L. Ye, Phys. Rev. D 89, 065022 (2014).
  * (37) S. M. Wu, Y. T. Cai, W. J. Peng, H. S. Zeng, Eur. Phys. J. C 82, 412 (2022).
  * (38) J. Wang, Z. Tian, J. Jing, and H. Fan, Phys. Rev. A 93, 062105 (2016).
  * (39) S. M. Wu, H. S. Zeng, and H. M. Cao, Class. Quantum Grav. 38, 185007 (2021).
  * (40) L. C. Céleri, A. G. S. Landulfo, R. M. Serra, and G. E. A. Matsas, Phys. Rev. A 81, 062130 (2010).
  * (41) W. C. Qiang, G. H. Sun, Q. Dong, and S. H. Dong, Phys. Rev. A 98, 022320 (2018).
  * (42) S. Bhattacharya, N. Joshi, Phys. Rev. D 105, 065007 (2022).
  * (43) M. R. Hwang, D. Park, and E. Jung, Phys. Rev. A 83, 012111 (2011).
  * (44) Q. Liu, S. M. Wu, C. Wen, J. Wang, Sci. China-Phys. Mech. Astron. 66, 120413 (2023).
  * (45) A. G. S. Landulfo and G. E. A. Matsas, Phys. Rev. A 80, 032315 (2009).
  * (46) S. Harikrishnan, S. Jambulingam, P. P. Rohde, C. Radhakrishnan, Phys. Rev. A 105, 052403 (2022).
  * (47) W.G. Brenna, R.B. Mann, E. Martín-Martínez, Phys. Lett. B 757, 307 (2016).
  * (48) T. Li, B. Zhang, L. You, Phys. Rev. D 97, 045005 (2018).
  * (49) Y. Pan and B. Zhang, Phys. Rev. A 101, 062111 (2020).
  * (50) S. M. Wu, H. S. Zeng, T. Liu, New J. Phys. 24, 073004 (2022).
  * (51) X. Ming, Eur. Phys. J. C 83, 1166 (2023).
  * (52) L. J. Henderson, R. A. Hennigar, R. B. Mann, A. R. H. Smithe, J. Zhang, Phys. Lett. B 809, 135732 (2020).
  * (53) H. Tsuchida, Opt. Lett. 15, 640 (1990).
  * (54) B. Peropadre, P. Forn-Díaz, E. Solano, J. J. García-Ripoll, Phys. Rev. Lett. 105, 023601 (2010).
  * (55) C. M. Wilson, G. Johansson, A. Pourkabirian, M. Simoen, J.R. Johansson, T. Duty, F. Nori, P. Delsing, Nature 479, 376 (2011).
  * (56) W. K. Wootters, Phys. Rev. Lett. 80, 2245 (1998).
  * (57) V. Coffman, J. Kundu, and W. K. Wootters, Phys. Rev. A 61, 052306 (2000).
  * (58) T. Baumgratz, M. Cramer, and M. B. Plenio, Phys. Rev. Lett. 113, 140401 (2014).


# Beyond Accuracy: ROI-driven Data Analytics of Empirical Data

Gouri Deshpande<EMAIL_ADDRESS>
Guenther Ruhe<EMAIL_ADDRESS>
Department of Computer Science, University of Calgary

###### Abstract

Background: The unprecedented access to data has rendered a remarkable
opportunity to analyze, understand, and optimize the investigation approaches
in almost all the areas of (Empirical) Software Engineering. However, data
analytics is time and effort consuming, thus, expensive, and not automatically
valuable.

Objective: This vision paper demonstrates that it is crucial to consider
Return-on-Investment (ROI) when performing Data Analytics. Decisions on ”How
much analytics is needed”? are hard to answer. ROI could guide for decision
support on the What?, How?, and How Much? analytics for a given problem.

Method: The proposed conceptual framework is validated through two empirical
studies that focus on requirements dependencies extraction in the Mozilla
Firefox project. The two case studies are (i) Evaluation of fine-tuned BERT
against Naive Bayes and Random Forest machine learners for binary dependency
classification and (ii) Active Learning against passive Learning (random
sampling) for REQUIRES dependency extraction. For both the cases, their
analysis investment (cost) is estimated, and the achievable benefit from DA is
predicted, to determine a break-even point of the investigation.

Results: For the first study, fine-tuned BERT performed superior to the Random
Forest, provided that more than 40% of training data is available. For the
second, Active Learning achieved higher F1 accuracy within fewer iterations
and higher ROI compared to Baseline (Random sampling based RF classifier). In
both the studies, estimate on, How much analysis likely would pay off for the
invested efforts?, was indicated by the break-even point.

Conclusions: Decisions for the depth and breadth of DA of empirical data
should not be made solely based on the accuracy measures. Since ROI-driven
Data Analytics provides a simple yet effective direction to discover when to
stop further investigation while considering the cost and value of the various
types of analysis, it helps to avoid over-analyzing empirical data.
keywords: Data Analytics, Return-on-Investment, Requirements Engineering,
Dependency extraction, BERT, Mozilla

##  1 Introduction

Return-on-Investment (ROI) is of great interest in engineering and business
for arriving at decisions. This is true in Software Engineering (SE) as well.
For example, Silverio et al. [16] evaluated cost-benefit analysis for the
adoption of software reference architectures for optimizing architectural
decision-making. Cleland et al. [8] studied the ROI of heterogeneous solutions
for the improvement of the ROI of requirements traceability. Recent data
explosion in the form of big data and advances in Machine Learning (ML) have
posed questions on the efficiency and effectiveness of these processes that
have become more relevant. In this paper, we present a retrospective
evaluation of two empirical studies taken from the field of requirements
dependency analysis for the benefit of ROI.

Data Analytics in SE (also called ”Software Analytics” by Bird et al. [6]) is
a term widely used, sometimes with a slightly different meaning. We subsume
all efforts devoted to collecting, cleaning, preparing, classifying, analyzing
data, and interpreting the results as Data Analytics (DA). In SE, the goal of
DA is to provide better insights into some aspects of the software development
life-cycle, which could facilitate some form of understanding, monitoring, or
improvement of processes, products or projects.

Figure 1: Break-even point from cost-benefit analysis of technology
investment.

SE is uncertain in various ways. SE is highly human-centric, and processes are
not strictly repeatable. The goals and constraints of software development are
dynamically changing. Experimentation and DA are inherently arduous under such
circumstances. The famous Aristotle [2] is widely attributed with a saying,
”It is the mark of an educated mind to rest satisfied with the degree of
precision which the nature of the subject admits and not to seek exactness
where only an approximation is possible”. Figure 1 shows a typical ROI (cost-
benefit) curve of technology usage. Following some phase of increase, the
curve reaches saturation, so, beyond that point, further investment does not
pay off. We contemplate that a similar behaviour holds true for applying DA.
Our research hypothesis is that ROI-driven DA helps to determine the break-
even point of investment and thus optimizes resources spent in this process.

Paper structure: Section 2 discusses related work. The problem formulation is
detailed in Section 3. Section 4 explains the empirical ROI investigation
approach for the two problems. A discussion of the applicability of the
results is elaborated in Section 5. Finally, Section 6, provides an outlook on
future research.

##  2 Related work

###  2.1 ROI Analysis in Software Engineering

Evaluating the profitability of expenditure helps to measure success over a
period of time thus takes the guesswork away from the concrete decision-making
process. For instance, Erdogmus et al. [15] analyzed the ROI of quality
investment to bring its importance in perspective and posed important
questions, “We generally want to increase a software products quality because
fixing existing software takes valuable time away from developing new
software. But how much investment in software quality is desirable? When
should we invest, and where?”.

Begel & Zimmermann [3] composed a set of 145 questions - based on a survey
with more than 200 developers and testers - that are considered relevant for
DA at Microsoft. One of the questions:“How important is it to have a software
DA team answer this question?”, expected answer on a five-point scale
(Essential to I don’t understand). Although it provides a sneak peek of the
development and testing environments of Microsoft, it does not prove any
emphasis on any form of ROI. Essentially, we speculate that the ROI aspect was
softened into asking for the perceived subjective importance through this
question.

Boehm et al. [7] presented quantitative results on the ROI of Systems
Engineering based on the analysis of the 161 software projects in the COCOMO
II database. Van Solingen [29] analyzed the ROI of software process
improvement and took a macro perspective to evaluate corporate programs
targeting the improvement of organizational maturity. Ferrari et al. [17]
studied the ROI for text mining and showed that it has not only a tangible
impact in terms of ROI but also an intangible benefits - which occur from the
investment in the knowledge management solution that is not directly
translated into returns, but that must be considered in the process of
judgment to integrate the financial perspective of analysis with the non-
financial ones. A lot of benefits occurring from the investment in this
knowledge management solution are not directly translated into returns, but
they must be considered in the process of judgment to integrate the financial
perspective of analysis with the non-financial ones.

Ruhe and Nayebi [23] proposed the Analytics Design Sheet as a means to sketch
the skeleton of the main components of the DA process. The four-quadrant
template provides direction to brainstorm candidate DA methods and techniques
in response to the problem statement and the data available. In its nature,
the sheet is qualitative. ROI analysis goes further and adds a quantitative
perspective for outlining DA.

###  2.2 Empirical Analysis for Requirements Dependency Extraction

The extraction of dependencies among requirements is an active field of SE
research. The practical importance of the topic was confirmed by our survey
[13]. More than 80% of the participants agreed or strongly agreed that (i)
dependency type extraction is difficult in practice, (ii) dependency
information has implications on maintenance, and (iii) ignoring dependencies
has a significant ill impact on project success.

In the recent past, many empirical studies have explored diverse computational
methods that used natural language processing (NLP) [10] [24], semi-supervised
technique [11], hybrid techniques [12] and deep learning [18]. However, none
of the approaches considered ROI to decide among techniques and the depth and
breadth of their execution level.

##  3 Conceptual Framework for ROI-driven Data Analytics

Different models exist that provide guidance to perform DA. Wieringa [30]
provides a checklist for what he calls the design cycle and the empirical
cycle. In this study, we use the term Scoping for defining the problem and the
analysis objectives. Scoping also means defining the boundaries that help to
exclude non-essential parts of the investigation. Analysis of the projected
Return-on-Investment (ROI) serves as an input for scoping.

###  3.1 Research Question

DA follows a resource and computation-intensive process constituting data
gathering and processing components that are the non-trivial proportion of the
total research cost. Thus, it is essential to account for these to compute the
overall cost-benefit and optimize it further.

Our aim is to look at DA for empirical studies retrospectively (already
conducted studies in the past). In particular, we are interested in
Requirements Dependency Analysis (RDA) based studies. Through this research,
we define and validate the principal concepts needed for ROI-driven DA. Our
research question is:

RQ: What are the benefits of ROI-driven Data Analytics in the studies focusing
on Requirements Dependency Analysis?

Justification: As for any investment, it is most important to know how much is
enough. There is no incentive to invest in analytics just for the sake of
performing some analysis. Although one cannot claim exactness from this, it is
worthwhile to get some form of guidance on where (which techniques) and how
far (how much of it) one should go. To make the analysis concrete, we have
selected RDA as the area of our specific investigations.

Table 1: Parameters used for ROI computation | Symbol |  Meaning | Unit
---|---|---|---
Cost | $C_{dg}$ |  Data gathering time | Minutes
$C_{pp}$ |  Pre-processing time | Minutes
$C_{e}$ |  Evaluation time | Minutes
$C_{l}$ |  Labeling time | Minutes
| $C_{resource}$ |  Human resource cost | $ per hour
Benefit | $B_{reward}$ |  Value per TP | $
$B_{penalty}$ |  Penalty per FN | $
$BF1_{iteration}$ |  F1 difference | Number
$PValue$ |  Projected value per 1% F1 improvement | $
Others | $H$ |  #Human resources | Number
$N_{train}$ |  Size of the training set | Number
$N_{test}$ |  Size of the test set | Number
$N$ |  $N_{train}$ \+ $N_{test}$ | Number

###  3.2 Cost Factors

Data processing is an umbrella term used to combine data collection
($C_{dg}$), pre-processing ($C_{pp}$) and labeling ($C_{l}$) under one hood,
each one of which is a cost component. However, not all costs are fixed and
some vary based on the solution approach used to tackle any decision problem.
For example, supervised Machine Learning (ML) requires a large amount of
annotated data, to begin with, whereas Active Learning acquires these
annotations over a period of time in iterations until a stopping condition for
classification operation is reached [25]. Additionally, there is a cost
associated with modeling and evaluation ($C_{e}$).

###  3.3 Value Factors

The value returns or “benefits” are defined based on the needs of the decision
problem. In the context of dependency extraction, the benefit could be modeled
in terms of the ability of the ML model to identify a larger number of
dependencies correctly (higher # of True Positives TP: $B_{reward}$) while
limiting misclassification (reduced # of False Negatives FN: $B_{penalty}$).
Conversely, the benefit could also be determined based on the net value
($PValue$) of change of accuracy ($BF1_{iteration}$) in every iteration,
especially when using Active Learning. Table 1 lists the relevant cost
components and their corresponding units. These will be utilized to compute
the $ROI$ later for the two different problems in Section 4.4.

###  3.4 ROI

To determine the ROI, we follow the simplest form of its calculation relating
to the difference between $Benefit$ and $Cost$ to the amount of $Cost$. Both
$Benefit$ and $Cost$ are measured as human effort in person hours.

$\centering ROI=(Benefit-Cost)/Cost\@add@centering$ (1)

Costa et al. [9] distinguished the “hard ROI” from the “soft ROI”. The former
refers to the direct additional revenue generated and cost savings. The latter
improved productivity, customer satisfaction, technological leadership, and
efficiencies.

##  4 ROI of Techniques for Requirements Dependency Analysis

We have selected the area of requirements dependency analysis (RDA) to
illustrate and initially validate our former conceptual framework. In what
follows, we introduce the key terms needed to formulate two Empirical Analysis
Studies called EAS 1 resp. EAS 2.

###  4.1 Problem statement

Following are the definitions of dependency types that are used to state the
two studies. For a set of requirements $R$ and a pair of requirements $(r,s)$
$\epsilon$ $R\times R$

  * 1)

An INDEPENDENT relationship is defined as the absence of any form of
relationship between a pair of requirements.

  * 2)

A DEPENDENT relationship is defined as the complement set of INDEPENDENT.
i.e., there exists at least one type of the dependency types such as REQUIRES,
SIMILAR, OR, AND, XOR, value synergy, effort synergy etc. between $r$ and $s$.

  * 3)

REQUIRES is a special form of DEPENDENT relationship. If $r$ requires $s$, or
$s$ requires $r$, then, $r$ and $s$ are in a REQUIRES relationship

  * 4)

OTHER type of dependency is when $(r,s)$ is DEPENDENT and the dependency type
is not REQUIRES (could be any of the other dependency types mentioned in (2))

  1. Problem 1-

Binary requirements dependency extraction: For a given set $R$ of requirements
and their textual description, the binary requirements dependency extraction
problem aims to classify each pair (r,s) $\epsilon$ $R\times R$ as DEPENDENT
or INDEPENDENT.

  2. Problem 2-

Specific requirements dependency extraction of the type REQUIRES: For a given
set $R$ of requirements and their textual description, the REQUIRES dependency
extraction problem aims to classify for each pair (r,s) $\epsilon$ $R\times R$
if they are in a $REQUIRES$ relationship.

###  4.2 Empirical Analysis Studies (EAS)

In this section, we formulate two Empirical Analysis Studies, EAS 1 and EAS 2,
to investigate the two problems explained above. We aim to analyze and compare
Bidirectional Encoder Representations from Transformers (BERT), and Active
Learning (AL), both proven to be of interest in general and pre-evaluated for
their applicability to the stated problems, with traditional ML. For the two
studies, we examine the (F1) accuracy and the ROI of the whole process of DA.

EAS 1: We compare two supervised classification algorithms: Naive Bayes (NB)
and Random Forest (RF) - ML algorithms successfully and prominently used for
text classification[19] in the past, with a fine-tuned BERT model [14]. The
analysis was performed for an incrementally growing training set size to
capture its impact on F1 accuracy and ROI.

BERT (Bidirectional Encoder Representations from Transformers) [14] is a
recent technique published by researchers from Google. BERT is applying
bidirectional training of Transformer, a popular attention model, to language
modeling, which claims to be state-of-the-art for NLP tasks. In this study
scenario, we explore the question, “How does fine-tune BERT compare with
traditional algorithms on an economical scale?” by comparing models’
effectiveness with incurred ROI.

EAS 2: Random sampling (Passive Learning) randomly selects a training set -
referred to as Baseline in the rest of the paper. Active Learning selects the
most informative instances using various sampling techniques such as MinMargin
and LeastConfidence [25]. We compare Baseline with AL using RF as a classifier
for this scenario. The analysis was done by adding a few training samples in
every iteration concurrently to classify the unlabeled instances.

Active Learning (AL) is a ML method that guides a selection of the instances
to be labeled by an oracle (e.g., human domain expert or a program) [25].
While this mechanism has been proven to positively address the question, “Can
machines learn with fewer labeled training instances if they are allowed to
ask questions?”, through this exploration, we try to answer the question,“Can
machines learn more economically if they are allowed to ask questions?” [26].

###  4.3 Data

The online bug tracking system Bugzilla [20] is widely used in open-source
software development. New requirements are logged into these systems in the
form issue reports [27] [4] which help software developers to track them for
effective implementation [28], testing, and release planning. In Bugzilla,
feature requests are a specific type of issue that is typically tagged as
“enhancement” [21]. We retrieved these feature requests or requirements from
Firefox and exported all related fields such as Title, Type, Priority,
Product, Depends_on, and See_also.

Data collection: Collecting data from Bugzilla was a substantial effort that
was carried out in multiple rounds. We collected 3,704 enhancements from
Firefox using REST API through a python script such that each one of the
enhancements considered for retrieval is dependent on at least another one in
the dataset. The data spanned from 08/05/2001 to 09/08/2019.

Data preparation: The complete data was analyzed to eliminate special
characters and numbers. Then dependent requirement pairs were created based on
the depends_on (interpreted as REQUIRES dependency) field information for each
one of the enhancements. Requirements with no dependency between them were
paired to generate INDEPENDENT class dataset. Further, sentence pairs that had
fewer than three words in them were filtered out resulting in 3,373 REQUIRES,
219 OTHER and 21,358 INDEPENDENT pairs.

Pre-processing and feature extraction: The data was first processed to
eliminate stop words and then lemmatized following the traditional NLP
pipeline [1]. For supervised and AL ML, we used the Bag Of Words (BOW) [22]
feature extraction method, which groups textual elements as tokens. For
applying BERT, we retained sentence pairs in their original form (without stop
word removal and lemmatization).

Classifiers: For both NB and RF, the data was split into train and test
(80:20) and balanced between classes. Also, hyper-parameter tuning was
performed and the results for 10-fold cross-validation were computed, followed
by testing (on unseen data).

To fine-tune the BERT model, we used
NextSentencePrediction111https://huggingface.co/transformers/model_doc/bert.html#bertfornextsentenceprediction,
a sentence pair classification pre-trained BERT model, and further fine-tuned
it for the RDA specific dataset on Tesla K80 GPU on Google
Colab222https://colab.research.google.com/.

###  4.4 ROI Modeling

####  4.4.1 EAS1

The classification algorithms such as RF and NB, have been explored in NLP
based SE problems. These algorithms are driven by the feature extraction
aspect to a great extent. Thus, could influence their effectiveness on
classification outcomes. However, feature extraction is problem specific and
incurs substantial cost and access to domain expertise.

On the other hand, BERT eliminates the need for feature extraction since it is
a language model based on deep learning. BERT, pre-trained on a large text
corpus, can be fine-tuned on specific tasks by providing only a small amount
of domain-specific data.

In this empirical analysis, we conducted classification by utilizing a
fraction of the whole dataset for training and testing for a small fixed data
set. This was repeated by slowly increasing the fraction of the training set
and results were captured.

During every classification, $Cost$ and $Benefit$ were computed using various
parameters explained in Table 1. $Cost$ is the sum of the data processing
costs ($(C_{dg}+C_{pp}+C_{e}+C_{l})/60$) (in hours) for a fraction (N%) of
training set. This is further translated into dollar cost based on hourly
charges ($C_{resource}$) of $H$ human resources.

$Cost=N\%*\frac{(C_{dg}+C_{pp}+C_{e}+C_{l})}{60}*H*C_{resource}$ (2)

$Return$ computations for RDA, assumes reward ($B_{reward}$) for identifying
the dependent requirements (TP) while penalizing ($B_{penalty}$) instances
that were falsely identified as independent (FN).

$Benefit=TP*B_{reward}-FN*B_{penalty}$ (3)

Table 2: Parameter settings for the two empirical analysis scenarios

[t]  Parameters Values $C_{fixed}=C_{dg}+C_{pp}+C_{e}$ 1 min/sample $C_{l}$
0.5 min/sample $C_{resource}$ $400/hr $H$ 1 $N$ 4,586 $B_{reward}$ $500/TP
$B_{penalty}$ $500/FN $BF1_{iteration}$ =$F_{cur}-F_{prev}$ $PValue$ $10,000
per percent F1 improvement

####  4.4.2 EAS 2

In this empirical analysis, we compared AL with a traditional random sampling
based classification- Baseline \- using the RF ML algorithm.

Beginning with 60 training samples of each class (REQUIRES, INDEPENDENT and
OTHER), we developed multi-class classifiers for both AL and Baseline for this
empirical study scenario. When AL used MinMargin sampling
technique333MinMargin sampling technique performed well compared to Least
Confidence and Entropy thus, we utilized MinMargin for this study to identify
20444The tests were performed with#samples = 10, 15 and 20. In this study, we
will discuss results related to #samples=20 most uncertain instance
(requirement pair) for oracle to label, baseline randomly selected 20
instances and added to the training set along with their label, thus, kept the
two approaches comparable in all the 20 iterations. Since data is already
labeled, for AL, we pretend they are unlabeled until queried and labeled by a
simulated oracle in this scenario.

The $Cost$ is determined by first computing the sum of total processing time
in person hours (= $Cost$) taken for data processing
($C_{fixed}=C_{dg}+C_{pp}+C_{e})$), labeling ($C_{l}$) of train set
($N_{train}$) and data processing cost ($C_{fixed}$) for testing. This is
further translated into dollar cost (=$C_{total}$) based on hourly charges
($C_{resource}$) of $H$ human resources.

$Cost=\frac{N_{train}*(C_{fixed}+C_{l})+N_{test}*C_{fixed}}{60}$
$C_{total}=Cost*H*C_{resource}$ (4)

Likewise, $Benefit$ is defined as the monetary value associated with a 1%
improvement in F1 score ($BF1_{iteration}$) between subsequent iterations.

$Benefit=BF1_{iteration}*PValue$ (5)

##  5 Results

Figure 2: F1 score plot for NB, RF and BERT trained over increasing training
set size, F1 improves, but plateaus beyond a certain point

In the real-world, cost and benefit values are hard to get and are uncertain.
All the results presented in this section are based on the parameter settings
given in Table 2. The settings reflect practical experience but are not taken
from a specific data collection procedure. We claim that the principal
arguments made in our paper are independent of these settings.

###  5.1 EAS 1

(a) F1 vs ROI for Random Forest

(b) F1 vs ROI for Fine tuned BERT

Figure 3: Empirical Analysis Scenario 1 (EAS 1)

Figure 2 provides the “accuracy only view” and shows that F1 gradually
increases with the increasing training size for the three ML algorithms: NB,
RF, and BERT. However, all three ML algorithms reach a saturation towards
larger training set sizes. While BERT performed exceptionally well when
training set size exceeded 42%, it could have been ideal to pre-determine “How
much training is enough?”. Thus we selected the top two classifiers (Figure
2): BERT and RF and applied the monetary values (Table 2) for the various cost
and benefit factors defined in Table 1 and computed the ROI.

Figure 3(a) and 3(b) show the results for RF and BERT, respectively. The ROI
behaviour is not monotonous and peaks for both cases. Although RF
classification achieved the highest ROI with just 20% of training set and
accuracy of F1 = 0.7, highest F1 value of 0.75 was achieved along with the
lowest ROI of 4.7.

For RF classification and applying ROI arguments, learning can be stopped with
20% of the training set.

Now looking at BERT classification, the best ROI-driven results: F1 = 0.84 and
an ROI = 8.43, were achieved with the 60% training set. Although F1 rose to
0.9 with 70% training set size, ROI dropped to 7.27. For the recommendation of
20% of training set size, ROI has a local optimum. BERT in general performs
well on the F1, however, is it worth the ROI? needs to be explored.

For training set sizes of at least 40% of the size of the whole set, BERT
performed better than RF in terms of both accuracy and ROI.

###  5.2 EAS 2

We analyzed the ROI for Baseline against AL for classifying the REQUIRES
class. The results are shown in Figure 4(a) and Figure 4(b). Similar to EAS 1,
we applied the values from Table 2 and equations (4) and (5) to compute cost
and benefit at every iteration for both the approaches. For the Baseline
approach, ROI peaked at 3.2 and F1 = 0.6, in the very 2nd iteration. Onwards,
ROI drastically decreased which indicated lesser value for increasing training
set by random sampling (Baseline) method.

Similar behavior was observed for the AL approach. shown in Figure 4(b). The
peak here was after three iterations with values ROI = 4.5 and F1 = 0.8.

Both Baseline and AL showed the best ROI performance in the early iterations.
Higher F1 accuracy needs additional human resources and reduces the ROI.

(a) F1 vs ROI for Baseline

(b) F1 vs ROI for AL

Figure 4: Empirical Analysis Scenario 2 (EAS2)

##  6 Discussion

For the problem of RDA, we explored the potential value of ROI-driven
decisions. When chasing higher accuracy, there is a risk of over analyzing
empirical data. In the sense that the value added due to increased accuracy is
not justifiable by the additional effort needed in achieving it.

What does a high or low ROI mean for DA? :  If available, a high ROI ratio
indicates that there is a substantial benefit expected from following the
recommendations derived from DA. Assuming that the ROI-driven suggestions are
implemented, the small improvements achieved for solving the decision problems
with high impact could justify the effort invested. Analysis related to effort
and benefit, targeting high ROI, also implies simplicity first. Advanced
methods are needed, but they are hard to justify practical application if a
similar type of insight could be reached from a much simpler analysis, e.g.,
from descriptive statistics.

What is the risk of ignoring analysis?:  The calculation of ROI is based on
the value and effort estimates and thus only provides an approximation. In all
types of exploratory data analysis, the emphasis is mainly on creating new
research hypotheses or validating existing assumptions. In these cases, the
notion of ROI is not the primary concern. Also, estimates for value and effort
needed are highly dependent; hence, the ROI might only serve as a soft
recommendation. On the other hand, whenever the ROI can be determined as a
reasonable estimate, even after using intervals of best and worst-case
performances, then ignoring ROI means to potentially waste effort for analysis
that does not pay off the investment made. For EAS 1, if the training size set
was limited to 30%, RF could be considered as a better choice over BERT.
However, with the possibility to increase the training set size, the BERT
approach could be favored.

##  7 Conclusions and Future Work

We proposed to complement Data Analytics of empirical studies with ROI
analysis to avoid over analyzing data in this vision paper. To validate the
need, we performed an analysis of accepted papers of ESEM conferences between
2015 and 2019 and found that 51 out of 190 papers (27%) were addressing some
form of DA. Among them, 39% included some consideration of cost, value, or
benefit. However, none of them directly explored or discussed ROI or used
cost-benefit analysis to decide the degree of DA needed. From a decision-
making perspective, selecting one out of many techniques, and for a selected
technique, deciding the termination of analysis amount to enlarge the scope
from one to two criteria.

Beyond accuracy, reflecting the benefit, it is essential to look into the
investment as well. Exclusively looking into the different aspects of accuracy
is cardinal, but it does not provide a full picture as the effort consumption
and impact are ignored. Effort estimation is well studied, however; prediction
of value [5] has not been explored as much. Even rough estimates may be
helpful to decide how much further investment into DA is reasonable. To make
this agenda successful, economical, business, and social concepts need to be
taken into account, apart from just the technical aspects.

## Acknowledgement

We thank Atharva Naik and Venessa Chan for useful comments. This work is
supported by the Natural Sciences and Engineering Research Council of Canada,
Discovery Grant RGPIN-2017-03948.

## References

  * [1] A. Arellano, E. Zontek-Carney, and M. A. Austin.  Frameworks for natural language processing of textual requirements.  International Journal On Advances in Systems and Measurements, 8:230–240, 2015.
  * [2] J. Barnes et al.  The Nicomachean Ethics.  Penguin, 2004.
  * [3] A. Begel and T. Zimmermann.  Analyze this! 145 questions for data scientists in software engineering.  In ICSE, pages 12–23, 2014.
  * [4] T. Bhowmik and S. Reddivari.  Resolution trend of just-in-time requirements in open source software development.  In 2015 IEEE Workshop on Just-In-Time Requirements Engineering (JITRE), pages 17–20. IEEE, 2015.
  * [5] S. Biffl, A. Aurum, B. Boehm, H. Erdogmus, and P. Grünbacher.  Value-based software engineering.  Springer Science & Biz Media, 2006.
  * [6] C. Bird, T. Menzies, and T. Zimmermann.  The art and science of analyzing software data.  Elsevier, 2015.
  * [7] B. Boehm et al.  The roi of systems engineering: Some quantitative results for software-intensive systems.  Systems Engineering, 11(3):221–234, 2008.
  * [8] J. Cleland-Huang, G. Zemont, and W. Lukasik.  A heterogeneous solution for improving the return on investment of requirements traceability.  In Proc. 12th IEEE Requirements Engineering Conference, 2004., pages 230–239. IEEE, 2004.
  * [9] A. Costa et al.  Intraoral hard and soft tissue depths for temporary anchorage devices.  In Seminars in orthodontics, volume 11, pages 10–15. Elsevier, 2005\.
  * [10] J. Dag et al.  A feasibility study of automated natural language requirements analysis in market-driven development.  RE, 7(1):20–33, 2002.
  * [11] G. Deshpande, C. Arora, and G. Ruhe.  Data-driven elicitation and optimization of dependencies between requirements.  Proc. RE, 2019.
  * [12] G. Deshpande et al.  Requirements dependency extraction by integrating active learning with ontology-based retrieval.  Proc. RE, 2020.
  * [13] G. Deshpande and G. Ruhe.  Survey: Elicitation and maintenance of requirements dependencies, Dec 2020\.
  * [14] J. Devlin et al.  Bert: Pre-training of deep bidirectional transformers for language understanding.  arXiv preprint arXiv:1810.04805, 2018.
  * [15] H. Erdogmus, J. Favaro, and W. Strigel.  Return on investment.  IEEE Software, 21(3):18–22, 2004.
  * [16] S. Fernández et al.  Rearm:a reuse-based economic model for software reference architectures.  In Int. Conf on Software Reuse, pages 97–112. Springer, 2013.
  * [17] M. Ferrari et al.  Roi in text mining projects.  WIT Transactions on State-of-the-art in Science and Engineering, 17, 2005.
  * [18] J. Guo et al.  Semantically enhanced software traceability using deep learning techniques.  In 2017 IEEE/ACM 39th ICSE, pages 3–14. IEEE, 2017.
  * [19] C. Manning et al.  Introduction to information retrieval.  Natural Language Engineering, 16(1):100–103, 2010.
  * [20] Mozilla.org.  Bugzilla: bug-tracking system, Apr 2020.
  * [21] Mozilla.org.  Userguide/bugfields, Apr 2020.
  * [22] J. Ramos et al.  Using tf-idf to determine word relevance in document queries.  In Proceedings of the first instructional conference on machine learning, volume 242, pages 133–142. New Jersey, USA, 2003.
  * [23] G. Ruhe and M. Nayebi.  What counts is decisions, not numbers — toward an analytics design sheet.  In Perspectives on Data Science for SE, pages 111–114. Elsevier, 2016.
  * [24] R. Samer, M. Stettinger, M. Atas, A. Felfernig, G. Ruhe, and G. Deshpande.  New approaches to the identification of dependencies between requirements.  In 31st Conference on Tools with Artificial Intelligence, ICTAI ’19. ACM, 2019.
  * [25] B. Settles.  Active learning literature survey.  Technical report, University of Wisconsin-Madison Department of Computer Sciences, 2009.
  * [26] B. Settles.  From theories to queries: Active learning in practice.  In Active Learning and Experimental Design workshop colocated with AISTATS 2010, pages 1–18, 2011.
  * [27] L. Shi et al.  Understanding feature requests by leveraging fuzzy method and linguistic analysis.  In Proc. of the 32nd Conf. on ASE, pages 440–450. IEEE Press, 2017\.
  * [28] Y. Shin, J. H. Hayes, and J. Cleland-Huang.  Guidelines for benchmarking automated software traceability techniques.  In Proceedings of the 8th Int. Symposium on Software and Systems Traceability, pages 61–67. IEEE Press, 2015.
  * [29] R. Van Solingen.  Measuring the roi of software process improvement.  IEEE software, 21(3):32–38, 2004.
  * [30] R. J. Wieringa.  Design science methodology for information systems and software engineering.  Springer, 2014.


# POSTER: spaceQUIC: Securing Communication in Computationally Constrained
Spacecraft

Joshua Smailes<EMAIL_ADDRESS>University of Oxford ,  Razvan
David<EMAIL_ADDRESS>University of Oxford ,  Sebastian Köhler
<EMAIL_ADDRESS>University of Oxford ,  Simon Birnbach
<EMAIL_ADDRESS>University of Oxford and  Ivan Martinovic
<EMAIL_ADDRESS>University of Oxford

###### Abstract.

Recent years have seen a rapid increase in the number of CubeSats and other
small satellites in orbit – these have highly constrained computational and
communication resources, but still require robust secure communication to
operate effectively. The QUIC transport layer protocol is designed to provide
efficient communication with cryptography guarantees built-in, with a
particular focus on networks with high latency and packet loss. In this work
we provide spaceQUIC, a proof of concept implementation of QUIC for NASA’s
“core Flight System” satellite operating system, and assess its performance.

††copyright: none

7cm(1.8cm,7.8cm) † Both authors contributed equally to this work.

##  1\. Motivation

Alongside a general upward trend in the number of satellites in orbit, there
has been a marked recent increase in the number of small satellites in space.
This increase has been driven by a number of factors, including a growing
availability of cheap Commercial Off-The-Shelf (COTS) components, satellite
ride-sharing (in which smaller satellites are launched alongside a larger
payload), and the rise in popularity of the CubeSat, a standardized design
enabling cheap ride-sharing.

Another significant factor has been software availability – access to open-
source operating systems and libraries allow operators to focus on building
payload-specific functionality, reducing unnecessary mission development. The
core Flight System (cFS) is a popular open-source satellite operating system
built by NASA from historical missions, and is used in many ongoing and
planned missions (mccomasCore2015, 1). It is actively maintained by NASA and
the open-source community surrounding the project, and is easily extensible
through the addition of libraries or apps to support specific payloads or
ancillary functions. It is built on top of an Operating System Abstraction
Layer (OSAL), making it easy to port to new hardware platforms, alongside
those for which it is already supported. For these reasons it is a popular
choice of satellite operating system and is used in a wide range of CubeSat
missions.

One key challenge in space systems is performant secure communication – the
vast majority of communication occurs over radio signals which are subject to
significant path loss, atmospheric noise, and multipath distortion. This
problem is exacerbated in CubeSats, which often use omnidirectional antennas
due to an inability to orient themselves, or antennas with lower gain due to
their limited size. As a result, much of these satellites’ communication is
low throughput and subject to data loss or corruption.

The TCP transport layer protocol is rarely used – its congestion control
algorithm assumes that packet loss is caused by a congested link and waits for
retransmission. These assumptions do not apply to point-to-point satellite
communications, and result in a significant decrease in throughput. Instead,
datagram-oriented protocols like UDP or the Space Packet Protocol (SPP) are
used. These are secured using symmetric cryptography through the Space Data
Link Security (SDLS) protocol, implemented in the CryptoLib cFS library
(ccsdsSpace2015, 2). This provides fewer security guarantees than asymmetric
cryptography, but requires less computational overhead.

The QUIC transport layer protocol, introduced in 2012, addresses these
concerns (iyengarQUIC2021, 3). The connection establishment process is highly
streamlined, and in many cases data can start being sent with 0 round trips of
setup. The protocol also provides improved congestion control and recovery
from losses, resulting in significantly better throughput over lossy and noisy
connections. Furthermore, security is built into the protocol, providing all
the security guarantees of an asymmetric cryptosystem with at most one network
round-trip time of setup. These factors make QUIC highly attractive in the
context of lossy satellite connections – existing work has shown that
combining QUIC with performance enhancing proxies can provide better
performance and security over satellite internet connections (pavurQPEP2021,
4). However, there is currently no way to leverage the benefits of QUIC when
in direct communication with satellites.

###  1.1. Contributions

In this work we introduce the spaceQUIC library, implementing QUIC
functionality on the cFS satellite operating system. This brings the
additional performance and resilience of QUIC to space missions, alongside the
increased security of asymmetric cryptography. This library can be used as a
replacement for CryptoLib, the existing library providing asymmetric
cryptography through the SDLS protocol.

We provide a high-level overview of the cFS architecture and describe how
spaceQUIC fits into this model. We also explain how to extend spaceQUIC to
work with real-world missions.

All code has been made open source under the Apache 2.0 license, and can be
found at https://github.com/ssloxford/spaceQUIC. For ease of setup, we also
provide an instance of cFS preconfigured to use spaceQUIC, both as source code
and a Docker container.

##  2\. Architecture

Figure 1. The overall architecture of a space system running cFS using
spaceQUIC.

Figure 1 shows the structure of a cFS system using spaceQUIC. Thanks to the
modular structure of cFS, both central cFS functionality, as well as mission-
specific applications and libraries, are left unchanged. All communication
occurs over a central software bus, with applications exchanging data through
a publish/subscribe message passing system. This means the underlying
communication stack is abstracted away from most of the system, with data sent
only via the software bus.

spaceQUIC is provided as a cFS library, giving access to the required QUIC
functionality. This library is used by modified Command Ingest (CI) and
Telemetry Output (TO) applications – these are provided as part of the
standard cFS system, and are used for processing commands, and sending
telemetry and housekeeping data back to the ground system. The provided “lab”
versions of these applications send data directly over the network, and are
modified on a per-mission basis to support that mission’s radio hardware.
Existing CI and TO applications can also be modified to support spaceQUIC,
replacing calls to CryptoLib or other security/networking libraries.

The spaceQUIC library supports two implementations of SSL/TLS: OpenSSL and
WolfSSL (wolfssl, 5). WolfSSL is designed for use in embedded devices and
optimized to minimize resource usage, making it ideal for small satellites.

##  3\. Performance

Table 1. Overall memory usage of cFS under each security configuration.

Security | Peak heap usage ($\text{\,}\mathrm{kB}$) | Peak RSS usage ($\text{\,}\mathrm{MB}$)
---|---|---
None | $84.5$ | $6.7$
SDLS | $89.5$ | $8.3$
QUIC (OpenSSL) | $583.3$ | $13.3$
QUIC (WolfSSL) | $344.8$ | $9.5$

Figure 2. Encryption and decryption times for SDLS and QUIC.

In this section we assess the performance and resource usage of spaceQUIC to
demonstrate its usefulness in embedded contexts.

All experiments were performed on a Dell laptop with an Intel i7-8750H CPU and
16GB of DDR4 memory, limited to a single thread. Embedded hardware comparable
to onboard satellite hardware was not available, so we focus on relative
performance.

Table 1 shows the memory usage of cFS under each configuration, looking at
both peak heap usage and Resident Set Size (RSS) of the process. We see that
QUIC uses significantly more heap space than SDLS, but when using WolfSSL
overall memory usage is only slightly increased. It is likely that there would
be no memory usage problems running QUIC on all but the most computationally
constrained spacecraft.

We also measured execution time, seen in Figure 2. From these results we
observe that QUIC is $1.5$ to $2.5$ times slower than SDLS – this is
unsurprising due to the greater requirements of asymmetric cryptography, but
further testing on embedded hardware is needed.

Further testing is also also required to measure performance when latency and
packet loss are high – due to the protocol’s design, spaceQUIC is likely to
perform well in these scenarios.

## References

  * (1) David McComas, Susanne Strege and Jonathan Wilmot  “Core Flight System (cFS) a Low Cost Solution for SmallSats”  In _Annual Small Satellite Conference_ , 2015
  * (2) CCSDS  “Space Data Link Security Protocol”, 2015
  * (3) Jana Iyengar and Martin Thomson  “QUIC: A UDP-Based Multiplexed and Secure Transport”, 2021  DOI: 10.17487/RFC9000
  * (4) James Pavur, Martin Strohmeier, Vincent Lenders and Ivan Martinovic  “QPEP: An Actionable Approach to Secure and Performant Broadband From Geostationary Orbit”  In _Proceedings 2021 Network and Distributed System Security Symposium_ Virtual: Internet Society, 2021  DOI: 10.14722/ndss.2021.24074
  * (5) wolfSSL  “wolfSSL”, 2023  URL: https://www.wolfssl.com/