# Quantum neural networks with multi-qubit potentials

Yue Ban  Department of Physical Chemistry, University of the Basque Country
UPV/EHU, Apartado 644, 48080 Bilbao, Spain  School of Materials Science and
Engineering, Shanghai University, 200444 Shanghai, People’s Republic of China
E. Torrontegui  Departamento de Física, Universidad Carlos III de Madrid,
Avda. de la Universidad 30, 28911 Leganés, Spain  Instituto de Física
Fundamental IFF-CSIC, Calle Serrano 113b, 28006 Madrid, Spain  J. Casanova
Department of Physical Chemistry, University of the Basque Country UPV/EHU,
Apartado 644, 48080 Bilbao, Spain  IKERBASQUE, Basque Foundation for Science,
Euskadi Plaza 5, 48009, Bilbao, Spain

###### Abstract

We propose quantum neural networks that include multi-qubit interactions in
the neural potential leading to a reduction of the network depth without
losing approximative power. We show that the presence of multi-qubit
potentials in the quantum perceptrons enables more efficient information
processing tasks such as XOR gate implementation and prime numbers search,
while it also provides a depth reduction to construct distinct entangling
quantum gates like CNOT, Toffoli, and Fredkin. This simplification in the
network architecture paves the way to address the connectivity challenge to
scale up a quantum neural network while facilitates its training.

##  I Introduction

Information has become a resource due to its advance and expansion in the
digitalization and control Castells96 . However, programing explicit
algorithms with good performance may become unfeasible due to the vertiginous
growth in (i) the amount of available information with which classical
algorithms have to deal Walter05 , and (ii) the inherent difficulty of finding
efficient algorithms for specific problems Kolmogorov63 . All this limits our
current capabilities in information processing tasks. Two alternative
approaches that would contest these limitations are machine learning and
quantum computing Samuel59 ; Feynman82 .

On the one hand, machine learning (ML) is a branch of artificial intelligence
that uses statistical techniques to give computers the ability to
progressively learn with input data without being explicitly programmed. ML is
based on the generation of a hypothesis that is optimized from sample inputs
and re-used to generate new predictions Hebb49 . Thus the algorithms can learn
from data and overcome the static program instructions by making data-driven
decisions from sample inputs. Among the distinct hypothesis models, neural
networks McCulloch43  are very extended due to the blooming of deep learning
Oh04 ; LeCun15 . Similar to biological networks, artificial neural networks
are organized in layers and each layer learns new behavior patterns Kleene56 .
The computational power of artificial neural networks relies on this
architecture where neurons in each layer feed signals into other neurons
allowing parallel-processed computing Rosenblatt57 ; Hopfield84 . In this
manner, several calculations can be performed at the same time, and large
computational problems can often be divided into smaller ones, which can be
then solved simultaneously. The versatility of neural networks to classify
complex data relies on the universal approximation theorem, which leads
artificial neural networks the capacity to approximate any function Cybenko89
. As result, they span a broad range of applications such as speech Dahl12  or
object recognition Hinton06 , spam filters Dada19 , vehicle control Buehler09
; Devi20 , trajectory prediction Valsamis17 , decision making Kashyap18 ,
game-playing Furnkranz10 , or automated trading systems Huang19 .

On the other hand, quantum computing represents a different paradigm from
classical information processing. Based on an alternative information encoding
that exploits the quantum properties of matter, systems that encompass several
quantum bits (qubits) are exponentially hard to simulate with classical
devices Feynman82  showing that quantum systems do not seem to obey Church
thesis Kleene36 , and consequently they are not polynomially equivalent to
classical systems. Then, quantum systems harnessed as computational devices,
might be dramatically more powerful than any other classical system Arute19 .
The universality of quantum computing Kitaev97  expands a broad range of
applications. Some examples are linear systems solvers Harrow09 , molecule
simulators Peruzzo14 , combinatorial optimizers Farhi14 , black-box Grover96
and factorization problems Shor94 , or Hamiltonian simulations Lloyd96 .
Although a set of single $N=1$ and two-qubit $N=2$ gates is a universal
approximator Nielsen11 , larger multi-qubit ($N>2$) gates may offer a
computational advantage that reduces complexity in existing algorithms
Mottonen04 ; Reagor18 ; Baekkegaard19 .

Quantum machine learning Kak95 ; Schuld14 ; Biamonte17 ; Farhi18 ; Schuld19 ;
Schuld20 ; Cao17 ; Salina20 ; Torrontegui19  aims for the symbiosis of both
paradigms to the mutual reinforcement. The use of quantum resources allows
improvements of the machine learning protocol accuracy in classification and
regression tasks compared to their classical counterparts Gupta01 ; Paparo14 ;
Benedetti17 ; Sentis15 ; Havlicek19 . On the other hand, the universality of
artificial networks may enhance the accuracy and efficiency of quantum
protocols Carrasquilla17 ; Deng17 ; Torlai18 ; Aharon19 ; Ban20 .

In this work, we propose an extension of a universal quantum neural network
(QNN) Torrontegui19  enabling multi-qubit ($N>2$) interactions that lead to an
enhancement of the approximation power, and a reduction of the network depth.
As result, achieving a simplification of existing protocols not only requires
shorter operation times, but it may also introduce less accumulation of errors
due to the reduction in the amount of requested gates. The paper is structured
as follows: We firstly introduce the quantum perceptron model under the
presence of multi-qubit interactions. We define the quantum perceptron as a
single-output quantum neuron connected to several-input quantum neurons
without hidden layers. The result of nesting several quantum perceptrons is a
QNN. We show that a quantum perceptron with multi-qubit interactions can do an
XOR gate and prime number search from $3$ to $5$ bits, improving the
performance of approximation power compared to the classical counterpart, as
well as with standard QNNs (i.e. QNNs that include several nested perceptrons
without multi-qubit interactions). Then, we show that quantum gates such as
CNOT, Toffoli, and Fredkin can be implemented by QNN involving quantum
perceptrons with multi-qubit interactions, thus reducing the circuit depth as
it is not needed to add hidden layers.

##  II Quantum perceptrons with multi-qubit potentials

A quantum perceptron, or quantum neuron, is the basic building block of a QNN.
It can be constructed as a qubit that presents a nonlinear response to an
input potential $\hat{x}_{j}$ in the excitation probability. This can be
written as the following quantum gate acting on a $j$th qubit that encodes the
quantum perceptron Torrontegui19 ; Ban20 :

$\displaystyle\hat{U}_{j}(\hat{x}_{j};f)|0_{j}\rangle=\sqrt{1-f(\hat{x}_{j})}|0_{j}\rangle+\sqrt{f(\hat{x}_{j})}|1_{j}\rangle.$
(1)

The transformation in Eq. (1) can be engineered by evolving adiabatically the
qubit with, for instance, the Hamiltonian

$\hat{H}=\frac{1}{2}\left[\hat{x}_{j}\hat{\sigma}^{z}_{j}+\Omega(t)\hat{\sigma}_{j}^{x}\right]$
(2)

where $\hat{x}_{j}$ is the potential exerted by other neurons on the
perceptron, and the applied external field $\Omega(t)$ leads to a tunable
energy gap in the dressed-state qubit basis $|\pm\rangle$, with
$\hat{\sigma}^{x}_{j}|\pm\rangle=\pm|\pm\rangle$. Typically,
$\hat{x}_{j}=\sum_{i=1}^{k}(w_{ji}\hat{\sigma}^{z}_{i})-b_{j}$ Torrontegui19 ;
Ban20  which implies that the $j$th perceptron is coupled to a number $k$ of
neurons (labelled with $i$) in the previous/input layer via standard spin-spin
interactions. The Hamiltonian in Eq. (2) has the following reduced eigenstate
(i.e. when the degrees of freedom of any other neuron are traced-out):

$\displaystyle|\Phi(x_{j}/\Omega(t))\rangle=\sqrt{1-f(x_{j}/\Omega(t))}|0_{j}\rangle+\sqrt{f(x_{j}/\Omega(t))}|1_{j}\rangle,~{}~{}~{}~{}$
(3)

where $f(x)$ corresponds to a sigmoid excitation probability

$\displaystyle f(x)=\frac{1}{2}\left(1+\frac{x}{\sqrt{1+x^{2}}}\right).$ (4)

Specifically, to accomplish $\hat{U}_{j}(\hat{x}_{j};f)|0_{j}\rangle$, one can
use a Hadamard gate to firstly get the transformation
$|0_{j}\rangle\rightarrow|+_{j}\rangle$ and finally obtain
$|\Psi\rangle=|\Phi(x_{j}/\Omega(t_{f}))\rangle$ at a certain time $t_{f}$ by
evolving the system adiabatically with Hamiltonian (2). In this manner, the
non-linear activation function of the quantum perceptron is encoded in the
probability of the excited state
$P_{j}=f(\hat{x}_{j})=\frac{1}{2}(1+\langle\hat{\sigma}_{j}^{z}\rangle)$
during the adiabatic evolution Torrontegui19 . In order to speed up the
operation of this perceptron, one can also use inverse engineering techniques
which directly impose conditions in the wave function evolution at the initial
and final time instants, resulting in a smooth control $\Omega(t)$ Ban20 . In
addition, this accelerated activation mechanism for the quantum neurons would
reduce the decoherence in the system leading to enhanced performance.

Now, we introduce a different type of potentials which rely on the possibility
to implement multi-qubit interactions. In particular, we consider a potential
of the kind

$\hat{x}_{j}=\sum_{i=1}^{k}(w_{ji}\hat{\sigma}^{z}_{i})+w_{\rm{m}}\hat{\sigma}_{l_{1}}^{z}...\
\hat{\sigma}_{l_{n}}^{z}-b_{j}.$ (5)

where $w_{\rm{m}}$ is a multi-qubit coefficient marked by the subscript
$\rm{m}$, and $l_{p}\in[1,2,...,k]$ (namely, the term involving several Pauli
matrices includes products of an arbitrary number of neurons in the previous
input). For the sake of simplicity in the presentation, there is only a single
multi-qubit term in Eq. (5). However, this can include several products of
distinct neurons in the input layer (i.e. additional multi-qubit terms). Later
we will provide examples of these of interactions each of them associated to a
specific problem.

The Hamiltonian given by Eqs. (2) and (5) corresponds to the linear Ising
Hamiltonian with higher order interactions. Such a model is present in
distinct quantum platforms Donskaya1984 ; Kumar20 ; Hartmann17 .
Interestingly, quantum evolutions governed by terms involving up to six-qubit
interactions has already been implemented in trapped-ion systems Lanyon11 .
Each of the next examples considered below correspond to a particular case of
multi-qubit interaction and its physical implementation would depend on the
specific considered platform.

In the following, we show that multi-qubit potentials enable tasks such as (i)
constructing XOR gates at the perceptron level (ii) searching prime numbers
and (iii) encoding quantum gates. All these are implemented without hidden
layers and/or ancillary qubits, thus, showing the significant role of multi-
qubit potentials in the simplification of QNNs.

###  II.1 XOR gate

As a classical perceptron is a linear separator, a nonlinear logic gate such
as the well-known XOR problem (i.e., the exclusive OR boolean function)
requires at least one hidden layer to be implemented in classical neural
networks perceptrons . Now we show that a quantum perceptron with multi-qubit
interactions in the neural potential is a non-linear classifier. In
particular, we illustrate the construction of an XOR gate by a single quantum
perceptron with multi-qubit interactions. We also show that the lack of hidden
layers prevents classical neural network and standard QNN to achieve the same
task.

Figure 1:  In order to (a) encode an XOR gate and to (b) search prime numbers
among the integers from $0$ to $7$, we show the value of the cost function at
different epoches, using a multi-qubit interaction perceptron (solid blue) and
a classical perceptron (dotted red). The latter is equivalent to a quantum
perceptron with only two-qubit terms, which is shown from the comparison of
the weights and bias during the training process, see Eqs. (II.1, II.1, 9),
where the learning rate is $\eta=1.5$. The transfer function in Eq. (4) is
used for both cases. In the quantum perceptron, this function indicates the
system evolves adiabatically.

As the values of the output neuron $0$ and $1$ cannot be separated linearly, a
classical perceptron with two inputs and one output (i.e. without hidden
layers) fails to solve the XOR gate. To show this, we use the standard
gradient descent algorithm to train this simple classical perceptron with a
sigmoidal activation potential $f(x)$, see Eq. (4), and
$x_{j}=\sum_{i=i}^{k}w_{ij}s_{i}-b_{j}$ with the classical input
$s_{j}\in\\{0,1\\}$ whose cost function is in the form of the mean square
value

$\displaystyle C=\frac{1}{2N}\sum_{n=1}^{N}(y^{(n)}-t^{(n)})^{2}.$ (6)

Here, $N=4$ determines the four possible examples $00,10,01,11$, while
$y^{(n)}$ and $t^{(n)}$ are the output and the target respectively, for the
$n$th example. During the training, the parameters of the classical perceptron
are updated after each epoch as

$\displaystyle\tilde{w}_{i}$ $\displaystyle=$ $\displaystyle
w_{i}-\eta\frac{\partial C}{\partial
w_{i}}=w_{i}-\frac{\eta}{N}\sum_{n=1}^{N}\left(y^{(n)}-t^{(n)}\right)f^{\prime}(x)s_{j},$
$\displaystyle\tilde{b}$ $\displaystyle=$ $\displaystyle b-\eta\frac{\partial
C}{\partial b}=b-\frac{\eta}{N}\sum_{i=1}^{N}(y^{(n)}-t^{(n)})f^{\prime}(x),$
(7)

with the learning rate $\eta$. Any other sigmoid-like function $g(x)$, similar
to $f(x)$, can be applied to train this perceptron in order to obtain the same
approximation power. As it is shown in Fig. 1 (a), the cost function value of
this classical perceptron stucks in $C=0.125$ (dotted-red line with
superimposed diamonds for a better identification). Thus, a classical
perceptron can only converge on linearly separable data, i.e. it is not able
to imitate the XOR function. In order to complete an XOR, a hidden layer with
two neurons is needed. These two neurons can be regarded to perform an OR and
a NAND gate.

To implement an XOR gate with a quantum perceptron without the multi-qubit
term in Eq. (5), this is by using the potential
$\hat{x}=w_{1}\hat{\sigma}_{1}^{z}+w_{2}\hat{\sigma}_{2}^{z}-b$, one can
follow the procedure described in Ref. Torrontegui19 . To encode an XOR gate,
the neural potential is derived from the four basis states $|00\rangle$,
$|01\rangle$, $|10\rangle$, $|11\rangle$, which play the role of the four
examples of the input in the XOR gate (namely, $00,10,01,11$). In this case,
as the input values of the XOR gate are bits, one can transform them into the
measurement value of $\hat{\sigma}^{z}$ of the input qubits for the
perceptron, i.e.
$\sigma^{z}_{\rm{in}}=\frac{1}{2}(1+\langle\hat{\sigma}^{z}\rangle)$.
Therefore, the input values $0$ and $1$ refer to the input states as the
ground state $|0\rangle$ ($\langle\hat{\sigma}^{z}\rangle=-1$) and the excited
state $|1\rangle$ ($\langle\hat{\sigma}^{z}\rangle=1$), respectively.

Such a quantum perceptron without hidden layers and with only two-qubit
interactions is equivalent to a classical perceptron, from the point of view
of the following training process. Aiming at training it with the gradient
descent method, one has to update the weights and bias as

$\displaystyle\tilde{w}_{i}$ $\displaystyle=$ $\displaystyle
w_{i}-\frac{\eta}{N}\sum_{n=1}^{N}\left(y^{(n)}-t^{(n)}\right)\frac{\partial
C}{\partial x}\frac{\partial x}{\partial w_{i}},$ $\displaystyle\tilde{b}$
$\displaystyle=$ $\displaystyle
b-\frac{\eta}{N}\sum_{i=1}^{N}(y^{(n)}-t^{(n)})\frac{\partial C}{\partial x},$
(8)

where

$\displaystyle\frac{\partial C}{\partial
x}=\frac{1}{2}\left(\left\langle\frac{\partial\Psi}{\partial
x}\bigl{|}\hat{\sigma}_{z}\bigr{|}\Psi\right\rangle+\left\langle\Psi\bigl{|}\hat{\sigma}_{z}\bigr{|}\frac{\partial\Psi}{\partial
x}\right\rangle\right)=f^{\prime}(x)$ (9)

and $|\Psi\rangle=\hat{U}(x,f)|0\rangle$ is the solution to the Schrödinger
equation driven by the Hamiltonian (2) with
$\hat{x}=w_{1}\hat{\sigma}_{1}^{z}+w_{2}\hat{\sigma}_{2}^{z}-b$. In the
previous equations we can see that weights and bias are obtained in the same
way as their classical counterparts. For that, one has to compare Eq. (II.1)
with Eqs. (II.1) and (9) provided that $\frac{\partial x}{\partial
w_{i}}=\hat{\sigma}_{j}^{z}\rightarrow s_{j}$. This indicates that a single
quantum perceptron with two-qubit interactions and the basis states
$|00\rangle$, $|01\rangle$, $|10\rangle$, $|11\rangle$ as input is equivalent
to a classical perceptron.

In order to implement the XOR gate with this quantum perceptron, one would
need to perform two adiabatic passages with different controls $\Omega(t)$, as
well as to use different neural potentials in each passage by appropriately
changing the weights and biases Torrontegui19 . Equivalently, one could also
do the XOR gate by including one hidden layer with two additional quantum
neurons and the application of a single $\Omega(t)$. It is worth mentioning
that Eq. (9) holds only for a quantum perceptron instead of a QNN with hidden
layers. If a QNN has more layers, one only needs to measure the output qubit
value $P(x_{j})=\frac{1}{2}(1+\langle\Psi|\hat{\sigma}^{z}_{j}|\Psi\rangle)$
instead of the intermediate ones, i.e.,
$|\Psi\rangle=\hat{U}_{\textrm{tot}}|0\rangle$ with
$\hat{U}_{\textrm{tot}}=\Pi_{j=1}^{M}\hat{U}_{j}$ where $M$ is the total
number of quantum perceptrons in QNN. Otherwise, shot noise is introduced by
measuring the neurons in the hidden layer.

This procedure to do the XOR gate can be simplified when considering a quantum
perceptron with multi-qubit interactions. In particular, we explore the
following multi-qubit potential

$\hat{x}=w_{1}\hat{\sigma}_{1}^{z}+w_{2}\hat{\sigma}_{2}^{z}-b+w_{\rm{m}}\hat{\sigma}_{1}^{z}\hat{\sigma}_{2}^{z}.$
(10)

For simplicity, the subscript $j$ labelling the perceptron is neglected, as we
are working with a single quantum perceptron. In this case, the weights
$w_{1}$, $w_{2}$ and $b$ are updated as in Eqs. (II.1), while the updating
formula for the weight of the
$w_{\rm{m}}\hat{\sigma}_{1}^{z}\hat{\sigma}_{2}^{z}$ term is

$\displaystyle\tilde{w}_{\rm{m}}$ $\displaystyle=$ $\displaystyle
w_{\rm{m}}-\frac{\eta}{N}\sum_{n=1}^{N}\left(y^{(n)}-t^{(n)}\right)\frac{\partial
C}{\partial x}\hat{\sigma}^{z}_{1}\hat{\sigma}^{z}_{2}.$ (11)

The existence of $w_{\rm{m}}\hat{\sigma}_{1}^{z}\hat{\sigma}_{2}^{z}$ in the
neural potential enables the quantum perceptron to construct an XOR gate as a
non-linear separator. To prove this point, we test the cost function value
(see, Eq. (6)) for our quantum perceptron and find that $C<1\%$ occurs at
epoch $=197$ as shown in Fig. 1 (a) (solid-blue line with squares). On the
contrary the classical and qubit-qubit interaction perceptron are not able to
produce an XOR gate, see plateau behavior of the cost function (dotted-red)
that remains constant after the epoch $\sim 50$.

###  II.2 Searching prime numbers

In Ref. Torrontegui19 , a specific example of two, three, and four perceptrons
per layer was illustrated to detect prime numbers from $0$ to $2^{i}-1$ where
$i$ ranges from $i=3$ to $i=7$ bits. In particular, it was shown that a QNN
with two perceptrons –one in the hidden layer and the other one as the output–
can classify prime numbers from $0$ to $7$ (3 bits). This exemplifies the
better performance of QNNs compared with classical ones where a hidden layer
with two neurons are necessary to accomplish the same task. A scheme of these
networks is shown figure in Fig. 2 (a) and (b), while details regarding QNN-
training to search prime numbers are in Appendix A.

Figure 2:  (a) Schematic configuration of a classical neural network with one
hidden layer of two neurons for the task to search prime numbers of the input
number $A=(a_{1},a_{2},a_{3})\in\\{0,1,...,7\\}$ with $3$ bits. After well
trained, the network gives the output $y=Q(A)$. (b) The same task can be
achieved by a QNN which has multi-qubit interactions in the neural potential
(red junction involving second and third neurons) without hidden layers. (c)
Truth table for prime-number search with $3$ input bits for a QNN. The input
values $A=(a_{1},a_{2},a_{3})$ are the binary numbers for the integers from
$0$ to $7$. The output value $y=Q(A)$ is $1$ for prime numbers and $0$ for
non-prime ones.

Now, we introduce the multi-qubit term into the neural potential and find that
the same task is achieved by a QNN without hidden layers, i.e. at the single
quantum perceptron level. To search a prime number for $3$ bits (Truth table
listed in Fig. 2 (c)) using a single quantum perceptron we consider the
potential

$\hat{x}=w_{1}\hat{\sigma}_{1}^{z}+w_{2}\hat{\sigma}_{2}^{z}+w_{3}\hat{\sigma}_{3}^{z}-b+w_{\textrm{m}}\hat{\sigma}_{2}^{z}\hat{\sigma}_{3}^{z}.$
(12)

As we demonstrate later, adding the multi-qubit term
$w_{\textrm{m}}\hat{\sigma}_{2}^{z}\hat{\sigma}_{3}^{z}$ is enough to fulfill
the prime numbers searching task. In this respect, one can include additional
multi-qubit terms in the neural potential. However, we use the simplest
example in Eq. (12) to simplify the network and training process.

We note that when the input of a quantum perceptron that presents only two-
qubit interactions are the basis states, this is equivalent to the classical
perceptron as shown in Eqs. (II.1, II.1, 9). For this reason, now we compare
the cost function of a classical perceptron with the cost function of a
quantum perceptron using the potential in Eq. (12) for the specific purpose of
searching prime numbers from $0$ to $7$. As it is shown in Fig. 1 (b) the
quantum perceptron achieves $C<1\%$ at epoch$=667$ (solid-blue with squares),
while the $C$ value saturates to $0.126$ in the classical counterpart (dotted-
red with diamonds). This indicates that our perceptron achieves the searching
task without hidden layers. We have also verified the ability of the quantum
perceptron by minimizing the cost function to an acceptable error for
searching prime numbers for $4$ bits with the potential

$\hat{x}=w_{1}\hat{\sigma}_{1}^{z}+w_{2}\hat{\sigma}_{2}^{z}+w_{3}\hat{\sigma}_{3}^{z}+w_{4}\hat{\sigma}_{4}^{z}-b+w_{\textrm{m}}\hat{\sigma}_{2}^{z}\hat{\sigma}_{3}^{z},$
(13)

and for $5$ bits with

$\displaystyle\hat{x}$ $\displaystyle=$ $\displaystyle
w_{1}\hat{\sigma}_{1}^{z}+w_{2}\hat{\sigma}_{2}^{z}+w_{3}\hat{\sigma}_{3}^{z}+w_{4}\hat{\sigma}_{4}^{z}+w_{5}\hat{\sigma}_{5}^{z}-b$
(14) $\displaystyle+w_{\textrm{m}}\hat{\sigma}_{2}^{z}\hat{\sigma}_{3}^{z}.$

The multi-qubit terms in the above two neural potentials for $4$ and $5$ bits
are chosen due to the fact that the cost function result in $C<1\%$ at epoch
$=1692$ for Eq. (13) and epoch $=1698$ for Eq. (14) with the consideration of
adopting the minimal number of multi-qubit terms. Further training the above
quantum perceptrons presents that the value of $C$ tends to $0$
asymptotically, proving the success to achieve these tasks.

###  II.3 Quantum gates

Now we show that one can use a QNN with multi-qubit interactions to construct
quantum gates such as CNOT gate, Toffoli, and Fredkin without the necessity of
hidden layers. In comparison, one can demonstrate that a QNN with only two-
qubit interactions cannot construct the above mentioned quantum gates in the
same conditions. This will be shown later. As quantum gates are reversible,
the number of input and output qubits of quantum gates should be equal. Hence,
the number of the neurons in the input equals that in the output.
Correspondingly, the cost function is changed into

$\displaystyle
C=\frac{1}{2Nk}\sum_{n=1}^{N}\sum_{i=1}^{k}(y_{i}^{(n)}-t_{i}^{(n)})^{2},$
(15)

where $k$ is the number of quantum perceptrons.

For CNOT gate where $k=2$, the truth table and the schematic configuration of
the QNN is shown in Fig. 3 (a) and (b) respectively, where each perceptron
(output neuron) gives the output value
$y_{i}=\frac{1}{2}(1+\langle\hat{\sigma}_{\rm{out,i}}^{z}\rangle)$. The first
perceptron should have the same value as the first input neuron, while the
second one aims to achieve the same value as the XOR output of two input
neurons, i.e.
$\hat{\sigma}^{z}_{\rm{out,2}}=\hat{\sigma}_{1}^{z}\oplus\hat{\sigma}_{2}^{z}$
quantum-computation-book . The neural potential of the second perceptron is

$\displaystyle\hat{x}_{2}=w_{21}\hat{\sigma}_{1}^{z}+w_{22}\hat{\sigma}_{2}^{z}-b+w_{\rm{m}}\hat{\sigma}_{1}^{z}\hat{\sigma}_{2}^{z},$
(16)

and the first perceptron has two-qubit interaction terms. We find that this is
the most concise choice, as it requires least connectivity. We obtain the cost
function value (using Eq. (15) $C<1\%$ at epoch $=116$ and $C\rightarrow 0$ at
a larger epoch, as shown in Fig. 4 (a) (solid-blue with squares) indicating
the satisfaction to train a CNOT gate well. On the contrary, training such a
CNOT gate by a QNN with only two-qubit interaction terms leads to a $C$
tending to $0.0625$ at large epoch, see Fig. 4 (b). This indicates that a
standard QNN cannot encode the CNOT without hidden layers.

Figure 3:  Truth table of a CNOT gate (a) constructed by a QNN illustrated in
a schematic configuration (b) with the multi-qubit interaction term
$w_{\rm{m}}\hat{\sigma}_{1}^{z}\sigma_{2}^{z}$.

Figure 4:  Cost function at different epoch for (a) QNN with the multi-qubit
term and (b) QNN with ony two-qubit interactions. In particular the solid-blue
line is the cost function obtained to construct a CNOT gate, a Toffoli gate
(dashed-red line) and a Fredkin gate (dotted-black line). Illustrated in the
inset of (a), the cost function values reach $C<1\%$ at $116$th, $140$th,
$162$th epoch for the above three quantum gates, respectively, indicating that
these gates can be constructed by QNNs with multi-qubit interaction without
hidden layers. In contrast, $C$ values go to their respective plateau by
quantum perceptrons with two-qubit interactions.

A QNN with multi-qubit terms also works for constructing a Toffoli gate
without hidden layers. Truth table, and QNN for Toffoli are shown in Fig. 5
(a) and (b). Known as Controlled-Controlled-Not gate, a Toffoli gate has
3-qubit inputs and outputs. In the QNN, the outputs of the first two qubits
should be the same value as their inputs
$\hat{\sigma}_{\rm{out,i}}^{z}=\hat{\sigma}^{z}_{i}$ ($i=1,2$), while the
output of the third qubit aims at
$\hat{\sigma}_{\rm{out,3}}^{z}=\hat{\sigma}_{3}^{z}\oplus\hat{\sigma}^{z}_{1}\hat{\sigma}^{z}_{2}$
quantum-computation-book . In this case, we propose the following neural
potential of the third perceptron

$\displaystyle\hat{x}_{3}=w_{31}\hat{\sigma}_{1}^{z}+w_{32}\hat{\sigma}_{2}^{z}+w_{33}\hat{\sigma}_{3}^{z}-b+w_{\textrm{m}}\hat{\sigma}_{1}^{z}\hat{\sigma}_{2}^{z}\hat{\sigma}_{3}^{z},$
(17)

with a minimal number of multi-qubit terms. Demonstrated in Fig. 4 (a) and
(b), the value $C<1\%$ (dashed-red line with crosses) occurs at epoch $=140$
and goes to $0$ at large epoch during the training of a Toffoli gate. In close
similarity with the previous case, a standard QNN without hidden layers fails
to achieve the Toffoli gate, as $C$ saturates at $0.0238$.

Figure 5:  Truth table of a Toffoli gate (a) constructed by a QNN illustrated
in a schematic configuration (b) with the multi-qubit interaction term
$w_{\rm{m}}\hat{\sigma}_{1}^{z}\hat{\sigma}_{2}^{z}\hat{\sigma}_{3}^{z}$.

Another example that can be successfully constructed by our QNNs including
multi-qubit interactions without hidden layers is the Fredkin gate (also known
as Controlled-SWAP gate) whose truth table and schematic configuration are in
Fig. 6 (a) and (b). In this case, the adopted potential of the third
perceptron is

$\displaystyle\hat{x}_{3}=w_{31}\hat{\sigma}_{1}^{z}+w_{32}\hat{\sigma}_{2}^{z}+w_{33}\hat{\sigma}_{3}^{z}-b+w_{\textrm{m}}\hat{\sigma}_{2}^{z}\hat{\sigma}_{3}^{z},$
(18)

The cost function value $C$ tends to zero asymptotically, shown in Fig. 4 (a)
(dotted-black line with asterisks), indicating that these gates can be
constructed by quantum perceptrons without hidden layers. Again, $C$ saturates
at the value $0.0417$ at large epoch for a standard QNN without hidden layers,
see Fig. 4 (b).

##  III Discussion

We have shown different application of QNNs which possess multi-qubit
interactions. These terms induce connectivities among quantum perceptrons that
deviate from the current network paradigm of additive activations. It is due
to the multi-qubit terms in the potentials that one can avoid the presence of
internal hidden layers without sacrificing approximative power. Meanwhile, the
performance is enhanced compared to the same topology of a QNN without multi-
qubit interactions. Such architecture allows us to address the connectivity
challenge in scaling up QNNs. Moreover, the simple configuration helps to
control the efficiency of the training processes. During the training process
of all the examples shown above, the activation function (Eq. 4) based on the
adiabatic evolution of the system is used. Instead, one may use shortcuts to
adiabaticity to accelerate the formation of the activation function in
physical registers Ban20 .

Figure 6:  Truth table of a Fredkin gate (a) constructed by a QNN illustrated
in a schematic configuration (b) with the multi-qubit interaction term
$w_{\rm{m}}\hat{\sigma}_{2}^{z}\sigma_{3}^{z}$.

##  IV Conclusion

We propose a QNN with multi-qubit interactions included in the potential. We
shown that this QNN can perform distinct tasks without hidden layers. Among
them we can list XOR gate encoding, searching prime number (in our case from
$3$ to $5$ bits) as well as to construct quantum gates such as the CNOT,
Toffoli, and Fredkin gates at the perceptron level. The reduction in the
topologies of the networks not only simplifies the training process but also
improve the accuracy of the approximation. The usage of less quantum
perceptrons in the network can increase the process fidelity due to less
decoherence effects when the QNN is encoded in physical registers.

## acknowledgements

We acknowledge financial support from Spanish Government via
PGC2018-095113-B-I00 (MCIU/AEI/FEDER, UE), Basque Government via IT986-16, as
well as from QMiCS (820505) and OpenSuperQ (820363) of the EU Flagship on
Quantum Technologies, and the EU FET Open Grant Quromorphic (828826). C.M.-J.
acknowledges the predoctoral MICINN grant PRE2019-088519. E. T. acknowledges
financial support from Project PGC2018-094792-B-I00 (MCIU/AEI/FEDER,UE), CSIC
Research Platform PTI-001, CAM/FEDER Project No. S2018/TCS-4342 (QUITEMAD-CM),
and by Comunidad de Madrid-EPUC3M14. J. C. acknowledges the Ramón y Cajal
program (RYC2018-025197-I) and the EUR2020-112117 project of the Spanish
MICINN, as well as support from the UPV/EHU through the grant EHUrOPE.

##  Appendix A Training for prime numbers search

To train a QNN for the task of searching prime numbers, a set of $2^{i}$ pairs
for $i$ bits containing the input and output values
$\\{A^{(n)},y^{(n)}\\}^{2^{i}}_{n=1}$ is taken, where the inputs are binary
numbers $A^{(n)}=(a_{1},a_{2},...,a_{i})$ corresponding to the integers
belonging to the set $\in\\{0,1,...,2^{i}-1\\}$. As the targets are
$t\in\\{0,1\\}$, the output of the network $y^{n}=Q(A^{(n)})=1$, if and only
if the input number is prime.

## References

  * (1) M. Castells, The Information Age: Economy, Society and Culture, Blackwell, Oxford, 1996.
  * (2) C. Walter, Scientific American 293, 32 (2005).
  * (3) A. Kolmogorov, Sankhya Ser. A. 25, 369 (1963).
  * (4) A. L. Samuel, IBM Journal of Research and Development 3, 210 (1959).
  * (5) R. P. Feynman, Int. J. Theor. Phys. 21, 467 (1982).
  * (6) D. Hebb, The Organization of Behavior, Wiley, New York, 1949.
  * (7) W. S. McCulloch and W. Pitts W, Bull. Math. Biol. 5, 115 (1943).
  * (8) K.-Su Oh and K. Jung, Pattern Recognition 37, 1311 (2004).
  * (9) Y. LeCun, Y. Bengio, and. G. Hinton, Nature 521, 436 (2015).
  * (10) S. C. Kleene, Representation of Events in Nerve Nets and Finite Automata, Annals of Mathematics Studies 34, Princeton University Press (1956).
  * (11) F. Rosenblatt, The perceptron - A perceiving and recognizing automaton, Tech. Rep. 85-460-1, Cornell Aeronautical Laboraroty, (1957).
  * (12) J. J. Hopfield, Proc. Nat. Acad. Sci. 81, 3088 (1984).
  * (13) G. Cybenko, Math. Control Signals Syst. 2, 303 (1989).
  * (14) G. E. Dahl, D. Yu, L. Deng, and A. Acero, IEEE Transactions on Audio, Speech and Language Processing, 20, 33 (2012).
  * (15) G. E. Hinton, S. Osindero, and Y. Teh, Neural Comput. 18, 1527 (2006).
  * (16) E. G. Dada, J. S. Bassi, H. Chiroma, S. M.Abdulhamid, A. O. Adetunmbi, and O. E. Ajibuwa, Heliyon 5, e01802 (2019).
  * (17) M. Buehler, K. Iagnemma, and S. Singh, The DARPA urbanchallenge: autonomous vehicles in city traffic. Springer, vol. 56, 2009.
  * (18) S. Devi, P. Malarvezhi, R. Dayana and K. Vadivukkarasi, Wirel. Pers. Commun. 114, 2121 (2020).
  * (19) A. Valsamis and K. Tserpes, D. Zissis and D. Anagnostopoulos, and T. Varvarigou, J. Syst. Softw., 127 249, (2017).
  * (20) P. Kashyap, Machine Learning for Decision Makers: Cognitive Computing Fundamentals for Better Decision Making, APress, U. S. (2018).
  * (21) J. Fürnkranz, Machine Learning and Game Playing, Encyclopedia of Machine Learning, Springer US, 2010.
  * (22) B. Huang, Y. Huan, L. D. Xu, L. Zheng, and Z. Zou, Enterp. Inf. Syst. 13, 132 (2019).
  * (23) S. C. Kleene, Duke Math. J. 2, 340 (1936).
  * (24) F. Arute, K. Arya, R. Babbush, et al., Nature 574, 505 (2019).
  * (25) A. Y. Kitaev, Russ. Math. Surv. 52, 1191 (1997).
  * (26) A. W. Harrow, A. Hassidim, and S. Lloyd, Phys. Rev. Lett. 103, 150502 (2009).
  * (27) A. Peruzzo, J. McClean, P. Shadbolt, M. Yung, X. Zhou, P. J. Love, A. Aspuru-Guzik, and J. L. O’Brien, Nat. commun. 5, 4213 (2014).
  * (28) E. Farhi, J. Goldstone, and S. Gutmann, arXiv:1411.4028 (2014).
  * (29) L. K. Grover, A fast quantum mechanical algorithm for database search, Proceedings, 28th Annual ACM Symposium on the Theory of Computing, 1996, p. 212.
  * (30) P. W. Shor, Algorithms for quantum computation: discrete logarithms and factoring, Proceedings 35th Annual Symposium on Foundations of Computer Science, 1994, p. 124.
  * (31) S. Lloyd, Universal quantum simulators, Science 273, 1073 (1996).
  * (32) M.A. Nielsen and I. Chuang, Quantum Computation and Quantum Information, Cambridge University Press, New York, U. S. (2011).
  * (33) Mikko Möttönen, J. J. Vartiainen, V. Bergholm, and M. M. Salomaa, Phys. Rev. Lett. 93, 130502 (2004).
  * (34) M. Reagor, C. B. Osborn, N. Tezak, et al., Sci. Adv. 8, eaao3603 (2018).
  * (35) T. Bækkegaard, L. B. Kristensen, N. J. S. Loft, C. K. Andersen, D. Petrosyan, and N. T. Zinner, Sci. Rep. 9, 13389 (2019).
  * (36) S. C. Kak, Adv. Imag. Elect. Phys. 94, 259 (1995).
  * (37) M. Schuld, I. Sinaysky, and F. Petruccione, Quan. Info. Proc. 13, 2567 (2014).
  * (38) J. Biamonte, P. Wittek, N. Pancotti, P. Rebentrost, N. Wiebe, and S. Lloyd, Seth, Nature 549, 195 (2017).
  * (39) E. Farhi and H. Neven, arXiv:1802.06002 (2018).
  * (40) M. Schuld and N. Killoran, Phys. Rev. Lett. 122 040504 (2019).
  * (41) M. Schuld, A. Bocharov, K. Svore, and N. Wiebe, Phys. Rev. A 101, 032308 (2020).
  * (42) Y. Cao, G. Guerreschi, A. Aspuru-Guzik, arXiv: 1711.11240 (2017).
  * (43) A. Pérez-Salinas, A. Cervera-Lierta, E. Gil-Fuster, and J. I. Latorre, Quantum 4, 226 (2020).
  * (44) E. Torrontegui and J. J. García-Ripoll, Europhys. Letts. 125, 30004 (2019).
  * (45) Y. Ban, X. Chen, E. Torrontegui, E. Solano and J. Casanova, Sci. Rep. 11, 5783 (2021).
  * (46) S. Gupta and R. K. P. Zia, J. Comput. Syst. Sci. 63, 355 (2001).
  * (47) G. D. Paparo, V. Dunjko, A. Makmal, M. A. Martin-Delgado, and H. J Briegel, Phys. Rev. X 4, 031002 (2014).
  * (48) M. Benedetti, J. Realpe-Gómez, R. Biswas, and A. Perdomo-Ortiz, Alejandro, Phys. Rev. X 7, 041052 (2017).
  * (49) G. Sentís, M. Guta, and G. Adesso, EPJ Quantum Technol. 2, 17 (2015).
  * (50) V. Havlícek, A. D. Corcóles, K. Temme, A. W. Harrow, A. Kandala, J. M. Chow, and J. M. Gambetta, Nature 567, 209 (2019).
  * (51) J. Carrasquilla and R. G. Melko, Nat. Phys. 13, 431 (2017).
  * (52) D.- L. Deng, X. Li, and S. Das Sarma, Phys. Rev. X 7, 021021 (2017).
  * (53) G. Torlai, G. Mazzola, J. Carrasquilla, M. Troyer, R. Melko, and G. Carleo, Nat. Phys. 14, 447 (2018).
  * (54) N. Aharon, A. Rotem, L. P. McGuinness, F. Jelezko, A. Retzker, and Z. Ringel, Sci. Rep. 9, 17802 (2019).
  * (55) Y. Ban, J. Echanobe, Y. Ding, R. Puebla, J. Casanova, arXiv: 2012.07677 (2020).
  * (56) I. S. Donskaya, Theor. Math. Phys. 74, 324 (1988).
  * (57) S. Kumar. H. Zhang, and Y. P. Huang, Commun. Phys. 3, 108 (2020).
  * (58) M. Sameti, A. Potočnik, D. E. Browne, A. Wallraff, and M. J. Hartmann, Phys. Rev. A 95, 042330 (2017).
  * (59) B. P. Lanyon, C. Hempel, D. Nigg, M. Müller, R. Gerritsma, F. Zähringer, P. Schindler, J. T. Barreiro, M. Rambach, G. Kirchmair, M. Hennrich, P. Zoller, R. Blatt, and C. F. Roos, Science 334, 57 (2011).
  * (60) M. Minsky and S. A. Papert, Perceptrons: An Introduction to Computational Geometry, Cambridge, MA, USA: MIT Press, 2017.
  * (61) M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantum Information, Cambridge university press, 2010.


# Widest scales in turbulent channels

R. Pozuelo  FLOW, Engineering Mechanics, KTH Royal Institute of Technology,
SE-100 44 Stockholm, Sweden  A. Cavalieri  Divisão de Engenharia Aeronáutica,
Instituto Tecnológico de Aeronáutica (ITA), 12228-900, São José dos Campos,
SP, Brazil  P. Schlatter  Institute of Fluid Mechanics (LSTM), Friedrich-
Alexander-Universität (FAU) Erlangen-Nürnberg, Germany  R. Vinuesa  FLOW,
Engineering Mechanics, KTH Royal Institute of Technology, SE-100 44 Stockholm,
Sweden

###### Abstract

The widest spanwise scales in turbulent channel flows are studied through the
use of three periodic channel-flow simulations at friction Reynolds number
$\mathrm{Re}_{\tau}=550$. The length and height of the channels are the same
in all cases ($L_{x}/h=8\pi$ and $L_{y}/h=2$ respectively), while the width is
progressively doubled: $L_{z}/h=\\{4\pi,8\pi,16\pi\\}$. The effects of
increasing the domain can not be determined with statistical significance in
our simulations, since the difference in the statistics between the
simulations is of the same order as the errors of convergence. A channel flow
similar to the smaller one (J. Fluid Mech., vol. 500, 2004, pp. 135–144),
which was averaged over a very long time, was used as a reference. The one-
dimensional spanwise spectrum of the streamwise velocity is computed with the
aim of assessing the domain-size effect on the widest scales. Our results
indicate that $90\%$ of the total streamwise energetic fluctuations is
recovered without a significant influence of the size of the domain. The
remaining $10\%$ of the energy reflects that the widest scales in the outer
layer are the ones most significantly affected by the spanwise length of the
domain. The power-spectral density for $k_{z}=0$ remains constant even if the
size of the domain in the spanwise direction is increased up to 4 times the
standard spanwise length, indicating that wide, spanwise coherent structures
are not an artifact of domain truncation.

##  I Introduction

In numerical simulations of turbulent flows, the size of the domain imposes a
limit to the size of the scales that can be simulated in the domain, and the
resolution is responsible for properly resolving the various simulated scales.
In the early days of direct numerical simulations (DNSs) of turbulent channel
flows, Kim, Moin, and Moser [9] showed their concern for the smallest scales
that could not be captured by the resolution they were using, and the domain
size was justified by the use of two-point correlations. The use of the one-
dimensional energy spectra in the streamwise and spanwise directions was then
used to justify the grid resolution, since the spectral energy for high
wavenumbers decreased by orders of magnitude. In this early simulation, the
values of the energy spectra exhibited large energy at low wavenumbers (large
scales).

Later, Jiménez [7] focused on the largest scales present in turbulence, where
the criterion for a domain that could fit the largest scales relied on the
function $k\phi$ (which is the power-spectral density $\phi$ premultiplied by
the wavenumber $k$) decaying for large and small wavenumber. Such function
decays, by construction, as $k$ goes towards zero. The largest scales indeed
contain a low spectral energy per unit of volume, but their big size can make
their total energetic contribution to be a substantial fraction of the
turbulence statistics. Their long lifetimes can also accumulate over time and
they have important implications on the flow from an acoustic point of view
[15]. More studies on the effect of the size of the domain were conducted by
Jiménez and Moin [8], who defined the minimal flow unit. In those simulations
the aim was to assess small domains that can capture “healthy” turbulence,
implying that the turbulence statistics for a certain region close to the wall
are correct. In Jiménez and Moin [8] the focus was on the minimal dimensions
for the viscous and buffer layers, while in Flores and Jiménez [5] the study
was extended to the logarithmic region. Those studies assessed the effects of
small channels with different aspect ratios and those results were compared
with highly-resolved turbulent channel flows at a similar friction Reynolds
numbers, i.e. $\mathrm{Re}_{\tau}=180$ from Kim, Moin, and Moser [9] and
$\mathrm{Re}_{\tau}=2000$ from Hoyas and Jiménez [6]. The smallest channels
exhibit lower values of the streamwise root-mean-square (rms) fluctuations
$u_{\mathrm{rms}}$ and larger values for the wall-normal component
$v_{\mathrm{rms}}$.

Focusing on the largest scales in turbulent channel flows, where the
premultiplied streamwise spectral density did not exhibit closed contours for
the standard domain sizes being simulated, Lozano-Durán and Jiménez [11]
performed a simulation of a very large channel at $\mathrm{Re}_{\tau}=550$
with streamwise and spanwise dimensions $(L_{x},L_{z})=(60\pi h,6\pi h)$,
where $h$ is the half-channel height. They showed that their one-point
statistics were barely affected by increasing the width of the domain beyond
$\pi h$ and their length beyond $2\pi h$ for $\mathrm{Re}_{\tau}$ up to 2000.
The effects observed in the premultiplied two-dimensional spectra at the
center of the channel showed that shorter domains exhibit trends similar to
those for larger domains. They were able to close the contour corresponding to
$10\%$ of the maximum, which they report contains $80\%$ of the streamwise
turbulent kinetic energy. Note that they did observe some accumulation of
energy in the probability density functions of the coherent structures studied
in Lozano-Durán, Flores, and Jiménez [12]. For shorter domains, the largest
coherent scales exhibited larger energy levels in comparison with the larger
domains. They argued that the largest resolved scales and those that were too
large to be simulated in the domain, which was seen as infinite in size due to
the periodicity, did not alter the dynamics of the rest of the simulated
scales. Motivated by the unexplored spanwise large scales and the studies that
related these scales with the noise generation at the trailing edge of a NACA
profile, Abreu et al. [1] studied these spanwise coherent structures. They
observed that the power-spectral density in zero-pressure-gradient (ZPG)
turbulent-boundary-layer (TBL) simulations collapsed for spanwise wavenumbers
$k_{z}\rightarrow 0$, when the domain was increased in the spanwise periodic
direction. The use of SPOD showed that the same hydrodynamic spanwise
structures appeared in NACA profiles, and argued that these waves are not an
artifact of the periodic conditions or the constrains in the spanwise
direction of the simulated domain.

The purpose of the present work is to quantify the effects of the widest
structures in turbulent channel flows. We start from a highly-resolved case
similar to the one used in Del Álamo et al. [4] for $\mathrm{Re}_{\tau}=550$,
and simulate two more channels at the same $\mathrm{Re}_{\tau}$, with the same
streamwise length and similar sampling conditions, with the only difference
being the width of the domain, which is doubled from one channel to another.

##  II Numerical setup

The parameters of the simulations used in this work are collected in Table 1,
where the physical resolution of the domain is maintained in all dimensions,
with the only difference being the spanwise size of the domain $L_{z}$ which
is doubled subsequently for channels C2 and C3. The resolution is reported in
viscous units, which are denoted by the superscript ‘+’, where the fluid
kinematic viscosity $\nu$ and the friction velocity
$u_{\tau}=\sqrt{\tau_{w}/\rho}$ ($\tau_{w}$ is the wall-shear stress and
$\rho$ the fluid density) are employed. The evolution equations for the
incompressible flow are integrated using the formulation based on the wall-
normal vorticity and the Laplacian of the wall-normal velocity for the current
simulations performed with SIMSON [2, 10]. The domain is discretized using
Fourier modes in the periodic streamwise $x$ and spanwise $z$ directions.
Chebychev polynomials are used in the wall-normal direction $y$. For the
channel case “torr”, the employed time integration was a third-order semi-
implicit Runge–Kutta scheme, while the one used for channels C1, C2 and C3 was
a four-stage Runge–Kutta scheme for the advective term, and a second-order
Crank–Nicolson scheme for the viscous terms for all the simulations. The time-
step of the simulation, as well as the sampling times for the statistics and
the velocity fields used to calculate the power spectrum of the velocity
fluctuations, are shown in Tab. 2. Note that the smaller channels torr and C1
have the same dimensions and resolution, and the time span of the torr case is
slightly larger than C1. We expect to obtain similar one-point statistics in
all these simulations. Since we will analyze the difference in these
quantities due to different size of the domains, we have also included the
simulation “torr” by Del Álamo and Jiménez [3] to illustrate how big can the
differences be for the same domain size.

Case | $\mathrm{Re}_{\tau}$ | $L_{x}/h$ | $L_{y}/h$ | $L_{z}/h$ | $m_{x}$ | $m_{y}$ | $m_{z}$ | $\Delta x^{+}$ | $\Delta y_{\rm wall}^{+}$ | $\Delta z^{+}$ | Colour
---|---|---|---|---|---|---|---|---|---|---|---
torr | 547 | 8$\pi$ | 2 | 4$\pi$ | 1536 | 257 | 1536 | 8.9 | 0.041 | 4.5 |
C1 | 545 | 8$\pi$ | 2 | 4$\pi$ | 1536 | 257 | 1536 | 8.92 | 0.041 | 4.46 |
C2 | 545 | 8$\pi$ | 2 | 8$\pi$ | 1536 | 257 | 3072 | 8.92 | 0.041 | 4.46 |
C3 | 545 | 8$\pi$ | 2 | 16$\pi$ | 1536 | 257 | 6144 | 8.92 | 0.041 | 4.46 |
Table 1: Parameters of the simulations used in this paper. The reference length $h$ corresponds to the channel half-height. The box size is $(L_{x}/h,L_{y}/h,L_{z}/h)$ and $(m_{x},m_{y},m_{z})$ are the number of collocation points including the $3/2$ factor for de-aliasing in the Fourier directions ($x$ and $z$). The spatial resolution in viscous units of the streamwise and spanwise directions $\Delta x^{+},\Delta z^{+}$ has been calculated using $(m_{x},m_{z})$. The largest grid spacing in the wall-normal direction is located at the center of the channel, and it is equal to $\Delta y_{\rm max}/h=0.0123$ or $\Delta y_{\rm max}^{+}=6.69$.  Name | $\Delta t_{\rm sim}$ | $\Delta t_{\rm stats}$ | $\Delta t_{\rm spec}$ | $\Delta t_{\rm sim}^{+}$ | $\Delta t_{\rm stats}^{+}$ | $\Delta t_{\rm spec}^{+}$ | Samples spec | $\Delta T$ | $\Delta T/(h/u_{\tau})$ | $\Delta TU_{b}/L_{x}$
---|---|---|---|---|---|---|---|---|---|---
torr | – | – | – | – | – | – | – | – | 12.3 | 10
C1 | 0.004 | 0.064 | 0.64 | 0.079 | 1.26 | 12.6 | 469 | 300 | 10.9 | 7.96
C2 | 0.004 | 0.064 | 0.64 | 0.079 | 1.26 | 12.6 | 469 | 300 | 10.9 | 7.96
C3 | 0.004 | 0.064 | 0.64 | 0.079 | 1.26 | 12.6 | 469 | 300 | 10.9 | 7.96
Table 2: Time steps for the simulation (sim), statistics sampling (stats) and
spectrum sampling (spec) of the new simulations, where $\Delta T$ is the total
simulated time. Viscous units are denoted with ‘+’, $h$ is the channel half-
height, $u_{\tau}$ is the friction and $U_{b}$ is the bulk velocity.

##  III Turbulence statistics

To assess the effects of the size of the domain we first evaluate its effect
on the overall turbulence statistics. The turbulence statistics for the
channel flows are obtained through the averaging over time and the homogeneous
wall-parallel directions $x$ and $z$ as well as in the wall-normal direction
using the property of symmetry or anti-symmetry of the measured quantity
around the center of the channel. Note that torr has been averaged over a
longer time period than the rest of the simulations, and C1, C2, C3 have been
averaged over the same time span. Since C2 has double the size of C1, it will
contain double of samples than C1 for the same averaged time. The same can be
applied to channel C3, which contains a larger number of samples than the
other channels. In Fig. 1 we examine the effects of the width of the domain on
the streamwise mean velocity. Fig. 1(a) shows the inner-scaled streamwise-
velocity profile, and to illustrate the differences between the various
simulations we show in Fig. 1(b) the absolute difference of each cases with
respect to channel C3. The relative deviation of the mean streamwise velocity
is represented in Fig. 1(c), showing a value of the order of $0.1\%$.

Figure 1:  (a) Mean streamwise velocity $U$ for channels C1 (blue), C2 (red),
C3 (yellow) and torr (black) scaled in viscous units. (b) Absolute deviation
of $U$ with respect to the values of channel C3. (c) Relative deviation of $U$
with respect to the values of channel C3. The data for the simulation torr was
extracted from Del Álamo and Jiménez [3].

Figure 2:  (a) Streamwise fluctuation profile $u^{+}_{\mathrm{rms}}$ for
channels C1 (blue), C2 (red), C3 (yellow) and torr (black) scaled in viscous
units. (b) Absolute deviation of $u_{\mathrm{rms}}$ with respect to the values
of channel C3. (c) Relative deviation of $u_{\mathrm{rms}}$ with respect to
the values of channel C3. The data for the simulation torr was extracted from
Del Álamo and Jiménez [3].

Figure 3:  (a) Spanwise fluctuation profile $w^{+}_{\mathrm{rms}}$ for
channels C1 (blue), C2 (red), C3 (yellow) and torr (black) scaled in viscous
units. (b) Absolute deviation of $w_{\mathrm{rms}}$ with respect to the values
of channel C3. (c) Relative deviation of $w_{\mathrm{rms}}$ with respect to
the values of channel C3. The data for the simulation torr was extracted from
Del Álamo and Jiménez [3].

As discussed by Townsend [16], the presence of the wall limits the wall-normal
perturbations, and the effects on the Reynolds stress components such as
$v^{2}$ or $uv$ should be mainly observed in the outer region. In Del Álamo
and Jiménez [3] they report that the longest fluctuating scales appear for the
streamwise velocity at $y=0.5h$ while the widest ones appearing for the
spanwise velocity are at the center of the channel. For this reason, we will
analyze the effect of the width of the domain in these components. In Figs. 2
and 3 we show the rms of the streamwise and spanwise velocity fluctuations,
together with the deviation and relative deviation with respect to C3. The
relative deviations, in this case, are of the order of $1\%$, with a larger
difference in the outer region. Based on these results and the order of
magnitude of the errors, from a point of view of the turbulence statistics,
the channels wider than C1 for $\mathrm{Re}_{\tau}=550$ do not introduce any
significant difference.

##  IV Spectrum of the widest turbulent scales

To assess whether wider domains constrain the widest scales, we use the one-
dimensional spectrum of the fluctuation velocities in the spanwise direction.
In the turbulence statistics, the averaging in the spanwise direction implies
that C3 has in principle better convergence than channels C1 and C2. However,
the average of the spanwise spectrum is performed only in time, the streamwise
direction and the symmetry around the center of the channel. Since the time
sampling and the streamwise size of the domain are the same for simulations
C1, C2 and C3, the energy contained in the different spanwise scales are
averaged over the same number of samples, and the convergence should be
similar.

The power spectrum is defined as
$\mathrm{PS}=\langle\hat{u}(k_{z},y)\hat{u}^{*}(k_{z},y)\rangle_{(x,t,Sy)}$,
where $\hat{u}$ indicates the Fourier coefficient of $u$ and $\hat{u}^{*}$ the
complex conjugate, while the power-spectral density is
$\phi(k_{z},y)=\mathrm{PS}/\mathrm{d}k_{z}$. Here $\langle\cdot\rangle$
denotes averaging over the quantities shown in the subscript and ‘Sy’
indicates the symmetry applied in the wall-normal direction. The widest scale
captured by Fourier analysis corresponds to $k_{z}=(2\pi)/L_{z}$ or
$\lambda_{z}=L_{z}$, and the energy contained in $k_{z}=0$ corresponds to the
energy of spanwise constant fluctuations.

---
---

Figure 4: (a) Inner-scaled premultiplied power-spectral density
$k_{z}\phi_{uu}/u_{\tau}^{2}$ as a function of the spanwise wavelength
$\lambda_{z}^{+}$ for C1 (top), C2 (middle), C3 (bottom). (b) Inner-scaled
power spectral density $\phi_{uu}/u_{\tau}^{2}$ as a function of the spanwise
wavenumber $k_{z}$; contours taken at $\phi_{uu}^{+}=[0.05,0.1,0.2]$, which
correspond to $15\%$, $31\%$ and $62\%$ of the maximum. (c) Same as (b) but
represented as a function of $\lambda_{z}$. Colors shown in Table 1.

In the spectral analysis of turbulence, the spectrum is usually shown
premultiplied by the wavenumber. In Fig. 4(a) we show the premultiplied
spectrum of the streamwise velocity for the three channel flows C1 (top), C2
(middle) and C3 (bottom), where it is possible to observe similar
characteristics and values of the energy, with the only differences being the
size of the scales that are captured and the spectral resolution being finer
in the wider channels. In Fig. 4(b) we represent the contours of $\phi_{uu}$,
without premultiplication by the wavenumber, as a function of the wavenumber,
and the levels $31\%$ and $15\%$ reach the wavenumber $k_{z}=0$, a fact that
is manifested in Fig. 4(c) as contours which are not closed since $k_{z}=0$
corresponds to $\lambda_{z}=\infty$, or scales wider than the domain.

In the spectra discussed above, the integral of $\phi$ over all the
wavenumbers (including the energy contained in the wavenumber $k_{z}=0$),
equals the corresponding Reynolds-stress component. The premultiplication by
$k_{z}$ automatically cancels that energy, making it inappropriate to use
premultiplied spectra for integral analysis. The question is whether it is
possible to have a domain $L_{z}$ where the largest Fourier scales
$\lambda_{z}=\\{L_{z},L_{z}/2,...\\}$ do not contain spectral energy, thus
leading to closed contours. It is important to analyze the evolution of the
spectral density of energy contained in $k_{z}=0$ with $L_{z}$, and even more
relevant from the physical point of view is how much is the density of energy
contained in the first wavenumbers or the largest scales
$\lambda_{z}=\\{L_{z},L_{z}/2,L_{z}/3,...\\}$, etc.

When comparing the spectra in channels of different widths, the wider the
domain, the finer the wavenumber resolution. The smallest channels torr and C1
share a width of $L_{z}=4\pi h$, which we will use as a parameter
$L_{z,0}=L_{z,C1}=4\pi h$. In channels torr and C1, the first wavelengths are
$k_{z}=(2\pi/L_{z,0})\\{0,1,2,i,...\\}$, while for channel C2 they are
$k_{z}=(2\pi/L_{z,0})\\{0,1/2,1,3/2,2,i/2,...\\}$ and for C3:
$k_{z}=(2\pi/L_{z,0})\\{0,1/4,1/2,3/4,1,i/4,...\\}$. We see how channels C1
and torr have the same wavenumber resolution, while C2 and C3 have
respectively 2 and 4 times the wavenumber resolution of C1. We can compare the
spectral density of energy obtained for scales of a similar width (same
wavenumber/wavelength) for all the channels, and it will give an idea of how
the spectral energy density changes due to numerical and time-average factors
(comparing C1 and torr), or due to the increasing size of the domain,
comparing C1 with C2 and C3.

Figure 5: (a) Relative differences in power-spectral density with respect to
that of torr channel $(\phi_{Ci}^{+}-\phi_{\rm torr}^{+})/\phi_{\rm torr}^{+}$
at the same wavenumbers and $y^{+}$. (b) Differences in power-spectral energy
with respect to that of torr channel assuming the same bandwidth
$\mathrm{d}k_{z,\mathrm{torr}}^{+}=\mathrm{d}k_{z,\mathrm{torr}}l_{\tau}$ and
relative to the total energy as: $(\phi_{Ci}^{+}-\phi_{\rm
torr}^{+})\mathrm{d}k_{z,\mathrm{torr}}^{+}/\int\phi_{\rm
torr}^{+}\mathrm{d}k_{z,\mathrm{torr}}^{+}$. The top row compares C1 with
torr, the middle row is for C2 and the row at the bottom compares C3.

In Fig. 5(a) we show the relative difference of $(\phi^{+}_{Ci}-\phi^{+}_{\rm
ref})/\phi^{+}_{Ci}$ where the reference $\phi^{+}_{\rm ref}$ has been chosen
as $\phi^{+}_{\rm torr}$. Here,
$\phi^{+}=(\mathrm{PS}/u_{\tau}^{2})(\mathrm{d}k_{z}l_{\tau})$ such that
$\int\phi^{+}\mathrm{d}k_{z}^{+}=\overline{uu}^{+}$. The largest relative
difference is of the order of $40\%$ and is located at $y=h$ for wavenumbers
$k_{z}h=0.5$ between torr and C1, but also between C1 and C2. The largest
differences $\approx 30\%$ are located above $y^{+}=200$ for wavenumbers
$k_{z}h<5$. Although these relative differences are large for the largest
scales, mostly due to an insufficient amount of time to converge the spectra
of these scales, it is also interesting to see how large are these differences
compared to the total energy. In Fig. 5(b) we assume a common wavenumber
bandwidth
$\mathrm{d}k_{z}^{+}=\mathrm{d}k_{z,\mathrm{torr}}^{+}=\mathrm{d}k_{z,C1}^{+}$
and we compare the differences in energy at similar wavenumbers with the total
energy $\overline{uu}^{+}$. In this figure, it is easier to see how the
largest differences are concentrated in the largest scales closer to the
center of the channel, and they can amount up to $2\%$ of the total Reynolds
stress. Notice how the differences alternate in sign for different
wavenumbers; this is also observed if the reference is taken as the spectral
density for C1, C2 or C3. This result shows that the energy of scales that
cannot be captured in narrower domains, is not accumulated in their largest
scales. The same method has been used to compare channels C1 and C2 with C3 as
a reference at matching wavenumbers/wavelengths. A similar pattern has been
observed where the differences in power-spectral density alternate sign for
different Fourier modes. The differences between C1 and C3 do not always share
the same sign as the differences between C2 and C3. For wider scales, we see
larger differences that extend all across the channel height, and for shorter
scales, the difference is smaller in magnitude and spans shorter regions in
the buffer and logarithmic layers. An example of $\phi^{+}$ is shown in Fig. 6
where it can be seen that for $k_{z}=0$ the level of $\phi^{+}$ is similar for
all the simulations and for $k_{z}h=0.5$ the lines for C1 and the wider C3 are
similar. This indicates that the low $k_{z}$ content is not an artifact of the
finite domain size, as the various domains considered here all lead to the
same power-spectral density for the lowest spanwise wavenumbers. The largest
Fourier scales always display non-zero power spectral density, even for wide
channels. For larger wavenumbers such as $k_{z}h=8$ the differences are
smaller between the different channels, and their relative error decreases
exponentially with the wavenumbers.

Figure 6: Power-spectral energy density $\phi_{uu}^{+}$ for the different
channels at wavenumbers (a) $k_{z}h=0$ (solid lines) and at $k_{z}h=0.5$
(dashed lines) for the different channels. (b) Power-spectral energy density
at wavenumbers $k_{z}h=4$ (solid lines) and at $k_{z}h=8$ (dashed lines).

In order to have a better understanding of the contribution of each scale to
the total Reynolds stress we will use the marginal contribution of energy
(MCE) defined in [14, 13] and applied to the streamwise velocity component.
For each wall-normal position, the MCE yields the percentage of
$\overline{u^{2}}$ that is provided by the small scales up to a certain
wavenumber. In Eq. (1), it can be observed that MCE is a function of $y$ and
$k_{z}$.

$\mathrm{MCE}(y,k_{z,c})=\int_{k_{z,c}}^{\infty}\phi_{uu}\mathrm{d}k_{z}\
\bigg{/}\int_{0}^{\infty}\phi_{uu}\mathrm{d}k_{z}.$ (1)

The integral in the numerator accumulates the energy of the smallest scale
with $k_{z}\rightarrow\infty$ up to the cut-off frequency $k_{z,c}=k_{z}$. The
integral at the bottom of the fraction spans all the scales and is equal to
$\overline{u^{2}}$. By representing the iso-contours of the MCE it is possible
to see how relevant is the contribution of large and small scales at different
wall-normal locations. Parallel iso-contours for consecutive levels indicate
that those scales have a similar energetic mechanism. Non-parallel iso-
contours are connected with a region where the scales have a different
energetic contribution. When the same iso-contour is compared for different
simulations, if the curves are parallel it can be stated that a similar
energetic behaviour of those scales is found, and if they are separated there
is an indication that in one simulation the larger scales are more relevant
than in the other one. In our channel simulations, which share the Reynolds
number and the same energy mechanisms, the differences in the iso-contours
provide a visual representation of the regions most significantly affected by
the different widths of the domain, both qualitatively and quantitatively.

In Fig. 7 the MCE contours show that up to $90\%$ of the energy provided by
the smaller scales remains unaltered by the width of these domains, with the
widest streamwise scales of size $\lambda_{z}\approx 2\pi$ being at the center
of the channel. The effects of the width of the domain starts to be seen
mostly in the outer region for the $95\%$ contours and for the $97\%$ contour
the channel C1 already exhibits scales of size $\lambda_{z}/h=4\pi$ at the
center line, while channel C2 exhibits scales with $\lambda_{z}/h=17$ and in
channel C3 there are scales with $\lambda_{z}/h\approx 21$. The viscous
sublayer is slightly affected by the width of the domain, since the largest
scales in the streamwise component can extend their influence down to the
region close to the wall.

Figure 7: (a) Power-spectral-density contours in dotted lines for levels
$\phi_{uu}^{+}=[0.05,0.1,0.2]$ and MCE contours in solid lines for levels
$90\%$, $95\%$ and $97\%$ from left to right, shown for the channels C1, C2
and C3. (b) Detail of the MCE contours. Colors shown in Table 1.

##  V Conclusions

The effects of the widest structures in turbulent channel flows have been
assessed through turbulence statistics and spectral analysis of the
fluctuations. As expected, the smallest channel C1 is wide enough to properly
reproduce the turbulence statistics, exhibiting very small differences with
respect to the widest channel C3 which are of the order of the convergence
error of our simulations. The unclosed contours on the power-spectral density
correspond to the region of low wavenumbers, or largest scales. The power
spectra for $k_{z}=0$ represent the variance of the average in the spanwise
direction and is different from zero in the various cases. The wider the
channel, the smaller is the energy contained in the largest modes, however,
the power-spectral density becomes independent of the spanwise size of the
domain. This result is in agreement with what has been observed in turbulent
boundary layers, and it could indicate the presence of spanwise-coherent
hydrodynamic waves as mentioned in Abreu et al. [1]. The effect of the wider
scales affects the flow all across the channel, and the widest resolved modes
are those more affected by the domain size.

## Acknowledgements

RV acknowledges the financial support provided by the Swedish Research Council
(VR). The computations and data handling were enabled by resources provided by
the National Academic Infrastructure for Supercomputing in Sweden (NAISS),
partially funded by the Swedish Research Council.

## References

  * Abreu et al. [2021] L. I. Abreu, A. Tanarro, A. V. G. Cavalieri, P. Schlatter, R. Vinuesa, A. Hanifi, and D. S. Henningson.  Spanwise-coherent hydrodynamic waves around flat plates and airfoils.  _J. Fluid Mech._ , 927:A1, 2021.  doi: 10.1017/jfm.2021.718.
  * Chevalier et al. [2007] M. Chevalier, P. Schlatter, A. Lundbladh, and D. S. Henningson.  A pseudo-spectral solver for incompressible boundary layer flows.  Technical report, TRITA-MEK. Stockholm, 2007.
  * Del Álamo and Jiménez [2003] J. C. Del Álamo and J. Jiménez.  Spectra of the very large anisotropic scales in turbulent channels.  _Phys. Fluids_ , 15(6):L41–L44, 2003.
  * Del Álamo et al. [2004] J. C. Del Álamo, J. Jiménez, P. Zandonade, and R. D. Moser.  Scaling of the energy spectra of turbulent channels.  _J. Fluid Mech._ , 500:135–144, 2004.  doi: 10.1017/S002211200300733X.
  * Flores and Jiménez [2010] O. Flores and J. Jiménez.  Hierarchy of minimal flow units in the logarithmic layer.  _Phys. Fluids_ , 22(7):071704, 2010.
  * Hoyas and Jiménez [2006] S. Hoyas and J. Jiménez.  Scaling of the velocity fluctuations in turbulent channels up to ${Re}_{\tau}=2003$.  _Phys. Fluids_ , 18(1):011702, 2006.  doi: 10.1063/1.2162185.  URL https://doi.org/10.1063/1.2162185.
  * Jiménez [1998] J. Jiménez.  The largest scales of turbulent wall flows.  _CTR Annual Research Briefs_ , 137:54, 1998.
  * Jiménez and Moin [1991] J. Jiménez and P. Moin.  The minimal flow unit in near-wall turbulence.  _J. Fluid Mech._ , 225:213–240, 1991.  doi: 10.1017/S0022112091002033.
  * Kim et al. [1987] J. Kim, P. Moin, and R. Moser.  Turbulence statistics in fully developed channel flow at low Reynolds number.  _J. Fluid Mech._ , 177:133–166, 1987.
  * Li et al. [2009] Q. Li, P. Schlatter, and D. S. Henningson.  Spectral simulations of wall-bounded flows on massively-parallel computers.  _Simulations of Turbulent Boundary Layers with Heat Transfer_ , page 43, 2009.
  * Lozano-Durán and Jiménez [2014] A. Lozano-Durán and J. Jiménez.  Effect of the computational domain on direct simulations of turbulent channels up to ${Re}_{\tau}=4,200$.  _Phys. Fluids_ , 26(1):011702, 2014.  doi: 10.1063/1.4862918.
  * Lozano-Durán et al. [2012] A. Lozano-Durán, O. Flores, and J. Jiménez.  The three-dimensional structure of momentum transfer in turbulent channels.  _J. Fluid Mech._ , 694:100–130, 2012.  doi: 10.1017/jfm.2011.524.
  * Pozuelo et al. [2022] R. Pozuelo, Q. Li, P. Schlatter, and R. Vinuesa.  An adverse-pressure-gradient turbulent boundary layer with nearly constant $\beta\simeq 1.4$ up to ${Re}_{\theta}\simeq 8700$.  _J. Fluid Mech._ , 939:A34, 2022.  doi: 10.1017/jfm.2022.221.
  * Pozuelo et al. [2023] R. Pozuelo, Q. Li, P. Schlatter, and R. Vinuesa.  Spectra of near-equilibrium adverse-pressure-gradient turbulent boundary layers.  _Phys. Rev. Fluids_ , 8:L022602, Feb 2023.  doi: 10.1103/PhysRevFluids.8.L022602.  URL https://link.aps.org/doi/10.1103/PhysRevFluids.8.L022602.
  * Sano et al. [2019] A. Sano, L. I. Abreu, A. V. G. Cavalieri, and W. R. Wolf.  Trailing-edge noise from the scattering of spanwise-coherent structures.  _Phys. Rev. Fluids_ , 4:094602, Sep 2019.  doi: 10.1103/PhysRevFluids.4.094602.  URL https://link.aps.org/doi/10.1103/PhysRevFluids.4.094602.
  * Townsend [1976] A. A. Townsend.  _The Structure of Turbulent Shear Flow. 2nd Edition_.  Cambridge University Press, 1976.


# Checking Fact Worthiness using Sentence Embeddings

Sidharth Singla<EMAIL_ADDRESS>University of WaterlooWaterlooOntario

###### Abstract.

Checking and confirming factual information in texts and speeches is vital to
determine the veracity and correctness of the factual statements. This work
was previously done by journalists and other manual means but it is a time-
consuming task. With the advancements in Information Retrieval and NLP,
research in the area of Fact-checking is getting attention for automating it.
CLEF-2018 and 2019 organised tasks related to Fact-checking and invited
participants. This project focuses on CLEF-2019 Task-1 Check-Worthiness and
experiments using the latest Sentence-BERT pre-trained embeddings, topic
Modeling and sentiment score are performed. Evaluation metrics such as MAP,
Mean Reciprocal Rank, Mean R-Precision and Mean Precision@N present the
improvement in the results using the techniques.

Fact checking; Check worthiness; Sentence-BERT; Speech

††copyright: acmcopyright††journalyear: ††doi: ††conference: ; ††booktitle:
;††price: ††isbn:

##  1\. Introduction

Fact-checking is a process that seeks to investigate an issue in order to
verify the facts, according to the Oxford dictionary. The spread of
misinformation is inherently human. Important facts can be turned and twisted
both knowingly and unknowingly and can lead to the spread of misinformation in
the masses. Fake News detection is thus becoming a hot topic of research. Thus
fact-checking and fake news detection is very much related to each other.
Political leaders, for their campaigns and rallies, tend to present twisted
facts with incorrect and disguised figures. This particularly caught public
attention during the 2016 US Presidential Campaign which was influenced by
fake news and false claims. It becomes essential to check if the facts being
presented are correct or not. Also, it is an important task to determine the
sentences or phrases out of text and speeches that should be factually-
checked. The manual process is cumbersome as one has to go through long text
and speeches and check each of the sentences if it needs to be checked. Also,
after extracting the phrases to be looked up for fact-checking it is required
to check them in order of the relevance of the phrase. Few phrases will be
more relevant to check for and should first become a priority for validation.
Doing this manually is very time-consuming and there is a lot of possibility
of human error.
With the social media getting stronger and used, a statement, an interview, a
press release, a blog note, or a tweet can spread almost instantaneously. With
the mass of information and little time for double-checking claims against the
facts, it becomes essential to have a quicker process and system for
validation. A number of organisations such as FactCheck and Snopes, among many
others have started with fact-checking initiatives but it has proved to be a
very demanding manual effort and only small number of claims could be fact-
checked.
This motivates researchers and engineers to research and build automatic
intelligent systems that are able to detect the statements that are important
to be fact-checked, rank them in order of priority and then check if the fact
statement is correct or twisted.

CheckThat! Lab at CLEF-2019(CheckThat, 2019) aimed to address these problems
and the participants were asked to build systems that help in automating the
systems. It states that a typical fact-checking pipeline includes the
following steps.

  1. (1)

Identification of check-worthy text fragments.

  2. (2)

Retrieval of documents that might be useful for fact-checking the claim from
various sources with the extraction of supporting evidence.

  3. (3)

Determination if the claim is true or not by comparing a claim against the
retrieved evidence.

The Task 1(Atanasova et al., 2019) is a ranking task. Given a political debate
or a transcribed speech, segmented into sentences with speakers annotated, the
task is to identify which sentence should be prioritized for fact-checking.
The systems are required to produce a score per sentence, according to which
the ranking will be performed and the task runs in English.

Eleven teams participated in Task 1. One of the participant team named TOBB
ETU(Kutlu and Altun, 2019) used linguistic features such as named entities,
topics extracted with IBM Watson’s NLP tools, PoS tags, bigram counts and
trained a multiple additive regression tree(MART).
This project improves their results by using Sentence BERT(Reimers and
Gurevych, 2019) embeddings for the sentences and using sentiment scores and
topic modeling scores of a sentence as features. Data Augmentation was also
used to create more sentence rows for the dataset. All the experiments and
results are presented.

##  2\. Related Work

###  2.1. Fact-checking systems

The very first work done to check-worthiness was the ClaimBuster (Hassan et
al., 2017) system. TF-IDF word representations were used with the features
such as sentiment, part-of-speech (POS) tags and named entities. SVM trained
classifier model was trained on data manually annotated by students,
professors, and journalists, where each sentence was annotated as non-factual,
unimportant factual, or check-worthy factual. The data consisted of
transcripts of 30 historical US election debates covering the period from 1960
until 2012 for a total of 28,029 transcribed sentences.It was later evaluated
against CNN and PolitiFact. Patwari et al.(Patwari et al., 2017) presented a
system called TATHYA based on the similar features, and also included
contextual features sentences immediately preceding and succeeding the one
being assessed, as well as certain hand-crafted POS patterns. Gencheva et
al.(Gencheva et al., 2017) extended ClaimBuster’s feature set by including
more contextual features, such as the sentence’s position in the debate text,
and whether the debate opponent is mentioned. Konstantinovskiy et
al.(Konstantinovskiy et al., 2018) in their work used neural networks for
checking worthiness. They used InferSent(Conneau et al., 2017) for the
universal neural sentence representation. A logistic regression classifier was
then trained. Thorne et al.(Thorne and Vlachos, 2018) presented automatic
fact-checking as a multi-step process that includes: identifying check-worthy
statements; generating questions to be asked about the statements; retrieving
relevant information to create a knowledge base; inferring the veracity of the
statements using text analysis or external sources. Vasileva et al.(Vasileva
et al., 2019) proposed a multi-task learning neural network that learns from
nine fact-checking organizations simultaneously and predicts if a sentence
will be selected for fact-checking by each of these organizations.

###  2.2. CLEF2019 Task1 Submissions

In this subsection, approaches used by the participants for the CLEF-19 Task1
primary subtask is discussed. Team Copenhagen(Hansen et al., 2019) achieved
the best performance. The system learned dual token embeddings: domain-
specific word embeddings and syntactic dependencies, and used them in an LSTM
network. They pre-trained this network with previous Trump and Clinton
debates, and then supervised it weakly with the ClaimBuster system. In their
primary submission, they used a contrastive ranking loss. Team
TheEarthIsFlat(Favano et al., 2019) trained a feed-forward neural network with
two hidden layers, which takes as input Standard Universal Sentence
Encoder(SUSE) embeddings for the current sentence as well as for the two
previous sentences as a context. Team IPIPAN(Gasior and Przybyła, 2019)
extracted features such as bag-of-words n-grams, word2vec vector
representations, named entity types, part-of-speech tags, sentiment scores,
and features from statistical analysis of the sentences. These features were
then used in an L1-regularized logistic regression to predict the check-
worthiness of the sentences. Team Terrier(Su et al., 2019) represented the
sentences using bag-of-words and named entities and used co-reference
resolution to substitute the pronouns by the referring entity/person name.
They also computed entity similarity and entity relatedness and used an SVM
classifier. Team UAICS(Coca et al., 2019) used a Naive Bayes classifier with
bag-of-words features. Team Factify used pre-trained ULMFiT model and fine-
tuned it on the training set. They over-sampled the minority class by
replacing words randomly based on word2vec similarity and also used data
augmentation based on back-translation, where each sentence was translated to
French, Arabic and Japanese and then back to English. Team JUNLP(Dhar et al.,
2019) extracted features such as syntactic n-grams, sentiment polarity, text
subjectivity, and LIX readability score, and used them to train a logistic
regression classifier with high recall. Then, they trained an LSTM model
having GloVe word representations and part-of-speech tags. The sentence
representations from the LSTM model were concatenated with the extracted
features and used for prediction by a fully connected layer, which had high
precision. Finally, average posterior probabilities from both models were used
as the final check-worthiness score. Team nlpir01 extracted TF-IDF word
vectors, TF-IDF PoS vectors, word, character, and PoS tag counts. These
features were then used in a multilayer perceptron regressor with two hidden
layers. Team IIT/ISM Dhanbad trained an LSTM network and fed the network with
word2vec embeddings and features extracted from constituency parse trees as
well as features based on named entities and sentiment analysis. Team ´e
proibido cochilar trained an SVM model on BoW representations of the
sentences, after performing co-reference resolution and removing all digits.
They further used an additional corpus of labelled claims extracted from fact-
checking websites to have a more balanced training corpus and potentially
better generalizations. The method used by team TOBB ETU(Kutlu and Altun,
2019) is discussed in the next subsection.

###  2.3. Approach used by TOBB ETU

The team used a hybrid approach in which claims are ranked using a supervised
method and then reranked based on hand-crafted rules. MART was used as
learning-to-rank algorithm to rank the claims. The features included topical
category of claims, named entities, partof-speech tags, bigrams, and speakers
of the claims. Rules were developed to detect the statements that are not
likely to be a claim, and the statements were put at the very end of the
ranked lists. The primary model ranked 9th based on MAP, and 6th based on R-P,
P@5, and P@20 metrics in the official evaluation of primary submissions. Named
Entities were used as claims about people and institutions are likely to be
check-worthy. They detected whether a statement is about a person or an
institution, and existence of numerical values, location, and date
information. Named entities were identified using Stanford Named Entity
Tagger2 which tags entities such as person, location, organization, money,
date, time and percentage. Part-of-speech Tags of words is helpful in
detecting the amount of information in a sentence. For instance, a sentence
having a noun tag is expected to have information to be fact-checked. POS tags
are detected using Stanford POS toolkit(Toutanova et al., 2003). Topical
Category is an effective indicator for check-worthy claims. For instance, a
claim about celebrities can be considered less check-worthy than a claim about
economics or wars. The topical categories such as finance, law, government,
and politics, where each topic could be branched up to two levels of subtopics
of statements was detected using IBM-Watson’s Natural Language Understanding
Tool. A limited but powerful set of bigrams were used instead of using a
large. Bigrams appearing at least N times only in check-worthy or only in not
check-worthy claims in the training set were considered. N was set to 50.
The contribution of each of the feature was shown by the authors by performing
ablation studies.

###  2.4. Sentence-BERT

Sentence-BERT(SBERT)(Reimers and Gurevych, 2019) is a modification of the
pretrained BERT network and uses siamese and triplet network structures to
derive semantically meaningful sentence embeddings that can be compared using
cosine-similarity. The effort for finding the most similar pair from 65 hours
with BERT / RoBERTa is reduced to about 5 seconds with SBERT, still
maintaining the accuracy from BERT. SBERT and SRoBERTa is evaluated on common
STS tasks and transfer learning tasks, where it outperforms other state-of-
the-art sentence embeddings methods. SBERT is fine-tuned on NLI data, which
creates sentence embeddings that significantly outperform other state-of-the-
art sentence embedding methods like InferSent(Conneau et al., 2017) and
Universal Sentence Encoder on seven Semantic Textual Similarity (STS) tasks.

##  3\. Dataset

The dataset for Task 1 is an extension of the Check That-18 dataset(Atanasova
et al., 2018) and the combined training and test English part of 2018 dataset
is the training data for 2019 Task. For the new test set, labelled data was
produced from three press conferences, six public speeches, six debates, and
one post. The annotations for the new instances were derived from the publicly
available analysis carried out by factcheck.org. The annotation is at the
sentence level. Therefore, if only part of a sentence was fact-checked, the
entire sentence was annotated as a positive instance. If a claim spans more
than one sentence, all the sentences are annotated as positive. The
participating systems were allowed to use external datasets with fact-checking
related annotations as well as to extract information from the Web, from
social media, etc.

Figure 1. Dataset Training example

##  4\. Approach

Table 1.  System Performance after using SBERT Embeddings, Sentiment Features and Topic Modeling Features and combining with TOBB ETU’s system. | MAP | RR | R-P | P@1 | P@3 | P@5 | P@10 | P@20 | P@50
---|---|---|---|---|---|---|---|---|---
TOBB ETU | .0884 | .2028 | .1150 | .0000 | .0952 | .1429 | .1286 | .1357 | .0829
SF only | .0832 | .3017 | .0873 | .1429 | .1905 | .1429 | .1286 | .1000 | .0800
SF + TOBB ETU | .0885 | .1992 | .1218 | .0000 | .1429 | .1143 | .1286 | .1500 | .0829
SBERT only | .1243 | .2207 | .1522 | .1429 | .0952 | .0857 | .1571 | .1643 | .1200
SBERT + TOBB ETU | .1287 | .2318 | .1577 | .1429 | .1429 | .1714 | .1429 | .1786 | .1171
TMF only | .1151 | .3487 | .0979 | .2857 | .1905 | .1714 | .1429 | .1071 | .0800
TMF + TOBB ETU | .1063 | .1930 | .1327 | .0000 | .0952 | .2000 | .1714 | .1429 | .0886
SBERT + TMF + SF + TOBB ETU | .1396 | .2780 | .1348 | .1429 | .1905 | .1714 | .1571 | .1643 | .1171

The TOBB ETU’s performance is improved in this project by using additional
features. The training and testing algorithm of the original system remains
the same and only new features are added for the performance. The original
system uses the MART algorithm with 50 trees and 2 leaves and the same is used
in this approach. One huge advantage of the approach is that training time is
very less and the results are also obtained quickly as there is no neural
model training involved. The system doesn’t require any GPU. Following
features were implemented.

###  4.1. Sentence BERT Embeddings

Each sentence is converted to the Sentence BERT (SBERT) Embeddings for both
training and test files. The model ’bert-base-nli-mean-tokens’ is used for
generating the embeddings. The model generates 768 sized vector embeddings for
a sentence. These embeddings are the best to be used as representations as
they capture the semantic meaning of a sentence.

###  4.2. Sentiment Score

Sentiment Analysis is the process of ‘computationally’ determining whether a
piece of writing is positive, negative or neutral. Sentiment score for each
sentence is calculated using NLTK’s VADER (Valence Aware Dictionary and
sEntiment Reasoner) SentimentIntensityAnalyzer module for both the training
and testing files. The analyzer computes negative, neutral and positive
sentiment scores for each of the sentences and is added as features.

###  4.3. Topic Modeling

Table 2.  CLEF-19 Task 1 Top 3 submissions scores compared to the presented model | MAP | RR | R-P | P@1 | P@3 | P@5 | P@10 | P@20 | P@50
---|---|---|---|---|---|---|---|---|---
Copenhagen | .1660 | .4176 | .1387 | .2857 | .2381 | .2571 | .2286 | .1571 | .1229
TheEarthIsFlat | .1597 | .19531 | .2052 | .0000 | .0952 | .2286 | .2143 | .1857 | .1457
Presented System | .1396 | .2780 | .1348 | .1429 | .1905 | .1714 | .1571 | .1643 | .1171
IPIPAN | .1332 | .2864 | .1481 | .1429 | .0952 | .1429 | .1714 | .1500 | .1171
TOBB ETU | .0884 | .2028 | .1150 | .0000 | .0952 | .1429 | .1286 | .1357 | .0829

Topic Modeling is done with the sentences having label ’1’ with the LDA
algorithm.

Latent Dirichlet allocation (LDA): It is an unsupervised learning approach for
Topic Modeling and a generative probabilistic model of a corpus. Documents are
represented as random mixtures over latent topics, where each topic is
characterized by a distribution over words.
Stopwords and punctuations are removed from the sentences and the retrieved
phrases are converted to Doc2Bow (Document to Bag of Words) model using
Gensim’s dictionary and LDA is performed after it. The number of topics
extracted is 40 and top 5 words with their scores are considered. Most of the
words in the topics contained Nouns. Feature vectors are created with the
scores of the words present in the topics. The scores of the words not in the
topics is considered as 0. From the results, it can be seen that it has
improved the performance considerably.

###  4.4. Data Augmentation

Data Augmentation was performed by replacing Noun and Adjective words in a
sentence with the corresponding most similar word using word2vec similarity.
New generated sentences were added to the training dataset. But the results
did not improve much and hence, are not reported.

##  5\. Evaluation and Results

Following metrics are considered for the task evaluation. Each metric is
described.

###  5.1. Mean Average Precision (MAP)

It is the official metric used in the competition for ranking. It is the mean
of the average precision scores for each query.

(1) $MAP=\sum_{q=1}^{Q}AveP(q)/Q$

###  5.2. Reciprocal Rank

Reciprocal rank(Craswell, 2009a) of a query response is the multiplicative
inverse of the rank of the first correct answer. Mean reciprocal rank is the
average of the reciprocal ranks of results for a sample of queries Q.

(2) $MRR=1/\left|Q\right|\sum_{i=1}^{\left|Q\right|}1/rank\textsubscript{i}$

###  5.3. R-Precision

R-Precision(Craswell, 2009c) is Precision at R, where R is the number of
relevant line-numbers for the evaluated set. R-precision requires knowing all
documents that are relevant to a query. If there are r relevant documents
among the top-R retrieved documents,

(3) $R-precision=r/R.$

###  5.4. Precision@N

Precision@N(Craswell, 2009b) is precision estimated for the first N line-
numbers in the provided ranked list. It is the proportion of the top-n
documents that are relevant. If r relevant documents have been retrieved at
rank N,

(4) $Precision@N=r/N$

Table 1 shows the results after adding the features on top of the TOBB ETU’s
implementation.

In the table:

  * •

SF refers to Semantic Features, the features extracted by performing sentiment
analysis on a sentence.

  * •

TMF refers to the Topic Modeling features, extracted by using Topic Modeling
scores for each sentence.

  * •

TOBB ETU refers to the original system built by the TOBB ETU’s team.

From the results, it can be seen that the approach and features used improve
the scores of all the evaluation metrics. The ablation studies and experiments
also show the contribution and improvement provided by each of the features.
The following analysis can be done from the table:

  * •

It can be seen that building a system using only Sentence BERT embeddings (Row
4) provide very good results. Combining the embeddings and using TOBB ETU’s
system (Row 5) improves the performance. The performance for the MAP score has
improved by 45.58%.

  * •

The sentiment features have not contributed much to the results. Comparing
Rows 1 and 3, it can be seen that the metric results for TOBB ETU’s system and
TOBB Etu’s system with the Sentiment Features are very much similar. Row 2
presents the scores when only Sentiment Features are used in the system.

  * •

Topic Modeling Features also contribute greatly to the results. Row 6 shows
the scores when only Topic Modeling features are used. Combining the features
with TOBB ETU’s system (Row 7) improves the performance in MAP score by
20.24%.

  * •

Combining all the features: SBERT embeddings, Topic Modeling Features and
Semantic Features with the TOBB ETU’s system improves the performance of
original TOBB ETU’s system by 57.91% in MAP score, 17.21% in R-P score and
41.25% in Precision@50.

Compared to CLEF-2019 competition TASK-1 submissions (Table 2), the MAP score
of the presented system is the third best compared to the ninth spot of the
TOBB ETU’s system. The top 2 submissions are neural-based models. The team
Copenhagen have used LSTM recurrent network as the model and the team
TheEarthisFlat has used Feedforward network as the model. Comparatively, the
presented approach doesn’t use any neural model in training and still achieves
very good performance and the results are comparable to the other two.

##  6\. Conclusion

Here, a system is presented for fact-checking. CLEF-2019 CheckThat Lab’s Task
- 1 Check Worthiness! is the focus of the project that checks if a statement
is worth fact-checking and ranks the statements according to which they should
be prioritized. The system presented is built on top of the submission by TOBB
ETU’s team in the challenge and improves the metrics scores of their system by
57.91% in Mean Average Precision Score by combining their implementation with
features such as Sentence BERT embeddings, Topic Modeling Feature Scores and
Semantic Feature scores. The contribution and performance of each of the
feature is also presented as part of the ablation studies. Data Augmentation
was also experimented but it didn’t improve the results. Also, another
advantage of the presented approach is that the model gets trained very
quickly and the results are also obtained very fast as there is no training
time spent like in training a neural model.

###### Acknowledgements.

I express my gratitude to Bahadir Altun and Dr. Mucahid Kutlu (TOBB ETU team)
for their implementation and code on the Github for the task.

## References

  * (1)
  * Atanasova et al. (2018) Pepa Atanasova, Alberto Barron-Cedeno, Tamer Elsayed, Reem Suwaileh, Wajdi Zaghouani, Spas Kyuchukov, Giovanni Martino, and Preslav Nakov. 2018.  Overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims. Task 1: Check-Worthiness.
  * Atanasova et al. (2019) Pepa Atanasova, Preslav Nakov, Georgi Karadzhov, Mitra Mohtarami, and Giovanni Da San Martino. 2019\.  Overview of the CLEF-2019 CheckThat! Lab: Automatic Identification and Verification of Claims. Task 1: Check-Worthiness. In _CLEF_.
  * CheckThat (2019) CheckThat. 2019\.  CLEF2019 - CheckThat!  Retrieved April 18, 2020 from https://sites.google.com/view/clef2019-checkthat/home
  * Coca et al. (2019) L Coca, Ciprian-Gabriel Cusmuliuc, and Adrian Iftene. 2019\.  CheckThat! 2019 UAICS. CLEF.
  * Conneau et al. (2017) Alexis Conneau, Douwe Kiela, Holger Schwenk, Loic Barrault, and Antoine Bordes. 2017.  Supervised learning of universal sentence representations from natural language inference data.  _arXiv preprint arXiv:1705.02364_ (2017).
  * Craswell (2009a) Nick Craswell. 2009a.  _Mean Reciprocal Rank_.  Springer US, Boston, MA, 1703–1703.  https://doi.org/10.1007/978-0-387-39940-9_488
  * Craswell (2009b) Nick Craswell. 2009b.  _Precision at n_.  Springer US, Boston, MA, 2127–2128.  https://doi.org/10.1007/978-0-387-39940-9_484
  * Craswell (2009c) Nick Craswell. 2009c.  _R-Precision_.  Springer US, Boston, MA, 2453–2453.  https://doi.org/10.1007/978-0-387-39940-9_486
  * Dhar et al. (2019) Rudra Dhar, Subhabrata Dutta, and Dipankar Das. 2019\.  A hybrid model to rank sentences for checkworthiness. CLEF.
  * Favano et al. (2019) Luca Favano, M Carman, and P Lanzi. 2019.  TheEarthIsFlat’s submission to CLEF’19 CheckThat! challenge. CLEF.
  * Gasior and Przybyła (2019) Jakub Gasior and Piotr Przybyła. 2019.  The IPIPAN Team Participation in the Check-Worthiness Task of the CLEF2019 CheckThat! Lab.  (2019).
  * Gencheva et al. (2017) Pepa Gencheva, Preslav Nakov, Lluís Màrquez, Alberto Barrón-Cedeño, and Ivan Koychev. 2017\.  A context-aware approach for detecting worth-checking claims in political debates. In _Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017_. 267–276.
  * Hansen et al. (2019) Casper Hansen, Christian Hansen, Stephen Alstrup, Jakob Grue Simonsen, and Christina Lioma. 2019\.  Neural check-worthiness ranking with weak supervision: finding sentences for fact-checking. In _Companion Proceedings of the 2019 World Wide Web Conference_. 994–1000.
  * Hassan et al. (2017) Naeemul Hassan, Fatma Arslan, Chengkai Li, and Mark Tremayne. 2017.  Toward automated fact-checking: Detecting check-worthy factual claims by ClaimBuster. In _Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_. 1803–1812.
  * Konstantinovskiy et al. (2018) Lev Konstantinovskiy, Oliver Price, Mevan Babakar, and Arkaitz Zubiaga. 2018.  Towards automated factchecking: Developing an annotation schema and benchmark for consistent automated claim detection.  _arXiv preprint arXiv:1809.08193_ (2018).
  * Kutlu and Altun (2019) Mücahid Kutlu and Bahadır Altun. 2019.  TOBB-ETU at CLEF 2019: Prioritizing claims based on check-worthiness.  (2019).
  * Patwari et al. (2017) Ayush Patwari, Dan Goldwasser, and Saurabh Bagchi. 2017\.  TATHYA: A multi-classifier system for detecting check-worthy statements in political debates. In _Proceedings of the 2017 ACM on Conference on Information and Knowledge Management_. 2259–2262.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-bert: Sentence embeddings using siamese bert-networks.  _arXiv preprint arXiv:1908.10084_ (2019).
  * Su et al. (2019) Ting Su, Craig Macdonald, and Iadh Ounis. 2019.  Entity detection for check-worthiness prediction: Glasgow Terrier at CLEF CheckThat! 2019. CLEF.
  * Thorne and Vlachos (2018) James Thorne and Andreas Vlachos. 2018.  Automated fact checking: Task formulations, methods and future directions.  _arXiv preprint arXiv:1806.07687_ (2018).
  * Toutanova et al. (2003) Kristina Toutanova, Dan Klein, Christopher D Manning, and Yoram Singer. 2003.  Feature-rich part-of-speech tagging with a cyclic dependency network. In _Proceedings of the 2003 conference of the North American chapter of the association for computational linguistics on human language technology-volume 1_. Association for Computational Linguistics, 173–180.
  * Vasileva et al. (2019) Slavena Vasileva, Pepa Atanasova, Lluís Màrquez, Alberto Barrón-Cedeño, and Preslav Nakov. 2019.  It takes nine to smell a rat: Neural multi-task learning for check-worthiness prediction.  _arXiv preprint arXiv:1908.07912_ (2019).