∎

11institutetext: Rajdeep Kumar Nath 22institutetext: University of Kentucky
Lexington, KY, USA
33institutetext: Himanshu Thapliyal* 44institutetext: University of Kentucky
Lexington, KY, USA
Corresponding Author<EMAIL_ADDRESS>55institutetext: Travis S. Humble
66institutetext: Oak Ridge National Laboratory
Oak Ridge, TN, USA

# A Review of Machine Learning Classification Using Quantum Annealing for
Real-world Applications

Rajdeep Kumar Nath  Himanshu Thapliyal  Travis S. Humble

(Received: date / Accepted: date)

###### Abstract

Optimizing the training of a machine learning pipeline helps in reducing
training costs and improving model performance. One such optimizing strategy
is quantum annealing, which is an emerging computing paradigm that has shown
potential in optimizing the training of a machine learning model. The
implementation of a physical quantum annealer has been realized by D-Wave
systems and is available to the research community for experiments. Recent
experimental results on a variety of machine learning applications using
quantum annealing have shown interesting results where the performance of
classical machine learning techniques is limited by limited training data and
high dimensional features. This article explores the application of D-Wave’s
quantum annealer for optimizing machine learning pipelines for real-world
classification problems. We review the application domains on which a physical
quantum annealer has been used to train machine learning classifiers. We
discuss and analyze the experiments performed on the D-Wave quantum annealer
for applications such as image recognition, remote sensing imagery,
computational biology, and particle physics. We discuss the possible
advantages and the problems for which quantum annealing is likely to be
advantageous over classical computation.

###### Keywords:

Classification Machine Learning Optimization Quantum Annealing Quantum
Computing

##  1 Introduction

Machine learning techniques to explore and harness the power of data have
found application in health care, finance, autonomous driving, security, etc.,
and are continuing to make their impact deeper every single day intro_002
intro_003 intro_004 . However, this technological boom is accompanied by
unprecedented challenges. These challenges arise from several factors such as
the scale of data being generated, hardware limitations, computational
complexity, and cost intro_001 . Although recent advances in hardware
technology have increased computational capability significantly, this is no
match for the amount of data generated globally, which is rapidly increasing,
as well as the amount of stored data, which is increasing at the rate of about
20% per year 001 . The current trend of technological advancement in terms of
computational power will become saturated in dealing with the massive scale of
data which will result in increased cost, and more importantly, the maximum
utilization of data won’t be possible in the future intro_001 . To deal with
this challenge, quantum computing is seen as a promising alternative that can
boost the computational prowess in utilizing the power of data to the full
extent intro_005 .

Figure 1: Processing in classical (top) and quantum systems (bottom). QPU can
generate many-solutions and counter-intuitive solutions faster than classical
processing units such as CPU/GPU.

Most of the real-world problems that machine learning seeks to solve are non-
convex in nature. These types of problems are hard to converge to an optimal
solution using classical algorithms because of the possibility of the presence
of several local minima and saddle points intro_006 . This computational
challenge limits the ability of machine learning models to train efficiently
especially in the presence of a large number of variables. To deal with this
problem, researchers have been exploring alternative computing techniques such
as quantum computing to augment the power of machine learning algorithms 003
015 017 . Quantum computing is inherently suited to carry out complex
computation which is not possible using classical computation 034 . Quantum
computing processes information in the form of qubits (quantum bits). This is
different from a classical bit in the sense that a classical bit can take the
value of either 0 or 1. However, a qubit can be represented as a combination
of two values at the same time. This principle is called superposition which
allows quantum computation to achieve exponential speedup over its classical
counterpart 002 . Another property of quantum computation that establishes
superiority over classical computation is the phenomenon of quantum tunneling.
Because of quantum tunneling, a quantum system can navigate through solution
space more efficiently by overcoming long and thin energy barriers 004 . These
characteristics of quantum computing have led to the hypothesis that QPU
(Quantum Processing Unit) can be used to discover more interesting and
counter-intuitive patterns for machine learning classification task than that
obtained using classical processing units such as CPU (Central Processing
Unit) or GPU (General Processing Unit) 005 (Figure 1).

The main contributions of this article are as follows:

  * •

The integration of D-wave’s quantum annealer as an optimization subroutine has
been analyzed and discussed in the context of machine learning classification
tasks.

  * •

Application domain where quantum annealing is applied for real-world
classification tasks has been identified and analyzed from existing
literature.

  * •

Finally, the possible advantages of using quantum annealer as an optimization
subroutine for machine learning classification have been analyzed based on the
findings from recent research.

Developing a hybrid classical-quantum system for machine learning
classification tasks is expected to perform better than using a classical
machine learning pipeline under data uncertainty conditions. Some of these
conditions are limited availability of training data, modeling under high
dimensional feature variables, better generalization to unknown data, and
reduced training time and cost. This article explores the effectiveness and
usefulness of quantum annealing as an optimization routine in machine learning
pipelines for classification tasks. We present a comprehensive survey on
quantum annealing for real-world machine learning classification tasks based
on recent research works presented in the literature.

The paper is organized as follows: Section 2 discusses the theoretical
background of quantum annealing and also the realization of the quantum
annealing in D-Wave’s quantum computer. Section 3 discusses the hybrid
classical-quantum computing system using D-Wave’s quantum computer. This
section details the D-Wave’s implementation and translation of real-world
problem instances into the quantum processing unit of D-Wave. Section 4 will
explore the existing literature in the area that has used quantum annealing
for machine learning classification tasks. Section 5 discusses the possible
advantages of using quantum annealing over classical techniques. Section 6
concludes the article.

##  2 Background on Quantum Annealing

Quantum annealing is a heuristic search algorithm for finding the lowest
energy state by traversing over the solution landscape 035 . The lowest energy
state is achieved through the evolution of a time-dependent Hamiltonian.
Quantum annealing uses quantum tunneling to traverse through the solution
space more efficiently than classical annealing to reach the optimal state 007
; 008 . As visualized in Figure 2, the time-dependent Hamiltonian can be
represented as a landscape of the cost of the solution. The two components of
the time-dependent Hamiltonian is the final Hamiltonian ($H_{F}$), that is the
ground state or the optimal solution to the problem and the second component
is a transverse field Hamiltonian ($H_{D}$) which is scaled by a time-
dependent coefficient also called transverse field which is initialized at a
high value and then decreased to zero. Large values of the transverse field
allow the algorithm to avoid local minima and tunnel through large energy
barriers to move towards the ground state. This same phenomenon is achieved in
classical annealing by thermal jumps. It has been theoretically proven that
quantum annealing is guaranteed to converge to the optimal solution and also
the convergence rate of quantum annealing was faster than that of classical
annealing 009 .

Figure 2: Solution landscape of a cost function represented by a time-
dependent Hamiltonian. High values of the transverse field coefficient drives
tunneling phenomenon in order to escape local minima and move towards the
optimal solution.

Even though quantum annealing optimization algorithm has proven to be
theoretically superior to classical annealing, a classical implementation of
such an algorithm is both costly and inefficient 007 . Whereas some operations
are more efficient and faster using quantum computing, there are also several
operations for which classical systems are inherently better than quantum
systems. A hybrid system that uses both classical and quantum computing is
gaining immense popularity in recent years. In the context of machine
learning, training becomes extremely expensive and inefficient as the
dimension of the feature vector increases which in turn makes it difficult to
find meaningful patterns. These types of computationally challenging tasks
could be offloaded to quantum processors for optimization 036 . Recent
advances made in the design of physical quantum annealer have made it possible
to explore the practical application of quantum algorithms. D-Wave system is
currently leading the market of commercial quantum computers which uses
quantum annealing for computation. We will now present a brief overview of the
implementation of quantum annealing on D-Wave systems.

###  2.1 Quantum Annealing in D-Wave Systems

In D-Wave’s implementation of quantum annealing, a qubit initially remains in
a superposition state. At the end of annealing, the qubit goes from the
superposition state to either 0 state or 1 state. Figure 3 shows the energy
diagram depicting the physics associated with the process of quantum annealing
and how the qubits attain the lowest energy state.

Figure 3: Energy diagram of the quantum annealing process of a single qubit.
Figure 4: Overview of the steps required for computation using D-Wave’s QPU
007

Figure 3 shows three configurations of the energy diagram. The initial
configuration (a) consists of only one valley, with the qubit in the
superpositioned state. When quantum annealing is run, a barrier is raised
which gives rise to the formation of a double-well potential (configuration
(b)). In this configuration, the qubits have an equal probability of ending up
either in the 0 state (low point of the left valley) or in 1 state (low point
of the right valley). An interesting feature of the quantum annealing
processor is that the probability with which a particular qubit will fall
either in 0 state or 1 state can be controlled by applying an external
magnetic field to the qubits also called the bias. In other words, the qubits
can minimize their energy under the influence of the external magnetic field
or bias.

Coupling is another important feature of the quantum annealing processor.
Coupling is the method through which two coupled qubits can be made to be
either in the same state or in a different state, that is both 0 or 1 or one 0
and the other 1 or vice versa. This phenomenon of coupling is known as
entanglement in quantum computing. For example, when two qubits are entangled,
they are considered as one object but with four possible states or
combinations. Hence, a two-qubit system will have potential with four states
where each state represents a different combination. This defines the energy
landscape of the two qubits governed by the relative energy between the
qubits. The relative energy between the qubits depends on the biases of each
qubit and the coupling between them. The programmer chooses the bias and
coupling to encode a specific problem instance.

##  3 Quantum Processing Unit of D-Wave Systems

In the previous section, we discussed the basics of computation using quantum
annealing and discussed some key concepts such as qubits, bias, and coupler.
These concepts are necessary for understanding the architecture of the
D-Wave’s Quantum Processing Unit (QPU) and the application of D-Wave’s QPU in
solving real-world problem instances. Figure 4 shows an overview of the steps
involved in performing computation using D-Wave’s QPU. The overall process can
be viewed as a hybrid system consisting of both classical computation and
quantum computation.

###  3.1 Classical Computation

The classical computation consists of three main parts, (i) Problem
initialization, (ii) Programming the problem instance into the quantum
annealing hardware through a software interface, and (iii) Resampling. The
first two steps take place before annealing and the last step takes place
post-annealing.

####  3.1.1 Problem Instance

To solve a problem with QPU, the problem needs to be translated into either an
Ising objective function or a QUBO objective function. Translation between
these two types of functions is straightforward. Both these functions support
only quadratic functions, the reason for which will be clear in section 3.1.2.
For example, considering an Ising objective seeks to find real-valued weights
$h_{i}$ and $J_{ij}$ to n-spin variables that will minimize the objective
function. In the context of machine learning classification, $h_{i}$ and
$J_{ij}$ can be thought of as biases and weights and the n-spin variables can
be thought of as the features or variables that are used for classification.
After the problem instance is defined, it is programmed into the quantum
annealing hardware using a software interface.

####  3.1.2 Programming into QPU

Programming into the QPU mainly consisting of initializing and loading the
bias and weight associated with qubits. The biases and weights of the spin
variables are mapped into the physical qubits of the quantum processor. The
qubits are physically arranged in the form of a unit cell, with each unit cell
consisting of 8 qubits. Figure 5 shows a $3\times 3$ layout of unit cells
consisting of a total of 72 qubits.

Figure 5: Architecture of D-Wave’s QPU layout

Out of the 8 qubits, 4 qubits are placed horizontally and 4 are placed
vertically. Each vertical qubit is fully connected to the four horizontal
qubits and vice-versa. The qubits in a unit cell are connected using internal
couplers (shown in green dots). External couplers can be used to connect
qubits between two different unit cells and also between vertically placed
qubits in the same unit cell. For example, the blue dots connected by a line
shows the interconnection between two horizontally placed qubits in two
different unit cells and the dotted purple line represents the connection
between two vertically placed qubits in the same unit cell. This type of
layout is also called a Chimera graph.

The qubits of the D-Wave’s QPU can be represented as nodes and the edges
connecting the nodes as couplers. Figure 6 shows a unit cell with 8 qubits.
The four qubits shown in blue vertical lines represent nodes 1-4 and the four
red horizontal lines represent qubits 5-8. A transformation of the layout to
its equivalent graph structure resembles a neural network with two fully
connected layers, with four neurons at each layer. Such a structure can be
used to represent an objective function in machine learning. For example, if
we consider a two-variable objective function, $H(a,b)=5a+7ab-3b$, can be
mapped into a two-qubit system as shown in Figure 6 (below).

Figure 6: Graphical representation of a two-variable objective function

In the objective function, we have two variable and the constant terms
associated with the two variables are the bias term that is 5 and 3 in this
case. The constant term in the quadratic expression of the two variables, that
is 7 in this case is the strength of the coupling between the two variables.
The structure of the D-Wave system’s QPU supports objective functions with a
maximum degree of 2, or in other words, quadratic expression. Alongside Ising,
QPU also supports QUBO problems known as Quadratic Unconstrained Binary
Optimization (QUBO). The objective function can be mathematically expressed as
$\underset{x\in(0,1)^{n}}{min}x^{T}Qx$, where $x$ is a binary vector and $Q$
is an upper triangular matrix of real weights, and the objective is to
minimize the above expression.

An important aspect of programming into D-Wave’s QPU, is the identification of
a representation of the cost function dependency graph within the constraints
of the Chimera hardware. A number of techniques have been developed for this
graph embedding procedure 037 ; 038 . Embedding increases the number of
resources required to represent the original problem but ensures that the
correct logical relationships are encoded. Specialized embeddings can be
developed for constrained input graph structures, such as the bipartite graph
that is frequently used in machine learning 039 . After programming the biases
and weights into the chimera graph structure for the given problem instance
and initializing other hyperparameters, the problem is sent to the quantum
hardware for computation.

####  3.1.3 Resample

The solution returned by the quantum annealing process may not end up in the
ground state because of the influences of external energy sources 035 . Hence,
in practice, the probability of the system staying in a low energy state might
be low for certain problems. This can be mitigated by taking several such
close-to-optimal solutions and generate a distribution of the possible
solutions. The distribution of the possible solutions determines the frequency
with which a specific solution is observed.

###  3.2 Quantum Computation

After the qubits are loaded with biases and weights, the system undergoes
quantum annealing to find solutions to the problem instance. This process is
distinguished into two parts: (i) Quantum Annealing and (ii) Readout. After
the process of quantum annealing, the initial ground energy states evolve to
their optimal ground energy states and the biases and weights change to their
optimal configurations. After the transition ends, the qubits have classical
spin values as governed by the objective function. After annealing, a readout
is performed to retrieve the classical spin values and perform a scaling
operation to convert the classical spin to real-valued weights. The total time
for performing an anneal operation can be estimated by summing up the
programming time, annealing time, and readout times. Hence, resampling the
solution greatly reduces the total computation time as only the readout phase
has to be executed instead of performing the annealing operation several
times.

With the background on quantum annealing and the D-Wave’s quantum annealing
processor, we will now explore the application of quantum annealing to machine
learning classification tasks. These theoretical sections on quantum annealing
background will help us analyze and understand the different ways in which
quantum annealing can be used to improve the efficiency of classification
algorithms.

##  4 Quantum Annealing for Machine Learning Classification

In recent years, researchers have been interested in applying quantum
annealing to improve various aspects of training a machine learning classifier
003 ; 015 ; 016 ; 018 . These works vary from each other in the ways they have
used quantum annealing for training, the physical D-Wave computer on which the
QA algorithm is implemented, the dataset, and the application domain. Table 1
shows the different versions of D-Wave systems along with attributes like the
number of qubits, couplers, and operating temperature 010 ; 011 ; 012 ; 007 ;
018 ; 032 . Recently, a 5000 qubit system has been also launched by D-Wave
systems called Advantage. However, because the latest version with 5000 qubits
has been launch very recently, it is not in the scope of the discussion of
this manuscript.

Table 1: Attributes of four versions of the D-Wave systems (Temp is the
operating temperature $T_{p}$ is the initialization/programming time, $T_{a}$
is the anneal time and $T_{r}$ is the read out time.)

Attributes | One | Two | 2X | 2000Q | Advantage
---|---|---|---|---|---
Qubits | 128 | 512 | 1152 | 2048 | 5640
Couplers | 352 | 1472 | 3360 | 6016 | 40,184
Temp (K) | NA | 0.02 | 0.015 | 0.015 | 0.015
$T_{p}$ | 270 ms | 36 ms | 10 ms | NA | NA
$T_{a}$ | 1 ms | 20 $\mu$s | 20 $\mu$s | 20 $\mu$s | 20 $\mu$s
$T_{r}$ | 1.5 ms | 0.13 ms | 120 $\mu$s | NA | NA

Machine learning tasks using quantum annealing are performed in a hybrid
classical and quantum system. The various aspects of training which need to be
optimized can be offloaded to the quantum processing unit for speed and
accuracy. Researchers have utilized the power of quantum annealing in various
aspects of training a machine learning classifier and validated the results on
a wide range of dataset suitable for various real-time applications. In this
section, we will present and analyze existing works in this area by
categorizing the works based on the applications for which quantum annealing
was intended. Table 2 lists the articles that have reported results on
performing machine learning classification tasks using D-Wave’s quantum
annealer based on recent research works. The works are tabulated by
categorizing them based on the intended application, the version of the D-Wave
on which the problem was solved, three aspects of training which include the
aspect of the training in which quantum annealing was used for and its
classical counterpart with which QA was compared. Finally, the table also
consists of aspects of the dataset such as source, the size of training, and
test data set. Listing these aspects in a tabular form will help us to
properly organize our discussion of these works.

Apart from the works discussed here, there are several works that have used
quantum annealing for machine learning tasks. For example, recently, quantum
annealing was also applied for training a restricted Boltzmann machine for
applications in cybersecurity 041 . Apart from the machine learning
classification tasks performed on physical D-Wave’s quantum computer, several
other works have also explored the idea of using quantum annealing for
optimization by simulation of the quantum phenomenon instead of the actual
hardware. Some of the examples of such works are in the area of lung cancer
detection 026 , multi-class classification 027 , training in presence of label
noise 028 , natural language processing, seizure prediction, and linear
separability testing 029 . However, in this paper, our focus is to discuss and
analyze the use of quantum annealing for training machine learning classifiers
using domain-specific dataset. Hence, we will be focusing only on those works
that have emphasized real-world applications using domain-specific dataset and
have used a physical quantum annealer.

Table 2: Related work on machine learning classification using quantum
annealing for optimizing training. Application refers to the broader on which
the use of quantum annealing was validated on. Quantum annealing application
summarizes the primary use of quantum annealing in the training procedure and
classical comparison refers to the alternative classical training approach
that is used for comparison. DCNN=Deep Convolutional Neural Network, DNN=Deep
Neural Network, SA=Simulated Annealing, QSA=Quantum Simulated Annealing,
MLR=Multivariate Linear Regression, SVM=Support Vector Machine, CD=Contrastive
Divergence.

|  |  | Training | Dataset
---|---|---|---|---
|  | D-Wave | Quantum Annealing | Classical | Source | Train | Test
Work | Application | Version | Application | Comparison |  | Size | Size
Adachi et al. 003  | Image | D-Wave 2 | Estimation of | Contrastive Divergence | MNIST | 60,000 | 10,000
| Recognition |  | model expectation | based training |  |  |
Nguyen et al. 014  | Image | D-Wave 2X | Generating sparse | Classical matching pursuit | MNIST | 50000 | 5000
| Recognition |  | Representation | SVM, Alex Net like DCNN |  |  |
Boyda et al. 015  | Remote Sensing | D-Wave 2X | Selecting optimal | Simulated Annealing | NAIP | 24,000 | 6,000
| Imagery |  | voting subset |  |  |  |
Mott et al. 016  | Particle Physics | D-Wave 2X | Selecting optimal set | DNN, XGBoost | Simulated | 20 x | NA
|  |  | of weak classifiers |  |  | 20000 |
Li et al. 017  | Computational | D-Wave 2X | Estimation of | SA, SQA, MLR, | gcPBM | 1500 | 166
| Biology |  | feature weights | Lasso, XGBoost | HT-SELEX | 4500 | 500
Willsch et al. 018  | Computational | 2000Q | Quantum SVM | Classical SVM | 019  | 4352 | 484
| Biology |  |  |  |  |  |
Dixit et al. 020  | Image | 2000Q | Estimation of | Contrastive Divergence | BAS | 300 | 200
| Recognition |  | model expectation | based training |  |  |
Caldeira et al. 024  | Image | 2000Q | Estimation of | CD, SA, | Galaxy Zoo | NA | NA
| Recognition |  | model expectation | Gradient Boosted DT |  |  |
Liu et al. 032  | Image | D-Wave 2X | Estimation of | High Performance | MNIST | NA | NA
| Recognition |  | model expectation | Computing |  |  |
| Particle Physics |  |  | Neuromorphic Computing | 033  | NA | NA

###  4.1 ML Applications Using D-Wave’s Quantum Annealing

In the current research landscape, researchers have used quantum annealing for
real-world application using domain-specific dataset in four broad areas: (i)
Image Recognition (ii) Remote Sensing Imagery, (iii) Particle Physics, and
(iv) Computational Biology (Figure 7). The representative works on these
domains will be discussed in detail in this section. The works discussed here
have adopted a hybrid classical-quantum system for optimizing machine learning
classifiers. Based on the works discussed here, an overview of such a hybrid
system can be illustrated with the help of Figure 8.

####  4.1.1 Image Recognition

Work done in 003 ; 014 ; 020 ; 024  has investigated the effectiveness of
using quantum annealing in image recognition tasks. Qualitatively, 003 ; 020
have used QA in a similar way but the implementation varied with respect to
the D-Wave version used and the dataset used for validation. For 003 , the
MNIST data set was used and the classifier’s task was to recognize handwritten
digit (0-9), whereas in 020 , a simple bar and strip dataset was used.
Further, the training size used in 003  is about 60,000 samples and that of
020  had about 300 training samples. On the other spectrum, 003  and 014  used
two different approaches in using QA but the validation was done on the same
dataset (MNIST) with 50000 training samples. A similar idea of RBM training
using QA to classify the shape of galaxies is used in 024 .

Figure 7: Representative works in some of the domains where quantum annealing
was applied in machine learning

In 003 , authors used quantum annealing to determine the model expectation of
a Deep Belief Network (DBN) which is a Deep Neural Network with stacked
Restricted Boltzmann Machine 021 . The model expectation of a Deep Belief
Network is typically performed using Contrastive Divergence. The experiment
was conducted on the MNIST dataset which is a benchmark dataset for image
recognition 022 . The dataset consists of handwritten digits between 0 and 9
with dimension $28\times 28$. To compare the quantum and the classical
methods, two DNNs of the same size were implemented. Each DNN had 32 input
nodes, two hidden layers followed by 10 output layers. The training was
divided into generative (quantum annealing is used) and discriminative
(completely classical). For the quantum model, quantum annealing was used to
estimate model expectation in the generative phase and for the classical
model, Contrastive Divergence was used. The generative phase calculates
updates to the weights and biases and the discriminative phase uses
backpropagation for fine-tuning the biases and weights. Results showed that
the quantum model could achieve a specific level of accuracy with fewer
iterations of generative and discriminative training. A quantitative
evaluation showed that the quantum model outperforms the classical model in
terms of accuracy with only 20 and 100 generative and discriminative
iterations respectively. The classical model took about 50 generative and 800
discriminative iterations to reach the same level of accuracy.

A similar approach was used in 020 , however, the idea was tested on BAS (bar
and strip database) and on D-Wave 2000Q. However, instead of a DBN, the
authors used a single RBM unit with a visible layer and hidden layer. The
article focused on the comparison of the quantum implementation of RBM and
classical implementation of RBM on the D-Wave computer. The implementation of
the RBM in the quantum model had 64 visible units and 64 hidden units with all
nodes of the visible layer fully connected to all nodes of the hidden layer.
However, due to faulty qubits, some connections were non-functional as was the
case in 003 . The structure of the RBM in the classical model was also the
same. The only difference between the two models is that the quantum model
estimated model expectation of the RBM using quantum annealing as opposed to
CD-based training in the classical model. Both the models were validated on a
BAS dataset containing 500 records consisting of 64 bit variables each. The
first 62 bits represent the image and the last two bits represent its label
(Bar or Strips). Results showed that both the approaches gave comparable
classification performance, however, higher fluctuations in the classification
accuracy were observed with CD-based training.

In 014 , the implementation of recognizing handwritten digits was implemented
on D-Wave 2X. This work followed a different approach to using quantum
annealing for training. In this work, the authors used QA to generate a sparse
representation of the images. First, the images were downsampled and were fed
into a fully connected graph of 47 logical qubits. The objective of the
quantum annealing procedure, in this case, was to obtain an optimal sparse
representation of the image that can most accurately represent the original
image. The method was compared with the classical approach in two ways: (i)
First, the sparse representation generated by QA was fed to a softmax
classifier and an SVM classifier and the result was compared with the sparse
representation generated by Alex Net like DCNN. (ii) An equivalent classical
sparse representation algorithm known as the classical matching pursuit was
used to find the optimal sparse representation of the images. The resulting
sparse representation from QA based approach and classical approach were fed
into a Multi-Layer Perceptron (MLP) for classification. In both cases,
superior performance of the quantum-based approach was observed. Another
comparison was done with state-of-the-art RESNET (with a large number of
neurons and parameters) which showed slightly better performance than the
quantum approach. However, the authors found that the accuracy of RESNET
decreased as the number of training samples was reduced.

In 024 , the idea of using QA to estimate model expectation of an RBM machine
in classifying the morphology of galaxies. The model was implemented on a
D-Wave 2000Q computer with 2000 qubits. The quantum approach was compared to
Contrastive Divergence based training, Simulated Annealing and gradient
boosted decision trees. Galaxy Zoo dataset was used for this experiment. The
images of the galaxies were first compressed using principle component
analysis to represent the image in the physical hardware. Experimental results
showed that quantum annealing could perform better than classical algorithms
if the dataset is small and with a limited number of training iterations.
Overall, the authors did not find any algorithmic advantage of using QA over
the classical approach for this particular dataset. In 032 , quantum annealing
was used to train a Limited Boltzmann Machine, which is a variation of
Restricted Boltzmann Machine to recognize handwritten digits.

Figure 8: Example of a generic machine learning pipeline illustrating some of
the phases that can be optimized using a quantum annealer.

####  4.1.2 Remote Sensing Imagery

In 015 , researchers have investigated the application of quantum annealing in
a remote sensing imagery problem. The problem objective was to classify image
data as either covered with trees or not covered with trees. In this problem,
quantum annealing was used to select an optimal set of voting classifiers that
could most accurately classify a given segment as covered with trees or not.
The idea was to build around a modified implementation of QBoost 023  which is
a quantum version of boosting algorithm. 537 image tiles were taken from the
National Agriculture Imagery Program (NAIP). The selected image tiles
contained 30000 8x8 labeled pixel squares. 24000 samples were used for
training and 6000 were used for validation and testing. 112 features were
extracted from the image tiles which resulted in 224 weak classifiers. After
performing some initial preprocessing, like discarding classifiers that did
not perform better than random guessing, 108 weak classifiers remained.
Quantum annealing was used to select the optimal set of weak classifiers. This
method was compared with simulated annealing and results showed that both
these methods performed comparably in terms of optimal solution and rate of
convergence.

####  4.1.3 Computational Biology

The application of quantum annealing in solving computational biology problems
has been investigated in 017 ; 018 . In 018 , the objective was to classify a
particular protein on its ability to bind with a particular DNA sequence.
Whereas in 017 , the objective was to classify the strength of the binding of
a protein and a DNA sequence based on a pre-determined threshold.

In 018 , the authors implemented a quantum version of Support Vector Machine
by representing the objective function of the SVM as QUBO formulation on
D-Wave 2000Q machine. Subsequently, quantum annealing was used to estimate the
weights of the objective function. This method was compared with the classical
implementation of SVM. The modeling was performed on an experimental dataset
019 , with 4352 training samples and 454 test samples. Results showed that the
combination of multiple solutions obtained from D-Wave quantum annealer proved
to be superior to the single solution obtained from classical SVM. The authors
also observed that the combination of the multiple solutions returned by
quantum annealing could possibly generalize better to unseen data than the
classical SVM. Based on the findings, the authors have concluded that the
quantum version of SVM could find useful practical applications for hard
classification problems where sufficient training data is not available.

In 017 , QA was used to estimate the weights of an objective function of the
form $F(\vec{w})=R(\vec{w})+\Omega(\vec{w})$, where $R$, $\Omega$, and
$\vec{w}$ are the loss, regularization and feature weights respectively. The
quantum approach was compared with Simulated Annealing (SA), Simulated Quantum
Annealing (SQA), Multiple Linear Regression (MLR), Lasso, XGBoost. The
objective of the experiment was to compare the performance of the quantum and
classical methods in classifying TF-DNA (Transcription Factor-DNA) binding.
The experiment was conducted on two types of dataset, one is the gcPBM
(genomic-context protein binding microarray) with 1500 training and about 166
test samples. The second is the HT-SELEX (high-throughput systematic evolution
of ligands by exponential enrichment) with 4500 train and 500 test sizes.
D-Wave 2X was used for this experiment with 1000 qubits. Experimental results
on the two dataset showed that the QA-based approach could outperform the
classical technique by a slight margin if the training data is limited in
size. Another interesting finding reported by the research is the ability of a
quantum-based solution to learn relevant biological patterns.

####  4.1.4 Particle Physics

Quantum annealing has also been applied to a particle physics problem in 016 .
The primary objective of the experiment was to evaluate the efficiency of
quantum annealing in classifying Higgs signal from background noise. The
quantum method was compared with the classical Deep Neural network and XGBoost
framework. QA was used in the selection of the optimal number of weak
classifiers from a total of 36 weak classifiers. The weak classifiers were
constructed from the distribution of kinematic variables. The experiment was
conducted in D-Wave 2X with 1000 qubits. A simulated dataset was used for the
experiment with a 20 x 20000 training dataset. Experimental results showed
that the QA-based approach was not very efficient in finding the true minimum.
The reason for this has been attributed to the noise in the physical system
which is exacerbated by the sparse connectivity among qubits. Further
investigation with dataset of varying sizes showed the clear advantage of
using the quantum approach over the classical approach for optimization.
However, the advantage diminishes as the dataset grows larger. Another work in
032  explored the application of quantum annealing in neutrino detection
dataset.

##  5 Advantages of using QA for Training ML Models

Analyzing the experimental results on the existing works on using quantum
annealing for training, the following possible advantages of using QA for
machine learning classification can be inferred.

###  5.1 Training under limited data

Quantum annealing could be a better choice for training ML classifiers in the
presence of limited training data. Results from 014 ; 024 ; 018 ; 017 ; 016
consistently reported higher performance of quantum annealing based approach
when the dataset is small. For smaller dataset, the performance of QA was
reported to be better than that of classical approaches. However, classical
approaches outperformed quantum-based training when the size of the data is
increased. The poor performance of the quantum approach was attributed to the
hardware constraint of the quantum hardware. Further research on different
dataset is needed to validate the effectiveness of using quantum annealing for
training models with limited data.

###  5.2 Dimension Reduction

Feature selection is an important subroutine for optimizing the training and
performance of machine learning models and experimental results indicate the
potential of QA over the classical approach for dimension reduction 014 ; 040
. Results reported in 014  confirmed the superior performance of the sparse
representation generated by QA over the classical approach. Further,
experimental results based on the application of QA to remote sensing imagery
015 , reported that the QA-based approach of selecting an optimal voting
subset from feature variables outperformed simulated annealing based approach.

###  5.3 Generalization to Test Data:

Machine learning classifiers based on quantum computation are hypothesized to
be able to generalize better to unseen data than purely classical machine
learning classifiers 025 ; 018 ; 015 . Results reported in 018 , showed that
the solutions returned by the quantum version of SVM were able to better
generalize to new data as opposed to the classical version of SVM. Also, the
results reported in the remote sensing imagery problem 015 , was also
consistent with the strong generalization performance of the the QA-based
classification approach. However, further research needs to be done to
validate the generalization performance on different data sets and also for
larger dataset.

###  5.4 Multiple Solutions

Quantum annealing can also possibly yield many counter-intuitive solutions to
a given optimization problem. In the readout phase (Figure 4), qubits stay in
a superposition state. The real-valued weights and biases are obtained by
scaling the classical spin values of the qubit. Hence, every time the readout
operation is performed, the hardware returns slightly different solutions.
This is validated in the experiment performed in 018  showed that the quantum
version of SVM was able to generate several solutions as compared to the
single solution obtained by the classical SVM which are also close to optimal
solutions. Further experiments performed in the application of quantum
annealing for signal isolation 016  also highlight the superior performance of
quantum annealing in generating diverse solutions.

###  5.5 Reduced Training Time

Quantum computing techniques owing to their superposition and entanglement
phenomenon can evaluate $2^{N}$ states at once along with speed up due to
quantum tunneling. Because of this, it is hypothesized that quantum computing
strategies can achieve significant speed-ups in machine learning applications.
This reduction in training time will also make the model learning more
effective in production when it has to update its weights in real-time in
presence of new data. Initial experimental results concerning speed-ups have
shown promise of the theoretical results derived so far. In 003 ; 024 ,
researchers noticed superior performance of the QA-based training when the
training iteration was limited. This implies that given similar training
conditions for quantum and classical approaches, it is expected that the
quantum-based training approach will converge to the solution with fewer
iterations than the classical approach. However, further investigations with
rigorous experimental proof are required to experimentally validate the effect
of quantum speed-up in reducing training time.

##  6 Conclusions and Discussions

In this article, we have explored a novel emerging computing paradigm known as
quantum annealing and its application in the context of machine learning
classification. We have presented a detailed survey of the existing efforts in
the direction of applying quantum annealing for improving various aspects of
training a machine learning classifier. Our survey has been limited to those
demonstrations that emphasized real-world applications. We have discussed the
background of quantum annealing and its implementation in D-Wave’s quantum
computer. We have categorized and analyzed the existing works that have used
D-Wave’s quantum computer for machine learning classification based on the
application domain. Based on the experimental results and analysis, we have
also listed several aspects of machine learning classification that can be
better handled with a hybrid classical-quantum system. Quantum annealing has
immense potential in advancing state-of-the-art machine learning classifiers
through improvement in speed and performance. However, several challenges need
to be overcome before quantum annealing can be widely used as an alternative
for classical computation for classification tasks. This includes but is not
limited to an increase in the number of qubits, the connections among the
qubits, etc. Despite the limitations faced currently, the current trend
indicates a bright possibility of successfully incorporating quantum
computation techniques in everyday real-world applications in the near future.

##  7 Compliance with Ethical Standards

Conflict of Interest: The authors declare that they have no conflict of
interest.
Ethical approval: This article does not contain any studies with human
participants or animals performed by any of the authors.

## References

  * (1) The D-Wave 2000q™ quantum computer technology overview (2019).  URL https://www.dwavesys.com/sites/default/files/D-Wave%202000Q%20Tech%20Collateral0117F.pdf
  * (2) D-Wave previews next-generation quantum computing platform (2019).  URL https://www.dwavesys.com/press-releases/d-wave-previews-next-generation-quantum-computing-platform
  * (3) Adachi, S.H., Henderson, M.P.: Application of quantum annealing to training of deep neural networks.  arXiv preprint arXiv:1510.06356 (2015)
  * (4) Ajagekar, A., Humble, T., You, F.: Quantum computing based hybrid solution strategies for large-scale discrete-continuous optimization problems.  Computers & Chemical Engineering 132, 106630 (2020)
  * (5) Alexeev, Y., Bacon, D., Brown, K.R., Calderbank, R., Carr, L.D., Chong, F.T., DeMarco, B., Englund, D., Farhi, E., Fefferman, B., et al.: Quantum computer systems for scientific discovery.  PRX Quantum 2(1), 017001 (2021)
  * (6) Biamonte, J., Wittek, P., Pancotti, N., Rebentrost, P., Wiebe, N., Lloyd, S.: Quantum machine learning.  Nature 549(7671), 195–202 (2017)
  * (7) Boothby, T., King, A.D., Roy, A.: Fast clique minor generation in chimera qubit connectivity graphs.  Quantum Information Processing 15(1), 495–508 (2016)
  * (8) Boyda, E., Basu, S., Ganguly, S., Michaelis, A., Mukhopadhyay, S., Nemani, R.R.: Deploying a quantum annealing processor to detect tree cover in aerial imagery of california.  PloS one 12(2), e0172505 (2017)
  * (9) Caldeira, J., Job, J., Adachi, S.H., Nord, B., Perdue, G.N.: Restricted boltzmann machines for galaxy morphology classification with a quantum annealer.  arXiv preprint arXiv:1911.06259 (2019)
  * (10) Creus Botella, G.: Exploring machine learning advances in finance.  B.S. thesis, Universitat Politècnica de Catalunya (2021)
  * (11) Dema, B., Arai, J., Horikawa, K.: Support vector machine for multiclass classification using quantum annealers (2020)
  * (12) Denchev, V.S., Ding, N., Vishwanathan, S., Neven, H.: Robust classification with adiabatic quantum optimization.  arXiv preprint arXiv:1205.1148 (2012)
  * (13) Dixit, V., Selvarajan, R., Alam, M.A., Humble, T.S., Kais, S.: Training a quantum annealing based restricted boltzmann machine on cybersecurity data.  arXiv preprint arXiv:2011.13996 (2020)
  * (14) Dixit, V., Selvarajan, R., Alam, M.A., Humble, T.S., Kais, S.: Training and classification using a restricted boltzmann machine on the d-wave 2000q.  arXiv preprint arXiv:2005.03247 (2020)
  * (15) Dulny III, J., Kim, M.: Developing quantum annealer driven data discovery.  arXiv preprint arXiv:1603.07980 (2016)
  * (16) Francia, G.A., El-Sheikh, E.: Applied machine learning to vehicle security.  In: Machine Intelligence and Big Data Analytics for Cybersecurity Applications, pp. 423–442. Springer (2021)
  * (17) Gao, H., Li, Y., Zhang, Z., Zhao, W.: Machine learning used in biomedical computing and intelligence healthcare, volume i.  Frontiers in Genetics 12 (2021)
  * (18) Garg, S., Ramakrishnan, G.: Advances in quantum deep learning: An overview.  arXiv preprint arXiv:2005.04316 (2020)
  * (19) Goodrich, T.D., Sullivan, B.D., Humble, T.S.: Optimizing adiabatic quantum program compilation using a graph-theoretic framework.  Quantum Information Processing 17(5), 1–26 (2018)
  * (20) Grant, E.K., Humble, T.S.: Adiabatic quantum computing and quantum annealing.  In: Oxford Research Encyclopedia of Physics (2020)
  * (21) Hilbert, M., López, P.: The world’s technological capacity to store, communicate, and compute information.  science 332(6025), 60–65 (2011)
  * (22) Humble, T.S., Thapliyal, H., Munoz-Coreas, E., Mohiyaddin, F.A., Bennink, R.S.: Quantum computing circuits and devices.  IEEE Design & Test 36(3), 69–94 (2019)
  * (23) Jin, C., Netrapalli, P., Ge, R., Kakade, S.M., Jordan, M.I.: On nonconvex optimization for machine learning: Gradients, stochasticity, and saddle points.  Journal of the ACM (JACM) 68(2), 1–29 (2021)
  * (24) Johnson, M.W.: Next generation quantum annealing system (2019).  URL https://www.dwavesys.com/sites/default/files/ 13DWJohnson.pdf
  * (25) Klymko, C., Sullivan, B.D., Humble, T.S.: Adiabatic quantum programming: minor embedding with hard faults.  Quantum information processing 13(3), 709–729 (2014)
  * (26) Larochelle, H., Bengio, Y.: Classification using discriminative restricted boltzmann machines.  In: Proceedings of the 25th international conference on Machine learning, pp. 536–543 (2008)
  * (27) LeCun, Y.: The mnist database of handwritten digits.  http://yann. lecun. com/exdb/mnist/ (1998)
  * (28) Li, R., He, Y., Zhang, S., Qin, J., Wang, J.: Cell membrane-based nanoparticles: a new biomimetic platform for tumor diagnosis and treatment.  Acta Pharmaceutica Sinica B 8(1), 14–22 (2018)
  * (29) Li, R.Y., Di Felice, R., Rohs, R., Lidar, D.A.: Quantum annealing versus classical machine learning applied to a simplified computational biology problem.  NPJ quantum information 4(1), 1–10 (2018)
  * (30) Liu, J., Spedalieri, F.M., Yao, K.T., Potok, T.E., Schuman, C., Young, S., Patton, R., Rose, G.S., Chamka, G.: Adiabatic quantum computation applied to deep learning networks.  Entropy 20(5), 380 (2018)
  * (31) McGeoch, C.C.: Adiabatic quantum computation and quantum annealing: Theory and practice.  Synthesis Lectures on Quantum Computing 5(2), 1–93 (2014)
  * (32) Morita, S., Nishimori, H.: Mathematical foundation of quantum annealing.  Journal of Mathematical Physics 49(12), 125210 (2008)
  * (33) Mott, A., Job, J., Vlimant, J.R., Lidar, D., Spiropulu, M.: Solving a higgs optimization problem with quantum annealing for machine learning.  Nature 550(7676), 375–379 (2017)
  * (34) Mukherjee, S., Chakrabarti, B.K.: Multivariable optimization: Quantum annealing and computation.  The European Physical Journal Special Topics 224(1), 17–24 (2015)
  * (35) Neven, H., Denchev, V.S., Drew-Brook, M., Zhang, J., Macready, W.G., Rose, G.: Nips 2009 demonstration: Binary classification using hardware implementation of quantum annealing.  Quantum pp. 1–17 (2009)
  * (36) Nguyen, N.T., Kenyon, G.T.: Image classification using quantum inference on the d-wave 2x.  In: 2018 IEEE International Conference on Rebooting Computing (ICRC), pp. 1–7. IEEE (2018)
  * (37) Pudenz, K.L., Lidar, D.A.: Quantum adiabatic machine learning.  Quantum information processing 12(5), 2027–2070 (2013)
  * (38) Rahul, K., Banyal, R.K., Goswami, P., Kumar, V.: Machine learning algorithms for big data analytics.  In: Computational Methods and Data Engineering, pp. 359–367. Springer (2021)
  * (39) Ray, P., Chakrabarti, B.K., Chakrabarti, A.: Sherrington-kirkpatrick model in a transverse field: Absence of replica symmetry breaking due to quantum fluctuations.  Physical Review B 39(16), 11828 (1989)
  * (40) Terwilliger, A.M., Perdue, G.N., Isele, D., Patton, R.M., Young, S.R.: Vertex reconstruction of neutrino interactions using deep learning.  In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 2275–2281. IEEE (2017)
  * (41) Ward, A., Bambos, N.: Quantum annealing assisted deep learning for lung cancer detection
  * (42) Willsch, D., Willsch, M., De Raedt, H., Michielsen, K.: Support vector machines on the d-wave quantum annealer.  Computer Physics Communications 248, 107006 (2020)
  * (43) Wittek, P.: Quantum machine learning: what quantum computing means to data mining.  Academic Press (2014)


# Loco-Manipulation Planning for Legged Robots:
Offline and Online Strategies

Luca Rossini1,2, Paolo Ferrari3, Francesco Ruscelli1,
Arturo Laurenzi1, Nikos G. Tsagarakis1, Enrico Mingo Hoffman4 1 Humanoid and
Human Centered Mechatronics (HHCM) Lab, Istituto Italiano di Tecnologia (IIT),
Via Morego 30, 16163 Genova, Italy. E-mail<EMAIL_ADDRESS>Dipartimento
di Informatica, Bioingegneria, Robotica e Ingegneria dei Sistemi (DIBRIS),
Università di Genova, Via Opera Pia 13, 16145 Genova, Italy.3 Dipartimento di
Ingegneria Informatica, Automatica e Gestionale (DIAG), Sapienza Università di
Roma, Via Ariosto 25, 00185 Roma, Italy.4 PAL Robotics, Carrer de Pujades,
77-79, 08005 Barcelona, Spain.

###### Abstract

The deployment of robots within realistic environments requires the capability
to plan and refine the loco-manipulation trajectories on the fly to avoid
unexpected interactions with a dynamic environment. This extended abstract
provides a pipeline to offline plan a configuration space global trajectory
based on a randomized strategy, and to online locally refine it depending on
any change of the dynamic environment and the robot state. The offline planner
directly plans in the contact space, and additionally seeks for whole-body
feasible configurations compliant with the sampled contact states. The planned
trajectory, made by a discrete set of contacts and configurations, can be seen
as a graph and it can be online refined during the execution of the global
trajectory. The online refinement is carried out by a graph optimization
planner exploiting visual information. It locally acts on the global initial
plan to account for possible changes in the environment. While the offline
planner is a concluded work, tested on the humanoid COMAN+, the online local
planner is still a work-in-progress which has been tested on a reduced model
of the CENTAURO robot to avoid dynamic and static obstacles interfering with a
wheeled motion task. Both the COMAN+ and the CENTAURO robots have been
designed at the Italian Institute of Technology (IIT).

Paper Type – Recent Work [1][2]

##  I Introduction

Figure 1: The COMAN+ huamnoid robot standing up taking advantage of a vertical
wall placed in front of it.

Humanoids have the potential to accomplish tasks that requires moving in
complex and confined spaces by braking and establishing multiple contacts with
the environment. Examples of these _multi-contact loco-manipulation tasks_
include crawling, climbing a ladder and standing up using vertical wall (Fig.
1). Additionally, introducing collaborative robots alongside human beings in a
domestic or working environment requires the capability of the robot to detect
any change in the surrounding environment and react to that on the fly while
executing a task.

Figure 2: Combination of global and local planning. The global planner works
offline and provides an initial guess to the online local planner (blue dashed
line on the left). While executing the trajectory, the local planner refines
it to recover any infeasibility, generating a new feasible motion depending on
the environment state. $\mathcal{O}_{j}$ is a generic obstacle interfering
with the global plan computed offline.

To effectively fulfill these tasks, the robot must be able to autonomously
decide and execute appropriate motion that respect several crucial constraints
such as collision avoidance, balance and kinematic/dynamic limitations. This
means addressing a complete Multi-Contact Planning and Control (MCPC) problem.
In this extended abstract, we focus specifically on the planning aspects of
this problem.

Sample-based global planners revealed to be very efficient over the last
decades, and they can easily be combined with perception algorithms avoiding
any simplification of the environment [3]. In the case of dynamic environment,
one possibility would be to continuously re-plan the global trajectory every
time a difference in the environment is perceived. However, sample-based
planners revealed to be strongly influenced by the complexity of the robot,
and its compuational cost increases exponentially depending on the number of
deegres of freedom (DoFs), such as for legged robots. As a consequence of
their massive computation cost, these methodologies are prevented to be used
in real-time application (a.k.a. _curse of dimensionality_). On the other
side, optimization-based motion planners can be employed in real-time but they
can easily stuck in local minima getting impossible to find an optimal
trajectory.

A possible solution is to couple a sample-based planner, generating offline a
global initial trajectory, with a optimization-based motion planner exploiting
visual information that acts locally while the robot executes the global
trajectory (see Fig. 2). In this way, the local planner is globally guided by
the global planner avoiding any local minima, while guaranteeing a
computational efficiency that allows an online implementation during the
execution of the assigned task. These kind of problems follow a common
pipeline made by three modules: _perception_ , _planning_ and _control_
modules [4].

In this extended abstract, we present preliminary results of our offline and
online planning strategies for loco-manipulation tasks, applied to legged
robots. The perception module uses state-of-art algorithms to process a dense
point cloud generated by the Intel${}^{\text{\textregistered}}$
Realsense${}^{\text{\texttrademark}}$ D435i depth camera embedded on the
robot. The planner layer is accompanied by a control layer which moves the
robot through an impedance controller using the references computed by the
planner. At the current stage, the offline planning strategy has been tested
on the real humanoid COMAN+ using a dedicated control layer that executes the
planned trajectory once available. Recently, we started developing the online
planning strategy, which has been successfully tested on the CENTAURO robot
executing a wheeled-locomotion task. We are currently working towards the
design of a unified planning and control framework, which is paramount skill
for the success deployment and use of legged robots in challenging
environments.

##  II Offline Planning

TABLE I: Averaged performance data of the stance planner. | Task
---
| Planning
---
time (s)
| Transition
---
generation time (s)
| Number of
---
iterations
| Number of
---
vertices in ${\cal T}$
| Number of
---
stances in ${\cal S}_{\sigma}$
Ladder climbing | 43.60 | 37.64 | 1926.14 | 205.34 | 39.04
Parallel walls climbing | 175.37 | 171.01 | 1557.37 | 151.28 | 44.12
Quadrupedal walking | 62.69 | 55.99 | 2540.10 | 228.32 | 53.19
Standing up | 7.56 | 6.02 | 459.24 | 62.19 | 17.00

In this section, we consider the problem of generating appropriate motions for
a torque-controlled humanoid robot that is assigned a multi-contact loco-
manipulation task, i.e., a task that requires the robot to move within a
_static_ and _completely known_ environment by repeatedly establishing and
breaking multiple, non-coplanar contacts.

To solve such problem, we propose a offline planning scheme consisting of the
two sequential sub-planner described in the following.

The _stance planner_ is in charge of finding three sequences

$\displaystyle{\cal S}_{\sigma}=\\{\sigma_{0},\dots,\sigma_{N}\\},$
$\displaystyle{\cal
S}_{q}=\\{{\mbox{\boldmath$q$}}_{0},\dots,{\mbox{\boldmath$q$}}_{N}\\},$
$\displaystyle{\cal
S}_{W}=\\{{\mbox{\boldmath$W$}}_{c,0},\dots,{\mbox{\boldmath$W$}}_{c,N}\\},$

where ${\cal S}_{\sigma}$ is the sequence of $N+1$ stances leading to the
desired final stance $\sigma^{\rm fin}$, i.e., $\sigma_{N}=\sigma^{\rm fin}$,
while ${\cal S}_{q}$ and ${\cal S}_{W}$ are sequences of associated
transitions [5] and contact wrenches, respectively. In particular, the generic
stance ${\sigma_{j}\in\cal S}_{\sigma}$ is a set of contacts (each one
specifying the pose of a certain end-effector in contact with the
environment), transition ${\mbox{\boldmath$q$}}_{j}\in{\cal S}_{q}$ is a
feasible configuration for both $\sigma_{j-1}$ and $\sigma_{j}$, vector
${\mbox{\boldmath$W$}}_{c,{j}}\in{\cal S}_{W}$ collects the contact wrenches
that guarantee static balance at ${\mbox{\boldmath$q$}}_{j}$ using the
contacts specified by $\sigma_{j}$. To produce the sequences ${\cal
S}_{\sigma}$, ${\cal S}_{q}$ and ${\cal S}_{W}$, our stance planner applies a
RRT-like strategy; it iteratively constructs a tree $\cal T$ in the search
space, where a vertex
$v=\langle\sigma,{\mbox{\boldmath$q$}},{\mbox{\boldmath$W$}}_{c}\rangle$
consists of a stance $\sigma$, a configuration $q$, and a vector
${\mbox{\boldmath$W$}}_{c}$ of contact wrenches, while an edge going from
vertex $v$ to vertex $v^{\prime}$ indicates that
${\mbox{\boldmath$q$}}^{\prime}$ is a transition from $\sigma$ and
$\sigma^{\prime}$. Transitions are efficiently generated during the planning
process using the method presented in [6].

The _whole-body planner_ , for each pair of consecutive configurations
${\mbox{\boldmath$q$}}_{j}$ and ${\mbox{\boldmath$q$}}_{j+1}$
($j=0,\dots,N-1$) belonging to ${\cal S}_{q}$, computes a feasible whole-body
motion ${\mbox{\boldmath$s$}}_{j}$, i.e., a configuration space trajectory
that connects the two and has duration $\delta_{j}$. Then, the result of this
planner consists in a sequence of motions

${\cal
S}_{s}=\\{{\mbox{\boldmath$s$}}_{0},\dots,{\mbox{\boldmath$s$}}_{N-1}\\}.$

To produce the generic motion ${\mbox{\boldmath$s$}}_{j}\in{\cal S}_{s}$, our
whole-body planner adopts the two-stage approach presented in [7]: first, a
path consisting of a sequence ${\cal S}_{q,j}$ of feasible configurations
joining ${\mbox{\boldmath$q$}}_{j}$ to ${\mbox{\boldmath$q$}}_{j+1}$ is found
using the AtlasRRT* algorithm [8]; then, a trajectory
${\mbox{\boldmath$s$}}_{j}$ interpolating the configurations in ${\cal
S}_{q,j}$ is computed, simultaneously determining its duration $\delta_{j}$,
via optimization.

To actually execute the planned motions, we have incorporated the presented
offline planning scheme into a complete MCPC framework. This includes a
control layer that is responsible for generating online the torque commands
$\bar{\bm{\tau}}$ for the humanoid actuators to execute the planned sequence
${\cal S}_{s}$ of whole-body motions throughout the planned sequence ${\cal
S}_{\sigma}$ of stances.

##  III Online Planning

Figure 3: COMAN+ sequentially performs the quadrupedal walking and standing up
tasks.

To produce the global configuration space trajectory $\cal{S}_{s}$, the
offline planning scheme previously described assumes that the environment is
static. However, a typical working environment for legged robots, is likely to
change during the execution of an assigned task, and the robot must be able to
rapidly adapt to it to avoid unexpected collisions. To this end, we designed a
configuration space local planner based on graph optimization that allows to
store the complete trajectory and efficiently perform local modifications
whenever needed. Indeed, the online planner possibly refines the motion
${\mbox{\boldmath$s$}}_{j}$ to be executed, instead of replanning the global
motion $\mathcal{S}_{s}$, avoiding useless optimizations of future motions
that are likely to be affected by further changes of the environment.
Additionally, the solver avoids to get stuck in local minima being guided by
the global information coming from the offline planner. Using the same
constraints both for the offline and online planner, they will generate whole-
body trajectories projected on the same manifold, thus guaranteeing the output
of the offline planner to be feasible also for the online one.

The idea behind the proposed methodology is based on the representation of a
global plan as a _hyper graph_ where each vertex is a configuration, while
each hyper edge represents the error term associated to a generic constraint
connecting a certain number of vertices. The aim of this optimization is to
move the vertices inside the feasible configuration space to minimize the sum
of the considered error terms.

We discretize the trajectory ${\mbox{\boldmath$s$}}_{j}$ into a sequence
${\cal Q}_{j}$ of equispaced configurations. Then, a vertex is created in
correspondence of each configuration ${\mbox{\boldmath$q$}}\in{\cal Q}_{j}$,
while a set of edges is created to build connections between them. In
particular, the $k$-th edge will connect a sub-sequence of vertices
$\mathcal{Q}_{j,k}\subset\mathcal{Q}_{j}$. Edges involve joint limits,
velocity limits, balance, and collision avoidance as well as a cost term to
track the global plan. A continuous error function
${\mbox{\boldmath$e$}}_{k}({\cal Q}_{j,k})$ is assigned to each edge to
penalize those solutions that violate the constraints. At each iteration, the
planner locally refines the global trajectory solving the Non-Linear Least
Squared Optimization problem (NLLSO):

$\mathcal{Q}_{j}^{*}=\operatorname*{argmin}_{\mathcal{Q}_{j}}~{}{\mbox{\boldmath$F$}}(\mathcal{Q}_{j})$
(1)

where the cost function is

${\mbox{\boldmath$F$}}(\mathcal{Q}_{j})=\sum_{k=0}^{m}{\mbox{\boldmath$e$}}_{k}^{T}(\mathcal{Q}_{j,k})\Omega_{k}{\mbox{\boldmath$e$}}_{k}(\mathcal{Q}_{j,k})=\sum_{k=0}^{m}{\mbox{\boldmath$F$}}_{k}(\mathcal{Q}_{j,k})$

with $\Omega_{k}$ being a diagonal square weight matrix having the same number
of rows and columns as the associated error function
${\mbox{\boldmath$e$}}_{k}(\mathcal{Q}_{j,k})$. A solution to (1) can be found
using the Levenberg-Marquardt algorithm which finds the optimal step size
$\Delta\mathcal{Q}_{j}^{*}$

$({\mbox{\boldmath$H$}}+\lambda{\mbox{\boldmath$I$}})\Delta\mathcal{Q}^{*}_{j}=-{\mbox{\boldmath$b$}}$
(2)

with $\lambda$ being a damping factor used to control the step size
$\Delta\mathcal{Q}_{j}^{*}$ on non-linear surfaces,
${\mbox{\boldmath$b$}}=\sum_{k=0}^{m}{\mbox{\boldmath$e$}}_{k}^{T}\Omega_{k}{\mbox{\boldmath$J$}}_{k}$,
and
${\mbox{\boldmath$H$}}=\sum_{k=0}^{m}{\mbox{\boldmath$J$}}_{k}^{T}\Omega_{k}{\mbox{\boldmath$J$}}_{k}$
being the $(n\cdot N)\times(n\cdot N)$ information matrix of the system (i.e.,
Hessian) and is sparse by construction, since the objective functions involved
depend on small subsets of neighbours states. ${\mbox{\boldmath$J$}}_{k}$ is
the $(n\cdot N)\times(n\cdot N)$ Jacobian of
${\mbox{\boldmath$e$}}_{k}(\mathcal{Q}_{j})$ evaluated in $\mathcal{Q}_{j}$.
The optimal refined trajectory is then found updating the initial guess
recursively:

$\mathcal{Q}_{j}^{*}=\mathcal{Q}_{j}+\Delta\mathcal{Q}_{j}^{*}.$ (3)

The optimized sequence of configurations $\mathcal{Q}_{j}^{*}$ is then
provided to the robot and used as a new refined reference.

##  IV Results

Both the offline and online planner were tested independently. Results are
collected in the following section.

###  IV-A Offline Planning

We now present some simulation and experimental results obtained using our
offline planning scheme on COMAN+, a torque-controlled humanoid robot designed
at Istituto Italiano di Tecnologia.

Figure 4: The four considered tasks. For each of them, a snapshot from a
typical planning solution is provided.

To analyze the performance of our offline planning scheme, we performed a
simulation campaign on an Intel Core i7-7500U CPU running at 2.70 GHz. We
considered four different multi-contact loco-manipulation tasks (Fig. 4)
described in the following.

  1. 1.

Ladder climbing. The robot must climb a ladder that is located in front of it.

  2. 2.

Parallel walls climbing. The robot must climb vertically between two parallel
walls located on its left and right flanks.

  3. 3.

Quadrupedal walking. The robot must go through a narrow passage that can not
be navigated by standard bipedal walking.

  4. 4.

Standing up. The robot must stand upright, possibly exploiting a wall located
in front of it as support.

Since our planning scheme is randomized (as both the stance and whole-body
planners rely on probabilistic strategies), we performed 100 runs for each of
the four tasks. Table I collects the most significant performance data,
averaged over the $100$ runs, of the proposed stance planner. For each task,
we report the time needed by the stance planner to find a solution, the time
spent in generating transitions, the number of performed iterations, the
number of vertices in the constructed tree $\cal T$, and the number of stances
in the solution sequence ${\cal S}_{\sigma}$. To show the feasibility of the
planned motions on the real robotic platform, we performed a complete
experiment in which COMAN+ was requested to sequentially perform the
quadrupedal walking and standing up tasks in a purposely constructed scenario
(Fig.3) 111https://youtu.be/zS44CegGqow.

###  IV-B Online Planner

The proposed perception-based online planner was evaluated through a number of
simulation studies and experimental trials carried out on the CENTAURO robot
[9] reduced considering the joints of the lower body as the only active
joints, resulting in a 24 DoFs robot. The robot is equipped with two
Intel${}^{\text{\textregistered}}$ Realsense${}^{\text{\texttrademark}}$ D435i
depth cameras mounted on its pelvis and head. No other external sensor has
been used to perceive the environment.

The online planner is tested independently from the offline global planner and
a straight wheeled motion is used as the initial global plan, moving the robot
4m forward. In this scenario, the stance does not change and the robot moves
simply rolling and steering its wheels instead of taking steps. The resulting
trajectory is discretized into 50 configurations and the online planner works
locally with a moving horizon of 20 vertices, thus ignoring the furthest ones
(Fig.5(a)). However, the global trajectory can be arbitrarily long, including
a higher number of vertices without under performing the local planner, which
depends on the number of local vertices involved in the refinement only.
During the execution of the task, obstacles are dynamically introduced to
force the robot to refine the whole-body trajectory, successfully avoiding any
collision (Figs. 5(b)-5(c)-5(d)) 222https://youtu.be/qaBNzVrZtF4.

((a))

((b))

((c))

((d))

Figure 5: Screenshots from the experiment carried out with the CENTAURO robot.
Blue an green dots are the poses of the four wheels corresponding to vertices
inside and outside the moving horizon of the planner.

The computation time fluctuates around 0.07s during the execution of the
trajectory.

##  V Conclusion

This extended abstract presents our recent work on planning strategies to
allow autonomous motion generation of legged robots in cluttered environments.
Specifically, our planners explore the feasible configuration space. The
offline planner finds a feasible global plan, that can be used as an initial
guess for the online planner, which refines the whole-body trajectory whenever
required.

Experiments and simulations were carried out on the two planning modules
separately, using two different robotic platforms, i.e., the COMAN+ humanoid
and the CENTAURO robot. Preliminary results confirmed the possibility to use
such a offline and online planning strategy to generate and adjust a feasible
configuration space trajectory in general dynamic environments. Future work
will focus on merging the two presented offline and online planners into a
unified framework allowing the real-time generation of appropriate motions to
accomplish loco-manipulation tasks with legged robots.

## References

  * [1] P. Ferrari, L. Rossini, F. Ruscelli, A. Laurenzi, G. Oriolo, E. Mingo Hoffman, and N. Tsagarakis, “Multi-contact planning and control for humanoid robots: design and validation of a complete framework,” _The International Journal of Robotics Research_ , Under Review.
  * [2] L. Rossini and N. Tsagarakis, “From offline to online: A perception-based local planner for dynamic obstacle avoidance applied to humanoid robots,” in _2022 IEEE/RSJ International Conference on Intelligent Robots and Systems_ , Under Review.
  * [3] M. Görner, R. Haschke, H. Ritter, and J. Zhang, “Moveit! task constructor for task-level motion planning,” 05 2019, pp. 190–196.
  * [4] S. Haddadin, A. De Luca, and A. Albu-Schäffer, “Robot collisions: A survey on detection, isolation, and identification,” _IEEE Transactions on Robotics_ , vol. 33, no. 6, pp. 1292–1312, 2017.
  * [5] K. Hauser, T. Bretl, J.-C. Latombe, K. Harada, and B. Wilcox, “Motion planning for legged robots on varied terrain,” _The International Journal of Robotics Research_ , vol. 27, no. 11-12, pp. 1325–1349, 2008.
  * [6] L. Rossini, E. Mingo Hoffman, A. Laurenzi, and N. Tsagarakis, “NSPG: An efficient posture generator based on null-space alteration and kinetostatics constraints,” _Frontiers in Robotics and AI_ , p. 236, 2021.
  * [7] F. Ruscelli, M. P. Polverini, A. Laurenzi, E. Mingo Hoffman, and N. G. Tsagarakis, “A multi-contact motion planning and control strategy for physical interaction tasks using a humanoid robot,” in _2020 IEEE/RSJ International Conference on Intelligent Robots and Systems_ , 2020, pp. 3869–3876.
  * [8] L. Jaillet and J. M. Porta, “Asymptotically-optimal path planning on manifolds,” in _Robotics: Science and Systems_ , 2012, pp. 1–8.
  * [9] N. Kashiri, L. Baccelliere, L. Muratore, A. Laurenzi, Z. Ren, E. M. Hoffman, M. Kamedula, G. F. Rigano, J. Malzahn, S. Cordasco, P. Guria, A. Margan, and N. G. Tsagarakis, “Centauro: A hybrid locomotion and high power resilient manipulation platform,” _IEEE Robotics and Automation Letters (RAL)_ , vol. 4, no. 2, pp. 1595–1602, 2019. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/8630605


# Electronic and Topological Properties of a Topological Insulator Thin Film
Sandwiched between Ferromagnetic Insulators

P. Pigoń1, A. Dyrdał2 1AGH University of Krakow, Faculty of Physics and
Applied Computer Science, al. Adama Mickiewicza 30, 30-059 Kraków, Poland
2Department of Mesoscopic Physics, ISQI, Faculty of Physics, Adam Mickiewicz
University, ul. Uniwersytetu Poznańskiego 2, 61-614 Poznań, Poland

###### Abstract

We consider a thin film of a topological insulator (TI) sandwiched between two
ferromagnetic (FM) layers. The system is additionally under an external gate
voltage. The surface electron states of TI are magnetized due to the magnetic
proximity effect to the ferromagnetic layers. The magnetization of
ferromagnetic layers can be changed by applying an external magnetic field or
by varying thickness of the topological insulator (owing to the interlayer
exchange coupling). The change in the magnetic configuration of the system
affects the transport properties of the surface electronic states. Using the
Green function formalism, we calculate spin polarization, anomalous Hall
effect, and magnetoresistance of the system. We show, among others, that by
tuning the gate voltage and magnetizations of the top and bottom FM layers,
one can observe the topological transition to the anomalous quantum Hall
state.

##  I Introduction

The first papers on topological insulators were published in the last century,
with the observation of quantum Hall effect [1]. A few years later, Haldane
proposed a mathematical model of two–dimensional topological insulator [2].
This model was then extended by Kane and Mele [3, 4], who included spin in the
model, making a more complete picture of topological insulators. These
materials act like trivial insulators in bulk, but they are conducting at the
edges [5], where the topologically protected surface states appear [6]. The
importance of topological insulators results from their potential application
in spintronics devices or in quantum information processing – mainly because
of dissipationless transport and/or possibility to control electron’s spin.
Nowadays, they are used in electronic and semiconducting devices, e.g., in
photodetectors, magnetic devices, and FET transistors [7]. A special class of
topological insulators is that of magnetic topological insulators, where the
quantum anomalous Hall effect can be observed. [8]. There are four ways for a
topological insulator to become a magnetic topological insulator [9]. The most
popular method is doping the bulk of topological insulators with transition
metal elements, e.g., Bi2Te3, Bi2Se3, and Sb2Te3 doped with Cr or Fe [10].
But, controlling the distribution and magnetic order of dopants is rather
difficult task [11]. These materials can have many applications, e.g. in
information storage and in dissipationless spin and current transport for low-
power-consuming electronics [8, 12].

Figure 1: A thin film of a topological insulator sandwiched between two
ferromagnets. A red color marks the bottom layer, and the top layer is in
blue. The layers are in the $x-y$ plane, while the axis $z$ is normal to the
layers.

In this work, we consider a thin film of a topological insulator sandwiched
between two ferromagnetic insulators. In such a system, the topological
surface states are affected by the magnetic proximity effect. The magnetic
layers are magnetized parallel either along or opposite to the $z$–axis (which
is normal to the plane of the heterostructure). However, this magnetization
orientation can be changed by an external magnetic field.

Figure 2: Band structure and the corresponding Berry curvatures for a fixed
magnetization of the bottom layer, $m_{z}^{t}=0.1$ eV and asymmetric potential
$V_{AS}=0.05eV$. (a) and (b) correspond to the parallel magnetic
configuration, i.e., $m_{z}^{t}=0.1$ eV, while (c) and (d) to the antiparallel
magnetic configuration, i.e., $m_{z}^{t}=-0.1$ eV.

Here we present the topological and transport properties of such a hybrid
structure. In principle we focus on the anomalous Hall effect and on current-
induced spin polarization (a.k. Edelstein effect). The anomalous Hall
conductivity will give us a direct information about the topological
properties of electronic surface states of TI. In turn, the current-induced
spin polarization is one of possible spin-to-charge interconversion effects,
that can be responsible for various magnetotransport phenomena and for spin-
orbit torque in the considered systems.

##  II Model and method

We consider a thin film of the topological insulator sandwiched between two
ferromagnetic insulators with easy axis magnetic anisotropy along the
z-direction (Fig. 1). The magnetization of the bottom layer is fixed and
oriented parallel to the z-axis, whereas the magnetization of the top layer is
free to rotate (in an external magnetic field or due to interlayer exchange
coupling) from antiparallel to parallel (or vice-versa) orientation to the
z-axis.

The Hamiltonian $\hat{H}$ of the electronic surface states of 3D TI, written
in the so-called top–bottom representation, takes the form [13]

$\displaystyle\hat{H}=v\hat{\tau}_{z}\otimes(k_{y}\sigma_{x}-k_{x}\sigma_{y})+V_{\small{AS}}\hat{\tau}_{z}\otimes\hat{\sigma}_{0}$
$\displaystyle+(m_{z}^{t}\hat{\tau}_{+}+m_{z}^{b}\hat{\tau}_{-})\otimes\hat{\sigma}_{z},$
(1)

where $\hat{\sigma}_{i}$, $\hat{\tau}_{i}$ ($i=x,y,z$) are Pauli matrices
acting in the spin and layer subspace, respectively,
$\tau_{\pm}=\frac{1}{2}(\tau_{0}\pm\tau_{z})$, $\textbf{k}=\\{k_{x},k_{y}\\}$
is the two-dimensional wave vector, $V_{AS}$ is the asymmetric part of the
gate voltage, while $m_{z}^{t/b}$ denote magnetizations of the top/bottom
layer and $\textbf{n}_{z}=(0,0,1)$. Here we assume that the layer of TI is
thick enough to neglect the effect of hybridization between top and bottom
surface states (the effect of hybridization as well as other orientatations of
the magnetizations will be published elsewhere).

To characterise the topological properties of the system, we calculate the
Berry curvature for each energy band based on the following equation:

$\Omega_{n}=i\nabla_{\textbf{k}}\times\left\langle\psi_{n}\right|\nabla_{\textbf{k}}\left|\psi_{n}\right\rangle,$
(2)

where $\left|\psi_{n}\right\rangle$ is the $n$-th eigenstate of the
Hamiltonian $\hat{H}$.

Figure 2 presents the band structure and the corresponding Berry curvatures
for all electronic bands for two specific cases, i.e., for parallel and
antiparalell configurations of the magnetic moments of the two FM layers (in
the following referred to shortly as parallel and antiparallel magnetic
configurations). In both cases one can see symmetric valence and conduction
bands formed by the surface states. Importantly, due to the magnetization and
gate voltage, there is a gap in the surface states. For the fixed
magnetization of the top and bottom layers, the width of the energy gap can
become shrunken, closed, and then reopened depending on the value of $V_{AS}$.
Moreover, depending on the magnetic configuration, the sign of the integrated
over the angle Berry curvature changes. In Fig. 2 one can see that the Berry
curvatures (integrated over the angle) for bands, $E_{1}$, and $E_{3}$, are
the same in antiparallel magnetic configuration and opposite in the parallel
one. This, in turn, leads to the two different topological phases, with Chern
number equal $\pm 2$ or $0$, respectively. The topological phase transition
will be seen in the anomalous Hall conductivity, that is directly related to
the Berry curvature [14, 15]:

$\sigma_{xy}=\frac{e^{2}}{\hbar}\sum_{{\small{n=1-4}}}\int\frac{d^{2}\mathbf{k}}{(2\pi)^{2}}\Omega_{n}f(E_{n})$
(3)

Note that the above expression gives only the intrinsic contribution to the
anomalous Hall effect. The scattering effects are neglected here, as we
consider quasi-ballistic limit and focuse on the topological properties of the
system, i.e., when chemical potential is located inside the energy gap.

Figure 3: Density plots of the anomalous Hall conductivity as a function of
asymmetric potential $V_{AS}$ and chemical potential $\mu$ (a) for parallel
magnetization direction ($m_{z}^{t}=m_{z}^{b}=0.1$ eV) and (b) for parallel
magnetization direction ($m_{z}^{t}=-0.1$ eV and $m_{z}^{b}=0.1$ eV) and as a
function of (c) magnetization of the top layer $m_{z}^{t}=0.1$ eV and chemical
potential $\mu$ with the fixed magnetization of bottom layers $m_{z}^{b}=0.1$
eV, $V_{AS}=5$ meV (d) magnetization of the layers with fixed $\mu=0$ and
$V_{AS}=5$ meV.

In turn, for chemical potential located in valence or conduction bands, we
focus on the current-induced spin polarization [16, 17]

$S_{y}=\frac{e\hbar}{2\pi}E_{x}\int\frac{\mathrm{d}^{2}\textbf{k}}{(2\pi)^{2}}\mathrm{Tr}\left\\{\hat{s}_{y}G_{k}^{R}\hat{v}_{x}G_{k}^{A}\right\\},$
(4)

where $\hat{s}_{y}=\frac{\hbar}{2}\tau_{0}\otimes\sigma_{y}$ is the spin
operator, $\hat{v}_{i}=\frac{1}{\hbar}\frac{\partial\hat{H}}{\partial k_{i}}$
is the velocity operator, and $G_{k}^{R/A}=\left[\left(\mu\pm
i\Gamma\right)\hat{\sigma}_{0}\otimes\hat{\tau}_{0}-\hat{H}\right]^{-1}$
defines the retarded (R) and advanced (A) Green function, respectively, with
$\Gamma$ denoting the relaxation rate. Note that the surfaces of TI are
spatially separated. Accordingly, apart from the spin operator $\hat{s}_{y}$,
that gives the information about the net current-induced spin polarization in
the system, one can define the spin polarization induced at a specific
surface. The spin polarization at the top/bottom surface is defined by the
operator:

$\hat{s}_{y}^{t/b}=\frac{\hbar}{2}\tau_{\pm}\otimes\sigma_{y}$ (5)

and therefore

$S_{y}^{t/b}=\frac{e\hbar}{2\pi}E_{x}\int\frac{\mathrm{d}^{2}\textbf{k}}{(2\pi)^{2}}\mathrm{Tr}\left\\{\hat{s}_{y}^{t/b}G_{k}^{R}\hat{v}_{x}G_{k}^{A}\right\\}.$
(6)

##  III Results

###  III.1 Anomalous Hall effect

Figure 3, presents the anomalous Hall conductivity plotted as a function of
the certain parameters defining Hamiltonian II. The most intriguing phenomena
happen when the chemical potential lies in the energy gap. The surface states
of TI in our model are gapped due to the interplay of gate voltage, $V_{AS}$,
and magnetizations of the top and bottom layers, $m_{z}^{t/b}$.

Figure 3(a) shows the transverse conductivity as a function of the chemical
potential $\mu$ and the asymmetric potential $V_{AS}$, for fixed values of the
magnetization of both layers $m_{z}^{t}=m_{z}^{b}=0.1$ eV (parallel magnetic
configuration). In the middle of the plot, there is a region of the diamond
shape, where the conductance is quantized, $\sigma_{xy}=-2$ $e^{2}/h$. This
region determines the range of the energy gap. With increasing absolute value
of the asymmetric potential, $|V_{AS}|$, this region becomes narrower. In
addition, when the chemical potential $\mu$ is far from the bandgap, the Hall
conductance diminishes. Figure 3(b) shows the conductivity as a function of
the chemical potential, $\mu$, and the asymmetric potential, $V_{AS}$. The
magnitude and direction of the magnetization of the top layer are changed in
comparison to those in Fig. 3(a), i.e., $m_{z}^{t}=-0.05$ eV. This corresponds
to antiparallel magnetic configuration, with asymmetric magnitudes of the
magnetizations. Now, one can distinguish four particular regions. The dark
areas, where the conductance reaches a maximum negative value (bands $E_{1}$
and $E_{3}$ are fully occupied). Thus, by changing the asymmetric potential,
one can control the anomalous Hall conductance of the system. However, for
this magnetic configuration, there is no area of quantized conductance.

In turn, in Fig. 3(c) we show the conductivity of the system as a function of
the chemical potential, $\mu$, magnetization of the top layer, $m_{z}^{t}$,
and for the fixed values of the magnetization of the bottom layer
$m_{z}^{t}=0.1$ eV and for $V_{AS}=5$ meV. When $m_{z}^{t}$ changes from
$m_{z}^{t}=-0.1$ eV to $m_{z}^{t}=0.1$ eV, the magnetic configuration of the
system changes from the antiparallel magnetic configuration to the parallel
one. Here the changes of amplitude and orientation of $m_{z}^{t}$ determine
the width of the energy band gap and the topological phase transition. For
$m_{z}^{t}<0$ (the antiparallel magnetic configuration) and chemical potential
located in the gap, one observes the trivial phase with zero anomalous Hall
conductivity (yellow triangle). In turn, for $m_{z}^{t}>0$ (the parallel
magnetic configuration) and chemical potential in the gap, the quantum
anomalous Hall phase occurs (dark triangle). The possibility of tuning the gap
width and topological phase behaviour externally seems to be promising for
application in spintronics and low-energy consumption electronics.

Figure 4: The current induced spin polarization as a function of a chemical
potential $\mu$ and asymmetric potential $V_{AS}$ with parallel (solid lines)
and antiparallel (dashed lines) direction of magnetization for (a) the top
layer, (b) the bottom layer and (c) the total system.

Figure 3(d) presents the variation of the transverse conductance with
magnetizations of both layers, for fixed values of the chemical potential,
i.e., $\mu=0$, and $V_{AS}=5$ meV. When the magnetic configuration is
parallel, the system is in the quantum anomalous Hall state with
$\sigma_{xy}=\mp 2$ $e^{2}/h$ (black and yellow areas in Fig 3(d)). The
negative sign of anomalous Hall conductance corresponds to the magnetizations
along $z$-axis while positive sign of anomalous Hall conductivity refers to
the magnetizations oriented antiparallel to the $z$-axis. For the antiparallel
configuration of magnetic layers, the system is a trivial insulator state (the
purple areas in Fig 3(d)), where the anomalous Hall conductance is zero.

Based on the above results one can clearly see the characteristic areas where
the conductance is quantized. These areas correspond to Chern number equal
$\pm 2$ depending on whether the magnetic configuration is parallel or
antiparallel. Accordingly, when chemical potential lies in the energy gap
(fully occupied valence bands) one can observe a topological phase transition
to the quantum anomalous Hall insulator state - the state corresponding to the
zero longitudinal charge current and fully quantized Hall current.

###  III.2 Current–induced spin polarization

Now, we consider the spin-to-charge interconversion effects in the system
under consideration, i.e. the spin polarization induced by a charge current.
This phenomenon appears as a result of spin-orbit coupling in the system (in
the TI layer). In Fig. 4, we present the current-induced spin polarization due
to the electric current flowing along the $x$-axis. The nonequilibrium
polarization is shown here as a function of $\mu$ and $V_{AS}$ for a fixed
magnetization of the bottom layer, $m_{z}^{b}=0.1$ eV. Here the orientation of
magnetization in the top layer doesn’t affect the results (solid and dashed
lines overlap in Fig. 4). Figure 4(a) shows the spin polarization at the top
surface of TI ,$S_{yx}^{t}$, while Fig. 4(b) presents the spin polarization at
the bottom surface of TI, $S_{yx}^{b}$. The neat spin polarization of the
system, i.e. sum of the polarizations of top and bottom surface is shown in
Fig. 4(c).

As only states that are at the Fermi level contribute to the polarization, and
there are no states at the Fermi level in the energy gap region, then the spin
polarization vanishes there, as clearly visible in the figures for chemical
potentials $|\mu|<0.2$ eV. As one can note, the spin polarizations of the
topological electron states at the top and bottom layers have opposite
orientations. Nevertheless, there is some asymmetry between top and bottom
polarizations, what can be seen in Fig 4(a) and 4(b). As a result, the net
spin polarization of the system is nonzero, as shown explicitly in Fig. 4(c).
One should emphasize that pairs of the plots: $S_{yx}^{b},\ S_{xy}^{t}$,
$S_{yx}^{t},\ S_{xy}^{b}$ and $S_{yx},\ S_{xy}$ are its mirror reflection
regarding to the line passing through the point $\mu=0$.

##  IV Discussion

In this work, we considered the electronic and topological properties of the
thin film of a topological insulator sandwiched between two layers of a
ferromagnetic insulator. Each layer can be characterized by magnetization
oriented along the $z$-axis, and we assume that an external magnetic field can
control the magnetic configuration of the magnetic layers. The presented
results show that magnetization direction has a significant impact on the
transport properties of the system. Firstly, for the parallel orientation of
the layer magnetizations, two of the Berry curvatures are positive, and two
are negative. If we switch the magnetization direction of the top layer, four
of Berry curvatures are negative. This results in a change of Chern numbers of
each band, and results in the change of the behaviour of anomalous Hall
conductivity. Accordingly, depending on the system parameters, $m_{z}^{t/b}$
and $V_{A}S$, one can switch our system between a trivial insulator state
(zero Hall conductance in the energy gap) and a topological insulator state
(quantized anomalous conductance in the energy gap). Moreover, one can induce
spin polarization in the system through an external electric field. We showed
that although the chemical potential is in the gap, the current-induced spin
polarization appears at each interface and is oriented in a particular
direction.

## Acknowledgement

This work has been supported by the National Science Centre in Poland under
the project Sonata-14 no. 2018/31/D/ST3/02351. P. Pigoń acknowledges B. Spisak
for fruitful discussions.

## References

  * Klitzing  _et al._ [1980] K. v. Klitzing, G. Dorda, and M. Pepper, New method for high-accuracy determination of the fine-structure constant based on quantized hall resistance, Phys. Rev. Lett. 45, 494 (1980).
  * Haldane [1988] F. D. M. Haldane, Model for a quantum hall effect without landau levels: Condensed-matter realization of the ”parity anomaly”, Phys. Rev. Lett. 61, 2015 (1988).
  * Kane and Mele [2005a] C. L. Kane and E. J. Mele, ${Z}_{2}$ topological order and the quantum spin hall effect, Phys. Rev. Lett. 95, 146802 (2005a).
  * Kane and Mele [2005b] C. L. Kane and E. J. Mele, Quantum spin hall effect in graphene, Phys. Rev. Lett. 95, 226801 (2005b).
  * Fruchart and Carpentier [2013] M. Fruchart and D. Carpentier, An introduction to topological insulators, C. R. Phys 14, 779 (2013).
  * Qi and Zhang [2010] X.-L. Qi and S.-C. Zhang, The quantum spin hall effect and topological insulators, Phys. Today 63, 33 (2010).
  * Tian  _et al._ [2017] W. Tian, W. Yu, J. Shi, and Y. Wang, The property, preparation and application of topological insulators: a review, Materials 10, 814 (2017).
  * Chang  _et al._ [2013] C.-Z. Chang, J. Zhang, X. Feng, J. Shen, Z. Zhang, M. Guo, K. Li, Y. Ou, P. Wei, L.-L. Wang, Z.-Q. Ji, Y. Feng, S. Ji, X. Chen, J. Jia, X. Dai, Z. Fang, S.-C. Zhang, K. He, Y. Wang, L. Lu, X.-C. Ma, and Q.-K. Xue, Experimental observation of the quantum anomalous hall effect in a magnetic topological insulator, Science 340, 167 (2013).
  * Bernevig  _et al._ [2022] B. A. Bernevig, C. Felser, and H. Beidenkopf, Progress and prospects in magnetic topological materials, Nature 603, 41 (2022).
  * Yu  _et al._ [2010] R. Yu, W. Zhang, H.-J. Zhang, S.-C. Zhang, X. Dai, and Z. Fang, Quantized anomalous hall effect in magnetic topological insulators, Science 329, 61 (2010).
  * Hou  _et al._ [2019] Y. Hou, J. Kim, and R. Wu, Magnetizing topological surface states of bi2se3 with a cri3 monolayer, Sci. Adv. 5, eaaw1874 (2019).
  * Lu  _et al._ [2013] H.-Z. Lu, A. Zhao, and S.-Q. Shen, Quantum transport in magnetic topological insulator thin films, Phys. Rev. Lett. 111, 146802 (2013).
  * Litvinov [2020] V. Litvinov,  _Magnetism in Topological Insulators_ (Springer, 2020).
  * Dyrdał and Barnaś [2017] A. Dyrdał and J. Barnaś, Anomalous, spin, and valley hall effects in graphene deposited on ferromagnetic substrates, 2D Materials 4, 034003 (2017).
  * Xiao  _et al._ [2010] D. Xiao, M.-C. Chang, and Q. Niu, Berry phase effects on electronic properties, Rev. Mod. Phys. 82, 1959 (2010).
  * Dyrdał and Barnaś [2015] A. Dyrdał and J. Barnaś, Current-induced spin polarization and spin-orbit torque in graphene, Phys. Rev. B 92, 165404 (2015).
  * Mahan [2000] G. D. Mahan,  _Many-Particle Physics_ (Springer New York, NY, 2000).
  * Fortunato and Barthelemy [2007] S. Fortunato and M. Barthelemy, Resolution limit in community detection, Proc. Natl. Acad. Sci. U. S. A. 104, 36 (2007).
  * Ando [2013] Y. Ando, Topological insulator materials, J. Phys. Soc. Jpn 82, 102001 (2013).
  * Tokura  _et al._ [2019] Y. Tokura, K. Yasuda, and A. Tsukazaki, Magnetic topological insulators, Nat. Rev. Phys. 1, 126 (2019).
  * Dyrdał and Barnaś [2017] A. Dyrdał and J. Barnaś, Anomalous, spin, and valley hall effects in graphene deposited on ferromagnetic substrates, 2D Mater. 4, 034003 (2017).
  * Liu  _et al._ [2016] C.-X. Liu, S.-C. Zhang, and X.-L. Qi, The quantum anomalous hall effect: Theory and experiment, Annu. Rev. Condens. Matter Phys. 7, 301 (2016).