# A Novel Framework Based on Variational Quantum Algorithms: Revolutionizing
Image Classification

Yixiong Chen  Email<EMAIL_ADDRESS>
###### Abstract

Image classification is a crucial task in machine learning. In recent years,
this field has witnessed rapid development, with a series of image
classification models being proposed and achieving state-of-the-art (SOTA)
results. Parallelly, with the advancement of quantum technologies, quantum
machine learning has attracted a lot of interest. In particular, a class of
algorithms known as variational quantum algorithms (VQAs) has been extensively
studied to improve the performance of classical machine learning. In this
paper, we propose a novel image classification framework using VQAs. The major
advantage of our framework is the elimination of the need for the global
pooling operation typically performed at the end of classical image
classification models. While global pooling can help to reduce computational
complexity, it often results in a significant loss of information. By removing
the global pooling module before the output layer, our approach allows for
effectively capturing more discriminative features and fine-grained details in
images, leading to improved classification performance. Moreover, employing
VQAs enables our framework to have fewer parameters compared to the classical
framework, even in the absence of global pooling, which makes it more
advantageous in preventing overfitting. We apply our method to different SOTA
image classification models and demonstrate the superiority of the proposed
quantum architecture over its classical counterpart through a series of
experiments on public datasets.

##  1 Introduction

In light of recent breakthroughs in quantum technologies, particularly the
availability of noisy intermediate-scale quantum (NISQ) processors
[cheng2023noisy], the field of quantum machine learning has attracted growing
concerns and triggered an enormous amount of work [schuld2015introduction,
biamonte2017quantum, dunjko2018machine, zhang2020recent,
cerezo2022challenges]. Quantum machine learning is an interdisciplinary field
that combines concepts from quantum physics and machine learning to develop
innovative algorithms and models. By harnessing the power of quantum
computing, quantum machine learning aims to improve the performance of machine
learning algorithms. While still an emerging discipline, quantum machine
learning has already demonstrated promising quantum extensions to classical
machine learning techniques including support vector machines
[varatharajan2018big], clustering [kerenidis2018q, otterbach2017unsupervised],
and principal component analysis [article].

Image classification is a fundamental task in computer vision that involves
categorizing images into predefined classes or categories. Over the past few
years, this field has witnessed rapid progress. Convolutional neural networks
(CNNs) emerged as a breakthrough technique, significantly improving image
classification accuracy. Various CNN architectures [krizhevsky2012imagenet,
lin2013network, simonyan2014very, szegedy2015going, he2016deep,
huang2017densely, tan2019efficientnet] have progressively advanced state-of-
the-art (SOTA) results on benchmark datasets such as ImageNet
[deng2009imagenet]. Meanwhile, self-attention models like Transformers
[vaswani2017attention] in natural language processing have been introduced to
computer vision. The corresponding transformer-based vision models, such as
Vision Transformer (ViT) [dosovitskiy2010image] and Swin Transformer
[liu2021swin], can achieve compelling results on image recognition and even
outperform CNNs on some tasks. More recently, hybrid approaches combining CNN
and transformer modules are also gaining attention. These hybrid CNN-
transformer models, including CoAtNet [dai2021coatnet] and MaxViT
[tu2022maxvit], have been demonstrated to improve the generalizability of pure
transformer-based models and achieve excellent performance in image
classification without being pre-trained on large-scale image datasets.

In most image classification architectures, the input images are processed by
a backbone model, also referred to as a feature extractor, which extracts
relevant features from the input. This backbone model generally consists of a
series of convolutional layers, transformer-based layers, or a combination of
both. The resulting feature maps are typically fed into a global pooling layer
followed by fully connected layers and classifier. Global pooling is a
commonly employed technique in the field of image recognition [lin2013network,
simonyan2014very, szegedy2015going, he2016deep, huang2017densely,
tan2019efficientnet, liu2021swin, dai2021coatnet, tu2022maxvit]. It aggregates
the spatial information of feature maps across the entire image into a single
vector representation. The advantage of global pooling lies in its ability to
capture the overall context and semantic information of an image, resulting in
a compact and fixed-length representation that can be easily fed into
subsequent layers for classification tasks. Global pooling also helps to
mitigate the spatial variance of features, enabling the model to achieve
translation invariance and robustness to object deformations. Moreover, it
reduces the dimensionality of the feature maps, leading to computational
efficiency and alleviating the risk of overfitting.

While global pooling provides some benefits for image classification, it has
several limitations. Collapsing an entire feature map into a single vector
results in the complete loss of fine-grained spatial information. The compact
fixed-length representation has also restricted expressive capacity compared
to a fully-connected layer operating on spatial data, which may hinder
capturing complex spatial relationships. This can be disadvantageous for tasks
that require precise object localization, discriminative information or
detailed spatial information. Furthermore, uninformative features
corresponding to background or irrelevant regions can dominate the global
summary statistic and render it unresponsive to salient features.
Additionally, since global pooling treats the entire image equally, it may not
effectively capture local variations or small-scale patterns that are
important for accurate recognition.

Researchers have explored different strategies to enhance the performance of
global pooling and mitigate its drawbacks. Firstly, several variants of global
pooling are proposed to provide a trade-off between the global average pooling
and global max pooling with a nonlinear log-average-exp (LAE)
[zhang2018alphamex, lowe2021logavgexp] or log-sum-exp (LSE) [sun2016pronet]
function, which helps extract features more effectively. Another approach is
to leverage pyramid pooling [he2015spatial, jose2018pyramid, qi2018concentric,
lin2019fourier] as an alternative to global pooling. Pyramid pooling methods
divide the image into multiple regions of different scales and extracts
features from each region independently. This allows for the capture of multi-
scale information and the preservation of spatial details. Moreover, attention
mechanisms have also been integrated with global pooling to improve its
discriminative power [qiu2018global, zhang2020global, behera2021context].
These mechanisms selectively emphasize salient regions or features while
suppressing irrelevant ones, enhancing the model’s ability to capture
important information. In addition, spatial transformer networks (STNs)
[jaderberg2015spatial] have been proposed to enable adaptive spatial
transformations prior to pooling. STNs apply learned transformations to align
features, enhancing the localization accuracy of global pooling. Furthermore,
the method of global second-order pooling [li2018towards, gao2019global,
wang2020deep, song2022eigenvalues] is also exploited for capturing more
discriminative image representations. This approach calculates covariance
matrices to obtain higher-order statistical information between features.

In contrast to the above methods, our work employs quantum machine learning
algorithms to address the issue of global pooling for the task of image
classification. Quantum machine learning has the potential to revolutionize
the area of image classification as quantum algorithms can process large
datasets of images more efficiently than classical algorithms, leading to
faster and more accurate classification [dang2018image, kerenidis2019quantum,
mari2020transfer, henderson2020quanvolutional, li2020quantum, liu2021hybrid,
henderson2021methods, houssein2021hybrid, sagingalieva2022hyperparameter,
hur2022quantum, senokosov2023quantum, melnikov2023quantum]. In this paper, we
propose a novel image classification framework based on variational quantum
algorithms (VQAs) [mcclean2016theory, bharti2022noisy, cerezo2021variational]
in the context of quantum machine learning. VQAs are a class of quantum
algorithms that utilize classical computers to optimize a parameterized
quantum circuit. Due to their hybrid quantum-classical approach that has
potential noise resilience [mcclean2016theory], VQAs have been considered as
one of the leading candidates to achieve application-oriented quantum
computational advantage on NIhyperrefSQ devices.

The proposed framework differs from classical architectures primarily in that
global pooling operations at the end of the network are no longer required.
This is due to the introduction of the variational quantum circuits (VQCs)
[cerezo2021variational, mitarai2018quantum, farhi2018classification] which
possess remarkable data storage capacity. Consequently, our framework can
fully utilize the feature maps extracted via the backbone model for image
classification, which is particularly advantageous for tasks requiring fine-
grained features. Moreover, despite the removal of the global pooling layer,
our framework does not suffer from the issue of excessive model parameters and
overfitting, as commonly encountered in classical deep learning frameworks. In
contrast, our framework actually has fewer parameters, thanks to the robust
data storage and expressive capabilities of the variational quantum circuits.
Furthermore, our framework exhibits high flexibility and adaptability, making
it applicable to most of existing deep learning models for image
classification.

In summary, the contributions of our work are

  * •

We propose a novel image classification framework based on variational quantum
algorithms, which does not require global pooling before the classification
layer and enables models to capture more discriminative details from images
using fewer parameters. To the best of our knowledge, our work is the first
attempt to tackle the problem of global pooling in the classical image
classification framework by employing quantum machine learning algorithms.

  * •

We evaluate our proposed framework on four challenging datasets, namely
Croatian Fish [jager2015croatian], Aircraft [maji2013fine], Breast Ultrasound
image [al2020dataset], and Apples or Tomatoes [AOT]. Experimental results
demonstrate that our framework significantly outperform the classical
framework.

##  2 Method

###  2.1 Preliminaries

####  2.1.1 Global Pooling

Pooling is a common operation in image classification models that plays a
vital role in reducing spatial dimensions and capturing invariant features. It
aims to down sampling feature maps by summarizing the presence of features in
patches of the feature map. Two commonly used pooling methods are average
pooling and max pooling. Average pooling calculates the average value of the
feature within each pooling region while max pooling selects the maximum value
of the feature. Examples of these two pooling approaches are shown in Fig. 1.

Figure 1: Examples of two types of pooling operations. (a) Average pooling
with a pooling area of size 2x2 and stride of 2. (b) Max pooling with a
pooling area of size 2x2 and stride of 2.

Global Pooling is a variant of pooling operation where the kernel size is
equal to the size of the input feature map. Essentially, it computes a single
value for each feature channel by summarizing the entire feature map. The two
most common types of global pooling are global average pooling (GAP) and
global max pooling (GMP), as illustrated in Fig. 2, which compute the average
value and maximum value of each feature channel respectively. Global pooling
is often used as a bridge between the backbone model and fully connected
layers in classical image classification architectures. They help to reduce
overfitting by minimizing the total number of parameters in the model.
However, they can also lead to substantial loss of spatial information and
critical details by transforming an entire feature map to a single value. This
prevents the model from capturing local details effectively and thus limits
the model performance.

Figure 2: Examples of two types of global pooling operations. (a) Global
average pooling. (b) Global max pooling.

####  2.1.2 Quantum Basics

Here, we provide a brief introduction to some fundamental concepts of quantum
computing that are essential for understanding this paper.

  * •

Qubit and Quantum State: The qubit is the basic unit of information in quantum
computing. Unlike a classical bit which has a value of either 0 or 1, a qubit
can exist in a combination of the two states. This is called superposition.
The quantum state of a qubit is described by a vector in a two-dimensional
Hilbert space, commonly represented as

$\displaystyle|\psi\rangle=\alpha|0\rangle+\beta|1\rangle,$ (1)

where $|0\rangle$ and $|1\rangle$ are two computational basis states, and
$\alpha$ and $\beta$ are complex probability amplitude corresponding to each
basis state, satisfying

$\displaystyle|\alpha|^{2}+|\beta|^{2}=1.$

  * •

Quantum gate and Quantum circuit: Quantum gates are unitary operators or
equivalently unitary matrices which are applied to states of qubits and
perform unitary transformation. Common single-qubit quantum gates include the
Hadamard gate, Pauli gates and their corresponding rotation gates. In
particular, single-qubit rotation gates are parametric gates that depend on
one parameter and allow for a more flexible range of operations. These
rotation gates, denoted by $R_{i}(\theta)$, perform a rotation around the $i$
axis on the Bloch sphere [bloch1946nuclear] by an angle $\theta$, where
$i\in\\{x,y,z\\}$ and $\theta\in[0,2\pi)$. Other examples of common gates
include two-qubit gates such as the CNOT gate which can induce quantum
entanglement between qubits. A summary of these quantum gates can be found in
Table LABEL:gates.