# CD-COCO: A Versatile Complex Distorted COCO Database for Scene-Context-Aware
Computer Vision

Ayman Beghdadi  IBISC Lab, Paris-Saclay University
Evry, France
<EMAIL_ADDRESS>Azeddine Beghdadi  L2TI Lab, Sorbonne Paris
Nord University
Villetaneuse, France
<EMAIL_ADDRESS>Malik Mallem  IBISC Lab, Paris-Saclay
University
Evry, France
<EMAIL_ADDRESS>Lotfi Beji  IBISC Lab, Paris-Saclay University
Evry, France
<EMAIL_ADDRESS>Faouzi Alaya Cheikh  Norwegian University of Science
and Technology (NTNU)
Gjovik, Norway
<EMAIL_ADDRESS>
###### Abstract

The recent development of deep learning methods applied to vision has enabled
their increasing integration into real-world applications to perform complex
Computer Vision (CV) tasks. However, image acquisition conditions have a major
impact on the performance of high-level image processing. A possible solution
to overcome these limitations is to artificially augment the training
databases or to design deep learning models that are robust to signal
distortions. We opt here for the first solution by enriching the database with
complex and realistic distortions which were ignored until now in the existing
databases. To this end, we built a new versatile database derived from the
well-known MS-COCO database to which we applied local and global photo-
realistic distortions. These new local distortions are generated by
considering the scene context of the images that guarantees a high level of
photo-realism. Distortions are generated by exploiting the depth information
of the objects in the scene as well as their semantics. This guarantees a high
level of photo-realism and allows to explore real scenarios ignored in
conventional databases dedicated to various CV applications. Our versatile
database offers an efficient solution to improve the robustness of various CV
tasks such as Object Detection (OD), scene segmentation, and distortion-type
classification methods. The image database, scene classification index, and
distortion generation codes are publicly available
111https://github.com/Aymanbegh/CD-COCO.

###### Index Terms:

Dataset, Deep learning, Depth, Distortion, Object detection, Scene analysis,
Segmentation

##  I Introduction

The interest in making databases available to the scientific community is
becoming more and more important with the development of data-driven
approaches, and in particular those based on deep neural network
architectures. Few studies have been conducted to analyse the relevance and
reliability of databases in the field of CV. However, we can point out some
interesting studies where some attributes and descriptors have been introduced
to measure the representativeness and the richness of the databases dedicated
to the evaluation of image and video quality metrics [1, 2]. To the best of
our knowledge, there have been no similar efforts to design realistic
databases dedicated to improve methods developed for solving problems in the
field of CV. Here, we are interested in the detection or segmentation of
objects in an uncontrolled environment and under various constraints related
to the image acquisition conditions. OD is still a hot topic and many methods
have been proposed during these last two decades [3, 4]. However, the impact
of the distortions on the performance of the proposed OD solutions was often
neglected apart a few studies limited to object recognition and image
classification under specific distortions (noise and blur) [5] and OD under
photometric and geometric distortions [6]. A previous study [7] highlighted
the distortion impact on the OD performance through global and local
distortions without any scene context consideration have been achieved, which
proved the usefulness of data augmentation by using a distorted database to
improve OD models robustness. Consequently, we propose a novel distorted image
database with complex and photorealistic distortions. This database offers the
diversity and quality of distortions necessary for designing robust deep-
learning models, in particular OD models. For this, we introduced the local
and realist atmospheric distortions in our database. Unlike the classic so-
called global distortions applied to the entire image, local distortions apply
to defined areas. Local distortions correspond to the local representation of
distortions resulting from scene conditions due to object motion or position
in the scene, such as motion blur from moving objects, defocus blur and
backlight phenomena.

(a) Gaussian noise

(b) Contrast changing

(c) Global defocus blur

(d) Global motion blur

Figure 1: Some examples of global distortions.

The proposed atmospheric distortions attempt to better replicate the natural
rain and fog phenomena by applying these distortions in a non-homogeneous
manner. These new distortions consider scene context through scene depth and
object annotation from MS-COCO’s ground truth for better photorealism.
Furthermore, a manual annotation of the original COCO database was done to
guide the choice of the distortion to be applied automatically to each image.
In addition, a scene classification (indoor/outdoor) was performed to
automatically manage the distortion intensity according to the type of scene.
The main contributions of our study are summarized as follows:

  * •

New distortions with improved realism are introduced, describing common
phenomena in computer vision through complex local and atmospheric
distortions.

  * •

This paper proposes efficient algorithms to generate local and global
photorealistic distortions that are not included in any existing database.

  * •

A novel dataset is built from the MS-COCO dataset, dedicated to the
improvement of the robustness of the OD and object segmentation models against
a broad type of distortion.

  * •

The image database, the proposed database scene classification index, and
distortion generation codes are publicly available.

The remainder of the paper is organized as follows. Section II summarizes
previous related literature. Section III is devoted to detail the methods of
generating complex distorted images. Then, section IV is dedicated to show
dataset details. Finally, conclusions and perspectives are provided in section
V.

##  II Related Work

Object detection in video sequences or still images is a research topic of
great interest given the numerous applications in the computer vision field
[8, 9] and especially in video surveillance [10]. With the development of deep
learning methods and the availability of many databases dedicated to this
problem, this field of research has seen a real progress. A comprehensive
survey on deep learning based OD approaches is provided in [11]. However, most
of the available databases do not consider real-world scenarios, especially
images and videos captured in uncontrolled environments, which are affected by
various types of distortions. In fact, many studies have shown that OD
performance is strongly influenced by the quality of the images [5, 6, 12, 13,
7]. It is worth noticing that the number and types of distortions considered
in these studies and the existing dedicated dataset are limited. Furthermore,
the case of multiple distortions appearing simultaneously has not been taken
into account in OD performance evaluation studies. Multiple distortion
scenarios have been considered in a few studies on video quality assessment
but in limited contexts [14, 15, 16]. Some interesting studies investigated
the impact of various distortions on the performance of CNN-based OD
architectures [17, 18]. However, all these studies are limited to a few
distortions and do not consider local distortions that really correspond to
real scenarios. Indeed, if we take, for example, the blur caused by movement,
it is usually simulated in a global way in the existing databases. While we
know that in an observed scene there can be objects moving at different speeds
and in different directions and therefore affected by blurs of different
amplitudes and directions. The same applies to the defocusing blur, which
depends on the depth of the objects in the filmed scene. In our database we
have taken into account these aspects and others such as lighting effects that
vary with the depth and geometry of objects. We have adopted the same approach
concerning the distortions due to atmospheric phenomena such as rain and fog.
Taking into account these aspects is not simple and it is one of the main
originalities of our contribution.

##  III Complex distortion generation algorithms

(a) Original image

(b) Local motion blur

(c) Mask annotation

(d) Depth image

(e) Original image

(f) Local motion blur

(g) Mask annotation

(h) Depth image

Figure 2: Illustration of the local motion blur.

Well-known global distortions have been applied to our database through
classical distortions methods. In our case, global distortion refers to the
classic distortions that apply more or less homogeneously to the entire image,
regardless of the context of the scene. Thus, we applied global distortions
for some images resulting from image acquisition (noise, compression, contrast
changing) or camera (motion and defocus blur) conditions without considering
the scene context (see fig.1). However, some images have specific scene
contexts that require the application of local or atmospheric distortions
using more sophisticated approaches. Our generated complex distortions use
scene depth information, ground truth information from COCO annotations
(object masks), and object and scene type to produce complex and
photorealistic distortions. Scene depth information is obtained using the
MiDaS depth estimation model [19].

###  III-A Local motion blur

Local motion blur is a local application of motion blur phenomena to the
annotated objects. It represents the cases of image acquisition where objects
move rapidly in front of the camera. This local distortion requires the ground
truth masks to define the pixel area where the blur motion is to be applied.
Furthermore, the object mask is also used to determine the distortion
orientation through a strategy specific to the nature of the object (object
classes). Another dual strategy allows us to compute the motion magnitude
applied to each object in the images. First, an interval of motion magnitude
is derived from the nature of the object and prior knowledge about the speed
of the object type. Then, a magnitude value is computed by considering the
object’s depth and the others. this value into the global scene context. Thus,
each magnitude value is obtained by correlating the nature and depth of
objects, ensuring the global consistency of each local blur motion distortions
relative to each other. Object orientation is obtained by computing the angle
between the X-axis and the ellipse’s major axis containing the object. Then, a
checking strategy of the orientation is adapted to apply a motion blur
according to the object’s nature. Furthermore, a checking of object
interaction is achieved to prioritize the magnitude and orientation of higher-
level objects on lower ones as shown in fig.3.

Figure 3: Interaction hierarchy related to the object type

This is done by correlating their depth proximity and their bounding box
overlap to ensure distortion consistency for linked objects. Thus, the
magnitude and orientation of higher-level objects are applied to lower-level
objects with which they are interacting. The complete algorithm follows the
following steps:

  1. 1.

Find the scene context: ski, riding, sport, skate or surf depending on present
objects in the image.

  2. 2.

Object classification: create object superclasses by grouping objects together
to think globally (vehicle, person, animal, food, etc…).

  3. 3.

Compute the average depth of each annotated objects.

  4. 4.

Calculate amplitude and orientation using depth and object type to distort
each object individually.

  5. 5.

Find interactions between objects by correlating their depth proximity and
their overlapping bounding boxes to apply the same distortion to interacted
objects.

  6. 6.

Sort the objects according to their depth to adjust their motion amplitude for
a global consistency of the scene and distortions.

###  III-B Local Defocus blur

Local defocusing blur results from focusing only on only the background
orforeground. To create a realistic defocus blur, we used successive smooth
thresholding to create three distinct areas related to scene depth. This
thresholding process is performed using a nonlinear smooth function $\Omega$
expressed as:

$\Omega(x)=1-\frac{1}{1+\exp{-15(x-0.5)}}$ (1)

Where $x$ represents the keypoint depth normalized by the average depth of the
closest object as follows:

$x=\frac{threshold-p_{i}}{threshold}$ (2)

(a) Original image

(b) Local defocus blur

(c) Mask annotation

(d) Depth image

(e) Original image

(f) Local defocus blur

(g) Mask annotation

(h) Depth image

Figure 4: Illustration of the local defocus blur.

Figure 5(b) illustrates the image splitting through the smooth thresholding of
the scene depth to get the three different grounds. Foreground corresponds to
depths with threshold coefficients higher than the high threshold, middle-
ground to coefficients between the high and low thresholds, and the background
for coefficients lower (see fig.5(a)). Then, the average depths $\delta$,
$\delta_{m}$, and $\delta_{b}$ of the three grounds are computed to perform a
proportional defocus blur related to the depth. We applied a cumulative
defocus blur magnitudes $\lambda$, $\lambda_{m}$, and $\lambda_{b}$ going from
foreground to background on each zone, expressed as follows:

$\displaystyle\lambda=0.5+\frac{\delta_{f}-threshold}{threshold}\cdot 1.5$ (3)
$\displaystyle\lambda_{m}=\lambda+\frac{\delta_{m}-threshold}{threshold}\cdot
1.2$ (4)
$\displaystyle\lambda_{b}=\lambda_{m}+\frac{\delta_{b}-threshold}{threshold}\cdot
1.2$ (5)

(a) Sigmoid curve for smooth thresholding

(b) Threshold related to the depth

Figure 5: Smooth thresholding related to the depth.

The aread bounded by the masks are distorted according to their corresponding
defocus blur magnitude ($\lambda,\lambda_{m}$ and $\lambda_{b}$), then fused
to get the complete distorted image $I_{d}$. as shown in Fig. 4. The proposed
local defocus blur algorithm is described in the algorithm 1.

Algorithm 1 Locale defocus blur algorithm

Input: Image I, Keypoints depth $p_{i}$
Output: Distorted Image $I_{d}$

Find the closest object depth: threshold

High threshold $th_{f}$ = 0.8176

Low threshold $th_{b}$ = 0.182

for each $p_{i}\in I$ do

$\sigma$ = $\frac{threshold-p_{i}}{threshold}$

$\Delta(p_{i})$ = $1-\frac{1}{1+\exp{(-15(\sigma-0.5))}}$

if $threshold>p_{i}$ then

$Foreground\leftarrow p_{i}$

else if $th_{f}<=\Delta(p_{i})$ then

$Foreground\leftarrow p_{i}$

else if $th_{b}<=\Delta(p_{i})$ then

$Middleground\leftarrow p_{i}$

else if $th_{f}>=\Delta(p_{i})$ and $th_{b}>\Delta(p_{i})$ then

$Background\leftarrow p_{i}$

end if

$\delta_{f}\leftarrow$ Foreground average depth

$\delta_{m}\leftarrow$ Middleground average depth

$\delta_{b}\leftarrow$Background average depth

end for

###  III-C Atmospheric distortion: the rain

Synthesizing the rain homogeneously, without any scene depth consideration,
lacks realism. Indeed, the size and density of the rain depends on the
distance from which it falls. Thereby, our rain generation algorithm used the
method from algorithm 1 for performing a scene depth classification into
foreground, middle-ground, and background. Each ground is assigned a rain
intensity level that replicates the rain density. Note that the rain masks are
obtained from images of flowing water like rain produced under experimental
conditions. We extract three rain densities from these rain masks by
performing some erosion and dilation processes. These three rain sub-masks are
applied for each ground according to a random constant $\alpha$ of blending,
achieving image blending as shown in fig.6. It is worth noticing that the rain
sub-masks are applied cumulatively from the foreground to the background as
follows:

$\displaystyle I_{f}=1-((1-I)\cdot(1-(\alpha\cdot R_{f})))$ (6) $\displaystyle
I_{m}=1-((1-I_{f})\cdot(1-(\alpha\cdot R_{m})))$ (7) $\displaystyle
I_{d}=1-((1-I_{m})\cdot(1-(\alpha\cdot R_{b})))$ (8)

Where $I$, $I_{f}$, $I_{m}$ and $I_{d}$ are the original, foreground,
middleground and final distorted images respectively. Likewise, $R_{f}$,
$R_{m}$, and $R_{b}$ are the three rain sub-masks.

Figure 6: Rain distortion example

Thus, this approach applies only the fine rain corresponding to the distant
rain stream to the distant parts of the scene for better realism, as shown in
Fig.6. Our complex atmospheric rain improves the global consistency of
distortion by considering the spatial relationship between scene depth and
rain phenomena.

###  III-D Atmospheric distortion: the fog

Generating synthetic fog is a complex task. Indeed, no mathematical fog model
could have been used for generating synthetic fog. Thus, we opted to use fog
masks extracted from images of experimental creations of fog with black
background.

(a) Original image

(b) Original image

(c) Fog distortion

(d) Fog distortion

Figure 7: Examples of atmospheric distortion: the fog.

Many masks have been extracted to provide a large fog sample with diverse
densities and forms. These masks are applied to the original images,
seamlessly blending through a mask adjustment. However, applying a mask
homogeneously produce a non-realistic fog. Considering the scene depth for
applying the mask to match the thick fog effect in real cases seems crucial.
Thereby, we carry out this mask $H$ with a variable factor $\kappa(i,j)$
proportional to the normalized depth $Depth_{n}(i,j)$ of each image pixel
$I(i,j)$ and a constant value $\alpha$ as summarized in algorithm 2.

Algorithm 2 Fog generation algorithm

Input: Image $I$, fog mask $H$
Output: Distorted Image $I_{d}$

$\alpha$ = 0.95

for each pixel $i,j\in I$ do

$Depth_{n}(i,j)$ = $\frac{Depth(I(i,j))}{Depth_{max}}$

$\kappa(i,j)$ = $\alpha\cdot Depth_{n}(i,j)$

$I_{d}(i,j)$ = $(1-(1-I(i,j))\cdot(1-(\kappa(i,j)\cdot H(i,j)))\cdot 255$

end for

To give the images generated more photo-realism, the thickness of the fog is
adapted to the depth of the observed scene, reproducing the effect of fog
accumulation, as shown in figure 7.

###  III-E Local Backlight

Local backlight distortion is generated by applying a local contrast
enhancement process to the luminance component by using the object
segmentation mask. This pixel-wise intensity transformation takes into account
the position of the light source and that of the illuminated object on which
the effect is to be brought out. This operation, which is nothing more than
tone mapping, is applied to three preselected intensity intervals, semi-
automatically and randomly. Figure 8 illustrates this type of photometric
distortion.

(a) Original image

(b) Distorted image

Figure 8: Local backlight distortion

##  IV Dataset

The generated dataset consists of more than 123K images with 80 object classes
organized in three sets: 95K, 5K and 23K images for train, validation and test
sets respectively. The ground truth annotation provides the objects’ classes,
bounding boxes, and masks for each image, which can be used for training
object detection, and segmentation models.

###  IV-A Distortions

Our distorted dataset is composed of ten distortion types, 5 global
distortions, 2 global atmospheric distortions and 3 local distortions. In
order to generate the different distortions in a coherent and relevant way, a
first scan of all the images is performed to prepare the distortion assignment
protocol according to the semantic content of the scene and the context.

TABLE I: Distribution of distortions. Distortion type |  Number of images |  Ratio
---|---|---
Compression artefact |  17989 |  15.3%
Contrast changing |  18038 |  15.4%
Gaussian noise |  18055 |  15.4%
Global motion blur |  18018 |  15.3%
Global defocus blur |  17792 |  15.1%
Fog |  787 |  0.7%
Rain |  845 |  0.7%
Local Backlight |  296 |  0.3%
Local defocus blur |  7061 |  6.0%
Local motion blur |  18625 |  15.9%

The different distortions are automatically applied to the images previously
annotated during the first process. Images annotated as global distortions are
then distorted by one of the global distortion types chosen randomly (see
table I).

###  IV-B Scene classification

The observed scenes are classified into indoor and outdoor scenes based on the
context of the images. Indoor scenes were attributed to scenes where most
information is included in indoor environments (room, building, hall, vehicle
interior, etc.). Conversely, outdoor scenes correspond to open environments.
The table II summarises the scene classification of our dataset.

TABLE II: Scene classification. Scene type |  Number of images
---|---
Indoor scene |  45884
Outdoor scene |  72404
Skiing scene |  4434
Surfing scene |  3635
Skating scene |  3603
Sport scene |  11965

##  V Conclusion

In this study, we presented novel local and global complex distortions
generated by reliable algorithms considering the scene context to achieve a
high level of photo-realism. The proposed database will improve not only OD
algorithms but also many scene analysis, classification and image segmentation
methods, providing a more complete and beneficial framework for deep learning-
based methods. As a perspective, it would be interesting to enrich this
database with other distortions and in particular those related to atmospheric
perturbations such as the heat diffusion effect and pollution. Another aspect
that could be considered in the future is to incorporate pose object
estimation when applying distortion.

## References

  * [1] Stefan Winkler.  Analysis of public image and video databases for quality assessment.  IEEE Journal of Selected Topics in Signal Processing, 6(6):616–625, 2012.
  * [2] Muhammad Ali Qureshi, Azeddine Beghdadi, and Mohamed Deriche.  Towards the design of a consistent image contrast enhancement evaluation measure.  Signal Processing: Image Communication, 58:212–227, 2017.
  * [3] Zhengxia Zou, Zhenwei Shi, Yuhong Guo, and Jieping Ye.  Object detection in 20 years: A survey.  arXiv preprint arXiv:1905.05055, 2019.
  * [4] Zhong-Qiu Zhao, Peng Zheng, Shou-tao Xu, and Xindong Wu.  Object detection with deep learning: A review.  IEEE transactions on neural networks and learning systems, 30(11):3212–3232, 2019.
  * [5] Tejas S Borkar and Lina J Karam.  Deepcorrect: Correcting dnn models against image distortions.  IEEE Transactions on Image Processing, 28(12):6022–6034, 2019.
  * [6] Zhongqi Lin, Zengwei Zheng, Jingdun Jia, Wanlin Gao, and Feng Huang.  Ml-capsnet meets vb-di-d: A novel distortion-tolerant baseline for perturbed object recognition.  Engineering Applications of Artificial Intelligence, 120:105937, 2023.
  * [7] Ayman Beghdadi, Malik Mallem, and Lotfi Beji.  Benchmarking performance of object detection under image distortions in an uncontrolled environment.  In 2022 IEEE International Conference on Image Processing (ICIP), pages 2071–2075. IEEE, 2022.
  * [8] Piotr Dollar, Christian Wojek, Bernt Schiele, and Pietro Perona.  Pedestrian detection: An evaluation of the state of the art.  IEEE transactions on pattern analysis and machine intelligence, 34(4):743–761, 2011.
  * [9] Constantine P Papageorgiou and Tomaso Poggio.  A trainable object detection system: Car detection in static images.  1999\.
  * [10] Azeddine Beghdadi, Muhammad Asim, Noor Almaadeed, and Muhammad Ali Qureshi.  Towards the design of smart video-surveillance system.  In 2018 NASA/ESA Conference on Adaptive Hardware and Systems (AHS), pages 162–167. IEEE, 2018.
  * [11] Licheng Jiao, Fan Zhang, Fang Liu, Shuyuan Yang, Lingling Li, Zhixi Feng, and Rong Qu.  A survey of deep learning-based object detection.  IEEE access, 7:128837–128868, 2019.
  * [12] Sebastian Cygert and Andrzej Czyżewski.  Toward robust pedestrian detection with data augmentation.  IEEE Access, 8:136674–136683, 2020.
  * [13] Xiangning Chen, Cihang Xie, Mingxing Tan, Li Zhang, Cho-Jui Hsieh, and Boqing Gong.  Robust and accurate object detection via adversarial learning.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16622–16631, 2021.
  * [14] Zohaib Amjad Khan, Azeddine Beghdadi, Mounir Kaaniche, and Faouzi Alaya Cheikh.  Residual networks based distortion classification and ranking for laparoscopic image quality assessment.  In 2020 IEEE International Conference on Image Processing (ICIP), pages 176–180. IEEE, 2020.
  * [15] Roger Gomez-Nieto, José Franciso Ruiz-Muñoz, Juan Beron, César A Ardila Franco, Hernán Darío Benítez-Restrepo, and Alan C Bovik.  Quality aware features for performance prediction and time reduction in video object tracking.  IEEE Access, 10:13290–13310, 2022.
  * [16] Roger Gomez Nieto, Hernan Dario Benitez Restrepo, and Ivan Cabezas.  How video object tracking is affected by in-capture distortions?  In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2227–2231. IEEE, 2019.
  * [17] Aharon Azulay and Yair Weiss.  Why do deep convolutional networks generalize so poorly to small image transformations?  J. Mach. Learn. Res., 20:184:1–184:25, 2019.
  * [18] Claudio Michaelis, Benjamin Mitzkus, Robert Geirhos, Evgenia Rusak, Oliver Bringmann, Alexander S Ecker, Matthias Bethge, and Wieland Brendel.  Benchmarking robustness in object detection: Autonomous driving when winter is coming.  CoRR, 2019.
  * [19] René Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun.  Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer.  IEEE transactions on pattern analysis and machine intelligence, 44(3):1623–1637, 2020.


# DISPERSIVE APPROACH TO HVP FOR MUON G-2

Z. ZHANG  Laboratoire de Physique des 2 infinis Irène Joliot-Curie – IJCLab,
CNRS/Univ. Paris-Saclay
15 rue Georges Clémenceau, 91405 Orsay cedex, France

###### Abstract

The dispersive approaches to the hadronic vacuum polarisation contributions to
the muon anomalous magnetic moment $a_{\mu}$ are described. The main issues
are discussed followed by perspectives in the next years.

##  1 Introduction

Vacuum polarisation (VP) originates from quantum fluctuations in the exchange
of gauge bosons occurring in particle interactions. The muon anomalous
magnetic moment $a_{\mu}$ is sensitive to such quantum fluctuations in all
three sectors, QED, Weak and QCD, of the Standard Model. The measurement of
$a_{\mu}$ has a long history and a strong interplay with the theoretical
predictions [1]. The measurement is dominated by the recent Run-1 result of
the Fermilab $g-2$ experiment [2] and the earlier BNL measurement [3]. They
are in good agreement and their average gives $116\,592\,061(41)\times
10^{-11}$ 111The units of $10^{-11}$ will be omitted for simplicity in the
following.. The comparison with the prediction of $116\,591\,810(43)$ provided
by the White Paper [4] of the Muon $g-2$ Theory Initiative results in,
however, a $4.2\sigma$ tension between the two. The theory prediction receives
dominant contributions from QED with a negligible uncertainty. The hadronic
vacuum polarisation (HVP) contribution, due to the fluctuations involving
strongly interacting particles, includes the leading-order (LO), higher-order
(HO) and light-by-light scattering components. The LO component,
$a_{\mu}^{\mathrm{Had,LO}}$, has the largest uncertainty of 40, to be compared
with the total uncertainty of 43 for $a_{\mu}$. It is evaluated using the
traditional dispersive approaches.

##  2 How are the evaluations performed?

The HVP contributions are induced by fluctuations of quark-antiquark pairs.
They can be computed at large energy scales in perturbative QCD but not at low
scales due to the non-perturbative nature of QCD at large distance. Using
unitarity and analyticity, the imaginary part of the two-point correlation (or
HVP) functions is connected to spectra of hadron production from $e^{+}e^{-}$
annihilation via a spin-one photon propagator by the dispersion relation [5]:

$\displaystyle
a_{\mu}^{\mathrm{Had,LO}}=\frac{1}{3}\left(\frac{\alpha}{\pi}\right)^{2}\int^{\infty}_{m^{2}_{\pi}}ds\frac{K(s)}{s}R^{(0)}(s)\,.$
(1)

The integration kernel [6] $K(s)/s\sim s^{-2}$ strongly emphasises the low-
energy part of
$R(s)\equiv\sigma(e^{+}e^{-}\to\textrm{hadrons})/\sigma(e^{+}e^{-}\to\mu^{+}\mu^{-})$.

The HVP evaluation with the dispersive approaches is thus data-driven and the
precision depends directly on the precision of the
$e^{+}e^{-}\to\textrm{hadrons}$ cross-section data. A large number of
exclusive hadronic final states have been measured over a large energy range
by many different experiments at $e^{+}e^{-}$ colliders in two general
categories: 1) energy scan at different centre-of-mass energies $\sqrt{s}$,
e.g. CMD2/3 and SND at VEPP-2M/2000; 2) varying energies $\sqrt{s^{\prime}}$
from a fixed value of $\sqrt{s}$ using the technique of radiative return with
a hard photon emitted from the initial state radiation (ISR), e.g. KLOE at
DAPHNE $\phi$-factory, BABAR at PEP-II B-factory, and BES III at BEPC II
collider. The number of final states and their cross sections as a function of
energy can be appreciated from Figure 1 (left). The dominant final state is
$e^{+}e^{-}\to\pi^{+}\pi^{-}$ which contributes about 73% to
$a_{\mu}^{\mathrm{Had,LO}}$ and 58% to its uncertainty.

Figure 1: Left: contributions to the total hadronic $R$-ratio from different
final states (plot taken from Ref. [7]). Right: total hadronic $R$-ratio as a
function of $\sqrt{s}$ (plot taken from Ref. [8]).

Two groups, DHMZ and KNT, have performed since about 2 decades several
complete evaluations of all exclusive final states below 1.8 GeV and 1.937
GeV, respectively. The two groups use different approaches for the data
combination and uncertainty estimation.

The combination and integration by DHMZ uses a dedicated package HVPTools
deployed first in Ref. [9]. It transforms the bare cross-section data and
associated statistical and systematic covariance matrices into fine-grained
bins, taking into account the correlations between the measurements of one
experiment, as well as inter-experiment and inter-final-state correlations.
The bin size is typically 1 MeV and smaller for the $\omega$ and $\phi$
resonances. The interpolation between adjacent measurements of a given
experiment uses second-order polynomials, to avoid potential bias when using
the linear trapezoidal interpolation. The averaging of the interpolated
measurements from different experiments is performed using pseudo-data
generated from the original measurements within wider averaging region so that
all locally available experiments contribute. The uncertainty of the average
is inflated if the local $\chi^{2}$/dof is greater than 1 following the PDG
procedure. For the dominant $\pi^{+}\pi^{-}$ final state, the
$a_{\mu}^{\mathrm{Had,LO}}$ evaluation at energies below 0.6 GeV uses
constraints from unitarity and analyticity. The evaluation of the total
hadronic $R$-ratio for energies up to 5 GeV is shown in Figure 1 (right).

The data combination by KNT for a given final state is performed using an
iterative procedure by minimising a $\chi^{2}$ function comparing the linear-
model interpolant cross section values of clusters against direct measurements
[7]. The cluster size depends on local data density and the correlation of
measurement uncertainties is taken into account in the fit. The fit gives the
mean $R$ values, a covariance uncertainty matrix, and local
$\chi^{2}_{\mathrm{min}}$/dof for each cluster, with dof being the number of
degree of freedom within a cluster. The quantity $a_{\mu}^{\mathrm{Had,LO}}$
is derived using trapezoidal integration between clusters. The uncertainty of
KNT’s recent evaluations is also inflated using $\chi^{2}_{\mathrm{min}}$/dof.

The HVP prediction in the White Paper was obtained in a conservative merging
procedure using the evaluations of these two groups [10, 8] together with a
partial evaluation in the $\pi^{+}\pi^{-}$ and $\pi^{+}\pi^{-}\pi^{0}$ final
states [11, 12] with constraints from unitarity and analyticity. The procedure
accounts for tensions among the measurements, for differences in methodologies
in the combination of experimental inputs, and for correlation between
systematic errors.

##  3 What are the issues?

The main issue concerns the discrepancy between two most precise measurements
from BABAR and KLOE in the dominant $\pi^{+}\pi^{-}$ final state. The DHMZ
group studied the discrepancy by removing either of them from the evaluation
and found a difference of 27.6 which is included in the channel-specific
systematic uncertainty to this final state shown in Table 1 (adapted from
Table 5 in Ref. [4]). Among the selected final states shown in the table, the
difference between the two evaluations for some of the channels is larger than
the quoted uncertainties from one evaluation reflecting again their different
methodologies.

Unfortunately, the $\pi^{+}\pi^{-}$ is not the only problematic case.
Differences between different measurements are also observed in other final
states, such as $K^{+}K^{-}$ around the $\phi$ resonance peak [8]. It should
be noted that it is difficult to determine kaon detection efficiency with
precision for CMD-2/3 and SND and easier for BABAR with large ISR photon
energies.

Table 1: Selected exclusive-mode contributions to $a_{\mu}^{\mathrm{Had,LO}}$ from DHMZ19 and KNT19, for the energy range $\leq 1.8$ GeV. Where three (or more) uncertainties are given for DHMZ19, the first is statistical, the second channel-specific systematic, and the third common systematic, which is correlated with at least one other channel. | DHMZ19 | KNT19 | Difference
---|---|---|---
$\pi^{+}\pi^{-}$ | $5078.5(8.3)(32.3)(5.5)$ | $5042.3(19.0)$ | $36.2$
$\pi^{+}\pi^{-}\pi^{0}$ | $462.1(4.0)(11.0)(8.6)$ | $466.3(9.4)$ | $-4.2$
$\pi^{+}\pi^{-}\pi^{+}\pi^{-}$ | $136.8(0.3)(2.7)(1.4)$ | $139.9(1.9)$ | $-3.1$
$\pi^{+}\pi^{-}\pi^{0}\pi^{0}$ | $180.3(0.6)(4.8)(2.6)$ | $181.5(7.4)$ | $-1.2$
$K^{+}K^{-}$ | $230.8(2.0)(3.3)(2.1)$ | $230.0(2.2)$ | $0.8$
$K_{S}K_{L}$ | $128.2(0.6)(1.8)(1.5)$ | $130.4(1.9)$ | $-2.2$
$\pi^{0}\gamma$ | $44.1(0.6)(0.4)(0.7)$ | $45.8(1.0)$ | $-1.7$
Sum of the above | $6260.8(9.5)(34.8)(14.7)$ | $6236.2(22.7)$ | $24.6$
$[1.8,3.7]$ GeV (without $c\bar{c}$) | $334.5(7.1)$ | $344.5(5.6)$ | $-10.0$
$J/\psi$, $\psi(2S)$ | $77.6(1.2)$ | $78.4(1.9)$ | $-0.8$
$[3.7,\infty)$ GeV | $171.5(3.1)$ | $169.5(1.9)$ | $2.0$
Total $a_{\mu}^{\mathrm{Had,LO}}$ | $6940(10)(35)(16)(1)_{\psi}(7)_{\mathrm{DV+QCD}}$ | $6928(24)$ | $12$

##  4 Perspectives

The Fermilab Run-1 result has an uncertainty of 54, which is statistics
dominated. The uncertainty is expected to be improved by a factor of 2 with
the subsequent runs in the next years and eventually by a factor of 4 with its
final measurement. The uncertainty of the HVP prediction has to be reduced
concurrently.

The most important and urgent task is to understand the discrepancy between
BABAR and KLOE. To resolve the discrepancy, new ideas, improved event
generators, and mostly new measurements with higher precision and using a
blind approach are needed.

Since the release of the White Paper, a new measurement of the
$\pi^{+}\pi^{-}$ final state between 525 and 883 MeV by SND has been published
[13]. Above $\sim 750$ MeV, the measurement, though with a limited systematic
uncertainty of 0.8% for $\sqrt{s}>600$ MeV, seems to be in better agreement
with BABAR than with KLOE. There are also two new measurements for the
$\pi^{+}\pi^{-}\pi^{0}$ final state by SND in the energy range of 1.15$-$2.0
GeV [14] and by BABAR in the energy range of 0.62$-$3.5 GeV [15]. The latter
measurement is measured with a systematic uncertainty of 1.3% in the energy
regions of the $\omega$ and $\phi$ resonances. This final state has the second
largest cross section after $\pi^{+}\pi^{-}$ below 1 GeV which contributes
currently with a precision of 3% to $a_{\mu}^{\mathrm{Had,LO}}$. The BES III
has published a new $R$ measurement at 14 energy points between 2.2324 and
3.6710 GeV with an accuracy of better than 2.6% below 3.1 GeV and 3% above
[16]. The measured cross section is, however, higher than the corresponding
KEDR result [17] and perturbative prediction, in particular in the energy
regions 3.4$-$3.6 GeV by 1.9 and 2.7 standard deviations. There are also
attempts to improve the uncertainty for the $\pi^{0}\gamma$ and $K\bar{K}$
final states [18, 19] using constraints from analyticity, unitarity, and
crossing symmetry as well as low-energy theorems, though they are not the
dominant final states contributing to $a_{\mu}^{\mathrm{Had,LO}}$.

One surprise after the publication of the White Paper is the Lattice
prediction of $a_{\mu}^{\mathrm{Had,LO}}$ by BMW [20] which has dramatically
improved the previous Lattice predictions by many factors to reach a precision
of 0.8% and has a value lying in between the direct measurements and the
dispersive prediction with a deviation of $1.5\sigma$ and $2.1\sigma$,
respectively. If this result will be confirmed with comparable precision by
the other Lattice groups, the discrepancy between the dispersive and Lattice
predictions must be understood.

## Acknowledgments

I am grateful for the fruitful collaboration with my colleagues and friends M.
Davier, A. Hoecker, and B. Malaescu.

## References

## References

  * [1] M. Davier, A. Hoecker, B. Malaescu and Z. Zhang, Adv. Ser. Direct. High Energy Phys. 26, 129 (2016).
  * [2] B. Abi et al. [Muon g-2], Phys. Rev. Lett. 126, 141801 (2021).
  * [3] G. W. Bennett et al. [Muon g-2], Phys. Rev. D 73, 072003 (2006).
  * [4] T. Aoyama et al. Phys. Rept. 887, 1 (2020).
  * [5] C. Bouchiat and L. Michel, J. Phys. Radium. 22, 121 (1961).
  * [6] M. Gourdin and E. De Rafael, Nucl. Phys. B 10, 667 (1969).
  * [7] A. Keshavarzi, D. Nomura and T. Teubner, Phys. Rev. D 97, 114025 (2018).
  * [8] M. Davier, A. Hoecker, B. Malaescu and Z. Zhang, Eur. Phys. J. C 80, 241 (2020) [erratum: Eur. Phys. J. C 80, 410 (2020)].
  * [9] M. Davier, A. Hoecker, G. Lopez Castro, B. Malaescu, X. H. Mo, G. Toledo Sanchez, P. Wang, C. Z. Yuan and Z. Zhang, Eur. Phys. J. C 66, 127 (2010).
  * [10] A. Keshavarzi, D. Nomura and T. Teubner, Phys. Rev. D 101, 014029 (2020).
  * [11] G. Colangelo, M. Hoferichter and P. Stoffer, JHEP 02, 006 (2019).
  * [12] M. Hoferichter, B. L. Hoid and B. Kubis, JHEP 08, 137 (2019).
  * [13] M. N. Achasov et al. [SND], JHEP 01, 113 (2021).
  * [14] M. N. Achasov et al. [SND], Eur. Phys. J. C 80, 993 (2020).
  * [15] J. P. Lees et al. [BABAR], Phys. Rev. D 104, 112003 (2021).
  * [16] M. Ablikim et al. [BES III], Phys. Rev. Lett. 128, 062004 (2022).
  * [17] M. Ablikim et al. [KEDR], Phys. Lett. B 753, 533 (2016); 770, 174 (2017); 788, 42 (2019).
  * [18] B. L. Hoid, M. Hoferichter and B. Kubis, Eur. Phys. J. C 80, 988 (2020).
  * [19] D. Stamen, D. Hariharan, M. Hoferichter, B. Kubis and P. Stoffer, [arXiv:2202.11106 [hep-ph]].
  * [20] S. Borsanyi, Z. Fodor, J. N. Guenther, C. Hoelbling, S. D. Katz, L. Lellouch, T. Lippert, K. Miura, L. Parato and K. K. Szabo et al., Nature 593, 51 (2021).


# Performance Analysis of Semi-supervised Learning in the Small-data Regime
using VAEs

Varun Mannam
Department of Electrical Engineering
University of Notre Dame
Notre Dame, IN 46556
<EMAIL_ADDRESS>
&Arman Kazemi
Department of Computer Science and Engineering
University of Notre Dame
Notre Dame, IN 46556
<EMAIL_ADDRESS>
Varun Mannam is with the Department of Electrical Engineering, University of
Notre Dame, Notre Dame, IN, 46556 USA e-mail<EMAIL_ADDRESS>
###### Abstract

Extracting large amounts of data from biological samples is not feasible due
to radiation issues, and image processing in the small-data regime is one of
the critical challenges when working with a limited amount of data. In this
work, we applied an existing algorithm named Variational Auto Encoder (VAE)
that pre-trains a latent space representation of the data that captures the
features in a lower-dimension for the small-data regime input. The latent
space representation will be fine-tuned, and its weights will be fixed. The
latent space will be used as a segment of the neural network that can be used
for classification. Here we will present the performance analysis of the VAE
algorithm with various latent space sizes in the semi-supervised learning
using the CIFAR-10 dataset.

_K_ eywords VAE, CIFAR-10, Small Data

##  1 Introduction

Artificial neural networks (ANNs), specifically Convolutional Neural Networks
(CNNS), have become popular due to their success in image classification,
feature extraction and object recognition and detection [1] in the recent
years. CNNs leverage the huge amount of labeled data available to train
networks that outperform humans in image recognition tasks. However, in the
small-data regime, the accuracy of trained networks using a limited number of
labeled samples is low [2]. This is a typical case when working with
biological samples where exposure to radiation (in order to capture an image)
is detrimental to the well-being of the sample. More images can be derived
from the initial data by some augmentation methods, but it is unhelpful due to
lack of labeled images.

To address this problem, there exists a framework called “Auto Encoder” (AE
[3]) that uses all the input data, labeled and unlabeled, to train a low-
dimensional embedding. AE is a neural network that takes unlabeled images as
the input and regards the input itself as the label. As illustrated in Figure
1, AE is comprised of two parts: the encoder and the decoder. The encoder part
tries to embed the features in a latent space that can extract the features of
the original image and the decoder tries to restore the image to the original
image. The process called “pre-training” trains the weights for both encoder
and decoder parts. Once trained, the encoder part of the AE will be a
representation of all the labeled and unlabeled data. This increases the
amount of usable information from all of the images.

Traditional semi-supervised models consists of pretraining using Restricted
Boltzmann machine (RBM)[4] or Gaussian-Restricted Boltzmann machine
(G-RBM)[4]. RBM is an energy based model that is represented using an
undirected graph containing a layer of observable variables and a single layer
of latent variables (similar to hidden units in a multi-layer perceptron) [1,
5]. This energy based model was first introduced in 1980’s [6] and have been
implemented using diverse datasets including image [4] and medical data [7].
Hinton and Salakhutdinov [8] showed that RBMs can be stacked and trained in a
greedy manner. This deep learning model, Deep belief networks(DBN), that
utilizes RBM as the learning model has been implemented on various
unsupervised and supervised learning problems. Later, Bengio et. al. [9]
showed that the pre-trained undirected graphical model in semi-supervised
setting performs well with deep architectures. However, the challenge working
with RBM is that it is constructed using sigmoid functions as the activation
function between the input and the hidden layer. As we are aware that the
major drawback of using sigmoid activation function is the vanishing gradient
problem. Hence, in this work, we pre-train the model using “Variational Auto
Encoder” (VAE [3]).

After pre-training the encoder is able to observe similar images to the
training images and extract the valuable features from it in a lower dimension
than the initial image. Now it is possible to couple the encoder with a small
neural network and train that network for classification tasks. The present
work is similar to reinforcement learning where the model is trained with one
dataset and uses the feature extraction part of that model to train another
model for a different dataset. It is important to mention that the encoder
weights are fixed and can’t be changed. It is only the small neural network
that will be trained. The input to this network is the small set of labeled
data. This is called “fine-tuning”.

In this project we will implement VAE that tries to capture not only the
compressed representation of the images, but also the parameters of a
probability distribution representing the data. We will examine the effect of
different size of the latent spaces and how it affects the accuracy of the
model. Later, we will analyze the performance of the semi-supervised model
with the optimum latent space.

##  2 Background

In this section we enumerate the basic details about AE and VAE.

###  2.1 Auto Encoder

An autoencoder is a type of ANN used to learn efficient data encoding in an
unsupervised manner. The aim of an autoencoder is to learn a representation of
a set of data, typically for dimensionality reduction, by training the network
to ignore signal noise. Along with the reduction side, a reconstructing side
is learned, where the autoencoder tries to generate from the reduced encoding
a representation as close as possible to its original input. An autoencoder
always consists of two parts, the encoder and the decoder, which can be
defined as transitions $\phi$ and $\psi$ such that [3]:

$\phi:X\rightarrow F$ (1) $\psi:F\rightarrow Y\newline $ (2)
$\phi,\psi=argmin_{\phi,\psi}||X-(\psi o\phi)(X)^{2}||$ (3)

where the given input is $X$ and the predicted output is $Y$. If the feature
space $F$ has lower dimensionality than the input space $X$, then the feature
vector $\phi(x)$ can be regarded as a compressed representation of the input
$X$.

Figure 1: Autoencoder block diagram

###  2.2 VAE

To extend the idea used in the AE, there is a variation called VAE which uses
the “KL-divergence” between the predicted probability distribution function
and actual posterior distribution in the latent space whereas traditional AEs
try to find the accurate mapping functions in the encoder (between input and
latent space) as well as in the decoder (between the latent space and output).
Using VAE, we generate a large dataset by adding the noise in the latent space
which is similar to input data augmentation (adding noise to images to
increase the number of examples in the input dataset).

VAE uses a variational approach for latent representation learning, which
results in an additional loss component. It assumes that the data is generated
by a directed graphical model $p(\textbf{X}|\textbf{Z})$ and that the encoder
is learning an approximation $q_{\phi}(\textbf{Z}|\textbf{X})$ of the
posterior distribution $p_{\theta}(\textbf{X}|\textbf{Z})$ where $\phi$ and
$\theta$ denote the parameters of the encoder (recognition model) and decoder
(generative model) respectively. We can write the conditional or posterior
distribution

$p(\bm{z}|\bm{x})=\frac{p(\bm{z},\bm{x})}{p(\bm{x})}=\frac{p(\bm{x}|\bm{z})p(\bm{z})}{p(\bm{x})}$
(4)

The denominator of above equation is the marginal distribution of the
observations and is calculated by marginalizing out the latent variables from
the joint distribution, i.e.

$p(x)=\int_{z}p(z,x)dz$ (5)

In many cases of interest this integral is not available in closed form or is
intractable (requires exponential time to compute). Hence, we consider
variational approximation as follows: consider a tractable distribution q(z).
The goal is to find the best approximation, e.g., the one that satisfies the
following optimization problem:

$Minimize:D_{KL}[q_{\phi}(\bm{z}|\bm{x})||p_{\theta}(\bm{z}|\bm{x})]$ (6)

Therefore, the objective of the variational autoencoder in this case has the
following form:

$\mathcal{L}(\phi,\theta,\mathbf{x})=D_{\mathrm{KL}}\left(q_{\phi}(\mathbf{z}|\mathbf{x})\|p_{\theta}(\mathbf{z})\right)-\mathbb{E}_{q_{\phi}(\mathbf{z}|\mathbf{x})}\left(\log
p_{\theta}(\mathbf{x}|\mathbf{z})\right)$ (7)

where $D_{KL}$ stands for the KL-divergence. In the VAE, the principle is to
minimize the loss between input and the restored image along with the loss
generated by the latent space to represent the features in the input images.

##  3 Methodology

Figure 2: Network architecture: pre-training with VAE(first row) In the second
row, parameter $\theta$ is underlined and bold to indicate that these
parameters are freezed when the fine-tuning the network

Consider a surrogate model $y=f(x,\theta)$ which is trained using limited
simulation data
$\mathcal{D}=\\{x^{i},y^{i}\\}_{i=1}^{N},\\{x^{j}\\}_{j=N+1}^{D}$. Where the
input data, $x^{i}\in\mathbb{R}^{d_{x}\times H\times W}$ is the input from
CIFAR-10 dataset. Here H and W are the height and width respectively and
$d_{x}$ is the number of dimensions for the input x at one location. $x^{j}$
is the additional data utilized for pretraining the model.
$y^{i}\in\mathbb{R}^{1}$ is the classified result. $\theta$ is the model
parameter and N is the total number of training data utilized during fine-
tuning and D is the total number of data utilized for pre-training. In semi-
supervised model, we pre-train the model with the input data
$\mathbb{R}^{d_{x}\times H\times W}$ and then perform image classification
problem $\mathbb{R}^{d_{x}\times H\times W}\rightarrow\mathbb{R}^{1}$. For
both the pre-training and fine tuning, we used stochastic gradient descent
with Adam optimizer to update the network weights and biases. The simulation
was performed using Pytorch machine learning package in Python.

###  3.1 VAE pre-training

We implemented dense-net [10],[11] version of VAE for the pre-training part.
Dense-net contains the encoder and decoder blocks along with the dense-layer
that has the simple and complex features.

###  3.2 VAE fine-tuning

We implemented a simple fully connected layer to classify the input images on
the CIFAR-10 [12] data. This is due to the expectation that the latent space
is smaller (either 4 $\times$ 4 or 8 $\times$ 8) (image size is 32 $\times$
32). If the number of channels are large at the latent space, we will add more
fully connected layers for classification of images.

##  4 Data

We perform the simulations on the CIFAR-10 dataset with ten image classes with
three input channels $(C=3)$ of size 32 $\times$ 32 (W $\times$ H). CIFAR-10
dataset has 50000 training images and 10000 test images.

##  5 Results

In this section we enumerate the results obtained using CIFAR-10 data. We
consider the following latent dimensions: 6400, 10,000 and 14,400. In order to
evaluate the performance of the model for above three latent space, we
consider the distribution estimated for the values at various pixel location.

###  5.1 Pre-training

For the results presented in this section, we have a dataset with 50,000
$\\{x^{k}\\}_{k=1}^{50000}$ examples for pre-training and the test set consist
of 10000, $\\{x^{k}\\}_{k=1}^{10000}$ examples. Adam optimizer was used for
training 100 epochs, with learning rate of 1e-4 and a plateau scheduler on the
test RMSE. Batch size is always smaller than the number of training data. In
this work, a batch size of 16 for pre-training was used. Weight decay was set
to 1e-3 for pre-training.

We consider Equation: 7 loss function to evaluate the trained model on test
data and also to monitor the convergence.

Figure 3: Error v/s epoch for 6400 latent space (top), Error v/s epoch for
10000 latent space (middle) and Error v/s epoch for 14400 latent space
(bottom)

From figure 1, we observe that the solution is converged after 50 epochs and
most importantly the loss for the three latent spaces is similar.

Figure 4: Distribution estimate for the values at various location of the
square domain for 6400, 10000 and 14400 latent space

From figure 4, we observe that even when the latent size is small (Batch
$\times$ 100 (channels) $\times$ 8 $\times$ 8) and (Batch $\times$ 100
(channels) $\times$ 10 $\times$ 10) the reconstructed density estimate is
close to actual input data. The PDF with the latent size 10000 is closer to
the actual input and also 6400 & 14400 latent space. Since, all the latent
spaces yield the similar outputs,we fine-tune and compare the classification
accuracy in the next section.

###  5.2 Fine-tuning

In this section we freeze the parameters (weights and bias) used in the pre-
training stage and fine tune the parameters (weights and bias) in the
classification network. For this problem, we consider small data from the
given CIFAR-10 dataset and use fully connected layers to perform
classification. Cross entropy loss function is commonly used for all
classification problems, we implemented cross entropy to measures the
performance of a classification mode.

Figure 5: Fine tuning results with three different latent space models

From figure 5, we observe the latent space 10000 yields better accuracy than
other two latent spaces (6400 and 14400). The smaller the latent space the
lower the test accuracy, this is due to insufficient features to classify the
data. Also, for the large latent space, the test accuracy is low, this is due
to model complexity [1].

##  6 Conclusions and Future work

The present document outlines the development of surrogate model for semi-
supervised problem. In this work, we have implemented VAE as a pre-training
model and a feed forward deep learning model for the classification. The
results obtained for differently sized latent spaces are presented. It was
observed that there is a slight improvement in the test accuracy when the
latent space is 10000 in comparison with latent space of 6400 and 14400.

For future work, a Bayesian approach can be explored. Due to a limited amount
of data, it is necessary to model appropriate surrogate, since it is important
to quantify the epistemic uncertainty induced by limited data [13], [14] and
hence, a Bayesian probabilistic approach is a natural way of addressing this
challenge.

## References

  * [1] Ian Goodfellow, Yoshua Bengio, and Aaron Courville.  Deep learning.  MIT press, 2016.
  * [2] Nitesh V Chawla et al.  Learning from labeled and unlabeled data: An empirical study across techniques and domains.  Journal of Artificial Intelligence Research, 23:331–366, 2005.
  * [3] Autoencoder.  Autoencoder — Wikipedia, the free encyclopedia, 2019.
  * [4] Geoffrey E Hinton and Ruslan R Salakhutdinov.  Reducing the dimensionality of data with neural networks.  science, 313(5786):504–507, 2006.
  * [5] Kevin P Murphy.  Machine learning: a probabilistic perspective.  MIT press, 2012.
  * [6] Paul Smolensky.  Information processing in dynamical systems: Foundations of harmony theory.  Technical report, Colorado Univ at Boulder Dept of Computer Science, 1986\.
  * [7] Tu Dinh Nguyen, Truyen Tran, Dinh Phung, and Svetha Venkatesh.  Latent patient profile modelling and applications with mixed-variate restricted boltzmann machine.  In Pacific-Asia conference on knowledge discovery and data mining, pages 123–135. Springer, 2013.
  * [8] G Hinton and R Salakhutdinov.  An efficient learning procedure for deep boltzmann machines.  Neural Computation, 24(8):1967–2006, 2012.
  * [9] Yoshua Bengio, Pascal Lamblin, Dan Popovici, and Hugo Larochelle.  Greedy layer-wise training of deep networks.  In Advances in neural information processing systems, pages 153–160, 2007.
  * [10] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger.  Densely connected convolutional networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700–4708, 2017.
  * [11] Yide Zhang, Yinhao Zhu, Evan Nichols, Qingfei Wang, Siyuan Zhang, Cody Smith, and Scott Howard.  A poisson-gaussian denoising dataset with real fluorescence microscopy images.  arXiv preprint arXiv:1812.10366, 2018.
  * [12] Alex Krizhevsky and Geoffrey Hinton.  Learning multiple layers of features from tiny images.  Technical report, Citeseer, 2009.
  * [13] Rohit K Tripathy and Ilias Bilionis.  Deep uq: Learning deep neural network surrogate models for high dimensional uncertainty quantification.  Journal of Computational Physics, 375:565–588, 2018.
  * [14] Yinhao Zhu and Nicholas Zabaras.  Bayesian deep convolutional encoder–decoder networks for surrogate modeling and uncertainty quantification.  Journal of Computational Physics, 366:415–447, 2018.