# Integrating Biological Data into Autonomous Remote Sensing Systems for In
Situ Imageomics: A Case Study for Kenyan Animal Behavior Sensing with Unmanned
Aerial Vehicles (UAVs)

Jenna M. Kline 1, Maksim Kholiavchenko 2, Otto Brookes 3, Tanya Berger-Wolf 1,
Charles V. Stewart 2, Christopher Stewart 1

###### Abstract

In situ imageomics leverages machine learning techniques to infer biological
traits from images collected in the field, or in situ, to study individuals
organisms, groups of wildlife, and whole ecosystems. Such datasets provide
real-time social and environmental context to inferred biological traits,
which can enable new, data-driven conservation and ecosystem management. The
development of machine learning techniques to extract biological traits from
images are impeded by the volume and quality data required to train these
models. Autonomous, unmanned aerial vehicles (UAVs), are well suited to
collect in situ imageomics data as they can traverse remote terrain quickly to
collect large volumes of data with greater consistency and reliability
compared to manually piloted UAV missions. However, little guidance exists on
optimizing autonomous UAV missions for the purposes of remote sensing for
conservation and biodiversity monitoring. The UAV video dataset curated by
KABR: In-Situ Dataset for Kenyan Animal Behavior Recognition from Drone Videos
(Kholiavchenko et al. 2024) required three weeks to collect, a time-consuming
and expensive endeavor. Our analysis of KABR revealed that a third of the
videos gathered were unusable for the purposes of inferring wildlife behavior.
We analyzed the flight telemetry data from portions of UAV videos that were
usable for inferring wildlife behavior, and demonstrate how these insights can
be integrated into an autonomous remote sensing system to track wildlife in
real time. Our autonomous remote sensing system optimizes the UAV’s actions to
increase the yield of usable data, and matches the flight path of an expert
pilot with an 87% accuracy rate, representing an 18.2% improvement in accuracy
over previously proposed methods.

## Introduction

Figure 1: Autonomous Remote Sensing System for In Situ Imageomics

Imageomics is a new scientific field that aims to understand and analyze
biological traits of organisms, species, populations, and ecosystems from
images (Imageomics 2023). In situ imageomics focuses on studying biological
organisms and systems within the context of their natural environment. Sources
of such datasets include photos and videos captured by experts using camera
traps (Brookes et al. 2023; Zualkernan et al. 2022) , UAVs (Koger et al. 2023;
Andrew, Greatwood, and Burghardt 2019), and citizen scientists, such as
iNaturalist (Horn et al. 2018). As wildlife respond to changing environmental
conditions due to climate change, new approaches to remote sensing data
collection and analysis are required to develop data-driven approaches to
conservation and ecosystem management. Technological advancements in remote
sensing data collection technology, namely camera traps (Brookes et al. 2023;
Tabak et al. 2019) and UAVs (Kholiavchenko et al. 2024; Koger et al. 2023),
have made it easier to gather large volumes of biological data more quickly
than traditional manual techniques. With an imageomics approach, biological
insights can be extracted from these datasets at scale using machine learning
techniques. However, these models require massive volumes of quality data far
exceeding what it typically captured for biological studies. This is
particularly vital for models built to accomplish specific tasks, such as
inferring behavior with the environmental and social context, which is
difficult to infer from camera traps or data gathered by citizen scientists.

UAVs are well suited to gather the volume of data required to train such
machine learning models for specific tasks. UAVs can dynamically track
wildlife to capture fine-grained details, such as behavior, while traversing
remote terrain quickly with little disturbance to the surrounding landscape.
Experts pilots are required to conduct UAV missions tailored to the specific
species and region. The ability to conduct such specialized missions with
autonomous UAVs would significantly reduce the barriers to conduct this
research. Previous works have demonstrated that autonomous UAV missions are 3x
cheaper, safer, more consistent, and reliable than manual flights (Boubin et
al. 2019; Kline et al. 2023).

Existing autonomous UAVs methods are not optimized for in situ imageomics data
collection. Autonomous UAVs may lose track of individuals over large
distances, or fail to capture fine-grain details such as social dynamics. For
both autonomous and manual approaches, crucial time may wasted on collecting
and storing images with poor lighting and viewing angles. Such datasets
require more storage and transport costs, and require more manual pre-
processing before they can be analyzed. Our analysis of a three-day subset of
the KABR dataset (Kholiavchenko et al. 2024) revealed that only 2.2 hours of
video was captured, and of that, only 65% was usable for the mission objective
of inferring wildlife behavior. UAVs have only recently become inexpensive
enough for widespread use in biology applications, so little guidance exists
on optimal strategies for maximizing the usable data yielded from such
missions.

In this work, we analyze the flight telemetry from portions of videos
successfully used to extract behavior data in KABR. We demonstrate how these
insights can be integrated into a remote sensing system to improve the
performance of autonomous UAVs to gather videos of wildlife behavior, with
Kenyan animal behavior as a use-case. Figure 1 illustrates such an autonomous
remote sensing system for in situ imageomics. We show that dynamically
adjusting the altitude and minimizing the movement of the UAV while ensuring
that the wildlife are in view leads to higher usability rate of the collected
videos.

## Methodology

Data | Description | Source
---|---|---
Date & Time | Date and time of video | Telemetry & KABR
Frame | Video frame (30 frames per second) | Telemetry & KABR
Behavior | Behavior of the animal | KABR
Bounding Box Dimensions | Dimensions of the bounding boxes surrounding the animal in pixels | KABR
Speed | Velocity of the UAV in x,y,z direction (meters per second) | Telemetry
Altitude | Altitude of UAV above ground (meters) | Telemetry
Table 1: Telemetry and KABR DatasetKholiavchenko et al. (2024)

We analyzed the flight telemetry data from portions of the UAV videos
successfully used to extract behavior data from KABR: In-Situ Dataset for
Kenyan Animal Behavior Recognition from Drone Videos from Kholiavchenko et al.
(2024). The videos were collected at the Mpala Research Centre in Laikipia,
Kenya using a DJI Air 2S in 4K resolution over a period of three weeks in
January 2023. The dataset includes three species: Plains zebra (Equus quagga),
Grevy’s zebra (Equus grevyi), and reticulated giraffes (Giraffa reticulata).
The KABR dataset includes the bounding box coordinates for each animal in the
frame, the behavior annotations for each bounding box in each frame of the
video, and the frame number and timestamp information. Since we were only
interested in frames that contained usable behavior data, we excluded the
subset of frames containing behavior annotations of “Occluded” “Out of Frame”
or “Out of Focus” from our count of usable frames. Next, we obtained the
telemetry data produced by the manually flown UAV missions. The telemetry data
for each mission began at the time the UAV was powered on and launched and
ended once the UAV was returned to the pilot and powered off. The telemetry
data contains data gathered during the UAV flight which describes the status
of the system, including altitude, speed, direction, and battery level. We
used the frame number and timestamp information to reconcile the behavioral
annotations with the corresponding telemetry data, as shown in Table 1. The
telemetry data used in the analysis can be found here:
https://huggingface.co/datasets/imageomics/KABR-telemetry After reconciling
the available telemetry and behavioral annotations, our final dataset
consisted of 133 minutes of raw videos from 10 missions over three days. Of
this final dataset, only 87 minutes contained video frames of suitable quality
to both detect the animals of interest and determine their behavior, which is
a 65% usability rate. In other words, a third of the video data collected in
the field could not be used to infer animal behavior using computer vision
methods. We analyzed the telemetry data to identify what range of altitudes
and bounding box values produced video that was suitable to be analyzed with
machine learning computer vision methods. Specifically, we focused on computer
visions methods for object detection with YOLO (Jocher, Chaurasia, and Qiu
2023) to automatically localize the animals in the scene and automatic
behavior detection proposed in KABR to automatically extract behavior. Using
these insights, we improved the autonomous navigation UAV model proposed in
Kline et al. (2023) to increase the yield of usable video data. We implemented
the autonomous navigation model from Kline et al. (2023) using the Yolov8n
model (Jocher, Chaurasia, and Qiu 2023) as the object detection model to
localize the animals.

## Results and Discussion

| Altitude (m) | Velocity (m/s) | Bounding Box Width (pixels) | Bounding Box Length (pixels)
---|---|---|---|---
Mean | 17.55 | 0.62 | 106.22 | 110.40
Standard Deviation | 7.80 | 1.31 | 61.65 | 69.01
Minimum | 4.40 | 0.00 | 11.00 | 14.00
25% | 12.70 | 0.00 | 61.00 | 61.67
50% | 14.90 | 0.00 | 92.00 | 96.67
75% | 19.80 | 0.63 | 136.00 | 137.00
Maximum | 71.60 | 12.47 | 583.00 | 767.00
Table 2: Telemetry Data Analysis: Ranges of Optimal Values for UAV altitude, velocity, and animal bounding box sizes in pixels Figure 2: Distribution of altitude, velocity, and bounding box values associated with usable video data for wildlife behavior inference Metric | Original Navigation Model | New & Improved Navigation Model | Improvement
---|---|---|---
% of actions matching original flight | 68.8 | 87.0 | +18.2
F1 Score | 82.1 | 90.4 | +8.3
Table 3: Autonomous Navigation Model Performance Compared to Approach from
(Kline et al. 2023) Figure 3: Example Autonomous Navigation Model Output

We used our telemetry analysis shown in Table 2 and Figure 2 to improve the
autonomous navigation model for tracking wildlife proposed in Kline et al.
(2023). This autonomous navigation model samples the video stream once every
second, and utilizes a YOLO model (Jocher, Chaurasia, and Qiu 2023) to
automatically detect the animals in the frame. Once a zebra or giraffe is
detected, the navigation model uses the bounding box dimensions generated by
YOLO to determine the commands to send to the UAV. For situations when
multiple animals are in the frame, this approach tracks the group by using the
bounding box values to calculate the herd centroid. If this value strays
sufficiently far from the centroid of the UAV’s camera, the navigating model
sends commands to the UAV to adjust its position to return the herd centroid
to the center of the camera’s view. One weakness of this autonomous navigation
approach is that it is constrained to generating commands in x-y plane only
(i.e. ‘fly left/right’ and ‘fly forward/backward’), since this approach
assumes a constant altitude.

Our approach adds the ability to generate commands in z-direction (altitude),
as well as the x and y directions, by integrating the insights from the
telemetry analysis. This new and improved autonomous navigation model includes
a range of preferred altitudes to tune the UAV’s flight path. We also improve
the autonomous navigation model by minimizing the movement of the UAV while
navigating the UAV close enough for the bounding boxes to measure at
approximately $100\times 100$ pixels. This produces a 18.2% increase in
accuracy and a 8.3% increase in the F1 score, as shown in Table 3. Figure 3
shows the output of the autonomous navigation model proposed here. We report
the F1 value, which combines the precision and recall scores of the model’s
predicted commands to the UAV. For this application, ‘positive’ values means
that the UAV is directed to move, while ‘negative’ values means that the UAV
is directed to hover in place. An example of a ‘true positive’ result would be
the model correctly predicting that the next command sent to the UAV should
‘fly left 3 meters’. An example of a ‘false positive’ result would be the
model incorrectly predicting that the next command sent to the UAV should ‘fly
left 3 meters’, when the correct decision was directing the UAV to hover in
place, as determined by the expert pilot executing the manual mission. The
accuracy values reported in Table 3 simply measures the number of predicted
commands that match the original flight path conducted by the expert pilot.
The telemetry dataset is imbalanced in favor of ‘positive’ values, so the F1
score is a better measure of performance compared to accuracy.

As shown in Table 2, the mean altitude of the usable behavior video frames is
17.55 meters with a standard deviation of 7.8 meters. The maximum altitude is
71.60 meters, however the 75th percentile is 19.8 meters, so this maximum is
an outlier. Figure 4 shows the relationship between behavior and altitude,
where behaviors are ordered by how often they occur in the KABR dataset. The
majority of the behavior samples were collected below 30 meters, with rare
behaviors such as ‘urinating’ and ‘defecating’ captured below 20 meters. These
results suggest that behavior is best captured between 10 and 30 meters.

Figure 4: Behavior Distribution by Altitude (behavior labels ordered from most
to least common, left to right)

The mean velocity of usable video frames is 0.62 meters per second, with a
standard deviation of 1.31 meters per second. The distribution of velocity
values is shown in Table 2 and Figure 2. For context, 0.62 meters per second
is approximately 1.4 miles per hour. The 75th percentile is only 0.63 meters
per second, while the maximum velocity is 12.47 meters per second and the
lower percentiles are 0 meters per second (i.e. hovering), suggesting the
maximum value may be skewed by the UAV approaching the wildlife. These results
suggest that behavior is best captured by minimizing the UAV’s movement and
hovering once it reaches a suitable altitude, enabling the generation of
bounding boxes of the appropriate size for behavioral analysis.

The mean bounding box dimensions is approximately $106\times 110$ pixels, with
a standard deviation of $62\times 69$ pixels. The distribution of bounding box
dimensions is shown in Figure 5. The bounding box size is related to the
distance between the UAV and the wildlife. Using the bounding box size as an
approximation for distance is an effective approach to tracking groups of
free-ranging wildlife autonomously, as shown in Kline et al. (2023). This
approach eliminates the need for costly distance calculations, and can
leverage the bounding box values produced by light-weight object detectors
such as the YOLO nano model (Jocher, Chaurasia, and Qiu 2023).

Figure 5: Bounding Box Dimension Distribution

## Conclusion & Future Work

We present guidelines for gathering in situ imageomics wildlife behavior video
data with UAVs and demonstrate how these guidelines can be integrated into an
autonomous UAV navigation model. The techniques proposed in this work can be
applied to others’ UAV wildlife video datasets to determine the optimal
guidelines for additional species. These techniques can also be adapted to
increase of the yield of usable UAV data beyond wildlife behavior studies; for
example, individual identification of wildlife, such as those implemented by
Wildbook (Berger-Wolf et al. 2017). Future work will explore integrating
behavior-adaptive flight data into autonomous UAV navigation models. This
approach can minimize the disturbance from UAV data collection by using a
behavior recognition model to determine if animals are exhibiting signs of
reacting to the UAV’s presence, such as vigilance behaviors or running.
Reinforcement learning approaches can also be used to balance competing
metrics, such as the range of altitudes and minimum desired bounding box size.
These approaches should be validated on free ranging wildlife to confirm the
proposed approaches produce data suitable for analysis with computer vision
techniques for imageomics applications.

## Acknowledgments

This work was supported by NSF Awards No. 2118240 and 2112606. Data was
collected in accordance with Kenya’s National Commission for Science,
Technology & Innovation research license NACOSTI/P/22/18214 and Princeton
University’s Institutional Animal Care and Use Committee 1835F.

## References

  * Andrew, Greatwood, and Burghardt (2019) Andrew, W.; Greatwood, C.; and Burghardt, T. 2019.  Aerial Animal Biometrics: Individual Friesian Cattle Recovery and Visual Identification via an Autonomous UAV with Onboard Deep Inference.  In _2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , 237–243.
  * Berger-Wolf et al. (2017) Berger-Wolf, T. Y.; Rubenstein, D. I.; Stewart, C. V.; Holmberg, J. A.; Parham, J.; Menon, S.; Crall, J.; Oast, J. V.; Kiciman, E.; and Joppa, L. 2017.  Wildbook: Crowdsourcing, computer vision, and data science for conservation.  arXiv:1710.08880.
  * Boubin et al. (2019) Boubin, J.; Chumley, J.; Stewart, C.; and Khanal, S. 2019.  Autonomic Computing Challenges in Fully Autonomous Precision Agriculture.  In _2019 IEEE International Conference on Autonomic Computing (ICAC)_ , 11–17.
  * Brookes et al. (2023) Brookes, O.; Mirmehdi, M.; Kühl, H.; and Burghardt, T. 2023.  Triple-stream Deep Metric Learning of Great Ape Behavioural Actions.  _arXiv:2301.02642 [cs]_ , (arXiv:2301.02642).  ArXiv:2301.02642 [cs].
  * Horn et al. (2018) Horn, G. V.; Aodha, O. M.; Song, Y.; Cui, Y.; Sun, C.; Shepard, A.; Adam, H.; Perona, P.; and Belongie, S. 2018.  The iNaturalist Species Classification and Detection Dataset.  In _CVPR_.
  * Imageomics (2023) Imageomics. 2023.  About the Imageomics Institute (NSF OAC-2118240).
  * Jocher, Chaurasia, and Qiu (2023) Jocher, G.; Chaurasia, A.; and Qiu, J. 2023.  YOLO by Ultralytics.
  * Kholiavchenko et al. (2024) Kholiavchenko, M.; Kline, J.; Ramirez, M.; Stevens, S.; Sheets, A.; Babu, R. R.; Banerji, N.; Campolongo, E.; Thompson, M.; Tiel, N. V.; Miliko, J.; Bessa, E.; Duporge, I.; Berger-Wolf, T.; Rubenstein, D.; and Stewart, C. 2024.  KABR: In-Situ Dataset for Kenyan Animal Behavior Recognition from Drone Videos.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_.
  * Kline et al. (2023) Kline, J. M.; Stewart, C.; Berger-Wolf, T.; Stewart, C. V.; and Rubenstein, D. I. 2023.  A Framework for Autonomic Computing for In Situ Imageomics.  In _4th IEEE International Conference on Autonomic Computing and Self-Organizing Systems - ACSOS 2023_.
  * Koger et al. (2023) Koger, B.; Deshpande, A.; Kerby, J.; Graving, J.; Costelloe, B.; and Couzin, I. 2023.  Quantifying the movement, behaviour and environmental context of group-living animals using drones and computer vision.  https://onlinelibrary.wiley.com/doi/abs/10.1111/1365-2656.13904.
  * Tabak et al. (2019) Tabak, M. A.; Norouzzadeh, M. S.; Wolfson, D. W.; Sweeney, S. J.; Vercauteren, K. C.; Snow, N. P.; Halseth, J. M.; Di Salvo, P. A.; Lewis, J. S.; White, M. D.; Teton, B.; Beasley, J. C.; Schlichting, P. E.; Boughton, R. K.; Wight, B.; Newkirk, E. S.; Ivan, J. S.; Odell, E. A.; Brook, R. K.; Lukacs, P. M.; Moeller, A. K.; Mandeville, E. G.; Clune, J.; and Miller, R. S. 2019.  Machine learning to classify animal species in camera trap images: Applications in ecology.  _Methods in Ecology and Evolution_ , 10(4): 585–590.
  * Zualkernan et al. (2022) Zualkernan, I.; Dhou, S.; Judas, J.; Sajun, A. R.; Gomez, B. R.; and Hussain, L. A. 2022.  An IoT System Using Deep Learning to Classify Camera Trap Images on the Edge.  _Computers_ , 11(11): 13.


# Tackling Diverse Minorities in Imbalanced Classification

Kwei-Herng Lai
Daochen Zha<EMAIL_ADDRESS>Rice University Houston, TXUSA ,  Huiyuan Chen
Mangesh Bendre<EMAIL_ADDRESS>Visa ResearchPalo Alto, CAUSA ,  Yuzhong Chen
Mahashweta Das
Hao Yang<EMAIL_ADDRESS>Visa ResearchPalo Alto, CAUSA and  Xia Hu
<EMAIL_ADDRESS>Rice UniversityHouston, TXUSA

(2023)

###### Abstract.

Imbalanced datasets are commonly observed in various real-world applications,
presenting significant challenges in training classifiers. When working with
large datasets, the imbalanced issue can be further exacerbated, making it
exceptionally difficult to train classifiers effectively. To address the
problem, over-sampling techniques have been developed to linearly
interpolating data instances between minorities and their neighbors. However,
in many real-world scenarios such as anomaly detection, minority instances are
often dispersed diversely in the feature space rather than clustered together.
Inspired by domain-agnostic data mix-up, we propose generating synthetic
samples iteratively by mixing data samples from both minority and majority
classes. It is non-trivial to develop such a framework, the challenges include
source sample selection, mix-up strategy selection, and the coordination
between the underlying model and mix-up strategies. To tackle these
challenges, we formulate the problem of iterative data mix-up as a Markov
decision process (MDP) that maps data attributes onto an augmentation
strategy. To solve the MDP, we employ an actor-critic framework to adapt the
discrete-continuous decision space. This framework is utilized to train a data
augmentation policy and design a reward signal that explores classifier
uncertainty and encourages performance improvement, irrespective of the
classifier’s convergence. We demonstrate the effectiveness of our proposed
framework through extensive experiments conducted on seven publicly available
benchmark datasets using three different types of classifiers. The results of
these experiments showcase the potential and promise of our framework in
addressing imbalanced datasets with diverse minorities.

Data Augmentation, Imbalanced Classification, Anomaly Detection

††journalyear: 2023††copyright: acmlicensed††conference: Proceedings of the
32nd ACM International Conference on Information and Knowledge Management;
October 21–25, 2023; Birmingham, United Kingdom††booktitle: Proceedings of the
32nd ACM International Conference on Information and Knowledge Management
(CIKM ’23), October 21–25, 2023, Birmingham, United Kingdom††price:
15.00††doi: 10.1145/3583780.3615071††isbn:
979-8-4007-0124-5/23/10††conference: CIKM’23: ACM International Conference on
Information and Knowledge Management; October 21– October 25, 2023;
Birmingham, UK††ccs: Computing methodologies Supervised learning by
classification††ccs: Computing methodologies Anomaly detection††ccs: Computing
methodologies Sequential decision making

##  1\. Introduction

Imbalanced data with a disproportionate ratio of training samples between
classes widely exists in many real-world datasets and their corresponding
applications, such as intrusion detection from server log (Kim et al., 2020),
fault detection in manufacturing (Hsu and Liu, 2021), and fraud detection in
finance (Al-Hashedi and Magalingam, 2021). In real-world scenarios, where the
dataset is vast, addressing the issue of data imbalance becomes exceptionally
difficult. The scale of the dataset often leads to extreme imbalances between
classes, making it challenging to tackle effectively. The dominance of the
majority class tends to overshadow the classifiers’ capacity to accurately
classify instances from the minority class. As a result, the performance of
the classifiers suffers, leading to subpar outcomes specifically for the
minority class.

To tackle the problem, over-sampling techniques are developed to directly
modify the dataset by injecting synthetic instances. One of the most
representative over-sampling techniques in the literature is SMOTE (Chawla et
al., 2002), which generates synthetic samples by performing linear
interpolation between minority instances and their neighbors. Alternative
solutions with under-sampling techniques (Yen and Lee, 2006; Liu et al., 2020)
remove existing majorities from the dataset to balance the majorities and
minorities. The under-sampling techniques may significantly lose information
and lead to poor performances when data is extremely imbalanced. Recent study
(Zha et al., 2022) suggests that over-sampling techniques are generally
favorable to extreme imbalance settings and show the promise of over-sampling
techniques.

Figure 1. Comparison between synthetic oversampling (left) and mix-up (right).

However, in real-world application scenarios such as anomaly detection,
minorities are seldom clustered within a certain distribution and may be
scattered diversely in the feature space. This will lead to unsatisfactory
performances due to the diverse distribution of minorities. Take the SMOTE
(Chawla et al., 2002; Siriseriwan and Sinapiromsaran, 2017; Nguyen et al.,
2011) as an example. As shown in the left side of Figure 1, if we perform
synthetic oversampling based on the abnormal accounts A and B, it is likely to
create a synthetic account C labeled as abnormal but with normal transaction
numbers and transaction amount. As a result, account C interleaves with the
majority of normal accounts, which may degrade the classification performance.

In parallel, instead of creating synthetic samples with minorities, data mix-
up (Zhang et al., 2018; Zha et al., 2023) techniques in the image domain
(Dabouei et al., 2021) have achieved great success in augmenting the training
data by mixing up data samples from two different classes such that fine-
grained supervisory signals of label differences between the two classes can
be captured. Recently, the mix-up technique has been extended to the imbalance
classification (Kabra et al., 2020), which outperforms synthetic minority
oversampling methods when minorities are generally within clusters. In light
of the previous success, we consider synthetic instance generation based on
_both minority and majority instances_ to address the diverse distribution of
minorities. The right of Figure 1 shows how mix-ups can help. A pair of normal
and abnormal accounts with a properly chosen mix-up strategy can generate not
only synthetic minorities but also synthetic majorities to better shape the
boundary between abnormal and normal accounts. Motivated by this, this work
aims to answer the following research question: _How can we leverage the mix-
up technique to better capture the limited supervisory signal from diverse
minorities?_

Nevertheless, it is challenging to develop a mix-up framework for the
following reasons. First, it is challenging to identify the source samples for
the mix-up. Randomly choosing two data samples for mix-up is likely to create
noise due to the extreme imbalance between minorities and majorities. The
noisy samples can harm the classification performance. Additionally, even if
we apply some strategies (e.g., only mixing up between anomalies), it is still
likely to create noise due to the diverse distribution of minorities. Second,
even if we are able to get a proper pair of source samples, how to mix up
remains to be a non-trivial problem. Randomly selecting a mix-up ratio may end
up creating noises. We argue that the mix-up ratio needs to be tailored for
the source samples. For example, if we select a pair of source samples with a
majority sample in the center of majorities and a minority sample that is
surrounded by majorities, a mix-up ratio that weighs more on minority samples
may create a synthetic sample that better shapes the decision boundary.
Conversely, for a pair of source samples with a majority sample that is
deviated from most majorities and a minority sample that is nearby other
minorities, a mix-up ratio that weighs more on the majority sample may lead to
a more informative synthetic sample. Third, even if we have a strategy to mix
up data samples, simultaneously considering the underlying classifier and
traversing the feature space to search for the most informative spot for
synthetic sample generation is still challenging. For example, how can we know
if the created sample can further benefit the formation of a model decision
boundary? Likewise, if we are able to find a sweet spot for model training,
how to guide the synthetic data generation process remains a challenging
problem.

To tackle the challenges above, we propose MixAnN, a universal data mixer that
is capable of incorporating different classifiers to explore and exploit
potentially beneficial information from extremely scarse and diverse
distributed minorities for training a classifier. Motivated by the success of
deep reinforcement learning and its applications on imbalanced classification
problem (Zha et al., 2022; Liu et al., 2020), we define an iterative mix-up
process that aims at iteratively creating synthetic samples by traversing in
the feature space through the nearest neighbor exploration. Then, we formulate
the iterative mix-up into a Markov decision process (MDP) and design an action
space to perform an automated selection of mix-up ratio with a tailored reward
function that provides model uncertainty and performance improvement to guide
feature space traversal. Finally, to solve the MDP, we tailor a deep actor-
critic framework to tackle the discrete-continuous action space and train a
mix-up policy to optimize the reward. Our contributions are summarized as
follows:

  * •

We propose an iterative mix-up process to generalize label information by
traversing the feature space.

  * •

We formulate the iterative mix-up into a Markov decision process and design a
combinatorial reward signal, which considers the convergence of the classifier
while providing model uncertainty, to guide the mix-up process.

  * •

We tailor a deep reinforcement learning algorithm to learn a mix-up policy for
the discrete-continuous action space.

  * •

We conduct extensive experiments to show that MixAnN outperforms the existing
data augmentation approaches and state-of-the-art label-informed anomaly
detectors on 7 publicly available benchmark datasets.

##  2\. Related Work

Here we briefly review the related work on anomaly detection, imbalanced
classification and mix-up strategies.

###  2.1. Label-informed Anomaly Detection

Weakly/semi-supervised anomaly detection (Ienco et al., 2016; Pang et al.,
2019a, b; Zhao and Hryniewicki, 2018; Ruff et al., 2020; Lai et al., 2023) are
two main strategies to tackle the problem when either normal or anomaly
samples are labeled. To leverage the large number of labeled normal samples,
SAnDCat (Ienco et al., 2016) selects top-K representative samples from the
dataset as a reference to evaluate anomaly scores based on a model learned
from pair-wise distances between labeled normal instances. On the other hand,
to exploit a limited number of labeled anomalies, DevNet (Pang et al., 2019a)
enforces the anomaly scores of individual data instances to fit a one-sided
Gaussian distribution for leveraging prior knowledge of labeled normal samples
and anomalies. PRO (Pang et al., 2019b) introduces a two-stream ordinal-
regression network to learn the pairwise relations between two data samples,
which is assumption-free on the probability distribution of the anomaly
scores.

Recently, several endeavors (Zhao and Hryniewicki, 2018; Ruff et al., 2020)
further generalize the label-informed anomaly detection problem into semi-
supervised classification setting (Chapelle et al., 2009) that limited numbers
of both normal and anomaly samples are accessible. The main underlying
assumptions are that similar points are likely to be of the same class and,
therefore, densely clustered in low-dimensional feature space. XGBOD (Zhao and
Hryniewicki, 2018) extracts feature representation based on the anomaly score
of multiple unsupervised anomaly detectors for training a supervised gradient
boosting tree classifier. DeepSAD (Ruff et al., 2020) points out that the
semi-supervision assumptions only work for normal samples and further develops
a one-class classification framework to cluster labeled normal samples while
maximizing the distance between the labeled anomalies and the cluster in the
high-dimensional space. However, weakly/semi-supervised learning methods focus
on modeling the given label information without considering the relations
between two labeled instances. Therefore, it is infeasible to generalize the
label information when anomaly behaviors are diverse. By considering
correlations between labeled samples and generating beneficial training data
correspondingly, we are able to generalize the label information for training
arbitrary classifiers.

###  2.2. Synthetic Oversampling for Imbalanced Classification

Data augmentation has been extensively studied for a wide range of data types
(Wen et al., 2020; Shorten and Khoshgoftaar, 2019; Ling et al., 2023b; Chen et
al., 2022b; Wang et al., 2023) to enlarge training data size and generalize
model decision boundaries for improving performance and robustness. Approaches
to solving the imbalanced classification problem can be categorized into two
types: algorithm-wise and data-wise. Algorithm-wise approaches directly tailor
the loss function of classification models (Liu and Zhou, 2006; Cao et al.,
2019; Li et al., 2021) to better fit the data distribution. However, modifying
the loss function only facilitates the fitting of label information and may
suffer from generalizing label information when the behavior between minority
classes varies dramatically. Data-wise approaches generate new samples into
the minority class (Chawla et al., 2002; He et al., 2008) or remove existing
samples from majority classes. Synthetic Minority Oversampling (SMOTE) (Chawla
et al., 2002) generates new minority samples by linearly combining a minority
sample with its k-nearest minority instances with a manually selected
neighborhood size and the number of synthetic instances. A series of
advancements on SMOTE (Kovács, 2019) further introduce density estimation (He
et al., 2008; Bunkhumpornpat et al., 2012), data distribution-aware sampling
(Han et al., 2005), and automated machine learning (Zha et al., 2022) to
tackle the class imbalance problem without manual selection of neighborhood
size and the number of synthetic instances.

###  2.3. Mix-up

Recently, instead of conducting synthetic data sampling on a single class,
Mixup (Zhang et al., 2018; Han et al., 2022b; Chen et al., 2022a; Ling et al.,
2023a) achieves a significant improvement in the image domain by synthesizing
data points through linearly combining two random samples from different
classes with a given combination ratio and creating soft labels for training
the neural networks. As Mixup assumes that all the classes are uniformly
distributed for image classification, it does not apply when the class
distribution is skewed. To tackle this limitation, MixBoost (Kabra et al.,
2020) introduces a skewed probability distribution to sample the ratio for
linearly combining two heterogeneous samples. However, the imbalanced
classification problem assumes that minority samples are clustered within the
feature space, which may not be true when the minorities are anomalies. To
this end, our work considers the attributes of a pair of majority and minority
samples for jointly identifying the best k-nearest neighborhood and
combination ratios. Then, we generate the synthetic samples with the combined
ratios and identify the next pair of samples within the k-nearest
neighborhood. In this way, our framework is capable of exploiting the label
information while exploring the diversely distributed minorities.

Figure 2. An illustration of the iterative mix-up process. The background
colors indicate the model decision boundary. The attributes of the source
samples (purple circled) are considered to specify the composition ratio
$\alpha$ for oversampling the synthetic sample (black points) with
corresponding labels (outer circle of black points) $n$ times. In the
meantime, a $k$-nearest neighborhood is identified for randomly sampling the
next pair of source samples. Finally, the process iterates to the next round
with the new pair of source samples.

##  3\. Methodology

Figure 3 depicts an overview of our framework. We establish our framework by
forming a classifier-driven Markov decision process with a data mixer that
learns to generate synthetic samples for training the underlying classifier.
In this section, we formally define the problem of strategic data augmentation
in imbalanced classification with diversely distributed minorities. Then, we
elaborate on the details of the Markov decision process, including the reward
signal for iterative data mix-up and a tailored sequential decision problem
solver. Finally, we provide algorithm and implementation details.

###  3.1. Problem Statement

A training dataset
$\mathcal{X}^{\text{train}}=\\{x_{1},x_{2},\dots,x_{n+m}\\}$ is composed of as
set of labeled majorities $\mathcal{N}=\\{x_{1},x_{2},\dots,x_{n}\\}$ and a
set of labeled minorities $\mathcal{A}=x_{n+1},x_{n+2},\dots,x_{n+m}\\}$,
where $n\gg m$, and the minorities can be anomalies that are diversely
distributed. The goal of imbalanced classification is to learn a classifier
$\phi:\mathcal{X}\rightarrow\mathbb{R}$ which evaluates the probability of
individual data points as majority or minority in the given dataset
$\mathcal{X}$ via exploiting the prior knowledge of labeled majorities
$\mathcal{N}$ and minorities $\mathcal{A}$, so that
$\sum_{i\in\mathcal{A}}{\phi(x_{i})}$ is maximized while
$\sum_{j\in\mathcal{N}}{\phi(x_{j})}$ is minimized with the constraints of
$\phi(x_{i})>0.5$ and $\phi(x_{j})\leq 0.5$.

To better generalize the knowledge from the label information, we formally
define the problem of strategic data augmentation as follows. Given a dataset
$\mathcal{X}^{\text{train}}=\\{\mathcal{N},\mathcal{A}\\}$ where
$\mathcal{X}^{\text{train}}\in\mathbb{R}^{(n+m)\times d}$, with a supervised
classifier $\phi$, we target on augmenting the $\mathcal{X}^{\text{train}}$
with a synthetic dataset $\mathcal{X}^{\text{syn}}$ according to the $\phi$,
where the synthetic dataset $\mathcal{X}_{\text{syn}}\in\mathbb{R}^{l\times
d}$ is generated via mixing up samples from $\mathcal{N}$ with samples from
$\mathcal{A}$. Specifically, our objective is to properly sample pairs of data
instances from $\mathcal{N}$ and $\mathcal{A}$ with the corresponding mix-up
ratio $\alpha$ to create synthetic instances
$x^{\text{syn}}\in\mathcal{X}^{\text{syn}}$, such that the performance of
$\phi$ can be maximized by being trained on
$\hat{\mathcal{X}}^{\text{train}}=\\{\mathcal{X}^{\text{train}}\cup\mathcal{X}^{\text{syn}}\\}$.

Figure 3. An illustration of our framework. In each step, a pair of a normal
sample and an anomaly is inputted to the data mixer. The corresponding action
(i.e., neighborhood size $k$, composition ratio $\alpha$, number of
oversampling $n$, and termination probability $\epsilon$) is then generated to
create synthetic samples. After that, the synthetic samples are adopted to
train the classifier and yield the reward signals (performance improvement
$\Delta\mathcal{M}(\phi_{t})$ and model confidence $\mathcal{C}(\phi_{t}|.)$)
for updating the data mixer.

###  3.2. Iterative Mix-up Process

To generalize label information from two different classes, Mixup (Zhang et
al., 2018) performs synthetic data generation over two samples from different
classes, which has been extensively studied to augment image and textual data.
The core idea of Mixup is to linearly combine two samples as follows:

(1) $x_{\text{syn}}=\alpha*x_{0}+(1-\alpha)*x_{1},$

where $x_{0}$ and $x_{1}$ are the two selected source samples and
$\alpha\in[0.0,1.0]$ controls the composition of $x_{\text{syn}}$. Although
existing works (Zhang et al., 2018; Kabra et al., 2020) generate a soft label
of $x_{\text{syn}}$ in the same fashion for the imbalance classification
problem, the diverse behaviors of the anomalies lead to similar labels with
high granularity on diverse synthetic samples, which may prompt the model to
over-fit on noisy synthetic labels. To this end, instead of generating soft
labels, we synthesize hard labels for $x_{\text{syn}}$ with a threshold $\eta$
as follows:

(2) $y_{\text{syn}}=\begin{cases}y_{0},&\alpha\geq\eta,\\\
y_{1},&\text{otherwise},\end{cases}\vspace{-5pt}$

where $0.5\leq\eta\leq 1.0$. Due to the diverse behavior of anomalies,
arbitrarily mixing up two random source samples from the $X^{train}$ may lead
to noisy samples. To tackle this problem, we seek to identify a meaningful
pair of samples for synthesizing new samples. As normal samples are often
concentrated in the latent space (Ruff et al., 2020), inspired by the
BorderlineSMOTE (Han et al., 2005) that shows that borderline samples are more
informative, we propose to traverse the feature space of $X^{\text{train}}$
with the guidance of the decision boundary of the model $\phi$ for synthetic
oversampling. Specifically, given a pair of arbitrary source samples, we
consider the attributes of the two samples to identify corresponding $\alpha$
and number of oversampling for generating a set of
$x_{syn}\in\mathcal{X}^{\text{syn}}$. Meanwhile, an optimal range for
uniformly sampling the next pair of source samples is identified according to
the model impact for iterating to the next round of the mix-up process. The
intuition behind uniform sampling is to consider the relationship between the
attributes of the source samples and their entire neighborhood information
instead of focusing on a certain sample in the neighborhood. Figure 2
illustrates an example of the iterative mix-up process. Based on the
attributes of source samples $x_{0}$ and $x_{1}$, we first specify the
composition ratio $\alpha=4$ and the number of oversampling $n=1$ for creating
synthetic sample $x_{\text{syn}}$. Then, a $k$-nearest neighborhood is
identified for sampling the next pair of source samples $x_{2}$ and $x_{3}$
for the next round of mix-up.

The proposed iterative mix-up process has several desirable properties. First,
it can make personalized decisions. For example, we can generate more samples
for some instances and fewer for other instances. Second, it can incorporate
various information to guide the mix-up process. For instance, data attributes
and model impact can be considered and serve as guidance for generating
samples. Third, by simultaneously considering the model impact with the
feature distribution, we directly generate information that is missing in the
original dataset but beneficial for model training.

###  3.3. Formulating Iterative Mix-up as Markov Decision Process

The iterative mix-up process can be formulated as a Markov decision process
with a quintuple $(\mathcal{S},\mathcal{A},\mathcal{T},\mathcal{R},\gamma)$,
where $\mathcal{S}$ is a finite set of states, $\mathcal{A}$ is a finite set
of actions, $\mathcal{T}:\mathcal{S}\times\mathcal{A}\to\mathcal{S}$ is the
state transition function that maps the current state $s$, action $a$ to the
next state $s^{\prime}$,
$\mathcal{R}:\mathcal{S}\times\mathcal{A}\to\mathbb{R}$ is the immediate
reward function that reflects the quality of action $a$ for the state $s$, and
$\gamma$ is a decade factor to gradually consider the future transitions.
Figure 3 illustrates the Markov decision process of the framework. To sample-
wise tailor the mix-up strategy, we target learning a data mixer that maps the
attributes of source data samples into an augmentation strategy for creating
synthetic samples while exploring model decision boundaries. The MDP can be
defined as follows:

  * •

State Space ($\mathcal{S}$): At each timestamp $t$, state
$s_{t}\in\mathcal{S}$ is defined as $s_{t}=(x_{0}^{t},x_{1}^{t})$, where
$s_{t}\in\mathbb{R}^{2m}$ is a concatenation of two $m$-dimensional feature
vectors of the two source samples. Therefore, the state space is defined as
$\mathcal{S}=\\{(x_{0}^{t},x_{1}^{t})|x_{0}^{t},x_{1}^{t}\in\mathcal{X}^{train}\\}$.

  * •

Action Space ($\mathcal{A}$): At each timestamp $t$, the action
$a_{t}\in\mathcal{A}$ where $a_{t}=(k,\alpha,n,\epsilon)$ is a vector composed
of the size of neighborhood $k$, composition ratio $\alpha$, number of
oversampling $n$ and the termination probability of the iterative mix-up
process $\epsilon$. Therefore, the action space is defined as a discrete-
continuous space
$\mathcal{A}=\\{(k_{t},\alpha_{t},n_{t},\epsilon_{t})|k_{t},n_{t},\epsilon_{t}\in\mathbb{N},\alpha_{t}\in\mathbb{R}^{+}\\}$.

  * •

Transition Function ($\mathcal{T}$): Given a state
$s_{t}=(x_{0}^{t},x_{1}^{t})$ and an action $a_{t}=(k,\alpha,n,\epsilon)$, the
transition function firstly adopts $\alpha$ with Eq. 1 and Eq. 2 to oversample
$x_{\text{syn}}$ for $n$ times. Then, the resulting synthetic samples
$\mathcal{X}_{\text{syn}}$ will be adopted for training the classifier and
lead to a classifier $\phi_{t}$ in timestamp $t$. Finally, the transition
function shifts to the next state $s_{t+1}=(x_{0}^{t+1},x_{1}^{t+1})$ where
the $x_{0}^{t+1}$ is randomly sampled from the $k$-nearest neighborhood of the
$x_{\text{syn}}$ and the $x_{1}^{t+1}$ is identified as the nearest data point
with a different label from $x_{0}^{t+1}$.

  * •

Reward Function ($\mathcal{R}$): The reward signal $r_{t}$ for each timestamp
$t$ is designed to encourage performance improvement while exploring the
decision boundaries of the classifier $\phi$. Therefore, the reward function
is defined as:

$\mathcal{R}(s_{t},a_{t})=\lambda*\Delta\mathcal{M}(\phi_{t})*\mathcal{C}(\phi_{t}|s_{t},a_{t}),$

where $\lambda$ is a hyperparameter to define the strength of the reward
signal, $\mathcal{M}$ is an evaluation metric, and
$\Delta\mathcal{M}(\phi_{t})$ measures the performance improvement of
$\phi_{t}$. The $\mathcal{C}(\phi_{t}|s_{t},a_{t})$ evaluates the model
confidence to encourage exploring the decision space of $\phi_{t}$. This way,
the reward signal drives the data mixer to explore the classifier while
achieving maximum improvement with the newly synthesized data samples. The
implementation details are provided in section 3.5.

###  3.4. Solving Markov Decision Process with Deterministic Actor-Critic

To solve the MDP above, we define a parameterized policy $\pi_{\theta}$ as the
data mixer to maximize the reward signal of the MDP where the ultimate goal is
to learn an optimal policy $\pi_{\theta}^{*}$ that maximize the cumulative
reward $\mathbb{E}_{\pi}[\sum_{t=0}^{\infty}\gamma^{t}r_{t}]$. However, the
action space of the iterative mix-up process is a discrete-continuous vector,
and the reward signals generated from an under-fitted $\phi_{t}$ may be in-
stable. To this end, we employ the deep deterministic policy gradient (DDPG)
(Lillicrap et al., 2015), an actor-critic framework that equips with two
separate networks: actor and critic. The critic network
$Q_{(}s_{t},a_{t}|\theta_{2})$ approximates the reward signal for a state-
action pair from the MDP, while the actor network $\pi_{(}s_{t}|\theta_{1})$
aims to learn the policy for a given state $s_{t}$ based on the critic
network. We note that an advanced actor-critic framework such as soft actor-
critic (Haarnoja et al., 2018) could be adopted to learn the
$\pi_{\theta}^{*}$, which will be our future exploration.

To perform a continuous action, DDPG learns an actor network
$\pi(s_{t}|\theta_{1})$ that deterministically maps a given state $s_{t}$ to
an action vector $a_{t}$ and trains the network by maximizing the approximated
cumulative reward that generated by the critic network $Q(.|\theta_{2})$.
Specifically, given $N$ transitions, a projected action
$\pi(s_{t}|\theta_{1})$ can be generated as the input of the critic to
minimize the following loss function:

(3)
$L_{\pi}(\theta_{1})=-\frac{1}{N}\sum_{i=1}^{N}Q(s_{i},\pi(s_{i})|\theta_{2}),$

where the action $\pi(s_{i})$ is a $4$-dimensional real number vector. To
fully leverage the expressive power of the deep neural network during the
training while outputting a discrete-continuous vector for the MDP, we
transform the continuous action vector $\pi(s_{i})$ with a sigmoid function
and yield the action vector $a_{t}=w\circ\sigma(\pi(s_{i}|\theta_{1}))$ where
$w$ specifies the value constraints of individual entries. For instance, if
the maximum for the $k$ and $n$ are $10$ and $5$, then $w=[10,1,5,1]$ since
$\alpha$ and $\epsilon$ are expected to be ranging from $0$ to $1$. The
outcome for the $k$ and $n$ are rounded to the nearest integer.

To tackle the in-stable reward signal issue, DDPG approximates the reward
signal with the critic network $Q_{(}.|\theta_{2})$ and trains the networks in
an off-policy fashion. It introduces a replay buffer to store historical and
randomly sampled transitions to minimize the temporal correlation between two
transitions for learning across a set of uncorrelated transitions.
Specifically, the critic network $Q_{(}.|\theta_{2})$ maps a state-action pair
into a real value $y_{t}$ via minimizing the following loss function:

(4) $L_{Q}(\theta_{2})=[Q(s_{t},a_{t})-b_{t}]^{2}$

Here, $b_{t}=\mathcal{R}(s_{t},a_{t})+\gamma
Q(s_{t+1},\pi(s_{t+1}|\theta_{1})|\theta_{2})$ is a signal derived from the
Bellman equation (Sutton and Barto, 2018) which considers the recursive
relation between the current reward and the future approximated reward signals
for maximizing cumulative reward. $\pi(s_{t+1}|\theta_{1})$ is an action from
the actor network and the $\gamma$ is the decade factor.

Algorithm 1 Training Procedure

1: Input: Input data $X^{\text{train}}$, maximum neighborhood size $K$, number
of training episode $E$, DDPG training step $S$, reward coefficient $\lambda$,
window size for baseline reward $m$.

2: Initialize classifier $\phi$, actor network $\pi$, critic network $Q$,
memory buffer $\mathcal{B}$, $\epsilon=0$.

3: Split partial $X^{\text{train}}$ to form $X^{\text{val}}$ and train $\phi$
on $X^{\text{train}}$.

4: Pre-compute $K$-nearest neighborhood for $X^{train}$.

5: Randomly sample a pair of source samples to form the initial state
$s_{0}=(x_{0},x_{1})$.

6: for $e$ = $0,1,2...,E$ do

7: $t\leftarrow 0$.

8: while $\epsilon<0.5$ do

9: Get the action $a_{t}=\pi(s_{t})$ from actor network.

10: Get $\alpha$, $n$ from $a_{t}$ and adopt Eq. 1, Eq. 2 to generate
$x_{\text{syn}}$.

11: Update $\phi$ with $x_{\text{syn}}$ and get $k$ from $a_{t}$.

12: Obtain $k$-nearest neighborhood of $x_{\text{syn}}$.

13: Form the reward $r_{t}=\lambda\Delta\mathcal{M}(.)\mathcal{C}(.)$ with
$X^{val}$ and the $k$-nearest neighborhood of $x_{\text{syn}}$.

14: Randomly choose next pair of source samples $(x_{0}^{t+1},x_{1}^{t+1})$
from the $k$-nearest neighborhood of $x_{\text{syn}}$

15: Form the next state $s_{t+1}=(x_{0}^{t+1},x_{1}^{t+1})$ and get $\epsilon$
from $a_{t}$.

16: Store the triplet $T_{t}=(s_{t},a_{t},s_{t+1},r_{t})$ into $\mathcal{B}$.

17: $t\leftarrow t+1$.

18: for step = $1$, $2$, .., $S$ do

19: Use data in $\mathcal{B}$ to update DDPG with Eq. 3 and Eq. 4.

20: end for

21: end while

22: end for

###  3.5. Algorithm Details

Algorithm 1 illustrates the training procedure. We design two components to
form the reward signal: improvement stimulation $\Delta\mathcal{M}(\phi_{t})$
and model exploration $P(\phi_{t}|s_{t},a_{t})$. To learn an optimal policy
for the target tasks, existing solutions (Pang et al., 2021; Wu and Ortiz,
2021; Zha et al., 2020) directly adopt the performance on a validation dataset
as a reward signal. However, as the convergence of the underlying classifier
is not guaranteed, directly learning a policy with the performance on a
validation set may lead to noisy reward signals. As a result, rather than
using the current model’s performance $\phi_{t}$, we design an improvement
stimulation to pursue the maximum model improvement on the validation set with
a baseline performance:

(5)
$\Delta\mathcal{M}(\phi_{t})=\mathcal{M}(\phi_{t}(\mathcal{X}^{\text{val}}),y^{\text{val}})-\frac{\sum_{i=t-m}^{t-1}\mathcal{M}(\phi_{i}(\mathcal{X}^{\text{val}}),y^{\text{val}})}{m-1},$

where
$\frac{\sum_{i=t-m}^{t-1}\mathcal{M}(\phi_{i}(\mathcal{X}^{\text{val}}),y^{\text{val}})}{m-1}$
is a baseline performance for the timestamp $t$, and $m$ controls the sample
size for the baseline.

In addition, as we aim at creating synthetic samples by mixing normal samples
and anomalies while iteratively training the classifier $\phi$, it is critical
to explore model decision boundaries to create beneficial samples and prevent
generating noisy samples. Inspired by the confidence quantification (Perini et
al., 2020) for anomaly detectors, we develop a model exploration signal to
quantify the instance-wise prediction uncertainty as follows:

(6)
$\mathcal{C}(\phi_{t}|s_{t},a_{t})=\frac{1}{k}\sum_{i=0}^{k}P(y_{i}=0|x_{i},\phi_{t})P(y_{i}=1|x_{i},\phi_{t}),$

where $x_{i}$ is the k-nearest neighbor of the synthesized sample in timestamp
$t$ because k is the size of the nearest neighborhood specified by $a_{t}$.
The intuition behind this is to encourage the data mixer to explore the
uncertain area in the feature space.

##  4\. Experiment

Our experiments aim to answer the following research questions: RQ1: How does
the proposed framework compare against the existing data augmentation methods?
RQ2: How does the framework compare against the existing label-informed
anomaly detection methods? RQ3: How does each component contribute to the
performance of the proposed framework? RQ4: How do the key hyperparameters
affect the model performance? RQ5: How does the learned strategy of the
proposed framework compare with the existing approaches?

###  4.1. Experiment Settings

We conduct horizontal analysis to compare the proposed framework with data
augmentation methods on three different classifiers. We also conduct a
vertical analysis that compares the proposed framework with label-informed
anomaly detectors. The details of the experimental settings are provided as
follows:

| # Samples | # Features | % Anomaly | Domain
---|---|---|---|---
Japanese Vowels | 1,456 | 12 | 3.43% | Utterance
Annthyroid | 7,200 | 22 | 7.42% | Clinical Record
Mammography | 11,183 | 6 | 2.32% | Medical Image
Satellite | 6,435 | 36 | 31.63% | Remote Sensing
SMTP | 95,156 | 41 | 0.03% | Server Log
CIFAR-10 | 5,263 | 512 | 5.00% | General
FashionMNIST | 6,315 | 512 | 5.00% | Fashion
Table 1. Dataset statistics.

Datasets: Table 1 summarizes the statistics of the datasets. We adopt 5
benchmark datasets from different domains. Japanese Vowels contains utterances
of /ae/that were recorded from nine speakers with 12 LPC cepstrum
coefficients. The goal is to identify the outlier speaker. Annthyroid is a set
of clinical records that record 21 physical attributes of over 7200 patients.
The goal is to identify the patients that are potentially suffering from
hypothyroidism. Mammography is composed of 6 features extracted from the
images, including shape, margin, density, etc. The goal is to identify
malignant cases that could potentially lead to breast cancer. Satellite
contains the remote sensing data of 6,435 regions, where each region is
segmented into a 3x3 square neighborhood region and is monitored by 4
different light wavelengths captured from the satellite images of the Earth.
The goal is to identify regions with abnormal soil status. SMTP has 95,156
server connections with 41 connection attributes, including duration,
src_byte, dst_byte, and so on. The task is to identify malicious attacks from
the connection log. CIFAR-10 and FashionMNIST are the benchmark datasets. We
follow the setting of previous works (Han et al., 2022a; Ruff et al., 2020) to
set one of the classes as normal and downsample the rest classes to 5% of the
total instances as anomalies. We report the average performance of airplane
and automobile for CIFAR-10; and the average performance of t-shirt/top and
trouser for FashionMNIST.

Horizontal Baselines: We conduct a horizontal comparison between the proposed
methods and the following representative data augmentation methods on 3
different classifiers (i.e., KNN, XGBoost, MLP): Random is a basic baseline
which generates synthetic anomalies by randomly averaging two existing
anomalies. SMOTE (Chawla et al., 2002) linearly combines existing anomalies
with their K-nearest anomalies through randomly sampled combination ratios.
BorderlineSMOTE (Han et al., 2005) identifies a borderline between anomalies
and normal samples with the K-nearest neighborhood of each anomaly. Then,
SMOTE is performed on the anomalies in the borderline area. SVMSMOTE (Nguyen
et al., 2011) introduces a support vector classifier to identify a borderline
between anomalies and normal samples and perform SMOTE on anomalies near to
the borderline. ADASYN (He et al., 2008) calculates the number of synthetic
samples generated from individual anomalies using the estimated local
distribution and then performs SMOTE on them. AutoSMOTE (Zha et al., 2022)
performs SMOTE by exploiting deep reinforcement learning to automatically
select the anomalies and the corresponding oversampling strategies. MixBoost
(Kabra et al., 2020) uses a randomly sampled combination ratio to mix
anomalies and normals. The number of oversamples for individual anomalies is
weightily sampled based on the entropy of the underlying classifier.

Vertical Baselines: We perform a vertical comparison between the proposed
method and label-informed detection algorithms to study the effectiveness of
synthetic oversampling. XGBOD (Zhao and Hryniewicki, 2018) exploits the
anomalous scores of individual data points generated from multiple
unsupervised algorithms as the input feature for the underlying XGBoost
classifier. DeepSAD (Ruff et al., 2020) develops a semi-supervised loss
function that classifies the known normal samples and unlabeled samples into a
unified cluster and deviates the known anomalies from the cluster. DevNet
(Pang et al., 2019a) uses label information with a prior probability to
enforce significant deviations in anomaly scores from the majority of normal
samples.

Evaluation Protocol: We adopt macro-averaged precision, recall, and F1-score,
which compute the scores separately for each class and average them. The
intuition behind this is to equalize the importance of anomaly detection and
normal sample classification since the minimum false alarms are also a
critical evaluation criterion. We use 80% data for training and 20% for
testing, where 20% of the training data is further split into a validation set
for our framework to generate reward signals or for baseline methods to
perform model tuning. We run the experiments with 5 random seeds and report
the average performance on the testing set.

Implementation Details: For the horizontal analysis, we use a KNN classifier
with $k=10$, an XGBoost classifier with the linear kernel, and the Adam
optimizer with ReLU activation function for a 128-64 multi-layer perceptron
classifier. For the vertical analysis, we adopt XGBOD
111https://github.com/yzhao062/pyod and public available implementations of
DevNet 222https://github.com/GuansongPang/deviation-network and DeepSAD
333https://github.com/lukasruff/Deep-SAD-PyTorch. Since the output of Dev and
DeepSAD are anomaly scores, we search for the thresholds for the two methods
from $\\{0.5x,1.0x,1.5x,2.0x\\}$ of the anomaly ratio to perform
classification and report the best result. For our own method, we select the
maximum neighborhood size $K$ from $\\{5,10,15,20,25\\}$, and set $\eta=0.3$,
the reward coefficient $\lambda=10.0$, the window size for baseline $m=25$ and
adopt the macro-averaged F1-score for the reward signal $\Delta\mathcal{M}$.

###  4.2. Horizontal Analysis

To answer RQ1, we compare the MixAnN to cutting-edge synthetic oversampling
methods for anomaly detection. Table 2 tabulates the macro-averaged precision,
recall, and F1-score of each data augmentation method across 3 different
classifiers. We also report the performances without data augmentation on the
original classifiers to show insights into how different classifiers impact
the performances. In general, the proposed MixAnN is able to outperform all of
the baseline oversampling methods and achieve at least $7.6\%$ and at most
$33.3\%$ improvements on the F1-score. Based on Table 2, we make the following
observations.

First, by comparing the baseline augmentation methods with the classifiers
without data augmentation, we observe that the performance of the baseline
augmentation methods is generally inferior to the classifier trained without
data augmentation. Specifically, the average F1-scores of the baseline
augmentation methods are consistently lower than the vanilla classifiers on
the five datasets. The only exception is the Mixboost with KNN classifier,
which is due to its decent performance on the SMTP dataset. Our further
investigation into this phenomenon suggests that randomly mixing up normal
samples with anomalies can create beneficial synthetic normal samples that
concrete the decision boundary when anomalies are extremely sparse. This
supports our claim that the existing data augmentation methods are not capable
of handling the diverse behavior of anomalies and may lead to noisy synthetic
samples. But generalizing label anomaly information by mixing normal samples
with anomalies could alleviate the problem.

Second, by comparing the MixAnN with the vanilla classifier without data
augmentation, we observe that the MixAnN consistently outperforms all vanilla
classifiers. On the five datasets, the MixAnN improves the F1-score of KNN,
XGBoost, and MLP classifiers by $9.6\%$, $14.2\%$, and $11.5\%$, respectively.
This phenomenon suggests that the MixAnN is able to adaptively create
synthetic samples for different classifiers toward performance improvements.
In addition, we also observe that the KNN classifier has the maximum
improvement, which suggests that the nearest-neighbor exploration of our
transition function favors the classifier with similar attributes. Another
interesting observation is that the more complex the models, the fewer the
improvements. A possible explanation is that complex models tend to be
overconfident in the prediction (Nguyen et al., 2015), which may lead to noisy
prediction uncertainty reward and mislead the learning procedure of the
augmentation strategy. We will study this issue in the future.

Third, by comparing the MixAnN with all other data augmentation methods, we
observe that the MixAnN outperforms all baselines with the three classifiers
on Macro-F1 scores. Specifically, the MixAnN averagely outperforms the
F1-score of the second-best augmentation method with KNN, XGBoost, and MLP
classifiers on the seven datasets by $4.9\%$, $10.1\%$ and $7.4\%$,
respectively. Because Mixboost is similar to MixAnN with a random mixture
policy, it implies that the proposed framework can learn tailored mix-up
policies for different classifiers and data samples. We can also observe that,
although the MixAnN is not always superior to all other baselines on precision
and recall, the F1-scores are always the best. This phenomenon suggests that
MixAnN is able to balance the trade-off between precision and recall, and
therefore leads to superior F1-scores in all settings. The main reason behind
this is that we adopt the Macro-F1 to form a reward signal. One may also
tailor their own metrics (e.g., precision, recall, or tailored metrics) to
obtain an anomaly detector that meets their requirements.

Fourth, by taking a detailed comparison between the MixAnN with SVMSMOTE and
BorderlineSMOTE, we observe that the F1-score of the MixAnN is superior to the
two baselines by at least $14.9\%$, $14.2\%$ and $14.5\%$ with the three
classifiers respectively. This phenomenon suggests that the instance-wise
prediction uncertainty in the reward function is a better approach to
generating tailored beneficial synthetic samples for different classifiers.
The rationale behind this is that the two baselines identify the class
boundary in the label space and the hyperspace of the SVM, where the MixAnN
identifies the boundary that is directly defined by the underlying classifier.
By encouraging the policy to generate samples on the boundary defined by the
classifier, it is more likely to create beneficial information that cannot be
observed from the original feature space or the hyperspace of another
classifier.

Classifier | Methods | Dataset | Avg. F1 $\uparrow$
---|---|---|---
(Prec./Rec./F1) | Vowels | Annthyroid | Mammography | Satellite | SMTP | CIFAR-10 | FashionMNIST
KNN | w/o Augmentation | 0.98/0.72/0.82 | 0.93/0.61/0.67 | 0.86/0.74/0.79 | 0.90/0.88/0.89 | 1.00/0.75/0.83 | 0.92/0.54/0.56 | 0.96/0.82/0.87 | +9.6%
Random | 0.87/0.82/0.84 | 0.67/0.63/0.65 | 0.64/0.77/0.68 | 0.90/0.88/0.89 | 0.51/0.99/0.51 | 0.55/0.55/0.55 | 0.85/0.85/0.85 | +19.7%
SMOTE | 0.82/0.99/0.88 | 0.61/0.70/0.61 | 0.65/0.91/0.71 | 0.89/0.88/0.89 | 0.52/1.00/0.53 | 0.56/0.74/0.56 | 0.81/0.91/0.85 | +18.1%
BorderlineSMOTE | 0.85/0.99/0.91 | 0.61/0.72/0.63 | 0.67/0.92/0.73 | 0.90/0.88/0.89 | 0.52/1.00/0.53 | 0.58/0.75/0.60 | 0.81/0.91/0.86 | +14.9%
SVMSMOTE | 0.85/0.99/0.91 | 0.64/0.72/0.66 | 0.70/0.92/0.76 | 0.90/0.88/0.89 | 0.53/1.00/0.56 | 0.59/0.72/0.62 | 0.84/0.91/0.87 | +13.3%
ADASYN | 0.82/0.99/0.88 | 0.59/0.71/0.60 | 0.63/0.92/0.69 | 0.90/0.88/0.89 | 0.55/1.00/0.60 | 0.56/0.73/0.55 | 0.79/0.92/0.83 | +19.7%
AutoSMOTE | 0.94/0.94/0.94 | 0.67/0.75/0.70 | 0.78/0.85/0.81 | 0.86/0.90/0.89 | 0.86/0.90/0.87 | 0.62/0.62/0.62 | 0.89/0.87/0.88 | +4.9%
Mixboost | 0.99/0.75/0.83 | 0.94/0.65/0.72 | 0.86/0.77/0.81 | 0.90/0.88/0.89 | 0.77/1.00/0.85 | 0.56/0.56/0.56 | 0.94/0.81/0.86 | +7.6%
MixAnN |  0.99/0.93/0.96 |  0.96/0.72/0.75 |  0.92/0.80/0.84 |  0.92/0.90/0.91 |  0.93/0.99/0.96 |  0.84/0.58/0.62 |  0.97/0.83/0.88 | -
XGBoost | w/o Augmentation | 0.99/0.86/0.91 | 0.94/0.67/0.73 | 0.86/0.76/0.80 | 0.90/0.76/0.79 | 1.00/0.58/0.64 | 0.69/0.66/0.67 | 0.87/0.87/0.87 | +14.2%
Random | 0.92/0.89/0.90 | 0.91/0.70/0.76 | 0.66/0.75/0.70 | 0.91/0.80/0.83 | 0.50/0.49/0.49 | 0.62/0.65/0.63 | 0.64/0.72/0.68 | +23.9%
SMOTE | 0.78/0.98/0.85 | 0.86/0.90/0.88 | 0.59/0.90/0.62 | 0.91/0.80/0.84 | 0.55/1.00/0.59 | 0.62/0.67/0.64 | 0.84/0.90/0.87 | +15.8%
BorderlineSMOTE | 0.79/0.98/0.86 | 0.87/0.92/0.89 | 0.58/0.90/0.60 | 0.86/0.81/0.83 | 0.50/0.94/0.47 | 0.63/0.66/0.64 | 0.87/0.88/0.87 | +18.9%
SVMSMOTE | 0.82/0.99/0.88 | 0.86/0.87/0.86 | 0.63/0.92/0.68 | 0.84/0.81/0.82 | 0.56/1.00/0.60 | 0.64/0.66/0.65 | 0.86/0.89/0.87 | +14.2%
ADASYN | 0.75/0.97/0.82 | 0.84/0.96/0.89 | 0.56/0.87/0.55 | 0.85/0.81/0.82 | 0.50/0.77/0.35 | 0.63/0.67/0.65 | 0.83/0.89/0.86 | +23.9%
AutoSMOTE | 0.90/0.96/0.93 | 0.88/0.92/0.90 | 0.80/0.82/0.81 | 0.68/0.70/0.62 | 1.00/0.75/0.83 | 0.64/0.67/0.66 | 0.87/0.90/0.88 | +10.1%
Mixboost | 0.99/0.79/0.86 | 0.96/0.55/0.57 | 0.90/0.62/0.68 | 0.89/0.70/0.73 | 0.50/0.50/0.50 | 0.67/0.61/0.63 | 0.66/0.70/0.68 | +33.3%
MixAnN |  1.00/0.96/0.98 |  0.87 /0.97/0.92 |  0.81/0.85/0.83 |  0.91/0.82/0.84 |  1.00/1.00/1.00 |  0.71/0.68/0.70 |  0.89/0.90/0.90 | -
MLP | w/o Augmentation | 0.95/0.89/0.92 | 0.94/0.53/0.53 | 0.88/0.76/0.81 | 0.84/0.78/0.80 | 1.00/0.75/0.83 | 0.86/0.65/0.71 | 0.97/0.85/0.89 | +11.5%
Random | 0.78/0.88/0.82 | 0.91/0.58/0.61 | 0.62/0.77/0.66 | 0.90/0.79/0.82 | 0.50/0.98/0.50 | 0.70/0.68/0.69 | 0.83/0.87/0.85 | +22.5%
SMOTE | 0.85/0.99/0.91 | 0.59/0.73/0.59 | 0.62/0.92/0.67 | 0.88/0.81/0.83 | 0.51/0.99/0.51 | 0.73/0.69/0.71 | 0.88/0.92/0.90 | +19.2%
BorderlineSMOTE | 0.82/0.99/0.88 | 0.59/0.73/0.59 | 0.64/0.92/0.70 | 0.85/0.80/0.82 | 0.50/0.97/0.49 | 0.73/0.71/0.72 | 0.89/0.92/0.90 | +19.2%
SVMSMOTE | 0.89/0.99/0.93 | 0.62/0.74/0.65 | 0.71/0.93/0.78 | 0.88/0.81/0.83 | 0.51/0.99/0.53 | 0.75/0.70/0.72 | 0.96/0.87/0.90 | +14.5%
ADASYN | 0.89/0.99/0.93 | 0.59/0.74/0.59 | 0.61/0.92/0.65 | 0.88/0.81/0.84 | 0.50/0.98/0.50 | 0.71/0.69/0.70 | 0.87/0.92/0.90 | +19.2%
AutoSMOTE | 0.96/0.93/0.94 | 0.90/0.64/0.70 | 0.78/0.86/0.82 | 0.70/0.73/0.66 | 1.00/0.92/0.95 | 0.74/0.79/0.73 | 0.94/0.90/0.91 | +7.4%
Mixboost | 0.65/0.60/0.62 | 0.96/0.52/0.52 | 0.90/0.72/0.78 | 0.89/0.73/0.76 | 0.90/0.83/0.86 | 0.67/0.58/0.61 | 0.93/0.95/0.88 | +20.8%
MixAnN |  0.99/0.95/0.96 |  0.93/0.74/0.79 |  0.87/0.84/0.85 |  0.91/0.83/0.85 |  1.00/0.92/0.95 |  0.84/0.71/0.76 |  0.97/0.91/0.93 | -
Table 2. Horizontal comparison with data-augmentation methods Category | Methods | Dataset | Avg. F1 $\uparrow$
---|---|---|---
(Prec./Rec./F1) | Vowels | Annthyroid | Mammography | Satellite | SMTP | CIFAR-10 | FashionMNIST
Label-informed | XGBOD | 0.86/0.71/0.76 | 0.88/0.52/0.52 | 0.95/0.59/0.65 | 0.91/0.80/0.84 | 1.00/0.58/0.64 | 0.87/0.59/0.63 | 0.96/0.83/0.88 | +28.2%
DevNet | 0.89/0.99/0.93 | 0.63/0.61/0.62 | 0.72/0.92/0.79 | 0.71/0.71/0.71 | 1.00/1.00/1.00 | 0.72/0.74/0.73 | 0.91/0.91/0.91 | +12.3%
DeepSAD | 0.99/0.71/0.80 | 0.90/0.56/0.59 | 0.66/0.65/0.65 | 0.81/0.73/0.75 | 0.99/0.71/0.80 | 0.71/0.73/0.72 | 0.92/0.93/0.93 | +21.3%
Data Augmentation | Best Baseline | 0.96/0.93/0.94 | 0.88/0.92/0.90 | 0.78/0.86/0.82 | 0.90/0.88/0.89 | 0.86/0.90/0.87 | 0.74/0.79/0.73 | 0.94/0.90/0.91 | +5.8%
MixAnN |  1.00/0.96/0.98 |  0.87/0.97/0.92 |  0.87/0.84/0.85 |  0.92/0.90/0.91 |  1.00/1.00/1.00 |  0.84/0.71/0.76 |  0.97/0.91/0.93 | -
Table 3. Vertical comparison with label-informed methods. The best baseline in
the data augmentation category refers to the horizontal baseline with the best
F1-score across the three classifiers.

###  4.3. Vertical Analysis

In order to answer RQ2, we compare the MixAnN to the most advanced label-
informed anomaly detection algorithms and baseline augmentation methods. Table
3 presents the macro-averaged precision, recall, and F1-score of each method
on the 5 datasets. ”Best baseline” refers to the horizontal baseline with the
best F1-score on individual datasets. We make the following two observations.

First, data augmentation methods generally outperform label-informed anomaly
detectors. Comparing the best data augmentation baseline to the three label-
informed approaches, the best data augmentation baseline outperforms the best
label-informed algorithm by $6.2\%$. This suggests that data augmentation may
be more effective in generalizing label information when incorporated with
proper strategy. Additionally, the proposed MixAnN with a properly learned
strategy achieves superior performance, which further validates the suggestion
above.

Second, label-informed methods achieve better precision and lower recall. By
comparing Table 3 with Table 2, we can observe that the average precision of
label-informed approaches is generally superior to data augmentation methods
on all three classifiers, whereas the average recall is generally inferior to
those baselines. One possible explanation is that label-informed approaches
can only exploit the labels themselves, while data augmentation methods are
capable of exploring potentially beneficial information from the limited label
information. This suggests that label-informed approaches tend to over-fit
existing labels, which may be sub-optimal.

Ablations | Precision | Recall | F1-score
---|---|---|---
Random reward | 0.85 | 0.60 | 0.66
w/o Improvement Stimulation (Eq. 5) | 0.98 | 0.64 | 0.71
w/o Model Exploration (Eq. 6) | 0.98 | 0.71 | 0.79
Full MixAnN | 0.99 | 0.93 | 0.96
Table 4. Macro-averaged scores of MixAnN with the KNN classifier and the
ablations on Japanese Vowels.

###  4.4. Ablation Study

To answer RQ3, we conducted an ablation study on the Japanese Vowels dataset
with the KNN classifier. As the reward signal is the most critical guidance
toward optimal augmentation strategies, we ablate the reward function to study
the contribution of individual components. Note that a random reward generates
a reward signal with a random floating number from 0 to 1, which potentially
leads to a random augmentation strategy. From Table 4, we can make the
following observations.

First, the proposed MDP is solvable, and the tailored RL agent is capable of
learning an optimal strategy. By comparing the random reward baseline with the
MixAnN, we observe that there are significant improvements in all three
metrics. Both the two ablations on Eq. 5 and 6 are significantly better than
the random reward baseline, which suggests that the tailored RL agent is
capable of addressing the MDP toward an optimal augmentation strategy.

Second, the two components in the proposed reward signal play significant
roles in an optimal augmentation strategy. Specifically, both components are
capable of increasing the exploitation of the label information and therefore
lead to significant improvements in precision. On one hand, as the classifier
may suffer from under-fitting during the training procedure, learning the
augmentation strategy without Eq. 5 may lead to a significant performance
drop. On the other hand, without considering the model impact via Eq. 6, it is
less possible to identify potentially beneficial information for the
underlying classifier and therefore leads to a lower recall.

Figure 4. Hyperparameter study on Vowels.

###  4.5. Hyper-parameter Study

To answer the RQ4, we study the two key hyperparameters (i.e., maximum
neighborhood size $K$ and the threshold $\eta$) on the vowels dataset with the
three classifiers. Based on the results shown in Figure 4, we make the
following observations.

For the size of the neighborhood $K$, simpler classifiers may require
collecting a larger amount of neighborhood information to extend the
supervisory information of the source samples. For a complex model such as a
multi-layer perceptron, we may need to carefully select the size to prevent
over-smoothing when collecting too much neighborhood information (Chen et al.,
2020) for the classifier. As for the threshold $\eta$, we observe that a lower
threshold for creating synthetic anomalies generally leads to better
performance. One possible explanation is that a lower $\eta$ prompts the data
mixer to create more synthetic anomalies, which is therefore beneficial to the
extreme imbalance setting of anomaly detection.

Figure 5. Visualization of the generated synthetic samples and the decision
boundary of DecisionTree on a toy data using AutoSMOTE and other over-sampling
techniques.

###  4.6. Case Study

To answer the RQ5, we apply MixAnN and the baselines to a publicly available
2-dimensional toy dataset
444https://github.com/shubhomoydas/ad_examples/tree/master/ad_examples/datasets/anomaly/toy2/fullsamples.
We split 60% of the data for training, 20% of the data for validation, and 20%
for testing and use Macro-F1 as the performance metric. Figure 5 illustrates
the generated synthetic samples and the decision boundary obtained by training
a KNN classifier on the over-sampled data. We observe that classical SMOTE-
based algorithms and Mixboost tend to generate some noisy samples that
interleave with the majorities between the anomalies from two different sides,
which degrades the performance. AutoSMOTE is capable of generating synthetic
samples that are less noisy. However, it has a limitation in cases where there
is a true anomaly located at the center. This can lead to the generation of
multiple synthetic anomalies that extend the left upper decision boundary,
causing misclassification of genuine normal instances. On the other hand,
MixAnN is a method that excels in generating synthetic samples of both normal
and anomalous instances without introducing any noise. As a result, MixAnN
achieves a higher Macro-F1 score compared to AutoSMOTE. This improvement in
performance clearly demonstrates the effectiveness of mixing majority and
minority instances guided by a learning-based agent.

##  5\. Conclusion and Future Work

We propose the MixAnN, a strategic data augmentation framework for imbalanced
classification with diverse minorities. We propose an iterative mix-up
approach to generalize label information and formulate the iterative mix-up
into a Markov decision process. In addition, we further tailor a deep actor-
critic framework to tackle the Markov decision process toward an optimal data
augmentation policy. We evaluate MixAnN in comparison to state-of-the-art
imbalanced classification algorithms and data augmentation approaches and
demonstrate the superiority of MixAnN. In the future, we will explore
potential approaches to overcome the overconfidence problem of complex
classifiers and further improve the scalability of MixAnN with under-sampling
methods.

## References

  * (1)
  * Al-Hashedi and Magalingam (2021) Khaled Gubran Al-Hashedi and Pritheega Magalingam. 2021.  Financial fraud detection applying data mining techniques: a comprehensive review from 2009 to 2019.  _Computer Science Review_ 40 (2021), 100402.
  * Bunkhumpornpat et al. (2012) Chumphol Bunkhumpornpat, Krung Sinapiromsaran, and Chidchanok Lursinsap. 2012.  DBSMOTE: density-based synthetic minority over-sampling technique.  _Applied Intelligence_ 36, 3 (2012), 664–684.
  * Cao et al. (2019) Zhangjie Cao, Kaichao You, Mingsheng Long, Jianmin Wang, and Qiang Yang. 2019.  Learning to transfer examples for partial domain adaptation. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_.
  * Chapelle et al. (2009) Olivier Chapelle, Bernhard Scholkopf, and Alexander Zien. 2009\.  Semi-supervised learning (chapelle, o. et al., eds.; 2006)[book reviews].  _IEEE Transactions on Neural Networks_ 20, 3 (2009), 542–542.
  * Chawla et al. (2002) Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. 2002.  SMOTE: synthetic minority over-sampling technique.  _Journal of artificial intelligence research_ 16 (2002), 321–357.
  * Chen et al. (2020) Deli Chen, Yankai Lin, Wei Li, Peng Li, Jie Zhou, and Xu Sun. 2020.  Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. In _Proceedings of the AAAI conference on artificial intelligence_.
  * Chen et al. (2022a) Huiyuan Chen, Chin-Chia Michael Yeh, Fei Wang, and Hao Yang. 2022a.  Graph neural transport networks with non-local attentions for recommender systems. In _Proceedings of the ACM Web Conference 2022_. 1955–1964.
  * Chen et al. (2022b) Huiyuan Chen, Kaixiong Zhou, Kwei-Herng Lai, Xia Hu, Fei Wang, and Hao Yang. 2022b.  Adversarial graph perturbations for recommendations at scale. In _Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval_. 1854–1858.
  * Dabouei et al. (2021) Ali Dabouei, Sobhan Soleymani, Fariborz Taherkhani, and Nasser M Nasrabadi. 2021.  Supermix: Supervising the mixing data augmentation. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 13794–13803.
  * Haarnoja et al. (2018) Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. 2018.  Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In _International conference on machine learning_. 1861–1870.
  * Han et al. (2005) Hui Han, Wen-Yuan Wang, and Bing-Huan Mao. 2005.  Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning. In _International conference on intelligent computing_. 878–887.
  * Han et al. (2022a) Songqiao Han, Xiyang Hu, Hailiang Huang, Mingqi Jiang, and Yue Zhao. 2022a.  ADBench: Anomaly Detection Benchmark. In _Neural Information Processing Systems_.
  * Han et al. (2022b) Xiaotian Han, Zhimeng Jiang, Ninghao Liu, and Xia Hu. 2022b.  G-Mixup: Graph Data Augmentation for Graph Classification. In _International Conference on Machine Learning_.
  * He et al. (2008) Haibo He, Yang Bai, Edwardo A Garcia, and Shutao Li. 2008\.  ADASYN: Adaptive synthetic sampling approach for imbalanced learning. In _2008 IEEE international joint conference on neural networks_. 1322–1328.
  * Hsu and Liu (2021) Chia-Yu Hsu and Wei-Chen Liu. 2021.  Multiple time-series convolutional neural network for fault detection and diagnosis and empirical study in semiconductor manufacturing.  _Journal of Intelligent Manufacturing_ 32 (2021), 823–836.
  * Ienco et al. (2016) Dino Ienco, Ruggero G Pensa, and Rosa Meo. 2016.  A semisupervised approach to the detection and characterization of outliers in categorical data.  _IEEE transactions on neural networks and learning systems_ 28, 5 (2016), 1017–1029.
  * Kabra et al. (2020) Anubha Kabra, Ayush Chopra, Nikaash Puri, Pinkesh Badjatiya, Sukriti Verma, Piyush Gupta, et al. 2020\.  MixBoost: Synthetic Oversampling with Boosted Mixup for Handling Extreme Imbalance.  _arXiv preprint arXiv:2009.01571_ (2020).
  * Kim et al. (2020) Aechan Kim, Mohyun Park, and Dong Hoon Lee. 2020.  AI-IDS: Application of deep learning to real-time Web intrusion detection.  _IEEE Access_ (2020).
  * Kovács (2019) György Kovács. 2019\.  Smote-variants: A python implementation of 85 minority oversampling techniques.  _Neurocomputing_ 366 (2019), 352–354.
  * Lai et al. (2023) Kwei-Herng Lai, Lan Wang, Huiyuan Chen, Kaixiong Zhou, Fei Wang, Hao Yang, and Xia Hu. 2023.  Context-aware Domain Adaptation for Time Series Anomaly Detection. In _Proceedings of the 2023 SIAM International Conference on Data Mining_. 676–684.
  * Li et al. (2021) Mingchen Li, Xuechen Zhang, Christos Thrampoulidis, Jiasi Chen, and Samet Oymak. 2021.  AutoBalance: Optimized Loss Functions for Imbalanced Data. In _Advances in Neural Information Processing Systems_.
  * Lillicrap et al. (2015) Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2015.  Continuous control with deep reinforcement learning.  _arXiv preprint arXiv:1509.02971_ (2015).
  * Ling et al. (2023a) Hongyi Ling, Zhimeng Jiang, Meng Liu, Shuiwang Ji, and Na Zou. 2023a.  Graph Mixup with Soft Alignments. In _International Conference on Machine Learning_.
  * Ling et al. (2023b) Hongyi Ling, Zhimeng Jiang, Youzhi Luo, Shuiwang Ji, and Na Zou. 2023b.  Learning Fair Graph Representations via Automated Data Augmentations. In _International Conference on Learning Representations_.
  * Liu and Zhou (2006) Xu-Ying Liu and Zhi-Hua Zhou. 2006.  The influence of class imbalance on cost-sensitive learning: An empirical study. In _International Conference on Data Mining_.
  * Liu et al. (2020) Zhining Liu, Pengfei Wei, Jing Jiang, Wei Cao, Jiang Bian, and Yi Chang. 2020\.  MESA: Boost Ensemble Imbalanced Learning with MEta-SAmpler. In _Conference on Neural Information Processing Systems_.
  * Nguyen et al. (2015) Anh Nguyen, Jason Yosinski, and Jeff Clune. 2015\.  Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 427–436.
  * Nguyen et al. (2011) Hien M Nguyen, Eric W Cooper, and Katsuari Kamei. 2011\.  Borderline over-sampling for imbalanced data classification.  _International Journal of Knowledge Engineering and Soft Data Paradigms_ 3, 1 (2011), 4–21.
  * Pang et al. (2019b) Guansong Pang, Chunhua Shen, Huidong Jin, and Anton van den Hengel. 2019b.  Deep weakly-supervised anomaly detection.  _arXiv preprint arXiv:1910.13601_ (2019).
  * Pang et al. (2019a) Guansong Pang, Chunhua Shen, and Anton van den Hengel. 2019a.  Deep anomaly detection with deviation networks. In _Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining_. 353–362.
  * Pang et al. (2021) Guansong Pang, Anton van den Hengel, Chunhua Shen, and Longbing Cao. 2021.  Toward deep supervised anomaly detection: Reinforcement learning from partially labeled anomaly data. In _Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining_. 1298–1308.
  * Perini et al. (2020) Lorenzo Perini, Vincent Vercruyssen, and Jesse Davis. 2020\.  Quantifying the confidence of anomaly detectors in their example-wise predictions. In _Joint European Conference on Machine Learning and Knowledge Discovery in Databases_. Springer, 227–243.
  * Ruff et al. (2020) Lukas Ruff, Robert A. Vandermeulen, Nico Görnitz, Alexander Binder, Emmanuel Müller, Klaus-Robert Müller, and Marius Kloft. 2020\.  Deep semi-supervised anomaly detection. In _International Conference on Learning Representations_.
  * Shorten and Khoshgoftaar (2019) Connor Shorten and Taghi M Khoshgoftaar. 2019.  A survey on image data augmentation for deep learning.  _Journal of big data_ 6, 1 (2019), 1–48.
  * Siriseriwan and Sinapiromsaran (2017) Wacharasak Siriseriwan and Krung Sinapiromsaran. 2017.  Adaptive neighbor synthetic minority oversampling technique under 1NN outcast handling.  _Songklanakarin J. Sci. Technol_ 39, 5 (2017), 565–576.
  * Sutton and Barto (2018) Richard S Sutton and Andrew G Barto. 2018.  _Reinforcement learning: An introduction_.  MIT press.
  * Wang et al. (2023) Song Wang, Xingbo Fu, Kaize Ding, Chen Chen, Huiyuan Chen, and Jundong Li. 2023\.  Federated Few-Shot Learning. In _Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining_.
  * Wen et al. (2020) Qingsong Wen, Liang Sun, Fan Yang, Xiaomin Song, Jingkun Gao, Xue Wang, and Huan Xu. 2020.  Time series data augmentation for deep learning: A survey.  _arXiv preprint arXiv:2002.12478_ (2020).
  * Wu and Ortiz (2021) Tong Wu and Jorge Ortiz. 2021.  Rlad: Time series anomaly detection through reinforcement learning and active learning.  _arXiv preprint arXiv:2104.00543_ (2021).
  * Yen and Lee (2006) Show-Jane Yen and Yue-Shi Lee. 2006.  Under-sampling approaches for improving prediction of the minority class in an imbalanced dataset.  In _Intelligent Control and Automation_. Springer, 731–740.
  * Zha et al. (2023) Daochen Zha, Zaid Pervaiz Bhat, Kwei-Herng Lai, Fan Yang, Zhimeng Jiang, Shaochen Zhong, and Xia Hu. 2023.  Data-centric artificial intelligence: A survey.  _arXiv preprint arXiv:2303.10158_ (2023).
  * Zha et al. (2022) Daochen Zha, Kwei-Herng Lai, Qiaoyu Tan, Sirui Ding, Na Zou, and Xia Ben Hu. 2022\.  Towards automated imbalanced learning with deep hierarchical reinforcement learning. In _Proceedings of the 31st ACM International Conference on Information & Knowledge Management_.
  * Zha et al. (2020) Daochen Zha, Kwei-Herng Lai, Mingyang Wan, and Xia Hu. 2020\.  Meta-AAD: Active anomaly detection with deep reinforcement learning. In _2020 IEEE International Conference on Data Mining_.
  * Zhang et al. (2018) Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. 2018.  mixup: Beyond Empirical Risk Minimization. In _International Conference on Learning Representations_.
  * Zhao and Hryniewicki (2018) Yue Zhao and Maciej K Hryniewicki. 2018.  XGBOD: improving supervised outlier detection with unsupervised representation learning. In _2018 International Joint Conference on Neural Networks_.