# From Seedling to Harvest: The GrowingSoy Dataset
for Weed Detection in Soy Crops
via Instance Segmentation

1st Raul Steinmetz  Universidade Federal de Santa Maria
Santa Maria, Brazil
<EMAIL_ADDRESS>2nd Victor Augusto Kich  University of Tsukuba
Tsukuba, Japan
<EMAIL_ADDRESS>3rd Henrique Krever  Universidade Federal de Santa
Maria
Santa Maria, Brazil
<EMAIL_ADDRESS>4rd João Davi Rigo Mazzarolo  Universidade Federal de
Santa Maria
Santa Maria, Brazil
<EMAIL_ADDRESS>5th Ricardo Bedin Grando  Universidad Tecnológica del
Uruguay
Rivera, Uruguay
<EMAIL_ADDRESS>6th Vinicius Marini  Universidade Federal de Santa
Maria
Santa Maria, Brazil
<EMAIL_ADDRESS>7th Celio Trois  Universidade Federal de Santa Maria
Santa Maria, Brazil
<EMAIL_ADDRESS>8th Ard Nieuwenhuizen  Wageningen University & Research
Wageningen, Netherlands
<EMAIL_ADDRESS>
###### Abstract

Deep learning, particularly Convolutional Neural Networks (CNNs), has gained
significant attention for its effectiveness in computer vision, especially in
agricultural tasks. Recent advancements in instance segmentation have improved
image classification accuracy. In this work, we introduce a comprehensive
dataset for training neural networks to detect weeds and soy plants through
instance segmentation. Our dataset covers various stages of soy growth,
offering a chronological perspective on weed invasion’s impact, with 1,000
annotated images. To validate our data, we also provide 6 state of the art
models, trained in this dataset, that can understand and detect soy and weed
in every stage of the plantation process, the best results achieved were a
segmentation average precision of 79.1% and an average recall of 73.3% across
all plant classes. Moreover, the YOLOv8M model attained 78.7% mean average
precision (mAp-50) in caruru weed segmentation, 69.6% in grassy weed
segmentation, and 90.1% in soy plant segmentation.111Video of the experiments
is available at: https://youtu.be/Ir0vwtAk8Tc222GitHub Repository is available
at: https://github.com/raulsteinmetz/soy-segmentation333SoyGrowing dataset is
available at: https://github.com/raulsteinmetz/soy-segmentation-ds

###### Index Terms:

Weed Detection, Instance Segmentation, Temporal Perspective Dataset, Soy

##  I Introduction

Figure 1: Some examples of instance segmentation on the GrowingSoy dataset.
Arrows highlight the transitions between temporal states, encompassing all
growth stages of the soy plant.

Effective crop management is essential for maximizing yields, particularly in
environments susceptible to issues like weed infestations, such as soybean
crops. Despite efforts to control weeds through methods such as chemical
applications, the persistent threat of unexpected weed invasions remains a
significant risk to crop quality and yield. Therefore, vigilant and responsive
agricultural practices are crucial.

Soybeans occupy a pivotal role globally, being the second most produced
vegetable oil and serving as a major source of protein for both animal feed
and human consumption [1]. Effective management techniques that enable early
identification and removal of weeds are essential for increasing crop
production.

In recent years, advancements in hardware and software technologies have
propelled neural networks into the spotlight as powerful tools capable of
automating crucial agricultural processes [2, 3]. These innovations include
autonomous robot control [4, 5] and computer vision algorithms, such as
Convolutional Neural Networks (CNNs) for image classification [6]. These
technologies facilitate automated disaster detection while simultaneously
conducting a comprehensive analysis of plantation status and key metrics [7].

A recent advancement in computer vision is instance semantic segmentation [8,
9], a method that divides images into distinct segments and entities with
pixel-level accuracy, enhancing object recognition and delineation. This
technique has significant potential in agricultural computer vision, where
accurate identification and analysis of individual components within a crop
field are essential for efficient crop management. However, the scarcity of
properly labeled and processed data for segmentation remains a challenge for
the development of advanced applied solutions.

The prevailing issue in crop datasets lies in the lack of segmentation labels,
as the majority remain unlabeled for this purpose. Notably, datasets such as
DeepWeeds [10], Weed25 [11], and the Soybean images dataset [12] are
exclusively annotated for image classification, overlooking the segmentation
aspect. Furthermore, a significant portion of these datasets consists of
images with low resolution and inadequate illumination.

In contrast, we introduce a dataset that addresses these limitations by
providing a 1000 high-quality images depicting soy and weeds. What
distinguishes our dataset is the inclusion of segmentation labels for all
images. This enhancement contributes to a more comprehensive dataset and fills
a crucial gap in the availability of well-annotated crop datasets for
segmentation tasks. A set of segmented images using a model trained on our
dataset is depicted in Fig. 1.

Hence, the contributions of this work are:

  * •

We present a novel dataset with 1,000 annotated and high-resolution images for
training instance segmentation neural networks, focusing on weed-infested soy
plantations. The dataset spans the entire soy growth process, from initial
stages to maturity with pervasive weed presence.

  * •

Additionally, to validate the dataset, we provide six state-of-the-art neural
network models and perform an in-depth comparison among them, leveraging this
temporal dataset to achieve impressive mean average precision in weed and soy
segmentation across all soy plantation stages.

The structure of this paper is as follows: Section II offers a literature
review, delving into previous research in computer vision for crop monitoring
and instance segmentation. In Section III, we outline our methodology for
creating the dataset. In Section IV, we depict the process of selecting and
training the neural networks involved in the study. Section V presents the
results obtained using the neural networks on our dataset. Section VI presents
an analysis of the performance in images that contain both soy and weed, in
seedling and harvest plantation stages. Section VII present some
clarifications about the results and analysis. Finally, Section VIII
summarizes our conclusions drawn from this study, highlighting the
significance of our findings for agricultural management and the potential for
further advancements in instance segmentation for crop monitoring.

##  II Related Work

Over the years, continuous advancements in artificial intelligence have
enhanced the widespread use of CNNs in precision agriculture and the use of
instance segmentation for computer vision. In this session, we will explore
some works that obtained success employing this technology.

In the field of plant disease detection, leveraging the use of a deep CNN,
Francis and Deisy [13] attained an 87% precision rate in the detection and
classification of diseases in apple and tomato leaf images. They proposed a
methodology consistent of an input image of 64x64 pixels and a smaller amount
of parameters when compared with the other models that can be used for this
problem. Similarly, Bedi and Gole [14] developed a hybrid AI model that
combines a convolutional autoencoder and a CNN to identify the presence of
Bacterial Spot disease in peach plants by analyzing images of their leaves,
the model avoids immense quantities of parameters and also increases overall
accuracy. Their main contribution was related to the small amount of
parameteres (less then 10K parameters) reducing the amount of time needed for
training and to perform the real time detection of possible plant deceases.

Weed detection through CNN’s has been widely adopted in the literature [15].
Ramirez et al. [16] employed the DeepLabv3 neural network architecture to
identify weed presence within a beet agricultural field by analyzing high-
resolution aerial imagery. For the latter, a comparison between three distinct
model was done, concluding that the DeepLabv3 presented a better performance,
besides being the one that needs the most computational resource. In a similar
vein, Hu et al. [17] utilized the YOLOv4 architecture to identify 12 distinct
types of weeds in a rice plantation. A comparative analysis with the YOLOv3
architecture was conducted, demonstrating enhancements in multiple metrics.

Several datasets containing images of both weeds and crops have been made
available in the literature like the one presented by Sudars et al. [18],
which consists of 1118 annotated images encompassing six different crop types
and eight different weed types. Another notable dataset is the TobSet,
presented by Alam et al. [19]. It is a collection of images featuring tobacco
plants and weeds captured from local fields, encompassing diverse growth
stages and varying lighting conditions. The dataset includes 7000 images of
tobacco plants and 1,000 images of weeds.

In the domain of crop monitoring, another significant application of CNNs is
in the detection of plant diseases. Medhi and Deb [20] introduced an image
dataset featuring various banana plant varieties and the diseases associated
with them, summing up to an extensive collection of over 8000 images for deep
neural network training. In a similar vein, Moupojou et al. [21] introduced
the FieldPlant dataset, consisting of 5,170 plant disease images acquired
directly from plantations. These images were annotated at the leaf level by
plant pathologists. The study demonstrates that FieldPlant surpasses current
datasets in terms of performance in classification tasks, establishing itself
as a more precise tool for disease detection in practical agricultural
scenarios.

Leveraging the power of semantic segmentation, various computer vision tasks
can have several improvements in accuracy. Yang et al. [22] conducted a
comprehensive study comparing instance segmentation methods on established
datasets, alongside an examination of practical applications of semantic
segmentation in autonomous driving. Also using instance segmentation concepts,
Zhou et al. [23] introduced an innovative approach, a combination of Swin
Transformer and CNN, for tunnel lining crack detection using instance
segmentation. This method achieved a mean intersection over union (mIoU) of
77.41% and a mean pixel accuracy of 84.42% on the custom datasets presented in
this paper. Notably, these results outperform previous CNN-based and
transformer-based semantic segmentation algorithms in terms of segmentation
accuracy.

Semantic segmentation shows notable enhancements in various fields of crop
management and monitoring. Anand et al. [24] introduced AgriSegNet, a deep
learning framework designed for the automatic identification of farmland
anomalies, including standing water and weed clusters, by employing multiscale
attention-based semantic segmentation on UAV-captured images. This framework
is tailored for IoT-assisted precision agriculture. Bosilj et al. [25]
conducted a study examining the feasibility of transfer learning across
different crop types, demonstrating the potential to reduce training time by
up to 80% in semantic segmentation-based architectures.

To the best of our knowledge, no dataset in the literature provides RGB, high-
quality images of soy plants along with human-annotated semantic segmentation
labels, covering their growth stages from seedling to harvest. As well as
encompassing the presence of weed invasions such as caruru and grassy weeds.
The temporal factor in the dataset is especially important since you can track
the plantation as well as the weeds invasion, for a better course of action
regarding ways to deal with the weeds. Furthermore, this paper represents the
pioneering effort to employ state-of-the-art YOLO architectures for accurate
segmentation of both soy and weeds throughout all growth stages of these
plants.

##  III The Dataset

Creating the soy instance segmentation dataset included recording videos,
selecting images, and manually labeling for instance segmentation. This
section illustrates these processes.

###  III-A Data Gathering

The data collection process involved recording multiple videos that captured
various distinct stages of the soy crop development, spanning from the initial
growth phases to more advanced stages. These videos also documented the
presence of caruru and grassy weeds. The selected areas of the plantation
where the videos were shot received different levels of chemical treatment,
including both medium and chemical-free approaches. This recording took place
at the Universidade Federal de Santa Maria, within a dedicated soy plantation
for research, located at approximately 29° 43’ 42” S, 53° 45’ 24” W, an aerial
image can be seen in Fig. 2.

Figure 2: Location where the videos were recorded.

The videos were captured using a GoPro Hero 12 camera with 4K resolution and a
frame rate of 120 frames per second. To ensure comprehensive coverage, the
camera was mounted on an ATV four-wheeled vehicle, which followed a consistent
14-meter path through the plantation at a steady pace of 2 kilometers per
hour, recording the crop’s growth at various intervals. The ATV and camera
setup can be seen in Fig. 3.

Figure 3: Setup used for the video recordings.

###  III-B Dataset Creation and Annotation

To curate a representative database of soy plantation stages, we selected 1000
frames from recorded videos. A sequential extraction, with a fixed step across
all videos, ensured comprehensive coverage of each stage in the soy plantation
life cycle. Through iterative experimentation, we determined the optimal
number of images for model training, ranging from 250 to 1500 images.
Interestingly, discernible model improvement plateaued after reaching 1000
images, affirming that this quantity sufficiently encapsulates the task’s
complexity.

Subsequently, the selected frames, initially with 4992 x 2496 pixels,
underwent resizing to 640x640 pixels using cubic interpolation, aligning them
with the specifications required for neural network models. To provide a
glimpse into the dataset, a subset of these images is showcased in Fig. 4.

Figure 4: Image sample from our dataset. Different stages of the plantation
can be easily visualized.

In the labeling process, instance segmentation was employed. This technique
involved delineating and fitting polygons around all the objects, which in
this case were the plants within the scenes. This detailed labeling approach
allowed the neural networks to learn how to accurately segment each individual
plant, enabling precise object recognition and classification within the
dataset. The labeling was performed manually using the Roboflow Framework
[26], which streamlines the creation of such labeled datasets for machine
learning purposes [27].

##  IV Models Trained for Data Validation

In this section, we analyze two significant frameworks in image segmentation,
used to validate the quality of our data: YOLOv5 [28] and YOLOv8 [29]. These
architectures and their various size models (medium, large, and x-large) were
chosen for our instance segmentation approach.

YOLOv5, developed by Ultralytics, is recognized for its real-time object
detection capabilities. It divides an image into a grid and predicts bounding
boxes and class probabilities for each grid cell using anchor boxes, which are
predefined shapes for predicting object shapes. YOLOv5’s success is largely
due to its flexible Pythonic structure, enabling rapid community-driven
improvements.

YOLOv8, the latest model in the YOLO series, also developed by Ultralytics,
supports object detection, image classification, and instance segmentation. It
introduces architectural enhancements over YOLOv5, including anchor-free
detection that predicts object centers directly, reducing the number of box
predictions and speeding up the Non-Maximum Suppression (NMS) process. YOLOv8
also features new convolutions and the C2f module, and it incorporates online
augmentations during training, such as mosaic augmentation.

The primary distinction between YOLOv5 and YOLOv8 lies in their approach to
object detection. YOLOv5 uses anchor boxes, while YOLOv8 employs anchor-free
detection, predicting object centers directly. This approach addresses the
limitations of anchor boxes, which may not represent the distribution of
custom datasets. YOLOv8 also introduces new convolutions and modules, such as
the C2f module, and utilizes online augmentations like mosaic augmentation,
enhancing the model’s ability to recognize objects under various conditions.
In terms of accuracy, YOLOv8 has shown improvements over YOLOv5, particularly
on benchmarks like COCO and Roboflow 100.

##  V Evaluation and Results

(a) YOLOv8 mAP increase.

(b) YOLOv8 class loss decrease.

(c) YOLOv8 segmentation loss decrease.

(d) YOLOv5 mAP increase.

(e) YOLOv5 class loss decrease.

(f) YOLOv5 segmentation loss decrease.

Figure 5: Performance metrics for the YOLOv8 and YOLOv5 models during training
and validation.

The neural networks trained on the temporal dataset demonstrated significant
precision, recall, and mean average pixel precision throughout all stages of
the plantation. This section provides an overview of the network training
process and presents the achieved metrics.

The neural networks underwent training for 200 epochs, employing early
stopping and a batch size of 8. The early stopping mechanism, with a patience
hyperparameter set to 50 epochs, allowed for the termination of training when
performance improvements became marginal. Pretraining the model on ImageNet
[30] provided a solid foundation for feature extraction. The training process
utilized a constant learning rate of 0.01, an Adam optimizer with a momentum
of 0.937, and a weight decay of 0.0005. Notably, dropout was not applied, and
the training mode operated in a deterministic manner, ensuring reproducibility
and stability throughout the training process. This comprehensive
configuration aimed to strike a balance between model complexity, convergence
speed, and generalization performance.

The learning curves can be seen in Fig. 5. In Figures 5(b) and 5(e), the class
loss curves for both the validation and training sets across all models are
displayed. In Figures 5(c) and 5(f), the loss curve for segmentation can be
seen. The loss curve for class is measured using Distribution Focal Loss (the
focal loss helps to address class imbalance in object detection by assigning
different weights to easy and hard examples), and for segmentation is Binary
Cross Entropy with Logits (binary classification loss function that combines
sigmoid transformation of logits with binary cross-entropy calculation, it’s
used for stable and efficient training in binary classification tasks). These
curves display a gradual and consistent decline, highlighting the models’
efficacy in comprehending the data throughout the training process.
Additionally, the mean average precision on both the training and validation
sets is presented in Figures 5(a) and 5(d), showcasing a steady and continuous
upward trend, ultimately converging to yield strong results. The mAp-50 is
calculated by averaging the precision values at the top 50 ranked predictions
for a given task, providing a powerful performance metric for object
detection.

Table I: Box mAP50 for all models and classes. Architecture | Caruru Weed | Grassy Weed | Soy Plant
---|---|---|---
YOLOv8m | 0.789 | 0.628 | 0.887
YOLOv8l | 0.759 | 0.593 | 0.880
YOLOv8x | 0.758 | 0.613 | 0.870
YOLOv5m | 0.744 | 0.623 | 0.866
YOLOv5l | 0.772 | 0.607 | 0.856
YOLOv5x | 0.720 | 0.606 | 0.860

In Table I, mAP-50 metrics for bounding boxes in the test set are shown. The
YOLOv8m model achieved the highest scores, with 0.789 for caruru weed objects,
0.628 for grassy weed objects, and 0.887 for Soy objects. Notably, the YOLOv8
models outperformed the YOLOv5 models across all classes, indicating a
significant improvement.

Table II presents the average bounding box precision and recall metrics for
all classes across all models. YOLOv5m exhibited the highest average precision
at 0.773, while YOLOv5l achieved the best recall at 0.738. Notably, YOLOv8 and
YOLOv5 models demonstrated similar results in these metrics.

Table II: Average box precision and recall for all models and classes. Architecture | Precision (all classes) | Recall (all classes)
---|---|---
YOLOv8m | 0.767 | 0.737
YOLOv8l | 0.729 | 0.719
YOLOv8x | 0.772 | 0.693
YOLOv5m | 0.773 | 0.728
YOLOv5l | 0.770 | 0.738
YOLOv5x | 0.746 | 0.730

The segmentation mAP-50 values can be seen in Table III, with YOLOv8m
consistently outperforming other models across all classes. Specifically,
YOLOv8m achieved a mAP of 0.787 for caruru and 0.696 for grassy weed, and an
impressive 0.901 for Soy. Once more, the YOLOv8 models demonstrated superior
performance compared to the YOLOv5 models.

Table III: Segmentation mAP50 for all models and classes. Architecture | Caruru Weed | Grassy Weed | Soy Plant
---|---|---|---
YOLOv8m | 0.787 | 0.696 | 0.901
YOLOv8l | 0.777 | 0.614 | 0.895
YOLOv8x | 0.761 | 0.591 | 0.888
YOLOv5m | 0.727 | 0.634 | 0.884
YOLOv5l | 0.742 | 0.610 | 0.875
YOLOv5x | 0.707 | 0.603 | 0.870

Precision and recall values for segmentation are provided at Table IV. YOLOv8x
achieved the highest precision at 0.791, while YOLOv8m attained the best
recall at 0.733. Notably, precision and recall values across all models were
quite similar for this task.

To provide a more comprehensive illustration of model predictions on test
images from the dataset, Fig. 6 presents sample predictions for YOLOv8x across
the initial, medium, and advanced stages of soy growth. This includes precise
segmentation and classification of soy, caruru, and grassy weeds,
demonstrating the model’s accuracy in delineating these plants.

##  VI Analysis of performance on initial and late stages

In this section, the primary focus revolves around the neural network’s role
in effectively detecting weed presence. Our analysis will be dedicated to
evaluating the performance of the YOLOv8 models on images featuring a
coexistence of both weed and soy elements, through box predictions rather then
mask predictions. We curated two distinct sets of 25 images, each depicting
varying degrees of weed presence. The initial group comprises images captured
during the seedling and early stages of plantation, while the second group
encapsulates images from the harvest stages. Our objective is to test the
YOLOv8 models’ efficacy in accurately identifying weed instances in these
diverse agricultural scenarios, and if there is a several difference in
performance from early to late stages.

Table IV: Average segmentation precision and recall for all models and classes. Architecture | Precision (all classes) | Recall (all classes)
---|---|---
YOLOv8m | 0.784 | 0.733
YOLOv8l | 0.743 | 0.732
YOLOv8x | 0.791 | 0.692
YOLOv5m | 0.770 | 0.721
YOLOv5l | 0.766 | 0.731
YOLOv5x | 0.743 | 0.716
Figure 6: YOLOv8X model predictions over multiple stages of soy growth.

Considering the box mAP50 results in Table V and Table VI, it is evident that
the models perform better when both soy and weeds are present in the image.
This improved performance might be attributed to the networks producing false
positives in the absence of weeds. However, this is not a significant concern
given the task at hand, especially since precision remains high in scenarios
where only soy is present.

Table V: Box mAP50 for all models and classes in early stages. Architecture | Caruru Weed | Grassy Weed | Soy Plant
---|---|---|---
YOLOv8m | 0.995 | 0.915 | 0.936
YOLOv8l | 0.995 | 0.838 | 0.930
YOLOv8x | 0.995 | 0.981 | 0.946
Table VI: Box mAP50 for all models and classes in late stages. Architecture | Caruru Weed | Grassy Weed | Soy Plant
---|---|---|---
YOLOv8m | 0.970 | 0.884 | 0.905
YOLOv8l | 0.933 | 0.901 | 0.901
YOLOv8x | 0.984 | 0.914 | 0.958

Another noteworthy observation from the tabulated data is the lack of
significant disparities in model performance across different stages of
plantation. This indicates that the dataset is diverse and that the trained
models can effectively handle a variety of agricultural scenarios.

Considering the box precision and recall metrics in Table VII and Table VIII,
it is apparent that no single model distinctly outperforms the others. All
YOLOv8 models consistently deliver reliable results in detecting both soy and
weed in the images. A subtle trend shows a slight improvement in precision and
recall metrics in the later stages of plantation, although this enhancement is
not particularly significant.

Table VII: Average box precision and recall for all models and classes in early stages. Architecture | Precision (all classes) | Recall (all classes)
---|---|---
YOLOv8m | 0.868 | 0.943
YOLOv8l | 0.954 | 0.873
YOLOv8x | 0.916 | 0.909
Table VIII: Average box precision and recall for all models and classes in late stages. Architecture | Precision (all classes) | Recall (all classes)
---|---|---
YOLOv8m | 0.885 | 0.841
YOLOv8l | 0.858 | 0.864
YOLOv8x | 0.879 | 0.933

In conclusion, this analysis highlights the consistent performance of the
networks across different stages of plantation, demonstrating the dataset’s
diversity and the models’ robust generalization capabilities. Additionally, it
reveals a significant performance boost in images featuring both soy and weed,
aligning with the primary objective of segmenting and detecting these
elements. This reaffirms the models’ effectiveness in achieving their intended
purpose.

##  VII Discussion

In this section, we discuss several key points that emerged from the results
and analysis.

Firstly, it is evident that the YOLOv8 models consistently outperform the
YOLOv5 models. However, it is intriguing that the YOLOv8m model, with
significantly fewer parameters, can sometimes outperform the YOLOv8x model.
Our best hypothesis is that the dataset, containing only three classes, may
lead to overfitting in the more complex YOLOv8x model, resulting in slightly
lower performance compared to the smaller YOLOv8m model in certain cases.

Another important observation is the distinct difference in both box and
segmentation mAP50 metrics for different classes. The precision for soy is
significantly higher than for the two weed classes. This discrepancy likely
arises because the dataset was collected in a soy plantation, resulting in a
higher prevalence of soy compared to weeds, both in terms of pixel count and
presence percentage in images.

It is crucial to clarify that the primary goal of employing these models was
not to achieve state-of-the-art results per se, but to validate our dataset.
This is why we utilized existing models rather than developing a new one
specifically tailored to this task.

##  VIII Conclusion

This paper introduces a dataset containing 1000 images capturing the entire
life cycle of soy crops, from seedling to harvest, while also documenting weed
infestation over time. Each image provides a comprehensive view of the
plantation and includes instance segmentation labeling. This dataset holds
promising scientific value for applications related to weed detection, soy
monitoring, and serves as a robust benchmark for neural network architectures
due to its comprehensiveness and practicality.

The paper further includes analysis and comparison of state-of-the-art YOLO
models using this dataset, contrasting the latest model iteration with the
previous YOLOv5. The models trained on this dataset demonstrate exceptional
precision in plant segmentation, and validate the quality of the data.

Looking ahead, potential future work could explore the extension of this
dataset’s methodology for crop disease detection and yield prediction,
capitalizing on the temporal aspect of the dataset, which offers valuable
practical applications.

## Acknowledgements

This research was funded by: Topsector TKI AgroFood under grant agreement
LWV20242 and Ministry of Agriculture, Nature and Food Quality grant agreement
BO-69-001-005 for “Smart technology for soybean production”.

The authors express their gratitude for the support and data contributions
from various organizations, including Wageningen University & Research, the
Aquarius Project at Universidade Federal de Santa Maria, the Advanced Farm 360
project, Rometron, and Stara. Special thanks are extended to Ard Nieuwenhuizen
and Telmo Jorge Carneiro Amado for their invaluable support throughout the
research process.

## References

  * [1] M. C. Pagano and M. Miransari, “The importance of soybean production worldwide,” in _Abiotic and biotic stresses in soybean production_. Elsevier, 2016, pp. 1–26.
  * [2] S. Kujawa and G. Niedbała, “Artificial neural networks in agriculture,” p. 497, 2021.
  * [3] A. Escamilla-García, G. M. Soto-Zarazúa, M. Toledano-Ayala, E. Rivas-Araiza, and A. Gastélum-Barrios, “Applications of artificial neural networks in greenhouse technology and overview for smart agriculture development,” _Applied Sciences_ , vol. 10, no. 11, p. 3835, 2020.
  * [4] H. A. Williams, M. H. Jones, M. Nejati, M. J. Seabright, J. Bell, N. D. Penhall, J. J. Barnett, M. D. Duke, A. J. Scarfe, H. S. Ahn _et al._ , “Robotic kiwifruit harvesting using machine vision, convolutional neural networks, and robotic arms,” _biosystems engineering_ , vol. 181, pp. 140–156, 2019.
  * [5] F. Vulpi, A. Milella, R. Marani, and G. Reina, “Recurrent and convolutional neural networks for deep terrain classification by autonomous robots,” _Journal of Terramechanics_ , vol. 96, pp. 119–131, 2021.
  * [6] W. Rawat and Z. Wang, “Deep convolutional neural networks for image classification: A comprehensive review,” _Neural computation_ , vol. 29, no. 9, pp. 2352–2449, 2017.
  * [7] A. Kamilaris and F. X. Prenafeta-Boldú, “A review of the use of convolutional neural networks in agriculture,” _The Journal of Agricultural Science_ , vol. 156, no. 3, pp. 312–322, 2018.
  * [8] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2015, pp. 3431–3440.
  * [9] Y. Guo, Y. Liu, T. Georgiou, and M. S. Lew, “A review of semantic segmentation using deep neural networks,” _International journal of multimedia information retrieval_ , vol. 7, pp. 87–93, 2018.
  * [10] A. Olsen, D. A. Konovalov, B. Philippa, P. Ridd, J. C. Wood, J. Johns, W. Banks, B. Girgenti, O. Kenny, J. Whinney _et al._ , “Deepweeds: A multiclass weed species image dataset for deep learning,” _Scientific reports_ , vol. 9, no. 1, p. 2058, 2019.
  * [11] P. Wang, Y. Tang, F. Luo, L. Wang, C. Li, Q. Niu, and H. Li, “Weed25: A deep learning dataset for weed identification,” _Frontiers in Plant Science_ , vol. 13, p. 1053329, 2022.
  * [12] M. E. Mignoni, A. Honorato, R. Kunst, R. Righi, and A. Massuquetti, “Soybean images dataset for caterpillar and diabrotica speciosa pest detection and classification,” _Data in Brief_ , vol. 40, p. 107756, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2352340921010313
  * [13] M. Francis and C. Deisy, “Disease detection and classification in agricultural plants using convolutional neural networks—a visual understanding,” in _2019 6th international conference on signal processing and integrated networks (SPIN)_. IEEE, 2019, pp. 1063–1068.
  * [14] P. Bedi and P. Gole, “Plant disease detection using hybrid model based on convolutional autoencoder and convolutional neural network,” _Artificial Intelligence in Agriculture_ , vol. 5, pp. 90–101, 2021.
  * [15] A. M. Hasan, F. Sohel, D. Diepeveen, H. Laga, and M. G. Jones, “A survey of deep learning techniques for weed detection from images,” _Computers and Electronics in Agriculture_ , vol. 184, p. 106067, 2021.
  * [16] W. Ramirez, P. Achanccaray, L. Mendoza, and M. Pacheco, “Deep convolutional neural networks for weed detection in agricultural crops using optical aerial images,” in _2020 IEEE Latin American GRSS & ISPRS Remote Sensing Conference (LAGIRS)_. IEEE, 2020, pp. 133–137.
  * [17] D. Hu, C. Ma, Z. Tian, G. Shen, and L. Li, “Rice weed detection method on yolov4 convolutional neural network,” in _2021 international conference on artificial intelligence, big data and algorithms (CAIBDA)_. IEEE, 2021, pp. 41–45.
  * [18] K. Sudars, J. Jasko, I. Namatevs, L. Ozola, and N. Badaukis, “Dataset of annotated food crops and weed images for robotic computer vision control,” _Data in brief_ , vol. 31, p. 105833, 2020.
  * [19] M. S. Alam, M. Alam, M. Tufail, M. U. Khan, A. Güneş, B. Salah, F. E. Nasir, W. Saleem, and M. T. Khan, “Tobset: A new tobacco crop and weeds image dataset and its utilization for vision-based spraying by agricultural robots,” _Applied Sciences_ , vol. 12, no. 3, p. 1308, 2022.
  * [20] E. Medhi and N. Deb, “Psfd-musa: A dataset of banana plant, stem, fruit, leaf, and disease,” _Data in Brief_ , vol. 43, p. 108427, 2022.
  * [21] E. Moupojou, A. Tagne, F. Retraint, A. Tadonkemwa, D. Wilfried, H. Tapamo, and M. Nkenlifack, “Fieldplant: A dataset of field plant images for plant disease detection and classification with deep learning,” _IEEE Access_ , vol. 11, pp. 35 398–35 410, 2023.
  * [22] J. Yang, S. Guo, M. J. Bocus, Q. Chen, and R. Fan, “Semantic segmentation for autonomous driving,” in _Autonomous Driving Perception: Fundamentals and Applications_. Springer, 2023, pp. 101–137.
  * [23] Z. Zhou, J. Zhang, and C. Gong, “Hybrid semantic segmentation for tunnel lining cracks based on swin transformer and convolutional neural network,” _Computer-Aided Civil and Infrastructure Engineering_ , 2023.
  * [24] T. Anand, S. Sinha, M. Mandal, V. Chamola, and F. R. Yu, “Agrisegnet: Deep aerial semantic segmentation framework for iot-assisted precision agriculture,” _IEEE Sensors Journal_ , vol. 21, no. 16, pp. 17 581–17 590, 2021.
  * [25] P. Bosilj, E. Aptoula, T. Duckett, and G. Cielniak, “Transfer learning between crop types for semantic segmentation of crops versus weeds in precision agriculture,” _Journal of Field Robotics_ , vol. 37, no. 1, pp. 7–19, 2020.
  * [26] Roboflow, “Roboflow: A tool for computer vision workflow,” 2023, https://roboflow.com/.
  * [27] Q. Lin, G. Ye, J. Wang, and H. Liu, “Roboflow: a data-centric workflow management system for developing ai-enhanced robots,” in _Conference on Robot Learning_. PMLR, 2022, pp. 1789–1794.
  * [28] G. Jocher _et al._ , “Yolov5: Fifth version of you only look once for real-time object detection,” 2020, https://github.com/ultralytics/yolov5.
  * [29] Ultralytics, “Ultralytics - yolov8,” 2023, https://github.com/ultralytics/ultralytics.
  * [30] T. Ridnik, E. Ben-Baruch, A. Noy, and L. Zelnik-Manor, “Imagenet-21k pretraining for the masses,” _arXiv preprint arXiv:2104.10972_ , 2021.


the interface. The shear stress scales as

$\displaystyle\tau^{b}_{\mu}\sim\frac{\mu^{b}U_{s}}{b},$

where $b$ is the thickness of the lower fluid and $U_{s}$ is the unknown
characteristic velocity scale to be determined in what follows. Furthermore,
the surface tension gradient scales as

$\displaystyle\frac{d\sigma}{dx}\sim\frac{d\sigma}{dT}\frac{\Delta T}{l},$

where $l$ is the length of the microchannel, and for SRFs with a quadratic
dependence of surface tension on temperature (see Eq. (4))

$\displaystyle\frac{d\sigma}{dT}=\sigma_{T}+2\sigma_{TT}\left(T-T_{ref}\right).$

Thus, the velocity scale can be deduced by from the first two equations above
by setting $\tau^{b}_{\mu}\sim d\sigma/dx$ and then substituting the last
equation for $d\sigma/dT$ as

$\displaystyle U_{s}\sim\frac{\Delta
T}{\mu^{b}}\left(\frac{b}{l}\right)\frac{d\sigma}{dT}=\frac{\Delta
T}{\mu^{b}}\left(\frac{b}{l}\right)\left[\sigma_{T}+2\sigma_{TT}\left(T-T_{ref}\right)\right]$
(64)

Here, we need a scale for the temperature on the interface, which we take it
to be temperature field at $x=0$ and $y=0$, where it reaches a maximum. Now,
using the temperature along the interface given as (see Appendix A)

$\displaystyle T(x,y=0)=C_{1}+C_{2}\cos\left(\omega x\right),$

and evaluating it at $x=0$, we get

$\displaystyle T(x=0,y=0)=C_{1}+C_{2},$ (65)

where

$\displaystyle
C_{1}=\frac{T_{h}\left(\frac{a}{b}\right)+T_{c}\tilde{k}}{\left(\frac{a}{b}\right)+\tilde{k}},\qquad
C_{2}=\frac{\Delta
T\sinh(\tilde{a})}{\tilde{k}\cosh(\tilde{a})\sinh(\tilde{b})+\cosh(\tilde{b})\sinh(\tilde{a})},$
(66)

Here, $\tilde{k}=k_{a}/k_{b}$, $\tilde{a}=a\omega$, and $\tilde{b}=b\omega$.
Substituting the above estimate for the temperature scale on the interface in
Eq. (64), we finally obtain the characteristic thermocapillary velocity scale
in SRFs as

$\displaystyle U_{s}\sim\frac{\Delta
T}{\mu^{b}}\left(\frac{b}{l}\right)\left[\sigma_{T}+2\sigma_{TT}\left(C_{1}+C_{2}-T_{ref}\right)\right].$
(67)

## References

  * [1] Pierre-Gilles De Gennes, Françoise Brochard-Wyart, David Quéré, et al.  Capillarity and wetting phenomena: drops, bubbles, pearls, waves, volume 315.  Springer, 2004.
  * [2] LE Scriven and CV Sternling.  The Marangoni effects.  Nature, 187(4733):186–188, 1960.
  * [3] Ronald F Probstein.  Physicochemical hydrodynamics: an introduction.  John Wiley & Sons, 2005.
  * [4] NO Young, JS Goldstein, and Mi J Block.  The motion of bubbles in a vertical temperature gradient.  Journal of Fluid Mechanics, 6(3):350–356, 1959.
  * [5] RS Subramanian, R Balasubramaniam, and N Clark.  Motion of bubbles and drops in reduced gravity.  Applied Mechanics Reviews, 55(3):B56, 2002.
  * [6] Samuel WJ Welch.  Transient thermocapillary migration of deformable bubbles.  Journal of colloid and interface science, 208(2):500–508, 1998\.
  * [7] Chen Ma and Dieter Bothe.  Direct numerical simulation of thermocapillary flow based on the volume of fluid method.  International Journal of Multiphase Flow, 37(9):1045–1058, 2011\.
  * [8] Anton A Darhuber and Sandra M Troian.  Principles of microfluidic actuation by modulation of surface stresses.  Annual Review of Fluid Mechanics, 37:425–455, 2005.
  * [9] Alireza Karbalaei, Ranganathan Kumar, and Hyoung Jin Cho.  Thermocapillarity in microfluidics—a review.  Micromachines, 7(1):13, 2016.
  * [10] Bhushan Pendse and Asghar Esmaeeli.  An analytical solution for thermocapillary-driven convection of superimposed fluids at zero reynolds and marangoni numbers.  International Journal of Thermal Sciences, 49(7):1147–1155, 2010\.
  * [11] Tatiana Gambaryan-Roisman.  Modulation of Marangoni convection in liquid films.  Advances in colloid and interface science, 222:319–331, 2015.
  * [12] R Vochten and Georges Petre.  Study of the heat of reversible adsorption at the air-solution interface. ii. experimental determination of the heat of reversible adsorption of some alcohols.  Journal of Colloid and Interface Science, 42(2):320–327, 1973.
  * [13] Georges Petre and Maherzia Aza Azouni.  Experimental evidence for the minimum of surface tension with temperature at aqueous alcohol solution/air interfaces.  Journal of Colloid and Interface Science, 98(1):261–263, 1984.
  * [14] MC Limbourg-Fontaine, Georges Pétré, and Jean Claude Legros.  Thermocapillary movements under microgravity at a minimum of surface tension.  Naturwissenschaften, 73(7):360–362, 1986.
  * [15] D Villers and JK Platten.  Temperature dependence of the interfacial tension between water and long-chain alcohols.  The Journal of Physical Chemistry, 92(14):4023–4024, 1988.
  * [16] Yoshiyuki Abe, Akira Iwasaki, and Kotaro Tanaka.  Microgravity experiments on phase change of self-rewetting fluids.  Annals of the New York Academy of Sciences, 1027(1):269–285, 2004\.
  * [17] Raffaele Savino, Anselmo Cecere, and Roberto Di Paola.  Surface tension-driven flow in wickless heat pipes with self-rewetting fluids.  International Journal of Heat and Fluid Flow, 30(2):380–388, 2009\.
  * [18] Yoshiyuki Abe.  Terrestrial and microgravity applications of self-rewetting fluids.  Microgravity Science and Technology, 19(3):11–12, 2007.
  * [19] Raffaele Savino, Anselmo Cecere, Stefan Van Vaerenbergh, Yoshiyuki Abe, G Pizzirusso, W Tzevelecos, Mohamed Mojahed, and Quentin Galand.  Some experimental progresses in the study of self-rewetting fluids for the SELENE experiment to be carried in the thermal platform 1 hardware.  Acta Astronautica, 89:179–188, 2013.
  * [20] Yanxin Hu, Tengqing Liu, Xuanyou Li, and Shuangfeng Wang.  Heat transfer enhancement of micro oscillating heat pipes with self-rewetting fluid.  International Journal of Heat and Mass Transfer, 70:496–503, 2014\.
  * [21] Shen-Chun Wu, Tien-Ju Lee, Wei-Jhih Lin, and Yau-Ming Chen.  Study of self-rewetting fluid applied to loop heat pipe with ptfe wick.  Applied Thermal Engineering, 119:622–628, 2017.
  * [22] Anselmo Cecere, Davide De Cristofaro, Raffaele Savino, Vincent Ayel, Thibaud Sole-Agostinelli, Marco Marengo, Cyril Romestant, and Yves Bertin.  Experimental analysis of a flat plate pulsating heat pipe with self-rewetting fluids during a parabolic flight campaign.  Acta Astronautica, 147:454–461, 2018.
  * [23] Minghan Zhu, Jin Huang, Mengjie Song, and Yanxin Hu.  Thermal performance of a thin flat heat pipe with grooved porous structure.  Applied Thermal Engineering, 173:115215, 2020.
  * [24] Anze Sitar and Iztok Golobic.  Heat transfer enhancement of self-rewetting aqueous n-butanol solutions boiling in microchannels.  International Journal of Heat and Mass Transfer, 81:198–206, 2015\.
  * [25] K Sefiane, X Yu, Gail Duursma, and J Xu.  On heat and mass transfer using evaporating self-rewetting mixtures in microchannels.  Applied Thermal Engineering, 179:115662, 2020.
  * [26] Yanxin Hu, Suling Zhang, Xuanyou Li, and Shuangfeng Wang.  Heat transfer enhancement of subcooled pool boiling with self-rewetting fluid.  International Journal of Heat and Mass Transfer, 83:64–68, 2015\.
  * [27] Yanxin Hu, Sixu Chen, Jin Huang, and Mengjie Song.  Marangoni effect on pool boiling heat transfer enhancement of self-rewetting fluid.  International Journal of Heat and Mass Transfer, 127:1263–1270, 2018.
  * [28] Yanxin Hu, Hai Wang, Mengjie Song, and Jin Huang.  Marangoni effect on microbubbles emission boiling generation during pool boiling of self-rewetting fluid.  International Journal of Heat and Mass Transfer, 134:10–16, 2019\.
  * [29] Yong Il Kim, Boo-Hyoung Bang, Keunhee Jang, Seongpil An, Alexander L Yarin, and Sam S Yoon.  Pool boiling enhancement via nanotexturing and self-propelled swing motion for bubble shedding.  International Communications in Heat and Mass Transfer, 133:105934, 2022.
  * [30] Ibrahim Zaaroura, Souad Harmand, Julien Carlier, Malika Toubal, Aurélie Fasquelle, and Bertrand Nongaillard.  Thermal performance of self-rewetting gold nanofluids: Application to two-phase heat transfer devices.  International Journal of Heat and Mass Transfer, 174:121322, 2021\.
  * [31] Martin ER Shanahan and Khellil Sefiane.  Recalcitrant bubbles.  Scientific Reports, 4(1):1–9, 2014.
  * [32] Dimitrios Mamalis, Vasileios Koutsos, and Khellil Sefiane.  Bubble rise in a non-isothermal self-rewetting fluid and the role of thermocapillarity.  International Journal of Thermal Sciences, 117:146–162, 2017.
  * [33] Yanxin Hu, Kaixin Huang, and Jin Huang.  A review of boiling heat transfer and heat pipes behaviour with self-rewetting fluids.  International Journal of Heat and Mass Transfer, 121:107–118, 2018\.
  * [34] Alexander Oron and Philip Rosenau.  On a nonlinear thermocapillary effect in thin liquid layers.  Journal of Fluid Mechanics, 273:361–374, 1994.
  * [35] William Batson, Yehuda Agnon, and Alex Oron.  Thermocapillary modulation of self-rewetting films.  Journal of Fluid Mechanics, 819:562–591, 2017.
  * [36] Z Yu.  Thermocapillary instability of self-rewetting films on vertical fibers.  Physics of Fluids, 30(8):082104, 2018.
  * [37] SG Slavtchev and SP Miladinova.  Thermocapillary flow in a liquid layer at minimum in surface tension.  Acta Mechanica, 127(1):209–224, 1998.
  * [38] Manoj Kumar Tripathi, KC Sahu, G Karapetsas, K Sefiane, and OK Matar.  Non-isothermal bubble rise: non-monotonic dependence of surface tension on temperature.  Journal of Fluid Mechanics, 763:82–108, 2015.
  * [39] BR Duffy, SK Wilson, JJA Conn, and K Sefiane.  Unsteady motion of a long bubble or droplet in a self-rewetting system.  Physical Review Fluids, 3(12):123603, 2018.
  * [40] Mounika Balla, Manoj Kumar Tripathi, Kirti Chandra Sahu, George Karapetsas, and Omar K Matar.  Non-isothermal bubble rise dynamics in a self-rewetting fluid: three-dimensional effects.  Journal of Fluid Mechanics, 858:689–713, 2019.
  * [41] Mohammad Majidi, Reza Haghani-Hassan-Abadi, and Mohammad-Hassan Rahimian.  Single recalcitrant bubble simulation using a hybrid lattice Boltzmann finite difference model.  International Journal of Multiphase Flow, 127:103289, 2020.
  * [42] TR Mitchell, M Majidi, MH Rahimian, and CR Leonardi.  Computational modeling of three-dimensional thermocapillary flow of recalcitrant bubbles using a coupled lattice Boltzmann-finite difference method.  Physics of Fluids, 33(3):032108, 2021.
  * [43] Roberto Benzi, Sauro Succi, and Massimo Vergassola.  The lattice Boltzmann equation: theory and applications.  Physics Reports, 222(3):145–197, 1992.
  * [44] Dazhi Yu, Renwei Mei, Li-Shi Luo, and Wei Shyy.  Viscous flow computations with the method of lattice Boltzmann equation.  Progress in Aerospace sciences, 39(5):329–367, 2003.
  * [45] Pierre Lallemand, Li-shi Luo, Manfred Krafczyk, and Wen-An Yong.  The lattice Boltzmann method for nearly incompressible flows.  Journal of Computational Physics, 431:109713, 2021.
  * [46] Xiaoyi He, Shiyi Chen, and Raoyang Zhang.  A lattice Boltzmann scheme for incompressible multiphase flow and its application in simulation of Rayleigh–Taylor instability.  Journal of computational physics, 152(2):642–663, 1999.
  * [47] Xiaoyi He and Gary D Doolen.  Thermodynamic foundations of kinetic theory and lattice Boltzmann models for multiphase flows.  Journal of Statistical Physics, 107(1):309–328, 2002.
  * [48] Taehun Lee and Ching-Long Lin.  A stable discretization of the lattice Boltzmann equation for simulation of incompressible two-phase flows at high density ratio.  Journal of Computational Physics, 206(1):16–47, 2005.
  * [49] Kannan N Premnath and John Abraham.  Three-dimensional multi-relaxation time (mrt) lattice-Boltzmann models for multiphase flow.  Journal of Computational Physics, 224(2):539–559, 2007.
  * [50] Farzaneh Hajabdollahi, Kannan N Premnath, and Samuel WJ Welch.  Central moment lattice Boltzmann method using a pressure-based formulation for multiphase flows at high density ratios and including effects of surface tension and Marangoni stresses.  Journal of Computational Physics, 425:109893, 2021.
  * [51] Haihu Liu, Yonghao Zhang, and Albert J Valocchi.  Modeling and simulation of thermocapillary flows using lattice Boltzmann method.  Journal of Computational Physics, 231(12):4433–4453, 2012.
  * [52] Martin Geier, Andreas Greiner, and Jan G Korvink.  Cascaded digital lattice Boltzmann automata for high reynolds number flow.  Physical Review E, 73(6):066705, 2006.
  * [53] William E Langlois and Michel O Deville.  Slow viscous flow, volume 173436.  Springer, 2014.
  * [54] Pao-Hsiung Chiu and Yan-Ting Lin.  A conservative phase field method for solving incompressible two-phase flows.  Journal of Computational Physics, 230(1):185–204, 2011.
  * [55] Ying Sun and Christoph Beckermann.  Sharp interface tracking using the phase-field equation.  Journal of Computational Physics, 220(2):626–653, 2007.
  * [56] R Folch, J Casademunt, A Hernández-Machado, and L Ramirez-Piscina.  Phase-field model for hele-shaw flows with arbitrary viscosity contrast. i. theoretical approach.  Physical Review E, 60(2):1724, 1999.
  * [57] Jeremiah U Brackbill, Douglas B Kothe, and Charles Zemach.  A continuum method for modeling surface tension.  Journal of computational physics, 100(2):335–354, 1992.
  * [58] Anand Kumar.  Isotropic finite-differences.  Journal of Computational Physics, 201(1):109–118, 2004.
  * [59] Hang Ding, Peter DM Spelt, and Chang Shu.  Diffuse interface model for incompressible two-phase flows with large density ratios.  Journal of Computational Physics, 226(2):2078–2095, 2007.
  * [60] Kannan N Premnath and Sanjoy Banerjee.  Incorporating forcing terms in cascaded lattice Boltzmann approach by method of central moments.  Physical Review E, 80(3):036702, 2009.
  * [61] Eman Yahia and Kannan N Premnath.  Central moment lattice Boltzmann method on a rectangular lattice.  Physics of Fluids, 33(5):057110, 2021.
  * [62] Eman Yahia, William Schupbach, and Kannan N Premnath.  Three-dimensional central moment lattice Boltzmann method on a cuboid lattice for anisotropic and inhomogeneous flows.  Fluids, 6(9):326, 2021.
  * [63] Martin Geier, Martin Schönherr, Andrea Pasquali, and Manfred Krafczyk.  The cumulant lattice Boltzmann equation in three dimensions: Theory and validation.  Computers & Mathematics with Applications, 70(4):507–547, 2015\.
  * [64] Timm Krüger, Halim Kusumaatmaja, Alexandr Kuzmin, Orest Shardt, Goncalo Silva, and Erlend Magnus Viggen.  The lattice Boltzmann method.  Springer International Publishing, 10(978-3):4–15, 2017.
  * [65] Zhenlin Guo and Ping Lin.  A thermodynamically consistent phase-field model for two-phase flows with thermocapillary effects.  Journal of Fluid Mechanics, 766:226–271, 2015.
  * [66] Lin Zheng, Song Zheng, and Qinglan Zhai.  Continuous surface force based lattice Boltzmann equation method for simulating thermocapillary flow.  Physics Letters A, 380(4):596–603, 2016.
  * [67] Seyed Amin Nabavizadeh, Mohsen Eshraghi, Sergio D Felicelli, Surendra N Tewari, and Richard N Grugel.  Effect of bubble-induced Marangoni convection on dendritic solidification.  International Journal of Multiphase Flow, 116:137–152, 2019.


5 | 0.15 | 40 | 4.26 $\pm$ 0.821 | 2.04 $\pm$ 0.315 | 1.53 $\pm$ 0.187 | 1.13 $\pm$ 0.137 | 1.33 $\pm$ 0.169

$D$ | $l$ | $H$ | $\phi$ | Attribution RMSE ($\downarrow$)
---|---|---|---|---
CAFE (0) | CAFE (0.25) | CAFE (0.5) | CAFE (0.75) | CAFE (1)
2 | 0.30 | 16 | ReLU | 3.27 $\pm$ 0.985 | 2.61 $\pm$ 0.737 | 1.85 $\pm$ 0.500 | 1.01 $\pm$ 0.274 | 0.35 $\pm$ 0.401
3 | 0.25 | 24 | 2.66 $\pm$ 0.375 | 2.06 $\pm$ 0.296 | 1.44 $\pm$ 0.227 | 0.80 $\pm$ 0.195 | 0.43 $\pm$ 0.216
4 | 0.20 | 32 | 2.57 $\pm$ 0.430 | 1.98 $\pm$ 0.355 | 1.39 $\pm$ 0.303 | 0.82 $\pm$ 0.309 | 0.52 $\pm$ 0.410
5 | 0.15 | 40 | 2.13 $\pm$ 0.304 | 1.62 $\pm$ 0.231 | 1.13 $\pm$ 0.156 | 0.67 $\pm$ 0.081 | 0.42 $\pm$ 0.056
2 | 0.30 | 16 | GELU | 3.14 $\pm$ 0.970 | 2.48 $\pm$ 0.744 | 1.71 $\pm$ 0.504 | 0.90 $\pm$ 0252 | 0.74 $\pm$ 0.242
3 | 0.25 | 24 | 2.56 $\pm$ 0.385 | 1.96 $\pm$ 0.297 | 1.29 $\pm$ 0.213 | 0.62 $\pm$ 0.129 | 0.60 $\pm$ 0.082
4 | 0.20 | 32 | 2.37 $\pm$ 0.406 | 1.82 $\pm$ 0.330 | 1.24 $\pm$ 0.262 | 0.75 $\pm$ 0.219 | 1.01 $\pm$ 0.280
5 | 0.15 | 40 | 2.05 $\pm$ 0.306 | 1.56 $\pm$ 0.222 | 1.03 $\pm$ 0.138 | 0.60 $\pm$ 0.052 | 0.88 $\pm$ 0.243

The data generation process for the synthetic data experiments is
parameterised with a dimension $D$ specifying the number of continuous
features, a cancellation likelihood $l$ determining the likelihood of each
feature in each sample being cancelled, a standard deviation $s$ to be used
for constructing a diagonal covariance matrix of a multivariate Gaussian
distribution and a range $[a,b]$ for the randomly generated feature weights.
During the data generation, we first sample the weights
${\bm{w}}\in\mathbb{R}^{D}$ for each of $D$ continuous features from a uniform
distribution $U(a,b)$. We then generate each sample $[{\bm{x}},{\bm{c}}]$ by
drawing the continuous features ${\bm{x}}\in\mathbb{R}^{D}$ from a Gaussian
distribution $\mathcal{N}(\mathbf{0},\text{diag}(s^{2}))$ and independently
drawing the binary cancellation features ${\bm{c}}\in\\{0,1\\}^{D}$ from a
Bernoulli distribution $\text{Bernoulli}(l)$. The continuous label for each
sample is then derived as $y=\sum_{i}\mathds{1}[c_{i}=0]w_{i}x_{i}$. That is,
the influence of each continuous feature is only reflected in the label if the
corresponding cancellation feature is zero and is cancelled otherwise. With
the labels derived in this way, the natural “ground-truth” attribution score
for the $i$-th continuous feature is $x_{i}w_{i}$, while the score for the
corresponding cancellation feature is $-\mathds{1}[c_{i}=1]w_{i}x_{i}$.

Using the described data generation procedure, we generate a collection of
datasets with different parameters and conduct a series of experiments. For
each experiment, we generated five datasets with $10000$ samples using
different random seeds. These datasets were split into training, validation
and test portions in the ratio of $6:2:2$. On each of the training portions,
we trained five differently initialised neural models with two hidden layers
for $2000$ epochs using the AdamW optimizer (Loshchilov and Hutter,, 2019) and
selected the model with the best performance on the validation dataset. All
the selected models achieved a low error on the test dataset, giving us
confidence that their internal reasoning should be aligned with the data
generation process. We then evaluated CAFE and the baseline feature
attribution methods666Our experimental comparison does not include DeepLIFT
RevealCancel. Although it is relevant to our work, the only available software
implementation is for Tensorflow v1, which is incompatible with the rest of
our software tooling. by computing the root-mean-square error (RMSE) between
the attribution scores returned by these methods for the samples in the test
set and the expected “ground-truth” scores. For the baseline methods, we used
the implementations available in Captum (Kokhlikyan et al.,, 2020), while CAFE
was implemented using PyTorch (Paszke et al.,, 2019). Attribution scores for
methods taking in a reference input (including CAFE) were computed with
respect to a zero vector. In order to make the comparison fair to methods not
distinguishing between positive and negative scores, we only considered the
combined CAFE scores obtained by subtracting the negative scores from the
positive ones. In each experiment, we averaged the results over the five
generated datasets and selected models. Although most of the results were
already stated in the main paper, in this supplement, we provide an expanded
version including the standard deviations — see Table 5.

###  E.2 Real Data Experiments

Table 6: Infidelity of the different attribution methods applied to NNs trained on MIMIC-IV for smaller (S, with Gaussian noise standard deviation 0.5 and categorical resampling probability 0.1) and larger (L, standard deviation 0.75, categorical resampling probability 0.2) perturbations. In addition to the means stated in the main paper, we additionally report the standard deviations. Method | Attribution Infidelity ($\downarrow$)
---|---
MIMIC-IV
S | L
Gradient $\cdot$ Input | 2.29 $\pm$ 0.177 | 4.97 $\pm$ 0.458
LRP | 2.29 $\pm$ 0.177 | 4.97 $\pm$ 0.458
DeepLIFT Rescale | 2.20 $\pm$ 0.210 | 4.80 $\pm$ 0.521
Integrated Gradients | 2.21 $\pm$ 0.214 | 4.82 $\pm$ 0.530
SmoothGrad | 3.14 $\pm$ 0.146 | 6.26 $\pm$ 0.449
Gradient SHAP | 2.28 $\pm$ 0.223 | 4.92 $\pm$ 0.559
Kernel SHAP | 2.75 $\pm$ 0.175 | 5.65 $\pm$ 0.474
Shapley Value Sampling | 2.19 $\pm$ 0.150 | 4.76 $\pm$ 0.423
LIME | 2.64 $\pm$ 0.173 | 5.48 $\pm$ 0.460
CAFE ($c=0.00$) | 2.29 $\pm$ 0.177 | 4.97 $\pm$ 0.458
CAFE ($c=0.25$) | 2.08 $\pm$ 0.179 | 4.49 $\pm$ 0.444
CAFE ($c=0.50$) | 2.13 $\pm$ 0.253 | 4.44 $\pm$ 0.520
CAFE ($c=0.75$) | 2.30 $\pm$ 0.325 | 4.64 $\pm$ 0.619
CAFE ($c=1.00$) | 2.48 $\pm$ 0.356 | 4.91 $\pm$ 0.676

As described in the main text, for our real data experiments, we used four
datasets and pretrained models from the OpenXAI benchmark (Agarwal et al.,,
2022) in addition to our own model strained on a subset of the MIMIC-IV
medical database (Johnson et al.,, 2023). Our version of the MIMIC-IV dataset
was constructed from a cohort of all patients with recorded ICU stays. For
each patient, we extracted the admission weight, heart rate (HR), mean blood
pressure (MBP), respiratory rate (RR), oxygen saturation (SpO2), body
temperature, admission type, gender and admission age values from the first
hour of their first ICU stay. These variables were selected as the most
relevant in cooperation with our clinical collaborators. Patients for which
some of the data were unavailable or contained clinically unrealistic values
were excluded from the dataset. We took the 30-day patient mortality as the
label and our prediction target. For the purposes of training of our models,
the continuous values were scaled using a robust scaler from scikit-learn
(Pedregosa et al.,, 2011) using the quantile range $[10,90]$ while the
categorical features were encoded using a one-hot encoding. Additionally, to
reduce the imbalance in the data, we randomly upsampled the patients with the
positive label to form one-third of all samples. The upsampling was performed
for a training set only. Following this process, we obtained a training set
with 52338 samples and a test set with 4586 samples. In the infidelity
evaluation experiment, we trained a collection of five differently initialised
models with two hidden layers of 32 neurons each on the training set for 100
epochs using the AdamW optimizer and averaged the results for these models.
Our models reached an average accuracy of $0.81\pm 0.003$ and AUC of $0.76\pm
0.003$ on the test set.

For computing the infidelity metrics in all experiments (including the ones
for the OpenXAI models), we used a perturbation function adding Gaussian noise
with zero mean and a specified standard deviation to continuous features and
randomly resampling categorical variables with a specified likelihood. We
evaluated the infidelity of the different attribution methods both on smaller
perturbations (with a standard deviation of the Gaussian noise of 0.5 and
categorical resampling probability of 0.1) and larger perturbations (with a
standard deviation of the Gaussian noise of 0.75 and categorical resampling
probability of 0.2). To compute the infidelity metric itself, we used the
respective implementation from Captum (Kokhlikyan et al.,, 2020), using the
normalization option to ensure that the metric is invariant to a constant
scaling of the attributions.

Most of the main results were provided in the main text. In Table 6, we
additionally report the means and standard deviations for the MIMIC-IV
experiments (there are no standard deviations for the OpenXAI experiments, as
the benchmark only contains a single pre-trained model for each dataset). Note
that the perturbations used for computing the infidelity measure were
generated using a different random seed for each of the five explained models,
resulting in a relatively high variability of the infidelity scores over the
different runs. However, considering paired samples for each seed, the
infidelity improvements of CAFE ($c=0.25$) on smaller perturbations and of
CAFE ($c=0.25$) and CAFE ($c=0.5$) on larger perturbations in comparison with
all the considered baseline methods are statistically significant at the
$0.05$ significance level (as determined by t-test for related samples from
SciPy777https://scipy.org/).

##  Appendix F CAFE ATTRIBUTION EXAMPLES

In this section, we consider example CAFE attribution scores for four data
points from our MIMIC-IV dataset — one for a true positive sample (Figure 9),
a true negative sample (Figure 10), a false positive sample (Figure 11) and a
false negative sample (Figure 12). The data points were selected as the first
occurring representatives of the given category in our randomly ordered test
set. In all examples, we compare the CAFE scores with DeepLIFT Rescale, as it
was the overall best-performing gradient-based baseline in our MIMIC
experiments. For fairness and ease of comparison between the two methods, we
only consider the combined positive and negative scores for CAFE.

Figure 9: CAFE and DeepLIFT Rescale attribution scores for a correctly
classified positive sample from our MIMIC-IV dataset (prediction score 0.51)
Figure 10: CAFE and DeepLIFT Rescale attribution scores for a correctly
classified negative sample from our MIMIC-IV dataset (prediction score 0.23)
Figure 11: CAFE and DeepLIFT Rescale attribution scores for an incorrectly
classified negative sample from our MIMIC-IV dataset (prediction score 0.63)
Figure 12: CAFE and DeepLIFT Rescale attribution scores for an incorrectly
classified positive sample from our MIMIC-IV dataset (prediction score 0.35)

In the first example (Figure 9) showing attributions for a true positive, we
can see that all methods except for CAFE (0.0) considered the high age to be
the key feature for predicting a higher likelihood of 30-day mortality, which
seems intuitive. It is possible that the age feature partially interfered with
the other features or the bias, explaining why it was not well captured by
CAFE (0.0). Additionally, we can observe that in comparison to DeepLIFT, CAFE
identifies a strong negative effect of the bias. This is to be expected, as
the model was trained on a partially imbalanced dataset with a majority of
negative samples. Female gender is identified by all methods as a strongly
negatively contributing feature, which may be somewhat counterintuitive. Our
hypothesis is that the model may be using the gender attribute for a partial
representation of its default bias against predicting a positive label, for
both of its possible values. This hypothesis also seems to be supported by the
attributions scores for the other examples, where the gender attribute always
has a relatively strong negative effect on the prediction, regardless of
whether it is male or female (though a female gender seems to be associated
with stronger negative biases in our examples).

In the second example (Figure 10, we consider the attribution scores for a
correctly classified negative sample. We can see that the most strongly
contributing features identified by all the considered methods, are a low mean
blood pressure and an urgent admission, both reasonable features in the given
context. In comparison with other methods, CAFE (0.5) and CAFE (1.0) identify
the relatively low age of 45 as a more important feature, which again seems
intuitive. All variants of CAFE capture a non-negligible negative effect of
the bias, though relatively less important than in the previous example.

The third example (Figure 11) considers an incorrectly classified negative
sample. In this case, the relatively high respiratory rate and heart rate are
most important for the positive classification. This is reasonable, as high
rates for these vital signs typically indicate higher distress and an overall
worse clinical state of the patient. Variants of CAFE with higher cancellation
sensitivity constant consider the heart rate to be the more important of the
two features, while CAFE (0.0) and DeepLIFT state the opposite.

Finally, the fourth example (Figure 12) perhaps most clearly demonstrates the
advantage of considering the effect of the bias in the attribution scores.
Here, the incorrect negative classification of the model seems to be
predominantly caused by the negative bias, which counteracted otherwise strong
positively contributing features, such as the high age (77 years). While all
variants of CAFE clearly show this, DeepLIFT does not.

##  Appendix G FAQ

In this section, we provide answers to several questions that may be relevant
to practitioners and researchers wishing to use or build upon CAFE in their
work.

###  G.1 What activation functions and model classes are supported by CAFE?

In our experiments, we evaluated CAFE on networks using the ReLU and GELU
activations, as we consider these to be one of the most common choices for
recent neural architectures. Nevertheless, CAFE does not make any particular
assumptions regarding the used activation function, and is thus readily
applicable to networks using other common activation functions, such as tanh,
sigmoid, swish or softplus. In theory, CAFE may fail to produce reasonable
attribution scores for highly irregular activation functions with high numbers
of local maxima and minima. However, such functions would likely be unsuitable
for use in neural networks, so we do not see this to be a practical
limitation.

As implemented and used in our work, CAFE is directly applicable to
feedforward model architectures. However, we believe that it would be
relatively easy to extend to other kinds of models, such as convolutional
neural networks and networks using residual connections. As described in (Wang
et al.,, 2019), specialised operations performed by these networks can often
be viewed as linear manipulations with sparse matrices or as applications of
special activation functions. Similarly, CAFE could also be adapted to
transformers using an approach analogical to the one recently proposed for LRP
(Ali et al.,, 2022). Further investigation of these extensions poses a
promising line of future work.

###  G.2 What values should I use for the cancellation sensitivity constants?

Typically, the choice of the cancellation sensitivity constants $c^{(n)}$
should be guided by the particular use-case for the produced attribution
scores. In cases where simpler and less noisy explanations are desired, it
might be preferable to use lower $c^{(n)}$s. However, we would generally
advise against using the value of $0$, as we believe that at least a limited
ability to surface conflicts is crucial for producing faithful explanations —
see the introduction of our paper for the relevant discussion. Meanwhile, in
cases in which a greater sensitivity to conflicts is preferred and where more
noisy scores are less of a concern, we would recommend the use of larger
values of $c^{(n)}$. In the absence of any specific preferences, a value of
$c^{(n)}=0.5$ appears to be a sensible default based on our experimental
results on real data. Alternately, the value of $c^{(n)}$ could be tuned to
maximize the fidelity to the considered model and data. In the case of deeper
models with a larger number of layers, we would suggest using lower values of
$c^{(n)}$ for lower layers and higher values for later layers, as the
conflicts in later layers would typically be more relevant to the model
output.

###  G.3 What value should I use for the reference input?

Similarly to the cancellation sensitivity constant, the appropriate reference
input is likely to be context-dependent. For continuous tabular data, vectors
of zeroes or mean/median values are common choices. We believe the zero values
to be a slightly more principled choice, as it forces the reference model
(i.e., the model without biases) to output zero in regression tasks or a
uniform distribution in classification tasks. Given this, the completeness
property ensures that all changes from this base output are explained by the
attribution scores. Additionally, the zero reference is the only option that
guarantees that the model does not use any of the reference values in its
computations, potentially masking their effect on the prediction during the
computation of the attribution scores. Nevertheless, the use of zero reference
vectors could also be criticised, as these will often be outside of the data
manifold, making the corresponding attributions potentially less grounded in
the behaviour of the model on the data distribution. From this point of view,
the use of mean or median values may be more appropriate. For categorical
data, we believe that a reasonable choice of reference is again zero, although
the mode or a domain-specific base category may also be reasonable choices.

###  G.4 Are there any benefits to using CAFE if I do not care about capturing
conflicts between features?

Firstly, we would like to highlight that consideration of conflicts is helpful
for producing explanations with a higher fidelity, as evidenced in our
experiments. Thus, we believe that using CAFE with non-zero conflict
sensitivity constants may produce generally more useful attribution scores,
even if surfacing conflicts is not the explicit goal. Nevertheless, even using
CAFE (0.0) offers several potential advantages over other methods — in
particular, quantifying the effects of the bias, safeguarding against
attribution score explosion and distinguishing between positive and negative
attribution scores.

###  G.5 What are the considerations to take when explaining models using
CAFE?

Note that the attribution scores computed by CAFE (as well as other
attribution methods) are only approximations of the true behaviour of the
explained model and may not provide a fully accurate account of the effects of
the individual input features. In particular, “reasonably-looking” attribution
scores for a particular example are not necessarily a guarantee that the
explained model functions correctly or makes correct decisions. Past research
has also suggested that cognitive biases of explanation recipients may lead
them to incorrect judgements about the behaviour of the model even if the
generated scores are accurate (Chu et al.,, 2020; Ghassemi et al.,, 2021).
Thus, while feature attribution scores may help uncover and diagnose issues,
they cannot exclude their presence and should be used critically in
combination with other model evaluation and verification techniques.

Also, while CAFE aims to accurately identify conflicts and safeguard against
attribution score explosion, there are still some cases in which its scores
may not fully capture feature conflicts or may overestimate the effects of the
input features. See the answers to the below questions for details.

###  G.6 Are there any cases in which CAFE cannot fully capture conflicts
between features?

While CAFE aims to surface most of the conflicts between features, there are
some cases in which it is unable to do so, even when using the conflict
sensitivity constant of $1$. In particular, CAFE may not always capture
hypothetical effects of positive or negative features that are masked by the
non-monotonicity of the considered activation function. For example, when
considering a GELU neuron with the inputs $x_{1}=-2$ and $x_{2}=-5$, CAFE will
conclude that both features have no effect on the output, since
$\text{GELU}(-7)-\text{GELU}(0)=0$. This ignores the fact that the GELU
function decreases and then increases again on the interval, and that the
$x_{1}$ input could change the neuron output to $-0.05$ in the absence of
$x_{2}$. Effects cancelled in this way could theoretically be captured by
considering all activation function extrema within the intervals examined by
CAFE. The computed effects could then be added to both positive and negative
attribution scores so as to maintain completeness. Nevertheless, we do not
believe that doing so would make the attribution scores particularly more
insightful. Additionally, having to consider arbitrary activation function
extrema would make our method considerably more complicated.

###  G.7 Are there any cases in which CAFE overestimates the effects of the
input features?

Yes, despite the safeguards against the attribution score explosion problem,
CAFE can overestimate the possible effects of certain features. For example,
consider a model $\mathcal{M}(x)=\text{ReLU}(-\text{ReLU}(1-x))$. It is easy
to see that the output of this model will always be zero regardless of the
value of $x$, even if the bias is removed. Nevertheless, for an input $x=1$,
CAFE (1.0) computes attribution score of $+1$ for $x$ and $-1$ for the bias.
This is because the CAFE activation rule can only consider the local behaviour
at the individual neurons rather than the global behaviour of the model.

###  G.8 Are there any relations between CAFE and other feature attribution
methods?

Since CAFE (0.0) effectively computes average gradients of the non-linearities
(with additional adjustments safeguarding against attribution score
explosion), its combined positive and negative scores are equivalent to
Gradient $\cdot$ Input, LRP and DeepLIFT when computed with respect to a zero
reference input for a network with ReLU activations and no additive biases.
This is because the linear attribution flow clipping does not affect CAFE
attributions computed for ReLU networks, since the slope of the activation
function on its active range is constant.

###  G.9 The definitions in the paper only describe how to compute the
individual elements of the CAFE attribution scores. Is it possible to
vectorise these operations?

Yes, computations of the CAFE scores can be easily vectorised and batched with
the use of indicator vectors and matrices to switch between the different
cases from our definitions. Please see our code for an example of how this can
be done — it relatively closely follows the definitions from the main text.

## References

  * Agarwal et al., (2022) Agarwal, C., Krishna, S., Saxena, E., Pawelczyk, M., Johnson, N., Puri, I., Zitnik, M., and Lakkaraju, H. (2022).  OpenXAI: Towards a transparent evaluation of model explanations.  Advances in Neural Information Processing Systems, 35:15784–15799.
  * Ali et al., (2022) Ali, A., Schnake, T., Eberle, O., Montavon, G., Müller, K.-R., and Wolf, L. (2022).  XAI for transformers: Better explanations through conservative propagation.  In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., and Sabato, S., editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 435–451. PMLR.
  * Ancona et al., (2018) Ancona, M., Ceolini, E., Öztireli, C., and Gross, M. (2018).  Towards better understanding of gradient-based attribution methods for deep neural networks.  In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net.
  * Angwin et al., (2016) Angwin, J., Larson, J., Mattu, S., and Kirchner, L. (2016).  Machine Bias.
  * Ayoobi et al., (2023) Ayoobi, H., Potyka, N., and Toni, F. (2023).  SpArX: Sparse Argumentative Explanations for Neural Networks.  In In proceedings of the 26th European Conference on Artificial Intelligence, ECAI 2023, volume 372 of Frontiers in Artificial Intelligence and Applications, pages 149 – 156, Kraków, Poland. IOS Press.
  * Bach et al., (2015) Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., and Samek, W. (2015).  On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation.  PloS one, 10(7):e0130140.
  * Balduzzi et al., (2017) Balduzzi, D., Frean, M., Leary, L., Lewis, J., Ma, K. W.-D., and McWilliams, B. (2017).  The shattered gradients problem: If resnets are the answer, then what is the question?  In International Conference on Machine Learning, pages 342–350. PMLR.
  * Chen et al., (2022) Chen, Z., Subhash, V., Havasi, M., Pan, W., and Doshi-Velez, F. (2022).  What makes a good explanation?: A harmonized view of properties of explanations.  In Workshop on Trustworthy and Socially Responsible Machine Learning, NeurIPS 2022.
  * Chu et al., (2020) Chu, E., Roy, D., and Andreas, J. (2020).  Are visual explanations useful? a case study in model-in-the-loop prediction.  arXiv preprint arXiv:2007.12248.
  * Dhurandhar et al., (2018) Dhurandhar, A., Chen, P.-Y., Luss, R., Tu, C.-C., Ting, P., Shanmugam, K., and Das, P. (2018).  Explanations based on the missing: Towards contrastive explanations with pertinent negatives.  In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS’18, page 590–601, Red Hook, NY, USA. Curran Associates Inc.
  * FICO, (2022) FICO (2022).  Explainable Machine Learning Challenge.
  * Ghassemi et al., (2021) Ghassemi, M., Oakden-Rayner, L., and Beam, A. L. (2021).  The false hope of current approaches to explainable artificial intelligence in health care.  The Lancet Digital Health, 3(11):e745–e750.
  * Guidotti et al., (2018) Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., and Pedreschi, D. (2018).  A survey of methods for explaining black box models.  ACM Comput. Surv., 51(5).
  * Hofmann, (1994) Hofmann, H. (1994).  Statlog (German Credit Data).
  * Johnson et al., (2023) Johnson, A. E. W., Bulgarelli, L., Shen, L., Gayles, A., Shammout, A., Horng, S., Pollard, T. J., Moody, B., Gow, B., wei H. Lehman, L., Celi, L. A., and Mark, R. G. (2023).  MIMIC-IV, a freely accessible electronic health record dataset.  Scientific Data, 10.
  * Kokhlikyan et al., (2020) Kokhlikyan, N., Miglani, V., Martin, M., Wang, E., Alsallakh, B., Reynolds, J., Melnikov, A., Kliushkina, N., Araya, C., Yan, S., and Reblitz-Richardson, O. (2020).  Captum: A unified and generic model interpretability library for PyTorch.  CoRR, abs/2009.07896.
  * Le et al., (2023) Le, P. Q., Nauta, M., Nguyen, V. B., Pathak, S., Schlötterer, J., and Seifert, C. (2023).  Benchmarking eXplainable AI - A survey on available toolkits and open challenges.  In International Joint Conference on Artificial Intelligence.
  * Loshchilov and Hutter, (2019) Loshchilov, I. and Hutter, F. (2019).  Decoupled weight decay regularization.  In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net.
  * Lundberg and Lee, (2017) Lundberg, S. M. and Lee, S.-I. (2017).  A unified approach to interpreting model predictions.  In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 4768–4777, Red Hook, NY, USA. Curran Associates Inc.
  * Meng et al., (2022) Meng, C., Trinh, L., Xu, N., Enouen, J., and Liu, Y. (2022).  Interpretability and fairness evaluation of deep learning models on MIMIC-IV dataset.  Scientific Reports, 12.
  * Montavon et al., (2019) Montavon, G., Binder, A., Lapuschkin, S., Samek, W., and Müller, K.-R. (2019).  Layer-wise relevance propagation: an overview.  Explainable AI: interpreting, explaining and visualizing deep learning, pages 193–209.
  * Nauta et al., (2023) Nauta, M., Trienes, J., Pathak, S., Nguyen, E., Peters, M., Schmitt, Y., Schlötterer, J., van Keulen, M., and Seifert, C. (2023).  From anecdotal evidence to quantitative evaluation methods: A systematic review on evaluating explainable AI.  ACM Comput. Surv., 55(13s).
  * Paszke et al., (2019) Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Köpf, A., Yang, E. Z., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. (2019).  PyTorch: An imperative style, high-performance deep learning library.  In Wallach, H. M., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E. B., and Garnett, R., editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 8024–8035.
  * Pedregosa et al., (2011) Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011).  Scikit-learn: Machine learning in python.  J. Mach. Learn. Res., 12(null):2825–2830.
  * Pezeshkpour et al., (2021) Pezeshkpour, P., Jain, S., Singh, S., and Wallace, B. C. (2021).  Combining feature and instance attribution to detect artifacts.  CoRR, abs/2107.00323.
  * Rahnama, (2023) Rahnama, A. H. A. (2023).  The blame problem in evaluating local explanations, and how to tackle it.
  * Ribeiro et al., (2016) Ribeiro, M. T., Singh, S., and Guestrin, C. (2016).  “Why should I trust you?”: Explaining the predictions of any classifier.  Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
  * Shapley, (1953) Shapley, L. S. (1953).  17\. A Value for n-Person Games, pages 307–318.  Princeton University Press, Princeton.
  * Shrikumar et al., (2017) Shrikumar, A., Greenside, P., and Kundaje, A. (2017).  Learning important features through propagating activation differences.  In Precup, D. and Teh, Y. W., editors, Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pages 3145–3153. PMLR.
  * Shrikumar et al., (2016) Shrikumar, A., Greenside, P., Shcherbina, A., and Kundaje, A. (2016).  Not just a black box: Learning important features through propagating activation differences.  arXiv preprint arXiv:1605.01713.
  * Simonyan et al., (2013) Simonyan, K., Vedaldi, A., and Zisserman, A. (2013).  Deep inside convolutional networks: Visualising image classification models and saliency maps.  CoRR, abs/1312.6034.
  * Smilkov et al., (2017) Smilkov, D., Thorat, N., Kim, B., Viégas, F. B., and Wattenberg, M. (2017).  SmoothGrad: Removing noise by adding noise.  CoRR, abs/1706.03825.
  * Sokol and Flach, (2020) Sokol, K. and Flach, P. (2020).  Explainability fact sheets: a framework for systematic assessment of explainable approaches.  In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 56–67.
  * Strumbelj and Kononenko, (2010) Strumbelj, E. and Kononenko, I. (2010).  An efficient explanation of individual classifications using game theory.  J. Mach. Learn. Res., 11:1–18.
  * Sundararajan et al., (2017) Sundararajan, M., Taly, A., and Yan, Q. (2017).  Axiomatic attribution for deep networks.  In Precup, D. and Teh, Y. W., editors, Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pages 3319–3328. PMLR.
  * Wang and Nvasconcelos, (2019) Wang, P. and Nvasconcelos, N. (2019).  Deliberative explanations: visualizing network insecurities.  In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., and Garnett, R., editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.
  * Wang et al., (2019) Wang, S., Zhou, T., and Bilmes, J. (2019).  Bias also matters: Bias attribution for deep neural network explanation.  In Chaudhuri, K. and Salakhutdinov, R., editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 6659–6667. PMLR.
  * Yeh et al., (2019) Yeh, C., Hsieh, C., Suggala, A. S., Inouye, D. I., and Ravikumar, P. (2019).  On the (in)fidelity and sensitivity of explanations.  In Wallach, H. M., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E. B., and Garnett, R., editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 10965–10976.
  * Yeh and hui Lien, (2009) Yeh, I.-C. and hui Lien, C. (2009).  The comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clients.  Expert Systems with Applications, 36(2, Part 1):2473–2480.
  * Zhang et al., (2021) Zhang, Y., Tiňo, P., Leonardis, A., and Tang, K. (2021).  A survey on neural network interpretability.  IEEE Transactions on Emerging Topics in Computational Intelligence, 5(5):726–742.
  * Zhou et al., (2021) Zhou, J., Gandomi, A. H., Chen, F., and Holzinger, A. (2021).  Evaluating the quality of machine learning explanations: A survey on methods and metrics.  Electronics, 10(5).