# A Critical Assessment of Interpretable and Explainable Machine Learning for
Intrusion Detection

Omer Subasi, Johnathan Cree, Joseph Manzano, Elena Peterson  Pacific Northwest
National Laboratory, 902 Battelle Blvd, Richland, 99354, WA, USA
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Interpretable and explainable Machine Learning (ML) refers to methods and
models that explain how and why a model makes a prediction and describe a
model’s behaviour in human understandable terms. There has been a large number
of studies in interpretable and explainable ML for cybersecurity, in
particular, for intrusion detection. Many of these studies have significant
amount of overlapping and repeated evaluations and analysis. At the same time,
these studies overlook crucial model, data, learning process, and utility
related issues and many times completely disregard them. These issues include
the use of overly complex and opaque ML models, unaccounted data imbalances
and correlated features, inconsistent influential features across different
explanation methods, the inconsistencies stemming from the constituents of a
learning process, and the implausible utility of explanations.

Therefore, in this work, we empirically demonstrate the aforementioned issues,
analyze them and propose practical solutions in the context of feature-based
model explanations. Specifically, we advise avoiding complex opaque models
such as Deep Neural Networks (DNNs) and instead using interpretable ML models
such as Decision Trees (DTs) as the available intrusion datasets are not
difficult for such interpretable models to classify successfully. Then, we
bring attention to the binary classification metrics such as Matthews
Correlation Coefficient (MCC) which are well-suited for imbalanced datasets.
Moreover, we find that feature-based model explanations are most often
inconsistent across different settings. In this respect, to further gauge the
extent of inconsistencies, we introduce the notion of cross explanations which
corroborates that the features that are determined to be impactful by one
explanation method most often differ from those by another method.
Furthermore, we show that strongly correlated data features and the
constituents of a learning process, such as hyper-parameters and the
optimization routine, become yet another source of inconsistent explanations.
Finally, we discuss the utility of feature-based explanations.

In our empirical study, we use two real-world intrusion datasets, two
interpretable classification models, namely, linear Ridge and DT classifiers,
and a DNN-based classifier together with SHapley Additive exPlanations (SHAPs)
and Permutation feature Importances (PIs) external explanation methods.

Keywords: Machine Learning, Interpretable, Explainable, Intrusion Detection,
Cybersecurity, Artificial Intelligence, Deep Learning, XAI, Interpretability,
Explainability.

##  1 Introduction

Artificial Intelligence (AI) and Machine Learning (ML) have achieved
significant advances in the last decade. The breathtaking progress in Deep
Learning (DL) has unleashed powerful Deep Neural Networks (DNNs) solving
complex tasks in computer vision, natural language processing, speech
recognition, and many others [1]. Naturally, the adoption of these methods and
models in cybersecurity research has greatly expanded [2]. Researchers have
applied and evaluated various models and algorithms to predict, detect, and
mitigate cyber-attacks [2, 3].

With ever-widening adoption of DNNs, understanding and interpreting their
predictions has become a significant problem [4]. This is because DNNs are
notoriously hard to understand since they have an opaque, often complex,
internal structure. For this reason, they are also called “black-box” methods.
As DNNs grow in size, explaining how they make their predictions becomes
harder and nontrivial. Consequently, this very problem of explainability and
interpretability of ML/DL models and methods carries to cybersecurity research
as their adoption becomes widespread in cybersecurity community [5].

To interpret and explain the outputs of ML models, there has been a large
number of studies [5] and, in particular, for intrusion detection [6]. The
vast majority of the existing studies on interpretable and explainable
intrusion detection evaluates ML/DL-based detection methods with a widely-used
cyber-attack (intrusion) dataset and some feature-based explanation method.
These studies inadvertently consist of significantly overlapping repeated work
[6]. As a result, they commonly fail to consider and address certain ML-
related issues. After a careful review of the state of the art literature, we
see that existing studies often do not investigate the selection of
appropriate ML classification models by taking the complexity of existing
datasets into account. Similarly, they do not examine class imbalances or
strongly correlated features. We also see that although feature-based model
explanations are largely inconsistent, the instigating aleatoric (stochastic)
uncertainties are not scrutinized. Moreover, the literature has not explored
the impact of the constituents (elements) of a learning process, such as
hyper-parameters and optimization methods. Lastly, we see that existing
studies do not assess the utility of feature-based model explanations.

Therefore, in this work, we focus on these critical, however unattended,
problems in explainability and interpretability of ML-based intrusion
detection methods. First, we empirically show that low-cost interpretable
methods such as Decision Trees (DTs) are successful in the classification of
intrusions in the existing cyber-attack datasets. Second, our examination of
these datasets reveals a significantly high level of class imbalance which is
also noted by [7]. In consequence, we advocate for the adaption of balanced
accuracy (BA) and Matthews Correlation Coefficient (MCC) [8], which is
considered as one of the most reliable and accurate classification metrics [9,
10].

Furthermore, we introduce the notion of cross-explanations: Given a fixed
dataset, we first compute the most influential features that impact the
predictions of an ML-based intrusion/attack classifier as determined by an
external explanation method or the ML classifier itself. We then evaluate the
classification performance of another (type of) classifier by only using these
features. This is to see if these features are transferable. That is, whether
the features that are found to be impactful for one ML-based classifier are
similarly impactful on the performance of another classifier. As a result,
cross-explanations provide a new way of probing if the explanations, i.e., the
most impactful features, are consistent across different classifiers and
settings.

Finally, we analyze how feature correlations and the constituents of a
learning process alter the explanations. Our results indicate that strongly
correlated features and these constituents, e.g., hyperparameters and the
optimization routine, become sources of aleatoric uncertainty and thus cause
unstable and contradictory explanations.

Overall, the experimental evidence clearly suggests that a vast majority of
the feature-based explanations, regardless of being obtained intrinsically or
externally, has little value in terms of practical applicability due to being
inconsistent, unstable, and unreliable. Coupled with the inability to offer
any actionable course, we conclude that these explanations do not provide any
tangible utility in cybersecurity applications and practices. Thus, research
studies should prioritize the approaches that are not based on feature
importance such as prototype-based methods and counterfactual explanations.

To summarize our main contributions,

  * •

We conduct a thorough analysis of feature-based explainable and interpretable
ML for intrusion detection in which we investigate the unattended problems in
cybersecurity research.

  * •

We empirically show that DTs successfully classify the intrusions in the
existing intrusion datasets. This signifies that complex DNNs are not needed.

  * •

We bring attention to class imbalance that exists in many cybersecurity
datasets and advise the adaption of BA and MCC [8].

  * •

We mathematically analyze how a misalignment between a model and its feature-
based explanation can lead to incorrect conclusions.

  * •

We experimentally substantiate that there are pervasive inconsistencies among
the most important features affecting the predictions of an ML model due to
aleatoric uncertainty.

  * •

We introduce the method of cross-explanations as a novel way to assess the
reliability and consistency of explanations.

  * •

We study the effect of feature correlations, hyper-parameters values and the
type of an optimization routine on the consistency of the most impactful
features.

  * •

We use SHAP [11], Permutation feature Importance (PI) [12], and classifier-
intrinsic feature coefficients and importances with the newly available 5G
dataset [13] and the UDBLag dataset [14]. Throughout our study, we highlight
key results and offer specific guidelines.

In the next section, we provide the background and related work. In Section 3,
we detail the experimental setup. In Section 4, we present our results and in-
depth analysis. Finally, Section 5 concludes the study.

##  2 Background and Related Work

In this section, we first provide the background for explainable and
interpretable ML. Second, we summarize the state of the art external
explanation methods. Then, we explain the task of (binary) classification and
several metrics to evaluate the classification performance of an ML model.
Finally, we discuss the related work.

###  2.1 Interpretable and Explainable ML

Interpretable and explainable ML [4] is a research field whose main purpose is
to reason, interpret, and explain why and how ML models make the predictions
they make. It often attempts to quantify the impact of data features on a
model’s prediction and to correctly understand and interpret the model’s
outputs. Interpretable and explainable ML are crucial and often required in
certain domains and fields, such as healthcare, finance and cybersecurity.
While the notions of interpretability and explainability are often used
interchangeably, there is some difference between them. Interpretability
refers to the extent to which the inner workings of an ML method can be
understood [4]. It leads to understanding how a model’s parameters and the
input data features determine the model’s outputs. To put differently,
interpretability is the extent to which a cause and effect can be observed in
an ML system. Explainability, on the other hand, refers to the extent to which
the inner workings of an ML model can be explained in human terms.

Interpretable ML models can be understood on their own. These models do not
require external methods. Examples are linear models and Decision Trees (DTs)
[15]. DTs are highly interpretable since a DT’s prediction can be understood
by simply following the path from the root to the leaf node and logically
conjoining the node conditions. The Feature Importances (FIs) of Decision
Trees (DTs) are intrinsically computed by the reduction a feature makes in the
criterion, such as Gini Index and Entropy, that is used to select the splits
in a DT. FIs can be viewed as self-feedback of DTs and they provide another
means to understand and interpret DTs. That is, DTs provide two forms of
explanations: i) the tree itself - from which boolean logic rules are
extracted - and ii) the FIs. As for linear models, the feature coefficients
(FCs) can be used to understand which features are more influential than
others on a model’s output. FCs are categorically different than DTs’ FIs.
They are conditional in the sense that they quantify the weight of a feature
assuming that all other features are fixed. To be on the same footing with FIs
scores, FCs need to be corrected such that all features have the same unit.
Standardization is an approximate way of achieving a common unit. In our
analysis, we choose to study Ridge classifiers which are linear models with
regularization that is quadratic in FCs. Regularization can prevent over-
fitting and alleviate the problems arising due to strongly correlated
features.

Figure 1: Interpretability vs. flexibility of ML models. Note that the
relationship shown here is a qualitative overview of ML models.

Compared to interpretable ML models, explainable models are too complex for a
human to understand on their own. These models also called “black-box” and
“opaque” models. They require external explanation methods to be understood by
humans. DNNs are the prototypical examples of black-box models.

Considering the overall relationship between model interpretability and
flexibility, Figure 1 shows how the interpretability of ML models varies with
respect to the flexibility of those models in general. We see that linear and
tree-based models are more interpretable than ensemble models and DNNs. This
comes at the cost of relatively lesser flexibility and potentially lower
performance. However, our results in Section 4 show that for the intrusion
detection datasets we evaluated, DTs perform as successfully as DNNs.

###  2.2 External Explanation Methods

Existing external explanation methods and techniques can be classified as
local and global methods. Local methods explain the specific outputs of
specific inputs. That is, they focus on individual input-output pairs. Local
Interpretable Model-agnostic Explanations (LIME) [16] and Individual
Conditional Expectations [17] are well-known local methods. In particular,
LIME explains an ML model by assuming that the model is approximately and
locally linear around the specific input instance that is under consideration.
The explanations are then computed by the weights of the linear approximation.
Considering global explanation methods, SHapley Additive exPlanations (SHAP)
[11] is one of the most popular approaches that quantifies the contribution of
input features to the model’s output by approximating the performance
difference between including and not including the feature in the training.
Explanation methods can also be classified as model-agnostic and model-
specific methods. Model-agnostic methods consider any ML model as a black-box
model, that is, the internal workings of the model are not known or
understood. Examples are LIME and SHAP as both assume no knowledge of a
model’s internal working and they work based on input-output pairs.
Permutation feature Importance (PI) [12] is another (global) model-agnostic
method that quantifies the impact of a feature on a model’s output by
computing the change in the model’s performance while shuffling the feature’s
values. On the other hand, model-specific methods are tailored for specific ML
models. As an example, Montavon et. al. [18] propose a model-specific method
for DNNs that is built on the loss function’s gradients.

Some external explanation methods are prototype-based [19, 20, 21, 22]. These
methods aim to construct a set of prototypes which is representative of a
dataset, e.g., a set of images of the ten digits (0-9). Depending on the
context, a prototype can be an input data instance from a dataset, an
observation that is very close to a data instance or a combination of several
data instances. In classification tasks, the class of a data instance is
typically determined to be the class of the “closest” or “the most similar”
prototype where closeness or similarity is defined by a distance metric.

Other than the aforementioned methods, counterfactual explanations are the
sets of features that need to change to achieve a desired model prediction
[23, 24], while anchors are sufficient local conditions consisting of if-then
rules [25]. Closely related to all these feature-based explanations, feature
selection and dimensionality reduction algorithms [26] also provide the set of
the most “important” features. The definition of importance varies across
different contexts and methods.

External explanation methods unsurprisingly have limitations. For instance,
SHAPs are exponential costly in the number of features and therefore heuristic
algorithms are required. LIME, on the other hand, can be unstable and there is
no guarantee that the linearity assumption will hold in all circumstances. PIs
fail when there are strongly correlated features or the trained model itself
performs poorly.

###  2.3 Binary Classification and Related Metrics

Machine classification refers to the problem of categorizing a data instance
into the predefined categories. As a special case, binary classification
assumes the number of categories or classes is two, where the classes can 1
and 0, or true and false, or cats and dogs, or attack and no attack. Binary
classification is typically performed by supervised learning methods such as
linear classifiers, e.g., Logistic Regression and Ridge, nearest neighbors
methods, support vector machines (SVMs), Decision Trees (DTs), random forests,
and deep neural networks (DNNs). There are many performance metrics for binary
classification and they are all based on the standard 2x2 binary
classification confusion matrix as shown in Table 1. Table 2 defines the
metrics we use in our study.

Table 1: Confusion Matrix |  | Predicted
---|---|---
|  | Positive (P) | Negative (N)
Actual | Positive (P) | True Positive (TP) | False Negative (FN)
Negative (N) | False Positive (FP) | True Negative (TN)
Table 2: Definition of Classification Metrics Metric | Definition
---|---
Accuracy | $\frac{TP+TN}{TP+TN+FP+FN}$
Balanced Accuracy (BA) | $\frac{1}{2}(\frac{TP}{TP+FN}+\frac{FP}{FP+TN})$
F1 Score | $\frac{2TP}{2TP+FP+FN}$
Precision | $\frac{TP}{TP+FP}$
Recall | $\frac{TP}{TP+FN}$
Matthews Correlation Coefficient (MCC) | $\frac{TP\times TN-FP\times FN}{\sqrt{(TP+FP)\times(TP+FN)\times(FP+TN)\times(TN+FN))}}$

These metrics include standard metrics such as accuracy, precision, recall and
F1 score. In contrast to much of the existing literature in cybersecurity and
ML in general, we include the lesser known metrics of Balanced Accuracy (BA)
and Matthews Correlation Coefficient (MCC). Our inclusion of these two metrics
is due to the fact that many popular cyber-attack (intrusion) datasets are
highly imbalanced and the standard metrics are misleading in the presence of
data imbalance. BA is defined as the arithmetic mean of sensitivity and
specificity. It is the better-known metric for imbalanced data. However, when
the class imbalance is extreme, which is common in publicly available
intrusion datasets, BA may prove unreliable and misleading just as the
standard metrics, e.g., accuracy and F1-score [9]. In those cases metrics such
as MCC are needed. MCC is introduced by Matthews [8] and it is defined as

$MCC=\frac{TP\times TN-FP\times
FN}{\sqrt{(TP+FP)\times(TP+FN)\times(FP+TN)\times(TN+FN))}}.$

It ranges between -1 and +1, with -1 for perfect misclassification (TP = TN =
0) and 1 for perfect classification (FP = FN = 0). MCC = 0 indicates random
classification (TP $\times$ TN = FP $\times$ FN). It is undefined when a whole
row or column of the confusion matrix is zero. However, it can be naturally
extended for these cases: MCC := +1 when TP (or TN) is nonzero and all other
three entries are zero. MCC := -1 when FP (or FN) is nonzero and all other
three entries are zero. MCC := 0 for all the remaining cases where it is
undefined. MCC is considered as the most accurate and reliable metric by many
scientists, especially experts in bioinformatics, computational biology, brain
informatics, and biomedical sciences [9, 10]. As we have not seen much
adoption in cybersecurity, computer science, and general AI/ML, we strive to
make MCC more visible to these fields with our study.

###  2.4 Related Work

There has been a significant number of publications for interpretable and
explainable ML in cybersecurity in the last ten years [5, 6]. The ML models
that have been studied range from traditional ML (DTs, nearest neighbors
methods, SVMs, ensembles etc.) to DL (feed-forward, convolutional, and
recurrent DNNs, autoencoders etc.) [5, 6]. The most used explanation methods
are SHAPs [11], LIME [16], and PDPs [27]. The applications of these methods
include botnet detection [28], malware detection [29], and intrusion detection
[30, 31, 32]. In these applications, the most evaluated datasets include CIC-
IDS2017 [33], NSL-KDD [34], CSE-CIC-IDS2018 [35], CIC-DDoS2019 [14], TON_IoT
[36], NF-TON-IoT V2 [37] CIDDS-001 [38], CIDDS-002 [39] and UNSW-NB15 [40].

Some studies, such as [32, 41], consider both binary and multi-class attack
classification. However, in real-world settings, intrusion detection systems
are typically tasked to determine if there is an attack or not while being
online. They do not concern for determining the types of attacks since
intrusion monitoring is continuously performed in real-time [42, 43] and the
priority is to detect the presence (or lack) of an attack. As a result, we
focus on the binary classification of network traffic into attack and benign
classes in our study.

Overall, we see that many published studies overlap in terms of ML models,
explanation methods and datasets. There is a common conduct in which many
authors most often use SHAP and/or LIME to report feature scores for a trained
ML model on one of the datasets we mention above and then conclude their
study. The problems we state in the introduction remain unattended and thus
open. This is what makes our study valuable. We empirically demonstrate the
problems, study them, offer practical solutions, and highlight the key
research insights we gain.

##  3 Experimental Setup and Datasets

We choose two of the publicly available and real-world intrusion datasets for
our experiments. One dataset is based on 5G network traffic [13] and the other
is based on traditional Internet traffic [14]. While there are other publicly
available intrusion datasets as stated in Section 2.4 and Section 4.2, these
two datasets are sufficient for our analysis: For the analysis and the
experiments regarding DTs in Section 4.1, there exist published studies
conducting the same DTs experiments with the other datasets. We provide the
corresponding references within that section. Our analysis in Sections 4.2 and
4.4 \- 4.8 holds for the other datasets because the very underlying assumption
of our analysis is the existence of aleatoric uncertainty. Such uncertainty
exists (for every dataset) because of the probabilistic nature of data pre-
processing and model training stages, the rampant approximations made
throughout these two stages, and the approximations made during the generation
of feature-based explanations. Lastly, our analysis in Section 4.3 is a
theoretical one.

The first dataset is the real-world 5G flow traffic provided by Samarakoon et.
al. [41, 13]. This dataset is generated using a real 5G testbed. It contains
several types of denial-of-service (DoS) attacks, such as ICMP, UDP, SYN and
HTTP Floods and Slowrate DoS, and Port Scans such as SYN, TCP Connect and UDP
Scan. We use the Encoded.csv file of the dataset files that are published on
the IEEE dataport [13]. We take the Label column as the true labels/classes.
We set the benign entries to 0 and the malicious ones to 1. We then omit the
columns Unnamed:0, nan, Label, Attack Type, Attack Tool and use the remaining
features. The dataset consists of 1,215,890 flow entries. There are 477,737
benign flow entries out of 1,215,890, which is about 39.3% of the whole
dataset. There are 96 fields in a flow entry and we use 91 of them as features
for training. Further details on the dataset can be found in the authors’
paper [41] and on the IEEE dataport [13].

The second DoS dataset we use is the UDBLag part of the CIC-DDoS2019 dataset
[14]. The entire dataset, including the UDBLag part, is severely imbalanced in
favor of attack flow entries. The UDPLag dataset has only 4016 benign flows
out of a total of 674463 flows. That is, only about 0.6% of the flows is
benign and the rest is attack. This dataset has 88 fields and 77 of them are
used for training.

We perform Python-based evaluations where we test each case 10$\times$ and
report the mean scores. Evaluations are performed on an Apple M2 Max system
with MacOS Sonoma 14.4.1. The system has 12 CPUs, 38 GPUs, and 32 GB of
memory. The versions of Python and Scikit-Learn [44] that we use are 3.11.7
and 1.4.2, respectively. The versions of Tensorflow [45] and Keras [46] are
2.16.1 and 3.2.1, respectively. We also use the SHAP python package
(https://shap.readthedocs.io/en/latest/) whose version is 0.45.1.

##  4 Evaluation and Analysis

We now present our experimental evaluation and analysis. Our analysis is
eight-fold:

  1. 1.

evaluating the classification performance of ML models (Section 4.1),

  2. 2.

assessment of binary classification metrics (Section 4.2),

  3. 3.

understanding the alignment among models and explanations (Section 4.3),

  4. 4.

obtaining feature importances intrinsically and through external methods
(Section 4.4),

  5. 5.

introducing cross-explanations (Section 4.5),

  6. 6.

studying strong feature correlations (Section 4.6),

  7. 7.

exploring the impact of hyper-parameters and optimization methods
(Section 4.7), and

  8. 8.

discussion of the utility of the explanations (Section 4.8).

Table 3: Classification Scores of DTs, Ridge and DNNs for the 5G and UDBLag datasets Classifier | DT | Ridge | DNN
---|---|---|---
Metric | 5G | UDBLag | 5G | UDBLag | 5G | UDBLag
Accuracy | 0.9995920 | 0.9999604 | 0.9912623 | 0.9993575 | 0.9996107 | 0.9998121
BA | 0.9995873 | 0.9983037 | 0.9900997 | 0.9596470 | 0.9996554 | 0.9958653
F1 | 0.9997179 | 0.9999801 | 0.9900906 | 0.9995228 | 0.9996805 | 0.9999055
Precision | 0.9996094 | 0.9999801 | 0.9955576 | 0.9998309 | 0.9999099 | 0.9999502
Recall | 0.9996636 | 0.9999801 | 0.9928166 | 0.9996769 | 0.9994512 | 0.9998607
MCC | 0.9991453 | 0.9966074 | 0.9816909 | 0.9440313 | 0.9991824 | 0.9845968

###  4.1 Classification Performance

We now present the binary classification performances of Ridge Classifiers and
DTs that are implemented in Scikit-Learn library [44], and a Keras-based DNN
[46]. The DNN has two fully connected hidden layers, each with ten nodes and
Relu activation. The output layer is a node with Sigmoid activation. Binary
cross entropy is selected as the loss function. We consciously choose a DNN
that is as simple as possible. As we report next, this simple DNN performs
very well. In addition, we use this two-layered DNN architecture for all
relevant evaluations in our study. Except for the evaluations presented in
Subsection 4.7, we use the default Keras optimizer RMSprop and we set the
batch size to 256 and the number of the epochs to 5.

Table 3 shows the average classification scores of DTs, Ridge classifiers, and
DNNs. DTs and Ridge classifiers are trained with default values. We see that
for both 5G and UDBLag datasets, DTs perform extremely well and achieve scores
that are all $>=$ 0.996. In particular, for the 5G dataset, for all metrics,
DTs achieve scores that are $>=$ 0.999. We report the results in high
precision which is valuable in real-world deployments. As for Ridge
classifiers, while not as good as those of DTs, they achieve scores $>=$ 0.98
for the 5G dataset and $>=$ 0.944 for the UDBLag dataset. These scores show
that simple linear classifiers perform well for both datasets. As for DNNs, we
see that for the 5G dataset, for all metrics, DNN achieves scores that are
$>=$ 0.999 - the same with DTs. For the UDBLag dataset, all scores $>=$ 0.99,
except MCC which is $>=$ 0.985.

Many intrusion detection studies have found that DTs perform extremely well
with the available intrusion datasets. To name a few, the authors of the 5G
dataset Samarakoon et. al. [41] report that DTs achieve accuracy, precision,
recall, and F1 scores that are all $>=0.998$ (Table X (10) in [41]) for the
dataset. Mahbooba et. al. [47] state that DTs achieve perfect precision,
recall, and F1 scores (Figure 4 in [47]) for NSL-KDD [34]. Gaitan-Cardenas,
Abdelsalam, and Roy [48] report that DTs obtain accuracy, precision, recall,
and F1 scores that are all $>=0.999$ for the CIDDS-001 [38] and NF-TON-IoT V2
[37] datasets. They also report that DTs achieve perfect scores of 1 for all
metrics for the CIDDS-002 dataset [39] (Table II in [48]). For the UNSW-NB15,
the study [49] (Table 7) reports that DTs achieve accuracy, precision, recall,
and F1 scores that are all $>=0.99$. Neto et. al. [50] report that AdaBoost
based on DTs obtains perfect scores for accuracy, precision, recall, and F1
for the CICIoV2024 (Figure 6a with decimal encoding of the data). For the CIC-
IDS2018, Songma, Sathuphan and Pamutha [51] report that DTs’ accuracy,
precision, recall, F1, BA, and MCC are 0.99, 0.96, 0.97, 0.96, 0.97, and
0.999, respectively. They further provide experiments with improved DT
implementations having higher scores. Rosay et. al. [52] (Table 2) report DT
scores $>=0.99$ for accuracy, precision, recall, and MCC for the CIC-IDS2017.

Main Result 1: Therefore, in light of our results and many published studies,
we stress that the existing intrusion datasets do not require the usage of
complex high-cost black-box models such as DNNs since the problem of binary
attack classification for these datasets is completely solvable by DTs, which
are highly interpretable and low-cost ML models.

Guide 1: DTs, their variants (e.g., gradient boosted trees) and their
ensembles (e.g., random forests) must be the starter ML models for intrusion
detection.

###  4.2 Proper Binary Classification Metrics

The usage of proper binary classification metrics is a key component of
accurate and reliable evaluation of ML classifiers. To showcase this, we train
a DNN with the features obtained by SHAPs as in Figure 7(f) for the UDBLag
dataset. From the figure, we see that the top three features are ACK Flag
Count, URG Flag Count, and Min Packet Length. Table 4 shows the DNN’s
classification scores with these features. We see that even though all
standard metrics, i.e., accuracy, F1, precision, and recall, are $>=0.998$, BA
and MCC are 0.8967 and 0.8672, respectively. The standard metric scores are
very misleading. Only after computing BA and MCC, the true classification
performance is known. To see the true picture, we check the confusion matrix
which is $\begin{bmatrix}504&131\\\ 27&100508\end{bmatrix}$. From the matrix,
we see that the DNN’s performance is actually poor. For instance, the false
positive rate is $\frac{131}{504+131}=0.206$, which is unacceptable and
unsustainable for real-world intrusion detection systems. Such a system would
raise a false alarm once for every five predictions made on average.

Table 4: DNN classification scores with the features ACK Flag Count, URG Flag Count, and Min Packet Length for the UDBLag dataset. Metric | Score | Metric | Score
---|---|---|---
Accuracy | 0.9984382722150835 | Precision | 0.9986983177495802
BA | 0.8967161121073125 | Recall | 0.9997314368130502
F1 | 0.9992146102379035 | MCC | 0.8672112869253433
Table 5: Binary Class Distribution of Intrusion Detection Datasets Dataset | Total Entries | Attack Entries | Attack - Benign | Severity
---|---|---|---|---
5G [13] | 1215890 | 738153 | 61% - 39% | Mild
UDBLag [14] | 674463 | 670447 | 99% - 1% | Extreme
CIC-DDoS2019 [14] | 50063112 | 50006249 | 99.9% - 0.1% | Extreme
CIC-IDS2018 [35] | 10974408 | 1865649 | 17% - 83% | Moderate
CIC-IDS2017 [33] | 2810677 | 2359087 | 84% - 16% | Moderate
NSL-KDD [34] | 311027 | 250436 | 81% - 19% | Moderate
NF-TON-IoT-V2 [37] | 16940496 | 10841027 | 64% - 36% | Mild
CIDDS-001 [38] | 172838 | 107343 | 62% - 38% | Mild
CIDDS-002 [39] | 1048576 | 699051 | 67% - 33% | Mild
UNSW-NB15 [40] | 2540044 | 321283 | 13% - 87% | Severe
CICIoV2024 [53] | 1408219 | 184482 | 13% - 87% | Severe
CICEV2023 [54] | 63284 | 58000 | 92% - 8% | Severe

For the next step, we collect statistical information regarding the binary
class distribution for the publicly available intrusion detection datasets.
This is to see if there is any class imbalance in the datasets. Table 5 shows
the binary class details for twelve datasets. In particular, it shows the
proportions of attack and benign entries. We see that all datasets have binary
class imbalance. Among the datasets, two datasets have a majority-class
proportion that is $>=$ 99%, another six datasets have a majority-class
proportion that is $>$ 80% and $<$ 99%.

To characterize the datasets qualitatively, we refine the definition of the
degree of imbalance provided by [55]. If in a dataset, the majority class has
a proportion that is $>=$ 60% and $<$ 75%, then the degree of imbalance is
defined as mild. If the majority class’s proportion is $>=$ 75% and $<$ 85%,
then the degree of imbalance is said to be moderate. If the majority class’s
proportion is $>=$ 85% and $<$ 99%, then the degree of imbalance is called
severe. Finally, if it is $>=$ 99%, the degree of imbalance is called extreme.
According to this qualitative characterization, we see that out of twelve
datasets, eight datasets have a degree of imbalance that is at least moderate.
Two datasets have extreme imbalance while another three have severe imbalance.
Overall, we conclude that a majority of the existing intrusion detection
datasets have a significant level of binary class imbalance.

Main Result 2: Given that the existing intrusion datasets are typically highly
imbalanced, standard classification metrics might be very misleading and
overestimating the true performance. One particular reason for this is that
even when all the standard scores are bigger that 0.99, the true
classification performance might be much worse.

Guide 2: Matthews Correlation Coefficient (MCC) and balanced accuracy (BA) are
a must for reliable and accurate evaluation of binary classification
performance.

(a) $M_{step}=c_{1}(T>threshold)+c_{2}H\\\ \qquad
M_{step}=0.9\times(T>7)+0.1\times H$

(b) $M_{smooth}=\sigma(c_{1}(T-threshold)+c_{2}H)$
$M_{smooth}=\sigma(0.9\times(T-7)+0.1\times H)$

Figure 2: The plots of models $M_{step}$ and $M_{smooth}$ where $c_{1}=0.9$,
$c_{2}=0.1$, and $threshold=7$. The plot is inspired by Chapter 33 - Section
33.1.2 of [56].

###  4.3 Alignment Among Models and Explanations

Considering the state of the art in interpretable and explainable ML for
intrusion detection, there is a need for more detailed studies that are
conducted with both experimental and mathematical rigor. In this section, we
use a simplified example where we show that if we are not careful enough, the
accurate alignment between the specification of an explanation, the
computation of that explanation, and the target model may be compromised. The
model may become misaligned with the intended computation of the explanation.
Ultimately, this misalignment may lead to incorrect explanations.

As a starter, we assume that we use two different ML models for predicting the
probability of a cyber attack. Let these models be the following simple,
linear and interpretable mathematical models:

$M_{step}=c_{1}\times(T>threshold)+c_{2}\times H$
$M_{smooth}=\sigma(c_{1}\times(T-threshold)+c_{2}\times H)$

where the feature traffic density $T$ represents the average traffic density
per day. We assume $T$ be between 0 and 10. Moreover, the feature recent
history $H$ represents the number of weeks that have seen at least one attack
over the last ten weeks. That is, $0<=H=<1$ and $H=1$ means that there has
been at least one attack per each of the last ten weeks. $H=0.5$ corresponds
to the scenario where there was at least one attack in five different weeks
over the last ten weeks. We choose these models such that they are very
similar and inherently interpretable. The only differences between the models
are the usage of a sigmoid function $\sigma(x)=\frac{1}{1+e^{-x}}$ in
$M_{smooth}$ and the application of a threshold ($threshold$). Proceeding
further, let us assume that both models are trained and their trained
parameters turn out to be the same, where $c_{1}=0.9$, $c_{2}=0.1$, and
$threshold=7$. With these parameter values, Figure 2 visualizes the models and
the colorized gradients with respect to the features traffic density $T$ and
recent history $H$.

We want to identify the most important feature in predicting the probability
of an attack. If we define the importance of a feature as to what extent the
feature dominates other features in determining a model’s prediction, then
both $M_{step}$ and $M_{smooth}$ agree on the feature traffic density $T$ due
to its coefficient (0.9 vs 0.1). In contrast, if the importance is defined by
the magnitude of the derivatives (gradients) of the prediction with respect to
the inputs, that is, if it is based on local geometry and curvature, then for
$M_{smooth}$, the most important feature is still the traffic density $T$.
However, for $M_{step}$, the most important feature becomes the recent history
$H$. This is because the local gradient/curvature of $H$ is 0.1 everywhere,
while the local gradient of $T$ is zero almost everywhere, except where the
threshold is located. Without carefully analyzing how a change in the
definition of the feature importance affects a model explanation, we could
have incorrectly concluded that the most important feature for $M_{step}$
would remain the same just because it was very similar to $M_{smooth}$.

Guide 3: Even for the simplest mathematical models, rigorous analysis is
required when computing a model explanation. Conducting such analysis helps
prevent unaccounted disagreements between the model and its intended
computation of explanations.

###  4.4 Feature Importance Computations

In this section, we evaluate and discuss the Feature Coefficients (FCs) of
Ridge, Feature Importances (FIs) of DTs, and Permutation feature Importances
(PIs) and SHapley Additive exPlanations (SHAPs) for Ridge, DTs, and DNNs.
Figures 3(a), 3(b) and 3(c) show the top feature scores computed intrinsically
by DTs themselves, i.e, FIs, by PIs, and by SHAPs, respectively when DTs are
trained on the 5G dataset. Focusing only on the top three features, we see
that all three methods agree on the top two features, which are Seq and sTtl.
These two top features do not seem to change across different DTs or random
seeds in our evaluations.

(a) FIs for DT.

(b) PIs for DT.

(c) SHAPs for DT.

Figure 3: The top features based on the FIs, PIs, and SHAPs for DTs for the 5G
dataset.

(a) FIs for DTs with random seed 1234.

(b) FIs for DTs with random seed 5678.

(c) PIs for DT with random seed 1234.

(d) PIs for DT with random seed 6789.

(e) SHAPs for DT with random seed 1234.

(f) SHAPs for DT with random seed 5678.

Figure 4: The top features based on the FIs, PIs, and SHAPs for DTs for the
UDBLag dataset with two different random seeds.

Figure 4 shows the top features computed by FIs, by PIs, and by SHAPs when DTs
are trained on the UDBLag dataset. We include two exemplary results for each
type of scores obtained. This is because when different random seeds are used,
the output of each method differs. We see that the resulting top three
features differ for the same explanation method and across different
explanation methods. For instance, for the FIs, only the third top feature is
the same, i.e., URG Flag Count. Across all six results in Figure 4, the very
top feature is different than others. Moreover, in contrast to the 5G dataset,
the top two features are not the same across different settings.

Figures 5(a) and 5(b) show the resulting DTs for the 5G and the UDBLag
dataset. For ease of illustration, we just plot the top three levels in the
trees. We can clearly reason about how a DT classifies a flow instance (entry)
by following the path from the root to the leaf node and using boolean logic
along the path. That is, conjoining the node conditions along the path
constructs the exact boolean expression that leads to the classification.
Here, we include just two examples of the resulting DTs for brevity. As with
the most impactful features, the resulting DTs differ with different random
seeds.

(a) The resulting DT for the 5G dataset.

(b) The resulting DT for the UDBLag dataset.

Figure 5: The resulting DTs for the two datasets.

(a) FCs of Ridge with the 5G dataset.

(b) PIs for Ridge with the 5G dataset.

(c) FCs of Ridge with the UDBLag dataset with random seed 1234.

(d) FCs of Ridge with the UDBLag dataset with random seed 5678.

(e) PIs for Ridge with the UDBLag dataset with random seed 1234.

(f) PIs for Ridge with the UDBLag dataset with random seed 5678.

Figure 6: The FCs and PIs for Ridge classification with the 5G and the UDBLag
dataset with two different random seeds.

(a) SHAPs for 5G
dataset with seed 1234.

(b) SHAPs for 5G
dataset with seed 6789.

(c) PIs for 5G dataset
with seed 1234.

(d) PIs for 5G dataset
with seed 6789.

(e) SHAPs for UDBLag
with seed 1234.

(f) SHAPs for UDBLag
with seed 6789.

(g) PIs for UDBLag
with seed 1234.

(h) PIs for UDBLag
with seed 6789.

Figure 7: The top features based on the SHAPs and PIs for DNN.

Figure 6 shows the top features for Ridge classification for the 5G and the
UDBLag datasets. In particular, Figures 6(a) and 6(b) show the FCs and the PIs
for the 5G dataset. We see that, as with DTs, the top two features are Seq and
sTtl and they do not seem to change with different random seeds. As for the
UDBLag dataset, the results plotted in Figures 6(c), 6(d), 6(e), and 6(f) show
that the top features differ largely across different methods and random
seeds. As a note, since for linear models, SHAPs and FCs are equivalent, we
only present FCs.

Figures 7 show the top scoring features obtained by SHAPs and PIs for DNNs for
the 5G and the UDBLag datasets with two different seed values. Figures 7(a),
7(b), 7(c) and 7(d) show that for the 5G dataset, Seq is consistently the most
impactful feature affecting a DNN’s prediction. After Seq, dTtL and dHops are
the two most impactful features for both SHAPs and PIs. Moreover, considering
all top features of Ridge classifiers, DTs, and DNNs, Seq is the only common
top feature. For the UDBLag dataset, Figures 7(e), 7(f), 7(g) and 7(h) show
that the most impactful features differ substantially across different
training runs and external explanation methods.

Main Result 3: Overall, our results indicate that there is little to no
consistency among the sets of the most impactful features across different ML
models having intrinsic self-feedback, external explanation methods, and
training runs due to aleatoric (stochastic) uncertainty – further discussed in
Sections 4.6 and 4.7.

(a) Average accuracy and MCC of DTs with the top cross-features with the 5G
dataset.

(b) Average accuracy and MCC of DTs with the top cross-features with the
UDBLag dataset.

Figure 8: The pairs of the average accuracy and MCC of DTs with the top three
cross-features. To be able to visualize, we plot the top two features. For
each pair, the accuracy is higher and plotted with a blue circle and the MCC
is lower and plotted with a red square. Also, for each pair, the accuracy and
MCC are on the same line that is perpendicular to the feature plane.

###  4.5 Cross-Explanations

We now evaluate the impact of the top features that are obtained by different
external explanation methods and classifiers on the classification performance
of a separate independent DT which is then trained by only using those top
features. We conduct this analysis to see whether the top features determined
by different trained models and external methods can be successful in other
settings. That is, to probe if those top features are transferable. This way
of cross-evaluation of top features, which we name as cross-explanations,
provides a novel way of checking if the explanations, i.e. the most impactful
features, are consistent across different classifiers and settings.

To decide whether a feature set is transferable, we use a threshold of
$MCC>=0.95$. We note that if $MCC>=0.95$ for a 2x2 confusion matrix, then all
other binary classification metrics are guaranteed to be $>=0.95$. We say a
set of features is transferable if an ML model that is only trained with the
feature set achieves an $MCC>=0.95$. In our analysis, we limit the set of top
features to have three features so that we are able to determine the impact of
a feature set in a nuanced way.

Figure 8(a) shows the pairs of the average accuracy and MCC of DTs that are
trained with the top three features obtained by different external methods or
classifiers for the 5G dataset. Specifically, the five sets of top three
features in Figure 8(a) are obtained

  * •

by the highest three FCs of a Ridge classifier,

  * •

by the highest three SHAPs of a DNN,

  * •

by the highest three SHAPs of a separate and independently trained DT,

  * •

the highest three PIs of a separate and independently trained DT, and

  * •

the highest three FIs of a separate and independently trained DT.

We choose to plot only the standard accuracy metric and MCC for a clear
presentation. From Figure 8(a), we see that among the five sets, only one set
is transferable (the leftmost point) for the 5G dataset. Similarly, for the
UDBLag dataset, we train and evaluate DTs with the top three features
determined by different external methods and classifiers. As shown in Figure
8(b), we report eleven sets of top three features. These eleven sets of top
three features consist of:

  * •

two sets having the highest three FCs of two different Ridge classifiers,

  * •

two sets having the highest three SHAPs of two different DNNs,

  * •

two sets having the highest three PIs of two different DNNs,

  * •

two sets having the highest three SHAPs of two separate and independently
trained DTs,

  * •

two sets having the highest three PIs of two separate and independently
trained DTs, and

  * •

one set having the highest three FIs of two separate and independently trained
DTs.

Figure 8(b) shows that among the eleven sets of top three features, only four
are transferable. In Figure 8(b), to improve the visualization, we also plot
the projections of the MCCs of those four sets onto the right plane, where
three projections are black dots and one is an orange square. We use an orange
square to make the two very close MCCs visibly distinguishable.

Main Result 4: To summarize, given a fixed input dataset, the classification
performance of a set of top features that are determined to be influential by
external explanation methods or intrinsic self-feedback of ML models is not
necessarily achieved in other settings where the model and/or the feature set
(for training) is different. This, in turn, corroborates the absence of
consistent explanations across different settings.

(a) FIs for the 5G Dataset.

(b) SHAPs for the 5G Dataset.

(c) PIs for the 5G Dataset.

(d) FIs for the UDBLag
Dataset.

(e) SHAPs for the UDBLag
Dataset.

(f) PIs for the UDBLag
Dataset.

Figure 9: The top features based on the FIs, SHAPs, and PIs for DTs after the
correlated features omitted.

(a) SHAPs for the 5G
Dataset.

(b) PIs for the 5G
Dataset.

(c) SHAPs for the
UDBLag Dataset.

(d) PIs for the
UDBLag Dataset.

Figure 10: The top features based on the SHAPs and PIs for DNNs after the
correlated features omitted.

###  4.6 Feature Correlations

As we stated earlier, some external explanation methods, such as PIs and
SHAPs, assume the independence of data features. In the cases where the
features are not independent, that is, correlated, such methods might mislead.
There might be cases where correlation causes the computed importance/impact
of the correlated features to be lower and other features to be higher. This,
in effect, makes all computed importances unreliable. For instance, for PIs,
if two features are strongly correlated and one of the features is permuted,
an ML model still has access to the permuted feature through its correlated
feature. This may cause lower PIs for both features and at the same time may
falsely inflate PIs for other features. In other cases, an external method may
falsely show that all features are unimportant. However, there are actually
important/impactful features (see [57] for a specific example).

In our correlation study, we consider a feature correlation that is $>=0.95$
as a strong correlation. To find the features that have at least one strong
correlation with another feature, we compute the correlation matrix based on
Pearson’s correlation. Then, if a feature has a strong correlation with at
least one other feature, we omit the feature. For the 5G dataset, we find that
there are 15 features that are strongly correlated with at least one other
feature out of 91 features. For the UDBLag dataset, there are 30 such features
out of 77 features. As the next step, we omit strongly correlated features and
then compute SHAPs and PIs for DTs and DNNs which are trained with only the
remaining features.

We note that when some features are omitted, the intrinsic explanations, i.e.,
FIs of DTs and FCs of Ridge, change by definition. In addition, as noted
above, for linear models, SHAPs and FCs are equivalent, which means SHAPs for
Ridge will change by definition as well. As a result, we do not discuss Ridge
separately.

Figure 9 shows the results for DTs. Compared to the results with all features
in Figure 3 for the 5G dataset, we see that since the feature ECO is omitted,
the set of top impactful features does not include it as in 9(a), 9(b) and
9(c). For the UDBLag dataset, because Min Packet Length and Max Packet Length
are omitted, they are no longer among the top impactful features for DTs as in
9(d), 9(e) and 9(f) as opposed to the results with all features in Figure 4.
Additionally, since Fwd IAT Total and Fwd IAT Mean are omitted, they are no
longer among the top impactful features as opposed to the top features in
Figure 4(f). Figure 10 shows the correlation results for DNNs. We see that
there are no top features that are also strongly correlated for the 5G
dataset. For the UDBLag dataset, since the features Fwd IAT Mean and Min
Packet Length are omitted, they are no longer among the top impactful features
in contrast with the results with all features in Figures 7(e) and 7(f).

Main Result 5: When we omit strongly correlated features to prevent computing
potentially inaccurate and unreliable feature importances, we may end up with
new sets of the top impactful features that are inconsistent with the top
features when all features are considered. Consequently, feature correlation
might become yet another source of uncertainty exacerbating the problem of
inconsistent explanations.

###  4.7 Constituents of a Learning Process

A learning process for an ML model refers to all aspects of data pre-
processing steps, such as data cleaning, standardization, feature selection
[26], and dimensionality reduction, as well as the training phase where model
and hyper-parameter optimizations are performed. We have already explored the
interplay between some of these aspects and the computed explanations above.
For instance, as a feature selection step, we have identified strongly
correlated features and omitted them from the training data. In this section,
we further investigate the impact of other aspects of the learning process,
namely, hyper-parameters and the optimization routine utilized during
training. While we do not report the results for all aspects of a learning
process for brevity, our conclusions applies to them too.

####  4.7.1 Hyper-parameters:

(a) SHAPs with a batch size of 512.

(b) SHAPs with a 75%-25% train-test split.

Figure 11: Top features based on SHAPs of a DNN for the UDBLag Dataset.

(a) SHAPs with Adam optimizer instead of the default RMSprop.

(b) SHAPs with Adam optimizer, batch size of 512 and a 75%-25% train-test
split.

Figure 12: Top features based on SHAPs of a DNN for the UDBLag Dataset.

Most ML models and algorithms have hyper-parameters that specify and control
the learning process as opposed to the parameters of an ML model which are
optimized during training. Examples include train-test split ratio, batch
size, learning rate, the number of iterations, and the number of (hidden)
layers in a DNN. In this section, we run experiments to check if an
explanation for a model changes when hyper-parameter values change. Ideally,
value changes should not influence model explanations. However, our results
show that they indeed do affect the resulting explanations. In our evaluation,
we change the hyper-parameters of batch size and train-test split ratio for a
DNN while keeping other hyper-parameters the same as they are specified in
Subsection 4.1. We choose SHAPs as the explanation method for a DNN for the
UDBLag Dataset. Compared to the SHAPs in Figures 7(e), 7(f), and 10(c), Figure
11(a) shows that the top features are different when we use a batch size of
512 instead of 256 which is used in all previous evaluations of DNNs above.
This is also true for the train-test split ratio. Figure 11(b) shows the SHAPs
when 75%-25% train-test split ratio is used instead of 85%-15% in the earlier
evaluations. While we only present batch size and train-test split ratio to
demonstrate that when their values change, SHAP explanations change, this
holds true for other hyper-parameters, classifiers, and explanation methods.

####  4.7.2 Optimization Methods:

ML algorithms and methods generally employ different optimization techniques.
Moreover, these algorithms typically are amenable to different
techniques/routines. While different optimization routines may have different
convergence speed and characteristics, they preferably should not affect the
final model and its performance. To check this, we perform experiments to
observe the effect of employing different optimizers on model explanations.
Figure 12(a) shows the resulting SHAPs when the default Keras optimizer
RMSprop is replaced with the Adam optimizer to train a DNN for the UDBLag
Dataset. When compared with Figures 7(e), 7(f), and 10(c), and 11, we see that
the top impactful features do not match, that is, the explanations are
different and inconsistent. As a further evaluation step, we conduct
experiments with the Adam optimizer, a batch size of 512, and a 75%-25% train-
test split to see how the changes in hyper-parameters and in the optimization
routine compound. Figure 12(b) shows the results. Compared with Figure 12(a),
Figure 12(b) shows that the Init_Win_bytes_forward is the only shared top
feature between the two explanations. This suggests that combining multiple
changes could severely degrade that the stability, the consistency, and the
reliability of explanations. Ideally, such changes in hyper-parameters and the
optimization technique should not drastically impact model explanations.

Main Result 6: In all experiments reported in this subsection, the
classification performances do not differ from each other for more than 0.1%
for the standard classification metrics and 2% for MCC when different hyper-
parameter values or optimization routines are tested. These limited
performance variations are critical: We see that while the classification
performances essentially remain the same, the explanations do not. As a
result, this observation discredits the merit of feature-based model
explanations.

###  4.8 Lack of Actionability and Utility

One of the ultimate goals of interpretable and explainable ML is to be able to
identify and provide necessary actions if there are issues regarding the model
under consideration. However, it is not clear that what actions are needed to
be taken if the explanation is a list of the most important/impactful features
based on some definition of importance. How does it help to know that the top
most important features are, say, Idle Min, Packet Length Std,
Init_Win_bytes_forward, Fwd IAT Mean and Avg Fwd Segment Size? A list of most
important features can not help much in deciding what to do next after a cyber
attack. The situation worsens when a DNN or any other black-box model is
employed to detect intrusions. This is because they do not offer any feedback
other than the feature importances provided by an external explanation method.
In comparison, DTs intrinsically provide two types of feedback for their
predictions. These are FIs and the boolean logic rules obtained by traversing
from the root to the output leaf node and conjoining the conditions along the
path. The logic rules also serve as a mechanism that reveals how the features
interact with each other. However, having said that, the rampant stochastic
inconsistencies in these feature-based explanations nullify the value of DTs’
intrinsic feedback. In consequence, current interpretable and explainable ML
methods are not able to recommend sound and effective after-the-fact actions
in cybersecurity and more specifically in intrusion detection.

Main Result 7: Considering the stochastic inconsistencies arising from random
variations and the changes in the constituents of a learning process, and
their inability to provide actionable feedback, the state of the art feature-
based explanations - obtained intrinsically or externally - exhibit no
tangible practical value for cybersecurity experts.

Guide 4: In conclusion, we advise that research studies on interpretable and
explainable ML for intrusion detection should focus on the approaches that are
not based on feature impact or importance. Potential approaches include
prototype-based models and counterfactual explanations.

##  5 Conclusion and Future Work

In this work, we present our assessment of the state of the art feature-based
interpretable and explainable ML for intrusion detection. In our study, we
highlight the concerning issues in ML-based intrusion detection research. One
such issue is the unjustified use of complex opaque DNNs while interpretable
ML models, such as Ridge and DT classifiers, are powerful enough for
successful intrusion detection. Another key issue is the use of improper
classification metrics. We empirically demonstrate that if proper metrics are
not used, the standard metrics, such as accuracy, precision, and F1-score, may
overestimate the true performance and thus mislead the researchers. Moreover,
we show that stochastic uncertainties lead to inconsistent and unreliable
feature-based explanations. In addition, we find that strong feature
correlations may exacerbate this inconsistency problem. Furthermore, we
introduce the novel method of cross-explanations to better gauge if
explanations are consistent and transferable. Applications of our method
corroborate that consistent and transferable explanations are absent for the
datasets, the ML classifiers, and the external explanation methods we study.
Finally, we find that hyper-parameters and optimization methods also cause
unstable and inconsistent explanations. Consequently, coupled with the lack of
actionable feedback, feature-based explanations and the constructing methods
do not offer convincing utility. Therefore, we advise that research should be
focused on the approaches that are not based on the data features.

As future work, we plan to expand our study into more general ML settings that
are beyond intrusion detection and cybersecurity.

## Acknowledgments

This work was supported by the U.S. DOE Office of Science, Office of Advanced
Scientific Computing Research, under award 66150: ”CENATE - Center for
Advanced Architecture Evaluation” project. The Pacific Northwest National
Laboratory is operated by Battelle for the U.S. Department of Energy under
contract DE-AC05-76RL01830.

## References

## References

  * [1] Ian Goodfellow, Yoshua Bengio, and Aaron Courville.  Deep Learning.  MIT Press, 2016.  http://www.deeplearningbook.org.
  * [2] Ramanpreet Kaur, Dušan Gabrijelčič, and Tomaž Klobučar.  Artificial intelligence for cybersecurity: Literature review and future research directions.  Information Fusion, 97:101804, 2023.
  * [3] Mayra Macas, Chunming Wu, and Walter Fuertes.  A survey on deep learning for cybersecurity: Progress, challenges, and opportunities.  Computer Networks, 212:109032, 2022.
  * [4] Christoph Molnar.  Interpretable machine learning.  Lulu. com, 2020.
  * [5] Gaith Rjoub, Jamal Bentahar, Omar Abdel Wahab, Rabeb Mizouni, Alyssa Song, Robin Cohen, Hadi Otrok, and Azzam Mourad.  A survey on explainable artificial intelligence for cybersecurity.  IEEE Transactions on Network and Service Management, 20(4):5115–5140, 2023.
  * [6] Nour Moustafa, Nickolaos Koroniotis, Marwa Keshk, Albert Y Zomaya, and Zahir Tari.  Explainable intrusion detection for cyber defences in the internet of things: Opportunities and solutions.  IEEE Communications Surveys & Tutorials, 2023.
  * [7] Omer Subasi, Joseph Manzano, and Kevin Barker.  Denial of service attack detection via differential analysis of generalized entropy progressions.  In 2023 IEEE International Conference on Cyber Security and Resilience (CSR), pages 219–226, 2023.
  * [8] B.W. Matthews.  Comparison of the predicted and observed secondary structure of t4 phage lysozyme.  Biochimica et Biophysica Acta (BBA) - Protein Structure, 405(2):442–451, 1975.
  * [9] Davide Chicco, Niklas Tötsch, and Giuseppe Jurman.  The matthews correlation coefficient (MCC) is more reliable than balanced accuracy, bookmaker informedness, and markedness in two-class confusion matrix evaluation.  BioData Min., 14(1):13, February 2021.
  * [10] Davide Chicco and Giuseppe Jurman.  The matthews correlation coefficient (MCC) should replace the ROC AUC as the standard metric for assessing binary classification.  BioData Min., 16(1):4, February 2023.
  * [11] Scott M Lundberg and Su-In Lee.  A unified approach to interpreting model predictions.  In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 4765–4774. Curran Associates, Inc., 2017.
  * [12] Leo Breiman.  Random forests.  Machine learning, 45:5–32, 2001.
  * [13] Sehan Samarakoon, Yushan Siriwardhana, Pawani Porambage, Madhusanka Liyanage, Sang-Yoon Chang, Jinoh Kim, Jonghyun Kim, and Mika Ylianttila.  5g-nidd: A comprehensive network intrusion detection dataset generated over 5g wireless network.  IEEE Dataport, 2022.
  * [14] Iman Sharafaldin, Arash Habibi Lashkari, Saqib Hakak, and Ali A. Ghorbani.  Developing realistic distributed denial of service (ddos) attack dataset and taxonomy.  In 2019 International Carnahan Conference on Security Technology (ICCST), pages 1–8, 2019.
  * [15] Leo Breiman.  Classification and regression trees.  Routledge, 2017.
  * [16] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin.  ”why should i trust you?”: Explaining the predictions of any classifier.  In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, page 1135–1144, New York, NY, USA, 2016. Association for Computing Machinery.
  * [17] Alex Goldstein, Adam Kapelner, Justin Bleich, and Emil Pitkin.  Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation.  journal of Computational and Graphical Statistics, 24(1):44–65, 2015.
  * [18] Grégoire Montavon, Alexander Binder, Sebastian Lapuschkin, Wojciech Samek, and Klaus-Robert Müller.  Layer-wise relevance propagation: an overview.  Explainable AI: interpreting, explaining and visualizing deep learning, pages 193–209, 2019.
  * [19] Jacob Bien and Robert Tibshirani.  Prototype selection for interpretable classification.  The Annals of Applied Statistics, 5(4):2403 – 2424, 2011.
  * [20] Been Kim, Cynthia Rudin, and Julie Shah.  The bayesian case model: a generative approach for case-based reasoning and prototype classification.  In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, NIPS’14, page 1952–1960, Cambridge, MA, USA, 2014. MIT Press.
  * [21] Oscar Li, Hao Liu, Chaofan Chen, and Cynthia Rudin.  Deep learning for case-based reasoning through prototypes: a neural network that explains its predictions.  In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence. AAAI Press, 2018.
  * [22] Meike Nauta, Ron Van Bree, and Christin Seifert.  Neural prototype trees for interpretable fine-grained image recognition.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14933–14943, 2021.
  * [23] Amir-Hossein Karimi, Gilles Barthe, Borja Balle, and Isabel Valera.  Model-agnostic counterfactual explanations for consequential decisions.  In International conference on artificial intelligence and statistics, pages 895–905. PMLR, 2020.
  * [24] Sandra Wachter, Brent Mittelstadt, and Chris Russell.  Counterfactual explanations without opening the black box: Automated decisions and the gdpr.  Harv. JL & Tech., 31:841, 2017.
  * [25] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin.  Anchors: High-precision model-agnostic explanations.  Proceedings of the AAAI Conference on Artificial Intelligence, 32(1), Apr. 2018.
  * [26] Omer Subasi, Sayan Ghosh, Joseph Manzano, Bruce Palmer, and Andrés Marquez.  Analysis and benchmarking of feature reduction for classification under computational constraints.  Machine Learning: Science and Technology, 5(2):020501, 2024.
  * [27] Jerome H Friedman.  Greedy function approximation: a gradient boosting machine.  Annals of statistics, pages 1189–1232, 2001.
  * [28] Hatma Suryotrisongko, Yasuo Musashi, Akio Tsuneda, and Kenichi Sugitani.  Robust botnet dga detection: Blending xai and osint for cyber threat intelligence sharing.  IEEE Access, 10:34613–34624, 2022.
  * [29] Boryau Hsupeng, Kun-Wei Lee, Te-En Wei, and Shih-Hao Wang.  Explainable malware detection using predefined network flow.  In 2022 24th International Conference on Advanced Communication Technology (ICACT), pages 27–33, 2022.
  * [30] Shraddha Mane and Dattaraj Rao.  Explaining network intrusion detection system using explainable ai framework.  arXiv preprint arXiv:2103.07110, 2021.
  * [31] Bhawana Sharma, Lokesh Sharma, Chhagan Lal, and Satyabrata Roy.  Explainable artificial intelligence for intrusion detection in iot networks: A deep learning based approach.  Expert Systems with Applications, 238:121751, 2024.
  * [32] Marwa Keshk, Nickolaos Koroniotis, Nam Pham, Nour Moustafa, Benjamin Turnbull, and Albert Y. Zomaya.  An explainable deep learning-enabled intrusion detection framework in iot networks.  Information Sciences, 639:119000, 2023.
  * [33] Iman Sharafaldin, Arash Habibi Lashkari, Ali A Ghorbani, et al.  Toward generating a new intrusion detection dataset and intrusion traffic characterization.  ICISSp, 1:108–116, 2018.
  * [34] Mahbod Tavallaee, Ebrahim Bagheri, Wei Lu, and Ali A. Ghorbani.  A detailed analysis of the kdd cup 99 data set.  In 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications, pages 1–6, 2009.
  * [35] Iman Sharafaldin, Arash Habibi Lashkari, and Ali A. Ghorbani.  Toward generating a new intrusion detection dataset and intrusion traffic characterization.  In International Conference on Information Systems Security and Privacy, 2018.
  * [36] Abdullah Alsaedi, Nour Moustafa, Zahir Tari, Abdun Mahmood, and Adnan Anwar.  Ton_iot telemetry dataset: A new generation dataset of iot and iiot for data-driven intrusion detection systems.  IEEE Access, 8:165130–165150, 2020.
  * [37] Mohanad Sarhan, Siamak Layeghy, and Marius Portmann.  Towards a standard feature set for network intrusion detection system datasets.  Mobile networks and applications, pages 1–14, 2022.
  * [38] Markus Ring, Sarah Wunderlich, Dominik Grüdl, Dieter Landes, and Andreas Hotho.  Flow-based benchmark data sets for intrusion detection.  In Proceedings of the 16th European Conference on Cyber Warfare and Security (ECCWS), pages 361–369. ACPI South Oxfordshire, UK, 2017.
  * [39] Markus Ring, Sarah Wunderlich, Dominik Grüdl, Dieter Landes, and Andreas Hotho.  Creation of flow-based data sets for intrusion detection.  Journal of Information Warfare, 16(4):41–54, 2017.
  * [40] Nour Moustafa and Jill Slay.  Unsw-nb15: a comprehensive data set for network intrusion detection systems (unsw-nb15 network data set).  In 2015 Military Communications and Information Systems Conference (MilCIS), pages 1–6, 2015.
  * [41] Sehan Samarakoon, Yushan Siriwardhana, Pawani Porambage, Madhusanka Liyanage, Sang-Yoon Chang, Jinoh Kim, Jonghyun Kim, and Mika Ylianttila.  5g-nidd: A comprehensive network intrusion detection dataset generated over 5g wireless network.  arXiv preprint arXiv:2212.01298, 2022.
  * [42] Camil Jichici, Bogdan Groza, Radu Ragobete, Pal-Stefan Murvay, and Tudor Andreica.  Effective intrusion detection and prevention for the commercial vehicle sae j1939 can bus.  IEEE Transactions on Intelligent Transportation Systems, 23(10):17425–17439, 2022.
  * [43] Aanshi Bhardwaj, Veenu Mangat, Renu Vig, Subir Halder, and Mauro Conti.  Distributed denial of service attacks in cloud: State-of-the-art of scientific and commercial solutions.  Computer Science Review, 39:100332, 2021.
  * [44] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay.  Scikit-learn: Machine learning in Python.  Journal of Machine Learning Research, 12:2825–2830, 2011.
  * [45] Martin Abadi, Ashish Agarwal, and Paul Barham et. al.  TensorFlow: Large-scale machine learning on heterogeneous systems.  2015\.  https://www.tensorflow.org/.
  * [46] François Chollet et al.  Keras.  2015\.  https://keras.io.
  * [47] Basim Mahbooba, Mohan Timilsina, Radhya Sahal, and Martin Serrano.  Explainable artificial intelligence (XAI) to enhance trust management in intrusion detection systems using decision tree model.  Complexity, 2021:6634811, January 2021.
  * [48] Maria Camila Gaitan-Cardenas, Mahmoud Abdelsalam, and Kaushik Roy.  Explainable ai-based intrusion detection systems for cloud and iot.  In 2023 32nd International Conference on Computer Communications and Networks (ICCCN), pages 1–7, 2023.
  * [49] Shweta More, Moad Idrissi, Haitham Mahmoud, and A. Taufiq Asyhari.  Enhanced intrusion detection systems performance with unsw-nb15 data analysis.  Algorithms, 17(2), 2024.
  * [50] Euclides Carlos Pinto Neto, Hamideh Taslimasa, Sajjad Dadkhah, Shahrear Iqbal, Pulei Xiong, Taufiq Rahman, and Ali A. Ghorbani.  Ciciov2024: Advancing realistic ids approaches against dos and spoofing attack in iov can bus.  Internet of Things, 26:101209, 2024.
  * [51] Surasit Songma, Theera Sathuphan, and Thanakorn Pamutha.  Optimizing intrusion detection systems in three phases on the cse-cic-ids-2018 dataset.  Computers, 12(12), 2023.
  * [52] Arnaud ROSAY, Florent CARLIER, Eloïse CHEVAL, and Pascal LEROUX.  From cic-ids2017 to lycos-ids2017: A corrected dataset for better performance.  In IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT ’21, page 570–575, New York, NY, USA, 2022. Association for Computing Machinery.
  * [53] Euclides Carlos Pinto Neto, Hamideh Taslimasa, Sajjad Dadkhah, Shahrear Iqbal, Pulei Xiong, Taufiq Rahman, and Ali A. Ghorbani.  Ciciov2024: Advancing realistic ids approaches against dos and spoofing attack in iov can bus.  Internet of Things, 26:101209, 2024.
  * [54] Yoonjib Kim, Saqib Hakak, and Ali Ghorbani.  Ddos attack dataset (cicev2023) against ev authentication in charging infrastructure.  In 2023 20th Annual International Conference on Privacy, Security and Trust (PST), pages 1–9, 2023.
  * [55] Google Developers.  Imbalanced data, Accessed on 06-17-2024.  https://developers.google.com/machine-learning/data-prep/construct/sampling-splitting/imbalanced-data.
  * [56] Kevin P. Murphy.  Probabilistic Machine Learning: Advanced Topics.  MIT Press, 2023.
  * [57] ScikitLearn Developers.  Permutation importance with multicollinear or correlated features.  Accessed on 05-17-2024.  https://scikit-learn.org/stable/auto$\\_$examples/inspection/ plot$\\_$permutation$\\_$importance$\\_$multicollinear.html#sphx-glr-auto-examples-inspection-plot-permutation-importance-multicollinear-py.