# Open or not open: Are conventional radio access networks more secure and
trustworthy than Open RAN?

Felix Klement  Computer Engineering, University of Passau, 94032 Passau,
Germany  Stefan Katzenbeisser  Computer Engineering, University of Passau,
94032 Passau, Germany  Vincent Ulitzsch  Security in Telecommunications, TU
Berlin, 10587 Berlin, Germany  Juliane Krämer  Data Security and Cryptography,
University of Regensburg, 93053 Regensburg, Germany
Slawomir Stanczak  Fraunhofer Heinrich Hertz Institute, 10587 Berlin, Germany
Zoran Utkovski  Fraunhofer Heinrich Hertz Institute, 10587 Berlin, Germany
Igor Bjelakovic  Fraunhofer Heinrich Hertz Institute, 10587 Berlin, Germany
Gerhard Wunder  Cybersecurity and AI Group, FU Berlin, 14195 Berlin, Germany

###### Abstract

The Open RAN architecture is a promising and future-oriented architecture. It
is intended to open up the radio access network (RAN) and enable more
innovation and competition in the market. This will lead to RANs for current
5G networks, but especially for future 6G networks, evolving from the current
highly integrated, vendor-specific RAN architecture towards disaggregated
architectures with open interfaces that will enable to better tailor RAN
solutions to the requirements of 5G and 6G applications. However, the
introduction of such an open architecture substantially broadens the attack
possibilities when compared to conventional RANs. In the past, this has often
led to negative headlines that in summary have associated Open RAN with faulty
or inadequate security. In this paper, we analyze what components are involved
in an Open RAN deployment, how to assess the current state of security, and
what measures need to be taken to ensure secure operation.

###### Index Terms:

Security, Open RAN, O-RAN, OpenRAN, 5G, 6G,

##  I Introduction to Open-RAN

###  I-A Motivation

Modern communication is one of the central pillars of successful digitization.
Particularly instrumental is the recently introduced 5G technology and its
ongoing evolution towards 6G. In addition to public mobile networks, 5G
technology - and in the long term 6G technology - will also be used for local
radio networks (so-called private networks or campus networks).

A 5G mobile network (whether public or private) typically consists of a
transport network (e.g., fiber optic network), a core network with central
elements for network control, and the radio access network (RAN) that provides
connections to mobile terminals. A schematic example of such a 5G mobile radio
network can be seen in Figure 1. While there is a plethora of vendors for
virtualized core networks, radio access networks are provided by only a
handful of major network equipment vendors. Today’s RANs are in addition
highly integrated solutions from individual vendors, with little
interoperability between products from different vendors. This inevitably
leads to innovation barriers.

A key to more innovation in mobile networks lies in the Open RAN approach,
which uses RAN technologies based on disaggregation and openness. In the Open
RAN approach, the RAN is divided into several RAN units, each of which
performs different RAN functions. The crucial point here is that the
interfaces between the RAN units are open and guarantee interoperability. The
open interfaces are therefore the basis for more flexibility and the much
needed trust in communication technologies. Finally, Open RAN promises
performance enhancements over the current integrated vendor-specific
solutions.

Fig. 1: Mobile Network

In addition to the architectural disaggregation and openness (in the sense of
interoperability), the aspects of cloudification and virtualisation [1],
network slicing [2], [3] and machine learning [4] also play an important role
in the Open RAN context. Yet, it is important to emphasize that except for
disaggregation and openness, the other aspects such as virtualization and
machine learning are not an inherent part of the Open RAN concept. This means
that Open RAN systems basically do not need to be virtualized, which can be
beneficial in some cases. For instance, in Massive MIMO systems, it may be
beneficial in terms of energy-efficiency to implement the lower layer RAN
functions on a dedicated system-on-chip (SoC) rather than running these
functions on general-purpose processors. Therefore, Open RAN is not equivalent
to virtualized RAN (vRAN), even though many Open RAN systems are highly
virtualized systems. However, this is also true for highly integrated and
closed RAN systems.

For a better understanding of this paper, it is also helpful to distinguish
between Open RAN, O-RAN, and OpenRAN (one word), as these terms are often
confused or used interchangeably. The acronym O-RAN originates from the O-RAN
Alliance, which focuses largely on the development of the O-RAN architecture.
This architecture forms the basis for the analysis in this paper. OpenRAN (one
word), on the other hand, is a project group established by the Telecom Infra
Project (TIP). This term plays no role in this paper. Finally, Open RAN is
used as a generic term for disaggregated systems with open and interoperable
interfaces. The O-RAN architecture is one possible Open RAN architecture.
Especially in in the context campus networks, there might be different flavors
of the Open RAN architecture.

###  I-B Architectural Overview

A traditional 3GPP-specified NG-RAN (Next Generation NodesBs Radio Access
Network) is divided into two logical RAN units: The CU (Central Unit) and the
DU (Distributed Unit). These basic units in turn comprise several logical
functional units. All of them together are then connected to the core network.
In the Open-RAN, the above two RAN functions are further divided according to
the 3GPP definition. Figure 2 shows a simplified version outlining the
breakdown of RAN functions and interfaces for a 3GPP compliant O-RAN
architecture [5].
New RAN functions specifically defined in the context of O-RAN are:

  1. 1.

Service Management and Orchestration (SMO) Framework

  2. 2.

RAN Intelligent Controllers (RICs) in the variants non real time (Non-RT RIC)
and near real time (Near-RT RIC)

  3. 3.

Remote Unit (O-RU)

  4. 4.

O-Cloud

We briefly explain the newly introduced interfaces within O-RAN in Section II.

Fig. 2: O-RAN specific interfaces [5]

###  I-C Security Advantages

While most critics of Open-RAN architectures often criticise the general
security or describe it as poor, we rather see the great opportunity to
effectively apply security solutions across the board. It is of extreme
importance in all projects to include security assessment at an early stage.
As we are still in the initial design phase for O-RAN, we can now develop and
incorporate security concepts in a timely manner. The concept of security by
design is based on the fact that the impact of iterating an architecture in
its early stages towards a secure design is many times more effective and
leads to more stable outcomes than conducting a security analysis in the later
stages of a technology’s lifecycle [6]. A risk analysis [7] has already been
commissioned by the German Federal Office for Information Security (BSI). We
have used this analysis as the basis for our further research and built on it.
The BSI study is an important contribution because it highlights the risks
associated with the O-RAN architecture and has triggered an important
discussion about security in 5G networks. In this paper, we would like to
focus more on the opportunities of the Open RAN approach and raise other
important issues such as post-quantum security.

##  II Important Interfaces

In this section we briefly mention the most important interfaces within an
O-RAN deployment and summarise them. The descriptions of the individual
interfaces are taken from the the official O-RAN specifications.

###  II-A O-Cloud

The O-Cloud is a cloud-based computing platform that includes a collection of
physical infrastructure nodes and hosts the components of relevant O-RAN
functions (e.g., Near-RT RIC, O-CU-CP, O-CU-UP and O-DU). Beyond that, it also
provides support for software components as well as the corresponding
management and orchestration functions. Generally speaking it can be seen as
the central execution environment for O-RAN components [8].

###  II-B O1-Interface

This interface grants access to network capabilities to the service management
and orchestration framework. Here, network management is implemented according
to the FCAPS model [9]. FCAPS follows to the ISO model for telecommunications
network management, which defines and incorporates the fault, configuration,
accounting, performance, and security management task areas [10], [11].

###  II-C O2-Interface

This interface plays a central role in the O-RAN environment. It is a tool for
managing and orchestrating open management and services. Its objective is to
guarantee secure communication between the SMO framework and the O-Cloud
platform. Therefore, the O2 interface as such is also extremely empowering
[12].

###  II-D A1-Interface

The A1-Interface facilitates communication between the Non-Real-Time RIC and
the Near-Real-Time RIC. This involves the transmission of data from internally
and externally O-RAN sources to the SMO-Framework. As an example, the
declarative A1 policy-based guidelines, which contain statements about goals
and resources for UEs and cells, can be mentioned here. Other administrative
information shared via this interface are used for ML models (training,
updating, use of ML models). Various internal as well as external O-RAN data
sources are made available as enrichment information. The availability and
general use of these sources is not crucial for the fulfilment of a task, but
only serves the purpose of general improvement [8].

###  II-E R1-Interface

To access RIC functions that do not run in real time, the so-called rApps
utilise the R1 interface. Examples here include the provision of policy-based
guidance and enrichment information which is obtained through the A1
interface. In addition, data analysis, AI/ML training and information
retrieval for RAN optimisation or for usage by other rApps is another main
component. As well as the recommendation of configurations which can be
transmitted via the O1 interface [13].

###  II-F E2-Interface

In order to connect the near-RT RIC with the so-called E2 nodes, the
E2-Interface is used. In general it shall support all protocol layers and
interfaces that are defined in 3GPP radio access networks. The objective is to
manage and improve the E2 nodes and the resources they consume. To make this
feasible, the ”RAN Function Network Interfaces (NI)” service can observe and,
if necessary, modify the entire data traffic of the network interface of each
individual direct node [14].

###  II-G Open Fronthaul M-Plane

The Open Fronthaul M-Plane interface allows the management of the open radio
unit components. It is used for performance reporting or for initialising and
configuring operating parameters. Particularly relevant from a risk
perspective is the possibility of updating the software of the components with
this interface [15].

###  II-H Open Fronthaul CUS-Plane

The user and control plane data of the Uu interface are transmitted via the
Open Fronthaul CUS-Plane interface. Furthermore, this component is in charge
of synchronizing the time between the Open Distributed Unit and the Open Radio
Unit. [16].

##  III Stakeholders

To be able to analyse risks within an architecture more precisely, it is
important to determine all stakeholders within the system and to define their
capabilities. In table I, possible attackers are therefore briefly explained
according to the models defined by [17], [7] and [18]. We have extracted what
we consider to be the most important stakeholders and categorised them
according to their level of access and overall safety-critical impact.

In total, we have assigned four (from L1 to L4) access levels. The higher the
level, the greater the potential impact of the capabilities available to the
stakeholder. The categorisation of the individual levels was declared by us on
the basis of the respective possibilities and their effects. All stakeholders
who use only a few or only individual areas of the RAN and can exploit it if
fall under L1. We therefore assume a relatively low common risk. At level two,
the stakeholder can further influence the component and also control it if
necessary, i.e. he has explicit knowledge of the respective device and either
operates it himself or knows to what extent and in what environment it is
operated. In terms of risk, L2 already causes considerably more damage than
L1, which is why we declare the overall risk to be medium. The next level, L3,
describes all stakeholder entities that have control and access to a complete
stand-alone RAN system. This class poses a very high risk, as the instances
have the ability to manipulate any services or hardware components. Thus, L3
creates a high risk with regard to the entire system. The last and highest
access category is level 4, representing the highest possible general risk
(severe). It refers to actors which are specialised in the construction or
provision of RAN subsystems as well as complete systems. The main difference
to L3 is that L4 stakeholders not only operate the network, but also configure
and create it themselves. This means that they have fundamental knowledge
about the functionality of the individual components. Thus, it can be assumed
that possibly malicious L4 stakeholders can cause the greatest possible
damage. Another point that underlines the seriousness of this is that L4
entities can already manipulate the overall RAN at an early stage. This means,
for example, before the RAN operator in L3 has the chance to do so. It may be
the case that an L3 stakeholder also falls into the L4 category at the same
time, e.g. if an MNO integrates its RAN system completely itself.

Later, in connection with the interfaces explained in section II, we will
derive which measures are necessary to reduce or mitigate the attack surfaces.

TABLE I: Itemised categorisation of the key stakeholder Stakeholder | Access | Short Description | Capability | Overall Risk
---|---|---|---|---
External [18] / Outsider [7] | L1 |  | Does not match with the other
---
defined stakeholder classes.
Is able to access the interfaces
defined by 3GPP or the O-RAN
Alliance.
| Is capable of performing designated types
---
of threats in several risk zones
Low
| Network consumer
---
(Similar to User [7]
& Connected Devices [18])
L1 |  | Is participating in the network as
---
a normal and legitimate service
consumer by connecting
entities or services that facilitate
network functionality.
| Is able to exploit specific types of
---
threats in different risk zones using
legitimate credentials/secrets to exploit
the respective network.
Low
Government Services [18] | L2 |  | Authority that has legal rights
---
to intercept/tap network traffic.
| Decoding and reading of data sets sent
---
over the network
Medium
| Hard-/Software Suppliers [18]
---
& Manufacturers [17]
L2 |  | Vendor of one or more specific
---
hard- or software component utilised in
the system ”… providing services or
infrastructure to MNOs in order
to build and/or operate their networks” [18]
| Can substitute benign hard-/software
---
components with malicious ones
Medium
| RAN-Operator [7]
---
& MNOs [18]
L3 |  | Refers to traditional Mobile (Virtual)
---
Network operators, as well as critical
infrastructure operators from non
telecommunication sectors.
Has complete and comprehensive
control over the respective RAN.
| Possesses enhanced capabilities to
---
tamper hardware components as well as
specific services
High
RAN-Integrator | L4 |  | Is specialised in assembling RAN
---
subsystems into a functioning entity
and ensuring that the subsystems
function smoothly together.
| Controls and operates software deployed
---
in the RAN as well as all hardware.
Thus, he has the possibility to
manipulate all individual entities at
each available stage.
Severe

##  IV Mitigations of Security Problems

In this section, we deal with the issue of how to solve the problems denounced
and, above all, what steps are necessary to do so and how much effort is
required. For this purpose, we identify security aspects that should be
applied.

###  IV-A Enforcement of clear safety & security concepts

One of the main components to fully secure a system is the use of standardised
mechanisms to ensure both safety and security. It would be necessary, for
example, to create a clear definition of a clear rights and role concept with
regard to the communication of interfaces and services. Furthermore, for
separation concepts, firewall-friendly designs, minimisation of the effects of
denial of service and the implementation of a zero trust model. To go one step
further in defining possible measures, we have extracted the individual
proposals from the enisa report [19] for network function virtualisation.
These measures can also be applied holistically to an Open-RAN system. In
Table 2 we have listed the best practices divided according to their level of
applicability. If we look at the individual practices, it becomes clear how
extensive our current security and safety options are in order to facilitate a
secure Open-RAN deployment.

###  IV-B Mandatory Encryption

Parts of the O-RAN definition currently require no or only weak encryption,
which weakens the overall security of a potential Open-RAN deployment. For
example, the encryption at the transport layer is only optional and legacy
protocols that rely on the weak encryption are not forbidden by default.
Consequently, it is recommended to increase the security of O-RAN by mandating
strong encryption and not allowing old protocols. This would indeed increase
the security of O-RAN. It is noteworthy however, that previous radio access
networks have the same shortcomings as Open-RAN when it comes to encryption,
and have suffered from security vulnerabilities as a result. The partial lack
of strong cryptography is thus not a threat introduced by O-RAN itself, but an
ever existing threat to telecommunication networks, that can now be fixed in
the O-RAN specification. Following the described security by design approach,
we strongly encourage strong cryptography should be mandatory. This would
result in a major security improvement compared to the previous standards.

###  IV-C Post-Quantum Security

Due to the dynamic development of quantum computers and their expected future
ability to break currently used classical public-key schemes, using post-
quantum schemes, i.e., quantum-resistant cryptographic schemes, should be at
least recommended within O-RAN. Thus, while encryption at large should be
mandatory, ”shall support” for post-quantum cryptography would be sufficient
for now. We want to stress that classically encrypted data is also at risk
before powerful quantum computers exist, since the encrypted data can be
stored now and be decrypted once powerful quantum computers exist. Hence, the
integrators have to carefully analyze which data need only short-term
protection and which data need to be protected for several decades, which
would then require post-quantum protection. Since the NIST process for
standardizing post-quantum cryptographic schemes is on the verge of announcing
first schemes that will be standardized, standardized PQC schemes will exist
once 6G is being deployed.

We want to emphasize two points that should be paid attention to when
developing post-quantum secure telecommunication protocols: First, O-RAN
relies mainly on symmetric cryptography to ensure authentication and
confidentiality of the data. So far, post-quantum security considerations for
telecommunication are often based on the assumption that quantum computers
mainly pose a serious threat to asymmetric cryptography [20, 21] \- and thus,
telecommunication protocols can ensure post-quantum security by doubling the
key-size. Arguably though, this claim is not backed by proofs but rather by a
truism that guides post-quantum security considerations; so in addition to
replacing the existing asymmetric cryptography, O-RAN’s symmetric
cryptosystems need to be re-evaluated w.r.t. quantum resilience (proofs) as
well to back up the claim that doubling the key size is indeed sufficient.
Second, the resource-constrained environment of telecommunication
infrastructure (e.g., SIMcards have low resources available, low network
bandwidth requirements are a must) and highly adversarial threat environment
(e.g., SIM cards are highly susceptible to side channels, the need to defend
against nation-station actors, and the fact that cloud environments introduce
new threats) require to carefully evaluate and tailor post-quantum
cryptography schemes towards use in telecommunication protocols. This process
is time-intensive - given the slow nature of telecommunication standardization
bodies and even more so the slow nature of actually implementing a new
telecommunication standard, it is urgent to address the topic (and integrate
post-quantum security in the standard) already now. Recent works have already
started to explore these trade-offs for the Subscriber Concealed Identifier
(SUCI), which aims at concealing subscriber identities [22]. The developed
post-quantum secure protocols for the SUCI, tailored towards use in
telecommunication protocols, serve as a great starting point to explore the
challenges and necessary changes required when bringing post-quantum security
to next generation telecommunication networks.

###  IV-D Cloud Environments

Moving Open-RAN components to the cloud results in a new threat landscape,
specifically when considering a malicious cloud provider. Recent risk analysis
correctly states that a cloud provider that controls the O-Cloud has the same
capabilities as the RAN-Operator. Currently, there are few mandatory safety
measures in the O-RAN requirements and therefore two recommendations can be
clearly made to mitigate the problems: 1) Integrate security measures to
defend against malicious cloud provider, for example, through Trusted
Execution Environments and 2) to integrate mandatory access control and
security requirements in the O-RAN definition.

A malicious cloud provider would indeed undermine a RAN’s security. In
practice however, operators expect to build and run their own data-centers
instead of relying on external cloud solutions. As a result, the O-Cloud
operator can be assigned the same level of trust as the RAN-Operator
themselves. This completely mitigates the malicious cloud provider scenario.

We also recommend to only use trusted datacenters and cloud solution for the
O-Cloud: Defending against malicious cloud providers through the usage of
confidential computing and trusted execution environments is non-trivial: The
security provided by confidential computing and trusted execution environments
have been undermined with various attack vectors, stemming from the very
powerful attacker model.

If the O-Cloud is trusted and adheres to the standard security best practices
in its configuration and design, we expect the security risk induced through a
cloud-based RAN to be minimal.

###  IV-E Clarification & concrete definition

From the point of view of security research, the current state of the O-RAN
specification still leaves a number of wide gaps in the specification of
security aspects. One of the reasons why this is the case is the philosophy of
the O-RAN Alliance to mainly provide some kind of guidelines. This is why the
term ”shall support” is often found in the documents. The actual
implementation of the necessary security concepts is, in their view, the
responsibility of the integrator or hardware manufacturer. Of course, this is
a clear thorn in the side for an authority like the German Federal Office for
Information Security, the BSI. The German Technical Inspection Agency (TÜV)
would also like to have more concrete definitions, e.g., in order to be able
to precisely evaluate and approve an Open-RAN system. However, this will
probably never be possible with the plan pursued by the O-RAN Alliance, since
they only provide a possible architecture and specifications for it, but do
not develop an additional standard to 3GPP. The O-RAN Alliance merely defines
a technical concept that is intended to improve interoperability in the radio
access networks of mobile networks. This fact, however, offers us a great
opportunity as security researchers in this field. We are now able to put an
additional security view on top of the concept and thus mitigate all security-
critical concerns.

###  IV-F Privacy

Technological innovations and emerging trends in the context of 6G, such as
the tighter integration/convergence of sensing and communication, may pose
unique challenges to both communication security and privacy. For example, in
dual radar and communication systems, the inclusion of data into the probing
signal, used to illuminate targets, makes it prone to eavesdropping from
potentially malicious targets. Even if the data itself is protected with
higher-layer encryption, the existence of a communication link can still be
detected from a malicious agent, thus making it prone to cyberattacks [23].
Similarly, the introduction of sensing capabilities and explicit localization
of users/devices to improve communication network performance, may pose
significant privacy challenges.

In face of these challenges, it is important to assess the potential of
certain approaches to provide secure, privacy-preserving solutions at the
radio access level. E.g. from a privacy perspective, it is essential to
collect only user related data that is absolutely necessary for the operation
of the network and to move away from the explicit localization paradigm as
much as possible. In addition, appropriate privacy-enhancing technologies
should be integrated, e.g. differential privacy. An example is provided by the
use of channel charting [24] to enhance network functionalities such as, e.g.,
radio resource management, beam management (mmWave and sub-THz), cell-
association and handover. Channel charting uses unsupervised/semi-supervised
learning to embed high-dimensional information about the radio environment
into a low-dimensional chart and relies on a pseudo-location (i.e. location in
the low-dimensional chart) of the users. The use of pseudo-location on a
channel chart can be seen as a privacy protection feature, allowing
localization-related services to be delivered without requiring the actual
user location to be estimated. While promising in general, the concept needs
to be further formalized and thoroughly investigated from the perspective of
privacy before inclusion as a design metric.

##  V Machine Learning

O-RAN seeks to utilize novel Machine Learning (ML) techniques such as deep
learning to automate operational network functions and reduce operational cost
once applied at both component and network levels [25]. However, the Open-RAN
architecture entails security challenges because of its inherently open and
modular nature. Devices within Open-RAN are able to run software that does not
trust the hardware it is running on. Due to the openness of Open-RAN, it is
susceptible to intrusion. Cyber-attacks pose a threat to security goals and
could result in denial-of-service (DoS) within the Open-RAN network. These
attacks present vulnerability to the Non-RT RIC and the Near-RT RIC controller
operations within Open-RAN [26].

###  V-A Anomaly Detection

Ideally, intrusions are detected with high performance, high speed, and a low
false-positive alarm rate. It is therefore clear that traditional human means
will not be enough to detect and combat cyber-attacks in the required manner.
Consequently, robust ML techniques will also play a big role in tackling
anomaly-based intrusion detection.
Anomaly-based IDS can be divided into three classes: statistical anomaly IDS,
knowledge-based IDS, and ML IDS [27]. The focus here is on novel ML-based IDS.
The ML-based anomaly detection methods in IDS [28] can be classified into
supervised learning, semi-supervised learning, unsupervised learning,
reinforcement learning, and graph neural networks [29]. The limitations of
traditional (shallow) ML-based IDS, such as the reliance on manual feature
engineering to extract useful information from network traffic and dealing
with unlabeled, high-dimensional data, have paved the way for Deep Learning
(DL)-based IDS [30] that do not require manual feature engineering and can
automatically learn complex features from raw data due to their deeper
structure [31].
Initial RAN-specific anomaly detection approaches have been presented by [32],
[33], [34], [35]. However, the O-RAN architecture with its different
interfaces poses new challenges for an anomaly detector, as next-generation
RAN data can be roughly divided into performance management, configuration
management, and fault management data. It has been shown that considering only
performance management data in anomaly detection can lead to sub-optimal
results [34], [35]. Therefore, it is important to define the requirements
under which the anomaly detector must operate, as not all approaches presented
so far take into account the different data streams in the next-generation
RAN.
In terms of where the anomaly detector is deployed, for example, the training
of ML models can be performed in the Non-RT RIC controller. Subsequently, the
learned model is inputted into the Near-RT RIC controller which uses this
model on Real-Time data and makes Real-Time decisions in an online manner
[36].

###  V-B Importance of Robustness

Moreover, every ML-based learning method mentioned above is prone to an attack
technique referred to as adversarial ML [37]. Hence, incorporating ML
solutions into the RAN poses new cyber-security threats. Consequently, having
a thorough understanding of the attack surface and ensuring the robustness of
the O-RAN to adversarial machine learning threats are mandatory for securing
the new Open-RAN architecture [38].

###  V-C Explainable AI

Notably, Explainable AI (XAI) has a great potential for securing ML systems as
explanations are key in identifying and defending against different types of
attacks. To elaborate, if explanations become available for adversarial
attacks, they become easier to defend against [39]. Additionally, explanations
can support effective root cause analysis and localisation [40].

TABLE II: List of best practices presented by enisa [19] Level | Practice | Level | Practice
---|---|---|---
Organisational | Trust model | Technical | Tracking version changes
SLAs establishment | Deployment security
Policy | Zero Trust | Software detection or relocation
Security Assessment of new or changes to existing Services | (Post-Quantum) Cryptography
Vulnerability handling patch management | Hypervisor protection
Security testing and assurance | Security Management and orchestration
Incident management | Remote attestation
Secure Update Management | Software compliance and integrity preservation
Restriction on installing applications | Segmentation/isolation between network functions
Defense in depth | Secure boot integrity
String password policy | Data protection and privacy
Secure supply chain | Encrypting Volume/swap Areas
Resources inventory management system and database | Trusted computing technologies
Apply hardening policies | Hardware security
Multi-vendors segregation and trust | Centralised log auditing
Security by design | Use and ownership of ”root” administration credentials
Life cycle management | Local or removal Blade Storage - SAN protection
Software Bill Of Materials (SBM) | Network security
Technical | Trusted time source | SDN security management
Secure 3rd party hosting environments | MANO access control and management
Redundancy and backup | VIM/CISM connectivity to Hypervisor/CIS
Specific container security controls | Recovery and reinstallation
OSS/BSS protection | Deploying VMs/Containers of differing trust levels
LI capabilities | Orchestration platform security management
User plane security |
MEC security |

##  VI Conclusion

In our research so far, we have not found that Open-RAN concepts such as O-RAN
introduce major problematic security issues. Of course, there is an increased
attack target due to the larger surface area of the ecosystem. However, such
complex systems can be secured through various practices such as in Table 2.
We can only agree with the opinion of Mimran et. al. [41] that the security
risks that arise are mainly due to the 5G requirements and less due to the
specific decisions in the Open-RAN architecture. In general, we can say that
now with O-RAN the attack surface is much clearer than in contrast to previous
proprietary implementations, which were unclear and in general rather a big
question mark. Our task now is to define the overall security methodologies
for the individual critical points and to apply them. This will then provide
integrators and network operators with options for operating their Open-RAN
deployment securely.

## Acknowledgment

The authors acknowledge the financial support by the Federal Ministry of
Education and Research of Germany in the programme of “Souverän. Digital.
Vernetzt.” Joint project 6G-RIC, project identification number: 16KISK020K and
16KISK0[21-35].

The authors of this paper also acknowledge the support of educational
conversations with Jean-Pierre Seifert around the topics discussed in this
paper, as well as the German Federal Office for Information Security (BSI).

## References

  * [1] Sameer Kumar Singh, Rohit Singh and Brijesh Kumbhani  “The Evolution of Radio Access Network Towards Open-RAN: Challenges and Opportunities”  In _2020 IEEE Wireless Communications and Networking Conference Workshops (WCNCW)_ , 2020, pp. 1–6  DOI: 10.1109/WCNCW48565.2020.9124820
  * [2] Sławomir Kukliński, Lechosław Tomaszewski and Robert Kołakowski  “On O-RAN, MEC, SON and Network Slicing integration”  In _2020 IEEE Globecom Workshops (GC Wkshps_ , 2020, pp. 1–6  DOI: 10.1109/GCWkshps50303.2020.9367527
  * [3] Salah Eddine Elayoubi, Sana Ben Jemaa, Zwi Altman and Ana Galindo-Serrano  “5G RAN Slicing for Verticals: Enablers and Challenges”  In _Comm. Mag._ 57.1 IEEE Press, 2019, pp. 28–34  DOI: 10.1109/MCOM.2018.1701319
  * [4] John S. Vardakas et al.  “Towards Machine-Learning-Based 5G and Beyond Intelligent Networks: The MARSAL Project Vision”  In _2021 IEEE International Mediterranean Conference on Communications and Networking (MeditCom)_ , 2021, pp. 488–493  DOI: 10.1109/MeditCom49071.2021.9647671
  * [5] “O-RAN Architecture Description 5.0, Technical Specification”  In _WG1: Use Cases and Overall Architecture Workgroup_ , 2021  URL: https://www.o-ran.org/specifications
  * [6] Terry Benzel et al.  “Design Principles for Security”, 2005
  * [7] BSI  “Open-RAN Risikoanalyse”, 2021  Bundesamt für Sicherheit in der Informationstechnik  URL: https://www.bsi.bund.de/SharedDocs/Downloads/DE/BSI/Publikationen/Studien/5G/5GRAN-Risikoanalyse.html
  * [8] “O-RAN Architecture-Description 6.0, Technical Specification”  In _WG1: Use Cases and Overall Architecture Workgroup_ , 2022  URL: https://www.o-ran.org/specifications
  * [9] “O-RAN Operations and Maintenance Interface Specification 6.0, Technical Specification”  In _WG10: OAM for O-RAN_ , 2022  URL: https://www.o-ran.org/specifications
  * [10] “ITU-T Recommendation M.3010 : Principles for a telecommunications management network”  URL: https://www.itu.int/rec/T-REC-M.3010
  * [11] Heinz-Gerd Hegering, Sebastian Abeck and Bernhard Neumair  “Integrated Management of Networked Systems: Concepts, Architectures, and Their Operational Application”  San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1999
  * [12] “Cloud Architecture and Deployment Scenarios for O-RAN Virtualized RAN v02.02”  In _WG6: Cloudification and Orchestration Workgroup_ , 2021  URL: https://www.o-ran.org/specifications
  * [13] “O-RAN Non-RT RIC Architecture 1.0”  In _WG2: Non-real-time RAN Intelligent Controller and A1 Interface Workgroup_ , 2021  URL: https://www.o-ran.org/specifications
  * [14] “Near-Real-time RAN Intelligent Controller Architecture and E2 General Aspects and Principles v02.01”  In _WG3: Near-real-time RIC and E2 Interface Workgroup_ , 2022  URL: https://www.o-ran.org/specifications
  * [15] “Near-Real-time RAN Intelligent Management Plane Specification 8.0”  In _WG4: Open Fronthaul Interfaces Workgroup_ , 2022  URL: https://www.o-ran.org/specifications
  * [16] “Fronthaul Control, User and Synchronization Plane Specification 8.0”  In _WG4: Open Fronthaul Interfaces Workgroup_ , 2022  URL: https://www.o-ran.org/specifications
  * [17] Dudu Mimran et al.  “Evaluating the Security of Open Radio Access Networks”  arXiv, 2022  DOI: 10.48550/ARXIV.2201.06080
  * [18] NIS  “Report on the cybersecurity of Open RAN”, 2022  NIS Cooperation Group  URL: https://digital-strategy.ec.europa.eu/en/library/cybersecurity-open-radio-access-networks
  * [19] enisa  “NFV Security in 5G - Challenges and Best Practices”, 2022  European Union Agency for Cybersecurity  URL: https://www.enisa.europa.eu/publications/nfv-security-in-5g-challenges-and-best-practices
  * [20] Chris J Mitchell  “The impact of quantum computing on real-world security: A 5G case study”  In _Computers & Security_ 93 Elsevier, 2020, pp. 101825
  * [21] Jing Yang and Thomas Johansson  “An overview of cryptographic primitives for possible use in 5G and beyond”  In _Science China Information Sciences_ 63.12 Springer, 2020, pp. 1–22
  * [22] Vincent Ulitzsch, Shinjo Park, Soundes Marzougui and Jean-Pierre Seifert  “A Post-Quantum Secure Subscription Concealed Identifier for 6G”  In _To appear in 15th ACM Conference on Security and Privacy in Wireless and Mobile Networks (WiSec 2022)_ , 2022
  * [23] Zhongxiang Wei et al.  “Towards Multi-Functional 6G Wireless Networks: Integrating Sensing, Communication and Security”  arXiv, 2021  DOI: 10.48550/ARXIV.2107.07735
  * [24] Christoph Studer et al.  “Channel Charting: Locating Users Within the Radio Environment Using Channel State Information”  In _IEEE Access_ 6, 2018, pp. 47682–47698  DOI: 10.1109/ACCESS.2018.2866979
  * [25] Open RAN Alliance  “O-RAN: towards an open and smart RAN”  In _White paper_ , 2018, pp. 1–19
  * [26] Paul H Masur and Jeffrey H Reed  “Artificial Intelligence in Open Radio Access Network”  In _arXiv preprint arXiv:2104.09445_ , 2021
  * [27] Geeta Kocher and Gulshan Kumar  “Machine learning and deep learning methods for intrusion detection systems: recent developments and challenges”  In _Soft Computing_ 25.15 Springer, 2021, pp. 9731–9763
  * [28] Song Wang et al.  “Machine Learning in Network Anomaly Detection: A Survey”  In _IEEE Access_ 9 IEEE, 2021, pp. 152379–152396
  * [29] Xiaoxiao Ma et al.  “A comprehensive survey on graph anomaly detection with deep learning”  In _IEEE Transactions on Knowledge and Data Engineering_ IEEE, 2021
  * [30] Hongyu Liu and Bo Lang  “Machine learning and deep learning methods for intrusion detection systems: A survey”  In _applied sciences_ 9.20 Multidisciplinary Digital Publishing Institute, 2019, pp. 4396
  * [31] Zeeshan Ahmad et al.  “Network intrusion detection system: A systematic study of machine learning and deep learning approaches”  In _Transactions on Emerging Telecommunications Technologies_ 32.1 Wiley Online Library, 2021, pp. e4150
  * [32] Dovile Momkute, Karolis Žvinys and Vaidotas Barzdėnas  “Adapted Anomaly Detection for RAN Performance”  In _2018 IEEE 6th Workshop on Advances in Information, Electronic and Electrical Engineering (AIEEE)_ , 2018, pp. 1–4  IEEE
  * [33] Yannan Yuan et al.  “Anomaly Detection and Root Cause Analysis Enabled by Artificial Intelligence”  In _2020 IEEE Globecom Workshops (GC Wkshps_ , 2020, pp. 1–6  IEEE
  * [34] Tobias Sundqvist, Monowar Bhuyan and Erik Elmroth  “Uncovering latency anomalies in 5G RAN-A combination learner approach”  In _2022 14th International Conference on COMmunication Systems & NETworkS (COMSNETS)_, 2022, pp. 621–629  IEEE
  * [35] Faris B Mismar and Jakob Hoydis  “Unsupervised learning in next-generation networks: Real-time performance self-diagnosis”  In _IEEE Communications Letters_ 25.10 IEEE, 2021, pp. 3330–3334
  * [36] Samad Ali et al.  “6G white paper on machine learning in wireless communication networks”  In _arXiv preprint arXiv:2004.13875_ , 2020
  * [37] Jinxin Liu, Michele Nogueira, Johan Fernandes and Burak Kantarci  “Adversarial Machine Learning: A Multilayer Review of the State-of-the-Art and Challenges for Wireless and Mobile Systems”  In _IEEE Communications Surveys Tutorials_ 24.1, 2022, pp. 123–159  DOI: 10.1109/COMST.2021.3136132
  * [38] Ron Bitton et al.  “Adversarial Machine Learning Threat Analysis in Open Radio Access Networks”  In _arXiv preprint arXiv:2201.06093_ , 2022
  * [39] Atul Rawal et al.  “Recent Advances in Trustworthy Explainable Artificial Intelligence: Status, Challenges and Perspectives”  In _IEEE Transactions on Artificial Intelligence_ 1.01 IEEE Computer Society, 2021, pp. 1–1
  * [40] Ashima Chawla et al.  “Interpretable Unsupervised Anomaly Detection For RAN Cell Trace Analysis”  In _2020 16th International Conference on Network and Service Management (CNSM)_ , 2020, pp. 1–5  IEEE
  * [41] Dudu Mimran et al.  “Evaluating the Security of Open Radio Access Networks”  In _CoRR_ abs/2201.06080, 2022  arXiv: https://arxiv.org/abs/2201.06080

TABLE III: List of abbreviations Term | Description
---|---
3GPP | Third Generation Partnership Project
BSS | Business Support System
CISM | Container Infrastructure Service Management
CU | Central Unit
DU | Distributed Unit
DL | Deep Learning
DoS | Denial-of-Service
FCAPS | Fault, Configuration, Accounting, Performance and Security
IDS | Intrusion Detection Systems
ISO | International Organization for Standardization
LI | Lawful Interception
MANO | Management and Orchestration
MEC | Multi-access Edge Computing
MIMO | Multiple Input Multiple Output
MNO | Mobile Network Operator
NG-RAN | Next Gerneration NodesBs Radio Access Network
OSS | Operations Support System
PQS | Post-Quantum Security
RAN | Radio Access Network
RIC | RAN Intelligent Controller
RU | Remote Unit
SAN | Storage Area Network
SDN | Software Defined Network
SMO | Service Management and Orchestration
SLA | Service-level Agreement
SoC | System-on-Chip
UE | User Equipment
VIM | Virtual Infrastructure Manager
VM | Virtual Manager
vRAN | Virtualized Radio Access Network
XAI | Explainable AI


# Evaluating Counterfactual Explanations Using Pearl’s Counterfactual Method

Bevan I. Smith
School of Mechanical, Industrial and Aeronautical Engineering,
University of the Witwatersrand,
Johannesburg, South Africa
<EMAIL_ADDRESS>

###### Abstract

Counterfactual explanations (CEs) are methods for generating an alternative
scenario that produces a different desirable outcome. For example, if a
student is predicted to fail a course, then counterfactual explanations can
provide the student with alternate ways so that they would be predicted to
pass. The applications are many. However, CEs are currently generated from
machine learning models that do not necessarily take into account the true
causal structure in the data. By doing this, bias can be introduced into the
CE quantities. I propose in this study to test the CEs using Judea Pearl’s
method of computing counterfactuals which has thus far, surprisingly, not been
seen in the counterfactual explanation (CE) literature. I furthermore evaluate
these CEs on three different causal structures to show how the true underlying
causal structure affects the CEs that are generated. This study presented a
method of evaluating CEs using Pearl’s method and it showed, (although using a
limited sample size), that thirty percent of the CEs conflicted with those
computed by Pearl’s method. This shows that we cannot simply trust CEs and it
is vital for us to know the true causal structure before we blindly compute
counterfactuals using the original machine learning model.

_K_ eywords Counterfactual explanations $\cdot$ Structural causal model (SCM)
$\cdot$ Counterfactuals

##  1 Background

This study presents a method for validating counterfactual explanations using
Pearl’s counterfactual method [1, 2]. Counterfactual explanations (CEs) form
part of the rapidly expanding field of explainable machine learning which aims
to explain why machine learning models make their predictions [3, 4]. If the
predictions made by the model were undesirable, knowing why the predictions
were made would allow us to generate counterfactual explanations that give us
alternative desirable outcomes. For example, if a student was predicted to
fail, we could generate CEs that would advise the student how to change
certain features to increase her probability of passing. A counterfactual
explanation (CE) “describes the smallest change to the feature values that
changes the prediction to a predefined output" [3]. Clearly we can see that
CEs promise much and the applications are vast, such as improving pass rates,
advising patients what to change to improve health, advising clients what to
do so they can obtain a bank loan, and so forth. The main problem however is
that CEs, and the algorithms that generate them, are yet to be properly
validated [5]. Why would they need to be validated or tested? This is because
CEs generated using the various optimization algorithms such as DiCE and WACH
[6, 4], are generated using the model trained on the original data [3]. This
is problematic because machine learning (ML) models do not care about the
causal relationships and causal structure between the features, only
correlation [1]. The ML model is designed to make good predictions without
concern for causality. Therefore if the algorithms that generate CEs are based
on these ML models, would they be the same as those generated via a ground
truth structural causal model? We already know that when we fit simple linear
regression models without taking into account the causal structure, there are
biases in the coefficients [7]. Examples include leaving out a confounding
feature in a regression model and including a collider in the model. If not
taking into account causal relationships in a regression model causes
problems, how much more if we generate counterfactual explanations using the
existing trained ML model. The problem that this study is aiming to address is
the following: Counterfactual explanations are generated using machine
learning models that do not necessarily take into account the true underlying
causal structure in the data. This can lead to erroneous estimations of the
counterfactuals.

To address this problem, I propose using the counterfactual methods found in
the work of Judea Pearl[1, 2]. Details about this method is found later in
Section 5. However, the essential idea is that to compute counterfactuals, we
first require a structural causal model (SCM) and associated graph that
describe the true data generating process. This allows us to compute
counterfactuals using the true causal relationships between the data, whether
it be between input features or between input and output features. Whereas
computing CEs from machine learning models would not necessarily incorporate
causal relationships, Pearl’s method does. I propose using this method to
evaluate and test the current CE methods.

The main contributions in this paper are:

  * •

I present a method for evaluating CEs using the counterfactual method
developed by Pearl. This has surprisingly not been applied in the CE
literature thus far. This would allow us to evaluate our CEs to know if we can
trust them.

  * •

Using this method, I show how the CEs generated using current methods conflict
with those computed via Pearl’s method and how different causal structures
affect the results. This should alert us to the fact that it is vital to
understand the true underlying causal structure before generating CEs.

##  2 Counterfactual Explanations

Recently, counterfactual explanations (CEs) have been gaining much traction
and interest in the machine learning literature [4, 3, 8, 5]. The aim of CEs
is to manipulate the inputs of a model to change the output to a desired one.
CEs are generated by solving an optimization problem where the input feature
space is perturbed to as close a feature space as possible, but one that leads
to a different, desirable output [8, 3]. This is therefore an optimization
problem that aims to perturb the input space as little as possible, in order
to change the output.

###  2.1 Method by Wachter et al.

An example of a CE method is that by Wachter et al. [4]. The loss function is
given below:

$L(x,x^{\prime},y^{\prime},\lambda)=\lambda\cdot(f(x^{\prime})-y^{\prime})^{2}+d(x,x^{\prime}),$
(1)

where $x$ and $x^{\prime}$ are the original and counterfactual input space
respectively, $y^{\prime}$ is the counterfactual output, and $\lambda$ a
tuning parameter. Importantly, $f$ refers to the original machine learning
model trained on the data, which does not necessarily take into account causal
relationships in the data. $L$ aims to balance between minimizing the distance
between original and counterfactual $x$ and $x^{\prime}$ and predicted
$f(x^{\prime})$ and original $y^{\prime}$ outputs.

###  2.2 Method by Mothilal et al.

Another CE method was developed by Mothilal et al. [6] and is the method used
in this study. It is known as DiCE: Diverse Counterfactual Explanations. Their
method of optimization is based on the following loss function:

$C(x)=\underset{{c_{1},...,c_{k}}}{\arg\min}\frac{1}{k}\sum_{i=1}^{k}yloss(f(c_{i}),y)+\frac{\lambda_{1}}{k}\sum_{i=1}^{k}dist(c_{i},x)-\lambda_{2}dpp\\_diversity(c_{1},...,c_{k}),$
(2)

where $c_{i}$ is a counterfactual explanation (CE), k is the total number of
CEs, f(.) refers to the ML model trained on the data, yloss(.) is a metric
minimizing the distance between counterfactual predictions and the desired
outcome, dist(.) is the distance between counterfactual input $c_{i}$ and
actual input x, dpp_diversity(.) functions to maximize diversity, and finally
the lambda values balance the three parts of the loss function. An important
aspect of this study is to show that CE methods shown above use the ML model
trained on the data.

###  2.3 Characteristics of CEs

CEs require important characteristics. I highlight the following:

  * •

Feasibility/plausibility: This refers to the CEs being realistic; not being
smaller or larger than those observed in the original data [9]. It means the
CEs must come from a possible world [4].

  * •

Actionable: Features must be able to be modified, to be changed. Features that
are non-actionable include age, gender, race etc. [9].

  * •

Causality: I discuss this in more detail in the Discussion section (Section
8). This characteristic requires that causal relationships be maintained in
the CEs. However, I push back on this requirement.

##  3 Structural Causal Models

Structural causal models (SCMs) refer to the true data generation process, and
allow for counterfactual analysis [5, 1]. SCMs are associated with graphical
causal models called directed acyclic graphs (DAGs), seen on the right side of
Figure 1. Consider a set of features $X_{1},...,X_{n}$, known as endogenous
features, where each feature $X_{i}$ is generated via a deterministic function
$f_{i}$ which is a function of its causes (or parents, PA). $PA_{i}$ refers to
the parents (known causes) of $X_{i}$,. and $U_{i}$ refers to some stochastic
unexplained cause of $X_{i}$ [10], also known as exogenous features (outside
the model). Each feature corresponds to a vertex in the DAG.

$X_{i}:=f_{i}(PA_{i},U_{i})\leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ (i=1,...,n)$ (3)

Figure 1 presents some intuition behind the difference between generating
counterfactuals based on a machine learning model (left image) and a true
structural causal model DAG (right image) [11]. SCMs would generate
counterfactuals based on the true causal structure but CEs are generated based
on the original machine learning model $f$ which does not necessarily take
into account causal structure. The question is: would the CE algorithms
generate the same counterfactuals as those via SCMs?

Figure 1: ML model (left) vs SCM (right) [11].

##  4 Causal Structures

Before presenting Pearl’s Counterfactual Method (PCM) in Section 5, I present
three important causal structures in causal inference: chains (or mediators),
forks (common cause) and colliders. This study will use these three structures
when evaluating the CEs using PCM. According to Pearl, these three types are
the building blocks of causal structures and “enable us to test a causal
model, discover new models, evaluate effects of interventions, and much more"
[2]. They are presented using a directed acyclic graph and an SCM.

###  4.1 Chain (mediator)

The chain or mediator DAG is shown in Figure 2. An example of a chain would be
a fire (X), smoke (Z) and alarm (y) system. The fire does not directly cause
the alarm to go off, but is required to first produce smoke that then sets off
the alarm. There is no direct causal path between fire and alarm, in this
case. Of course, there can be a causal path between X and y in other cases. We
are restricting ourselves here to the basic chain. What this model tells us is
that if we control for Z, then we block the causal flow between X and y and do
not allow for measuring the true effect of X on y. That is, X and y are
conditionally independent given Z. What this implies practically is that if we
include X, Z and y in our model, we are effectively controlling for Z and
blocking the path between X and y. The SCM for Figure 2 is seen in Equations 4
and 5. Here we introduce the exogenous variables, U, that are vital to Pearl’s
method seen later. Z is a function of X and $U_{z}$, where X is an endogenous
feature (e.g. fire) and $U_{z}$ refers to all external features causing Z that
we can not account for one by one. This goes for all U features seen later.

$X$$y$$Z$$U_{y}$$U_{z}$ Figure 2: DAG showing chain path between X and Z and
y. Causality flow from X to Z to y.

$\displaystyle Z=f(X,U_{z})$ (4) $\displaystyle y=f(Z,U_{y})$ (5)

###  4.2 Fork (common cause)

Figure 3 shows a DAG representing a fork or common cause causal structure.
Practically, it represents spurious correlation where X and y are
conditionally independent given Z. That is, in this case, to obtain the true
causal effect of X on y requires conditioning (controlling) on Z. The SCM for
Figure 3 is shown in Equations 6 to 8. An example of this particular causal
structure could be where X refers to shark attacks and y refers to ice-cream
sales. We may obtain data that shows that these two events are correlated: as
shark attacks increase, ice-cream sales also increase. However, what explains
this spurious correlation is Z, which refers to summertime. Both shark attacks
and ice-cream sales are conditionally dependent on the seasons.

$X$$U_{x}$$y$$Z$$U_{y}$$U_{z}$ Figure 3: DAG showing common parent confounding
where Z is the parent of both X and y, but with no causal flow from X to y in
this case. Also known as spurious correlation.

$\displaystyle X=f(Ux)$ (6) $\displaystyle Z=f(Uz)$ (7) $\displaystyle
y=f(Z,Uy)$ (8)

###  4.3 Collider

The final causal structure used in this study is the collider, presented by
the DAG in Figure 4 with no causal path between X and y. An example of a
collider is where X is a sprinkler, y is rain, and Z is wet grass. Each
feature can be on or off, so to speak. Z is therefore a function of the
sprinkler and rain. For example, if Z = dry (off), then both sprinkler (X) and
rain (y) must be off. However, if Z = wet, either X or y is on, or both are
on. The important point to note here is that X and y are marginally
independent; they are independent, conditioned on nothing. Rain being on has
nothing to do with the sprinkler being on, for example. However, when we
condition on Z, we make sprinkler and rain conditionally dependent. This means
that by conditioning on grass, we create the illusion that there is a causal
relationship between otherwise independent features. For example, if we know
that grass is wet (Z = on) and that it is raining (y = on), then we
immediately know the value of the sprinkler (X = off). Two variables (X and y)
that are independent, become conditionally dependent, given z. This is the
opposite of chains and forks where conditioning on Z results in estimating the
actual causal relationship between X and y. The SCMs for the collider is seen
in Equations 9 to 10.

$X$$U_{x}$$y$$Z$$U_{y}$$U_{z}$ Figure 4: DAG showing collider path between X,
Z and y, but with no causal path between X and y.

$\displaystyle Z=f(X,y,U_{z})$ (9) $\displaystyle X=f(U_{x})$ $\displaystyle
y=f(U_{y})$ (10)

###  4.4 Difference between SCM and ML

What is vital to note from these three basic causal structures is that whereas
the SCMs would take into account the causal structure (i.e. the data
generating process), the ML model samples from the joint distribution of X, y
and Z without any thought for the causal structure. Clearly these three types
are distinctly different. For example, the DAG and linear model for an ML
model might look like that shown in Figure 5 and Equation 11. The question is,
if we generate CEs from the ML model, would they be the same as those
generated by the true SCM using Pearl’s method shown next?

$X$$y$$Z$$U_{y}$ Figure 5: DAG showing how a supervised machine learning model
would train on the data.

$\displaystyle y=f(X,Z,Uy)$ (11)

##  5 Pearl’s Counterfactual Method (PCM)

We now come to Pearl’s Counterfactual Method (PCM). This sections presents how
we go about generating counterfactuals using SCMs. The method is based on
Judea Pearl’s work that can be found in Chapter 8 of The Book of Why [2] and
Chapter 4 of Causal Inference in Statistics, A Primer [1]. The method involves
three steps: abduction, action and prediction and works as follows.

###  5.1 Abduction: to compute exogenous variables, U

Consider data with a known true causal structure. This structure has a DAG and
SCMs. For example, consider the DAG and SCM for Figure 3 where we have
spurious correlation from a common parent. The SCMs are given as Z = f(Uz) and
y = f(Z,Uy). The initial aim in PCM is to compute the values of the exogenous
variables Uz and Uy. These refer to all unmeasured variables for an individual
unit, that affect Z and y, respectively. These U variables, also referred to
as noise variables, describe the world of an individual person or observation.
It is these that we need to compute first before perturbing the input features
to compute counterfactuals. We need to compute these variables first because
they describe the "situation" of an individual unit that must remain the same
in both the factual and counterfactual worlds. Recall that the idea behind the
counterfactual is to keep all other things constant and to change only one
feature. These other things refer to the exogenous variables.

To compute Uz and Uy in our example, we select a unit (eg. a patient, a
student etc.) out of the data that we are interested in, and input observed
factual features into our SCMs. For example, say we measured X, Z and y from
observation. We input the individual unit’s Z into the first SCM, Z = f(Uz),
and compute Uz. To compute Uy, input measured y and Z from the individual
unit, into the second SCM, y = f(Z,Uy). We now have the exogenous variables
for the selected unit. This is called abduction and was computed based on
actual measured data.

###  5.2 Action: to intervene on counterfactual variables

In the second step, called action, we input the new counterfactual features,
for example, Z=z. This means that Z is no longer a function of Uz and we have
therefore deleted the arrow from Uz to Z and removed the relationship that Z
had with its parent, Uz. See Figure 6. When we apply a counterfactual, we
remove the relationship the feature has with its parents, regardless if they
are endogenous or exogenous features. This is also called applying Pearl’s do-
operator to Z. This is a vital step in computing counterfactuals. we now have
a new modified (or surgically altered) model with which we compute
counterfactual outcomes.

###  5.3 Prediction

In the final step, prediction, we use the new Z = z feature value as well as
the U variables computed in step 1, and compute the new y = f(Z=z,Uy).
Detailed examples will be presented in the results of how to perform this
using more complicated data.

$X$$y$$z$$U_{y}$$U_{z}$ Figure 6: DAG showing common parent confounding but
now with deleted edge between $U_{z}$ and z, showing a counterfactual quantity
independent of its’ parents.

##  6 Methodology

The CE method used in this study to generate CEs was DiCE ML [6]. This was
used because it is readily available as a Python package that is easy to
implement. The following presents the workflow for evaluating CEs using SCMs.
Also see Figure 7.

  1. 1.

Select a causal structure based on the three types discussed above and
generate a dataset based on the SCM.

  2. 2.

Train a supervised learning model on the data. This model is used in the DiCE
CE generation process.

  3. 3.

Identify a case/unit in the dataset.

  4. 4.

Generate CEs using DiCE which utilizes the model trained in the earlier step.
The CEs cause the output to switch to the opposite class.

  5. 5.

Now separately, carry out Abduction in Pearl’s method from Section 5.1. This
is to compute all the exogenous features.

  6. 6.

Input DiCE CEs into the SCMs according to Pearl’s Action method (Section 5.2).

  7. 7.

Estimate the output class using Pearl’s Prediction method ((Section 5.3).

$data$$ML$$DiCE$$CEs$$PCM$$class_{CE}$$class_{PCM}$$?$ Figure 7: Methodology
flow generating DiCE CEs, feeding the CEs into PCM and comparing the output
classes.

###  6.1 Experiments

This section details the experiments based on the three causal structures
above, but the datasets were made more complex with the aim of more closely
modeling real life. The dataset shown in Table 1 comprised seven features and
was based very loosely on features describing a university student taking a
course.

Table 1: Description of features used in this study. Feature | Description | Statistics
---|---|---
x1 | Grades |  $\mu$=50, $\sigma$=5
x2 | Age |  $\mu$=20, $\sigma$=1
x3 | Grades |  $\mu$=45, $\sigma$=6
x4 | Gender |  p = 0.6
x5 | Bursary |  p = 0.3
x6 | Grades |  $\mu$=70, $\sigma$=5
x7 | Grades |  $\mu$=50, $\sigma$=5

###  6.2 Experiment 1: Chain causal structure

The DAG and SCMs for experiment 1 are based on the chain causal structure
(Section 4.1) and is shown in Figure 8 and Equations 12 to 19. Note that in
the SCM, y is not a function of $x_{7}$ and would not make use of it in
predictions; however an ML model will include $x_{7}$ in the training and
prediction. In the DAG, for brevity and space-saving, note that X refers to
all the features not represented by $x_{7}$ and $x_{3}$, namely $x_{1}$,
$x_{2}$, $x_{4}$, $x_{5}$, $x_{6}$. They are features feeding directly into y
and are considered parents of y. Also, U refers to all the exogenous variables
feeding into X, namely $U_{1}$, $U_{2}$, $U_{4}$, $U_{5}$, $U_{6}$. In the
SCMs, the $\beta$ coefficients from 0 to 8 were arbitrarily selected as
follows: 0.4, 0.6, 0.4, 0.6, 0.7, 0.4, 0.4, 0.3, 0.7.

$u_{3}$$x_{7}$$y$$X$$x_{3}$$u_{y}$$u_{7}$$U$ Figure 8: Causal structure for
experiment 1 that is based on a chain (mediator) causal structure.

$\displaystyle x_{1}=u_{1}$ (12) $\displaystyle x_{2}=u_{2}$ (13)
$\displaystyle x_{3}=\beta_{1}x_{7}+u_{3}$ (14) $\displaystyle x_{4}=u_{4}$
(15) $\displaystyle x_{5}=u_{5}$ (16) $\displaystyle x_{6}=u_{6}$ (17)
$\displaystyle x_{7}=u_{7}$ (18) $\displaystyle
y=\beta_{0}+\beta_{1}x_{1}+\beta_{2}x_{2}+\beta_{3}x_{3}+\beta_{4}x_{4}+\beta_{5}x_{5}+\beta_{6}x_{6}+U_{y}$
(19)

####  6.2.1 Experiment 2: Fork (common cause) causal structure

The second experiment was based on the fork structure from Section 4.2. The
DAG and SCMs are presented in Figure 9 and Equations 20 and 27. Whereas the
causal path in Experiment 1 flowed from $x_{7}$ to $x_{3}$ to y, in Experiment
2, $x_{3}$ is the cause of both $x_{7}$ and y. Everything else is the same as
Experiment 1.

$u_{3}$$x_{7}$$y$$X$$x_{3}$$u_{y}$$u_{7}$$U$ Figure 9: Causal structure for
experiment 2 that is based on a fork (common cause/parent).

$\displaystyle x_{1}=u_{1}$ (20) $\displaystyle x_{2}=u_{2}$ (21)
$\displaystyle x_{3}=u_{3}$ (22) $\displaystyle x_{4}=u_{4}$ (23)
$\displaystyle x_{5}=u_{5}$ (24) $\displaystyle x_{6}=u_{6}$ (25)
$\displaystyle x_{7}=\beta_{8}x_{3}+u_{7}$ (26) $\displaystyle
y=\beta_{0}+\beta_{1}x_{1}+\beta_{2}x_{2}+\beta_{3}x_{3}+\beta 4x_{4}+\beta
5x_{5}+\beta_{6}x_{6}+u_{y}$ (27)

###  6.3 Experiment 3: Collider causal structure

The DAG and SCMs for experiment 3 based on the collider causal structure shown
in Figure 10 and Equations 28 to 35. Note again that there is no causal path
directly between $x_{7}$ and y and $x_{3}$ is not a parent of y. Therefore the
SCM does not consider $x_{7}$ nor $x_{3}$ as a parent of y and is not included
in computing y (see Equation 35). However, in an ML model, $x_{7}$ and $x_{3}$
would most likely be included. This again highlights the distinction between
ML modelling and SCM modelling.

$u_{3}$$x_{7}$$y$$X$$x_{3}$$u_{y}$$u_{7}$$U$ Figure 10: Causal structure for
experiment 3 that is based a collider causal structure.

$\displaystyle x_{1}=u_{1}$ (28) $\displaystyle x_{2}=u_{2}$ (29)
$\displaystyle x_{3}=\beta_{1}x_{7}+u_{3}$ (30) $\displaystyle x_{4}=u_{4}$
(31) $\displaystyle x_{5}=u_{5}$ (32) $\displaystyle x_{6}=u_{6}$ (33)
$\displaystyle x_{7}=u_{7}$ (34) $\displaystyle
y=\beta_{0}+\beta_{1}x_{1}+\beta_{2}x_{2}+\beta_{4}x_{4}+\beta_{5}x_{5}+\beta_{6}x_{6}+u_{y}$
(35)

###  6.4 Limitations and Assumptions

In all the CE computations and evaluations in this study, I make the
assumption that all features are actionable and all counterfactual values are
plausible. The aim of this study was not to study actionability and
plausibility but rather causality. In real life, however, it is highly
unlikely that all features are actionable and all counterfactuals are
plausible. This was assumed to simplify the study.

##  7 Results

This section presents two main results. The first is to show the PCM method
using CEs from DiCE (Section 7.1). The second is to show the results of this
method on 30 examples from the three causal structures (Section 7.2).

###  7.1 Pearl’s Counterfactual Method (PCM)

####  7.1.1 Experiment 1: Chain causal structure

Following the steps outlined under Section 6, we begin by:

##### Step 1:

generating a dataset by sampling each feature in Table 1 and generating the
data according to the SCMs of Equations 12 to 19. A sample of the data
generated is shown in the first five rows in Figure 11. Note that y is a
numeric value and class is a binary 0 or 1. DiCE only operates on changing
class values so I converted the output y to a class by making any value
greater or equal to the mean value, a one, and anything below the mean value,
a zero. This was the same in all subsequent results.

Figure 11: First five rows of the data generated in step 1.

##### Step 2:

Next, train a logistic regression model on the data which is used in the DiCE
operations. Logistic regression is used because the output is a class. Any
supervised learning model that performs classification can be used here.

##### Step 3:

Select any unit in the dataset that you are interested in, on which to compute
counterfactuals. This unit is presented in Figure 12, together with two CEs
computed by DiCE. Notice the original class was 1 and the counterfactuals
switched the class to 0.

Figure 12: Random unit selected for experiment 1, shown at the top. In this
DiCE computation, two DiCE counterfactual explanations were generated, shown
by the two rows at the bottom.

##### Steps 4 and 5:

Next we carry out Abduction, which is to compute the exogenous variables by
using the actual feature values for this unit shown in Figure 12. Using the
values from the top row in Figure 12, I reproduce Equations 28 to 35 but now
with values inserted to compute the U values which are shown in Table 2.

$\displaystyle 54.5=\boldsymbol{u_{1}}$ (36) $\displaystyle
20.2=\boldsymbol{u_{2}}$ (37) $\displaystyle 18.9=0.6\cdot
53.3+\boldsymbol{u_{3}}$ (38) $\displaystyle 0.0\leavevmode\nobreak\
\leavevmode\nobreak\ =\boldsymbol{u_{4}}$ (39) $\displaystyle
0.0\leavevmode\nobreak\ \leavevmode\nobreak\ =\boldsymbol{u_{5}}$ (40)
$\displaystyle 69.2=\boldsymbol{u_{6}}$ (41) $\displaystyle
53.3=\boldsymbol{u_{7}}$ (42) $\displaystyle 81.7=0.4+0.6\cdot 54.5+0.4\cdot
20.2+0.6\cdot 18.9+0.7\cdot 0+0.4\cdot 0+0.4\cdot 77.0+\boldsymbol{u_{y}}$
(43)

Table 2: Exogenous variables for experiment 1. $u_{1}$ | $u_{2}$ | $u_{3}$ | $u_{4}$ | $u_{5}$ | $u_{6}$ | $u_{7}$ | $u_{y}$
---|---|---|---|---|---|---|---
54.4 | 20.2 | -1.58 | 0 | 0 | 77.0 | 51.4 | -1.58

##### Step 6:

In this example I select counterfactual 0: i.e. the first row of
counterfactuals in Figure 12. Here we can see that two features, $x_{6}=69.2$
and $x_{7}=53.4$ have been changed by DiCE in order to obtain a different
output class of 0. The rest remain unchanged. Therefore when applying Pearl’s
method, Action, if the feature is different, we delete its’ parents and input
the new counterfactual value. This is because when we intervene on a feature,
we now introduce a new mechanism that determines the state of those features,
and it no longer “listens" to it’s parents [2].

We now apply the above (i.e the exogenous variables and the CEs) to the SCMs
for this experiment shown below. Ultimately, we are trying to compute the
output y from the SCM and compare it with the output y from the DiCE CEs.

$\displaystyle x_{1}=u_{1}=54.4$ (44) $\displaystyle x_{2}=u_{2}=20.2$ (45)
$\displaystyle x_{3}=0.6\boldsymbol{\cdot}53.3+(-1.58)$ (46) $\displaystyle
x_{4}=u_{4}=0.0$ (47) $\displaystyle x_{5}=u_{5}=0.0$ (48) $\displaystyle
x_{6}=CE_{6}=69.2$ (49) $\displaystyle x_{7}=CE_{7}=53.3$ (50) $\displaystyle
y=\beta_{0}+\beta_{1}u_{1}+\beta_{2}u_{2}+\beta_{3}x_{3}+\beta_{4}u_{4}+\beta_{5}u_{5}+\beta_{6}CE_{6}+u_{y}=79.08$
(51)

The output y = 79.08 is greater than the mean of 79.07. Therefore when
applying the CEs computed by DiCE to PCM, we obtain a class of 1, whereas DiCE
for this unit, predicted a class of 0.

####  7.1.2 Experiment 2 - Fork causal structure

Following the same steps as above, for the fork causal structure, we generate
the dataset (Step 1), train a classification model (Step 2), select a
unit/case and generate DiCE CEs (Steps 3 and 4, see Figure 13). compute
exogeneous variables (Steps 4 and 5), identify counterfactual values and apply
Action (Step 6), and finally compute the SCM output and compare with DiCE
output (Step 7).

Figure 13 shows the unit that was selected as well as the two DiCE CEs.

Figure 13: Selected unit and two DiCE CEs generated for the fork causal
structure.

Table 3 shows the calculated exogenous variables.

Table 3: Exogenous variables for experiment 2. $u_{1}$ | $u_{2}$ | $u_{3}$ | $u_{4}$ | $u_{5}$ | $u_{6}$ | $u_{7}$ | $u_{y}$
---|---|---|---|---|---|---|---
53.2 | 19.5 | 51.9 | 0 | 0 | 76.0 | 0.84 | 0.84

If we again use $CE_{0}$ in our example, we can see from Figure 13, that
features $x_{1}=34.4$ and $x_{4}=1.0$ are different from the original: they
are counterfactuals. We therefore pay special attention to them and delete
their original causal mechanism and replace it with the new values; the rest
of the features remain unchanged in the SCMs.

$\displaystyle y=\beta_{0}+\beta_{1}\cdot
CE_{1}+\beta_{2}u_{2}+\beta_{3}u_{3}+\beta 4\cdot
CE_{4}+\beta_{5}u_{5}+\beta_{6}u_{6}+u_{y}=91.9$ (52)

The output y = 91.6 is smaller than the mean of 93.6. Therefore when applying
the CEs computed by DiCE to PCM, we obtain a class of 0 which is the same
class predicted by DiCE.

####  7.1.3 Experiment 3 - Collider causal structure

The final example is for the collider causal structure and we again follow the
same six steps as before. We will not repeat the details here, only the
results. Figure 14 shows the unit selected and the DiCE counterfactuals and
Table 4 shows the computed U variables.

Figure 14: Selected unit for Experiment 3 with two DiCE CEs generated for the collider causal structure. Table 4: Exogenous variables for experiment 3. $u_{1}$ | $u_{2}$ | $u_{3}$ | $u_{4}$ | $u_{5}$ | $u_{6}$ | $u_{7}$ | $u_{y}$
---|---|---|---|---|---|---|---
44.8 | 20.4 | 0.43 | 0 | 1 | 76.7 | 44.3 | 0.43

If we again use $CE_{0}$ in this example from Figure 14, we can see that only
features $x_{3}=61.9$ is different from the original. However, due to the
causal structure of the collider, $x_{3}$ is not a parent of y and is
therefore ignored. The output y is therefore calculated as follows:

$\displaystyle
y=\beta_{0}+\beta_{1}u_{1}+\beta_{2}u_{2}+\beta_{4}u_{4}+\beta_{5}u_{5}+\beta_{6}u_{6}+u_{y}=67.0$
(53)

The output y = 67.0 is greater than the mean of 66.84. Therefore when applying
the CEs computed by DiCE to PCM, we obtain a class of 1 which is the same
class predicted by DiCE.

###  7.2 Effect of Causal Structures

Section 7.1 presented the step by step method for computing CEs via DiCE and
then using these values in PCM to compute counterfactual outputs. The aim was
to present the method of using PCM. This section, still using the same method,
aims to obtain an idea of whether different causal structures produce
different counterfactual outcomes using PCM. We do this by performing this
method on thirty DiCE CEs; ten for each causal structure.

####  7.2.1 Chain (mediator)

Table 5 shows the results of selecting 5 units, with DiCE generating two
counterfactual explanations for each unit, totalling ten CEs. Each CE was
input into the chain SCMs and the 6 steps were followed as above. The column
“class" refers to the class of the original unit and the CE classes. The
column “class (PCM)" is the class predicted using Pearl’s counterfactual
method. We can see that three out of the ten times, PCM predicted a different
class than the DiCE method. This is highlighted in gray in the table.

Table 5: Results for chain causal structure Item | $x_{1}$ | $x_{2}$ | $x_{3}$ | $x_{4}$ | $x_{5}$ | $x_{6}$ | $x_{7}$ | y | class | class (PCM) | y (PCM)
---|---|---|---|---|---|---|---|---|---|---|---
Unit 1 | 54.4 | 20.2 | 18.9 | 0 | 0 | 77.0 | 51.3 | 81.7 | 1 | - |
U | 54.4 | 20.2 | -1.58 | 0 | 0 | 77.0 | 51.4 | -1.58 | - | - |
$CE_{0}$ | 54.4 | 20.2 | 18.9 | 0 | 0 | 69.2 | 53.4 | - | 0 | 1 | 79.1
$CE_{1}$ | 39.9 | 20.2 | 18.9 | 1 | 0 | 77.0 | 51.3 | - | 0 | 0 | 77.8
Unit 2 | 52.5 | 18.7 | 15.9 | 0 | 0 | 80.5 | 40.1 | 80.9 | 1 | - |
U | 52.5 | 18.7 | -0.16 | 0 | 0 | 80.5 | 40.1 | -0.16 | - | - |
$CE_{0}$ | 52.5 | 18.7 | 15.9 | 0 | 0 | 73.1 | 40.1 | - | 0 | 0 | 77.5
$CE_{1}$ | 41.4 | 18.7 | 15.9 | 0 | 0 | 70.9 | 40.1 | - | 0 | 0 | 70.4
Unit 3 | 57.5 | 21.3 | 23.2 | 0 | 1 | 61.9 | 58.6 | 82.3 | 1 | - |
U | 57.5 | 21.3 | -0.2 | 0 | 1 | 61.9 | 58.6 | -0.2 | - | - |
$CE_{0}$ | 57.5 | 21.3 | 19.0 | 0 | 1 | 61.9 | 58.6 | - | 0 | 1 | 79.8
$CE_{1}$ | 57.5 | 21.3 | 15.1 | 0 | 0 | 61.9 | 58.6 | - | 0 | 0 | 77.1
Unit 4 | 54.6 | 20.2 | 19.9 | 0 | 0 | 66.2 | 54.1 | 77.9 | 0 | - |
U | 54.6 | 20.2 | -1.7 | 0 | 0 | 66.2 | 54.1 | -1.7 | - | - |
$CE_{0}$ | 62.7 | 20.2 | 19.9 | 0 | 0 | 66.2 | 54.1 | - | 1 | 1 | 82.8
$CE_{1}$ | 62.7 | 18.5 | 19.9 | 0 | 0 | 66.2 | 54.1 | - | 1 | 1 | 82.1
Unit 5 | 50.4 | 20.4 | 25.6 | 0 | 1 | 67.5 | 58.8 | 83.7 | 1 | - |
U | 50.4 | 20.4 | 2.1 | 0 | 1 | 67.5 | 58.8 | 2.1 | - | - |
$CE_{0}$ | 50.4 | 20.4 | 21.7 | 0 | 1 | 67.5 | 58.8 | - | 0 | 1 | 80.9
$CE_{1}$ | 35.3 | 20.4 | 25.6 | 0 | 1 | 67.5 | 58.8 | - | 0 | 0 | 74.2

####  7.2.2 Fork causal structure

Table 6 presents the results for ten CEs using the fork causal structure. The
gray rows show where the PCM method generates the opposite class to DiCE. This
occurred two out of the ten times.

Table 6: Results for fork causal structure.  Item | $x_{1}$ | $x_{2}$ | $x_{3}$ | $x_{4}$ | $x_{5}$ | $x_{6}$ | $x_{7}$ | y | class | class (PCM) | y (PCM)
---|---|---|---|---|---|---|---|---|---|---|---
Unit 1 | 47.5 | 19.53 | 50.6 | 0 | 0 | 68.5 | 19.9 | 94.1 | 0 | - |
U | 47.5 | 19.53 | 50.6 | 0 | 0 | 68.5 | -0.3 | -0.3 | - | - |
$CE_{0}$ | 47.5 | 19.53 | 51.7 | 1 | 0 | 68.5 | 19.9 | - | 1 | 1 | 95.5
$CE_{1}$ | 47.5 | 19.53 | 55.8 | 0 | 0 | 68.5 | 19.9 | - | 1 | 1 | 97.0
Unit 2 | 56.7 | 18.9 | 42.4 | 1 | 0 | 63.9 | 15.9 | 92.6 | 1 | - |
U | 56.7 | 18.9 | 42.4 | 1 | 0 | 63.9 | -1.11 | -1.11 | - | - |
$CE_{0}$ | 51.2 | 20.5 | 42.4 | 1 | 0 | 63.9 | 15.9 | - | 0 | 0 | 89.9
$CE_{1}$ | 62.4 | 18.9 | 28.2 | 1 | 0 | 63.9 | 15.9 | - | 0 | 0 | 87.5
Unit 3 | 52.3 | 18.7 | 43.1 | 1 | 0 | 64.4 | 18.6 | 92.9 | 1 | - |
U | 52.3 | 18.7 | 43.1 | 1 | 0 | 64.4 | 1.36 | 1.36 | - | - |
$CE_{0}$ | 53.5 | 22.1 | 43.1 | 1 | 0 | 64.4 | 18.6 | - | 0 | 1 | 95.0
$CE_{1}$ | 52.3 | 20.3 | 43.1 | 1 | 0 | 64.4 | 18.6 | - | 0 | 0 | 93.6
Unit 4 | 46.6 | 20.2 | 52.9 | 1 | 0 | 72.2 | 20.4 | 96.6 | 1 | - |
U | 46.6 | 20.2 | 52.9 | 1 | 0 | 72.2 | -0.8 | -0.8 | - | - |
$CE_{0}$ | 36.4 | 22.3 | 52.9 | 1 | 0 | 72.2 | 20.4 | - | 0 | 0 | 91.7
$CE_{1}$ | 38.8 | 20.2 | 52.9 | 1 | 0 | 72.2 | 20.4 | - | 0 | 0 | 92.3
Unit 5 | 54.4 | 21.1 | 46.7 | 0 | 0 | 74.9 | 19.9 | 100.7 | 1 | - |
U | 54.4 | 21.1 | 46.7 | 0 | 0 | 74.9 | 19.9 | 2.1 | - | - |
$CE_{0}$ | 36.4 | 21.1 | 46.7 | 0 | 0 | 70.1 | 19.9 | - | 0 | 0 | 87.9
$CE_{1}$ | 46.1 | 21.1 | 46.7 | 0 | 0 | 74.9 | 19.9 | - | 0 | 1 | 95.7

####  7.2.3 Collider causal structure

Table 7 presents the PCM results for ten CEs generated from the collider
causal structure. We can see here that five out of ten (50%) of the DiCE CEs
produced outputs that conflicted with the PCM output classes. These are shown
in gray.

Table 7: Results for collider causal structure Item | $x_{1}$ | $x_{2}$ | $x_{3}$ | $x_{4}$ | $x_{5}$ | $x_{6}$ | $x_{7}$ | y | class | class (PCM) | y (PCM)
---|---|---|---|---|---|---|---|---|---|---|---
Unit 1 | 47.3 | 21.1 | 57.4 | 1 | 0 | 59.7 | 48.9 | 62.3 | 0 | - |
U | 47.3 | 21.1 | 0.48 | 1 | 0 | 59.7 | 48.9 | 0.48 | - | - |
$CE_{0}$ | 47.3 | 21.1 | 57.4 | 1 | 0 | 85.2 | 48.9 | - | 1 | 1 | 72.5
$CE_{1}$ | 47.3 | 20.7 | 69.5 | 1 | 0 | 59.7 | 48.9 | - | 1 | 0 | 62.1
Unit 2 | 46.8 | 19.7 | 58.9 | 0 | 1 | 68.9 | 51.9 | 64.0 | 0 | - |
U | 46.8 | 19.7 | -0.3 | 0 | 1 | 68.9 | 51.9 | -0.3 | - | - |
$CE_{0}$ | 46.8 | 19.7 | 69.7 | 0 | 1 | 68.9 | 51.9 | - | 1 | 0 | 64.05
$CE_{1}$ | 61.4 | 19.7 | 58.9 | 0 | 1 | 68.9 | 51.9 | - | 1 | 1 | 72.7
Unit 3 | 43.3 | 19.4 | 56.7 | 0 | 0 | 75.3 | 51.4 | 62.7 | 0 | - |
U | 43.3 | 19.4 | -1.5 | 0 | 0 | 75.3 | 51.4 | -1.5 | - | - |
$CE_{0}$ | 46.9 | 19.4 | 63.1 | 0 | 0 | 75.3 | 51.4 | - | 1 | 0 | 64.8
$CE_{1}$ | 60.5 | 16.8 | 56.7 | 0 | 0 | 75.3 | 51.4 | - | 1 | 1 | 72.0
Unit 4 | 45.5 | 19.9 | 63.8 | 0 | 1 | 66.3 | 59.7 | 64.1 | 0 | - |
U | 45.5 | 19.9 | 1.5 | 0 | 1 | 66.3 | 59.7 | 1.5 | - | - |
$CE_{0}$ | 45.5 | 19.9 | 63.8 | 0 | 1 | 66.3 | 47.17 | - | 1 | 1 | 73.1
$CE_{1}$ | 45.5 | 19.9 | 63.8 | 0 | 1 | 66.3 | 48.6 | - | 1 | 1 | 73.1
Unit 5 | 43.6 | 20.4 | 63.9 | 0 | 1 | 77.3 | 55.3 | 67.4 | 1 | - |
U | 43.6 | 20.4 | 1.3 | 0 | 1 | 77.3 | 55.3 | 1.3 | - | - |
$CE_{0}$ | 43.6 | 20.4 | 54.8 | 0 | 1 | 77.3 | 55.3 | - | 0 | 1 | 67.4
$CE_{1}$ | 43.6 | 20.4 | 61.5 | 0 | 0 | 77.3 | 55.3 | - | 0 | 1 | 67.4

##  8 Discussion

This section discusses the results and specifically how this method compares
and fits in with current literature.

###  8.1 Pearl’s Counterfactual Method

Verma et al. [5] identified a current challenge in the CE literature being the
lack of using causal models (SCMs) to guide the counterfactual explanations.
Mahajan et al. [8] stated that feasibility (i.e. coming from a possible world)
in CEs is fundamentally a causal concept and that it is important to preserve
causal relationships. That is, if we vary one feature, we have to consider how
other features are causally related to that feature. They give an example
where a CE might recommend increasing education level to masters to obtain a
different desired outcome. According to them, if we change education level,
then the age of the person also needs to change due to being causally linked
with education level. To preserve causal constraints, they proposed to
constrain the distance between features based on an SCM. Riccardo Guidotti [9]
also mentions causality as a requirement for CEs: for a counterfactual
explanation to be plausible and actionable, it should maintain causal
relationships between features. Both of these papers make sense. We want to
use the true causal structure in the data. This is even a main contribution of
this paper. However, when dealing with counterfactuals, there is some nuance.
According to Pearl, when we intervene on a feature and perform do-calculus, we
manipulate and change the original causal structure in the data by deleting
the relationship between that feature and its’ parents. This is slightly
different to what Mahajan et al. [8] and Guidotti [9] are requiring. Indeed,
we want to make use of the true causal structure in the data, however
counterfactual features are no longer constrained by their parents as
described above in Pearl’s Action method (see Section 5.2). Therefore if we
apply Pearl’s method to Mahajan et al’s example, if age is a parent of
education level in the original causal structure, and the CE has recommended
increasing education level, then because we are now intervening on education,
we free it from the influence of its surrounding and it becomes only a
function of our intervention. Age is no longer a cause of education and these
two features are no longer causally constrained. However, this differs
slightly from Mahajan et al. who aim to preserve the causal relationship.

Furthermore, by preserving the causal structure of the counterfactual
variable, are we not simply maintaining the same distribution of data? Recall
that counterfactuals are by nature not in the distribution. Of course, our
features must still be feasible; i.e. they must come from the real world.
However, it seems that maintaining the causal structure between a
counterfactual and its parents keeps the problem on Rung 1 of the Pearl causal
hierarchy: association [2]. A counterfactual by definition is to keep
everything else the same, and only change that feature. If other things are
changing with it according to the original data generation process, then it
seems that it is not a counterfactual. I acknowledge however, that if I have
misunderstood the above mentioned literature, then I am open for correction.

The reader may then ask why am I making such a fuss over using SCMs if it
seems that I don’t care for the constraints in the causal structure. Recall
that it is only the counterfactual feature that is deleted from its’ parents;
the other input features maintain their causal structure. Furthermore, the
causal relationship between the counterfactual feature and the output remains
untouched. Indeed, we must preserve feasibility by keeping the data in the
real world, but we cannot do this by preserving a counterfactual feature’s
causal relationship with its’ parents.

###  8.2 Effect of Causal Structures

The second aim was to see, using PCM, how different causal structures affected
the CEs generated using DiCE. We see that first of all, out of 30 different
results, 10 (33%) showed conflict between the CE output and the PCM output.
Although definitely not conclusive, the results suggest that DiCE CEs produce
incorrect estimates (assuming PCM is correct). For more conclusive results, I
suggest performing a simulation study of 500 to 1000 estimates. The challenge
here was that, at least for now, each PCM calculation was carried out by hand
and therefore very challenging to perform hundreds of times. Future work would
develop code to simulate this type of study.

Another important result was that the collider causal structure
counterfactuals showed 50% conflict with the DiCE CEs. Chain had 3/10
conflicts, fork had 2/10 and collider, 5/10 conflicts. Again, these are not
conclusive results, but they suggest that the type of causal structure may
have a significant effect on the true counterfactual explanations generated.
Whereas ML models simply include all features in the model, the true SCM would
not include all the features. Actually, to include a collider in a statistical
predictive model produces unwanted bias in the causal estimates [7] that can
“sabotage a causal analysis"[12]. These results suggest that simply training a
model on the entire dataset may introduce unwanted error into our
counterfactual estimates. It is vital that we understand the underlying causal
structure before blindly generating CEs.

###  8.3 Causal Discovery

This study used simulated data with known ground truth causal structures. The
reader may argue that this method only works with simulated data and how can
we validate CEs using PCM if we have real life data and don’t know the causal
structure? This is where causal discovery comes in which refers to discovering
the true causal structure in real life data [13]. More explanation of this
method is outside the scope of this study. However, I believe that for us to
have significantly more trust in CEs from DiCE and others, we must first
perform causal discovery on our data in order to determine as best we can, the
true causal structure. We then perform CE validation by feeding the CEs into
the PCM method on the true structure. Real life datasets may be substantially
more complicated that the ones I created in this study. This is left for
future work.

###  8.4 Concluding Remarks and Future Work

In the beginning of the article I argued that it is vital to evaluate
(validate) counterfactual explanations (CEs). A main reason is that CEs are
generated from machine learning models that do not necessarily take into
account the causal structure in the data. This could lead to errors in the
predictions and estimations of counterfactuals. CEs are gaining much interest
in the data community and if we don’t deeply understand how we are generating
CEs, then at best they will add no value.

The results of this study showed that there was some conflict between the ML
method for generating CEs, and Pearl’s method. This casts some doubt on how
much we can trust CEs that are blindly generated by machine learning models.
Different causal structures showed different levels of conflict.

Where do we go from here? It is vital that before we generate CEs, we do our
best to know the true causal structure and modify our models accordingly.
Causal discovery is essential for the future of CEs. Future work would include
carrying out PCM using CEs on real life data with a known causal structure.

I also believe that the ultimate way of validating CEs is to implement them on
real data and measure the true output after a period of time. This is of
course, highly challenging.

For more confidence in these results, it is best to simulate on hundreds of
CEs and also apply it to real life data.

## References

  * [1] Judea Pearl, Madelyn Glymour, and Nicholas P. Jewell.  Causal inference in statistics A primer.  Wiley, 2016.
  * [2] Judea Pearl and Dana Mackenzie.  The book of why.  Basic books, 2018.
  * [3] Christoph Molnar.  Interpretable machine learning: A guide for making black box models explainable.  2019\.
  * [4] Sandra Wachter, Brent Mittelstadt, and Chris Russel.  Counterfactual explanations without opening the black box: Automated decisions and the GDPR.  arXiv preprint arXiv:1711.00399, 2017.
  * [5] Sahil Verma, John P. Dickerson, and Keegan Hines.  Counterfactual explanations for machine learning: Challenges revisited.  arXiv:2106.07756v1, 2021.
  * [6] Ramaravind K. Mothilal, Amit Sharma, and Chenhao Tan.  Explaining machine learning classifiers through diverse counterfactual explanations.  Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 2020.
  * [7] M. Luque-Fernandez, M.A.and Schomaker, D. Redondo-Sanchez, M. Jose Sanchez Perez, A. Vaidya, and M.E. Schnitzer.  Educational Note: Paradoxical collider effect in the analysis of non-communicable disease epidemiological data: a reproducible illustration and web application.  Int J Epidemiol., 48:640–653, 2019.
  * [8] Divyat Mahajan, Chenhao Tan, and Amit Sharma.  Preserving causal constraints in counterfactual explanations for machine learning classifiers.  arXiv:1912.03277v3, 2020.
  * [9] Riccardo Guidotti.  Counterfactual explanations and how to find them: literature review and benchmarking.  Data Mining and Knowledge Discovery, 2022.
  * [10] Bernhard Schölkopf.  Causality for machine learning.  arXiv:1911.10500v2, 2019.
  * [11] Peter Tennant.  Introduction to causal inference and directed acyclic graphs, 2022.
  * [12] Stephen L. Morgan and Christopher Winship.  Counterfactuals and causal inference: Methods and principles for social research.  Cambridge University Press, 2007.
  * [13] Clark Glymour, Kun Zhang, and Peter Spirtes.  Review of causal discovery methods based on graphical models.  Front. Genet. 10:524.doi: 10.3389/fgene.2019.00524, 2019.