# Attributing Responsibility in AI-Induced Incidents: A Computational
Reflective Equilibrium Framework for Accountability

Yunfei Ge and Quanyan Zhu

###### Abstract

The pervasive integration of Artificial Intelligence (AI) has introduced
complex challenges in the responsibility and accountability in the event of
incidents involving AI-enabled systems. The interconnectivity of these
systems, ethical concerns of AI-induced incidents, coupled with uncertainties
in AI technology and the absence of corresponding regulations, have made
traditional responsibility attribution challenging. To this end, this work
proposes a Computational Reflective Equilibrium (CRE) approach to establish a
coherent and ethically acceptable responsibility attribution framework for all
stakeholders. The computational approach provides a structured analysis that
overcomes the limitations of conceptual approaches in dealing with dynamic and
multifaceted scenarios, showcasing the framework’s explainability, coherence,
and adaptivity properties in the responsibility attribution process. We
examine the pivotal role of the initial activation level associated with
claims in equilibrium computation. Using an AI-assisted medical decision-
support system as a case study, we illustrate how different initializations
lead to diverse responsibility distributions. The framework offers valuable
insights into accountability in AI-induced incidents, facilitating the
development of a sustainable and resilient system through continuous
monitoring, revision, and reflection.

###### Index Terms:

Responsibility Attribution, Accountability, Reflective Equilibrium, AI Ethics

##  I Introduction

In today’s context, Artificial Intelligence (AI) has been integrated into
systems with critical functionalities. Cyber-physical systems, such as smart
health systems, cloud-enabled critical infrastructure, and connected vehicles,
are quintessential examples. The wide adoption of AI, however, introduces a
range of issues, encompassing safety concerns in autonomous driving, the
reliability of AI-driven medical diagnosis and treatment, and the unforeseen
outcomes arising from medical recommendation systems. The recent White House
executive order [1] places the highest priority on governing the development
and use of AI in a safe and responsible manner. Nonetheless, as an emerging
technology with a black-box nature, AI introduces uncertainties that cannot be
easily identified, assessed, and attributed by traditional approaches.
Concerns arise due to the absence of mechanisms for responsibility attribution
in incidents caused by AI systems. The lack of accountability can result in
hesitancy in the pervasive adoption of AI, thereby limiting the growth and the
impact of the technology. To this end, there is a need to establish an
accountability framework for AI-enabled systems, facilitating the future
development of safe, secure, and trustworthy AI.

Accountability involves recognizing and accepting responsibility for the
potential outcomes of one’s actions. In the context of AI-related incidents,
it revolves around identifying responsibility for failures in AI systems. For
example, AI-enabled medical decision-support systems rely on AI algorithms to
provide medical diagnosis and recommendations. Typically, a physician
administers a medical test to a patient, and the results are input into the
system. The doctor, relying on the AI’s suggested results, prescribes
medication to the patient. If medical incidents occur due to the patient
following the doctor’s prescription, various parties could be accountable.
Possible points of accountability include the physician’s actions during the
operation, potential errors in AI development, the doctor’s dependence on AI
recommendations, and the patient’s adherence to the treatment plan.
Determining how to allocate responsibility among all involved parties proves
to be a challenging task.

Figure 1: General process of reflective equilibrium: The process involves
formulating a set of moral intuitions, applying evidence and moral principles
to support the claim, and iteratively revising and reflecting on intuitions to
establish a coherent equilibrium.

Attributing AI responsibility has the following challenges. The first
challenge arises from the interconnected and complex nature of AI systems,
where inherent vulnerabilities in each component can contribute to potential
malfunctions, making responsibility attribution to individual components a
complex task [2]. The second challenge involves uncertainties in AI, both in
the technology itself and in regulations governing AI. The AI system provides
a diagnosis, but the reasoning behind the recommendation is not transparent or
explainable, creating difficulties in determining accountability without clear
examination tools. Additionally, there is a lack of legal regulations and
guidelines outlining expected conduct, responsibilities, and ethical
principles for practitioners when utilizing AI in their practice. Furthermore,
new challenges also arise from ethical constraints. One example is the
allocation of limited medical resources by an AI decision-support system,
prioritizing certain groups of patients over others. Determining
accountability becomes complex when such decisions result in inadequate
medical resources for underprivileged patients.

To address these inherent challenges, we seek to establish a coherent
responsibility attribution framework among stakeholders supported by justified
beliefs. Drawing inspirations from moral and political philosophy,
particularly associated with the work of John Rawls [3, 4, 5], we employ
Reflective Equilibrium (RE) as our approach. RE begins with moral intuitions
or judgments regarding specific cases or general principles, representing
immediate and unreflective responses to given situations. Subsequently,
evidence and theoretical principles from philosophical frameworks or other
theories are introduced. A comparison is then made between these intuitions
and the theoretical principles, whether new or existing. In the presence of
inconsistencies or conflicts, adjustments are made to establish a coherent and
balanced set of principles and intuitions that mutually support each other.
The iterative process, as illustrated in Fig. 1, eventually leads us to
consistency in ethical responsibility attribution. In the context of AI-
induced incidents, many inconsistencies arise due to the absence of applicable
theory conflicting with human intuition.

Applying RE to AI-induced incidents, we first establish a framework utilizing
reflective equilibrium to assess and assign responsibility to achieve a
coherent and ethically acceptable equilibrium for all stakeholders. Initially,
we formally state all possible responsibility attribution principles for AI-
enabled systems involving multiple parties. Subsequently, we apply evidence
and moral principles to support the responsibility attribution accordingly. We
refer to the intuitions and principles as claims. To find the equilibrium, we
formulate the problem as a constraint satisfaction problem using coherence
theory and employ a computational approach, defining our solution as
Computational Reflective Equilibrium (CRE). This computational framework
provides a structured analysis that overcomes the limitations of conceptual
approaches in dealing with dynamic and multifaceted scenarios. To enhance
results, we incorporate hypothesis testing and gather feedback from those
involved in or contributing to accountability measures to determine the
initial activation level for each claim. We showcase that different
initializations would lead to diverse responsibility distributions in the
equilibrium.

The computational approach aims to achieve a coherent and ethically
justifiable equilibrium that minimizes conflicts and maximizes support,
achieving consistency among the stakeholders. By employing computation, our
framework not only enhances clarity in ethical reasoning but also demonstrates
adaptability to diverse ethical contexts, promoting transparency and coherence
in responsibility attribution. Computational Reflective Equilibrium (CRE)
facilitates a dynamic balance among conflicting ethical principles,
obligations, and evidence, offering context-sensitive solutions. To ensure
transparency, it is imperative to communicate the principles and
accountability standards to all stakeholders. Moreover, continuous monitoring
and revisiting the equilibrium for necessary adjustments in case of
discrepancies are also crucial components of the process. We use a medical
decision-support system as a case study to illustrate the framework and
provide computational results to highlight its properties.

The remainder of the paper is organized as follows. In Section II, we delve
into the related work within the field. Section III outlines the computational
reflective equilibrium framework. In Section IV, we discuss the properties of
the proposed model. The performance of our model through simulations is
presented in Section V. Lastly, we draw our conclusions in Section VI.

##  II Related Work

The concept of responsibility has several connotations. In this study, our
focus is on backward-looking responsibilities, specifically understanding
responsibility as blameworthiness [6]. A primary challenge in assigning
responsibility in AI-induced incidents lies in identifying the morally
responsible agent. While humans are traditionally associated with agency and
accountability for their actions, it remains unclear whether AI technologies
can be considered responsible agents themselves. Extensive discussions about
the moral agency of AI have taken place in the ethics of computing and robot
ethics literature [7, 8, 9, 10]. Depending on the moral theory adopted, one
may choose to include or exclude AI as the responsible agent. Currently, there
is no legal regulation treating AI as a responsible agent, but as technology
evolves, future perspectives may emerge [11]. In our framework, we can
accommodate both arguments as long as they are supported by the justified
belief system in the end.

Once we identify the responsible agents, another challenge arises in how to
attribute responsibilities among these agents, a problem often referred to as
the “many hands” problem [12]. Several works have focused on addressing this
challenge. Duch et al. [13] applied weighted votes to distribute
responsibility among multiple collective decision-makers, but this approach
may not be applicable to technologies like AI, as we cannot collect their
votes. Taddeo et al. [14] proposed the conceptual idea of distributed
responsibility to address responsibility attribution in the AI context;
however, practical solutions for its implementation are lacking. Other
detailed research has focused on determining “who did what at when”, seeking
to establish causality and the role of each agent to attribute responsibility
on a case-by-case basis [15, 16]. While there is no definitive correct answer
to responsibility attribution, what is notably missing is a general model
addressing how to attribute responsibility in AI, supported by reasoning and
justified outcomes, preferably in a computationally supported manner.

In applied ethics, the reflective equilibrium (RE) model has been proposed as
an approach to reconcile a pluralism of ethical views. RE enables decision-
makers to achieve coherence and consistency in ethical reasoning by balancing
and refining moral intuitions, principles, and theories [4]. It has
demonstrated success in various areas, including justice [3], moral decision-
making [17], and public reasoning [18]. While Doorn [19] explored
responsibility rationales in research and development using RE, their
contribution was primarily descriptive. Yilmaz et al. developed a domain-
specific language to establish coherence-governed models for ethical decision-
making. Building on the literature of RE and its associated AI frameworks, we
aim to propose a computational methodology to support ethical AI reasoning and
coherent responsibility attribution.

##  III Computational Reflective Equilibrium to Accountability

Figure 2: Workflow of computational reflective equilibrium approach to
responsibility attribution.

In this section, we describe the computational reflective equilibrium
computation (CRE) process for achieving coherent and ethical responsibility
attribution in AI-induced incidents. The basic workflow is depicted in Fig. 2.
We focus on the example of a medical decision-support system for diagnosis and
treatment to elaborate the proposed approach.

###  III-A Stage $1$: Identify Responsible Agents

In the initial stage, we formulate hypotheses for accountability attribution
that align with our intuitions and identify all potential responsible agents
involved in the AI incident [20]. This process includes recognizing all
relevant parties that can contribute to the final consequences. As discussed
earlier, there is deliberation on whether to include AI itself as a moral
agent. We assert that such inclusion is justifiable as long as it is supported
by theories and evidence. This highlights the adaptivity of our framework,
which accommodates diverse claims as initial input from various perspectives.
It is essential to note that we should also consider opposing claims where the
agent is not responsible. This comprehensive approach ensures a balanced
consideration, incorporating both affirmative and negative perspectives for a
thorough analysis of responsibility attribution.

For illustrative purposes, we consider a scenario in which the AI decision-
support system generates a diagnosis, and the doctor adopts a treatment plan
based on the recommendation, ultimately resulting in an incorrect treatment
for the patient. Our focus is specifically on the interaction between the AI
system and the doctor, excluding other factors such as the manufacturer
producing the product, the physician collecting data, or the patient failing
to adhere to the prescription. For a more comprehensive analysis, all these
factors can be incorporated into the graph. In this instance, we can outline
the following initial claims regarding responsibility attribution, as depicted
in Fig. 3.

Figure 3: Initial claims about responsibility attribution in AI medical
decision-support systems. Solid lines indicate supporting claims, while dotted
lines represent conflicting claims.

###  III-B Stage $2$: Find Supporting Claims

After identifying the initial claims, the next step involves formulating an
initial set of theoretical principles or relevant facts that generalize or
support these initial claims. For each initial claim, we seek supportive
claims in the following directions [21]:

  1. 1.

Relative Facts: This entails identifying the domain knowledge or evidence that
supports the initial claim. Such supportive claims can often be quantified
through evaluations like hypothesis testing.

  2. 2.

Moral Concerns and Theories: This direction involves finding the generalized
moral theory to ethically support the claim.

  3. 3.

Analogy: Drawing parallels with similar cases in other areas serves as an
analogy to support the claim. For example, referring to robot-induced
incidents can provide insights into AI-induced incidents as a support factor.

  4. 4.

Opposite opinions: In addition to identifying supportive claims, it is crucial
to list possible opposing opinions to these claims, forming a more
comprehensive analysis.

By considering these claims, we can construct a coherent explanation of
accountability that is acceptable and explainable. The details about the
supportive claims we found for the AI medical decision-support system case can
be found in Fig. 4.

It is important to highlight that the claims, including initial claims and
their supportive claims, should not be arbitrary but rather directly or
analogically related to the relevant domain. Holding any unrelated party
accountable in the case of AI-induced incidents would be unreasonable.
Moreover, several philosophers have suggested that RE is best interpreted as a
hybrid of foundationalism and coherentism [20]. This suggests that we should
adhere to the following constraints for the claims.

######  Definition 1 (Claim Constraints).

The claims for computational reflective equilibrium should adhere to the
following conditions:

  1. 1.

The claim is directly or analogically related to the relevant domain,

  2. 2.

The claim must already be justified to some degree,

  3. 3.

The justification is non-inferential, based directly on the mere fact of the
subject’s believing or intuiting as they do.

###  III-C Stage $3$: Construct Constraint Network

In the third stage, we attempt to formulate the constraint network that
describes the relations among all claims. The network can be represented as a
graph $G=(V,E)$, where $V$ represents a finite set of claims and $E\subseteq
V\times V$ represents the set of relations between two claims. These relations
could be supports or conflicts. The support between the two components
strengthens one’s belief in certain claims. However, conflicts are likely to
arise between one’s initial claims and the supportive claims that aim to
account for those claims. Furthermore, conflicts may arise within the initial
claims or within the supportive claims, reflecting the challenge of
determining accountability in complex systems like AI.

Figure 4: Constraint network for responsibility attribution in the AI medical
decision-support system case. Solid lines indicate supporting claims with
positive constraints, while dotted lines represent conflicting claims with
negative constraints.

Inspired by the notion of constraint satisfaction studied in computer science,
we can formalize support (coherence) and conflict (incoherence) relations as
positive and negative constraints, respectively [22].

######  Definition 2 (Satisfaction conditions).

The coherence conditions for the positive and negative constraints are:

  1. (C1)

A positive constraint $(u,v)\in C^{+}$ is satisfied if and only if the
elements $u$ and $v$ are both rejected or both accepted;

  2. (C2)

A negative constraint $(u,v)\in C^{-}$ is satisfied if and only if the element
$u$ is accepted and $v$ is rejected, or conversely.

In our AI medical decision-support case, the full constraint network with
initial responsibility attribution claims and supportive claims are
illustrated in Fig. 4, where solid lines indicate supporting claims with
positive constraint and dotted lines represent conflicting claims with
negative constraint.

###  III-D Stage $4$: Coherence-driven Computation

Once we establish these positive and negative constraints, our goal is to
identify a set of claims that exhibit coherence, forming our responsibility
attribution result with supporting explanations. The process of computing
reflective equilibrium can be understood as a coherence problem, seeking to
satisfy as many constraints as possible. Formally, we can define the coherence
problem for finding RE as follows.

######  Definition 3 (Coherence problem).

Given a graph $G=(V,E)$ with vertex set $V$ represents a finite set of
elements (belief, propositions, evidence, etc.) and edge set $E\subseteq
V\times V$ that partitions into positive constraints $C^{+}$ and negative
constraints $C^{-}$, i.e., $C^{+}\cup C^{-}=E$ and $C^{+}\cap
C^{-}=\emptyset$. Each constraint $(u,v)\in E$ is associated with a weight
$w(u,v)\in\mathbb{R}$ that indicates the weight of the constraint. The
coherence problem aims to partition the vertices $V$ into two sets, $A$
(accepted) and $R$ (rejected) such that the sum of the weights of the
satisfied, given by:

$\displaystyle\max_{A\cup R=V}\quad
W(A,R)=\sum\limits_{\begin{subarray}{c}(u,v)\in C^{+}\\\ \text{satisfy
C1}\end{subarray}}w(u,v)+\sum\limits_{\begin{subarray}{c}(u,v)\in C^{-}\\\
\text{satisfy C2}\end{subarray}}w(u,v).$ (1)

The challenge of coherence is recognized as NP-hard; however, practical
approximations can be reliably employed. One computational approach to
approximating coherence involves leveraging connectionist algorithms, such as
neural networks. In neural science, this model aligns with the problem of
maximizing harmony in Hopfield networks [23]. Under this interpretation, the
focus shifts from partition identification to assigning each element an
activation level $a:V\mapsto[-1,1]$ in a manner that maximizes the subsequent
function:

$\displaystyle\max_{a}H(a)=\sum_{u,v\in V}a(u)a(v)w(u,v).$ (2)

######  Lemma 1.

Given a constraint network $G=(V,E)$ and weight function
$w:E\mapsto\mathbb{R}$, there exists an activation level
$a^{*}:V\mapsto\\{+1,-1\\}$ that maximizes $H(a)$.

The detailed proof of the lemma can be found in [23]. The lemma suggests that
at least one partition can be identified to achieve reflective equilibrium.
It’s worth noting that the reached equilibrium offers the most fitting
explanation only for the particular context and situation. Individuals with
different preferences may reach different equilibria, each guided by distinct
principles and judgments.

Algorithm 1 outlines the procedure for updating the activation levels using
connectionist approximation. In this algorithm,
$net^{t}_{u}=\sum_{v}\hat{w}(v,u)a^{t}(u)$ represents the net input to element
$u\in V$. If $(v,u)\in C^{+}$, we assign $\hat{w}(v,u)=w(v,u)$; else if
$(v,u)\in C^{-}$, we let $\hat{w}(v,u)=-w(v,u)$. The parameter
$\gamma\in(0,1)$ introduces a decay parameter, reducing the activation level
of each element in each iteration. The values of $M$ and $m$, usually set to
$M=1$ and $m=-1$, define the maximum and minimum activation ranges. The
algorithm produces a decision for claims $u\in V$ where $a^{*}(u)>0$,
indicating acceptance of the claim; otherwise, it is rejected. The accepted
claims constitute a coherent and explainable set that aligns closely with
ethical considerations. To determine the final responsibility attribution, we
examine the accepted sets to identify which involved party should be
accountable, along with the supportive claims that bolster the assignment.

Input: Constraint graph $G=(V,E)$, positive constraint set $C^{+}$, negative
constraint set $C^{-}$, weight function $w:E\mapsto\mathbb{R}$, decay
parameter $\gamma\in(0,1).$

1 Initialization: assign $a^{0}(u)$ for each element $u\in V$ ;

2 repeat

3 Update the activation of all the elements parallel:  $\displaystyle
a^{t+1}(u)=$ $\displaystyle a^{t}(u)(1-\gamma)$
$\displaystyle+\begin{cases}net^{t}_{u}\cdot(M-a^{t}(u))&\text{if
}net^{t}_{u}>0,\\\ net^{t}_{u}\cdot(a^{t}(u)-m)&\text{otherwise.}\end{cases}$

4until  _activation levels reach equilibrium_ ;

Result: If $a^{*}(u)>0$, the element $u$ is accepted. Otherwise, $u$ is
rejected.

Algorithm 1 Computational Reflective Equilibrium Algorithm

### Initial Activation Level

As mentioned earlier, the equilibrium is influenced by individuals’ initial
beliefs in the given claim. In this context, we argue that the initial
activation level of each claim, $a^{0}(u)$ for all $u\in V$, effectively
represents the initial belief associated with that specific claim. The
following properties in cognitive science provide insights into the effect of
the initial activation level.

  * •

The Rich Get Richer Effect [24]: Elements $u\in V$ that possess slight initial
advantages, be it in terms of their external inputs or initial activation
values, tend to magnify this advantage when compared to their competitors,
specifically those denoted as $v\in V$ with a negative constraint $(u,v)\in
C^{-}$.

  * •

Resonance Effect [25]: If element $u\in V$ and $v\in V$ have mutually
excitatory connections (positive constraints $(u,v)\in C^{+}$), then once one
of the elements becomes active, they will tend to keep each other active.

Building upon these effects, we present the following proposition.

######  Proposition 1.

For an input claim $u\in V$ with higher initial activation levels $a^{0}(u)$,
they possess greater priority and a higher likelihood of sustaining activation
throughout the equilibrium, denoted as $a^{*}(u)=+1$.

To quantitatively assess the initial activation levels of each input claim, we
can categorize the claims into two sets. The first set comprises ethical,
theoretical, or regulatory claims that are either unquantifiable or
untestable, and the second set includes claims that can be evaluated through
scientific testing or evidence gathering. We present the quantitative
investigation methods for these two types of claims in the following sections.

#### Public Preference

Diverse individuals may possess different intuitions or perceptions regarding
a single ethical claim. For example, there may be one group of people who
strongly believe that AI algorithms provide misinformation to doctors, leading
them to argue that AI developers should be held accountable. Conversely,
another group may hold the belief that holding developers accountable could
hinder their exploration in scientific research, subsequently delaying the
development of AI systems and their potential benefits. It is hard to quantify
the claims using numerical values, as we cannot find a method to
quantitatively evaluate the importance of the claims.

For this category of claims, the assessment of initial activation levels
relies on subjective judgments. Individuals have the flexibility to initialize
values based on their personal judgments, where $a^{0}(u)=-1$ indicates strong
disagreement, $a^{0}(u)=0$ denotes neutrality, and $a^{0}(u)=1$ signifies
strong agreement. In this work, we consider public preferences to determine
the initial activation level for this type of claim.

######  Proposition 2.

Given the claim $u\in V$, Let $f_{u}(x)$ be the probability distribution
function representing the distribution of preferences $x\in[-1,1]$ for claim
$u$ across the population. The initial activation level of claim $u$ can be
represented as

$\displaystyle a^{0}(u)=\int_{-}1^{1}xf_{u}(x)dx.$ (3)

The activation value can be derived by assessing people’s acceptance of a
specific claim through surveys, seeking expert opinions, referring to
regulatory suggestions, or consulting academic research. The integration of
public preference aims to translate the intuitions and beliefs of individuals
into practical data, thereby contributing to the balancing process. This
approach aligns with the framework in medical ethics [26], emphasizing
autonomy, beneficence, nonmaleficence, and justice principles, all of which
are closely related to the well-being and perspectives of the people involved.
However, it is worth noting that the quantitative survey should have a
sufficient sample size to draw a conclusion and should be distributed to each
population fairly to eliminate potential bias. Otherwise, the results could
incline towards particularity rather than universality [27].

#### Quantitative Investigation

For claims of the second type, their initial activation values can be
quantified through evaluations. The key lies in investigating whether
empirical evidence supports the claim. One appropriate approach for
determining these values is through hypothesis testing. In line with the
framework outlined in [2], we employ hypothesis testing to illustrate the
computation of initial activation levels for individual claims. It’s worth
noting that alternative evaluation methods are also acceptable.

Consider a claim in the constraint network that states that “the developed
algorithm has no error”. Assume the claim has an authentic binary type
$\Theta=\\{0,1\\}$, where $\theta=1$ means the statement is true and
$\theta=0$ means the statement is false. Since the algorithm performance
cannot be directly accessed but only can be tested through operations, we
consider the following. The responsible party of this statement (algorithm
developer) sends a message to the public stating whether the statement is true
or not, denoting $m\in\mathcal{M}=\\{0,1\\}$. The doctor decides how to
incorporate the algorithm into medical practices, denoted as an operation
function $\delta:\mathcal{M}\mapsto\mathcal{D}$. The final performance follows
a distribution $p_{y}(y;\theta,\delta(m))$. Consider a sequence of repeated
but independent investigation observations
$Y^{k}=\\{y_{1},y_{2},\cdots,y_{k}\\}$, $k\in\mathbb{N}$. Hypothesis $H_{0}$
is set to be the case when the observations follow the anticipated performance
given the received message $m$ and $H_{1}$ otherwise. Depending on whether
$H_{0}$ or $H_{1}$ holds, each observation $y_{i}$ follows the respective
distribution:

$\displaystyle H_{0}:\,y_{i}\sim f_{m}(y|H_{0})=p_{y}(y;\theta=m,\delta(m)),$
(4) $\displaystyle H_{1}:\,y_{i}\sim f_{m}(y|H_{1})=p_{y}(y;\theta=\neg
m,\delta(m)).$ (5)

The optimal Bayesian investigation rule is based on the likelihood ratio test
(LRT). LRT provides the decision rule that $H_{1}$ is established when
$L(Y^{k})$ exceeds a defined threshold value $\tau\in\mathbb{R}$; otherwise,
$H_{0}$ is established. It can be formulated by

$\displaystyle L(Y^{k})=\prod_{j=1}^{k}\frac{p_{y}(y_{j};\delta(m)|\theta=\neg
m)p_{\theta}(\theta=\neg
m)}{p_{y}(y_{j};\delta(m)|\theta=m)p_{\theta}(\theta=m)}\mathop{\gtreqless}_{H_{0}}^{H_{1}}\tau.$
(6)

where $p_{\theta}$ is the prior distribution of the claim, which indicates the
prior probability that the developed algorithm has an error or not. Assume the
investigation cost is symmetric and incurred only when an error occurs. In the
binary case, the optimum decision rule will consequently minimize the error
probability, and the threshold value $\tau$ in LRT (6) will reduce to
$\tau=\Pr(H_{0})/\Pr(H_{1})$, where $\Pr(H_{i})_{i=0,1}$ is the prior
distribution of the hypothesis, indicating the reputation of the developer.

######  Definition 4 (Claim Authenticity).

For a given claim $u\in V$, given an investigation rule, i.e., the threshold
$\tau_{u}$, claim authenticity $P_{A}\in[0,1]$ is defined as the probability
of correct establishment of hypothesis $H_{1}$ based on the observations
$Y^{k}$ and message $m$, which is given by

$\displaystyle
P_{A}(\tau_{u})=\int_{\mathcal{Y}_{1}}f_{m}(Y^{k}|H_{1})dy^{k},$ (7)

where $\mathcal{Y}_{1}$ is the observation space in which
$\mathcal{Y}_{1}=\\{Y^{k}:L(Y^{k})\geq\tau_{u}\\}$.

Input: Original claim $u$, received message $m$, operation function
$\delta(m)$, and reputation regarding the claim $\pi_{u}$;

1 Establish hypotheses $H_{0}$ and $H_{1}$ based on received message $m$;

2 Determine the investigation rule $\tau_{u}$ in LRT based on the reputation
$\pi_{u}$;

3 Perform $k$ times of i.i.d. investigation tests and record the observations
$Y^{k}$;

4 Compute the claim authenticity according to (7) ;

Result: Claim Authenticity $P_{A}(\tau_{u})$.

Algorithm 2 Claim Authenticity Investigation Procedure

The claim authenticity Investigation investigation procedure is summarized in
Algorithm 2. $P_{A}(\tau_{u})$ indicates the probability that the claim holds
within the constraint network. This value is a probability that lies within
the range $P_{A}(\tau_{u})\in[0,1]$. To effectively represent the initial
activation level, it is necessary to normalize the outcomes of the claim
authenticity assessment to fit within the initial activation level region
$a^{0}(u)\in[-1,1]$.

######  Proposition 3.

Given the claim authenticity investigation result $P_{A}(\tau_{u})\in[0,1]$ of
a claim $u\in V$, the initial activation level of the element is

$\displaystyle a^{0}(u)=2P_{A}(\tau_{u})-1.$ (8)

The individual claim investigation process integrates one’s prior beliefs
about the claim with the testing performance acquired through investigative
efforts. This combination of beliefs and observations aims to provide a more
comprehensive interpretation, thereby guiding the allocation of accountability
to the relevant element.

###  III-E Result and Revision

After determining the initial activation levels for all input claims, the
reflective equilibrium is computed using the approximation algorithm outlined
in Algorithm 1. The algorithm returns a set of coherent and mutually
supporting claims. By inspecting this set of accepted claims, we can identify
which of the initial responsibility claims are accepted. Based on these
accepted claims, responsibility attribution under CRE is determined. This
attribution may involve one or multiple parties responsible for the AI-induced
incidents. It is important to note that CRE provides responsibility
attribution in the sense of identifying who should be responsible, but it does
not quantify the extent of their responsibility.

It is important to recognize that the CRE is a dynamic process requiring
ongoing reflection and revision. This need arises for various reasons.
Firstly, public preferences towards certain ethical claims may vary based on
their perception of the claim under specific circumstances, leading to
potential changes in the initial activation levels. Additionally, as AI
technology advances, new claims or estimation methods may emerge, allowing for
more informed computation based on additional evidence. Moreover, regulations
or supporting principles may evolve over time. All these considerations
emphasize that the CRE obtained is coherent and optimal only for the current
moment. Therefore, it is essential to periodically revisit the initial stage
and repeat the process whenever the current responsibility attribution appears
inappropriate or requires adjustment.

##  IV Properties

In this section, we present several crucial properties of the CRE framework
for responsibility attribution in AI-induced incidents.

###  IV-A Explainability

The responsibility attribution derived from CRE is explainable due to the
supportive principles embedded within the acceptance set. The initial claims
offer an intuition about potential responsibility attribution, while the
supportive claims further reinforce these assertions. They bring in
principles, knowledge, evidence, analogies, and ethical considerations to
explain several aspects: what explains the claim, what the claim signifies,
similar cases that help clarify it, and why opposite opinions cannot hold.
This combined explanation helps illustrate why certain decisions are made,
drawing from ethical considerations to scientific evidence to provide a
comprehensive understanding. This explainability not only adds transparency to
the decision-making process but also promotes the trustworthiness of AI
systems in the public.

###  IV-B Coherence

CRE is an equilibrium seeking to satisfy the maximum number of constraints
while minimizing conflicts within the final decisions, aiming to establish a
coherent system of beliefs that people would find acceptable. It is important
to note that the result is not necessarily truth-conducive; instead, it tries
to acquire justified beliefs under ethical constraints. This coherence goes
beyond scientific proof or evidence, as it also takes into consideration
people’s preferences and acceptances towards a certain claim. Ultimately, the
final judgment is made by humans, and accountability serves as a means to
ensure satisfaction with the results, providing a balanced and coherent
framework that resonates with the human perspective.

###  IV-C Adaptivity

The adaptivity of CRE is evident in its continuous revision and reflection
process. When conditions change, the responsibility attribution can be
promptly adjusted to align with the current situation. CRE is not a one-time
iteration but an evolving procedure that encourages adaptation to challenges
and diverse perspectives. While objections against RE in philosophy highlight
its potential arbitrariness based on initial beliefs or intuitions, this
characteristic proves advantageous in the context of responsibility
attribution. Given the dynamic nature of situations, the ability to adapt
equilibrium along the way is crucial. A fixed accountability framework could
be exploited by malicious actors and would ultimately lack resonance with
people’s evolving understanding of the subject.

TABLE I: Baseline Parameter Settings. The claim abbreviations correspond to those in Fig. 4. Claim | AGS | AGNS | AIDR | AIDNR | DR | DNR | AIR | AINR | NR | DE | OM | DJW | SLOW | UT | UNFAIR
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
$a^{0}(u)$ | 0.2 | 0.1 | 0.01 | -0.01 | 0.01 | -0.01 | -0.2 | 0.1 | -0.2 | 0.1 | 0.01 | 0.2 | 0.01 | 0.01 | 0.01
Claim | INFO | BETTER | OIC | PRO | NON | OWN | RIGHT | AIM | ATT | AINM | LACK | SET | FIND | UBER | PRAC
$a^{0}(u)$ | 0.3 | 0.3 | 0.01 | 0.5 | 0.7 | 0.01 | 0.3 | -0.3 | 0 | 0.3 | 0.5 | -0.2 | 0 | 0.3 | 0.6

##  V Case Study

In this section, we use the medical decision-support system illustrated in
Fig. 4 as a case study to discuss responsibility attribution under different
situations. We evaluate the performance of our CRE model using numerical
experiments, implemented in a self-built Python simulator. The weight of each
edge is set to $w(u,v)=1,\forall(u,v)\in E$, and the decay parameter $\gamma$
is set to be $0.05$. For the base case value, the initial activation levels
are listed in Table I. The claim abbreviations correspond to those in Fig. 4.

###  V-A Case $1$: Algorithm Design Error

Consider a scenario where quantitative investigation reveals a significant
design error in the algorithm. The claim authenticity for the algorithm design
error (DE) is assessed to be $P_{A}(u)=0.9$. In the constraint network, we
initialize the activation level of this claim as
$a^{0}(\text{DE})=2P_{A}(u)-1=0.8$. Additionally, we assume that the society
does not strongly agree with the idea that AI could be a moral agent, setting
$a^{0}(\text{AIM})=-0.3$ and $a^{0}(\text{AINM})=0.3$

(a) Initial

(b) Equilibrium

Figure 5: Constraint network illustrating algorithm design errors. The node
positions correspond to those in Fig. 4. Nodes colored in red represent
accepted claims; any initial responsibility claim falling within this set is
accepted for responsibility attribution. Nodes in blue represent rejected
claims.

Fig. 5 illustrate the initial claim status (Fig. 5a)and the claim status at
the equilibrium (Fig. 5b). The evidence suggesting algorithm design error
enhances the probability that the algorithm developer should be responsible
for the incident, resulting in the rejection of the opposing claim that the
algorithm developer should not be responsible. Furthermore, the analogy cases
and ethical concerns indicate that the doctor should also share responsibility
for the patient, even with the algorithm design error, as the doctor holds
professional responsibility and should exercise careful judgment if the AI is
wrong. Since we initialize the idea with the belief that AI should not be a
moral agent, AI itself is not attributed responsibility in this case. Finally,
the equilibrium suggests that people believe accountability should be assigned
to individuals involved in the incident, rather than expecting society to bear
the loss caused by such an AI system.

###  V-B Case $2$: Doctor Malpractice

Next, we consider the scenario where the doctor has proved to have operational
malpractice through investigation. In this case, we keep the base initial
activation value in Table I but set the initial activation level of the claim
operational malpractice to $a^{0}(\text{OM})=0.6$ and the claim doctor’s
judgment is wrong to $a^{0}(\text{DJW})=0.2$.

Figure 6: Constraint network with doctor operational malpractice at
equilibrium.

The division of the equilibrium claim set is illustrated in Fig. 6. Unlike the
previous case, only the doctor is determined to be responsible for the
incident in this scenario. This is attributed to the high initial value of the
operational malpractice claim. Additionally, the positive activation level of
the analogy case provides support for the claim. In traditional medical
operations, the doctor is responsible for any malpractice. Besides, in the
Uber autonomous vehicle case, the responsibility is determined to be
attributed to the test operator, instead of the algorithm developer. All these
analogy cases provide legal support to the new area of AI-induced incidents,
which is lacking regulation or laws for now.

###  V-C Case $3$: Social Belief in Collective Responsibility

In the final scenario, we explore the societal belief that responsibility for
AI-induced incidents should be collectively shared, diverging from the
instinct to pinpoint an individual culprit. Despite challenging traditional
intuitions, we examine the consequences of this collective responsibility
attribution, envisioning a situation where such a belief is enforced by the
government or other factors. In this setting, we set $a^{0}(\text{SET})$ and
$a^{0}(\text{FIND})$ to be $0.8$.

As depicted in Fig. 7, at the equilibrium, the claims accepted indicate that
no individual should be held responsible for the incident. By embracing this
perspective, society can collectively address the responsibility gap,
proposing a shared responsibility model where funds are allocated for
compensation. This approach provides a practical solution to dissolve the
problem of individual responsibility while ensuring compensation in AI-induced
incidents.

Figure 7: Constraint network with social belief in collective responsibility
at equilibrium.

##  VI Conclusion

Responsibility attribution and accountability in the realm of AI pose
intricate challenges in today’s society. In this study, we introduce a
computational reflective equilibrium (CRE) approach to responsibility
attribution in AI-induced incidents, with the goal of achieving a coherent and
ethically acceptable equilibrium for all stakeholders. We describe the
workflow of the CRE computation process and discuss the importance of the
initial activation level in equilibrium computation. Using an AI-assisted
medical decision-support system as an illustrative example, we demonstrate how
different initialization leads to distinct responsibility attributions. Our
framework offers explainability in responsibility attribution reasoning,
coherence by ensuring mutual support among claims, and adaptability to varying
ethical perspectives. It is essential to note that the framework is not
necessarily truth-conducive but serves as a tool to provide ethically reasoned
justifications that are coherently acceptable to all stakeholders. To enhance
precision in responsibility attribution, continuous monitoring, revision, and
reflection of the claims and activation levels are imperative for adjustments,
contributing to the development of a more sustainable and enduring framework.

## References

  * [1] J. R. Biden, “Executive order on the safe, secure, and trustworthy development and use of artificial intelligence,” 2023.
  * [2] Y. Ge and Q. Zhu, “Accountability and insurance in iot supply chain,” _arXiv preprint arXiv:2201.11855_ , 2022.
  * [3] J. Rawls, “A theory of justice,” in _Applied Ethics_. Routledge, 2017, pp. 21–29.
  * [4] N. Daniels, “Reflective equilibrium,” 2003.
  * [5] ——, “Wide reflective equilibrium and theory acceptance in ethics,” _The journal of philosophy_ , vol. 76, no. 5, pp. 256–282, 1979.
  * [6] I. Van de Poel, J. Nihlén Fahlquist, N. Doorn, S. Zwart, and L. Royakkers, “The problem of many hands: Climate change as an example,” _Science and engineering ethics_ , vol. 18, pp. 49–67, 2012.
  * [7] J.-A. Cervantes, S. López, L.-F. Rodríguez, S. Cervantes, F. Cervantes, and F. Ramos, “Artificial moral agents: A survey of the current status,” _Science and engineering ethics_ , vol. 26, pp. 501–532, 2020.
  * [8] J. Hage, “Theoretical foundations for the responsibility of autonomous agents,” _Artificial Intelligence and Law_ , vol. 25, pp. 255–271, 2017.
  * [9] M. Coeckelbergh, “Moral appearances: emotions, robots, and human morality,” _Ethics and Information Technology_ , vol. 12, pp. 235–241, 2010.
  * [10] D. J. Gunkel, “Mind the gap: responsible robotics and the problem of responsibility,” _Ethics and Information Technology_ , vol. 22, no. 4, pp. 307–320, 2020.
  * [11] R. Van den Hoven van Genderen, “Do we need new legal personhood in the age of robots and ai?” in _Robotics, AI and the Future of Law_. Springer, 2018, pp. 15–55.
  * [12] D. F. Thompson, “Moral responsibility of public officials: The problem of many hands,” _American Political Science Review_ , vol. 74, no. 4, pp. 905–916, 1980.
  * [13] R. Duch, W. Przepiorka, and R. Stevenson, “Responsibility attribution for collective decision makers,” _American Journal of Political Science_ , vol. 59, no. 2, pp. 372–389, 2015.
  * [14] M. Taddeo and L. Floridi, “How ai can be a force for good,” _Science_ , vol. 361, no. 6404, pp. 751–752, 2018.
  * [15] P. Liu and Y. Du, “Blame attribution asymmetry in human–automation cooperation,” _Risk Analysis_ , vol. 42, no. 8, pp. 1769–1783, 2022.
  * [16] M. Coeckelbergh, “Artificial intelligence, responsibility attribution, and a relational justification of explainability,” _Science and engineering ethics_ , vol. 26, no. 4, pp. 2051–2068, 2020.
  * [17] L. Yilmaz, A. Franco-Watkins, and T. S. Kroecker, “Coherence-driven reflective equilibrium model of ethical decision-making,” in _2016 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA)_. IEEE, 2016, pp. 42–48.
  * [18] E. Brandstedt and J. Brännmark, “Rawlsian constructivism: a practical guide to reflective equilibrium,” _The Journal of Ethics_ , vol. 24, pp. 355–373, 2020.
  * [19] N. Doorn, “Exploring responsibility rationales in research and development (r&d),” _Science, Technology, & Human Values_, vol. 37, no. 3, pp. 180–209, 2012.
  * [20] Y. Cath, “Reflective equilibrium,” _The Oxford handbook of philosophical methodology_ , pp. 213–230, 2016.
  * [21] P. Thagard, _The cognitive science of science: Explanation, discovery, and conceptual change_. Mit Press, 2012.
  * [22] P. Thagard and K. Verbeurgt, “Coherence as constraint satisfaction,” _Cognitive science_ , vol. 22, no. 1, pp. 1–24, 1998.
  * [23] I. van Rooij and T. Wareham, “Intractability and approximation of optimization theories of cognition,” _Journal of Mathematical Psychology_ , vol. 56, no. 4, pp. 232–247, 2012.
  * [24] S. Grossberg, “Adaptive pattern classification and universal recoding: I. parallel development and coding of neural feature detectors,” _Biological cybernetics_ , vol. 23, no. 3, pp. 121–134, 1976.
  * [25] ——, “Competitive learning: From interactive activation to adaptive resonance,” _Cognitive science_ , vol. 11, no. 1, pp. 23–63, 1987.
  * [26] T. L. Beauchamp, “Methods and principles in biomedical ethics,” _Journal of Medical ethics_ , vol. 29, no. 5, pp. 269–274, 2003.
  * [27] M. Dunn, M. Sheehan, T. Hope, and M. Parker, “Toward methodological innovation in empirical ethics research,” _Cambridge Quarterly of Healthcare Ethics_ , vol. 21, no. 4, pp. 466–480, 2012.