# Bell’s nonlocality and gravity

Yuri Bonder∗ and Johas D. Morales  Instituto de Ciencias Nucleares
Universidad Nacional Autónoma de México
Apartado Postal 70-543, Coyoacán 04510 Cd.Mx., México
∗Corresponding author, email<EMAIL_ADDRESS>
(Essay written for the Gravity Research Foundation
2022 Awards for Essays on Gravitation
)

###### Abstract

The experimental results that test Bell’s inequality have found strong
evidence suggesting that there are nonlocal aspects in nature. Evidently,
these nonlocal effects, which concern spacelike separated regions, create an
enormous tension between general relativity and quantum mechanics. In
addition, by avoiding the coincidence limit, semiclassical gravity can also
accommodate nonlocal aspects. Motivated by these results, we study if it is
possible to construct geometrical theories of gravitation that are nonlocal in
the sense of Bell. We propose three constructions of such theories, which
could constitute an important step towards our understanding of the interplay
between quantum mechanics and gravitation.

##  1 General motivation

It is an empirical fact that Bell’s inequalities are violated. This, in turn,
strongly suggests that there are nonlocal features in nature. What is more,
this nonlocality relates spacelike separated regions of spacetime. At first
sight, this reveals an almost insurmountable tension with general relativity
(GR), where gravity is described by local geometrical entities that have the
causal information. (For an excellent discussion on this issue see Ref. [1],
see also Ref. [2]). The logical conclusion seems to be that, without leaving
the local/geometrical paradigm, we will never be able to construct a theory of
gravitation that can account for matter in every possible state.

In this essay we show that it is in principle possible to build nonlocal
theories of gravity that are still geometrical. To motivate our proposals, we
first present a brief overview of Bell seminal work. We then argue that,
within our most fundamental description of nature, namely, in semiclassical
gravity, it is also possible to incorporate nonlocal effects. Subsequently, we
turn to the question of how to describe gravity geometrically while, at the
same time, it can accommodate nonlocal effects. It should be mentioned that
the nonlocalities considered here are completely different from those found in
the so-called “nonlocal theories of gravitation” where the nonlocalities are
related to the formal appearance of infinitely many derivatives in the action,
or they relate causally connected regions of spacetime [3].

Clearly, addressing the question of nonlocality in gravity could constitute a
crucial step towards a successful quantization of gravity. Nevertheless, the
fact that we are trying to incorporate a nonlocal structure into the
geometrical framework does not imply that we believe that gravity is
fundamentally geometrical; we are simply tackling some questions that, we
believe, could shed light into the quantum nature of gravity.

##  2 Bell’s theorem

Entanglement is one of the most innovative and counterintuitive notions that
arise in quantum mechanics. In fact, since the early years of quantum
mechanics, the nature of entanglement was disputed; the best known example is
the work by Einstein, Podolsky, and Rosen [4]. The issues raised in these type
of studies motivated Bell to study these issues, who, in 1964, presented his
first theorem [5]. This theorem states that, under fairly reasonable
assumptions, any deterministic theory capable of reproducing the predictions
of quantum mechanics must exhibit some kind of nonlocality. This paper was the
first of several works where Bell refined his hypothesis, until he was able to
include probabilistic theories, such as quantum mechanics [6]. Bell’s works
can be summed up in an inequality that constrains the predictions of local
theories. As it is well known, quantum mechanics, having the aforementioned
entangled states, is nonlocal, and it violates Bell’s inequalities.

It is worth mentioning that the assumptions of Bell’s theoriem are quite
general. Besides giving a precise mathematical notion of locality, Bell
assumed “statistical independence.” This hypothesis assumes that the
measurements are independent of the experimental composition that generates
the entangled pair [7], which is essentially what is done every time the
scientific method is applied. Moreover, it is important to emphasize that
Bell’s theorem can be applied in a wide range of circumstances [1], even
outside the realm of quantum mechanics.

Experimental work to test Bell’s inequality began in the early 1980s, using
polarized photons [8, 9, 10]. These experiments found that Bell’s inequalities
are indeed violated. What is more, over time there have been other experiments
that have tested the inequality in extreme situations and that have eliminated
gaps in the theorem’s hypotheses [11, 12, 13, 14, 15, 16]. The results, in all
cases, is that the inequalities are violated, and hence, they provide evidence
that the world has nonlocal features.

##  3 Semiclassical gravity

In quantum field theory in curved spacetimes [17] people came up with a
concrete recipe to build the energy-momentum tensor, for a given quantum
field, such that, its renormalized expectation value can take the place of its
classical counterpart in Einstein’s equation. This scheme is known as
semiclassical gravity and it is the best framework available to study the
effects that quantum systems have on a classical spacetime. For the following
part of the discussion we can focus on a real free Klein-Gordon field $\phi$
of mass $m$. In this case the energy-momentum tensor is given by111Throughout
the essay, the notations a conventions of Ref. [18] are used.

$T_{ab}=\nabla_{a}\phi\ \nabla_{b}\phi-\frac{1}{2}g_{ab}\left(\nabla_{c}\phi\
\nabla^{c}\phi+m^{2}\phi^{2}\right),$ (1)

where $g_{ab}$ and $\nabla_{a}$ are the spacetime metric and the associated
derivative, respectively, and raised indexes indicate the contraction with an
inverse metric, $g^{ab}$.

Importantly, in quantum field theory in curved spacetimes, $\phi$ only makes
sense as an operator-valued distribution, and quadratic distributions, like
$\phi^{2}$, are ill defined [cf. Eq. (1)]. Therefore, to have a well-defined
energy-momentum tensor, one must appeal to a point splitting procedure, where
the fields are evaluated at different points, and, in the end, one takes the
limit where the two points coincide; this last step is known as the
coincidence limit. However, the coincidence limit produces divergences that
must be systematically removed. This process of removing divergences is called
Hadamard renormalization and it has the undesirable feature that it selects a
subset of states from the Fock space. In fact, to compute the renormalized
expectation value of the energy-momentum tensor, only Hadamard states, which
have the same divergent structure in the two-point function than the
corresponding function in flat spacetime, can be utilized.

As we have already discussed, we are taking the point of view that nature is
nonlocal. On the other hand, Hadamard’s renormalization process, and the
corresponding restriction in the Fock space, only play a role in if the
coincidence limit is taken. Therefore, it is relevant to study if this
limiting procedure can be avoided. In this scenario, the energy-momentum
expectation value will depend on two spacetime points; this nonlocal object
will be denoted by $\braket{T_{ab}}_{\psi}$. Note that
$\braket{T_{ab}}_{\psi}$ depends on $g_{ab}$ and it is intimately related with
the two-point functions [19]. Therefore it is reasonable to expected that
$\braket{T_{ab}}_{\psi}$ is sensitive to the causal spacetime structure. In
what follows we assume that $\braket{T_{ab}}_{\psi}$ can be built out of the
causal spacetime structure, and we focus on the possible nonlocal objects that
could take the place of the Einstein tensor in a nonlocal Einstein-like
equation.

##  4 Nonlocal geometries

The goal is to discuss what kind of objects can we use in the geometric side
of an Einstein-like nonlocal equation. We require that all candidates to take
this place have the appropriate limiting behavior. Namely, they must become
the conventional Einstein tensor, $G_{ab}(x)$, in the coincidence limit to be
able to make contact with semiclassical gravity. In addition, we need to have
an object describing the causal spacetime structure. This last assumption is
related to the properties of $\braket{T_{ab}}_{\psi}$, but it is also
compatible with our motivation that the nonlocal effects are Bell-like, and
hence, they concern spatially related spacetime regions.

Bitensors are geometrical objects that have a nonlocal character [20].
Concretely, a bitensor is a multilinear map that takes $k$ covectors and
$\ell$ vectors in $x$, and $k^{\prime}$ covectors and $\ell^{\prime}$ vectors
in $x^{\prime}$, and produces a real number. Such an object can be represented
in an abstract index notation by
$\tensor{T}{{}^{a_{1}}{}^{...}{}^{a_{k}}_{b_{1}}{}_{...}{}_{b_{\ell}}^{a^{\prime}_{1}}{}^{...}{}^{a^{\prime}_{k^{\prime}}}_{b^{\prime}_{1}}{}_{...}{}_{b^{\prime}_{\ell^{\prime}}}}$,
where the (un)primed indices act on vectors and covectors in
($x$)$x^{\prime}$.

Perhaps the best known bitensor, which is well defined within a normal convex
hull, is the parallel propagator
$\tensor{g}{{}^{a^{\prime}}_{b}}=\tensor{g}{{}^{a^{\prime}}_{b}}(x,x^{\prime})$.
This object takes a vector in $x$ and produces the parallel transported vector
at $x^{\prime}$ along the geodesic joining $x$ and $x^{\prime}$. To transport
arbitrary tensors from $x^{\prime}$ to $x$, the inverse of
$\tensor{g}{{}^{a^{\prime}}_{b}}$ is also needed, which is denoted by
$\tensor{g}{{}^{a}_{b^{\prime}}}=\tensor{g}{{}^{a}_{b^{\prime}}}(x,x^{\prime})$.
Obviously
$\tensor{g}{{}^{a}_{b^{\prime}}}\tensor{g}{{}^{b^{\prime}}_{c}}=\delta^{a}_{c}$;
however, in general
$\tensor{g}{{}^{a^{\prime\prime}}_{b^{\prime}}}\tensor{g}{{}^{b^{\prime}}_{c}}\neq\tensor{g}{{}^{a^{\prime\prime}}_{c}}$,
where conventional contraction is defined for each type of index. It should
also be mentioned that bitensors can be derived with respect to each of its
two points; these derivatives are defined as standard (covariant) derivatives
in the point of the derivation, and by keeping everything at other point
fixed. Thus, derivatives associated to different points commute. Notice that,
for any bitensor, one can take a “coincidence” limit $x^{\prime}\to x$.
Throughout this essay, we consider that such limits exist regardless of the
direction in which $x^{\prime}$ approaches $x$.

The bitensors that replace $G_{ab}(x)$ in a nonlocal Einstein-like equation
are called Einstein bitensors; we require that they coincide with $G_{ab}(x)$
in the coincidence limit. We turn to describe three concrete constructions of
Einstein bitensors.

### Conformal factor

It is well known that, for a given metric $g_{ab}$ and a real smooth positive
function $\bar{\Omega}$, the conformally related metric,
$\bar{\Omega}^{2}g_{ab}$, has the same causal structure than $g_{ab}$. The
first of the strategies we put forward to construct an Einstein’s bitensor is
motivated by this simple observation. The idea is to allow the conformal
factor to be a bifunction that will be determined when solving the
corresponding equations of motion. The only requirement on
$\Omega(x,x^{\prime})$, besides being positive, is that
$\Omega(x,x^{\prime})\to 1$ as $x^{\prime}\to x$, fast enough to recover
semiclassical gravity.

We can construct an Einstein bitensor by using the expressions for the
curvature of a conformally related metric [18]. The outcome is

$\displaystyle G_{ab}(x,x^{\prime})$ $\displaystyle=$ $\displaystyle
G_{ab}(x)-2\nabla_{a}\nabla_{b}\ln\Omega(x,x^{\prime})+2\nabla_{a}\ln\Omega\
\nabla_{b}\ln\Omega(x,x^{\prime})$ (2)
$\displaystyle+2g_{ab}\nabla_{c}\nabla^{c}\ln\Omega(x,x^{\prime})+g_{ab}\nabla_{c}\ln\Omega\
\nabla^{c}\ln\Omega(x,x^{\prime}).$

This $G_{a{b}}(x,x^{\prime})$ satisfies all the requirements to be considered
an Einstein’s bitensor since, clearly, $\displaystyle{\lim_{x^{\prime}\to
x}}G_{a{b}}(x,x^{\prime})=G_{ab}(x)$ and the causal structure is codified in
$g_{ab}$. Thus, Eq. (2) is a concrete proposal that satisfies all the required
properties.

### Synge bifunction

Another well-known bitensor is the Synge bifunction $\sigma(x,x^{\prime})$
[21]. This bifunction corresponds to the geodesic distance between $x$ and
$x^{\prime}$ (again, we work in a normal convex hull). Importantly, starting
with $\sigma(x,x^{\prime})$, we can produce new bitensors by taking
derivatives. It turns out that $\displaystyle{\lim_{x^{\prime}\to
x}}\nabla_{a}\nabla_{b}\sigma(x,x^{\prime})=g_{ab}(x)$; and there are similar
relations obtained by taking derivatives at $x^{\prime}$. Notably, this last
relation has motivated some approaches where the metric is considered to
emerge from $\sigma(x,x^{\prime})$ [22].

The proposal then is to build an Einstein bitensor using
$\nabla_{a}\nabla_{b}\sigma(x,x^{\prime})$ in place of $g_{ab}$, without
taking the coincidence limit. Clearly, by construction, this Einstein bitensor
will reduce to the conventional Einstein tensor in such a limit. Moreover, the
causal structure of spacetime is encoded in the sign of
$\sigma(x,x^{\prime})$. The main issue within this strategy is that one need
to take the coincidence limit to some
$\nabla_{a}\nabla_{b}\sigma(x,x^{\prime})$ to have a conventional metric (and
its inverse), which is needed in $\braket{T_{ab}}_{\psi}$ and to make the
contractions necessary to compute the corresponding Einstein bitensor.
Nevertheless, it is clear that an Einstein bitensor with the desired
properties can also be constructed form the Synge function.

### Bitensorial connexion

It is well known that GR accepts a variational formulation where the metric
and the connection are a priori independent [23, 24]. Assuming further that
the action can be separated into a gravity term, that is independent of the
matter fields, and a matter part, $S_{M}$, then, the variation with respect to
the metric, the nonmetric part of the connection,
$\tensor{C}{{}^{c}_{a}{}_{b}}$, and the matter fields, $\varphi$, produces the
equations of motion:

$\displaystyle G_{ab}(g,C)$ $\displaystyle=$ $\displaystyle 8\pi
T_{ab}(g,C,\varphi),$ (3) $\displaystyle
g^{de}\tensor{C}{{}^{a}_{de}}\delta^{b}_{c}+\tensor{C}{{}^{d}_{dc}}g^{ab}-2\tensor{C}{{}^{a}_{ce}}g^{be}$
$\displaystyle=$ $\displaystyle
16\pi\tensor{\Sigma}{{}_{c}^{ab}}(g,C,\varphi),$ (4)
$\displaystyle\frac{\delta S_{M}}{\delta\varphi}$ $\displaystyle=$
$\displaystyle 0,$ (5)

where $T_{ab}$ and $\tensor{\Sigma}{{}_{c}^{a}{}^{b}}$ are, respectively, the
variations of $S_{M}$ with respect to $g_{ab}$ and
$\tensor{C}{{}^{c}_{a}{}_{b}}$. Evidently, if
$\tensor{\Sigma}{{}_{c}^{a}{}^{b}}=0$, then $\tensor{C}{{}^{c}_{a}{}_{b}}=0$,
and GR is recovered.

Unfortunately, to our knowledge, the semiclassical theory associated with Eqs.
(3)-(4) has not been studied. Still, this semiclassical theory would require
that $T_{ab}$ and $\tensor{\Sigma}{{}_{c}^{a}{}^{b}}$ are replaced by their
expectation values. Thus, it is expected that both, $T_{ab}$ and
$\tensor{\Sigma}{{}_{c}^{a}{}^{b}}$, must become nonlocal objects222If $S_{M}$
is the standard model action, only Dirac spinors produce a nontrivial
$\tensor{\Sigma}{{}_{c}^{a}{}^{b}}$. Importantly, this
$\tensor{\Sigma}{{}_{c}^{a}{}^{b}}$ is quadratic in the spinors, and thus, its
expectation value is nonlocal in the same sense as $\braket{T_{ab}}_{\psi}$..
Moreover, it is clear from the fact that Eq. (4) is simply an algebraic
equation that, if we do not take the coincidence limit and we impose that the
metric is a conventional tensor, then $\tensor{C}{{}^{a}_{b}{}_{c}}$ has to be
a bitensor333In case some indexes in
$\tensor{C}{{}^{a}_{b}{}_{c}}(x,x^{\prime})$ are actually primed indexes, as
we naively expect from simple geometrical considerations, one can use the
parallel propagator bitensor to “bring” all indexes to $x$.
$\tensor{C}{{}^{a}_{b}{}_{c}}(x,x^{\prime})$. The important point is that this
bitensorial connection leads to an Einstein bitensor of the form
$G_{ab}(x,x^{\prime})=G_{ab}[g(x),C(x,x^{\prime})]$. What is more, the
semiclassical theory is automatically recovered in the coincidence limit since
Eq.(4) requires $\tensor{C}{{}^{a}_{b}{}_{c}}$ to be a conventional tensor.

Recall that the Riemann tensor associated with an arbitrary connection
$\tensor{C}{{}^{c}_{ab}}$ satisfies

$\tensor{R}{{}_{a}{}_{b}{}_{c}^{d}}=\tensor{R}{{}_{a}{}_{b}{}_{c}^{d}}(x)-2\nabla_{[a}\tensor{C}{{}^{d}_{b]c}}+2\tensor{C}{{}^{e}_{[a|c|}}\tensor{C}{{}^{d}_{b]e}},$
(6)

where $\tensor{R}{{}_{a}{}_{b}{}_{c}^{d}}(x)$ is the Riemann tensor associated
with $g_{ab}$. Clearly, this gives rise to a Riemann bitensor when
$\tensor{C}{{}^{a}_{b}{}_{c}}=\tensor{C}{{}^{a}_{b}{}_{c}}(x,x^{\prime})$. One
can then use the metric to build the corresponding Einstein bitensor, which
has the form

$\displaystyle G_{ab}(x,x^{\prime})$ $\displaystyle=$ $\displaystyle
G_{ab}(x)-2\nabla_{[a}\tensor{C}{{}^{c}_{c]b}}(x,x^{\prime})+g_{ab}(x)g^{cd}(x)\nabla_{[c}\tensor{C}{{}^{e}_{e]d}}(x,x^{\prime})$
(7)
$\displaystyle+2\tensor{C}{{}^{d}_{[a|b|}}(x,x^{\prime})\tensor{C}{{}^{c}_{c]}{}_{d}}(x,x^{\prime})-g_{ab}(x)g^{cd}(x)\tensor{C}{{}^{e}_{[c|d|}}(x,x^{\prime})\tensor{C}{{}^{f}_{f]e}}(x,x^{\prime}).$

Again, this construction has the required properties of producing conventional
semiclassical gravity in the coincidence limit and having the information on
the causal spacetime structure. Moreover, by construction, this tensor
satisfies Eq. (3), which is the Einstein-like equation, but where $T_{ab}$
must be replaced by its expectation value.

##  5 Discussion

In this essay, three methods are presented to construct an Einstein bitensor
that can be used in nonlocal semiclassical gravity, i.e., in semiclassical
gravity before taking the coincidence limit. The guide is to keep an object
that describes the causal structure and to get the correct result in the
coincidence limit. Here we presented concrete examples on how to use several
nonlocal objects to build Einstein bitensors. However, all the constructions
we presented have several ambiguities and there is no additional criteria to
fix them. In trying to make the smallest possible departure from conventional
semiclassical gravity, one can require the Einstein bitensors to be symmetric
under the interchange of its indexes. This is strongly related to having a
scheme that is symmetric when interchanging the two points in the bitensors.
Constructing a point-symmetric bitensor is straightforward: one can always
take one bitensor plus the bitensor with the points in the oposite order. This
simple proposal could restrict the number of models, which, in turn, given the
amount of decisions when building models, could be extremely valuable. Another
requirement could be to have some sort of Bianchi identity, which does not
seem to be present in the context of bitensors.

The mere fact that for a given point $x$ there is a “random” point
$x^{\prime}$ associated to it also seems puzzling. Perhaps this should be
remedied by integrating in some spacetime region, say, the spacelike related
region to $x$. Then, to have well-defined integrals, one would need to produce
a conventional volume form. This seems to be feasible, but it seems unnatural
in the models where the the natural volume form is a bitensor. This is simply
because one would need to take the coincidence limit only for some of the
bitensors. However, this is precisely the same issue that appears in the
construction of the Einstein bitensor out of the Synge bifunction.

The immediate goal of this research plan is to construct a model that is
mathematically self consistent to start making calculations. It is reasonable
to expect that the calculations will be nontrivial, particularly as our
bitensor toolbox is limited as compared to that of conventional tensors. Yet,
two natural simplifications can be implemented. First, one can try to use a
perturbation scheme in which, for example, one expands in powers of
$\sigma(x,x^{\prime})$. In this way, one would get the conventional local
theory to zero order, and then, it seems possible to look for first order
deviations. The second simplification is to study highly symmetric situations.
In fact, we could start by working in maximally symmetric spacetimes where
there are methods to compute $n$-point functions using bitensors [25] (see
also Refs. [26, 27]).

The long-term objective is to select a handful of promising models to be
analized in depth. These studies should tackle mathematical and
phenomenological questions. In the mathematical front, one should try to
extend some of the well-known GR notions. For example, given a fair definition
of the energy-conditions for $\braket{T_{ab}}_{\psi}$, one could study if the
conclusions of the singularity theorems can be avoided. Or, given that, by
construction, the causal spacetime structure is conventional and hence one can
use definitions such as global hyperbolicity, one can verify if the dynamics
of these nonlocal theories is well-posed as an initial value problem.
Regarding the phenomenology, recall that these models reduce to GR whenever
$\braket{T_{ab}}_{\psi}$ behaves as a conventional energy-momentum tensor.
Thus, one should not expect new signals in the usual tests of GR (solar system
tests, gravity waves, etc.). Perhaps the place to look for phenomenological
effects is in cosmology, where there is plenty of data to analyze, and some
apparent tensions [28] to resolve.

##  6 Conclussions

The motivation of our proposal stems from the results of Bell’s work and the
subsequent experiments testing Bell’s inequalities. In addition, semiclassical
gravity, without taking the coincidence limit, produces a nonlocal theory,
which is natural in the sense that it does not impose restrictions in the Fock
space. Thus, we propose that a more accurate description of nature could be
performed using nonlocal geometrical objects, e.g., bitensors. We present the
three constructions and briefly discuss some of their potential issues.

The research program outlined here is at a very early stage. Yet, we are
convinced that it could produce very valuable lessons. If solutions with some
clear physical interpretation are found, we could gain insight into the
question of how entangled states gravitate. In turn, this should shed
invaluable light to construct a satisfactory theory of quantum gravity.
Another possible conclusion is that there is simply no way to generate a self
consistent nonlocal geometrical theory. This would strengthen the argument
that gravity, at a more fundamental level, must be nongeometrical [29].
Whatever the results end up being, we are proposing a very exciting and well-
motivated research program, with the potential to illuminate some of the most
obscure —yet relevant— corners of modern physics.

### Acknowledgment

This research was financially supported by UNAM-DGAPA-PAPIIT Grant IG100120,
CONACyT FORDECYT-PRONACES grant 140630, and the CONACyT scholarships program.

## References

  * Maudlin [2011] T. Maudlin.  _Quantum Non-Locality and Relativity: Metaphysical Intimations of Modern Physics_.  Wiley, third edition, 2011.
  * Tumulka [2006] R. Tumulka.  Relativistic version of the Ghirardi–Rimini–Weber model.  _J. Stat. Phys._ , 125:821, 2006.
  * Mashhoon [2017] B. Mashhoon.  _Nonlocal Gravity_.  Oxford University Press, 2017.
  * Einstein et al. [1935] A. Einstein, B. Podolsky, and N. Rosen.  Can quantum-mechanical description of physical reality be considered complete?  _Phys. Rev._ , 47:777, 1935.
  * Bell [1964] J.S. Bell.  On the Einstein Podolsky Rosen paradox.  _Physics Physique Fizika_ , 1:195, 1964.
  * Bell [2004] J.S. Bell.  _Speakable and Unspeakable in Quantum Mechanics: Collected Papers on Quantum Philosophy_.  Cambridge University Press, 2004.
  * Clauser and Horne [1974] J.F. Clauser and M.A. Horne.  Experimental consequences of objective local theories.  _Phys. Rev. D_ , 10:526, 1974.
  * Aspect et al. [1981] A. Aspect, P. Grangier, and G. Roger.  Experimental tests of realistic local theories via Bell’s theorem.  _Phys. Rev. Lett._ , 47:460, 1981.
  * Aspect et al. [1982a] A. Aspect, J. Dalibard, and G. Roger.  Experimental test of Bell’s inequalities using time-varying analyzers.  _Phys. Rev. Lett._ , 49:1804, 1982a.
  * Aspect et al. [1982b] A. Aspect, P. Grangier, and G. Roger.  Experimental realization of Einstein-Podolsky-Rosen-Bohm gedankenexperiment: A new violation of Bell’s inequalities.  _Phys. Rev. Lett._ , 49:91, 1982b.
  * Tittel et al. [1998] W. Tittel, J. Brendel, H. Zbinden, and N. Gisin.  Violation of Bell inequalities by photons more than 10 km apart.  _Phys. Rev. Lett._ , 81:3563, 1998.
  * Weihs et al. [1998] G. Weihs, T. Jennewein, C. Simon, H. Weinfurter, and A. Zeilinger.  Violation of Bell’s inequality under strict Einstein locality conditions.  _Phys. Rev. Lett._ , 81:5039, 1998.
  * Gallicchio et al. [2014] J. Gallicchio, A.S. Friedman, and D.I. Kaiser.  Testing Bell’s inequality with cosmic photons: Closing the setting-independence loophole.  _Phys. Rev. Lett._ , 112, 2014.
  * Giustina et al. [2015] M. Giustina et al.  Significant-loophole-free test of Bell’s theorem with entangled photons.  _Phys. Rev. Lett._ , 115, 2015.
  * Shalm et al. [2015] L.K. Shalm et al.  Strong loophole-free test of local realism.  _Phys. Rev. Lett._ , 115, 2015.
  * Hensen et al. [2015] B.J. Hensen et al.  Loophole-free Bell inequality violation using electron spins separated by 1.3 kilometres.  _Nature_ , 526:682, 2015.
  * Wald [1995] R.M. Wald.  _Quantum Field Theory in Curved Space-Time and Black Hole Thermodynamics_.  University of Chicago Press, 1995.
  * Wald [1984] R.M. Wald.  _General Relativity_.  University of Chicago Press, 1984.
  * Fulling [1989] S.A. Fulling.  _Aspects of Quantum Field Theory in Curved Spacetime_.  Cambridge University Press, 1989.
  * Poisson [2004] E. Poisson.  The motion of point particles in curved spacetime.  _Liv. Rev. Rel._ , 7, 2004.
  * Synge [1960] J.L. Synge.  _Relativity: The General Theory_.  North-Holland Publishing Company, 1960.
  * Kempf [2021] A. Kempf.  Replacing the notion of spacetime distance by the notion of correlation.  _Frontiers in Physics_ , 9, 2021.
  * Hehl et al. [1995] F.W. Hehl, J.D. Mccrea, E.W. Mielke, and Y. Ne’eman.  Metric affine gauge theory of gravity: Field equations, Noether identities, world spinors, and breaking of dilation invariance.  _Phys. Rep._ , 258:1, 1995.
  * Vitagliano et al. [2011] V. Vitagliano, T.P. Sotiriou, and S. Liberati.  The dynamics of metric-affine gravity.  _Ann. Phys._ , 326:1259, 2011.
  * Allen and Jacobson [1986] B. Allen and T. Jacobson.  Vector two-point functions in maximally symmetric spaces.  _Commun. Math. Phys._ , 103:669, 1986.
  * Allen and Lutken [1986] B. Allen and C.A. Lutken.  Spinor Two Point Functions in Maximally Symmetric Spaces.  _Commun. Math. Phys._ , 106:201, 1986.
  * Perez-Nadal et al. [2010] G. Perez-Nadal, A. Roura, and E. Verdaguer.  Stress tensor fluctuations in de Sitter spacetime.  _JCAP_ , 2010.
  * Di Valentino et al. [2021] E. Di Valentino et al.  Snowmass2021 - Letter of interest cosmology intertwined II: The Hubble constant tension.  _Astropart. Physics_ , 131:102605, 2021.
  * Bonder et al. [2018] Y. Bonder, C. Chryssomalakos, and D. Sudarsky.  Extracting geometry from quantum spacetime.  _Foundations of Physics_ , 48:1038, 2018.


# AuditMAI: Towards An Infrastructure for
Continuous AI Auditing

Laura Waltersdorfer
TU Wien & WU Wien
<EMAIL_ADDRESS>
Fajar J. Ekaputra
WU Wien & TU Wien
<EMAIL_ADDRESS>
Tomasz Miksa
TU Wien & SBA Research
<EMAIL_ADDRESS>
Marta Sabou
WU Wien
<EMAIL_ADDRESS>

###### Abstract

Artificial Intelligence (AI) Auditability is a core requirement for achieving
responsible AI system design. However, it is not yet a prominent design
feature in current applications. Existing AI auditing tools typically lack
integration features and remain as isolated approaches. This results in
manual, high-effort, and mostly one-off AI audits, necessitating alternative
methods. Inspired by other domains such as finance, continuous AI auditing is
a promising direction to conduct regular assessments of AI systems. The issue
remains, however, since the methods for continuous AI auditing are not mature
yet at the moment. To address this gap, we propose the Auditability Method for
AI (AuditMAI), which is intended as a blueprint for an infrastructure towards
continuous AI auditing. For this purpose, we first clarified the definition of
AI auditability based on literature. Secondly, we derived requirements from
two industrial use cases for continuous AI auditing tool support. Finally, we
developed AuditMAI and discussed its elements as a blueprint for a continuous
AI auditability infrastructure.

##  1 Introduction

Due to the increasing reports of incidents caused by Artificial Intelligence
(AI) systems McGregor (2021), calls for countermeasures and alternative design
principles for AI systems have been raised. AI auditability is often mentioned
as one of the high-level requirements for responsible AI design AI (2019);
UNESCO (2021), since AI audits are considered as relevant means to check and
assess AI systems Prem (2023); Raji et al. (2020); Bandy (2021).

Furthermore, a move towards continuous AI auditing practices to empower
regular assessments of critical AI systems is needed due to the expected
increased deployments of AI systems in critical and sensitive domains
Minkkinen et al. (2022). Continuous AI auditing –inspired by financial
auditing– implies automatic periodic control and risk assessments of AI
systems. However, the absence of clear standards, requirements, and technical
tools leaves much space for ambiguity of how to achieve AI auditability
especially continuous AI auditing.

We argue that current approaches in AI auditing are hampered by the following
two challenges:

(C1) Unclear definitions of AI auditability and requirements. Current AI
auditing approaches are described in heterogeneous vocabulary (e.g., impact
assessment, technical or governance audits), leading to different expectations
and requirements Ayling and Chapman (2022). Although AI auditability is
frequently mentioned as a desired system characteristic to build responsible
algorithmic systems Williams et al. (2022), existing AI Principles failed to
provide concrete recommendations and requirements on how to implement AI
auditability Shneiderman (2020).

(C2) Lack of Automated AI auditing approaches and methods. AI auditing methods
have been proposed and are mostly in the form of multiple one-time audits by
external parties being conducted with detective purposes Bandy (2021).
Furthermore, the majority of published governance processes focus on manual
trace and data collection Ayling and Chapman (2022); Minkkinen et al. (2022).
These facts reinforced the notion that continuous AI auditing practices
Minkkinen et al. (2022); Ojewale et al. (2024) are not yet established. Audit
artefacts are often described as static documents or plain text to be
extracted and analysed by humans. Logging services are often only tailored to
specific components and provide limited to no flexible configuration. Only a
few approaches provide ways to increase the degree of automation through
technical AI audits to increase the repeatability of audits and, thus, their
frequency Minkkinen et al. (2022); Ojewale et al. (2024).

While human oversight, evaluation, and interpretation –in the form of manual
efforts– is and will be of crucial importance in AI auditability, increasing
the degree of automation through continuous AI auditing is crucial towards
achieving responsible AI design AI (2019); UNESCO (2021). Therefore, to
overcome these challenges, in this paper, we focus on the following research
question:

What are key elements of an infrastructure to enable continuous AI
auditability?

To answer this question, we propose AuditMAI (Auditability Method for AI), a
framework to enable continuous AI audits. We describe key elements of an
infrastructure enabling continuous AI auditing based on three levels: 1)
knowledge, 2) process, and 3) architecture. To frame the scope of AuditMAI, we
analysed literature for a working definition of AI auditability. In contrast
to the top-down approach for the definition, we derived four main requirements
bottom-up from two industrial research projects Ekaputra et al. (2021); Breit
et al. (2023) focused on enabling continuous AI auditing. AuditMAI is
comprised of three views: Knowledge, process, and architecture, targeting
different interaction levels. AuditMAI framework aims to close the gap in
current auditability research by offering a holistic view of essential
elements relevant for establishing continuous AI auditing.

The rest of this paper is structured as follows: In Section 2, we discuss
related work and propose our own AI auditability definition. In Section 3, we
present our two industrial use cases and derive requirements (cf. Figure 2)
for technical support of continuous AI auditing. In Section 4, we propose
AuditMAI (cf. Figure 3), distilling our results and finally concluding in
Section 5.

##  2 AI Auditability Definition

AI auditability is frequently mentioned in responsible AI principles Díaz-
Rodríguez et al. (2023). However, despite the common description of this
concept, a well-accepted definition with concrete technical implementation is
missing. This lack of concrete requirements and definitions is hampering the
creation of a continuous AI auditing framework. To overcome this gap, we
summarized common themes in AI auditability (Section 2.1) as a foundation for
our AI auditability definition to scope our context (Section 2.2).

###  2.1 Background: Common Discussion Themes in AI Auditability

In this section, we briefly discuss the following common themes on AI
auditability: (i) AI audit goals, (ii) AI audit techniques, (iii) AI audit
methods, (iv) AI auditing tools and (v) challenges for AI auditability.

Goals. AI auditability is often aimed to achieve accountability Cobbe et al.
(2021); Díaz-Rodríguez et al. (2023); Cloete et al. (2021) and transparency
(either as a goal Cobbe et al. (2021); Cloete et al. (2021) or enabler of AI
auditability Díaz-Rodríguez et al. (2023)). Accountability is the obligation
of providing information related to standardization and regulation or
consequences after incidents, while transparency means the provision of
relevant information about the system.

Techniques. Audit traces are mentioned to be essential to AI auditability
Brundage et al. (2020); Shneiderman (2020), meaning record and log-keeping
support provision of required information for AI auditability Cobbe et al.
(2021); Cloete et al. (2021). Traces are collected to provide a holistic
overview, calling for inspection of the entire system context: Such
information includes technical aspects such as data and algorithms and also
contextual information or design decisions Cobbe et al. (2021); AI (2019);
Díaz-Rodríguez et al. (2023), but what exactly is left to be decided at the
practical level.

Methods. The following methods do not specifically define AI auditability but
rather provide methods for different types of AI audits: Internal AI Audit
methods Raji et al. (2020), specific model types such as Large Language Models
Mökander et al. (2023) or for specific purposes, e.g. ethics Mökander and
Floridi (2023). While these methods provide valuable insights, they do not
focus much on technical aspects of audit data models and tool support.

Tools. A multitude of tools have been developed due to rising interest in
responsible AI: Surveys collect frameworks and tools for ethical AI audits
Ayling and Chapman (2022), continuous AI auditing Minkkinen et al. (2022), and
a broader context Ojewale et al. (2024). While these attempts establish a
useful basis by categorizing single tools, there are remaining challenges: i)
tools is a fuzzy term in AI auditing, not necessarily describing executable
software but also guidelines Ayling and Chapman (2022), ii) isolated
approaches since often tools are created for specific application contexts and
use cases, it is unclear how to combine them along an AI auditing workflow
Ojewale et al. (2024), and iii) context-dependence Forum (2023), e.g., the
World Privacy Forum has identified that many popular fairness auditing tools
implement US-specific legislation (US Four-Fifths Employment Rule for AI
Fairness) Forum (2023), not suitable for the rest of the world.

Needs. The need for a holistic assessment poses challenges to AI system
providers and operators, because tracking everything is seldom feasible and
also not sensible due to opacity Cobbe et al. (2021) or other issues (e.g.
privacy, data management) Cloete et al. (2021). This makes the selection and
access of critical traces essential. Furthermore, traces typically come from
very different system components in heterogeneous formats, so logs need to be
integrated at a meaningful semantic level to gain insights from them.

###  2.2 Proposed AI Auditability Definition

Following our discussion on the common themes in AI Auditability and to
establish the basis for our work, we define AI auditability as the following:

AI auditability is the ability of an auditor to obtain accurate and relevant
auditable artefacts for answering concrete audit questions, when examining an
AI system.

Figure 1: Auditability Definition Key Elements (E1-E4) and Challenges (C1-C2b)

To further clarify our definition, we explain the key AI auditability elements
mentioned within our definition (auditor, AI system, audit questions and
auditable artefacts) in the following text and illustrate them in Figure 1
(E1-E4) together with two identified challenges (C2a and C2b).

(1) Auditor is any stakeholder being authorized to examine a given AI system
for a specific purpose, typically by asking a set of audit questions. In
literature, auditors are classified as being first, second and third party
auditors, as well as internal and external auditors Raji et al. (2022a).

(2) AI system includes its algorithmic and technical components, as well as
non-technical components, e.g. implicit knowledge involving humans or
organisational aspects. However, the major challenge remains for system
developers and providers what exactly needs to be captured.

(3) Audit questions are defining the information needs for auditing purposes.
These questions can be related to applicable legislation or certification
standards. Some questions target specific AI system components (e.g., machine
learning component) or the entire AI system (e.g., overall performance
statistics) and have different audit objectives (e.g., fairness or
robustness). An example audit question is Who is the creator of this AI
model?, and a more complicated one is Which bias mitigation measures were
taken throughout the development of this AI model? Other forms of audit
questions might be test cases to mitigate AI risks related to the system
domain or task and also the chosen AI model (e.g., testing for adversarial
input).

(4) Auditable artefacts (also including audit traces, data statements, logs
Brundage et al. (2020); Shneiderman (2020); Weigand et al. (2013); Cloete et
al. (2021)), are the evidence used during AI audits to verify and validate the
core functionality of an AI system by answering audit questions. We
distinguish between static and dynamic artefacts: Static artefacts include
documentation that is less frequently updated, e.g. design decisions. Raji et
al. Raji and Buolamwini (2019) describe examples for more static auditable
artefacts for internal audits, e.g. stakeholder map. Dynamic artefacts,
including audit traces collected from the live systems, from the training or
execution phase of a system (e.g. status or output logs). Furthermore, to
avoid opacity and other issues as mentioned in Section 2.1, not all
information that can be collected should be collected. Thus, it is essential
to consider relevancy and accuracy of auditable artefacts. Both factors are
closely linked to the overall audit goal and thus, related audit questions.

The definition in this section clarified and addressed the first challenge
(C1) unclear AI auditability definiton mentioned in Section 1. Furthermore, it
also helped to clarify the scope of the second challenge (C2) lack of
automated AI auditing approaches and methods. This challenge consists now of
(C2a) the selection of relevant traces and documents within the AI system
context, and (C2b) the provision of an AI auditability infrastructure to
support the continuous AI auditing process.

##  3 Use-case based AI Auditability Requirements

We present two use cases (UC) from industrial research projects to derive
requirements for the development of continuous AI auditing tool support in
order to address challenge C2 (cf. Section 1). To this end, we first discuss
them based on the previously introduced AI auditability elements, namely,
Auditors, AI system, Audit questions and Auditable artefacts (cf. Section
2.2). Then, we summarize requirements derived from the use cases (cf. Fig. 2
and Table 1).

###  3.1 Use Case Analysis

The use cases are from different domains and UC partners: UC1 is situated in
the environmental-legal domain in cooperation with a federal agency Breit et
al. (2023); while UC2 is conducted in the medical domain with several small
companies Ekaputra et al. (2021).

UC 1 The use case covers the ecological and legal domain of a federal agency
dealing with environmental permits. The to-be audited AI system is to
complement human manual extraction from legal permits through automatic
extraction through machine learning. Key information items are extracted from
legal permits (PDFs) to add metadata information of those permits for search
purposes (e.g. operator or applicable law).

Auditors. System operators, who are no machine learning experts are interested
in underlying patterns of wrong predictions and frequently corrected key
information types. Legal analysts are interested in single executions. Both
qualify as internal auditors.

AI System. Main components are: The user interface for document upload and
transformation of PDF documents in machine-readable format. After data
extraction from the documents, four services (2 ML and 2 non-ML services and
an ontology) provide suggestions for key entity types. Finally, users can
correct suggestions by the AI system.

Audit Question. Examples included: What is the average confidence value per
entity type over time? to analyse the overall model fit continuously. Another
audit question was: Did this system execution run successfully? to ensure the
correct functioning of single system runs.

Auditable Artefact. Traces from the identified main AI system components need
to be collected, such as version numbers, timestamps, and completion states,
but also more concrete ones, such as the confidence values for key entity
types or correction states.

UC 2 This use case is situated in the medical domain of startups integrating
health data from multiple application providers, under regulation, e.g. the
General Data Protection Regulation (GDPR) in the European Union. Small health
application providers choose for compliance reasons to store their users’ data
in a cloud platform. This way, they do not have to develop and run the
platform themselves. The centralised platform manages data access to permit
data analysis through privacy-preserving data analysis. Service provider
upload their data with the applicable consent rights from users to the
platform. Researchers can specify study criteria for applicable health data.
Consent management is managed by the platform.

Auditors. Analysts are interested in which software libraries were used for a
certain analysis. System operators can inquire about the integrated logging
concerning the data access and analysis actions performed by analysts for
accountability of past studies. The former are considered external, while the
latter are internal auditors.

AI System. Components include: user data upload, data selection (including
consent management) and analysis (through ML models).

Audit Question. Example questions are: Was the consent evaluation executed
during data collection to ensure compliance. To audit data analysis: Which
software libraries have been used to analyse data in a specific study?

Auditable Artefact. Traces from many sources needed to be collected. For
example, the consent management (in RDF) for the consent result and the
analysis module (in R) for used libraries.

###  3.2 Auditability Requirements

Several requirements can be derived from the generic continuous auditing
approach Minkkinen et al. (2022). However, since we aim to specify more
concrete requirements specifically for AI systems, we derived a set of
requirements through multiple rounds of workshops with the project partners to
design and implement tool support for continuous AI auditing. We visualised
the requirements (R1-R4) in Figure 2 and explain them in the following.

Figure 2: A schematic overview of the UCs and derived requirements (R1-R4)

R1: Identification of relevant auditable artefacts logging over different AI
lifecycle phases: Similar to software engineering, AI systems undergo
lifecycle phases from design until decommissioning. A key challenge is the
identification of relevant audit information. In both UCs, the audit questions
were iteratively derived and refined. In UC 1, the audit focus was put on the
exploitation phase capturing runtime phrases of the system, with just a
limited number of training parameters being captured, while in UC 2, the focus
was on the overall system, also tracking main data flows during execution.
Supporting the identification of relevant artefacts is vital to the
preparation of AI auditing.

R2: Flexibility of auditable artefact collection mechanisms: AI systems can
handle multi-modal data ranging from text, images, video or sound, encoded in
various formats. In both UCs, we encountered different software languages
(Python, R, Java) and libraries, as well as artefact formats, such as JSON,
CSV, TIFF, or RDF. Some log structures were predefined, while others could be
adapted to the AI auditing needs. Flexibility is also needed in terms of
granularity, since for different auditing objectives different information is
needed (e.g. fairness vs. accountability). This flexibility in the system
context must also be represented and feasible for AI auditability by offering
flexible and variable collection mechanisms.

R3: Automation of auditable artefact collection, transformation and
management: Auditable artefacts are created by multiple components, in
different system states as described in the previous requirements and include
various artefacts in the design phase or high numbers of transactions per
system execution during system execution. Therefore, collection,
transformation and management of these artefacts during the system development
and operation is critical, while manual logging is undesired. For both UCs,
automatic collection and management was needed for effective AI auditability.
Integrating heterogeneous trace data into a metadata representation is also
needed to provide improved analysis capabilities.

R4: Ease of analysing auditable artefacts: The artefacts need to be analysed
to obtain audit results and plan further steps based on the outcome. Analysis
capabilities include the investigation of auditable artefacts, writing of
queries or computing metrics and visualisation of results for communication or
reporting purposes. Different levels of expertise and professional backgrounds
of potential auditors require easy analytics and reporting possibilities. In
UC 1, there was only limited technical knowledge to be expected from the
internal auditors and for UC 2, main requirement was to offer a query
interface with pre-defined queries and possibility to write custom queries.

Elements |  UC 1 |  UC 2
---|---|---
Auditor |  (both internal) System operators, legal analysts |  (internal) System operators, (external) analysts
AI System |  UI, 2 ML components, 2 non-ML services, ontology |  UI, consent check, ML-based analysis
AQ System Operator |  What is the average confidence value per key entity type? |  Was the consent evaluation executed during data collection?
AQ Analyst |  Did this system execution run successfully? |  Which software libraries have been used in a specific study?
Auditable Artefacts |  RDF traces, (ML) services output, user corrections |  R scripts, consent result
Requirements |  R1 (focus on exploitation phase of ML systems), R2 (log structure was partly pre-defined), R3 (full automation), R4 (query interface with pre-defined queries, detailed custom queries) |  R1 (focus on data flows in entire system), R2 (log structure was flexible), R3 (full automation), R4 (query interface with pre-defined queries, custom ones)
Table 1: AI Auditability elements in UCs - AQ (Audit Question)

All of these requirements, focusing on the handling and management of
auditable artefacts are enablers for AI auditability. To the best of our
knowledge, there does not yet exist a tool suite to cover all these
requirements, but selected aspects. For example, MLOps tools e.g. MLflow
provide capabilities especially for R2 and R3. Logging and analysis of common
parameters and metadata for ML models can be easily executed with such tools.
The selection and higher degree of granularity for specific parameters and
metrics are more difficult to achieve. For R1 especially auditable artefacts
beyond common MLOps artefacts are not covered. Also for R4 the integration and
analysis mechanisms of auditable artefacts is not supported beyond common
ones. To address this gap, we propose AuditMAI.

##  4 AuditMAI Framework

To address both the identified challenges (cf. Section 2.2: C2a-C2b) as well
as the requirements (cf. Section 3.2: R1-R4), we present AuditMAI
(Auditability Method for AI) framework. The aim is to provide a blueprint for
AI auditability through technical tools for the auditing process. The
framework was iteratively built and refined based on our experiences of past
projects. AuditMAI is partly implemented through our prototype AuditBox,
leveraging semantic web technologies for auditable artefact integration and
management Ekaputra et al. (2021); Breit et al. (2023).

#### Framework Description.

AuditMAI is divided into three views targeting different levels of the AI
auditability infrastructure: (i) Knowledge View, (ii) Process View and (iii)
Architecture View. The framework integrates the aspects of knowledge
management, process understanding and architectural design for supporting
continuous AI auditing showing relevant flows between each layer. In the
following, we discuss them:

I. Knowledge View \- gathers knowledge about derived AI audit questions, test
cases and risks, documentation standards and existing tools and metrics. It
consists of three knowledge bases:

Audit Questions, (cf. Section 2) suggest which information needs to be
collected during the audit process. Initial collections have been made for
different purposes e.g. explainability Liao et al. (2020) or accountability
Naja et al. (2022) or trustworthy AI Poretschkin et al. (2021). Initial risk
taxonomies have been synthesised Raji et al. (2022b) through AI Incident
databases, such as Incident DB McGregor (2021) and
AIAAIC111https://www.aiaaic.org.

Documentation Standards, collect relevant metadata standards to describe and
document relevant information of auditable artefacts. Well-known approaches
such as ModelCards Mitchell et al. (2019) and Datasheets Gebru et al. (2021)
have reached a certain adoption due their integration on popular online
platforms e.g. HuggingFace or Github. However, current research indicates that
AI documentation is failing to adequately include and communicate relevant AI
system features Bhat et al. (2023); Longpre et al. (2023).

Tool and Metrics differ depending on task and domain. Knowledge bases such as
the OECD Catalogue222 https://oecd.ai/en/catalogue/overview or surveys
Minkkinen et al. (2022); Ayling and Chapman (2022) are helpful to investigate
available tools. However, since terminology in AI auditing is still fuzzy,
attempts such as the recently published AI audit tooling landscape map333
https://tools.auditing-ai.com aim to go beyond prior classifications by
providing a more comprehensive audit tool taxonomy. The same is needed to
organise metrics and evaluation techniques for AI auditability.

II. Process View \- focuses on the auditing process steps incorporating audit
questions and auditable artefacts. It consists of four auditing steps:

Figure 3: AuditMAI: Combining knowledge management, process understanding and
architectural design for AI Auditability Infrastructure

Step 1: Audit Scoping. To scope the audit process, two inputs are needed: i)
the AI system description describing main data flows and algorithmic modules
and ii) the overall audit goal, e.g. transparency or fairness. Both
descriptions are then also input to the architectural component (cf. Audit
Scoping Service) to provide recommendations and subsequent selection of
applicable audit questions and risks also based on the input from the
Knowledge View.

Step 2: Audit Setup. After scoping the audit, this step is concerned with
setting up the collection of auditable artefacts. These artefacts are derived
based on the AI system description from the previous step. Furthermore,
identified artefacts need to be integrated into a common view across the
entire AI system to answer complex audit questions e.g. in a graph-based
representation (cf. architectural component Audit Setup Service).

Step 3: Audit Artefact Collection. This step deals with collecting auditable
artefacts during the runtime of a system. Ideally, artefact collection is
automated, with only limited data being collected or tested manually (cf.
architectural component Audit Collection Service).

Step 4: Audit Analysis and Reporting. In the final step, auditable artefacts
are analysed to report results of the audit process to stakeholders and
identify system improvements (cf. architectural component Audit Analytics &
Reporting Service). Another audit process can be started then.

III. Architecture View \- describes technical services to support the
continuous AI auditing process, especially the definition of audit questions
and management of relevant auditable artefacts. The four requirements (cf.
Section 3.2) are targeted by the following four services:

Audit Scoping Service. The audit scoping includes the selection of applicable
audit questions, risks and test cases (R1 Identification of relevant auditable
artefacts). This is highly dependent on the AI system, context and domain. The
AI system workflow description with main data flows and algorithmic components
assists in suggesting example audit questions or test cases in each phase of
the system lifecycle. In line with continuous auditing, knowledge from past
experiences is integrated, e.g., from AI incident databases, and an audit data
model is output for subsequent steps.

Audit Setup Service. This service aims to prepare the collection of auditable
artefacts from available data sources (R2 Flexibility of auditable artefact
collection mechanisms). Example data sources can be textual descriptions
(e.g., Model Cards Mitchell et al. (2019)), automatic MLOps tools (e.g.,
MLflow444 https://mlflow.org), or specific automatic ones (e.g., Aequitas for
fairness Saleiro et al. (2018)). Also, logs might be already collected for
other purposes (e.g., debugging or security) and could be extended for
auditing purposes. An overview of the required auditable artefacts and the
needed artefact collection tools is the output of this step.

Artefact Management Service. This service is responsible for storing and
persisting collected artefacts (R3 Automation of auditable artefact
management) for analysis and reporting. The choice of storage and retrieval of
artefacts might depend on organisational contexts and requirements, such as
given infrastructure. Due to the heterogeneity of auditable artefacts and to
answer complex audit queries, data integration is important e.g. through
Knowledge Graphs and Semantic Web technologies Naja et al. (2022); Breit et
al. (2023).

Audit Artefact Analytics and Reporting Service. This component provides
analysis and investigation capabilities of auditable artefacts. Artefacts can
be analysed through a provided dashboard, predefined or custom queries or
generated reports accessing the artefact management service (R4 Ease of
analysing auditable artefacts).

##  5 Conclusion

Through AuditMAI, we provide a framework for AI auditability infrastructure
for continuous AI auditing. We aim to counteract unclear requirements
hampering both process and technical maturity in AI auditability and the
currently fragmented emerging AI audit tools landscape. To achieve this, the
framework is scoped around our AI auditability definition and requirements
from industrial projects structured into three different views. The aim is to
reduce manual efforts and support preventive and continuous AI auditing.

Discussion. While the use cases provided important insights into the needs and
requirements for continuous AI auditing practices, they might be subject to
selection and observer bias.

Future Work. We plan to further analyze the capabilities of existing AI
auditing tools and audit data models to extend our work on the AuditBox tool
for AI Auditability Breit et al. (2023), specifically to cover all four steps
of the AuditMAI framework. Furthermore, we plan to conduct a comprehensive
evaluation of the AuditMAI framework with AuditBox as the reference
implementation.

## References

  * McGregor [2021] Sean McGregor.  Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , 2021.
  * AI [2019] HLEG AI.  _High-level Expert Group on Artificial Intelligence_.  European Commission, 2019.
  * UNESCO [2021] UNESCO.  Recommendation on the Ethics of Artificial Intelligence, 2021.
  * Prem [2023] Erich Prem.  From Ethical AI Frameworks to Tools: A Review of Approaches.  _AI and Ethics_ , pages 1–18, 2023.
  * Raji et al. [2020] Inioluwa Deborah Raji, Andrew Smart, Rebecca N White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes.  Closing the AI Acountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing.  In _Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency_ , pages 33–44, 2020.
  * Bandy [2021] Jack Bandy.  Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits.  _Proceedings of the ACM on Human-Computer Interaction_ , 5(CSCW1):1–34, 2021.
  * Minkkinen et al. [2022] Matti Minkkinen, Joakim Laine, and Matti Mäntymäki.  Continuous Auditing of Artificial Intelligence: A Conceptualization and Assessment of Tools and Frameworks.  _Digital Society_ , 1(3):21, 2022.
  * Ayling and Chapman [2022] Jacqui Ayling and Adriane Chapman.  Putting AI Ethics to Work: Are the Tools Fit for Purpose?  _AI and Ethics_ , 2(3):405–429, 2022.
  * Williams et al. [2022] Rebecca Williams, Richard Cloete, Jennifer Cobbe, Caitlin Cottrill, Peter Edwards, Milan Markovic, Iman Naja, Frances Ryan, Jatinder Singh, and Wei Pang.  From Transparency to Accountability of Intelligent Systems: Moving beyond Aspirations.  _Data & Policy_, 4:e7, 2022.
  * Shneiderman [2020] Ben Shneiderman.  Bridging the Gap Between Ethics and Practice: Guidelines for Reliable, Safe, and Trustworthy Human-centered AI systems.  _ACM Transactions on Interactive Intelligent Systems (TiiS)_ , 10(4):1–31, 2020.
  * Ojewale et al. [2024] Victor Ojewale, Ryan Steed, Briana Vecchione, Abeba Birhane, and Inioluwa Deborah Raji.  Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling.  _arXiv preprint arXiv:2402.17861_ , 2024.
  * Ekaputra et al. [2021] Fajar J. Ekaputra, Andreas Ekelhart, Rudolf Mayer, Tomasz Miksa, Tanja Sarčević, Sotiris Tsepelakis, and Laura Waltersdorfer.  Semantic-enabled Architecture for Auditable Privacy-Preserving Data Analysis.  _Semantic Web Journal_ , 2021.
  * Breit et al. [2023] Anna Breit, Laura Waltersdorfer, Fajar J Ekaputra, Sotirios Karampatakis, Tomasz Miksa, and Gregor Käfer.  Combining Semantic Web and Machine Learning for Auditable Legal Key Element Extraction.  In _European Semantic Web Conference_ , pages 609–624. Springer, 2023.
  * Díaz-Rodríguez et al. [2023] Natalia Díaz-Rodríguez, Javier Del Ser, Mark Coeckelbergh, Marcos López de Prado, Enrique Herrera-Viedma, and Francisco Herrera.  Connecting the Dots in Trustworthy Artificial Intelligence: From AI Principles, Ethics, and Key Requirements to Responsible AI Systems and Regulation.  _Information Fusion_ , page 101896, 2023.
  * Cobbe et al. [2021] Jennifer Cobbe, Michelle Seng Ah Lee, and Jatinder Singh.  Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems.  In _Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency_ , pages 598–609, 2021.
  * Cloete et al. [2021] Richard Cloete, Chris Norval, and Jatinder Singh.  Auditable augmented/mixed/virtual reality: The practicalities of mobile system transparency.  _Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies_ , 5(4):1–24, 2021.
  * Brundage et al. [2020] Miles Brundage, Shahar Avin, Jasmine Wang, Haydn Belfield, Gretchen Krueger, Gillian Hadfield, Heidy Khlaaf, Jingying Yang, Helen Toner, Ruth Fong, Tegan Maharaj, Pang Wei Koh, Sara Hooker, Jade Leung, Andrew Trask, Emma Bluemke, Jonathan Lebensold, Cullen O’Keefe, Mark Koren, Théo Ryffel, JB Rubinovitz, Tamay Besiroglu, Federica Carugati, Jack Clark, Peter Eckersley, Sarah de Haas, Maritza Johnson, Ben Laurie, Alex Ingerman, Igor Krawczuk, Amanda Askell, Rosario Cammarota, Andrew Lohn, David Krueger, Charlotte Stix, Peter Henderson, Logan Graham, Carina Prunkl, Bianca Martin, Elizabeth Seger, Noa Zilberman, Sean O hEigeartaigh, Frens Kroeger, Girish Sastry, Rebecca Kagan, Adrian Weller, Brian Tse, Elizabeth Barnes, Allan Dafoe, Paul Scharre, Ariel Herbert-Voss, Martijn Rasser, Shagun Sodhani, Carrick Flynn, Thomas Krendl Gilbert, Lisa Dyer, Saif Khan, Yoshua Bengio, and Markus Anderljung.  Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims.  4 2020.  URL http://arxiv.org/abs/2004.07213.
  * Mökander et al. [2023] Jakob Mökander, Jonas Schuett, Hannah Rose Kirk, and Luciano Floridi.  Auditing Large Language Models: A Three-Layered Approach.  _AI and Ethics_ , pages 1–31, 2023.
  * Mökander and Floridi [2023] Jakob Mökander and Luciano Floridi.  Operationalising AI Governance through Ethics-Based Auditing: an Industry Case Study.  _AI and Ethics_ , 3(2):451–468, 2023.
  * Forum [2023] World Privacy Forum.  _Risky Analysis: Assessing and Improving AI Governance Tools_.  2023\.
  * Raji et al. [2022a] Inioluwa Deborah Raji, Peggy Xu, Colleen Honigsberg, and Daniel Ho.  Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance.  In _Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society_ , pages 557–571, 2022a.
  * Weigand et al. [2013] Hans Weigand, Paul Johannesson, Birger Andersson, Maria Bergholtz, Faiza Allah Bukhsh, R Deneckere, and H Proper.  Conceptualizing Auditability.  In _CAiSE forum_ , pages 49–56, 2013.
  * Raji and Buolamwini [2019] Inioluwa Deborah Raji and Joy Buolamwini.  Actionable auditing: Investigating the impact of publicly naming biased performance results of commercial ai products.  In _Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society_ , pages 429–435, 2019.
  * Liao et al. [2020] Q Vera Liao, Daniel Gruen, and Sarah Miller.  Questioning the AI: Informing Design Practices for Explainable AI User Experiences.  In _Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems_ , pages 1–15, 2020.
  * Naja et al. [2022] Iman Naja, Milan Markovic, Peter Edwards, Wei Pang, Caitlin Cottrill, and Rebecca Williams.  Using Knowledge Graphs to Unlock Practical Collection, Integration, and Audit of AI Accountability Information.  _IEEE Access_ , 10:74383–74411, 2022.
  * Poretschkin et al. [2021] Maximilian Poretschkin, Anna Schmitz, Maram Akila, Linara Adilova, Daniel Becker, Armin B Cremers, Dirk Hecker, Sebastian Houben, Michael Mock, Julia Rosenzweig, et al.  Leitfaden zur Gestaltung vertrauenswürdiger künstlicher Intelligenz.  _KI-Prüfkatalog. Sankt Augustin: Fraunhofer-Institut für Intelligente Analyse-und Informationssysteme IAIS_ , 2021.
  * Raji et al. [2022b] Inioluwa Deborah Raji, I Elizabeth Kumar, Aaron Horowitz, and Andrew Selbst.  The Fallacy of AI Functionality.  In _Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency_ , pages 959–972, 2022b.
  * Mitchell et al. [2019] Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru.  Model Cards for Model Reporting.  In _Proceedings of the Conference on Fairness, Accountability, and Transparency_ , pages 220–229, 2019.
  * Gebru et al. [2021] Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé Iii, and Kate Crawford.  Datasheets for datasets.  _Communications of the ACM_ , 64(12):86–92, 2021.
  * Bhat et al. [2023] Avinash Bhat, Austin Coursey, Grace Hu, Sixian Li, Nadia Nahar, Shurui Zhou, Christian Kästner, and Jin LC Guo.  Aspirations and Practice of ML Model Documentation: Moving the Needle with Nudging and Traceability.  In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_ , pages 1–17, 2023.
  * Longpre et al. [2023] Shayne Longpre, Robert Mahari, Anthony Chen, Naana Obeng-Marnu, Damien Sileo, William Brannon, Niklas Muennighoff, Nathan Khazam, Jad Kabbara, Kartik Perisetla, et al.  The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI.  _arXiv preprint arXiv:2310.16787_ , 2023.
  * Saleiro et al. [2018] Pedro Saleiro, Benedict Kuester, Loren Hinkson, Jesse London, Abby Stevens, Ari Anisfeld, Kit T Rodolfa, and Rayid Ghani.  Aequitas: A Bias and Fairness Audit Toolkit.  _arXiv preprint arXiv:1811.05577_ , 2018.