[1,2]Stefan Sandfeld

[1]Institute of Advanced Simulation–Materials Data Science and Informatics
(IAS-9), Forschungszentrum Jülich, Germany 2]Faculty of Georesources and
Materials Engineering, RWTH Aachen University, Germany

# Modeling Dislocation Dynamics Data Using Semantic Web Technologies

Ahmad Zainul Ihsan<EMAIL_ADDRESS>Said Fathalla  s.fathalla@fz-
juelich.de<EMAIL_ADDRESS>*  [

###### Abstract

Research in the field of Materials Science and Engineering focuses on the
design, synthesis, properties, and performance of materials. An important
class of materials that is widely investigated are crystalline materials,
including metals and semiconductors. Crystalline material typically contains a
distinct type of defect called “dislocation”. This defect significantly
affects various material properties, including strength, fracture toughness,
and ductility. Researchers have devoted a significant effort in recent years
to understanding dislocation behavior through experimental characterization
techniques and simulations, e.g., dislocation dynamics simulations. This paper
presents how data from dislocation dynamics simulations can be modeled using
semantic web technologies through annotating data with ontologies. We extend
the already existing Dislocation Ontology by adding missing concepts and
aligning it with two other domain-related ontologies (i.e., the Elementary
Multi-perspective Material Ontology and the Materials Design Ontology)
allowing for representing the dislocation simulation data efficiently.
Moreover, we show a real-world use case by representing the discrete
dislocation dynamics data as a knowledge graph (DisLocKG) that illustrates the
relationship between them. We also developed a SPARQL endpoint that brings
extensive flexibility to query DisLocKG.

###### keywords:

Ontology, Knowledge Graph, Reasoning, Dislocation, Crystallographic Defects,
Semantic Web

##  1 Introduction

Plastic deformation in metals and other crystalline materials can be
attributed to a one-dimensional lattice defect type known as dislocation. The
concept of the dislocation was introduced in the 1930s by Taylor [1] and
Polanyi [2]. Dislocations determine mechanical properties of materials, such
as strength, hardness, and ductility. For instance, materials engineers have
discovered the strengthening mechanism of crystalline materials by studying
the relationship between dislocation motion and the mechanical behavior of
metals [3]. By controlling the motion of dislocations in crystalline
materials, materials engineers can build, for example, an airplane turbine
blade that can withstand an operation temperature of $\sim 1000^{\circ}C$ and
creep deformation due to centrifugal forces while the turbine is rotating [4].
Significant efforts have been made to understand dislocation systems using
dedicated microscopy techniques and simulation methods. These simulation
methods along with other techniques have been created to predict dislocation
evolution.

In recent years, data-driven approaches have brought new methods and tools for
analyzing and understanding the evolution of dislocation systems [5, 6, 7, 8,
9, 10]. This intensely transforms the Materials Science and Engineering (MSE),
combining simulations, data mining, and experiments, making the digital
transformation possible [11, 12]. However, a digital transformation without
being supported by the appropriate data infrastructure often ends up with
isolated and inaccessible data repositories, the so-called “data silos”. In
this regard, materials informatics plays a significant role in materials
science research to overcome the data silos problem. This is because materials
informatics combines two discourses of materials science and information
technologies to tackle the major problems in materials science, such as data
management and analysis. Moreover, it helps to develop intelligent systems to,
e.g., explore materials, find novel materials properties, or study the
behavior of a specific materials phenomenon.

To fully understand the behavior of materials and in particular dislocations,
aspects from different length scales need to be considered. This variety of
length scales makes the knowledge representation of systems of dislocations
challenging, even though this has yet to be perceived as a significant
research hindrance in materials science. Generally, the schematic
representation of knowledge, i.e. the representation through ontologies, can
significantly boost data management and analysis; it helps to extract
knowledge from data. Ontologies also allow the domain knowledge to be machine-
understandable, meaning that machines can read and interpret this knowledge
efficiently. Furthermore, it has become an essential part of achieving FAIR
(Discoverable, Accessible, Interoperable, and Reusable) data [13].

This paper presents how Discrete Dislocation Dynamics (DDD) data can be
enriched using Semantic Web technologies, such as the Resource Description
Framework (RDF) [14], the Web Ontology Language (OWL) [15] and SPARQL [16].
The first step we have taken is to adapt and extend the dislocation ontology
(DISO) [17, 18] so that it can model various concepts and relationships in the
DDD domain. The adaption includes adding missing concepts, improving class
definition, exploring additional relationships between concepts, and finally
aligning it with other domain-related ontologies, including the Elementary
Multi-perspective Material Ontology (EMMO) and the Materials Design Ontology
(MDO). This allows for representing the dislocation simulation data
efficiently. DISO is one of the Dislocation Ontology Suite
(DISOS)111https://purls.helmholtz-metadaten.de/disos ontologies that represent
the concepts and relationships of linear defects in crystalline materials. In
fact, DISOS comprises several modules describing materials scientific
concepts, representations of dislocations, and different simulation models in
the dislocation domain. The adapted version of DISO is developed and
maintained in the DISOS GitHub repository. The ontology is available in
several RDF serializations via a persistent identifier (i.e.,
https://purls.helmholtz-metadaten.de/disos/diso) provided by PIDA (Persistent
Identifiers for Digital Assets)222https://purls.helmholtz-metadaten.de/. PIDA
employs content negotiation [19] to serve different versions of the ontology
(i.e., the HTML documentation or an RDF representation) via its IRI. DISO has
been syntactically validated by the W3C RDF validation
service333https://www.w3.org/RDF/Validator/ to conform with the W3C RDF
standards. The documentation of the ontology is available via its IRI.

The next step after adapting the ontology is to annotate the data gathered
from multiple DDD simulations with the adapted version of DISO resulting in a
knowledge graph (DisLocKG) of DDD data (more details can be found in section
6). This knowledge graph connects DDD data concepts via the relationship
between them, thus enabling machine actionability [20], allowing for semantic
querying, inferring implicit knowledge that does not exist, and ensuring data
consistency and integrity. The objective is to convert the unstructured DDD
data to linked data with dereferenceable IRIs that adheres to W3C standards
and best practices. This will enable not only reasoning about the dislocation
data but also to integrate it into other MSE-related fields. We have made
DisLocKG publicly available via its GitHub
repository444https://purls.helmholtz-metadaten.de/dislockg.

##  2 Related work

Over the past few years, many researchers gave a particular attention on
developing ontologies to represent scientific data in different fields of
science, such as physics [21], agriculture [22], and pharmaceutical science
[23]. Specifically in the MSE field, several efforts have been up in creating
ontologies representing materials-related notions or semantically presenting
actual materials data as knowledge graphs. This section will discuss related
works of knowledge graphs with and without semantic web technology (including
RDF, OWL, and SPARQL). Two examples from the latter are the Propnet Knowledge
Graph [24] and the Materials Knowledge Graph (MatKG) [25].

The Propnet is a knowledge graph enhancing materials properties data from the
Materials Project [26] Repository555https://next-gen.materialsproject.org. It
augments base properties data (e.g., lattice, basis, chemical formula, band
gap, and total energy) resulting from the ab-initio calculation into derived
properties, e.g., Debye temperature, bulk modulus, and shear modulus. The
workflow and input for generating the augmented data are subsequently stored
in the knowledge graph.

On the other hand, the MatKG stores metadata from over 2.9 million materials
science articles. This metadata includes abstracts, titles, keywords, and
author data (e.g., name, email, affiliation, and ORCID). By accessing the
MatKG, we can retrieve information such as the milestones of a material
developed by multiple authors.

Ashino [27] have developed a “Materials Ontology” which is an ontology
describing substances, processes, environments, and properties. This ontology
also has been used to exchange data between three different thermal property
databases.

In the solid-state physics domain, Li et al. [28] have developed the Materials
Design Ontology (MDO) which is an ontology covering knowledge in the field of
materials design, e.g., with regards to ab-initio methods. MDO is used to
represent materials’ data related to ab-initio calculations over disparate
materials data repositories as RDF triples. At the time of writing the paper,
a total of $\approx 4.3$K RDF triples have been collected in their
repository666https://github.com/LiUSemWeb/materials-design-ontology. While the
work is related to the representation of crystalline material by means of the
crystal structure, MDO does not represent data related to crystalline defects.

Another effort in the experimental materials science community that uses
semantic web technologies is the NanoMine Knowledge
Graph777http://nanomine.org/ [29]. It is a knowledge graph for polymer
nanocomposite materials integrating diverse data from more than 1,700 polymer
nanocomposite experiments. Moreover, the authors of the NanoMine knowledge
graph have developed the NanoMine ontology888https://github.com/tetherless-
world/nanomine-ontology, which is a backbone ontology to describe polymer
nanocomposite experiments.

In conclusion, it is evident that even though several efforts and groups
utilizing the semantic web in various MSE-related fields have progressed
significantly, work for semantically representing dislocations simulation data
is still missing. We believe that this work is the first attempt at creating a
knowledge graph in an MSE-related domain that deals with data governing
details of linear crystallographic defects, i.e. dislocation data. As a
result, the unstructured dislocation data is transformed into linked data with
dereferenceable IRIs using persistent URLs, adhering to W3C standards and best
practices. This enables not only to annotation of dislocation data by an
ontology but also to integration of dislocation data into other MSE-related
fields.

##  3 Description of the Domain

This section briefly describes the relevant notions and concepts of line
defects within the crystalline materials domain.

###  3.1 Representation of Crystalline Materials

Most of the metals and metallic materials have a crystalline structure, which
implies that the atoms are arranged in a periodic structure with a high degree
of symmetry. This periodic arrangement is at the basis of the _crystal
structure_ model, idealizing the physical concept of crystalline materials.
For example, in Figure 1 atoms are shown in an idealized manner as small
spheres.

Figure 1: The crystal structure of face-centered cubic comprises an aggregate
of atoms within one unit cell in crystalline materials.

The crystal structure is represented by the _lattice_ together with a _motif_
: the lattice is a mathematical concept of an infinite, repeating arrangement
of points in space (3D), in a plane (2D), or on a line (1D), in which all
points have the same surrounding and coincide with atom positions. The motif
(or base) consists of an arrangement of chemical species, which can be atoms,
ions, or molecules in crystalline materials. By putting a motif of one or more
atoms at every lattice point, the crystal structure can be represented.

It is now possible to identify the smallest atom pattern that can be repeated
along all spatial directions to cover the entire structure. This pattern is
called a _unit cell_ , shown as the black cube in Figure 1. The lattice
parameters of the unit cell consist of the angles between the edges and the
edge lengths.

Figure 2: The geometry of a unit cell is exactly defined through the three
lengths, $a$, $b$, and $c$, and the three angles $\alpha$, $\beta$, and
$\gamma$.

Figure 2 shows the six lattice parameters needed to characterize the unit
cell: three lengths ($a,b,c$) and three angles ($\alpha,\beta,\gamma$). These
parameters also constitute the basis vectors in the crystal coordinate system;
they are not necessarily mutually perpendicular. Unit cells are often
classified into a systematic based on the lattice parameters (cf. Figure 3).

Figure 3: The seven crystal systems. These seven crystal systems are also
seven primitive Bravais lattices. Each of them only corresponds to a single
lattice point.

For instance, the cubic system has $a=b=c,\;\alpha=\beta=\gamma=90^{\circ}$
and the orthorhombic system has $a\neq b\neq
c,\;\alpha=\beta=\gamma=90^{\circ}$. Seven crystal systems are often ordered
according to the increasing symmetry: cubic, tetragonal, orthorhombic,
hexagonal, rhombohedral, monoclinic, and triclinic.

In the unit cell, we can also define _lattice points_ , _lattice directions_ ,
and _lattice planes_ : A lattice consists of lattice points where the atoms,
ions, or molecules are located (the leftmost cube in Figure 4). The vector
position of lattice points, $\overrightarrow{R}$, is described by the equation

$\mathbf{\overrightarrow{R}}=n_{1}\mathbf{a}+n_{2}\mathbf{b}+n_{3}\mathbf{c}\;,$
(1)

where $n_{i}$ are arbitrary integers and $\mathbf{a},\mathbf{b},\mathbf{c}$
are basis vectors (pointing along the axes in Figure 2) derived from the
lattice parameters. As illustrated in Figure 4, a lattice direction or lattice
vector is a vector connecting two lattice points, whereas a lattice plane
forms an infinitely stretched plane (characterized through a plane normal)
that cuts through lattice points such that a regular arrangement of lattice
points in the plane occurs.

Figure 4: On the very left, a unit cell is shown that corresponds to a ”face-
centered cubic” structure. The black points indicate the lattice points, i.e.
the positions of the atoms. Second from left: a direction vector that connects
to these lattice points. Lastly, two different lattice planes are shown.

###  3.2 Description of Linear Defects

In crystalline materials, atoms are not always arranged or positioned
perfectly. Typically, different kinds of crystallographic defects lead to
disruption of the local order in a material (in addition to thermal
fluctuations affecting the atomic positions). A common type of such defect is
the dislocation, which causes a strongly localized, tube-like region of
disorder (illustrated by the dashed circle in the right panel of Figure 5;
this tube-like region stretches along the $z$-direction).

Figure 5: The left figure shows the perfect order of atoms in a crystalline
material. The right crystal contains a dislocation that destroys the local
order of the crystal structure. Note that red-colored atoms only show the
‘irregularity”, and are not different atom types.

This region contains the highly disordered dislocation core at the center.
Further away from the dislocation core, the perfect lattice structure is
restored, even though there is now a row of atoms shifted into the new
position as indicated by the red spheres in Figure 5.

The “Burgers vector” of a dislocation can be defined through the “Burgers
circuit”, as shown in 6. A Burgers circuit is an atom-to-atom path that is
closed in a perfect crystal (left panel of the figure). The length of the path
is given as multiple of the atomic distances in two directions. In the
presence of a dislocation, a path of the same lengths would not be closed
(right panel of the figure). The step-by-step procedure is as follows: We
define a reference point C as the start point of the path. The line sense of
the path is given by $\boldsymbol{\xi}$ assuming the ”right-hand convention”
(the thumb points along the vector $\boldsymbol{\xi}$ into the picture plane
and the other fingers curl around that vector; their fingertips indicate the
direction of the path).

Figure 6: The Burgers circuit in the crystalline materials. While the left
panel has the Burgers circuit in the perfect crystal material. The right panel
has the Burgers circuit around the dislocation in the crystalline materials.
Due to the closure failure in the defective crystalline materials, we can
define the Burgers vector, $\mathbf{b}$.

The symbol $\otimes$ in the figure indicates that the vector
$\boldsymbol{\xi}$ points into the picture plane. In the perfect crystal, this
circuit goes up five atoms from the reference point, four atoms to the left,
five atoms down, and four atoms to the left to close a circuit at point C
again. With the same reference point and the same number of atomic spacings as
in the circuit, the perfect crystalline material can be used with the crystal
containing a dislocation. However, the circuit does not end at the same point
as in the perfect crystal, but rather at point G as shown in the right panel
of Figure 6. The vector connecting the start point C with the finish point G
is the Burgers vector $\mathbf{b}$.

There are two fundamentally different types of dislocations: “screw” and
“edge” dislocation. A screw dislocation has a line sense parallel to its
Burgers vector, $\boldsymbol{\xi}||\boldsymbol{b}$, whereas for an edge
dislocations the line sense is perpendicular to its Burgers vector,
$\boldsymbol{\xi}\perp\boldsymbol{b}$. Thus, Figure 6 shows an edge
dislocation.

In reality, screw and edge dislocations are extreme cases, while the most
general case of a dislocation type in a crystalline material is a “mixed
dislocation.” This is a dislocation with the line sense, $\boldsymbol{\xi}$,
neither parallel nor perpendicular to its Burgers vector, $\mathbf{b}$.

Since the atoms around dislocations are not positioned at the perfect lattice
points, the lattice is distorted near a dislocation. This distortion results
in a stress field in the crystalline material around the dislocation which is
the reason why dislocations move: they try to minimize energy. In the context
of plastic deformation, a dislocation is defined as the boundary of a slipped
area within which atoms are displaced by the size of an elementary unit
translation given by the Burgers vector.

In materials science, often the question arises on which “granularity level” a
dislocation should be defined. Clearly, if we are interested in phenomena on
the nanometer scale then we should resolve individual atoms (e.g., through
high-resolution transmission electron microscopy or molecular dynamics
simulations). When taking the _mesoscopic_ perspective, typically the
individual atoms can not be seen anymore and are not of interest (as, e.g.,
done through regular transmission electron microscopy or dislocations dynamics
simulations). However, the dislocation line itself is still observable: the
tube-like defect “region” is reduced to an idealized mathematical line, as
demonstrated in the right panel of Figure 7.

Figure 7: The idealization represents the dislocation in the mesoscale. Here,
the individual atoms are no longer visible. This idealization reduced the
tube-like defect “region” to a mathematical line. Note that the line on the
right does not correspond to the dislocation in the left (this would be a
vertical, straight line).

Therefore, the transition from the atomic scale to the mesoscale requires a
conceptual and mathematical idealization that significantly reduces the amount
of information. These idealizations require to be accompanied by further
details and definitions from the atomic scale, including the crystal
structure, the lattice, the lattice plane, and the lattice direction
information, all of which have an impact on the dislocation’s motion. For
example, the motion of a dislocation line through a crystal is constrained to
a specific crystallographic plane or lattice plane. Thus, it still requires
crystallographic information, even though the “defect region” is now only
represented as a mathematical line. These two different levels of information
require particular attention when designing the dislocation ontology.

The particular crystallographic or lattice plane constraining the dislocation
motion is called the _slip plane_ (see the green plane in Figure 8). There are
specific _slip directions_ , which are lattice directions along which plastic
deformation occurs within the slip plane, given by the Burgers vector. A _slip
system_ is a set of slip planes with the same unit normal vector and the same
slip direction. Thus, the unit normal vector and the slip direction or the
Burgers vector (where the latter is not a unit vector) determine the slip
system.

Figure 8: Depiction of the mathematical dislocation line on the mesoscale as a
mathematical object that has start and end points. The object is characterized
by the Burgers vector and the line sense. Furthermore, the dislocation motion
is constrained by the slip plane.

The mathematical representation of a mesoscale dislocation, as shown in Figure
8, is an oriented curve with a start point and an endpoint. The local line
orientation changes along the line, while the Burgers vector is constant for
each point. Since the dislocation is a directed curve, it has a line sense.
Unlike the local line orientation, it is a property of the whole line.

Various computational and experimental techniques are leveraged to predict and
observe dislocations in crystalline materials, some of which were already
mentioned above. For instance, high-resolution transmission electron
microscopy (or field ion microscopy) is used on the atomic scale to image the
arrangement of atoms. On the mesoscale, the focus is on examining the
characteristics of individual dislocations and analyzing the distribution,
arrangement, and density of dislocation in materials. Transmission Electron
Microscopy (TEM) and Discrete Dislocation Dynamics (DDD) simulations are
techniques for investigating these properties and simulating the dislocation
behaviour, respectively.

TEM is a microscopy technique that generates a highly-magnified image of a
material specimen. This technique involves an electron beam passing through
the specimen and several lenses. In strongly simplified terms, if the electron
beam hits an atom, then it is deflected. As a result of the deflection, the
intensity of the transmitted beam is reduced, and the intensity of the
diffracted beam is increased. The dislocation can be seen as a dark line in
such a bright-field image.

Above, it was already mentioned that the displaced atoms around a dislocation
result in stresses, because the atoms are no longer in their preferred
equilibrium position. Dislocations move in such stress fields which are mainly
described by the governing equations of (linear) elasticity theory. DDD
simulations employ mathematical lines (polygons or splines) to represent
dislocations, which are moved based on elastic interactions and further “local
rules”.

The numerical schemes used in DDD simulations require to numerically
discretize the mathematical line, e.g., by a number of straight line segments.
The discretization steps are illustrated in Figure 9. Further details can be
found in [30].

Figure 9: The discretization of dislocation to a numerical representation. The
oriented curve dislocation line shown in the left panel is discretized into a
number of segments shown in the right panel.

The discretization process can be easiest described based on the example of a
polygonal chain. There, the smooth mathematical line is approximated by a
polygonal chain, $\mathbf{C}$. $\mathbf{C}$ is a curve defined by a sequence
of points ($\mathbf{P}_{0}$, $\mathbf{P}_{1}$,…, $\mathbf{P}_{n}$), and these
points are called vertices. In addition, the curve consists of segments
connecting consecutive pairs of vertices. In general, we can define the shape
of a segment through the shape function which allows to have not only straight
line segments but also spline curves of different order.

##  4 The Dislocation Ontology

The dislocation ontology, DISO, is developed using several well-known ontology
development methodologies, such as [31]. The process is iterative, starting
with an initial version and continuously revising and refining the evolving
ontology. The development process is outlined in Figure 10, which includes the
main phases, their sub-tasks, and the roles involved.

Figure 10: The workflow of the dislocation ontology development, illustrating
the main phases, subprocesses, and roles involved in the whole process.

###  4.1 Metadata

It is essential to provide a systematic and comprehensive description of the
ontology, also known as ontology metadata, thus supporting its reusability and
findability [32]. When ontology metadata is missing, several potential issues
can occur. These include reduced accessibility for potential users, decreased
reusability, and ontologies not being recognized as relevant for specific use
cases. Accordingly, several DCMI Metadata
Terms999https://www.dublincore.org/specifications/dublin-core/dcmi-terms/ have
been added to the ontology, involving terms:contributor, terms:created,
terms:title, and vann:preferredNamespacePrefix.

###  4.2 Reuse of existing models

When developing an ontology, one of the first steps is to utilize or reuse
terms (i.e., classes or properties) from existing ontologies that describe the
same domain or subject matter. Deciding which ontologies are appropriate for
reuse is a challenging task for ontology engineers. Ontology reuse involves
several activities, including merging, extending, specializing, or adapting
other ontologies. In DISO, we reuse concepts from two related ontologies in
the MSE domain: the Crystal Structure Ontology101010https://purls.helmholtz-
metadaten.de/disos/cso (CSO) and the Crystalline Defect
Ontology111111https://purls.helmholtz-metadaten.de/disos/cdo (CDO). CSO
describes crystallographic data related to dislocations, while CDO links
physical material entities to crystal structures and different defect types
within a crystal, such as point defect, dislocation, and planar defect.

In CDO, the EMMO:Crystal class (from the EMMO121212https://github.com/emmo-
repo/domain-crystallography ontology) is reused to describe the physical
entity of crystalline materials. The CDO:CrystallineMaterial class is defined
as a subclass of EMMO:Crystal which is used to represent crystalline
materials.

In CSO, several MDO [28] classes are reused to describe the crystal coordinate
system, the motif in a crystal structure, point groups, and space groups.
Furthermore, the CSO defines the unit quantity of a property by reusing
several classes from QUDT (Quantities, Units, Dimensions and Data Types
Ontologies) [33]. Overall, the semantic data value of the developed ontology
increases as more ontologies are included, making the reuse of terms from
other ontologies a worthwhile undertaking [34].

###  4.3 Classes

Our ontology classes are separated into two groups: (i) those imported from
existing ontologies (as explained in subsection 4.2) and (ii) newly created
classes that are not already defined in any existing ontologies.

_Imported classes_. DISO reuses several classes from CSO: CSO:Lattice
represents the periodic arrangement of one or more atoms, and CSO:Vector
represents quantities with both magnitude and direction. Additionally, DISO
reuses classes from CDO, including CDO:CrystallographicDefect, which
represents lattice irregularity or lattice defects.

_Newly defined classes_. For new classes, we focus on specific classes of
crystalline materials and line defects, including 1) Dislocation, the focal
class in the DISO which represents a linear or one-dimensional defect that
causes some atoms to be displaced, 2) SlipPlane, which models the lattice
plane to which the dislocation is constrained to move in, 3) SlipDirection,
which models the lattice direction where the slip occurs in the crystalline
materials, 4) LatticePlane, which represents the lattice plane where it forms
an infinitely stretched plane that cuts through the lattice points, 5)
LatticeDirection, which models the direction inside the lattice that connects
two lattice points, and 6) DiscretizedLine, which provides a numerical
representation of the dislocation line as a mathematical line, such as an
oriented curve, that is discretized into several segments.

Figure 11: Core concepts and interconnected relationships in the DISO
ontology. Arrows with open arrowheads denote rdfs:subClassOf properties
between classes. Regular arrows represent rdfs:domain and rdfs:range
restrictions on properties and coloured boxes represent classes belonging to
different ontologies, e.g., yellow boxes represent DISO’s classes.

###  4.4 Properties

Similarly, both data and object properties in DISO are divided into two
categories which are newly defined properties and reused ones.

_Newly defined properties_. Object properties constitute the relationship
between various concepts in the ontology. For instance, the relationship
between TransmissionElectronMicroscopy and Dislocation classes can be
represented through the observedBy object property. Similarly, the
hasLineSense object property represents the relationship between Dislocation
and LineSense. Additionally, a number of data properties, including
directionMillerIndice and planeMillerIndice are defined, which typically
provide a relation to attaching an entity instance to some literal datatype
value, such as a string or a date.

_Reused properties_. Several properties from the reused ontologies have been
used, e.g., cso:hasPositionVector, cdo:hasCrystallographicDefect,
mdo:hasComposition, and emmo:hasProperty from the CSO ontology, CDO ontology,
MDO ontology, and EMMO ontology, respectively. Moreover, we reused several
data properties from DCterms for adding ontology metadata (see subsection
4.1). After defining new properties and identifying reused ones, the domain
and range for each property using rdfs:domain and rdfs:range are defined,
respectively. For instance, the domain of the data property
diso:planeMillerIndice is diso:LatticePlane and the range is xsd:string. while
the domain of the object property diso:hasLatticePoint is diso:LatticePlane
and the range is diso:LatticePoint.

Figure 12: DISO alignment with EMMO. The starting point to align the DISO with
the EMMO by importing the domain ontology crystallography developed by EMMO.

_Restricting properties_. In the DISO, several classes use property
restrictions, e.g., value constraints. For example, the resultsIn property
which connects Dislocation and LatticeDisplacement is restricted by a value
constraint of owl:someValuesFrom representing the fact that every dislocation
individual results in some or at least one lattice displacement individual(s).
The hasLineSense property which connects Dislocation and LineSense is
restricted by a value constraint of owl:allValuesFrom representing that every
dislocation individual can only have a line sense individual.

###  4.5 Reasoning

DISO’s inference capability is increased through the use of several property
characteristics, such as functional relations, transitivity, and the inverse
property [35]. hasMathematicalRepresentation is a functional property because
it means that a dislocation can be represented by exactly one mathematical
line, i.e., it can not have any different mathematical representation than
that. The transitive property can be demonstrated through the
hasRepresentation relationship. This relationship refers to the connection
between a dislocation and its representation. For instance, if a dislocation
has a line representation, and this line has a discretized line
representation, it can be inferred that the dislocation also has a discretized
line representation. In order to enable bidirectional navigation between two
classes in the ontology, inverse properties are established for each
corresponding property. For instance, the isSegmentOf property is the inverse
property of hasSegment. This means that if a discretized line A has a segment
B, then B is a segment of A.

##  5 Ontology Alignment

Ontology alignment is the process of identifying relations between entities
among different ontologies in order to establish connections between them
[36]. These entities include classes, properties, and individuals. For
successful ontology alignment, it is crucial to identify similarities between
source and target ontologies. The analysis entails examining concepts that
overlap but may have different names (i.e. synonyms) or types in the
ontologies [37].

This section will cover the extension of DISO, which involves aligning two
ontologies, namely EMMO and MDO. This alignment plays a crucial role in
allowing DISO to annotate the DDD data and transform it into linked data while
also facilitating knowledge graph generation.

Figure 13: DISO alignment with the MDO Core and Provenance module.

###  5.1 Alignment with EMMO

EMMO is a continuous initiative aimed at establishing semantic standards that
can be implemented at the highest level of abstraction. This makes it possible
for all potential domain ontologies, especially in the MSE field, to be
integrated and to work together seamlessly. Currently, EMMO consists of two
modules: a top-level and a mid-level module. The former includes the
fundamental axioms that constitute the philosophical foundation of the EMMO,
while the latter consists of a set of perspectives to develop more specialized
domain ontologies. These two ontologies serve as the basis for building
further domain and applications ontologies, e.g., the application of EMMO in
the domain of mechanical testing [38].

The starting point to align the DISO with EMMO is by aligning with the
Crystallography Domain Ontology131313https://github.com/emmo-repo/domain-
crystallography, a domain ontology based on EMMO and the CIF core
dictionary141414https://www.iucr.org/resources/cif/dictionaries/cif_core. As
shown in Figure 12, CDO:CrystallographicDefect subsumes Dislocation, while
also being a subclass of EMMO:Crystallographical class. Similarly,
EMMO:CrystalStructure, an equivalent class to CSO:CrystalStructure, is also a
subclass of EMMO:Crystallographical. Overall, EMMO:Crystallographical is a
class that ideally represents the physical concepts associated with
crystalline materials.

As we mentioned in section 3, on the mesoscale, a dislocation is represented
by a mathematical line, which can be further idealized as a pixel or
discretized line depending on the application (e.g., microscopy or
simulation). To align the dislocation mathematical line concept with an EMMO
class, EMMO:MathematicalModel subsumes Line and the discretized representation
of the mathematical dislocation line, DiscretizedLine, is subsumed
EMMO:Numerical.

###  5.2 Alignment with MDO

The MDO is a domain ontology that defines concepts and relations to cover the
knowledge of materials design, especially in the ab-initio calculation. MDO
consists of several modules, a Core, the Provenance module, and two domain-
specific modules: Structure and Calculation.

To align the DISO with the MDO, we reused several classes in the MDO Core
module. The MDO Core module describes the structure or the virtual specimen of
interest via MDO:Structure class. As shown in Figure 13, we defined a
DislocationStructure class as a subclass of MDO:Structure. This class
describes a dislocation (micro)structure, which is a virtual specimen used by
a DDD simulation to study the mechanical properties of a crystalline material.
Furthermore, DislocationStructure as an idealized representation relates to a
physical concept called CDO:CrystallineMaterial.

Figure 14: The core concepts in DISO ontology after the alignment with MDO and
EMMO. Arrows with open arrowheads denote rdfs:subClassOf properties between
classes, while regular arrows represent the relationships between them.
Classes that belong to the same ontology share the same color.

In the MDO Core module, an instance of the MDO:Structure class is used as a
virtual specimen input or output for a simulation. Here, the simulation
concept is represented as the MDO:Calculation class. We subsumed the
MDO:Calculation class to define the DDDSimulation class, which is a class to
describe the DDD simulation. Thus, the DDDSimulation can have
DislocationStructure as an input or an output. Moreover, the DDDSimulation has
an input and output relationship with MDO:Property to run a calculation. In
addition, the DDDSimulation is related to the DDDSimulationParameter, a
simulation parameter concept configuring the DDD simulation, e.g., the
activation parameter for cross-slip, junction formation, and external load.

Figure 15: Sample of dislocation microstructures used as input for simulation
as well as yielded by the simulation as output.

To preserve the provenance information of a DDD simulation, we reused several
classes from the MDO Provenance module and the PROV ontology [39]. Running a
DDD simulation requires specific software to solve materials science problems.
It is quite helpful to store information about the software used and its
version, as this can help scientists reuse data through post-processing
methods specific to DDD software. In this regard, DDDSimulation has a
relationship with PROV:SoftwareAgent, which has two data properties:
softwareVersion and MDO:softwareName. Furthermore, to preserve the provenance
information related to when a DDD simulation starts and ends, PROV:Activity
subsumed the DDDSimulation and inherited two data properties:
PROV:startedAtTime and PROV:endedAtTime. Apart from that, we reused
PROV:Person to annotate the person running or responsible for the simulation.
It has three data properties to define a person: FOAF:firstName,
FOAF:family_name, and MWO:hasORCID. The latter is a data property that we
reused from the MatWerk Ontology (MWO)151515hhttp://purls.helmholtz-
metadaten.de/mwo/.

To summarize, core concepts and interconnected relationships in DISO after the
alignment can be seen in Figure 14. The advantages of ontology alignment for
DISO are promoting knowledge transfer from other ontologies when describing
the DDD simulation. Furthermore, ontology alignment fosters interoperability
between ontologies in MSE-related domains. The objective is to assist in
building a knowledge graph for the dislocation domain.

##  6 The Dislocation Knowledge Graph

In the field of materials science, researchers use a numerical method called
DDD simulation to analyze the behaviour of dislocations within crystalline
materials. This technique helps identify the specific characteristics of each
dislocation, as well as their interaction, arrangement, and collective
behaviour within the material. The simulation observes the motion and
interaction of many dislocations which ultimately creates the relationship
between the microstructure, loading conditions, and the mechanical properties
of a crystalline material. For simulations of dislocations, there are various
software options available such as MoDELib [40], ParaDiS [41], and microMegas
[42]. Every software has a distinct collection of metadata that organizes the
inputs and outputs of the simulation.

DISO was utilized in this specific scenario to accurately annotate the
information collected from various DDD simulations. The ultimate goal is to
generate a comprehensive dislocation knowledge graph (DisLocKG) using this
data. The DDD data used in this work was generated through the MoDELib
software and took different initial dislocation densities and specimen sizes
into account. The cube-shaped Copper specimen, with an edge length of either
50 or 100 nanometers, was randomly filled with dipolar edge loops on all slip
systems until the initial density of either $1\cdot 10^{16}$ m-2 or $5\cdot
10^{16}$ m-2 was reached. A sample of the generated cube-shaped Copper
specimen can be seen on the left panel of the Figure 15. During the
simulation, the dislocation microstructure was allowed to relax without any
external load, meaning that internal stress and image forces solely influenced
the dislocation evolution. The simulation resulted in the relaxed dislocation
microstructure shown on the right-hand side of Figure 15. An important aspect
for a materials scientist is that some simulations do not have cross-slip or
junction formation. This has significant implications when it comes to
analyzing simulation results. E.g., Demirci et al. [43] investigated the
influence of cross-slip on the evolution of dislocation structures and
therefore could benefit from this information. Such relaxation calculations
are also important for creating a realistic microstructure. For example, Motz
et al. [44] investigated how the relaxed dislocation microstructure influences
the plasticity in subsequent tensile test simulations. Furthermore, several
authors [9, 45, 46, 47] conducted machine learning and data mining studies
utilizing the dislocation relaxed microstructure to classify the structure and
express the strain energy density of a dislocation microstructure,
respectively. This explains why there is a strong need for a detailed and
formal representation of such simulations – here, contained in the class of
DDDSimulation.

Figure 16: DDD simulation data as linked data. Colored rectangles on the left
depict data types in the DDD simulation: dislocation structure, provenance,
simulation parameters, and crystal structure data. The data subsequently is
linked using the DISO as a reference ontology. The rdflib Python module
supports data linking and generates the DisLocKG. Via the SPARQL Endpoint, end
users can query the data to retrieve the information in the DisLocKG.

For our example, we have collected a total of 25 data points – where each data
point is one DDD simulation consisting of initial and final microstructure.
Each of those was annotated with DISO. Any data point gives information about
the simulation details, such as parameters used for a simulation, initial
dislocation microstructure used as input, and the resulting dislocation
microstructure produced by the simulation. Additionally, each dislocation
microstructure includes information about the crystal structure, Bravais
lattice, dislocation, slip plane, Burgers vector, and numerical representation
of dislocation. The results of the simulations are stored and parsed in the
HDF5161616https://www.hdfgroup.org/solutions/hdf5/ format. Subsequently, we
utilize our in-house Python scripts (using the rdflib 6.0 [48] Python library)
to create a knowledge graph called DisLocKG from this data using DISO as a
reference ontology (cf. Figure 16). The DisLocKG is a semantic network that
holds information about dislocations in crystalline materials. Note that,
DisLocKG also stores the provenance information related to the data,
particularly the creator data, software, and software version used to generate
the data. In total, we have generated a number of $\sim 2.2$M triples that are
stored as RDF files which are available via its persistent
identifier171717https://purls.helmholtz-metadaten.de/dislockg.

Figure 17: A visual representation of CQ3 results. Colored boxes represent
classes and the red dot represents an individual belonging to that class. Each
individual is defined by a directed arrow having the rdf:type relationship to
the respective class and connected to other individuals by object properties.

Publishing DDD data as linked data has several benefits [49], including 1)
establishing links between dislocation-related datasets, enabling machines to
understand and discover new information, 2) supporting semantic querying via
the SPARQL query language, 3) supporting data enrichment, where machines can
infer implicit knowledge that does not exist, and 4) promoting semantic
validation of the data, ensuring consistency and accuracy.

We have listed some competency questions in Table 1 to give an idea of the
vast information available in DisLocKG. For instance, CQ1 can retrieve the
history and origin information of DDD data generated by the MoDELib software,
and CQ2 and CQ3 can retrieve information on the specimen geometry and the
initial dislocation of each dislocation simulation. These CQs are important if
one wants to query a dislocation simulation to be reused for the processing
step if they need a specific density of a dislocation structure and
information concerning the geometry. CQ4 retrieves the input parameters to run
the simulation, while CQ5 queries all dislocation structures generated by the
relaxation calculation. The SPARQL query corresponding to CQ3 is shown in 1,
and the complete set of the competency questions and the corresponding SPARQL
queries can also be found in the DISO GitHub repository. Figure 17 visualizes
the results of CQ3, which contains three individuals (shown as the red
markers) of the DDD simulation class. Each of the DDD simulation individuals
has a relationship with dislocation structure individuals. Moreover, the
dislocation density data relates to the dislocation structure individual.

Table 1: A sample of competency questions for DisLocKG. No. |  Question
---|---
CQ1 |  Provide detailed information on the dislocation structures simulated using the MODELIB software, including the software version and creator associated with these simulations.
CQ2 |  Which dislocation structures possess a specimen shape resembling a cube with an edge length greater than 30 nanometers?
CQ3 |  List all DDD simulations that have an initial density of dislocation $=5e16$ m-2
CQ4 |  List all DDD simulations that do not activate the cross slip formation and junction formation
CQ5 |  What are dislocation structures generated by the relaxation calculation? List also the initial density of a dislocation structure used for a relaxation calculation, simulation parameters: cross-slip activation, junction formation activation, and external load activation.
Listing 1: SPARQL query corresponding to CQ3.

⬇

PREFIX diso:<https://purls.helmholtz-metadaten.de/disos/diso#>

PREFIX qudt:<http://qudt.org/schema/qudt/>

PREFIX xsd:<http://www.w3.org/2001/XMLSchema#>

PREFIX mdo_core:<https://w3id.org/mdo/core/>

SELECT ?ddd_simulation ?initial_density ?unit

WHERE{

?ddd_simulation diso:hasInputDislocationStructure ?input_dislocstructure.

?initial_density_qu mdo_core:relatesToStructure ?input_dislocstructure;

qudt:quantityValue ?density_qv.

?density_qv qudt:numericalValue ?initial_density;

qudt:unit ?unit.

FILTER(?initial_density = "5e16"^^xsd:double)

}

##  7 Evaluation

Employing predefined metrics that evaluate an ontology’s richness through
criteria-based assessment is one way of evaluating its quality [34]. In this
section, we evaluate the adapted version of DISO using the OntoQA [50]
evaluation model.

Ontology | C | SC | AT | P | RR | AR | IR
---|---|---|---|---|---|---|---
MDO | 37 | 49 | 32 | 32 | 0.40 | 0.86 | 1.32
CSO | 30 | 49 | 19 | 25 | 0.34 | 0.63 | 1.63
DISO v1.0 | 33 | 62 | 12 | 33 | 0.35 | 0.32 | 1.63
DISO v1.1 | 70 | 116 | 47 | 80 | 0.41 | 0.67 | 1.66
Table 2: Evaluation of DISO compared to DISO v1.0, MDO and CSO using the
OntoQA model. C is the number of classes, SC is the number of sub-classes, AT
is the number of attributes, and P denotes the number of relationships.

This model can assess an ontology based on two dimensions: Schema and
Instances. Here, we focus on the schema evaluation which evaluates the quality
of the ontology’s design. We determine the effectiveness of the ontology and
its ability to represent rich knowledge using the following metrics:

  * •

_Relationship richness (RR)_ shows the diversity of relations and placement of
relations in the ontology (Equation 2).

$RR=\frac{|P|}{|SC|+|P|}$ (2)

where P is the number of relationships and SC is the number of sub-classes.
The more relations an ontology owns, the richer it is (_is-a_ relations are
not considered).

  * •

_Attribute richness (AR)_ shows that the more attributes are defined the more
knowledge the ontology delivers (Equation 3).

$AR=\frac{|AT|}{|C|}$ (3)

where AT is the number of attributes for all classes and C is the number of
classes.

  * •

_Inheritance richness (IR)_ describes the distribution of information across
different levels of the ontology inheritance tree. IR indicates how knowledge
is classified into different classes and subclasses in an ontology. (Equation
4).

$IR=\frac{|SC|}{|C|}$ (4)

In Table 2, we compare the evaluation outcomes of DISO with MDO [28],
CSO181818https://purls.helmholtz-metadaten.de/disos/cso and the previous
version of DISO. DISO has the most significant value of RR, which implies that
it has a greater relation diversity. Moreover, DISO has the highest IR value,
representing a more comprehensive knowledge range than MDO, CSO, and the
previous version of DISO. The AR value of DISO is lower than that of MDO and
higher than CSO and the previous version of DISO. To conclude, DISO possesses
the most extensive knowledge representation and diversity in terms of
relationships, achieving the highest IR and RR, respectively. Moreover, the
adapted version of DISO surpasses its predecessor in all evaluation metrics.

##  8 Conclusion and Outlook

This paper showcases how semantic web technologies can be utilized to
transform unstructured DDD data into well-organized and structured data.
Furthermore, we extended the dislocation ontology by aligning it with commonly
used materials science ontologies (i.e., EMMO and MDO core) to be able to
model simulation data efficiently. Moreover, we presented a real-world use
case that utilized the DISO to construct a semantic network of DDD data (i.e.,
linked data) called DisLocKG, where individual entities are connected,
enabling semantic query and supporting intelligent tasks. To support querying
DisLocKG, the graph has been made publicly available and steps to set up a
SPARQL endpoint are described in its GitHub repository. The evaluation results
indicate that the adapted version of DISO is the most comprehensive and
diverse knowledge representation among the state-of-the-art ontologies.

In the future, we plan to improve DISO by modelling the linear elasticity
theory of dislocations and extending the real-world use case, e.g., another
DDD simulation software and TEM data. In addition, developing the DisLocKG
Application Programming Interface (API) will also be a worthwhile undertaking.
The idea is to develop several interactive features, e.g., querying, data
mining, visualizing, updating, and deleting data with the DisLocKG via its
API. DISO and DisLocKG will continue to be maintained and extended in the
context of the Helmholtz Metadata Collaboration (HMC) and NFDI-MatWerk efforts
of facilitating machine readability and reuse of research data.

Acknowledgments AI and SS acknowledge financial support from the European
Research Council through the ERC Grant Agreement No. 759419 MuDiLingo (”A
Multiscale Dislocation Language for Data-Driven Materials Science”). AI, SF,
and SS acknowledge Helmholtz Metadata Collaboration (HMC) within the Hub
Information at the Forschungszentrum Jülich (FZJ). We are grateful to Aytekin
Demirci for the MoDELib parser that we used.

## Declarations

Conflict of interest The authors have no conflicts of interest to declare.

## References

  *   * Taylor [1934] Taylor, G.I.: The mechanism of plastic deformation of crystals. part I.–Theoretical. Proceedings of the Royal Society of London. Series A, Containing Papers of a Mathematical and Physical Character 145(855), 362–387 (1934)
  * Polanyi [1934] Polanyi, M.: Über eine Art Gitterstörung, die einen Kristall plastisch machen könnte. Zeitschrift für Physik 89(9-10), 660–664 (1934)
  * Callister and Rethwisch [2018] Callister, W.D., Rethwisch, D.G.: Materials Science and Engineering: an Introduction vol. 9. Wiley New York, ??? (2018)
  * Murakumo et al. [2004] Murakumo, T., Kobayashi, T., Koizumi, Y., Harada, H.: Creep behaviour of ni-base single-crystal superalloys with various $\gamma$’ volume fraction. Acta Materialia 52(12), 3737–3744 (2004)
  * Govind et al. [2023] Govind, K., Oliveros, D., Dlouhy, A., Legros, M., Sandfeld, S.: Deep learning of crystalline defects from tem images: A solution for the problem of” never enough training data”. arXiv preprint arXiv:2307.06322 (2023)
  * Bertin and Zhou [2023] Bertin, N., Zhou, F.: Accelerating discrete dislocation dynamics simulations with graph neural networks. Journal of Computational Physics 487, 112180 (2023)
  * Zhang et al. [2022] Zhang, C., Song, H., Oliveros, D., Fraczkiewicz, A., Legros, M., Sandfeld, S.: Data-mining of in-situ tem experiments: On the dynamics of dislocations in cocrfemnni alloys. Acta Materialia 241, 118394 (2022)
  * Yang et al. [2020] Yang, Z., Papanikolaou, S., Reid, A.C., Liao, W.-k., Choudhary, A.N., Campbell, C., Agrawal, A.: Learning to predict crystal plasticity at the nanoscale: Deep residual networks and size effects in uniaxial compression discrete dislocation simulations. Scientific reports 10(1), 8262 (2020)
  * Song et al. [2021] Song, H., Gunkelmann, N., Po, G., Sandfeld, S.: Data-mining of dislocation microstructures: concepts for coarse-graining of internal energies. Modelling and Simulation in Materials Science and Engineering 29(3), 035005 (2021)
  * Salmenjoki et al. [2018] Salmenjoki, H., Alava, M.J., Laurson, L.: Machine learning plastic deformation of crystals. Nature communications 9(1), 5307 (2018)
  * Prakash and Sandfeld [2018] Prakash, A., Sandfeld, S.: Chances and challenges in fusing data science with materials science. Practical Metallography 55(8), 493–514 (2018) https://doi.org/10.3139/147.110539
  * Adamovic et al. [2020] Adamovic, N., Friis, J., Goldbeck, G., Hashibon, A., Hermansson, K., Hristova‐Bogaerds, D., Koopmans, R., Wimmer, E.: The emmc roadmap for materials modelling and digitalisation of the materials sciences (2020) https://doi.org/10.5281/zenodo.4272033
  * Wilkinson et al. [2016] Wilkinson, M.D., Dumontier, M., Aalbersberg, I.J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., Silva Santos, L.B., Bourne, P.E., et al.: The fair guiding principles for scientific data management and stewardship. Scientific data 3(1), 1–9 (2016)
  * McBride [2004] McBride, B.: The resource description framework (rdf) and its vocabulary description language rdfs. Handbook on ontologies, 51–65 (2004)
  * Bechhofer et al. [2004] Bechhofer, S., Van Harmelen, F., Hendler, J., Horrocks, I., McGuinness, D.L., Patel-Schneider, P.F., Stein, L.A., et al.: Owl web ontology language reference. W3C recommendation 10(2), 1–53 (2004)
  * Pérez et al. [2006] Pérez, J., Arenas, M., Gutierrez, C.: Semantics and complexity of sparql. In: The Semantic Web-ISWC 2006: 5th International Semantic Web Conference, ISWC 2006, Athens, GA, USA, November 5-9, 2006. Proceedings 5, pp. 30–43 (2006). Springer
  * Ihsan et al. [2023] Ihsan, A.Z., Fathalla, S., Sandfeld, S.: Diso: A domain ontology for modeling dislocations in crystalline materials. In: Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing. SAC ’23, pp. 1746–1753. Association for Computing Machinery, New York, NY, USA (2023). https://doi.org/10.1145/3555776.3578739
  * Ihsan et al. [2021] Ihsan, A.Z., Dessi, D., Alam, M., Sack, H., Sandfeld, S.: Steps towards a dislocation ontology for crystalline materials. In: 2nd International Workshop on Semantic Digital Twins, SeDiT 2021, vol. 2887 (2021). CEUR-WS
  * Berrueta et al. [2008] Berrueta, D., Fernández, S., Frade, I.: Cooking http content negotiation with vapour. In: Proceedings of 4th Workshop on Scripting for the Semantic Web (SFSW2008), vol. 72 (2008). Citeseer
  * Loibl et al. [2020] Loibl, A., Manoharan, T., Nagarajah, A.: Procedure for the transfer of standards into machine-actionability. Journal of Advanced Mechanical Design, Systems, and Manufacturing 14(2), 0022–0022 (2020)
  * Say et al. [2020] Say, A., Fathalla, S., Vahdati, S., Lehmann, J., Auer, S.: Semantic representation of physics research data. In: Proceedings of the 12th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (2020)
  * Hu et al. [2011] Hu, S., Wang, H., She, C., Wang, J.: Agont: ontology for agriculture internet of things. In: Computer and Computing Technologies in Agriculture IV: 4th IFIP TC 12 Conference, CCTA 2010, Nanchang, China, October 22-25, 2010, Selected Papers, Part I 4, pp. 131–137 (2011). Springer
  * Say et al. [2020] Say, Z., Fathalla, S., Vahdati, S., Lehmann, J., Auer, S.: Ontology design for pharmaceutical research outcomes. In: International Conference on Theory and Practice of Digital Libraries, pp. 119–132 (2020). Springer
  * Mrdjenovich et al. [2020] Mrdjenovich, D., Horton, M.K., Montoya, J.H., Legaspi, C.M., Dwaraknath, S., Tshitoyan, V., Jain, A., Persson, K.A.: Propnet: a knowledge graph for materials science. Matter 2(2), 464–480 (2020)
  * Zhao et al. [2021] Zhao, X., Greenberg, J., McClellan, S., Hu, Y.-J., Lopez, S., Saikin, S.K., Hu, X., An, Y.: Knowledge graph-empowered materials discovery. In: 2021 IEEE International Conference on Big Data (Big Data), pp. 4628–4632 (2021). IEEE
  * Jain et al. [2013] Jain, A., Ong, S.P., Hautier, G., Chen, W., Richards, W.D., Dacek, S., Cholia, S., Gunter, D., Skinner, D., Ceder, G., Persson, K.a.: The Materials Project: A materials genome approach to accelerating materials innovation. APL Materials 1(1), 011002 (2013) https://doi.org/10.1063/1.4812323
  * Ashino [2010] Ashino, T.: Materials ontology: An infrastructure for exchanging materials information and knowledge. Data Science Journal 9, 54–61 (2010)
  * Li et al. [2020] Li, H., Armiento, R., Lambrix, P.: An ontology for the materials design domain. In: International Semantic Web Conference, pp. 212–227 (2020). Springer
  * McCusker et al. [2020] McCusker, J.P., Keshan, N., Rashid, S., Deagen, M., Brinson, C., McGuinness, D.L.: Nanomine: A knowledge graph for nanocomposite materials science. In: International Semantic Web Conference, pp. 144–159 (2020). Springer
  * Ghoniem and Sun [1999] Ghoniem, N.M., Sun, L.: Fast-sum method for the elastic field of three-dimensional dislocation ensembles. Physical Review B 60(1), 128 (1999)
  * Suárez-Figueroa et al. [2015] Suárez-Figueroa, M.C., Gómez-Pérez, A., Fernandez-Lopez, M.: The neon methodology framework: A scenario-based methodology for ontology development. Applied ontology 10(2), 107–145 (2015)
  * Simperl et al. [2011] Simperl, E., Sarasua, C., Ungrangsi, R., Bürger, T.: Ontology metadata for ontology reuse. International Journal of Metadata, Semantics and Ontologies 6(2), 126–145 (2011) https://doi.org/10.1504/IJMSO.2011.046579
  * Hodgson et al. [2014] Hodgson, R., Keller, P.J., Hodges, J., Spivak, J.: Qudt-quantities, units, dimensions and data types ontologies. USA Available http://qudt. org March 156 (2014)
  * Fathalla et al. [2019] Fathalla, S., Vahdati, S., Lange, C., Auer, S.: Seo: A scientific events data model. In: Ghidini, C., Hartig, O., Maleshkova, M., Svátek, V., Cruz, I., Hogan, A., Song, J., Lefrançois, M., Gandon, F. (eds.) The Semantic Web – ISWC 2019, pp. 79–95. Springer, Cham (2019)
  * Fathalla et al. [2023] Fathalla, S., Lange, C., Auer, S.: An upper ontology for modern science branches and related entities. In: European Semantic Web Conference, pp. 436–453 (2023). Springer
  * Ehrig [2006] Ehrig, M.: Ontology Alignment: Bridging the Semantic Gap vol. 4. Springer, ??? (2006)
  * Noy et al. [2000] Noy, N.F., Musen, M.A., et al.: Algorithm and tool for automated ontology merging and alignment. (2000)
  * Morgado et al. [2020] Morgado, J.F., Ghedini, E., Goldbeck, G., Hashibon, A., Schmitz, G.J., Friis, J., Baas, A.: Mechanical testing ontology for digital-twins: A roadmap based on emmo. SeDiT 2020: Semantic Digital Twins 2020, 3 (2020)
  * Lebo et al. [2013] Lebo, T., Sahoo, S., McGuinness, D., Belhajjame, K., Cheney, J., Corsar, D., Garijo, D., Soiland-Reyes, S., Zednik, S., Zhao, J.: Prov-o: The prov ontology. w3c recommendation. World Wide Web Consortium (2013)
  * Po and Ghoniem [2014] Po, G., Ghoniem, N.: A variational formulation of constrained dislocation dynamics coupled with heat and vacancy diffusion. Journal of the Mechanics and Physics of Solids 66, 103–116 (2014) https://doi.org/10.1016/j.jmps.2014.01.012
  * Arsenlis et al. [2007] Arsenlis, A., Cai, W., Tang, M., Rhee, M., Oppelstrup, T., Hommes, G., Pierce, T.G., Bulatov, V.V.: Enabling strain hardening simulations with dislocation dynamics. Modelling and Simulation in Materials Science and Engineering 15(6), 553 (2007)
  * Devincre et al. [2011] Devincre, B., Madec, R., Monnet, G., Queyreau, S., Gatti, R., Kubin, L.: Modeling crystal plasticity with dislocation dynamics simulations: The ‘micromegas’ code. Mechanics of Nano-objects 1, 81–100 (2011)
  * Demirci et al. [2023] Demirci, A., Steinberger, D., Stricker, M., Merkert, N., Weygand, D., Sandfeld, S.: Statistical analysis of discrete dislocation dynamics simulations: initial structures, cross-slip and microstructure evolution. Modelling and Simulation in Materials Science and Engineering (2023)
  * Motz et al. [2009] Motz, C., Weygand, D., Senger, J., Gumbsch, P.: Initial dislocation structures in 3-d discrete dislocation dynamics and their influence on microscale plasticity. Acta Materialia 57(6), 1744–1754 (2009)
  * Fan et al. [2021] Fan, H., Wang, Q., El-Awady, J.A., Raabe, D., Zaiser, M.: Strain rate dependency of dislocation plasticity. Nature communications 12(1), 1845 (2021)
  * Steinberger et al. [2019] Steinberger, D., Song, H., Sandfeld, S.: Machine learning-based classification of dislocation microstructures. Frontiers in Materials 6, 141 (2019) https://doi.org/10.3389/fmats.2019.00141
  * Stricker et al. [2018] Stricker, M., Sudmanns, M., Schulz, K., Hochrainer, T., Weygand, D.: Dislocation multiplication in stage ii deformation of fcc multi-slip single crystals. Journal of the Mechanics and Physics of Solids 119, 319–333 (2018)
  * Boettiger [2018] Boettiger, C.: Rdflib: A High Level Wrapper Around the Redland Package for Common Rdf Applications. (2018). https://doi.org/10.5281/zenodo.1098478
  * Fathalla et al. [2019] Fathalla, S., Lange, C., Auer, S.: Eventskg: A 5-star dataset of top-ranked events in eight computer science communities. In: European Semantic Web Conference, pp. 427–442 (2019). Springer
  * Tartir et al. [2005] Tartir, S., Arpinar, I.B., Moore, M., Sheth, A.P., Aleman-Meza, B.: Ontoqa: Metric-based ontology quality analysis (2005)


# Modeling Mistrust in End-of-Life Care

Willie Boag  Harini Suresh  Leo Anthony Celi  Peter Szolovits  Marzyeh
Ghassemi

###### Abstract

In this work, we characterize the doctor-patient relationship using a machine
learning-derived trust score. We show that this score has statistically
significant racial associations, and that by modeling trust directly we find
stronger disparities in care than by stratifying on race. We further
demonstrate that mistrust is indicative of worse outcomes, but is only weakly
associated with physiologically-created severity scores. Finally, we describe
sentiment analysis experiments indicating patients with higher levels of
mistrust have worse experiences and interactions with their caregivers. This
work is a step towards measuring fairer machine learning in the healthcare
domain.

##  1 Introduction

There are well-established gaps in the American healthcare system for minority
populations. Groups that have been historically marginalized have also had
worse treatment options and longitudinal health outcomes. Biases are
especially troubling in the context of machine learning applied to clinical
data. Bias can be replicated and exacerbated in the model’s future
recommendations (Ensign et al., 2017). For example, black and Hispanic
patients are often given less pain medication for equivalent injuries and
reported pain levels (Goyal et al., 2015; Singhal et al., 2016). If this
pattern is present in the training data for a model built to recommend
treatment, it would learn to associate race with pain medication dosage.

Differences in care have also been established during end-of-life (EOL), when
critically ill patients are confronting death (Muni et al., 2011; Lee et al.,
2016). Previous work has suggested that medical disparities can reflect higher
levels of mistrust for the healthcare system among black patients. It is said
that blacks are more suspicious of the clinical motives in advance directives
and do-not-resuscitate (DNR) orders (Wunsch et al., 2010), and believe that
the healthcare system was controlling which treatments they can receive
(Perkins et al., 2002). When the doctor-patient relationship lacks trust,
patients may believe that limiting any intensive treatment is unjustly
motivated, and demand higher levels of aggressive care. While there are
clinical examples of exemplary end-of-life care, studies have highlighted that
aggressive care can lead to painful final moments, and may not improve patient
outcomes (Cipolletta & Oprandi, 2014).

Prior works in the FATML community have attempted to mask out features that
may lead to disparate treatment (Zemel et al., 2013), but including
information about race may be important for some clinical tasks (e.g., if
there are differences in recommended care by genetic makeup). In such a
setting, quantifying bias and establishing proxy measures for medical trust is
particularly important.

In this work, we present three contributions.

  * •

We present a trust metric derived from coded doctor-patient interactions.

  * •

We demonstrate that our trust score captures treatment differences by showing
disparities in end-of-life care are pronounced when patients are stratified on
trust.

  * •

We validate our mistrust metric using sentiment analysis of patients’ notes.

For further analysis – including the analysis of three different proxy scores
for trust – see (Boag, 2018).

##  2 Background and Related Work

The quantity of health-related data is increasing rapidly, from genetic data
to medical images like x-rays (Kruse et al., 2016; Raghupathi & Raghupathi,
2014). These rapid advancements have facilitated large-scale machine learning
methods to guide care. Ferryman & Pitcan give an overview of fairness issues
that may arise with such advances in personalized medicine. However, further
research into these risks and the feasibility of applying existing FATML work
to healthcare domains is limited.

Socialized mistrust of the medical community in minority groups has been
established as a factor in care differences (Washington, 2007). Family members
of African American patients are more likely to cite absent or problematic
communication with physicians about EOL care (Hauser et al., 1997). Similarly,
in surveys, African Americans report lower rates of satisfaction with the
quality of care that they received by physicians (Hanchate et al., 2009). In
end-of-life care, a mistrustful patient could be more resistant to a doctor’s
recommendation of comfort-based care, and instead insist receiving all
possible treatments even if they are overly aggressive (Garrett et al., 1993;
Hopp & Duffy, 2000).

Trust is difficult to quantify, and shaped by subtle interactions such as
perceived discrimination, racial discordance, poor communication, language
barriers, unsatisfied expectations, cultural stigmas and reputations, and more
(L. Whaley, 2001). Trust is very important to success of a hospital stay;
previous work has found that increased levels of doctor-patient trust were
associated with stronger adherence to a physician’s advice, increased patient
satisfaction and improved health status (Gelb-Safran et al., 1998).

Previous efforts to create trust-based measures that correlate with outcomes
have relied on surveys, which can be difficult to conduct for both theoretical
(selection bias) and practical (cannot be done for retrospective, de-
identified data) concerns (Lee et al., 2016).

##  3 Data

We use the publicly-available Medical Information Mart for Intensive Care
(MIMIC-III) v1.4 (Johnson et al., 2016). This database contains de-identified
EHR data from over 58,000 hospital admissions for nearly 38,600 adult
patients. The data was collected from Beth Israel Deaconess Medical Center
from 2001–2012. We examine a cohort of black and white patients in end-of-life
care. We examine patients who have a hospital stay which lasted at least 6
hours, and have either died in the hospital, were discharged to hospice, or
were discharged to a skilled nursing facility111This was done because the
notes indicate some SNF patients are discharged on hospice without coding that
into the EHR. (SNF). These experiments are repeated on a stricter definition
of an EOL cohort (which excludes SNF patients) in (Boag, 2018), shows the same
trends as this work, but with less statistical power because of smaller sample
sizes. Both our data extraction and modelling code are made
available222https://github.com/wboag/eol-mistrust to enable reproducibility
and further study (Johnson et al., 2017).

In order to measure disparities in aggressive end-of-life procedures, we
extracted treatment durations (in minutes) from MIMIC’s derived mechanical
ventilation (ventdurations) and vasopressor (vasopressordurations) tables
333Available freely at https://github.com/MIT-LCP/mimic-
code/tree/master/concepts/durations.. Due to the noisiness of clinical
measurements,444for instance, when one treatment span is erroneously coded as
two back-to-back smaller spans we merge any treatment spans that occurred
within 10 hours of each other.555This heuristic was suggested by MIMIC staff
because 10 hours is approximately the shift of a nurse, and treatment duration
events might get recorded once at the beginning of each shift. If a patient
had multiple spans, such as an intubation-extubation-reintubation, then we
consider the patient’s treatment duration to be the sum of the individual
spans.

In this work, we wish to better understand and quantify the nuances of a
patient’s interactions with their nurses and doctors. We accomplish this using
two sources: clinical notes and coded chart events. We obtain the notes of
every patient who had a stay of at least 12 hours in the ICU. This resulted in
48,273 admissions and over 800,000 notes. Most notes are nursing notes,
discharge summaries, physician notes, and social worker notes. To supplement
this narrative prose, we also extract coded information from the MIMIC
chartevents table, which records many interpersonal aspects of the patient’s
stay, including: code status, health literacy (e.g. whether there is a
healthcare proxy), behavioral and mental status assessments, family
communications, pain management, whether the patient was restrained, whether
the patient wanted help bathing, support services, and more.

##  4 Methods and Experiments

We aim to replicate previously demonstrated racial disparities in end-of-life
care using MIMIC-III (Johnson et al., 2016). We take as reference a set of
three recent papers which examined the racial disparities in end-of-life care
for nonwhite or minority populations (Yarnell et al., 2017; Muni et al., 2011;
Lee et al., 2016). We compared the differences of patient outcomes between
white and black populations using Mann-Whitney analysis for non-normally
distributed variables (treatment durations, mistrust metric scores). In
accordance with prior work, we consider p-values $<.05$ to be statistically
significant.

###  4.1 Establishing a Medical Mistrust Metric

Ideally, the gold standard for measuring trust would be a survey where the
patient – in their own words – describes their feelings and relationship with
their caregiver (Gelb-Safran et al., 1998). However, such surveys were not
recorded for MIMIC patients. But we believe that there is still a useful
signal of the trust which can be inferred from the EHR. We quantify mistrust
in a novel way by seeding a supervised machine learning task with labels which
serve as a proxy for mistrust. Our goal is to model the underlying
relationship and create a trust score which aims to explain treatment
disparities better than race does.

We extract coded interpersonal features from the chartevents table for all
MIMIC patients. Some of the information extracted includes: indication of
family meetings, patient education, whether the patient needed to be
restrained, how thoroughly pain is being monitored and treated, healthcare
literacy (e.g. whether the patient has a healthcare proxy), whether the
patient has a support system (such as family, social workers, and religion),
and agitation scales (Riker-SAS and Richmond-RAS). In total, we extract 620
unique binary indicators.

We use a simple rule-based search through the notes to determine whether the
patient has non-compliance documented somewhere in their notes (e.g. medical
advice, regimens, follow-ups, etc.). Noncompliance indicates a very overt
mistrust; rather than just holding an unspoken resentment, the patient
actually defies their doctor’s orders. Because crossing this line explicitly
demonstrates that the patient is willing to disregard physician decisions, it
is a suitable prediction target for training the model to quantify mistrust.

Of the 48,273 hospital admissions, we find 464 with notes that document
noncompliance, and fit an L1-regularized Logistic Regression666http://scikit-
learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html
model with chartvents features to predict whether the patient was
noncompliant. Once the model is trained, we use the classifier’s predicted
probability for a new patient as a proxy for their degree of mistrust.

###  4.2 Validating the Mistrust Metric

Throughout a patient’s stay, caregivers write narrative prose notes to
document administered care, family meetings, patient preferences, reminders,
warnings, and the patient’s quality of care. In documenting their impressions
of how to best understand and interact with their patients, caregivers can
give clues into their relationship with the patient and family.

Clinical notes have been used for prediction tasks in previous work (Ghassemi
et al., 2014) but not for investigating mistrust. Sentiment analysis of
clinical notes has also been used to measure whether one group of patients has
a better experience, on average, than another group (McCoy et al., 2015). We
use the Pattern software package for sentiment analysis (De Smedt & Daelemans,
2012).

For a given hospital admission, we compute the sentiment score of the
concatenation of that patient stay’s notes. Once we compute the score for each
stay in our population, we scale the distribution of scores to be zero-mean
and unit-variance in order to give better sense to the differences in
sentiment, relative to average. Of particular interest is the differences in
sentiment between different groups: white-and-black; trustful-and-mistrustful.
As a sanity check, we hypothesize that sick patients have more negative stays
than healthy patients, so we also stratify into low- and high-risk
subpopulations using the Oxford Acute Severity of Illness Score (OASIS) score
(E W Johnson et al., 2013). We create subpopulations to be same size as
black/white split, i.e. the white:black dataset size ratio is the same as the
trustful:mistrustful dataset size ratio.

##  5 Results

###  5.1 Creation of a Mistrust Metric

Table 1: Top-3 most positively and negatively informative chartevent features for tuning the mistrust metric. feature | weight
---|---
state: alert | -1.0156
riker-sas scale: agitated | 0.7013
pain: none | -0.5427
richmond-ras scale: 0 alert and calm | -0.3598
education readiness: no | 0.2540
pain level: 7-mod to severe | 0.2168

Table 1 shows the three most positively and most negatively informative
weights used to predict a mistrust metric (Section 4.1). The features align
well with our intuitive notion of mistrust: patients who are agitated and not
receptive to education are more likely to be mistrustful, whereas calm, pain-
free patients are more willing to trust their doctor.

We observe a statistically significant racial disparity in the mistrust
metric, where the median black patient has a higher level of mistrust than the
median white patient using the Mann-Whitney test (p=0.003). This is not
surprising, given the extensive literature investigating differences in
iatrophobia by race (Washington, 2007).

###  5.2 Significant Disparities in EOL Care

####  5.2.1 Race-Based Disparities

We demonstrate racial treatment disparities in the MIMIC dataset. Figure 1(b)
highlights the differences in white and black populations for aggressive
treatment durations. Figures 1(b)a and 1(b)b show that for both mechanical
ventilation and vasopressors, the median black patient receives a longer
duration of treatment, perhaps suggesting a reluctance to transition to
palliative care. While these results only show statistical significance for
ventilation, the same trends are also observable for vasopressor
administration.

####  5.2.2 Trust-Based Disparities

Figure 1: We observe racial disparities in for black patients (when compared
to white patients) for the duration of aggressive interventions (vasopressors
and ventilation). Medians are indicated by dotted lines; differences are
significant ($p<0.05$) for ventilation but not for vasopressors.

(a) CDF of ventilation duration by race ($p=.005$).

(b) CDF of vasopressor duration by race ($p=0.12$).

Figure 2: Stratifying patients by mistrust directly shows starker disparities
in care than race. Medians for ventilation and vasopressor durations are
indicated with dotted lines and all differences between the groups are
significant.

(a) CDF of ventilation duration ($p<.0001$).

(b) CDF of vasopressor duration ($p<.0001$).

Using the mistrust metric, we can rank the patients by trust score and
stratify them into two groups: low- and high-mistrust. Figure 2(b) revisits
the experiments from Figure 1(b) except stratified into low and high mistrust
instead of white and black populations.777For each treatment, we preserve the
same size difference of stratified groups in order to maintain consistency in
sample sizes for significance testing, e.g,. because the black group contains
510 patients for ventilation, we compare the 510 lowest trust patients against
the 4811 highest trust patients. We can see from Figure 2(b)b that trust-based
disparities in vasopressor durations are significant. The difference between
medians of each group is 650 minutes for vasopressors (whereas the difference
stratified by race was 200 minutes). This gap is even larger for ventilation
durations, as shown in Figure 2(b)a: the trust-based stratification shows a
2150-minute difference between medians, in contrast to the 550-minute gap for
the race split in Figure 1(b)a.

###  5.3 Low Trust Patients Have The Most Negative Notes

Table 2 shows the differences in sentiment analysis scores between race,
severity of illness, and trust.888Note that a naive application of
tokenization is misleadings, as even positive notes containing
”Date:[**5-1-18**]” were tagged as negative because the tool’s string-matching
algorithm was identifying “:[” as negative emoticon use. As a reminder, the
scores were normalized to be zero-mean and unit-variance. It is interesting
that every subpopulation (and indeed the full population) median score is at
least slightly negative, indicating that the distribution has a positive skew.

We observed statistically significant differences in the population means
($p<.05$) for all three stratifications using the Mann-Whitney test. In
particular, we see that black patients, high risk patients, and low trust
patients all have stronger levels of negative sentiment in their notes.
However, the low trust cohort had the most extreme negative sentiments. The
median low trust sentiment (-0.242) was more than twice as far from the center
as the median black sentiment (-0.110), further suggesting that the mistrust
metric is able to tease out the cases with poor caregiver interactions and
impressions.

Table 2: Median sentiment analysis of cohorts stratified by race, severity,
and trust.
population | N | median
---|---|---
White | 9629 | -0.064
Black | 1164 | -0.110
Low Severity | 9629 | -0.058
High Severity | 1164 | -0.167
High Trust | 9629 | -0.049
Low Trust | 1164 | -0.242

###  5.4 Not Just Some Severity Score Proxy

One initial concern we had was that this mistrust metric might have actually
been more similar to a severity score like OASIS than intended. Certainly,
high-risk patients are treated differently than the general population. To
dispel this concern, we compared the pairwise correlations between the
mistrust score, OASIS, and SAPS II – another severity measure (Le Gall et al.,
1993). Table 3 shows that the two well-established acuity scores, OASIS and
SAPS II, have a strong correlation of 0.68. On the other hand, the mistrust
score does not seem to simply recapitulating severity of illness, as indicated
by its weak (0.095 and 0.045) correlations with the other two scores.

Table 3: Pairwise Pearson correlations between severity scores and mistrust
score.
| OASIS | SAPS II | Mistrust
---|---|---|---
OASIS | 1.0 | 0.680 | 0.095
SAPS II | 0.680 | 1.0 | 0.045
Mistrust | 0.095 | 0.045 | 1.0

##  6 Limitations

The primary limitation of this study is that the labels for tuning the weights
for the mistrust metric were generated with a simple rule-based search for the
word ”noncompliant” in a patient’s clinical notes. Not only does this narrow
definition of mistrust fail to capture some of the more subtle interactions in
unhealthy doctor-patient relationships, but it also could falsely attribute
malice to logistic issues such as being noncompliant with home medications
because of a lack of access to prescriptions. In practice, however, we did not
observe many false positive examples, and the mistrust metric – both in
feature weights and in analysis of treatment/sentiment disparities – indicates
that it is a sufficient first-attempt proxy to capture the more difficult-to-
measure quantity of ”trust.”

##  7 Conclusion

In this work, we demonstrate that black patients receive – sometimes
significantly – longer durations of invasive treatments in the MIMIC database.
Though these trends have been studied in private datasets, we present our
replicable analysis on a public dataset.

We create a mistrust score by using coded interpersonal features to predict
patient noncompliance. This mistrust metric is a better identifier than race
to show disparities in both end-of-life care and sentiment. However, this
score also indicates a higher level of mistrust held by black patients than
white patients.

Medical machine learning is moving forward at an exciting pace; we hope that
this work will be a step towards creating models of human physiology that
serve everyone, and do not propagate existing disparities in care. In order to
achieve that goal, we must make better efforts to measure and understand these
disparities.

## References

  * Boag (2018) Boag, William.  Quantifying racial disparities in end-of-life care.  Master’s thesis, MIT, 2018.
  * Cipolletta & Oprandi (2014) Cipolletta, Sabrina and Oprandi, Nadia.  What is a good death? health care professionals’ narrations on end-of-life care.  _Death studies_ , 38(1):20–27, 2014.
  * De Smedt & Daelemans (2012) De Smedt, T. and Daelemans, W.  Pattern for python.  13:2031–2035, 2012.
  * E W Johnson et al. (2013) E W Johnson, Alistair, Kramer, Andrew, and D Clifford, Gari.  A new severity of illness scale using a subset of acute physiology, age, and chronic health evaluation data elements shows comparable predictive accuracy.  41, 05 2013.
  * Ensign et al. (2017) Ensign, Danielle, Friedler, Sorelle A, Neville, Scott, Scheidegger, Carlos, and Venkatasubramanian, Suresh.  Runaway feedback loops in predictive policing.  _arXiv preprint arXiv:1706.09847_ , 2017.
  * Ferryman & Pitcan (2018) Ferryman, Kadija and Pitcan, Mikaela.  Fairness in precision medicine, 2018.
  * Garrett et al. (1993) Garrett, Joanne Mills, Harris, Russell P., Norburn, Jean K., Patrick, Donald L., and Danis, Marion.  Life-sustaining treatments during terminal illness - who wants what?  _Journal of General Internal Medicine_ , 8(7):361–368, 7 1993.  ISSN 0884-8734.  doi: 10.1007/BF02600073.
  * Gelb-Safran et al. (1998) Gelb-Safran, Dana, Taira, DA, Rogers, William, Kosinski, Mark, Ware, John, and Tarlov, AR.  Linking primary care performance to outcome of care.  47:213–20, 10 1998.
  * Ghassemi et al. (2014) Ghassemi, Marzyeh, Naumann, Tristan, Doshi-Velez, Finale, Brimmer, Nicole, Joshi, Rohit, Rumshisky, Anna, and Szolovits, Peter.  Unfolding physiological state: Mortality modelling in intensive care units.  In _Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining_ , pp. 75–84. ACM, 2014.
  * Goyal et al. (2015) Goyal, Monika K., Kuppermann, Nathan, Cleary, Sean D., Teach, Stephen J., and Chamberlain, James M.  Racial disparities in pain management of children with appendicitis in emergency departments.  _JAMA Pediatrics_ , 169(11):996–1002, 2015.
  * Hanchate et al. (2009) Hanchate, Amresh, Kronman, Andrea C., Young-Xu, Yinong, Ash, Arlene S., and Emanuel, Ezekiel.  Racial and ethnic differences in end-of-life costs: Why do minorities cost more than whites?  _Archives of Internal Medicine_ , 169(5):493–501, 2009.
  * Hauser et al. (1997) Hauser, Joshua M., Kleefield, Sharon F., Brennan, Troyen A., and Fischbach, Ruth L.  Minority populations and advance directives: Insights from a focus group methodology.  _Cambridge Quarterly of Healthcare Ethics_ , 6(1):58–71, 1997.  ISSN 0963-1801.  doi: 10.1017/S0963180100007611.
  * Hopp & Duffy (2000) Hopp, Faith P. and Duffy, Sonia A.  Racial variations in end-of-life care.  _Journal of the American Geriatrics Society_ , 2000.
  * Johnson et al. (2017) Johnson, Alistair E. W., Pollard, Tom J., and Mark, Roger G.  Reproducibility in critical care: a mortality prediction case study.  In _MLHC 2018_ , volume 68, pp. 361–376, Boston, Massachusetts, 18–19 Aug 2017.
  * Johnson et al. (2016) Johnson, Alistair EW, Pollard, Tom J, Shen, Lu, Lehman, Li-wei H, Feng, Mengling, Ghassemi, Mohammad, Moody, Benjamin, Szolovits, Peter, Celi, Leo Anthony, and Mark, Roger G.  Mimic-iii, a freely accessible critical care database.  _Scientific data_ , 3, 2016.
  * Kruse et al. (2016) Kruse, Clemens Scott, Goswamy, Rishi, Raval, Yesha, and Marawi, Sarah.  Challenges and opportunities of big data in health care: a systematic review.  _JMIR medical informatics_ , 4(4), 2016.
  * L. Whaley (2001) L. Whaley, Arthur.  Cultural mistrust: An important psychological construct for diagnosis and treatment of african americans.  32:555–562, 12 2001.
  * Le Gall et al. (1993) Le Gall, J.R., Lemeshow, S., and Saulnier, F.  A new simplified acute physiology score (saps ii) based on a european/north american multicenter study.  _JAMA_ , 270(24):2957–2963, 1993.
  * Lee et al. (2016) Lee, Janet J., Long, Ann C., Curtis, J. Randall, and Engelberg, Ruth A.  The influence of race/ethnicity and education on family ratings of the quality of dying in the icu.  _Journal of Pain and Symptom Management_ , 51(1):9 – 16, 2016.  ISSN 0885-3924.  doi: https://doi.org/10.1016/j.jpainsymman.2015.08.008.  URL http://www.sciencedirect.com/science/article/pii/S0885392415004558.
  * McCoy et al. (2015) McCoy, Thomas H., Castro, Victor M., Cagan, Andrew, Roberson, Ashlee M., Kohane, Isaac S., and Perlis, Roy H.  Sentiment measured in hospital discharge notes is associated with readmission and mortality risk: An electronic health record study.  _PLOS ONE_ , 10(8):1–10, 08 2015.  doi: 10.1371/journal.pone.0136341.  URL https://doi.org/10.1371/journal.pone.0136341.
  * Muni et al. (2011) Muni, Sarah, Engelberg, Ruth A., Treece, Patsy D., Dotolo, Danae, and Curtis, J. Randall.  The influence of race/ethnicity and socioeconomic status on end-of-life care in the icu.  _Chest_ , 139(5):1025–1033, 2011.  ISSN 0012-3692.  doi: 10.1378/chest.10-3011.
  * Perkins et al. (2002) Perkins, Henry S, Geppert, Cynthia, Gonzales, Adelita, Cortez, Josie D, and Hazuda, Helen P.  Cross-cultural similarities and differences in attitudes about advance care planning.  _Journal of General Internal Medicine_ , 17(1):48–57, 2002.
  * Raghupathi & Raghupathi (2014) Raghupathi, Wullianallur and Raghupathi, Viju.  Big data analytics in healthcare: promise and potential.  _Health information science and systems_ , 2(1):3, 2014.
  * Singhal et al. (2016) Singhal, Astha, Tien, Yu-Yu, and Hsia, Renee Y.  Racial-ethnic disparities in opioid prescriptions at emergency department visits for conditions commonly associated with prescription drug abuse.  _PLOS ONE_ , 11(8):1–14, 08 2016.  doi: 10.1371/journal.pone.0159224.  URL https://doi.org/10.1371/journal.pone.0159224.
  * Washington (2007) Washington, Harriet.  _Medical Apartheid: The Dark History of Medical Experimentation on Black Americans from Colonial Times to the Present_.  2007\.  ISBN 978-0385509930.
  * Wunsch et al. (2010) Wunsch, Hannah, Guerra, Carmen, Barnato, Amber E, Angus, Derek C, Li, Guohua, and Linde-Zwirble, Walter T.  Three-year outcomes for medicare beneficiaries who survive intensive care.  _Jama_ , 303(9):849–856, 2010.
  * Yarnell et al. (2017) Yarnell, Christopher J., Fu, Longdi, Manuel, Doug, Tanuseputro, Peter, Stukel, Theres, Pinto, Ruxandra, Scales, Damon C., Laupacis, Andreas, and Fowler, Robert A.  Association between immigrant status and end-of-life care in ontario, canada.  In _JAMA_ , 2017.
  * Zemel et al. (2013) Zemel, Rich, Wu, Yu, Swersky, Kevin, Pitassi, Toni, and Dwork, Cynthia.  Learning fair representations.  In _International Conference on Machine Learning_ , pp. 325–333, 2013.