# Tutela: An Open-Source Tool for Assessing User-Privacy
on Ethereum and Tornado Cash

Mike Wu, Will McTighe, Kaili Wang
Stanford University
&István A. Seres
Eötvös Loránd University
Nick Bax
Convex Research
&Manuel Puebla, Mariano Mendez
Universidad de Buenos Aires
Facultad de Ingenieria
Federico Carrone, Tomás De Mattey, Herman O. Demaestri, Mariano Nicolini,
Pedro Fontana
LambdaClass

###### Abstract

A common misconception among blockchain users is that pseudonymity guarantees
privacy. The reality is almost the opposite. Every transaction one makes is
recorded on a public ledger and reveals information about one’s identity.
Mixers, such as Tornado Cash, were developed to preserve privacy through
“mixing” transactions with those of others in an anonymity pool, making it
harder to link deposits and withdrawals from the pool. Unfortunately, it is
still possible to reveal information about those in the anonymity pool if
users are not careful. We introduce Tutela, an application built on expert
heuristics to report the true anonymity of an Ethereum address. In particular,
Tutela has three functionalities: first, it clusters together Ethereum
addresses based on interaction history such that for an Ethereum address, we
can identify other addresses likely owned by the same entity; second, it shows
Ethereum users their potentially compromised transactions; third, Tutela
computes the true size of the anonymity pool of each Tornado Cash mixer by
excluding potentially compromised transactions. A public implementation of
Tutela can be found at https://github.com/TutelaLabs/tutela-app. To use
Tutela, visit https://www.tutela.xyz111For related questions and inquiries,
please contact the corresponding author at<EMAIL_ADDRESS>
##  1 Introduction

On any public blockchain, the cost of creating a new wallet is virtually zero,
enabling the same entity to manage several pseudonymous addresses. The
pseudonymity underpinning blockchains like Bitcoin (Nakamoto, 2008) and
Ethereum (Buterin et al., 2013) breeds a sense of privacy. This often leads to
misuse (Christin, 2013), such as money laundering through a large number of
addresses (Möser et al., 2013), or unfair voting power distributed among
multiple addresses owned by the same user. Thus, it is of interest in many
investigations to identify addresses linked to the same entity. This is
predominantly done through heuristics. Every transaction an address makes on a
blockchain is recorded and public, revealing information about the underlying
entity. As such, with graph analysis tools, one can cluster addresses together
that, with reasonable confidence, possess the same owner.

Such anonymity tools have been widely explored for Bitcoin Haslhofer et al.
(2016), leveraging heuristics targeting the unspent transaction output (UTXO)
model. However, this has limited application to more recent blockchain
implementations like Ethereum, which forgo the UTXO model for an account (or
sometimes balance) model. Ethereum, in particular, has an account-based
protocol that implicitly encourages an entity to reuse a handful of addresses.
As such, this poses greater challenges to user privacy than UTXO-based
blockchains.

In response to this shortcoming, several coin mixing protocols have been
proposed like Möbius (Meiklejohn and Mercer, 2018), MixEth (Seres et al.,
2019), and Tornado Cash (Pertsev et al., 2019) to obfuscate transaction
tracing, the final of which is deployed in practice. Still, new heuristics
have surfaced (Victor, 2020; Béres et al., 2021) that deanonymize Ethereum
users. These heuristics largely exist in academic silos, and not been combined
nor demonstrated in public application.

#### Our contributions.

We develop a web application that combines several state-of-the-art heuristics
to measure the anonymity of Ethereum addresses. To the best of our knowledge,
this is the first instance to deploy these algorithms at scale. In doing so,
we create a rich depiction of user behavior and privacy. We also propose a set
of new heuristics targeted at Tornado Cash, highlighting that careless user
behavior, despite using a mixer, can still reveal identity. A Python
implementation is open sourced at https://github.com/TutelaLabs/tutela-app and
the tool is available at https://www.tutela.xyz.

#### Paper organization.

The rest of this paper is organized as follows. We provide some pertinent
preliminaries in Section 2. In Section 3, we provide an overview of Tutela,
our developed anonymity tool. In Section 4, we describe our data processing
methods and the used datasets. In Section 5, we describe two heuristics that
allowed us to cluster Ethereum addresses that are likely owned by the same
entity. In Section 6, we assess the privacy guarantees of Tornado Cash
applying five novel heuristics. In Section 7, we provide a quantitative
analysis. Finally, we conclude our work with some discussions and future work
in Section 8.

##  2 Preliminaries

In this section, we provide some background on Ethereum and its account model.
For a deeper dive into Ethereum, we refer the reader to Antonopoulos and Wood
(2018). Moreover, we briefly describe the high-level workings of Tornado Cash.

###  2.1 Ethereum and the Account Model

Ethereum is the second-largest cryptocurrency platform in terms of market
capitalization.222For more details, see:
https://coinmarketcap.com/currencies/ethereum. However, it is the largest
smart contract platform, and it is also the most used public blockchain for
settling transactions. Therefore, it is imperative to understand better and
assess the privacy guarantees of Ethereum quantitatively.

Ethereum employs the account model. There are two types of accounts:
externally owned accounts (EOA) and contract accounts. EOAs are controlled by
users via a cryptographic key-pair (a private and a public key) owned by them.
The private key of the EOA enables users to send transactions from that
account, while the public key is used to derive an address for the EOA (the
hash of the public key is its address). On the other hand, contract accounts
are controlled by contract code. The contract account’s address is the hash of
their contract code. Accounts are referred to by their addresses (a
pseudonym). Contract accounts cannot initiate transactions. However, EOAs can
send transactions to contract accounts that can run the code of the contract
account. In the following, we will use the words accounts and addresses
interchangeably.

Ethereum’s account model has several implications from a privacy point of
view. First, the account model incentivizes the reuse of accounts across many
transactions. Imagine the following scenario. Alice owns an EOA with address A
and a balance of three ether. Alice wishes to buy a product from Bob for two
ether. Alice sends a transaction worth two ether to Bob. After this purchase,
address A has a balance of one ether. Therefore, if Alice wants to spend her
change, she necessarily needs to send a transaction _again_ from her address
A. Address reuse facilitates the profiling of rich transaction histories:
complete financial history, list of all counterparties, time-of-day activity,
and more. Furthermore, account reuse implies that most users only own a
handful of accounts. Finally, the accounts owned by the same user can
effectively be clustered; see Section 5.

###  2.2 Preserving Privacy on Ethereum: Tornado Cash

Users can break links with their transaction history using a so-called mixer
contract. Tornado Cash (TC) is the most widely used, non-custodial mixer on
Ethereum. TC works as follows. Users deposit equal amounts to a TC smart
contract (e.g. 1 ETH as shown below). After some time, users can withdraw
their funds from the mixer contract to _a freshly generated EOA_ by providing
a zero-knowledge proof that proves that the withdrawing user is one of the
depositors. Therefore, the withdrawing EOA has enhanced its privacy since it
has become unlinkable to any unique depositor EOA. Note that each TC contract
applies a fixed denomination for the mixed funds (e.g. 1 ETH). Otherwise,
linking deposits to withdraws would be trivial.

Figure 1: Example of the Tornado Cash 1 ETH pool: addresses A through F
deposit to and withdraw from the pool. It quickly becomes impossible to
associate withdraw and deposit transactions given a growing mixer.

A user’s anonymity is defined by the number of equal user deposits in a given
pool. This is the pool’s _Anonymity Set_. In the example above, D’s withdrawal
could have come from A, B or C, so the anonymity set is 3 and the probability
of correctly guessing the deposit / withdrawal connection is 1/3.

The more users that deposit in the pool, the greater the number of people that
a withdrawal could have come from. If you add a fourth deposit to the example
above, the probability of being correctly detected decreases to 1/4. However,
there are lots of ways users can compromise their privacy. If you can link A’s
deposit to E’s withdrawal, then the pool’s anonymity set decreases from 3 to
2. This means the probability of correctly guessing your deposit / withdrawal
connection increases to 1/2 because any withdrawal could only have come from B
or C’s deposits.

Tornado Cash provides mixing contracts (otherwise known as mixing pools) for
Ether and several Ethereum-based tokens (e.g., DAI, USDC, wBTC, etc.) in
different denominations (1, 10, 100, etc.).

##  3 Tutela Overview

Tutela333At the time of publication, Tutela is hosted at
https://www.tutela.xyz., latin for protection, is a web application that has
three functions, it inform users which of their Ethereum addresses are
affiliated, how they are linked and audits the anonymity sets of Tornado Cash
Pools. More on each of these below.

###  3.1 Ethereum Address Clustering

Users can search an ethereum address (e.g. 0x…) or ENS (e.g. tutela.eth), and
receive a summary of its anonymity.

Figure 2: Tutela address page when searching an Ethereum address. This address
is also a TC user.

We summarize the main functionalities shown in Figure 1. The response page is
separated into three sections. The top left section summarizes the anonymity
of the searched address. It contains an “anonymity score” out of 100, where a
lower number represents less anonymity. In this example, the searched address
has an anonymity score of 0, representing a large amount of leaked identity
information. Other information, such as balance in ETH or ENS names, are shown
when relevant.

The top right section is only populated if the searched address is a Tornado
Cash user. In this example, the searched address has deposited 97 times to a
Tornado Cash pool and withdrawn 88 times. Interestingly, we find that through
heuristics, we are able to tie 87 of those withdraw transactions to deposit
transactions, thereby meaningfully reducing the useful size of the Tornado
Cash pool. See Section 6 for more details.

The bottom section labeled “Linked Addresses” shows a list of addresses
clustered with the searched one. Each item in this list is denoted as either
an externally owned address (EOA), a deposit address, or an exchange address.
Each item also contains a confidence score denoting the strength of
association with the searched address, and the heuristic that bound it to the
searched address. This example shows a thousand clustered addresses
representing an entity with a large wallet portfolio (e.g., bot) – hence why
the anonymity score is zero.

###  3.2 Ethereum Address Reveals

Figure 3: Tutela transaction page when searching an Ethereum address. This
address is also a TC user.

To see a history of when an Ethereum user potentially committed reveals, users
can select the transactions tab on the landing page and input an Ethereum
address or ENS. The right hand side of Figure 3 shows a graph of when these
revealing transactions occurred where the x-axis denotes the weeks prior to
the current date. Each bar in the graph corresponds to an individual week and
upon clicking on each bar, Tutela will show the details of the potentially
revealing transactions underneath. On the left hand side, users can see
statistics on their potential reveals as well as a comparison to the average
Tornado Cash user.

###  3.3 Tornado Cash Anonymity Set Auditor

The Tornado Cash Anonymity Set Auditor, computes the five heuristics described
in Section 6 for each Tornado Cash pool to determine how many potentially
compromised deposits are in each pool.

Figure 4: Tutela interface when searching a Tornado Cash pool.

If a user selects a Tornado Cash Pool from the dropdown list on the landing
page, the top right of the results page will show the headline number of equal
user deposits in a Tornado Cash pool, otherwise known as its anonymity set.
Below that, we compute the “true” anonymity set size for the pool, subtracting
out all equal user deposits that may have been compromised through our
heuristics. This gives the number of “uncompromised deposits”. At the time of
publication, the Tornado Cash website reports only the full anonymity set
size, not taking into account potential compromises.

At the bottom of Figure 4, users can supply an address to check, in a private
way, if it has made any compromising transactions, inspired by the popular
website “Have I been pwned?”444See https://haveibeenpwned.com..

A concrete use case of Tutela is to adjust anonymity set to protect the
privacy of Tornado Cash users more faithfully. As described in Section 2, if a
pool has many compromised deposits, it provides less privacy than a user
believes.

##  4 Data and Setup

We discuss the data sources used to build Tutela.

Ethereum transactions are downloaded from the crypto_ethereum dataset using
BigQuery, including all transactions from August 7th, 2015 to October 1st,
2021555In the web application, Tutela is updated weekly.. In total, this
amounts to 4 terabytes of data with over 1B rows. In addition, we assume
access to a list of known addresses obtained from a public Kaggle
challenge666See the list of labelled Ethereum addresses found at
https://www.kaggle.com/hamishhall/labelled-ethereum-addresses., containing
almost 20,000 labelled addresses corresponding to different centralized
exchanges, decentralized exchanges, relayers, DeFi applications, and more.
This list will be used to identify exchange addresses for heuristics and apply
known constraints on the inferred identity of clustered addresses.

Additionally, we create a partition of the transaction data from
crypto_ethereum pertaining to Tornado Cash pools. This is done by checking
that the address receiving a transaction is a Tornado Cash smart contract
(taken from the BigQuery dataset tornado_cash_transactions). To capture the
transactions executed by the Ethereum virtual machine (e.g., through a smart
contract), we use the crypto_ethereum.traces table. In the special case that a
withdrawal from a Tornado Cash pool is made by a relayer, we decode the input
code using the contract ABI to find the recipient address. In total, we
uncover around 97,365 deposit and 83,782 withdraw transactions across all
pools. These two transaction sets will be used for Tornado Cash-specific
heuristics.

##  5 Ethereum Heuristics

Using this large dataset, we describe two Ethereum-wide heuristics used to
cluster together addresses potentially belonging to the same entity.

###  5.1 Deposit Address Reuse

Deposit address reuse (DAR) links together EOAs through the usage of a
centralized exchange (CEX). We refer to the original paper (Victor, 2020) for
a detailed description but provide an overview here.

Figure 5: Graph of transactions between EOA, deposit, and CEX addresses. A
cluster is defined as a weakly connected component in an undirected subgraph
containing only EOA and deposit nodes. The gray circles show EOA addresses in
two clusters.

When users deposit Ether to a CEX, the exchange typically creates “deposit
addresses”, which receive the assets from the EOA and then forward these funds
to the main addresses associated with the CEX. Critically, these deposit
addresses are created per customer, not per address. That is, multiple
addresses that send funds to the same deposit address are highly likely to be
owned by the same entity. However, these deposit addresses are not known. The
DAR algorithm seeks to identify deposit addresses through heuristics. It uses
two hyperparameters: the maximum amount difference $\alpha$ and the maximum
time difference $\tau$ between two transactions: a “receiving” transaction
from an EOA to a suspected deposit address, and a “forwarding” transaction
from a deposit to a main exchange address (we assume access to a list of main
CEX addresses). The intuition is that true deposits likely forward funds
quickly, and differences in amount should only be due to transaction fees. We
exclude known entities (e.g. CEX, DEX, miner, etc.) from being classified as
deposit addresses.

Then, we can search for $($EOA, deposit, exchange$)$ tuples that match the
constraints set by $\alpha$ and $\tau$. If we construct an undirected graph
with addresses as nodes and edges between associated EOA and deposits, a
“cluster” is defined as a weakly connected component. This definition captures
interesting cases where a single entity has multiple EOAs that send to
different deposits of multiple CEXs (see Figure 5). In practice, we pick
$\alpha=0.01$ and $\tau=3,200$ (Victor, 2020).

For each cluster, we would like to assign a confidence score, representing our
belief in this cluster representing a single entity. We note that any
uncertainty in clustering must come from uncertainty in defining deposit
addresses. So, for any address in a cluster $C$, we assign confidence as the
average confidence for deposits in $C$. Now, for any deposit $v$, we define
confidence as:

$\kappa(v)=1-\left\\{\frac{1}{2}\left(\frac{|a_{f}-a_{r}|}{\alpha}\right)+\frac{1}{2}\left(\frac{t_{f}-t_{r}}{\tau}\right)\right\\}$

where $a_{f}$ and $a_{r}$ are the amounts from the forwarding and receiving
transactions, respectively. Similarly, $t_{f}$ and $t_{r}$ are the forwarding
and receiving block numbers. A larger difference in either amount or time
would decrease the confidence, which is bound to be between 0 and 1.

###  5.2 Learned Node Embedding

The benefit of DAR is interpretability: we can understand (to a degree) why
the algorithm believes two addresses are linked. However, this comes at the
cost of recall and the ability to identify dense clusters. DAR searches for a
very specific behavior, and most addresses will not be in a cluster of greater
than size 1. Initial feedback from Tutela users reported limited success in
finding clusters. To supplement DAR, we consider a second Ethereum-wide
heuristic (NODE) that projects addresses to points in a low-dimensional vector
space based on who it transacts with. In this vector space, addresses
belonging to the same entity should be close together in Euclidean distance.

Consider constructing an undirected graph $G(V,E)$ from all Ethereum
transactions, where nodes $V$ are composed of distinct addresses, and an edge
is placed between two nodes if there is a transaction between them. Each edge
is given a weight $w:V\times V\rightarrow\mathbb{N}$ designating the number of
interactions between two addresses. For instance, and if Alice has sent Bob 1
ETH five times, then $w(\textup{Alice},\textup{Bob})=5$. Note that this graph
is distinct from the one used in Section 5.1.

At this abstraction layer, we seek to learn a “node embedding function”
$f:V\rightarrow\mathbb{R}^{d}$ that projects a node to a $d$-dimensional
vector representation. Importantly, we want this embedding to capture semantic
information about the node, such as which other addresses it frequently
interacts with. To do so, we leverage popular graph representation learning
algorithms (Grover and Leskovec, 2016; Rozemberczki and Sarkar, 2018). In
particular, we focus on Diff2Vec (Rozemberczki and Sarkar, 2018), which has
been applied to blockchain transactions (Béres et al., 2021), though not at
scale.

The intuition of Diff2Vec is to summarize a node by its neighborhood through a
diffusion-like random process. Specifically, there are four steps to Diff2Vec:
(1) generating a “diffusion graph”, (2) sampling a “node sequence”, (3)
extracting features, (4) learning a neural network embedding. We briefly
summarize each step below.

#### Step One

Fixing a starting node $v\in V$, we initialize a diffusion subgraph
$\tilde{G}$ containing only $\\{v\\}$. Randomly sample two nodes, $u$ from
$\tilde{G}$ and $w\in\textup{Neighbors}(u,G)$, where $G$ is the original
graph. Add node $w$ and the edge $(u,w)$ to $\tilde{G}$. Repeat until
$\tilde{G}$ has $l$ nodes, where $l$ is a hyperparameter representing the
amount of information we want to capture in our eventual node embedding. A
larger $l$ may capture a large neighborhood but sacrifice in granularity.

#### Step Two

Given $\tilde{G}$, we generate a sequence $s=(v_{1},v_{2},\ldots)$ recording
the nodes visited during an Euler walk. To do this, we must ensure that
$\tilde{G}$ is Eulerian, which holds if every node has an even degree. A
simple way to achieve this is to double each edge in $\tilde{G}$. At this
point, we can summarize a node $v\in V$ by multiple sequences
$S=(s_{1},s_{2},\ldots)$, each from a random walk.

#### Step Three

Given a set of sequences $S$, we aim to produce a single feature vector based
on frequencies of which vertices appear near each other in sequences $s\in S$.
Specifically, fix a node $v$. Then, pick a window size $h$. Count how many
times other nodes appear within $h$ positions before and after when $v$
appears, summed over all sequences $s\in S$. This will result in $2h$ vectors
each of length $|V|$ — $2$ from counting before and after; $h$ from counting
frequencies 1 to $h$ positions away from $v$; $|V|$ since each vector stores
counts of all nodes in $V$. Denote this feature vector by
$y(v)\in\mathbb{R}^{2h|V|}$.

#### Step Four

Finally, we wish to compress the feature vector from Step Three to a
$d$-dimensional continuous space. To do this, we train a two layer perceptron
$f_{\theta}=f_{1}\circ f_{2}$ using stochastic gradient descent (Goodfellow et
al., 2016; Kingma and Ba, 2014). Define
$f_{1}:\mathbb{R}^{|V|}\rightarrow\mathbb{R}^{d}$ and
$f_{2}:\mathbb{R}^{d}\rightarrow\mathbb{R}^{2h|V|}$, with a ReLU nonlinearity
in between. The input to the network is a one hotted representation of the
current node $v$. To train the parameters $\theta$, we optimize the objective:

$\mathcal{L}(v;\theta)=\texttt{dist}(f_{\theta}(\texttt{one\\_hot}(v)),y(v))$
(1)

where dist is a distance function. Examples include Euclidean distance, or a
cross entropy loss. That is, Equation 1 optimizes $f_{\theta}$ to predict the
frequency vector. Then, assign $f_{1}(v)\in\mathbb{R}^{d}$ to be the final
embedding for $v$.

(a) Subgraph $\tilde{G}$

(b) Sequences $S$

(c) Feature $y(a)$

(d) Word2Vec $f_{\theta}$

Figure 6: The four steps of the Diff2Vec algorithm, focusing on the individual
node $a\in V$.

Steps three and four amount to Word2Vec777We use gensim.models.word2vec.
(Mikolov et al., 2013a, b). In practice we optimize with SGD for 5 epochs with
a learning rate of 0.025. We set $d=128$, $l=40$, and $h=5$. Given an address
$v\in V$, to find its cluster, we can search for the closest $k$ vectors in
$\mathbb{R}^{d}$. In practice, we accomplish this efficiently using FAISS
(Johnson et al., 2019). Unlike DAR, this heuristic will always return $k$
addresses. We score the confidence of an address $u$ in the cluster by the
inverse of its Euclidean distance to the embedding of $v$:

$\kappa(u)=\frac{1}{\|f_{\theta}(\texttt{one\\_hot}(u))-f_{\theta}(\texttt{one\\_hot}(v))\|_{2}}$

since a smaller distance (closer to 0) represents more semantic similarity of
$u$ to $v$.

###  5.3 Anonymity Score

Given a query address $v$, using one or both heuristics, we obtain a set of
clustered addresses $C$ with confidence scores $\kappa$ for every member. We
wish to compute a statistic summarizing the anonymity of the query address,
such that a larger cluster reveals more identity information, hence lower
anonymity. We define the anonymity score as:

$\texttt{anonymity}(v)=1-\texttt{tanh}(\beta*\kappa(v)*|C|)$ (2)

where $\beta$ is a hyperparameter controlling slope. A larger value for
$\beta$ more aggressively penalizes larger clusters. We chose $\beta=0.1$.
This anonymity score is a summary statistic representing how easily connected
an address is to other addresses potentially owned by the same entity.

##  6 Tornado Cash Heuristics

Focusing on the subset of Ethereum transaction data involving Tornado cash
deposits and withdrawals, we discuss a five heuristics for identifying
compromised deposits.

###  6.1 Address Match

Figure 7: Address match – The triangle represents a single address withdrawing
and depositing to a TC pool.

Suppose the address making a deposit transaction to a Tornado Cash pool
matches the address making a withdrawal transaction (from the same pool). In
that case, the two transactions can be linked, and the corresponding deposit
is compromised as the user identity may be revealed. These may be TORN yield
farmers who deposit and withdraw to the same address and are only profit-
motivated or clumsy Tornado cash users.

###  6.2 Unique Gas Price

Figure 8: Unique gas price – two addresses depositing and withdrawing with the
same 27.4 gwei gas price.

Prior to EIP-1559, Ethereum users could specify the gas price when making a
deposit or withdrawal to a Tornado Cash pool. Those who do so tend to specify
gas prices that are identical for deposit and withdrawal transactions. User-
specified gas prices follow common patterns such as being round numbers or
being unique to an individual. Care must be taken to remove transactions made
by a relayer service which may set gas prices as well. Relayer-specified
prices also tend to be oddly specific and not round numbers. In practice,
relayers can be filtered out by decoding the input code from Ethereum
transaction data.

###  6.3 Linked ETH Addresses

Figure 9: Linked ETH addresses – the green arrows represent interactions
between two addresses A and D outside of TC. Addresses A and D deposit and
withdraw from the same Tornado Cash pool, respectively.

This heuristic aims to link withdraw and deposit transactions on Tornado Cash
by inspecting ETH (non-Tornado Cash) interactions. This is done by
constructing two sets, one corresponding to the unique Tornado Cash deposit
addresses and one to the unique Tornado Cash withdraw addresses, to then make
a query to reveal transactions between addresses of each set: when at least
three such transactions are found, the withdraw and deposits addresses will be
considered heuristically linked in Tornado Cash. The more transactions found,
the more confident we are in the link.

###  6.4 Multiple Denomination

Figure 10: Multi Denomination – Addresses A and D deposit and withdraw the
same number of times from the same three Tornado Cash pools, respectively.

Previous heuristics examine isolated pairs of deposit and withdraw
transactions. This heuristic, however, studies an address’ history of
transactions. We compute the portfolio of an address’ withdrawals across
Tornado Cash pools, hence “multi-denomination”. For instance, Alice may have
withdrawn from the 1 ETH pool twice, the 0.1 ETH pool five times, the DAI pool
once. Then we search for all addresses whose portfolio of deposit transactions
is exactly the same as Alice’ withdrawal portfolio. An address depositing into
the 1 ETH pool three times, the 0.1 ETH pool five times, and the DAI pool once
would be a valid match. (Optionally, we can relax this to search for deposits
with at least as many as Alice’ withdrawals, though this risks more false
positives). All Tornado Cash transactions under the matched deposit and
withdrawal address are linked to each other.

We ignore all addresses that make fewer than three transactions to Tornado
Cash pools, and ignore all addresses that interact with only one pool.
Additionally, to reduce the likelihood of false positives, we further
constrain deposits within a multi-denomination reveal to occur within a 24
hour window; similarly we separately constrain withdraws within a multi-
denomination reveal to occur within a 24 hour window. This intuition being
that careless users will deposit or withdraw all at once.

###  6.5 TORN Mining

Figure 11: TORN Mining – Address D was given 10 TORN upon withdrawing from the
1 ETH pool in return for anonymity points, linking address D to a deposit 100
blocks prior. Searching records, only address A deposited in the 1 ETH pool
100 blocks prior, compromising address D. Note that the numbers presented here
are for explanatory purposes.

In February 2021, Tornado Cash introduced anonymity mining. It was an
incentive scheme to encourage more deposits in Tornado Cash pools, thereby
increasing their anonymity sets. Tornado Cash rewarded participants a fixed
amount of anonymity points (AP) based on how long they left their assets in a
pool.

After withdrawing assets, users can claim Anonymity Points. The amount
withdrawn is recorded in the transaction. If a user uses a single address to
claim all of their anonymity points, one can calculate the exact number of
Ethereum blocks that their assets were in the pool because the AP yields were
public and fixed. If there is a unique deposit / withdrawal combination in a
pool separated by this number of Ethereum blocks, the transactions are assumed
linked. This is more likely when the deposit or withdrawal in the pair also
claimed the AP. This heuristic is most effective if AP is being claimed for a
single deposit and is harder to compute for multiple deposits.

###  6.6 Connection to Ethereum-based Clusters

Unlike the Ethereum-wide heuristics (i.e., DAR and NODE), which find clusters
of compromised addresses, Tornado Cash heuristics find clusters of compromised
transactions. However, aside from address matching, a subset of these Tornado
Cash heuristics can also be applied on the address level: given a cluster of
compromised transactions, find the sender address for each transaction, and
compute the unique set, removing any addresses in our list of known addresses.
These clusters can be added to the ones discussed in Section 5. This in turn,
will further enrich the anonymity score.

##  7 Analysis

We provide a brief quantitative analysis of the scale and efficacy of Tutela
heuristics.

Figure 12: 10k subgraph of 26M graph created via deposit address reuse.
Contains EOA (green), deposits (blue), and exchanges (red).

###  7.1 Ethereum Heuristics

Using DAR, we found 26M EOA addresses, resulting in 2.5M clusters of Ethereum
addresses. The average cluster size contains 4.3 ($\pm$ 8.6) EOA addresses;
the largest cluster contains 2.1k EOA addresses. Figure 12 shows a
visualization of 10,000 random nodes from the DAR graph. In particular, we
observe interesting structure with many small clusters scattered uniformly,
balanced by several large clusters in the perimeter.

Next, we can measure the quality of the DAR clusters using a held out “test
set” of known clustered addresses. We obtain such a set from Béres et al.
(2021) where 1,028 clusters of addresses (average size of 4.0 $\pm$ 3.6 EOA
addresses per cluster) are derived from ENS names. We find a recall of 39.4%
using DAR. While there is room for improvement, we are cautiously optimistic
that DAR is able to recover a nontrivial number without any knowledge of ENS
names. This provides evidence for the generality of DAR clusters.

Figure 13: 100k random subset of 131M embedding of Ethereum addresses
projected from to 3D using PCA.

Using Node, we found 131M clusters of Ethereum addresses, with each cluster
having exactly 9 members by design (10 including itself). Figure 13 shows a
visualization of 100,000 random embeddings of Ethereum addresses from the NODE
set, where embeddings are projected down to two dimensions using PCA (trained
on a subset of 1M address embeddings). The color in Figure 13 shows the
density, estimated using a Gaussian kernel, where a lighter (yellow) color
represents higher density. That is, the figure shows a non-hollow ball of
address embeddings. For any query address, Tutela returns the closest
neighbors from the point cloud. Again, we access quality using the held-out
known clusters, finding a recall of 37.8% with NODE, two percent lower than
DAR. However, as shown in Figure 14, when DAR and NODE are used in
combination, the recall increases 7% to 44.8%, indicating that DAR and NODE
find clusters of different “types”. In Tutela, searching an address will
return both clusters types.

Figure 14: Plot of the recall of held-out address clusters through ENS reveals
using deposit address reuse (DAR), diff2vec (NODE), and the combination of
both (BOTH). A higher recall represents a better heuristic.

###  7.2 Tornado Cash Heuristics

Of the 97.3k Tornado Cash equal user deposits, we found 42.8k are potentially
compromised: 18.6K from the address match reveal, 102 from the unique gas
price reveal, 18.9K from the linked ETH address reveal, 16.2K from the multi-
denomination reveal, and 358 from the TORN mining reveal (with overlap between
reveals). Splitting this by pool, we find the anonymity set to be reduced by
37% ($\pm$ 15%) on average. Figure 15 shows the uncompromised anonymity sets
by pool.

We find that some of the pools could be heavily compromised (such as the cDAI
and WBTC pools), whereas other pools are less effected (e.g. USDC). In
summary, while many of the Tornado Cash heuristics are simple, they are quite
powerful. These findings could help Tornado Cash developers and users alike,
measure and understand the degree user privacy offered.

Figure 15: Plot of the percentage of compromised versus uncompromised (pink)
deposits by pool.

##  8 Discussion and Future Work

We conclude with a few discussion points and final remarks. In particular, we
present possible limitations and extensions to the proposed system.

###  8.1 Limitations

We acknowledge that heuristics are not perfect measures. In return for
simplicity, there will likely be false positives in practice, e.g., addresses
in a cluster that should not be there, or faithful Tornado Cash transactions
labeled as compromised.

For the Ethereum heuristics, we emphasize that picking proper hyperparameters
is a challenge. In DAR, the quality of the algorithm is very sensitive to the
choice of maximum thresholds $\alpha$ and $\tau$. With too small thresholds,
no clusters will be found; with too large thresholds, clusters will be low
quality, containing many addresses they should not. Currently, the best
practice is to tune these by hand. In NODE, the size of subgraph $l$ and the
dimensionality $d$ similarly determine the resulting quality. The choice of
$l$ should reflect the size of the full graph $G$: a bigger graph requires the
embedding to summarize a larger neighborhood. Other NODE hyperparameters, such
as window size or optimization choices, are less important to the final
embedding. Running the DAR or NODE on Ethereum is computationally expensive,
requiring both a large RAM and storage.

On the other hand, the Tornado Cash heuristics are much simpler than the
Ethereum ones, and more deterministic. However, given that only a small subset
of Ethereum addresses are Tornado Cash users, they have limited applicability
to the majority of potential Tutela users.

###  8.2 Extensions

As described, Tutela uses only on-chain data to access anonymity. However,
extensions can be made to include off-chain data, such as from decentralized
applications (e.g. DeFI, NFT, games, etc.), layer two data, external
blockchains, and more. The inclusion of off-chain data could lead to more
powerful de-anonymization attacks.

###  8.3 Broader Impact

There is a need for greater privacy on the blockchain to accelerate adoption.
Consumers would not be willing to receive their salaries publically or have
their online purchasing history in the public for all to see. Businesses would
not pay suppliers on the blockchain if their competitors could see who and how
much they pay for supplies. Similarly, investment funds want to keep their
strategies private and not copied before their trades have even been recorded
on-chain.

That said, blockchain privacy is a difficult issue to navigate. Currently,
blockchain privacy and finality create opportunities for money laundering and
nefarious activities, so privacy solutions combined with regulation will need
to account for these considerations. In the meantime, we hope that Tutela will
help law-abiding blockchain users better protect themselves in the current
ecosystem.

## Acknowledgments

We would like to acknowledge both the Tornado Cash team and the Convex
Research team for their support. This work was funded by the Tornado Cash
community bounty to develop anonymity tools to protect user privacy. We also
thank the Tornado Cash community at large for their consistent feedback and
patience through our application’s development. Finally, we thank the Stanford
Venture Studio for their support with compute credits.

##  Appendix A Appendix

###  A.1 Tornado Cash Heuristic: Wallet Fingerprints

We present a Tornado Cash heuristic for identifying compromised transactions
that is not yet implemented in Tutela at the time of publication.

In the Ethereum ecosystem, users can select from numerous wallet softwares.
Wallet fingerprints allow one to establish the wallet software a user sent
their transaction from. Such wallet fingerprints could largely fragment the
anonymity set of Tornado Cash users. For instance, a withdrawal transaction
sent with wallet software $\mathcal{A}$ can only be indistinguishable among
the deposit transactions sent by users of the same wallet software
$\mathcal{A}$.

Transaction fees (i.e., the $\mathsf{max\\_fee}$ and
$\mathsf{max\\_priority\\_fee}$ fields of a transaction) are typically set by
the wallet software. These values are often computed by open-source algorithms
that are different for each wallet. For instance, Blocknative’s
algorithm888See for more details: https://www.blocknative.com/gas-estimator.
uses the following:

$\mathsf{max\\_fee}=\mathsf{base\\_fee}+2\cdot\mathsf{max\\_priority\\_fee}$

Therefore, it is straightforward to assess the wallet software for many
transactions. Assuming that accounts do not change their used wallet software,
we can more confidently identify compromises.

###  A.2 Improvements to Measuring Tornado Cash’s True Anonymity Set Size

The anonymity set size listed on Tornado Cash’s
webpage999https://tornadocash.eth.link does not take into account compromised
deposits. We can compute the uncompromised (or useful) anonymity set size as
the total number of deposits in this pool removing revealed deposits from each
of the heuristics above. This resulting number is a more faithful
representation of the privacy provided by a Tornado Cash pool.

In this regard, the very definition of anonymity set offers a way of
interpreting heuristics producing multiple deposit addresses as output for a
single withdraw address input by measuring anonymity in terms of the Shannon
entropy Diaz et al. (2002).

Concretely, we are able to quantify how much anonymity is lost after running a
heuristic $\mathcal{H}$ for a given withdrawal address. Heuristics linking
addresses by scanning ETH transactions outside the TC environment or linking
addresses by matching deposit to withdraw portfolios typically produce a large
number of candidate addresses: organizing these outputs as a non-empty set
$\mathcal{C}$, they provide a natural ”refined anonymity set” notion - of
course, these outputs need not to include the actual deposit address, but in a
worst-case scenario a clear interpretation can be given. Calling the anonymity
set $\mathcal{D}$, $D$ its cardinality and recalling its definition, we can
compute its Shannon entropy by simply taking the logarithm of its cardinality:

$H(\mathcal{D})=-\sum\limits_{d\in\mathcal{D}}{\frac{1}{D}\ln\Big{(}\frac{1}{D}\Big{)}}=\ln(D)$

This quantity is a central notion in classical information theory and
represents how much ”regularity” there is in a random variable: higher entropy
will mean low information. In our context, the definition of anonymity set
implies that any of its members is equally likely to be the deposit address
actually linked to a given withdraw address and we interpret this fact as not
having more information about the participant addresses of the actual
transaction than the size of the TC anonymity set. A heuristic $\mathcal{H}$
(or combination of heuristics) producing a refined set of candidates
$\mathcal{C}=\mathcal{C}(y)$ will then update our prior belief and potentially
locate the actual deposit address in this set: now the conditional entropy
$H(\mathcal{C})$ is subtracted from the a priori entropy resulting in a
quantity known as conditional information gain:

$\displaystyle I(X,X|Y=y)=H(X)-H(X|Y=y)=$
$\displaystyle=\ln(\lvert\mathcal{D}\rvert)-\ln(\lvert\mathcal{C}\rvert)$

This turns out to be equal to the Kullback-Leibler divergence between the
prior and posterior probability mass functions for the anonymity sets. It
measures how much information is gained upon running the heuristic. In this
sense, larger sets $\mathcal{C}$ produce little information gain, whereas
smaller ones produce larger information gains: in the case of $\mathcal{C}$
containing the actual deposit address corresponding to a given withdrawal, we
interpret this information gain as a greater anonymity loss.

## References

  * Antonopoulos and Wood (2018) Andreas M Antonopoulos and Gavin Wood. 2018.  _Mastering ethereum: building smart contracts and dapps_.  O’reilly Media.
  * Béres et al. (2021) Ferenc Béres, István A Seres, András A Benczúr, and Mikerah Quintyne-Collins. 2021.  Blockchain is watching you: Profiling and deanonymizing ethereum users.  In _2021 IEEE International Conference on Decentralized Applications and Infrastructures (DAPPS)_ , pages 69–78. IEEE.
  * Buterin et al. (2013) Vitalik Buterin et al. 2013.  Ethereum white paper.  _GitHub repository_ , 1:22–23.
  * Christin (2013) Nicolas Christin. 2013.  Traveling the silk road: A measurement analysis of a large anonymous online marketplace.  In _Proceedings of the 22nd international conference on World Wide Web_ , pages 213–224.
  * Diaz et al. (2002) Claudia Diaz, Stefaan Seys, Joris Claessens, and Bart Preneel. 2002.  Towards measuring anonymity.  In _International Workshop on Privacy Enhancing Technologies_ , pages 54–68. Springer.
  * Goodfellow et al. (2016) Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016.  _Deep learning_.  MIT press.
  * Grover and Leskovec (2016) Aditya Grover and Jure Leskovec. 2016.  node2vec: Scalable feature learning for networks.  In _Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining_ , pages 855–864.
  * Haslhofer et al. (2016) Bernhard Haslhofer, Roman Karl, and Erwin Filtz. 2016.  O bitcoin where art thou? insight into large-scale transaction graphs.  In _SEMANTiCS (Posters, Demos, SuCCESS)_.
  * Johnson et al. (2019) Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019.  Billion-scale similarity search with gpus.  _IEEE Transactions on Big Data_.
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_.
  * Meiklejohn and Mercer (2018) Sarah Meiklejohn and Rebekah Mercer. 2018.  Möbius: Trustless tumbling for transaction privacy.
  * Mikolov et al. (2013a) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a.  Efficient estimation of word representations in vector space.  _arXiv preprint arXiv:1301.3781_.
  * Mikolov et al. (2013b) Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013b.  Distributed representations of words and phrases and their compositionality.  In _Advances in neural information processing systems_ , pages 3111–3119.
  * Möser et al. (2013) Malte Möser, Rainer Böhme, and Dominic Breuker. 2013.  An inquiry into money laundering tools in the bitcoin ecosystem.  In _2013 APWG eCrime researchers summit_ , pages 1–14. Ieee.
  * Nakamoto (2008) Satoshi Nakamoto. 2008.  Bitcoin: A peer-to-peer electronic cash system.  _Decentralized Business Review_ , page 21260.
  * Pertsev et al. (2019) Alexey Pertsev, Roman Semenov, and Roman Storm. 2019.  Tornado cash privacy solution version 1.4.
  * Rozemberczki and Sarkar (2018) Benedek Rozemberczki and Rik Sarkar. 2018.  Fast sequence-based embedding with diffusion graphs.  In _International Workshop on Complex Networks_ , pages 99–107. Springer.
  * Seres et al. (2019) István András Seres, Dániel A Nagy, Chris Buckland, and Péter Burcsi. 2019.  Mixeth: efficient, trustless coin mixing service for ethereum.  In _International Conference on Blockchain Economics, Security and Protocols (Tokenomics 2019)_. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik.
  * Victor (2020) Friedhelm Victor. 2020.  Address clustering heuristics for ethereum.  In _International Conference on Financial Cryptography and Data Security_ , pages 617–633. Springer.


11institutetext: Information Sciences Institute, University of Southern
California
Marina del Rey CA 90292, USA
11email: {nananuku<EMAIL_ADDRESS>
# HALO: An Ontology for Representing Hallucinations in Generative Models

Navapat Nananukul 11 Mayank Kejriwal 11

###### Abstract

Recent progress in generative AI, including large language models (LLMs) like
ChatGPT, has opened up significant opportunities in fields ranging from
natural language processing to knowledge discovery and data mining. However,
there is also a growing awareness that the models can be prone to problems
such as making information up or ‘hallucinations’, and faulty reasoning on
seemingly simple problems. Because of the popularity of models like ChatGPT,
both academic scholars and citizen scientists have documented hallucinations
of several different types and severity. Despite this body of work, a formal
model for describing and representing these hallucinations (with relevant
meta-data) at a fine-grained level, is still lacking. In this paper, we
address this gap by presenting the _Hallucination Ontology_ or HALO, a formal,
extensible ontology written in OWL that currently offers support for six
different types of hallucinations known to arise in LLMs, along with support
for provenance and experimental metadata. We also collect and publish a
dataset containing hallucinations that we inductively gathered across multiple
independent Web sources, and show that HALO can be successfully used to model
this dataset and answer competency questions.

###### Keywords:

Hallucinations Generative AI Large Language Model.

Ontology https://bitly.ws/34Sdp

Documentation: https://bitly.ws/34L6U

GitHub: https://github.com/navapatn/halo-ontology

License: CC BY 4.0 International

DOI: https://doi.org/10.5281/zenodo.10279463

##  1 Introduction

The recent advent of large language models (LLMs) [16, 23, 31] and other
generative AI (GenAI) systems [19, 30, 35] has led to impressive success on a
wide set of tasks, in communities ranging from natural language processing to
computer vision [3, 26, 28]. While the success of these models is promising,
concerns have been raised about their responsible use and deployment [36].
Like other deep learning models preceding them, such models can be prone to
faulty (and often, unpredictable) reasoning on relatively simple problems, and
can also ‘make up’ or _hallucinate_ information [18].

The issue of hallucinations, in particular, has raised some important concerns
such as misinformation and over-reliance on the system’s output, especially
when the system _seems_ to be giving the correct answer in response to a
prompt [37]. It has also become evident through several recently published
studies [28, 33] that hallucinations are not uncommon occurrences. Even
‘citizen scientists,’ who have been playing with models like ChatGPT owing to
its easy-to-use interface, have documented a wide body of hallucinations even
in common Web forums.

This paper is motivated by the observation that, despite the growing number of
studies (whether academic or informal), a standard vocabulary or ontology for
representing hallucinations has thus far been lacking. Such an ontology, if it
existed, and datasets modeled using the classes and predicates in the
ontology, would serve some important hypothetical use-cases. For example, they
would allow us to gain a better understanding of, and represent, different
_categories_ of hallucinations that have been documented. However, without a
common vocabulary, data collected on this important phenomenon would be ad-hoc
and require significant manual work, making it more difficult and time-
consuming to conduct systematic empirical studies.

At the same time, any such ontology can only have practical utility if it is
extensible, well-documented, and amenable to modeling both hallucination
_instances_ and their _metadata_ , which include details such as what LLMs the
hallucination instance was tried on, when it was tried, its provenance, and so
on. Even preliminary evaluations show that all LLMs don’t hallucinate equally:
on average, given a set of prompts, there are clear differences between the
major language models currently in circulation, as we subsequently show.

With these motivations in mind, we propose the _Hallucination Ontology_
(HALO), a resource for modeling hallucination instances and their metadata.
Specific contributions in this work are as follows:

  * •

We propose and describe HALO, an ontology that treats hallucinations in GenAI
models as first-class citizens of study. HALO is released under an open
license, follows the FAIR principles, and is intended to be sustainable and
extensible in the foreseeable future, serving as an ontological resource for
researchers who continue to study and gather data on hallucinations in GenAI
models and LLMs.

  * •

HALO was constructed following a rigorous design process and has support for
multiple hallucination categories and sub-categories, metadata fields and
attributes, and an attractive set of technical specifications and complete
documentation. It contains two high-level modules, called the _Hallucination
Module_ and _Metadata Module_ , which allows us to maintain clear separation
between the (more extensible) categories representing different hallucination
types, and the (more standard) concepts and properties that are better suited
for capturing experimental data on hallucinations.

  * •

We demonstrate the utility of HALO using a set of competency questions (CQs)
that are similar to those normally studied in empirical studies of
hallucination. We use a real dataset on hallucinations collected from a
diverse set of sources on the Web, and show that even relatively complex CQs
can be formalized as SPARQL queries using HALO concepts and properties. We
verify the correctness of the answers obtained from executing these queries.

##  2 Related Work

Hallucinations in Generative AI (GenAI): The advent of GenAI and large
language models (LLMs) like ChatGPT and BARD mark a significant evolution from
an earlier generation of models. Technically, such models are distinguished by
their use of reinforcement learning in addition to deep transformer neural
networks based on attention. Although deep learning models had been known to
be prone to problems such as overconfidence and adversarial attacks,
hallucinations have been recognized as a relatively novel class of errors and
‘false predictions’ in the context of GenAI [34, 29]. Hallucinations are not
uncommon, and researchers have documented them (or errors resembling them) in
a diverse range of generative tasks, such as text summarization [18], text
generation [32, 22], multi-modal problem solving [3], dialogue [14], question-
answering [7, 20, 36], and language translation [15, 12]. Inevitably, as we
have learned more about hallucinations as a community, their definition and
scope has expanded. Much more recently, the evaluation of hallucinations in
LLMs has transitioned to focus more on the distinction between factual
reality, fabricated results, and the ability of LLMs to follow human
instructions [28], as opposed to problems that can be traced back purely to
faults in abstract reasoning or over-confidence.

Based on performance metrics, modern LLMs are considered definitively superior
compared to an earlier generation of language models like BERT. They can
handle complex prompts and instructions, but this complexity has also resulted
in more sophisticated hallucination patterns [5]. Researchers are already
attempting to define and categorize hallucinations in LLMs in a rigorous way,
but efforts are ad-hoc and do not rely on the formal machinery that is a
strength of the Semantic Web [13, 37]. Nevertheless, such works do offer
insights into hallucination causes, types, and potential solutions, even
though they do not always take into account the hallucinations published in
non-academic sources.

Ontology development and guidelines: To the best of our knowledge, there is no
existing hallucination ontology in the literature that has a formal
representation. Recognizing this gap, we drew upon the Linked Open Terms (LOT)
methodology [25] as our guiding framework. LOT’s methodology is particularly
suited to our goal of developing a robust, extensible ontology from scratch.
Moreover, to connect hallucination in LLMs with existing classes and enhance
interoperability, we also follow the _Minimum Information to Reference an
External Ontology Term (MIREOT)_ guidelines [6], as subsequently discussed.
Beyond hallucinations, obviously, there are many resources and decades of
related work in the Semantic Web on best practices for publishing ontologies
expressing novel phenomena. An exhaustive survey of this research is well
beyond the scope of any reasonable work, but some recent research that is
relevant to the design and development of HALO include [2, 17, 11, 9, 1].

##  3 HALO: Design and Development

In this section, we present the ontology design and development process for
HALO. Much of this process followed the Linked Open Terms (LOT) methodology
[25], an industrial method for developing ontologies and vocabularies. The LOT
methodology is structured around iterative cycles through a foundational
workflow, including four phases: (i) ontological requirements specification,
(ii) ontology implementation, (iii) ontology publication, and (iv) ontology
maintenance. The methodology also places much emphasis on the reuse of terms
(ontology classes, properties, and attributes) in published vocabularies or
ontologies. Before describing these four steps and how they apply to HALO, we
begin by describing a use case motivating the need for an ontology like HALO.

###  3.1 Use Case

An important use case for HALO is to serve as a standard vocabulary and model
for representing the many hallucinations that are being discovered and
described by a broad set of individuals, not just academic scholars. Examples
of hallucination can be found on the Web, especially for the more popular LLMs
such as ChatGPT, published by the likes of journalists, technologist
enthusiasts, and others, who in their role as “citizen scientists”, are
experimenting organically with the LLMs. This data is valuable and may even
serve an important role both in formulating interesting research questions and
in helping companies and organizations improve the LLMs in further iterations,
but because the data and “experiments” are so decentralized, they are
currently not being captured either in a central forum or using a standardized
vocabulary. HALO is meant to serve the latter purpose more than the former,
but we also demonstrate its feasibility for the former when we evaluate its
merits in Section 6. Some of the types of questions that HALO can help us
answer include: are LLMs relatively uniform in which prompts they hallucinate
on, or are there significant differences between LLMs? What kinds of
hallucination categories are most common? Furthermore, given longitudinal
data, can we determine if LLMs are ‘improving’ on some categories of
hallucinations, compared to others? While the last of these cannot be
addressed with the data we currently have available, due to the recency of the
models, we do address the first two in Section 6.

###  3.2 Specification of Ontology Requirements

This subsection details two types of requirements considered in HALO’s
construction:

  1. 1.

Functional requirements are defined by competency questions (CQs), which
contain multiple questions on different views related to GenAI hallucinations.
Additionally, SPARQL queries corresponding to each CQ were created to validate
HALO (Section 6.2).

  2. 2.

Non-functional requirements for HALO include _extensibility_ ,
_interoperability_ , and _maintainability_. _Extensibility_ is important
because HALO was designed to evolve and expand as we learn more about GenAI
and the different categories of hallucinations it is prone to. Hence, we
followed hallucination taxonomies presented in the most recent literature on
the subject, and which have a clear hierarchical structure and are easy to
understand. _Interoperability_ is also an important feature we aimed to
achieve when building HALO, by connecting each hallucination type with
external classes in order to reuse existing resources from published
vocabularies. Finally, we also considered _maintainability_ as an important
design element when building HALO. In addition to following standard
guidelines in the ontology’s construction and design, we also created a
pipeline to extend the hallucination categories and broaden the scope of
external classes. This pipeline includes Google Forms, Sheets, and a Python-
based program to convert the data into RDF/XML format.

###  3.3 Ontology Implementation

In developing HALO, we aimed to closely adhere to the general use cases and
motivations we had previously introduced in Section 1. For example, we had
stated therein that supporting a broad set of experimental studies involving
actual hallucinations was an important reason for proposing a standard
vocabulary. To conceptualize this need, HALO was designed to contain two
modules: (1) the _Metadata Module_ and (2) the _Hallucination Module_. We
created the initial conceptual model using
Chowlk111https://chowlk.linkeddata.es/index.html and implemented the OWL
version using Protégé [21]. From a socio-technical perspective, the design and
developmental process of HALO involved iterative internal discussions aimed at
refining the ontology to implement the requirements (both functional and non-
functional) noted earlier. These discussions spanned both data collection and
literature review strategies for ensuring that, even in its first version,
HALO contained a minimal set of hallucination categories and external entities
for representing the hallucinations that people had already discovered.
Through the review of the literature, we also sought to align our categories
with hallucination terminologies to ensure that we were using commonly used
terms. This ensured that our ontology remained up-to-date with the latest
findings and best practices in the growing literature in this area. Finally,
we evaluated HALO followed the standard LOT process by asking for feedback and
reviewing external knowledge, not just in the academic literature but also
informal sources on the Web. Following these guidelines, we created competency
questions and corresponding SPARQL queries to fully validate the ontology.
Using this iterative process, we were able to refine HALO to its current
version, the OWL version of which is provided in the resource links included
with this paper.

###  3.4 Linking External Vocabularies

HALO is created based on two existing primary vocabulary sources: (1)
hallucination categories and (2) external classes/entities. As a key objective
is to ensure interoperability, we took the following systematic approach:

Identify Top-Level Hallucination Categories: As a first step, we identified
primary categories of hallucinations observed in LLMs through a careful review
of the literature [13, 37]. These categories form the foundational classes in
HALO’s _Hallucination Module_.

Link Hallucinations with External Classes: We planned to import related
vocabularies to make HALO as comprehensive as possible, but because not all
classes and properties in such vocabularies (like FOAF) are relevant, and to
avoid considerable overhead of full imports, we adhered to the _Minimum
Information to Reference an External Ontology Term (MIREOT)_ guidelines, which
suggest that the essential information needed to reference an external concept
includes the ontology namespace, the URI of the specific term, and the URI for
the term’s superclass [6]. By following MIREOT, we selectively integrated
relevant external concepts without also including superfluous terms. We also
established connections between external classes that might appear in actual
hallucination instances e.g., when an LLM hallucinates a fact about a person,
we connect foaf:person to our hallucination module, ensuring that the
semantics of the hallucination are reliably represented in reused vocabulary
terms.

###  3.5 Ontology Publication and Maintenance

HALO is published under the open CC-BY 4.0 license in two formats: (1) a
machine-readable (OWL) file following ontology-publication best practices [4],
with the files available both in a GitHub
repository222https://github.com/navapatn/halo-ontology and a
Zenodo333https://zenodo.org/records/10279463 repository; (2) a human-readable
document that allows for more conceptual debate on how to represent
hallucination categories and metadata. We documented HALO using Widoco [10],
which generates a series of interlinked HTML pages offering a human-readable
version of HALO, derived directly from its content. This documentation is made
publicly accessible online through a persistent identifier, allowing us to
fulfill FAIR’s _Findability_ (specifically, F1, F2, and F4) requirements.

Concerning ontology maintenance, we plan to keep an active GitHub issue
tracker, with the repository open for users to submit issues and suggestions.
In particular, we solicit discovery of bugs through this avenue, and pitfalls
that we may have missed in this version, although we also intend the tracker
to enable external collaboration, requests for extension and general feedback.

To further maintain and promote the ontology, we will also be launching a
Google form that will allow users or researchers to report actual
hallucination instances that we will model using the ontology. This form,
although not directly related to the development or extension of the ontology
itself, is expected to facilitate an important use case that motivated the
development of HALO (collecting and analyzing experimental data on
hallucinations in a standard and relatively automated way). Particularly, we
hope to periodically review submitted hallucinations to determine whether the
ontology should be extended with more hallucination categories.

##  4 HALO: Hallucination Ontology

This section describes the architecture and elements of the HALO, which is
structured into two principal modules: (1) the _Hallucination_ module, and (2)
the _Metadata_ module. We begin by defining (Section 4.1) the different
hallucination categories that HALO supports and that evidence has shown that
the popular LLMs (such as ChatGPT and BARD) exhibit. The categories and
definitions below provide the foundation for HALO and the phenomena it aims to
capture. Figure 1 schematizes the ontology in a visually accessible manner. As
discussed earlier, the ontology has been formally published and released under
an open-source license as an OWL file, and is designed to be extensible.
Following a discussion of the hallucination categories and sub-categories, we
detail the two different modules in Sections 4.2 and 4.3.

Figure 1: A visualization of HALO, including the _Metadata_ and
_Hallucination_ modules that describe it at the highest level and are further
articulated in the main text.

###  4.1 Hallucination Categories

Even in the short time that LLMs have been exposed to the public, several
studies have found that hallucinations that they are prone to are not all
equal [13, 37]. Guided by this literature, we chose to include two main
hallucination categories (with subcategories) into HALO. Eventually, we
believe that more hallucination categories and subcategories might be
discovered, which is the reason that the hallucinations below are sub-classes
of a broader _hallucination_ class.

1\. Factuality Hallucinations refers to errors in LLMs outputs by
misrepresenting real-world facts. These errors manifest in two distinct ways:

  * •

Factual Inconsistency occurs when LLMs produce answers or responses that
reference real-world facts but include contradictions or misinformation.

Example: LLMs incorrectly identify “Yuri Gagarin” as the first person to land
on the Moon, a clear contradiction of established historical facts.

  * •

Factual Fabrication refers to LLMs producing responses with supporting
evidence that cannot be verified by established real-world information.

Example: An LLM fabricates a detailed historical narrative about dragons
existing on Earth despite no empirical evidence or historical records
supporting such a claim.

2\. Faithfulness Hallucination refers to LLM outputs that fail to align
faithfully with user instructions, when provided with context, including well-
known logical rules based in objective fields like math or physics, for
example. We further sub-categorize it into three subtypes:

  * •

Instruction Inconsistency arises when LLM outputs do not follow the user’s
prompt instructions.

Example: Users intend to request a translation of a specific sentence, but the
LLM erroneously performs a question-answering to that sentence instead of
translating it.

  * •

Context Inconsistency occurs when the LLM’s output is not aligned to the
user’s provided contextual information.

Example: LLMs provided an answer about great food for dinner when a user asked
about breakfast, and it clearly stated in the users’ prompt

  * •

Logical Inconsistency occurs when an LLM’s output exhibits internal logical
contradictions, often observed in reasoning tasks444There is some controversy
whether artifacts like logical inconsistencies are ‘hallucinations’ or just
ordinary drops in performance. No AI-based reasoning system is known to have
perfect performance, given non-trivial natural language inputs. However, there
is reason to believe that the errors produced by LLMs is grounded, not in
faulty reasoning, but faulty interpretation of inputs or context, which better
resembles hallucinatory phenomena. In this paper, we err on the side of
caution and take a reasonably broad view of hallucinations..

Example: LLM might correctly perform mathematical steps, but then conclude
with an incorrect final answer

###  4.2 Hallucination Module

The _Hallucination_ module contains classes for each of the categories and
sub-categories defined in Section 4.1 and is the main module in HALO. A key
feature of the module is its integration with terms from existing
vocabularies, in adherence to the Linked Data standards. Below, we discuss the
two important features of the module.

Hierarchical structure: The hallucination categories and sub-categories we
defined earlier are best encoded as a taxonomy, using appropriate owl:
subclassOf relations. This is also shown in Figure 1. In line with the
categorization provided earlier, there are two main branches from the major
LLMsHallucination root class: (1) factualityHallucination which comprises two
sub-classes FactualFabrication and factualInconsistency (2)
faithfulnessHallucination which comprises of three sub-classes
logicalInconsistency, instrctionInconsistency, and contextInconsistency. Note
also that the taxonomy, laid out using a relatively simple structure, ensures
that the ontology can be expanded as new hallucination categories and sub-
categories emerge from the intense research that is being conducted at the
present moment on LLMs.

Instances within this module correspond to specific hallucinated prompts and
responses identified by a researcher, or through mechanisms like citizen
science. As we show in Section 6, when we evaluate the merits of HALO using a
real-world hallucination dataset collected from multiple sources from the Web,
these instances often involve real-world entities such as individuals’ names,
geographic locations, and documented events, varying according to the user-
generated prompts. Examples of external classes we connected to this module
include foaf:person, foaf:document, schema:date, schema:schorlarlyArticle,
etc.

Finally, the ontology also provides support for recording the metadata and
provenance of the different hallucinations that can be represented as
instances of the hallucination classes and sub-classes (Section 4.3).

###  4.3 Metadata Module

The _Metadata_ module represents the core concept and properties of the
prompts and answers we gathered. We visually represent its structure in the
top pane in Figure 1. The module comprises four primary concepts:
GenerativeAI, LargeLanguageModel, LLMsPrompt, and LLMsAnswers, each briefly
described below:

  1. 1.

GenerativeAI describes broadly the types of AI agents that showed
hallucinations. The instance of this class can be an AI agent from any field
such as text generation models, image generation models, or LLMs. Since our
initial experiments are tested on LLMs, the GenerativeAI class has only one
subclass (LargeLanguageModel) for the current version of HALO.

  2. 2.

LargeLanguageModel describes instances of LLMs that exhibit hallucination,
including names and model versions.

  3. 3.

LLMsPrompt instantiates the metadata of the collected prompts, with attributes
such as (1) hasPromptID, an internal identifier for tracking prompts; (2)
CollectedOn, the date of prompt acquisition; and (3) hasSource, the internet
source we found the prompt.

  4. 4.

LLMsAnswer encapsulates metadata about the LLMs’ responses, featuring (1)
hasAnswerID for internal answer tracking and (2) hasAnswerDate for recording
the date of the response. Additionally, LLMsAnswer serves as a bridge between
the Metadata and Hallucination modules, linked by the relation metadata:
hallucinationGeneratedBy, indicating that an LLM’s answer we add into our
ontology generated a specific hallucination type.

##  5 Technical Specifications

This section summarizes HALO’s technical specifications augmenting those
mentioned in Section 3, including _interoperability, indexing and
availability, logical correctness and validation_ , and _documentation_.

Interoperability: We imported 7 external classes into HALO, as one of our
goals in designing HALO is to represent hallucinations that involve real-world
entities and concepts, which are already present in vocabularies like FOAF.
Note that the core implementation of HALO is OWL-based in order to achieve
FAIR’s Interoperability (I1) principle.

Indexing and availability: We release the ontology publicly, under an open CC
BY 4.0 International license, in Zenodo
555https://zenodo.org/records/10279463. HALO can be browsed using any web
browser, and using visualization tools such as WebVOWL.

Logical correctness and validation: We validated HALO against a known ontology
scanner tool called _OOPS! Ontology Pitfall Scanner_ [24]. This tool verifies
29 common pitfalls in ontology development, including creation of polysemous
elements, synonyms as classes, and so on. The tool notifies users of three
critical levels based on the number of pitfalls and the impacts on the
ontology. HALO passed the evaluation with zero pitfalls using this evaluation
tool.

Documentation: We documented HALO using the _Widoco wizard_ [10] as mentioned
in subsection 3.5. The documentation666https://bitly.ws/34L6U aims to provide
a comprehensive explanation to broad classes of users, not just domain-
specific researchers. The documentation is fully accessible online, and a live
visualization of the ontology is also included therein using the WebVOWL
plugin, as mentioned earlier.

##  6 Evaluation

###  6.1 Hallucination Dataset and Sources

In our effort to model and evaluate hallucination instances observed in actual
LLMs using HALO, we compiled 40 prompts known to induce hallucinatory
responses in LLMs from a diverse set of Web sources. The prompts span multiple
document types, including Reddit posts, news articles from The New York Times,
and reports from Opengov, among others. During this collection process, we
also compiled information on applicable metadata, such as the specific LLM
known to have produced the hallucination, date when we found the
hallucination, Web source link, name, and type (e.g., blog post, newspaper
article). We tabulate descriptive statistics on this collected data in Table
1.

Table 1: Descriptive statistics and representative examples from the hallucination dataset that was collected for evaluating HALO Statistics |  Count |  Example
---|---|---
Number of Prompts |  40 |  [How many games did the New Jersey Devils win in 2014]
Number of Sources |  9 |  [Reddit]
Number of Metadata Collected |  5 |  [Source, Source_link, Date, etc.]
Number of Document Types |  4 |  [Post]
Date found |  - |  [April 1, 2023]

When studying hallucinations in LLMs, an important analytical objective is to
assess whether hallucinations _persist_ across LLMs i.e., if a prompt led to a
hallucination in ChatGPT, does it also lead to a hallucination in BARD, more
often than not? HALO can enable us to pursue such objectives if the dataset
contains a diverse set of prompts, hallucinations, and LLMs. All
hallucinations in the dataset that we described earlier were originally tested
on one of three different LLMs: ChatGPT (GPT-3.5 version), BARD, and Claude.
At present, these models represent the forefront in language-based GenAI
research and are briefly described below:

ChatGPT (v3.5)777https://chat.openai.com/ is a variant of the GPT (Generative
Pretrained Transformer) family of generative language models developed by
OpenAI, and has become widely known for its conversational abilities and its
general knowledge.

BARD888https://bard.google.com/chat is an LLM developed by Google AI. This
model utilizes a transformer-based encoder-decoder architecture, incorporating
a novel attention mechanism that allows it to effectively capture long-range
dependencies within the input text.

Claude999https://claude.ai/chats is an AI assistant developed by Anthropic.
Similar to other LLMs, it was also designed to learn from human feedback and
conversations, and to be more introspective about admitting its mistakes,
refusing inappropriate requests, and discussing options that are more aligned
with human ethics.

Because the original dataset we collected did not evaluate all prompts on all
three LLMs, to showcase HALO’s utility, we first ‘completed’ the dataset by
running all 40 prompts to all three LLMs, yielding a total of 120 data points.
Figure 2 shows the knowledge graph of a data point from HALO. This served two
purposes: first, it allowed us to verify that a hallucination was indeed an
hallucination on the model on which it was documented; second, it allows us to
pursue the stated objective through a competency query (CQ), as subsequently
described. As shown in Table 2, even this relatively small dataset suggests
that there is variation in the frequency and nature of hallucinations across
the LLMs. A cursory analysis shows that, of the 40 prompts tested, BARD had
the highest hallucination rate at 62.5%, followed by ChatGPT (42.5%) and
Claude (40.0%). Particularly, BARD’s inability to refuse to respond to user
prompts might have contributed to its higher rate of hallucinations. These
verified hallucination instances, including metadata and corresponding
responses from LLMs, form the foundational dataset for evaluating HALO and
showcasing its utility using the CQs described in the next section.

Table 2: Validation statistics on hallucination experiments (using the dataset described in Table 1) on the three LLMs. Statistic |  ChatGPT (GPT-3.5) |  BARD |  Claude
---|---|---|---
# Hallucinations |  17 |  25 |  16
# Correct answers |  13 |  15 |  13
# Refuse to answer |  10 |  0 |  11
Hallucination Rate |  42.5% |  62.5% |  40.0%
Figure 2: An illustrative example of a hallucination modeled by HALO

###  6.2 Ontology Evaluation

We evaluated HALO following the guidelines proposed in [25]. Our primary
objective in this section is to show that HALO is comprehensive enough to
model a range of observed hallucination types and can support a relatively
rich set of analytical queries. For the latter, we devised a set of competency
questions (CQs), shown in Table 3.

Table 3: A sample of the competency questions (CQs). Id |  Question Text
---|---
CQ1 |  What different types of generative AIs exhibit hallucination?
CQ2 |  What are the different types of hallucination that occurred by large language models :x?
CQ3 |  What is the ranking of hallucination types generated by model :x?
CQ4 |  Which large language model exhibits the most hallucination type :x?
CQ5 |  What pair of LLMs shared the same type of hallucinations the most?
Table 4: Sample test cases for evaluating HALO on CQs 2, 3, and 4. CQ |  Input(s) |  Expected Result(s)
---|---|---
CQ2 |  GPT-3.5 |  FF, FI LI, CI
CQ3 |  BARD |  (1) FI, (2) FF, (3) CI, (4) II, (5) LI
CQ4 |  FactualityFabrication |  BARD

Given these CQs, the evaluation is predicated on showing that HALO can be used
to provide accurate answers to them. For example, based on the dataset, we
would expect the response to CQ1 to be that _LLMs_ (which are sub-types of
GenAI) exhibit hallucination, and the response to CQ2 to be the _Factuality_
hallucination type is the most frequent type to be present. To conduct the
actual evaluation, as is standard in other related work [8, 27], we first
translate the CQs (expressed informally in natural language as shown in Table
3) into formal SPARQL queries framed using the concepts and properties in
HALO. We note that, because HALO contains both a _Hallucination_ and
_Metadata_ module, it contains the necessary ontological elements for modeling
the CQs as equivalent SPARQL queries.

Table 4 presents a selection of the specific test cases used to verify HALO
based on the CQs for the verification of HALO. We found that HALO achieved
perfect scores across these deployed test cases, showing (to a reasonable
first approximation) not only that the ontology is rich enough for us to
formally express the types of CQs in Table 3 as SPARQL queries, but also that
the current version of HALO has been correctly implemented and is giving
expected results for those queries. We also conducted further testing to
verify the latter of these claims, and to ensure to the best of our ability
that the public version of the ontology does not have implementation or
logical errors.

Figure 3: An example of SPARQL used to validate CQ5 Table 5: The results from executing the SPARQL query corresponding to CQ5, expressed as a confusion matrix. Since the query only counts hallucinations common to distinct (pairs of) LLMs, the diagonal entries do not apply. | GPT-3.5 | Claude | BARD
---|---|---|---
GPT-3.5 | - | 9 | 12
Claude | 9 | - | 10
BARD | 12 | 10 | -

Due to space limitations, we only show the most complex SPARQL query (shown in
Figure 3) used to verify CQ5, with execution results tabulated in Table 5. We
ran this query over 60 hallucination instances and consequently populated the
confusion matrix, which shows the pairwise association between different
models’ (GPT-3.5, Claude, and BARD) hallucinations. The first row shows, for
example, that there are nine common prompts on which GPT-3.5 and Claude both
hallucinate. Overall, the successful execution of this query on HALO enables
this type of analysis that enables us to compare different LLMs on their
ability to hallucinate. In the future, as more instances of hallucination
classes in the _Hallucination Module_ are collected, and the dataset used in
this section is augmented, it becomes possible to conduct analyses of even
broader scope using a set of CQs modeled in the same vein as CQ5.

##  7 Conclusion and Future Work

Since the release of ChatGPT, DALL-E and other such models, there has been
enormous progress on GenAI problems across domains and applications. However,
the complexity of these deep learning systems has also given rise to unique
problems, such as hallucinations. Over the last few months, both academics and
ordinary people have discovered and documented interesting sets of
hallucinations when ‘playing’ with these models. A systematic study of
hallucinations remains uncommon, and to our knowledge, a standard vocabulary
for expressing and analyzing them is still lacking. In this paper, we proposed
HALO, a hallucination ontology that provides rich facilities for modeling six
diverse types of hallucinations (themselves categorized into two larger
categories), and that also provides support for modeling experimental metadata
on actual hallucinations. We demonstrated the utility of HALO using a set of
practical competency queries that would likely be of interest to researchers
and practitioners alike.

There are many avenues for future research. In releasing HALO publicly, and
adhering to time-established principles (such as using external vocabularies
where possible), we hope to sustain and extend the ontology in the long-term
future. Without a doubt, more categories and sub-categories of hallucinations
will have to be added to HALO both as the existing GenAI systems and LLMs
become more powerful (and consequently, hallucinate in ways we have not
observed yet), and as we study existing LLMs in more detail. We also hope to
add support for longitudinal experimental studies to HALO, and to supplement
the competency queries with documented hallucinations from other (e.g., non-
language-based) GenAI models.

## References

  * [1] M. Amith, K. Fujimoto, R. Mauldin, and C. Tao.  Friend of a friend with benefits ontology (foaf+): extending a social network ontology for public health.  BMC Medical Informatics and Decision Making, 20(10):1–14, 2020.
  * [2] S. Bader, J. Pullmann, C. Mader, S. Tramp, C. Quix, A. W. Müller, H. Akyürek, M. Böckmann, B. T. Imbusch, J. Lipp, et al.  The international data spaces information model–an ontology for sovereign exchange of digital content.  In International Semantic Web Conference, pages 176–192. Springer, 2020.
  * [3] Y. Bang, S. Cahyawijaya, N. Lee, W. Dai, D. Su, B. Wilie, H. Lovenia, Z. Ji, T. Yu, W. Chung, Q. V. Do, Y. Xu, and P. Fung.  A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity, 2023.
  * [4] D. Berrueta and J. Phipps.  Best practice recipes for publishing rdf vocabularies, 08 2008.
  * [5] S. Bubeck, V. Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Kamar, P. Lee, Y. T. Lee, Y. Li, S. Lundberg, H. Nori, H. Palangi, M. T. Ribeiro, and Y. Zhang.  Sparks of artificial general intelligence: Early experiments with gpt-4, 2023.
  * [6] M. Courtot, F. Gibson, A. Lister, J. Malone, D. Schober, R. Brinkman, and A. Ruttenberg.  Mireot: the minimum information to reference an external ontology term.  Applied Ontology, 6:23–33, 01 2011.
  * [7] J. Cui, Z. Li, Y. Yan, B. Chen, and L. Yuan.  Chatlaw: Open-source legal large language model with integrated external knowledge bases, 2023.
  * [8] S. Fathalla, C. Lange, and S. Auer.  An upper ontology for modern science branches and related entities.  In The Semantic Web: 20th International Conference, ESWC 2023, Hersonissos, Crete, Greece, May 28–June 1, 2023, Proceedings, page 436–453, Berlin, Heidelberg, 2023. Springer-Verlag.
  * [9] A. P.-D. l. Gala, Y. Cardinale, I. Dongo, and R. Ticona-Herrera.  Towards an ontology for urban tourism.  In Proceedings of the 36th annual ACM symposium on applied computing, pages 1887–1890, 2021.
  * [10] D. Garijo.  Widoco: A wizard for documenting ontologies.  In C. d’Amato, M. Fernandez, V. Tamma, F. Lecue, P. Cudré-Mauroux, J. Sequeda, C. Lange, and J. Heflin, editors, The Semantic Web – ISWC 2017, pages 94–102, Cham, 2017. Springer International Publishing.
  * [11] D. Garijo and M. Osorio.  Oba: An ontology-based framework for creating rest apis for knowledge graphs.  In The Semantic Web–ISWC 2020: 19th International Semantic Web Conference, Athens, Greece, November 2–6, 2020, Proceedings, Part II 19, pages 48–64. Springer, 2020.
  * [12] N. M. Guerreiro, D. Alves, J. Waldendorf, B. Haddow, A. Birch, P. Colombo, and A. F. T. Martins.  Hallucinations in large multilingual translation models, 2023.
  * [13] L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin, and T. Liu.  A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions, 2023.
  * [14] S. Jha, S. K. Jha, P. Lincoln, N. D. Bastian, A. Velasquez, and S. Neema.  Dehallucinating large language models using formal methods guided iterative prompting.  In 2023 IEEE International Conference on Assured Autonomy (ICAA), pages 149–152, 2023.
  * [15] Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, and P. Fung.  Survey of hallucination in natural language generation.  ACM Comput. Surv., 55(12), mar 2023.
  * [16] M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer.  Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension, 2019.
  * [17] H. Li, R. Armiento, and P. Lambrix.  An ontology for the materials design domain.  In International Semantic Web Conference, pages 212–227. Springer, 2020.
  * [18] J. Li, X. Cheng, W. X. Zhao, J.-Y. Nie, and J.-R. Wen.  Halueval: A large-scale hallucination evaluation benchmark for large language models, 2023.
  * [19] Y. Li, D. Choi, J. Chung, N. Kushman, J. Schrittwieser, R. Leblond, T. Eccles, J. Keeling, F. Gimeno, A. D. Lago, T. Hubert, P. Choy, C. de Masson d’Autume, I. Babuschkin, X. Chen, P.-S. Huang, J. Welbl, S. Gowal, A. Cherepanov, J. Molloy, D. J. Mankowitz, E. S. Robson, P. Kohli, N. de Freitas, K. Kavukcuoglu, and O. Vinyals.  Competition-level code generation with alphacode.  Science, 378(6624):1092–1097, 2022.
  * [20] P. Manakul, A. Liusie, and M. J. F. Gales.  Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models, 2023.
  * [21] M. A. Musen.  The protégé project: A look back and a look forward.  AI Matters, 1(4):4–12, jun 2015.
  * [22] N. Mündler, J. He, S. Jenko, and M. Vechev.  Self-contradictory hallucinations of large language models: Evaluation, detection and mitigation, 2023.
  * [23] OpenAI.  Gpt-4 technical report, 2023.
  * [24] M. Poveda-Villalón, A. Gómez-Pérez, and M. C. Suárez-Figueroa.  OOPS! (OntOlogy Pitfall Scanner!): An On-line Tool for Ontology Evaluation.  International Journal on Semantic Web and Information Systems (IJSWIS), 10(2):7–34, 2014.
  * [25] M. Poveda-Villalón, A. Fernández-Izquierdo, M. Fernández-López, and R. García-Castro.  Lot: An industrial oriented ontology engineering framework.  Engineering Applications of Artificial Intelligence, 111:104755, 2022.
  * [26] A. Ramesh, M. Pavlov, G. Goh, S. Gray, C. Voss, A. Radford, M. Chen, and I. Sutskever.  Zero-shot text-to-image generation, 2021.
  * [27] S. M. Rashid, J. McCusker, D. Gruen, O. Seneviratne, and D. L. McGuinness.  A concise ontology to support research on complex, multimodal clinical reasoning.  In The Semantic Web: 20th International Conference, ESWC 2023, Hersonissos, Crete, Greece, May 28–June 1, 2023, Proceedings, page 390–407, Berlin, Heidelberg, 2023. Springer-Verlag.
  * [28] V. Rawte, A. Sheth, and A. Das.  A survey of hallucination in large foundation models, 2023.
  * [29] S. Semeniuta, A. Severyn, and S. Gelly.  On accurate evaluation of gans for language generation, 2019.
  * [30] K. Singhal, T. Tu, J. Gottweis, R. Sayres, E. Wulczyn, L. Hou, K. Clark, S. Pfohl, H. Cole-Lewis, D. Neal, M. Schaekermann, A. Wang, M. Amin, S. Lachgar, P. Mansfield, S. Prakash, B. Green, E. Dominowska, B. A. y Arcas, N. Tomasev, Y. Liu, R. Wong, C. Semturs, S. S. Mahdavi, J. Barral, D. Webster, G. S. Corrado, Y. Matias, S. Azizi, A. Karthikesalingam, and V. Natarajan.  Towards expert-level medical question answering with large language models, 2023.
  * [31] R. Thoppilan, D. D. Freitas, J. Hall, N. Shazeer, A. Kulshreshtha, H.-T. Cheng, A. Jin, T. Bos, L. Baker, Y. Du, Y. Li, H. Lee, H. S. Zheng, A. Ghafouri, M. Menegali, Y. Huang, M. Krikun, D. Lepikhin, J. Qin, D. Chen, Y. Xu, Z. Chen, A. Roberts, M. Bosma, V. Zhao, Y. Zhou, C.-C. Chang, I. Krivokon, W. Rusch, M. Pickett, P. Srinivasan, L. Man, K. Meier-Hellstern, M. R. Morris, T. Doshi, R. D. Santos, T. Duke, J. Soraker, B. Zevenbergen, V. Prabhakaran, M. Diaz, B. Hutchinson, K. Olson, A. Molina, E. Hoffman-John, J. Lee, L. Aroyo, R. Rajakumar, A. Butryna, M. Lamm, V. Kuzmina, J. Fenton, A. Cohen, R. Bernstein, R. Kurzweil, B. Aguera-Arcas, C. Cui, M. Croak, E. Chi, and Q. Le.  Lamda: Language models for dialog applications, 2022.
  * [32] N. Varshney, W. Yao, H. Zhang, J. Chen, and D. Yu.  A stitch in time saves nine: Detecting and mitigating hallucinations of llms by validating low-confidence generation, 2023.
  * [33] H. Ye, T. Liu, A. Zhang, W. Hua, and W. Jia.  Cognitive mirage: A review of hallucinations in large language models, 2023.
  * [34] H. Yuan, Z. Yuan, R. Gan, J. Zhang, Y. Xie, and S. Yu.  Biobart: Pretraining and evaluation of a biomedical generative language model, 2022.
  * [35] X. Zeng, A. Vahdat, F. Williams, Z. Gojcic, O. Litany, S. Fidler, and K. Kreis.  Lion: Latent point diffusion models for 3d shape generation, 2022.
  * [36] M. Zhang, O. Press, W. Merrill, A. Liu, and N. A. Smith.  How language model hallucinations can snowball, 2023.
  * [37] Y. Zhang, Y. Li, L. Cui, D. Cai, L. Liu, T. Fu, X. Huang, E. Zhao, Y. Zhang, Y. Chen, L. Wang, A. T. Luu, W. Bi, F. Shi, and S. Shi.  Siren’s song in the ai ocean: A survey on hallucination in large language models, 2023.