# Probabilistically-autoencoded horseshoe-disentangled multidomain item-
response theory models

Joshua C. Chang
Epidemiology and Biostatistics
Rehabilitation Medicine, CC, NIH
& team med$\varepsilon_{\text{rrata}}$
<EMAIL_ADDRESS>
Shashaank Vattikuti & Carson C. Chow
Laboratory of Biological Modeling, NIDDK, NIH
& team med$\varepsilon_{\text{rrata}}$
{shashaank.vattikuti<EMAIL_ADDRESS>

###### Abstract

Item response theory (IRT) is a non-linear generative probabilistic paradigm
for using exams to identify, quantify, and compare latent traits of
individuals, relative to their peers, within a population of interest. In pre-
existing multidimensional IRT methods, one requires a factorization of the
test items. For this task, linear exploratory factor analysis is used, making
IRT a posthoc model. We propose skipping the initial factor analysis by using
a sparsity-promoting horseshoe prior to perform factorization directly within
the IRT model so that all training occurs in a single self-consistent step.
Being a hierarchical Bayesian model, we adapt the WAIC to the problem of
dimensionality selection. IRT models are analogous to probabilistic
autoencoders. By binding the generative IRT model to a Bayesian neural network
(forming a probabilistic autoencoder), one obtains a scoring algorithm
consistent with the interpretable Bayesian model. In some IRT applications the
black-box nature of a neural network scoring machine is desirable. In this
manuscript, we demonstrate within-IRT factorization and comment on scoring
approaches.

##  1 Introduction

###  1.1 Item response theory (IRT)

IRT [7, 10] is the dominant statistical paradigm for using tests in order to
quantify latent traits for individuals in the context of their populations. In
both form and function, IRT models resemble autoencoders. Like in an
autoencoder, the traits are a nonlinear factor representation of input data,
and the latent representation is a major analytical product of interest. While
additional latent variables exist both in IRT and in neural network-based
autoencoders, of these, only in an IRT model are the latent variables
interpretable. These parameters constitute item calibrations – difficulty and
discrimination parameters – that correspond to the information content of each
test question. In effect, IRT models are able to separate individual effects
from test-specific effects in understanding responses. In contrast, the
weights of a neural network lack mechanistic interpretability.

Model interpretability is a weakness for deep learning because artificial
neural networks are intrinsically black boxes. It is difficult to understand
how neural networks encode relationships between variables in their
multidimensional mappings. In practical terms, this opaqueness has made it
difficult to determine whether neural networks are making good objective
inferences, or are simply codifying biases inherent in their training data.

Unfortunately, these issues are an impediment to using autoencoders in
testing, where testing outcomes have real-life consequences. Disentangled
autoencoders like the $\beta$-variational autoencoder [16] ($\beta$-VAE), and
related methods [17, 3, 20, 6] show promise – they provide improved
interpretability by producing disentangled representations. What is meant by
disentanglement, however, is specific to the method. Disentanglement typically
refers to statistical independence of these latent factors, and not on
dependencies to the input data itself.

Typically, for IRT, one desires disentanglement of original test items through
partitioning, so that each partition informs a different dimension of the
latent trait. The partitioning is performed prior to training of the IRT
model, using exploratory factor analysis. Subdomains of the IRT model are then
fit independently, conditional on the partition, using various empirical
Bayesian [5] or Bayesian methods [31, 28].

Hence, IRT models are dependent on the initial partitioning of items into
domains. The domain partitioning is a linear covariance analysis, where
various criteria are commonly-used for accepting and rejecting items within
each factor domain, without attention to consistency with the nonlinear IRT
model. Additionally, partitional uncertainty is not propagated into the IRT
model.

We propose factorization directly within IRT models using sparse coding.
Additionally, we interpret IRT as a probabilistic decoder, and couple it to a
neural network encoder. In training the encoder, the neural network learns the
inversion of the generative model. Hence, one obtains a scoring machine that
is consistent with the learned posterior of the IRT model. Testing is a rare
application in which black box scoring machines may have value. The
uninterpretable nature of the neural network can help impede gaming of exams
if parameters from the scoring algorithm become partially compromised.

##  2 Modeling

In the parlance of neural networks, we present the overall method under the
framework of a probabilistic autoencoder, depicted in Fig. 1. Item response
theory (IRT) provides a substitute for the less-structured decoder neural
network present in vanilla autoencoders.

###  2.1 Hierarchical Bayesian item response theory (IRT)

Figure 1: Architecture: The encoder (left) computes via an uninterpretable
neural network an interpretable representation of the input data consistent
with the mechanistic Bayesian hierarchical model (right).

In item response theory (IRT), a person’s test responses are modeled using
both personal traits and item-specific parameters. The item parameters relate
to the difficulty of the item and the discrimination of the item, or the
degree to which the question’s responses are determined by personal traits.
The two types of attributes are linked together via item response functions.

We focus on the parametric graded response model (GRM), which is used to model
tests with ordinal responses. According to the GRM, the probability of a
response of $j\in\\{1,2,\ldots,J\\}$ to item $i$ for person $p$ is the
likelihood function

$\displaystyle\Pr(X_{pi}=j|\theta_{p},\tau_{i,j},\tau_{i,j+1},\lambda_{i})$
$\displaystyle=\Pr(X_{pi}\geq
j|\theta_{p},\tau_{ij},\lambda_{i})-\Pr(X_{pi}\geq
j+1|\theta_{p},\tau_{i,j+1},\lambda_{i})$
$\displaystyle=S(\lambda_{i}(\theta_{p}-\tau_{i,j}))-S(\lambda_{i}(\theta_{p}-\tau_{i,j+1})),$
(1)

where $S$ is the sigmoid function

$S(x)=\frac{1}{1+e^{-x}}.$

The item discrimination parameters $\lambda_{i}$ control the scale of the
response functions, or the strengths of the relationships between trait and
response: $\lambda_{i}=0$ implies that a item is non-informative. The item-
specific parameters $\tau_{ij}<\tau_{i,j+1}$ and the personal traits
$\theta_{p}$ set the locations of the response distribution. For notational
simplicity, we set $\tau_{i,0}=-\infty$ and $\tau_{i,J}=\infty$.

Extending the GRM to multidimensional traits, we consider a mixture model
where each component of the trait is informed by a weighted combination of the
original items. Our objective in setting priors for the training problem is to
simultaneously regularize an IRT model and factorize the original test items.
To this end, we encourage sparsity in the mappings between the latent
representation and the original items, by using hierarchical shrinkage on the
item-domain discrimination parameters.

To encourage item-to-domain sparsity, we set horseshoe priors [24] on the
discrimination parameters, yielding the hierarchical model equations

$\begin{split}&{\textbf{Likelihood}}\\\
\@ADDCLASS{ltx_eqn_lefteqn}$\displaystyle\Pr(X_{pi}=j|\\{\theta_{p}^{(d)}\\}_{d},\\{\\{\tau_{ij}^{(d)}\\}_{j}\\}_{d},\\{\lambda_{i}^{(d)}\\}_{d})$\mbox{}\hfil\\\
&\quad=\sum_{d=1}^{D}w_{id}\Pr(X_{pi}=j|\theta_{p}^{(d)},\tau_{i,j}^{(d)},\tau_{i,j+1}^{(d)},\lambda_{i}^{(d)})\\\
&w_{id}={\left(\lambda_{i}^{(d)}\right)^{\nu}}/{\sum_{d=0}^{D-1}\left(\lambda_{i}^{(d)}\right)^{\nu}}\\\
&\vec{\theta}_{p}|\mathcal{W}=\texttt{encoder}(\vec{X}_{p};\mathcal{W})\\\ \\\
\end{split}\qquad\begin{split}\textbf{Prior}\\\
\lambda_{i}^{(d)}|\eta_{i},\xi_{i}^{(d)},\kappa^{(d)}&\sim\mathcal{N}^{+}\left(0,\eta_{i}\xi_{i}^{(d)}\kappa^{(d)}\right)\\\
\xi_{i}^{(d)}&\sim C^{+}(0,\xi_{0})\\\ \eta_{i}&\sim C^{+}(0,\eta_{0})\\\
\kappa^{(d)}&\sim C^{+}(0,\kappa^{(d)}_{0})\\\
\tau_{i,0}^{(d)}&\sim\mathcal{N}(\mu_{i}^{(d)},1)\\\
\mu_{i}^{(d)}&\sim\mathcal{N}(0,1)\\\
\tau_{i,j}^{(d)}|\tau_{i,j-1}^{(d)}&\sim\mathcal{N}^{+}(\tau_{i,j-1}^{(d)},1)\\\
\theta_{p},\mathcal{W}&\sim\mathcal{N}(\theta_{p};0,1)p(\mathcal{W},\ldots),\end{split}$
(2)

defined for $i\in 1,\ldots,I\quad d\in 1,\ldots,D,\quad p\in 1,\ldots,P$, and
$\mathcal{W}$ represents neural-network parameters.

The half-Cauchy hyperparameters determining the variance of the discrimination
parameters constitute a variant of the horseshoe prior. In this context, the
horseshoe prior factors items into domains by encouraging sparsity in the
discrimination parameters. We also use horseshoe priors in regularizing the
probabilistic encoder. When applied to weights of neural networks, the
horseshoe prior has been shown to improve out of sample performance [14, 13],
as it is also known to do in the general setting of Bayesian hierarchical
models [25, 26].

###  2.2 Probabilistic encoder

Referring to Fig. 1, we use the graded response model as a probabilistic
decoder. Note that the decoder model is itself sufficient for calibrating and
implementing item response theory. However, by coupling a neural network
encoder to the decoder, we obtain a test scoring algorithm for a new subject’s
responses

$\texttt{encoder}:\begin{subarray}{c}\text{new}\\\
\text{responses}\end{subarray}\to\mathbb{E}_{\begin{subarray}{c}\text{posterior}\\\
\text{parameters}\end{subarray}}\Pr\left(\mathrm{traits}{\bigg{|}}\
\begin{subarray}{c}\text{training}\\\ \text{responses}\end{subarray},\
\begin{subarray}{c}\text{calibrated}\\\ \text{model}\end{subarray}\right),$
(3)

consistent with the Bayesian inference performed in training the IRT model.
Note that Eq. 3 is from the viewpoint that the encoder represents probability
densities. There are many ways to accomplish this representation. One way of
doing so is to use weight-uncertain Bayesian neural networks [4], where the
encoder is an ensemble of neural networks, in a manner true to Fig. 1. Other
computationally-efficient approximate methods also exist such as dropout [30,
11], or simply using arbitrary neural network architectures to express either
parametric [18], semiparametric, or non-parametric probability distributions
[19, 21].

Looking at implementation details, there are some further similarities between
the GRM and neural networks. The GRM (Eq. 1) has scaling degeneracies that are
absolved by our hierarchical priors. Similarly, neural networks are also
scale-degenerate, in the absence of scale-enforcing priors or regularization.
Additionally, there is a permutation degeneracy in the GRM, that compromises
identifiability, that we ameliorate by setting
$\kappa_{0}^{(d+1)}<\kappa_{0}^{(d)}$. The also exists the same degeneracy in
DNNs: each dense layer of $n$ nodes has $n!$ computationally-identical
configurations, suggesting a similar remedy.

###  2.3 Training

This model makes use of the horseshoe prior, for which the Cauchy distribution
plays a starring role. The heavy peak and fat tails of Cauchy priors give it
excellent shrinkage properties, however, they make it difficult to sample from
directly. In lieu of direct sampling, we use the following auxiliary
parameterization [22],

$\begin{split}x\sim\mathcal{C}^{+}(0,\sigma)\end{split}\qquad\Longleftrightarrow\qquad\begin{split}x^{2}\sim\text{Inverse-
Gamma}\left(\frac{1}{2},\ \frac{1}{\lambda}\right)\\\
\lambda\sim\text{Inverse-Gamma}\left(\frac{1}{2},\
\frac{1}{\sigma^{2}}\right).\end{split}$ (4)

Directly training the conjoined model of Fig. 1 is challenging. Our strategy
has been to proceed in blocks, by initially training the IRT model on its own,
and then training the neural network separately, conditional on the IRT model.
For both sets of training, we used mean-field ADVI with inverse-Gamma
distributions parameterizing the scale parameters.

###  2.4 Model evaluation

To decide on a metric for model comparison, we consider the real-world use-
case of IRT models. In practice, IRT models are first calibrated to a training
set of responses from which both person- and item-specific model parameters
are learned. Then, conditional on these pre-trained item-specific parameters,
new respondents are scored by learning their person-specific parameters
(traits). Hence, the scoring procedure finds the traits of new respondents
relative to those of the calibration sample.

Implicitly, the assumption is that the calibrated model is predictive of how
new respondents interact with the test. Hence, the out-of-sample predictive
power of any calibrated IRT model is important. The general way to assess this
power is to use held-out data, through k-fold cross validation, which requires
multiple model fits. To save from having to pay the added computational cost
of refitting the model, we adapt an approximation of Bayesian leave-one-out
cross validation (LOO) known as the Widely Applicable Information Criterion
(WAIC) [29, 12, 23, 27].

The WAIC is a simple-to-compute asymptotic approximation of LOO under minimal
regularity conditions. In particular, the WAIC does not assume that the
mapping between data and model parameters is bijective. The WAIC consists of
the balance between two simple terms which often have exact algebraic
expressions [8]. The first term is the log pointwise predictive density
(lppd), which is the “pointwise” sum of the logarithm of the marginal
likelihoods for the training data, or

$\text{lppd}=\sum_{p}\log\left\\{\mathbb{E}_{\begin{subarray}{c}\text{param}\\\
\text{posterior}\end{subarray}}\left[\Pr\left(\mathbf{X}_{p}\Big{|}\text{params}\right)\right]\right\\}\approx\sum_{p}\log\left\\{\frac{1}{S}\sum_{s=0}^{S-1}\left[\Pr\left(\mathbf{X}_{p}\Big{|}\,\begin{subarray}{c}\text{param}\\\
\text{sample }s\end{subarray}\right)\right\\}\right],$ (5)

where we treat each person $p$’s collection of responses $\mathbf{X}_{p}$ as a
statistically interchangeable point and assume that the joint posterior
distribution of the calibration parameters is represented by a sample of size
$S$. The second term in the WAIC is a penalty term of which there are two
commonly used variants. The second variant has lower variance than the first
[12], so we adapt it to our purposes in computing

$\text{pwaic}=\sum_{p}\text{Var}_{\begin{subarray}{c}\text{param}\\\
\text{posterior}\end{subarray}}\left[\log\Pr\left(\mathbf{X}_{p}\Big{|}\,\begin{subarray}{c}\text{param}\\\
\text{sample }s\end{subarray}\right)\right].$ (6)

Eqs 5 and 6 combine together to constitute the WAIC,

$\text{WAIC}=-2\times(\text{lppd}-\text{pwaic}),$ (7)

which is on the same deviance scale as used in the AIC.

The WAIC, being a model statistic, is a stochastic quantity that one estimates
conditional on the on-hand data. Hence, it should not be treated as certain
and one must also take its estimation error into account. To this end, one may
approximate the standard error of the WAIC using the expression

$\displaystyle\widehat{\text{se}}\left(\text{WAIC}\right)\approx$
$\displaystyle\quad\sqrt{P\times\text{Var}_{\text{people}}\left[\log\left\\{\mathbb{E}_{\begin{subarray}{c}\text{param}\\\
\text{posterior}\end{subarray}}\left[\Pr\left(\mathbf{X}_{p}\Big{|}\text{params}\right)\right]\right\\}-\text{Var}_{\begin{subarray}{c}\text{param}\\\
\text{posterior}\end{subarray}}\left[\log\Pr\left(\mathbf{X}_{p}\Big{|}\,\begin{subarray}{c}\text{param}\\\
\text{sample }s\end{subarray}\right)\right]\right]}.$ (8)

##  3 Experiments

Figure 2: Two-dimensional within-IRT factorization vs linear factorization of
Right Wing Authoritarianism (RWA) scale test items, colored by discrimination
weights $w_{i}^{(d)}$. Shown are posterior expectations for the weights as
estimated by mean-field ADVI. For comparison, linear factor loadings, colored
by absolute value, are contrasted on the right hand side. Figure 3: Three-
dimensional within-IRT factorization vs linear factorization of Right Wing
Authoritarianism (RWA) scale test items, colored by discrimination weights
$w_{i}^{(d)}$. Shown are posterior expectations for the weights as estimated
by mean-field ADVI. For comparison, linear factor loadings, colored by
absolute value, are contrasted on the right hand side.

As a testing ground for the overall method, we used the Right Wing
Authoritarianism (RWA) scale [2], for which there exists an openly available
dataset [1] representing $9881$ college-aged respondents. The scale consists
of twenty two questions on cultural and societal beliefs, with responses
reported on a 9-level Likert scale.

Fig. 2 demonstrates factorization of the scale into two dimensions, learned
using mean field ADVI, with inverse-gamma distributions to approximate all
scale parameters. In these examples, we set the following scale parameters:
$\eta_{0}=0.01$, $\xi_{0}=0.01$, $\kappa^{(d)}_{0}=0.01\times(0.1)^{d}$,
$\nu=1$. For comparison, the linear factor loadings are also given.

Fig. 3 demonstrates factorization of the scale into three dimensions, also
learned using mean field ADVI, with inverse-gamma distributions to approximate
all scale parameters. Also in these examples, we set the following scale
parameters: $\eta_{0}=0.01$, $\xi_{0}=0.01$,
$\kappa^{(d)}_{0}=0.01\times(0.1)^{d}$, $\nu=1$.

In both Figs. 2 and 3, we initialized the ADVI algorithm so that the mean of
the discrimination parameters is one plus the corresponding factor loading. We
found this choice to lead to stable training using the Adam optimizer with an
initial step size ranging between $10^{-3}$ and $10^{-4}$, and with an
exponentially decaying learning rate.

The WAIC statistics for the two and three-dimensional models were
$\text{WAIC}_{D=2}\approx 5.67\pm 0.025\times 10^{5}$ and
$\text{WAIC}_{D=3}\approx 5.66\pm 0.025\times 10^{5}$ respectively, mildly
supporting the adaptation of a three-dimensional factorization on purely
predictive grounds.

We performed all experiments using TensorFlow probability [9]. The source code
is available upon request. The python package factor-analyzer provided the
algorithm for our exploratory factor analysis results that we contrasted
against the within-IRT factorizations.

##  4 Discussion

In this manuscript we have defined a Bayesian hierarchical model for
performing factorization of test items in item response theory, making an
analogy to probabilistic autoencoders. We demonstrated the method on the Right
Wing Authoritarianism scale and used Bayesian predictive model comparison to
contrast two and three-dimensional factorizations on the basis of prediction.

###  4.1 Within-IRT versus linear factorization

In practice, people use factor analysis as a first step in item response
theory, in order to segregate items into separate domains for fitting IRT.
Factor analysis results in a non-sparse matrix of loadings. To use factor
analysis for this purpose, one must adapt an ultimately arbitrary cut-off
criteria. A common criterion used in the literature is to set the cutoff at an
absolute value of $0.4$, though other advised cutoffs dependent on sample size
exist [15].

For the RWA factorization, an absolute cut-off of $|F|<0.4$ in the factor
analysis leads to a different factorization from that obtained within-IRT,
though with some similarities. Looking at the two dimensional factorization of
Fig. 2, the within-IRT factorization assigns fewer overall items into the
first dimension relative the linear factor analysis. Looking at the item
separation within-IRT, there are clear differences between the two overall
factor dimensions. The first dimension tends to pick up on items that describe
attitudes towards openness to liberal cultural attitudes. The second dimension
tends to pick up on items describing adherence to traditional values and
deference to authorities.

The two methods clearly disagreed in their categorization of two items: Women
should have to promise to obey their husbands when they get married, and God’s
laws about abortion, pornography and marriage must be strictly followed before
it is too late, and those who break them must be strongly punished. Factor
analysis placed both of these items into the first factor, though with
negative loading. Within-IRT factorization placed these two items into the
second factor instead.

Extending the factorizations to three dimensions (Fig. 3), one sees that for
the within-IRT factorization, some of the items that weighted nearly
completely into the second dimension leak partially into the third dimension,
while retaining the bulk of their weight in the second dimension. In contrast,
for linear factorization, two of the items switch completely from the second
factor to the third factor. This behavior suggests that the IRT factorization
is inherently more-stable than linear factor analysis, demonstrating the
partial pooling feature of Bayesian hierarchical modeling.

###  4.2 Selecting between two and three dimensions

Based purely on predictive accuracy, the three-dimensional model achieves a
slight edge over the two-dimensional one, though the difference between the
WAIC statistics for these models was within one standard error.

For practical purposes, the two models have very similar predictive
capabilities, as estimated using the WAIC. Hence, other considerations must be
used for deciding between the models. These considerations are highly context-
dependent, and should be influenced by how one wishes to interpret or use the
testing instrument.

The interpretability of the two-dimensional model is fairly clear, given the
content of the items within the resulting factors. The two- and three-
dimensional models have similar factorizations, except for some cases where
the second dimension leaks some weight into the third dimension. The
meaningfulness of the third dimension requires subject-matter knowledge to
adjudicate.

###  4.3 Scoring

When training a Bayesian model using mean-field variational methods, one is
breaking dependencies between the variables of interest. For this reason, the
sampling distribution of the surrogate posterior distribution does not
necessarily yield the correct prediction distribution, unless either the
posterior parameter densities are heavily concentrated around an unimodal
peak, or are factorizable.

The entire purpose of IRT is to obtain statistics of the prediction
distribution for latent traits conditional on input responses. Mindful of this
purpose, it is improper to directly use the surrogate posterior parameter
distributions to perform scoring – this holds true regardless of whether one
wishes to score using the neural network, or by marginalizing over the
statistics of the IRT model. In either case, one must use either more-
structured variational approximations, or direct Monte-Carlo samplings from
the joint posterior density, to preserve at least some of the inter-variable
associations.

Using direct Monte-Carlo simulations of the parameters is conceptually
straightforward, yet computationally intensive, both at the time of training
and at the time of prediction. One can reduce the computation needed at
training by performing MCMC only after ADVI, but then still be left with the
computation of propagating the parameter samples at the time of prediction.

Since Bayesian computation is approximate by nature, it is perhaps fruitful to
forgo Bayesian neural networks and seek deterministic neural networks that
directly parameterize Gaussian or other low-order approximations of the
prediction distributions. Doing so would also come with the added benefit of
making scoring incredibly fast. Direct Bayesian marginalization of the IRT
model requires computation of a multidimensional integral either directly or
through Monte Carlo methods. For this reason, the neural network architecture
would be advantageous. Future work will focus on finding efficient
architectures for this task.

##  5 Conclusion

We have demonstrated that one may perform multidimensional within-IRT
factorization and model calibration in a single step, using sparse coding. By
acknowledging the analogy between IRT and probabilistic autoencoders, one also
obtains a versatile tool for inverting IRT model products to score new test
respondents. On the flip side, regularization used for Bayesian
multidimensional IRT shows promise for taming Bayesian neural networks. Future
work will concentrate on translating the pooling properties of Bayesian
hierarchical models to neural networks.

## Acknowledgements

This work is supported by the Intramural Research Programs of the National
Institutes of Health Clinical Center (CC) and the National Institute of
Diabetes and Digestive and Kidney Diseases (NIDDK), and the US Social Security
Administration.

## References

  * [1] Take a personality test - Open Source Psychometrics Project.  URL https://openpsychometrics.org/.
  * Altemeyer [1988] Bob Altemeyer.  _Enemies of freedom: Understanding right-wing authoritarianism_.  Enemies of freedom: Understanding right-wing authoritarianism. Jossey-Bass, San Francisco, CA, US, 1988.  ISBN 978-1-55542-097-0.
  * Antoran and Miguel [2019] Javier Antoran and Antonio Miguel.  Disentangling in Variational Autoencoders with Natural Clustering.  _arXiv:1901.09415 [cs, stat]_ , January 2019.  URL http://arxiv.org/abs/1901.09415.  arXiv: 1901.09415.
  * Blundell et al. [2015] Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra.  Weight Uncertainty in Neural Networks.  May 2015.  URL https://arxiv.org/abs/1505.05424v2.
  * Bock et al. [1997] R. Darrell Bock, David Thissen, and Michele F. Zimowski.  IRT Estimation of Domain Scores.  _Journal of Educational Measurement_ , 34(3):197–211, 1997.  ISSN 1745-3984.  doi: 10.1111/j.1745-3984.1997.tb00515.x.  URL https://www.onlinelibrary.wiley.com/doi/abs/10.1111/j.1745-3984.1997.tb00515.x.
  * Burgess et al. [2018] Christopher P. Burgess, Irina Higgins, Arka Pal, Loic Matthey, Nick Watters, Guillaume Desjardins, and Alexander Lerchner.  Understanding disentangling in $\beta$-VAE.  _arXiv:1804.03599 [cs, stat]_ , April 2018.  URL http://arxiv.org/abs/1804.03599.  arXiv: 1804.03599.
  * Chang and Reeve [2005] Chih-Hung Chang and Bryce B. Reeve.  Item Response Theory and its Applications to Patient-Reported Outcomes Measurement.  _Evaluation & the Health Professions_, 28(3):264–282, September 2005.  ISSN 0163-2787.  doi: 10.1177/0163278705278275.  URL https://doi.org/10.1177/0163278705278275.
  * [8] Joshua C. Chang.  Predictive Bayesian selection of multistep Markov chains, applied to the detection of the hot hand and other statistical dependencies in free throws.  _Royal Society Open Science_ , 6(3):182174.  doi: 10.1098/rsos.182174.  URL https://royalsocietypublishing.org/doi/full/10.1098/rsos.182174.
  * Dillon et al. [2017] Joshua V. Dillon, Ian Langmore, Dustin Tran, Eugene Brevdo, Srinivas Vasudevan, Dave Moore, Brian Patton, Alex Alemi, Matt Hoffman, and Rif A. Saurous.  TensorFlow Distributions.  _arXiv:1711.10604 [cs, stat]_ , November 2017.  URL http://arxiv.org/abs/1711.10604.  arXiv: 1711.10604.
  * Fries et al. [2014] James F. Fries, James Witter, Matthias Rose, David Cella, Dinesh Khanna, and Esi Morgan-DeWitt.  Item Response Theory, Computerized Adaptive Testing, and PROMIS: Assessment of Physical Function.  _The Journal of Rheumatology_ , 41(1):153–158, January 2014.  ISSN 0315-162X, 1499-2752.  doi: 10.3899/jrheum.130813.  URL http://www.jrheum.org/content/41/1/153.
  * Gal and Ghahramani [2015] Yarin Gal and Zoubin Ghahramani.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning.  June 2015.  URL https://arxiv.org/abs/1506.02142v6.
  * Gelman et al. [2014] Andrew Gelman, Jessica Hwang, and Aki Vehtari.  Understanding predictive information criteria for Bayesian models.  _Statistics and Computing_ , 24(6):997–1016, November 2014.  ISSN 1573-1375.  doi: 10.1007/s11222-013-9416-2.  URL https://doi.org/10.1007/s11222-013-9416-2.
  * Ghosh and Doshi-Velez [2017] Soumya Ghosh and Finale Doshi-Velez.  Model Selection in Bayesian Neural Networks via Horseshoe Priors.  May 2017.  URL https://arxiv.org/abs/1705.10388v1.
  * Ghosh et al. [2018] Soumya Ghosh, Jiayu Yao, and Finale Doshi-Velez.  Structured Variational Learning of Bayesian Neural Networks with Horseshoe Priors.  _arXiv:1806.05975 [cs, stat]_ , June 2018.  URL http://arxiv.org/abs/1806.05975.  arXiv: 1806.05975.
  * Hair et al. [1998] J. F. Hair, R. E. Anderson, R. L. Tatham, and W. C. Black.  _Multivariate Data Analysis, 5th international ed_.  Prentice Hall, Upper Saddle River, 1998.
  * Higgins et al. [2016] Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner.  beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework.  November 2016.  URL https://openreview.net/forum?id=Sy2fzU9gl.
  * Kim and Mnih [2018] Hyunjik Kim and Andriy Mnih.  Disentangling by Factorising.  In _International Conference on Machine Learning_ , pages 2649–2658, July 2018.  URL http://proceedings.mlr.press/v80/kim18b.html.
  * Kingma and Welling [2013] Diederik P. Kingma and Max Welling.  Auto-Encoding Variational Bayes.  _arXiv:1312.6114 [cs, stat]_ , December 2013.  URL http://arxiv.org/abs/1312.6114.  arXiv: 1312.6114.
  * Kobyz and Zamyatin [2015] G.V. Kobyz and A.V. Zamyatin.  Conditional probability density estimation using artificial neural network.  In _2015 9th International Conference on Application of Information and Communication Technologies (AICT)_ , pages 441–445, October 2015.  doi: 10.1109/ICAICT.2015.7338597.  ISSN: null.
  * Li and Mandt [2018] Yingzhen Li and Stephan Mandt.  Disentangled Sequential Autoencoder.  _arXiv:1803.02991 [cs]_ , March 2018.  URL http://arxiv.org/abs/1803.02991.  arXiv: 1803.02991.
  * Likas [2001] Aristidis Likas.  Probability density estimation using artificial neural networks.  _Computer Physics Communications_ , 135(2):167–175, April 2001.  ISSN 0010-4655.  doi: 10.1016/S0010-4655(00)00235-6.  URL http://www.sciencedirect.com/science/article/pii/S0010465500002356.
  * Makalic and Schmidt [2016] Enes Makalic and Daniel F. Schmidt.  A simple sampler for the horseshoe estimator.  _IEEE Signal Processing Letters_ , 23(1):179–182, January 2016.  ISSN 1070-9908, 1558-2361.  doi: 10.1109/LSP.2015.2503725.  URL http://arxiv.org/abs/1508.03884.  arXiv: 1508.03884.
  * Piironen and Vehtari [2017a] Juho Piironen and Aki Vehtari.  Comparison of Bayesian predictive methods for model selection.  _Statistics and Computing_ , 27(3):711–735, May 2017a.  ISSN 1573-1375.  doi: 10.1007/s11222-016-9649-y.  URL https://doi.org/10.1007/s11222-016-9649-y.
  * Piironen and Vehtari [2017b] Juho Piironen and Aki Vehtari.  Sparsity information and regularization in the horseshoe and other shrinkage priors.  _Electronic Journal of Statistics_ , 11(2):5018–5051, 2017b.  ISSN 1935-7524.  doi: 10.1214/17-EJS1337SI.  URL http://arxiv.org/abs/1707.01694.  arXiv: 1707.01694.
  * Polson and Scott [2011a] Nicholas G. Polson and James G. Scott.  On the half-Cauchy prior for a global scale parameter.  _arXiv:1104.4937 [stat]_ , April 2011a.  URL http://arxiv.org/abs/1104.4937.  arXiv: 1104.4937.
  * Polson and Scott [2011b] Nicholas G. Polson and James G. Scott.  _Shrink Globally, Act Locally: Sparse Bayesian Regularization and Prediction *_.  Oxford University Press, October 2011b.  ISBN 978-0-19-173192-1.  URL https://www.oxfordscholarship.com/view/10.1093/acprof:oso/9780199694587.001.0001/acprof-9780199694587-chapter-17.
  * Vehtari et al. [2017] Aki Vehtari, Andrew Gelman, and Jonah Gabry.  Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC.  _Statistics and Computing_ , 27(5):1413–1432, September 2017.  ISSN 1573-1375.  doi: 10.1007/s11222-016-9696-4.  URL https://doi.org/10.1007/s11222-016-9696-4.
  * Vispoel et al. [2018] Walter P. Vispoel, Carrie A. Morris, and Murat Kilinc.  Applications of generalizability theory and their relations to classical test theory and structural equation modeling.  _Psychological Methods_ , 23(1):1–26, 2018.  ISSN 1939-1463(Electronic),1082-989X(Print).  doi: 10.1037/met0000107.
  * Watanabe [2010] Sumio Watanabe.  Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory.  _Journal of Machine Learning Research_ , 11(Dec):3571–3594, 2010.  ISSN ISSN 1533-7928.  URL http://www.jmlr.org/papers/v11/watanabe10a.html.
  * Wen et al. [2018] Yeming Wen, Paul Vicol, Jimmy Ba, Dustin Tran, and Roger Grosse.  Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches.  _arXiv:1803.04386 [cs, stat]_ , March 2018.  URL http://arxiv.org/abs/1803.04386.  arXiv: 1803.04386.
  * Weng and Coad [2018] Ruby Chiu-Hsing Weng and D. Stephen Coad.  Real-Time Bayesian Parameter Estimation for Item Response Models.  _Bayesian Analysis_ , 13(1):115–137, March 2018\.  ISSN 1936-0975, 1931-6690.  doi: 10.1214/16-BA1043.  URL https://projecteuclid.org/euclid.ba/1482138050.


11institutetext: Information Sciences Institute, University of Southern
California
11email<EMAIL_ADDRESS>
22institutetext: Ontology Engineering Group, Universidad Politécnica de Madrid
22email<EMAIL_ADDRESS>
# Best Practices for Implementing FAIR Vocabularies and Ontologies on the Web

Daniel Garijo 11 0000-0003-0454-7145
María Poveda-Villalón 22 0000-0003-3587-0367

###### Abstract

With the adoption of Semantic Web technologies, an increasing number of
vocabularies and ontologies have been developed in different domains, ranging
from Biology to Agronomy or Geosciences. However, many of these ontologies are
still difficult to find, access and understand by researchers due to a lack of
documentation, URI resolving issues, versioning problems, etc. In this chapter
we describe guidelines and best practices for creating accessible,
understandable and reusable ontologies on the Web, using standard practices
and pointing to existing tools and frameworks developed by the Semantic Web
community. We illustrate our guidelines with concrete examples, in order to
help researchers implement these practices in their future vocabularies.

###### Keywords:

Ontology metadata Ontology publication Ontology access FAIR principles Linked
Data principles.

##  1 Introduction

In the last decade, a series of initiatives for open data, transparency and
open science have led to the development of a myriad of datasets and linked
Knowledge Graphs on the Web.111https://lod-cloud.net/ Ontologies and
vocabularies have been developed accordingly to represent the contents of
these datasets and Knowledge Graphs and help in their integration and linking.
However, while significant effort has been spent on making data Findable,
Accessible, Interoperable and Reusable (FAIR) [18], ontologies and
vocabularies are often difficult to access, understand and reuse. This may be
due to several reasons, including a lack of definitions of ontology classes
and properties; deprecated or unavailable imported ontologies, non-resolvable
ontology URIs, lack of examples and diagrams in the documentation, or having
scientific publications describing an ontology without any reference to its
implementation.

The scientific community has started to acknowledge the need for ontologies to
be properly documented, versioned, published and maintained following the
Linked Data Principles [6] and adapting the FAIR principles for data [8]. But
these recommendations do not include guidelines on how to implement them for a
target vocabulary. In this chapter we address this issue by describing how to
make an ontology or vocabulary comply with the FAIR principles, including
examples summarizing best practices from the community and our own experience;
and pointing to popular existing tools and frameworks.

Our guidelines are aimed at ontology engineers, and therefore the paper is
structured according to their ontology development processes: Section 2
describes several design decisions for an ontology URI (naming conventions,
versioning, permanent URIs); Section 3 describes how to create a documentation
that is easy to reuse and understand by others (minimum metadata, diagrams to
include); Section 4 illustrates how to make an ontology accessible and
findable in the Web; Section 5 points to existing end-to-end frameworks that
support the ontology publication process; and Section 6 concludes our paper.
We consider the design and development of an ontology out of the scope of this
paper, as it has been covered by methodologies for ontology development (e.g.,
LOT222https://lot.linkeddata.es/ or NeOn [14]).

##  2 Accessible Ontology URI Design

Ontologies are digital artifacts, and therefore their URIs should follow the
Linked Data Principles,333https://www.w3.org/DesignIssues/LinkedData.html and
use a URI namespace under the developer’s control. The rationale is simple:
only in a domain under our control we will be able to serve the right
serialization of the ontology.

When creating an ontology, it is also important to think carefully about its
name, prefix and URI design. Well engineered ontologies are costly to produce,
and therefore they should be accessible to other potential users (e.g., by
avoiding complex URIs) and easy to differentiate from existing vocabularies.

In this section we describe our proposed best practices for designing URIs for
ontologies to make them unique, easy to remember, and easy to maintain. We
acknowledge that naming conventions may be subjective, but these practices
reflect our experience in ontology development over a wide range of domains
(smart cities, open science, meteorology, neuroscience, etc.) and therefore
provide a good starting point. We divide our guidelines over four main key
points for accessible ontology URI design: how to select a name and prefix
(Section 2.1), whether to use hash or slash namespaces (Section 2.2), whether
to use opaque URIs (Section 2.3), how to incorporate semantic versioning in
your ontology URI (Section 2.4) and how to make your ontology URI permanent
(Section 2.5).

For illustrating purposes, we will be using an example ontology throughout
this paper, with the following URI: https://w3id.org/example#.

###  2.1 Ontology name and prefix

The name and prefix of an ontology are in most cases related to its
application domain. Short and simple names will help others remember your
ontology easily. An extended practice is to use acronyms to abbreviate the
title of longer ontologies, as in “The Data Catalog
Ontology”444https://www.w3.org/TR/vocab-dcat-2/ (with prefix dcat) or the
“Friend of a Friend Ontology”555http://xmlns.com/foaf/spec/ (with prefix
foaf).

Another aspect to consider when designing the name of your ontology is to
avoid overlapping with other existing vocabularies. While it is possible to
overload existing prefixes by assigning them a different URI, this often
confuses potential re-users that are already familiar with the state of the
art. A good strategy to prevent this problem is to look for existing prefixes
in common vocabulary registries such as prefix.cc,666http://prefix.cc Linked
Open Vocabularies (LOV) [15] or Bioportal [17]. In our example ontology, by
doing a quick search LOV and Bioportal, we can see that our example ontology
URI has not been defined. However, the prefix “example” has already been
registered to refer to “example.org”, a namespace defined to create sample
URIs. Therefore we decide on exo (derived from _example ontology_) as our
namespace prefix.

###  2.2 Hash versus slash URIs

When designing the URI of an ontology, it is important to determine whether
the trailing element will be a hash (“#”) or a slash (“/”). On the one hand,
using a hash makes serving the ontology
easier,777https://www.w3.org/wiki/HashVsSlash as the client looking at the URI
strips only the part of the URI before the hash symbol. Everything after the
hash symbol is interpreted as a _fragment identifier_ , and may be ignored or
used by browsers to refer to the right section of the HTML documentation (if
available). W3C standards usually follow the hash convention, and we will
follow it in our example as well.

On the other hand, using a slash allows treating each element of the ontology
as a separate entity that may be described in an independent manner. This can
be useful for organizational purposes when the ontology is big (thousands of
classes) and returning a full serialization or rendering a single HTML
documentation may be deemed too slow, showing instead each class and property
separately. The Open Biomedical Ontology network888http://www.obofoundry.org/
is an example of this convention.

###  2.3 Opaque URIs for classes and properties

Opaque URIs obfuscate the name of classes and properties that are part of your
ontology. For instance, let us assume we have an “ExampleClassA” in our
example ontology. In order to use opaque URIs, instead of using
“ExampleClassA” we would associate the class with an identifier such as
“EXO_C0001”, and use it as part of its URI. This has two main advantages:
First, if we decide to change the name of the class in the future, the URI of
the class would not be affected by it. Second, identifiers are language
agnostic. For instance, someone using another alphabet (e.g., chinese,
cyrillic, etc.) would be able to refer to the same URI with the corresponding
label. Examples of ontologies that follow this convention can be found in the
Open Biomedical Ontologies, but it is also followed by commonly used knowledge
graphs such as Wikidata [16].

The main drawback of using opaque URIs is the difficulty of interpreting
properly classes and properties, which usually requires additional tooling for
displaying the right labels. For this reason, we will not use them in our
example.

###  2.4 Ontology versioning

Ontologies often have multiple versions, and these should be appropriately
annotated as part of the ontology metadata (e.g., with the property
_owl:versionIRI_ and _owl:versionInfo_).We recommend using _semantic
versioning_ 999https://semver.org/ as a guideline for identifying the
different versions of an ontology, as it has become a common practice in
software engineering. In semantic versioning, each version id should follow
the format X.Y.Z, where X represents a major version (e.g., defining a set of
classes and properties to support new use cases), Y represents a minor version
(e.g., adding a single property or class), and Z represents patches or quick
bug fixes (updating a label, adding examples, etc.). In our example ontology,
the first version is 1.0.0 (first major release), with the IRI
“https://w3id.org/example/1.0.0”, which we would represent in the ontology as
follows:

<https://w3id.org/example> rdf:type owl:Ontology ; owl:versionIRI
<https://w3id.org/example/1.0.0> ; owl:versionInfo "1.0.0"@en .

As shown in the example, the version IRI is independent from the URI of the
ontology (“https://w3id.org/example#”). It is discouraged to include version
numbers as part of the ontology URI, as it would deeply affect
interoperability of its instances. For example, consider we had used
“https://w3id.org/example/1.0.0#” as the ontology URI, and we had populated a
Knowledge Graph with two instances of “ExampleClassA”:

@prefix exo: <https://w3id.org/example/1.0.0#> . @prefix ex-inst:
<https://example.org/instance/> . ex-inst:instance1 a exo:ExampleClassA . ex-
inst:instance2 a exo:ExampleClassA .

If we now released another version of the ontology (1.0.1), all the URIs of
ExampleClassA would change (highlighted in blue below):

@prefix exo2: <https://w3id.org/example/1.0.1#> . @prefix ex-inst:
<https://example.org/instance/> . ex-inst:instance1 a exo2:ExampleClassA . ex-
inst:instance2 a exo2:ExampleClassA .

This is an undesired behavior, because it makes ex-inst:instance1 and ex-
inst:instance2 instances of two different classes (exo:ExampleClassA and
exo2:exampleClassA) instead of a single one. Instead, we want all the
instances of a class to be compatible across different ontology versions:

@prefix exo: <https://w3id.org/example#> . @prefix ex-inst:
<https://example.org/instance/> . ex-inst:instance1 a exo:ExampleClassA . ex-
inst:instance2 a exo:ExampleClassA .

By following this convention, we can continue doing ontology releases with
appropriate versioning, while keeping the classes and properties consistent in
terms of their URI.

###  2.5 Using permanent URIs

When publishing an ontology on the Web, it is recommended to think about its
long term sustainability, specifically if it gets widely adopted. For example,
what will happen to the domain used for the namespace URI of an ontology after
several years? (i.e., when the funding for the related research project is
over). Likewise, if the ontology is hosted on a server in a university or
company, what will happen if the server name changes; or if the person in
charge of maintaing it needs to move the ontology hosting to another
institution?

Permanent URIs services are community driven initiatives designed to address
these issues. The idea behind them is simple: instead of minting a URI for a
resource, users may use these services to create a _proxy_ URI which can then
be redirected to wherever the target resource is stored at any point in time.
That way, if the target resource is moved, developers just have to update its
location without changing its permanent URL. There are several open, free
services to create permanent URLs on the Web, among which
purl.org101010https://archive.org/services/purl/ (now hosted by the Internet
Archive) and w3id111111https://w3id.org/ (created by the W3C Permanent
Identifier Community Group and supported by several companies) are perhaps the
most commonly used. We recommend using permanent URIs in ontologies in order
to support their long term sustainability. In fact, our example ontology uses
a w3id: https://w3id.org/example. Creating a w3id is as simple as forking a
GitHub repository and following the instructions in the readme
file.121212https://github.com/perma-id/w3id.org An advantage of w3id versus
_purl.org_ is that you have control on how to redirect the ontology to its
different serializations (a full example is available in Section 4).

##  3 Generating Reusable Ontology Documentation

We refer to “ontology documentation” as the collection of documents and
explanatory comments generated during the entire ontology building process
[14]. These documents are critical for helping others understand an ontology:
documentation provides context, accurate definitions and examples of the
different concepts that are included. In fact, an important part of the
documentation is usually provided within the ontology itself through ontology
metadata and natural language annotations. However, some of the documents may
be external to the ontology, such as the ontology requirements document, other
sources used during the knowledge acquisition phase, the conceptualization
diagrams, examples of use, etc.

In this section we describe our recommended best practices to generate
ontology metadata and human oriented documentation, including diagramming
guidelines to show the relationships between classes in a visual manner.

###  3.1 Ontology Metadata

When creating an ontology, it is crucial to describe it with appropriate
metadata for others to understand it correctly. For example, if some of the
classes are ambiguously defined, other researchers may misinterpret their
meaning when incorporating them into their work. We distinguish two main
categories of metadata in an ontology: the metadata associated with the
ontology itself and the metadata associated with its elements (classes, object
properties, datatype properties and individuals).

The metadata associated to the ontology itself is important to provide an
overview and identify an ontology, understand its usage conditions and
understand its provenance. Table 1 shows our recommended and optional
annotation properties for describing ontologies, along with candidate
properties that can be reused from existing vocabularies and
standards.131313Other vocabularies (e.g., https://schema.org) are alternatives
to the ones proposed. See https://w3id.org/widoco/bestPractices for additional
suggestions. The _recommended_ properties are license (critical for others to
know how the ontology may be used; we recommend a CC-BY license); _creator_ ,
_contributor_ , _creation date_ and _previous version_ to track the provenance
of the ontology and compare against earlier versions; _namespace URI_ and
_version IRI_ to properly identify the ontology; and _prefix_ , _title_ and
_description_ to provide a quick overview on what the ontology does and how to
properly refer to it. Finally, a _citation_ is recommended for letting other
users know how to attribute the authors of the ontology. For the rest of the
section, we will be using the following namespaces:141414For reference, the
TTL version of our example ontology is available at
https://dgarijo.github.io/example/release/1.0.1/ontology.ttl

rdfs <http://www.w3.org/2000/01/rdf-schema#> owl
<http://www.w3.org/2002/07/owl#> bibo <http://purl.org/ontology/bibo/> foaf
<http://xmlns.com/foaf/0.1/> dcterms <http://purl.org/dc/terms/> vaem
<http://www.linkedmodel.org/schema/vaem> vann <http://purl.org/vocab/vann/> sw
<http://www.w3.org/2003/06/sw-vocab-status/ns#>

The _optional_ properties included in Table 1 are not critical to identify or
reuse a target ontology, but provide additional insight and ease its
understanding. These properties include having an _abstract_ and _see also_
with an additional overview of the ontology and links to related resources; a
_status_ to describe its maturity (first draft, specification, etc.);
information about the _backward compatibility_ or other _incompatible_
versions of the ontology; the _modification_ and _issue_ dates; the _original
source_ that led to the definition of the ontology (requirement document, use
cases, etc.); information about the _publisher_ organization supporting the
ontology; the _DOI_ to accessed to a publication about the ontology; and
information about the _logo_ and _diagrams_ that can be used as a visual aid
for the ontology.

Table 1: Recommended and optional metadata for describing ontologies

Property name | Annotation Property | Rationale | Guideline
---|---|---|---
License | dcterms:license | Usage conditions | Recommended
Creator | dcterms:creator | Provenance and attribution | Recommended
Contributor | dcterms:contributor | Provenance and attribution | Recommended
Creation date | dcterms:created | Provenance | Recommended
Previous version | owl:priorVersion | Provenance and comparison | Recommended
Namespace URI | vann:preferredNamespaceUri | Identifying the ontology | Recommended
Version IRI | owl:versionIRI | Versioning | Recommended
Prefix | vann:preferredNamespacePrefix | Identifying the ontology | Recommended
Title | dcterms:title | Understanding | Recommended
Description | dcterms:description | Understanding | Recommended
Citation | dcterms:bibliographicCitation | Credit | Recommended
Abstract | dcterms:abstract | Additional information | Optional
See also | rdfs:seeAlso | Additional information | Optional
Status | sw:status | Maturity information | Optional
Backward compatibility | owl:backwardCompatibility | Version compatibility | Optional
Incompatibility | owl:incompatibleWith | Version compatibility | Optional
Modification Date | dcterms:modified | Provenance and timeliness | Optional
Issued date | dcterms:issued | Provenance and timeliness | Optional
Source | dcterms:source | Provenance | Optional
Publisher | dcterms:published | Provenance | Optional
DOI | bibo:doi | Bibliographic information | Optional
Logo | foaf:logo | Identifying the ontology | Optional
Diagram | foaf:depiction | Visual documentation | Optional

Table 2 shows the recommended and optional metadata properties for classes,
properties, data properties and individuals. Recommended metadata properties
include a human-readable _label_ to identify the term (using as many languages
as needed); and a _definition_ for the ontology term that is as accurate as
possible. Definitions should be clear and illustrative, as classes and
property names may have different meanings to different researchers.

The rest of the properties in Table 2 are nice to have to improve the
understanding of ontology terms. These include _examples_ that illustrate how
to use a term; its _status_ (e.g., deprecated, under discussion, etc.); the
_rationale_ for including a term in the ontology (which may reflect consensus
from a discussion); and the _source_ material that motivated the inclusion of
the term.

###  3.2 Creating a Human-Readable Documentation

Table 2: Recommended and optional properties for describing ontology terms Property name | Annotation Property | Rationale | Guideline
---|---|---|---
Label | rdfs:label | Readibility | Recommended
Definition | rdfs:comment | Understanding | Recommended
Example | vann:example | Understanding | Optional
Status | sw:term_status | Understanding | Optional
Rationale | vaem:rationale | Understanding | Optional
Source | dcterms:source | Provenance | Optional

Ontologies are usually designed in editors and then exported in formats
(Turtle, RDF/XML, JSON-LD, etc.) that are difficult to navigate by humans.
Researchers often address this issue by pointing to an associated paper or
report, but these usually describe a scientific contribution rather than the
definitions of each ontology concept in detail. A better solution is to create
a documentation for all the terms in the ontology. Since this can be a time-
consuming task, the Semantic Web community has developed tools to help
ontology documentation. Given an OWL file as input, these tools generate an
HTML documentation from the metadata included in the ontology itself (by
retrieving the annotation properties recommended in Section 3.1), creating
sections for all classes, properties, data properties and named individuals.
Popular tools for ontology documentation include WIDOCO [3] (an evolution of
the Live OWL Documentation Environment [12] that includes automated
visualization diagrams through the WebVowl tool [9]), Parrot [2] or
OwlDoc151515https://protegewiki.stanford.edu/wiki/OWLDoc (integrated with the
Protégé Ontology editor [11]).

We recommend creating an HTML documentation for ontologies, as it makes them
easier for others to understand and navigate on the Web. The tools introduced
above are a good starting point for documentation generation, but we recommend
expanding their results with additional motivation and context for the target
ontology, pointers to the requirements and rationale; and custom diagrams with
examples that illustrate how to use ontologies in practice.

###  3.3 Ontology visualization

Graphical representations of ontologies help users understand their structure,
relationships and usage. Since there is no standard convention for ontology
diagrams, developers have adopted several approaches, such as UML-alike
diagrams in the SAREF ontologies;161616See the SAREF4AGRI extension
https://w3id.org/def/saref4agri semantic network oriented diagrams in the SSN
Ontology;171717Semantic Sensor Network Ontology https://www.w3.org/TR/vocab-
ssn/ or custom diagrams as in the Provenance Ontology.181818PROV-O: The PROV
Ontology http://www.w3.org/TR/prov-o/

In the last years, conventions for ontology diagrams have been proposed (e.g.,
VOWL [10] and Graffoo191919Specification available at
https://essepuntato.it/graffoo/specification) but none have been standardized
yet. In this section we suggest guidelines for generating ontology diagrams
based on the UML_Ont profile proposed at [4].202020The original UML_Ont
profile uses custom labels and dependencies to cover OWL 1 constructs. Here
labels are mapped to the OWL and RDF(S) constructs. The rationale for our
recommendation is that UML is commonly used in software engineering, and it is
familiar to software developers.

Figure 1 depicts our proposed graphical representations for classes, class
restrictions and class axioms. Named classes are represented by labelled boxes
(1.a); while class restrictions or anonymous classes are represented by empty
boxes (1.b). Intersection class descriptions are represented either by using
an empty circle with the <<owl:intersectionOf>> stereotype (1.c.i); or an icon
including the symbol “$\sqcap$” (1.c.ii). Similarly, union class descriptions
may use an empty circle with the  <<owl:unionOf>> stereotype (1.d.i); or an
icon including the symbol “$\sqcup$”(1.d.ii).

Figure 1: Recommended notation for classes.

Subclasses are represented using the generalization arrow used in UML (1.e.i)
or with a dependency arrow with the <<rdfs:subClassOf>> stereotype (1.e.ii);
and equivalent classes are represented with double-sided UML dependency arrows
with the <<owl:equivalentClass>> (1.f.i) stereotype or by a circle including
the symbol “$\equiv$” (1.f.ii). Lastly, disjoint classes may be represented
with double-sided UML dependency arrows, using the <<owl:disjointWith>>
stereotype (1.g.i) or with a circle including the symbol “$\bot$” (1.g.ii).

Figure 2 illustrates guidelines on how to represent object properties. When
the domain or range are not known (2.a), properties can be represented with
dotted arrows (2.a.i); or with a diamond with the <<owl:ObjectProperty>>
stereotype (2.a.ii). Subproperties may be represented with the UML dependency
arrow with the <<owl:subPropertyOf>> stereotype linking the arrows that
represent the involved object properties (2.b.i) or with an UML dependency
unidirectional arrow with the <<owl:subPropertyOf>> stereotype linking the
diamonds that represent the involved object properties (2.b.ii). When domain
and range are known, properties can be represented with a solid line between
source and target classes (2.c.i) or with a labelled diamond accompanied by
dotted arrows labelled with <<rdfs:domain>> and <<rdfs:range>> respectively
(2.c.ii). Equivalent (2.d) and inverse object properties (2.e) can be
represented by using a bidirectional arrow with the <<owl:equivalentProperty>>
and <<owl:inverseOf>> stereotypes between the lines (2.d.i and 2.e.i) or using
diamond shapes (2.d.ii and 2.e.ii). Lastly, functional (2.f), transitive (2.g)
and symmetric (2.h) object properties can be represented using a shared
notation: either by adding the first initial of the property type (F, T or S)
to the object property label attached to the arrow that represents the object
property; or by a labelled diamond, which represents the object property
itself, including the corresponding stereotype (e.g.,
<<owl:FunctionalProperty>>).

Figure 2: Recommended notation for object properties.

Figure 3 shows how to represent datatype properties. When the domain or range
are not known (3.a) datatype properties may be represented as labelled dashed
boxes attached to boxes representing classes (3.a.i); or as a diamond with
<<owl:DatatypeProperty>>. Subproperties for datatypes may be represented with
a UML dependency arrow with the <<owl:subPropertyOf>> stereotype linking the
diamonds that represent the involved datatype properties (3.b). When the
domain and/or range are known, the box representing the datatype property may
be depicted with a solid line indicating that the domain of the datatype
property is the attached class and the range may be included following the
characters “::” after the datatype label (3.c.i). Alternatively, a labelled
diamond may be used accompanied by dotted arrows labelled with the
<<rdfs:domain>> and <<rdfs:range>> stereotypes respectively (3.c.ii).
Equivalent datatype properties may be represented by a UML dependency
bidirectional arrow with the <<owl:EquivalentProperty>> stereotype linking the
diamonds that represent the datatype properties (3.d). Functional datatype
properties may be represented by adding “(F)” to the datatype property label
(3.e.i) or by a labelled diamond including the <<owl:FunctionalProperty>>
stereotype (3.e.ii).

Figure 3: Recommended notation for datatype properties.

Finally, Figure 4 proposes how to represent individuals and class membership.
Individuals or instances may be represented by labelled boxes with underlined
names or identifiers (4.a). Class membership may be represented with labelled
boxes containing the individual name followed by the character “:” and the
class name, all underlined (4.b.i); by a linking the individual box with the
class using a unidirectional UML dependency arrow with the stereotype
<<rdf:type>> (4.b.ii) or with a underlined labelled box for the individual
attached to the class.

Figure 4: Recommended notation for individuals.

##  4 Ontology Publication on the Web

Once an ontology is fully implemented and documented, it is time to make it
accessible and findable in the Web. In this section we briefly describe the
best practices to perform content negotiation to serve a target ontology in
multiple formats (Section 4.1) and registries for describing new ontologies
(Section 4.2).

###  4.1 Ontology Accessibility in Multiple Interoperable Formats

Ontologies should be made available in both human and machine readable manner
using a single identifier: the ontology URI. This way we can make any ontology
resolve to its HTML documentation when accessed by a user in a browser; and
resolve to a standard RDF serialization when importing it in an ontology
editor. In order to distinguish the target resource to serve (HTML or RDF
serialization), we must implement a _303 See Other redirect_
,212121https://tools.ietf.org/html/rfc7231#page-57 a common practice in the
Semantic Web community for doing _content negotiaiton_ over URIs. This type of
redirect indicates the location of a target resource in the server based on
the received request, but has to be appropriately configured on the server
where we are hosting the ontology. Fortunately, there are W3C best practices
(recipes) on how to configure an Apache HTTP server -a commonly used type of
server for serving files- for hash ended and slash ended
ontologies.222222http://www.w3.org/TR/swbp-vocab-pub/ Here we expand these
practices with our example ontology to illustrate: 1) How to support multiple
serializations of an ontology (HTML and Turtle); 2) how to support version
redirection, as we would like all the versions of the ontology to be
appropriately available, not only the latest; 3) How to specify if a
serialization is not supported (for example, requests for JSON-LD will return
a 406 non-acceptable code, rather than an RDF/XML serialization); and 4) how
to implement a default response in case the user agent doing the request does
not specify a target in the request (by default we return Turtle). The
_htaccess_ file to be placed in the server (in an “/example” folder) would
look as follows:

# Turn off MultiViews (Apache-specific command) Options -MultiViews #
Directive to specify supported types besides html and xml # Standard RDF
serialization formats include Turtle, RDF/XML, N-Triples and JSON-LD AddType
text/turtle .ttl RewriteEngine on # Rewrite rule for accessing the latest
version. RewriteCond %{HTTP_ACCEPT}
!application/rdf\\+xml.*(text/html|application/xhtml\\+xml) RewriteCond
%{HTTP_ACCEPT} text/html [OR] RewriteCond %{HTTP_ACCEPT}
application/xhtml\\+xml [OR] RewriteCond %{HTTP_USER_AGENT} ^Mozilla/.*
RewriteRule ^$ https://dgarijo.github.io/example/release/1.0.1/index-en.html
[R=303,L] # Rewrite rule to serve the Turtle serialization from the vocabulary
URI (latest version) RewriteCond %{HTTP_ACCEPT} text/turtle [OR] RewriteCond
%{HTTP_ACCEPT} text/\\* [OR] RewriteCond %{HTTP_ACCEPT} \\*/turtle RewriteRule
^$ https://dgarijo.github.io/example/release/1.0.1/ontology.ttl [R=303,L] #
Rewrite rules for retrieving a particular version. RewriteCond %{HTTP_ACCEPT}
!application/rdf\\+xml.*(text/html|application/xhtml\\+xml) RewriteCond
%{HTTP_ACCEPT} text/html [OR] RewriteCond %{HTTP_ACCEPT}
application/xhtml\\+xml [OR] RewriteCond %{HTTP_USER_AGENT} ^Mozilla/.*
RewriteRule ^(.+)$ https://dgarijo.github.io/example/release/$1/index-en.html
[R=303,L] # Rewrite rule to serve Turtle serialization of a particular version
RewriteCond %{HTTP_ACCEPT} text/turtle [OR] RewriteCond %{HTTP_ACCEPT}
text/\\* [OR] RewriteCond %{HTTP_ACCEPT} \\*/turtle RewriteRule ^(.+)$
https://dgarijo.github.io/example/release/$1/ontology.ttl [R=303,L] # Rewrite
rule for other non accepted formats RewriteCond %{HTTP_ACCEPT} .+ RewriteRule
^(.*)$ https://dgarijo.github.io/example/release/1.0.1/406.html [R=406,L] #
Rewrite rule to serve the Turtle content from the vocabulary URI by default
RewriteRule ^$https://dgarijo.github.io/example/release/1.0.1/ontology.ttl
[R=303,L]

In order to test the redirection, the easiest way is just to paste the URI of
the ontology in Protégé or in your browser and check that both load the right
serialization. Another possibility is to use a _curl_
command,232323https://curl.haxx.se/ e.g., to retrieve the Turtle
serialization:

    curl -sH "Accept: text/turtle" -L https://w3id.org/example#

If interested in a particular version, we can use its version IRI. For
example, for the documentation of version 1.0.0:

    curl -sH "Accept: text/html" -L https://w3id.org/example/1.0.0

###  4.2 Making an Ontology Findable on the Web

Once an ontology is published in the Web, the next step is to ensure it can be
easily found by others. There are three main activities that can help the
visibility of an ontology:

  1. 1.

Register the namespace prefix: prefix.cc,242424http://prefix.cc is a
crowdsourced registry where users can vote the most popular URL for a given
prefix.

  2. 2.

Register the ontology: There are a number of existing metadata registries that
can be used for browsing existing ontologies [15][17]. Our recommendation is
to look first for domain-specific registries (e.g., Bioportal [17] in the
biomedical domain, Agroportal [7] in Agronomy, etc.) commonly used by the
target community of interest. When domain-specific registries do not exist, we
suggest registering the ontology in a domain-generic metadata registry, such
as Linked Open Vocabularies [15] (which has a manual curation process to
ensure that minimum metadata is provided) or
FAIRsharing.252525https://fairsharing.org/standards/

  3. 3.

In-document annotations to help crawlers understand the metadata of the
ontology when publishing it on the Web. For instance, annotations can be added
in your HTML documentation through JSON-LD
snippets:262626https://www.w3.org/TR/json-ld11/

<!-- Annotations for the example ontology --> <script
type="application/ld+json">{ "@context":"http://schema.org",
"@type":"WebPage", "url":"https://w3id.org/example", "name":"The example
ontology", "datePublished":"4-2-2020", "version":"1.0.1",
"license":"http://creativecommons.org/licenses/by/2.0/",
"author":[{"@type":"Person","name":"Daniel Garijo"},
{"@type":"Person","name":"Maria Poveda"}], }</script>

##  5 Ontology Documentation and Publication Frameworks

Semantic Web researchers and practitioners have developed methods and tools
for easing ontology engineering, development, publication and exploitation. A
number these tools have already been mentioned in the corresponding sections
of this chapter, however, they are not always integrated as part of an end-to-
end framework, and researchers have to use them separately.

More recently, frameworks inspired in the software continuous integration
practices have arisen to support ontology engineering activities in
collaborative environments. These frameworks offer end-to-end solutions that
support ontology engineers documenting, visualizing, testing and publishing
their ontologies; and we recommend them as an entry point to adopt some of the
practices described in this chapter. One example is OnToology [1], a web
application272727http://ontoology.linkeddata.es/ that orchestrates ontology
documentation, evaluation and publication on the Web with permanent URLs.
Another similar approach is VoCol [5], which provides ontology developers with
feedback on syntax and other errors and gives access to a human-readable
presentation of a target ontology. Finally, PoolParty [13] is a commercial
solution that also includes publication of thesauri, taxonomies and ontology
management among other features.

##  6 Conclusions

In this chapter we have described implementation guidelines and
recommendations for making ontologies findable (through metadata registries
and annotations); accessible (through good practices in URI design and content
negotiation), interoperable (showing how to serve ontologies in different
standard serializations) and reusable (by describing the metadata and diagram
guidelines needed for proper understanding) on the Web while following the
Linked Data principles. A distinct feature of our guidelines is that we have
illustrated how to carry out our recommendations with an example ontology and
pointers to usable tools developed by the Semantic Web community in the last
decade. Our recommendations reflect years of experience on ontology
engineering and also summarize community discussions for ontology design and
publication. Hence, we believe these guidelines are a comprehensive starting
point for ontology developers who aim to make their ontologies FAIR and
available on the Web.

## References

  * [1] Alobaid, A., Garijo, D., Poveda-Villalón, M., Santana-Perez, I., Fernández-Izquierdo, A., Corcho, O.: Automating ontology engineering support activities with OnToology. Journal of Web Semantics (2018)
  * [2] Alonso, C.T., Berrueta, D., Polo, L., Fernández, S.: Current practices and perspectives for metadata on web ontologies and rules. International Journal of Metadata, Semantics and Ontologies 7(2), 93 (2012)
  * [3] Garijo, D.: WIDOCO: A Wizard for Documenting Ontologies. In: The Semantic Web – ISWC 2017, vol. 10588, pp. 94–102. Cham (2017)
  * [4] Haase, P., Brockmans, S., Palma, R., Euzenat, J., d’Aquin, M.: D1.1.2 updated version of the networked ontology model. Tech. rep., Universität Karlsruhe (2009), NeOn Project. http://www. neon-project. org
  * [5] Halilaj, L., Petersen, N., Grangel-González, I., Lange, C., Auer, S., Coskun, G., Lohmann, S.: Vocol: An integrated environment to support version-controlled vocabulary development. In: 20th International Conference on Knowledge Engineering and Knowledge Management - EKAW 2016, Bologna, Italy. Lecture Notes in Computer Science, vol. 10024, pp. 303–319 (2016)
  * [6] Janowicz, K., Hitzler, P., Adams, B., Kolas, D., Vardeman II, C.: Five stars of Linked Data vocabulary use. Semantic Web 5(3), 173–176 (2014)
  * [7] Jonquet, C., Toulet, A., Arnaud, E., Aubin, S., Yeumo, E.D., Emonet, V., Graybeal, J., Laporte, M.A., Musen, M.A., Pesce, V., Larmande, P.: Agroportal: A vocabulary and ontology repository for agronomy. Computers and Electronics in Agriculture 144, 126 – 143 (2018)
  * [8] Le Franc, Y., Parland-von Essen, J., Bonino, L., Lehväslaiho, H., Coen, G., Staiger, C.: D2.2 fair semantics: First recommendations (Mar 2020), https://doi.org/10.5281/zenodo.3707985
  * [9] Lohmann, S., Link, V., Marbach, E., Negru, S.: WebVOWL: Web-based visualization of ontologies. In: Proceedings of EKAW 2014 Satellite Events. LNAI, vol. 8982, pp. 154–158. Springer (2015)
  * [10] Lohmann, S., Negru, S., Haag, F., Ertl, T.: VOWL 2: User-oriented visualization of ontologies. In: Proceedings of the 19th International Conference on Knowledge Engineering and Knowledge Management (EKAW ’14). LNAI, vol. 8876, pp. 266–281. Springer (2014)
  * [11] Musen, M.A.: The Protégé project: a look back and a look forward. AI Matters 1(4), 4–12 (Jun 2015)
  * [12] Peroni, S., Shotton, D., Vitali, F.: The Live OWL Documentation Environment: A Tool for the Automatic Generation of Ontology Documentation. In: Knowledge Engineering and Knowledge Management, vol. 7603, pp. 398–412. Springer Berlin Heidelberg, Berlin, Heidelberg (2012)
  * [13] Schandl, T., Blumauer, A.: Poolparty: Skos thesaurus management utilizing linked data. In: Extended Semantic Web Conference. pp. 421–425. Springer (2010)
  * [14] Suárez-Figueroa, M.C.: NeOn Methodology for building ontology networks: specification, scheduling and reuse. Ph.D. thesis, Facultad de Informatica, Universidad Politécnica de Madrid (2010)
  * [15] Vandenbussche, P.Y., Atemezing, G.A., Poveda-Villalón, M., Vatant, B.: Linked Open Vocabularies (LOV): A gateway to reusable semantic vocabularies on the Web. Semantic Web 8(3), 437–452 (Jan 2017)
  * [16] Vrandečić, D., Krötzsch, M.: Wikidata: a free collaborative knowledgebase. Communications of the ACM 57(10), 78–85 (Sep 2014)
  * [17] Whetzel, P.L., Noy, N.F., Shah, N.H., Alexander, P.R., Nyulas, C., Tudorache, T., Musen, M.A.: BioPortal: enhanced functionality via new Web services from the National Center for Biomedical Ontology to access and use ontologies in software applications. Nucleic Acids Research 39(suppl), W541–W545 (Jul 2011)
  * [18] Wilkinson, M.D., Dumontier, M., Aalbersberg, I.J., Appleton, G., Axton, M., Baak, A., Blomberg, N., et.al.: The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data 3, 160018 (Mar 2016)