# Results from the Baksan Experiment on Sterile Transitions (BEST)

V.V. Barinov  Institute for Nuclear Research of the Russian Academy of
Sciences, Moscow 117312, Russia  B.T. Cleveland  SNOLAB, Sudbury, ON P3Y 1N2,
Canada  S.N. Danshin  Institute for Nuclear Research of the Russian Academy of
Sciences, Moscow 117312, Russia  H. Ejiri  Research Center for Nuclear
Physics, Osaka University, Osaka, Japan  S.R. Elliott  Los Alamos National
Laboratory, Los Alamos, NM, USA  D. Frekers  Institut für Kernphysik,
Westfälische Wilhelms-Universität Munster, D-48149 Munster, Germany  V.N.
Gavrin<EMAIL_ADDRESS>Institute for Nuclear Research of the Russian Academy of
Sciences, Moscow 117312, Russia  V.V. Gorbachev  Institute for Nuclear
Research of the Russian Academy of Sciences, Moscow 117312, Russia  D.S.
Gorbunov  Institute for Nuclear Research of the Russian Academy of Sciences,
Moscow 117312, Russia  W.C. Haxton  Nuclear Science Division, Lawrence
Berkeley National Laboratory, Berkeley, CA 94720, USA  Department of Physics,
University of California, Berkeley, CA 94720, USA  T.V. Ibragimova  Institute
for Nuclear Research of the Russian Academy of Sciences, Moscow 117312, Russia
I. Kim  Los Alamos National Laboratory, Los Alamos, NM, USA  Yu.P. Kozlova
Institute for Nuclear Research of the Russian Academy of Sciences, Moscow
117312, Russia  L.V. Kravchuk  Institute for Nuclear Research of the Russian
Academy of Sciences, Moscow 117312, Russia  V.V. Kuzminov  Institute for
Nuclear Research of the Russian Academy of Sciences, Moscow 117312, Russia
B.K. Lubsandorzhiev  Institute for Nuclear Research of the Russian Academy of
Sciences, Moscow 117312, Russia  Yu.M. Malyshkin  Institute for Nuclear
Research of the Russian Academy of Sciences, Moscow 117312, Russia  R.
Massarczyk  Los Alamos National Laboratory, Los Alamos, NM, USA  V.A. Matveev
Joint Institute for Nuclear Research (JINR) Joliot-Curie 6, 141980, Dubna,
Moscow Region, Russia  I.N. Mirmov  Institute for Nuclear Research of the
Russian Academy of Sciences, Moscow 117312, Russia  J.S. Nico  National
Institute of Standards and Technology, 100 Bureau Dr, Gaithersburg, MD 20899,
USA  A.L. Petelin  JSC ‘State Scientific Center Research Institute of Atomic
Reactors’, Dimitrovgrad, 433510, Russia  R.G.H. Robertson  Center for
Experimental Nuclear Physics and Astrophysics, and Department of Physics,
University of Washington, Seattle, WA 98195, USA  D. Sinclair  Carleton
University 1125 Colonel By Drive Ottawa, K1S 5B6, Canada  A.A. Shikhin
Institute for Nuclear Research of the Russian Academy of Sciences, Moscow
117312, Russia  V.A. Tarasov  JSC ‘State Scientific Center Research Institute
of Atomic Reactors’, Dimitrovgrad, 433510, Russia  G.V. Trubnikov  Joint
Institute for Nuclear Research (JINR) Joliot-Curie 6, 141980, Dubna, Moscow
Region, Russia  E.P. Veretenkin  Institute for Nuclear Research of the Russian
Academy of Sciences, Moscow 117312, Russia  J.F. Wilkerson  Department of
Physics and Astronomy, University of North Carolina, Chapel Hill, NC 27599,
USA  Triangle Universities Nuclear Laboratory, Durham, NC 27708, USA  A.I.
Zvir  JSC ‘State Scientific Center Research Institute of Atomic Reactors’,
Dimitrovgrad, 433510, Russia

###### Abstract

The Baksan Experiment on Sterile Transitions (BEST) was designed to
investigate the deficit of electron neutrinos, $\nu_{e}$, observed in previous
gallium-based radiochemical measurements with high-intensity neutrino sources,
commonly referred to as the gallium anomaly, which could be interpreted as
evidence for oscillations between $\nu_{e}$ and sterile neutrino ($\nu_{s}$)
states. A 3.414-MCi 51Cr $\nu_{e}$ source was placed at the center of two
nested Ga volumes and measurements were made of the production of 71Ge through
the charged current reaction, 71Ga($\nu_{e}$,e-)71Ge, at two average
distances. The measured production rates for the inner and the outer targets
respectively are ($54.9^{+2.5}_{-2.4}(\mbox{stat})\pm 1.4(\mbox{syst})$) and
($55.6^{+2.7}_{-2.6}(\mbox{stat})\pm 1.4(\mbox{syst})$) atoms of 71Ge/d. The
ratio ($R$) of the measured rate of 71Ge production at each distance to the
expected rate from the known cross section and experimental efficiencies are
$R_{in}=0.791\pm 0.05$ and $R_{out}=0.766\pm 0.05$. The ratio of the outer to
the inner result is 0.97$\pm$0.07, which is consistent with unity within
uncertainty. The rates at each distance were found to be similar, but 20-24%
lower than expected, thus reaffirming the anomaly. These results are
consistent with $\nu_{e}\rightarrow\nu_{s}$ oscillations with a relatively
large $\Delta m^{2}$ ($>$1 eV2) and mixing sin${}^{2}2\theta$ ($\approx$0.4).

The possibility of the existence of light sterile neutrinos ($\nu_{s}$) is
presently a major field of inquiry. The literature on this topic is extensive
but has been summarized well in a number of recent reviews Abazajian  _et al._
(2012); Gariazzo  _et al._ (2015); Giunti and Lasserre (2019); Böser  _et al._
(2020); Diaz  _et al._ (2020); Seo (2020); Dasgupta and Kopp (2021). Much of
the evidence for $\nu_{s}$’s comes from oscillation experiments that search
for the conversion of an active neutrino into a sterile state.

The SAGE Abdurashitov  _et al._ (2009) and GALLEX Kaether  _et al._ (2010)
radiochemical experiments detected neutrinos from the Sun through the charged-
current reaction 71Ga($\nu_{e}$,e-)71Ge. The method exposed a large mass of Ga
metal (30-50 t) to the Sun for about a month and then chemically extracted the
radioactive 71Ge atoms ($\tau_{1/2}$=(11.43$\pm$0.03) d Hampel and Remsberg
(1985)), mixed the Ge with a proportional counter gas, and counted the
decaying 71Ge in a low-background system. Both collaborations followed up the
solar neutrino studies with strong radioactive electron-capture sources to
confirm their sensitivity to interactions with $\nu_{e}$ from the Sun. These
experiments, using 51Cr Abdurashitov  _et al._ (1999a); Hampel  _et al._
(1998) and 37Ar Abdurashitov  _et al._ (2006) placed at the center of their Ga
targets, found a 71Ge production rate of about 87% of that expected
Abdurashitov  _et al._ (2006). This led to extensive studies of the cross
section Barinov  _et al._ (2018); Frekers  _et al._ (2011, 2013, 2015);
Semenov (2020), the extraction efficiency, and counting efficiencies Hampel
_et al._ (1998); Abdurashitov  _et al._ (2002) by both collaborations and a
number of outside interested groups Kaether  _et al._ (2010); Giunti  _et al._
(2012); Kostensalo  _et al._ (2019). This discrepancy between the expected and
measured rates defines the gallium anomaly and has been interpreted in the
context of $\nu_{e}\rightarrow\nu_{s}$ oscillations Laveder (2007). Although
the statistical evidence for a deviation from expectation is modest, about
2-3$\sigma$, it has persisted motivating the need for further investigation.
Furthermore, given the simplicity of the electron-capture neutrino energy
spectrum and the well-known cross section ($\sigma$) at these low energies,
this is an effective technique to search for $\nu_{s}$’s.

The previous source measurements used a single target and, therefore, required
comparison of a measured rate to a theoretical expectation. The Baksan
Experiment on Sterile Transitions (BEST) was designed as a two-distance
oscillation experiment. The experimental concept is depicted in Fig. 1. An
inner spherical volume, with diameter 133.5 cm, contains (7.4691$\pm$0.0631) t
of Ga. An outer cylindrical volume (234.5 cm high, 218 cm diam.) contains
(39.9593$\pm$0.0024) t of Ga. The active core of the source consisted of 26
irradiated Cr disks, placed into a stainless-steel cylinder with a radius of
4.3 cm and height 10.8 cm, was shielded for radiation safety with a tungsten
alloy of thickness of $\approx$30 mm. The source was placed at the center
irradiating both volumes simultaneously, permitting the production rate of
71Ge to be measured at two different distances. After exposure, the Ga was
pumped to reactors for the extraction chemistry.

Figure 1: The Ga target and extraction piping diagram also indicating the
source handling apparatus.

The source was delivered to the Baksan Neutrino Observatory (BNO) on July 5,
2019 and was placed into the two-zone target at 14:02 that same day and this
is our chosen reference time for the source strength. The source was
manufactured by irradiating 4 kg of 50Cr-enriched metal for 100 d in a reactor
at the State Scientific Center Research Institute of Atomic Reactors,
Dimitrovgrad, Russia. The activity ($A$) at the reference time is
(3.414$\pm$0.008) MCi. A full description of the source and the calorimetric
measurements of its intensity can be found in Refs. Kozlova  _et al._ (2020);
Gavrin  _et al._ (2021). The 51Cr isotope emits $\nu_{e}$’s at four energies;
747 keV (81.63%), 427 keV (8.95%), 752 keV (8.49%) and 432 keV (0.93%).

Twenty extractions, 10 from each volume, were conducted between July 15 and
Oct. 13, 2019. Each exposure lasted approximately 9 d and counting of the
sample commenced approximately 24 h after the extractions. At the start of
each exposure, $\approx$250 $\mu$g of Ge carrier was added to each Ga volume.
At the end of the exposure, the carrier Ge and any produced 71Ge was extracted
using the procedure described in Ref. Cleveland  _et al._ (2015). The process
ensured the independent extraction of 71Ge atoms from each zone of the Ga
target. The gas germane was synthesized, mixed with Xe, and inserted into
small ($\sim$0.6 cm3), low-background proportional counters. The counters were
installed into a NaI well volume of one of the two counting systems. The
counting duration varied from 60 to 150 days. The irradiations were scheduled
to maximize the number of extracted 71Ge atoms. The first extraction’s
counting times were shorter due to the limited number of working counters. The
shorter counting time had little effect on the number of measured 71Ge decays,
but the statistical uncertainty was increased due to the lower statistical
determination of the counter background.

Two 8-channel data acquisition systems were used Shikhin  _et al._ (2017,
2018); Abdurashitov  _et al._ (1999b). Pulses from the proportional counters
were digitized at 1 GHz with a bandwidth of 100 MHz and a rise time of 3.5 ns.
The energy range of the 8-bit digitizer was 0.37-15 keV. The energy range of
the NaI counters was 60-3000 keV. Data were collected in event mode, recording
the time of occurrence.

The digitized pulse shapes were analyzed for energy and rise time Abdurashitov
_et al._ (1999b). The measure of energy is the integral of the pulse waveform
for 800 ns after pulse onset. The peak position for each counter is based on
routine periodic calibrations with 55Fe. The rise time ($T_{N}$) was
determined by a functional fit to the waveform Elliott (1990). After counting
of the samples from the Cr experiment was completed in fall of 2020,
measurements of the counting efficiency were made for each counter used in the
experiment. Two different techniques and two different isotopes were employed:
37Ar to measure volume efficiency, and 71Ge to measure the L- and K-peak
efficiencies and the $T_{N}$ acceptance for each counter. The $T_{N}$ values
were arranged in ascending order and an upper limit was set such that 4% of
the events were excluded Abdurashitov  _et al._ (2006). The volume
efficiencies of all counters used in the experiment were directly measured
with 37Ar. The calculated counting efficiency using the measured pressure,
GeH4 fraction, and 37Ar volume efficiency was determined for each extraction.
The total uncertainty in these calculated efficiencies is $\pm$1.1%.

The likelihood fits to the time distribution of the candidate events were
performed as in Ref. Abdurashitov  _et al._ (1999a). This analysis includes a
71Ge contribution with its 11-d half life and a constant background rate. For
joint fits of all extractions, the decay of the 51Cr source was taken into
account. Table 1 presents a summary of the K+L fit results for each extraction
for the inner and outer volumes. Additionally, a combined fit for each is
given. Figure 2 shows the K+L production rate fits for the two volumes
indicating the resulting production rate at the reference time. Two
independent analyses were pursued and both obtained similar results to within
about 2%. This difference is due to minor event-selection differences at the
edges of the selection borders in energy and rise time. This difference is
accounted for by the estimated systematic uncertainties in the efficiencies
for those cuts.

Table 1: A summary of the likelihood fits for the production rate from each extraction, the combined fit of all extractions, and the predicted production rate. The quoted measurement uncertainties are statistical. | Inner Volume | Outer Volume
---|---|---
Exposure | K+L | Number fit |  51Cr | Production | K+L | Number fit |  51Cr | Production
Dates (DoY) | Candidates | to 71Ge | Production | Rate (Atoms/d) | Candidates | to 71Ge | Production | Rate (Atoms/d)
186.585-196.376 | 180 | 176.3 | 175.5 | 49.4${}^{+4.2}_{-4.0}$ | 181 | 133.4 | 129.6 | $41.1^{+5.3}_{-5.2}$
197.362-206.372 | 129 | 111.5 | 107.7 | 44.9${}^{+5.9}_{-5.6}$ | 174 | 163.8 | 158.6 | 63.6${}^{+5.7}_{-5.5}$
207.282-216.374 | 132 | 117.6 | 115.4 | 62.9${}^{+7.4}_{-7.1}$ | 116 | 92.5 | 88.2 | 51.4${}^{+7.3}_{-6.9}$
217.286-226.371 | 93 | 87.3 | 85.6 | 73.3${}^{+8.6}_{-8.0}$ | 98 | 82.3 | 78.9 | 66.6${}^{+9.8}_{-9.2}$
227.258-236.458 | 134 | 60.2 | 58.4 | 49.8${}^{+8.2}_{-7.7}$ | 120 | 64.0 | 59.5 | 46.9${}^{+7.9}_{-7.2}$
237.342-246.369 | 81 | 48.8 | 47.7 | 69.5${}^{+12.0}_{-11.0}$ | 97 | 62.3 | 59.3 | 87.3${}^{+13.2}_{-12.3}$
247.243-256.368 | 91 | 45.0 | 43.9 | 64.6${}^{+12.6}_{-11.6}$ | 69 | 38.0 | 34.4 | 50.4${}^{+10.6}_{-9.6}$
257.241-266.369 | 59 | 33.6 | 32.4 | 53.8${}^{+12.2}_{-11.0}$ | 68 | 43.4 | 39.2 | 59.7${}^{+11.7}_{-10.8}$
267.240-276.369 | 106 | 23.7 | 22.7 | 49.9${}^{+16.5}_{-14.9}$ | 66 | 20.2 | 17.0 | 43.0${}^{+15.3}_{-13.5}$
277.201-286.367 | 88 | 25.2 | 24.3 | 69.1${}^{+19.4}_{-17.3}$ | 81 | 31.8 | 28.0 | 78.8${}^{+20.0}_{-18.1}$
Combined | 1093 | 724.0 | 708.2 | 54.9${}^{+2.5}_{-2.4}$ | 1069 | 738.8 | 699.8 | 55.6${}^{+2.7}_{-2.6}$
Predicted |  |  |  | 69.41${}^{+2.5}_{-2.0}$ |  |  |  | 72.59${}^{+2.6}_{-2.1}$

Figure 2: Top: (a) the measured K+L peak rates of the inner target volume; (b)
normalizes the production rate to the reference time, the combined results for
events in the the L and K peaks are shown. The blue (red) region represents
the predicted (measured) production rate. Bottom: Similar but for the outer
volume. The dotted lines enclose the $\pm 1\sigma$ uncertainty regions.

During each extraction a small fraction of the production is due to solar
neutrinos. The measured solar neutrino capture rate is (66.1$\pm$3.1) SNU1111
SNU corresponds to one neutrino capture per second in a target that contains
1036 atoms of the neutrino absorbing isotope, in our case 71Ga Abdurashitov
_et al._ (2009) and typically results in about 0.51 (3) counts per extraction
attributed to the K+L counts for the inner (outer) target. Due to the
inefficiency of the extraction, there are also some 71Ge atoms that carryover
from one extraction to the next. Typically this is about 1 count for each
volume. Both of these effects were taken into account, extraction by
extraction.

The systematic uncertainties have been estimated from auxiliary tests. The
chemical extraction efficiency is typically about 95% with an uncertainty of
$\pm$1.6%. The summed K+L peak counting efficiency is typically about 70% with
an uncertainty of -1.8/+2.0%. There are small uncertainties due to the Rn cut
(-0.05%), the solar neutrino correction ($\pm$0.20%), and the carryover
correction ($\pm$0.04%). The total systematic uncertainty is estimated to be
-2.5/+2.6%. Note that the uncertainty in the extraction efficiency has been
greatly reduced as compared to Ref. Abdurashitov  _et al._ (2006). This is due
to the use of mass spectrometry to determine with high accuracy the efficiency
of extraction of minute quantities of Ge from a large mass of Ga Cleveland
_et al._ (2015). The details of the systematic uncertainties are described in
Ref Barinov  _et al._ (2021).

The use of an electron capture $\nu_{e}$ source is a powerful technique to
search for $\nu$ oscillations. The $\nu_{e}$ spectrum is simple, being
comprised of 2 dominant monoenergetic components (and 2 sub-dominant lines).
It is a well-understood spectrum relying on well-known nuclear and atomic
physics parameters.

The cross section has to be calculated from nuclear physics input and when the
original Ga anomaly was observed, there was concern that the transition
strengths to excited states were not fully understood. Bahcall Bahcall (1997)
derived the ground state contribution from the 71Ge half-life, but the excited
state contributions were estimated from charge exchange (i.e. (p,n))
reactions. For the central value, Bahcall used the best estimate of the
transition strength values to the excited states with an estimated uncertainty
to be the change in $\sigma$ (-1.6/+2.8%), if one ignores the excited states.
The charge exchange data has been improved by recent work Frekers  _et al._
(2011, 2013, 2015) indicating that they are not the cause of the discrepancy.
However, the excited-state contribution uncertainty is critical because the
(p,n) measurements have a significant cancellation between the Gamow-Teller
and tensor matrix elements resulting in an underestimate of the transition
strengths Haxton (1998). Kostensalo et al. Kostensalo  _et al._ (2019) used a
nuclear shell model calculation to avoid the (p,n) measurement drawback. The
paper of Semenov et al. Semenov (2020) reproduces Bahcall’s approach but uses
modern values for the transition strengths Alanssari  _et al._ (2016). The
Semenov et al. and Kostensalo et al. results differ by about 4%, which is
about 2-3 times larger than the uncertainty estimated for each. Interestingly,
the original Bahcall number is half way between these two results with a
$\pm$4% uncertainty. We therefore use the Bahcall $\sigma$ value and the
associated conservative uncertainties from his estimate:
$(5.81^{+0.21}_{-0.16})\times 10^{-45}$ cm2.

The survival probability at a distance $d$ for two-component oscillation for a
given $\nu$ energy ($E_{\nu}$) is

$P_{ee}(d)=1-\mbox{sin}^{2}2\theta\mbox{sin}^{2}\left(1.27\frac{\Delta
m^{2}[\mbox{eV}^{2}]d[\mbox{m}]}{E_{\nu}[\mbox{MeV}]}\right).$ (1)

where $\Delta m^{2}$ is the difference of the masses squared between the two
neutrino species and $\theta$ is the angle that defines the mixing between
them. The capture rate ($r$) can be written

$r=\int_{V}F\sum_{i=1}^{4}(f_{i}P_{ee}^{i})\sigma nd\vec{x},$ (2)

where $F$ is the flux of $\nu_{e}$, $P_{ee}^{i}$ is the oscillation survival
probability for the $i^{th}$ neutrino branch with branching fraction $f_{i}$,
$\sigma$ is the cross section, $n$ is the 71Ga number density ($(2.1001\pm
0.0008)\times 10^{22}$/cm3) and the integral is calculated over the target
volume ($V$). With $A$ as the source activity and $d$ the distance between
emission and absorption of the $\nu_{e}$, this can be written

$r=\frac{n\sigma
A}{4\pi}\int_{V}\frac{\sum_{i}(f_{i}P_{ee}^{i}(d))}{d^{2}}d\vec{x}.$ (3)

The integral is calculated by Monte Carlo due to the complexity of the target
geometry. The average path length $<L>$ of a neutrino through the target is
given by the integral when $P_{ee}=1$. The average path lengths for the BEST
volumes are $<L>_{in}=(52.03\pm 0.18$) cm and $<L>_{out}=(54.41\pm 0.18$) cm.
The uncertainties on these numbers are dominated by dimensional uncertainties
of the apparatus.

For $n=1,\dots,N$ experiments (the two BEST volumes are treated separately),
oscillation parameters are estimated by a global minimization of

$\chi^{2}(\Delta m^{2},\sin^{2}2\theta)=({\bf r^{meas}}-{\bf
r^{calc}})^{T}{\bf V}^{-1}({\bf r^{meas}}-{\bf r^{calc}})$ (4)

with the ${\bf r^{meas}}$ (${\bf r^{calc}}$) is the vector of the measured
(calculated) rates with $r_{i}^{calc}(\Delta m^{2};\mbox{sin}^{2}2\theta)$ and
the covariance matrix

$V_{nk}=\delta_{nk}\varepsilon_{n}^{2}+\varepsilon_{CS}^{n}\times\varepsilon_{CS}^{k}$
(5)

where $\varepsilon_{n}^{2}=\varepsilon_{n,stat}^{2}+\varepsilon_{n,syst}^{2}$
are uncorrelated uncertainties comprised of statistical and systematic
measurement uncertainties, and $\varepsilon_{CS}^{n}$ represent the correlated
uncertainties of $\sigma$ Fogli  _et al._ (2002).

The calculation of the confidence level contours corresponding to a
$\Delta\chi^{2}=\chi^{2}-\chi^{2}_{min}$ with two degrees of freedom:
$\Delta\chi^{2}$ = 2.30, 6.18, 11.83 for 68.27% (1$\sigma$), 95.45%
(2$\sigma$) and 99.73% (3$\sigma$) C.L., respectively. The two BEST results
for the measured to expected ratios are $R_{out}$ = 0.766$\pm$0.05 and
$R_{in}$ = 0.791$\pm$0.05. The results from SAGE are $R_{Cr}=0.95\pm 0.12$
Abdurashitov  _et al._ (1999a), $R_{Ar}=0.79^{+0.09}_{-0.10}$ Abdurashitov
_et al._ (2006) and for GALLEX are $R_{Cr1}=0.95\pm 0.11$ and $R_{Cr2}=0.81\pm
0.11$ Kaether  _et al._ (2010); Altmann  _et al._ (2005).

Figure 3 shows the allowed $\Delta m^{2}$ \- sin${}^{2}2\theta$ parameter
space assuming that $\nu_{e}\rightarrow\nu_{s}$ oscillations is the origin of
the gallium anomaly. The best fit for BEST only data is $\Delta m^{2}=3.3$ eV2
and sin${}^{2}2\theta$ = 0.42. Including all the Ga data, the result is
$\Delta m^{2}=1.25$ eV2 and sin${}^{2}2\theta$ = 0.34.

Figure 3: Top: Exclusion for the results from BEST. The best fit point
(b.f.p.) is for $\Delta m^{2}$ = 3.3 eV2 and sin${}^{2}2\theta$ = 0.42.
Bottom: The exclusion plot for the BEST result combined with the previous SAGE
and GALLEX results. The best fit point is for $\Delta m^{2}$ = 1.25 eV2 and
sin${}^{2}2\theta$ = 0.34.

The $\nu_{e}-\nu_{s}$ oscillation parameter space minimum (Fig. 3) is very
broad and gradual with very small $\chi^{2}$ difference between the two best
fit points. Because the values for $R$ are similar for the two volumes, the
deduced oscillation length is similar to, or smaller, than the volumes’
dimensions. As a result, the acceptable $\Delta m^{2}$ range extends above a
lower limit. As a consequence, it is not well determined and the results are
consistent with values above about 1 eV2. The large deviation of the $R$’s
from 1 drives the mixing angle to a large value within an extended range. This
description is similar to the previous Ga results and hence, given the broad
minimum, the difference in parameter values at the minima points is
inconsequential.

Because the measured $R$’s for the two volumes are similar, an alternative
explanation for the results could be an overall error in $\sigma$ or
efficiency. Since the observed $R$’s would require a smaller $\sigma$ than the
ground state contribution alone, some fundamental misunderstanding of the
nuclear or atomic physics would be necessary for a reduced $\sigma$ to resolve
the Ga anomaly. Given the known 71Ge decay rate, $\sigma$ to the ground state
is assumed to be well determined and the inclusion of excited state
contributions cannot decrease $\sigma$.

An error in the efficiency also cannot be ruled out but the experimental
procedures have been verified extensively over the past two decades. Many
aspects of BEST have been double-checked, including the Ga target masses, the
extraction efficiency, the source strength, the source placement, the counting
efficiency and the counting system operation. No cause for concern was found.

After the BEST measurements the Ga anomaly looks more pronounced; the weighted
average value of the neutrino capture rate relative to the expected value for
all Ga experiments is 0.80$\pm$0.05, accounting for the correlated uncertainty
for $\sigma$. If one ignores the excited state contribution to $\sigma$, the
value would increase to 0.84$\pm$0.04, but still be significantly below 1. The
hypothesis of $\nu_{e}\rightarrow\nu_{s}$ oscillations is consistent with
these results. A future source experiment with a smaller inner volume might be
considered, but the required source strength would be challenging.

We thanks to V. A. Rubakov for the interest, stimulation and fruitful
discussions. This work is supported by former Federal Agency for Scientific
Organizations (FANO till 2019) of Russian Federation, Ministry of Science and
Higher Education of Russian Federation under agreement no. 14.619.21.0009
(unique project identifier no.RFMEFI61917X0009), State Corporation ROSATOM,
and the Office of Nuclear Physics of the US Department of Energy.

## References

  * Abazajian  _et al._ (2012) K. N. Abazajian _et al._ , “Light sterile neutrinos: A white paper,” (2012), arXiv:1204.5379 [hep-ph] .
  * Gariazzo  _et al._ (2015) S. Gariazzo, C. Giunti, M. Laveder, Y. F. Li, and E. M. Zavanin, Journal of Physics G: Nuclear and Particle Physics 43, 033001 (2015).
  * Giunti and Lasserre (2019) C. Giunti and T. Lasserre, Annual Review of Nuclear and Particle Science 69, 163 (2019).
  * Böser  _et al._ (2020) S. Böser, C. Buck, C. Giunti, J. Lesgourgues, L. Ludhova, S. Mertens, A. Schukraft, and M. Wurm, Progress in Particle and Nuclear Physics 111, 103736 (2020).
  * Diaz  _et al._ (2020) A. Diaz, C. Argüelles, G. Collin, J. Conrad, and M. Shaevitz, Physics Reports 884, 1 (2020).
  * Seo (2020) S.-H. Seo, “Review of sterile neutrino experiments,” (2020), arXiv:2001.03349 [hep-ex] .
  * Dasgupta and Kopp (2021) B. Dasgupta and J. Kopp, “Sterile neutrinos,” (2021), arXiv:2106.05913 [hep-ph] .
  * Abdurashitov  _et al._ (2009) J. Abdurashitov _et al._ , Phys. Rev. C 80, 015807 (2009).
  * Kaether  _et al._ (2010) F. Kaether _et al._ , Phys. Lett. B 685, 47 (2010).
  * Hampel and Remsberg (1985) W. Hampel and L. P. Remsberg, Phys. Rev. C 31, 666 (1985).
  * Abdurashitov  _et al._ (1999a) J. N. Abdurashitov _et al._ , Phys. Rev. C 59, 2246 (1999a).
  * Hampel  _et al._ (1998) W. Hampel _et al._ , Physics Letters B 420, 114 (1998).
  * Abdurashitov  _et al._ (2006) J. N. Abdurashitov _et al._ , Phys. Rev. C 73, 045805 (2006).
  * Barinov  _et al._ (2018) V. Barinov _et al._ , Phys. Rev. D 97, 073001 (2018).
  * Frekers  _et al._ (2011) D. Frekers _et al._ , Phys. Lett. B 706, 134 (2011).
  * Frekers  _et al._ (2013) D. Frekers _et al._ , Phys. Lett. B 722, 233 (2013).
  * Frekers  _et al._ (2015) D. Frekers _et al._ , Phys. Rev. C 91, 034608 (2015).
  * Semenov (2020) S. V. Semenov, Phys. At. Nucl. 83, 1549 (2020).
  * Abdurashitov  _et al._ (2002) J. Abdurashitov _et al._ , JETP 95, 181 (2002), Zh.Eksp.Teor.Fiz. 122, 211-226 (2002).
  * Giunti  _et al._ (2012) C. Giunti, M. Laveder, Y. F. Li, Q. Y. Liu, and H. W. Long, Physical Review D 86 (2012), 10.1103/physrevd.86.113014.
  * Kostensalo  _et al._ (2019) J. Kostensalo _et al._ , Phys. Lett. B 795, 542 (2019).
  * Laveder (2007) M. Laveder, Nuclear Physics B - Proceedings Supplements 168, 344 (2007), proceedings of the Neutrino Oscillation Workshop.
  * Kozlova  _et al._ (2020) J. Kozlova, V. Gavrin, T. Ibragimova, V. Tarasov, E. Veretenkin, and A. Zvir, Phys. At. Nucl. 83, 958 (2020), presentation at Session of the Department of Nuclear Physics March 10-12, 2020.
  * Gavrin  _et al._ (2021) V. Gavrin, T. Ibragimova, J. Kozlova, V. Tarasov, E. Veretenkin, and A. Zvir, J. of Instrum. 16, P04012 (2021).
  * Cleveland  _et al._ (2015) B. Cleveland, V. Gavrin, V. Gorbachev, T. Ibragimova, T. Knodel, Y. Malyshkin, I. Mirmov, and E. Veretenkin, International Journal of Mass Spectrometry 392, 41 (2015).
  * Shikhin  _et al._ (2017) A. A. Shikhin _et al._ , J. Phys.: Conf. Ser. 798, 012201 (2017).
  * Shikhin  _et al._ (2018) A. A. Shikhin _et al._ , Phys. Part. Nucl. 49, 799 (2018).
  * Abdurashitov  _et al._ (1999b) J. N. Abdurashitov _et al._ , Phys. Rev. C 60, 055801 (1999b).
  * Elliott (1990) S. R. Elliott, Nucl. Instr. and Meth. A 290, 158 (1990), erratum Nucl. Instr. and Meth. A 297, 536 (1990).
  * Note (1) 1 SNU corresponds to one neutrino capture per second in a target that contains 1036 atoms of the neutrino absorbing isotope, in our case 71Ga.
  * Barinov  _et al._ (2021) V. Barinov _et al._ , “Search for sterile neutrinos in the best experiment,” (2021), citation will be replaced with long paper information.
  * Bahcall (1997) J. Bahcall, Phys. Rev. C 56, 3391 (1997).
  * Haxton (1998) W. Haxton, Phys. Lett. B 431, 110 (1998).
  * Alanssari  _et al._ (2016) M. Alanssari, D. Frekers, T. Eronen, L. Canete, J. Hakala, M. Holl, A. Jokinen, A. Kankainen, J. Koponen, I. Moore, D. Nesterenko, I. Pohjalainen, J. Reinikainen, S. Rinta-Antila, and A. Voss, International Journal of Mass Spectrometry 406, 1 (2016).
  * Fogli  _et al._ (2002) G. L. Fogli, E. Lisi, A. Marrone, D. Montanino, and A. Palazzo, Phys. Rev. D 66, 053010 (2002).
  * Altmann  _et al._ (2005) M. Altmann _et al._ , Phys. Lett. B 616, 174 (2005).


# The Hebrew Bible as Data: Laboratory – Sharing - Experiences

Dirk Roorda∗ ∗∗<EMAIL_ADDRESS>∗Data Archiving and Networked
Services - Royal Netherlands Academy of Arts and Sciences, Anna van Saksenlaan
10; 2593 HT Den Haag, Netherlands ∗The Language Archive - Max Planck Institute
for Psycholinguistics, Wundtlaan 1; 6525 XD Nijmegen, Netherlands

###### Abstract

The systematic study of ancient texts including their production, transmission
and interpretation is greatly aided by the digital methods that started taking
off in the 1970s. But how is that research in turn transmitted to new
generations of researchers? We tell a story of Bible and computer across the
decades and then point out the current challenges: (1) finding a stable data
representation for changing methods of computation; (2) sharing results in
inter- and intra-disciplinary ways, for reproducibility and cross-
fertilization. We report recent developments in meeting these challenges. The
scene is the text database of the Hebrew Bible, constructed by the Eep Talstra
Centre for Bible and Computer (ETCBC), which is still growing in detail and
sophistication. We show how a subtle mix of computational ingredients enable
scholars to research the transmission and interpretation of the Hebrew Bible
in new ways: (1) a standard data format, Linguistic Annotation Framework
(LAF); (2) the methods of scientific computing, made accessible by
(interactive) Python and its associated ecosystem. Additionally, we show how
these efforts have culminated in the construction of a new, publicly
accessible search engine SHEBANQ, where the text of the Hebrew Bible and its
underlying data can be queried in a simple, yet powerful query language MQL,
and where those queries can be saved and shared.

##  1 Introduction

The Hebrew Bible is a collection of ancient texts resulting from a ten-
centuries long tradition. It is one of the most studied texts in human
culture. Information processing by machines is less than two centuries old,
but since its inception its capabilities have evolved in an exponential manner
up till now [Gleick2011]. We are interested in what happens when the Hebrew
Bible as an object of study is brought under the scope of the current methods
of information processing. The Eep Talstra Centre for Bible and Computing
(ETCBC) formerly known as Werkgroep Informatica Vrije Universiteit (WIVU), has
been involved in just this since the 1970s and their members are dedicated to
this approach. The combination of a relatively stable set of data and a
rapidly evolving set of methods urges for reflection. Add to that a growing
set of ambitious research questions, and it becomes clear that not only
reflection is needed but also action. Methods from computational linguistics
and the wider digital humanities are to be used, hence people from different
disciplines have to be involved. How can the ETCBC share its data and way of
working productively with people that are used to a wide variety of
computational ways?

In this article we tell a story of reflection and action, and the characters
are databases, data formats, query languages, annotations, computer languages,
archives, repositories and social media. This story has a beginning in
February 2012, when a group of biblical scholars convened at the Lorentz
center at Leiden for the workshop Biblical Scholarship and Humanities
Computing: Data Types, Text, Language and Interpretation [Roorda2012c]. They
searched for new ways to obtain computational tools that matched their
research interests. The author was part of that meeting and had prepared a
demo application: a query saver. It was an attempt to improve the sharing of
knowledge. It is a craft to write successful queries for the ETCBC Hebrew Text
database, and by publishing their queries, researchers might teach each other
how to do it.

In the years that followed, this idea has materialized as the result of the
SHEBANQ project (System for HEBrew text: ANnotations for Queries and markup),
a curation and demonstrator project funded by CLARIN-NL, the Dutch department
of the Common LAnguage Resource INfrastructure in Europe http://www.clarin.eu.
We have chosen a modern standard format for the data: Linguistic Annotation
Framework (LAF), and have built a web-application for saving queries. During
the execution of this project we also have built LAF-Fabric, a tool to analyze
and manipulate LAF resources. Now, in 2014, we can say that we have a modern
data laboratory for historico-linguistic data, plus ways to share results, not
only among a small circle of theological experts, but also among computational
linguists on the one hand and students and interested lay people on the other.

Of course, every beginning of such a story is arbitrary. There is always so
much more that happened before. In order to provide the reader with enough
context, we shall also relate key moments of that greater story. Moreover, we
cannot tell the whole story: our perspective is biased to the computational
side. We shall not delve into the intricacies of manuscript research, but
focus on the data models and computational methods that help analyze a rather
fixed body of transcribed text. Yet, we believe that this simplified context
is rich enough material for a good story. Whereas this paper deliberately
scratches only the surface of the computational methods, there is also a joint
paper with researchers, which contains a more technical account in
[Roorda2014a].

##  2 Ground work: WIVU and ETCBC

Since the 1970s, Eep Talstra, Constantijn Sikkel and a group of researchers at
the VU University Amsterdam have been compiling a text database of the Hebrew
Bible. This database started as a set of files, containing the transliterated
Hebrew text of the Bible according to the Biblia Hebraica Stuttgartensia
edition [BHS]. To this text, they added files with their observations of
linguistic patterns in it as coded annotations, anchored to the individual
words, phrases, clauses, and sentences. They tested tentative patterns against
the data, refined them, and added manual exceptions. This led to a complex web
of files, containing the base text and a set of semi-automatically generated
annotations. They refrained from shaping these annotations in a hierarchical,
linguistic model, because they wanted to represent observations, not theory
[Talstra2000]. The result of this work is a database in the sense of being
observational data on which theories can be based. It is not a database in the
sense of a modern relational database system.

The advantages of a proper (in the sense of computer science) database are
obvious indeed, but the relational model does not represent textual data in a
natural way, and does not facilitate queries that are linguistically
meaningful. In the 1990s there have been promising efforts to define the
notion of a text database. In his Ph.D. thesis, Crist-Jan Doedens
[Doedens1994] defined a data model for texts and the notion of a topographic
query language (QL) to retrieve linguistic results. He identified the
relations of sequence and embedding as the key structures to store and
retrieve texts. A query is topographic if its internal structure exhibits the
same sequence and embedding relations as the results it is meant to retrieve.
Interestingly, he did not postulate that a text is one hierarchy. In his data
model, textual data may be organized by means of multiple, overlapping
hierarchies.

The definition of a data model and a query language are not yet a working
database system. In the 2000s, Ulrik Petersen undertook to create an
implementation of Doedens’s ideas. This led to the Emdros database system with
the MQL (Mini-QL) query language Petersen2004
Petersen2004,Petersen2006,Petersen2002. Emdros consists of a front-end, which
is an MQL interpreter, and a back-end, which is an existing production class
relational database system such as Postgres or MySQL. Despite the fact that
MQL is a concession to practicality, it is still a topographic query language
and very convenient to express real-life textual queries without invoking
programming skills. Since then, an Emdros export of the current Hebrew text
database is being maintained by the ETCBC team. Emdros is open source
software, the data model is very clear, so this export is a communication
device: the intricacies of the internal annotation-creation of the ETCBC
workflow are largely left behind, and users of the export have a well-defined
dataset at their disposal.

##  3 Idea: Queries As Annotations

During the aforementioned Lorentz workshop [Roorda2012c], an international
group of experts reflected on how to bring biblical data resources to better
fruition in the digital age. The ETCBC database had been incorporated in Bible
study software, but developments there were not being driven by agendas set by
academic research. Yet those bible study applications offered attractive
interfaces to browse the text, look up words and more. The problem was: how
can theologians, with limited ICT resources, regain control over the
development of software that works with their data? The workshop offered no
concrete solutions, but some ingredients of potential long-term solutions did
get mentioned: open up the data and develop open source tools. Theologians can
only hope to keep up with ICT developments if they allow people building on
each others’ accomplishments. A very concrete articulation of this statement
was made by Eep Talstra himself, when he deposited the ETCBC database into
EASY, the research archive of DANS [Talstra2012]. It must be admitted that
there remained barriers: the data was not Open Access and the format in which
it was deposited was MQL, which is not a very well-known format, so the
experimenting theological programmer still has a hard time to do some
meaningful work with this data. But it was definitely a step towards increased
sharing of resources.

Figure 1: Queries/Features as Annotations

In that same workshop, the author showed a demo application [Roorda2012b] (see
Figure 1) by which the user could browse the Hebrew text and highlight a
number of linguistic features. The idea to highlight features, which are
essentially annotations to the text, triggered another idea: to view queries
as annotations to the passages that contain their results [Roorda2012a]. If
researchers can save their carefully crafted queries as annotations, and if
those annotations are centrally stored, then other researchers have access to
them and may encounter them when they are reading a passage. Just as readers
encounter ordinary annotations by other scholars in printed books, they will
encounter results of queries of others when they are browsing a chapter of the
Hebrew Bible in their web browser. With a single click they are led to not
only the query instruction itself but also a description of the provenance and
motivation of the query. This could be the basis of interesting scenarios for
cross-fertilization.

It is interesting to note the stack of computational tools needed to write
this demo. Its construction involved a data preparation tool for transforming
the contents of the ETCBC database into a relational database for driving a
website. The web app itself was based on web2py, a lightweight python based
web-application framework [Pierro].

Table 1 is a list of languages used to implement both the data-preparation
tool and the web-site, together with the amount of code needed in each
formalism. There are several things to note:

  1. 1.

The numbers of lines of code are very small.

  2. 2.

The formalisms, while considerable in number, are utterly commonplace.

  3. 3.

The number of formalisms may be reduced by one by dropping Perl in favor of
Python

It can be concluded that mastering commonplace ICT techniques may generate a
good return on investment, in the form of a web application that expose data
on the web in rich interfaces.

formalism | web app | data prep tool
---|---|---
sql | 90 | 80
python | 250 |
perl |  | 650
javascript | 300 |
html | 50 |
css | 60 |
shell script |  | 280
Table 1: Amount of lines of code per formalism per application

##  4 Realization: LAF-Fabric and SHEBANQ

In 2013-2014, ETCBC together with DANS has carried out the CLARIN-NL project
SHEBANQ. We seized the opportunity to implement the idea of queries-as-
annotations, but to make it possible at all more work had to be done.

###  4.1 LAF and LAF-Fabric

First of all, a new representation of the data had to be selected, one that
conformed to a standard used in linguistics. Linguistic Annotation Framework,
an ISO standard [Ide2012], was chosen. LAF defines a data model in which an
immutable stream of primary data is annotated by feature structures. The data
stream is addressed by means of a graph of nodes and edges, where the nodes
may be linked to regions of the primary data, and where edges serve to connect
smaller parts to bigger wholes. Both nodes and edges can act as targets of
annotations, which contain the feature structures. Finally, all entities,
except the primary data, are serialized in XML.

In concrete terms, we have extracted the complete text of the Hebrew Bible as
a plain Unicode text file. As far as LAF is concerned, this is our primary
data. For the books, chapters and verses we have created nodes that are linked
to the stretches of text that they correspond to. For every individual word
there is a node, linked to a region defined by the character positions of the
first and last character of that word. For the phrases, clauses and sentences
there are nodes, linked to the regions corresponding to the words they
contain. Relationships between constituents correspond to edges. The
properties of sectional units, words, and constituents are key-value pairs
targeted at the corresponding nodes.

The LAF data model shares a lot of structure with the Emdros data model of
text, objects and features. We only had to map objects to nodes and features
to key-value pairs inside annotations targeting the proper nodes, so this
conversion has been a straightforward process with only a few devilish
details.

The result is a good example of stand-off markup. The primary data is left
untouched, and around it is a graph of annotations. It is perfectly possible
to add new annotations without interfering with the primary data or the other
annotations. The annotations are like a fabric, into which new threads can be
woven, and that can be stitched to other fabrics. In this way, the stand-off
way of adding information to sources facilitates cooperation and sharing much
better than adding markup inline, such as TEI prescribes. This bold assertion
must be qualified by two considerations, however:

  1. 1.

Stand-off markup works best in those cases where the primary sources are
immutable. As easy as it is to add new annotations, so difficult it is to
insert new primary data.

  2. 2.

Stand-off markup flourishes in cases where the main access mode to the sources
is by programmatic means. Manual inspection of stand-off data and their
annotations becomes quickly overwhelming.

In our case, condition 1 is satisfied for years in a row. How we will deal
with major updates remains to be seen.

Table 2 indicates some quantities of the ETCBC data, both in their Emdros form
and in their LAF form. These numbers suggest that manual inspection of
individual files is so cumbersome that it pays off to invest in programmatic
access of the data.

quantity | Emdros | LAF
---|---|---
words | 426,555 | 426,555
linguistic objects resp. nodes | 945,726 | 945,726
total number of features | 22,622,100 | 25,504,388
serialized size (MQL resp. XML) | 455 MB in 1 file | 1640 MB in 14 files
compiled size (SQLite3 resp. binary) | 126 MB | 260 MB
Table 2: Quantities in the ETCBC data

The LAF version of the Hebrew text database has been archived at Data
Archiving and Networked Services (DANS), the research archive for the
humanities and social sciences in the Netherlands [Peursen2014a].

As LAF is a relative new standard, there are few LAF-compatible tools. A LAF
resource is represented in XML, but the nature and size of this XML make it
difficult to be handled by ordinary XML tools. Looking through the surface
syntax, a LAF resource is neither a relational database, nor a document, but a
graph. XML processing works well when the underlying data structure is a
single hierarchy, no matter how deep, or a table of records, no matter how
large, but it grinds to a halt when the data is a large and intricate web of
nodes and edges, i.e. a graph.

In order to facilitate productive work with the freshly created LAF
representation of the Hebrew Bible, we have developed LAF-Fabric [Roorda2013],
which is a LAF compiler and loader. In a typical workflow, a researcher wants
to inspect the LAF data, focus on some aspects, sort, collate, link and
transform selected data, and finally export results. Without LAF-Fabric, the
obvious way to do so is read the XML data, apply XPATH, XSLT or XQUERY scripts
and collect the results. Reading the XML data means parsing it and building an
internal representation in memory, and this alone takes an annoying 15 minutes
on a average laptop and uses a prohibitive amount of memory. This is not
conducive to an interactive, explorative, agile use of the data, and LAF-
Fabric remedies this. When first invoked on a LAF-resource, it compiles it
into efficient data structures and writes those to disk, in such a way that
this data can be loaded fast. This one-time compilation process takes roughly
15 minutes, but then the data loads in a matter of seconds every time you want
to work with it. Furthermore, LAF-Fabric offers a programmers interface (API)
to the LAF data, by which the programmer can walk over the nodes and edges and
collect feature information on the fly. These walks are fast, and can be
programmed easily.

The idea to create LAF-Fabric arose after we tried to use a library called
graf-python [Bouda2013], part of POIO [Bouda2012], for the biblical LAF data.
Unfortunately, the way graf-python was programmed made it unsuitable for
dealing with our LAF resource because of its size. Python is a scripting
language with a clean syntax and a good performance if used judiciously, hence
we undertook to write LAF-Fabric in Python as well. We use those parts of
Python that perform best for the heavy data lifting, and those parts that are
most user friendly for the programmers interface. LAF-Fabric is a package that
can be imported in any Python script, and it behaves particularly well when
invoked in an IPython notebook.

IPython Notebook is an interactive way of writing Python scripts and
documentation [Perez2007]. A notebook is a document in which the data analyst
writes cells with Python code and other cells with documentation. Code cells
can be run individually, in any order, while the results of the execution
remain in memory. The notebook has powerful capabilities of formatting
results. A notebook can be published easily on the web, so that others can
download it and execute it as well, provided they have the same data and
packages installed. IPython notebook belongs to a branch of computer
programming called scientific computing. It is about explorative data analysis
by means of computing power. The scientific programmer produces analyses,
charts, and documents that account for his data and results. By contrast, the
typical software engineer produces applications that perform well-defined
tasks for end users. The scientific programmer works close to the researchers,
and writes special purpose code fast, and reacts to changing demands in an
agile way. The software engineer works at a greater distance from the actual
use cases. He uses programming languages that support good software
organization at the cost of a much slower development process. He is less
prepared to accomodate fast-changing requirements.

When LAF-Fabric runs in an IPython notebook, even the few seconds it needs to
load data are required only once. The programmer can experiment with his code
cells at will, without the need to reload the data all the time.

LAF-Fabric has already been used for some significant data extractions. There
is a varied and growing set of notebooks [Roorda2014b] on Github that is
testimony to the extent of use cases that can be served. Not only data
analysis, but also adding new annotations is supported. One of the use cases
is the query saver itself.

###  4.2 SHEBANQ

The actual goal of the SHEBANQ project was to create a demonstrator query
saver for the ETCBC data. This has been achieved, and the resulting web
application is called SHEBANQ [Peursen2014b].

Figure 2: A saved query in SHEBANQ

It went live on 2014-08-01, and contains now, on 2015-01-06, 309 public
queries, saved by 42 users. The public part of the application offers users
the options to read the Hebrew Bible chapter by chapter, to see query results
of public queries as annotations in the margin, and to jump from query
annotations to query descriptions and result lists. Figure 2 shows a
screenshot of the page of a saved query. No matter how many query results
there are, the user is able so navigate through them all, as can be seen in
Figure 3.

Figure 3: Navigating through results of the query for the definite article
(12,835 result verses)

When a user clicks on the verse indicator of a result, he is led to the
browsing interface, where other queries show up and can be navigated to, see
Figure 4.

Figure 4: Reading a passage and seeing the results of various queries.

When users register and log in, they can write their own queries, have them
executed, save them, including the query results, and make them public. In
order to execute MQL queries, SHEBANQ communicates with a web-service that is
wrapped around the Emdros text database.

While the underlying idea of SHEBANQ is straightforward, turning it into
practice posed several challenges. To begin with, the data had to be modeled
in a way suitable for driving web applications. We have programmed a MySQL
export in a notebook, invoking LAF-Fabric. Every now and then the
functionality of SHEBANQ is extended. For example, it can now show additional
linguistic information in layers below the plain text. We first experimented
in LAF-Fabric by generating HTML for a visual prototype, then we collectd
feedback, and adapted our notebook. We ran consistency checks whenever we
wanted to make use of perceived regularities in the data. After everything had
crystallized out satisfactorily, we built the new data representation into
SHEBANQ, see Figure 5. The fact that both the notebook and the SHEBANQ website
are written in Python turned out very convenient.

Figure 5: Text and underlying data

Rendering the Hebrew text turned out to be a problem because of subtle bugs in
some platform/browser combinations. On Mac OSX, Chrome and Safari mangled
whitespace between particular combinations of consonants and diacritics. We
have explored the problems using LAF-Fabric and found a mitigating work-
around.

If the demonstrator shows one thing, then it is the fact that there are many
additional desiderata. Whereas SHEBANQ has been designed to fulfill the
sharing function, researchers also want to use it as a research tool. It is
not easy to write a good MQL query, because many of the linguistic aspects of
the data are not shown on the interface. If, for instance, a user wants to use
the dictionary entries of the words or the syntactic features of clauses and
phrases, he has no immediate, visual clues. So SHEBANQ has been extended
again. The user can now click on any word in the text for easy access to
lexical information.

Other users see SHEBANQ as a preprocessor tool: they need data exports of
query results. The next iteration of SHEBANQ is planned to deliver that.

Another matter is usability: the number of queries is becoming too large to
display them all in the margin. Users want to be able to filter the queries
they see on the basis of who wrote them, both in the browsing interface and in
the list of public queries. Last-but-not-least, query execution is CPU-hungry.
We have already started thinking about measures to prevent excessive processor
loads, or ways to distribute the load over multiple servers.

##  5 Reflection

Back in 2012 we faced the challenge to provide better data models and better
programs for biblical scholars. It had become clear that the software
companies that were developing the bible study applications were not
interested in building software for researchers. The researchers did not have
funds to hire programmers themselves. There seemed to be only one way out:
researchers should take their fate in their own hands and write the software
themselves, which looked like a daunting proposition at best and an impossible
one at worst. Yet, now in 2014, we have a publicly accessible tool for
querying the linguistic data of the Hebrew Bible, with a means to share those
queries. We also have a data laboratory where the programming theologian can
take control over her data. Collectively, biblical scholars can use the data
laboratory to help the query tool evolve according to their needs. Several
factors have contributed to this achievement.

  1. 1.

The existence of the LAF standard, which turned out to be a natural fit for
this kind of data.

  2. 2.

The realization that the plain text of the Hebrew Bible is not subject to
copyright, and hence that the ETCBC database of text and annotations can be
made available as Open Source.

  3. 3.

The existence of a research archive, DANS, acting as a data-hub; the
intellectual heritage of many years of ETCBC work lays deposited there and is
open to scrutiny by anyone at any time.

  4. 4.

The existence of a social medium for program code, Github; all software for
LAF-Fabric and SHEBANQ (and even some of the supporting software) lies there
ready to be cloned and re-used.

  5. 5.

The rise of scientific computing and its paraphernalia, such as (interactive)
Python and auxiliary packages; it offers an unprecedented level of user-
friendliness to novice programmers; it has the potential to draw a much wider
range of humanities scholars into the enticing world of computing. A
researcher is much closer to a scientific programmer than to a software
engineer.

Yet, this is not sufficient to get the job done. The ETCBC is steeped in its
own ways, it has an efficient internal data workflow, run with the best tools
that were available in the late 1980s. The internet existed then, but had not
yet morphed into the world-wide web. Data sharing is not in the genes of the
ETCBC. Doing unique things in relative isolation for a prolonged stretch of
time tends to make you idiosyncratic. The ETCBC has its own transliteration of
Hebrew, its own, locally documented way of coding data into forms that are
optimal for local data processing.

Opening up to the world poses new requirements on the ways the data is coded
and how it is documented. While we have archived the existing ETCBC
documentation at DANS, we started publishing a new kind of feature
documentation on the web [Roorda2014c]. There we document not only the
intended meaning of features, but we also provide frequency lists of their
complete value sets, things that are easily computed by means of LAF-Fabric.

Can we say that we have succeeded in meeting the challenges posed in 2012? It
is too early for that. Proof of success would be the adoption of LAF-Fabric by
at least some theological researchers, interest in the Hebrew data from the
side of computational linguistics and artificial intelligence, and large
access log files of the SHEBANQ web application.

At the moment of writing, all these indicators are non-zero [Roorda2014a],
Kalkman2013 Kalkman2013,Kalkman2015, which is promising, given the fact that
we just started.

## Acknowledgements

The following people contributed significantly to the work described in this
paper, in very different ways:

My colleagues at DANS: Henk Harmsen and Andrea Scharnhorst for granting
additional time for research in these topics; Henk van den Berg and Heleen van
de Schraaf for developing fundamental portions of the web-service and web
application of SHEBANQ.

The ETCBC people: Eep Talstra and Constantijn Sikkel for creating and
maintaining the ETCBC database and sharing so much knowledge about it; Wido
van Peursen for paving the way for increased sharing of data. Oliver Glanz for
showing order in the forest of features of the database; Gino Kalkman and
Martijn Naaijer for using the new tools and challenging me. Grietje en Johan
Commelin for their efforts to get LAF-Fabric working in the cloud. Reinoud
Oosting for finding subtle bugs in SHEBANQ and Janet Dyk introducing new use
cases.

Wider Digital Humanities: Joris van Zundert (HuygensING) for leading an
inspiring Interedition bootcamp [Zundert2012] which set me on the track of
rapid development for the humanities; Rens Bod (Univ. of Amsterdam) and
Andreas van Cranenburgh (Univ. of Amsterdam and HuygensING) who asked for the
Hebrew data as tree structures in order to try out Data Oriented Parsing for
classical Hebrew.

## References

  * [Bouda]Bouda2013-2014Bouda2013Bouda, Peter (2013-2014), graf-python.  Python software on Github.  https://github.com/cidles/graf-python. [Bouda et al.]Bouda et al.2012Bouda2012Bouda, Peter, Vera Ferreira, and António Lopes (2012), Poio API - an annotation framework to bridge language documentation and natural language processing., Proceedings of The Second Workshop on Annotation of Corpora for Research in the Humanities, Lisbon, 2012, Lisbon, Portugal.  ISBN: 978-989-689-273-9, http://alfclul.clul.ul.pt/crpc/acrh2/ACRH-2_papers/Bouda-Ferreira-Lopes.pdf. [Di Pierro]Di Pierro2015PierroDi Pierro, Massimo (2015), Web2py. full stack web framework, 6th edition.  Online book.  http://web2py.com/book. [Doedens]Doedens1994Doedens1994Doedens, Crist-Jan (1994), Text Databases. One Database Model and Several Retrieval Languages, number 14 in Language and Computers, Editions Rodopi, Amsterdam, Netherlands and Atlanta, USA.  ISBN: 90-5183-729-1, http://books.google.nl/books?id=9ggOBRz1dO4C. [Gleick]Gleick2011Gleick2011Gleick, James, editor (2011), The Information: a History, a Theory, a Flood, HarperCollins.  ISBN: 0007225741, http://en.wikipedia.org/wiki/The_Information:_A_History,_a_Theory,_a_Flood. [Ide and Romary]Ide and Romary2012Ide2012Ide, Nancy and Laurent Romary (2012), Linguistic Annotation Framework.  ISO standard 24612:2012. Edition 1, 2012-06-15.  http://www.iso.org/iso/home/store/catalogue_tc/catalogue_detail.htm?csnumber=37326. [Kalkman]Kalkman2013Kalkman2013Kalkman, Gino J. (2013), Functions of asyndetic clause relations in biblical Hebrew.  IPython Notebook.  http://nbviewer.ipython.org/github/ETCBC/Biblical_Hebrew_Analysis/blob/master/Miscellaneous/AsyndeticClauseFunctions.ipynb. [Kalkman]Kalkmanto be published 2015Kalkman2015Kalkman, Gino J. (to be published 2015), Verbal Forms in Biblical Hebrew Poetry: Poetical Freedom or Linguistic System?, PhD thesis, VU University, Amsterdam.  http://nbviewer.ipython.org/github/ETCBC/Biblical_Hebrew_Analysis/blob/master/PhD/Introduction.ipynb. [Kittel]Kittel1968-1997BHSKittel, Rudolf, editor (1968-1997), Biblia Hebraica Stuttgartensia, Deutsche Bibelgesellschaft, Stuttgart, Germany.  http://www.bibelwissenschaft.de/startseite/wissenschaftliche-bibelausgaben/biblia-hebraica/bhs/. [Pérez and Granger]Pérez and Granger2007Perez2007Pérez, Fernando and Brian E. Granger (2007), IPython: a system for interactive scientific computing, Computing in Science and Engineering 9 (3), pp. 21–29, IEEE Computer Society.  http://ipython.org, ISSN: 1521-9615, DOI: 10.1109/MCSE.2007.53. [Petersen]Petersen2002-2014Petersen2002Petersen, Ulrik (2002-2014), Emdros. text database engine for analyzed or annotated text.  Open Source software.  http://emdros.org. [Petersen]Petersen2004Petersen2004Petersen, Ulrik (2004), Emdros - a text database engine for analyzed or annotated text, Proceedings of COLING 2004, p. 1190–1193.  http://emdros.org/petersen-emdros-COLING-2004.pdf. [Petersen]Petersen2006Petersen2006Petersen, Ulrik (2006), Principles, Implementation Strategies, and Evaluation of a Corpus Query System, Vol. 4002, Springer, p. 215–226.  http://link.springer.com/chapter/10.1007%2F11780885_21. [Peursen and Roorda]Peursen and Roorda2014Peursen2014aPeursen, Wido Th. van and Dirk Roorda (2014), Hebrew text database in Linguistic Annotation Framework.  Dataset available online at Data Archiving and Networked services, Den Haag, Netherlands.  PID: urn:nbn:nl:ui:13-048i-71, http://www.persistent-identifier.nl/?identifier=urn:nbn:nl:ui:13-048i-71. [Roorda]Roorda2012Roorda2012bRoorda, Dirk (2012), Queries-as-annotations.  Github repository.  https://github.com/Dans-labs/annotation-paradigm. [Roorda]Roorda2013-2014bRoorda2013Roorda, Dirk (2013-2014b), LAF-Fabric. workbench for analysing LAF resources.  Python software on Github.  https://github.com/ETCBC/laf-fabric. [Roorda]Roorda2014aRoorda2014bRoorda, Dirk (2014a), LAF-Fabric notebooks. examples of data processing for the hebrew bible.  Python software on Github.  https://github.com/ETCBC/laf-fabric-nbs. [Roorda and van den Heuvel]Roorda and van den Heuvel2012Roorda2012aRoorda, Dirk and Charles M.J.M. van den Heuvel (2012), Annotation as a new paradigm in research archiving, Proceedings of ASIS&T 2012 Annual Meeting. Final Papers, Panels and Posters.  https://www.asis.org/asist2012/proceedings/Submissions/84.pdf (author’s version: http://arxiv.org/abs/1412.6069). [Roorda et al.]Roorda et al.2014Roorda2014cRoorda, Dirk, Constantijn Sikkel, and Wido Th van Peursen (2014), Feature documentation of shebanq.  Github repository published on readthedocs.  https://github.com/ETCBC/shebanq-doc. [Roorda et al.]Roorda et al.2012Roorda2012cRoorda, Dirk, Jan Krans, Bert-Jan Lietaert-Peerbolte, Wido Th. van Peursen, Ulrik Sandborg-Petersen, and Eep Talstra (2012), Scientific report of the workshop biblical scholarship and humanities computing: Data types, text, language and interpretation, held at the lorentz centre leiden from 6 feb 2012 through 10 feb 2012, Technical report, Lorentz Center, Leiden.  http://www.lorentzcenter.nl/lc/web/2012/480/report.php3?wsid=480&venue=Oort. [Roorda et al.]Roorda et al.to appear 2015Roorda2014aRoorda, Dirk, Martijn Naaijer, Gino J. Kalkman, and Andreas van Cranenburgh (to appear 2015), Laf-fabric: a data analysis tool for linguistic annotation framework with an application to the hebrew bible, Computational Linguistics in the Netherlands Journal.  ISSN: 2211-4009, preprint http://arxiv.org/abs/1410.0286. [Talstra and Sikkel]Talstra and Sikkel2000Talstra2000Talstra, Eep and Constantijn J. Sikkel (2000), Genese und kategorienentwicklung der wivu-datenbank, in Hardmeier, C. et al, editor, Ad Fontes! Quellen erfassen – lesen – deuten. Wat ist Computerphilologie? Ansatzpunkte und Methodologie – Instrument und Praxis, VU University Press, Amsterdam, Netherlands, pp. 33–68. [Talstra et al.]Talstra et al.2012Talstra2012Talstra, Eep, Constantijn J. Sikkel, Oliver Glanz, Reinoud Oosting, and Janet W. Dyk (2012), Text database of the hebrew bible.  Dataset available online after permission of the depositor at Data Archiving and Networked services, Den Haag, Netherlands.  http://www.persistent-identifier.nl/?identifier=urn:nbn:nl:ui:13-ukhm-eb. [van Peursen et al.]van Peursen et al.2014Peursen2014bvan Peursen, Wido Th., Dirk Roorda, Henk van den Berg, and Heleen van de Schraaf (2014), SHEBANQ, a search engine and query saver for the etcbc text database of the hebrew bible.  web application.  http://shebanq.ancient-data.org. [van Zundert]van Zundert2012Zundert2012van Zundert, Joris et al. (2012), Proceedings of the leuven2012 interedition workshop.  Wiki page.  http://interedition.eu/wiki/index.php/Leuven2012_Proceedings.
  * [1]