∎

e1e-mail<EMAIL_ADDRESS>e2e-mail<EMAIL_ADDRESS><EMAIL_ADDRESS>
11institutetext: Dept. Atomic, Molecular and Nuclear Physics, University of
Granada, Spain 22institutetext: Technical University of Munich, TUM School of
Natural Sciences, Physics Department, James-Franck-Str. 1, 85748 Garching,
Germany 33institutetext: Cluster of Excellence PRISMA+, Johannes Gutenberg
University Mainz, Staudingerweg 9, 55128 Mainz, Germany 44institutetext:
Istituto Nazionale di Fisica Nucleare, Legnaro Division, viale dell’Università
2, 35020 Legnaro, Italy 55institutetext: Istituto Nazionale di Fisica
Nucleare, Padova Division, via Marzolo 8, 35131 Padova, Italy,
66institutetext: University of Cassino and Southern Lazio, Dipartimento di
Ingegneria Civile e Meccanica, via G. Di Biasio 43, 03043 Cassino (FR), Italy
77institutetext: University of Padova, Dept. of Physics and Astronomy, via
Marzolo 8, 35131 Padova, Italy

# Fast neutron production at the LNL Tandem from the 7Li(14N,xn)X reaction

Pablo Torres-Sáncheze1,addr1  Hans Th. J. Steigere2,addr2, addr3
Pierfrancesco Mastinuaddr4  Jeffery L. Wyssaddr5,addr6  Lennard Kayseraddr2
Luca Silvestrinaddr5,addr7  Elizabeth Musacchio-Gonzálezaddr4  Matthias
Raphael Stockaddr2  David Dörflingeraddr2  Ulrike Fahrendholzaddr2  Gianfranco
Preteaddr7  Osvaldo Carlettoaddr4  Lothar Oberaueraddr2  Ignacio Porrasaddr1

(Received: date / Accepted: date)

###### Abstract

Fast neutron beams are of relevance for many scientific and industrial
applications. This paper explores fast neutron production using a TANDEM
accelerator at the Legnaro National Laboratories, via an energetic ion beam
(90 MeV ${}^{14}N$) onto a lithium target. The high energy models for nuclear
collision of FLUKA foresee large neutron yields for reactions of this kind.
The experiment aimed at validating the expected neutron yields from FLUKA
simulations, using two separate and independent set-ups: one based on the
multi-foil activation technique, and the other on the time of flight
technique, by using liquid scintillator detectors.

The results of the experiment show clear agreement of the measured spectra
with the FLUKA simulations, both in the shape and the magnitude of the neutron
flux at the measured positions. The neutron spectrum is centered around the 8
MeV range with mild tails, and a maximum neutron energy spanning up to 50 MeV.

These advantageous results provide a starting point in the development of fast
neutron beams based on high energy ion beams from medium-sized accelerator
facilities.

###### Keywords:

Neutron production nitrogen ion beam multifoil activation time of flight FLUKA
simulation

††journal: Eur. Phys. J. C

##  1 Introduction

Neutron beams have important applications in many different fields. Most
neutron beams are available at reactors and have cold or thermal spectra; such
neutrons are uniquely suited to the study of the structure and dynamics of
materials at the atomic level. On the other hand, few facilities can provide
fast neutron beams ($E_{n}>1$ MeV); the access is limited and costs high.

Fast neutron sources are of importance not only for basic nuclear physics. The
list of other fields of applications includes: dosimetry, neutron detector
development, fast neutron oncology, radiation protection shielding materials
for accelerator based oncology and Space missions, and the study of single-
neutron induced effects in digital and power electronics.

Tandem accelerators are still widely used in many small-medium size
laboratories and could provide cost effective neutrons on existing beam lines.
In addition to the direct applications to fundamental and applied physics
mentioned above, important is also the possibility of training students in
neutron physics.

##  2 Fast neutrons at the LNL Tandem

The Legnaro National Laboratories (LNL) of the Italian National Institute of
Nuclear Physics is developing new sources of fast neutrons that would
integrate the existing one at the CN Van de Graaff 6 MV accelerator and the
one under design at the SPES 70 MeV proton cyclotron mastinu2020fast
Silvestrin2016 . To produce fast neutrons, we studied reactions involving
heavy ion beams on light targets, as inverse kinematics can produce forwardly
focused high energy neutrons in the laboratory reference frame.

Energetic ions beams are available at the Tandem-PIAVE-ALPI accelerator
complex SIGNORINI198627 . The experiment described below used the maximum
energy available at the Tandem XTU accelerator; a followup experiment will use
the PIAVE-ALPI heavy ion injector and superconducting linac to deliver higher
energy beams.

As a solid target material, the lightest possible elements to be considered
are lithium or beryllium. With regards to the beam and the performance of the
Tandem, a balance between ion mass and available current gives nitrogen as the
best compromise. The initial target choice is lithium, routinely used at the
BELINA neutron facility at the CN accelerator; beryllium will soon be
evaluated too.

###  2.1 The experimental reaction

The selected reaction 7Li(14N,xn)X reaction has a large $Q$-value (16741.137
keV); the kinematics, therefore, allows the production of fast neutrons with
energies of tens of MeV, even at ion beam energies of less than 10 MeV/u. The
cross section and yield for this reaction are not available in literature. We
used the Monte Carlo code FLUKA to study the neutron production spectra; the
calculated FLUKA neutron spectra are based on theoretical models since no data
are available for the reaction.

Two independent measurement methods were used to measure the energy spectrum
and validate the results of the simulations: the multi-activation foil
technique and time-of-flight (TOF).

The TOF part of this experiment is actually part of a broad research program
for characterizing Liquid Scintillators developed for Neutrino Physics and
Rare Event Astrophysics. In general, a detailed understanding of neutron
signals in detectors is crucial to construct realistic simulations and
predictions of natural phenomena; this is the immediate application of this
new neutron source at LNL.

###  2.2 Fluka Simulations

Monte Carlo simulations of the nuclear reaction were performed using the
models available in FLUKA Fluka . Evaporation including heavy ion fragments
and coalescence hadronic models were activated in the simulations.

Figure 1 shows the expected neutron yield at zero and 10 degrees,
corresponding to the two positions where the fluence was measured. The
position of the activation foil was tilted 10$\degree$ from the beam axis and
was located 2.5 cm from the lithium target, covering a solid angle of 0.193
sr. The position of the TOF detector was located 4.56 m away from the lithium
target in the forward (0∘) beam direction, covering a solid angle of
$0.235\times 10^{-3}$ sr. The estimated yield at the activation position is
lower than the one at the TOF detector position, confirming the strong forward
angle dependence of the neutron production, typical of inverse kinematics
reactions.

Figure 1: FLUKA simulations of the neutron production at the two measurement
positions. The integral yield towards the TOF detector in forward direction
(blue distribution) is 1.6 times larger than that towards the activation foils
at 10° (red).

###  2.3 The beam

Negative BN molecules were injected into the Tandem, accelerated by a 13.7 MV
terminal voltage, and dissociated by one carbon stripping foil to produce
positive N ions that were then further accelerated. Magnetic Analysis selected
14N+6 90 MeV ions that were then transported along the 30∘ beam line in Hall 1
(Figure 2) towards the lithium neutron production target.

Figure 2: The 30∘ beam line in Hall 1. The lithium target is encapsulated into
a copper backing and beam stopper visible at the extreme right; a close-up is
shown below in Figure 5

The beam was pulsed (800 ns repetition rate and 1 ns sigma pulse time width)
in order to perform TOF measurements, thereby providing a cross-check of the
simultaneous multi-foil activation measurement.

###  2.4 The neutron production target

Figure 3: Lithium pressed into a 300 $\mu$m thick copper backing and beam
stopper. The production procedure, carried out inside an argon glovebox, is
described in Section 2.4

The target uses metal lithium encapsulated into a copper backing and beam
stopper designed to have low mass and thus a low neutron spectra
perturbability in forward direction.

The target system is produced in an inert atmosphere inside an argon glovebox.
The lithium is attached to the 300 µm thick Cu backing by pressing them
between a male and a female mold, producing a lithium layer of the desired
thickness, in this case, 200 $\mu$m (Figure 3). This technique is used
regularly to make thin lithium targets used to produce neutrons at the CN
accelerator. The copper backing has a hemispherical shape (see Figure 3)
designed to minimize the material, and it is electrically insulated from the
beam line. The current at the target is acquired directly from the copper
backing. A two-stage tantalum collimator of 5 mm diameter is placed 10 cm
upstream of the copper backing and the current delivered on it was also
acquired.

The total charge accumulated during the experiment amounted to 0.153 mC.
Figure 4 shows the time-dependent ion current during the irradiation.

Figure 4: Irradiation history of the measurement.

###  2.5 The multi-foil activation technique

The activation technique is a well-established and very powerful method that
can be used to measure neutron spectra. The main goal with this setup was to
obtain broad spectral information including the low energy range below few MeV
and specially to provide an absolute value for the yield at the measured
position.

####  2.5.1 Experimental setup

For the multi-foil activation measurement, a series of samples especially
sensitive to neutrons in the energy range of interest were selected(i.e., with
large cross sections or with thresholds ranging from hundreds of keV to tens
of MeV). A list of these activation foils and the reactions used are shown in
Table 1. The activation foils were purchased from ShieldWerx shieldwerxfoils .
These samples were arranged in a stack configuration and were surrounded with
two cadmium covers, to minimize the activation due to low energy neutrons. The
particular ordering in the arrangement of samples was chosen based on the
relative attenuation or scattering produced by each of the samples within the
stack. These alterations in the neutron flux at each position were later
corrected via dedicated Monte Carlo simulations using MCNP6.2 MCNP62 .
Additional samples of Au were placed outside the Cd covers (Au-a sample) and
in the last position (Au-c sample) and used for cross-checking the thermal
contamination and possible scattering back from the experimental hall.
However, these samples were not included in the unfolding of the neutron
spectrum.

Figure 5: (Top): The final portion of the beam line with the lithium target
installed (copper support), the air-cooling system for the target, and the
aluminum support for the activation foils (aluminum mount).
(Bottom): Detail of the aluminum holder used to keep the activation samples in
front of the target. The foils were secured using a kapton foil at the tip.

Figure 5 shows (top) the aluminum support and holder at its tilted position
during the experiment; and (bottom) a detail of the sample holder. The stack
of samples was mounted on an aluminum support and placed 2.5 cm from the
center of the lithium target. The aluminum support was perforated near the tip
of the holder so that the stack could fit in there. The samples were secured
in that position using a fine layer of kapton foil. The samples were tilted by
10$\degree$ with respect to the beam axis in order to reduce the interference
with the measurements by the TOF detectors located in the forward direction at
0∘ in a farther position. Given the close distance between the neutron source
and the samples, the difference in solid angle covered by the first and last
samples in the stack was not negligible. This effect was also incorporated
into the unfolding by correcting the activation depending on their distance to
the source.

Table 1: List of samples and reactions used in the activation measurement setup. Cd-A and Cd-B stand for the cadmium thermal neutron covers. The ordering in the table represents the actual arrangement of the samples in upstream-to-downstream configuration.  Sample | Mass (g) | Reactions | Half-life
---|---|---|---
Au-a | 0.25874(2) |  197Au(n,$\gamma$)198Au | 2.6941 d
Cd-A | 1.13244(3) | - |
Au-b | 0.26644(5) |  197Au(n,$\gamma$)198Au | 2.6941 d
|  |  197Au(n,2n)196Au | 6.1669 d
|  |  197Au(n,4n)194Au | 38.02 h
Al | 0.06468(5) |  27Al(n,$\alpha$)24Na | 14.997 h
In | 0.26002(1) |  113In(n,3n)111In | 2.8047 d
|  |  113In(n,4n)110In | 4.92 h
|  |  115In(n,$\gamma$)116mIn | 54.29 min
|  |  115In(n,n’)115mIn | 4.486 h
Ti | 0.28573(3) |  46Ti(n,p)46Sc | 83.79 d
|  |  48Ti(n,p)48Sc | 43.71 d
Co | 0.31926(8) |  59Co(n,2n)58Co | 70.86 d
|  |  59Co(n,4n)56Co | 77.236 d
|  |  59Co(n,p)59Fe | 44.49 d
|  |  59Co(n,$\alpha$)56Mn | 2.5789 h
Ni | 0.57104(4) |  58Ni(n,p)58Co | 70.86 d
|  |  58Ni(n,2n)57Ni | 35.6 h
Au-c | 0.24924(2) | - |
Cd-B | 0.93009(2) | - |

####  2.5.2 Activation measurements

The activity of each isotope produced by irradiation of the samples was
measured using Gamma spectroscopy with an HPGe detector. It was placed inside
a lead well and cooled every 12 hours with liquid nitrogen. A shaping
amplifier ORTEC 672 was used and pulses were digitized with an ORTEC 928 MCB
and the MAESTRO 7.01 DAQ. The efficiency $\varepsilon(E_{\gamma})$ of the HPGe
detector was calibrated with several certificated radioactive gamma sources
(22Na, 57Co, 60Co, 88Y, 133Ba and 137Cs) for that purpose. Several positions
at different distances (3-15 cm) between the detector and samples were used to
keep a balance between detection efficiency and dead-time losses. A dedicated
low perturbating sample holder has been designed and 3D printed to guarantee
the accuracy of the efficiency calibration and measurements. Simulations were
carried out to account for the different sizes of the calibration radioactive
sources and the measured irradiated samples, obtaining correction factors to
the efficiency, $\kappa_{\varepsilon}$. The low activity of the irradiated
samples impelled in most cases to close the distance down to 3 cm to increase
the count rates. Even if generally small, dead time corrections were taken
into account. The self-shielding correction factor, $\kappa_{s}$, was also
computed in dedicated simulations for every gamma line of each sample. Figure
6 shows the setup of the gamma spectroscopy station.

Figure 6: Gamma spectroscopy station with the HPGe detector and the
surrounding lead shielding. A thin methacrylate support was used to position
the samples. A gold foil is shown in the sample position at 3 cm from the
detector.

After the end of irradiation, a schedule of measurements was followed in order
to obtain the activity data with the best precision, given the time
constraints. Special regards had to be considered given the relatively short
half-lives of some of the produced isotopes, which were assessed by measuring
some of the samples jointly (e.g., Al (containing 24Na), In (containing 110In,
115mIn and 116mIn) and Co (containing 56Mn). Figure 7 shows some of the
measured spectra for these samples.

Figure 7: (Top): Gamma spectrum of the Au sample compared to the overall
background.
(Bottom): Gamma spectrum of the joint measurement of the Al, In, and Co
samples compared to the overall background.

The samples were measured after some cooling time, $t_{c}$, from the end of
irradiation (EOI). The activity of isotope $k$ at the end of irradiation,
$A_{k}^{EOI}$, was extrapolated from the measured counts in a certain gamma
line, $C_{\gamma}$, during the measuring time, $t_{m}$ using the following
expression:

$A_{k}^{EOI}=\frac{C_{\gamma}}{n_{\gamma}\varepsilon\cdot\kappa_{\varepsilon}\cdot\kappa_{s}}\cdot\frac{\lambda_{k}}{1-e^{-\lambda_{k}t_{m}}}e^{\lambda_{k}t_{w}}$
(1)

where energy $E_{\gamma}$, relative intensities of the gamma rays
$n_{\gamma}$, and the decay constants $\lambda_{k}$ of the produced isotopes
were taken from ENSDF ENSDF . At least one gamma line was used for each
reaction. More than one was employed when available. An average value between
the activity values from each gamma line was used in these cases.

####  2.5.3 Unfolding routine

In order to recover the spectral information from the activity of the samples,
an unfolding procedure has to be performed. In the case of a steady-state
neutron field, the activation rate equals the decay rate after several half-
lives of the produced isotope. Therefore the saturated activity, $A_{S}$, can
be used directly as input to the unfolding ($A_{S}=n\sigma\dot{\Phi}$).
However, if the neutron flux is not stable in time (or not enough compared to
the half-lives of the isotopes of interest), a correction factor has to be
applied. In this sense, instead of keep using the saturated activity outside
of its true definition, the total number of activations per atom in the
samples, $R/n$, can be used. This in turn can be done by integrating in time
(using the information from the irradiation history). This leads to an
activations-to-fluence relation in order to perform the deconvolution:

$\frac{R}{n}=\sigma\cdot\tilde{\Phi}$ (2)

In a real experiment, the fluence measured by a foil, $\tilde{\Phi}$, is not
exactly the true fluence in that position, $\Phi$, due to multiple scattering
and self-shielding effects (and other effects from the surrounding support and
other samples in the case of a stack), therefore, a correction factor $\kappa$
has to be introduced to the response function, which is dominated by the
cross-section:

$\frac{R}{n}=\sigma\cdot\kappa\cdot\Phi$ (3)

The energy spectrum can be subsequently divided into several energy groups,
$\Phi_{j}$, that provide information on the shape of the spectrum and can be
chosen depending on the relative contribution to the activations on sample
$i$, $\left(\frac{R}{n}\right)_{i}$, mediated by the response function, $M$:

$\left(\frac{R}{n}\right)_{i}=\sum_{j}\sigma_{ij}\kappa_{ij}\Phi_{j}=\sum_{j}M_{ij}\Phi_{j}$
(4)

Under this approach, the response matrix includes an effective cross-section
within each energy group, $\sigma_{ij}$, that has to be calculated and
included in the response function as:

$M_{ij}=\frac{\int^{E_{j+1}}_{E_{j}}\sigma_{i}(E)\kappa_{i}(E)\Phi(E)dE}{\int^{E_{j+1}}_{E_{j}}\Phi(E)dE}$
(5)

For the computation of the response function, cross-section data spanning from
the low energy range up to above tens of MeV is needed. For this reason, the
TENDL-19 extended cross-section evaluation database was used (including up to
60 MeV or 200 MeV depending on the isotope and reaction) tendl19 . In the
cases where the cross-section data was not available above 60 MeV, linear
extrapolation was applied. These matrix elements are thus dependent on the
intra-group shape of the neutron spectrum. The selection of any spectrum for
the calculation of this matrix impacts the final unfolding, which should be
chosen carefully. This impact can be assessed by performing variations in the
guess spectrum and including that in the unfolded flux uncertainty.

In order to perform the unfolding of the spectrum, several algorithms are
available, including iterative methods such as the SAND-II code SANDii .
Lately, there have been efforts to use other methods to manage the
uncertainties more comprehensively. One such strategy uses a Bayesian
unfolding routine coupled to Markov Chain Monte Carlo simulations (MCMC) using
the code JAGS JAGS . This methodology is based on the one presented by Chiesa
et al. ChiesaBatman2 ; ChiesaBaTMAN . Different sources of uncertainty can be
handled, including the uncertainty in the experimental data from the
activation of the samples, the aforementioned uncertainty in the spectrum
shape and the uncertainties in the cross-section data, which prove to be very
relevant in the MeV energy range. This approach has recently been used to
unfold the spectrum of the newly built NEAR station of n_TOF at CERN
NEAR_ND2022 .

###  2.6 The TOF technique

To provide a complementary and separate measurement of the spectral shape with
higher energy resolution a fast liquid scintillation detector was deployed in
the neutron beam to allow TOF measurements, especially for the high energy
part of the spectrum. In the following paragraph, both the detector used and
the associated electronics are briefly introduced.

####  2.6.1 Experimental setup

A custom-made liquid scintillator detector was used for the measurements
presented here. Figure 8 shows a conceptual drawing of of this device. A 3x3
inch cylindrical cell of borosilicate glass with 1 mm thin optical windows was
filled with a Linear alkylbenzene (LAB) based scintillation cocktail with
admixtures of 2.5 g/l PPO (2,5-Diphenyloxazole) and 3mg/l BisMSB
(1,4-bis-(o-methylstyryl)-benzene). LAB was chosen as the detector medium for
safety reasons, due to its low toxicity and high flashpoint ($\sim$ 140 ∘C).
To protect the scintillator from oxygen, it was previously flushed extensively
with high-purity nitrogen. After filling, the glass cell was sealed gas-tight
under moderate overpressure (a few mbar) of nitrogen. To enhance the light
collection efficiency the mantle surface of the cylindrical cell was
previously covered by a directly evaporated aluminum mirror protected by a
layer of plasma polymer. This vessel is coupled to two fast 3 inch 9821B
photomultiplier tubes (PMTs) housed in mu-metal shields provided by ET
Enterprises pmt . The detector was housed in a thin-walled, fully enclosed
aluminum dark box that serves as both a light shield and a Faraday cage
against electromagnetic noise. This assembly was placed in the beam axis in a
distance of 4.76 m behind the Li-target (distance with respect to liquid
scintillators (LS) target cell center).

Figure 8: Conceptual drawing of the experimental TOF setup consisting of:
cylindrical glass cell filled with an LAB based liquid scintillator (LS), two
ETEL 9821B photomultiplier tubes (PMTs), the whole setup is surrounded by a
mu-metal shield.

The readout electronics of the TOF setup is based on a staged coincidence
circuit. The first coincidence stage is realized between the two PMT branches
and the second between the output of the first stage and the discriminated RF
signal of the accelerator’s beam buncher. The block diagram of the used
electronics can be found in Figure 9. For TOF measurements described here, the
PMT signals are fed to a CAEN N978 10x Fast Amplifier fastamp . While one of
the two outputs of each stage is directly connected to the waveform digitizer
Agilent U1065A Acqiris DC282 (ADC), the other output is connected to a CAEN
N844 LTD (Low Threshold Discriminator) ltd .

Figure 9: Block diagram of the readout electronics: Photomultiplier Tube
(PMT), RF Signal Buncher, Analog to Digital Converter (ADC), Fan in Fan out
(FIFO), Low Threshold Discriminator (LTD), Constant Fraction Discriminator
(CFD), 2 Coincidence Modules.

The sinusoidal RF Signal used for the XTU Tandem’s Beam Buncher is
discriminated on the falling edge of the negative half-wave by a CAEN N842 CFD
(Constant Fraction Discriminator) cfd . The resulting logic signal is
multiplied by a CAEN N625 Fan-In Fan-Out n625fifo  and sent to the ADC on the
one hand and to the second stage of the double coincidence module on the other
hand. In case of a coincident event in both PMTs in coincidence with the
buncher signal, the waveform digitizer is triggered. While the LS detector
reaches an internal time resolution of $\sigma_{LS}$=(769$\pm$6) ps for energy
depositions of 1 MeVee, the buncher of the XTU tandem was capable of producing
beam bunches with a nearly gaussian shape and a width of $\sigma_{Bunch}$
=(1.08$\pm$0.02) ns. Therefore, the beam properties are fully dominating the
time and with that the energy resolution of the TOF spectrum.

####  2.6.2 Unfolding routine

The efficiency and the TOF-to-energy resolution function of the TOF detection
setup were estimated from simulations with the MCNP6.2 code MCNP62 . Several
simulations with monoenergetic neutron beams were carried out to compute the
total detection efficiency, shown in Figure 10 for the energy range covered in
the experiment. Also time discretization in the experimental time window of
200 ns was included in the simulations in order to fill the resolution
function matrix, as shown in Figure 11. The resolution function matrix
provides a means to convert TOF distributions to neutron energy spectra,
including the total flight path from the neutron source to the actual
detection inside the liquid scintillator. Also the time width of the pulse was
reproduced in the simulations by using the distribution observed in the
detection of the $\gamma$-flash. For the calculations of the efficiency and
resolution function, elastic scattering of neutrons off hydrogen and charged
particle reactions from both hydrogen and carbon were included. Elastic
scattering off carbon was not included due to the heavy quenching that
prevented the detection of the carbon recoils.

The unfolding routine to convert TOF to energy spectra was a bayesian-based
iterative algorithm using the maximum likelihood expectation-maximization
(MLEM) method. This method has been previously used for comparable
applications, for instance the unfolding of $\beta$-decay total absorption
spectra Tain2007  and for $\gamma$-ray spectroscopy decompositionMatta2022 .

Figure 10: Detection efficiency of the liquid scintillator located at 4.76 m
from the neutron production target.  Figure 11: Resolution function matrix
used to convert TOF to neutron energy. The matrix includes the detection
efficiency.

##  3 Results

###  3.1 Results from Activation and TOF measurements

The activation produced during the irradiation was measured via Gamma
spectroscopy, from which the activity at the end of irradiation, $A^{EOI}$,
was inferred. Table 2 lists the $A^{EOI}$ for each sample and reaction.
Subsequent spectral results from the analysis using the activation unfolding
method and the total neutron yield are shown in the following section,
compared to the Fluka simulations.

Table 2: List of measured activity at end of irradiation, $A^{EOI}$, for each sample and reaction. Sample | Reaction |  $A^{EOI}$ (Bq)
---|---|---
Au-a |  197Au(n,$\gamma$)198Au | $0.59\pm 0.05$
Au-b |  197Au(n,$\gamma$)198Au | $0.36\pm 0.03$
|  197Au(n,2n)196Au | $5.39\pm 0.22$
|  197Au(n,4n)194Au | $0.57\pm 0.05$
Al |  27Al(n,$\alpha$)24Na | $4.0\pm 0.4$
In |  113In(n,3n)111In | $0.051\pm 0.004$
|  113In(n,4n)110In | $0.056\pm 0.020$
|  115In(n,$\gamma$)116mIn | $3.8\pm 0.6$
|  115In(n,n’)115mIn | $22.0\pm 0.4$
Ti |  46Ti(n,p)46Sc | $0.117\pm 0.019$
|  48Ti(n,p)48Sc | $1.72\pm 0.05$
Co |  59Co(n,2n)58Co | $0.534\pm 0.013$
|  59Co(n,4n)56Co | $0.107\pm 0.006$
|  59Co(n,p)59Fe | $0.113\pm 0.009$
|  59Co(n,$\alpha$)56Mn | $2.05\pm 0.16$
Ni |  58Ni(n,p)58Co | $1.73\pm 0.04$
|  58Ni(n,2n)57Ni | $1.76\pm 0.11$

The experimental TOF spectrum measured with the LS detectors is shown in
Figure 12. This TOF spectrum includes the $\gamma$-flash from the target and
the neutron distribution. The long TOF tail is altered due to event rejection
in triggering due to the fast bunch rate used in the experiment. This makes us
reject the low energy tail in the reconstruction. Also this part of the
experiment lacked a concise determination of the ion current used to produce
the spectrum, and therefore only the shape of the spectrum is retained. The
unfolded energy spectrum is shown in the next section.

Figure 12: Time of Flight data for events below 200 ns. The left peak shows
the gamma flash events from the neutron production target, and the right peak
shows the neutron distribution. From the gamma flash the time and width that
energy resolution of the TOF spectrum was derived. The arbitrary offset for
the TOF measurement is calibrated out by shifting the mean value of the gamma
flash to the value derived from the division of the distance (between detector
and Li target) by the speed of light.

###  3.2 Comparison between FLUKA simulations and the experimental data

Table 3 reports the measured neutron flux from unfolding at the activation
foils position, compared with the FLUKA simulation; the unfolded neutron
spectrum is shown in Figure 13. The integral neutron flux of $(4.04\pm
0.42)\times 10^{5}$ n/cm²/s/pnA at that position, confirms the estimates by
FLUKA ( $(3.79\pm 0.10)\times 10^{5}$ n/cm²/s/pnA), in good agreement within
uncertainties. The unfolded spectrum obtained from activation data is shown in
blue in Figure 13. Five energy groups above 0.1 MeV show good agreement both
in shape and in the absolute value of the production yield.

Table 3: Measured neutron flux from unfolding at the activation position, compared with FLUKA simulation. The integral neutron flux is also given in the last row. Energy Range (MeV) |  $\Phi$ ($10^{3}$ n/cm²/pnA)
---|---
Min | Max | Unfolded | FLUKA Sim
$10^{-9}$ | $0.1$ | $11\pm 35$ | $4.2\pm 0.7$
$0.1$ | $3.16$ | $64\pm 27$ | $70.0\pm 2.6$
$3.16$ | $10.0$ | $198\pm 32$ | $187.1\pm 4.0$
$10.0$ | $20.0$ | $104\pm 12$ | $94.0\pm 2.4$
$20.0$ | $35.5$ | $21\pm 6$ | $22.2\pm 1.0$
$35.5$ | $100$ | $7\pm 7$ | $1.42\pm 0.23$
$10^{-9}$ | $100$ | $404\pm 42$ | $379\pm 10$

Moreover, the unfolded data from the TOF measurement, covering the high energy
part of the spectrum (above 8 MeV), shows a spectral shape with increased
energy resolution and also in excellent agreement with the simulation. The
results of the unfolding are shown also in Figure 13 in comparison with the
activation and simulated data. The spectral data from TOF has been scaled to
the activation data in that range as this part of the experiment did not
include an independent normalization.

Figure 13: Unfolded neutron spectrum obtained from activation and unfolded
from TOF, compared with FLUKA simulations. FLUKA simulations are also shown
with the same binning of the activation unfolding to facilitate the
comparison.

##  4 Discussion and prospects

A practical limitation of the 7Li(14N,xn)X reaction at LNL is the intensity of
the ion beam current, measured in particle nanoamperes compared to
microamperes at the LNL proton machines. This limitation can be partially
compensated for by placing test points just a few centimeters from the neutron
production target.

If only a Tandem is available, the voltage and charge state, hence the beam
energy, should be chosen to optimize the current and the energy dependent
neutron yield. At LNL, the ALPI booster can be used to reach higher beam
energies: according to FLUKA simulations, the energy spectrum can be stiffened
and the neutron yield can be increased by a factor 5 by raising both the beam
energy up to 170 MeV and the lithium thickness up to 1 mm (Figure 14).

In practice the transmission of the ALPI booster is only 8%. If the Tandem is
used as an injector for ALPI, the gain in flux is more than canceled, but by
using the more intense Electron Cyclotron Resonance (ECR) based PIAVE
injection system for ALPI, the neutron flux can be significantly increased:
the PIAVE system is capable of producing high positive charge state ions at
the source, unlike the Tandem were an inefficient stripping foil must be used.
To verify the higher energy and thicker lithium simulations, we plan to
perform the PIAVE-ALPI version of this experiment as soon as possible.

Finally, we are investigating the possibility of using a thin beryllium easy
to install self-standing foil target instead of the lithium targets fabricated
in a glovebox (described in Section 2.1). Beryllium has better physical
properties (higher melting point, thermal conductivity, and lower linear
thermal expansion coefficient) and is stable in air making it much more
manageable compared to lithium. However the issue of radiation damage to the
Be foil must be evaluated with great care: Be is toxic and the risk of
polluting the beam line might be too great.

Figure 14: FLUKA simulations comparing forward direction neutron spectra for
90 MeV energy using 200 $\mu$m with 170 MeV using 1 mm thick Lithium and 320
$\mu$m thick Be.

##  5 Summary

The 7Li(14N,xn)X reaction is cost effective and easy to implement at Tandem
machines. LNL can use this reaction to immediately produce fast neutrons in an
interesting energy range while waiting for the construction and commissioning
of a dedicated neutron irradiation facility at the SPES proton cyclotron. The
experimental neutron energy distribution measured is broad and peaked at 8
MeV, ranging from hundreds of keV up to tens of MeV. Activation foils and TOF
technique have been used to measure the produced energy spectra, showing a
good agreement between them and the heavy ion fragmentation and coalescence
models available in FLUKA.

###### Acknowledgements.

This work has been supported by the Cluster of Excellence PRISMA+, the Cluster
of Excellence ORIGINS as well as the Collaborative Research Center “Neutrinos
and Dark Matter in Astro- and Particle Physics” (SFB 1258) and the University
of Granada Chair ”Neutrons for Medicine”. Moreover, we are greatful for the
support from the DFG Research Unit JUNO 2319 and 5519 (Project Number:
498394246) and the Spanish Junta de Andalucía (Projects FEDER no.
A-FQM-371-UGR18 and B-FQM-156-UGR20). I.P. acknowledges finantial support from
the Spanish Ministerio de Ciencia, Innovación y Universidades for his research
stay at LNL under contract PRX18/00551. Furthermore, we would like to thank
Francisco Ogallar Ruiz for his helpful advice and suggestions on the FLUKA
simulations.

## References

  * (1) P. Mastinu, D. Bisello, R.A. Barrera, I. Porras, G. Prete, L. Silvestrin, J. Wyss, Journal of Neutron Research 22(2-3), 233 (2020)
  * (2) L. Silvestrin, D. Bisello, J. Esposito, P. Mastinu, G. Prete, J. Wyss, The European Physical Journal Plus 131(72) (2016).  DOI https://doi.org/10.1140/epjp/i2016-16072-0.  URL https://doi.org/10.1140/epjp/i2016-16072-0
  * (3) C. Signorini, F. Cervellera, G. Bezzon, Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 244(1), 27 (1986).  DOI https://doi.org/10.1016/0168-9002(86)90731-X
  * (4) C. Ahdida, D. Bozzato, D. Calzolari, F. Cerutti, N. Charitonidis, A. Cimmino, A. Coronetti, G. D’Alessandro, A. Donadon Servelle, L. Esposito, R. Froeschl, R. García Alía, A. Gerbershagen, S. Gilardoni, D. Horváth, G. Hugo, A. Infantino, V. Kouskoura, A. Lechner, B. Lefebvre, G. Lerner, M. Magistris, A. Manousos, G. Moryc, F. Ogallar Ruiz, F. Pozzi, D. Prelipcean, S. Roesler, R. Rossi, M. Sabaté Gilarte, F. Salvat Pujol, P. Schoofs, V. Stránský, C. Theis, A. Tsinganis, R. Versaci, V. Vlachoudis, A. Waets, M. Widorski, Frontiers in Physics 9(788253) (2022).  DOI 10.3389/fphy.2021.788253.  URL https://doi.org/10.3389/fphy.2021.788253
  * (5) Shieldwerx, a division of Bladewerx LLC.  Activation foils.  URL https://www.shieldwerx.com/activation-foils.html
  * (6) C.J. Werner, J.S. Bull, C.J. Solomon, F.B. Brown, G.W. McKinney, M.E. Rising, D.A. Dixon, R.L. Martz, H.G. Hughes, L.J. Cox, A.J. Zukaitis, J.C. Armstrong, R.A. Forster, L. Casswell, _MCNP6.2 Release Notes - report LA-UR-18-20808_ (Los Alamos National Laboratory (LANL), 2018)
  * (7) Evaluated Nuclear Structure Data Files (2023).  URL https://www.nndc.bnl.gov/ensdf/.  Last accessed 21 July 2023
  * (8) A. Koning, D. Rochman, J.C. Sublet, N. Dzysiuk, M. Fleming, S. van der Marck, Nuclear Data Sheets 155, 1 (2019).  DOI https://doi.org/10.1016/j.nds.2019.01.002.  Special Issue on Nuclear Reaction Data
  * (9) P. Griffin, J. Kelly, J. VanDenburg, _User’s Manual for SNL-SAND-II Code_ (1994).  URL https://rsicc.ornl.gov/codes/psr/psr3/psr-345.html
  * (10) M. Plummer, _JAGS Version 4.3.0 user manual_ (2017).  URL https://martynplummer.wordpress.com/2017/07/18/jags-4-3-0-is-released/
  * (11) D. Chiesa, E. Previtali, M. Sisti, Annals of Nuclear Energy 70, 157 (2014).  DOI 10.1016/j.anucene.2014.02.012
  * (12) D. Chiesa, M. Nastasi, C. Cazzaniga, M. Rebai, L. Arcidiacono, E. Previtali, G. Gorini, C.D. Frost, Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 902, 14 (2018).  DOI 10.1016/j.nima.2018.06.016
  * (13) Stamati, M. E., Torres-Sánchez, P., Pérez-Maroto, P., Cecchetto, M., Goula, S., Mastromarco, M., Chasapoglou, S., Beltrami, C., Chiesa, D., Manna, A., Mucciola, R., Patronis, N., Praena, J., Guerrero, C., Colonna, N., Mengoni, A., Massimi, C., Aberle, O., Alcayne, V., Altieri, S., Amaducci, S., Amar Es-Sghir, H., Andrzejewski, J., Babiano-Suarez, V., Bacak, M., Balibrea, J., Bennett, S., Bernardes, A. P., Berthoumieux, E., Bosnar, D., Busso, M., Caamaño, M., Calviño, F., Calviani, M., Cano-Ott, D., Casanovas, A., Castelluccio, D. M., Cerutti, F., Cescutti, G., Chiaveri, E., Colombetti, P., Console Camprini, P. C., Cortés, G., Cortés-Giraldo, M. A., Cosentino, L., Cristallo, S., Di Castro, M., Diacono, D., Diakaki, M., Dietz, M., Domingo-Pardo, C., Dressler, R., Dupont, E., Durán, I., Eleme, Z., Fargier, S., Fernández-Domínguez, B., Finocchiaro, P., Fiore, S., Furman, V., García-Infantes, F., Gawlik-Ramiega, A., Gervino, G., Gilardoni, S., González-Romero, E., Gunsing, F., Gustavino, C., Heyse, J., Jenkins, D. G., Jericha, E., Junghans, A., Kadi, Y., Katabuchi, T., Katabuchi, T., Knapová, I., Kokkoris, M., Kopatch, Y., Krticka, M., Kurtulgil, D., Ladarescu, I., Lederer-Woods, C., Lerendegui-Marco, J., Lerner, G., Martínez, T., Masi, A., Mastinu, P., Matteucci, F., Maugeri, E. A., Mazzone, A., Mendoza, E., Michalopoulou, V., Milazzo, P. M., Murtas, F., Musacchio-Gonzalez, E., Musumarra, A., Negret, A., Oprea, A., Pavón-Rodríguez, J. A., Pellegriti, M. G., Perkowski, J., Petrone, C., Piersanti, L., Pirovano, E., Pomp, S., Porras, I., Protti, N., Quesada, J. M., Rauscher, T., Reifarth, R., Rochman, D., Romanets, Y., Romano, F., Rubbia, C., Sánchez, A., Sabaté-Gilarte, M., Schillebeeckx, P., Schumann, D., Sekhar, A., Smith, A. G., Sosnin, N. V., Spelta, M., Tagliente, G., Tarifeño-Saldivia, A., Taríro, D., Terranova, N., Urlass, S., Valenta, S., Variale, V., Vaz, P., Vescovi, D., Vlachoudis, V., Vlastou, R., Wallner, A., Woods, P. J., Wright, T., Zugec, P., n_TOF Collaboration, EPJ Web of Conf. 284, 06009 (2023).  DOI 10.1051/epjconf/202328406009.  URL https://doi.org/10.1051/epjconf/202328406009
  * (14) ET Enterprises Electron Tubes, _78 mm (3”) photomultiplier 9821B series data sheet_.  ET Enterprises Limited, Uxbridge, United Kingdom (2012)
  * (15) CAEN, _N978 - 4 Channel Variable Gain Fast Amplifier_.  CAEN, URL:
https://www.caen.it/products/n978/ (2022)

  * (16) CAEN, _N844 - 8 Channel Low Threshold Discriminator_.  CAEN, URL:
https://www.caen.it/products/n844/ (2022)

  * (17) CAEN, _N842 - 8 Channel Constant Fraction Discriminator_.  CAEN, URL:
https://www.caen.it/products/n842/ (2022)

  * (18) CAEN, _N625 - Quad Linear FAN-IN FAN-OUT_.  CAEN, URL:
https://www.caen.it/products/n625/ (2022)

  * (19) J. Tain, D. Cano-Ott, Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 571(3), 728 (2007).  DOI https://doi.org/10.1016/j.nima.2006.10.098
  * (20) J.T. Matta, A.J. Rowe, M.P. Dion, M.J. Willis, A.D. Nicholson, D.E. Archer, H.H. Wightman, IEEE Transactions on Nuclear Science 69(6), 1212 (2022).  DOI 10.1109/TNS.2022.3162986


# Never a Dull Moment: Distributional Properties as a Baseline for Time-Series
Classification

Trent Henderson
The University of Sydney Annie G. Bryant
The University of Sydney Ben D. Fulcher
The University of Sydney

###### Abstract

The variety of complex algorithmic approaches for tackling time-series
classification problems has grown considerably over the past decades,
including the development of sophisticated but challenging-to-interpret deep-
learning-based methods. But without comparison to simpler methods it can be
difficult to determine when such complexity is required to obtain strong
performance on a given problem. Here we evaluate the performance of an
extremely simple classification approach—a linear classifier in the space of
two simple features that ignore the sequential ordering of the data: the mean
and standard deviation of time-series values. Across a large repository of 128
univariate time-series classification problems, this simple distributional
moment-based approach outperformed chance on 69 problems, and reached 100%
accuracy on two problems. With a neuroimaging time-series case study, we find
that a simple linear model based on the mean and standard deviation performs
better at classifying individuals with schizophrenia than a model that
additionally includes features of the time-series dynamics. Comparing the
performance of simple distributional features of a time series provides
important context for interpreting the performance of complex time-series
classification models, which may not always be required to obtain high
accuracy.

_Keywords_ Time series $\cdot$ Time-series classification $\cdot$
Benchmarking.

##  1 Introduction

Time-series classification is a key problem in the sciences and industry
wherein time-varying data is used to distinguish labeled classes. The quantity
and diversity of time-series classification algorithms is large and
increasing, from simple linear decision boundaries in interpretable feature
spaces [1] to complex methods based on deep neural networks, such as long
short-term memory networks [2]. Complex new algorithms can yield impressive
classification accuracy on challenging problems, but often at the expense of
clear human interpretability [3].

The UEA/UCR univariate time-series classification repository, which currently
contains 128 problems spanning a variety of domains [4], has been crucial for
encouraging transparent reporting of the relative strengths and weaknesses of
classification algorithms. It has also enabled the field to overcome key
limitations in prior standard practice of reporting and comparison, including
avoiding cherry-picking of optimistic datasets when reporting algorithm
performance [5]. Systematic comparisons of time-series classification
algorithms across this database have been essential for benchmarking the
accuracy of state-of-the-art algorithms [6]. However, in settings ranging from
policy-making [7] to healthcare [8], deriving interpretable understanding that
can guide subsequent decision-making can be more important than raw
classification accuracy. In such settings, simpler methods that are faster to
train and clearer to interpret, are often preferred.

Recent work has shown that parsimonious and interpretable methods can match
(or even outperform) more sophisticated ones, in settings ranging from sleep-
stage classification [9] to earthquake detection [10]. A particularly striking
example is the recent demonstration of a two-parameter logistic regression
model with equivalent performance in earthquake aftershock forecasting to a
deep neural network with thousands of parameters [11]. For such settings, in
which simple methods can perform well, the choice to instead use overly
complex models risks over-fitting, leading to poor generalizability on out-of-
sample data. This effect has been demonstrated in a recent meta-analysis of
deep-learning models for classifying autism spectrum disorder from resting-
state neuroimaging data, which exhibited inferior performance to linear
support vector machine (SVM) models on unseen data [12].

Defaulting to complex classification models and comparing their performance to
chance-level accuracy—or even other complex methods—can thus result in
classifiers that are over-complicated and difficult to interpret. This issue
is well-illustrated by the time-series classification task of distinguishing
‘epilepsy’ from ‘eyes-open’ states from electroencelphalogram (EEG) data [13].
While state-of-the-art approaches had applied complex algorithmic approaches,
ranging from independent components analysis and discrete wavelet transforms
to multi-layer neural networks [14], it was found that the time-series
standard deviation alone could completely separate the two classes [15]. The
strong performance of a simple threshold classifier on standard deviation thus
suggests it as an interpretable and parsimonious alternative to overly
complicated algorithmic approaches to this problem, highlighting the utility
of starting with a simple and parsimonious model and building in complexity
only when it yields clearly demonstrable benefits.

Given that the characteristic complexity of time-series classification tasks
(compared to classification using non-sequential data) relates to the
challenge of quantifying class-informative dynamical patterns, here we aimed
to investigate the performance of an extremely simple benchmark: a linear
classifier in the two-dimensional space of two extracted features that ignore
the sequential ordering entirely: the mean and standard deviation. As these
features characterize the distribution of time-series values (and are thus
unrelated to the sequential ordering), high performance of this simple
benchmark indicates problems for which trivial properties of the distribution
are already sufficient to perform well, undermining the need for more complex
approaches that aim to quantify informative temporal patterns [16]. While some
previous work [1, 17] has $z$-scored all time series prior to analysis—thereby
insuring mean and variance are uninformative, to focus on dynamical
patterns—such normalization has not been applied consistently across all
problems in the UEA/UCR Repository [6]. We also investigate the performance
gains of additionally incorporating a simple set of time-series features that
capture different types of dynamical patterns in the data, using the catch22
feature set [17]. None of these features are sensitive to the mean or standard
deviation of the data; they instead capture more subtle properties of the time
series, including properties of distributional shape; periodic patterns;
temporal spacing of outliers; and linear and nonlinear autocorrelation [17].
Finally, to demonstrate the practical implications of our findings, we present
a case study of classifying individuals with schizophrenia using functional
neuroimaging time series.

##  2 Methods

This study aims to evaluate the performance of a simple baseline linear
classifier for time series in the two-dimensional space of their mean ($\mu$)
and standard deviation ($\sigma$)—features related to the First Two Moments
(FTM) of the distribution. We first discuss methods related to fitting the FTM
and catch22 feature-based time-series classifiers across the UEA/UCR
Repository [4] (in Sec. 2.1) and then describe specific methods related to our
neuroimaging case study (in Sec. 2.2). Code for reproducing our results is
available on GitHub111https://github.com/hendersontrent/mean-var-ts-classify.

###  2.1 Feature-based classifier performance on the UEA/UCR Repository

In the UEA/UCR Time-Series Classification Repository [4], each of the 128
problems (as downloaded on 9 Feb 2023) are partitioned into designated train
and test sets. These problems vary widely in the number of time series, the
relative size of train and test set sizes, time-series lengths, and number of
classes (see [4, 6] for details). For all time series, we computed the FTM
feature set containing two features: the mean ($\mu)$ and standard deviation
($\sigma$). For comparison to more sophisticated features of the time-series
dynamics, we used the catch22 set of 22 time-series features [17]. Given a
feature space, we fit and evaluated classification models following the
resample-based procedure outlined in [6], using 30 resamples of train–test
splits. In addition to the designated split, we generated 29 additional
(seeded) resamples of the data that preserve the class proportions of the
designated split, for a total of 30 train–test splits. Prior to fitting a
linear SVM, features were normalized as a $z$-score by computing the mean and
standard deviation of each feature in the train set, and using these values to
rescale both the train and test sets (ensuring that test data was completely
unseen). A linear SVM was fit on the train set and used to generate
predictions for the test set. Classification accuracy was used as the
performance metric, following prior work. To statistically compare the
performance of FTM, and the union set of 24 features from both catch22 and FTM
(catch22 \+ FTM), we implemented a correlated test statistic [18] that
corrects the traditional $T$-statistic to account for the violation of the
assumption of independence incurred in the usage of resampling using the
correctR package [19]. To compare the FTM and catch22 \+ FTM feature sets, we
removed four problems (AllGestureWiimoteX, AllGestureWiimoteY,
AllGestureWiimoteX, and PLAID) due to presence of time series that had $<10$
real values before containing all or mostly missing values—the minimum
threshold for calculations in catch22.

###  2.2 Case study: Schizophrenia classification

As a case study, we investigated the performance of simple feature-based
classifiers to distinguish adults with schizophrenia versus cognitively
healthy controls based on their whole-brain activity dynamics. We obtained
resting-state functional magnetic resonance imaging (rs-fMRI) data from the
University of California at Los Angeles Consortium for Neuropsychiatric
Phenomics LA5c Study [20], which was pre-processed using the ICA-AROMA + 2P +
GMR method, as described previously [21]. Blood oxygen level-dependent (BOLD)
signals consisting of 152 time samples were extracted for each of 68 cortical
[22] and 14 subcortical [23] regions per participant. Any participants with
flatline time series across all brain regions after preprocessing were
excluded, yielding a final sample of $N=166$ participants ($N=118$ control and
$N=48$ schizophrenia). The two groups differ in terms of age (Control
$=31.5\pm 8.8$ years; Schizophrenia $=36.6\pm 9.0$ years; Wilcoxon rank sum
test $p<0.01$) and sex (Control $=46.6\%$ Female; Schizophrenia = $25\%$
Female; $X^{2}(1,N=166)=5.75$, $p<0.05$).

After feature extraction (using theft [24]), the dataset was in the form of an
$N\times R\times F$ matrix, for $N$ subjects, $R$ brain regions, and $F$
features. We incorporated all combinations of $82$ brain regions and features
(either $2$ FTM features or $24$ FTM + catch22 features) as inputs to a
regularized linear SVM classifier (from scikit-learn [25]), setting the
regularization parameter C = 1, disabling the shrinking heuristic, and using
balanced class weights. In other words, the FTM-only model had a total of
$2\times 82=164$ SVM input features and the FTM + catch22 model had a total of
$24\times 82=1968$ features.

Model performance was evaluated as balanced accuracy using $10$-fold cross-
validation (CV) with 10 repeats (setting the random state such that all
compared models received the same resampled sets of test folds). For each
$k$-fold, the same $z$-score feature-normalization was applied as described in
Sec. 2.1. Balanced accuracy is reported as the mean $\pm$ SD across the 10
repeats, in which each repeat contains the mean balanced accuracy across the
10 CV folds. Statistical significance of a given balanced accuracy value was
assessed using permutation testing using 1000 null samples obtained from
shuffling class labels, pooling null samples across all combinations of
regions and features for each feature set [26]. False positives were
controlled at the $\alpha=0.05$ level using Bonferroni correction.

##  3 Results

Results are structured as follows. First, in Sec. 3.1, we describe findings
across the UEA/UCR Repository, where we demonstrate that the FTM classifier
outperforms chance on the majority of problems. We then highlight the
practical ramifications of these results using the example of a neuroimaging
biomarker classification task in Sec. 3.2, in which the mean and standard
deviation exhibit strong performance that is weakened by adding dynamical
properties of the functional neuroimaging time series.

###  3.1 Linear FTM classifier performance across the UEA/UCR Database

We first investigated how a linear classifier based on the two FTM features,
$(\mu,\sigma)$, performs across the 128 problems from the UEA/UCR Repository.
To determine the problems on which FTM beats chance, we calculated a $p$-value
of the chance probability against the distribution of 30 resampled FTM
accuracy values for each problem. We found that the FTM-based classifier
statistically outperformed chance ($p<0.05$) on 69 of the 128 problems (i.e.,
on $53.9\%$ of problems). FTM-based classification accuracies relative to
chance level are plotted for these 69 problems in Fig. 1. We note that chance
is often beaten by a substantial margin on these problems. Indeed, this simple
classifier achieves 100% accuracy on two problems: InsectEPGRegularTrain and
GunPointOldVersusYoung. These results demonstrate that, even on a repository
devoted to time-series classification tasks, strong performance can be
obtained using the two simplest distributional statistics that are unrelated
to the sequential ordering of the data, due to the database containing
problems in which labeled classes have distinctive levels (means) or scales
(variances).

Figure 1:  Mean and standard deviation as features in a linear SVM classifier
statistically outperforms chance-level accuracy on 69 of 128 problems in the
UEA/UCR time-series repository. Classification accuracy ($\%$) is displayed
along the horizontal axis and the name of each problem is shown on the
vertical axis. Chance accuracies are displayed as black crosses. Points
indicate mean classification accuracy and error bars show $\pm 1$ standard
deviation (across train–test resamples).

To better understand how the FTM classifier is behaving, we examined the two
problems for which it achieved 100% accuracy. For GunPointOldVersusYoung, the
classes are clearly distinguished in the $(\mu,\sigma)$ feature space (Fig.
2A), while for InsectEPGRegularTrain each class has a characteristic mean
level (Fig. 2B). For the binary (‘Young’ vs ‘Old’) GunPointOldVersusYoung
task, the time series is the $x$-axis coordinate of their centre of the hand
at each frame while moving it from rest position to a gun pose and back again.
Figure 2A shows that the ‘Young’ actor exhibits characteristically lower mean
and variation of their hand coordinate, consistent with them being shorter and
having a shorter arm than the ‘Old’ actor. It is worth noting that, in
contrast to features of the time-series dynamics (many of which are invariant
to linear rescalings of the time-series values), $\mu$ and $\sigma$ are highly
sensitive to the calibration of experimental measurement. For the GunPoint
collection of problems, this makes these features less likely to generalize
well to new examples, with small differences in the angle of the camera used
to measure hand coordinates, or the distance from camera to actor, strongly
affecting differences in $\mu$ and $\sigma$ and hence accurate classification
[27]. In InsectEPGRegularTrain, each class has a characteristic mean voltage
level of the electrical circuit that connects insects with their food source,
allowing the classes to be accurately distinguished by ignoring sequential
patterns, and simply focusing on this mean voltage.

Figure 2:  Simple distributional moments can perfectly separate classes in the
GunPointOldVersusYoung and InsectEPGRegularTrain datasets. A Individual time
series in GunPointOldVersusYoung are represented in the two-dimensional
feature space of mean and standard deviation. Points are colored by class (as
labeled) and class-level covariances are shown as shaded ellipses to guide the
eye. B Histogram of time-series means by class in InsectEPGRegularTrain.

Finally, we aimed to investigate the extent to which measuring the catch22
features in addition to the FTM features would improve performance. We
compared the performance of the FTM feature set (2 features) to that of the
catch22 \+ FTM feature set (24 features) across the 124 problems where catch22
features could be successfully calculated (four problems were excluded due to
a high number of missing values, cf. Methods). Adding catch22 features
resulted in an average absolute improvement in classification accuracy of
$33.7\%$ across the 124 problems, confirming the general importance of
capturing dynamical properties for time-series classification problems.
However, pairwise comparisons using the corrected test statistic revealed that
there was no statistical difference between FTM and catch22 \+ FTM on $45$ of
the $124$ (or $36.4\%$ of) problems. This demonstrates that for many problems,
simple distributional properties yield a surprisingly strong baseline against
which to assess the benefits gained by using more complex approaches.

###  3.2 Neuroimaging Biomarker Case Study

Figure 3:  The FTM feature set shows strong performance at classifying
individuals with schizophrenia from resting-state fMRI time series. The
distribution of mean balanced accuracy (across 10 repeats), using the
combination of all brain regions and either the two FTM features (green), or
the 24 FTM + catch22 features (orange), is shown as a combined spaghetti plot
and boxplot. Each gray line indicates the same resampled set of $k=10$ folds
evaluated for each feature set along the $x$-axis. ∗∗: $p=0.003$, corrected
resampled $T$ $=3.63$.

We next extended our investigation of simple baseline time-series classifiers
to a schizophrenia classification problem using rs-fMRI time series, a setting
in which complex, opaque classifiers are common [28, 29]. Given the strong
performance of simple distributional statistics for some time-series
classification problems above, we evaluated the performance of FTM alone (and
with the addition of the catch22 feature set) in classifying individuals with
schizophrenia from healthy controls.

As shown in Fig. 3, the model based on FTM features displayed a high balanced
accuracy ($67.5\pm 2.2\%$), which sits well within the range of a recent meta-
analysis of schizophrenia classification studies using rs-fMRI [30]. Adding
catch22 features to the model decreased its performance to $63.6\pm 1.0\%$
(corrected resampled $T$-statistic = $3.63$, $p=0.003$). Consistent with the
findings on some problems in the UAE/UCR Repository above, these results
demonstrate the surprisingly strong performance of basic properties of the
distribution of time-series values for this fMRI classification task.
Surprisingly, for this rich and complex whole-brain time-series dataset,
incorporating information about temporal patterns in the data (using catch22)
yields models with inferior performance than the simple two-dimensional model
based on mean and standard deviation.

##  4 Discussion

With the growing sophistication of machine-learning algorithms, modern data
analysts have increasingly adopted a methodological approach that defaults to
more complex statistical methods, which can obscure their clear
interpretation. Presenting such complex approaches without direct comparison
to simpler alternatives makes it difficult to discern whether the
methodological complexity is beneficial relative to simpler approaches, an
assumption that has been challenged in some recent studies [9, 11, 10]. In
this work, focusing on time-series classification problems, we demonstrate
that a method at the extreme end of simplicity (using just two distributional-
moment-based features, $\mu$ and $\sigma$, and a linear classifier) performs
surprisingly well on many such problems. Despite being insensitive to the
unique property of sequential data relative to an unordered vector—its
ordering (in time)—this simple benchmark classifier statistically outperformed
chance on approximately half of the problems in a prominent time-series
classification archive, the UEA/UCR Repository [4], and even achieved $100\%$
accuracy on two problems. Our results emphasize the importance of carefully
considering the factors that contribute to model performance, assessing model
gains relative to simpler models, and favoring parsimony by carefully building
up model complexity incrementally from simple baselines.

We demonstrated the applicability of our findings to a time-series
classification problem in neuroimaging, a setting in which many prior models
have been highly complex models [28] but can fail to generalize [12].
Importantly, classification performance was stronger with FTM than with a
model that also included catch22 features of the dynamics using an SVM
classifier, which is generally the top-performing classifier type in rs-fMRI
analysis for schizophrenia [30]. The combination of $\sigma$ and $\mu$ yielded
a mean balanced accuracy of $67.5\%$ while affording clear statistical and
biological interpretations.

The strong performance of mean and standard deviation across the UEA/UCR
Repository demonstrates that time series in these problems have not been
consistently normalized, as prior work has highlighted [6]. If all time series
were individually $z$-score transformed, there would be no class differences
in mean or standard deviation for any problem (and the FTM-based classifier
would exhibit null performance). Our results have implications for comparing
time-series classification algorithms on problems for which labeled classes
can be distinguished based on distributional properties alone. For example,
consider the catch22 features, which are all insensitive to the mean and
variance of the input time series [17]. Relative to catch22, the superior
performance of an alternative time-series classifier (which is sensitive to
$\mu$ or $\sigma$ of the input series) could be driven entirely by a class-
relevant difference in $\mu$ or $\sigma$: properties that are unrelated to
dynamical patterns (and that catch22 cannot access). One approach to testing
the ability of different classification algorithms to capture properties
related to the dynamics of a uniformly sampled univariate time series would be
to normalize time series such that there are no class differences in basic
distributional properties. A second approach would be to compare model
performance to a benchmark in which only simple distributional properties are
included, with gains relative to this benchmark then being attributable to
class-relevant differences in more complex properties. In this latter
approach, there is scope for extending the two simple (FTM) features used here
by adding higher-order moments, or other types of distributional features
(such as those included in hctsa [31]).

This work also highlights the need for careful consideration of the
generalizability of models that use features sensitive to the calibration of
time-series measurements (like $\mu$ and $\sigma$), relative to features that
are invariant to linear rescalings of the input time series (like the catch22
feature set). This is because such measurement-scale-dependent features are
highly sensitive to changes in the calibration of experimental measurements
that may not be precisely maintained in new data. An illustrative example is
shown here in the GunPointOldVersusYoung dataset, for which our FTM-based
method achieved 100% accuracy. However, this may be a deceptively high value,
as it is reliant on the precise calibration of new data. A prior analysis of
this dataset, focusing on early classification, has shown that classification
accuracy can plummet with only slight variability in experimental calibration
(to changes as small as a $\approx 1.9^{\circ}$ tilt in the angle of the
camera used to measure hand coordinates) [27]. In general, the decision of
whether or not to include features that are sensitive to measurement scale (or
focus on scale-invariant features of the dynamics or distribution shape, as in
catch22) should be motivated by domain expertise to avoid overly optimistic
classification results.

In summary, by highlighting many time-series classification problems for which
simple distributional properties of a time series can achieve surprisingly
high classification accuracy, our results raise important issues for the
development and interpretation of time-series classification models. Future
work on evaluating time-series classification algorithms may consider using
simple benchmarks for comparison to aid interpretation and provide evidence
for the contribution of model complexity to any performance advantage,
particularly for problems highlighted here for which simple distributional
features are highly informative of class differences.

### Acknowledgements

The authors would like to thank Kevin Aquino for sharing preprocessed fMRI
data [32] used in the case study, and Eamonn Keogh for providing useful
feedback on a manuscript draft.

## References

  * Fulcher and Jones [2014] B. D. Fulcher and N. S. Jones.  Highly comparative feature-based time-series classification.  _IEEE Transactions on Knowledge and Data Engineering_ 26, 3026 (2014).
  * Ismail Fawaz et al. [2019] H. Ismail Fawaz, G. Forestier, J. Weber, L. Idoumghar, and P.-A. Muller.  Deep learning for time series classification: A review.  _Data Mining and Knowledge Discovery_ 33, 917 (2019).
  * Rudin [2019] C. Rudin.  Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.  _Nature Machine Intelligence_ 1, 206 (2019).
  * Bagnall et al. [2022] A. Bagnall, J. Lines, W. Vickers, and E. Keogh.  The uea & ucr time series classification repository, www.timeseriesclassification.com (2022).
  * Keogh and Kasetty [2003] E. Keogh and S. Kasetty.  On the Need for Time Series Data Mining Benchmarks: A Survey and Empirical Demonstration.  _Data Mining and Knowledge Discovery_ 7, 349 (2003).
  * Bagnall et al. [2017] A. Bagnall, J. Lines, A. Bostrom, J. Large, and E. Keogh.  The great time series classification bake off: A review and experimental evaluation of recent algorithmic advances.  _Data Mining and Knowledge Discovery_ 31, 606 (2017).
  * Goodman and Flaxman [2017] B. Goodman and S. Flaxman.  European union regulations on algorithmic decision-making and a “right to explanation”.  _AI Magazine_ 38, 50 (2017).
  * Caruana et al. [2015] R. Caruana, Y. Lou, J. Gehrke, et al.  Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission.  In _Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ , p. 1721. Association for Computing Machinery (2015).
  * Van Der Donckt et al. [2023] J. Van Der Donckt, J. Van Der Donckt, E. Deprost, et al.  Do not sleep on traditional machine learning: Simple and interpretable techniques are competitive to deep learning for sleep scoring.  _Biomedical Signal Processing and Control_ 81, 104429 (2023).
  * Waheed et al. [2020] U. Waheed, A. Afify, M. Fehler, and B. Fulcher.  Winning with simple learning models: Detecting earthquakes in groningen, the netherlands.  _EAGE 2020 Annual Conference & Exhibition Online_ 2020, 1 (2020).
  * Mignan and Broccardo [2019] A. Mignan and M. Broccardo.  One neuron versus deep learning in aftershock prediction.  _Nature_ 574, E1 (2019).
  * Traut et al. [2022] N. Traut, K. Heuer, G. Lemaître, et al.  Insights from an autism imaging biomarker challenge: promises and threats to biomarker discovery.  _NeuroImage_ 255, 119171 (2022).
  * Andrzejak et al. [2001] R. G. Andrzejak, K. Lehnertz, F. Mormann, et al.  Indications of nonlinear deterministic and finite-dimensional structures in time series of brain electrical activity: Dependence on recording region and brain state.  _Physical Review E_ 64, 061907 (2001).
  * Subasi and Ismail Gursoy [2010] A. Subasi and M. Ismail Gursoy.  EEG signal classification using PCA, ICA, LDA and support vector machines.  _Expert Systems with Applications_ 37 (2010).
  * Fulcher et al. [2013] B. D. Fulcher, M. A. Little, and N. S. Jones.  Highly comparative time-series analysis: The empirical structure of time series and their methods.  _Journal of The Royal Society Interface_ 10, 20130048 (2013).
  * Fulcher [2018] B. D. Fulcher.  Feature-Based Time-Series Analysis.  In _Feature Engineering for Machine Learning and Data Analytics_. CRC Press (2018).
  * Lubba et al. [2019] C. H. Lubba, S. S. Sethi, P. Knaute, et al.  Catch22: CAnonical Time-series CHaracteristics.  _Data Mining and Knowledge Discovery_ 33, 1821 (2019).
  * Nadeau and Bengio [2003] C. Nadeau and Y. Bengio.  Inference for the Generalization Error.  _Machine Learning_ 52, 239 (2003).
  * Henderson [2022] T. Henderson.  _correctR: Corrected Test Statistics for Comparing Machine Learning Models on Correlated Samples_ (2022).  R package version 0.1.2.
  * Poldrack et al. [2016] R. A. Poldrack, E. Congdon, W. Triplett, et al.  A phenome-wide examination of neural and cognitive function.  _Scientific Data_ 3, 1 (2016).
  * Aquino et al. [2020] K. M. Aquino, B. D. Fulcher, L. Parkes, K. Sabaroedin, and A. Fornito.  Identifying and removing widespread signal deflections from fmri data: Rethinking the global signal regression problem.  _Neuroimage_ 212, 116614 (2020).
  * Desikan et al. [2006] R. S. Desikan, F. Ségonne, B. Fischl, et al.  An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest.  _Neuroimage_ 31, 968 (2006).
  * Fischl et al. [2002] B. Fischl, D. H. Salat, E. Busa, et al.  Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain.  _Neuron_ 33, 341 (2002).
  * Henderson and Fulcher [2022] T. Henderson and B. D. Fulcher.  Feature-Based Time-Series Analysis in R using the theft Package (2022).
  * Pedregosa et al. [2011] F. Pedregosa, G. Varoquaux, A. Gramfort, et al.  Scikit-learn: Machine learning in Python.  _Journal of Machine Learning Research_ 12, 2825 (2011).
  * Leek and Storey [2011] J. T. Leek and J. D. Storey.  The joint null criterion for multiple hypothesis tests.  _Statistical Applications in Genetics and Molecular Biology_ 10 (2011).
  * Wu et al. [2023] R. Wu, A. Der, and E. J. Keogh.  When is Early Classification of Time Series Meaningful?  _IEEE Transactions on Knowledge and Data Engineering_ 35, 3253 (2023).
  * Quaak et al. [2021] M. Quaak, L. van de Mortel, R. M. Thomas, and G. van Wingen.  Deep learning applications for the classification of psychiatric disorders using neuroimaging data: systematic review and meta-analysis.  _NeuroImage: Clinical_ 30, 102584 (2021).
  * Zeng et al. [2018] L.-L. Zeng, H. Wang, P. Hu, et al.  Multi-site diagnostic classification of schizophrenia using discriminant deep learning with functional connectivity mri.  _EBioMedicine_ 30, 74 (2018).
  * de Filippis et al. [2019] R. de Filippis, E. A. Carbone, R. Gaetano, et al.  Machine learning techniques in a structural and functional mri diagnostic approach in schizophrenia: a systematic review.  _Neuropsychiatric disease and treatment_ p. 1605 (2019).
  * Fulcher and Jones [2017] B. D. Fulcher and N. S. Jones.  Hctsa: A Computational Framework for Automated Time-Series Phenotyping Using Massive Feature Extraction.  _Cell Systems_ 5, 527 (2017).
  * Aquino et al. [2022] K. M. Aquino, B. Fulcher, S. Oldham, et al.  On the intersection between data quality and dynamical modelling of large-scale fMRI signals.  _NeuroImage_ 256, 119051 (2022).