$\widetilde{\tau}\subseteq\left\llbracket\psi\right\rrbracket^{\mathcal{X}}$,
which by the inductive hypothesis amounts to
$\mathcal{M}(\mathcal{X}),\widetilde{\sigma}\vDash\Box\psi$. Note that we have
equivalences in both directions of the proof, so the bi-implication is
obtained “for free” for this part.

Case $\gamma(\phi,\psi)$: Firstly, suppose that
$\mathcal{M}(\mathcal{X}),\widetilde{\sigma}\vDash\gamma(\phi,\psi)$. Then
there exists a $\pm$-path
$\pi:\\{0,\dots,k\\}\stackrel{{\scriptstyle\pm}}{{\to}}\widetilde{\mathrm{K}}$
such that $\pi(0)=\widetilde{\sigma}$,
$\pi(\\{1,\ldots,k-1\\})\subseteq\left\llbracket\phi\right\rrbracket^{\mathcal{M}}$
and $\pi(k)\in\left\llbracket\psi\right\rrbracket^{\mathcal{M}}$.

By inductive hypothesis and Lemma 3,
$\pi(j)\subseteq\left\llbracket\phi\right\rrbracket^{\mathcal{X}}$ for
$j\in\\{1,\dots,k-1\\}$ and
$\pi(k)\subseteq\left\llbracket\psi\right\rrbracket^{\mathcal{X}}$.

For any two cells $\widetilde{\sigma}\,\widetilde{\preceq}\,\widetilde{\tau}$,
we have $b_{\sigma}\in\widetilde{\sigma}\subseteq\tau$ (recall that
$b_{\sigma}$ indicates the _barycentre_ of $\sigma$). So by Proposition 14
(where $b_{\sigma}$ and $b_{\tau}$ play the role of $y$ and $x$ respectively
in the statement of the proposition) there exists a linear path connecting
$b_{\sigma}\in\tau$ to $b_{\tau}\in\widetilde{\tau}$ and mapping the interval
$(0,1)$ to $\widetilde{\tau}$. Using this fact, for every $i=1,\dots k$ we can
find a linear path $\pi^{\prime}_{i}$ connecting $b_{\pi(i-1)}$ to
$b_{\pi(i)}$ with $\pi^{\prime}_{i}((0,1))\subseteq\pi(i-1)$ or
$\pi^{\prime}_{i}((0,1))\subseteq\pi(i)$—depending on whether
$\pi(i-1)\,\widetilde{\preceq}\,\pi(i)$ or
$\pi(i)\,\widetilde{\preceq}\,\pi(i-1)$. Concatenating these paths, we obtain
a path $\pi^{\prime}:=\pi^{\prime}_{1};\dots;\pi^{\prime}_{k}$ from
$b_{\pi(0)}$ to
$b_{\pi(k)}\in\left\llbracket\psi\right\rrbracket^{\mathcal{X}}$ such that
$\pi^{\prime}((0,1))\subseteq\pi(1)\cup\dots\cup\pi(k-1)\subseteq\left\llbracket\phi\right\rrbracket^{\mathcal{X}}$.
This shows that $\mathcal{X},b_{\pi(0)}\vDash\gamma(\phi,\psi)$, and so by
Lemma 3 since $x,{b_{\pi(0)}}\in\widetilde{\sigma}$ we have
$\mathcal{X},x\vDash\gamma(\phi,\psi)$.

Conversely, assume that $\mathcal{X},x\vDash\gamma(\phi,\psi)$, which by Lemma
7 amounts to the existence of a PL-path $\pi:[0,1]\stackrel{{\scriptstyle
PL}}{{\to}}P$ such that $\pi(0)=x$,
$\pi(1)\in\left\llbracket\psi\right\rrbracket^{\mathcal{X}}$ and
$\pi((0,1))\in\left\llbracket\phi\right\rrbracket^{\mathcal{X}}$. $\pi$ is a
simplicial path by Lemma 8, so there exist points
$s_{0}=0<s_{1}<\dots<s_{l}=1$ such that each $\pi((s_{i},s_{i+1}))$ is fully
contained in a single cell. In particular, we can find cells
$\widetilde{\tau}_{1},\dots,\widetilde{\tau}_{l},\widetilde{\tau}^{\prime}_{0},\dots,\widetilde{\tau}^{\prime}_{l}$
such that $\pi((s_{j-1},s_{j}))\subseteq\widetilde{\tau}_{j}$ for
$j=1,\dots,l$, and such that $\pi(s_{j})\in\widetilde{\tau}^{\prime}_{j}$ for
$j=0,\dots,l$. Notice that $\widetilde{\tau}^{\prime}_{0}=\widetilde{\sigma}$,
$\widetilde{\tau}^{\prime}_{l}\subseteq\left\llbracket\psi\right\rrbracket^{\mathcal{X}}$
(since $\pi(1)\in\widetilde{\tau}^{\prime}_{l}$) and
$\widetilde{\tau}_{i},\widetilde{\tau}^{\prime}_{j}\subseteq\left\llbracket\phi\right\rrbracket^{\mathcal{X}}$
for $i=1,\dots,l$ and $j=1,\dots,l-1$ (since they contain points in
$\left\llbracket\phi\right\rrbracket^{\mathcal{X}}$).

Observe that
$\pi(s_{j})\in\widetilde{\tau}^{\prime}_{j}\cap\mathcal{C}(\widetilde{\tau}_{j+1})$
for $j=0,\dots,l-1$, and so by Lemma 15 we have
$\widetilde{\tau}^{\prime}_{j}\,\widetilde{\preceq}\widetilde{\tau}_{j+1}$.
With a similar argument, we also have
$\widetilde{\tau}^{\prime}_{j}\,\widetilde{\preceq}\,\widetilde{\tau}_{j}$ for
$j=1,\dots,l$. Rewriting the previous conditions we have
$\widetilde{\tau}^{\prime}_{0}\,\widetilde{\preceq}\,\widetilde{\tau}_{1}\,\widetilde{\succeq}\,\widetilde{\tau}^{\prime}_{1}\,\widetilde{\preceq}\,\widetilde{\tau}_{2}\,\widetilde{\succeq}\,\dots\,\widetilde{\succeq}\,\widetilde{\tau}^{\prime}_{l}$,
which by definition means that the sequence
$\left\langle\widetilde{\tau}^{\prime}_{0},\widetilde{\tau}_{1},\widetilde{\tau}^{\prime}_{1},\widetilde{\tau}_{2},\widetilde{\tau}^{\prime}_{2},\dots,\widetilde{\tau}_{l},\widetilde{\tau}^{\prime}_{l}\right\rangle$
is a $\pm$-path of $\mathcal{M}(\mathcal{X})$. By previous considerations
together with the inductive hypothesis applied to $\phi$ and $\psi$, we also
have that $\widetilde{\tau}^{\prime}_{0}=\widetilde{\sigma}$, that
$\widetilde{\tau}^{\prime}_{l}\in\left\llbracket\psi\right\rrbracket^{\mathcal{M}}$
and that
$\widetilde{\tau}_{i},\widetilde{\tau}^{\prime}_{j}\in\left\llbracket\phi\right\rrbracket^{\mathcal{M}}$
for $i=1,\dots,l$ and $j=1,\dots,l-1$. Thus we have
$\mathcal{M}(\mathcal{X}),\widetilde{\sigma}\vDash\gamma(\phi,\psi)$, as
desired.

Theorem 10. Logical equivalence is a simplicial bisimulation.

######  Proof A.13.

In the proof, for $S$ a set of logically equivalent points, we call
“characteristic formula” of $S$ the characteristic formula of the equivalence
class that includes $S$; similarly, we also speak of the “characteristic
formula” of a point $x$.

Consider two points $x$ and $y$, with $x\equiv y$. Let us look at the
conditions of Definition 6. First observe that, since $\phi$ can be an atomic
proposition symbol, Condition 1 holds. We only prove Condition 2, as the proof
of Condition 3 follows the same pattern.

Equivalently, we shall prove, by induction on $k$, the following statement:
for each $k\geq 1$, for each pair $x,y$ with $x\equiv y$, for each path
$\pi_{x}$ with $\pi_{x}(0)=x$, points $s_{0},\ldots,s_{k}$, and cells
$\widetilde{\sigma_{1}},\ldots,\widetilde{\sigma_{k}}$ making $\pi_{x}$ a
simplicial path according to Definition 3, there is a simplicial path
$\pi_{y}$ with $\pi_{y}(0)=y$ and $\pi_{x}\widehat{\equiv}\pi_{y}$.

To ease readability, below, given the data above, we let
$\phi_{1},\dots,\phi_{k}$ be the characteristic formulas of the sets
$\pi_{x}((s_{0},s_{1})),\dots,\pi_{x}((s_{k-1},s_{k}))$ respectively.
Moreover, let $\phi^{\prime}_{0},\dots,\phi^{\prime}_{k}$ be the
characteristic formulas of the points $\pi_{x}(s_{0}),\dots,\pi_{x}(s_{k})$
respectively.

Next, the proof proceeds by induction on $k$.

For $k=1$, observe that, by Definition 3, we have
$\mathcal{X},x\vDash\gamma(\phi_{1},\phi^{\prime}_{1})$. By $x\equiv y$, we
have $\mathcal{X},y\vDash\gamma(\phi_{1},\phi^{\prime}_{1})$. By Lemma 7,
there is a _piecewise linear_ path $\pi_{y}$ with $\pi_{y}(0)=y$,
$\pi_{y}((0,1))\subseteq\left\llbracket\phi_{1}\right\rrbracket^{\mathcal{X}}$,
and
$\pi_{y}(1)\in\left\llbracket\phi^{\prime}_{1}\right\rrbracket^{\mathcal{X}}$.
By Lemma 8, $\pi_{y}$ is simplicial. Note that, since all the $\phi_{i}$ and
$\phi^{\prime}_{i}$ are characteristic formulas of equivalence classes, all
the points in $\pi_{x}((0,1))\cup\pi_{y}((0,1))$ are logically equivalent, and
$\pi_{x}(1)\equiv\pi_{y}(1)$. Therefore, $\pi_{x}\hat{\equiv}\pi_{y}$.

For $k>1$, consider the sub-paths $\pi_{x}^{a}={\pi_{x}}^{[0,s_{1}]}$ and
$\pi_{x}^{b}={\pi_{x}}^{[s_{1},1]}$. By Lemma 16, both sub-paths are
simplicial. By the previous case, there is a simplicial path $\pi_{y}^{a}$
with $\pi_{x}^{a}\hat{\equiv}\pi_{y}^{a}$. In particular, we have
$\pi_{x}^{a}(1)\equiv\pi_{y}^{a}(1)$. Noting that
$\pi_{x}^{a}(1)=\pi_{x}(s_{1})=\pi_{x}^{b}(0)$, we now apply the inductive
hypothesis to the points $\pi_{x}^{b}(0)$, $\pi_{y}^{a}(1)$ and the simplicial
path $\pi_{x}^{b}$, obtaining the simplicial path $\pi_{y}^{b}$ with
$\pi_{y}^{b}(0)=\pi_{y}^{a}(1)$ and $\pi_{x}^{b}\hat{\equiv}\pi_{y}^{b}$. Let
$\pi_{y}=\pi_{y}^{a};\pi_{y}^{b}$. By Lemma 16, $\pi_{y}$ is simplicial, and
we have
$\pi_{x}=\pi_{x}^{a};\pi_{x}^{b}\hat{\equiv}\pi_{y}^{a};\pi_{y}^{b}=\pi_{y}$,
proving the thesis.

Theorem 11. Each simplicial bisimulation is included in logical equivalence.

######  Proof A.14.

Given a simplicial bisimulation $\sim$, we need to show that for every formula
$\phi$ and all points $x$ and $y$, if $x\sim y$ then
$\mathcal{X},x\models\phi\iff\mathcal{X},y\models\phi$. We use induction on
the structure of $\phi$. The cases for atomic propositions and Boolean
operations are trivial, thus omitted. The case for the $\Box$ operator is also
omitted, as $\Box$ is derived from $\gamma$ by Theorem 5.

Suppose $x\sim y$ and $\mathcal{X},x\vDash\phi=\gamma(\phi_{1},\phi_{2})$. By
Lemma 7, there is a piecewise linear path $\pi_{x}$ with $\pi_{x}(0)=x$,
$\pi_{x}((0,1))\subseteq\left\llbracket\phi_{1}\right\rrbracket^{\mathcal{X}}$,
and $\pi_{x}(1)\in\left\llbracket\phi_{2}\right\rrbracket^{\mathcal{X}}$. By
Lemma 8, $\pi_{x}$ is simplicial. By Condition 2 of Definition 6, and the fact
that $x\sim y$, there is a simplicial path $\pi_{y}$ with $\pi_{y}(0)=y$, such
that $\pi_{x}\hat{\sim}\pi_{y}$. For each $r\in(0,1)$, we have
$\pi_{x}(r)\sim\pi_{y}(r)$ and $\mathcal{X},\pi_{x}(r)\models\phi_{1}$. By
induction hypothesis, $\mathcal{X},\pi_{y}(r)\vDash\phi_{1}$, therefore
$\pi_{y}((0,1))\subseteq\left\llbracket\phi_{1}\right\rrbracket^{\mathcal{X}}$.
We also have $\mathcal{X},\pi_{x}(1)\vDash\phi_{2}$ and
$\pi_{x}(1)\sim\pi_{y}(1)$. Thus, by induction hypothesis,
$\pi_{y}(1)\in\left\llbracket\phi_{2}\right\rrbracket^{\mathcal{X}}$. Summing
up, we have $\mathcal{X},y\vDash\gamma(\phi_{1},\phi_{2})$, concluding one
direction of our proof. Next, we should prove that conversely, if
$\mathcal{X},y\vDash\gamma(\phi_{1},\phi_{2})$ then
$\mathcal{X},x\vDash\gamma(\phi_{1},\phi_{2})$. The argument is similar to the
other case, using Condition 3 of Definition 6 instead of Condition 2.


# Efficient prediction of attosecond two-colour pulses from an X-ray free-
electron laser with machine learning

Karim K. Alaa El-Din  Blackett Laboratory, Imperial College London, SW7 2AZ,
London, UK<EMAIL_ADDRESS>Oliver G. Alexander  Blackett
Laboratory, Imperial College London, SW7 2AZ, London, UK  Leszek J. Frasinski
Blackett Laboratory, Imperial College London, SW7 2AZ, London, UK  Florian
Mintert  Blackett Laboratory, Imperial College London, SW7 2AZ, London, UK
Helmholtz-Zentrum Dresden-Rossendorf, Bautzner Landstraße 400, 01328 Dresden,
Germany  Zhaoheng Guo  SLAC National Accelerator Laboratory, Menlo Park,
California 94025, USA  Joseph Duris  SLAC National Accelerator Laboratory,
Menlo Park, California 94025, USA  Zhen Zhang  SLAC National Accelerator
Laboratory, Menlo Park, California 94025, USA  David B. Cesar  SLAC National
Accelerator Laboratory, Menlo Park, California 94025, USA  Paris Franz  SLAC
National Accelerator Laboratory, Menlo Park, California 94025, USA  Taran
Driver  SLAC National Accelerator Laboratory, Menlo Park, California 94025,
USA  Peter Walter  SLAC National Accelerator Laboratory, Menlo Park,
California 94025, USA  James P. Cryan  SLAC National Accelerator Laboratory,
Menlo Park, California 94025, USA  Agostino Marinelli  SLAC National
Accelerator Laboratory, Menlo Park, California 94025, USA  Jon P. Marangos
Blackett Laboratory, Imperial College London, SW7 2AZ, London, UK  Rick
Mukherjee  Blackett Laboratory, Imperial College London, SW7 2AZ, London, UK

###### Abstract

X-ray free-electron lasers are sources of coherent, high-intensity X-rays with
numerous applications in ultra-fast measurements and dynamic structural
imaging. Due to the stochastic nature of the self-amplified spontaneous
emission process and the difficulty in controlling injection of electrons,
output pulses exhibit significant noise and limited temporal coherence.
Standard measurement techniques used for characterizing two-coloured X-ray
pulses are challenging, as they are either invasive or diagnostically
expensive. In this work, we employ machine learning methods such as neural
networks and decision trees to predict the central photon energies of pairs of
attosecond fundamental and second harmonic pulses using parameters that are
easily recorded at the high-repetition rate of a single shot. Using real
experimental data, we apply a detailed feature analysis on the input
parameters while optimizing the training time of the machine learning methods.
Our predictive models are able to make predictions of central photon energy
for one of the pulses without measuring the other pulse, thereby leveraging
the use of the spectrometer without having to extend its detection window. We
anticipate applications in X-ray spectroscopy using XFELs, such as in time-
resolved X-ray absorption and photoemission spectroscopy, where improved
measurement of input spectra will lead to better experimental outcomes.

## Introduction

In recent years, X-ray free-electron lasers (XFELs) [1, 2, 3] have emerged as
a versatile tool for research with applications ranging from damage-free
dynamic imaging of molecules[4] and proteins [5, 6, 7], new spectroscopic
methods for quantum chemistry [8, 9] and resonant X-ray spectroscopy of
nanostructures in condensed matter [10, 11]. The versatility of XFELs is based
on their tunability, brightness and very short pulse durations, which make the
tracking of ultra-fast dynamics of electrons in matter feasible.

XFEL sources generate X-ray pulses by accelerating electron bunches to
relativistic speeds in a linear accelerator of radiofrequency (RF) cavities
and allowing them to interact with magnetic fields generated by an
undulator[1, 2, 3], see Fig. 1. An XFEL can emit coherent or partially
coherent radiation because of a favourable self-organization of the electrons
in a relativistic beam as it passes through an appropriately tuned undulator.
Different configurations are chosen that lead to the modulation of the phase
space for the electron bunch and lasing. This can be used to generate pulses
with different properties. Using an additional pre-modulation of the electron
beam energy in a short wiggler section, followed by phase space manipulation
to transfer the energy into a very short duration high electron current, leads
to so-called enhanced SASE that results in sub-femtosecond pulses of the kind
studied here [12]. SASE and enhanced SASE pulse are important techniques in
ultrafast science [13], where dynamics can be resolved using pump-probe
configurations with synchronization to infra-red or optical laser fields [14,
6] or by using two-pulse XFEL modes [15, 16, 17]. Despite the versatility of
XFELs in creating two-colour pulses in the femtosecond regime [18], single-
shot variation of the pulse energy is significant; for example, photon energy
fluctuation of more than 1% of the mean, pulse energy up to 100% of the mean
and bandwidth more than 20% of the mean are common in existing machines.
Multiple factors contribute to the instability of output X-ray properties. The
working principle of XFEL machines relies on SASE, which is inherently a
stochastic process, with amplification seeded broadband emission from noise in
the distribution of electrons in the bunch [19]. In the case of traditional
SASE operation, there are several temporal spikes within the width of the
pulse that are not coherent with each other and are amplified, producing only
partial longitudinal coherence across the XFEL pulse. This is compounded by
fluctuations in the RF amplitudes or RF phases, which can translate to
variation of the spatial and energy distribution of the electrons within a
bunch.

Techniques like XFEL seeding and optical active stabilization may improve
stability, but the issue of temporal fluctuations is still relevant at the
few-femtosecond level. Alternatively, one can also circumvent issues of
unstable pulse properties by performing a full X-ray characterization for each
XFEL shot. However, single-shot characterization of XFEL pulses requires
higher-dimensional inputs, such as the X-ray spectrum, which are obtained in a
data expensive manner e.g. using an X-ray spectrometer with a CCD image
readout. In addition to the slow and invasive diagnostics, the processing of
large volumes of image data, given inevitable limits to computational power
and data transfer rates, restricts the rate of characterization [20, 21, 22].
Diagnostics in current machines operate at kHz repetition rates, and
technological advances in high speed diagnostics must be accompanied by
increased efficiency to reduce complexity and cost. An interesting solution to
the issue of slow characterization of XFEL pulses was suggested in [23], where
machine learning techniques were used to make accurate predictions of XFEL
properties using data collected solely from fast diagnostics. The key concept
relies on exploiting the correlation of various XFEL properties such as photon
energy and spectral shape of the X-ray pulses with data that can be acquired
at a higher repetition rate, such as electron beam properties. Since the
detailed modelling of every experimental aspect that determines this
correlation is currently out of reach, machine learning methods can prove to
be extremely useful in this context, as further illustrated in [24]. Whilst
the quantum fluctuations associated with SASE will not be amenable in
principle to machine learning, the complex interplay of the other fluctuating
parameters gives some hope that machine learning strategies can be applied to
predict the X-ray parameters with improved fidelity.

Figure 1: (a) Diagram of the XFEL configuration for two-colour X-ray pulse
generation: An electron bunch is modulated in energy-time phase space to yield
a high peak current that propagates between two undulator sections separated
by a chicane that introduces delay between two pulses. In each undulator
section, self-amplified spontaneous emission (SASE) generates a bright,
coherent X-ray pulse. A CCD camera is used to measure the spectrum of the two
pulses. (b) Diagnostics are used to measure the energies of the two-colour
XFEL pulse $y(\mathbf{x})$ which depend on the input feature vector
$\mathbf{x}$. Both $y(\mathbf{x})$ and $\mathbf{x}$ are used to build the
prediction model, which consists of three main steps: pre-processing of data,
feature extraction, and training/validating/testing of the prediction model.
Two different prediction models were used in this work: neural networks and
decision tree based on gradient boosting classifier. (c) An optimized neural
network or gradient boosting classifier is applied directly to real-time
experiments for efficient prediction of central photon energies for two-colour
XFEL pulses.

In this work, we use techniques of supervised learning to make efficient
predictions of central photon energies for attosecond fundamental and harmonic
pulses with high fidelity that can be applied to any XFEL facility. Enhanced
SASE is realised by manipulation of the electron bunch spikes from the
photoinjector with the undulator split into two sections for radiation of
$\omega$ and $2\omega$ frequencies [25]. By combining feature selection
analysis with artificial neural networks/decision trees, we reduce the
dimensionality of the entire input space to the most relevant ones. This leads
to a simpler neural network architecture and optimal decision trees that make
accurate predictions for real experimental data while enhancing the training
efficiency when compared to [23]. Moreover, despite XFEL beamlines being
typically designed with the flexibility to allow for different experimental
configurations (targets, diagnostics, etc.), at current facility beamlines it
is not usually possible to measure the X-ray spectrum before and after a
sample. Many experiments are also unable to measure multiple pulses
simultaneously, due to the limited spectral range of available spectrometers.
One of the key results of our work is the intriguing possibility of using
machine learning methods to predict the photon energy for the second harmonic
pulse without relying on the measurements of the fundamental pulse. Thus our
methods offer a more pragmatic approach to maximising useful information from
available resources whilst adding little experimental overhead.

## Building the prediction model

A prediction model mathematically connects the output variables to the input
parameters. This mathematical function is often non-trivial, especially for
noisy experiments, which exhibit large variance of the affected parameters and
variables. This leads to difficulties in discriminating between noise and
signal, while further establishing an upper bound on the quality of
predictions we can achieve. Naturally, the quality of the model is benchmarked
by its ability to make successful predictions for future measurements.

Figure 1 illustrates machine learning of the prediction model. The objective
is to predict the pulse characterization y from the diagnostics x. There are
three main stages to building the theoretical prediction model. The first step
is to perform pre-processing on the raw experimental data, which mainly
involves filtering and normalizing the data [26]. Here, filtering implies
removing outlier events, such as events that correspond to low variance or
based on not properly recorded measurements. The next step is to randomly
split the pre-processed data into three different data sets: $70\%$ of the
data set used for training to fit different models, $15\%$ of the data set
used for testing while another $15\%$ used for validation. The models chosen
for this work are artificial neural networks [27] and gradient boosting [28].
We train, validate and benchmark the performance of the prediction models on
the test set. Later in this work, the performance of the machine learning
methods are compared with a simpler model, namely a linear regression model
[29]. The final step is to optimize the prediction model in terms of its
training cycle period. For this, it is important to identify the most relevant
input features that contribute to the prediction of the pulse properties,
especially since an unnecessarily large number of input features can slow down
the fitting of estimators as well as decrease the quality of model predictions
by over-fitting. The reduced input space leads to a simpler and more robust
prediction model.

## Results

### Reducing the dimensionality of input space

The goal is to identify the most relevant set of input features, which in this
case are the XFEL electron beam properties, by assessing their importance in
the prediction of the output. Typically, a few hundred parameters are recorded
for each event, including measurements of the electron beam properties, basic
photon diagnostics (such as gas detectors for the pulse energy) and large
numbers of other environmental variables. Many of the environmental features
are collected at a reduced rate of 1 Hz and therefore are only measuring
slower fluctuations. This is done to reduce data flow rates, as these
variables are generally uninformative at high repetition rates but could, in
principle, be measured on every shot. A lot of these parameters such as
environmental variables are empirically known to be disconnected from the XFEL
operation and thus have no predictive value. These are systematically removed
to reduce the total number of input features for an event from hundreds to
$N\simeq 80$. Focusing on the remaining features, especially with those that
have large fluctuations, it is a priori unclear whether they are expected to
have predictive value. For such instances, it is useful to perform a thorough
statistical analysis on the remaining features and rank them in order of their
relevance using the permutation feature importance function [30].

Figure 2: Panel (a) depicts the permutation importance function $I_{j}$ for a
particular input parameter $j$ while predicting the central photon energy of
the second pulse ($E_{2}$) using neural networks. Mean absolute error
$\mathcal{M}$ (solid blue line, in $eV$) is plotted for a varying number of
input features. Panel (b) is a scatter plot that compares the measured values
of $E_{2}$ with the predicted values of neural networks. Predictions of the
reduced input space (red dots) agree with the full input space (blue dots)
with a mean absolute error of $\mathcal{M}=2.48$ eV.

Before describing the importance function, we define the input matrix denoted
by $\tilde{\mathbf{x}}$ whose dimensions are $S$ (total number of events)
$\times$ $N$ (input features for each event)111Throughout this work, the tilde
will be used to indicate that the data have been normalized to zero mean and
unit standard deviation, see [26].. Thus, for $i^{\text{th}}$ event, the row
vector has $N$ input features denoted by the vector
$\mathbf{\tilde{x}}_{i}=(\tilde{x}^{1}_{i},\tilde{x}^{2}_{i},\ldots,\tilde{x}^{N}_{i})$
while for the $j^{\text{th}}$ input feature, the column vector has $S$ events
denoted by
$\tilde{\mathbf{x}}^{j}=(\tilde{x}^{j}_{1},\tilde{x}^{j}_{2},\ldots,\tilde{x}^{j}_{S})^{\mathsf{T}}$.
The mean absolute error calculated over $S$ events is given as

$\mathcal{M}(\tilde{\mathbf{x}},N)=\frac{1}{S}\sum\limits^{S}_{i=1}|\tilde{Y}_{i}-f(\tilde{\mathbf{x}}_{i},N)|,$
(1)

where $\tilde{Y}_{i}$ denotes the output for the $i^{\text{th}}$ event and
$f(\tilde{\mathbf{x}}_{i},N)$ is the estimator for the output observable
generated using the input vector $\mathbf{\tilde{x}}_{i}$. The relevance of a
particular $j^{\text{th}}$ input feature is given using the normalized
permutation feature importance function [30] which is denoted here by $I_{j}$.
It measures the increase in the mean absolute error when the $j^{\text{th}}$
input feature is randomly replaced by an incorrect one and is defined as
follows,

$I_{j}=\frac{1}{\mathcal{M}(\tilde{\mathbf{x}},N)}\left(\frac{1}{R}\sum\limits^{R}_{r=1}\mathcal{M}(\mathbf{p}^{r}(j),N)-\mathcal{M}(\tilde{\mathbf{x}},N)\right),$
(2)

where
$\mathbf{p}^{r}(j)=(\mathbf{p}_{1}^{r}(j),\mathbf{p}_{2}^{r}(j),\ldots,\mathbf{p}_{S}^{r}(j))^{\mathsf{T}}$
is a matrix of the $r^{\text{th}}$ permutation to the $j^{\text{th}}$ input
feature. Its individual row vectors are denoted as
$\mathbf{p}_{i}^{r}(j)=(p^{1,r}_{i}(j),p^{2,r}_{i}(j)\ldots,p^{N,r}_{i}(j))$.
These vectors have elements where only the $j^{\text{th}}$ input feature is
replaced using a permutation operator $\Pi^{r}$ which gives the element

$p_{i}^{k,r}(j)=\begin{cases}\tilde{x}_{i}^{k}&\quad\text{if}\quad k\neq j,\\\
[\Pi^{r}(\tilde{\mathbf{x}}^{j})]_{i}&\quad\text{if}\quad k=j.\\\ \end{cases}$
(3)

Here $\Pi^{r}(\tilde{\mathbf{x}}^{j})$ gives the $r^{\text{th}}$ permutation
from a series of random permutations applied to column vector
$\tilde{\mathbf{x}}^{j}$. The $i^{\text{th}}$ value of the resultant vector
obtained after the permutation is given by the element
$[\Pi^{r}(\tilde{\mathbf{x}}^{j})]_{i}$. All other column vectors
$\tilde{\mathbf{x}}^{k\neq j}$ remain unaltered.

Figure 2(a) is a plot which ranks the input features using the permutation
feature importance $I_{j}$ while predicting the central photon energy of the
second pulse $E_{2}$ using only non-pulse measurement data. The relevance of a
particular input feature is ranked with descending values of $j$ and the plot
of the mean absolute error
($\mathcal{M}(j)=\mathcal{M}(\tilde{\mathbf{x}},j)$) reaches its lowest value
for the top ten relevant features, most of which are related to the electron
beam properties. Adding further features leads to over-fitting, as is seen
with the rise in $\mathcal{M}(j)$ for higher $j$ values.

Figure 2(b) shows a scatter plot which compares the measured values of the
central photon energy of the pulse $E_{2}$ with the predicted values estimated
by the neural-network. For a perfect predictor, the points would all lie
exactly along the diagonal, with deviations from this distribution indicating
reduced accuracy of prediction. The blue and red scatter points correspond to
full input space ($M=N=87$) and reduced input space ($M=10$) respectively. In
both cases, the overall quality of predictions was identical, with a mean
absolute error of 2.48 eV. Thus, we can perform training of simpler estimators
with smaller architectures by using the reduced input space, without
compromising the quality of predictions. By including only the most relevant
features, we introduce a feature-restricted mean absolute error
$\mathcal{M}=\mathcal{M}(\tilde{\mathbf{x}},10)$, which will be used to
estimate the performance of predictor models for the rest of this work. To
further allow for comparability between different prediction targets, we will
proceed by normalizing the mean absolute error $\mathcal{M}$ with respect to
the standard deviation $\sigma$ of the target data. Using this notation, the
results seen in Fig. 2(b) are equivalent to $\mathcal{M}=0.54\sigma$. Whilst
the accuracy of the predictions is modest, this was achieved without addition
probes of electron and X-ray properties to those already in use at LCLS. The
methods employed in this work can be used generally for the prediction of beam
properties with an accuracy that will depend on many different factors such as
the inherent noise in the available data, choice of predicted parameter,
ground-truth accuracy, and experimental configuration. For example, the
results shown in Figs. (S1)-(S2) of the Supplemental Material indicate that
the input-output correlation in the data for the time delay parameter between
the pulses is much higher than that of the central photon energies of the
pulses. The data for Figs. (S1)-(S2) were performed using a completely
different experimental setup [23] and provide $\mathcal{M}\sim 0.2\sigma$ and
$\mathcal{M}\sim 0.3\sigma$ for the prediction of the time delay and central
energies respectively.

### Independent prediction of a single pulse

Figure 3 focuses on predicting the central photon energy of Pulse 2 ($E_{2}$)
using two different detection schemes for the experiment. One setting
corresponds to a configuration of the spectrometer which detects both the
pulses simultaneously (depicted with blue lines) and using the energy of Pulse
1 ($E_{1}$) as an input feature, while the other measures only the second
pulse (depicted with dashed lines). The green dashed line is the prediction of
the central photon energy for Pulse 2 with ANN, while the magenta dashed line
is with LIN model. These predictions are made with experimental data where
different numbers of undulators were used between the pulses. Although both
LIN and ANN models make accurate predictions of $E_{2}$ without the spectral
information of Pulse 1, we find that the accuracy of their predictions depends
on the number of undulators between the pulses. Predictions of $E_{2}$ improve
with increasing number of undulators that are used for generating the second
pulse. One plausible explanation for this is that, as each additional
undulator provides amplification to Pulse 2, the accuracy of central-photon
energy prediction, the ground truth of our prediction models, improves.
Alternatively, this could be understood by how the photon emission energy
depends on the undulator period, the bunch energy and the position within the
bunch initiated by SASE; as the number of undulators increases, the energy is
less determined by the stochastic nature of SASE and is better characterized
by the measured parameters.

It should be noted that although Pulse 2 is a harmonic of Pulse 1 and is
generated from the same electron bunch, the spectrometer was optimized for
Pulse 2 and thus the accuracy in determining $E_{2}$ for both the training and
test data is improved when compared to the setup where both pulses were
measured. Often in experiments, measurement of the energy spectrum of both
pulses simultaneously is not possible due to the limited spectral range of the
spectrometer. Furthermore, it may only be possible to measure photon spectra
after transmission through target samples, which in many settings alter the
spectrum, e.g. due to absorption. This result allows for prediction of the
photon energy without input from the spectrometer, except for training, adds
directly to the capabilities of current XFEL experiments, allowing for
important information about the incoming pulses to be extracted within typical
experimental constraints.

Figure 3: The plot shows the precision with which the central photon energy of
the second pulse ($E_{2}$) can be predicted as a function of the number of
undulators between the pulses for two different detection scenarios: one where
both pulses are measured simultaneously, while for the other only pulse 2 is
measured. Using the former data, ANN is used to predict $E_{2}$ (blue solid
line with circle), while using the latter data, both LIN (magenta dashed line
with circle) and ANN (green dashed line with triangle) were used to predict
$E_{2}$.

## Discussion

Conventional X-ray spectrometers involve high volumes of data and are still
too slow for future XFEL experiments (which will run at MHz repetition rates)
and proposed high data rate models using photo-electron spectrometers [31, 32]
would add significantly to experimental cost and complexity. Another issue is
the limited spectral range of the available spectrometers. In both cases,
machine learning methods can be advantageous as demonstrated in this work.
Although there have been prior works relying on the concept of using data from
the photon spectrometer to train the neural networks, our work suggests that
gradient boosting methods are more efficient (orders of magnitude) than neural
networks in making spectroscopic predictions while giving comparable
predictions. It is well-established that there is strong dependence of the
properties of two-colour pulses on the electron beam parameters. However,
while most of the environmental variables are usually not relevant, it could
be that certain environmental parameters specific to that facility beamline
play a crucial role in making more accurate predictions. One of the challenges
in pre-processing of data used for predictor models is to filter the relevant
features from the redundant ones. In this work, the dimension of the input
parameter space was drastically reduced without having to compromise with the
prediction results using the feature selection analysis. However, the data
collected in the experiment was not tailored to machine learning, and the
electron beam and photon properties recorded were incidentally of use for
predictions and, in future experiments, collection of more relevant electron
beam properties may allow for improved prediction accuracy.

## Methods

### A: Experiment details for attosecond two-colour pulses pulses

In our experiment, data with two pulses at different energies were obtained
from a configuration similar to [12], utilizing an enhanced SASE mode. The
phases between SASE emitting microbunches are not predetermined and, as a
result, the temporal properties are difficult to predict from purely spectral
measurements. The photon energy of the emission is determined by the period of
the undulators, the energy of the electron bunch and the position of the SASE
emission within the bunch. The spatial and energetic distribution of electrons
within the bunch varies on a shot-to-shot basis due to fluctuations in the
electron accelerator. In the two-colour mode, a second set of undulators was
used to produce a second pulse (see Fig. 1), either at the second or third
harmonic of the first, with the emission from the first pulse seeding the
second.

Separation of the X-rays from the electrons due to a difference in their group
velocities, i.e. slippage, was used to create a time delay between the pulses,
for use in a separate pump-probe experiment. With more undulators in the
second section, the slippage is larger at the centre of mass of the second
pulse generation, so the delay is greater. Both pulses are estimated to have
temporal length below 500 attoseconds [12]. Pulses were generated at 120 Hz
with photon energies of approximately 250 eV and using either the second or
third harmonics at 500 and 750 eV respectively, 2–10 eV FWHM bandwidth, and up
to 50 $\mu$J energy in each pulse.

### B: Pre-processing of data

#### Data Filtering

A typical experimental data set will contain many events which are labelled by
$i\in 1,2\ldots,S$, where $S=35000\text{--}40000$. After filtering, the total
number of events in each data set reduces to $S=16000\text{--}32000$ (varying
between the different data sets) that can actually be used for building the
predictive model. For each event, we typically have around 300 recorded input
features that are collected during the experiment. These include environmental
variables such as current and voltage measures for different XFEL machines,
total photon energies of the pulses as measured by gas monitor detectors as
well as electron beam properties at the dump which include electron beam
charge and energy. We remove from this set of features any that take less than
$10$ distinct values across the full dataset. Furthermore, we eliminate any
features that are perfectly correlated (correlation coefficient above 0.995).
The combination of these two methods brings our overall feature count down to
around 80 (depending on the individual data set). Based on the statistical
dispersion of the data, we also remove outlier events which can negatively
impact the prediction results. Thus, any events with features with a median
absolute deviation greater than four are removed. Finally, we impose a lower
limit of $5\leavevmode\nobreak\ \mu$J on the total central photon energy of
the pulse as measured by the gas monitor detectors.

#### Normalisation of data

Let the vector of input features for event $i$ be denoted by
$\mathbf{x}_{i}=(x^{1}_{i},x^{2}_{i},\ldots,x^{N}_{i})$ where $N$ is the total
number of recorded features and the output for this event be denoted by
$Y_{i}$. Then the normalised input and output data are given as

$\displaystyle\tilde{x}^{j}_{i}$ $\displaystyle=$
$\displaystyle(x^{j}_{i}-\mu_{\mathbf{x}^{j}})/\sigma_{\mathbf{x}^{j}}$
$\displaystyle\tilde{Y}_{i}$ $\displaystyle=$
$\displaystyle(Y_{i}-\mu_{\mathbf{Y}})/\sigma_{\mathbf{Y}}$ (4)

Here $\mathbf{x}^{j}=(x^{j}_{1},x^{j}_{2}\ldots,x^{j}_{S})^{\mathsf{T}}$ is a
vector consisting of $j$th input variable from every event and
$\mathbf{Y}=(Y_{1},Y_{2}\ldots,Y_{S})^{\mathsf{T}}$ is the output vector.
Additionally, $\mu$ and $\sigma$ respectively correspond to the mean and
standard deviation of the subscripted data column across all events.

### C: Key Code for top ten input parameters for Fig. 2(a)

Rank |  Label |  Description |  Data rate
---|---|---|---
1 |  ebeam_ebeamLTU450 |  A position measurement of the transverse electron beam orbit in the linac-to-undulator beamline |  120 Hz
2 |  ebeam_ebeamLTU250 |  A position measurement of the transverse electron beam orbit in the linac-to-undulator beamline |  120 Hz
3 |  epics_UND_34_gap_act |  The measured vertical gap between the undulator magnet arrays in one undulator, which is tuned to adjust the K-factor of the undulator |  1 Hz
4 |  epics_UND_36_gap_act |  The measured vertical gap between the undulator magnet arrays in another undulator |  1 Hz
5 |  xgmd_energy |  The pulse energy measured after all attenuation using the total ionisation of an $N_{2}$ filled cell (X-ray gas monitor detector) |  120 Hz
6 |  epics_UND_34_gap_des |  The desired vertical gap between the undulator magnet arrays in the first undulator |  1 Hz
7 |  ebeam_ebeamPkCurrBC2 |  The peak electron bunch current measured in the second bunch compressor |  120 Hz
8 |  epics_GMD_ElectronMesh |  The voltage applied to an electron mesh in the gas monitor detector (GMD), to extract the electrons towards the electrode |  1 Hz
9 |  gmd_energy |  The pulse energy measured after all attenuation using the total ionisation of an Kr filled cell (X-ray gas monitor detector) |  120 Hz
10 |  epics_UND_28_gap_act |  The measured vertical gap between the undulator magnet arrays in a third undulator |  1 Hz
Table 1: Input feature ranking by permutation feature importance for pulse
energy data

### D: ML methods

#### Linear modeling

A linear regression model (LIN) fits a general linear function

$\bar{Y}^{(LIN)}_{i}=\tilde{\mathbf{x}}_{i}\cdot\mathbf{c}+c_{0}$ (5)

across $S$ events. The parameters $\mathbf{c},c_{0}$ are varied to minimize
the residuals-squared, given by

$RS=\frac{\sum\limits^{S}_{i=1}(\tilde{Y}_{i}-\bar{Y}_{i})^{2}}{\sum\limits^{S}_{i=1}(\tilde{Y}_{i}-\tilde{\mu}_{Y})^{2}}.$
(6)

Here, $\tilde{\mu}_{Y}$ is the mean of the normalized labels $\tilde{Y}$ such
that $\tilde{\mu}_{Y}\equiv 0$. We then use the mean absolute error
$\mathcal{M}$ to calculate the model performance. While linear regression
methods can be very useful and simple to implement, they naturally fail with
data that are highly non-linear. Since the generation of XFEL pulses are
highly non-linear processes, it is helpful to use this method to get a sense
of the level of non-linearity in the data set.

#### Gradient Boosting Decision trees

Decision tree learning is a supervised machine learning approach often used
for predicting classification or regression type of problems. A decision tree
is built by splitting the root node (which is at the apex) into subsets, and
this process of splitting continues for each subset recursively until further
splitting does not improve the predictions. The rules for splitting a node are
determined by the classification features. Gradient boosted decision trees is
an ensemble learning method where rather than using a single decision tree to
make predictions, we combine multiple decision trees to enhance the model’s
accuracy. The basic premise of boosting is to combine weak “learners" into a
single strong learner iteratively. The success of the boosting scheme is
evaluated by defining a suitable loss function that is minimized using a
gradient descent scheme.

In our case, the full set of events $S$ forms the root node, which is
subsequently split into subsets $\mathcal{S}_{i}$ and are distinguished based
on the values of different categorical or numerical features. We partition out
the input space into $D$ regions $d\in 1,\ldots,D$ where we split the data
using

$z_{d}(\tilde{\mathbf{x}}_{i})=\begin{cases}1&\quad\text{if}\quad\tilde{\mathbf{x}}_{i}\in
d,\\\ 0&\quad\text{if}\quad\tilde{\mathbf{x}}_{i}\notin d.\\\ \end{cases}$ (7)

By predicting a constant value $h_{d}$ across each of these regions, we can
define the output of the decision tree as

$\bar{y}_{t}(\tilde{\mathbf{x}}_{i};N)=\sum\limits_{d=1}^{D}h_{d}z_{d}(\tilde{\mathbf{x}}_{i}).$
(8)

Here, $h_{d}$ is the average of the target across all points within the region
$d$, and is used as the model output for all points where $z_{d}=1$. The
predictions of an individual decision tree are generally heavily biased, and
thus ensemble methods are often used. Apart from random forests, which use
independent decision tree predictors, gradient boosting (GB) is another
commonly used method where trees are added to the estimator successively and
fitted to the pseudo-residuals of all the previous tree’s predictions. A
gradient boosting regressor[28] is an ensemble method that gives an estimate
$\bar{Y}^{(GB)}_{i}$ from the weighted sum of estimates given by $T$ base
regressors $\bar{y}_{t}(\tilde{\mathbf{x}}_{i};N)$, written as

$\bar{Y}^{(GB)}_{i}=\sum\limits_{t=1}^{T}\gamma_{t}\bar{y}_{t}(\tilde{\mathbf{x}}_{i};N),$
(9)

where we used the decision trees to define our base estimator. The gradient
boosting regressor is then constructed iteratively under consideration of a
differentiable loss function

$\mathcal{L}=\frac{1}{S}\sum\limits^{S}_{i=1}(\tilde{Y}_{i}-\bar{Y}_{i})^{2}.$
(10)

We begin by considering a constant average estimate
$\bar{Y}^{(GB)}_{i,0}=\tilde{\mu}_{Y}=0$, where the subscript 0 indicates that
no estimators have been added yet. We then iterate over $t\in 1,\ldots,T$ and
at each step perform the following:

  1. 1.

For each $i$, find the pseudo-residuals given by

$q_{i,t}=-\frac{\partial\mathcal{L}}{\partial\bar{Y}_{i,t-1}}.$ (11)

  2. 2.

Fit a decision tree estimator $y_{t}(\tilde{\mathbf{x}}_{i};N)$ to the set of
pseudo-residuals.

  3. 3.

Find $\gamma_{t}$ to minimize $\mathcal{L}$ for the new set of estimates

$\bar{Y}_{i,t}=\bar{Y}_{i,t-1}+\gamma_{t}y_{t}(\tilde{\mathbf{x}}_{i}).$ (12)

After adding $T$ base estimators in this manner, we have our fully fitted
estimator $\bar{Y}^{(GB)}_{i}=\bar{Y}_{i,T}$. This approach has the advantage
of focusing on regions of bad prediction and improving them. While many tree
parameters are fit in the algorithm, others are hyperparameters that have to
be specified a priory, such as the number of trees, the number of decisions
per tree, the use of regularization and the number of data points to consider
for each decision. Often the intuitive interpretation of the regressor
obtained from decision trees can be lost when using an ensemble of decision
trees. We found an estimator with 20 trees without specified depth limit and
l2 regularization to yield the best results with only minor overfitting as
seen in Fig. 4. To evaluate the performance of the gradient boosting
estimator, we evaluated the mean absolute error across the test set and
compare it to the performance of the ANN and the linear model.

Figure 4: Convergence of the mean absolute error as function of the number of
events S used for training the decision trees/neural networks for (a)
Prediction of central photon energy of the pulse using [12] and (b) time delay
prediction using [23]. Figure 5: Convergence of mean absolute error as
function of the number of epochs used in the neural networks for (a)
predicting the central photon energy of the pulse using [12] and (b) time
delay prediction using [23].

#### Neural networks

Artificial Neural Networks (ANNs) are one of the most widely used modern
machine learning techniques and have been very successful in making
predictions for various physical systems. In this work, we use Feed-Forward
Neural Networks as we are performing supervised learning on a set of
independent data points. Conceptually, a neural network can be represented by
a graph, with values and biases associated with each node (or neuron) and
weights associated with each edge. We group the nodes into layers, and allow
edges only between nodes of neighbouring layers. The data propagates through
this network layer by layer in one direction (Feed-Forward) only. The overall
architecture of the neural network is defined by the hyperparameters which
include the number of neurons in each layer, number of layers and choice of
activation function applied to the outputs of different nodes. Regularization
schemes and choice of optimizer constitute further hyperparameters, while bias
$b$ and weights $W$ are parameters fit using the backpropagation algorithm.
The last layer must have the same size as the number of prediction labels in
the data, 1 in our case. For each of the $L+1$ layers labelled by $l\in
0,\ldots,L$, we define the node activation by a vector $\mathbf{v}_{l}$, the
node bias by a vector $\mathbf{b}_{l}$, the edge weights for edges between
layers $l$ and $l+1$ by a matrix $\mathbf{W}_{l}$ and the differentiable
activation function for each node in the layer as $a_{l}$. We then perform
forward propagation of the data for event $i$ by setting
$\mathbf{v}^{i}_{0}=\tilde{\mathbf{x}}_{i}$. We then propagate the data using

$\mathbf{v}^{i}_{l+1}=a_{l}\left(\mathbf{W}_{l}\mathbf{v}^{i}_{l}+\mathbf{b}_{l}\right)$
(13)

and use $\bar{Y}^{(ANN)}_{i}=\mathbf{v}^{i}_{L}$ as our estimate of
$\tilde{Y}_{i}$. The crucial task is then to train the estimator by finding
$\mathbf{W}_{l}$ and $\mathbf{b}_{l}$ such that our loss, chosen as
$\mathcal{M}$ is minimized. We initialize these parameters randomly, and then
perform backpropagation with gradient descent, implemented through the Adagrad
algorithm [33]. We used Bayesian optimization to find the optimal neural
network architecture, activation functions, regularization and drop out. This
technique uses Bayesian inference to guess combinations of hyperparameters
that yield the best predictions for the smallest computational cost. We find
that the optimal network sufficient to make accurate predictions for both the
two-pulse delay and the pump-probe energies consists of two hidden layers of
20 cells each. The network is also l2-regularized and there is no drop-out,
leading to no overfitting (Figure 4) and training convergence after few
thousand of epochs (Figure 5). The choice of the activation function on hidden
layers is chosen to be a ReLU (regularized linear unit function). In
combination with the reduced feature count, this results in a substantial
speed-up of model fitting and requires far fewer data to be collected.

## Author contributions statement

The project was conceived by KKA and RM. KKA performed the machine learning
and data analysis guided by RM. The experimental data was provided by ZG, JD,
ZZ, DBC, PF, TD, PW, AM, JPC, and JPM, while OGA did the pre-processing of the
data. RM, KA, OGA, LJF, FM, JPM and JPC contributed to the writing of the
manuscript.

## Acknowledgement

JPM would like to acknowledge EPSRC funding EP/X026094/1. AM would like to
acknowledge support from US Department of Energy (DOE), BES Scientific User
Facilities Division Field Work Proposal 100317; JD and AM were supported by
the Laboratory Directed Research and Development Program in support of the
Panofsky fellowship. The contributions from TD and JPC were supported by the
US DOE, Office of Science, Office of Basic Energy Sciences (BES), Chemical
Sciences, Geosciences, and Biosciences Division (CSGB). Use of the Linac
Coherent Light Source (LCLS), SLAC National Accelerator Laboratory, is
supported by the US DOE, Office of Science, BES, under Contract DE-
AC02-76SF00515.

## Data availability

The raw data for this research was generated at the Linear Coherent Light
Source, both raw and processed datasets are available upon reasonable request
to the corresponding author.

## Code availability

The codes used for this work are available upon reasonable request to the
corresponding author.

## Competing interests

The authors declare no competing interests.

## Supplemental Material

Here we present the results of our ML prediction for central photon energies
using the experimental setup [12] described in the main text and compare it
against the predictions of energies and time delay using data obtained from
another two-colour experiment [23] which has a different modus operandi than
[12]. Although [23] does not utilize enhanced SASE scheme like in [12], both
methods use a variable line spectrometer to measure the X-ray spectrum. To
create the two pulses, a double slotted foil in inserted into the bunch
compressor. In the bunch compressor, there is a space-to-energy mapping, so
the spatial windows spoil the bunch except in two energy regions which are
then the only regions able to lase. As energy maps to time in the undulators,
this space-to-energy mapping becomes a space-to-time mapping for the emission.
The result is reduced total brightness and emission confined to two short
periods, i.e. pulses. The widths of the slits determine the widths of the
pulses and the slits’ separation scales linearly with the delay [34]. The
space-to-energy mapping in the bunch compressor is equally important, and will
jitter with the electron beam energy. Pulses up to 30 $\mu$J were produced in
this way with photon energy centred close to 540 eV and separated by 14 eV.
The repetition rate was 120 Hz, though complete pulse diagnostics operated at
only 60 Hz. The temporal structure of the pulses is retrieved for the double
slotted foil method using XTCAV.

In general, we find higher input-output correlation for the data from [23].
Testing on the highly correlated data with limited non-linearity helps to
benchmark our theoretical prediction models.

##  Appendix A Predicting central photon energies with two-pulse data from
experiment

Figure S1 shows the validity of predicting central photon energies of the
individual pulses ($E_{1}$ and $E_{2}$) using different machine learning
methods. Despite the complex inter-dependence of these energies on the
diagnostics, which is in some cases highly non-linear, the linear regression
model (LIN) makes reasonable predictions of the central photon energy for
either pulse, as seen in Fig. S1(a) and (d).

Both gradient boosting (GB) and artificial neural networks (ANN) make better
predictions than the LIN models for the central photon energies of the
individual pulses as depicted in Fig. S1(b), (e) and Fig. S1(c), (f)
respectively. In general, independent of the prediction model, the mean
absolute error for the predictions of $E_{2}$ are 2.4 times larger than for
$E_{1}$ which can be attributed to the fact that Pulse 2 is the second
harmonic of Pulse 1. Thus, the second pulse will experience effects of
electron bunch energy jitter twice as much compared to Pulse 1 while the
remaining difference may be attributed to the error in our energy
measurements. It is promising to find that the GB model and the ANN model have
similar accuracy in their predictions, especially since we find the GB models
are faster to train compared to ANN models, at least by a factor of three.

Figure S1: Prediction of XFEL energies of the individual pulses for two-colour
data: (a–c) compare the measured values of energies $E_{1}$ with the values
predicted by different ML methods, while (d–f) is the same for $E_{2}$. Top
row panels represent linear regression model (LIN), middle row panels
represent gradient boosting method (GB) and bottom row panels represent neural
networks (ANN). The 2D histogram plots are constructed by grouping the data
into 50 bins along each direction, where the density is indicated by the
intensity of the blue colouring.

##  Appendix B Predictions central photon energy and time delay for the two-
colour data from experiment

Figures S2 and S3 depicts the prediction of the central photon energy of the
second pulse and time delay between the two pulses using the machine learning
methods as used in the main manuscript. While the results agree with [23], it
is more efficient in training time due to reduced input parameter space, which
is the result of our feature analysis. Interestingly, for this data, the
linear model here is sufficient to make accurate predictions with mean
absolute error that is a fraction of the variance of the data. This perhaps
can be understood by considering how the double-slotted foil affects the
electron bunch used to create the pulses. In summary, the data in [23] seems
to be less nonlinear in nature with high input-output correlation compared to
[12].

Figure S2: Comparing the measured values of the higher energy pulse from the
two-colour data of an older operation mode [23] using the prediction from
different ML methods: (a) linear regression model (LIN), (b) gradient boosting
method (GB) and (c) neural networks (ANN). 2D histogram plots are constructed
in similar way as Fig. S1. Note that the energies shown here differ from those
presented in [23] by a small offset owing to a scaling factor but doesn’t
affect the performance of the fit. Figure S3: Comparing the measured values of
time delay for the two-colour data [23] using the prediction from different ML
methods: (a) linear regression model (LIN), (b) gradient boosting method (GB)
and (c) neural networks (ANN). The $\mathcal{M}$ values correspond to 2.39 fs,
1.87 fs, and 1.77 fs for LIN, GB, and ANN, respectively.

## References

  * [1] Emma, P. _et al._ First lasing and operation of an ångstrom-wavelength free-electron laser.  _Nature Photonics_ 4, 641–647 (2010).
  * [2] Ishikawa, T. _et al._ A compact x-ray free-electron laser emitting in the sub-ångström region.  _Nature Photonics_ 6, 540–544 (2012).
  * [3] Allaria, E. _et al._ Two-stage seeded soft-x-ray free-electron laser.  _Nature Photonics_ 7, 913–918 (2013).
  * [4] Glownia, J. M. _et al._ Self-referenced coherent diffraction x-ray movie of ångstrom- and femtosecond-scale atomic motion.  _Physical Review Letters_ 117, 153003 (2016).
  * [5] Seibert, M. M. _et al._ Single mimivirus particles intercepted and imaged with an x-ray laser.  _Nature_ 470, 78–82 (2011).
  * [6] Pande, K. _et al._ Femtosecond structural dynamics drives the trans/cis isomerization in photoactive yellow protein.  _Science_ 352, 725–729 (2016).
  * [7] Chapman, H. N. _et al._ Femtosecond x-ray protein nanocrystallography.  _Nature_ 470, 73–78 (2011).
  * [8] Biggs, J. D., Zhang, Y., Healion, D. & Mukamel, S.  Watching energy transfer in metalloporphyrin heterodimers using stimulated X-ray Raman spectroscopy.  _Proceedings of the National Academy of Sciences of the United States of America_ 110, 15597–15601 (2013).
  * [9] Berrah, N. _et al._ Double-core-hole spectroscopy for chemical analysis with an intense x-ray femtosecond laser.  _Proceedings of the National Academy of Sciences_ 108, 16912–16915 (2011).
  * [10] Wernet, P. _et al._ Orbital-specific mapping of the ligand exchange dynamics of Fe(CO)5 in solution.  _Nature_ 520, 78–81 (2015).
  * [11] Kroll, T. _et al._ Stimulated x-ray emission spectroscopy in transition metal complexes.  _Physical Review Letters_ 120, 133203 (2018).
  * [12] Duris, J. _et al._ Tunable isolated attosecond x-ray pulses with gigawatt peak power from a free-electron laser.  _Nature Photonics_ 14, 30–36 (2020).
  * [13] Young, L. _et al._ Roadmap of ultrafast x-ray atomic and molecular physics (2018).
  * [14] Erk, B. _et al._ Imaging charge transfer in iodomethane upon x-ray photoabsorption.  _Science_ 345, 288–291 (2014).
  * [15] Liekhus-Schmaltz, C. E. _et al._ Ultrafast isomerization initiated by X-ray core ionization.  _Nature Communications_ 6, 1–7 (2015).
  * [16] Barillot, T. _et al._ Correlation-driven transient hole dynamics resolved in space and time in the isopropanol molecule.  _Physical Review X_ 11, 031048 (2021).
  * [17] Picón, A. _et al._ Hetero-site-specific X-ray pump-probe spectroscopy for femtosecond intramolecular dynamics.  _Nature Communications_ 7, 1–6 (2016).
  * [18] Lutman, A. A. _et al._ Experimental demonstration of femtosecond two-color x-ray free-electron lasers.  _Phys. Rev. Lett._ 110, 134801 (2013).
  * [19] Bonifacio, R., De Salvo, L., Pierini, P., Piovella, N. & Pellegrini, C.  Spectrum, temporal structure, and fluctuations in a high-gain free-electron laser starting from noise.  _Physical Review Letters_ 73, 70–73 (1994).
  * [20] Ding, Y. _et al._ Femtosecond x-ray pulse temporal characterization in free-electron lasers using a transverse deflector.  _Phys. Rev. ST Accel. Beams_ 14, 120701 (2011).
  * [21] Harmand, M. _et al._ Achieving few-femtosecond time-sorting at hard x-ray free-electron lasers.  _Nature Photonics_ 7, 215–218 (2013).
  * [22] Kimberg, V. _et al._ Stimulated x-ray raman scattering – a critical assessment of the building block of nonlinear x-ray spectroscopy.  _Faraday Discuss._ 194, 305–324 (2016).
  * [23] Sanchez-Gonzalez, A. _et al._ Accurate prediction of x-ray pulse properties from a free-electron laser using machine learning.  _Nature Communications_ 8, 15461 (2017).
  * [24] Ren, X. _et al._ Temporal power reconstruction for an x-ray free-electron laser using convolutional neural networks.  _Physical Review Accelerators and Beams_ 23, 040701 (2020).
  * [25] Guo, Z. _et al._ Experimental demonstration of attosecond pump-probe spectroscopy with an x-ray free-electron laser (2023).  [Manuscript submitted for publication].
  * [26] Details provided in the supplemental material.
  * [27] Cheng, B. & Titterington, D. M.  Neural Networks: A Review from a Statistical Perspective.  _Statistical Science_ 9, 2–30 (1994).
  * [28] Chen, T. & Guestrin, C.  XGBoost: A scalable tree boosting system.  In _Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ (ACM, New York, NY, USA, 2016).
  * [29] A. Schneider, G. H. & Blettner, M.  Linear regression analysis: part 14 of a series on evaluation of scientific publications.  _Deutsches Arzteblatt international_ 107 (2010).
  * [30] Breiman, L.  Random forests.  _Machine Learning_ 45, 5–32 (2001).
  * [31] Li, K. _et al._ Ghost-imaging-enhanced noninvasive spectral characterization of stochastic x-ray free-electron-laser pulses.  _Communications Physics_ 5, 1–8 (2022).
  * [32] Heider, R. _et al._ Megahertz-compatible angular streaking with few-femtosecond resolution at x-ray free-electron lasers.  _Physical Review A_ 100, 053420 (2019).
  * [33] Pedregosa, F. _et al._ Scikit-learn: Machine learning in python.  _Journal of Machine Learning Research_ 12, 2825–2830 (2011).
  * [34] Ding, Y. _et al._ Generating femtosecond X-ray pulses using an emittance-spoiling foil in free-electron lasers.  _Applied Physics Letters_ 107, 191104 (2015).