# An MDP approach for radio resource allocation
in urban Future Railway Mobile Communication System (FRMCS) scenarios

V. Corlay and J.-C. Sibel  Mitsubishi Electric R&D Centre Europe
Rennes, France
Email: {v.corlay<EMAIL_ADDRESS>
###### Abstract

In the context of railway systems, the application performance can be very
critical and the radio conditions not advantageous. Hence, the communication
problem parameters include both a survival time stemming from the application
layer and a channel error probability stemming from the PHY layer. This paper
proposes to consider the framework of Markov Decision Process (MDP) to design
a strategy for scheduling radio resources based on both application and PHY
layer parameters. The MDP approach enables to obtain the optimal strategy via
the value iteration algorithm. The performance of this algorithm can thus
serve as a benchmark to assess lower complexity schedulers. We show numerical
evaluations where we compare the value iteration algorithm with other
schedulers, including one based on deep Q learning.

###### Index Terms:

Scheduling, application-oriented systems, cross-layer, neural networks, Markov
decision process.

##  I Introduction

On the one hand, the automated train control is a crucial railway service use
case and induces a change of communication paradigm compared to the current
railway system: It might require at the same time a large throughput, a very
high reliability, and a sufficient availability. On the other hand, the
success of the 3GPP 5G NR standard for communication systems makes the
underlying technology relevant for specific scenarios such as that of the
railway systems. As a result, the Future Railway Mobile Communication Systems
(FRMCS) propose mechanisms to take advantage of 5G-related aspects to offer
specific railway services such as the automated train control [1].

Within this context, the scheduling of radio resources plays an important role
to efficiently share the said resources between several users. The Round-Robin
[2] and the Priority-Queue [3] are well-known schedulers whose computational
complexity is very low but whose performance is not optimal with respect to
application-level metrics. As a matter of fact, they do not take into account
all the parameters impacting the application performance. For the current
purpose, the objective is to consider both application-level parameters and
lower layer parameters, such as the radio conditions, to adapt the scheduling
strategy.

This approach is in line with the Release 19 of 3GPP [4] which specifies the
service requirements for the 5G system. In this latter reference, it is
explained that “the communication service is considered unavailable if it does
not meet the pertinent Quality of Service (QoS) requirements. For example, the
communication service is unavailable if a message is not correctly received
within a specified time, which is the sum of maximum allowed end-to-end
latency and survival time”.

Recently, we introduced a new paradigm [5] for the design of a radio resource
scheduler in a multi-agent setting. It takes into account in the meantime an
application layer parameter, the survival time, and a PHY layer parameter, the
channel error probability. To enhance the scheduler of [5], a low-complexity
heuristic to approximately solve the scheduling optimization problem, we
formalize in this paper the scheduling problem as a Markov Decision Process
(MDP) [6, 7]. One advantage is that MDP provide an optimal solution for the
optimization problem. Another advantage is that MDP have been widely
experienced and give also access to many sub-optimal existing algorithms. More
specifically, we shall consider the value iteration algorithm as well as the
deep Q learning algorithm. The first algorithm optimally solves the scheduling
problem but with a prohibitive complexity as the problem size grows. The
second algorithms is sub-optimal but scales with the problem size, similarly
to the heuristic proposed in [5]. Consequently, we compare these algorithms
and discuss the performance complexity trade-off.

This paper is organized as follows. Section II presents the scheduling problem
in the scope of FRMCS, Section III describes the system, Section IV introduces
MDP within the scheduling framework, and Section V exhibits numerical results
to challenge MDP with other schedulers.

##  II FRMCS context

Within the various scenarios under the umbrella of the train control [1], we
focus on the remote driving for train shutting yards. This implies a remote
driver driving the train in order to bring it back to the train station. The
data provided to the said driver is mainly made of images and videos to allow
the driver to be aware, in real-time, of the train surrounding environment. In
other words, the data to be transmitted involves a large quantity of payloads.
Moreover, as this environment is shared with other trains, moving or not,
remote driving raises a safety problem.

Combining a high throughput with a high reliability and a high availability is
one of the 5G proposals, e.g., for the V2X scenarios [8] or the factory
automation scenarios [9]. The application requirements provided by the 3GPP
specifications embody the performance target for the access layer design
through the QoS. The access layer is understood in this paper to comprise the
PHY and MAC layers. For example, the QoS comprises a guaranteed bit-rate, a
latency, etc. However, the metrics that are used as inputs and outputs for the
mechanisms of the access layer are low-level metrics, e.g., the channel error
probability, the frame error rate, the channel busy/occupancy ratio. Even
though these metrics are helpful to underline the behaviour of a single layer,
namely the access layer, they do not well reflect the expected synergy with
the application layer.

From another perspective, some companies provide in [10] results of radio
performance for NR railway systems considering a system-level framework.
Within this framework, the scheduler plays an important role to efficiently
share the radio resources between the various users. The said companies only
consider the Round-Robin scheduler [2], a scheduler that assigns equal amount
of resources to each user, regardless of the channel quality or performance
requirements for the applications. Among other consequences, a user with a
high channel quality achieves a much higher throughput than what it needs
while, at the same time, a user with a low channel quality cannot achieve the
required throughput. Furthermore, a proportional-fair strategy would bring
only limited benefits as it does not consider the application parameters.
Consequently, following the scope in [5], we orient the current paper on the
scheduling problem taking into account application aspects as well as access
layer aspects.

##  III Description of the system

This section presents the scheduling problem by defining the application
traffic model, the radio resources used by the scheduler and the application
behaviour, similarly to [5, Sec. II.A]. The difference with [5] is the
inclusion of the payload size, and the division of the payload in several
packets.

###  III-A Traffic model

We consider a discrete-time system divided in time slots whose length is $dt$,
e.g., $dt=1$ msec. Let $N$ be the number of agents in the system. Two main
traffic models for the application layer are commonly proposed in the
literature [11]:

  * •

Full buffer traffic model: The buffers of an agent have always an unlimited
amount of data to transmit.

  * •

Finite buffer traffic model: An agent is assigned a finite payload to transmit
when it arrives. The agent leaves the system when the payload reception is
completed.

In this study, we consider a full-finite buffer traffic model. An agent
$A_{k}$ is assigned a finite payload of size $P_{k}$ to transmit. As soon as
the payload has been fully received, the application buffer of $A_{k}$ is
immediately refilled with a payload of the same size. In this model, all the
agents are always active as they have always a payload in their buffer.

###  III-B Scheduling resource & Payload aspect

We consider a single radio resource per time slot $t$. All agents
simultaneously compete for the resource at $t$ and only one agent finally
obtains it. The channel error probability $p_{k}$ is the probability for the
agent $A_{k}$ that a transmission at any time $t$ fails because of the
channel. It is assumed to be constant over time.

We assume that only a packet of size $\Gamma_{k}$ can be transmitted in one
time slot for $A_{k}$. The payload is thus necessarily split into $C_{k}$
packets at the scheduler level, i.e., $P_{k}=C_{k}\Gamma_{k}$. The payload is
transmitted once the associated packets have all been successfully
transmitted.

###  III-C Application behaviour and performance metric

The application is monitored according to three events for any agent. We
assume that the agents use the same application, i.e., they have the same
survival time $\tau$:

  * •

The event survival time failure for an agent (E1): “No payload is successfully
transmitted during the last $\tau$ time slots” where $\tau$ is a strictly
positive integer. For an agent $A_{k}$ at time $t$, we accordingly introduce
$\tau_{k}(t)$ as the remaining time before (E1). This means that
$0\leq\tau_{k}(t)\leq\tau$ and $\tau_{k}(t)$ is a decreasing function of $t$.

  * •

The micro-event successful transmission of a packet at time $t$ (e0).

  * •

The macro-event successful transmission of a payload at time $t$ (E0): This
event is the result of $C_{k}$ events (e0) required to to transfer a whole
message.

With both the events (E0) and (E1) for any agent $A_{k}$ at time $t$,
$\tau_{k}(t)$ is immediately set to its maximum value $\tau$.

For a single agent $A_{k}$, at any time $t$, the quantity $V_{k}(t)$ is the
number of failures (E1) met by $A_{k}$ until time $t$. The performance metric
is chosen as the failure rate $F(t)$:

$\displaystyle F(t)=\frac{\sum_{k=1}^{N}V_{k}(t)}{t}.$ (1)

Given that several agents might fail at a single time slot, the value of the
failure rate can be greater than one. This is to expect for very small values
of the survival time $\tau$.

##  IV Framework of the Markov Decision Process

We consider the infinite-horizon MDP with discounted sum-reward criterion, as
presented in [6, Chap. 6], to model the scheduling problem.

###  IV-A Model

As for a standard MDP, we use four variables to model the scheduling problem:

  * •

State $\textbf{S}_{t}\in\mathbb{N}^{N\times 2}$: The element
$\textbf{S}_{t}(k,1)$ is the number $\tau_{k}(t)$ of remaining time slots at
time $t$ before meeting a failure for agent $A_{k}$. The element
$\textbf{S}_{t}(k,2)$ is the number $c_{k}(t)$ of remaining packets at time
$t$ before the application message is fully transmitted. As
$0\leq\tau_{k}(t)\leq\tau$ and $1\leq c_{k}(t)\leq C_{k}$, the state
$\textbf{S}_{t}$ belongs to a state space $\mathcal{S}$ whose size is
$|\mathcal{S}|=(\tau+1)^{N}\prod_{k=1}^{N}C_{k}$.

  * •

Action $a_{t}\in\\{0,1,...,N\\}$: The index of the agent who gets the resource
at time $t$. The action $a_{t}$ belongs to an action space $\mathcal{A}$ whose
size is the number of agent $N$.

  * •

Short-term reward $r_{t}\in\\{-N,...,-1,0\\}$: Minus the number of events (E1)
at time $t$.

  * •

Transition probability: If $a_{t}=k$, the packet for $A_{k}$ is well received
with probability $1-p_{k}$. Hence, $a_{t}$ leads to only two states
$\textbf{S}_{t+1}$ with the non-zero transition probability
$p(\textbf{S}_{t+1}|\textbf{S}_{t},a_{t})$.

The MDP introduces the long-term reward or gain $G_{t}$ at time $t$ defined
as:

$\displaystyle
G_{t}=\sum_{t^{\prime}=t}^{\infty}\lambda^{t^{\prime}-t}r_{t^{\prime}},$ (2)

where $\lambda<1$ is the discount factor. If $\lambda$ is close to 1, this
quantity is almost the same as the (unnormalized) failure rate (1). The goal
for the scheduling problem is thus to find a policy $\pi$, being the time
sequence of allocation decisions, that maximizes the expected gain
$\mathbb{E}[G_{t}]$.

The MDP then defines $v^{\pi}(\textbf{S})$ the value of the state S under the
policy $\pi$ as the expected gain given that $\textbf{S}_{t}=\textbf{S}$:

$\displaystyle
v^{\pi}(\textbf{S})=\mathbb{E}[G_{t}|\textbf{S}_{t}=\textbf{S}].$ (3)

Then, a policy $\pi^{*}$ is optimal if:

$\displaystyle v^{\pi^{*}}(\textbf{S})\geq v^{\pi}(\textbf{S}),\
\forall\textbf{S}\in\mathcal{S}\text{ and }\forall\pi\neq\pi^{*}.$ (4)

Under the optimal policy, the value of the state S can be expressed via the
Bellman’s equation as:

$\displaystyle v^{\pi^{*}}(\textbf{S})=\max_{a\in\mathcal{A}}Q(\textbf{S},a),$
(5)

where $Q(\textbf{S},a)$ is the state-action value computed as:

$\displaystyle
Q(\textbf{S},a)=\sum_{\textbf{S}^{\prime}\in\mathcal{S}}p(\textbf{S}^{\prime}|\textbf{S},a)\Bigl{(}r(\textbf{S}^{\prime},\textbf{S})+\lambda
v^{\pi^{*}}(\textbf{S}^{\prime})\Bigr{)},$ (6)

where $r(\textbf{S}^{\prime},\textbf{S})$ is the reward obtained when going
from S to $\textbf{S}^{\prime}$. Given an optimal policy $\pi^{*}$, the
optimal action to take when in a state S is obtained as:

$\displaystyle a^{*}=\arg\max_{a\in\mathcal{A}}Q(\textbf{S},a).$ (7)

###  IV-B Value iteration

One standard approach to obtain (3) under $\pi^{*}$ is to operate the Value
Iteration algorithm (VI) that consists in computing the state values in an
iterative manner. Let us define $v^{(i)}(\textbf{S})$ as the approximate of
$v^{\pi^{*}}(\textbf{S})$ at iteration $i$. We accordingly define the
approximate of $Q(\textbf{S},a)$ at iteration $i$ as:

$\displaystyle
Q^{(i)}(\textbf{S},a)=\sum_{\textbf{S}^{\prime}\in\mathcal{S}}p(\textbf{S}^{\prime}|\textbf{S},a)\Bigl{(}r(\textbf{S}^{\prime},\textbf{S})+\lambda
v^{(i)}(\textbf{S}^{\prime})\Bigr{)},$ (8)

which leads to:

$\displaystyle v^{(i+1)}(\textbf{S})=\max_{a\in\mathcal{A}}\
Q^{(i)}(\textbf{S},a),$ (9)

where $v^{(i)}(\textbf{S})$ converges to $v^{\pi^{*}}(\textbf{S})$ for all S
[6]. The optimal action is then chosen as
$a^{(I)}(\textbf{S})=\arg\max_{a\in\mathcal{A}}\ Q^{(I)}(\textbf{S},a)$, where
$I$ denotes the index of the last iteration.

As already mentioned, the VI finds the optimal policy with respect to
$\mathbb{E}[G_{t}]$. Hence, modelling the scheduling problem in the MDP
framework enables to get the optimal performance in the cases where the VI is
tractable. As a direct consequence, the VI can be used as a benchmark to
assess the performance of less complex heuristics.

###  IV-C Deep Q learning

Even though the VI provides the optimal estimate of the states value, it
suffers from significant computational complexity and memory consumption as it
needs to cover all the states in $\mathcal{S}$ of size
$|\mathcal{S}|=(\tau+1)^{N}\prod_{k=1}^{N}C_{k}$. A practical alternative to
the VI is the temporal-difference learning [7, Chap. 6] on which $Q$-learning
and deep $Q$ learning rely.

Instead of looping over all the states to estimate $v^{\pi^{*}}(\textbf{S})$,
these algorithms walk across a few states within $\mathcal{S}$. In the
standard approach, the system is in state $\textbf{S}_{t}$ at $t$ and an
action $a_{t}$ is done. One then gets a reward
$r(\textbf{S},\textbf{S}^{\prime})$ and the new state $\textbf{S}^{\prime}$ of
the system at time $t+1$. It is therefore a Monte Carlo approach. The model to
estimate $Q^{\pi^{*}}(\textbf{S},a)$ is updated via an error signal
$\Delta_{t}$, obtained from two estimates of $Q^{\pi^{*}}(\textbf{S},a)$:

$\displaystyle\Delta_{t}=\hat{Q}(\textbf{S},a)-\hat{Q}^{\prime}(\textbf{S},a)$
(10)

where:

  * •

$\hat{Q}^{\prime}(\textbf{S},a)=r(\textbf{S},\textbf{S}^{\prime})+\lambda\max_{a^{\prime}}\hat{Q}(\textbf{S}^{\prime},a^{\prime})$
is a first estimate of $Q^{\pi^{*}}(\textbf{S},a)$ based on the observed
reward and the subsequent state $\textbf{S}^{\prime}$ obtained when taking the
action $a$ in state S, and where $\hat{Q}(\textbf{S}^{\prime},a^{\prime})$ is
obtained via the current model.

  * •

$\hat{Q}(\textbf{S},a)$ is a second estimate of $Q^{\pi^{*}}(\textbf{S},a)$
obtained via the current model.

With deep Q learning the model is a neural network which is trained via the
gradient of the error signal with respect to the parameters of the neural
network [12].

##  V Numerical evaluations

This section presents some performance results of the VI and deep Q learning
algorithms, and compare them with other schedulers.

###  V-A Challengers

We compare the MDP-based algorithms with three other schedulers:

  * •

Round-Robin (RR). It consists in allocating the resource to the $N$ agents
following a buffer of the agents index. The said buffer is a random
permutation of $[0,\dots,N-1]$. After $N$ allocations, the RR replaces its
buffer with a new random permutation of $[0,\dots,N-1]$. This randomization
prevents an agent from being always out at each period of the RR. The RR is a
low complexity algorithm.

  * •

Priority-queue (PQ). It consists in allocating the resource to the agent whose
number of remaining time slots before meeting a failure is the lowest, i.e.,:
$a_{t}=\text{arg min}_{k}\tau_{k}(t)$. This scheduler is very easy to
implement with no concern neither on the computational complexity nor on the
memory consumption. Nevertheless, it does not use the channel error
probabilities $p_{k}$ and the number of remaining packets $c_{k}(t)$. The PQ
is a low complexity algorithm.

  * •

On-line (OL) [5]. It uses a heuristic $f_{r}(t,k)$ to estimate a probability
of survival time failure for every agent $A_{k}$. The agent to allocate is
then selected to minimize a subsequent global probability of failure. In [5],
there is no notion of division of the payload in $C_{k}$ packets. Therefore,
we need to slightly modify the heuristic $f_{r}(t,k)=p_{k}^{\tau_{k}(t)}$
with:

$\displaystyle f_{r}(t,k)=p_{k}^{\frac{\tau_{k}(t)}{c_{k}(t)}}.$ (11)

The OL is a moderately low complexity algorithm.

###  V-B Scenario and assumptions

We consider a small-sized scenario with $N=3$ agents and with $\tau$ spanning
from $N$ to 12. We assume the channel error probabilities to be
$p_{1}=10^{-3},p_{2}=10^{-2},p_{3}=10^{-1}$, i.e., such that $A_{1}$ has a
very good channel, $A_{3}$ suffers from a difficult channel, and $A_{2}$ is in
between. Several situations are studied regarding the number of packets
$C_{1},C_{2},C_{3}$ per payload for the agents denoted by the set
$\\{C_{1},C_{2},C_{3}\\}$: $\\{1,1,2\\}$, $\\{1,1,3\\}$, $\\{2,2,2\\}$ and
$\\{2,2,3\\}$. Consequently, the size of the state space varies from 128 to
26364 elements. In these configurations, the system has the greatest channel
error probability on the agent with the greatest number of packets per
payload. This is a way to stress the scheduler by setting difficult
situations. The purpose of the evaluations is to observe the performance
dependence on the values of $\\{C_{k}\\}_{k}$ given $\\{p_{k}\\}_{k}$. We let
all the schedulers run for a time duration of $10^{6}$ time slots to be able
to observe failure rates up to around $10^{-5}$. We recall that several
failures might occur at a given time slot according to (1). Hence, the
performance metric is $F=F(10^{6})$ (see (1)).

The challengers are then compared taking the value of $F$ for each of them,
namely $F_{\text{RR}}$ for the Round-Robin, $F_{\text{PQ}}$ for the Priority-
Queue, $F_{\text{OL}}$ for the On-Line, $F_{\text{VI}}$ for the VI,
$F_{\text{DQ}}$ for the deep Q learning. To simplify the wording here after,
we denote by $F_{X}(\\{a,b,c\\})$ the value of the failure rate for the
scheduler $X\in\\{\text{RR},\text{PQ},\text{OL},\text{VI}\\}$ with the
configuration $C_{1}=a,C_{2}=b,C_{3}=c$.

###  V-C Details for Deep Q learning

We consider deep Q learning for the case
$\\{C_{1},C_{2},C_{3}\\}=\\{2,2,3\\}$. We consider two neural networks whose
input is the state $\textbf{S}_{t}$. The first network NN(1) is trained with
the true situation $p_{1}=10^{-3},p_{2}=10^{-2},p_{3}=10^{-1}$ considering
$\tau=10$. With such parameters, $A_{1}$ and $A_{2}$ encounter errors at a
very low frequency, which may be an issue as deep Q learning is a Monte Carlo
algorithm (analysis validated by the experiments, see the next section). To
mitigate this issue, we train a second network NN(2) in a situation with
$p_{1}=10^{-2},p_{2}=10^{-1},p_{3}=0.5$ such that $A_{1}$ and $A_{2}$ have
more channel errors, while keeping the hierarchy $p_{1}<p_{2}<p_{3}$. In other
words, we introduce a model mismatch for the training. Deep Q learning with
NN(1) is called DQ(1) and DQ(2) with NN(2).

The neural network used in the simulation is a residual neural network of 22
layers comprising a total of 26200 parameters. Note however that a smaller
neural network with only 3000 parameters yields similar results. Such a large
network is considered such that its size is not the performance bottleneck.

###  V-D Simulation results

The results are displayed in Fig. 1. First of all, we observe that all
schedulers have similar performance with large values of $F$ for low values of
$\tau$. Unsurprisingly, a larger $\tau$ is required to reduce the failure
rate. Moreover, the challenging zone for discriminating between the schedulers
is then for middle or large values of $\tau$, e.g., for $\tau\geq 4$ with
{1,1,2}, for $\tau\geq 5$ with {1,1,3}, for $\tau\geq 7$ with {2,2,2}, and for
$\tau\geq 8$ with {2,2,3}.

$4$$6$$8$$10$$12$$10^{-4}$$10^{-2}$$10^{0}$$\tau$$F$RRPQOLVI (a) {1,1,2}

$4$$6$$8$$10$$12$$10^{-4}$$10^{-2}$$10^{0}$$\tau$$F$RRPQOLVI (b) {1,1,3}

$4$$6$$8$$10$$12$$10^{-4}$$10^{-2}$$10^{0}$$\tau$$F$RRPQOLVI (c) {2,2,2}

$4$$6$$8$$10$$12$$10^{-4}$$10^{-2}$$10^{0}$$\tau$$F$RRPQOLVIDQ-1DQ-2 (d)
{2,2,3}

Figure 1: Simulation results in several scenarios with
$p_{1}=10^{-3},p_{2}=10^{-2},p_{3}=10^{-1}$.

Then, we observe that for such challenging zones, $F_{\text{RR}}$ is far
beyond $F_{\text{PQ}},F_{\text{OL}}$ and $F_{\text{VI}}$. This is expected as
RR is the only scheduler which does not consider any agent parameter. Another
common observation in every figure is that:

$F_{\text{VI}}\leq F_{\text{OL}}\leq F_{\text{PQ}}\leq F_{\text{RR}},$ (12)

This confirms that the VI is the reference scheduler. This also shows that the
low complexity schedulers PQ and OL perform better than the naive RR.
Consequently, the analysis will focus on the performance of OL and PQ in
comparison with that of VI.

The configurations {1,1,2},{2,2,2} result in $F_{\text{OL}}\approx
F_{\text{PQ}}$ whereas the other configurations {1,1,3},{2,2,3} result in
$F_{\text{OL}}<F_{\text{PQ}}$ in a more obvious manner. As an example, at
$\tau=10$, $F_{\text{PQ}}(\\{1,1,3\\})=3F_{\text{OL}}(\\{1,1,3\\})$ and at
$\tau=12$, $F_{\text{PQ}}(\\{2,2,3\\})=4F_{\text{OL}}(\\{2,2,3\\})$. Also, the
y-distance between $F_{\text{OL}}$ and $F_{\text{PQ}}$ increases when
increasing $\tau$ in a faster way with $C_{3}=3$ than with $C_{3}=2$. For
example, $F_{\text{PQ}}(\\{2,2,2\\})\approx 1.5F_{\text{OL}}(\\{2,2,2\\})$ at
$\tau=10$ and $F_{\text{PQ}}(\\{2,2,2\\})\approx F_{\text{OL}}(\\{2,2,2\\})$
at $\tau=6$ while $F_{\text{PQ}}(\\{2,2,3\\})\approx
2.6F_{\text{OL}}(\\{2,2,3\\})$ at $\tau=10$ and
$F_{\text{PQ}}(\\{2,2,3\\})\approx F_{\text{OL}}(\\{2,2,3\\})$ at $\tau=6$.
This expansion effect is also observed between OL and VI, e.g.,
$F_{\text{OL}}(\\{2,2,3\\})\approx 4.2F_{\text{VI}}(\\{2,2,3\\})$ at $\tau=10$
and $F_{\text{OL}}(\\{2,2,3\\})\approx F_{\text{VI}}(\\{2,2,3\\})$ at
$\tau=6$. As $F_{\text{OL}}\leq F_{\text{PQ}}$, though, OL can be said to be
more robust than PQ when increasing the payload of the worse channel agent. In
other words, enlarging the payload of the worse channel agent leads to select
OL rather than PQ.

Alternatively, we can compare the schedulers with respect to the survival time
they require to reach a target failure rate $F_{\text{trgt}}$. As $\tau$ is an
integer, we round $\tau$ to the nearest greater integer for comparison
purpose. For example, when fixing $F_{\text{trgt}}=10^{-4}$, we see that for
{1,1,2},{1,1,3},{2,2,2},{2,2,3}, respectively, VI fulfills the target with
$\tau=7,8,9,10$, OL fulfills the target with $\tau=7,9,9,11$, and PQ fulfills
the target with $\tau=8,9,10,12$. PQ is then always one time slot ahead from
VI whereas OL may reach the same survival time. Consequently, OL provides
nearly optimal performance with less constrains on the application compared
with PQ.

Finally, we focus on the curves showing the performance of deep Q learning
with {2,2,3}. We observe that $F_{\text{DQ}^{(1)}},F_{\text{DQ}^{(2)}}$ are
both greater than $F_{\text{VI}}$, i.e., they do not reach the optimal policy.
However, when $\tau\geq 11$, we see that $F_{\text{DQ}^{(2)}}\leq
F_{\text{PQ}}$ and $F_{\text{DQ}^{(2)}}\leq F_{\text{OL}}$. Extrapolating the
curves to $\tau\geq 12$, we expect DQ(1) and DQ(2) to perform even better than
PQ and OL. In other words, when going further than the $\tau$ value used for
training, DQ seems to well behave whereas running DQ for $\tau$ values lower
than that of training, DQ is clearly suboptimal. We also observe that DQ(2),
trained with the model mismatch, performs better than the standard training
DQ(1). This highlights that the rare channel errors is indeed a problem for
deep Q learning, and having a model mismatch for the training improves the
performance. Nevertheless, this raises a new problem: How to chose the most
adequate model parameters for the training? These preliminary performance
results also indicate, though, that it might be possible to approach the VI
performance provided good training parameters. Therefore, as DQ is less
complex than VI and less memory consuming than VI, this opens the room for
scaling the scheduling problem to a larger state space, e.g., with more
agents, with greater values of $\tau$, and with greater values of $C_{k}$.

##  VI Conclusions

In this paper, we formalized the scheduling problem in the framework of MDP
and we showed how to find the optimal scheduling strategy. This enables to
assess the performance of candidate lower complexity schedulers. Indeed, the
optimal scheduler suffers from complexity and storage issues. Among the lower
complexity scheduler, the deep Q learning approach based on neural networks is
investigated. We observed that training the neural network is not
straightforward because of the scarcity of error events of some agents.
Nevertheless, the performance can be improved by introducing a model mismatch
for the training step. These preliminary results indicate that it may be
possible to approach the optimal performance with this lower complexity
scheduler. This may offer the possibility to scale the scheduler and therefore
address larger systems.

## References

  * [1] _TS22.289 Technical Specification Group Services and System Aspects; Mobile Communication System for Railways; Stage 1 (Release 17)_ , 3rd Generation Partnership Project (3GPP), December 2019, v17.0.0.
  * [2] R. H. Arpaci-Dusseau and A. C. Arpaci-Dusseau, _Operating Systems: Three Easy Pieces_ , 1st ed. Arpaci-Dusseau Books, August 2018.
  * [3] J. Liebeherr and D. Wrege, “Priority queue schedulers with approximate sorting in output-buffered switches,” _IEEE Journal on Selected Areas in Communications_ , vol. 17, no. 6, pp. 1127–1144, 1999.
  * [4] _TS22.261 Technical Specification Group Services and System Aspects; Service requirements for the 5G system; Stage 1 (Release 19)_ , 3rd Generation Partnership Project (3GPP), December 2022, v19.1.0.
  * [5] J.-C. Sibel, N. Gresset, and V. Corlay, “An application-oriented scheduler,” in _2023 IEEE Wireless Communications and Networking Conference (WCNC)_ , March 2023, to appear. [Online]. Available: https://arxiv.org/abs/2302.09926
  * [6] M. L. Puterman, _Markov Decision Processes: Discrete Stochastic Dynamic Programming_ , 1st ed. USA: John Wiley and Sons, Inc., 1994.
  * [7] R. S. Sutton and A. G. Barto, _Reinforcement Learning: An Introduction_. Cambridge, MA, USA: A Bradford Book, 2018.
  * [8] _TS22.886 Technical Specification Group Services and System Aspects; Study on enhancement of 3GPP Support for 5G V2X Services (Release 16)_ , 3rd Generation Partnership Project (3GPP), December 2018, v16.2.0.
  * [9] _TS22.104 Technical Specification Group Services and System Aspects; Service requirements for cyber-physical control applications in vertical domains; Stage 1 (Release 18)_ , 3rd Generation Partnership Project (3GPP), December 2021, v18.3.0.
  * [10] _TR103.554-2 Technical Report; Rail Telecommunications (RT); Next Generation Communication System; Radio performance simulations and evaluations in rail environment; Part 2: New Radio (NR)_ , European Telecommunications Standards Institute (ETSI), February 2021, v1.1.1.
  * [11] P. Ameigeiras, Y. Wang, J. Navarro-Ortiz, P. E. Mogensen, and J. M. Lopez-Soler, “Traffic models impact on ofdma scheduling design,” _EURASIP Journal on Wireless Communications and Networking_ , no. 61, 2012\.
  * [12] M. et al., “Human-level control through deep reinforcement learning,” _Nature_ , vol. 518, no. 7540, p. 529–533, 2015.


organizations, the CAP service shows potential for broader application across
research disciplines. Furthermore, the technology and tools used to develop
the service are not only open source, but are discipline agnostic. More
specifically, CAP is built on top of the open source framework Invenio, which
already has a wide variety of applications in many disciplines, and all the
underlying source code for CAP is openly available on GitHub.

###  5.4 REANA Reproducible Analyses

REANA is a reproducible analysis platform [23] that aims to facilitate
reusable science by allowing researchers to structure and run parametrised
computational data analysis workflows. REANA was launched in 2017 as a sister
project to ensure the reuse of content preserved in the CERN Open Data portal
and the CERN Analysis Preservation framework services [8].

The data analysis process can be generally described by means of specifying
input data, the analysis code, and the computational recipe used to arrive at
the final results trough a sequence of computational steps. This process can
be expressed by means of Directed Acyclic Graph (DAG) where graph vertices
represent units of computation with their inputs and outputs and graph edges
describe the interconnection of various computational steps [92]. REANA
supports several such DAG standards (CWL, Snakemake, Yadage), parses the
workflow specification described by the user and dispatches its computational
steps to various supported compute backends (Kubernetes, HTCondor, Slurm). The
reproducibility of computations is assisted by means of using software
containers (Docker, Singularity) that fully encapsulate the original
computational environments of each analysis step.

The REANA approach was successfully tested in several data production and data
analysis scenarios. For example, REANA was used for ATLAS reinterpretation
searches for new physics or CMS reconstruction and jet energy corrections
[93]. In the CERN Open Data portal, REANA is used both on the “data
production” side to ensure the correctness of preserved data sets’ provenance
information [90] as well as on the “data analysis” side by running several
data analysis examples such as CMS open data Higgs-to-four-lepton example
analysis (see Fig. 11).

Figure 11: Example of Higgs-to-four-leptons analysis of CMS open data running
on REANA reproducible analysis platform. The analysis consists of four steps
and is expressed in the Snakemake workflow specification language. The
specification for one of the steps is illustrated on the right.

REANA promotes early adoption of computational reproducibility principles by
researchers. The integration with source code management platforms such as
GitLab allows researchers to develop analyses on GitLab and run either full
data analysis tasks on REANA or at least test the correctness of analysis
workflow after each code change. If an analysis is developed in this
“continuous integration” manner [93], the preservation of knowledge associated
with the data analysis as well as the future deposit of analysis assets into
digital repositories are largely facilitated. REANA therefore complements the
data preservation repositories by promoting active “preproducibility” of data
analyses during the active analysis phase rather than only relying on passive
data deposition and subsequent “reproducibility” once the analysis is
completed [94].

REANA was developed with particle physics use cases in mind, but the platform
profits from synergies with general reproducible data analysis patterns in
other scientific disciplines, such as astronomy, bioinformatics and life
sciences [95].

###  5.5 Bit Preservation at CERN

During 2021 CERN migrated all archival storage from the earlier CASTOR system
to the new CERN Tape Archive (CTA) which consequently now assumes CERN’s bit
preservation responsibilities. All preservation use cases, including of course
the LEP data, were moved into CTA.

In terms of bit preservation CTA provides essentially the same guarantees as
CASTOR. Indeed, the migration was purely a matter of moving metadata and the
data on tapes remained unaffected. File access permissions are managed
slightly differently in CTA and this triggered discussion on the appropriate
model for authorising access to preservation data. A model where a privileged
curator is able to maintain a list of reader accounts (using CERN’s e-groups
system) is being elaborated.

During 2021 CERN deployed a new tape library in a building separate from the
computer centre. This library will be commissioned for use with CTA, which
will allow dual-replica data to benefit from geographical separation between
the replicas.

CTA now holds the data that CERN has received from BABAR for the purposes of
data preservation. The integrity of this archive has yet to be validated by
BABAR and thus the transfer is not yet considered complete.

###  5.6 CERNVM

CERNVM provides a portable platform to develop and execute HEP experiment
applications [96]. The CERNVM platform consists of a virtual appliance with a
minimal operating system (OS) and a distributed file system (CVMFS) that
provides a global shared area for scientific applications and conditions data
[97]. Both the appliance and the distributed file system distribute data
processing environments to the heterogeneous, distributed computing
infrastructure used by running experiments. The CERNVM technology is designed
to take software preservation aspects into account and to allow for an easy
transition from an actively maintained experiment software stack to the
preservation phase [98].

The CERNVM appliance provides both an image for a full virtual machine (VM)
for use with hardware virtualization technology such as KVM and Amazon EC2 as
well as a container image for use with container tools such as Docker and
Kubernetes. Full VMs give access to hardware emulation technology, so that
software stacks compiled for deprecated CPU architectures can run on
contemporary hardware, albeit at a significant performance cost (often
$\times$10 and more slower). Container virtualization has a negligible
performance overhead; it relies, however, on a stable interface from the Linux
kernel to user-land applications.

Retrofitting the CERNVM technology to software stacks from the LEP
experiments, the approach has shown to bridge more that 20 years. A CERNVM
environment with a CMS software stack from 2011 to process LHC run 1 data was
created with minimal effort, since the CVMFS and CERNVM infrastructure has
been in use from the start by LHC experiments.

Key in the CERNVM approach is the split between the minimal appliance (the OS
platform) and the experiment software directory tree provided by the CVMFS
network file system. The naïve use of virtualization technology, in contrast,
which bundles OS and application software in a single VM or container image,
is easily susceptible to container management problems. Due to the complexity
and variety of experiment application software, the naïve approach results
either in huge images (e.g., 100GB+ for all ATLAS releases) that are difficult
to distribute or in a proliferation of special purpose containers that are
difficult to book-keep. Software stacks provided through CVMFS, on the other
hand, are well-maintained by the experiments’ software librarians and the
necessary binaries for any given task are loaded on-demand at runtime. As
CVMFS is a versioning file system, software stacks installed on CVMFS are
automatically preserved remain available for future use.

Several lessons were learned from exercising CERNVM and CVMFS for the
preservation of LEP and LHC experiment applications:

  1. 1.

It is important to distinguish between _scientific_ applications (e.g., event
generators, detector simulation, histogramming tools) and _system_
applications (e.g., web clients, data access software). In contrast to
scientific applications, system applications generally do not freeze well.
This is due to the fact that system applications often interact with the
outside world or at least the (virtualized) hardware. For instance, web
clients may unsuccessfully try to connect with an outdated SSL protocol to
HTTPS servers. Hard disk utilities may be subject to integer overflows on
today’s volume sizes.

  2. 2.

As a result, it is important to identify network dependencies of data
processing workflows before freezing. There may be dependencies, for instance,
to online databases (such as conditions databases), grid configuration, or
data access servers. A stable frozen software environment should only use the
POSIX file system as a means to interact with the outside world. The
experiment software as well as all input and output data should be stored on
network file systems, which can be mapped into virtualized environments. In
CERNVM, CVMFS is used to store software and detector conditions data and EOS
is used for experiment data.

  3. 3.

Lifting authorization barriers (such as X.509 certificate authorization) has
been useful in the context of data preservation efforts. Authorization
protocols are particularly ill-suited for freezing due to sudden upgrades of
protocols or software following the revelation of vulnerabilities. Also, there
are often expiry mechanisms in security protocols such as certificate
lifetimes. Software and data on CVMFS is publicly readable by design;
providing the data on public EOS servers instead of protected shares
significantly simplifies the provisioning of historic data processing
environments.

Looking ahead, new challenges arise for the preservation of the LHC experiment
software of current and future runs. Compared to Run 1, the software
complexity rose substantially, especially with respect to external, non-HEP
software such as mathematical and ML libraries and scientific Python tools
(Fig. 12). Growing software stacks can be mitigated by strengthening the role
of software librarians and generally a disciplined approach to software
dependency management. Secondly, the last few years saw a rise of greater
architectural variety compared to the previous decades which were dominated by
the Intel X86_64 architecture. Today, there is a mix of Intel, ARM, and POWER
CPUs, accompanied by GPUs and (rarely) FPGAs. When transitioning a software
stack into a frozen state, it will likely be necessary to streamline the
available binaries to one or few reference architectures.

Figure 12: Growth of the CMS software stack in approximately 5 years.

###  5.7 Managed service migration/retirement: the CERNLIB example

The management of the retired external software packages poses a problem for
the rapidly developing but not optimally managed physics-related software.

One of the approaches that could be used to maintain the external software
packages is to create a digital archive/library implemented on top of the
popular software versioning platforms as github or GitLab.

CERNLIB is one of the key software components that have been extensively re-
used by many collaborations and still plays an key role for the preservation
activities e.g. for LEP- but also HERA-era experiments. The CERNLIB libraries
and tools have since their inception been migrated to a large set of platforms
and also several code management systems – still preserving most of the
original functionality and development(commit) history.

The CERNLIB contains multiple software sub-libraries, many of which are
actually independent software packages. Logically CERNLIB is split into a set
of sub-packages

  * •

mathlib, a library with mathematical routines

  * •

packlib, general purpose library

  * •

kernlib, general purpose library

  * •

geant, a version of the Geant3 [88] simulation toolkit.

  * •

mclib, which contains a set of most used MC event generators.

  * •

graflib, a library with interfaces to graphical routines

  * •

pawlib, Physics Analysis Workstation

Each sub-package is split into smaller sub-sub-subpackages, very often with
their own requirements and compilation options. Depending on the host system,
the CERNLIB might requre the following software

  * •

C preprocessor, C99 and Fortran77 compilers

  * •

sed and zip utilities

  * •

CMake and any build system – for sc CMake builds

  * •

Imake and make – for sc Imake builds

  * •

BLAS with Lapack compiled with the same Fortran compiler (optionally)

  * •

Freetype libraries and headers

  * •

X11 libraries and headers of version R6.6+

  * •

Motif libraries and headers

  * •

OpenSSL libraries (and headers for CMake builds)

The community of users may soon be facing the challenges of a forced migration
(caused by hardware and OS obsolescence) and consequently validation on recent
64-bit platforms. Since such a validation poses significant technical and
personnel effort, the idea to consolidate the validation activities on a
single, community maintained software base with a small set of actively
validated platforms emerged at a recent DPHEP community workshop. The effort
in a consolidated, up-to-date issue tracking and, as much as possible,
automated build and validation pipeline would then allow to exploit the small
remaining expert effort for the benefit all remaining CERNLIB deployments and
prepare for a period in which expertise in CERNLIB internals is rare or not
anymore readily available.

After the end of active development of the CERNLIB in 2007, it was widely used
by many collaborations and individuals. The intensive usage of CERNLIB on the
constantly updated operating systems resulted in a development of large number
of code patches. In the end of 200x most of these patches were consolidated by
Kevin McCarthy and used to create Debian packages for CERNLIB. Those patches,
as well as patches from the DESY theory group, Fedora and Debian projects
became the starting point for the updates of CERNLIB in 2022. All the updates
were implemented in the newly created CERNLIB git repository hosted by
CERN151515https://gitlab.cern.ch/DPHEP/cernlib/cernlib and later extended by
various improvements by the CERNLIB/DPHEP initiative. The repository is a
merger of previously separate repositories with CERNLIB sub-libraries and
contains all the development history of CERNLIB since 1990s. The code is also
supplemented with a continuous integration system (CI) that performs builds of
CERNLIB on selected platforms, namely older Scientific Linux 4, 5, 6; CentOS
7, 8, 9; Alma 8, 9; Fedora 35; in 32- and 64-bit variations, as well as Ubuntu
18, 20, 22; in 64-bit.161616Upstream Canonical dropped certain 32bit
development libraries, therefore for Ubuntu only 64-bit variations are
available. While some of the images used in CI are stock images from the
vendors, the older Scientific Linux 4, 5, and 6 container images are
maintained under the DPHEP cover.

While CERNLIB contains codes written in Fortran and C, there are little
worries about future ability to compile the updated CERNLIB code. However, the
native CERNLIB build system, Imake [99], is relatively outdated and can
potentially disappear in the nearest future. Therefore, while the Imake
scripts are still maintained in the CERNLIB, significant efforts were put
recently into a re-implementation of the build system of CERNLIB in CMake
[100]. CMake is an open-source, free cross-platform build tool with a history
for more than 20 years. As of 2022 CMake is de-facto the standard for software
written in C/C++/Fortran. With the CMake-based build system it became possible
to perform checks of CERNLIB code with many more compilers on multiple
historical and modern platforms. This approach is a carbon copy of the
techniques used to port the JADE software, see Sec. 4.5. As a result, the
updated CERNLIB version can be compiled on RHEL4+ i686, x86_64, ppc64 systems
and modern 64bit MacOSX using GNU 3+, CLang, Intel and NVidia compiler
collections. However, only certain combinations of the compilers and systems
are fully backed by the DPHEP and included in the CI.

##### Distribution

The CERNLIB code is available in https://gitlab.cern.ch/DPHEP/cernlib/cernlib
and should serve as a drop-in replacement for the CERNLIB 2006. The updated
CERNLIB can be also packaged into RPM package. While no further development of
CERNLIB is foreseen, we encourage the current and future users of CERNLIB to
give a feedback on the updated CERNLIB and submit their suggestions,
bugreports and bugfixes.

###  5.8 ARCHIVER

The project ARCHIVER [101, 102] develops innovative services for Long Term
Digital Preservation of scientific datasets using the Pre-Commercial
Procurement instrument funded by the European Commission. R&D was performed
competitively by commercial suppliers, across different implementation phases.
The services were developed by Arkivum and Libova co-designed with input from
research clusters members: CERN, PIC/IFAE and DESY are members of the ESCAPE
cluster, DESY is also a partner in the ExPaNDs cluster, while EMBL-EBI is a
member of EOSC-Life. These research performing organizations deployed use
cases from Astrophysics, High-Energy Physics, Life Sciences and Photon-Neutron
Sciences. The use-cases driving the ARCHIVER consortium’s need for research
and development of innovative data preservation services extended the
preservation ecosystems of research organizations to create more dynamic
solutions to be deployed primarily by using a hybrid model, on-premise or
exclusively in cloud environments, operated by European SMEs that were
enhanced and assessed against best practices such of CoreTrustSeal and DPC
RAM, so that datasets remain FAIR (Findable, Accessible, Interoperable,
Reusable) for decade timescales or more.

The services resulting from the ARCHIVER R&D are addressing the gaps that put
data at long-term risk as they prevent the construction and operation of
sustainable Trusted Digital Repository services, affecting organizations both
large and small who are tasked with being custodians of valuable research
artifacts. Thanks to its outstanding results, the ARCHIVER project has been
recognized by the international community and been awarded by the
International Council on Archives, category for Collaboration and Cooperation,
at the DPC awards ceremony in Glasgow, on the 12th of September 2022. The
model developed in ARCHIVER is considered very beneficial for the research
community and contributes to address concerns about sharing and re-use of FAIR
data to reproduce research as a pillar for Open Science and long-term
preservation of the EOSC federated data, by delivering sustainable, production
quality long-term data preservation services for user communities that fills a
gap in the existing European Open Science Cloud portfolio.

##  6 DPHEP: the way forward

DPHEP is an unique forum ensuring continuity, expertise exchange and knowledge
preservation within a field where the usual HEP collaborative models are
modified to deal with a rather uncommon situation: obtain new scientific
results at constant level of complexity with reduced resources and therefore
refactored analysis environments. With an overview of over ten years, one can
observe that the concepts, the methodologies, the technologies and the
collaborative configurations required to preserve data at long term require a
significant amount of innovation. While DPHEP focused so far on collider
experiments, it is obvious that other collaborations – in particle physics,
astroparticles, cosmology, nuclear physics etc. – face similar problems in
preserving large and complex data sets. An increased synergy within and beyond
the HEP community remains possible and desirable. The links with the
industrial and economical worlds are an interesting perspective as well, still
to be explored.

Moreover, data preservation is one of the multiple possible actions towards
improving the current and future computing systems in HEP and beyond. As it
becomes more and more obvious from the case studies presented above, the data
preservation activity is not only performing research on existing unique data
sets, it is also shaping the future, in coherence with the open science
policies and methodologies. Furthermore, there is a clear potential for
training the new generations and improving their technological skills.

The DPHEP Collaboration commits to drive a culture of open sharing of
knowledge, data, software, algorithms, infrastructures and other research
resources. The objectives for the next period are to:

  1. 1.

Improve the awareness and stimulate improvements of data preservation:

     * •

Explore and document the aspects related to data preservation: scientific
motivation, organisation options, technologies, standards, outreach and
education.

     * •

Attract new collaborators, enlarge the community, organise workshops, issue
regular Global Reports.

     * •

Ensure links to other communities.

  2. 2.

Reinforce and support the ongoing laboratory-based projects, encourage inter-
laboratory cooperation and strengthen the links to other projects in different
host institutions (external computing centers, Universities etc.).

     * •

Keep alive data sets that (can) still produce science and keep track of parked
data sets.

  3. 3.

Support/develop the DP aspects for future experiments and encourage the
transfer of knowledge.

  4. 4.

Encourage open data and open science as a way to preserve data and knowledge.

Data preservation is one of the building blocks of the HEP scientific outcome
and the DPHEP Collaboration intends to stimulate and support it.

## Acknowledgements

The authors would like to thank the respective funding agencies for support in
pursuing the topic. In particular, the endorsing of DPHEP as an ICFA panel has
been and continues to be an essential base of sustainable international
collaboration in the area of data preservation across collaborations and
laboratories.

## References

  * [1] DPHEP Study Group Collaboration, R. Mount et al., “Data Preservation in High Energy Physics”, arXiv:0912.0255 [hep-ex].
  * [2] DPHEP Study Group Collaboration, Z. Akopov et al., “Status Report of the DPHEP Study Group: Towards a Global Effort for Sustainable Data Preservation in High Energy Physics”, arXiv:1205.4667 [hep-ex].
  * [3] DPHEP Collaboration, S. Amerio et al., “Status Report of the DPHEP Collaboration: A Global Effort for Sustainable Data Preservation in High Energy Physics”, arXiv:1512.02019 [hep-ex].
  * [4] CERN, “CERN Open Data Policy for the LHC Experiments.” CERN-OPEN-2020-013, Geneva, Nov, 2020.  https://cds.cern.ch/record/2745133.
  * [5] LHC Collaboration, “Update of the Computing Models of the WLCG and the LHC Experiments”,. {http://cds.cern.ch/record/1695401}.
  * [6] HEP Software Foundation Collaboration, J. Albrecht et al., “A Roadmap for HEP Software and Computing R&D for the 2020s”, Comput. Softw. Big Sci. 3 no. 1, (2019) , arXiv:1712.06982 [physics.comp-ph].
  * [7] D. Ozerov and D. M. South, “A Validation Framework for the Long Term Preservation of High Energy Physics Data”, J. Phys. Conf. Ser. 513 (2014) , arXiv:1310.7814 [hep-ex].
  * [8] X. Chen et al., “Open is not enough”, Nature Phys. 15 no. 2, (2019) .
  * [9] M. Wilkinson et al., “The fair guiding principles for scientific data management and stewardship”, Scientific Data Article No.160018 no. 3, (2016) . 10.1038/sdata.2016.18.
  * [10] “ISO 14721: OAIS – a reference model for what is required for an archive to provide long-term preservation of digital information.”  {https://www.iso.org/standard/57284.html}.
  * [11] “ISO 16363: Audit and certification of trustworthy digital repositories.”  {http://www.iso16363.org/}.
  * [12] J. Shiers and M. Hildreth, “DPHEP - Certification: Motivation, Benefits and Status.”  {https://indico.cern.ch/event/658060/contributions/2889539/}.
  * [13] G. Ganis and J. Blomer, “Achilles’ heels in LTDP or how do we address risks.”  {https://indico.cern.ch/event/658060/contributions/2915946/}.
  * [14] “HEPData: Repository for publication-related High-Energy Physics data.”  {https://www.hepdata.net}.
  * [15] “Rivet: Analyses reference.”  {https://rivet.hepforge.org/analyses.html}.
  * [16] ATLAS Collaboration, “Simplified ATLAS SUSY Analysis Framework.”  {http://simpleanalysis.docs.cern.ch}.
  * [17] “Rivet: Analysis coverage.”  {https://rivet.hepforge.org/rivet-coverage}.
  * [18] ATLAS Collaboration, “SimpleAnalysis software release.”  {https://zenodo.org/communities/atlas_experiment/}.
  * [19] ATLAS Collaboration, “Implementation of simplified likelihoods in HistFactory for searches for supersymmetry”,. {http://cdsweb.cern.ch/record/2782654}.
  * [20] ATLAS Collaboration, “Reproducing searches for new physics with the ATLAS experiment through publication of full statistical likelihoods”,. {http://cdsweb.cern.ch/record/2684863}.
  * [21] K. Cranmer and I. Yavin, “RECAST: Extending the Impact of Existing Analyses”, JHEP 04 (2011) , arXiv:1010.2506 [hep-ex].
  * [22] ATLAS Collaboration, “Reinterpretation of the ATLAS Search for Displaced Hadronic Jets with the RECAST Framework”,. {http://cdsweb.cern.ch/record/2714064}.
  * [23] “REANA: Reproducible research data analysis platform.”  {https://www.reana.io/}.
  * [24] ATLAS Collaboration, “ATLAS 13 TeV samples collection Gamma-Gamma, for 2020 Open Data release. CERN Open Data Portal.”  {http://opendata.cern.ch/record/15006}.
  * [25] ATLAS Collaboration, “ATLAS 13 TeV samples collection exactly three leptons (electron or muon), for 2020 Open Data release. CERN Open Data Portal.”  {http://opendata.cern.ch/record/15004}.
  * [26] ATLAS Collaboration, “ATLAS Virtual Machine, for 2020 Open Data release. CERN Open Data Portal.”  {http://opendata.cern.ch/record/15008}.
  * [27] ATLAS Collaboration, “Athena.”  {https://gitlab.cern.ch/atlas/athena}.
  * [28] ATLAS Collaboration, “Athena”, May, 2021.  https://doi.org/10.5281/zenodo.4772550.
  * [29] ATLAS Collaboration, “Athena Container Repository.”  {https://gitlab.cern.ch/atlas/athena/container_registry}.
  * [30] ATLAS Collaboration, “ATLAS Data Access Policy. CERN Open Data Portal.”  {https://opendata.cern.ch/record/413}.
  * [31] M. Elsing et al., “TrackML Particle Tracking Challenge.”  {https://sites.google.com/site/trackmlparticle/}.
  * [32] ATLAS Collaboration, “Dataset from the ATLAS Higgs Boson Machine Learning Challenge 2014. CERN Open Data Portal.”  {http://opendata.cern.ch/record/328}.
  * [33] ATLAS Collaboration, “FastCaloGAN Training Project”, October, 2021\.  https://doi.org/10.5281/zenodo.5589623.
  * [34] ATLAS Collaboration, “Datasets used to train the Generative Adversarial Networks used in ATLFast3. CERN Open Data Portal.”  {https://opendata.cern.ch/record/15012}.
  * [35] “CERN analysis preservation.”  {https://analysispreservation.cern.ch/}.
  * [36] ATLAS Collaboration, “Performance of Multi-threaded Reconstruction in ATLAS”,. {http://cdsweb.cern.ch/record/2771777}.
  * [37] CMS Collaboration, A. M. Sirunyan et al., “2020 CMS data preservation, re-use and open access policy. CERN Open Data Portal.”  {https://opendata.cern.ch/record/415}.
  * [38] CERN, “CERN open data portal.”  {http://opendata.cern.ch}.
  * [39] CMS Collaboration, A. M. Sirunyan et al., “Container image.”  {https://gitlab.cern.ch/cms-cloud/cmssw-docker-opendata/container_registry}.
  * [40] CMS Collaboration, A. M. Sirunyan et al., “Container image.”  {https://hub.docker.com/u/cmsopendata}.
  * [41] CMS Collaboration, A. M. Sirunyan et al., “VM images.”  {http://opendata.cern.ch/search?page=1&size=20&q=VM&subtype=VM&type=Environment&experiment=CMS&file_type=ova}.
  * [42] CMS Collaboration, A. M. Sirunyan et al., “CMS Open data guide.”  {https://cms-opendata-guide.web.cern.ch/}.
  * [43] “Rivet – the particle-physics MC analysis toolkit.”  {https://rivet.hepforge.org}.
  * [44] LHCb, “Davinci physics analysis application.”  https://twiki.cern.ch/twiki/bin/view/LHCb/DaVinci.
  * [45] H1 Collaboration, I. Abt et al., “The Tracking, calorimeter and muon detectors of the H1 experiment at HERA”, Nucl. Instrum. Meth. A 386 (1997) .
  * [46] H1 Collaboration, I. Abt et al., “The H1 detector at HERA”, Nucl. Instrum. Meth. A 386 (1997) .
  * [47] H1 Collaboration, D. Britzger, S. Levonian, S. Schmitt, and D. South, “Preservation through modernisation: The software of the H1 experiment at HERA”, EPJ Web Conf. 251 (2021) , arXiv:2106.11058 [hep-ex].
  * [48] A. Pfeiffer, “Overview of the lcg applications area software projects”, in IEEE Symposium Conference Record Nuclear Science 2004., vol. 4, pp. 2020–2023 Vol. 4.  2004\.
  * [49] S. Roiser, A. Gaspar, Y. Perrin, and K. Kruzelecki, “Servicing HEP experiments with a complete set of ready integreated and configured common software components”, Journal of Physics: Conference Series 219 no. 4, (Apr, 2010) .
  * [50] H1 Collaboration, V. Andreev et al., “Measurement of multijet production in $ep$ collisions at high $Q^{2}$ and determination of the strong coupling $\alpha_{s}$”, Eur. Phys. J. C 75 no. 2, (2015) , arXiv:1406.4709 [hep-ex].
  * [51] H1 Collaboration, V. Andreev et al., “Measurement of Jet Production Cross Sections in Deep-inelastic $ep$ Scattering at HERA”, Eur. Phys. J. C 77 no. 4, (2017) , arXiv:1611.03421 [hep-ex]. [Erratum: Eur.Phys.J.C 81, 739 (2021)].
  * [52] H1 Collaboration, F. D. Aaron et al., “Inclusive Deep Inelastic Scattering at High $Q^{2}$ with Longitudinally Polarised Lepton Beams at HERA”, JHEP 09 (2012) , arXiv:1206.7007 [hep-ex].
  * [53] H1 Collaboration, V. Andreev et al., “Measurement of lepton-jet correlation in deep-inelastic scattering with the H1 detector using machine learning for unfolding”, arXiv:2108.12376 [hep-ex].
  * [54] M. Arratia, D. Britzger, O. Long, and B. Nachman, “Reconstructing the kinematics of deep inelastic scattering with deep learning”, Nucl. Instrum. Meth. A 1025 (2022) , arXiv:2110.05505 [hep-ex].
  * [55] ZEUS, DESY DPHEP Group Collaboration, J. Malka, “The ZEUS data preservation project”, in 2012 IEEE Nuclear Science Symposium and Medical Imaging Conference and 19th Workshop on Room-Temperature Semiconductor X-ray and Gamma-ray Detectors, pp. 2022–2023.  2012\.
  * [56] A. Bacchetta et al., “Summary of Workshop on Future Physics with HERA Data”, in Future Physics with HERA Data for Current and Planned Experiments.  1, 2016.  arXiv:1601.01499 [hep-ex].
  * [57] A. Geiser, “Possible future HERA analyses”, in Future Physics with HERA Data for Current and Planned Experiments.  12, 2015.  arXiv:1512.03624 [hep-ex].
  * [58] “arXiv.org.”  {https://arxiv.org}.
  * [59] “iNSPIRE: Discover High-Energy Physics Content.”  {https://inspirehep.net}.
  * [60] ZEUS Collaboration, A. Verbytskyi, “The ZEUS long term data preservation project”, PoS DIS2016 (2016) , arXiv:1607.01898 [hep-ex].
  * [61] D. Krücker, K. Schwank, P. Fuhrmann, B. Lewendel, and D. M. South, “Data preservation for the HERA experiments at DESY using dCache technology”, J. Phys. Conf. Ser. 664 no. 4, (2015) .
  * [62] “PUNCH4NFDI: Particles, Universe, Nuclei and Hadrons for the NFDI.”  {https://www.punch4nfdi.de}.
  * [63] “Key4hep: A Common Software Ecosystem.”  {https://github.com/key4hep}.
  * [64] B. Naroska, “$e^{+}e^{-}$ Physics with the JADE Detector at PETRA”, Phys. Rept. 148 (1987) .
  * [65] G. Corcella et al., “HERWIG 6: An Event generator for hadron emission reactions with interfering gluons (including supersymmetric processes)”, JHEP 01 (2001) , arXiv:hep-ph/0011363.
  * [66] T. Sjostrand, S. Mrenna, and P.Z. Skands, “PYTHIA 6.4 Physics and Manual”, JHEP 05 (2006) , arXiv:hep-ph/0603175.
  * [67] T. Sjostrand, “High-energy physics event generation with PYTHIA 5.7 and JETSET 7.4”, Comput. Phys. Commun. 82 (1994) .
  * [68] I. Antcheva et al., “ROOT: A C++ framework for petabyte data storage, statistical analysis and visualization”, Comput. Phys. Commun. 182 (2011) .
  * [69] R. Brun et al., “PAW: PHYSICS ANALYSIS WORKSTATION: INCLUDING VERSION 1.03/05.” CERN-Q-121, 2, 1988.
  * [70] G. Behrmann et al., “A distributed storage system with dCache”, J. Phys. Conf. Ser. 119 (2008) .
  * [71] P.A. Movilla-Fernandez, “Studien zur Quantenchromodynamik und Messung der starken Kopplungskonstanten $\alpha(s)$ bei $S^{1/2}=14-GeV-44-GeV$ mit dem JADE-Detektor”, other thesis, Aachen, Tech. Hochsch., 11, 2002.
  * [72] R. Bock et al., “HIGZ: HIGH LEVEL INTERFACE TO GRAPHICS AND ZEBRA.” CERN-Q-120, 3, 1988.
  * [73] JADE Collaboration, J. Schieck et al, “Measurement of the strong coupling $alpha_{S}$ from the three-jet rate in $e^{+}e^{-}$ \- annihilation using JADE data”, Eur. Phys. J. C 73 no. 3, (2013) , arXiv:1205.3714 [hep-ex].
  * [74] JADE Collaboration, C. Pahl et al., “Study of moments of event shapes and a determination of alpha(S) using e+ e- annihilation data from Jade”, Eur. Phys. J. C 60 (2009) , arXiv:0810.2933 [hep-ex]. [Erratum: Eur.Phys.J.C 62, 451–452 (2009)].
  * [75] S. Amerio et al., “Data preservation at the Fermilab Tevatron”, Nucl. Instrum. Meth. A 851 (2017) , arXiv:1701.07773 [hep-ex].
  * [76] BESIII Collaboration, M. Ablikim et al., “Future Physics Programme of BESIII”, Chin. Phys. C 44 no. 4, (2020) , arXiv:1912.05983 [hep-ex].
  * [77] Belle-II Collaboration, T. Abe et al., “Belle II Technical Design Report”, arXiv:1011.0352 [physics.ins-det].
  * [78] Belle-II Collaboration, “Belle II luminosity.”  {https://confluence.desy.de/display/BI/Belle+II+Luminosity#BelleIILuminosity-Totalrecordedintegratedluminosity}.
  * [79] Belle-II Collaboration, I. Adachi et al., “Search for an Invisibly Decaying $Z^{\prime}$ Boson at Belle II in $e^{+}e^{-}\to\mu^{+}\mu^{-}(e^{\pm}\mu^{\mp})$ Plus Missing Energy Final States”, Phys. Rev. Lett. 124 no. 14, (2020) , arXiv:1912.11276 [hep-ex].
  * [80] Belle-II Collaboration, F. Abudinén et al., “Search for Axion-Like Particles produced in $e^{+}e^{-}$ collisions at Belle II”, Phys. Rev. Lett. 125 no. 16, (2020) , arXiv:2007.13071 [hep-ex].
  * [81] Belle Collaboration, A. Abashian et al., “The Belle Detector”, Nucl. Instrum. Meth. A 479 (2002) .
  * [82] Belle Collaboration, “Belle luminosity.”  {https://belle.kek.jp/bdocs/lumi_belle.png}.
  * [83] MINERvA Collaboration, L. Aliaga et al., “MINERvA neutrino detector response measured with test beam data”, Nucl. Instrum. Meth. A 789 (2015) , arXiv:1501.06431 [physics.ins-det].
  * [84] MINERvA Collaboration, “Publications web page”, 2023.  https://minerva.fnal.gov/recent-minerva-results/.
  * [85] R. Fine et al., “Data Preservation at MINERvA”, arXiv:2009.04548 [hep-ex].
  * [86] MAT, “Git repository”, 2023.  https://github.com/MinervaExpt/MAT.
  * [87] MAT-MINERvA, “Git repository”, 2023.  https://github.com/MinervaExpt/MAT-MINERvA.
  * [88] R. Brun et al., “GEANT3.” CERN-DD-EE-84-1, 9, 1987.
  * [89] E. Maguire, L. Heinrich, and G. Watt, “HEPData: a repository for high energy physics data”, J. Phys. Conf. Ser. 898 no. 10, (2017) , arXiv:1704.05473 [hep-ex].
  * [90] T. Šimko et al., “Open data provenance and reproducibility: a case study from publishing CMS open data”, EPJ Web Conf. 245 (2020) .
  * [91] G. De Lellis, S. Dmitrievsky, G. Galati, A. Lavasa, T. Šimko, I. Tsanaktsidis, and A. Ustyuzhanin, “Dataset of tau neutrino interactions recorded by the OPERA experiment”, EPJ Web Conf. 245 (2020) .
  * [92] T. Šimko, L. Heinrich, H. Hirvonsalo, D. Kousidis, and D. Rodríguez, “REANA: A System for Reusable Research Data Analyses”, EPJ Web Conf. 214 (2019) .
  * [93] T. Šimko, L. A. Heinrich, C. Lange, A. E. Lintuluoto, D. M. MacDonell, A. Mečionis, D. Rodríguez Rodríguez, P. Shandilya, and M. Vidal García, “Scalable Declarative HEP Analysis Workflows for Containerised Compute Clouds”, Front. Big Data 4 (2021) .
  * [94] P. B. Stark, “Before reproducibility must come preproducibility”, Nature 557 (2018) .
  * [95] T. Šimko, K. Cranmer, M. R. Crusoe, L. Heinrich, A. Khodak, D. Kousidis, and D. Rodriguez, “Search for computational workflow synergies in reproducible research data analyses in particle physics and life sciences”, in 14th International Conference on e-Science, pp. 403–404.  2018\.
  * [96] P. Buncic, C. A. Sanchez, J. Blomer, A. Harutyunyan, and M. Mudrinic, “A practical approach to virtualiztion in HEP”, The European Physical Journal Plus 126 no. 1, (2011) .
  * [97] J. Blomer, C. Aguado-Sanchez, P. Buncic, and A. Harutyunyan, “Distributing LHC application software and conditions databases using the CernVM file system”, Journal of Physics: Conference Series 331 no. 042003, (2011) .
  * [98] F. Berghaus, J. Blomer, S. D. Tiessen, G. C. Melia, G. Ganis, J. Shiers, and T. Simko, “CERN services for long term data preservation”, in Proc. 13th int. conf. Digital Preservation (iPres’16).  2016\.
  * [99] P. DuBois, Software portability with imake - practical software engineering (2. ed.).  O’Reilly, 1996.
  * [100] Kitware, “Cmake”, 2022.  https://cmake.org/.
  * [101] “ARCHIVER Project.”  {https://archiver-project.eu/}.
  * [102] “ARCHIVER white paper draft:.”  {https://docs.google.com/document/d/1jc1J3ouLI1LtRKGQhVsRl3XoIrkFFF5bwhcX4UEZ-Q4/edit#}.

##  Appendix A The DPHEP Collaboration

DPHEP Collaboration

T. Basaglia1, M. BellisI,2, J. Blomer1, J. Boyd1, C. Bozzi3, D. Britzger4, S.
Campana1, C. Cartaro5, G. Chen6, B. Couturier1, G. DavidII,7, C. Diaconu8, A.
Dobrin9, D. Duellmann1, M. Ebert10, P. Elmer11, J. Fernandes1, L. Fields21, P.
Fokianos1, G. Ganis1, A. Geiser12, M. Gheata9, J. B. Gonzalez Lopez1, T.
Hara13, L. Heinrich1, M. Hildreth21, K. Herner14, B. Jayatilaka14, M. Kado1,
O. Keeble1, A. Kohls1, K. Naim1, C. Lange20, K. Lassila-Perini15, S.
Levonian12, M. Maggi16, Z. Marshall18, P. Mato Vila1, A. Mečionis1, A.
Morris17, S. Piano16, M. Potekhin7 M. Schröder1, U. Schwickerath1, E. Sexton-
Kennedy14, T. Šimko1, T. Smith1, D. South12, A. Verbytskyi4, M. Vidal1, A.
Vivace1, L. Wang6, G. Watt19, T. Wenaus7

Affiliation Notes

I Also at: Siena College

II Also at: Stony Brook University

Collaboration Institutes

1 CERN, Geneva, Switzerland

2 Cornell University, USA

3 INFN Ferrara, Italy

4 Max-Planck-Institut für Physik, München, Germany

5 SLAC National Accelerator Laboratory, USA

6 Institute of High Energy Physics, IHEP, CAS, Bejing, China

7 Brookhaven National Laboratory, BNL, USA

8 Aix Marseille Univ, CNRS/IN2P3, CPPM, Marseille, France

9 Institute of Space Science, ISS, Bucharest, Magurele, Romania

10 HEP Research Computing, University of Victoria, BC, Canada

11 Princeton University, USA

12 Deutsches Elektronen Synchrotron, DESY, Hamburg, Germany

13 High Energy Accelerator Research Organization, KEK, Tsukuba, Japan

14 Fermi National Accelerator Laboratory, Batavia, USA

15 Helsinki Institute of Physics, Finland

16 INFN Trieste, Italy

17 University of Bonn, Germany

18 Lawrence Berkeley National Laboratory, Berkeley, USA

19 IPPP, Durham University, Durham, UK

20 Paul Scherrer Institut, Villigen, Switzerland

21 University of Notre Dame, Notre Dame, USA