# SCC5G: A PQC-based Architecture for Highly Secure Critical Communication
over Cellular Network in Zero-Trust Environment ††thanks: This material is
based upon the work supported by the US National Science Foundation under
Grant No. IIP-2204502 and the Air Force Office of Scientific Research under
award number FA9550-20-1-0090.

Mohammed Gharib1, Fatemeh Afghah1  1Department of Electrical and Computer
Engineering, Clemson University, Clemson, SC, USA
<EMAIL_ADDRESS>
###### Abstract

5G made a significant jump in cellular network security by offering enhanced
subscriber identity protection and a user-network mutual authentication
implementation. However, it still does not fully follow the zero-trust (ZT)
requirements, as users need to trust the network, 5G network is not
necessarily authenticated in each communication instance, and there is no
mutual authentication between end users. When critical communications need to
use commercial networks, but the environment is ZT, specific security
architecture is needed to provide security services that do not rely on any 5G
network trusted authority. In this paper, we propose SCC5G Secure Critical-
mission Communication over a 5G network in ZT setting. SCC5G is a post-quantum
cryptography (PQC) security solution that loads an embedded hardware root of
authentication (HRA), such as physically unclonable functions (PUF), into the
users’ devices, to achieve tamper-resistant and unclonability features for
authentication and key agreement. We evaluate the performance of the proposed
architecture through an exhaustive simulation of a 5G network in an ns-3
network simulator. Results verify the scalability and efficiency of SCC5G by
showing that it poses only a few kilobytes of traffic overhead and adds only
an order of $O(0.1)$ second of latency under the normal traffic load.

###### Index Terms:

Security, Cellular Network, Zero-Trust, PUF, 5G

##  I Introduction

Communication is an essential component of any critical-mission application
such as disaster relief, military missions, etc. In many cases, mission-
specific communication might not be available due to the coverage shortage,
capacity limitation, and very fast network topology change. Availability,
network capacity, mobility support, and large coverage area of cellular
networks make them an exceptional choice for critical-mission use when a
dedicated communication link is unavailable. Three different scenarios are
discussed in [1] where the mission-critical applications require 5G
communication, for machine-to-machine communication, for hypersonic weapon
command and control, and for millimeter wave (mmWave) usecases where there
will be a large bandwidth with ultra-low latency. The focus of the Department
of Defence (DoD) on 5G testing and experimentation for warfighting is also
evidence of the importance of mission-critical 5G communication. DoD invested
more than 600 million in 2020 for 5G testbeds [2], deployed at 12 sites across
the country. Working with more than 100 industry partners, DoD expects the
full operation of the dual-use 5G cellular network in 2023 [2]. Besides the
DoD effort, many researchers proposed various mission-critical use cases for
5G networks and showed how they can advance such mission operations [3, 4, 5,
6, 7].

5G network, in addition to the aforementioned cellular network advantages, has
a striking enhancement in terms of communication security in comparison with
its predecessors, especially in identity protection and utilizing more robust
encryption algorithms. However, security is still the main concern in
utilizing them for critical-mission communication since 5G does not fully
follow the National Institute of Standard and Technology (NIST) and the
Department of Defense (DoD) recommendation of considering ZT architecture. In
ZT architecture there is no pre-assumed trust and every communication instance
should be protected by verifying the communicating parties regardless of their
location and ownership [8, 9]. Critical-mission agents not only deal with a ZT
network where they cannot rely on the network’s regular security protocols,
but they are also at constant risk of physical attacks when closely
interacting with adversary agents in “unknown” or “contested” environments.

Generally, three main reasons make a significant need for a new architecture
for critical-mission communication over a 5G network, $i)$ in a 5G network,
the mutual authentication is between the user and the network, not between the
end users, $ii)$ the entire authentication process is based on the Sim card
which can be stolen, and $iii)$ the network is responsible for the
authentication verification, and hence the critical-mission agents need to
trust the network. Regardless of the application domain, NIST [10] and NSA
[11] recommended that critical-mission communication should be migrated to
post-quantum cryptography (PQC) architectures in addition to accommodating ZT
requirements. To achieve this goal, a robust PQC-based key management system
capable of providing end-to-end encrypted communication is required where each
node should be authenticated for each communication instance.

In this paper, we present a PQC-based security architecture for mission-
critical communication, utilizing a 5G cellular network as the communicating
layer, considering ZT settings. This architecture is referred to as SCC5G,
Secure Critical-mission Communication over a 5G network. It protects the
agents’ devices, e.g. cell phones, ground vehicles, unmanned aerial vehicles,
IoT devices, etc., from revealing secret information even if they are targeted
by physical hijacking attacks. SCC5G forces a mutual authentication for each
communication session to fulfill the ZT requirement. To protect the critical-
mission users’ devices, SCC5G provides each user with a hardware root of
authentication (HRA), which is a pre-loaded hardware in the user device. The
HRA should be able to take a challenge and respond with a unique response.
SCC5G requires the HRA to be physically unclonable, tamper-resistant, and have
high entropy in its challenge-response pairs, i.e. have a large enough state-
space of challenge-response pairs. Commercial 5G networks can provide mutual
authentication among users using the users’ SIM cards as the hardware root of
trust (HRT). However, a SIM card is just used as hardware identification and
could be cloned or tampered.

We assume that the commercial 5G network cooperates with the critical-mission
agency to add trusted authority referredd to as HRA authentication and keying
function (HAKF) into its structure. SCC5G keeps the security protocols of the
5G network and adds another layer of security by utilizing the HRA devices
embedded in the users’ communication devices, to provide an asymmetric
cryptosystem that does not rely on any 5G network trusted authority. The idea
behind SCC5G is to bind the public-private key pair generated by a critical-
mission user’s device to the embedded HRA by using the hashed readings stored
in the HAKF. It will verify the originality of the generated public key as
well as the identity of the device. The entire solution relies on ID
verification in every communication session to address the ZT requirements.
The unclonability and high entropy properties of the HRA guarantee the
uniqueness of the verification part. While any hardware with the mentioned
properties could be utilized within SCC5G architecture, we suggest using a
physical unclonable function (PUF) as an HRA. PUFs exploit the fabrication
variation of the microelectronic devices to generate a unique response for the
corresponding input.

SCC5G has an initialization phase before the network starts its operation. At
the initialization phase, the network admin chooses the required security
parameters and forms a certification storage. The certification storage is
used for certificate verification and includes non-confidential information.
It includes the hashed values of the image of each HRA, bound to the ID of the
carrying user. The network starts its operation after the initialization
phase. By starting the operation, each user can utilize the cellular network
to connect to any other user. The proposed architecture encrypts the
communication from end to end and authenticates end users for any
communication instance. In SCC5G, users are responsible for generating their
cryptographic keys based on the PQC cryptosystem and the pre-defined
cryptographic parameters. The security and the performance of the proposed
architecture are comprehensively studied. We show that SCC5G carries the NIST-
recommended properties [12] for the secure key management system and it is
resistant to well-known key management attacks such as man-in-the-middle
(MitM). We use network simulator ns-3 [13] to evaluate the performance of the
proposed architecture and verify its efficiency and scalability. Results show
that both the communication overhead and latency of SCC5G architecture are
negligible with the order of $O(1)$ KB and $O(0.1)$ second, respectively, in
the network with a normal load.

The main contribution of this paper is to propose a novel architecture for
utilizing the existing 5G cellular network for critical-mission communication
in a zero-trust setting. It adds another layer of security by encrypting the
data communication in an end-to-end manner, mutually authenticating the end
users, and supporting PQC cryptography. It further enhances data transfer
communication by providing a session key agreement for data communication.
Comprehensive security and performance evaluation validates the feasibility
and dependability of the proposed architecture.

The rest of this paper is organized as follows. We review the related work in
Section (II) and present the system model in Section (III). The details of the
proposed architecture are reviewed in Section (IV). We evaluate the security
and the performance of the proposed architecture in Section (V). Finally, we
conclude the paper and mention the future directions in Section (VI).

##  II Related Work

There are over 40 types of suggested PUF technologies where some of which are
based on the embedded memory of the hardware. The advantage of utilizing
embedded memory as a PUF is the fact that they are widely available in cyber-
physical systems. Examples of the utilized embedded memories include SRAMs,
DRAMs, Flash RAMs, MRAM, and ReRAM [14]. The ReRAM advantage for SCC5G over
the other memory-based PUF technologies is its tamper-resistant property and
self-destruction mechanism in case of outsider manipulation [15]. However,
there are two main challenges in utilizing PUF in cryptosystems, the need for
a trusted server to store the image of challenge-response pairs, and the
extreme sensitivity of the entire system to the PUF response variation.

The PUF-based security solutions have been mostly utilized in trustable
networks since they require a secure server to keep a record of original PUF
responses. The initial PUF responses are generated with a set of challenges
and are stored as references in secure servers. During the authentication
cycle, PUFs are challenged to generate fresh responses that should match the
reference responses stored in the network. Ren et al. [16] proposed a PUF-
based group authentication scheme, where a repository of all challenge-
response pairs is stored in the Authentication Server Function (AUSF) in the
home network. As an instance, Alladi et al. [17] proposed an authentication
scheme for drone-assisted 5G networks. While the protocol offers user-to-user
mutual authentication, it stores the entire challenge-response pairs in the
connected base station and assumes that all base stations have secure
communication in between. Mall et al. [18] published a survey including 44
PUF-based authentication and key agreement protocols for the Internet of thing
(IoT), wireless sensor network (WSN), and smart grids. However, all of the
reviewed works need to store the plain challenge-response pairs somewhere in
the network. Although the mentioned requirement may not seem to be
unacceptable in WSN, IoT, and smart grids, it is an obvious security threat to
critical-mission communication networks. SCC5G architecture does not store the
challenge-response pairs, instead, it just keeps the hash value of the PUF
image in HAKF. Although the HAKF is not a 5G component and could be trusted by
the critical-mission agents, it does not have access to the PUF images.

The use of PUFs for cryptographic key generation is the most challenging
problem since even a single-bit mismatch between the challenge and response
could not be tolerated. Robust secret key generation heavily relies on fuzzy
extractors[19] and error-correcting methods which use a set of publicly
available helper data. Most of the current PUF technologies suffer from
vulnerability to side-channel analysis. The newly developed ultra-low power
tamper-resistant ReRAM-based PUFs with a self-destruction mechanism, however,
represented promising performance in mitigating this shortcoming [19, 15]. The
developed masking techniques added also another layer of protection for the
PUF-based key generation against helper data manipulation attacks [20]. Noting
the extreme sensitivity of PUF-based key generation to variations of PUF
responses, such methods require long helper data and trustable servers, which
is not feasible in a zero-trust environment. In this work, we utilize the
embedded PUFs as a means to certify the devices’ public keys.

##  III System Model

SCC5G utilizes the existing 5G cellular communication as an underlay and adds
an overlay secure communication level at the top of it. The sage cloud in Fig.
(1) represents the existing 5G cellular network where both third-generation
partnership project (3GPP) users, i.e. SIM card-based users, and non-3GPP
users, i.e. users without SIM card chipset on their devices, can connect to
the network. Similarly, the cloud with cantaloupe color represents the overlay
formed by SCC5G architecture for critical-mission secure communication, where
the users can be 3GPP or non-3GPP users, but need to have their own HRA. We
will use user and critical-mission user interchangeably throughout the paper,
since the architecture is proposed for mission-critical users, even in the
presence of the typical 5G users. All non-3GPP users, critical-mission or
typical, connect to the 5G core network through Non-3GPP Interworking Function
(N3IWF) across IP security (IPsec) tunnel. N3IWF plays the role of a virtual
private network (VPN) server allowing the non-3GPP users to establish an IPsec
tunnel into the 5G core network.

Figure 1: SCC5G architecture; Secure Critical-mission Communication
architecture over a 5G network.

The user’s physical connections to the 5G core network or N3IWF could be held
through the radio channel provided by WiFi or next generation node B (gNodeB)
or even through the cable network. We divided the 5G core network into two
parts based on their physical connections with the other network parts to be
direct radio access or to be behind the IP network. We name the part of the
core network with direct radio access as the serving network and the latter
one as the home network. Such naming is not novel as it could be seen in [21].
Access and Mobility Management Function (AMF) which is responsible for
connecting the users through different interfaces with 5G core networks, and
Security Anchor Function (SEAF) which is responsible for initializing the 5G
authentication process are located in the serving network.

The second part of the 5G core network is located behind the IP network,
hence, theoretically, its building blocks could be located at any physical
location. Unified Data Management (UDM) which is responsible for managing the
subscribers’ data, and Subscription Identifier De-concealing Function (SIDF)
which is responsible for decrypting subscribers’ IDs, i.e. Subscription
Concealed Identifier (SUCI), are two of the 5G home network building blocks.
The Authentication Server Function (AUSF) and Authentication Credential
Repository and Processing Function (ARPF) are two other building blocks of the
5G home network. ARPF is responsible for authentication method selection based
on the configured policy and subscriber identity. AUSF is the responsible
party for making the decision about authentication or denying the
authentication of the users based on some keying material that ARPF is
responsible for computing them. In SCC5G, we add HRA authentication and keying
function (HAKF) to the 5G home network considering that the 5G network is
cooperative with the critical-mission administration. We leave the extension
of this work to be compatible with a non-cooperative 5G network for future
work. HAKF contains the database of the hashed images of the HRAs. Each image
is bound to the corresponding user ID. Since the HAKF is placed in the home
network, it could be located at any secure location, and it does not interfere
with the regular 5G core network functionality.

######  Assumption 1.

In SCC5G architecture, we assume that the 5G network is cooperative with the
critical-mission network administration.

######  Assumption 2.

We assume that the critical-mission agents will not communicate or respond to
any critical-mission user communication if the authentication process fails.

######  Assumption 3.

We assume that the utilized cryptosystem saves the confidentiality and
integrity of the encrypted data for everyone except for the ones who have the
proper key. The proof of the correctness of the utilized cryptosystem is
beyond the scope of this paper.

##  IV Proposed SCC5G Architecture

In this section, we present the details of SCC5G, secure critical-mission
communication over a 5G network. The aim is to utilize the existing 5G network
for critical-mission communication with end-to-end encryption, considering a
ZT setting, where none of the network entities is assumed to be trusted. The
proposed architecture load each user with an HRA and store the hash of the
HRA’s image in a central database. SCC5G architecture has two phases, the
initialization phase, and the network operation phase. In this architecture,
each user is responsible for generating its own keys. To authenticate the keys
and bind them to the users, the image of the HRA is pre-hashed and loaded in a
part of the network, behind the IP network. HRA authentication and keying
function (HAKF) is the network part that we added to the general 5G
architecture to be responsible for storing the hashed HRA images and
performing the authentication and certification-related tasks. In this
section, we describe the details of the network initialization and operation
phases in separate parts.

Network Initialization Phase: The network initialization phase includes three
main steps, providing each critical-mission user with an HRA, choosing the
cryptosystem parameters, and forming the certification database. During an
enrolment phase performed in a secure ground, each user is loaded with an HRA.
The HRA could be any hardware with four properties, $i)$ to be physically
unclonable, $ii)$ to be tamper resistant, $iii)$ returns a unique response for
any unique challenge, and $iv)$ has high entropy. The high entropy means
having a large enough number of challenge-response pairs. The network
initialization phase further contains a selection for cryptosystem parameters.
SCC5G architecture is designed to be compatible with any asymmetric
cryptosystem. However, we use the recent NIST-recommended PQC cryptosystem
CRYSTAL-Kyber [22] for key encapsulation mechanism (KEM) and CRYSTAL-Dilithium
[23] for digital signature, without the loss of generality. The network admin
chooses the required cryptosystem parameters at the initialization phase of
the network. In the case of CRYSTAL-Kyber and CRYSTAL-Dilithium, the network
admin chooses $k,d_{t},d_{u},d_{v},$ and $q$, representing the number of
columns in matrix A, the size of $t$, $u$, and $v$ vectors, and the size of
the ring of the learning with error (LWE) problem, respectively. Each user, in
its turn, forms the image of its HRA, i.e. all possible challenge-response
pairs, hashes the responses, and sends the resulting table to the network
admin. Network admin, in his turn, stores the hashed challenge-response pairs
of each user bounded to their IDs in a database, for certification purposes.
Considering the ZT setting, each user for any communication instance needs to
be authenticated. To authenticate the user, two verification parts are used, a
signed public key by the user’s private key, and a hashed response of the
user’s HRA where the challenge is extracted from the public key. Algorithm (1)
represents the pseudocode of the initialization process.

TABLE I: Table of notations

$C$ | The challenge
---|---
$R$ | The response
$f(.)$ | The HRA function transforms C into its corresponding R.
$H(.)$ | Hash function.
$n$ | Number of critical-mission users.
$U_{i}$ | The $i$th user.
$HRA_{i}$ | The HRA loaded into the user $U_{i}$.
$\rho$ | The seed to be expanded to generate matrix $A$.
$\sigma$ | The seed to be expanded to generate vector $s$.
$e$ | Additive error.
$s_{i}$ | The private key of the user $U_{i}$.
$Y_{i}=(A,t)$ | The public key of the user $U_{i}$.
$m$ | The plain text
$c=(u,v)$ | The cipher text, i.e. the concatenation of vectors $u$ and $v$.
$d_{t}$ | The dimension of vector $t$.
$d_{u}$ | The dimension of the vector $u$.
$d_{v}$ | The dimension of the vector $v$.
$k$ | Number of columns in matrix $A$.
$S_{i,j}$ | The session key between users $U_{i}$ and $U_{j}$.
$q$ | A large prime number representing the
| ring size of the LWE problem.

$[k,d_{t},d_{u},d_{v},q]\leftarrow$defineNetParam(), {The network admin
chooses the network parameters.}

Form($CertificateDB$), {The network admin forms a database $CertificateDB$ for
certification process.}

for $(i=1:n)$ do

Load($U_{i},HRA_{i}$), {The $i$th user is loaded with an HRA.}

$j\leftarrow 1$, {A counter for the number of challenge-response pairs.}

Form($DB_{i}$), {User $U_{i}$ forms its own database $DB_{i}$.}

while $!endOf(HRA_{i})$ do

$H_{j}\leftarrow$ Form($ID_{i},C_{j},H(f(C_{j}))$), {User $U_{i}$ forms a
database entry containing its ID, challenges $C_{j}$, and the hashed value of
the corresponding response.}

Concat($H_{j},DB_{i}$), {User $U_{i}$ concatenate the $H_{j}$ entry to its own
database.}

end while

Concat($DB_{i},CertificateDB$), {Newtork admin concatenate the user $U_{i}$
database into the certification database.}

end for

ALGORITHM 1 SCC5G Initialization Phase

Network Operation Phase: In this architecture, each user is responsible for
generating its pair of public-private keys. When the network starts working,
i.e. operational phase of the network, each user chooses two random numbers
$\rho$ and $\sigma$ to generate

$A\sim R^{k\times k}_{q}:=Sam(\rho),$ (1)
$(s,e)\sim\beta_{\eta}^{k}\times\beta_{\eta}^{k}:=Sam(\sigma),$ (2)

and

$t:=Compress_{q}(As+e,d_{t}),$ (3)

where $(A,t)$ and $s$ are the user’s public key and private key, respectively.
Here, $A$ is a $k\times k$ matrix in the Galois Field of order $q$ where
$Sam(.)$ and $Compress(.)$ are sampling and compression functions,
respectively. In this system, $A\times s+e=t$, where $e$ is an additive error
and the security of the system is based on the hardness of module learning
with error (M-LWE) problem. For more detailed information about how this PQC
system works, we refer the researchers to read CRYSTAL-Kyber main paper [22].

Figure 2: SCC5G mutual authentication handshake process.

1 Handshake process between $U_{i}$ and $U_{j}$, initiated by $U_{i}$.User
$U_{i}$

$R_{i}\leftarrow f(Y_{i})$

$Sgn_{i}\leftarrow Sign(Y_{i})_{s_{i}}$, {Function $Sign(m)_{s_{i}}$ signs the
message $m$ with the key $s_{i}$.}

$Cert_{i}\leftarrow$ Concat($ID_{i},Sgn(i),H(R_{i})$), {The function Concat()
concatenates the included arguments.}

Send($Y_{i},Cert_{i})\rightarrow U_{j}$

2User $U_{j}$

$v\leftarrow Verify(Y_{i},Cert_{i})$,{Function Verify() returns a boolean
value representing the verification process result for the included sign and
key.}

if $(v)$ then

ver_Req$\leftarrow$ Concat($ID_{i},Y_{i},H(R_{i})$)

Send$($ver_Req$)\rightarrow$ HAKF

else

Terminate the handshake process, {Handshake failure}

end if

3HAKF

$R\leftarrow$ Retrieve($ID_{i},Y_{i}$),{Function Retrieve() retrieves the
content of the provided address from the database.}

if $(R==H(R_{i}))$ then

ver_Res$\leftarrow$ True

else

ver_Res$\leftarrow$ False

end if

Send(ver_Res)$\rightarrow U_{j}$

4User $U_{j}$

if ($!$ ver_Res) then

$R_{j}\leftarrow f(Y_{j})$

$Sgn_{j}\leftarrow Sign(Y_{j})_{s_{j}}$

$Cert_{j}\leftarrow$ Concat($ID_{j},Sgn(j),H(R_{j})$)

Send($Y_{j},Cert_{j})\rightarrow U_{i}$

$S_{i,j}\leftarrow Encapsulate(Y_{i})$

else

Terminate the handshake process,{Handshake failure}

end if

5User $U_{i}$

$v\leftarrow Verify(Y_{j},Cert_{j})$

if $(v)$ then

ver_Req$\leftarrow$ Concat($ID_{j},Y_{j},H(R_{j})$)

Send$($ver_Req$)\rightarrow$ HAKF

else

Terminate the handshake process,{Handshake failure}

end if

6HAKF

$R\leftarrow$ Retrieve($ID_{j},Y_{j}$)

if $(R==H(R_{j}))$ then

ver_Res$\leftarrow$ True

else

ver_Res$\leftarrow$ False

end if

Send(ver_Res)$\rightarrow U_{i}$

7User $U_{i}$

if ($!$ ver_Res) then

$S_{i,j}\leftarrow Encapsulate(Y_{j})$

else

Terminate the handshake process,{Handshake failure}

end if

ALGORITHM 2 SCC5G Handshake Process

For any communication instance, a mutual authentication followed by a session
key agreement between the two communicating end users is required. Fig. (2)
represents the handshake process in a diagram. Considering the challenge-
response process of the HRA as a function $f(.)$, the user who requires to
establish a new communication instance, gives its public key as the input to
its HRA, takes the response $f(A,t)$, hashes the response to generate the
verification part, i.e. $H(f(A,t))$, and forms a certificate by concatenating
his ID, public key, signed public key by the private key, and the hashed
response of the HRA. The initiating user sends the concatenated message to the
destination through the 5G network. The destination receives the message and
takes four actions, it checks the validity of the signed public key, sends a
verification request to HAKF to verify the verification part, generates and
sends a public key certificate in the same way as the source user, and
calculates a session key $S_{i,j}$. The source node, in its turn, gets the
certificate, extracts the public key of the destination, verifies the
destination public key with the help of HAKF, and calculates a session key
$S_{i,j}$. The session key can be calculated by the two endpoints when they
have the public keys of one another as

$S_{i,j}=Encapsulate(Y_{i})=Encapsulate(Y_{j}).$ (4)

It is worth mentioning that the users send a certificate including their
public key in all of their communications with HAKF. While the communication
from each user to the HAKF includes only public information, the HAKF encrypts
its response to the verification request by the public key of the requesting
node, to protect the response from manipulation. The pseudocode of the SCC5G
mutual authentication handshake process is represented in Algorithm (2). In
Section (V), we show that the proposed handshake has a negligible overhead,
follows the NIST key management recommendations [12], and is resistant to the
well-known attacks.

##  V Security and Performance Evaluation

Security and performance of the proposed architecture are required to be
carefully considered together in the evaluation process, as there may exist a
highly secure architecture but carrying high complexity in terms of
computation, time, storage, etc. which makes it infeasible to be implemented
in the real world. In this section, we first evaluate the security of SCC5G in
terms of its security properties that might be interpreted as resiliency
against different well-known attacks. Then, we exhaustively evaluate the
performance of the mutual authentication and session key generation algorithm
which is considered the core algorithm of the proposed architecture, in terms
of latency and communication overhead.

###  V-A Security Evaluation

We show in this section that SCC5G carries the essential key management
properties (explicit key authentication, key freshness, perfect forward
secrecy, backward secrecy, key independence), provides the primary security
services (confidentiality, integrity, authentication, authorization, and non-
repudiation), and is resistant against the most well-known key management
attacks (MitM attack and ID-spoofing).

######  Definition 1.

Explicit key authentication is the process of authenticating a key with the
knowledge that the intended user owns the corresponding private key.

######  Lemma 1.

SCC5G has explicit key authentication property.

###### Proof.

Since the public key certificate in SCC5G binds the ID, public key, signed
public key, and the unclonable tamper-resistant response of the user’s HRA
together, it authenticates the key and proves its ownership. The public key
along with the sign verify that the certificate ties the public key and its
private key, whereas the verification part extracted from the HRA certifies
that the public key is for the intended user. Hence, SCC5G carries the
explicit key authentication property. ∎

######  Definition 2.

Forward and backward secrecy are two properties for the cryptographic keys
which state that revealing a key should not reveal any other upcoming key or
former key, respectively. In other words, there should be no relationship
between a key and its successor and/or predecessor keys that make them
extractable from one another. The perfect forward secrecy ensures that there
is no long-term key that if revealed may lead to revealing a new key. A long-
term key example is the master private key in the cryptosystems where the
entire system security is built based on a single master key.

It is obvious that in SCC5G there is no dependency between the keys, as they
are randomly chosen by the users. Hence, it carries perfect forward and
backward secrecy in addition to key independence property. The keys can also
be updated at any time by the user, thus, SCC5G carries key freshness
property, too.

######  Lemma 2.

SCC5G is resistant to ID spoofing attacks.

###### Proof.

First, SCC5G requires mutual authentication for each and every communication
instance. Second, SCC5G binds the user’s ID to the key, and to the
certificate. First and second together guarantee the verification of the
users’ IDs for each communication instance which makes the SCC5G resistant to
ID spoofing attacks. ∎

######  Lemma 3.

SCC5G is resistant to man-in-the-middle attacks.

###### Proof.

In a man-in-the-middle attack, the attacker inserts itself between the user
and the certifying entity or between two users and foists itself instead of
the other communication party. The uniqueness of the HRA responses and the
bind between the user ID, public key, and the HRA response makes the man-in-
the-middle attack infeasible. ∎

######  Lemma 4.

SCC5G architecture provides confidentiality and integrity services for
critical-mission users communicating over a commercial 5G network.

###### Proof.

To provide confidentiality, end-to-end encryption is required to assure that
the encryption key(s) has/have not been revealed or forged. To provide
integrity, in addition to the end-to-end encryption condition, the
cryptosystem is also required to keep the encrypted data correlated such that
altering or omitting any bit damages the entire block of data. SCC5G provides
the users with end-to-end PQC encryption. Hence, the confidentiality and the
integrity of the communication are conditioned by the confidentiality of the
utilized cryptosystem (CRYSTAL-Kyber and CRYSTAL-Dilithium), and the
resistance of the architecture against revealing the keys. According to
Assumption (3), the utilized cryptosystem keeps the encrypted data
confidential and integrated. Since the private key of each user is generated
by the user himself and not communicated over the network, the attacker cannot
get access to it except by physically compromising the user. Even in the case
of the physical compromise, since the key is bound to the HRA and the HRA is
tamper-resistant, the attacker cannot generate a new key. The only other known
way to attack the confidentiality and the integrity of communication is to
perform a man-in-the-middle attack, where according to Lemma (3), SCC5G is
resistant to this attack and hence, it provides confidentiality and integrity.
∎

######  Lemma 5.

SCC5G provides the users with authentication, authorization, and non-
repudiation services.

###### Proof.

SCC5G fulfills the ZT requirement by performing mutual authentication for each
communication instance. Hence, to prove providing authentication service, we
need to prove that mutual authentication works perfectly secure. The mutual
authentication process, according to Algorithm (2), has seven steps and six
data communication instances (as also shown in Fig. 2). The first and the
fourth communication instances are between the users, where they exchange
their certificates. The certificates are public information transferred
plainly, i.e. without encryption. The second and fifth communication instances
include verifiable public information. The rest of the communication instances
are encrypted communication from HAKF to the users. Hence, to prove the
security of the authentication and authorization process, we need to show that
no one else can forge the certificate. Each certificate binds four values
together, user ID, the user’s public key, the signed public key by the
corresponding private key, and the HRA verification part. Binding the user ID
and the unclonable tamper-resistant HRA to the key guarantees authorization,
in case of authentication success. The presence of the digital signature is
also guaranteeing non-repudiation. Hence, the authentication, authorization,
and non-repudiation security are now conditioned by the correctness of the
certificate. To forge a certificate, the attacker needs to find the hashed
value of the users’ HRA response to the challenge consisting of the
combination of the public key and user ID. Since the HRA is assumed to be
unclonable and tamper-resistant, no one other than the corresponding user can
extract the response. Hence, the certificate cannot be forged. However, if the
attacker becomes able to change the HAKF response to verify a forged
certificate, can perform a man-in-the-middle attack. While the communication
from HAKF is encrypted, we further show in Lemma (3), that SCC5G is resistant
to man-in-the-middle attacks. Hence SCC5G provides authentication,
authorization, and non-repudiation services. ∎

###  V-B Performance Evaluation

To evaluate the performance of the proposed architecture, we focus on the
efficiency and scalability of the proposed architecture. Since SCC5G adds a
mutual authentication to each communication instance, to verify its
scalability and efficiency, we need to investigate the latency and traffic
overhead it poses to each communication instance. High latency and/or
authentication traffic overhead lead to the infeasibility of the proposed
architecture in the real world. While those parameters could be easily
measured via a simple simulation scenario, the results of such an evaluation
might not be dependable. Hence, we designed a comprehensive simulation
scenario using ns-3 network simulator [13] to test the feasibility and to
evaluate the performance of SCC5G. We first describe the simulation scenario
and then show the results of the evaluation process.

Table (II) represents the simulation setting in brief. We simulated a 5G
network with a single base station and different numbers of user equipment
(UE) ranging from two to 40 in the increment steps of two. The network is a 5G
cellular network working on the mid-band, including two bands one for time
division duplexing (TDD) and the other for frequency division duplexing (FDD)
each with 100 MHz bandwidth centered on 2 and 2.2 GHz, respectively. The TDD
band includes one bandwidth part (BWP) specified for voice calls where the FDD
is divided into two BWPs one for video streaming (download) and the other for
gaming (upload). All users move according to a random waypoint (RWP) mobility
model in a squared area cell of $500\times 500$ meters. We selected these area
dimensions to encompass network evaluation under congested conditions. All
users generate all three types of traffic and one of them initiates an
authentication process after a warmup time. We ran each simulation instance
with different random generator seeds over 100 times and reported the average
of the results.

Figure 3: SCC5G mutual authentication traffic overhead (KB).  TABLE II:
Simulation Setting

Simulator version | ns-3 3.37 with 5G-LENA module
---|---
Number of UEs | $[2\quad 40]$
Area size | $500\times 500m$
Warmup time | 100 msec
Speed range | $[0\quad 20]m/s$
Mobility models | RWP
Traffic type | UDP
Voice flow type | GBR_CONV_VOICE
Video flow type | GBR_CONV_VIDEO
Gaming flow type | GBR_GAMING
UE antenna | $2\times 4$
Base station antenna | $4\times 8$
Antenna model | Isotropical
Base station antenna power | 4 dB
No. of component carriers(CC) | Two $CC_{0}$ and $CC_{1}$
$CC_{0}$ band type | TDD
$CC_{1}$ band type | FDD
No. of bandwidth parts | Three
$CC_{0}$ Central frequency | 2 GHz
$CC_{1}$ Central frequency | 2.2 GHz
CC Bandwidth | 100 MHz
Certificate packet size | 1 KB

Fig. (3) shows the average amount of the imposed authentication traffic of
SCC5G algorithm. This metric is directly related to the throughput of the
network, as the more congested network leads to more packet drops and
accordingly more retransmissions. As Fig. (3) shows, the increment in the
network traffic slightly increases the average traffic overhead. This
parameter starts with around 6 KB for the uncongested network which represents
the six transmissions of one KB each for a mutual authentication process. The
handshake process includes six packet transmissions where two of which are
user-to-user and the rest are between the users and the HAKF. In the most
congested network, this number is as low as 13 KB. Overall, this result shows
that SCC5G traffic is negligible in comparison with the data transmission of
such a congested network.

Fig. (4) shows the average time required for the mutual authentication process
of SCC5G starting from the authentication initialization of the source node
and ending with the authentication success at both the source and the
destination nodes. The reported times are in seconds starting at about 0.25
sec for the uncongested network and slightly less than 0.55 second for the
highly-congested network. While not represented here, we investigated the end-
to-end delay of a single small packet latency in such a network without any
extra traffic. It takes about 0.04 sec for such a packet to be transferred
from the source and received by the destination node. Recalling that the
mutual authentication process has six transmission instances, the overall
mutual authentication time is low for the less congested network. Considering
that each data transmission communication needs only one mutual authentication
process to take place, the reported authentication process time sounds
feasible, even for highly congested networks.

Figure 4: SCC5G mutual authentication latency (sec).

##  VI Conclusion

In this paper, we present SCC5G, a secure critical-mission communication
architecture over a 5G network. It includes a full architecture design with a
careful protocol design for PQC end-to-end encrypted communication considering
the ZT setting. The comprehensive security and performance evaluation show
that SCC5G is not only feasible to be implemented, efficient, and scalable but
also it is resistant to well-known security attacks such as man-in-the-middle
and ID spoofing attacks. The proposed architecture is based on an assumption
that the existing cellular network is cooperative with the critical-mission
management system, such that the network admins are able to add a repository
behind the IP network to the 5G home network. As a future direction of this
work, we aim at relaxing his assumption and distributing the repository among
the critical-mission users such that any $k$ out of $n$ of them can cooperate
to validate a certificate.

## References

  * [1] “5g for military communications,” _Procedia Computer Science_ , vol. 171, pp. 2665–2674, 2020, third International Conference on Computing and Network Communications (CoCoNet’19).
  * [2] Under Secretary of Defense for Research and Engineering USD(R&E). (2023) Advancing 5g communications for america’s warfighters. [Online]. Available: {https://www.cto.mil/5g/about}
  * [3] S. Lukman, Y. Y. Nazaruddin, E. Joelianto, and B. Ai, “Military 5g mobile networking as driver of uavs in detecting ram for stealth operation,” in _2019 IEEE ACDT_ , 2019, pp. 91–96.
  * [4] L. Bastos, G. Capela, A. Koprulu, and G. Elzinga, “Potential of 5g technologies for military application,” in _2021 International Conference on Military Communication and Information Systems (ICMCIS)_.
  * [5] P. Grønsund, A. Gonzalez, K. Mahmood, K. Nomeland, J. Pitter, A. Dimitriadis, T.-K. Berg, and S. Gelardi, “5g service and slice implementation for a military use case,” in _2020 IEEE International Conference on Communications Workshops (ICC Workshops)_.
  * [6] J. Liao and X. Ou, “5g military application scenarios and private network architectures,” in _2020 IEEE International Conference on Advances in Electrical Engineering and Computer Applications( AEECA)_.
  * [7] G. Elmasry and P. Corwin, “Operational views of vertical tactical 5g,” in _MILCOM 2021 - 2021 IEEE Military Communications Conference (MILCOM)_ , 2021, pp. 727–732.
  * [8] S. Rose, O. Borchert, S. Mitchell, and S. Connelly, “Zero trust architecture,” 2020. [Online]. Available: https://doi.org/10.6028/NIST
  * [9] Defense Information Systems Agency (DISA) and National Security Agency (NSA) Zero Trust Engineering Team, “Department of defense (dod) zero trust reference architecture,” 2022.
  * [10] N. P. quantum cryptography. (2021, May) %****␣ASILOMAR2023.bbl␣Line␣75␣****https://csrc.nist.gov/projects/post-quantum-cryptography.
  * [11] NSA Cybersecurity Advisory, “Announcing the commercial national security algorithm suite 2.0,” 2022.
  * [12] E. Barker, “Recommendation for key management,” 2020. [Online]. Available: https://doi.org/10.6028/NIST.SP.800-57pt1r5
  * [13] (2023) ns-3. [Online]. Available: https://www.nsnam.org/
  * [14] A. Shamsoshoara, A. Korenda, F. Afghah, and S. Zeadally, “A survey on physical unclonable function (puf)-based security solutions for internet of things,” _Computer Networks_ , vol. 183, p. 107593, 2020.
  * [15] A. Chen, “Comprehensive assessment of rram-based puf for hardware security applications,” in _2015 IEEE International Electron Devices Meeting (IEDM)_ , 2015, pp. 10.7.1–10.7.4.
  * [16] X. Ren, J. Cao, M. Ma, H. Li, and Y. Zhang, “A novel puf-based group authentication and data transmission scheme for nb-iot in 3gpp 5g networks,” _IEEE Internet of Things Journal_ , vol. 9, no. 5, 2022.
  * [17] T. Alladi, V. Venkatesh, V. Chamola, and N. Chaturvedi, “Drone-map: A novel authentication scheme for drone-assisted 5g networks,” in _IEEE INFOCOM 2021 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)_ , 2021, pp. 1–6.
  * [18] P. Mall, R. Amin, A. K. Das, M. T. Leung, and K.-K. R. Choo, “Puf-based authentication and key agreement protocols for iot, wsns, and smart grids: A comprehensive survey,” _IEEE Internet of Things Journal_ , vol. 9, no. 11, pp. 8205–8228, 2022.
  * [19] A. Korenda, F. Afghah, A. Razi, B. Cambou, and T. Begay, “Fuzzy key generator design using reram-based physically unclonable functions,” in _IEEE International Conference on Physical Assurance and Inspection of Electronics_ , 2021.
  * [20] A. Alipour, F. Afghah, D. Hely, V. Beroulle, G. Di-Natale, A. Korenda, and B. Cambou, “Helper data masking for physically unclonable function-based key generation algorithms,” _IEEE ACCESS_ , 2022.
  * [21] CableLabs. (2023) A comparative introduction to 4g and 5g authentication. [Online]. Available: https://www.cablelabs.com
  * [22] J. Bos, L. Ducas, E. Kiltz, T. Lepoint, V. Lyubashevsky, J. M. Schanck, P. Schwabe, G. Seiler, and D. Stehlé, “CRYSTALS-Kyber: a CCA-secure module-lattice-based KEM,” in _2018 IEEE European Symposium on Security and Privacy_. IEEE, 2018, pp. 353–367.
  * [23] CRYSTALS-Dilithium, “CRYSTALS-Dilithium Algorithm Specifications and Supporting Documentation. Part of the Round 3 Submission Package to NIST.” Apr. 2022. [Online]. Available: https://pq-crystals.org/dilithium/


#  The LoDoPaB-CT Dataset
A Benchmark Dataset for Low-Dose CT Reconstruction Methods

Johannes Leuschner<EMAIL_ADDRESS>University of Bremen, Center for
Industrial Mathematics  Maximilian Schmidt<EMAIL_ADDRESS>University of Bremen, Center for Industrial Mathematics  Daniel Otero Baguer
<EMAIL_ADDRESS>University of Bremen, Center for Industrial Mathematics
Peter Maaß<EMAIL_ADDRESS>University of Bremen, Center for Industrial
Mathematics

(October 2019)

###### Abstract

Deep Learning approaches for solving Inverse Problems in imaging have become
very effective and are demonstrated to be quite competitive in the field.
Comparing these approaches is a challenging task since they highly rely on the
data and the setup that is used for training. We provide a public dataset of
computed tomography images and simulated low-dose measurements suitable for
training this kind of methods. With the LoDoPaB-CT Dataset we aim to create a
benchmark that allows for a fair comparison. It contains over $40\,000$ scan
slices from around $800$ patients selected from the LIDC/IDRI Database. In
this paper we describe how we processed the original slices and how we
simulated the measurements. We also include first baseline results.

##  1 Introduction

Tomographic image reconstruction is an extensively studied field.
Traditionally, analytical methods, like filtered back-projection (FBP), or
iterative reconstruction (IR) techniques are used for this task. Recently,
image reconstruction approaches involving Deep Learning (DL) have been
developed and demonstrated to be very competitive [4, 9, 18, 22, 23, 31, 34,
38].

One popular imaging modality is Computed Tomography (CT). As high doses of
applied radiation are potentially harmful to the subjects, better
reconstruction methods for low-dose scans are highly desirable. Different
learned methods have been successfully applied to this task [4, 34]. However,
comparing these approaches is a challenging task since they highly rely on the
data and the setup that is used for training. The main goal of this work is to
provide a standard dataset that can be used to train and benchmark learned
low-dose CT reconstruction methods.

To this end we introduce the Low-Dose Parallel Beam (LoDoPaB)-CT, which uses
the public LIDC/IDRI Database [6] of human chest CT reconstructions. We
consider these (in the form of 2D images) to be the so called ground truths.
Paired samples, created by simulating low-dose CT measurements, constitute the
most complete training data and could be used for all kinds of learning. In
particular, methods trained only on prior samples, i.e. CT images, and methods
relying on samples of the joint distribution of CT images and the
measurements, can make use of (part of) the dataset. In total, the dataset
features more than $40\,000$ sample pairs.

The dataset is published on zenodo.org and accessible via DOI
10.5281/zenodo.3384092.111https://doi.org/10.5281/zenodo.3384092 It is subject
to the Open Data Commons Attribution License (ODC-By)
v1.0.222https://opendatacommons.org/licenses/by/1.0/ Each dataset part is
stored in multiple HDF5 files [33], which can be processed with simple tools
in many programming languages. Nevertheless, the python library
DIV$\alpha\ell$333https://github.com/jleuschn/dival. Library for testing and
comparing DL based methods for inverse problems. might be useful for accessing
the data easily.

##  2 Related Work

Machine learning based approaches benefit strongly from the availability of
comprehensive datasets. In the last years a wide variety of different CT data
has been published, covering different body parts and scan scenarios.

For the training of reconstruction models the projection data is crucial, but
is rarely provided. Recently, Low Dose CT Image and Projection Data (LDCT-and-
Projection-data) [26] was published by investigators from the Mayo Clinic,
which includes measured normal-dose projection data in the new open DICOM-CT-
PD format of $299$ patients. The AAPM Low Dose CT Grand Challenge data [25] is
another dataset that includes (simulated) sinograms, featuring $30$ different
patients. The Finish Inverse Problems Society (FIPS) provides complete
datasets of a walnut [15] and a lotus root [7] aimed at sparse data
tomography. Recently, Der Sarkissian et al. [13] published cone-beam CT
projection data and reconstructions of $42$ walnuts. Their dataset is directly
aimed at the training and comparison of machine learning methods. In magnetic
resonance imaging, fastMRI [39] with $1600$ scans of humans knees is another
prominent example.

Other CT datasets focus on the detection and segmentation of special
structures like lesions in the reconstructions for the development of
computer-aided diagnostic (CAD) methods. Therefore, they do not include the
sinograms. FUMPE [24] contains CT angiography images of $35$ subjects for the
detection of pulmonary embolisms. KiTS2019 [16] is build around the
segmentation of kidney tumor in CT images. The Japanese Society of Radiology
Technology (JSRT) database [32] and the National Lung Screening Trial (NLST)
in cooperation with the CT Image Library (CTIL) [10, 12] each contain scans of
the lung.

The LIDC/IDRI Database [6, 5] consist of $1018$ helical thoracic CT scans (cf.
section 4). It forms the basis for our LoDoPaB-CT Dataset. We simulate the
missing projection data to make the dataset usable for the training of
reconstruction algorithms. The whole process is described in section 5.

##  3 CT Imaging

Introduced in the 1970s, computed tomography has become one of the most
valuable technologies in medical imaging. A mathematical description of CT was
developed by Radon [29] over 50 years earlier. This section briefly introduces
the mathematical foundation and a model for the stochastic noise.

Figure 1: The Shepp-Logan phantom (left) and its corresponding sinogram
(right). $\varphi$$s$

X-ray source

detector

Figure 2: Parallel beam geometry

###  3.1 Mathematical Foundation

The task in CT belongs to the class of inverse problems

$\displaystyle\mathcal{A}x+\varepsilon=y^{\delta}.$ (1)

Given noisy measurements $y^{\delta}$, so called sinograms, the goal is to
calculate an accurate reconstruction $\tilde{x}$ of the interior distribution
$x$ of a body $\Omega$, with respect to some quality measure. The noise
$\varepsilon$ has a level of $\delta$.

For CT we use the 2D Radon transform as forward operator $\mathcal{A}$ to
model the attenuation of the X-ray when passing trough body $\Omega$ with
interior distribution $x$. We can parameterize the path of an X-ray beam by
the distance from the origin $s\in\mathbb{R}$ and angle $\varphi\in[0,\pi]$
(cf. Figure 2)

$\displaystyle
L_{s,\varphi}(t)=s\omega\left(\varphi\right)+t\omega^{\perp}\left(\varphi\right),\quad\omega\left(\varphi\right):=\begin{bmatrix}\cos(\varphi)\\\
\sin(\varphi)\end{bmatrix},\quad\omega^{\perp}\left(\varphi\right):=\begin{bmatrix}-\sin(\varphi)\\\
\phantom{-}\cos(\varphi)\end{bmatrix}.$

The Radon transform then calculates the integral along the line for parameters
$s$ and $\varphi$

$\displaystyle\mathcal{A}x(s,\varphi)=\int_{\mathbb{R}}x\left(L_{s,\varphi}(t)\right)\,\mathrm{d}t.$
(2)

According to Beer-Lambert’s law, the result is the logarithm of the ratio
between the intensity $I_{0}$ at the X-ray source and $I_{1}$ at the detector

$\displaystyle\mathcal{A}x(s,\varphi)=-\ln\left(\frac{I_{1}\left(s,\varphi\right)}{I_{0}\left(s,\varphi\right)}\right)=y\left(s,\varphi\right).$
(3)

Calculating the transform for all pairs $(s,\varphi)$ results in a sinogram.
An example for the famous Shepp-Logan phantom is shown in Figure 1.

To get a reconstruction $\tilde{x}$ from the sinogram, we have to invert the
forward model. Since the Radon transform is linear and compact, the inverse
problem is ill-posed in the sense of Nashed [27, 28]. In practice, we can only
measure for a finite number of distances $s$ and angles $\varphi$. In the
discrete case the problem is ill-conditioned.

###  3.2 Photon Statistics and Noise

Noise in CT can be classified into quantum noise and detector noise. The whole
process of photon generation, attenuation and detection can be modelled by a
Poisson distribution, while the detector noise is often assumed to be
Gaussian. The number of measured photons $N_{1}$ at the detector is therefore
given by

$\displaystyle N_{1}(s,\varphi)=n_{1}(s,\varphi)+\varepsilon,\quad
n_{1}(s,\varphi)\sim\operatorname{Poisson}\left(N_{0}\exp\left(-\mathcal{A}x(s,\varphi)\right)\right),\quad\varepsilon\sim\mathcal{N}\left(0,\sigma^{2}\right).$
(4)

For the expected number of photons follows

$\displaystyle\mathbb{E}\left[N_{1}(s,\varphi)\right]=\mathbb{E}\left[n_{1}(s,\varphi)\right]+\mathbb{E}\left[\varepsilon\right]=N_{0}\exp\left(-\mathcal{A}x(s,\varphi)\right),$
(5)

which is again Beer-Lambert’s law. Besides the stochastic fluctuations,
artifacts can occur due to material properties, physical effects or movement
of the patient during the scan (cf. Buzug [8]).

##  4 LIDC/IDRI Database and Data Selection

The Lung Image Database Consortium (LIDC) and Image Database Resource
Initiative (IDRI) published the LIDC/IDRI Database [6] to support the
development of CAD methods for the detection of lung nodules. The dataset
consists of $1018$ helical thoracic CT scans of $1010$ individuals. Seven
academic centers and eight medical imaging companies collaborated for the
creation of the database. As a result, the data is heterogeneous with respect
to the technical parameters and scanner models.

Figure 3: Scans from the LIDC/IDRI Database with poor quality, good quality
and an artifact. The shown HU window is $[-1024,1023]$.

Both standard-dose and lower-dose scans are part of the dataset. Tube peak
voltages range from $120\text{\,}\mathrm{k}\mathrm{V}$ to
$140\text{\,}\mathrm{k}\mathrm{V}$ and tube current from
$40\text{\,}\mathrm{m}\mathrm{A}$ to $627\text{\,}\mathrm{m}\mathrm{A}$ with a
mean of $222.1\text{\,}\mathrm{m}\mathrm{A}$. Labels for the lung nodules were
create by a group of $12$ radiologists in a two-phase process. The image
reconstruction was performed with different convolution kernels, depending on
the manufacturer of the scanner. Figure 3 shows examples of the provided
reconstructions. The LIDC/IDRI Database is freely available from The Cancer
Imaging Archive (TCIA) [11]. It is published under the Creative Commons
Attribution 3.0 Unported
License444https://creativecommons.org/licenses/by/3.0/.

The LoDoPaB-CT Dataset builds on the LIDC/IDRI scans. Our dataset is intended
for the evaluation of reconstruction methods in a low-dose setting. Therefore,
we simulate the projection data, which is not included in the LIDC/IDRI
dataset (cf. section 5). To guarantee a fair comparison with good ground
truths, patients whose scans are too noisy were removed in a manual selection
process. Additional scans were excluded due to their geometric properties. In
the end, $812$ patients remain in the LoDoPaB-CT Dataset.

The dataset is split into four parts: three parts for training, validation and
testing, respectively, and a “challenge” part that is kept private for the
time being.555If you are interested in participating in our challenge-like
comparison, please contact us via<EMAIL_ADDRESS>Each part contains
scans from a distinct set of patients as we want to study the case of learned
reconstructors being applied to patients that are not known from training. The
training set features scans from $632$ patients, while the other parts contain
scans from $60$ patients each. Every scan contains multiple slices (2D images)
for different $z$-positions, of which only a subset is included. The amount of
extracted slices depends on the slice thickness obtained from the
metadata.666As slices with small distances are similar, they may not provide
much additional information while increasing the chances to overfit. With this
reasoning we attempted to only include slices that have a minimum distance of
$7.5\text{\,}\mathrm{m}\mathrm{m}$ to each other, which we stated in the
previous version of this paper (arXiv:1910.01113v1). However, we have to
correct ourselves, since we mistakenly assumed the slice files to be ordered
by their $z$-positions when using the filename as sort key (and also assumed
the distances to be equal to the slice thickness). The actual distances of the
extracted slices are larger than $5.0\text{\,}\mathrm{m}\mathrm{m}$ for
$>$45\text{\,}\%$$ and larger than $2.5\text{\,}\mathrm{m}\mathrm{m}$ for
$>$75\text{\,}\%$$ of the slices. In total, the dataset contains $35\,820$
training images, $3522$ validation images, $3553$ test images and $3678$
challenge images.

######  Remark.

We propose to use our default dataset split, as it allows for fair comparison
with other methods that use the same split. However, users are free to remix
or resplit the dataset parts. For this purpose randomized patient IDs are
provided, i.e., the same random ID is given for all slices obtained from one
patient. Thus, when creating custom splits it can be regulated whether—and to
what extent—the same patients are contained in different parts.

##  5 Simulation Setup

For the simulated CT measurements the following setup was chosen:

  * •

parallel beam geometry (cf. Figure 2)

  * •

$1000$ projection angles

  * •

$513$ projection beams (lowest number meeting Nyquist-criterion)

  * •

Poisson-distributed noise matching a mean photon count of $4096$ per detector
pixel before attenuation.

The computation steps that were applied to the images from the LIDC/IDRI
Database are listed in algorithm 1 in more detail and will be discussed in the
next sections. For technical reference, the script used for simulation is
available online.777https://github.com/jleuschn/lodopab_tech_ref

###  5.1 Image preprocessing

First of all, each image is cropped to the central rectangle of
$$362\text{\,}\mathrm{p}\mathrm{x}$\times$362\text{\,}\mathrm{p}\mathrm{x}$$.
This is done because most of the images contain (approximately) circle-shaped
reconstructions with a diameter of $512\text{\,}\mathrm{p}\mathrm{x}$ (cf.
Figure 3). After the crop, the image only contains pixels that lie inside this
circle, which avoids value jumps occuring at the border of the circle. While
this yields natural ground truth images, we need to point out that the cropped
images in general do not show the full subject but some interior part. Hence,
it is unlikely for methods trained with this dataset to perform well on full-
subject measurements.

For some scan series, the circle is subject to a geometric transformation
either shrinking or expanding the circle in some directions. In particular,
for few scan series the circle is shrinked such that it is smaller than the
cropped rectangle. We exclude these series, i.e. those with patient IDs 0004,
0032, 0102, 0116, 0120, 0289, 0368, 0418, 0541, 0798, 0926, 0972 and 1000,
from our dataset, which allows to crop all included images consistently to
$$362\text{\,}\mathrm{p}\mathrm{x}$\times$362\text{\,}\mathrm{p}\mathrm{x}$$.

The integer Hounsfield unit (HU) values obtained from the DICOM files are
dequantized by adding uniform noise from the interval $[0,1)$. Next, the
linear attenuations $\mu$ are computed from the dequantized HU values using
the definition of the HU,

$\displaystyle\mathrm{HU}=1000\,\frac{\mu-\mu_{\mathrm{water}}}{\mu_{\mathrm{water}}-\mu_{\mathrm{air}}}\qquad\Leftrightarrow\qquad\mu=\mathrm{HU}\,\frac{\mu_{\mathrm{water}}-\mu_{\mathrm{air}}}{1000}+\mu_{\mathrm{water}},$
(6)

where we use the linear attenuation coefficients

$\displaystyle\mu_{\mathrm{water}}$
$\displaystyle=$20\text{\,}\mathrm{/}\mathrm{m}$,$ (7)
$\displaystyle\mu_{\mathrm{air}}$
$\displaystyle=$0.02\text{\,}\mathrm{/}\mathrm{m}$,$ (8)

which approximately correspond to an X-ray energy of
$60\text{\,}\mathrm{k}\mathrm{e}\mathrm{V}$ [17]. Finally, the $\mu$-values
are normalized into $[0,1]$ by dividing by

$\displaystyle\mu_{\mathrm{max}}=3071\,\frac{\mu_{\mathrm{water}}-\mu_{\mathrm{air}}}{1000}+\mu_{\mathrm{water}}=$81.358\,58\text{\,}\mathrm{/}\mathrm{m}$,$
(9)

which corresponds to the largest HU value that can be represented with the
standard 12-bit encoding, i.e.
$(2^{12}-1-1024)\,$\mathrm{H}\mathrm{U}$=$3071\text{\,}\mathrm{H}\mathrm{U}$$,
followed by clipping of all values into $[0,1]$,

$\displaystyle\hat{\mu}=\mathrm{clip}(\mu/\mu_{\mathrm{max}},[0,1])=\begin{cases}0&,\mu/\mu_{\mathrm{max}}\leq
0\\\ \mu/\mu_{\mathrm{max}}&,0<\mu/\mu_{\mathrm{max}}\leq 1\\\
1&,1<\mu/\mu_{\mathrm{max}}\end{cases}.$ (10)

The equations (6) and (10) are applied pixel-wise to the images.

Algorithm 1 Simulation

0:
$\mu_{\mathrm{air}}=$0.02$,\mu_{\mathrm{water}}=$20$,\mu_{\mathrm{max}}=$81.358\,58$$

0: Preprocessing:

1: image_HU = dcm.pixel_array[75:$-75$, 75:$-75$] $*$ dcm.RescaleSlope $+$
dcm.RescaleIntercept

2: image_HU += dequantization_noise $\sim\mathcal{U}(0,1)$

3: image_mu = image_HU $*$ ($\mu_{\mathrm{water}}$ $-$ $\mu_{\mathrm{air}}$)
$/$ 1000 $+$ $\mu_{\mathrm{water}}$

4: image = clip(image_mu $/$ $\mu_{\mathrm{max}}$, min=0, max=1)

4: Simulation:

5: image_upscaled = bilinear_interp(image_mu, 1000$\times$1000)

6: proj_data = ray_trafo(image_upscaled, rect=$[$-0.13$,$0.13$]^{2}$,
angles=1000, detector_pixels=513)

7: photons = exp($-$proj_data) $*$ $4096$

8: noisy_photons $\sim$ max($0.1$, Pois(photons))

9: measurement = $-$log(noisy_photons $/$ $4096$) $/$ $\mu_{\mathrm{max}}$

###  5.2 Simulation of CT measurements

To simulate the measurements based on the virtual ground truth images, the
main step is to apply the forward operator, which is the ray transform for CT.
For this task we utilize the Operator Discretization Library (ODL) [3] with
the ’astra_cpu’ backend [1].888We choose ’astra_cpu’ over the usually favored
’astra_cuda’ because of small inaccuracies observed in the sinograms when
using ’astra_cuda’, specifically at angles $0$, $\frac{\pi}{2}$ and $\pi$ and
detector positions $-1/\sqrt{2}\,\frac{l}{2}$ and $1/\sqrt{2}\,\frac{l}{2}$
with $l$ being the length of the detector. The used version is astra-
toolbox==1.8.3 on Python 3.6. The tested CUDA version is 9.1 combined with
cudatoolkit==8.0.

In order to avoid “committing the inverse crime” [37], which, in our scenario,
would be to use the same discrete model both for simulation and
reconstruction, we use a higher resolution for the simulation. Otherwise, a
good performance of reconstructors for the specific resolution of this dataset
($362\times 362$) could also stem from the properties of the specific
discretized problem, rather than from good inversion of the analytical model.
We use bilinear interpolation for the upscaling of the virtual ground truth
from $362\times 362$ to $1000\times 1000$.

The non-normalized, upscaled image is projected by the ray transform. By Beer-
Lambert’s law, the intensity quotient $I_{1}/I_{0}$ then is $\exp(-y)$, where
$y$ is the result of the projection. In order to compute the photon count per
detector pixel, this quotient is multiplied with the number of photons per
detector pixel before attenuation, which is chosen to be $4096$. This photon
count is used as the mean (and variance) of the Poisson distribution from
which the measured photon count is sampled. In order to avoid photon
starvation, photon counts that are zero are replaced by a photon count of
$0.1$. Hereby strictly positive values are ensured, which is a prerequisite
for the log-transform in the next step (see e.g. [35]). The log-transform of
the noisy intensity quotient is calculated and divided by $\mu_{\mathrm{max}}$
to match the normalized ground truth images.

######  Remark.

Although the linear model obtained by the log-transform is easier to study, in
some cases pre-log models are more accurate. See Fu et al. [14] for a detailed
comparison. For applying a pre-log method, the stored observation data
$\hat{y}$ must be back-transformed by $\exp(-\mu_{\mathrm{max}}\cdot\hat{y})$.
In this case the ground truth images also should be multiplied with
$\mu_{\mathrm{max}}$ to create physically consistent data pairs.

######  Remark.

Note that the minimum photon count of $0.1$ can be adapted subsequently. This
is most easily done by filtering out the highest observation values and
replacing them with $-\log(\epsilon/4096)/\mu_{\mathrm{max}}$, where
$\epsilon$ is the minimum photon count.

##  6 Evaluation Criteria

The baseline models in section 7 are evaluated against two standard image
quality metrics often used in CT applications [19, 2]. Since ground truth data
is provided in the data set, we restrict ourselves to so called full-reference
methods. Namely these are the peak signal-to-noise ratio and the structural
similarity by Wang et al. [36]. While the first metric calculates pixel-wise
intensity comparisons between ground truth and reconstruction, structural
similarity captures structural distortions.

###  6.1 Peak Signal-to-Noise Ratio

The peak signal-to-noise ratio (PSNR) expresses the ratio between the maximum
possible image intensity and the distorting noise

$\displaystyle\operatorname{PSNR}\left(\tilde{x},x\right)=10\log_{10}\left(\frac{\max\left(x\right)^{2}}{\operatorname{MSE}\left(\tilde{x},x\right)}\right).$
(11)

Here $x$ is the ground truth image and $\tilde{x}$ the reconstruction. Higher
PSNR values are an indication for a better reconstruction. Since the reference
value of $3071$ HU is far from the most common values, we choose
$\max\left(x\right)$ to be the difference between highest and lowest entry in
$x$ and not the maximum possible intensity. Otherwise, the results would often
be too optimistic.

###  6.2 Structural Similarity

Based on assumptions about the human visual perception, the structural
similarity (SSIM) compares the overall image structure of ground truth and
reconstruction. It is computed through a sliding window at $M$ locations

$\displaystyle\operatorname{SSIM}\left(\tilde{x},x\right)=\frac{1}{M}\sum_{j=1}^{M}\frac{\left(2\tilde{\mu}_{j}\mu_{j}+C_{1}\right)\left(2\Sigma_{j}+C_{2}\right)}{\left(\tilde{\mu}_{j}^{2}+\mu_{j}^{2}+C_{1}\right)\left(\tilde{\sigma}_{j}^{2}+\sigma_{j}^{2}+C_{2}\right)},$
(12)

where $\tilde{\mu}_{j}$ and $\mu_{j}$ are the average pixel intensities,
$\tilde{\sigma}_{j}$ and $\sigma_{j}$ the variances and $\Sigma_{j}$ the
covariance of $\tilde{x}$ and $x$ at the $j$-th local window. Constants
$C_{1}=(K_{1}L)^{2}$ and $C_{2}=(K_{2}L)^{2}$ stabilize the division.
Following Wang et al. [36] we choose $K_{1}=0.01$ and $K_{2}=0.03$. The window
size is $7\times 7$ and $L=\max(x)$.

##  7 Baseline Reconstructions

We provide reference reconstructions as well as quantitative results for the
standard filtered back-projection (FBP) and for a learned post-processing
method, which was trained using the proposed dataset. FBP is a widely used
analytical reconstruction technique (cf. Buzug [8] for an introduction). In
this case we used the Ram-Lak filter without frequency scaling.

If the measurements are noisy, FBP reconstructions tend to include streaking
artifacts. A typical approach to overcome this problem is to apply some kind
of post-processing such as denoising. Recent works [9, 18, 38] have
successfully used convolutional neural networks, such as the U-Net [30]. The
idea is to train a neural network to create clean reconstructions out of the
noisy FBP results.

In our implementation we used a U-Net-like architecture which is shown in
Figure 4 and is similar to the one used in [21]. There, the authors show that
such an architecture is a good image prior, i.e., its output is biased towards
natural images, for example, noise-free images. Training was performed by
minimizing the mean squared error loss with the Adam algorithm [20] for $20$
epochs with batch size $64$ and learning rate starting at $0.01$ which is
decayed by $0.1$ after every $5$ epochs. The model with the highest mean PSNR
on the validation set is selected from the models reached during training.

$z$$\varphi_{\theta}(z)$$\displaystyle 3\times 3$ Conv + Bn + LeakyRelu$\displaystyle 3\times 3$ Stride-Conv + Bn + LeakyReluUpsample + $\displaystyle 3\times 3$ Conv + Bn + LeakyRelu$\displaystyle 1\times 1$ conv + sigmoid$\displaystyle 1$$\displaystyle 16$$\displaystyle 32$$\displaystyle 32$$\displaystyle 64$$\displaystyle 64$$\displaystyle 64$$\displaystyle 64$$\displaystyle 128$$\displaystyle 64$$\displaystyle 64$$\displaystyle 128$$\displaystyle 64$$\displaystyle 64$$\displaystyle 32$$\displaystyle 32$$\displaystyle 1$$\displaystyle 16$$\displaystyle 1\times 1$ conv (skip connection) Figure 4: U-Net architecture. Numbers of channels are denoted below the layers. Skip connections have $4$ channels. Leaky ReLU activations with $a=$0.2$$ are used. | training set | validation set | test set
---|---|---|---
| PSNR ($\mathrm{d}\mathrm{B}$) | SSIM | PSNR ($\mathrm{d}\mathrm{B}$) | SSIM | PSNR ($\mathrm{d}\mathrm{B}$) | SSIM
FBP | $23.99$ | $0.408$ | $23.94$ | $0.407$ | $24.43$ | $0.426$
U-Net | $34.48$ | $0.838$ | $35.00$ | $0.858$ | $34.24$ | $0.820$
Table 1: Baseline performance. Values are the mean over all samples.

Figure 5: Baseline reconstructions. The window $[0,$0.45$]$ corresponds to a
HU range of $\approx[-1001,831]$.

In Table 1, the performance of the baseline methods on the training,
validation and test set is denoted. Figure 5 shows the reconstructions for the
first samples from the test set. While the post-processing leads to major
improvements in terms of the evaluated metrics, slight corruptions, such as
missing details, can be observed in some of the reconstructions. Thus, in
addition to the standard metrics, which are objective and easy to evaluate,
also the value of reconstructions for e.g. diagnosis by a medical professional
should be considered.

##  8 Conclusion

A large dataset, suitable for training deep learning approaches for low-dose
CT image reconstruction, was created and made publicly available via
zenodo.org. Human chest reconstructions from the LIDC/IDRI database were
processed and filtered in order to be used as ground truths. The corresponding
projection data was simulated in a low-intensity setting by applying Poisson
noise. We demonstrate its usability by means of two baseline reconstruction
methods, FBP (not learned) and post-processing (learned). These were applied
and evaluated using standard metrics. The results confirm the potential of
learned reconstruction methods over analytical ones, especially in imaging
applications such as CT. A “challenge” part of the data is kept private for
the time being in order to be used in a challenge-like comparison.

##  9 Acknowledgements

We thank Simon Arridge, Ozan Öktem and Carola-Bibiane Schönlieb for fruitful
discussion about the procedure, and Felix Lucka and Jonas Adler for their
ideas and helpful feedback on the simulation setup.

## References

  * [1] Wim Aarle et al.  “The ASTRA Toolbox: A platform for advanced algorithm development in electron tomography”  In _Ultramicroscopy_ 157, 2015, pp. 35–47  DOI: 10.1016/j.ultramic.2015.05.002
  * [2] J. Adler and O. Öktem  “Learned Primal-Dual Reconstruction”  In _IEEE Transactions on Medical Imaging_ 37.6, 2018, pp. 1322–1332  DOI: 10.1109/TMI.2018.2799231
  * [3] Jonas Adler, Holger Kohr and Ozan Öktem  “Operator Discretization Library (ODL)”, 2017  DOI: 10.5281/zenodo.249479
  * [4] Jonas Adler and Ozan Öktem  “Solving ill-posed inverse problems using iterative deep neural networks”  In _Inverse Problems_ 33.12 IOP Publishing, 2017, pp. 124007  DOI: 10.1088/1361-6420/aa9581
  * [5] Samuel G. Armato III et al.  “Data From LIDC-IDRI”, The Cancer Imaging Archive, 2015  DOI: 10.7937/K9/TCIA.2015.LO9QL9SX
  * [6] Samuel G. Armato III et al.  “The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A Completed Reference Database of Lung Nodules on CT Scans”  In _Medical Physics_ 38.2 John Wiley & Sons, Ltd, 2011, pp. 915–931  DOI: 10.1118/1.3528204
  * [7] Tatiana A. Bubba et al.  “Tomographic X-ray data of a lotus root filled with attenuating objects”, 2016  arXiv:1609.07299
  * [8] T.M. Buzug  “Computed Tomography: From Photon Statistics to Modern Cone-Beam CT”  Springer Berlin Heidelberg, 2008  DOI: 10.1007/978-3-540-39408-2
  * [9] H. Chen et al.  “Low-Dose CT With a Residual Encoder-Decoder Convolutional Neural Network”  In _IEEE Transactions on Medical Imaging_ 36.12, 2017, pp. 2524–2535  DOI: 10.1109/TMI.2017.2715284
  * [10] K.. Clark et al.  “Creation of a CT Image Library for the Lung Screening Study of the National Lung Screening Trial”  In _Journal of Digital Imaging_ 20.1, 2007, pp. 23–31  DOI: 10.1007/s10278-006-0589-5
  * [11] Kenneth Clark et al.  “The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository”  In _Journal of Digital Imaging_ 26.6, 2013, pp. 1045–1057  DOI: 10.1007/s10278-013-9622-7
  * [12] Dianna D. Cody et al.  “Normalized CT Dose Index of the CT Scanners Used in the National Lung Screening Trial”  In _American Journal of Roentgenology_ 194.6 American Roentgen Ray Society, 2010, pp. 1539–1546  DOI: 10.2214/AJR.09.3268
  * [13] Henri Der Sarkissian et al.  “A Cone-Beam X-Ray CT Data Collection designed for Machine Learning”, 2019  arXiv:1905.04787
  * [14] L. Fu et al.  “Comparison Between Pre-Log and Post-Log Statistical Models in Ultra-Low-Dose CT Reconstruction”  In _IEEE Transactions on Medical Imaging_ 36.3, 2017, pp. 707–720  DOI: 10.1109/TMI.2016.2627004
  * [15] Keijo Hämäläinen et al.  “Tomographic X-ray data of a walnut”, 2015  arXiv:1502.04064
  * [16] Nicholas Heller et al.  “The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomes”, 2019  arXiv:1904.00445
  * [17] J.H. Hubbell and S.M. Seltzer  “Tables of x-ray mass attenuation coefficients and mass energy-absorption coefficients 1 keV to 20 meV for elements z = 1 to 92 and 48 additional substances of dosimetric interest”, 1995  DOI: 10.18434/T4D01F
  * [18] K.. Jin et al.  “Deep Convolutional Neural Network for Inverse Problems in Imaging”  In _IEEE Transactions on Image Processing_ 26.9, 2017, pp. 4509–4522  DOI: 10.1109/TIP.2017.2713099
  * [19] Raoul M.. Joemai and Jacob Geleijns  “Assessment of structural similarity in CT using filtered backprojection and iterative reconstruction: a phantom study with 3D printed lung vessels” PMID: 28830200  In _The British Journal of Radiology_ 90.1079, 2017, pp. 20160519  DOI: 10.1259/bjr.20160519
  * [20] Diederik P. Kingma and Jimmy Ba  “Adam: A Method for Stochastic Optimization”  In _arXiv e-prints_ , 2014  arXiv:1412.6980
  * [21] V. Lempitsky, A. Vedaldi and D. Ulyanov  “Deep Image Prior”  In _2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 9446–9454  DOI: 10.1109/CVPR.2018.00984
  * [22] Housen Li et al.  “NETT: Solving Inverse Problems with Deep Neural Networks”, 2018  arXiv:1803.00092
  * [23] Sebastian Lunz, Ozan Öktem and Carola-Bibiane Schönlieb  “Adversarial Regularizers in Inverse Problems”  In _Advances in Neural Information Processing Systems 31_ Curran Associates, Inc., 2018, pp. 8507–8516  URL: https://papers.nips.cc/paper/8070-adversarial-regularizers-in-inverse-problems.pdf
  * [24] Mojtaba Masoudi et al.  “A new dataset of computed-tomography angiography images for computer-aided detection of pulmonary embolism” Data Descriptor  In _Scientific Data_ 5 The Author(s), 2018, pp. 180180 EP  DOI: 10.1038/sdata.2018.180
  * [25] C McCollough  “TU-FG-207A-04: Overview of the Low Dose CT Grand Challenge”  In _Medical Physics_ 43, 2016, pp. 3759–3760  DOI: 10.1118/1.4957556
  * [26] C.H. McCollough et al.  “Data from Low Dose CT Image and Projection Data [Data set]. The Cancer Imaging Archive”, 2020  DOI: 10.7937/9npb-2637
  * [27] M.Z. Nashed  “A new approach to classification and regularization of ill-posed operator equations”  In _Inverse and Ill-Posed Problems_ Academic Press, 1987, pp. 53–75  DOI: 10.1016/B978-0-12-239040-1.50009-0
  * [28] Frank Natterer  “The mathematics of computerized tomography” XVII, 222 S., [2] Bl. ; 23 cm : Ill., graph. Darst., Classics in applied mathematics ; 32  Philadelphia: Society for IndustrialApplied Mathematics, 2001  DOI: 10.1137/1.9780898719284
  * [29] J. Radon  “On the determination of functions from their integral values along certain manifolds”  In _IEEE Transactions on Medical Imaging_ 5.4, 1986, pp. 170–176  DOI: 10.1109/TMI.1986.4307775
  * [30] Olaf Ronneberger, Philipp Fischer and Thomas Brox  “U-Net: Convolutional Networks for Biomedical Image Segmentation”, 2015  arXiv:1505.04597
  * [31] Hongming Shan et al.  “Competitive performance of a modularized deep neural network compared to commercial algorithms for low-dose CT image reconstruction”  In _Nature Machine Intelligence_ 1.6, 2019, pp. 269–276  DOI: 10.1038/s42256-019-0057-9
  * [32] Junji Shiraishi et al.  “Development of a Digital Image Database for Chest Radiographs With and Without a Lung Nodule”  In _American Journal of Roentgenology_ 174.1 American Roentgen Ray Society, 2000, pp. 71–74  DOI: 10.2214/ajr.174.1.1740071
  * [33] The HDF Group  “Hierarchical Data Format, version 5” https://www.hdfgroup.org/HDF5/, 1997
  * [34] G. Wang et al.  “Image Reconstruction is a New Frontier of Machine Learning”  In _IEEE Transactions on Medical Imaging_ 37.6, 2018, pp. 1289–1296  DOI: 10.1109/TMI.2018.2833635
  * [35] G. Wang et al.  “Hybrid Pre-Log and Post-Log Image Reconstruction for Computed Tomography”  In _IEEE Transactions on Medical Imaging_ 36.12, 2017, pp. 2457–2465  DOI: 10.1109/TMI.2017.2751679
  * [36] Zhou Wang et al.  “Image quality assessment: from error visibility to structural similarity”  In _IEEE Transactions on Image Processing_ 13.4, 2004, pp. 600–612  DOI: 10.1109/TIP.2003.819861
  * [37] Armand Wirgin  “The inverse crime”, 2004  arXiv:math-ph/0401050
  * [38] Q. Yang et al.  “Low-Dose CT Image Denoising Using a Generative Adversarial Network With Wasserstein Distance and Perceptual Loss”  In _IEEE Transactions on Medical Imaging_ 37.6, 2018, pp. 1348–1357  DOI: 10.1109/TMI.2018.2827462
  * [39] Jure Zbontar et al.  “fastMRI: An Open Dataset and Benchmarks for Accelerated MRI”, 2018  arXiv:1811.08839