# Retraining-free Model Quantization via One-Shot Weight-Coupling Learning

Chen Tang1∗ Yuan Meng1∗ Jiacheng Jiang1 Shuzhao Xie1 Rongwei Lu1
Xinzhu Ma2,3 Zhi Wang1† Wenwu Zhu1†
1Tsinghua University 2The University of Sydney 3Shanghai AI Laboratory

###### Abstract

Quantization is of significance for compressing the over-parameterized deep
neural models and deploying them on resource-limited devices. Fixed-precision
quantization suffers from performance drop due to the limited numerical
representation ability. Conversely, mixed-precision quantization (MPQ) is
advocated to compress the model effectively by allocating heterogeneous bit-
width for layers. MPQ is typically organized into a searching-retraining two-
stage process. Previous works only focus on determining the optimal bit-width
configuration in the first stage efficiently, while ignoring the considerable
time costs in the second stage. However, retraining always consumes hundreds
of GPU-hours on the cutting-edge GPUs, thus hindering deployment efficiency
significantly. In this paper, we devise a one-shot training-searching paradigm
for mixed-precision model compression. Specifically, in the first stage, all
potential bit-width configurations are coupled and thus optimized
simultaneously within a set of shared weights. However, our observations
reveal a previously unseen and severe bit-width interference phenomenon among
highly coupled weights during optimization, leading to considerable
performance degradation under a high compression ratio. To tackle this
problem, we first design a bit-width scheduler to dynamically freeze the most
turbulent bit-width of layers during training, to ensure the rest bit-widths
converged properly. Then, taking inspiration from information theory, we
present an information distortion mitigation technique to align the behaviour
of the bad-performing bit-widths to the well-performing ones. In the second
stage, an inference-only greedy search scheme is devised to evaluate the
goodness of configurations without introducing any additional training costs.
Extensive experiments on three representative models and three datasets
demonstrate the effectiveness of the proposed method.

**footnotetext: Equal contributions$\dagger$$\dagger$footnotetext:
Corresponding authors

##  1 Introduction

Recent years have witnessed the tremendous achievements made by deep network-
driven applications, ranging from classification [17, 20, 37], object
detection [41, 36, 35], and segmentation [4, 18]. However, the huge number of
parameters in these deep models poses a challenge for deploying them. To
enable efficient deep learning, several techniques are proposed, including
pruning [29, 28], quantization [42, 11, 62], and neural architecture search
[19, 45].

Ultra-low bit-width neural network quantization [62, 11, 48] is an appealing
model compression technique to simplify the hardware complexity and improve
the runtime efficiency of over-parameterized AI models. However, under
severely limited numerical representation capabilities, performing such
compression across the whole neural network usually incurs an unacceptable
performance drop. Mixed-precision quantization (MPQ) [52, 14, 3, 21, 42], by
allocating unequally bit-width for weight and activation tensors of each
layer, can largely avoid accuracy degradation while maintaining the proper
model size and runtime overhead (_e.g_. on-device latency). The underlying
principle of MPQ is that layers contribute very differently to the final
accuracy [3, 52, 42], so the compression algorithm should apply heterogeneous
precision rather than a uniform one across the whole network. Besides,
hardware starts to support mixed-precision computation [52, 5] in these years,
which further pushes the study for mixed-precision compression.

Recently, MPQ has been extensively studied from several perspectives, _e.g_.
through reinforcement learning [52, 10], differentiable methods [55, 3], and
proxy-based approaches [8, 5]. They all try to solve one challenge, that says,
how to find the optimal bit-width configuration for each layer, in an
exponentially large $\mathcal{O}(n^{2L})$ space, where $n$ is the number of
bit-width candidates and $L$ is the number of layers in the deep network. To
this end, they organize a _searching-then-retraining pipeline_ , in which the
first stage aims to finish the bit-width allocation as fast as possible, and
naturally becomes the focus of the research. Nevertheless, previous works tend
to ignore the importance of the second stage, which in fact consumes
considerable time cost for retraining the model to fit the obtained bit-width
configurations (a.k.a, the policy). To recover the performance, LIMPQ [42]
needs about 200 GPU-hours to retrain a single ResNet18 policy. This impedes
the real-world quantized mixed-precision model deployment—we might not have
much time to retrain every policies for all devices.

Instead, we consider a new paradigm termed as _training-then-searching_ ,
repositioning the resource-intensive training process to the forefront of the
mixed-precision quantization pipeline. The initial stage focuses on the
development of a weight-sharing quantization model, where all possible bit-
width configurations are concurrently optimized within unified network of
weights to fulfill extensive search requirements. Importantly, this weight-
sharing model undergoes a singular training session and, notably, _requires no
subsequent retraining following the search_. Subsequently, in the second
stage, we present an inference-only search employing a bidirectional greedy
scheme to judiciously determine the optimal bit-width for each layer.

The primary focus of this paper lies in the training of a high-quality weight-
sharing quantization model, which highly relies on ingenious weight-coupling
learning method with heterogeneous bit-widths. We identify a distinctive
phenomenon inherent in weight-sharing quantization—referred to as the bit-
width interference problem. This problem arises from the highly shared weights
between bit-widths, the same weight could be quantized to very different
discrete values for various bit-widths, so significantly superimposed
quantization noise of various bit-widths leads to daunting optimization
challenges, as we will discuss later. We illustrate the bit-width interference
problem in Fig. 1, one can see that even the introduction of a single
additional bit-width can cause the shared weight to frequently traverse
quantization bound, resulting in training instability and large gradient
variance.

To understand and circumvent the issue of weight-sharing quantization, we
conduct a detailed analysis of the bit-width interference problem (Sec. 3.2).
Building upon this understanding, we introduce a bit-width scheduler designed
to freeze the bit-widths that contribute to weight interference, ensuring
proper convergence for the remaining bit-widths. Furthermore, during dynamic
training, we observe an information distortion phenomenon associated with the
unfrozen bit-widths. To mitigate this distortion, we propose to align the
behavior of poorly performing bit-widths with their well-performing
counterparts. Extensive experiments demonstrate that these two complementary
techniques not only unravel the intricacies of the bit-width interference
problem but also provide meaningful performance improvements of weight-sharing
quantization models. To summarize, our contributions are as follows:

  * •

We identify and analyze the bit-width interference problem in weight-sharing
quantization models, revealing its impact on optimization challenges, training
stability, and convergence.

  * •

To train the weight-sharing quantization model, we first design a novel bit-
width scheduler that freezes interfering bit-widths during training, ensuring
proper convergence and addressing instability caused by the introduction of
additional bit-widths.

  * •

We also propose a strategy inspired by information theory to align poorly
performing bit-widths with their well-performing counterparts, mitigating
information distortion during dynamic training and enhancing the overall
performance.

  * •

Extensive experiments on three representative models and three benchmarks
demonstrate the effectiveness of proposed method. For example, under an
average 4-bit constraint, our method leads on ResNet with a top accuracy of
71.0% at only 31.6G BitOPs and no retraining cost, compared to the second-best
at 70.8% accuracy with higher 33.7G operations and 90 epochs of retraining.

Figure 1: (a) 2D regression of _single_ 4bit quantization, (b) 2D regression
of 4bit quantization with an additional 2bit (_i.e.,_ weight-sharing
quantization), and (c) the L2-normalized gradients of these two regressions.
Compared with Fig. 1(a), the weight in Fig. 1(b) is more unstable due to the
bit-width interference. Notably, the gradient of 4bit also has a larger
variance under weight-sharing.  Figure 2: Distance between full-precision
weight and quantized weight on MobileNetV2 of a point-wise conv layer. Left: 4
bits. Right: 6 bits.

##  2 Related Work

###  2.1 Neural Network Quantization

In this paper, we only consider the context in quantization-aware training, as
it can achieve higher compression ratio than post-training quantization [30,
22]. Quantization can be generally categorized into two classes: fixed-
precision quantization and mixed-precision quantization.
Fixed-Precision Quantization. Fixed-precision quantization involves assigning
a uniform bit-width to all layers. Specifically, methods such as Dorefa [62]
and PACT [6] employ a low-precision representation for weights and activations
during forward propagation. They leverage the Straight-Through Estimation
(STE) technique [1] to estimate the gradient of the piece-wise quantization
function for backward propagation. LSQ [11] scales the weight and activation
distributions by introducing learnable step-size scale factors for
quantization functions.
Mixed-Precision Quantization. Mixed-Precision Quantization (MPQ) delves into
the intricate aspects of low-precision quantization by recognizing the
inherent variability in redundancy across different layers of the deep model.
By allocating smaller bit-widths to layers with high redundancy, MPQ optimizes
model complexity without causing a significant performance decline. The
challenge, however, lies in determining the most suitable bit-width for each
layer, considering that the bit-width selection is a discrete process, and the
potential combinations of bit-width and layer (referred to as “policy”) grow
exponentially.

Strategies such as HAQ [52] and ReleQ [10] leverage reinforcement learning
(RL) techniques to derive a bit-width allocator. SPOS [13], EdMIPS [3], BP-NAS
[60], GMPQ [55] and SEAM [44] adopt differential neural architecture search
(NAS) methods to learn the bit-width. However, these methods require to both
search-from-scratch and train-from-scratch for the models when changing the
search constraints. Unlike learning the optimal MPQ policy, HAWQ [8, 9] and
MPQCO [5] use the Hessian information as the quantization sensitivity metrics
to assist bit-width assignment. LIMPQ [42] proposes to learn the layer-wise
importance within a single quantization-aware training cycle.

###  2.2 Deep Learning with Weight-Sharing

Weight-sharing is an effective and practical technique to reuse weight to deal
with joint task learning or avoid the need to store multiple copies of
weights. Generally speaking, there have been two relevant topics to weight-
sharing with this work, covering neural architecture search (NAS) and dynamic
neural network.
Neural Architecture Search. NAS [19, 40, 64] aims to automatically discover
the well-performing topology of deep neural network in a vast search space,
which composes of various operators (_e.g_. convolutional layers with
different kernel-size or channels). To improve the search efficiency, recent
works [34, 13, 45] both adopt the idea of weight-sharing to stuff the
candidates into a shared and large meta-topology (a.k.a. the super-network).
Weight-sharing allows to train directly the meta-topology and derive a sub-
topology from the meta-topology to evaluate. This significantly shortens the
evaluation time of the goodness for a given topology [27, 34]. Although
certain MPQ research [3, 55] leverages this NAS-style searching process, they
still pay significant time for retraining.

Dynamic Neural Network. Unlike conventional neural networks which are
architecturally fixed during inference, dynamic neural networks [16] enable
dynamic computational paths on demand according to various input samples or
running environments. For example, runtime channel pruning [26, 54]
dynamically activates channels of layers and layer skipping [53, 51, 38]
adjusts depths of layers for different input images. To support the adaptive
inference, the weight-sharing idea is applied to avoid substantial copies of
weights. Therefore, they can both achieve a better accuracy-efficiency trade-
off compared to their static counterparts. These have also been several works
of dynamic bit-width neural networks [23, 43, 2, 57]. However, they either
provide only the fixed-precision quantization that supports very limited bit-
width configurations or have to drop the ultra-low bit-widths (_e.g_. 2 bits,
3bits, _etc_.) to guarantee the training convergence but lose the chance for
achieving high compression ratio.

##  3 Methodology

###  3.1 Preliminary

Quantization. The uniform quantization function under $b$ bits in
quantization-aware training (QAT) maps the input full-precision activations
and weights to the homologous quantized values $[0,2^{b}-1]$ and
$[-2^{b-1},2^{b-1}-1]$. The quantization functions $Q_{b}(\cdot)$ that
quantize the input values $x$ to quantized values $\hat{x}$ can be expressed
as follows:

$\hat{x}=Q_{b}(x;\gamma)=\lfloor\text{\rm clip}(\frac{x}{\gamma},{\rm
N_{min}},{\rm N_{max}})\rceil\times\gamma_{b},\frac{\partial\lfloor
x\rceil}{\partial x}\triangleq 1,$ (1)

where $\lfloor\cdot\rceil$ is the rounding-to-nearest function, and $\gamma$
is the scale factor. The Straight-Through Estimation (STE) is used to pass the
gradients for back-propagation. ${\rm N_{min}}$ and ${\rm N_{max}}$ are the
lower bound and upper bound of quantization [11, 62]. For ${\rm ReLU}$
activations, ${\rm N_{min}}=0$ and ${\rm N_{max}}=2^{b}-1$. For weights, ${\rm
N_{min}}=-2^{b-1}$ and ${\rm N_{max}}=2^{b-1}-1$. $\gamma_{b}$ is the
learnable scalar parameter used to adjust the quantization mappings, called
the _step-size scale factor_. For a network, each layer has two distinct scale
factors in the weights and activations quantizer.
Weight-Sharing for Mixed-Precision Quantization. We consider a model
$f(\cdot)=f_{\rm\textbf{W}_{L-1}}\circ...\circ f_{\rm\textbf{W}_{0}}(\cdot)$
with $L$ layers, and each layer has $N$ bit-width choices. Let
${\rm\textbf{W}:=\\{{\rm\textbf{W}}_{l}\\}}_{l=0}^{L-1}$ be the set of
flattened weight tensors of these $L$ layers. Therefore, the corresponding
search space $\mathcal{A}$ with $N^{2L}$ mixed-precision quantization policies
$\\{(b^{(w)}_{l},b^{(a))}_{l}\\}_{l=0}^{L-1}$ share the same latent weights W.
To track the time-prohibitive training costs of traversing the whole search
space, Monte-Carlo sampling is used to approximate the expectation term [43,
61, 42]. The overall optimization objective is formulated as follows

$\displaystyle\mathop{\arg\min}_{\rm\textbf{W}}\quad\mathbb{E}_{\mathcal{S}}\left[\mathcal{L}(f(\textbf{x};\mathcal{S},w^{(\mathcal{S})}),\textbf{y})\right]$
(2)
$\displaystyle\approx\mathop{\arg\min}_{\rm\textbf{W}}\frac{1}{K}\sum_{\mathcal{S}_{k}\thicksim\text{\rm
U}(\mathcal{A})}^{K}\left[\mathcal{L}(f(\textbf{x};\mathcal{S}_{k},\hat{\rm\textbf{W}}^{(\mathcal{S}_{k})}),\textbf{y})\right],$

where $\hat{\rm\textbf{W}}^{(\mathcal{S}_{k})}$ is the quantized weights of
sampled policy $S$ derived from the latent weights W. This weight-sharing of
layer $l$ is achieved by

$\hat{\rm\textbf{W}}^{(\mathcal{S}_{k})}:=\\{\hat{{\rm\textbf{W}}}^{(\mathcal{S}_{k})}_{l}\\}_{l=0}^{L-1},\quad\text{where}\quad\hat{\rm\textbf{W}}_{l}^{(\mathcal{S}_{k})}=Q_{b^{(k)}_{l,{\rm
w}}\in\mathcal{S}_{k}}({\rm\textbf{W}}_{l};\gamma)$ (3)

according to Eq. 1, where $b^{(k)}_{l,{\rm w}}\in\mathcal{S}_{k}$ is the bit-
width of weight of $l$-th layer in the policy $\mathcal{S}_{k}$.

Table 1: Accuracy of the weight-sharing quantization with/without low bit-width.  |  Top-1 Acc. (%) _w/_ 2bits (decreased sampling prob) | Top-1 Acc. (%) _w/o_ 2bits
---|---|---
6 bit | 69.2 | 70.4
4 bit | 68.1 | 69.1

###  3.2 Interference in Highly Coupled Weight-sharing

While training is possible, there still many challenges exist in weight-
sharing in practice. For example, ABN [43] observes a large accuracy gap
between the lower bit-widths and higher bit-widths. These works simply bypass
this problem and remove the ultra-low bit-width until the training becomes
stable, however, doing so does not achieve high compression ratios.

Here, we discuss the bit-width interference caused by highly coupled latent
weights W. Suppose we have $K=2$ sampled policies {$\mathcal{S}_{0}$,
$\mathcal{S}_{1}$} at training step $t$ in Eq. 2, and the bit-width of weight
is sampled differently, namely
$\hat{\rm\textbf{W}}^{(\mathcal{S}_{0})}\neq\hat{\rm\textbf{W}}^{(\mathcal{S}_{1})}$.
$\forall b^{(0)}_{l,{\rm w}}\in\mathcal{S}_{0},\forall b^{(1)}_{l,{\rm
w}}\in\mathcal{S}_{1}:b^{(0)}_{l,{\rm w}}<b^{(1)}_{l,{\rm w}}$.

######  Assumption 3.1 (Non-uniform Bit-width Convergence).

Quantized weights
$\hat{\rm\textbf{W}}_{l}^{(\mathcal{S}_{1})}=Q_{b^{(1)}_{l,{\rm
w}}\in\mathcal{S}_{1}}({\rm\textbf{W}}_{l};\gamma)$ of bit-width $b_{k}$ at
step $t$ is nearly converged while the
$\hat{\rm\textbf{W}}_{l}^{(\mathcal{S}_{0})}=Q_{b^{(0)}_{l,{\rm
w}}\in\mathcal{S}_{0}}({\rm\textbf{W}}_{l};\gamma)$ is not converged properly.
The smaller and not fully converged bit-width in $\mathcal{S}_{0}$ will pose
negative impact to the larger but well converged bit-width in
$\mathcal{S}_{1}$.

The situation in Assump. 3.1 is observed in weight-sharing network when the
learning capacity gap between sub-networks is large enough [61, 43]. To
further analyze the impact of $\mathcal{S}_{0}$, we can approximates the loss
perturbation with the second-order Taylor expansion:

$\displaystyle\Delta\mathcal{L}$
$\displaystyle=\sum_{n=1}^{N}\ell(f(\hat{\rm\textbf{W}}^{(\mathcal{S}_{1})}+\Delta{\rm\textbf{W}},\text{x}^{n}),\text{y}^{n})-\sum_{n=1}^{N}\ell(f(\hat{\rm\textbf{W}}^{(\mathcal{S}_{1})},\text{x}^{n}),\text{y}^{n})$
(4)
$\displaystyle\approx\nabla_{\hat{\rm\textbf{W}}^{(\mathcal{S}_{1})}}\ell(\hat{\rm\textbf{W}}^{(\mathcal{S}_{1})})\Delta{\rm\textbf{W}}+\Delta{\rm\textbf{W}}^{\mathsf{T}}\nabla_{\hat{\rm\textbf{W}}^{(\mathcal{S}_{1})}}^{2}\ell(\hat{\rm\textbf{W}}^{(\mathcal{S}_{1})})\Delta{\rm\textbf{W}},$

where $\ell(\cdot)$ is the loss function,
$\Delta{\rm\textbf{W}}:=\\{{\Delta}{\rm\textbf{W}}^{(\mathcal{S}_{1})}_{l}\\}_{l=0}^{L-1}$
is the quantization noise introduced by policy $\mathcal{S}_{1}$ to policy
$\mathcal{S}_{0}$ on each layer. It is noteworthy that the lower the bit-
width, the larger the quantization noise introduced [63, 23, 21], caused by
the large rounding and clipping error under very limited available discrete
values, _e.g_. quantization error for 2 bits is roughly 5$\times$ for 3 bits.
Therefore, putting small bit-width (_e.g_. 2bit) into the weight-sharing will
lead to large loss perturbation $\Delta\mathcal{L}$ and disturb the overall
performance eventually (see Tab. 1). Accordingly, one can draw such
conclusions in Eq. 4: (i) removing the low bit-width is the simplest way to
erase the effects of quantization noise but loses the chance to compress the
model with high compression ratio and (ii) one can track the loss perturbation
through $\Delta{\rm\textbf{W}}$ to iteratively freeze the most unstable bit-
width, which motivates our method described later in Sec. 3.3.

To illustrate the bit-width interference in weight-sharing quantization, we
use a 2D regression quantization experiment depicted in Fig. 1. Our
optimization objective minimizes the empirical risk [31, 7]:

$\mathop{\arg\min}_{w}\mathbbm{E}_{\textbf{x}\sim\mathcal{N}(0,1)}\Big{[}\|\textbf{x}w^{*}-\textbf{x}Q_{b}(w,\gamma)\|_{2}^{2}\Big{]},$
(5)

where $w^{*}$ represents the target weight and x is sampled from a normal
distribution. In Fig. 1(a), the single-bit optimization showcases relatively
stable quantized weights, occasionally crossing the quantization boundary due
to gradient approximation of STE [1]. Comparatively, weight-sharing
quantization exhibits more instability and the _weight moves closer to the
quantization bound more frequently_ (Fig. 1(b)), even with higher variance in
gradients as in Fig. 1(c).

This interference extends beyond toy regression to modern neural networks,
shown in Fig. 2 and Tab. 1. Furthermore, Fig. 2 demonstrates the distance
between quantized weights of 6 and 4 bits in one training epoch. Removing the
smallest bit-width (2 bits) notably stabilizes the higher bit-width curve.
However, introducing extra small bit-widths induces significant random
oscillations, signifying heightened model training instability.

###  3.3 Dynamic Bit-width Schedule

Eq. 4 reveals the decomposition of overall quantization noise
$\Delta{\rm\textbf{W}}$ into layer-specific perturbation components, offering
a metric to identify unstable layers. Therefore, _dynamically freezing the
bit-width causing weight interference ensures proper convergence for remaining
bit-widths during training_. However, direct use of Eq. 4 poses computational
challenges, particularly in calculating the Hessian and quantization noise
terms, prompting us to devise an alternative method.

We approximate layer perturbations by focusing on rounding errors due to their
significant impact on overall performance [15, 31]. Rounding errors portray
the distance between full-precision weights and their discrete quantization
levels, and reach maximums when at the midpoint between two quantization
levels (i.e., the _quantization bound_ in Fig. 1) because the possible
quantization levels change. In other words, the closer to the quantization
bounds, the more unstable the weights are, and therefore the unstable weights
are more vulnerable to the weight-sharing. Therefore, tracking the round
errors provide effective proxies for constructing our bit-width scheduler. For
clarity, we first definite the Bit-width Representation Set (BRS) as follows:

######  Definition 3.1 (Bit-width Representation Set).

For bit-width $b$ under uniform weight quantization, the bit-width
representation set $\Phi_{b}:=\gamma\times\\{-2^{b-1},...,0,...,2^{b-1}-1\\}$,
representing $2^{b}$ decomposed values of discrete quantization levels
according to Eq. 1.

The midpoints between two adjacent elements in a BRS are quantization bounds,
where they have a uniform distance $\gamma$. Given a pre-defined weight bit-
width candidates $B^{(w)}$, we can accumulate bit-specific unstable weights
for BRS of each bit-width of each layer’s shared weights
${\rm\textbf{W}}_{l}$. Therefore, we calculate the unstable weight criterion
$\hat{\Delta}\rm\textbf{W}^{\text{unstable}}$ by

$\displaystyle\hat{\Delta}{\rm\textbf{W}^{\text{unstable}}}$
$\displaystyle\triangleq\\{\hat{\Delta}{\rm\textbf{W}^{\text{unstable}}_{l}}\\}_{l=0}^{L-1},\text{where}$
(6) $\displaystyle\hat{\Delta}{\rm\textbf{W}}_{l}^{\text{unstable}}$
$\displaystyle=\sum_{b\in
B^{(w)}}\quad\frac{1}{2^{b-1}}\frac{1}{\|{\rm\textbf{W}}_{l}\|_{0}}\cdot$
$\displaystyle\sum_{q_{b}\in\Phi_{b}}\sum_{{\rm\textbf{w}}_{l,*}\in{\rm\textbf{W}}_{l}}\mathbbm{1}_{\mid{\rm\textbf{w}}_{l,*}\mid\leq\gamma\times\left(\frac{1-\epsilon}{2}+\frac{q_{b}}{\gamma}\right)},$

where $\epsilon\in[0,1]$ is to control the range of weights we care about.
After that, we choose the frozen layer set $\Omega$ with a Top-K selector from
the weight criterion,

$\large\Omega\leftarrow\texttt{{TopKToFreeze}}(\hat{\Delta}{\rm\textbf{W}^{\text{unstable}}};\text{K}),$
(7)

and the smallest bit-width of selected layers in $\Omega$ will be temporarily
frozen periodically. In practice, we use a cosine scheduler to gradually
decrease the value of K to guarantee that more unstable low bits will be
frozen early to improve the convergence of more high bits.

Figure 3: Output density at 2bit and 6bits. Small bit-width shows noteworthy
information distortion.

###  3.4 Optimization during Dynamic Training

Information Distortion Mitigation. While freezing the bit-width of layers, we
observe the outputs of the remaining small bit-widths of layers still exhibit
a information distortion compared to their high precision counterparts, as
shown in Fig. 3. Inspired by the information bottleneck principle [46, 58,
47], we expect if the smallest bit-width is sampled of a layer $l$, its
outputs ${\rm\textbf{O}}_{l}^{S}$ can preserve the information of its large
counterparts ${\rm\textbf{O}}_{l}^{H}$. However, directly optimizing this
mutual information term $I({\rm\textbf{O}}_{l}^{S};{\rm\textbf{O}}_{l}^{H})$
is infeasible, so we consider a feature alignment loss function to optimize
their rectified Euclidean distance as follows:

$\displaystyle\mathbb{E}\Big{[}\|$
$\displaystyle\max\\{Q,{\small\frac{{\rm\textbf{O}}^{S}-\mu({{\rm\textbf{O}}}^{S})}{\sqrt{\sigma({{\rm\textbf{O}}}^{S})+\epsilon}}}\eta_{{\rm\textbf{O}}^{S}}+\xi_{{\rm\textbf{O}}^{S}}\\}-$
(8)
$\displaystyle\max\\{Q,{\small\frac{{\rm\textbf{O}}^{H}-\mu({{\rm\textbf{O}}}^{H})}{\sqrt{\sigma({{\rm\textbf{O}}}^{H})+\epsilon}}}\eta_{{\rm\textbf{O}}^{H}}+\xi_{{\rm\textbf{O}}^{H}}\\}\|\Big{]},$

where $\epsilon$ is a small constant to avoid Divide-by-Zero errors, $\eta$
and $\xi$ are the learnable parameters for adapting the features, $\mu(\cdot)$
and $\sigma(\cdot)$ return the channel-wise mean and variance of input. Eq. 8
not only scales the features for better optimization but uses a $\max$
operator to avoid needless activations. See Fig. 4 for visualization with
proposed Information Distortion Mitigation technique.
Fairness Weight Regularization. Low-bit weights are actually subsets of high-
bit weights, when a layer is sampled with different bit-width, low-bit weights
will receive additional updates from high-bit weights. In other words, low-bit
weights are subjected to very aggressive weight regularization, which
exacerbates their underfitting issues [61]. To ensure regularization fairness,
we disable weight regularization for small-bit weights during training.

###  3.5 Bidirectional Greedy Search

To find the optimal quantization policy $\mathcal{S^{*}}$, the existing MPQ
methods can be formulated to a bi-level optimization problem [42]. In this
paper, our well-trained weight-sharing model can serve as a good performance
indicator to perform inference-only searching [61, 50]. This simplified
procedure motivates us to devise a bidirectional greedy search scheme to
determine the per-layer bit-width efficiently.

Consider a mixed-precision quantization policy, $\mathcal{S}^{(t)}$,
implemented at step $t$, with $L$ being the total number of layers. To evolve
this policy to $\mathcal{S}^{(t+1)}$, rather than concurrently adjusting the
bit-width of most layers (e.g., employing reinforcement learning), a step-by-
step approach is taken. Specifically, the bit-width of a single layer is
adjusted at a time, either increasing or decreasing by a single bit-width to
create a provisional policy, $\mathcal{S}^{(t)}_{i}$, where
$i\in\\{0,...,2L-1\\}$. This method yields a search space of complexity
$\mathcal{O}(2L)$ for each iteration. During each iteration, the permanent
policy $\mathcal{S}^{(t+1)}$ is chosen in a greedy manner between these $2L$
policies, considering the trade-off between accuracy and efficiency, denoted
as
$J_{i}=\bar{\mathcal{L}}_{val}(\hat{\rm\textbf{W}}^{(\mathcal{S}_{i}^{(t)})})+\lambda*\accentset{{\cc@style\underline{\mskip
18.0mu}}}{\texttt{{BitOps}}}(\mathcal{S}_{i}^{(t)})$ for each layer:

$\displaystyle\mathcal{S}^{(t+1)}$
$\displaystyle\leftarrow\mathop{\arg\min}_{i}\left[\Theta\right],$ (9)
$\displaystyle\Theta\triangleq\\{J_{i}|J_{i}=\bar{\mathcal{L}}_{val}$
$\displaystyle(\hat{\rm\textbf{W}}^{(\mathcal{S}_{i}^{(t)})})+\lambda*\accentset{{\cc@style\underline{\mskip
18.0mu}}}{\texttt{{BitOps}}}(\mathcal{S}_{i}^{(t)})\\}_{i=0}^{2L-1},$

where $\bar{\mathcal{L}}$ and $\accentset{{\cc@style\underline{\mskip
18.0mu}}}{\texttt{{BitOps}}}$ are the min-max normalization loss and BitOPs to
ensure their values fall into the interval $[0,1]$, and $\lambda$ is the
hyper-parameter to control the trade-off, respectively. By this means, the
solution $\mathcal{S^{*}}$ is reached when the BitOPs is satisfied at final
step $T$, i.e., $\mathcal{S}^{*}\leftarrow\mathcal{S}^{(T)},\text{if}\
\texttt{{BitOps}}(\mathcal{S}^{(T)})\leq C$.

Table 2: Accuracy and efficiency results for ResNet. “Top-1 Acc.” represents the Top-1 accuracy of the quantized model and full-precision model. “MP” means mixed-precision quantization. “Retrain Cost” denotes the epochs required to retrain the MPQ policy. “*”: reproduces through the vanilla ResNet architecture [17]. The best results are bolded in each metric, the second best results are underlined.  Method |  Bit-width (W/A) |  Top-1 Acc. (%) $\uparrow$ |  BitOPs (G) $\downarrow$ |  Retrain Cost (Epoch) $\downarrow$
---|---|---|---|---
Baseline | 32 / 32 | 70.5 | - | -
PACT [6] | 3 / 3 | 68.1 | 23.0 | -
LSQ∗ [11] | 3 / 3 | 69.4 | 23.0 | 90
EWGS [25] | 3 / 3 | 69.7 | 23.0 | 100
EdMIPS [11] | 3MP / 3MP | 68.2 | - | 40
GMPQ∗ [55] | 3MP / 3MP | 68.6 | 22.8 | 40
DNAS [56] | 3MP / 3MP | 68.7 | 24.3 | 120
FracBits [59] | 3MP / 3MP | 69.4 | 22.9 | 150
LIMPQ [42] | 3MP / 3MP | 69.7 | 23.0 | 90
SEAM [44] | 3MP / 3MP | 70.0 | 23.0 | 90
Ours | 2MP / 3MP | 67.7 | 17.3 | 0
Ours | 3MP / 3MP | 70.2 | 23.3 | 0
PACT [6] | 4 / 4 | 69.2 | 35.0 | -
LSQ∗ [11] | 4 / 4 | 70.5 | 35.0 | 90
EWGS [25] | 4 / 4 | 70.6 | 35.0 | 100
MPQCO [5] | 4MP / 4MP | 69.8 | - | 30
DNAS [12] | 4MP / 4MP | 70.6 | 35.1 | 120
FracBits [59] | 4MP / 4MP | 70.6 | 34.7 | 150
LIMPQ [42] | 4MP / 4MP | 70.8 | 35.0 | 90
SEAM [44] | 4MP / 4MP | 70.8 | 33.7 | 90
Ours | 4MP / 4MP | 71.0 | 31.6 | 0

##  4 Experiments

###  4.1 Experimental Setups

Dataset. We conduct experiments on three benchmarks, _i.e_. classification on
ImageNet, CIFAR-100, and Pets, for classification task. Specifically, we
pretrain the model on ImageNet and verify the transfer learning performance on
CIFAR and Pets. For ImageNet classification, we use the basic data
augmentation methods during training, _i.e_. the input images are randomly
cropped to 224$\times$224 with the horizontal flip.
Model. We conduct the experiments on three representative lightweight models
including the ResNet18 [17], MobileNetV2 [37], and EfficientNetLite-B0 [40].
Hyper-parameters. The bit-width candidates are set to {2, 3, 4, 5, 6} for both
weight and activation quantization. Following the previous practices [52, 11,
42], the first and last layers are fixed to 8 bits. We set $\lambda=1.5$ for
searching.

For training, we adopt the SGD optimizer with cosine learning rate scheduler,
the initial learning rate is set to $0.04$ and the first 5 epochs are used as
warm-up. The hyper-parameter $\epsilon$ in Eq. 6 is set to $0.25$. The weight-
decay is set to $2.5\times 10^{-5}$ [11, 42].

###  4.2 ImageNet Classification

Table 3: Accuracy and efficiency results for MobileNetV2. †: QBitOPT uses channel-wise quantization to retain performance.  Method |  Bit-width (W/A) |  Top-1 Acc. (%) $\uparrow$ |  BitOPs (G) $\downarrow$ |  Retrain Cost (Epoch) $\downarrow$
---|---|---|---|---
Baseline | 32 / 32 | 72.6 | - | -
LSQ [11] | 3 / 3 | 65.2 | 3.4 | 90
QBR [15] | 3 / 3 | 67.4 | 3.4 | 90
HMQ [14] | 2MP / 4MP | 64.5 | - | 50
QBitOPT [33] | 3MP / 3MP | 65.7 | - | 30
NIPQ [39] | 3MP / 3MP | 62.3 | - | 43
Ours | 3MP / 3MP | 67.8 | 3.6 | 0
LSQ [11] | 4 / 4 | 69.5 | 5.4 | 90
EWGS [25] | 4 / 4 | 70.3 | 5.4 | 100
AdaBits [23] | 4 / 4 | 70.4 | 5.4 | 0
QBR [15] | 4 / 4 | 70.4 | 5.4 | 90
MPDNN [48] | 3.75MP / 4MP | 69.8 | - | 50
QBitOPT [33] | 4MP / 4MP | 69.7 | - | 30
NIPQ [39] | 4MP / 4MP | 69.2 | - | 43
BayesianBits [49] | 4MP / 4MP | 69.0 | 5.9 | 40
GMPQ [15] |  $\sim$ 4MP / 4MP | 70.4 | 7.4 | 40
HAQ [52] | 6MP / 4MP | 69.5 | 8.3 | 30
Ours | 4MP / 4MP | 70.7 | 5.5 | 0

ResNet. PACT demonstrates accuracy with 3-bit for both weights and
activations, achieving 68.1%. LSQ reaches 69.4% accuracy but requires 90
retraining epochs. EdMIPS and GMPQ employ MPQ (3${\rm MP}$ / 3${\rm MP}$) for
68.2% and 68.6% accuracy but still require considerable retraining costs. DNAS
and FracBits adopt longer retraining epochs and yield better accuracy.

When increasing the bit-width to 4 bits, PACT achieves 69.2% accuracy with
35.0G BitOPs, while LSQ reaches 70.5% accuracy. DNAS and FracBits demonstrate
a 4-bit MPQ with slightly different results, while LIMPQ and SEAM both achieve
the highest accuracy but still need 90 retraining epochs. Notably, our method
with varying bit-width configurations (2${\rm MP}$/3${\rm MP}$, 3${\rm
MP}$/3${\rm MP}$, and 4${\rm MP}$/4${\rm MP}$). The 4${\rm MP}$/4${\rm MP}$
configuration achieves the highest accuracy in the table at 71.0%, with
competitive BitOPs (31.6G) and no retraining cost.
MobileNetV2. QBR demonstrates a competitive Top-1 accuracy of 67.4% with 3/3
bit-width and 3G BitOPs. QBitOPT achieves 65.7% accuracy in the 3${\rm
MP}$/3${\rm MP}$ configuration and requires retraining split into 15 + 15
epochs [33], suggesting a more complex process. In the 4/4 bit-width category,
QBR stands out with 70.4% accuracy and 5.4G BitOPs, demonstrating efficiency.
GMPQ delivers 70.4% accuracy but requires 40 retraining epochs. HAQ achieves
69.5% accuracy but incurs higher BitOPs (8.3G) and demands 30 retraining
epochs.

With 3MP/3MP bit-width, our method reaches 67.8% accuracy with 3.6G BitOPs and
no retraining. Moreover, in the 4MP/4MP configuration, it excels with a Top-1
accuracy of 70.7% and competitive BitOPs (5.5G), all while eliminating
retraining costs.

Table 4: Accuracy and efficiency results for EfficientNetLite-B0. †: QBitOPT uses channel-wise quantization to retain performance.  Method |  Bit-width (W/A) |  Top-1 Acc. (%) $\uparrow$ |  BitOPs (G) $\downarrow$ |  Retrain Cost (Epoch) $\downarrow$
---|---|---|---|---
Baseline | 32 / 32 | 75.4 | - | -
LSQ [11] | 3 / 3 | 69.7 | 4.2 | 90
NIPQ [39] | 3MP / 3MP | 66.5 | - | 43
QBitOPT† [33] | 3MP / 3MP | 70.0 | - | 30
MPQDNN [48] | 3MP / 3MP | 68.8 | - | 50
Ours | 3MP / 3MP | 70.4 | 4.5 | 0
LSQ [11] | 4 / 4 | 72.3 | 6.8 | 90
NIPQ [39] | 4MP / 4MP | 72.3 | - | 43
QBitOPT† [33] | 4MP / 4MP | 73.3 | - | 30
Ours | 4MP / 4MP | 73.2 | 6.9 | 0

EfficientNet. LSQ achieves 69.7% accuracy with 4.2G BitOPs and requires 90
retraining epochs. In contrast, our 3${\rm MP}$/3${\rm MP}$ method attains
70.4% accuracy with 4.5G BitOPs but eliminates the need for retraining,
showcasing improved accuracy at a lower cost. QBitOPT achieves 70.0% accuracy
under 3${\rm MP}$/3${\rm MP}$ with 30 epochs for retraining. Our method at the
same setting achieves 70.4% accuracy without any retraining, highlighting
superior performance without complex retraining. While LSQ and NIPQ achieve
72.3% accuracy at 4/4 bit-width, they demand 90 retraining epochs. Our 4${\rm
MP}$/4${\rm MP}$ method surpasses both, achieving 73.2% accuracy with 6.9G
BitOPs and no retraining. Our method consistently achieves comparable or
superior accuracy with no retraining costs, demonstrating efficacy and
simplicity in EfficientNet quantization.

Table 5: Accuracy and efficiency results for ResNet with knowledge distillation.  Method |  Bit-width (W/A) |  Top-1 Acc. (%) $\uparrow$ |  BitOPs (G) $\downarrow$ |  Retrain Cost (Epoch) $\downarrow$
---|---|---|---|---
Baseline | 32 / 32 | 70.5 | - | -
GMPQ [55] | 3MP / 3MP | 69.5 | 22.8 | 90
SEAM [44] | 3MP / 3MP | 70.7 | 23.0 | 90
EQNet [57] | 3MP / 3MP | 69.8 | - | 0
SDQ [21] | 3MP / 3 | 70.2 | 25.1 | 90
Ours | 3MP / 3MP | 70.9 | 23.9 | 0
NIPQ [39] | 4MP / 4MP | 71.2 | 34.2 | 40
SDQ [21] | 4MP / 3 | 71.7 | 33.4 | 90
Ours | 4MP / 4MP | 71.6 | 31.6 | 0

###  4.3 Ablation Study

Efficientness with KD. In comparison to the existing methods in Tab. 5 when
knowledge distillation (KD) is enabled with a ResNet101 teacher model, our
method exhibits compelling advantages. GMPQ achieves a respectable 69.5%
accuracy with 3MP bit-width but requires 90 retraining epochs. Our method
surpasses it significantly, achieving a 70.9% accuracy without retraining.
Similarly, SEAM marginally improves accuracy to 70.7%, but our method still
outperforms with 70.9% accuracy and no retraining costs. EQNet stands out with
zero retraining epochs but falls significantly short of our method in accuracy
(69.8%). SDQ shows varied performance, but our method consistently outperforms
it, particularly with 3MP / 3MP and 4MP / 4MP bit-width configurations,
achieving higher accuracy and requiring no retraining compared to SDQ’s 90
retraining epochs.

Table 6: Effectiveness of proposed dynamic bit-width schedule scheme and information distortion mitigation (IDM) training technique. To save training cost, we train the weight-sharing model 80 epochs.  Dynamic Bit Schedule | IDM Training | 4 Bit Top-1 (%)
---|---|---
✗ | ✗ | 68.3
✓ | ✗ | 69.1
✓ | ✓ | 69.5

Effectiveness of Proposed Techniques. Tab. 6 investigates the impact of a
dynamic bit-width schedule and our information distortion mitigation (IDM)
training technique on the weight-sharing model. It presents three experimental
scenarios: without both dynamic bit scheduling and IDM training resulting in
68.3% Top-1 accuracy, dynamic bit scheduling alone with an improvement to
69.1%, and the combination of both techniques achieving the highest Top-1
accuracy of 69.5%. The results suggest that both dynamic bit scheduling and
IDM training contribute positively to the model’s performance, and their
combination yields the most significant improvement. Moreover, our IDM
training technique significantly mitigates information distortion, as shown in
Fig. 4.

Figure 4: Output density at 2bit and 6bits with our IDM training. Compared
with Fig. 3, information distortion of the small bit-widths is significantly
mitigated.

###  4.4 Transfer Learning

We transfer the quantized weights for downstream benchmarks to verify the
generalization ability of the proposed method. We directly use the pretrained
checkpoints on ImageNet and then finetune the classifiers. As shown in Tab. 7,
our method achieves the same accuracy as a full-precision model at 4 bits with
smaller model complexity, which further confirms the superiority of the
proposed method.

Table 7: Performance of transfer learning using the pretrained weights on ImageNet.  Model |  Bit-width (W/A) |  CIFAR100 [24] Top-1 Acc. (%) |  Pets [32] Top-1 Acc. (%)
---|---|---|---
ResNet18 | 32 / 32 | 79.4 | 88.9
4MP / 4MP | 79.5 | 88.7
3MP / 3MP | 78.7 | 87.9
MobileNetV2 | 32 / 32 | 78.9 | 86.0
4MP / 4MP | 79.0 | 86.1
3MP / 3MP | 78.2 | 84.1

##  5 Conclusion

In this paper, we introduce a novel one-shot training-searching paradigm for
mixed-precision model compression. More specifically, traditional approaches
focus on bit-width configurations but overlook significant retraining costs.
We identified and addressed bit-width interference issues by introducing a
dynamic scheduler and an information distortion mitigation technique. Together
with an inference-only greedy search scheme, our method can significantly
reduce the costs of mixed-precision quantization. Experiments on three
commonly used benchmarks across various network architectures validate the
effectiveness and efficiency of the proposed method in compressing models.
Overall, our method offers a promising solution for deploying compressed
models without compromising performance on resource-limited devices.

## References

  * Bengio et al. [2013] Yoshua Bengio, Nicholas Léonard, and Aaron C. Courville.  Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation.  _CoRR_ , abs/1308.3432, 2013.
  * Bulat and Tzimiropoulos [2021] Adrian Bulat and Georgios Tzimiropoulos.  Bit-mixer: Mixed-precision networks with runtime bit-width selection.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 5188–5197, 2021.
  * Cai and Vasconcelos [2020] Zhaowei Cai and Nuno Vasconcelos.  Rethinking Differentiable Search for Mixed-precision Neural Networks.  In _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020_ , pages 2346–2355. Computer Vision Foundation / IEEE, 2020.
  * Chen et al. [2017] Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille.  Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs.  _IEEE transactions on pattern analysis and machine intelligence_ , 40(4):834–848, 2017.
  * Chen et al. [2021] Weihan Chen, Peisong Wang, and Jian Cheng.  Towards Mixed-precision Quantization of Neural Networks via Constrained Optimization.  In _2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021_ , pages 5330–5339. IEEE, 2021.
  * Choi et al. [2018] Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, and Kailash Gopalakrishnan.  PACT: Parameterized Clipping Activation for Quantized Neural Networks.  _CoRR_ , abs/1805.06085, 2018.
  * Défossez et al. [2022] Alexandre Défossez, Yossi Adi, and Gabriel Synnaeve.  Differentiable model compression via pseudo quantization noise.  _Transactions on Machine Learning Research_ , 2022.
  * Dong et al. [2019] Zhen Dong, Zhewei Yao, Amir Gholami, Michael W. Mahoney, and Kurt Keutzer.  HAWQ: Hessian AWare Quantization of Neural Networks With Mixed-precision.  In _2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019_ , pages 293–302. IEEE, 2019.
  * Dong et al. [2020] Zhen Dong, Zhewei Yao, Daiyaan Arfeen, Amir Gholami, Michael W. Mahoney, and Kurt Keutzer.  HAWQ-V2: Hessian Aware trace-weighted Quantization of Neural Networks.  In _Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual_ , 2020.
  * Elthakeb et al. [2020] Ahmed T. Elthakeb, Prannoy Pilligundla, Fatemehsadat Mireshghallah, Amir Yazdanbakhsh, and Hadi Esmaeilzadeh.  ReLeQ : A Reinforcement Learning Approach for Automatic Deep Quantization of Neural Networks.  _IEEE Micro_ , 40(5):37–45, 2020.
  * Esser et al. [2020] Steven K. Esser, Jeffrey L. McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmendra S. Modha.  Learned Step Size quantization.  In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net, 2020.
  * Gong et al. [2019] Ruihao Gong, Xianglong Liu, Shenghu Jiang, Tianxiang Li, Peng Hu, Jiazhen Lin, Fengwei Yu, and Junjie Yan.  Differentiable Soft Quantization: Bridging Full-precision and Low-bit Neural Networks.  In _2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019_ , pages 4851–4860. IEEE, 2019.
  * Guo et al. [2020] Zichao Guo, Xiangyu Zhang, Haoyuan Mu, Wen Heng, Zechun Liu, Yichen Wei, and Jian Sun.  Single Path One-shot Neural Architecture Search with Uniform Sampling.  In _Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XVI_ , pages 544–560. Springer, 2020.
  * Habi et al. [2020] Hai Victor Habi, Roy H. Jennings, and Arnon Netzer.  HMQ: Hardware Friendly Mixed Precision Quantization Block for CNNs.  In _Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXVI_ , pages 448–463. Springer, 2020.
  * Han et al. [2021a] Tiantian Han, Dong Li, Ji Liu, Lu Tian, and Yi Shan.  Improving low-precision network quantization via bin regularization.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 5261–5270, 2021a.
  * Han et al. [2021b] Yizeng Han, Gao Huang, Shiji Song, Le Yang, Honghui Wang, and Yulin Wang.  Dynamic neural networks: A survey.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 44(11):7436–7456, 2021b.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep Residual Learning for Image Recognition.  In _2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016_ , pages 770–778. IEEE Computer Society, 2016.
  * He et al. [2017] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick.  Mask r-cnn.  In _Proceedings of the IEEE international conference on computer vision_ , 2017.
  * Howard et al. [2019] Andrew Howard, Ruoming Pang, Hartwig Adam, Quoc V. Le, Mark Sandler, Bo Chen, Weijun Wang, Liang-Chieh Chen, Mingxing Tan, Grace Chu, Vijay Vasudevan, and Yukun Zhu.  Searching for MobileNetV3.  In _2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019_ , pages 1314–1324. IEEE, 2019.
  * Howard et al. [2017] Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications.  _CoRR_ , abs/1704.04861, 2017.
  * Huang et al. [2022] Xijie Huang, Zhiqiang Shen, Shichao Li, Zechun Liu, Xianghong Hu, Jeffry Wicaksana, Eric P. Xing, and Kwang-Ting Cheng.  SDQ: Stochastic Differentiable Quantization with Mixed Precision.  In _International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA_ , pages 9295–9309. PMLR, 2022.
  * Hubara et al. [2021] Itay Hubara, Yury Nahshan, Yair Hanani, Ron Banner, and Daniel Soudry.  Accurate post training quantization with small calibration sets.  In _International Conference on Machine Learning_ , pages 4466–4475. PMLR, 2021.
  * Jin et al. [2020] Qing Jin, Linjie Yang, and Zhenyu Liao.  Adabits: Neural network quantization with adaptive bit-widths.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 2146–2156, 2020.
  * Krizhevsky et al. [2009] Alex Krizhevsky, Geoffrey Hinton, et al.  Learning multiple layers of features from tiny images.  2009\.
  * Lee et al. [2021] Junghyup Lee, Dohyung Kim, and Bumsub Ham.  Network quantization with element-wise gradient scaling.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 6448–6457, 2021.
  * Lin et al. [2017] Ji Lin, Yongming Rao, Jiwen Lu, and Jie Zhou.  Runtime neural pruning.  _Advances in neural information processing systems_ , 30, 2017.
  * Liu et al. [2018] Hanxiao Liu, Karen Simonyan, and Yiming Yang.  Darts: Differentiable architecture search.  _arXiv preprint arXiv:1806.09055_ , 2018.
  * Liu et al. [2019] Zhuang Liu, Mingjie Sun, Tinghui Zhou, Gao Huang, and Trevor Darrell.  Rethinking the Value of Network Pruning.  In _7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019_. OpenReview.net, 2019.
  * Molchanov et al. [2019] Pavlo Molchanov, Arun Mallya, Stephen Tyree, Iuri Frosio, and Jan Kautz.  Importance estimation for neural network pruning.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 11264–11272, 2019.
  * Nagel et al. [2020] Markus Nagel, Rana Ali Amjad, Mart Van Baalen, Christos Louizos, and Tijmen Blankevoort.  Up or down? adaptive rounding for post-training quantization.  In _International Conference on Machine Learning_ , pages 7197–7206. PMLR, 2020.
  * Nagel et al. [2022] Markus Nagel, Marios Fournarakis, Yelysei Bondarenko, and Tijmen Blankevoort.  Overcoming oscillations in quantization-aware training.  In _International Conference on Machine Learning_ , pages 16318–16330. PMLR, 2022.
  * Parkhi et al. [2012] Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar.  Cats and dogs.  In _2012 IEEE conference on computer vision and pattern recognition_ , pages 3498–3505. IEEE, 2012.
  * Peters et al. [2023] Jorn Peters, Marios Fournarakis, Markus Nagel, Mart van Baalen, and Tijmen Blankevoort.  Qbitopt: Fast and accurate bitwidth reallocation during training.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 1282–1291, 2023.
  * Pham et al. [2018] Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, and Jeff Dean.  Efficient neural architecture search via parameters sharing.  In _International conference on machine learning_ , pages 4095–4104. PMLR, 2018.
  * Redmon and Farhadi [2018] Joseph Redmon and Ali Farhadi.  Yolov3: An incremental improvement.  _arXiv preprint arXiv:1804.02767_ , 2018.
  * Ren et al. [2015] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks.  _Advances in neural information processing systems_ , 28, 2015.
  * Sandler et al. [2018] Mark Sandler, Andrew G. Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen.  MobileNetV2: Inverted Residuals and Linear Bottlenecks.  In _2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018_ , pages 4510–4520. Computer Vision Foundation / IEEE Computer Society, 2018.
  * Shen et al. [2020] Jianghao Shen, Yue Wang, Pengfei Xu, Yonggan Fu, Zhangyang Wang, and Yingyan Lin.  Fractional skipping: Towards finer-grained dynamic cnn inference.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , pages 5700–5708, 2020.
  * Shin et al. [2023] Juncheol Shin, Junhyuk So, Sein Park, Seungyeop Kang, Sungjoo Yoo, and Eunhyeok Park.  Nipq: Noise proxy-based integrated pseudo-quantization.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 3852–3861, 2023.
  * Tan and Le [2019] Mingxing Tan and Quoc Le.  Efficientnet: Rethinking model scaling for convolutional neural networks.  In _International conference on machine learning_ , pages 6105–6114. PMLR, 2019.
  * Tan et al. [2020] Mingxing Tan, Ruoming Pang, and Quoc V Le.  Efficientdet: Scalable and efficient object detection.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 10781–10790, 2020.
  * Tang et al. [2022a] Chen Tang, Kai Ouyang, Zhi Wang, Yifei Zhu, Wen Ji, Yaowei Wang, and Wenwu Zhu.  Mixed-precision Neural Network Quantization via Learned Layer-wise Importance.  In _Computer Vision - ECCV 2022 - 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XI_ , pages 259–275. Springer, 2022a.
  * Tang et al. [2022b] Chen Tang, Haoyu Zhai, Kai Ouyang, Zhi Wang, Yifei Zhu, and Wenwu Zhu.  Arbitrary bit-width network: A joint layer-wise quantization and adaptive inference approach.  In _Proceedings of the 30th ACM International Conference on Multimedia_ , pages 2899–2908, 2022b.
  * Tang et al. [2023a] Chen Tang, Kai Ouyang, Zenghao Chai, Yunpeng Bai, Yuan Meng, Zhi Wang, and Wenwu Zhu.  Seam: Searching transferable mixed-precision quantization policy through large margin regularization.  In _Proceedings of the 31st ACM International Conference on Multimedia_ , pages 7971–7980, 2023a.
  * Tang et al. [2023b] Chen Tang, Li Lyna Zhang, Huiqiang Jiang, Jiahang Xu, Ting Cao, Quanlu Zhang, Yuqing Yang, Zhi Wang, and Mao Yang.  Elasticvit: Conflict-aware supernet training for deploying fast vision transformer on diverse mobile devices.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , 2023b.
  * Tishby and Zaslavsky [2015] Naftali Tishby and Noga Zaslavsky.  Deep learning and the information bottleneck principle.  In _2015 ieee information theory workshop (itw)_ , pages 1–5. IEEE, 2015.
  * Tishby et al. [2000] Naftali Tishby, Fernando C Pereira, and William Bialek.  The information bottleneck method.  _arXiv preprint physics/0004057_ , 2000.
  * Uhlich et al. [2019] Stefan Uhlich, Lukas Mauch, Fabien Cardinaux, Kazuki Yoshiyama, Javier Alonso Garcia, Stephen Tiedemann, Thomas Kemp, and Akira Nakamura.  Mixed precision dnns: All you need is a good parametrization.  _arXiv preprint arXiv:1905.11452_ , 2019.
  * Van Baalen et al. [2020] Mart Van Baalen, Christos Louizos, Markus Nagel, Rana Ali Amjad, Ying Wang, Tijmen Blankevoort, and Max Welling.  Bayesian bits: Unifying quantization and pruning.  _Advances in neural information processing systems_ , 33:5741–5752, 2020.
  * Wang et al. [2021a] Dilin Wang, Meng Li, Chengyue Gong, and Vikas Chandra.  Attentivenas: Improving neural architecture search via attentive sampling.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR)_ , 2021a.
  * Wang et al. [2022] Jue Wang, Ke Chen, Gang Chen, Lidan Shou, and Julian McAuley.  Skipbert: Efficient inference with shallow layer skipping.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 7287–7301, 2022.
  * Wang et al. [2019] Kuan Wang, Zhijian Liu, Yujun Lin, Ji Lin, and Song Han.  HAQ: Hardware-aware Automated Quantization With Mixed Precision.  In _IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019_ , pages 8612–8620. Computer Vision Foundation / IEEE, 2019.
  * Wang et al. [2018] Xin Wang, Fisher Yu, Zi-Yi Dou, Trevor Darrell, and Joseph E Gonzalez.  Skipnet: Learning dynamic routing in convolutional networks.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , pages 409–424, 2018.
  * Wang et al. [2020] Yulong Wang, Xiaolu Zhang, Xiaolin Hu, Bo Zhang, and Hang Su.  Dynamic network pruning with interpretable layerwise channel selection.  In _Proceedings of the AAAI conference on artificial intelligence_ , pages 6299–6306, 2020.
  * Wang et al. [2021b] Ziwei Wang, Han Xiao, Jiwen Lu, and Jie Zhou.  Generalizable Mixed-precision Quantization via Attribution Rank Preservation.  In _2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021_ , pages 5271–5280. IEEE, 2021b.
  * Wu et al. [2018] Bichen Wu, Yanghan Wang, Peizhao Zhang, Yuandong Tian, Peter Vajda, and Kurt Keutzer.  Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search.  _CoRR_ , abs/1812.00090, 2018.
  * Xu et al. [2023a] Ke Xu, Lei Han, Ye Tian, Shangshang Yang, and Xingyi Zhang.  Eq-net: Elastic quantization neural networks.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 1505–1514, 2023a.
  * Xu et al. [2023b] Sheng Xu, Yanjing Li, Mingbao Lin, Peng Gao, Guodong Guo, Jinhu Lü, and Baochang Zhang.  Q-detr: An efficient low-bit quantized detection transformer.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 3842–3851, 2023b.
  * Yang and Jin [2021] Linjie Yang and Qing Jin.  FracBits: Mixed Precision Quantization via Fractional Bit-widths.  In _Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021_ , pages 10612–10620. AAAI Press, 2021.
  * Yu et al. [2020a] Haibao Yu, Qi Han, Jianbo Li, Jianping Shi, Guangliang Cheng, and Bin Fan.  Search What You Want: Barrier Panelty NAS for Mixed Precision Quantization.  In _Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part IX_ , pages 1–16. Springer, 2020a.
  * Yu et al. [2020b] Jiahui Yu, Pengchong Jin, Hanxiao Liu, Gabriel Bender, Pieter-Jan Kindermans, Mingxing Tan, Thomas Huang, Xiaodan Song, Ruoming Pang, and Quoc Le.  Bignas: Scaling up neural architecture search with big single-stage models.  In _Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VII 16_ , pages 702–717. Springer, 2020b.
  * Zhou et al. [2016] Shuchang Zhou, Zekun Ni, Xinyu Zhou, He Wen, Yuxin Wu, and Yuheng Zou.  DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients.  _CoRR_ , abs/1606.06160, 2016.
  * Zhou et al. [2018] Yiren Zhou, Seyed-Mohsen Moosavi-Dezfooli, Ngai-Man Cheung, and Pascal Frossard.  Adaptive quantization for deep neural network.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , 2018.
  * Zoph and Le [2016] Barret Zoph and Quoc V Le.  Neural architecture search with reinforcement learning.  _arXiv preprint arXiv:1611.01578_ , 2016.


# X-ray absorption spectroscopy using an ultrafast laboratory-scale laser-
plasma accelerator source

B. Kettle<EMAIL_ADDRESS>The John Adams Institute for Accelerator
Science, Imperial College London, London, SW7 2AZ, UK  C. Colgan  The John
Adams Institute for Accelerator Science, Imperial College London, London, SW7
2AZ, UK  E. Los  The John Adams Institute for Accelerator Science, Imperial
College London, London, SW7 2AZ, UK  E. Gerstmayr  School of Mathematics and
Physics, Queen’s University of Belfast, BT7 1NN, Belfast, UK  The John Adams
Institute for Accelerator Science, Imperial College London, London, SW7 2AZ,
UK  M.J.V. Streeter  School of Mathematics and Physics, Queen’s University of
Belfast, BT7 1NN, Belfast, UK  F. Albert  Lawrence Livermore National
Laboratory (LLNL), Livermore, California 94550, USA  S. Astbury  Central Laser
Facility, STFC Rutherford Appleton Laboratory, Didcot OX11 0QX, UK  R.A.
Baggott  The John Adams Institute for Accelerator Science, Imperial College
London, London, SW7 2AZ, UK  N. Cavanagh  School of Mathematics and Physics,
Queen’s University of Belfast, BT7 1NN, Belfast, UK  K. Falk  Helmholtz-
Zentrum Dresden-Rossendorf, Bautzner Landstrasse 400, 01328 Dresden, Germany
Technische Universität Dresden, D-01069 Dresden, Germany  Institute of Physics
of the ASCR, Na Slovance 1999/2, 182 21 Prague, Czech Republic  T.I. Hyde
Johnson Matthey Technology Centre, Blount’s Court, Sonning Common, Reading,
RG4 9NH, United Kingdom  O. Lundh  Department of Physics, Lund University,
P.O. Box 118, S-22100, Lund, Sweden  P.P. Rajeev  Central Laser Facility, STFC
Rutherford Appleton Laboratory, Didcot OX11 0QX, UK  D. Riley  School of
Mathematics and Physics, Queen’s University of Belfast, BT7 1NN, Belfast, UK
S.J. Rose  The John Adams Institute for Accelerator Science, Imperial College
London, London, SW7 2AZ, UK  G. Sarri  School of Mathematics and Physics,
Queen’s University of Belfast, BT7 1NN, Belfast, UK  C. Spindloe  Central
Laser Facility, STFC Rutherford Appleton Laboratory, Didcot OX11 0QX, UK  K.
Svendsen  Department of Physics, Lund University, P.O. Box 118, S-22100, Lund,
Sweden  D.R. Symes  Central Laser Facility, STFC Rutherford Appleton
Laboratory, Didcot OX11 0QX, UK  M. Šmíd  Helmholtz-Zentrum Dresden-
Rossendorf, Bautzner Landstrasse 400, 01328 Dresden, Germany  A.G.R. Thomas
Gérard Mourou Center for Ultrafast Optical Science, University of Michigan,
Ann Arbor, Michigan 48109-2099, USA  C. Thornton  Central Laser Facility, STFC
Rutherford Appleton Laboratory, Didcot OX11 0QX, UK  R. Watt  Stanford PULSE
Institute, SLAC National Accelerator Laboratory, Menlo Park, California 94025,
USA  The John Adams Institute for Accelerator Science, Imperial College
London, London, SW7 2AZ, UK  S.P.D. Mangles  The John Adams Institute for
Accelerator Science, Imperial College London, London, SW7 2AZ, UK
<EMAIL_ADDRESS>
###### Abstract

The absorption profile of the copper K-edge was measured over a 250 eV window
using ultrashort X-rays from a laser-plasma wakefield accelerator. For the
first time with a femtosecond probe, Extended X-ray Absorption Fine Structure
(EXAFS) features were observed in a single shot, detailing the local atomic
structure. This unique capability will allow the investigation of novel
ultrafast processes, and in particular probing high energy density matter and
physics far-from-equilibrium. A perspective on the additional strengths of a
laboratory-based ultrafast X-ray absorption source is presented.

In recent years laser-plasma based particle accelerators have provided access
to gigaelectronvolt electron energies within the small-scale laboratory
environment Esarey  _et al._ (2009); Albert  _et al._ (2021); Gonsalves  _et
al._ (2019). This has led to new research involving strong field QED studies
Cole  _et al._ (2018); Poder  _et al._ (2018), electron-positron pair
generation Kettle  _et al._ (2021); Sarri  _et al._ (2015) and X-ray and gamma
ray applications Albert  _et al._ (2014); Cole  _et al._ (2015); Sarri  _et
al._ (2014); Schumaker  _et al._ (2014). One growing area makes use of the
X-rays generated in tandem with the accelerated beam as the electrons wiggle
in the back of the plasma wakefield bubble Kneip  _et al._ (2010); Corde  _et
al._ (2013). The X-rays have a pulse duration comparable to the emitted
electron bunch; usually 10’s of femtoseconds Debus  _et al._ (2010); Lundh
_et al._ (2011); Mangles  _et al._ (2006).

A key strength of this X-ray source is that it has a smooth, broadband
synchrotron-like spectrum, making it ideal for X-ray absorption spectroscopy
(XAS) techniques such as X-ray Absorption Near-Edge Structure (XANES) and
Extended X-ray Absorption Fine Structure (EXAFS) spectroscopy. In these
techniques the absorption, scattering and interference of ejected
photoelectrons from neighboring atoms manifest as modulations in the
absorption profile near resonant edges. These modulations are directly linked
to the local electronic and ionic structure of the sample Rehr and Albers
(2000); Koningsberger and Prins (1988). In fact the electron temperature, ion
temperature, ionisation state and local ionic positions can be simultaneously
measured Dorchies and Recoules (2016).

Over the last few decades XANES and EXAFS has been used with great success on
synchrotron facilities to measure the structure of materials, across a wide
range of sciences Syn (2018); Bilderback  _et al._ (2005). However, when the
duration of the X-ray probe is important, measurements are limited to the
pulse duration of the synchrotron; generally on the order of hundreds of
picoseconds Torchio  _et al._ (2016), unless additional slicing techniques are
implemented (that sacrifice flux). X-ray Free Electron Laser (XFEL) facilities
operate at femtosecond pulse duration but provide a narrow bandwidth,
requiring a scan over many shots to obtain a broad absorption spectrum Lemke
_et al._ (2013). The combined ultrashort and broadband capabilities of a
laser-plasma accelerator X-ray source makes it uniquely suited to investigate
novel ultrafast processes on a femtosecond scale through XAS techniques. These
include femtochemistry phenomena, such as photodissociation, spin crossover
processes Bressler  _et al._ (2008); Gawelda  _et al._ (2007); Chen (2001,
2005), photobiology Bressler and Chergui (2010); Jonas  _et al._ (1996); van
Grondelle  _et al._ (1994); Arcovito  _et al._ (2005), or photocatalysts and
photoelectrodes Uemura  _et al._ (2017); Hu  _et al._ (2021); Park  _et al._
(2022), which are becoming increasingly important, for example in energy
storage research. Another active area is that of spintronics for data storage
and transfer, with specific interest in antiferromagnets Hirohata  _et al._
(2020); Li  _et al._ (2019); Johnson  _et al._ (2015); Strungaru  _et al._
(2022). Other ultrafast phenomena such as non-thermal phase changes in semi-
conductors Rousse  _et al._ (2001) or bond-hardening effects Ernstorfer  _et
al._ (2009) are also of interest.

Figure 1:  Overview of the instrument setup. An f/40 laser creates a LWFA
(inset) in a high pressure gas jet, generating a beam of on-axis electrons and
X-rays. The electrons are swept away by a dipole magnet and the X-rays are
either diffracted off a cylindrical crystal through a sample and spectrally
spread onto a shielded CCD, or alternatively (with the crystal translated out)
straight onto a CCD through a multi-element filter array used to infer the
broad spectral shape.

A major application in physics is the study of extreme states far from
equilibrium Mahieu  _et al._ (2018), or those with high energy density, such
as the tremendous pressures and temperatures of planetary formations Koenig
_et al._ (2004), or inertial confinement fusion experiments Betti and
Hurricane (2016). For example, nanosecond EXAFS has been performed on high-
energy laser systems to investigate materials at high pressures Yaakobi  _et
al._ (2004); Ping  _et al._ (2013). For these experiments a single-shot
measurement is of major benefit, as driving the sample to the appropriate
conditions requires extensive resources and shot rate is limited.

Until now, the photon flux from laser-plasma accelerator based XAS
measurements has been lacking, requiring 10’s to 100’s of shots to form a full
absorption profile, which often suffers from noise and limited spectral range
Ta Phuoc  _et al._ (2007); Mahieu  _et al._ (2018). Recently single-shot XANES
measurements using a laser-plasma accelerator have been demonstrated Kettle
_et al._ (2019), although due to the noise levels of the absorption profile,
the spectral range was limited to 40 eV. In this article, using a new
experimental geometry, we demonstrate a marked increase in signal-over-noise
and extend the single-shot spectral range to over 250 eV. This allows access
to the EXAFS region for the first time, an important capability for performing
pump-probe experiments of high-energy-density physics and extreme states.
Using these developments, we present a platform for a laser-plasma accelerator
based XAS system that will enable high quality laboratory-scale measurements,
while providing the unique capability of measuring ultrafast processes on a
femtosecond timescale.

The experiment was conducted using the Gemini Laser at the Central Laser
Facility (U.K.). An overview of the setup can be seen in Fig. 1 Our (2023).
The drive laser was focused using an $f$/40 geometry onto a 15 mm diameter gas
jet nozzle. Each laser pulse (provided at 0.05 Hz) had a duration of $45\pm 5$
fs and contained $6.0\pm 0.3$ J, corresponding to a peak laser power of
$134\pm 18$ TW. This provided an on-target intensity of ($3.0\pm 0.4)\times
10^{18}~{}\textrm{W/cm}^{2}$ and a laser strength parameter of $a_{0}=1.2\pm
0.1$, before any self-focusing effects. As this pulse propagates through the
gas, it drives a laser wakefield accelerator (LWFA), expelling electrons from
the atoms and creating a charge cavity in the wake of the laser pulse. The
electrons are accelerated to relativistic energies in this wake, emitting
X-ray synchrotron light as they oscillate at the back of the cavity. A 99% He
and 1% $\mathrm{N}_{2}$ mix was used to promote ionisation injection of
electrons into the charge cavity Pak  _et al._ (2010); McGuffey  _et al._
(2010). The nozzle provided a plasma density of $n_{e}=(2.1\pm 0.2)\times
10^{18}~{}\mathrm{cm}^{-3}$. After the LWFA the residual drive laser was
blocked by a refreshable tape drive (125 $\rm{\mu m}$ Kapton), and the
electron beam was diagnosed with a 0.3 $\mathrm{T}\,\mathrm{m}$ magnetic
spectrometer that dispersed the beam onto an imaged scintillating Lanex
screen. The electron beam was then dumped into a lead shielded cavity. The
broadband X-ray beam is reflected off a cylindrical HAPG (Highly Annealed
Pyrolytic Graphite) crystal, onto a well shielded X-ray CCD. The crystal
spectrally disperses the X-rays in one axis, while focusing them spatially in
the other axis. The crystal was tilted 11.8∘ from the laser and X-ray axis to
observe X-rays in the 9 keV region. Various copper samples were placed on a
translation stage before the CCD, so that their absorption could be measured.
By temporarily moving the crystal, it was also possible to infer the full
broadband spectral shape of the X-rays using an on-axis CCD with an elemental
filter array placed in front Rousse  _et al._ (2004); Kneip  _et al._ (2008,
2010); King  _et al._ (2019).

Figure 2:  (a) Electron spectra for ten consecutive shots. (b) Mean and
standard deviation of these spectra (shaded). (c) Raw XAS CCD data for the
same shots. The X-rays are spatially focused to a line, and spectrally spread
in the vertical axis. (d) Smoothed X-ray reference profile shape (red), the
unsmoothed average of the 10 normalised profiles (black), and the standard
deviation (shaded blue); the X-rays on the spectrometer demonstrate a stable
and consistent spectral shape despite the variation in electron beam
properties.

For the data discussed here, the LWFA was operated in a high-charge mode (as
opposed to high electron energy), as this has previously been seen to produce
more photons in our desired spectral range Kettle  _et al._ (2019). Fig. 2 (a)
depicts the electron spectra for 10 consecutive example shots. The 90th
percentile electron energy was $684\pm 39$ MeV, with a charge of $99\pm 29$ pC
per shot. This led to the generation of an on-axis X-ray beam with an average
best-fit critical energy of $E_{\rm{crit}}=19.6\pm 1.0~{}\mathrm{keV}$, and a
standard deviation of $\sigma_{\mathrm{E_{crit}}}=3.4~{}\mathrm{keV}$
(inferred over 10 shots in the run prior to the data of Fig. 2 (a)). Fig. 2
(c) depicts the X-ray signal for the same 10 shots of Fig. 2 (a). On average
$(2.4\pm 0.5)\times 10^{5}$ photons/eV were emitted in the 9 keV region per
shot. It is important to note that while the spectral content of the electron
beam is somewhat erratic, with the 90th percentile electron beam energy
varying by 150 MeV (22%) across the run and a shot-to-shot variation in total
charge of 29%, the measured portion of 9 keV radiation is seen to be
spectrally smooth, and can be easily scaled in magnitude. This is because the
crystal spectrometer observes a small slice of the broadband synchrotron
spectrum, a relatively slowly varying function with electron beam energy. By
averaging over the profiles of the X-ray data in Fig. 2 (c), a reference
spectral profile shape is found (each single shot is normalised to its peak
level, and a rolling average smoothing function is applied). This profile is
given in Fig. 2 (d). The variation from the measured signal is on order of a
few percent and so it is possible for the absorption profile of a sample to be
measured without an on-shot reference (but rather an assumed profile). In
summary, the stability of the X-ray source for absorption measurements is
evident, despite the fluctuations of the electron beam properties.

The absorption profile of a $4~{}\mathrm{\mu m}$ copper sample has been
measured, assuming the reference profile of Fig. 2 (d) as the unattenuated
signal (the known transmission before the copper K-edge is used to scale the
magnitude). Once an absorption profile is found, a normalised absorption is
calculated by fitting below and above the copper K-edge, subtracting the
former, before dividing by the latter. The process is a standard XAS
procedure, and we follow the method outlined in our previous work Kettle  _et
al._ (2019), and that of Cook and Sayers Cook and Sayers (1981).

Figure 3:  Normalised absorption profile of a $4~{}\mathrm{\mu m}$ copper
foil. Data is shown for a single shot (grey) and an average of ten shots
(black). Both are compared to a synchrotron reference (blue) convolved with
and without our simulated instrument response. Inset: zoomed view of the XANES
profile near the edge.

Fig. 3 depicts the resulting normalised absorption data for the copper sample.
The inset gives the XANES profile close to the edge. The profile for a single
shot (grey) and the average profile for a 10 shot run (black) is shown. Both
are compared to the absorption profile of a similar copper foil sample
measured at B18, the general purpose XAS beamline at the Diamond Light Source
synchrotron facility. This reference is presented both with the simulated
instrument function of our system applied (blue solid), and the unaltered
synchrotron data (light blue dashed). Our instrument response was simulated
using the MMPXRT code Šmíd  _et al._ (2021) and indicates a spectral
resolution of $\approx 5$ eV (FWHM). Note that the reference data without our
instrument function applied differs only slightly at the edge (inset). The
measured profile stretches over a 250 eV spectral range, allowing access to
the EXAFS region of the profile. It demonstrates a marked increase in signal-
to-noise over our previous work Kettle  _et al._ (2019), as well as overall
photon number from other LWFA X-ray absorption measurements Mahieu  _et al._
(2018). The two most important alterations to the experimental setup were to
use a spatially focusing cylindrical crystal, and to mitigate the noise on the
X-ray detector from the electron beam by sweeping the beam into a shielded
dump and separately enclosing the X-ray CCD. Both these factors contributed to
greatly increased signal-to-noise. In fact, we infer that the noise level on a
single shot is close to Poisson limited (where for peak signal we have
$\approx 500$ photons per spectral pixel, with a standard deviation of
$\approx 23$ photons, or $\approx\sqrt{N_{ph}}$). This indicates we have
removed the majority of any environmental noise sources, and are limited by
the measured photon number of our signal. The CCD is cooled to $5^{o}$C and
readout noise is on order of 1 count per spectral pixel, or $\approx 0.1\%$ of
a single photon.

Figure 4: (a) Isolated EXAFS oscillations in $k$-space (b) The Fourier
transform to $R$-space, where peaks correspond to the distance to neighbouring
atoms, or coordination shells (labelled).

In Fig. 3 the EXAFS features after the edge are visible on a single shot, a
first of its kind measurement for a laser-plasma accelerator XAS system. We
investigate these features further by subtracting a spline fit to the post-
edge background and converting to $\chi(k)$, the EXAFS fine-structure
function, where $k=\sqrt{2m_{e}(E-E_{0})/\hbar^{2}}$ is wavenumber and $E_{0}$
is the absorption threshold energy. See Fig. 4 (a). Note that while our
spectral resolution results in a small amount of broadening on the fine XANES
features, there is no meaningful effect on the broader EXAFS structure. The
next step is to Fourier transform to $\chi(R)$. See Fig. 4 (b) for a
comparison of our data to the synchrotron reference data, processed under the
same conditions. $|\chi(R)|$ is similar to a radial distribution function, and
the peaks correspond to the distance to neighboring atoms (or coordination
shells). The scattering from the first four shells are labelled Zabinsky  _et
al._ (1995). Our single shot data matches the first shell of the reference
data to within 1.5%, and shells 2-4 within 5% (a peak finding algorithm was
used to determine their location). The 10 shot data improves on this, with the
second, third and fourth shells matching within 2%, 3% and 5% respectively.
This analysis is a direct measurement of the local atomic structure, and
unique to the species of a sample. Interpretation of this structure also
allows access to information on the ion temperature, or any sample compression
or phase changes.

It is evident that the signal-to-noise ratio we have achieved has made it
possible to produce high quality measurements of both the XANES and EXAFS
features in a single shot using a femtosecond probe. This enables XAS probing
of ultrafast processes, but especially those samples in extreme conditions
which are limited to a low shot rate. The development of this ultrafast XAS
platform is on-going, with clear routes for further improvements on flux,
photon energy and spectral resolution. Petawatt-class laser systems are
becoming more commonplace, providing an order of magnitude more laser power
than the experiment we detail. These systems can access higher electron
energies Gonsalves  _et al._ (2019); Thomas (2010), and an estimated order of
magnitude increase in the associated X-ray flux Bloom  _et al._ (2020); Thomas
(2010). Novel targetry techniques can also be used to increase the source
X-ray flux. Recent results Rakowski  _et al._ (2022); Kozlova  _et al._
(2020); Ta Phuoc  _et al._ (2008) have shown that introducing a density
gradient into the target, using a wire or razor blade obstruction, can
increase the electron oscillation radius and frequency, enhancing the X-ray
emission by an order of magnitude. Combining the above improvements in laser
technology and targetry one could speculate two orders of magnitude
improvement in signal-to-noise. See Fig. 5 (a) for a simulated comparison of
these improvements, based on the noise being Poisson-limited by the photon
number. The spectral resolution can be increased by using a perfect crystal
(such as Si or Ge) instead of a mosaic crystal, if a sacrifice in X-ray flux
can be allowed. Simulations using MMPXRT Šmíd  _et al._ (2021) suggest sub-eV
resolution in this configuration. The small source size of the X-rays (sub-
micron Kneip  _et al._ (2012)) can be harnessed in conjunction with a suitable
focusing optic, such as a toroidal crystal, to allow micron focusing of the
probe beam. This can be used to probe small sample volumes (for example in HED
targets) or high-resolution scanning of inhomogeneities in industrial samples
such as batteries or spintronics materials. Accessing higher electron beam
energies will also generate synchrotron spectra with higher critical energies,
and thus provide photons of 100’s keV Kneip  _et al._ (2012); Hojbota  _et
al._ (2022). These photon energies cannot be accessed easily by conventional
synchrotron facilities, and will provide the capability to probe higher-Z
elements and inner atomic edges. Important, for example, in nuclear research
Lind  _et al._ (2007). The stability of the electron accelerator can be
improved through laser techniques Döpp  _et al._ (2017); Ferran Pousa  _et
al._ (2022), and recent studies have shown that machine learning can be used
to optimise the X-ray emission Shalloo  _et al._ (2020); Ye  _et al._ (2022).

Figure 5:  (a) Predictions for signal quality improvement with tailored gas
targetry (purple) and increased laser power (green), where the plots have been
shifted vertically for clarity. (b) Example absorption profile for a copper
oxide catalyst (black), compared to a synchrotron scan (cyan) and a standard
copper oxide sample (blue dashed).

In addition to the above, petawatt lasers can now operate at repetition rates
of 1 Hz, providing an increased rate of data taking ($\times 20$ increase from
this experiment). This would allow laser-plasma accelerator based XAS
platforms to complement synchrotron systems, i.e. making a similar quality
scan in a few minutes. As an example, using our current source capability,
Fig. 5 (b) depicts an ex situ absorption profile of a copper based
hydrogenation catalyst provided by Johnson Matthey, UK. Our data (black), and
the profile for the same sample scanned at the Diamond Light Source (cyan), is
compared to a CuO reference sample (dashed blue). The characteristic features
of this industrial catalyst are clear, indicated by a shifted edge at 8995 eV
and an altered near-edge shape at 9015 eV, even with the Cu loading at 13 wt%
as expressed as an oxide. The oxidation state shows the Cu to be oxidic and
predominantly in the Cu2+ oxidation state in comparison to the CuO reference
sample. The data shown took 10 minutes of shots, but with the improvements
discussed in the previous paragraph, a profile with an improved signal-over-
noise could be generated in under a minute. The benefit of a laser-plasma
based XAS system as described here is that it could be achieved in a
relatively small laboratory environment, in comparison to a conventional
synchrotron light source that requires a large national facility. This reduced
scale makes it an accessible tool for single institutions and makes long-term
programmatic research possible. For example detailed studies of catalysts
(which can improve conversion rates, lower energy consumption, or tune the
selectivity of chemical reactions) Fernández-García (2002), or new energy
storage technologies, including long term effects such as repeated charge
cycle degradation of batteries Jahrman  _et al._ (2019); Preger  _et al._
(2020); Bi  _et al._ (2019); Wang  _et al._ (2016); Nowack  _et al._ (2016). A
laboratory scale instrument also allows dedicated user access for immediate
probing of delicate samples or the appropriate handling of hazardous samples,
for example biological hazards, or the analysis of radiological materials for
environmental remediation (nuclear waste management) Zhong  _et al._ (2021);
Mottram  _et al._ (2020).

In conclusion, we have demonstrated that the X-rays from a LWFA source are
sufficiently bright and stable to measure the absorption profile around the
copper K-edge, over a 250 eV spectral window in a single shot. The absorption
profile is of sufficient quality to make single-shot EXAFS measurements with a
LFWA-based source for the first time, providing direct access to the local
atomic structure of a sample using a femtosecond probe. In particular EXAFS is
sensitive to short-range order and can provide the ion temperature of a
sample, both difficult measurements to make. Single shot capability of an
ultrafast probe is of significant benefit for studying transient pump-probe
samples of high-energy-density physics such as warm dense matter and other
non-equilibrium states, as well as other ultrafast electronic and ionic
processes. The compact size will also allow the instrument to be co-located
with other laboratories more easily, especially XFEL’s or high-power laser
facilities (where it can be inherently synchronised on a femtosecond level),
acting as an auxiliary probe. In addition, by implementing an XAS source on a
laboratory scale, the breadth and depth of both industrial and programmatic
scientific research that can be achieved on a wide-scale basis is vastly
increased.

We wish to acknowledge the support of the staff at the Central Laser Facility.
This project has received funding from the European Research Council (ERC)
under the European Union’s Horizon 2020 research and innovation programme
(grant agreement no 682399), as well as support from the John Adams Institute
for Accelerator Science (STFC: ST/P002021/1, ST/V001639/1), the EPSRC
(EP/S001379/1, EP/V044397/1, EP/N027175/1), the US DOE grant No. DE-SC0022109,
the Knut and Alice Wallenberg Foundation (KAW 2019.0318), the Swedish Research
Council (2019-04784), and the Helmholtz Association (VH-NG-1338). The data
that support the findings of this study are openly available Our (2023).

## References

  * Esarey  _et al._ (2009) E. Esarey, C. B. Schroeder, and W. P. Leemans, Rev. Mod. Phys. 81, 1229 (2009).
  * Albert  _et al._ (2021) F. Albert, M. E. Couprie, A. Debus, M. C. Downer, J. Faure, A. Flacco, L. A. Gizzi, T. Grismayer, A. Huebl, C. Joshi, M. Labat, W. P. Leemans, A. R. Maier, S. P. D. Mangles, P. Mason, F. Mathieu, P. Muggli, M. Nishiuchi, J. Osterhoff, P. P. Rajeev, U. Schramm, J. Schreiber, A. G. R. Thomas, J.-L. Vay, M. Vranic, and K. Zeil, New Journal of Physics 23, 031101 (2021).
  * Gonsalves  _et al._ (2019) A. J. Gonsalves, K. Nakamura, J. Daniels, C. Benedetti, C. Pieronek, T. C. H. de Raadt, S. Steinke, J. H. Bin, S. S. Bulanov, J. van Tilborg, C. G. R. Geddes, C. B. Schroeder, C. Tóth, E. Esarey, K. Swanson, L. Fan-Chiang, G. Bagdasarov, N. Bobrova, V. Gasilov, G. Korn, P. Sasorov, and W. P. Leemans, Phys. Rev. Lett. 122, 084801 (2019).
  * Cole  _et al._ (2018) J. M. Cole, K. T. Behm, E. Gerstmayr, T. G. Blackburn, J. C. Wood, C. D. Baird, M. J. Duff, C. Harvey, A. Ilderton, A. S. Joglekar, K. Krushelnick, S. Kuschel, M. Marklund, P. McKenna, C. D. Murphy, K. Poder, C. P. Ridgers, G. M. Samarin, G. Sarri, D. R. Symes, A. G. R. Thomas, J. Warwick, M. Zepf, Z. Najmudin, and S. P. D. Mangles, Phys. Rev. X 8, 011020 (2018).
  * Poder  _et al._ (2018) K. Poder, M. Tamburini, G. Sarri, A. Di Piazza, S. Kuschel, C. D. Baird, K. Behm, S. Bohlen, J. M. Cole, D. J. Corvan, M. Duff, E. Gerstmayr, C. H. Keitel, K. Krushelnick, S. P. D. Mangles, P. McKenna, C. D. Murphy, Z. Najmudin, C. P. Ridgers, G. M. Samarin, D. R. Symes, A. G. R. Thomas, J. Warwick, and M. Zepf, Phys. Rev. X 8, 031004 (2018).
  * Kettle  _et al._ (2021) B. Kettle, D. Hollatz, E. Gerstmayr, G. M. Samarin, A. Alejo, S. Astbury, C. Baird, S. Bohlen, M. Campbell, C. Colgan, D. Dannheim, C. Gregory, H. Harsh, P. Hatfield, J. Hinojosa, Y. Katzir, J. Morton, C. D. Murphy, A. Nurnberg, J. Osterhoff, G. Pérez-Callejo, K. Põder, P. P. Rajeev, C. Roedel, F. Roeder, F. C. Salgado, G. Sarri, A. Seidel, S. Spannagel, C. Spindloe, S. Steinke, M. J. V. Streeter, A. G. R. Thomas, C. Underwood, R. Watt, M. Zepf, S. J. Rose, and S. P. D. Mangles, New Journal of Physics 23, 115006 (2021).
  * Sarri  _et al._ (2015) G. Sarri, K. Poder, J. M. Cole, W. Schumaker, A. Di Piazza, B. Reville, T. Dzelzainis, D. Doria, L. A. Gizzi, G. Grittani, S. Kar, C. H. Keitel, K. Krushelnick, S. Kuschel, S. P. D. Mangles, Z. Najmudin, N. Shukla, L. O. Silva, D. Symes, A. G. R. Thomas, M. Vargas, J. Vieira, and M. Zepf, Nature Communications 6, 6747 (2015).
  * Albert  _et al._ (2014) F. Albert, A. G. R. Thomas, S. P. D. Mangles, S. Banerjee, S. Corde, A. Flacco, M. Litos, D. Neely, J. Vieira, and Z. Najmudin, Plasma Phys. Control. Fusion 56, 084015 (2014).
  * Cole  _et al._ (2015) J. M. Cole, J. C. Wood, N. C. Lopes, K. Poder, R. L. Abel, S. Alatabi, J. S. J. Bryant, A. Jin, S. Kneip, K. Mecseki, _et al._ , Sci. Rep. 5 (2015), 10.1038/srep13244.
  * Sarri  _et al._ (2014) G. Sarri, D. J. Corvan, W. Schumaker, J. M. Cole, A. Di Piazza, H. Ahmed, C. Harvey, C. H. Keitel, K. Krushelnick, S. P. D. Mangles, Z. Najmudin, D. Symes, A. G. R. Thomas, M. Yeung, Z. Zhao, and M. Zepf, Phys. Rev. Lett. 113, 224801 (2014).
  * Schumaker  _et al._ (2014) W. Schumaker, G. Sarri, M. Vargas, Z. Zhao, K. Behm, V. Chvykov, B. Dromey, B. Hou, A. Maksimchuk, J. Nees, V. Yanovsky, M. Zepf, A. G. R. Thomas, and K. Krushelnick, Physics of Plasmas 21, 056704 (2014), https://doi.org/10.1063/1.4875336 .
  * Kneip  _et al._ (2010) S. Kneip, C. McGuffey, J. L. Martins, S. F. Martins, C. Bellei, V. Chvykov, F. Dollar, R. Fonseca, C. Huntington, G. Kalintchenko, _et al._ , Nat. Phys. 6, 980 (2010).
  * Corde  _et al._ (2013) S. Corde, K. Ta Phuoc, G. Lambert, R. Fitour, V. Malka, A. Rousse, A. Beck, and E. Lefebvre, Rev. Mod. Phys. 85, 1 (2013).
  * Debus  _et al._ (2010) A. D. Debus, M. Bussmann, U. Schramm, R. Sauerbrey, C. D. Murphy, Z. Major, R. Hörlein, L. Veisz, K. Schmid, J. Schreiber, K. Witte, S. P. Jamison, J. G. Gallacher, D. A. Jaroszynski, M. C. Kaluza, B. Hidding, S. Kiselev, R. Heathcote, P. S. Foster, D. Neely, E. J. Divall, C. J. Hooker, J. M. Smith, K. Ertel, A. J. Langley, P. Norreys, J. L. Collier, and S. Karsch, Phys. Rev. Lett. 104, 084802 (2010).
  * Lundh  _et al._ (2011) O. Lundh, J. Lim, C. Rechatin, L. Ammoura, A. Ben-Ismaïl, X. Davoine, G. Gallot, J.-P. Goddet, E. Lefebvre, V. Malka, and J. Faure, Nat. Phys. 7, 219–222 (2011).
  * Mangles  _et al._ (2006) S. P. D. Mangles, A. G. R. Thomas, M. C. Kaluza, O. Lundh, F. Lindau, A. Persson, F. S. Tsung, Z. Najmudin, W. B. Mori, C.-G. Wahlström, and K. Krushelnick, Phys. Rev. Lett. 96, 215001 (2006).
  * Rehr and Albers (2000) J. J. Rehr and R. C. Albers, Rev. Mod. Phys. 72, 621–654 (2000).
  * Koningsberger and Prins (1988) D. C. Koningsberger and R. Prins,  _X-Ray Absorption: Principles, Applications, Techniques of EXAFS, SEXAFS and XANES_ (Wiley, 1988).
  * Dorchies and Recoules (2016) F. Dorchies and V. Recoules, Phys. Rep. 657, 1 (2016).
  * Syn (2018) Nature Reviews Materials 3, 281 (2018).
  * Bilderback  _et al._ (2005) D. H. Bilderback, P. Elleaume, and E. Weckert, Journal of Physics B: Atomic, Molecular and Optical Physics 38, S773 (2005).
  * Torchio  _et al._ (2016) R. Torchio, F. Occelli, O. Mathon, A. Sollier, E. Lescoute, L. Videau, T. Vinci, A. Benuzzi-Mounaix, J. Headspith, W. Helsby, S. Bland, D. Eakins, D. Chapman, S. Pascarelli, and P. Loubeyre, Scientific Reports 6, 26402 (2016).
  * Lemke  _et al._ (2013) H. T. Lemke, C. Bressler, L. X. Chen, D. M. Fritz, K. J. Gaffney, A. Galler, W. Gawelda, K. Haldrup, R. W. Hartsock, H. Ihee, J. Kim, K. H. Kim, J. H. Lee, M. M. Nielsen, A. B. Stickrath, W. Zhang, D. Zhu, and M. Cammarata, The Journal of Physical Chemistry A 117, 735 (2013), pMID: 23281652, https://doi.org/10.1021/jp312559h .
  * Bressler  _et al._ (2008) C. Bressler, R. Abela, and M. Chergui, Zeitschrift für Kristallographie - Crystalline Materials 223, 307 (2008).
  * Gawelda  _et al._ (2007) W. Gawelda, A. Cannizzo, V.-T. Pham, A. El Nahhas, C. J. Milne, R. van der Veen, C. Bressler, and M. Chergui, CHIMIA 61, 179 (2007).
  * Chen (2001) L. X. Chen, Journal of Electron Spectroscopy and Related Phenomena 119, 161 (2001), chemical Applications of Synchrotron Radiation.
  * Chen (2005) L. X. Chen, Annual Review of Physical Chemistry 56, 221 (2005), pMID: 15796701.
  * Bressler and Chergui (2010) C. Bressler and M. Chergui, Annual Review of Physical Chemistry 61, 263 (2010), pMID: 20055677.
  * Jonas  _et al._ (1996) D. M. Jonas, M. J. Lang, Y. Nagasawa, S. E. Bradforth, S. N. Dikshit, R. Jimenez, T. Joo, and G. R. Fleming, in  _The Reaction Center of Photosynthetic Bacteria_ , edited by M.-E. Michel-Beyerle (Springer Berlin Heidelberg, Berlin, Heidelberg, 1996) pp. 187–198.
  * van Grondelle  _et al._ (1994) R. van Grondelle, J. P. Dekker, T. Gillbro, and V. Sundstrom, Biochimica et Biophysica Acta (BBA) - Bioenergetics 1187, 1 (1994).
  * Arcovito  _et al._ (2005) A. Arcovito, D. Lamb, G. Nienhaus, J. Hazemann, M. Benfatto, and S. D. Longa, Biophysical Journal 88, 2954 (2005).
  * Uemura  _et al._ (2017) Y. Uemura, D. Kido, A. Koide, Y. Wakisaka, Y. Niwa, S. Nozawa, K. Ichiyanagi, R. Fukaya, S.-i. Adachi, T. Katayama, T. Togashi, S. Owada, M. Yabashi, K. Hatada, A. Iwase, A. Kudo, S. Takakusagi, T. Yokoyama, and K. Asakura, Chem. Commun. 53, 7314 (2017).
  * Hu  _et al._ (2021) Y. Hu, C. Gao, and Y. Xiong, Solar RRL 5, 2000468 (2021), https://onlinelibrary.wiley.com/doi/pdf/10.1002/solr.202000468 .
  * Park  _et al._ (2022) S. H. Park, A. Katoch, K. H. Chae, S. Gautam, P. Miedema, S. W. Cho, M. Kim, R.-P. Wang, M. Lazemi, F. de Groot, and S. Kwon, Nature Communications 13, 2531 (2022).
  * Hirohata  _et al._ (2020) A. Hirohata, K. Yamada, Y. Nakatani, I.-L. Prejbeanu, B. Diény, P. Pirro, and B. Hillebrands, Journal of Magnetism and Magnetic Materials 509, 166711 (2020).
  * Li  _et al._ (2019) J. Li, R. Wang, H. Guo, Y. Zhu, Y. Cao, J. Liu, H. Ding, H. Wen, and X. Liu, Journal of Physics: Condensed Matter 31, 255801 (2019).
  * Johnson  _et al._ (2015) J. A. Johnson, T. Kubacka, M. C. Hoffmann, C. Vicario, S. de Jong, P. Beaud, S. Grübel, S.-W. Huang, L. Huber, Y. W. Windsor, E. M. Bothschafter, L. Rettig, M. Ramakrishnan, A. Alberca, L. Patthey, Y.-D. Chuang, J. J. Turner, G. L. Dakovski, W.-S. Lee, M. P. Minitti, W. Schlotter, R. G. Moore, C. P. Hauri, S. M. Koohpayeh, V. Scagnoli, G. Ingold, S. L. Johnson, and U. Staub, Phys. Rev. B 92, 184429 (2015).
  * Strungaru  _et al._ (2022) M. Strungaru, M. Augustin, and E. J. G. Santos, npj Computational Materials 8, 169 (2022).
  * Rousse  _et al._ (2001) A. Rousse, C. Rischel, S. Fourmaux, I. Uschmann, S. Sebban, G. Grillon, P. Balcou, E. Förster, J. P. Geindre, P. Audebert, J. C. Gauthier, and D. Hulin, Nature 410, 65 (2001).
  * Ernstorfer  _et al._ (2009) R. Ernstorfer, M. Harb, C. T. Hebeisen, G. Sciaini, T. Dartigalongue, and R. J. D. Miller, Science 323, 1033 (2009), https://www.science.org/doi/pdf/10.1126/science.1162697 .
  * Mahieu  _et al._ (2018) B. Mahieu, N. Jourdain, K. T. Phuoc, F. Dorchies, J.-P. Goddet, A. Lifschitz, P. Renaudin, and L. Lecherbourg, Nat. Commun. 9 (2018), 10.1038/s41467-018-05791-4.
  * Koenig  _et al._ (2004) M. Koenig, E. Henry, G. Huser, A. Benuzzi-Mounaix, B. Faral, E. Martinolli, S. Lepape, T. Vinci, D. Batani, M. Tomasini, B. Telaro, P. Loubeyre, T. Hall, P. Celliers, G. Collins, L. DaSilva, R. Cauble, D. Hicks, D. Bradley, A. MacKinnon, P. Patel, J. Eggert, J. Pasley, O. Willi, D. Neely, M. Notley, C. Danson, M. Borghesi, L. Romagnani, T. Boehly, and K. Lee, Nuclear Fusion 44, S208 (2004).
  * Betti and Hurricane (2016) R. Betti and O. A. Hurricane, Nature Physics 12, 435 (2016).
  * Yaakobi  _et al._ (2004) B. Yaakobi, D. D. Meyerhofer, T. R. Boehly, J. J. Rehr, B. A. Remington, P. G. Allen, S. M. Pollaine, and R. C. Albers, Phys. Rev. Lett. 92, 095504 (2004).
  * Ping  _et al._ (2013) Y. Ping, F. Coppari, D. G. Hicks, B. Yaakobi, D. E. Fratanduono, S. Hamel, J. H. Eggert, J. R. Rygg, R. F. Smith, D. C. Swift, _et al._ , Phys. Rev. Lett. 111, 065501 (2013).
  * Ta Phuoc  _et al._ (2007) K. Ta Phuoc, R. Fitour, A. Tafzi, T. Garl, N. Artemiev, R. Shah, F. Albert, D. Boschetto, A. Rousse, _et al._ , Physics of Plasmas 14, 080701 (2007).
  * Kettle  _et al._ (2019) B. Kettle, E. Gerstmayr, M. J. V. Streeter, F. Albert, R. A. Baggott, N. Bourgeois, J. M. Cole, S. Dann, K. Falk, I. Gallardo González, A. E. Hussein, N. Lemos, N. C. Lopes, O. Lundh, Y. Ma, S. J. Rose, C. Spindloe, D. R. Symes, M. Šmíd, A. G. R. Thomas, R. Watt, and S. P. D. Mangles, Phys. Rev. Lett. 123, 254801 (2019).
  * Our (2023) https://doi.org/10.5281/zenodo.7876186 (2023), ultrafast X-ray absorption spectroscopy using a laboratory-scale laser-plasma accelerator [Data set].
  * Pak  _et al._ (2010) A. Pak, K. A. Marsh, S. F. Martins, W. Lu, W. B. Mori, and C. Joshi, Phys. Rev. Lett. 104, 025003 (2010).
  * McGuffey  _et al._ (2010) C. McGuffey, A. G. R. Thomas, W. Schumaker, T. Matsuoka, V. Chvykov, F. J. Dollar, G. Kalintchenko, V. Yanovsky, A. Maksimchuk, K. Krushelnick, _et al._ , Phys. Rev. Lett. 104, 025004 (2010).
  * Rousse  _et al._ (2004) A. Rousse, K. T. Phuoc, R. Shah, A. Pukhov, E. Lefebvre, V. Malka, S. Kiselev, F. Burgy, J.-P. Rousseau, D. Umstadter, _et al._ , Phys. Rev. Lett. 93, 135005 (2004).
  * Kneip  _et al._ (2008) S. Kneip, S. R. Nagel, C. Bellei, N. Bourgeois, A. E. Dangor, A. Gopal, R. Heathcote, S. P. D. Mangles, J. R. Marquès, A. Maksimchuk, P. M. Nilson, K. T. Phuoc, S. Reed, M. Tzoufras, F. S. Tsung, L. Willingale, W. B. Mori, A. Rousse, K. Krushelnick, and Z. Najmudin, Phys. Rev. Lett. 100, 105006 (2008).
  * King  _et al._ (2019) P. M. King, N. Lemos, J. L. Shaw, A. L. Milder, K. A. Marsh, A. Pak, B. M. Hegelich, P. Michel, J. Moody, C. Joshi, and F. Albert, Review of Scientific Instruments 90, 033503 (2019).
  * Cook and Sayers (1981) J. W. Cook and D. E. Sayers, Journal of Applied Physics 52, 5024 (1981), https://doi.org/10.1063/1.329444 .
  * Šmíd  _et al._ (2021) M. Šmíd, X. Pan, and K. Falk, Computer Physics Communications 262, 107811 (2021).
  * Zabinsky  _et al._ (1995) S. I. Zabinsky, J. J. Rehr, A. Ankudinov, R. C. Albers, and M. J. Eller, Phys. Rev. B 52, 2995 (1995).
  * Thomas (2010) A. G. R. Thomas, Physics of Plasmas 17, 056708 (2010), https://doi.org/10.1063/1.3368678 .
  * Bloom  _et al._ (2020) M. S. Bloom, M. J. V. Streeter, S. Kneip, R. A. Bendoyro, O. Cheklov, J. M. Cole, A. Döpp, C. J. Hooker, J. Holloway, J. Jiang, N. C. Lopes, H. Nakamura, P. A. Norreys, P. P. Rajeev, D. R. Symes, J. Schreiber, J. C. Wood, M. Wing, Z. Najmudin, and S. P. D. Mangles, Phys. Rev. Accel. Beams 23, 061301 (2020).
  * Rakowski  _et al._ (2022) R. Rakowski, P. Zhang, K. Jensen, B. Kettle, T. Kawamoto, S. Banerjee, C. Fruhling, G. Golovin, D. Haden, M. S. Robinson, D. Umstadter, B. A. Shadwick, and M. Fuchs, Scientific Reports 12, 10855 (2022).
  * Kozlova  _et al._ (2020) M. Kozlova, I. Andriyash, J. Gautier, S. Sebban, S. Smartsev, N. Jourdain, U. Chaulagain, Y. Azamoum, A. Tafzi, J.-P. Goddet, K. Oubrerie, C. Thaury, A. Rousse, and K. Ta Phuoc, Phys. Rev. X 10, 011061 (2020).
  * Ta Phuoc  _et al._ (2008) K. Ta Phuoc, E. Esarey, V. Leurent, E. Cormier-Michel, C. G. R. Geddes, C. B. Schroeder, A. Rousse, and W. P. Leemans, Physics of Plasmas 15, 063102 (2008), https://doi.org/10.1063/1.2918657 .
  * Kneip  _et al._ (2012) S. Kneip, Z. Najmudin, and A. G. Thomas, High Energy Density Physics 8, 133 (2012).
  * Hojbota  _et al._ (2022) C. I. Hojbota, M. Mirzaie, D. Y. Kim, T. G. Pak, M. Rezaei-Pandari, V. B. Pathak, J. H. Jeon, J. W. Yoon, J. H. Sung, S. K. Lee, C. M. Kim, K. Y. Kim, and C. H. Nam, “High-energy betatron source driven by a 4-pw laser with applications to non-destructive imaging,”  (2022).
  * Lind  _et al._ (2007) O. Lind, B. Salbu, K. Janssens, K. Proost, M. García-León, and R. García-Tenorio, Science of The Total Environment 376, 294 (2007).
  * Döpp  _et al._ (2017) A. Döpp, B. Mahieu, A. Lifschitz, C. Thaury, A. Doche, E. Guillaume, G. Grittani, O. Lundh, M. Hansson, J. Gautier, M. Kozlova, J. P. Goddet, P. Rousseau, A. Tafzi, V. Malka, A. Rousse, S. Corde, and K. Ta Phuoc, Light: Science & Applications 6, e17086 (2017).
  * Ferran Pousa  _et al._ (2022) A. Ferran Pousa, I. Agapov, S. A. Antipov, R. W. Assmann, R. Brinkmann, S. Jalas, M. Kirchen, W. P. Leemans, A. R. Maier, A. Martinez de la Ossa, J. Osterhoff, and M. Thévenet, Phys. Rev. Lett. 129, 094801 (2022).
  * Shalloo  _et al._ (2020) R. J. Shalloo, S. J. D. Dann, J.-N. Gruse, C. I. D. Underwood, A. F. Antoine, C. Arran, M. Backhouse, C. D. Baird, M. D. Balcazar, N. Bourgeois, J. A. Cardarelli, P. Hatfield, J. Kang, K. Krushelnick, S. P. D. Mangles, C. D. Murphy, N. Lu, J. Osterhoff, K. Põder, P. P. Rajeev, C. P. Ridgers, S. Rozario, M. P. Selwood, A. J. Shahani, D. R. Symes, A. G. R. Thomas, C. Thornton, Z. Najmudin, and M. J. V. Streeter, Nature Communications 11, 6355 (2020).
  * Ye  _et al._ (2022) H. Ye, Y. Gu, X. Zhang, S. Wang, F. Tan, J. Zhang, Y. Yang, Y. Yan, Y. Wu, W. Huang, and W. Zhou, Results in Physics 43, 106116 (2022).
  * Fernández-García (2002) M. Fernández-García, Catalysis Reviews 44, 59 (2002), https://doi.org/10.1081/CR-120001459 .
  * Jahrman  _et al._ (2019) E. P. Jahrman, L. A. Pellerin, A. S. Ditter, L. R. Bradshaw, T. T. Fister, B. J. Polzin, S. E. Trask, A. R. Dunlop, and G. T. Seidler, Journal of The Electrochemical Society 166, A2549 (2019).
  * Preger  _et al._ (2020) Y. Preger, H. M. Barkholtz, A. Fresquez, D. L. Campbell, B. W. Juba, J. Romàn-Kustas, S. R. Ferreira, and B. Chalamala, Journal of The Electrochemical Society 167, 120532 (2020).
  * Bi  _et al._ (2019) W. Bi, E. Jahrman, G. Seidler, J. Wang, G. Gao, G. Wu, M. Atif, M. AlSalhi, and G. Cao, ACS Applied Materials & Interfaces 11, 16647 (2019), https://doi.org/10.1021/acsami.9b03830 .
  * Wang  _et al._ (2016) J. Wang, Y.-c. Karen Chen-Wiegart, C. Eng, Q. Shen, and J. Wang, Nature Communications 7, 12372 (2016).
  * Nowack  _et al._ (2016) L. Nowack, D. Grolimund, V. Samson, F. Marone, and V. Wood, Scientific Reports 6, 21479 (2016).
  * Zhong  _et al._ (2021) M.-X. Zhong, B. Walkley, D. J. Bailey, L. R. Blackburn, H. Ding, S.-Q. Wang, W.-C. Bao, L. J. Gardner, S.-K. Sun, M. C. Stennett, C. L. Corkhill, and N. C. Hyatt, Journal of Nuclear Materials 556, 153198 (2021).
  * Mottram  _et al._ (2020) L. M. Mottram, M. C. Dixon Wilkins, L. R. Blackburn, T. Oulton, M. C. Stennett, S. K. Sun, C. L. Corkhill, and N. C. Hyatt, MRS Advances 5, 27 (2020).