# Progressive Neural Networks for Transfer Learning in Emotion Recognition

###### Abstract

Many paralinguistic tasks are closely related and thus representations learned
in one domain can be leveraged for another. In this paper, we investigate how
knowledge can be transferred between three paralinguistic tasks: speaker,
emotion, and gender recognition. Further, we extend this problem to cross-
dataset tasks, asking how knowledge captured in one emotion dataset can be
transferred to another. We focus on progressive neural networks and compare
these networks to the conventional deep learning method of pre-training and
fine-tuning. Progressive neural networks provide a way to transfer knowledge
and avoid the forgetting effect present when pre-training neural networks on
different tasks. Our experiments demonstrate that: (1) emotion recognition can
benefit from using representations originally learned for different
paralinguistic tasks and (2) transfer learning can effectively leverage
additional datasets to improve the performance of emotion recognition systems.

Index Terms: neural networks, transfer learning, progressive neural networks,
computational paralinguistics, emotion recognition

00footnotetext: ∗These authors contributed equally to this work

##  1 Introduction

Automatic emotion recognition has been actively explored by researchers for
the past few decades. However, compared with other tasks such as automatic
speech recognition (ASR) or speaker verification, the sizes of emotion
datasets are relatively small. This makes it difficult to create models that
generalize beyond the recording conditions and subject demographics of a
particular dataset. One possible approach to alleviate this problem is to
incorporate related knowledge that can help in learning a better system. Many
paralinguistic tasks are closely related and dependent on one another. As a
result, we hypothesize that emotion recognition models can be augmented with
additional information from other paralinguistic information, such as speaker
ID and gender, to improve classification performance. In this paper, we
explore transfer learning both as a method to leverage knowledge from other
paralinguistic tasks and to augment an emotion model with another model
trained on a different emotion dataset.

Previous work demonstrated that multi-task learning techniques (MTL) could be
used to jointly model both gender and emotion [1, 2, 3], resulting in
consistent performance increases compared to gender-agnostic models. Previous
work also showed that emotion recognition systems could be improved by
incorporating speaker identity as a feature, along with the other emotion-
related features [4]. It has also been demonstrated that some features (e.g.,
pitch) are not only valuable for predicting emotions, but are also effective
for detecting weight, height, gender, etc. [5]. Zhang et al. [6] explored MTL
frameworks for leveraging data from different domains (speech and song) and
gender in emotion recognition systems. Xia et al. [7] treated dimensional and
categorical labels as two different tasks.

MTL is best suited to situations where it is possible to train with all data
from scratch. However, in some cases an existing model needs to be adapted to
a new situation. Transfer learning provides a framework from which to address
this problem. The most common approach to transfer learning is to train a
model in one domain and fine-tune it in a related domain [8, 9, 10]. This pre-
training and fine-tuning (PT/FT) approach has been successful in cases where
the size of available data for the source domain is abundant in comparison to
size of the data in the target domain. For example, in ASR, transfer learning
is used to transfer knowledge from a richly-resourced language to an under-
resourced language [8]. In emotion recognition, Deng et al. [9] presented a
sparse auto-encoder method for transferring knowledge between six emotion
recognition datasets. The authors used auto-encoders to transform a source
domain and its features to a domain that is more consistent with the target
domain. The authors then used the transformed source domain to learn SVM
classifiers to predict emotions in the target domain. Ng et al. [10] studied
transfer learning in the context of facial expression recognition. The authors
first trained a model on a general large-scale dataset (ImageNet) that
contains $1.2$ million images. They then fine-tuned the trained model on four
datasets that contain relevant facial expressions similar to those in the
target domain. Finally, the authors fine-tuned the network using the target
data. The authors found that the two-step fine tuning approach provided
improvements over one-step fine-tuning and no fine-tuning.

However, the PT/FT approach has a number of limitations. First, it is unclear
how to initialize a model given learned weights from a sequence of related
tasks [11]. Second, when a model is fine-tuned using initial weights learned
from a source task, the end-model loses its ability to solve the source task,
a phenomenon termed the “forgetting effect” [12]. Finally, transferring
learned parameters between networks can be challenging if the networks have
inconsistent architectures. In this work, we use the PT/FT approach as our
baseline for transfer learning.

Another recent approach for transfer learning is progressive neural networks
(ProgNets) [11, 13]. ProgNets train sequences of tasks by freezing the
previously trained tasks and using their intermediate representations as
inputs into the new network. This allows ProgNets to overcome the above-
mentioned limitations associated with the traditional method of PT/FT,
including the challenge of initializing a model from a sequence of models, at
the expense of added parameters. Additionally, it prevents the forgetting
effect present in the PT/FT methods by freezing and preserving the source task
weights.

In this work, we investigate transfer learning between three paralinguistic
tasks: emotion, speaker, and gender recognition, with a focus on emotion
recognition as the target domain. In addition, we investigate the efficacy of
transfer learning applied between two emotion datasets: IEMOCAP [14] and MSP-
IMPROV [15]. Finally, we study the effect of transfer learning between
datasets when the target task has limited amount of data available. In all
cases, we investigate three methods: (1) deep neural network (DNN); (2) DNN
with PT/FT; and (3) progressive networks. Our results demonstrate significant
improvements over the conventional PT/FT methods when using ProgNets for
transferring knowledge from speaker recognition to emotion recognition tasks.
Furthermore, our results suggest that ProgNets show promise as a method for
transferring knowledge from gender detection to emotion recognition tasks, as
well as transferring knowledge across datasets, with results significantly
better than a DNN without transfer learning.

##  2 Datasets and Features

###  2.1 Datasets

We use speech utterances from two datasets in our study: IEMOCAP [14] and MSP-
IMPROV [15], two of the most commonly used datasets in emotion classification
[16, 17]. Both datasets were collected to simulate natural dyadic interactions
between actors and have similar labeling schemes. We use utterances with
majority agreement ground-truth labels. We only consider utterances with
happy, sad, angry, and neutral labels.

IEMOCAP: The IEMOCAP dataset contains utterances from ten speakers grouped
into five sessions. Each session contains one male and one female speaker. We
combine excitement and happiness utterances to form the happy category, as in
[14]. The final dataset contains 5531 utterances (1103 angry, 1708 neutral,
1084 sad, 1636 happy).

MSP-IMPROV: The MSP-IMPROV dataset contains utterances from 12 speakers
grouped into six sessions. A session has one male and one female. The final
dataset contains 7798 utterances (792 angry, 3477 neutral, 885 sad, 2644
happy).

###  2.2 Features

We use the eGeMAPS [18] feature set designed to standardize features used in
affective computing. The eGeMAPS feature set contains a total of $88$
features, including frequency, energy, spectral, cepstral, and dynamic
information. The final feature vectors for each utterance are obtained by
applying the following statistics: mean, coefficient of variation, 20-th,
50-th, and 80-th percentile, range of 20-th to 80-th percentile, mean and
standard deviation of the slope of rising/falling signal parts, mean of the
Alpha Ratio, the Hammarberg Index, and the spectral slopes from $0$–$500$Hz
and $500$–$1500$Hz. We perform dataset-specific global $z$-normalization on
all features.

Figure 1: Deep Neural Network (DNN) used in our experiments. The arrows
represent dense connections between each layer. The number of outputs (N)
varies depending on the experiment. Figure 2: Progressive Neural Network
(ProgNet) used in our experiments. The arrows represent dense connections
between each layer. The black arrows show frozen weights from the transferred
representations. The number of outputs (N) varies depending on the experiment.

##  3 Methods

We compare three methods in the context of transfer learning. As a baseline
method, we consider the performance of a DNN trained on the target task
without any extra knowledge. Additionally, we use the common transfer learning
approach of pre-training a DNN on the source task and fine-tuning on the
target task (PT/FT). The underlying assumption of PT/FT is that the target
model can leverage prior knowledge present in the source task. This approach
has been effective in many applications, including ASR [8] and natural
language processing [19].

Both these methods are compared to the recently introduced progressive neural
networks (ProgNets) [11]. Instead of using learned parameters as a starting
point for training a model on a target task, ProgNets do the following: (1)
freeze all parameters of the old model; (2) add a new model that is
initialized randomly; (3) add connections between the old (frozen) model and
the new model; (4) learn parameters of the new model using backpropagation.
ProgNets do not disrupt the learned information in existing source tasks,
which avoids the forgetting effect present in PT/FT [11].

In the construction of ProgNets, it is important to carefully select a method
for combining representations across network and to identify where these
representations will be combined. Adaptation layers can be included to
transform from one task’s representation to another. However, due to the small
amount of data available for training, we use ProgNets without adaptation
layers. For the same reason, we simplify the network by using an equal number
of layers in each column and transfer the representations between neighboring
layers in a one-to-one fashion: the representations produced at layer $k$ from
the frozen column is fed as an input to layer $k+1$ of the new column (Figure
2).

Table 1 shows the neural network parameters used by all experiments in this
paper. These values were selected using a standard DNN without transfer
learning to determine the best structure suited to our data (Figure 1).

(a) IEMOCAP

(b) MSP-IMPROV

Figure 3: The learning curves of different methods when transferring
representations from speaker to emotion. The regions around each curve show
the standard deviation of the UARs found by averaging across the folds of each
iteration.

We report unweighted average recall (UAR) as our comparison measure. UAR is an
unweighted accuracy that gives the same weights to different classes and is a
popular metric for emotion recognition, used to account for unbalanced
datasets [20]. We evaluate the performance of the methods using a repeated
ten-fold cross-validation scheme, as used in [21]. The folds are stratified
based on speaker ID. In each step of cross-validation, one fold is used for
testing, another is reserved for early stopping, and the remaining eight folds
are used for training. We repeat this evaluation scheme ten times, resulting
in ten UARs for each iteration. We calculate the mean and standard deviation
UAR within folds and report the mean of these statistics over all iterations.
We perform significance tests using a repeated cross-validation paired
$t$-test with ten degrees of freedom, as shown in [21], and note significance
when $p<0.05$.

Table 1: The hyper-parameters used in our experiments. Hyper-parameter | Value
---|---
number of layers | 4
layers width | 256
hidden activation function | sigmoid
output activation function | softmax
dropout rate | 0.5
learning rate | 0.0005
maximum number of epochs | 600

##  4 Paralinguistic Experiments

###  4.1 Experimental Setup

In the first set of experiments, we investigate the effectiveness of
transferring knowledge from speaker or gender recognition to emotion
recognition using the three methods mentioned above. In this section, we first
report UAR of the systems on both IEMOCAP and MSP-IMPROV. We analyze the
learning curves to compare the convergence behaviors of the systems. Prior
work demonstrated that using the weights of a pre-trained model to initialize
a new model to be trained on a related task can increase convergence speed
[11].

###  4.2 Results

Table 2: Paralinguistic experimental results comparing different techniques for transferring knowledge from speaker/gender to emotion. Mean and standard deviation UARs are given for each method. A cross shows a result is significantly better than the other two methods for a given task, while an asterisk notes results significantly better than a standard DNN. The mean within-fold standard deviations are shown. Source | Method | IEMOCAP | MSP-IMPROV
---|---|---|---
N/A | DNN |  0.640$\pm$0.017 |  0.584$\pm$0.022
Speaker | PT/FT |  0.645$\pm$0.017 |  0.592$\pm$0.018
Speaker | ProgNet | 0.657$\pm$0.018∗† | 0.605$\pm$0.021∗†
Gender | PT/FT |  0.640$\pm$0.016 |  0.586$\pm$0.021
Gender | ProgNet |  0.642$\pm$0.018 | 0.593$\pm$0.022∗

(a) Source: MSP-IMPROV; Target: IEMOCAP

(b) Source: IEMOCAP; Target: MSP-IMPROV

Figure 4: Cross-dataset experimental results under different amounts of
training folds used (out of 8 total available training folds). Each test is
run for ten iterations with different random folds to control for variations
in selected data. All experiments use emotion as the source and target label.
The regions around each curve show the standard deviation of the UARs found by
averaging across the folds of each iteration. Circles mark results that are
statistically significantly better than DNN.

Table 2 summarizes the results obtained from speaker-emotion and gender-
emotion transfer learning. When transferring from speaker recognition to
emotion recognition, ProgNets significantly outperform both standard DNN
($p=2.6$E-3) and PT/FT ($p=2.0$E-2) for IEMOCAP and both standard DNN
($p=8.8$E-4) and PT/FT ($p=1.2$E-2) for MSP-IMPROV. The PT/FT system slightly
outperforms the standard DNN, but the improvement is not significant. This
suggests that ProgNets can efficiently incorporate representations learned by
speaker recognition systems into emotion recognition ones, but PT/FT cannot
leverage this knowledge as effectively.

The performance of transferring gender information using ProgNets is not
consistent between IEMOCAP and MSP-IMPROV. ProgNets significantly ($p=$
1.2E-2) outperform the standard DNN when transferring knowledge from gender
recognition to emotion recognition in the case of MSP-IMPROV, but not IEMOCAP.
We hypothesize that this is due to the stronger gender recognition performance
on MSP-IMPROV. Gender recognition UAR on MSP-IMPROV and IEMOCAP are $98.1\%$
and $93.1\%$, respectively. For both datasets, PT/FT is not effective at
transferring gender information and performs no better than the standard DNN.

Figure 3 shows learning curves of the three reported systems for the specific
case of transferring speaker knowledge to an emotion detection system. The
figure shows that the PT/FT system reaches its best solution faster than the
other two methods. The PT/FT system, however, achieves lower final performance
than that of ProgNets. The learning curves of both transfer learning systems
(i.e., PT/FT and ProgNet) start with a larger slope compared to the standard
DNN. This slope vanishes quickly in PT/FT (after approximately $150$ epochs),
but the slope for the ProgNet preserves a positive value up to approximately
$400$ epochs. We hypothesize that this vanishing slope is due to PT/FT’s
inability to effectively incorporate representations learned for solving the
source task.

##  5 Cross-Dataset Experiments

###  5.1 Experimental Setup

In this set of experiments, we explore transfer learning as a way to improve
emotion recognition using an existing emotion model. In this experiment, the
model is trained on the source dataset and is then adapted (PT/FT and ProgNet)
to the target dataset. The standard DNN is trained only on the target dataset.
We examine the impact of transfer learning when the target training data size
is small by using different subsets of the training folds: 8, 4, 2, and 1.
Previous work has shown that transferring knowledge from a large source data
set to a smaller target datasets can be beneficial [8]. The source model is
always trained using the full source dataset (all eight folds). We perform
transfer learning by first treating IEMOCAP as the source and MSP-IMPROV as
the target and we reverse the source/target designations.

###  5.2 Results

Figure 4 shows a summary of the results when transferring across different
corpora. ProgNet significantly outperforms the standard DNN when transferring
from MSP-IMPROV to IEMOCAP for training fold sizes of 1 ($p=2.0$E-3), 2
($p=1.1$E-2), and 4 ($p=1.6$E-2) and when transferring from IEMOCAP to MSP-
IMPROV for training fold sizes of 1 ($p=5.0$E-3), 2 ($p=3.2$E-2), and 8
($p=3.6$E-2). PT/FT only achieves significant improvement versus the standard
DNN baseline when transferring from MSP-IMPROV to IEMOCAP for training fold
sizes of 1 ($p=7.1$E-4) and 2 ($p=1.0$E-2).

Because ProgNet has a larger number of weights to transfer knowledge, it is
most beneficial when the target dataset is larger, compared with PT/FT. We
hypothesize that this is what causes PT/FT to perform better in cases of
smaller training fold amounts (1 and 2) on a smaller target dataset (IEMOCAP).
This indicates that in some cases of small data, PT/FT may still be the better
choice. However, in cases where the size of the target dataset is sufficient,
ProgNet can effectively utilize the previous task representation better than
PT/FT.

##  6 Conclusion

Transfer learning provides a method for using additional paralinguistic data,
such as speaker ID, as well as a technique for combining models trained on
different datasets. This paper demonstrates the usefulness of progressive
neural networks for this task. While pre-training a DNN on a source dataset
has been previously used for transferring knowledge between tasks, progressive
neural networks provide an alternative way of avoiding the forgetting effect
by allowing the network to retain representations learned for solving the
original task. ProgNets significantly outperformed the standard DNN and PT/FT
networks when transferring knowledge between speaker identity and emotion.
This suggests the utility of ProgNets. We also demonstrated that ProgNets can
provide significant improvements for gender to emotion transfer tasks and
dataset transfer tasks when compared to systems that do not utilize source
information.

In this work, we concentrated on transferring knowledge across paralinguistic
tasks and datasets. However, future work will explore the utility of
transferring knowledge across both simultaneously, even when the datasets have
different characteristics (e.g. telephone vs. high fidelity speech). Speech
and speaker recognition datasets tend to contain more training utterances than
emotion datasets and contain paralinguistic information, such as subject ID.
This data could be used to train a classifier for different types of speakers
(e.g., accent, demographics). This model could then be transferred to emotion
to account for person-specific variations tied to phenomena outside of
emotion. This would allow for the transfer of knowledge from multiple datasets
or paralinguistic tasks. Future work will explore these ideas in order to
better augment emotion classification with existing data.

##  7 Acknowledgement

This work was partially supported by IBM under the Sapphire project. We would
like to thank Dr. David Nahamoo and Dr. Lazaros Polymenakos, IBM Research,
Yorktown Heights, for their support.

## References

  * [1] D. Ververidis and C. Kotropoulos, “Automatic speech classification to five emotional states based on gender information,” in _Signal Processing Conference, 2004 12th European_. IEEE, 2004, pp. 341–344.
  * [2] C. M. Lee and S. S. Narayanan, “Toward detecting emotions in spoken dialogs,” _IEEE transactions on speech and audio processing_ , vol. 13, no. 2, pp. 293–303, 2005.
  * [3] T. Vogt and E. André, “Improving automatic emotion recognition from speech via gender differentiation,” in _Proc. Language Resources and Evaluation Conference (LREC 2006), Genoa_ , 2006.
  * [4] M. Sidorov, S. Ultes, and A. Schmitt, “Comparison of gender-and speaker-adaptive emotion recognition.” in _LREC_ , 2014, pp. 3476–3480.
  * [5] B. Schuller, S. Steidl, A. Batliner, F. Burkhardt, L. Devillers, C. MüLler, and S. Narayanan, “Paralinguistics in speech and language—state-of-the-art and the challenge,” _Computer Speech & Language_, vol. 27, no. 1, pp. 4–39, 2013.
  * [6] B. Zhang, E. M. Provost, and G. Essi, “Cross-corpus acoustic emotion recognition from singing and speaking: A multi-task learning approach,” in _Acoustics, Speech and Signal Processing (ICASSP), 2016 IEEE International Conference on_. IEEE, 2016, pp. 5805–5809.
  * [7] R. Xia and Y. Liu, “A multi-task learning framework for emotion recognition using 2d continuous space,” _IEEE Transactions on Affective Computing_ , 2015\.
  * [8] A. Das and M. Hasegawa-Johnson, “Cross-lingual transfer learning during supervised training in low resource scenarios.” in _INTERSPEECH_ , 2015, pp. 3531–3535.
  * [9] J. Deng, Z. Zhang, E. Marchi, and B. Schuller, “Sparse autoencoder-based feature transfer learning for speech emotion recognition,” in _Affective Computing and Intelligent Interaction (ACII), 2013 Humaine Association Conference on_. IEEE, 2013, pp. 511–516.
  * [10] H.-W. Ng, V. D. Nguyen, V. Vonikakis, and S. Winkler, “Deep learning for emotion recognition on small datasets using transfer learning,” in _Proceedings of the 2015 ACM on international conference on multimodal interaction_. ACM, 2015, pp. 443–449.
  * [11] A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, and R. Hadsell, “Progressive neural networks,” _arXiv preprint arXiv:1606.04671_ , 2016.
  * [12] S. E. Fahlman and C. Lebiere, “The cascade-correlation learning architecture,” 1990.
  * [13] A. A. Rusu, M. Vecerik, T. Rothörl, N. Heess, R. Pascanu, and R. Hadsell, “Sim-to-real robot learning from pixels with progressive nets,” _arXiv preprint arXiv:1610.04286_ , 2016.
  * [14] C. Busso, M. Bulut, C.-C. Lee, A. Kazemzadeh, E. Mower, S. Kim, J. N. Chang, S. Lee, and S. S. Narayanan, “Iemocap: Interactive emotional dyadic motion capture database,” _Language resources and evaluation_ , vol. 42, no. 4, p. 335, 2008.
  * [15] C. Busso, S. Parthasarathy, A. Burmania, M. Abdel-Wahab, N. Sadoughi, and E. M. Provost, “Msp-improv: An acted corpus of dyadic interactions to study emotion perception,” _IEEE Transactions on Affective Computing_ , 2016.
  * [16] Y. Kim and E. M. Provost, “Emotion spotting: Discovering regions of evidence in audio-visual emotion expressions,” in _Proceedings of the 18th ACM International Conference on Multimodal Interaction_. ACM, 2016, pp. 92–99.
  * [17] R. Lotfian and C. Busso, “Retrieving categorical emotions using a probabilistic framework to define preference learning samples,” in _Proc. Interspeech_ , 2016.
  * [18] F. Eyben, K. R. Scherer, B. W. Schuller, J. Sundberg, E. André, C. Busso, L. Y. Devillers, J. Epps, P. Laukka, S. S. Narayanan _et al._ , “The geneva minimalistic acoustic parameter set (gemaps) for voice research and affective computing,” _IEEE Transactions on Affective Computing_ , vol. 7, no. 2, pp. 190–202, 2016.
  * [19] L. Mou, Z. Meng, R. Yan, G. Li, Y. Xu, L. Zhang, and Z. Jin, “How transferable are neural networks in nlp applications?” _arXiv preprint arXiv:1603.06111_ , 2016.
  * [20] B. W. Schuller, S. Steidl, A. Batliner _et al._ , “The interspeech 2009 emotion challenge.” in _Interspeech_ , vol. 2009, 2009, pp. 312–315.
  * [21] R. R. Bouckaert and E. Frank, “Evaluating the replicability of significance tests for comparing learning algorithms,” in _Pacific-Asia Conference on Knowledge Discovery and Data Mining_. Springer, 2004, pp. 3–12.


# Fast implicit diffusive dark-field retrieval for single-exposure, single-
mask x-ray imaging

Mario A. Beltran1, David M. Paganin1, Michelle K. Croughan1, Kaye S. Morgan1

1 School of Physics and Astronomy, Monash University, Clayton, VIC 3800,
Australia
email<EMAIL_ADDRESS>
###### Abstract

Complementary to conventional and phase X-ray radiography, dark-field imaging
has become central in visualizing diffusive scattering signal due to the
spatially-unresolved texture within an object. To date most diffusive-dark-
field retrieval methods require either the acquisition of multiple images at
the cost of higher radiation dose or significant amounts of computational
memory and time. In this work, a simple method of X-ray diffusive dark-field
retrieval is presented, applicable to any single-mask imaging setup, with only
one exposure of the sample. The approach, which is based on a model of
geometric and diffusive reverse-flow conservation, is implicit and non-
iterative. This numerically fast methodology is applied to experimental X-ray
images acquired using both a random mask and a grid mask, giving high quality
reconstructions that are very stable in the presence of noise. The method
should be useful for high-speed imaging and/or imaging with low-flux sources.

_K_ eywords X-ray Imaging, Dark-field Imaging, Image Retrieval, Fokker-Planck,
Transport of Intensity

##  1 Introduction

Attenuation-based X-ray radiography is a powerful non-invasive technique to
study samples at sub-millimeter length scales. However, when the difference in
absorption of the transmitted beam by adjacent materials is small, the sample
features become difficult to visualize. Hence, absorption-based X-ray
radiography has limited applications, encountering difficulties when imaging
weakly absorbing specimens without the use of contrast enhancement agents [1].

In recent years, complementary phase (refraction) and diffusive dark-field
signals have also been acquired using X-ray phase-contrast imaging set-ups.
These include analyzer-based systems [2], grating interferometers [3, 4, 5,
6], edge-illumination set-ups [7, 8], single-mask imaging (grating or random)
[9, 10, 11, 12] and propagation-based imaging (PBI) [13, 14, 15]. Here we use
a single-mask imaging set-up, where the diffusive dark-field signal has been
shown to be successfully extracted [16, 17, 18, 19]. Such a dark-field signal
often stems from small-angle or ultra-small-angle X-ray scattering
(SAXS/USAXS) from unresolved texture within the object, revealing features
that would otherwise not be revealed in either attenuation or phase contrast
X-ray images. Therefore diffusive dark-field signal extraction has potential
use in multiple fields, such as biomedical studies, material science and
industrial inspection. Further to this, in recent years dark-field retrieval
methods have been extended to recover directional diffusive dark-field signal,
enabling the orientation of SAXS/USAXS-producing texture or fibers to also be
visualized [20, 21, 22, 23, 24, 25]. In the single-mask approach, a mask
patterns the X-ray intensity incident on the sample, then an image captured
some distance downstream of the sample reveals how it has distorted this
pattern. Local transverse shifts of the pattern reveal phase gradients
introduced by the sample and a local blurring of the pattern reveals micro-
structure in the sample.

Methods used to extract the diffusive dark-field signal from single-mask set-
ups usually require some kind of pixel-by-pixel analysis, which can lead to
large computation times. To improve spatial resolution, some approaches
acquire exposures at multiple translational mask positions [26, 19], at the
cost of higher radiation dose. Diffusive dark-field retrieval methods can be
classified into two categories, explicit or implicit. Explicit methods
directly track the distortions of the mask features, via a suitable form of
correlation analysis or iterative computational approach, to retrieve the
phase and diffusive dark-field signals [16, 27, 19, 18]. Implicit methods
indirectly track mask distortions, extracting the signals in a deterministic
manner via direct solution of a suitable partial differential equation. This
was proposed by Paganin $et$ $al$. [28] for phase imaging, founded on the
concept of local optical energy conservation and geometric flow to extract
phase information, assuming the sample to be fully transparent. This algorithm
was later extended by Pavlov $et$ $al$. [29] and Quénot $et$ $al$. [30] to
include sample attenuation effects. The idea was taken further by Pavlov $et$
$al$. [31, 32] and Alloo $et$ $al$. [33], to recover a diffusive dark-field
signal. As a starting point, their approach utilized the Fokker-Planck
generalization [34, 35] of the transport-of-intensity equation of paraxial
wave optics [36], adapted to random mask-based imaging. The dark-field
retrieval algorithm of Pavlov et al. and Alloo et al. requires a minimum of
two sample and two reference images, captured at different transverse
positions of the mask.

In this paper we present an implicit diffusive-dark-field reverse-flow
retrieval method that is also inspired by the Fokker-Planck equation. Our
method requires only a single exposure of the mask and a single exposure of
the mask-and-sample, greatly reducing the complexity and duration of the
acquisition process and drastically improving computation time. The
theoretical forward model and procedure is presented in Section 2, followed by
an experimental verification in Section 3. Section 4 discusses our key
findings, followed by some concluding remarks.

##  2 Theory

Consider Fig. 1, where paraxial forward-propagating X-rays illuminate a mask,
before passing through a sample and then traversing a distance $\Delta$ to the
planar surface of a position-sensitive detector. Let $\textup{I}_{R}$ be the
recorded intensity of the reference mask, i.e., the image taken in the absence
of the sample, as a function of Cartesian coordinates
$\textbf{r}=\left(x,y\right)$ in a plane perpendicular to the optical axis
$z$. The recorded image in the presence of both the sample and mask is denoted
by $\textup{I}_{S}$. Assume the sample to obey the projection approximation
[37] at sufficiently high X-ray energies. The presence of the sample will
distort the illumination pattern created by the reference X-ray mask. These
distortions are physically manifested as: (i) Geometric flow, where certain
features of the reference image are transversely displaced due to the
refraction of rays induced by the sample, in a manner that locally conserves
the total radiant exposure; (ii) Flux loss, where the brightness of the
reference image is diminished as a result of the attenuation of rays as they
traverse the sample; (iii) Flux-conservative diffusion, where some features of
the reference pattern are blurred out as a consequence of SAXS/USAXS or
unresolved multiple-scattering of the beam, often occurring when the rays
interact with micro-structures within the volume.

Figure 1: Schematic illustrating our single-mask imaging setup. Exposures of a
periodic grid mask are shown in (a) and (b), without and with a sample
present. Exposures of a piece of sandpaper are shown in (c) and (d), without
and with a sample present. The scale bar in (a) also applies to the images in
(b-d). The sample here consists of a shaped PMMA block, a PMMA tube containing
polystyrene microspheres and a rectangular wooden skewer.

We have the freedom to choose whether we consider $\textup{I}_{R}$ as evolving
into $\textup{I}_{S}$ due to the introduction of the sample, or the reverse-
flow case where $\textup{I}_{S}$ devolves into $\textup{I}_{R}$ due to the
removal of the sample. Physically, these two viewpoints are equivalent, but
the logical possibility exists that they are inequivalent from the perspective
of the inverse problem of retrieving flow, flux loss and diffusion. As we now
show, the reverse-flow perspective enables us to solve the inverse problem in
a surprisingly simple manner.

Treating the devolution-type distortions from $\textup{I}_{S}$ to
$\textup{I}_{R}$ in a reverse-evolution Fokker-Planck-type manner [34], we
write down a continuity-type equation incorporating terms that describe
geometric flow, flux loss and diffusion (cf. Refs. [28, 32, 29, 31]):

$\overset{{\textup{Flux}}}{\overbrace{\textup{I}_{S}-\textup{I}_{R}\textup{t}_{0}}}=\overset{{\textup{Diffusion}}}{\overbrace{\Delta^{2}\nabla_{\perp}^{2}\left(\mathfrak{D}\textup{I}_{S}\right)}}-\overset{{\textup{Flow}}}{\overbrace{\nabla_{\perp}\cdot\left(\textup{I}_{S}\textbf{S}_{0}\right)}}.$
(1)

Here, $\textup{I}_{S}$, $\textup{I}_{R}$, $\textup{t}_{0}$, $\mathfrak{D}$ and
$\textbf{S}_{0}$ are all dependent on the transverse coordinates r, with the
transverse gradient and Laplacian being denoted as
$\nabla_{\perp}=(\partial_{x},\partial_{y})$ and
$\nabla_{\perp}^{2}=\partial_{x}^{2}+\partial_{y}^{2}$, respectively. The
dimensionless transmission function $\textup{t}_{0}$ quantifies the
attenuation properties of the object, yielding a flux loss in the reference
pattern [32, 33]. $\mathfrak{D}$ is the dimensionless diffusive function [38,
39]

$\mathfrak{D}=\tfrac{1}{2}\theta_{S}^{2}F$ (2)

due to SAXS/USAXS which we wish to recover [34, 35], with $\theta_{S}$ being
the position-dependent blur-cone half-angle of the local SAXS/USAXS diffuse
scatter that arises due to the interaction of the X-rays with the spatially-
unresolved random microstructure in the sample, and $F$ being the position-
dependent proportion of the incident rays that are converted to diffuse
scatter. The vector function
$\textbf{S}_{0}=(\textup{S}_{0x},\textup{S}_{0y})$, denotes the transverse
displacements (geometric flow [28]) of the reference pattern upon introduction
of the sample, where each component has units of distance. Under coherent
illumination, $\textbf{S}_{0}$ is related to gradients in the X-ray wavefield
phase $\phi_{0}$ introduced by the sample, via [10]

$\textbf{S}_{0}=\frac{\Delta}{k}\nabla_{\perp}\phi_{0},$ (3)

where the wavenumber $k=2\pi/\lambda$ comes from the wavelength $\lambda$ of
the radiation. Note, also, that no particular functional form is assumed for
the previously-mentioned position-dependent blur cone associated with the
diffuse scatter from each transverse location in the sample [39].

From this point, we restrict ourselves to objects comprised of a single
material of possibly varying density, that is, materials whose position-
dependent complex refractive index $n=1-\delta+i\beta$ is such that the ratio
$\delta/\beta$ is everywhere the same, within the volume occupied by the
sample [40, 41]. Under this assumption, the object transmission
$\textup{t}_{0}$ can be recovered using the method of Pavlov $et$ $al$. [29],
which is based on the transport-of-intensity equation [36] (cf. the method of
Ref. [40]):

$\textup{t}_{0}\approx\mathcal{F}^{-1}\left(\frac{1}{1+\frac{\Delta\delta}{\mu}\left|\textbf{q}\right|^{2}}\mathcal{F}\left\\{\frac{\textup{I}_{S}}{\textup{I}_{R}}\right\\}\right).$
(4)

Here, $\mu=2k\beta$ is the object’s linear attenuation coefficient,
$\mathcal{F}$ denotes Fourier transformation with respect to r,
$\mathcal{F}^{-1}$ denotes the corresponding inverse transformation,
$\textbf{q}=\left(q_{x},q_{y}\right)$ are transverse Fourier space coordinates
reciprocal to r, and the utilized Fourier-transform convention and notation is

$\mathcal{F}[g\left(\textbf{r}\right)]\equiv\widetilde{g}\left(\textbf{q}\right)=\frac{1}{2\pi}\iint_{\infty}^{\infty}e^{-i\textbf{q}\cdot\textbf{r}}g\left(\textbf{r}\right)d\textbf{r},\\\
$ (5a)
$\\!\\!\\!\\!\\!\\!\\!\\!\mathcal{F}^{-1}[\widetilde{g}\left(\textbf{q}\right)]\equiv
g\left(\textbf{r}\right)=\frac{1}{2\pi}\iint_{\infty}^{\infty}e^{i\textbf{q}\cdot\textbf{r}}\widetilde{g}\left(\textbf{q}\right)d\textbf{q}.$
(5b)

Although the single-material assumption seems highly restrictive, it has been
demonstrated in numerous practical X-ray phase contrast studies that retrieval
methods work effectively for samples that stretch beyond this assumption, even
to samples containing multiple materials (see e.g. [42, 43, 44]). For high-
energy X-rays and low-atomic-number materials, there is a simple physical
reason why different elements may be considered as a single material for PBI
purposes. This single material is electrons, because the influence of atomic
nuclei may often be ignored in such a regime, with electron density dominating
the photon–matter interaction.

In Fig. 2(a) we show the recovered transmission, $\textup{t}_{0}$, found by
applying Eq. (4) to the $\textup{I}_{R}$ and $\textup{I}_{S}$ images shown in
Figs. 1(c) and 1(d) respectively. The experimental details describing how the
images were captured are discussed in Section 3, and we concentrate here on
describing the retrieval procedure. Once the sample transmission,
$\textup{t}_{0}$, has been obtained, we are now able to easily calculate the
“Flux” term in Eq. (1), with the resulting image shown in Fig. 2(d).

Figure 2: Processed images used as inputs to calculate the “Flux”, “Flow” and
“Diffusion”, with the particular sample discussed in Section 3. (a) Logarithm
of transmission, $-\textup{ln}[\textup{t}_{0}]$, recovered using Eq. (4). (b)
Derivative in the $x$-direction, $\partial_{x}\textup{ln}[\textup{t}_{0}]$,
calculated from (a). (c) Derivative in the $y$-direction,
$\partial_{y}\textup{ln}[\textup{t}_{0}]$, also calculated from (a). (d) Image
of calculated “Flux”. (e) Image of calculated “Flow”. (f) “Flux+Flow” image.
The scale bar in (c) applies to all panels.

Turning attention to the “Flow” term, in the case of a single-material sample
this can be re-expressed in terms of the transmission using the
phase–intensity relation
$\phi_{0}=\frac{k\delta}{\mu}\textup{ln}(\textup{t}_{0})$ [40], hence

$\displaystyle\overset{\textup{Flow}}{\overbrace{\frac{\Delta}{k}\nabla_{\perp}\cdot\left(\textup{I}_{S}\nabla_{\perp}\phi_{0}\right)}}$
$\displaystyle=$
$\displaystyle\frac{\Delta\delta}{\mu}\nabla_{\perp}\cdot\left(\textup{I}_{S}\nabla_{\perp}\textup{ln}[\textup{t}_{0}]\right)$
(6) $\displaystyle=$
$\displaystyle\frac{\Delta\delta}{\mu}\left\\{\partial_{x}\left(\textup{I}_{S}\partial_{x}\textup{ln}[\textup{t}_{0}]\right)+\partial_{y}\left(\textup{I}_{S}\partial_{y}\textup{ln}[\textup{t}_{0}]\right)\right\\}.$

The Fourier operators $\partial_{x}=\mathcal{F}^{-1}iq_{x}\mathcal{F}$ and
$\partial_{y}=\mathcal{F}^{-1}iq_{y}\mathcal{F}$ [37] can be used to compute
the transverse spatial derivatives. This gives the calculated derivatives in
the $x$\- and $y$-directions of the transmission in Fig. 2 (a), as shown in
Figs. 2(b) and 2(c), respectively. These are then subsequently used to compute
the “Flow” term seen in Fig. 2(e).

Having obtained the “Flux” and “Flow” we can simply add these to get a
“Flux+Flow” image (see Fig. 2(f)), and then apply an inverse Laplacian
operation to solve for the diffusive dark-field $\mathfrak{D}$. Hence

$\mathfrak{D}=\frac{\nabla_{\perp}^{-2}\left[{\textup{Flux}}+{\textup{Flow}}\right]}{\Delta^{2}\textup{I}_{S}}.$
(7)

The inverse Laplacian can be efficiently numerically implemented using

$\nabla^{-2}=-\mathcal{F}^{-1}\frac{1}{\left|\textbf{q}\right|^{2}+\varepsilon}\mathcal{F}.$
(8)

The added Tikhonov regularizing term $\varepsilon>0$ is small compared to
$\left|\textbf{q}\right|^{2}$, with the exception of the origin of Fourier
space, in order to avoid a division-by-zero singularity when
$(q_{x},q_{y})=(0,0)$.

##  3 Methods and results

To test the approach described in Section 2, X-ray images were collected in
hutch 3B of the Imaging and Medical Beamline (IMBL) at the Australian
Synchrotron. The setup of our experiment is the same as a typical single-mask
imaging technique, as illustrated in Fig. 1. A large source-to-object distance
(135 m) and double-crystal monochromator provided a near planar X-ray beam. We
used 25 keV X-rays to produce visible phase contrast fringes and diffusive
dark-field effects for these small samples. The beam size was collimated to
approximately 31.5 mm wide and 26.5 mm high, which was large enough to
illuminate each sample. A 25 $\upmu$m Gadox scintillator and lens-coupled
PCO.Edge camera, producing an effective pixel size of 12.3 $\upmu$m, was used
to collect the images.

The masks were positioned 0.5 m upstream from the object (as close as
possible) and the detector at a distance $\Delta=2$ m downstream from the
object. This relatively intermediate sample-to-detector propagation distance
was sufficient to render visible the blurring of the reference pattern from
the diffusive dark-field-producing parts of the sample. Each image was
acquired with an exposure time of 1 s. Flat field images (with no object in
the beam) and dark current images (with no incident X-rays) were recorded to
normalize the image intensity, correct for the beam non-uniformities and
correct for pixel-to-pixel variations in detector sensitivity.

For the periodic-mask experiments, an attenuating grid was used, in this case
a geological stainless steel sieve with holes of 90 $\upmu$m and wires 61
$\upmu$m thick, which attenuates 0-72% of the 25 keV X-rays. In the case of a
random mask, 3 sheets of sandpaper were used as an analyzer mask.

The first sample was (i) a plastic tube filled with polystyrene microspheres
of around 6 $\upmu$m in diameter and (ii) a wooden skewer, placed in front of
(iii) a smoothly undulating PMMA plastic slab of approximately 7 mm mean
thickness (see center of Fig. 1). For a second sample, a slice of lemon was
mounted with some wooden craft sticks (seen in the top-right of the images).

###  3.1 Diffusive dark-field imaging using a single random mask

In this section we demonstrate diffusive dark-field recovery using the
methodology described in Section 2. This is applied to images acquired using a
random mask (sandpaper) as a reference mask, namely the raw data displayed in
Figs. 1(c) and 1(d). For a beam of 25 keV X-rays, effective values of
$\delta=4.265\times 10^{7}$ and $\beta=1.53\times 10^{10}$, specific for
water, were used [1]. This is consistent with the $\delta/\beta$ ratio and
energies used in previous X-ray phase imaging studies of similar samples [43].
These values were obtained from the Center for X-Ray Optics (CXRO) website
[45]. A regularization value of $\varepsilon=1.5\times 10^{7}$ was used, which
is several orders in magnitude less than $\left|\textbf{q}\right|^{2}$ even
near the origin.

Figure 3: Speckle-based transmission and diffusive dark-field retrieval
results, for the sample shown in Fig. 1. (a) Image of the ratio
$\textup{I}_{S}/\textup{I}_{R}$. (b) Diffusive dark-field image recovered
using Eq. (7). The scale bar in (b) also applies to (a). The region outlined
with a white dashed line is shown magnified in the top right of each panel.
The dark-field image is shown with a linear grayscale from $\mathfrak{D}=0$ to
$\mathfrak{D}=6.0\times 10^{-11}$ ($\theta_{S}=11\upmu$rad), corresponding to
$F=1$ according to Eq. (2).

Figure 3(a) shows a PBI-like image of the test sample, in which the region of
the image behind the polystyrene micro-spheres yields a considerable level of
residual speckle originating from the dark-field effects on the reference
mask. We believe these artifacts arise where the term
$\textup{I}_{S}/\textup{I}_{R}$ in Eq. (4) does not entirely remove the mask
pattern, because the mask pattern has been substantially changed by the
presence of the sample in these regions. The associated recovered diffusive
dark-field image in Fig. 3(b) yields a strong signal in the regions where
micro-spheres are present, but no signal from the surrounding solid plastic
tube, indicating that this method is indeed highly sensitive to diffusive
effects induced by SAXS/USAXS due to micro-structure. Note, also, that: (i)
the stated value of $F=1$, as given in the caption to Fig. 3, is consistent
with the fact that the Fresnel number, associated with propagation from the
entrance surface to the exit surface of the sample, is much greater than unity
[39]; (ii) the recovered position-dependent blur angles $\theta_{S}$
correspond to microstructure-induced blur at the detector plane over a maximum
transverse distance on the order of twice the effective pixel size.

The horizontally-placed wooden skewer also reveals a strong signal in the
recovered dark-field. The inner structure of wood contains high levels of
porosity and fibers, known to induce diffusive effects and therefore producing
a strong diffusive dark-field signal. Interestingly, incorporating a wood
skewer undoubtedly violates the premise of the single-material sample
restriction assumed in deriving this method, yet we still obtain a usable
signal that is valuable for qualitative assessment.

###  3.2 Diffusive dark-field imaging using a single periodic grid

In this section we apply our diffusive dark-field retrieval method to the same
sample and setup, but now the random mask is replaced with a grid containing a
two-dimensional periodic structure. The raw images of (i) the mask and (ii)
the mask and sample are shown in Figs. 1(a-b), respectively. Note that all
parameters of the experiment other than the mask remain unchanged, hence the
same input values for $\delta$ and $\beta$ were used as in the previous
section.

Figure 4: Grid-based retrieved transmission and diffusive dark-field images,
for the sample shown in Fig. 1. (a) The PBI-like image obtained from the ratio
$\textup{I}_{S}/\textup{I}_{R}$. (b) The diffusive dark-field image recovered
using Eq. (7). (c) The Fourier transform of (b), shown as the logarithm of the
absolute value of the complex image, showing the grid artifacts as harmonic
peaks. (d) The retrieved diffusive dark-field image, where a Fourier filter
has removed the grid artifacts seen in (b). The scale bar in (b) also applies
to (a) and (d). The dark-field images are shown with a linear grayscale from
$\mathfrak{D}=0$ to $\mathfrak{D}=7.2\times 10^{-10}$ ($\theta_{S}=38$
$\upmu$rad), corresponding to $F=1$ according to Eq. (2).

Diffusive dark-field images retrieved from data collected with a grid-based
mask are shown in Fig. 4. Immediately one can notice residual periodic
artifacts from the grid in the transmission image (Fig. 4(a)) in the regions
where micro-structure is found, in the same way residual speckles are seen in
these regions when using the speckle mask. These artifacts propagate to some
degree to the recovered dark-field image (inset, Fig. 4(b)). However because
the differences are local, these high spatial frequencies are reduced by the
Fourier filter used in the transmission retrieval (Eq. (4)), and this does not
prevent the successful application of the algorithm. As seen in the previous
section, the retrieved diffusive dark-field image in Fig. 4(b) displays high
sensitivity to areas where local blurring takes place, from both the
polystyrene micro-spheres and the wooden fibers.

To remove the periodic artifacts we utilized an image processing method which
blocks the periodicity signatures in Fourier space, by locally setting them to
zero. These periodicity signatures are clearly visualized as the bright spots
in the magnitude of the Fourier transform of Fig. 4(b), shown here as Fig.
4(c). Once these harmonic peaks are blocked, using a computational form of the
technique of spatial filtering (see e.g. pp 642–647 of Ref. [46]), the image
can be inverse Fourier transformed back to real space. In this way, the
artifacts can be almost completely suppressed, as shown in Fig. 4(d).

###  3.3 Testing the stability under varying noise levels

Despite the crude assumptions made as a trade-off in order to reach
mathematical simplicity and computational efficiency, we note another
benefit—in addition to computational speed—namely the robustness of the method
with respect to noise. Accordingly, we tested the stability of the retrieval
algorithm on random-mask-based data of a lemon slice (see Fig. 5(a)). The
experimental parameters and setup were the same as in the previous two
sections. Poisson noise was added to the raw image $\textup{I}_{S}$ prior to
applying our retrieval approach. The increasing added noise in the raw images
is shown in Figs. 5(a-c) but is considerably more obvious in Figs. 5(d-f),
which are magnified regions, shown by the square boxed area in the first row
of the figure. Signal-to-noise ratios ($\textup{SNR}=\sqrt{N}$, with $N$
defined below) of the input images in the top row were calculated to be 107
for (a), 7.6 for (b) and 3.4 for (c), indicating an effective lower average
photon count per pixel with decreasing SNR as the added noise level is
increased. $N=11\,572$ is the mean gray value of the original image in (a),
which was varied as $N/200$ for (b) and $N/1000$ for (c). For the respective
added noise levels, diffusive dark-field reconstructions are shown in Figs.
5(g-i). Evidently, the higher the level of added noise, the poorer the
diffusive dark-field reconstruction, yet the SAXS/USAXS signal is still easily
distinguished in all cases. In terms of practicality, this extreme noise
stability can prove beneficial when it comes to radiation-dose considerations,
especially when dealing with biological specimens.

Figure 5: Raw random mask-based images $\textup{I}_{S}$ of a lemon sample are
shown with increasing levels of Poisson noise, where (b) has $7\%$ of the SNR
of (a), and (c) $3\%$ of the SNR of (a). Magnified regions of (a), (b) and (c)
are seen in (d), (e) and (f), respectively. Retrieved diffusive dark-field
images for the different noise levels are shown in (g), (h) and (i). The scale
bar in (a) applies to all images except for those in the middle row. The dark-
field images are shown with a linear grayscale from $\mathfrak{D}=0$ to
$\mathfrak{D}=4.2\times 10^{-11}$ ($\theta_{S}=9.1\upmu$rad), corresponding to
$F=1$ according to Eq. (2).

##  4 Discussion and conclusion

We have derived a fast implicit diffusive-dark-field retrieval algorithm for
data measured using a single position of a mask, which may be periodic or
random. The main advantage of single-mask imaging is that only one exposure of
the sample is required for the data acquisition process. This makes the
technique highly suitable for high-speed dynamic imaging, compared to the
other mask-based diffusive dark-field X-ray retrieval methods which require
multiple sample exposures. Note also that because the mask is stationary, the
retrieval algorithm should be robust to any motion blur due to a moving
sample, since this would not blur the mask and hence would not be interpreted
as dark-field.

The primary advantage of our method is its robustness in retrieving SAXS/USAXS
signal pertaining to sample unresolved micro-structure. Propagation-based
phase contrast effects are incorporated within this approach, and hence do not
appear as edge signal in the retrieved dark-field images. This means that the
resulting images extract the diffusive type scattering signal. Another benefit
of the approach is high stability under shot noise whilst only utilizing one
exposure. This makes it potentially useful for low-dose imaging using
laboratory-based sources that have considerably less photon flux than
synchrotron sources. We believe the robustness and sensitivity will be
maximized when using a high-visibility pattern, as reflected in Figs. 3 and 4,
where the grid pattern enables a lower-noise diffusive dark-field retrieval
than the lower-visibility speckle pattern does. The sensitivity of the set-up
can be tuned by adjusting the sample-to-detector distance, such that the
reference pattern is appreciably blurred, but not entirely blurred out. While
we have found this optimization to be most easily performed experimentally,
the propagation distance should not be so long that the linearization of the
Fokker-Planck equation is no longer valid (for detail, see Eq. (30) in Ref.
[39]). While there are reference-pattern remnants in areas of strong diffusive
dark-field in the “transmission” images in panel a) of Figs. 3 and 4, this
contrast is effectively the source of our dark-field contrast. If someone
wished to retrieve a transmission image without these remnants, the retrieved
dark-field image (e.g. Fig. 4(d)) could be used to regionally blur the
reference image, $\textup{I}_{R}$, before calculating the transmission
$\textup{t}_{0}$ using Eq. (4). Future work could potentially introduce an
iteration to the algorithm where the resulting refined transmission image
$\textup{t}_{0}$ is used to improve the estimate of the flow term (Eq. (6))
and hence potentially reduce the reference pattern remnants seen in the dark-
field images.

In terms of drawbacks of the approach, the assumption that the object is
composed of a single material clearly restricts the classes of specimen this
method can be applied to. Despite this limitation, we saw that even though the
samples used in this investigation were not entirely homogeneous, the
reconstruction showed a high sensitivity to the wood interfaces placed within
the main object, thus providing extremely useful visual information indicating
that these produce a high degree of diffusive signal. This opens up the idea
that this method could be extended to a wider class of samples (non-
homogeneous) where one could focus in on a particular material interface by
tuning the values of the refractive index corresponding to the material of
interest and at the same time still only using one exposure. This would be
analogous to what has been done in studies by Gureyev $et$ $al$. [42] in the
context of quantitative material-specific X-ray phase retrieval using PBI
[43]. In the case that the sample thickness retrieval step (Eq. (4)) over- or
under-smooths some material interfaces within a multi-material sample, we
expect any resulting artifacts in the dark-field image to simply highlight
edges. Further to this, another particular class of samples that would be
interesting to consider in future studies are those classified as pure phase
objects ($\textup{t}_{0}=1$). In this case estimation of the transmission
would not be needed and the single-material assumption could be dropped, with
sample thickness being retrieved from the phase shift by tracking speckles
[28]. Another possible avenue of research is to make use of the retrieved
SAXS/USAXS signal as an initial guess (a seed) to more computationally-
intensive iterative diffusive dark-field reconstruction approaches where no
restrictions on the sample are made. Studying the influence of the nonzero
distance, between the mask and the object, is another interesting avenue for
future work.

It is also worth commenting on the possible influence of long-range tails in
the scattering function, namely particularly slow transverse spatial decay
associated with the position-dependent SAXS/USAXS distribution functions. If
such long-range tails are present, then the first term on the right side of
Eq. (1) will need to be augmented with higher-order spatial derivatives. Our
position-dependent diffusion coefficient will then become the first member in
an infinite hierarchy of diffusion tensors, with our Fokker–Planck equation
thereby generalizing to an equation of the Kramers–Moyal form. The associated
higher-order diffusion coefficients are closely related to moments of the
local SAXS/USAXS distribution. For more information on these points, we refer
to our earlier papers on the application of the Fokker–Planck equation to
paraxial optics [34, 35], together with references contained therein.

Finally, we believe this method could be applied to data measured from single-
mask setups where the probing radiation is comprised of matter waves such as
neutrons and electrons. We base this statement on the fact that our
underpinning expression, namely Eq. (1), is in essence a statement of local
energy conservation under the simultaneous presence of both coherent and
diffusive flow, together with the presence of a sink term associated with a
dimensionless sample-transmission function. Such local energy conservation,
embodied in a suitable reverse-flow Fokker–Planck form of the continuity
equation, holds for a very broad class of matter and radiation fields.

## Acknowledgments

The authors are grateful for help provided by the beamline scientists, Chris
Hall, Daniel Hausermann, Anton Maksimenko and Matthew Cameron, at the Imaging
and Medical beamline at the Australian Synchrotron, part of ANSTO, where the
experiment was completed under proposal 18642. KM acknowledges support from
the Australian Research Council (FT18010037) and MC acknowledges funding from
the Australian Research Training Program (RTP).

## Disclosures

The authors declare no conflicts of interest.

## Data availability

The raw and diffusive-dark-field retrieved images presented in this paper can
be obtained from https://github.com/Xray-grill/DiDaFi_MKCroughan

## References

  * [1] Alberto Bravin, Paola Coan, and Pekka Suortti.  X-ray phase-contrast imaging: from pre-clinical applications towards clinics.  Physics in Medicine and Biology, 58:R1, 2013.
  * [2] E. Förster, K. Goetz, and P. Zaumseil.  Double crystal diffractometry for the characterization of targets for laser fusion experiments.  Kristall und Technik, 15:937–945, 1980.
  * [3] Atsushi Momose, Shinya Kawamoto, Ichiro Koyama, Yoshitaka Hamaishi, Kengo Takai, and Yoshio Suzuki.  Demonstration of x-ray Talbot interferometry.  Jpn J. Appl. Phys., 42(7B):L866, 2003.
  * [4] Timm Weitkamp, Ana Diaz, Christian David, Franz Pfeiffer, Marco Stampanoni, Peter Cloetens, and Eric Ziegler.  X-ray phase imaging with a grating interferometer.  Optics Express, 16:6296–6304, 2005.
  * [5] Franz Pfeiffer, Timm Weitkamp, Oliver Bunk, and Christian David.  Phase retrieval and differential phase-contrast imaging with low-brilliance x-ray sources.  Nature Physics, 2:258–261, 2006.
  * [6] Franz Pfeiffer, Martin Bech, Oliver Bunk, Philipp Kraft, Eric F. Eikenberry, Ch Brönnimann, Christian Grünzweig, and Christian David.  Hard-x-ray dark-field imaging using a grating interferometer.  Nature Materials, 7:134–137, 2008.
  * [7] Alessandro Olivo and Robert Speller.  A coded-aperture technique allowing x-ray phase contrast imaging with conventional sources.  Applied Physics Letters, 91:074106, 2007.
  * [8] Marco Endrizzi, Paul C. Diemoz, Thomas P. Millard, J. Louise Jones, Robert D. Speller, Ian K. Robinson, and Alessandro Olivo.  Hard x-ray dark-field imaging with incoherent sample illumination.  Applied Physics Letters, 104:024106, 2014.
  * [9] Harold H. Wen, Eric E. Bennett, Rael Kopace, Ashley F. Stein, and Vinay Pai.  Single-shot x-ray differential phase-contrast and diffraction imaging using two-dimensional transmission gratings.  Optics Letters, 35:1932–1934, 2010.
  * [10] Kaye S. Morgan, David M. Paganin, and Karen K. W. Siu.  Quantitative single-exposure x-ray phase contrast imaging using a single attenuation grid.  Optics Express, 19(20):19781–19789, 2011.
  * [11] Sébastien Bérujon, Eric Ziegler, Roberto Cerbino, and Luca Peverini.  Two-dimensional x-ray beam phase sensing.  Physical Review Letters, 108(15):158102, 2012.
  * [12] Kaye S. Morgan, David M. Paganin, and Karen K. W. Siu.  X-ray phase imaging with a paper analyzer.  Applied Physics Letters, 100:124102, 2012.
  * [13] Peter Cloetens, Raymond Barrett, José Baruchel, Jean-Pierre Guigay, and Michel Schlenker.  Phase objects in synchrotron radiation hard x-ray imaging.  Journal of Physics D: Applied Physics, 29:133–146, 1996.
  * [14] Anatoly Snigirev, Irina Snigireva, V. Kohn, S. Kuznetsov, and I. Schelokov.  On the possibilities of x-ray phase contrast microimaging by coherent high-energy synchrotron radiation.  Review of Scientific Instruments, 66:5486, 1995.
  * [15] S. W. Wilkins, T. E. Gureyev, D. Gao, A. Pogany, and A. W. Stevenson.  Phase-contrast imaging using polychromatic hard x-rays.  Nature, 384:335–338, 1996.
  * [16] Sebastien Berujon, Hongchang Wang, and Kawal Sawhney.  X-ray multimodal imaging using a random-phase object.  Physical Review A, 86:063813, 2012.
  * [17] Kaye S. Morgan, Peter Modregger, Sarah C. Irvine, Simon Rutishauser, Vitaliy A. Guzenko, Marco Stampanoni, and Christian David.  A sensitive x-ray phase contrast technique for rapid imaging using a single phase grid analyzer.  Optics Letters, 38:4605–4608, 2013.
  * [18] I. Zanette, T. Zhou, A. Burvall, U. Lundström, D.H. Larsson, M. Zdora, P. Thibault, F. Pfeiffer, and H.M. Hertz.  Speckle-based x-ray phase-contrast and dark-field imaging with a laboratory source.  Physical Review Letters, 112:253903, 2014.
  * [19] Marie Christine Zdora, Pierre Thibault, Tunhe Zhou, Frieder J. Koch, Jenny Romell, Simone Sala, Arndt Last, Christoph Rau, and Irene Zanette.  X-ray phase-contrast imaging and metrology through unified modulated pattern analysis.  Physical Review Letters, 118(3):203903, 2017.
  * [20] Torben H. Jensen, Martin Bech, Oliver Bunk, Tilman Donath, Christian David, Robert Feidenhans’l, and Franz Pfeiffer.  Directional x-ray dark-field imaging.  Physics in Medicine and Biology, 55:3317–23, 2010.
  * [21] Torben Haugaard Jensen, Martin Bech, Irene Zanette, Timm Weitkamp, Christian David, Hans Deyhle, Simon Rutishauser, Elena Reznikova, Jürgen Mohr, Robert Feidenhans’l, and Franz Pfeiffer.  Directional x-ray dark-field imaging of strongly ordered systems.  Phys. Rev. B, 82:214103, Dec 2010.
  * [22] Tunhe Zhou, Hongchang Wang, and Kawal Sawhney.  Single-shot x-ray dark-field imaging with omnidirectional sensitivity using random-pattern wavefront modulator.  Applied Physics Letters, 113:091102, 2018.
  * [23] Florian Schaff, Friedrich Prade, Yash Sharma, Martin Bech, and Franz Pfeiffer.  Non-iterative directional dark-field tomography.  Scientific Reports, 7:03307, 2017.
  * [24] Erik S Dreier, Chantal Silvestre, Jan Kehres, Daniel Turecek, Mohamad Khalil, Jens H Hemmingsen, Ole Hansen, Jan Jakubek, Robert Feidenhans’l, and Ulrik L Olsen.  Single-shot, omni-directional x-ray scattering imaging with a laboratory source and single-photon localization.  Optics Letters, 45(4):1021–1024, 2020.
  * [25] Ronan Smith, Fabio De Marco, Ludovic Broche, Marie-Christine Zdora, Nicholas W Phillips, Richard Boardman, and Pierre Thibault.  X-ray directional dark-field imaging using unified modulated pattern analysis.  PLoS One, 17(8):e0273315, 2022.
  * [26] Sebastien Berujon and Eric Ziegler.  Near-field speckle-scanning-based x-ray imaging.  Physical Review A, 92(1):013837, 2015.
  * [27] Ying Y. How and Kaye S. Morgan.  Quantifying the x-ray dark-field signal in single-grid imaging.  Optics Express, 30:10899–10918, 2022.
  * [28] David M. Paganin, Hélène Labriet, Emmanuel Brun, and Sebastien Berujon.  Single-image geometric-flow x-ray speckle tracking.  Physical Review A, 98:053813, 2018.
  * [29] Konstantin M. Pavlov, Heyang. Li, David M. Paganin, Sebastien Berujon, Helene Rouge-Labriet, and Emmanuel Brun.  Single-shot x-ray speckle-based imaging of a single-material object.  Physical Review Applied, 13(3):054023, 2020.
  * [30] Laurène Quénot, Helene Rougé-Labriet, Sylvain Bohic, Sebastien Berujon, and Emmanuel Brun.  Implicit tracking approach for x-ray phase-contrast imaging with a random mask and a conventional system.  Optica, 8:111412–04, 2021.
  * [31] Konstantin M Pavlov, David M Paganin, Heyang (Thomas) Li, Sebastien Berujon, Hélène Rougé-Labriet, and Emmanuel Brun.  X-ray multi-modal intrinsic-speckle-tracking.  Journal of Optics, 22(12):125604, nov 2020.
  * [32] Konstantin M. Pavlov, David M. Paganin, Kaye S. Morgan, Heyang (Thomas) Li, Sebastien Berujon, Laurène Quénot, and Emmanuel Brun.  Directional dark-field implicit x-ray speckle tracking using an anisotropic-diffusion Fokker-Planck equation.  Physical Review A, 104(15):053505, 2021.
  * [33] Samantha J. Alloo, David M. Paganin, Kaye S. Morgan, Marcus J. Kitchen, Andrew W. Stevenson, Sheridan C. Mayo, Heyang T. Li, Ben M. Kennedy, Anton Maksimenko, Joshua C. Bowden, and Konstantin M. Pavlov.  Dark-field tomography of an attenuating object using intrinsic x-ray speckle tracking.  Journal of Medical Imaging, 9(3):031502–14, 2022.
  * [34] David M. Paganin and Kaye S. Morgan.  X-ray Fokker–Planck equation for paraxial imaging.  Scientific Reports, 9(1):17537, 2019.
  * [35] Kaye S. Morgan and David M. Paganin.  Applying the Fokker–Planck equation to grating-based x-ray phase and dark-field imaging.  Scientific Reports, 9(1):17465, 2019.
  * [36] Michael Reed Teague.  Deterministic phase retrieval: a Green’s function solution.  Journal of the Optical Society of America, 73:1434–1441, 1983.
  * [37] D. M. Paganin.  Coherent X-Ray Optics.  Oxford University Press, Oxford, 2006.
  * [38] D. M. Paganin and D. Pelliccia.  X-ray phase-contrast imaging: a broad overview of some fundamentals.  Adv. Imaging Electron Phys., 218:63–158, 2021.
  * [39] David M. Paganin, Daniele Pelliccia, and Kaye S. Morgan.  Paraxial diffusion-field retrieval, 2023.  arXiv:2301.09046.
  * [40] David Paganin, Sherry C. Mayo, Timothy E. Gureyev, P. R. Miller, and Stephen W. Wilkins.  Simultaneous phase and amplitude extraction from a single defocused image of a homogeneous object.  Journal of Microscopy, 206(3):33–40, 2002.
  * [41] D. Paganin, T. E. Gureyev, S. C. Mayo, A. W. Stevenson, Ya. I. Nesterets, and S. W. Wilkins.  X-ray omni microscopy.  Journal of Microscopy, 214(3):315–327, 2004.
  * [42] Timothy E. Gureyev, Andrew W. Stevenson, David M. Paganin, Timm Weitkamp, Anatoly Snigirev, Irina Snigireva, and Stephen W. Wilkins.  Quantitative analysis of two-component samples using in-line hard x-ray images.  Journal of Synchrotron Radiation, 9:148–153, 2002.
  * [43] Mario A. Beltran, David M. Paganin, Kentaro Uesugi, and Marcus J. Kitchen.  2D and 3D x-ray phase retrieval of multi-material objects using a single defocus distance.  Optics Express, 18:6423–6436, 2010.
  * [44] S Irvine, R Mokso, P Modregger, Z Wang, F Marone, and M Stampanoni.  Simple merging technique for improving resolution in qualitative single image phase contrast tomography.  Optics Express, 22(22):27257–27269, 2014.
  * [45] X-Ray Interactions With Matter.  https://henke.lbl.gov/optical_constants/.  Hosted at website of Center for X-Ray Optics, Lawrence Berkeley National Laboratory (LBNL), Materials Sciences Division.
  * [46] E. Hecht.  Optics.  Pearson, Boston, 5 edition, 2017.