#  Lossy Compression of Electron Diffraction Patterns for Ptychography via
Change of Basis

Anton Gladyshev  Thomas C. Pekin  Marcel Schloz  Benedikt Haas  Johannes
Müller and Christoph T. Koch  AG SEM, Department of Physics, Humboldt
Universität zu Berlin & IRIS Adlershof, Berlin, Germany
Email<EMAIL_ADDRESS>

###### Abstract

Ptychography is a computational imaging technique that has risen in popularity
in the x-ray and electron microscopy communities in the past half decade. One
of the reasons for this success is the development of new high performance
electron detectors [1] with increased dynamic range and readout speed, both of
which are necessary for a successful application of this technique. Despite
the advances made in computing power, processing the recorded data remains a
challenging task, and the growth in data rate has made the size of the
resulting datasets a bottleneck for the whole process. Here we present an
investigation into lossy compression methods for electron diffraction patterns
that retain the necessary information for ptychographic reconstructions, yet
lead to a decrease in data set size by three or four orders of magnitude. We
apply several compression methods to both simulated and experimental data -
all with promising results.

##  1 Introduction

Used at synchrotron beamlines, in optical microscopes, or transmission
electron microscopes (TEMs), ptychography recovers a two-dimensional or three-
dimensional complex transmission function of an object from a four-dimensional
dataset, comprised of a set of diffraction patterns that have been collected
for many (often thousands) of (overlapping) beam positions on the sample [1].
Through the use of a direct [2, 3], or a more complex iterative [4, 5, 6, 7,
8, 9, 10] reconstruction algorithm, ptychography can provide a higher
resolution than the conventional Rayleigh limit determined by a microscope’s
aperture [11] and correct for microscope aberrations [12, 13].

The foundations of ptychography were first described in a series of papers by
Hoppe and coauthors in the ’60s [14, 15, 16, 17], but a number of
breakthroughs such as fast cameras, improved reconstruction algorithms and an
increase of computational power had to be made before this technique started
to provide results that overcame limitations of other imaging techniques,
first in photon imaging (optical and X-ray) [18, 19, 20], and ultimately also
in electron imaging [21].

Nowadays, acquisition of 4D-STEM datasets [1] is becoming routine, and one can
choose between several ptychographic reconstruction algorithms [21, 1, 5, 2,
3, 10, 4, 6, 7, 8, 9] to process the data [22, 3, 23, 24]. However, one of the
main drawbacks of ptychography remains –– the ratio between the amount of
information required to be recorded and the amount of the output information
that can be reconstructed from it. The combination of partial spatial
coherence of the illumination, detector size, desired resolution of the
reconstruction, and the requirement to sample overlapping specimen areas by
adjacent probe positions imposes a lower limit on the amount of acquired data.
This data size can quickly reach the limits of the storage system used to
record the data and thus limits the range of some experimental parameters
crucial for the new trends in ptychography (e.g. multifocus ptychography [25]
or ptychotomography [26]) such as the number of sequential scans that can be
acquired or the size of field of view that can be scanned.

In X-ray ptychography, data compression [27, 28, 29] is mainly based on two
concepts – singular value decomposition that uses the fact that the
diffraction intensities from neighboring and overlapping scan positions are
typically correlated according to the ptychographic oversampling [10] and on
constrained pixel sums, where the reduction of the diffraction data is
achieved by summation over given regions. The second concept is closely
related to binning, a compression technique that binds together neighboring
pixels, however the two approaches are not identical. The main advantage of
the constrained pixel sums compression is that not the pixels in a given
region, but only their sums are constrained, similar to Sudoku [27]. Another
compression technique [30, 29] is based on an assumption that the measured
diffraction patterns are well described by the Poisson statistics. The
measured intensities are rounded to 8bit unsigned integers.

Binning, having its disadvantages mentioned above, is the most common
compression strategy in electron ptychography [10] and leaves plenty of room
for novel compression techniques to arise, e.g. the usage of binary 4D-STEM
data [31] with a linear reconstruction algorithm (single side band [3]). Yet
the specific geometry of electron diffraction patterns in the context of an
iterative ptychographic reconstruction has not been fully explored.

During the research we used ADORYM [32]. It is one implementation of an
iterative gradient-based ptychographic reconstruction algorithm capable of
directly minimizing the discrepancy between experimental and model-predicted
data expressed by specific choices of error metrics, optionally combined with
various regularization terms (see also [7, 10]). It is written in Python and
computes the necessary derivatives using pytorch [33] or autograd [34], two
different automatic differentiation packages, that can be chosen as a backend.
Initially developed for X-ray ptychography, ADORYM offers a generic forward
model, includes multislice reconstruction, partial coherence of the probe and
a flexible Python implementation. After replacing the dispersion law of
photons by the one for electrons, we used ADORYM’s default sparse multi-slice
forward model for electron ptychography.

##  2 Theory

###  2.1 Ptychography

A 2D ptychographic reconstruction algorithm retrieves amplitude and phase of
the scattering object from diffraction patterns that are stored as a 4D-STEM
data set in terms of probe positions $\bm{\rho}_{p}=\rho_{p,x}$, $\rho_{p,y}$
and spatial frequencies in the detector plane $k_{f,x}$ and $k_{f,y}$ (in the
remainder of the paper, bold face variables indicate vectors). Here we will
briefly explain main theoretical aspects of the reconstruction procedure.

The interaction of thin specimens with the incident probe is mathematically
described as the real-space product of a two dimensional wavefunction
representing the incoming probe $P^{(in)}(\bm{\rho}-\bm{\rho}_{p})$ with a
complex transmission function that represents the object $O(\bm{\rho})$.

$\displaystyle P^{(out)}(\bm{\rho})=P^{(in)}(\bm{\rho}-\bm{\rho}_{p})\cdot
O(\bm{\rho}).$ (1)

In real-space the two-dimensional transmission function $O(\bm{\rho})$ depends
on the spatial coordinate $\bm{\rho}=(x,y)$ and physically represents the
modulation of amplitude and phase of an incident electron wave. The
introduction of a relative coordinate $\bm{\rho}-\bm{\rho}_{p}$ is used to
describe the position within the transmission function relative to different
probe positions. Very thin specimens do not absorb electrons, but impose
mainly a phase shift on the traversing probe wave function. For this reason
ptychographic reconstructions typically aim to reconstruct the phase
$\phi(\bm{\rho})$ of the object (transmission) function $O(\bm{\rho})$, which
is directly proportional to the projected electrostatic potential
$V_{proj}(\bm{\rho})$ of the specimen ($\phi(\bm{\rho})=\sigma\cdot
V_{proj}(\bm{\rho})$, where the interaction strength $\sigma$ depends on the
incident beam energy). Absorption of traversing electrons by the 2D object
function $O(\bm{\rho})=\exp[j\phi(\bm{\rho})]$ can be included by allowing the
potential and thus the phase to also have a (positive) imaginary part [7].

The diffraction patterns measured in a real scattering experiment are
conventionally stored as 4D-STEM data sets and will be referred to hereinafter
as $I^{(m)}_{\bm{\rho}_{p},\bm{k}_{f}}$. Starting from an initial guess for
the object function (e.g. $O(\bm{\rho})=1)$, one calculates the expected
diffraction patterns $I^{(e)}_{\bm{\rho}_{p},\bm{k}_{f}}$ as the squared
modulus of the Fourier transform of the wave defined in Equation (1).

$I^{(e)}_{\bm{\rho}_{p},\bm{k}_{f}}=|\mathscr{F}_{\bm{\rho}\rightarrow\bm{k}_{f}}\left(P^{(out)}(\bm{\rho})\right)|^{2}$
(2)

where $\mathscr{F}_{\bm{\rho}\rightarrow\bm{k}_{f}}$ denotes a Fourier
transform with respect to $\bm{\rho}$ as a function of $\bm{k}_{f}$.

The discrepancy between measured and expected intensities is calculated at
each scan position with a loss function $\mathscr{L}$. It can be formed by
various error metrics (solely or in combination). For our calculations we used
the squared $L_{2}$ norm of the wave magnitudes [35]:

$\displaystyle\mathscr{L}(\bm{\rho}_{p})=\sum_{\bm{k}_{f}}\left(\sqrt{I^{(e)}_{\bm{\rho}_{p},\bm{k}_{f}}}-\sqrt{I^{(m)}_{\bm{\rho}_{p},\bm{k}_{f}}}\right)^{2}.$
(3)

After the loss estimation the algorithm computes the derivatives with respect
to the quantities to be optimized (the phase $\phi(\bm{\rho})$ and probe
function $P^{(in)}(\bm{\rho})$) and updates them in a line search reducing the
loss $\mathscr{L}$. The reconstruction iterates until the value of the loss
function becomes sufficiently small.

####  2.1.1 Multi-slice formalism

To describe the interaction of the probe with thicker specimens, one has to
account for multiple scattering events [36]. In this case, the transmission
function of the specimen cannot be treated as a two-dimensional quantity. The
propagation of the beam is divided into intervals $\left[z_{i},z_{i+1}\right)$
and the projected object potential is broken into multiple two-dimensional
slices $V_{proj}^{(i)}(\bm{\rho})=\int_{z_{i}}^{z_{i+1}}V(\bm{\rho},z)dz$
along it resulting in object slices $O^{(i)}(\bm{\rho})=\exp[j\sigma
V_{proj}^{(i)}(\bm{\rho})]$. For a given two-dimensional wave function of the
incoming electron beam $P^{(in)}(\bm{\rho}-\bm{\rho}_{p})$ entering the object
at position $\bm{\rho}_{p}$, a 2D-wave $P^{out}$ exiting the object is
computed by a sequence of propagation and interaction steps:

$\displaystyle
P^{out}(\bm{\rho})=\left(P^{in}(\bm{\rho}-\bm{\rho}_{p})\underbrace{\cdot
O^{i}(\bm{\rho})}_{interaction}\right)\underbrace{\otimes_{\bm{\rho}}P_{Fr}(\bm{\rho},z_{i+1}-z_{i})}_{propagation}$
(4)
$\displaystyle{P}_{Fr}(\bm{\rho},d)=\mathscr{F}_{\bm{q}\rightarrow\bm{\rho}}\left\\{e^{-i\pi\lambda
d|\bm{q}|^{2}}\right\\}$ (5)

Here ${P}_{Fr}(\bm{\rho},d)$ is a Fresnel propagator, $\otimes_{\bm{\rho}}$
denotes a real-space convolution, $\mathscr{F}$ represents a Fourier-transform
and $\lambda$ is the de Broglie wavelength of the beam electrons. The
subsequent calculations of the expected intensity and the loss function remain
the same as described in subsection 2.1.

###  2.2 Compression methodologies

For each probe position $\bm{\rho}_{p}$ diffraction patterns recorded by a
detector with $N_{x}\times N_{y}$ pixels are conventionally stored as
matrices, whose elements contain information about the intensities recorded at
individual pixels. Mathematically, these matrices can be treated as elements
of a $N_{x}\cdot N_{y}$ vector space of real numbers with the dot product
$\langle.,.\rangle_{F}$ and norm $\|.\|_{F}$ computed by the Frobenius inner
product. With two $N_{x}\times N_{y}$ matrices $\bm{A}$ and $\bm{B}$ this can
be defined as follows:

$\displaystyle\langle\bm{A},\bm{B}\rangle_{F}=\sum_{ij}A^{*}_{ij}B_{ik}$ (6)
$\displaystyle\|\bm{A}\|_{F}=\sqrt{\langle\bm{A},\bm{A}\rangle_{F}}$ (7)

The traditional basis set for this vector space, hereinafter referred to as
the ”pixel basis”, is composed of basis vectors representing single pixels.
Using this basis set, one can describe any $N_{x}\times N_{y}$ matrix.
However, the diffraction patterns recorded within a 4D-STEM experiment are
quite similar in their intensity distributions. It should thus be possible to
approximate them using a basis set having fewer elements than the original
pixel basis. An ideal basis for compressing the data of a 4D-STEM experiment
would satisfy the following requirements:

  1. 1.

Provide the best achievable resolution given the amount of data used.

  2. 2.

Provide a good match between the original patterns and their compressed
representations.

  3. 3.

Satisfy the previous two items with as few degrees of freedom as possible

As previously, we assign a subscript $e$ to the expected patterns predicted by
forward propagation and $m$ to the projected (measured) patterns stored in the
4D-STEM data set. The decomposition coefficients $c_{n}$ in a new basis were
calculated for the square roots of both intensities as Frobenius inner
products:

$\displaystyle
c_{n,\bm{\rho}_{p}}=\sum_{\bm{k}_{f}}\sqrt{I_{\bm{\rho}_{p},\bm{k}_{f}}}\cdot
M_{\bm{k}_{f},n}$ (8)

Here $M_{\bm{k}_{f},n}$ is a pixel-basis representation of the n-th new basis
vector. When calculating the decomposition coefficients with a pytorch
convolution layer within the ADORYM framework we interchange the terms ”basis
vector” and ”mask”. The panel a of Figure 1 illustrates this masking process.

Figure 1: a) Schematic of the masking process. b) Illustration of the
uncompressed ($\vec{u}$) and compressed ($\vec{c}$) representations of a
vector. $\vec{c}$ and and $\vec{u}$ create an angle $\Theta$. A compression
basis is considered suitable when it reduces this angle. c) A typical log-
scaled electron diffraction pattern.

Fig. 1c) shows a simulated electron diffraction pattern that can be separated
in three non-overlapping regions: bright field, dark field and the outer
regions containing unmeasured pixels. Because of this specific geometry, it is
convenient to use two separate non-overlapping basis sets to describe the
bright and dark field. The pixels in the unmeasured areas of the detector (in
an actual experiment these would for example be shadowed by the HAADF
detector) contain only zeros, so one does not have to store them.

####  2.2.1 Loss function for compressed patterns

When using the new basis set, the loss function $\mathscr{L}$ is computed not
by comparing individual pixel values, but rather the decomposition
coefficients:

$\displaystyle\mathscr{L}(\bm{\rho}_{p})=\sum_{n}\left(c^{(e)}_{n,\bm{\rho}_{p}}-c^{(m)}_{n,\bm{\rho}_{p}}\right)^{2}.$
(9)

This loss function can be extended by various terms. Excluding the unmeasured
pixels of the measured diffraction patterns from the dataset may cause
divergence of the reconstruction. To prevent this a sum over the unmeasured
(edge) pixels of the expected intensity can be added to the loss function:

$\displaystyle\mathscr{L}_{edge}(\bm{\rho}_{p})=\sum_{\bm{k}_{f}\in\text{
}edge}I^{(e)}_{\bm{\rho}_{p},\bm{k}_{f}}$ (10)

which implicitly forces the values in those unmeasured pixels towards zero.

Panel b of Figure 1 demonstrates a situation, where an uncompressed 2D-vector
$\vec{u}$ is projected onto a compression axis, producing a compressed vector
$\vec{c}$. In the context of our discussion, the squared length of $\vec{u}$
can be interpreted as a total intensity of a diffraction pattern and
$|\vec{c}|^{2}$ is the amount of intensity included by the new basis. We found
it useful (see Appendix A) to separately constrain the ”floating parts” of the
intensity in the bright and dark fields.

Knowing the total intensity of a pattern in the bright or dark field region of
the diffraction pattern and the decomposition coefficients in the new basis,
one can estimate the squared length of a ”perpendicular” component
$c^{(\perp)}$ not included into a new basis with the Pythagoras theorem:

$\displaystyle\left(c^{(e/m)}_{\perp,\bm{\rho}_{p}}\right)^{2}=\sum_{\bm{k}_{f}}\left(I^{(e/m)}_{\bm{\rho}_{p},\bm{k}_{f}}\right)-\sum_{n}\left(c^{(e/m)}_{n,\bm{\rho}_{p}}\right)^{2}$
(11)

In the Equation above $\bm{k}_{f}$ and $n$ correspond either to bright or dark
field. For both bright and dark field detector areas the loss function
contributions $\mathscr{L}_{\perp}(\bm{\rho}_{p})$ were formulated as the
$l^{1}$ norm of the difference between the expected and measured squared
perpendicular components. A heavy weighting of this part of the loss function
minimizes gross intensity differences in the diffraction patterns that were
experimentally acquired and ones simulated in the ptychographic
reconstruction.

Finally, we define a constraint for the probe. The values of the phase-space
wavefunction of the electron beam lying outside of the physical probe-forming
aperture with a radius $k_{a}$ were constrained to be zero:

$\displaystyle\mathscr{L}_{probe}=\sum_{|\bm{k}|>k_{a}}\left|\mathscr{F}_{\bm{\rho}\rightarrow\bm{k}}\left\\{P^{(in)}(\bm{\rho})\right\\}\right|^{2}.$
(12)

In addition to the loss-function contributions described above we considered
two conventional regularization terms [10]: total variation (TV) of the
reconstructed object function and the $l^{1}$-norm of its phase. The total
mismatch was calculated as follows:

$\displaystyle\mathscr{L}(\bm{\rho}_{p})$
$\displaystyle=\underbrace{\sum_{n}\left|c^{(e)}_{n,\bm{\rho}_{p}}-c^{(m)}_{n,\bm{\rho}_{p}}\right|^{2}}_{\text{bright
and
dark}}+\left|\left(c^{(e,bright)}_{\perp,\bm{\rho}_{p}}\right)^{2}-\left(c^{(m,bright)}_{\perp,\bm{\rho}_{p}}\right)^{2}\right|+\left|\left(c^{(e,dark)}_{\perp,\bm{\rho}_{p}}\right)^{2}-\left(c^{(m,dark)}_{\perp,\bm{\rho}_{p}}\right)^{2}\right|+$
$\displaystyle+\mathscr{L}_{edge}(\bm{\rho}_{p})+\mathscr{L}_{probe}+TV\left\\{O(\bm{r})\right\\}+\sum_{\bm{r}}|O_{phase}(\bm{r})|.$
(13)

All parts of the loss function can be supplied with adjustable weights, which
are dropped from the discussion in order to keep the expressions short. The
main interest of this work was to investigate the behavior of the compressed
loss-function, thence the weights of the conventional regularization terms
were kept low. The parameters used in all further shown reconstructions are
listed in Table 1 in Appendix D.

The form of the loss function defined in Eq. 13 ensures that the predicted
patterns match those measured along the directions specified by the basis,
while the constraints try to make their total length in terms of the $L_{2}$
norm equal in separated areas of the detector plane. The impact of the
constraints accounting for the total length is discussed in Appendix C.

####  2.2.2 Basis options

In the following section we compare three orthonormal basis set systems:
binning, Zernike polynomials, and a data-dependent basis set.

Binning can be computed by using square-shaped non-overlapping masks that
average the pattern value over an area (for an application of conventional
binning to ptychography see, e.g. [10]). The difference between conventional
binning and the implementation here is a scaling factor. In the implementation
described here an orthonormal basis is used in which the $l^{2}$ norm of each
mask is equal to one. An example of bright field binning masks is shown in
panels a and e of Figure 2.

An alternative option for the basis set are Zernike polynomials - an
orthonormal set of polynomials within the unit circle. Two sets of Zernike
polynomials were used – one for the bright field and the second one for the
dark field. Zernike polynomials can be described using two indices $n$ and
$m$.

$\displaystyle Z_{n}^{m}=N_{n}^{m}\cdot R_{n}^{m}(r)cos(\phi)$ (14)
$\displaystyle Z_{n}^{-m}=N_{n}^{m}\cdot R_{n}^{m}(r)sin(\phi)$ (15)
$\displaystyle
R^{m}_{n}(r)=\sum_{k=0}^{\tfrac{n-m}{2}}\frac{(-1)^{k}\,(n-k)!}{k!\left(\tfrac{n+m}{2}-k\right)!\left(\tfrac{n-m}{2}-k\right)!}\;r^{n-2k}$
(16)

$N^{m}_{n}$ is a normalisation factor and $R_{n}^{m}$ is a radial part of the
two dimensional Zernike polynomial $Z^{m}_{n}(r,\phi)$. The dark field
polynomials have a hole in the bright field area and are thus doughnut-shaped.
Because of that, Gram-Schmidt orthonormalization was used to make the modified
Zernike dark field polynomials orthonormal to each other.

In principle one can use any arbitrary set of 2D orthonormal polynomials to
describe a diffraction pattern. One nontrivial option is a basis set based on
diffraction patterns acquired from the same sample, that the actual data is
later recorded from. Due to the periodic structure of crystalline specimens,
many of the recorded diffraction patterns are similar. Therefore, one can
record diffraction patterns by a quick sampling across the sample (e.g. in an
area adjacent to the area of interest or very coarsely in the same area) and
use them to create a basis for the subsequently recorded diffraction patterns
comprising the actual data set.

In the current work, diffraction patterns from random positions in the same
area covered by the actual data set were used. To generate the compression
basis, we used only the bright field region and applied a Gram-Schmidt
algorithm to make the masks orthonormal. An example bright field mask
generated in this way is presented in panel b of Fig. 2. This approach appears
similar to a principal component decomposition [37, 38, 39], however it does
not require the complete data set to be recorded before compression, since one
selects the raw data randomly. Another benefit of our approach is that it is
computationally much cheaper, a new basis can be generated almost immediately
and the compression can be applied directly during acquisition and before the
data is stored to disk. Nevertheless, we have also tested a principal
component compression approach, the results of which are presented in Appendix
C.

Figure 2: a) Bright field binning masks (denoted with individual colors)
polynomial used for the description of the bright field. b) A mask generated
from a prerecorded pattern. c) An example of a Zernike polynomial. d) A
modified Zernike polynomial used for the description of the dark field. Note
the change in scale. e) Binning masks used for the dark field description. The
individual binning masks do not overlap, so all masks can be shown in a single
picture. Even though the Zernike and diffraction pattern basis sets are
orthonormal, the individual masks cover the same pixels, so it is not possible
to show multiple masks in a single panel.

###  2.3 Evaluation Metrics

####  2.3.1 Storage space

In order to describe the relation between the storage space required for
uncompressed and compressed 4D-STEM data sets, one can introduce a simple
compression metric C. This is defined as the ratio between the degrees of
freedom of the uncompressed pattern, i.e. the number of pixels, and the number
of stored values per pattern:

$\displaystyle C=\frac{\\#\text{ of pixels}}{n_{bright}+n_{dark}+2}$ (17)

$n_{bright}$ and $n_{dark}$ describe the number of used bright and dark field
masks, the term 2 corresponds to the perpendicular components
$c^{(e)}_{\perp}$ one each for the bright and dark fields defined in the
Equation 11.

####  2.3.2 Intact information metric

In panel b of Fig. 1 we showed the process of the dimension reduction from
2D-space to 1D-space. To describe how well the chosen basis system is able to
match a given vector one can use the angle $\Theta$ presented in this Figure.
Ideally, the vector to be compressed is an element of the space spanned by the
basis, in which case the angle $\Theta$ is zero. In the worst case scenario
all basis vectors are orthogonal to the chosen vector, and the angle $\Theta$
is equal to $\pi/2$. We calculated $\Theta$ separately for the bright and dark
field areas of the data to estimate how well various basis systems describe
the diffraction patterns. This metric is closely related to cosine similarity
(in biology also known as Otsuka–Ochiai or Ochiai–Barkman coefficient [40, 41,
42]), a more explicit discussion about this error metric can be found in
Appendix A.

$\displaystyle\Theta_{bright/dark}(\bm{\rho}_{p})=\arccos\left(\sqrt{\frac{\sum_{n\in
bright/dark}(c_{n,\bm{\rho}_{p}})^{2}}{\sum_{\bm{k}_{f}\in\text{
}bright/dark}I_{\bm{\rho}_{p},\bm{k}_{f}}}}\right).$ (18)

####  2.3.3 Fourier shell correlation

Fourier shell correlation (FSC) [43, 44] is a widely used metric for achieved
resolution. It describes how well the phase-space representations of two real-
space three dimensional data sets match each other. This is useful, for
example, when the ground truth transmission function is known. For two object
functions $O_{1}(\bm{r})$ and $O_{2}(\bm{r})$ the Fourier shell correlation
(FSC) is calculated as follows:

$FSC(q)=\frac{\sum_{|\bm{k}|=q}\mathcal{F}^{*}_{\bm{r}\rightarrow\bm{k}}\left\\{O_{1}{(\bm{r})}\right\\}\cdot\mathcal{F}_{\bm{r}\rightarrow\bm{k}}\left\\{O_{2}{(\bm{r})}\right\\}}{\sqrt{\sum_{|\bm{k}|=q}|\mathcal{F}_{\bm{r}\rightarrow\bm{k}}\left\\{O_{1}{(\bm{r})}\right\\}|^{2}\cdot\sum_{|\bm{k}|=q}|\mathcal{F}_{\bm{r}\rightarrow\bm{k}}\left\\{O_{2}{(\bm{r})}\right\\}|^{2}}}$
(19)

The Fourier-transformed images of atoms arranged in a perfect crystal are
typically comprised only by very well defined (Bragg) peaks, with mostly noise
or zeros in between. Defects produce very weak signal between the Bragg peaks.
In order to exclude the futile information, we evaluated the Fourier shell
correlation only up to a cutoff-frequency of $2$ Å-1. The value of the FSC is
in general complex, we found it useful to evaluate the correlation using its
real part.

##  3 Simulated data

In order to be able to compare the reconstruction result with the ground
truth, the compression was first tested using a simulated data set. This data
set was simulated using qstem [45], including thermal diffuse scattering and
partial spatial coherence. A 92 Å thick atomistic model of a silicon crystal
containing a $60^{\circ}$ edge dislocation dissociated into a stacking fault
bounded by a $30^{\circ}$ and $90^{\circ}$ partial dislocation [45, 46] was
scanned by the probe with an accelerating voltage of 60 kV and with a step
size of 1 Å along both, x- and y-direction. The probe was simulated for a
convergence angle of 30 mrad using the following aberrations: defocus = 30 nm,
astigmatism = 5 nm ($\phi_{A1}=25^{\circ}$), coma = 500 nm
($\phi_{B2}=80^{\circ}$), spherical aberration = -0.04 mm, chromatic
aberration = 1.2 mm, energy spread = 0.3 eV, effective source size = 0.5 Å.
The simulated detector had 500x500 pixels covering scattering angles that
corresponded to a real-space sampling of 0.07 Å in both dimensions. A gray-
scale image presented in panel a of Fig. 3 shows the projected potential of
this model, the pseudo-color images in panels c-f show the phase maps that are
proportional to the four slices of the object potential being reconstructed in
this multi-slice ptychography reconstruction from the uncompressed data set.
The upper and lower slice of the potential have weaker contrast than the two
central slices, as one would expect from the limited resolution along the
z-direction that corresponds to a beam convergence angle of 30 mrad at 60 kV.

Figure 3: a Projected potential of the 92 Å thick Si dislocation model. Red
box indicates the area presented in panels c)-f). b Azimuthally integrated
power spectrum of the projected potential at spatial frequencies $|\bm{q}|$.
c-f Four slices of the phase of the 3D object reconstructed from the
uncompressed 4D-STEM data set. The parameters of the reconstruction are listed
in Table 1 in Appendix D.

We found that the quality of the reconstruction improved with the number of
bright field masks, independent of the decomposition basis. The object
function converged with only one stored value accounting for the mean
intensity in the case of all three dark-field basis sets (binning, Zernike
polynomials and diffraction pattern basis), i.e. representing the total
diffracted intensity. Storing more dark field values when using the Zernike
basis only marginally improved the quality of the reconstruction. In Figure
4a-h, we compare two reconstructions that both used the first 45 Zernike
bright field masks and 2 perpendicular coefficients defined by Equation (11),
one each for bright and dark fields, but differed in the number of dark field
masks. We used a single value (mean dark-field intensity) in a-d and the first
22 azimuthally invariant (with $m=0$) Zernike dark field masks in e-h. The
reconstruction result improved slightly when including more dark-field masks.
In Figs. 4i-p a similar comparison has been carried out for the case of
compression by binning. Even though one would expect more dark field basis
vectors to provide a more accurate description of patterns in terms of the
metric defined in Equation 18, for compression by binning, the quality of the
reconstruction worsens when using 172 dark-field masks (each being 22 px wide
along both dimensions) versus a single dark-field mask (334 px wide along both
dimensions).

Figure 4: Multislice reconstructions of the phase of the object, reconstructed
over four slices. The reconstruction in row a)-d) was done with the first 45
Zernike polynomials for the bright-field area of the detector and 1 dark field
Zernike polynomial, in row e)-h) with the first 45 Zernike polynomials for the
bright field area and 22 dark field Zernike polynomials for the dark-field
area, in row i)-l) with 45 bright field and 1 dark field binning masks and in
row m)-p) with 45 bright field and 172 dark field binning masks. The
parameters of the reconstructions can be found in Table 1 in Appendix D.
Figure 5: Summed phase along the beam direction of the reconstructed object
using various compression basis sets and electron doses. Colorbar on the right
is in radians. In all presented cases we used a single dark field mask and
varied the number of bright field masks and the amount of noise. The
reconstructions a), e), g), k), m), q) (first column) are done assuming ideal
experimental situation, i.e. infinite dose and perfect coherence of the
source. The reconstructions b-d, h-j, n-p are done assuming the partial
spatial coherence and a finite dose, the value of which is given in the top
right corner of the corresponding reconstruction. In a)-d) the bright field
was described with 45 7 px wide binning masks, in e) with 52 6 px wide binning
masks, in g)-j) with first 36 Zernike polynomials, in k) with first 45 Zernike
polynomials, in m)-p) with 15 masks generated from diffraction patterns
generated before the main acquisition and in q) with 30 masks generated from
diffraction patterns.The plots f), l), r) show the Fourier shell correlations
of the reconstructed objects with ground truth, which was the potential used
to simulate the 4D-STEM data. The parameters of the reconstructions are listed
in Table 1 in Appendix D.

Fig. 4 shows that the dimension reduction in the diffraction pattern makes the
reconstruction slightly underdetermined. The shape of the reconstructed atoms
varies from circular (e.g. panel k) to triangular (e.g. panel j), more noise
is present (e.g. panel b) and/or the overall resolution of the reconstruction
is relatively low in some of the cases (panels m \- p).

Another important aspect is that the probe $P^{in}(\bm{\rho})$, which was
optimized simultaneously with the object during the reconstructions, converges
to different defocus values depending on the basis set. Thus, some
reconstructed slices in Fig. 4 (e.g. panels d, h and l) of the object are
empty and the object appears to end.

After finding the optimal approach for the dark field compression, we
investigated the limits of compression in terms of the minimal number of
bright field masks and the lowest electron dose. Convergence of the
reconstruction could be achieved by using only the first 36 Zernike masks, 45
binning masks (each 7 px wide and high) or 15 diffraction pattern basis masks
to describe the bright-field region of the detector. A single coefficient
describing the mean intensity in the dark-field area was used in all 3 cases.
It corresponds to the signal one would collect using an ADF detector with a
large outer radius and an inner radius being slightly larger than the semi-
convergence angle of the illuminating probe.

With these limits discovered, the effect of a finite electron dose were
explored. We considered an effective source of size 0.5 Å and used a Poisson
noise model to emulate the finite electron doses indicated for each of the
panels in Fig. 5. Aside from some fluctuations in apparent reconstruction
quality induced by the randomness of the applied noise, the results (phases
summed over all 4 slices) shown in Figure 5 are consistently better for the
diffraction pattern basis (row 5). This is reflected in both the visual
appearance of the phase maps across the noise levels as well as the FSC plots,
especially for spatial frequencies corresponding to Bragg peaks. Fig. 3.b
shows that the Fourier coefficients between the Bragg peaks are only very
weakly represented in the original potential, which explains their poor
representation in the FSC of in all three cases of compression basis. However,
the FSC of spatial frequencies around 0.3 and 0.6 Å-1 are the highest for the
diffraction pattern basis, even though only 15 coefficients have been used, in
comparison to 36 in the case of Zernike polynomials and 45 in case of Binning.
It is, however, noticeable that the diffuse signal between the Bragg peaks is
represented weaker in the case of the diffraction pattern basis than in both
other cases. Besides the lower number of coefficients, this may also be due to
the random choice of diffraction patterns, which makes it likely that the
perfect crystal information is more faithfully represented by this basis than
all the defect configurations possible.

##  4 Experimental data

With the various compression bases’ suitability for ptychography confirmed in
simulation, we turn to experimental data. Although the authors of [35] have
shown that using the magnitude of a wave, i.e. the square root of its
intensity, is more beneficial for the calculation of the loss than the
intensity itself, the calculation of the square roots during on-the-fly
compression may dramatically affect the acquisition time, depending on the
recorded frame size and computational resources. Moreover, there already exist
a number of tools that allow the use of the compression approach described
above directly on the intensity without any adjustments.

Several software packages implement the computation of the dot product of a
diffraction pattern with any mask as described by expression (7) very
efficiently. As an example, the software driving the Dectris ELA detector
attached to the Nion HERMES microscope used for acquiring the experimental
data in this section applies this operation to the data ‘on the fly’ at frame
rates up to 100,000 frames per second [47]. From the perspective of the
microscope operator, the diffraction patterns are thus transferred from the
microscope already decomposed into the chosen basis set, obviating the need to
store hundreds of gigabytes of data which must then later be decomposed.

A monolayer MoS2 was scanned at an accelerating voltage of $60$ kV with a
convergence semi-angle of $33$ mrad with a step size of $0.2$ Å. $128\times
128$ px wide diffraction patterns were compressed using a single dark field
mask accounting for the mean dark-field intensity and the following three
different compression bases for the bright-field region: 1) 15 masks generated
from recorded diffraction patterns; 2) the first 36 Zernike polynomials and 3)
45 binning masks $8\times 8$ pixels in size. Subsequently the dark field mask
was padded so that the pixel width in real space was $0.138$ Å. To reduce the
storage space we decided to neglect the perpendicular components
$c_{\perp,\bm{\rho}_{p}}$ defined in the Equation 11. In Figure 6 we show the
results of the reconstructions.

The probe functions that have been reconstructed along with the object
transmission functions show very clearly that only in the case of the
diffraction pattern basis the probe does not seem to contain any object
information, although this data was compressed the most. In both, the Zernike
and binning compression the probe contains side lobes that correspond to
inter-atomic distances of the MoS2-lattice. This is particularly strong in the
case of compression by binning.

In addition to the probe functions, also the reconstructed object transmission
functions itself is more faithfully reconstructed in the case of the
diffraction pattern basis. The reconstruction shown in Fig. 6a (and its
diffractogram in Fig. 6d) contains more diffuse scattering as well as defects,
which is physically more reasonable, since MoS2 not sandwiched in hBN is
typically covered by contamination [48].

Figure 6: a-c Reconstructed phases of a complex transmission function of a
MoS2 monolayer from compressed 4D-STEM datasets. $128\times 128$ px wide
diffraction patterns were compressed using a single dark field mask accounting
for the mean dark-field intensity and multiple bright field mask
configurations: a) 15 bright field masks generated from diffraction patterns,
b) the first 36 Zernike polynomials, and c) 45 bright field binning masks
(each $8\times 8$ px in size). The diffractograms of these reconstructions
shown in d-f are the log-scaled amplitudes of the Fourier transforms of the
complex transmission functions presented in panels a-c, respectively. The
white circles indicate spatial frequencies of 1 Å-1. In g-i we show the
magnitudes of the reconstructed probe functions that correspond to the
reconstructions a-c, respectively. The plot f) shows azimuthal averages of the
Fourier transformed object functions. The parameters of the reconstruction are
listed in Table 1 in Appendix D.

##  5 Conclusion

We have shown for the case of atomic-resolution ptychography in the STEM, data
can be compressed by a factor of 5,000 to 15,000, while still achieving a
resolution comparable to the uncompressed reconstruction. Three different
compression approaches have been compared. For both simulated and experimental
data the compression approach based on orthogonal masks produced from randomly
recorded diffraction patterns seems to preserve a sufficient amount of
detailed real-space information to allow for a faithful reconstruction even in
the case of reconstructing multiple object slices. The estimation of the
amount of “intact” information is discussed in Appendix A.

For the presented reconstructions, it was sufficient to represent the dark-
field intensity of the diffraction patterns by a single value accounting for
the mean intensity, although a better knowledge of the dark field provides a
slightly better result (see in Appendix B), albeit at severely increased
storage needs. The loss function used contained only two conventional
regularization terms: total variation (TV) of the reconstructed object
function and the $l^{1}$-norm of its phase, both with relatively small weights
(see in Appendix D). It has been shown that regularization can prevent
artifacts due to underdeterminedness of the solution (see, e.g. [10]),
however, in Appendix C we show that the $l^{1}$ constraint does not help to
prevent the artifacts appearing in the reconstructed object. This fact
suggests the need to develop new, more complex concepts of regularization that
would specifically account for the compressed nature of the input data and
potentially improve the visual appearance of the reconstructions. The
investigation of regularization constraints, as well as the evaluation of the
strength of improvement of including more bright-field vs. more dark-field
compression coefficients on the reconstruction quality are reserved for
subsequent studies.

The presented tests with experimental and simulated data show that our
compression algorithm is suitable for real experiments and works also for
reconstructing objects under strong multiple scattering, realistic electron
doses and partial spatial and temporal coherence. The new basis sets proposed
here – Zernike polynomials and a diffraction pattern basis require less
storage capacity than the conventional compression by binning, allowing for
more patterns to be acquired during an experiment and a more efficient use of
data storage.

## Acknowledgements

The authors acknowledge financial support by the Volkswagen Foundation
(Initiative: ”Experiment!”, Project ”Beyond mechanical stiffness”) and by the
Deutsche Forschungsgemeinschaft (DFG) in project nr. 182087777 (CRC951) and
project nr. 414984028 (CRC1404).

## References

  * [1] Colin Ophus  “Four-dimensional scanning transmission electron microscopy (4D-STEM): From scanning nanodiffraction to ptychography and beyond”  In _Microscopy and Microanalysis_ 25.3 Cambridge University Press, 2019, pp. 563–582
  * [2] J.. Rodenburg and R… Bates  “The Theory of Super-Resolution Electron Microscopy Via Wigner-Distribution Deconvolution”  In _Philosophical Transactions of The Royal Society A Mathematical Physical and Engineering Sciences 339(1655):521-553 DOI: 10.1098/rsta.1992.0050_ , 1992
  * [3] Timothy Pennycook et al.  “Efficient phase contrast imaging in STEM using a pixelated detector. Part 1: Experimental demonstration at atomic resolution”  In _Ultramicroscopy_ 151, 2014  DOI: 10.1016/j.ultramic.2014.09.013
  * [4] J. Rodenburg and H. Faulkner  “A phase retrieval algorithm for shifting illumination”  In _Applied Physics Letters_ 85, 2004, pp. 4795–4797  DOI: 10.1063/1.1823034
  * [5] H.M.L. Faulkner and J.M. Rodenburg  “Error tolerance of an iterative phase retrieval algorithm for moveable illumination microscopy”  In _Ultramicroscopy 103(2):153–164, doi: 10.1016/j.ultramic.2004.11.006_, 2005
  * [6] A.M. Maiden and J.M. Rodenburg  “An improved ptychographical phase retrieval algorithm for diffractive imaging”  In _Ultramicroscopy 109(10):1256-62, doi: 10.1016/j.ultramic.2009.05.012_, 2009
  * [7] W. Van den Broek and C.. Koch  “General framework for quantitative three-dimensional reconstruction from arbitrary detection geometries in TEM”  In _Physical Review B_ 87, 2013, pp. 184108  DOI: 10.1103/PhysRevB.87.184108
  * [8] P. Thibault and Guizar-Sicairos M.  “Maximum-likelihood refinement for coherent diffractive imaging”  In _New Journal of Physics 14(6):063004, doi: 10.1088/1367-2630/14/6/063004_, 2012
  * [9] Michal Kronenberg, Andreas Menzel and Manuel Guizar-Sicairos  “Iterative least-squares solver for generalized maximum-likelihood ptychography”  In _Optics Express_ 26, 2018, pp. 3108  DOI: 10.1364/OE.26.003108
  * [10] Marcel Schloz et al.  “Overcoming information reduced data and experimentally uncertain parameters in ptychography with regularized optimization”  In _OpticsExpress_ 28.19, 2020, pp. 28306–28323
  * [11] P.. Nellist, B.. McCallum and J.. Rodenburg  “Resolution beyond the ’information limit’ in transmission electron microscopy”  In _Nature_ 374, 1995, pp. 630–632  DOI: 10.1038/374630a0
  * [12] J.M. Rodenburg, B.C. McCallum and P.D. Nellist  “Experimental tests on double-resolution coherentimaging via STEM”  In _Ultramicroscopy_ 48, 1993, pp. 304–314  DOI: 10.1016/0304-3991(93)90105-7
  * [13] Ryusuke Sagawa et al.  “Ptychographic phase reconstruction and aberration correction of STEM image using 4D dataset recorded by pixelated detector”  In _The 16th European Microscopy Congress, Lyon, France_ , The 16th European Microscopy Congress, Lyon, France, 2016, pp. 5291
  * [14] Walter Hoppe  “Beugung im inhomogenen primärstrahlwellenfeld. i. prinzip einer phasenmessung von elektronenbeungungsinterferenzen”  In _Acta Crystallographica Section A: Crystal Physics, Diffraction, Theoretical and General Crystallography_ 25.4 International Union of Crystallography, 1969, pp. 495–501
  * [15] Walter Hoppe and G. Strube  “Beugung in inhomogenen primärstrahlenwellenfeld. II. lichtoptische analogieversuche zur phasenmessung von gitterinterferenzen.”  In _Acta crystallographica. Section A, Foundations of crystallography_ 25.4, 1969, pp. 501–508
  * [16] Walter Hoppe  “Beugung im inhomogenen Primärstrahlwellenfeld. III. Amplituden-und Phasenbestimmung bei unperiodischen Objekten”  In _Acta Crystallographica Section A: Crystal Physics, Diffraction, Theoretical and General Crystallography_ 25.4 International Union of Crystallography, 1969, pp. 508–514
  * [17] Reiner Hegerl and Walter Hoppe  “Dynamische theorie der kristallstrukturanalyse durch elektronenbeugung im inhomogenen primärstrahlwellenfeld”  In _Berichte der Bunsengesellschaft für physikalische Chemie_ 74.11 Wiley Online Library, 1970, pp. 1148–1154
  * [18] Franz Pfeiffer  “X-ray ptychography”  In _Nature Photonics_ 12.1, 2018, pp. 9–17  DOI: 10.1038/s41566-017-0072-5
  * [19] Pierre Thibault et al.  “High-Resolution Scanning X-Ray Diffraction Microscopy”  In _Science (New York, N.Y.)_ 321, 2008, pp. 379–82  DOI: 10.1126/science.1158573
  * [20] Pierre Thibault et al.  “Probe retrieval in ptychographic coherent diffractive imaging”  In _Ultramicroscopy_ 109, 2009, pp. 338–43  DOI: 10.1016/j.ultramic.2008.12.011
  * [21] J.M. Rodenburg  “Ptychography and Related Diffractive Imaging Methods”  In _Advances in Imaging and Electron Physics_ 150, 2008, pp. 97–184  DOI: 10.1016/S1076-5670(07)00003-1
  * [22] Hao Yang et al.  “Electron ptychographic phase imaging of light elements in crystalline materials using Wigner distribution deconvolution”  In _Ultramicroscopy_ 180, 2017  DOI: 10.1016/j.ultramic.2017.02.006
  * [23] Corey Putkunz et al.  “Atom-Scale Ptychographic Electron Diffractive Imaging of Boron Nitride Cones”  In _Physical review letters_ 108, 2012, pp. 073901  DOI: 10.1103/PhysRevLett.108.073901
  * [24] Yi Jiang et al.  “Electron ptychography of 2D materials to deep sub-ångström resolution”  In _Nature_ 559, 2018  DOI: 10.1038/s41586-018-0298-5
  * [25] Marcel Schloz et al.  “High Resolution Three-Dimensional Reconstructions in Electron Microscopy Through Multifocus Ptychography”  In _Microscopy and Microanalysis_ 28.S1 Cambridge University Press, 2022, pp. 364–366  DOI: 10.1017/S1431927622002203
  * [26] Philipp Pelz et al.  “Solving Complex Nanostructures With Ptychographic Atomic Electron Tomography”, 2022  DOI: 10.48550/arXiv.2206.08958
  * [27] Lars Loetgering et al.  “Data compression strategies for ptychographic diffraction imaging”  In _Advanced Optical Technologies_ 6, 2017  DOI: 10.1515/aot-2017-0053
  * [28] Lars Loetgering, David Treffer and Thomas Wilhein  “Compression and information recovery in ptychography”  In _Journal of Instrumentation_ 13, 2018, pp. C04019–C04019  DOI: 10.1088/1748-0221/13/04/C04019
  * [29] Panpan Huang et al.  “Fast digital lossy compression for X-ray ptychographic data”  In _Journal of Synchrotron Radiation_ 28, 2021, pp. 292–300  DOI: 10.1107/S1600577520013326
  * [30] Klaus Wakonig et al.  “PtychoShelves , a versatile high-level framework for high-performance analysis of ptychographic data”  In _Journal of Applied Crystallography_ 53, 2020  DOI: 10.1107/S1600576720001776
  * [31] Colum O’Leary et al.  “Phase reconstruction using fast binary 4D STEM data”  In _Applied Physics Letters_ 116, 2020, pp. 124101  DOI: 10.1063/1.5143213
  * [32] Ming Du et al.  “Adorym: A multi-platform generic X-ray image reconstruction framework based on automatic differentiation”  In _Optics express_ 29.7 Optica Publishing Group, 2021, pp. 10000–10035
  * [33] Adam Paszke et al.  “PyTorch: An Imperative Style, High-Performance Deep Learning Library”  In _Advances in Neural Information Processing Systems_ 32 Curran Associates, Inc., 2019
  * [34] ““Autograd tutorial,” https://github.com/HIPS/autograd/blob/master/docs/tutorial.md. 48. last visited: 30.08.2022”, 2020
  * [35] Pierre Godard, Marc Allain, Virginie Chamard and John Rodenburg  “Noise models for low counting rate coherent diffraction imaging”  In _Opt. Express_ 20.23 Optica Publishing Group, 2012, pp. 25914–25934  DOI: 10.1364/OE.20.025914
  * [36] J.. Cowley and A.. Moodie  “The scattering of electrons by atoms and crystals. I. A new theoretical approach”  In _Acta Crystallographica_ 10.10, 1957, pp. 609–619  DOI: 10.1107/S0365110X57002194
  * [37] Karl Pearson  “LIII. On lines and planes of closest fit to systems of points in space”  In _Phil. Mag._ 2, 1901, pp. 559–572  DOI: 10.1080/14786440109462720
  * [38] H Hotelling  “Analysis of a complex of statistical variables into principal components”  In _Journal of Educational Psychology_ 24(6), 1933, pp. 417–441  DOI: 10.1037/h0071325
  * [39] Ian Jolliffe and Jorge Cadima  “Principal component analysis: A review and recent developments”  In _Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences_ 374, 2016, pp. 20150202  DOI: 10.1098/rsta.2015.0202
  * [40] Yanosuke Otsuka  “The faunal character of the Japanese Pleistocene marine Mollusca, as evidence of the climate having become colder during the Pleistocene in Japan”  In _Bulletin of the Biogeographical Society of Japan_ 6.16, 1936, pp. 165–170
  * [41] Akira Ochiai  “Zoogeographical studies on the soleoid fishes found in Japan and its neighhouring regions-II”  In _Bulletin of the Japanese Society of Scientific Fisheries_ , 1957, pp. 526–530  DOI: 10.2331/suisan.22.526
  * [42] Howard Crum and J.. Barkman  “Phytosociology and Ecology of Cryptogamic Epiphytes: Including a Taxonomic Survey and Description of Their Vegetation Units in Europe”  In _The Bryologist_ 62.1, 1959  DOI: 10.2307/3240418
  * [43] George Harauz, Lisa Borland and Marin Zeitler  “Three-dimensional reconstruction of a human metaphase chromosome from electron micrographs”  In _Chromosoma_ 95, 1987, pp. 366–374  DOI: 10.1007/BF00293184
  * [44] W.. Saxton and W. Baumeister  “The correlation averaging of a regularly arranged bacterial cell envelope protein”  In _Journal of Microscopy_ 127.2, 1982, pp. 127–138  DOI: https://doi.org/10.1111/j.1365-2818.1982.tb00405.x
  * [45] C.. Koch  “Determination of core structure periodicity and point defect density along dislocations”, 2002
  * [46] J.C.H. Spence et al.  “Imaging dislocation cores - the way forward”  In _Philosophical Magazine_ 86, 2006, pp. 4781–4796
  * [47] Benjamin Plotkin-Swing et al.  “100,000 Diffraction Patterns per Second with Live Processing for 4D-STEM”  In _Microsc. Microanal. (Suppl 1)_ 28, 2022, pp. 422–423  DOI: 10.1017/S1431927622002392
  * [48] Fuhui Shao et al.  “Substrate influence on transition metal dichalcogenide monolayer exciton absorption linewidth broadening”  In _Physical Review Materials_ 6, 2022  DOI: 10.1103/PhysRevMaterials.6.074005

##  Appendix A Measurement of basis quality

Here we show the relationship between the angle $\Theta$ defined in the
Equation 18 and the numbers of basis vectors in the bright and dark fields for
the simulated 4D-STEM dataset.

Figure 7: Angle $\Theta$ described in Equation 18 averaged over multiple scan
positions for bright- and dark-field areas of the diffraction pattern and
various basis systems. a) $\Theta_{bright}$ (mean angle enclosed by the bright
field of a diffraction pattern and its projection on the bright field basis)
for Zernike polynomials (red line), masks generated from randomly selected
diffraction patterns (orange line) and binning masks (black dashed line). b)
$\Theta_{dark}$ (mean angle enclosed by the dark field of a diffraction
pattern and its projection on the dark field basis) for Zernike continuous
(all values of $m$) indexing (red line), azimuthally symmetric ($m=0$) Zernike
polynomials (violet line) and dark field binning masks whose central area has
been cropped so that they can be combined with 7px wide bright field binning
masks (black dashed line).

The plot presented in panel a) of Figure 7 shows that a single mask generated
from randomly selected diffraction patterns on average describes the
diffraction patterns worse then the Zernike or binning masks. However, as the
number of masks increases, the angle $\Theta_{bright}$ decreases much faster
for this basis set and only 10-15 masks generated from randomly selected
diffraction patterns perform noticeably better then a similar number of
Zernike or binning masks.

In contrast to Zernike masks that perfectly matched to the geometry of the
diffraction patterns, 7 px wide bright field binning masks extend slightly out
of the true bright field area and partially cover the transition areas between
the bright and high angle annular dark field (HAADF). These transition pixels
carry values that are higher than the average count value in the rest of the
dark field. Thus, as the panel b) of Figure 7 shows, a single binning mask
that ignores these pixels describes the dark field better then the the first
Zernike dark field mask.

It is noteworthy that the angle $\Theta$ is not a universal tool. To determine
whether the reconstruction will converge, one has to consider the effects
caused by partial spatial coherence, finite dose, and combine it with a total
over-determination ratio in the pixel-basis, derived in [10].

##  Appendix B Anomalous effects in regions not covered by the basis set

Figure 8: Magnitudes of the reconstructed probes in phase space when using
Zernike masks covering a circle with a diameter a) 334px and b) 500 px. In
both cases, the probe intensity in the uncovered area grows uncontrollably.

When performing a dimensional reduction of a vector space, only some part of
the initial information is saved. Even though the measured patterns might be
well described with the chosen basis, the algorithm has no control over its
predictions in this directions.

This problem can be demonstrated on an example with uncovered areas. We
attempted to use Zernike polynomials which only covered a part of the
detector, i.e. not including the edges of the detector, while simultaneously
optimising probe and objects functions. In Figure 8.a, we show a reconstructed
probe function in phase-space. The bright field polynomials covered the pixels
up to 44 px away from the center, the dark field polynomials covered the
pixels with distances between 44 px and 334 px, and the pixels that are 335 px
or further away from the center were completely ignored. During optimization,
the probe collected artifacts in the uncovered/ignored areas.

We discovered the same behaviour of the reconstruction also at different radii
of the dark field polynomials. Figure 8.b shows a Fourier transform of a
reconstructed probe function. The dark field polynomials used covered the
pixels up to 500 px away from the center. In this case, the probe also
collected artifacts in the uncovered areas.

In order to prevent it, we introduced the probe constraint described in
Equation 12 and took the edges of the predicted patterns into account. The
usability of the constants for the parts of bright and dark fields not
included into the new basis can be seen in Figure 9. Here we used first 45
Zernike polynomials for the description of the bright field and first 22
angular symmetric, i.e. with $m=0$, Zernike dark field polynomials.

Figure 9: Two multislice reconstructions of the phase of the object,
reconstructed over four slices that are positioned along the beams propagation
direction at 0 Å, 30.7 Å, 61.4 Å and 92 Å, respectively. Both reconstructions
a)-d) and e)-h) are done using first 45 Zernike bright field masks and first
22 angular-symmetric dark field Zernike masks. The reconstruction a)-d) was
performed without constraint defined in the Equation (11). The reconstruction
e)-h) was performed without it. It is apparent that in a)-d) the object
function collected artifacts in form of stripes, while e)-h) did not. The
parameters of the reconstructions are listed in Table 1 in Appendix D.

##  Appendix C PCA basis and $l^{1}$ regularization.

The concept of a basis generation with the Gram-Schmidt algorithm from
patterns recorded before the main acquisition is closely related to principal
component analysis (PCA) [37, 38, 39]. The main difference between our
approach and the PCA is that the Gram-Schmidt compression proposed by us does
not require to store the whole dataset.

Figure 10: Four multi-slice reconstructions of the phase of object
transmission function from a compressed 4D-STEM dataset using 15 PCA
components of 320 randomly selected diffraction patterns as a basis. a)-d) and
e)-f) show the results of the same reconstruction containing ”peak”-formed
artifacts, in first row we display the reconstruction with its full dynamic
range, and in the second row we set the maximum displayed value to 0.5. i)-l)
and m)-p) show another reconstruction, that was done with the same set of
masks, but the weight corresponding to the regularization constraints, total
variation of the reconstructed potential and the $l^{1}$ norm of its phase,
were increased by a factor of $10^{4}$ and $10^{7}$, respectively. The exact
parameters of the two reconstructions are listed in Table 1 in Appendix D.

To compare the two basis sets we randomly selected 320 diffraction patterns
from the initial dataset and generated a basis from first 15 PCA components of
the small 4D-STEM dataset. The result of the reconstruction presented first
row of Figure 10 contained artifacts – point-shaped peaks, which shifted the
maximum of the phase up to 6 radians. In the second row of the same Figure we
show the same reconstruction, but with an adjusted threshold. As one can see,
the reconstructed phase looks correct, but the resolution of the
reconstruction is not sufficient to resolve the individual atoms. We obtained
the same artifact with ”Gram-Schmidt” compression method, but there we could
suppress this behaviour by adjusting the $\mathscr{L}_{\perp}$-constraints.
Setting higher weights for the $\mathscr{L}_{\perp}$-constraints did not help
to prevent the high phase spikes for the PCA basis. We also tried to increase
the weight corresponding to a regularization constraints (TV of the object
function and $l^{1}$-norm of the reconstructed phase), but, as demonstrated in
panels i)-p) of Figure 10, this reduced the noise between the atoms, but did
not help prevent the appearance of point-shaped peaks. In summary, the basis
created through the Gram-Schmidt approach is more robust than the one created
via PCA. In addition, the Gram-Schmidt compression is computationally cheaper
than the principal component analysis.

##  Appendix D Reconstruction parameters

For the sake of reproducibility of the presented results, we provide the
reconstruction parameters. In the following table one can find the learning
rates (initial step sizes for the updates of the object function, probe
function and the probe positions) and the weights corresponding to the
individual parts of the loss function (main part describing the discrepancy
between the measurement and algorithms predictions, parts describing the
discrepancy between the predictions and the parts of the measured intensity
not included into the basis predicted values on the edge of the detector,
weight corresponding to the probe constraint and two regularization terms-
$l^{1}$ norm of the reconstructed object and its total variation).

Table 1: Parameters of the presented reconstruction results.

figure | Slice positions in Å | Learning rates | Weights corresponding to the individual parts of the loss function
---|---|---|---
|  | object | probe | probe position | ${\mathscr{L}_{main}}$ | ${\mathscr{L}_{\perp}^{bright}}$ | ${\mathscr{L}_{\perp}^{dark}}$ | ${\mathscr{L}_{edge}}$ | ${\mathscr{L}_{Probe}}$ | l1-norm of the phase of the object | TV of the object
Loss function based on uncompressed magnitude
3.c)-f) | [$0$ , $30.7$ , $61.4$ , $92$ ] | 1e-3 | 1e-3 | — | 4e-6 | — | — | — | — | 7e-7 | 7e-6
Loss function based on compressed magnitude
4. a)-d) | [$0$ , $30.7$ , $61.4$ , $92$ ] | 1e-5 | 1e-3 | — | 2.2e-2 | 5e-4 | 5e-5 | 5e-2 | 5e-2 | 7e-7 | 7e-6
4. e)-h) | [$0$ , $30.7$ , $61.4$ , $92$ ] | 1e-5 | 1e-3 | — | 1.5e-2 | 1e-4 | 1e-3 | 5e-2 | 5e-2 | 7e-7 | 7e-6
4. i)-l) | [$0$ , $30.7$ , $61.4$ , $92$ ] | 1e-5 | 1e-3 | — | 2.2e-2 | 1e-4 | 1e-3 | 5e-2 | 5e-2 | 7e-7 | 7e-6
4. m)-p) | [$0$ , $30.7$ , $61.4$ , $92$ ] | 1e-5 | 1e-3 | — | 4.6e-3 | 1e-4 | 1e-3 | 5e-2 | 1e-4 | 7e-7 | 7e-6
5.a)-d) | [$0$ , $30.7$ , $61.4$ , $92$ ] | 1e-5 | 1e-3 | — | 2.2e-2 | 1e-4 | 1e-3 | 5e-2 | 1e-4 | 7e-7 | 7e-6
5.e) | [$0$ , $30.7$ , $61.4$ , $92$ ] | 1e-5 | 1e-3 | — | 1.9e-2 | 1e-4 | 1e-3 | 5e-2 | 1e-4 | 7e-7 | 7e-6
5.g)-j) | [$0$ , $30.7$ , $61.4$ , $92$ ] | 1e-5 | 1e-3 | — | 2.7e-2 | 5e-5 | 5e-4 | 5e-2 | 5e-2 | 7e-7 | 7e-6
5.k) | [$0$ , $30.7$ , $61.4$ , $92$ ] | 1e-5 | 1e-3 | — | 2.2e-2 | 5e-5 | 5e-4 | 5e-2 | 5e-2 | 7e-7 | 7e-6
5.m)-p) | [$0$ , $30.7$ , $61.4$ , $92$ ] | 1e-5 | 1e-3 | — | 6.3e-2 | 5e-5 | 5e-4 | 5e-2 | 5e-2 | 7e-7 | 7e-6
5.q) | [$0$ , $30.7$ , $61.4$ , $92$ ] | 1e-5 | 1e-3 | — | 3.2e-2 | 5e-5 | 5e-4 | 5e-2 | 5e-2 | 7e-7 | 7e-6
9.a)-d) | [$0$ , $30.7$ , $61.4$ , $92$ ] | 1e-5 | 1e-3 | — | 1.5e-2 | 0 | 0 | 5e-2 | 5e-2 | 7e-7 | 7e-6
9.a)-d) | [$0$ , $30.7$ , $61.4$ , $92$ ] | 1e-5 | 1e-3 | — | 1.5e-2 | 1e-4 | 1e-3 | 5e-2 | 5e-2 | 7e-7 | 7e-6
10.a)-h) | [$0$ , $30.7$ , $61.4$ , $92$ ] | 1e-5 | 1e-3 | — | 6.3e-2 | 5e-2 | 5e-2 | 5e-1 | 1e-1 | 7e-7 | 7e-6
10.i)-p) | [$0$ , $30.7$ , $61.4$ , $92$ ] | 1e-5 | 1e-3 | — | 6.3e-2 | 5e-2 | 5e-2 | 5e-1 | 1e-1 | 7e0 | 7e-2
Loss function based on compressed intensity
6.a) | single slice | 1e-3 | 1e-3 | 1e-2 | 6.3e-2 | 0 | 0 | 0 | 1e-6 | 3e-6 | 3e-5
6.b) | single slice | 1e-3 | 1e-3 | 1e-2 | 2.7e-2 | 0 | 0 | 0 | 1e-6 | 3e-6 | 3e-5
6.c) | single slice | 1e-3 | 1e-3 | 1e-2 | 2.2e-2 | 0 | 0 | 0 | 1e-6 | 3e-6 | 3e-5