# High-flux 100-kHz attosecond pulse source driven by a high average power
annular laser beam

Peng Ye  ELI-ALPS, ELI-HU Non-Profit Ltd., Wolfgang Sandner utca 3., Szeged,
H-6728, Hungary.  Corresponding author<EMAIL_ADDRESS>Lénárd Gulyás
Oldal  ELI-ALPS, ELI-HU Non-Profit Ltd., Wolfgang Sandner utca 3., Szeged,
H-6728, Hungary.  Institute of Physics, University of Szeged, Dóm tér 9,
Szeged 6720, Hungary.  Tamás Csizmadia  ELI-ALPS, ELI-HU Non-Profit Ltd.,
Wolfgang Sandner utca 3., Szeged, H-6728, Hungary.  Zoltán Filus  ELI-ALPS,
ELI-HU Non-Profit Ltd., Wolfgang Sandner utca 3., Szeged, H-6728, Hungary.
Tímea Grósz  ELI-ALPS, ELI-HU Non-Profit Ltd., Wolfgang Sandner utca 3.,
Szeged, H-6728, Hungary.  Péter Jójárt  ELI-ALPS, ELI-HU Non-Profit Ltd.,
Wolfgang Sandner utca 3., Szeged, H-6728, Hungary.  Imre Seres  ELI-ALPS, ELI-
HU Non-Profit Ltd., Wolfgang Sandner utca 3., Szeged, H-6728, Hungary.  Zsolt
Bengery  ELI-ALPS, ELI-HU Non-Profit Ltd., Wolfgang Sandner utca 3., Szeged,
H-6728, Hungary.  Barnabás Gilicze  ELI-ALPS, ELI-HU Non-Profit Ltd., Wolfgang
Sandner utca 3., Szeged, H-6728, Hungary.  Subhendu Kahaly  ELI-ALPS, ELI-HU
Non-Profit Ltd., Wolfgang Sandner utca 3., Szeged, H-6728, Hungary.  Institute
of Physics, University of Szeged, Dóm tér 9, Szeged 6720, Hungary.  Katalin
Varjú  ELI-ALPS, ELI-HU Non-Profit Ltd., Wolfgang Sandner utca 3., Szeged,
H-6728, Hungary.  Department of Optics and Quantum Electronics, University of
Szeged, Dóm tér 9, Szeged 6720, Hungary.  Balázs Major  ELI-ALPS, ELI-HU Non-
Profit Ltd., Wolfgang Sandner utca 3., Szeged, H-6728, Hungary.  Department of
Optics and Quantum Electronics, University of Szeged, Dóm tér 9, Szeged 6720,
Hungary.  Corresponding author<EMAIL_ADDRESS>
###### Abstract

High-repetition-rate attosecond pulse sources are indispensable tools of time-
resolved studies of electron dynamics, such as coincidence spectroscopy and
experiments with high demands on statistics or signal-to-noise ratio,
especially in case of solid and big molecule samples in chemistry and biology.
Although with the high-repetition-rate lasers such attosecond pulses in a
pump-probe configuration are possible to achieve, until now only a few such
light sources have been demonstrated. Here, by shaping the driving laser to an
annular beam, a 100-kHz attosecond pulse train (APT) is reported with the
highest energy so far (51 pJ/shot) on target (269 pJ at generation) among the
high-repetition-rate systems ($>$ 10 kHz) in which the attosecond pulses were
temporally characterized. The on-target pulse energy is maximized by reducing
the losses from the reflections and filtering of the high harmonics, and an
unprecedented $19\%$ transmission rate from the generation point to the target
position is achieved. At the same time, the probe beam is also annular, and
low loss of this beam is reached by using another holey mirror to combine with
the APT. The advantages of using an annular beam to generate attosecond pulses
with a high average power laser is demonstrated experimentally and
theoretically. The effect of nonlinear propagation in the generation medium on
the annular-beam generation concept is also analyzed in detail.

##  1 Introduction

Since the first experimental realizations of an attosecond pulse train (APT)
[1] and an isolated attosecond pulse (IAP) [2] at 1 kHz in 2001, attosecond
pulses have been widely used to investigate electron dynamics in gases [3],
liquids [4] and solids [5]. Nowadays, more effort is put towards scaling up
the flux of APTs and IAPs using the lasers of high repetition rate. One route
is the multi-pass high-harmonic generation (HHG) in a laser cavity or a
resonant enhancement cavity [6], in which cases the lasers with a low pulse
energy (nJ $\sim$ $\mu$J) and a very high repetition rate ($>$ MHz) are used.
Another route is the single-pass HHG [7]. In this case, since the laser energy
is close to that of a typical kHz system ($\sim$ mJ), one can keep the same
pump-probe ability of the kHz system, and at the same time increase the
repetition rate to 100 kHz. For the applications in which it is crucial to
avoid space charge effects, such as the photoemission spectroscopy [8], and in
time-resolved coincidence measurements which require few events in each laser
shot [9, 10], in order to achieve a high signal-to-noise ratio, a high
repetition rate and a moderate attosecond pulse energy are preferred.
Furthermore, because the time necessary for data collection can be shortened,
high repetition rate is beneficial in a wide range of experiments such as
coherent diffraction imaging [11], transient absorption [12], and attosecond
pump-probe spectroscopy [13]. For example, it will enhance the scope of single
particle structural dynamics studies [14] and allows to investigate the newly
emerged Schrödinger cat states using strong laser fields [15, 16]. Thanks to
the continuous development of laser technology, high-repetition-rate and high-
average-power lasers have become available, and as a result there is a
continuous increase in the achievable photon flux. In this work, we call the
high-order harmonics with measured attosecond temporal duration as attosecond
pulses, and call those without temporal characterization as high harmonics. We
make this distinction because temporal characterization is a demonstration of
attosecond pump-probe capability. As shown in Figure 1, the single-pass HHG
can provide the high harmonics [17, 18, 19, 20, 21, 22, 23, 24, 25, 25, 26,
27, 28, 29, 30, 31, 32, 33, 34, 35, 36] up to tens of nJ per shot at 1 MHz by
using powerful driving lasers with an average power up to $\sim$ 100 W [37,
38, 39, 40, 41, 42, 43], and the attosecond pulses [44, 45, 46, 47, 48, 49,
50, 51, 52, 53, 54, 55] up to hundreds of pJ per shot at 100 kHz. Intra-cavity
HHG can deliver the high harmonics with the repetition rates up to hundreds of
MHz [56, 57, 58, 59, 60, 61, 62, 63].

For single-pass HHG, two difficulties emerge when the high-harmonic generation
(HHG) flux is scaled up by increasing the average power of such high-
repetition-rate driving lasers. In a typical attosecond beamline, the incident
laser beam is divided into a driving beam for HHG and a probe beam used in
extreme ultraviolet - infrared (XUV-IR) pump-probe schemes for either the
temporal characterization of attosecond pulses or for studying dynamics in the
attosecond regime. The first challenge is to remove the high-average-power
residual laser after generation process without attenuating the attosecond
pulses drastically. Conventionally, a metal foil with a thickness of a few
hundred nanometers is used to block the residual driving laser, allowing the
transmission of the attosecond pulses with some losses. This method fails when
the laser power increases because the thin foil is destroyed. The second
difficulty arises due to the probe beam, the energy of which should be high
enough for probing the system, such as what is needed for temporal
characterization of the attosecond pulses. Most energy of the laser is given
to the driving beam and only a small portion is in the probe beam.
Conventionally, a holey mirror is used to combine the high-order harmonics
which transmits through the central hole and the probe beam which is
reflected. Because of a prominent energy loss due to the central hole, the
energy of the probe beam available at target is even lower.

Figure 1:  Typical energies per shot of the high-order harmonics at the
repetition rates ranging from 10 Hz to 200 MHz. Hollow symbols show the
attosecond pulses (AP) with measured attosecond duration. Solid symbols show
high harmonics (HH) without temporal characterization.
Circle/Square/Diamond/Triangle represent the technologies used: Fiber-
CPA/Ti:Sapphire/OPA/Cavity. The three blue lines show the photon numbers at 40
eV at different repetition rates. The black/green/red colors differentiate by
the photon energy range covered.

In this work, by shaping the driving laser to an annular beam, we present a
record-high APT energy in our 100-kHz attosecond beamline and demonstrate the
advantages of utilizing annular beams to generate and characterize attosecond
pulses with high average power IR laser beams. We show a proper technique with
which the residual annular driving beam can be easily filtered out after HHG.
We use an IR probe beam which is also annular at the holey recombination
mirror. In this way it can be combined with the attosecond pulses with low
loss, so an even bigger fraction of the driving laser energy can be used for
HHG, which altogether results in a higher XUV flux. With this configuration we
demonstrate 51 pJ energy of the 100 kHz attosecond pulses at the experimental
target position. As shown in Figure 1, to the best of our knowledge this is
the highest energy of attosecond pulses with the temporal characterization at
the target achieved with high-repetition-rate systems. The $19.0\%$
transmission rate from generation to target is also the highest rate achieved
so far in cases of using a high average power laser in the 100 W regime. The
full width at half-maximum (FWHM) duration of the APTs was measured to be 166
as.

##  2 Methods

We used a 100-kHz fiber laser system as the input to this beamline to drive
the HHG process (see Section 1 in the supplementary material for the details).

Figure 2:  Schematic representation of attosecond pulse generation and
measurement using an annular beam. (a) Ray tracing of the beam path. HSM:
holey splitting mirror. FM: focusing mirror. HDM: holey dump mirror. HRM:
holey recombination mirror. The red beam is the fundamental laser beam. The
blue beam is the high-order harmonic beam. Figures (b) to (d) show the
simulated electric field of the generating pulsed laser beam at different
positions. (b) in the front of the FM, (c) at focus, (d) in the front of the
HDM. (e) is the electric field of the probe beam at the HRM. TOF: time-of-
flight electron spectrometer. PD: photodiode. The hole sizes of the HSM, HDM
and HRM are the same (6 mm in diameter).

Figure 2 is a schematic of the beamline analyzed in this work which
illustrates the XUV-IR pump-probe configuration. Figure 2(a) shows the results
of ray tracing of the beam path based on geometric optics. Figures 2(b) to (e)
show the electric field calculated using the paraxial wave equation at
different positions. Red and blue colors represent positive and negative
amplitudes respectively, while the white color show zero amplitude. The laser
beam is magnified to 11 mm FWHM before reaching the beamline (blue arrow in
Figure 2). A holey splitting mirror (HSM) splits the input laser into a
reflected annular beam (generation beam) and a transmitted central beam (probe
beam). The generation beam is focused by the focusing mirror (FM) with the
focusing length of 0.9 m onto a spot at the gas cell to generate high-order
harmonics, as shown in Figure 2(c). The gas cell is a home-made water-cooled
gas cavity designed to be used in combination with high average power laser
beams [64]. This generation beam propagates to an annular shape with a hole in
the center after the HHG shown in both Figure 2(a) and (d), so it can be
reflected off fully by another holey mirror, the holey dump mirror (HDM). The
XUV goes through the center without any attenuation. It must be noted that
based on ray tracing the generation beam is perfectly annular everywhere
except at the focus in Figure 2(a). However, wave propagation gives a
different behavior. The shape of the generation beam is a diffraction pattern
evolving along the beam path with substantial energy in the center. A typical
pattern is shown in Figure 2(b). The perfect annular shape with no energy in
the center can be observed only in a small range, which is the suitable place
for the HDM. The probe beam goes through the central hole of the HSM. After
the delay stage and the telescope, the magnified and delayed IR beam is
combined with the high-order harmonics using a holey recombination mirror
(HRM). As indicated in Figure 2(a), there is some energy loss in the probe
beam after the HRM as the transmitted central part is lost through the hole.
However, wave propagation predicts an annular shape of the probe beam at the
HRM, shown in Figure 2(e), therefore the energy loss can be avoided. After the
recombination, the high-order harmonics and the probe beam are focused into
the time-of-flight (TOF) electron spectrometer for the temporal
characterization of the XUV. An XUV photodiode (PD) and an XUV spectrometer
placed after the TOF are used to measure the energy and the spectrum of the
XUV. The XUV beam path and the measurement of the flux can be found in the
supplementary material (Figure S1 and S2 in section 2).

The propagation of the infrared beam is simulated using the Huygens-Fresnel
integral and paraxial wave equation without the source term in free space [65,
66](See section 3 for the details in Supplementary Material). While analyzing
the spatial profile of the beams, we found that in the studied aspects a
monochromatic beam and a pulse yield the same conclusion, so in the following
we will only consider the monochromatic beam for simplicity. In the following,
we will describe the evolution of the generation beam and the probe beam based
on the wave equation, and show the proper arrangement of optics for dumping
the generation beam and for recombining the XUV and probe beams.

##  3 Results

###  3.1 Generation Beam (XUV Arm)

In order to block the high-average-power residual generation beam, two methods
have been proposed so far for high average power driven HHG: (i) one method is
to use plates with special coating to reflect the attosecond pulses and
transmit the driving laser. The reflection of two fused silica plates is as
low as $17\%$ at 30 eV, and the attosecond pulses retain only $10\%$ of their
energy after filtering [21]. Furthermore, the coating must be individually
designed to fit the laser spectrum, and it is challenging especially in the
case of few-cycle lasers with a broad spectrum. The other method (ii) is to
use an annular beam to generate high-order harmonics [67, 68]. The annular
beam converges at focus to generate high harmonics, and becomes annular again,
so it can be reflected off easily by a holey mirror or blocked by a holey
plate after HHG. Attosecond pulses have already been generated and
characterized using this approach with a 1 kHz laser [69] having a much lower
average power (1 W). Generally, a small portion of the driving laser always
co-propagates with the XUV, so the residual driving laser beam cannot be fully
blocked. In case a low power laser is used, this small portion can be
neglected. However, with the increase of laser power, this portion will become
stronger, and it must be considered. In the high-repetition-rate regime of
$\sim 100$ kHz, several laboratories have used annular laser beams to generate
high-order harmonics [27, 70], while the measurement of the attosecond
temporal duration was only reported in our previous work at ELI-ALPS [52].

Figure 3:  (Simulation) (a) Beam propagation after the gas cell using paraxial
wave equation. z = 0 is set as the image plane of the FM with the HSM as the
object. (b) Radial intensity profile of the beam at different z positions.
Blue: z = 0, the image plane of the HSM. Black: z = 0.2 m. Red: z = -0.2 m.
The positions of z = $\pm$ 0.2 m are marked with the white arrows in (a). (c)
The central part of the beam ( around r = 0 ). (d) The intensity distribution
inside the hole.

Figure 3 shows beam propagation after the gas cell for a monochromatic beam of
1030-nm wavelength. In Figure 3(a) we observe that at the positions of
$z<-0.2$ m the beam profile considerably differs from the geometrically
expected central shadow, exhibiting substantial intensity of diffraction
rings. In the range from z = -2 m to z = 0, the beam evolves gradually from
the diffraction pattern to an ideal annular beam as predicted by geometrical
optics. At the position of z = 0, i.e., the image plane of the HSM (the FM
mirror serving as the imaging optic), the beam has a perfect annular shape
with no light in the center, as can be seen in Figure 3(c) and (d). After
further propagation towards $z>0$, the beam again shows substantial
diffraction in its profile. In Figure 3(a) and (b) we observe the Arago spot
before and after the image plane of the HSM ( z = 0 ). In addition, at the
position of z = 0 there is a circular on-axis area of the beam with no light
inside it, as shown in Figure 3(c) and (d). However, when z = 0.2 m and z =
–0.2 m, a considerable amount of light can still be observed in the center.
When low average power laser beams are used [69], the Arago spot and the
diffraction rings do not have sufficient intensity to cause practical issues.
However, when the average power of the laser is increased, these diffraction
rings must be considered, since they can damage the optical elements and
detectors and can produce unwanted noise in the signal. In order to reflect
such a beam fully with a holey mirror, the mirror must be put at the image
plane. Practically, the mirror must be placed in the diamond-shaped dark area
in Figure 3(a). In our case, the HDM can be located within 10 cm around z = 0.
We have also simulated the cases when focusing mirrors with different focal
lengths between 0.5 m and 3 m are used. The results have shown that the
optimal position range is not directly related to the focal length, and it is
between 10 cm to 30 cm in the case of all studied focal lengths. Researchers
aiming to design such a beamline must simulate beam propagation using the wave
equation instead of ray tracing to find the appropriate range, and must
experimentally measure the beam profile to check the correctness of the
positions. In order to directly record the beam profile, we put a CMOS sensor
at the position of the HDM, and measured the beam profile at low power (1 W)
and atmospheric pressure. As shown in Figure 4(a), the beam is perfectly
annular, so it can be fully reflected by the HDM.

In the above discussion, we only considered the pulse propagation in vacuum.
If the spatio-temporal distribution of the generation beam is not prominently
changed by the medium, i.e., from all the possible nonlinear effects only the
process of HHG takes place, the conclusion of this work is valid without
restrictions. In real experiments, if HHG works under the usual phase matching
conditions, where the ionization is lower than the critical ionization rate
(usually less than few percent) [71], the driving laser can be considered
unmodified by the gas, and our conclusions are not affected by these effects.
However, in the case of high ionization, the shape of the laser beam during
propagation will be modified by the electrons in the medium [33, 72, 73, 74],
and the far-field shape of the laser beam profile is expected to change
relevantly. To analyze the effect of ionization of the generation medium, we
carried out simulations. To analyze the effect of ionization of the generation
medium, we carried out additional simulations (See section 4 in Supplementary
Material for the details). In the simulations, a gas cell with 4-mm length and
1.2-mm diameter was put at the position of the laser focus, matching the
experimental conditions. We changed the pressure of argon and calculated the
beam profile at the position of the HDM in Figure 2. As shown in Figure 5, by
increasing the pressure, the transmitted energy also increases. Using the same
parameters as in the experiments, when the pressure is $\sim$ 200 mbar in the
gas cell, the transmitted energy is below $1\%$. In the case of using 100-W
laser, the transmitted power is below 1 W (the same level of 1-kHz system) and
can be safely blocked by a metallic filter. At higher pressure of $\sim$ 500
mbar and higher free-electron density, the transmitted energy is still below
$2\%$. It should be noted that there is a certain percentage of beam energy
always transmitted through the hole of the HDM because the cell aperture acts
as a spatial filter distorting ideal imaging conditions (see details in
section 4 for the Supplementary material). Also, the almost unchanged
transmission percentages ($t$) and beam profiles in Figure 3 up to a medium
pressure of $p_{Ar}\sim$ 10 mbar suggests that at these pressures with our
focused laser intensities the situation is identical to propagating in vacuum.
In a recent theoretical work, Cheng Jin et al. also investigated HHG in the
overdriven regime (high ionization) using an annular beam and indicated that
XUV and IR can separate in the far field [75].

Figure 4:  (Experiment) (a) The measured beam profile of the residual
generation beam at the HDM using a CMOS camera. (b) The beam profile recorded
on an IR card of the probe beam in front of the HRM.  Figure 5:  (Simulation)
The beam profile of the driving laser at the HDM (see position in Figure 2) at
different pressures of Ar medium ($p_{Ar}$) in the gas cell. The ideal imaging
(dashed gray curve) shows the case of free propagation in vacuum without
diffraction on any obstacle. The transmission percentages (t) in the legend
provide the ratio of the beam energy transmitted through the central hole of
the HDM with respect to the input beam energy reaching the beamline. The solid
blue curve corresponds to the experimental case.

###  3.2 Probe Beam (IR Arm)

Figure 6:  (Simulation) (a) The amplitude of the probe beam after the HSM with
a 6 mm diameter hole. L1 = 0 is the position of the HSM shown in Figure 2. The
telescope is positioned to image an annular part of the propagating beam. The
entrance of the telescope is around the position of L1 = 4.1 m. (b) The
amplitude of the annular probe beam after the telescope. L2 = 0 is the exit of
the telescope, also shown in Figure 2. The amplitudes in the two pictures are
in linear scale.

To match the focus of the attosecond and IR pulses in the pump-probe setup,
the probe beam is recombined with the attosecond pulses by a holey mirror. The
attosecond pulses propagate through the central hole, while the IR is
reflected. This scheme wastes the central part of the IR probe causing a
relevant loss in its energy during recombination. Although the average power
of the laser is high, the energy of the individual pulses is low (below mJ or
even 100 $\mu$J in most of the currently available systems). In order to
perform a reconstruction of attosecond beating by interference of two-photon
transitions (RABBITT) [1] or a streaking measurement [2], the laser intensity
of the probe pulse must be above $10^{11}$ Wcm-2, so these losses need to be
minimized.

In our beamline, shown in Figure 2(a), the probe beam is magnified by a
telescope and then combined with the high-order harmonics using the HRM, where
a substantial amount of energy in the center would be lost. However,
diffraction allows for system optimization. As shown in Figure 2(a), the
transmitted probe beam from the HSM (L1 = 0) evolves as a diffraction pattern.
The central intensity exhibits an oscillating behavior along the laser
propagation direction. For certain positions of the HRM, e.g., at z = 3 m,
most energy would be lost through the hole. However, by positioning it at z =
4.5 m, almost the entire energy of the beam could be preserved after
reflection, since the transmitted central part is a hole with a low portion of
energy. However, as the position of HRM cannot be set completely arbitrarily
in most beamlines, therefore we use a telescope to position the annular
profile of the probe beam to a suitable geometrical position.

We build the telescope at the position of L1 = 4.1 m in Figure 6(a), where the
probe beam exhibits an annular shape. The telescope has threefold
magnification, and the propagation distance is virtually reduced by 70 cm as a
result of imaging. L2 = 0 is defined as the output of the telescope in Figure
6(b) (see also Figure 2). The magnified beam propagates further and keeps its
annular shape within 2 meters, as shown in Figure 6(b). Figure 4(b) shows the
annular beam profile of the probe beam on an IR card in front of a holey
mirror. In our experiment the hole diameter of HRM was 6 mm, and the loss due
to reflection was $15\%$. According to our simulations, this loss can be
decreased to $3.5\%$ by reducing the hole diameter to 4 mm.

###  3.3 Attosecond Pulse Duration Measurement

To demonstrate the performance of our system optimized according to the
description above, high-order harmonics were generated in a 4-mm gas cell
filled with 200-mbar argon gas. The generated harmonic beam propagates through
a 100-nm aluminum (Al) foil, and combines with the delayed IR beam. The two
beams are focused to ionize neon (Ne) gas from a gas jet in front of a TOF
spectrometer that collects the emitted electrons. By changing the delay
between the two beams we can record the delay dependent electron kinetic
energy spectrogram, i.e., RABBITT trace, shown in Fig. 7(a). The photon energy
covered by the APT was between 30 eV and 70 eV. As a result, the electron
kinetic energies were ranging from 8 eV to 48 eV, obtained by subtracting the
21.56-eV ionization potential of Ne. The whole temporal range of the trace is
approximately $\sim$ 70 fs, which is consistent with the 40-fs duration (FWHM)
of the independent measurement of the driving laser [52]. The reconstruction
gave an average FWHM duration of 166$\pm$12 as of the attosecond pulses in the
APT, as shown in Fig. 7(b) (blue line). After the TOF, a photodiode was
inserted in the beam path to measure the energy of high-order harmonics at the
target position. The pulse energy was measured to be 51.0$\pm$3.1 pJ. The
pulse energy at generation was calculated to be 269.0 pJ. Further details of
the beamline and the laser system can be found in our previous works [68, 52].
Details of the energy measurement of the high-order can be found in the Figure
S1 and Figure S2 in Section 2 of the supplementary material. Compared to our
previous work [52], we optimized the experimental conditions by using a new
water-cooled gas cell, an extensive parametric optimization of phase matching
conditions and improving the stability of the laser. We reached a five-fold
decrease in necessary integration time while also improving the signal-to-
noise ratio. These improved results demonstrate the possibility to carry out
attosecond pump-probe measurements at 100 kHz repetition rate with our
beamline at XUV fluxes not available before.

Figure 7:  (Experiment) (a) Measured RABBITT trace (linear scale). (b) The
FWHMs of the fourier-transform-limited pulse(red) and the reconstructed pulse
(blue). Both the red line and blue line were normalized to the peak of the red
line.

##  4 Conclusion

In conclusion, in this work we have reported the generation and temporal
characterization of attosecond pulses using the high average power HR laser of
ELI-ALPS. In our approach, a holey mirror is used to split the laser into two
independent beams. The reflected annular beam is used for attosecond pulse
generation, while the transmitted central part serves as the probe beam for
experiments and temporal characterization. After HHG, the generation beam
becomes annular again upon further propagation. As predicted by wave optics,
if a holey mirror is placed at a proper position, the residual annular IR beam
can be almost fully reflected, and the harmonics can transmit through the
central hole. This way the possible damage of the optics and detectors by the
residual generating IR beam can be avoided, and the unwanted background in the
signal can also be suppressed. Since the diffracted central probe beam also
becomes annular, it can be recombined with the XUV beam via reflection on a
holey mirror with minimal energy loss, provided that this mirror is placed at
the correct position. This ensures a sufficiently intense probe beam for pump-
probe experiments such as RABBITT or streaking measurements. These assumptions
have been verified both by experiments and simulations, and have provided
guidance in finding the proper positions of the key mirrors in our beamline.
As a result, we could optimize HHG and delivered 51.0 pJ attosecond pulse
trains with an average duration of 166 as to the target position after
transmission through a 100-nm thick Al metal foil. This is the highest pulse
energy of attosecond pulses with temporal characterization achieved so far on
target using a laser with a repetition rate higher than 10 kHz and an average
power in 100 W regime. As a future step, we plan to compress the laser pulses
further to a few-cycle duration [76]. We expect that this approach will even
increase the conversion efficiency and the flux of the attosecond pulses. With
this 100-kHz high-energy attosecond pulse, we believe that many experiments
which need both the high repetition rate and enough energy can be performed
now, especially for the studies of solid samples and big molecules.

## Acknowledgments

We thank the groups of Mauro Nisoli and Luca Poletto for the help provided in
developing the beamline and the discussions. We also thank Harshitha Nandiga
Gopalakrishna, Miklos Füle, and Amelle Zaïr for the early contributions to the
implementation of this beamline. We are grateful to Valer Tosa for the
availability of the macroscopic high-harmonic generation simulation code. We
acknowledge KIFÜ for awarding us high-performance computing access to resource
based in Hungary.

### Author Contributions

P.Y., with the input from B.M., conceived the idea. K.V. initiated the project
of the beamline. B.M., S.K., and K.V. supervised the project. P.Y, L.G.O.,
T.C., Z.F., T.G., and B.M. operated the beamline and performed the
experiments. P.Y. and B.M. performed the simulations, and with L.G.O. analyzed
the experiment results. P.J., I.S., Z.B., and B.G. operated the laser system.
P.Y. wrote the manuscript with the inputs from B.M, S.K., and K.V. All people
contributed to the preparation of the manuscript.

### Funding

The ELI-ALPS project (GINOP-2.3.6-15-2015-00001) is supported by the European
Union and co-financed by the European Regional Development Fund.

### Conflicts of Interest

The authors declare no conflict of interest.

### Data Availability

All data discussed in the article will be made available upon request.

## Supplementary Materials

Section 1: The details of the laser system. Section 2: The beampath of the XUV
path (Figure S-1) and the flux measruement (Figure S-2). Section 3: The
simulation of beam propagation in free space. Section 4: The simulation of
beam propagation in ionized gas and the aperture effect.

## References

  * [1] P. M. Paul, E. S. Toma, P. Breger, G. Mullot, F. Auge, P. Balcou, H. G. Muller, and P. Agostini, “Observation of a train of attosecond pulses from high harmonic generation,” Science, vol. 292, no. 5522, pp. 1689–1692, 2001\.
  * [2] M. Hentschel, R. Kienberger, C. Spielmann, G. Reider, N. Milosevic, T. Brabec, P. Corkum, U. Heinzmann, M. Drescher, and F. Krausz, “Attosecond metrology,” Nature (London), vol. 414, no. 6863, pp. 509–513, 2001.
  * [3] J. Itatani, J. Levesque, D. Zeidler, H. Niikura, H. Pépin, J.-C. Kieffer, P. B. Corkum, and D. M. Villeneuve, “Tomographic imaging of molecular orbitals,” Nature, vol. 432, no. 7019, pp. 867–871, 2004.
  * [4] T. T. Luu, Z. Yin, A. Jain, T. Gaumnitz, Y. Pertot, J. Ma, and H. J. Wörner, “Extreme–ultraviolet high–harmonic generation in liquids,” Nature Communications, vol. 9, no. 1, pp. 1–10, 2018.
  * [5] S. Ghimire and D. A. Reis, “High-harmonic generation from solids,” Nature Physics, vol. 15, no. 1, pp. 10–16, 2019.
  * [6] I. Pupeza, C. Zhang, M. Högner, and J. Ye, “Extreme-ultraviolet frequency combs for precision metrology and attosecond science,” Nature Photonics, vol. 15, no. 3, pp. 175–186, 2021.
  * [7] S. Hädrich, J. Rothhardt, M. Krebs, S. Demmler, A. Klenke, A. Tünnermann, and J. Limpert, “Single-pass high harmonic generation at high repetition rate and photon flux,” Journal of Physics B: Atomic, Molecular and Optical Physics, vol. 49, no. 17, p. 172002, 2016.
  * [8] W. Zheng, P. Jiang, L. Zhang, Y. Wang, Q. Sun, Y. Liu, Q. Gong, and C. Wu, “Ultrafast extreme ultraviolet photoemission electron microscope,” Review of Scientific Instruments, vol. 92, no. 4, p. 043709, 2021.
  * [9] L. Frasinski, V. Zhaunerchyk, M. Mucke, R. J. Squibb, M. Siano, J. H. Eland, P. Linusson, P. Vd Meulen, P. Salén, R. Thomas, et al., “Dynamics of hollow atom formation in intense x-ray pulses probed by partial covariance mapping,” Physical Review Letters, vol. 111, no. 7, p. 073002, 2013.
  * [10] L. Cattaneo, J. Vos, R. Y. Bello, A. Palacios, S. Heuser, L. Pedrelli, M. Lucchini, C. Cirelli, F. Martín, and U. Keller, “Attosecond coupled electron and nuclear dynamics in dissociative ionization of H2,” Nature Physics, vol. 14, no. 7, pp. 733–738, 2018.
  * [11] J. Miao, T. Ishikawa, I. K. Robinson, and M. M. Murnane, “Beyond crystallography: Diffractive imaging using coherent x-ray light sources,” Science, vol. 348, no. 6234, pp. 530–535, 2015.
  * [12] A. Johnson, L. Miseikis, D. Wood, D. Austin, C. Brahms, S. Jarosch, C. Strüber, P. Ye, and J. Marangos, “Measurement of sulfur L2,3 and carbon k edge xanes in a polythiophene film using a high harmonic supercontinuum,” Structural Dynamics, vol. 3, no. 6, p. 062603, 2016.
  * [13] K. Ramasesha, S. R. Leone, and D. M. Neumark, “Real-time probing of electron dynamics using attosecond time-resolved spectroscopy,” Annual Review of Physical Chemistry, vol. 67, pp. 41–63, 2016.
  * [14] T. Gorkhover, S. Schorb, R. Coffee, M. Adolph, L. Foucar, D. Rupp, A. Aquila, J. D. Bozek, S. W. Epp, B. Erk, et al., “Femtosecond and nanometre visualization of structural dynamics in superheated nanoparticles,” Nature Photonics, vol. 10, no. 2, pp. 93–97, 2016.
  * [15] J. Rivera-Dean, P. Stammer, E. Pisanty, T. Lamprou, P. Tzallas, M. Lewenstein, and M. F. Ciappina, “New schemes for creating large optical schrödinger cat states using strong laser fields,” Journal of Computational Electronics, pp. 1–13, 2021.
  * [16] M. Lewenstein, M. Ciappina, E. Pisanty, J. Rivera-Dean, P. Stammer, T. Lamprou, and P. Tzallas, “Generation of optical schrödinger cat states in intense laser–matter interactions,” Nature Physics, vol. 17, no. 10, pp. 1104–1108, 2021.
  * [17] A. Nayak, I. Orfanos, I. Makos, M. Dumergue, S. Kühn, E. Skantzakis, B. Bodi, K. Varju, C. Kalpouzos, H. Banks, et al., “Multiple ionization of argon via multi-xuv-photon absorption induced by 20-gw high-order harmonic laser pulses,” Physical Review A, vol. 98, no. 2, p. 023426, 2018.
  * [18] H. Wang, Y. Xu, S. Ulonska, J. S. Robinson, P. Ranitovic, and R. A. Kaindl, “Bright high-repetition-rate source of narrowband extreme-ultraviolet harmonics beyond 22 ev,” Nature Communications, vol. 6, no. 1, pp. 1–7, 2015.
  * [19] R. Klas, S. Demmler, M. Tschernajew, S. Hädrich, Y. Shamir, A. Tünnermann, J. Rothhardt, and J. Limpert, “Table-top milliwatt-class extreme ultraviolet high harmonic light source,” Optica, vol. 3, no. 11, pp. 1167–1170, 2016.
  * [20] A. Comby, D. Descamps, S. Beauvarlet, A. Gonzalez, F. Guichard, S. Petit, Y. Zaouter, and Y. Mairesse, “Cascaded harmonic generation from a fiber laser: a milliwatt xuv source,” Optics Express, vol. 27, no. 15, pp. 20383–20396, 2019.
  * [21] S. Hädrich, A. Klenke, J. Rothhardt, M. Krebs, A. Hoffmann, O. Pronin, V. Pervak, J. Limpert, and A. Tünnermann, “High photon flux table-top coherent extreme-ultraviolet source,” Nature Photonics, vol. 8, no. 10, pp. 779–783, 2014.
  * [22] R. Klas, A. Kirsche, M. Gebhardt, J. Buldt, H. Stark, S. Hädrich, J. Rothhardt, and J. Limpert, “Ultra-short-pulse high-average-power megahertz-repetition-rate coherent extreme-ultraviolet light source,” PhotoniX, vol. 2, no. 1, pp. 1–8, 2021.
  * [23] S. Hädrich, M. Krebs, J. Rothhardt, H. Carstens, S. Demmler, J. Limpert, and A. Tünnermann, “Generation of $\mu$w level plateau harmonics at high repetition rate,” Optics Express, vol. 19, no. 20, pp. 19374–19383, 2011\.
  * [24] E. Lorek, E. W. Larsen, C. M. Heyl, S. Carlström, D. Paleček, D. Zigmantas, and J. Mauritsson, “High-order harmonic generation using a high-repetition-rate turnkey laser,” Review of Scientific Instruments, vol. 85, no. 12, p. 123106, 2014.
  * [25] J. Rothhardt, S. Hädrich, Y. Shamir, M. Tschnernajew, R. Klas, A. Hoffmann, G. K. Tadesse, A. Klenke, T. Gottschall, T. Eidam, et al., “High-repetition-rate and high-photon-flux 70 eV high-harmonic source for coincidence ion imaging of gas-phase molecules,” Optics Express, vol. 24, no. 16, pp. 18133–18147, 2016.
  * [26] A. Harth, C. Guo, Y.-C. Cheng, A. Losquin, M. Miranda, S. Mikaelsson, C. M. Heyl, O. Prochnow, J. Ahrens, U. Morgner, et al., “Compact 200 kHz hhg source driven by a few-cycle OPCPA,” Journal of Optics, vol. 20, no. 1, p. 014007, 2017.
  * [27] R. Klas, A. Kirsche, M. Tschernajew, J. Rothhardt, and J. Limpert, “Annular beam driven high harmonic generation for high flux coherent xuv and soft x-ray radiation,” Optics Express, vol. 26, no. 15, pp. 19318–19327, 2018\.
  * [28] M. Keunecke, C. Möller, D. Schmitt, H. Nolte, G. M. Jansen, M. Reutzel, M. Gutberlet, G. Halasi, D. Steil, S. Steil, et al., “Time-resolved momentum microscopy with a 1 MHz high-harmonic extreme ultraviolet beamline,” Review of Scientific Instruments, vol. 91, no. 6, p. 063905, 2020.
  * [29] C.-T. Chiang, M. Huth, A. Trützschler, M. Kiel, F. O. Schumann, J. Kirschner, and W. Widdra, “Boosting laboratory photoelectron spectroscopy by megahertz high-order harmonics,” New Journal of Physics, vol. 17, no. 1, p. 013035, 2015.
  * [30] E. Takahashi, Y. Nabekawa, and K. Midorikawa, “Generation of 10-$\mu$J coherent extreme-ultraviolet light by use of high-order harmonics,” Optics Letters, vol. 27, no. 21, pp. 1920–1922, 2002.
  * [31] E. J. Takahashi, Y. Nabekawa, and K. Midorikawa, “Low-divergence coherent soft x-ray source at 13 nm by high-order harmonics,” Applied Physics Letters, vol. 84, no. 1, pp. 4–6, 2004.
  * [32] S. Hädrich, M. Krebs, A. Hoffmann, A. Klenke, J. Rothhardt, J. Limpert, and A. Tünnermann, “Exploring new avenues in high repetition rate table-top coherent extreme ultraviolet sources,” Light: Science & Applications, vol. 4, no. 8, pp. e320–e320, 2015.
  * [33] A. S. Johnson, D. R. Austin, D. A. Wood, C. Brahms, A. Gregory, K. B. Holzner, S. Jarosch, E. W. Larsen, S. Parker, C. S. Strüber, et al., “High-flux soft x-ray harmonic generation from ionization-shaped few-cycle laser pulses,” Science Advances, vol. 4, no. 5, p. eaar3761, 2018.
  * [34] Y. Fu, K. Nishimura, R. Shao, A. Suda, K. Midorikawa, P. Lan, and E. J. Takahashi, “High efficiency ultrafast water-window harmonic generation for single-shot soft x-ray spectroscopy,” Communications Physics, vol. 3, no. 1, pp. 1–10, 2020.
  * [35] P.-A. Chevreuil, F. Brunner, S. Hrisafov, J. Pupeikis, C. R. Phillips, U. Keller, and L. Gallmann, “Water-window high harmonic generation with 0.8-$\mu$m and 2.2-$\mu$m OPCPAs at 100 kHz,” Optics Express, vol. 29, no. 21, pp. 32996–33008, 2021.
  * [36] M. Gebhardt, T. Heuermann, R. Klas, C. Liu, A. Kirsche, M. Lenski, Z. Wang, C. Gaida, J. Antonio-Lopez, A. Schülzgen, et al., “Bright, high-repetition-rate water window soft x-ray source enabled by nonlinear pulse self-compression in an antiresonant hollow-core fibre,” Light: Science & Applications, vol. 10, no. 1, pp. 1–7, 2021.
  * [37] M. Mero, Z. Heiner, V. Petrov, H. Rottke, F. Branchi, G. M. Thomas, and M. J. Vrakking, “43 W, 1.55 $\mu$m and 12.5 W, 3.1 $\mu$m dual-beam, sub-10 cycle, 100 kHz optical parametric chirped pulse amplifier,” Optics Letters, vol. 43, no. 21, pp. 5246–5249, 2018.
  * [38] T. Nagy, S. Hädrich, P. Simon, A. Blumenstein, N. Walther, R. Klas, J. Buldt, H. Stark, S. Breitkopf, P. Jójárt, et al., “Generation of three-cycle multi-millijoule laser pulses at 318 W average power,” Optica, vol. 6, no. 11, pp. 1423–1424, 2019.
  * [39] P. Storz, J. Tauch, M. Wunram, A. Leitenstorfer, and D. Brida, “Parametric amplification of phase-locked few-cycle pulses and ultraviolet harmonics generation in solids at high repetition rate,” Laser & Photonics Reviews, vol. 11, no. 6, p. 1700062, 2017.
  * [40] X. Zou, W. Li, S. Qu, K. Liu, H. Li, Q. J. Wang, Y. Zhang, and H. Liang, “Flat-top pumped multi-millijoule mid-infrared parametric chirped-pulse amplifier at 10 kHz repetition rate,” Laser & Photonics Reviews, p. 2000292, 2021.
  * [41] M. Natile, A. Golinelli, L. Lavenu, F. Guichard, M. Hanna, Y. Zaouter, R. Chiche, X. Chen, J. Hergott, W. Boutu, et al., “CEP-stable high-energy ytterbium-doped fiber amplifier,” Optics Letters, vol. 44, no. 16, pp. 3909–3912, 2019.
  * [42] L. Young, K. Ueda, M. Gühr, P. H. Bucksbaum, M. Simon, S. Mukamel, N. Rohringer, K. C. Prince, C. Masciovecchio, M. Meyer, et al., “Roadmap of ultrafast x-ray atomic and molecular physics,” The Journal of Physics B: Atomic, Molecular and Optical Physics, vol. 51, no. 3, p. 032003, 2018.
  * [43] S. Toth, T. Stanislauskas, I. Balciunas, R. Budriunas, J. Adamonis, R. Danilevicius, K. Viskontas, D. Lengvinas, G. Veitas, D. Gadonas, et al., “SYLOS lasers–the frontier of few-cycle, multi-TW, kHz lasers for ultrafast applications at extreme light infrastructure attosecond light pulse source,” Journal of Physics: Photonics, vol. 2, no. 4, p. 045003, 2020.
  * [44] S. Mikaelsson, J. Vogelsang, C. Guo, I. Sytcevich, A.-L. Viotti, F. Langer, Y.-C. Cheng, S. Nandi, W. Jin, A. Olofsson, et al., “A high-repetition rate attosecond light source for time-resolved coincidence spectroscopy,” Nanophotonics, vol. 1, no. ahead-of-print, 2020.
  * [45] E. Goulielmakis, M. Schultze, M. Hofstetter, V. S. Yakovlev, J. Gagnon, M. Uiberacker, A. L. Aquila, E. Gullikson, D. T. Attwood, R. Kienberger, et al., “Single-cycle nonlinear optics,” Science, vol. 320, no. 5883, pp. 1614–1617, 2008.
  * [46] E. J. Takahashi, P. Lan, O. D. Mücke, Y. Nabekawa, and K. Midorikawa, “Attosecond nonlinear optics using gigawatt-scale isolated attosecond pulses,” Nature Communications, vol. 4, no. 1, pp. 1–9, 2013.
  * [47] D. Fabris, T. Witting, W. Okell, D. Walke, P. Matia-Hernando, J. Henkel, T. Barillot, M. Lein, J. Marangos, and J. Tisch, “Synchronized pulses generated at 20 eV and 90 eV for attosecond pump–probe experiments,” Nature Photonics, vol. 9, no. 6, pp. 383–387, 2015.
  * [48] B. Manschwetus, L. Rading, F. Campi, S. Maclot, H. Coudert-Alteirac, J. Lahl, H. Wikmark, P. Rudawski, C. Heyl, B. Farkas, et al., “Two-photon double ionization of neon using an intense attosecond pulse train,” Physical Review A, vol. 93, no. 6, p. 061402, 2016.
  * [49] H. Timmers, M. Sabbar, J. Hellwagner, Y. Kobayashi, D. M. Neumark, and S. R. Leone, “Polarization-assisted amplitude gating as a route to tunable, high-contrast attosecond pulses,” Optica, vol. 3, no. 7, pp. 707–710, 2016\.
  * [50] S. L. Cousin, N. Di Palo, B. Buades, S. M. Teichmann, M. Reduzzi, M. Devetta, A. Kheifets, G. Sansone, and J. Biegert, “Attosecond streaking in the water window: A new regime of attosecond pulse characterization,” Physical Review X, vol. 7, no. 4, p. 041030, 2017.
  * [51] J. Li, A. Chew, S. Hu, J. White, X. Ren, S. Han, Y. Yin, Y. Wang, Y. Wu, and Z. Chang, “Double optical gating for generating high flux isolated attosecond pulses in the soft x-ray regime,” Optics Express, vol. 27, no. 21, pp. 30280–30286, 2019.
  * [52] P. Ye, T. Csizmadia, L. G. Oldal, H. N. Gopalakrishna, M. Füle, Z. Filus, B. Nagyillés, Z. Divéki, T. Grósz, M. Dumergue, et al., “Attosecond pulse generation at ELI-ALPS 100 kHz repetition rate beamline,” Journal of Physics B: Atomic, Molecular and Optical Physics, vol. 53, no. 15, p. 154004, 2020.
  * [53] M. Osolodkov, F. J. Furch, F. Schell, P. Šušnjar, F. Cavalcante, C. S. Menoni, C. P. Schulz, T. Witting, and M. J. Vrakking, “Generation and characterisation of few-pulse attosecond pulse trains at 100 kHz repetition rate,” Journal of Physics B: Atomic, Molecular and Optical Physics, vol. 53, no. 19, p. 194003, 2020.
  * [54] I. Makos, I. Orfanos, A. Nayak, J. Peschel, B. Major, I. Liontos, E. Skantzakis, N. Papadakis, C. Kalpouzos, M. Dumergue, et al., “A 10-gigawatt attosecond source for non-linear XUV optics and XUV-pump-XUV-probe studies,” Scientific Reports, vol. 10, no. 1, pp. 1–18, 2020.
  * [55] T. Witting, M. Osolodkov, F. Schell, F. Morales, S. Patchkovskii, P. Susnjar, F. Cavalcante, C. S. Menoni, C. P. Schulz, F. J. Furch, et al., “Generation and characterisation of isolated attosecond pulses at 100 kHz repetition rate,” Optica, accepted, DOI 10.1364/OPTICA.443521, 2021.
  * [56] A. Cingöz, D. C. Yost, T. K. Allison, A. Ruehl, M. E. Fermann, I. Hartl, and J. Ye, “Direct frequency comb spectroscopy in the extreme ultraviolet,” Nature, vol. 482, no. 7383, pp. 68–71, 2012.
  * [57] I. Pupeza, S. Holzberger, T. Eidam, H. Carstens, D. Esser, J. Weitenberg, P. Rußbüldt, J. Rauschenberger, J. Limpert, T. Udem, et al., “Compact high-repetition-rate source of coherent 100 eV radiation,” Nature Photonics, vol. 7, no. 8, pp. 608–612, 2013.
  * [58] A. Ozawa, Z. Zhao, M. Kuwata-Gonokami, and Y. Kobayashi, “High average power coherent vuv generation at 10 MHz repetition frequency by intracavity high harmonic generation,” Optics Express, vol. 23, no. 12, pp. 15107–15118, 2015.
  * [59] G. Porat, C. M. Heyl, S. B. Schoun, C. Benko, N. Dörre, K. L. Corwin, and J. Ye, “Phase-matched extreme-ultraviolet frequency-comb generation,” Nature Photonics, vol. 12, no. 7, pp. 387–391, 2018.
  * [60] T. Saule, S. Heinrich, J. Schötz, N. Lilienfein, M. Högner, O. deVries, M. Plötner, J. Weitenberg, D. Esser, J. Schulte, et al., “High-flux ultrafast extreme-ultraviolet photoemission spectroscopy at 18.4 MHz pulse repetition rate,” Nature Communications, vol. 10, no. 1, pp. 1–10, 2019.
  * [61] J. Lee, D. R. Carlson, and R. J. Jones, “Optimizing intracavity high harmonic generation for xuv fs frequency combs,” Optics Express, vol. 19, no. 23, pp. 23315–23326, 2011.
  * [62] H. Carstens, M. Högner, T. Saule, S. Holzberger, N. Lilienfein, A. Guggenmos, C. Jocher, T. Eidam, D. Esser, V. Tosa, et al., “High-harmonic generation at 250 MHz with photon energies exceeding 100 eV,” Optica, vol. 3, no. 4, pp. 366–369, 2016.
  * [63] C. Corder, P. Zhao, J. Bakalis, X. Li, M. D. Kershis, A. R. Muraca, M. G. White, and T. K. Allison, “Ultrafast extreme ultraviolet photoemission without space charge,” Structural Dynamics, vol. 5, no. 5, p. 054301, 2018\.
  * [64] Z. Filus, “Liquid-cooled modular gas target cell system for high-order harmonic generation using high average power laser systems,” In preparation, 2021.
  * [65] A. E. Siegman, “Lasers,” Mill Valley, CA, vol. 37, no. 208, p. 661, 1986\.
  * [66] P. Ye, H. Teng, X.-K. He, S.-Y. Zhong, L.-F. Wang, M.-J. Zhan, W. Zhang, C.-X. Yun, and Z.-Y. Wei, “Minimizing the angular divergence of high-order harmonics by truncating the truncated bessel beam,” Physical Review A, vol. 90, no. 6, p. 063808, 2014.
  * [67] J. Peatross, J. Chaloupka, and D. Meyerhofer, “High-order harmonic generation with an annular laser beam,” Optics Letters, vol. 19, no. 13, pp. 942–944, 1994.
  * [68] S. Kühn, M. Dumergue, S. Kahaly, S. Mondal, M. Füle, T. Csizmadia, B. Farkas, B. Major, Z. Várallyay, E. Cormier, et al., “The ELI-ALPS facility: the next generation of attosecond sources,” The Journal of Physics B: Atomic, Molecular and Optical Physics, vol. 50, no. 13, p. 132002, 2017.
  * [69] Y. Mairesse, A. De Bohan, L. Frasinski, H. Merdji, L. Dinu, P. Monchicourt, P. Breger, M. Kovačev, R. Taïeb, B. Carré, et al., “Attosecond synchronization of high-harmonic soft x-rays,” Science, vol. 302, no. 5650, pp. 1540–1543, 2003.
  * [70] T. Gaumnitz, A. Jain, and H. J. Wörner, “Extreme-ultraviolet high-order harmonic generation from few-cycle annular beams,” Optics Letters, vol. 43, no. 18, pp. 4506–4509, 2018.
  * [71] Z. Chang, Fundamentals of attosecond optics.  CRC press, 2016.
  * [72] D. Rivas, B. Major, M. Weidman, W. Helml, G. Marcus, R. Kienberger, D. Charalambidis, P. Tzallas, E. Balogh, K. Kovács, et al., “Propagation-enhanced generation of intense high-harmonic continua in the 100-eV spectral region,” Optica, vol. 5, no. 10, pp. 1283–1289, 2018\.
  * [73] B. Major, K. Kovács, V. Tosa, P. Rudawski, A. L’Huillier, and K. Varjú, “Effect of plasma-core-induced self-guiding on phase matching of high-order harmonic generation in gases,” Journal of the Optical Society of America B, vol. 36, no. 6, pp. 1594–1601, 2019.
  * [74] B. Major, M. Kretschmar, O. Ghafur, A. Hoffmann, K. Kovács, K. Varjú, B. Senfftleben, J. Tümmler, I. Will, T. Nagy, et al., “Propagation-assisted generation of intense few-femtosecond high-harmonic pulses,” Journal of Physics: Photonics, vol. 2, no. 3, p. 034002, 2020\.
  * [75] C. Jin, X. Tang, B. Li, K. Wang, and C. Lin, “Optimal spatial separation of high-order harmonics from infrared driving lasers with an annular beam in the overdriven regime,” Physical Review Applied, vol. 14, no. 1, p. 014057, 2020.
  * [76] E. Shestaev, D. Hoff, A. Sayler, A. Klenke, S. Hädrich, F. Just, T. Eidam, P. Jójárt, Z. Várallyay, K. Osvay, et al., “High-power ytterbium-doped fiber laser delivering few-cycle, carrier-envelope phase-stable 100 $\mu$J pulses at 100 kHz,” Optics Letters, vol. 45, no. 1, pp. 97–100, 2020.


11institutetext: University of Glasgow, School of Computing Science, UK
11email<EMAIL_ADDRESS>
{roderick.murray-smith<EMAIL_ADDRESS>
# Learning a low dimensional manifold of real cancer tissue with PathologyGAN

Adalberto Claudio Quiros 11 0000-0003-4804-0741 Roderick Murray-Smith 11
0000-0003-4228-7962 Ke Yuan 11 0000-0002-2318-1460

###### Abstract

Application of deep learning in digital pathology shows promise on improving
disease diagnosis and understanding. We present a deep generative model that
learns to simulate high-fidelity cancer tissue images while mapping the real
images onto an interpretable low dimensional latent space. The key to the
model is an encoder trained by a previously developed generative adversarial
network, PathologyGAN. We study the latent space using 249K images from two
breast cancer cohorts. We find that the latent space encodes morphological
characteristics of tissues (e.g. patterns of cancer, lymphocytes, and stromal
cells). In addition, the latent space reveals distinctly enriched clusters of
tissue architectures in the high-risk patient group.

###### Keywords:

Generative Adversarial Networks Digital Pathology.

##  1 Introduction

Diagnosis and treatment of cancer are commonly based on assessment of
histopathological images such as the haematoxylin and eosin (H&E) stained
tissue images. The clinical utility of H&E images is due to the rich
information about the tumor microenvironment recorded in them, such as the
phenotype of cancer cells, immune cells, tissue architecture, and how they
interact. Recently, advanced machine learning and deep learning approaches
have been developed to improve our understanding of the tumor microenvironment
[7]. A common theme of these approaches is to correlate the quantification of
tumor microenvironment to a known clinically significant phenotype [2, 35] and
molecular characteristics [14, 10]. The quality of such correlation-based
studies largely depends on the heterogeneity within the response and
explanatory variables. Large cancer genome sequencing projects have revealed
substantial diversities of molecular and clinical characteristics within and
between patients [9]. Although it has been studied in breast and ovarian
cancers [29, 36], the heterogeneity of tumor microenvironment is largely
unknown.

Here, we propose a representation learning and disentanglement framework for
unsupervised quantification and clustering of tissue architectures, which
relates phenotype and patient survival. We use Generative Adversarial Networks
(GANs) as a tool to find useful representation of cancer tissue architectures.
We summarize our contributions as:

  1. 1.

Based on PathologyGAN model [30], we introduce an encoder that can be trained
to act as an inverse function of the generator, taking advantage of the
generator’s ability to capture tissue characteristics. This allows us to
project real tissue onto the generative model’s latent space.

  2. 2.

We demonstrate that the encoder is able to interpret the morphological
attributes of the cancer tissue and place the tissue images in distinct
regions of the latent space.

  3. 3.

We capture the change of cancer tissue morphology densities between patients
with survival times greater and lesser than five years, aligning with with
previous findings [2].

##  2 Background

Generative Adversarial Networks [17] are models that are able to learn diverse
and faithful data representations from a given distribution. This is done with
a generator, $G(z)$, that maps random noise, $\boldsymbol{z}\sim
p_{\boldsymbol{z}}(z)$, to samples that resemble the target data,
$\boldsymbol{x}\sim p_{\text{ data }}(\boldsymbol{x})$, and a discriminator,
$D(x)$, whose goal is to distinguish between real and generated samples. The
goal of a GAN is find the equilibrium in the min-max problem:

$\min_{G}\max_{D}V(D,G)=\mathbb{E}_{\boldsymbol{x}\sim p_{\text{ data
}}(\boldsymbol{x})}[\log D(\boldsymbol{x})]+\mathbb{E}_{\boldsymbol{z}\sim
p_{\boldsymbol{z}}(\boldsymbol{z})}[\log(1-D(G(\boldsymbol{z})))].$ (1)

GANs have since improved in image resolution, quality, and diversity with
models as SNGAN [28], BigGAN [4], ProGAN [21], RealsnessGAN [33], and StyleGAN
[22, 23]. There has been an increased focus on improving GANs for
disentanglement and representation learning, such as InfoGAN [8], BiGAN [11],
StyleGAN [22, 23], and BiBigGAN [12]. These models allow certain control in
image generation with specific feature properties. Simultaneously, projecting
real images onto a GAN’s latent space has also gained interest in the
literature. Some works have used pre-trained generators and to find the real
image projections through an iterative process [1, 25, 22, 23], yet these
methods are usually costly and image-by-image based. Alternatively, other
models include an encoder with different optimization goals like gradual
latent space [31], representation learning [11, 12, 26], or disentanglement
[26]. Given the computational capacity and current state of generative models
for representation learning, GANs or VAEs can make an impact on real-task
applications such as histopathology.

Machine learning and especially deep learning approaches have shown early
success in digital pathology, not only on achieving high accuracy
classification [13, 32, 18], but also in assisting in the decision process
with computer-human interaction [6]. These methods are usually either
supervised or weakly supervised, which require previous knowledge to train the
models. On the other hand, unsupervised models only require the data samples
(cancer tissue images) to find common attributes or properties that can
explain the data. Unsupervised models are gaining interest in histopathology
and they have been applied to tasks including tissue or nuclei segmentation
[34, 19, 3, 15, 16], classification [5], high resolution image generation [24,
30], and representation learning [19, 30]). Our work focuses on building a GAN
with an encoder that has disentanglement and representation learning
properties, and that way providing a framework for unsupervised quantification
and clustering of tissue architectures.

Figure 1: High level architecture of our GAN model.

##  3 PathologyGAN Encoder

We build upon PathologyGAN [30], which used techniques from BigGAN [4] and
StyleGAN [22] to successfully reproduce cancer tissue images while having an
interpretable latent space. In our model, the encoder $E$ learns to interpret
tissue morphology through generated images, effectively acting as the inverse
of the generator $G$. In PathologyGAN, the generator has disentanglement and
representation learning properties. We take advantage of this fact by forcing
the encoder to learn to place generated images back to the latent space. This
process trains an encoder that is able to map tissue with different properties
(e.g. cancer cell, lymphocyte, and stromal density) to distinct regions of the
latent space. Figure 1 captures the high level network architecture of our
model. After training, the encoder can be used independently to map real
images to their representations in the latent space.

We define the loss functions for the discriminator as $L_{Dis}$ and the
generator as $L_{Gen}$, which remain the same as in the GAN model (Equations 2
and 3):

$\displaystyle L_{Dis}=-\mathbb{E}_{x_{r}\sim P_{data}}$
$\displaystyle\left[\log\left(\tilde{D}\left(x_{r}\right)\right)\right]-\mathbb{E}_{x_{f}\sim
G(w)}\left[\log\left(1-\tilde{D}\left(x_{f}\right)\right)\right],$ (2)
$\displaystyle L_{Gen}=-\mathbb{E}_{x_{f}\sim G(w)}$
$\displaystyle\left[\log\left(\tilde{D}\left(x_{f}\right)\right)\right]-\mathbb{E}_{x_{r}\sim
P_{data}}\left[\log\left(1-\tilde{D}\left(x_{r}\right)\right)\right],$ (3)
$\displaystyle\quad\quad\tilde{D}\left(x_{r}\right)$
$\displaystyle=\text{sigmoid}\left(C\left(x_{r}\right)-\mathbb{E}_{x_{f}\sim
G(w)}C\left(x_{f}\right)\right)),$
$\displaystyle\quad\quad\tilde{D}\left(x_{f}\right)$
$\displaystyle=\text{sigmoid}\left(C\left(x_{f}\right)-\mathbb{E}_{x_{r}\sim
P_{data}}C\left(x_{r}\right)\right),$ $\displaystyle\quad\quad w$
$\displaystyle\sim M(z),\ z\sim P_{z}.$

The encoder loss function, $L_{Enc}$, is defined to minimize the mean square
error of the latent vectors $w$ and its reconstruction $w^{\prime}=E(G(w))$
through generated images (Equation 4):

$\displaystyle L_{Enc}$ $\displaystyle=\mathbb{E}_{z\sim
P_{z}}\left[\frac{1}{n}\sum_{i=1}^{n}(w_{i}-w^{\prime}_{i})^{2}\right]$ (4)
$\displaystyle\quad\quad n$ $\displaystyle=dim(w),\ w^{\prime}=E(G(w)),\ w\sim
M(z).$

Although the encoder $E$ is simultaneously trained with the GAN model, we can
separate the model training into two parts: the mapping network $M$, generator
$G$, and discriminator $D$ that are trained as a GAN with Relativistic Average
Discriminator [20], and the encoder $E$, which is trained to project back the
generated cancer tissue images onto the latent space. In practice, the encoder
$E$ learns simultaneously with the Generator $G$.

We trained our encoder based on the assumption that the generator is
successful in reproducing real cancer tissue. Therefore the encoder will learn
to project real tissue images if it is able to do so with generated ones.
Based on this logic, we use only generated images to train the encoder.

The encoder is only updated when the generator is not trained with style
mixing regularization [22]. Style mixing regularization uses two latent
vectors $w_{1}$ and $w_{2}$ to force disentanglement in the generator. It
becomes impractical to train the encoder in these steps because these images
have no clear assignation in the latent space. The style mixing regularization
is only preformed $50$% of times in the generator training, so our encoder is
updated every two steps per the generator.

To train our model we used two haematoxylin and eosin (H&E) breast cancer
databases from the Netherlands Cancer Institute (NKI) cohort and the Vancouver
General Hospital (VGH) cohort with 248 and 328 patients, respectively [2]. In
total, this corresponded to a training set of $249K$ images of $224\times 224$
pixels. We used an NVIDIA Titan RTX 24 GB to train the model for approximately
80 hours.

Figure 2: Real tissue images and its reconstructions. We take real tissue
images and map them to the latent space with our encoder, then we use the
generator with the latent vector representations to generate the image
reconstructions. (a) correspond to the real tissue images and (b) to the
reconstructions, the images are paired in columns. We show different examples
of stromal, lymphocytes, cancer cells, and combinations of these, the
reconstructions follow the real image attributes. Figure 3: Uniform Manifold
Approximation and Projection (UMAP) representation of real tissue samples in
the latent space using samples from Netherlands Cancer Institute (NKI) and
Vancouver General Hospital (VGH) patient cohorts. In this Figure, we fitted a
Gaussian mixture model over the complete dataset and used 100 components to
cluster the latent representations. We show different tissue images belonging
to various unique clusters, demonstrating how tissues with similar features
get assigned to common regions in the latent space.

##  4 Results and discussion

Our results focus on analyzing our model’s comprehension of tissue
characteristics, such as colour, texture, spatial features of cancer,
lymphocytes, and stromal cells. For these results we used only real H&E breast
tissue samples of VGH and NKI cohorts.

###  4.1 Tissue image Reconstruction

We start by analyzing how much information about the tissue the is model
capturing. The assumption is that if the encoder truly finds meaningful
representations of tissue morphology, the generator will reconstruct the held
attributes in the latent vectors.

We use the encoder to find the latent vector of real tissue images, and then
use the generator on those same vector representations to obtain the tissue
image reconstruction. Figure 2 shows these reconstruction results. Although
the reconstruction does not have a one-to-one match at the pixel level, we are
judging our model by how it finds high level features and assigns
representations based on them. The reconstructions keep the same tissue
attributes whether if we analyze stromal, lymphocytes, or cancer cells.

Figure 4: Four different linear interpolations between clusters in extreme
positions of the latent space at ten equally distributed points each. In
contrast to Figure 3, this figure shows the global structure of the latent
space where consecutive image points have gradual morphological changes in the
tissue.

###  4.2 Analysis of real tissue representations

We also study the latent space representations of all available real tissue
images of VGH and NKI cohorts. We perform a Uniform Manifold Approximation and
Projection (UMAP) [27] reduction on the $w$ latent vectors from $200$ to $2$
dimensions, and then fit a Gaussian mixture model of $100$ components to
cluster the tissue points. We reason that a good representation should have
the following properties: 1) points in close proximity should encode similar
tissue architectures; 2) far apart points should encode drastically different
tissue architectures; 3) changes in tissue architectures should correspond to
smooth manifolds in the representation space.

Figure 3 shows the UMAP plot, which demonstrates that tissue points belonging
to the same cluster have common characteristics, not only color and texture,
but also cell types presented in the tissue. Figure 4 captures the global
structure of the latent space, by displaying four different linear
interpolations between clusters across extreme positions of the latent space.
Each interpolation is made of ten equally distributed points. We can see that
transitions between consecutive points show morphological similarities without
abrupt changes. Between Figures 3 and 4, we conclude that our model learns to
capture and place gradual changes in cancer tissue, where images with similar
morphology can be clustered together.

###  4.3 Analysis of survival data using latent representations

Figure 5: Densities of tissue architectures in patients with greater (a) and
lesser (b) than 5 year survival of VGH cohort. We highlight six Gaussian
mixture components of tissue architectures that are enriched in high-risk
(less than 5 year survival) patients. (#-A) Tissue images of in the cluster,
(#-B) Percentage of patients with the tissue pattern in the survival group.

The NKI cohort consists of $198$ patients with survival times greater than
five years, and $49$ patients with survival times lesser or equal to five
years. In the case of VGH, $156$, $59$ with survival times greater, and lesser
or equal to 5 years respectively. We use the clustering properties over tissue
morphology of our model, to show the differences in tissue density between
patients with greater and lesser than 5 years of survival time. Previous
literature found different tissue architectures can dictate improved or worse
prognosis [2].

Figure 5 highlights tissue architectures that are enriched in patients with
lesser than 5 year survival, but less frequent in cases with greater than 5
year survival. Across both VGH and NKI cohorts, we find a wide spread
distinctly enriched clusters of tissue architectures. These results are
included in the Appendix.

##  5 Conclusion and future work

We presented an improvement to PathologyGAN that includes an encoder and can
integrate real tissue image data. We showed that this model distinguishes
features of real tissue, such as color, texture, cancer, lymphocyte, and
stromal cells; the model assigns low dimensional representations that maintain
meaning associated with its morphological characteristics. Furthermore, it
revealed distinctly enriched clusters of tissue architectures in the high-risk
patient groups.

This model opens the door for identifying common/distinct patterns of tissue
architecture. This could greatly improve our understanding of tumor
mircoenvironment and its relation to patient outcome and underlying molecular
characteristics. We are working towards generalizing our findings across large
patient cohorts such as The Cancer Genome Atlas.

## References

  * [1] Abdal, R., Qin, Y., Wonka, P.: Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space? 2019 IEEE/CVF International Conference on Computer Vision (ICCV) (Oct 2019)
  * [2] Beck, A., Sangoi, A., Leung, S.: Systematic analysis of breast cancer morphology uncovers stromal features associated with survival. Science translational medicine 3 (01 2011)
  * [3] de Bel, T., Hermsen, M., Smeets, B., Hilbrands, L., van der Laak, J., Litjens, G.: Automatic segmentation of histopathological slides of renal tissue using deep learning. In: Tomaszewski, J.E., Gurcan, M.N. (eds.) Medical Imaging 2018: Digital Pathology. vol. 10581, pp. 285 – 290. International Society for Optics and Photonics, SPIE (2018)
  * [4] Brock, A., Donahue, J., Simonyan, K.: Large scale GAN training for high fidelity natural image synthesis. CoRR (2018)
  * [5] Bulten, W., Litjens, G.: Unsupervised prostate cancer detection on H&E using Convolutional Adversarial Autoencoders (2018)
  * [6] Cai, C.J., Reif, E., Hegde, N., Hipp, J.D., Kim, B., Smilkov, D., Wattenberg, M., Viégas, F.B., Corrado, G.S., Stumpe, M.C., Terry, M.: Human-centered tools for coping with imperfect algorithms during medical decision-making. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI 2019. p. 4 (2019)
  * [7] Campanella, G., Hanna, M.G., Geneslaw, L., Miraflor, A., Werneck Krauss Silva, V., Busam, K.J., Brogi, E., Reuter, V.E., Klimstra, D.S., Fuchs, T.J.: Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nature Medicine 25(8), 1301–1309 (Aug 2019)
  * [8] Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., Abbeel, P.: InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets (2016)
  * [9] Ciriello, G., Miller, M.L., Aksoy, B.A., Senbabaoglu, Y., Schultz, N., Sander, C.: Emerging landscape of oncogenic signatures across human cancers. Nature Genetics 45(10), 1127–1133 (oct 2013)
  * [10] Coudray, N., Ocampo, P.S., Sakellaropoulos, T., Narula, N., Snuderl, M., Fenyö, D., Moreira, A.L., Razavian, N., Tsirigos, A.: Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nature Medicine 24(10), 1559–1567 (oct 2018)
  * [11] Donahue, J., Krähenbühl, P., Darrell, T.: Adversarial feature learning (2016)
  * [12] Donahue, J., Simonyan, K.: Large scale adversarial representation learning (2019)
  * [13] Esteva, A., Kuprel, B., Novoa, R., Ko, J., M Swetter, S., M Blau, H., Thrun, S.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542 (01 2017)
  * [14] Fu, Y., Jung, A.W., Torne, R.V., Gonzalez, S., Vohringer, H., Jimenez-Linan, M., Moore, L., Gerstung, M.: Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis. bioRxiv (0), 813543 (Oct 2019)
  * [15] Gadermayr, M., Gupta, L., Appel, V., Boor, P., Klinkhammer, B.M., Merhof, D.: Generative adversarial networks for facilitating stain-independent supervised and unsupervised segmentation: A study on kidney histology. IEEE Transactions on Medical Imaging 38(10), 2293–2302 (2019)
  * [16] Gadermayr, M., Gupta, L., Klinkhammer, B.M., Boor, P., Merhof, D.: Unsupervisedly Training GANs for Segmenting Digital Pathology with Automatically Generated Annotations. In: Cardoso, M.J., Feragen, A., Glocker, B., Konukoglu, E., Oguz, I., Unal, G., Vercauteren, T. (eds.) Proceedings of The 2nd ference on Medical Imaging with Deep Learning. Proceedings of Machine Learning Research, vol. 102, pp. 175–184. PMLR (2019)
  * [17] Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A.C., Bengio, Y.: Generative adversarial networks. CoRR (2014)
  * [18] Han, Z., Wei, B., Zheng, Y., Yin, Y., Li, K., Li, S.: Breast cancer multi-classification from histopathological images with structured deep learning model. Scientific Reports 7 (06 2017)
  * [19] Hou, L., Nguyen, V., Kanevsky, A.B., Samaras, D., Kurc, T.M., Zhao, T., Gupta, R.R., Gao, Y., Chen, W., Foran, D., et al.: Sparse autoencoder for unsupervised nucleus detection and representation in histopathology images. Pattern Recognition 86, 188–200 (Feb 2019)
  * [20] Jolicoeur-Martineau, A.: The relativistic discriminator: a key element missing from standard GAN. CoRR (2018)
  * [21] Karras, T., Aila, T., Laine, S., Lehtinen, J.: Progressive growing of GANs for improved quality, stability, and variation. In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings (2018)
  * [22] Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 4396–4405 (2018)
  * [23] Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., Aila, T.: Analyzing and Improving the Image Quality of StyleGAN (2019)
  * [24] Levine, A.B., Peng, J., Farnell, D., Nursey, M., Wang, Y., Naso, J.R., Ren, H., Farahani, H., Chen, C., Chiu, D., Talhouk, A., Sheffield, B., Riazy, M., Ip, P.P., Parra-Herran, C., Mills, A., Singh, N., Tessier-Cloutier, B., Salisbury, T., Lee, J., Salcudean, T., Jones, S.J., Huntsman, D.G., Gilks, C.B., Yip, S., Bashashati, A.: Synthesis of diagnostic quality cancer pathology images (2020)
  * [25] Lipton, Z.C., Tripathi, S.: Precise recovery of latent vectors from generative adversarial networks (2017)
  * [26] Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I., Frey, B.: Adversarial autoencoders (2015)
  * [27] McInnes, L., Healy, J., Melville, J.: Umap: Uniform manifold approximation and projection for dimension reduction (2018)
  * [28] Miyato, T., Kataoka, T., Koyama, M., Yoshida, Y.: Spectral normalization for generative adversarial networks. In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings (2018)
  * [29] Natrajan, R., Sailem, H., Mardakheh, F.K., Arias Garcia, M., Tape, C.J., Dowsett, M., Bakal, C., Yuan, Y.: Microenvironmental Heterogeneity Parallels Breast Cancer Progression: A Histology–Genomic Integration Analysis. PLoS Medicine 13(2), e1001961 (feb 2016)
  * [30] Quiros, A.C., Murray-Smith, R., Yuan, K.: Pathology gan: Learning deep representations of cancer tissue (2019)
  * [31] Sainburg, T., Thielk, M., Theilman, B., Migliori, B., Gentner, T.: Generative adversarial interpolative autoencoding: adversarial training on latent space interpolations encourage convex latent distributions (2018)
  * [32] Wei, J.W., Tafe, L.J., Linnik, Y.A., Vaickus, L.J., Tomita, N., Hassanpour, S.: Pathologist-level classification of histologic patterns on resected lung adenocarcinoma slides with deep neural networks
  * [33] Xiangli*, Y., Deng*, Y., Dai*, B., Loy, C.C., Lin, D.: Real or not real, that is the question. In: International Conference on Learning Representations (2020)
  * [34] Xu, J., Xiang, L., Liu, Q., Gilmore, H., Wu, J., Tang, J., Madabhushi, A.: Stacked sparse autoencoder (ssae) for nuclei detection on breast cancer histopathology images. IEEE Transactions on Medical Imaging 35(1), 119–130 (2016)
  * [35] Yuan, Y., Failmezger, H., M Rueda, O., Ali, H., Gräf, S., Chin, S.F., F Schwarz, R., Curtis, C., Dunning, M., Bardwell, H., Johnson, N., Doyle, S., Turashvili, G., Provenzano, E., Aparicio, S., Caldas, C., Markowetz, F.: Quantitative image analysis of cellular heterogeneity in breast tumors complements genomic profiling. Science translational medicine 4, 157ra143 (10 2012)
  * [36] Zhang, A.W., McPherson, A., Milne, K., Kroeger, D.R., Hamilton, P.T., Miranda, A., Funnell, T., Little, N., de Souza, C.P., Laan, S., LeDoux, S., Cochrane, D.R., Lim, J.L., Yang, W., Roth, A., Smith, M.A., Ho, J., Tse, K., Zeng, T., Shlafman, I., Mayo, M.R., Moore, R., Failmezger, H., Heindl, A., Wang, Y.K., Bashashati, A., Grewal, D.S., Brown, S.D., Lai, D., Wan, A.N., Nielsen, C.B., Huebner, C., Tessier-Cloutier, B., Anglesio, M.S., Bouchard-Côté, A., Yuan, Y., Wasserman, W.W., Gilks, C.B., Karnezis, A.N., Aparicio, S., McAlpine, J.N., Huntsman, D.G., Holt, R.A., Nelson, B.H., Shah, S.P.: Interfaces of Malignant and Immunologic Clonal Dynamics in Ovarian Cancer. Cell 173(7), 1755–1769.e22 (jun 2018)

##  Appendix 0.A Code

We provide the code at this location.

##  Appendix 0.B Tissue image Reconstruction

Real tissue images and its reconstructions. We take real tissue images and map
them to the latent space with our encoder, then we use the generator with the
latent vector representations to generate the image reconstructions.

In all these samples we use the following labeling: (a) correspond to the real
tissue images and (b) to the reconstructions, the images are paired in
columns. We show different examples of stromal, lymphocytes, cancer cells, and
combinations of these, the reconstructions follow the real image attributes.

Figure 6: Real tissue images and its reconstructions. (a) corresponds to the
real tissue images and (b) to the reconstructions, the images are paired in
columns. We show different examples of stromal, lymphocytes, cancer cells, and
combinations of these, the reconstructions follow the real image attributes.

Figure 7: Real tissue images and its reconstructions. (a) corresponds to the
real tissue images and (b) to the reconstructions, the images are paired in
columns. We show different examples of stromal, lymphocytes, cancer cells, and
combinations of these, the reconstructions follow the real image attributes.

##  Appendix 0.C Analysis of real tissue representations

Uniform Manifold Approximation and Projection (UMAP) representation of real
tissue samples in our model’s latent space using samples from Netherlands
Cancer Institute (NKI) and Vancouver General Hospital (VGH) patient cohorts.
In this Figure, we placed a Gaussian mixture model over the complete dataset
and used 100 components to cluster the latent representations.

We show two different type of figures for the combined datasets VGH and NKi,
and for NKI and VGH independently:

  1. 1.

Clustering of tissue architectures into common regions of the latent space,
Figures 8,10,12.

  2. 2.

Global structure of the latent space, Figures 9,11,13.

Figure 8: Uniform Manifold Approximation and Projection (UMAP) representation
of real tissue samples in the latent space using samples from Netherlands
Cancer Institute (NKI) and Vancouver General Hospital (VGH) patient cohorts.
In this Figure, we fitted a Gaussian mixture model over the complete dataset
and used 100 components to cluster the latent representations. We show
different tissue images belonging to various unique clusters, demonstrating
how tissues with similar features get assigned to common regions in the latent
space. Figure 9: Four different linear interpolations between clusters in
extreme positions of the latent space at ten equally distributed points each.
In contrast to Figure 8, this figure shows the global structure of the latent
space where consecutive image points have gradual morphological changes in the
tissue. Figure 10: Uniform Manifold Approximation and Projection (UMAP)
representation of real tissue samples in the latent space using samples from
Netherlands Cancer Institute (NKI) patient cohorts. In this Figure, we fitted
a Gaussian mixture model over the complete dataset and used 100 components to
cluster the latent representations. We show different tissue images belonging
to various unique clusters, demonstrating how tissues with similar features
get assigned to common regions in the latent space. Figure 11: Four different
linear interpolations between clusters in extreme positions of the latent
space at ten equally distributed points each. In contrast to Figure 10, this
figure shows the global structure of the latent space where consecutive image
points have gradual morphological changes in the tissue in the NKI patient
cohort. Figure 12: Uniform Manifold Approximation and Projection (UMAP)
representation of real tissue samples in the latent space using samples from
Vancouver General Hospital (VGH) patient cohorts. In this Figure, we fitted a
Gaussian mixture model over the complete dataset and used 100 components to
cluster the latent representations. We show different tissue images belonging
to various unique clusters, demonstrating how tissues with similar features
get assigned to common regions in the latent space. Figure 13: Four different
linear interpolations between clusters in extreme positions of the latent
space at ten equally distributed points each. In contrast to Figure 12, this
figure shows the global structure of the latent space where consecutive image
points have gradual morphological changes in the tissue in the VGH patient
cohort.

##  Appendix 0.D Analysis of survival data using latent representations

In this appendix we provide the collection of figures for NKI and VGH patient
cohorts. We show the density difference in tissue architectures between high-
risk (less than 5 year survival (a)) and low-risk patients (greater than 5
year survival (b)).

Figures 14 and 16 present tissue architectures predominant on high-risk
patients, and Figures 15 and 17 on low-risk patients.

(#-A) Tissue images belonging to the cluster, (#-B) Percentage of patients
with the tissue pattern in the survival group.

Figure 14: VGH Cohort, tissue architecture more predominant on high-risk
patients, survival times lesser than 5 years.

Figure 15: VGH Cohort, tissue architecture more predominant on low-risk
patients, survival times greater than 5 years.

Figure 16: NKI Cohort, tissue architecture more predominant on high-risk
patients, survival times lesser than 5 years.

Figure 17: NKI Cohort, tissue architecture more predominant on low-risk
patients, survival times greater than 5 years.

##  Appendix 0.E Model Architecture

Mapping Network $M:z\rightarrow w$
---
$z\in\sim\mathbb{R}^{200}\sim\mathcal{N}(0,I)$
ResNet Dense Layer and ReLU, $200\rightarrow 200$
ResNet Dense Layer and ReLU, $200\rightarrow 200$
ResNet Dense Layer and ReLU, $200\rightarrow 200$
ResNet Dense Layer and ReLU, $200\rightarrow 200$
Dense Layer, $200\rightarrow 200$
Table 1: Mapping Network Architecture details of Pathology GAN model.

Generator Network $G:w\rightarrow x$
---
Dense Layer, adaptive instance normalization (AdaIN), and leakyReLU
$200\rightarrow 1024$
Dense Layer, AdaIN, and leakyReLU
$1024\rightarrow 12544$
Reshape $7\times 7\times 256$
ResNet Conv2D Layer, 3x3, stride 1, pad same, AdaIN, and leakyReLU $0.2$
$7\times 7\times 256\rightarrow 7\times 7\times 256$
ConvTranspose2D Layer, 2x2, stride 2, pad upscale, AdaIN, and leakyReLU $0.2$
$7\times 7\times 256\rightarrow 14\times 14\times 512$
ResNet Conv2D Layer, 3x3, stride 1, pad same, AdaIN, and leakyReLU $0.2$
$14\times 14\times 512\rightarrow 14\times 14\times 512$
ConvTranspose2D Layer, 2x2, stride 2, pad upscale, AdaIN, and leakyReLU $0.2$
$14\times 14\times 512\rightarrow 28\times 28\times 256$
ResNet Conv2D Layer, 3x3, stride 1, pad same, AdaIN, and leakyReLU $0.2$
$28\times 28\times 256\rightarrow 28\times 28\times 256$
Attention Layer at $28\times 28\times 256$
ConvTranspose2D Layer, 2x2, stride 2, pad upscale, AdaIN, and leakyReLU $0.2$
$28\times 28\times 256\rightarrow 56\times 56\times 128$
ResNet Conv2D Layer, 3x3, stride 1, pad same, AdaIN, and leakyReLU $0.2$
$56\times 56\times 128\rightarrow 56\times 56\times 128$
ConvTranspose2D Layer, 2x2, stride 2, pad upscale, AdaIN, and leakyReLU $0.2$
$56\times 56\times 128\rightarrow 112\times 112\times 64$
ResNet Conv2D Layer, 3x3, stride 1, pad same, AdaIN, and leakyReLU $0.2$
$112\times 112\times 64\rightarrow 112\times 112\times 64$
ConvTranspose2D Layer, 2x2, stride 2, pad upscale, AdaIN, and leakyReLU $0.2$
$112\times 112\times 64\rightarrow 224\times 224\times 32$
Conv2D Layer, 3x3, stride 1, pad same, $32\rightarrow 3$
$224\times 224\times 32\rightarrow 224\times 224\times 3$
Sigmoid
Table 2: Generator Network Architecture details of Pathology GAN model.
Discriminator Network $C:x\rightarrow d$
---
$x\in\mathbb{R}^{224\times 224\times 3}$
ResNet Conv2D Layer, 3x3, stride 1, pad same, and leakyReLU $0.2$
$224\times 224\times 3\rightarrow 224\times 224\times 3$
Conv2D Layer, 2x2, stride 2, pad downscale, and leakyReLU $0.2$
$224\times 224\times 3\rightarrow 122\times 122\times 32$
ResNet Conv2D Layer, 3x3, stride 1, pad same, and leakyReLU $0.2$
$122\times 122\times 32\rightarrow 122\times 122\times 32$
Conv2D Layer, 2x2, stride 2, pad downscale, and leakyReLU $0.2$
$122\times 122\times 32\rightarrow 56\times 56\times 64$
ResNet Conv2D Layer, 3x3, stride 1, pad same, and leakyReLU $0.2$
$56\times 56\times 64\rightarrow 56\times 56\times 64$
Conv2D Layer, 2x2, stride 2, pad downscale, and leakyReLU $0.2$
$56\times 56\times 64\rightarrow 28\times 28\times 128$
ResNet Conv2D Layer, 3x3, stride 1, pad same, and leakyReLU $0.2$
$28\times 28\times 128\rightarrow 28\times 28\times 128$
Attention Layer at $28\times 28\times 128$
Conv2D Layer, 2x2, stride 2, pad downscale, and leakyReLU $0.2$
$28\times 28\times 128\rightarrow 14\times 14\times 256$
ResNet Conv2D Layer, 3x3, stride 1, pad same, and leakyReLU $0.2$
$14\times 14\times 256\rightarrow 14\times 14\times 256$
Conv2D Layer, 2x2, stride 2, pad downscale, and leakyReLU $0.2$
$14\times 14\times 256\rightarrow 7\times 7\times 512$
Flatten $7\times 7\times 512\rightarrow 25088$
Dense Layer and leakyReLU, $25088\rightarrow 1024$
Dense Layer and leakyReLU, $1024\rightarrow 1$
Table 3: Discriminator Network Architecture details of Pathology GAN model.
Encoder Network $E:x\rightarrow w^{\prime}$
---
$x\in\mathbb{R}^{224\times 224\times 3}$
Conv2D Layer, 2x2, stride 2, pad downscale, and leakyReLU $0.2$
$224\times 224\times 3\rightarrow 224\times 224\times 32$
ResNet Conv2D Layer, 3x3, stride 1, pad same, and leakyReLU $0.2$
$224\times 224\times 32\rightarrow 224\times 224\times 32$
Conv2D Layer, 2x2, stride 2, pad downscale, and leakyReLU $0.2$
$224\times 224\times 32\rightarrow 122\times 122\times 64$
ResNet Conv2D Layer, 3x3, stride 1, pad same, and leakyReLU $0.2$
$122\times 122\times 64\rightarrow 122\times 122\times 64$
Conv2D Layer, 2x2, stride 2, pad downscale, and leakyReLU $0.2$
$122\times 122\times 64\rightarrow 56\times 56\times 128$
ResNet Conv2D Layer, 3x3, stride 1, pad same, and leakyReLU $0.2$
$56\times 56\times 128\rightarrow 56\times 56\times 128$
Conv2D Layer, 2x2, stride 2, pad downscale, and leakyReLU $0.2$
$56\times 56\times 128\rightarrow 28\times 28\times 256$
ResNet Conv2D Layer, 3x3, stride 1, pad same, and leakyReLU $0.2$
$28\times 28\times 256\rightarrow 28\times 28\times 256$
Attention Layer at $28\times 28\times 256$
Conv2D Layer, 2x2, stride 2, pad downscale, and leakyReLU $0.2$
$28\times 28\times 256\rightarrow 14\times 14\times 512$
ResNet Conv2D Layer, 3x3, stride 1, pad same, and leakyReLU $0.2$
$14\times 14\times 512\rightarrow 14\times 14\times 512$
Conv2D Layer, 2x2, stride 2, pad downscale, and leakyReLU $0.2$
$14\times 14\times 512\rightarrow 7\times 7\times 512$
Flatten $7\times 7\times 512\rightarrow 25088$
Dense Layer and leakyReLU, $25088\rightarrow 1024$
Dense Layer and leakyReLU, $1024\rightarrow 200$
Table 4: Encoder Network Architecture details of Pathology GAN model.