# Neural style transfer of weak lensing mass maps

Masato Shirasaki<EMAIL_ADDRESS>National Astronomical Observatory
of Japan (NAOJ), National Institutes of Natural Science, Mitaka, Tokyo
181-8588, Japan  The Institute of Statistical Mathematics, Tachikawa, Tokyo
190-8562, Japan  Shiro Ikeda  The Institute of Statistical Mathematics,
Tachikawa, Tokyo 190-8562, Japan  Department of Statistical Science, School of
Multidisciplinary Sciences, Graduate University for Advanced Studies, 10-3
Midori-cho, Tachikawa, Tokyo 190-8562, Japan  Kavli Institute for the Physics
and Mathematics of the Universe, The University of Tokyo, 5-1-5 Kashiwanoha,
Kashiwa, 277-8583, Japan

###### Abstract

We propose a new generative model of projected cosmic mass density maps
inferred from weak gravitational lensing observations of distant galaxies
(weak lensing mass maps). We construct the model based on a neural style
transfer so that it can transform Gaussian weak lensing mass maps into deeply
non-Gaussian counterparts as predicted in ray-tracing lensing simulations. We
develop an unpaired image-to-image translation method with Cycle-Consistent
Generative Adversarial Networks (Cycle GAN), which learn efficient mapping
from an input domain to a target domain. Our model is designed to enjoy
important advantages; it is trainable with no need for paired simulation data,
flexible to make the input domain visually meaningful, and expandable to
rapidly-produce a map with a larger sky coverage than training data without
additional learning. Using 10,000 lensing simulations, we find that
appropriate labeling of training data based on field variance requires the
model to exhibit a desired diversity of various summary statistics for weak
lensing mass maps. Compared with a popular log-normal model, our model
improves in predicting the statistical natures of three-point correlations and
local properties of rare high-density regions. We also demonstrate that our
model enables us to produce a continuous map with a sky coverage of $\sim
166\,\mathrm{deg}^{2}$ but similar non-Gaussian features to training data
covering $\sim 12\,\mathrm{deg}^{2}$ in a GPU minute. Hence, our model can be
beneficial to massive productions of synthetic weak lensing mass maps, which
is of great importance in future precise real-world analyses.

##  I Introduction

General relativity predicts that an apparent image of a single distant
astrophysical source should be distorted by gravitational lensing effects. The
distortion is governed by the gravitational potential of intervening mass
(lens) and the geometry between the lens and source objects Bartelmann and
Schneider (2001). Extracting coherent distortions over a large number of
sources allows us to infer large-scale gravitating mass distributions of the
Universe in an unbiased manner. The lensing effect caused by the large-scale
structure is commonly referred to as the weak lensing effect, whereas the mass
distributions inferred from the weak lensing effect are called weak lensing
mass maps in the literature. Statistical analyses of weak lensing mass maps
can provide vital information of gravitational clumping of dark matter and
cosmic expansion (see, Refs Kilbinger (2015); Mandelbaum (2018) for reviews).

The two-point angular correlation function of shapes of sources, or its
Fourier-space counterpart known as the power spectrum, are commonly used to
characterize weak lensing mass maps. Although the power spectrum provides a
complete statistical description of a random Gaussian field, numerical
simulations of weak lensing effects have shown that weak lensing mass maps
have non-Gaussian properties. Hence the power spectrum alone cannot fully
describe a weak lensing mass map Jain  _et al._ (2000); White and Hu (2000).
Various statistical methods have been proposed to study the non-Gaussian
features of weak lensing mass maps (see Ref. Ajani  _et al._ (2023) and
references therein for a list of the statistics proposed so far). At present,
there are no unique methods to extract all the information imprinted in weak
lensing mass maps.

A generative model of weak lensing mass maps plays an essential role in modern
weak lensing analyses. It is highly demanded in practice for the reasons
below;

  * •

Valid analytic approaches are yet unknown to derive cosmological dependence of
most of the non-Gaussian statistics proposed so far. An ensemble average of a
given non-Gaussian statistic over a set of generated weak lensing mass maps is
commonly used to study its cosmological dependence.

  * •

A precise estimation of statistical errors in a given lensing statistic is
crucial in cosmological inference Dodelson and Schneider (2013). A strong non-
Gaussianity in the weak lensing mass maps and several observational effects
make it difficult to estimate the statistical errors in an analytic way. The
best practice is to run a set of numerical simulations with the relevant
physics to lensing analyses and to create realistic synthetic weak lensing
data by including various observational effects with appropriate recipes
Shirasaki  _et al._ (2019a).

  * •

Blind analyses of weak lensing mass maps become a standard procedure to remove
any human-based systematic errors as some cosmological tensions have been
claimed (see Ref. Abdalla  _et al._ (2022) for a review of the cosmological
tensions). The current blinding scheme of weak lensing data is based on
changes of observed galaxy shapes with a single number, introducing
modulations in the amplitude of lensing power spectrum Asgari  _et al._
(2021); Secco  _et al._ (2022); Dalal  _et al._ (2023); Li  _et al._ (2023).
More sophisticated blinding is required to test the accuracy of a theoretical
template for a given lensing statistic. A blinded challenge, analyzing
generated weak lensing mass maps by making input cosmological parameters
unknown, is among the most effective approaches for this purpose (see Ref.
Nishimichi  _et al._ (2020) for an example in galaxy clustering data). A fast
precise generative model of weak lensing mass maps enables one to perform the
blinded challenge in a user-friendly manner.

  * •

Field-level cosmological inference of weak lensing mass maps receives plenty
of attention from the community Porqueres  _et al._ (2021); Boruah  _et al._
(2022); Dai and Seljak (2022) because it can have no loss of cosmological
information in principle. Reconstructions of weak lensing mass maps within a
Bayesian framework are also an important subject in modern astronomy even for
a fixed cosmological model, whereas the reconstruction from noisy galaxy
imaging data often requires a training dataset of noiseless weak lensing mass
maps Shirasaki  _et al._ (2019b); Jeffrey  _et al._ (2020); Fiedorowicz  _et
al._ (2022); Remy  _et al._ (2023). A rapid but realistic generator of weak
lensing mass maps is fundamental to the field-level inference and Bayesian
reconstruction as a matter of course.

Ray-tracing simulations of weak lensing effects are among the most robust
generative models thus far, but those need multiple outputs from cosmological
simulations of large-scale structures. Because of its computational cost, the
ray-tracing simulation can typically provide 100-10,000 realizations of weak
lensing mass maps on a yearly basis Sato  _et al._ (2009); Dietrich and
Hartlap (2010); Harnois-Déraps  _et al._ (2012); Liu  _et al._ (2018);
Takahashi  _et al._ (2017); Harnois-Déraps  _et al._ (2018); Harnois-Deraps
_et al._ (2019); Shirasaki  _et al._ (2021); Osato  _et al._ (2021); Kacprzak
_et al._ (2023); Hadzhiyska  _et al._ (2023); Ferlito  _et al._ (2023). To
reduce the computational cost, previous studies have proposed that the time-
consuming cosmological N-body simulation can be replaced with some
alternatives. The proposals include the use of multivariate log-normal models
Taruya  _et al._ (2002); Xavier  _et al._ (2016); Makiya  _et al._ (2021) or
its advanced versions Yu  _et al._ (2016); Tessore  _et al._ (2023), an
approximate gravity solver Izard  _et al._ (2018); Sgier  _et al._ (2019);
Böhm  _et al._ (2021), and halo-based models Giocoli  _et al._ (2017, 2020).
Ray-tracing simulations with the alternative methods drastically shorten
computational time with a fixed computing environment, while they have limited
predictive power for non-Gaussian observables in general.

Image processing with neural networks provides another interesting approach
for the generative model of weak lensing effects. Motivated by recent
successes in deep learning, there exist several generative models of weak
lensing mass maps based on deep-learning neural networks, including Generative
Adversarial Networks (GANs) Mustafa  _et al._ (2019); Perraudin  _et al._
(2021); Tamosiunas  _et al._ (2021), Score-based Diffusion Models Remy  _et
al._ (2023), and Normalizing Flow Dai and Seljak (2022). Those deep-learning-
based generators commonly produce a new weak lensing mass map from a given set
of random numbers, and the generators learn the relation between the mass map
and latent random numbers from a large set of training data. Because a field
of view and the number of pixels in the training data has to be set a priori,
the generators are not able to produce weak lensing mass maps at different sky
coverage and angular resolution from the training setup. This is a weak point
of the deep-learning-based generative models as ongoing and future galaxy
imaging surveys aim at measuring weak lensing effects over a sky coverage of
$1000\,\mathrm{deg}^{2}$ or larger. In general, it becomes difficult to train
a given neural network as the size of output data increases, because the
number of trainable parameters in the network increases dramatically.

In this paper, we propose a new generative model of weak lensing mass maps
based on a deep-learning method, which is designed to be able to increase a
field of view without additional training. We train the neural network
translating a latent image into a target image of a non-Gaussian weak lensing
mass map. We here set the latent image to a multivariate Gaussian map with the
power spectrum being the same as the target counterpart. The network will be
trained to learn the translation between the Gaussian and non-Gaussian mass
maps in a given field of view within the framework of Cycle-Consistent
Generative Adversarial Networks (Cycle GAN) Zhu  _et al._ (2017). After
training, we can extend a sky coverage of fake mass maps by (i) producing a
large-sky Gaussian map from the given power spectrum, (ii) dividing the
Gaussian map into subregions, (iii) performing the learned translation on a
patch-by-patch basis, and (iv) combining the non-Gaussian maps on all the
subregions. This procedure allows our generator to produce a lensing mass map
with an arbitrary sky coverage while not degrading the angular resolution
compared to the training datasets. Note that our training data does not
require a large sky coverage, enabling the neural network to learn small-scale
non-Gaussian features in a weak lensing mass map in a cost-efficient manner.
Also, our approach enables us to rapidly increase the number of realistic
lensing simulations with low costs of producing a new set of the Gaussian mass
maps as long as we thoroughly train the Cycle GAN.

Similar approaches based on an image-to-image translation have been explored
in Refs. Han  _et al._ (2021); Piras  _et al._ (2022). A distinct difference
from the previous studies is that our training does not need a paired or
aligned set of images for the image-to-image translation. This is achieved by
recent progress in unsupervised learning for style transfer of natural images
as in Ref. Zhu  _et al._ (2017). Note that paired sets of images at two
different domains are not available in common, that is the main difficulty in
constructing generative models for the task of image-to-image translation. Our
training strategy only requires an unpaired set of images at two different
domains, allowing one to easily increase the number of training data as well
as examine various translations in an efficient way.

The rest of the present paper is organized as follows. In Section II, we
summarize the basics of weak gravitational lensing effects. In Section III, we
describe three generative models of weak lensing mass maps based on
multivariate random Gaussian variables, the log-normal model, and the
generative adversarial networks adopted in this paper. We provide how to
produce the data set for training and testing the networks in Section IV. In
Section V, we show the performance of our trained networks at the sky coverage
as same as in the training datasets. We then study the applicability of our
method in extending a field of view in Section VI. Limitations of our
generators are listed in Section VII. Concluding remarks and discussions are
given in Section VIII.

##  II Weak Gravitational Lensing

The image distortion of a source object (galaxy) induced by weak gravitational
lensing is commonly characterized by the $2\times 2$ matrix below:

$\displaystyle A_{ij}=\frac{\partial\theta_{\rm true}^{i}}{\partial\theta_{\rm
obs}^{j}}\equiv\left(\begin{array}[]{cc}1-\kappa-\gamma_{1}&-\gamma_{2}-\omega\\\
-\gamma_{2}+\omega&1-\kappa+\gamma_{1}\\\ \end{array}\right),$ (3)

where $\kappa$ is the convergence, $\gamma$ is the shear, and $\omega$ is the
rotation, $\mbox{\boldmath$\theta$}_{\rm obs}$ and
$\mbox{\boldmath$\theta$}_{\rm true}$ represent the observed position of the
source object and the true (unlensed) position, respectively. In the limit of
$\kappa,\gamma\ll 1$ which we are interested in, we can express the
convergence as the integral of the density contrast of underlying matter
density field $\delta_{\rm m}(\mbox{\boldmath$x$})$ with a weight function
over redshift Bartelmann and Schneider (2001),

$\displaystyle\kappa(\mbox{\boldmath$\theta$})$ $\displaystyle=$
$\displaystyle\int_{0}^{\infty}{\rm d}\chi\,W_{\kappa}(\chi)\delta_{\rm
m}(r(\chi)\mbox{\boldmath$\theta$},\chi),$ (4) $\displaystyle
W_{\kappa}(\chi)$ $\displaystyle=$
$\displaystyle\frac{3}{2}\left(\frac{H_{0}}{c}\right)^{2}\Omega_{\rm
m0}(1+z(\chi))r(\chi)$ (5)
$\displaystyle\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\times\int_{\chi}^{\infty}{\rm
d}\chi^{\prime}\,p(\chi^{\prime})\frac{r(\chi^{\prime}-\chi)}{r(\chi^{\prime})},$

where $c$ is the speed of light, $H_{0}$ is the present-day Hubble constant,
$\Omega_{\rm m0}$ is the matter density parameter at present, $\chi(z)$ is the
radial comoving distance to redshift $z$, $r(\chi)$ is the angular diameter
distance, and $p(\chi)$ represents the source distribution normalized to
$\int{\rm d}\chi\,p(\chi)=1$. Throughout this paper, we assume a single source
plane at redshift of $z_{\rm source}=1$, i.e.
$p(\chi)=\delta_{\mathrm{D}}(\chi-\chi_{1})$ where $\delta_{\mathrm{D}}(x)$ is
the Dirac delta function and $\chi_{1}=\chi(z=1)$ for simplicity.

##  III Model

###  III.1 Gaussian

The simplest generative model of the lensing convergence field is given by a
multivariate random Gaussian distribution. The full statistical information of
$\kappa$ can be characterized by the distribution in Fourier mode
$\tilde{\kappa}_{\bm{\ell}}$. In general, the distribution of Fourier mode
$\kappa_{\bm{\ell}}$ is expressed as the function of norm
$|\tilde{\kappa}_{\bm{\ell}}|$ and phase $\theta_{\kappa,{\bm{\ell}}}$.
Assuming zero mean, we can write the distribution of Gaussian
$\kappa_{\bm{\ell}}$ as

$\displaystyle{\rm
Prob}(|\tilde{\kappa}_{\bm{\ell}}|,\theta_{\kappa,{\bm{\ell}}})$
$\displaystyle\equiv$ $\displaystyle{\cal
P}_{G}(|\tilde{\kappa}_{\bm{\ell}}|){\cal P}_{G}(\theta_{\kappa,\bm{\ell}}),$
(6) $\displaystyle{\cal P}_{G}(|\tilde{\kappa}_{\bm{\ell}}|)$ $\displaystyle=$
$\displaystyle\frac{2|\tilde{\kappa}_{\bm{\ell}}|}{[C(\ell)]^{3/2}}\exp\left[-\left(\frac{|\tilde{\kappa}_{\bm{\ell}}|}{\sqrt{C(\ell)}}\right)^{2}\right],$
(7) $\displaystyle{\cal P}_{G}(\theta_{\kappa,{\bm{\ell}}})$ $\displaystyle=$
$\displaystyle\frac{1}{2\pi},$ (8)

where $C(\ell)$ is the power spectrum of $\kappa$ and it is defined by

$\displaystyle\langle\tilde{\kappa}(\bm{\ell}_{1})\tilde{\kappa}(\bm{\ell}_{2})\rangle=(2\pi)^{2}\delta_{D}(\bm{\ell}_{1}+\bm{\ell}_{2})C(\ell_{1}).$
(9)

###  III.2 Log Normal

Non-linear gravitational growth in the matter density contrast
$\delta_{\mathrm{m}}$ makes the lensing convergence non-Gaussian. A prominent
non-Gaussin feature of $\kappa$ in numerical simulations is the presence of a
heavy positive tail in one-point probability distribution function (PDF) of
$\kappa$ (e.g. Refs. Jain  _et al._ (2000); Takahashi  _et al._ (2011); Castro
_et al._ (2018)). An empirical log-normal model can account for the tail in a
simple way Taruya  _et al._ (2002). The model assumes that the convergence
field follows the one-point PDF of

$\displaystyle\mathrm{Prob}(\kappa)$ $\displaystyle=$
$\displaystyle\frac{1}{\sqrt{2\pi}\sigma_{\ln}}\frac{1}{\kappa+|\kappa_{\mathrm{min}}|}$
(10)
$\displaystyle\times\exp\left\\{-\frac{\left[\ln(1+\kappa/|\kappa_{\mathrm{min}}|)+\sigma^{2}_{\ln}/2\right]^{2}}{2\sigma^{2}_{\ln}}\right\\},$

for $\kappa>\kappa_{\mathrm{min}}$, and it holds that
$\mathrm{Prob}(\kappa)=0$ otherwise. In Eq. (10), the variance
$\sigma^{2}_{\ln}$ is given by

$\displaystyle\sigma^{2}_{\ln}\equiv\ln\left(1+\frac{\langle\kappa^{2}\rangle}{|\kappa_{\mathrm{min}}|^{2}}\right),$
(11)

where $\langle\kappa^{2}\rangle$ is the variance of $\kappa$. Note that the
variance is determined by the convergence power spectrum of $C$.

One can rewrite the log-normal model in terms of a new Gaussian variable $y$
by considering a local-type translation of

$\displaystyle
y(\mbox{\boldmath$\theta$})=\ln\left(1+\frac{\kappa(\mbox{\boldmath$\theta$})}{|\kappa_{\mathrm{min}}|}\right)+\frac{\sigma^{2}_{\ln}}{2}.$
(12)

This new Gaussian field $y$ has zero mean and unit variance. Using the
relation of Eq. (12) and the PDF of Eq. (10), one expect that the two-point
correlation of $y$ should be related with the counterpart of $\kappa$ as

$\displaystyle\xi_{y}(\theta)$ $\displaystyle\equiv$ $\displaystyle\langle
y(\mbox{\boldmath$\theta$}+\mbox{\boldmath$\phi$})y(\mbox{\boldmath$\phi$})\rangle,$
(13) $\displaystyle\xi_{\kappa}(\theta)$ $\displaystyle\equiv$
$\displaystyle\langle\kappa(\mbox{\boldmath$\theta$}+\mbox{\boldmath$\phi$})\kappa(\mbox{\boldmath$\phi$})\rangle$
(14) $\displaystyle=$
$\displaystyle\frac{1}{2\pi[1-\xi^{2}_{y}(\theta)]^{1/2}}\int_{-\infty}^{\infty}\mathrm{d}u_{1}\int_{-\infty}^{\infty}\mathrm{d}u_{2}$
$\displaystyle\times\exp\left(-\frac{u^{2}_{1}}{2[1-\xi^{2}_{y}(\theta)]}\right)\exp\left(-\frac{u^{2}_{2}}{2}\right)$
$\displaystyle\times{\cal F}(u_{1}+\xi_{y}(\theta)u_{2}){\cal F}(u_{2}),$

where ${\cal F}$ is the inverse of Eq. (12), i.e. ${\cal
F}(u)=|\kappa_{\mathrm{min}}|\exp(u\sigma_{\ln}-\sigma^{2}_{\ln}/2)-|\kappa_{\mathrm{min}}|$.
Note that the two-point correlation of $\xi_{\kappa}(\theta)$ can be computed
as the Fourier transform of the power spectrum $C$;

$\displaystyle\xi_{\kappa}(\theta)=\int\frac{\mathrm{d}^{2}\ell}{(2\pi)^{2}}\,C(|\mbox{\boldmath$\ell$}|)e^{-i\mbox{\boldmath$\ell$}\cdot\mbox{\boldmath$\theta$}}.$
(15)

Therefore, Eq. (14) sets $\xi_{y}$ for a given convergence power spectrum. In
the end, the log-normal generative model is fully specified by two parameters
of the minimum convergence $\kappa_{\mathrm{min}}$ and convergence power
spectrum $C$.

###  III.3 Cycle-Consistent Generative Adversarial Networks

The log-normal model partly accounts for non-linear gravitational effects on
the lensing convergence, while its validity is still limited. For instance,
the log-normal model cannot reproduce summary statistics sensitive to rare
events such as skewness and kurtosis Taruya  _et al._ (2002). Also, multiple-
point correlation functions in the log-normal model can be expressed as a
function of the two-point counterpart Hilbert  _et al._ (2011), but this is
not the case in numerical simulations of cosmic large-scale structures
Colavincenzo  _et al._ (2019); Piras  _et al._ (2022).

To improve the log-normal model, we here examine a machine-learning approach
for image synthesis. The GAN is a generative model Goodfellow  _et al._ (2014)
that contains two neural networks: a generator $G$ and a discriminator $D$.
Given the real data set, $G$ aims to create false data that looks like the
genuine data from the real sample, while $D$ tries to discriminate the false
data and the genuine counterpart. The GAN can be effectively trained with the
back-propagation algorithm and appropriate network architecture even when
limited training data are available (e.g. see Ref. Huang  _et al._ (2018) for
a review).

Among several models of the GAN, we adopt the Cycle-Consistent GAN (Cycle GAN)
consisting of a pair of GANs Zhu  _et al._ (2017). The Cycle GAN is designed
to learn a style transfer between two different image domains. A
representative example is the translation between a landscape photograph and a
Monet painting. Because there are no paired image sets between landscape
photographs and Monet paintings in practice, one requires the Cycle GAN to
train with unpaired image collections, consisting of a source set of
$\\{x_{i}\\}_{i=1}^{N}\,(x_{i}\in\mathrm{landscape\,photos})$ and a target set
$\\{y_{j}\\}_{j=1}^{M}\,(y_{j}\in\mathrm{Monet\,paintings})$. In this setup,
one is not able to use the information that $x_{i}$ in the input domain
matches to $y_{j}$ in the target for training of the Cycle GAN. In this paper,
we train the Cycle GAN with the network architecture proposed in Ref. Zhu  _et
al._ (2017) by using unpaired image sets of the Gaussian convergence and the
convergence produced in lensing simulations of Ref. Liu  _et al._ (2018). It
would be worth noting that the Gaussian convergence model is computationally
efficient, while a massive production of lensing simulations requires high
computational costs. This situation is similar to the translation between a
landscape photograph and a Monet painting, i.e. it is easy to prepare a large
set of landscape photographs with low costs, but it is really expensive
(impossible in fact) to increase the number of Monet paintings. Details of the
unpaired sets and training process are described in the next section, while
the network architecture in the Cycle GAN is provided in Appendix A.

We here briefly summarize how to train the Cycle GAN in an unsupervised
fashion. Let $X$ and $Y$ be the image sets produced by the Gaussian model as
in Section III.1 and the set of the lensing convergence in the simulations,
respectively. We refer to $G_{X}$ as the generator translating an input image
of $x\in X$ to $y\in Y$, whereas $D_{X}$ judges if an input image is produced
by $G_{X}$ or not. We also define the generator of $G_{Y}$ and the
discriminator of $D_{Y}$ for the translation of $y\rightarrow x$ in a similar
way. Our objective is then summarized as

$\displaystyle
G_{X},G_{Y}=\mathrm{arg}\min_{G_{X},G_{Y}}\max_{D_{X},D_{Y}}{\cal
L}(G_{X},G_{Y},D_{X},D_{Y}),$ (16)

where ${\cal L}(G_{X},G_{Y},D_{X},D_{Y})$ is the loss function for the Cycle
GAN containing four parts below;

$\displaystyle{\cal L}(G_{X},G_{Y},D_{X},D_{Y})$ $\displaystyle=$
$\displaystyle{\cal L}_{\mathrm{GAN}}(G_{X},D_{X})+{\cal
L}_{\mathrm{GAN}}(G_{Y},D_{Y})$ (17)
$\displaystyle\quad\quad\quad\quad+\lambda_{1}{\cal
L}_{\mathrm{cyc}}(G_{X},G_{Y})$
$\displaystyle\quad\quad\quad\quad+\lambda_{2}{\cal
L}_{\mathrm{id}}(G_{X},G_{Y}).$

The first two functions in the right-hand side of Eq. (17) represent the
losses for the two GANs and those are defined by

$\displaystyle{\cal L}_{\mathrm{GAN}}(G_{X},D_{X})$ $\displaystyle=$
$\displaystyle\mathbb{E}_{y}\log\left[D_{X}(y)\right]$ (18)
$\displaystyle+\mathbb{E}_{x}\log\left\\{1-D_{X}\left[G_{X}(x)\right]\right\\},$
$\displaystyle{\cal L}_{\mathrm{GAN}}(G_{Y},D_{Y})$ $\displaystyle=$
$\displaystyle\mathbb{E}_{x}\log\left[D_{Y}(x)\right]$ (19)
$\displaystyle+\mathbb{E}_{y}\log\left\\{1-D_{Y}\left[G_{Y}(y)\right]\right\\},$

where $\mathbb{E}_{x}$ represents the expected value over a sample of $x$ and
so on. The term of ${\cal L}_{\mathrm{cyc}}$ in Eq. (17) regularizes the two
generators of $G_{X}$ and $G_{Y}$ so that the learned mapping between $X$ and
$Y$ should be cycle-consistent. It is given by

$\displaystyle{\cal L}_{\mathrm{cyc}}(G_{X},G_{Y})$ $\displaystyle=$
$\displaystyle\mathbb{E}_{x}\left|G_{Y}\left[G_{X}(x)\right]-x\right|_{\mathrm{1}}$
(20)
$\displaystyle+\mathbb{E}_{y}\left|G_{X}\left[G_{Y}(y)\right]-y\right|_{\mathrm{1}},$

where $|\cdots|_{\mathrm{1}}$ is the L1 norm. We further constrain the
generators with the term of ${\cal L}_{\mathrm{id}}(G_{X},G_{Y})$ so that they
can provide an identity mapping when real samples of the target domain are
used as their input, i.e.

$\displaystyle{\cal L}_{\mathrm{id}}(G_{X},G_{Y})$ $\displaystyle=$
$\displaystyle\mathbb{E}_{y}\left|G_{X}(y)-y\right|_{\mathrm{1}}+\mathbb{E}_{x}\left|G_{Y}(x)-x\right|_{\mathrm{1}}.$
(21)

We have two hyperparameters of $\lambda_{1}$ and $\lambda_{2}$ in the loss and
set $\lambda_{1}=10$ and $\lambda_{2}=5$ throughout this paper.

##  IV Data and Methods

###  IV.1 High-cost Lensing Simulations

To train the Cycle GAN, we use publicly available numerical simulations of
weak gravitational lensing as performed in Ref. Liu  _et al._ (2018). The
simulation suite is referred to as MassiveNuS (Cosmological Massive Neutrino
Simulations), aiming at detailed studies of cosmic large-scale structures in
the presence of massive neutrinos. MassiveNuS consists of 101 models in
flat-$\Lambda$CDM cosmology. In this paper, we adopt the simplest
flat-$\Lambda$CDM model including massless neutrinos with cosmological
parameters below; total mass density $\Omega_{\mathrm{m0}}=0.3$, baryon
density $\Omega_{\mathrm{b0}}=0.046$, cosmological constant
$\Omega_{\Lambda}=1-\Omega_{\mathrm{m0}}=0.7$, present-day Hubble parameter
$H_{0}=100h=70.0\,\mathrm{km}/\mathrm{s}/\mathrm{Mpc}$, primordial scalar
spectrum power-law index $n_{s}=0.97$, and the primordial curvature power
spectrum at the pivot scale $k=0.05\,\mathrm{Mpc}^{-1}$ being $A_{s}=2.1\times
10^{-9}$. The authors in Ref. Liu  _et al._ (2018) run N-body simulations with
the public tree-Particle Mesh code Gadget-2 Springel (2005), adopting a box
size of $512\,h^{-1}\mathrm{Mpc}$ and $1024^{3}$ particles. The Zel’dovich
approximation Zeldovich (1970) was adopted when they generated the initial
condition at $z=99$ for the N-body simulations. From the N-body simulation
snapshots, they then produced 10,000 convergence maps at five different source
redshifts of $z_{\mathrm{source}}=0.5,1.0,1.5,2.0$, and $2.5$ by using the
public ray-tracing code LensTools Petri (2016). Each map covers a sky of
$3.5^{2}\,\mathrm{deg}^{2}$ with the number of pixels on a side being 512. In
this paper, we degrade the convergence map at $z_{\mathrm{source}}=1.0$ from
the original resolution of $512\times 512$ to $256\times 256$ as it is used
for the input of the generators in our GAN.

###  IV.2 Low-cost Gaussian Simulations

As another training dataset for the Cycle GAN, we produce 10,000 convergence
maps assuming the Gaussian model as described in Section III.1. In the
Gaussian model, we set the convergence power spectrum of $C$ so that it can be
in agreement with the average power spectrum in the lensing simulations. To be
specific, we assume a functional form of the power spectrum below;

$\displaystyle
C(\ell)=C_{\mathrm{ref}}(\ell)\,\left[1+\left(\frac{\ell}{\ell_{0}}\right)^{\alpha}\right]^{-\beta},$
(22)

where $C_{\mathrm{ref}}(\ell)$ represents a reference model of the power
spectrum, and there are three free parameters of $\ell_{0}$, $\alpha$ and
$\beta$ to account for resolution effects in the lensing simulations. Using
Eq. (5) with the Limber approximation Limber (1954), we compute the reference
model of $C_{\mathrm{ref}}$ as

$\displaystyle C_{\mathrm{ref}}(\ell)=\int_{0}^{\infty}{\rm
d}\chi\frac{W^{2}_{\kappa}(\chi)}{r^{2}(\chi)}P_{\delta}\left(k=\frac{\ell}{r(\chi)},z(\chi)\right),$
(23)

where $P_{\delta}(k,z)$ is the three dimensional matter power spectrum at
redshift $z$, and it is computed as the fitting formula developed in Ref.
Takahashi  _et al._ (2012). We constrain the free parameters of $\ell_{0}$,
$\alpha$, and $\beta$ by least-squares fitting for the power spectrum in the
lensing simulations. When estimating the convergence power spectrum from the
simulations, we first adopt the fast Fourier transform of the convergence
field and then measure the binned power spectrum of the convergence field by
averaging the product of Fourier modes $|\kappa_{\bm{\ell}}|^{2}$. We employ
15 bins logarithmically spaced in the range of $\ell=100$ to $\ell=10^{4}$. In
the end, we find that Eq. (23) can explain the average power spectrum over
10,000 simulations at $z_{\mathrm{source}}=1$ within a $5\%$-level accuracy
when setting $\ell_{0}=1.360\times 10^{4}$, $\alpha=1.643$, and $\beta=4.088$.
This pre-training of the Gaussian model allows the Cycle GAN to learn non-
Gaussian information about the lensing simulations in an efficient way. Also,
it is important to prepare training sets with reasonable similarities when one
trains the GAN for an image-to-image translation Isola  _et al._ (2017); Zhu
_et al._ (2017); Shirasaki  _et al._ (2019b).

###  IV.3 Training Cycle GAN

Figure 1:  labeling of training datasets with field variances of lensing
convergence fields. Different colored squares in the lower left panel stand
for labels of two different datasets. The labels are set by the field variance
of $\langle\kappa^{2}\rangle$ for either simulated or Gaussian fields. In this
figure, we consider 10 labels. The horizontal axis in the lower left
represents the variance of the Gaussian convergence fields, while the vertical
axis shows the simulated counterpart. The upper and right panels show the
cumulative distribution of the variances over 10,000 realizations of the two
different fields. When training the Cycle GAN, we use the datasets with
identical labels, requiring each of 10 different generators to be trained with
1000 realizations of the training sets. It is worth noting that each
realization of Gaussian and simulated fields is not aligned, e.g. the two do
not share phase information in a survey window with each other.

For training the Cycle GAN, we follow the procedure as adopted in Ref. Zhu
_et al._ (2017). To make the training stable, we replace the negative log-
likelihood objectives in Eqs. (18) and (19) with least-squares loss functions
Mao  _et al._ (2017). In practice, we train $G_{X}$ to minimize
$\mathbb{E}_{x}[D_{X}(G_{X}(x))-1]^{2}$ and then update the parameters in
$D_{X}$ to minimize
$\mathbb{E}_{y}[D_{X}(y)-1]^{2}+\mathbb{E}_{x}[D_{X}(G_{X}(x))]^{2}$ in Eq.
(18). A similar replacement is applied in Eq. (19). Furthermore, we update the
discriminators using a history of generated images rather than the ones
produced by the latest generators. We keep an image buffer that stores the 50
previously created images. This strategy is effective in reducing a gap
between synthetic and real image distributions Shrivastava  _et al._ (2017).
We use the Adam solver Kingma and Ba (2015) with a batch size of 1 to update
the parameters in the Cycle GAN. All networks are trained with a learning rate
of 0.0002 for the first 100 epochs, while we linearly decay the rate to zero
over the next 100 epochs. Weights are initialized from a Gaussian distribution
with zero mean and standard deviation of 0.02.

When using the training datasets without any annotations, we find that
convergence fields produced by the generator $G_{X}$ can reproduce some
summary statistics observed in the lensing simulations on average, whereas
$G_{X}$ fails to predict the variance as in the lensing simulations. To
improve the performance of the generator, we divide the training sets into $N$
groups by using a label that is relevant to variations in the convergence
fields. Among several candidates, we decided to use the field variance
$\langle\kappa^{2}\rangle$ to measure the variations. This choice is
physically motivated as the scatter in the lensing power spectrum $C$ at large
$\ell$ is mostly determined by four-point correlations with squeezed
quadrilaterals including a shared infinite wavelength mode Takada and Hu
(2013). According to a halo-model prescription Cooray and Sheth (2002), the
large-$\ell$ variance in $C$ depends on the variance of matter density in a
survey window, sharing some information with the field variance of
$\langle\kappa^{2}\rangle$. Hence, we expect that the field variance
$\langle\kappa^{2}\rangle$ provides a good measure of the variation in lensing
convergence fields. An example of labeling the training sets is shown in
Figure 1. In the figure, we divide the training sets into $N=10$ groups by
their field variances. We then train the Cycle GAN by using the lensing
simulations and the Gaussian data with the same label of $i$
($i=1,2,\cdots,N$). In the end, we will have $N$ different generators to
produce non-Gaussian convergence fields from given Gaussian fields. When
producing a new non-Gaussian field, we select which generator will be used by
the field variance of an input Gaussian field. Throughout this paper, we set
$N=10$ for our fiducial choice. The $N$ dependence of our results is examined
in Section V.2.

As a by-product, the Cycle GAN provides a generator to translate non-Gaussian
convergence fields into Gaussian counterparts. Although this translation is
not our primary purpose, it is interesting to see if the Cycle GAN can be used
for a reasonable Gaussianization of weak lensing mass maps. Note that
appropriate Gaussianization enhances the cosmological constraining power of
weak lensing fields Yu  _et al._ (2011); Seo  _et al._ (2011, 2012); Simpson
_et al._ (2016). Appedix B summarizes the results of the translation from the
non-Gaussian to the Gaussian mass maps. A simple training without any labels
can gaussianize the non-Gaussian weak lensing mass maps obtained by the ray-
tracing simulation with a reasonable precision; The Gaussianized power
spectrum can reproduce the true Gaussian counterpart with a level of $2\%$ at
$\ell\lower 2.15277pt\hbox{$\;\buildrel>\over{\sim}\;$}400$, while the
Gaussianized one-point PDF is well described by the expected Gaussian
distribution. Also, the average bispectrum over 10,000 Gaussianized lensing
mass maps is consistent with a null detection within the statistical
uncertainty for a sky coverage of $3.5^{2}\,\mathrm{deg}^{2}$. We also find
that field-variance-based labels do not help to improve the performance in the
translation when the target domain is set to the Gaussian model.

###  IV.4 Summary Staistics

Because our neutral-network-based model is trained in an unsupervised fashion,
it is non-trivial to assess the performance of our model. In this paper, we
validate the model by studying various summary statistics of generated (fake)
convergence fields and comparing them with their counterparts in the ground-
truth lensing simulations. Because of non-Gaussian natures in the lensing
convergence, several summary statistics have been studied in the literature.
Apart from the standard power spectrum, we here consider five non-Gaussian
statistics of bispectrum, peak counts, one-point probability distribution
function (PDF), Minkowski functionals (MFs), and scattering transform
coefficients. Characteristics of each non-Gaussian statistic are briefly
summarized below. Table 1 provides all the summary statistics of interest in
this paper with a short description.

Table 1: Summary statistics of lensing convergence fields. Those statistics are used for validation of the performance of our neural network-based generative model. We are interested in not only the ensemble average but also the covariance matrix of each statistic.  Name | Symbol | Note
---|---|---
Power Spectrum | $C$ | Variance of Fourier modes at a given multiple $\ell$
Bispectrum | $B$ | Three-point correlations among Fourier modes at a given triangle configuration
Peak counts | $N_{\mathrm{pk}}$ | Number density of local maxima
One-point PDF | ${\cal P}$ | Histogram of pixel values
Minkowski functionals (MFs) |  $V_{1}$ and $V_{2}$ | Total boundary length and integral of geodesic curvature along contours
Scattering transform coefficients |  $s_{1}$ and $s_{2}$ | A compressed set of two- and four-point spatial correlations

#### Bispectrum

The bispectrum is the three-point correlation in Fourier space and is defined
as

$\displaystyle\langle\tilde{\kappa}(\bm{\ell}_{1})\tilde{\kappa}(\bm{\ell}_{2})\tilde{\kappa}(\bm{\ell}_{3})\rangle$
$\displaystyle=$
$\displaystyle(2\pi)^{2}\delta_{D}(\bm{\ell}_{1}+\bm{\ell}_{2}+\bm{\ell}_{3})$
(24) $\displaystyle\quad\quad\quad\quad\times
B(\bm{\ell}_{1},\bm{\ell}_{2},\bm{\ell}_{3}),$

where $B$ represents the bispectrum. Because it holds that $B=0$ at any
Fourier-mode configurations for the Gaussian convergence model, the bispectrum
provides a powerful means of studying the lowest-order non-Gaussian
information in the convergence field Hui (1999); Cooray and Hu (2001); Benabed
and Bernardeau (2001); Takada and Jain (2004); Refregier  _et al._ (2004);
Dodelson and Zhang (2005); Coulton  _et al._ (2019). As in the case of power
spectra, one can relate the bispectrum to the three-dimensional matter
bispectrum $B_{\delta}$;

$\displaystyle B(\bm{\ell}_{1},\bm{\ell}_{2},\bm{\ell}_{3})$ $\displaystyle=$
$\displaystyle\int_{0}^{\infty}{\rm
d}\chi\frac{W^{3}_{\kappa}(\chi)}{r^{4}(\chi)}$ (25)
$\displaystyle\quad\quad\times
B_{\delta}\left(\bm{k}_{1},\bm{k}_{2},\bm{k}_{3},z(\chi)\right),$

where we evaluate the wavenumbers in the three-dimensional space as
${\bm{k}_{i}=\bm{\ell}_{i}/r(\chi)}$. Theoretical and numerical studies have
predicted that the bispectrum can add a $20-50$ percent-level gain to signal-
to-noise ratio of the power spectrum up to a maximum multipole of a few
thousands Kayo  _et al._ (2013); Sato and Nishimichi (2013).

To estimate the bispectrum from a given pixelized data of the convergence, we
measure an average of the product of three Fourier modes
$\mathrm{Re}[\tilde{\kappa}(\bm{\ell}_{1})\tilde{\kappa}(\bm{\ell}_{2})\tilde{\kappa}(\bm{\ell}_{3})]$
over a triangle configuration of
$\bm{\ell}_{1}+\bm{\ell}_{2}+\bm{\ell}_{3}=\bm{0}$ where $\mathrm{Re}[\cdots]$
stands for the real part of a complex number. Throughout this paper, we adopt
15 bins logarithmically spaced in the range of $\ell_{i}\,(i=1,2,3)=100$ to
$10^{4}$ for each of the three multipoles.

#### Peak Counts

The number count of local maxima in a convergence map referred to as peak
counts, has become a popular summary statistic to extract non-Gaussian
cosmological information beyond the standard power-spectrum-based analysis in
practice Liu  _et al._ (2015); Hamana  _et al._ (2015); Kacprzak  _et al._
(2016); Shan  _et al._ (2018); Martinet  _et al._ (2018); Harnois-Déraps  _et
al._ (2021); Zürcher  _et al._ (2022); Liu  _et al._ (2022). Peaks with
heights greater than a $\sim 4\sigma$ noise level arise from single massive
dark matter haloes Hamana  _et al._ (2004); Hennawi and Spergel (2005); Maturi
_et al._ (2005); Marian  _et al._ (2012); Hamana  _et al._ (2012), whereas
peaks with modest heights can be created by superposition of large-scale
structures and multiple dark matter halos Yang  _et al._ (2011); Liu and
Haiman (2016); Sabyr  _et al._ (2022).

In this paper, we define the peak as a pixel that has a higher value than all
of its eight neighbor pixels in the discretized convergence field. Note that
we do not apply any additional smoothing to count peaks. We consider 41 bins
in the range of $-0.05\leq\kappa\leq 0.15$. Since a typical field variance of
the convergence field in our simulations is found to be $3\times 10^{-4}$ (see
Figure 1), the largest peak heights in our analysis correspond to a $\sim
9\sigma$ significance level.

#### One-point Probability Distribution Function

The one-point PDF of the convergence contains vital information about the non-
Gaussian matter density field Bernardeau and Valageas (2000); Liu and
Madhavacheril (2019); Barthelemy  _et al._ (2021); Thiele  _et al._ (2020);
Boyle  _et al._ (2021); Giblin  _et al._ (2023). It encodes the information
originating from moments of all orders in the convergence, allowing us to
study the non-Gaussian nature of the weak lensing field at a given length
scale. As a one-point statistic, the PDF can be easily obtained from simulated
and real data even in the presence of complex survey boundaries Friedrich  _et
al._ (2018); Gruen  _et al._ (2018). In this paper, we measure the PDF from a
histogram using 41 linearly spaced bins in the range of $-0.05\leq\kappa\leq
0.15$.

#### Minkowski Functionals

MFs are a useful statistic to study morphological information encoded in
smoothed random fields. For two-dimensional maps as the lensing convergence,
three MFs of $V_{0}$, $V_{1}$, and $V_{2}$ are given by the area in which the
convergence is above a threshold of $\kappa_{\mathrm{thre}}$, the total
boundary length, and the integral of geodesic curvature along the contours,
respectively. In particular, $V_{2}$ is equal to the number of connected
regions above the threshold, minus those below the threshold. Therefore, as
the threshold increases, $V_{2}$ provides almost the same information as the
peak counts.

MFs can be evaluated by a perturbative approach when the field of interest
exhibits a weak non-Gaussianity Matsubara (2003a); Munshi  _et al._ (2012);
Matsubara and Kuriki (2021), but it is yet difficult to predict MFs of highly
non-Gaussian fields with analytic approaches Petri  _et al._ (2013).
Simulation-based predictions of the convergence MFs have been constructed so
far, demonstrating that the MFs can carry cosmological information beyond the
standard power spectrum analysis Kratochvil  _et al._ (2012); Shirasaki and
Yoshida (2014); Petri  _et al._ (2013). Note that $V_{0}$ provides a
cumulative PDF of the lensing convergence, sharing some information with the
aforementioned one-point PDF. Hence, we focus on the two MFs of $V_{1}$ and
$V_{2}$ in this paper. We estimate the MFs from a pixelized map by following
the method in Refs. Lim and Simon (2012); Kratochvil  _et al._ (2012). We
compute MFs for 41 equally spaced bins of $\kappa_{\mathrm{thre}}$ between
$-0.05$ and $0.15$.

#### Scattering Transform Coefficients

The scattering transform (ST) provides a powerful means of extracting
information from high-dimensional data, while it has mathematically tractable
properties of translational invariance, non-expanding variance, and Lipschitz
continuous to spatial deformation Mallat (2012). The ST generates a set of new
fields from a given input field by applying a wavelet convolution and a
modulus recursively and uses the expected values of the new fields as a
collection of statistical information encoded in the original input field.
Hence, the ST performs a similar operation to what a common convolutional
neural network (CNN) does in practice, i.e. either the ST or the CNN relies on
the use of localized convolution kernels, and a non-expansive non-linear
operator. Ref. Cheng  _et al._ (2020) has shown that the ST statistic can
provide cosmological constraints as tight as the CNN does Ribli  _et al._
(2019) for given weak lensing mass maps. It would be worth noting that the
CNN-based cosmological inference requires a costly training process with
simulations, whereas the ST statistic simply needs to emulate its expected
values as a function of cosmology (as same as other summary statistics do).
Considering these points, it is interesting to see if fake convergence fields
generated by our model can reproduce the ST statistic as in the true lensing
simulations.

In this paper, we measure the ST statistic by following the method in Ref.
Cheng  _et al._ (2020). For a given lensing convergence field $\kappa$, we
first generate a set of first-order fields as
$I_{1}\equiv|\kappa\star\Psi^{j_{1},l_{1}}|$, where $\kappa\star\Psi^{j,l}$
represents a convolution with wavelets $\Psi^{j,l}$ with a size index $j$ and
orientation index $l$, and $|\cdot|$ stands for the modulus of a complex-value
field. The second-order fields are then defined as
$I_{2}\equiv|I_{1}\star\Psi^{j_{2},l_{2}}|$. In this paper, we consider the ST
statistic up to second order, which can be interpreted as a compact set of
spatial information about two- and four-point correlation functions Cheng  _et
al._ (2020). See also Ref. Cheng and Ménard (2021) for a more intuitive
understanding of the ST statistic.

From a set of $\kappa,I_{1},$ and $I_{2}$, we define the ST statistic as

$\displaystyle S_{0}$ $\displaystyle\equiv$
$\displaystyle\langle\kappa\rangle,$ (26) $\displaystyle S_{1}(j_{1},l_{1})$
$\displaystyle\equiv$ $\displaystyle\langle I_{1}\rangle,$ (27) $\displaystyle
S_{2}(j_{1},j_{2},l_{1},l_{2})$ $\displaystyle\equiv$ $\displaystyle\langle
I_{2}\rangle,$ (28)

where $\langle\cdot\rangle$ represents the spatial average over a survey
window. We refer to $S_{0},S_{1}$, and $S_{2}$ as the ST coefficients. In this
paper, we work with Morlet wavelets with $L=4$ different rorations. One can
construct a family of Morlet wavelets by dilating a prototype wavelet by
factors of $2^{j}$ and rotating it by $l\times 45$ degrees. The prototype
Morlet wavelet is defined as

$\displaystyle\Psi_{0}(x,y)$ $\displaystyle=$
$\displaystyle\frac{\exp\left(-\frac{x^{2}+y^{2}}{2\sigma^{2}}\right)}{\sqrt{2}\sigma}$
(29)
$\displaystyle\times\left[\exp\left(ik_{0}x\right)-\exp\left(-\frac{k^{2}_{0}\sigma^{2}}{2}\right)\right],$

where $\sigma=0.8\,\mathrm{pixel}$ and $k_{0}=3\pi/4\,\mathrm{pixel}^{-1}$.

As proposed in Ref. Cheng  _et al._ (2020), we further reduce the ST
coefficients by averaging over the rotation indices of $l$ because of isotropy
in the lensing convergence. The relevant ST coefficients to the convergence up
to the second order are then given by

$\displaystyle s_{1}(j_{1})$ $\displaystyle=$
$\displaystyle\frac{\sum_{l_{1}}S_{1}(j_{1},l_{1})}{\sum_{l_{1}}1},$ (30)
$\displaystyle s_{2}(j_{1},j_{2})$ $\displaystyle=$
$\displaystyle\frac{\sum_{l_{1},l_{2}}S_{2}(j_{1},j_{2},l_{1},l_{2})}{\sum_{l_{1},l_{2}}1}.$
(31)

Throughout this paper, we consider the Morlet wavelets with $J=7$ dyadic
scales. In this case, we have 7 and 21 independent elements in $s_{1}$ and
$s_{2}$, respectively. We measure the ST coefficients of $s_{1}$ and $s_{2}$
from a given convergence field with a publicly available code developed by S.
Cheng111https://github.com/SihaoCheng/scattering_transform.

##  V Results

###  V.1 Visual impression

Figure 2:  An example of lensing convergence fields generated by our neural
network. The left panel shows an input Gaussian field, while the generated
convergence by our model is presented on the right. For comparison, the middle
panel represents a log-normal model with the same phase information as the
Gaussian field. In each panel, the inset figure highlights a zoom-in view
around the highest convergence value in a field of view covering
$3.5^{2}\,\mathrm{deg}^{2}$.

We first see what convergence fields generated by our GAN-based model look
like. Figure 2 shows an example of the fake convergence fields as well as an
input Gaussian convergence and a popular log-normal model. For the log-normal
model, we assume the power spectrum of Eq. (22) so that the model can
reproduce the average power spectrum in the lensing simulations. We also set
the minimum convergence of $\kappa_{\mathrm{min}}=-0.0367$ for the log-normal
model, which is equal to the minimum value over 10,000 lensing simulations.

The right panel in the figure displays the convergence field produced by our
generators with the input being the Gaussian field shown on the left. Our
generator adds prominent features at positive convergence regions, mimicking
highly non-linear gravitational effects in dark matter haloes found in N-body
simulations. Similar effects can be seen in the log-normal model as shown in
the middle, but we find that our generators tend to make convergence values
higher than the log-normal counterpart. This is necessary to make a long tail
in the one-point PDF of the fake convergence fields, as predicted by previous
numerical simulations Das and Ostriker (2006); Wang  _et al._ (2009);
Takahashi  _et al._ (2011); Shirasaki (2017). It is worth noting that our
model preserves large-scale phase information encoded in the input Gaussian
model. Hence, the model enables us to produce a set of lensing convergence
fields in an intuitive way for cosmological analyses.

###  V.2 Power spectrum

We then investigate the most basic summary statistic of the convergence power
spectrum $C(\ell)$ generated by our neural network-based model. Figure 3
summarizes the statistical nature of $C(\ell)$ predicted by our model and
compares it with the counterparts of the true lensing simulations. In the
figure, we look into three properties of ensemble average, variance, and a
cumulative signal-to-noise ratio defined as

$\displaystyle\left(\frac{\mathrm{S}}{\mathrm{N}}\right)[\ell_{\mathrm{max}}]=\left[\sum_{\ell_{i},\ell_{j}\leq\ell_{\mathrm{max}}}C(\ell_{i})\,\mathrm{Cov}^{-1}(\ell_{i},\ell_{j})\,C(\ell_{j})\right]^{1/2},$
(32)

where $C(\ell_{i})$ is the power spectrum at the i-th multipole bin of
$\ell_{i}$, $\mathrm{Cov}(\ell_{i},\ell_{j})$ is the covariance between
$C(\ell_{i})$ and $C(\ell_{j})$, and the sum in Eq. (32) runs over all indices
with $\ell_{i}\leq\ell_{\mathrm{max}}$ and $\ell_{j}\leq\ell_{\mathrm{max}}$.

We find that the convergence power spectrum predicted by our model is in
agreement with the counterparts in the true lensing simulations in many
aspects. On the ensemble average, the prediction by our generators provides an
accurate fit to the simulation counterpart as shown in the left panels in
Figure 3. Apart from the largest angular scales of $\ell\lower
2.15277pt\hbox{$\;\buildrel<\over{\sim}\;$}300$ (corresponding to $\lower
2.15277pt\hbox{$\;\buildrel>\over{\sim}\;$}1\,\mathrm{degree}$), our
generators can reproduce the information about two-point correlation functions
in the lensing simulations within a 1%-level accuracy. The variance in
$C(\ell)$ by our generators is also consistent with the simulation counterpart
in a wide range of multipoles. We observe that our generators can enhance the
variance at smaller angular scales as predicted by the ground truth. Note that
this enhancement arises from non-linear effects in gravitational growth of
cosmic mass density Takada and Hu (2013), sourcing the non-Gaussanity in the
lensing convergence. Similar non-Gaussian contributions are also prominent in
off-diagonal elements in the covariance matrix, reducing the cumulative S/N of
$C(\ell)$ as small-scale information is added Sato  _et al._ (2009). The right
panels in Figure 3 demonstrate that the power spectra produced by our
generators preserve almost the same information contents as in the true
simulation. The degradation in the S/N with respect to the Gaussian
expectation can be explained by our model as well.

Figure 3:  Comparisons of convergence power spectra among several generative
models. The left panel shows the average power spectra, while the middle
presents the variance at a given multipole $\ell$. In the right panel, we
compare the cumulative signal-to-noise ratio (S/N) defined by Eq. (32). In
each panel, the blue circles represent the results of the true lensing
simulations, whereas the black solid and orange dashed lines stand for ones by
our GAN-based and the Gaussian models, respectively. At the bottom, we also
show the fractional difference between the GAN predictions and the simulation
counterparts, highlighting that our generators can reproduce average,
variance, and the cumulative S/N at $300\lower
2.15277pt\hbox{$\;\buildrel<\over{\sim}\;$}\ell\lower
2.15277pt\hbox{$\;\buildrel<\over{\sim}\;$}2000$ within a 10%-level accuracy.
Note that all the results in this figure are based on 10,000 realizations of
the simulations, fake convergence fields generated by our networks, and
Gaussian models at a source redshift being $z_{\mathrm{source}}=1$. Each
realization covers a sky of $3.5^{2}\,\mathrm{deg}^{2}$.

It is beneficial to compare the covariance matrix of $C(\ell)$ by our model
with the simulation counterpart in a direct manner because the covariance
plays a central role in determining the constraining power of $C(\ell)$ for a
given survey setup. Figure 4 summarizes the covariance matrix predicted by our
generators as well as the simulation counterparts. Each panel represents the
covariance $\mathrm{Cov}(\ell,\ell^{\prime})$ as a function of $\ell$ at a
fixed $\ell^{\prime}$. Note that we normalize the covariance so that its
diagonal element can be unity. For comparison, the predictions by the popular
log-normal models are shown in the orange dashed lines in the figure. We find
that the log-normal model can account for some levels of the non-Gaussian
covariance, but its validity is still limited. The log-normal model introduces
a non-zero covariance at the smallest scales of $\ell\sim 10^{4}$, but it
cannot explain the cross-correlations between two $\ell$ bins when the
difference in $\ell$ becomes large. Our neutral-network-based model can
account for significant covariance at a wide range of $\ell$ as seen in the
true simulation data. Note that the significant non-Gaussian covariance in the
true lensing simulations can be mostly explained by statistical fluctuations
in the number of dark matter halos sampled by a finite survey area, referred
to as halo sample variance (HSV) Sato  _et al._ (2009); Takada and Hu (2013).
Our model can efficiently learn the HSV effects.

Figure 4:  A close look at covariance matrices of convergence power spectra
among several models. Each panel summarizes the covariance at different two
scales of $\ell$ and $\ell^{\prime}$ as normalized to unity at the diagonal
elements. The blue circles in each panel represent the simulation results,
while the black solid lines stand for the predictions by our generative model.
For comparison, the orange dashed lines in each panel show the covariance
matrix estimated by 10,000 realizations of the log-normal model. This figure
validates that the covariance matrix by our generators can provide a
reasonable fit to the simulation results, making our model valuable for
standard cosmological analyses based on two-point correlation functions.

To get some sense of the learning experience of our model, we examine to what
extent our model performance can depend on the choice of the number of labels
in the training datasets. It is worth noting that we introduce the labeling of
the training datasets by the field variance of lensing convergence fields as
in Figure 1. This labeling ensures that the convergence field generated by our
generators can have a similar field variance to the true counterpart. Because
the HSV effects in the covariance of $C(\ell)$ depend on the variance in
cosmic mass density in a survey window, we expect that the field-variance-
based labeling can make our model efficiently learn the HSV. Figure 5 shows
the model performance when we vary the number of labels in the training
datasets. We observe that the training with no labels cannot appropriately
teach the HSV effects. The model trained without the field-variance-based
labels cannot fully explain the non-Gaussian covariance at small angular
scales which is a distinct feature in the true lensing simulations. The
labeling significantly improves in predicting the scatter in $C(\ell)$ at a
wide range of $\ell$, indicating that the field variance is closely related to
the HSV effects in the lensing convergence field as expected. We also confirm
that our model performance can be well converged as long as the number of
labels is set to 10. Our results highlight the importance of physically-
motivated annotations in unsupervised training of neural networks for
cosmological purposes.

Figure 5:  Dependence of our model performance on the number of labels based
on field variances. Similar to Figure 3, but we show the results when varying
the number of labels $N$ in the training datasets. The blue circles in each
panel are the results obtained from 10,000 lensing simulations, while
different colored lines stand for the model predictions with different $N$.

###  V.3 Bispectrum

We study the lowest-order non-Gaussian statistic of the lensing bispectrum
predicted by our neural network-based model. The bispectrum provides
information about three-point spatial correlations among several length
scales. Because our model aims for an image-to-image translation and the input
Gaussian convergence should have zero bispectrum on average, the bispectrum
can be a good measure to look into the non-Gaussianity acquired through the
translation by our model.

Figure 6 summarizes the average and variance of the bispectrum over 10,000
realizations of the fake convergence fields by our model. In the figure, we
consider three representative configurations of Fourier modes, i.e.
equilateral, flattened and squeezed shapes. We find that our model can
reproduce the equilateral and flattened shaped bispectra in the simulations
with a level of $<10\%$ on average, while its variance is also consistent with
the true simulation counterparts. It would be worth emphasizing that the
popular log-normal model cannot provide a reasonable fit to the bispectrum in
the true simulation as seen in Section V.4. On the squeezed configuration, we
observe that the model prediction deviates from the true counterparts with a
level of $\sim 30\%$ and $\sim 50\%$ for the average and variance,
respectively. Note that the current state-of-the-art fitting formula of the
matter bispectrum can explain the lensing bispectrum in simulations with a
$10\%$-level deviation Takahashi  _et al._ (2020).

Hence, our model learns the image-to-image translation so that it can preserve
novel information about the three-point correlations in the lensing
simulations. This highlights that our model does not modulate convergence
values from the input Gaussian one on a pixel-by-pixel basis. Phase
correlations in the Fourier space are a key ingredient for non-zero bispectrum
Matsubara (2003b), whereas there are no phase correlations in the input
Gaussian model by construction (see Section III.1). The non-zero bispectra
predicted by our model indicates that the image-to-image translation is
operated in a multi-scale manner.

Figure 6:  Comparison of convergence bispectra predicted by the lensing
simulation and our model. The left, middle, and right panels show the bispecra
with three different configurations of equilateral, flattened, and squeezed
shapes, respectively. The blue circles with the error bars in the upper panels
represent the simulation results, while the black solid lines stand for the
model prediction. The shaded region in the upper panels highlights the
statistical uncertainty of the bispectra by our model. In the second row, we
show the ratio of the average bispectra between the two (the black solid lines
there show the model bispectra divided by the simulation counterparts). The
bottom panels summarize the ratio of the variance. The gray lines at the
bottom are the model variance divided by the simulation counterparts. All the
results are based on 10,000 realizations of the simulations and the neural
network-based model.

###  V.4 Comparison with log-normal model

In this section, we check the performance of our model to predict various
summary statistics and compare the model predictions with the log-normal
counterparts. Figure 7 shows a set of summary statistics by our neural-
network-based model, including the equilateral-shaped bispectrum, the peak
counts, the one-point PDF, the MFs, and the ST coefficients. We see that the
log-normal bispectrum is off from the simulation counterparts, whereas our
model can reproduce the equilateral bispectrum in the simulations in an
accurate way. On the peak count, the one-point PDF, and the MFs, we find that
our model improves in predicting statistical properties around rare
convergence regions with a $\lower
2.15277pt\hbox{$\;\buildrel>\over{\sim}\;$}5\sigma$ significance compared to
the log-normal model. We also observe that the ST coefficients by our model
are in good agreement with the simulation counterparts, implying that our
model may be useful for cosmological inferences based on neural networks.

Importantly, we also find that our model can reproduce the variance of each
statistic in the true simulations. For instance, our model can predict the
variance of the equilateral bispectrum over a wide range of $\ell$ with a
10%-level accuracy, while it can provide the variances of the peak counts, the
one-point PDF, and the MFs in the range of $0\lower
2.15277pt\hbox{$\;\buildrel<\over{\sim}\;$}\kappa\lower
2.15277pt\hbox{$\;\buildrel<\over{\sim}\;$}0.15$ with a 20-30% level accuracy.
We also observe that the labeling of the training datasets with the field
variance is effective in predicting the variance of various summary
statistics. When we trained our model without introducing any labels, the
model tended to underestimate the variance of the peak counts, the one-point
PDF, and the MFs at a wide range of $\kappa$ by $50\%$ or larger.

Figure 7:  Convergence summary statistics predicted by different generative
models. Different seven panels summarize the statistics of the equilateral
bispectrum, the peak count, the one-point PDF, the first MF ($V_{1}$), the
second MF ($V_{2}$), and the ST coefficients up to second order ($s_{1}$ and
$s_{2}$). In each panel, the blue circles with error bars show the average
statistic and standard deviation obtained from the 10,000 lensing simulations.
The black solid lines in the panels are our model predictions, and the gray-
shaded regions stand for the standard deviation in our model. For comparison,
the orange dashed lines represent the average statistics estimated by 10,000
realizations of the log-normal model.  Figure 8:  Distributions of chi-square
quantities for various summary statistics. The chi-square quantity for a given
statistic $S$ is defined as in Eq. (33). In each panel, the blue histogram
shows the distribution obtained from 10,000 lensing simulations, while the
black solid lines are our model predictions based on the neural networks. For
comparison, the orange dashed line in each panel represents the distribution
for 10,000 log-normal realizations. In this figure, we use all bins of
statistics $S$ measured in a given convergence field with a sky coverage of
$3.5^{2}\,\mathrm{deg}^{2}$. Details of binning are found in Section IV.4. It
is worth noting that the results in this figure do not account for any
observational effects.

To compare our model predictions with the log-normal counterparts in a
quantitative way, we introduce a chi-square quantity defined as

$\displaystyle\chi_{r}^{2}[S]=\sum_{i,j}(S^{(r)}_{i}-\bar{S}_{i})\,\mathrm{Cov}^{-1}_{ij}[S]\,(S^{(r)}_{j}-\bar{S}_{i}),$
(33)

where $\bar{S}_{i}$ represents the ensemble average at the i-th bin of a given
binned summary statistic $S$, $\mathrm{Cov}_{ij}[S]$ is the covariance matrix
for the binned $S$, and $S^{(r)}_{i}$ is the i-th summary statistic at the
$r$-th realization. We evaluate the ensemble averages and covariance matrices
for various summary statistics over 10,000 realizations of the true lensing
simulations as well as the fake convergence fields generated by our model and
the log-normal counterparts. We then compute 10,000 chi-square quantities for
the three generative models and compare the histogram of $\chi^{2}[S]$ among
the different models. The comparisons are important in practice because the
chi-square quantity in Eq. (33) commonly provides an estimate of log-
likelihood for the statistic $S$, which is a central part of modern
cosmological analyses.

Figure 8 compares the histograms of $\chi^{2}[S]$ for six different sets of
summary statistics $S$ among the three cases of the true lensing simulations,
the log-normal model, and our neural-network-based model. We find that our
model can reproduce the distribution of $\chi^{2}$ in the true simulations for
every statistic, while the log-normal counterpart exhibits notable differences
from the true one for some statistics. The clearest difference between the
true simulations and the log-normal model is found in the distribution of
$\chi^{2}$ for the bispectrum. Apart from the bispectrum, we observe that the
distribution by our model explains the median in the true $\chi^{2}$
distribution better than the log-normal counterpart. Note that we do not
include any observational effects in the comparison, indicating that the
results in Figure 8 do not provide a realistic log-likelihood function for
given statistics.

##  VI Extension of Sky Coverage

For an application of our generative model, we here examine to produce a weak
lensing mass map covering a continuous sky larger than the field-of-view of
individual training data. Note that previous neutral-network-based models in
the literature Mustafa  _et al._ (2019); Perraudin  _et al._ (2021); Remy  _et
al._ (2023) cannot produce a continuous map if the sky coverage becomes larger
than the one assumed in the training datasets. Considering that our model
utilizes the Gaussian model as latent information, we propose a four-step
procedure to generate a non-Gaussian convergence field with an arbitrary sky
coverage;

  1. (i)

Create a Gaussian realization of the convergence field with a given sky
coverage $\Omega_{\mathrm{large}}$,

  2. (ii)

Chop the Gaussian convergence field into a set of subregions so that each
subregion can cover the field-of-view of single training datasets, denoted as
$\Omega_{\mathrm{small}}=3.5^{2}\,\mathrm{deg}^{2}$,

  3. (iii)

Conduct an image-to-image translation at each subregion with our generators,

  4. (iv)

Concatenate the non-Gaussian fields generated in the previous step (iii).

Figure 9:  Comparison of summary statistics for lensing convergence fields
with different sky coverages. Six panels show the ensemble average of
different summary statistics including power spectrum, equilateral-shaped
bispectrum, peak counts, one-point PDF, and two Minkowski functionals. In each
panel, the blue points represent the average summary statistics over 10,000
realizations of the true lensing simulations, whereas the black solid line
stands for the average obtained from 500 realizations of the fake convergence
fields generated by our neural network. Each lensing simulation covers a
$3.5^{2}\,\mathrm{deg}^{2}$, and the fake field assumes a larger sky coverage
of $12.9^{2}\,\mathrm{deg}^{2}$. For the power spectrum and the bispectrum, we
overlay theoretical predictions based on Eqs. (23) and (25) as shown in the
orange dashed lines. To compute the theoretical predictions, we use the
fitting formula of non-linear matter power spectrum and bispectrum developed
in Refs. Takahashi  _et al._ (2012, 2020).

We then summarize some additional treatments on a step-by-step basis.

##### Step (i)

Because our neural-network-based model needs an input image with the size of
$256\times 256$, we set the size of a large-sky Gaussian convergence field to
$(256M_{x})\times(256M_{y})$ where $M_{x}$ and $M_{y}$ are integers. We
consider $M_{x}=M_{y}=4$ for the demonstration below. Note that the pixel size
of $0.82\,\mathrm{arcmin}$ on a side is required to be the same as in the
training datasets. Hence, a single Gaussian convergence field produced in this
step covers a sky of $\Omega_{\mathrm{large}}=14^{2}\,\mathrm{deg}^{2}$.

##### Step (ii)

When dividing the large-sky Gaussian convergence field into small subregions,
we decompose it into two terms of

$\displaystyle\kappa_{\mathrm{G,large}}(\bm{\theta})=\kappa^{(L)}_{\mathrm{G,large}}(\bm{\theta})+\kappa^{(S)}_{\mathrm{G,large}}(\bm{\theta}),$
(34)

where $\kappa_{\mathrm{G,large}}$ represents the Gaussian convergence field
produced at step (i), and the two fields on the right-hand side are produced
by high- and low-pass filtering of $\kappa_{\mathrm{G,large}}$ such that

$\displaystyle\kappa^{(L)}_{\mathrm{G,large}}(\bm{\theta})$
$\displaystyle\equiv$
$\displaystyle\int\,\mathrm{d}^{2}\theta^{\prime}W^{(L)}(\bm{\theta}-\bm{\theta}^{\prime})\,\kappa_{\mathrm{G,large}}(\bm{\theta}^{\prime}),$
(35) $\displaystyle\kappa^{(S)}_{\mathrm{G,large}}(\bm{\theta})$
$\displaystyle\equiv$
$\displaystyle\int\,\mathrm{d}^{2}\theta^{\prime}W^{(S)}(\bm{\theta}-\bm{\theta}^{\prime})\,\kappa_{\mathrm{G,large}}(\bm{\theta}^{\prime}).$
(36)

In the above, we introduce the high- and low-pass filters of $W^{(L)}$ and
$W^{(S)}$, respectively. In this paper, we define those two filters to be top-
hat filters in Fourier space. To be specific, each filter in Fourier space is
given by

$\displaystyle\tilde{W}^{L}(\bm{\ell})$ $\displaystyle=$ $\displaystyle{\cal
H}(\ell_{\mathrm{cut}}-|\bm{\ell}|),$ (37)
$\displaystyle\tilde{W}^{S}(\bm{\ell})$ $\displaystyle=$ $\displaystyle
1-\tilde{W}^{L}(\bm{\ell}),$ (38)

where ${\cal H}(x)$ is the Heaviside step function which is 1 for $x>0$ and 0
otherwise, and $\ell_{\mathrm{cut}}$ is a cutoff multipole. We calibrate
$\ell_{\mathrm{cut}}$ so that the field variance of
$\langle[\kappa^{(S)}_{\mathrm{G,large}}]^{2}\rangle$ over a
$3.5\,\mathrm{deg}^{2}$ sky can follow the same distribution as in the
Gaussian simulations used for the training. This calibration is important to
avoid covariate shifts Shimodaira (2000), ensuring that input Gaussian images
for our generators should have the same statistical nature as the training
data. We find that $\ell_{\mathrm{cut}}=0.55\ell_{\mathrm{f}}$ provides a good
fit, where $\ell_{\mathrm{f}}=2\pi/\Omega^{1/2}_{\mathrm{small}}=102.8$
represents the fundamental Fourier mode in the training datasets.

We then divide the field of $\kappa^{(S)}_{\mathrm{G,large}}$ into
$M_{x}\times M_{y}$ subregion so that each subregion can cover the sky
coverage of $\Omega_{\mathrm{small}}$. We set the number of pixels to be
$256\times 256$ in each subregion, but we extract each subregion with a
20-pixel overlap region around the edges of every subfield, ensuring that the
final map is continuous after combining subfields Han  _et al._ (2021). Note
that the overlap regions are not used to produce the final map. Hence, this
procedure removes $(M_{x}\times 20)\times(M_{y}\times 20)$ pixels in our map
making, leading the final map to cover a sky coverage of
$12.9^{2}\,\mathrm{deg}^{2}$. We hereafter define
$\kappa^{(S)}_{\mathrm{G,small}}$ as the high-pass filtered Gaussian field
with a sky coverage of $\Omega_{\mathrm{small}}$.

##### Step (iii)

Given the $M_{x}\times M_{y}$ fields of $\kappa^{(S)}_{\mathrm{G,small}}$, we
add non-Gaussian features to individual fields by our generators. This process
is formally written as

$\displaystyle\kappa^{(S)}_{\mathrm{NG,small}}(\bm{\theta})=f_{\mathrm{NN}}\left[\kappa^{(S)}_{\mathrm{G,small}}(\bm{\theta})|\sigma^{2}_{\mathrm{G,small}}\right],$
(39)

where $f_{\mathrm{NN}}(\cdot|\sigma^{2}_{\mathrm{G,small}})$ represents our
neural-network-based model conditioned by the field variance of
$\sigma^{2}_{\mathrm{G,small}}$. For the field variance, we compute
$\sigma^{2}_{\mathrm{G,small}}$ as
$\langle[\kappa^{(S)}_{\mathrm{G,small}}]^{2}\rangle$ over each subregion. It
would be worth noting that we trained 10 generators according to the field
variance of the input Gaussian fields. Hence, we have 10 different models of
$f_{\mathrm{NN}}$ in this paper.

##### Step (iv)

We then combine the fields of $\kappa^{(S)}_{\mathrm{NG,small}}$ after
removing the 20-pixel overlap regions so that a single combined field covers a
$12.9^{2}\,\mathrm{deg}^{2}$ sky. In the joining process, we also add the low-
pass filtered Gaussian field $\kappa^{(L)}_{\mathrm{G,large}}(\bm{\theta})$ in
order to preserve the large-scale information inherent with the latent field
of $\kappa_{\mathrm{G,large}}(\bm{\theta})$.

Figure 9 shows ensemble averages of various lensing statistics for the non-
Gaussian convergence field covering a sky of $12.9^{2}\,\mathrm{deg}^{2}$. In
the figure, we summarize the results obtained from 500 realizations of the
large-sky non-Gaussian convergence fields by following the steps (i)-(iv).
Note that it takes about 60 seconds to produce a large-sky map with
$M_{x}=M_{y}=4$ with the use of a single GPU. The figure highlights that our
model enables us to reproduce the lensing summary statistics as same as the
true lensing simulations even if extending a sky coverage. The upper left
panel shows the average power spectra with two different sky coverages. For
comparison, the orange dashed line in the upper left represents a prediction
by Eq. (23). On the large scale at $\ell\lower
2.15277pt\hbox{$\;\buildrel<\over{\sim}\;$}100$, our model does not learn any
information about the power spectrum in the training process, but it can
provide a good fit to the theoretical prediction. A similar result can be
found in the upper middle panel showing the equilateral bispectrum. We also
observe that our model is still able to reproduce the ensemble averages of
peak counts, one-point PDF, and the MFs as in the true lensing simulations
when the sky coverage of the model is set to be larger than the counterpart of
the training data. Note that the peak counts and MFs can be affected by
possible discontinuity across the subregions in our map-making because those
statistics are defined with the information of spatial derivatives in the
lensing convergence. The results in Figure 9 demonstrate that extension of the
sky coverage based on steps (i)-(iv) works with no practical problems.

##  VII Limitation

Before concluding this paper, we summarize the major limitations of our neural
network-based model. The following issues will be addressed in future studies.

###  VII.1 Squeezed bispectrum

Although our model can reproduce most of the lensing summary statistics as in
the ground truth, there is room for further improvements in the model
performance. We find that the squeezed-shaped bispectrum is one of the most
difficult statistic to accurately predict our current model. The squeezed
bispectrum is commonly defined by the bispectrum
$B(\bm{\ell}_{1},\bm{\ell}_{2},\bm{\ell}_{3})$ with one multipole being
significatly smaller than other multipoles. In the end, the number of
triangles at the squeezed configurations is limited in our training datasets.
To improve the model prediction of the squeezed bispectrum without increasing
the size of training datasets, one may consider a weakly non-Gaussian
convergence field as the input of our generators instead of the simplest
Gaussian model.

For a given theoretical template of the bispectrum
$B_{\mathrm{ref}}(\bm{\ell}_{1},\bm{\ell}_{2},\bm{\ell}_{3})$, one can
construct the non-Gaussian field so that its average bispectrum can be equal
to the theoretical counterpart by Smith and Zaldarriaga (2011); Hall and Mead
(2018)

$\displaystyle\tilde{\kappa}_{\mathrm{NG,pert}}(\bm{\ell})$ $\displaystyle=$
$\displaystyle\tilde{\kappa}_{\mathrm{G}}(\bm{\ell})+\frac{1}{6}\int\frac{\mathrm{d}^{2}\ell^{\prime}}{(2\pi)^{2}}\,B_{\mathrm{ref}}(\bm{\ell},-\bm{\ell}^{\prime},-\bm{\ell}+\bm{\ell}^{\prime})$
(40)
$\displaystyle\quad\quad\quad\quad\quad\quad\times\frac{\tilde{\kappa}_{\mathrm{G}}(\bm{\ell}^{\prime})\tilde{\kappa}_{\mathrm{G}}(\bm{\ell}-\bm{\ell}^{\prime})}{C_{\mathrm{G}}(\ell^{\prime})C_{\mathrm{G}}(|\bm{\ell}-\bm{\ell}^{\prime}|)},$

where $\tilde{\kappa}_{\mathrm{G}}(\bm{\ell})$ represents a Gaussian
convergence in Fourier space with zero mean and its average power spectrum
being $C_{\mathrm{G}}$. The bispectrum of $\kappa_{\mathrm{NG,pert}}$ is equal
to the reference $B_{\mathrm{ref}}$ on average by construction. Once the
squeezed bispectrum template can be obtained by numerical simulations and/or
analytic approaches, one can include the information with Eq. (40) in the
training datasets. This pre-training would be among the simplest approaches to
update our neural network-based model in a physically intuitive way, but
detailed examinations are required.

###  VII.2 Curved Sky

Throughout the paper, we assume a flat sky to generate weak lensing mass maps,
whereas the assumption can break as the survey area increases shortly Wallis
_et al._ (2021); Kansal (2023). The flat-sky approximation is valid for the
standard power-spectrum analysis in most cases Kitching  _et al._ (2017);
Kilbinger  _et al._ (2017); Lemos  _et al._ (2017), but the validity would
depend on sky positions for other non-Gaussian statistics. For instance,
projections of a sphere on a plane can induce some distortions of the surface,
affecting precise estimates of peaks and contours in the lensing convergence.
A promising approach to account for curved-sky effects is to upgrade the
architecture of our neural networks with a graph convolutional neural network
as introduced by Ref. Perraudin  _et al._ (2019). However, it seems difficult
to learn a sphere-to-sphere translation by keeping the angular resolution to
sub-arcminutes, because the number of grids on an input sphere can be of an
order of $O(10^{8})$ in the common HEALPix pixelization Górski  _et al._
(2005). A well-organized training strategy on a patch-by-patch basis is needed
in practice Han  _et al._ (2021).

###  VII.3 Generative models for lensing tomography

Modern weak lensing analyses commonly employ a tomographic setup consisting of
multiple lensing convergence fields with different source redshift
information. Although it is easy to increase the number of channels of inputs
and outputs for our neural networks, we expect that appropriate labels for
multiple-channel training datasets would be important to make our generative
model applicable to realistic tomographic setups. There are several candidates
for the label even when we assume input images to be produced by a
multivariate Gaussian model. The candidates include the field covariance
matrix $\langle\kappa_{\mathrm{G,i}}\kappa_{\mathrm{G,j}}\rangle$ where
$\kappa_{\mathrm{G,i}}$ represents the Gaussian convergence field at the i-th
tomographic bin and the average is computed as the spatial average over a
field-of-view in individual training datasets. We assume that the field
covariance can be a good measure to quantify the HSV effects in the
tomographic lensing analysis as motivated by our single-channel analysis, but
this assumption should be examined with actual simulation data. Note that the
labeling of the field covariance cannot be determined uniquely, indicating
that some trial and error would be crucial.

###  VII.4 Dependence of cosmological and astrophysical parameters

The ultimate goal in the generative model of weak lensing mass maps is to
rapidly-produce a map for various cosmological models while accounting for
different astrophysical effects such as source redshift distributions,
intrinsic alignments Troxel and Ishak (2014), and baryonic effects in the
cosmic mass density Chisari  _et al._ (2019). Our study can provide an
important first step toward such an all-in-one generative model, but it is
still challenging to add realistic cosmological and astrophysical dependence
to a deep-learning-assisted simulator in general. There are some attempts to
develop a cosmology-dependent generative model of weak lensing or two-
dimensional mass density maps in the literature Perraudin  _et al._ (2021);
Tamosiunas  _et al._ (2021); Dai and Seljak (2022). It would be worth noting
that previous cosmology-dependent generators limit themselves to work with a
fixed sky coverage, questioning their utilities in a real-world analysis. Few-
shot image generation (e.g. Ref. Ojha  _et al._ (2021)) is among the most
promising approaches to update our model so that it can be adapted to various
cosmological and astrophysical scenarios. For instance, a baseline generative
model can be trained with a large set of fiducial training data, while few-
shot learning allows one to update the baseline generator with a small set of
training data at different cosmological and astrophysical models. The few-shot
learning already has been examined in a classification problem of galaxies
Zhang  _et al._ (2022), and it might be effective for image generation tasks
in astronomy.

##  VIII Conclusion

In this paper, we have proposed a new generative model of the weak lensing
mass maps (convergence fields) based on an image-to-image translation by
neural networks. The model aims at translating Gaussian convergence fields to
fully non-Gaussian counterparts as simulated by cosmological ray-tracing
simulations Liu  _et al._ (2018). We have trained the neural networks within
the framework of Cycle-Consistent Generative Adversarial Networks (Cycle GAN)
Zhu  _et al._ (2017), enabling the networks to learn the style transfer
between two different convergence models in an unsupervised manner. We have
found that annotating the training datasets is key to realizing the desired
level of diversity in the convergence fields generated by our model. Labeling
individual training images with field variances is effective in learning non-
Gaussian covariance structures in the convergence power spectrum for our
neural network-based model.

Compared with a popular log-normal convergence model, our model can provide
more accurate predictions of convergence bispectra as well as local structures
around rare high-density regions. We have validated our model by studying
multiple sets of convergence summary statistics, including the bispectra,
number density of convergence peaks, one-point probability distribution
function, Minkowski functionals, and scattering transform coefficients. The
model predictions of those summary statistics are statistically consistent
with the counterparts in the true lensing simulations. Likelihood functions of
the model summary statistics can give a reasonable fit to the ground truth,
highlighting that our generative model can play a central role in validating
pipelines for cosmological inference.

We have also examined to produce a non-Gaussian convergence field with an
arbitrary sky coverage by our neural network-based model. For this purpose, we
have laid out a four-step procedure; (i) Generate a Gaussian convergence map,
(ii) Divide the Gaussian map into small patches, (iii) perform an image-to-
image translation on a patch-by-patch basis, and (iv) combine the resultant
non-Gaussian fields in all the patches. We have demonstrated that the four-
step procedure works in producing a non-Gaussian convergence field with a sky
coverage of $\sim 166\,\mathrm{deg}^{2}$ in a GPU-minute, whereas our neural
networks have been trained with the simulations covering a sky of $\sim
12\,\mathrm{deg}^{2}$. We have confirmed that the model summary statistics are
still consistent with the ground truth even when extending sky coverage. This
method opens a valuable opportunity for massive productions of weak lensing
mass maps with realistic non-Gaussianities but covering large areas of the
sky. Note that our trained generators only require low-cost Gaussian
convergence fields to produce a new set of realistic non-Gaussian
counterparts, allowing us to increase the number of independent non-Gaussian
convergence fields in a rapid manner.

Our generative model accelerates the production of mock weak lensing mass maps
in a simple and efficient way, but it has to be revised in various aspects for
real-world applications. The model tends to overestimate a squeezed-shaped
bispectrum with a level of $30\%$, indicating that the input convergence field
for our model should preserve weakly non-Gaussian structures. We constructed
our generative model assuming single source redshift in a flat sky, but this
is not valid for future lensing surveys with a sky coverage larger than
$1000\,\mathrm{deg}^{2}$ and tomographic setups. Our model is not able to
produce a weak lensing mass map as a function of various cosmological and
astrophysical parameters, which is a common weak point in deep-learning-
assisted approaches. We expect that the few-shot learning method can be an
effective breakthrough in cosmology- and astrophysics-dependent generators, as
a set of various simulation data becomes available. This is along the lines of
our future study.

###### Acknowledgements.

We thank Naoki Yoshida and Kana Moriwaki for the useful comments. This work is
supported by MEXT KAKENHI Grant Numbers 19K14767 and 20H05861. A part of
numerical computations was carried out on Cray XC50 at the Center for
Computational Astrophysics in NAOJ.

## References

  * Bartelmann and Schneider (2001) M. Bartelmann and P. Schneider, Phys.Rept. 340, 291 (2001), arXiv:astro-ph/9912508 [astro-ph] .
  * Kilbinger (2015) M. Kilbinger, Rept. Prog. Phys. 78, 086901 (2015), arXiv:1411.0115 [astro-ph.CO] .
  * Mandelbaum (2018) R. Mandelbaum, Ann. Rev. Astron. Astrophys. 56, 393 (2018), arXiv:1710.03235 [astro-ph.CO] .
  * Jain  _et al._ (2000) B. Jain, U. Seljak, and S. D. M. White, Astrophys. J. 530, 547 (2000), arXiv:astro-ph/9901191 .
  * White and Hu (2000) M. J. White and W. Hu, Astrophys. J. 537, 1 (2000), arXiv:astro-ph/9909165 .
  * Ajani  _et al._ (2023) V. Ajani _et al._ (Euclid), Astron. Astrophys. 675, A120 (2023), arXiv:2301.12890 [astro-ph.CO] .
  * Dodelson and Schneider (2013) S. Dodelson and M. D. Schneider, Phys. Rev. D 88, 063537 (2013), arXiv:1304.2593 [astro-ph.CO] .
  * Shirasaki  _et al._ (2019a) M. Shirasaki, T. Hamana, M. Takada, R. Takahashi, and H. Miyatake, Mon. Not. Roy. Astron. Soc. 486, 52 (2019a), arXiv:1901.09488 [astro-ph.CO] .
  * Abdalla  _et al._ (2022) E. Abdalla _et al._ , JHEAp 34, 49 (2022), arXiv:2203.06142 [astro-ph.CO] .
  * Asgari  _et al._ (2021) M. Asgari _et al._ (KiDS), Astron. Astrophys. 645, A104 (2021), arXiv:2007.15633 [astro-ph.CO] .
  * Secco  _et al._ (2022) L. F. Secco _et al._ (DES), Phys. Rev. D 105, 023515 (2022), arXiv:2105.13544 [astro-ph.CO] .
  * Dalal  _et al._ (2023) R. Dalal _et al._ , (2023), arXiv:2304.00701 [astro-ph.CO] .
  * Li  _et al._ (2023) X. Li _et al._ , (2023), arXiv:2304.00702 [astro-ph.CO] .
  * Nishimichi  _et al._ (2020) T. Nishimichi, G. D’Amico, M. M. Ivanov, L. Senatore, M. Simonović, M. Takada, M. Zaldarriaga, and P. Zhang, Phys. Rev. D 102, 123541 (2020), arXiv:2003.08277 [astro-ph.CO] .
  * Porqueres  _et al._ (2021) N. Porqueres, A. Heavens, D. Mortlock, and G. Lavaux, Mon. Not. Roy. Astron. Soc. 509, 3194 (2021), arXiv:2108.04825 [astro-ph.CO] .
  * Boruah  _et al._ (2022) S. S. Boruah, E. Rozo, and P. Fiedorowicz, Mon. Not. Roy. Astron. Soc. 516, 4111 (2022), arXiv:2204.13216 [astro-ph.CO] .
  * Dai and Seljak (2022) B. Dai and U. Seljak, Mon. Not. Roy. Astron. Soc. 516, 2363 (2022), arXiv:2202.05282 [astro-ph.CO] .
  * Shirasaki  _et al._ (2019b) M. Shirasaki, N. Yoshida, and S. Ikeda, Phys. Rev. D 100, 043527 (2019b), arXiv:1812.05781 [astro-ph.CO] .
  * Jeffrey  _et al._ (2020) N. Jeffrey, F. Lanusse, O. Lahav, and J.-L. Starck, Mon. Not. Roy. Astron. Soc. 492, 5023 (2020), arXiv:1908.00543 [astro-ph.CO] .
  * Fiedorowicz  _et al._ (2022) P. Fiedorowicz, E. Rozo, S. S. Boruah, C. Chang, and M. Gatti, Mon. Not. Roy. Astron. Soc. 512, 73 (2022), arXiv:2105.14699 [astro-ph.CO] .
  * Remy  _et al._ (2023) B. Remy, F. Lanusse, N. Jeffrey, J. Liu, J.-L. Starck, K. Osato, and T. Schrabback, Astron. Astrophys. 672, A51 (2023), arXiv:2201.05561 [astro-ph.CO] .
  * Sato  _et al._ (2009) M. Sato, T. Hamana, R. Takahashi, M. Takada, N. Yoshida, T. Matsubara, and N. Sugiyama, Astrophys. J. 701, 945 (2009), arXiv:0906.2237 [astro-ph.CO] .
  * Dietrich and Hartlap (2010) J. P. Dietrich and J. Hartlap, Mon. Not. Roy. Astron. Soc. 402, 1049 (2010), arXiv:0906.3512 [astro-ph.CO] .
  * Harnois-Déraps  _et al._ (2012) J. Harnois-Déraps, S. Vafaei, and L. Van Waerbeke, Mon. Not. Roy. Astron. Soc. 426, 1262 (2012), arXiv:1202.2332 [astro-ph.CO] .
  * Liu  _et al._ (2018) J. Liu, S. Bird, J. M. Z. Matilla, J. C. Hill, Z. Haiman, M. S. Madhavacheril, A. Petri, and D. N. Spergel, JCAP 03, 049 (2018), arXiv:1711.10524 [astro-ph.CO] .
  * Takahashi  _et al._ (2017) R. Takahashi, T. Hamana, M. Shirasaki, T. Namikawa, T. Nishimichi, K. Osato, and K. Shiroyama, Astrophys. J. 850, 24 (2017), arXiv:1706.01472 [astro-ph.CO] .
  * Harnois-Déraps  _et al._ (2018) J. Harnois-Déraps _et al._ , Mon. Not. Roy. Astron. Soc. 481, 1337 (2018), arXiv:1805.04511 [astro-ph.CO] .
  * Harnois-Deraps  _et al._ (2019) J. Harnois-Deraps, B. Giblin, and B. Joachimi, Astron. Astrophys. 631, A160 (2019), arXiv:1905.06454 [astro-ph.CO] .
  * Shirasaki  _et al._ (2021) M. Shirasaki, K. Moriwaki, T. Oogi, N. Yoshida, S. Ikeda, and T. Nishimichi, Mon. Not. Roy. Astron. Soc. 504, 1825 (2021), arXiv:1911.12890 [astro-ph.CO] .
  * Osato  _et al._ (2021) K. Osato, J. Liu, and Z. Haiman, Mon. Not. Roy. Astron. Soc. 502, 5593 (2021), arXiv:2010.09731 [astro-ph.CO] .
  * Kacprzak  _et al._ (2023) T. Kacprzak, J. Fluri, A. Schneider, A. Refregier, and J. Stadel, JCAP 02, 050 (2023), arXiv:2209.04662 [astro-ph.CO] .
  * Hadzhiyska  _et al._ (2023) B. Hadzhiyska _et al._ , (2023), arXiv:2305.11935 [astro-ph.CO] .
  * Ferlito  _et al._ (2023) F. Ferlito _et al._ , Mon. Not. Roy. Astron. Soc. 524, 5591 (2023), arXiv:2304.12338 [astro-ph.CO] .
  * Taruya  _et al._ (2002) A. Taruya, M. Takada, T. Hamana, I. Kayo, and T. Futamase, Astrophys. J. 571, 638 (2002), arXiv:astro-ph/0202090 .
  * Xavier  _et al._ (2016) H. S. Xavier, F. B. Abdalla, and B. Joachimi, Mon. Not. Roy. Astron. Soc. 459, 3693 (2016), arXiv:1602.08503 [astro-ph.CO] .
  * Makiya  _et al._ (2021) R. Makiya, I. Kayo, and E. Komatsu, JCAP 03, 095 (2021), arXiv:2008.13195 [astro-ph.CO] .
  * Yu  _et al._ (2016) Y. Yu, P. Zhang, and Y. Jing, Phys. Rev. D 94, 083520 (2016), arXiv:1607.05007 [astro-ph.CO] .
  * Tessore  _et al._ (2023) N. Tessore, A. Loureiro, B. Joachimi, M. von Wietersheim-Kramsta, and N. Jeffrey,  (2023), 10.21105/astro.2302.01942, arXiv:2302.01942 [astro-ph.CO] .
  * Izard  _et al._ (2018) A. Izard, P. Fosalba, and M. Crocce, Mon. Not. Roy. Astron. Soc. 473, 3051 (2018), arXiv:1707.06312 [astro-ph.CO] .
  * Sgier  _et al._ (2019) R. Sgier, A. Réfrégier, A. Amara, and A. Nicola, JCAP 01, 044 (2019), arXiv:1801.05745 [astro-ph.CO] .
  * Böhm  _et al._ (2021) V. Böhm, Y. Feng, M. E. Lee, and B. Dai, Astron. Comput. 36, 100490 (2021), arXiv:2012.07266 [astro-ph.CO] .
  * Giocoli  _et al._ (2017) C. Giocoli, S. Di Meo, M. Meneghetti, E. Jullo, S. de la Torre, L. Moscardini, M. Baldi, P. Mazzotta, and R. Benton Metcalf, Mon. Not. Roy. Astron. Soc. 470, 3574 (2017), arXiv:1701.02739 [astro-ph.CO] .
  * Giocoli  _et al._ (2020) C. Giocoli, P. Monaco, L. Moscardini, T. Castro, M. Meneghetti, R. B. Metcalf, and M. Baldi, Mon. Not. Roy. Astron. Soc. 496, 1307 (2020), arXiv:2001.11512 [astro-ph.CO] .
  * Mustafa  _et al._ (2019) M. Mustafa, D. Bard, W. Bhimji, Z. Lukić, R. Al-Rfou, and J. M. Kratochvil, Computational Astrophysics and Cosmology 6, 1 (2019), arXiv:1706.02390 [astro-ph.IM] .
  * Perraudin  _et al._ (2021) N. Perraudin, S. Marcon, A. Lucchi, and T. Kacprzak, Front. Artif. Intell. 4, 673062 (2021), arXiv:2004.08139 [astro-ph.CO] .
  * Tamosiunas  _et al._ (2021) A. Tamosiunas, H. A. Winther, K. Koyama, D. J. Bacon, R. C. Nichol, and B. Mawdsley, Mon. Not. Roy. Astron. Soc. 506, 3049 (2021), arXiv:2004.10223 [astro-ph.CO] .
  * Zhu  _et al._ (2017) J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, in  _Computer Vision (ICCV), 2017 IEEE International Conference on_ (2017).
  * Han  _et al._ (2021) D. Han, N. Sehgal, and F. Villaescusa-Navarro, Phys. Rev. D 104, 123521 (2021), arXiv:2105.11444 [astro-ph.CO] .
  * Piras  _et al._ (2022) D. Piras, B. Joachimi, and F. Villaescusa-Navarro,  (2022), 10.1093/mnras/stad052, arXiv:2205.07898 [astro-ph.CO] .
  * Takahashi  _et al._ (2011) R. Takahashi, M. Oguri, M. Sato, and T. Hamana, Astrophys. J. 742, 15 (2011), arXiv:1106.3823 [astro-ph.CO] .
  * Castro  _et al._ (2018) T. Castro, M. Quartin, C. Giocoli, S. Borgani, and K. Dolag, Mon. Not. Roy. Astron. Soc. 478, 1305 (2018), arXiv:1711.10017 [astro-ph.CO] .
  * Hilbert  _et al._ (2011) S. Hilbert, J. Hartlap, and P. Schneider, Astron. Astrophys. 536, A85 (2011), arXiv:1105.3980 [astro-ph.CO] .
  * Colavincenzo  _et al._ (2019) M. Colavincenzo _et al._ , Mon. Not. Roy. Astron. Soc. 482, 4883 (2019), arXiv:1806.09499 [astro-ph.CO] .
  * Goodfellow  _et al._ (2014) I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, in  _Advances in Neural Information Processing Systems_, Vol. 27, edited by Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Weinberger (Curran Associates, Inc., 2014).
  * Huang  _et al._ (2018) H. Huang, P. S. Yu, and C. Wang, arXiv e-prints , arXiv:1803.04469 (2018), arXiv:1803.04469 [cs.CV] .
  * Springel (2005) V. Springel, Mon. Not. Roy. Astron. Soc. 364, 1105 (2005), arXiv:astro-ph/0505010 .
  * Zeldovich (1970) Y. B. Zeldovich, Astron. Astrophys. 5, 84 (1970).
  * Petri (2016) A. Petri, Astron. Comput. 17, 73 (2016), arXiv:1606.01903 [astro-ph.CO] .
  * Limber (1954) D. N. Limber, Astrophys. J. 119, 655 (1954).
  * Takahashi  _et al._ (2012) R. Takahashi, M. Sato, T. Nishimichi, A. Taruya, and M. Oguri, Astrophys. J. 761, 152 (2012), arXiv:1208.2701 [astro-ph.CO] .
  * Isola  _et al._ (2017) P. Isola, J. Zhu, T. Zhou, and A. A. Efros, in  _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ (IEEE Computer Society, Los Alamitos, CA, USA, 2017) pp. 5967–5976.
  * Mao  _et al._ (2017) X. Mao, Q. Li, H. Xie, R. Y. Lau, Z. Wang, and S. P. Smolley, in  _2017 IEEE International Conference on Computer Vision (ICCV)_ (2017) pp. 2813–2821.
  * Shrivastava  _et al._ (2017) A. Shrivastava, T. Pfister, O. Tuzel, J. Susskind, W. Wang, and R. Webb, in  _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ (2017) pp. 2242–2251.
  * Kingma and Ba (2015) D. Kingma and J. Ba, in  _International Conference on Learning Representations (ICLR)_ (San Diega, CA, USA, 2015).
  * Takada and Hu (2013) M. Takada and W. Hu, Phys. Rev. D 87, 123504 (2013), arXiv:1302.6994 [astro-ph.CO] .
  * Cooray and Sheth (2002) A. Cooray and R. K. Sheth, Phys. Rept. 372, 1 (2002), arXiv:astro-ph/0206508 .
  * Yu  _et al._ (2011) Y. Yu, P. Zhang, W. Lin, W. Cui, and J. N. Fry, Phys. Rev. D 84, 023523 (2011), arXiv:1103.2858 [astro-ph.CO] .
  * Seo  _et al._ (2011) H.-J. Seo, M. Sato, S. Dodelson, B. Jain, and M. Takada, ApJ 729, L11 (2011), arXiv:1008.0349 [astro-ph.CO] .
  * Seo  _et al._ (2012) H.-J. Seo, M. Sato, M. Takada, and S. Dodelson, Astrophys. J. 748, 57 (2012), arXiv:1109.5639 [astro-ph.CO] .
  * Simpson  _et al._ (2016) F. Simpson, J. Harnois-Déraps, C. Heymans, R. Jimenez, L. Verde, and B. Joachimi, Mon. Not. Roy. Astron. Soc. 456, 278 (2016), arXiv:1507.04862 [astro-ph.CO] .
  * Hui (1999) L. Hui, Astrophys. J. Lett. 519, L9 (1999), arXiv:astro-ph/9902275 .
  * Cooray and Hu (2001) A. Cooray and W. Hu, Astrophys. J. 548, 7 (2001), arXiv:astro-ph/0004151 .
  * Benabed and Bernardeau (2001) K. Benabed and F. Bernardeau, Phys. Rev. D 64, 083501 (2001), arXiv:astro-ph/0104371 .
  * Takada and Jain (2004) M. Takada and B. Jain, Mon. Not. Roy. Astron. Soc. 348, 897 (2004), arXiv:astro-ph/0310125 .
  * Refregier  _et al._ (2004) A. Refregier, R. Massey, J. Rhodes, R. Ellis, J. Albert, D. Bacon, G. Bernstein, T. McKay, and S. Perlmutter, Astron. J. 127, 3102 (2004), arXiv:astro-ph/0304419 .
  * Dodelson and Zhang (2005) S. Dodelson and P. Zhang, Phys. Rev. D 72, 083001 (2005), arXiv:astro-ph/0501063 .
  * Coulton  _et al._ (2019) W. R. Coulton, J. Liu, M. S. Madhavacheril, V. Böhm, and D. N. Spergel, JCAP 05, 043 (2019), arXiv:1810.02374 [astro-ph.CO] .
  * Kayo  _et al._ (2013) I. Kayo, M. Takada, and B. Jain, Mon. Not. Roy. Astron. Soc. 429, 344 (2013), arXiv:1207.6322 [astro-ph.CO] .
  * Sato and Nishimichi (2013) M. Sato and T. Nishimichi, Phys. Rev. D 87, 123538 (2013), arXiv:1301.3588 [astro-ph.CO] .
  * Liu  _et al._ (2015) J. Liu, A. Petri, Z. Haiman, L. Hui, J. M. Kratochvil, and M. May, Phys. Rev. D 91, 063507 (2015), arXiv:1412.0757 [astro-ph.CO] .
  * Hamana  _et al._ (2015) T. Hamana, J. Sakurai, M. Koike, and L. Miller, Publ. Astron. Soc. Jap. 67, 34 (2015), arXiv:1503.01851 [astro-ph.CO] .
  * Kacprzak  _et al._ (2016) T. Kacprzak _et al._ (DES), Mon. Not. Roy. Astron. Soc. 463, 3653 (2016), arXiv:1603.05040 [astro-ph.CO] .
  * Shan  _et al._ (2018) H. Shan _et al._ , Mon. Not. Roy. Astron. Soc. 474, 1116 (2018), arXiv:1709.07651 [astro-ph.CO] .
  * Martinet  _et al._ (2018) N. Martinet _et al._ , Mon. Not. Roy. Astron. Soc. 474, 712 (2018), arXiv:1709.07678 [astro-ph.CO] .
  * Harnois-Déraps  _et al._ (2021) J. Harnois-Déraps, N. Martinet, T. Castro, K. Dolag, B. Giblin, C. Heymans, H. Hildebrandt, and Q. Xia, Mon. Not. Roy. Astron. Soc. 506, 1623 (2021), arXiv:2012.02777 [astro-ph.CO] .
  * Zürcher  _et al._ (2022) D. Zürcher _et al._ (DES), Mon. Not. Roy. Astron. Soc. 511, 2075 (2022), arXiv:2110.10135 [astro-ph.CO] .
  * Liu  _et al._ (2022) X. Liu, S. Yuan, C. Pan, T. Zhang, Q. Wang, and Z. Fan,  (2022), 10.1093/mnras/stac2971, arXiv:2210.07853 [astro-ph.CO] .
  * Hamana  _et al._ (2004) T. Hamana, M. Takada, and N. Yoshida, Mon. Not. Roy. Astron. Soc. 350, 893 (2004), arXiv:astro-ph/0310607 .
  * Hennawi and Spergel (2005) J. F. Hennawi and D. N. Spergel, Astrophys. J. 624, 59 (2005), arXiv:astro-ph/0404349 .
  * Maturi  _et al._ (2005) M. Maturi, M. Meneghetti, M. Bartelmann, K. Dolag, and L. Moscardini, Astron. Astrophys. 442, 851 (2005), arXiv:astro-ph/0412604 .
  * Marian  _et al._ (2012) L. Marian, R. E. Smith, S. Hilbert, and P. Schneider, Mon. Not. Roy. Astron. Soc. 423, 1711 (2012), arXiv:1110.4635 [astro-ph.CO] .
  * Hamana  _et al._ (2012) T. Hamana, M. Oguri, M. Shirasaki, and M. Sato, Mon. Not. Roy. Astron. Soc. 425, 2287 (2012), arXiv:1204.6117 [astro-ph.CO] .
  * Yang  _et al._ (2011) X. Yang, J. M. Kratochvil, S. Wang, E. A. Lim, Z. Haiman, and M. May, Phys. Rev. D 84, 043529 (2011), arXiv:1109.6333 [astro-ph.CO] .
  * Liu and Haiman (2016) J. Liu and Z. Haiman, Phys. Rev. D 94, 043533 (2016), arXiv:1606.01318 [astro-ph.CO] .
  * Sabyr  _et al._ (2022) A. Sabyr, Z. Haiman, J. M. Z. Matilla, and T. Lu, Phys. Rev. D 105, 023505 (2022), arXiv:2109.00547 [astro-ph.CO] .
  * Bernardeau and Valageas (2000) F. Bernardeau and P. Valageas, Astron. Astrophys. 364, 1 (2000), arXiv:astro-ph/0006270 .
  * Liu and Madhavacheril (2019) J. Liu and M. S. Madhavacheril, Phys. Rev. D 99, 083508 (2019), arXiv:1809.10747 [astro-ph.CO] .
  * Barthelemy  _et al._ (2021) A. Barthelemy, S. Codis, and F. Bernardeau, Mon. Not. Roy. Astron. Soc. 503, 5204 (2021), arXiv:2012.03831 [astro-ph.CO] .
  * Thiele  _et al._ (2020) L. Thiele, J. C. Hill, and K. M. Smith, Phys. Rev. D 102, 123545 (2020), arXiv:2009.06547 [astro-ph.CO] .
  * Boyle  _et al._ (2021) A. Boyle, C. Uhlemann, O. Friedrich, A. Barthelemy, S. Codis, F. Bernardeau, C. Giocoli, and M. Baldi, Mon. Not. Roy. Astron. Soc. 505, 2886 (2021), arXiv:2012.07771 [astro-ph.CO] .
  * Giblin  _et al._ (2023) B. Giblin, Y.-C. Cai, and J. Harnois-Déraps, Mon. Not. Roy. Astron. Soc. 520, 1721 (2023), arXiv:2211.05708 [astro-ph.CO] .
  * Friedrich  _et al._ (2018) O. Friedrich _et al._ (DES), Phys. Rev. D 98, 023508 (2018), arXiv:1710.05162 [astro-ph.CO] .
  * Gruen  _et al._ (2018) D. Gruen _et al._ (DES), Phys. Rev. D 98, 023507 (2018), arXiv:1710.05045 [astro-ph.CO] .
  * Matsubara (2003a) T. Matsubara, Astrophys. J. 584, 1 (2003a).
  * Munshi  _et al._ (2012) D. Munshi, L. van Waerbeke, J. Smidt, and P. Coles, Mon. Not. Roy. Astron. Soc. 419, 536 (2012), arXiv:1103.1876 [astro-ph.CO] .
  * Matsubara and Kuriki (2021) T. Matsubara and S. Kuriki, Phys. Rev. D 104, 103522 (2021), arXiv:2011.04954 [astro-ph.CO] .
  * Petri  _et al._ (2013) A. Petri, Z. Haiman, L. Hui, M. May, and J. M. Kratochvil, Phys. Rev. D 88, 123002 (2013), arXiv:1309.4460 [astro-ph.CO] .
  * Kratochvil  _et al._ (2012) J. M. Kratochvil, E. A. Lim, S. Wang, Z. Haiman, M. May, and K. Huffenberger, Phys. Rev. D 85, 103513 (2012), arXiv:1109.6334 [astro-ph.CO] .
  * Shirasaki and Yoshida (2014) M. Shirasaki and N. Yoshida, Astrophys. J. 786, 43 (2014), arXiv:1312.5032 [astro-ph.CO] .
  * Lim and Simon (2012) E. A. Lim and D. Simon, JCAP 01, 048 (2012), arXiv:1103.4300 [astro-ph.CO] .
  * Mallat (2012) S. Mallat, Communications on Pure and Applied Mathematics 65, 1331 (2012), https://onlinelibrary.wiley.com/doi/pdf/10.1002/cpa.21413 .
  * Cheng  _et al._ (2020) S. Cheng, Y.-S. Ting, B. Ménard, and J. Bruna, Mon. Not. Roy. Astron. Soc. 499, 5902 (2020), arXiv:2006.08561 [astro-ph.CO] .
  * Ribli  _et al._ (2019) D. Ribli, B. A. Pataki, J. M. Zorrilla Matilla, D. Hsu, Z. Haiman, and I. Csabai, Mon. Not. Roy. Astron. Soc. 490, 1843 (2019), arXiv:1902.03663 [astro-ph.CO] .
  * Cheng and Ménard (2021) S. Cheng and B. Ménard, (2021), arXiv:2112.01288 [astro-ph.IM] .
  * Das and Ostriker (2006) S. Das and J. P. Ostriker, Astrophys. J. 645, 1 (2006), arXiv:astro-ph/0512644 .
  * Wang  _et al._ (2009) S. Wang, Z. Haiman, and M. May, Astrophys. J. 691, 547 (2009), arXiv:0809.4052 [astro-ph] .
  * Shirasaki (2017) M. Shirasaki, Mon. Not. Roy. Astron. Soc. 465, 1974 (2017), arXiv:1610.00840 [astro-ph.CO] .
  * Takahashi  _et al._ (2020) R. Takahashi, T. Nishimichi, T. Namikawa, A. Taruya, I. Kayo, K. Osato, Y. Kobayashi, and M. Shirasaki, Astrophys. J. 895, 113 (2020), arXiv:1911.07886 [astro-ph.CO] .
  * Matsubara (2003b) T. Matsubara, Astrophys. J. Lett. 591, L79 (2003b), arXiv:astro-ph/0303278 .
  * Shimodaira (2000) H. Shimodaira, Journal of Statistical Planning and Inference 90, 227 (2000).
  * Smith and Zaldarriaga (2011) K. M. Smith and M. Zaldarriaga, Mon. Not. Roy. Astron. Soc. 417, 2 (2011), arXiv:astro-ph/0612571 .
  * Hall and Mead (2018) A. Hall and A. Mead, Mon. Not. Roy. Astron. Soc. 473, 3190 (2018), arXiv:1709.03924 [astro-ph.CO] .
  * Wallis  _et al._ (2021) C. G. R. Wallis, M. A. Price, J. D. McEwen, T. D. Kitching, B. Leistedt, and A. Plouviez, Mon. Not. Roy. Astron. Soc. 509, 4480 (2021), arXiv:1703.09233 [astro-ph.CO] .
  * Kansal (2023) V. Kansal, Astron. Astrophys. 670, A34 (2023), arXiv:2301.10482 [astro-ph.CO] .
  * Kitching  _et al._ (2017) T. D. Kitching, J. Alsing, A. F. Heavens, R. Jimenez, J. D. McEwen, and L. Verde, Mon. Not. Roy. Astron. Soc. 469, 2737 (2017), arXiv:1611.04954 [astro-ph.CO] .
  * Kilbinger  _et al._ (2017) M. Kilbinger _et al._ , Mon. Not. Roy. Astron. Soc. 472, 2126 (2017), arXiv:1702.05301 [astro-ph.CO] .
  * Lemos  _et al._ (2017) P. Lemos, A. Challinor, and G. Efstathiou, JCAP 05, 014 (2017), arXiv:1704.01054 [astro-ph.CO] .
  * Perraudin  _et al._ (2019) N. Perraudin, M. Defferrard, T. Kacprzak, and R. Sgier, Astron. Comput. 27, 130 (2019), arXiv:1810.12186 [astro-ph.CO] .
  * Górski  _et al._ (2005) K. M. Górski, E. Hivon, A. J. Banday, B. D. Wandelt, F. K. Hansen, M. Reinecke, and M. Bartelman, Astrophys. J. 622, 759 (2005), arXiv:astro-ph/0409513 .
  * Troxel and Ishak (2014) M. A. Troxel and M. Ishak, Phys. Rept. 558, 1 (2014), arXiv:1407.6990 [astro-ph.CO] .
  * Chisari  _et al._ (2019) N. E. Chisari _et al._ , Open J. Astrophys. 2, 4 (2019), arXiv:1905.06082 [astro-ph.CO] .
  * Ojha  _et al._ (2021) U. Ojha, Y. Li, J. Lu, A. A. Efros, Y. J. Lee, E. Shechtman, and R. Zhang, in  _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ (IEEE Computer Society, Los Alamitos, CA, USA, 2021) pp. 10738–10747.
  * Zhang  _et al._ (2022) Z. Zhang, Z. Zou, N. Li, and Y. Chen, Research in Astronomy and Astrophysics 22, 055002 (2022), arXiv:2202.08172 [astro-ph.GA] .
  * Johnson  _et al._ (2016) J. Johnson, A. Alahi, and L. Fei-Fei, in  _Computer Vision – ECCV 2016_ , edited by B. Leibe, J. Matas, N. Sebe, and M. Welling (Springer International Publishing, Cham, 2016) pp. 694–711.
  * He  _et al._ (2016) K. He, X. Zhang, S. Ren, and J. Sun, in  _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ (IEEE Computer Society, Los Alamitos, CA, USA, 2016) pp. 770–778.
  * Ulyanov  _et al._ (2017) D. Ulyanov, A. Vedaldi, and V. Lempitsky, in  _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ (IEEE Computer Society, Los Alamitos, CA, USA, 2017) pp. 4105–4113.
  * Ledig  _et al._ (2017) C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi, in  _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ (IEEE Computer Society, Los Alamitos, CA, USA, 2017) pp. 105–114.
  * Li and Wand (2016) C. Li and M. Wand, in  _Computer Vision – ECCV 2016_ , edited by B. Leibe, J. Matas, N. Sebe, and M. Welling (Springer International Publishing, Cham, 2016) pp. 702–716.

##  Appendix A Network Architecture in Cycle GAN

In this appendix, we provide some details of the architecture for our
generative networks. In our Cycle GAN, we adopt the same architecture for a
pair of GANs. For the generator, we follow the architecture designed by Ref.
Johnson  _et al._ (2016). This network contains three convolutions, nine
residual blocks He  _et al._ (2016), two transposed convolutions, and one
convolution that generates $256\times 256$ images. As in Ref. Johnson  _et
al._ (2016), we insert an instance normalization module Ulyanov  _et al._
(2017), improving the quality of image stylization. For the discriminator, we
adopt $70\times 70$ PatchGANs Isola  _et al._ (2017); Ledig  _et al._ (2017);
Li and Wand (2016), aiming at authenticity determination across 70 × 70
overlapping image patches. Note that a patch-level discriminator architecture
allows us to reduce the number of parameters compared to a full-image
discriminator Isola  _et al._ (2017). Also, it can be implemented on
arbitrarily sized images in a fully convolutional fashion, making the training
efficient with GPU environments. Details of the architecture are provided in
Tables 2 and 3.

Table 2: Generator network architecture: layer types, activations, output shapes (channels $\times$ height $\times$ width) and the number of trainable parameters for each layer. Conv($k$, $s$) are $k\times k$ Convolution layers with strides=$s$, while TConv($k$, $s$) stands for the transposed counterpart. ResNetBlock denotes a residual block that contains two $3\times 3$ convolutional layers with the same number of filters on both layers. ReflectionPad represents a reflection padding used for the reduction of artifacts. InstanceNorm is the instance normalization.  | Activ. | Output shape | Params.
---|---|---|---
Input | – | [1, 256, 256] | –
ReflectionPad | – | [1, 262, 262] | –
Conv(7,1) | – | [64, 256, 256] | 3,200
InstanceNorm | ReLU | [64, 256, 256] | –
Conv(3,2) | – | [128, 128, 128] | 73,856
InstanceNorm | ReLU | [128, 128, 128] | –
Conv(3,2) | – | [256, 64, 64] | 295,168
InstanceNorm | ReLU | [256, 64, 64] | –
ResNetBlock | – | [256, 64, 64] | 1,180,160
ResNetBlock | – | [256, 64, 64] | 1,180,160
ResNetBlock | – | [256, 64, 64] | 1,180,160
ResNetBlock | – | [256, 64, 64] | 1,180,160
ResNetBlock | – | [256, 64, 64] | 1,180,160
ResNetBlock | – | [256, 64, 64] | 1,180,160
ResNetBlock | – | [256, 64, 64] | 1,180,160
ResNetBlock | – | [256, 64, 64] | 1,180,160
TConv(3,2) | – | [128, 128, 128] | 295,040
InstanceNorm | ReLU | [128, 128, 128] | –
TConv(3,2) | – | [64, 256, 256] | 73,792
InstanceNorm | ReLU | [64, 256, 256] | –
ReflectionPad | – | [1, 262, 262] | –
Conv(7,1) | Tanh | [1, 256, 256] | 3,137
|  | Total params | 11,365,633
Table 3: Similar to Table 2, this table shows the discriminator network architecture. In the discriminator, we set every LeakyReLU’s leakine to be 0.2.  | Activ. | Output shape | Params.
---|---|---|---
Input | – | [1, 256, 256] | –
Conv(4,2) | LReLU | [64, 128, 128] | 1,088
Conv(4,2) | – | [128, 64, 64] | 131,200
InstanceNorm | LReLU | [128, 64, 64] | –
Conv(4,2) | – | [256, 32, 32] | 524,544
InstanceNorm | LReLU | [256, 32, 32] | –
Conv(4,1) | – | [512, 31, 31] | 2,097,664
InstanceNorm | LReLU | [512, 31, 31] | –
Conv(4,1) | – | [1, 30, 30] | 8,193
|  | Total params | 2,762,689

##  Appendix B Translation from non-Gaussian convergence to the Gaussian
counterpart

Figure 10:  Comparisons of lensing summary statistics predicted by our neural-
network-based “Gaussianizer” and target Gaussian counterparts. The left,
middle, and right panels summarize the comparisons of power spectra,
equilateral-shaped bispectra, and one-point PDFs, respectively. In each panel,
the blue points show the average statistics over 10,000 realizations of the
target Gaussian generative model, whereas the black solid lines stand for the
prediction by our Gaussianizer. The blue error bars in each panel correspond
to the statistical uncertainty for a survey of $3.5\,\mathrm{deg}^{2}$. For
ease of comparison, we also include the average summary statistics over 10,000
non-Gaussian convergence fields in the MassiveNuS simulation Liu  _et al._
(2018) as shown by the orange dashed lines in each panel. This figure
clarifies that our neural network can be used for a precise Gaussianization of
non-Gaussian convergence fields.

We here summarize the performance of the generator translating from non-
Gaussian convergence maps into the Gaussian counterparts (i.e.
Gaussianization), that is an interesting by-product of our Cycle GAN. In this
appendix, we show the results of the neural network trained without any
labeling.

Figure 11:  Cumulative signal-to-noise ratios (S/Ns) of lensing power spectra
for three different lensing maps. The cumulative S/N is defined in Eq. (32).
The blue points represent the results of the target Gaussian maps, while the
black solid and orange dashed lines stand for the Gaussianized and input non-
Gaussian maps, respectively. A perfect Gaussianization ensures that the blue
points and the solid line should be consistent with each other.

Figure 10 compares three different summary statistics between the target
Gaussian and our generative model. We chose power spectra, equilateral-shaped
bispectra, and one-point PDFs for the comparison because these are primarily
important to check the degree of Gaussianity left in a given lensing map. The
figure shows that our neural-network-based Gaussianization works well as long
as the network was sufficiently trained. It is worth noting that our
Gaussianization vanishes the non-zero three-point correlations found in an
input non-Gaussian lensing map. This is an improvement compared to the
previous Gaussianization methods Yu  _et al._ (2011); Seo  _et al._ (2011,
2012); Simpson  _et al._ (2016), because the previous approaches rely on a
local transformation of non-Gaussian weak lensing mass maps in real space,
making it difficult to erase the three-point correlations.

In Figure 11, we study the degree of Gaussianity in power spectra produced by
our generator in detail. We compute the cumulative signal-to-noise ratio (S/N)
as in Eq. (32) for the target Gaussian, the input non-Gaussian, and the
Gaussianized maps. If our generator can perform a perfect Gaussianization, the
cumulative S/N should match the target Gaussian counterpart. The figure
highlights that our neural network-based Gaussianization cannot be perfect. We
observe that the cumulative S/N of the Gaussianized power spectrum deviates
from the true Gaussian counterpart at $\ell\lower
2.15277pt\hbox{$\;\buildrel>\over{\sim}\;$}3000$. This means that the
Gaussianization by our generator remains at some level of non-Gaussianity at
angular scales of $\lower
2.15277pt\hbox{$\;\buildrel<\over{\sim}\;$}0.1\,\mathrm{deg}$.

We also caution that our Gaussianization assumes a fiducial cosmology for the
image-to-image translation. Hence, it is still uncertain if the
Gaussianization can work even for non-Gaussian lensing maps at various
cosmological models. Another caveat in our method is that our image-to-image
translation does not account for the presence of observational noises, which
is the main practical difficulty in the Gaussianization of weak lensing
fields.