# On the use of automatically generated synthetic image datasets for
benchmarking face recognition

Laurent Colbois  Tiago de Freitas Pereira  Sébastien Marcel
Idiap Research Institute, Martigny, Switzerland
{laurent.colbois, tiago.pereira<EMAIL_ADDRESS>
###### Abstract

The availability of large-scale face datasets has been key in the progress of
face recognition. However, due to licensing issues or copyright infringement,
some datasets are not available anymore (e.g. MS-Celeb-1M). Recent advances in
Generative Adversarial Networks (GANs), to synthesize realistic face images,
provide a pathway to replace real datasets by synthetic datasets, both to
train and benchmark face recognition (FR) systems. The work presented in this
paper provides a study on benchmarking FR systems using a synthetic dataset.
First, we introduce the proposed methodology to generate a synthetic dataset,
without the need for human intervention, by exploiting the latent structure of
a StyleGAN2 model with multiple controlled factors of variation. Then, we
confirm that (i) the generated synthetic identities are not data subjects from
the GAN’s training dataset, which is verified on a synthetic dataset with 10K+
identities; (ii) benchmarking results on the synthetic dataset are a good
substitution, often providing error rates and system ranking similar to the
benchmarking on the real dataset.

##  1 Introduction

Figure 1: Example of the full set of variations generated for a single
synthetic identity. All displayed images are synthetic. The first image
(highlighted in red) is the main reference (neutral expression, frontal view,
frontal illumination). Then, using a latent editing approach we automatically
generate expression variations (first row), pose variations (second row) and
illumination variations (third row).

Face datasets used to train and benchmark face recognition (FR) systems have
reached the scale of hundreds of thousand identities. These large-scale
datasets have been responsible for pushing the state-of-the-art face
recognition systems in terms of recognition rates. Many of them claim to be
based on images of “celebrities” scraped from the web. However, this is
partially inaccurate as these datasets are not only composed of public figures
from media, but may include as well ordinary people [25] uninformed about the
use of their data. As a consequence, well established datasets had to be
discontinued [16]. In parallel, legislation is trying to address these issues.
For instance, in Europe the GDPR [10] has established that biometric data is
personal data and requires that informed consent is obtained from data
subjects. Hence, it becomes harder to collect, curate and distribute face
recognition datasets that comply with all regulations such as anonymity or
informed consent. At the same time, the past few years have seen major
improvements in synthetic face image generation, in particular using
Generative Adversarial Networks (GANs). The most recent GANs enable easy
sampling of high-resolution and very realistic face images. Moreover, their
latent space present some structure that can be exploited to edit face
attributes in a disentangled manner, and thus to generate many different
samples of the same synthetic identity.

The aim of this work is to explore if those synthetic images can be
substituted to real data when benchmarking FR systems. In a past study on this
topic [29], three required characteristic of such an evaluation dataset have
been identified: (i) Privacy: each biometric data in the synthetic dataset
should not represent any real person, (ii) Precision: the evaluation results
derived from a synthetic biometric dataset should be equal to the one from the
real dataset and (iii) Universality: the precision requirement should be
satisfied for all evaluated authentication algorithms. In the present work, we
will mainly assess whether the _precision_ and _privacy_ requirements are
satisfied with our synthetic dataset. The _universality_ requirement is much
more difficult to evaluate. We cover it partially in this work by considering
several systems, but it will be more extensively addressed in future work.

This paper first describes our dataset generation method, which relies on
exploiting semantic editing directions in the latent space of a face-based
StyleGAN2 [19]. Then we will check whether the synthetic identities are
actually new or if they are similar to real identities from the GAN’s training
dataset. We will also compare the performance of several FR systems on precise
protocols using real and synthetic datasets independently. Finally, we will
inspect some of the synthetic samples causing recognition errors to identify
possible shortcomings of our method. Overall, our main contributions are the
following:

  * •

We introduce a new method for finding semantic editing directions in the
latent space of StyleGAN2, based on the projection in this space of a real
dataset with labeled covariates. This method enables fully automatic editing,
unlike current semantic editing methods (cf. fig. 2)

  * •

We create a synthetic dataset imitating the Multi-PIE dataset [13], called
_Syn-Multi-PIE_ , to investigate if it can be substituted to a real dataset
when evaluating the robustness of FR systems to illumination, expression and
pose variations. We hypothesize that this assessment can give an insight about
the adequacy of synthetic datasets to benchmark FR systems.

##  2 Related work

###  2.1 Synthetic face generation

Since the early introduction of generative adversarial networks (GANs) in
[12], their generative ability has often been showcased on the task of face
generation. The current state-of-the-art model is the StyleGAN2 architecture
trained on the FFHQ dataset [19], which provides the most perceptually
compelling face images, while also performing best regarding the Fréchet
Inception Distance (FID), the most common metric used to quantify realism of
the generation. However, the StyleGAN2 model is unconditioned : while it
allows to sample face images from random input latent vectors, it does not
provide _a priori_ control on the semantic content of the result (e.g. pose,
expression or race of the person). It is notably non trivial to generate
several variations of the same identity. Several approaches work towards
overcoming this limitation.

First, one can use image-to-image translation networks to generate variations
through _a posteriori_ editing, the same way one would edit a real image. We
can mention in particular [17] and [21], which propose methods for editing a
broad set of facial attributes, but there exists many references proposing an
editing method targeting a specific attribute such as age [3] or expression
[8].

A second approach is instead to retrain the generative model to make it
conditional. This is generally done by ensuring the latent input to the
generator can be split between a component specifying identity and a component
specifying all other factors of variations ([9], [31], [4]), with sometimes a
even finer control on the factors of variation to manipulate several semantic
attributes in a disentangled manner ([23], [11]).

Finally, it is also possible to exploit StyleGAN2’s properties. In [26], the
authors propose to decouple disentanglement from synthesis by learning a
mapping into StyleGAN2’s latent space, from which they are able to generate
face images by respectively mixing the identity and attributes of two input
synthetic images, while keeping the synthesis quality. In [28], the authors
exploit the property of linear separation of semantic attributes in
StyleGAN2’s latent space, and propose a general framework to find latent
directions corresponding to editing only a particular semantic attribute,
while keeping other attributes untouched. In a sense, this makes the
unconditional model actually conditional, by leveraging latent properties of
the model that were learned in an unsupervised manner.

Figure 2: Usual methods to find editing directions do not provide insight on
the scale of the editing. Human intervention is required at generation time to
explore how much one can move along a direction while preserving identity and
realism of the image, which is unpractical for automatic dataset generation.
(Illustration from [18])

###  2.2 Synthetic data in face recognition

Usage of synthetic data in face recognition usually occurs at the training
step, as a form of data augmentation. A distinction can be made between depth
augmentation (increasing the number of samples per subject) and width
augmentation (increasing the number of subjects). Depth augmentation is
typically done by mapping face images to a 3D morphable model (3DMM), enabling
generation of pose, expression or illumination augmentations [6]. For width
augmentation, it is also possible to use similar 3DMMs by randomizing their
parameters [20]. GAN based approaches can be exploited both for depth- and
width- augmentation, although with them identity preservation in depth-
augmentation is a major challenge. We can mention [31], in which they train an
identity-disentangled GAN and use it to augment a FR train dataset, and [22],
where they design a 3D GAN conditioned on the parameters of a 3DMM to improve
facial recognition on extreme poses.

In this work however, we focus on the use of synthetic data to benchmark face
recognition systems. We are not aware of any recent work on this topic. We can
mention [29] that proposes a procedure to replace a real evaluation dataset
with a synthetic one that produces similar biometric score distributions, but
their work completely predates recent progresses in GAN-based face generation.

##  3 Synthetic generation methodology

For the face generator, we use the StyleGAN2 model pretrained on FFHQ [19].
The network architecture decomposes in two parts. First, a _mapping_ network,
which is a fully-connected network, takes Gaussian-sampled latent vectors as
input and transforms them into a new latent of same dimension. Its input and
output spaces are respectively called $\mathcal{Z}$ and $\mathcal{W}$.
Secondly, a _synthesis_ network uses $\mathcal{W}$-latents as input style
vectors used to control the actual face generation process. The output of this
network is a face image. One can easily generate a database with many
independent faces by randomly sampling latent vectors. However, in order to
use this synthetic data in face recognition applications, we need to generate
several images where the identity is preserved but other attributes can vary.
In this work, we focus on three types of variability : illumination, pose and
expression. This is done by exploiting the remarkable property of _linear
separability_ of StyleGAN2’s $\mathcal{W}$ space. This property emerges as a
byproduct of the training of the GAN. Its impact is that when considering
binary semantic attributes characterizing a face (young/old, female/male,
neutral/smiling expression…), we can usually find a hyperplane in
$\mathcal{W}$ that separates well the $\mathcal{W}$-latents from each class.
The normal to this hyperplane can then be used as a latent direction that
specifically edits the considered covariate, thus providing finer control on
the generation, despite the network being unconditional in the first place. In
this section, we describe our protocol for constructing a face recognition
dataset by exploiting the linear separability of $\mathcal{W}$.

###  3.1 Finding latent directions

We take inspiration from [28], which proposes an approach to find semantic
latent directions by labeling two distinct populations of
$\mathcal{W}$-latents vectors according to a binary target attribute (e.g left
vs right profile), then fitting a linear SVM in $\mathcal{W}$ to separate
those two populations. In their case, the latent vectors are randomly sampled
then labeled by running their associated image through an auxiliary pretrained
attribute classification network. Their method has the partial drawback of
needing this auxiliary network, but more importantly it does not provide any
notion of scaling of the editing, i.e. how much along one latent direction one
can move while preserving the identity and the realism of the image. To avoid
those issues, we propose an alternative approach for obtaining the latent
populations on which the SVMs are fitted. Instead of using random images, we
_project_ into the $W$ space the content of a dataset of real images. By
projection is meant the process of finding a $\mathcal{W}$-latent for which
the associated image is perceptually close to a target real image. This is
done through optimization, generally using the perceptual loss as the
objective to minimize. Several variants of this approach exists ([1], [2]),
mainly differing in the details of the optimization process. We use the
implementation from [19].

This projection-based approach removes the need for an auxiliary
classification network, and it gives access to a sense of scale : as a
reasonable heuristic, we can keep track of the _average distance to the
hyperplane_ of each population of projected latents. We hypothesize that by
using this distance as the editing scale for our synthetic identities, the
resulting range of variation for each attribute will be similar to the one
observed in the real dataset. This enables us to generate all variations of
each synthetic identity in a fully automatic manner, without requiring human
intervention to decide the strength of each editing.

We project the Multi-PIE dataset [13]. For each identity, it contains face
images with labeled covariates for pose, illumination and expression. The
available expressions are _neutral, smile, disgust, scream, squint, surprise_.
After projection, we use the resulting latents to fit SVMs in $W$ and find the
following interesting latent directions: left-right pose edition, left-right
illumination edition, and edition between any pair of expressions, along with
the associated maximum editing scales. Figure 3 illustrate the full process.

Figure 3: To obtain a latent direction, we project in $\mathcal{W}$ labeled
images, then fit a linear SVM to separate the latents. The normal to the
resulting hyperplane is used as the editing direction for this attribute. New
identities are obtained by randomly sampling a $\mathcal{Z}$-latent. We then
edit its associated $\mathcal{W}$-latent using our computed latent directions
to obtain variations of the original image.

###  3.2 Syn-Multi-PIE Dataset

Finally, we generate a dataset of synthetic faces with controlled variations.
New identities are obtained by sampling a latent in the $\mathcal{Z}$ space,
and we create the main reference image by neutralizing the face in the
$\mathcal{W}$ space, which means we edit it to ensure it has frontal view,
frontal lightning and neutral expression. For pose and illumination
neutralization, we simply project the $\mathcal{W}$-latent onto the separation
hyperplane. For expression neutralization, a preliminary qualitative
observation of the typical output of StyleGAN2 shows that randomly sampled
latents mostly produce neutral or smiling expressions. We thus simply edit the
latent along the neutral-to-smile direction to increase its neutral component.

To ensure enough variability between identities, we optionally apply a
constraint on the interclass threshold (ICT), which specifies the minimum face
embedding distance required between any identity pair. We only accept new
candidate identities if they satisfy the constraint w.r.t every previous
identity. The embedding distance is measured using a Inception-ResNet-v2 model
[30] pretrained on Ms-Celeb [15]. In a second pass, we generate variations for
each identity using our precomputed latent directions. For scaling our edits,
we use the measured average distance of Multi-PIE projections to the
hyperplanes as the maximal variation. Figure 1 presents example of the full
set of variations obtained for a given face. We will showcase in the following
sections that this synthetic database satisfies the _privacy_ and _precision_
requirements, making it a good substitution to Multi-PIE to benchmark FR
systems. We thus name it Syn-Multi-PIE. With a concern for reproducibility of
our work, we also release the code to regenerate this
database.111https://gitlab.idiap.ch/bob/bob.paper.ijcb2021_synthetic_dataset
Additional details on the generation algorithm can also be found in the
supplementary material.

##  4 Are StyleGAN2 identities new ?

To assess the requirement of _privacy_ , we need to verify that generated
identities do not simply reproduce existing identities from the FFHQ dataset.
We evaluate this by reproducing on our synthetic dataset an experiment
originally proposed in [32] on the first version of StyleGAN. It consists in
comparing the identity similarity between a synthetic dataset (Sy) and a seed
dataset (Se), which is the dataset used to train the face generator. A third
dataset (Ref) of real data with labeled identities is used as reference to
estimate the density of lookalikes in a standard population.

For this experiment we use an Inception-ResNet v2 trained on Casia-WebFace
[33] for face embedding extraction. We measure face similarity as the negative
cosine distance between face embeddings. We then compare three ROC curves. The
Ref-ROC is built solely from identities from the reference dataset. For the
Sy-Se-ROC, we keep genuine pairs from Ref, and assuming that Sy identities
should all be different from Se ones, we use Sy-Se comparisons as impostor
pairs. If this ROC curve is shifted downwards with respect to the Ref-ROC, it
means that Sy-Se pairs contain more false matches than Ref impostors pairs,
i.e. Sy-Se pairs contains more lookalikes than the Ref dataset, and thus Sy
identities are not truly new. If however the Sy-Se ROC is superposed to the
Ref-ROC, then we can say Sy identities are new. Doing the same experiment with
Sy-Sy pairs, we can also assess the amount of lookalikes _between_ Sy
identities. For establishing the Ref-ROC, we work with the IJB-C dataset under
the 1:1 verification protocol [24]. We generate two sets of 11k synthetic
identities, once with no ICT constraint, and once with an ICT constraint of
0.1. The value of 0.1 was empirically chosen as a good-trade off between
identity variability, and number of rejected faces during the generation.
Those 11k identities are compared with the 70k images of the seed dataset
(FFHQ) to obtain the Sy-Se curves, and they are compared between themselves to
get the Sy-Sy curves. The resulting ROC curves are presented in figure 4.

We first focus on the Sy-Se curves. We observe that without an ICT constraint,
the Sy-Se-ROC lies below the Ref-ROC. This indicates there is a higher density
of lookalikes between Sy and Se than in a real population, which is here
modeled by IJB-C. This is similar to the observation made in [32] using
StyleGAN. In the article, they notice that this is caused by the presence of
children in FFHQ, and thus also in StyleGAN’s output distribution, while SOTA
face recognition networks are typically not trained on faces of children and
thus perform poorly on this population. We observe the same behavior, as
showcased in the leftmost columns of figure 5 that present the closest Sy-Se
matches. They solved the issue by manually removing the children, but this is
unpractical if we plan to generate a large number of identities.
Alternatively, we can apply the ICT constraint. Indeed, this greatly reduces
the amount of children in the synthetic dataset: many children candidate faces
are rejected by the model used to apply the ICT constraint, as this model
itself fails at distinguishing different children. We showcase on the
rightmost columns of figure 5 how introducing the ICT constraint impacts the
closest Sy-Se lookalikes: while children are mostly removed, the lookalikes
now appear to be from an east-asian population. We can hypothesize that this
FR model might still be underperforming this demographic, maybe due to it
being underrepresented in the training data. However, a study on the
demographic differentials of this model is out of the scope of this work.
Despite this, our Sy-Se-ROC is now quite well superposed to the Ref-ROC. This
suggests there is the same density of lookalikes between the Sy and Se
database as in IJB-C - at least under the scrutiny of this particular FR
model. Therefore the Sy identities are as “novel” as possible. Focusing on the
Sy-Sy curves, we observe however that the variability of identities _inside_
the synthetic database is lesser than in real data, but this does not
invalidate the results of the Sy-Se experiment that shows that the generator
is indeed not just reproducing existing FFHQ identities. This is an important
point which validates the use of synthetic face data as a way to satisfy the
_privacy_ requirement.

Figure 4: ROC curves obtained when comparing identities from the synthetic
dataset (Sy) to the seed dataset (FFHQ). The Ref-ROC is obtained using the
IJB-C dataset (with the 1:1 verification protocol). Figure 5: Closest matches
in FFHQ to synthetic identities, with no ICT constraint (leftmost columns) and
with an ICT constraint of 0.1 (rightmost columns).

##  5 Can we use a synthetic dataset for benchmarking face recognition systems
?

The central question of this work is whether we can completely replace real,
sensitive FR data by synthetic and private one in a _benchmark_ setup. We thus
have to assess the _precision_ requirement, i.e. verify whether we can do this
real-to-synthetic substitution and still obtain similar conclusions on the
performance of several FR systems (error rate, system ranking and robustness
to different factors of variation). We hypothesize that the answer to this
question could depend on the range of considered variations, and on the
quality of our face editing process, which might be better for some covariates
than for others. For this reason, we choose not to perform a large-scale
experiment which would not enable a fine analysis. Instead, we reproduce
synthetically 3 Multi-PIE evaluation protocols, each of which targets a single
covariate, and we compare the performance on the equivalent synthetic and real
protocols. The U protocol targets illumination (enrollment with frontal
lightning, probing with lightning variations), the E protocol targets
expression (enrollment with neutral expression, probing with other
expressions), and the P protocol targets pose (enrollment with frontal view,
probing with other views). Equivalent protocols with Syn-Multi-PIE are also
defined, using the generated variations shown in figure 1. We always use the
reference image for the enrollment, and probe the system only with variations
of the target covariate. We use the same number of 64 identities in both
setups. 6 FR systems, listed in the caption of table 1, are then benchmarked
on those protocols.

###  5.1 Experiment results

Database | Multi-PIE | Syn-Multi-PIE
---|---|---
Protocol | U | E | P | U | E | P
Gabor | 0.42 | 0.46 | 0.66 | 0.43 | 0.46 | 0.67
LGBPHS | 0.43 | 0.42 | 0.72 | 0.30 | 0.38 | 0.67
AF | 0.11 | 0.3 | 0.55 | 0.12 | 0.25 | 0.51
Inc-Res.v1 | 0.05 | 0.14 | 0.50 | 0.09 | 0.50 | 0.47
Inc-Res.v2 | 0.07 | 0.15 | 0.26 | 0.08 | 0.45 | 0.44
AF-VGG2 | 0.08 | 0.16 | 0.40 | 0.10 | 0.44 | 0.47
Table 1: FNMR values at FMR@1E-3. We highlighted all the scores where the
error rate on Syn-Multi-PIE was more that 5% away from the error rate on the
equivalent Multi-PIE protocol. The considered systems are a Gabor graph
(_Gabor_) [14], a local Gabor binary pattern histogram sequence (_LGBPHS_)
[14], and 4 neural net based systems : original ArcFace (_AF_) [7], ArcFace
retrained on VGGFace2 (_AF-VGG2_) [5], Inception-ResNet models trained on
Casia-WebFace (_Inc-Res.v1_ and _v2_) [27].

(a) U - False non-matches

(b) U - False matches

(c) E - False non-matches

(d) E - False matches

(e) P - False non-matches

(f) P - False matches

Figure 6: Worst matches at FMR@1E-3 with the Inception-ResNet v2 system evaluated on Syn-Multi-PIE. Additional examples are provided in the supplementary material. Protocol | U | E | P
---|---|---|---
Compared | MGS | SEP | MGS | SEP | MGS | SEP
Gabor | 0.17 | 1.6 | 0.05 | 1.11 | 0.09 | 1.19
LGBPHS | 0.49 | 2.58 | 0.14 | 1.34 | 0.17 | 1.36
AF | 0.27 | 1.2 | -0.22 | 0.97 | 0.21 | 1.27
Inc-Res.v1 | 0.52 | 0.85 | -0.36 | 0.7 | 0.43 | 0.89
Inc-Res.v2 | 0.32 | 1.06 | -0.22 | 0.68 | 0.05 | 0.84
AF-VGG2 | 0.37 | 0.97 | -0.4 | 0.68 | 0.16 | 0.92
Table 2: Comparison of summary statistics of the score histograms. We report
under MGS : $(\mu_{GS}-\mu_{GR})/|\mu_{GR}|$ (comparison of the Mean Genuine
Similarity), and under SEP : $|\mu_{GS}-\mu_{IS}|/|\mu_{GR}-\mu_{IR}|$
(comparison of the mean SEParation between genuine and impostor scores), with
$\mu_{GS},\mu_{IS}$ resp. the mean Genuine & Impostor scores for the Synthetic
protocol, and $\mu_{GR},\mu_{IR}$ the same but for the equivalent Real
protocol. For MGS, any value $>0$ indicates a higher average similarity in
genuine synthetic comparisons than in genuine real comparisons. For SEP, any
value $>1$ indicates a stronger separation of the synthetic impostor & genuine
distributions than of the real ones.

Table 1 presents the measured error rates. In the ArcFace row for example
(AF), the first 3 columns (Multi-PIE protocols) inform on the robustness of
the system to each considered factor of variation. The system shows good
robustness to illumination variation (U protocol, FNMR of 0.11), less
robustness to expression variation (E protocol, FNMR of 0.3) and the least
robustness to pose variation (P protocol, FNMR of 0.5). The last 3 columns
provide the same information using Syn-Multi-PIE instead. We observe that our
synthetic protocols are a good substitute to the real protocols, with a
majority of Syn-Multi-PIE error rates lying at less than $5\%$ from the error
rate on the corresponding Multi-PIE protocol. Qualitatively speaking, this
means Syn-Multi-PIE satisfies the requirement of _precision_ , but there are
still some imperfections. For illumination protocols the error rates follow a
very similar trend, with the exception of the LGBPHS system. For pose
protocols, there is a bit more discrepancy with the Inception-ResNet v2 and
the ArcFace-VGG2 models, however the ranking of the systems remain very
similar, with a clear performance improvement when using neural nets, and with
Inception-ResNet v2 appearing as the best model on this protocol. The picture
is less clear on the expression protocols, with the Inception-ResNet and
ArcFace-VGG2 models presenting significantly subpar performance on Syn-Multi-
PIE. We now try to explain those failure cases. To do so, we study some visual
examples of the worst false matches and non-matches in Syn-Multi-PIE
protocols, in figure 6. We especially focus on the Inception-ResNet v2 system,
on which we also performed the identity uniqueness experiment in section 4.
Moreover, we also compare some summary statistics of the score histograms,
which are presented in table 2.

###  5.2 Identity preservation

We crucially want to check that we are correctly preserving identity during
the latent editing process. Let us emphasize that the “difficulty” of the
synthetic protocols and hence the obtained error rates will be correlated with
the strength of the editing. In particular, when we get a false non-match, is
it because the FR system lacks robustness to the particular variation we are
considering, or is it because the FR system is actually right and our image
editing did not preserve the identity ? The highlighted discrepancies in table
1 showcasing mainly _excessive_ error rates, it suggests we might indeed be in
the situation of overly strong edits that do not preserve identity well. Those
would indeed appear as false non matches in the analysis, as we still label
those overly edited images as having the same identity as the reference. We
thus consider the worst false non matches : examples are presented in figures
6(a), 6(c) and 6(e).

Assessing preservation of identity is a subjective question. Readers can make
their own opinion from the provided examples, but an extensive perceptual
study would be necessary to answer this question more thoroughly. The examples
at least suggest that identity preservation is good when doing illumination
and expression editing. For pose editing however, the worst false matches
contain enough perturbations to the facial details to possibly alter the
identity. In this context our identity labeling could be partially erroneous,
thus causing an increased number of false non matches. We could limit this
issue by reducing the strength of the pose editing and renouncing to the most
extreme angles. But the root cause might be an under-representation of extreme
poses in the FFHQ dataset (and thus in StyleGAN2’s output distribution) making
the model fundamentally unable to generate far-from-frontal images. To perform
this analysis in a more objective manner, we can also read the Mean Genuine
Similarity (MGS) comparison values in table 2. An overly edited face image
would generate an excessively low similarity score when compared to its
reference. If our editing is too strong, we should thus observe a significant
drop in the average synthetic genuine similarity score. In contrary, we
observe that for the U and P protocol, the MGS is never lower in Syn-Multi-PIE
than it is in Multi-PIE (all reported values are positive), which suggests
that for both those covariates the editing is not too strong. The negative MGS
comparison values with the E protocol might however hint towards slightly
excessive expression editing.

###  5.3 High similarity impostors

Another possible cause of score discrepancy between synthetic and real
protocols could be an excessive amount of very close synthetic identities
causing a large number of high-score impostors. We showcase some of the worst
false matches in figures 6(b), 6(d) and 6(f). We first notice that all those
false matches are between feminine looking faces. This could be a possible
indicator of a demographic differential of the Inception-ResNet v2 system
against a feminine population, however it should then also occur during the
Multi-PIE evaluation and it should not cause such a strong difference in
performance. It can however be argued that some of the impostor identities
actually do look very similar to the reference, to the point where a human
judge might also consider them to be the same person. As seen in section 4,
the amount of variability between synthetic identities has been measured to be
lower than in real data: we might be seeing the effect of this lack of
variability. The summary statistics (SEP values in table 2) lead to the same
conclusion : for neural-net based models, on E and P protocols, the distance
between the impostor and genuine distributions is significantly smaller with
Syn-Multi-PIE than it is with Multi-PIE. Even though we applied the ICT
constraint when generating the references, it seems the subsequent face
editions are enough to give way for many high score impostors, which are then
responsible for the performance discrepancy w.r.t. the Multi-PIE benchmarks.
In other terms, although for each synthetic identity we seem to span a
realistic set of variations, the identities still are _closer_ from each other
than real ones, and so there is more overlap between each of those identity-
variation sets, causing an excessive amount of false matches. A natural way to
fix this issue would be to increase the ICT value to spread out synthetic
identities even more. But the issue that comes then is that with pure random
sampling, we start to reject a high number of candidates when generating the
reference identities, making the runtime grow quickly with the number of
identities. Moreover, the ICT constraint is just a tentative to “fix” the lack
of variability of StyleGAN2 identities. We can hope future improvements in
face GANs will further increase the richness of their output distribution, in
particular identity variability.

##  6 Conclusion

In this work, we have presented a method to generate synthetic databases for
face recognition by exploiting the StyleGAN2 latent space. This has enabled us
to introduce the Syn-Multi-PIE dataset, which presents the same factors of
variation as the Multi-PIE database, but does not contain any real identity
and thus can be shared much less restrictively. Moreover, our database
construction process is automatic and can be extended to other covariates.
Benchmark experiments carried out with 6 face recognition systems have shown
that Syn-Multi-PIE can be used in place of Multi-PIE and lead to similar
conclusion on the systems’ error rates and ranking. While we have noticed some
performance discrepancies in some of the setups, our analysis suggests those
discrepancies generally do not seem to be caused by our editing method, which
looks to preserve identity quite well. We do note that our quantitative
observations (MGS) on expression editing raise some doubt on the quality of
the identity preservation in this context, however the visual examples (false
non-matches) reinforce the claim. In a future study, it could be useful to
perform a perceptual study with human raters, as it remains the only way to
fully validate identity preservation.

Overall, performance discrepancies seem rather caused by limitations of the
generative capacity of StyleGAN2, mainly the lack of variability of generated
identities, and a lack of extreme poses in the bulk of the output
distribution. The field of generative face models currently following a very
fast evolution, we expect those issues to improve as new models emerge able to
produce an even more realistic output image distribution.

Other usages for synthetic data can be conceived. Given the available semantic
control in the $\mathcal{W}$ space, one could for example generate datasets
with a balanced density of each demographics (gender, age or race), which
could be of use to expose possible demographic differentials in current SOTA
systems. Finally, our generation process scaling at least to 10k+ identities,
a natural next step would be to generate a large-scale dataset and assess its
usability for _training_ FR models while solving the data privacy issue. This
direction has indeed not been deeply explored, at least using GAN-based
generative models. Again, it would probably require a stronger preliminary
verification of the identity preservation at the editing step, else we can
foresee mislabeled synthetic images causing issues during the training,
leading to subpar performance.

## References

  * [1] https://github.com/rolux/stylegan2encoder.
  * [2] R. Abdal et al.  Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?  In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 4431–4440, Oct. 2019.  ISSN: 2380-7504.
  * [3] G. Antipov et al.  Face aging with conditional generative adversarial networks.  In 2017 IEEE International Conference on Image Processing (ICIP), pages 2089–2093, Sept. 2017.  ISSN: 2381-8549.
  * [4] J. Bao et al.  Towards Open-Set Identity Preserving Face Synthesis.  In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6713–6722, June 2018.  ISSN: 2575-7075.
  * [5] Q. Cao et al.  VGGFace2: A Dataset for Recognising Faces across Pose and Age.  In 2018 13th IEEE International Conference on Automatic Face Gesture Recognition (FG 2018), pages 67–74, May 2018.
  * [6] D. Crispell et al.  Dataset Augmentation for Pose and Lighting Invariant Face Recognition.  arXiv:1704.04326 [cs.CV], Apr. 2017.
  * [7] J. Deng et al.  ArcFace: Additive Angular Margin Loss for Deep Face Recognition.  arXiv:1801.07698 [cs], Feb. 2019.
  * [8] H. Ding et al.  ExprGAN: Facial Expression Editing With Controllable Expression Intensity.  Proceedings of the AAAI Conference on Artificial Intelligence, 32(1), Apr. 2018.
  * [9] C. Donahue et al.  Semantically decomposing the latent spaces of generative adversarial networks.  In International Conference on Learning Representations, 2018.
  * [10] European Commission.  General data protection regulation - processing of special categories of personal data, 2018.  https://gdpr-info.eu/art-9-gdpr/.
  * [11] P. Ghosh et al.  GIF: Generative interpretable faces.  In International Conference on 3D Vision (3DV), 2020.
  * [12] I. Goodfellow et al.  Generative Adversarial Nets.  In Z. Ghahramani et al., editors, Advances in Neural Information Processing Systems 27, pages 2672–2680. Curran Associates, Inc., 2014.
  * [13] R. Gross et al.  Multi-PIE.  In 2008 8th IEEE International Conference on Automatic Face Gesture Recognition, pages 1–8, Sept. 2008.
  * [14] M. Günther, L. El Shafey, and S. Marcel.  Face recognition in challenging environments: An experimental and reproducible research survey.  In T. Bourlai, editor, Face Recognition Across the Imaging Spectrum. Springer, 1 edition, Feb. 2016.
  * [15] Y. Guo et al.  MS-Celeb-1M: A dataset and benchmark for large-scale face recognition.  In ECCV, 2016.
  * [16] A. Harvey.  Exposing.ai: Microsoft Celeb.
  * [17] Z. He et al.  AttGAN: Facial Attribute Editing by Only Changing What You Want.  IEEE Transactions on Image Processing, 28(11):5464–5478, Nov. 2019\.
  * [18] K. Hill and J. White.  Designed to Deceive: Do These People Look Real to You?  The New York Times, Nov. 2020.
  * [19] T. Karras et al.  Analyzing and Improving the Image Quality of StyleGAN.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8110–8119, 2020.
  * [20] A. Kortylewski et al.  Training Deep Face Recognition Systems with Synthetic Data.  arXiv:1802.05891 [cs.CV], Feb. 2018.
  * [21] M. Liu et al.  STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing.  In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3668–3677, June 2019.  ISSN: 2575-7075.
  * [22] R. T. Marriott et al.  A 3D GAN for Improved Large-pose Facial Recognition.  arXiv:2012.10545 [cs], Dec. 2020.
  * [23] R. T. Marriott et al.  Taking control of intra-class variation in conditional gans under weak supervision.  In 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020) (FG), pages 257–264, Los Alamitos, CA, USA, may 2020. IEEE Computer Society.
  * [24] B. Maze et al.  IARPA Janus Benchmark - C: Face Dataset and Protocol.  In 2018 International Conference on Biometrics (ICB), pages 158–165, Feb. 2018.
  * [25] M. Murgia and M. Harlow.  Who’s using your face? The ugly truth about facial recognition, Apr. 2019.  https://www.ft.com/content/cf19b956-60a2-11e9-b285-3acd5d43599e.
  * [26] Y. Nitzan et al.  Face identity disentanglement via latent space mapping.  ACM Transactions on Graphics, 39(6):225:1–225:14, Nov. 2020.
  * [27] T. d. F. Pereira et al.  Heterogeneous Face Recognition Using Domain Specific Units.  IEEE Transactions on Information Forensics and Security, 14(7):1803–1816, July 2019.
  * [28] Y. Shen et al.  Interpreting the Latent Space of GANs for Semantic Face Editing.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9243–9252, 2020.
  * [29] K. Sumi et al.  Study on Synthetic Face Database for Performance Evaluation.  In D. Zhang and A. K. Jain, editors, Advances in Biometrics, Lecture Notes in Computer Science, pages 598–604, Berlin, Heidelberg, 2005\. Springer.
  * [30] C. Szegedy et al.  Inception-v4, Inception-ResNet and the impact of residual connections on learning.  In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI’17, pages 4278–4284, San Francisco, California, USA, Feb. 2017. AAAI Press.
  * [31] D. S. Trigueros et al.  Generating Photo-Realistic Training Data to Improve Face Recognition Accuracy.  arXiv:1811.00112 [cs, stat], Oct. 2018.
  * [32] V. Varkarakis et al.  Validating Seed Data Samples for Synthetic Identities – Methodology and Uniqueness Metrics.  IEEE Access, 8:152532–152550, 2020.
  * [33] D. Yi et al.  Learning Face Representation from Scratch.  arXiv:1411.7923, Nov. 2014.

##  Appendix A Database generation details

In this section we provide a bit more technical details on the synthetic
database generation process. First, we describe the database projection
protocol, which for each covariate of interest specifies which subset of
images we project to form the two latent populations on which the SVM is
fitted. We then provide some details on the generation of reference images, in
particular on the face neutralization process, including a pseudocode
algorithm. Finally, we present in a second pseudocode the process of
generating variations for each identity, using the precomputed latent
directions.

###  A.1 Database projection protocol

We project the Multi-PIE database [13]. For each identity, it contains face
images with labeled covariates for expression, view angle and lightning
direction. The available expressions are _neutral, smile, disgust, scream,
squint, surprise_. For notation convenience, we number them in that order from
0 to 5.

We project the world set of Multi-PIE, then use the projected latents to fit
SVMs in $W$ and find interesting latent directions. Table 3 specifies our
protocol, i.e. which subset of the projected latents are used to compute each
of the latent directions of interest. After this process we have access to the
following latent directions (unit vector normals to the SVMs hyperplanes) :
left-right pose edition ($\mathbf{\hat{n}_{P}}$), left-right illumination
edition ($\mathbf{\hat{n}_{I}}$), and edition between any pair of expressions,
for example $\mathbf{\hat{n}_{01}}$ for neutral-to-smile and
$\mathbf{\hat{n}_{02}}$ for neutral-to-disgust.

We also need some notion of scaling, i.e. to determine how much we can move
along a given latent direction while still preserving the identity and realism
of the face. As a reasonable heuristic, we thus also keep track of the mean
distance to the hyperplane of each population of projected latents. We denote
it by $d_{P}^{L}$, $d_{P}^{R}$ for the left and right pose,
$d_{I}^{L}$,$d_{I}^{R}$ for the left and right illumination, and by
$d_{01}^{0}$ and $d_{01}^{1}$ for example for the neutral and smile
populations with respect to the $\mathbf{\hat{n}_{01}}$ hyperplane.

Attribute | Expressions | Cameras | Lightning | Binary classes
---|---|---|---|---
Expression | All | Frontal view | Frontal flash | All expression pairs
Pose | Neutral | Non frontal views in $[-45\degree,45\degree]$ | Frontal flash | Left & right profiles
Illumination | Neutral | Frontal view | All non frontal with flash | Left & right flashes
Table 3: Description of the Multi-Pie subsets used to compute the latent
directions for each attribute.

###  A.2 Generation of identities

Figure 7: Examples of generated faces before and after the face neutralization
process. While the expression neutralization seems qualitatively successful,
it is not the case for pose neutralization. But we do not need the faces to be
perfectly frontal : the only aim is to make sure to keep reasonable poses when
editing the image along the pose editing direction, so we simply need to start
close to a frontal view. Illumination neutralization is not very visible, due
to the absence of extreme illuminations in the typical uncontrolled output of
StyleGAN2.

Algorithm 1 explains in pseudocode the process of generating the references,
using our notation from before for the latent directions. It includes a
neutralization step, during which the candidate face is edited to have frontal
view, frontal lightning and neutral expression. After neutralization,
candidate faces are optionally rejected based on the comparison of their
embedding distance to all previous faces w.r.t the required ICT. Figure 7
presents example of generated faces before and after the neutralization
process.

Algorithm 1 Creation of reference identities

procedure New_ID($W_{prev}$, ICT)

repeat

$\mathbf{z}\leftarrow\text{Random z-latent}$

$\mathbf{w}\leftarrow\text{{Mapping}}(\mathbf{z})$ $\triangleright$ StyleGAN2
mapping

$\mathbf{w_{ref}}$ $\leftarrow{\text{{Neutralize}}(\mathbf{w})}$

until ClosestDistance($\mathbf{w_{ref}}$, $W_{prev}$) $>$ ICT

Append($W_{prev}$, ref)

end procedure

function Neutralize($\mathbf{w}$)

$\mathbf{w}\leftarrow\mathbf{w}-(\mathbf{w}^{\top}\mathbf{\hat{n}_{P}})\cdot\mathbf{\hat{n}_{P}}$
$\triangleright$ Pose

$\mathbf{w}\leftarrow\mathbf{w}-(\mathbf{w}^{\top}\mathbf{\hat{n}_{I}})\cdot\mathbf{\hat{n}_{I}}$
$\triangleright$ Illumination

$\mathbf{w}\leftarrow\mathbf{w}-(\mathbf{w}^{\top}\mathbf{\hat{n}_{01}}+d_{01}^{0})\cdot\mathbf{\hat{n}_{01}}$
$\triangleright$ Expression

return $\mathbf{w}$

end function

function ClosestDistance($\mathbf{w}$, $W_{prev}$)

$\mathbf{e}=\text{{Embedding}({Synthesis}}(\mathbf{w}))$ $\triangleright$
Compute image and face embedding

dists $\leftarrow$ []

for all $\mathbf{w^{\prime}}$ in $W_{prev}$ do

$\mathbf{e^{\prime}}=\text{{Embedding}({Synthesis}}(\mathbf{w^{\prime}}))$

Append(dists, $\text{{CosineDistance}}(e,e^{\prime})$)

end for

return Minimum(dists)

end function

###  A.3 Generation of variations

Algorithm 2 presents in pseudocode the process of generating variations for
each reference, using the precomputed latent editing directions.

Algorithm 2 Creation of variations

procedure Variations($\mathbf{w_{r}ef}$, $n_{var}$)

pose_var $\leftarrow$ LRVar($\mathbf{w_{ref}}$, $\mathbf{\hat{n}_{P}}$,
$d_{P}^{L}$, $d_{P}^{R}$, $n_{var}$)

illum_var $\leftarrow$ LRVar($\mathbf{w_{ref}}$, $\mathbf{\hat{n}_{I}}$,
$d_{I}^{L}$, $d_{I}^{R}$, $n_{var}$)

expr_var $\leftarrow$ ExprVar($\mathbf{w_{ref}}$)

end procedure

Left-Right editing

function LRVar($\mathbf{w}$, $\mathbf{\hat{n}}$, $d^{L}$, $d^{R}$, $n_{var}$)

variations $\leftarrow$ []

$D\leftarrow$ Maximum($d^{L}$, $d^{R}$)

dists $\leftarrow$ LinSpace($-D$, $D$,$n_{var}$)

for all $d$ in dists do

Append(variations, $\mathbf{w}+d\cdot\mathbf{\hat{n}}$)

end for

return variations

end function

Expression editing

function ExprVar($\mathbf{w}$)

variations $\leftarrow$ []

for $j$ in 1..5 do

Append(variations,
$\mathbf{w}+(-\mathbf{w}^{\top}\mathbf{\hat{n}_{0j}}+d_{0j}^{j})\cdot\mathbf{\hat{n}_{0j}}$)

end for

end function

##  Appendix B Visual examples

In this section, we provide some more visual examples of false matches / non-
matches obtained at FAR@1E-3 when evaluating the Inception-ResNet v2 system on
the Syn-Multi-PIE protocols, respectively U, E and P in figures 8, 9 and 10.

(a) U - False non-matches

(b) U - False matches

Figure 8: Worst matches at FMR@1E-3 with the Inception-ResNet v2 system
evaluated on the Syn-Multi-PIE U protocol.

(a) E - False non-matches

(b) E - False matches

Figure 9: Worst matches at FMR@1E-3 with the Inception-ResNet v2 system
evaluated on the Syn-Multi-PIE E protocol.

(a) P - False non-matches

(b) P - False matches

Figure 10: Worst matches at FMR@1E-3 with the Inception-ResNet v2 system
evaluated on the Syn-Multi-PIE P protocol.


B1b | HLN1 | 20:14:25.934 | 41:13:33.708 | 10.2$\pm$0.8 | 80$\pm$6 | 324$\pm$4 | 180$\pm$13
B2a | HLN2 | 20:14:25.873 | 41:13:33.470 | 4.8$\pm$0.6 | 37$\pm$5 | 344$\pm$7 | 526$\pm$68
B2b | HLN2 | 20:14:25.879 | 41:13:33.635 | 16.5$\pm$0.2 | 129$\pm$2 | 286$\pm$3 | 152$\pm$2
B2c | HLN5 | 20:14:25.867 | 41:13:34.279 | 14.5$\pm$0.7 | 113$\pm$5 | 316$\pm$3 | 206$\pm$10
B2d | HLN5 | 20:14:25.853 | 41:13:34.360 | 12.9$\pm$0.7 | 100$\pm$5 | 320$\pm$3 | 248$\pm$13
B2da | HLN5 | 20:14:25.852 | 41:13:34.305 | 10.8$\pm$1.0 | 84$\pm$8 | 320$\pm$5 | 295$\pm$28
B2db | HLN5 | 20:14:25.836 | 41:13:34.296 | 13.8$\pm$1.3 | 108$\pm$10 | 304$\pm$5 | 245$\pm$24
B2dc | HLN5 | 20:14:25.804 | 41:13:34.340 | 15.3$\pm$1.2 | 119$\pm$10 | 312$\pm$5 | 251$\pm$21
B2e | HLN4 | 20:14:25.825 | 41:13:33.861 | 14.7$\pm$0.5 | 114$\pm$4 | 292$\pm$4 | 228$\pm$7
B2ea | HLN4 | 20:14:25.814 | 41:13:33.962 | 14.4$\pm$1.2 | 112$\pm$10 | 293$\pm$5 | 245$\pm$21
B2f | HLN3 | 20:14:25.850 | 41:13:33.948 | 16.6$\pm$1.2 | 129$\pm$10 | 314$\pm$5 | 183$\pm$14
B2fa | HLN3 | 20:14:25.844 | 41:13:33.869 | 17.7$\pm$1.2 | 138$\pm$10 | 315$\pm$5 | 174$\pm$12
B3a | HLN7 | 20:14:25.765 | 41:13:35.010 | 16.9$\pm$0.9 | 132$\pm$7 | 310$\pm$3 | 276$\pm$15
B3b | HLN8 | 20:14:25.764 | 41:13:35.307 | 16.3$\pm$0.1 | 127$\pm$1 | 319$\pm$1 | 297$\pm$2
B3ba | HLN8 | 20:14:25.769 | 41:13:35.488 | 20.3$\pm$1.5 | 158$\pm$12 | 323$\pm$4 | 241$\pm$18
B3bb | HLN8 | 20:14:25.768 | 41:13:35.667 | 13.5$\pm$1.5 | 105$\pm$12 | 312$\pm$6 | 370$\pm$41
B3c | HLN9 | 20:14:25.749 | 41:13:36.070 | 15.1$\pm$0.1 | 117$\pm$1 | 321$\pm$1 | 363$\pm$2
B3d | HLN10 | 20:14:25.718 | 41:13:36.250 | 6.3$\pm$0.3 | 49$\pm$2 | 309$\pm$3 | 951$\pm$47
C1a0 |  | 20:14:25.940 | 41:13:35.350 | 11.1$\pm$1.1 | 86$\pm$9 | 316$\pm$6 | 278$\pm$28
C1b |  | 20:14:25.841 | 41:13:36.440 | 14.6$\pm$2.5 | 113$\pm$20 | 300$\pm$10 | 329$\pm$57
C1c |  | 20:14:25.882 | 41:13:36.820 | 4.3$\pm$0.2 | 33$\pm$2 | 5$\pm$3 | 1060$\pm$55
C1d |  | 20:14:25.832 | 41:13:36.882 | 6.4$\pm$1.2 | 50$\pm$10 | 326$\pm$1 | 816$\pm$159
C1e | HLN11 | 20:14:25.792 | 41:13:37.100 | 6.7$\pm$1.0 | 52$\pm$8 | 326$\pm$9 | 690$\pm$104
C1f | HLN11 | 20:14:25.768 | 41:13:37.060 | 12.0$\pm$1.0 | 93$\pm$8 | 291$\pm$5 | 591$\pm$50
C1g | HLN11 | 20:14:25.750 | 41:13:37.060 | 4.1$\pm$0.5 | 32$\pm$4 | 342$\pm$6 | 1507$\pm$197
C1h1 | HLN12 | 20:14:25.724 | 41:13:37.250 | 3.2$\pm$0.5 | 25$\pm$4 | 315$\pm$10 | 2031$\pm$337
C1h2 | HLN12 | 20:14:25.720 | 41:13:37.254 | 6.0$\pm$1.0 | 46$\pm$8 | 314$\pm$10 | 1107$\pm$188
C1i | HLN15b | 20:14:25.681 | 41:13:37.256 | 6.4$\pm$0.4 | 50$\pm$3 | 313$\pm$4 | 1096$\pm$76
C1hh | HLN14 | 20:14:25.652 | 41:13:36.561 | 16.9$\pm$1.2 | 131$\pm$10 | 293$\pm$4 | 440$\pm$33
C1hi | HLN14 | 20:14:25.660 | 41:13:36.609 | 1.7$\pm$1.2 | 13$\pm$10 | 316$\pm$42 | 4319$\pm$3128
C1hj | HLN15a | 20:14:25.683 | 41:13:36.940 | 8.1$\pm$1.2 | 63$\pm$10 | 322$\pm$9 | 812$\pm$125
C1hk | HLN14 | 20:14:25.697 | 41:13:36.600 | 9.1$\pm$1.2 | 71$\pm$10 | 318$\pm$8 | 708$\pm$97
C1k1 | HLN13 | 20:14:25.632 | 41:13:36.300 | 7.7$\pm$1.2 | 60$\pm$10 | 244$\pm$9 | 914$\pm$148
C1k | HLN13 | 20:14:25.634 | 41:13:36.330 | 5.4$\pm$1.2 | 42$\pm$10 | 276$\pm$13 | 1314$\pm$307
C1l | HLN14 | 20:14:25.627 | 41:13:36.620 | 10.2$\pm$1.2 | 79$\pm$10 | 288$\pm$7 | 713$\pm$87
C1m | HLN16a | 20:14:25.607 | 41:13:36.650 | 10.1$\pm$1.2 | 79$\pm$10 | 292$\pm$7 | 747$\pm$92
C1n | HLN16b | 20:14:25.594 | 41:13:36.866 | 8.2$\pm$0.8 | 64$\pm$6 | 323$\pm$5 | 954$\pm$88
C1o | HLN17a | 20:14:25.563 | 41:13:36.710 | 7.7$\pm$0.3 | 60$\pm$3 | 322$\pm$3 | 914$\pm$41
C1p | HLN17b | 20:14:25.548 | 41:13:36.860 | 5.9$\pm$0.8 | 46$\pm$6 | 303$\pm$7 | 1417$\pm$181
C1qu | HLN13 | 20:14:25.645 | 41:13:36.026 | 10.1$\pm$0.9 | 78$\pm$7 | 325$\pm$5 | 651$\pm$57
C1qd | HLN13 | 20:14:25.617 | 41:13:35.913 | 16.0$\pm$1.2 | 124$\pm$10 | 297$\pm$5 | 410$\pm$32
C1r | HLN13 | 20:14:25.671 | 41:13:36.170 | 6.3$\pm$1.2 | 49$\pm$10 | 315$\pm$11 | 1028$\pm$204
C2a | HLN18 | 20:14:25.514 | 41:13:36.752 | 5.8$\pm$0.6 | 45$\pm$5 | 312$\pm$5 | 1585$\pm$157
C2b | HLN19 | 20:14:25.464 | 41:13:36.771 | 12.9$\pm$0.5 | 100$\pm$4 | 287$\pm$2 | 735$\pm$29
C2ba | HLN19 | 20:14:25.475 | 41:13:36.838 | 8.8$\pm$1.2 | 69$\pm$10 | 300$\pm$8 | 1060$\pm$150
C2bb | HLN19 | 20:14:25.500 | 41:13:36.471 | 17.2$\pm$1.2 | 134$\pm$10 | 315$\pm$4 | 513$\pm$37
C2bc | HLN19 | 20:14:25.472 | 41:13:36.504 | 8.3$\pm$1.2 | 65$\pm$10 | 303$\pm$9 | 1113$\pm$167
C2bd | HLN19 | 20:14:25.490 | 41:13:36.690 | 11.5$\pm$1.2 | 90$\pm$10 | 283$\pm$6 | 891$\pm$96
C2c | HLN20 | 20:14:25.413 | 41:13:36.810 | 11.3$\pm$0.6 | 88$\pm$5 | 288$\pm$3 | 905$\pm$50
C2ca | HLN20 | 20:14:25.408 | 41:13:36.889 | 13.5$\pm$1.2 | 105$\pm$10 | 278$\pm$5 | 761$\pm$70
C2d | HLN20 | 20:14:25.401 | 41:13:36.580 | 12.3$\pm$0.4 | 95$\pm$3 | 292$\pm$3 | 836$\pm$30
C2da | HLN20 | 20:14:25.395 | 41:13:36.420 | 11.4$\pm$1.2 | 88$\pm$10 | 286$\pm$6 | 904$\pm$99
C2e | HLN21 | 20:14:25.362 | 41:13:36.570 | 7.2$\pm$0.3 | 56$\pm$2 | 284$\pm$3 | 1497$\pm$62
C2es | HLN21 | 20:14:25.356 | 41:13:36.438 | 11.9$\pm$1.0 | 92$\pm$8 | 293$\pm$4 | 911$\pm$74
C2f | HLN22 | 20:14:25.341 | 41:13:36.670 | 9.6$\pm$0.3 | 75$\pm$2 | 280$\pm$4 | 1156$\pm$32
C2fa | HLN22 | 20:14:25.335 | 41:13:36.420 | 13.1$\pm$1.2 | 102$\pm$10 | 264$\pm$5 | 847$\pm$81
C2g | HLN22 | 20:14:25.323 | 41:13:36.852 | 7.6$\pm$0.9 | 59$\pm$7 | 279$\pm$5 | 1512$\pm$175
C2ga | HLN22 | 20:14:25.325 | 41:13:36.730 | 6.9$\pm$0.9 | 53$\pm$7 | 303$\pm$7 | 1658$\pm$212
C2h1 | HLN22 | 20:14:25.323 | 41:13:37.110 | 11.6$\pm$0.8 | 90$\pm$6 | 280$\pm$4 | 996$\pm$65
C2h2 | HLN22 | 20:14:25.314 | 41:13:37.207 | 15.0$\pm$1.0 | 117$\pm$8 | 292$\pm$4 | 779$\pm$52
C2i | HLN23a | 20:14:25.288 | 41:13:37.435 | 11.1$\pm$1.1 | 86$\pm$9 | 296$\pm$11 | 1097$\pm$112
C2j | HLN23b | 20:14:25.259 | 41:13:37.310 | 10.1$\pm$0.4 | 78$\pm$3 | 293$\pm$2 | 1240$\pm$44
C2k | HLN23d | 20:14:25.230 | 41:13:37.348 | 14.0$\pm$1.1 | 109$\pm$8 | 293$\pm$4 | 923$\pm$72
C2ka | HLN23b | 20:14:25.245 | 41:13:37.250 | 8.0$\pm$1.2 | 62$\pm$10 | 283$\pm$9 | 1581$\pm$246
C2m | HLN25 | 20:14:25.118 | 41:13:38.690 | 12.6$\pm$1.0 | 98$\pm$8 | 297$\pm$4 | 1189$\pm$93
Table 6: continued. Table 7: Total velocity and inclination to the line of sight of the vectors’ velocity of the knots in IRAS20126+4104 (i.e. $i<90$ indicates a blue-shifted knot and $i>90$ indicates a red-shifted knot). Structure | knot | $\varv_{\rm tot}\pm{\rm d}\varv_{\rm tot}$ | i$\pm$di
---|---|---|---
| ID | (km s-1) | ($\degr$)
A1 | A1a | 45$\pm$3 | 93$\pm$3
A2 | A2a | 64$\pm$4 | 98$\pm$3
B | B2d | 101$\pm$6 | 82$\pm$2
C1 | C1hh | 134$\pm$10 | 82$\pm$3
C2 | C2b | 103$\pm$5 | 76$\pm$3


# Characterizations of open and semi-open maps of compact Hausdorff spaces by
induced maps

Xiongping Dai<EMAIL_ADDRESS>Yuxuan Xie<EMAIL_ADDRESS>Department of Mathematics, Nanjing University, Nanjing 210093, People’s
Republic of China

###### Abstract

Let $f\colon X\rightarrow Y$ be a continuous surjection of compact Hausdorff
spaces. By

  1. $f_{*}\colon\mathfrak{M}(X)\rightarrow\mathfrak{M}(Y),\ \mu\mapsto\mu\circ f^{-1}$ and $2^{f}\colon 2^{X}\rightarrow 2^{Y},\ A\mapsto f[A]$

we denote the induced continuous surjections on the probability measure spaces
and hyperspaces, respectively. In this paper we mainly show the following
facts:

  1. 1.

If $f_{*}$ is semi-open, then $f$ is semi-open.

  2. 2.

If $f$ is semi-open densely open, then $f_{*}$ is semi-open densely open.

  3. 3.

$f$ is open iff $2^{f}$ is open.

  4. 4.

$f$ is semi-open iff $2^{f}$ is semi-open.

  5. 5.

$f$ is irreducible iff $2^{f}$ is irreducible.

###### keywords:

Hyperspace, (densely, semi-)open map, irreducible/induced map, minimal flow

###### MSC:

[2010] 37B05, 54B20

††journal: Topology and its Applications (submitted 8 Feb 2024, accepted 18
Apr 2024)

Let $f\colon X\rightarrow Y$ be a continuous map from a topological space $X$
onto another $Y$. As usual, $f$ is open iff the image of every open subset of
$X$ is open in $Y$; $f$ is semi-open, or almost open, iff for every non-empty
open subset $U$ of $X$, the interior of $f[U]$, denoted $\mathrm{int}\,f[U]$,
is non-empty in $Y$. The “open” and “semi-open” properties are important for
the structure theory of compact minimal dynamics (see, e.g., [21, 20, 8, 9]).

By $\mathfrak{M}(X)$, it means the set of all regular Borel probability
measures on $X$ equipped with the weak-$*$ topology. Then, there exists a
naturally induced continuous surjective map:

  1. $f_{*}\colon\mathfrak{M}(X)\rightarrow\mathfrak{M}(Y)$, $\mu\mapsto\mu\circ f^{-1}$.

First of all there are equivalent descriptions of openness and semi-openness
of $f$ with the help of the induced map of $\mathfrak{M}(X)$ to
$\mathfrak{M}(Y)$ as follows:

######  Theorem A (Ditor-Eifler [7]).

Let $f\colon X\rightarrow Y$ be a continuous surjection of compact Hausdorff
spaces. Then $f$ is open iff the induced map
$f_{*}\colon\mathfrak{M}(X)\rightarrow\mathfrak{M}(Y)$ is an open surjection.

######  Theorem B′ (Glasner [8, 9]).

Let $f\colon X\rightarrow Y$ be a continuous surjection between compact metric
spaces. If $f$ is semi-open, then
$f_{*}\colon\mathfrak{M}(X)\rightarrow\mathfrak{M}(Y)$ is a semi-open
surjection.

In fact, it turns out that condition “$f$ is semi-open” is also necessary for
that “$f_{*}$ is semi-open” as follows:

######  Theorem B′′.

Let $f\colon X\rightarrow Y$ be a continuous surjection of compact Hausdorff
spaces. If the induced map
$f_{*}\colon\mathfrak{M}(X)\rightarrow\mathfrak{M}(Y)$ is semi-open, then $f$
is semi-open.

###### Proof.

Let $U\not=\emptyset$ be an open set in $X$. We shall prove that
$\mathrm{int}\,f[U]\not=\emptyset$ in $Y$. For that, let

  1. $\mathcal{U}=\\{\mu\in\mathfrak{M}(X)\,|\,U\cap\mathrm{supp}(\mu)\not=\emptyset\\}$.

Since $\mu\in\mathfrak{M}(X)\mapsto\mathrm{supp}(\mu)\in 2^{X}$ is lower semi-
continuous (cf. [20, Lem. VII.1.4]), $\mathcal{U}$ is open in
$\mathfrak{M}(X)$. As $f_{*}$ is semi-open by hypothesis, it follows that
there exists an open subset $\mathcal{V}$ of $\mathfrak{M}(Y)$ with
$\emptyset\not=\mathcal{V}\subseteq f_{*}[\mathcal{U}]$.

Because $\overline{\mathrm{co}}(\delta_{Y})=\mathfrak{M}(Y)$ where
$\delta_{Y}=\\{\delta_{y}\,|\,y\in Y\\}$ is the set of Dirac measures in $Y$,
we can choose a measure
$\nu=\sum_{i=1}^{n}\alpha_{i}\delta_{y_{i}}\in\mathcal{V}$ with $\alpha_{i}>0$
for $i=1,\dotsc,n$ and $\sum_{i}\alpha_{i}=1$. Further, we can choose an
$\varepsilon\in\mathscr{U}_{Y}$, the uniformity structure of $Y$, such that
for all
$(y_{1}^{\prime},\dotsc,y_{n}^{\prime})\in\varepsilon[y_{1}]\times\dotsm\times\varepsilon[y_{n}]$,
there exists an irreducible convex combination
$\nu^{\prime}=\alpha_{1}^{\prime}\delta_{y_{1}^{\prime}}+\dotsm+\alpha_{n}^{\prime}\delta_{y_{n}^{\prime}}\in\mathcal{V}$.
By $\mathcal{V}\subseteq f_{*}[\mathcal{U}]$, it follows that there is some
$\mu^{\prime}\in\mathcal{U}$ with $f_{*}(\mu^{\prime})=\nu^{\prime}$. As
$\mathrm{supp}(\nu^{\prime})=\\{y_{1}^{\prime},\dotsc,y_{n}^{\prime}\\}$,
$U\cap\mathrm{supp}(\mu^{\prime})\not=\emptyset$ and
$\mathrm{supp}(\mu^{\prime})\subseteq
f^{-1}[\mathrm{supp}(\nu^{\prime})]\subseteq\bigcup_{i=1}^{n}f^{-1}(y_{i}^{\prime})$,
it follows that

  1. $\\{y_{1}^{\prime},\dotsc,y_{n}^{\prime}\\}\cap f[U]\not=\emptyset$.

Clearly, this implies that $\varepsilon[y_{i}]\subseteq f[U]$ for some $i$
with $1\leq i\leq n$. Thus, $\mathrm{int}\,f[U]\not=\emptyset$. The proof is
completed. ∎

Consequently we have concluded the following theorem by a combination of
Theorem B′ and Theorem B′′:

######  Theorem B.

Let $f\colon X\rightarrow Y$ be a continuous surjection of compact metric
spaces. Then $f$ is semi-open iff
$f_{*}\colon\mathfrak{M}(X)\rightarrow\mathfrak{M}(Y)$ is semi-open.

On the other hand, recall that the (largest) hyperspace, denoted $2^{X}$, of
$X$ is defined to be the collection of all non-empty closed subsets of $X$
equipped with the Vietoris topology (see [20, $\S$II.1] or [13, Thm. I.1.2]).
Note here that a base for the Vietoris topology is formed by the sets of the
form

  1. $\langle U_{1},\dotsc,U_{n}\rangle:=\left\\{K\in 2^{X}\,|\,K\subseteq U_{1}\cup\dotsm\cup U_{n},K\cap U_{i}\not=\emptyset,1\leq i\leq n\right\\}$

for all $n\geq 1$ and all non-empty open sets $U_{1}$, $\dotsc$, $U_{n}$ in
$X$. Then $X$ is a compact Hausdorff space iff so is $2^{X}$, and $X$ is
metrizable iff $2^{X}$ is metrizable (cf. [19], [20, Thm. II.1.1], [13, Thm.
I.3.3]).

Since Kelley 1942 [15] the hyperspace theory became an important way of
obtaining information on the structure of a topological space $X$ (in
continua–compact connected metric spaces) by studying properties of the
hyperspace $2^{X}$ and its hyperspace $2^{2^{X}}$. In this note we shall give
other characterizations of open and semi-open maps with the help of the
hyperspaces $2^{X}$ and $2^{Y}$ (see Thm. 3 and Thm. 4). Moreover, we shall
consider the interrelation of the irreducibility of $f$ and its induced map
$2^{f}$ (see Thm. 9B), and improve Theorem B′ (see Thm. 10C).

######  1.

Let $X$, $Y$ be compact Hausdorff spaces. Let $\phi\colon X\rightarrow Y$ be a
continuous surjection. Then $\phi$ induces maps

  1. $2^{\phi}\colon 2^{X}\rightarrow 2^{Y}$ by $K\in 2^{X}\mapsto\phi[K]\in 2^{Y}$ and $\phi_{\mathrm{ad}}\colon 2^{Y}\rightarrow 2^{X}$ by $K\in 2^{Y}\mapsto\phi^{-1}[K]\in 2^{X}$.

Then, $2^{\phi}$ is a continuous surjection. Moreover, $\phi_{\mathrm{ad}}$ is
continuous iff $\phi_{\mathrm{ad}}|_{Y}=\phi^{-1}$ is continuous where $Y$ is
identified with $\\{\\{y\\}\,|\,y\in Y\\}\subset 2^{Y}$ by Lemma below, iff
$\phi$ is open; see [19] and [20, Thm. II.1.3].

######  Lemma (cf. [20, Rem. II.1.4]).

Let $X$ be a compact Hausdorff space and let $n\geq 1$ be an integer. Then the
map

  1. $i_{n}\colon X^{n}\rightarrow 2^{X}$ defined by $(x_{1},\dotsc,x_{n})\mapsto\\{x_{1},\dotsc,x_{n}\\}$

is continuous. Moreover, it is locally 1-1 in the points
$(x_{1},\dotsc,x_{n})$ with $x_{i}\not=x_{j}$ for all $i\not=j$. Also note
that $\bigcup\\{i_{n}[X^{n}]\,|\,n\in\mathbb{N}\\}$ is dense in $2^{X}$.

It is natural to wonder about which properties are transmitted between $\phi$
and $2^{\phi}$. This problem has been addressed by several authors (cf., e.g.,
[19, 6, 5]). We shall be concerned with the “openness” and “semi-openness”
here.

######  2 Lemma.

Let $\phi\colon X\rightarrow Y$ be a continuous surjection between Hausdorff
spaces with $X$ locally compact. Then:

  1. 1.

$\phi$ is semi-open iff the preimage of every dense subset of $Y$ is dense in
$X$ (cf. [9, Lem. 2.1]).

  2. 2.

$\phi$ is semi-open iff the preimage of every residual subset of $Y$ is
residual in $X$.

###### Proof.

(a): If $\phi$ is semi-open and let $A\subset Y$ be a dense set and set
$U=X\setminus\overline{\phi^{-1}[A]}$, then $\phi[U]\cap A\not=\emptyset$
whenever $U\not=\emptyset$, a contradiction. Thus, $U=\emptyset$ so that
$\phi^{-1}[A]$ is dense in $X$. Conversely, suppose the preimage of every
dense subset of $Y$ is dense in $X$. Let $U\not=\emptyset$ be open in $X$. If
$\textrm{int}\,\phi[U]=\emptyset$, then $\phi^{-1}[Y\setminus\phi[U]]\cap
U\not=\emptyset$, a contradiction. Thus,
$\textrm{int}\,\phi[U]\not=\emptyset$.

(b)-Necessity: Let $A=\bigcap_{i=1}^{\infty}A_{i}$, where $A_{i}$,
$i=1,2,\dots$ are open dense subsets of $Y$. Then, obviously,
$\phi^{-1}[A]=\bigcap_{i}\phi^{-1}[A_{i}]$. It follows from (a) that
$\phi^{-1}[A_{i}]$ are open dense in $X$. Thus, $\phi^{-1}[A]$ is a residual
subset of $X$.

(b)-Sufficiency: Let $U\not=\emptyset$ be open in $X$. If
$\textrm{int}\,\phi[U]=\emptyset$, then there exists an open set $V$ with
$\emptyset\not=V\subseteq\bar{V}\subseteq U$ such that $\bar{V}$ is compact in
$X$. Further,

  1. $\phi^{-1}[Y\setminus\phi[\bar{V}]]\cap\bar{V}\not=\emptyset$ and $\phi[\bar{V}]\cap(Y\setminus\phi[\bar{V}])\not=\emptyset$,

a contradiction. Thus, $\mathrm{int}\,\phi[U]\not=\emptyset$. The proof is
completed. ∎

######  3 Theorem (cf. [11, Thm. 4.3] for $X,Y$ in continua).

Let $f\colon X\rightarrow Y$ be a continuous surjection between compact
Hausdorff spaces. Then $f$ is open iff $2^{f}\colon 2^{X}\rightarrow 2^{Y}$ is
open.

###### Proof.

Necessity: Let $\mathcal{A}$ be a member of the base of the hyperspace
$2^{X}$. Then by definition of the Vietoris topology, it follows that there
exist nonempty open sets $U_{1}$, $\dotsc$, $U_{n}$ in $X$ such that

  1. $\mathcal{A}=\\{K\in 2^{X}\,|\,K\subseteq U_{1}\cup\dotsm\cup U_{n},K\cap U_{i}\not=\emptyset\textrm{ for }1\leq i\leq n\\}$.

Let $U=U_{1}\cup\dotsm\cup U_{n}$. Since $f$ is open, $f[U]$ and $f[U_{i}]$,
$1\leq i\leq n$, are all open subsets of $Y$. Let

  1. $\mathcal{B}=\\{K\in 2^{Y}\,|\,K\subseteq f[U],K\cap f[U_{i}]\not=\emptyset\textrm{ for }1\leq i\leq n\\}$.

Clearly, $2^{f}[\mathcal{A}]\subseteq\mathcal{B}$. In order to prove that
$2^{f}$ is open, it suffices to prove that $2^{f}[\mathcal{A}]=\mathcal{B}$.
For that, we need only prove $\mathcal{B}\subseteq 2^{f}[\mathcal{A}]$.

Let $B\in\mathcal{B}$ be arbitrarily given. By definitions, there exist points
$y_{i}\in B\cap f[U_{i}]$ for $i=1,\dotsc,n$. So we can select points
$x_{i}\in U_{i}$ with $f(x_{i})=y_{i}$ for $1\leq i\leq n$. Moreover, as $f$
is open and $X,Y$ compact, it follows that there exists a closed set
$A^{\prime}\in 2^{X}$ with $A^{\prime}\subseteq U$ such that
$f[A^{\prime}]=B$. Let

  1. $A=A^{\prime}\cup\\{x_{1},\dotsc,x_{n}\\}$.

Then $A\in 2^{X}$ such that $A\subseteq U$ and $x_{i}\in A\cap
U_{i}\not=\emptyset$ for $i=1,\dotsc,n$. Thus, $A\in\mathcal{A}$ and then
$2^{f}[\mathcal{A}]=\mathcal{B}$.

Sufficiency: Let $V\subseteq X$ be an open nonempty set. We need prove that
$f[V]$ is open in $Y$. As

  1. $\langle V\rangle=\\{K\in 2^{X}\,|\,K\subseteq V\\}$

is an open subset of $2^{X}$ and $2^{f}\colon 2^{X}\rightarrow 2^{Y}$ is open,
it follows that $2^{f}[\langle V\rangle]\subseteq 2^{Y}$ is open. Since
$V=\bigcup\\{F\,|\,F\in\langle V\rangle\\}$, hence $f[V]=\bigcup\\{K\,|\,K\in
2^{f}[\langle V\rangle]\\}$. Given $y\in f[V]$, $\\{y\\}\in 2^{f}[\langle
V\rangle]$ and there exists an open neighborhood $\langle
V_{y,1},\dotsc,V_{y,n}\rangle$ of $\\{y\\}$ in $2^{Y}$ such that
$\\{y\\}\in\langle V_{y,1},\dotsc,V_{y,n}\rangle\subseteq 2^{f}[\langle
V\rangle]$. Then

  1. $f[V]\subseteq\bigcup_{y\in f[V]}(\langle V_{y,1},\dotsc,V_{y,n}\rangle)\subseteq\bigcup\\{K\,|\,K\in 2^{f}[\langle V\rangle]\\}=f[V]$.

Thus, $f[V]=\bigcup_{y\in f[V]}(\langle V_{y,1},\dotsc,V_{y,n}\rangle)$ is
open in $Y$, for each $\langle V_{y,1},\dotsc,V_{y,n}\rangle$ is open in $Y$.
The proof is completed. ∎

Let $f\colon X\rightarrow Y$ be a continuous surjection between compact
Hausdorff spaces. Now define

  1. $2^{X,f}=\\{A\in 2^{X}\,|\,\exists y\in Y\textrm{ s.t. }A\subseteq f^{-1}(y)\\}$,

which is called the quasifactor representation of $Y$ in $X$; and, as
$i_{1}\colon Y\rightarrow 2^{Y}$ is an embedding (see Lem. 1), we may identify
$Y$ with $i_{1}[Y]=\\{\\{y\\}\,|\,y\in Y\\}\subseteq 2^{Y}$ as mentioned
before. Thus,

  1. $f^{\prime}=2^{f}|_{2^{X,f}}\colon 2^{X,f}\rightarrow Y$.

is a well-defined continuous surjection, called the quasifactor of $f$. It is
of interest to know when $f^{\prime}$ is actually a factor of $f$. Then
Theorem 3 has an interesting variation as follows:

######  3′. Theorem.

Let $f\colon X\rightarrow Y$ be a continuous surjection between compact
Hausdorff spaces. Then $f$ is open iff $f^{\prime}$ is open iff $f_{*}$ is
open.

###### Proof.

In view of Theorem A we need only prove the first “iff”. Suppose $f$ is open.
Then $2^{f}$ is open by Theorem 3. Set $\mathcal{Y}=\\{\\{y\\}\,|\,y\in Y\\}$.
Then $2^{X,f}=(2^{f})^{-1}[\mathcal{Y}]$. Thus, $f^{\prime}$ is open. Now
conversely, if $f^{\prime}\colon 2^{X,f}\rightarrow Y$ is open, then $f$ is
obviously open. The proof is completed. ∎

######  4 Theorem (cf. [12, Lem. 2.3] for “only if” part).

Let $f\colon X\rightarrow Y$ be a continuous surjection between compact
Hausdorff spaces. Then $f$ is semi-open iff $2^{f}\colon 2^{X}\rightarrow
2^{Y}$ is semi-open.

###### Proof.

Necessity (different with [12]): Let $\mathcal{B}\subseteq 2^{Y}$ be a dense
subset. By Lemma 2a, we need only prove that
$\mathcal{A}:=(2^{f})^{-1}[\mathcal{B}]$ is dense in $2^{X}$. For that, let
$\langle U_{1},\dotsc,U_{n}\rangle$ be a basic open set in $2^{X}$. It is
enough to prove that $\mathcal{A}\cap\langle
U_{1},\dotsc,U_{n}\rangle\not=\emptyset$. Indeed, since $f$ is semi-open,
hence

  1. $V_{i}:=\mathrm{int}\,f[U_{i}]\not=\emptyset$, $i=1,\dotsc,n$.

Let $V=V_{1}\cup\dotsm\cup V_{n}$, $U=U_{1}\cup\dotsm\cup U_{n}$. Then
$\langle V_{1},\dotsc,V_{n}\rangle$ is an open non-empty subset of $2^{Y}$. So
$\mathcal{B}\cap\langle V_{1},\dotsc,V_{n}\rangle\not=\emptyset$. Let
$B\in\mathcal{B}\cap\langle V_{1},\dotsc,V_{n}\rangle$. As $V\subseteq f[U]$
and $B\in 2^{Y}$ is compact, it follows that we can select $A\in 2^{X}$ with
$A\subseteq U$ such that $f[A]=B$. Moreover, by $B\cap V_{i}\not=\emptyset$,
we can take a point $x_{i}\in U_{i}$ with $f(x_{i})\in B$ for $i=1,\dotsc,n$.
Now set

  1. $K=A\cup\\{x_{1},\dotsc,x_{n}\\}$.

Then $K\subseteq U$, $K\cap U_{i}\not=\emptyset$ for all $i=1,\dotsc,n$ and
$f[K]=B$. Thus, $K\in\mathcal{A}$ and further $\mathcal{A}\cap\langle
U_{1},\dotsc,U_{n}\rangle\not=\emptyset$.

Sufficiency: Let $U\not=\emptyset$ be an open set in $X$. Since $\langle
U\rangle=\\{K\in 2^{X}\,|\,K\subseteq U\\}$ is open in $2^{X}$ and $2^{f}$ is
semi-open, it follows that $2^{f}[\langle U\rangle]=\\{f[K]\,|\,K\in\langle
U\rangle\\}$ has a non-empty interior. Therefore, there exists a non-empty
basic open subset of $2^{Y}$, say $\langle V_{1},\dotsc,V_{n}\rangle\subseteq
2^{f}[\langle U\rangle]$. Put

  1. $B=V_{1}\cup\dotsm\cup V_{n}$.

Then $B\not=\emptyset$ is open in $Y$ such that $B\subseteq f[U]$. For, as
$B\in\langle V_{1},\dotsc,V_{n}\rangle\subseteq 2^{f}[\langle U\rangle]$,
there exists $A\in\langle U\rangle$ such that $f[A]=B$; thus, $A\subseteq U$
implies $B\subseteq f[U]$. The proof is completed. ∎

Note here that we do not know whether or not Theorem 4 has a variation similar
to Theorem 3′. That is, we do not know how to characterize the semi-openness
of $f^{\prime}\colon 2^{X,f}\rightarrow Y$.

######  5.

A flow is a triple $(T,X,\pi)$, simply denoted $\mathscr{X}$ or
$T\curvearrowright X$ if no confusion, where $T$ is a topological group,
called the phase group; $X$ is a topological space, called the phase space;
and where $\pi\colon T\times X\xrightarrow{(t,x)\mapsto x}X$, the action, is a
jointly continuous map, such that $ex=x$ and $(st)x=s(tx)$ for all $x\in X$
and $s,t\in T$. Here $e$ is the identity of $T$.

Every flow $\mathscr{X}$ can induce a so-called hyperflow $(T,2^{X},2^{\pi})$
(cf. [16] or [20, Thm. II.1.6]), denoted $2^{\mathscr{X}}$, where the phase
group is $T$, the phase space is $2^{X}$ and the action is the induced mapping
$2^{\pi}\colon T\times 2^{X}\xrightarrow{(t,K)\mapsto tK}2^{X}$. Here
$tK=\\{tx\,|\,x\in K\\}$ for $t\in T$ and $K\in 2^{X}$.

If $\overline{Tx}=X\ \forall x\in X$, then $\mathscr{X}$ is called a minimal
flow. Let $\mathscr{X}$ be a compact flow; that is, $\mathscr{X}$ is a flow
with compact Hausdorff phase space $X$. If $x\in X$ such that $\overline{Tx}$
is a minimal subset of $\mathscr{X}$, then $x$ is refereed to as an almost
periodic (a.p.) point for $\mathscr{X}$. The induced affine flow
$T\curvearrowright\mathfrak{M}(X)$ and hyperflow $2^{\mathscr{X}}$ of a
compact minimal flow $\mathscr{X}$ are not minimal unless $X=\\{pt\\}$ a
singleton. So, the dynamics on $2^{\mathscr{X}}$ is more richer than that on
$\mathscr{X}$. See, e.g., [3, 10, 18, 1, 17, 14].

Let $\mathscr{X},\mathscr{Y}$ be two compact flows. Then
$\phi\colon\mathscr{X}\rightarrow\mathscr{Y}$ is called an extension if
$\phi\colon X\rightarrow Y$ is a continuous surjection such that
$\phi(tx)=t\phi(x)$ for all $t\in T$ and $x\in X$. If
$\phi\colon\mathscr{X}\rightarrow\mathscr{Y}$ is an extension of compact
flows, then $2^{\phi}\colon 2^{\mathscr{X}}\rightarrow 2^{\mathscr{Y}}$ is an
extension of compact hyperflows (see [20, Thm. II.1.8]).

######  6 Lemma (cf. [4, Lem. 3.12.15] or [20, Thm. I.1.4]).

If $\phi\colon\mathscr{X}\rightarrow\mathscr{Y}$ is an extension of compact
flows with $\mathscr{X}$ having a dense set of a.p. points and $\mathscr{Y}$
minimal, then $\phi$ is semi-open.

######  6A.

In Definition 5, if $T$ is only a topological semigroup, then $\mathscr{X}$ or
$T\curvearrowright X$ is refereed to as a semiflow. In this case, $x\in X$ is
called a.p. for $\mathscr{X}$ iff $x\in\overline{Tx}$ and $\overline{Tx}$ is a
minimal subset of $\mathscr{X}$. In fact, it is not known whether or not Lemma
6 is still true if $T$ is only a semigroup. See Theorem 6C for a confirmative
conditional case.

######  6B.

Let $\phi\colon\mathscr{X}\rightarrow\mathscr{Y}$ be an extension of compact
semiflows. We say that $\phi$ is highly proximal (h.p.) iff, for all $y\in Y$,
there is a net $t_{n}\in T$ with $t_{n}\phi^{-1}(y)\to\\{pt\\}$ in $2^{X}$
(cf. [20, p.104]).

######  6C. Theorem.

Let $\phi\colon\mathscr{X}\rightarrow\mathscr{Y}$ be an extension of compact
semiflows such that $t\phi^{-1}(y)=\phi^{-1}(ty)$ for all $t\in T,y\in Y$. If
$\phi$ is h.p. and $\mathscr{X}$ has a dense set of a.p. points, then $\phi$
is semi-open.

###### Proof.

Firstly we claim that every nonempty open subset of $X$ contains a
$\phi$-fiber. Indeed, let $U\subset X$ be open with $U\not=\emptyset$. Then
there is an a.p. point $x_{0}\in U$. Since $\phi$ is h.p., there is a net
$t_{n}\in T$ such that $t_{n}\phi^{-1}\phi(x_{0})\to\\{x^{\prime}\\}$, with
$x^{\prime}=\lim t_{n}x_{0}$, in $2^{X}$. As $x_{0}$ is a.p., it follows that
there is a net $s_{j}\in T$ with $s_{j}x^{\prime}\to x_{0}$. Thus, there is a
net $\tau_{i}\in T$ such that $\tau_{i}\phi^{-1}\phi(x_{0})\to L\subseteq U$,
and so that $\phi^{-1}\phi(\tau_{i}x_{0})\subseteq U$ eventually.

Now to prove that $\phi$ is semi-open, let $U\subset X$ be open with
$U\not=\emptyset$. By our claim above, there exists a point $y_{0}\in Y$ such
that $\phi^{-1}(y_{0})\subseteq U$. Since $\phi_{\textrm{ad}}|_{Y}\colon
Y\rightarrow 2^{X}$, $y\mapsto\phi^{-1}(y)$ is upper semi-continuous and $U$
is open, there is an open neighborhood $V$ of $y_{0}$ in $Y$ such that
$\phi^{-1}[V]\subseteq U$. Thus, $\phi[U]\supseteq V$ so that
$\textrm{int}\,\phi[U]\not=\emptyset$. The proof is completed. ∎

Therefore, if $\phi\colon\mathscr{X}\rightarrow\mathscr{Y}$ is an h.p.
extension of compact flows with $\mathscr{X}$ having a dense set of a.p.
points, then $\phi$ is semi-open. A point of Theorem 6C is that $\mathscr{Y}$
need be minimal here.

As was mentioned before, $2^{\mathscr{X}}$ need not have a dense set of a.p.
points and $2^{\mathscr{Y}}$ are generally not minimal, so Lemma 6 is not
applicable straightforwardly for $2^{\phi}\colon 2^{\mathscr{X}}\rightarrow
2^{\mathscr{Y}}$. However, using Theorem 4 we can obtain the following:

######  7 Corollary.

Let $\phi\colon\mathscr{X}\rightarrow\mathscr{Y}$ be an extension of compact
minimal flows. Then:

  1. 1.

$2^{\phi}\colon 2^{\mathscr{X}}\rightarrow 2^{\mathscr{Y}}$ and
$2^{2^{\phi}}\colon 2^{2^{\mathscr{X}}}\rightarrow 2^{2^{\mathscr{Y}}}$ both
are semi-open extensions of compact hyperflows.

  2. 2.

If $X$ and $Y$ both are compact metric spaces, then

    1. $\phi_{*}\colon\mathfrak{M}(X)\rightarrow\mathfrak{M}(Y)$ and $(\phi_{*})_{*}\colon T\curvearrowright\mathfrak{M}(\mathfrak{M}(X))\rightarrow T\curvearrowright\mathfrak{M}(\mathfrak{M}(Y))$

both are semi-open extensions of compact flows.

###### Proof.

By Lemma 6, Theorem 4 and Theorem B′. ∎

######  8 Corollary.

Let $\phi\colon\mathscr{X}\rightarrow\mathscr{Y}$ be an extension of compact
flows. Then $\phi$ is open iff $2^{\phi}\colon 2^{\mathscr{X}}\rightarrow
2^{\mathscr{Y}}$ is open iff $2^{2^{\phi}}\colon
2^{2^{\mathscr{X}}}\rightarrow 2^{2^{\mathscr{Y}}}$ is open.

###### Proof.

By Theorem 3. ∎

######  9Irreducibility of maps.

In what follows, let $\phi\colon X\rightarrow Y$ be a continuous surjection
between compact Hausdorff spaces. We say that $\phi$ is irreducible if the
only member $A\in 2^{X}$ with $\phi[A]=Y$ is $X$ itself. This notion is
closely related to “highly proximal” in extensions of minimal flows [20].

######  9A. Lemma.

$\phi$ is irreducible iff every non-empty open subset $U$ of $X$ contains a
fiber $\phi^{-1}(y)$ for some point $y\in Y$.

###### Proof.

It is straightforward. ∎

######  9B. Theorem.

$\phi$ is irreducible iff $2^{\phi}\colon 2^{X}\rightarrow 2^{Y}$ is
irreducible.

###### Proof.

Sufficiency: Assume $2^{\phi}$ is irreducible. Let $A\in 2^{X}$ with
$\phi[A]=Y$. Then $2^{A}\subseteq 2^{X}$ is a closed set such that
$2^{\phi}[2^{A}]=2^{Y}$. Thus, $2^{A}=2^{X}$ and $A=X$. This shows that $\phi$
is irreducible.

Necessity: Suppose $\phi$ is irreducible. Let $\langle
U_{1},\dotsc,U_{n}\rangle$ be a basic open set in $2^{X}$. In view of Lemma
9A, it is sufficient to prove that $\langle U_{1},\dotsc,U_{n}\rangle$
includes a fiber of $2^{\phi}$. Indeed, as $\phi$ is irreducible, it follows
by Lemma 9A that there is a point $y_{i}\in\phi[U_{i}]$ with
$\phi^{-1}(y_{i})\subseteq U_{i}$ for all $i=1,\dotsc,n$. Let
$B=\\{y_{1},\dotsc,y_{n}\\}\in 2^{Y}$. Then
$(2^{\phi})^{-1}(B)\subseteq\langle U_{1},\dotsc,U_{n}\rangle$. Thus,
$2^{\phi}$ is irreducible. The proof is completed. ∎

######  10.

Let $f\colon X\rightarrow Y$ be a semi-open continuous surjection of compact
Hausdorff spaces, where $X$ is not metrizable. In view of Theorem B′′ we
naturally wonder whether or not
$f_{*}\colon\mathfrak{M}(X)\rightarrow\mathfrak{M}(Y)$ is semi-open. The
following lemma seems to be helpful for the this question, which is of
interest independently.

######  10A. Lemma.

Let $f\colon X\rightarrow Y$ be a continuous surjection between compact
Hausdorff spaces, where $X$ is not necessarily metrizable. Let $\psi\colon
X\rightarrow\mathbb{R}$ be a continuous function and define functions

  1. $\psi^{*}\colon Y\xrightarrow{y\mapsto\psi^{*}(y)=\sup_{x\in f^{-1}(y)}\psi(x)}\mathbb{R}$ and $\psi_{*}\colon Y\xrightarrow{y\mapsto\psi_{*}(y)=\inf_{x\in f^{-1}(y)}\psi(x)}\mathbb{R}$.

Then there is a residual set $Y_{c}(\psi)\subseteq Y$ such that $\psi^{*}$ and
$\psi_{*}$ are continuous at each point of $Y_{c}(\psi)$.

###### Proof.

Let $\rho$ be a continuous pseudo-metric on $X$ and let $\tilde{\rho}$ be the
naturally induced one on $2^{X}$. Since $f^{-1}\colon Y\rightarrow 2^{X}$,
defined by $y\mapsto f^{-1}y$, is upper semi-continuous, hence $f^{-1}\colon
Y\rightarrow(2^{X},\tilde{\rho})$ is also upper semi-continuous. Thus, there
exists a residual set $Y_{\rho}\subseteq Y$ such that $f^{-1}$ is continuous
at every point of $Y_{\rho}$. Now, for every $\varepsilon>0$, there exists a
continuous pseudo-metric $\rho$ on $X$ and a positive $r>0$ such that if
$x,x^{\prime}\in X$ with $\rho(x,x^{\prime})<r$, then
$|\psi(x)-\psi(x^{\prime})|<\varepsilon/3$. Then there exists a residual set
$Y_{\varepsilon}=Y_{\rho}\subseteq Y$ such that for every $y\in
Y_{\varepsilon}$, we have that
$|\psi^{*}(y)-\psi^{*}(y^{\prime})|+|\psi_{*}(y)-\psi_{*}(y^{\prime})|<\varepsilon$
as $y^{\prime}\in Y$ close sufficiently to $y$. Let
$Y_{c}=\bigcap_{n=1}^{\infty}Y_{1/n}$. Clearly, $Y_{c}$ is a residual subset
of $Y$ as desired. The proof is completed. ∎

######  10B (Densely open mappings).

Let $f\colon X\rightarrow Y$ be a continuous surjection between compact
Hausdorff spaces. We say that $f$ is densely open if there exists a dense set
$Y_{o}\subseteq Y$ such that $f^{-1}\colon Y\rightarrow 2^{X}$ is continuous
at each point of $Y_{o}$. For example, if $X$ is a separable metric space,
then $f$ is always densely open. We notice here that the “densely open” is
different with “almost open” considered in [12, Def. 1.5] and [2].

######  10C. Theorem.

Let $f\colon X\rightarrow Y$ be a continuous surjection between compact
Hausdorff spaces. Then:

  1. 1.

If $f$ is semi-open densely open, then $f_{*}$ is semi-open densely open.

  2. 2.

If $f$ is densely open, then $f_{*}$ is densely open.

###### Proof.

Based on Def. 10B, let $Y_{o}$ be the dense set of points of $Y$ at which the
set-valued map $f^{-1}\colon Y\rightarrow 2^{X}$ is continuous. Let
$\delta_{Y}=\\{\delta_{y}\,|\,y\in Y\\}$.

(1): As $f$ is semi-open, it follows from Lemma 2 that $X_{o}:=f^{-1}[Y_{o}]$
is dense in $X$. In view of (2), we need only prove that $f_{*}$ is semi-open.
For that let $\mathcal{U}\subset\mathfrak{M}(X)$ be a closed set with non-
empty interior. We need only show that
$\mathrm{int}\,f_{*}[\mathcal{U}]\not=\emptyset$. Suppose to the contrary that
$\mathrm{int}\,f_{*}[\mathcal{U}]=\emptyset$. We now fix a measure

  1. $\mu_{0}=\sum_{i=1}^{m}c_{i}\delta_{x_{i}}\in\mathrm{int}\,\mathcal{U}$ with $x_{i}\in X_{o}$, $0<c_{i}\leq 1$, $\sum_{i}c_{i}=1$.

Set

  1. $\nu_{0}=f_{*}(\mu_{0})=\sum_{i=1}^{m}c_{i}\delta_{y_{i}}$ where $y_{i}=f(x_{i})\in Y_{o}$.

Let

  1. $\nu_{j}=\sum_{i=1}^{m_{j}}c_{j,i}\delta_{y_{j,i}}\in\mathrm{co}(\delta_{Y})\setminus f_{*}[\mathcal{U}]$ be a net with $\nu_{j}\to\nu_{0}$.

So $\nu_{j}\to\nu_{0}$. Each $Q_{j}$ is a closed convex subset of
$\mathfrak{M}(X)$ and, with no loss of generality, we may assume that $Q=\lim
Q_{j}$ exists in $2^{\mathfrak{M}(X)}$. Then $Q$ is a compact convex subset of
$\mathfrak{M}(X)$ with $f_{*}[Q]=\\{\nu_{0}\\}$.

If $\mu_{0}\in Q$, then $Q_{j}\cap\mathcal{U}\not=\emptyset$ eventually so
that $\nu_{j}\in f_{*}[\mathcal{U}]$, contradicting our choice of $\nu_{j}$.
Thus, $\mu_{0}\notin Q$ and by the Separation Theorem there exists a $\psi\in
C(X)$ and $\epsilon>0$ such that

  1. $\mu_{0}(\psi)\geq q(\psi)+\epsilon\ \forall q\in Q$.

Define the associated function

  1. $\psi^{*}\colon Y\rightarrow\mathbb{R}$, by $y\mapsto\psi(y)=\sup\\{\psi(x)\,|\,x\in f^{-1}y\\}$, such that $\psi^{*}\circ f\geq\psi$.

We can choose points $x_{j,i}\in X$ with $f(x_{j,i})=y_{j,i}$ and
$\psi^{*}(y_{j,i})=\psi(x_{j,i})$. Now form the measures
$\mu_{j}=\sum\limits_{i=1}^{m_{j}}c_{j,i}\delta_{x_{j,i}}$ and assume, with no
loss of generality, that $\mu=\lim_{j}\mu_{j}$ exists in $\mathfrak{M}(X)$.
Since $\mu_{j}\in Q_{j}$ for each $j$, we have $\mu\in Q$. Thus,
$\mu_{0}(\psi)\geq\mu(\psi)+\epsilon$. By our construction
$\nu_{j}(\psi^{*})=\mu_{j}(\psi)$ for every $j$. By assumption we have that
$\textrm{supp}(\nu_{0})=\\{y_{1},\dotsc,y_{m}\\}$ is a subset of $Y_{o}$ and
therefore, each $y_{i}$ is a continuity point of $\psi^{*}$. Thus,
$\lim\nu_{j}(\psi^{*})=\nu_{0}(\psi^{*})$. It then follows that

$\mu(\psi)=\lim_{j}\mu_{j}(\psi)=\lim_{j}\nu_{j}(\psi^{*})=\nu_{0}(\psi^{*})=\sum_{i=1}^{m}c_{i}\psi^{*}(y_{i})\geq\sum_{i=1}^{m}c_{i}\psi(x_{i})=\mu_{0}(\psi).$

This contradicts the choice of $\psi$. Therefore $f_{*}$ is semi-open.

(2): We will now prove that $f_{*}$ is densely open. Suppose to the contrary
that $f_{*}$ is not densely open. Then there exists an open set
$\mathcal{V}\not=\emptyset$ in $\mathfrak{M}(Y)$ such that
$f_{*}^{-1}\colon\mathfrak{M}(Y)\rightarrow 2^{\mathfrak{M}(X)}$ is not
continuous at every point of $\mathcal{V}$. Since
$\mathrm{co}(\delta_{Y_{o}})$ is dense in $\mathfrak{M}(Y)$, there exists a
measure of the form

  1. $\nu_{o}=\sum_{i=1}^{m}c_{i}\delta_{y_{i}}$, where $y_{i}\in Y_{o}$ and $0\leq c_{i}\leq 1$ with $\sum c_{i}=1$,

such that $\nu_{0}\in\mathcal{V}$. This implies that there is a measure

  1. $\mu_{o}\in f_{*}^{-1}(\nu_{o})$ such that there is $\mathcal{U}\in\mathfrak{N}_{\mu_{o}}(\mathfrak{M}(X))$ with $f_{*}[\mathcal{U}]\notin\mathfrak{N}_{\nu_{o}}(\mathfrak{M}(Y))$.

Then we can select a net
$\nu_{j}=\sum_{i=1}^{m_{j}}c_{j,i}\delta_{y_{j,i}}\in\mathrm{co}(\delta_{Y_{o}})\setminus
f_{*}[\mathcal{U}]$ such that $\nu_{j}\to\nu_{o}$. Next by an argument as in
(1) we can reach a contradiction. The proof is completed. ∎

## Acknowledgements

This work was supported by National Natural Science Foundation of China (Grant
No. 12271245) and PAPD of Jiangsu Higher Education Institutions.

## References

  * [1] E. Akin, J. Auslander and A. Nagar,  Dynamics of induced systems,  Ergod. Th. $\&$ Dynam. Sys., 37 (2017), 2034-2059.
  * [2] A. Andrade and J. Camargo,  A note on semi-open and almost open induced maps,  Topology Appl., 302 (2021), No. 107823, 6 pp.
  * [3] W. Bauer and K. Sigmund,  Topological dynamics of transformations induced on space of probability measures,  Monatsh. Math., 79 (1975), 81–92.
  * [4] I. U. Bronšteǐn,  Extensions of Minimal Transformation Groups,  Sijthoff $\&$ Noordhoff, 1975/79.
  * [5] J. J. Charatonik,  Recent research in hyperspace theory,  Extracta Math., 18 (2003), 235-262.
  * [6] J. J. Charatonik, A. Illanes and S. Macías,  Induced mappings on hyperspace $C_{n}(X)$ of a continuum $X$,  Houston J. Math., 28 (2002), 781-805.
  * [7] S. Z. Ditor and L. Q. Eifler,  Some open mapping theorems for measures,  Trans. Amer. Math. Soc., 164 (1972), 287-293.
  * [8] E. Glasner,  The structure of tame minimal dynamical systems,  Ergod. Th. $\&$ Dynam. Sys., 27 (2007), 1819-1837.
  * [9] E. Glasner,  The structure of tame minimal dynamical systems for general groups,  Invent. Math., 211 (2018), 213–244.
  * [10] E. Glasner and B. Weiss,  Quasi-factors of zero-entropy systems,  J. Amer. Math. Soc., 8 (1995), 655-686.
  * [11] H. Hosokawa,  Induced mappings on hyperspaces,  Tsukuba J. Math., 21 (1997), 239–250.
  * [12] X. Huang, F. Zeng and G. Zhang,  Semi-openness and almost-openness of induced mappings,  Appl. Math. J. Chinese Univ. Ser. B, 20 (2005), 21–26.
  * [13] A. Illanes and S. B. Nadler, Jr.,  Hyperspaces, Fundamentals and Recent Advances,  New York, Marcel Dekker Inc, 1999.
  * [14] D. Jelić and P. Oprocha,  On recurrence and entropy in the hyperspace of continua in dimension one,  Fund. Math., 263 (2023), 23–50.
  * [15] J. L. Kelley,  Hyperspaces of a continuum,  Trans. Amer. Math. Soc., 52 (1942), 22–36.
  * [16] S. C. Koo,  Recursive properties of transformations groups in hyperspaces,  Math. Systems Theory, 9 (1975), 75–82.
  * [17] J. Li, P. Oprocha, X. Ye and R. Zhang,  When are all closed subsets recurrent?,  Ergod. Th. $\&$ Dynam. Sys., 37 (2017), 2223-2254.
  * [18] J. Li, K. Yan and X. Ye,  Recurrence properties and disjointness on the induced spaces,  Discrete Contin. Dyn. Syst., 35 (2015), 1059-1073.
  * [19] E. Michael,  Topologies on spaces of subsets,  Trans. Amer. Math. Soc., 71 (1951), 152–182.
  * [20] J. C. S. P. van der Woude,  Topological Dynamix,  CWI Tracts No. 22, Centre for Mathematics and Computer Science (CWI), Amsterdan, 1986.
  * [21] W. A. Veech,  Topological dynamics,  Bull. Amer. Math. Soc., 83 (1977), 775–830.