# GroupGazer: A Tool to Compute the Gaze per Participant in Groups with
integrated Calibration to Map the Gaze Online to a Screen or Beamer Projection

Wolfgang Fuhl, Daniel Weber, Shahram Eivazi
Department of Human Computer Interaction
University Tübingen
Tübingen, 72076
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>

###### Abstract

In this paper we present GroupGaze. It is a tool that can be used to calculate
the gaze direction and the gaze position of whole groups. GroupGazer
calculates the gaze direction of every single person in the image and allows
to map these gaze vectors to a projection like a projector. In addition to the
person-specific gaze direction, the person affiliation of each gaze vector is
stored based on the position in the image. Also, it is possible to save the
group attention after a calibration. The software is free to use and requires
a simple webcam as well as an NVIDIA GPU and the operating system Windows or
Linux.
https://es-cloud.cs.uni-
tuebingen.de/d/8e2ab8c3fdd444e1a135/?p=%2FGroupGazer&mode=list.

##  1 Introduction

Figure 1: Detection results of the proposed tool GroupGazer on a group photo.

Eye tracking is an important input modality and information source in the
modern world [12, 39, 19, 40, 42]. The signal itself is usually computed by
first detecting the pupil [43, 55, 58, 53, 52, 30] and additional eye features
[37, 28, 46, 24]. Based on the gaze signal it is possible to extract further
information like the eye movement types [44, 54, 21, 22, 31, 34]. In the field
of human-machine interaction [20], the gaze signal is used and further
researched for interaction with robots [112] but also other technical devices
[111, 18, 23, 45]. This involves not only simple control but also
collaboration in which a human communicates complex behavior to a robot or
system [95]. Interaction with the eyes based on pupil movements [56, 41, 29,
27, 25, 57] is also an interesting source of information in the field of
computer games [2]. Eye interaction is many times faster than mouse
interaction, which could revolutionize the professional computer gaming field
[73]. In the field of virtual reality, gaze information can be used to render
only small areas of the scene in high resolution, leading to a significant
reduction in the resources consumption of the devices [91]. Another important
area in which the gaze signal plays an important role is driver observation.
Here it is necessary to assess whether the driver is able to control the
vehicle or is too tired in the case of autonomous driving to take over the
vehicle [118]. Of course, this also applies to car rental companies, for which
it is important to know whether the driver is, for example, intoxicated or an
unsafe driver [88, 49, 48, 50, 26, 51]. In the field of medicine, research is
also being conducted into methods of self-diagnosis [10]. This involves, for
example, the early detection of Alzheimer’s disease [13], strokes [87], as
well as eye defects [16] or autism [6]. In the field of safety, the eye signal
also gains increasingly more interest [75, 60]. This is due to the fact that
personal behavior is reflected in the gaze signal, which can be used to
identify the person [47, 60]. Other information contained in the eye is the
cognitive load based on pupil dilation [7], attention [9], procedural
strategies [71] and many others. A relatively new area in which the eye
tracking signal is used is behavioral research [117, 14]. Here it is on the
one hand about extracting expert knowledge from the eye signal and passing
this knowledge to trainees [85, 65]. This concerns, all areas in which the
training is only possible with expensive tools and training devices [109]. In
the area of medicine the main interest is to distill the expert knowledge
better [96, 85]. Another area of behavioral research which is also the subject
of this thesis is group behavior [69, 100, 81]. Here there is research in the
area of teaching [79, 105, 70] but also in dynamic environments like sports
[94, 15, 102].

The current problems in the field of behavioral research for groups, is that
there is no freely available software for this. Therefore, research groups
have to resort to expensive solutions such as multiple worn eye trackers. This
creates further issues like the assignment of the important areas between the
different scene cameras. One way around this is to use virtual reality
together with eye tracking. However, this also changes the behavior of the
test persons and cannot be carried out over longer periods of time with regard
to motion sickness. Alternatively to worn eye trackers, there is also the
possibility to use external cameras. In this case, the researchers have to
implement their technical solutions independently, which often leads to
dependencies on other working groups and is also an expensive undertaking due
to the image processing cameras which are usually used.

In this paper, we present software that allows anyone to use a simple webcam
for gaze estimation of groups and calibration each subject in parallel. By
doing so, we hope to enable anyone to conduct behavioral group research. Our
contributions to the state of the art are:

  1. 1.

A tool to record the gaze of groups and calibrate each individual in parallel.

  2. 2.

The tool has no specialized hardware requirements and only needs an NVIDIA GPU
with at least 4 GB memory (We used a 1050 ti with 4 GB).

  3. 3.

Stores the gaze per person as well as the average gaze location of the group.

##  2 Related work

Since our software is the combination of several research fields, we have
divided the related work into three categories. The first category is face
recognition, the second category is appearance based gaze estimation, and the
third category is gaze based group behavior research.

###  2.1 Face detection

Face recognition in arbitrary environments is still a very challenging field
of research. Here, an arbitrarily large image is given, and all faces must be
detected. This often involves occlusions, different head positions, changes in
lighting conditions, and of course the faces in the image have different
resolutions. The first very successful approach was presented by Viola and
Jones [110]. This is based on hair features and trained using AdaBoost. The
next major step was achieved with deformable part models (DPM) [116]. Compared
to feature-based approaches, DPM is much more robust but requires
significantly more computational effort. With the advent of deep neural
networks, however, the state of the art was again significantly improved [4,
72, 82, 122]. The first extension of neural networks was the combination of
face detection with face matching [119, 120]. Current methods for face
detection follow two directions. The first direction is the multistage
approach, which is based on a region proposal neural network followed by
validation of the proposed faces. The most notable representatives of this
direction of development are RCNN [63], almost RCNN [62], and faster RCNN
[101]. The second direction of development is direct methods such as single
shot multibox detector (SSD) [83] or you only look once (YOLO) [99]. For YOLO,
there are already multiple versions, which consume even less resources at
approximately the same detection rate. The advantage of the direct approaches,
is the faster execution and the smaller resource consumption. The multilayer
methods, on the other hand, provide a better detection rate and fewer
misclassifications. Further ressource consumption approaches have also been
proposed in the literature [38, 33, 35].

###  2.2 Appearance based gaze estimation

Here, the entire facial image or eye area of a person is used to directly
determine the gaze vector via a neural network. The first work in this area is
from 1994 [5] and was extended in [108] by linear projection functions. These
methods require very expensive calibration, since the neural network was
trained for each person individually with many training examples. The first
extensions to reduce the effort in calibration were a Gaussian process
regression [113], saliency maps [107], and optimal selection using a linear
regression [84]. While all of these methods advanced the state of the art, the
appearance based approach still had many limitations, such as a fixed head
position and per-person calibration. With deep neural networks and the advent
of big data, this has changed significantly. In [121] the first successful
approach was presented, which realized appearance based gaze estimation with
deep neural networks. The first extensions used, in addition to eye images,
the subjects’ faces, which resulted in a significant improvement [80, 76]. For
extreme head positions and strongly deviating gaze angles to the head
orientation, an asymmetric regression was presented [8].

###  2.3 Gaze based group behavior research

In this section, we would like to mention and briefly explain only some works
from this area, since our software is made for this purpose but does not
perform a behavior research study.

The first area in behavioral research which can also be applied to groups is
mind wandering [68, 67]. Mind wandering is a shift in attention to task-
unrelated thoughts. This is an interesting effect for teaching since it
negatively influences the learning performance of students [103, 106, 67].
Mind wandering itself is a special form of disengagement and has to be
separated from boredom or off-task behaviors [11, 92, 67]. Another interesting
social behavior is gaze following [3]. This gaze following is a form of
communication and socializing. In some scenarios it has to be done only for a
single person [74, 17] but in modern research entire scenes with multiple
persons are evaluated and analyzed [3, 93, 86, 97, 98]. Nowadays,
psychologists use behavior observation methods in classrooms as well as direct
behavior ratings [114]. While both methods are valid and also used by teachers
themselves, they are limited in effectiveness due to the attentional limits of
the human observers as well as their induced biases [89]. Modern research
focuses on establishing intelligent classroom technologies with eye tracking
and voice recording [114, 89]. Those methods have their limitations due to the
data security but deliver more insights and allow reducing the induced bias by
humans [114, 89, 90].

##  3 Method

Figure 2: The workflow of our approach. We first detect the faces and compute the gaze vector using an appearance based approach. Each person is calibrated using mouse clicks on a projection in parallel. The fitted polynomials are afterwards used to map the gaze to the projection. Table 1: Shows the architecture of our face detection deep neural network. The architecture is copied from dlib [77] and uses the max margin [78] training procedure. We modified the model in terms of tensor normalization [36] and gradient centralization [61] as well as convolution size and depth. Level | Gaze estimator
---|---
Input | RGB image any resolution
1 | Pyramid layer with six stages
2 |  $5\times 5$ Conv, depth 8, $2\times 2$ down scaling, BN, ReLu with tensor normalization
3 |  $3\times 3$ Conv, depth 8, $2\times 2$ down scaling, BN, ReLu with tensor normalization
4 |  $3\times 3$ Conv, depth 8, $2\times 2$ down scaling, BN, ReLu with tensor normalization
5 |  $5\times 5$ Conv, depth 16, BN, ReLu with tensor normalization
6 |  $3\times 3$ Conv, depth 16, BN, ReLu with tensor normalization
7 |  $3\times 3$ Conv, depth 16, BN, ReLu with tensor normalization
8 |  $7\times 7$ Conv, depth 1
Table 2: Shows the architecture of our gaze estimation deep neural network. It has the structure of a ResNet-34 [64] and uses the leaky maximum propagation blocks [59], tensor normalization [36], as well as the weight and gradient centralization [61]. Level | Gaze estimator
---|---
Input | Gray scale image $100\times 100$
1 |  $5\times 5$ Conv, depth 32
2 | ReLu with tensor normalization
3 |  $2\times 2$ Max pooling
4 | 3 Maxium connection blocks, $2\times 2$ down scaling, $3\times 3$ Conv, depth 64, BN
5 | ReLu with tensor normalization
6 | 3 Maxium connection blocks, $2\times 2$ down scaling, $3\times 3$ Conv, depth 128, BN
7 | ReLu with tensor normalization
8 | 3 Maxium connection blocks, $2\times 2$ down scaling, $3\times 3$ Conv, depth 256, BN
9 | ReLu with tensor normalization
10 | Fully connected, 512 outputs
11 | ReLu
12 | Fully connected, 7 outputs (3 and 7 are the accuracy of the estimation [32])

Figure 2 shows the workflow of our approach. GroupGazer first opens a video
stream on an available camera. Afterwards, all faces in the image are
detected. If not all desired faces are detected, GroupGazer offers an
upscaling factor, which can be set by the user. This upscaling factor resizes
the input image to allow the face detection to detect even very small faces in
the image. After the face detection, all detected faces are extracted from the
image and resized to $100\times 100$ pixels in a gray scale image. These
images are grouped together to form a batch which is given to the gaze vector
estimation DNN. The batch size can also be set by the user. This fixed batch
size allows GroupGazer to have a static runtime and if there are fewer faces
in the image, the rest of the batch is filled with black images. GroupGazer
can be used with a 1050 ti graphics card for up to 40 faces in real time,
which is also dependent on the input resolution to the face detection DNN. For
newer GPUs more faces can be set by the user as well as larger input image
resolutions for the face detection. The gaze estimation DNN processes the
entire batch and computes a starting position (First two values), an accuracy
of the starting position (Third value), the gaze vector (Forth to sixth
value), as well as an accuracy of the gaze vector (Seventh value). With this
information, each face has a gaze vector and an estimated accuracy. With the
gaze vector and the starting position, a polynomial is used to map the gaze
vector to a projection or monitor. The degree of the polynomial can be
specified by the user, and the calibration procedure works as follows. The
teacher or adviser tells the students to look at his mouse cursor position. On
a left mouse click, all gaze vectors which are seen as valid and accurate are
stored together with the click location. This is repeated multiple times.
Afterwards, for each user, the polynomial is fitted in the least squares
sense. With those polynomials, the mapping and therefore the gaze location is
computed for each user. The reidentification of users is done by the smallest
euclidean distance to the last detections, and the new position is not allowed
to leaf the last face detection bounding box. This is a simple procedure but
saves a lot of computational resources since no additional network has to be
used. In addition, it is much more robust since fine-tuning a Network online
usually needs multiple examples to deliver reliable results, even if we use
the hypersphere approach [115] or siam networks [1].

The used model architectures can be seen in Table 1 and 2. Our face detection
model is similar to the model from dlib [77] we only made some slight changes
which improve the accuracy of the model and only impact the runtime slightly.
For gaze estimation we used the architecture of a ResNet-34 [64] since it has
a good accuracy and is resource saving in contrast to the other networks. We
modified the ResNet-34 architecture only by adding some novel normalization
[36, 61], the landmark validation loss [32], as well as leaky maximum
propagations instead of the residual connections [59].

##  4 Evaluation

Gaze360 [76] is a huge data set with 3D gaze annotations recorded using
multiple cameras covering 360 degree. The recordings were conducted indoor and
outdoor with 238 subjects. The dataset contains large head variations as well
as distances of the subjects to the camera. We only used approximately 80,000
images of this data set since the data set contains also human heads from
behind as well as some partially covered heads which we removed from our data
for training and evaluation. The train and test split was done by randomly
selecting 20% for testing and 80% for training.

DLIB [77] data set contains images of various resolutions. Each image can have
multiple faces which are annotated with bounding boxes. In total the data set
contains 7213 images and 11480 annotated faces. We made a random 50% to 50%
split and used the first half for training and the second half for evaluation
(One image more for training due to the uneven number). The images in the
dataset are taken from other public data sets and annotated by the authors of
[77].

Table 3: Face detection results on DLIB data set [77] with percition and recall. We compare our model to other approaches in therms of detection percentage as well as runtime in milliseconds (ms) for one hundred images in average. OoM means out of memory exception. Method | Percision | Recall | Runtime GPU (ms)
---|---|---|---
|  |  | $1920\times 1280$ | $300\times 300$
Proposed | 0,99 | 0,89 | 67 | 3
dlib std. arch. [77] | 0,99 | 0,88 | 175 | 8
ResNet-34 & Faster-RCNN [101] | 0,99 | 0,91 | OoM | 22 & 1
Yolov5s [99] | 0,99 | 0,89 | OoM | 10
Table 4: Appearance based gaze estimation results on the Gaze360 [76] dataset. We compared our model to other approaches and evaluated the gaze start estimation in average euclidean distance in pixel as well as the gaze vector estimation in degree. Time is measured for one face image as average over one thousand. Method | Gaze start | Gaze vector | Runtime GPU (ms)
---|---|---|---
Proposed | 0,6 | 0,2 | 3
ResNet-34 [64] | 0,9 | 0,5 | 8
ResNet-50 [64] | 0,5 | 0,2 | 12
MobileNet [66] | 1,8 | 1,6 | 7
MobileNetv2 [104] | 1,7 | 1,6 | 7
Table 5: Accuracy of the proposed tool for different distances to the camera. The results are the average accuracy over three subjects on a TV screen with a diagonal of 108cm. 1m | 2m | 3m | 4m | 5m | 6m
---|---|---|---|---|---
4cm | 5cm | 8cm | 12cm | 15cm | 19cm

Figure 3: Qualitative evaluation on different images from pixabay.com, which
are free to use. Left original image, right detections. The images have a high
resolution in the pdf so you can zoom in to see everything.

In Table 3 and 4 our models are compared with other approaches. For face
detection (Table 3), it can be seen that we have chosen a tradeoff between
detection rate and runtime. The recognition rate of our approach can be
further increased via the upscaling factor. However, this also increases the
computation time, which also increases the runtime per image. For Yolo this is
not possible, because the memory usage for images larger than $300$ becomes
too large. For the backbone of the faster-RCNN, the memory consumption is also
too high for a large resolution. Which is also the main reason why we decided
against YOLO and the faster-RCNN. In addition, both the faster-RCNN with
backbone and the YOLO need a fixed input resolution with which they have to be
trained. For our fully convolutional approach inspired by the dlib
architecture, this is not necessary.

For the gaze direction determination, you can clearly see that our net runs
significantly faster than the other nets. This is due to the fact that our
layers use less depth than, for example, ResNet-34. The MobileNets cannot show
their advantage on the GPU, since they cause cache conflicts here, whereby
parts of the code are executed serialized. On a CPU, MobileNet would be
significantly faster than our net, but with about 160 ms per face too slow for
a real-time evaluation. In terms of results, ResNet-50 is the most accurate,
closely followed by our network. In addition to accuracy, if we consider
runtime on a GPU, our network is clearly ahead, which is why we chose our
architecture.

The accuracy of our approach for different distances can be seen in Table 5.
As can be seen, the distance has a huge impact on the accuracy. For a 6-meter
distance, the error of 19 cm is approximately 20% of the projection area. For
a beamer projection this would not be as crucial since the diagonal here is
usually 2 to 3 meters or more. Therefore, our system is capable of computing
useful gaze positions, but does not have the accuracy of a professional remote
eye tracker which can only be used from persons in front of the screen. On the
other side, our system can compute and map the gaze of multiple persons in
parallel and is therefore made for classrooms or meetings.

In addition to the quantitative analysis from Table 3 and 4, we also made a
qualitative analysis. The images in Figure 3 have a very high resolution in
the PDF, so everything can be seen in detail when using the zoom function.
Also, all images are in supplementary material. Looking at the first image in
Figure 3 you can see that our mesh can also appreciate the head-only pose when
the eyes are covered. On the second, third and last image, one can see that
our network works with both color and gray scale images. Likewise, it is
capable of good results on crowd gatherings, as can be seen in the fourth
image.

##  5 Limitations

One limitation of the presented software is that persons are not recognized
via a neural network, but are assigned based on their last position. This has
the limitation that the persons cannot move freely, but the software is only
suitable for slightly dynamic scenarios like a classroom. In the future,
recognition, which is adapted online, will be integrated. Another limitation
of our software is that currently only gaze data is extracted. Since other
features such as the eyelids as well as a more accurate gaze determination via
the pupils are also interesting for research, these will also be integrated in
future updates. The final limitation of our software is the need for a GPU.
This limitation will not be changed in the near future, since the problem of
face recognition and fast gaze determination can be solved by cheaper methods,
but these have strong losses in detection rate and accuracy.

##  6 Conclusion

In this paper, we have presented GroupGazer. This is a software that allows to
determine the gaze direction of groups per person. This gaze determination is
done online on a conventional computer with an NVIDIA GPU. GroupGazer allows
each person in the group to be calibrated in parallel so that the individual
gaze vectors can be mapped to a projection, such as that of a projector or
large monitor. The software is intended to support behavioral research and
thus make it possible to easily record the gaze positions of groups. Future
enhancements to the software include person recognition for dynamic processes
such as sports, pose determination, emotion determination, advanced feature
extraction such as eyelids, and more accurate gaze determination based on eye
features such as the pupil. We hope that the software will help other
researchers in their work and further advance behavioral research as well as
the application area of group-based gaze determination.

## References

  * [1] Mohamed H Abdelpakey and Mohamed S Shehata.  Dp-siam: Dynamic policy siamese network for robust object tracking.  IEEE Transactions on Image Processing, 29:1479–1492, 2019.
  * [2] Serkan Alkan and Kursat Cagiltay.  Studying computer game learning experience through eye tracking.  British Journal of Educational Technology, 38(3):538–542, 2007\.
  * [3] Arkar Min Aung, Anand Ramakrishnan, and Jacob R Whitehill.  Who are they looking at? automatic eye gaze following for classroom observation video analysis.  International Educational Data Mining Society, 2018.
  * [4] Yancheng Bai, Yongqiang Zhang, Mingli Ding, and Bernard Ghanem.  Finding tiny faces in the wild with generative adversarial network.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 21–30, 2018.
  * [5] Shumeet Baluja and Dean Pomerleau.  Non-intrusive gaze tracking using artificial neural networks.  Technical report, CARNEGIE-MELLON UNIV PITTSBURGH PA DEPT OF COMPUTER SCIENCE, 1994.
  * [6] Zillah Boraston and Sarah-Jayne Blakemore.  The application of eye-tracking technology in the study of autism.  The Journal of physiology, 581(3):893–898, 2007.
  * [7] Margot Chauliac, Leen Catrysse, David Gijbels, and Vincent Donche.  It is all in the" surv-eye": Can eye tracking data shed light on the internal consistency in self-report questionnaires on cognitive processing strategies?.  Frontline Learning Research, 8(3):26–39, 2020.
  * [8] Yihua Cheng, Xucong Zhang, Feng Lu, and Yoichi Sato.  Gaze estimation by exploring two-eye asymmetry.  IEEE Transactions on Image Processing, 29:5259–5272, 2020.
  * [9] Meia Chita-Tegmark.  Social attention in asd: A review and meta-analysis of eye-tracking studies.  Research in developmental disabilities, 48:79–93, 2016.
  * [10] Rosie Clark, James Blundell, Matt J Dunn, Jonathan T Erichsen, Mario E Giardini, Irene Gottlob, Chris Harris, Helena Lee, Lee Mcilreavy, Andrew Olson, et al.  The potential and value of objective eye tracking in the ophthalmology clinic.  Eye, 33(8):1200–1202, 2019.
  * [11] Mihaela Cocea and Stephan Weibelzahl.  Disengagement detection in online learning: Validation studies and perspectives.  IEEE transactions on learning technologies, 4(2):114–124, 2010\.
  * [12] Matteo Cognolato, Manfredo Atzori, and Henning Müller.  Head-mounted eye gaze tracking devices: An overview of modern devices and recent advances.  Journal of rehabilitation and assistive technologies engineering, 5:2055668318773991, 2018.
  * [13] Trevor J Crawford.  The disengagement of visual attention in alzheimer’s disease: a longitudinal eye-tracking study.  Frontiers in aging neuroscience, 7:118, 2015.
  * [14] Marcel Das, Peter Ester, and Lars Kaczmirek.  Social and behavioral research and the internet: Advances in applied methods and research strategies.  Routledge, 2018.
  * [15] Peet J Du Toit, Pieter Ernst Kruger, NZ Chamane, Jolene Campher, and Dalene Crafford.  Sport vision assessment in soccer players and sport science.  African Journal for Physical Health Education, Recreation and Dance, 15(4):594–604, 2009.
  * [16] Mads Gjerstad Eide, Ruben Watanabe, Ilona Heldal, Carsten Helgesen, Atle Geitung, and Harald Soleim.  Detecting oculomotor problems using eye tracking: Comparing eyex and tx300.  In 2019 10th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), pages 381–388. IEEE, 2019.
  * [17] Alircza Fathi, Jessica K Hodgins, and James M Rehg.  Social interactions: A first-person perspective.  In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pages 1226–1233. IEEE, 2012.
  * [18] E. Fuhl, W.and Bozkir, B. Hosp, N. Castner, D. Geisler, T. Santini, and E. Kasneci.  Encodji: encoding gaze data into emoji space for an amusing scanpath classification approach.  In Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications, pages 1–4, 2019.
  * [19] W. Fuhl.  Image-based extraction of eye features for robust eye tracking.  PhD thesis, University of Tübingen, 04 2019.
  * [20] W. Fuhl.  From perception to action using observed actions to learn gestures.  User Modeling and User-Adapted Interaction, pages 1–18, 08 2020\.
  * [21] W. Fuhl, N. Castner, and E. Kasneci.  Histogram of oriented velocities for eye movement detection.  In International Conference on Multimodal Interaction Workshops, ICMIW, 2018.
  * [22] W. Fuhl, N. Castner, and E. Kasneci.  Rule based learning for eye movement type detection.  In International Conference on Multimodal Interaction Workshops, ICMIW, 2018.
  * [23] W. Fuhl, N. Castner, T. C. Kübler, A. Lotz, W. Rosenstiel, and E. Kasneci.  Ferns for area of interest free scanpath classification.  In Proceedings of the 2019 ACM Symposium on Eye Tracking Research & Applications (ETRA), 06 2019.
  * [24] W. Fuhl, N. Castner, L. Zhuang, M. Holzer, W. Rosenstiel, and E. Kasneci.  Mam: Transfer learning for fully automatic video annotation and specialized detector creation.  In International Conference on Computer Vision Workshops, ICCVW, 2018.
  * [25] W. Fuhl, S. Eivazi, B. Hosp, A. Eivazi, W. Rosenstiel, and E. Kasneci.  Bore: Boosted-oriented edge optimization for robust, real time remote pupil center detection.  In Eye Tracking Research and Applications, ETRA, 2018.
  * [26] W. Fuhl, H. Gao, and E. Kasneci.  Neural networks for optical vector and eye ball parameter estimation.  In ACM Symposium on Eye Tracking Research & Applications, ETRA 2020. ACM, 01 2020.
  * [27] W. Fuhl, H. Gao, and E. Kasneci.  Tiny convolution, decision tree, and binary neuronal networks for robust and real time pupil outline estimation.  In ACM Symposium on Eye Tracking Research & Applications, ETRA 2020. ACM, 01 2020.
  * [28] W. Fuhl, D. Geisler, W. Rosenstiel, and E. Kasneci.  The applicability of cycle gans for pupil and eyelid segmentation, data generation and image refinement.  In International Conference on Computer Vision Workshops, ICCVW, 11 2019.
  * [29] W. Fuhl, D. Geisler, T. Santini, T. Appel, W. Rosenstiel, and E. Kasneci.  Cbf:circular binary features for robust and real-time pupil center detection.  In ACM Symposium on Eye Tracking Research & Applications, 06 2018\.
  * [30] W. Fuhl, D. Geisler, T. Santini, and E. Kasneci.  Evaluation of state-of-the-art pupil detection algorithms on remote eye images.  In ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct publication – PETMEI 2016, 09 2016.
  * [31] W. Fuhl and E. Kasneci.  Eye movement velocity and gaze data generator for evaluation, robustness testing and assess of eye tracking software and visualization tools.  In Poster at Egocentric Perception, Interaction and Computing, EPIC, 2018.
  * [32] W. Fuhl and E. Kasneci.  Learning to validate the quality of detected landmarks.  In International Conference on Machine Vision, ICMV, nov 2019.
  * [33] W. Fuhl and E. Kasneci.  Multi layer neural networks as replacement for pooling operations.  arXiv preprint arXiv:2006.06969, 08 2020.
  * [34] W. Fuhl and E. Kasneci.  A multimodal eye movement dataset and a multimodal eye movement segmentation analysis.  In Proceedings of the ACM Symposium on Eye Tracking Research & Applications (ETRA), 2021.
  * [35] W. Fuhl and E. Kasneci.  Rotated ring, radial and depth wise separable radial convolutions.  In Proceedings of the International Joint Conference on Neural Networks. IEEE, 2021.
  * [36] W. Fuhl and E. Kasneci.  Tensor normalization and full distribution training.  In Conference on Artificial Intelligence Workshops, AAAI Workshop Adversarial Machine Learning and Beyond, 02 2022.
  * [37] W. Fuhl, G. Kasneci, and E. Kasneci.  Teyed: Over 20 million real-world eye images with pupil, eyelid, and iris 2d and 3d segmentations, 2d and 3d landmarks, 3d eyeball, gaze vector, and eye movement types.  In IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2021.
  * [38] W. Fuhl, G. Kasneci, W. Rosenstiel, and E. Kasneci.  Training decision trees as replacement for convolution layers.  In Conference on Artificial Intelligence, AAAI, 02 2020.
  * [39] W. Fuhl, T. Kübler, T. Santini, and E. Kasneci.  Automatic generation of saliency-based areas of interest for the visualization and analysis of eye-tracking data.  In VMV, pages 47–54, 2018.
  * [40] W. Fuhl, T. C. Kübler, H. Brinkmann, R. Rosenberg, W. Rosenstiel, and E. Kasneci.  Region of interest generation algorithms for eye tracking data.  In Third Workshop on Eye Tracking and Visualization (ETVIS), in conjunction with ACM ETRA, 06 2018.
  * [41] W. Fuhl, T. C. Kübler, D. Hospach, O. Bringmann, W. Rosenstiel, and E. Kasneci.  Ways of improving the precision of eye tracking data: Controlling the influence of dirt and dust on pupil detection.  Journal of Eye Movement Research, 10(3), 05 2017.
  * [42] W. Fuhl, T. C. Kübler, K. Sippel, W. Rosenstiel, and E. Kasneci.  Arbitrarily shaped areas of interest based on gaze density gradient.  In European Conference on Eye Movements, ECEM 2015, 08 2015.
  * [43] W. Fuhl, T. C. Kübler, K. Sippel, W. Rosenstiel, and E. Kasneci.  Excuse: Robust pupil detection in real-world scenarios.  In 16th International Conference on Computer Analysis of Images and Patterns (CAIP 2015), 09 2015.
  * [44] W. Fuhl, Y. Rong, and K. Enkelejda.  Fully convolutional neural networks for raw eye tracking data segmentation, generation, and reconstruction.  In Proceedings of the International Conference on Pattern Recognition, pages 0–0, 2020.
  * [45] W. Fuhl, Y. Rong, T. Motz, M. Scheidt, A. Hartel, A. Koch, and E. Kasneci.  Explainable online validation of machine learning models for practical applications.  In Proceedings of the International Conference on Pattern Recognition, pages 0–0, 2020.
  * [46] W. Fuhl, W. Rosenstiel, and E. Kasneci.  500,000 images closer to eyelid and pupil segmentation.  In Computer Analysis of Images and Patterns, CAIP, 11 2019.
  * [47] W Fuhl, N Sanamrad, and E Kasneci.  The gaze and mouse signal as additional source for user fingerprints in browser applications.  arXiv preprint arXiv:2101.03793, 01 2021.
  * [48] W. Fuhl, T. Santini, D. Geisler, T. C. Kübler, and E. Kasneci.  Eyelad: Remote eye tracking image labeling tool.  In 12th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2017), 02 2017.
  * [49] W. Fuhl, T. Santini, D. Geisler, T. C. Kübler, W. Rosenstiel, and E. Kasneci.  Eyes wide open? eyelid location and eye aperture estimation for pervasive eye tracking in real-world scenarios.  In ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct publication – PETMEI 2016, 09 2016.
  * [50] W. Fuhl, T. Santini, and E. Kasneci.  Fast and robust eyelid outline and aperture detection in real-world scenarios.  In IEEE Winter Conference on Applications of Computer Vision (WACV 2017), 03 2017.
  * [51] W. Fuhl, T. Santini, and E. Kasneci.  Fast camera focus estimation for gaze-based focus control.  arXiv preprint arXiv:1711.03306, 2017.
  * [52] W. Fuhl, T. Santini, G. Kasneci, and E. Kasneci.  Pupilnet: Convolutional neural networks for robust pupil detection.  arXiv preprint arXiv:1601.04902, 2016.
  * [53] W. Fuhl, T. Santini, G. Kasneci, W. Rosenstiel, and E. Kasneci.  Pupilnet v2.0: Convolutional neural networks for cpu based real time robust pupil detection.  arXiv preprint arXiv:1711.00112, 2017.
  * [54] W. Fuhl, T. Santini, T. Kuebler, N. Castner, W. Rosenstiel, and E. Kasneci.  Eye movement simulation and detector creation to reduce laborious parameter adjustments.  arXiv preprint arXiv:1804.00970, 2018.
  * [55] W. Fuhl, T. Santini, T. C. Kübler, and E. Kasneci.  Else: Ellipse selection for robust pupil detection in real-world environments.  In Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications (ETRA), pages 123–130, 03 2016.
  * [56] W. Fuhl, T. Santini, C. Reichert, D. Claus, A. Herkommer, H. Bahmani, K. Rifai, S. Wahl, and E. Kasneci.  Non-intrusive practitioner pupil detection for unmodified microscope oculars.  Elsevier Computers in Biology and Medicine, 79:36–44, 12 2016.
  * [57] W. Fuhl, J. Schneider, and E. Kasneci.  1000 pupil segmentations in a second using haar like features and statistical learning.  In International Conference on Computer Vision Workshops, ICCVW, 2021.
  * [58] W. Fuhl, M. Tonsen, A. Bulling, and E. Kasneci.  Pupil detection for head-mounted eye tracking in the wild: An evaluation of the state of the art.  In Machine Vision and Applications, pages 1–14, 06 2016.
  * [59] Wolfgang Fuhl.  Maximum and leaky maximum propagation.  arXiv preprint arXiv:2105.10277, 2021.
  * [60] Wolfgang Fuhl, Efe Bozkir, and Enkelejda Kasneci.  Reinforcement learning for the privacy preservation and manipulation of eye tracking data.  In Proceedings of IEEE International Joint Conference on Neural Networks, 2021.
  * [61] Wolfgang Fuhl and Enkelejda Kasneci.  Weight and gradient centralization in deep neural networks.  In Proceedings of IEEE International Joint Conference on Neural Networks, 2021.
  * [62] Ross Girshick.  Fast r-cnn.  In Proceedings of the IEEE international conference on computer vision, pages 1440–1448, 2015.
  * [63] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik.  Rich feature hierarchies for accurate object detection and semantic segmentation.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 580–587, 2014.
  * [64] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [65] Shahab Hoghooghi, Vesna Popovic, and Levi Swann.  Novice to expert real-time knowledge transition in the context of x-ray airport security.  In Proceedings of DRS 2020 International Conference: Synergy. Vol. 4. Design Research Society (UK), 2020.
  * [66] Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam.  Mobilenets: Efficient convolutional neural networks for mobile vision applications.  arXiv preprint arXiv:1704.04861, 2017.
  * [67] Stephen Hutt, Kristina Krasich, Caitlin Mills, Nigel Bosch, Shelby White, James R Brockmole, and Sidney K D’Mello.  Automated gaze-based mind wandering detection during computerized learning in classrooms.  User Modeling and User-Adapted Interaction, 29(4):821–867, 2019\.
  * [68] Stephen Hutt, Caitlin Mills, Nigel Bosch, Kristina Krasich, James Brockmole, and Sidney D’mello.  " out of the fr-eye-ing pan" towards gaze-based models of attention during learning with technology in the classroom.  In Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization, pages 94–103, 2017.
  * [69] Yoon Min Hwang and Kun Chang Lee.  An eye-tracking paradigm to explore the effect of online consumers’ emotion on their visual behaviour between desktop screen and mobile screen.  Behaviour & Information Technology, pages 1–12, 2020.
  * [70] Halszka Jarodzka, Irene Skuballa, and Hans Gruber.  Eye-tracking in educational practice: Investigating visual perception underlying teaching and learning in the classroom.  Educational Psychology Review, pages 1–10, 2020.
  * [71] Libby Jenke, Kirk Bansak, Jens Hainmueller, and Dominik Hangartner.  Using eye-tracking to understand decision-making in conjoint experiments.  Political Analysis, 29(1):75–101, 2021.
  * [72] Huaizu Jiang and Erik Learned-Miller.  Face detection with the faster r-cnn.  In 2017 12th IEEE international conference on automatic face & gesture recognition (FG 2017), pages 650–657. IEEE, 2017.
  * [73] Erika Jönsson.  If looks could kill–an evaluation of eye tracking in computer games.  Unpublished Master’s Thesis, Royal Institute of Technology (KTH), Stockholm, Sweden, 2005.
  * [74] Tilke Judd, Krista Ehinger, Frédo Durand, and Antonio Torralba.  Learning to predict where humans look.  In 2009 IEEE 12th international conference on computer vision, pages 2106–2113. IEEE, 2009.
  * [75] Christina Katsini, Yasmeen Abdrabou, George E Raptis, Mohamed Khamis, and Florian Alt.  The role of eye gaze in security and privacy applications: Survey and future hci research directions.  In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pages 1–21, 2020.
  * [76] Petr Kellnhofer, Adria Recasens, Simon Stent, Wojciech Matusik, and Antonio Torralba.  Gaze360: Physically unconstrained gaze estimation in the wild.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6912–6921, 2019.
  * [77] Davis E King.  Dlib-ml: A machine learning toolkit.  The Journal of Machine Learning Research, 10:1755–1758, 2009.
  * [78] Davis E King.  Max-margin object detection.  arXiv preprint arXiv:1502.00046, 2015.
  * [79] Andreas Korbach, Paul Ginns, Roland Brünken, and Babette Park.  Should learners use their hands for learning? results from an eye-tracking study.  Journal of Computer Assisted Learning, 36(1):102–113, 2020.
  * [80] Kyle Krafka, Aditya Khosla, Petr Kellnhofer, Harini Kannan, Suchendra Bhandarkar, Wojciech Matusik, and Antonio Torralba.  Eye tracking for everyone.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2176–2184, 2016.
  * [81] Ralf Kredel, Christian Vater, André Klostermann, and Ernst-Joachim Hossner.  Eye-tracking technology and the dynamics of natural gaze behavior in sports: A systematic review of 40 years of research.  Frontiers in psychology, 8:1845, 2017.
  * [82] Zhihang Li, Xu Tang, Junyu Han, Jingtuo Liu, and Ran He.  Pyramidbox++: high performance detector for finding tiny face.  arXiv preprint arXiv:1904.00386, 2019.
  * [83] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg.  Ssd: Single shot multibox detector.  In European conference on computer vision, pages 21–37. Springer, 2016.
  * [84] Feng Lu, Yusuke Sugano, Takahiro Okabe, and Yoichi Sato.  Adaptive linear regression for appearance-based gaze estimation.  IEEE transactions on pattern analysis and machine intelligence, 36(10):2033–2046, 2014.
  * [85] David Manning, Susan C Ethell, and Trevor Crawford.  Eye-tracking afroc study of the influence of experience and training on chest x-ray interpretation.  In Medical Imaging 2003: Image Perception, Observer Performance, and Technology Assessment, volume 5034, pages 257–266. International Society for Optics and Photonics, 2003.
  * [86] Manuel Jesús Marin-Jimenez, Andrew Zisserman, Marcin Eichner, and Vittorio Ferrari.  Detecting people looking at each other in videos.  International Journal of Computer Vision, 106(3):282–296, 2014\.
  * [87] Hideyuki Matsumoto, Yasuo Terao, Akihiro Yugeta, Hideki Fukuda, Masaki Emoto, Toshiaki Furubayashi, Tomoko Okano, Ritsuko Hanajima, and Yoshikazu Ugawa.  Where do neurologists look when viewing brain ct images? an eye-tracking study involving stroke cases.  PloS one, 6(12):e28928, 2011.
  * [88] Pierre Maurage, Nicolas Masson, Zoé Bollen, and Fabien D’Hondt.  Eye tracking correlates of acute alcohol consumption: A systematic and critical review.  Neuroscience & Biobehavioral Reviews, 108:400–422, 2020.
  * [89] Nora A McIntyre and Tom Foulsham.  Scanpath analysis of expertise and culture in teacher gaze in real-world classrooms.  Instructional Science, 46(3):435–455, 2018.
  * [90] Aideen McParland, Stephen Gallagher, and Mickey Keenan.  Investigating gaze behaviour of children diagnosed with autism spectrum disorders in a classroom setting.  Journal of Autism and Developmental Disorders, pages 1–16, 2021\.
  * [91] Xiaoxu Meng, Ruofei Du, Matthias Zwicker, and Amitabh Varshney.  Kernel foveated rendering.  Proceedings of the ACM on Computer Graphics and Interactive Techniques, 1(1):1–20, 2018.
  * [92] Caitlin Mills, Nigel Bosch, Art Graesser, and Sidney D’Mello.  To quit or not to quit: predicting future behavioral disengagement from reading patterns.  In International Conference on Intelligent Tutoring Systems, pages 19–28. Springer, 2014.
  * [93] Sankha S Mukherjee and Neil Martin Robertson.  Deep head pose: Gaze-direction estimation in multimodal video.  IEEE Transactions on Multimedia, 17(11):2094–2107, 2015.
  * [94] Jessie R Oldham, Christina L Master, Gregory A Walker, William P Meehan III, and David R Howell.  The association between baseline eye tracking performance and concussion assessments in high school football players.  Optometry and Vision Science, 98(7):826–832, 2021.
  * [95] Oskar Palinko, Francesco Rea, Giulio Sandini, and Alessandra Sciutti.  Robot reading human gaze: Why eye tracking is better than head tracking for human-robot collaboration.  In 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5048–5054. IEEE, 2016.
  * [96] Mathieson Tan Zui Quen, J Mountstephens, Yong Guang Teh, and J Teo.  Medical image interpretation training with a low-cost eye tracking and feedback system: A preliminary study.  Healthcare Technology Letters, 2021.
  * [97] Adria Recasens, Carl Vondrick, Aditya Khosla, and Antonio Torralba.  Following gaze in video.  In Proceedings of the IEEE International Conference on Computer Vision, pages 1435–1443, 2017.
  * [98] Adriá Recasens Continente Recasens.  Where are they looking? PhD thesis, Massachusetts Institute of Technology, 2016.
  * [99] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi.  You only look once: Unified, real-time object detection.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 779–788, 2016.
  * [100] Jonas Reichenberger, Michael Pfaller, and Andreas Mühlberger.  Gaze behavior in social fear conditioning: An eye-tracking study in virtual reality.  Frontiers in psychology, 11:35, 2020.
  * [101] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks.  Advances in neural information processing systems, 28:91–99, 2015\.
  * [102] Jennifer C Reneker, W Cody Pannell, Ryan M Babl, Yunxi Zhang, Seth T Lirette, Felix Adah, and Matthew R Reneker.  Virtual immersive sensorimotor training (vist) in collegiate soccer athletes: A quasi-experimental study.  Heliyon, 6(7):e04527, 2020.
  * [103] Ian H Robertson, Tom Manly, Jackie Andrade, Bart T Baddeley, and Jenny Yiend.  Oops!’: performance correlates of everyday attentional failures in traumatic brain injured and normal subjects.  Neuropsychologia, 35(6):747–758, 1997.
  * [104] Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen.  Mobilenetv2: Inverted residuals and linear bottlenecks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4510–4520, 2018.
  * [105] Michael Schneider, Angela Heine, Verena Thaler, Joke Torbeyns, Bert De Smedt, Lieven Verschaffel, Arthur M Jacobs, and Elsbeth Stern.  A validation of eye movements as a measure of elementary school children’s developing number sense.  Cognitive Development, 23(3):409–422, 2008.
  * [106] Jonathan Smallwood, Merrill McSpadden, and Jonathan W Schooler.  When attention matters: The curious incident of the wandering mind.  Memory & Cognition, 36(6):1144–1150, 2008.
  * [107] Yusuke Sugano, Yasuyuki Matsushita, and Yoichi Sato.  Appearance-based gaze estimation using visual saliency.  IEEE transactions on pattern analysis and machine intelligence, 35(2):329–341, 2012.
  * [108] Kar-Han Tan, David J Kriegman, and Narendra Ahuja.  Appearance-based eye gaze estimation.  In Sixth IEEE Workshop on Applications of Computer Vision, 2002.(WACV 2002). Proceedings., pages 191–195. IEEE, 2002.
  * [109] Kavin Kathiresh Vijayan, Ola Jon Mork, and Irina Emily Hansen.  Eye tracker as a tool for engineering education.  Universal Journal of Educational Research, 6(11):2647–2655, 2018\.
  * [110] Paul Viola and Michael J Jones.  Robust real-time face detection.  International journal of computer vision, 57(2):137–154, 2004.
  * [111] Nutthanan Wanluk, Sarinporn Visitsattapongse, Aniwat Juhong, and C Pintavirooj.  Smart wheelchair based on eye tracking.  In 2016 9th Biomedical Engineering International Conference (BMEiCON), pages 1–4. IEEE, 2016.
  * [112] Cesco Willemse and Agnieszka Wykowska.  In natural interaction with embodied robots, we prefer it when they follow our gaze: a gaze-contingent mobile eyetracking study.  Philosophical Transactions of the Royal Society B, 374(1771):20180036, 2019.
  * [113] Oliver Williams, Andrew Blake, and Roberto Cipolla.  Sparse and semi-supervised visual mapping with the s^ 3gp.  In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), volume 1, pages 230–237. IEEE, 2006.
  * [114] Genevieve Alice Woolverton and Alisha R Pollastri.  An exploration and critical examination of how “intelligent classroom technologies” can improve specific uses of direct student behavior observation methods.  Educational Measurement: Issues and Practice, 2021.
  * [115] Renjie Xie, Yanzhi Chen, Yan Wo, and Qiao Wang.  A deep, information-theoretic framework for robust biometric recognition.  arXiv preprint arXiv:1902.08785, 2019.
  * [116] Junjie Yan, Zhen Lei, Longyin Wen, and Stan Z Li.  The fastest deformable part model for object detection.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2497–2504, 2014.
  * [117] Xiaozhi Yang and Ian Krajbich.  Webcam-based online eye-tracking for behavioral research.  Judgment and Decision Making, 16(6):1486, 2021.
  * [118] Ali Shahidi Zandi, Azhar Quddus, Laura Prest, and Felix JE Comeau.  Non-intrusive detection of drowsy driving based on eye tracking data.  Transportation research record, 2673(6):247–257, 2019.
  * [119] Cha Zhang and Zhengyou Zhang.  Improving multiview face detection with multi-task deep convolutional neural networks.  In IEEE Winter Conference on Applications of Computer Vision, pages 1036–1041. IEEE, 2014.
  * [120] Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, and Yu Qiao.  Joint face detection and alignment using multitask cascaded convolutional networks.  IEEE Signal Processing Letters, 23(10):1499–1503, 2016.
  * [121] Xucong Zhang, Yusuke Sugano, Mario Fritz, and Andreas Bulling.  Appearance-based gaze estimation in the wild.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4511–4520, 2015.
  * [122] Zhishuai Zhang, Wei Shen, Siyuan Qiao, Yan Wang, Bo Wang, and Alan Yuille.  Robust face detection via learning small faces on hard images.  In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1361–1370, 2020.


# Attribute Controllable Beautiful Caucasian Face Generation by Aesthetics
Driven Reinforcement Learning

Xin Jin  Beijing Electronic Science and Technology
InstituteBeijingBeijingChina100070 ,  Shu Zhao  Beijing Electronic Science and
Technology InstituteBeijingBeijingChina100070 ,  Le Zhang  Beijing Electronic
Science and Technology InstituteBeijingBeijingChina100070 ,  Xin Zhao  Beijing
Electronic Science and Technology InstituteBeijingBeijingChina100070 ,  Qiang
Deng  Beijing Electronic Science and Technology
InstituteBeijingBeijingChina100070 and  Chaoen Xiao  Beijing Electronic
Science and Technology InstituteBeijingBeijingChina100070

(2018)

###### Abstract.

In recent years, image generation has made great strides in improving the
quality of images, producing high-fidelity ones. Also, quite recently, there
are architecture designs, which enable GAN to unsupervisedly learn the
semantic attributes represented in different layers. However, there is still a
lack of research on generating face images more consistent with human
aesthetics. Based on EigenGAN [He et al., ICCV 2021], we build the techniques
of reinforcement learning into the generator of EigenGAN. The agent tries to
figure out how to alter the semantic attributes of the generated human faces
towards more preferable ones. To accomplish this, we trained an aesthetics
scoring model that can conduct facial beauty prediction. We also can utilize
this scoring model to analyze the correlation between face attributes and
aesthetics scores. Empirically, using off-the-shelf techniques from
reinforcement learning would not work well. So instead, we present a new
variant incorporating the ingredients emerging in the reinforcement learning
communities in recent years. Compared to the original generated images, the
adjusted ones show clear distinctions concerning various attributes.
Experimental results using the MindSpore, show the effectiveness of the
proposed method. Altered facial images are commonly more attractive, with
significantly improved aesthetic levels.

image aesthetics, face generation, reinforcement learning

††copyright: acmcopyright††journalyear: 2018††doi:
10.1145/1122445.1122456††journal: JACM††journalvolume: 37††journalnumber:
4††article: 111††publicationmonth: 8 Figure 1. Top: the original faces
generated by EigenGAN (He et al., 2021). Bottom: aesthetically improved ones
by aesthetics-driven reinforcement learning over semantic attributes of the
generator.

##  1\. Introduction

In recent years, Generative Adversarial Networks (GANs) (Goodfellow et al.,
2014) show promising performance on image synthesis. A parallel line of
research tries to learn interpretable GAN generators, which can discover
explicit dimensions controlling the semantic attributes. InfoGAN in (Chen et
al., 2016) learns disentangled representations by decomposing the conventional
single unstructured noise vector into incompressible noise and a set of
structured latent variables. Motivated by style transfer literature, Karras et
al. proposed StyleGAN in (Karras et al., 2019), enabling separations of high-
level attributes like pose and identity. Most recently, He et al. proposed
EigenGAN (He et al., 2021), which for each generator layer directly embeds in
one linear subspace model with an orthogonal basis. During the training,
EigenGAN automatically learns a set of “eigen-dimensions” corresponding to
interpretable semantic attributes. So until now, people have several ways to
manipulate the semantics within the GANs, or said in other words, semantically
edit them. One problem followed is how to enhance the generated face images
positively.

In this paper, we incorporate the techniques of reinforcement learning (RL)
into the attribute values searching in EigenGAN. The first requirement of the
RL agent is the design of the reward function. The objective is to enhance the
images to the ones more aesthetically attractive. So for this, we trained an
aesthetics score network based on SCUT-FBP5500 released in (Liang et al.,
2018). By this, we give a multi-attribute model using MindSpore(MindSpore,
2022), which can conduct automatic Caucasian face beautification for the
generated images, as shown in Fig. 1. The user can select different facial
attributes in the interaction section to change the generated image.

##  2\. Related Work

Reinforcement Learning in Aesthetics. Li et al. proposed an RL-based framework
named Fast A3RL (Aesthetics-Aware Adversarial) (Li et al., 2018, 2019), which
dedicates to the task of image cropping. This task requires the agent to
extract a well-composed region from the input image. Li et al. first
formulated the cropping as a decision-making process. Next, they introduced an
RL-based cropping network, which contains an actor-critic architecture (Mnih
et al., 2016). Although Fast A3RL is also an application of RL in aesthetics,
the work proposed here differs in several aspects. On one side, Fast A3RL
relies on a specially designed reward function, which involves human-designed
heuristics besides an aesthetics score network. The solution discussed here,
however, only requires a straightforward regression over SCUT-FBP5500 (Liang
et al., 2018) to give facial beauty predictions. The solution does not need
any hand-craft features. On the other side, RL techniques utilized in Fast
A3RL are conventional policy gradient methods. So it requires two networks,
i.e., a policy network and a value network. Whereas we follow the approach
taken in the literature on image captioning (Rennie et al., 2017; Pan et al.,
2020; Zhang et al., 2021), and use a self-critical baseline. This form not
only simplifies the network design but also liberates the agent from learning
accurate value estimations.

Facial Attractiveness Enhancement. There are other facial image beautification
techniques proposed in recent years. Diamant et al. proposed Beholder-GAN
(Diamant et al., 2019), which is a variant of PGGAN (Progressive Growing)
(Karras et al., 2018) and CGAN (Conditional) (Mirza and Osindero, 2014).
Diamant et al. first trained a GAN based on beauty scores. Then for real
images, the latent vectors and the beauty scores are recovered using GAN
inversion techniques. To make input images more attractive, one only needs to
increase the beauty scores recovered. Although effective, Beholder-GAN
struggles to preserve the facial identity of the input image. In (Zhou and
Xiao, 2020), Zhou and Xiao proposed an approach based on InterFaceGAN (Shen et
al., 2020), and treated the “beauty” as a binary semantic. We note that in
InterFaceGAN, the main assumption is “for any binary semantic (eg, male v.s.
female), there exists a hyperplane in the latent space serving as the
separation boundary.” Although intuitively implausible (as beauty is not
obvious binary semantic as other semantics like the presence of eyeglasses or
not), Zhou and Xiao managed to validate the existence of such a beauty hyper-
plane. Like InterFaceGAN, they can now semantically edit this beauty semantic
of input images, generating aesthetically more attractive ones. Besides, by
inserting an identity preserving loss, they proposed an enhanced version of
Beholder-GAN. Nonetheless, as presented (and illustrated) in their paper, this
enhanced version only solved identity-preserving weakness to some extend. We
note that for these approaches, GANs are still acting like a black box.
Whereas, by contrast, we seek for precise controlling of explicit semantics.
The pre-trained model in (Shen et al., 2020) is not released, so the
comparison to it is not conducted in the experiment.

Figure 2. Illustration of the training procedure over two semantics $z_{3,4}$
and $z_{4,5}$. After setting $z_{3,4}$ and $z_{4,5}$ to zero, by cloning
$\mathbf{z}$, we have five identical $\mathbf{y}$’s. We generate five images
by feeding $\mathbf{y}$’s into the generator. We use ResNet18 to extract the
features of the five images, and input them into a GRUCell. Next, the hidden
state of GRUCell is input into a fully connected layer, and a softmax layer
then gives the action distribution for choosing values for $z_{3,4}$. We renew
these newly chosen values into $\mathbf{y}$’s. Again, by feeding these updated
$\mathbf{y}$’s into the generator, we have five new fresh images. The
procedure repeats, and we get the sampling values for $z_{4,5}$. After
updating the values for $z_{4,5}$ into $\mathbf{y}$’s, we get the final
vectors $\mathbf{y}^{(k)}$’s. We again feed $\mathbf{y}^{(k)}$’s into the
generator to get the final five images corresponding to five randomly sampled
trajectories. The aesthetics score network will score them and return five
scores. These scores provide an informative baseline for the soft policy
gradient. The RL module is next updated to raise the probabilities for the
trajectories with higher scores. Finally, for the inference, the agent will
not sample actions after the softmax layer. In contrast, it greedily chooses
the most probable one.

##  3\. EigenGAN + RL

In this section, we detail the building blocks for incorporating the policy
gradient algorithm into the generator of EigenGAN. We first briefly review the
preliminaries of RL and the architecture of the generator of EigenGAN. We
depict it as part of Fig. 2.

###  3.1. Preliminaries for RL

In RL, the agent constantly interacts with the environment, and this process
is described as a Markov Decision Process (MDP). We define the MDP as
$(\mathcal{S},\mathcal{A},P,r,\rho_{0},\gamma)$, where $\mathcal{S}$ and
$\mathcal{A}$ are the state and action space resp.,
$P:\mathcal{S}\times\mathcal{A}\times\mathcal{S}\to\mathbb{R}$ is the state
transition probability, $r:\mathcal{S}\to\mathbb{R}$ is the reward function,
$\rho_{0}$ is the distribution of the initial state $s_{0}$, and $\gamma$ is
the discounting factor satisfying $0\leq\gamma\leq 1$.

The objective is to learn a stochastic policy
$\pi:\mathcal{S}\times\mathcal{A}\to[0,1]$, such that when taking actions
$\mathbf{a}_{t}\sim\pi(\mathbf{a}_{t}|\mathbf{s}_{t})$, the expected reward of
the trajectories is maximized, i.e.,

(1)
$\pi^{*}=\underset{\pi}{\arg\max}\mathbb{E}_{\mathbf{s}_{0},\mathbf{a}_{0},\dotsc}\left[\gamma^{t}r(\mathbf{s}_{t})\right]\text{,}$

where $\mathbf{s}_{0}\sim\rho_{0}(\mathbf{s}_{0})$,
$\mathbf{a}_{t}\sim\pi(\mathbf{a}_{t}|\mathbf{s}_{t})$, $\mathbf{s}_{t+1}\sim
P(\mathbf{s}_{t+1}|\mathbf{s}_{t},\mathbf{a}_{t})$.

In this paper, the circumstance considered is a simplified version of the one
above. The discounting factor $\gamma$ equals 1. Also, no rewards exist for
intermediate states. The agent only gets a reward for the terminal state. The
reward value is the aesthetic score of the image after a series of
adjustments.

###  3.2. Generator of EigenGAN

The controlling parameters of the generator consist of not only the
conventionally noise input $\mathbf{\epsilon}\in\mathcal{N}(0,\mathbf{I})$,
but also a sequence of latent variables $\mathbf{z}_{i}\in\mathbb{R}^{q}$,
which are similarly sampled from $\mathcal{N}_{q}(0,\mathbf{I})$ (for
EigenGAN, $q$ is set to 6). Six blocks of transposed convolution layers
constitute the main part of the neural architecture. Strides of the two
transposed convolutions are 2 and 1 resp. Hence, the size of the image is
scaled by a factor of 2 when traveling through one such block. Within the
block, the randomly sampled point (due to the randomness of $\mathbf{z}_{i}$)
from the subspace model
$S_{i}=(\mathbf{U}_{i},\mathbf{L}_{i},\mathbf{\mu}_{i})$ is embedded into the
image by element-wise addition. For the notations, $\mathbf{U}_{i}$ is the
orthonormal basis, $\mathbf{L}_{i}$ is a diagonal matrix indicating the
“importance,” and $\mathbf{\mu}_{i}$ is the origin of the subspace.

For the input part, it is a fully connected layer mapping $\mathbf{\epsilon}$
into a $4\times 4\times 512$ image. Correspondingly for the output, it is
passed into a convolutional layer, getting a generated $256\times 256$ image
of 3 channels.

###  3.3. Learn to set semantic attributes

For EigenGAN, dimensions from latent variables $\mathbf{z}_{i}$ correspond to
interpretable or controllable semantic attributes. As an example, the fourth
dimension of $\mathbf{z}_{3}$ is related to the control of the bangs. The
sixth dimension of $\mathbf{z}_{5}$, for a specific pre-trained generator,
corresponds to the lipstick color. So as we move along these dimensions, the
generator produces a sequence of images with continuous changes corresponding
to the relevant semantic attributes. Six layers in EigenGAN brings us six such
$\mathbf{z}$’s. Recall every $\mathbf{z}_{i}\in\mathbb{R}^{q}$, and $q=6$. We
only adopt a subset of these controllable semantic attributes, as the others
are a bit entangled or drastic, like the first dimension of $\mathbf{z}_{3}$
is possible to be unsupervisedly trained to change the age or even the gender
of the generated face. The premise of the adjustments is to keep the unaltered
version still recognizable. Let $C$ be the set of selected controllable
semantic attributes. We only discuss how to let the RL agent learn to choose
the preferred values. We defer the details of the RL algorithm to the next
section.

Formally as shown in 2 (for conciseness, only two controllable dimensions are
involved), we first set the dimensions corresponding to the attributes in $C$
as zero. Any other “sensible” values (EigenGAN samples these values according
to a standard normal distribution) can also be okay, as the agent will set or
override these values anyway. The agent learns how to select over this
dimension during the training process. After these setting zero operations, we
concatenate these altered $\mathbf{z}_{i}$’s into one single vector. Let it be
$\mathbf{y}$. We clone the single vector five times. The value of five caters
to the five trajectories or decision sequences sampled by the RL agent.
Intuitively, we train the agent over these sampled trajectories. The agent
then gradually learns to select the preferred one.

As depicted in Fig. 2, these five (identical) $\mathbf{y}$’s will be passed
into the generator. Accompanied with the same $\mathbf{\epsilon}$, these five
$\mathbf{y}$’s induce five (identical) images. Remind that for a frozen pre-
trained EigenGAN generator, passing in the same $\mathbf{\epsilon}$ and
$\mathbf{z}_{i}$’s deterministic-ally generates the same image. The next step
is the conventional feature extraction process. In this paper, we directly use
a pre-trained ResNet18 model, for which we replace the last fully connected
layer with a counterpart of output dimension 512. The value of 512 is
consistent with the length of $\mathbf{\epsilon}$. We also choose this value
as the default dimension of latent vectors. We do not freeze the other layers
of ResNet18. By contrast, we finetune these weights in the RL training.

The extracted features vectors are the input for a gated recurrent unit (GRU)
cell, of which the size of the hidden state is 512. With the embedding of the
current state, we feed the hidden states of the GRU cell into a fully
connected module. With a softmax layer, the output of the RL module indicates
the action probability distributions. The agent then samples an action. For
example in Fig. 2, it will sample a value for $z_{3,4}$. Recall that there are
five identical images for the first step. So we will sample five (possibly
different) values for $z_{3,4}$.

Next, one obtains five renewed $\mathbf{y}$’s by setting the newly sampled
values for $z_{3,4}$. So these $\mathbf{y}$’s can start to diverge from each
other. Next, the process is repeated, i.e., feature extracting, fully
connected mapping over the updated hidden state of GRU cell, sampling from
distributions given by the softmax layer. The only difference is to randomly
sample the values for $z_{4,5}$, not $z_{3,4}$.

Consider the simplified case in which we only need to set two attributes
$z_{3,4}$ and $z_{4,5}$. At this very moment, the final $\mathbf{y}^{(k)}$’s
with the same $\mathbf{\epsilon}$ determine the final generated images. These
five (probably different) images are input into the aesthetics score network,
generating five scores for RL training.

###  3.4. Soft policy gradient + self-critical

The standard objective in reinforcement learning is the expected sum of
rewards as follows
$\sum_{t}\mathbb{E}_{(\mathbf{s}_{t},\mathbf{a}_{t})\sim\rho_{\pi}}[r(\mathbf{s}_{t},\mathbf{a}_{t})]$.
Most RL algorithms are working on the conventional notion of deterministic
optimality. That is, the optimal policy we are searching for is always
deterministic. And this holds at least for the circumstances of full
observability (Haarnoja et al., 2017). However, intuitively the conventional
objective does not consider the importance of stochasticity for exploration. A
sufficient amount of randomness is critical for searching for near-optimal
policies, preventing premature convergence to suboptimal ones. One way of
keeping randomness is including an entropy regularization term (referred to as
entropy bonus) in the loss function, like in (Mnih et al., 2016; Schulman et
al., 2017). Even so, this is not a systematic way of ensuring the
stochasticity of the policy. In that way, the entropy term serves as a
regularizer, not as one of the objectives of the RL agent.

In this paper, we instead follows the framework of maximum entropy
reinforcement learning (see e.g. (Ziebart, 2010)), which aims for a policy
$\pi^{*}$ as follows:

(2)
$\pi^{*}=\underset{\pi}{\arg\max}\sum_{t}\mathbb{E}_{(\mathbf{s}_{t},\mathbf{a}_{t})\sim\rho_{\pi}}[r(\mathbf{s}_{t},\mathbf{a}_{t})+\alpha\mathcal{H}(\pi(\cdot|\mathbf{s}_{t})],$

where $\mathcal{H}(\pi(\cdot|\mathbf{s}_{t})$ is the entropy, and $\alpha$ is
a hyper-parameter controlling the relative importance of
$\mathcal{H}(\pi(\cdot|\mathbf{s}_{t})$ in the objective.

The next step is to get the gradient of this objective in 2 w.r.t. the policy
parameters. In (Haarnoja et al., 2017), Haarnoja et al.implicitly gave the
following form of the gradient (see Eqs. (27) and (28) in Appendix B of the
arXiv version), and this form is explicitly stated in (Shi et al., 2019). The
result is as follows:

###### Theorem 3.1 (Soft Policy Gradient (Haarnoja et al., 2017; Shi et al.,
2019; Sutton and Barto, 2018)).

Suppose that the policy $\pi(\cdot|\mathbf{s})$ is differentiable w.r.t. its
parameters $\mathbf{\theta}$. Then,

(3)
$\begin{split}&\nabla_{\mathbf{\theta}}J_{\mathrm{PG}}(\pi_{\mathbf{\theta}})\\\
=&\mathbb{E}_{(\mathbf{s}_{t},\mathbf{a}_{t})\sim\rho_{\pi_{\mathbf{\theta}}}}\left[\left(\hat{Q}_{\pi_{\mathbf{\theta}}}(\mathbf{s}_{t},\mathbf{a}_{t})-b(\mathbf{s}_{t})\right.\right.\\\
&\left.-\alpha(1+\log\pi_{\mathbf{\theta}}(\mathbf{a}_{t}|\mathbf{s}_{t}))\vphantom{\hat{Q}_{\pi_{\mathbf{\theta}}}(\mathbf{s}_{t},\mathbf{a}_{t})}\right)\left.\nabla_{\mathbf{\theta}}\log\pi_{\mathbf{\theta}}(\mathbf{a}_{t}|\mathbf{s}_{t})\vphantom{\left(\hat{Q}_{\pi_{\mathbf{\theta}}}(\mathbf{s}_{t},\mathbf{a}_{t})-b(\mathbf{s}_{t})-\alpha(1+\log\pi_{\mathbf{\theta}}(\mathbf{a}_{t}|\mathbf{s}_{t}))\right)}\right]\text{,}\end{split}$

where $\alpha$ is the temperature hyper-parameter controlling the the extent
of exploring, $\hat{Q}_{\pi_{\mathbf{\theta}}}(\mathbf{s}_{t},\mathbf{a}_{t})$
is an empirical estimate of the Q-value of the policy, and $b(\mathbf{s}_{t})$
is a state-dependent baseline, which can be any function as long as it does
not vary with the actions.

With the theorem above, we follow the approach proposed in (Rennie et al.,
2017), i.e., incorporating the self-critical way of training into the soft
policy gradient updates. More concretely,
$\hat{Q}_{\pi_{\mathbf{\theta}}}(\mathbf{s}_{t},\mathbf{a}_{t})$’s are
replaced with Monte Carlo estimations over single trajectories. These
estimations correspond to the final five aesthetics scores given by the
aesthetics score network. This disposes the need for estimations via separate
neural networks for predicting
$\hat{Q}_{\pi_{\mathbf{\theta}}}(\mathbf{s}_{t},\mathbf{a}_{t})$’s. For
$b(\mathbf{s}_{t})$ in 3.1, it is set as the average of the five scores. This
baseline deviates slightly from (Rennie et al., 2017) (which uses the baseline
score got by the greedy policy under current $\pi_{\mathbf{\theta}}$) but
experimentally it is more effective as shown in (Luo, 2020).

In a word, the main difference from conventional self-critical training is
applying the way of self-critical on the soft (not the vanilla) policy
gradient. In the experiment section, we show this change is critical to ensure
sufficient exploration, and thus this guarantees better performance.

###  3.5. Aesthetics score network

We designed a network dedicated to scoring the aesthetics of human faces. We
use the lightweight module ECA (Efficient Channel Attention) (Wang et al.,
2020), which acts as a plug-and-play block, and we apply it to the proposed
network. We adopt EfficientNet-B4 (Tan and Le, 2019) as a pre-trained model to
extract 1,792 features of the image. Then a convolution operation is performed
with a kernel size of 3. We normalize the features before using the ReLU
activation function. Next, it is input to the ECA module to obtain a
4-dimensional vector, and we flatten this vector into a 1-dimensional vector
after activation and adaptive average pooling. After this, it is input into
the regression network, which returns the aesthetics score.

We trained this aesthetic face scoring model above using the SCUT-FBP5500
dataset (Liang et al., 2018). The SCUT-FBP5500 dataset contains a total of
5,500 frontal faces and their beauty scores, including 2,000 Asian females,
2,000 Asian males, 750 Caucasian females, and 750 Caucasian males. The batch
size is 64 for the training, and the initial learning rate is 1e-4. The
training consists of 10 epochs. For validation, we remap the prediction score
of our model to $[1,5]$, which is the beauty score range in SCUT-FBP5500. The
RMSE (root-mean-square error) of our model is 0.3058, which is better than the
best result of 0.3325 presented in the paper of SCUT-FBP5500 (Liang et al.,
2018). The latter adopts a CNN-based ResNeXt-50 architecture.

##  4\. Experiments

All the RL models presented in this section use a batch size of 16. Since the
RL agent samples five random trajectories, the literal batch size is 80 for
each gradient updating step. By fixing the random seed, we maintain the same
validation set across all comparisons. Identical to the experiments conducted
in the original paper of EigenGAN, for the Adam optimizer (Kingma and Ba,
2015; Loshchilov and Hutter, 2019), we use a learning rate of 1e-5 with no
$L_{2}$ regularization over the parameters. For the GRU cell, the size of the
hidden state is 512, which is the same value for the output of the ResNet18
module. The ResNet18 module is a pre-trained version on ImageNet (Deng et al.,
2009). We set the hyperparameter $\alpha$ in 3 as 0.01 for soft policy
gradient.

For the action space, for each semantic attribute, we discretize the range
$[-4.5,4.5]$, taking equally spaced 17 points. It works well, although one
could use continuous values or split into more bins. Nonetheless, we observe
no significant improvement with more fine-grained splits, partially because
the generated images are insensitive to small fluctuations of attribute
values. The final models are versions with 20,000 updating iterations, and we
report the models with the highest scores on the validation set (please refer
to Fig. 3).

Figure 3. A qualitative comparison of aesthetic improvement with different
sets of attributes. Six rows are listed, which indicates six distinct suits of
semantics available. Also, next to each row of illustrated pictures, we type
vertically the average scores of 1,000 randomly generated test images.

###  4.1. Attribute selection

EigenGAN discovers various facial semantic attributes learned in each layer of
subspace, and we wish to analyze their correlations with the aesthetic
quality. We use Spearman’s rank correlation coefficient or Spearman’s $\rho$,
which measures the dependence between two variables by performing linear
correlation analysis on the ranks of two variables. For the analysis here, the
aesthetics score range is $[0,1]$. The searching range of semantic attributes
is $[-4.5,4.5]$. We discretize it with step size 0.5. We calculate Spearman’s
$\rho$ for 30 randomly generated pictures. Correlation coefficients greater
than 0.8 indicate high correlation; We interpret those with values less than
0.3 as irrelevant. In addition, we perform a significance test over these
coefficients. Hypotheses as to this test are $H_{0}:\rho=0$ and
$H_{1}:\rho\neq 0$. We set the threshold of indicating significant difference
as 0.05 for the $p$-values. We list the results in Table 1.

Lastly, we have to choose the dimensions with better decoupling effects, i.e.,
one only observes continuous changes of a _single_ semantic attribute when
traversing the coefficients of those eigen-dimensions. This step is necessary
mainly because, in EigenGAN, different eigen-dimensions might control the same
semantic. For example, changes in $z_{5,2}$ and $z_{5,3}$ both affect the hair
color. To break the ties, we adopt a greedy policy by choosing the more
relevant one. To further narrow the scope, only the attributes with strong
correlation ($|\rho|>0.3$) and also being significant ($p$-value$<0.01$) are
selected. We further filter the candidate set by observing whether they
preserve the facial identities or not. After this step, we choose only five
semantics, as marked boldly in Table 1.

Table 1. Correlation test results of 30 pictures randomly generated by EigenGAN. The notation $z_{i,j}$ indicates the $j^{th}$ dimension of the $i^{th}$ layer. When the $p$-value is less than $10^{-4}$, we treat it as 0.0. For repeated attributes, we choose the most relevant one. The final selected ones are marked in bold. D | Attribute | Spearman’s $\rho$ |  $p$-value
---|---|---|---
$z_{2,5}$ | Body Pose | 0.418 | 0.0
$z_{3,4}$ | Bangs | -0.857 | 0.0
$z_{3,5}$ | Hair Style | -0.125 | 2.875$\times 10^{-3}$
$z_{3,6}$ | Body Side | 0.188 | 0.0
$z_{4,1}$ | Pose | 0.328 | 0.0
$z_{4,2}$ | Background | -0.978 | 0.0
$z_{4,3}$ | Pose | 0.027 | 0.1551
$z_{4,4}$ | Lighting | -0.561 | 0.0
$z_{4,5}$ | Smiling | 0.786 | 0.0
$z_{4,6}$ | Face Shape | -0.720 | 0.0
$z_{5,1}$ | Background | -0.198 | 0.0
$z_{5,2}$ | Hair Color | -0.164 | 0.0
$z_{5,3}$ | Hair Color | 0.685 | 0.0
$z_{5,4}$ | Lighting | -0.061 | 0.1484
$z_{5,5}$ | Gaze | 0.585 | 0.0
$z_{5,6}$ | Lipstick Color | 0.471 | 0.0

###  4.2. Effect of soft policy gradient on entropy

In 3.4, we emphasize the role of soft policy gradient in ensuring the higher
entropy of $\pi_{\mathbf{\theta}}$. Without the soft policy gradient, the main
phenomenon observed is the quick diminishing of entropy, which indicates the
possibility of exploration of diverse choices is gradually becoming zero. For
RL agents, this might induce premature convergence to suboptimal deterministic
policies. To simplify the observation, we only consider the case of setting
only one dimension, i.e., $z_{3,4}$. Thus it is a single-step environment now.
The difficulty of achieving higher scores by changing only $z_{3,4}$ justifies
the choice of this dimension. Fig. 4 depicts the changing of entropy w.r.t.
training iterations. The initial entropy value is slightly higher than 2.82.
As can be observed, if following vanilla policy gradient updates, the average
entropy of distributions defined by
$\pi_{\mathbf{\theta}}(\cdot|\mathbf{s}_{t})$ decreases to values less than
0.3, whereas, by using soft policy gradient, this value is kept still above
2.3. Higher entropy values indicate the fact that there are still
possibilities for exploration. It helps the agents to learn and perform better
finally.

Figure 4. 4 Changing of entropy w.r.t. the number of iterations. 4 Performance
w.r.t. the number of iterations when adjusting more attributes or using soft
policy gradient. Figure 5. Top: Schematic of subjective experiment. Bottom:
Participants’ selection statistics for five variants of the proposed model. By
fixing the latent variables, these five variants, namely $M_{\text{b}}$,
$M_{\text{l}}$, $M_{\text{bs}}$, $M_{\text{ll}}$, and $M_{\text{aes}}$,
generate 100 groups, 600 images in total. Participants chose the best-looking
among each group. $M_{\text{aes}}$ receives the highest votes.

###  4.3. Improved performance with soft policy gradient and more attributes

In this section, we extend the former one-step environment to a counterpart of
two steps, i.e., $z_{3,4}$ and $z_{4,5}$. So one can observe the improved
performance brought by considering more attributes adjustable. Meanwhile, we
also show the results on using soft policy gradient or not. Intuitively higher
entropy enables the possibility of still exploring the environment and thus
getting higher scores. We verify this intuition in the experiment. We note
higher entropy of $\pi_{\mathbf{\theta}}(\mathbf{a}_{t}|\mathbf{s}_{t})$ does
not necessarily mean the agent cannot perform optimally. By contrast, as in
the inference phase, we only select
$\mathbf{a}^{*}={\arg\max}_{\mathbf{a}_{t}}\pi_{\mathbf{\theta}}(\cdot|\mathbf{s}_{t})$,
so as long as preferred actions still dominate the probability mass,
intuitively the agent still performs optimally. We adopt a randomly generated
set of 80 images for validation. The original average score of these images is
0.54530, shown in Fig. 4 for iteration 0. Consider the increase of scores by
adjusting one more attribute. If using vanilla policy gradient, for one-step
setting, the validation score is improved to 0.55991; for two-steps, this
value is 0.58539. Meanwhile, for soft policy gradient, these two values are
0.56649 and 0.60051 resp. So this indicates higher scores with more attributes
adjustable. Also, most of the time, the curves concerning soft policy gradient
are above the curves of vanilla policy gradient, no matter the environment is
one-step or two-steps. This observation verifies the advantage of the soft
version of policy gradient.

###  4.4. Experimental analysis of combinations of attributes

We evaluate five variants of the proposed method, called $M_{\text{b}}$
(_bangs_), $M_{\text{l}}$ (_lighting_), $M_{\text{bs}}$ (_bangs & smile_),
$M_{\text{ll}}$ (_lighting & lipstick color_), and $M_{\text{aes}}$ (_joint
control over full five attributes_). Fig. 3 shows the qualitative comparison
of the face images obtained by these five variants, which are tested over a
total of 1,000 images, and six of which are presented as an illustration. Most
images changed by these variants are distinct from the original ones.
Moreover, they are all with promoted average aesthetics scores. We use FID to
evaluate the quality of these images. The FID value of the image generated by
$M_{\text{aes}}$ is 47.11, and the FID value of the initially generated image
is 41.07, indicating that the attribute adjustment does not make the quality
better, just makes them more beautiful. Also, Fig. 3 shows that controlling
the brightness achieves a higher average score than altering the bangs, which
indicates that the enhancement differs with distinct semantics. In the fourth
row, we observe mouthes of the faces change significantly, from bright smiles
to slightly pursed mouthes. Intuitively, as grinning faces have been scarce in
SCUT-FBP5500, this might give misleading reward signals to the RL agent. In
addition, as Asian faces dominate SCUT-FBP5500 up to 72%, the aesthetics score
network tends to prefer darker hair colors, and hence we remove this semantic
in $M_{\text{aes}}$. Also, we observe $M_{\text{aes}}$ alters the images in a
way, not via a straightforward composing of the changes in each separate
single-step model. For example, for the first column, the image generated by
$M_{\text{aes}}$ is different from several other variant models in terms of
bangs, lighting, lipstick color, body pose, and gaze. For $M_{\text{aes}}$, we
observe that for lighting, it tends to choose the butterfly light, which
highlights the characters and reduces the environmental impact. Some changes
are not in line with common perceptions, such as some male faces choosing
reddish lipstick colors, partly because the generator lacks gender perception.

###  4.5. Subjective analysis

Because of the subjectivity of image aesthetics, humans show diverse
characteristics in their aesthetic thinking. Even on commonly viewed
attractive ones, people might differ subtly in their preferences. We,
therefore, conduct a subjective comparative experiment on the five variants
presented in the previous section. We recruit eight users to participate in
the subjective study. All users do not know the experiment contents in
advance. In the subjective study, we compared the face images obtained by
EigenGAN and our five variants, namely $M_{\text{b}}$,
$M_{\text{l}}$,$M_{\text{bs}}$, $M_{\text{ll}}$ , and $M_{\text{aes}}$. We
show 100 groups of images to all participants. The images generated by the
same latent variables are a group, and we display each group of images in
random order. The user selects the best image by clicking the screen without
limiting the selection time. Fig. 5 shows that $M_{\text{aes}}$ has obtained
the most votes, which indicates its effective controlling of various
attributes of the face.

##  5\. Conclusions

In this paper, by applying a novel variant of the ingredients in RL, we
propose an RL-based model which can learn to precisely control the semantics
underlying the generator of EigenGAN. We experimentally show the resulting
model can alter the images in an aesthetically more attractive way. In future
work, we will verify the efficacy of the proposed method on real-world image
editing. A straightforward way for this task is to first recover the latent
codes in EigenGAN corresponding to the input images.

Limitations. The work presented in this paper largely depends on how well
EigenGAN can discover disentangled semantics and can reveal these
interpretable factors explicitly in the generators. However, we have to
manually observe the continuous changes by traversing the coefficient of
specific eigen-dimensions. This step helps us filter out these semantics (such
as gender, age, race, etc.) that drastically alter the identities of the
original images. Another drawback of the method here is the quality of RL
policy largely depends on the facial beauty prediction network, this might
misleading the RL agent.

###### Acknowledgements.

This work is partially supported by the National Natural Science Foundation of
China (62072014 & 62106118), the CAAI-Huawei MindSpore Open Fund
(CAAIXSJLJJ-2021-022A), the Open Fund Project of the State Key Laboratory of
Complex System Management and Control (2022111), the Project of Philosophy and
Social Science Research, Ministry of Education of China (20YJC760115), and the
Advanced Discipline Construction Project of Beijing Universities
(20210051Z0401). We gratefully acknowledge the support of MindSpore, CANN and
Ascend AI Processor used for this research.

## References

  * (1)
  * Chen et al. (2016) Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. 2016\.  Infogan: Interpretable representation learning by information maximizing generative adversarial nets. In _Proceedings of the 30th International Conference on Neural Information Processing Systems_. Curran Associates Inc., Red Hook, NY, USA, 2180–2188.
  * Deng et al. (2009) Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009\.  ImageNet: A large-scale hierarchical image database. In _2009 IEEE Conference on Computer Vision and Pattern Recognition_. 248–255.
  * Diamant et al. (2019) Nir Diamant, Dean Zadok, Chaim Baskin, Eli Schwartz, and Alexander M. Bronstein. 2019.  Beholder-GAN: Generation and Beautification of Facial Images with Conditioning on Their Beauty Level.  _CoRR_ abs/1902.02593 (2019).  arXiv:1902.02593 http://arxiv.org/abs/1902.02593
  * Goodfellow et al. (2014) Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014\.  Generative Adversarial Nets. In _Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2_ (Montreal, Canada) _(NIPS’14)_. MIT Press, Cambridge, MA, USA, 2672–2680.
  * Haarnoja et al. (2017) Tuomas Haarnoja, Haoran Tang, Pieter Abbeel, and Sergey Levine. 2017.  Reinforcement learning with deep energy-based policies. In _International Conference on Machine Learning_. PMLR, 1352–1361.
  * He et al. (2021) Zhenliang He, Meina Kan, and Shiguang Shan. 2021.  EigenGAN: Layer-Wise Eigen-Learning for GANs. In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_. IEEE, Montreal, QC, Canada, 14408–14417.
  * Karras et al. (2018) Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2018\.  Progressive Growing of GANs for Improved Quality, Stability, and Variation. In _6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings_. OpenReview.net.  https://openreview.net/forum?id=Hk99zCeAb
  * Karras et al. (2019) Tero Karras, Samuli Laine, and Timo Aila. 2019.  A style-based generator architecture for generative adversarial networks. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. IEEE, Long Beach, CA, USA, 4401–4410.
  * Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015.  Adam: A Method for Stochastic Optimization. In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_ , Yoshua Bengio and Yann LeCun (Eds.).  http://arxiv.org/abs/1412.6980
  * Li et al. (2018) Debang Li, Huikai Wu, Junge Zhang, and Kaiqi Huang. 2018\.  A2-RL: Aesthetics Aware Reinforcement Learning for Image Cropping. In _IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 8193–8201.  https://doi.org/10.1109/CVPR.2018.00855
  * Li et al. (2019) Debang Li, Huikai Wu, Junge Zhang, and Kaiqi Huang. 2019\.  Fast A3RL: Aesthetics-Aware Adversarial Reinforcement Learning for Image Cropping.  _IEEE Transactions on Image Processing_ 28, 10 (2019), 5105–5120.  https://doi.org/10.1109/TIP.2019.2914360
  * Liang et al. (2018) Lingyu Liang, Luojun Lin, Lianwen Jin, Duorui Xie, and Mengru Li. 2018.  Scut-fbp5500: A diverse benchmark dataset for multi-paradigm facial beauty prediction. In _2018 24th International Conference on Pattern Recognition (ICPR)_. IEEE, Beijing, China, 1598–1603.
  * Loshchilov and Hutter (2019) Ilya Loshchilov and Frank Hutter. 2019.  Decoupled Weight Decay Regularization. In _7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019_. OpenReview.net.  https://openreview.net/forum?id=Bkg6RiCqY7
  * Luo (2020) Ruotian Luo. 2020\.  A Better Variant of Self-Critical Sequence Training.  _CoRR_ abs/2003.09971 (2020).  arXiv:2003.09971 https://arxiv.org/abs/2003.09971
  * MindSpore (2022) MindSpore. 2022\.  https://www.mindspore.cn/.
  * Mirza and Osindero (2014) Mehdi Mirza and Simon Osindero. 2014.  Conditional generative adversarial nets.  _arXiv preprint arXiv:1411.1784_ (2014).
  * Mnih et al. (2016) Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016.  Asynchronous methods for deep reinforcement learning. In _International conference on machine learning_. PMLR, 1928–1937.
  * Pan et al. (2020) Yingwei Pan, Ting Yao, Yehao Li, and Tao Mei. 2020\.  X-linear attention networks for image captioning. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 10971–10980.
  * Rennie et al. (2017) Steven J. Rennie, Etienne Marcheret, Youssef Mroueh, Jerret Ross, and Vaibhava Goel. 2017\.  Self-Critical Sequence Training for Image Captioning. In _IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE, Honolulu, HI, USA, 1179–1195.
  * Schulman et al. (2017) John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017.  Proximal policy optimization algorithms.  _arXiv preprint arXiv:1707.06347_ (2017).
  * Shen et al. (2020) Yujun Shen, Jinjin Gu, Xiaoou Tang, and Bolei Zhou. 2020\.  Interpreting the Latent Space of GANs for Semantic Face Editing. In _IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. 9240–9249.  https://doi.org/10.1109/CVPR42600.2020.00926
  * Shi et al. (2019) Wenjie Shi, Shiji Song, and Cheng Wu. 2019\.  Soft Policy Gradient Method for Maximum Entropy Deep Reinforcement Learning.. In _Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence_. 3425–3431.
  * Sutton and Barto (2018) Richard S. Sutton and Andrew G. Barto. 2018.  _Reinforcement Learning: An Introduction_.  A Bradford Book, Cambridge, MA, USA.
  * Tan and Le (2019) Mingxing Tan and Quoc Le. 2019.  Efficientnet: Rethinking model scaling for convolutional neural networks. In _International conference on machine learning_. PMLR, 6105–6114.
  * Wang et al. (2020) Qilong Wang, Banggu Wu, Pengfei Zhu, Peihua Li, Wangmeng Zuo, and Qinghua Hu. 2020\.  ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. In _IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE, Seattle, WA, USA, 11531–11539.  https://doi.org/10.1109/CVPR42600.2020.01155
  * Zhang et al. (2021) Pengchuan Zhang, Xiujun Li, Xiaowei Hu, Jianwei Yang, Lei Zhang, Lijuan Wang, Yejin Choi, and Jianfeng Gao. 2021\.  Vinvl: Revisiting visual representations in vision-language models. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 5579–5588.
  * Zhou and Xiao (2020) Yuhongze Zhou and Qinjie Xiao. 2020.  GAN-Based Facial Attractiveness Enhancement.  _CoRR_ abs/2006.02766 (2020).  arXiv:2006.02766 https://arxiv.org/abs/2006.02766
  * Ziebart (2010) Brian D Ziebart. 2010\.  _Modeling purposeful adaptive behavior with the principle of maximum causal entropy_.  Carnegie Mellon University, Pittsburgh, PA 15213.