# MoSAR: Monocular Semi-Supervised Model for Avatar Reconstruction using
Differentiable Shading

Abdallah Dib1 Luiz Gustavo Hafemann1∗ Emeline Got1 Trevor Anderson1 Amin
Fadaeinejad1,2 Rafael M. O. Cruz3 Marc-André Carbonneau1

1Ubisoft LaForge York University2 Ecole de Technologie Supérieure3 Equal
contribution

###### Abstract

Reconstructing an avatar from a portrait image has many applications in
multimedia, but remains a challenging research problem. Extracting reflectance
maps and geometry from one image is ill-posed: recovering geometry is a one-
to-many mapping problem and reflectance and light are difficult to
disentangle. Accurate geometry and reflectance can be captured under the
controlled conditions of a light stage, but it is costly to acquire large
datasets in this fashion. Moreover, training solely with this type of data
leads to poor generalization with in-the-wild images. This motivates the
introduction of MoSAR, a method for 3D avatar generation from monocular
images. We propose a semi-supervised training scheme that improves
generalization by learning from both light stage and in-the-wild datasets.
This is achieved using a novel differentiable shading formulation. We show
that our approach effectively disentangles the intrinsic face parameters,
producing relightable avatars. As a result, MoSAR estimates a richer set of
skin reflectance maps, and generates more realistic avatars than existing
state-of-the-art methods. We also introduce a new dataset, named _FFHQ-UV-
Intrinsics_ , the first public dataset providing intrinsic face attributes at
scale (diffuse, specular, ambient occlusion and translucency maps) for a total
of 10k subjects. The project website and the dataset are available on the
following link: https://ubisoft-laforge.github.io/character/mosar

Figure 1:  Our method estimates detailed geometry and reflectance maps,
yielding convincing rendering under new lighting conditions.

##  1 Introduction

Avatars are an important component of virtual worlds, being widely used in
virtual reality, multimedia and video-games. Realistic personalized avatars,
often called digital doubles, serve as bridge between the physical world and
digital realities. The light stage [12] has long been regarded as the primary
solution to obtain high-quality realistic avatars, however their high cost
restricts their availability to the general public. They also require physical
presence of users for scanning, which is impractical for many applications.

In contrast, creating avatars from a single monocular image enables a wide
range of applications [51, 52, 53, 15, 17, 18, 10, 63, 6, 32]. These methods
are fast and robust to arbitrary capture conditions, however, they fall short
in terms of quality when compared to high-level multimedia production
standards.

Moreover, these approaches often produce avatars that cannot seamlessly be
integrated in tools used by content creators and artists, which limits the
ability to edit the captured avatars. For example, modern graphic engines
render realistic faces by relying on advanced materials and texture maps that
these methods do not support. Even state-of-the-art methods [60, 28, 30, 29,
38] are limited to estimating diffuse, specular and normal maps, while modern
engines use a broader range of maps including ambient occlusion and
translucency to simulate global illumination and sub-surface scattering.

In this paper, we propose a new method for generating 3D avatars from a single
monocular image. It estimates rich reflectance maps including ambient
occlusion and translucency, in addition to the conventional diffuse, specular
and normal maps, all at a 4K resolution. Our method can separate light
contributions from the ambient occlusion and translucency from the diffuse
maps because it relies on high-quality light stage data. It generalizes well
to uncontrolled capture conditions because it also trains on a large quantity
of in-the-wild monocular images using self-supervision. This is made possible
by our new differentiable shading formulation. This paper makes the following
main contributions:

  * •

We propose a semi-supervised training scheme leveraging high-quality light
stage data, and large quantities of in-the-wild images. This allows for the
estimation of a rich set of reflectance maps, as well as accurate geometry.

  * •

We establish non-linear morphable models as a promising avenue to recover
geometry from a single image by ranking in 2nd place on the REALY benchmark
[4], so far largely dominated by linear models.

  * •

We introduce a new differentiable shading formulation that incorporates, for
the first time, the ambient occlusion and translucency maps with the spherical
harmonics lighting model.

These contributions culminate in a more accurate and detailed head model than
other state-of-the-art monocular avatar reconstruction methods. Figure MoSAR:
Monocular Semi-Supervised Model for Avatar Reconstruction using Differentiable
Shading shows results from our method. In addition to our method’s
contribution, we introduce a new dataset, named _FFHQ-UV-Intrinsics_ , the
first public dataset providing intrisic face attributes at scale (diffuse,
specular, ambient occlusion and translucency maps) for a total of 10k
subjects. We built it by applying our method to the dataset _FFHQ-UV_[2].

##  2 Related works

Figure 2: Overview of the proposed pipeline. In a first step, the 3D geometry
of the face is estimated, and the image is projected onto the UV space. This
texture is then used to estimate reflectance and displacement maps in 4K in a
series of steps describe in Section 3.

#### 3D Geometry reconstruction

Face geometry estimation from images has been widely investigated in the
literature [64]. Accurate geometry estimation can be obtained in controlled
capture settings with multiple cameras (e.g. in a light stage [12]), but this
is costly and labor intensive, which makes it impractical for many use cases.
This motivated research on geometry recovery from a single image, which
enables a wider variety of applications.

Monocular face reconstruction is commonly treated in a self-supervised way
[64], by modeling a parametric scene with geometry, light, reflectance and
camera parameters. These are used in conjunction with a differentiable
renderer [26], enabling learning from images only. There are three main types
of approaches for the parametric geometry model: (i) using linear morphable
models (3DMM), (ii) learning unconstrained geometry, and (iii) learning non-
linear 3DMMs.

Methods based on linear 3DMMs [51, 18, 62, 14, 11, 47, 16, 5, 32] are bound by
the statistical prior space of the 3DMM, which restricts their expressiveness.
To go beyond this prior, methods such as [49, 45, 61] estimate unrestricted
geometry. These methods output dense meshes that require a registration
process to a standard topology in order to be used in most applications.

Other methods such as [55, 56] learn a non-linear morphable model. These
methods train a decoder that estimates the final geometry under the weak
supervision of another decoder that predicts 3DMM coefficients. The face
geometry produced by these methods often contain artifacts as noted in [16].
Graph neural networks (GNN) have been used to refine the geometry obtained
from 3DMMs [34, 19], or to directly estimate geometry and reflectance [31]. In
[1], a non-linear part-based GNN auto-encoder is trained for local shape
editing.

Our method is not restricted nor regularized by linear 3DMMs, and instead
implements a non-linear morphable model using GNN auto-encoders [44], inspired
by [31] and [1]. Our proposed semi-supervised training scheme and loss
functions mitigate artifact problems in the geometry, while still keeping it
unbounded.

#### Skin reflectance

To reconstruct a realistic and relightable avatar representation from an
image, the intrinsic skin reflectance attributes must be disentangled.
Recovering face attributes from single in-the-wild images is an ill-posed
problem. Most existing methods [51, 18] use a Lambertian Bidirectional
Reflectance Distribution Function (BRDF) [8] and a statistical basis, usually
a PCA, for albedo estimation, which largely limits their expressivity.

Some methods relax the constraint imposed by using a statistical basis. For
instance in [17, 16, 15], skin reflectance is modeled using a Microfacet BRDF
[54], with diffuse and specular components recovered using only a weak PCA
regularization. Nevertheless, this leads to blurry textures, and fails to
capture fine-grained details of the skin. Lee et al.[31] learn an unbounded
skin reflectance model, however the estimated texture map bakes light,
geometry and skin color (diffuse and specular) making it unsuitable for
relighting. Finally, other methods [60, 28, 29, 30, 38] train a conditional
GAN to estimate diffuse, specular, and normal maps for relightable avatars,
but use only light stage data.

Our proposed method extends the Microfacet BRDF and estimates a richer skin
reflectance than existing methods. It produces Ambient Occlusion and
Translucency maps, all in 4K, enabling more realistic avatars. These maps are
also compatible with modern technical artist tools and rendering engines. Our
semi-supervised training scheme, that uses a novel differentiable shading
formulation, allows for disentangling diffuse from ambient occlusion, as well
as specular and translucency maps, from in-the-wild images.

##  3 Method

An overview of our method is shown in Figure 2. First, the _3D shape
reconstruction_ module (Section 3.1) estimates the geometry and camera
position used to project the input image onto UV space. Then, the _texture
completion_ network (Section 3.2) estimates the missing parts of the texture.
Next, light is normalized across the entire face (Section 3.3) and the
_intrinsic texture map estimation_ (Section 3.4) recovers various texture maps
(i.e diffuse, specular, displacement, ambient occlusion and translucency).
Finally, a _super-resolution_ network upscales the maps to 4K. We train this
architecture in a semi-supervised manner, using (i) light stage data, that
contains raw images, ground truth geometry and reflectance maps, and (ii) in-
the-wild images, for which we do not have ground truth.

###  3.1 3D Shape Reconstruction

Figure 3: 3D Face reconstruction pipeline: An encoder $\mathcal{E}$ estimates
the scene parameters $\chi$, used by a Differential Renderer (DR) to obtain
the estimated vertex colors, displayed here as an image.

We represent the scene with a parametric model, defined by
$\chi=\\{\boldsymbol{\alpha},\boldsymbol{\delta},\boldsymbol{\beta},\boldsymbol{\gamma},\boldsymbol{\phi}\\}$,
with neutral shape (identity) parameters $\boldsymbol{\alpha}$, expression
coefficients $\boldsymbol{\delta}$, reflectance coefficients
$\boldsymbol{\beta}$, light coefficients $\boldsymbol{\gamma}$ and camera
parameters $\boldsymbol{\phi}$, as shown in Figure 3. This decomposition is
similar to previous work [20, 51, 15], but we propose a novel procedure to
learn a non-linear representation of identity.

We use a part-based GNN model inspired by [44, 1] to generate a detailed
neutral mesh. While this method was proposed for artist-controlled geometry
editing, our proposed semi-supervised training procedure makes it effective
for monocular face reconstruction, improving modeling accuracy when compared
to linear 3DMMs.

The GNN model is composed of a collection of encoders $\mathcal{G}_{i}$ each
responsible for translating the vertices of a specific part of the face into a
latent representation $\boldsymbol{\alpha}_{i}$ where $i\in[0,\ldots,7]$.
These are concatenated in a single vector $\boldsymbol{\alpha}$, and fed to a
decoder $\mathcal{F}(\boldsymbol{\alpha})$, that reconstructs the entire input
mesh. We use the same architecture as [1].

We represent facial expressions as a set of linear displacement blendshapes
$\boldsymbol{E}$. Given parameters $\boldsymbol{\alpha}$ and
$\boldsymbol{\delta}$, we obtain the geometry as
$\hat{V}=\mathcal{F}(\boldsymbol{\alpha})+\sum_{i}{\delta_{i}E_{i}}$, where
$E_{i}$ is the $i$th blendshape in the set.

For reflectance, we use a Lambertian BRDF model, with a PCA basis on per-
vertex albedo colors. We model light using 3rd-order spherical harmonics, and
we use a pinhole camera model.

#### Training

We train two components at this stage: the GNN model producing neutral meshes
and a scene-encoder $\mathcal{E}$ translating images into scene parameters
$\chi$. We first pre-train the GNN model in a supervised way with the light
stage data, following the work of [1]. Next, we train the scene encoder
$\mathcal{E}$ and the GNN decoder $\mathcal{F}$ in a semi-supervised manner.
We combine the in-the-wild data with light stage data, for which we have
ground truth vertex positions used for supervision.

When training with in-the-wild data, we rely solely on self-supervision. The
differentiable renderer uses the estimated geometry $\hat{V}$, reflectance
coefficients $\boldsymbol{\beta}$, light coefficients $\boldsymbol{\gamma}$
and camera parameters $\boldsymbol{\phi}$ to compute the final color
(irradiance) for each vertex, as in [51]. These are then compared to their
corresponding pixels in the input image $X$. Additionally, we extract
landmarks $l$ using an off-the-shelf detector [3]. During training, we
minimize the following loss function:

$\mathcal{L}_{u}=L_{\text{photo}}+L_{\text{landmark}}+L_{\text{reg}},$ (1)

where the photo-consistency loss $L_{\text{photo}}$ is the $\ell_{1}$-norm
between the predicted vertex colors and their associated pixel colors in the
input image; $L_{\text{landmark}}$ is the $\ell_{2}$-norm of the distance
between the landmarks $l$ and the perspective projection of their associated
mesh vertices. $L_{\text{reg}}$ is the regularization term defined as:

$L_{\text{reg}}=L_{\text{lap}}+L_{\text{light}}+L_{\text{exp}}+L_{\text{alb}}.$
(2)

$L_{\text{lap}}$ is the Laplacian smoothing operator from [21] applied to the
estimated mesh $\hat{V}$, which minimizes the mean surface curvature.
$L_{\text{exp}}$ is the $\ell_{1}$-norm of the expression coefficient vector
acting as a regularization term. $L_{\text{light}}$ encourages monochromatic
light (similar to [15]), while $L_{\text{alb}}$ regularizes against
implausible face deformations using prior albedo statistics as defined in
[51]. We note that the term $L_{\text{lap}}$ if required since $\mathcal{F}$
is not restricted to a parametric model, and thus requires regularization to
avoid producing implausible geometry.

When training with light stage data, we leverage the ground-truth mesh $v_{i}$
associated with the image $X_{i}$. In that case we complement
$\mathcal{L}_{u}$ with $L_{\text{supervised}}$ and $L_{\text{nrm}}$ which are
respectively the $\ell_{2}$-distance between the vertices of the estimated and
GT mesh, and the cosine distance between their normal vectors:

$\mathcal{L}_{s}=\mathcal{L}_{u}+L_{\text{supervised}}+L_{\text{nrm}}.$ (3)

We omitted weighting coefficients ($\lambda$) of the loss term for better
readability. They are specified in the supplementary material.

Finally, at inference time, we further refine the scene parameters for the
input image. Similarly to [15], we directly optimize the parameters $\chi$ to
minimize Equation 1, using the Adam optimizer [27], with the values given by
the scene encoder $\chi=\mathcal{E}(X)$ as initialization.

###  3.2 Texture completion

We project the image onto the UV space using the estimated geometry and camera
information. This produces an incomplete texture because certain regions of
the face are occluded, as illustrated in Figure 2.

Similarly to [13], we estimate a complete texture map from an incomplete map
alongside its mirrored version. To train this model, we create synthetic
textures simulating in-the-wild settings from light stage data. We render
faces from random camera angles and HDR maps, then project onto UV space. This
creates self-occlusions and non-visible areas similar to real images. We train
an image-to-image translation network, by minimizing the $\ell_{1}$-distance
between the estimated texture and the ground truth.

###  3.3 Light normalization

Figure 4: Light normalization step: We render faces in UV space with random
light and camera positions. The network then estimates a light-normalized map
$\mathcal{M}$.

The light normalization step aims to produce uniform light on the subject,
mimicking the controlled light conditions of the light stage. This enables
using simple light models for the remaining of the pipeline, such as Spherical
Harmonics [42, 50], even for in-the-wild images, that can present complex
lighting conditions. If this step is omitted, unexplained light contributions
(e.g. strong directional lights) end up baked in the final estimated
reflectance maps.

Figure 4 illustrates the training procedure for the light normalization
network. We first perform the shading in UV space, using the mesh geometry $V$
and its associated texture maps. We use random environment maps, camera
positions with the rendering Equation 5. The resulting texture is fed to a
U-Net architecture that generates a normalized texture $\mathcal{M}$. The
network is trained to minimize the $\ell_{1}$-distance between the estimated
normalized texture and the target ground truth. The target texture is obtained
by projecting the images from all light stage camera views onto UV space, then
aggregating them in a single image.

###  3.4 Intrinsic texture maps estimation

The normalized texture $\mathcal{M}$ and the estimated geometry $V$ are used
to estimate different face attribute maps: Diffuse $\mathcal{D}$, Specular
$\mathcal{S}$, Ambient Occlusion $\mathcal{A}$, Translucency $\mathcal{T}$ and
Normal map $\mathcal{N}$, as depicted in Figure 5.

Figure 5: Texture estimation pipeline: Reflectance and normal maps are
estimated by reconstructing the light-normalized texture $\mathcal{M}$ using
Differentiable Shading (DS).

A separate network is used to estimate each of the reflectance maps. This
architecture could be trained using only light stage data by supervising the
output of each network. To achieve better generalization outside the light
stage, we seek training on in-the-wild images as well. For this, we add an
encoder $C$ to estimate the residual light $\hat{\gamma}$ and the camera
position $\hat{\phi}$, and introduce a new differentiable shading formulation
(DS in Figure 5). DS outputs a new shaded texture $\hat{\mathcal{M}}$ which
can then be compared to $\mathcal{M}$. This allows our model to be trained in
a semi-supervised manner, combining light stage and in-the-wild datasets. The
encoder $C$ captures residual light that was not removed during the Light
Normalization step (Section 3.3). This prevents the model from baking this
light into one of the texture maps.

To estimate the normal map $\mathcal{N}$, we first predict the displacement
map following a patch-based approach as in [7]. First, a PCA basis is built on
patches of displacement maps from the light stage data. A coarse displacement
encoder (refer to Figure 5), estimates the PCA coefficients that best
reproduce a patch of the texture $\mathcal{M}$. This PCA captures repeated
patterns (e.g. wrinkles) and low to medium frequency details. This
approximation is further refined by a second U-Net model that complements the
patch with high frequency details (e.g. skin pores and moles). Finally, the
normal map is obtained from the displacement map using a Fast Fourier
convolution.

To estimate the remaining maps
($\mathcal{D},\mathcal{S},\mathcal{A},\mathcal{T}$), we use image-to-image
translation networks. For the diffuse $\mathcal{D}$ and specular
$\mathcal{S}$, the generators predict the maps from the normalized texture
image $\mathcal{M}$. For $\mathcal{A}$ and translucency $\mathcal{T}$, the
generators also use the geometry $V$ projected onto UV space. We use
adversarial training, by defining a discriminator for each texture map. These
networks are trained jointly, with the following loss:

$\displaystyle\mathcal{L}=L_{\text{shading}}+\sum_{\mathcal{D},\mathcal{S},\mathcal{A},\mathcal{T}}{\left(L_{\text{GAN}}(G,D)+L_{\text{sup}}\right)},$
(4)

where $L_{\text{shading}}$ is the $\ell_{1}$ reconstruction loss between
$\mathcal{M}$ and $\hat{\mathcal{M}}$, $L_{\text{GAN}}(G,D)$ is the
adversarial loss described in [23], applied for each texture map separately.
Finally, when training with light stage data, we also compute $L_{\text{sup}}$
as the $\ell_{1}$ loss between the ground truth and predicted maps. We omitted
weighting coefficients ($\lambda$) of the loss term for better readability.
They are specified in the supplementary material.

#### Differentiable shading.

The differentiable shading _DS_ (in Figure 5), takes the predicted textures
($\mathcal{D}$, $\mathcal{S}$, $\mathcal{A}$, $\mathcal{T}$ and
$\mathcal{N}$), camera parameters $\hat{\phi}$, light coefficients
$\hat{\gamma}$ and outputs a new texture map $\hat{\mathcal{M}}$. We formulate
our shading equation, for each pixel $\hat{m}$ in $\hat{\mathcal{M}}$,
extending the Microfacet BRDF model [54] as follows:

$\displaystyle\hat{m}=\mathcal{B}_{d}+\mathcal{B}_{sss}+\mathcal{B}_{s},$ (5)

where $\mathcal{B}_{d}$, $\mathcal{B}_{sss}$ and $\mathcal{B}_{s}$ are the
diffuse, subsurface scattering and specular contribution respectively,
described in the equations below.

To model the diffuse component of the skin $\mathcal{B}_{d}$, we extend the
diffuse Lambertian model with an additional term $a$ that captures residual
diffuse ambient occlusion in areas that receive less light such as wrinkles,
folds and nostrils. $\mathcal{B}_{d}$ is calculated as follows:

$\mathcal{B}_{d}=d\cdot
a\cdot\sum_{l=0}^{2}\sum_{m=-l}^{l}A_{l}\cdot\hat{\gamma}^{m}_{l}\cdot
Y^{m}_{l}(n),$ (6)

where $d$, is the diffuse albedo obtained from the diffuse $\mathcal{D}$ map,
and $a$ is the residual ambient occlusion term from $\mathcal{A}$. $A_{l}$ are
the Lambertian BRDF coefficients [41, 42]. $\hat{\gamma}$ are the SH
coefficients, and $Y^{m}_{l}$ are the SH basis [42]. $n$ is the pixel normal
(using normal mapping). For the diffuse component, we use 3rd-order spherical
harmonics ($l=2$) as it has been shown in [41] that it captures 99% of the
reflected radiance.

Next, we introduce a new subsurface scattering contribution
$\mathcal{B}_{sss}$ which indicates where thinner parts of the face (e.g.
nose, lips and ears) scatter more light than other thicker areas, defined as:

$\displaystyle\mathcal{B}_{sss}=d\cdot\sum_{l=1}^{2}\sum_{m=-l}^{l}S_{l}\cdot\hat{\gamma}^{m}_{l}\cdot
Y^{m}_{l}(n),$ (7)

with $S_{l}=e^{-l^{2}/t^{4}}$, with ${t}$ being the translucency value from
$\mathcal{T}$. Intuitively, thicker parts of the face would have a low value
in $\mathcal{T}$ which in turn would negate the $\mathcal{B}_{sss}$
contribution through $S_{l}$ nearing $0$.

Finally, we compute the specular component $\mathcal{B}_{s}$ as the spatial
convolution of the SH light representation with the BRDF roughness kernel.
This kernel is constant in the simplified Microfacet BRDF model that we use.
Thus our specular contribution $\mathcal{B}_{s}$ is defined as:

$\displaystyle\mathcal{B}_{s}=f\cdot\sum_{l=0}^{8}\sum_{m=-l}^{l}R_{l}\cdot\hat{\gamma}^{m}_{l}\cdot
Y^{m}_{l}(r),$ (8)

where $f=s+(1-s)(1-\cos{\theta})^{5}$ is the Fresnel reflection [57]
calculated using Schlick’s approximation [48]. $f$ quantifies the proportion
of light that is reflected and depends on the angle at which light hits the
surface. $\theta$ is approximated with the angle between the normal vector and
the camera view direction [43]. $s\in\mathbb{R}$ is specular intensity from
$\mathcal{S}$. $r_{i}$ is the reflection vector of the viewing vector
(obtained from the camera position $\hat{\phi}$) according to the surface
normal [57], and $R_{l}$ are the SH coefficients of the BRDF function
corresponding to the roughness [43, 35]. For the specular contribution, we use
9th-order order spherical harmonics ($l=8$) for better approximation of the
specular reflection [16].

Equation 5 is fully differentiable with respect to all texture maps, allowing
for gradient-based optimization of the generators. We note that the diffuse
$d$ and ambient occlusion $a$ have the same contribution, and therefore cannot
be separated using only self-supervised training. For instance, the model can
bake all the information in a single diffuse map which results in a sub-
optimal separation. Our semi-supervised training scheme (Equation 4) allows
for this separation, by leveraging ground truth data from light stage in
conjunction with the unlabeled data.

###  3.5 Super-resolution

We use super-resolution at the last step, to obtain 4K texture maps. We train
two ESRGAN models [58]: one that upsamples the input to 1K and one that
upsamples to 4K, as in [33]. Both are trained with data from the light stage.

##  4 Experimental protocol

#### Datasets

The light stage dataset is composed of 890 subjects, captured with a neutral
pose. For every subject, 12 cameras capture frontal and side views. The
geometry is obtained using a Multi-View reconstruction pipeline, followed by
registration to a standard topology. Each subject has diffuse, specular,
displacement, ambient occlusion and translucency maps in 4K. We keep 50
subjects for validation, and use the remaining for training.

We train the 3D reconstruction model with the light stage dataset and 70k
images from the FFHQ dataset [24]. We crop and resize the images to the
resolution of $512\times 512$, we apply standard data augmentation
(brightness, rotation, scale and flip). For texture estimation, we use the
light stage dataset and FFHQ-UV [2], that contains 54k textures obtained from
StyleGAN [25]. We normalize the FFHQ-UV textures using our light normalization
network (Section 3.3). The textures are processed in resolution $512\times
512$.

#### Training

For the 3D geometry estimation (Section 3.1), $\mathcal{E}$ is a ResNet-50
[22] while for $\mathcal{F}$ we use the same architecture as [1] from their
open-source implementation. We train both networks for 20 epochs, using batch
size of 32 images, equally divided between light stage and in-the-wild data.

The texture completion network is trained for 20 epochs on the light stage
data following the data augmentation described in Section 3.2. We train the
light normalization network for 20 epochs, using random environment maps
sampled from a collection of 190 HDR maps111https://polyhaven.com.

The displacement map estimation is done in patches in the full resolution
(4K). We manually selected 20 wrinkled subjects from the training set to
obtain the PCA basis for coarse displacement, as done in [7]. After training
the coarse displacement encoder for 5 epochs, we jointly train the coarse and
fine displacement networks for 15 epochs.

The texture estimation networks are trained on the light stage data and the
FFHQ-UV dataset. We use a batch size of 2 containing a labeled and an
unlabeled image. All the generators are 5 layer U-Nets with skip connections
[46]. We use the patch GAN multi-scale discriminators from [39].

##  5 Results

###  5.1 Geometry estimation

In this section, we evaluate our method for 3D shape reconstruction. We
compare to different state-of-the-art methods, including methods that
specialize in geometry prediction without supporting texture map generation.

Figure 6: Comparison of 3D face reconstruction methods.

First we visually compare our results to 3 available state-of-the-art methods:
DECA [18], HRN [32] and Deep3D [14]. Figure 6 shows the estimated geometry of
each method under the same rendering conditions. Our method produces meshes
with more similar likeness to the input images, notably on the nose, eyes,
cheeks and the shape of the face. It also better captures higher-frequency
details such as wrinkles and folds than the other methods. While HRN captures
some of the details, they are not as fine and precise as ours. Moreover, HRN
produces noticeable artifacts around the eyes and nose. The details captured
by DECA are not as visible and sometimes missing. Deep3D does not predict
displacement maps, and does not capture any of the folds and wrinkles.
However, DECA and Deep3D avoid baking glasses in the geometry as HRN and our
method do. The supplementary material contains additional comparisons.

Error (mm) Method Nose Mouth Forehead Cheeks All MICA [63] 1.585 $\pm$ 0.325
3.478 $\pm$ 1.204 2.374 $\pm$ 0.683 1.099 $\pm$ 0.324 2.134 DECA [18] 1.697
$\pm$ 0.355 2.516 $\pm$ 0.839 2.394 $\pm$ 0.576 1.479 $\pm$ 0.535 2.010 PSL
[37] 1.708 $\pm$ 0.349 1.708 $\pm$ 0.563 2.350 $\pm$ 0.551 1.593 $\pm$ 0.540
1.882 AlbedoGan [40] 1.656 $\pm$ 0.374 2.087 $\pm$ 0.839 2.102 $\pm$ 0.512
1.141 $\pm$ 0.303 1.746 Deep3D [14] 1.719 $\pm$ 0.354 1.368 $\pm$ 0.439 2.015
$\pm$ 0.449 1.528 $\pm$ 0.501 1.657 HRN [32] 1.722 $\pm$ 0.330 1.357 $\pm$
0.523 1.995 $\pm$ 0.476 1.072 $\pm$0.333 1.537 HiFace (w/o synthetic) [5]
1.227 $\pm$ 0.407 1.787 $\pm$ 0.439 1.454 $\pm$ 0.382 1.762 $\pm$ 0.436 1.558
HiFace [5] 1.036 $\pm$ 0.280 1.450 $\pm$ 0.413 1.324 $\pm$ 0.334 1.291 $\pm$
0.362 1.275 Baseline (linear 3DMM) 1.815 $\pm$ 0.516 1.725 $\pm$ 0.576 2.550
$\pm$ 0.825 1.469 $\pm$ 0.511 1.890 MoSAR (ours) 1.499 $\pm$ 0.366 1.424 $\pm$
0.462 1.950 $\pm$ 0.559 1.128 $\pm$ 0.303 1.500

Table 1: Reconstruction errors on the REALY benchmark.

For a quantitative analysis, we compare our method to state-of-the-art methods
using REALY [4], a publicly available benchmark. This evaluation framework
uses multi-view rendered portrait images of 100 high quality scans from the
HeadSpace dataset [9]. Geometric reconstruction errors are computed separately
on different face regions which reduces imprecision caused by alignment errors
and allows for a fine-grained analysis of the results. For each face region,
the mean of the per vertex $\ell_{2}$-distance between the predictions and the
ground truth is reported in Table 1.

Our method ranks second on reconstruction error averaged for all parts, being
surpassed only by the full HiFace model [5] when trained with 200k synthetic
images with ground-truth meshes. The addition of the synthetic dataset
significantly improves HiFace’s results, which otherwise would perform a bit
worst than ours. We believe that our method would similarly benefit from this
additional data.

Our results indicate that non-linear morphable models can perform
competitively compared to the dominating linear 3DMM. We included in Table 1
an implementation of our method using a linear 3DMM, a PCA created with the
same light stage dataset. Except for the geometry model, all other factors
were kept constant (i.e. dataset, encoder architecture). These results show
that the improvement stems from the higher expressivity of non-linear
modeling.

###  5.2 Texture estimation ablation

This ablation study evaluates the effect of light normalization (Section 3.3)
and the proposed semi-supervised training scheme (Section 3.4) on the quality
of the predicted texture maps. As a baseline, we train our system in a fully-
supervised manner without light normalization, using only light stage data.
This means that our differential shading is not required for self-supervision,
thus, we ignore the term $L_{\text{shading}}$ on Equation 4. Then, we include
the light normalization to the baseline model to measure its impact. For
completeness, we train a model in a purely self-supervised manner, without any
light stage data.

To evaluate these systems on settings similar to in-the-wild, while having
access to ground truth maps, we render subjects from our light stage
validation data under novel lighting conditions using random HDR maps. From 50
subjects, we generate 1000 renders. We report the Structural Similarity Index
(SSIM) [59] between the predicted and ground-truth maps.

Figure 7: Comparison of the estimated geometry and renders under 3 lighting
conditions, against FitMe [30] and Relightify [38]

Method Diffuse Specular AO Transl. Supervised 0.69 0.28 0.61 0.67 \+ Light
Normalization 0.80 0.55 0.82 0.79 Self-supervised 0.80 0.19 0.36 0.65 Full
(Semi-supervised) 0.83 0.65 0.85 0.82

Table 2: SSIM metric between estimated and ground truth texture maps, for
different settings of our method (higher is better).

Results are shown in Table 2. The supervised method, trained using only light
stage data underperforms, indicating poor generalization when confronted to
the variety of lighting conditions in-the-wild. Our light normalization step
reduces this problem by attenuating specular highlights and strong shadows.
Consequently, it improves SSIM, specially on the specular map. On the other
hand, the self-supervised method, which is trained without any ground-truth
textures, fails to disentangle the contributions of different maps, as
expected, since they are entangled in the rendering Equation 5. It bakes most
of the information in the diffuse albedo map and is therefore unsuitable for
relighting applications. Our full model obtains the best SSIM. It takes
advantages of high-quality light stage data to better disentangle the
contributions of different maps, as opposed to the self-supervised-only
version. Moreover, training from in-the-wild images exposes the model to a
wide variety of lighting conditions and allows for training with a larger
quantity of data. This translates to improvements on every map, but especially
the specular map.

Aside from improving SSIM, our full model produces visually sharper and
cleaner maps when applied to in-the-wild data, as shown in Figure 8. Our semi-
supervised training helps to better separate the intrinsic face attributes.
For instance, the supervised-only model bakes specular reflections in the
diffuse map. This is observed on the specular highlights of the nose, which
our full model successfully removed from the diffuse map. The supervised model
also bakes into the diffuse map both light and shadows that were not
completely removed by the light normalization step. The full model correctly
handles the diffuse and specular terms. Finally, the AO map produced by the
full method better captures the shading of the wrinkles and folds.

Figure 8: Reflectance estimation using the “supervised + Light Normalization”
model (first and third rows) vs the full method (second and fourth rows).

###  5.3 Avatar reconstruction

In this section, we perform a visual comparison with the most recent state-of-
the art methods that perform full-avatar reconstruction: FitMe [30] and
Relightify [38]. Both methods estimate geometry, diffuse, specular and normal
maps from a single image. Figure 7 shows, for each method, the estimated
geometry (including normal map), and a rendering of the avatar, under the same
conditions (camera placement and environment map). We focus the comparison on
the skin reflectance, and therefore we removed the scalp, eyes and mouth
interior of FitMe and Relightify, since our method do not estimate them. We
refer the reader to the supplementary material for the comparison with more
subjects and lighting conditions.

Our approach captures subtle geometric details, including wrinkles and folds,
in contrast to other methods, which struggle to reproduce these nuances.
Notably, the shadows caused by wrinkles (see the first row) are baked in the
albedo for both FitMe and Relightify, since they were not captured in the
geometry, which cause unrealistic wrinkles in other light conditions. Our
method is better at capturing the facial structure (cheek, nose, eyes) and
expression, resulting in a likeness that more faithfully resembles the
original image. For darker skin tones, while our method has a better shape
estimation, it tends to yield lighter skin tones.

##  6 Limitations

Our method does not explicitly model external occluders (e.g. eyeglass, hair)
which get baked into the texture maps. Also, our method is more successful in
preserving the skin tones for caucasian subjects. This is mainly due to the
representation imbalance in our in-the-wild dataset. As shown in [36], FFHQ is
composed of around 69% white, 4% black, and 27% of other races. This biases
the model towards generating lighter skin tones. Interestingly, we do not
notice the same bias in the 3D shape reconstruction.

##  7 Conclusion

In this work, we presented MoSAR, a complete framework for creating realistic,
relightable avatars from a single portrait image. It produces detailed
geometry and skin reflectance maps at 4K resolution, compatible with modern
rendering engines. We proposed a semi-supervised training procedure that
includes a novel differentiable shading formulation that allows for estimating
ambient occlusion and translucency. Our experiments highlighted the benefit of
our semi-supervised training and showed that we obtain competitive results for
geometry estimation using non-linear morphable models. This amounts to
realistic and visually appealing renderings when compared to existing state-
of-the-art methods. Finally, we introduced a dataset containing intrinsic
facial attributes for $10$k subjects of the FFHQ-UV dataset to accelerate
research on this field. For future work, we intend to collect more balanced
dataset to reduce bias in our results. We want to model external occluders and
reduce artifacts caused by hairs.

##  8 Acknowledgments

We would like to thank Dr. Alexandros Lattas and Foivos Papantoniou for
running the FitMe and Relightify methods on the selected subjects. We thank
the authors of DECA, HRN and Deep3D for sharing their code source publicly. We
thank Amaury Depierre for assistance with the super-resolution network.

## References

  * Aliari et al. [2023] Mohammad Amin Aliari, Andre Beauchamp, Tiberiu Popa, and Eric Paquette.  Face editing using part-based optimization of the latent space.  In _Computer Graphics Forum_ , pages 269–279. Wiley Online Library, 2023.
  * Bai et al. [2023] Haoran Bai, Di Kang, Haoxian Zhang, Jinshan Pan, and Linchao Bao.  Ffhq-uv: Normalized facial uv-texture dataset for 3d face reconstruction.  In _IEEE Conference on Computer Vision and Pattern Recognition_ , 2023\.
  * Bulat and Tzimiropoulos [2017] Adrian Bulat and Georgios Tzimiropoulos.  How far are we from solving the 2d & 3d face alignment problem? (and a dataset of 230,000 3d facial landmarks).  In _International Conference on Computer Vision_ , 2017.
  * Chai et al. [2022] Zenghao Chai, Haoxian Zhang, Jing Ren, Di Kang, Zhengzhuo Xu, Xuefei Zhe, Chun Yuan, and Linchao Bao.  Realy: Rethinking the evaluation of 3d face reconstruction.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , 2022.
  * Chai et al. [2023] Zenghao Chai, Tianke Zhang, Tianyu He, Xu Tan, Tadas Baltrusaitis, HsiangTao Wu, Runnan Li, Sheng Zhao, Chun Yuan, and Jiang Bian.  Hiface: High-fidelity 3d face reconstruction by learning static and dynamic details.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 9087–9098, 2023.
  * Chandran et al. [2021] Prashanth Chandran, Sebastian Winberg, Gaspard Zoss, Jérémy Riviere, Markus Gross, Paulo Gotardo, and Derek Bradley.  Rendering with style: combining traditional and neural approaches for high-quality face rendering.  _ACM Transactions on Graphics (ToG)_ , 40(6):1–14, 2021.
  * Chen et al. [2019] Anpei Chen, Zhang Chen, Guli Zhang, Kenny Mitchell, and Jingyi Yu.  Photo-realistic facial details synthesis from single image.  In _Proceedings of the IEEE International Conference on Computer Vision_ , pages 9429–9439, 2019.
  * Cook and Torrance [1982] Robert L Cook and Kenneth E. Torrance.  A reflectance model for computer graphics.  _ACM Transactions on Graphics (TOG)_ , 1(1):7–24, 1982.
  * Dai et al. [2020] Hang Dai, Nick Pears, William Smith, and Christian Duncan.  Statistical modeling of craniofacial shape and texture.  _International Journal of Computer Vision_ , 128:547–571, 2020.
  * Danecek et al. [2022] Radek Danecek, Michael J. Black, and Timo Bolkart.  EMOCA: Emotion driven monocular face capture and animation.  In _Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 20311–20322, 2022.
  * Daněček et al. [2022] Radek Daněček, Michael J Black, and Timo Bolkart.  Emoca: Emotion driven monocular face capture and animation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 20311–20322, 2022.
  * Debevec [2012] Paul Debevec.  The light stages and their applications to photoreal digital actors.  _SIGGRAPH Asia_ , 2(4):1–6, 2012.
  * Deng et al. [2018] Jiankang Deng, Shiyang Cheng, Niannan Xue, Yuxiang Zhou, and Stefanos Zafeiriou.  Uv-gan: Adversarial facial uv map completion for pose-invariant face recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 7093–7102, 2018.
  * Deng et al. [2019] Yu Deng, Jiaolong Yang, Sicheng Xu, Dong Chen, Yunde Jia, and Xin Tong.  Accurate 3d face reconstruction with weakly-supervised learning: From single image to image set.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , pages 0–0, 2019.
  * Dib et al. [2021a] Abdallah Dib, Gaurav Bharaj, Junghyun Ahn, Cédric Thébault, Philippe-Henri Gosselin, Marco Romeo, and Louis Chevallier.  Practical face reconstruction via differentiable ray tracing.  _Computer Graphics Forum_ , 2021a.
  * Dib et al. [2021b] Abdallah Dib, Cedric Thebault, Junghyun Ahn, Philippe-Henri Gosselin, Christian Theobalt, and Louis Chevallier.  Towards high fidelity monocular face reconstruction with rich reflectance using self-supervised learning and ray tracing.  In _The IEEE International Conference on Computer Vision (ICCV)_ , 2021b.
  * Dib et al. [2023] Abdallah Dib, Junghyun Ahn, Cedric Thebault, Philippe-Henri Gosselin, and Louis Chevallier.  S2f2: Self-supervised high fidelity face reconstruction from monocular image.  In _2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition (FG)_ , pages 1–8. IEEE, 2023.
  * Feng et al. [2021] Yao Feng, Haiwen Feng, Michael J Black, and Timo Bolkart.  Learning an animatable detailed 3d face model from in-the-wild images.  _ACM Transactions on Graphics (TOG)_ , 40(4):1–13, 2021.
  * Gao et al. [2020] Zhongpai Gao, Juyong Zhang, Yudong Guo, Chao Ma, Guangtao Zhai, and Xiaokang Yang.  Semi-supervised 3d face representation learning from unconstrained photo collections.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops_ , pages 348–349, 2020.
  * Garrido et al. [2016] Pablo Garrido, Michael Zollhöfer, Dan Casas, Levi Valgaerts, Kiran Varanasi, Patrick Perez, and Christian Theobalt.  Reconstruction of Personalized 3D Face Rigs from Monocular Video.  _{ ACM} Trans. Graph. (Presented at SIGGRAPH 2016)_, 35(3):28:1–28:15, 2016.
  * Groueix et al. [2018] Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry.  3d-coded: 3d correspondences by deep deformation.  In _Proceedings of the european conference on computer vision (ECCV)_ , pages 230–246, 2018.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Identity mappings in deep residual networks.  In _Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14_ , pages 630–645. Springer, 2016.
  * Isola et al. [2017] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros.  Image-to-image translation with conditional adversarial networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 1125–1134, 2017.
  * Karras et al. [2019] Tero Karras, Samuli Laine, and Timo Aila.  A style-based generator architecture for generative adversarial networks.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 4401–4410, 2019.
  * Karras et al. [2020] Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila.  Analyzing and improving the image quality of StyleGAN.  In _Proc. CVPR_ , 2020.
  * Kato et al. [2020] Hiroharu Kato, Deniz Beker, Mihai Morariu, Takahiro Ando, Toru Matsuoka, Wadim Kehl, and Adrien Gaidon.  Differentiable rendering: A survey.  _arXiv preprint arXiv:2006.12057_ , 2020.
  * Kingma and Ba [2015] Diederik Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  In _International Conference on Learning Representations (ICLR)_ , San Diega, CA, USA, 2015.
  * Lattas et al. [2020] Alexandros Lattas, Stylianos Moschoglou, Baris Gecer, Stylianos Ploumpis, Vasileios Triantafyllou, Abhijeet Ghosh, and Stefanos Zafeiriou.  Avatarme: Realistically renderable 3d facial reconstruction” in-the-wild”.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 760–769, 2020.
  * Lattas et al. [2021] Alexandros Lattas, Stylianos Moschoglou, Stylianos Ploumpis, Baris Gecer, Abhijeet Ghosh, and Stefanos Zafeiriou.  Avatarme++: Facial shape and brdf inference with photorealistic rendering-aware gans.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 44(12):9269–9284, 2021.
  * Lattas et al. [2023] Alexandros Lattas, Stylianos Moschoglou, Stylianos Ploumpis, Baris Gecer, Jiankang Deng, and Stefanos Zafeiriou.  Fitme: Deep photorealistic 3d morphable model avatars.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 8629–8640, 2023.
  * Lee and Lee [2020] Gun-Hee Lee and Seong-Whan Lee.  Uncertainty-aware mesh decoder for high fidelity 3d face reconstruction.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 6100–6109, 2020.
  * Lei et al. [2023] Biwen Lei, Jianqiang Ren, Mengyang Feng, Miaomiao Cui, and Xuansong Xie.  A hierarchical representation network for accurate and detailed face reconstruction from in-the-wild images.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 394–403, 2023.
  * Li et al. [2020] Ruilong Li, Karl Bladin, Yajie Zhao, Chinmay Chinara, Owen Ingraham, Pengda Xiang, Xinglei Ren, Pratusha Prasad, Bipin Kishore, Jun Xing, et al.  Learning formation of physically-based face attributes.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 3410–3419, 2020.
  * Lin et al. [2020] Jiangke Lin, Yi Yuan, Tianjia Shao, and Kun Zhou.  Towards high-fidelity 3d face reconstruction from in-the-wild images using graph convolutional networks.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 5891–5900, 2020.
  * Mahajan et al. [2007] Dhruv Mahajan, Ravi Ramamoorthi, and Brian Curless.  A theory of frequency domain invariants: Spherical harmonic identities for brdf/lighting transfer and image consistency.  _IEEE transactions on pattern analysis and machine intelligence_ , 30(2):197–213, 2007.
  * Maluleke et al. [2022] Vongani H. Maluleke, Neerja Thakkar, Tim Brooks, Ethan Weber, Trevor Darrell, Alexei A. Efros, Angjoo Kanazawa, and Devin Guillory.  Studying bias in gans through the lens of race.  In _Computer Vision – ECCV 2022: 17th European Conference_ , page 344–360, Berlin, Heidelberg, 2022. Springer-Verlag.
  * Otto et al. [2023] Christopher Otto, Prashanth Chandran, Gaspard Zoss, Markus Gross, Paulo Gotardo, and Derek Bradley.  A Perceptual Shape Loss for Monocular 3D Face Reconstruction.  _Computer Graphics Forum_ , 2023.
  * Paraperas Papantoniou et al. [2023] Foivos Paraperas Papantoniou, Alexandros Lattas, Stylianos Moschoglou, and Stefanos Zafeiriou.  Relightify: Relightable 3d faces from a single image via diffusion models.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , 2023.
  * Park et al. [2019] Taesung Park, Ming-Yu Liu, Ting-Chun Wang, and Jun-Yan Zhu.  Semantic image synthesis with spatially-adaptive normalization.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2019.
  * Rai et al. [2023] Aashish Rai, Hiresh Gupta, Ayush Pandey, Francisco Vicente Carrasco, Shingo Jason Takagi, Amaury Aubel, Daeil Kim, Aayush Prakash, and Fernando De la Torre.  Towards realistic generative 3d face models.  _arXiv preprint arXiv:2304.12483_ , 2023.
  * Ramamoorthi and Hanrahan [2001a] Ravi Ramamoorthi and Pat Hanrahan.  An Efficient Representation for Irradiance Environment Maps.  In _Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques_ , pages 497–500, New York, NY, USA, 2001a. ACM.
  * Ramamoorthi and Hanrahan [2001b] Ravi Ramamoorthi and Pat Hanrahan.  On the relationship between radiance and irradiance: determining the illumination from images of a convex Lambertian object.  _JOSA A_ , 18(10):2448–2459, 2001b.
  * Ramamoorthi and Hanrahan [2001c] Ravi Ramamoorthi and Pat Hanrahan.  A signal-processing framework for inverse rendering.  In _Proceedings of the 28th annual conference on Computer graphics and interactive techniques_ , pages 117–128, 2001c.
  * Ranjan et al. [2018] Anurag Ranjan, Timo Bolkart, Soubhik Sanyal, and Michael J. Black.  Generating 3D faces using convolutional mesh autoencoders.  In _European Conference on Computer Vision (ECCV)_ , pages 725–741, 2018.
  * Richardson et al. [2017] Elad Richardson, Matan Sela, Roy Or-El, and Ron Kimmel.  Learning detailed face reconstruction from a single image.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 1259–1268, 2017.
  * Ronneberger et al. [2015] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-net: Convolutional networks for biomedical image segmentation.  In _Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18_ , pages 234–241. Springer, 2015.
  * Sanyal et al. [2019] Soubhik Sanyal, Timo Bolkart, Haiwen Feng, and Michael J Black.  Learning to regress 3d face shape and expression from an image without 3d supervision.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 7763–7772, 2019.
  * Schlick [1994] Christophe Schlick.  An inexpensive brdf model for physically-based rendering.  In _Computer graphics forum_ , pages 233–246. Wiley Online Library, 1994.
  * Sela et al. [2017] Matan Sela, Elad Richardson, and Ron Kimmel.  Unrestricted facial geometry reconstruction using image-to-image translation.  In _Proceedings of the IEEE International Conference on Computer Vision_ , pages 1576–1585, 2017.
  * Sloan et al. [2023] Peter-Pike Sloan, Jan Kautz, and John Snyder.  Precomputed radiance transfer for real-time rendering in dynamic, low-frequency lighting environments.  In _Seminal Graphics Papers: Pushing the Boundaries, Volume 2_ , pages 339–348. 2023.
  * Tewari et al. [2017] Ayush Tewari, Michael Zollöfer, Hyeongwoo Kim, Pablo Garrido, Florian Bernard, Patrick Perez, and Theobalt Christian.  MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction.  In _The IEEE International Conference on Computer Vision (ICCV)_ , 2017.
  * Tewari et al. [2018] Ayush Tewari, Michael Zollöfer, Florian Bernard, Pablo Garrido, Hyeongwoo Kim, Patrick Perez, and Christian Theobalt.  High-fidelity monocular face reconstruction based on an unsupervised model-based face autoencoder.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , pages 1–1, 2018.
  * Tewari et al. [2019] A Tewari, F Bernard, P Garrido, G Bharaj, et al.  _FML: Face Model Learning from Videos_.  CVPR, 2019.
  * Torrance and Sparrow [1967] Kenneth E Torrance and Ephraim M Sparrow.  Theory for off-specular reflection from roughened surfaces.  _Josa_ , 57(9):1105–1114, 1967.
  * Tran and Liu [2018] Luan Tran and Xiaoming Liu.  Nonlinear 3d face morphable model.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 7346–7355, 2018.
  * Tran et al. [2019] Luan Tran, Feng Liu, and Xiaoming Liu.  Towards high-fidelity nonlinear 3d face morphable model.  In _In Proceeding of IEEE Computer Vision and Pattern Recognition_ , Long Beach, CA, 2019.
  * Walter et al. [2007] Bruce Walter, Stephen Marschner, Hongsong Li, and Kenneth Torrance.  Microfacet models for refraction through rough surfaces.  pages 195–206, 2007.
  * Wang et al. [2021] Xintao Wang, Liangbin Xie, Chao Dong, and Ying Shan.  Real-esrgan: Training real-world blind super-resolution with pure synthetic data.  In _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 1905–1914, 2021.
  * Wang et al. [2004] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli.  Image quality assessment: from error visibility to structural similarity.  _IEEE transactions on image processing_ , 13(4):600–612, 2004.
  * Yamaguchi et al. [2018] S Yamaguchi, S Saito, et al.  _High-fidelity Facial Reflectance and Geometry Inference from an Unconstrained Image_.  ACM TOG, 2018.
  * Zeng et al. [2019] Xiaoxing Zeng, Xiaojiang Peng, and Yu Qiao.  Df2net: A dense-fine-finer network for detailed 3d face reconstruction.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 2315–2324, 2019.
  * Zhu et al. [2017] Xiangyu Zhu, Xiaoming Liu, Zhen Lei, and Stan Z Li.  Face alignment in full pose range: A 3d total solution.  _IEEE transactions on pattern analysis and machine intelligence_ , 2017.
  * Zielonka et al. [2022] Wojciech Zielonka, Timo Bolkart, and Justus Thies.  Towards metrical reconstruction of human faces.  In _European Conference on Computer Vision_ , pages 250–269. Springer, 2022.
  * Zollhöfer et al. [2018] Michael Zollhöfer, Justus Thies, Pablo Garrido, Derek Bradley, Thabo Beeler, Patrick Pérez, Marc Stamminger, Matthias Nießner, and Christian Theobalt.  State of the art on monocular 3d face reconstruction, tracking, and applications.  In _Computer graphics forum_ , pages 523–550. Wiley Online Library, 2018.


# Can the exciton–polariton regime be defined by its quantum properties?

D. G. Suárez-Forero  Universidad Nacional de Colombia - Bogotá, Facultad de
Ciencias, Departamento de Física, Grupo de Óptica e Información Cuántica,
Carrera 30 Calle 45-03, C.P. 111321, Bogotá, Colombia  G. Cipagauta
Universidad Nacional de Colombia - Bogotá, Facultad de Ciencias, Departamento
de Física, Grupo de Óptica e Información Cuántica, Carrera 30 Calle 45-03,
C.P. 111321, Bogotá, Colombia  H. Vinck-Posada<EMAIL_ADDRESS>Universidad Nacional de Colombia - Bogotá, Facultad de Ciencias, Departamento
de Física, Grupo de Óptica e Información Cuántica, Carrera 30 Calle 45-03,
C.P. 111321, Bogotá, Colombia  K. M. Fonseca-Romero  Universidad Nacional de
Colombia - Bogotá, Facultad de Ciencias, Departamento de Física, Grupo de
Óptica e Información Cuántica, Carrera 30 Calle 45-03, C.P. 111321, Bogotá,
Colombia  B. A. Rodríguez  Instituto de Física, Universidad de Antioquia,
Medellín, A.A. 1226, Medellín, Colombia

###### Abstract

Using a simple fully quantum model in an effective exciton scheme that takes
into account the system–environment interaction, we study the different
regimes arising in a microcavity–quantum dot system. Our numerical
calculations of the emission linewidth, emission energy, integrated intensity
and second- and third-order correlation functions are in good qualitative
agreement with reported experimental results. We show that the transition from
the polariton-laser to the photon-laser regime can be defined through the
critical points of both the negativity and the linear entropy of the steady
state.

###### pacs:

42.50.ct, 78.67.Hc, 42.55.sa, 03.75.Gg

The solid–state realization of Bose–Einstein Condensation (BEC) has been
achieved in exciton–polariton systems Kasprzak  _et al._ (2006); Deng  _et
al._ (2006). These quasi–bosons, arising from the strong coupling between
photons and electron–hole pairs in semiconductor microcavities ($\mu$-C), have
a high critical temperature due to their small effective mass (eight orders of
magnitude smaller than hydrogen atom mass). After more than two decades of
theoretical and experimental investigations, nowadays it is understood that
the large occupation of the polariton ground state cannot be identified with
usual thermodynamic equilibrium BEC states Deng  _et al._ (2010); Kasprzak
_et al._ (2006); Snoke (2006). Instead, the corresponding experimentally
observed regime has been called polariton laser Kasprzak  _et al._ (2006);
Bajoni  _et al._ (2008); Azzini  _et al._ (2011) because of its dynamical
nature and the gain in the light–emission intensity. The transition from this
regime to a second one, identified with the well-known photon laser, has also
been observed Bajoni  _et al._ (2008); Azzini  _et al._ (2011); Christopoulos
_et al._ (2007).

Cavity polariton systems have been studied from different theoretical
perspectives. Assuming thermal equilibrium, a trial wave function that takes
into account the coherence properties of both light and matter, has been able
to predict multiple phase transitions Littlewood  _et al._ (2004); Kamide and
Ogawa (2010). On the other hand, when the matter–light state is obtained from
some equation of motion (mean field dynamics Vinck-Posada  _et al._ (2005);
Lagoudakis  _et al._ (2011); Liew  _et al._ (2008), master equation in
multiexcitonic scheme Vera  _et al._ (2009a), dissipative Jaynes–Cummings
model Perea  _et al._ (2004); Laussy  _et al._ (2009); del Valle  _et al._
(2009); Laussy  _et al._ (2011); Vera  _et al._ (2009b); del Valle  _et al._
(2007); Auffèves  _et al._ (2009); Laucht  _et al._ (2009)), the dynamical
character of the polariton laser regime is conspicuous, and the non-Gibbsian
character of the stationary state of the system is revealed. However, the
elucidation of the mechanisms behind the appearance of the different observed
regimes is still an open problem. Our work is a first step in this direction.

The aim of this Letter, the identification of the regimes observed in current
experiments from the quantum properties of the steady state, is possible due
to the simplicity of our model. Indeed, we consider a single pumped radiator
interacting with a leaky mode of the electromagnetic field, ignoring
collective effects. Our calculations, however, are in qualitative good
agreement with the experimental results. Moreover, we can correlate the
entanglement, mixedness and the coherence functions of the steady state not
only with the observed regimes but also with important physical parameters
like the pumping rate and the detuning. In addition, we are able to provide a
criterion to identify the “best” polariton that can be sustained by the
system.

We model a quantum dot (QD) embedded in a $\mu$-C, as a two-level system
(ground $\ket{G}$ and excited $\ket{X}$ states). Its interaction with a single
electromagnetic mode of frequency $\omega_{C}$, in the dipole and rotating
wave approximations, is described by the Hamiltonian ($\hbar=1$):

$\hat{H}=\omega_{C}\hat{a}^{\dagger}\hat{a}+(\omega_{C}-\Delta)\hat{\sigma}^{\dagger}\hat{\sigma}+g(\hat{a}\hat{\sigma}^{\dagger}+\hat{a}^{\dagger}\hat{\sigma}).$
(1)

The detuning $\Delta$ is the difference between cavity mode and exciton
energies, $g$ is the matter–light coupling constant,
$\hat{\sigma}=\ket{G}\bra{X}$ is the QD ladder operator, and
$\hat{a}^{\dagger}$ ($\hat{a}$) is the usual creation (annihilation) operator
of the cavity mode. The Hamiltonian (1) commutes with the excitation number
$\hat{N}=\hat{N}_{ph}+\hat{N}_{ex}=\hat{a}^{\dagger}\hat{a}+\hat{\sigma}^{\dagger}\hat{\sigma}$;
hence, it only causes transitions between matter–light states of the same
excitation manifold. Polaritons are defined as the energy eigenstates
$\hat{H}$, and are explicitly given by

$\displaystyle\ket{n,+}$ $\displaystyle=$
$\displaystyle\sin{\Phi_{n}}\ket{G,n}+\cos{\Phi_{n}}\ket{X,n-1}$
$\displaystyle\ket{n,-}$ $\displaystyle=$
$\displaystyle\cos{\Phi_{n}}\ket{G,n}-\sin{\Phi_{n}}\ket{X,n-1},$ (2)

where $\\{\ket{n}\\}$ denotes the Fock number states of the field and
$\tan{2\Phi_{n}}=2g\sqrt{n}/\Delta$. We include two non-conservative
processes, the loss of photons in the $\mu$-C ($\kappa$) and the continuous
pumping of excitons ($P$), in the master equation for the density operator
$\hat{\rho}$ of the system

$\displaystyle\frac{d\hat{\rho}}{dt}$ $\displaystyle=$ $\displaystyle
i[\hat{\rho},\hat{H}]+\tfrac{1}{2}P(2\hat{\sigma}^{\dagger}\hat{\rho}\hat{\sigma}-\hat{\sigma}\hat{\sigma}^{\dagger}\hat{\rho}-\hat{\rho}\hat{\sigma}\hat{\sigma}^{\dagger})$
(3) $\displaystyle+$
$\displaystyle\tfrac{1}{2}\kappa(2\hat{a}\hat{\rho}\hat{a}^{\dagger}-\hat{a}^{\dagger}\hat{a}\hat{\rho}-\hat{\rho}\hat{a}^{\dagger}\hat{a}),$

where we have made the Born-Markov approximation. We neglect other
system–environment interaction mechanisms (e.g., spontaneous emission,
dephasing, photon pumping, polariton pumping, etc.) because their effect is
either small or is already effectively contained in the master equation. If a
better adjustment with the experimental results is desired those mechanisms
can be included and fitted Laucht  _et al._ (2009), but the qualitative
physical image remains essentially unchanged.

The basic assumption behind our approach, which focuses on the steady state
$\hat{\rho}_{\textrm{ss}}$ of the equation of motion (3), is that polariton
lifetime is much longer than the time required to reach the asymptotic
solution Deng  _et al._ (2006). The steady state
$\hat{\rho}_{\textrm{ss}}=\hat{\rho}(\kappa,P,g,\Delta)$ of the system, is a
function of the dissipative rates $\kappa$ and $P$, the matter-light coupling
constant $g$ and the detuning $\Delta$. In the weak-coupling regime $g\ll
P,\kappa$, the pumping keeps the QD in its excited state while the dissipation
steers the electromagnetic field to its ground state. Other states are not
significantly populated because matter excitation cannot be converted into
photons. On the other extreme, the ultra-strong coupling regime $g\ggg
P,\kappa,\Delta$, the long-time density matrix becomes (almost) diagonal in
the basis of bare states $\ket{G/X,n}$. The larger the coupling $g$ the
smaller the difference of the populations of $\ket{G,n}$ and $\ket{X,n-1}$
($\propto 1/g^{2}$). The coherences $|\rho_{GnXn-1}|$, which decay as $1/g$,
also vanish as the coupling $g$ increases.

In this work we focus on the strong-coupling regime $g\gg P,\kappa$, in which
the coherences $\rho_{GnXn-1}$ are small, but different from zero. In
resonance they are purely imaginary. For small detunings they acquire a small
real part. If the detuning increases, $\Delta\gg g$, the matter-light
interaction becomes dispersive, i.e., the energies of the matter states depend
on the number of photons, and the mechanism which converts matter excitations
into photons is suppressed. Thereby, large detunings correspond to a weak
coupling regime. We conclude that in the regime $|\Delta|\sim g\gg P,\kappa$,
the steady state of the system is expected to exhibit a polaritonic behavior.
Unless stated otherwise, the steady state solution $\hat{\rho}_{ss}$ of (3) is
obtained for the initial condition $\hat{\rho}(0)=\ket{G0}\bra{G0}$, and
setting $\omega_{C}=1$ eV, $g=1$ meV and $\kappa=5\times 10^{-2}$ meV, while
$\Delta$ and $P$ are varied in ranges similar to those of current experiments
Bajoni  _et al._ (2008); Azzini  _et al._ (2011); Reithmaier  _et al._ (2004).

Evidence of the spontaneous coherence build-up associated with polariton
states are currently detected through the photoluminescence properties in
quantum wells (QW) Bajoni  _et al._ (2008); Azzini  _et al._ (2011); Kasprzak
_et al._ (2006). We compare our theoretical predictions in QDs with the
experimental findings in QWs because: $i$) due to experimental difficulties no
analogous results for QDs have been reported and $ii$) it is reasonable to
assume that some of the physical mechanisms behind the exciton–polariton laser
regime are the same in both cases. Additionally, the present approach may shed
light on the separation of the collective effects in QWs from those of a QD
single emitter.

The calculated emission linewidth, emission energy, integrated intensity and
number of photons are shown in fig. 1 as a function of the pumping rate. This
numerical calculation used the quantum regression theorem Perea  _et al._
(2004); Laussy  _et al._ (2009); del Valle  _et al._ (2009); Vera  _et al._
(2009b) and the integrated intensity corresponds to the area under the curve
of the peak associated to the transition between the polaritons in the
manifolds $n=1$ and $n=2$. The linewidth (fig.1.a) exhibits the characteristic
reduction in the polariton regime, and the subsequent growth and decrease
Bajoni  _et al._ (2008); Azzini  _et al._ (2011); Kasprzak  _et al._ (2006).
The emission energy blueshifts (fig.1.b) from the exciton transition frequency
to the cavity mode frequency. In the polaritonic region the blueshift is
smaller than observed in QWs Bajoni  _et al._ (2008); Azzini  _et al._ (2011).
In the intermediate region, $10^{-1}$ meV $\lesssim P\lesssim 1$ meV, the
calculated blueshift grows faster than the measured one. Despite the small
slope changes in the integrated intensity as a function of $P$ (fig.1.c) our
results are consistent with the previous prediction of absence of threshold in
a one–atom laser Rice and Carmichael (1994). However, the nonlinearity of the
model is evident in the curve of the average number of photons. Only for large
values of the pumping ($P\gtrsim 1$ meV), this curve is parallel to that of
the integrated emission intensity (whose slope is approximately one).

Figure 1: (color online). (a) Emission linewidth, (b) emission energy, (c)
integrated intensity (continuous line) and average number of photons (dashed
line) and (d) second– and third–order correlation functions versus the
incoherent exciton pumping $P$, for $\kappa=5\times 10^{-2}$ meV and
$\Delta=2.5$ meV. The marked regions correspond to the polariton–laser and
photon–laser regimes.

Statistics of the emitted light can be characterized by the normalized second–
$g^{(2)}(0)=\braket{\hat{a}^{\dagger}\hat{a}^{\dagger}\hat{a}\hat{a}}/\braket{\hat{a}^{\dagger}\hat{a}}^{{}_{2}}$
and third–order
$g^{(3)}(0)=\braket{\hat{a}^{\dagger}\hat{a}^{\dagger}\hat{a}^{\dagger}\hat{a}\hat{a}\hat{a}}/\braket{\hat{a}^{\dagger}\hat{a}}^{{}_{3}}$
coherence functions, plotted in fig.(1.d). For small pumping power ($P\lesssim
2\times 10^{-2}$ meV) $g^{(2)}(0),g^{(3)}(0)<1$, a footprint of quantum-like
light, i.e. the partial state of the field is nearly a Fock state with small
number of photons, as expected. Indeed, our calculations show that in this
regime
$\hat{\rho}_{ss}\approx\ket{G}\bra{G}\otimes\\{\sin{\varphi}\ket{0}\bra{0}+\cos{\varphi}\ket{1}\bra{1}\\}$,
with $\varphi\approx 0$. For intermediate values of the pumping ($10^{-2}$ meV
$\lesssim P\lesssim 10^{-1}$ meV) both correlations functions monotonically
grow beyond one (inset fig.1.d), as has been experimentally observed Horikiri
_et al._ (2010). We analyze this behavior below in the text. For larger
pumping rates ($P\gtrsim 2$ meV), the state of the field becomes coherent up
to third order (when $g^{(2)}(0)=g^{(3)}(0)=1$), and thus the linewidth falls
(fig.1.a). In this region, the state of the field obtained by the partial
trace over the excitonic degrees of freedom, has a fidelity of more than 0.99
with the random-phase coherent state
$\int_{0}^{2\pi}\tfrac{d\phi}{2\pi}\Ket{\alpha e^{i\phi}}\Bra{\alpha
e^{i\phi}}$, where $\Ket{\alpha e^{i\phi}}$ is an usual coherent state with
$|\alpha|^{2}$ average number of photons. This type of state has been proposed
to describe the features of a (true) photon-laser Mølmer (1997).

We stress that the regimes identified through the characteristics of the
emitted light, mirror quantum properties of the system state. We describe
those quantum properties by entanglement and mixedness of the steady state.
Linear entropy and negativity are employed to quantify mixedness and matter-
light entanglement, respectively. The former, defined as
$S_{L}(\hat{\rho})=1-\textrm{Tr}\hat{\rho}^{{}_{2}}$, vanishes for pure states
and is maximum for maximally mixed states. The latter is defined as
$\mathcal{N}(\hat{\rho})=2\sum_{\lambda<0}|\lambda|$, where $\lambda$ denotes
the eigenvalues of the partial transpose of $\hat{\rho}$ Życzkowski  _et al._
(1998); Vidal and Werner (2002). Finally, searching for a relation between the
energy of the system and its quantum characteristics (such as entanglement),
we introduce the differential energy per excitation,
$\nu(\kappa,P)=\tfrac{1}{2}d\braket{\hat{H}}/d\braket{\hat{N}}$, as a
convenient measure of energy per particle. The factor of 2 in the definition
of $\nu$ has been chosen to satisfy the condition $\nu(P\ll
g,\kappa\ll\Delta)\approx\omega_{C}-\Delta$. The negativity, linear entropy
and the differential energy per excitation are depicted in fig. 2.

Figure 2:  (color online). $\mathcal{N}(\hat{\rho}_{ss})$ (continuous red
line), $S_{L}(\hat{\rho}_{ss})$ (dashed black line) and $\nu$ (dashed-dotted
blue line) as a function of the incoherent exciton pumping $P$, for
$\kappa=5\times 10^{-2}$ meV and (a) $\Delta=0$, (b) $\Delta=1$ meV, (c)
$\Delta=2$ meV, (d) $\Delta=3$ meV, (e) $\Delta=7$ meV and (f) $\Delta=10$
meV. Note that for $\Delta\neq 0$ the maximum of
$\mathcal{N}(\hat{\rho}_{ss})$, the minimum of $S_{L}(\hat{\rho}_{ss})$ and
the inflection point of $\nu$ coincide.

Assuming the strong-coupling regime, the formation of the polariton is
hindered by two different mechanisms, which depend on the detuning. While the
mixedness of the state is large for small detuning (fig. 2.a and 2.b), matter
and light decouple for large detuning (fig. 2.f). In the intermediate region
$g\lesssim|\Delta|\lesssim 10g$, where the polariton is well defined, both
mechanisms compete. The matter-light entanglement is enhanced, and the entropy
of the asymptotic state of the system decreases with increasing detuning. The
negativity of the steady state $\mathcal{N}(\hat{\rho}_{ss})$ vanishes for
small and large values of the pumping power, and attains a maximum at the
point $P=\kappa$ –the mid-point of the polariton-laser region.

Now, we are able to give a possible explanation of the behavior of the
correlation functions in the polariton regime. The asymptotic state in the
polaritonic region is a mixed entangled state which satisfies
$g^{(3)}(0)>g^{(2)}(0)>1$. This behavior of the correlation functions is a
rather generic feature. As an example, we consider another mixed entangled
state $\hat{\rho}_{pol}(\bar{n})=\sum_{n}P_{n}(\bar{n})\ket{n,+}\bra{n,+}$.
Since the probabilities $P_{n}(\bar{n})=e^{-\bar{n}}\bar{n}^{n}/(n!)$ are
Poisson weights, this is a polariton coherent state. However, the reduced
photon state has super-poissonian statistics, i.e., the second- and third-
order coherence functions can not be expected to be unity. Hence, matter-light
entanglement is a viable alternative to the standard explanation of the
unexpected behavior of these functions, based on polariton-polariton and
polariton-phonon interactions.

The differential energy per excitation $\nu$ is plotted in fig. 2 as a
function of $P$ for several values of the detuning. For $\Delta=0$, $\nu$ is
independent of $P$ and equals to the cavity mode energy, since the number of
photons becomes much larger than the number of matter excitations. Our
numerical results show that for $|\Delta|\gtrsim g$, and small ($P<10^{-2}$
meV) or large ($10$ meV $>P>10^{-1}$ meV) pumping rates, $\nu$ is almost a
constant, equal to the exciton (photon) energy in the former (latter) case.
The same constant values would had been obtained with the definition
$\tilde{\nu}=H/N$. For intermediate values of $P$, $\nu$ displays an inflexion
point at $P=\kappa$, where $\nu=\omega_{C}-(\Delta/2)$ is halfway between the
exciton and photon energies. Moreover, since the mean number of excitations is
one, it is tempting to define $P=\kappa$ as the condition for the “optimum”
polariton. In order to quantify this idea we compare the steady state with the
polariton states $\ket{n,\pm}$ defined by (2), using the sequence of non-zero
fidelities $F_{n\pm}=\sqrt{\braket{n,\pm}{\hat{\rho}_{ss}}{n,\pm}}$. For small
values of $P$ (fig.3.a), when $\rho_{G0G0}$ is much larger than the other
populations, only $F_{1-}$ does not vanish –however it is relatively small–.
For $P=\kappa$ (fig.3.b) $F_{1-}$ increases up to more than 0.95, while the
remaining fidelities are still small. Hence, the steady state
$\hat{\rho}_{ss}$ is quite similar to the $\Lambda_{1}$–polariton $\ket{1-}$.
As $P$ increases, $F_{n\pm}$ is non-zero for larger excitation-numbers, but
their values are very small.

Figure 3: (color online). Sequence of non-zero fidelities $F_{n\pm}$ between
the steady state $\hat{\rho}_{ss}$ and the
$\Lambda_{n}$–lower(black)$/$upper(green) polaritons $\ket{n,\pm}$ for
$\kappa=5\times 10^{-2}$ meV, $\Delta=3$ meV and (a) $P=2\times 10^{-3}$ meV,
(b) $P=5\times 10^{-2}$ meV, (c) $P=1$ meV and (d) $P=5$ meV. $|\Lambda_{n}|$
denotes the excitation number of the polariton manifold $\Lambda_{n}$.

The excitation number ($\hat{N}$) symmetry associated with the Hamiltonian (1)
is broken in the time evolution provided by the master equation (3), in the
sense that the asymptotic state of the system cannot be labeled with a single
eigenvalue of $\hat{N}$. If polariton-like behavior is actually present,
restoration of the symmetry is expected. In order to account for this effect,
we introduce the participation ratio $PR=\sum_{n=0}^{\infty}P_{n}^{2}$, where
$P_{n}$ is the probability to have $n$ excitations in the asymptotic state.
This quantity, which varies from zero –all excitation numbers are
equiprobable– to one –only one occupied manifold–, displays a global maximum
at zero pumping rate and a local maximum at $P\approx\kappa$, in the strong
coupling regime, signaling a partial restoration of symmetry. This can be
understood as a combined effect of the decrease of the mixedness of the state
and the increase of its entanglement, occurring at $P\approx\kappa$, as
discussed above (again, $\Delta\sim g$).

As we have seen, the optimum polariton exhibits maximum negativity, minimum
linear entropy, a local maximum of the participation ratio and an inflection
point of the differential energy per excitation, provided that the system is
in strong coupling. To quantify our previous qualitative arguments –which show
that polaritons cannot be sustained neither for small nor for large
detunings–, it is worth examining the behavior of the linear entropy and the
negativity at $P=\kappa$, as a function of the detuning (fig.4). Three regions
can be identified. In the first region, $|\Delta|<0.68$ meV, the negativity of
the steady state is exactly zero and the linear entropy is larger than $0.72$.
In the second region, $0.68$ meV $<|\Delta|\lesssim 10$ meV, the linear
entropy and the negativity are still significative. In the third region the
dissipative polariton is very close to the Hamiltonian polariton
$\ket{1,\pm}$, for the corresponding $\Delta$. Nevertheless, the dressed
states (2) are virtually the bare states.

Figure 4: (color online). Linear entropy (dashed black line) and negativity
(continuous red line) of the steady state of the system $\hat{\rho}_{ss}$, and
negativity of polaritons of the excitation manifold $\Lambda_{1}$ (dashed-
dotted blue line) as a function of $\Delta$ for $P=\kappa=5\times 10^{-2}$
meV.

Our results might provide a guide to experiments, in the sense that, in the
strong-coupling regime, both the pumping rate and the detuning have to be
carefully adjusted. In our model, the best matter-light correlation properties
occur at $P=\kappa$ and $\Delta\approx 3g$, where $\mathcal{N}\approx 0.32$
and $S_{L}\approx 0.29$. From the theoretical point of view we propose the
following criterion: if the negativity rises above 0.25, close to its maximum
possible value, its inflection points, as a function of the pumping power, can
be used to define the polaritonic regime. When this condition is fulfilled all
the quantifiers that we have examined exhibit a characteristic change. The
emission energy presents a blueshift, the differential energy per excitation
has an inflection point, the emission line decreases, the second- and third-
order correlation functions increase beyond their value for photon coherent
states, the entropy decreases and the negativity increases. With the exception
of the first two, these changes can be understood as a coherence gain of the
asymptotic state of the system.

We are grateful with Prof. P.S.S. Guimarães from UFMG (Brazil), Prof. J.
Mahecha from UdeA (Colombia) and Dra. J. Restrepo from UAN (Colombia) for
critical reading of the manuscript. The authors acknowledge partial financial
support from Dirección de Investigación - Sede Bogotá, Universidad Nacional de
Colombia (DIB-UNAL) under project 12584, and technical and computational
support from Grupo de Óptica e Información Cuántica (GOIC-UNAL), and Grupo de
Física Atómica y Molecular (GFAM-UDEA).

## References

  * Kasprzak  _et al._ (2006) J. Kasprzak, M. Richard, S. Kundermann, A. Baas, P. Jeambrun, J. Keeling, F. Marchetti, M. Szymanska, R. Andre, J. Staehli, V. Savona, P. Littlewood, B. Deveaud, and L. Dang, Nature 443, 409 (2006).
  * Deng  _et al._ (2006) H. Deng, D. Press, S. Götzinger, G. S. Solomon, R. Hey, K. H. Ploog, and Y. Yamamoto, Phys. Rev. Lett. 97, 146402 (2006).
  * Deng  _et al._ (2010) H. Deng, H. Haug, and Y. Yamamoto, Rev. Mod. Phys. 82, 1489 (2010).
  * Snoke (2006) D. Snoke, Nature 403, 047401 (2006).
  * Bajoni  _et al._ (2008) D. Bajoni, P. Senelllart, E. Wertz, I. Sagnes, A. Miard, A. Lemaitre, and J. Bloch, Phys. Rev. Lett. 100, 047401 (2008).
  * Azzini  _et al._ (2011) S. Azzini, D. Gerace, M. Galli, I. Sagnes, R. Braive, A. Lemaître, J. Bloch, and D. Bajoni, Appl. Phys. Lett. 99, 111106 (2011).
  * Christopoulos  _et al._ (2007) S. Christopoulos, G. von Högersthal, A. Grundy, P. Lagoudakis, A. Kavokin, J. Baumberg, G. Christmann, R. Butté, E. Feltin, J.-F. Carlin, and N. Grandjean, Phys. Rev. Lett. 98, 126405 (2007).
  * Littlewood  _et al._ (2004) P. Littlewood, P. Eastham, J. Keeling, F. Marchetti, B. Simons, and M. Szymanska, Journal of Physics: Condensed Matter 16, S3597 (2004).
  * Kamide and Ogawa (2010) K. Kamide and T. Ogawa, Phys. Rev. Lett. 105, 056401 (2010).
  * Vinck-Posada  _et al._ (2005) H. Vinck-Posada, B. A. Rodriguez, and A. Gonzalez, Physica E: Low-dimensional Systems and Nanostructures 27, 427 (2005).
  * Lagoudakis  _et al._ (2011) K. G. Lagoudakis, F. Manni, B. Pietka, M. Wouters, T. C. H. Liew, V. Savona, A. V. Kavokin, R. André, and B. Deveaud-Plédran, Phys. Rev. Lett. 106, 115301 (2011).
  * Liew  _et al._ (2008) T. C. H. Liew, Y. G. Rubo, and A. V. Kavokin, Phys. Rev. Lett. 101, 187401 (2008).
  * Vera  _et al._ (2009a) C. Vera, A. Cabo, and A. González, Phys. Rev. Lett. 102, 126404 (2009a).
  * Perea  _et al._ (2004) J. Perea, D. Porras, and C. Tejedor, Phys. Rev. B 70, 115304 (2004).
  * Laussy  _et al._ (2009) F. Laussy, E. del Valle, and C. Tejedor, Phys. Rev. B 79, 235325 (2009).
  * del Valle  _et al._ (2009) E. del Valle, F. Laussy, and C. Tejedor, Phys. Rev. B 79, 235326 (2009).
  * Laussy  _et al._ (2011) F. Laussy, A. Laucht, E. del Valle, J. Finley, and J. Villas-Bôas, Phys. Rev. B 84, 195313 (2011).
  * Vera  _et al._ (2009b) C. Vera, N. Quesada, H. Vinck-Posada, and B. Rodríguez, J. Phys.: Condens. Matter 21, 395603 (2009b).
  * del Valle  _et al._ (2007) E. del Valle, F. Laussy, F. Troiani, and C. Tejedor, Phys. Rev. B 76, 235317 (2007).
  * Auffèves  _et al._ (2009) A. Auffèves, J.-M. Gérard, and J.-P. Poizat, Phys. Rev. A 79, 053838 (2009).
  * Laucht  _et al._ (2009) A. Laucht, N. Hauke, J. M. Villas-Bôas, F. Hofbauer, G. Böhm, M. Kaniber, and J. Finley, Phys. Rev. Lett. 103, 087405 (2009).
  * Reithmaier  _et al._ (2004) J. P. Reithmaier, G. Sek, A. Löffler, C. Hofmann, S. Kuhn, S. Reitzenstein, L. V. Keldysh, V. D. Kulakovskii, T. L. Reinecke, and A. Forchel, Nature 432, 197 (2004).
  * Rice and Carmichael (1994) P. R. Rice and H. J. Carmichael, Phys. Rev. A 50, 4318 (1994).
  * Horikiri  _et al._ (2010) T. Horikiri, P. Schwendimann, A. Quattropani, S. Höfling, A. Forchel, and Y. Yamamoto, Phys. Rev. B 81, 033307 (2010).
  * Mølmer (1997) K. Mølmer, Phys. Rev. A 55, 3195 (1997).
  * Życzkowski  _et al._ (1998) K. Życzkowski, P. Horodecki, A. Sanpera, and M. Lewenstein, Phys. Rev. A 58, 883 (1998).
  * Vidal and Werner (2002) G. Vidal and R. F. Werner, Phys. Rev. A 65, 032314 (2002).