11institutetext:  Tsinghua University 22institutetext: Megvii Technology

# Practical Deep Raw Image Denoising
on Mobile Devices

Yuzhi Wang 1122 Haibin Huang 22 Qin Xu 22 Jiaming Liu 22 Yiqun Liu 11 Jue Wang
22

###### Abstract

Deep learning-based image denoising approaches have been extensively studied
in recent years, prevailing in many public benchmark datasets. However, the
stat-of-the-art networks are computationally too expensive to be directly
applied on mobile devices. In this work, we propose a light-weight, efficient
neural network-based raw image denoiser that runs smoothly on mainstream
mobile devices, and produces high quality denoising results. Our key insights
are twofold: (1) by measuring and estimating sensor noise level, a smaller
network trained on synthetic sensor-specific data can out-perform larger ones
trained on general data; (2) the large noise level variation under different
ISO settings can be removed by a novel k-Sigma Transform, allowing a small
network to efficiently handle a wide range of noise levels. We conduct
extensive experiments to demonstrate the efficiency and accuracy of our
approach. Our proposed mobile-friendly denoising model runs at $\sim$70
milliseconds per megapixel on Qualcomm Snapdragon 855 chipset, and it is the
basis of the night shot feature of several flagship smartphones released in
2019.

00footnotetext: This work is supported by The National Key Research and
Development Program of China under Grant 2018YFC0831700.

##  1 Introduction

Figure 1: Our proposed denoising method can run smoothly on smartphones with
high quality noise reduction even in low-light conditions (see (b),(d)). In
contrast, the default ISP image denoiser produces images with over-smoothed
high texture regions (e.g. ground in (a)) and noisy smooth regions (e.g. sky
in (c)).

Smartphones have become the go-to devices for consumer photography in recent
years. Compared with DSLR cameras, images captured with mobile devices are
more easily contaminated with higher level of noise due to the use of
relatively low-cost sensors and lenses, especially in low-light scenarios.

Despite decades of development in image denoising technologies, it remains
challenging to restore high quality images from extremely noisy ones on mobile
devices. Recently, deep neural network (DNN) based denoising methods [36, 10,
42, 21, 38, 30, 37] have achieved tremendous success and outperformed most
traditional methods [43, 12, 18, 39, 40]. It is however not practical to
directly deploy these heavy-weight DNNs on mobile devices due to the limited
computational resources available on them.

In this work, we propose a simple yet efficient approach for deep raw image
denoising. It can run efficiently on off-the-shelf smartphones with high
quality noise reduction. Our key observation is that the noise characteristics
for a specific sensor model are consistent and can be measured with sufficient
accuracy. By capturing and modeling sensor noise, we can generate synthetic
datasets with clean and noisy image pairs, and train a light-weight neural
network on them. The trained model remains highly effective on real images
captured by the same sensor (i.e. the same smartphone model). Furthermore,
based on the parametric sensor noise model, we derive a unique linear
transform in luminance space, dubbed as k-Sigma Transform, that maps noisy
images captured under different ISO settings into an ISO-invariant signal-
noise space, allowing a single network to handle different noise levels in
different scenes. We show that this approach is not only theoretically
elegant, but in practice is more efficient than training a separate model for
each ISO setting, or having one large model trained on images with different
noise levels.

To summarize, the main contributions of this work are as follows:

  * •

A systematic approach to estimate sensor noise and train a sensor-specific
denoising neural network using properly constructed synthetic data.

  * •

A novel k-Sigma Transform to map noisy images under different ISO settings
into a ISO-invariant signal-noise space. Instead of training separate models
per ISO or a larger model to cover the variations, the proposed transform
allows a single small network trained in this space to handle images with
different noise levels.

  * •

A mobile-friendly network architecture for efficient image denoising. We
provide in-depth analysis and comparison with different network architectures
and denoising methods, demonstrating that our method has compatible
performance with state-of-the-art approaches with significantly less
computational resources.

To the best of our knowledge, our solution is the first practical deep-
learning-based image denoising approach that has satisfactory efficiency and
accuracy on mobile devices. In Fig. 1 we show examplar images captured by an
off-the-shelf smartphone that use our approach in low-light photography.
Compared with the default ISP image denoising technique, our results contain
much more fine details of the scene.

##  2 Related work

Image denoising is a fundamental task in image processing and computer vision.
Classical methods often rely on using sparse image priors, such as non-local
means (NLM) [6], sparse coding [13, 28, 2], 3D transform-domain filtering
(BM3D) [12], and others [18, 32]. Among them BM3D is usually deemed as the
leading method considering its accuracy and robustness. Most of these methods
are designed for general noise and do not take advantage of known sensor noise
characteristics. Their algorithmic complexity is usually high, making full-
fledged implementation difficult on smartphones.

With the recent development of convolutional neural networks (CNNs), training
end-to-end denoising CNNs has gained considerable attention. Earlier work that
uses multi-layer perceptron (MLP) [7] has achieved comparable results with
BM3D. Further improvements have been achieved with the introduction of more
advanced network architectures, resulting in a large number of CNN-based
denoising methods [36, 10, 21, 38, 30, 37, 41, 23]. These works are primarily
focused on novel network structures for improving the accuracy, without paying
much attention to their adaptability to mobile devices.

Our work focuses on denoising raw image, i.e., images read out from the sensor
in the raw Bayer format before demosaicing and other ISP-processing. On the
recently proposed public raw image denoising benchmark datasets [1, 8, 3],
CNN-based methods [8, 20, 17] have achieved the best results. It is however a
very tedious work to construct such high quality real datasets with clean and
noisy image pairs. Thus, the problem of synthesizing realistic image noise for
training has also been extensively studied, including Gaussian-Poisson
noise[16, 27], Gaussian Mixture Model (GMM) [44], in-camera process simulation
[25, 34], GAN-generated noises [9] and so on. It has been shown that networks
properly trained from the synthetic data can generalize well to real data [42,
5].

The existing best practice for raw image denoising on mobile devices is to
capture and merge multiple frames [19, 24, 31]. These methods generally
require accurate and fast image alignment, which is hard to achieve when
moving objects present in the scene. Furthermore, when noise level is high,
averaging multiple frames can reduce, but not completely remove the noise,
leading to unsatisfactory results. To the best of our knowledge, our proposed
method is the first single frame, deep learning-based raw image denoiser
specifically designed for mobile devices.

##  3 Method

In this section, we first revisit the general ISO-dependent noise model of
camera sensor, and then describe how to estimate noise parameters given a new
sensor. We further show how to synthesize ISO-independent training data using
the proposed k-Sigma Transform, and use it to train a small neural network
that can handle a wide range of noise levels.

Figure 2: Photon transfer pipeline: multiple noise sources like shot noise,
read-out noise and thermal noise are involved along the camera pipeline. Check
[14] for more details.

###  3.1 The Noise Model

A camera sensor converts the photons hitting the pixel area during the
exposure time into a digitized luminance map. As shown in the photon transfer
pipeline shown in Fig. 2, this process contains multiple stages, where each
stage introduces specific noise. Let us first consider an ideal system with no
noise. Under the linear camera model, at each pixel, the sensor conversion is
a linear amplification as:

$x^{*}=g\alpha u^{*},$ (1)

where $u^{*}$ is the expected number of photons hitting the pixel area,
$\alpha$ is the quantum efficiency factor and $g$ is the analog gain. Now
considering the system noise in each step of the pipeline in Fig. 2, we have:

$x=g(\alpha u+n_{\text{d}})+n_{\text{r}},$ (2)

where $u$ denotes the actual collected amount of photons, and
$n_{d}\sim\mathcal{N}(0,\sigma_{d}^{2})$ and
$n_{r}\sim\mathcal{N}(0,\sigma_{r}^{2})$ are Gaussian noise before and after
applying the analog gain. Furthermore, it is demonstrated in [14] that $u$
obeys a Poisson distribution of $u^{*}$, given by

$u\sim\mathcal{P}(u^{*}).$ (3)

Combining Eqn. (1) to Eqn. (3), we have:

$x\sim(g\alpha)\mathcal{P}(\frac{x^{*}}{g\alpha})+\mathcal{N}(0,g^{2}\sigma_{\text{d}}^{2}+\sigma_{\text{r}}^{2}).$
(4)

This is consistent with the Poisson-Gaussian noise model that has been
extensively studied in previous work [16, 27]. This formulation can be further
simplified by replacing $k=g\alpha$ and
$\sigma^{2}=g^{2}\sigma_{d}^{2}+\sigma_{r}^{2}$:

$x\sim k\mathcal{P}(\frac{x^{*}}{k})+\mathcal{N}(0,\sigma^{2}).$ (5)

Note that both $k$ and $\sigma^{2}$ are related to $g$, which is determined by
the ISO setting of the camera.

###  3.2 Parameter Estimation

To sample the distribution described in Eqn. (5), we need an accurate
estimation of $k$ and $\sigma$ under a specified ISO setting of a specific
sensor. Luckily, as we check the mean and variance over $x$, shown in Eqn.
(6), we can turn it into the following linear regression problem:

$\left\\{\begin{array}[]{ll}\mathrm{E}(x)&=x^{*},\\\
\mathrm{Var}(x)&=kx^{*}+\sigma^{2}.\end{array}\right.$ (6)

Figure 3: Noise parameter estimation with a burst series of raw images of a
static grayscale chart.

Similar to [15], we capture a series of raw images of a static grayscale chart
in burst mode, depicted in Fig. 3a, and compute $\mathrm{E}(x)$ from the
series of luminance values at the same pixel location. Next, as shown in Fig.
3b, we bracket all pixels that have the same estimated luminance, and compute
$\mathrm{Var}(x)$ from them. A linear regression is then applied to find the
optimal estimation of $k$ and $\sigma^{2}$, illustrated in Fig. 3c.

###  3.3 The k-Sigma Transform

In real applications the camera will automatically adjust the ISO settings
according to the scene illumination, thus one has to consider different noise
levels when training the denoising neural network. A straightforward solution
is to train a single network to cover a wide range of ISO settings, but it
puts extra burden on the network itself as the noise variation in the training
data becomes quite large. Inspired by variance stabilizing transformations [4,
29], here we propose a k-Sigma Transform to avoid this problem.

Specifically, we define a linear transform

$f(x)=\frac{x}{k}+\frac{\sigma^{2}}{k^{2}}.$ (7)

According to our noise model of Eqn. (5),

$f(x)\sim\mathcal{P}(\frac{x^{*}}{k})+\mathcal{N}(\frac{\sigma^{2}}{k^{2}},\frac{\sigma^{2}}{k^{2}}).$
(8)

To analyze this distribution, a usual simplification is to treat the Poisson
distribution $\mathcal{P}(\lambda)$ as a Gaussian distribution of
$\mathcal{N}(\lambda,\lambda)$ [16]. Therefore:

$\displaystyle
P(\frac{x^{*}}{k})+\mathcal{N}(\frac{\sigma^{2}}{k^{2}},\frac{\sigma^{2}}{k^{2}})$
(9) $\displaystyle\approx$
$\displaystyle\mathcal{N}(\frac{x^{*}}{k},\frac{x^{*}}{k})+\mathcal{N}(\frac{\sigma^{2}}{k^{2}},\frac{\sigma^{2}}{k^{2}})$
$\displaystyle=$
$\displaystyle\mathcal{N}(\frac{x^{*}}{k}+\frac{\sigma^{2}}{k^{2}},\frac{x^{*}}{k}+\frac{\sigma^{2}}{k^{2}})$
$\displaystyle=$ $\displaystyle\mathcal{N}[f(x^{*}),f(x^{*})].$

Combining Eqn. (8) and Eqn. (9), the approximate distribution of $f(x)$ is:

$f(x)\sim\mathcal{N}[f(x^{*}),f(x^{*})].$ (10)

Eqn. (10) indicates that the distribution of $f(x)$ only depends on
$f(x^{*})$. As shown in Fig. 4, we can train a single network that takes
$f(x)$ as input and outputs $f(\hat{x}^{*})$ as an estimation of $f(x^{*})$.
The estimated true image value $x^{*}$ can then be computed by applying the
inverted k-Sigma Transform $f^{-1}(\cdot)$ to $f(\hat{x}^{*})$. In other
words, we apply ISO-dependent transforms to the input and output of the neural
network, so that the network can be trained using normalized data without
considering the ISO setting.

Figure 4: The pipeline of running ISO-independent denoising network with
k-Sigma Transform.

##  4 Learning to Denoise

###  4.1 Mobile-friendly Network Architecture

We further introduce a mobile-friendly convolutional neural network for image
denoising, as shown in Fig. 5. We use a U-Net-like [33] architecture with 4
encoder and 4 decoder stages with skip connections, illustrated in Fig. 5(a).

(a) The U-Net-like overall structure of the denoising network.

(b) Detailed stucture of network blocks.

Figure 5: The architecture of the proposed denoising network.

Fig. 5(b) depicts the detailed structures of the network blocks. Specifically,
in order to run on mobile devices, we use separable-conv [11] in all encoder
and decoder stages to reduce the computation cost, and normal dense
convolution layers are only used in the input and output stage. In encoders,
we use $5\times 5$ kernel size to increase receptive field and decrease
network depth, and downsample feature maps with stride-2 convolutions. In
decoder, we only use $3\times 3$-speconv and upsample feature maps with
$2\times 2$ deconvolutions. The inputs of each encoder stages are combined
into its corresponding decoder stage by element-wise adding, a $3\times
3$-speconv is adopted in the skip connect to match the channel shape. Finally,
the last convolution layer outputs a residual added to the input image as the
denoising result.

###  4.2 Training Dataset

To train our denoising network, we need pairs of noisy and clean RAW images.
In this paper, we use a subset of See-in-the-Dark (SID) dataset proposed in
[8] as the ground truth clean images. The SID dataset contains RAW images
captured from a Sony $\alpha$7s II and a Fujifilm X-T2 camera, we choose the
10s and 30s long-exposure subset captured by the Sony $\alpha$7s II camera,
and manually take out those with visible noise, leaving 214 high quality RAW
images.

According to our noise model described in Section 3, if clean RAW images were
available, we can synthesize noisy images by sampling from a Poisson-Gaussian
distribution with estimated noise parameters measured from the target sensor.

###  4.3 Settings and Details

To generate tranining samples, we randomly crop $1024\times 1024$-sized bayer
patches from the original dataset. We adopt the bayer-aug method described in
[26] with random horizontal and vertical flipping and ensure the input bayer
pattern is in R-G-G-B order. We then pack the bayer image to $512\times
512\times 4$-shaped RGGB tensor. We also randomly adjust the brightness and
contrast of the cropped images for data augmentation. Noisy images are then
synthesized according to the noise model, with noise parameters of randomly
selected ISO value. Finally, we apply k-Sigma Transform to both the noisy and
clean images so that the denosing network is trained in the ISO-independent
space.

We use $\ell_{1}$ distance between the noisy and clean images as the loss
function, and train the network using Adam [22] optimizer. We adopt the
triangular cyclical learning rate scheduling [35] with the maximum learning
rate of 1e-3, cycle step of 50$\times$214 iterations, and the base learning
rate linearly decays to 1e-5 after 4000$\times$214 iterations. The batch size
is set to 1, and the training converges at 8000$\times$214 iterations.

##  5 Experiments

In this section, we evaluate our denoising method with a real world dataset
collected with an OPPO Reno-10x smartphone. This smartphone has three rear
cameras, and we use the most commonly used main camera for our test. The
sensor of this camera is Sony IMX586 sized at
$1\text{/}2\text{\,}\mathrm{\SIUnitSymbolArcsecond}$ with 48 megapixels, and
the pixel size is $0.8\text{\,}\mathrm{\SIUnitSymbolMicro m}$. This sensor is
widely used in many smartphones of 2019, including OPPO Reno series, Xiaomi 9,
etc.

###  5.1 Noise Parameters Estimation

We first measure and estimate the noise parameters with the method described
in Section 3.2. We write a simple app to collect RAW images with Android
Camera-2 API, which allows us manually control the camera’s ISO and exposure
time. To keep a stable light condition, we use an Xrite SpectraLight QC light
booth in a dark room. At each ISO and exposure time setting, we adjust the
luminance of the light source to avoid over or under exposure, and the final
values of the captured image are kept in an appropriate range.

At each ISO setting, 64 RAW images are captured in burst mode, and the mean
image is considered as the clean image. With the method described in Section
3.2, we can estimate the noise params $k$ and $\sigma^{2}$ at each specified
ISO setting. Fig. 6(a) plots the value-variance curve of the test phone under
ISO $4800$, where the scattered dots represent the measured variances
corresponding to each raw value on the mean image, and the blue line plots the
linear regression result of Eqn. (6). From the figure we can see that our
theoretical noise model can well fit the measurement results. The slope of the
fitted line is the estimated noise parameter $\hat{k}$ and the y-intercept
value is the estimated noise parameter $\hat{\sigma}^{2}$.

(a)

(b)

(c)

Figure 6: Noise param estimation of Reno-10x smartphone: (a) parameter
estimation at ISO-4800 (b) $k$ values at different ISOs (c) $\sigma^{2}$ at
differnt ISOs.

The adjustable range of IMX586 sensor analog gain is $[1.0,64.0]$,
corresponding to the ISO value of OPPO Reno-10x camera as $[100,6400]$.
According to our noise model Eqn. (4), the params $k$ and $\sigma^{2}$ are
linearly, and qudraticly correlated to the ISO value, respectively. We measure
and estimate the noise params at each ISO setting, and plot the ISO-$k$ and
ISO-$\sigma^{2}$ curve in Fig. 6. The scattered dots represent the estimated
noise params under each ISO setting, and the blue lines in Fig. 6(b) and Fig.
6(c) respectively represent the linearly and qudraticly fitted curves, which
demostrate that our theoretical model matches the measurements well.

With the ISO-$k$ and ISO-$\sigma^{2}$ curves well fitted, the noise params
under any ISO setting can be easily calculated, and thus satisfying the
requirements of both synthesizing training data and applying the k-Sigma
transform.

###  5.2 Test Dataset and Metrics

Since our proposed denoising network needs to be trained for specific sensors,
we cannot directly use public benchmarks such as SIDD [1] due to the
mismatching of sensors. Therefore, we build a testing dataset to evaluate our
denoising method.

We use an Xrite SpectraLight QC light booth in a dark room to build a stable
light condition. For each capturing process, 64 RAW images are captured and
the mean image can be used as the estimated ground truth. As shown in Fig. 7,
we capture 4 static scenes as the content of testing images, and 2 luminance
conditions are set for each scene. When capturing one scene, the camera
position and scene contents are kept fixed. We set 5 exposure combinations at
each scene and luminance, which are ISO-800@160ms, ISO-1600@80ms,
ISO-3200@40ms, ISO-4800@30ms and ISO-6400@20ms, respectively. These settings
share an identical value of the product of ISO and exposure time, so that the
captured images have similar brightness but different noise levels.

Figure 7: The 4 scenes from the test dataset: the blue boxes represent the
regions of interest where image quality metrics are calculated.

We use peak signal-noise-ratio (PSNR) and structural similarity (SSIM) to
measure the performance of denoising methods. The PSNR and SSIM between the
denoising results and the clean images are measured in sRGB domain with a
simple post processing pipeline, including  (1) white balance correction,  (2)
demosaicking,  (3) color correction, and  (4) gamma correction.  The
parameters for white balance correction and color correction are obtained from
the metadata of the RAW image. The demosaicking algorithm is pixel-grouping
(PPG) and the gamma value is $2.2$.

###  5.3 Results

We first show the comparison between our method and the previous state-of-the-
art raw image denoising method proposed in [26], which is trained and tested
using the SIDD dataset. An extra-large UNet-like network which costs 1T
multiply-and-cumulate operations (MACs) per megapixel is proposed in [26] and
achieved the state-of-the-art performance in NTIRE 2019 denoising challenge.
In addition to the UNet-1T architecture, we modify the network by reducing the
channel width and layer depth to fit various computation complexities. We
train these models with two different data sources: SIDD dataset and the
training data generated by our method.

As shown in Fig. 8, because of our accurate noise modeling, the models trained
with our synthetic data outperform those trained with the SIDD dataset by a
large margin. Moreover, our mobile-friendly network trained on our synthetic
data achieves comparable performance with the UNet-36G with only $10\%$ of its
computational complexity (3.6G vs 36.3G). More visual comparisons are provided
in Fig. 9.

$10^{1}$$10^{2}$$10^{3}$$38$$39$$40$Ours(39.76@3.6G)Ours + Liu’s Model(40.04@1T)Liu [26](39.49@1T)Computational Cost (MACs)PSNR ($\mathrm{dB}$)Ours+LiuLiu [26]Ours Figure 8: PSNRs at different computation costs of our method, our method using Liu’s model [26], and Liu’s method. Note that the MACs are in the log space. Table 1: Running time of denoising 1 MP on Qualcomm Snapdragon 855 GPU Method | Ours | UNet-5G | UNet-21G | UNet-36G
---|---|---|---|---
$\mathrm{ms}$/MP | 70.70 | 79.19 | 292.0 | 383.9

We further test the actual running time of different models on mobile devices,
listed in Table 1. Our mobile-friendly model can process a $1024\times 1024$
Bayer input with $70.70\text{\,}\mathrm{ms}$ (aka
$\sim$$850\text{\,}\mathrm{ms}$ for a 12MP full-sized image) on a Qualcomm
Snapdragon 855 GPU, while other models with comparable performance require
significantly longer time, $292\text{\,}\mathrm{ms}$ for 21G network and
$383\text{\,}\mathrm{ms}$ for 36G network (aka $4.45\text{\,}\mathrm{s}$ for
12MP), making them impractical to be deployed on mobile devices.

Figure 9: More visual results on our real dataset. From left to right: input
image; ground truth; result based on method in [26]; our result. Compared with
[26] which generates blurred areas, our method can efficiently reduce the
noise as well maintain underlying details. Moreover, our method utilizes a
significantly smaller network (3.6G vs 1T).

###  5.4 Ablation Studies

####  5.4.1 Data synthesis method

To verify the effectiveness of our data synthesis method, we train our
denosing network with four different training datasets, including

  * •

testset overfitting: directly use the inputs and ground truth from the testset
as the training data;

  * •

testset with synthetic: use the ground truth of the testset and add synthetic
noise as the input;

  * •

testset with noise scaling: use our noise synthesis method but scale the noise
parameters to purposely mismatch with the target sensor;

  * •

SID: synthesize the training set with SID dataset described in Section 4.2.

The test results are listed in Table 2. Not surprisingly, the overfitting
experiment obtains the highest PSNR of $40.06\text{\,}\mathrm{dB}$ and SSIM of
$0.9335$, which sets an upper-bound for this comparison. The testset with our
synthetic noise achieves the second best result of $39.97\text{\,}\mathrm{dB}$
PSNR. When using inaccurate noise parameters by scaling $k$ and $\sigma^{2}$
by 0.5 or 2.0, the network results in the lowest PSNRs in this experiment. Our
method of using the SID dataset achieves $39.76\text{\,}\mathrm{dB}$ PSNR.

This experiment shows that our noise model and synthetic noise can well match
the characteristics of the real input noise, and the testing results can be
close to the upper bound performance. Inaccurate noise parameters, even in the
overfitting experiment, lead to noticeable performance degradation.

Table 2: Comparison of training datasets and noise parameters Dataset | PSNR ($\mathrm{dB}$) | SSIM
---|---|---
testset overfitting | $40.057\,062\,88$ | $0.933\,505\,932$
testset+synthetic | $39.965\,606\,21$ | $0.933\,691\,115$
testset+synthetic scale 0.5 | $36.873\,840\,14$ | $0.885\,018\,911$
testset+synthetic scale 2.0 | $39.680\,421\,45$ | $0.930\,105\,38$
SID | $39.764\,889\,57$ | $0.931\,048\,368$

####  5.4.2 Robustness to ISO

We compare several strategies of denoising images of different ISO settings to
verify the effectiveness of the k-Sigma Transform. We compare two ways of
handling multiple ISO settings:  (1) iso-augmentation: randomly choose ISO
settings when synthesizing training samples and directly feed them into the
denosing network; and  (2) concat-variance: the method proposed in [5], where
the estimated noise variance are concatenated to the input as 4 additional
channels.  In addition, we also test the performance of the single-ISO method,
where the training data is synthesized using noise parameters of a single ISO.

The results are listed in Table 3. The concat-variance strategy achieves the
PSNR of $39.65\text{\,}\mathrm{dB}$, which is about $0.09\text{\,}\mathrm{dB}$
higher than the iso-augmentation strategy. This means that explicit noise
level information can help the model achieve better results than blind
denoising. With the proposed k-Sigma Transform, our network achieves the
highest PSNR in this experiment. In comparison, all single-ISO methods perform
much worse than multi-ISO ones.

Table 3: Strategies of denoising for multiple ISOs Method | PSNR ($\mathrm{dB}$) | SSIM
---|---|---
k-Sigma Transform | $39.76$ | $0.931\,048\,368$
concat-variance [5] | $39.653\,042\,266\,876$ | $0.930\,745\,060\,240\,976$
iso-augmentation | $39.566\,629\,78$ | $0.929\,922\,157$
single-iso-1600 | $35.735\,691\,715\,240\,29$ | $0.807\,202\,519\,813\,696\,7$
single-iso-3200 | $38.561\,628\,588\,767\,086$ | $0.908\,908\,179\,257\,805\,1$
single-iso-6400 | $38.159\,799\,6$ | $0.916\,650\,591$

Table 4 gives a more detailed analysis of single-ISO methods and our approach,
where the PSNRs measured in the testset are grouped into different ISO
settings. From the table we can see that when the ISO setting of the testset
matches with the single-ISO model, it can produce competitive denoising
results. Our method base on the k-Sigma Transform performs consistently well
under all ISO settings.

Table 4: PSNRs under different ISO settings | ISO-800 | ISO-1600 | ISO-3200 | ISO-4800 | ISO-6400
---|---|---|---|---|---
k-Sigma Transform | 43.21 | 41.48 | 39.49 | 38.17 | 36.94
single-iso-1600 | 42.96 | 41.48 | 35.01 | 31.33 | 28.86
single-iso-3200 | 41.79 | 40.87 | 39.51 | 36.97 | 34.08
single-iso-6400 | 39.59 | 38.59 | 38.11 | 37.80 | 36.91

##  6 Conclusion

We have presented a new raw image denoiser designed for mobile devices. By
accurate sensor noise estimation, we can utilize a light-weight network
trained on sensor-specific synthetic data that generalizes well to real noise.
We also propose a k-Sigma Transform to process the input and output data, so
that denoising can be learned in an ISO-independent space. This allows the
network to handle a wide range of noise level without increasing network
complexity. Our results show that the proposed method can achieve compatible
performance with state-of-the-art methods, which typically employ much larger
networks that cannot be directly applied for mobile applications.

In applications, our method can be integrated into existing camera pipeline
and replace its denoising component. Since our method can produce high quality
denoised raw images, it gives a strong base for the ISP to apply more
aggressive post-processing. Our methods have been featured in the night shot
mode of several flagship phones released in 2019, with stable and outstanding
performance on mobile devices.

In the future, we would like to explore how to further reduce the
computational complexity of the proposed method, so that we can apply it on
video streams in real-time. Also, we believe it will be interesting and
promising to explore deep learning based approaches for raw image processing
that can improve or even replace camera’s ISP pipeline.

## References

  * [1] Abdelhamed, A., Lin, S., Brown, M.S.: A high-quality denoising dataset for smartphone cameras. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2018)
  * [2] Aharon, M., Elad, M., Bruckstein, A., et al.: K-svd: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on signal processing 54(11), 4311 (2006)
  * [3] Anaya, J., Barbu, A.: Renoir–a dataset for real low-light image noise reduction. Journal of Visual Communication and Image Representation 51, 144–154 (2018)
  * [4] Anscombe, F.J.: The transformation of poisson, binomial and negative-binomial data. Biometrika 35(3/4), 246–254 (1948)
  * [5] Brooks, T., Mildenhall, B., Xue, T., Chen, J., Sharlet, D., Barron, J.T.: Unprocessing images for learned raw denoising. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 11036–11045 (2019)
  * [6] Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05). vol. 2, pp. 60–65. IEEE (2005)
  * [7] Burger, H.C., Schuler, C.J., Harmeling, S.: Image denoising: Can plain neural networks compete with bm3d? In: CVPR (2012)
  * [8] Chen, C., Chen, Q., Xu, J., Koltun, V.: Learning to see in the dark. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 3291–3300 (2018)
  * [9] Chen, J., Chen, J., Chao, H., Yang, M.: Image blind denoising with generative adversarial network based noise modeling. In: CVPR (2018)
  * [10] Chen, Y., Pock, T.: Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration. IEEE transactions on pattern analysis and machine intelligence 39(6), 1256–1272 (2017)
  * [11] Chollet, F.: Xception: Deep Learning with Depthwise Separable Convolutions (oct 2016), http://arxiv.org/abs/1610.02357
  * [12] Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image restoration by sparse 3d transform-domain collaborative filtering. In: Image Processing: Algorithms and Systems VI. vol. 6812, p. 681207. International Society for Optics and Photonics (2008)
  * [13] Elad, M., Aharon, M.: Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image processing 15(12), 3736–3745 (2006)
  * [14] European Machine Vision Association.: Standard for Characterization of Image Sensors and Cameras (2010). https://doi.org/10.1063/1.1518010
  * [15] Foi, A., Alenius, S., Katkovnik, V., Egiazarian, K.: Noise measurement for raw-data of digital imaging sensors by automatic segmentation of nonuniform targets. IEEE Sensors Journal 7(10), 1456–1461 (2007)
  * [16] Foi, A., Trimeche, M., Katkovnik, V., Egiazarian, K.: Practical poissonian-gaussian noise modeling and fitting for single-image raw-data. IEEE Transactions on Image Processing 17(10), 1737–1754 (2008)
  * [17] Gharbi, M., Chaurasia, G., Paris, S., Durand, F.: Deep joint demosaicking and denoising. ACM Transactions on Graphics (TOG) 35(6), 191 (2016)
  * [18] Gu, S., Zhang, L., Zuo, W., Feng, X.: Weighted nuclear norm minimization with application to image denoising. In: CVPR (2014)
  * [19] Hasinoff, S.W., Sharlet, D., Geiss, R., Adams, A., Barron, J.T., Kainz, F., Chen, J., Levoy, M.: Burst photography for high dynamic range and low-light imaging on mobile cameras. ACM Transactions on Graphics 35(6), 1–12 (2016). https://doi.org/10.1145/2980179.2980254, http://dl.acm.org/citation.cfm?doid=2980179.2980254
  * [20] Hirakawa, K., Parks, T.W.: Joint demosaicing and denoising. IEEE Transactions on Image Processing 15(8), 2146–2157 (2006)
  * [21] Jain, V., Seung, S.: Natural image denoising with convolutional networks. In: Advances in neural information processing systems. pp. 769–776 (2009)
  * [22] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  * [23] Lehtinen, J., Munkberg, J., Hasselgren, J., Laine, S., Karras, T., Aittala, M., Aila, T.: Noise2noise: Learning image restoration without clean data. arXiv preprint arXiv:1803.04189 (2018)
  * [24] Liba, O., Murthy, K., Tsai, Y.T., Brooks, T., Xue, T., Karnad, N., He, Q., Barron, J.T., Sharlet, D., Geiss, R., Hasinoff, S.W., Pritch, Y., Levoy, M.: Handheld mobile photography in very low light. ACM Transactions on Graphics 38(6) (2019). https://doi.org/10.1145/3355089.3356508
  * [25] Liu, C., Szeliski, R., Kang, S.B., Zitnick, C.L., Freeman, W.T.: Automatic estimation and removal of noise from a single image. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 299–314 (2008)
  * [26] Liu, J., Wu, C.H., Wang, Y., Xu, Q., Zhou, Y., Huang, H., Wang, C., Cai, S., Ding, Y., Fan, H., Wang, J.: Learning Raw Image Denoising with Bayer Pattern Unification and Bayer Preserving Augmentation (apr 2019), http://arxiv.org/abs/1904.12945
  * [27] Liu, X., Tanaka, M., Okutomi, M.: Practical signal-dependent noise parameter estimation from a single noisy image. IEEE Transactions on Image Processing 23(10), 4361–4371 (2014)
  * [28] Mairal, J., Bach, F.R., Ponce, J., Sapiro, G., Zisserman, A.: Non-local sparse models for image restoration. In: ICCV. vol. 29, pp. 54–62. Citeseer (2009)
  * [29] Makitalo, M., Foi, A.: Optimal inversion of the anscombe transformation in low-count poisson image denoising. IEEE transactions on Image Processing 20(1), 99–109 (2010)
  * [30] Mao, X., Shen, C., Yang, Y.B.: Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In: NeurIPS (2016)
  * [31] Mildenhall, B., Barron, J.T., Chen, J., Sharlet, D., Ng, R., Carroll, R.: Burst Denoising with Kernel Prediction Networks (dec 2017), https://arxiv.org/abs/1712.02327
  * [32] Portilla, J., Strela, V., Wainwright, M.J., Simoncelli, E.P.: Image denoising using scale mixtures of gaussians in the wavelet domain. IEEE Trans Image Processing 12(11) (2003)
  * [33] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. pp. 234–241. Springer (2015)
  * [34] Shi, G., Zifei, Y., Kai, Z., Wangmeng, Z., Lei, Z.: Toward convolutional blind denoising of real photographs. In: arXiv preprint arXiv:1807.04686 (2018)
  * [35] Smith, L.N.: Cyclical learning rates for training neural networks. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV). pp. 464–472. IEEE (2017)
  * [36] Tai, Y., Yang, J., Liu, X., Xu, C.: Memnet: A persistent memory network for image restoration. In: Proceedings of the IEEE international conference on computer vision. pp. 4539–4547 (2017)
  * [37] Ulyanov, D., Vedaldi, A., Lempitsky, V.: Deep image prior. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 9446–9454 (2018)
  * [38] Xie, J., Xu, L., Chen, E.: Image denoising and inpainting with deep neural networks. In: Advances in neural information processing systems. pp. 341–349 (2012)
  * [39] Xu, J., Zhang, L., Zhang, D., Feng, X.: Multi-channel weighted nuclear norm minimization for real color image denoising. In: ICCV (2017)
  * [40] Yair, N., Michaeli, T.: Multi-scale weighted nuclear norm image restoration. In: CVPR (2018)
  * [41] Zhang, K., Zuo, W., Zhang, L.: Ffdnet: Toward a fast and flexible solution for cnn based image denoising. IEEE Transactions on Image Processing (2018)
  * [42] Zhou, Y., Jiao, J., Huang, H., Wang, Y., Wang, J., Shi, H., Huang, T.: When awgn-based denoiser meets real noises. arXiv preprint arXiv:1904.03485 (2019)
  * [43] Zhou, Y., Liu, D., Huang, T.: Survey of face detection on low-quality images. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018). pp. 769–773. IEEE (2018)
  * [44] Zhu, F., Chen, G., Heng, P.A.: From noise modeling to blind image denoising. In: CVPR (2016)


* Guo and Caliskan (2021) Wei Guo and Aylin Caliskan. 2021.  Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases. In _AAAI/ACM Conference on AI, Ethics, and Society_. 122–133.
  * Gupta et al. (2022) Umang Gupta et al. 2022\.  Mitigating Gender Bias in Distilled Language Models via Counterfactual Role Reversal.  https://arxiv.org/abs/2203.12574
  * Gyamfi et al. (2020) Enoch Opanin Gyamfi, Yunbo Rao, Miao Gou, and Yanhua Shao. 2020\.  Deb2viz: Debiasing gender in word embedding data using subspace visualization.  _Proceedings of SPIE - The International Society for Optical Engineering_ 11373 (2020).
  * Haas (2019) Christian Haas. 2019\.  The price of fairness-A framework to explore trade-offs in algorithmic fairness. In _40th International Conference on Information Systems, ICIS 2019_. Association for Information Systems.
  * Hendricks et al. (2018) Lisa Anne Hendricks, Kaylee Burns, Kate Saenko, Trevor Darrell, and Anna Rohrbach. 2018\.  Women also snowboard: Overcoming bias in captioning models. In _Proceedings of the European Conference on Computer Vision (ECCV)_. 771–787.
  * Higgins et al. (2018) Irina Higgins, David Amos, David Pfau, Sebastien Racaniere, Loic Matthey, Danilo Rezende, and Alexander Lerchner. 2018.  Towards a Definition of Disentangled Representations.  https://arxiv.org/abs/1812.02230
  * Higgins et al. (2017) Irina Higgins, Loïc Matthey, Arka Pal, Christopher P. Burgess, Xavier Glorot, Matthew M. Botvinick, Shakir Mohamed, and Alexander Lerchner. 2017.  beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework. In _ICLR_.
  * Hinton et al. (2015) Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. 2015\.  Distilling the Knowledge in a Neural Network.  _CoRR_ abs/1503.02531 (2015).  arXiv:1503.02531 http://arxiv.org/abs/1503.02531
  * Hirota et al. (2022) Yusuke Hirota, Yuta Nakashima, and Noa Garcia. 2022\.  Quantifying Societal Bias Amplification in Image Captioning. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 13450–13459.
  * Holland et al. (2020) Sarah Holland, Ahmed Hosny, Sarah Newman, Joshua Joseph, and Kasia Chmielinski. 2020.  The dataset nutrition label.  _Data Protection and Privacy, Volume 12: Data Protection and Democracy_ 12 (2020), 1\.
  * Holstein et al. (2019) Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé III, Miro Dudik, and Hanna Wallach. 2019\.  Improving fairness in machine learning systems: What do industry practitioners need?. In _Proceedings of the 2019 CHI conference on human factors in computing systems_. 1–16.
  * Jain et al. (2021) Bhanu Jain, Manfred Huber, and Ramez Elmasri. 2021\.  Increasing Fairness in Predictions Using Bias Parity Score Based Loss Function Regularization.  https://arxiv.org/abs/2111.03638
  * Julia Angwin and Kirchner (2016) Surya Mattu Julia Angwin, Jeff Larson and Lauren Kirchner. 2016.  Machine Bias.  https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing.  Accessed: 2022-08-23.
  * Jung et al. (2021) Sangwon Jung, Donggyu Lee, Taeeon Park, and Taesup Moon. 2021\.  Fair Feature Distillation for Visual Recognition. In _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE.
  * Kaneko and Bollegala (2019) Masahiro Kaneko and Danushka Bollegala. 2019.  Gender-preserving Debiasing for Pre-trained Word Embeddings. In _Conference of the Association for Computational Linguistics_. 1641–1650.
  * Kaneko and Bollegala (2021a) Masahiro Kaneko and Danushka Bollegala. 2021a.  Debiasing Pre-trained Contextualised Embeddings. In _Conference of the European Chapter of the Association for Computational Linguistics_. 1256–1266.
  * Kaneko and Bollegala (2021b) Masahiro Kaneko and Danushka Bollegala. 2021b.  Dictionary-based Debiasing of Pre-trained Word Embeddings. In _Conference of the European Chapter of the Association for Computational Linguistics_. 212–223.
  * Kaneko and Bollegala (2021c) Masahiro Kaneko and Danushka Bollegala. 2021c.  Unmasking the Mask–Evaluating Social Biases in Masked Language Models.  _arXiv preprint arXiv:2104.07496_ (2021).
  * Kang et al. (2022) Sunghun Kang, Gwangsu Kim, and Chang D Yoo. 2022.  Fair Facial Attribute Classification via Causal Graph-Based Attribute Translation.  _Sensors_ 22, 14 (2022), 5271.
  * Karakas et al. (2022) Cemre Karakas, Alara Dirik, Eylul Yalcinkaya, and Pinar Yanardag. 2022.  FairStyle: Debiasing StyleGAN2 with Style Channel Manipulations.  _CoRR_ abs/2202.06240 (2022).  arXiv:2202.06240 https://arxiv.org/abs/2202.06240
  * Karras et al. (2019) Tero Karras, Samuli Laine, and Timo Aila. 2019.  A Style-Based Generator Architecture for Generative Adversarial Networks. In _IEEE Conference on Computer Vision and Pattern Recognition_. 4401–4410.
  * Kenfack et al. (2022) Patrik Joslin Kenfack, Kamil Sabbagh, Adín Ramírez Rivera, and Adil Khan. 2022.  RepFair-GAN: Mitigating Representation Bias in GANs Using Gradient Clipping.  https://arxiv.org/abs/2207.10653
  * Kim et al. (2019) Byungju Kim, Hyunwoo Kim, Kyungsu Kim, Sungjin Kim, and Junmo Kim. 2019.  Learning Not to Learn: Training Deep Neural Networks With Biased Data. In _2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE.
  * Kim et al. (2021) Hyemi Kim, Seungjae Shin, JoonHo Jang, Kyungwoo Song, Weonyoung Joo, Wanmo Kang, and Il-Chul Moon. 2021.  Counterfactual Fairness with Disentangled Causal Effect Variational Autoencoder. In _AAAI Conference on Artificial Intelligence_. 8128–8136.
  * Kim and Cho (2022) Jin-Young Kim and Sung-Bae Cho. 2022.  An information theoretic approach to reducing algorithmic bias for machine learning.  _Neurocomputing_ 500 (2022), 26 – 38.
  * King and Zeng (2006) Gary King and Langche Zeng. 2006.  The dangers of extreme counterfactuals.  (2006), 131–159.
  * Kirk et al. (2021) Hannah Rose Kirk et al. 2021\.  Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Models. In _Advances in Neural Information Processing Systems_. 2611–2624.
  * Klare et al. (2012a) Brendan F Klare, Mark J Burge, Joshua C Klontz, Richard W Vorder Bruegge, and Anil K Jain. 2012a.  Face recognition performance: Role of demographic information.  _IEEE Transactions on Information Forensics and Security_ 7, 6 (2012), 1789–1801.
  * Klare et al. (2012b) Brendan F. Klare, Mark J. Burge, Joshua C. Klontz, Richard W. Vorder Bruegge, and Anil K. Jain. 2012b.  Face Recognition Performance: Role of Demographic Information.  _IEEE Transactions on Information Forensics and Security_ 7, 6 (2012), 1789–1801.
  * Kleinberg et al. (2018) Jon Kleinberg, Jens Ludwig, Sendhil Mullainathan, and Ashesh Rambachan. 2018.  Algorithmic fairness. In _Aea papers and proceedings_ , Vol. 108. 22–27.
  * Kolling et al. (2022) Camila Kolling, Martin More, Nathan Gavenski, Eduardo Pooch, Otávio Parraga, and Rodrigo C Barros. 2022.  courfactual Debiasing for Visual Question Answering. In _IEEE/CVF Winter Conference on Applications of Computer Vision_. 3001–3010.
  * Kumar et al. (2020) Vaibhav Kumar, Tenzin Singhay Bhotia, and Tanmoy Chakraborty. 2020.  Nurse is Closer to Woman than Surgeon? Mitigating Gender-Biased Proximities in Word Embeddings.  _Trans. Assoc. Comput. Linguistics_ 8 (2020), 486–503.
  * Kumar et al. (2021) Vaibhav Kumar, Tenzin Singhay Bhotia, Vaibhav Kumar, and Tanmoy Chakraborty. 2021.  Identifying and Mitigating Gender Bias in Hyperbolic Word Embeddings.  https://arxiv.org/abs/2109.13767
  * Kurita et al. (2019) Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. 2019.  Measuring bias in contextualized word representations.  _arXiv preprint arXiv:1906.07337_ (2019).
  * Kusner et al. (2017) Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. 2017.  Counterfactual Fairness. In _Advances in Neural Information Processing Systems_.
  * Lambrecht and Tucker (2019) Anja Lambrecht and Catherine Tucker. 2019.  Algorithmic bias? An empirical study of apparent gender-based discrimination in the display of STEM career ads.  _Management science_ 65, 7 (2019), 2966–2981.
  * Lauscher et al. (2020) Anne Lauscher, Goran Glavas, Simone Paolo Ponzetto, and Ivan Vulic. 2020.  A General Framework for Implicit and Explicit Debiasing of Distributional Word Vector Spaces. In _AAAI Conference on Artificial Intelligence_. 8131–8138.
  * Lauscher et al. (2021) Anne Lauscher, Tobias Lueken, and Goran Glavaš. 2021\.  Sustainable Modular Debiasing of Language Models. In _Findings of the Association for Computational Linguistics_. ACL, 4782–4797.
  * Le Quy et al. (2022) Tai Le Quy, Arjun Roy, Vasileios Iosifidis, Wenbin Zhang, and Eirini Ntoutsi. 2022.  A survey on datasets for fairness-aware machine learning.  _Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery_ (2022), e1452.
  * Levi and Hassncer (2015) Gil Levi and Tal Hassncer. 2015.  Age and gender classification using convolutional neural networks. In _2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)_. 34–42.
  * Li et al. (2020) Peizhao Li, Han Zhao, and Hongfu Liu. 2020.  Deep Fair Clustering for Visual Learning. In _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020_. Computer Vision Foundation / IEEE, 9067–9076.
  * Li et al. (2021) Yong Li, Yufei Sun, Zhen Cui, Shiguang Shan, and Jian Yang. 2021.  Learning Fair Face Representation With Progressive Cross Transformer.  arXiv:2108.04983
  * Li and Vasconcelos (2019) Yi Li and Nuno Vasconcelos. 2019.  REPAIR: Removing Representation Bias by Dataset Resampling. In _2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE.
  * Li et al. (2022) Yuantong Li, Xiaokai Wei, Zijian Wang, Shen Wang, Parminder Bhatia, Xiaofei Ma, and Andrew Arnold. 2022.  Debiasing Neural Retrieval via In-batch Balancing Regularization.  _Workshop on Gender Bias in Natural Language Processing_ (2022), 58 – 66.
  * Liang et al. (2020) Paul Pu Liang, Irene Mengze Li, Emily Zheng, Yao Chong Lim, Ruslan Salakhutdinov, and Louis-Philippe Morency. 2020\.  Towards Debiasing Sentence Representations. In _Annual Meeting of the Association for Computational Linguistics_. ACL, 5502–5515.
  * Liang et al. (2021) Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2021.  Towards Understanding and Mitigating Social Biases in Language Models.  https://arxiv.org/abs/2106.13219
  * Lin et al. (2019) Xi Lin, Hui-Ling Zhen, Zhenhua Li, Qing-Fu Zhang, and Sam Kwong. 2019.  Pareto multi-task learning.  _Advances in neural information processing systems_ 32 (2019).
  * Liu et al. (2021) Ruibo Liu, Chenyan Jia, Jason Wei, Guangxuan Xu, Lili Wang, and Soroush Vosoughi. 2021\.  Mitigating Political Bias in Language Models Through Reinforced Calibration.  _35th AAAI Conference on Artificial Intelligence, AAAI 2021_ 17A (2021), 14857 – 14866.
  * Liu et al. (2015) Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. 2015\.  Deep Learning Face Attributes in the Wild. In _2015 IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, December 7-13, 2015_. IEEE Computer Society, Santiago, Chile, 3730–3738.
  * Locatello et al. (2019b) Francesco Locatello et al. 2019b.  Challenging common assumptions in the unsupervised learning of disentangled representations. In _International Conference on Machine Learning_. 4114–4124.
  * Locatello et al. (2019a) Francesco Locatello, Gabriele Abbati, Thomas Rainforth, Stefan Bauer, Bernhard Schölkopf, and Olivier Bachem. 2019a.  On the fairness of disentangled representations.  _Advances in Neural Information Processing Systems_ 32 (2019).
  * Lu et al. (2020) Kaiji Lu, Piotr Mardziel, Fangjing Wu, Preetam Amancharla, and Anupam Datta. 2020.  Gender bias in neural natural language processing.  In _Logic, Language, and Security_. Springer, 189–202.
  * Ma et al. (2021) Martin Q Ma, Yao-Hung Hubert Tsai, Paul Pu Liang, Han Zhao, Kun Zhang, Ruslan Salakhutdinov, and Louis-Philippe Morency. 2021.  Conditional Contrastive Learning for Improving Fairness in Self-Supervised Learning.  _arXiv e-prints_ (2021), arXiv–2106.
  * Madaio et al. (2020) Michael A Madaio, Luke Stark, Jennifer Wortman Vaughan, and Hanna Wallach. 2020.  Co-designing checklists to understand organizational challenges and opportunities around fairness in AI. In _Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems_. 1–14.
  * Maheshwari et al. (2022) Gaurav Maheshwari, Pascal Denis, Mikaela Keller, and Aurélien Bellet. 2022.  Fair NLP Models with Differentially Private Text Encoders.  https://arxiv.org/abs/2205.06135
  * Martinez et al. (2020) Natalia Martinez, Martin Bertran, and Guillermo Sapiro. 2020\.  Minimax pareto fairness: A multi objective perspective. In _International Conference on Machine Learning_. PMLR, 6755–6764.
  * May et al. (2019) Chandler May, Alex Wang, Shikha Bordia, Samuel R. Bowman, and Rachel Rudinger. 2019.  On Measuring Social Biases in Sentence Encoders. In _Conference of the North American Chapter of the Association for Computational Linguistics_. 622–628.
  * Mazumder et al. (2022) Pratik Mazumder, Pravendra Singh, and Vinay P. Namboodiri. 2022\.  Fair Visual Recognition in Limited Data Regime using Self-Supervision and Self-Distillation.  _2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)_ (2022), 3889–3897.
  * McGovern (2021) Hope McGovern. 2021\.  A Source-Criticism Debiasing Method for GloVe Embeddings.  _CoRR_ abs/2106.13382 (2021).  arXiv:2106.13382 https://arxiv.org/abs/2106.13382
  * Meade et al. (2021) Nicholas Meade, Elinor Poole-Dayan, and Siva Reddy. 2021\.  An empirical survey of the effectiveness of debiasing techniques for pre-trained language models.  _arXiv preprint arXiv:2110.08527_ (2021).
  * Mehrabi et al. (2021) Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021\.  A survey on bias and fairness in machine learning.  _ACM Computing Surveys (CSUR)_ 54, 6 (2021), 1–35.
  * Mikolov et al. (2013) Tomás Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013\.  Efficient Estimation of Word Representations in Vector Space. In _International Conference on Learning Representations_.
  * Mishkin et al. (2022) Pamela Mishkin, Lama Ahmad, Miles Brundage, Gretchen Krueger, and Girish Sastry. 2022.  DALL·E 2 Preview - Risks and Limitations.  (2022).
  * Mitchell et al. (2019) Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019.  Model cards for model reporting. In _Conference on Fairness, Accountability, and Transparency_. 220–229.
  * Mitchell (1980) Tom M Mitchell. 1980\.  _The need for biases in learning generalizations_.  Department of Computer Science, Laboratory for Computer Science Research.
  * Mittelstadt (2019) Brent Mittelstadt. 2019\.  Principles alone cannot guarantee ethical AI.  _Nature Machine Intelligence_ 1, 11 (2019), 501–507.
  * Nadeem et al. (2021) Moin Nadeem, Anna Bethke, and Siva Reddy. 2021.  StereoSet: Measuring stereotypical bias in pretrained language models. In _Annual Meeting of the Association for Computational Linguistics_. ACL, 5356–5371.
  * Nanayakkara et al. (2021) Priyanka Nanayakkara, Jessica Hullman, and Nicholas Diakopoulos. 2021.  Unpacking the expressed consequences of AI research in broader impact statements. In _Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society_. 795–806.
  * Nangia et al. (2020) Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R. Bowman. 2020.  CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models. In _Conference on Empirical Methods in Natural Language Processing_. ACL, 1953–1967.
  * NeurIPS (2020) NeurIPS. 2020.  Call for Papers.  https://neurips.cc/Conferences/2020/CallForPapers
  * NeurIPS (2021a) NeurIPS. 2021a.  Introducing the NeurIPS 2021 Paper Checklist.  https://neuripsconf.medium.com/introducing-the-neurips-2021-paper-checklist-3220d6df500b
  * NeurIPS (2021b) NeurIPS. 2021b.  NeurIPS 2021 Paper Checklist Guidelines.  https://neurips.cc/Conferences/2021/PaperInformation/PaperChecklist
  * Ngxande et al. (2020) Mkhuseli Ngxande, Jules-Raymond Tapamo, and Michael Burke. 2020.  Bias Remediation in Driver Drowsiness Detection Systems Using Generative Adversarial Networks.  _IEEE Access_ 8 (2020), 55592–55601.
  * Nia et al. (2022) Vahid Partovi Nia, Alireza Ghaffari, Mahdi Zolnouri, and Yvon Savaria. 2022.  Rethinking Pareto Frontier for Performance Evaluation of Deep Neural Networks.  _arXiv preprint arXiv:2202.09275_ (2022).
  * Niu et al. (2021) Yulei Niu, Kaihua Tang, Hanwang Zhang, Zhiwu Lu, Xian-Sheng Hua, and Ji-Rong Wen. 2021\.  Counterfactual vqa: A cause-effect look at language bias. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 12700–12710.
  * Odbal et al. (2022) Odbal, Guanhong Zhang, and Sophia Ananiadou. 2022.  Examining and mitigating gender bias in text emotion detection task.  _Neurocomputing_ 493 (2022), 422 – 434.
  * Olteanu et al. (2019) Alexandra Olteanu, Carlos Castillo, Fernando Diaz, and Emre Kıcıman. 2019.  Social data: Biases, methodological pitfalls, and ethical boundaries.  _Frontiers in Big Data_ 2 (2019), 13.
  * Park et al. (2020) Sungho Park, Sunhee Hwang, Jongkwang Hong, and Hyeran Byun. 2020.  Fair-VQA: Fairness-Aware Visual Question Answering Through Sensitive Attribute Prediction.  _IEEE Access_ 8 (2020), 215091–215099.
  * Park et al. (2021) Sungho Park, Sunhee Hwang, Dohyung Kim, and Hyeran Byun. 2021\.  Learning Disentangled Representation for Fair Facial Attribute Classification via Fairness-aware Information Alignment.  _AAAI Conference on Artificial Intelligence_ 35, 3 (2021), 2403–2411.
  * Patil and Purcell (2022) Pranita Patil and Kevin Purcell. 2022.  Decorrelation-Based Deep Learning for Bias Mitigation.  _Future Internet_ 14, 4 (2022), 110.
  * Pena et al. (2020) Alejandro Pena, Ignacio Serna, Aythami Morales, and Julian Fierrez. 2020.  Bias in multimodal AI: Testbed for fair automatic recruitment. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_. 28–29.
  * Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014.  Glove: Global vectors for word representation. In _Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)_. 1532–1543.
  * Perianes-Rodriguez et al. (2016) Antonio Perianes-Rodriguez, Ludo Waltman, and Nees Jan van Eck. 2016\.  Constructing bibliometric networks: A comparison between full and fractional counting.  _Journal of Informetrics_ 10, 4 (2016), 1178–1195.
  * Pessach and Shmueli (2020) Dana Pessach and Erez Shmueli. 2020.  Algorithmic fairness.  _arXiv preprint arXiv:2001.09784_ (2020).
  * Prunkl et al. (2021) Carina EA Prunkl, Carolyn Ashurst, Markus Anderljung, Helena Webb, Jan Leike, and Allan Dafoe. 2021\.  Institutionalizing ethics in AI through broader impact requirements.  _Nature Machine Intelligence_ 3, 2 (2021), 104–110.
  * Qian et al. (2022) Rebecca Qian, Candace Ross, Jude Fernandes, Eric Smith, Douwe Kiela, and Adina Williams. 2022\.  Perturbation Augmentation for Fairer NLP.  https://arxiv.org/abs/2205.12586
  * Qian et al. (2019) Yusu Qian, Urwa Muaz, Ben Zhang, and Jae Won Hyun. 2019\.  Reducing Gender Bias in Word-Level Language Models with a Gender-Equalizing Loss Function.  _Annual Meeting of the Association for Computational Linguistics_ (2019), 223 – 228\.
  * Radford et al. (2021) Alec Radford et al. 2021\.  Learning Transferable Visual Models From Natural Language Supervision. In _International Conference on Machine Learning_ , Vol. 139. 8748–8763.
  * Rajabi et al. (2022) Amirarsalan Rajabi, Mehdi Yazdani-Jahromi, Ozlem Ozmen Garibay, and Gita Sukthankar. 2022.  Through a fair looking-glass: mitigating bias in image datasets.  https://arxiv.org/abs/2209.08648
  * Raji et al. (2020) Inioluwa Deborah Raji, Timnit Gebru, Margaret Mitchell, Joy Buolamwini, Joonseok Lee, and Emily Denton. 2020.  Saving face: Investigating the ethical concerns of facial recognition auditing. In _Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society_. 145–151.
  * Ramaswamy et al. (2021) Vikram V. Ramaswamy, Sunnie S. Y. Kim, and Olga Russakovsky. 2021.  Fair Attribute Classification through Latent Space De-biasing. In _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE.
  * Ramesh et al. (2022) Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022.  Hierarchical Text-Conditional Image Generation with CLIP Latents.  _CoRR_ abs/2204.06125 (2022).  arXiv:2204.06125
  * Rekabsaz et al. (2021) Navid Rekabsaz, Simone Kopeinik, and Markus Schedl. 2021\.  Societal biases in retrieved contents: Measurement framework and adversarial mitigation of bert rankers. In _International ACM SIGIR Conference on Research and Development in Information Retrieval_. 306–316.
  * Reynolds and McDonell (2021) Laria Reynolds and Kyle McDonell. 2021.  Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm. In _CHI Conference on Human Factors in Computing Systems_. 314:1–314:7.
  * Richardson et al. (2021) Brianna Richardson, Jean Garcia-Gathright, Samuel F Way, Jennifer Thom, and Henriette Cramer. 2021\.  Towards Fairness in Practice: A Practitioner-Oriented Rubric for Evaluating Fair ML Toolkits. In _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_. 1–13.
  * Richardson and Gilbert (2021) Brianna Richardson and Juan E Gilbert. 2021.  A Framework for Fairness: A Systematic Review of Existing Fair AI Solutions.  _arXiv preprint arXiv:2112.05700_ (2021).
  * Ruchte and Grabocka (2021) Michael Ruchte and Josif Grabocka. 2021.  Scalable Pareto Front Approximation for Deep Multi-Objective Learning. In _2021 IEEE International Conference on Data Mining (ICDM)_. IEEE, 1306–1311.
  * Russakovsky et al. (2015) Olga Russakovsky et al. 2015\.  ImageNet Large Scale Visual Recognition Challenge.  _International Journal of Computer Vision_ 115, 3 (2015), 211–252.
  * Saleiro et al. (2018) Pedro Saleiro, Benedict Kuester, Loren Hinkson, Jesse London, Abby Stevens, Ari Anisfeld, Kit T Rodolfa, and Rayid Ghani. 2018\.  Aequitas: A bias and fairness audit toolkit.  _arXiv preprint arXiv:1811.05577_ (2018).
  * Salvador et al. (2022) Tiago Salvador, Stephanie Cairns, Vikram Voleti, Noah Marshall, and Adam M. Oberman. 2022.  FairCal: Fairness Calibration for Face Verification. In _International Conference on Learning Representations_.
  * Sanh et al. (2019) Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019.  DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter.  _CoRR_ abs/1910.01108 (2019).  arXiv:1910.01108 http://arxiv.org/abs/1910.01108
  * Schick et al. (2021) Timo Schick, Sahana Udupa, and Hinrich Schütze. 2021\.  Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP.  _Transactions of the Association for Computational Linguistics_ 9 (2021), 1408–1424.
  * Selvaraju et al. (2016) Ramprasaath R. Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, and Dhruv Batra. 2016.  Grad-CAM: Why did you say that? Visual Explanations from Deep Networks via Gradient-based Localization.  arXiv:1610.02391
  * Serna et al. (2022) Ignacio Serna, Aythami Morales, Julian Fierrez, and Nick Obradovich. 2022.  Sensitive loss: Improving accuracy and fairness of face representations with discrimination-aware deep learning.  _Artificial Intelligence_ 305 (2022).
  * Shah et al. (2020) Deven Shah, H. Andrew Schwartz, and Dirk Hovy. 2020\.  Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview. In _Annual Meeting of the Association for Computational Linguistics_. 5248–5264.
  * Shah et al. (2021) Kulin Shah, Pooja Gupta, Amit Deshpande, and Chiranjib Bhattacharyya. 2021.  Rawlsian fair adaptation of deep learning classifiers. In _Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society_. 936–945.
  * Sharma et al. (2022) Shanya Sharma, Manan Dey, and Koustuv Sinha. 2022.  How sensitive are translation systems to extra contexts? Mitigating gender bias in Neural Machine Translation models through relevant contexts.  https://arxiv.org/abs/2205.10762
  * Shen et al. (2021) Aili Shen, Xudong Han, Trevor Cohn, Timothy Baldwin, and Lea Frermann. 2021.  Contrastive Learning for Fair Representations.  https://arxiv.org/abs/2109.10645
  * Shen et al. (2022) Tianshu Shen, Jiaru Li, Mohamed Reda Bouadjenek, Zheda Mai, and Scott Sanner. 2022.  Unintended Bias in Language Model-drivenConversational Recommendation.  _arXiv preprint arXiv:2201.06224_ (2022).
  * Sheng et al. (2020) Emily Sheng, Kai-Wei Chang, Prem Natarajan, and Nanyun Peng. 2020.  Towards Controllable Biases in Language Generation. In _Findings of the Association for Computational Linguistics: EMNLP 2020_. Association for Computational Linguistics, Online, 3239–3254.
  * Srinivasan et al. (2019) Ramya Srinivasan, Ajay Chander, and Pouya Pezeshkpour. 2019\.  Generating User-friendly Explanations for Loan Denials using GANs.
  * Stafanovičs et al. (2020) Artūrs Stafanovičs, Toms Bergmanis, and Mārcis Pinnis. 2020\.  Mitigating Gender Bias in Machine Translation with Target Gender Annotations. In _Conference on Machine Translation_. ACL, 629–638.
  * Stanton et al. (2021) Samuel Don Stanton, Pavel Izmailov, Polina Kirichenko, Alexander A Alemi, and Andrew Gordon Wilson. 2021\.  Does Knowledge Distillation Really Work?. In _Advances in Neural Information Processing Systems_.
  * Stark and Hoffmann (2019) Luke Stark and Anna Lauren Hoffmann. 2019.  Data is the new what? Popular metaphors & professional ethics in emerging data culture.  (2019).
  * Subramanian et al. (2021) Shivashankar Subramanian, Xudong Han, Timothy Baldwin, Trevor Cohn, and Lea Frermann. 2021\.  Evaluating Debiasing Techniques for Intersectional Biases. In _Conference on Empirical Methods in Natural Language Processing_. 2492–2498.
  * Suresh and Guttag (2019) Harini Suresh and John V. Guttag. 2019.  A Framework for Understanding Unintended Consequences of Machine Learning.  _CoRR_ abs/1901.10002 (2019).  arXiv:1901.10002 http://arxiv.org/abs/1901.10002
  * Tang et al. (2021) Ruixiang Tang, Mengnan Du, Yuening Li, Zirui Liu, Na Zou, and Xia Hu. 2021\.  Mitigating gender bias in captioning systems. In _Proceedings of the Web Conference 2021_. 633–645.
  * Tanjim et al. (2022) Md Mehrab Tanjim et al. 2022\.  Generating and Controlling Diversity in Image Search.  _IEEE/CVF Winter Conference on Applications of Computer Vision_ (2022), 3908 – 3916.
  * Tartaglione et al. (2021) Enzo Tartaglione, Carlo Alberto Barbano, and Marco Grangetto. 2021.  EnD: Entangling and Disentangling deep representations for bias correction. In _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE.
  * TensorFlow (2020) TensorFlow. 2020\.  Fairness Indicators.  https://www.tensorflow.org/responsible_ai/fairness_indicators/guide
  * Tian et al. (2022) Huan Tian, Tianqing Zhu, Wei Liu, and Wanlei Zhou. 2022\.  Image fairness in deep learning: problems, models, and challenges.  _Neural Computing and Applications_ (March 2022).
  * Vandenhende et al. (2021) Simon Vandenhende, Stamatios Georgoulis, Wouter Van Gansbeke, Marc Proesmans, Dengxin Dai, and Luc Van Gool. 2021.  Multi-task learning for dense prediction tasks: A survey.  _IEEE transactions on pattern analysis and machine intelligence_ (2021).
  * Vargas and Cotterell (2020) Francisco Vargas and Ryan Cotterell. 2020.  Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation. In _Conference on Empirical Methods in Natural Language Processing_. 2902–2913.
  * Vasudevan and Kenthapadi (2020) Sriram Vasudevan and Krishnaram Kenthapadi. 2020.  Lift: A scalable framework for measuring fairness in ml applications. In _Proceedings of the 29th ACM International Conference on Information & Knowledge Management_. 2773–2780.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017\.  Attention is All you Need. In _Advances in Neural Information Processing Systems, NeurIPS_. 5998–6008.
  * Verma and Rubin (2018) Sahil Verma and Julia Rubin. 2018.  Fairness definitions explained. In _2018 ieee/acm international workshop on software fairness (fairware)_. IEEE, 1–7.
  * Vig et al. (2020) Jesse Vig et al. 2020\.  Investigating gender bias in language models using causal mediation analysis.  _Advances in Neural Information Processing Systems_ (2020), 12388–12401.
  * Wallace et al. (2019) Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. 2019.  Universal Adversarial Triggers for Attacking and Analyzing NLP. In _Conference on Empirical Methods in Natural Language Processing_. 2153–2162.
  * Wang et al. (2021) Jialu Wang, Yang Liu, and Xin Eric Wang. 2021.  Are gender-neutral queries really gender-neutral? mitigating gender bias in image search.  _arXiv preprint arXiv:2109.05433_ (2021).
  * Wang and Deng (2020) Mei Wang and Weihong Deng. 2020.  Mitigating Bias in Face Recognition Using Skewness-Aware Reinforcement Learning. In _IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 9319–9328.
  * Wang et al. (2018) Mei Wang, Weihong Deng, Jiani Hu, Xunqiang Tao, and Yaohai Huang. 2018.  Racial Faces in-the-Wild: Reducing Racial Bias by Information Maximization Adaptation Network.  (2018).
  * Wang et al. (2019) Tianlu Wang, Jieyu Zhao, Mark Yatskar, Kai-Wei Chang, and Vicente Ordonez. 2019.  Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations. In _IEEE/CVF International Conference on Computer Vision_.
  * Webster et al. (2020) Kellie Webster, Xuezhi Wang, Ian Tenney, Alex Beutel, Emily Pitler, Ellie Pavlick, Jilin Chen, Ed Chi, and Slav Petrov. 2020.  Measuring and reducing gendered correlations in pre-trained models.  _arXiv preprint arXiv:2010.06032_ (2020).
  * Wexler et al. (2019) James Wexler, Mahima Pushkarna, Tolga Bolukbasi, Martin Wattenberg, Fernanda Viégas, and Jimbo Wilson. 2019.  The what-if tool: Interactive probing of machine learning models.  _IEEE transactions on visualization and computer graphics_ 26, 1 (2019), 56–65.
  * Wu et al. (2021b) Chuhan Wu, Fangzhao Wu, Xiting Wang, Yongfeng Huang, and Xing Xie. 2021b.  Fairness-aware news recommendation with decomposed adversarial learning. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 35. 4462–4469.
  * Wu et al. (2021a) Fangsheng Wu et al. 2021a.  Understanding Social Biases Behind Location Names in Contextual Word Embedding Models.  _IEEE Transactions on Computational Social Systems_ 9, 2 (2021), 458–468.
  * Wu et al. (2022) Yawen Wu, Dewen Zeng, Xiaowei Xu, Yiyu Shi, and Jingtong Hu. 2022.  FairPrune: Achieving Fairness Through Pruning for Dermatological Disease Diagnosis.  https://arxiv.org/abs/2203.02110
  * Xu et al. (2019) Depeng Xu, Yongkai Wu, Shuhan Yuan, Lu Zhang, and Xintao Wu. 2019.  Achieving Causal Fairness through Generative Adversarial Networks. In _International Joint Conference on Artificial Intelligence_. 1452–1458.
  * Xu et al. (2020) Tian Xu, Jennifer White, Sinan Kalkan, and Hatice Gunes. 2020\.  Investigating Bias and Fairness in Facial Expression Recognition.  In _Computer Vision – ECCV 2020 Workshops_. Springer International Publishing, 506–523.
  * Yan et al. (2020) Shen Yan, Di Huang, and Mohammad Soleymani. 2020.  Mitigating biases in multimodal personality assessment. In _Proceedings of the 2020 International Conference on Multimodal Interaction_. 361–369.
  * Yang et al. (2021) Xu Yang, Hanwang Zhang, Guojun Qi, and Jianfei Cai. 2021\.  Causal attention for vision-language tasks. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 9847–9857.
  * Yang and Feng (2020) Zekun Yang and Juan Feng. 2020.  A Causal Inference Method for Reducing Gender Bias in Word Embedding Relations. In _AAAI Conference on Artificial Intelligence_. 9434–9441.
  * Yucer et al. (2020) Seyma Yucer, Samet Akçay, Noura Al Moubayed, and Toby P. Breckon. 2020.  Exploring Racial Bias within Face Recognition via per-subject Adversarially-Enabled Data Augmentation. In _IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 83–92.
  * Zhang et al. (2018) B. Zhang, Blake Lemoine, and Margaret Mitchell. 2018.  Mitigating Unwanted Biases with Adversarial Learning.  _Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society_ (2018).
  * Zhang et al. (2022) Susan Zhang et al. 2022\.  OPT: Open Pre-trained Transformer Language Models.  _CoRR_ abs/2205.01068 (2022).
  * Zhao et al. (2017) Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017.  Men also like shopping: Reducing gender bias amplification using corpus-level constraints.  _arXiv preprint arXiv:1707.09457_ (2017).