# Mitigating Bias Using Model-Agnostic Data Attribution

Sander De Coninck, Sam Leroux, Pieter Simoens
IDLab, Department of Information Technology at Ghent University - imec
Technologiepark 126, B-9052 Ghent, Belgium
{sander.deconinck, sam.leroux<EMAIL_ADDRESS>
###### Abstract

Mitigating bias in machine learning models is a critical endeavor for ensuring
fairness and equity. In this paper, we propose a novel approach to address
bias by leveraging pixel image attributions to identify and regularize regions
of images containing significant information about bias attributes. Our method
utilizes a model-agnostic approach to extract pixel attributions by employing
a convolutional neural network (CNN) classifier trained on small image
patches. By training the classifier to predict a property of the entire image
using only a single patch, we achieve region-based attributions that provide
insights into the distribution of important information across the image. We
propose utilizing these attributions to introduce targeted noise into datasets
with confounding attributes that bias the data, thereby constraining neural
networks from learning these biases and emphasizing the primary attributes.
Our approach demonstrates its efficacy in enabling the training of unbiased
classifiers on heavily biased datasets.

##  1 Introduction

Figure 1: Our strategy to address biased learning. We utilize a region-based
classifier to classify individual image patches based on attributes that
introduce bias to the data. By leveraging the confidence of the trained model,
we identify the regions within an image where attribute-related information is
most concentrated. Subsequently, we introduce targeted noise to these areas in
the training data to prevent models from overfitting to the confounding
attributes.

In the domain of computer vision, various methodologies are employed to
mitigate bias and uphold fairness in machine learning models operating on
sensitive attributes. Data bias can manifest in various ways, such as when the
data representation disproportionately favors a specific group of subjects
[22]. Alternatively, bias may arise from substantial differences between the
training and production environments [23]. Moreover, biases within the data
can be either explicit and acknowledged beforehand or concealed. We
specifically focus on situations where bias is pre-existing and originates
from disparities between the training and evaluation datasets due to a
confounding variable heavily influencing the former. For instance, in the
CelebA [15] dataset, a significant majority of women are depicted with blond
hair, whereas only a minority of men exhibit this trait. Consequently, a model
trained on such data may exhibit unequal performance across men and women in a
production setting where this distribution does not hold true as it has
learned to associate having blond hair with being a woman.

One prevalent technique to mitigate the learning of biases is dataset re-
sampling, where instances from minority or majority classes are either over-
sampled or under-sampled to achieve a more equitable distribution. However, in
scenarios such as facial data, where numerous attributes are present,
achieving a perfectly balanced dataset may be unattainable as this would
require an equal number of samples for each subcategory. In the case of
CelebA, which contains $40$ binary attributes, this would mean you require an
equal amount of samples for each of the $2^{40}$ possible combinations.
Another approach is to utilize dataset augmentation [23], which involves
altering the training data to rectify imbalances in underrepresented classes
or counter biases inherent in the training dataset, for example by generating
synthetic data [25, 32]. Our solution falls under this category as well. We
propose a solution where we identify critical pixel areas for classifying
attributes that introduce a bias in the data (e.g., blond hair in face data).
By strategically introducing targeted noise to these regions in the training
data, we aim to prevent the model from overfitting to these confounding
attributes. A conceptualisation of our technique can be seen in Figure 1.

Attributing pixels that are important for classification is typically referred
to as saliency mapping. Saliency mapping techniques were originally developed
to elucidate machine learning models rather than datasets. Consequently, they
attempt to identify pixel regions crucial for a specific model’s
classification. However, for mitigating biases being learned, it’s more
pertinent to determine which regions potentially contribute to classification.
In other words, it’s essential to discern which image regions harbour valuable
information for a particular class (e.g., identifying facial features aiding
in ethnicity classification). This will not always align with specific model
attribution maps, as not all networks learn in the same way, and some may lay
more importance on a certain pixel region than others.

To tackle this issue, we introduce data attribution through classification. We
employ a small-scale classifier that predicts a sensitive label for the entire
image from a single image patch. By identifying regions where the classifier
confidently predicts the label, assuming the model is well calibrated, we
establish a measure of information content regarding the sensitive label
within that patch. As a result, this methodology allows us to attribute
specific image regions to particular classes.

The resulting attributions could be used for a number of purposes, but we see
the most promise in fields where it is undesirable that certain attributes are
learned. The most notable of these is learning using a biased dataset. We
propose to use our attributions as a way to regularize the training process,
if some biases in the data are known a priori (e.g. due to a high correlation
between labels), then we can compel the model learning to classify desired
attributes to divert focus away from those regions important for classifying
the confounder.

We demonstrate the effectiveness of our attribution technique on two prominent
face datasets, namely FairFace [12] and CelebA [15]. These datasets are
selected due to their comprehensive annotations and consistent alignment of
posture, facilitating the validation of our attributions’ utility through
averaged attributions across multiple images. Subsequently, we explore the
application of our attributions in training unbiased perceived gender
classifiers on biased data, leveraging the CelebA dataset. By creating biased
subsets where specific attributes are prominent for women but absent for men,
we evaluate subgroup accuracy on a balanced dataset. Our experiments reveal
that incorporating noise based on our attributions to the training data proves
beneficial in mitigating the adverse effects of dataset bias on classifier
performance.

We make the following contributions in this paper:

  * •

We developed a general model-independent data attribution technique that can
foster data understanding

  * •

We showcase the use of this attribution technique for data augmentation to
train neural networks under known biases and experiment with different noise
addition types and settings

The remainder of this paper is organized as follows. We position related works
in Section 2. Following this, we introduce our region classifier and show how
we obtain attributions in Section 3. Lastly, we experiment with using the
attributions for developing robust classifiers in Section 4. We conclude our
work in Section 5.

##  2 Related works

###  2.1 Dataset bias and fairness

Correlation is a widely studied problem in fair research, mainly concerning
spurious correlations in the dataset through societal biases or dataset
construction. For example, for the CelebA dataset, it is known that the
attractive attribute is biased towards women and that a majority of the women
have blonde hair. Rajabi et al. [24] demonstrate that mitigating bias between
gender and attractiveness for the CelebA dataset influences classification of
several other attributes related to attractiveness as well. Similarly, Denton
et al. [4] found that manipulating non-hair attributes such as heavy makeup
resulted in a change in hair style/and or color.

Addressing these spurious correlations and biases represents an active area of
investigation within the research community. Some studies concentrate on
improving datasets to ensure a more equitable distribution of data samples,
either through resampling techniques or augmenting the dataset with new data
generated, for example, using generative models [25, 32]. Others employ
techniques during model training to guide the learning process, facilitating
the acquisition or elimination of specific biases [33]. For instance, Kim et
al. utilize a regularization loss based on mutual information to prevent the
model from learning biased attributes [14]. Additionally, certain approaches
leverage causal learning methodologies to achieve unbiased recognition [31].

###  2.2 Attribution beyond explainability

Pixel attribution techniques are utilized to identify the areas or pixels
within an image that significantly contribute to a model’s prediction. These
techniques typically fall into two categories: occlusion or perturbation-
based, and gradient-based methods. The former, being model-agnostic, assesses
the impact on prediction when certain regions are omitted, while the latter
computes gradients concerning the input image [19]. Noteworthy gradient-based
techniques include Grad-CAM [27], Integrated Gradients [30], and XRAI [11].
For occlusion-based approaches, LIME [26] and SHAP [17] are widely recognized.

These methods primarily serve to enhance the interpretability of deep neural
networks. However, they also see use beyond, in improving the generalization
of deep learning models [2], such as those used in facial expression
recognition [13, 18] and person detection tasks [1], including scenarios
involving thermal imagery [6]. Particularly relevant to our work, Huang et al.
[9] employ saliency techniques to address biases in facial recognition models.
Their approach employs gradient attention maps, ensuring consistent attention
patterns across diverse racial backgrounds. However, our work differs in that
we utilize saliency to mitigate confounding variables being learned, whereas
they focus on ensuring uniform learning across different subgroups regardless
of what is learned.

##  3 Region classifier

In our efforts to alleviate learned biases, our focus is directed towards
identifying the regions containing vital information for classification. While
saliency techniques are commonly used for this purpose, they often focus
solely on pinpointing important pixels for a specific model, potentially
overlooking broader regions of interest. To address this, we deploy a region
classifier capable of analyzing image patches and attributing characteristics
of the entire image, such as determining a person’s age. By assessing the
confidence levels of a well-trained classifier, we gain valuable insights into
the specific areas where significant information is concentrated.

###  3.1 Training setup

Our region classifier employs a ResNet18 model architecture [8], which
operates on patches of size $k\times k$. Additionally, the network takes in a
region indicator, which indicates the region from which the patch was taken in
the original image. This region indicator is translated into an embedding akin
to the vision transformer approach [5]. This embedding is then concatenated to
the input image before execution. The embeddings and the classification
network are trained simultaneously. During the training process, a single
patch per image is randomly selected for training. Patches are randomly
selected from $p$ possible positions, where patch $i$ corresponds to having
$((i\mod\sqrt{p})*s,\lfloor i/\sqrt{p}\rfloor*s)$ as the top-left coordinates
of the patch. Here, the stride $s$ is calculated as $I-k/\sqrt{p}$, with $I$
being the image size.

Our experiments utilize the FairFace [12] and CelebA [15] dataset, providing
centred and aligned face images, which we resized to $128\times 128$ pixels.
We employ the AdamW optimizer [16] with an initial learning rate of $10^{-3}$,
reduced by a factor of $10$ every $40$ epochs during training, for a total of
$90$ epochs. Patch dimensions $k$ are set to $32$, and the number of possible
patch positions $p$ is $2041$. A batch size of $256$ is utilized, and the
cross-entropy loss function with label smoothing of $0.1$ is employed.

###  3.2 Confidence estimation

To effectively utilize our region classifier for determining whether a patch
contains relevant information regarding the subject, it is imperative that our
classifier can provide an indication of its confidence level. There are three
prevalent methods for estimating confidence when working with classification
networks. Each relies on softmax values derived from the output logits $l$,
which are computed as follows:

$S(l_{i})=\frac{e^{l_{i}}}{\sum_{j}e^{l_{j}}}$ (1)

The three confidence indicators used are the top softmax score, the margin,
and the negative entropy score, which are calculated as follows:

  * •

Top softmax score: this indicator is determined by selecting the highest
softmax value among all classes.

$\max_{i}s_{i}$ (2)

  * •

Margin: we determine it as the difference between the highest and second-
highest softmax values.

$\max_{i}s_{i}-\max_{j\neq\text{argmax}(s)}s_{j}$ (3)

  * •

Negative entropy score: it is computed as the negative sum of the softmax
probabilities multiplied by their natural logarithms.

$-\sum_{i}s_{i}\log s_{i}$ (4)

In our experiments, we primarily rely on the negative entropy score as it
considers the entire output distribution. While less straightforward to
interpret than the top softmax score or the margin, we strive to enhance
interpretability by normalizing outputs to a $[0,1]$ range wherever feasible.

###  3.3 Calibration

When using our classifier for attribution, it’s important that its confidence
outputs are grounded in how accurately it can predict and, thus, how much
information there is. A calibrated model is a model whose estimated confidence
is close to its accuracy. The most common metrics of calibration are the
Expected Calibration Error (ECE) [20], and reliability diagrams [3, 21].

Expected Calibration Error (ECE) quantifies the discrepancy between predicted
and actual confidence levels. It is calculated as follows:

$ECE=\sum^{N}_{i}b_{i}||(p_{i}-c_{i})||$ (5)

Here, $N$ represents the number of bins used for calibration, $b_{i}$ is the
proportion of samples falling into bin $i$, $p_{i}$ denotes the average
predicted confidence, and $c_{i}$ represents the average true accuracy. ECE
measures the average difference between predicted confidence and actual
accuracy across different confidence levels. It provides a holistic assessment
of the calibration performance of a classifier.

Reliability diagrams, also known as calibration diagrams, graphically
illustrate the alignment between predicted confidence levels and actual
accuracy. They plot predicted confidence values against observed accuracy
within equally spaced bins. Ideally, a well-calibrated classifier shows a
diagonal line, indicating accurate confidence estimates. Deviations from this
line highlight areas of overconfidence or underconfidence. Reliability
diagrams offer a visual tool for evaluating a classifier’s calibration
performance.

Calibration diagrams, along with the corresponding Expected Calibration Error
(ECE) scores, for networks trained to classify the three FairFace attributes
are depicted in Figure 2. These diagrams were generated using all possible
patches from the validation set using $100$ bins. We observe that our networks
are well-calibrated without having made any specific modifications. This is
likely attributed to the high variability in data quality. Many regions may
lack sufficient information for accurate classification but are still
included, thereby mitigating overfitting and ensuring consistent calibration.
Notably, for the age attribute, calibration appears to decrease significantly
at high confidences, although such instances are infrequent, as evidenced by
the lower plots.

(a) Age

(b) Gender

(c) Race

Figure 2: Reliability diagrams illustrating the performance of region
classifiers trained on FairFace attributes. Our models exhibit strong
calibration without any adjustments, as evidenced by the close alignment
between average confidence and accuracy, alongside low Expected Calibration
Error (ECE).

###  3.4 Region-based attributions

For the final step, we can use our patch-based classifiers, which we have
shown to be well-calibrated, to calculate attribution maps. To attribute an
image with respect to a specific attribute, we employ our region classifier to
assess confidence across numerous regions. This is achieved by iteratively
collecting regions in a sliding window manner, as can also be seen in Figure
1. Subsequently, the obtained regions are batched and classified using the
region classifier. Finally, a confidence score, such as negative entropy, is
computed for each patch. The resulting attribution can be visualized as a
$\sqrt{p}\times\sqrt{p}$ map, with each pixel $(x,y)$ representing a region
whose top left coordinate in the original image is $(s*x,s*y)$. Illustrative
examples of our region attributions for the FairFace dataset are showcased in
Figure 3. Note that we solely rely on confidence scores to calculate
attributions, allowing a trained model to be applied to novel data without the
need for ground-truth labels. One limitation of this attribution technique is
its computational cost, as it requires classifying $p$ image patches per
image.

Figure 3: Examples of region-based attributions.

By averaging over all samples, mean confidence maps can be generated. These
maps offer valuable insights into the spatial distribution of important
attribute information within an image and can be used to relate these
attributes to one another. The calculated maps for the FairFace attributes are
presented in Figure 4, revealing distinct differences between the attributes.
In addition to this, we trained region classifiers on select attributes of the
CelebA dataset, for which we can see the mean attributions in Figure 5. We
observe that the attribution maps for attributes such as Blond Hair and
Eyeglasses correspond with human intuition.

(a) Gender

(b) Race

(c) Age

Figure 4: Mean attribution maps for the FairFace attributes.

(a) Blond Hair

(b) Smiling

(c) Wearing Hat

(d) Eyeglasses

Figure 5: Mean attribution maps for CelebA attributes.

For many applications, a pixel-space attribution map is more practical. We can
achieve this by converting the region-based map to pixel space using Algorithm
1. The algorithm employs the confidences and locations of all patches present
on the original image to produce a pixel-level confidence map matching the
original image’s dimensions. It calculates the confidence for each pixel by
averaging the confidences of all patches containing that pixel. For all
experiments in Section 4, we used pixel space attributions, which were
normalized to a $[0,1]$ range.

Algorithm 1 Map region attributions to pixel space

1: Input: $confidences$, $locations$, $k$, $image\\_size$

2: $map\leftarrow$ zeros tensor of size $image\\_size$

3: $counts\leftarrow$ zeros tensor of size $image\\_size$

4: for $i=0$ to length($locations$) - 1 do

5: $(x,y)\leftarrow$ $locations[i]$

6: $map[y:y+k,x:x+k]\mathrel{+}=confidences[i]$

7: $counts[y:y+k,x:x+k]\mathrel{+}=1$

8: end for

9: return $map/counts$

##  4 Regularized training on biased data

To develop a robust training algorithm, we aim to leverage our attribution
maps to address biases in the training dataset by introducing targeted noise
to regions critical for detecting confounding attributes. Our attribution
technique could enable manipulation of the training data to mitigate the
model’s reliance on these confounders. An illustrative figure can be seen in
Figure 1.

###  4.1 Experiment Setup

In order to evaluate the feasibility of this approach, we conducted
experiments utilizing the CelebA dataset, which comprises a comprehensive
collection of annotated facial attributes. Our investigation focuses on a
scenario wherein a classifier is trained for predicting perceived gender
classification111In the CelebA dataset, perceived gender is denoted as Male,
where a value of $1$ represents individuals belonging to the category of men,
while a value of $0$ is assigned to those who do not, which we label as women
for simplicity’s sake.. The dataset is not sampled uniformly random from the
CelebA dataset. Specifically, the dataset is perfectly balanced for the
classification attribute (3000 instances of men, 3000 instances of women), but
contains bias in terms of another attribute. Within the subset of men, none
exhibit the confounding attribute, whereas among women, 2000 individuals
possess the confounder while 1000 do not. We created datasets in this manner
using Blond Hair, Eyeglasses, Smiling and Wearing Hat as confounding
attributes. Subsequently, we assess performance on a test dataset that is
balanced with respect to gender as well as the hidden attribute. By
introducing this bias into the dataset, our hypothesis posits a significant
discrepancy in test accuracy between male and female subjects, owing to the
disparate distribution of data instances across gender categories in the test
set.

To cultivate a robust classifier, we conducted experiments exploring various
methods of utilizing attribution maps to alter the training data. We
investigated the incorporation of both image-specific and general attributions
(calculated over a large number of samples) of confounding attributes, as
exemplified in Figure 5. For both mask types, we experimented with greying out
as well as additive noise. An illustration of adding noise for the Hair Color
attribute in all manners (General Mask, Specific Mask, General Noise, Specific
Noise) is provided in Figure 6. When introducing noise, we drew samples from a
Gaussian distribution with a mean of $0$ and a standard deviation of $0.5$. In
all cases, we masked out regions deemed most crucial for classifying
unintended biases, such as the top $30\%$ most confident regions within an
image. The selection of the noise addition scheme and the quantile used for
masking out regions were treated as hyperparameters in our analysis.

Figure 6: Example of regularizing the data based on attributions for hair
color. From left to right: Original, General Mask, Specific Mask, General
Noise, Specific Noise.

Several metrics are available for evaluating model fairness, with Demographic
Parity and Equality of Opportunity being the most prominent. In our
assessment, focusing on perceived gender classification, we deemed it
appropriate to examine accuracy per subgroup. This approach allows us to
determine if any particular category (in this case, men or women) is
disproportionately affected by the known bias.

We employed a customized Convolutional Neural Network (CNN) architecture for
gender classification, as we observed that deeper network architectures such
as VGG [28] and ResNet [8] were susceptible to overfitting. Our CNN
architecture comprised three convolutional layers, supplemented with dropout
regularization [29] and batch normalization [10]. To optimize the model
parameters, we utilized the AdamW optimizer [16] with an initial learning rate
set to $10^{-5}$, followed by exponential decay with a decay rate ($\gamma$)
of $0.95$. We conducted training with a batch size of $128$. Robust models
incorporating noise were trained for $20$ epochs, while those utilizing
masking techniques were trained until convergence for $10$ epochs.

###  4.2 Results

|  |  | Accuracy $\uparrow$ | Accuracy Men $\uparrow$ | Accuracy Women $\uparrow$ | Gap $\downarrow$
---|---|---|---|---|---|---
|  | Type | Original | Balanced $\Delta$ | Ours $\Delta$ | Original | Balanced $\Delta$ | Ours $\Delta$ | Original | Balanced $\Delta$ | Ours $\Delta$ | Original | Balanced | Ours
Attribute | Noise | Quantile |  |  |  |  |  |  |  |  |  |  |  |
Blond Hair | General Mask | 0.60 | 0.74 | +0.09 | 0.0 | 0.6 | +0.2 | +0.02 | 0.88 | -0.03 | -0.03 | 0.28 | 0.05 | 0.22
| General Noise | 0.70 | 0.77 | +0.06 | -0.02 | 0.64 | +0.16 | +0.08 | 0.9 | -0.05 | -0.12 | 0.26 | 0.05 | 0.09
| Specific Mask | 0.60 | 0.74 | +0.09 | +0.01 | 0.6 | +0.2 | +0.09 | 0.88 | -0.03 | -0.08 | 0.28 | 0.05 | 0.11
| Specific Noise | 0.80 | 0.79 | +0.04 | -0.05 | 0.67 | +0.13 | +0.01 | 0.9 | -0.05 | -0.1 | 0.23 | 0.05 | 0.15
Eyeglasses | General Mask | 0.60 | 0.71 | +0.06 | -0.02 | 0.61 | +0.11 | +0.03 | 0.82 | +0.0 | -0.07 | 0.21 | 0.10 | 0.12
| General Noise | 0.60 | 0.72 | +0.05 | +0.01 | 0.61 | +0.11 | +0.05 | 0.84 | -0.02 | -0.04 | 0.23 | 0.10 | 0.14
| Specific Mask | 0.60 | 0.71 | +0.06 | +0.03 | 0.61 | +0.11 | +0.13 | 0.82 | +0.0 | -0.08 | 0.21 | 0.10 | 0.05
| Specific Noise | 0.80 | 0.72 | +0.05 | -0.01 | 0.61 | +0.11 | +0.1 | 0.84 | -0.02 | -0.13 | 0.23 | 0.10 | 0.09
Smiling | General Mask | 0.80 | 0.81 | +0.03 | -0.04 | 0.71 | +0.06 | -0.02 | 0.9 | +0.01 | -0.05 | 0.19 | 0.13 | 0.16
| General Noise | 0.60 | 0.85 | -0.01 | -0.05 | 0.79 | -0.02 | -0.04 | 0.91 | -0.0 | -0.07 | 0.12 | 0.13 | 0.09
| Specific Mask | 0.95 | 0.81 | +0.03 | -0.02 | 0.71 | +0.06 | -0.02 | 0.9 | +0.01 | -0.01 | 0.19 | 0.13 | 0.20
| Specific Noise | 0.60 | 0.85 | -0.01 | -0.05 | 0.79 | -0.02 | +0.01 | 0.91 | -0.0 | -0.1 | 0.12 | 0.13 | 0.05
Wearing Hat | General Mask | 0.80 | 0.71 | +0.08 | +0.05 | 0.63 | +0.11 | +0.08 | 0.79 | +0.05 | +0.03 | 0.16 | 0.10 | 0.10
| General Noise | 0.95 | 0.73 | +0.06 | -0.02 | 0.63 | +0.11 | +0.03 | 0.84 | +0.0 | -0.07 | 0.21 | 0.10 | 0.11
| Specific Mask | 0.95 | 0.71 | +0.08 | 0.0 | 0.63 | +0.11 | +0.01 | 0.79 | +0.05 | -0.01 | 0.16 | 0.10 | 0.15
| Specific Noise | 0.95 | 0.73 | +0.06 | -0.03 | 0.63 | +0.11 | +0.04 | 0.84 | +0.0 | -0.1 | 0.21 | 0.10 | 0.08

Table 1: Summary of the noise addition experiment across multiple attributes
of the CelebA dataset. Models are trained on a highly biased dataset regarding
each attribute, leading to a disparity in performance between men and women.
This disparity between performance for men and women is denoted in the gap
column. Accuracies for models trained on a balanced dataset, and those using
our noise addition regularization are shown relative to those trained on the
original, biased dataset. Through the strategic addition of noise to regions
crucial for each attribute based on a specified noise type, we diminish the
discrepancy in accuracies between genders.

We conducted experiments using all attributes illustrated in Figure 5 as
confounding attributes, with the results summarized in Table 1. The table
presents overall accuracy, as well as accuracy for men and women separately on
a balanced test set, with the gap column indicating the difference between
these two. Notably, our experiments encompassed multiple quantiles, yet we
only included those yielding the most significant results for brevity.
Additionally, we compared these results against classifiers trained on a
dataset of equivalent size but balanced with respect to the attribute. It’s
worth noting that the data was not fully balanced in the case of Blond Hair as
the confounder for men but rather $54/46$ in favor of non-blond hair, as the
original CelebA train set contains only $1387$ men with blond hair.

First of all, we can observe that all models have some discrepancy between the
accuracy for men and women, with those trained on the biased dataset without
any modifications having the largest gap. Nevertheless, we observe that for
each confounding attribute, there exists a specific combination of noise type
and quantile for regularizing that yields notably improved equality in
accuracy, despite the significant divergence in the training distribution of
men compared to the balanced test data. Surprisingly, even when trained on
balanced data concerning the attribute, we still observe a performance gap
between men and women, albeit significantly smaller than with highly
unbalanced data. This discrepancy likely persists due to other confounders
that remain present, as we did not balance the data on the remaining $38$
attributes. Furthermore, in the case of smiling, there is almost no
discernible difference between balanced and unbalanced data. This observation
may be attributed to the inherent difficulty in classifying smiling compared
to gender, resulting in a lesser impact of this bias on the model [7].
However, our technique notably reduces the accuracy gap between men and women,
approaching the performance achieved with balanced data, suggesting that our
method is a more practical approach than obtaining a balanced dataset on all
attributes.

Subsequently, we zoom in on the choice of the two hyperparameters. Figure 7
illustrates a comparison of various noise addition strategies for the Blond
Hair attribute, where we removed the $30\%$ most influential information
related to the attribute from all training data. All robust training schemes
either enhance or maintain the accuracy for men, while diminishing the
accuracy for women, thus aligning subgroup accuracies and mitigating the
effect of training on a biased dataset. Notably, the incorporation of General
Noise brings both subset accuracies closest to parity.

Figure 7: Comparison of noise addition techniques for the attribute Blond Hair
using Quantile $0.7$

Next to the type of noise, the quantile of information blocked is an important
hyperparameter as well. Figure 8 shows how the accuracy of the subgroups
evolves with the differing percentages. In the case of Specific Mask for the
Blond Hair attribute, the optimal point seems to be the $70\%$ mark. Masking
out more reduces accuracy for women further while more specific noise
additions increase the inequality in performance to that of unmodified data.

Figure 8: Comparison of metrics for the Specific Mask noise addition, for the
attribute Blond Hair. Dotted lines indicate the accuracy of the original model
for the same metric.

##  5 Conclusion

In conclusion, our paper introduces a novel method for mitigating bias in
machine learning models, crucial for ensuring fairness and equity. We propose
leveraging pixel image attributions to identify and regulate regions within
images containing significant information about biased attributes. Our
approach, employing a model-agnostic technique to extract pixel attributions
through a CNN classifier trained on small image patches, enables the
identification of critical information distribution across images. By
utilizing these attributions to introduce targeted noise into datasets with
confounding attributes, we effectively prevent neural networks from learning
biases and prioritize primary attributes. Our method demonstrates its
effectiveness in training unbiased classifiers on heavily biased datasets,
offering promise for enhancing fairness and equity in machine learning
applications.

## Acknowledgments

Sander De Coninck receives funding from the Special Research Fund of Ghent
University under grant no. BOF22/DOC/093. This research received funding from
the Flemish Government under the “Onderzoeksprogramma Artificiële
Intelligentie (AI) Vlaanderen” programme.

## References

  * Aguilar et al. [2017] Wilbert G Aguilar, Marco A Luna, Julio F Moya, Vanessa Abad, Hugo Ruiz, Humberto Parra, and William Lopez.  Cascade classifiers and saliency maps based people detection.  In _Augmented Reality, Virtual Reality, and Computer Graphics: 4th International Conference, AVR 2017, Ugento, Italy, June 12-15, 2017, Proceedings, Part II 4_ , pages 501–510. Springer, 2017.
  * Boyd et al. [2022] Aidan Boyd, Kevin W. Bowyer, and Adam Czajka.  Human-aided saliency maps improve generalization of deep learning.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)_ , pages 2735–2744, 2022.
  * DeGroot and Fienberg [1983] Morris H DeGroot and Stephen E Fienberg.  The comparison and evaluation of forecasters.  _Journal of the Royal Statistical Society: Series D (The Statistician)_ , 32(1-2):12–22, 1983.
  * Denton et al. [2019] Emily Denton, Ben Hutchinson, Margaret Mitchell, Timnit Gebru, and Andrew Zaldivar.  Image counterfactual sensitivity analysis for detecting unintended bias.  _arXiv preprint arXiv:1906.06439_ , 2019.
  * Dosovitskiy et al. [2021] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby.  An image is worth 16x16 words: Transformers for image recognition at scale.  _ICLR_ , 2021.
  * Ghose et al. [2019] Debasmita Ghose, Shasvat M. Desai, Sneha Bhattacharya, Deep Chakraborty, Madalina Fiterau, and Tauhidur Rahman.  Pedestrian detection in thermal images using saliency maps.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops_ , 2019.
  * Hall et al. [2022] Melissa Hall, Laurens van der Maaten, Laura Gustafson, Maxwell Jones, and Aaron Adcock.  A systematic study of bias amplification.  _arXiv preprint arXiv:2201.11706_ , 2022.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 770–778, 2016.
  * Huang et al. [2023] Linzhi Huang, Mei Wang, Jiahao Liang, Weihong Deng, Hongzhi Shi, Dongchao Wen, Yingjie Zhang, and Jian Zhao.  Gradient attention balance network: Mitigating face recognition racial bias via gradient attention.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 38–47, 2023.
  * Ioffe and Szegedy [2015] Sergey Ioffe and Christian Szegedy.  Batch normalization: Accelerating deep network training by reducing internal covariate shift.  In _International conference on machine learning_ , pages 448–456. pmlr, 2015.
  * Kapishnikov et al. [2019] Andrei Kapishnikov, Tolga Bolukbasi, Fernanda Viégas, and Michael Terry.  Xrai: Better attributions through regions.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 4948–4957, 2019.
  * Karkkainen and Joo [2021] Kimmo Karkkainen and Jungseock Joo.  Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation.  In _Proceedings of the IEEE/CVF winter conference on applications of computer vision_ , pages 1548–1558, 2021.
  * Khan et al. [2019] Rizwan Ahmed Khan, Alexandre Meyer, Hubert Konik, and Saida Bouakaz.  Saliency-based framework for facial expression recognition.  _Frontiers of Computer Science_ , 13:183–198, 2019.
  * Kim et al. [2019] Byungju Kim, Hyunwoo Kim, Kyungsu Kim, Sungjin Kim, and Junmo Kim.  Learning not to learn: Training deep neural networks with biased data.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 9012–9020, 2019.
  * Liu et al. [2015] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang.  Deep learning face attributes in the wild.  In _Proceedings of International Conference on Computer Vision (ICCV)_ , 2015.
  * Loshchilov and Hutter [2017] Ilya Loshchilov and Frank Hutter.  Decoupled weight decay regularization.  _arXiv preprint arXiv:1711.05101_ , 2017.
  * Lundberg and Lee [2017] Scott M Lundberg and Su-In Lee.  A unified approach to interpreting model predictions.  _Advances in neural information processing systems_ , 30, 2017.
  * Mavani et al. [2017] Viraj Mavani, Shanmuganathan Raman, and Krishna P Miyapuram.  Facial expression recognition using visual saliency and deep learning.  In _Proceedings of the IEEE international conference on computer vision workshops_ , pages 2783–2788, 2017.
  * Molnar [2022] Christoph Molnar.  _Interpretable Machine Learning_.  2 edition, 2022.
  * Naeini et al. [2015] Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht.  Obtaining well calibrated probabilities using bayesian binning.  In _Proceedings of the AAAI conference on artificial intelligence_ , 2015.
  * Niculescu-Mizil and Caruana [2005] Alexandru Niculescu-Mizil and Rich Caruana.  Predicting good probabilities with supervised learning.  In _Proceedings of the 22nd international conference on Machine learning_ , pages 625–632, 2005.
  * Olteanu et al. [2019] Alexandra Olteanu, Carlos Castillo, Fernando Diaz, and Emre Kıcıman.  Social data: Biases, methodological pitfalls, and ethical boundaries.  _Frontiers in big data_ , 2:13, 2019.
  * Parraga et al. [2023] Otavio Parraga, Martin D More, Christian M Oliveira, Nathan S Gavenski, Lucas S Kupssinskü, Adilson Medronha, Luis V Moura, Gabriel S Simões, and Rodrigo C Barros.  Fairness in deep learning: A survey on vision and language research.  _ACM Computing Surveys_ , 2023.
  * Rajabi et al. [2023] Amirarsalan Rajabi, Mehdi Yazdani-Jahromi, Ozlem Ozmen Garibay, and Gita Sukthankar.  Through a fair looking-glass: mitigating bias in image datasets.  In _International Conference on Human-Computer Interaction_ , pages 446–459. Springer, 2023.
  * Ramaswamy et al. [2021] Vikram V Ramaswamy, Sunnie SY Kim, and Olga Russakovsky.  Fair attribute classification through latent space de-biasing.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 9301–9310, 2021.
  * Ribeiro et al. [2016] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin.  ” why should i trust you?” explaining the predictions of any classifier.  In _Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining_ , pages 1135–1144, 2016.
  * Selvaraju et al. [2017] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  In _Proceedings of the IEEE international conference on computer vision_ , pages 618–626, 2017.
  * Simonyan and Zisserman [2014] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  _arXiv preprint arXiv:1409.1556_ , 2014.
  * Srivastava et al. [2014] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov.  Dropout: a simple way to prevent neural networks from overfitting.  _The journal of machine learning research_ , 15(1):1929–1958, 2014.
  * Sundararajan et al. [2017] Mukund Sundararajan, Ankur Taly, and Qiqi Yan.  Axiomatic attribution for deep networks.  In _International conference on machine learning_ , pages 3319–3328. PMLR, 2017.
  * Wang et al. [2021] Tan Wang, Chang Zhou, Qianru Sun, and Hanwang Zhang.  Causal attention for unbiased visual recognition.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 3091–3100, 2021.
  * Yucer et al. [2020] Seyma Yucer, Samet Akçay, Noura Al-Moubayed, and Toby P Breckon.  Exploring racial bias within face recognition via per-subject adversarially-enabled data augmentation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , pages 18–19, 2020.
  * Zhang et al. [2018] Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell.  Mitigating unwanted biases with adversarial learning.  In _Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society_ , pages 335–340, 2018.


# Galois Cohomology of Reductive Groups over Function Fields over
${\mathbb{C}}((t))$

Dylon Chow

###### Abstract.

Over a global field (number field or function field of a curve over a finite
field), theorems for the Galois cohomology of algebraic groups have long been
known. For $F$ the function field of a curve over the formal series field
${\mathbb{C}}((t))$, under an additional assumption on the curve, we establish
an explicit description of the localization map in Galois cohomology. This is
achieved by extending the theory of $B(F,G)$ due to Kottwitz.

##  1\. Introduction

Let $k={\mathbb{C}}((t))$ and let $F$ be the function field of a smooth,
projective, geometrically integral curve $X$ over $k$. Let $G$ be a connected
reductive group over $F$. This article discusses, under certain conditions on
$X$, a pointed set $B(G)$ attached to $G$ and the field $F$, as well as a
pointed set $B(G)$ attached to $G$ and the completions of $F$ at closed points
of $X$. These results are in parallel with those established by Kottwitz [7]
for local and global fields.

We let $X^{(1)}$ denote the set of closed points on the $k$-curve $X$. This is
in bijection with the set $V_{F}$ of places (discrete valuations of rank one)
of the field $F=k(X)$ that are trivial on $k$. From our results on $B(G)$, we
obtain results on the first Galois cohomology set $H^{1}(F,G)$ of $G$ and the
localization map

$H^{1}(F,G)\to\prod_{v\in V_{F}}H^{1}(F_{v},G).$

This extends results of Borovoi [2], Kottwitz [7], and Borovoi-Kaletha [3],
who worked over global fields.

The analogous localization map over global fields has been much studied in
connection with the local-global principle. A number of recent articles
address Galois cohomology over what are called fields of arithmetic type.
These are fields that are not necessarily global fields but that share many
properties of global fields. (See [9] for more on this topic.) The field $F$
is a field of arithmetic type. In particular, $F$ has cohomological dimension
$2$ and presents analogies with global fields of positive characteristic,
namely, with function fields in one variable over a finite field.

The main local result is Theorem 4, which gives a bijection between the Galois
coinvariants of Borovoi’s fundamental group with the set of basic elements in
$B(F,G)$. The main global result is Theorem 14, which gives a bijection
between the set of basic elements in $B(F,G)$ and the Galois coinvariants of
Borovoi’s fundamental group with the tensor product of a certain subset of the
divisor group of $X$. For global function fields, this result is due to
Kottwitz [7, Proposition 13.1, Corollary 15.2], whose proofs we adapt to the
context of function fields over ${\mathbb{C}}((t))$. For global function
fields there is a natural topological generator of the absolute Galois group
of a finite field, namely the Frobenius automorphism. The absolute Galois
group of ${\mathbb{C}}((t))$ is also isomorphic to $\widehat{{\mathbb{Z}}}$,
the profinite completion of ${\mathbb{Z}}$, but the bijections we obtain are
not canonical because they depend on a choice of a canonical generator of the
absolute Galois group.

For other results related to those obtained in this article, see the paper
[4], which establishes arithmetic duality theorems for the Galois cohomology
of tori and finite Galois modules in the context of function fields over
${\mathbb{C}}((t))$.

Section 2 of this article discusses the local theory, while section 3
discusses the global theory.

##  2\. Local Theory

###  2.1. Review of Tate-Nakayama triples

Throughout this article, if $\Gamma$ is a finite group and $M$ is a
$\Gamma$-module, $H^{r}(\Gamma,M)$ denotes the $r$th Tate cohomology group of
$\Gamma$ with coefficients in $M$. We recall the notion of a Tate-Nakayama
triple due to Kottwitz [7]. Let $G$ be a finite group. Let $X$ and $A$ be
$G$-modules, and let $\alpha\in H^{2}(G,\mathrm{Hom}(X,A))$ We say that
$(X,A,\alpha)$ is a weak Tate-Nakayama triple for $G$ if the following
condition holds for every subgroup $G^{\prime}$ of $G$:

  * •

For all $r\in{\mathbb{Z}}$, cup product with
$\mathrm{Res}_{G/G^{\prime}}(\alpha)$ induces isomorphisms

$H^{r}(G^{\prime},X)\to H^{r+2}(G^{\prime},A).$

We say that $(X,A,\alpha)$ is rigid if

  * •

$H^{1}(G^{\prime},\mathrm{Hom}(X,A))$ is trivial for every subgroup
$G^{\prime}$ of $G$.

A Tate-Nakayama triple is a weak Tate-Nakayama triple that is also rigid.

###  2.2. Notation

We consider a function field over $k={\mathbb{C}}((t))$. We let $F$ be a
completion of this field with respect to a discrete valuation on the field
that is trivial on ${\mathbb{C}}((t))$. The residue field of $F$ is a finite
extension of ${\mathbb{C}}((t))$. We let
$\overline{k}=\cup_{n}{\mathbb{C}}((t^{1/n}))$ and choose an algebraic closure
$\overline{F}$ of $F$. We let $\Gamma(\overline{F}/F)$ denote its Galois
group. If $E$ is a field, $\mathrm{Br}(E)$ denotes the Brauer group of $E$.

###  2.3. Review of local class field theory

Although $F$ is not a local field, the main results of local class field
theory hold for $F$. This is summarized as follows, see for instance, [6,
Section 6.3]. First we note that $k={\mathbb{C}}((t))$ is quasi-finite, i.e.,
it is perfect and its Galois group is isomorphic to the profinite completion
$\widehat{{\mathbb{Z}}}$ of ${\mathbb{Z}}$. We fix a topological generator
$\sigma=\sigma_{k}$ of $\Gamma(\overline{k}/k)$, or equivalently, a fixed
isomorphism $\varphi_{k}\colon\Xi(\Gamma_{k})\to{\mathbb{Q}}/{\mathbb{Z}}$
where $\Xi(\Gamma_{k})$ is the character group
$\text{Hom}_{cts}(\Gamma_{k},{\mathbb{Q}}/{\mathbb{Z}})$ of $\Gamma_{k}$. The
relation between $\sigma$ and $\varphi_{k}$ is that
$\varphi_{k}(\chi)=\chi(\sigma)$ for all $\chi\in\Xi(\Gamma_{k})$. A finite
extension $l$ of a quasi-finite field $k$ is again quasi-finite with generator
$\sigma_{l}=\sigma_{k}^{[l:k]}$.

We refer to [8] for the definition of a class formation that is used in this
article.

######  Proposition 1.

The pair $(\Gamma(\overline{F}/F),\overline{F}^{\times})$ is a class
formation.

###### Proof.

There is a split exact sequence [6, Corollary 6.3.5]

$0\to\mathrm{Br}(\kappa)\to\mathrm{Br}(K)\to\mathrm{Hom}_{\mathrm{cts}}(\Gamma(\kappa_{s}/\kappa),{\mathbb{Z}})\to
0.$

Since ${\mathbb{C}}((t))$ is of dimension $\leq 1$, $\kappa$ is also of
dimension 1, and so $\mathrm{Br}(\kappa)=0$. The commutativity of the diagram
follows from [6, Proposition 6.3.7]. This proposition is stated only for
finite residue field, but the proof works in this setting, since we have fixed
a topological generator of $\Gamma(k^{s}/k)$. ∎

Let $K$ be a finite Galois extension of $F$. The isomorphism

$\mathrm{inv}_{F}\colon\mathrm{Br}(F)\cong{\mathbb{Q}}/{\mathbb{Z}}$

obtained from the class formation
$(\Gamma(\overline{F}/F),\overline{F}^{\times})$ induces an isomorphism

$\mathrm{inv}_{K/F}\colon
H^{2}(\Gamma(K/F),K^{\times})\cong\frac{1}{n}{\mathbb{Z}}/{\mathbb{Z}}$

(See Theorem 2.1 in Milne’s notes on class field theory). We define the
(local) fundamental class $\alpha(K/F)$ of $K/F$ to be the element
$\alpha(K/F)$ of $H^{2}(\Gamma(K/F),K^{\times})$ such that

$\mathrm{inv}_{K/F}(\alpha(K/F))=\frac{1}{[K:F]}\ \mathrm{mod}\ {\mathbb{Z}}.$

One can rephrase this theorem as saying that we get a Tate-Nakayama triple
$(X,A,\alpha)$ for $\Gamma(K/F)$ by taking

  * •

$X$ to be ${\mathbb{Z}}$, with $\Gamma(K/F)$ acting trivially;

  * •

$A$ to be $K^{\times}$, with the natural $\Gamma(K/F)$-action, and

  * •

$\alpha$ to be the fundamental class in $H^{2}(\Gamma(K/F),K^{\times})$.

###  2.4. Algebraic 1-cocycles

We choose an extension

$1\to K^{\times}\to W(K/F)\to\Gamma(K/F)\to 1$

that corresponds to the local fundamental class $\alpha(K/F)$. Notice that
$W(K/F)$ is a local Weil group for the extension $K/F$. If $M$ is a
$\Gamma(K/F)$-module, the extension defines an action of $W(K/F)$ on $M$, with
$K^{\times}$ acting trivially.

We review the definition of an algebraic 1-cocycle, following [7]. Let $G$ be
a linear algebraic group over $F$. The Galois group $\Gamma=\Gamma(K/F)$ acts
on $G(K)$. We consider the set $Z^{1}(W(F/K),G(K))$ of 1-cocycles of $W(F/K)$
in $G(K)$. An algebraic $1$-cocycle of $W(F/K)$ in $G(K)$ is a pair $(\nu,x)$
consisting of

  * •

a homomorphism $\nu\colon D\to G$ over $K$, and

  * •

an abstract $1$-cocycle $x$ of $W(K/F)$ in $G(K)$,

satisfying the following two compatibilities:

  * •

$x_{d}=\nu(d)$ for all $d\in D(E)$,

  * •

$\text{Int}(x_{w})\circ\sigma(\nu)=\nu$ whenever $w\in\mathcal{E}$ maps to
$\sigma\in\Gamma(E/F)$.

We consider the set $Z^{1}(W(K/F),G(K))$ of abstract 1-cocycles of $W(K/F)$ in
$G(K)$ and the set $Z^{1}_{\mathrm{alg}}(W(K/F),G(K))$ of algebraic
$1$-cocycles of $W(K/F)$ in $G(K)$. We write
$H^{1}_{\mathrm{alg}}(W(K/F),G(K))$ for the pointed set obtained as the
quotient of $Z^{1}_{\mathrm{alg}}(W(K/F),G(K))$ by the action of $G(K)$.

Let $T$ be an $F$-torus that splits over $K$. Kottwitz has defined an
isomorphism

$\kappa_{T}\colon H^{1}_{\mathrm{alg}}(W(K/F),T(K))\cong
X_{*}(T)_{\Gamma(K/F)},$

uniquely characterized by certain compatibility properties (see [7], Lemma
4.1).

Let $\Lambda_{G}$ denote Borovoi’s algebraic fundamental group (see [2] for
the definition and basic properties).

######  Proposition 2.

There is a unique map of natural transformations

$\kappa_{G}\colon
H^{1}_{\mathrm{alg}}(W(K/F),G(K))\to(\Lambda_{G})_{\Gamma(K/F)}$

that agrees with $\kappa_{T}$ when $G=T$ is an $F$-torus split by $K$.

###### Proof.

See [7], Prop. 9.1. ∎

###  2.5. Local $B(F,G)$

We consider local fields $L\supset K\supset F$ with both $L$ and $K$ finite
Galois over $F$. One has a natural map

$H^{1}_{\mathrm{alg}}(W(K/F),G(K))\to H^{1}_{\mathrm{alg}}(W(L/F),G(L))$

(see expression (8.32) in [7], page 35) and it is an isomorphism when $G$ is a
torus split by $K$ [7, Lemma 8.1]. For any linear algebraic $F$-group $G$ we
define a pointed set $B(F,G)$ to be the direct limit

$B(F,G)=\varinjlim H^{1}_{\mathrm{alg}}(W(K/F),G(K)),$

the direct limit being taken over the set of finite Galois extensions $K$ of
$F$ in $\overline{F}$. For $L\supset K$, the transition map is the inflation
map

$H^{1}_{alg}(W(K/F),G(K))\to H^{1}_{alg}(W(L/F),G(L))$

defined above.

###  2.6. Basic elements (local)

We review the definition a subset $B(F,G)_{\mathrm{bsc}}\subset B(F,G)$ of
basic elements, following Kottwitz. For fields $L\supset K\supset F$ with both
$L$ and $K$ finite Galois over $F$ and contained in $\overline{F}$, let
${\mathbb{Z}}_{K}$ and ${\mathbb{Z}}_{L}$ be copies of ${\mathbb{Z}}$ attached
to $K$ and $L$. Let $p\colon{\mathbb{Z}}_{K}\to{\mathbb{Z}}_{L}$ be defined by
multiplication by $[L:K]$. We define $\mathbb{D}_{F}$ to be the protorus over
$F$ whose character group is

$X^{*}(\mathbb{D}_{F})=\varinjlim{\mathbb{Z}}_{K}={\mathbb{Q}}$

with the transition maps $p\colon{\mathbb{Z}}_{K}\to{\mathbb{Z}}_{L}$. Thus
$\mathbb{D}_{F}=\varprojlim\mathbb{G}_{m,K}$, the limit being taken over the
directed set of finite Galois extensions of $F$ in $\overline{F}$. The map
$(\nu,x)\mapsto\nu$ induces a well-defined map

$H^{1}_{\mathrm{alg}}(W(K/F),G(K))\to(\mathrm{Hom}_{K}(\mathbb{G}_{m},G)/\mathrm{Int}(G(K))^{\Gamma(K/F)}$

which is called a Newton map. These maps fit together to give a Newton map

$B(F,G)\to[\mathrm{Hom}_{\overline{F}}(\mathbb{D}_{F},G)/G(\overline{F})]^{\Gamma}.$

The image of $b\in B(F,G)$ under the Newton map is called the Newton point of
$b$.

The natural maps $H^{1}(\Gamma(K/F),G(K))\hookrightarrow
H^{1}_{\mathrm{alg}}(W(K/F),G(K))$ fit together to give an injective map

$H^{1}(F,G)\hookrightarrow B(F,G),$

whose image is the kernel of the Newton map.

We denote by $Z(G)$ the center of $G$. The inclusion $Z(G)\hookrightarrow G$
induces an injection

$\mathrm{Hom}_{\overline{F}}(\mathbb{D}_{F},Z(G))\hookrightarrow[\mathrm{Hom}_{\overline{F}}(\mathbb{D}_{F},G)/G(\overline{F})]^{\Gamma}.$

We say that $b\in B(F,G)$ is basic if its Newton point lies in the image of
the map above. We write $B(F,G)_{\mathrm{bsc}}$ for the set of basic elements
in $B(F,G)$. The set $B(F,G)_{bsc}$ contains the image of $H^{1}(F,G)$ under
the inclusion (1.1).

###  2.7. Main theorem (local)

######  Lemma 3.

Let $G$ be a connected reductive group over $F$ and let $G^{\mathrm{ss}}$
denote the derived subgroup of $G$. Then $G$ contains an elliptic maximal
$F$-torus, i.e., a maximal $F$-torus $T$ such that $T\cap G^{\mathrm{ss}}$ is
an anisotropic $F$-torus.

###### Proof.

Since every maximal torus of $G^{\mathrm{ss}}$ is contained in a maximal torus
of $G$, we may assume that $G$ is semisimple. We use the proof of [11, Theorem
6.21]. The lemma in this reference states the result for non-archimedean local
fields, but the proof works in our setting. We have to adapt the proof because
we need to know that there exists a cyclic extension of $K$ of degree $n$ for
every integer $n\geq 2$. But this follows from the fact that the residue field
of $F$ is quasi-finite. ∎

The main result of this section is the following theorem. Let
$\Gamma=\Gamma(\overline{F}/F)$.

######  Theorem 4.

The map $\kappa_{G}$ restricts to a bijection

$\kappa_{G}\colon B(F,G)_{\mathrm{bsc}}\to(\Lambda_{G})_{\Gamma}.$

###### Proof.

The proof is the same as that of [7, Prop. 13.1], using the lemma above. ∎

######  Corollary 5.

The map $\kappa_{G}$ restricts to a bijection
$H^{1}(F,G)\to(\Lambda_{G})_{\Gamma,\mathrm{tors}}$.

We note that it is possible to give another proof of the corollary without
using $B(G)$, for example by using hypercohomology as in [2]. One obtains a
map

$H^{1}(K,G)\cong(\Lambda_{G})_{\Gamma,\mathrm{tors}}$

as the composition of an abelianization map with an isomorphism coming from
local Tate-Nakayama duality (see [2, Prop. 4.1(i)]). However, since the theory
of $B(G)$ over local and global fields has applications to automorphic forms,
we hope that developing a theory of $B(G)$ will yield interesting
applications.

##  3\. Global Theory

###  3.1. Notation

In this section, we let $F$ be a function field over $k={\mathbb{C}}((t))$ and
let $X$ be a geometrically connected projective smooth curve over $k$ with
function field $F$. We fix a topological generator $\sigma$ of the absolute
Galois group of ${\mathbb{C}}((t))$. The set of closed points of $X$ will be
denoted by $X^{(1)}$ (thus $X^{(1)}$ omits only the generic point of $X$). We
are interested in the family of completions $F_{v}$ of $F$ relative to
discrete valuations of rank 1 that are trivial on $k={\mathbb{C}}((t))$, that
is to say the valuations corresponding to the closed points of the $k$-curve
$X$. We let $R_{v}$ denote the ring of integers in $K_{v}$. For such a
valuation $v$, the residue field $k(v)$ is a quasi-finite field of degree
$\mathrm{deg}(v)$ over $k={\mathbb{C}}((t))$ with $\sigma^{\mathrm{deg}(v)}$
as the chosen generator of $\Gamma(\overline{k(v)}/k(v))$. It is of the form
${\mathbb{C}}((s))$ with $s=t^{1/n}$ for suitable $n$.

We let $\mathrm{Jac}_{X}$ denote the Jacobian variety of $X$. From now on, we
make the following assumption on the curve $X$.

  * •

For every finite extension $k^{\prime}$ of $k$, the cohomology group
$H^{1}(\Gamma_{k^{\prime}},\mathrm{Jac}_{X}(\overline{k}))=0$.

In particular, the Brauer group $\text{Br}(X)$ is assumed to be trivial [8,
Proposition A.13]. We make this assumption in order to obtain a global class
formation, as reviewed below.

We define the group of ideles $I_{F}$ of $F$ to be the subgroup of
$\prod_{v\in X^{(1)}}F_{v}^{\times}$ comprising those elements $a=(a_{v})$
such that $a_{v}\in R_{v}^{\times}$ for all but finitely many $v$. The
quotient of $I_{F}$ by $F^{\times}$ (embedded diagonally) is the idele class
group $C_{F}$ of $F$. We set $C=\varinjlim C_{K}$ (limit over all finite
extensions $K$ of $F$, $K\subset\overline{F}$).

Let $K/F$ be a finite Galois extension with Galois group $\Gamma(K/F)$. Then
$\Gamma(K/F)$ operates on the idele group $I_{K}$ of $K$, on the
multiplicative group $K^{\times}$ of $K$, and on the idele class group $C_{K}$
of $K$. Corresponding to the thee $\Gamma(K/F)$-modules $I_{K},C_{K}$, and
$K^{\times}$ there are three $\Gamma(K/F)$-modules of much simpler structure.

We let ${\mathbb{Z}}[V_{K}]$ denote the divisor class group of the function
field $K$, i.e., the free abelian group having one basis element $x_{P}$ for
each closed point $P$ of $X$, and we let $\Gamma(K/F)$ operate on
${\mathbb{Z}}[V_{K}]$ according to its effect on the primes, namely,
$\lambda(x_{P})=x_{\lambda(P)}$. The module that corresponds to $C_{K}$ is
${\mathbb{Z}}$ with the trivial action. The natural homomorphism
${\mathbb{Z}}[V_{K}]\to{\mathbb{Z}}$ defined by $x_{P}\mapsto 1$ for all $P$
is surjective, and we define ${\mathbb{Z}}[V_{K}]_{0}$ to be the kernel of
this homomorphism. In other words, ${\mathbb{Z}}[V_{K}]_{0}$ is the kernel of
the homomorphism ${\mathbb{Z}}[V_{K}]\to{\mathbb{Z}}$ defined by $\sum_{v\in
V_{K}}n_{v}v\mapsto\sum_{v\in V_{K}}n_{v}$.

We define $\mathbb{D}_{K/F}$ to be the $F$-group of multiplicative type whose
character group is ${\mathbb{Z}}[V_{K}]_{0}$.

###  3.2. Review of global class field theory over function fields

Write $a_{v}$ for the image of an element $a$ of $\mathrm{Br}(F)$ in
$\mathrm{Br}(F_{v})$, and define

$\mathrm{inv}_{F}\colon\mathrm{Br}(F)\to{\mathbb{Q}}/{\mathbb{Z}}$

to be $a\mapsto\sum_{v}\mathrm{inv}_{v}(a_{v})$, where $\text{inv}_{v}$ is
$\mathrm{inv}_{F_{v}}$.

######  Lemma 6.

There is an exact sequence

$0\to
H^{1}(\Gamma_{k},\mathrm{Jac}_{X}(\overline{k}))\to\mathrm{Br}(F)\to\bigoplus_{v\in
X^{(1)}}\mathrm{Br}(F_{v})\to{\mathbb{Q}}/{\mathbb{Z}}\to 0.$

###### Proof.

This is [8, Theorem A.7, page 131]. ∎

######  Lemma 7.

If $H^{1}(\Gamma_{k^{\prime}},\mathrm{Jac}_{X}(\overline{k}))=0$ for all
finite extensions $k^{\prime}$ of $k$, then it is possible to define on
$(\Gamma_{F},C)$ a natural structure of a class formation.

###### Proof.

This is [8, Corollary A.8, page 132]. ∎

######  Corollary 8.

Suppose that $H^{1}(\Gamma_{k^{\prime}},\mathrm{Jac}_{X}(\overline{k}))=0$ for
all finite extensions $k^{\prime}$ of $k$. For every finite Galois extension
$K$ of $F$, there is an isomorphism

$\mathrm{inv}_{K/F}\colon
H^{2}(\Gamma(K/F),C_{K})\to\frac{1}{n}{\mathbb{Z}}/{\mathbb{Z}}.$

The cohomology class $\alpha\in H^{2}(\Gamma(K/F),C_{K})$ such that
$\mathrm{inv}_{K/F}(\alpha)=1/n$ is called the global fundamental class.

###  3.3. Cohomology of tori

In this section, we extend the results of [14] to our context. We compare the
two exact sequences of $\Gamma(K/F)$-modules

$1\to K^{\times}\to I_{K}\to C_{K}\to 1$

and

$1\to\mathbb{Z}[V_{K}]_{0}\to\mathbb{Z}[V_{K}]\to\mathbb{Z}\to 1.$

######  Theorem 9.

There are elements

$\displaystyle\alpha_{1}$ $\displaystyle\in
H^{2}(\Gamma(K/F),\mathrm{Hom}(\mathbb{Z},C_{K})),$ $\displaystyle\alpha_{2}$
$\displaystyle\in H^{2}(\Gamma(K/F),\mathrm{Hom}({\mathbb{Z}}[V_{E}],I_{K})),$
$\displaystyle\alpha_{3}$ $\displaystyle\in
H^{2}(\Gamma(K/F),\mathrm{Hom}({\mathbb{Z}}[V_{K}]_{0},K^{\times}))$

such that cup product by these cohomology classes induces isomorphisms

$\displaystyle H^{r}(\Gamma(K/F),{\mathbb{Z}})$ $\displaystyle\to
H^{r+2}(\Gamma(K/F),C_{K}),$ $\displaystyle
H^{r}(\Gamma(K/F),{\mathbb{Z}}[V_{K}])$ $\displaystyle\to
H^{r+2}(\Gamma(K/F),I_{K}),$ $\displaystyle
H^{r}(\Gamma(K/F),{\mathbb{Z}}[V_{K}]_{0})$ $\displaystyle\to
H^{r+2}(\Gamma(K/F),K^{\times})$

for all $r$, $-\infty<r<\infty$.

###### Proof.

The class $\alpha_{1}$ is the global fundamental class of $K/F$. For
$\alpha_{2}$ and $\alpha_{3}$, see [14]. ∎

######  Proposition 10.

The triple $({\mathbb{Z}}[V_{K}]_{0},K^{\times},\alpha_{3})$ is a Tate-
Nakayama triple.

###### Proof.

See [7], section 6.3. ∎

###  3.4. $B(F,G)$ (global)

We choose an extension

$1\to\mathrm{Hom}({\mathbb{Z}}[V_{K}]_{0},K^{\times})\to\mathcal{E}(K/F)\to\Gamma(K/F)\to
1$

whose associated cohomology class is $\alpha_{3}$. Using this extension, we
get (see [7]), for each linear algebraic group over $F$, a pointed set
$H^{1}_{\mathrm{alg}}(\mathcal{E}(K/F),G(K))$.

Given a larger finite Galois extension $L\supset K$, there is a natural
$\Gamma(L/F)$-map $p\colon{\mathbb{Z}}[V_{K}]_{0}\to{\mathbb{Z}}[V_{L}]_{0}$
defined as follows. The value of $p$ on the basis element $v\in V_{K}$ is
defined to be

$p(v)=\sum_{w\lvert v}[L_{w}:K_{v}]w.$

Note that since $[L:K]=\sum_{w\lvert v}[L_{w}:K_{v}]$, $p$ is well-defined.

Using $p$, one forms an inflation map

$H^{1}_{\mathrm{alg}}(\mathcal{E}(K/F),G(K))\to
H^{1}_{\mathrm{alg}}(\mathcal{E}(L/F),G(L))$

(see [7], expression (8.32) on page 42). Using these inflation maps as
transition morphisms, we form a pointed set $B(F,G)$ as the colimit of
$H^{1}_{\mathrm{alg}}(\mathcal{E}(K/F),G(K))$, with $K$ varying over the
directed set of finite Galois extensions of $F$ in some fixed separable
closure $\overline{F}$ of $F$:

$B(F,G)=\varinjlim H^{1}_{\mathrm{alg}}(\mathcal{E}(K/F),G(K)).$

For a place $u$ of $F$ there is a localization map

$B(F,G)\to B(F_{u},G).$

The definition of this map is identical to that in [7], expression (10.7) on
page 49.

######  Proposition 11.

Let $G$ be a connected reductive $F$-group, the total localization map

$B(F,G)\to\prod_{u\in V_{F}}B(F_{u},G)$

takes values in $\bigoplus_{u\in V_{F}}B(F_{u},G)$. That is, the components of
any element in the image of the total localization map are trivial for all but
finitely many $u\in V_{F}$.

###### Proof.

The same proof of Corollary 14.3 in [7] works, after replacing Lang’s Theorem
by Steinberg’s vanishing theorem for connected linear algebraic groups over
fields of dimension $\leq 1$ [13, Theorem 1.9]. ∎

We choose a finite Galois extension $K$ of $F$ in $\overline{F}$ such that
$\Gamma(\overline{F}/K)$ acts trivially on $\Lambda_{G}$. Put

$A(F,G)=(\Lambda_{G}\otimes{\mathbb{Z}}[V_{K}]_{0})_{\Gamma(K/F)}.$

There is a natural map (see [7, Section 11.5])

$\kappa_{G}\colon B(F,G)\to A(F,G).$

We have the composed map

${\mathbb{Z}}[V_{K}]_{0}\hookrightarrow{\mathbb{Z}}[V_{K}]\to{\mathbb{Z}}[V_{u}],$

where $V_{u}$ is the set of places of $K$ lying over $u$. Tensoring with
$\Lambda_{G}$ and forming $\Gamma(K/F)$-coinvariants, we obtain the composed
map

$(\Lambda_{G}\otimes{\mathbb{Z}}[V_{K}]_{0})_{\Gamma(K/F)}\to(\Lambda_{G}\otimes{\mathbb{Z}}[V_{K}])_{\Gamma(K/F)}\to(\Lambda_{G}\otimes{\mathbb{Z}}[V_{u}])_{\Gamma(K/F)}\cong(\Lambda_{G})_{\Gamma(K_{v}/F_{u})}.$

Then we get a map

$(\Lambda_{G}\otimes{\mathbb{Z}}[V_{K}]_{0})_{\Gamma(K/F)}\to(\Lambda_{G})_{\Gamma(K_{v}/F_{u})}.$

By taking the colimit, we obtain a localization map

$A(F,G)\to A(F_{u},G).$

###  3.5. Basic elements (global)

There is a (global) Newton map

$B(F,G)\to[\mathrm{Hom}_{\overline{F}}(\mathbb{D}_{F},G)/G(\overline{F})]^{\Gamma},$

where $\Gamma=\mathrm{Gal}(\overline{F}/F)$ and
$\mathbb{D}_{F}=\varprojlim\mathbb{D}_{K/F}$, the limit being taken over the
directed set of finite Galois extensions $K$ of $F$ in $\overline{F}$. The
kernel of the Newton map is the image of $H^{1}(F,G)$ under the natural
inclusion.

Inside the target of the Newton map is the subset
$\text{Hom}_{F}(\mathbb{D}_{F},Z(G))$. The preimage of
$\mathrm{Hom}_{F}(\mathbb{D}_{F},Z(G))$ is by definition the set
$B(F,G)_{\mathrm{bsc}}$ of basic elements of $B(F,G)$. The image of
$H^{1}(F,G)$ in $B(F,G)$ is contained in $B(F,G)_{\mathrm{bsc}}$.

###  3.6. Main theorem (global)

We make some preliminary remarks.

######  Lemma 12.

Let $H$ be a semisimple simply connected algebraic group over $F$. Then
$H^{1}(F,H)=0$.

###### Proof.

One can reduce to the case where $H$ is absolutely almost simple. For inner
forms of type $A_{n}$, this is due to Merkurjev and Suslin and for the other
classical groups it is a theorem of Bayer and Parimala [1]. For exceptional
groups, it follows from a theorem of Gille [5] as well as index-period
properties of $C_{2}$ fields (see [10], Theorem on page 194). ∎

######  Lemma 13.

Let $S$ be a finite nonempty set of places of $F$. There exists a maximal
$F$-torus $T$ in $G$ such that $T$ is elliptic over $F_{u}$ for every $u\in
S$.

###### Proof.

We choose an elliptic torus $M^{v}$ for each $v\in S$. We follow the proof of
Section 7.1 Corollary 3 of [11]; see also the proof of Theorem 2 of [12]. Let
$\mathcal{T}$ denote the variety of maximal tori of $G$. For any valuation $v$
of $K$, the map $\phi\colon G(K_{v})\to\mathcal{T}(K_{v})$ given by $g\mapsto
g\cdot M^{v}$, where $g\cdot M_{v}=g^{-1}M^{v}g$, is open in the $v$-adic
topology. Thus, for any $v\in S$, the orbit set $G(F_{v})\cdot M^{v}$ is open
in $\mathcal{T}(K_{v})$. Since $\mathcal{T}$ is rational and smooth, it
satisfies weak approximation, and so there exists a maximal torus

$T\in\mathcal{T}(F)\cap\prod_{v\in S}(G(F_{v})\cdot M^{v}).$

Now $G(F_{v})\cdot M^{v}$ consists of $F_{v}$-tori that are conjugate to
$M^{v}$ and so are in particular elliptic. This completes the proof. ∎

The main results of this section are the following theorem and corollaries.

######  Theorem 14.

The map $\kappa_{G}\colon B(F,G)\to A(F,G)$ restricts to a bijection

$\kappa_{G}\colon B(F,G)_{\mathrm{bsc}}\to
A(F,G)=(\Lambda_{G}\otimes{\mathbb{Z}}[V_{K}]_{0})_{\Gamma(K/F)}.$

###### Proof.

One only needs to modify the proof of [7, Proposition 15.1] by using the two
lemmas above. ∎

######  Corollary 15.

The diagram

${B(F,G)_{\mathrm{bsc}}}$${\bigoplus_{u\in
V_{F}}B(F_{u},G)_{\mathrm{bsc}}}$${A(F,G)}$${\bigoplus_{u\in
V_{F}}A(F_{v},G)}$$\scriptstyle{\kappa_{G}}$$\scriptstyle{\kappa_{G}}$

commutes and the vertical arrows are bijections.

###### Proof.

It remains to prove the compatibility of the local and global maps
$\kappa_{G}$ with the localization maps. This follows from [7, Lemma 11.7]. ∎

######  Corollary 16.

An element in $\bigoplus_{u\in V_{F}}B(F_{u},G)_{\mathrm{bsc}}$ lies in the
image of the localization map

$B(F,G)_{\mathrm{bsc}}\to\bigoplus_{u\in V_{F}}B(F_{u},G)_{\mathrm{bsc}}$

if and only if its image under

$\bigoplus_{u\in V_{F}}B(F_{u},G)_{\mathrm{bsc}}\to\bigoplus_{u\in
V_{F}}(\Lambda_{G})_{\Gamma(K_{v}/F_{u})}\to(\Lambda_{G})_{\Gamma(K/F)}$

is trivial.

######  Corollary 17.

The map $B(F,G)_{\mathrm{bsc}}\to A(F,G)$ restricts to a bijection

$H^{1}(F,G)\to(\Lambda_{G}\otimes{\mathbb{Z}}[V_{K}]_{0})_{\Gamma(K/F),\mathrm{tors}}=A(F,G)_{\mathrm{tors}}.$

The bijection gives $H^{1}(F,G)$ the structure of an abelian group.

###  3.7. Concluding Remarks

The approach of Borovoi-Kaletha [3] can probably be modified to prove the
corollary above. In this paper the assumption made on the curve $X$ was
crucial, in order to get global Tate-Nakayama triples. We cannot say anything
about what happens if the assumption does not hold, in particular, if the
Brauer group of the $k$-curve $X$ is not trivial.

## References

  * [1] E. Bayer-Fluckiger and R. Parimala.  Galois cohomology of the classical groups over fields of cohomological dimension $\leq 2$.  Invent. Math., 122(2):195–229, 1995.
  * [2] Mikhail Borovoi.  Abelian Galois cohomology of reductive groups, volume 626.  American Mathematical Soc., 1998.
  * [3] Mikhail Borovoi and Tasho Kaletha.  Galois cohomology of reductive groups over global fields.  arXiv preprint arXiv:2303.04120, 2023.
  * [4] Jean-Louis Colliot-Thélène and David Harari.  Dualité et principe local-global pour les tores sur une courbe au-dessus de $\mathbb{C}((t))$.  Proc. Lond. Math. Soc. (3), 110(6):1475–1516, 2015.
  * [5] Philippe Gille.  Cohomologie galoisienne des groupes quasi-déployés sur des corps de dimension cohomologique $\leq 2$.  Compositio Math., 125(3):283–325, 2001.
  * [6] Philippe Gille and Tamás Szamuely.  Central simple algebras and Galois cohomology, volume 165 of Cambridge Studies in Advanced Mathematics.  Cambridge University Press, Cambridge, second edition, 2017.
  * [7] Robert Kottwitz.  B(G) for all local and global fields, 2014.
  * [8] J. S. Milne.  Arithmetic duality theorems.  BookSurge, LLC, Charleston, SC, second edition, 2006.
  * [9] R. Parimala.  Arithmetic of linear algebraic groups over two-dimensional fields.  In Proceedings of the International Congress of Mathematicians. Volume I, pages 339–361. Hindustan Book Agency, New Delhi, 2010.
  * [10] V. P. Platonov and V. I. Yanchevskii.  Finite-dimensional division algebras.  In Algebra, IX, volume 77 of Encyclopaedia Math. Sci., pages 121–239. Springer, Berlin, 1995.  Translated from the Russian by P. M. Cohn.
  * [11] Vladimir Platonov and Andrei Rapinchuk.  Algebraic groups and number theory, volume 139 of Pure and Applied Mathematics.  Academic Press, Inc., Boston, MA, 1994.  Translated from the 1991 Russian original by Rachel Rowen.
  * [12] Gopal Prasad and Andrei S. Rapinchuk.  Subnormal subgroups of the groups of rational points of reductive algebraic groups.  Proc. Amer. Math. Soc., 130(8):2219–2227, 2002.
  * [13] Robert Steinberg.  Regular elements of semisimple algebraic groups.  Inst. Hautes Études Sci. Publ. Math., (25):49–80, 1965.
  * [14] J. Tate.  The cohomology groups of tori in finite Galois extensions of number fields.  Nagoya Math. J., 27:709–719, 1966.


* Martinez et al. (2017) Martinez, J., Hossain, R., Romero, J., Little, J.J., 2017\.  A simple yet effective baseline for 3d human pose estimation.  arXiv:1705.03098 [cs] URL: http://arxiv.org/abs/1705.03098. arXiv: 1705.03098.
  * Mathis et al. (2018) Mathis, A., Mamidanna, P., Cury, K.M., Abe, T., Murthy, V.N., Mathis, M.W., Bethge, M., 2018.  DeepLabCut: markerless pose estimation of user-defined body parts with deep learning.  Nature Neuroscience 21, 1281--1289.  URL: https://www.nature.com/articles/s41593-018-0209-y, doi:10.1038/s41593-018-0209-y. number: 9 Publisher: Nature Publishing Group.
  * Mathis and Mathis (2019) Mathis, M.W., Mathis, A., 2019\.  Deep learning tools for the measurement of animal behavior in neuroscience.  arXiv:1909.13868 [cs, q-bio] URL: http://arxiv.org/abs/1909.13868. arXiv: 1909.13868.
  * Mehta et al. (2016) Mehta, D., Rhodin, H., Casas, D., Fua, P., Sotnychenko, O., Xu, W., Theobalt, C., 2016.  [1611.09813] Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision.  URL: https://arxiv.org/abs/1611.09813.
  * Mehta et al. (2020) Mehta, D., Sotnychenko, O., Mueller, F., Xu, W., Elgharib, M., Fua, P., Seidel, H.P., Rhodin, H., Pons-Moll, G., Theobalt, C., 2020\.  XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera.  ACM Transactions on Graphics 39.  URL: http://arxiv.org/abs/1907.00837, doi:10.1145/3386569.3392410. arXiv: 1907.00837.
  * Mehta et al. (2018) Mehta, D., Sotnychenko, O., Mueller, F., Xu, W., Sridhar, S., Pons-Moll, G., Theobalt, C., 2018.  Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB.  arXiv:1712.03453 [cs] URL: http://arxiv.org/abs/1712.03453. arXiv: 1712.03453.
  * Mehta et al. (2017) Mehta, D., Sridhar, S., Sotnychenko, O., Rhodin, H., Shafiei, M., Seidel, H.P., Xu, W., Casas, D., Theobalt, C., 2017.  VNect: real-time 3D human pose estimation with a single RGB camera.  ACM Transactions on Graphics 36, 1--14.  URL: http://dl.acm.org/citation.cfm?doid=3072959.3073596, doi:10.1145/3072959.3073596.
  * Moeslund and Granum (2001) Moeslund, T.B., Granum, E., 2001\.  A Survey of Computer Vision-Based Human Motion Capture.  Computer Vision and Image Understanding 81, 231--268.  URL: http://www.sciencedirect.com/science/article/pii/S107731420090897X, doi:10.1006/cviu.2000.0897.
  * Moro et al. (2020) Moro, M., Marchesi, G., Odone, F., Casadio, M., 2020\.  Markerless gait analysis in stroke survivors based on computer vision and deep learning: A pilot study, in: Proceedings of the 35th Annual ACM Symposium on Applied Computing, Association for Computing Machinery, New York, NY, USA. p. 2097–2104.  URL: https://doi.org/10.1145/3341105.3373963, doi:10.1145/3341105.3373963.
  * Nath et al. (2018) Nath, T., Mathis, A., Chen, A.C., Patel, A., Bethge, M., Mathis, M.W., 2018\.  Using DeepLabCut for 3D markerless pose estimation across species and behaviors.  preprint. Neuroscience.  URL: http://biorxiv.org/lookup/doi/10.1101/476531, doi:10.1101/476531.
  * Newell et al. (2016) Newell, A., Yang, K., Deng, J., 2016.  Stacked Hourglass Networks for Human Pose Estimation.  arXiv:1603.06937 [cs] URL: http://arxiv.org/abs/1603.06937. arXiv: 1603.06937.
  * (55) Olah, C., .  Understanding LSTM Networks -- colah’s blog.  URL: https://colah.github.io/posts/2015-08-Understanding-LSTMs/.
  * Omran et al. (2018) Omran, M., Lassner, C., Pons-Moll, G., Gehler, P.V., Schiele, B., 2018.  Neural Body Fitting: Unifying Deep Learning and Model-Based Human Pose and Shape Estimation.  arXiv:1808.05942 [cs] URL: http://arxiv.org/abs/1808.05942. arXiv: 1808.05942.
  * Pavlakos et al. (2017) Pavlakos, G., Zhou, X., Derpanis, K.G., Daniilidis, K., 2017\.  Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose.  arXiv:1611.07828 [cs] URL: http://arxiv.org/abs/1611.07828. arXiv: 1611.07828.
  * Pavllo et al. (2019) Pavllo, D., Feichtenhofer, C., Grangier, D., Auli, M., 2019\.  3D Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training, in: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Long Beach, CA, USA. pp. 7745--7754.  URL: https://ieeexplore.ieee.org/document/8954163/, doi:10.1109/CVPR.2019.00794.
  * Qiu et al. (2019) Qiu, H., Wang, C., Wang, J., Wang, N., Zeng, W., 2019\.  Cross View Fusion for 3D Human Pose Estimation.  arXiv:1909.01203 [cs] URL: http://arxiv.org/abs/1909.01203. arXiv: 1909.01203.
  * Redmon et al. (2016) Redmon, J., Divvala, S., Girshick, R., Farhadi, A., 2016\.  You Only Look Once: Unified, Real-Time Object Detection.  arXiv:1506.02640 [cs] URL: http://arxiv.org/abs/1506.02640. arXiv: 1506.02640.
  * Ronneberger et al. (2015) Ronneberger, O., Fischer, P., Brox, T., 2015.  U-Net: Convolutional Networks for Biomedical Image Segmentation.  arXiv:1505.04597 [cs] URL: http://arxiv.org/abs/1505.04597. arXiv: 1505.04597.
  * Sarafinos et al. (2016) Sarafinos, Boteanu, Ionescu, Kakadiaris, 2016\.  (PDF) 3D Human Pose Estimation: A Review of the Literature and Analysis of Covariates.  URL: https://www.researchgate.net/publication/307905073_3D_Human_Pose_Estimation_A_Review_of_the_Literature_and_Analysis_of_Covariates. library Catalog: www.researchgate.net.
  * Sheshadri et al. (2020) Sheshadri, S., Dann, B., Hueser, T., Scherberger, H., 2020\.  3D reconstruction toolbox for behavior tracked with multiple cameras.  Journal of Open Source Software 5, 1849.  URL: https://joss.theoj.org/papers/10.21105/joss.01849, doi:10.21105/joss.01849.
  * Sigal et al. (2010) Sigal, L., Balan, A.O., Black, M.J., 2010.  HumanEva: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human Motion.  International Journal of Computer Vision 87, 4--27.  URL: http://link.springer.com/10.1007/s11263-009-0273-6, doi:10.1007/s11263-009-0273-6.
  * Sun et al. (2019) Sun, K., Xiao, B., Liu, D., Wang, J., 2019.  Deep high-resolution representation learning for human pose estimation.  arXiv:1902.09212.
  * Sun et al. (2017) Sun, X., Shang, J., Liang, S., Wei, Y., 2017.  Compositional Human Pose Regression.  arXiv:1704.00159 [cs] URL: http://arxiv.org/abs/1704.00159. arXiv: 1704.00159.
  * Sun et al. (2018) Sun, X., Xiao, B., Wei, F., Liang, S., Wei, Y., 2018\.  Integral Human Pose Regression.  arXiv:1711.08229 [cs] URL: http://arxiv.org/abs/1711.08229. arXiv: 1711.08229.
  * Tompson et al. (2014) Tompson, J., Jain, A., LeCun, Y., Bregler, C., 2014\.  Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation.  arXiv:1406.2984 [cs] URL: http://arxiv.org/abs/1406.2984. arXiv: 1406.2984.
  * De la Torre et al. (2008) De la Torre, F., Hodgins, J., Bargteil, A., Martin, X., Macey, J., Collado, A., Beltran, P., 2008.  Guide to the Carnegie Mellon University Multimodal Activity (CMU-MMAC) Database.  URL: https://www.ri.cmu.edu/publications/guide-to-the-carnegie-mellon-university-multimodal-activity-cmu-mmac-database/. library Catalog: www.ri.cmu.edu.
  * Toshev and Szegedy (2014) Toshev, A., Szegedy, C., 2014\.  DeepPose: Human Pose Estimation via Deep Neural Networks.  2014 IEEE Conference on Computer Vision and Pattern Recognition , 1653--1660URL: http://arxiv.org/abs/1312.4659, doi:10.1109/CVPR.2014.214. arXiv: 1312.4659.
  * Trumble et al. (2017) Trumble, M., Gilbert, A., Malleson, C., Hilton, A., Collomosse, J., 2017.  Total Capture: 3D Human Pose Estimation Fusing Video and Inertial Sensors, in: Procedings of the British Machine Vision Conference 2017, British Machine Vision Association, London, UK. p. 14.  URL: http://www.bmva.org/bmvc/2017/papers/paper014/index.html, doi:10.5244/C.31.14.
  * (72) University, C.M., .  Graphic lab motion capture database.  http://mocap.cs.cmu.edu/.  URL: http://mocap.cs.cmu.edu/. accessed: 2020-04-29.
  * Wandt and Rosenhahn (2019) Wandt, B., Rosenhahn, B., 2019\.  RepNet: Weakly Supervised Training of an Adversarial Reprojection Network for 3D Human Pose Estimation.  arXiv:1902.09868 [cs] URL: http://arxiv.org/abs/1902.09868. arXiv: 1902.09868.
  * Wang et al. (2020) Wang, J., Yan, S., Xiong, Y., Lin, D., 2020.  Motion Guided 3D Pose Estimation from Videos, in: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.M. (Eds.), Computer Vision – ECCV 2020. Springer International Publishing, Cham. volume 12358, pp. 764--780.  URL: http://link.springer.com/10.1007/978-3-030-58601-0_45, doi:10.1007/978-3-030-58601-0_45. series Title: Lecture Notes in Computer Science.
  * Xiao et al. (2018) Xiao, B., Wu, H., Wei, Y., 2018.  Simple baselines for human pose estimation and tracking.  arXiv:1804.06208.
  * Xu et al. (2019) Xu, Y., Zhu, S.C., Tung, T., 2019.  DenseRaC: Joint 3D Pose and Shape Estimation by Dense Render-and-Compare.  arXiv:1910.00116 [cs, eess] URL: http://arxiv.org/abs/1910.00116. arXiv: 1910.00116.
  * Yu et al. (2017) Yu, T., Guo, K., Xu, F., Dong, Y., Su, Z., Zhao, J., Li, J., Dai, Q., Liu, Y., 2017.  BodyFusion: Real-Time Capture of Human Motion and Surface Geometry Using a Single Depth Camera, in: 2017 IEEE International Conference on Computer Vision (ICCV), IEEE, Venice. pp. 910--919.  URL: http://ieeexplore.ieee.org/document/8237366/, doi:10.1109/ICCV.2017.104.
  * Yu et al. (2018) Yu, T., Zheng, Z., Guo, K., Zhao, J., Dai, Q., Li, H., Pons-Moll, G., Liu, Y., 2018.  DoubleFusion: Real-time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensor.  arXiv:1804.06023 [cs] URL: http://arxiv.org/abs/1804.06023. arXiv: 1804.06023.
  * Zhang and Park (2020) Zhang, Y., Park, H.S., 2020\.  Multiview Supervision By Registration, pp. 420--428.  URL: http://openaccess.thecvf.com/content_WACV_2020/html/Zhang_Multiview_Supervision_By_Registration_WACV_2020_paper.html.
  * Zhang et al. (2020) Zhang, Z., Wang, C., Qin, W., Zeng, W., 2020.  Fusing wearable imus with multi-view images for human pose estimation: A geometric approach.  arXiv:2003.11163.
  * Zhou et al. (2017) Zhou, X., Huang, Q., Sun, X., Xue, X., Wei, Y., 2017\.  Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach.  arXiv:1704.02447 [cs] URL: http://arxiv.org/abs/1704.02447. arXiv: 1704.02447.