# Applications of Information Theory:
statistics and statistical mechanics

Khizar Qureshi Prof. Peter Shor provided generous amounts of feedback
Department of Mathematics
Massachusetts Institute of Technology
18.424: Seminar in Information Theory

###### Abstract

The method of optimizing entropy is used to (i) conduct Asymptotic Hypothesis
Testing and (ii) determine the energy distribution for which Entropy is
maximized. This paper focuses on two related applications of information
theory: Statistics and Statistical Mechanics.

## Introduction

Entropy is one measure of uncertainty within a system, and is often used to
describe the disorder of sequences of quantized random variables. However,
entropy can also be extended to methods within optimization, in which the
disorder of a system of interest may be maximized or minimized. Such methods
are prevalent within statistics, the physical sciences, and econometrics.

#####

The concerns of a statistician observing a sequence of outcomes include the
validity of an explanatory hypothesis, its degree of significance, and any
assumptions underlying the statistical tests. While linear hypothesis testing
is often sufficient, larger sequences exhibit large deviations in behavior
that should receive separate treatment. Traditional linear hypothesis testing
trivially assigns a constant multiple of an explanatory parameter $\beta$ to
an observation when forming a hypothesis ($H_{0}=k\beta=0$). Optimal entropy,
in which disorder is locally minimized or maximized, can be used to construct
asymptotic, non-linear hypothesis tests. Unlike linear hypothesis testing,
error probability can be minimized.

Optimizing entropy extends to thermodynamic systems. The Third Law of
Thermodynamics states that the entropy of a closed system, i.e. one in which
no mass or energy is added or removed, must be bounded from below by zero.
Achieving a non-entropic system is nearly impossible, except within a perfect
crystal lattice. A more probable state is one for which the entropy of a
system is maximized, and observations follow a Boltzmann distribution.

In our study of optimal entropy, we will use classical Statistics and
Statistical Mechanics as a lens. We will demonstrate the concept of minimal
entropy through two statistical tests: the univariate optimality test defined
by Stein’s Lemma, and a multivariate optimality test, as defined by the
Chernoff Bounds. We will see that there exists a distribution, the Boltzmann
distribution, that approaches maximum entropy as temperature goes to infinity.
Finally, we will apply the concept of asymptotic hypothesis testing to
Statistical Mechanics. In particular, we will test and observe the evolution
of error probability with a growing sample size. We will also compare
Q-function error probability with that of the Chernoff bound. The remainder of
the paper is organized as follows. To better understand the atypicality of
sequences, we will study the method of types. We will then learn how such
sequences behave through the large deviation theory. The focus of the paper
will then shift to hypothesis testing, in which we will develop tools for
recognizing the asymptotic optimality of entropy. Illustrative examples of
this will include Stein’s Lemma and Chernoff bounds. For the interest of the
physical sciences, we will rigorously derive the Boltzmann distribution, for
which entropy is nearly maximized. Finally, we will converge the
aforementioned topics through simulations of asymptotic statistical testing.

## Hypothesis Testing

Statisticians are often concerned with not just observed data, but the several
possible underlying explanations. A few examples include: testing for the
effectiveness of a drug, determining whether or not a coin is biased, and the
effect of gender on wage growth. We begin with a simple case in which we
decide between two hypothesis, each of which is represented by an independent
and identical distribution, or i.i.d. Let $X_{1},X_{2},\ldots,X_{n}$ be i.i.d.
$\sim Q(x)$. For an observed outcome, we have two possible explanations:

  * •

$H_{1}:Q=P_{1}$

  * •

$H_{2}:Q=P_{2}$

We now define a general decision function, whose value reflects the acceptance
and rejection of the above hypothesis. Namely, for general decision
$g(x_{1},x_{2},\ldots,x_{n})$, $g(x)=i$ indicates that $H_{i}$ is accepted. In
the binary case, the set $A$ over which $g(x)=i$ is complemented by the set
$A^{c}$, over which $g(x)\neq i$.

#####

Quite often, statisticians are concerned with accepting incorrect hypotheses
and rejecting correct ones. Such occurrences, recognized as Type I/II errors,
often occur when sequences exhibit atypicality and large deviating behavior
(See appendix). Error probabilities are reflected through the decision
function using weights $\alpha,\beta$:

$\begin{split}\alpha=P(g(x)=2|H_{1}\text{ true})=P^{n}_{1}(A^{c})\\\
\beta=P(g(x)=1|H_{2}\text{ true})=P^{n}_{1}(A)\end{split}$ (1)

Notice that the general decision function takes on values contradicting those
implied by the conditional hypothesis. The first implies that $H_{2}$ was
accepted even though $H_{1}$ was true, and the second implies that $H_{1}$ was
accepted even though $H_{2}$ was true. Type I (reject true) and Type II
(accept false) errors similarly prove detrimental to experiments, and so we
wish to minimize probabilities $\alpha$ and $\beta$. Minimizing $\alpha$
increases $\beta$, and minimizing $\beta$ increases $\alpha$. We will now
explore methodology to minimize the overall probability of error by optimizing
entropy as a weighted sum of $\alpha$ and $\beta$.

### Stein’s Lemma

We first fix either $\alpha$ or $\beta$, and manipulate the other to minimize
the probability of error.

######  Theorem 1 (Stein’s Lemma).

Let $X_{1},X_{2},\ldots,X_{n}$ be i.i.d. $\sim Q$. Further, let
$D(P_{1}\|P_{2})$ represent the Kullback-Leibler distance, or relative entropy
between the probability densities. Consider the hypothesis test between two
alternatives $Q=P_{1}$ and $Q=P_{2}$ where $D(P_{1}\|P_{2})<\infty$. Let
$A_{n}\subseteq\mathcal{H}^{n}$ be an acceptance region for hypothesis 1. Let
the probabilities of error be

$\begin{split}\alpha_{n}=P^{n}_{1}(A^{c}_{n})\\\
\beta_{n}=P^{n}_{2}(A_{n})\end{split}$ (2)

and for $0<\epsilon<\frac{1}{2}$, define

$\beta^{\epsilon}_{n}=\text{min}_{A_{n}\subseteq X^{n}}\beta_{n}$ (3)

Then,

$\text{lim}_{\epsilon\rightarrow 0}\
\text{lim}_{n\rightarrow\infty}\frac{1}{n}\text{log}\beta^{\epsilon}_{n}=-D\left(P_{1}\|P_{2}\right)$
(4)

###### Proof.

See Appendix ∎

Thus, no sequence of sets $B_{n}$ has an exponent better than
$D\left(P_{1}\|P_{2}\right)$. But the sequence $A_{n}$ achieves the exponent
$D\left(P_{1}\|P_{2}\right)$. Thus $A_{n}$ is asymptotically optimal, and the
best error exponent is $D\left(P_{1}\|P_{2}\right)$.

### Chernoff Bound

Thus far, $\alpha$ and $\beta$ have been treated separately. The approach
underlying Stein’s Lemma was to set one error probability to be
infinitesimally small, and measure the effect on the resulting probability. We
saw that setting $\alpha\leq\epsilon$ achieved $\beta_{n}=2^{-nD}$. However,
the distribution of error amongst $\alpha$ and $\beta$ may be highly
asymmetrical, in which case univariate optimization may not suffice. We now
explore methodology for a bivariate optimization.

#####

An alternative approach is to minimize the weighted sum of $\alpha$ and
$\beta$. The resulting error exponent is known as the Chernoff Information.
Consider a distribution of i.i.d. random variables: $X_{1},X_{2},\ldots,X_{n}$
representative of the decision function. We assign $P_{1}$ to Q with
probability $\pi_{1}$ and $P_{2}$ to Q with probability $\pi_{2}$. Upholding
the definition of $\alpha$ and $\beta$, the overall probability of error is

$P^{n}_{\epsilon}=\pi_{1}\alpha_{n}+\pi_{2}\beta_{n}$ (5)

######  Theorem 2 (Chernoff).

The best achievable exponent in the Bayesian probability of error is $D^{*}$,
where

$D^{*}=\\\
text{lim}_{n\rightarrow\infty}\text{min}_{A_{n}\subseteq\mathcal{H}^{n}}-\frac{1}{n}\text{log}P^{n}_{\epsilon}=D\left(P_{\lambda^{*}}\|P_{1}\right)=D\left(P_{\lambda^{*}}\|P_{2}\right)$
(6)

where

$P_{\lambda}=\frac{P^{\lambda}_{1}(x)P^{1-\lambda}_{2}(x)}{\sum_{a\in
X}P^{\lambda}_{1}(a)P^{1-\lambda}_{2}(a)}$ (7)

and $\lambda^{*}$ the value of $\lambda$ such that

$D\left(P_{\lambda^{*}}\|P_{1}\right)=D\left(P_{\lambda^{*}}\|P_{2}\right).$
(8)

###### Proof.

See Appendix ∎

## Physical Chemistry

Claude Shannon first proposed that the uncertainty due to possible errors in a
message could be encapsulated by

$U(W)=\text{log}W$ (9)

where W is the number of possible ways (state space) of encoding random
information. Intuitively, the uncertainty increases with increasing W, and is
zero if W=1. The concept of entropy provides a deep-rooted link between
information theory and statistical mechanics. The state with the least
information available, or greatest entropy occurs when the set of all states
are equiprobable. This is also the state with maximum uncertainty. An
information theoretic perspective dictates that explicit knowledge of various
probabilities associated with the system constitutes greater information.
Similarly, the thermodynamics of a system of isolated particles indicate that
entropy is directly correlated with expected energy level. Below is a
molecular orbital diagram that illustrates the possible energy states, all of
which depend on the position an electron occupies.

Figure 1: The figure above is a molecular orbital diagram. When two atoms
interact, and possibly share their valence (outermost) electrons, the
electrons must occupy a particular orbital. Once occupying an orbital, the
atoms are able to form a bond. Here, we are not concerned with the type of the
bond, but rather, paired occupancy. The empty orbitals are indistinguishable,
and will be occupied with equal probability.

#####

Entropy may also be observed in macroscopic states. The second law of
thermodynamics states that in equilibrium, changes in entropy are proportional
to changes in system heat per unit temperature. We have
$dW=\frac{dQ_{sys}}{T}$. We can better understand this through an
illustration. Consider a system of gas particles that may be expanded or
compressed. We can study the system under various entropic regimes. The
diagram below illustrates how available work decreases (increases) for gaseous
expansion (compression) under bivariate states of pressure and volume.

Figure 2: The figure above is a pressure-volume diagram for a system of Argon
gas particles. Expanding or compressing a gas requires energy, the extent of
which depends on the state of the system. Notice that for an adiabatic system
$\left(dQ_{sys}=0\right)$, compressing the gas requires the least relative
energy. That is, when when the change in entropy in minimized, a system can be
most naturally expanded/compressed. To reach minimum work available, we move
down the gradient of steepest descent until entropy is globally minimized.

#####

Consider a perfectly structured crystal lattice structure, in which the
positions of each contributing molecule is fixed. If we observe such a system,
depart, and return after $n$ periods, the position of each molecule within the
crystal will have remained the same almost surely. If the particles did not
displace, then they also carried zero kinetic energy, which is representative
of a zero temperature system. This near certainty of a thermodynamic system is
an example of an optimization in which entropy is minimized. If instead, the
system consisted of a fair coin toss, with no extra information, entropy would
be maximized.

### The Boltzmann Distribution

The distribution that maximizes the state space for a fixed energy level is
the Boltzmann distribution. We will now derive such a distribution, and show
that it uniquely maximizes entropy on each energy level. Consider a crystal
containing $N$particles, each of which has available energy levels,
$\epsilon_{n}$. The state space, W, is the number of ways the total energy
$E=\sum_{n}N_{n}\epsilon_{n}$ can be distributed amongst the the particles in
each energy level, across all energy levels, $N=\sum_{n}N_{n}$. The expected
number of particles in each energy level, $N_{n}$, is the product of the
probability that a particle is at an energy level, $P_{n}$, and the total
number of particles, N. The only consideration for such a distribution is the
number of particles in each energy levels, not necessarily the amount of
energy allocated to each particle. Energy is conserved amongst states and
across energy levels. While there exist several ways of assigning the number
of particles in each energy level $\epsilon_{n}$, we wish to find the state
with the distribution achievable in the most number of ways for fixed energy
levels. Our first constraint is that the total state space W is the sum of
individual states occupied, $W_{i}$ for all possible distributions

$W=\sum W_{i}$ (10)

Amongst all possible distributions of particles, there exists one that can be
achieved in more ways than any other. A distribution that approaches maximum
entropy for fixed energy levels is the Boltzmann distribution.

$\text{log}W\cong\text{log}W_{B}$ (11)

To find the most probable distribution that maximizes W, we first note that
each particle in the crystal can be distinguished from the others because it
occupies a defined position in space. Therefore, such a setting allows us to
number the particles $1,2,\ldots,N$. We assume a large N, to maintain
consistency with typical non-deficient states. S A particular microstate of
the crystal will place particle 1 in energy level $\epsilon_{i}$, particle 2
in energy level $\epsilon_{j}$, and so on. Initially, we seek the number of
$W_{i}$ microstates in a distribution for which there are $N_{1}$ particles in
$\epsilon_{1}$, $N_{2}$ particles in $\epsilon_{2},$, and so on. We choose, at
random, particles from the crystal, $N_{i}$, and assign them to energy levels,
$\epsilon_{i}$. The number of ways this can be done is equal to the number of
different orders in which the particles can be chosen from the crystal. The
first particle can be chosen from a group of N. With $N-1$ particles
remaining, the second can be chosen in $N-1$ ways. We see that the number of
ways for selecting the first two particles is $N(N-1)$. Following this
procedure, we then we see that the number of ways for selecting the $N$
particles is $N(N-1)(N-2)(N-3)\dots(3)(2)(1)$, or $N!$.

#####

We have over counted the ways of achieving a given distribution, and have
assumed that all states are distinguishable. Consider the placement of the
first two particles into energy level $\epsilon_{1}$. It makes no difference
whether the first particle is placed into $\epsilon_{1}$ prior to or following
particle 2. That is, the states are indistinguishable. This relaxes the
strictness on order, and so permutation are ignored. Thus, the state space,
$W_{i}$, for a given distribution, is $N!$ divided by the product of all $N!$

$W_{i}=\frac{N!}{\prod_{n}N_{n}!}$ (12)

To find the distribution that maximizes $W_{i}$, we note Stirling’s
approximation for log N!

$\text{log}\ N!\approx N\ \text{log}\ N-N$ (13)

Finding the maximum of $W_{i}$ is equivalent to finding the maximum of
$\text{log}\ W_{i}$, so we combine equations (13) and (14), and re-arrange as
follows

$\text{log}\ W_{i}\approx\text{log}\ N!-\sum_{n}\text{log}N_{n}!=N\
\text{log}\ N-\sum_{n}N_{n}\ \text{log}N_{n}=\left(\sum_{n}N_{n}\right)\
\text{log}\sum_{n}N_{n}-\sum_{n}N_{n}\ \text{log}N_{n}$ (14)

However, the set of particles is conserved. Moreover, the net energy within
the system is conserved. This provides the following two constraints

$\begin{split}\sum_{j}N_{j}=N\\\ \sum_{j}\epsilon_{j}N_{j}=E\end{split}$ (15)

We now use Lagrange’s method of undetermined multipliers. When $W_{i}$ is
maximized, its differential log must be zero

$d\ \text{log}\ W_{i}=\sum_{j}\frac{\partial\text{log}W_{i}}{\partial
N_{j}}_{j}dN_{j}=0$ (16)

We multiply the constraints on particle count and energy by constants
$\alpha,\beta$, and then take the differential to obtain

$\begin{split}\alpha\sum_{j}dN_{j}=0\\\
\beta\sum_{j}\epsilon_{j}dN_{j}=0\end{split}$ (17)

Subtracting these two constraints from the log-entropy, we obtain

$\sum_{j}\left(\frac{\partial\text{log}W_{i}}{\partial
N_{j}}-\alpha-\beta\epsilon_{j}\right)dN_{j}=0$ (18)

Through use of log-properties and algebraic manipulation (See Appendix), the
expression above is reduced to

$\text{log}N_{j}=\text{log}N-\alpha-\beta\epsilon_{j}$ (19)

which, after exponentiating both sides is

$N_{j}=Ne^{-\alpha}e^{-\beta\epsilon_{j}}$ (20)

The significance of this result is that it shows the occupancy of en energy
state $\epsilon_{j}$ is proportional to $e^{-\beta\epsilon_{j}}$.

#### Key Result

To account for energy states, thermodynamicists often make use of temperature,
an intrinsic quantity. Temperature is equivalent to the average kinetic energy
of a system of particles. Because this varies across systems, we normalize.
For a temperature T, and the Boltzmann constant, $k_{B}$,

$\beta=\frac{1}{k_{B}T}.$ (21)

Inducting on the one particle case, in which,
$e^{\alpha}=\sum_{j}e^{\frac{-\epsilon_{j}}{k_{B}T}}$, and combining the
preceding three expressions, providing the desired result

$\begin{split}P_{n}=\frac{N_{n}}{N}=\frac{e^{z}}{\sum_{N}e^{z}}\\\ \text{where
}z=\frac{-\epsilon_{j}}{k_{B}T}\end{split}$ (22)

We have now found the Boltzmann distribution, for which entropy is maximized.
The Boltzmann probability above expresses the fraction of particles placed in
each quantum state $n$ to maximize entropy $W_{i}$ of the distribution over
each energy level, $\epsilon_{j}$.

## Simulation: Asymptotic Hypothesis Testing

Thus far, we have studied various methods of statistical testing, highlighting
the importance of asymptotic tests such as the Chernoff Information Bound. We
have also (briefly) explored Statistical Mechanics, in which we show that
entropy is maximized for a Boltzmann distribution. We now demonstrate the
importance of our learnings through a representative example.

#####

Robust methods of signal interpretation allow for communication, and involve
the separation of signal and noise. A simple signal will follow a Gaussian
distribution, for which entropy is maximized. A hypothesis consists of
assigning observations as either signals, or as noise. Such hypotheses carry
error probabilities, and should be studied with both linear testing, as well
as asymptotic testing.

### Example: Binary Detection

The classical binary detection problem involves the reception of finite-length
signals realized as a random process $r[n],n=1,2,\ldots,$ [3]. The signal can
be attributed to either Gaussian white noise $n[i]$ or a deterministic signal
$s[i]$. Basic studies involve the interpretation of the signal-to-noise ratio,
a measure of quality. Consider a binary detection problem:

$r=s_{i}+n,,\ i\in\\{1,2\\}$ (23)

  * •

Detections are composed of signals and noise

  * •

n: N-dimensional noise vector

  * •

i.i.d. Gaussian random variables and $\sim\left(0,1\right)$

  * •

$s_{1}=\left(m,m,\ldots,m\right)$

  * •

$s_{2}=\left(0,0,\ldots 0\right)$

Evaluate the error probability for both $N=1$ and $N=4$ when
$m\in\\{1,2,3,4,5,6\\}$.

Traditionally, error probability is evaluated through the Q-function, which
represents the probability that a normal random variable will obtain a value
larger than $x$ standard deviations above the mean. It can also be thought of
as the "tail" probability of the standard normal distribution, and is useful
for linear hypothesis testing.

$Q(x)=\frac{1}{\sqrt{2\pi}}\int_{x}^{\infty}exp\left(\frac{-t^{2}}{2}\right)dt=\frac{1}{2}erf\left(\frac{x}{\sqrt{2}}\right)$
(24)

Given a signal-to-noise Ratio, the Q-function can be used to determine the
error probability

$P^{e}=Q\left(\frac{\rho}{\sigma}\right)\text{where }\
\frac{\rho^{2}}{\sigma^{2}}=\frac{Nm^{2}}{4}$ (25)

We also know that any error probability is bounded from above by the Chernoff
Information bound. For the Q-function,

$Q(x)\leq exp\left(-\frac{x^{2}}{2}\right)\text{erf}$ (26)

And so,

$P^{e}=Q\left(\frac{\sqrt{N}m}{2}\right)\leq
exp\left(-\frac{m^{2}N}{8}\right)$ (27)

The figure below illustrates the growth of error in both forms of testing.

Figure 3: The figure above shows the evolution of error probability with an
increasing signal length (m). Error probability decreases as: (i) the number
of bits in the signal increases, and (ii) the number of elements in the noise
vector increases.

## Concluding Remarks

Optimizing entropy demonstrates the applicability of information theory beyond
computing. Asymptotic testing captures error probability in atypical
sequences, and a Boltzmann distribution of particles approaches maximum
entropy, as temperature goes to infinity.

## Appendix

Motivation for asymptotic testing arises from atypicality and large deviations
in sequences. We briefly review this, and encourage the ambitious reader to
study further.

### The Method of Types

The Asymptotic Equipartition Property formalizes that although there exist
several possible outcomes of a stochastic process, there exists a set from
which sequences are typical , or most frequently observed. The centric
approach underlying the AEP involves defining an almost sure convergence in
probability between the expectation of a sequence to its entropy. Similarly,
the Method of Types defines strong bounds on the number of sequences of a
particular distribution, as well as the probability of each such sequence
being observed.

## Large Deviation Theory

Recall that type of a sequence $x^{n}_{i}\in A^{n}$ is representative of its
empirical distribution $\hat{P}=\hat{P}_{x^{n}_{i}}$ where:

$\hat{P}(a)=\frac{|\\{i:x_{i}=a\\}|}{n},a\in A.$ (28)

A distribution P on A is called an n-type if it is the type of some
$x^{n}_{1}\in A^{n}$. The set of all $x^{n}_{1}\in A^{n}$ of type P is called
the type class of the n-type P and is denoted by $\mathcal{T}^{n}_{p}$.

######  Lemma 3.

The number of possible n-types is

$\binom{n+|A|-1}{|A|-1}$ (29)

###### Proof.

$\mathcal{T}(P)=\\{x\in\mathcal{X}^{n}:P_{x}=P\\}$ (30)

The combinatoric cardinality of $\mathcal{T}(P)$ provides the result ∎

######  Lemma 4.

For any n-type P,

$\binom{n+|A|-1}{|A|-1}^{-1}2^{nH\left(P\right)}\leq|\mathcal{T}^{n}_{p}|\leq
2^{nH\left(P\right)}$ (31)

###### Proof.

First, we prove the upper bound using $P(\mathcal{T}(P)\leq 1$.

$1\geq P^{n}(\mathcal{T}(P))=\sum_{x\ inT(P)}P^{n}(x)=\sum_{x\
inT(P)}2^{-nH(P)}=\|T(P)\|2^{-nH(P)}$ (32)

Consequently, $\|T(P)\|\leq 2^{nH(P)}$. For the lower bound, using the fact
that $T(P)$ has the highest probability amongst all type classes in P, we can
bound the ratio of probabilities

$\frac{P^{n}(T(P))}{P^{n}(T(\hat{P}))}\\\
=\frac{\|T(P)\|\prod_{a\in\mathcal{X}P(a)^{nP(a)}}}{\|T(\hat{P})\|\prod_{a\in\mathcal{X}P(a)^{n\hat{P}(a)}}}=\prod_{a\in\mathcal{X}}\frac{(n\hat{P}(a))!}{(n{P}(a))!}P(a)^{n(P(a)-\hat{P}(a))}\\\
$ (33)

Using the identity $\frac{m!}{n!}\geq n^{m-n}$, we see

$\frac{P^{n}(\mathcal{T}(P))}{P^{n}(\mathcal{T}(\hat{P}))}\geq\prod_{a\in\mathcal{X}}n^{n(P(a)-\hat{P}(a))}=n^{n(1-1)}=1$
(34)

So $P^{n}(T(P))\geq P^{n}(T(\hat{P}))$. The lower bound can now be found as

$\displaystyle
1=\sum_{Q\in\mathcal{P}_{n}}P^{n}(T(Q))\leq\sum_{Q\in\mathcal{P}_{n}}$ (35)
$\displaystyle=max_{Q}P^{n}(T(Q))$
$\displaystyle=\sum_{Q\in\mathcal{P}_{n}}P^{n}(T(P))$
$\displaystyle\leq(n+1)^{\|\mathcal{X}\|}P^{n}(T(P))=(n+1)^{\|\mathcal{X}\|}\sum_{x\in
T(P)}P^{n}(x)$ $\displaystyle=(n+1)^{\|\mathcal{X}\|}\sum_{x\in
T(P)}2^{-nH(P)}$ $\displaystyle=(n+1)^{\|\mathcal{X}\|}\|T(P)\|2^{-nH(P)}$

∎

To connect the theory of types with general probability theory, we must
develop a sense of relative entropy. For any distribution P on A, let $P^{n}$
denote the distribution of n independent drawings from P, that is,
$P^{n}\left(x^{n}_{1}\right)=\prod_{i=1}^{n}P\left(x_{i}\right),x^{n}_{1}\in
A^{n}$.

######  Lemma 5.

For any distribution P on A and any n-type Q

$\begin{split}\frac{P^{n}\left(x^{n}_{1}\right)}{Q^{n}\left(x^{n}_{1}\right)}=2^{-nD\left(Q\|P\right)},ifx^{n}_{1}\in\mathcal{T}^{n}_{Q}\\\
\binom{n+|A|-1}{|A|-1}^{-1}2^{-nD\left(Q\|P\right)}\leq
P\left(\mathcal{T}^{n}_{p}\right)\leq 2^{-nD\left(Q\|P\right)}\end{split}$
(36)

###### Proof.

For probability $P\in P_{n}$, distribution $Q$, the probability of type class
$T(P)$ under $Q^{n}$ is $2^{-nD(P\|Q)}$. We see

$\displaystyle Q^{n}(T(P))=\sum_{x\in T(P)}Q^{n}(x)$ (37)
$\displaystyle=\sum_{x\in T(P)}2^{-n(D(P\|Q)+H(P))}$
$\displaystyle=\|T(P)\|2^{-n(D(P\|Q)+H(P))}$

Replacing $\|T(P)\|$ with the result from Lemma 3, we see the result. ∎

######  Corollary 6.

Let $\hat{P}_{n}$ denote the empirical distribution (type) of a random sample
of size n drawn from P. Then

$P\left(D\left(\hat{P}_{n}\|P\right)\geq\delta\right)\leq\binom{n+|A|-1}{|A|-1}2^{-n\delta},\forall\delta>0$
(38)

###### Proof.

Given an $\epsilon>0$, we can define a typical set
$\mathcal{T}^{\epsilon}_{Q}$ of sequences for the distribution Q as
$\mathcal{T}^{\epsilon}_{Q}=\\{x^{n}:D(P_{x^{n}}\|Q)\leq\epsilon\\}$. Then the
probability of an atypical sequence is

$\begin{split}1-Q^{n}(\mathcal{T}^{\epsilon}_{Q})=\sum_{P:D(P\|Q)\geq\epsilon}Q^{n}(T(P))\\\
\leq\sum_{P:D(P\|Q)\geq\epsilon}2^{-nD(P\|Q)}\\\
\leq\sum_{P:D(P\|Q)\geq\epsilon}2^{-n\delta}\\\
\leq\binom{n+|A|-1}{|A|-1}2^{-n\delta},\forall\delta\geq 0\end{split}$ (39)

∎

######  Theorem 7.

Sanov’s Theorem Let $\Pi$ be a set of distributions on A whose closure is
equal to the closure of its interior. Then for the empirical distribution of a
sample from a strictly positive distribution P on A,

$-\frac{1}{n}\text{\text{log}}P\left(\hat{P}\in\Pi\right)\rightarrow
D\left(\Pi\|P\right)$ (40)

###### Proof.

Sanov’s Theorem Let $\mathcal{P}_{n}$ be the set of possible n-types and let
$\Pi_{n}=\Pi\cap\mathcal{P}_{n}$. The previous lemma implies that

$\begin{split}Prob\left(\hat{P}_{n}\in\Pi_{n}\right)=P^{n}\left(\cup_{Q\in\Pi_{n}}\mathcal{T}^{n}_{Q}\right)\
\text{is upper bounded by}\\\
\binom{n+|A|-1}{|A|-1}2^{-nD\left(\Pi_{n}\|P\right)}\ \text{and lower bounded
by}\\\ \binom{n+|A|-1}{|A|-1}^{-1}2^{-nD\left(\Pi_{n}\|P\right)}\end{split}$
(41)

Since $D\left(Q\|P\right)$ is continuous in Q, the hypothesis on $\Pi$ implies
that $D\left(\Pi_{n}\|P\right)$ is arbitrarily close to $D\left(\Pi\|P\right)$
if n is large. ∎

### Proofs

###### Stein’s Lemma.

To prove the theorem, we construct a sequence of acceptance regions
$A_{n}\subseteq X^{n}$ such that $A_{n}<\epsilon$ and
$\beta_{n}=2^{-nD(P_{1}\|P_{2})}$. We then show that no other sequence of
tests has an asymptotically better exponent.
First, we define

$A_{n}=\left\\{x\in
X^{n}:2^{+n(D(P_{1}\|P_{2})^{-\delta})}\leq\frac{P_{1}x}{P_{2}x}\leq
2^{+n(D(P_{1}\|P_{2})^{+\delta}}\right\\}$ (42)

Then, we have the following properties:

  1. 1.

$P^{n}_{1}(A_{n})\rightarrow 1$. This follows from:

$P^{n}_{1}(A_{n})=P^{n}_{1}\left(\frac{1}{n}\sum_{i=1}^{n}\text{log}\frac{P_{1}(X_{i})}{P_{2}(X_{i})}\in\left(D(P_{1}\|P_{2}\right)-\delta_{1}D\left(P_{1}\|P_{2}\right)+\delta\right)$
(43)

by the strong LLN, since
$D\left(P_{1}\|P_{2}\right)=E_{P_{1}}\left(\text{log}\frac{P_{1}(X)}{P_{2}(X)}\right)$.
Hence, for sufficiently large n, $A_{n}<\epsilon$.

  2. 2.

$P^{n}_{2}(A_{n})\leq^{+n(D(P_{1}\|P_{2})^{-\delta})}$. Using the definition
of $A_{n}$, we have

$\begin{split}P^{n}_{2}(A_{n})=\sum_{A_{n}}P_{2}(x)\leq\sum_{A_{n}}P_{1}(x)2^{+n(D(P_{1}\|P_{2})^{-\delta})}\\\
=2^{+n(D(P_{1}\|P_{2})^{-\delta})}\sum_{A_{n}}P_{1}(x)\\\
=2^{+n(D(P_{1}\|P_{2})^{-\delta})}(1-\alpha_{n}).\end{split}$ (44)

Similarly, $P^{n}_{2}(A_{n})\geq
2^{+n(D(P_{1}\|P_{2})^{+\delta})}(1-\alpha_{n})$. And so,
$\frac{1}{n}\text{log}\beta_{n}\leq-D\left(P_{1}\|P_{2}\right)+\delta+\frac{\text{log}(1-\alpha_{n})}{n}$,
and
$\frac{1}{n}\text{log}\beta_{n}\geq-D\left(P_{1}\|P_{2}\right)-\delta+\frac{\text{log}(1-\alpha_{n})}{n}$,
hence $\
\text{lim}_{n\rightarrow\infty}\frac{1}{n}\text{\text{log}}\beta_{n}=-D\left(P_{1}\|P_{2}\right).$

∎

###### Chernoff Bound.

The optimum hypothesis test is a likelihood ratio test, which follows the
form:

$D\left(P_{X^{n}}\|P_{2}\right)-D\left(P_{X^{n}}\|P_{1}\right)>T$ (45)

The test divides the probability simplex into regions corresponding to
hypothesis 1 and hypothesis 2, respectively. This is illustrated below:

Figure 4: The figure above shows the probability simplex and the Chernoff
bound. Notice that for error probabilities $P_{1}$ and $P_{2}$, there exists
an optimal error probability $P_{\lambda}$. This is determined as a weighted
argmin.

Let A be the set of types associated with hypothesis 1. From the preceding
discussions, it follows that the closest point in the set $A^{c}$ to $P_{1}$
is on the boundary of A, and is of the form given by $[8]$. Then, it is clear
that $P_{\lambda}$ is the distribution in A that is closest to $P_{2}$. It is
also the distribution in $A^{c}$ that is closest to $P_{1}$. By Sanov’s
theorem, we can calculate the associated probabilities of error:

$\begin{split}\alpha_{n}=P^{n}_{1}(A^{c})=2^{-nD\left(P_{\lambda^{*}}\|P_{1}\right)}\\\
\beta_{n}=P^{n}_{2}(A)=2^{-nD\left(P_{\lambda^{*}}\|P_{2}\right)}\end{split}$
(46)

In the Bayesian case, the overall probability of error is the weighted sum of
the two probabilities of error,

$P_{e}=\pi_{1}2^{-nD\left(P_{\lambda^{*}}\|P_{1}\right)}+\pi_{2}2^{-nD\left(P_{\lambda^{*}}\|P_{2}\right)}=2^{-n\
\text{min}\\{{-D\left(P_{\lambda^{*}}\|P_{1}\right),-D\left(P_{\lambda^{*}}\|P_{2}\right)\\}}}$
(47)

since the exponential rate is determined by the worst exponent. Since
$D\left(P_{\lambda}\|P_{1}\right)$ increases with $\lambda$ and
$D\left(P_{\lambda}\|P_{2}\right)$ decreases with $\lambda$, the maximum value
of the minimum of
$\\{D\left(P_{\lambda}\|P_{1}\right),D\left(P_{\lambda}\|P_{2}\right)\\}$ is
attained when they are equal. We choose $\lambda$ so that

$D\left(P_{\lambda}\|P_{1}\right)=D\left(P_{\lambda}\|P_{2}\right)=C(P_{1},P_{2})$
(48)

Thus $C\left(P_{1},P_{2}\right)$ is the highest achievable exponent for the
probability of error, and is called the Chernoff information.

The closest point in the set $A^{c}$ to $P_{1}$ is on the boundary of A, and
is of the form given by []. Then from the previous discussion, it is clear
that $P_{\lambda}$ is the distribution in A that is closest to $P_{2}$; it is
also the distribution in $A^{c}$ that is closest to $P_{1}$. By Sanov’s
theorem, we can calculate the associated probabilities of error:

$\begin{split}\alpha_{n}=P^{n}_{1}\left(A^{c}\right)=2^{-nD\left(P_{\lambda^{*}}\|P_{1}\right)}\\\
\beta_{n}=P^{n}_{2}\left(A^{c}\right)=2^{-nD\left(P_{\lambda^{*}}\|P_{2}\right)}\end{split}$
(49)

In the Bayesian case, the overall probability of error is the weighted sum of
the individual two probabilities of error,

$P_{e}=\pi_{1}2^{-nD\left(P_{\lambda}\|P_{1}\right)}+\pi_{2}2^{-nD\left(P_{\lambda}\|P_{2}\right)}=2^{-n\
\text{min}\left(D(P_{\lambda}\|P_{1}\right),D\left(P_{\lambda}\|P_{2}\right)}$
(50)

since the exponential rate is determined by the worst exponent. Since
$D\left(P_{\lambda}\|P_{1}\right)=D\left(P_{\lambda}\|P_{1}\right)$ increases
with $\lambda$ and $D\left(P_{\lambda}\|P_{2}\right)$ decreases with
$\lambda$, the maximum value of the minimum of
$D\left(P_{\lambda}\|P_{1}\right),D\left(P_{\lambda}\|P_{2}\right)$ is
attained when they are equal.

This is illustrated below:

Figure 5: The figure above shows the relative entropy for each error
probability as a function of $\lambda$.

We choose $\lambda$ so that

$D\left(P_{\lambda}\|P_{1}\right)=D\left(P_{\lambda}\|P_{2}\right)=C(P_{1},P_{2})$
(51)

Thus $C\left(P_{1},P_{2}\right)$ is the highest achievable exponent for the
probability of error, and is called the Chernoff information.

∎

## References

  * [1] T.M. Cover, J.A. Thomas, _Elements of Information Theory_ , Wiley, 2nd edition, 2006\.
  * [2] D. Eisenberg, D. Crothers, _Physical Chemistry with Applications to the Life Sciences_ , Benjamin Cummings, 3rd edition, 1979\.
  * [3] P. Gopych, _Sensitivity and Bias within the Binary Signal Detection Theory, BSDT_ , Information Theories & Applications, Vol. 11, 2004\.


11institutetext: Department of Electrical and Computer Engineering, National
University of Singapore 22institutetext: Department of Computer and
Information Science, University of Mississippi 33institutetext: CtrsVision
33email: {yihao<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>{xinchao<EMAIL_ADDRESS>
# Domain-Adaptive 2D Human Pose Estimation via Dual Teachers in Extremely Low-
Light Conditions

Yihao Ai  Equal contribution.11 0009-0005-2336-4813 Yifei Qi ⋆ 11
0009-0007-4197-947X Bo Wang 2233 0000-0002-0127-2281 Yu Cheng 11
0000-0002-9830-0081
Xinchao Wang 11 0000-0003-0057-1404 Robby T. Tan 11 0000-0001-7532-6919

###### Abstract

Existing 2D human pose estimation research predominantly concentrates on well-
lit scenarios, with limited exploration of poor lighting conditions, which are
a prevalent aspect of daily life. Recent studies on low-light pose estimation
require the use of paired well-lit and low-light images with ground truths for
training, which are impractical due to the inherent challenges associated with
annotation on low-light images. To this end, we introduce a novel approach
that eliminates the need for low-light ground truths. Our primary novelty lies
in leveraging two complementary-teacher networks to generate more reliable
pseudo labels, enabling our model achieves competitive performance on
extremely low-light images without the need for training with low-light ground
truths. Our framework consists of two stages. In the first stage, our model is
trained on well-lit data with low-light augmentations. In the second stage, we
propose a dual-teacher framework to utilize the unlabeled low-light data,
where a center-based main teacher produces the pseudo labels for relatively
visible cases, while a keypoints-based complementary teacher focuses on
producing the pseudo labels for the missed persons of the main teacher. With
the pseudo labels from both teachers, we propose a person-specific low-light
augmentation to challenge a student model in training to outperform the
teachers. Experimental results on real low-light dataset (ExLPose-OCN) show,
our method achieves 6.8% (2.4 AP) improvement over the state-of-the-art (SOTA)
method, despite no low-light ground-truth data is used in our approach, in
contrast to the SOTA method. Our code is available at: DA-LLPose.

###### Keywords:

Human pose estimation Low-light Domain adaptation

##  1 Introduction

Human 2D pose estimation is a fundamental task in computer vision and
important for many downstream tasks like human activity recognition [59],
virtual tryon [12], motion capture [11], augmented reality [55], or Metaverse
[21]. The existing methods and benchmarks [34, 60, 31] primarily focus on
well-illuminated scenarios, where human subjects exhibit both clear visibility
and high recognizability in the majority of cases. However, poor lighting
conditions such as low light or nighttime are a prevalent aspect of daily
life, which signifies the importance of low-light human pose estimation
research. Despite the significance of this research area, its exploration has
been constrained by the lack of suitable low-light datasets or benchmarks.

A previous dataset for human pose estimation under poor lighting conditions
primarily revolves around nighttime scenarios [10, 9], where human subjects
generally remain recognizable due to the presence of light sources such as
lamps or car lights. Thus, these nighttime datasets do not focus on extremely
low-light conditions, which pose severe challenges to existing human pose
estimation methods. Recently, Lee et al. [29] proposed a new benchmark for
human pose estimation in extremely low-light conditions, which covers
different levels of low-light conditions, and a low-light pose estimation
method, which requires paired well-lit and low-light images with ground truths
for training. However, the scarcity of paired images with ground truths in
practical scenarios makes the application of this method impractical. To this
end, we aim to develop an innovative domain adaptive method for low-light
human pose estimation, utilizing well-lit ground-truth data only. Our goal is
to achieve performance on par with the SOTA methods that rely on both low-
light and well-lit ground truths.

To the best of our knowledge, we are the first to propose a solution to
estimate human poses in extremely low-light conditions utilizing well-lit
ground-truth data only. This is an extremely challenging task because the
visibility of human subjects is poor and low-light images are corrupted due to
severe low-light noise. The intensity and contrast are much lower in the low-
light images (e.g., average pixel intensity of well-lit and low-light images
are 90.5 and 2.0). Without paired well-lit and low-light images with ground-
truth data, two types of existing methods could be used for low-light pose
estimation: low-light image enhancement [52, 1] and domain adaptation [46,
26]. However, these two groups of methods have their own limitations. The
former is difficult to restore the extremely low-light images faithfully and
the latter still lacks a specific solution for bridging the gap between well-
lit and low-light conditions.

In response to these challenges, we introduce an innovative domain-adaptive
dual-teacher framework for human pose estimation that facilitates knowledge
transfer from well-lit to low-light domains. This innovation enables our model
to exceed state-of-the-art (SOTA) methods by eliminating the requirement for
training with low-light ground truths, which are often impractical to obtain.
Ideally, cross-domain knowledge transfer is achievable using a single teacher,
as demonstrated by existing methods [44, 26, 25]. However, a single center-
based teacher, designed to predict the entire human pose based on each
person’s center, may encounter difficulties in detecting individuals in low-
light images due to inherent architectural limitations that focus solely on
the detection of human centers. Consequently, this approach may fail to
identify individuals when their centers cannot be detected, resulting in a
limited number of pseudo labels. To generate a more comprehensive set of human
pose predictions for use as pseudo labels, we propose the incorporation of a
complementary teacher: a keypoint-based bottom-up human pose estimation model.
This model is capable of predicting partial human poses by grouping detected
keypoints, offering a complementary approach to the primary center-based
teacher.

Our method comprises two stages: pre-training and dual-teacher knowledge
acquisition. During the first stage, we perform supervised training with
labeled well-lit data alone. This initial step equips both teachers to
recognize human poses in well-lit images and their corresponding low-light
augmentations. In the subsequent second stage, our focus shifts to harnessing
knowledge directly from unlabeled low-light images. To challenge the student
model during the second stage, we introduce a novel augmentation technique
called Person-specific Degradation Augmentation (PDA). PDA is selectively
applied to images with pseudo labels generated by the teachers. Given that
low-light conditions pose a significant challenge in pose estimation, our
degradation augmentations primarily darken the individuals, making them less
distinguishable from their backgrounds to mimic realistic low-light scenarios.
Such a simple, flexible and effective degradation neither necessitates the
camera calibration as various physical-based augmentations did [53, 54], nor
relies on an additional network, which would cost extra computational
resources [39]. And it facilitates the student model’s exploration of
knowledge in the low-light domain, thereby encouraging it to surpass the
performance of the teachers in the second stage.

LABEL:fig:representative_result shows three images from ExLPose-OCN [29] and
the human pose estimation results of the existing methods [52, 1, 26, 46] and
our proposed method. Compared with the existing methods, our method can
effectively estimate human poses in extremely low-light conditions.
Specifically, our method is capable of handling low-contrast (last row) and
partially visible (second row) persons. For the relatively visible persons
(first row), our method can produce more accurate pose estimation. In summary,
our major contributions are listed as follows.

  * •

We investigate a challenging yet practical task, i.e., estimating human poses
in extremely low-light conditions with well-lit ground-truth data only. To the
best of our knowledge, we are the first to propose a solution in this
challenging setting.

  * •

We introduce a novel domain-adaptive dual-teacher framework, utilizing both
center-based and keypoint-based teachers to generate enriched pseudo labels
for effective student model training, eliminating the need for low-light
ground truths, which are often impractical to obtain. To our knowledge, this
represents a significant advancement in low-light human pose estimation.

  * •

Our proposed method outperforms the SOTA method by 6.8% (2.4 AP) on the real
low-light dataset, ExlPose-OCN. Notably, our method utilizes well-lit ground-
truth data alone, while the SOTA method is trained with paired well-lit and
low-light images with ground truths.

##  2 Related Work

2D Human Pose Estimation Two primary paradigms exist within 2D human pose
estimation: top-down and bottom-up approaches. In general, top-down methods,
benefiting from applying human detection first [17, 5, 56, 45, 43], tend to
outperform their bottom-up counterparts [3, 37, 27, 6]. Early approaches of
bottom-up estimation [3, 37] are keypoint-based, detecting keypoints for each
individual in an image before grouping them to construct individual human
poses. Recently, Geng et al. propose a center-based bottom-up method [14],
which predicts human centers instead of grouping individual keypoints, and
regresses offsets between a human center and the keypoints for each
individual. This method sets a new trend among the bottom-up methods [58, 47,
48]. However, existing 2D pose estimation methods focus on well-lit
conditions, primarily due to the prevalence of well-lit images within the
benchmark datasets [34, 60, 31]. Additionally, since existing methods are
fully supervised, they demonstrate poor performance when applied directly to
low-light images after being trained on well-lit data.

Domain Adaptive Pose Estimation Domain adaptive pose estimation is potentially
useful for low-light conditions. A number of domain adaptation methods are
proposed in human pose [26, 41, 38], animal pose [36, 2, 30] and hand pose
estimations [22, 16, 23]. However, these methods are not designed for human
pose estimation in extremely low-light conditions. Source-free domain adaptive
human pose methods are developed [41, 38] to transfer knowledge from synthetic
to real data in the absence of the source data. This is in opposition to low-
light scenarios, where well-lit (source) domain data is typically available.
Kim et al. propose a unified framework for domain adaptive human pose
estimation [26], where the student is challenged to learn the target-domain
knowledge by applying affine transform-based augmentations. However, this
augmentation is not tailored for the severe lighting conditions.

Low-light Human Pose Estimation Apart from the domain adaptation-based
methods, fully-supervised approaches are developed as well. Crescitelli et al.
propose a nighttime human pose estimation method that combines images from
both RGB and infrared cameras [10, 9]. However, the reliance on infrared
cameras restricts the practical applicability of this method. Recently, Lee et
al. [29] propose a top-down human pose estimation approach for extremely low-
light conditions. Nevertheless, this method depends on the use of paired well-
lit and low-light images with ground-truth data during training, which are
impractical for real-world applications due to the scarcity of the paired data
and the inherent challenges associated with annotating low-light images. In
the context of semi-supervised learning for human pose estimation, the
utilization of multiple teachers, especially in dual-teacher configurations
[57, 19], is related to our work. However, semi-supervised learning typically
leverages both labeled and unlabeled data under the assumption that they come
from similar distributions. This assumption differs significantly from the
low-light setting, underscoring the necessity for our innovative domain-
adaptive dual-teacher framework.

Low-light Image Enhancement Low-light image enhancement might be used to
enhance low-light images prior to human pose estimation. Early approaches
primarily rely on gamma correction [40] or histogram equalization [7, 4]. More
advanced methods restore the low-light images based on the Retinex theory [32,
51]. Unfortunately, these methods tend to introduce additional noise with
color distortion. In the deep learning era, a number of convolutional neural
network (CNN)-based [35, 42, 50] and GAN-based [24, 20] methods are developed.
Recently, Wang et al. introduce an invertible network to capture the
distribution of the well-lit images [52]. Cai et al. propose a transformer-
based approach to restore the hidden corruption in low-light images [1].
However, these image enhancement methods still face difficulties in faithfully
restoring extremely low-light images. Thus, even with enhanced images of these
methods, low-light pose estimation is still challenging.

Figure 1: The overview of our method. The grey uni-directional arrows indicate
the supervised workflow in the pre-training stage. The grey and pink arrow
indicates the workflow of the supervised and unsupervised losses in the
knowledge acquisition stage. The low-light images, $I_{\rm fake}$, $I_{\rm
real}$ and $I_{\rm pda}$ are all brightened for improved visibility. NMS
stands for non-maximum suppression.

##  3 Proposed Method

Given an image $I\in{\mathbb{R}^{3\times{h}\times{w}}}$ which contains persons
in the low-light conditions, 2D human pose estimation is to obtain the human
skeletons from an image by locating $K$ types of human keypoints for each
person. An overview of our proposed framework is shown in Fig. 1, where we
present a novel domain-adaptive dual-teacher framework, utilizing both center-
based and keypoint-based teachers to generate enriched pseudo labels for
effective student model training, as shown in the yellow boxes.

###  3.1 Main Teacher and Complementary Teacher

#### Main Teacher

We build our main teacher based on DKER [14], where a person’s pose is
represented as the center and offset in addition to the heatmap for each
keypoint. In particular, the center location of the $i$-th person is
$\textbf{c}^{i}=\frac{1}{K}\sum_{k=1}^{K}\textbf{p}^{i}_{k}$ and the offsets
from the human center to each keypoints are
$\textbf{o}^{i}=\\{\textbf{p}^{i}_{1}-\textbf{c}^{i},\textbf{p}^{i}_{2}-\textbf{c}^{i},...,\textbf{p}^{i}_{k}-\textbf{c}^{i}\\}$,
where $\textbf{p}^{i}_{k}\in{\mathbb{R}^{2}}$ indicates the 2D location of the
$k$-th keypoints for the $i$-th person. The model is supervised by both
heatmap and offset losses:

$L_{\rm sup}^{m}=L_{H}^{m}+\lambda_{m}L_{O}^{m},$ (1)

where $L_{H}^{m}$ denotes a MSE loss of the main teacher’s heatmaps, and
$L_{O}^{m}$ is a smooth L1 loss of the offset map.

#### Complementary Teacher

The main drawback of the center-based representation is that the pose
estimation fully relies on the center’s predictions. If the center of a person
is failed to be detected, the whole person is missed, especially for the low-
contrast and partially visible persons in the low-light conditions. Therefore,
we build a complementary teacher to predict human poses for partially visible
persons following HigherHRNet-style design [6]. Specifically, the network is
trained to estimate the keypoints’ heatmap $\hat{H}_{c}$ and the corresponding
tag map $\hat{T}_{c}$. A person’s pose is grouped based on all the identity
information via Hungarian algorithm [28]. The method is supervised by
following losses:

$L_{\rm sup}^{c}=L_{H}^{c}+\lambda_{c}L_{\rm tag}^{c},$ (2)

where, $L_{H}^{c}$ is a MSE loss of the complementary teacher’s heatmaps, and
$L_{\rm tag}^{c}$ consists of push and pull losses of the tag maps. More
details are provided in the supplementary.

###  3.2 Pre-Training Stage

Firstly, the two teachers are trained on the well-lit dataset [29] to gain the
common knowledge of the human bodies. They are supervised by the losses
defined in Eq. 1 and Eq. 2 respectively. After the supervised training with
the labeled well-lit data, teachers can predict human poses on well-lit
images, but they cannot process low-light images due to the substantial domain
gap between well-lit and low-light conditions. As our framework relies solely
on well-lit ground-truth data, we propose Extreme Low-Light Augmentation
(ELLA) to augment low-light images from well-lit ones. This process involves
incorporating a set of low-light characteristics into the well-lit images.

The distinguishing characteristics of low-light images include extreme
darkness, high levels of noise, and low contrast. To simulate these low-light
specific features, ELLA applies a series of random augmentations sequentially
to well-lit images, including gamma correction, brightness adjustment,
contrast reduction, and Gaussian noise. The first two augmentations aim to
intensify the darkness of the images, while contrast reduction is employed to
decrease image contrast, and Gaussian noise is added to introduce noise into
the images. It is important to note that in ELLA, we do not explicitly
simulate light sources. This is because our primary focus is on low-light
conditions, where no light source is typically available. An example of the
augmented images using ELLA is shown in Fig. 2. The following equations, Eq.
3, Eq. 4, Eq. 5, and Eq. 6, mathematically represent the four ELLA
augmentations:

$\displaystyle I_{\rm out}$
$\displaystyle=255\cdot\left(I_{in}/255\right)^{\gamma},$ (3) $\displaystyle
I_{\rm out}$ $\displaystyle=bI_{\rm in},$ (4)

$\displaystyle I_{\rm out}$ $\displaystyle=cI_{\rm in}+(1-c)I_{\rm grey},$ (5)
$\displaystyle I_{\rm out}$ $\displaystyle=I_{\rm in}+{\rm Gaussian}(0,var),$
(6)

where $I_{\rm in}$ is the input image, $I_{\rm grey}$ is the greyscale image
of $I_{\rm in}$, and $I_{\rm out}$ stands for the output image.
$\gamma\in[2,5]$, $b\in[0.01,0.05]$, $c\in[0.2,1.0]$ and $var\in[0,40]$ are
the parameters uniformly sampled from their respective ranges.

The augmented well-lit images are named fake low-light images, denoted as
$I_{\rm fake}$. The augmentation is added randomly so that both teachers are
trained with a mixture of well-lit images and fake low-light images and
supervised by the losses in Eq. 1 and Eq. 2 respectively (each of the ELLA
augmentations has a 0.5 probability to be added). It is important to use both
well-lit images and fake low-light images in the training because it is
difficult for the model to transfer the knowledge to low-light domain if all
inputs suddenly change to the fake low-light images.

###  3.3 Dual-Teacher Knowledge Acquisition Stage

After the pre-training stage, the teachers possess the preliminary capability
to perform human pose estimation for extremely low-light images. In this
stage, our primary objective is to enhance the student model’s ability for
detecting and localizing human keypoints by leveraging real low-light images
and guiding the student to surpass both teachers. In particular, given a non-
paired fake low-light image $I_{\rm fake}$, a real low-light image $I_{\rm
real}$, and the pre-trained two teachers $T_{m}$, $T_{c}$, we train the
student $S$, who shares the same architecture of the main teacher at the
beginning, by a supervised loss and an unsupervised loss. The workflow of the
supervised and unsupervised losses is illustrated in Fig. 1 with grey and pink
arrows respectively.

#### Supervised Loss

The supervised loss is a continuation of the pre-training stage. To strengthen
the student’s knowledge on the low-light domain, a larger proportion of fake
low-light images is expected to be added in this stage. Ideally, all well-lit
images that are used in the student’s training should be augmented in the low-
light fashion by ELLA. However, this would trigger the knowledge forgetting
problem [33, 49] which is an inherent problem in domain adaptation methods. To
strengthen the learning in the target domain while keeping the source
knowledge, we make a few adjustment to the ELLA as follows.

#### Adjust ELLA

On the one hand, we set ELLA to do gamma correction and brightness adjustment
as Eq. 4 and Eq. 3 to every input well-lit images. This aims to augment more
samples in the low-light domain. On the other hand, to avoid the forgetting
problem, we randomly crop a few image patches in an augmented image and
restore it to the well-lit domain. The cropping probability for an image is
0.15 to guarantee the majority of the inputs are still fully-augmented images.
The supervised loss is computed by the following equations (shown in Fig. 1
with grey arrows). First, well-lit images $I_{\rm well}$ are augmented to fake
low-light images $I_{\rm fake}$ with adjusted ELLA as:

$I_{\rm fake}={\rm AdjustELLA}(I_{\rm well}).$ (7)

Second, student $S$ produces the prediction for $I_{\rm fake}$:

$\hat{H}_{f},\hat{O}_{f}=S(I_{\rm fake}),$ (8)

where $\hat{H}$ and $\hat{O}$ refer to the heatmaps and offset maps, which are
used in the calculation of the supervised loss following Eq. 2.

(a) Gamma

(b) Brightness

(c) Contrast

(d) Noise

(e) All Aug.

(f) Bri. All

Figure 2: Examples of our ELLA. (a)-(d) are the results of the individual
augmentation in ELLA. (e) is the image with all the augmentations, and Aug.
stands for augmentations. (e) is brightened up for improved visibility as
shown in (f), where Bri. All denotes brightened images with all the
augmentations.

#### Unsupervised Loss

Apart from the supervised loss, we teach the student with the pseudo labels
selected and fused by the predictions from both teachers. As shown in the Fig.
1, the two teachers firstly generate their own pose predictions $P_{m}$ and
$P_{c}$ by $P_{m}=T_{m}(I_{\rm real})$ and $P_{c}=T_{c}(I_{\rm real})$. Since
the two teachers score the poses in different manner (the main teacher
directly uses the center’s activation, $C_{m}$, while the complementary
teacher averages the heatmap activation of all keypoints as the confidence
score, $C_{c}$), we use different thresholds $s_{m}$ and $s_{c}$ to select out
the high-confidence poses. Then the poses from both teachers are concatenated
by and applied non-maximum suppression (NMS) [18] to rule out the redundant
poses predictions to obtain the pseudo labels $P_{\rm all}$ defined as:

$P_{\rm all}={\rm NMS}(P_{m}[C_{m}>s_{m}]\oplus P_{c}[C_{c}>s_{c}]),$ (9)

where $\oplus$ indicates concatenation operation.

#### Person-specific Degradation Augmentation

The student needs to gain low-light domain-specific knowledge in order to
outperform the teachers in challenging scenarios under low illumination. For
example, the right person with black shirt in the last row of
LABEL:fig:representative_result is largely indistinguishable from the dark
background, which demonstrates that low contrast is a major challenge for low-
light human pose estimation. Therefore, the network has to learn the specific
human structure in a texture-less condition. To this end, we propose a Person-
specific Degradation Augmentation (PDA) to realistically mimic the extremely
low-light conditions at individual-level. Specifically, after getting the
selected poses $P_{\rm all}$ from the teachers, a set of bounding boxes is
generated to crop the individuals. The cropped image patches are augmented
based on Eq. 4, which are then used to replace the original ones from input
$I_{\rm real}$ to generate output $I_{\rm pda}$.

The last step of the unsupervised loss computation is to generate the pseudo
labels based on $P_{\rm all}$ and to supervise the output of the student:

$\hat{H}_{r},\hat{O}_{r}=S({\rm PDA}(I_{\rm real},P_{\rm all})).$ (10)

The unsupervised loss $L_{\rm unsup}^{m}=L_{H}^{m}+\lambda_{m}L_{O}^{m}$ is
similar to Eq. 1. Combining both supervised and unsupervised losses, the loss
of the student $S$ is defined as follows:

$L{s}=\lambda_{\rm sup}L_{\rm sup}^{m}+\lambda_{\rm unsup}L_{\rm unsup}.$ (11)

##  4 Experiments

###  4.1 Implementations and Dataset

#### Dataset

ExLPose datasets [29] are used in validating our method, which is a new
dataset that is specifically designed for evaluating 2D human pose estimation
under extremely low-light conditions. The training set of ExLPose provides
$2,065$ pairs of well-lit and low-light images and the ground-truth definition
follows the CrowdPose [31] format. These images were taken from 251 indoor and
outdoor scenes in the daytime and the paired low-light images are artificially
produced by using a dual-camera system with different settings to ensure a
wide coverage of diverse low-light conditions.

There are two different testing sets in ExLPose [29]. One is ExLPose-OCN , in
which 360 extremely low-light images are captured at night with two different
cameras, A7M3 and RICOH3. Another one is ExLPose-test, which consists of 491
man-made low-light images captured and produced similarly as those in the
training set. Low-light All (LL-A) is used to denote the whole ExLPose-test,
which is further divided into three subsets according to their difficulty
level including Low-Light Normal (LL-N), Hard (LL-H), and Extreme (LL-E).

Among the two testing sets, ExLPose-OCN is captured in real low-light
environments, while the low-light images in the ExLPose-test are obtained by
artificially reducing the amount of light by 100 times [29]. In addition,
ExLPose-OCN is a test-only dataset, which is used to evaluate the
generalization capability of a model trained on ExLPose-test. Therefore, the
evaluation on ExLPose-OCN provides a more objective measurement of the
performance of low-light human pose estimation. In our method, we only use the
unpaired labeled well-lit images and unlabeled low-light images in the
training set to train our models. Low-light ground-truth data is never used in
training our model.

Table 1: Evaluation on ExLPose-OCN with top-down baselines (lower part of the
table) and the bottom-up baselines (upper part of the table) tested in our
paper. Our method is a bottom-up method. The reported top-down baselines are
all fully-supervised methods. LL indicates low-light labels. WL indicates
well-lit labels. The best is bold. The second best is underlined.

Methods | Training Labels | AP↑@0.5:0.95
---|---|---
|  LL |  WL |  A7M3 |  RICOH3 |  Avg.
Base-low [14] |  ✓ |  |  27.1 |  15.9 |  21.5
Base-well [14] |  |  ✓ |  5.3 |  7.4 |  6.3
RFormer [1] |  |  ✓ |  18.9 |  17.7 |  18.3
LLFlow [52] |  |  ✓ |  23.7 |  19.1 |  21.4
UDA-HE [26] |  |  ✓ |  6.5 |  7.2 |  6.9
AdvEnt [46] |  |  ✓ |  9.1 |  11.2 |  10.1
CPN-low [5] |  |  ✓ |  23.7 |  23.9 |  23.8
CPN-well [5] |  ✓ |  |  15.2 |  15.6 |  15.4
CPN-all [5] |  ✓ |  ✓ |  32.8 |  31.7 |  32.2
LLFlow [52] |  ✓ |  ✓ |  25.6 |  28.2 |  27.0
LIME [15] |  ✓ |  ✓ |  33.2 |  28.4 |  30.7
DANN [13] |  ✓ |  ✓ |  27.9 |  30.6 |  29.3
AdvEnt [46] |  ✓ |  ✓ |  28.2 |  29.0 |  28.6
LSBN+LUPI [29] |  ✓ |  ✓ |  35.3 |  35.1 |  35.2
Ours |  |  ✓ |  39.1 |  36.2 |  37.6

#### Evaluation Protocol

We follow the COCO protocol [34], as used in LSBN+LUPI [29], to validate on
ExLPose-test and ExLPose-OCN. Average precision at various thresholds (i.e.,
$AP@0.5:0.95$) is reported for every subset.

#### Implementation Details

DEKR-W32 [14] is adopted as our backbone network for the main teacher and the
student. HigherHRNet [6] is modified to be the complementary teacher. In the
pre-training stage, the input image of our framework is cropped to $512\times
512$. The data augmentations follow the DEKR paper setting, which include
random rotation $[-30^{\circ},30^{\circ}]$, random scaling $[0.75,1.5]$,
random translation $[-40,40]$ and random flipping. The training strategy of
both teachers is the same with DEKR. The only difference is the loss weighting
factors, the $\lambda_{m}=0.03$ and $\lambda_{c}=0.001$.

In the knowledge acquisition stage, the confidence score thresholds are set to
be $s_{m}=0.9$ and $s_{c}=0.5$ respectively for dual teachers. In training, we
use Adam optimizer. The learning rate is set as $1e-4$. $\lambda_{\rm sup}$
and $\lambda_{unsup}$ are set to be 1.0. Our framework is trained on 4 RTX3090
GPUs (24GB of VRAM each) with a per-GPU batch size of 9 for the pre-training
and 8 for the dual teacher knowledge acquisition stage. The training time for
the dual-teacher setup is 139 seconds per epoch, which is comparable to the
single-teacher setting at 103 seconds per epoch. Additional information is
provided in supplementary material.

#### Baselines

To compare with existing image enhancement and domain adaptation methods, two
SOTA image enhancement methods (RFormer, LLFlow) [1, 52] and two SOTA domain
adaptation methods (UDA-HE, AdvEnt) [26, 46] are used as baselines. To ensure
a fair comparison, we employ the same base human pose estimator (DEKR) [14]
for all the aforementioned methods, as well as for our method, utilizing only
well-lit ground-truth data during training. Following the setup in ExLPose
[29], we replace AdvEnt’s semantic segmentation backbone with the same 2D pose
estimator used in our method. UDA-HE is a model-agnostic domain-adaptive human
pose estimation method. We upgrade the backbone network (ResNet) in UDA-HE by
HRNet, the same backbone used by our method and re-train AdaIN to ensure a
fair comparison. Note, UDA-HE is configured under a bottom-up paradigm, which
is included as a bottom-up baseline in Tab. 3. For a comprehensive comparison,
we add the top-down baselines reported in LSBN+LUPI [29], which are trained
with both well-lit and low-light ground-truth data (unpaired). CPN [5] is used
as the base human pose estimator for these top-down baselines. Among the
additional baselines [29], there is an additional domain adaptation method
(DANN) [13] and another image enhancement method (LIME) [15]. Note that the
training data of LSBN+LUPI [29] is different from all the baselines because it
relies on paired well-lit and low-light images with ground truths. All top-
down baselines employ the same top-down method (CPN) [5] as the base human
pose estimator.

Table 2: Evaluation on ExLPose-test with a bottom-up baseline [14], image
enhancement [1, 52], and domain adaptation [26, 46] methods. All the methods
are fully-supervised by well-lit labels. The best is bold. The second best is
underlined.

Methods | AP↑@0.5:0.95
---|---
LL-N | LL-H | LL-E | LL-A | WL
Base-well [14] | 1.3 | 0.0 | 0.0 | 0.8 | 60.3
RFormer [1] |  16.9 | 2.3 | 0.1 | 5.7 | 60.0
LLFlow [52] | 11.0 | 0.8 |  0.9 | 4.5 | 60.1
UDA-HE [26] | 16.4 |  3.6 | 0.1 |  7.4 | 53.7
AdvEnt [46] | 4.6 | 0.7 | 0.0 | 2.0 | 56.4
Ours | 35.6 | 18.6 | 5.0 | 21.1 | 61.6

Table 3: Evaluation on ExLPose-test with top-down methods trained on both low-
light and well-lit labels (denoted by ${\dagger}$). Differently, our method is
a bottom-up method trained with well-lit labels alone. The best is bold. The
second best is underlined.

Methods | AP↑@0.5:0.95
---|---
LL-N | LL-H | LL-E | LL-A | WL
CPN-low† [5] | 25.4 | 18.2 | 5.0 | 17.2 | 1.2
CPN-well† [5] | 15.3 | 2.7 | 0.4 | 6.7 | 59.9
CPN-all† [5] | 26.4 | 18.2 |  6.1 | 17.6 | 52.3
LLFlow† [52] | 28.7 | 15.7 | 5.3 | 17.4 |  60.7
LIME† [15] |  31.9 | 21.2 | 7.6 | 21.1 | 57.7
DANN† [13] | 28.0 | 17.5 | 5.3 |  17.8 | 52.0
Ours | 35.6 |  18.6 | 5.0 | 21.1 | 61.6

###  4.2 Performance on ExLPose-OCN

Quantitative comparison on ExLPose-OCN is provided in Tab. 1. The upper part
of the table is the bottom-up baselines and the lower part is top-down. Note
that our method is a bottom-up one. The top-down baselines include CPN fully
supervised on low-light images (CPN-low), well-lit images (CPN-well), or both
of them (CPN-all), CPN integrated with low-light image enhancement methods
[52, 15], and domain adaptation methods [13, 46]. It is worth mentioning all
the top-down baselines (except the CPN-well) are fully supervised by both the
well-lit and low-light ground truths and the reported performance, all using
ground-truth person detections. The bottom-up baselines include DEKR trained
on labeled low-light or well-lit images, denoted as Base-low and Base-well,
respectively. Also, we integrate SOTA image enhancement methods, including
RFormer [1] and LLFlow [52], as well as domain adaptation methods, including
output-level method UDA-HE [26] and feature-level method AdvEnt [46]. The
performance of the image enhancement methods is tested by applying the Base-
well to the enhanced images. Therefore, all the bottom-up baselines including
ours are trained with the well-lit ground-truth data alone with the training
set of ExLPose and we only use ExLPose-OCN for evaluation.

Our method outperforms the SOTA method [29] by 6.8% (2.4 AP) on average. As
ExLPose-OCN captures actual low-light images, the superior performance
validates the effectiveness of our method in addressing the low-light
challenges, where our method surpasses all the bottom-up and top-down
baselines without using low-light ground truths or paired well-lit and low-
light data. This highlights the potential of our method in real-world
applications, where we achieve performance surpassing the SOTA methods without
using low-light ground truths.

###  4.3 Performance on ExLPose-test

We compare our method with various bottom-up baselines on ExLPose-test,
including a fully-supervised baseline [14] and four baselines using well-lit
ground-truth data only for training based on image enhancement [52, 1] or
domain adaptation [26, 46], as shown in Tab. 3. As expected, the fully-
supervised baseline, Base-well, is heavily biased towards their training data
due to the lack of cross-domain generalization. Its poor performance clearly
demonstrates the substantial gap between well-lit and extremely low-light
domains. Our method outperforms the baselines using image enhancement or
domain adaptation. In particular, our performance beats the second best method
on the LL-N subset by 18.7 AP which almost doubled the second-best. For LL-H
and LL-E subsets, all methods except ours struggled to achieve reasonable
performance. Our method also maintains the best performance on the WL (well-
lit) subset compared with DEKR-well.

To make a comprehensive comparison, we further compare with top-down baselines
incorporated with image enhancement and domain adaptation methods in Tab. 3.
Note that the comparison is not fair to our approach as the top-down baselines
reported in [29] were all trained using both low-light and well-lit ground
truths. Furthermore, top-down human pose estimation methods generally
outperform their bottom-up counterparts due to their utilization of human
detection, as well as the reduction in human scale variation achieved through
detection cropping and human patch normalization [8].

However, we still achieve the highest performance on the LL-N, LL-A, and WL
subsets in Tab. 3, while obtaining comparable performance on other subsets
without utilizing the low-light ground-truth data. Note LSBN+LUPI [29] is not
included because its training data is different from the baselines in Tab. 3.
LSBN+LUPI utilizes paired well-lit and low-light data and the ground truths,
while the baselines included in the table use unpaired well-lit and low-light
datasets in training. With the extra paired training data and ground truths,
LSBN+LUPI manages to attain competitive performance on ExLPose-test, where its
performance on the WL subset (61.5) is still worse than ours, but it is better
than ours in LL-A (25.0). However, when evaluating the generalization ability
of each method on the real low-light dataset (ExLPose-OCN), where there is no
training and only testing, our method surpasses all the baselines, including
LSBN+LUPI, as demonstrated in Tab. 1.

Table 4: Ablation Study of the proposed modules in our method over ExLPose-
test. PT stands for Pre-Training stage. Step 1 is the fully-supervised
training on well-lit images. Step 2 is the fully supervised training with fake
low-lit images after adding ELLA. KA indicates the Knowledge Acquisition
stage. Single indicates the training with single teacher. Dual indicates our
dual-teacher framework. The best is bold. The second best is underlined.

| Our Design | AP↑@0.5:0.95
---|---|---
ELLA |  PDA | LL-N | LL-H | LL-E | LL-A | WL
PT Step 1 |  |  | 1.3 | 0.0 | 0.0 | 0.8 | 60.3
PT Step 2 |  ✓ |  | 29.4 | 13.6 | 1.6 | 15.8 | 62.1
KA |  ✓ |  | 29.5 | 14.6 | 2.9 | 16.8 | 60.5
Single |  ✓ |  ✓ |  33.4 |  16.6 |  3.1 |  19.0 | 59.4
Dual |  ✓ |  ✓ | 35.6 | 18.6 | 5.0 | 21.1 |  61.6

Table 5: Number of the additional pseudo labels provided by the complementary
teacher in ExLPose training set. OKS Thres., referring to OKS threshold, is
used to determine the validity of pseudo labels by comparing the OKS between
pseudo labels and the ground truth with them. PL indicates pseudo labels. Main
indicates the mean teacher. Comp. indicates complementary teacher.

OKS Thres. | # of PL in Main | # of PL in Comp. | # of additional PL
---|---|---|---
0.3 | 10203 | 12155 | 4870 (+47.7%)
0.5 | 5056 | 8735 | 2136 (+49.8%)
0.7 | 3293 | 5435 | 1180 (+35.8%)
0.9 | 1463 | 1574 | 501 (+34.2%)

###  4.4 Ablation Studies

Ablation study is performed to validate the effectiveness of the individual
modules in our method is shown in Tab. 4. We use our framework in the pre-
training stage without any newly proposed components as the baseline, and
subsequently add each new module in separate experiments.

#### ELLA

We evaluate our method’s performance with and without the supervision of fake
low-light images generated from WL. The results are in the first two rows of
Tab. 4. The inclusion of ELLA significantly boosts the performance on LL-N,
LL-H, and LL-A test subsets from almost zero to a reasonable level.

#### PDA

We investigate our method’s performance with and without PDA in Stage 2
training. When comparing the third row (single teacher without PDA) to the
fourth row (single teacher with PDA), we observe a $20.2\mathcal{\%}$ (3.4 AP)
improvement on LL-A, accompanied by a marginal 0.4 AP drop on WL. This
highlights the effectiveness of PDA in enabling the student to surpass the
teachers.

#### Dual Teacher

Comparing the last two rows, we observe that the model’s performance with dual
teachers (last row) surpasses that of a single teacher (second last row) in
every split. Notably, The complementary teacher achieves a $11\mathcal{\%}$
(2.1 AP) improvement, which is comparable to the $13\mathcal{\%}$ (2.2 AP)
improvement obtained by the main teacher on LL-A (all subsets), underscoring
the effectiveness of our dual-teacher framework. Additionally, with the help
of the dual teachers, the student model exhibits enhanced performance on the
well-lit split.

To validate that the complementary teacher provides more valid pseudo labels,
quantitative evaluation is provided in Table 5. We measure the validity of
pseudo labels by computing their OKS with corresponding ground truths and
count the number of non-overlapping pseudo labels by thresholding the OKS at
0.5 between the dual teachers, following [34, 31]. The evaluation demonstrates
that the complementary teacher consistently produces additional valid pseudo
labels across different thresholds of quality, providing an average of 41.8%
more valid pseudo labels compared to using the main teacher alone. The
increased number of valid pseudo labels leads to further improvement in the
student’s training, highlighting the effectiveness of our dual-teacher
framework.

##  5 Conclusion

We introduce the first human pose estimation method for extremely low-light
conditions, utilizing well-lit ground-truth data exclusively. Our novelty lies
in leveraging center-based and keypoint-based teachers to generate enriched
pseudo labels, enabling the student model to achieve competitive performance
on extremely low-light images without the need for training with low-light
ground truths. To our knowledge, this represents a significant advancement in
the field of low-light human pose estimation.

## Acknowledgements

This research is in part supported by the Singapore Ministry of Education
Academic Research Fund Tier 1 (WBS: A-8001229-00-00), a project titled
“Towards Robust Single Domain Generalization in Deep Learning”.

## References

  * [1] Cai, Y., Bian, H., Lin, J., Wang, H., Timofte, R., Zhang, Y.: Retinexformer: One-stage retinex-based transformer for low-light image enhancement. arXiv preprint arXiv:2303.06705 (2023)
  * [2] Cao, J., Tang, H., Fang, H.S., Shen, X., Lu, C., Tai, Y.W.: Cross-domain adaptation for animal pose estimation. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 9498–9507 (2019)
  * [3] Cao, Z., Hidalgo, G., Simon, T., Wei, S.E., Sheikh, Y.: Openpose: realtime multi-person 2d pose estimation using part affinity fields. IEEE transactions on pattern analysis and machine intelligence 43(1), 172–186 (2019)
  * [4] Celik, T., Tjahjadi, T.: Contextual and variational contrast enhancement. IEEE Transactions on Image Processing 20(12), 3431–3441 (2011)
  * [5] Chen, Y., Wang, Z., Peng, Y., Zhang, Z., Yu, G., Sun, J.: Cascaded pyramid network for multi-person pose estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 7103–7112 (2018)
  * [6] Cheng, B., Xiao, B., Wang, J., Shi, H., Huang, T.S., Zhang, L.: Higherhrnet: Scale-aware representation learning for bottom-up human pose estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 5386–5395 (2020)
  * [7] Cheng, H.D., Shi, X.: A simple and effective histogram equalization approach to image enhancement. Digital signal processing 14(2), 158–170 (2004)
  * [8] Cheng, Y., Ai, Y., Wang, B., Wang, X., Tan, R.T.: Bottom-up 2d pose estimation via dual anatomical centers for small-scale persons. Pattern Recognition 139, 109403 (2023)
  * [9] Crescitelli, V., Kosuge, A., Oshima, T.: Poison: Human pose estimation in insufficient lighting conditions using sensor fusion. IEEE Transactions on Instrumentation and Measurement 70, 1–8 (2020)
  * [10] Crescitelli, V., Kosuge, A., Oshima, T.: An rgb/infra-red camera fusion approach for multi-person pose estimation in low light environments. In: 2020 IEEE Sensors Applications Symposium (SAS). pp. 1–6. IEEE (2020)
  * [11] Desmarais, Y., Mottet, D., Slangen, P., Montesinos, P.: A review of 3d human pose estimation algorithms for markerless motion capture. Computer Vision and Image Understanding p. 103275 (2021)
  * [12] Dong, H., Liang, X., Shen, X., Wang, B., Lai, H., Zhu, J., Hu, Z., Yin, J.: Towards multi-pose guided virtual try-on network. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 9026–9035 (2019)
  * [13] Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., Lempitsky, V.: Domain-adversarial training of neural networks. The journal of machine learning research 17(1), 2096–2030 (2016)
  * [14] Geng, Z., Sun, K., Xiao, B., Zhang, Z., Wang, J.: Bottom-up human pose estimation via disentangled keypoint regression. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 14676–14686 (2021)
  * [15] Guo, X.: Lime: A method for low-light image enhancement. In: Proceedings of the 24th ACM international conference on Multimedia. pp. 87–91 (2016)
  * [16] Han, Z., Sun, H., Yin, Y.: Learning transferable parameters for unsupervised domain adaptation. IEEE Transactions on Image Processing 31, 6424–6439 (2022)
  * [17] He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask r-cnn. In: Proceedings of the IEEE international conference on computer vision. pp. 2961–2969 (2017)
  * [18] Hosang, J., Benenson, R., Schiele, B.: Learning non-maximum suppression. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 4507–4515 (2017)
  * [19] Huang, L., Li, Y., Tian, H., Yang, Y., Li, X., Deng, W., Ye, J.: Semi-supervised 2d human pose estimation driven by position inconsistency pseudo label correction module. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 693–703 (2023)
  * [20] Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. In: Proceedings of the IEEE international conference on computer vision. pp. 1501–1510 (2017)
  * [21] Jiang, J., Streli, P., Qiu, H., Fender, A., Laich, L., Snape, P., Holz, C.: Avatarposer: Articulated full-body pose tracking from sparse motion sensing. In: European Conference on Computer Vision. pp. 443–460. Springer (2022)
  * [22] Jiang, J., Ji, Y., Wang, X., Liu, Y., Wang, J., Long, M.: Regressive domain adaptation for unsupervised keypoint detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6780–6789 (2021)
  * [23] Jin, R., Zhang, J., Yang, J., Tao, D.: Multibranch adversarial regression for domain adaptative hand pose estimation. IEEE Transactions on Circuits and Systems for Video Technology 32(9), 6125–6136 (2022)
  * [24] Jin, Y., Yang, W., Tan, R.T.: Unsupervised night image enhancement: When layer decomposition meets light-effects suppression. In: European Conference on Computer Vision. pp. 404–421. Springer (2022)
  * [25] Kennerley, M., Wang, J.G., Veeravalli, B., Tan, R.T.: 2pcnet: Two-phase consistency training for day-to-night unsupervised domain adaptive object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11484–11493 (2023)
  * [26] Kim, D., Wang, K., Saenko, K., Betke, M., Sclaroff, S.: A unified framework for domain adaptive pose estimation. In: European Conference on Computer Vision. pp. 603–620. Springer (2022)
  * [27] Kocabas, M., Karagoz, S., Akbas, E.: Multiposenet: Fast multi-person pose estimation using pose residual network. In: Proceedings of the European conference on computer vision (ECCV). pp. 417–433 (2018)
  * [28] Kuhn, H.W.: The hungarian method for the assignment problem. Naval research logistics quarterly 2(1-2), 83–97 (1955)
  * [29] Lee, S., Rim, J., Jeong, B., Geonu Kim, B.W., Lee, H., Cho, S., Kwak, S.: Human pose estimation in extremely low-light conditions. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2023)
  * [30] Li, C., Lee, G.H.: From synthetic to real: Unsupervised domain adaptation for animal pose estimation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 1482–1491 (2021)
  * [31] Li, J., Wang, C., Zhu, H., Mao, Y., Fang, H.S., Lu, C.: Crowdpose: Efficient crowded scenes pose estimation and a new benchmark. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 10863–10872 (2019)
  * [32] Li, M., Liu, J., Yang, W., Sun, X., Guo, Z.: Structure-revealing low-light image enhancement via robust retinex model. IEEE Transactions on Image Processing 27(6), 2828–2841 (2018)
  * [33] Lin, H., Zhang, Y., Qiu, Z., Niu, S., Gan, C., Liu, Y., Tan, M.: Prototype-guided continual adaptation for class-incremental unsupervised domain adaptation. In: European Conference on Computer Vision. pp. 351–368. Springer (2022)
  * [34] Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: European conference on computer vision. pp. 740–755. Springer (2014)
  * [35] Moran, S., Marza, P., McDonagh, S., Parisot, S., Slabaugh, G.: Deeplpf: Deep local parametric filters for image enhancement. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 12826–12835 (2020)
  * [36] Mu, J., Qiu, W., Hager, G.D., Yuille, A.L.: Learning from synthetic animals. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 12386–12395 (2020)
  * [37] Newell, A., Huang, Z., Deng, J.: Associative embedding: End-to-end learning for joint detection and grouping. Advances in Neural Information Processing Systems 30 (2017)
  * [38] Peng, Q., Zheng, C., Chen, C.: Source-free domain adaptive human pose estimation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 4826–4836 (2023)
  * [39] Punnappurath, A., Abuolaim, A., Abdelhamed, A., Levinshtein, A., Brown, M.S.: Day-to-night image synthesis for training nighttime neural isps. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 10769–10778 (June 2022)
  * [40] Rahman, S., Rahman, M.M., Abdullah-Al-Wadud, M., Al-Quaderi, G.D., Shoyaib, M.: An adaptive gamma correction for image enhancement. EURASIP Journal on Image and Video Processing 2016(1), 1–13 (2016)
  * [41] Raychaudhuri, D.S., Ta, C.K., Dutta, A., Lal, R., Roy-Chowdhury, A.K.: Prior-guided source-free domain adaptation for human pose estimation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 14996–15006 (2023)
  * [42] Sharma, A., Tan, R.T.: Nighttime visibility enhancement by increasing the dynamic range and suppression of light effects. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11977–11986 (2021)
  * [43] Sun, K., Xiao, B., Liu, D., Wang, J.: Deep high-resolution representation learning for human pose estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 5693–5703 (2019)
  * [44] Tarvainen, A., Valpola, H.: Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. Advances in neural information processing systems 30 (2017)
  * [45] Tian, Z., Chen, H., Shen, C.: Directpose: Direct end-to-end multi-person pose estimation. arXiv preprint arXiv:1911.07451 (2019)
  * [46] Vu, T.H., Jain, H., Bucher, M., Cord, M., Pérez, P.: Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 2517–2526 (2019)
  * [47] Wang, D., Zhang, S.: Contextual instance decoupling for robust multi-person pose estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11060–11068 (2022)
  * [48] Wang, D., Zhang, S., Hua, G.: Robust pose estimation in crowded scenes with direct pose-level inference. Advances in Neural Information Processing Systems 34 (2021)
  * [49] Wang, Q., Fink, O., Van Gool, L., Dai, D.: Continual test-time domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7201–7211 (2022)
  * [50] Wang, R., Zhang, Q., Fu, C.W., Shen, X., Zheng, W.S., Jia, J.: Underexposed photo enhancement using deep illumination estimation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 6849–6857 (2019)
  * [51] Wang, S., Zheng, J., Hu, H.M., Li, B.: Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE transactions on image processing 22(9), 3538–3548 (2013)
  * [52] Wang, Y., Wan, R., Yang, W., Li, H., Chau, L.P., Kot, A.: Low-light image enhancement with normalizing flow. In: Proceedings of the AAAI conference on artificial intelligence. vol. 36, pp. 2604–2612 (2022)
  * [53] Wei, K., Fu, Y., Yang, J., Huang, H.: A physics-based noise formation model for extreme low-light raw denoising. In: IEEE Conference on Computer Vision and Pattern Recognition (2020)
  * [54] Wei, K., Fu, Y., Zheng, Y., Yang, J.: Physics-based noise modeling for extreme low-light photography. IEEE Transactions on Pattern Analysis and Machine Intelligence 44(11), 8520–8537 (2021)
  * [55] Weng, C.Y., Curless, B., Kemelmacher-Shlizerman, I.: Photo wake-up: 3d character animation from a single photo. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 5908–5917 (2019)
  * [56] Xiao, B., Wu, H., Wei, Y.: Simple baselines for human pose estimation and tracking. In: Proceedings of the European conference on computer vision (ECCV). pp. 466–481 (2018)
  * [57] Xie, R., Wang, C., Zeng, W., Wang, Y.: An empirical study of the collapsing problem in semi-supervised 2d human pose estimation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 11240–11249 (2021)
  * [58] Xue, N., Wu, T., Xia, G.S., Zhang, L.: Learning local-global contextual adaptation for multi-person pose estimation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2022)
  * [59] Yan, S., Xiong, Y., Lin, D.: Spatial temporal graph convolutional networks for skeleton-based action recognition. In: Thirty-second AAAI conference on artificial intelligence (2018)
  * [60] Zhang, S.H., Li, R., Dong, X., Rosin, P., Cai, Z., Han, X., Yang, D., Huang, H., Hu, S.M.: Pose2seg: Detection free human instance segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 889–898 (2019)