# Modeling of a multiple source heating plate

Stephan Scholz, Lothar Berger
Control and Process Engineering
RWU Ravensburg-Weingarten University of Applied Sciences
Weingarten, Germany
Email to<EMAIL_ADDRESS>
Web: https://www.rwu.de/en/research/labs/control-and-process-engineering

###### Abstract

Heating plates describe the transfer of heat from actuators to a target
object. In other words, they separate the heat sources and heated object and
can be further used to apply a specific heat distribution on this object.
Therefore, an exact description of their thermal dynamics and an efficient
coordination of their actuators is necessary to achieve a desired time-
dependent temperature profile accurately. In this contribution, the thermal
dynamics of a multiple source heating plate is modeled as a quasi-linear heat
equation and the configuration of the spatially distributed actuators and
sensors are discussed. Furthermore, the distributed parameter system is
approximated using a Finite Volume scheme, and the influence of the actuators’
spatial characterization on the plate’s thermal dynamics is studied with the
resulting high-dimensional system.

_Keywords_ Heating plate $\cdot$ Multiple sources $\cdot$ Heat equation

## Introduction

Thermal processing for advanced manufacturing, like semiconductor fabrication
[1] , additive manufacturing [2], or biotechnology processing [3], requires
ever higher precise temperature controllability, temperature ramp-up
performance, and temperature profile stability. This is achieved by thermal
systems with multiple spatially distributed heating sources, and cooling, for
example, multiple source heating plates [4].

Since existing physical models are insufficient to achieve the required
precision, existing control schemes for multiple source heating plates depend
on data-driven models [5] or various lumped models [6]-[12].

Existing control schemes for multiple source heating plates also utilize
offline calibration methods like wireless temperature sensor arrays [5]
because no observability models for multiple spatially distributed heating
sources were known. Meanwhile, there exist observer design based approaches
for one-dimensional thermal problems using boundary [13, 14] or in-domain
control and measurement [15], however if these are applicable to multiple
spatially distributed heating sources problems, requires further
investigation.

Hence, renewed interest has come to physical modeling and designing efficient
solvers for multiple spatially distributed heating sources thermal processing
[16]. Due to ever increasing process requirements, there is also renewed
interest in observability modeling [17] to eliminate the need for offline
calibration methods.

Therefore, modeling and control of the heat equation [18] and related
reaction-diffusion equation [19] \- and observability [20] \- were discussed
extensively in recent years. For example, in [21] the authors approximate a
two-dimensional quasi-linear heat equation using the finite difference method
and apply on its resulting high-dimensional nonlinear system flatness-based
methods to reach trajectory planning. The authors of [22, 23] construct a
sliding-mode controller and observer for a cylindrical rapid thermal
processing system which is modeled as a nonlinear one-dimensional heat
equation.

In this work, the thermal dynamics of a cuboid heating plate is introduced as
a three-dimensional distributed parameter system in the next section with
boundary conditions. The idea of actuator- and sensor-typical spatial
characterization is introduced in section two and a numerical two-dimensional
example which illustrates the influence of the actuators’ spatial
characterization is discussed from section three to five.

This work states first ideas to model and control the quasi-linear heat
equation in two (and three) dimensions with actuators and sensors limited by
their spatial characterization. This publication neither states a complete
analytical or numerical treatment of the quasi-linear heat equation, nor yet
an advanced controller design.

##  1 Problem formulation

The heating plate is assumed as an open cuboid

$\displaystyle\Omega:=\left(0,L\right)\times\left(0,W\right)\times\left(0,H\right)\subset\mathbb{R}^{3}$

with a constant length $L>0$, width $W>0$ and height $H>0$. The cuboids
surface $\partial\Omega:=B_{U}\cup B_{T}\cup B_{L}$ is subdivided in the
disjoint areas $B_{U}$ on the underside, $B_{T}$ on the topside and $B_{L}$ on
the lateral surfaces (see Figure 1). The cuboid’s temperature
$\vartheta:\left[0,T_{final}\right]\times\overline{\Omega}\rightarrow\mathbb{R}_{\geq
0}$ in Kelvin varies in time and space depending on the induced $\Phi_{in}$
and emitted heat flux $\Phi_{out}$ on $B_{U}$ and on $B_{T}\cup B_{L}$,
respectively. The heating plate’s physical properties

  * •

thermal conductivity $\lambda:\mathbb{R}_{\geq 0}\rightarrow\mathbb{R}_{\geq
0}$ in $\left[\frac{W}{m\leavevmode\nobreak\ K}\right]$,

  * •

specific heat capacity $c:\mathbb{R}_{\geq 0}\rightarrow\mathbb{R}_{\geq 0}$
in $\left[\frac{J}{kg\leavevmode\nobreak\ K}\right]$ and

  * •

density $\rho:\mathbb{R}_{\geq 0}\rightarrow\mathbb{R}_{\geq 0}$ in
$\left[\frac{kg}{m^{3}}\right]$

are assumed as continuous functions of the temperature.

The evolution of heat in the plate is modeled as quasi-linear heat equation

$\displaystyle\rho(\vartheta(t,x))\leavevmode\nobreak\
c(\vartheta(t,x))\leavevmode\nobreak\ \frac{\partial\vartheta(t,x)}{\partial
t}\leavevmode\nobreak\ =\leavevmode\nobreak\
\operatorname{div}\left[\lambda(\vartheta(t,x))\leavevmode\nobreak\
\nabla\vartheta(t,x)\right]$ (1)

for $(t,x)\in\left(0,T_{final}\right)\times\Omega$ with initial value
$\vartheta(0,x)=\vartheta_{init}(x)$ for $x\in\overline{\Omega}$ and boundary
condition

$\displaystyle\lambda(\vartheta(t,x))\leavevmode\nobreak\
\frac{\partial\vartheta(t,x)}{\partial x}\cdot\vec{n}\leavevmode\nobreak\
=\leavevmode\nobreak\ \begin{cases}\phi_{in}(t,x)&\leavevmode\nobreak\
\text{on}\leavevmode\nobreak\ B_{U}\text{,}\\\
\phi_{out}(t,x)&\leavevmode\nobreak\ \text{on}\leavevmode\nobreak\ B_{T}\cup
B_{L}\text{.}\end{cases}$ (2)

Vector $\vec{n}$ denotes the normal vector on the described boundary. Flux
$\phi_{in}$ denotes the induced heat flux on the underside of the heating
plate and $\phi_{out}$ denotes the emitted heat flux on the topside and the
lateral surfaces. The induced heat is stored in the plate and emitted via
linear heat conduction

$\displaystyle-h\leavevmode\nobreak\
\left[\vartheta(t,x)-\vartheta_{amb}(x)\right]$

with coefficient $h\geq 0$ in $\left[\frac{W}{m^{2}\leavevmode\nobreak\
K}\right]$ and nonlinear heat radiation

$\displaystyle-\epsilon(\vartheta(t,x))\varrho\leavevmode\nobreak\
\left[\vartheta(t,x)^{4}-\vartheta_{amb}(x)^{4}\right]$

with temperature-dependent emissivity $\epsilon:\mathbb{R}_{\geq
0}\rightarrow\left[0,1\right]$, Stefan-Boltzmann constant $\varrho\approx
5.67\cdot 10^{-8}$ in $\left[\frac{W}{m^{2}\leavevmode\nobreak\ K^{4}}\right]$
and ambient temperature $\vartheta_{amb}:B_{T}\cup
B_{U}\rightarrow\mathbb{R}_{\geq 0}$ (see also [24]). The emitted heat flux
results in a sum of both terms as

$\displaystyle\phi_{out}(t,x):=-h\leavevmode\nobreak\
\left[\vartheta(t,x)-\vartheta_{amb}(x)\right]-\epsilon(\vartheta(t,x))\varrho\leavevmode\nobreak\
\left[\vartheta(t,x)^{4}-\vartheta_{amb}(x)^{4}\right]$ (3)

for $(t,x)\in\left[0,T_{final}\right]\times B_{T}\cup B_{U}$.

$L$

$H$

$B_{T}$$B_{L}$$B_{U}$ Figure 1: Front view on heating plate with length L and height H. Boundaries $B_{U}$, $B_{T}$ and $B_{L}$ denote the underside, topside and lateral surfaces. Actuators are applied on the underside and sensors are assumed on the topside. Table 1: ASSUMED PHYSICAL PROPERTIES AND CONSTANTS Coefficient | Symbol | Value
---|---|---
Mass density | $\rho$ | $7800$
Heat capacity | $\left\\{c_{0},c_{1}\right\\}$ | $\left\\{330,0.4\right\\}$
Thermal conductivity | $\left\\{\lambda_{0},\lambda_{1}\right\\}$ | $\left\\{10,0.1\right\\}$
Heat transfer | $h$ | $10$
Emissivity | $\varepsilon$ | $0.6$
Ambient temperature | $\vartheta_{amb}$ | $300$

##  2 Spatial characterization

It is assumed that the underside $B_{U}$ has a partition $\beta_{n}\subset
B_{U}$ with $\bigcap\limits_{n=0}^{N_{u}-1}\beta_{n}=\left\\{\right\\}$ and
$\bigcup\limits_{n=0}^{N_{u}-1}\beta_{n}=B_{U}$ where $N_{u}$ denotes the
number of heating elements (see Figure 2). Each heating element has a spatial
characterization $b_{n}:B_{U}\rightarrow\left[0,1\right]$ with restrictions

$\displaystyle b_{n}(x)\geq$ $\displaystyle\leavevmode\nobreak\
0\quad\text{for}\leavevmode\nobreak\ x\in\beta_{n}\text{,}$ $\displaystyle
b_{n}(x)=$ $\displaystyle\leavevmode\nobreak\
0\quad\text{for}\leavevmode\nobreak\ x\in B_{U}\setminus\beta_{n}$

to describe the physical nature of the actuator including imperfection or
abrasion. Each partition $\beta_{j}$ correspond to the spatial
characterization $b_{j}$ and thus to its heating element and input signal
$u_{j}$. This means only separated inputs without any superposition are
considered. Feedback design with Backstepping [25, 26] or optimal control [27]
can be found in the literature and will not be discussed here. The $N_{u}$
actuators are assumed to have input signals
$u_{n}:\left[0,T_{final}\right)\rightarrow[u_{min},u_{max}]$ with the bounds
$-\infty<u_{min}<u_{max}<\infty$. The induced heat flux is described by

$\displaystyle\Phi_{in}(t,x):=b(x)^{\top}\leavevmode\nobreak\ u(t)$

for $\left(t,x\right)\in\left[0,T_{final}\right)\times B_{U}$ and with

$\displaystyle
b(x)=\left(b_{0}(x),b_{1}(x),\cdots,b_{N_{u}-1}(x)\right)^{\top}$

and

$\displaystyle
u(t)=\left(u_{0}(t),u_{1}(t),\cdots,u_{N_{u}-1}(t)\right)^{\top}\text{.}$

$L$

$W$

$\beta_{1}$$\beta_{2}$$\beta_{3}$$\beta_{4}$$\beta_{5}$$\beta_{6}$$\beta_{7}$$\beta_{8}$
Figure 2: Example partition of underside $B_{U}$. Each actuator belongs to one
partition $\beta$ and its spatial characterization is assumed as zero outside
its partition. Topside $B_{T}$ can be partitioned similar to the presented
one.

Analog to the plate’s underside, it is assumed that the temperature on topside
$B_{T}$ is measured by $N_{y}$ sensors where each of it correspond to a
partition $\gamma_{n}\subset B_{T}$ with
$n\in\left\\{0,1,\cdots,N_{y}-1\right\\}$. The sensor’s spatial
characterization $g_{n}:B_{T}\rightarrow\left[0,1\right]$ has to fulfill the
requirements

$\displaystyle g_{n}(x)\geq$ $\displaystyle\leavevmode\nobreak\
0\quad\text{for}\leavevmode\nobreak\ x\in\gamma_{n}\text{,}$ $\displaystyle
g_{n}(x)=$ $\displaystyle\leavevmode\nobreak\
0\quad\text{for}\leavevmode\nobreak\ x\in B_{T}\setminus\gamma_{n}\text{.}$

and additionally

$\displaystyle 0<\int_{\gamma_{n}}g_{n}(x)\operatorname{d}x<\infty\text{.}$

The system output at the n-th sensor is defined by

$\displaystyle
y_{n}(t):=\frac{1}{\int_{\gamma_{n}}g_{n}(x)\operatorname{d}x}\leavevmode\nobreak\
\int_{\gamma_{n}}g_{n}(x)\leavevmode\nobreak\
\vartheta(t,x)\operatorname{d}x\text{.}$ (4)

One notes, that the system output is bounded with $0\leq y_{n}(t)<\infty$ for
all $n\in\left\\{0,1,\cdots,N_{y}-1\right\\}$ and
$t\in\left(0,T_{final}\right]$.

As mentioned above, the choice of $b_{n}$ and $g_{n}$ depend strongly on the
nature of the actuators and sensors. Here, the prototype formula

$\displaystyle b_{n}(x)=\begin{cases}m\cdot\exp\left(-\lVert
M(x-x_{c,n})\rVert^{\nu}\right)\quad&\text{for}\quad x\in\beta_{n}\text{,}\\\
0\quad&\text{for}\quad x\in B_{U}\setminus\beta_{n}\end{cases}$ (5)

with $m\in[0,1]$, $M\in\mathbb{R}^{2\times 2}$, $\nu\geq 0$ and central point
$x_{c,n}\in\beta_{n}$ is chosen to describe the actuator’s spatial
characterization. The same formula (5) is used to describe the sensor’s
spatial characterization $g_{n}$ on $B_{T}$. In the case of $m=1$ and
$M=0_{2\times 2}$ one holds the indicator function

$\displaystyle\tilde{b}_{n}(x)\leavevmode\nobreak\ =\leavevmode\nobreak\
\begin{cases}1\quad&\text{for}\leavevmode\nobreak\ x\in\beta_{n}\text{,}\\\
0\quad&\text{for}\leavevmode\nobreak\ x\in
B_{U}\setminus\beta_{n}\text{.}\end{cases}$

##  3 Application to reduced two-dimensional case

The heating plate is reduced to the two-dimensional side view
$\Omega=\left(0,L\right)\times\left(0,H\right)$ with $L=0.30$ and $H=0.01$
meter. The density $\rho$ is assumed as constant, whereas the specific heat
capacity and thermal conductivity are assumed as linear functions
$c(\theta)=\leavevmode\nobreak\ c_{0}+c_{1}\theta$ and
$\lambda(\theta)=\leavevmode\nobreak\ \lambda_{0}+\lambda_{1}\theta$, see
Table 1. The heat equation in conservative form is noted as

$\displaystyle\int\limits_{\Omega}\rho\leavevmode\nobreak\
c(\vartheta(t,x))\leavevmode\nobreak\ \frac{\partial\vartheta(t,x)}{\partial
t}\operatorname{d}x\leavevmode\nobreak\ =\leavevmode\nobreak\
\int\limits_{\Omega}\operatorname{div}\left[\lambda(\vartheta(t,x))\leavevmode\nobreak\
\nabla\vartheta(t,x)\right]\operatorname{d}x$ (6)

with $\vartheta(0,x)=\vartheta_{init}(x)$ for all $x\in\Omega$ and boundary
conditions as described by Equations (2) and (3) with constants as in Table 1.
The initial temperature variation is assumed as $\vartheta(0,x)=300+\delta(x)$
including a noise term

$\displaystyle\delta(x)=a_{0}\leavevmode\nobreak\
\cos\left(2\pi\leavevmode\nobreak\
a_{1}\frac{x_{1}}{L}\right)\cos\left(2\pi\leavevmode\nobreak\
a_{2}\frac{x_{2}}{H}\right)$

with $a_{0}=3$, $a_{1}=10$ and $a_{2}=5$. The number of heating elements and
sensors is assumed as $N_{u}=N_{y}=5$ and a naive proportional controller

$\displaystyle u_{n}(t):=\begin{cases}K_{p,n}\leavevmode\nobreak\
e(t)\quad&\text{for}\leavevmode\nobreak\ e(t)>0\\\
0\quad&\text{otherwise}\end{cases}$

with $e_{n}(t):=y_{ref}-y_{n}(t)$, $K_{p,n}=10^{4}$ for
$n\in\left\\{0,1,\cdots,4\right\\}$ is designed to reach a constant reference
temperature $y_{ref}=400$ Kelvin. The partitions of $B_{U}$ and $B_{T}$ are
defined by

$\displaystyle\beta_{n}:=$ $\displaystyle\leavevmode\nobreak\
\left(n\leavevmode\nobreak\ \frac{L}{N_{u}},(n+1)\leavevmode\nobreak\
\frac{L}{N_{u}}\right)\times\left\\{0\right\\}\quad\text{and}$
$\displaystyle\gamma_{n}:=$ $\displaystyle\leavevmode\nobreak\
\left(n\leavevmode\nobreak\ \frac{L}{N_{y}},(n+1)\leavevmode\nobreak\
\frac{L}{N_{y}}\right)\times\left\\{L\right\\}\text{.}$

Two test scenarios are considered for the actuators: the nominal and the
realistic case with different values for the spatial characterization, see
Table 2. The sensors’ spatial characterization is the same for both scenarios.

Table 2: SPATIAL CHARACTERIZATION FOR TEST SCENARIOS | m | M | $\nu$
---|---|---|---
Sensors | $1.0$ | $10.0$ | $4.0$
Actuators (nominal) | $1.0$ | $0.0$ | $4.0$
Actuators (realistic) | $1.0$ | $30.0$ | $4.0$

##  4 Spatial approximation

The plate’s volume $\Omega$ is discretized with $J\times K$ cells where $J$
and $K$ denote the number of cells in $x_{1}$\- and $x_{2}$-direction,
respectively. The finite volume method

$\displaystyle p(t,x^{j,k})=\frac{1}{\Delta x_{1}\Delta
x_{2}}\int\limits_{x_{2}^{k-\frac{1}{2}}}^{x_{2}^{k+\frac{1}{2}}}\int\limits_{x_{1}^{j-\frac{1}{2}}}^{x_{1}^{j+\frac{1}{2}}}p(t,x)\operatorname{d}x_{1}\operatorname{d}x_{2}$

with $\lvert\Omega_{j,k}\rvert=\Delta x_{1}\Delta x_{2}$ is used to
approximate heat equation (6) in each cell $\Omega_{j,k}$ with
$j\in\left\\{0,1,\cdots,J-1\right\\}$ and
$k\in\left\\{0,1,\cdots,K-1\right\\}$. The left-hand side of the heat equation
results in

$\displaystyle\frac{1}{\lvert\Omega_{j,k}\rvert}\int\limits_{\Omega_{j,k}}\rho\leavevmode\nobreak\
c(\vartheta(t,x))\leavevmode\nobreak\
\dot{\vartheta}(t,x)\operatorname{d}x\approx\rho\leavevmode\nobreak\
c(\vartheta(t,x^{j,k}))\leavevmode\nobreak\
\dot{\vartheta}(t,x^{j,k})\text{.}$

The heat flux vector
$q(\vartheta(t,x))=\lambda(\vartheta(t,x))\nabla\vartheta(t,x)$ with its
elements

$\displaystyle
q_{i}(\vartheta(t,x))=\lambda(\vartheta(t,x))\frac{\partial}{\partial
x_{i}}\vartheta(t,x)$

are introduced to simplify the notation. Likewise, new (relative) coordinates

$\displaystyle\tilde{x}:=x^{j,k}=\begin{pmatrix}\left[j+\frac{1}{2}\right]\Delta
x_{1}\\\\[4.30554pt] \left[k+\frac{1}{2}\right]\Delta x_{2}\end{pmatrix}$

and $\tilde{x}+\delta x_{1}=x^{j+1,k}$ with $\delta x_{1}:=\left(\Delta
x_{1},0\right)^{\top}$ are established. The right-hand side of Equation (6) is
separated for each flux as

$\displaystyle\frac{1}{\lvert\Omega_{j,k}\rvert}\int\limits_{\Omega_{j,k}}\operatorname{div}\left[\lambda(\vartheta(t,x))\nabla\vartheta(t,x)\right]\operatorname{d}x\leavevmode\nobreak\
=$ $\displaystyle\leavevmode\nobreak\
\frac{1}{\lvert\Omega_{j,k}\rvert}\int\limits_{\Omega_{j,k}}\operatorname{div}\left[q(\vartheta(t,x))\right]\operatorname{d}x$
$\displaystyle\leavevmode\nobreak\ =$ $\displaystyle\leavevmode\nobreak\
\frac{1}{\lvert\Omega_{j,k}\rvert}\int\limits_{\Omega_{j,k}}\frac{\partial}{\partial
x_{1}}q_{1}(\vartheta(t,x))+\frac{\partial}{\partial
x_{2}}q_{2}(\vartheta(t,x))\operatorname{d}x\text{.}$

From here on, only the flux $q_{1}$ is considered - the calculations for
$q_{2}$ work similar. The integral over cell $\Omega_{j,k}$ is solved and the
derivatives $\frac{\partial q_{1}}{\partial x_{1}}$ and
$\frac{\partial\vartheta}{\partial x_{1}}$ are approximated using finite
differences. Consequently, one yields

$\displaystyle\frac{1}{\lvert\Omega_{j,k}\rvert}\int\limits_{\Omega_{j,k}}\frac{\partial}{\partial
x_{1}}q_{1}(\vartheta(t,x))\operatorname{d}x\leavevmode\nobreak\
=\leavevmode\nobreak\ \frac{\Delta
x_{2}}{\lvert\Omega_{j,k}\rvert}\left[q_{1}(\vartheta\left(t,\tilde{x}+\frac{\delta
x_{1}}{2}\right))-q_{1}(\vartheta\left(t,\tilde{x}-\frac{\delta
x_{1}}{2}\right))\right]$ $\displaystyle\quad\leavevmode\nobreak\
=\leavevmode\nobreak\ \frac{1}{\Delta
x_{1}^{2}}\left[\lambda(\vartheta\left(t,\tilde{x}+\delta
x_{1}/2\right))\leavevmode\nobreak\ \vartheta(t,\tilde{x}+\delta
x_{1})+\lambda(\vartheta\left(t,\tilde{x}-\delta
x_{1}/2\right))\leavevmode\nobreak\ \vartheta(t,\tilde{x}-\delta
x_{1})\right.$
$\displaystyle\qquad\qquad\left.-\left[\lambda(\vartheta\left(t,\tilde{x}+\delta
x_{1}/2\right))+\lambda(\vartheta\left(t,\tilde{x}-\delta
x_{1}/2\right))\right]\leavevmode\nobreak\
\vartheta(t,\tilde{x})\right]\text{.}$ (7)

The thermal conductivity at the cell boundaries are approximated with

$\displaystyle\lambda(\vartheta\left(t,\tilde{x}+\delta
x_{1}/2\right))\approx\lambda(\left[\vartheta\left(t,\tilde{x}\right)+\vartheta\left(t,\tilde{x}+\delta
x_{1}\right)\right]/2)\text{.}$

$x_{1}=0$$x_{1}=L$$x_{2}=0$$x_{2}=H$$-\frac{\partial\vartheta}{\partial
x_{1}}$$\frac{\partial\vartheta}{\partial
x_{1}}$$-\frac{\partial\vartheta}{\partial
x_{2}}$$\frac{\partial\vartheta}{\partial
x_{2}}$$j=-\frac{3}{2}$-$\frac{1}{2}$$\frac{1}{2}$$J-\frac{3}{2}$$J-\frac{1}{2}$$J+\frac{1}{2}$$k=-\frac{3}{2}$$-\frac{1}{2}$$\frac{1}{2}$$K-\frac{3}{2}$$K-\frac{1}{2}$$K+\frac{1}{2}$
Figure 3: Two-dimensional heating plate with heat flux at boundaries (above)
and Finite Volume grid (below).

###  4.1 Boundary conditions

The boundary conditions are taken into account to complete the approximation
as a high-dimensional ODE. Here, the cells at the left boundary with
$x_{1}=\frac{1}{2}$ and $\tilde{x}=x^{0,k}$ are considered and therefore one
holds

$\displaystyle\left.-\lambda(\vartheta(t,x))\frac{\partial\vartheta(t,x)}{\partial
x}\right\rvert_{x=\begin{pmatrix}0\\\
x_{2}\end{pmatrix}}\approx\lambda(\vartheta(t,\tilde{x}-\delta
x_{1}/2))\frac{1}{\Delta x_{1}}\left[\vartheta(t,\tilde{x}-\delta
x_{1})-\vartheta(t,\tilde{x})\right]=\phi_{out}(t,\tilde{x})$

and equivalently

$\displaystyle\vartheta(t,\tilde{x}-\delta
x_{1})=\vartheta(t,\tilde{x})+\frac{\Delta x_{1}\leavevmode\nobreak\
\phi_{out}(t,\tilde{x})}{\lambda(\vartheta(t,\tilde{x}-\delta
x_{1}/2))}\text{.}$

The temperature of the virtual cell $\vartheta(t,\tilde{x}-\delta x_{1})$ in
Equation (7) is substituted with the boundary conditions to yield

$\displaystyle\frac{1}{\lvert\Omega_{0,k}\rvert}\int\limits_{\Omega_{0,k}}\frac{\partial}{\partial
x_{1}}q_{1}(\vartheta(t,x))\operatorname{d}x\leavevmode\nobreak\ =$
$\displaystyle\leavevmode\nobreak\ \frac{1}{\Delta
x_{1}^{2}}\left[\lambda(\vartheta\left(t,\tilde{x}+\delta
x_{1}/2\right))\leavevmode\nobreak\ \vartheta(t,\tilde{x}+\delta
x_{1})-\right.$
$\displaystyle\qquad\left.\lambda(\vartheta\left(t,\tilde{x}+\delta
x_{1}/2\right))\leavevmode\nobreak\
\vartheta(t,\tilde{x})\right]+\frac{1}{\Delta x_{1}}\leavevmode\nobreak\
\phi_{out}(t,\tilde{x})\text{.}$

The same procedure is applied on the remaining boundaries with respect to the
emitted and induced heat fluxes $\phi_{out}$ and $\phi_{in}$. The variables at
the discrete points $x^{j,k}$ are summarized as vectors and matrices

  * •

$\Theta\in\mathbb{R}^{J\cdot K}$ for the temperature,

  * •

$C\in\mathbb{R}^{J\cdot K\times J\cdot K}$ for the specific heat capacity,

  * •

$\Lambda_{1},\Lambda_{2}\in\mathbb{R}^{J\cdot K\times J\cdot K}$ for the
thermal conductivity and

  * •

$\Phi_{in},\Phi_{out,x_{1}},\Phi_{out,x_{2}}\in\mathbb{R}^{J\cdot K}$ for the
induced and emitted heat flux.

The approximated heat dynamics is governed by the large-scale differential
equation

$\displaystyle\rho\leavevmode\nobreak\ C(\Theta)\leavevmode\nobreak\
\dot{\Theta}(t)\leavevmode\nobreak\ =\leavevmode\nobreak\
\left[\frac{1}{\Delta x_{1}^{2}}\Lambda_{1}(\Theta(t))+\frac{1}{\Delta
x_{2}^{2}}\Lambda_{2}(\Theta(t))\right]\Theta(t)$
$\displaystyle+\frac{1}{\Delta x_{1}}\Phi_{out,x_{1}}(t)+\frac{1}{\Delta
x_{2}}\left[\Phi_{in}(t)+\Phi_{out,x_{2}}(t)\right]$

with $\Theta(0)=\Theta_{init}$.

##  5 Simulation

The simulation is built with Julia [28] and the package
DifferentialEquations.jl [29]. The visualization is carried out with
PlotlyJS.jl. The source code of this simulation is available on GitHub:

github.com/stephans3/MultipleSourceHeatingPlate2D.jl.

The number of cells in horizontal and vertical axis are set to $J=100$ and
$K=40$ which corresponds to $\Delta x_{1}=3.0\cdot 10^{-3}$ and $\Delta
x_{2}=2.5\cdot 10^{-4}$ meter, the final Time is set to $T_{final}=10$ and the
sampling time is set to $\Delta t=10^{-3}$ seconds. The time integration was
solved by the forward Euler method.

Figure 4: Temperature distribution on topside $B_{T}$. The actuators’ spatial
characterization is transferred to the final temperature and can be observed
in case of the nominal actuators (Scenario 1) as almost constant (desired),
and in the case of realistic actuators (Scenario 2) as oscillating
(undesired).

The final temperature on topside $B_{T}$, depicted in Figure 4, shows an error
from the reference $y_{ref}=400$ of approximately one Kelvin for the first
scenario. However, in the second scenario the temperature distribution unveils
an oscillatory behavior along the horizontal axis. This means, the realistic
actuators’ spatial characterization cannot be compensated by the diffusive
character of the heat equation. Same results can be observed in Figure 5: the
temperature distributions - constant in Scenario 1 and oscillatory in Scenario
2 - are propagated through the two-dimensional plate. The naive controller
approach leads to minimal differences of the input signals in both scenarios
as presented in Figure 6. The input and output averages are computed with

$\displaystyle\overline{u}(t)=\frac{1}{N_{u}}\sum_{n=0}^{N_{u}-1}u_{n}(t)\quad\text{and}\quad\overline{y}(t)=\frac{1}{N_{y}}\sum_{n=0}^{N_{y}-1}y_{n}(t)\text{.}$

Figure 5: Two-dimensional temperature distribution of Scenario 1 (left) and 2
(right). The comparison unveils a distinct difference of the thermal behavior
in the whole plate between both scenarios.

Figure 6: Averaged input and output signals of Scenario 1 (left) and 2
(right). The graphs show a comparable trend, even though the thermal behaviors
are completely different in both cases.

Despite the comparable controller signals one has to note the distinct
differences of the final temperature between the first and second scenario.
Therefore, modeling systems like heating plates with multiple heat sources has
to incorporate distributed parameter systems (up to three spatial dimensions),
distributed actuators and sensors and their spatial characterization.
Furthermore, a suitable controller has to be designed that can deal large-
scale differential equations and the interaction between distributed actuators
and sensors.

##  6 Conclusion

The thermal dynamics of a heating plate with multiple sources was discussed in
the sense of a distributed parameter system in two (and three) dimensions.
This general approach opens up the opportunity to analyze the complete heat
evolution and the target temperature distribution in depth. The spatially
distributed actuators and sensors were modeled including their typical
characterization which has a significant influence on the heating process as
illustrated in the numerical example. Although, the measured output reaches
the reference temperature, the assumption of temperature-uniformity along the
horizontal axis had to be rejected at least in the case of realistic
actuators. This simple example points out the need to model thermal dynamics
as precisely as possible despite its complexity and high computational effort.
In future work the focus will lie on the development of a numerical framework
to simulate and control two- and three-dimensional thermal processes with
multiple heat sources on the underside and distributed sensors on the topside.
This includes the modeling from a physical and analytical point of view, the
spatial approximation with Finite Volumes and Galerkin approaches and the
design of an efficient modern control scheme to compensate the (realistic)
actuators’ spatial character. Finally, for verifying the examined thermal
dynamic models, a heating plate set-up with multiple spatial symmetric Joule
heating sources is available, which can be operated using either any single
source or $n\times n$ concurrent multiple sources with $n$ up to five.

## References

  * [1] Hong Xiao: _Introduction to Semiconductor Manufacturing Technology_. Chapter 5: Thermal Processes. SPIE Press, Bellingham 2012.
  * [2] Zhengying Wei, Jun Du: _Heat and Mass Transfer of Additive Manufacturing Processes for Metals_. Heat and Mass Transfer-Advances in Science and Technology Applications. IntechOpen, 2019.
  * [3] Zhiwei Zhou, Boyan Song, Likuan Zhu, Zuntao Li, Yang Wang: _Optimizing of Bioreactor Heat Supply and Material Feeding by Numerical Calculation_. International Conference on Intelligent Computing and Information Science. Springer, Berlin, Heidelberg, 2011.
  * [4] Lothar Berger, Peter Dress, Shun-Ho Yang, Chien-Hsien Kuo: _Qualification of design-optimized multizone hotplate for 45-nm node mask making_. Photomask and Next-Generation Lithography Mask Technology XIV. Vol. 6607. International Society for Optics and Photonics, 2007.
  * [5] Lothar Berger, Peter Dress, Thomas M. Gairing, Chia-Jen Chen, Ren-Guey Hsieh, Hsin-Chang Lee, Hung-Chang Hsieh: _Global critical dimension uniformity improvement for mask fabrication with negative-tone chemically amplified resists by zone-controlled postexposure bake_. Journal of Micro/Nanolithography, MEMS, and MOEMS 3.2 (2004): 203-211.
  * [6] A. Tay, K.K. Tan, S. Zhao, T.H. Lee: _Predictive Ratio Control of Multizone Thermal Processing System in Lithography_. IFAC Proceedings Volumes 41.2 (2008): 10863-10868.
  * [7] Yan Han: _Temperature sensing and control in multi-zone semiconductor thermal processing_. Diss. 2009. Available: https://scholarbank.nus.edu.sg/handle/10635/17322.
  * [8] Hiroto Hamane, Koudai Matuki, Fujio Hiroki, Kazuyoshi Miyazaki: _Thermal MIMO controller for setpoint regulation and load disturbance rejection_. Control engineering practice 18.2 (2010): 198-208.
  * [9] Endra Joelianto, Iqbal Ginanjar Prasetia: _Bake Plate Control using A Robust Multiplexed Model Predictive Control (RMMPC)_. 2011 2nd International Conference on Instrumentation Control and Automation. IEEE, 2011.
  * [10] Yong Feng: _Critical dimension and temperature control in multi-zone thermal processing_. Diss. 2011. Available: https://scholarbank.nus.edu.sg/handle/10635/35821.
  * [11] Poom Jatunitanon, Withit Chatlatanagulchai: _Multivariable Robust Control for Two–Zone Thermal Plate System_. TSME International Conference on Mechanical Engineering (2013).
  * [12] Poom Jatunitanon, Sarawoot Watechagit, Withit Chatlatanaguchai: _Robust multi-model predictive control of multi-zone thermal plate system_. Songklanakarin Journal of Science & Technology 40.1 (2018).
  * [13] Andrey Smyshlyaev, Miroslav Krstic: _Backstepping observers for a class of parabolic PDEs_. Systems & Control Letters 54.7 (2005): 613-625.
  * [14] T. Ahmed-Ali, F. Giri, M. Krstic, F. Lamnabhi-Lagarrigue, and L. Burlion: _Adaptive Observer for a Class of Parabolic PDEs_. IEEE Transactions on Automatic Control 61.10 (2015): 3083-3090.
  * [15] Tatiana Kharkovskaia, Denis Efimov, Emilia Fridman, Andrey Polyakov, Jean-Pierre Richard: _Interval observer design and control of uncertain non-homogeneous heat equations_. Automatica 111 (2020): 108595.
  * [16] Karsten Eppler, Fredi Tröltzsch: _Optimal Control Problems for the Nonlinear Heat Equation_ Online Optimization of Large Scale Systems. Springer, Berlin, Heidelberg, 2001. 173-183.
  * [17] Thomas Seidman: _Control of the heat equation_. The Control Handbook. CRC Press, Boca Raton (1996): 1157-1168. Available: http://www.math.umbc.edu/~seidman/Papers/ctrl_heq.pdf.
  * [18] B.K. Tan, X.Y. Huang, T.N. Wong, K.T. Ooi: _A study of multiple heat sources on a flat plate heat pipe using a point source approach_ , International journal of heat and mass transfer 43.20 (2000): 3755-3764.
  * [19] Yiming Song, Xiaoxiao Wang, Haipeng Teng, Yulei Guan: _Nonlinear parametric predictive control for the temperature control of bench-scale batch reactor_. Applied Thermal Engineering, Volume 102 (2016): 134-143.
  * [20] Abbas Tutcuoglu, Carmel Majidi, Wanliang Shan: _Nonlinear thermal parameter estimation for embedded internal Joule heaters_. International Journal of Heat and Mass Transfer 97 (2016): 412-421.
  * [21] Tilman Utz, Thomas Meurer, Andreas Kugi: _Trajectory planning for a two-dimensional quasi-linear parabolic PDE based on finite difference semi-discretizations_. IFAC Proceedings Volumes 44.1 (2011): 12632-12637.
  * [22] Tengfei Xiao, Han-XiongLi: _Sliding mode control design for a rapid thermal processing system_. Chemical Engineering Science 143 (2016): 76-85.
  * [23] Tengfei Xiao, Xiao-Dong Li, Shuqiang Wang: _Dominant-Modes-Based Sliding-Mode Observer for Estimation of Temperature Distribution in Rapid Thermal Processing System_. IEEE Transactions on Industrial Informatics 15.5 (2018): 2673-2681.
  * [24] Hans Dieter Baehr, Karl Stephan: _Heat and mass transfer_. Springer Science & Business Media, Berlin 2013.
  * [25] Thomas Meurer: _Control of higher–dimensional PDEs: Flatness and backstepping designs_. Springer Science & Business Media, Berlin 2012.
  * [26] Miroslav Krstic, and Andrey Smyshlyaev: _Boundary control of PDEs: A course on backstepping designs_. SIAM, Philadelphia 2008.
  * [27] Sabine Hein: _MPC/LQG-Based Optimal Control of Nonlinear Parabolic PDEs_. Diss. 2009. Available: https://nbn-resolving.org/urn:nbn:de:bsz:ch1-201000134.
  * [28] Jeff Bezanson, Alan Edelman, Stefan Karpinski, Viral B. Shah: _Julia: A fresh approach to numerical computing_. SIAM Review 59.1 (2017): 65-98.
  * [29] Christopher Rackauckas, et al.:_SciML/DifferentialEquations.jl: v6.15.0_. [Computer software]. Zenodo (2020). Available: https://doi.org/10.5281/ZENODO.3929125.

## License

This work is licensed under a Creative Commons “Attribution-ShareAlike 4.0
International” license.


# Taming Small-sample Bias in Low-budget Active Learning

Linxin Song 1 Jieyu Zhang 2 Xiaotian Lu 3 Tianyi Zhou 4

###### Abstract

Active learning (AL) aims to minimize the annotation cost by only querying a
few informative examples for each model training stage. However, training a
model on a few queried examples suffers from the small-sample bias. In this
paper, we address this small-sample bias issue in low-budget AL by exploring a
regularizer called Firth bias reduction, which can provably reduce the bias
during the model training process but might hinder learning if its coefficient
is not adaptive to the learning progress. Instead of tuning the coefficient
for each query round, which is sensitive and time-consuming, we propose the
curriculum Firth bias reduction (CHAIN) that can automatically adjust the
coefficient to be adaptive to the training process. Under both deep learning
and linear model settings, experiments on three benchmark datasets with
several widely used query strategies and hyperparameter searching methods show
that CHAIN can be used to build more efficient AL and can substantially
improve the progress made by each active learning query.

## Introduction

The last decade has witnessed the emergence of deep learning as the dominant
force in advancing machine learning and its applications. To reduce the data
annotation cost, researchers and practitioners have resorted to active
learning (AL), which aims to reduce the total number of annotations by
iteratively querying the most informative annotations for the learning process
from an oracle (Settles 2009; Schröder and Niekler 2020). AL uses a finite
budget to select and label a subset of a data pool for downstream supervised
learning. By iteratively training a model on a current set of labeled data and
querying labels for more new points, this paradigm yields models that use as
fewer data as possible to achieve nearly the same accuracy as classifiers with
unlimited labeling budgets.

Many traditional active learning approaches are based on uncertainty sampling
(e.g., Lewis (1995); Scheffer, Decomain, and Wrobel (2001); Culotta and
McCallum (2005); Joshi, Porikli, and Papanikolopoulos (2009a); Li and Guo
(2013a); Gissin and Shalev-Shwartz (2019); Sinha, Ebrahimi, and Darrell
(2019a)), wherein the learner queries the most uncertain examples for
annotations, presumably because such examples are most informative to the
learner. However, such a type of approach primarily relies on the quality of
estimated uncertainty. Besides, existing deep active learning strategies
require a large amount of labeled examples to work properly (Yuan, Lin, and
Boyd-Graber 2020; Pourahmadi, Nooralinejad, and Pirsiavash 2021). However, the
labeling budget might be limited in practice. Under this low-budget regime, it
is well-established that the conventional Maximum Likelihood Estimation (MLE)
used for training the model is biased with a term of $O(N^{-1})$ (Cox and
Snell 1968; Box 1971; Whitehead 1986; Steyerberg, Eijkemans, and Habbema
1999). Such small-sample bias in low-budget setup might deteriorate the
performance of uncertainty-based AL due to (1) an inaccurate model trained
with MLE; and (2) a poor uncertainty estimation caused by the inaccurate
model.

To tackle the small-sample bias in AL, we, for the first time, leverage the
Firth bias reduction (Firth 1993; Ghaffari et al. 2021) as a plug-in
regularizer for the MLE-based model training. It reduces the small-sample bias
by applying a regularizer in addition to the cross-entropy loss, which
computes the log-determinant of Fisher information matrix
$\log(\text{det}(F))$. However, the performance of the Firth regularizer is
sensitive to its coefficient (Ghaffari et al. 2021), a hyperparameter that
controls the strength of the regularizer. Thus, how to search for the optimal
coefficient becomes crucial when adopting the Firth regularizer. Grid search,
as a standard solution for hyperparameter search, requires repeating the
training process multiple times to search for the best coefficient. It is even
more time-consuming in AL since the model has to be re-trained after each time
the new queried data is added, and so does the hyperparameter search process.

To reduce the time complexity, we propose to optimize the Firth coefficient on
the fly without heavy parameter tuning. Specifically, we formulate the
optimization of both the model parameters and the Firth coefficient as a
bilevel optimization where the inner level optimization corresponds to the
model parameters, and the outer one optimizes the Firth coefficient. Such a
bilevel optimization can be solved efficiently by interleaving the gradient
descent steps of the model parameter and the Firth coefficient. In
experiments, it only requires a few gradient descent steps (e.g., 50) to
obtain a good coefficient.

Furthermore, the impact of the bias can vary drastically across different
stages of the training process, while over-regularization might degrade the
generalization performance, and under-regularization might not be able to
alleviate the small-sample bias issue effectively. Hence, the strength of
Firth regularization (i.e., the Firth coefficient) should be adaptively
adjusted rather than fixed. To this end, we propose a simple curriculum of the
Firth coefficient that dynamically adjust the Firth coefficient to be adaptive
to the model’s training process.

Finally, we incorporate the aforementioned components seamlessly in CHAIN
(Curriculum firtH biAs ReductIoN), which is a general query-strategy-agnostic
method for tackling the small-sample bias issue in low-budget AL effectively
and efficiently. We conduct extensive experiments on three benchmark datasets
with five widely used query strategies under deep learning and linear model
settings. Compared with the baselines, CHAIN achieves the highest performance
in most of the query rounds. To help understand the curriculum of the Firth
coefficient, we provide the analysis of the coefficient curriculum on both
deep model and logistic regression, showing that the curriculum exists during
training and is related to the size of the queried training set. We also
conduct ablation studies for major hyperparameters used in CHAIN to
demonstrate the consistent superiority of CHAIN under different hyperparameter
choices. Our main contributions are summarized as: (1) we propose a query-
strategy-agnostic framework that tackles the small-sample bias issue in low-
budget AL via a plug-in regularization and efficient optimization of the
coefficient of the regularizer, (2) we further improve the framework by a
simple curriculum of the coefficient that adaptively changes the coefficient
for the model in different training stages, and (3) experimental results on
three benchmark datasets with three widely used query strategies and two
hyperparameter searching methods showing that CHAIN has a distinct and robust
performance in most of the circumstances.

## Related Works

##### Low-budget Active Learning.

The goal of active learning (AL) is to render the best model given a limited
budget of annotations. One of the fundamental components in AL is the query
strategy, which selects a subset of instances from the pool of unlabeled data
and queries their labels from an oracle (Settles 2009). Existing query
strategies can be categorized into uncertainty-based (Li and Guo 2013b; Roth
and Small 2006; Wang and Shang 2014), representation-based (Sener and Savarese
2017a; Contardo, Denoyer, and Artières 2017; Yu, Bi, and Tresp 2006) or hybrid
(Sener and Savarese 2017b; Shui et al. 2020; Sinha, Ebrahimi, and Darrell
2019b). However, in the low-budget regime (e.g., less than 400 instances are
labeled), the sub-optimality of the heuristic can lead to significant loss
(Mahmood, Fidler, and Law 2021), which is called cold-start phenomenon. To
tackle the cold-start issue, Mahmood, Fidler, and Law (2021) proposed a core
set query strategy which minimized the discrete Wasserstein distance between
the selected and unlabeled, while Hacohen, Dekel, and Weinshall (2022)
proposed a clustering algorithm for querying most typical instances from the
max density region with diversity. Both of these methods partially alleviated
the cold-start phenomenon by proposing new query strategies but overlooked the
small-sample bias issue brought by using MLE in model training. In this work,
instead of developing a new query strategy to compete with existing ones, we
focus on the small-sample bias during model training and propose a general
method that could improve over a given query strategy in low-budget AL.

##### Firth Bias Reduction.

Firth bias reduction focuses on reducing the bias caused by maximum likelihood
estimators while the dataset is small, known as Firths’ Penalized Maximum
Likelihood Estimator (PMLE) (Firth 1993). In the case of the exponential
family of distributions, it has a simplified form that penalizes the
likelihood by Jeffrey’s invariant prior (Firth 1993; Poirier 1994), which is
proportional to the determinant of the Fisher Information Matrix $F$. For
logistic and cosine classifiers (Chen et al. 2019), which are widely used in
few-shot classification, since they belong to the exponential family, Firth
bias reduction can be further cast as adding a log-determinant penalty
($\log(\det(F))$) to the cross-entropy loss. In this work, we, for the first
time, explore the utility of Firth bias reduction in reducing the small-sample
bias in low-budget AL.

## Preliminary

Assume a $K$-way classification task and let
$\mathcal{U}=\\{\textbf{x}_{1},...,\textbf{x}_{N}\\}$ be the input unlabeled
dataset with unknown ground truth $Y=[y_{1},...,y_{N}]$. Given a limited
budget of annotations, the goal of active learning (AL) is to wisely spend the
budget on querying annotations from an oracle (e.g., human expert) for model
training. Specifically, we have $R>1$ rounds of querying, and each time we
query annotations for $M$ instances according to some query strategy.

One of the popular query strategies is the uncertainty-based query, wherein
different measurements of the model’s uncertainty (e.g., entropy, the least
confidence) on each instance can be leveraged. Given a specific query strategy
$Q$, at round $r$ we select the top-$M$ uncertain instances from the remaining
unlabeled instances. We include them in training set after obtaining their
labels from the oracle (Li and Guo 2013b; Roth and Small 2006; Wang and Shang
2014). The resultant training set $\mathcal{D}_{r}$ in round $r$ is in turn
used to train a classifier $f_{r}(\cdot;\bm{\beta}_{r})$ parameterized by
$\bm{\beta}_{r}$. We would omit the subscript $r$ when there is no confusion.
In this work, we focus on low-budget AL, where the budget is very low compared
to the size of the pool of unlabeled data. In such a regime, each time model
is trained with only a small set of labeled data, and usually with the Maximum
Likelihood Estimation (MLE) (i.e., the cross-entropy loss). As a consequence,
the optimized model parameter $\bm{\beta}$ would be biased (Firth 1993) and
hurt both the model performance and the quality of the queries in the latter
rounds, especially those uncertainty-based query strategies that largely rely
on the trained model of last round to estimate the uncertainty of remaining
unlabeled instances.

## Methods

In this section, we describe the proposed method CHAIN (Curriculum firtH biAs
reductIoN) that aims to tackle the small-sample bias issue in low-budget
active learning. First, we show how to leverage the Firth bias reduction
technique as a plug-in regularizer to reduce the small-sample bias in the
model training procedure at each round of active learning. Then, we propose an
efficient approach to optimize the coefficient of the Firth regularizer
without heavy parameter tuning. Finally, we present a simple curriculum of the
coefficient that adaptively adjusts the strength of the regularizer for the
model at different training stages.

### Firth Bias Reduction

Based on (Firth 1993; Ghaffari et al. 2021), in this section, we introduce a
penalized cross-entropy loss by adding the Firth regularizer to reduce the
small-sample bias. We denote the parameter of the logistic model as
$\bm{\beta}$, and the assignment probability of the $i^{th}$ data to class $k$
can be represented as

$\textbf{P}_{i,k}=\text{Pr}(y_{i}=k|\textbf{x}_{i};\bm{\beta})=\frac{\exp(\beta_{k}^{\top}\textbf{x}_{i})}{\sum_{k^{\prime}=0}^{K}\exp(\beta_{k^{\prime}}^{\top}\textbf{x}_{i})}.$
(1)

The likelihood of the current training set $\mathcal{D}$ given the
$\bm{\beta}$ is

$\text{Pr}(\textbf{y}|\mathcal{D};\bm{\beta})=\prod_{i=1}^{N}\sum_{k=0}^{K}\mathds{1}[y_{i}=k]\cdot\textbf{P}_{i,k},$
(2)

where $\mathds{1}[\cdot]$ denotes the binary indicator function. We use the
maximum likelihood estimation (MLE) to acquire the optimal parameter
$\hat{\bm{\beta}}_{\text{MLE}}$, which can be defined as

$\hat{\bm{\beta}}_{\text{MLE}}=\underset{\bm{\beta}}{\text{arg max}}\
\text{Pr}(\textbf{y}|\textbf{x};\bm{\beta}).$ (3)

Oftentimes the cross-entropy (CE) loss is used as the objective of the MLE,
which can be interpreted as the log of Eq. 2. This draws out the optimization
objective

$\displaystyle\mathcal{L}_{\text{CE}}=-\sum_{i=1}^{N}l_{\text{CE}}(y_{i}\|\textbf{P}_{i,k})=-\sum_{i=1}^{N}\sum_{k=0}^{K}\mathds{1}[y_{i}=k]\log(\textbf{P}_{i,k}).$
(4)

In logistic models, the penalized likelihood function proposed by Firth is
equivalent to imposing Jeffery’s prior (Poirier 1994) on the parameters and
making a maximum posterior estimation as

$\text{Pr}(\bm{\beta}|\mathcal{D},\textbf{y})=\frac{1}{Z}\cdot\text{Pr}(\textbf{y}|\mathcal{D};\bm{\beta})\cdot\text{det}(F|r)^{\frac{1}{2}}$
(5)

where $Z$ is a normalization constant and $\text{det}(F|r)^{\frac{1}{2}}$ is
the Jeffery’s prior, the product of all $r$ non-zero eigenvalues of the Fisher
information matrix $F$ to the power $1/2$, and $F$ is defined as the Hessian
of the negative log-likelihood function with

$F=-\text{Hess}_{\bm{\beta}}(\mathcal{L}_{\text{CE}})=\mathbb{E}_{y}[\nabla_{\bm{\beta}}\mathcal{L}_{\text{CE}}\cdot\nabla_{\bm{\beta}}\mathcal{L}_{\text{CE}}^{\top}].$
(6)

Taking the log of both sides of Eq.5 yields the log-posterior as a sum of the
cross-entropy loss and the Firth bias reduction term

$\mathcal{L}(\mathcal{D};\bm{\beta})=\log(\text{Pr}(\bm{\beta}|\mathcal{D},\textbf{y}))=\mathcal{L}_{\text{CE}}(\mathcal{D};\bm{\beta})+\lambda\cdot\mathcal{L}_{\text{Firth}}(\mathcal{D};\bm{\beta}),$
(7)

where

$\mathcal{L}_{\text{Firth}}=\frac{1}{2}\log(\text{det}(F|r))+\text{const},$
(8)

and $\lambda$ is the coefficient of the regularizer. Following (Ghaffari et
al. 2021), we further acquire an easy-to-implement optimization objective for
the Firth regularizer. We firth decompose $F$ as

$F_{(dK)\times(dK)}=X^{\top}_{(dK)\times(NK)}\cdot M_{(NK)\times(NK)}\cdot
X_{(NK)\times(dK)},$

where $d$ is the dimension of data feature space, $M$ is a block-diagonal
matrix whose $i^{th}$ diagonal block is denoted as $M_{i}$. By adopting the
Determinant Amendment and Constant Dropping (Ghaffari et al. 2021), and using
the Matrix-Determinant Lemma (Harville 1998), it comes out that

$\displaystyle\mathcal{L}_{\text{Firth}}$
$\displaystyle=\frac{1}{N}\sum_{i=1}^{N}\log(\text{det}(M_{i}))+\text{Const}$
$\displaystyle=\frac{1}{N}\sum_{i=1}^{N}\sum_{k=0}^{K}\log(\textbf{P}_{i,k}).$
(9)

It is well known that Jeffery’s prior encourages uniform class assignment
probabilities (Firth 1993). Therefore, we treat the
$\sum_{k=0}^{K}\log(\textbf{P}_{i,k})$ as a scaled average over a uniform
distribution of the number of classes

$\mathcal{L}_{\text{Firth}}\propto\frac{1}{N}\sum_{i=1}^{N}\left[\sum_{k=0}^{K}\frac{1}{K+1}\cdot\log(\textbf{P}_{i,k})\right],$
(10)

and it further comes out that

$\mathcal{L}_{\text{Firth}}=\frac{-1}{N}\sum_{i=1}^{N}D_{\text{KL}}(\text{U}_{[0,K]}\|\textbf{P}_{i,k})+\text{Const},$
(11)

where $\text{U}_{[0,K]}$ is the uniform distribution over the number of
classes. By combining with cross-entropy and dropping the constants, we have
the final penalized optimization objective:

$\mathcal{L}=\frac{-1}{N}\sum_{i=1}^{N}\left[l_{\text{CE}}(y_{i}\|\textbf{P}_{i,k})+\lambda\cdot
D_{\text{KL}}(\text{U}_{[0,K]}\|\textbf{P}_{i,k})\right].$ (12)

### Bilevel Firth Coefficient Optimization

According to Eq. 12, we use $\lambda$ as a coefficient to adjust the strength
of the Firth regularizer. However, we found that the model performance is
sensitive to the $\lambda$, because the variance of model performance with
different $\lambda$ is around 4 points on average. It indicates that one needs
to perform some search algorithm to find the best $\lambda$. Furthermore,
after each round of query, as new labeled instances are added and the size of
the training set increases, we have to redo the search of the optimal
$\lambda$, making the whole AL pipeline immensely time-consuming.

To attack this issue, we optimize $\lambda$ without heavy parameter tuning via
bilevel optimization. Specifically, let $\mathcal{D}$ and $\mathcal{V}$ be the
current training set and a hold-out validation set, respectively. The bilevel
optimization objective can be formulated as

$\displaystyle\lambda^{*}$
$\displaystyle=\operatorname*{arg\,min}_{\lambda}\mathcal{L}_{\text{CE}}(\mathcal{V};\bm{\beta}^{*}(\lambda))$
$\displaystyle s.t.\ \bm{\beta}^{*}(\lambda)$
$\displaystyle=\operatorname*{arg\,min}_{\bm{\beta}}\mathcal{L}_{\text{CE}}(\mathcal{D};\bm{\beta})+\lambda\cdot\mathcal{L}_{\text{Firth}}(\mathcal{D};\bm{\beta}).$
(13)

The above bilevel optimization problem is nontrivial to solve since computing
the exact gradient of $\lambda$ regarding the objective of the outer-level
optimization requires an optimal solution of inner optimization. Specifically,
Eq. Bilevel Firth Coefficient Optimization can be separated as the
optimization of $\lambda^{*}$ for which the best response
$\bm{\beta}^{*}(\lambda)$ is required, and the optimization of the best
response $\bm{\beta}^{*}(\lambda)$. For the optimization of
$\bm{\beta}^{*}(\lambda)$, we adopt the SGD as an optimizer to update
iteratively with time-step $t$

$\bm{\beta}^{(t+1)}:=\bm{\beta}^{(t)}-\eta\cdot\nabla_{\bm{\beta}^{(t)}}\mathcal{L}_{\text{T}}(\bm{\beta}^{(t)},\lambda),$
(14)

where $\eta$ is the learning rate and $\mathcal{L}_{\text{T}}$ denotes the
training loss. Assume we have $T_{2}$ steps totally for update the
$\bm{\beta}$, the best response$\bm{\beta}^{*}(\lambda)$ can be represented as

$\displaystyle\bm{\beta}^{*}(\lambda)$
$\displaystyle\approx\bm{\beta}^{(T_{2})}-\eta\cdot\nabla_{\bm{\beta}^{(T_{2})}}\mathcal{L}_{\text{T}}(\bm{\beta}^{(T_{2})},\lambda)$
(15) $\displaystyle\text{where}\quad\bm{\beta}^{(T_{2})}$
$\displaystyle:=\bm{\beta}^{(T_{2}-1)}-\eta\cdot\nabla_{\bm{\beta}^{(T_{2}-1)}}\mathcal{L}_{\text{T}}(\bm{\beta}^{(T_{2}-1)},\lambda)$
$\displaystyle\ \ \vdots$ $\displaystyle\text{where}\quad\bm{\beta}^{(1)}$
$\displaystyle:=\bm{\beta}^{(0)}-\eta\cdot\nabla_{\bm{\beta}^{(0)}}\mathcal{L}_{\text{T}}(\bm{\beta}^{(0)},\lambda)$

To solve the optimization objective of $\lambda$, which requires the best
response $\bm{\beta}^{*}(\lambda)$, we propose using an outer loop that wraps
the optimization process for $\bm{\beta}^{*}(\lambda)$ described above. We
seek a value $\lambda^{*}$ which ensures that the training process
$\mathcal{L}_{\text{T}}(\bm{\beta},\lambda^{*})$ produces parameter
$\bm{\beta}^{*}(\lambda)$ which perform best against some metric
$\mathcal{L}_{\text{V}}(\bm{\beta}^{*}(\lambda))$ we care about. Essentially,
an estimate of $\bm{\beta}^{*}(\lambda)$ is obtained following $T_{2}$ steps
training as outlined in Eq. 15, which is then evaluated against
$\mathcal{L}_{\text{V}}$. The gradient of this evaluation,
$\nabla_{\lambda}\mathcal{L}_{\text{V}}(\bm{\beta}^{*}(\lambda))$ is then
obtained through backpropagation, and used to update $\lambda$. Using
$\tau=1,...,T_{1}$ as a time-step counter to symbolize the time-scale being
different from that used in the inner-loop, we formalize this update as

$\displaystyle\lambda^{*}$
$\displaystyle\approx\lambda^{(T_{1})}-\gamma\cdot\nabla_{\lambda^{(T_{1})}}\mathcal{L}_{\text{V}}(\bm{\beta}^{*}(\lambda^{(T_{1})})$
(16) $\displaystyle\text{where}\quad\lambda^{(T_{1})}$
$\displaystyle:=\lambda^{(T_{1}-1)}-\gamma\cdot\nabla_{\lambda^{(T_{1}-1)}}\mathcal{L}_{\text{V}}(\bm{\beta}^{*}(\lambda^{(T_{1}-1)}))$
$\displaystyle\ \ \vdots$ $\displaystyle\text{where}\quad\lambda^{(1)}$
$\displaystyle:=\lambda^{(0)}-\gamma\cdot\nabla_{\lambda^{(0)}}\mathcal{L}_{\text{V}}(\bm{\beta}^{*}(\lambda^{(0)}))$

where $\gamma$ is the learning rate, $\mathcal{L}_{\text{V}}$ denotes the
validation loss, and $\bm{\beta}^{*}(\lambda)$ can be solved by Eq. 15.
Implementing an iterative process as described in Eq. 16 will exploit the
chain rule for partial derivatives in order to run back propagation. By
decompose the
$\nabla_{\lambda}\mathcal{L}_{\text{V}}(\bm{\beta}^{*}(\lambda))$, we have

$\nabla_{\lambda}\mathcal{L}_{\text{V}}(\bm{\beta}^{*}(\lambda))=\nabla_{\bm{\beta}^{(t)}}\mathcal{L}_{\text{V}}(\bm{\beta}^{*}(\lambda))\cdot\nabla_{\lambda}\bm{\beta}^{*}(\lambda)$
(17)

Obviously,
$\nabla_{\bm{\beta}^{(t)}}\mathcal{L}_{\text{V}}(\bm{\beta}^{*}(\lambda))$
exists for $t\in\\{0,...,T_{2}\\}$. For
$\nabla_{\lambda}\bm{\beta}^{*}(\lambda)$, the gradients trivially exist
because: (1) $\lambda$ is a continuous hyperparameter and (2) Eq. 16 is a
differentiable function of $\lambda$. In this work, we use the GIMLI algorithm
(Grefenstette et al. 2019) to implement the efficient calculation of
$\nabla_{\lambda}\mathcal{L}_{\text{V}}(\bm{\beta}^{*}(\lambda))$.

Input: Unlabeled dataset $\mathcal{U}$; Query strategy $Q$; Size of each query
$M$.

$\mathcal{D}_{1}\leftarrow$ Randomly sample $M$ instances from $\mathcal{U}$
and query the labels.

$\mathcal{U}_{1}\leftarrow\mathcal{U}\setminus\mathcal{D}_{1}$

for  _$r=1,2,...,R$_ do

for  _$t=1,2,...,T$_ do

if  _$t\ mod\ T_{2}$ = 0_ then

$\lambda_{t}^{*}\leftarrow$ solve Eq. (Bilevel Firth Coefficient
Optimization).

$\bm{\beta}^{(t+1)}\leftarrow$ perform a gradient descent step.

$\bm{\beta}^{*}_{r}\leftarrow\bm{\beta}^{(T)}$.

$\mathcal{S}_{r+1}\leftarrow
Q(\text{from}:\mathcal{U}_{r},\text{size}:M,\text{model}:f(\cdot;\bm{\beta}^{*}_{r}))$.

Query annotations of $\mathcal{S}_{r+1}$ from Oracle.

$\mathcal{D}_{r+1}\leftarrow\mathcal{D}_{r}\cup\mathcal{S}_{r+1}$.

$\mathcal{U}_{r+1}\leftarrow\mathcal{U}_{r}\setminus\mathcal{D}_{r+1}$.

Output: Output final model $f(\cdot;\bm{\beta}^{*}_{R})$.

Algorithm 1 CHAIN

### The Curriculum of Firth Coefficient

Figure 1: Example of the Firth coefficient curriculum. Left is the coefficient
curve in a single model training. Right is the curve of the final coefficient
of a model training across AL query rounds.

Regarding the impact of the Firth coefficient on the model performance, it is
natural to hypothesize that the model at different training stages may require
different $\lambda$ to obtain the best performance since the model may
accumulate different levels of bias during the training. Therefore, we propose
a simple curriculum of the Firth coefficient. That is, we repeatedly update
the $\lambda$ via, again, solving the bilevel optimization but regarding the
currently trained model. Specifically, for every $T_{2}$ model training step,
we use the current model parameters as the initialization of the $\bm{\beta}$
in the inner optimization of Eq. Bilevel Firth Coefficient Optimization and
then solve the bilevel optimization to get the new $\lambda$. In this way, the
$\lambda$ is specific to the currently trained model and dynamically changes
over the training process.

As in Fig. 1, the coefficient we obtained by solving the bilevel optimization
is increasing within a single model training. However, the final coefficient
(the very last coefficient of a model training process) across different AL
query rounds decreases with the increase in the size of the training set. This
phenomenon provides some insights into using a curriculum of $\lambda$: (1)
within one model training, the bias is gradually increasing as the model saw a
small set of training data multiple times, and therefore the $\lambda$
increases as well to strengthen the regularizer; (2) when we have more and
more training data as the AL proceeds, the small-sample bias issue is
naturally alleviated so the final $\lambda$ decreases. Finally, the overall
algorithm of CHAIN is summarized in Alg. 1.

Table 1: Final round performance comparison with and without CHAIN of fine-
tunning ResNet18 on CIFAR10, CIFAR100 and Fashion MNIST. Blue denotes the best
result in each $M$, and red denotes the best results over all query strategies
and $M$ in the corresponding dataset. Performance overall rounds are listed in
the appendix.

Dataset($\downarrow$) | Methods($\downarrow$) | M=10 | M=20
---|---|---|---
Entropy | CoreSet | BADGE | Random | Entropy | CoreSet | BADGE | Random
CIFAR10 | Orig. | 52.26 | 52.84 | 53.04 | 56.06 | 51.88 | 52.61 | 53.26 | 54.43
w/ CHAIN | 57.65 | 59.56 | 58.03 | 60.15 | 55.90 | 56.90 | 56.94 | 57.47
CIFAR100 | Orig. | 15.52 | 14.63 | 16.50 | 15.63 | 15.55 | 15.01 | 15.77 | 15.79
w/ CHAIN | 17.13 | 16.12 | 17.82 | 18.08 | 16.62 | 16.36 | 16.21 | 18.82
Fashion MNIST | Orig. | 88.50 | 89.44 | 87.47 | 90.08 | 90.72 | 89.93 | 88.64 | 90.04
w/ CHAIN | 91.08 | 93.41 | 92.22 | 91.28 | 90.99 | 91.36 | 91.64 | 91.48

Dataset($\downarrow$) | Methods($\downarrow$) | M=50 | M=100
---|---|---|---
Entropy | CoreSet | BADGE | Random | Entropy | CoreSet | BADGE | Random
CIFAR10 | Orig. | 53.57 | 53.51 | 54.23 | 52.25 | 52.26 | 53.58 | 52.12 | 54.01
w/ CHAIN | 58.49 | 57.14 | 58.39 | 58.63 | 57.48 | 57.37 | 58.56 | 58.74
CIFAR100 | Orig. | 14.52 | 15.28 | 16.38 | 16.63 | 14.49 | 15.68 | 16.76 | 15.82
w/ CHAIN | 15.95 | 17.25 | 17.58 | 16.83 | 14.69 | 16.05 | 17.51 | 17.70
Fashion MNIST | Orig. | 89.20 | 88.32 | 91.88 | 90.43 | 90.32 | 90.13 | 90.11 | 89.68
w/ CHAIN | 92.12 | 90.94 | 92.78 | 90.90 | 90.09 | 90.65 | 93.47 | 91.16

(a) CIFAR10

(b) CIFAR100

Figure 2: Comparison of the performance gain over all rounds between FBR+BO,
FBR+GS and CHAIN of fine-tunning ResNet18 on CIFAR10, CIFAR100. Both of the
sub-figures share the same figure legend shown in Fig.2(a). Results on Fashion
MNIST are listed in the appendix.

## Experiment

In this section, we summarize the details of the experiment setting, including
the datasets, AL setups, implementation details, and baselines. We further
provide the comparison result, ablation study, and Firth coefficient
curriculum study on fine-tuning and logistic regression tasks.

### Experiment Setups

##### Datasets and AL setups.

We use three image classification benchmark datasets: CIFAR10 (Krizhevsky,
Hinton et al. 2009), CIFAR100 (Krizhevsky, Hinton et al. 2009) and Fashion
MNIST (Xiao, Rasul, and Vollgraf 2017) to evaluate CHAIN. The datasets are
separated into training and test sets originally, and we randomly sample a
small-size validation set from the training set. We study both deep learning
and linear model settings. For the former, we fix the model being trained to
be ResNet-18 (He et al. 2016) pre-trained on ImageNet (Deng et al. 2009),
while for the latter, we use the features extracted from also a ResNet-18 pre-
trained on ImageNet and fit a logistic regression model on top of the
extracted features. To demonstrate the efficacy of CHAIN for reducing the
small-sample bias in low-budget AL, we set our total budget as $500=M\times
R$, where the budget of the training set at each query round is
$M\in\\{10,20,50,100\\}$, and total query round $R=500/M$.

##### Implementations.

In the proposed CHAIN, we have two major hyperparameters: $T_{1}$ is the
number of steps we used in solving the bilevel optimization, and within one
model training, we solve the bilevel optimization to get a new $\lambda$ every
$T_{2}$ model training steps. For the deep learning setting, we set $T_{1}=50$
and $T_{2}=100$ for all datasets. For logistic regression, we set $T_{1}=1$
and $T_{2}=5$ for all the datasets. We use SGD with a momentum of $0.9$ for
fine-tuning the ResNet-18 and Adam for training the logistic regression model.
We use $0.01$ as the learning rate for fine-tuning and $0.001$ for logistic
regression. For bilevel optimization, we use Adam as our optimizer, and we set
the learning rate as $0.01$ for the deep learning setting and $0.05$ for the
linear model setting. To reduce the impact of changing the training set to the
AL optimization process, we keep our batch size $|\mathcal{B}|$ positive
proportional to the training set size $|\mathcal{S}_{t}|$, i.e.,
$|\mathcal{B}|=0.1|\mathcal{S}_{t}|$. All our experiments are conducted on 6
Nvidia A6000 GPUs and Intel Xeon Gold 6226R CPU, and our code111We will
release the source code after the paper is accepted. is implemented in Python
3.8 and PyTorch 1.9.

### Baselines

CHAIN can be readily combined with any query strategy since it only modifies
the model training procedure. To demonstrate the above claim, we choose three
well-known uncertainty-based, representation-based, and hybrid query
strategies and the naive random query strategy to show that CHAIN can improve
the model performance by reducing the small-sample bias in low-budget AL.
Besides those selected query strategies, we also compare CHAIN with two widely
used hyperparameter searching methods to show that CHAIN performs better
without efficiency loss.

#### Query strategies

Entropy (Joshi, Porikli, and Papanikolopoulos 2009b) Entropy is the most
typical uncertainty-based query strategy. Higher entropy values indicate
greater uncertainty in the probability distribution. CoreSet (Sener and
Savarese 2017c) is a representation-based query strategy that adopts the
coreset selection based on data embeddings(representations). BADGE (Ash et al.
2019) is a hybrid query strategy incorporating both predictive uncertainty and
data diversity into every selected batch.

#### Hyperparameter searching methods

To show the efficacy of our bilevel optimization approach for seeking the best
Firth coefficient, we include the Firth bias reduction (FBR) with grid search
over the coefficient as a baseline (marked as FBR w/ GS, GS: Grid Search). To
further verify the usefulness of the proposed curriculum of the coefficient,
our last baseline is FBR w/ BO (BO: Bilevel Optimization), which only applies
the bilevel optimization once before the model training process and uses the
output $\lambda$ as a fixed hyperparameter during the model training.

### Main Result on Fine-tuning

We evaluated CHAIN against multiple baselines across four query strategies and
three benchmark datasets, averaging the performance over three runs per query
round. The results, summarized in Tab. 1, showcase CHAIN’ performance
enhancement for the chosen query strategies. We conducted a t-test to verify
this improvement, taking $\mu_{\text{Orig.}}=\mu_{\text{w/ CHAIN}}$ as the
null hypothesis and $\mu_{\text{Orig.}}<\mu_{\text{w/ CHAIN}}$ as the
alternative. With $t$-values of $-15.52,-5.24$, and $-6.19$ for CIFAR10,
CIFAR100, and Fashion MNIST, respectively, all values fall below the critical
value $-1.6859$, strongly favoring the alternative hypothesis. Notably,
CIFAR10 and CIFAR100 perform best with a small $M$, likely due to their
complex features and consequential small-sample bias. Conversely, Fashion
MNIST excels with larger $M$, likely due to its simpler features, as
highlighted in Tab. 1 (additional discussions in Sec. The Curriculum of the
Firth Coefficient and the appendix).

We also compared the over-round performance gain of our method with other
well-known hyperparameter searching methods across all datasets, and we
recorded the results in Fig. 2. Upon analyzing the results of CHAIN, we
observed that the performance per round was relatively consistent overall $M$
for entropy and random query strategies. However, for CoreSet and BADGE, the
performance gain displayed a decreasing trend (for CoreSet) and an increasing
trend (for BADGE), respectively. This indicates that the CoreSet strategy is
highly biased when the dataset is small, while the bias of BADGE increases as
the dataset grows larger. Therefore, we recommend using BADGE with lower $M$
and CoreSet with higher $M$.

Table 2: Final round performance comparison with and without CHAIN of logistic
regression on CIFAR10. blue denotes the best result in each $M$, and red
denotes the best results over all query strategies and $M$ in the
corresponding dataset. Performance overall rounds are listed in the appendix.

Dataset($\downarrow$) | Methods($\downarrow$) | M=10 | M=20
---|---|---|---
Entropy | CoreSet | BADGE | Random | Entropy | CoreSet | BADGE | Random
CIFAR10 | Orig. | 28.28 | 11.62 | 43.03 | 39.33 | 31.27 | 10.56 | 40.18 | 40.96
w/ CHAIN | 37.47 | 32.15 | 44.12 | 43.44 | 37.79 | 33.90 | 44.86 | 41.36

Dataset($\downarrow$) | Methods($\downarrow$) | M=50 | M=100
---|---|---|---
Entropy | CoreSet | BADGE | Random | Entropy | CoreSet | BADGE | Random
CIFAR10 | Orig. | 33.50 | 11.49 | 44.06 | 40.35 | 28.35 | 11.51 | 42.04 | 39.87
w/ CHAIN | 36.79 | 34.29 | 44.93 | 42.19 | 39.61 | 37.21 | 45.61 | 42.99

(a) Fine-tuning a ResNet-18

(b) Logistic Regression

Figure 3: The trend of the Firth coefficient across model training steps and
AL query rounds on CIFAR10 with random query strategy and $M=100$.

(a) Fine-tuning a ResNet18

(b) Logistic Regression

Figure 4: Ablation studies of the two hyperparameters $T_{1}$ and $T_{2}$ on
CIFAR10 dataset with random query strategy and $M=100$. All the results are
averaged over three runs.

### Main Result on Logistic Regression

To theoretically demonstrate the effectiveness of CHAIN, we evaluated CHAIN
and other baseline methods using a logistic regression model on CIFAR10. The
results are presented in Tab. 2, from which we can observe that CHAIN
consistently improves the overall performance across all query strategies.
Using the same t-test as in Sec. Main Result on Fine-tuning, we obtained a
$t$-value of $-3.32$, which is smaller than the critical value of
$-t(N_{\text{Orig.}}+N_{\text{w/ CHAIN}}-2,0.10)=-1.6859$. Based on this
finding, we can conclude that the adoption of CHAIN leads to a significant
improvement in performance. Upon further examining Tab. 1, we observed that
CoreSet failed to converge to good performance with logistic regression.
However, a significant improvement was observed after adopting CHAIN,
indicating that the bias of CoreSet is amplified with less-parameterized
models such as logistic regression. On the other hand, BADGE has a strong
bias-restricting ability, enabling it to achieve better performance even with
less-parameterized models. Further discussion of the performance of logistic
regression can be found in the appendix.

### Ablation Study

We provide ablation studies on two major hyperparameters: $T_{1}$ and $T_{2}$.
Tuning $T_{1}$ will influence the bilevel optimization, which determines the
quality of the learned coefficient, while tuning the $T_{2}$ affects the
frequency of the coefficient being updated during the model training. We run
CHAIN with varying $T_{1}\in\\{10,20,50\\}$ and $T_{2}\in\\{50,100\\}$, and
compare it with FBR w/ GS. The results can be found in Fig. 4. Following the
results, CHAIN outperforms grid search in all combinations of $T_{1}$ and
$T_{2}$. We can see that performance of CHAIN with different hyperparameters
is more consistent with lower variance when using the logistic regression
model than the deep model. This observation indicates that the complex deep
model is relatively more sensitive to the bilevel optimization process and the
curriculum of the Firth coefficient.

### The Curriculum of the Firth Coefficient

We examine the progression of the optimization coefficient during model
training and across various Active Learning (AL) rounds, with results depicted
in Fig. 3 (details for CIFAR100 and Fashion MNIST are in the appendix). Both
when fine-tuning deep models and training logistic regression models, the
coefficient increases each round, eventually plateauing. This increase
reflects the model’s transition from a bias-free random initialization to a
bias-impacted state due to small-sample size as training progresses,
necessitating stronger regularization. The plateau likely arises when model
parameters undergo minor changes, reflecting well-trained status, resulting in
a stable coefficient.

Regarding the coefficient’s dynamic across AL rounds, we observe a decrease in
the final step coefficient during deep model fine-tuning. In contrast,
logistic regression models see early-round coefficient increases followed by
later-round decreases. This decrease in deep learning settings aligns with
expectations: more labeled data naturally mitigate small-sample bias. The
initial increase in the logistic regression model’s coefficient is somewhat
counterintuitive, as more data should typically decrease the need for Firth
regularization. We posit that early query rounds trigger early stopping in
fewer training steps for logistic regression models, resulting in lower final
coefficients. In contrast, later rounds involve more training steps and
subsequent coefficient updates, leading to higher coefficients.

## Conclusion

In this work, we focus on the small-sample bias issue in low-budget active
learning. To mitigate the small-sample bias, we leverage the Firth bias
reduction. Since Firth bias reduction is sensitive to its coefficient, we
adopt bilevel optimization to acquire the optimal coefficient for model
training to avoid the time-consuming hyperparameter tuning. Furthermore, we
introduce a simple curriculum of the coefficient, which can adjust the
regularization strength according to the current model training stage. The
experimental results show that CHAIN outperforms all baselines in most cases.
by comparing with three query strategies and two widely used hyperparameter
searching methods on three datasets.

## References

  * Ash et al. (2019) Ash, J. T.; Zhang, C.; Krishnamurthy, A.; Langford, J.; and Agarwal, A. 2019.  Deep batch active learning by diverse, uncertain gradient lower bounds.  _arXiv preprint arXiv:1906.03671_.
  * Box (1971) Box, M. J. 1971.  Bias in Nonlinear Estimation.  _Journal of the royal statistical society series b-methodological_.
  * Chen et al. (2019) Chen, W.-Y.; Liu, Y.-C.; Kira, Z.; Wang, Y.-C. F.; and Huang, J.-B. 2019.  A closer look at few-shot classification.  _arXiv preprint arXiv:1904.04232_.
  * Contardo, Denoyer, and Artières (2017) Contardo, G.; Denoyer, L.; and Artières, T. 2017.  A meta-learning approach to one-step active learning.  _arXiv preprint arXiv:1706.08334_.
  * Cox and Snell (1968) Cox, D.; and Snell, E. J. 1968.  A General Definition of Residuals.  _Journal of the royal statistical society series b-methodological_.
  * Culotta and McCallum (2005) Culotta, A.; and McCallum, A. 2005.  Reducing labeling effort for structured prediction tasks.  _national conference on artificial intelligence_.
  * Deng et al. (2009) Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei-Fei, L. 2009.  ImageNet: A large-scale hierarchical image database.  In _2009 IEEE Conference on Computer Vision and Pattern Recognition_ , 248–255.
  * Firth (1993) Firth, D. 1993.  Bias reduction of maximum likelihood estimates.  _Biometrika_ , 80(1): 27–38.
  * Ghaffari et al. (2021) Ghaffari, S.; Saleh, E.; Forsyth, D.; and Wang, Y.-X. 2021.  On the Importance of Firth Bias Reduction in Few-Shot Classification.  In _International Conference on Learning Representations_.
  * Gissin and Shalev-Shwartz (2019) Gissin, D.; and Shalev-Shwartz, S. 2019.  Discriminative Active Learning.  _arXiv: Learning_.
  * Grefenstette et al. (2019) Grefenstette, E.; Amos, B.; Yarats, D.; Htut, P. M.; Molchanov, A.; Meier, F.; Kiela, D.; Cho, K.; and Chintala, S. 2019.  Generalized Inner Loop Meta-Learning.  _arXiv preprint arXiv:1910.01727_.
  * Hacohen, Dekel, and Weinshall (2022) Hacohen, G.; Dekel, A.; and Weinshall, D. 2022.  Active learning on a budget: Opposite strategies suit high and low budgets.  _arXiv preprint arXiv:2202.02794_.
  * Harville (1998) Harville, D. A. 1998.  Matrix algebra from a statistician’s perspective.
  * He et al. (2016) He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 770–778.
  * Joshi, Porikli, and Papanikolopoulos (2009a) Joshi, A.; Porikli, F.; and Papanikolopoulos, N. 2009a.  Multi-class active learning for image classification.  _computer vision and pattern recognition_.
  * Joshi, Porikli, and Papanikolopoulos (2009b) Joshi, A. J.; Porikli, F.; and Papanikolopoulos, N. 2009b.  Multi-class active learning for image classification.  In _2009 ieee conference on computer vision and pattern recognition_ , 2372–2379. IEEE.
  * Krizhevsky, Hinton et al. (2009) Krizhevsky, A.; Hinton, G.; et al. 2009.  Learning multiple layers of features from tiny images.
  * Lewis (1995) Lewis, D. D. 1995.  A sequential algorithm for training text classifiers: Corrigendum and additional data.  In _Acm Sigir Forum_ , volume 29, 13–19. ACM New York, NY, USA.
  * Li and Guo (2013a) Li, X.; and Guo, Y. 2013a.  Adaptive Active Learning for Image Classification.  _computer vision and pattern recognition_.
  * Li and Guo (2013b) Li, X.; and Guo, Y. 2013b.  Adaptive active learning for image classification.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 859–866.
  * Lorraine, Vicol, and Duvenaud (2020) Lorraine, J.; Vicol, P.; and Duvenaud, D. 2020.  Optimizing millions of hyperparameters by implicit differentiation.  In _International Conference on Artificial Intelligence and Statistics_ , 1540–1552. PMLR.
  * Mahmood, Fidler, and Law (2021) Mahmood, R.; Fidler, S.; and Law, M. T. 2021.  Low Budget Active Learning via Wasserstein Distance: An Integer Programming Approach.  _arXiv preprint arXiv:2106.02968_.
  * Poirier (1994) Poirier, D. 1994.  Jeffreys’ prior for logit models.  _Journal of Econometrics_ , 63(2): 327–339.
  * Pourahmadi, Nooralinejad, and Pirsiavash (2021) Pourahmadi, K.; Nooralinejad, P.; and Pirsiavash, H. 2021.  A Simple Baseline for Low-Budget Active Learning.  _arXiv: Computer Vision and Pattern Recognition_.
  * Roth and Small (2006) Roth, D.; and Small, K. 2006.  Margin-based active learning for structured output spaces.  In _European Conference on Machine Learning_ , 413–424. Springer.
  * Scheffer, Decomain, and Wrobel (2001) Scheffer, T.; Decomain, C.; and Wrobel, S. 2001.  Active Hidden Markov Models for Information Extraction.  _Intelligent Data Analysis_.
  * Schröder and Niekler (2020) Schröder, C.; and Niekler, A. 2020.  A survey of active learning for text classification using deep neural networks.  _arXiv preprint arXiv:2008.07267_.
  * Sener and Savarese (2017a) Sener, O.; and Savarese, S. 2017a.  Active learning for convolutional neural networks: A core-set approach.  _arXiv preprint arXiv:1708.00489_.
  * Sener and Savarese (2017b) Sener, O.; and Savarese, S. 2017b.  Active learning for convolutional neural networks: A core-set approach.  _arXiv preprint arXiv:1708.00489_.
  * Sener and Savarese (2017c) Sener, O.; and Savarese, S. 2017c.  Active learning for convolutional neural networks: A core-set approach.  _arXiv preprint arXiv:1708.00489_.
  * Settles (2009) Settles, B. 2009.  Active Learning Literature Survey.
  * Shui et al. (2020) Shui, C.; Zhou, F.; Gagné, C.; and Wang, B. 2020.  Deep active learning: Unified and principled method for query and training.  In _International Conference on Artificial Intelligence and Statistics_ , 1308–1318. PMLR.
  * Sinha, Ebrahimi, and Darrell (2019a) Sinha, S.; Ebrahimi, S.; and Darrell, T. 2019a.  Variational Adversarial Active Learning.  _international conference on computer vision_.
  * Sinha, Ebrahimi, and Darrell (2019b) Sinha, S.; Ebrahimi, S.; and Darrell, T. 2019b.  Variational adversarial active learning.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 5972–5981.
  * Steyerberg, Eijkemans, and Habbema (1999) Steyerberg, E. W.; Eijkemans, M. J.; and Habbema, J. D. F. 1999.  Stepwise selection in small data sets : A simulation study of bias in logistic regression analysis.  _Journal of Clinical Epidemiology_.
  * Wang and Shang (2014) Wang, D.; and Shang, Y. 2014.  A new active labeling method for deep learning.  In _2014 International joint conference on neural networks (IJCNN)_ , 112–119. IEEE.
  * Whitehead (1986) Whitehead, J. 1986.  On the bias of maximum likelihood estimation following a sequential test.  _Biometrika_.
  * Xiao, Rasul, and Vollgraf (2017) Xiao, H.; Rasul, K.; and Vollgraf, R. 2017.  Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms.  _arXiv preprint arXiv:1708.07747_.
  * Yu, Bi, and Tresp (2006) Yu, K.; Bi, J.; and Tresp, V. 2006.  Active learning via transductive experimental design.  In _Proceedings of the 23rd international conference on Machine learning_ , 1081–1088.
  * Yuan, Lin, and Boyd-Graber (2020) Yuan, M.; Lin, H.-T.; and Boyd-Graber, J. 2020.  Cold-start Active Learning through Self-supervised Language Modeling.  _empirical methods in natural language processing_.

## Extra Experiment

We provide the per-round comparison in Fig. 5, and the final round results are
reported in the main context. To clarify the figure, we omit the curve of w/
GS and w/ BO of all query strategies and only report the final round results
in Tab.3. We can see from the results that in most circumstances, training
with a curriculum coefficient provides a promising performance gain over all
rounds in CIFAR10 and CIFAR100, which is also reported in Fig. 2(a).

### Other hyperparameter searching methods

From Tab. 3, we can find that FBR+GS also performs better than the origin
under most situations but less than CHAIN because of the limited searching
space. In this work, we search the $\lambda$ in {0.0, 0.01, 0.1, 1.0, 3.0} in
each round, which means we will train five different models from scratch in
each round, comparing their performance, and choose the best one to query data
by the baseline query strategies from the dataset. While FBR+GS yields good
results, supporting such a search process requires huge performance,
especially in active learning, which has multiple rounds, and we need to train
a model from scratch at each round to prevent the influence of changing the
distribution of the training set. On the other hand, FBR+BO performs less than
the original in many circumstances. This is because bi-level optimization has
a huge noise when the model is under-fitting, which is why we require $T_{2}$
steps in CHAIN to get a ”best response” of $\lambda$ (Lorraine, Vicol, and
Duvenaud 2020).

### Less perform in Fashion MNIST

In the Fashion MNIST, the over-round performance gain of CHAIN is sometimes
less than FBR w/ GS, but CHAIN still achieves the best final-round result in
most circumstances. By deep into the curriculum of the $\lambda$ of CHAIN when
performing on Fashion MNIST (Fig. 6), we observe that CHAIN tends to learn a
negative value of $\lambda$, which according to Eq. 12 means that the model
tends to ”sharpen” the output confidence by minimizing the KL-divergence
between the uniform distribution and the output confidence. This observation
indicates that the model can capture the features of Fashion MNIST under a
low-budget regime with only a few steps, such that the model’s performance on
the training set can be improved by giving a more confident output (reflected
in a sharp confidence distribution). However, such an optimization process
will speed up the over-fitting, leading to less performance of CHAIN at the
start of training. Fortunately, the ”over-fitting” process can be mitigated by
CHAIN in the later round, which can be observed in Fig. 7(b) later round.
Moreover, this can be further alleviated by increasing the query rounds so
that the CHAIN achieve the best average performance in $M=10$ on Fashion
MNIST.

### CoreSet versus BADGE in logistic regression

We found that in logistic regression, the CoreSet cannot converge in CIFAR10
overall situations, and the bad performance appears consistently. This is
because CoreSet is a representation-based query strategy, and in the logistic
regression, we froze the parameter of the feature extractor (ResNet18), which
means the representation is fixed during training. As a result, CoreSet will
always query data with noised embeddings, leading to a bad performance. While
in the first round, the data is queried randomly to eliminate the bias of
dataset initialization, so the coreset achieves better performance in the
first round, and the performance gain is eliminated by the later selected
noise data points. However, BADGE does not suffer from the invariant
embeddings even though it leverages the same embeddings of CoreSet. This is
because BADGE encodes the gradient, a representation of uncertainty, of the
current model (logistic regression) into the embeddings, which helps the model
query the data with large uncertainty, which can provide the most information
gain.

Table 3: Final round performance comparison with and without CHAIN of fine-
tunning ResNet18 on CIFAR10, CIFAR100 and Fashion MNIST. Blue denotes the best
result in each $M$, and red denotes the best results over all query strategies
and $M$ in the corresponding dataset.

Dataset($\downarrow$) | Methods($\downarrow$) | M=10 | M=20
---|---|---|---
Entropy | CoreSet | BADGE | Random | Entropy | CoreSet | BADGE | Random
CIFAR10 | Orig. | 52.26 | 52.84 | 53.04 | 56.06 | 51.88 | 52.61 | 53.26 | 54.43
w/ FBR+BO | 51.02 | 51.98 | 54.26 | 55.25 | 54.32 | 50.92 | 56.98 | 54.86
w/ FBR+GS | 53.68 | 54.31 | 56.90 | 57.63 | 56.00 | 56.22 | 56.37 | 56.42
w/ CHAIN | 57.65 | 59.56 | 58.03 | 60.15 | 55.90 | 56.90 | 56.94 | 57.47
CIFAR100 | Orig. | 15.52 | 14.63 | 16.50 | 15.63 | 15.55 | 15.01 | 15.77 | 15.79
w/ FBR+BO | 14.77 | 14.32 | 15.92 | 16.73 | 14.39 | 15.44 | 16.21 | 16.20
w/ FBR+GS | 16.01 | 15.59 | 15.65 | 16.17 | 15.34 | 15.94 | 16.22 | 16.17
w/ CHAIN | 17.13 | 16.12 | 17.82 | 18.08 | 16.62 | 16.36 | 16.21 | 18.82
Fashion MNIST | Orig. | 88.50 | 89.44 | 87.47 | 90.08 | 90.72 | 89.93 | 88.64 | 90.04
w/ FBR+BO | 88.66 | 86.99 | 90.30 | 91.81 | 88.63 | 89.46 | 90.15 | 90.45
w/ FBR+GS | 89.72 | 90.54 | 91.16 | 89.46 | 90.44 | 90.85 | 91.50 | 90.06
w/ CHAIN | 91.08 | 93.41 | 92.22 | 91.28 | 90.99 | 91.36 | 91.64 | 91.48

Dataset($\downarrow$) | Methods($\downarrow$) | M=50 | M=100
---|---|---|---
Entropy | CoreSet | BADGE | Random | Entropy | CoreSet | BADGE | Random
CIFAR10 | Orig. | 53.57 | 53.51 | 54.23 | 52.25 | 52.26 | 53.58 | 52.12 | 54.01
w/ FBR+BO | 53.55 | 53.77 | 54.20 | 54.88 | 51.70 | 54.65 | 54.85 | 54.39
w/ FBR+GS | 54.89 | 56.65 | 56.59 | 55.24 | 54.63 | 56.77 | 56.36 | 55.89
w/ CHAIN | 58.49 | 57.14 | 58.39 | 58.63 | 57.48 | 57.37 | 58.56 | 58.74
CIFAR100 | Orig. | 14.52 | 15.28 | 16.38 | 16.63 | 14.49 | 15.68 | 16.76 | 15.82
w/ FBR+BO | 13.72 | 16.13 | 16.58 | 16.46 | 14.04 | 15.93 | 14.92 | 16.36
w/ FBR+GS | 15.37 | 16.33 | 16.58 | 16.57 | 14.22 | 16.39 | 16.36 | 16.51
w/ CHAIN | 15.95 | 17.25 | 17.58 | 16.83 | 14.69 | 16.05 | 17.51 | 17.70
Fashion MNIST | Orig. | 89.20 | 88.32 | 91.88 | 90.43 | 90.32 | 90.13 | 90.11 | 89.68
w/ FBR+BO | 90.32 | 89.59 | 89.87 | 90.76 | 89.77 | 90.55 | 89.39 | 90.89
w/ FBR+GS | 90.44 | 91.25 | 91.14 | 91.56 | 90.11 | 91.73 | 90.66 | 90.35
w/ CHAIN | 92.12 | 90.94 | 92.78 | 90.90 | 90.09 | 90.65 | 93.47 | 91.16

(a) CIFAR10

(b) CIFAR100

(c) Fashion MNIST

(d) CIFAR10 (logistic regression)

Figure 5: Per-round performance comparison (both of the fine-tuning and
logistic regression) on three benchmark classification datasets: CIFAR10,
CIFAR100, and Fashion MNIST. All of the above sub-figures share the same
legend in Fig. 5(a). The solid lines are the average result over 3 runs, and
the shadow area indicates the standard deviation. Figure 6: Comparison of the
performance gain over all rounds between FBR+BO, FBR+GS and CHAIN of fine-
tunning ResNet18 on Fashion MNIST.

(a) Fine-tuning a ResNet-18 on CIFAR100

(b) Fine-tuning a ResNet-18 on Fashion MNIST

Figure 7: The trend of the Firth coefficient across model training steps and
AL query rounds on CIFAR100 and Fashion MNIST with random query strategy and
$M=100$.