# Rethinking Out-of-Distribution Detection on Imbalanced Data Distribution

Kai Liu1,2111Work done during Kai Liu’s research internship at Alibaba Cloud.
Email<EMAIL_ADDRESS>, Zhihang Fu2222Corresponding authors. Email:
<EMAIL_ADDRESS><EMAIL_ADDRESS>, Sheng Jin2, Chao Chen2, Ze
Chen1,
Rongxin Jiang1, Fan Zhou1222Corresponding authors. Email: zhihang.fzh@alibaba-
inc.com<EMAIL_ADDRESS>, Yaowu Chen1, Jieping Ye2
1Zhejiang University, 2Alibaba Cloud

###### Abstract

Detecting and rejecting unknown out-of-distribution (OOD) samples is critical
for deployed neural networks to void unreliable predictions. In real-world
scenarios, however, the efficacy of existing OOD detection methods is often
impeded by the inherent imbalance of in-distribution (ID) data, which causes
significant performance decline. Through statistical observations, we have
identified two common challenges faced by different OOD detectors:
misidentifying tail class ID samples as OOD, while erroneously predicting OOD
samples as head class from ID. To explain this phenomenon, we introduce a
generalized statistical framework, termed ImOOD, to formulate the OOD
detection problem on imbalanced data distribution. Consequently, the
theoretical analysis reveals that there exists a class-aware bias item between
balanced and imbalanced OOD detection, which contributes to the performance
gap. Building upon this finding, we present a unified training-time
regularization technique to mitigate the bias and boost imbalanced OOD
detectors across architecture designs. Our theoretically grounded method
translates into consistent improvements on the representative CIFAR10-LT,
CIFAR100-LT, and ImageNet-LT benchmarks against several state-of-the-art OOD
detection approaches. Code will be made public soon.

##  1 Introduction

Identifying and rejecting unknown samples during models’ deployments, aka OOD
detection, has garnered significant attention and witnessed promising
advancements in recent years [53, 3, 37, 44, 27]. Nevertheless, most advanced
OOD detection methods are designed and evaluated in ideal settings with
category-balanced in-distribution (ID) data. However, in practical scenarios,
long-tailed class distribution (a typical imbalance problem) not only limits
classifiers’ capability [5], but also causes a substantial performance decline
for OOD detectors [47].

Several efforts have been applied to enhance OOD detection on imbalanced data
distributions [47, 19, 41]. They mainly attribute the performance degradation
to misidentifying samples from tail classes as OOD (due to the lack of
training data), and concentrate on improving the discriminability for tail
classes and out-of-distribution samples [47, 51]. Whereas, we argue that the
confusion between tail class and OOD samples presents only one aspect of the
imbalance problem arising from the long-tailed data distribution.

To comprehensively understand the imbalance issue, we investigate a wide range
of representative OOD detection methods (_i.e._ , OE [17], Energy [28], and
PASCL [47]) on the CIFAR10-LT dataset [6]. For each model, we statistic the
distribution of wrongly detected ID samples and wrongly detected OOD samples,
respectively. The results in Fig. 1(a) reveal that different approaches
encounter the same two challenges: (1) ID samples from tail classes are prone
to be detected as OOD, and (2) OOD samples are prone to be predicted as ID
from head classes. As illustrated in Fig. 1(b), we argue that the disparate ID
decision spaces on head and tail classes jointly result in the performance
decline for OOD detection, which has also been confirmed by Miao et al. [36].

(a) Statistics on wrongly-detected ID and OOD samples.

(b) Dicision illustration.

Figure 1: Issues of OOD detection on imbalanced data. (a) Statistics of the
class labels of ID samples that are wrongly detected as OOD, and the class
predictions of OOD samples that are wrongly detected as ID. (b) Illustration
of the OOD detection process in feature space. Head classes’ huge decision
space and tail classes’ small decision space jointly damage the OOD detection.

To mitigate this problem, Miao et al. [36] developed a heuristic outlier class
learning approach (namely COCL) to respectively separate OOD samples from head
and tail ID classes in the feature space. Different from COCL, this paper
introduces a generalized statistical framework, termed ImOOD, to formulate and
explain the fundamental issue of imbalanced OOD detection from a probabilistic
perspective. We start by extending closed-set ID classification to open-set
scenarios and derive a unified posterior probabilistic model for ID/OOD
identification. Consequently, we find that between balanced and imbalanced ID
data distributions exists a class-aware bias item, which concurrently explains
the inferior OOD detection performance on both head and tail classes.

Based on ImOOD, we derive a unified loss function to regularize the posterior
ID/OOD probability during training, which simultaneously encourages the
separability between tail ID classes and OOD samples, and prevents predicting
OOD samples as head ID classes. Furthermore, ImOOD can readily generalize to
various OOD detection methods, including OE [17], Energy [28], and BinDisc
[3], Mahalanobis-distance [25], _etc._ Besides, our method can easily
integrate with other feature-level optimization techniques, including PASCL
[47] and COCL [36], to derive stronger OOD detectors. With the support of
theoretical analysis, our statistical framework consistently translates into
strong empirical performance on the CIFAR10-LT, CIFAR100-LT [6], and ImageNet-
LT [47] benchmarks.

Our contribution can be summarized as follows:

  * •

Through statistical observation and theoretical analysis, we reveal that OOD
detection approaches collectively suffer from the disparate decision spaces
between tail and head classes in the imbalanced data distribution.

  * •

We establish a generalized statistical framework to formulate and explain the
imbalanced OOD detection issue, and further provide a unified training
regularization to alleviate the problem.

  * •

We achieve superior OOD detection performance on three representative
benchmarks, outperforming state-of-the-art methods by a large margin.

##  2 Related Work

Out-of-distribution detection. To reduce the overconfidence on unseen OOD
samples [2], a surge of post-hoc scoring functions has been devised based on
various information, including output confidence [16, 26, 29], free energy
[28, 12, 23], Bayesian inference [32, 7], gradient information [18],
model/data sparsity [42, 56, 11, 1], and visual distance [43, 45], _etc._
Vision-language models like CLIP [40] have been recently leveraged to
explicitly collect potential OOD labels [14, 13] or conduct zero-shot OOD
detections [37]. Other researchers add open-set regularization in the training
time [33, 17, 50, 49, 31, 27], making models produce lower confidence or
higher energy on OOD data. Manually-collected [17, 48] or synthesized [12, 45]
outliers are required for auxiliary constraints.

Works insofar have mostly focused on the ideal setting with balanced data
distribution for optimization and evaluation. This paper aims at OOD detection
on practically imbalanced data distribution.

OOD detection on imbalanced data distribution. In real-world scenarios, the
deployed data frequently exhibits long-tailed distribution, and Liu et al.
[30] start to study the open-set classification on class-imbalanced setup.
Wang et al. [47] systematically investigate the performance degradation for
OOD detection on imbalanced data distribution, and develop a partial and
asymmetric contrastive learning (PASCL) technique to tackle this problem. Wei
et al. [51] and [36] extend the feature-space optimization by introducing
abstention classes or outlier class learning and integrating with data
augmentation or margin learning techniques, respectively. Sapkota and Yu [41]
employ adaptive distributively robust optimization (DRO) to quantify the
sample uncertainty from imbalanced distributions. Choi et al. [8] focus on the
imbalance problem in OOD data, and develop an adaptive regularization for each
OOD sample during optimization. In particular, Jiang et al. [20] also utilize
class prior to boost imbalanced OOD detector, which is however constrained as
a heuristic post-hoc normalization for pre-trained models and unable to
translate into training a better detector.

Different from previous efforts, this paper establishes a generalized
probabilistic framework to formulate and explain the imbalanced OOD detection
issue, and provides a unified training-time regularization technique to
alleviate this problem across different OOD detectors.

##  3 Rethinking Imbalanced OOD Detection

In this section, we start from revisiting the closed-set imbalanced image
recognition (in-distribution classification), and extend to open-set out-of-
distribution detection problem. Finally, we will reveal the class-aware bias
between balanced and imbalanced OOD detectors, and derive a unified training-
time regularization technique to alleviate the bias for different detectors.

###  3.1 Preliminaries

Imbalanced Image Recognition. Let $\mathcal{X}^{in}$ and
$\mathcal{Y}^{in}=\\{1,2,\cdots,K\\}$ denote the ID feature space and label
space with $K$ categories in total. Let ${\bm{x}}\in\mathcal{X}^{in}$ and
$y\in\mathcal{Y}^{in}$ be the random variables with respect to
$\mathcal{X}^{in}$ and $\mathcal{Y}^{in}$. The posterior probability for
predicting sample ${\bm{x}}$ into class $y$ is given by:

$\displaystyle P(y|{\bm{x}})=\frac{P({\bm{x}}|y)\cdot
P(y)}{P({\bm{x}})}\propto P({\bm{x}}|y)\cdot P(y)$ (1)

Given a learned classifier $f\colon\mathcal{X}^{in}\mapsto\mathbb{R}^{K}$ that
estimates $P(y|{\bm{x}})$, in the class-imbalance setting where the label
prior $P(y)$ is highly skewed, $f$ is evaluated with the balanced error (BER)
[4, 34, 35]:

$\displaystyle\text{BER}(f)=\frac{1}{K}\sum_{y}P_{{\bm{x}}|y}(y\neq\text{argmax}_{y^{\prime}}f_{y^{\prime}}({\bm{x}}))$
(2)

This can be seen as implicitly estimating a class-balanced posterior
probability [35]:

$\displaystyle P^{\text{bal}}(y|{\bm{x}})\propto\frac{1}{K}\cdot
P({\bm{x}}|y)\propto\frac{P(y|{\bm{x}})}{P(y)}$ (3)

The ideal Bayesian-optimal classification becomes
$y^{*}=\text{argmax}_{y\in[K]}P^{\text{bal}}(y|{\bm{x}})$.

Out-of-distribution Detection. In the open world, input sample $x$ may also
come from the OOD space $\mathcal{X}^{out}$. Let $o$ be the random variable
for an unknown label $o\notin\mathcal{Y}^{in}$, and $i$ be the union variable
of all ID class labels (_i.e._ , $i=\cup~{}y$). Given an input ${\bm{x}}$ from
the union space $\mathcal{X}^{in}\cup\mathcal{X}^{out}\triangleq\mathcal{X}$,
the posterior probability for identifying ${\bm{x}}$ as in-distribution is
formulated as:

$\displaystyle P(i|{\bm{x}})=\sum_{y}{P(y|{\bm{x}})}=1-P(o|{\bm{x}})\not\equiv
1$ (4)

Correspondingly, $P(o|{\bm{x}})$ measures the probability that sample
${\bm{x}}$ does not belong to any known ID class, aka OOD probability. Hence,
the OOD detection task can be viewed as a binary classification problem [3].
Given a learned OOD detector
$g\colon\mathcal{X}^{in}\cup\mathcal{X}^{out}\mapsto\mathbb{R}^{1}$ that
estimate the $P(i|{\bm{x}})$, samples with lower scores $g({\bm{x}})$ are
detected as OOD and vice versa.

###  3.2 Analysis of OOD Detection on Imbalanced Data Distribution

When ID classification meets OOD detection, slightly different from Eq. 1, the
classifier $f$ is actually estimating the posterior class probability for
sample ${\bm{x}}$ from ID space $\mathcal{X}^{in}$ merely, that is,
$P(y|{\bm{x}},i)$ [17, 47]. Considering a sample ${\bm{x}}$ from the open
space ${\bm{x}}\in\mathcal{X}^{in}\cup\mathcal{X}^{out}$, the classification
posterior in Eq. 1 is re-formulated as $P(y|{\bm{x}})=P(y|{\bm{x}},i)\cdot
P(i|{\bm{x}})$, the probability that sample ${\bm{x}}$ comes from ID data
multiply the probability that sample ${\bm{x}}$ belongs to the specific $y$-th
ID class. According to Eq. 3, we assume the proportion between balanced
classification $P^{\text{bal}}(y|{\bm{x}})$ and vanilla $P(y|{\bm{x}})$ for
each class $y$ still holds.

######  Lemma 3.1.

For each ID class $y$ in open-set, there exists a non-negative variable
$\gamma_{y}({\bm{x}})$, so that
$P^{\text{bal}}(y|{\bm{x}})=\gamma_{y}({\bm{x}})\cdot\frac{P(y|{\bm{x}})}{P(y)}$,
where
$\gamma_{y}({\bm{x}})=\frac{1}{K}\frac{P^{\text{bal}}({\bm{x}}|y)}{P({\bm{x}}|y)}\in(0,\infty)$,
$P(y|{\bm{x}}),P^{\text{bal}}(y|{\bm{x}}),P(y)\in[0,1]$.

In fact, $\gamma_{y}({\bm{x}})$ captures the likelihood difference for the
same sample ${\bm{x}}$ between balanced and imbalanced distributions, and also
plays the role in constraining the multiplication results to $(0,1)$. The
proof can be found in Sec. A.1. Using Lemma 3.1, we reveal that there exists a
class-aware bias term $\beta({\bm{x}})$ between the OOD detection on balanced
and imbalanced data distribution.

######  Theorem 3.2.

According to Lemma 3.1, there exists a bias term
$\beta({\bm{x}})=\sum_{y}{\gamma_{y}({\bm{x}})\frac{P(y|{\bm{x}},i)}{P(y)}}$
between $P^{\text{bal}}(i|{\bm{x}})$ and $P(i|{\bm{x}})$, _i.e._ ,
$P^{\text{bal}}(i|{\bm{x}})=\beta({\bm{x}})\cdot P(i|{\bm{x}})$.

###### Proof.

Since $P(y|{\bm{x}})=P(y|{\bm{x}},i)\cdot P(i|{\bm{x}})$, from Lemma 3.1,
$P^{\text{bal}}(y|{\bm{x}})$ can be further expressed as
$P^{\text{bal}}(y|{\bm{x}})=\gamma_{y}({\bm{x}})\cdot\frac{P(y|{\bm{x}},i)}{P(y)}\cdot
P(i|{\bm{x}})$. According to Eq. 4, it can be formulated as:
$P^{\text{bal}}(i|{\bm{x}})=\sum_{y}{P^{\text{bal}}(y|{\bm{x}})}=\sum_{y}{\gamma_{y}({\bm{x}})\frac{P(y|{\bm{x}},i)}{P(y)}P(i|{\bm{x}})}\triangleq\beta({\bm{x}})\cdot
P(i|{\bm{x}})$, where
$\beta({\bm{x}})=\sum_{y}{\gamma_{y}({\bm{x}})\frac{P(y|{\bm{x}},i)}{P(y)}}$.
∎

Based on Theorem 3.2, we conclude that the original OOD posterior
$P(i|{\bm{x}})$ estimated by the detector $g({\bm{x}})$, with the bias term
$\beta({\bm{x}})$, causes the performance gap for OOD detection on balanced
and imbalanced data distributions. To understand this further, we will first
discuss the scenario under ideal conditions and then extend our analysis to
real-world scenarios, attempting to analyze the intrinsic bias of the out-of-
distribution problem in both cases.

On ideal class-balanced distribution, the data likelihood
$P({\bm{x}}|y)=P^{\text{bal}}({\bm{x}}|y)$, so that
$\gamma_{y}({\bm{x}})\equiv\frac{1}{K}$. From Theorem 3.2,
$\beta({\bm{x}})=\sum_{y}{\frac{1}{K}\frac{P(y|{\bm{x}},i)}{P(y)}}$.
Meanwhile, the class prior $P(y)\equiv\frac{1}{K}$, and the summary of in-
distribution classification probabilities equals 1 (_i.e._ ,
$\sum_{y}{P(y|{\bm{x}},i)}=1$), making the bias item
$\beta({\bm{x}})=\sum_{y}{P(y|{\bm{x}},i)}=1$. Ultimately, Theorem 3.2
indicates $P^{\text{bal}}(i|{\bm{x}})=P(i|{\bm{x}})$, where the detector
$g({\bm{x}})$ exactly models the balanced OOD detection.

On more challenging class-imbalanced distribution, the class prior $P(y)$ is a
class-specific variable for ID categories, and
$\beta(x)=\sum_{y}{\gamma_{y}({\bm{x}})\frac{P(y|{\bm{x}},i)}{P(y)}}\not\equiv
1$. In previous works[17, 47], the class posterior $P(y|{\bm{x}},i)$ is
usually estimated with a softmax function by classifier $f$, and the class
prior $P(y)$ adopts the sample frequency for each ID class.
$\gamma_{y}({\bm{x}})$ is under-explored and simply treated as a constant.
Under this circumstance, since $\sum_{y}{P(y|{\bm{x}},i)}=1$, the bias item
can be viewed as a weighted sum of the reciprocal prior $\frac{1}{P(y)}$.
Theorem 3.2 explains how $\beta(x)$ causes the gap between balanced (ideal)
ID/OOD probability $P^{\text{bal}}(i|{\bm{x}})$ and imbalanced (learned)
$P(i|{\bm{x}})$:

  * •

Given a sample ${\bm{x}}$ from an ID tail-class $y_{t}$ with a small prior
$P(y_{t})$, when the classification probability $P(y_{t}|{\bm{x}},i)$ gets
higher, the term $\beta({\bm{x}})$ becomes larger. Compared to the original
$P(i|{\bm{x}})$ (learned by $g$), the calibrated probability
$P^{\text{bal}}(i|{\bm{x}})$ _i.e._ , $P(i|{\bm{x}})\cdot\beta({\bm{x}})$ is
more likely to identify the sample $x$ as in-distribution, rather than OOD.

  * •

Given a sample ${\bm{x}}^{\prime}$ from OOD data, as the classifier $f$ tends
to produce a higher head-class probability $P(y_{h}|{\bm{x}}^{\prime},i)$ and
a lower tail-class $P(y_{t}|{\bm{x}}^{\prime},i)$ [20], the term
$\beta({\bm{x}}^{\prime})$ becomes smaller. Compared to the original
$P(i|{\bm{x}}^{\prime})$, the calibrated probability
$P^{\text{bal}}(i|{\bm{x}}^{\prime})$ _i.e._ ,
$P(i|{\bm{x}}^{\prime})\cdot\beta({\bm{x}}^{\prime})$ is more likely to
identify the sample $x^{\prime}$ as out-of-distribution, rather than ID.

The above analysis is consistent with the statistical behaviors (see Fig. 1)
of a vanilla OOD detector $g$. Compared to an ideal balanced detector
$g^{\text{bal}}$, $g$ is prone to wrongly detect ID samples from tail class as
OOD, and simultaneously wrongly detect OOD samples as head class from ID.

###  3.3 Towards Balanced OOD Detector Learning

To address the identified bias in OOD detection due to class imbalance, we
present a unified approach from a statistical perspective. Our goal is to push
the learned detector $g$ towards the balanced $g^{\text{bal}}$. We outline the
overall formula for estimating the OOD posterior below.

Specially, we use the common practices [35, 20] to estimate the probability
distribution
$P^{\text{bal}}(i|{\bm{x}})=\sum_{y}{P^{\text{bal}}(y|{\bm{x}})}=\sum_{y}{\gamma_{y}({\bm{x}})\frac{P(y|{\bm{x}},i)}{P(y)}P(i|{\bm{x}})}$
in Theorem 3.2:

First, for the class prior $P(y)$, we use the label frequency of the training
dataset [6, 35], expressed as
$P(y)\coloneqq\frac{n_{y}}{\sum_{y^{\prime}}n_{y^{\prime}}}\triangleq\pi_{y}$,
where $n_{y}$ refers to the number of instances in class $y$. For the class
posterior $P(y|{\bm{x}},i)$, given a learned classifier $f$, the
classification probability can be estimated using a softmax function [35, 20]:
$P(y|{\bm{x}},i)\coloneqq\frac{e^{f_{y}({\bm{x}})}}{\sum_{y^{\prime}}e^{f_{y^{\prime}}}({\bm{x}})}\triangleq
p_{y|{\bm{x}},i}$, where $f_{y}({\bm{x}})$ represents the logit for class $y$.
For the OOD posterior $P(i|{\bm{x}})$, since OOD detection is a binary
classification task [3], the posterior probability for an arbitrary OOD
detector $g$ [17, 28, 25] can be estimated using a sigmoid function:
$P(i|{\bm{x}})\coloneqq\frac{1}{1+e^{-g({\bm{x}})}}$, where $g({\bm{x}})$ is
the ID/OOD logit. Finally, for the class-specific scaling factor
$\gamma_{y}({\bm{x}})$, estimating $\gamma_{y}({\bm{x}})$ is a sophisticated
problem in long-tailed image recognition [35, 22]. To focus on the OOD
detection problem, we use a parametric mapping
$\gamma_{y;\theta}:{\bm{x}}\mapsto(0,\infty)$, where $\theta$ are learnable
parameters that are optimized through gradient back-propagation.

This unified approach allows us to systematically estimate and correct for the
bias introduced by class imbalance, thereby improving the performance of OOD
detection in real-world scenarios. According to Theorem 3.2, the balanced OOD
detectors $g^{\text{bal}}$ is modeled as:

$\displaystyle\sigma(g^{\text{bal}}({\bm{x}}))=\left(\sum_{y}{\gamma_{y;\theta}({\bm{x}})\frac{p_{y|{\bm{x}},i}}{\pi_{y}}}\right)\cdot\sigma(g({\bm{x}}))\triangleq\beta({\bm{x}})\cdot\sigma(g({\bm{x}}))$
(5)

Substitute the sigmoid function $\sigma(z)=\frac{1}{1+e^{-z}}$ into Eq. 5, we
have:

$\displaystyle
g({\bm{x}})=g^{\text{bal}}({\bm{x}})-\log{\left[(\beta({\bm{x}})-1)\cdot
e^{g^{\text{bal}}({\bm{x}})}+\beta({\bm{x}})\right]}$ (6)

The derivation is displayed in Sec. A.3. In order to make the detector
$g({\bm{x}})$ directly estimate the balanced OOD detection distribution, we
can apply the binary cross-entropy loss on calibrated logits:

$\displaystyle\mathcal{L}_{\text{ood}}=\mathcal{L}_{\text{BCE}}\left(g({\bm{x}})-\log{\left[(\beta({\bm{x}})-1)\cdot
e^{g({\bm{x}})}+\beta({\bm{x}})\right]},t\right)\triangleq\mathcal{L}_{\text{BCE}}\left(g({\bm{x}})-\Delta({\bm{x}}),t\right)$
(7)

where $t=\bm{1}\\{{\bm{x}}\in\mathcal{X}^{in}\\}$ indicates whether the sample
${\bm{x}}$ comes from ID or not. For a ID sample from tail classes with $t=1$,
discussed in Sec. 3.2, the bias $\beta$ is larger than samples from head
classes. In this situation, the punishment $\Delta$ correspondingly increases,
which encourages the detector $g$ to generate a higher score $g({\bm{x}})$ to
reduce the loss. On the other hand, for a OOD sample that predicted as a head
ID class by classifier $f$, $\beta$ and $\Delta$ become smaller than those
predicted as tail ID classes, and $g$ are further forced to reduce the score
$g({\bm{x}})$ to decrease the loss, as the label $t$ for OOD samples is 0. In
practice, to alleviate the optimization difficulty, the $\Delta({\bm{x}})$ for
ID samples (with $t=1$) is cut off to be non-negative value, and
$\Delta({\bm{x}}\prime)$ for OOD samples (with $t=0$) is cut off to be non-
positive values, ensuring the optimization direct dose not conflict with the
vanilla BCE loss function.

Meanwhile, according to Lemma 3.1, we add an extra constraint on
$\gamma_{y;\theta}$ to ensure the posterior estimate
$P^{\text{bal}}(y|{\bm{x}})=\gamma_{y}({\bm{x}})\cdot\frac{p_{y|{\bm{x}},i}}{\pi_{y}}\cdot\sigma(g({\bm{x}}))$
and $P^{\text{bal}}(i|{\bm{x}})=\sum_{y}{P^{\text{bal}}(y|{\bm{x}})}$ will not
exceed 1:

$\displaystyle\mathcal{L}_{\gamma}=\max\left\\{0,\sum_{y}{\gamma_{y;\theta}({\bm{x}})\cdot\frac{p_{y|{\bm{x}},i}}{\pi_{y}}}\cdot\sigma(g({\bm{x}}))-1\right\\}=\max\left\\{0,\beta({\bm{x}})\cdot\sigma(g({\bm{x}}))-1\right\\}$
(8)

Note that for each class $y$, the term
$\frac{p_{y|{\bm{x}},i}}{\pi_{y}}\cdot\sigma(g({\bm{x}}))>0$ always holds, so
that we only have to constrain the summary on all classes
$P^{\text{bal}}(i|{\bm{x}})=\sum_{y}{P^{\text{bal}}(y|{\bm{x}})}$ within 1, as
indicated in Eq. 8, and $P^{\text{bal}}(y|{\bm{x}})$ for each class $y$ will
be constrained as well.

Combining with $\mathcal{L}_{\text{ood}}$ and $\mathcal{L}_{\gamma}$ for
optimization, the learned OOD detector $g^{*}$ has already estimated the
balanced $P^{\text{bal}}(i|{\bm{x}})$. We thus predict the ID/OOD probability
as usual: $\hat{p}(i|{\bm{x}})=\frac{1}{1+e^{-g^{*}({\bm{x}})}}$. The other
terms like $\gamma$, $\beta$, and $\Delta$ are no longer needed to compute,
maintaining the inference efficiency and simplicity for OOD detection
applications.

##  4 Experiments

In this section, we empirically validate the effectiveness of our ImOOD on
several representative imbalanced OOD detection benchmarks. The experimental
setup is described in Sec. 4.1, based on which extensive experiments and
discussions are displayed in Sec. 4.2 and Sec. 4.3.

###  4.1 Setup

Datasets. Following the literature [47, 20, 8, 36], we use the popular
CIFAR10-LT, CIFAR100-LT [6], and ImageNet-LT [30] as imbalanced in-
distribution datasets.

For CIFAR10/100-LT benchmarks, the imbalance ratio (_i.e._ ,
$\rho=\max_{y}(n_{y})/\min_{y}(n_{y})$) is set as 100 [6, 47]. The original
CIAFR10/100 test sets are kept for evaluating the ID classification
capability. For OOD detection, the TinyImages80M [46] is adopted as the
auxiliary OOD training data, and the test set is semantically coherent out-of-
distribution (SC-OOD) benchmark [52].

For the large-scale ImageNet-LT benchmark, training samples are sampled from
the original ImageNet-1k [10] dataset, and the validation set is taken for
evaluation. We follow the OOD detection setting as Wang et al. [47] to use
ImageNet-Extra as auxiliary OOD training and ImageNet-1k-OOD for testing.
Randomly sampled from ImageNet-22k [10], ImageNet-Extra contains 517,711
images belonging to 500 classes, and ImageNet-1k-OOD consists of 50,000 images
from 1,000 classes. All the classes in ImageNet-LT, ImageNet-Extra, and
ImageNet-1k-OOD are not overlapped.

Evaluation Metrics. For OOD detection, we follow Hendrycks et al. [17], Yang
et al. [52], Wang et al. [47] to report three metrics: (1) AUROC, the area
under the receiver operating characteristic curve, (2) AUPR, the area under
precision-recall curve., and (3) FPR95, the false positive rate of OOD samples
when the true positive rate of ID samples are 95%. For ID classification, we
measure the macro accuracy of the classifier. We report the mean and standard
deviation of performance (%) over six random runs for each method.

Implementation Details. For the ID classifier $f$, following the settings of
Wang et al. [47], we train ResNet18 [15] models on the CIFAR10/100LT
benchmarks, and leverage ResNet50 models for the ImageNet-LT benchmark. Logit
adjustment loss [35] is adopted to alleviate the imbalanced ID classification.
Detailed settings are displayed in Sec. B.1. For the OOD detector $g$, as
Bitterwolf et al. [3] suggest, we implement $g$ as a binary discriminator
(abbreviated as BinDisc) to perform ID/OOD identification. Detector $g$ shares
the same backbone (feature extractor) as classifier $f$, and $g$ only attaches
an additional output node to the classification layer of $f$. In addition, we
also add a linear layer on top of the backbone to produce the $\gamma$ factors
to perform the training regularization with Eq. 7 and Eq. 8. To reduce the
optimization difficulty, the gradient is stopped between Eq. 7 and Eq. 8 (but
still shared in the backbone), where Eq. 7 aims at training $g$ while Eq. 8
only optimize $\gamma$. Furthermore, to verify the versatility of our method,
we also implement several representative OOD detection methods (_e.g._ , OE
[17], Energy [28], _etc._) into binary discriminators, and equip them with our
ImOOD framework. For more details please refer to Sec. B.2.

Methods for comparison. In the following sections, we mainly compare our
method on three benchmarks with the typical OOD detectors including OE [17],
Energy [28], and BinDisc [3], as well as some state-of-the-art detectors such
as PASCL [47], ClassPrior [20], EAT [51], COCL [36], _etc._. Specifically, as
the results on the ImageNet-LT benchmark reported by COCL share a large
discrepancy against PASCL (especially the AUPR measure), we re-implement COCL
based on their released code111https://github.com/mala-lab/COCL and report the
aligned results in Tab. 2.

###  4.2 Main Results

Table 1: OOD detection evaluation on CIFAR10/100-LT benchmarks. The best
results are marked in bold, and the secondary results are marked with
underlines. The base model is ResNet18.
Method | CIFAR10-LT | CIFAR100-LT
---|---|---
AUROC↑ | AUPR↑ | FPR95↓ | ACC↑ | AUROC↑ | AUPR↑ | FPR95↓ | ACC↑
MSP | 72.28 | 70.27 | 66.07 | 72.34 | 61.00 | 57.54 | 82.01 | 40.97
OECC | 87.28 | 86.29 | 45.24 | 60.16 | 70.38 | 66.87 | 73.15 | 32.93
EnergyOE | 89.31 | 88.92 | 40.88 | 74.68 | 71.10 | 67.23 | 71.78 | 39.05
OE | 89.77 | 87.25 | 34.65 | 73.84 | 72.91 | 67.16 | 68.89 | 39.04
PASCL | 90.99 | 89.24 | 33.36 | 77.08 | 73.32 | 67.18 | 67.44 | 43.10
OpenSampling | 91.94 | 91.08 | 36.92 | 75.78 | 74.37 | 75.80 | 78.18 | 40.87
ClassPrior | 92.08 | 91.17 | 34.42 | 74.33 | 76.03 | 77.31 | 76.43 | 40.77
BalEnergy | 92.56 | 91.41 | 32.83 | 81.37 | 77.75 | 78.61 | 73.10 | 45.88
EAT | 92.87 | 92.40 | 28.83 | 81.31 | 75.45 | 70.87 | 64.01 | 46.23
COCL | 93.28 | 92.24 | 30.88 | 81.56 | 78.25 | 79.37 | 74.09 | 46.41
PASCL + Ours | 92.93 | 92.51 | 28.73 | 78.96 | 74.23 | 68.63 | 65.65 | 44.60
COCL + Ours | 93.55 | 92.83 | 28.52 | 81.83 | 78.50 | 79.96 | 71.65 | 46.80

ImOOD significantly outperforms previous SOTA methods on CIFAR10/100-LT
benchmarks. As shown in Tab. 1, our ImOOD achieves new SOTA performance on
both of CIFAR10/100-LT benchmarks. Built on top of the strong baseline PASCL
[47], our method leads to 1.9% increase of AUROC, 3.2% increase of AUPR, and
4.6% decrease of FPR95 on CIFAR10-LT, with 0.9% - 1.8% enhancements of
respective evaluation metrics on CIFAR100-LT. By integrating with COCL [36],
our ImOOD further pushes the imbalanced OOD detection on CIFAR10/100-LT
benchmarks towards a higher performance, _e.g._ , achieving 93.55%/78.50% of
AUROC respectively. To further demonstrate the efficacy, we validate our
method on the real-world large-scale ImageNet-LT [30] benchmark, and the
results are displayed below.

Table 2: OOD detection evaluation on the ImageNet-LT benchmark. The base model
is ResNet50.
Method | AUROC↑ | AUPR↑ | FPR95↓ | ACC↑
---|---|---|---|---
MSP | 53.81 | 51.63 | 90.15 | 39.65
OECC | 63.07 | 63.05 | 86.90 | 38.25
EnergyOE | 64.76 | 64.77 | 87.72 | 38.50
OE | 66.33 | 68.29 | 88.22 | 37.60
PASCL | 68.00 | 70.15 | 87.53 | 45.49
EAT | 69.84 | 69.25 | 87.63 | 46.79
COCL | 73.87 | 72.63 | 76.35 | 51.00
PASCL + Ours | 74.69 | 73.08 | 74.37 | 46.63
COCL + Ours | 75.84 | 73.19 | 74.96 | 52.43

ImOOD achieves superior performance on the ImageNet-LT benchmark. As Tab. 2
implies, our ImOOD brings significant improvements against PASCL, _e.g._ ,
6.7% increase on AUROC and 13.2% decrease on FPR95, and further enhance the
SOTA method COCL for a better OOD detection performance, _e.g._ , 75.84 of
AUROC. Since the performance enhancement is much greater than those on
CIFAR10/100-LT benchmarks, we further statistic the class-aware error
distribution on wrongly detected ID/OOD sample in Fig. A1. The results
indicate our method builds a relatively better-balanced OOD detector on
ImageNet-LT, which leads to higher performance improvements. Besides, as we
follow the literature [47, 36] to employ ResNet18 on CIFAR10/100-LT while
adopt ResNet50 on ImageNet-LT, the model capacity also seems to play a vital
role in balancing the OOD detection on imbalanced data distribution,
particularly in more challenging real-world scenarios.

###  4.3 Ablation Studies

In this section, we conduct in-depth ablation studies on the CIFAR10-LT
benchmark to assess the validity and versatility of our proposed ImOOD
framework, and the results are reported as follows.

Table 3: Ablation on the $\gamma_{y}$ estimates and technique integration on
the CIFAR10-LT benchmark.
$\gamma_{y}$ Estimates | AUROC↑ | AUPR↑ | FPR95↓ | ID ACC↑
---|---|---|---|---
none | 90.06 | 88.72 | 33.39 | 78.22
$\gamma_{y}\coloneqq const$ | 89.75 | 86.28 | 32.67 | 78.50
$\gamma_{y}\coloneqq\gamma_{y;\theta}$ | 92.04 | 91.32 | 31.24 | 79.16
$\gamma_{y}\coloneqq\gamma_{y;\theta}({\bm{x}})$ | 92.23 | 91.92 | 29.95 | 79.56
\+ PASCL | 92.93 | 92.51 | 28.73 | 78.96
\+ COCL | 93.55 | 92.83 | 28.52 | 81.83

Figure 2: Statistics on $\gamma$, $\beta$, and $\Delta$ from the CIFAR10-LT
benchmark. (1) Upper: distributions on ID samples from head to tail (left to
right) class indices; (2) Lower: distributions on OOD samples predicted as
head to tail (left to right) ID classes.

Lemma 3.1
($P^{\text{bal}}(y|{\bm{x}})=\gamma_{y}({\bm{x}})\cdot\frac{P(y|{\bm{x}})}{P(y)}$)
is consistent with empirical results. To validate that the coefficient
$\gamma_{y}({\bm{x}})$ depends on input sample ${\bm{x}}$ and differs for each
class $y$, we perform a series of ablation studies in Tab. 3. We first build a
baseline model with BinDisc [3] only, and no extra regularization is adopted
to mitigate the imbalanced OOD detection. As shown in the first row from Tab.
3, the baseline (denoted as none of $\gamma_{y}$ estimates) presents a fair
OOD detection performance (_e.g._ , 90.06% of AUROC and 33.39% of FPR95).
Then, we simply take $\gamma_{y}$ as a constant for all classes (denoted as
$\gamma_{y}\coloneqq const$) by assuming
$P^{\text{bal}}({\bm{x}}|y)=P({\bm{x}}|y)$ and
$\gamma_{y}\coloneqq\frac{1}{K}$ (see Sec. A.1) to apply the training
regularization in Sec. 3.3. According to Tab. 3, the OOD detection performance
receives a slight decline of AUROC (from 90.06% to 89.75%), despite the better
FPR95 result. Consequently, after treating $\gamma_{y}$ as a learnable
variable for each class $y$ (denoted as
$\gamma_{y}\coloneqq\gamma_{y;\theta}$), the detector receives significant
improvement on all the three measures of AUROC, AUPR, and FPR95. Finally,
setting $\gamma_{y}$ as an input-dependent and class-aware learnable variable
(denoted as $\gamma_{y}\coloneqq\gamma_{y;\theta}({\bm{x}})$) brings further
OOD detection enhancement.

In addition, we further statistics the practical distribution of $\gamma$ in
Fig. 2, where $\gamma_{y}({\bm{x}})$ is relatively higher for the ID sample
from head classes. It is consistent with
$\gamma_{y}({\bm{x}})=\frac{1}{K}\frac{P^{\text{bal}}({\bm{x}}|y)}{P({\bm{x}}|y)}$
(see Sec. A.1), as the data likelihood $P({\bm{x}}|y)$ is close to the
balanced situation ($P^{\text{bal}}({\bm{x}}|y)$) for head samples while over-
estimated for tail classes, leading to a higher fraction of
$\frac{P^{\text{bal}}({\bm{x}}|y)}{P({\bm{x}}|y)}$ for head classes and lower
for tail classes. Detailed discussion is presented in Sec. A.2.

The designed training-time regularization in Sec. 3.3 is effective. As shown
in Tab. 3, compared with the BinDisc baseline in the first row, adding Eq. 7
and Eq. 8 by setting $\gamma_{y}\coloneqq\gamma_{y;\theta}({\bm{x}})$ leads to
significant improvement on the OOD detection performance (_i.e._ , 2.2%
increase of AUROC and 3.4% decrease of FPR95). As Fig. 2 shows, the
automatically learned adjustments (_e.g._ , $\beta$ and $\Delta$) are
consistent with our motivation, where we aim to punish the ID data from tail
classes with a large positive $\Delta$, as well as the OOD samples predicted
as ID head classes with a large negative $\Delta$ (Sec. 3.3). Moreover, as
indicated by Tab. 3, integrating with PASCL [47] and COCL [36] techniques
further boosts the imbalanced OOD detection, ultimately resulting in a new
SOTA performance.

ImOOD generalizes to various OOD detection methods. To verify the versatility
of our statistical framework, we implement the ImOOD framework with different
OOD detectors beside BinDisc, including the OE [17], Energy [28], and the
Mahalanobis distance [21]. For those detectors, we add an extra linear layer
to conduct logistic regression on top of their vanilla OOD scores (see Sec.
B.2), and leverage our training-time regularization to optimize those
detectors in a unified manner. According to the results presented in Sec. 4.3,
our ImOOD consistently boosts the original OOD detectors by a large margin on
all the AUROC, AUPR, and FPR95 measures. In real-world applications, one may
choose a proper formulation of our ImOOD to meet specialized needs.

Table 4: Generalization to OOD detectors. OOD Detector | Method | AUROC↑ | AUPR↑ | FPR95↓
---|---|---|---|---
Prob-Based | OE | 89.91 | 87.32 | 34.06
| +Ours | 91.52 | 89.03 | 30.64
Energy-Based | Energy | 90.27 | 88.73 | 34.42
| +Ours | 91.41 | 90.63 | 31.81
Dist-Based | Maha | 88.26 | 87.94 | 42.74
| +Ours | 89.30 | 88.68 | 39.54
Table 5: Robustness to OOD test sets. OOD Dataset | Method | AUROC↑ | AUPR↑ | FPR95↓
---|---|---|---|---
Far-OOD | PASCL | 96.63 | 98.06 | 12.18
| +Ours | 97.50 | 98.26 | 9.65
Near-OOD | PASCL | 84.43 | 82.99 | 57.27
| +Ours | 86.61 | 85.71 | 55.51
Spurious-OOD | PASCL | 79.00 | 81.83 | 63.57
| +Ours | 83.27 | 84.19 | 57.69

ImOOD is robust to different OOD test sets. In the preceding sections, we
evaluated our method on the CIFAR10-LT benchmark, where the SCOOD test set
[52] comprises 6 subsets covering different scenarios. As suggested by Fort et
al. [14], the SVHN subset can be viewed as far OOD, and the CIFAR100 subset
can be seen as near OOD (with CIFAR10-LT as ID). According to the detailed
results in Sec. 4.3, our ImOOD brings consistent enhancement against the
strong baseline PASCL regardless of the near or far OOD test set. Furthermore,
we also report the spurious OOD detection evaluation in Sec. 4.3.
Specifically, we follow Ming et al. [38] to take WaterBird as the imbalanced
ID dataset, which also suffers from the imbalance problem (on water birds and
land birds), and a subset of Places [55] as the spurious OOD test set (with
spurious correlation to background). Results in Sec. 4.3 also demonstrate our
method’s robustness in handling spurious OOD problems, with a considerable
improvement of 4.3% increase on AUROC and 5.96% decrease on FPR95. The
robustness of ImOOD to various OOD testing scenarios is verified.

##  5 Conclusion and Discussion

This paper establishes a statistical framework ImOOD to formulate OOD
detection on imbalanced data distribution. Through theoretical analysis, we
find there exists a class-aware biased item between balanced and imbalanced
OOD detection models. Based on it, our ImOOD provides a unified training-time
regularization technique to alleviate the imbalance problem. On three popular
imbalanced OOD detection benchmarks, extensive experiments and ablation
studies to demonstrate the validity and versatility of our method. We hope our
work can inspire new research in this community.

Limitations. Following the literature, ImOOD utilizes auxiliary OOD training
samples to refine the the ID/OOD decision boundary. However, unforeseen OOD
samples in real-world applications could potentially challenge this boundary.
To mitigate this issue, integrating online-learning strategies for adaptive
decision-making during testing is a promising avenue. We view this as our
future work.

## References

  * Ahn et al. [2023] Yong Hyun Ahn, Gyeong-Moon Park, and Seong Tae Kim.  Line: Out-of-distribution detection by leveraging important neurons.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 19852–19862, 2023.
  * Bendale and Boult [2015] Abhijit Bendale and Terrance Boult.  Towards open world recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 1893–1902, 2015.
  * Bitterwolf et al. [2022] Julian Bitterwolf, Alexander Meinke, Maximilian Augustin, and Matthias Hein.  Breaking down out-of-distribution detection: Many methods based on ood training data estimate a combination of the same core quantities.  In _International Conference on Machine Learning_ , pages 2041–2074. PMLR, 2022.
  * Brodersen et al. [2010] Kay Henning Brodersen, Cheng Soon Ong, Klaas Enno Stephan, and Joachim M Buhmann.  The balanced accuracy and its posterior distribution.  In _2010 20th international conference on pattern recognition_ , pages 3121–3124. IEEE, 2010.
  * Buda et al. [2018] Mateusz Buda, Atsuto Maki, and Maciej A Mazurowski.  A systematic study of the class imbalance problem in convolutional neural networks.  _Neural networks_ , 106:249–259, 2018.
  * Cao et al. [2019] Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma.  Learning imbalanced datasets with label-distribution-aware margin loss.  _Advances in neural information processing systems_ , 32, 2019.
  * Cao and Zhang [2022] Senqi Cao and Zhongfei Zhang.  Deep hybrid models for out-of-distribution detection.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 4733–4743, 2022.
  * Choi et al. [2023] Hyunjun Choi, Hawook Jeong, and Jin Young Choi.  Balanced energy regularization loss for out-of-distribution detection.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 15691–15700, 2023.
  * Cimpoi et al. [2014] Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi.  Describing textures in the wild.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 3606–3613, 2014.
  * Deng et al. [2009] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE conference on computer vision and pattern recognition_ , pages 248–255. Ieee, 2009.
  * Djurisic et al. [2023] Andrija Djurisic, Nebojsa Bozanic, Arjun Ashok, and Rosanne Liu.  Extremely simple activation shaping for out-of-distribution detection.  In _International Conference on Learning Representations_ , 2023.
  * Du et al. [2022] Xuefeng Du, Zhaoning Wang, Mu Cai, and Yixuan Li.  Vos: Learning what you don’t know by virtual outlier synthesis.  In _International Conference on Learning Representations_ , 2022.
  * Esmaeilpour et al. [2022] Sepideh Esmaeilpour, Bing Liu, Eric Robertson, and Lei Shu.  Zero-shot out-of-distribution detection based on the pre-trained model clip.  _Proceedings of the AAAI conference on artificial intelligence_ , 36(6):6568–6576, 2022.
  * Fort et al. [2021] Stanislav Fort, Jie Ren, and Balaji Lakshminarayanan.  Exploring the limits of out-of-distribution detection.  _Advances in Neural Information Processing Systems_ , 34:7068–7081, 2021.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 770–778, 2016.
  * Hendrycks and Gimpel [2017] Dan Hendrycks and Kevin Gimpel.  A baseline for detecting misclassified and out-of-distribution examples in neural networks.  In _International Conference on Learning Representations_ , 2017.
  * Hendrycks et al. [2019] Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich.  Deep anomaly detection with outlier exposure.  In _International Conference on Learning Representations_ , 2019.
  * Huang et al. [2021] Rui Huang, Andrew Geng, and Yixuan Li.  On the importance of gradients for detecting distributional shifts in the wild.  _Advances in Neural Information Processing Systems_ , 34:677–689, 2021.
  * Huang et al. [2023] Zhuo Huang, Xiaobo Xia, Li Shen, Bo Han, Mingming Gong, Chen Gong, and Tongliang Liu.  Harnessing out-of-distribution examples via augmenting content and style.  In _International Conference on Learning Representations_ , 2023.
  * Jiang et al. [2023] Xue Jiang, Feng Liu, Zhen Fang, Hong Chen, Tongliang Liu, Feng Zheng, and Bo Han.  Detecting out-of-distribution data through in-distribution class prior.  In _International Conference on Machine Learning_ , 2023.
  * Jie et al. [2021] Ren Jie, Fort Stanislav, Liu Jeremiah, Roy Abhijit Guha, Padhy Shreyas, and Lakshminarayanan Balaji.  A simple fix to mahalanobis distance for improving near-ood detection, 2021.
  * Kirichenko et al. [2020] Polina Kirichenko, Pavel Izmailov, and Andrew G Wilson.  Why normalizing flows fail to detect out-of-distribution data.  _Advances in neural information processing systems_ , 33:20578–20589, 2020.
  * Lafon et al. [2023] Marc Lafon, Elias Ramzi, Clément Rambour, and Nicolas Thome.  Hybrid energy based model in the feature space for out-of-distribution detection.  In _International Conference on Machine Learning_ , 2023.
  * Le and Yang [2015] Ya Le and Xuan Yang.  Tiny imagenet visual recognition challenge.  _CS 231N_ , 7(7):3, 2015.
  * Lee et al. [2018] Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin.  A simple unified framework for detecting out-of-distribution samples and adversarial attacks.  _Advances in neural information processing systems_ , 31, 2018.
  * Liang et al. [2018] Shiyu Liang, Yixuan Li, and R Srikant.  Enhancing the reliability of out-of-distribution image detection in neural networks.  In _International Conference on Learning Representations_ , 2018.
  * Liu et al. [2023a] Kai Liu, Zhihang Fu, Chao Chen, Sheng Jin, Ze Chen, Mingyuan Tao, Rongxin Jiang, and Jieping Ye.  Category-extensible out-of-distribution detection via hierarchical context descriptions.  In _Thirty-seventh Conference on Neural Information Processing Systems_ , 2023a.
  * Liu et al. [2020] Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li.  Energy-based out-of-distribution detection.  _Advances in neural information processing systems_ , 33:21464–21475, 2020.
  * Liu et al. [2023b] Xixi Liu, Yaroslava Lochman, and Christopher Zach.  Gen: Pushing the limits of softmax-based out-of-distribution detection.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 23946–23955, 2023b.
  * Liu et al. [2019] Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, and Stella X Yu.  Large-scale long-tailed recognition in an open world.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 2537–2546, 2019.
  * Lu et al. [2023] Fan Lu, Kai Zhu, Wei Zhai, Kecheng Zheng, and Yang Cao.  Uncertainty-aware optimal transport for semantically coherent out-of-distribution detection.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 3282–3291, 2023.
  * Maddox et al. [2019] Wesley J Maddox, Pavel Izmailov, Timur Garipov, Dmitry P Vetrov, and Andrew Gordon Wilson.  A simple baseline for bayesian uncertainty in deep learning.  _Advances in neural information processing systems_ , 32, 2019.
  * Malinin and Gales [2018] Andrey Malinin and Mark Gales.  Predictive uncertainty estimation via prior networks.  _Advances in neural information processing systems_ , 31, 2018.
  * Menon et al. [2013] Aditya Menon, Harikrishna Narasimhan, Shivani Agarwal, and Sanjay Chawla.  On the statistical consistency of algorithms for binary classification under class imbalance.  In _International Conference on Machine Learning_ , pages 603–611. PMLR, 2013.
  * Menon et al. [2021] Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, and Sanjiv Kumar.  Long-tail learning via logit adjustment.  In _International Conference on Learning Representations_ , 2021.
  * Miao et al. [2024] Wenjun Miao, Guansong Pang, Tianqi Li, Xiao Bai, and Jin Zheng.  Out-of-distribution detection in long-tailed recognition with calibrated outlier class learning.  _Proceedings of the 38th AAAI Conference on Artificial Intelligence_ , 2024.
  * Ming et al. [2022a] Yifei Ming, Ziyang Cai, Jiuxiang Gu, Yiyou Sun, Wei Li, and Yixuan Li.  Delving into out-of-distribution detection with vision-language representations.  _Advances in Neural Information Processing Systems_ , 35:35087–35102, 2022a.
  * Ming et al. [2022b] Yifei Ming, Hang Yin, and Yixuan Li.  On the impact of spurious correlation for out-of-distribution detection.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 36, pages 10051–10059, 2022b.
  * Netzer et al. [2011] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng.  Reading digits in natural images with unsupervised feature learning.  2011\.
  * Radford et al. [2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In _International Conference on Machine Learning_ , pages 8748–8763. PMLR, 2021.
  * Sapkota and Yu [2023] Hitesh Sapkota and Qi Yu.  Adaptive robust evidential optimization for open set detection from imbalanced data.  In _International Conference on Learning Representations_ , 2023.
  * Sun et al. [2021] Yiyou Sun, Chuan Guo, and Yixuan Li.  React: Out-of-distribution detection with rectified activations.  In _Advances in Neural Information Processing Systems_ , 2021.
  * Sun et al. [2022] Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li.  Out-of-distribution detection with deep nearest neighbors.  In _International Conference on Machine Learning_ , pages 20827–20840. PMLR, 2022.
  * Tao et al. [2023a] Leitian Tao, Xuefeng Du, Jerry Zhu, and Yixuan Li.  Non-parametric outlier synthesis.  In _International Conference on Learning Representations_ , 2023a.
  * Tao et al. [2023b] Leitian Tao, Xuefeng Du, Jerry Zhu, and Yixuan Li.  Non-parametric outlier synthesis.  In _International Conference on Learning Representations_ , 2023b.
  * Torralba et al. [2008] Antonio Torralba, Rob Fergus, and William T Freeman.  80 million tiny images: A large data set for nonparametric object and scene recognition.  _IEEE transactions on pattern analysis and machine intelligence_ , 30(11):1958–1970, 2008.
  * Wang et al. [2022] Haotao Wang, Aston Zhang, Yi Zhu, Shuai Zheng, Mu Li, Alex J Smola, and Zhangyang Wang.  Partial and asymmetric contrastive learning for out-of-distribution detection in long-tailed recognition.  In _International Conference on Machine Learning_ , pages 23446–23458. PMLR, 2022.
  * Wang et al. [2023a] Qizhou Wang, Junjie Ye, Feng Liu, Quanyu Dai, Marcus Kalander, Tongliang Liu, HAO Jianye, and Bo Han.  Out-of-distribution detection with implicit outlier transformation.  In _International Conference on Learning Representations_ , 2023a.
  * Wang et al. [2023b] Qizhou Wang, Junjie Ye, Feng Liu, Quanyu Dai, Marcus Kalander, Tongliang Liu, HAO Jianye, and Bo Han.  Out-of-distribution detection with implicit outlier transformation.  In _International Conference on Learning Representations_ , 2023b.
  * Wei et al. [2022] Hongxin Wei, Renchunzi Xie, Hao Cheng, Lei Feng, Bo An, and Yixuan Li.  Mitigating neural network overconfidence with logit normalization.  In _International Conference on Machine Learning_ , pages 23631–23644. PMLR, 2022.
  * Wei et al. [2024] Tong Wei, Bo-Lin Wang, and Min-Ling Zhang.  Eat: Towards long-tailed out-of-distribution detection.  _Proceedings of the 38th AAAI Conference on Artificial Intelligence_ , 2024.
  * Yang et al. [2021a] Jingkang Yang, Haoqi Wang, Litong Feng, Xiaopeng Yan, Huabin Zheng, Wayne Zhang, and Ziwei Liu.  Semantically coherent out-of-distribution detection.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 8301–8309, 2021a.
  * Yang et al. [2021b] Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu.  Generalized out-of-distribution detection: A survey.  _arXiv preprint arXiv:2110.11334_ , 2021b.
  * Yu et al. [2015] Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao.  Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop.  _arXiv preprint arXiv:1506.03365_ , 2015.
  * Zhou et al. [2017] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba.  Places: A 10 million image database for scene recognition.  _IEEE transactions on pattern analysis and machine intelligence_ , 40(6):1452–1464, 2017.
  * Zhu et al. [2022] Yao Zhu, YueFeng Chen, Chuanlong Xie, Xiaodan Li, Rong Zhang, Hui Xue, Xiang Tian, Yaowu Chen, et al.  Boosting out-of-distribution detection with typical features.  _Advances in Neural Information Processing Systems_ , 35:20758–20769, 2022.

##  Appendix A Theorem Proofs

###  A.1 Proof for Lemma 3.1

From Bayesian Theorem, we have:

$\displaystyle P(y|{\bm{x}})=\frac{P({\bm{x}}|y)\cdot P(y)}{P({\bm{x}})}$ (A1)

In class-balanced scenarios, the class prior $P^{\text{bal}}(y)$ equals
$\frac{1}{K}$, where $K$ is the class number. The balanced classification
posterior is given by:

$\displaystyle
P^{\text{bal}}(y|{\bm{x}})=\frac{P^{\text{bal}}({\bm{x}}|y)\cdot\frac{1}{K}}{P^{\text{bal}}({\bm{x}})}$
(A2)

where the marginal probability $P^{\text{bal}}({\bm{x}})\equiv P({\bm{x}})$ is
independent from balanced or imbalanced class distribution. Combining with Eq.
A1 and Eq. A2, we have:

$\displaystyle
P^{\text{bal}}(y|{\bm{x}})=\frac{P^{\text{bal}}({\bm{x}}|y)}{P({\bm{x}}|y)}\cdot\frac{1}{K}\cdot\frac{P(y|vx)}{P(y)}\triangleq\gamma_{y}({\bm{x}})\cdot\frac{P(y|vx)}{P(y)}$
(A3)

where
$\gamma_{y}({\bm{x}})=\frac{1}{K}\frac{P^{\text{bal}}({\bm{x}}|y)}{P({\bm{x}}|y)}$
captures the likelihood difference for the same sample x between balanced and
imbalanced distributions.

###  A.2 Discussion for Lemma 3.1 and statistic results in Fig. 2

According to Fig. 2 that depicts the statistical distribution on the
CIFAR10-LT benchmark, $\gamma_{y}({\bm{x}})$ is higher for the ID sample from
head classes than tail classes. This phenomenon is consistent with
$\gamma_{y}({\bm{x}})=\frac{1}{K}\frac{P^{\text{bal}}({\bm{x}}|y)}{P({\bm{x}}|y)}$.
As the model has seen sufficient training examples for head classes, the data
likelihood $P({\bm{x}}|y^{h})$ is approaching the balanced situation of
$P^{\text{bal}}({\bm{x}}|y)$, the fraction of
$\frac{P^{\text{bal}}({\bm{x}}|y)}{P({\bm{x}}|y)}$ is closed to 1. Meanwhile,
since a large number of tail samples are not presented during training, the
sample space is narrowed down and the data likelihood $P({\bm{x}}|y)$ for tail
classes is over-estimated (even higher than $P^{\text{bal}}({\bm{x}}|y)$),
leading to a lower fraction of
$\frac{P^{\text{bal}}({\bm{x}}|y)}{P({\bm{x}}|y)}$. Therefore,
$\gamma_{y}({\bm{x}})=\frac{1}{K}\frac{P^{\text{bal}}({\bm{x}}|y)}{P({\bm{x}}|y)}$
presents higher values for head classes than tail classes. The statistical
results are well-aligned with the theoretical analysis.

###  A.3 Derivation for Eq. 6

Substituting the sigmoid function $\sigma(z)=\frac{1}{1+e^{-z}}$ into Eq. 5,
we have:

$\displaystyle\frac{1}{1+e^{-g^{\text{bal}}({\bm{x}})}}=\beta({\bm{x}})\cdot\frac{1}{1+e^{-g({\bm{x}})}}$
(A4)

Consequently, we have:

$\displaystyle e^{-g({\bm{x}})}=\beta({\bm{x}})\cdot
e^{-g^{\text{bal}}({\bm{x}})}+\beta({\bm{x}})-1$ (A5)

And then:

$\displaystyle g({\bm{x}})$ $\displaystyle=-\log{\beta({\bm{x}})\cdot
e^{-g^{\text{bal}}({\bm{x}})}+\beta({\bm{x}})-1}$ (A6) $\displaystyle=-\log
e^{-g^{\text{bal}}({\bm{x}})}\left[1+(\beta({\bm{x}})-1)\cdot
e^{g^{\text{bal}}({\bm{x}})}\right]$
$\displaystyle=g^{\text{bal}}({\bm{x}})-\log{\left[(\beta({\bm{x}})-1)\cdot
e^{g^{\text{bal}}({\bm{x}})}+1\right]}$

##  Appendix B Experimental settings and implementations

###  B.1 Training settings

For a fair comparison, we mainly follow PASCL’s [47] settings. On
CIFAR10/100-LT benchmarks, we train ResNet18 [15] for 200 epochs using Adam
optimizer, with a batch size of 256. The initial learning rate is 0.001, which
is decayed to 0 using a cosine annealing scheduler. The weight decay is
$5\times 10^{-4}$. On the ImageNet-LT benchmark, we train ResNet50 [15] for
100 epochs with SGD optimizer with the momentum of 0.9. The batch size is 256.
The initial learning rate is 0.1, which is decayed by a factor of 10 at epochs
60 and 80. The weight decay is $5\times 10^{-5}$. During training, each batch
contains an equal number of ID and OOD data samples (_i.e._ , 256 ID samples
and 256 OOD samples).

For a better performance, one may carefully tune the hyper-parameters to train
the models.

###  B.2 Implementation details

In the manuscript, we proposed a generalized statistical framework to
formularize and alleviate the imbalanced OOD detection problem. This section
provides more details on implementing different OOD detectors into a unified
formulation, _e.g._ , a binary ID/OOD classifier [3]:

  * •

For BinDisc [3], we simply append an extra ID/OOD output node to the
classifier layer of a standard ResNet model, where ID classifier $f$ and OOD
detector $g$ share the same feature extractor. Then we adopt the sigmoid
function to convert the logit $g({\bm{x}})$ into the ID/OOD probability
$\hat{p}(i|{\bm{x}})=\frac{1}{1+e^{-g({\bm{x}})}}$.

  * •

For Energy [28], following Du et al. [12], we first compute the negative free-
energy $E({\bm{x}};f)=\log\sum_{y}e^{f_{y}({\bm{x}})}$, and then attach an
extra linear layer to calculate the ID/OOD logit $g({\bm{x}})=w\cdot
E({\bm{x}};f)+b$, where $w,b$ are learnable scalars. Hence, the sigmoid
function is able to convert the logit $g({\bm{x}})$ into the probability
$\hat{p}(i|{\bm{x}})$.

  * •

For OE [17], similarly, we compute the maximum softmax-probability
$\text{MSP}({\bm{x}};f)=\max_{y}\frac{e^{f_{y}({\bm{x}})}}{\sum_{y^{\prime}}e^{f_{y^{\prime}}({\bm{x}})}}$,
and use another linear layer to obtain the ID/OOD logit
$g({\bm{x}})=w\cdot\text{MSP}({\bm{x}};f)+b$.

  * •

For Mahalanobis distance [21], we maintain an online feature-pool for each ID
class $y$, so as to calculate the Mahalanobis distances for the test samples
as $D_{m}({\bm{x}})$. Then, an extra linear layer is adopted to transform the
distances to ID/OOD logits as $g({\bm{x}})=w\cdot D_{m}({\bm{x}})+b$.

By doing so, one can exploit the unified training-time regularization in Sec.
3.3 to derive a strong OOD detector.

##  Appendix C Additional experimental results

###  C.1 Additional error statistics

In this section, we present the class-aware error statistics for OOD detection
on different benchmarks (see Fig. A1) and different detectors (see Fig. A2).
For each OOD detector on each benchmark, we first compute the OOD scores
$g({\bm{x}})$ for all the ID and OOD test data. Then, a threshold $\lambda$ is
determined to ensure a high fraction of OOD data (_i.e._ , 95%) is correctly
detected as out-of-distribution. Recall that $g({\bm{x}})$ indicates the in-
distribution probability for a given sample (_i.e._ , $P(i|{\bm{x}})$).
Finally, we statistic the distributions of wrongly detected ID/OOD samples.

Specifically, in Fig. A1 and Fig. A2, the first row displays class labels of
ID samples that are wrongly detected as OOD (_i.e._ , $g({\bm{x}})<\lambda$),
and the second row exhibits class predictions of OOD samples that are wrongly
detected as ID (_i.e._ , $g({\bm{x}})>\lambda$). In each subplot, we statistic
the distribution over head, middle, and tail classes (the division rule
follows Wang et al. [47]) for simplicity. Note that the total count of wrong
OOD samples (in the second row) is constant, and a better OOD detector $g$
will receive fewer wrong ID samples (in the first row).

Fig. A1 compares our method with PASCL [47] on CIFAR10/100-LT and ImageNet-LT
benchmarks. The results indicate our method (dashed bar) performs relatively
more balanced OOD detection on all benchmarks. We reduce the error number of
ID samples from tail classes, and simultaneously decrease the error number of
OOD samples that are predicted as head classes. In particular, our method
achieves considerable balanced OOD detection on ImageNet-LT (the right
column), which brings a significant performance improvement, as discussed in
Sec. 4.2.

Fig. A2 compares our integrated version and vanilla OOD detectors (_i.e._ ,
OE, Energy, and BinDisc) on the CIFAR10-LT benchmark. Similarly, the results
indicate our method performs relatively more balanced OOD detection against
all original detectors. The versatility of our ImOOD framework is validated,
as discussed in Sec. 4.3.

However, the statistics in Fig. A1 and Fig. A2 indicate the imbalanced problem
has not been fully solved, since the scarce training samples of tail classes
still affect the data-driven learning process. More data-level re-balancing
techniques may be leveraged to further address this issue.

Figure A1: Class-aware error statistics for OOD detection on different
benchmarks. Figure A2: Class-aware error statistics for different OOD
detectors on CIFAR10-LT.

###  C.2 Detailed results on CIFAR10/100-LT benchmarks

For CIFAR10/100-LT benchmark, Textures [9], SVHN [39], CIFAR100/10
(respectively), TinyImageNet [24], LSUN [54], and Places365 [55] from SC-OOD
dataset [52] are adopted as OOD test sets. The mean results on the six OOD
sets are reported in Sec. 4.

In this section, we reported the detailed measures on those OOD test sets in
Sec. C.2 and Sec. C.2, as the supplementary to Tab. 1. The results indicate
our method consistently outperforms the state-of-the-art PASCL [47] on most of
the subsets.

Table A1: Detailed results on CIFAR10-LT. $\bm{\mathcal{D}_{\text{out}}^{\text{test}}}$ | Method |  AUROC ($\uparrow$) |  AUPR ($\uparrow$) |  FPR95 ($\downarrow$)
---|---|---|---|---
Texture | PASCL |  93.16$\,\pm\,$0.37 |  84.80$\,\pm\,$1.50 |  23.26$\,\pm\,$0.91
Ours |  96.58$\,\pm\,$0.21 |  94.09$\,\pm\,$0.29 |  16.22$\,\pm\,$0.47
SVHN | PASCL |  96.63$\,\pm\,$0.90 |  98.06$\,\pm\,$0.56 |  12.18$\,\pm\,$3.33
Ours |  97.50$\,\pm\,$0.47 |  98.26$\,\pm\,$0.41 |  9.65$\,\pm\,$0.72
CIFAR100 | PASCL |  84.43$\,\pm\,$0.23 |  82.99$\,\pm\,$0.48 |  57.27$\,\pm\,$0.88
Ours |  86.61$\,\pm\,$0.19 |  85.71$\,\pm\,$0.12 |  55.51$\,\pm\,$0.41
TIN | PASCL |  87.14$\,\pm\,$0.18 |  81.54$\,\pm\,$0.38 |  47.69$\,\pm\,$0.59
Ours |  88.75$\,\pm\,$0.35 |  86.27$\,\pm\,$0.39 |  40.52$\,\pm\,$0.32
LSUN | PASCL |  93.17$\,\pm\,$0.15 |  91.76$\,\pm\,$0.53 |  26.40$\,\pm\,$1.00
Ours |  94.55$\,\pm\,$0.23 |  93.70$\,\pm\,$0.34 |  22.02$\,\pm\,$0.37
Places365 | PASCL |  91.43$\,\pm\,$0.17 |  96.28$\,\pm\,$0.14 |  33.40$\,\pm\,$0.88
Ours |  93.57$\,\pm\,$0.14 |  97.04$\,\pm\,$0.21 |  28.43$\,\pm\,$0.61
Average | PASCL |  90.99$\,\pm\,$0.19 |  89.24$\,\pm\,$0.34 |  33.36$\,\pm\,$0.79
Ours |  92.93$\,\pm\,$0.26 |  92.51$\,\pm\,$0.29 |  28.73$\,\pm\,$0.48
Table A2: Detailed results on CIFAR100-LT. $\bm{\mathcal{D}_{\text{out}}^{\text{test}}}$ | Method |  AUROC ($\uparrow$) |  AUPR ($\uparrow$) |  FPR95 ($\downarrow$)
---|---|---|---|---
Texture | PASCL |  76.01$\,\pm\,$0.66 |  58.12$\,\pm\,$1.06 |  67.43$\,\pm\,$1.93
Ours |  77.34$\,\pm\,$0.55 |  62.76$\,\pm\,$0.69 |  68.87$\,\pm\,$0.44
SVHN | PASCL |  80.19$\,\pm\,$2.19 |  88.49$\,\pm\,$1.59 |  53.45$\,\pm\,$3.60
Ours |  84.20$\,\pm\,$0.34 |  91.86$\,\pm\,$0.50 |  47.50$\,\pm\,$0.32
CIFAR10 | PASCL |  62.33$\,\pm\,$0.38 |  57.14$\,\pm\,$0.20 |  79.55$\,\pm\,$0.84
Ours |  61.53$\,\pm\,$0.43 |  56.56$\,\pm\,$0.42 |  79.19$\,\pm\,$0.65
TIN | PASCL |  68.20$\,\pm\,$0.37 |  51.53$\,\pm\,$0.42 |  76.11$\,\pm\,$0.80
Ours |  68.42$\,\pm\,$0.27 |  52.15$\,\pm\,$0.21 |  75.54$\,\pm\,$0.60
LSUN | PASCL |  77.19$\,\pm\,$0.44 |  61.27$\,\pm\,$0.72 |  63.31$\,\pm\,$0.87
Ours |  77.68$\,\pm\,$0.29 |  61.66$\,\pm\,$0.38 |  60.32$\,\pm\,$0.32
Places365 | PASCL |  76.02$\,\pm\,$0.21 |  86.52$\,\pm\,$0.29 |  64.81$\,\pm\,$0.27
Ours |  76.19$\,\pm\,$0.13 |  86.79$\,\pm\,$0.20 |  62.48$\,\pm\,$0.45
Average | PASCL |  73.32$\,\pm\,$0.32 |  67.18$\,\pm\,$0.10 |  67.44$\,\pm\,$0.58
Ours |  74.21$\,\pm\,$0.35 |  68.60$\,\pm\,$0.43 |  65.65$\,\pm\,$0.26

##  Appendix D Resource Requirement

We use 2 NVIDIA V100-32G GPUs in all our experiments.


# Analyzing Cross-Phase Effects of Reactive Power Intervention on Distribution
Voltage Control

Dhaval Dalal, Senior Member, IEEE, Anamitra Pal, Senior Member, IEEE, and Raja
Ayyanar, Fellow, IEEE This work was supported in part by the US Department of
Energy under grant DE-EE0009355. School of Electrical, Computer and Energy
Engineering
Arizona State University, Tempe, AZ, USA
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Increasing photovoltaic (PV) penetration in the distribution system can often
lead to voltage violations. Mitigation of these violations requires reactive
power intervention from PV inverters. However, the unbalanced nature of the
distribution system leads to mixed effects on the voltages of nearby nodes for
each inverter injecting or absorbing reactive power. In particular, reactive
power absorption to reduce over-voltage in one phase can exacerbate over-
voltage in a different phase. In this paper, the factors impacting the
incremental and decremental voltage effects of reactive power intervention are
analyzed in detail. The result of these effects on the distribution system
performance is presented to highlight their significance and the need to
factor them in for any coordinated voltage control algorithm.

###### Index Terms:

Cross-phase effects, Photovoltaic penetration, Reactive power intervention,
Voltage control

##  I Introduction

The road to net zero emissions (NZE) has milestones for achieving
significantly high level of photovoltaic (PV) penetration in the distribution
systems [1]. However, due to limited planning control and operational
visibility from the utility side, the proliferation of PVs can lead to
potential voltage violations which negatively impact the distribution system
quality and reliability [2]. The contributors to voltage violations include a
combination of factors such as power flow reversal, low X/R ratio,
inappropriate settings of step voltage regulators (SVRs) and capacitor banks
(CBs), and sudden drastic changes in PV generation and load profiles [3].

Two major avenues for addressing the voltage violations are: (a) on-load-tap-
changer (OLTC)/SVR/CB control, or (b) reactive power (Q) intervention from PV
inverters. A combination of the two provides a third option. OLTC/SVR/CB
control has severe maintenance cost penalty if implemented to address the
diurnal variations in PV generation. Hence, the focus of current research is
on inverter Q control. Localized voltage control techniques such as inverter
volt-VAr (VV) control require no coordination, but have limited effectiveness
when the PV penetration is high [4]. Communication-based methodologies, in
which the inverters are actively monitored and commanded in a coordinated
fashion using information not locally available to each inverter, are more
effective for PV-rich systems [5, 6]. Since coordinated control on large
distribution systems have significant communication and computation overhead,
a number of recent works have proposed decentralized voltage regulation by
partitioning the distribution networks into zones [7, 8]. However, any
partitioning of the system results in certain parametric effects being
ignored, because they cannot transcend the partition boundaries. Specifically,
the unbalanced nature of distribution systems create significant cross-phase
voltage effects, often leading to new voltage violations when Q intervention
is used in a single-phase PV inverter. Since many of the zoning approaches
rely on phase-based partitions (e.g., [7, 8, 9]), the cross-phase effects are
disregarded, resulting in sub-optimal outcomes.

In this paper, a methodical approach is developed to characterize the cross-
phase voltage effects caused by Q intervention, including their causes,
sensitivities, and end-effects on system performance. The value of considering
these effects is demonstrated by testing on a renewable-rich complex
feeder111A complex feeder is one that has both primary and secondary circuits,
unbalanced multi-phase lines/loads, and OLTCs/SVRs/capacitors. with and
without these effects factored in.

The major contributions of this paper are as follows:

  * •

Identification of cross-phase voltage effects as a major impediment in
achieving desired voltage regulation in distribution networks with high PV
penetration

  * •

Sensitivity analyses to identify and quantify the factors impacting cross-
phase voltage effects

  * •

Proposing and validating a novel methodology to integrate cross-phase voltage
effects into a voltage regulation algorithm to achieve higher hosting capacity
(HC)

##  II Characteristics of Unbalanced Distribution Systems

Distribution systems are significantly different from transmission systems in
terms of power flow balance between phases, line impedances, and/or types of
distributed energy resources (DERs). The role of these factors in maintaining
reliable system operation is well understood for traditional load-dominant
distribution systems or systems with low levels of DER penetration [10].
However, there is a need to investigate the behavior of distribution systems
with a high percentage of DERs that are participating in voltage regulation by
injecting or absorbing significant levels of Q. Following characteristics of
distribution systems merit special attention:

  1. 1.

Low X/R ratios: Smaller conductor sizes in the distribution system lead to
much larger increases in the per unit resistance values
($\propto\frac{1}{d^{2}}$) compared to the increases in (self and mutual)
inductances ($\propto\log\frac{1}{d}$). For instance, X/R ratios in many
distribution systems are in between 2 and 4 as compared to $>$10 for
transmission systems.

  2. 2.

Diversity of line configurations: Distribution systems consist of single-
phase, two-phase, and untransposed three-phase lines serving unbalanced loads.
Therefore, a positive sequence analysis approach is not valid. As stated in
[10], “it is necessary to retain the identity of the self- and mutual
impedance terms of the conductors in addition to taking into account the
ground return path for the unbalanced currents”.

  3. 3.

DER types: The addition of behind-the-meter DERs (typically, PVs) is not
controlled by the utilities. They are likely to add to the phase imbalance
caused by the loads, while also adding high degrees of uncertainty and
variability. Particularly, the PV inverters have varying capability to inject
or absorb Q.

In summary, when voltage mitigation is done in PV-rich systems by providing Q
absorption/injection, the imbalance and low X/R ratios play a major role in
determining the outcome of the Q intervention [11].

##  III Simple Distribution System Illustration

To fully understand the effects of Q intervention in distribution networks, it
is useful to first construct a simple 4-wire, 2-bus system as shown in Fig. 1.
Note that this system is a reduced version of the 4-bus test case available in
OpenDSS, with the transformer removed and the feeder-head line-to-line voltage
changed to 4.16 kV. All other configuration details including wire data, line
geometry, and load were retained from the original case. The system has a
balanced 3-phase load (5400 kW, 0.9 power factor (PF)), representing the
$0^{th}$ instance. A primitive impedance matrix,
$Z_{\mathrm{prim}}\in\mathbb{R}^{4\times 4}$, consisting of $Z_{\mathrm{mut}}$
and $Z_{\mathrm{earth}}$ components was constructed from the available data
using modified Carson’s equations. With the given parameters, the X/R ratio is
about 3:1. The power flow equations at instance $i$ are given by:

$\mathbf{S_{i}}=\mathbf{P_{i}}+j\mathbf{Q_{i}}$ (1)
$\mathbf{I_{L_{i}}}=\Bigg{(}\frac{\mathbf{{S_{i}}}}{\mathbf{V_{{N4}_{i}}}}\Bigg{)}^{*}$
(2) $\mathbf{V_{{N4}_{i}}}=\mathbf{V_{S}}-\mathbf{I_{L_{i}}}\times
Z_{\mathrm{prim}}$ (3)

where, $\mathbf{S_{i}}$, $\mathbf{P_{i}}$, and $\mathbf{Q_{i}}$ are the load
apparent, real and reactive power vectors, $\mathbf{I_{L_{i}}}$ is the complex
vector representing the line current, ∗ in the superscript denotes the
conjugate operation, $\mathbf{V_{S}}$ is the base voltage vector (a constant),
and $\mathbf{V_{{N4}_{i}}}$ is the voltage vector at the load node. It was
observed that despite a balanced load, the $\mathbf{V_{{N4}_{0}}}$ values
exhibited imbalance, which is attributable to the mutual impedance mismatches
related to wire location differences. The results obtained by solving (1)-(3)
numerically were found to be consistent with those obtained from the OpenDSS
solution of this system. The phase voltage magnitudes are shown in the first
row of Table I.

Figure 1: Simple 4-wire, 2-bus system

Next, a perturbation of 100 kVAr ($Q_{1_{a}}=Q_{0_{a}}+100$ kVAR) is applied
to phase A to test the effects of PV inverter Q absorption. Solutions to
(1)-(3) were again found to be consistent with the OpenDSS results. The phase
voltage magnitudes are provided in the second row of Table I. The voltage
difference between the two cases is shown in the third row of the table. Since
the objective of Q absorption is to reduce the voltage, the results for phase
A are consistent with that objective. However, it is observed that both phase
B and phase C have voltage differences of opposite polarity.

TABLE I: N4 phase voltages (in Volts) Case | Phase A | Phase B | Phase C
---|---|---|---
Base (i=0) | 2103 | 2206 | 2150
+100 kVAr (i=1) | 2068 | 2217 | 2160
Difference | -35.3 | 11.6 | 10.0
Difference due to $Z_{\mathrm{earth}}$ | 19.2 $\angle 180\degree$ | 19.2 $\angle 180\degree$ | 19.2 $\angle 180\degree$
Difference due to $Z_{\mathrm{mut}}$ | 16.4 $\angle 165\degree$ | 2.1 $\angle-57\degree$ | 3.4 $\angle 57\degree$

Before providing contextual analysis of these results, it is instructive to
find the contributions to the voltage differences from $Z_{\mathrm{earth}}$
($\mathbf{\Delta V_{{N4}_{e}}}$) and $Z_{\mathrm{mut}}$ ($\mathbf{\Delta
V_{{N4}_{m}}}$), respectively. Note that this breakdown is not available from
the OpenDSS power flow results, but can be obtained by solving the following
equations that decompose the contributions. The dominant component of
$\mathbf{\Delta I_{L}}$ is the imaginary component of phase A. It leads to the
computed values of $\mathbf{\Delta V_{{N4}_{e}}}$ and $\mathbf{\Delta
V_{{N4}_{m}}}$ from (5) and (6), respectively, which are shown in the last two
rows of Table I.

$\mathbf{\Delta I_{L}}=\mathbf{I_{L_{1}}}-\mathbf{I_{L_{0}}}$ (4)
$\mathbf{\Delta V_{{N4}_{e}}}=\mathbf{\Delta I_{L}}\times Z_{\mathrm{earth}}$
(5) $\mathbf{\Delta V_{{N4}_{m}}}=\mathbf{\Delta I_{L}}\times
Z_{\mathrm{mut}}$ (6)

The results in Table I provide significant insights into the incremental and
decremental effects of changes in reactive power ($\Delta Q$). The positive
voltage deviations in phases B and C as a result of Q absorption in phase A
show unintended consequences of a corrective action (i.e., voltage mitigation
for phase A) that must be recognized. Additionally, the decomposition of the
contributions from $Z_{\mathrm{mut}}$ and $Z_{\mathrm{earth}}$ show that the
corrective effect for phase A has nearly equal and significant contributions
from both factors, while the phase B and phase C effects are dominated by the
$Z_{\mathrm{earth}}$ factor (compare the entries in the last two rows of Table
I for phases B and C). To understand why the earth current has such a large
opposite-polarity effect on the other phases, we look at the phasor diagrams
of the voltages in Fig. 2. Individual phasors are color coded and their values
are shown next to the diagrams for clarity.

(a) Phase A

(b) Phase B

(c) Phase C

Figure 2: Phasor diagrams showing effects of Q perturbation

For phase A, the contributions from $Z_{\mathrm{mut}}$ and
$Z_{\mathrm{earth}}$ have similar phase and magnitudes, and the combination of
the two results in phase A voltage reduction (which is expected). Since the
earth current is the same, the $\mathbf{\Delta V_{{N4}_{e}}}$ is identical for
all three phases, as depicted by the vectors in black color and values in the
phasor diagrams. Moreover, since the perturbation was made in phase A, the
effects of $Z_{\mathrm{mut}}$ on phases B and C voltages are minimal as shown
by the red-colored vectors and values. As a result, the composite effect of
$\mathbf{\Delta V_{N4}}$ for phases B and C is dominated by the
$\mathbf{\Delta V_{{N4}_{e}}}$ component, which is 19.2 V at -180°. As seen in
Figs. 2(b) and 2(c), the phasor addition of $\mathbf{\Delta V_{N4}}$ and
$\mathbf{V_{{N4}_{0}}}$ for phases B and C result in higher magnitudes of
$\mathbf{V_{{N4}_{1}}}$ with a very small change in angle.

While these results allow a good analytical basis for understanding the cross-
phase effects and their causes, they need to be put into proper context.
Firstly, the reported voltage deviations are fairly small in p.u. terms
(-0.015 p.u. for phase A, and about 0.005 p.u. for phases B and C). Secondly,
a real distribution system rarely has the 4-wire line configuration shown in
Fig. 1 with a single-phase PV on one of the lines. Hence, the question to
address is whether actual distribution systems need to contend with both
incremental and decremental effects when using Q intervention from DERs as a
voltage control technique. This is analyzed in the next section.

##  IV Evaluation on a Complex Distribution System

For this purpose, we evaluate the J1 Feeder [12], which represents a
distribution system in the U.S. Northeast, with open-source data and models
provided by EPRI. Key attributes of the feeder are summarized in Table II. In
addition, J1 has a diverse mix of underground and overhead lines with
different impedance characteristics and single-phase loads/transformers
ranging from 0.3 kW/5 kVA to 20 kW/75 kVA. With this level of complexity, the
phasor-based approach used in the previous section may not yield sufficient
insights. Instead, the results from OpenDSS are used to perform the analysis.
Since there is no time series data available for the J1 feeder, data from
Pecan Street [13] were applied to the loads and PVs in the J1 feeder to
perform the analysis across different time instants.

TABLE II: Attributes of EPRI J1 feeder

Parameter | Value | Comments
---|---|---
Primary Voltage | 12.47 kV | Substation transformer at 69 kV
Secondary Voltage | 240 V |
Total Customers | 1384 | 363/375/643/3 (Phase A/B/C/3-ph)
Total Nodes | 4245 | 2037 Primary/2208 Secondary
Total Load | 10.95 MW | Includes 5 MW aggregated load
Transformers | 819 | 218/225/372/4 (Phase A/B/C/3-ph)
SVRs | 9 | 3/3/2/1 (Phase A/B/C/Substation)
Capacitors | 3900 kVAR | 5 Capacitors
Existing PV | 1813.6 kW | 1.71 MW commercial, 103.6 kW residential

###  IV-A Effect of Reactive Power (Q) Intervention from PV Nodes

To accomplish the objective of increasing PV penetration in the J1 feeder
without incurring voltage violations under extreme scenarios, one extreme use
case (11 AM, August) was selected. For this use case, a voltage-reactive power
sensitivity matrix (VQ-SM) was constructed, with its elements given by (7),
where $\Delta Q_{j}$ is the reactive power perturbation applied to PV inverter
$j$, and $v_{i}^{0}$ and $v_{i,j}^{Q}$ are the voltages at node $i$ prior to
the perturbation and after the perturbation, respectively.

$sm_{i,j}=\frac{v_{i,j}^{Q}-v_{i}^{0}}{\Delta Q_{j}}$ (7)

Since the VQ-SM is large (rows corresponding to number of PVs and columns
corresponding to number of nodes in the system) we can look at one column
which depicts the effect on a single system node from all the PVs where
$\Delta Q$ can be applied. It was observed that the secondary node of a
single-phase SVR in phase B, B19007reg.2, exhibited highest voltage on a
consistent basis under moderate-to-high PV penetration. This SVR is located
downstream from the feeder-head and has a step-up ratio to restore the voltage
levels under high load conditions. At 11 AM, when PV generation is high with
moderate load conditions, the voltage on the primary side of the SVR goes up
(but does not have violations). However, due to the step-up ratio, the
secondary node and other nodes close to the SVR (including some customer
sites) experience voltage violations. The method to address these violations
is to identify PV nodes that have the highest impact on this SVR secondary
node and activate them for Q intervention. Fig. 3 depicts the sensitivity of
all the PV nodes in the system to the selected node (B19007reg.2).

Figure 3: Sensitivity of a single SVR node to all PVs

The following observations are made from Fig. 3:

  * •

Sensitivity values for phase B absorption are negative as the node under
consideration is a phase B node. Both phase A and phase C sensitivities are
positive - a result consistent with the explanation in Section III.

  * •

The sensitivity magnitudes generally decrease as the distance from the node
increases. However, there are clear exceptions indicated by the choppiness of
the plots. These can be attributed to the fact that the physical distances do
not always equate to the electrical distances (impedances). Additional factors
such as the presence of CBs and SVRs, and location relative to the substation
(upstream or downstream) also contribute to these incongruous results.

  * •

If only the Q interventions of phase B are considered to address the voltage
violations, it may not be sufficient. This is because many of the nearby PVs
in phases A and C may be activated to address voltage violations in the nodes
located on those phases, and they could trigger decremental effects on the
selected SVR node. Hence, one must consider all phase effects together to
achieve the desired results.

The inferences drawn from these observations are used to implement a
prioritized voltage control strategy that is explained in the next sub-
section.

###  IV-B Prioritized Q Intervention for Effective Voltage Control

Algorithm 1 was devised to add PVs into a distribution system while accounting
for the cross-phase effects explained previously. It was also used to validate
the effectiveness of a new voltage control technique (see Section V).

Algorithm 1 PV addition/Voltage control algorithm

1: Load network details into a distribution system solver

2: Add $n$ PV system, where $n=1$

3: Apply appropriate PV and load profiles

4: Select evaluation condition (time instance)

5: Run power flow with unity PF (UPF) and identify locations and values of
voltage violations by phase

6: Identify PVs with highest sensitivity to the UPF violation nodes and
incrementally add $\Delta Q$ on them till zero violations are achieved in
power flows

7: Repeat Steps 3 to 6 with $n=n+1$ PVs

The results of running Algorithm 1 on EPRI J1 feeder are depicted in Fig. 4.
These results clearly identify the decremental effects of Q intervention. As
shown in the first two rows, for 90 PV additions, all violations are in phase
B (68 of them), and they were addressed (Step 6) by activating Q intervention
in a prioritized list of 15 PVs in phase B, guided by the VQ-SM. When 10 more
PVs were added (Step 7a), there were more phase B violations (121 of them),
but no phase A or phase C violations under UPF. However, when the Q
intervention from Step 6 was activated (Step 7b), there were two new
violations in phase A. The dominant contributor to the new violations in phase
A was the decremental effect from Q intervention on the 15 phase B PVs. This
was further confirmed by reducing the Q intervention in phase B, which removed
the phase A violation, but reintroduced the phase B violations. The final
solution, as shown in the last row (Step 7c), was to add Q intervention at one
phase A PV.

Figure 4: Cross-phase sensitivity effects using Algorithm 1

##  V Impact on Voltage Control Strategies

The simple example in Section IV-B clearly showed the decremental effect of
cross-phase sensitivity. Looking at the larger picture, it also points to the
challenges in achieving effective coordinated voltage control with Q
intervention in real-world, large, complex distribution systems. For example,
with any partition that is phase-based or distance-based, there will be a
significant loss of accuracy that can lead to sub-optimal results from a
voltage control perspective.

In order to demonstrate this impact, a large number (460) of residential PVs
were added in a random fashion to the EPRI J1 feeder, taking the total PV
capacity to 6.4 MW. With the inverters set at UPF (i.e., without any Q
intervention), there were many voltage violations for different hours of the
day with the maximum voltage exceeding 1.1 p.u. during some hours as shown in
the left part of Fig. 5.

Next, the network is partitioned into three zones, one for each phase. For
each phase, a VQ-SM is created using only the PVs and nodes in that phase, and
three optimization algorithms are run independently of each other for the
three phases. The goal of each of the optimization algorithms was to find the
minimum Q intervention in a given phase to minimize the voltage violations of
that phase. The results from the three algorithms are combined and applied to
the distribution system solver (OpenDSS) to check for voltage violations. The
results in the middle part of Fig. 5 clearly depict the relative
ineffectiveness of the phase-based partitioning. By omitting the cross-phase
sensitivity from the optimization formulation, the optimizer is misled into
providing Q intervention solutions that may work for one phase, but not for
the other phases.

Finally, an optimization-based iterative voltage control algorithm was
developed to mitigate the violations, while also accounting for the cross-
phase sensitivity effects. This algorithm relied on availability of system-
wide voltage information and iterative refinement of the VQ-SM for its
effectiveness. More details about this algorithm can be found in [14]. As
shown in the right part of Fig. 5, this algorithm is successful in removing
voltage violations at all hours, which is impressive considering the extent of
violations in the UPF case in terms of numbers (3000+) and voltage magnitudes
(1.1045 p.u.).

Figure 5: Cross-phase sensitivity effects with high PV penetration

##  VI Conclusion

This paper presented a robust argument for considering the cross-phase effects
of Q intervention in distribution systems to achieve better voltage control
and higher levels of PV penetration. The underlying reasons for the cross-
phase sensitivity were identified first. Then, a simple network model was
created and phasor diagrams were employed to analytically depict how the
cross-phase effects manifest in the system.

Afterwards, successive illustrations on a complex real-world distribution
feeder underlined how the cross-phase effects negatively impact voltage
control algorithms if they are not fully considered within the control
strategy. Finally, an example was provided that demonstrated how the
incorporation of these effects in the control algorithms lead to significantly
better results.

## References

  * [1] SolarPower Europe, “Global market outlook for solar power 2023-2027.” [Online]. Available: {https://www.solarpowereurope.org/insights/outlooks/global-market-outlook-for-solar-power-2023-2027/detail#global-solar-market-update-2000-2022}
  * [2] T. S. Ustun, J. Hashimoto, and K. Otani, “Impact of smart inverters on feeder hosting capacity of distribution networks,” _IEEE Access_ , vol. 7, pp. 163 526–163 536, 2019.
  * [3] E. Demirok, P. C. González, K. H. B. Frederiksen, D. Sera, P. Rodriguez, and R. Teodorescu, “Local reactive power control methods for overvoltage prevention of distributed solar inverters in low-voltage grids,” _IEEE Journal of Photovoltaics_ , vol. 1, no. 2, pp. 174–182, 2011.
  * [4] F. Ding and B. Mather, “On distributed PV hosting capacity estimation, sensitivity study, and improvement,” _IEEE Transactions on Sustainable Energy_ , vol. 8, no. 3, pp. 1010–1020, 2017.
  * [5] P. Lusis, L. L. H. Andrew, A. Liebman, and G. Tack, “The added value of coordinating inverter control,” _IEEE Transactions on Smart Grid_ , vol. 12, no. 2, pp. 1238–1248, 2021.
  * [6] Y. Yao, F. Ding, K. Horowitz, and A. Jain, “Coordinated inverter control to increase dynamic PV hosting capacity: A real-time optimal power flow approach,” _IEEE Systems Journal_ , vol. 16, no. 2, pp. 1933–1944, 2022.
  * [7] P. Li, C. Zhang, Z. Wu, Y. Xu, M. Hu, and Z. Dong, “Distributed adaptive robust voltage/var control with network partition in active distribution networks,” _IEEE Transactions on Smart Grid_ , vol. 11, no. 3, pp. 2245–2256, 2020.
  * [8] B. Zhao, Z. Xu, C. Xu, C. Wang, and F. Lin, “Network partition-based zonal voltage control for distribution networks with distributed PV systems,” _IEEE Transactions on Smart Grid_ , vol. 9, no. 5, pp. 4087–4098, 2018.
  * [9] A. Alrushoud, C. McEntee, and N. Lu, “A zonal Volt/VAr control mechanism for high PV penetration distribution systems,” in _2021 IEEE Power and Energy Society General Meeting (PESGM)_ , 2021, pp. 1–5.
  * [10] W. Kersting, _Distribution System Modeling and Analysis_. Taylor & Francis, CRC Press, 2017. [Online]. Available: https://books.google.com/books?id=TnJIvwEACAAJ
  * [11] T. Niknam, “A new HBMO algorithm for multiobjective daily Volt/VAr control in distribution systems considering distributed generators,” _Applied energy_ , vol. 88, no. 3, pp. 778–788, 2011.
  * [12] Electric Power Research Institute, “EPRI Test Circuits J1 Feeder OpenDSS Code.” [Online]. Available: {https://sourceforge.net/p/electricdss/code/HEAD/tree/trunk/Distrib/EPRITestCircuits/epr/_dpv/J1/}
  * [13] Pecan Street Research Institute, “The Dataport Database.” [Online]. Available: https://pecanstreet.org/dataport/
  * [14] D. Dalal, M. Sondharangalla, R. Ayyanar, and A. Pal, “Improving Photovoltaic Hosting Capacity of Distribution Networks with Coordinated Inverter Control – A Case Study of the EPRI J1 Feeder,” 2023. [Online]. Available: https://arxiv.org/abs/2311.02793