# Attention Guided Global Enhancement and Local Refinement Network for
Semantic Segmentation

Jiangyun Li, Sen Zha, Chen Chen, Meng Ding, Tianxiang Zhang, and Hong Yu J.
Li, S. Zha, T. Zhang and H. Yu are with the School of Automation and
Electrical Engineering, University of Science and Technology Beijing, Beijing
100083, China (e-mail<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>g20198754@xs.ustb.edu.cn)C. Chen is with the Center for
Research in Computer Vision, University of Central Florida, Orlando, FL 32816
USA (e-mail: chen.chen@crcv.ucf.edu)M. Ding is with the Scoop Medical,
Houston, TX 77007 USA (e-mail<EMAIL_ADDRESS>
###### Abstract

The encoder-decoder architecture is widely used as a lightweight semantic
segmentation network. However, it struggles with a limited performance
compared to a well-designed Dilated-FCN model for two major problems. First,
commonly used upsampling methods in the decoder such as interpolation and
deconvolution suffer from a local receptive field, unable to encode global
contexts. Second, low-level features may bring noises to the network decoder
through skip connections for the inadequacy of semantic concepts in early
encoder layers. To tackle these challenges, a Global Enhancement Method is
proposed to aggregate global information from high-level feature maps and
adaptively distribute them to different decoder layers, alleviating the
shortage of global contexts in the upsampling process. Besides, a Local
Refinement Module is developed by utilizing the decoder features as the
semantic guidance to refine the noisy encoder features before the fusion of
these two (the decoder features and the encoder features). Then, the two
methods are integrated into a Context Fusion Block, and based on that, a novel
Attention guided Global enhancement and Local refinement Network (AGLN) is
elaborately designed. Extensive experiments on PASCAL Context, ADE20K, and
PASCAL VOC 2012 datasets have demonstrated the effectiveness of the proposed
approach. In particular, with a vanilla ResNet-101 backbone, AGLN achieves the
state-of-the-art result ($56.23\%$ mean IOU) on the PASCAL Context dataset.
The code is available at https://github.com/zhasen1996/AGLN.

###### Index Terms:

Semantic segmentation, encoder-decoder, global enhancement, local refinement,
context fusion.

##  I Introduction

Semantic segmentation is a fundamental problem in computer vision, aiming to
segment an image into different regions by assigning a predefined class label
to each pixel in the image based on their respective semantic categories. It
plays a crucial role in various practical applications, such as autonomous
driving [1], computational photography [2], and human-machine interaction [3].

Most of the recent approaches for semantic segmentation are based on Fully
Convolutional Network (FCN) [4]. However, the consecutive downsampling
operations, such as pooling and strided convolution, lead to a significant
decrease in the spatial details of the original input image. To alleviate this
problem, most state-of-the-art methods [5, 6, 7, 8, 9, 10, 11, 12, 13] discard
some of the downsampling operations to maintain relatively high-resolution
feature maps and exploit dilated convolutions to enlarge the receptive field,
termed Dilated-FCN-based models. Instead of that, another approach carefully
designs a decoder on the top of a classification network (i.e. encoder) to
gradually reconstruct a high-resolution segmentation map, forming a “U-shape”
Encoder-Decoder architecture [14, 15, 16]. Benefiting from the low-resolution
intermediate feature maps, the encoder-decoder networks usually require much
lower computation and memory costs compared to the widely used Dilated-FCN
models. However, there are two major limitations impairing the performance of
the encoder-decoder models.

Firstly, existing decoders mainly utilize bilinear interpolations or
deconvolutions to upsample the high-level/low-res feature maps to match the
pixel-wise supervision. However, research studies [17, 18] revealed that these
upsampling methods have limited capability of recovering the pixel-wise
prediction accurately. Specifically, both interpolation and deconvolution only
consider the limited neighborhood and are unable to capture long-range
contextual information, meaning that the feature representation at each
location of the upsampled feature maps is recovered from a limited receptive
field. As a result, the absence of global contexts in the upsampling process
leads to common misclassification on large-scale objects. As shown in the FPN
result (first row of Fig. 1(d)), an evident mistake (green color) occurs at
the top left corner of the water (red color), and this type of error should be
avoided. In comparison, our proposed AGLN generates an outstanding result
shown in the first row of Fig. 1(c).

(a) Image

(b) GT

(c) Ours

(d) FPN

Figure 1: Visualization examples of the segmentation results on PASCAL Context
dataset [19]. From left to right: (a) Input image, (b) Ground truth, (c) Ours,
(d) FPN [20].

In addition, low-level features from encoder layers are passed to the decoder
through skip connections. However, these encoder feature maps need further
refinement since they are unable to provide enough fine-grained details to
achieve accurate segmentation boundaries and recognition of small objects [21,
22]. Concretely, early encoder convolutions (i.e. sliding windows) traverse
all local areas of the image to capture local contexts due to the limited
receptive fields. These contexts may be critical for the formulation of
semantic concepts in later convolutional layers. However, the massive and
confusing local representations (i.e. noises) are not conducive to the
recognition of semantic boundaries. Some encoder-decoder networks combine low-
and high-level features by simple addition or concatenation operations [14,
23, 20], drowning critical details in considerable noises, and further
resulting in confusion between different objects as well as ignorance of small
targets. As shown in the second row of Fig. 1(d), FPN cannot recognize the
accurate edge of the bird claws since the fine-grained details are lost in the
deep network architecture.

The aforementioned two drawbacks make the encoder-decoder models fail to take
full advantage of the global semantic information from high-level features and
are unable to obtain effective spatial details. Consequently, these problems
lead to unacceptable segmentation results, including the misclassification of
large-scale objects, rough segmentation boundaries, and ignorance of small
targets. To address these challenges, a framework called Attention guided
Global enhancement and Local refinement Network (AGLN) is proposed in this
paper. Specifically, in terms of the inadequate global information in the
feature upsampling process, a Global Enhancement Method is introduced to first
selectively aggregate global contexts from high-level features and then
adaptively distribute them to the upsampled feature maps. In this way, global
category clues are aggregated and delivered to each stage of the decoder,
making up for the deficiency of global contexts in the upsampling process and
generating the global enhanced decoder feature maps. In addition, utilizing
the output of Global Enhancement Method as semantic guidance, a Local
Refinement Module is designed to carefully refine low-level features from the
encoder, filtering out the noises and generating more informative local
details. After that, these two methods are incorporated into the FPN baseline.
The resulting AGLN can provide global enhanced decoder features and local
refined spatial details, achieving more effective context fusions in different
decoder layers. Consequently, Our AGLN improves the performance of baseline
and achieves competitive results on popular benchmarks, including the PASCAL
Context [19], ADE20K [24], and PASCAL VOC 2012 [25] datasets.

Our main contributions can be summarized as follows:

1) We propose a Global Enhancement Method to aggregate global contexts from
high-level features and deliver them to different decoder layers, compensating
for the absence of global information in the upsampling process.

2) We introduce a Local Refinement Module, providing semantic guidance to
refine the encoder features, filtering out the noises before the low- and
high-level context fusion.

3) Our proposed AGLN incorporates the aforementioned two methods into the FPN
baseline. Extensive experiments show that AGLN achieves state-of-the-art
results on PASCAL Context (56.23$\%$ mIOU) and competitive results on ADE20K
(45.38$\%$ mIOU) and PASCAL VOC 2012 (84.9$\%$ mIOU).

##  II Related Work

In this section, we introduce some popular encoder-decoder models for semantic
segmentation and related works on the global context, local context, relation
modeling and attention mechanism contributing to the improvements of
segmentation performance.

###  II-A Encoder-Decoder Models

Based on a U-shape encoder-decoder architecture, different approaches have
been explored to improve the segmentation performance. For example, U-Net [14]
and Seg-Net [26] employ skip connections on a symmetrical structure to enable
the fusion of information from the encoder and decoder. To further exploit the
context information along the downsampling process, RefineNet [16] carefully
designs a generic multi-path refinement decoder, fusing coarse high-level
semantic features with fine-grained low-level features to generate high-
resolution semantic feature maps. In addition, the Feature Pyramid Network
(FPN) [20] builds a high-level semantic feature pyramid as the decoder to
solve the multi-scale object detection problem, guiding the semantic
segmentation models at the same time. To summarize, the aforementioned works
aim at an effective strategy to utilize different levels of contexts and
explore the potential of the encoder-decoder network. For similar purposes,
based on the FPN, our AGLN is focused on the enhancement of global contexts
and the refinement of local spatial details.

###  II-B Global Context

Global context encoding is effective in improving the categorization of large-
scale areas and has been well-explored during these years. For instance,
PSPNet [8] utilizes the global average pooling to collect global contexts and
different sub-region representations. EncNet [9] introduces a Context Encoding
Module to capture the global semantic contexts and predict scaling factors to
selectively highlight class-dependent feature maps. ACNet [27] adopts a global
average pooling following a convolution layer to generate the global feature
and fuse it with each pixel by an adaptive coefficient measuring the
similarity between the global feature and per-pixel feature. OCR [13] presents
an object-contextual representation approach to aggregate the contexts of all
the locations weighted by their degrees belonging to a certain object region.
Although these methods improve the performances on different segmentation
benchmarks, they completely ignore the inadequacy of global contexts in the
upsampling operations. Instead, our Global Enhancement Method is designed to
improve the global contexts during the upsampling process.

###  II-C Local Context

Local contexts or spatial details are essential to the recognition of small
objects and discrimination of segmentation boundaries. Thus, different models
have been proposed to exploit the low-level features from the shallow layers.
For example, U-Net-based methods [14, 23, 20] often adopt local contexts from
low- and middle-level visual features by simple addition or concatenation
operations. However, this simple combination of low- and high-level feature
maps is less effective because of the gap in semantic levels and spatial
resolution between them. Hence, several researches attempt to apply the local
contexts in a more effective way. ExFuse [21] finds that introducing semantic
information into low-level features and high-resolution details into high-
level features is more effective for the later fusion. RefineNet [16] utilizes
fine-grained features from earlier convolutions to directly refine the deeper
layers that capture high-level semantic features through long-range residual
connections. ACNet [27] attaches more local contexts to the locations
presenting lower similarities with the global feature and reuses the gated
local features multiple times. In addition to the gap between different levels
of features, noises from the encoder feature also impair the context fusion.
Although some of the previous works [21, 28] point out the problem of noisy
low-level features, few of them give a detailed solution to deal with it. In
this paper, we propose to refine the noisy encoder features in both spatial
and channel dimensions.

###  II-D Relation Modeling and Attention Mechanism

The self-attention mechanism has been demonstrated to be capable of modeling
relationships between different locations by calculating their similarities
with each other. For instance, Non-local Network [29] and its variants [10,
28, 30, 12] have demonstrated the significance of the self-attention mechanism
in computer vision tasks. Besides, some other models measure feature
correlations by learned attention weights. For example, SENet [31] utilizes
the global pooling feature followed by fully connected layers to generate
channel weights, selectively highlighting some of the channel maps. CBAM [32]
further explores the squeeze-and-excitation module and expands it to the
spatial dimension. CPNet [33] directly supervises the learning of attention
weights by developing a Context Prior with the supervision of the Affinity
Loss. Similar implementation of the learned attention includes EncNet [9],
PSANet [34], GANet [35], and APCNet [36], etc. Given the demand for global
feature extraction and channel relation modeling, the attention mechanism is
also well-exploited in our AGLN. Firstly, a learned global attention pooling
is adopted in the Global Enhancement Method to adaptively aggregate semantic
information in the global spatial space. Additionally, channel attention is
employed to model the similarity between the low-level spatial contexts and
the enhanced decoder feature, resampling the encoder feature maps in channel
dimension.

##  III Proposed Approach

In this section, we first illustrate the major shortcomings of the encoder-
decoder segmentation models and then propose two corresponding solutions to
overcome these shortages. Specifically, on one hand, a Global Enhancement
Method consisting of a Semantic Aggregation Block (SAB) and a Semantic
Distribution Module (SDM) is proposed to provide global features inadequate in
upsampling process. On the other hand, a Local Refinement Module (LRM) is
designed to filter out the noises from the low-level encoder features and
generate more informative local details. Based on the proposed methods, a
novel AGLN is built to enhance global contexts and refine local textures
before the feature fusion in the decoder, leading to improved performance
compared to the original encoder-decoder baseline.

Figure 2: The overall architecture and design details of the proposed AGLN.
(a) Details of the proposed Semantic Aggregation Block and Context Fusion
Block (consisting of Semantic Distribution Module and Local Refinement
Module), (b) Overall architecture of AGLN (best viewed in color).

###  III-A Global Enhancement for Decoder Features

####  III-A1 Absence of Global Features in Upsampling Process

In an encoder-decoder architecture, the deeper layers of encoder contain more
high-level semantic information. However, upsampling methods such as nearest
interpolation and linear interpolation are incapable of utilizing the global
contexts by their hand-crafted kernels in a sub-pixel scope. In addition to
the interpolations, another preferred upsampling operator deconvolution also
suffers from a limited receptive field (i.e. $3\times 3$ kernel size).
Meanwhile, the deconvolution applies the same kernel across the entire spatial
space, despite the various instance-specific contents from different spatial
regions. The aforementioned limitations become the obstacle to the accurate
recovery of the pixel-level prediction.

In this work, we aim to break the limits of current upsampling methods and
seek an approach to improve the reconstruction of high-resolution segmentation
maps. For this purpose, an ideal enhancement method for the upsampling process
should be capable of (1) aggregating contextual information in a global space,
(2) adaptively updating the representation for each specific location of
feature maps, and (3) maintaining computation efficiency. To this end, the
Global Enhancement Method is proposed.

####  III-A2 Global Enhancement

In our method, the original upsampling operator of the baseline (i.e. bilinear
interpolation) is kept. Moreover, we propose to aggregate global features from
low-resolution/high-level features and attach them to upsampled decoder
feature maps, enabling the restoration of global contexts during the
upsampling process.

Semantic Aggregation Block: Firstly, a Semantic Aggregation Block is
introduced to aggregate global features. As shown in the green box of Fig. 2,
$X\in\mathbb{R}^{C\times H\times W}$ denotes the output of the last encoder
layer. A global attention pooling is applied on $X$ to selectively gather
visual primitives from the entire space and generate $N$ semantic descriptors
$D\in\mathbb{R}^{C\times N}$, formulated as:

$D=G_{gather}(X)=X_{feat}\times X_{am}^{\mathsf{T}},$ (1)

where $G_{gather}(*)$ is the global attention pooling, and “$\times$” denotes
the matrix multiplication. $X_{feat}=\phi(X;W_{\phi})\in\mathbb{R}^{C\times
HW}$ and $X_{am}=softmax(\theta(X;W_{\theta}))\in\mathbb{R}^{N\times HW}$ are
outputs of different $1\times 1$ convolutional layers ($\phi$ and $\theta$),
transforming the input $X$. Specifically, the softmax layer is performed on
the spatial dimension ($HW$) to obtain the attention maps $X_{am}$.

Due to the diverse representations from different channels ($N$) of the
attention map ($X_{am}$), each semantic descriptor is a specific weighted
aggregation of global features from the entire spatial space and therefore
implicitly encodes a certain semantic region of the input feature map.

Semantic Distribution Module: To compensate for the inadequacy of global
semantic clues in repeated upsampling operators, Semantic Distribution Modules
are employed to the upsampling process of the encoder-decoder baseline. In the
yellow box of Fig. 2, $A\in\mathbb{R}^{C\times kH\times kW}(k=2,4,8)$ denotes
the decoder feature map gradually upsampled from the lowest-resolution feature
map $X$. Given the input feature A, the Semantic Distribution Module first
applies a $1\times 1$ convolution and a softmax layer in channel dimension,
transforming the unique representations from all positions to $kH\times kW$
$N$-dimensional attention vectors. Then, $N$ semantic descriptors are
adaptively integrated to each position according to the attention vectors,
generating the semantic descriptor map $M\in\mathbb{R}^{C\times kH\times kW}$.

$M=F_{distr}(D,A)=D\times A_{av},$ (2)

where $F_{distr}(*)$ denotes the semantic distribution function, while
$A_{av}=softmax(\varphi(A;W_{\varphi}))\in\mathbb{R}^{N\times kH\times kW}$
are $kH\times kW$ $N$-dimensional attention vectors obtained by channel-wise
softmax on the convolved input feature map $A$.

Next, the semantic descriptor map $M$ is fused with the original decoder
feature $A$ by an addition operation followed with convolution, BN, and ReLU
layer, generating the output, enhanced decoder feature
$E\in\mathbb{R}^{C\times kH\times kW}$.

$E=\Psi(A+\alpha M;W_{\Psi}).$ (3)

Note that we multiply $M$ with a learnable scaler parameter $\alpha$ for
flexibility, and $\Psi(*;W_{\Psi})$ stands for the convolution together with
BN and ReLU layer.

####  III-A3 Discussion

Here we provide an in-depth discussion about our Global Enhancement Method.

In the global attention pooling of the Semantic Aggregation Block, the
representation of the attention map $X_{am}$ determines the information that a
semantic descriptor implicitly encodes. For example, if an attention map has
higher activation values on foreground objects (e.g., person, car, and
airplane), the semantic descriptor will capture more representations from
corresponding foreground objects. By setting the number of semantic
descriptors $N$ to a relatively small value (e.g., 64, 128, 256), various
global semantic features from the entire spatial space are efficiently
aggregated into a set of compact semantic descriptors (The number of semantic
descriptors $N$ will be further discussed by an ablation study in the
experiment section). After that, Eq. 2 of the Semantic Distribution Module
formulates the soft attention for feature selection. Hence, the representation
of each pixel from the semantic descriptor map $M$ is the adaptive aggregation
of the various semantic descriptors. Fusing the semantic descriptor map $M$
with $A$ helps to provide the content-aware global context prior to original
decoder features, greatly facilitating the pixel-level classification. Further
investigation into the mechanics of the Global Enhancement Method will be
elaborated in the experiment Sec. IV-E.

Furthermore, our Global Enhancement Method is computationally efficient with a
complexity of $O(CNHW)$. More specifically, it costs only $CN/2^{18}$ MB
memory to store the intermediate result semantic descriptors, meaning saving
128 semantic descriptors for an input data array with 256 channels
($N=128,C=256$, implemented in our method) costs only 0.125 MB memory.

####  III-A4 Comparison

We compare our Global Enhancement Method with other popular global context
encoding methods. The self-attention mechanism is applied to capture long-
range dependency since its first success in video classification [29], and its
potential has been well-explored during recent years [10, 30, 37]. However,
the high computational complexity (quadratic of $HW$) of the self-attention
mechanism is still unaffordable in most cases. In comparison, our Global
Enhancement Method is more efficient for the linear complexity of $HW$
($N<<HW$). In the process of gathering global information, a great many
methods, such as SENet [31], CBAM [32], ACNet [27] and CGNet [38], tend to use
a global pooling to generate a single global vector. Our Global Enhancement
Method, instead, utilizes a learned global attention pooling (Eq. 1) to get
multiple semantic descriptors, thus encoding various semantic representations
from different positions in the global feature space. Recently, OCRNet [13]
also employs the global attention pooling to generate multiple object region
representations, however, it needs additional supervision from the ground-
truth segmentation to get the object regions (equivalent to the attention map
$X_{am}$ in Eq. 1). Moreover, most global context encoding methods design a
plugin module and apply it within a single stage of a network. As far as we
know, we take the first attempt to enhance the global information during the
upsampling process.

###  III-B Local Refinement for Encoder Features

####  III-B1 Noisy Encoder Features in Context Fusion

In context fusion, the low-level visual primitives encoded by shallow layer
features are essential for the accurate prediction of object boundaries and
details. However, due to the limited receptive field of the early encoder
layers, primary features usually capture massive low-level textures from the
entire image, blurring the boundaries between the objects and the background.
The needless textures can be regarded as noises in the context fusion since
they do few contributions to the recognition of the segmentation boundaries
and severely impair the categorization of delicate objects. Compared to the
low-level encoder features, representations from the decoder encodes more
semantic information, presenting relatively distinct discriminations between
different objects and can be even regarded as coarse segmentation maps.
Therefore, it is reasonable to utilize the decoder features as the semantic
guidance to capture the valuable spatial details and filter out the noises. In
the next section, we elaborate on the problem of noisy encoder features from
two aspects (i.e. channel dimension and spatial dimension) and propose
corresponding solutions.

####  III-B2 Local Refinement

A Local Refinement Module is designed to refine the encoder features in both
channel (Channel Resampling) and spatial (Spatial Gating) dimensions.

Channel Resampling: Considerable channels of the low-level features present no
beneficial information for segmentation and need to be filtered out before the
context fusion. Specifically, from the visualization of the encoder features
(first two rows of Fig. 5 and 5), it is observed that a substantial portion of
channels (marked by green boxes) shows no boundaries or rough shape of
segmentation targets. These channels can be regarded as pure noises for being
useless to the recognition of object boundaries. Only a few channels depict
clear structure details that truly contribute to the segmentation. To filter
out the noisy channels of encoder features, the channel attention mechanism is
employed to resample the encoder feature maps in channel dimension, namely
Channel Resampling. To be specific, an attention module is built to model the
channel dependencies between the noisy encoder features and the semantically
rich decoder features. In our design, highly correlated channels of the
encoder feature showing stronger dependencies with the decoder feature are
emphasized, and consequently, the representations of specific semantic
concepts are improved. Conversely, channels of the encoder feature presenting
no discrimination between different objects will be filtered out with a high
probability. The implementation details are illustrated in the red box (“CR”
denotes the Channel Resampling) of Fig. 2. Given an encoder feature
$B\in\mathbb{R}^{C\times kH\times kW}(k=2,4,8)$ and the enhanced decoder
feature $E\in\mathbb{R}^{C\times kH\times kW}$ (the output of Semantic
Distribution Module), the channel attention is implemented to get resampled
encoder feature map $F$ by

$F=B\times softmax(B\times E^{\mathsf{T}}),$ (4)

where the softmax operator is applied to obtain the channel attention maps.

Spatial Gating: Low-level features usually encode local details and textures
without category awareness. On one hand, the overabundant textural
representations outside the target region may blur the object boundaries. On
the other hand, detailed features within an object may lead to the
miscategorization of other regions. Our insight is to utilize the semantic
descriptor map $M$ as a spatial gating map to ‘crop’ the valuable details from
the region of interest, discarding useless or even harmful textures out the
target area. The semantic descriptor map is chosen for its strong semantic
consistency making it more suitable to generate an integrated semantic region
compared to the decoder feature map. As illustrated in the red box (“SG”
denotes the Spatial Gating) of Fig. 2, a sigmoid layer is performed on the
semantic descriptor map $M$ to obtain the spatial gating map, which is then
multiplied with the resampled encoder feature element-wise. The spatial gating
process is formulated as

$G=F\cdot sigmoid(M),$ (5)

where “$\cdot$” denotes the element-wise multiplication.

Through these two steps of refinement (Channel Resampling and Spatial Gating),
the output of Local Refinement Module, a refined encoder feature
$G\in\mathbb{R}^{C\times kH\times kW}$ is finally obtained.

####  III-B3 Comparison

We present comparisons between our Local Refinement Method and some related
works. RefineNet [16] utilizes fine-grained local features to directly refine
the deeper layers, while we do the opposite for the insight that deeper layer
decoder features can provide semantic guidance to capture more valuable
spatial details and filter out the noises. In addition, our design of the
Channel Resampling is derived from the Channel Attention Module (CAM) of DANet
[10], but the key difference between these two is nonnegligible. Specifically,
the CAM performs a self-attention on a single feature map to explicitly model
interdependencies between different channels and reweight the channel maps.
Our Channel Resampling creatively employs a cross-attention between the
encoder and decoder features to enhance fine-grained details and suppress
noisy channels from the former (the encoder features).

###  III-C Context Fusion Block

We combine the Semantic Distribution Module with the Local Refinement Module
in a Context Fusion Block (blue box of Fig. 2), enabling it to capture global
contexts from semantic descriptors and to refine the encoder features for
better local details. Specifically, the refined encoder feature $G$ and the
enhanced decoder feature $E$ is combined by element-wise addition, and the
result is then fed into a convolution together with BN and ReLU layer,
generating the final output of the Context Fusion Block as:

$O=\Phi(E+\beta G;W_{\Phi}),$ (6)

where $O$ is the final output of the Context Fusion Block. Here the refined
encoder feature $G$ is also multiplied with a learnable scalar parameter
$\beta$, and $\Phi(*;W_{\Phi})$ denotes the convolution together with BN and
ReLU layer.

###  III-D Overall Architecture

The overall architecture of the proposed AGLN is shown in Fig. 2. A pre-
trained vallina ResNet [39] is adopted as the encoder, downsampling the image
to 1/32 of the original input size, and then a Semantic Aggregation Block with
three Context Fusion Blocks are stacked on top as the decoder. Following the
Feature Pyramid Network [20], the output from different stages of the decoder
is upsampled to 1/4 input size and concatenated for final segmentation. In the
upsampling process, the Semantic Aggregation Block and three Context Fusion
Blocks are employed with different resolutions. Concretely, the Semantic
Aggregation Block aggregates global semantic descriptors from the output of
the encoder, acting as the first stage of the decoder. After that, the Context
Fusion Blocks serve as the subsequent stages, distributing the global semantic
descriptors to different stages of decoder features and refining the local
details from encoder layers. The output of each decoder block is then
interpolated to 1/4 of the original input size for final prediction. Moreover,
a pixel-wise cross-entropy loss is applied on the final segmentation result.

##  IV Experiments and Results

To evaluate our proposed method, comprehensive experiments are conducted on
three popular segmentation datasets, including PASCAL Context [19], ADE20K
[24], and PASCAL VOC 2012 [25]. Experimental results demonstrate that the AGLN
achieves competitive performances. In the next subsections, we first introduce
the datasets, evaluation metrics, and implementation details, and then present
our best results compared with state-of-the-art methods on PASCAL Context,
ADE20K, and PASCAL VOC 2012 datasets. After that, a series of ablation
experiments are performed on the PASCAL Context dataset to validate the
effectiveness of our proposed Global Enhancement and Local Refinement.
Finally, we propose a lightweight version of the AGLN and conduct experiments
to verify its high performance and efficiency.

###  IV-A Experimental Settings

####  IV-A1 PASCAL Context

The dataset is widely used for scene parsing, which contains 4,998 images for
training and 5,105 images for testing. Following [40] and [28], our method is
evaluated on the most frequent 59 classes (60 classes in total, 59 classes +
background).

####  IV-A2 ADE20K

The dataset is a scene parsing benchmark containing 20210 images for training,
2000 images for validation, and 3352 images for testing. There are 150 labels
in this dataset, including 35 stuff concepts and 115 discrete objects. The
unbalanced distribution of labels and diverse scales of images make the
dataset more challenging.

####  IV-A3 PASCAL VOC 2012

The PASCAL VOC 2012 dataset consists of 1,464 images for training, 1,449 for
validation, and 1,456 for testing, which is a major benchmark dataset for
semantic object segmentation. It includes 20 foreground object classes and one
background class. Following the common practice [41, 6, 9, 11], the training
set is augmented by additional 10,582 annotated images, namely the PASCAL Aug
training set.

####  IV-A4 Evaluation Metrics

The standard mean Intersection of Union (mIoU) and pixel Accuracy (pixAcc) are
chosen as the evaluation metrics. For ablation studies, the single-scale
evaluation is performed if not mentioned. While for the best results,
following the common practice [9, 22, 13], the multi-scale evaluation is used
to compute mIoU and pixAcc. Specifically, for each input image, we first
randomly resize the input image with a scaling factor sampled uniformly from
$[0.5,2.0]$ and then randomly flip the image horizontally. Next, these
predictions are averaged to generate the final prediction.

####  IV-A5 Implementation Details

The FPN is employed as the baseline, and a pre-trained vanilla ResNet with an
output stride of 32 is utilized as the backbone. Note that we apply ResNet-101
as the backbone for the best performance when compared to other state-of-the-
art models and ResNet-50 in the ablation study for a shorter experiment time.
Following [20], the intermediate outputs of the encoder are passed to $3\times
3$ convolution with BN and ReLU layers to reduce the number of channels to 256
before feeding into the decoder blocks. Particularly, in this paper, the
output of the last encoder layer (i.e. $X$ in Fig. 2) is fed into Semantic
Aggregation Block to obtain the semantic descriptors. After that, three
Context Fusion Blocks are stacked on top of the Semantic Aggregation Block,
forming the decoder of our AGLN.

We implement our method based on Pytorch. Following [8, 9], our model is
trained with Stochastic Gradient Descent (SGD). During the training phase, a
poly learning rate policy is employed where the initial learning rate is
multiplied by $(1-iter/total\\_iter)^{0.9}$ after each iteration. The base
learning rate is set to 0.001 for PASCAL Context, ADE20K, and 0.0005 for
PASCAL VOC 2012. Momentum and weight decay coefficients are set to 0.9 and
0.0001 respectively. The batch size is set to 16 for all datasets. Meanwhile,
The training time is set to 240 epochs for PASCAL Context, ADE20K, and 120
epochs for PASCAL VOC 2012. In addition, random scaling, random cropping, and
random left-right flipping are applied as data augmentation during training.

TABLE I: Segmentation results on the PASCAL context validation set. D-ResNet denotes the Dilated-ResNet with the output stride = 8 or 16, ResNet denotes the vallina ResNet with the output stride = 32 Method | Backbone | mIoU$\%$
---|---|---
PSPNet[8] (CVPR2017) | D-ResNet-101 | 47.8
EncNet[9] (CVPR2018) | D-ResNet-101 | 52.6
EMANet[12] (ICCV2019) | D-ResNet-101 | 53.1
CGBNet[42] (TIP2020) | ResNet-101 | 53.4
CPNet[33] (CVPR2020) | D-ResNet-101 | 53.9
CFNet[11] (CVPR2019) | D-ResNet-101 | 54.0
HRNetV2[43] (CVPR2019) | HRNetV2-W48 | 54.0
ACNet[27] (ICCV2019) | D-ResNet-101 | 54.1
GFFNet[22] (AAAI2020) | D-ResNet-101 | 54.2
OCRNet[13] (ECCV2020) | D-ResNet-101 | 54.8
DRANet[28] (TNNLS2020) | D-ResNet-101 | 55.4
SETR-MLA[44] (CVPR2021) | Transformer | 55.8
| HRNetV2[40] (TPAMI2020)
---
\+ OCRNet[13] (ECCV2020)
HRNetV2-W48 | 56.2
AGLN (Ours) | ResNet-101 | 56.23
TABLE II: Segmentation results on the ADE20K Val Set. D-ResNet denotes the
Dilated-ResNet with the output stride = 8 or 16, ResNet denotes the vallina
ResNet with the output stride = 32

Method | Backbone | mIoU$\%$ | PixAcc$\%$
---|---|---|---
PSPNet[8] (CVPR2017) | D-ResNet-101 | 43.29 | 81.39
PSANet[34] (ECCV2018) | D-ResNet-101 | 43.77 | 81.51
EncNet[9] (CVPR2018) | D-ResNet-101 | 44.65 | 81.19
CFNet[11] (CVPR2019) | D-ResNet-101 | 44.89 | -
CGBNet[42] (TIP2020) | ResNet-101 | 44.90 | -
CCNet[30] (ICCV2019) | D-ResNet-101 | 45.22 | -
OCRNet[13] (ECCV2020) | D-ResNet-101 | 45.28 | -
GFFNet[22] (AAAI2020) | D-ResNet-101 | 45.33 | 82.01
GANet[35] (PR2019) | D-ResNet-101 | 45.36 | 82.14
| HRNetV2[40] (TPAMI2020)
---
\+ OCRNet[13] (ECCV2020)
HRNetV2-W48 | 45.66 | -
AGLN (Ours) | ResNet-101 | 45.38 | 81.78

TABLE III: Segmentation results on PASCAL VOC 2012 Test Set. The bold numbers
indicate the best one result
and the gray background highlights the best two results. (without MS COCO
dataset pre-trained)

Method | aero | bike | bird | boat | bottle | bus | car | cat | chair | cow | table | dog | horse | mbike | person | plant | sheep | sofa | train | tv | mIOU$\%$
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
FCN-8s[4] | 76.8 | 34.2 | 68.9 | 49.4 | 60.3 | 75.3 | 74.7 | 77.6 | 21.4 | 62.5 | 46.8 | 71.8 | 63.9 | 76.5 | 73.9 | 45.2 | 72.4 | 37.4 | 70.9 | 55.1 | 62.2
DeepLabv2[6] | 84.4 | 54.5 | 81.5 | 63.6 | 65.9 | 85.1 | 79.1 | 83.4 | 30.7 | 74.1 | 59.8 | 79.0 | 76.1 | 83.2 | 80.8 | 59.7 | 82.2 | 50.4 | 73.1 | 63.7 | 71.6
DeconvNet[15] | 89.9 | 39.3 | 79.7 | 63.9 | 68.2 | 87.4 | 81.2 | 86.1 | 28.5 | 77.0 | 62.0 | 79.0 | 80.3 | 83.6 | 80.2 | 58.8 | 83.4 | 54.3 | 80.7 | 65.0 | 72.5
DPN[41] | 87.7 | 59.4 | 78.4 | 64.9 | 70.3 | 89.3 | 83.5 | 86.1 | 31.7 | 79.9 | 62.6 | 81.9 | 80.0 | 83.5 | 82.3 | 60.5 | 83.2 | 53.4 | 77.9 | 65.0 | 74.1
RefineNet[16] | 94.9 | 60.2 | 92.8 | 77.5 | 81.5 | 95.0 | 87.4 | 93.3 | 39.6 | 89.3 | 73.0 | 92.7 | 92.4 | 85.4 | 88.3 | 69.7 | 92.2 | 65.3 | 84.2 | 78.7 | 82.4
PSPNet[8] | 91.8 | 71.9 | 94.7 | 71.2 | 75.8 | 95.2 | 89.9 | 95.9 | 39.3 | 90.7 | 71.7 | 90.5 | 94.5 | 88.8 | 89.6 | 72.8 | 89.6 | 64.0 | 85.1 | 76.3 | 82.6
EncNet[9] | 94.1 | 69.2 | 96.3 | 76.7 | 86.2 | 96.3 | 90.7 | 94.2 | 38.8 | 90.7 | 73.3 | 90.0 | 92.5 | 88.8 | 87.9 | 68.7 | 92.6 | 59.0 | 86.4 | 73.4 | 82.9
APCNet[36] | 95.8 | 75.8 | 84.5 | 76.0 | 80.6 | 96.9 | 90.0 | 96.0 | 42.0 | 93.7 | 75.4 | 91.6 | 95.0 | 90.5 | 89.3 | 75.8 | 92.8 | 61.9 | 88.9 | 79.6 | 84.2
CFNet[11] | 95.7 | 71.9 | 95.0 | 76.3 | 82.8 | 94.8 | 90.0 | 95.9 | 37.1 | 92.6 | 73.0 | 93.4 | 94.6 | 89.6 | 88.4 | 74.9 | 95.2 | 63.2 | 89.7 | 78.2 | 84.2
DMNet[45] | 96.1 | 77.3 | 94.1 | 72.8 | 78.1 | 97.1 | 92.7 | 96.4 | 39.8 | 91.4 | 75.5 | 92.7 | 95.8 | 91.0 | 90.3 | 76.6 | 94.1 | 62.1 | 85.5 | 77.6 | 84.4
AGLN (Ours) | 94.6 | 77.0 | 93.0 | 71.9 | 81.7 | 95.9 | 90.5 | 96.1 | 38.7 | 95.2 | 74.9 | 94.2 | 96.1 | 90.4 | 90.8 | 78.6 | 94.6 | 59.8 | 89.2 | 83.0 | 84.9

###  IV-B Comparison With State-of-the-Arts

We compare the best results of our AGLN (based on the vallina ResNet-101
backbone) with recent state-of-the-art models on three popular segmentation
benchmarks, PASCAL Context, ADE20K, and PASCAL VOC 2012.

####  IV-B1 Results on PASCAL Context

We present the evaluation results of the proposed AGLN on PASCAL Context
validation set and compare them with other state-of-the-art methods in Table
I. As can be seen, our model achieves the state-of-the-art performance of
56.23$\%$. With the vanilla backbone ResNet-101, our model outperforms
Dilated-ResNet-101 based models (e.g., DRAN, GFFNet, and ACNet, etc.).
Besides, our result is comparable with HRNetV2 + OCRNet, which is based on a
much stronger backbone HRNetV2-W48. The visualization of example results on
the PASCAL Context validation set is shown in Fig. 3 and Fig. 5. Compared to
FPN, our network generates more accurate segmentation results whether on
large-scale background or delicate objects.

####  IV-B2 Results on ADE20K

Experiments are also conducted on the ADE20K dataset to validate the
performance of our AGLN. Quantitative results (evaluated by both PixelAcc and
mIoU) are shown in Table II. With vallina ResNet-101, our AGLN achieves a
competitive performance of 45.38$\%$ mIoU and 81.78$\%$ PixAcc compared to
other state-of-the-art models. Note that the HRNetV2 + OCRNet outperforms our
network with a much stronger backbone HRNetV2-W48. Without the HRNetV2
backbone, OCRNet shows an inferior performance when compared with our method.
The visualization of the example results from FPN and our AGLN is presented in
Fig. 6. The main contrasts marked by the white rectangles show that our
results present stronger classification consistency within an object and more
accurate boundaries.

####  IV-B3 Results on PASCAL VOC 2012

The performance of the proposed AGLN is also evaluated on the PASCAL VOC 2012
dataset. Following [9, 11], our model is first trained on the PASCAL Aug
training data for 120 epochs and then fine-tuned on the original PASCAL VOC
2012 dataset for another 120 epochs. As shown in Table III, AGLN achieves
84.9$\%$ mIOU without COCO pre-training and performs best in many categories,
from large objects (e.g., train, cow, horse, and tv) to small, delicate ones
(e.g., bike, cat, dog, and plant).

TABLE IV: Ablation study for the Global Enhancement Method. GEM represents the Global Enhancement Method while GAP-GEM replaces the global attention pooling to global average pooling in GEM. N denotes the number of semantic descriptors Method | Backbone | N | mIoU$\%$
---|---|---|---
FPN (w/o GEM) | ResNet-50 |  | 46.91
AGLN-(w/ GAP-GEM) | ResNet-50 |  | 48.17
AGLN-(w/ GEM) | ResNet-50 | 1 | 47.93
AGLN-(w/ GEM) | ResNet-50 | 64 | 50.98
AGLN-(w/ GEM) | ResNet-50 | 128 | 51.24
AGLN-(w/ GEM) | ResNet-50 | 256 | 50.88
TABLE V: Ablation study for Local Refinement Module. CR represents Channel Resampling and SG represents Spatial Gating Method | Backbone | CR | SG | mIoU$\%$
---|---|---|---|---
FPN | ResNet-50 |  |  | 46.91
AGLN- | ResNet-50 |  |  | 51.24
AGLN- | ResNet-50 | $\surd$ |  | 51.50
AGLN- | ResNet-50 |  | $\surd$ | 51.58
AGLN | ResNet-50 | $\surd$ | $\surd$ | 51.98
TABLE VI: Ablation study for learnable parameters $\alpha$ and $\beta$. 1 means the parameter is fixed to a constant value of 1, $\ell$ means the parameter is learnable Method | Backbone | $\alpha$ | $\beta$ | mIoU$\%$
---|---|---|---|---
AGLN | ResNet-50 | $\ell$ | $\ell$ | 51.98
AGLN | ResNet-50 | 1 | $\ell$ | 51.20 (-0.78)
AGLN | ResNet-50 | $\ell$ | 1 | 51.68 (-0.30)
AGLN | ResNet-50 | 1 | 1 | 50.93 (-1.05)

Figure 3: The visualization of feature maps and segmentation results from FPN
and our AGLN on the PASCAL Context dataset. From left to right: (a) Input
images; (b) Decoder features; (c) Semantic descriptor maps; (d) Enhanced
decoder features (the fusion results of decoder features and semantic
descriptor maps); (e) FPN results; (f) Our results; (g) Ground truth. (The
feature maps are from the last stage of the decoder)

###  IV-C Ablation Studies

Ablation studies are performed on the PASCAL Context validation set to verify
the effectiveness of different components of our network. All experiments are
conducted on the ResNet-50 backbone.

####  IV-C1 Ablation Study for Global Enhancement Method

To evaluate the effectiveness of our proposed Global Enhancement Method
(consisting of Semantic Aggregation Block and Semantic Distribution Modules),
the performance of the FPN with and without Global Enhancement is compared. To
this end, we apply the Global Enhancement Method on the FPN baseline. The
resulting AGLN- shares a similar structure with AGLN while its decoder is
composed of a Semantic Aggregation Block and three Semantic Distribution
Modules. As shown in Table IV, the Global Enhancement Method improves the
performance significantly. Compared with the FPN baseline, our AGLN- obtains a
result of more than 51$\%$ in mIoU, bringing at least 4$\%$ improvement. In
our design, the Semantic Aggregation Block plays a vital role in capturing
various global semantic features, the performance witnesses an obvious decline
if we replace the global attention pooling in SAB with a global average
pooling (GAP-GEM in Table IV).

####  IV-C2 Ablation Study for Number of Semantic Descriptors

Different numbers of semantic descriptors ($N$) are applied in experiments for
a comprehensive comparison to show their effect on the segmentation results.
As shown in Table IV, compared to 64, increasing the number of semantic
descriptors to 128 could improve the performance (i.e. the bold number
51.24$\%$). However, continue raising it to 256 gains no further improvement
for the segmentation results. Considering the limited segmentation classes in
an image, a compact set of semantic descriptors may lead to an easier
categorization. In addition, if $N$ is set to 1, only one semantic descriptor
is obtained and the attention pooling is degraded to a global average pooling
in performance. Our later experiments are based on the best setting of 128
semantic descriptors ($N$ = 128).

####  IV-C3 Ablation Study for Local Refinement Module

Based on AGLN-, further experiments are conducted to evaluate the
effectiveness of the Local Refinement Module. As shown in Table V, the Channel
Resampling (CR) and Spatial Gating (SG) separately improve the performance to
51.50$\%$ and 51.58$\%$. When we combine these two methods into a Local
Refinement Module and form the complete AGLN, the result further goes up to
51.98$\%$. The comparison of segmentation results between our AGLN and the
baseline in Fig. 5 shows the effectiveness of the Local Refinement Module.
Apparently, Our AGLN generates more accurate segmentation edges compared to
the FPN.

####  IV-C4 Ablation Study for Learnable Parameters $\alpha$ and $\beta$

When fusing semantic descriptor maps and refined encoder features into the
decoder, we respectively multiply them with learnable parameters $\alpha$ and
$\beta$ for flexibility. Experiments are also performed to investigate their
effect on the segmentation results. In practice, the initial value of $\alpha$
is set to 0 to avoid the disturbance from semantic descriptor maps at the
beginning of training, and $\beta$ is initialized as 1. As shown in Table VI,
without the learnable $\alpha$ and $\beta$, the performance of AGLN decreases
by different margins (-0.78% and -0.30% respectively). When $\alpha$ and
$\beta$ are both fixed to 1 during the training process, the mIoU of AGLN
drops down by 1%.

####  IV-C5 Densely Connected Decoder and Multi-scale Evaluation

Inspired by the dense connection in [46] and feature reuse in [27], a further
improvement of our network is developed by applying the Semantic Aggregation
Block and Context Fusion Blocks multi-times in a dense connection way. As
shown in Fig. 4, compared to the straight connection, the densely connected
implementation has two advantages: (1) global semantic information from the
second and third stages of decoder features could be well-exploited through
the repeatedly applied Semantic Aggregation Blocks (i.e. “SAB_2” and “SAB_3”
in Fig. 4); (2) the spatial details from encoder features are further enhanced
in such a recurrent learning process through the consecutive Context Fusion
Blocks (i.e. “CFB1_2”, “CFB1_3” and “CFB2_2” in Fig. 4). As shown in Table
VII, the implementation of the densely connected decoder improves the
performance of AGLN to 52.45$\%$, which is a significant improvement over the
baseline by 5.54$\%$. Moreover, following [9, 10], multi-scale testing is
performed on AGLN to further improve the mIoU to 53.42$\%$. Our best results
in Sec. IV-B are obtained by the best implementation described in this
section.

Figure 4: Different decoder implementations of our network: (a) Straight
connection, (b) Dense connection of SAB and CFBs.

Figure 5: Comparison of the segmentation results and the encoder features between FPN and AGLN. (a) and (c) show input images and segmentation maps (left-top: input image, left-bottom: groud truth, right-top: FPN, right-bottom: Ours.). (b) and (d) show the comparison of the encoder features before and after Local Refinement (First two rows: original encoder features, last two rows: refined encoder features.). Red box: valuable local details, green box: noisy features (best viewed in color). Figure 6: Example results on ADE20K validation set. The images in each column from top to bottom are (a) Input images, (b) Ground truth, (c) FPN, and (d) Ours. The main contrasts are marked by the white dashed lines (best viewed in color). TABLE VII: The results of densely connected decoder and multi-scale evaluation. Dense represents the densely connected decoder and MS represents the multi-scale evaluation Method | Backbone | Dense | MS | mIoU$\%$
---|---|---|---|---
FPN | ResNet-50 |  |  | 46.91
AGLN | ResNet-50 |  |  | 51.98 (+5.07)
AGLN | ResNet-50 | $\surd$ |  | 52.45 (+5.54)
AGLN | ResNet-50 | $\surd$ | $\surd$ |  53.42 (+6.51)
TABLE VIII: Performance (mIoU%) and Complexity Comparison of the AGLN and AGLN-Lite. FCN-8s denotes that the output stride of the FCN encoder is 8 Method | Backbone | PASCAL Context | ADE20K | PASCAL VOC 2012 | Parameters | FLOPs
---|---|---|---|---|---|---
FCN-8s | D-ResNet-101 | 51.38 | 41.40 | 73.57 | 52.09 M | 196.75 G
FPN | ResNet-101 | 52.82 | 42.58 | 81.26 | 54.32 M | 124.59 G
AGLN | ResNet-101 | 56.23 | 45.38 | 84.89 | 61.71 M | 187.16 G
AGLN-Lite | ResNet-101 | 56.21 | 44.04 | 84.38 | 55.74 M | 135.56 G

###  IV-D AGLN-Lite

Although the AGLN gains an impressive improvement in the segmentation
performance compared to the baseline, it results in extra parameters and
computational cost due to the additional $3\times 3$ convolutions in the
repeated Context Fusion Blocks, especially in the densely connected decoder.
Therefore, a lightweight version of AGLN is developed by applying the $3\times
3$ depth-wise separable convolutions as the replacement of the original
$3\times 3$ convolutions in Context Fusion Blocks, namely AGLN-Lite. The
comparison of our AGLN, AGLN-Lite, and FPN shown in Table VIII is performed
with the same experimental setting, while the results of FCN are from the
model zoo (best implementation) of MMSegmentation111https://github.com/open-
mmlab/mmsegmentation/tree/master/conFig/fcn. As can be seen, the AGLN-Lite
gains a considerable reduction in parameters and FLOPs at the cost of a slight
decrease in segmentation performance. Notably, taking Dilated-ResNet as the
backbone, FCN-8s suffers from high computational consumption and so do its
variants (i.e. the Dilated-FCN-based models). In comparison, based on the
vanilla ResNet, our AGLN-Lite becomes a much better choice when considering
the balance between segmentation accuracy and computation efficiency.

###  IV-E Understanding AGLN

To illustrate the ability of AGLN, we visualize the enhanced decoder features
produced by Global Enhancement Method and the refined encoder features
predicted by Local Refinement Module.

####  IV-E1 Global Enhancement Visualization

Fig. 3 depicts the semantic descriptor maps and the enhanced decoder features
generated by Global Enhancement Method on PASCAL Context. As shown in Fig. 3,
original feature maps from the last decoder layer of FPN show vague semantic
boundaries and weak discriminations between different categories, resulting in
misclassifications of large-scale objects (e.g., FPN results in Fig. 3).
Obviously, the semantic descriptor maps in Fig. 3 represent minor intra-class
diversity within a certain object and large inter-class diversity between
different targets. The main reason is that the semantic descriptors are
obtained by the global attention pooling, which owns a clustering effect and
can aggregate the similar semantic concepts together. Moreover, each location
of the decoder feature adaptively integrates the semantic descriptors based on
their specific attention weights, distributing similar semantic concepts in a
consistent semantic region. In Fig. 3, the combinations of original decoder
features and semantic descriptor maps (i.e. the enhanced decoder features)
show much stronger spatial semantic consistency. The comparison of the
segmentation results shown in Fig. 3, 3, and 3 demonstrates that the Global
Enhancement Method helps to generate more semantically consistent and accurate
segmentation results for large-scale objects.

####  IV-E2 Local Refinement Visualization

Fig. 5 and 5 present the visualization samples from the original encoder
feature maps (first two rows) and our refined encoder feature maps (last two
rows). These visualized channels are randomly sampled from the highest-
resolution (1/4 of the input size) encoder feature maps. In these figures, the
red box denotes the valuable local details which may contribute to the
segmentation of delicate object edges (e.g., the aircraft propeller in Fig. 5
and the chair legs in Fig. 5). In contrast, the green box marks out the noisy
features showing no visible distinction between foreground objects and
background.

As shown in Fig. 5 and 5, the effect of the refinement module is obvious.
Specifically, the noisy features are filtered out and the fine-grained details
are greatly enhanced. This remarkable improvement benefits from three aspects:
First, the channel attention significantly suppresses the noisy channels of
the encoder features for the gap between them and the semantically rich
decoder features. Second, the informative channels of encoder features are
greatly enhanced for their relatively high similarity with the enhanced
decoder features. Third, the spatial gating map is employed to selectively
enhance spatial details in a specific semantic area and suppress other
irrelevant regions. The comparison of the segmentation results between the
proposed AGLN and FPN (Fig. 5 and 5) demonstrates the effectiveness of our
Local Refinement Module.

##  V Conclusion

In this paper, a novel AGLN is proposed to improve the encoder-decoder network
for image segmentation. First, a Global Enhancement Method is designed to
capture global semantic information from high-level features to complement the
deficiency of global contexts in the upsampling process. Then, a Local
Refinement Module is built to refine the noisy encoder features in both
channel and spatial dimensions before the context fusion. After that, the
proposed two methods are integrated into the Context Fusion Blocks, enabling
the AGLN to generate semantically consistent segmentation masks on large-scale
stuff and accurate boundaries on delicate objects. Furthermore, a lightweight
implementation AGLN-Lite is developed, achieving a balance of performance and
computational complexity. Experiments on the PASCAL Context, ADE20K, and
PASCAL VOC 2012 datasets show that the proposed AGLN provides an effective
solution to improve the performance of the FPN baseline and achieves state-of-
the-art segmentation results ($56.23\%$ mean IOU on the PASCAL Context
dataset). In conclusion, the proposed AGLN is simple, effective, and model-
agnostic in the sense that it can be applied to various encoder-decoder
architectures.

## Acknowledgment

This work was supported by the Fundamental Research Funds for the China
Central Universities of USTB (FRF-DF-19-002), Beijing Key Discipline
Development Program (No. XK100080537).

## References

  * [1] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele, “The cityscapes dataset for semantic urban scene understanding,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 3213–3223.
  * [2] Y. Yoon, H.-G. Jeon, D. Yoo, J.-Y. Lee, and I. So Kweon, “Learning a deep convolutional network for light-field image super-resolution,” in _Proceedings of the IEEE international conference on computer vision workshops_ , 2015, pp. 24–32.
  * [3] M. Oberweger, P. Wohlhart, and V. Lepetit, “Hands deep in deep learning for hand pose estimation,” _arXiv preprint arXiv:1502.06807_ , 2015.
  * [4] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2015, pp. 3431–3440.
  * [5] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Semantic image segmentation with deep convolutional nets and fully connected crfs,” _arXiv preprint arXiv:1412.7062_ , 2014.
  * [6] ——, “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 40, no. 4, pp. 834–848, 2017\.
  * [7] L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam, “Rethinking atrous convolution for semantic image segmentation,” _arXiv preprint arXiv:1706.05587_ , 2017.
  * [8] H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, “Pyramid scene parsing network,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 2881–2890.
  * [9] H. Zhang, K. Dana, J. Shi, Z. Zhang, X. Wang, A. Tyagi, and A. Agrawal, “Context encoding for semantic segmentation,” in _Proceedings of the IEEE conference on Computer Vision and Pattern Recognition_ , 2018, pp. 7151–7160.
  * [10] J. Fu, J. Liu, H. Tian, Y. Li, Y. Bao, Z. Fang, and H. Lu, “Dual attention network for scene segmentation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 3146–3154.
  * [11] H. Zhang, H. Zhang, C. Wang, and J. Xie, “Co-occurrent features in semantic segmentation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 548–557.
  * [12] X. Li, Z. Zhong, J. Wu, Y. Yang, Z. Lin, and H. Liu, “Expectation-maximization attention networks for semantic segmentation,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 9167–9176.
  * [13] Y. Yuan, X. Chen, and J. Wang, “Object-contextual representations for semantic segmentation,” in _Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VI 16_. Springer, 2020, pp. 173–190.
  * [14] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in _International Conference on Medical image computing and computer-assisted intervention_. Springer, 2015, pp. 234–241.
  * [15] H. Noh, S. Hong, and B. Han, “Learning deconvolution network for semantic segmentation,” in _Proceedings of the IEEE international conference on computer vision_ , 2015, pp. 1520–1528.
  * [16] G. Lin, A. Milan, C. Shen, and I. Reid, “Refinenet: Multi-path refinement networks for high-resolution semantic segmentation,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 1925–1934.
  * [17] Z. Tian, T. He, C. Shen, and Y. Yan, “Decoders matter for semantic segmentation: Data-dependent decoding enables flexible feature aggregation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 3126–3135.
  * [18] J. Wang, K. Chen, R. Xu, Z. Liu, C. C. Loy, and D. Lin, “Carafe: Content-aware reassembly of features,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 3007–3016.
  * [19] R. Mottaghi, X. Chen, X. Liu, N.-G. Cho, S.-W. Lee, S. Fidler, R. Urtasun, and A. Yuille, “The role of context for object detection and semantic segmentation in the wild,” in _IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2014.
  * [20] T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 2117–2125.
  * [21] Z. Zhang, X. Zhang, C. Peng, X. Xue, and J. Sun, “Exfuse: Enhancing feature fusion for semantic segmentation,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , 2018, pp. 269–284.
  * [22] X. Li, H. Zhao, L. Han, Y. Tong, S. Tan, and K. Yang, “Gated fully fusion for semantic segmentation,” in _Proceedings of the AAAI conference on artificial intelligence_ , vol. 34, no. 07, 2020, pp. 11 418–11 425.
  * [23] F. Milletari, N. Navab, and S.-A. Ahmadi, “V-net: Fully convolutional neural networks for volumetric medical image segmentation,” in _2016 fourth international conference on 3D vision (3DV)_. IEEE, 2016, pp. 565–571.
  * [24] B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso, and A. Torralba, “Scene parsing through ade20k dataset,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 633–641.
  * [25] M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman, “The pascal visual object classes (voc) challenge,” _International journal of computer vision_ , vol. 88, no. 2, pp. 303–338, 2010.
  * [26] V. Badrinarayanan, A. Kendall, and R. Cipolla, “Segnet: A deep convolutional encoder-decoder architecture for image segmentation,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 39, no. 12, pp. 2481–2495, 2017.
  * [27] J. Fu, J. Liu, Y. Wang, Y. Li, Y. Bao, J. Tang, and H. Lu, “Adaptive context network for scene parsing,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 6748–6757.
  * [28] J. Fu, J. Liu, J. Jiang, Y. Li, Y. Bao, and H. Lu, “Scene segmentation with dual relation-aware attention network,” _IEEE Transactions on Neural Networks and Learning Systems_ , 2020.
  * [29] X. Wang, R. Girshick, A. Gupta, and K. He, “Non-local neural networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 7794–7803.
  * [30] Z. Huang, X. Wang, L. Huang, C. Huang, Y. Wei, and W. Liu, “Ccnet: Criss-cross attention for semantic segmentation,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 603–612.
  * [31] J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 7132–7141.
  * [32] S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, “Cbam: Convolutional block attention module,” in _Proceedings of the European conference on computer vision (ECCV)_ , 2018, pp. 3–19.
  * [33] C. Yu, J. Wang, C. Gao, G. Yu, C. Shen, and N. Sang, “Context prior for scene segmentation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 12 416–12 425.
  * [34] H. Zhao, Y. Zhang, S. Liu, J. Shi, C. C. Loy, D. Lin, and J. Jia, “Psanet: Point-wise spatial attention network for scene parsing,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , 2018, pp. 267–283.
  * [35] P. Zhang, W. Liu, H. Wang, Y. Lei, and H. Lu, “Deep gated attention networks for large-scale street-level scene segmentation,” _Pattern Recognition_ , vol. 88, pp. 702–714, 2019.
  * [36] J. He, Z. Deng, L. Zhou, Y. Wang, and Y. Qiao, “Adaptive pyramid context network for semantic segmentation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 7519–7528.
  * [37] Y. Cao, J. Xu, S. Lin, F. Wei, and H. Hu, “Gcnet: Non-local networks meet squeeze-excitation networks and beyond,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops_ , 2019, pp. 0–0.
  * [38] T. Wu, S. Tang, R. Zhang, J. Cao, and Y. Zhang, “Cgnet: A light-weight context guided network for semantic segmentation,” _IEEE Transactions on Image Processing_ , vol. 30, pp. 1169–1179, 2020.
  * [39] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 770–778.
  * [40] J. Wang, K. Sun, T. Cheng, B. Jiang, C. Deng, Y. Zhao, D. Liu, Y. Mu, M. Tan, X. Wang _et al._ , “Deep high-resolution representation learning for visual recognition,” _IEEE transactions on pattern analysis and machine intelligence_ , 2020.
  * [41] Z. Liu, X. Li, P. Luo, C.-C. Loy, and X. Tang, “Semantic image segmentation via deep parsing network,” in _Proceedings of the IEEE international conference on computer vision_ , 2015, pp. 1377–1385.
  * [42] H. Ding, X. Jiang, B. Shuai, A. Q. Liu, and G. Wang, “Semantic segmentation with context encoding and multi-path decoding,” _IEEE Transactions on Image Processing_ , vol. 29, pp. 3520–3533, 2020.
  * [43] K. Sun, Y. Zhao, B. Jiang, T. Cheng, B. Xiao, D. Liu, Y. Mu, X. Wang, W. Liu, and J. Wang, “High-resolution representations for labeling pixels and regions,” _arXiv preprint arXiv:1904.04514_ , 2019.
  * [44] S. Zheng, J. Lu, H. Zhao, X. Zhu, Z. Luo, Y. Wang, Y. Fu, J. Feng, T. Xiang, P. H. Torr _et al._ , “Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 6881–6890.
  * [45] J. He, Z. Deng, and Y. Qiao, “Dynamic multi-scale filters for semantic segmentation,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 3562–3572.
  * [46] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 4700–4708.


# Interesting magnetic response of the nuclear fuel material UO2

Sudip Pal1, L. S. Sharath Chandra2, M. K. Chattopadhyay2,3 and S. B. Roy1 1
UGC DAE Consortium for Scientific Research, University Campus
Khandwa Road, Indore 452001, India  2 Free Electron Laser Utilization
Laboratory, Raja Rammana Centre for Advanced Technology, Indore-452 013, India
3 Homi Bhabha National Institute, Training School Complex, Anushakti Nagar,
Mumbai 400 094, India<EMAIL_ADDRESS>
###### Abstract

Magnetic response of uranium dioxide (UO2) has been investigated through
temperature and magnetic field dependent dc magnetization measurements. UO2 is
a paramagnet at room temperature. The magnetic susceptibility, however,
deviates from Curie-Weiss (CW) like paramagnetic behavior below $T$ = 280 K.
Further down the temperature UO2 undergoes phase transition to an
antiferromagnetic state below $T_{N}$ = 30.6 K. The zero field cooled (ZFC)
and field cooled (FC) magnetizations exhibit some distinct thermomagnetic
irreversibility below $T_{N}$. The temperature dependence of the FC
magnetization is more like a ferromagnet, whereas ZFC magnetization exhibits
distinct structures not usually observed in the antiferromagnets. In low
applied magnetic field this thermomagnetic irreversibility in magnetization
exists in a subtle way even in the paramagnetic regime above $T_{N}$ up to a
fairly high temperature, but vanishes in high applied magnetic fields.
Deviation from CW law and irreversibility between ZFC and FC magnetization
indicate that the paramagnetic state above $T_{N}$ is not a trivial one.
Magnetic response below $T_{N}$ changes significantly with the increase in the
applied magnetic field. Thermomagnetic irreversibility in magnetization
initially increases with the increase in the strength of applied magnetic
field, but then gets reduced in the high applied fields. A subtle signature of
a magnetic field induced phase transition is also observed in the isothermal
magnetic field vartaion of magnetization. All these experimetal results
highlight the non-trivial nature of the antiferromagnetic state in UO2

##  1 Introduction

Uranium di-oxide (UO2) is a well known nuclear fuel material and used
worldwide in nuclear reactors for electrical power generation and research.
UO2 is also recognized as a Mott-Hubbard insulator [1, 2], and it promises
other technological applications apart from a nuclear fuel [3, 4]. Thermal
conductivity is very important for the removal of heat produced in a nuclear
reactor by fission in the nuclear fuel materials. As a result thermal
properties of UO2 particularly have drawn much attention over the years [1, 5,
6, 7, 8]. UO2 crystallizes in face centred cubic (fcc) calcium fluorite
structure ($Fm\overline{3}m$), in which U4+ ions are surrounded by eight O2-
ions forming a cube [9]. Therefore, the anisotropic thermal conductivity
reported in this compound is rather unexpected and emphasizes on the relevance
of spin-phonon coupling, which is associated with the magnetic state of the
system [8]. The Mott insulating state in UO2 further highlights the importance
of strong electron-electron correlation in the system [2, 4]. Various
techniques, including neutron scattering and nuclear magnetic resonance (NMR)
have revealed a complex 3k- non-collinear antiferromagnetic (AFM) spin
ordering below $T_{N}$ = 30.6 K. The transition is first order in nature and
is accompanied by a small lattice distortion, predominantly in the oxygen cage
[10, 11, 12]. In cubic crystal field, the nine fold degenerate ($5f^{2}$,
$J=4$) state splits up with a 3-fold degenerate ground state, resulting into
Jahn-Teller (JT) instability [13].

Spin-orbit coupling, Coulomb interaction, antiferromagnetic exchange
interaction and JT distortion are of comparable strength in UO2. Below
$T_{N}$, a quadrupoler ordering is established together with AFM spin
ordering, facilitated by the interaction between cooperative JT distortion,
antiferromagnetic exchange interaction and $5f$ quadrupoles [14, 15, 16, 17].
While, there is some understanding of the physical properties of UO2 in the
antiferromagnetic state below $T_{N}$, characteristics of the high temperature
state are not quite clear. The nature of the high temperature state, although
considered to be paramagnetic, is not so straightforward. Temperature
dependence of thermal conductivity exhibits a minimum at $T_{N}$ and a maximum
around $T$ = 220 K, which clearly highlight unusual physical state above
$T_{N}$. Moreover, a positive magnetostriction has been observed above
$T_{N}$, whereas magnetostriction is negative in the AFM state. Neutron
scattering has provided evidence for dynamic JT distortion of the oxygen
sublattice above $T_{N}$ [18, 19, 20, 21]. Temperature dependence of elastic
constant above $T_{N}$ also exhibit unconventional behavior [11]. UO2 has been
reported to be showing some other interesting physical properties, such as
piezomagnetism and magnetoelastic memory driven by spin-lattice interaction
[9].

Here we present a detailed study of the temperature ($T$) and magnetic field
($H$) dependence of magnetization ($M$) in UO2. The results of our study
highlight various hitherto unknown interesting aspects of the magnetic respose
of UO2. We show that below $T$ = 280 K the low field magnetization or
susceptibility deviates from standard Curie-Weiss paramagnetic behavior.
Abrupt changes in both the zero field cooled (ZFC) and field cooled (FC)
magnetization are observed at $T_{N}$ = 30.6 K where UO2 undergoes a phase
transition to antiferromagnetic state along with distinct thermomagnetic
irreversibility i.e. $M_{ZFC}(T)$ $\neq$ $M_{FC}(T)$ inside the
antiferromagnetic state. The magnetic response in the antiferromagnetic state
changes considerably with the increase in the applied magnetic field. The
observed behaviour may be correlated to a magnetic field induced phase
transition. The thermomagnetic irreversibility in the antiferromagnetic state
continues to exist in the temperature range above $T_{N}$ and below $T$ = 150
K, but it gets suppressed completely at higher applied magnetic fields.

Figure 1:  X-ray diffraction pattern of UO2 powder sample recorded at room
temperature using wavelength, $\lambda$ = 0.7121 Å. The data have been fitted
using rietveld refinement method. Indexes of the planes corresponding to peaks
are also mentioned. The blue line at the bottom shows the difference between
experimental data and fitted curve.

##  2 Experimental details

Powder sample of UO2 has been prepared by reducing UO3 in hydrogen atmosphere
at 700∘C at Bhabha Atomic Research Centre, Trombay. Room temperature X-ray
diffraction data have been recorded at the wavelength of $\lambda$ = 0.7121 Å
in the beamline-BL12 in Indus-2 synchrotron radiation source at Raja Ramanna
Centre for Advanced technology (RRCAT), Indore, India. Magnetic measurements
have been carried out in a MPMS-3 SQUID-VSM magnetometer (M/S Quantum Design,
USA). Temperature dependence measurements of magnetization have been performed
in temperature sweep mode of measurement at 0.5 K/min cooling and heating
rate.

##  3 Results and discussion

Fig.1 shows the X-ray diffraction data of UO2 measured at room temperature in
$\theta-2\theta$ geometry. In the inset, we have shown a magnified view of the
data around (111) peak, which is the most intense peak in this case. Sharp
diffraction peaks and flat background reveal the good crystalline nature of
the sample. The data have been analyzed by Rietveld refinement method using
Fullprof software package. All the peaks can be indexed in cubic fcc structure
(space group $Fm\overline{3}m$), which rules out the presence of any secondary
phase in the sample. The optimized lattice parameter obtained from fitting is,
$a$ = 5.4594 Å. Lattice parameter of UO2 is sensitive to oxygen stoichiometry
of the sample and follows an empirical law given by, $a=5.4705-0.132x$, where
$x$ is defined as UO2+x [22]. In our case, the obtained lattice parameter
corresponds to, $x=0.08$.

Figure 2:  (a) Magnetization ($M$) versus temperature ($T$) plots measured in
applied fields of (a) $H$ = 100 Oe, (b) $H$ = 1 kOe, (c) $H$ = 50 kOe and (d)
$H$ = 70 kOe in the ZFC, FCC and FCW modes. FCC and FCW curves overlap at all
temperatures. The $M_{ZFC}(T)$ and $M_{FC}(T)$ curves show bifurcation below
$T_{N}$. The bifurcation also persists above $T_{N}$ at least upto 150 K in
the applied fields of 100 Oe. The inset in Fig. (a) shows the magnified view
of ZFC and FCW curve above $T_{N}$ in applied field of 100 Oe. The inset in
Fig. (b) shows $\Delta M=M_{FC}-M_{ZFC}$ in fields of $H$ = 100 Oe and 1 kOe.
In applied fields of 50 kOe and 70 kOe $M_{ZFC}(T)$ and $M_{FC}(T)$ curves
show a sudden and discontinuous fall in magnetization at $T_{N}$. $M_{ZFC}(T)$
and $M_{FC}(T)$ bifurcate below temperature $T_{irr}<T_{N}$ (see Figs. c and
d).

Fig. 2(a) shows the temperature dependence of magnetization ($M$) in UO2
measured at an applied field of $H$ = 100 Oe in the zero field cooled (ZFC)
warming, field cooled cooling (FCC) and field cooled warming (FCW) modes. In
ZFC mode the measuring field $H$ = 100 Oe is applied after cooling the sample
to the lowest temperature (here 2 K) of measurement in zero external field.
ZFC magnetization ($M_{ZFC}$) is then measured while warming the sample. After
reaching the highest temperature (room temperature) of measurement the sample
is subsequently cooled back to 2 K in same field while measuring the FCC
magnetization ($M_{FCC}$). After FCC measurements the FCW magnetization
($M_{FCW}$) is measured while by warming the sample again gradually to room
temperature in the same field. Temperature variation of magnetization in the
high temperature regime in all the modes (ZFC, FCC and FCW) indicate the
presence of magnetic moment in UO2, which is in line with the Mott insulator
status of the sample. Below $T$ = 50 K, $M$ tends to flatten while the
temperature is further reduced and then $M$ changes abruptly around $T_{N}$ =
30.6 K. This temperature matches well with the reported temperature where UO2
supposedly undergoes a phase transition to a complex antferromagnetic state
[12]. Figure 2(a) highlights the presence of a very distinct thermomagnetic
irreversibility i.e. $M_{ZFC}(T)$ $\neq$ $M_{FC}(T)$ in the antiferromagnetic
state. Such thermomagnetic irreversibility is not expected in any standard
antiferromagnet [23], and to the best of our knowledge has not been reported
earlier for UO2. M${}_{FCC}(T)$ and $M_{FCW}(T)$ curves overlap at all
temperatures of measurement and now onwords we will designate the field cooled
magnetization as $M_{FC}$. Interestingly in an expanded scale (see the inset
of Fig. 2(a)) one can see that the $M_{ZFC}(T)$ and $M_{FC}(T)$ curves
actually start to bifurcate below $T$ = 150 K, which is well over $T_{N}$ and
nearly equal to $5T_{N}$; the bifurcation gradually increases with decrease in
temperature. The bifurcation between the $M_{ZFC}(T)$ and $M_{FC}(T)$ curves
below the antiferromagnetic transition temperature $T_{N}$ is much larger than
the bifurcation above $T_{N}$. Just below the transition temperature,
$M_{ZFC}(T)$ curve starts to rise sharply with increase in temperature, which
is followed by a rapid fall resulting into a narrow peak around $T$ = 29 K.
Below about $T$ = 10 K, $M_{ZFC}$(T) curve increases slowly with decreasing
temperature. All these features are not typical of a collinear
antiferromagnet. Field cooled magnetization $M_{FC}(T)$ on the other hand
rises sharply with decrease in temperature below $T_{N}$ and tends to saturate
below $T$ = 15 K. Such a sharp rise in magnetization followed by a tendency
towards saturation is not expected in the case of a collinear AFM state and is
usually a characteristic feature of a ferromagnetic state. This kind of
behaviour possibly arises due to non-collinear spin ordering within the AFM
state of UO2, which results into weak ferromagnetic response below $T_{N}$. It
may be noted here that, although the antiferromagnetic transition in UO2 is
reported to be first order in nature, $M_{FCC}(T)$ and $M_{FCW}(T)$ do not
show any thermal hysteresis acrosss $T_{N}$. It means that the supercooling
and superheating phenomena, which are usually observed across a first order
phase transition in many systems[24, 25], are absent across the
antiferromagnetic transition in UO2.

Fig. 2(b) presents the results of the magnetization measurements as a function
of temperature carried out in an applied magnetic field of $H$ = 1 kOe.
$M_{ZFC}$, $M_{FCC}$ and $M_{FCW}$ curves appear to be qualitatively similar
to those obtained at $H$ = 100 Oe. However, there are important differences,
which are noted below:

  1. 1.

The difference between $M_{ZFC}$ and $M_{FC}$ curves i.e. the thermomagnetic
irreversibility above $T_{N}$ is completely erased.

  2. 2.

The thermomagnetic irreversibility below $T_{N}$ increases by a large extent
than that at $H$ = 100 Oe. The value of $\Delta M=M_{FCW}-M_{ZFC}$ at $H$ =
100 and 1 kOe is shown in the inset of Fig. 2(b).

  3. 3.

Bifurcation of $M_{ZFC}$ and $M_{FC}$ curves starts just below $T_{N}$ and
this temperature does not seem to change with increase in applied magnetic
field from 100 Oe to 1 kOe.

  4. 4.

The low temperature increase in $M_{ZFC}$ is more pronounced.

Figure 3:  Isothermal field ($H$) variation of magnetization($M$) in the
antiferromagnetic regime at $T$ = 29 K and 2 K: (a) and (b) show the virgin
curves at $T$ = 29 and 2 K near origin. $M(H)$ in the low field region is
fitted and extrapolated to show the change in slope in the virgin $M-H$ curve.
(c) and (d) show isothermal $M-H$ loops at $T$ = 29 and 2 K. In the Inset of
(c) $M-H$ curve at $T$ = 32 K has been shown only in the first quadrant.

Thermomagnetic irreversibility is a characteristic feature of spin glass or
cluster glass systems, where it arises due to freezing of spins or magnetic
moments originating from the competing magnetic interactions and associated
frustration [26, 27]. Thermomagnetic irrersivility is also observed in
ferromagnets, which is associated with domain wall pinning [28]. Even some
antiferromagnets with ThCr2Si2 structure exhibits thermomagnetic
irrersivility, which was attributed to the stacking faults in those compunds
with layered structures [29]. However, in all these magnetic systems the
thermomagnetic irreversibility gets suppressed with increase in the applied
magnetic field. This is in contrast with the observed increase in
thermomagnetic irreversibility in UO2 with the increase in applied fields from
100 Oe to 1 kOe. Such behaviour in UO2, however, changes in the region of high
applied fields. Fig. 2(c) and 2(d) present the temperature dependence of
magnetization at $H$ = 50 kOe and 70 kOe measured in ZFC, FCC and FCW modes.
In this case the thermomagnetic irreversibility as well as $T_{irr}$ get
reduced as the applied magnetic field is increased from 50 kOe to 70 kOe,
whereas the antiferromagnetic transition temperature $T_{N}$ remains largely
unaffected. It may also be noted that the thermomagnetic irreversibility in
the paramagentic regime is totally absent in this high applied field regime.

From the results discussed above it is clear that magnetic response in low
applied magnetic fields in both ZFC and FC states below $T_{N}$ in UO2 is
distinctly different from that at high applied magnetic fields. More evidences
in this direction emerge if one looks carefully to the temperature dependence
of high field magnetization. As the UO2 sample is cooled from high
temperature, magnetization increases and tends to saturate around 35 K.
However, just above $T_{N}$, magnetization decreases by a small amount,
resulting into a small hump prior to $T_{N}$. The magnetic response at
$T_{N}$, and also at low temperatures, change drastically both in the ZFC and
FC states, as compared to magnetic behavior at low fields, shown in Fig. 2(a)
and 2(b). At $H$ = 50 kOe, both the $M_{ZFC}(T)$ and $M_{FC}(T)$ curves
undergo an abrupt and discontinuous fall at $T_{N}$, which is immediately
followed by a small rise in magnetization with decrease in temperature. Then,
they show a relatively flat region over a temperature region of 28-22 K. In
the lower temperature region, the $M_{ZFC}$ and $M_{FCC}$ curves bifurcate:
the $M_{ZFC}$ curve starts to decrease, whereas the $M_{FCC}$ curve slowly
increases. Note that, the $M_{ZFC}$ also shows an additional shallow dip
around $T_{d}$ = 15 K. Note that, the bifurcation between $M_{ZFC}(T)$ and
$M_{FC}(T)$ curves appears only below a temperature $T_{irr}$ (see Fig. 2(c)),
which is lower than the transition temperature $T_{N}$. At $H$ = 70 kOe,
magnetization shows larger drop at $T_{N}$ and all the three curves gradually
decrease with further decrease in temperature (see Fig. 2(d)). Below
$T_{irr}$, the $M_{ZFC}$ curve continues to decrease, whereas the
$M_{FCC}/M_{FCW}$ curve increases. This is in clear contrast to the
temperature dependence of magnetization obtained in applied magnetic fields of
100 Oe and 1 kOe (see Fig.2(a) and 2(b)).

To investigate further on the antiferromagnetic state in UO2, we present in
Fig. 3 the isothermal $M-H$ curves at $T$ = 29 and 2 K measured starting from
the ZFC state. Here, the sample is initially cooled to the temperature of
measurement in the absence of any external field. Then $M$ is measured while
increasing $H$ isothermally to 70 kOe to record initial (virgin) curve, which
is shown in Figs. 3(a) and 3(b) for $T$ = 29 and 2 K, respectively. After
recording the virgin curve, $M$ is measured while varying $H$ between $\pm$70
kOe to record the envelope curves, which are shown in Figs. 3(c) and 3(d). The
virgin curves at both temperature lie within the envelope curves. Note that,
at $T$ = 29 K, the $M-H$ curve shows a small change in the slope above $H_{a}$
= 1.2 kOe and continues to increase at higher field up to the highest applied
field of $H$ = 70 kOe. Whereas, the virgin $M-H$ curve at $T$ = 2 K deviates
from linearity above $H_{a}$ = 15 kOe. The envelope $M-H$ curve at $T$ = 29 K
shows a small hysteresis with a coercive field of around $H_{C}$ = 2.2 kOe,
which increases to 4.5 kOe at 2 K. The $M-H$ curves do not show any tendency
of saturation till $H$ = 70 kOe. The nonsaturating $M-H$ curve highlights the
antiferromagnetic state but the presence of hysteresis is rather unexpected in
AFM state. At further lower temperature, both coercive field and remanant
magnetization further increase as evident from the $M-H$ curve at $T$ = 2 K.
This change in the slope of $M-H$ curves above $H=H_{a}$, may indicate some
sort of field induced transition of the zero field cooled AFM state.

The increase in the difference between $M_{ZFC}(T)$ and $M_{FC}(T)$ with
magnetic applied magnetic field is one of the important signatures of a
kinetically arrested first order transitions[25, 31, 32, 33, 34]. In this
case, the dynamics of a first order phase transition gets arrested in a $H-T$
window. The difference between $M_{ZFC}(T)$ and $M_{FC}(T)$ increases because
cooling in different magnetic fields produce different volume fractions of the
high and low temperature magnetic phases. As stated earlier, the transition in
UO2 is first order in nature, and like a kinetically arrested system
thermomagnetic irreversibility in the low field regime increases with the
applied magnetic field. In this context, the $M-H$ curves shown in Fig. 3, can
be rationalized in the following manner: the ZFC state undergoes a field
induced transition above a critical field $H_{a}$, as observed from the change
in slope. However, in the field decreasing cycle, the reverse transition is
not observed, so that the magnetic state induced by applied magnetic field
persists during the entire envelope curve and gives rise to hysteresis loop.
Such behaviour has been reported in the literature in the cases of kinetic
arrest of first order phase transition in various magnetic systems, where the
zero field cooled state may be either equilibrium low-$T$ phase, or the
kinetically arrested high-$T$ phase, depending on the nature of the ground
state [35, 34]. Now, after zero field cooling, in the first case, when
magnetic field is increased, above certain field, the equilibrium phase
undergoes a field induced transition to the high-$T$ phase as the superheating
band is crossed [32, 34, 35]. On the other hand, in the second case, the
kinetically arrested state devitrify into the equilibrium low-$T$ phase while
increasing the field. In either cases, the envelope curve does not show the
reverse transition.

Figure 4:  $\chi^{-1}$ versus $T$ data measured at $H$ = 100 Oe in the ZFC
mode. The inset shows the CW law fitting above 280 K.

The unusual dependence of magnetization on temperature and the finite
difference between $M_{ZFC}(T)$ and $M_{FC}(T)$ curves much above $T_{N}$
clearly highlight an unconventional paramagnetic state at high temperatures.
Therefore, to further understand the magnetic property of UO2 at high
temperatures, $\chi^{-1}$ versus $T$ at $H$ = 100 Oe measured in ZFC mode is
plotted in Fig. 4. The $\chi^{-1}$ versus T data appears to be grossly linear
in the temperature range from $T$ = 300 K to 70 K. We tried to fit the results
in this temperature regime by using Curie-Weiss (CW) law, given by
$\chi=\frac{C}{T+T_{0}}$, where
$C=\frac{N\mu_{eff}^{2}}{3k_{B}}=\frac{Ng^{2}J(J+1)}{3k_{B}}$. However, when
looked carefully, the CW law does not fit the experimental data in the entire
temperature regime. In fact, it deviates from linearity below around $T$ = 280
K (see the inset of Fig. 4). The effective magnetic moment $\mu_{eff}$
obtained to be 2.82 $\mu_{B}$/f.u., which is smaller than the expected value
of 3.57 $\mu_{B}$/f.u. for $J$ = 4 (L =5 and S= 1). However, it matches
extremely well with value of effective moment expected for the threefold
degenerate ground state in UO2 in a cubic crystal field [12]. $T_{0}$ is
obtained to be around 171 K, which is around 5.6 times higher than the
transition temperature. The deviation of the susceptibility from CW law, the
presence of irreversibility much above $T_{N}$, higher value of $T_{0}$ than
$T_{N}$ are interesting. There exist some earlier reports of the unusual
characteristics above $T_{N}$ [8]. In inelastic neutron scattering
experiments, magnetic inelastic response has been observed above $T_{N}$ up to
as high as $T$ = 200 K [20]. It has been suggested that the coherent motion of
the neighbouring oxygen cages produces uncorrelated 1-k type dynamical JT
distortion in the paramagnetic state of UO2 [20]. With the lowering in
temperature correlation builds up and a static 3-k type distortion condenses
at $T_{N}$. The tendency of $M$ to saturate as $T_{N}$ is approached during
cooling, the deviation from the CW law, existence of bifurcation between the
ZFC and FC curves highlight the unusual paramagnetic state at high
temperature, which may arise due to short range 1-k type dynamic JT
distortion.

##  4 Conclusion

Summarizing, we can say from the results of temperature and magnetic field
dependent studies of dc magnetization that the low temperature
antiferromagnetic state in UO2 is non-trivial in nature and is accompanied
with large thermomagnetic irreversibility. The field cooled magnetization
$M_{FC}(T)$ measured in low fields of 100 Oe and 1 kOe indicates the presence
of ferromagnetic character in UO2. This inference is further corroborated by
the presence of hysteresis in isothermal field variation of magnetization
alongwith appreciable coercive field. The nature of the thermomagnetic
irreversibility changes in the presence of high applied magnetic field. In the
low magnetic field region the thermomagnetic irreversibility irreversibility
increases with the increase in applied field, which can be rationalized in
terms of the kinetic arrest of magnetic field induced first order phase
transition. The thermomagnetic irreversibility in the high magnetic field
regime deceases with the increase in applied field. Prima facie this behaviour
is quite similar to that observed in some ferro and antiferromagets, where it
was attributed to the hindrance in domain wall motion. Furthermore the
paramagetic state in UO2 is also unusual with a deviation of Curie-Weiss law
and the presence of thermomagnetic irreversibility in the temperature region
much above $T_{N}$. These results indicate the existence of some short range
magnetic correlations well inside the paramagnetic region of UO2. Overall the
results of our present study are expected to stimulate further microscopic
measurements involving neutron scattering and muon spin rotation ($\mu$SR)
measurents to find out the exact microscopic nature of magnetic states in
various magnetic field ($H$) - temperature ($T$) regime of UO2.

##  5 Acknowledgment

We acknowledge Dr. Vinay Kumar of Bhabha Atomic Research Centre, Trombay for
providing us with powdered UO2 sample and Dr. A. Sagdeo and Dr. A. K. Sinha of
Raja Ramanna Centre for Advanced Technology, Indore for help in X-ray
diffraction measurement. S B Roy acknowledges financial support from
Department of Atomic Energy, India in the form of Raja Ramanna Fellowship.
Sudip Pal and S B Roy thank Dr. A J. Pal, Director, UGC-DAE CSR and Dr. D. M.
Phase, Centre Director, UGC-DAE CSR, Indore Centre for suuport and
encouragement.

##  6 References:

## References

  * [1] Quan Yin and Sergey Y. Savrasov, Phys. Rev. Lett 100, 225504 (2008).
  * [2] S. -W. Yu et al, Phys. Rev. B 83, 165102 (2011).
  * [3] J. Schoenes, J. Appl. Phys. 49 1463 (1978).
  * [4] S. B. Roy, Physics of Mott insulator: Physics and applications, IOP Publishing (2019).
  * [5] Jack Leland Daniel, J. Matolich, H. W. Deem,  Thermal conductivity of UO2, Hanford atomic products operation (1962).
  * [6] J. H. Harding and D. G. Martin, J. Nucl. Mater. 166, 223 (1989).
  * [7] P.G.Lucuta, Hj. Matzke, I.J.Hastings, J. Nucl. Mater. 232, 166 (1996).
  * [8] K. Gofryk et al, Nature Communication 5:4551 (2014).
  * [9] M. Jaime et al, Nature Communication 8: 99 (2017).
  * [10] P. Burlet, J. Rossat-Mignod, S. Quezel, O. Vogt, J.C. Spirlet, and J. Rebizant, J. Less-Common Met. 121, 121 (1986).
  * [11] Ogden G. Brandt and Charles T. Walker, Phys. Rev. Lett 18, 11 (1967).
  * [12] K. Ikushima et al, Phys. Rev. B 63 104404 (2001).
  * [13] H.U. Rahman and W.A. Runciman, J. Phys. Chem. Solids 27, 1833 (1966); 30, 2497 (1969); H.U. Rahman, Physica (Amsterdam) , 511 (1970).
  * [14] S.J. Allen, Phys. Rev. 166, 530 (1968); 167, 492 (1968).
  * [15] S. B. Wilkins, R. Caciuffo, C. Detlefs, J. Rebizant, E. Colineau, F. Wastin, and G. H. Lander, Phys. Rev. B 73, 060406(R) (2006).
  * [16] R. Caciuffo, G. van der Laan, L. Simonelli, T. Vitova, C. Mazzoli, M. A. Denecke, and G. H. Lander, Phys. Rev. B 81, 195104, (2010).
  * [17] Leonid V. Pourovskii and Sergii Khmelevskyi, Phys. Rev. B 99 094439 (2019).
  * [18] J. Faber, G. Lander and B. Cooper, Phys. Rev. Lett. 35, 1770 (1975).
  * [19] J. Faber and G. Lander, Phys. Rev. B 14, 1151 (1976)
  * [20] R. Caciuffo, G. Amoretti, P. Santini, G. H. Lander, J. Kulda, and P. de V. Du Plessis, Phys. Rev. B 59, 13 892 (1999).
  * [21] R. Caciuffo, P. Santini, S. Carretta, G. Amoretti, A. Hiess, N. Magnani, L.-P. Regnault, and G. H. Lander, Phys. Rev. B 84, 104409 (2011).
  * [22] K. Teske, H. Ullmann, and D. Rettig, J. Nucl. Mater. 116, 260 (1983).
  * [23] S. J. Blundell, Magnetism in Condensed Matter (Oxford University Press, 2001).
  * [24] S. B. Roy, G. K. Perkins, M. K. Chattopadhyay, A. K. Nigam, K. J. S. Sokhey, P. Chaddah, A. D. Caplin and L. F. Cohen, Phys. Rev. Lett,. 92 147203 (2004).
  * [25] S. B, Roy, J. Phys.: Condens. Matter 25 183201 (2013).
  * [26] J. A. Mydosh, Rep. Prog. Phys. 78, 052501 (2015).
  * [27] Sudip Pal, Kranti Kumar, A. Banerjee, S. B. Roy and A. K. Nigam, J. Phys.: Condens. Matter 33 (2021) 025801 (2021) and references therein.
  * [28] S.B. Roy, A.K. Pradhan, P. Chaddah and B.R. Coles, Solid St. Commun. 99 563 (1996).
  * [29] S. B. Roy, A.K. Pradhan and P. Chaddah, J. Phys.: Condens. Matter 6 5155 (1994).
  * [30] S.B. Roy, A.K. Pradhan, P. Chaddah and E. V. Sampathkumaran, J. Phys.: Condens. Matter 9 2465 (1997).
  * [31] M. K. Chattopadhyay, S. B. Roy, and P. Chaddah, Phys. Rev. B 72, 180401(R) (2005).
  * [32] A. Banerjee, K. Mukherjee, Kranti Kumar, and P. Chaddah, Phys. Rev. B 74, 224445 (2006).
  * [33] S. B. Roy and M. K. Chattopadhyay, Phys. Rev. B 79, 052407 (2009).
  * [34] P. Chaddah, First Order Phase Transitions of Magnetic Materials (Taylor and Francis, 2017).
  * [35] Kranti Kumar et al, Phys. Rev. B 73 184435 (2006).