(eccv) Package eccv Warning: Package ‘hyperref’ is loaded with option
‘pagebackref’, which is *not* recommended for camera-ready version

11institutetext:  Korea Advanced Institute of Science and Technology (KAIST)
11email: {madin, smwoo95, suminlee94, jinyoungpark, yswang, kdu3613,
<EMAIL_ADDRESS>
# Flow-Assisted Motion Learning Network for Weakly-Supervised Group Activity
Recognition

Muhammad Adi Nugroho  0000-0002-9360-5441 Sangmin Woo  0000-0003-4451-9675
Sumin Lee  Jinyoung Park  Yooseung Wang  Donguk Kim  Changick Kim
0000-0001-9323-8488

###### Abstract

Weakly-Supervised Group Activity Recognition (WSGAR) aims to understand the
activity performed together by a group of individuals with the video-level
label and without actor-level labels. We propose Flow-Assisted Motion Learning
Network (Flaming-Net) for WSGAR, which consists of the motion-aware actor
encoder to extract actor features and the two-pathways relation module to
infer the interaction among actors and their activity. Flaming-Net leverages
an additional optical flow modality in the training stage to enhance its
motion awareness when finding locally active actors. The first pathway of the
relation module, the actor-centric path, initially captures the temporal
dynamics of individual actors and then constructs inter-actor relationships.
In parallel, the group-centric path starts by building spatial connections
between actors within the same timeframe and then captures simultaneous
spatio-temporal dynamics among them. We demonstrate that Flaming-Net achieves
new state-of-the-art WSGAR results on two benchmarks, including a 2.8%p higher
MPCA score on the NBA dataset. Importantly, we use the optical flow modality
only for training and not for inference.

###### Keywords:

Group Activity Recognition Weakly-Supervised Group Activity Recognition
Optical Flow

##  1 Introduction

The Group Activity Recognition (GAR) task involves classifying the collective
activity performed by a group of individuals within a given video. This area
has garnered growing interest owing to its diverse applications, spanning
sports video analysis, video surveillance, and comprehension of social scenes.
GAR introduces new challenges in human action understanding due to its
distinction from the primarily studied individual or paired action recognition
[23, 35, 24]. It involves complex spatio-temporal relationships among multiple
actors in a long temporal sequence. GAR is studied initially in a fully
supervised setting, where individual bounding boxes and actor actions are
available. However, these annotations are costly to acquire, both for the
dataset creation and for the inference stage. This challenge motivates the
research area of Weakly-Supervised Group Activity Recognition (WSGAR), where
only the group activity label is available.

Figure 1:  Importance of motion-awareness. In WSGAR, it is crucial to capture
the movements of key actors. In the second frame, DFWSGAR [19] fails to
recognize the layup action, as the player doing the layup is not strongly
highlighted. In contrast, Flaming-Net strongly highlights the player
performing the layup, thanks to its motion awareness. Flaming-Net leverages
optical flow as a learning guidance motivated by how an area with intense
optical flow indicates the existence of key actors, as seen in the second
frame. Lastly, in the third frame, our method successfully identifies multiple
key actors, such as the celebrating scoring team, the defending team looking
down, and the referee, as strong cues to correctly predict a successful layup
attempt.

WSGAR poses a significant challenge in acquiring the actor features, requiring
the model to learn distinctive cues that help determine the location or
feature that highly correlates with key actors. Flow information is one of the
useful clues that highlight the key actors. In Fig. 1, we show a clipped
sequence of a successful layup activity in a basketball game. The leftmost
column demonstrates how adding flow information strongly accentuates the key
actors, especially the player performing the layup and defending players.
Unfortunately, multiple non-important players are also highlighted. DFWSGAR
[19], a state-of-the-art WSGAR model, more selectively highlights the
important player compared to flow information, reducing needless noises.
However, the attention to players in the under-ring area is also reduced,
causing a loss of detailed information and resulting in an incorrect
prediction. We propose the Flow-Assisted Motion Learning Network (Flaming-Net)
for WSGAR, in which one of the key work principles is to selectively highlight
key actors while also paying high attention to the crucial, highly active
areas. The attention maps of our model, shown in the third column, show how it
focuses on the under-ring area in the second frame, especially on actively
moving players. Additionally, the third frame also shows how the model
highlights supporting key actors, resulting in correct prediction.

Figure 2: Visualization of the motion-aware actor encoder attention maps on
the activity of (i) 3p-success and (ii) 2p-layup-fail.-def. Flaming-Net
highlights the key actors, e.g. 3-point shooter and referee giving hand sign
in the sequence (i), and offensive player doing layup and counter-attacking
defensive players in (ii).

The second key principle of our model is combining powerful cues from a long
temporal dynamic of individual actors with multi-actor interactions in a short
time window when inferring video-level inter-actor relationships. Figure 2
shows a longer clip of (i) 3p-success and (ii) 2p-failed-def. rebound activity
sequences. In Fig. 2-(i), the long-range individual temporal dynamic of the
3p-shooter is crucial to recognize the activity. In the last two frames, the
referee and the ring are also highlighted as the points of focus, due to their
importance in determining whether the 3-point attempt is successful or not. In
Fig. 2-(ii), it is more crucial to determine the interaction among actors in a
short time window, particularly in frames 1 and 2 where the players are
hustling to catch the ball. The progressing sequence also shows how multiple
actors interact together, e.g. players on the same team running to the
opposition side. The spatio-temporal relation module combines the local
dynamics of an actor-centric approach with the comprehensive global motion-
oriented approach to establish spatio-temporal relationships among actors with
parallel aggregation paths.

We evaluate the proposed framework on two datasets, Volleyball [17] and NBA
[40]. Our framework achieves competitive performance with state-of-the-art
methods on these two benchmarks in the weakly supervised learning setting
compared with both fully supervised and weakly supervised models. The
contribution of this paper is threefold:

  * $\bullet$

We propose a novel learning method guided by optical flow frames and an
auxiliary temporal contrastive loss to build motion-awareness when extracting
key actor features without bounding box annotations. Note that optical flow is
not required in the inference stage.

  * $\bullet$

We build a spatio-temporal actor relationship module to infer collective
activity containing two parallel paths: (i) actor motion path that models
activity as an interaction between sequence long individual long-range
temporal dynamic, and (ii) group motion path that models activity as
progressing spatio-temporal connections between one or multiple groups of
actors.

  * $\bullet$

Our method achieves state-of-the-art performance on two benchmark datasets
while maintaining the computational costs and not requiring expensive ground
truth bounding boxes or individual action annotations.

##  2 Related Work

###  2.1 Group Activity Recognition

Group activity recognition requires modeling the relationships between
multiple actors. Typically, individual actor features are extracted from the
image feature map using methods such as RoI-Align [14] or RoI-Pooling [30],
with the help of bounding boxes. Then, the actor features are formed into a
group-level representation by constructing spatio-temporal relationships among
actor features. Earlier methods utilize hand-crafted features with
probabilistic graphical models [2, 7] or AND-OR graphs [1, 32] to model the
connections between individual actions and group activity. Some other works
employ multi-stage RNNs to encode the hierarchical temporal structure of
connections inside the group [4, 17, 8, 31]. Alternatively, the RNN constructs
a graphical structure with actor features as its element [8, 29]. RNNs such as
LSTM capture individual action dynamics and aggregate individual information
to infer group activity.

Recent approaches show trend towards adopting relational modeling [3] to model
relationships between actors, including the widely used graph-based approaches
[11, 16, 36, 42, 39], where actor features are assigned as nodes and their
relationships are represented as edges. Graph-based methods subsequently
employ the way to evolve relation graphs, such as Graph Convolutional Network
(GCN) [36] or Graph Attention Network (GAT) [11]. Some approaches build
hierarchical relationships from body-level representation to group-level
representation by constructing a cross-inference module to embed spatio-
temporal features [39] or by utilizing dynamic relation and dynamic walk
offsets to build person-specific interaction graph [42]. On the other hand,
Azar et al. [3] introduce the notion of activity map to encode spatial
relations between individuals.

Transformer-based methods [12, 21, 6, 34, 37] have emerged as effective
approaches for modeling the relationships between actors. These methods
typically employ a transformer module on top of the actor features, such as
SACRF [28] which embed spatio-temporal relational contexts with conditional
random fields (CRF) or joint spatio-temporal contexts regarding intra- and
inter-group relations[21]. Yuan  _et al_. [41] encode person-specific scene
contexts per individual feature. Tamura  _et al_. [34] combine social group
detection and group activity classification with a region-aware transformer to
identify and aggregate features relevant to social group activities. Dual-AI
[13] construct two complementary spatio-temporal views to infer relations
between actors. Wu  _et al_. [37] use the transformer as a hierarchical
relation inference network to process multi-scale feature maps based on active
spatial positions.

Our work differs from the aforementioned approaches in that they are primarily
designed for the fully supervised setting. In contrast, we focus on the
challenging weakly supervised setting, where obtaining the features of key
actors is more difficult due to the lack of any actor-level annotations.
Despite this difference, we draw inspiration from common practices found in
state-of-the-art fully supervised GAR methods, particularly in using an inter-
actor relation module to build video-level representation and two parallel
aggregation paths.

###  2.2 Weakly-Supervised Group Activity Recognition

The goal of WSGAR is to reduce the labeling cost associated with detailed
actor-level annotations, such as actor bounding boxes and action classes. Yan
_et al_. [40] introduce the NBA dataset for WSGAR task, along with a
specialized WSGAR model called SAM. SAM utilizes an off-the-shelf object
detector to provide bounding box annotations, followed by a relation graph to
prune noisy outputs from the detector. SAM has a drawback of reliance on an
object detector not directly optimized for the GAR problem. To overcome this,
Kim  _et al_. [19] propose a detector-free network based on a transformer
decoder. Similar to DETR [5], they use the transformer to find relevant actors
using learnable queries and a cross-attention decoder. The identified actors
are then transformed into tokens and processed with a temporal convolution
encoder and actor relation self-attention module. Du  _et al_. [9] address the
information loss in the encoding process by using a decoupled 3D convolutional
network intended to better separate camera motion and actor motions. Several
methods employ a pretraining self-supervision strategy, for example, Chappa
_et al_. [6] adopt the video transformer for activity recognition with a self-
supervised pretraining strategy. Con-RPM [43] uses the transformer
architecture for self-supervised GAR by modeling high-level semantic patterns
and learning how to predict future group relations.

Compared to all these works, our method combines the strengths of actor-
focused methods and group motion learning approaches to build relationships
within groups. We leverage strong actor features for robust group relationship
modeling and incorporate group motion learning, where motions are built from
spatio-temporal local connections. We further enhance our model by adding an
extrinsic objective of local motion learning, guided by an additional optical
flow modality. Groupformer [21] also uses flow modality by concatenating the
flow-extracted feature to the actor feature to enhance the actor relation
network. SBGAR [22] combines RGB and flow features for semantic-based action
and activity recognition. However, unlike these methods, we do not use flow in
the inference stage. Instead, we utilize it as guidance in the learning stage,
in a straightforward manner, without the need for an additional extraction
network.

Figure 3:  Overall architecture of Flaming-Net. For each frame, a 2D CNN
backbone generates a feature map and the motion-aware actor encoder extracts
actor features that represent important actors or entities. The sequence of
actor features is then forwarded to two different paths: actor motion path and
group motion path. To improve the learning process, we add local motion
learning with a flow-map-based label to help the model learn local motion.
Additionally, we include a temporal consistency loss and a frame-level
classifier. Note that optical flow is only used in the training stage.

##  3 Flow-Assisted Motion Learning Network

The major challenge of WSGAR is finding the key actors in the video.
Interestingly, they tend to be more active than their surroundings in an
activity sequence, so we can use visual cues associated with motion to find
them. Flaming-Net is built to exploit this knowledge of individual actor
motion and interactive multi-actor motion to infer group activity. The
overview of the network is presented in Fig. 3. The backbone network firstly
extracts RGB frames into feature maps, $\textbf{F}_{0}\in\mathbb{R}^{T\times
H\times W\times C_{0}}$. Then, they are processed frame-per-frame through a
motion-aware actor encoder to extract a set of actor features. We incorporate
the knowledge from the optical flow map into the encoder using a contrastive
loss, along with a temporal consistency objective to enhance the encoder
motion awareness. Following this, the actor features are processed through a
relation module consisting of the actor motion path and the group motion path.

###  3.1 Motion-Aware Actor Encoder

Figure 4:  Motion-aware actor encoder transforms actor queries and the feature
map using a series of multi-head attention modules to generate actor features.
The flow learning loss guides the encoders using the optical flow map to
highlight active key actors. The temporal consistency loss $\ell_{\text{tco}}$
encourages each token to represent the same actor across frames. The loss
attracts temporally adjacent actor features belonging to the same index, while
repels those with different indices. Here, we assume $L=2$.

Motion-aware actor encoder transforms a set of learnable actor queries and the
extracted feature map into actor features frame-per-frame. The encoder
consists of $L$ layer stacks, each containing a multi-head self-attention
(MHSA) and a multi-head cross-attention (MHCA) as visualized in Fig. 4. In the
layer $l$, the MHCA Key and Value are the flattened and compressed feature map
at frame $t$, $F^{t}\in\mathbb{R}^{HW\times C}$, while the Query is the sum of
two elements: (1) the processed actor queries from previous layer
$Z^{l-1}\in\mathbb{R}^{K\times C}$, and (2) the output of the MHSA applied to
$Z^{l-1}$. The encoder attention mechanism localizes and encodes actors from
the given feature maps into tokens. For $T$ frames, the encoder output
$\textbf{W}^{(t)}=\\{\textbf{w}_{i}^{(t)}\\}_{i=1}^{K}$ is stacked to form
$\textbf{W}=[\textbf{W}^{(1)},\textbf{W}^{(2)},\ldots,\textbf{W}^{(T)}]\in\mathbb{R}^{T\times
K\times C}$, the sequence of action features in form of a bag of $T\times K$
token embeddings of dimension $C$. Note that $\textbf{w}_{i}^{(t)}$ denotes
the $i$-th token embedding for the frame $t$.

####  3.1.1 Flow-guided learning.

The actor encoder ideally attends more to the areas with relevant actors. To
achieve this, we introduce an auxiliary learning objective using extra
guidance of optical flow as it can capture strong cues of actor motion in a
frame-per-frame manner. We align the magnitude of optical flow with the
attention map of the encoder block. In an action encoder block, the attention
map value is the average of the softmax result from the query and key
multiplication. Every block generates $K$ attention maps as there are $K$
actor queries. Then, these attention maps are averaged to representative
attention map value to be aligned, $\textrm{att}_{t}(l)$, for encoder block
$l$ on $t$ time frame. However, not all relevant key actors contain strong
flow magnitude, so we only averaged across the first $K_{\text{flm}}$ number
of attention maps and let other attention maps focus on important yet low flow
value areas.

We formulate the flow learning loss ($\mathcal{L}_{\text{flm}}$) to align the
attention and the optical flow maps, m. Firstly, we reshape the attention and
the flow maps into $\mathbb{R}^{NT\times HW}$ vectors. Then, we pair the
attention and optical flow map by assigning maps with the same time frame as
the positive pairs and those with different time frames as the negative pairs.
Then, for each MHCA block, we calculate the contrastive loss and accumulate
them over $L$ stack as expressed in Eq. 3,

$\displaystyle\begin{split}&h(u,v)=\exp((\text{sim}(u,v)/\tau),\>\;&\text{sim}(u,v)=u^{\intercal}v/(\|u\|\|v\|),&\end{split}$
(1) $\displaystyle\ell_{\mbox{\scriptsize
flm}}(\textbf{att}^{(l)},\textbf{m})=-\frac{1}{NT}\sum\limits_{i=1}^{NT}\mathrm{log}\frac{h(\text{att}_{i}^{(l)},\text{m}_{i})}{\sum\limits_{j=1}^{NT}\mathbbm{1}_{i\neq
j}h(\text{att}^{(l)}_{i},\text{m}_{j})},$ (2)
$\displaystyle\mathcal{L}_{\text{flm}}=\frac{1}{L}\sum\limits_{l=1}^{L}\ell_{\mbox{\scriptsize
flm}}(\textbf{att}^{(l)},\textbf{m}),$ (3)

where $l$ is the index of motion-aware encoder block, $\tau$ denotes a
temperature parameter, $N$ denotes the number of data in one batch, and
$\mathbbm{1}_{j\neq i}\in\\{0,1\\}$ is an indicator function evaluating to 1
if $j\neq i$. To generate the flow map, we use FlowNet2 [18] in $H_{0}\times
W_{0}$ resolution, then resize the result to $H\times W$.

####  3.1.2 Temporal consistency.

We add auxiliary loss $\mathcal{L}_{\text{tco}}$ to encourage features with
the same index across time to represent the same actors or have similar
feature values. We achieve this by using bidirectional contrastive loss
between temporally adjacent feature tokens. Tokens with the same index
constitute the positive pairs, and tokens with different indices constitute
the negative pairs.

$\ell_{\mbox{\scriptsize
tco}}(\textbf{W}^{(t)})=-\frac{1}{NK}\sum\limits_{i=1}^{NK}\left(\mathrm{log}\frac{h(\textbf{w}_{i}^{(t)},\textbf{w}_{i}^{(t+1)})}{\sum\limits_{j=1}^{NK}\mathbbm{1}_{j\neq
i}h(\textbf{w}^{(t)}_{i},\textbf{w}^{(t+1)}_{j})}+\\\
\mathrm{log}\frac{h(\textbf{w}_{i}^{(t+1)},\textbf{w}_{i}^{(t)})}{\sum\limits_{j=1}^{NK}\mathbbm{1}_{j\neq
i}h(\textbf{w}^{(t+1)}_{i},\textbf{w}^{(t)}_{j})}\right),$

(4)

$\displaystyle\mathcal{L}_{\text{tco}}=\frac{1}{T-1}\sum\limits_{t=1}^{T-1}\ell_{\mbox{\scriptsize
tco}}(\textbf{W}^{(t)}).$ (5)

Figure 5:  Actor relation module. The actor motion path initially generates
temporally evolved representations of each actor, then builds inter-actor
relations using MHSA. In contrast, the group motion path initially builds
inter-actor relationships using MHSA per frame. Then, these features are
stacked along the temporal dimension and processed spatio-temporally using a
2D convolution stack. We use separate MLP layer classifiers on the paths and
average the outputs of these two classifiers for final prediction.

###  3.2 Actor-centric Motion Path

Within this path, the model first infers dynamic motion for each actor
individually, then builds a group representation by combining them spatially,
as shown visually in the upper part of Fig. 5. In the first step, the action
features
$\textbf{W}_{i}=\\{\textbf{w}_{i}^{(t)}\\}_{t=1}^{T}\in\mathbb{R}^{T\times C}$
from the $i$-th actor at different frames are fused to form an $i$-th
aggregated actor feature $\tilde{\textbf{w}_{i}}\in\mathbb{R}^{D}$ for all
$i$. We use a stack of 1D convolution layers that operate on the temporal
dimension to aggregate them. Specifically, we reshape the output token
embedding W to $K\times T\times C$ tensor, then feed it to the convolution
stack with each layer followed by ReLU[26] operation. The tokens are then
aggregated in the temporal dimension with $AvgPool$ operation, generating a
set of $K$ vectors with dimension $C$ for each video. Then, we construct an
inter-actor relationship using a single-layer multi-head self-attention.
Finally, the actor-path group representation $\textbf{f}_{A}$ is obtained by
applying $AvgPool$ operation over $K$ dimension.

###  3.3 Group-centric Motion Path

The group-centric motion path facilitates interactions of actor features from
different frames, as these represent cues such as multiple actors forming
small groups moving together. The lower part of Fig. 5 illustrates the
process, which begins by constructing spatial actor relationships per frame.
Specifically, the actor relation MHSA, processes action features on time $t$,
$\textbf{W}^{t}$, for each frame. We then reshape the output of this MHSA
across frames into $C\times K\times T$ tensor. After that, we process them
with a 2D convolution stack, where each convolution layer is activated by
ReLU[26], to build spatio-temporal relationships across these features.
Lastly, we perform $AvgPool$ across the spatio-temporal dimension to obtain
the vectorized representation. To improve performance, we add per-frame
classification loss $\mathcal{L}_{\text{gf}}$ by aggregating the token
features per frame and putting an additional classifier after constructing the
spatial relationship with MHSA. We stop the gradient before the 2D convolution
stack to improve learning stability.

###  3.4 Learning Objective and Inference

We use separate MLP classifiers on the actor-centric and group-centric motion
paths. The prediction score is obtained by averaging the outputs of these two
classifiers. The network is trained end-to-end by summing all the losses,
where $\mathcal{L}_{\text{CE}}$ is the cross-entropy loss of the activity
prediction, and $\rho$ is the maximum softmax prediction score for each video
to adjust the influence of the $\mathcal{L}_{\text{flm}}$.

$\mathcal{L}_{\text{sum}}=\mathcal{L}_{\text{CE}}+(1-\rho)\mathcal{L}_{\text{flm}}+\mathcal{L}_{\text{tco}}+\mathcal{L}_{\text{gf}}.$
(6)

##  4 Experiments

###  4.1 Datasets

####  4.1.1 Volleyball dataset.

This dataset [17] consists of 4830 clips taken from 55 videos. 3494 clips
among them are for training and 1337 clips are for testing. The center frame
of each clip is labeled with the activity label and action label along with
the bounding box per player. Bounding box tracklets of players along the 10
frames before and after the center frame are provided by Bagautdinov  _et al_.
[4] and serve as the ground truth. In the WSGAR setting, only the group
activity labels are available for training. We adopted Multi-class
Classification Accuracy (MCA) and Merged MCA for our experiment evaluation.
For computing Merged MCA, we merge the classes right set and right pass into
right pass-set, and left set and left pass into left pass-set as in SAM[40].

####  4.1.2 NBA dataset.

This dataset [40] consists of 7624 clips for training and 1548 clips for
testing. Only 1 out of 9 group activity labels is provided for each clip,
without any individual-level annotation. Since each video clip is 6 seconds
long and usually exhibits a nontrivial temporal structure, the dataset
requires a model that captures long-term temporal dynamics compared with other
GAR benchmarks. Also, it is a challenging benchmark due to fast movement,
camera view change, and a varying number of people in each frame. For
evaluation, we adopt Multi-class Classification Accuracy (MCA) and Mean Per
Class Accuracy (MPCA) metrics; MPCA is adopted due to the class imbalance
issue of the dataset.

###  4.2 Implementation Details

####  4.2.1 Hyperparameters.

We use an ImageNet pretrained ResNet-18 [15] or Inception-v3 [33] with motion
augmentation as used in [19] for the backbone. We set $L=6$ with four number
of attention heads and $C=128$. The temporal convolution in the actor motion
path consists of three 1D convolutional layers with a kernel size of 5 without
padding for the NBA dataset and two 1D convolutional layers with a kernel size
of 3 with zero-padding are used for Volleyball. The spatio-temporal
convolution consists of two 2D convolution layers with a kernel size of
$5\times 3$ for NBA and $3\times 3$ for Volleyball. We use segment-based
sampling [35] to select $T$ frames from each video in the NBA dataset, and
random sampling in the Volleyball dataset. For the NBA dataset, we set $T=18$
and for the Volleyball dataset $T=5$, and we resize each frame to $W_{0}=1280$
and $H_{0}=720$.

####  4.2.2 Training.

Our model is optimized by ADAM[20] with $\beta_{1}=0.9$, $\beta_{2}=0.999$,
and $\epsilon={1}\mathrm{e}{-8}$ for 30 epochs. Weight decay is set to
${1}\mathrm{e}{-4}$ for the NBA dataset. Learning rate is initially set to
${1}\mathrm{e}{-6}$ with linear warmup to ${1}\mathrm{e}{-4}$ for 5 epochs,
and linearly decayed after the $6^{\textrm{th}}$ epoch. We use a mini-batch of
size 4.

Table 1: Comparison with the state-of-the-art GAR models and video backbones
on the NBA[40] dataset. Numbers in bold indicate the best performance and
underlined ones are the second best. ’†’ indicates that the results are from
Kim  _et al_. [19] reproduction, and others are as reported from the original
paper. ${\ast}$ indicates the method requires additional pre-training before a
supervised fine-tuning.

Method | Published on | Backbone | #Params $\downarrow$ | FLOPs $\downarrow$ | MCA $\uparrow$ | MPCA $\uparrow$
---|---|---|---|---|---|---
Video backbone |  |  |  |
† TSM[23] | ICCV’19 | Resnet-18 | 11.2M | 303G | 66.6% | 60.3%
† VideoSwin[24] | CVPR’22 | VideoSwin-T | 27.9M | 478G | 64.3% | 60.6%
Supervised GAR |  |  |  |
† ARG[36] | CVPR’19 | Resnet-18 | 49.5M | 307G | 59.0% | 56.8%
† AT[12] | CVPR’20 | Resnet-18 | 29.6M | 305G | 47.1% | 41.5%
† SACRF[28] | ECCV’20 | Resnet-18 | 53.7M | 339G | 56.3% | 52.8%
† DIN[42] | AAAI’21 | Resnet-18 | 26.0M | 304G | 61.6% | 56.0%
KRG-GAR [27] | TCSVT’23 | Resnet-18 | - | - | 72.4% | 67.1%
Weakly supervised GAR |  |  |  |
SAM [40] | ECCV’20 | Resnet-18 | - | - | 49.1% | 47.5%
† SAM [40] | ECCV’20 | Resnet-18 | 25.5M | 304G | 54.3% | 51.5%
Dual-AI (RGB) [13] | CVPR’22 | Inception-v3 | - | - | 58.1% | 50.2%
DFWSGAR [19] | CVPR’22 | Resnet-18 | 17.5M | 313G | 75.8% | 71.2%
${\ast}$ GSTCo-PCE [10] | TCSVT’23 | Resnet-18 | - | - | 68.6% | 64.2%
LRMM+GCM [9] | ImaVis’23 | Resnet-18 | 14.2M | 306G | 77.8% | 73.2%
${\ast}$ SPARTAN[6] | CVPRW’23 | ViT-Base | - | - | 82.1% | 72.8%
Flaming-Net(Ours) | - | Resnet-18 | 13.8M | 307G | 79.1% | 76.0%

###  4.3 Comparison with State-of-the-art

We compare our method with the state-of-the-art methods in GAR[36, 12, 28, 27]
and WSGAR [40, 19, 9, 6] on the NBA and Volleyball dataset. Table 1 summarizes
the NBA dataset results. NBA dataset does not provide ground truth bounding
boxes, so we use the bounding box proposals provided by SAM [40] to
accommodate fully supervised learning if needed. Only the RGB frames were
available during the inference stage. Among methods that require only one end-
to-end training stage, Flaming-Net performs best with 1.3%p higher MCA and
2.8%p higher MPCA. SPARTAN[6] achieved higher MCA, but it requires additional
training stages due to pretraining and a different backbone of the video
transformer network. We also achieved a higher MPCA of 3.2%p compared to
SPARTAN, which shows the better ability of Flaming-Net to tackle imbalanced
classes. Our model also requires a smaller parameter size with a similar
number of FLOPs compared with the lowest one among the reported models. Our
performance also surpassed the recent video backbones, ResNet-18 TSM [23] and
VideoSwin-T [24], used in conventional action recognition studies. Our model
also performed competitively in the Volleyball dataset as shown in Tab. 2,
surpassing several fully-supervised and all weakly-supervised methods in terms
of the 8-classes MCA. The Volleyball dataset contains fewer long-range
activity sequences compared to the NBA dataset. The higher difference of
Flaming-Net with other methods on the NBA dataset shows that it is better
suited for long-range activity sequences with more complex inter-frame inter-
actor relationships.

Table 2: Comparison with the state-of-the-art GAR models and video backbones
on the Volleyball [17] dataset. Numbers in bold indicate the best performance
among WSGAR. ’†’ indicates that the results are from Kim  _et al_. [19]
reproduction, and others are as reported from the original paper. ${\ast}$
indicates the method requires additional pre-training before a supervised
fine-tuning.

Method | Published on | Backbone | MCA $\uparrow$ | Merged MCA $\uparrow$
---|---|---|---|---
Supervised GAR |  |
† PCTDM[38] | ACMMM’18 | ResNet-18 | 90.3% | 94.3%
† ARG[36] | CVPR’19 | ResNet-18 | 91.1% | 95.1%
† AT[12] | CVPR’20 | ResNet-18 | 90.0% | 94.0%
† SACRF[28] | ECCV’20 | ResNet-18 | 90.7% | 92.7%
† DIN[42] | ICCV’21 | ResNet-18 | 93.1% | 95.6%
GroupFormer [21] | ICCV’21 | Inception-v3 | 94.1% | -
Dual-AI (RGB) [13] | CVPR’22 | Inception-v3 | 94.4% | -
KRG-GAR [27] | TCSVT’23 | Inception-v3 | 95.4% | -
Weakly supervised GAR |  |
SAM [40] | ECCV’20 | ResNet-18 | 86.3% | 93.1%
† SAM [40] | ECCV’20 | Inception-v3 | - | 94.0%
DFWSGAR [19] | CVPR’22 | ResNet-18 | 90.5% | 94.4%
Dual-AI (RGB) [13] | CVPR’22 | Inception-v3 | - | 95.8%
LRMM+GCM [9] | ImaVis’23 | ResNet-18 | 92.8% | 95.6%
${\ast}$ SPARTAN [6] | CVPRW’23 | ViT-Base | 92.9% | 95.6%
Wu  _et al_. [37] | TCSVT’23 | Inception-v3 | 90.2% | 94.9%
Flaming-Net(Ours) | - | ResNet-18 | 91.6% | 94.5%
Flaming-Net(Ours) | - | Inception-v3 | 93.3% | 95.2%

Table 3: Ablation on actor relation module.

Actor relation module | MCA | MPCA
---|---|---
Actor motion path |  74.3% | 73.9%
Actor motion path + Aux. losses |  77.1% | 71.7%
Mirror dual path |  77.0% | 70.7%
Flaming-Net |  79.1% | 76.0%

###  4.4 Ablations

####  4.4.1 Spatio-temporal relationship between actors.

Table 3 shows the result of experimenting methods to construct the spatio-
temporal relationships among the actor features. First, we set the baseline of
using only the actor motion path. In the second row, we show the effectiveness
of our proposed additional losses
($\mathcal{L}_{\text{gf}},\mathcal{L}_{\text{tco}},\mathcal{L}_{\text{flm}}$)
as we obtained an increase in MCA. In the third experiment, we added a dual
path that is set as a mirror of the actor-centric path, i.e. simply reversed
the order of spatial and temporal processing, resulting in no improvement.
Lastly, we experimented with the Flaming-Net strategy that adds the group-
centric path combining actor-relation MHSA and 2D spatio-temporal convolution.
The high performance proves the effectiveness of our strategy in building
complementing actor-motion and group-motion representation.

Table 4: Ablation on the number of $K_{\text{flw}}$.

$K_{\text{flm}}$ |  MCA |  MPCA
---|---|---
4 |  78.2% |  72.6%
6 |  79.1% |  76.0%
9 |  78.2% |  72.9%
12 |  77.2% |  69.9%

Table 5: Ablation on the number of aligned attention map encoder blocks.

$L_{\text{flm}}$ |  MCA |  MPCA
---|---|---
$1^{\text{st}}$ Half |  77.2% |  71.1%
$2^{\text{nd}}$ Half |  77.9% |  71.9%
All |  79.1% |  76.0%

Table 6: Ablation on alignment auxiliary losses.

Loss Formula |  MCA |  MPCA
---|---|---
L1 $\mathcal{L}_{\text{tco}}$ |  76.8% |  72.2%
L1 $\mathcal{L}_{\text{flm}}$ |  76.2% |  73.3%
Contra. |  79.1% |  76.0%

####  4.4.2 Effect of $K_{\text{flw}}$.

We varied the number of actor attention token maps averaged and aligned to the
flow map in Tab. 4. The best result of a balanced number of $K_{\text{flw}}$
shows the need to maintain a balance between using optical flow to find key
actors and using other factors.

####  4.4.3 Multi-level attention map alignment

. In Eq. 3, we align every level of the motion-aware encoder block to the flow
map. For the ablation study in Tab. 5, we varied the level of blocks aligned
with the flow map. There is a slight improvement when aligning only the second
half of blocks compared to only the first half of blocks. We achieved further
improvement once we aligned all the blocks, showing all encoder blocks benefit
from the alignment process.

####  4.4.4 Alignment function.

We use the contrastive loss in Eq. 3 to align the attention map with the
optical flow map and in Eq. 5 to align the temporally adjacent actor tokens
with the same index. We alternatively experimented using L1 loss function to
calculate the $\ell_{\mbox{\scriptsize flm}}$ and $\ell_{\mbox{\scriptsize
tco}}$ value, with results shown in Tab. 6. Changing either of the contrastive
losses to L1 loss results in a lower performance score, thus justifying the
usage of contrastive loss in our formulation.

Table 7: Ablation study on training losses.

No | $\mathcal{L}_{\text{gf}}$ | $\mathcal{L}_{\text{tco}}$ | $\mathcal{L}_{\text{flm}}$ | MCA | MPCA
---|---|---|---|---|---
1 | ✗ | ✗ | ✗ |  75.3% | 70.1%
2 | ✓ | ✗ | ✗ |  78.3% | 72.3%
3 | ✓ | ✓ | ✗ |  77.5% | 73.0%
4 | ✓ | ✗ | ✓ |  77.5% | 73.0%
5 | ✓ | ✓ | ✓ |  79.1% | 76.0%

Table 8: Ablation on detaching loss from spatio-temporal convolution module.

Detach | MCA | MPCA
---|---|---
loss
✗ | 77.3% | 72.5%
✓ | 79.1% | 76.0%

####  4.4.5 Effect of auxiliary losses.

We validated the effect our auxiliary losses in Tab. 7. Firstly, we used only
the activity cross-entropy loss as the learning function, which results in
suboptimal performance. Then, we added the frame-level classifier loss
$\mathcal{L}_{\text{gf}}$ which increases the performance as the group motion
path properly functions. Experiments 3 and 4 show that if the motion-aware
actor encoder auxiliary losses $L_{\text{tco}}$ and $L_{\text{flm}}$ are added
individually, they can be contra-productive. But, when combined as in Exp. 5,
they improved the model performance.

####  4.4.6 Detached loss before spatiotemporal convolution.

We compared the model learning effectiveness between attaching the loss from
the spatio-temporal convolution module in the group motion path and detaching
them. Results in Tab. 8 show better performance by detaching the loss.

Figure 6: t-SNE [25] visualization of feature embedding learned by different
model variants on the NBA [40] dataset. Figure 7: Per-class accuracy on NBA
[40] dataset across model variants.

###  4.5 Qualitative Analysis

Figure 6 displays the t-SNE[25] visualization result, where the final group
representation of each model on NBA is visualized in two-dimensional space, of
4 model variants; (i) model with only the actor motion path, (ii) model with
actor motion path and group motion path along with $\mathcal{L_{\text{gf}}}$,
(iii) model (ii) with additional ${\mathcal{L}_{\text{flm}}}$, and (iv)
complete Flaming-Net model. To help understand the impact of Flaming-Net in
projecting features between activities, we draw a boundary line that divides
the area into 3-point related activities to the right side of the line, and
other activities on the opposite side. As Flaming-Net components are added,
the activities are better separated, i.e. less amount of misclassified
non-3-point activities to the right of the line. Interestingly, it also puts
activities related to successful attempts nearly together. This shows the
model understands the visual context after scoring a point is similar for
these activities.

Figure 8: Token attention visualization. In the first row, we show RGB frames
overlayed with the flow map. In the second row, we overlay the RGB frames with
the encoder attention map. In the third row, we display separately the
attention value of three different tokens, and also draw a circle on actors
related to each token, to show how our encoder uses different tokens for
different actors.

Figure 7 shows the per-class accuracy of Flaming-Net variants. Without
auxiliary losses and flow guidance, the dual path (actor motion path + group
motion path) underperforms the actor-centric path-only variant. Adding the
flow-guided loss and other auxiliary losses enhances the performance,
particularly for the activities related to offensive rebounds and successful
scorings, achieving state-of-the-art performance. Figure 8 displays the
comparison between the flow map and the attention of the actor encoder.
Comparing the flow map (a) with the attention map (b), the model selectively
follows the flow guide if the actor is useful to the activity and tends to
ignore the influence of irrelevant flow. The third row (c) visualizes separate
token attention maps differentiated by colors, and we also circle the actors
represented by the tokens. Our model has diverse attention with each token
representing different relevant actors to the activity.

##  5 Conclusion

We have introduced the Flaming-Net to address the challenge of WSGAR. The
optical flow map is used as learning guidance to enhance the motion awareness
of the actor encoder when extracting key actors. Then, we construct a
comprehensive spatio-temporal dual-path inference module that combines the
strengths of building group representation from relations among multiple
individual actor motions and direct inter-actor inter-frame connections.
Flaming-Net achieves state-of-the-art performance on two GAR datasets.
Ablation studies validate the effectiveness of our learning strategy and the
actor-relation module aggregation.

####  5.0.1 Discussion.

In future work, we aim to alleviate the limitations of our current approach,
such as removing the need for an external optical map generation network and
minimizing the number of network hyperparameters. Besides that, there are
multiple interesting topics to expand our GAR solutions. For example,
increasing efficiency for the feasibility of real-time or online scenarios or
adding an explainable inference to inform the user explicitly on how the
actors interact with each other and their role in the activity.

## References

  * [1] Amer, M.R., Xie, D., Zhao, M., Todorovic, S., Zhu, S.C.: Cost-sensitive top-down/bottom-up inference for multiscale activity recognition. In: Computer Vision - ECCV 2012 - 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part IV. pp. 187–200. Springer (2012). https://doi.org/10.1007/978-3-642-33765-9_14
  * [2] Amer, M.R., Lei, P., Todorovic, S.: Hirf: Hierarchical random field for collective activity recognition in videos. In: Computer Vision - ECCV 2014 - 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VI. pp. 572–585. Springer (2014). https://doi.org/10.1007/978-3-319-10599-4_37
  * [3] Azar, S.M., Atigh, M.G., Nickabadi, A., Alahi, A.: Convolutional relational machine for group activity recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019. pp. 7892–7901. Computer Vision Foundation / IEEE (2019). https://doi.org/10.1109/CVPR.2019.00808
  * [4] Bagautdinov, T., Alahi, A., Fleuret, F., Fua, P., Savarese, S.: Social scene understanding: End-to-end multi-person action localization and collective activity recognition. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (Jul 2017)
  * [5] Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part I. Lecture Notes in Computer Science, vol. 12346, pp. 213–229. Springer (2020). https://doi.org/10.1007/978-3-030-58452-8_13
  * [6] Chappa, N.V.S.R., Nguyen, P.A., Nelson, A.H., Seo, H., Li, X., Dobbs, P.D., Luu, K.: SPARTAN: self-supervised spatiotemporal transformers approach to group activity recognition. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023 - Workshops, Vancouver, BC, Canada, June 17-24, 2023. pp. 5158–5168. IEEE (2023). https://doi.org/10.1109/CVPRW59228.2023.00544
  * [7] Choi, W., Savarese, S.: A unified framework for multi-target tracking and collective activity recognition. In: Computer Vision - ECCV 2012 - 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part IV. Lecture Notes in Computer Science, vol. 7575, pp. 215–230. Springer (2012). https://doi.org/10.1007/978-3-642-33765-9_16
  * [8] Deng, Z., Vahdat, A., Hu, H., Mori, G.: Structure inference machines: Recurrent neural networks for analyzing relations in group activity recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016. pp. 4772–4781. IEEE Computer Society (2016). https://doi.org/10.1109/CVPR.2016.516
  * [9] Du, Z., Wang, X., Wang, Q.: Perceiving local relative motion and global correlations for weakly supervised group activity recognition. Image Vis. Comput. 137, 104789 (2023). https://doi.org/10.1016/J.IMAVIS.2023.104789
  * [10] Du, Z., Wang, X., Wang, Q.: Self-supervised global spatio-temporal interaction pre-training for group activity recognition. IEEE TCSVT 33(9), 5076–5088 (2023). https://doi.org/10.1109/TCSVT.2023.3249906
  * [11] Ehsanpour, M., Abedin, A., Saleh, F.S., Shi, J., Reid, I.D., Rezatofighi, H.: Joint learning of social groups, individuals action and sub-group activities in videos. In: Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part IX. Lecture Notes in Computer Science, vol. 12354, pp. 177–195. Springer (2020). https://doi.org/10.1007/978-3-030-58545-7_11
  * [12] Gavrilyuk, K., Sanford, R., Javan, M., Snoek, C.G.M.: Actor-transformers for group activity recognition. In: CVPR (June 2020)
  * [13] Han, M., Zhang, D.J., Wang, Y., Yan, R., Yao, L., Chang, X., Qiao, Y.: Dual-AI: Dual-path actor interaction learning for group activity recognition. In: CVPR (June 2022)
  * [14] He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 2961–2969. openaccess.thecvf.com (2017)
  * [15] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)
  * [16] Hu, G., Cui, B., He, Y., Yu, S.: Progressive relation learning for group activity recognition. In: CVPR (June 2020)
  * [17] Ibrahim, M.S., Muralidharan, S., Deng, Z., Vahdat, A., Mori, G.: A hierarchical deep temporal model for group activity recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016. pp. 1971–1980. IEEE Computer Society (2016). https://doi.org/10.1109/CVPR.2016.217
  * [18] Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., Brox, T.: Flownet 2.0: Evolution of optical flow estimation with deep networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017. pp. 1647–1655. IEEE Computer Society (2017). https://doi.org/10.1109/CVPR.2017.179
  * [19] Kim, D., Lee, J., Cho, M., Kwak, S.: Detector-Free weakly supervised group activity recognition. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022. pp. 20051–20061. IEEE (2022)
  * [20] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings (2015), http://arxiv.org/abs/1412.6980
  * [21] Li, S., Cao, Q., Liu, L., Yang, K., Liu, S., Hou, J., Yi, S.: Groupformer: Group activity recognition with clustered spatial-temporal transformer. In: 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021. pp. 13648–13657. IEEE (2021). https://doi.org/10.1109/ICCV48922.2021.01341
  * [22] Li, X., Choo Chuah, M.: Sbgar: Semantics based group activity recognition. In: Proceedings of the IEEE international conference on computer vision. pp. 2876–2885 (2017)
  * [23] Lin, J., Gan, C., Han, S.: TSM: temporal shift module for efficient video understanding. In: 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019. pp. 7082–7092. IEEE (2019). https://doi.org/10.1109/ICCV.2019.00718
  * [24] Liu, Z., Ning, J., Cao, Y., Wei, Y., Zhang, Z., Lin, S., Hu, H.: Video swin transformer. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022. pp. 3192–3201. IEEE (2022). https://doi.org/10.1109/CVPR52688.2022.00320
  * [25] Van der Maaten, L., Hinton, G.: Visualizing data using t-sne. Journal of machine learning research 9(11) (2008)
  * [26] Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10), June 21-24, 2010, Haifa, Israel. pp. 807–814. Omnipress (2010), https://icml.cc/Conferences/2010/papers/432.pdf
  * [27] Pei, D., Huang, D., Kong, L., Wang, Y.: Key role guided transformer for group activity recognition. IEEE Transactions on Circuits and Systems for Video Technology pp. 1–1 (2023). https://doi.org/10.1109/TCSVT.2023.3283282
  * [28] Pramono, R.R.A., Chen, Y., Fang, W.: Empowering relational network by self-attention augmented conditional random fields for group activity recognition. In: Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part I. Lecture Notes in Computer Science, vol. 12346, pp. 71–90. Springer (2020). https://doi.org/10.1007/978-3-030-58452-8_5
  * [29] Qi, M., Wang, Y., Qin, J., Li, A., Luo, J., Gool, L.V.: stagnet: An attentive semantic RNN for group activity and individual action recognition. IEEE Trans. Circuits Syst. Video Technol. 30(2), 549–565 (2020). https://doi.org/10.1109/TCSVT.2019.2894161
  * [30] Ren, S., He, K., Girshick, R.B., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada. pp. 91–99 (2015), https://proceedings.neurips.cc/paper/2015/hash/14bfa6bb14875e45bba028a21ed38046-Abstract.html
  * [31] Shu, T., Todorovic, S., Zhu, S.: CERN: confidence-energy recurrent network for group activity recognition. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017. pp. 4255–4263. IEEE Computer Society (2017). https://doi.org/10.1109/CVPR.2017.453
  * [32] Shu, T., Xie, D., Rothrock, B., Todorovic, S., Zhu, S.: Joint inference of groups, events and human roles in aerial videos. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015. pp. 4576–4584. IEEE Computer Society (2015). https://doi.org/10.1109/CVPR.2015.7299088
  * [33] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision (2015)
  * [34] Tamura, M., Vishwakarma, R., Vennelakanti, R.: Hunting group clues with transformers for social group activity recognition. In: Computer Vision - ECCV 2022 - 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part IV. pp. 19–35. Springer (2022). https://doi.org/10.1007/978-3-031-19772-7_2
  * [35] Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., Gool, L.V.: Temporal segment networks: Towards good practices for deep action recognition. In: Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VIII. pp. 20–36. Springer (2016). https://doi.org/10.1007/978-3-319-46484-8_2
  * [36] Wu, J., Wang, L., Wang, L., Guo, J., Wu, G.: Learning actor relation graphs for group activity recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019. pp. 9964–9974. Computer Vision Foundation / IEEE (2019). https://doi.org/10.1109/CVPR.2019.01020
  * [37] Wu, L., Lang, X., Xiang, Y., Chen, C., Li, Z., Wang, Z.: Active spatial positions based hierarchical relation inference for group activity recognition. IEEE TCSVT 33(6), 2839–2851 (2023). https://doi.org/10.1109/TCSVT.2022.3228731
  * [38] Yan, R., Tang, J., Shu, X., Li, Z., Tian, Q.: Participation-contributed temporal dynamic model for group activity recognition. In: Proceedings of the 26th ACM international conference on Multimedia. pp. 1292–1300 (2018)
  * [39] Yan, R., Xie, L., Tang, J., Shu, X., Tian, Q.: Higcin: Hierarchical graph-based cross inference network for group activity recognition. IEEE TPAMI (2020)
  * [40] Yan, R., Xie, L., Tang, J., Shu, X., Tian, Q.: Social adaptive module for weakly-supervised group activity recognition. In: Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part VIII. pp. 208–224. Springer (2020). https://doi.org/10.1007/978-3-030-58598-3_13
  * [41] Yuan, H., Ni, D.: Learning visual context for group activity recognition. In: Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021. pp. 3261–3269. AAAI Press (2021). https://doi.org/10.1609/AAAI.V35I4.16437
  * [42] Yuan, H., Ni, D., Wang, M.: Spatio-temporal dynamic inference network for group activity recognition. In: 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021. pp. 7456–7465. IEEE (2021). https://doi.org/10.1109/ICCV48922.2021.00738
  * [43] Zhou, W., Kong, L., Han, Y., Qin, J., Sun, Z.: Contextualized relation predictive model for self-supervised group activity representation learning. IEEE Transactions on Multimedia (2023)


# RESULTS OF STEREO AND PROSPECT, AND STATUS OF STERILE NEUTRINO SEARCHES

M. LICCIARDI  on behalf of the Stereo collaboration  Univ. Grenoble Alpes,
CNRS, Grenoble INP, LPSC-IN2P3, 38000 Grenoble, France

###### Abstract

Reactor neutrinos have been an intense field of investigation for the last
decade. Two anomalies are discussed in this document. First, a status of the
sterile neutrino searches by Stereo, PROSPECT and DANSS is presented. The
best-fit parameters of active-to-sterile oscillations from the Reactor
Antineutrino Anomaly are strongly rejected by these experiments. Second, the
analyses of the shape anomaly (”5 MeV bump”) by Stereo and PROSPECT, both
using a virtually pure 235U neutrino flux, are detailed. Results show a
significant excess of events at 5-6 MeV and indicate that the bump observed at
commercial reactors is not specific to a particular isotope but rather shared
among U and Pu.

##  1 Introduction

The Reactor Antineutrino Anomaly (RAA) [1] appeared in 2011, following a
revision of the predicted neutrino fluxes for the main isotopes in nuclear
fuel (235U, 238U, 239Pu, 241Pu) [2, 3]. The upward reevaluation of predicted
fluxes resulted in a data-to-prediction deficit of about 5%. One hypothesis to
explain this deficit consists in oscillations to a sterile neutrino state,
since sterile neutrinos are not observable in detectors. The survival
probability writes for $L\lesssim 1$ km:

$P_{ee}=1-\cos^{4}(\theta_{14})\sin^{2}(2\theta_{13})\sin^{2}\left(\frac{\Delta
m^{2}_{31}L}{4E}\right)-\sin^{2}(2\theta_{14})\sin^{2}\left(\frac{\Delta
m^{2}_{14}L}{4E}\right)$ (1)

with $E$ the neutrino energy, $L$ the distance from source to detector,
$\theta_{13}$ and $\Delta m^{2}_{13}$ the mixing angle and mass splitting in
the 3-neutrino paradigm, $\theta_{\mathrm{14}}$ and $\Delta
m^{2}_{\mathrm{14}}$ the parameters ruling the oscillation to the sterile
state. Based on data available at the time, the RAA best-fit parameters are of
the order of $\sin^{2}2\theta_{\mathrm{new}}\sim 0.1$ and $\Delta
m^{2}_{\mathrm{new}}\sim 2\leavevmode\nobreak\ \mathrm{eV}^{2}$. Notably, this
mass splitting translates into an oscillation length from 2 to 10 meters
(depending on the neutrino energy), which is the typical length on which
oscillations would develop. Therefore, a new generation of neutrino detectors
were designed to study such oscillations, combining two key requirements: (i)
a distance from detector to reactor core of about 10 m, to be able to probe
the RAA hypothesis; (ii) a segmented detector covering several baselines over
$\sim 2$ m, so that oscillations could develop inside the detector and be seen
by comparing spectra from the detector’s subparts. Experiments operating such
detectors are, for instance, Stereo, PROSPECT and DANSS.

Another intriguing aspect of the reactor antineutrino spectrum is the excess
of events observed by experiments running at commercial reactors (Daya Bay
[4], RENO [5], Double Chooz [6]) around 5 MeV, so-called ”5 MeV bump”. In
order to disentangle the contribution from the several isotopes present in
low-enriched nuclear fuel, pure 235U measurements from research reactors are
helpful. Stereo and PROSPECT have thus performed analyses of the spectrum
shape, and have formed a joint collaboration to increase the sensitivity of
their respective analyses.

##  2 Experimental context

###  2.1 Stereo, PROSPECT and DANSS experiments

Three experiments will be presented in this document: i) Stereo [7], operated
near the High Flux Reactor at Institut Laue-Langevin in Grenoble, France; ii)
PROSPECT [8], operated near the High Flux Isotope Reactor at the Oak Ridge
National Laboratory, USA; iii) DANSS [9], operated below a commercial reactor
core at the Kalininskaya nuclear power plant in Russia. Tab. 1 compares some
important instrumental characteristics of these three experiments.

Table 1: Comparison of key instrumental aspects of the Stereo, PROSPECT and DANSS experiments. | Stereo | PROSPECT | DANSS
---|---|---|---
Reactor power | 58 MWth | 85 MWth | 3 GWth
235U enrichement | 93 % (HEU) | 93 % (HEU) | Low (LEU)
$\overline{\nu}_{e}$ spectrum |  $>99\%$ from 235U |  $>99\%$ from 235U | Mixed isotopes
Core size |  $h=80$cm, $\diameter=40$cm |  $h=50$cm, $\diameter=45$cm |  $h=3.5$m, $\diameter=3.1$m
Detector structure | Segmented (6 cells) | Segmented | On a movable platform
Baselines | 9.4 to 11.2 m | 6.7 to 9.3 m | 10.9, 11.9 and 12.9 m
Detection principle | Gd-doped liq. scint. |  6Li-doped liq. scint. | Gd-coated plastic scint.
$\overline{\nu}_{e}$ rate | 360 / day | 500 / day | 5 000 / day
S/B | $\simeq 0.8$ | $\simeq 1.4$ | $\sim 20$

###  2.2 Analysis key points: Stereo as an illustration

Let us now illustrate some key aspects of the analysis using Stereo as an
example. Reactor antineutrinos are detected through the inverse beta decay
(IBD) reaction: $\overline{\nu}_{e}+p\rightarrow e^{+}+n$. This produces two
coincident signals in the detector: i) a prompt signal from the $e^{+}$
kinetic energy and annihilation, and ii) a delayed signal from the neutron
capture on a Gd nucleus.

The prompt signal is related to the neutrino energy as
$E_{\mathrm{pr}}=E_{\nu}-0.782$ MeV. An accurate energy reconstruction is
crucial for the oscillation analysis since they develop as a function of
$L/E$, and for the spectral analysis as well. Most importantly the
reconstructed energy has to be well reproduced in simulation: the quantity of
interest is the energy scale, namely the data/MC ratio of reconstructed
energies $E_{\mathrm{rec}}^{\mathrm{data}}/E_{\mathrm{rec}}^{\mathrm{MC}}$.
The control of the energy scale is done in 3 steps. First, a set of
calibration sources are deployed in each cell, at 5 different heights, and
cover energies from 0.5 MeV to $\simeq$8 MeV. Second, the $\beta$-decay of
cosmic-muon-induced 12B provides a continuous spectrum up to $Q_{\beta}=13.4$
MeV. Finally, a joint fit of the data/MC residuals from sources and boron
constrains the ratio
$E_{\mathrm{rec}}^{\mathrm{data}}/E_{\mathrm{rec}}^{\mathrm{MC}}$ to be in
agreement with 1 within $\pm 1\%$ accuracy across all energies.

The delayed signal originates from the de-excitation cascade of the excited Gd
nucleus after $n$ capture. The computation of the capture efficiency is of
prime importance for the absolute measurement of the IBD rate which provides a
measurement of the deficit with respect to the reference Huber-Mueller (HM)
model [3, 2]. It is done with an AmBe source circulated in and around the
detector, systematic uncertainties being evaluated with data/MC comparison.
For this, the FIFRELIN software provides improved modelling of the de-
excitation cascades [10], the result of which is made available to the
community. In addition, correction factors acount for spatial data/MC
discrepancies that could be related to neutron physics at the edges of the
detector, or Gd non-uniformity. This careful evaluation allows Stereo to
measure the rate deficit of the 235U spectrum as

$(5.2\pm 0.8[\mathrm{stat}]\pm 2.3[\mathrm{syst}]\pm 2.3[\mathrm{model}])\%$
(2)

which is the most accurate result for HEU experiments [11].

A final key point in the analysis comes from the reactor’s alternation of
reactor-on and reactor-off periods. Cosmogenic background is measured in
reactor-off periods and used as a model for such a background in the reactor-
on collected data. Pulse Shape Discrimination then allows to keep a signal-to-
background ratio of order 1, leading to the sterile neutrino search and the
spectral analysis presented in Section 3 and 4. Note that the following
results use only the first 2 phases of Stereo’s collected data, with a
doubling of statistics yet to come.

##  3 Sterile neutrino searches

Stereo, PROSPECT and DANSS use different analysis methods, but all rely on the
same principle: looking for oscillations by comparing spectra at different
baselines (and not comparing the data to a model). Analyses can then be model-
independent and provide more reliable results.

In order to do so, Stereo introduces free $\phi_{j}$ parameters in its
$\chi^{2}$ minimization [12]:

$\chi^{2}=\sum_{l=1}^{N_{\mathrm{cells}}}\sum_{j=1}^{N_{\mathrm{Ebins}}}\left(\frac{A_{l,j}-\phi_{j}M_{l,j}}{\sigma_{j}}\right)^{2}+\mathrm{pull\leavevmode\nobreak\
terms}$ (3)

with $A_{l,j}$ the measured neutrino rate in cell $l$ and energy bin $j$, and
$M_{l,j}=M_{l,j}(\sin\theta_{14},\Delta m_{14}^{2};\boldsymbol{\alpha})$ the
predicted rate as a function of the active-to-sterile oscillation parameters
from Eq. 1 and nuisance parameters $\boldsymbol{\alpha}$. Any bias in the
initial model in energy bin $j$ would be absorbed by the $\phi_{j}$ in this
bin: then, only cell-to-cell differences in the oscillated prediction
$M_{l,j}$ remain relevant.

A similar method is pursued by PROSPECT [13]. Specifically, the $\phi_{j}$ are
not free parameters anymore but are set to

$\phi_{j}=\sum_{l=1}^{N_{\mathrm{baselines}}}A_{l,j}\,/\sum_{l=1}^{N_{\mathrm{baselines}}}M_{l,j}$
(4)

using the ratio of the baseline-summed measured and predicted spectra.

Finally, the analysis of the DANSS collaboration uses the ratio $R_{j}$ of
spectra between bottom ($L=12.9$ m) and top ($L=10.9$ m) positions [15]. The
$\chi^{2}$ then writes

$\chi^{2}=\sum_{j=1}^{N_{\mathrm{Ebins}}}\left(\frac{R^{\mathrm{obs}}_{j}-k\times
R^{\mathrm{pred}}_{j}}{\sigma_{j}}\right)^{2}$ (5)

with $R^{\mathrm{obs}}_{j}$ and $R^{\mathrm{pred}}_{j}$ the observed and
predicted ratios, and $k$ a normalization parameter to consider only
differences in positron energy shapes.

Figure 1: Exclusion and sensitivity contours from (top left) Stereo [12], (top
right) PROSPECT [13] and (bottom) DANSS [14], showing strong rejection of the
RAA best-fit point as well as a large surrounding region.

The exclusion contours from the 3 experiments are gathered in Figure 1, using
either the CLs (PROSPECT, DANSS) or the usual $\Delta\chi^{2}$ method (Stereo,
PROSPECT). Importantly for the latter, since $\Delta\chi^{2}$ does not follow
a standard $\chi^{2}$ law with 2 d.o.f., the Feldman-Cousins procedure
(buiding $\Delta\chi^{2}$ p.d.f.s via MC toys) is used. The best-fit point of
the RAA is strongly excluded: at $>99\%$ C.L. by Stereo, at $>95\%$ C.L. by
PROSPECT, and at $>5\sigma$ by DANSS. A large portion of the RAA 95% C.L.
region is excluded as well. The only remaining region of the parameter space
still unrejected corresponds to $\Delta m^{2}\gtrsim 5\leavevmode\nobreak\
\mathrm{eV}^{2}$ where experiments at $O(10\leavevmode\nobreak\ \mathrm{m})$
have little sensitivity.

In this region of high-frequency oscillations, only an averaged effect could
be observable in detectors, and no cell-to-cell distortions are to be
expected. Oscillations to a sterile state would only appear as a deficit to a
certain model, and are thus beyond the model-independent searches discussed
here. One would need, for instance, to add a model-dependent constraint on the
normalization.

##  4 Spectral analysis

For the experiments at HEU reactors, the spectral analysis has been developped
for two main purposes: i) to investigate the 235U contribution to the event
excess around 5 MeV, and ii) to provide an experimental reference to the
community, since the current prediction fails to reproduce the experimental
data in this specific region. Therefore, the measured energy spectra have been
unfolded and expressed as a function of the antineutrino energy.

###  4.1 Analysis of Stereo data

The unknown unfolded spectrum is parametrized by a set of parameters
$\Phi_{i}$ (one per $E_{\nu}$ bin $i$) and is folded into measured-energy
space through the experimental response matrix $R$. There, it is fitted
against the measured spectrum $D$, using one of the following two methods:

$\chi^{2}(\Phi)=\big{(}R\Phi-D\big{)}^{T}V_{D}^{-1}\big{(}R\Phi-D\big{)}+r\cdot\mathcal{R}_{1}(\Phi)$
(6)

or

$\chi^{2}(\Phi)=\big{(}R(\vec{\alpha})\Phi-D\big{)}^{T}V_{\mathrm{stat}}^{-1}\big{(}R(\vec{\alpha})\Phi-D\big{)}+|\vec{\alpha}|^{2}+r\cdot\mathcal{R}_{1}(\Phi).$
(7)

The first method uses the total covariance matrix $V_{D}$ in the $\chi^{2}$
computation, while systematic uncertainties are implemented using nuisance
parameters in the second. As statistical noise is enhanced by unfolding
processes, both uses a Tikhonov-like regularization term

$\mathcal{R}_{1}(\Phi)=\sum_{i}\left(\frac{\Phi_{i+1}}{\Phi_{i+1}^{0}}-\frac{\Phi_{i}}{\Phi_{i}^{0}}\right)^{2}$
(8)

to smooth the first derivative of the deviation between the unfolded spectrum
and a prior shape $\Phi^{0}$ (set to the shape of the HM model [3, 2]). The
strength of this smoothing can be varied through the tunable parameter $r$. In
the first method, $r$ is set following the General Cross-Validation
prescription [16]; in the second method, $r$ is empirically chosen to ensure
that the choice of the prior shape has negligible impact on the unfolded
spectrum [17]. The compatibility between the two methods has been validated on
a pseudo-data set, and excellent agreement was found. Finally, biases have
been studied on various toy models (including models with a bump) and show
that the unfolding is well controlled, as biases remain $<1\%$ across all the
energy range.

Figure 2: (Top) Unfolded 235U spectrum along with area-normalized Huber-
Mueller (HM) and Summation Model (SM) [18] predictions. The non-trivial
correlation matrix is displayed. (Bottom) Ratios to HM prediction. Adapted
from Ref. [17].

The unfolded spectrum is shown in Fig. 2. The agreement with the area-
normalized HM model is $\chi^{2}/\mathrm{ndf}=26.7/18$ (p-value: 0.084). When
fitted by a gaussian, the excess of events around 5-5.5 MeV gives an amplitude
of $(12.1\pm 3.4)\%$. The significance of the bump observation in a 235U
spectrum by Stereo reaches thus 3.5$\sigma$. However, no preference can be
inferred about the isotopic origin of the bump observed by Daya Bay [4] with
commercial reactors. A bump shared among isotopes (resp. a pure-235U bump)
would correspond to an amplitude of $\sim 9\%$ (resp. $\sim 16\%$) in Stereo
data.

###  4.2 Analysis of PROSPECT data

The investigation for the bump is done on the measured-energy spectrum by
PROSPECT. The analysis focuses on the isotopic origin of the bump observed by
Daya Bay [4]. A gaussian with mean 5.678 MeV and width 0.562 MeV
(corresponding to Daya Bay’s observation) is folded through the response
matrix, and its amplitude is fitted againt PROSPECT data [13].

PROSPECT spectrum and the best-fit excess are displayed on Fig. 3. The scale
factor, ratio of PROSPECT’s to Daya Bay’s amplitudes, has a best-fit value of
$0.84\pm 0.39$. It is consistent with 1, indicating that 235U does not play a
specific role on the bump origin with respect to the other isotopes present in
LEU neutrino spectra. The no-235U bump and all-235U bump hypotheses,
corresponding to scale factors of 0 and 1.78 respectively, are both
disfavoured at more than $2\sigma$.

Figure 3:  Measured-energy spectrum observed by the PROSPECT experiment [13].
The event excess is fitted by a gaussian with mean and width set to correspond
to Daya Bay’s observation.

###  4.3 Joint analysis by Stereo and PROSPECT

As both Stereo and PROSPECT spectral analyses are statistically limited, both
collaborations have worked together on a joint analysis to increase the
physics reach of the experiments. Stereo’s unfolding methods have been
extended to perform the joint unfolding of the data sets; for instance, the
first method now writes

$\displaystyle\chi^{2}(\Phi)$ $\displaystyle=\big{(}R_{\mathrm{ST}}\Phi-
D_{\mathrm{ST}}\big{)}^{T}V_{\mathrm{ST}}^{-1}\big{(}R_{\mathrm{ST}}\Phi-
D_{\mathrm{ST}}\big{)}$ $\displaystyle+\big{(}\alpha\cdot R_{\mathrm{PR}}\Phi-
D_{\mathrm{PR}}\big{)}^{T}V_{\mathrm{PR}}^{-1}\big{(}\alpha\cdot
R_{\mathrm{PR}}\Phi-D_{\mathrm{PR}}\big{)}+r\cdot\mathcal{R}_{1}(\Phi)$ (9)

where PR and ST are used to refer to each experiment, and $\alpha$ is a free-
floating scaling parameter accounting for the different normalizations of the
experiments.

Another unfolding framework has been developped within the PROSPECT
collaboration, based on the Wiener-SVD method [19]. The unfolding is done by
using the Singular Value Decomposition to invert the response matrix. A
filter, optimized according to the signal-to-background ratio, is applied in
this inversion process to mitigate the noise amplification due to small
diagonal elements in the SVD decomposition.

Figure 4: Joint fit of Stereo and PROSPECT data, along with the unfolded
spectra of each experiments (all unfolding performed with the method of Eqn.
9).

Stereo’s and PROSPECT’s unfolding methods have been compared on the single
unfolding of Stereo or PROSPECT data, and show excellent agreement. Moreover,
a $\chi^{2}$ test between the spectra from the two experiments (unfolded with
the method of Eqn. 9) gives $\chi^{2}/\mathrm{ndf}=22.3/17$ (p-value: 0.17).
Since the two experiments are statistically compatible, it is fully relevant
to attempt a joint fit combining both of them. A preliminary $E_{\nu}$
spectrum from the joint unfolding of Stereo and PROSPECT data is shown in Fig.
4. As suggested in Ref. [19], a filter matrix encoding all unfolding biases
will be provided with the unfolded spectrum in the forthcoming publication.

This 235U spectrum extracted by Stereo and PROSPECT, from HEU reactor cores,
is complementary to results from LEU experiments. While commercial reactors
provide more intense neutrino fluxes, our analysis is completely independent
on flux models related to other main fuel isotopes. In the near future, one
could even imagine to use this 235U spectrum as an external constraint in
analyses aiming to separate U and Pu contributions in LEU spectra [20].

##  5 Summary

Since the emergence of the RAA, an intense experimental effort has developped
around very-short-baseline reactor neutrino detectors, in order to search for
active-to-sterile oscillations. A decade later, the best-fit parameters and a
large portion of the allowed region in parameter space are ruled out by
Stereo, PROSPECT or DANSS. With these segmented detectors, model-independent
analyses have been performed by comparing the antineutrino spectra from
several baselines, and no sign of oscillations have been found.

Another explanation is then required to understand the data-to-prediction
deficit of about 5%, which first suggested the hypothesis of a sterile
neutrino. The contributions of 235U and 239Pu to this deficit have been
separated by the Daya Bay collaboration [21] and favor that the deficit is
mostly carried by 235U. The 235U deficit is measured at $(7.8\pm 2.7)\%$. The
measurement by Stereo on a pure 235U flux yields a compatible deficit of
$(5.2\pm 2.4)\%$, with a pure-235U world-average now at $(5.0\pm 1.3)\%$ [11].
Finally, a recent repetition by Kopeikin et al. of the measurement of 235U and
239Pu $\beta$ spectra, used as inputs for the Huber-Mueller model, indicates
that the ratio of 235U/239Pu fluxes may have been overestimated by 5.4% [22].
The global picture is illustrated in Fig. 5: it suggests that the 239Pu
normalization may be correct, and the 235U normalization overestimated in the
HM model by 5-6%.

Figure 5: Global picture of the 235U and 239Pu cross-section per fission. The
black ellipse corresponds to the HM model; the green contours are from the
U/Pu separation from Daya Bay [21] ; the purple band reprensents the world-
average for pure-235U experiments [11]; the orange dashed line corresponds to
Kopeikin’s new measurement of the ratio of $\beta$ spectra 235U / 239Pu [22].

Concerning the shape anomaly, the complementarity of HEU and LEU experiments
is once again an asset for the community. First revealed by reactor neutrinos
experiments at commercial reactors, the ”5 MeV bump” has been studied by
Stereo and PROSPECT on pure-235U fluxes. Combining all analyses, there is a
small preference for a bump equally shared by 235U and 239Pu. As the modelling
of these neutrino fluxes has proven to be a difficult task, Stereo and
PROSPECT provide a experimental reference for the 235U spectrum. Thanks to
well controlled unfolding methods, it is expressed in antineutrino energy.

Finally, let us note that the accuracy on all these observables will increase
as more data is currently taken and/or analyzed by Stereo, PROSPECT and DANSS.

## References

## References

  * [1] G. Mention et al, Phys. Rev. D 83, 073006 (2011).
  * [2] Th.A. Mueller et al, Phys. Rev. C 83, 054615 (2011).
  * [3] P. Huber, Phys. Rev. C 84, 024617 (2011).
  * [4] F.P. An et al, Chin. Phys. C 41, 013002 (2017).
  * [5] Z. Atif et al, arXiv:2010.14989 (2020).
  * [6] H. de Kerret et al, Nature Physics 16, 558-564 (2020).
  * [7] N. Allemandou et al, JINST 13, P07009 (2018).
  * [8] J. Ashenfelter et al, Nucl. Instrum. Methods A 922, 287-309 (2019).
  * [9] I. Alekseev et al, JINST 11, P11011 (2016).
  * [10] H. Almazán et al, Eur. Phys. J. A 55, 183 (2019).
  * [11] H. Almazán et al, Phys. Rev. Lett. 125, 201801 (2020).
  * [12] H. Almazán et al, Phys. Rev. D 102, 052002 (2020).
  * [13] M. Andriamirado et al, Phys. Rev. D 103, 032001 (2021).
  * [14] M. Danilov, arXiv : 2012.10255 (2020).
  * [15] N. Skrobova, J. Phys. Conf. Ser. 1390, 012072 (2019).
  * [16] G.H. Golub, M. Heath and G Wahba, Technometrics 21:2, 215-223 (1979).
  * [17] H. Almazán et al, J. Phys. G: Nucl. Part. Phys. 48, 075107 (2021).
  * [18] M. Estienne et al, Phys. Rev. Lett. 123, 022502 (2019).
  * [19] W. Tang et al, JINST 12, P10002 (2017).
  * [20] F.P. An et al, arXiv : 2102.04614 (2021).
  * [21] F.P. An et al, Phys. Rev. Lett. 118, 251801 (2017).
  * [22] V. Kopeikin, M. Skorokhvatov and O. Titov, arXiv : 2103.01684 (2021).