# Bullying10K: A Neuromorphic Dataset towards Privacy-Preserving Bullying
Recognition

Yiting Dong1,3, Yang Li2,3∗, Dongcheng Zhao3∗, Guobin Shen1,3, Yi Zeng1,2,3
1 School of Future Technology, University of Chinese Academy of Sciences
2 School of Artificial Intelligence, University of Chinese Academy of Sciences
3 Brain-inspired Cognitive Intelligence Lab,
Institute of Automation, Chinese Academy of Sciences
Equally contributionCorresponding author<EMAIL_ADDRESS>
###### Abstract

The prevalence of violence in daily life poses significant threats to
individuals’ physical and mental well-being. Using surveillance cameras in
public spaces has proven effective in proactively deterring and preventing
such incidents. However, concerns regarding privacy invasion have emerged due
to their widespread deployment. To address the problem, we leverage Dynamic
Vision Sensors (DVS) cameras to detect violent incidents and preserve privacy
since it captures pixel brightness variations instead of static imagery. We
introduce the Bullying10K dataset, encompassing various actions, complex
movements, and occlusions from real-life scenarios. It provides three
benchmarks for evaluating different tasks: action recognition, temporal action
localization, and pose estimation. With 10,000 event segments, totaling 12
billion events and 255 GB of data, Bullying10K contributes significantly by
balancing violence detection and personal privacy persevering. And it also
poses a challenge to the neuromorphic dataset. It will serve as a valuable
resource for training and developing privacy-protecting video systems. The
Bullying10K opens new possibilities for innovative approaches in these
domains.††project website
https://figshare.com/articles/dataset/Bullying10k/19160663

##  1 Introduction

The issue of violence in daily life poses a significant threat to individuals’
physical and mental well-being. In addition to merely punishing for violent
actions, it is crucial to deter and prevent their occurrence proactively.
Implementing surveillance cameras in public spaces has effectively facilitated
the prompt detection of emerging violent behavior. While this strategy has
somewhat curbed violent incidents, the widespread deployment of these cameras
stirs up concerns over potential invasions of individuals’ privacy, leading to
significant apprehensions.

The proliferation of cameras has dramatically enhanced the ease of data
collection. Cameras are commonly employed for indoor and outdoor surveillance,
capturing instances of violence or emergencies [1, 2, 3]. Nonetheless, this
data-gathering method frequently requires obtaining explicit consent from
recorded participants. Obtaining comprehensive consent from individuals
captured on camera poses significant challenges [1, 4]. In addition to
capturing movement data, personal information related to privacy, such as
facial features and attire, is recorded and potentially stored on untrusted
third-party servers with high-performance capabilities, thereby intensifying
the potential for privacy breaches. At present, most of the commonly employed
violence detection datasets primarily utilize RGB images. We aim to devise a
strategy that effectively identifies unusual and violent incidents while
minimizing the risk of privacy breaches during normal circumstances.

Figure 1: Visualization of the Bullying10K dataset. For each example, the
right section illustrates the stream of events captured by a Dynamic Visual
Sensor (DVS) camera, showcasing the dynamic changes in brightness at each
pixel. The left section demonstrates the related event frame transformed from
the event stream and corresponding human pose keypoints labels. This enables
us to observe specific actions and scenes.

Dynamic Vision Sensors (DVS) cameras [5], which capture pixel brightness
variations, provide an innovative alternative to conventional cameras that
produce image frames at fixed frequencies. Instead, DVS cameras generate an
event stream that records each pixel’s brightness changes, either enhancement
or reduction. As a result, it becomes challenging to visually identify the
captured objects. Although some techniques attempt to reconstruct images from
DVS data [6, 7], these approaches grapple with issues such as low contrast and
blurriness and may even require additional sensor information for assistance
[7]. Consequently, extracting detailed user information suitable for
recognition systems from DVS cameras becomes a significant challenge,
naturally reinforcing privacy persevering measures. At the same time, the high
sensitivity of DVS cameras ensures their stable performance under uncontrolled
luminary conditions and diverse environmental states [8]. As an event-driven
camera, DVS consumes low power when the scene is static, reducing energy
consumption and mitigating information redundancy compared to traditional
cameras. However, although datasets captured using DVS cameras exist [9, 10,
11], most are employed for traditional image classification tasks. Existing
action recognition datasets [12, 13] primarily focus on generic simple action
recognition with limited scale and simplistic labels. Thus, they are
insufficient for detecting complex and rapid actions and overlapping
individuals, characteristic of violent incidents.

To address these concerns, we leverage the unique characteristics of DVS
cameras and propose an event-based dataset called Bullying10K. The dataset
aims to detect violent incidents in videos while ensuring privacy protection.
Instead of relying on conversion algorithms or reproduction methods that can
be time-saving and resource-saving, we chose to capture real-life scenarios
and subjects using DVS cameras. This approach allows us to avoid data biases
that may arise from the process of RGB cameras of original datasets. The
dataset captures subjects engaging in various actions under different views
and lighting conditions. In addition to data privacy persevering, the
Bullying10K dataset stands out from other DVS datasets by encompassing more
complex and rapid actions and instances where individuals may obscure each
other. This inclusion introduces new challenges to event-based neuromorphic
datasets.

In conclusion, the design of the Bullying10K dataset aims to fulfill the real-
time detection requirements of violent behavior while maximizing the privacy
protection of the individuals captured in the footage. This dataset will serve
as valuable training data for developing privacy-preserving video systems,
providing new insights and opportunities for future research.

Our contributions are as follows:

  1. 1.

We propose a large-scale DVS violence detection dataset: Bullying10K. It
contains 10,000 event segments, totaling 12 billion events and 255 GB of data.
The actions in the videos are characterized by their complexity, rapidity, and
occlusion of individuals.

  2. 2.

We provide three benchmarks for comparing the performance of different
methods: an action recognition benchmark, an action temporal localization
benchmark, and a pose estimation benchmark. For the pose estimation task, we
provide the keypoints of human pose.

  3. 3.

We present the DVS community with a trainable dataset to detect violent scenes
without compromising privacy. It makes the anticipation and research of
violent scenarios possible.

##  2 Related Work

Table 1: Comparison of different neuromorphic datasets.

Dataset | #Year | #Sensor | #Type | #Object | #Sec Per Example | #pose | #Event Count
---|---|---|---|---|---|---|---
ASLAN-DVS [14] | 2019 | Davis240c | reproduced | action | - | No | -
N-MNIST [10] | 2015 | ATIS | reproduced | digit images | 0.3s | - | 300M/4K
N-CALTECH101 [10] | 2015 | ATIS | reproduced | images | 0.3s | - | 1B/0.1M
DVS-CIFAR10 [9] | 2017 | Davis128 | reproduced | images | 1.2s | - | 2B/0.2M
HMDB-DVS [14, 15] | 2019 | Davis240c | reproduced | action | 19s | No | 3B/0.5M
ES-ImageNet [16] | 2021 | - | conversion | images | - | - | -
UCF-DVS [14, 17] | 2019 | Davis240c | reproduced | action | 25s | No | 12B/0.9M
N-Omniglot [11] | 2022 | Davis346 | reproduced | char images | - | - | -
DVS-Gesture [13] | 2017 | Davis128 | real | action | 6s | No | 500M/0.4M
NCARS [18] | 2018 | ATIS | real | cars | 0.1s | - | 95M/4K
ASL-DVS [19] | 2020 | Davis240 | real | hand | 0.1s | - | 2B/21K
PAF [12] | 2019 | Davis346 | real | action | 5s | No | -
DailyAction [20] | 2021 | Davis346 | real | action | 5s | No | -
Bullying10K | 2023 | Davis346 | real | action | 2-20s | Yes | 12B/1.2M

Figure 2: The flow of the data acquisition process. We shot with two DVS
cameras, and we will get the event stream for post-processing and apply it to
three tasks.

##### DVS Dataset

Early DVS datasets were typically derived from pre-existing image
classification datasets [9, 10]. They captured the brightness differences of
pixels caused by camera or image motion using DVS cameras. However, generating
meaningful temporal data from static images proved challenging. [13, 12]
captured people in real scenarios, showcasing different actions through hand
movements and providing early event-based classification tasks benchmarks.
However, these datasets were relatively small, and the actions displayed were
somewhat repetitive. In contrast to traditional classification tasks, [11]
introduced a dataset for few-shot tasks, reconstructing the drawing process of
character strokes and creating meaningful temporal data. [14] attempted to
capture existing action recognition datasets using DVS cameras. However, video
reproduction failed to capture the event characteristics in natural scenes,
especially motion blur caused by high-speed motion or significant changes in
illumination conditions. Constructing a dataset suitable for violence
detection requires capturing data with complex actions, fast movement, and
occlusion, which existing datasets are not explicitly designed for. In Table
1, we provide an overview of several DVS datasets, where #Event Count means
the total number of events of the dataset and the average number of each
example.

##### Violent Dataset

The construction of appropriate datasets for detecting violent actions are
crucial to promote research in automated detection technology. [21] proposed a
dataset created by extracting clips from short films comprising only 200 video
segments. [22] collected 1,000 data samples by capturing snippets from hockey
games. [23] gathered realistic scenes involving multiple groups engaged in
specific actions. [24] compiled the RWF-2000 dataset by amassing 2,000 sample
clips from the internet. [25] extracted segments from Hollywood movies to
create a dataset.

Spiking Neural Networks (SNNs) are models that simulate the behavior of
neurons in the brain. In contrast to Artificial Neural Networks (ANNs), SNNs
transmit signals through discrete spikes, and the accumulation of membrane
potentials in SNNs allows them to handle time series data effectively, making
them well-suited for processing event-based data. However, due to the non-
differentiability of spike sequences, applying the traditional backpropagation
(BP) algorithm directly to training SNNs poses significant challenges. As a
result, various methods have been proposed to explore effective training
approaches for SNNs [26, 27, 28, 29, 30, 31, 32, 33].

##  3 Bullying10K Dataset

In this section, we elaborate on the acquisition process, preprocessing
methods, and annotation details of the Bullying10K dataset. Simultaneously, we
analyze multiple attributes of the dataset, including its temporal length,
keypoints motion, and spatial event distribution.

###  3.1 Data Acquisition

##### Environment Setting

For data collection, we utilize two Davis346 [34], a high-speed event camera
that captures pixel brightness changes with microsecond precision. Each
pixel’s brightness change triggers an event $(t,x,y,p)$, where $(x,y)$
represents the spatial coordinates of the pixel, $t$ denotes the event’s time,
and $p$ is either $0$ or $1$, indicating the polarity of the brightness change
(enhancement or reduction). To capture multiple viewing angles and ensure
diversity in the collected data, we position two DVS cameras on the left and
right sides of the filming scene, as depicted in Figure 2. Additionally, we
incorporate two lighting conditions: bright and dark, to simulate various
real-life scenarios.

Figure 3: Visualization of human pose keypoints labels on event frames

Each event segment contains two participants who assume the roles of a
perpetrator and a victim during segments involving violent actions. In
contrast, sections depicting friendly actions involve the cooperation between
the two participants. Actors are instructed to perform a specific action in
each video segment, and we collect ten valid clips for each action. The
duration of each sample segment is action-dependent, ranging from 2 to 20
seconds.

##### Preprocessing

The Davis346 camera directly outputs data in the aedat4 file format,
specifically designed for storing event streams. To facilitate subsequent
processing and analysis, we transform the raw data into the widely used npy
format. npy is a common file format for storing NumPy [35] array data,
enabling effortless preservation and recovery of multidimensional data,
matrices, and other data structures. Throughout this transformation, we
organized the event stream into 10-millisecond units to maintain temporal
precision while effectively compressing the data. This strategy improves
convenience and reduces file size. For user-friendly data manipulation, we
supply code that merges the event stream into frames and reads the data.

##### Quality Control

To ensure data quality and usability of the data, we marked the position of
each camera before the start of filming, along with the relevant settings of
the DVS camera (including aperture, focal length, etc.), and maintained
consistency in these settings for each capture. To enhance data redundancy and
robustness, we introduced a manual screening step to optimize the data
collection process. We captured twelve sample segments for each action group.
Following data collection, we conducted manual screening to exclude poorly
captured segments, using only ten segments per group for the final dataset.
This ensures that every sample segment in the dataset possesses a good
quality, facilitating subsequent research and analysis.

###  3.2 Data Annotation

####  3.2.1 Category Label

Figure 4: Statistical data and analysis of Bullying10K, (a)motion of keypoints
(b)ratio of event in different polar (c)distribution of event (d)IoU of
individual(e) distribution of frame number (f) distribution of event number
(g)event number in sample

We conduct detailed classification and annotation for each sample after
filming for action recognition tasks to identify the represented action
accurately. Our dataset consists of ten actions, including six violent actions
(punching, kicking, hair grabbing, strangling, pushing, and slapping) and four
friendly actions (handshaking, finger guessing, greeting, and walking). We
further organized each category based on subjects, lighting scenes, and camera
positions. Each group is named using the subject’s code, action name,
illumination, and camera position.

####  3.2.2 Pose Estimation

Pose estimation is a task that involves identifying a person’s body position
and keypoints in a video or an image. Precise pose estimation facilitates
effective action recognition, making it an essential precursor to subsequent
action recognition tasks. To acquire human pose data for each DVS video
segment, we simultaneously leverage the RGB data captured with the event data.
The same camera captures both data types and offers overlapping scenes with
highly consistent content. This advantage allows us to employ well-established
pose estimation algorithms to predict the RGB dataset and obtain initial human
pose labels. Specifically, we utilize AlphaPose [36] as an automated labeling
tool, a multi-person pose estimation system. We employ the ResNet50 [37]
backbone pre-trained on the Haple dataset, while the annotation process
involved using the YOLOX [38] algorithm trained on the COCO dataset [39] as an
object detector. Our annotation target includes 26 keypoints of the human
body, as specified in [36], and the label information is saved in the COCO
format. Upon obtaining initial labels, we manually calibrated the labels, as
illustrated in Figure 3. It is important to note that direct pose estimation
algorithms for DVS data are still in the early stages of development.
Moreover, due to the inherent characteristics of DVS data, the events do not
explicitly represent human poses, which compounds the complexity of directly
utilizing DVS data for pose annotation.

###  3.3 Data Analysis

The Bullying10K dataset encompasses 10,000 sample clips, each with a duration
ranging from 2 to 20 seconds. It contains a staggering 12 billion valid
events, resulting in a total data volume of 255 GB. Figure 4 (e,f,g) presents
the distribution of frames, events, and events per frame in each sample clip
within the dataset. Notably, the average sample length of Bullying10K
surpasses existing event datasets captured with DVS cameras. This addresses
the limitation of shorter datasets captured by DVS cameras, providing new
challenges in establishing long-range dependencies between event data.

Figure 4 (a) displays the results obtained by analyzing the movement distance
of keypoints between consecutive frames. Different keypoints exhibit distinct
motion distribution patterns, with the most prominent trends observed in the
wrist and elbow movements, aligning with human motion characteristics. To
quantify the degree of task overlap in the video, we calculate the
Intersection over Union (IoU) of the bounding boxes for two characters
appearing in the same frame. Figure 4 (d) visualizes the distribution of IoU
values, which are primarily concentrated between 0.1 and 0.5. This indicates a
significant number of instances where characters overlap within this dataset.

As illustrated in Figure 4 (b), our analysis reveals a slightly lower
occurrence ratio of positive and negative polarity events. Moreover, we
visualize the ratio of positive and negative events for each action category
within the dataset. As shown in Figure 4 (c). The movement distribution of
actions such as punching and strangling predominantly occurs in the upper part
of the image, while walking exhibits a higher likelihood of occurring in both
the upper and lower parts of the image.

##  4 Evaluation and Task

Bullying10K, as a neuromorphic dataset, offers valuable advantages in terms of
user privacy persevering. We have provided three benchmark tasks: action
recognition, temporal action localization, and pose estimation. Through the
validation of these benchmarks, we aim to use neuromorphic datasets to address
these complex challenges and foster advancements in the field.

###  4.1 Action Recognition

##### Experimental Setting

Figure 5: PR curves for each category obtained from the x3d model trained on
Bullying10k.

The action recognition task aims to predict the corresponding action labels
based on input images. Each sample segment in the dataset includes a single
behavior, making it suitable for single-label classification tasks. During
preprocessing, the event streams are combined into frames, forming a sequence
that is then used to perform the action. The Bullying10K dataset has been
divided into training and testing sets with an 8:2 ratio, and the temporal
unit for integrating the event stream is set to 10 ms. Given many frames and
the need for batch training, we randomly crop video segments to a fixed
length, sampling at 0, 2, and 4 intervals. We employed various commonly used
action recognition models to evaluate the performance of the Bullying10K
dataset. We explored the performance of spiking neural networks on this
dataset, showcasing their potential for action recognition tasks.

##### Evaluation Metric

In the classification task, we measure the performance of the network by
calculating the accuracy of the output corresponding to the labels. However,
misjudgment of violent events can have severe consequences and cause
irreparable harm. Violent scenarios occur less frequently than non-violent
ones, implying a substantial class imbalance in real-world situations. Relying
solely on accuracy as an evaluation metric may not adequately reflect the
model’s ability to predict violent events accurately. To address this issue,
we have employed the Precision-Recall (PR) curve. The PR curve allows us to
examine the model’s predictive capabilities across varying judgment
thresholds.

##### Results and Analysis

Table 2 presents a detailed comparison of the performance of several widely-
used action recognition models on the Bullying10K dataset. It includes the
backbone architectures of each model and their respective operational
configurations. We conducted experiments using frame intervals of 0, 2, and 4,
with three different temporal step lengths of 4, 10, and 16. The results
demonstrate that increasing the frame interval and temporal step length
contributes to improved precision of the models. However, even models that
exhibit strong performance in traditional visual classification tasks did not
yield satisfactory results on the Bullying10K dataset, indicating the
dataset’s unique challenges. Furthermore, performed a visual analysis of the
corresponding PR curve of the X3D model on the Bullying10K dataset. The PR
curve reveals significant variations in performance across different action
categories, underscoring the dataset’s complexity and the need for robust
recognition models.

Table 2: Performance of different action recognition models on Bullying10k.

Model | backbone | step 4 | step 10 | step 16
---|---|---|---|---
gap 0 | gap 2 | gap 4 | gap 0 | gap 2 | gap 4 | gap 0 | gap 2 | gap 4
C3D [40] | Conv3D | 47.60 | 49.05 | 52.10 | 51.70 | 54.75 | 57.20 | 60.35 | 68.55 | 71.25
TAM [41] | ResNet | 54.85 | 59.20 | 62.00 | 57.85 | 60.80 | 65.05 | 58.20 | 66.90 | 71.20
R2Plus1D [42] | ResNet18 | 54.60 | 57.15 | 60.15 | 56.50 | 61.60 | 65.45 | 58.20 | 65.70 | 69.25
R3D [43] | ResNet18 | 57.20 | 61.05 | 62.10 | 58.90 | 65.46 | 68.60 | 62.70 | 69.75 | 72.50
SlowFast [44] | ResNet50 | 57.80 | 60.30 | 61.55 | 60.10 | 66.50 | 70.90 | 61.70 | 70.55 | 74.00
X3D [45] | ResNet | 60.30 | 62.40 | 64.80 | 63.30 | 69.40 | 72.15 | 65.75 | 72.45 | 76.90
SNN [46] | SEW-ResNet19 | 51.75 | 53.00 | 53.85 | 56.40 | 59.50 | 62.85 | 58.48 | 64.05 | 67.05

###  4.2 Temporal action localization

##### Experimental Setting

In surveillance videos, violent scenarios often occur irregularly and
sporadically. Due to the few background disturbance and noise in event data,
we can naturally concatenate multiple samples to form extended sequences. In
our task, we input a video frame sequence that encompasses segments from
different action categories randomly extracted from the dataset and integrate
them into longer video sequences. And aim to predict the action label along
with its corresponding start and end times. The ground truth annotations are
constructed based on the respective time intervals. For training on the
Bullying10K dataset, we have chosen commonly used temporal action localization
models. Initially, we pre-trained the TSN [47] and TSM [48] models on the
action recognition task, which serve as the feature extractors for the
localization models. We extract features from the dataset and store them for
subsequent analysis and processing.

##### Evaluation Metric

Average recall and Area Under the Curve (AUC) are commonly used metrics to
evaluate action localization. Concurrently, we employ the AR@N indicator for
assessment, representing the recall rate under the condition of N proposals.
This study considers N to be 1, 5, 10, 100. Additionally, we calculate the AUC
for the AR-AN curve.

##### Results and Analysis

We present the accuracy of several commonly used temporal action localization
algorithms on Bullying10K for comparative evaluation. They exhibit varying
performances under different features. On the other hand, performance
significantly diminishes with the reduction of proposals. It is worth
exploring a model designed for processing event datasets that can maintain
higher precision with fewer proposals.

Table 3: Performance of different temporal action localization models on Bullying10k Model | Feature | $AUC$ | $AR@1$ | $AR@5$ | $AR@10$ | $AR@100$
---|---|---|---|---|---|---
BSN [49] | TSN [47] | 75.9 | 31.1 | 70.5 | 75.8 | 77.6
BSN | TSM [48] | 75.5 | 31.8 | 71.2 | 75.8 | 77.2
BMN [50] | TSN | 80.3 | 34.6 | 74.3 | 81.0 | 82.0
BMN | TSM | 81.0 | 36.0 | 75.3 | 82.4 | 82.7
Table 4: Performance of different pose recognition models on Bullying10k

Model |  SimpleBaseline [51] (ResNet-50) |  SimpleBaseline [51] (ResNet-101) |  HRNet [52] (HRNet-ws32) |  HRNet [52] (HRNet-ws48) |  SimpleBaseline [51] (Spiking ResNet-50) |
---|---|---|---|---|---|---
AP | 62.6 | 63.6 | 62.7 | 62.8 | 54.1 |
AP50 | 88.3 | 88.2 | 88.2 | 87.8 | 84.6 |
AP75 | 67.4 | 67.7 | 67.5 | 67.2 | 57.6 |
APM | 58.3 | 59.3 | 58.6 | 59.5 | 49.3 |
APL | 73.4 | 74.6 | 73.9 | 74.9 | 64.9 |
AR | 65.9 | 67.0 | 66.3 | 66.7 | 58.5 |

###  4.3 Pose estimation

##### Experimental Setting

Human pose estimation typically involves providing an image or video
containing one or multiple individuals and outputting the corresponding
locations of various keypoints for each person. These keypoints include head,
shoulders, arms, and legs. Different datasets may have different numbers of
joints, and in the case of Bullying10K, we use 26 keypoints. Pose estimation
can provide additional information for action recognition. We evaluate the
performance of Bullying10K using several commonly used pose estimation models.

Figure 6: Taking the prediction results of the SimpleBaseline model with
ReseNet101 as the backbone, we visualize the skeleton and heatmap of the
prediction results, respectively.

##### Evaluation Metric

The standard metric used to measure pose estimation is usually based on Object
Keypoint Similarity (OKS). OKS is a scale-invariant measure of localization
accuracy and is utilized to evaluate how closely the predicted keypoints of a
model match the ground truth keypoints.
$OKS=\frac{\sum_{i}[\exp(-d^{2}_{i}/2s^{2}k^{2}_{i})\delta(v_{i}>0)]}{\sum_{i}[\delta(v_{i}>0)]}$,
where $d_{i}$ is the distance between the position of predicted keypoints and
the ground truth. $v_{i}$ denotes the visibility of related keypoints. $s$
indicates the object scale. $k_{i}$ is the predefined constant that controls
falloff.

Average Precision (AP) and Recall (AR) are used to measure the accuracy of
different models on the Bullying10K dataset: $AP$, $AP^{50}$ (OKS is at
0.50),$AP^{75}$,$AP^{M}$,$AP^{L}$,$AR$.

##### Results and Analysis

Table 4 presents the results of multiple commonly used pose estimation models
on the Bullying10K dataset. However, these models exhibit relatively low
accuracy on the dataset, indicating that Bullying10K poses substantial
challenges for pose estimation tasks. Additionally, although HRNet has shown
superior capabilities to SimpleBaseline on other RGB datasets, it achieved
lower accuracy on our dataset. This discrepancy may be attributed to the
unique nature of our event-based dataset, which significantly differs from RGB
images. Furthermore, we investigated the potential of SNN in posture
estimation by testing the SNN backbone using the SimpleBaseline algorithm. To
enhance the interpretability of the results, we visualized relevant output
images and presented heatmaps for selected samples.

##  5 Conclusion

This research introduces a novel event-driven dataset called Bullying10K,
which utilizes Dynamic Vision Sensor (DVS) cameras to detect instances of
violent behavior while preserving individual privacy. The dataset is designed
to address the limitations of existing event-driven datasets by featuring
complex, rapid movements and overlapping figures, presenting higher complexity
and challenges. By offering a large-scale dataset, Bullying10K enables
researchers to explore complex actions and contributes to advancements in
violence detection and privacy-preservation techniques.

## Acknowledgments and Disclosure of Funding

## References

  * [1] Qasim Mahmood Rajpoot and Christian Damsgaard Jensen.  Video surveillance: Privacy issues and legal compliance.  In Promoting social change and democracy through information technology, pages 69–92. IGI global, 2015.
  * [2] Niels Haering, Péter L Venetianer, and Alan Lipton.  The evolution of video surveillance: an overview.  Machine Vision and Applications, 19(5-6):279–290, 2008.
  * [3] Omar Elharrouss, Noor Almaadeed, and Somaya Al-Maadeed.  A review of video surveillance systems.  Journal of Visual Communication and Image Representation, 77:103116, 2021.
  * [4] Christopher Slobogin.  Public privacy: camera surveillance of public places and the right to anonymity.  Miss. lJ, 72:213, 2002.
  * [5] Guillermo Gallego, Tobi Delbrück, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew J Davison, Jörg Conradt, Kostas Daniilidis, et al.  Event-based vision: A survey.  IEEE transactions on pattern analysis and machine intelligence, 44(1):154–180, 2020.
  * [6] Lin Zhu, Siwei Dong, Tiejun Huang, and Yonghong Tian.  A retina-inspired sampling method for visual texture reconstruction.  In 2019 IEEE International Conference on Multimedia and Expo (ICME), pages 1432–1437. IEEE, 2019.
  * [7] Lin Zhu, Siwei Dong, Jianing Li, Tiejun Huang, and Yonghong Tian.  Retina-like visual image reconstruction via spiking neural model.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1438–1446, 2020.
  * [8] Gemma Taverni, Diederik Paul Moeys, Chenghan Li, Celso Cavaco, Vasyl Motsnyi, David San Segundo Bello, and Tobi Delbruck.  Front and back illuminated dynamic and active pixel vision sensors comparison.  IEEE Transactions on Circuits and Systems II: Express Briefs, 65(5):677–681, 2018.
  * [9] Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li, and Luping Shi.  Cifar10-dvs: an event-stream dataset for object classification.  Frontiers in neuroscience, 11:309, 2017.
  * [10] Garrick Orchard, Ajinkya Jayawant, Gregory K Cohen, and Nitish Thakor.  Converting static image datasets to spiking neuromorphic datasets using saccades.  Frontiers in neuroscience, 9:437, 2015.
  * [11] Yang Li, Yiting Dong, Dongcheng Zhao, and Yi Zeng.  N-omniglot, a large-scale neuromorphic dataset for spatio-temporal sparse few-shot learning.  Scientific Data, 9(1):746, 2022.
  * [12] Shu Miao, Guang Chen, Xiangyu Ning, Yang Zi, Kejia Ren, Zhenshan Bing, and Alois Knoll.  Neuromorphic vision datasets for pedestrian detection, action recognition, and fall detection.  Frontiers in neurorobotics, 13:38, 2019.
  * [13] Arnon Amir, Brian Taba, David Berg, Timothy Melano, Jeffrey McKinstry, Carmelo Di Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Garreau, Marcela Mendoza, et al.  A low power, fully event-based gesture recognition system.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7243–7252, 2017.
  * [14] Yin Bi, Aaron Chadha, Alhabib Abbas, Eirina Bourtsoulatze, and Yiannis Andreopoulos.  Graph-based spatio-temporal feature learning for neuromorphic vision sensing.  IEEE Transactions on Image Processing, 29:9084–9098, 2020.
  * [15] Hildegard Kuehne, Hueihan Jhuang, Estíbaliz Garrote, Tomaso Poggio, and Thomas Serre.  Hmdb: a large video database for human motion recognition.  In 2011 International conference on computer vision, pages 2556–2563. IEEE, 2011.
  * [16] Yihan Lin, Wei Ding, Shaohua Qiang, Lei Deng, and Guoqi Li.  Es-imagenet: A million event-stream classification dataset for spiking neural networks.  Frontiers in neuroscience, page 1546, 2021.
  * [17] Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah.  Ucf101: A dataset of 101 human actions classes from videos in the wild.  arXiv preprint arXiv:1212.0402, 2012.
  * [18] Amos Sironi, Manuele Brambilla, Nicolas Bourdis, Xavier Lagorce, and Ryad Benosman.  Hats: Histograms of averaged time surfaces for robust event-based object classification.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1731–1740, 2018.
  * [19] Yin Bi, Aaron Chadha, Alhabib Abbas, Eirina Bourtsoulatze, and Yiannis Andreopoulos.  Graph-based object classification for neuromorphic vision sensing.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 491–501, 2019.
  * [20] Qianhui Liu, Dong Xing, Huajin Tang, De Ma, and Gang Pan.  Event-based action recognition using motion information and spiking neural networks.  In IJCAI, pages 1743–1749, 2021.
  * [21] Enrique Bermejo Nievas, Oscar Deniz Suarez, Gloria Bueno García, and Rahul Sukthankar.  Violence detection in video using computer vision techniques.  In Computer Analysis of Images and Patterns: 14th International Conference, CAIP 2011, Seville, Spain, August 29-31, 2011, Proceedings, Part II 14, pages 332–339. Springer, 2011.
  * [22] Enrique Bermejo Nievas, Oscar Deniz Suarez, Gloria Bueno Garcia, and Rahul Sukthankar.  Hockey fight detection dataset.  In Computer Analysis of Images and Patterns, pages 332–339. Springer Berlin/Heidelberg, Germany, 2011.
  * [23] Waqas Sultani, Chen Chen, and Mubarak Shah.  Real-world anomaly detection in surveillance videos.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6479–6488, 2018.
  * [24] Ming Cheng, Kunjing Cai, and Ming Li.  Rwf-2000: an open large scale video database for violence detection.  In 2020 25th International Conference on Pattern Recognition (ICPR), pages 4183–4190. IEEE, 2021.
  * [25] Claire-Hélène Demarty, Cédric Penet, Mohammad Soleymani, and Guillaume Gravier.  Vsd, a public dataset for the detection of violent scenes in movies: design, annotation, analysis and evaluation.  Multimedia Tools and Applications, 74:7379–7404, 2015.
  * [26] Peter U Diehl and Matthew Cook.  Unsupervised learning of digit recognition using spike-timing-dependent plasticity.  Frontiers in computational neuroscience, 9:99, 2015.
  * [27] Tielin Zhang, Yi Zeng, Dongcheng Zhao, and Mengting Shi.  A plasticity-centric approach to train the non-differential spiking neural networks.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
  * [28] Bodo Rueckauer, Iulia-Alexandra Lungu, Yuhuang Hu, Michael Pfeiffer, and Shih-Chii Liu.  Conversion of continuous-valued deep networks to efficient event-driven networks for image classification.  Frontiers in neuroscience, 11:682, 2017.
  * [29] Shikuang Deng and Shi Gu.  Optimal conversion of conventional artificial neural networks to spiking neural networks.  arXiv preprint arXiv:2103.00476, 2021.
  * [30] Yang Li and Yi Zeng.  Efficient and accurate conversion of spiking neural network with burst spikes.  arXiv preprint arXiv:2204.13271, 2022.
  * [31] Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi.  Spatio-temporal backpropagation for training high-performance spiking neural networks.  Frontiers in neuroscience, 12:331, 2018.
  * [32] Emre O Neftci, Hesham Mostafa, and Friedemann Zenke.  Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks.  IEEE Signal Processing Magazine, 36(6):51–63, 2019.
  * [33] Kaiwei Che, Luziwei Leng, Kaixuan Zhang, Jianguo Zhang, Qinghu Meng, Jie Cheng, Qinghai Guo, and Jianxing Liao.  Differentiable hierarchical and surrogate gradient search for spiking neural networks.  Advances in Neural Information Processing Systems, 35:24975–24990, 2022.
  * [34] Christian Brandli, Lorenz Muller, and Tobi Delbruck.  Real-time, high-speed video decompression using a frame-and event-based davis sensor.  In 2014 IEEE International Symposium on Circuits and Systems (ISCAS), pages 686–689. IEEE, 2014.
  * [35] Charles R Harris, K Jarrod Millman, Stéfan J Van Der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J Smith, et al.  Array programming with numpy.  Nature, 585(7825):357–362, 2020.
  * [36] Hao-Shu Fang, Jiefeng Li, Hongyang Tang, Chao Xu, Haoyi Zhu, Yuliang Xiu, Yong-Lu Li, and Cewu Lu.  Alphapose: Whole-body regional multi-person pose estimation and tracking in real-time.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022\.
  * [37] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [38] Zheng Ge, Songtao Liu, Feng Wang, Zeming Li, and Jian Sun.  Yolox: Exceeding yolo series in 2021.  arXiv preprint arXiv:2107.08430, 2021.
  * [39] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick.  Microsoft coco: Common objects in context.  In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740–755. Springer, 2014.
  * [40] Shuiwang Ji, Wei Xu, Ming Yang, and Kai Yu.  3d convolutional neural networks for human action recognition.  IEEE transactions on pattern analysis and machine intelligence, 35(1):221–231, 2012.
  * [41] Zhaoyang Liu, Limin Wang, Wayne Wu, Chen Qian, and Tong Lu.  Tam: Temporal adaptive module for video recognition.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 13708–13718, 2021.
  * [42] Du Tran, Heng Wang, Lorenzo Torresani, Jamie Ray, Yann LeCun, and Manohar Paluri.  A closer look at spatiotemporal convolutions for action recognition.  In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 6450–6459, 2018.
  * [43] Du Tran, Jamie Ray, Zheng Shou, Shih-Fu Chang, and Manohar Paluri.  Convnet architecture search for spatiotemporal feature learning.  arXiv preprint arXiv:1708.05038, 2017.
  * [44] Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He.  Slowfast networks for video recognition.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 6202–6211, 2019.
  * [45] Christoph Feichtenhofer.  X3d: Expanding architectures for efficient video recognition.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 203–213, 2020.
  * [46] Wei Fang, Zhaofei Yu, Yanqi Chen, Tiejun Huang, Timothée Masquelier, and Yonghong Tian.  Deep residual learning in spiking neural networks.  Advances in Neural Information Processing Systems, 34:21056–21069, 2021.
  * [47] Limin Wang, Yuanjun Xiong, Zhe Wang, Yu Qiao, Dahua Lin, Xiaoou Tang, and Luc Van Gool.  Temporal segment networks: Towards good practices for deep action recognition.  In European conference on computer vision, pages 20–36. Springer, 2016.
  * [48] Ji Lin, Chuang Gan, and Song Han.  Tsm: Temporal shift module for efficient video understanding.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 7083–7093, 2019.
  * [49] Tianwei Lin, Xu Zhao, Haisheng Su, Chongjing Wang, and Ming Yang.  Bsn: Boundary sensitive network for temporal action proposal generation.  In Proceedings of the European conference on computer vision (ECCV), pages 3–19, 2018.
  * [50] Tianwei Lin, Xiao Liu, Xin Li, Errui Ding, and Shilei Wen.  Bmn: Boundary-matching network for temporal action proposal generation.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 3889–3898, 2019.
  * [51] Bin Xiao, Haiping Wu, and Yichen Wei.  Simple baselines for human pose estimation and tracking.  In Proceedings of the European conference on computer vision (ECCV), pages 466–481, 2018.
  * [52] Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang.  Deep high-resolution representation learning for human pose estimation.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5693–5703, 2019.

## Checklist

  1. 1.

For all authors…

    1. (a)

Do the main claims made in the abstract and introduction accurately reflect
the paper’s contributions and scope? [Yes] see Section 1

    2. (b)

Did you describe the limitations of your work? [Yes] see Section 5

    3. (c)

Did you discuss any potential negative societal impacts of your work? [N/A]

    4. (d)

Have you read the ethics review guidelines and ensured that your paper
conforms to them? [Yes]

  2. 2.

If you are including theoretical results…

    1. (a)

Did you state the full set of assumptions of all theoretical results? [N/A]

    2. (b)

Did you include complete proofs of all theoretical results? [N/A]

  3. 3.

If you ran experiments (e.g. for benchmarks)…

    1. (a)

Did you include the code, data, and instructions needed to reproduce the main
experimental results (either in the supplemental material or as a URL)? [Yes]
see the supplemental material

    2. (b)

Did you specify all the training details (e.g., data splits, hyperparameters,
how they were chosen)? [Yes] see the supplemental material

    3. (c)

Did you report error bars (e.g., with respect to the random seed after running
experiments multiple times)? [No]

    4. (d)

Did you include the total amount of compute and the type of resources used
(e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] see the
supplemental material

  4. 4.

If you are using existing assets (e.g., code, data, models) or
curating/releasing new assets…

    1. (a)

If your work uses existing assets, did you cite the creators? [Yes] see
section 4

    2. (b)

Did you mention the license of the assets? [N/A]

    3. (c)

Did you include any new assets either in the supplemental material or as a
URL? [Yes] see the supplemental material and section 1

    4. (d)

Did you discuss whether and how consent was obtained from people whose data
you’re using/curating? [N/A]

    5. (e)

Did you discuss whether the data you are using/curating contains personally
identifiable information or offensive content? [Yes] see the supplemental
material

  5. 5.

If you used crowdsourcing or conducted research with human subjects…

    1. (a)

Did you include the full text of instructions given to participants and
screenshots, if applicable? [Yes] see the supplemental material

    2. (b)

Did you describe any potential participant risks, with links to Institutional
Review Board (IRB) approvals, if applicable? [N/A]

    3. (c)

Did you include the estimated hourly wage paid to participants and the total
amount spent on participant compensation? [Yes] see the supplemental material