# PURE: Passive mUlti-peRson idEntification via Deep Footstep Separation and
Recognition

Chao Cai, Ruinan Jin, Peng Wang, Liyuan Ye, Hongbo Jiang, and Jun Luo  C. Cai
and J. Luo is with the School of Computer Engineering, Nanyang Technological
University, Singapore 639798. E-mail: {chris.cai<EMAIL_ADDRESS>R. Jin
is with the School of Information and Safety Engineering, Zhongnan University
of Economics and Law, e-mail<EMAIL_ADDRESS>P. Wang and L. Ye are with
the School of Electronic Information and Communications, Huazhong University
of Science and Technology, China. E-mail<EMAIL_ADDRESS>H.
Jiang (corresponding author) is with the College of Computer Science and
Electronic Engineering, Hunan University. E-mail<EMAIL_ADDRESS>
###### Abstract

Recently, passive behavioral biometrics (e.g., gesture or footstep) have
become promising complements to conventional user identification methods
(e.g., face or fingerprint) under special situations, yet existing sensing
technologies require lengthy measurement traces and cannot identify multiple
users at the same time. To this end, we propose PURE as a passive multi-person
identification system leveraging deep learning enabled footstep separation and
recognition. PURE passively identifies a user by deciphering the unique
“footprints” in its footstep. Different from existing gait-enabled recognition
systems incurring a long sensing delay to acquire many footsteps, PURE can
recognize a person by as few as only one step, substantially cutting the
identification latency. To make PURE adaptive to walking pace variations,
environmental dynamics, and even unseen targets, we apply an adversarial
learning technique to improve its domain generalisability and identification
accuracy. Finally, PURE can defend itself against replay attack, enabled by
the richness of footstep and spatial awareness. We implement a PURE prototype
using commodity hardware and evaluate it in typical indoor settings.
Evaluation results demonstrate a cross-domain identification accuracy of over
90%.

###### Index Terms:

User identification, source separation, footstep recognition, adversarial
learning.

##  1 Introduction

User identification technology is an indispensable element for building smart
indoor applications, such as elderly and child care, personalized custom
service, and surveillance in sensitive zones. As one example, a smart home
recognizing its elderly host (also his/her walking direction) may predict
his/her intentions and switch on home appliances (e.g., lights or TVs)
accordingly. Taking supermarket as another example, identifying customer
identities and hence retrieving the relevant shopping histories may enable a
salesman (or simply an advertising system) to make more appropriate
recommendations and thus more likely to cut a deal. Existing solutions for
user identification, ranging from conventional computer vision [1, 2, 3, 4]
and biometrics [5, 6, 7] to the emerging WiFi sensing [8, 9, 10, 11], have
shown promising results for several important applications.

However, the inherent limitations of existing solutions have prevented them
from being widely applicable. In particular, computer vision techniques
require a good lighting condition and line-of-sight (LoS), while they may
raise critical privacy concerns. Biometrics often demand wearable instruments
or users’ active involvements, hence causing potential discomfort and
inconvenience. WiFi sensing approaches leveraging gait profile or breathing
pattern appear to be viable, but they often fail to work in practice due to
the severe interference from WiFi’s main function of communications and other
co-spectrum devices. Recent developments on passive behavioral biometrics [12,
13] (e.g., footstep-enabled identification [14, 15]) can be promising
alternatives to the aforementioned solutions as they incur no privacy issues
and have less interference due to their limited sensing range. However, these
approaches often require special sensing hardware [15, 12, 13] or multiple
long-time footstep measurements [14]. Besides, they can only handle one user
each time, rendering them less appealing for practical applications where
multiple users may appear at the same time.

In this paper, we revisit the footstep-enabled passive behavioral biometrics,
and we propose PURE (Passive mUlti-peRson idEntification) to achieve a
lightweight user identification, exploiting both commodity sensing devices and
up-to-date deep learning techniques. PURE is built on a key observation that
footsteps carry “footprints” unique to individuals and thus can be leveraged
for effective user identification. These footprints can be passively captured
by commodity acoustic sensing hardware, totally removing the need for active
user involvements. PURE aims to extract such footprint information from as few
as a single step, thus enabling a much faster identification than existing
gait-oriented systems. In addition, PURE targets simultaneous multi-user
identification, robustness against various background interference, as well as
immunity to replay attack.

Implementing the above ideas into a practical system entails several technical
challenges. First of all, background noise and interferences (especially
voices) may overwhelm footsteps, significantly affecting the system
performance. Both detecting and extracting footsteps under such strong
interference can be a formidable task. Second, footstep variations, caused by
factors such as different walking paces, can lead to distinctive features
hence affect the identification performance. In addition, footstep can carry
environment-dependent information (e.g., floor material). Such domain
conditions are largely irrelevant to individuals’ unique footprints, and can
thus seriously degrade the identification performance if not sufficiently
removed. Last but not least, replay attack that records someone’s footstep and
cheats an identification system later by playing the recorded sounds could be
a major hurdle for practical adoption in certain applications.

Figure 1: Mel-Frequency Cepstral (MFC) of footsteps generated by five persons
clearly show distinctive profiles.

In PURE, we tackle these challenges via a series of delicate designs. In the
presence of continuous voice signals, we explore the rhythmic patterns in the
time-frequency (TF) representation to detect footsteps and employ a blind
source separation algorithm with the a priori knowledge to extract footsteps.
To exclude feature variations caused by environment-dependent information and
walking pace discrepancy for user identification, we train the user identifier
(a predictor) via an adversarial domain adaptation scheme to improve its
generalizability. Finally, we leverage the following fact to thwart replay
attack: the replayed sounds exhibit static spatial characters (e.g., Angle-of-
Arrival) and reveal inconsistency between walking speed and step frequency. To
summarize, this paper makes the following contributions:

  * •

We propose PURE, an acoustic passive multi-person identification system with
little infrastructure cost.

  * •

We innovatively employ an adversarial learning scheme to combat feature
variations introduced by environment-independent information or heterogeneous
walking paces, thus improving the system generalizability and identification
performance.

  * •

We leverage the dynamic and smoothly changing spatial characters extracted
from both structure-borne and air-borne footsteps to thwart the challenging
replay attack.

  * •

We implement PURE prototype using commodity hardware and extensively evaluate
its performance under various practical settings; the results demonstrate a
cross-domain identification accuracy up to 90%.

The rest of this paper is organized as followed: Sec. 2 discusses footstep
basics. In Sec. 3, we elaborate on the details of system design. Sec. 4
reports extensive performance evaluation results. We present a literature
review in Sec. 5, and finally conclude the paper in Sec. 6.

##  2 Background and Motivation

We first explain the basic theories about footstep, then we further provide
rationales for our later designs via a few preliminary measurement studies.

###  2.1 Basic Acoustics of Footstep

When a foot touches the ground, it causes minor vibrations at the impact point
and radiates energy via both the air and solid medium behind the surface. The
acoustic pressure radiated from this impact point can be characterized by
Rayleigh’s surface integral [16]:

$\displaystyle p\left(r,\theta,t\right)$ $\displaystyle=\frac{-2\rho}{\pi
md}\int_{R}\int_{\phi}R\sum_{j}\sum_{k}\left(-1\right)^{\frac{j-1}{2}}\sin\left[j\pi\left(\frac{1}{2}+\frac{R}{a}\right)\right]$
$\displaystyle\times\left[\frac{\text{d}F}{\text{d}t}\left(t-d/c\right)*\cos\left(\omega_{jk}\left(t-d/c\right)\right)\right]\mathrm{d}R\mathrm{d}\phi,$
(1)

where $(r,\theta)$ describes the position relative to the impact point with
respect to the floor plane, $\rho$ is the mass density that characterizes the
medium properties, $c$ is the speed of acoustic signals in a certain medium,
$\omega$ is the frequency, $a$ is a constant, $d$ is the length of the leg,
$F$ is the impact force being zero at all time except during the impact
period, and $*$ denotes convolution. The acoustic pressure, generated by the
impact event, mostly radiates through two common mediums. The one propagating
through piston-like and non-dispersive air channel (we refer to as air-borne
hereafter) has a constant speed (e.g., a speed of 340 m/s at a temperature of
25∘C [16]) and the corresponding waveform remains identical along the
propagating path as far as there are no multipath reverberations. The other
one traversing in solid medium, known as blending wave (we refer to as
structure-borne hereafter), exhibits a dispersive phenomenon where the
propagation speed $c_{f}$ (ranging from 2000 m/s to 3000 m/s) of a specific
signal component is a function of its frequency $f$:

${c_{f}}=\sqrt[4]{Eh{f^{2}}\left[12\rho(1-v_{p}^{2})\right]^{-1}},$ (2)

where $E,\rho,h$ are constants that characterize the property of a medium: $E$
quantifies the elastic property, $\rho$ is the mass density that indicates the
stiffness, $h$ is the thickness, and $v_{p}$ is the phase velocity. Eqn. (2)
implies that when detecting structure-borne footsteps from different distances
relative to the impact point, the corresponding waveform can be different due
to the frequency-dependent propagation speeds.

(a) Feature embeddings of different users under different environments.

(b) Feature embeddings under walking pace variations.

Figure 2: MFC features embeddings (a) under different environments, and (b)
under different walking speeds: s1 $=$ 0.2 m/s, s2 $=$ 0.5 m/s, s3 $=$ 1
m/s, “other” denotes other users at 0.2 m/s.

###  2.2 Richness of Footstep Acoustic Profile

Eqn. (2.1) and (2) together show that footstep contains both air-borne and
structure-borne signals and they involve a rich set of frequency components,
carrying unique identity information. For example, both a person’s weight
(impact force) and walking style (duration of impact) are closely related to
the generated acoustic pressures. To verify the above intuition, we record
footsteps from five persons under the same circumstance and inspect the
corresponding Mel-Frequency Cepstrals (MFC) [17]. The MFC features depicted in
Fig. 1 clearly show distinctive visual clues and thus imply the possibility of
using footsteps for person identification. We then carry out measurements
under different environments and Fig. 2(a) plots the embedded features. One
may observe that the low-dimensional features among different users show clear
boundaries, demonstrating the possibility of effective identification using
footsteps. We also conduct a simple user study by playing several recording of
footsteps produced by different persons; all audiences participating in our
user study indeed claim that they can tell perceivable differences among these
recordings. The distinctiveness of acoustic features in footsteps from
different persons lays the foundation for our footstep enabled user
identification system. However, the footstep features of the same person can
be affected by domain conditions such as varying walking paces as demonstrated
by Fig. 2(b). These feature distinctions, introduced by domain conditions
(including, e.g., environment heterogeneity and walking pace variations), can
be particularly detrimental to the identification accuracy and thus should be
excluded.

###  2.3 Differences between Structure- and Air-borne Footsteps

The structure-borne and air-borne components of footsteps have a sharp
difference in their propagation speeds, as already discussed in Sec. 2.1; this
offers us an opportunity to physically separate them. For instance, when the
sampling rate is 192 kHz and we detect the footsteps at a distance of 2 m, a
clean set of structure-borne signals (due to propagation difference) lasts for
$\frac{1}{340}-\frac{1}{3000}\approx 5.4$ ms, an equivalent of 1000 samples.

Figure 3: A footstep waveform contains both structure- and air-borne
components that are temporally separated.

As clear shown by the measurements illustrated in Fig. 3, the structure-borne
component indeed arrives ahead of the air-borne one. Separating these two
components allows us to exam their respective natures. As the air-borne
component appears to have more complicated waveforms, we conjecture that it
may be more suitable for the identification purpose than its structure-borne
counterpart. To verify the above hypothesis, we have conducted measurements on
the five persons in several environments using commodity microphones. We then
utilize a Gaussian Mixture Model (GMM) [18] to identify those persons based on
either structure-borne or air-borne components. The identification accuracy
comparison under different Signal-to-Noise-Ratio (SNR) is shown in Fig. 4(a),
which confirms the advantage of adopting the air-borne component for achieving
a higher identification accuracy. Also, the experiments reveal that using
traditional GMM method for identification is practically not feasible since
the required SNR is over 60 dB.

(a) Identification accuracy comparison.

(b) Feature embeddings from different locations.

Figure 4: Examining the properties of structure-borne and air-borne components
of footsteps. (a) GMM applied on air- and structure-borne signals for
identification. (b) Structure-borne component carries distance information.

###  2.4 The Edge of Structure-borne Footstep

Although structure-borne footstep is less representative than its air-borne
counterpart for identifying persons, it has a unique property, namely acoustic
dispersion, as shown by Eqn. (2). The dispersion phenomenon indicates that the
structure-borne waveform is modulated by distance, suggesting the possibility
of using it for ranging. Our measurements in Fig. 4(b), showcasing clear
boundaries between feature embeddings of structure-borne footsteps from
different locations, further demonstrate the feasibility of using structure-
borne footsteps for ranging. This range information, together with the angle
of signal arrival, may give us an edge over attackers who replay footstep
recordings, because the smoothly changing spatial characters of human
footsteps cannot be fully imitated.

Figure 5: The system architecture of PURE.

##  3 System Design

PURE consists of a pipeline of signal processing and deep learning modules as
shown in Fig. 5. It employs a microphone array [19] to capture raw audio
streams and then denoise them via a background spectral subtraction, removing
most stationary noises. Then footstep detection is performed by inspecting the
energy change, as well as by adopting an audio classifier. During the above
process elaborated in Sec. 3.1, if there exist continuous interferences such
as voices, a source separation algorithm kicks in to decouple footsteps from
the sound mixture, as detailed in Sec. 3.2. Finally, PURE applies a neural
network to identify the user behind each footstep (Sec. 3.3), and it also
extracts spatial information to counter replay attack (Sec. 3.4).

###  3.1 Background Noise Suppression and Footstep Detection

Indoor places often contain common color noises [20] that can affect the SNR
of captured footsteps and thus should be removed. From the measurements in
Fig. 2, we can see that the spectra of footsteps almost spread over the entire
available bandwidth. Therefore, using common bandwidth oriented filter to trim
the out-of-band noise is infeasible. To this end, we utilize a multi-band
spectral subtraction method [20] to suppress background noise and obtain about
3 dB gain in SNR.

Figure 6: Low-dimensional outputs of GMM classifier show clear boundaries
between different acoustic noises.

After that, we apply abrupt energy detection, characterized by Root Mean
Square (RMS) [21], for candidate footstep detection. The RMS of a sequence
$\mathbf{x}=\\{x_{1},x_{2},\cdots,x_{L}\\}$ is defined by
$E_{\mathrm{RMS}}(\mathbf{x})=\sum_{i=1}^{L}\sqrt{\frac{x^{2}_{1}+x^{2}_{2}+...+x^{2}_{L}}{L}}$.
If the detected energy $E_{RMS}$ is above a certain threshold, a footstep may
be captured. Since many transient noises can also exhibit high energy, a
simple energy-based detection method would not be sufficient. To this end, we
further utilize a Gaussian Mixture Model (GMM) [18] based audio classifier to
recognize footsteps. We train this GMM classifier against common background
noises and it almost achieves an oracle performance: as shown in Fig. 6,
different acoustic signals can be clearly identified.

###  3.2 Footstep Extraction under Continuous Interference

To extract footstep overwhelmed by continuous strong interference such as
voice signals, we further utilize a Flexible Audio Source Separation Toolbox
(FASST) [22]. This procedure is often suspended for the sake of efficiency; it
is only invoked when footsteps are heavily interfered. To leverage FASST, we
use the rhythmic pattern of footsteps in frequency domain to detect the
ongoing walking paces. More specifically, we utilize auto-correlation of STFT
magnitude to detect the presence of footsteps. The reason for not directly
using auto-correlation in time-domain is that the repetitive features in time-
domain waveform are likely to be under noise floor due to their relatively low
volume.

Suppose $\mathbf{V},\mathbf{V}\in\mathbb{R}^{P\times Q}$ denotes the STFT
power spectrum where $P,Q$ are the dimensions of time and frequency,
respectively, we first calculate the auto-correlation of $\mathbf{V}$ along
the time frames to obtain $\mathbf{B}$, the operation of which can enhance the
rhythmic pattern of footsteps:

$\mathbf{B}(i,j)=\frac{1}{P-j+1}\sum_{k=1}^{P-j+1}\mathbf{V}(i,k)\mathbf{V}(i,k+j-1).$
(3)

We then take the average of $\mathbf{B}$ over the frequency dimension and
normalize the result with its first term,
$b(j)=\frac{1}{Q}\sum_{i=1}^{Q}\mathbf{B}(i,j),b(j)=\frac{b(j)}{b(1)}$. The
walking rhythmic features can then be inspected in $\mathbf{b}=[b(j)]$, which
we denote as the Averaged Spectrogram Auto-Correlation Coefficient (ASACC). We
further check whether the periodicity exhibited in $\mathbf{b}$ lies in a
reasonable range as the frequency of normal walking pace is usually within
$[0.8,2]$Hz [23]. Ideally, we can utilize $\mathbf{b}$ to obtain a soft binary
mask for $\mathbf{V}$ and then extract the footsteps in the same vein as the
proposal in [24]. However, as explain by [22], the mask-based approaches, even
with ground truth labels, are inferior to the local instantaneous Gaussian
mixture models in source separation. Therefore, we still resort to FASST for
better separation performance. Our measurements in Fig. 7 further verify the
above intuition as correlation in time domain shown in Fig. 7(a) exhibit no
rhythmic features while the rhythm is evident in frequency domain as shown in
Fig. 7(b). To further remove the residual noises in the separated signals, we
apply a trained time-domain denoising network in [25] for footstep
enhancement.

(a) Time domain correlation.

(b) Spectrum domain correlation.

Figure 7: Time domain correlation of a mixture of footsteps and voice fails to
reveal rhythmic features (a), while in frequency domain, such features can be
clearly observed (b).  Figure 8: Identification network (ID-Net) based on
domain adversarial adaptation.

###  3.3 Domain Adapted Identification

As we have mentioned in Sec. 2.1, the domain conditions, including walking
speed variations and environment dynamics, can be particularly detrimental to
our identification system. To preclude these domain information hence improve
identification accuracy, we adopt domain adversarial adaptation. We formulate
the identification problem as a classification problem: given a specific
footstep $\mathbf{x}\in{X}$, we aim to learn a maximum likelihood estimator
$\mathcal{G}:{X}\rightarrow{I}$, where $X$ and $I$ represent the spaces of
input (STFT of footstep waveform) and user identity, respectively. In reality,
an footstep $\mathbf{x}$ is sampled from a joint distribution
$P(\mathbf{x},u,d,v,e)$, where $u\in I$ that characterizes footstep diversity,
$d\in D$ denotes distance information, $v\in V$ represents walking pace, and
$e\in E$ describe specific properties incurred by environment dynamics. We
denote $d,v,e$ potentially harmful to our identification purpose as domain
conditions. Apparently, only features characterizing the joint distribution
$P(\mathbf{x},u)$ are desirable for our identification purpose but features
induced by domain conditions $d,v,e$ should be eliminated. To this end, we use
a deep neural network $G(\mathbf{x})$ to approximate $\mathcal{G}(\mathbf{x})$
and adopt an adversarial learning [26] to train $G$ so as to preclude the
impact from $d,v,e$.

Our ID-Net is made up of three parts, namely feature extractor, identity
predictor, and domain discriminator, as shown in Fig. 8. The feature extractor
$\mathbf{f}=G_{f}(\mathbf{x},{\boldsymbol{\theta}}_{f})$, parameterized by
${\boldsymbol{\theta}}_{f}$, compresses an STFF of footstep waveform
$\mathbf{x}$ into $\mathbf{f}\in\mathbb{R}^{Q}$, a lower-dimensional feature
vector. With $\mathbf{f}$, the identity predictor aims to recognize the user
behind the footstep, while the goal of domain discriminator is to identify
different domains. The input to the domain discriminator is $\mathbf{f}$
weighted by a latent vector extracted from the identity predictor. The goal of
ID-Net is to extract a domain independent representation $\mathbf{f}$ so as to
i) achieve high-accuracy user identification and ii) deceive the domain
discriminator to misidentify domains. It is this domain independent
$\mathbf{f}$ (involving only identity-specific features) that allows a cross-
domain generalization of ID-Net.

The identity predictor
$\hat{P}_{u}=G_{u}(\mathbf{f},{\boldsymbol{\theta}}_{u})$ outputs a
probability matrix, whose element $\hat{p}^{(i,j)}_{u}$ represents the
probability of the $i$-th footstep belonging to the $j$-th user. The loss
function for training the parameter ${\boldsymbol{\theta}}_{u}$ is categorical
cross-entropy:

$\mathcal{L}_{u}=-|I|^{-1}\textstyle{\sum_{i=1}^{|I|}\sum_{j=1}^{N_{u}}}\log\left(\hat{p}^{(i,j)}_{u}\right),$
(4)

where $N_{u}$ denotes the number of users. To further force the identity
predictor to learn features sufficiently discriminative and generalizable to
identify unseen users, we introduce a center loss [3] in training the identity
predictor:

$\mathcal{L}_{C}=0.5~{}\textstyle{\sum_{i=1}^{|I|}}\left\|\mathbf{f}_{i}-\mathbf{c}_{z_{i}}\right\|^{2}_{2},$
(5)

where $\mathbf{c}_{z_{i}}\in\mathbb{R}^{Q}$ is the center for the $z_{i}$-th
class deep features, $\mathbf{f}_{i}\in\mathbb{R}^{Q}$ is the $i$-th deep
feature, and the summation is performed over the input set $I$. We update
$\mathbf{c}_{z_{i}}$ in a mini-batch (with size $m$) manner where the gradient
of $\mathcal{L}_{C}$ with respect to $\mathbf{f}_{i}$ is calculated as
$\mathbf{f}_{i}-\mathbf{c}_{z_{i}}$ with
$\mathbf{c}_{j}=\mathbf{c}_{j}+\frac{\sum^{|I|}_{i=1}\mathcal{I}(z_{i}=j)(\mathbf{c}_{j}-\mathbf{f}_{i})}{1+\sum^{|I|}_{i=1}\mathcal{I}(z_{i}=j)}$,
$\mathcal{I}(z_{i}=j)$ is an indicator function whose value is 1 if $z_{i}=j$;
otherwise 0. We combine the center loss $\mathcal{L}_{c}$ with the categorical
cross-entropy loss $\mathcal{L}_{u}$ to train the identity predictor as
$\mathcal{L}_{\eta}=\mathcal{L}_{u}+\lambda\mathcal{L}_{c}$, where $\lambda$
is a scalar that balances the two losses. This loss function enables ID-Net’s
identity predictor to maximize inter-class margins and minimize intra-class
distances, thereby improving its generalizability.

The loss for training the domain discriminator
$G_{d}(\mathbf{f},{\boldsymbol{\theta}}_{d})$ is also categorical cross-
entropy:

$\mathcal{L}_{\delta}=-|I|^{-1}\textstyle{\sum_{i=1}^{|I|}\sum_{j=1}^{N_{d}}}\log(\hat{\delta}^{(i,j)}),$
(6)

where $\hat{\delta}^{(i,j)}$ is the output probability for the $i$-th footstep
originating from the $j$-th domain and $N_{d}$ denotes the number of domains.
The overall training process aims to minimize both $\mathcal{L}_{\eta}$ and
$\mathcal{L}_{\delta}$ by tuning their respective network parameters
$\mathbf{\theta}_{u}$ and $\mathbf{\theta}_{d}$. In the meantime,
$\mathbf{\theta}_{f}$ is tuned to minimize $\mathcal{L}_{\eta}$ but maximize
$\mathcal{L}_{\delta}$ (via gradient reversal). This procedure forces
$\mathbf{f}$ to retain only user-specific properties but discard those induced
by domains, allowing ID-Net to handle footstep samples taken from unseen
domains.

###  3.4 Thwarting Replay Attack via Spatial Clues

As a user identification system, PURE may be adopted in applications where
security is a concern (e.g., assisting authentication and surveillance). Under
these circumstances, PURE needs to be resilient to potential attacks, among
which replay attack is the most lethal one. Similar to all other acoustic-
related identification methods (e.g., speaker recognition [27]), footsteps can
be overheard (and recorded) by a microphone and then be replayed to attack the
system in the sense of faking a certain user. Fortunately, footsteps contain
dynamic and smoothly changing spatial clues induced by user movements. On the
contrary, a recording clip of footsteps often exhibit only static spatial
characters or may otherwise suggest abnormal trajectories, so it should be
readily recognizable upon fully extracting the spatial clues contained in
footsteps.

To thwart replay attacks, we propose to leverage two pieces of spatial
information, namely Time-of-Arrival (ToA) and Angle-of-Arrival (AoA), to
filter out replayed footsteps. In particular, ToA is extracted from the
structure-borne footstep and AoA is obtained from the air-borne counterpart.
The key rationale is that footsteps from a lively walking person exhibit two
traits: i) the moving speed suggested by ToA is bounded, as one should not
move too fast indoors (it is also suspicious for a real person to move too
fast in a security-concerned area) and ii) the moving trajectory implied by
ToA should be naturally irregular, i.e., no person may move along a straight
line. On the contrary, an replayed footstep clip always violate at least one
of these two criteria, as explained next, regardless of whether the attack is
aware of our countermeasure or not.

We formulate the defense against replay attack as a Hypothesis Test [28], in
which we define significance thresholds to judge whether a footstep can be
accepted as an authentic one or rejected as a replayed one. In our
implementation, two thresholds are defined: i) we use Spearman coefficient
[29] $\pi$ to quantify the correlation between walking speeds $\mathbf{v}$ and
step frequency $\mathbf{f}$, as replayed footsteps often exhibit rather low
correlation while those of authentic footsteps are high. ii) we exploit the
maximum difference between any detected AoA $\gamma_{\mathrm{diff}}$ to check
the motion states: the AoAs should exhibit variance instead of being static.
To be more specific, if $\pi\geq\bar{\pi}$ and
$\gamma_{\mathrm{diff}}\geq\bar{\gamma}_{\mathrm{diff}}$ are both satisfied,
we accept the footsteps; otherwise, we reject them, where $\bar{\pi}=0.8$ and
$\bar{\gamma}_{\mathrm{diff}}=10^{\circ}$ are empirically set based on
measurements.

We apply beamforming techniques [30] to extract AoAs from footstep, but
obtaining ToA, or equivalently range, is non-trivial. Intuitively, a range-
sensitive acoustic fingerprinting strategy based on Sec. 2.1 can be used, but
such fingerprints can be interfered by domain conditions. Therefore, we again
resort to adversarial learning to preclude domain conditions with respect to
ranging. Essentially, we train a neural network, R-Net, to infer range from a
footstep $\mathbf{x}$. This R-Net follows the same design as ID-Net except
that the identity predictor is replaced by a range estimator (for regression)
and the domain conditions are modified accordingly. We omit the training
details given their similarity to those presented in Sec. 3.3.

We emulate a case to show how replayed footsteps exhibit abnormal properties
and can thus be detected. We let a user (hence his/her footsteps) move from
$[1,0]$ m to $[1,3.4]$ m, with a speed of 0.7 m/s (0.1 m/s variance) and a
step frequency of 1 Hz (0.05 Hz variance). In the meantime, the microphone
of PURE is located at $[1.5,2]$ m, while an attacker hides at the origin to
record the footsteps. After a while, the attacker replays the recorded
footsteps, trying to impersonate the legitimate user. As shown in Fig. 9(a),
$\mathbf{v}_{\mathrm{replay}}$ has a significantly lower correlation with
$\mathbf{f}$, compare with that of $\mathbf{v}_{\mathrm{live}}$:
$\pi_{\mathrm{replay}}=0.32<\bar{\pi}=0.8<\pi_{\mathrm{live}}=0.87$.
Meanwhile, Fig. 9(b) shows a fixed $\gamma_{\mathrm{diff}}=0$ for replayed
footsteps, as opposed to the meaningful one for the live ones.

(a) Correlation between $\mathbf{v}$ and $\mathbf{f}$.

(b) AoA vs. range changes.

Figure 9: Emulated scenario to compare (a) walking speed $\mathbf{v}$ vs. step
frequency $\mathbf{f}$ (representing $\pi$) and (b) AoA vs. range changes
(suggesting $\gamma_{\mathrm{diff}}$) between live and replayed footsteps.

##  4 Implementation and Performance Evaluation

(a) A microphone array.

(b) Experiment setting.

Figure 10: Implementation with a microphone array (a) and corresponding
experiment setting for evaluations (b).

(a) Clean footsteps.

(b) Voice.

(c) Mixture.

Figure 11: ASACC calculated from (a) clean footsteps, (b) voice, and (c) a
mixture of clean footsteps and voice.

###  4.1 Implementation

We implement a PURE prototype using a circular array backed by a Raspberry
Pi4, as shown in Fig. 10(a). We configure the sampling rate as 192 kHz, the
highest configuration on this platform, to capture as much structure-borne
signals as possible due to their short duration. However, we downsample the
air-borne footsteps to 16 kHz for identification purpose, in order to achieve
computational efficiency. The signal processing modules (including footstep
detection, GMM based classifier, and FASST audio source separation) are
implemented using C++. The deep learning modules, namely the denoising
network, ID-Net, and R-Net, are implemented using Tensorflow [31]. For the
denoising network, we follow the routines in [25], which allows PURE to
achieve a salient denoising performance and also a low runtime complexity.

ID-Net takes STFT magnitude of dimension $32\times 16$ as its input,
representing a footstep that lasts around $30$ ms. The feature extractor has
two convolution layers each followed by a batch normalization layer and ReLU
activation layer. The first one has 32 filters with a $5\times 3$ kernel and
the second one has 64 filters with a $3\times 2$ kernel. After then, we
flatten the output of the last convolution layer, add a dropout layer with
drop probability of 0.65, and project it into a 16-dimensional feature vector.
The identity predictor and domain discriminator both have only one fully
connected layer with 16 neurons and adopt a sigmoid activation function. Their
respective outputs depend on the number of users and domains involved in the
training set.

For the R-Net, the input is a footstep waveform with 500 samples. The feature
extractor in R-Net has a 1D convolution layer, followed by a pooling layer,
and a fully-connected layer. The corresponding filter size is 64, 16, and 16,
respectively. A dropout layer with a probability of 0.5 is inserted before the
fully-connected layer. The range estimator, together with the domain
discriminator has only one hidden layer, and both has a filter size of 16. To
gather training data for R-Net, we first deploy a centimeter-level
localization system using Decwave UWB-based sensors [32]. We tie one sensor on
a user’s foot when he/she walks so as to obtain ground truth locations, i.e.,
ToA and AoA labels. Also, we attach an IMU sensor on the user’s leg to help
triggering the microphone array, so that it may correctly capture a footstep.

We synthesize a training data set for the denosing network with clean
footsteps from [33, 34] and speech from TED talks [35]. The noise is extracted
from Diverse Environments Multichannel Acoustic Noise Database (DEMAND) [36].
We also collect footsteps under common floors (wood, stone) and circumstances
(hall, indoor office, home appliance). We configure the signal SNR in a range
of 5 dB, 10 dB, 20 dB, and 30 dB during training. We also utilize the
footsteps from source separation to train the network in gaining the ability
of minimizing residual interference signals.

###  4.2 Performance Evaluation

We present extensive experiments in this section. We start with evaluating the
source separation algorithm, followed by the denoising network. Then we
seriously verify the identification accuracy. Finally, we report performance
of defending against the replay attack. The experimental statistics, unless
otherwise noted, are all obtained by repeating the same experiment 1,000
times.

####  4.2.1 Source Separation Performance

As we mentioned in Sec. 3.2, the source separation module is activated if we
detect rhythmic features in STFT spectrogram. Therefore, before source
separation, we first evaluate the performance of this detection algorithm.
Recall that we utilize the ASACC $\mathbf{b}$, calculated by Eqn. (3) for
footstep detection, as the strongest energy of footsteps lies in the low
frequency range, we thus only use the first three bins. As shown in Fig. 11,
only footsteps that exhibit rhythmic features can generate periodic peaks in
ASACC of Fig. 11(a), whereas voice signals hold no such properties in Fig.
11(b). And when footsteps are mixed with voice, using ASACC can still identify
this rhythmic feature as shown in Fig. 11(c). The detailed steps of this
detection algorithm proceed as follows. After obtaining $\mathbf{b}$, we
estimate the beating frequency $k$ in $\mathbf{b}$ via Discrete Fourier
Transform. If the magnitude $m_{k}$ of bin $k$ in spectrum goes beyond the
average of its subsequent 20 bins by a certain threshold (10 in our case),
namely $m_{k}>{\frac{1}{20}\sum^{k+20}_{i=k}m_{i}}+10$, we accept that current
audio signals contain footsteps; otherwise, no footstep is detected and source
separation is deactivated. This detection method allows us to achieve a 100%
footstep detection even when the magnitude of voice is higher than footstep.

To inspect the source separation performance, we first use clean footsteps
blended with voice signals under different configurations as inputs and then
check the quality of separated signals. Specifically, we synthesize mixtures
of footstep and voice under different Source to Interference Ratio (SIR) [37]
as inputs and evaluate the performance using SIR and Source to Distortion
Ratio (SDR) [37]. These two evaluation matrices, namely SIR and SDR, are
widely used to quantify source separation performance where
$\mathrm{SIR}=10\log\frac{\|\mathbf{s}_{\mathrm{tgt}}\|^{2}_{2}}{\|\mathbf{e}_{\mathrm{itf}}\|^{2}_{2}}$,
$\mathrm{SDR}=10\log\frac{\|\mathbf{s}_{\mathrm{tgt}}\|^{2}_{2}}{\|\mathbf{e}_{\mathrm{itf}}+\mathbf{e}_{\mathrm{nie}}+\mathbf{e}_{\mathrm{atf}}\|^{2}_{2}}$,
with
$\mathbf{s}_{\mathrm{tgt}},\mathbf{e}_{\mathrm{itf}},\mathbf{e}_{\mathrm{nie}}$,
and $\mathbf{e}_{\mathrm{atf}}$ being respectively the target signal,
interference signal, noise signal, and signal artifacts. The higher the value
of these matrices, the better the target signal quality is. Both SIR and SDR
in this experiment are calculated given footsteps as the primary signals, as
opposed to common speech enhancement tasks where voices are the major
concerns.

(a) SIR before separation vs SDR after separation.

(b) SIR before separation vs SIR after separation.

Figure 12: Source separation performance. SDR (a) remains almost constant and
SIR (b) is enhanced by source separation.

The results are shown in Fig. 12. From Fig. 12(a), we can see that the SDR
after source separation remains almost constant under different SIRs. This
simply implies that the source separation algorithm introduces little
distortion to the original footsteps, which is notably important for our later
identification performance. As a matter of fact, we can barely perceive any
distortion when playing the separated footsteps, except some residual voices.
It is observable that after source separation in Fig. 12(b), SIR is
significantly boosted, indicating a success removal of voice interference.

(a) Time domain waveform.

(b) STFT spectrogram.

Figure 13: Time domain waveform (a) and STFT spectrogram (b) of mixed,
original, and separated footsteps.

We finally showcase waveforms and STFT spectrogram of footsteps after source
separation, compared with the mixed and original recorded ones, in Fig. 13(a)
and Fig. 13(b), respectively. In this experiments, the maximum voice magnitude
is identical to that of footsteps, under which case, a user mostly notice
voice but ignores footstep hence interference is strong. Though under severe
interference, the separated footstep waveform is already rather clean as we
can see from both Fig. 13(a) and Fig. 13(b): minor distortions and residuals
may exist, but none of them introduce perceivable artifacts.

####  4.2.2 Denoising Performance

The denoising network is used to filter out interference from the background
subtraction and get rid of residual signals from the source separation. To
evaluate the denoising performance, we synthesize mixtures of real-life
recorded footsteps and voices under different SNRs and SDRs, generating a
total of 1400 sound clips whose duration is within 20 s. Then we check the
respective SNR and SDR after denosing.

Our measurements in Fig. 14(a) reveal a maximum SNR gain around 30 dB
(depending on the background noise type). The SNR gain can be noticeable when
the SNR of input noisy signals is relatively low, e.g., below 20 dB. However,
the network introduces little distortion to its inputs when SNR is high. The
same goes for SDR as shown in Fig. 14(b). But this little distortion
introduces no perceivable difference to the inputs. Meanwhile, we can
deactivate the denoising module when SNR or SDR is high so as to prevent the
possible distortion since the SNR or SDR can be roughly calculated. We finally
showcase our denoising network in residual removal in Fig. 15. Both time
domain waveform (Fig. 15(a)) and Frequency domain spectrogram (Fig. 15(b))
indicate the success of residual noise removal. The noise removal effect can
be visualized by the less signal magnitude variations in time domain waveform
and the less “blurred image” in spectrogram.

(a) SNR.

(b) SDR.

Figure 14: SNR (a) and SDR (b) before and after denoising.

(a) Time domain waveform.

(b) STFT spectrogram.

Figure 15: Denoising performance for residual removal.

####  4.2.3 Identification Performance

(a) Accuracy over SNR.

(b) Accuracy over SIR.

Figure 16: Identification accuracy at various SNRs and SIRs.

We conduct extensive measurements to evaluate the identification performance
utilizing real-life recorded footsteps from six users. In our first study, we
utilize data samples from the same domain (the same environment and walking
speed) but only vary the user identity. In this study, we deactive center loss
and apply no adversarial learning for ID-Net. The results shown in Fig. 16(a)
reveal that even under a SNR of -12.5 dB, ID-Net still achieves 87% accuracy,
demonstrating the feasibility of applying footstep for user identification.

(a) Identification accuracy after different processing methods.

(b) Impact of sampling rate.

Figure 17: (a) Identification accuracy improved after each processing step.
(b) Improving the sampling rate can boost the identification accuracy at low
SNR.

It should be noted that we only utilize one step for identification, if we
incorporate multiple footsteps and use a majority voting strategy, the
accuracy can be boosted to $1-C^{2}_{3}\times\left(1-0.90\right)^{2}\times
0.9-C_{3}^{3}(1-0.9)^{3}=97.2$%.

We then check the identification accuracy under different levels of voice
interference (SIR). The results in Fig. 16(b) show that even under severe
interference (SIR = 0), ID-Net can still achieve an accuracy up to 87.13%. And
if adding noise and thereby reducing the SNR for footsteps, the accuracy would
drop to 60%, indicating ID-Net’s vulnerability to strong background
interference. This also emphasizes the need for source separation and
denoising, as the former can deliver SIR gain and the latter provides SNR
gain, thereby promoting the identification accuracy.

We next extensively explore the identification accuracy after source
separation and denoising. The results shown in Fig. 17(a) reveals that source
separation can achieve a maximum accuracy gain of 59.9% while denoising
network can boost the performance by an average of 5%. We then explore the
impact of sampling rate on the final identification performance and Fig. 17(b)
shows the results. It can be observed that improving the sampling rate can
contribute to better identification performance. The performance gain is
marginal when SNR is sufficiently high ($>20$ dB), if the sampling rate hits
48 kHz. Since a higher sampling rate requires more computational power but
achieves little performance improvement, we therefore adopt 16 kHz in our
system.

(a) Low-dimensional features without center loss.

(b) Low-dimensional features with center loss.

Figure 18: Low-dimensional feature visualization without (a) and with (b)
center loss. It clearly shows the power of center loss in maximizing inter-
class boundaries while minimizing intra-class distances.

We next conduct an ablation study on the effectiveness of center loss, the
impact of which on the classified features can be visualized in Fig. 18. It is
observable that center loss can effectively maximize the inter-class
boundaries and minimize intra-class distances. This ability could not only
enhance the identification performance but also improve the generalizability
of ID-Net. Applying center loss sometimes can push the identification accuracy
to almost 100%.

TABLE I: Accuracy without domain adaptation. Accuracy (%) | Distance | Speed | Environment
---|---|---|---
Distance | 76.4 | 57.2 | 9.11
Speed | 57.2 | 62.01 | 14.31
Environment | 9.11 | 14.31 | 12.04

We test the identification performance under different domains including
speed, distance, and environment variations in the following experiments.
Specifically, our footsteps are captured under: 1) three levels of walking
speed, namely 0.2 m/s, 0.5 m/s, and 1 m/s, 2) different distances ranging
from $0$ to $3$ m, 3) heterogeneous environments including common indoor
office, home appliance, hall, corridor, etc that exhibit different ground
materials and background interference.

We first show the impact of domains on the identification performance when we
deactivate center loss and domain predictor. The results are displayed in
Table I and they tells us that domain conditions can have a notable impact on
the identification accuracy. To read the statistics in Table I, each row and
column indicate the number of domains involved in the training data. For
instance,
$(\text{row},\text{column})=\left(\text{Speed},\text{Distance}\right)=57.2\%$
means when the training data involves speed and distance variations, the
identification accuracy is 57.2%. According to Eqn. (2.1), distance should not
impose any negative impacts on the final results. But when data only contains
distance variations, the identification accuracy is only 76.42%. We believe
that this is caused by 1) SNR degradation due to propagation loss and 2)
structural differences from place to place that cause heterogeneous features.
Speed variations, equivalently leading to different impact forces, sabotage
the identification accuracy to only 62.01%. And environment dynamics,
introducing different medium properties, undermine the identification accuracy
most.

TABLE II: Accuracy with domain adaptation. Accuracy (%) | 0 Domain | 2 D. | 2 D. | 3 D.
---|---|---|---|---
One footstep | 1 | 88.6 | 84.92 | 81.75
2/3 | 1 | 95.92 | 90.33 | 88.16
3/5 | 1 | 96.53 | 94.47 | 90.73

We next explore the identification accuracy under domain adaptation.
Particularly, we evaluate the identification accuracy under different number
of domains and number of footsteps. The average results from 100 trials are
shown in Table II and “2/3” means we incorporate three steps to identify each
user and we accept the result if the same identity appears twice. It can be
observable that domain adaptation can significantly improve identification
accuracy, pushing the resulting accuracy from a minimal of 9% (without domain
adaptation) to 81.75%. And if incorporating multiple footsteps, the accuracy
can be further improved to 90.73%.

(a) Identification accuracy under one footstep interference.

(b) Identification accuracy under multiple footsteps.

Figure 19: Identification accuracy when one footstep is mixed with partial
another footstep components (a). Such a collision can reduce the
identification accuracy. Identification accuracy drops when the number of
users increases (b). As collision rate increases when more users are involved,
the accuracy drops.

We finally explore the identification accuracy under multiple user scenarios.
We first explore the identification accuracy under the case when each footstep
is interfered by only one another footstep. We check the identification
accuracy when the footsteps are overlapped at different percentages, the
results of which are displayed in Fig. 19(a). As the figure tells, the
identification accuracy drops monotonically if the percentage of overlapped
region increases. However, the accuracy is still around 50% if two steps are
totally overlapped. This simply implies that ID-Net can still recognize these
two footsteps but is unable to distinguish them. We then evaluate the
performance when multiple person randomly walk in an indoor meeting room where
we place the microphone array in the center. The results in Fig. 19(b) show
that an identification accuracy around 80% can still be achieved even when
there are three users. As the number of user increases, the collision between
different footsteps happens more frequently hence worse performance.

####  4.2.4 Defend Against Replay Attack

PURE leverages R-Net to extract spatial information, including range and AoA,
from multiple consecutive footsteps to defend against replay attack. In this
section, we first present the performance of R-Net in spatial information
extraction and then inspect the defending performance based on these
signatures. We run over 1000 trials in an indoor office $6.8\times 4.2$ m2
where we place the microphone array in the center.

Fig. 20 shows the ranging and AoA estimation performance. In Fig. 20(a), We
verify the ranging performance of R-Net under three cases to demonstrate its
capability in domain adaptation. First, we utilize training data (70% of all
data) from all the identities and domains to train R-Net, and we then verify
the ranging performance using test data (the remaining 30%), which we refer to
as Test with Domain Adaptation (Test w/DA). Second, we randomly remove one
identity from the training data and after training, we apply inference on this
particular identity, referred as Test new samples with Domain Adaptation (Test
new samples w/DA). Third, we cut the domain predictor from R-Net and apply
inference using the same setting with the second case, denoted as Test new
samples without Domain adaptation (Test new samples w/o DA). The results in
Fig. 20(a) exhibit an median error of around 0.3 m, even testing on samples
that come from other domains and never participant in the training process.
And if without domain adaptation, the median ranging errors would reach up to
1 m. The comparison of afore-mentioned results clearly demonstrates the
salient performance of R-Net in domain adaptation, as well as in ranging. Fig.
20(b) shows AoA estimation errors are below $10^{\circ}$. These salient
performance lays the foundation of our defending mechanism against replay
attack.

(a) Ranging performance.

(b) AoA estimation performance.

Figure 20: Performance in spatial clues extraction.

We have checked our defending mechanism under several replay attack scenarios
including different walking trajectories and hacking positions. Our
measurements reveal that if the trajectories contain complex shapes such as
“L” or circle shape segments, the detected $\pi$ could easily violates the
threshold $\bar{\pi}$ so that we achieve 100% success in defending these
attacks. When there involves only straight line trajectories, the variance of
detected AoA revealed by replayed attacks never exceeds the preset
$10^{\circ}$ threshold while live footsteps reveal a minimum variation around
32.8∘ due to location swing caused by the alternation between left and right
legs. In conclusion, PURE can successfully defend against replay attack.

##  5 Related Work

In this section, we survey the literature on user identification. Whereas
common identification techniques have a broad categories, ranging from
traditional computer vision, fingerprint sensing, and iris scan, they are
rather irrelevant to our proposal. Therefore, we shall not review these common
techniques but focus only on solutions that leverage emergent sensing
techniques and adopt behavioral biometrics for user identification.

The proposals of [11, 10, 8, 9] exploit the gait information to identify users
during their walking. The basic idea behind these systems is that the
particular walking cycle of each user can be sampled by WiFi signals. However,
they may not be able to adapt to environment dynamics, and even walking
direction variations can severely affect the identification accuracy.
Meanwhile, they often fail to work in practice due to the severe interference
from WiFi’s main function of communications and other co-spectrum devices.

FootprintID [14] is a structural vibration based identification system. It
employs Gephone [38] to sense the structural vibration caused by a footstep.
For identification, it again relies on the gait patterns extracted from
multiple structural vibration measurements; the reported identification
accuracy for 10 people may reach up to 96%. However, this promising solution
still leaves many open issues, including sensor location variation, multiple
pedestrians interference, footwear variation. Other similar behavioral
biometrics enabled identification system can be found in [39, 40, 41], a well
as [13] where accelerometer and camera are used together as sensors.

To summarize, existing technologies driven by behavioral biometrics often
require multiple measurements hence long latency identification experience. On
the contrary, PURE solves this problem elegantly by requiring as few as only
one step. While PURE can be deemed as a type of behavioral biometrics, it is
actually quite related to voiceprint recognition [42, 43]; it can be deemed as
a “footstep-print” enabled identification system. PURE is similar to those
acoustic fingerprint based systems [42, 43] but PURE is totally passive and
thus can provide better user experience. The most similar work to PURE is the
one from [15] where footstep patterns rather than gaits are used for
identification. But this proposal requires an excessive number of
piezoelectric sensors to capture footstep signals while PURE utilizes only
commodity microphone, significantly reducing the deployment cost and rendering
itself widely applicable for indoor scenarios.

##  6 Conclusion

In this paper, we have explored the possibility of exploiting footsteps for
passive user identification. We have proposed PURE as a multi-person
identification system driven by a pipeline of signal processing and deep
learning techniques. PURE demands as few as a single footstep to enable user
identification and is immune to replay attacks. PURE is even feasible to work
under continuous voice interference, thanks to a novel source separation and
denoising network. To have PURE working across different domains, we have
exploited domain adversarial adaptation scheme with a center loss to further
enhance its generalization ability across different domains. We have
implemented a prototype for PURE and extensively evaluated its performance;
the results confirm that PURE achieves a cross-domain identification accuracy
up to 90%. Since PURE outperforms existing passive identification system in
both deployment cost and identification latency, we have the reason to believe
that PURE has the potential for a wide adoption.

## References

  * [1] W. Liu, Y. Wen, Z. Yu, M. Li, B. Raj, and L. Song, “SphereFace: Deep Hypersphere Embedding for Face Recognition,” in _Proc. of IEEE/CVF CVPR_ , 2017, pp. 6738–6746.
  * [2] J. Deng, J. Guo, N. Xue, and S. Zafeiriou, “ArcFace: Additive Angular Margin Loss for Deep Face Recognition,” in _Proc. of IEEE/CVF CVPR_ , 2019, pp. 4685–4694.
  * [3] Y. Wen, K. Zhang, Z. Li, and Y. Qiao, “A Discriminative Feature Learning Approach for Deep Face Recognition,” in _Proc. of ECCV_ , 2016, pp. 499–515.
  * [4] F. Schroff, D. Kalenichenko, and J. Philbin, “FaceNet: A Unified Embedding for Face Recognition and Clustering,” in _Proc. of IEEE/CVF CVPR_ , 2015, pp. 815–823.
  * [5] Bin Fang and Yuan Yan Tang, “Elastic Registration for Retinal Images based on Reconstructed Vascular Trees,” _IEEE Transactions on Biomedical Engineering_ , vol. 53, no. 6, pp. 1183–1187, 2006.
  * [6] A. K. Jain, S. Prabhakar, and Lin Hong, “A Multichannel Approach to Fingerprint Cassification,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 21, no. 4, pp. 348–359, 1999.
  * [7] A. Jain, L. Hong, and S. Pankanti, “Biometric Identification,” _Communications of the ACM_ , vol. 43, no. 2, p. 90–98, 2000.
  * [8] W. Wang, A. X. Liu, and M. Shahzad, “Gait Recognition Using WiFi Signals,” in _Proc. of ACM UbiComp_ , 2016, p. 363–373.
  * [9] C. Shi, J. Liu, H. Liu, and Y. Chen, “Smart User Authentication through Actuation of Daily Activities Leveraging WiFi-Enabled IoT,” in _Proc. of ACM Mobihoc_ , 2017, pp. 5:1–10.
  * [10] Y. Zeng, P. H. Pathak, and P. Mohapatra, “WiWho: WiFi-Based Person Identification in Smart Spaces,” in _Proc. of ACM/IEEE IPSN_ , 2016, pp. 1–12.
  * [11] J. Zhang, B. Wei, W. Hu, and S. S. Kanhere, “WiFi-ID: Human Identification using WiFi Signal,” in _Proc. of IEEE DCOSS_ , 2016, pp. 75–82.
  * [12] P. Connor and A. Ross, “Biometric recognition by gait: A survey of modalities and features,” _Comput. Vis. Image Underst._ , vol. 167, pp. 1–27, 2018\.
  * [13] Y. He, J. Zhang, H. Shan, and L. Wang, “Multi-Task GANs for View-Specific Feature Learning in Gait Recognition,” _IEEE Transactions on Information Forensics and Security_ , vol. 14, pp. 102–113, 2019.
  * [14] S. Pan, T. Yu, M. Mirshekari, J. Fagert, A. Bonde, O. J. Mengshoel, H. Y. Noh, and P. Zhang, “FootprintID: Indoor Pedestrian Identification through Ambient Structural Vibration Sensing,” in _Proc. of ACM UbiComp_ , 2017, pp. 89:1–31.
  * [15] R. Vera-Rodríguez, J. Mason, J. Fierrez, and J. Ortega-Garcia, “Comparative Analysis and Fusion of Spatiotemporal Information for Footstep Recognition,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 35, pp. 823–834, 2013.
  * [16] A. Ross and G. Ostiguy, “Propagation of the Initial Transient Noise From an Impacted Plate,” _Elevier Journal of Sound and Vibration_ , vol. 301, pp. 28–42, 03 2007.
  * [17] S. Nakagawa, L. Wang, and S. Ohtsuka, “Speaker Identification and Verification by Combining MFCC and Phase Information,” _IEEE Transactions on Audio, Speech, and Language Processing_ , vol. 20, no. 4, pp. 1085–1095, 2012.
  * [18] J. Portelo, M. Bugalho, I. Trancoso, J. Neto, A. Abad, and A. Serralheiro, “Non-speech Audio Event Detection,” in _2009 IEEE International Conference on Acoustics, Speech and Signal Processing_ , 2009, pp. 1973–1976.
  * [19] RaspberryPi, “ReSpeaker 6-Mic Circular Array Kit for Raspberry Pi,” https://wiki.seeedstudio.com/ReSpeaker_6-Mic_Circular_Array_kit_for_Raspberry_Pi/, accessed: 2020-08-08.
  * [20] S. Kamath and P. Loizou, “A Multi-band Spectral Subtraction Method for Enhancing Speech Corrupted by Colored Noise,” in _Proc. of IEEE ICASSP_ , 2002, pp. 4164–4172.
  * [21] E. Yuce, S. Minaei, and S. Tokat, “Root-Mean-Square Measurement of Distinct Voltage Signals,” _IEEE Transactions on Instrumentation and Measurement_ , vol. 56, no. 6, pp. 2782–2787, 2007.
  * [22] A. Ozerov, E. Vincent, and F. Bimbot, “A General Flexible Framework for the Handling of Prior Information in Audio Source Separation,” _IEEE Transactions on Audio, Speech and Language Processing_ , vol. 20, no. 4, pp. 1118 – 1133, May 2012.
  * [23] A. Rai, K. K. Chintalapudi, V. N. Padmanabhan, and R. Sen, “Zee: Zero-Effort Crowdsourcing for Indoor Localization,” in _Proceedings of the 18th Annual International Conference on Mobile Computing and Networking_ , 2012, p. 293–304.
  * [24] Z. Rafii and B. Pardo, “REpeating Pattern Extraction Technique (REPET): A Simple Method for Music/Voice Separation,” _IEEE Transactions on Audio, Speech, and Language Processing_ , vol. 21, no. 1, pp. 73–84, 2013.
  * [25] A. Defossez, G. Synnaeve, and Y. Adi, “Real Time Speech Enhancement in the Waveform Domain,” 2020.
  * [26] Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky, “Domain-Adversarial Training of Neural Networks,” _The Journal of Machine Learning Research (JMLR)_ , vol. 17, no. 1, pp. 2096–2030, Jan. 2016.
  * [27] A. Poddar, M. Sahidullah, and G. Saha, “Speaker Verification with Short Utterances: A Review of Challenges, Trends and Opportunities,” _IET Biometrics_ , vol. 7, no. 2, pp. 91–101, 2018.
  * [28] N. Jerzy and S. P. Egon, “On the problem of the most efficient tests of statistical hypotheses,” _Philosophical Transactions of the Royal Society of London_ , 1933.
  * [29] J. H. Zar, “Significance Testing of the Spearman Rank Correlation Coefficient,” _Journal of the American Statistical Association_ , vol. 67, no. 339, pp. 578–580, 1972.
  * [30] J. C. Chen, K. Yao, and R. E. Hudson, “Source Localization and Beamforming,” _IEEE Signal Processing Magazine_ , vol. 19, no. 2, pp. 30–39, 2002.
  * [31] TensorFlow, “An End-to-End Open Source Machine Learning Platform,” https://www.tensorflow.org/, accessed: 2020-08-08.
  * [32] Decwave, “EVK1000 Evaluation Kit,” https://www.decawave.com/product/evk1000-evaluation-kit/, accessed: 2020-11-11.
  * [33] F. Studio, “Footsteps Sound Effects Soundboard,” https://www.fesliyanstudios.com/royalty-free-sound-effects-download/footsteps-31?soundboard=1, 2020, accessed: 2020-11-6.
  * [34] E. Sound, “Footsteps Sound Effects,” https://www.epidemicsound.com/sound-effects/footsteps/, 2020, accessed: 2020-11-6.
  * [35] TED, “TED: ideas worth spreading,” https://www.ted.com, 2020, accessed: 2020-11-6.
  * [36] J. Thiemann, N. Ito, and E. Vincent, “The Diverse Environments Multi-Channel Acoustic Noise Database (DEMAND): A database of multichannel environmental noise recordings,” _The Journal of the Acoustical Society of America_ , vol. 133, p. 3591, 2013.
  * [37] E. Vincent, R. Gribonval, and C. Fevotte, “Performance Measurement in Blind Audio Source Separation,” _IEEE Transactions on Audio, Speech, and Language Processing_ , vol. 14, no. 4, pp. 1462–1469, 2006.
  * [38] J. M. Reynolds, _An introduction to applied and environmental geophysics; 2nd ed._ Hoboken, NJ: Wiley, 2011.
  * [39] D. Gafurov, E. Snekkenes, and P. Bours, “Gait Authentication and Identification Using Wearable Accelerometer Sensor,” in _2007 IEEE Workshop on Automatic Identification Advanced Technologies_ , 2007, pp. 220–225.
  * [40] H. J. Ailisto, M. Lindholm, J. Mantyjarvi, E. Vildjiounaite, and S.-M. Makela, “Identifying People from Gait Pattern with Accelerometers,” in _Biometric Technology for Human Identification II_ , vol. 5779, 2005, pp. 7–14.
  * [41] L. Rong, D. Zhiguo, Z. Jianzhong, and L. Ming, “Identification of Individual Walking Patterns Using Gait Acceleration,” in _2007 1st International Conference on Bioinformatics and Biomedical Engineering_ , 2007, pp. 543–546.
  * [42] N. Obin and A. Roebel, “Similarity Search of Acted Voices for Automatic Voice Casting,” _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , vol. 24, no. 9, pp. 1642–1651, 2016.
  * [43] R. A. Rashid, N. H. Mahalin, M. A. Sarijari, and A. A. Abdul Aziz, “Security system using biometric technology: Design and implementation of Voice Recognition System (VRS),” in _2008 International Conference on Computer and Communication Engineering_ , 2008, pp. 898–902.