# Deep Learning-based Real-time Smartphone Pose Detection for Ultra-wideband
Tagless Gate

Junyoung Choi, Sagnik Bhattacharya  Communications Standards Research Team,
Samsung Electronics, Seoul, South Korea
Email: {juny.choi<EMAIL_ADDRESS>
###### Abstract

As commercial interest in proximity services increased, the development of
various wireless localization techniques was promoted. In line with this
trend, Ultra-wideband (UWB) is emerging as a promising solution that can
realize proximity services thanks to centimeter-level localization accuracy.
In addition, since the actual location of the mobile device (MD) on the human
body, called pose, affects the localization accuracy, poses are also important
to provide accurate proximity services, especially for the UWB tagless gate
(UTG). In this paper, a real-time pose detector, termed $D^{3}$, is proposed
to estimate the pose of MD when users pass through UTG. $D^{3}$ is based on
line-of-sight (LOS) and non-LOS (NLOS) classification using UWB channel
impulse response and utilizes the inertial measurement unit embedded in
smartphone to estimate the pose. $D^{3}$ is implemented on Samsung Galaxy
Note20 Ultra (i.e., SM-N986B) and Qorvo UWB board to show the feasibility and
applicability. $D^{3}$ achieved an LOS/NLOS classification accuracy of 0.984,
and ultimately detected four different poses of MD with an accuracy of 0.961
in real-time.

###### Index Terms:

Ultra-wideband, tagless gate, deep learning

##  I Introduction

Recently, Ultra-wideband (UWB) is emerging as a promising solution for
proximity services with centimeter-level localization performance compared to
other narrowband radios such as Wi-Fi and Bluetooth [1, 2]. UWB-based
proximity services, such as UWB tagless gate (UTG) [3], UWB payment [4], and
UWB car key [5, 6] are preparing to come out to the world with a lot of
attention due to commercial and social values. In addition, the flagship
smartphones of Samsung (e.g., Galaxy Note 20 Ultra and later) are equipped
with UWB module.

The mentioned applications are based on the distance between the two UWB
modules to ensure proximity, and the most common way to calculate the distance
is double-sided two-way ranging (DS-TWR). However, even if the DS-TWR-based
ranging result provides a centimeter level of localization accuracy, the
actual location of the user may be different from that of the mobile device
(MD) [7]. For example, when the user approaches the UTG, the MD and the UTG
perform DS-TWR to smoothly open the gate when the user enters a specific area
of the gate. In this situation, due to the position of MD on the human body,
which is called pose in this paper, the user can experience unexpected
situations as shown in Fig. 1.

When the user is close enough to the gate, the gate must be automatically
opened without causing any interference to the user, as shown in Fig. 1(a).
However, if the MD is in the back pocket, the gate might not be opened because
of the incorrect ranging results due to a blockage, signal attenuation, and
multipath, or the gate might be opened too early when the MD is in front of
the body, as shown in Fig. 1(b). Due to the difference between the actual
location of the human body and the pose of the MD, both unexpected behaviors
occur. These unexpected operations may be resolved when the opening of the
gate is adaptively operated based on the pose of the MD.

In this paper, we present $D^{3}$, Deep learning-based pose Detection of the
MD on UWB DS-TWR. We newly design and implement a novel pose detection system
based on UWB channel impulse response (CIR) and inertial measurement unit
(IMU) sensor. We define four different poses of the MD, and classify them with
two steps using deep learning method. First, $D^{3}$ collects CIR information
during DS-TWR operation and determines whether the signal is line-of-sight
(LOS) or non-LOS (NLOS). When performing LOS/NLOS classification, $D^{3}$
sorts out effective CIR (eCIR) to minimize the input size of the CNN model,
and hence, achieves real-time operation. Finally, $D^{3}$ determines the pose
of the MD based on the LOS/NLOS classification result.

(a) Normal operation

(b) Unexpected operations

Figure 1: The normal and unexpected operations of UTG. The opening and closing
of the gate are colored in green and red, respectively.

Our major contributions are summarized as follows:

  * •

As far as we know, this is the first work to propose real-time LOS/NLOS
classification and pose detection operating on Android OS with help of the UWB
chipset. We believe that our research can be an important technology to
encourage UTG proximity service using UWB.

  * •

Based on deep learning models, $D^{3}$ is designed by combining the LOS/NLOS
classifier and pose detector. To realize a real-time operation, only eCIRs are
used to classify the LOS/NLOS condition to minimize the data transfer latency
between the smartphone and UWB board.

  * •

We implemented $D^{3}$ in the commercial Android OS and Qorvo UWB module, and
evaluated its performance to show the feasibility. $D^{3}$ classified LOS/NLOS
condition of four poses with an accuracy of 0.984 and achieved an accuracy of
0.961 for pose detection in real-time.

Figure 2: The procedure of DS-TWR with ranging messages.

##  II Background and related work

###  II-A Background

UWB is a wireless communication technique that occupies a wide bandwidth over
500 MHz, and hence, this technique uses radio frequency pulses with a very
short time-duration, resulting in high time resolution. To measure the
distance between two UWB devices, the IEEE 802.15.4z standard, published in
2020, defines DS-TWR for ranging [8]. DS-TWR is an extension of single-sided
two-way ranging in which two round-trip time measurements are used and
combined to give the time-of-flight (ToF) result. The operation of DS-TWR is
shown in Fig. 2. The initiator measures the first round-trip time by sending
ranging initiation message (RIM) and the responder replies by sending ranging
reply message (RRM). After sending RRM, the responder measures the second-trip
time to which the initiator replies by sending ranging final message (RFM).
Each device precisely measures the transmission and reception times of the
messages, and the resultant of ToF can be estimated by the following equation:

${ToF}=\frac{T_{round1}T_{round2}-T_{reply1}T_{reply2}}{T_{round1}+T_{round2}+T_{reply1}+T_{reply2}}.$
(1)

###  II-B Related work

To the best of our knowledge, this is the first work to implement real-time
pose detection based on UWB on commercial smartphones, including LOS/NLOS
classification. The previous work focuses on LOS/NLOS classification itself or
UWB error mitigation based on the classification results, and none of the
previous researches have considered how to use the classification results for
practical UWB application. We are focusing on applications such as pose
detection by combining IMU sensor data for proximity services as well as real-
time LOS/NLOS classification.

Jiang  _et al._ [9] propose CNN-LSTM deep learning method for UWB LOS/NLOS
classification based on CIR, which does not consider the input size of CIR
samples for real-time operation on MD. Jiang  _et al._ [10] focus on de-nosing
the CIR data to identify NLOS signal based on CNN model, and test the effect
of the de-nosing CIR method. Wymeersch  _et al._ [11] present the approach to
directly mitigate ranging errors in both LOS and NLOS conditions. Zeng  _et
al._ [12] take consideration of the necessary size of CIRs among whole CIR
samples, but does not conduct any implementation and real-world experiment.
Barral  _et al._ [13] employ machine learning techniques to analyze several
sets of real UWB measurements to identify the NLOS propagation condition, and
conduct experiments with diverse machine learning algorithms.

Figure 3: The example of CIR data including channel diagnostics. The values of
FP_INDEX and maxNoise are 747 and 1409, respectively.

(a) LOS-HAND

(b) NLOS-HAND

(c) FRONT

(d) BACK

Figure 4: The illustration of four different poses for LOS and NLOS.

##  III Preliminary: UWB data collection

###  III-A Channel impulse response and received signal quality

We use Qorvo MDEK1001 equipped with DWM1001 UWB module to collect CIR data and
the received signal quality [14]. Both data are collected from the responder
after receiving RFM during DS-TWR operation. First, the way to obtaining CIRs
is access to the register file and calculating the complex value (i.e., a
16-bit real integer and a 16-bit imaginary integer), and then 1016 CIRs per
signal can be generated from UWB module111The span of the CIR data is one
symbol time. This is 992 (1016) samples for the nominal 16 (64) MHz mean pulse
repetition frequency.. In addition, 8 kinds of information about the received
signal quality include first path index (FP_INDEX), first path amplitude
(FP_AMPL), and the maximum noise (maxNoise)222The APIs, such as
dwt_readaccdata() and dwt_readdiagnostics(), are used for CIRs and diagnostics
collection. Each CIR index interval is 1 ns.

Fig. 3 shows the value of CIR from 720 to 820 among total 1016 CIRs and three
diagnostics. The horizontal line, black stem, and blue stems represent
FP_INDEX, FP_AMPL, and maxNoise, respectively. The FP_INDEX index gives the
actual signal reception timing to help find the start of a meaningful CIR. The
three consecutive indices after FP_INDEX are three FP_AMPLs, including the
signature of the received signal whether the signal is LOS or NLOS. Based on
the above information, the signals can be classified as LOS or NLOS.

###  III-B Definition of four poses of the mobile device

We adopt deep learning model for LOS/NLOS classification, and hence, the CIR
samples with the ground truth value of LOS/NLOS are essential. In order to
generate the LOS/NLOS signals, four representative poses of the MD are defined
as shown in Fig. 4. Note that the pose refers to the position of the MD on the
human body. We first divide the pose into two conditions, such as LOS or NLOS,
and both conditions can exist when the MD is in the hand or pocket. Thus, each
combination of $\\{$LOS, NLOS$\\}\times\\{$HAND, POCKET$\\}$ will be one of
the poses of the MD. First, LOS-HAND is the pose in which the user holds the
MD without blocking the UWB antenna (Fig. 4(a)). However, in the case of NLOS-
HAND, the LOS path is blocked because the user covers the UWB antenna by hand
(Fig. 4(b)). If the MD is in the front pocket or back pocket, it is considered
as an LOS or NLOS environment, respectively. Since we are considering the
scenario that the user approaches the gate, the MD can receive the LOS path
directly from the gate if it is in the front pocket, which is called
FRONT333We intentionally took a picture of the UWB antenna outside the front
pocket to show the position of MD more clearly. During the actual experiments,
the MD is completely in the front pocket. (Fig. 4(c)). Conversely, when the MD
is in the back pocket, the signal becomes NLOS because the human body blocks
the signal, which is called BACK (Fig. 4(d)).

Figure 5: The diagnostics/CIR and IMU data collection during DS-TWR.

###  III-C CIR and IMU data collection

In order to train the classification model, CIR and IMU data with labels
including the ground truth of LOS/NLOS and pose are required. Based on the
four different poses of the MD presented in Fig. 4, CIR and IMU data samples
through preliminary experiments. Fig. 5 shows the concept of data collection.
We assume that the user gets close to the gate while the MD and the gate
perform DS-TWR. The ranging interval is set to as the default value of 200 ms,
Therefore, the MD collects the ranging results (i.e., 1016 CIRs and a channel
diagnostics) once every 200 ms. We collect 2,000 DS-TWR results for each pose,
which means that 4,000 LOS and 4,000 NLOS samples are collected, for
subsequent training. We also include an open source dataset to prevent the
overfitting problem associated with data shortage [15]. The open dataset
contains 42,000 samples444This LOS/NLOS open datasets were created by using
Qorvo DWM1001 UWB radio module. Measurements were taken on 7 different indoor
locations. In every indoor location 3,000 LOS and 3,000 NLOS samples were
collected.. Therefore, a total of 50,000 samples are used for training and
testing the CNN-based model.

On the other hand, IMU sensor data is simultaneously collected once every 60
ms. The IMU sensor data is composed of the gravity and a pure acceleration of
x,y, and z coordinates, which means that 6 sensor values are collected at each
timestep. The reason that we collect those two type of data is that the
gravity values contain the information about the orientation of the smartphone
and the accelerometer values reflect the signature of the user movement,
especially during walking [16]. Since IMU data resolution is approximately 3
times higher than DS-TWR, we collect 6,600 IMU samples for each pose. In case
of IMU, no open source dataset is used, and hence, in total 26,400 samples are
used.

Figure 6: The overall architecture of $D^{3}$. MD is a set of the commercial
smartphone and the Qorvo board using wired connection.

##  IV $D^{3}$: Proposed system

###  IV-A Overview

Fig. 6 shows the overall architecture of $D^{3}$. MD is the combination of the
smartphone and Qorvo board (i.e., MDEK1001) through USB serial555The latest
smartphones from Samsung, such as Galaxy S22+ and Galaxy Z Fold3, have
embedded UWB module, but there is no developer API to obtain CIR yet. This is
the reason why we choose 3rd party UWB module.. The UTG is also equipped with
the same UWB module of MD to perform UWB ranging. After receiving RFM, UWB
module on the MD collects CIRs and channel diagnostics, and sends them to the
eCIR updater. The eCIR updater generates eCIR from 1016 CIRs based on channel
diagnostics such as maxNoise and FP_INDEX (Section IV-B). After eCIR is
generated, it is transferred from Qorvo board to the smartphone through USB
port. The smartphone runs the LOS/NLOS classifier to determine whether the
received signal is LOS or NLOS using eCIR whenever eCIR reception occurs
(Section IV-C). The LOS/NLOS classifier sends the result of the classification
to the pose detector. Finally, the pose detector determines the pose by
combining IMU sensor data, especially accelerometer and gravity measurements
(Section IV-D). In both MD and UTG, each process of all the components
operates in real-time.

###  IV-B Effective channel impulse response

Since $D^{3}$ performs LOS/NLOS classification and the pose detection of MD on
Android application, the data should be transferred from the Qorvo board to
the smartphone through USB port. Therefore, the latency of data transfer
between the Qorvo board and the smartphone should be optimized to achieve the
real-time operation. If the processing time of the whole procedures from Qorvo
board to smartphone is larger than the ranging interval, the ranging operation
between Qorvo board of MD and UTG will be interrupted.

First, we experimentally measured the latency when 8 channel diagnostics and
1016 CIRs are transferred to the smartphone through USB without data loss, and
223.4 ms is consumed on average, which is longer than the default ranging
interval. Therefore, the latency should be minimized by selective data
transfer, and hence, we adopt the eCIR updater to collect the only effective
CIRs. Fig. 7(a) represents the CIRs divided into a real signal and a noise
using maxNoise. If the amplitude of CIR is less than maxNoise, the CIR is
considered as a noise. Thus, CIRs before FP_INDEX are noise and can be
eliminated. Therefore, we focus on CIRs after FP_INDEX, which contains the
real information of the signal.

Fig. 7(b) shows only 135 CIRs of two LOS and NLOS CIR samples. The 0 index is
FP_INDEX-5 index as a margin for FP_INDEX in both CIR samples. The CIR trends
have clear differences between LOS and NLOS in the max amplitude of the peak
and the presence of multiple peaks after the first peak. Those signatures
represent the nature of LOS and NLOS signals. In the case of LOS, the
amplitude of CIRs quickly goes below under maxNoise, and 61 CIRs are actual
signal after FP_INDEX on average. However, the first peak amplitude of the
NLOS signal is much smaller than that of LOS and multiple peaks are observed
after the first peak because of the multipath signals in NLOS situation.
Therefore, 130 CIRs are larger than maxNoise after FP_INDEX in case of NLOS on
average. As a result, at least 135 CIRs (i.e., 130 and 5 margin CIRs before
FP_INDEX) should be transferred to the smartphone for LOS/NLOS classification
since those CIRs contain all the signatures of the received signal. We
consider those 135 CIRs as eCIR and using only 135 CIRs for LOS/NLOS
classification instead of 1016 CIRs, and hence, the latency is reduced to 17.8
ms from 223.4 ms, which is much smaller than the ranging interval.

###  IV-C LOS/NLOS classifier

(a) The CIR divided into signal and noise based on maxNoise

(b) The difference in the amplitude and the number of peaks

Figure 7: The example of effective CIR and difference between LOS/NLOS.

Fig. 8(a) shows the deep learning model architecture adopted for the CIR based
LOS/NLOS classification. The input layer comprises the 135 consecutive CIRs
from FP_INDEX-5. It is followed by 4 CNN layers, which are highly efficient in
extracting meaningful practical features from spatial/temporal data. The
output of the CNN is then flattened from the 2D image format to a 1D vector,
using the Flatten layer. This is followed by a Dense layer, in which all the
neurons of the previous and the current hidden layer are pairwise connected
using weights. The output of the Dense layer is passed into the Sigmoid
activation layer, which applies Sigmoid function to produce a number between 0
and 1, which in this case indicates the probability of the CIR corresponding
to a NLOS condition. Finally, the output of the Sigmoid is smoothed by the
low-pass filter (LPF) to remove the outliers, and the signal is classified as
0 (LOS) or 1 (NLOS).

The detailed architecture of an individual CNN layer is shown in Fig. 8(b).
The convolution layer Conv1D consists of a pre-specified number of 1D vectors
called filters, all of them having a pre-specified size called kernel size.
These filters help to extract significant spatial/temporal features from the
input data. Each of these filters is convolved with the input data to produce
a vector. These output vectors are stacked one above another to give a 2D
image type output. The output of the convolution layer is passed into the
Instance Normalization (IN) layer, which applies normalization per data sample
along a specified axis. Unlike the conventional Batch Normalization layer, IN
thus reduces incoming noise and model dependence on the training data
statistics. The normalized output is sent into the rectified linear units
(ReLU) layer, which applies the ReLU activation function. This is followed by
the Dropout layer, which randomly drops a fraction of the neurons at algorithm
runtime. This reduces the overfitting error of the model to the training data.
The final layer is a Maxpool layer, which also operates based on a pre-
specified kernel size. It does the maxpooling operation (i.e. chooses the
maximum element along a specified axis) in the pre-specified kernel size, and
replaces the elements in that kernel with this chosen maximum element. This
helps to bring down the dimensionality of the data and helps to simplify the
model and prevent overfitting. The entire set of parameters used in the CNN
based model for the LOS/NLOS classification are listed in Table I.

(a) LOS/NLOS classification model architecture with 4 CNN layers

(b) CNN layer architecture

Figure 8: CNN based model for LOS/NLOS classification using eCIR input. TABLE I: The set of LOS/NLOS classification model parameters Model parameters | Value
---|---
Conv1D layer [1,2,3,4] kernel size | [5,11,17,5]
Conv1D layer [1,2,3,4] no. of filters | [64,128,256,512]
Dropout rate | 0.2
Maxpool layer kernel size | 2
Optimizer | Adam
Loss function | Binary cross-entropy
Batch size | 50
Maximum no. of epochs | 20

###  IV-D Pose detector

Fig. 9 shows the snapshots of IMU sensor values, especially accelerometer of
each poses, divided into LOS and NLOS conditions. The red and blue plot
represent the poses of MD, respectively, and three lines of each pose
represents the acceleration value of x,y, and z coordinates. The trends of
acceleration value show clear difference whether pose is HAND or POCKET. If
the pose of MD is close to the ground such as POCKET, the magnitude of
accelerometer is much higher than that of HAND. Therefore, the difference in
acceleration is a signature of HAND and POCKET, and can be classified by deep
neural network.

(a) LOS condition

(b) NLOS condition

Figure 9: The snapshots of 18 accelerometer values (1080 ms) of four poses.

Fig. 10(a) shows the deep neural network model used for pose classification.
The input layer comprises the IMU sensor data with 6 features, namely the
accelerometer and gravity sensor readings along x,y and z coordinates. We also
accumulate this IMU sensor data for the previous 18 timesteps, which leads to
the input being a matrix of dimensions $18\times 6$. This number of timesteps
is derived based on the logic that we want to send the sequential data of the
previous 2-3 steps of the user as input data. Considering that the average
human walking speed is 3 steps per second, that corresponds to sending 1 s
worth of IMU sensor data [16]. The IMU sensor collects data every 60 ms,
hence, to collect the IMU data for 1 s, we need to collect 18 IMU data
samples.

The IMU input data is passed into one of two CNN-LSTM models based on whether
the CIR based LOS/NLOS classification in the previous stage produced an output
LOS or NLOS. The two CNN-LSTM based models are designed identically but get
trained with different sets of training data, resulting in different optimal
weights and filters. The model starts with 3 CNN layers, followed by the LSTM
layer. While the CNN layers extract meaningful spatial/temporal features from
the IMU data, the LSTM layer takes advantage of the sequential (18 timesteps)
data to produce predictions based on current timestep as well as the previous
timesteps [17]. The LSTM output is flattened using a Flatten layer, and then
input to a Dense Layer. Finally, Sigmoid activation layer operates on the
previous layer output, to produce a number between 0 and 1. The output of
Sigmoid is directly used for pose detection without passing through LPF
because LOS/NLOS classification results are already filtered by LPF.

For the model corresponding to the LOS case, the output of Sigmoid corresponds
to the probability of MD being FRONT, as opposed to being LOS-HAND. Similarly,
for the NLOS case, the output of Sigmoid corresponds to the probability of MD
being BACK, as opposed to being NLOS-HAND. The detailed CNN layer architecture
used for the pose detection models is shown in Fig. 10(b). The convolution
layer Conv2D contains 2D filters of pre-specified kernel size. The output of
the Conv2D layer is successively passed through an IN layer, ReLU activation,
Dropout, and a Maxpool layer. The entire set of parameters used in CNN-LSTM
based model for the pose classification are listed in Table II.

(a) Pose detection model architecture with 3 CNN layers

(b) CNN layer architecture

Figure 10: CNN-LSTM based model for pose detection using IMU input based on LOS/NLOS classification model output. TABLE II: The set of pose detection model parameters Model parameters | Value
---|---
Conv2D layer [1,2,3] kernel size | [2,2,2]
Conv2D layer [1,2,3] no. of filters | [64,128,256]
Dropout rate | 0.2
Maxpool layer kernel size | 2
LSTM layer no. of units | 128
Optimizer | Adam
Loss function | Binary cross-entropy
Batch size | 100
Maximum no. of epochs | 100

##  V Performance Evaluation

###  V-A Implementation

MD: We implemented the application side of $D^{3}$ on the commercial
smartphone (i.e., SM-N986B) running on Android 11 to show the feasibility.
Both LOS/NLOS classifier and pose detector are implemented using TensorFlow
2.10.0 and converted to TensorFlow-Lite for mobile deployment. Other modules
including CNN models and IMU are implemented using Android Studio. The
exponentially weighted moving average is used as a LPF with a weighting factor
of 0.8. If the user turns on $D^{3}$, the results of LOS/NLOS classification
and pose detector are showed on application UI.

UWB: We implemented both UWB modules of MD and UTG based on the example code
from the Qorvo DWM1001 [18]. The DS-TWR operation between MD/UTG and eCIR
updater in Qorvo board is implemented by following the user manual using
SEGGER Embedded Studio for ARM 5.32a [19, 20]. The UWB module is connected to
MD via a wired connection.

###  V-B Experiment setup

Datasets and CNN training: We trained our LOS/NLOS classifier on open datasets
and collected data with preliminary experiments to prevent overfitting, and
pose detector is trained by using only collected IMU data. All datasets were
randomly divided into training (80%) and testing (20%) data.

Performance evaluation: We conducted our experiments for a test area of
$8\times 8~{}m^{2}$ in an office building. As shown in Fig. 11, UTG is
installed with UWB module and MD is composed of the smartphone and UWB module.
We also design the case of MD using 3D printer for the consistency of
experiments. The experiments are focused to show the real-time accuracy of
LOS/NLOS classification and pose detection, and hence, we iterate 50 times for
each poses. Each iteration returns the LOS/NLOS condition and pose of MD every
200 ms while the user with MD starts walking towards UTG from 4 m away.

Figure 11: The snapshot of the real-world UTG experiment environment.

###  V-C Performance

TABLE III: The accuracy of LOS/NLOS classification and pose detection. | LOS/NLOS classification (w/o LPF) | Pose detection
---|---|---
LOS-HAND | 0.994 (0.849) | 0.983
NLOS-HAND | 0.993 (0.839) | 0.982
FRONT | 0.968 (0.783) | 0.931
BACK | 0.982 (0.803) | 0.948

We measure the LOS/NLOS classification and pose detection performance of
$D^{3}$ as the rate of true positive an false positive, in which the
classification and detection are correctly/wrongly conducted. Table III
summarizes the accuracy of each pose measured in real-time.

LOS/NLOS classification: The two values are obtained from experiments. The
first value is the actual performance of LOS/NLOS classification model and the
second value in the parentheses represents the accuracy without LPF, which is
the output of Sigmoid in Fig. 8(a). The performance without LPF is similar to
that of [9], which proposes CNN-LSTM model for classification, and overall
accuracy of 0.819. $D^{3}$ improves the accuracy by adding LPF to CNN model,
and the overall LOS/NLOS classification accuracy is 0.984. As a result,
$D^{3}$ improved classification performance by 20.1%.

Pose detection: Considering that the accuracy of pose detection is affected by
the result of LOS/NLOS classification, the pose detection accuracy can not
outperform LOS/NLOS classification accuracy. The average performance of pose
detection is 0.961, and there is only accuracy degradation of 0.023 compared
to LOS/NLOS classification. Therefore, we conclude that $D^{3}$ successfully
classifies four different poses.

Pose transition delay: $D^{3}$ utilizes LPF to improve the accuracy of
LOS/NLOS classification, but, LPF has trade-off between the accuracy and a
delay. The higher the weighting factor, the longer delay. Therefore, we
evaluate the pose transition delay between LOS and NLOS condition. To this
end, we record the timing of the transition and measure the delay using
application. If $D^{3}$ does not adopt LPF on LOS/NLOS classification, the
average pose transition delay should be 200 ms, which is same as the default
ranging interval of DS-TWR. Table IV summarizes the average transition delay
and standard deviation for the possible combinations in ms. The transition
delay between HANDs takes 734.8 ms, which is the shortest among four delays,
because it has no change of the position on the human body. The remaining
transitions contain pose change from/to POCKET that have more trajectory to
move on the body, and consume additional 111.1 ms on average. All pose
transitions are completed within one second.

TABLE IV: The transition delay between LOS and NLOS conditions. | LOS-HAND | FRONT
---|---|---
NLOS-HAND | 734.8 (137.4) | 838.4 (168.9)
BACK | 848.0 (173.8) | 851.2 (188.4)

##  VI Conclusion and future work

In this paper, we have proposed $D^{3}$, a novel pose detection system for the
proximity services based on UWB CIR and fusion of IMU sensors. We implemented
and evaluated $D^{3}$ in a real-world environment using SM-N986B and MDEK1001.
$D^{3}$ achieved 0.984 and 0.961 accuracy in LOS/NLOS classification and pose
detection in real-time. As a future work, we will design the algorithm for
alleviating ranging error in UWB by adopting $D^{3}$. In addition, we have a
plan to develop the whole UTG system, and will implement $D^{3}$ as a one
device without external UWB module.

## References

  * [1] J. Choi, G. Lee, S. Choi, and S. Bahk, “Smartphone based Indoor Path Estimation and Localization without Human Intervention,” _IEEE Transactions on Mobile Computing_ , vol. 21, no. 2, pp. 681–695, 2020.
  * [2] J. Choi, M. Lee, T. Kim, and H. Jun, “Wand: Remote Control System based on Ultra-wideband Localization using Smartphone,” in _Proc. IEEE GlobeCom_ , 2022.
  * [3] UWB Minutes: Access Control. [Online]. Available: https://www.nxp.com/video/uwb-minutes-uwb-for-access-control:UWB-MINUTES-ACCESS-CONTROL
  * [4] NXP Collaborates with ING and Samsung to Pilot Industry’s First UWB-Based Peer-to-Peer Payment Application. [Online]. Available: https://www.nxp.com/company/about-nxp/newsroom:NEWSROOM
  * [5] Car Connectivity Consortium. [Online]. Available: https://carconnectivity.org/
  * [6] What’s the deal with Ultra Wideband. [Online]. Available: https://www.bmw.com/en/innovation/bmw-digital-key-plus-ultra-wideband.html
  * [7] B. Großwindhager, M. Rath, J. Kulmer, M. S. Bakr, C. A. Boano, K. Witrisal, and K. Römer, “SALMA: UWB-based Single-anchor Localization System using Multipath Assistance,” in _Proc. ACM SenSys_ , 2018.
  * [8] _IEEE 802.15z-2012_ , IEEE Computer Society, Aug. 2020.
  * [9] C. Jiang, J. Shen, S. Chen, Y. Chen, D. Liu, and Y. Bo, “UWB NLOS/LOS Classification Using Deep Learning Method,” _IEEE Communications Letters_ , vol. 24, no. 10, pp. 2226–2230, 2020.
  * [10] C. Jiang, S. Chen, Y. Chen, D. Liu, and Y. Bo, “An UWB Channel Impulse Response De-Noising Method for NLOS/LOS Classification Boosting,” _IEEE Communications Letters_ , vol. 24, no. 11, pp. 2513–2517, 2020.
  * [11] H. Wymeersch, S. Maranò, W. M. Gifford, and M. Z. Win, “A Machine Learning Approach to Ranging Error Mitigation for UWB Localization,” _IEEE Transactions on Communications_ , vol. 60, no. 6, pp. 1719–1728, 2012\.
  * [12] Z. Zeng, S. Liu, and L. Wang, “NLOS Identification for UWB based on Channel Impulse Response,” in _Proc. IEEE ICSPCS_ , 2018.
  * [13] V. Barral, C. J. Escudero, J. A. García-Naya, and R. Maneiro-Catoira, “LOS Identification and Mitigation Using Low-Cost UWB Devices,” _Sensors_ , vol. 19, no. 16, p. 3464, 2019.
  * [14] MDEK1001 Ultra-Wideband (UWB) Transceiver Development Kit. [Online]. Available: https://www.qorvo.com/products/p/MDEK1001
  * [15] K. Bregar, A. Hrovat, and M. Mohorcic, “NLOS Channel Detection with Multilayer Perceptron in Low-Rate Personal Area Networks for Indoor Localization Accuracy Improvement,” in _Proc. IPSSC_ , vol. 31, 2016.
  * [16] Y. Shu, K. G. Shin, T. He, and J. Chen, “Last-mile Navigation using Smartphones,” in _Proc. ACM MobiCom_ , 2015.
  * [17] S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” _Neural computation_ , vol. 9, no. 8, pp. 1735–1780, 1997.
  * [18] Basic Application using the UWB Features of the DWM1001C. [Online]. Available: https://github.com/Decawave/dwm1001-examples
  * [19] DW1000 USER MANUAL (PDF). [Online]. Available: https://www.qorvo.com/products/d/da007967/
  * [20] Embedded Studio Downloads. [Online]. Available: https://www.segger.com/downloads/embedded-studio/