# Low-cost low-power in-vehicle occupant detection with mm-wave FMCW radar

Mostafa Alizadeh  Eletrical and Computer Engineering
University of Waterloo
Canada, Ontario,
Email<EMAIL_ADDRESS>Hajar Abedi  System Design Engineering
University of Waterloo
Canada, Ontario,
Email<EMAIL_ADDRESS>George Shaker  Eletrical and Computer
Engineering
University of Waterloo
Canada, Ontario,
Email<EMAIL_ADDRESS>
###### Abstract

In this paper, we use a low-cost low-power mm-wave frequency modulated
continuous wave (FMCW) radar for the in-vehicle occupant detection. We propose
an algorithm using Capon filter for the joint range-azimuth estimation. Then,
the minimum necessary features are extracted to train machine learning
classifiers to have reasonable computational complexity while achieving high
accuracy. In addition, experiments were carried out in a minivan to detect
occupancy of each row using support vector machine (SVM). Finally, our
proposed system achieved 97.8% accuracy on average in finding the defined
scenarios. Moreover, The system can correctly identify if the vehicle is
occupied or not with 100% accuracy.

##  I Introduction

Frequency modulated continuous wave (FMCW) radars have unique advantages that
they differentiate them from other radar systems. The benefits include
simultaneous detection of range, Doppler (or velocity), and angle which makes
this type of radar popular for a variety of applications. The major advantages
of these radars are being low-cost and low-power so that make them suitable
for most internet-of-things (IoT) applications such as vital signs detection
[1], wireless finger print identification [2], gesture recognition [3], and
occupant detection without imposing any potential long-term health risks.

The sensor imaging resolution is defined as the minimum spatial separation of
two targets revolvable by the radar which contains both range and azimuth
resolutions. For FMCW radars, the larger the sweeping bandwidth is, the more
range resolution is [4]. On the other hand, the angle of arrival resolution
increases by increasing the number of transmitters and receivers. However, the
implementation of a radar system with a large number of transmitters and
receivers would result in higher system cost and more operational complexity.
Hence it is desirable to achieve accurate occupancy detection with low-
resolution radars. For instance, authors in [5] used a single transceiver
ultra-wideband (UWB) radar with only 5.35 cm without angle of arrival
estimation. Indeed, Santra et al. in [6] used a mm-wave FMCW radar with 50 cm
resolution at 1 m distance to the radar. In [5], they applied principle
component analysis (PCA) on the range profile for counting people in a room
and the first component was only considered. The range profile is represents
peaks corresponding to the object distributions in the scene and its first
basis may not contain all target locations reducing the system performance. In
[6], they tried to find human activities by categorizing them in three classes
of macro-Doppler, micro-Doppler, and vital signs without invoking the azimuth
information. In contrast in our work, we will exploit range-azimuth maps
obtained by 2D Capon filtering as our input features for machine learning
classification.

In section II, we introduce the proposed algorithm and the system formulation
based on FMCW radar and the high-resolution Capon filtering for joint range-
azimuth estimation. In section II-A, we briefly explain the classification. In
section III, we discuss on the experimental results and finally, we will
conclude the paper with wrapping up the achievements and further possible
extensions for the future works in section IV.

##  II System design

Figure 1: Vehicle occupant detection algorithm.

In Fig. 1, the proposed signal processing chain is illustrated. The range FFT
is applied on the received chirp samples. In a time division multiplexed (TDM)
MIMO FMCW radar, a sequence of chirps are sent in a frame from different
transmit antennas. At the receiver, the signal is collected and assigned to a
virtual channel such that each channel contains the data transmitted and
received from and to a unique pair of transceiver which is done in stationary
clutter removal stage together with removing the average of each range bin. In
fact, removing the average is equivalent to eliminating the stationary
scatterers. Then, the covariance matrix of the virtual channel vector across
each range bin is calculated and capon filter will be applied to estimate the
angle of arrival spectrum of each range. By putting them in a matrix, range-
azimuth map will be constructed and is delivered to ML section to perform
classification.

###  II-A Range-azimuth map derivation

In a FMCW radar having up chirps i.e. only positive-slope chirps, the received
baseband signal from one target at l’th antenna element is:

$\displaystyle x_{l}(t_{f},t_{s})$
$\displaystyle=b_{l}\exp\Big{[}-j\big{(}2\pi
f_{b}t_{f}+2\frac{vt_{s}}{\lambda_{max}}+\tau_{l}$
$\displaystyle+\xi_{l}+\Delta\psi_{l}(t_{f},t_{s})\big{)}\Big{]}+e_{l}(t_{f},t_{s})$
(1)

where $t_{f}$ and $t_{s}$ are referring to two different time scales. The
former refers to the time scale of a chirp period while the latter is
referring to the time scale greater than a chirp period. $b_{l}$, $f_{b}$,
$v$, $\lambda_{max}$, $\tau_{l}$, $\xi_{l}$, $\Delta\psi_{l}(t_{f},t_{s})$,
and $e_{l}(t_{f},t_{s})$ are the channel gain, beat frequency 111it contains
the range of the target. , target’s velocity, the wavelength corresponding to
the start frequency of the FMCW ramp, the phase shift at l’th receiver due to
the angle of arrival, the channel phase, the residual phase noise existing in
both $t_{f}$, $t_{s}$, and the additive noise, respectively. By reordering
terms in (1) and stacking the received signal from all receiver channels in a
column vector, it can be expresses as:

$\displaystyle\boldsymbol{x}(t_{f},t_{s})$
$\displaystyle=\Gamma\boldsymbol{a}(\theta)y(v,t_{s})s(t_{f},t_{s})+\boldsymbol{e}(t_{f},t_{s})$
(2) $\displaystyle\Gamma$
$\displaystyle:=\text{diag}\Big{(}b_{1}\exp(-j\xi_{1}),\cdots,b_{L}\exp(-j\xi_{L})\Big{)}$
$\displaystyle\boldsymbol{a}(\theta)$
$\displaystyle:=\left[\exp\left(-j\tau_{1}\right),\cdots,\exp\left(-j\tau_{L}\right)\right]^{T}$
$\displaystyle y(v,f_{b},t_{f},t_{s})$
$\displaystyle:=\exp\left(-j\left(\frac{2vt_{s}}{\lambda_{max}}+2\pi
f_{b}t_{f}+\Delta\psi(t_{f},t_{s})\right)\right)$

in which $\theta$ is the angle of arrival (AoA) of the target. $\Gamma$
depends on the channel gain/phase mismatches and $\boldsymbol{a}$ depends on
(AoA) and it is called the steering vector. If there is more than one target
at different ranges i.e. with different $f_{b}$, then the received vector is
the summation of all the vectors received from each target, thus:

$\displaystyle\boldsymbol{x}(t_{f},t_{s})$ $\displaystyle=\Gamma
A(\boldsymbol{\theta})Y(\boldsymbol{v},\boldsymbol{f}_{b},t_{f},t_{s})+\boldsymbol{e}(t_{f},t_{s})$
(3)

where A is $L\times K$ matrix with K is the number of targets and it has
columns corresponding to the steering vector of each target. Matrix
$\boldsymbol{Y}$ is diagonal matrix with the elements of
$y(v,f_{b},t_{f},t_{s})$. In (3), the vectors $\boldsymbol{\theta}$,
$\boldsymbol{v}$, and $\boldsymbol{f}_{b}$ are the unknown parameters of the
targets; however, only elements of A are functions of receiver channel
indexes. Hence, matrix $\boldsymbol{Y}$ does not contribute to the covariance
of $\boldsymbol{x}$. In fact, the covariance matrix of $\boldsymbol{x}$ can be
computed as the following when the additive noise is uncorrelated to
$\boldsymbol{Y}$:

$R=\mathbb{E}(\boldsymbol{x}\boldsymbol{x}^{H})=P_{s}\Gamma
A(\boldsymbol{\theta})A^{H}(\boldsymbol{\theta})\Gamma^{H}+R_{n}$ (4)

in which $P_{s}$ is the power of $s(t_{s})$. The matrix $R_{n}$ is the noise
covariance and it is positive definite matrix by assuming that the noise at
each receiver is independent of the others. Moreover, the first term in the
covariance of (4), is positive definite since $A(\boldsymbol{\theta})$ is a
Vandermonde matrix with positive kernels [7]. Therefore, $R$ is positive
definite and it is invertible. The Capon output filter spectrum is computed as
follows:

$\Phi(\boldsymbol{\hat{\theta}})=\dfrac{1}{\boldsymbol{a}^{H}(\hat{\theta})R^{-1}\boldsymbol{a}(\hat{\theta})}$
(5)

where $\hat{\theta}$ is a test unknown AoA. In an

###  II-B Machine learning

TABLE I: Definition of classes class | Definition
---|---
No-one | the car is empty
Row1 | one person is in the first row
Row2 | one person is in the second row
Row3 | one person is in the third row
Row12 | two persons, one in the first and one in the second rows
Row13 | two persons, one in the first and one in the last rows
Row23 | two persons, one in the second and one in the last rows
Row123 | three persons, each on different row

As Fig. 1 shows, machine learning classification is the last stage of our
proposed in-vehicle occupant detection. After finding range-azimuth dataset,
the size of it reduced by using principle component analysis (PCA) without
information loss. The dataset is split to training and test datasets by the
ratio of 8 to 2. The split is fair such that the size of data for each class
is equal in both the training and test sets. We applied SVM, a supervised
classifier, to classify the Capon filter output to 8 classes defined in Table
I.

It is also necessary to optimize the hyperparameters utilized by the
classifier to get the optimum classification. Thus, a grid search with k-fold
cross validation is employed to search for the best hyperparameters that will
increase the prediction accuracy of the classifiers. For evaluation, we tested
the model using 5-fold cross validation.

##  III Experimental results

TABLE II: FMCW Radar configuration Parameter | Ka | $T_{r}^{\mathtt{b}}$ | $B_{s}^{\mathtt{c}}$ | $f_{r}^{\mathtt{d}}$ | $f_{b,max}^{\mathtt{e}}$
---|---|---|---|---|---
Value | $98MHz/\mu s$ | $580\mu s$ | 3920 MHz | 6.25/s | 2.2 MHz
a chirp slope |  b chirp duration |  c sweeping BW
d frame rate |  e ADC sampling rate |  |
Figure 2: Minivan indoor look Figure 3: Range-azimuth map when the seat number
1, 4, and 5 are occupied.

We used Texas Instrument (TI) mm-wave FMCW chip for our experiments which
operates at 77GHz. In addition, there were three rows in minivan as shown in
Fig. 2. The seats are numbered starting from the driver and each row from
right to left of the figure. During recordings, passengers were allowed to
freely move their hands, talk to each other, and work with their phones. We
consider eight classes, empty car and either a row is occupied or not (Table
I). In fact, our class definition is a small collection of all possible
situations in a car; otherwise, the total number of seat occupations is
$2^{7}=128$. The recording duration is the same for all classes in order to
have unbiased training.

TI radar chip has 3Tx and 4Rx and two transmitters were used in TDM MIMO mode
to construct eight virtual receivers (further radar parameters are listed in
Table II). Thus, for azimuth detection, we have 8 spatial samples resulting in
$2/8=0.25\;Radian$ or 14-degree resolution. This means that at 1 meter away
from the radar, two targets can be resolved only if they are separated more
than 25 cm; otherwise, the targets are spatially correlated. For instance, in
Fig. 3, represents a sample range-azimuth map obtained from (5) when the
driver (seat number 1) and two other passengers are in the minivan (see Fig.
2). The figure does not show clear visual passenger separation due to low
angular resolution. However, SVM classifier can identify this map with three
occupied rows.

Confusion matrix of SVM classifier is shown in Fig. 4. From Fig. 4, when the
car is vacant, there is no false alarm. Additionally, if the car is not empty,
SVM correctly indicates that there is at least one person (see the first
column of Fig. 4). Moreover, in the presence of an occupant; i.e. row1, row2,
row3; SVM certainly identifies the case. However, most of the
misclassifications occurred for localization of more than one occupant in the
car i.e. row123 case. Finally, by using 5-fold cross validation, 97.8% correct
detection rate is obtained.

Figure 4: Confusion matrix of SVM

##  IV Conclusion

In this paper, we investigated the in-vehicle occupancy detection with a mm-
wave FMCW radar. Also, we addressed the low angular resolution which limits
the visual perception of the occupant location. However, with the aim of
machine learning classifier, namely SVM, a high accuracy was obtained to
determine one of the defined scenarios. Therefore, although the resolution is
low, by defining efficient features the accuracy can be enhanced.

## Acknowledgment

The authors acknowledge financial support of Natural Sciences and Engineering
Research Council of Canada (NSERC) and Ontario Centres of Excellence (OCE).

## References

  * [1] M. Alizadeh, G. Shaker, and S. Safavi-Naeini, “Remote heart rate sensing with mm-wave radar,” in _2018 18th International Symposium on Antenna Technology and Applied Electromagnetics (ANTEM)_ , pp. 1–2.
  * [2] K. Diederichs, A. Qiu, and G. Shaker, “Wireless biometric individual identification utilizing millimeter waves,” vol. 1, no. 1, pp. 1–4.
  * [3] K. A. Smith, C. Csech, D. Murdoch, and G. Shaker, “Gesture recognition using mm-wave sensor for human-car interface,” _IEEE Sensors Letters_ , vol. 2, no. 2, pp. 1–4, June 2018.
  * [4] M. Alizadeh, G. Shaker, J. C. M. D. Almeida, P. P. Morita, and S. Safavi-Naeini, “Remote monitoring of human vital signs using mm-wave fmcw radar,” _IEEE Access_ , vol. 7, pp. 54 958–54 968, 2019.
  * [5] Z. Baird, I. Gunasekara, M. Bolic, and S. Rajan, “Principal component analysis-based occupancy detection with ultra wideband radar,” in _2017 IEEE 60th International Midwest Symposium on Circuits and Systems (MWSCAS)_ , Aug 2017, pp. 1573–1576.
  * [6] A. Santra, R. V. Ulaganathan, and T. Finke, “Short-range millimetric-wave radar system for occupancy sensing application,” _IEEE Sensors Letters_ , vol. 2, no. 3, pp. 1–4, Sep. 2018.
  * [7] A. Pinkus, “Spectral properties of totally positive kernels and matrices,” in _Total positivity and its applications_. Springer, 1996, pp. 477–511.