[a]Juan W. Pedersen

# Quantum Simulation of Finite Temperature Schwinger Model via Quantum
Imaginary Time Evolution

Etsuko Itou  Rong-Yang Sun  Seiji Yunoki

###### Abstract

We study the Schwinger model at finite-temperature regime using a quantum-
classical hybrid algorithm. The preparation of thermal state on quantum
circuit presents significant challenges. To address this, we adopt the Thermal
Pure Quantum (TPQ) state approach and apply the Quantum Imaginary Time
Evolution (QITE) algorithm to implement the necessary imaginary time
evolution. We first compute the chiral condensate in the massless Schwinger
model, verifying its consistency with the analytical solution. We then
simulate the massive Schwinger model with non-zero topological $\theta$-term
to investigate the temperature and $\theta$-dependence of the chiral
condensate. Our method works well even at non-zero $\theta$ regime, while the
conventional lattice Monte Carlo method suffers from the sign problem in this
system.

##  1 Introduction

Quantum simulation of quantum field theories at finite-temperature regime is a
challenging endeavor. This complexity is mainly due to the nature of thermal
state as a mixed state, which complicates its preparation on quantum circuit.
A possible approach to this problem is usage of a “typical pure state”, the
so-called Thermal Pure Quantum (TPQ) state [1, 2]. In thermal equilibrium
system, the TPQ state is defined as one whose expectation value approximates
the thermal ensemble average of local observables at finite volume. It is
proven that the expectation value calculated with the TPQ state converges to
the thermal ensemble average in the thermodynamic limit [1]. Although this
theoretical framework was initially introduced as a novel formulation of
statistical mechanics, it also has the potential for use in the quantum
simulation of gauge theories. For example, its applicability to $Z_{2}$ gauge
theory has been discussed in Ref. [3].

We use a specific realization of the TPQ state termed as the canonical TPQ
state [2]. This state is defined by applying the imaginary time evolution
operator to random state. Implementing imaginary time evolution on quantum
circuit is also challenging since it is expressed by non-unitarity operators.
Several algorithms have been proposed, mainly to address the ground state
energy problem, such as variational methods [4], probabilistic methods [5],
power methods [6], and the Quantum Imaginary Time Evolution (QITE) algorithm
[7]. Here, we employ the QITE algorithm combining the random-state generation
algorithm to generate the TPQ state.

In this work, we simulate the temperature-dependence of the chiral condensate
of the Schwinger model using the TPQ state and the QITE algorithm. We perform
the massless case to confirm the validity of our method by comparing our
results with the analytic result. Furthermore, we explore the massive and non-
zero $\theta$ regime where the sign problem appears in the conventional Monte
Carlo method.

##  2 The Schwinger model

Let us first briefly overview the Schwinger model. The Schwinger model is a
1+1 dimensional U(1) gauge theory. The qubit description of the Schwinger
model Hamiltonian [8] is given by

$H=\frac{1}{4a}\sum_{n=1}^{N-1}\left[X_{n}X_{n+1}+Y_{n}Y_{n+1}\right]+\frac{m}{2}\sum_{n=1}^{N}(-1)^{n}Z_{n}+\frac{ag^{2}}{2}\sum_{n=1}^{N-1}\left[\sum_{i=1}^{n-1}\frac{Z_{i}+(-1)^{i}}{2}+\frac{\theta}{2\pi}\right]^{2},$
(1)

where $N,a$ and $g$ denote the number of lattice site, the lattice spacing,
and the coupling constant, respectively. $X_{i},Y_{i},Z_{i}$ are the Pauli
matrices. Here, we solve the Gauss’s law constraint and impose the open
boundary condition to remove the degree of freedom for the U($1$) gauge field.
Note that the non-local terms are induced due to the Gauss’s law constraint in
the last term in Eq. (1). The topological $\theta$-term is shown as a constant
shift of the electric-field ($Z_{i}$), and thus no additional difficulty
emerges by the insertion in this formula.

In this work, we investigate the (discrete) chiral symmetry as a function of
temperature. The order parameter is given by the chiral condensate,
$\langle\bar{\psi}\psi\rangle$, where $\psi$ is a fermionic field operator.
The quantity can be also expressed by the Pauli matrices using the Jordan-
Wigner transformation as follows:

$\langle\bar{\psi}\psi\rangle=\frac{1}{N}\sum_{i=1}^{N}(-1)^{i}\frac{1+\left\langle
Z_{i}\right\rangle}{2a}.$ (2)

In the massless case, its temperature-dependence has been derived analytically
based on the Lagrangian formalism [9] as

$\langle\bar{\psi}\psi\rangle=-\frac{m_{\gamma}}{2\pi}e^{\gamma}e^{2I\left(\beta
m_{\gamma}\right)}\quad,\quad
I(x)=\int_{0}^{\infty}\frac{1}{1-e^{x\cosh(t)}}dt,$ (3)

where $\gamma$ is the Euler constant and $m_{\gamma}$ is the mass gap
$m_{\gamma}=g/\sqrt{\pi}$. Note that at zero temperature, it takes the value
$\langle\bar{\psi}\psi\rangle\simeq-0.1599$ [10]. On the other hand, it is
difficult to solve this model analytically in massive regime. Furthermore,
when dealing with non-zero $\theta$ regime, the conventional lattice Monte
Carlo method is not doable due to the sign problem.

##  3 Calculation strategy

To express a thermal state, we prepare a TPQ state [1] on a quantum circuit.
The TPQ state, $|\psi\rangle^{\mathrm{TPQ}}$, is defined as a state that
satisfies for low-degree polynomials of local operators,
${}^{\forall}\hat{O}$, and $\forall\epsilon>0$,

$\mathrm{P}\left(\left|\frac{\langle\psi|\hat{O}|\psi\rangle^{\mathrm{TPQ}}}{\langle\psi|\psi\rangle^{\mathrm{TPQ}}}-\langle\hat{O}\rangle^{\mathrm{ens}}\right|\geq\epsilon\right)\leq\eta_{\epsilon}(N),$
(4)

at finite size system $N$. Here $\langle\hat{O}\rangle^{\mathrm{ens}}$
presents the ensemble average and $\eta_{\epsilon}(N)$ is a function that
vanishes at the thermodynamic limit ($N\rightarrow\infty$). We use a specific
representation of the TPQ state termed as the canonical TPQ state [2],

$|\beta,N\rangle\equiv
e^{-\frac{\beta}{2}\hat{H}}\left|\psi_{R}\right\rangle,$ (5)

where $\beta$ denotes the inverse temperature, and the state
$|\psi_{R}\rangle$ is random state. Calculating the expectation value of the
local operator
$\langle\hat{\mathcal{O}}\rangle^{\mathrm{TPQ}}_{\beta,N}\coloneqq\langle\beta,N|\hat{\mathcal{O}}|\beta,N\rangle/\langle\beta,N|\beta,N\rangle$,
then taking an average for initial random states,
$\overline{\langle\hat{\mathcal{O}}\rangle_{\beta,N}^{\mathrm{TPQ}}}$,
approaches the thermal ensemble average in the thermodynamic limit,

$\left(\overline{\langle\hat{\mathcal{O}}\rangle_{\beta,N}^{\mathrm{TPQ}}}-\langle\hat{\mathcal{O}}\rangle_{\beta,N}^{\mathrm{ens}}\right)\xrightarrow{N\rightarrow\infty}0.$
(6)

To prepare the canonical TPQ state, Eq. (5), on a quantum circuit, we first
prepare the random state $|\psi_{R}\rangle$. Here, we use an algorithm
proposed in Ref. [11], which incorporates a staggered arrangement of local
unitary operations.

The problem is how to perform the imaginary time evolution
$e^{-\frac{\beta}{2}\hat{H}}$. Here, we utilize the QITE algorithm [7], which
is a quantum-classical hybrid algorithm designed to perform the imaginary time
evolution on a quantum circuit. In an infinitesimal time step ($\Delta\beta$)
of the Trotter decomposition of a given Hamiltonian
$\hat{H}=\sum_{l=1}^{L}\hat{h}[l]$, the imaginary time evolution, which is
expressed by a non-unitary operator, can be approximated by a unitary
operator, $e^{-i\Delta\beta\hat{A}[l]}$. The unitary operator can be expanded
by the following set of unitary operators,

$e^{-i\Delta\beta\hat{A}[l]}=e^{-i\Delta\beta\sum_{I}a_{I}[l]\hat{\sigma}_{I}}.$
(7)

Here, $\hat{\sigma}_{I}\in\\{I,X,Y,Z\\}^{\otimes D}$ denotes the Pauli string,
which acts on $D$-qubit domain. Since the index $I$ runs from $1$ to $4^{D}$,
the coefficients $\\{a_{I}[l]\\}$ form a real vector of size $4^{D}$. On the
other hand, we suppose that a part of the Hamiltonian in each Trotter step,
$\hat{h}[l]$, is a $k$-local operator. The domain size $D$ should be larger
than $k$ for each Trotter step. In the case of a local Hamiltonian system, the
algorithm works well even in $D\ll N$ [7]. In our Hamiltonian (1), it includes
all-to-all qubit interactions, so that we take $D=k(=N)$ in our calculation.

Using the unitary operators, we obtain a state
$|\Phi\rangle_{\text{unitary}}=e^{-i\Delta\beta\sum_{I}a_{I}[l]\hat{\sigma}_{I}}|\psi\rangle$.
On the other hand, a desired state is $|\Phi\rangle_{\text{non-
unitary}}=Ce^{-\Delta\beta\hat{h}[l]}|\psi\rangle$, where $C$ is a
normalization factor $C=\langle\psi|e^{-2\Delta\beta\hat{h}[l]}|\psi\rangle$.
Here, we initialize the state $|\psi\rangle$ with a random state for our
purpose of preparing the canonical TPQ state. To find an optimized value of
$a_{I}[l]$, we minimize the norm difference between these two states, $\|\
|\Phi\rangle_{\text{unitary}}-|\Phi\rangle_{\text{non-unitary}}\ \|,$ using
classical computers.

In the process of fixing the value of $a_{I}[l]$, we calculate ($4^{D}\times
4^{D}$) matrix elements containing the expectation values of the Pauli
strings. Ideally, the calculation of the expectation values is performed by
quantum computers in the QITE algorithm. The bottleneck of this algorithm is
the calculation of these numerous expectation values, in particular, for our
non-local Hamiltonian case which requires a large value of $D$. To address
this computational obstacle, we developed a method to handle multiple
Hamiltonian terms with different localities in a single Trotter step.
Furthermore, by utilizing the symmetry of Pauli strings, we reduce the
computational cost for both quantum and classical calculations. These
improvements allow us to compute with large system sizes.

To obtain the thermodynamic quantity from the calculation data given by the
QITE algorithm, we have to take double limits; namely the
$\Delta\beta\rightarrow 0$ limit at finite $N$ to remove the Trotter errors
and then $N\rightarrow\infty$. After taking the first extrapolation, we can
see that the result of QITE will be consistent with that of TPQ by classical
calculation. After the second extrapolation, we eventually obtain the value of
thermodynamic quantity.

##  4 Simulation results

###  4.1 Calculation methods and simulation parameters

Now, we calculate the chiral condensate, Eq. (2), as a function of temperature
using three independent methods: the exact diagonalization method, the “TPQ
method”, and the “QITE method”. In the second method, we classically prepare
the TPQ state exactly as defined in Eq. (5) and calculate the expectation
value of the chiral condensate. Also in the third method, we generate the TPQ
state, but here we perform the imaginary time evolution using the QITE
algorithm. This is a quantum-classical hybrid algorithm, but here we simulate
the quantum part classically using a state vector simulator.

The lattice spacing and the coupling constant are set to $a=0.80$ and
$g=1.00$, respectively. The lattice size $N$ is $4$–$12$. The mass parameters
in this work are massless and $m/g=0.15$. In the massive case, we also study
the non-zero $\theta$ regime. In both the TPQ and QITE methods, we utilize the
same set of $100$ initial random states and take the average over them.
Furthermore, in the QITE method, we take the Trotter step in the range of
$\Delta\beta=0.05$–$0.25$.

###  4.2 Results of the massless case

First, we study the massless case as a feasibility test of our calculation
strategy. In this case, the analytical result is known as shown in Eq. (3).
Figure 1 depicts the comparison between the results of the TPQ method (blue
circle symbol) and the exact diagonalization results (red curve) at finite
$N$.

Figure 1:  The comparison between the $\beta$-dependence of the chiral
condensates calculated by the TPQ method (blue circle symbol) and the exact
diagonalization (red curve) at finite $N$.

The results of the TPQ method do not agree with the exact diagonalization
results for each $N$, but the discrepancy becomes smaller in larger $N$. We
extrapolate these data at each $\beta$ at $N=4,6,8,10$, and $12$ with the
quadratic function and estimate the value at $N\rightarrow\infty$.

Figure 2 illustrates the $\beta$-dependence of the chiral condensate at the
thermodynamic limit. Here, we plot the analytical result obtained by the
Lagrangian formalism (black solid curve), the extrapolated value of the exact
diagonalization results (red triangle symbol), and the extrapolated value of
TPQ method (blue circle symbol). The statistical error of the extrapolated
values comes from the fitting error of the extrapolation. These three results
agree with each other within the statistical error as expected from Eq. (6).

Figure 2:  Comparison of the results in the thermodynamic limit obtained by
the TPQ method (blue circle symbol), the exact diagonalization (red triangle
symbol), and the analytical calculation, Eq. (3) (black solid curve). The
black dotted line shows the value at zero temperature [10].

Furthermore, we can see that at $\beta=10$ the result converges to the value
at zero temperature $\langle\bar{\psi}\psi\rangle\simeq-0.1599$ [10]. This
indicates that $\beta=10$ corresponds to a sufficiently low-temperature.

Next, we examine the QITE method. Here, we fix the lattice size as $N=6$. We
take $\Delta\beta=0.05,0.10,0.125,0.20$, and $0.25$ and investigate the
effects of the Trotter error. In Figure 3, we plot the absolute difference
between results by the QITE and TPQ methods as a function of the Trotter step
size ($\Delta\beta$). It can be fitted well using the quadratic function of
$\Delta\beta$ and the extrapolated value at $\Delta\beta\rightarrow 0$ is
consistent with zero within the statistical error. This result indicates that
the Trotter error is under control and the QITE method reproduces the results
of the TPQ method. Combined with the consistency between the results of the
TPQ method and the analytical results in the thermodynamic limit, our strategy
based on TPQ and QITE looks promising to study the local observables at
finite-temperature even for the Schwinger model Hamiltonian, which includes
non-local terms.

Figure 3:  The $\Delta\beta$-dependence of the Trotter error in the QITE
method. The blue circle symbol shows the absolute difference of the chiral
condensate calculated by the TPQ and QITE methods and the blue square symbol
shows the extrapolated value of the absolute difference in the
$\Delta\beta\rightarrow 0$ limit.

###  4.3 Results of the massive case

We now consider the massive and finite $\theta$ regimes. Figure 4 shows the
value of the chiral condensate computed by the QITE method in the $\beta$ –
$\theta$ plane. Here, we take $m/g=0.15$ and $0\leq\theta\leq\pi$ with the
$\theta/2\pi=0.1$ interval.

Figure 4:  The density plot of the chiral condensate
$\langle\bar{\psi}\psi\rangle$ with $N=6$, calculated by the QITE method.

We can see that the absolute value of the chiral condensate is large only in
small $\theta$ and large $\beta$ regimes. Outside this region, it gradually
approaches zero, indicating that the (discrete) chiral symmetry is being
restored. The next task is the double extrapolation of $\Delta\beta\rightarrow
0$ and $N\rightarrow\infty$, which is an ongoing work.

##  5 Summary and discussion

In this study, we simulated the Schwinger model at finite temperatures using
the TPQ state approach and the QITE algorithm. As a feasibility test, we first
considered the massless Schwinger model and showed that the (classical) TPQ
method reproduces the result given by the exact diagonalization in the
thermodynamic limit. Furthermore, these results are consistent with the
analytical result in the continuum limit based on the Lagrangian formalism. We
next confirmed that the results of the QITE method converge to those of the
(classical) TPQ method in the Trotter error $\Delta\beta\rightarrow 0$ limit.
Thus, we found that the Trotter error for the discrete imaginary time
evolution is under control in our calculation setup. We conclude that our
method based on TPQ and QITE works well even for the Schwinger model
Hamiltonian, which includes non-local terms.

We also simulated the massive Schwinger model with non-zero $\theta$-term. We
explored the $\theta$ and $\beta$-dependence of the chiral condensate for a
finite system. The chiral condensate exhibits non-zero values in the region of
small $\theta$ and high $\beta$, and gradually approaches zero outside this
region. Taking both $\Delta\beta\rightarrow 0$ and $N\rightarrow\infty$ limits
in the massive and non-zero $\theta$ case will be reported in the near future.

## Acknowledgments

We would like to thank K. Nitadori for the useful discussions. The numerical
simulations are supported by the Yukawa-21 in YITP, Kyoto University and
HOKUSAI supercomputer at RIKEN (Project ID No. Q22577). The work of J. W. P is
supported in part by the JSPS Grant-in-Aid for Research Fellow Number 22J14732
and the JST SPRING, Grant Number JPMJSP2108. The work of E. I. and S. Y. is
supported by JST Grant Number JPMJPF2221 and Program for Promoting Researches
on the Supercomputer “Fugaku”, Joint Institute for Computational Fundamental
Science (JICFuS), Grant Number JPMXP1020230411.The work of E. I. is supported
by JSPS KAKENHI with Grant Number 23H05439, JST PRESTO Grant Number
JPMJPR2113, and JSPS Grant-in-Aid for Transformative Research Areas (A)
JP21H05190. The work of S. Y. is supported by JSPS Grant-in-Aid for Scientific
Research (A) JP21H03455, and the COE research grant in computational science
from Hyogo Prefecture and Kobe City through Foundation for Computational
Science.

## References

  * [1] S. Sugiura and A. Shimizu Physical Review Letters 108 no. 24, (Jun, 2012) .
  * [2] S. Sugiura and A. Shimizu Physical Review Letters 111 no. 1, (Jul, 2013) .
  * [3] Z. Davoudi, N. Mueller, and C. Powers arXiv:2208.13112 [hep-lat].
  * [4] S. McArdle, T. Jones, S. Endo, Y. Li, S. C. Benjamin, and X. Yuan npj Quantum Information 5 no. 1, (Sep, 2019) .
  * [5] T. Liu, J.-G. Liu, and H. Fan Quantum Information Processing 20 no. 6, (Jun, 2021) .
  * [6] K. Seki and S. Yunoki PRX Quantum 2 (Feb, 2021) 010333.
  * [7] M. Motta, C. Sun, A. T. K. Tan, M. J. O’Rourke, E. Ye, A. J. Minnich, F. G. S. L. Brandão, and G. K.-L. Chan Nature Physics 16 no. 2, (Nov, 2019) 205–210.
  * [8] M. Honda, E. Itou, Y. Kikuchi, L. Nagano, and T. Okuda Phys. Rev. D 105 no. 1, (2022) 014504, arXiv:2105.03276 [hep-lat].
  * [9] I. Sachs and A. Wipf arXiv:1005.1822 [hep-th].
  * [10] D. J. Gross, I. R. Klebanov, A. V. Matytsin, and A. V. Smilga Nuclear Physics B 461 no. 1-2, (Feb, 1996) 109–130.
  * [11] N. Hunter-Jones arXiv:1905.12053 [quant-ph].


# Leveraging IoT and Weather Conditions to Estimate the Riders Waiting for the
Bus Transit on Campus
††thanks: 1 Ismail Arai was also affiliated as a visiting scholar at James
Madison University, Virginia, USA while working on parts of this paper.

Ismail Arai 1 Information Initiative Center
Nara Institute of Science and Technology
Nara, Japan
<EMAIL_ADDRESS>Ahmed Elnoshokaty  Computer Information Systems
Northern Michigan University
Michigan, USA
<EMAIL_ADDRESS>Samy El-Tawab  College of Integrated Science and Engineering
James Madison University
Virginia, USA
<EMAIL_ADDRESS>
###### Abstract

The communication technology revolution in this era has increased the use of
smartphones in the world of transportation. In this paper, we propose to
leverage IoT device data, capturing passengers’ smartphones’ Wi-Fi data in
conjunction with weather conditions to predict the expected number of
passengers waiting at a bus stop at a specific time using deep learning
models. Our study collected data from the transit bus system at James Madison
University (JMU) in Virginia, USA. This paper studies the correlation between
the number of passengers waiting at bus stops and weather conditions.
Empirically, an experiment with several bus stops in JMU, was utilized to
confirm a high precision level. We compared our Deep Neural Network (DNN)
model against two baseline models: Linear Regression (LR) and a Wide Neural
Network (WNN). The gap between the baseline models and DNN was 35% and 14%
better Mean Squared Error (MSE) scores for predictions in favor of the DNN
compared to LR and WNN, respectively.

###### Index Terms:

IoT (Internet of Things), Transit Systems, Machine Learning (ML), Data
Analytics, Intelligent Transportation Systems (ITS)

##  I Introduction

For the Transit Bus Management System, data-driven fleet management strategies
empowered by precise models that predict the number of passengers waiting at
the bus stops are essential. On a university campus, students often have to
wait in additional time when a packed bus arrives. To solve the problem, we
have developed the Internet of Things (IoT) system [1] that estimates the
number of passengers by analyzing passengers’ smartphones’ Wi-Fi frames. Even
though the system can grasp the bus stop’s occasion, it still has not been
spun out from a research prototype with the lack of feasibility in the real
field. Other data sources captured from the bus (e.g., camera feed) can be
considered ground truth [2].

This paper proposes a machine learning utilization technique that employs the
IoT passenger counter system to provide the labeled training data. The
learning phase inputs the weather data, transportation information, campus
schedule, and the number of passengers waiting at the bus stop. After building
the machine learning model, it can estimate the number of passengers without
the IoT system. Our group has noticed a significant relationship between the
number of passengers waiting for the bus at a particular bus station and the
weather conditions. Using these machine learning models, we can achieve a
realistic fleet management system without having IoT sensors installed
permanently at each bus stop.

The experiment took place in public transit systems at James Madison
University (JMU) in Harrisonburg, Virginia, USA, in Spring 2017. With a month
of data collected at seven bus stops, the Mean Squared Error (MSE) of Deep
Neural Network (DNN), Wide Neural Network (WNN), and Linear Regression (LR)
were 1.15, 1.34, 1.77, respectively. As a result of the experiment, we
achieved the best MSE with the DNN.

##  II Related Work

Several researchers have used IoT in the intelligent transportation world.
Applications inside cities such as Smart Parking, Bus Monitoring have been
easier to deploy with the integration of IoT devices to sense data [3, 4]. At
the same time, other highway applications (e.g., highway monitoring and
incident detection) advanced with the use of IoT devices [5, 6]. The IoT
devices’ role has improved with the advance of power batteries and reliability
of IoT [7]. Our system depends on IoT devices to improve the public transit
system’s quality of service [8]. Several researchers have integrated the power
of communication and public transportation to improve service quality (e.g.,
ridership and waiting time in public bus transit) [9, 10]. Dunlap et al. [9]
estimated passenger origin and destination (OD) information for transit lines
using IoT sensors to collect Wi-Fi and Bluetooth beacon.

The rise of data-mining-based studies and machine learning techniques has
recently improved research quality in many fields. In Intelligent
Transportation, Lathia et al. proposed a machine learning technique to enhance
the the passenger’s ticket choice by studying travel history patterns and
mining the public transport fare data collected from the bus system [11]. On
the other hand, Amato et al. studied car parking occupancy with deep learning
techniques [12]. Tahere et al. predicted five crowding levels with rich data
such as ridership 15 minutes ago [13]. Machine learning techniques and deep
learning will open the door toward more improvement to big data and data-
driven systems.

In this paper, we leverage machine learning models to study bus ridership in
one of the USA’s college cities. We integrate several parameters such as class
schedule and weather information to predict hourly passengers waiting at bus
stops.

Figure 1: System overview

##  III Framework

The proposed system estimates the number of passengers with a machine learning
technique taking as an input anonymous and abstracted Wi-Fi capture data at
each bus stop and weather data as shown in Figure 1. Some of the Wi-Fi
captured data might correlate with the number of persons since they have a Wi-
Fi enabled smartphone. On the other hand, we should keep in mind the privacy
concerns during capturing Wi-Fi frames, including MAC addresses. Weather data
is also informative for estimating the number of passengers. For example,
students would not wait for a bus for several minutes on a sunny day. The
research question is whether the weather data and the campus schedule data
effectively work as an input to machine learning for estimating the number of
passengers.

###  III-A System Design and Rational

Figure 2: Edge nodes located through the bus route

The proposed system is based on an IoT cloud computing model. The edge nodes
are Raspberry Pi $3$ (located at $7$ different stations as shown in Figure 2)
with a monitor mode enabled Wi-Fi interface that captures frames and uploads
them to the cloud-based database. Since edge nodes do not require heavy
computational need, it works with a solar panel and a rechargeable battery.
These edge nodes’ main functions are to capture the unencrypted frames,
anonymize the MAC addresses with SHA-1, and upload the data to the cloud-based
database. The other processes work on the cloud as follows:

  * •

Cleaning: Filter only the passengers’ smartphones frames. Many Wi-Fi frames
have noises due to MAC randomization, smartphones passing by a bus stop, and
devices in personal vehicles and buildings.

  * •

Learning: Training the model by learning weather data, Wi-Fi sensor data, and
some categorical information for estimating the number of passengers waiting
for the bus at the bus stop.

  * •

Estimating: Estimating the number of passengers from weather information and
categorical information.

Our system makes sure that only light processing tasks are done on the edge
nodes, while the heavy tasks of machine learning and computing are done on the
cloud. We installed an IoT node at each bus stop to capture the Wi-Fi frames
of passengers’ smartphones. There are mainly two ways to set the IoT devices
in the field. One is to install them inside the bus (hereafter call it mobile
sensing), the other is to place them at each bus stop (fixed point sensing).
Estimating the ridership is relatively easy with the mobile sensing against
the fixed point sensing because the mobile sensing can track all the
passengers inside the bus during driving. However, mobile sensing has no way
to know the number of passengers waiting at the bus stop when the bus is not
at the bus stop. The number of required devices is the same as the number of
buses and bus stops with mobile sensing and fixed point sensing. The fixed
point sensing is reasonable in our field because the number of bus stops is
less than the buses.

TABLE I: Weather data Name |  Content
---|---
dt |  Time of data calculation, unix, UTC
temp |  Temperature
feels_like |  Accounting for the human perception of weather
temp_min |  Minimum temperature among sensors in city
temp_max |  Maximum temperature among sensors in city
pressure |  Atmospheric pressure (on the sea level), hPa
sea_level |  Sea Level, meters
grnd_level |  Ground Level, meters
humidity |  Humidity, %
wind_speed |  Wind speed. Unit Default: meter/sec
wind_deg |  Wind direction, degrees (meteorological)
rain_1h |  Rain volume for the last hour, mm
rain_3h |  Rain volume for the last 3h, mm
snow_1h |  Snow volume for the last hour in liquid state, mm
snow_3h |  Snow volume for the last 3h in liquid state, mm
clouds_all |  Cloudiness, %
weather_id |  Weather condition id
weather_main |  Rain, Snow, Extreme etc.
weather_description |  Weather condition within the group

###  III-B Data Preparation

TABLE II: Raw sensor data Name | Type
---|---
Bus stop | String
Date and time in UTC | YYYY-MM-DD hh:mm:ss
MAC Address | 6 Octet
Signal Strength (dBm) | Integer
TABLE III: The number of people data Name | Type
---|---
Bus stop | String
Date and time in UTC | YYYY-MM-DD hh:mm:ss
Count | Integer

As described in Figure 1, the proposed system employs four types of data for
estimating the number of passengers: the weather data (Table I) obtained from
OpenWeather[14], the transportation data (e.g., the locations of the bus
stops), the campus schedule (e.g., class schedule), and the number of people
data converted from the raw data. Table II shows the specification of the raw
data. The IoT sensor data is cleaned and transformed into the number of people
data (Table III) to feed the machine learning. In the training phase, the
machine learning model inputs all the data, while in the testing phase, it
does not input the passenger count data since it is the expected output of the
model.

Figure 3: Overview of the segmentation and the filtering

(a) Raw data (b) Cleaned data

Figure 4: MAC addresses every minutes at 017 (PhysChem) on 4/21

We have cleaned the data as follows:

  1. 1.

Random MAC Address: For a privacy measure, iOS 8 and Android 8 started to
provide a MAC randomization feature, randomizing a source MAC address when the
device is trying to find a Wi-Fi network with a probe request frame. When we
identify the devices with the probe request frame, the random MAC address will
be noise. We have removed such frames by checking G/L and I/G bits in the MAC
address. 111Our data collection was done while iOS versions implementing
randomization were not yet released.

  2. 2.

MAC Addresses observed at only one bus station: The passengers’ smartphone’s
Wi-Fi frames should be captured at two or more bus stations. If a MAC address
is found at only one bus stop, we assume that a person only passed by the bus
stop or a device is installed around the bus stop (e.g., Wi-Fi enabled PC in a
building). We know that a passenger phone battery may die while riding the
bus; we assume that this case can be ignored with big data.

  3. 3.

Segmentation and filtering based on the duration: As shown in Figure 3,
samples are split into segments with a given threshold time. If the segment’s
period is shorter than $D_{\text{min}}$ or longer than $D_{\text{max}}$, it is
discarded. The rest of the segments will be stored in the cloud-based
database.

  4. 4.

Unrealistic RSSI: Usually, the waiting passengers’ length is within ten or
more meters. In this condition, Received Signal Strength Indication (RSSI)
will be in a certain range.

Figure 4 shows the result of cleaning the data of bus stop $017$ (PhysChem) on
April 21 in 2017. We used 2 and 30 minutes for $D_{\text{min}}$ and
$D_{\text{max}}$, respectively. And we assumed the RSSI range for capturing
Wi-Fi frames from smartphones within 10 meters is between -30 dBm and -80 dBm.
Against the raw data shows the unrealistic numbers of unique MAC addresses all
over the time, the cleaned data shows like a transition of the number of
passengers in a day. Though we have not confirmed the cleaned data’s
correctness due to the lack of the ground truth, it will be no problem because
it is generally consistent with the trends we have seen visually at the bus
stop.

To prepare machine learning data, we derived features (e.g., academic week) as
a week of the semester from the date. Also, we derived the morning/evening
feature from time. We further created dummy variables for all values of
categorical features (e.g., weather_description and Bus_stop) in Tables I and
II, respectively. For example, the weather_description having ”Rain” and
”Cloud” will be True/False values of weather_description_Rain, and
weather_description_Cloud.

###  III-C Machine Learning

For the Transit Bus Management System, data-driven fleet management strategies
empowered by precise models that predict the number of passengers waiting at
transit bus stops based on real-time weather conditions are essential. For
example, the system could suggest dispatching many buses to a particular route
depending on the weather conditions, instead of having near-empty buses some
days and full one’s others. We highlight the following machine learning
techniques: Linear Regression (LR) analysis is a predictive modeling technique
that estimates the relationship between two or more variables. Recall that a
correlation analysis does not assume the causal relationship between two
variables.

$Y_{i}=f(X_{i},\theta)+\epsilon_{i}$ (1)

where for $k$ features and $n$ data points, $i=1,...,n$ and $\theta$ =
($\theta_{1}$,…,$\theta{k})^{T}$ is the vector of the parameters to be
estimated. The error $\epsilon_{i}$ are with a mean equal to zero and an
unknown $\sigma^{2}$. LR can be used when the features measured have a linear
correlation with the dependent variable. Since not all features are having a
linear relationship with the passengers’ count to be predicted, it is expected
that good results cannot be obtained using a plain LR model.

Classification and regression trees (CART) are binary decision trees. The tree
is constructed by splitting the entire data into subsets by using all the
independent variables. The goal is to produce terminal leaves that are as
homogeneous as possible with respect to the target variable. Regression trees
can be notable accurate in the case of nonlinear problems. For every node $t$,
$1/n\sum_{i=1}^{n}(Y_{i}-\bar{Y_{i}}(t))^{2}$ is the node sum of squares. In
other words, it is the total squared deviations of $Y_{i}$ in $t$ from their
average. The regression tree is formed by splitting the nodes iteratively so
that the decrease in R(T) is maximized, where R(T) sums up all the sums of
squares within all the nodes.

Neural networks (NN) are models in which input features flow through hidden
layers towards the output [15]. The neural network learns new feature spaces
by first computing the affine (linear) transformations of the given inputs and
then applying a nonlinear function (rectified linear unit ReLU), which will be
the next layer’s input. This process will continue just before the output
layer when a linear transformation will be applied for predicting the hourly
passenger demand. NN with no hidden layers can only learn how to solve
linearly separable problems. NN with two or more hidden layers capture non-
linearity is a natural aspect. NN with three hidden layers is usually
considered a deep-neural network. For this study, we expect the performance of
a deep neural network (DNN) model that consists of an input layer, three
hidden layers, and an output layer to outperform LR, CART, and Neural Networks
with one hidden layer (WNN).

In the first layer, layer 1 of the NN architecture, each neuron receives a set
of X-values (numbered from 1 to $n$) from input vector $X$ and computes the
predicted $\hat{Y}$ value. Vector $X$ contains the weather’s value and the bus
features, for one example, from the training set. Each node in layer 1 has its
own set of parameters, usually referred to as $W$ (column vector of weights)
and $b$ (bias), as shown in Figure 5. In each iteration, the neuron calculates
a weighted average of the vector $X$, based on its current weight vector $W$.
Then, it adds bias where weights and bias change during the learning process
to minimize prediction error. Finally, the result of this calculation is
passed through a ReLU.

Figure 5: Neural Network Neuron

The equation in Figure 5 used $X$ and $\hat{Y}$, which are the column vector
of features and the predicted value, respectively, for a node in layer 1. The
$X$ vector is, therefore, layer 0 (input layer). When switching to the general
notation for layer $k$, like layer 2 and layer 3, we use $a^{[k-1]}$ and
$a^{[k]}$, which are the input to the hidden layer $k$ and the activated
output predicted value of layer $k$, respectively (activation function $g$
used is ReLU). Accordingly, for any hidden layer $k$ with $n^{[k]}$ neuron,
each neuron performs a similar calculation according to the following
equations:

$Z_{i}^{[k]}=W_{i}^{T}.a^{[k-1]}+b_{i}$ (2) $a_{i}^{[k]}=g^{[k]}(Z_{i}^{[k]})$
(3)

The activation function $g$ for any hidden layer is ReLU, except for the
output layer, which is a linear function to predict the hourly passenger
demand. The model learns through iterations of forward-feeding and backward
propagation from the input layer, hidden layers, to output and back. Back-
propagation is seen as a common approach where random weights are assigned;
the output seen is compared with the test data; the output error is calculated
comparing the two (i.e., actual output vs. expected output in MSE loss
function). The layer immediately closer to the output layer adjusts its
weights, leading to weight adjustments in the subsequent inner layers until
the error rate is reduced [15]. NN models extract features by weight
allocation and weight decay through iterations of forward-feeding and backward
propagation techniques.

##  IV Experimental results and discussion

TABLE IV: Experimental data specification Name | Value
---|---
Date | From 2017-04-05 to 2017-05-04
Bus stops | 7
Raw data records | 210,161,203
Cleaned data records | 9,981,883
Figure 6: Estimated numbers of passengers waiting at each bus stop with the
cleaned data.

We have collected the experimental data for a month, as shown in Table IV. IoT
devices were installed at the seven bus stops shown in Figure 2 from April 5
to May 4 in 2017. Figure 6 shows the estimated numbers of passengers waiting
at each bus stop with the cleaned data. The numbers of passengers were around
20.

The weather predictors (Table I) and the passenger counts (Table III) are
recorded hourly and every second, respectively. We aggregated the passenger
counts data on an hourly basis and averaged the reading for the predictors. We
then merged the aggregated bus data with the weather data to have our
granularity of analysis on the hourly level. We further normalized our data
and derived new predictors like weekdays/weekends and morning/evenings. We
also extracted the week of the academic semester from the date, which is shown
to be informative predictive features, as in Figure 7.

Figure 7: Predictors Impurity Index

After the data preparation and transformation stage, we benchmarked our DNN
model (an input layer, three hidden layers, and one output layer) against
baseline models: a WNN model with one hidden layer only and traditional LR.
Unlike traditional computational intelligence approaches, deep learning
predictive performs much better with large datasets. The model learns features
to look for and to make better predictions. We split the data randomly to 80%
training and 20% testing.

For better prediction, we further fine-tuned the DNN and WNN through another
round of random splitting of the training dataset into 80% training and 20%
validation. We then plotted and analyzed the MSE of the training and
validation datasets for WNN and DNN over 100 epochs, as shown in Figures 8 and
9.

Figure 8: Deep Neural Network MSE Figure 9: Wide Neural Network MSE Figure 10:
Mean Squared Error (MSE) of Medels.

As a final step, and after fine-tuning the parameters of NN models using the
validation dataset, we compared the DNN model against the two baseline models
using the testing dataset. The MSE for the testing dataset for the DNN model
was $1.15$, compared to $1.34$ and $1.77$ for WNN and LR models, respectively,
as shown in Figure 10. We offer empirical evidence on DNN models’
effectiveness for predicting the hourly number of passengers waiting in bus
stations by achieving $35\%$ and $14\%$ better prediction for the DNN model
over LR and WNN models, respectively.

Furthermore, to study the influence of our dataset’s informative features, we
use an ensemble of gradient boosting decision trees. A benefit of using
ensembles of decision tree methods like gradient boosting is that they can
automatically estimate feature importance from a trained predictive model.
Decision trees provide more information about the relationships of the
features than a standard correlation index. We rank the predictive power of
the input features, and we find that there are seven most decisive features
for partitioning the decision tree to predict the number of passengers waiting
at the bus stop. The seven features as shown in Figure 7 are Bus Stop, Week
Day, Pressure, Minimum Temperature, Humidity, Clouds, and Week of Semester. It
summarizes the reduction in the impurity index over all trees when a
particular feature is pointed during the trees’ internal space partition over
several epochs.

##  V Conclusion

This paper studied the correlation between the passengers waiting at bus
stations and the weather conditions using a deep learning algorithm. This
paper gives empirical evidence on the importance of incorporating predictive
modeling in intelligent transportation to maximize fleet utilization. We
highlighted the importance of applying deep learning models to precisely
predict the number of passengers waiting at bus stops in intelligent
transportation systems. This research also leveraged a broad set of features
from IoT devices like smartphone Wi-Fi data in conjunction with detailed
weather information. Our results show four of the top seven decisive features
(Pressure, Minimum Temperature, Humidity, and Clouds) were related to
temperature and weather. It gives empirical evidence on the importance of
weather information in influencing riders’ behavior and, hence, better
predicting the number of passengers waiting at the bus stops.

We studied only one month of campus-level data in 2017. In future work, data
collection could take place yearly and on the city scale to confirm the
generalization of the predictive model’s performance. A limitation of this
study is that we did not take into consideration any unusual event like a
pandemic. In future work, we will incorporate data of 2020 and investigate
more informative features. Among the features to be explored are the
government’s restriction levels (stay-at-home, business is open with
restrictions or no restrictions, etc.) The study should also incorporate
features like the number of daily reported cases reported in the city and
lockdown dates to improve the prediction of the number of passengers waiting
at bus stops during the COVID-19 pandemic.

## ACKNOWLEDGMENT

This work is funded through a 4-VA research grant (https://4-va.org) and JSPS
KAKENHI Grant Numbers JP20K11789, JP20H04183.

## References

  * [1] S. El-Tawab, R. Oram, M. Garcia, C. Johns, and B. B. Park, “Data analysis of transit systems using low-cost IoT technology,” in _2017 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops)_. IEEE, 2017, pp. 497–502.
  * [2] S. Ryu, B. B. Park, and S. El-Tawab, “WiFi Sensing System for Monitoring Public Transportation Ridership: A Case Study,” _KSCE Journal of Civil Engineering_ , vol. 24, no. 10, pp. 3092–3104, 2020.
  * [3] A. Khanna and R. Anand, “IoT based smart parking system,” in _2016 International Conference on Internet of Things and Applications (IOTA)_. IEEE, 2016, pp. 266–270.
  * [4] M. Garcia, P. Rose, R. Sung, and S. El-Tawab, “Secure Smart Parking at James Madison University via the Cloud Environment (SPACE),” in _2016 IEEE Systems and Information Engineering Design Symposium (SIEDS)_ , 2016, pp. pp–271.
  * [5] A. Salem, T. Nadeem, M. Cetin, and S. El-Tawab, “Driveblue: Traffic incident prediction through single site bluetooth,” in _2015 IEEE 18th International Conference on Intelligent Transportation Systems_. IEEE, 2015, pp. 725–730.
  * [6] O. Popescu, S. Sha-Mohammad, H. Abdel-Wahab, D. C. Popescu, and S. El-Tawab, “Automatic incident detection in intelligent transportation systems using aggregation of traffic parameters collected through V2I communications,” _IEEE Intelligent Transportation Systems Magazine_ , vol. 9, no. 2, pp. 64–75, 2017.
  * [7] H. V. Chand and J. Karthikeyan, “Survey on the Role of IoT in Intelligent Transportation System,” _Indonesian Journal of Electrical Engineering and Computer Science_ , vol. 11, no. 3, pp. 936–941, 2018.
  * [8] S. El-Tawab, I. Arai, A. Salman, and B. B. Park, “A Framework for Transit Monitoring System Using IoT Technology: Two Case Studies,” in _2020 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops)_. IEEE, 2020, pp. 1–6.
  * [9] M. Dunlap, Z. Li, K. Henrickson, and Y. Wang, “Estimation of Origin and Destination Information from Bluetooth and Wi-Fi Sensing for Transit,” in _Transportation Research Board 95th Annual Meeting_ , no. 16-6837, 2016.
  * [10] T. Oransirikul, R. Nishide, I. Piumarta, and H. Takada, “Feasibility of Analyzing Wi-Fi Activity to Estimate Transit Passenger Population,” in _IEEE 30th International Conference on Advanced Information Networking and Applications (AINA)_ , 2016, pp. 362–369.
  * [11] N. Lathia, C. Smith, J. Froehlich, and L. Capra, “Individuals among commuters: Building personalised transport information services from fare collection systems,” _Pervasive and Mobile Computing_ , vol. 9, no. 5, pp. 643–664, 2013.
  * [12] G. Amato, F. Carrara, F. Falchi, C. Gennaro, and C. Vairo, “Car parking occupancy detection using smart camera networks and deep learning,” in _2016 IEEE Symposium on Computers and Communication (ISCC)_. IEEE, 2016, pp. 1212–1217.
  * [13] T. Arabghalizi and A. Labrinidis, “How full will my next bus be? a framework to predict bus crowding levels,” in _Urbcomp 2019_ , 2019.
  * [14] “Openweather,” https://openweathermap.org/, [Online; accessed 16-December-2020].
  * [15] R. Hecht-Nielsen, “Theory of the backpropagation neural network,” in _Neural networks for perception_. Elsevier, 1992, pp. 65–93.