# Temporal Multimodal Multivariate Learning

Hyoshin Park<EMAIL_ADDRESS>0002-1490-5404 North Carolina A&T State
University
Dept. of Comput. Data Sci. & Eng. ,  Justice Darko  ,  Niharika Deshpande
<EMAIL_ADDRESS>North Carolina A&T State University
Dept. of Comput. Data Sci. & Eng. ,  Venktesh Pandey<EMAIL_ADDRESS>North
Carolina A&T State University
Dept. of Civil, Archi., & Envir. Eng.Greensboro, NCUSA ,  Hui Su  Stratosphere
& Upper Troposphere
Jet Propulsion Laboratory
CalTech, Pasadena, CA, USA<EMAIL_ADDRESS>,  Masahiro Ono  Robotics
Jet Propulsion Laboratory
CalTech, Pasadena, CA, USA<EMAIL_ADDRESS>,  Dedrick Barkely  ,
Larkin Folsom  North Carolina A&T State University
Dept. of Comput. Data Sci. & Eng.Greensboro, NCUSA ,  Derek Posselt
Atmospheric Physics & Weather
Jet Propulsion Laboratory, California Institute of Technology, Pasadena
<EMAIL_ADDRESS>and  Steve Chien  Artificial Intelligence
Jet Propulsion Laboratory, California Institute of Technology, Pasadena
<EMAIL_ADDRESS>
(2022)

###### Abstract.

We introduce temporal multimodal multivariate learning, a new family of
decision making models that can indirectly learn and transfer online
information from simultaneous observations of a probability distribution with
more than one peak or more than one outcome variable from one time stage to
another. We approximate the posterior by sequentially removing additional
uncertainties across different variables and time, based on data-physics
driven correlation, to address a broader class of challenging time-dependent
decision-making problems under uncertainty. Extensive experiments on real-
world datasets ( i.e., urban traffic data and hurricane ensemble forecasting
data) demonstrate the superior performance of the proposed targeted decision-
making over the state-of-the-art baseline prediction methods across various
settings.

Multimodal Entropy, Multivariate Learning, Spatiotemporal Information,
Sequential decision

††copyright: acmcopyright††journalyear: 2022††doi:
10.1145/3534678.3539159††conference: KDD ’22: ACM SIGKDD Conference on
Knowledge Discovery and Data Mining; August 14–18, 2022; Washington
DC††booktitle: KDD ’22: ACM SIGKDD Conference on Knowledge Discovery and Data
Mining, August 14–18, 2022, Washington DC††price: 15.00††isbn:
978-1-4503-9385-0/22/08

Figure 1. Online gain in temporal, multimodal, and multivariate prediction
uncertainty between prior and posterior. Each cell can be assumed to have a
combination of discrete travel time distributions (i.e., 2, 5, 10, 30min) with
different weights.

##  1\. Introduction

In recent years, deep reinforcement learning (RL) models have improved the
solution quality of online combinatorial optimization problems (Bello et al.,
2017; Dai et al., 2017), yet cannot match the real-world online systems (Gauci
et al., 2019). Consider a Mars autonomous navigation problem under uncertainty
where information sampled from aerial agents mapping large areas and ground
agents observing traversability can be transferred to other agents for safe
and efficient navigation (Ono et al., 2020). For the posterior approximation
(Bengio et al., 2021) (e.g., Markov decision process) with online data
assimilation (Folsom et al., 2021), we need a new framework to actively sample
useful information. Traditional information metrics like Shannon Entropy or
Kullback-Leibler (KL) Divergence fail to incorporate future uncertainty with
more than one peak probability distributions (Folsom et al., 2021). Shannon
Entropy (Shannon and Weaver, 1949) cannot distinguish distributions with
multiple weights (e.g., bimodal distributions) because it only considers raw
information gain, treating all information as equally valuable. KL Divergence
(Kullback and Leibler, 1951) introduces a bias toward only one mode (e.g.,
Exclusive, Reverse) or toward the mean of the modes (e.g. Inclusive, Forward)
with non-symmetrical measures of information gain. Recent computer vision
models (Das Gupta et al., 2015) cannot address unobserved heterogeneity
causing multimodal distributions since representing the information gain using
KL Divergence requires comparison to an “ideal” distribution. This biases the
model toward searching only for some types of uncertain solutions while
ignoring other potentially more valuable solutions. When the probability
distribution is heavily weighted at either extreme, the system cost either
experience very high true savings or negative true savings. Those combinations
of probability distributions vary across time and location and evolve as new
observations become available.

Our main contribution is the development of a new family of online predictive
decision making models, Temporal Multimodal Multivariate Learning (TMML), that
can indirectly learn and transfer online information from multiple modes of
probability distributions and multiple variables between different time
stages, which can be applied to many routing problems under uncertainty such
as Mars exploration (Folsom et al., 2021), Hurricane sensing (Darko et al.,
2022a), and urban routing (folsomdynamic). Preliminary remedy (Folsom et al.,
2021) partially filled this gap by grouping similar types of locations based
on their classified output (e.g., sandy or rocky), used in optimizing vehicle
routing to improve the prediction uncertainty proven to be superior to
partially observable Markov decision processes. Locations with broad bimodal
distributions offered the greatest potential delta between the expected and
true savings. We expand this bimodal learning to multimodal learning and the
maximum information gain is accomplished by identifying the time-dependent
similarity between the probability distribution of variables. With existing
routing algorithms, opportunities for data collection are commonly skipped or
missed entirely. A technology to collect more valuable observations while
carefully spending system resources will add significant value to the
autonomous decisions. The result will be an increased likelihood of
encountering unexpected scientific discoveries, creating new opportunities to
characterize uncertainties, reconciling the desire to explore further with the
desire to explore in-depth, and eliminating the dichotomy between engineering
limitations and scientific discovery.

Cells in the grid of Figure 1 with a similar combination of distributions are
clustered together based on the similarity between the combinations (e.g., 6
cells outlined in black). As users traverse the map, exploration of a cell in
a cluster will remove the travel time uncertainty of other cells in the same
cluster. In other words, exploring one cell of the cluster will identify which
of the two travel time distributions applies to that explored cell, and to all
other unexplored cells in the cluster. However, each cell has two travel time
distributions with peaks of different heights. Therefore, we do not update all
cells that share a single travel time distribution; we update cells that have
similar combinations of distributions. This technique can be applied to
several real-life applications. For example, assume that each cell with
heterogeneous users presents a mixture of traffic conditions (Park et al.,
2010). An online RL simplifies multi-modality to a unimodal distribution
$X\sim\mathcal{N}(35,\,10^{2})$ resulting in a lost opportunity to remove
uncertainties in other locations.

Several techniques learn and transfer information gained from multimodal
distribution data in information theory (Shannon and Weaver, 1949; Kullback
and Leibler, 1951) for global uncertainty removal: grouping similar
combinations of distributions, sampling from similar groups and updating
posteriors, and solving probabilistic optimization (Bello et al., 2017; Bengio
et al., 2021) for online routing. Those are necessary to optimize the
probabilistic global routing problems based on knowledge learned and
transferred in a sequence, and data is typically obtained from parts and not
analyzed as a whole. While previous research (Folsom et al., 2021; Folsom,
2021; Darko et al., 2022a) addressed bimodal learning and full uncertainty
removal, we address multimodal learning integrated with partial information
gains from temporal and multivariate learning applied to urban traffic and
hurricane data.

##  2\. Multimodal Learning

Reduced uncertainty in bimodal travel time information can be processed and
transferred from one agent to another agent (Folsom et al., 2021). A prototype
of bimodal uncertainty removal in an 10$\times$10 grid map is extended to
multimodality of each cell through clustering similar probability
distributions for multimodal learning (Figure 2). The agent is allowed to move
in four directions: up, down, left, and right. Diagonal moves are not allowed.
Cells in the grid are numbered row-wise, starting with zero for the first
cell. Grids are used because the prior state of the map will be defined using
image analysis, which defines the state of a region using pixels of a fixed
size. Those 2, 5, 10, and 30 minutes from Figure 1 bimodal is expressed as
clusters. The key statistics for travel time distribution in each cell are
based on lower and upper bounds, with probability $P(T)$ represented as real
numbers between 0 and 1 in the model (Folsom et al., 2021) in Figure 2 further
extended to incorporate multimodality.

Figure 2. Multimodal extensions to the proposed path (Folsom et al., 2021),
compared against theoretically minimizing the initial expected travel time
(ETT) or the highest probability classification-based travel time the
Max$[P(T)]$. The grid shows each cell type by number (white) and filled color.

In this study, multimodal learning enhances the scientific and engineering
value of autonomous vehicles by finding the best routes based on the desired
level of exploration, risk, and constraints. In the proposed exploration
framework, each grid cell (Figure 2) contains a unique probabilistic
distribution of travel time for formulating the best options to travel with
partial, sequential, and mixture of information gain, with various probability
distributions. An example application is the Machine learning-based Analytics
for Rover Systems (MAARS) (Ono et al., 2020), where agents analyze images for
autonomous driving feature detection, and assist scientists by selectively
collecting data without interrupting drives. When agents travel through a grid
map, information can be gained by visiting cells classified with uncertainty,
observing the conditions in those cells, and estimating the true state of
other cells and observations from other agents. Existing work on energy-
optimal path planning (Tompkins et al., 2006; Wettergreen et al., 2005; Sutoh
et al., 2015) assume that energy consumption and generation are given or
immediately derivable from an existing height map. However, without prediction
of the agent’s energy consumption and generation, these methods are myopic,
simplified, and not a realistic optimization approach. An agent’s energy
consumption and generation depend on interrelated factors such as
terramechanics, the agent’s dynamics and kinematics, and terrain topography.

##  3\. Multivariate multimodal learning

Traditional machine learning frameworks overlook simultaneous observations of
more than one outcome variable in different locations and times without
lowering the prediction errors. Real-life data, behaviors, and problems
(referring to objects, values, and attributes) are non-independent and non-
identically distributed, whereas most analytical methods assume independent
and identically distributed (IID) random variables. Unfortunately, the
interdependent event relationship has been overlooked and future posterior
events have been assumed independent from other events and systems. The
dynamic impact area of a prior event could predict the probability of
posterior events (Park and Haghani, 2016b; Park et al., 2018). However, when
frequent minor events are occurring in a sequence, due to high uncertainty,
the literature could not reliably predict the dynamic spatiotemporal evolution
of a mutual relationship between events (Park and Haghani, 2016a). Machine
learning with rule extraction (Park et al., 2016) partially alleviates Black
box issues, but without an effort to reduce uncertainty by observing a ground
truth, the routing solutions are still unreliable and intractable. In this
paper, those dependencies are partially addressed by clustering
multidimensional correlation data from multiple variables through deep
clustering and when one cluster is updated, other variable data from the same
cluster are also updated.

In our case study, the multivariate clustering applied on hurricane Small
Unmanned Aerial Systems (sUAS) observation includes four variables in each
cell: wind speed, temperature, relative humidity, and pressure observed from
the boundary layer. Each cell represents a specific location in the hurricane.
Utilizing data manipulation techniques, we transform each variable to a single
vector and combine each of the four vectors to create a multidimensional data
matrix. By aggregating all cells in the map and grouping similar types of
probability distributions of multiple variables, when we observe those
variables at one location, uncertainties of other correlated variables at
other locations are realized in this study.

##  4\. Online Learning framework

Temporal learning addresses the time-dependent realization of uncertainties of
other correlated variables at other locations at other time stages, when we
observe variables at one-time stage. The online learning framework updates
sequential information based on the rapidly exploring random tree star (RRT*)
algorithm. The RRT* algorithm finds an initial path solution based on an
originally developed utility map of the environment conditioned on some
constraint. As the agent follows this path of connected way-points (nodes) and
makes new observations, the utility map is sequentially updated, generating
recourse actions to accommodate the new information. Specifically, in each
time stage, the utilities of the cells in the map are updated based on
observations made at the agent’s current location. At the first successor node
from the agent’s current location, the updated utility map is used to find a
new node in a defined search region centered at the agent’s current location.
The search regions’ radius is equal to the length of the edge connecting the
current location and the first successor node.

Considering the nodes in this region, we evaluate their utilities and select
the node with the highest utility, replacing the initial first successor node
from the current location. After pruning the previous edge connection, we then
rewire the current location node to the new node.The new node is also rewired
to the current location’s second successor node. We repeat pruning and
rewiring as the agent moves through the nodes and receives new information
(updated utilities) until it reaches the target location.

Algorithm 1 :TMML-RRT* with Online Recourse

T $\leftarrow$ InitializeTree()

T $\leftarrow$ InsertNode($\emptyset$,$z_{init}$,T)

for i=0 to i=N do

$z_{rand}\leftarrow$ Sample(i)

$z_{nearest}\leftarrow$ NearestTMML(T,$z_{rand}$)

$(z_{new},U_{new})\leftarrow$ Steer($z_{nearest},z_{rand}$)

if NoExceed($z_{new}$) then

$z_{near}\leftarrow$ Near($T,z_{new},|V|$)

$z_{max}\leftarrow$ ChooseParentTMML($z_{near},z_{nearest},z_{new}$)

T $\leftarrow$ InsertNode($z_{max},z_{new},T$)

T $\leftarrow$ Rewire($T,z_{near},z_{max},z_{new}$)

end if

end for

OptPath $\leftarrow$ OnlineRecourse(T, $n_{s},n_{g}$)

The path search in Algorithm 1 shows the Nearest function in TMML-RRT*
(NearestTMML) considering the utility at the nearest node. The NoExceed
conditional statement implements constraints. The ChooseParent function
considers the node with maximum utility within a defined region. The Rewire
reevaluates previous connections from the agent’s start location and extends
the new node to the node that can be accessed through the maximum utility.
This process in Algorithm 1 is repeated until we find the target location. As
shown in the pseudocode (Algorithm 2) in Appendix C, after the initial path
solution (based on the originally developed utility map) is found, the
OnlineRecourse function is applied to find alternative waypoints (nodes) as
the agent follows this path.

As the vehicle traverses its planned path, observations are made of the
environment. Variational Bayesian inference generates a posterior belief given
the prior belief of cell type distributions within each of the clusters. To
measure how well variational multinomial posterior distribution
$q_{\lambda}(z|x)$ approximates the true posterior $p(z|x)$, the
$\mathbb{K}\mathbb{L}$ divergence
$\mathbb{K}\mathbb{L}(q_{\lambda}(z|x)||p(z|x))$ estimates the information
lost minimized with optimal variational parameters $\lambda$. The belief about
the properties of different cell type clusters is updated en route to improve
the travel. Clustering is performed using an expectation maximization
algorithm on multinomial mixture models of the cells to identify cells with
similar probability distributions. The likelihood of observing the dataset
$P(\Upsilon|\mathbf{\alpha},\mathbf{\beta})$ for data $\Upsilon$ and Dirichlet
parameters $\mathbf{\alpha}$, $\mathbf{\beta}$ is the sum of
$\alpha_{k}P(x_{i}|\beta_{k})$ as observations $i$ goes from 1 to $N$ and
clusters $k$ goes from 1 to $K$. Using Expectation Maximization, the optimal
distribution of the data over $K$ clusters is determined by maximizing the
lower bound of the log of the likelihood. The optimal cluster index minimizes
the Bayesian Information Criterion as the difference between $D\ln{(N)}$ and
$2\ln{(\widehat{L})}$ where $D$ is the number of parameters, $N$ is the total
number of observations, and $\widehat{L}$ is the likelihood of the model.

$D_{KL}$ quantifies the information gained by revising belief from the prior
probability distribution $\mathcal{Q}$ to the posterior probability
distribution $\mathcal{P}$. The mutual information
$I(\mathcal{X};\mathcal{Y})$ between two discrete random variables
$\mathcal{X}$ and $\mathcal{Y}$ is
$I(\mathcal{X};\mathcal{Y})=D_{KL}(P_{(\mathcal{X},\mathcal{Y})}||P_{\mathcal{X}}\otimes
P_{\mathcal{Y}})$ where $P_{(\mathcal{X},\mathcal{Y})}$ is the joint
probability mass function and $P_{\mathcal{X}}$ and $P_{\mathcal{Y}}$ are the
marginal probability mass functions of the random varaibles $\mathcal{X}$ and
$\mathcal{Y}$. If $\mathcal{X}$ and $\mathcal{Y}$ are independent, then the
joint distribution $P_{(\mathcal{X},\mathcal{Y})}$ will be identical to the
product of the marginal distributions, implying that the mutual information is
zero in given equation (1).

(1)
$I(\mathcal{X};Y)=\sum_{y\in\mathcal{Y}}\sum_{\mathrm{x}\in\mathcal{X}}p_{(\mathcal{X},\mathcal{Y})}(\mathrm{x},\mathrm{y})log\Big{(}\frac{p_{(\mathcal{X},\mathcal{Y})}(\mathrm{x},\mathrm{y})}{p_{\mathcal{X}}(\mathrm{x})p_{\mathcal{Y}}(\mathrm{y})}\Big{)}$

This provides a measure of how much uncertainty is reduced for one random
variable by knowing information about the other. Mutual information can also
be formulated as an expectation value of the KL Divergence, as shown in
equation (2).

(2)
$I(\mathcal{X};\mathcal{Y})=\mathbb{E}_{\mathcal{Y}}\big{[}D_{KL}(p_{\mathcal{X}|\mathcal{Y}}||p_{\mathcal{X}})\big{]}$

##  5\. Temporal Multimodal Learning

###  5.1. Learning urban traffic

Daily commutes can be unexpectedly protracted by road closures, accidents, and
inclement weather. The quick restoration of traffic flow through the
coordinated responses of emergency vehicles may help alleviate the traffic
delays impacting the road network (Darko et al., 2021; Darko and Park, 2022).
However, the delays, which can exceed users’ planned commuting time, can cause
missed meetings, canceled appointments, and child care fees, accumulating
costs. The majority of users react similarly to the unforeseen traffic delays
and may unknowingly, collectively transfer congestion from one route to
another (Ben-Akiva et al., 1999). Current navigation systems (e.g., Google
Maps) are not customized to users’ tolerance for unexpected delays therefore
they cannot predict optimal routes (Macfarlane, 2019). Because the network is
dynamic, the route suggestion users receive at the outset of their commute may
not be optimal when they are on the road (Claes et al., 2011). In the
literature, other traffic sensing technologies commonly fail to provide
network-scale predictions under unexpected conditions. For example, current
dynamic route choice models (Gendreau et al., 2015) consider that the link
travel time realization is only based on nearby links. The multimodal
multivariate uncertainty caused by unobserved varying traffic patterns through
the day has not been considered.

To accommodate this, recent Google Deep-mind research has been using many
factors and real-time updates of traffic data for more accurate prediction of
travel time. Anticipatory routing guidance (Mahajan et al., 2019) is effective
in knowledge transfer, however, ignores the potential information gain from
probability density functions with more than one peak. Consider a network with
a grid laid on top in Figure 1, where each cell represents a small
geographical region. To find an optimal route from an origin cell to a
destination, forecasting the condition of intermediate cells is critical.
Routing literature (Darko et al., 2022b) did not use a location’s observed
data to forecast conditions at distant non-contiguous locations’ unobserved
data. We aggregate the data from all cells in the grid and cluster cells that
have similar combinations of probability distributions. When one cell of a
cluster is explored, the information gained from the explored cell can
partially remove uncertainty about the conditions in distant non-contiguous
unexplored cells of the same cluster.

Multimodal traffic learning. Assume we know a freeway link $\mathbb{A}$
historically takes 2-minutes without congestion but it may take 8-minutes due
to an unexpected event (e.g., incidents). We can cluster $\mathbb{A}$ and
$\mathbb{A}^{\prime}$ in the same correlated group assuming the bimodal travel
distributions for both links are similar. Literature ignores three benefits of
sending a platoon of vehicles to $\mathbb{A}$ instead of $\mathbb{B}$ shown at
the bottom in Figure 3: For a scenario that turned out to be 2-minutes due to
the fast clearance of the incident, 1) we can update the predicted travel time
on this link $\mathbb{A}$ so other drivers can switch either their departure
time or route to take this 2-minutes shortcut, 2) we can update travel time on
other links (e.g., $\mathbb{A}^{\prime}$) having the same type of probability
distributions. By knowing that the total travel time of a route is 4-minutes,
we can send more vehicles to this route and relieve other route congestion
that turned out to be 8-minutes due to the long clearance of the incident, 3)
we update travel time on other links (e.g., $\mathbb{A}^{\prime}$) having the
same type of probability distributions. By knowing that the total travel time
of a route $\mathbb{A}\mathbb{A}^{\prime}$ is 16-minutes, we can inform fewer
vehicles to use this route, redistribute traffic to other routes (i.e.,
$\mathbb{B}\mathbb{C}$) having shorter travel times. While the current routing
literature realize only nearby links, the realization of multimodal travel
time distributions that are derived from real-world data have not been
studied.

Figure 3. Temporal Multimodal Learning (TML) from correlation of time-varying
bimodal distributions between links.

However, recent studies (Guo et al., 2010; Zheng and Zuylen, 2010) have shown
that travel time distributions on freeways have two or more modes as distinct
peaks in the probability density function due to the mixes of driving patterns
and vehicle types. This multimodal (or bimodal) distribution exists on
arterial roads, where a vehicle passing a signal at the end of the green would
experience quite different travel time than the vehicle following behind it
that must make a stop at the red, although they traveled next to each other.
Without knowing the future traffic with confidence, the traditional choice
theory considers the bounded rationality (Han and Timmermans, 2006; Han et
al., 2015; Guo, 2013; Di et al., 2016) of the majority of agents taking a
detour to link $\mathbb{B}$, which causes congestion on $\mathbb{B}$ and
nearby roads.

Temporal multimodal traffic learning. In Figure 3, $\mathbb{A}$ has a bimodal
distribution with a mode at 8 and 2 between time stages 1 to 5, switching to a
bimodal distribution with a mode at 6 and 4 at time stage 6. For departing at
time stage 1, the time-invariant method adds travel time together either the
high or low modes of link and route $\mathbb{AA^{\prime}}$ travel time to be
either 8+8 or 2+2. The time-variant method accounts for the time needed to
traverse $\mathbb{A}$ either 8 or 2 minutes and re-evaluates the travel time
at $\mathbb{A^{\prime}}$ based on the time of entering $\mathbb{A^{\prime}}$,
may encourage a detour to Link $\mathbb{C}$ in case of 8 minutes of
realization. We assume that the state change is given based on event models
(Park and Haghani, 2016a).

The previous examples assume that the primary factor contributing to travel
time variation on a given link is the time of day. The focus of this study is
to use travel time correlation information to remove uncertainty in within-day
travel. Travel time may depend on other factors such as day-of-week, weather
patterns during the day, and special events in the region like post-game-day
traffic near a sports stadium. If data on these other variables are made
available, the same temporal learning process can be extended (see Section 6
for temporal learning under the presence of multiple variables of interest).

###  5.2. Improving KF prediction

Prediction uncertainty in travel time is improved by considering TML on real-
world traffic data. Let $C$ be the set of all links, traffic message channels
(TMCs), across the network and $T$ be a finite set of discrete-time intervals
over the morning peak period (Park and Haghani, 2015, 2016a). We consider 39
TMCs ($|C|=39$) on Interstate 540 in Raleigh, NC during 24 ten-minute time
intervals from 8:00 am to 12 noon ($|T|=24$). Probe-vehicle-based speed for
each TMC was obtained from the National Performance Management Research Data
Set (NPRMDS).

NPRMDS contains the travel time and speed information for each TMC for each
time interval across different days over the course of eight months. Due to
the day-to-day traffic randomness, the traffic speed on TMC $c\in C$ for time
interval $t\in T$, denoted by $v_{c}^{t}$, is a random variable. As argued in
the literature (Park et al., 2010; Guo et al., 2010; Zheng and Zuylen, 2010),
$v_{c}^{t}$ is likely to have a probability distribution with multiple modes
(multimodal distribution). We learn and predict $v_{c}^{t}$ within day by
analyzing the spatiotemporal correlations between random variables $v_{c}^{t}$
for all $(c,t)\in C\times T$. By clustering all $v_{c}^{t}$ variables, we
identify spatiotemporal patterns and different combinations of traffic speed
distributions with following steps:

  * •

Analyze the spatiotemporal probability distribution of variables $v_{c}^{t}$
by aggregating variation of traffic speed for a specific time interval across
eight months. For this case study, we assume that the only factors influencing
travel time are the location of TMC ($c$) and time-of-day interval ($t$).

  * •

Clustering is performed across all $C$ and $T$ using minimum message length
criteria to identify TMC’s with similar probability distributions (Wallace and
Dowe, 1997). Clustering algorithm will automatically discover the optimal
number of clusters.

Kalman Filtering (KF) Prediction $v_{c}^{t}$. We model the evolution of random
variable $v_{c}^{t}$ from one 10 minute interval to the next interval within-
day with and without information gain using KF. The data of $v_{c}^{t}$
acquired from TMCs have inherent noise due to sensor errors. Employing KF can
produce an accurate estimate of $v_{c}^{t}$ using noisy measurements over the
period (24 intervals of time). In this paper, the traditional KF is expanded
to consider the information gain from the clustering step. We model evolution
of variable $v_{c}^{t}$ from the first time interval $t=8-8:10am$ to the last
interval $t=11:50-12pm$ within a day. Figure 4 shows the KF process which is
formulated in the following equations.

Figure 4. Two steps in KF-TML: In the predict step, a model is employed to
predict the chosen state variable at next time interval $t+1$ using
measurement from previous time interval ($k$). In the update step, the
predicted state is corrected using the noisy measurements at $t+1$.

Prediction step. Projection of the state at time $t$ using the prediction at
previous time $t-1$ is given by:

(3) $\vspace{-1.0em}\hat{x}^{-}_{t}=A\hat{x}_{t-1}^{+}+B\mu_{t}$

where,

  * •

$\hat{x}_{t-1}^{+}$ is the state vector of the process at time $t-1$. In this
case, state vector considered is $\begin{bmatrix}\text{speed}\\\
\text{acceleration}\end{bmatrix}$, where, acceleration is defined as the rate
of change of speed of TMC with respect to previous time period.

  * •

Matrix $A$ is the state transition matrix of the process from the state at
$t-1$ to state at $t$ and is assumed stationary over time. That is,
$A=\begin{bmatrix}1&dt\\\ 0&1\end{bmatrix}$

  * •

$dt=1$ according to definition of acceleration defined above.

  * •

Matrix $B$ is a matrix of all zeros as there is no known external control
input factor that affects speed measurement.

  * •

$P$ is the error covariance matrix. It is interpreted as the error in
estimation according to filter.

  * •

$Q$ is the process noise defined as $Q=\begin{bmatrix}0.04&0\\\
0&1\end{bmatrix}$

  * •

We assumed the speed with a variance of 0.04 in prediction step.

Projection of error covariance of state

(4) $P^{-}_{t}=P_{t-1}^{+}A^{T}+Q$

Correction step. In this step, we determine the Kalman Gain at time $t$
(denoted by $K_{t}$) which can be interpreted as,

(5) $\text{Kalman gain}=\frac{\text{Uncertainty in
prediction}}{\text{Uncertainty in prediction + measurements}}.$

We can write,

(6) $K_{t}=P^{-}_{t}H^{T}(HP_{t}^{-}H^{T}+R)^{-1}$

where, $H$ is the connection matrix between the state vector and the
measurement vector and $R$ is the data precision matrix. In our case,
$H=\begin{bmatrix}1&0\end{bmatrix}.$

In the next step of KF, speed prediction is updated using observations
$Z_{t}$. In case of KF-no TML, $Z_{t}$ are the speed observations on a given
day while in case of KF-TML, $Z_{t}$ are mean and variance of historical speed
data.

(7) $\hat{x}_{t}^{+}=\hat{x}_{t}^{-}+K_{t}(Z_{t}-H\hat{x}_{t}^{-})$

Error covariance matrix is also updated in this step using the Kalman gain.

(8) $P_{t}^{+}=(I-K_{t}H)P_{t}^{-}$

KF-TML has an additional step as the speed prediction update with data
$Z_{t}^{+}$ obtained from information gain of correlated links.

(9) $\hat{x}_{t}^{++}=\hat{x}_{t}^{+}+K_{t}^{+}(Z_{t}^{+}-H\hat{x}_{t}^{+})$

This step also updates the error covariance matrix.

(10) $P_{t}^{++}=(I-K_{t}^{+}H)P_{t}^{+}$

The hat operator indicates an estimate of a variable. The superscripts -, +
and ++ denote predicted (prior), updated 1 (posterior 1) and updated 2
(posterior 2) estimates, respectively. The posterior 1 will be the final
prediction in KF-no TML while posterior 2 will be final outcome in KF-TML.

During the update step, observations available from the correlated links from
previous time intervals are considered. The mean and variance of speeds of all
correlated links are used as the new observation in the update step.
Therefore, traditional KF has only one update step but in this paper, the
algorithm is modified to have two updates, one with mean and variance of
historical data of 8 months and the other with mean and variance of correlated
speed data obtained from the clustering step.

The $(1,1)$ element in matrix P denotes the variance in estimation of speed.
Percentage change in $P(1,1)$ with information gain with respect to $P(1,1)$
without information gain is calculated.

(11) $\Delta P=\frac{P(1,1)_{\text{without info gain}}-P(1,1)_{\text{with info
gain}}}{P(1,1)_{\text{without info gain}}}*100$

Results. The performance of KF with TML is compared against the benchmark.
Traditional KF without TML ignores the correlation information where the
observation is simply the observed speed from the sensor on a given day, and
KF with TML is modified to include the mean observation of speed from other
TMCs and previous time-periods that are within the same cluster as the given
TMC and time-period. Figure 5 shows that the KF prediction with TML has fewer
errors compared to the KF prediction without TML. Figure 10 in Appendix B
shows the percentage change in uncertainty of predictions when TML is
considered. A significant reduction in uncertainty indicates more confidence
in the predictions with TML. In KF without TML, the update step uses
measurements with noise at time $t$ to get accurate predictions at time step
$t$. In KF with TML, we improve the prediction performance of traditional KF
by using the correlated observations from previous periods and it helps to
achieve the estimation of speed at $t$ on the previous time step, $t-1$. This
improved method is useful in getting more accurate predictions ahead of time.

Figure 5. Speed predictions with and without TML and corresponding
observations

##  6\. Temporal Multivariate Multimodal Learning

###  6.1. Learning storm atmosphere

With observations from sUAS, NOAA’s National Hurricane Center can better
measure critical variables and parameters in the boundary layers of hurricanes
(Cione et al., 2016). The accuracy of the data collected by the sUAS agreed
well with that of the manned measurement, with the sUAS sometimes capturing
more variability than the manned measurement (Cione et al., 2020). However,
the predefined navigation procedures do not necessarily consider how data
gathered from a flight path improves the hurricane forecasting. The criteria
for location selection was “difficult to observe in sufficient detail by
remote sensing”. We find the “optimal routes considering importance of
observations” gained from the data in a precise target location among high-
dimensional spaces. We analyze how online updating from sUAS collected
meteorological data would benefit hurricane intensity forecasting considering
the temporal variation in the uncertainty of hurricane prediction. The
temporal multivariate learning and in-situ data collections can significantly
improve understanding of hurricane movement, relevant dynamics, and track
prediction with the same effort and less risk.

TMML in hurricane forecasting. To generate the uncertainty distribution for
Hurricane Harvey, conventional in situ observations (e.g., Dropsondes) and
all-sky satellite radiance from GOES-16 were assimilated in a state-of-the-art
data assimilation system (ensemble KF) and built around the Advanced Weather
Research and Forecasting Model (WRF-ARW) and the Community Radiative Transfer
Model (CRTM) to provide hourly temporal resolution forecast of Hurricanes
(Minamide et al., 2020). Indirect learning will overcome possible limitations
in observing only one reliable sample variable in a cell out of many sensor
payloads (Poterjoy and Zhang, 2011). The temporal multivariate cluster groups
cells with a similar combination of distribution of multiple variables in each
cell: e.g., temperature (T), pressure (P), wind speed(W), relative humidity
(RH) (Figure 6). Once we have an observation of one variable, we update
posterior of other variables at the same location/other locations at the same
time/other time stages with the same cluster as the observed location.

Figure 6. The clustered temporal multivariate distribution of four variables
across all time stages (4 $\times$ 12 dimensional). For example, one
observation of temperature within cluster 1 at time stage 1 are used in
removing prediction uncertainties of humidity in cluster 2 at time stage 3.

###  6.2. Improving sUAS routing

Once the first sUAS is launched from a tube attached to a P-3 hurricane hunter
aircraft and controlled remotely from the airplane to be deployed to the lower
layer, the benefit of the online update of the information presents an
increase in accuracy relative to the initial hurricane prediction (Figure 7).
The overall improvement in prediction uncertainty after observations are made
along a path is computed as the difference between the sum of measurement
variances before and after observations.

Figure 7. Active sUAS In-situ Sensing Figure 8. Posterior update after agents
A & B depart from red and observe 64 cells out of 2500. As the probability
distribution for each variable for each cluster in the plot is narrower, it
represents less uncertainty. Variational Bayesian inference generates a
posterior belief given the prior belief of cell-type distributions. As
observations made along the route, particularly the cluster 2, 6, and 9
observations are made directly and indirectly would be further reduced
uncertainty and present narrower probability distribution. Figure 9. Safe and
efficient sUAS path solution by TMML-RRT* and MDM on centered horizontal
cross-sections for Hurricane Harvey at a 1.1-km level.

The utility map for the agent-guided observations of the hurricane environment
is constituted from the standard deviation and multivariate cluster entropy
between the four state variables: pressure, temperature, wind speed, and
relative humidity. These components are normalized to ensure a consistent
scale. First, the normalized standard deviation $\hat{\sigma}_{(x,j)}$ of the
measurement for these state variables $x\in X$ are linearly combined as
$\sum_{x\in X}\hat{\sigma}_{(x,j)}$ to quantify the overall standard deviation
$\sigma_{(j)}$ in each cell of the hurricane space. Second, the entropy
$H(x,n_{type})$ for each multivariate cluster distribution is computed as

(12)
$H(\textbf{x},n_{type})=-\mathop{\sum_{t=1}^{T}\sum_{k=1}^{K}}P\left(\textbf{x},n_{type}(k),t\right)\log_{2}P\left(\textbf{x},n_{type}(k),t\right)$

where $P\left(\textbf{x},n_{type}(k),t\right)$ is probability of category
$k\in K$ in cell $j$ of cluster $n_{type}$ at time $t$ and x is an $N$
dimensional vector of the state variables. For illustration, in a 12-hour
forecast window for Hurricane Harvey, cells with similar characteristics were
grouped into 11 clusters as different combinations with multiple variables,
where the optimal number of clusters and distribution parameters in each
cluster were estimated to maximize posterior probability (Figure 8).
Variational Bayesian inference approximates the true posterior based on the
$\mathbb{K}\mathbb{L}$ divergence minimizing the information lost with optimal
variational parameters. In Appendix A, the estimation of expected observation
and posterior estimate of the variances after observations by the sUAS is
presented.

Results. Although unobserved non-contiguous cells may not share any inherent
correlation with locally observed cells, classification errors are found be
correlated with certain features found in different locations. By clustering
cells with similar distributions (e.g., predicted wind speed) and correcting
those errors as more evidence becomes available from sUAS observations, the
reliability of the prediction was greatly improved (Figure 9). Anticipatory
sUAS routing lowered the overall energy usage, and maximized the reduction of
forecasting error by exploring and sampling unobserved cells along the path to
a target location. This new method significantly improved the combined quality
of observations (pressure, temperature, and wind speed) against the Minimum
Distance Method (MDM) in Hurricane Harvey (Darko et al., 2022a). In this
paper, the multistage online decisions are made by combining the benefits of
direct sensing through the sensitivity map and the additional benefits from
indirect learning through the correlation structure considering three
components: 1) temporal learning, 2) multimodal learning from one observed
geographical location to other similar locations, 3) deep multivariate
learning by grouping similar covariance of all four variables rather than
estimating correlation of each pair. Compared against unimodal univariate
learning, the combination of all three components presents a larger reduction
in prediction uncertainty.

Table 1. Improvement in Prediction Uncertainty MDM = 2.36% | Non temporal | Temp.
---|---|---
Unimodal | Multimodal
Correlation | Univariate | 2.66% | 4.34% | -
Multivariate | 3.28% | 6.25% | 11.26%
Deep multivariate | - | 7.37% | 13.45%

The average improvement in predicted multimodal measurement was significantly
higher when TMML were considered. MDM (Darko et al., 2022a) ignores those two
properties by averaging to reduce the dimension of the data, but these results
show that while the dimension may have increased, uncertainty reduction in
temporal correlation may have reduced the size of the deterministic problem.

##  7\. Conclusion

With the Temporal Multimodal Multivariate Learning (TMML), we have introduced
a new family of RL models that can indirectly learn and transfer information
from multiple modes of probability distributions of multiple data variables in
different time stages. These models can solve challenging tasks where the
uncertainty is revealed in a sequence by grouping samples within similar
distribution types and inferring the posterior based on expected observations.
The effectiveness of TMML has been demonstrated on real-world autonomous
navigation in urban transportation and Hurricane. TMML opens appealing
research opportunities in the study of information-theoretic decision making
that exhibit nontrivial indirect learning from spatiotemporal correlation.

##  8\. Acknowledgments

Part of the research work was carried out at the NASA Jet Propulsion
Laboratory (JPL), California Institute of Technology, under a contract with
the National Aeronautics and Space Administration (NASA) [80NM0018D0004].
Funding for this research was provided by NSF [1910397, 2106989], NASA JPL
[RSA 1625294, 1646362, 1659540], and NCDOT [TCE2020-01]. The authors thank JPL
Education Office for providing internships and faculty visiting opportunities
to conduct this research and Dr. Joe Cione for giving us valuable insight into
past, current, and future sUAS missions and key ﬁndings. The data used for
these analyses are available at: https://figshare.com/s/90f31f60e5821dae90bd

## References

  * (1)
  * Bello et al. (2017) Irwan Bello, Hieu Pham, Quoc V. Le, Mohammad Norouzi, and Samy Bengio. 2017.  Neural Combinatorial Optimization with Reinforcement Learning. In _5th International Conference on Learning Representations (ICLR 2017)_.
  * Ben-Akiva et al. (1999) M. Ben-Akiva, D. McFadden, and T. et al. Gärling. 1999\.  Extended Framework for Modeling Choice Behavior.  _Marketing Letters_ 10 (1999), 187–203.
  * Bengio et al. (2021) Yoshua Bengio, Andrea Lodi, and Antoine Prouvost. 2021\.  Machine learning for combinatorial optimization: A methodological tour d’horizon.  _European Journal of Operational Research_ 290, 2 (2021), 405–421.
  * Cione et al. (2020) Joseph J Cione, George H Bryan, Ronald Dobosy, Jun A Zhang, Gijs de Boer, Altug Aksoy, Joshua B Wadler, Evan A Kalina, Brittany A Dahl, Kelly Ryan, et al. 2020\.  Eye of the storm: observing hurricanes with a small unmanned aircraft system.  _Bulletin of the American Meteorological Society_ 101, 2 (2020).
  * Cione et al. (2016) Joseph J Cione, EA Kalina, EW Uhlhorn, AM Farber, and B Damiano. 2016.  Coyote unmanned aircraft system observations in Hurricane Edouard (2014).  _Earth and Space Science_ 3, 9 (2016), 370–380.
  * Claes et al. (2011) Rutger Claes, Tom Holvoet, and Danny Weyns. 2011\.  A Decentralized Approach for Anticipatory Vehicle Routing Using Delegate Multiagent Systems.  _IEEE Transactions on Intelligent Transportation Systems_ 12, 2 (2011), 364–373.
  * Dai et al. (2017) Hanjun Dai, Elias B. Khalil, Yuyu Zhang, Bistra Dilkina, and Le Song. 2017.  Learning Combinatorial Optimization Algorithms over Graphs. In _Proceedings of the 31st International Conference on Neural Information Processing Systems_ (Long Beach, California, USA) _(NIPS’17)_. Curran Associates Inc., Red Hook, NY, USA, 6351–6361.
  * Darko et al. (2021) Justice Darko, Larkin Folsom, Niharika Deshpande, and Hyoshin Park. 2021.  Distributed Constraint Optimization Problem for Coordinated Response of Unmanned Aerial Vehicles and Ground Vehicles. In _2021 55th Annual Conference on Information Sciences and Systems (CISS)_. IEEE, 1–6.
  * Darko et al. (2022a) Justice Darko, Larkin Folsom, Hyoshin Park, Masashi Minamide, Masahiro Ono, and Hui Su. 2022a.  A Sampling-Based Path Planning Algorithm for Improving Observations in Tropical Cyclones.  _Earth and Space Science_ 9, 1 (2022), e2020EA001498.  https://doi.org/10.1029/2020EA001498
  * Darko et al. (2022b) Justice Darko, Larkin Folsom, Nigel Pugh, Hyoshin Park, Khadijeh Shirzad, Justin Owens, and Andrew Miller. 2022b.  Adaptive personalized routing for vulnerable road users.  _IET Intelligent Transport Systems_ (2022).
  * Darko and Park (2022) Justice Darko and Hyoshin Park. 2022.  A Proactive Dynamic-Distributed Constraint Optimization Framework for Unmanned Aerial and Ground Vehicles in Traffic Incident Management. In _2021 6th International Conference on Intelligent Transportation Engineering (ICITE 2021)_ , Zhenyuan Zhang (Ed.). Springer Nature Singapore, Singapore, 708–721.
  * Das Gupta et al. (2015) Mithun Das Gupta, Srinidhi Srinivasa, J. Madhukara, and Meryl Antony. 2015.  KL Divergence based Agglomerative Clustering for Automated Vitiligo Grading. In _IEEE Computer Vision and Pattern Recognition_.
  * Di et al. (2016) Xuan Di, Henry X. Liu, and Xuegang Jeff Ban. 2016.  Second best toll pricing within the framework of bounded rationality.  _Transportation Research Part B: Methodological_ 83 (2016), 74–90.  https://doi.org/10.1016/j.trb.2015.11.002
  * Folsom (2021) Larkin Folsom. 2021\.  Information-Theoretic Dynamic Decision Making of Multiple Agents Under Extreme Uncertain Conditions, Dissertation, Computational Data Science and Engineering, North Carolina A and T State University.  https://www.proquest.com/openview/b141c77418b28498481750a282235211/1?pq-origsite=gscholar&cbl=18750&diss=y
  * Folsom et al. (2021) Larkin Folsom, Masahiro Ono, Kyohei Otsu, and Hyoshin Park. 2021\.  Scalable Information-Theoretic Path Planning for a Rover-Helicopter Team in Uncertain Environments, 18(2): 1-16.  _International Journal of Advanced Robotic Systems_ (2021).
  * Folsom et al. ([n. d.) ]folsomdynamic Larkin Folsom, Hyoshin Park, and Venktesh Pandey. [n. d.].  Dynamic Routing of Heterogeneous Users After Traffic Disruptions under a Mixed Information Framework.  _Frontiers in Future Transportation_ ([n. d.]), 11.
  * Gauci et al. (2019) Jason Gauci, Edoardo Conti, Yitao Liang, Kittipat Virochsiri, Yuchen He, Zachary Kaden, Vivek Narayanan, Xiaohui Ye, Zhengxing Chen, and Scott Fujimoto. 2019\.  Horizon: Facebook’s Open Source Applied Reinforcement Learning Platform. In _Workshop in the 36 th International Conference on Machine Learning (ICML), Long Beach, California, USA_.
  * Gendreau et al. (2015) Michel Gendreau, Gianpaolo Ghiani, and Emanuela Guerriero. 2015\.  Time-dependent routing problems: A review.  _Computers & operations research_ 64 (2015), 189–197.
  * Guo et al. (2010) Feng Guo, Hesham Rakha, and Sangjun Park. 2010.  Multistate Model for Travel Time Reliability.  _Transportation Research Record_ 2188, 1 (2010), 46–54.
  * Guo (2013) Xiaolei Guo. 2013\.  Toll sequence operation to realize target flow pattern under bounded rationality.  _Transportation Research Part B: Methodological_ 56 (2013), 203–216.  http://www.sciencedirect.com/science/article/pii/S0191261513001422
  * Han et al. (2015) Ke Han, Wai Yuen Szeto, and Terry L. Friesz. 2015.  Formulation, existence, and computation of boundedly rational dynamic user equilibrium with fixed or endogenous user tolerance.  _Transportation Research Part B: Methodological_ 79 (2015), 16–49.  https://doi.org/10.1016/j.trb.2015.05.002
  * Han and Timmermans (2006) Qi Han and Harry Timmermans. 2006.  Interactive Learning in Transportation Networks with Uncertainty, Bounded Rationality, and Strategic Choice Behavior: Quantal Response Model.  _Transportation Research Record: Journal of the Transportation Research Board_ 1964 (2006), 27–34.
  * Kullback and Leibler (1951) Solomon Kullback and Richard A. Leibler. 1951.  On Information and Sufficiency.  _The Annals of Mathematical Statistics_ 22, 1 (1951), 79–86.
  * Macfarlane (2019) Jane Macfarlane. 2019\.  When Apps Rule the Road: Your Navigation App Is Making Traffic Unmanageable.  _IEEE Spectrum_ (2019).
  * Mahajan et al. (2019) Niharika Mahajan, Andreas Hegyi, Serge P. Hoogendoorn, and Bart van Arem. 2019.  Design analysis of a decentralized equilibrium-routing strategy for intelligent vehicles.  _Transportation Research Part C: Emerging Technologies_ 103 (2019), 308–327.  https://doi.org/10.1016/j.trc.2019.03.028
  * Minamide et al. (2020) Masashi Minamide, Fuqing Zhang, and Eugene E. Clothiaux. 2020\.  Nonlinear Forecast Error Growth of Rapidly Intensifying Hurricane Harvey (2017) Examined through Convection-Permitting Ensemble Assimilation of GOES-16 All-Sky Radiances.  _Journal of the Atmospheric Sciences_ 77, 12 (2020), 4277 – 4296.
  * Ono et al. (2020) Masahiro Ono, Brandon Rothrock, Kyohei Otsu, Shoya Higa, Yumi Iwashita, Annie Didier, Tanvir Islam, Christopher Laporte, Vivian Sun, Kathryn Stack, Jacek Sawoniewicz, Shreyansh Daftry, Virisha Timmaraju, Sami Sahnoune, Chris A. Mattmann, Olivier Lamarre, Sourish Ghosh, Dicong Qiu, Shunichiro Nomura, Hemanth Sarabu, Gabrielle Hedrick, Larkin Folsom, Sean Suehr, and Hyoshin Park. 2020\.  MAARS: Machine learning-based Analytics for Rover Systems. In _IEEE Aerospace Conference_.
  * Park et al. (2010) Byung-Jung Park, Yunlong Zhang, and Dominique Lord. 2010\.  Bayesian mixture modeling approach to account for heterogeneity in speed data.  _Transportation Research Part B: Methodological_ 44, 5 (2010), 662–673.
  * Park and Haghani (2015) Hyoshin Park and Ali Haghani. 2015.  Optimal Number and Location of Bluetooth Sensors Considering Stochastic Travel Time Prediction.  _Transportation Research Part C: Emerging Technologies_ 55 (2015), 203–216.
  * Park and Haghani (2016a) Hyoshin Park and Ali Haghani. 2016a.  Real-time prediction of secondary incident occurrences using vehicle probe data.  _Transportation Research Part C: Emerging Technologies_ 70 (2016), 69 – 85.
  * Park and Haghani (2016b) H. Park and A. Haghani. 2016b.  Stochastic Capacity Adjustment Considering Secondary Incidents.  _IEEE Transactions on Intelligent Transportation Systems_ 17, 10 (2016), 2843–2853.
  * Park et al. (2018) Hyoshin Park, Ali Haghani, Siby Samuel, and Michael A. Knodler. 2018.  Real-time prediction and avoidance of secondary crashes under unexpected traffic congestion.  _Accident Analysis & Prevention_ 112 (2018), 39–49.
  * Park et al. (2016) Hyoshin Park, Ali Haghani, and Xin Zhang. 2016\.  Interpretation of Bayesian neural networks for predicting the duration of detected incidents.  _Journal of Intelligent Transportation Systems_ 20, 4 (2016), 385–400.
  * Poterjoy and Zhang (2011) Jonathan Poterjoy and Fuqing Zhang. 2011.  Dynamics and structure of forecast error covariance in the core of a developing hurricane.  _Journal of the atmospheric sciences_ 68, 8 (2011), 1586–1606.
  * Shannon and Weaver (1949) Claude Elwood Shannon and Warren Weaver. 1949.  _The Mathematical Theory of Communication_.  Urbana, The University of Illinois Press. 1–117 pages.
  * Sutoh et al. (2015) M. Sutoh, M. Otsuki, S. Wakabayashi, T. Hoshino, and T. Hashimoto. 2015.  The Right Path: Comprehensive Path Planning for Lunar Exploration Rovers.  _IEEE Robotics Automation Magazine_ 22, 1 (March 2015), 22–33.
  * Tompkins et al. (2006) Paul Tompkins, Anthony Stentz, and David Wettergreen. 2006\.  Mission-level path planning and re-planning for rover exploration.  _Robotics and Autonomous Systems_ 54 (2006), 174–183.
  * Wallace and Dowe (1997) Chris S. Wallace and David L. Dowe. 1997.  MML mixture modelling of multi-state, Poisson, von Mises circular and Gaussian distributions.  (1997).
  * Wettergreen et al. (2005) David Wettergreen, Paul Tompkins, Chris Urmson, Michael Wagner, and William Whittaker. 2005.  Sun-Synchronous Robotic Exploration: Technical Description and Field Experimentation.  _The International Journal of Robotics Research_ 24, 1 (2005), 3–30.
  * Zheng and Zuylen (2010) Fangfang Zheng and Henk Van Zuylen. 2010.  Uncertainty and Predictability of Urban Link Travel Time: Delay Distribution–Based Analysis.  _Transportation Research Record_ 2192, 1 (2010), 136–146.

##  Appendix A Posterior approximation

We combine observations and prior data with the importance of information. The
sequence of observations are assimilated with the prior predicted measurements
to provide the best estimate (posterior) of the measurements. The multivariate
measurements are represented as grid-point values with background prior
information at location $i$:

(13)
$M_{\textnormal{Pred}(\textbf{x},i)}+\sigma_{\textnormal{Pred}(\textbf{x},i)}^{2},$

and after observation at location $i$:

(14)
$M_{\textnormal{Obs}(\textbf{x},i)}+\sigma_{\textnormal{Obs}(\textbf{x},i)}^{2},$

where $M_{\textnormal{Pred}(\textbf{x},i)}$ is the mean predicted measurement
at location $i$, $\sigma_{\textnormal{Pred}(\textbf{x},i)}^{2}$ is the
variance of predicted measurement at location $i$,
$M_{\textnormal{Obs}(\textbf{x},i)}$ is the sUAS observation at location $i$,
$\sigma_{\textnormal{Obs}(\textbf{x},i)}^{2}$ variance of sUAS observation at
location $i$. The variance $\sigma_{\textnormal{Obs}(\textbf{x},i)}^{2}$
represents the imperfections of observations made by sUAS sensors. Our model
assumes that this variance is known apriori based on the type of sensors used.
The best estimate of the measurement of variable $x$ at location $i$ is
written as:

(15) $M_{\textbf{best
estimate}(\textbf{x},i)}=(1-\beta)M_{\textnormal{Pred}(x,i)}+\beta
M_{\textnormal{Obs}(\textbf{x},i)}.$

$\beta$ is the weight between the predicted measurements and observation. The
best estimate of weight considers the variance of the predicted measurement
and observation, written as:

(16)
$\beta=\frac{\sigma_{\textnormal{Pred}(\textbf{x},i)}^{2}}{\sigma_{\textnormal{Pred}(\textbf{x},i)}^{2}+\sigma_{\textnormal{Obs}(\textbf{x},i)}^{2}}.$

The variance of the best estimate of measurement for variable x at location
$i$ is less than that of either the prediction or the observation:

(17) $\sigma_{\textbf{best
estimate}(\textbf{x},i)}^{2}=(1-\beta)\sigma_{\textnormal{Pred}(\textbf{x},i)}^{2}.$

To account for the effect of an influence region around each observation
point, we introduce a weighting function $\boldsymbol{\omega(i,j)}$, to update
the best estimates of the variance at each grid locations $j$ in the vicinity
of observation point $i$ written as:

(18)
$\boldsymbol{\omega(i,j)}=\max\left(0,\frac{R^{2}-d_{i,j}^{2}}{R^{2}+d_{i,j}^{2}}\right)$

where $d_{i,j}$ is a measure of the distance between points $i$ and $j$. The
weighting function $\boldsymbol{\omega(i,j)}$ equals to one if the grid point
$j$ is collocated with observation $i$. It is a deceasing function of distance
which is zero if $d_{i,j}\geq R$. $R$ (“the influence region or radius”) is a
user defined constant beyond which the observations have no weight. The
modified best estimate of the variance at each grid point location $j$ can now
be written as:

(19) $\sigma_{\textbf{best
estimate}(\textbf{x},j)}^{2}=(1-\beta*\boldsymbol{\omega(i,j)})\sigma_{\textnormal{Pred}(\textbf{x},j)}^{2}.$

Sequential learning updates a cells entropy and in extension the utilities
each time an observation is made in other cells belonging to the same cluster.
We introduce a weight $\omega(x,o_{n_{type}})$ (decreasing function of sample
size $o_{n_{type}}$) that updates the entropy $H(x,n_{type})$ as observation
of cells belonging to the same cluster type are made. Since there is high
confidence in the measurement in cells belonging to clusters with low entropy,
we exploit those low entropy cluster types through a few sampling of their
member cells. A posterior update will be applied to the variance of all
similar type cells in such scenarios.

Conversely, there is low confidence in the measurement in cells belonging to
clusters with high entropy. Therefore, we explore the high entropy cluster
types through a large sampling of their member cells. With enough sampling of
member cells, we can reduce the entropy to a set threshold and apply a
posterior update to measurements in its member cells. The sequential learning
in environments learn the optimal sample size and update as more observations
are obtained. Online recourse in Algorithm 2 shows the sequential update of
the TMML-RRT* (Algorithm 1).

Algorithm 2 :OnlineRecourse

X $\leftarrow$ $n_{s}$

for t=1 to i=$\tau$ do

if X = $n_{g}$ then

break;

else

$(X_{new},U_{new})\leftarrow$ max(X, $R_{X,X_{successor}}$)

if $U_{new}$ ¿ $U_{X_{successor}}$ then

$X_{successor}$ $\leftarrow$ $X_{new}$

Rewire(X, $X_{successor}$, $X_{successor2}$)

end if

X $\leftarrow$ $X_{successor}$

end if

end for

##  Appendix B KF-TML Prediction

The main goal of the KF-TML is to use the information gain from spatiotemporal
correlation between TMCs to reduce uncertainty in KF speed prediction. Figure
10 shows the significant percent reduction in uncertainty of predictions when
the KF-TML is employed. When speed observations with TML are close to historic
observations, the reduction in uncertainty is higher.

Figure 10. Percent change in uncertainty of KF prediction when TML is
considered

##  Appendix C TMML in sUAS Routing

Using k-means clustering, the map is divided into regions of similar cell
types. Clusters are formed based on the entropy and the expected value of each
cell. The optimal number of clusters for the dataset was calculated using a
Gap function, demonstrated in Figure 11. Because of differences in scale,
k-means clustering performs better when entropy is expressed as a percentage
rather than a decimal value.

Figure 11. Based on sum of squares of hurricane prediction output uncertainty,
optimal cluster of 12 is used. We cluster the four factors across 12 time
stages using 12 clusters to create a heap map to highlight the different
clusters.

Figure 12 shows principle components of those 12 clusters for each variables
across 12 hours in hurricane case study. As long as any observation is within
the same cluster, based on the correlation measure, prediction uncertainty of
multiple variables are updated.

Figure 12. The result of multimodal multivariate clustering and temporal
multimodal multivariate is described by transforming four variables and time
dimensions to three principle components in a three-dimensional graph with
cumulative variation proportion of 90$\%$.


* Dang  _et al._ [2023] H. Dang, D. Qi, M. Zhao, C. Fan, and C. S. Lu, Thermal-induced interfacial behavior of a thin one-dimensional hexagonal quasicrystal film, Appl. Math. Mech. -Engl. Ed. 44, 841 (2023).
  * Lubin  _et al._ [2012] S. M. Lubin, W. Zhou, A. J. Hryn, M. D. Huntington, and T. W. Odom, High-rotational symmetry lattices fabricated by moiré nanolithography, Nano Lett. 12, 4948 (2012).
  * Ahn  _et al._ [2018] S. J. Ahn, P. Moon, T.-H. Kim, H.-W. Kim, H.-C. Shin, E. H. Kim, H. W. Cha, S.-J. Kahng, P. Kim, M. Koshino, Y.-W. Son, C.-W. Yang, and J. R. Ahn, Dirac electrons in a dodecagonal graphene quasicrystal, Science 361, 782 (2018).
  * Mahmood  _et al._ [2021] R. Mahmood, A. V. Ramirez, and A. C. Hillier, Creating two-dimensional quasicrystal, supercell, and moiré lattices with laser interference lithography: Implications for photonic bandgap materials, ACS Appl. Nano Mater. 4, 8851 (2021).
  * Uri  _et al._ [2023] A. Uri, S. C. de la Barrera, M. T. Randeria, D. Rodan-Legrain, T. Devakul, P. J. D. Crowley, N. Paul, K. Watanabe, T. Taniguchi, R. Lifshitz, L. Fu, R. C. Ashoori, and P. Jarillo-Herrero, Superconductivity and strong interactions in a tunable moiré quasicrystal, Nature 620, 762 (2023).
  * Merlin  _et al._ [1985] R. Merlin, K. Bajema, R. Clarke, F. Y. Juang, and P. K. Bhattacharya, Quasiperiodic gaas-alas heterostructures, Phys. Rev. Lett. 55, 1768 (1985).
  * Todd  _et al._ [1986] J. Todd, R. Merlin, R. Clarke, K. M. Mohanty, and J. D. Axe, Synchrotron x-ray study of a fibonacci superlattice, Phys. Rev. Lett. 57, 1157 (1986).
  * Bajema and Merlin [1987] K. Bajema and R. Merlin, Raman scattering by acoustic phonons in fibonacci gaas-aias superlattices, Phys. Rev. B 36, 4555 (1987).
  * Cohn  _et al._ [1988] J. L. Cohn, J. J. Lin, F. J. Lamelas, H. He, R. Clarke, and C. Uher, Upper critical fields of periodic and quasiperiodic nb-ta superlattices, Phys. Rev. B 38, 2326 (1988).
  * Zhu  _et al._ [1997] S.-n. Zhu, Y.-y. Zhu, Y.-q. Qin, H.-f. Wang, C.-z. Ge, and N.-b. Ming, Experimental realization of second harmonic generation in a fibonacci optical superlattice of lita${\mathrm{o}}_{3}$, Phys. Rev. Lett. 78, 2752 (1997).
  * Dal Negro  _et al._ [2003] L. Dal Negro, C. J. Oton, Z. Gaburro, L. Pavesi, P. Johnson, A. Lagendijk, R. Righini, M. Colocci, and D. S. Wiersma, Light transport through the band-edge states of fibonacci quasicrystals, Phys. Rev. Lett. 90, 055501 (2003).
  * Yuan  _et al._ [2018] L. Yuan, Q. Lin, M. Xiao, and S. Fan, Synthetic dimension in photonics, Optica 5, 1396 (2018).
  * Steurer and Sutter-Widmer [2007] W. Steurer and D. Sutter-Widmer, Photonic and phononic quasicrystals, J. Phys. D: Appl. Phys. 40, R229 (2007).
  * Tanese  _et al._ [2014] D. Tanese, E. Gurevich, F. Baboux, T. Jacqmin, A. Lemaître, E. Galopin, I. Sagnes, A. Amo, J. Bloch, and E. Akkermans, Fractal energy spectrum of a polariton gas in a fibonacci quasiperiodic potential, Phys. Rev. Lett. 112, 146404 (2014).
  * Baboux  _et al._ [2017] F. Baboux, E. Levy, A. Lemaître, C. Gómez, E. Galopin, L. Le Gratiet, I. Sagnes, A. Amo, J. Bloch, and E. Akkermans, Measuring topological invariants from generalized edge states in polaritonic quasicrystals, Phys. Rev. B 95, 161114(R) (2017).
  * Goblot  _et al._ [2020] V. Goblot, A. Štrkalj, N. Pernet, J. L. Lado, C. Dorow, A. Lemaître, L. Le Gratiet, A. Harouri, I. Sagnes, S. Ravets, A. Amo, J. Bloch, and O. Zilberberg, Emergence of criticality through a cascade of delocalization transitions in quasiperiodic chains, Nat. Phys. 16, 832 (2020).
  * Singh  _et al._ [2015] K. Singh, K. Saha, S. A. Parameswaran, and D. M. Weld, Fibonacci optical lattices for tunable quantum quasicrystals, Phys. Rev. A 92, 063426 (2015).
  * Reisner  _et al._ [2023] M. Reisner, Y. Tahmi, F. Piéchon, U. Kuhl, and F. Mortessagne, Experimental observation of multifractality in fibonacci chains, Phys. Rev. B 108, 064210 (2023).
  * Lisiecki  _et al._ [2019] F. Lisiecki, J. Rychły, P. Kuświk, H. Głowiński, J. W. Kłos, F. Groß, N. Träger, I. Bykova, M. Weigand, M. Zelent, E. J. Goering, G. Schütz, M. Krawczyk, F. Stobiecki, J. Dubowik, and J. Gräfe, Magnons in a quasicrystal: Propagation, extinction, and localization of spin waves in fibonacci structures, Phys. Rev. Appl. 11, 054061 (2019).
  * Josephson [1962] B. Josephson, Possible new effects in superconductive tunnelling, Phys. Lett. 1, 251 (1962).
  * Josephson [1974] B. D. Josephson, The discovery of tunnelling supercurrents, Rev. Mod. Phys. 46, 251 (1974).
  * Likharev [1979] K. K. Likharev, Superconducting weak links, Rev. Mod. Phys. 51, 101 (1979).
  * Golubov  _et al._ [2004] A. A. Golubov, M. Y. Kupriyanov, and E. Il’ichev, The current-phase relation in josephson junctions, Rev. Mod. Phys. 76, 411 (2004).
  * Ishii [1970] C. Ishii, Josephson Currents through Junctions with Normal Metal Barriers, Prog. Theor. Phys. 44, 1525 (1970).
  * Affleck  _et al._ [2000] I. Affleck, J.-S. Caux, and A. M. Zagoskin, Andreev scattering and josephson current in a one-dimensional electron liquid, Phys. Rev. B 62, 1433 (2000).
  * Nikolić  _et al._ [2001] B. K. Nikolić, J. K. Freericks, and P. Miller, Intrinsic reduction of josephson critical current in short ballistic sns weak links, Phys. Rev. B 64, 212507 (2001).
  * Sandberg [2022] A. Sandberg,  _Quasicrystal nanowires in SNS-junctions_ , Master’s thesis, Uppsala University (2022).
  * Su  _et al._ [1979] W. P. Su, J. R. Schrieffer, and A. J. Heeger, Solitons in polyacetylene, Phys. Rev. Lett. 42, 1698 (1979).
  * Harper [1955] P. G. Harper, Single band motion of conduction electrons in a uniform magnetic field, Proc. Phys. Soc. London A 68, 874 (1955).
  * Aubry and André [1980] S. Aubry and G. André, Analyticity breaking and anderson localization in incommensurate lattices, Ann. Israel Phys. Soc 3, 18 (1980).
  * Gordon  _et al._ [1997] A. Y. Gordon, S. Jitomirskaya, Y. Last, and B. Simon, Duality and singular continuous spectrum in the almost Mathieu equation, Acta Math. 178, 169 (1997).
  * Holmvall  _et al._ [2023] P. Holmvall, N. Wall Wennerdal, M. Håkansson, P. Stadler, O. Shevtsov, T. Löfwander, and M. Fogelström, SuperConga: An open-source framework for mesoscopic superconductivity, Appl. Phys. Rev. 10, 011317 (2023).
  * Zhu [2016] J.-X. Zhu,  _Bogoliubov-de Gennes method and its applications_, Lecture Notes in Physics, Vol. 924 (Springer International Publishing, Switzerland, 2016).
  * Shumeiko  _et al._ [1997] V. S. Shumeiko, E. N. Bratus, and G. Wendin, Scattering theory of superconductive tunneling in quantum junctions, Low Temp. Phys. 23, 181 (1997).
  * Bena [2012] C. Bena, Metamorphosis and taxonomy of andreev bound states, Eur. Phys. J. B 85, 196 (2012).
  * Jackiw and Rebbi [1976] R. Jackiw and C. Rebbi, Solitons with fermion number ½, Phys. Rev. D 13, 3398 (1976).
  * Furusaki  _et al._ [1992] A. Furusaki, H. Takayanagi, and M. Tsukada, Josephson effect of the superconducting quantum point contact, Phys. Rev. B 45, 10563 (1992).
  * Beenakker [1992] C. W. J. Beenakker, Three “universal” mesoscopic josephson effects, in  _Transport Phenomena in Mesoscopic Systems_ , edited by H. Fukuyama and T. Ando (Springer Berlin Heidelberg, Berlin, Heidelberg, 1992) pp. 235–253.
  * Kouwenhoven  _et al._ [1990] L. P. Kouwenhoven, F. W. J. Hekking, B. J. van Wees, C. J. P. M. Harmans, C. E. Timmering, and C. T. Foxon, Transport through a finite one-dimensional crystal, Phys. Rev. Lett. 65, 361 (1990).
  * Kuzmanovski  _et al._ [2023] D. Kuzmanovski, R. S. Souto, P. J. Wong, and A. V. Balatsky, Mobile topological su-schrieffer-heeger soliton in a josephson metamaterial (2023), arXiv:2312.03456 [cond-mat.supr-con] .
  * Amundsen  _et al._ [2024] M. Amundsen, V. Juričić, and J. A. Ouassou, Josephson effect in a fractal geometry (2024), arXiv:2404.01373 [cond-mat.supr-con] .