# Integration of Digital Twin and Federated Learning for Securing Vehicular
Internet of Things

Deepti Gupta  Texas A&M University - Central Texas1001 Leadership Pl,Texas,
USA76549<EMAIL_ADDRESS>,  Shafika Showkat Moni  Embry-Riddle
Aeronautical UniversityP.O. Box 1212Daytona BeachFlorida, USA43017-6221
<EMAIL_ADDRESS>and  Ali Saman Tosun  University of North Carolina at
PembrokeNorth Carolina, USA<EMAIL_ADDRESS>
(2023)

###### Abstract.

In the present era of advanced technology, the Internet of Things (IoT) plays
a crucial role in enabling smart connected environments. This includes various
domains such as smart homes, smart healthcare, smart cities, smart vehicles,
and many others. The IoT facilitates the integration and interconnection of
devices, enabling them to communicate, share data, and work together to create
intelligent and efficient systems. With ubiquitous smart connected devices and
systems, a large amount of data associated with them is at a prime risk from
malicious entities (e.g., users, devices, applications) in these systems.
Innovative technologies, including cloud computing, Machine Learning (ML), and
data analytics, support the development of anomaly detection models for the
Vehicular Internet of Things (V-IoT), which encompasses collaborative
automatic driving and enhanced transportation systems. However, traditional
centralized anomaly detection models fail to provide better services for
connected vehicles due to issues such as high latency, privacy leakage,
performance overhead, and model drift.

Recently, Federated Learning (FL) has gained significant recognition for its
ability to address data privacy concerns in the IoT domain. In the context of
V-IoT, which involves autonomous vehicles and intelligent transportation
systems with connected vehicles communicating with various sensors and
devices, FL is used to develop an anomaly detection model. Current technology,
the Digital Twin (DT), proves beneficial in addressing uncertain crises and
data security issues by creating a virtual replica that simulates various
factors, including traffic trajectories, city policies, and vehicle
utilization. This enables the system to facilitate efficient and inclusive
decision-making. However, the effectiveness of a V-IoT DT system heavily
relies on the collection of long-term and high-quality data to make
appropriate decisions. Consequently, its advantages may be limited when
confronted with urgent crises like the COVID-19 pandemic.

This paper introduces a Hierarchical Federated Learning (HFL) based anomaly
detection model for V-IoT, aiming to enhance the accuracy of the model. Our
proposed model integrates both DT and HFL approaches to create a comprehensive
system for detecting malicious activities using an anomaly detection model.
Additionally, real-world V-IoT use case scenarios are presented to demonstrate
the application of the proposed model.

Vehicular Internet of Things, Hierarchical Federated Learning, Digital Twin,
Anomaly Detection Model

††copyright: rightsretained††journalyear: 2023††copyright:
acmlicensed††conference: International Conference on Research in Adaptive and
Convergent Systems; August 6–10, 2023; Gdansk, Poland††booktitle:
International Conference on Research in Adaptive and Convergent Systems (RACS
’23), August 6–10, 2023, Gdansk, Poland††price: 15.00††doi:
10.1145/3599957.3606250††isbn: 979-8-4007-0228-0/23/08††ccs: Computer systems
organization Embedded systems††ccs: Computer systems organization
Redundancy††ccs: Computer systems organization Robotics††ccs: Networks Network
reliability

##  1\. Introduction

The proliferation of Internet of Things (IoT) technology has become
increasingly prevalent in our daily lives, primarily driven by the
advancements in low latency and high-speed cellular networks. This
technological progress has facilitated seamless connectivity and communication
between various smart devices, enabling them to exchange data and interact in
real-time. As a result, IoT has found widespread applications in diverse
domains such as smart homes, healthcare, transportation, industrial
automation, and more, enhancing efficiency, convenience, and automation in our
day-to-day activities. As a result of such enormous growth, the number of IoT
and connected devices is expected to increase to 60 billion by 2025 (Moni and
Manivannan, 2021). A significant part of this rapid increase is likely linked
to the Vehicular Internet of Things (V-IoT). V-IoT comprises connected
vehicles, Road Side Units (RSUs), sensors, base stations, edge servers, cloud
servers, and other devices capable of data sharing and communication with
humans. This information generated through V-IoT will play a vital role in
traffic management, traffic safety, infotainment services, smart city, and
Intelligent Transportation Systems (ITSs), as shown in Figure1.

Federated Learning (FL) is an innovative approach to Machine Learning (ML)
that emphasizes collaborative learning and privacy preservation by avoiding
the need to share raw data with a centralized server. In FL, multiple learning
agents can efficiently and securely collaborate their computing capabilities
to achieve an improved quality of services. The deployment of FL for V-IoT
enables vehicles, Roadside Units (RSUs), base stations, and other connected
devices to enhance learning efficiency in various aspects such as intelligent
environment sensing, intelligent networking, cooperative autonomous driving,
and intelligent processing of large volumes of vehicular data. By leveraging
FL, V-IoT can benefit from collective intelligence while preserving data
privacy and promoting efficient knowledge sharing among the connected
entities.

Figure 1. Applications of Smart Connected Automotive System.

Recent research studies (Du et al., 2020; Li et al., 2023; Pokhrel and Choi,
2020; Chai et al., 2020; Lu et al., 2020) have made significant contributions
in securing V-IoT environment from anomalies and malicious entities. These
studies have proposed and developed various methods and techniques, leveraging
the power of FL, to enhance the security and privacy of the V-IoT systems.
Additionally, energy-efficient models have been designed, utilizing the FL
approach, to optimize energy consumption in V-IoT deployments. The collective
findings of these research works contribute to the ongoing efforts in
establishing robust security mechanisms and energy efficiency strategies in
the V-IoT domain. Despite advances in FL based models for V-IoT system, there
are still growing concerns about safety, security and privacy of users.
Therefore, this system requires a comprehensive and robust anomaly detection
approach to detect anomalous behavior of various entities effectively.

A Digital Twin (DT) enables connectivity, interaction, and synchronization
between the physical entity and its virtual representation in real time. The
DT is considered to be one of the most promising technology due to its
advanced capabilities, intelligent services, and bridging the gap between the
digital model and its physical counterpart. A digital model of a vehicle can
be placed at the edge computing node that can serve as an edge middleware in
the V-IoT. With the help of this cloud-edge computing paradigm, large-scale
data analysis, storage, and modeling are made possible. In addition to that, a
huge volume of geographically dispersed information shared by many DTs can be
aggregated to derive synthesized information with effectiveness. By creating a
virtual replica that mimics real-world conditions, the DT of V-IoT enables
simulations and analysis of crucial factors like traffic trajectories, city
policies, and vehicle utilization. This virtual representation enhances
decision-making processes and assists in developing effective strategies for
managing crises while ensuring the security of data within the V-IoT system.

In this paper, we integrate both DT and FL technologies into the V-IoT
framework. Our objective is to harness the advanced computing capabilities
offered by DT and leverage the collaborative learning potential of FL to
address the security and privacy challenges present in V-IoT systems. By
combining these technologies, we aim to enhance the overall performance,
efficiency, and privacy preservation in the V-IoT environments. The
utilization of DT technology can significantly enhance the efficiency of the
anomaly detection model by incorporating data from various sources such as
smart sensors, traffic light data, weather statistics, vehicle data, and city
policies. The integration of data from multiple vendors enables the provision
of more accurate and expedited service delivery for the V-IoT system. In our
proposed model, DT facilitates data synchronization and weight aggregation in
a synchronous manner, reducing the wait time during FL process. This
streamlined approach allows participants to efficiently share their data with
the model, ultimately improving the overall performance and effectiveness of
the FL-based anomaly detection system.

In our research, we have implemented a Hierarchical Federated Learning (HFL)
approach to develop an anomaly detection model. For example, in region-1,
where vendor-1 operates with two smart vehicles, these vehicles collaborate to
build their local anomaly detection model. Similarly, in region-2, where
Vendor-2 operates with five smart vehicles, those vehicles collaborate to
construct their own local anomaly detection model. This hierarchical approach
allows smart vehicles from multiple regions to collaborate at different
levels, enabling the development of robust anomaly detection models for the
V-IoT within the same smart city.

By leveraging this HFL approach, our research aims to identify anomalies and
enhance the security of the V-IoT systems. Through collaborative learning and
data aggregation at different levels, we can improve the accuracy and
reliability of the anomaly detection models, ultimately ensuring the integrity
and safety of the V-IoT ecosystem. The main contribution of this paper are as
follows-

  * •

In our research, we have identified a research gap in the development of FL
based anomaly detection models specifically tailored for the V-IoT domain.

  * •

We present the concept of a Hierarchical Federated Learning (HFL) based
anomaly detection model.

  * •

We propose a system model where we integrate both the emerging DT and FL
technologies. This approach provides a powerful framework for enhanced
collaboration, learning, and decision-making in the V-IoT domain, ultimately
leading to improved performance, efficiency, and security.

  * •

We also present a use case scenario to demonstrate the feasibility of our
proposed model.

The remainder of this paper is organized as follows. Section 2 presents the
literature review on DT and FL technologies in vehicular internet domains. We
also discuss about the V-IoT, DT, and FL in this section. The concept of a
Hierarchical Federated Learning (HFL) based anomaly detection model is
presented in Section 3. Section 4 presents the proposed system model for
identifying anomalies and securing V-IoT. We discuss a use case scenario to
demonstrate the practical application of our proposed model in Section 5.
Conclusion and future work are discussed in Section 6.

##  2\. Related Work and Background

This section provides an in-depth discussion of fundamental concepts and
background information that are essential to understand the research
contributions. It covers key topics such as the concept of the V-IoT, security
and privacy concerns, as well as FL models and DT.

Recently, there has been a growing interest in various technologies such as
cloud computing, edge computing, ML, FL, and DT in both academic and
industrial sectors. These technologies are seen as promising solutions for
enabling smart cities and ITSs. Lu et al. (Lu et al., 2020) proposed an
asynchronous federated framework to implement secure and effective data
sharing in the Internet of Vehicles (IoV). In this approach, each vehicle
serves as FL client and shares data with an aggregation server at macro BS
(MBS). Vehicles can request a variety of services, including traffic
prediction and path selection to the MBS. The MBS develops a shared global
model based on accumulated vehicular datasets. Next, the MBS transforms the
sharing process into a computing task and resolves the sharing request of
vehicles by using an actor-critic reinforcement learning framework.

Chai et al. (Chai et al., 2020) proposed a hierarchical blockchain-enabled FL
scheme for IoV. They presented a feasibility analysis of adapting the
hierarchical model to manage large-scale vehicles. In this scheme, each
vehicle serves as an FL client and uses its hardware resources to implement
local learning. Road side units (RSUs) are responsible for collecting
transactions from vehicles within their communication region in a blockchain
framework. Each RSU compute the FL model and append into the blockchain
framework to ensure security. The blockchain framework is shared to all RSUs
and vehicles in IoV. Shrivastava et al. (Shrivastava and Dwivedi, 2021)
presented a brief survey on Security in V-Iot using blockchain. In addition,
several security models for protecting IoT devices in other domains are
discussed in (Gupta et al., 2020a, b, 2021a, 2021b; Aslan et al., 2021; Ozkan-
Okay et al., 2021; Gupta et al., 2021c, d; Akbarfam et al., 2023; Moni and
Gupta, 2022; Pei et al., 2022).

###  2.1. Vehicular Internet of Things

V-IoT can be described as a platform that enables the exchange of information
between vehicles and their surroundings. This communication is facilitated
through various channels, including vehicle-to-vehicle (V2V), vehicle-to-
infrastructure (V2I), and vehicle-to-everything (V2X) communication. These
interactions allow vehicles to connect with other vehicles, infrastructure
elements, and various entities in their environment, creating a networked
ecosystem that enhances safety, efficiency, and overall driving experience.
Yang et al. (Yang et al., 2014) put forward an abstract network model for the
IoV. Their research focuses on discussing the necessary technologies to
establish the IoV framework. They explore various applications that can be
built upon existing technologies, highlighting the potential of IoV in
different domains. V-IoT enables drivers, pedestrians, and other vehicles to
utilize the data produced by vehicular ad hoc networks (VANETs) with the aid
of roadside infrastructure (Moni and Manivannan, 2020, 2022). V-IoT integrates
the IoT technology with ITSs to improve transportation efficiency and
security. It is anticipated that the V-IoT will play a vital role in enabling
connected, shared, autonomous, and electric future mobility. This article (Ji
et al., 2020) conducted an extensive literature review focusing on the
fundamental aspects of the IoV. They covered essential information related to
IoV, including basic VANET technology, different network architectures
employed in IoV systems, and typical applications of IoV.

Figure 2. Overview of Federated Learning Model.

###  2.2. Federated Learning

FL is an approach that places a strong emphasis on privacy by allowing ML
models to be trained locally on individual devices, without the need to share
the underlying data with a centralized server. This decentralized training
process, depicted in Figure 2, ensures that sensitive data remains on the
devices where it is generated, reducing the risk of privacy breaches. By
adopting FL, these privacy risks can be mitigated, as the data remains
securely stored on the local devices, and only aggregated model updates are
shared with the central server. This way, FL strikes a balance between data
privacy and the need for accurate model training, making it a reliable
solution for privacy-preserving ML in various applications. Du et al. (Du et
al., 2020) conducted a comprehensive survey of existing studies on FL and its
use in wireless IoT. Then, they highlighted the potential benefits of FL in
addressing the unique requirements and complexities of vehicular IoT
environments.

Mothukuri et al. (Mothukuri et al., 2021) introduced a novel approach for
anomaly detection in IoT networks using FL. Their proposed method leverages
decentralized on-device data to proactively identify intrusions in IoT
networks.

###  2.3. Digital Twin

DT is referred to as a virtual representation of the real-world entity,
devices, machines, process, or other abstraction. Physical sensors, computer
programs, machine learning algorithms, and software models are used to
simulate real-time digital models of the physical entity. Tao et al. (Tao et
al., 2018) mentioned that It is considered as one of the most promising
enabling technologies for realizing smart manufacturing and Industry 4.0. Wang
et al. (Wang et al., 2023) conducted a comprehensive review of the Internet of
Digital Twins (IoDT), focusing on various aspects such as system architecture,
enabling technologies, and security/privacy concerns. It facilitates real-time
interaction, close monitoring, and reliable communication between the digital
model and its physical counterpart. DT is considered to be one of the most
promising technology due to its advanced capabilities, intelligent services,
and bridging the gap between the digital model and its physical counterpart.
For instance, the DT of a vehicle can observe the driving pattern of the
driver and communicate, interact, and share this information with other DTs to
notify the driver about possible issues or emergencies on road.

Previous research has introduced several anomaly detection models based on FL
in various domains. These models have been deployed either on centralized
cloud servers or edge devices. Additionally, the concept of digital twins (DT)
has been utilized to identify anomalies in the industrial domain. However, as
mentioned earlier, these anomaly detection models often suffer from low
accuracy rates due to limited volumes of data available for training.
Consequently, a robust anomaly detection model that can provide effective
security and privacy solutions for protecting V-IoT systems is still lacking.
To bridge this gap, our proposed integrated approach-based anomaly detection
model offers a novel perspective for detecting anomalies in the vehicular
domain. By combining the strengths of different technologies, such as FL and
DT, we aim to enhance the accuracy and effectiveness of anomaly detection in
the V-IoT systems. Our approach provides a comprehensive solution that
leverages collaborative learning among distributed entities and utilizes the
virtual replicas created by digital twins to simulate and analyze various
factors contributing to anomalies. We strongly believe that our integrated
approach presents a promising solution to address the challenges of anomaly
detection in the vehicular domain.

Figure 3. Overview of Hierarchical Federated Learning Model.

##  3\. Hierarchical Federated Learning based Anomaly Detection Model

FL can be widely adopted in the V-IoT domain to train various ML models, such
as prediction analysis and anomaly detection, by collecting data from vehicles
in a privacy-preserving environment. This approach offers several advantages,
including low latency, high efficiency, data privacy, and improved security
mechanisms. For RPM use case, a FL-based anomaly detection model (Gupta et
al., 2021d) is proposed. This model leverages edge computing to execute the
anomaly detection models locally on the edge devices without sharing patients’
data with a centralized server. To further enhance the capabilities of the FL
model for multi-user scenarios, HFL approach is developed. HFL allows the
aggregation of gradients at multiple levels, enabling the participation of
multiple entities while leveraging edge computing and DT technologies. Figure
3 provides an overview of this approach. In this research, HFL approach is
used to develop anomaly detection model for the V-IoT systems.

In the connected automotive environment, there are various types of anomalies,
such as traffic congestion, collision detection, malicious attacks, vehicle
breakdown, traffic violations and driver fatigue or distraction. The detection
and timely response to these anomalies can contribute to improving safety,
efficiency, and overall performance in connected vehicle environments.
Detecting and understanding anomalies in the V-IoT can lead to enhanced
safety, security, performance optimization, and better management of traffic
and resources. It allows for proactive decision-making and timely
interventions to ensure a smoother and more efficient functioning of the
connected vehicle ecosystem.

To develop a HFL based anomaly detection model for the V-IoT, we define the
objectives, performance metrics, and requirements for the anomaly detection
model. Then, gather relevant data from vehicles in the V-IoT, which may
include sensor data, vehicle telemetry, weather statistics, traffic light data
and historical records. Ensure that the data collection process preserves
privacy and follows ethical guidelines, which is provided by city policies.
After that, clean and preprocess the collected data to remove noise, handle
missing values, and normalize the features. This step is crucial for preparing
the data for further analysis and training. In next step, design the
hierarchical architecture for FL in the V-IoT. Where, we need to determine the
levels of aggregation such as vehicle-level, region-level, or vendor-level,
based on the collaboration requirements and privacy considerations.

To train the data, we utilize the FedTimeDis LSTM (Gupta et al., 2021d)
approach, which is specifically designed for the connected automotive
environment. Now, each smart vehicle performs local training using their own
data. This training is done in a privacy-preserving manner, where data remains
on the local device and only model updates (e.g., gradients) are shared. To
perform gradient aggregation at each level of the hierarchy to combine the
model updates from different participants. This aggregation process ensures
that the collective knowledge of the participating entities is utilized to
improve the overall anomaly detection model. After developing the model,
evaluate the performance of the aggregated anomaly detection model using
evaluation metrics such as accuracy, precision, recall, or F1-score. Refine
the model if necessary by adjusting hyperparameters, incorporating feedback,
or retraining with additional data. This developed HFL-based anomaly detection
model can be deployed in a real-world V-IoT environment and the performance of
the deployed model can be monitored continuously. Incorporate new data, update
the model periodically, and iterate on the anomaly detection process to
enhance its accuracy, efficiency, and robustness.

Algorithm 1 Anomaly Detection of all data nodes $N$

1: Collect vehicular the data $D_{i}$ from all data nodes $N$

2: Preparing the data $D_{i}$ by converting into numerical form

3: Normalize the data $D_{i}$

4: Create the sequences sets $(X^{n},y^{n})$ of data based on correlations

5: Take these input sequence sets $(X^{n},y^{n})$, where $n=1,2,$…$,N$ from
$N$ data nodes, initial model parameter $w_{z}$, local minibatch size $J$,
number of local epochs $H$, learning rate $\alpha$, number of rounds $Q$, $h$
hidden layer.

6: Split local dataset $D_{i}$ to mini batches of size $J$ which are included
into the set $J_{i}$ and fed horizontally to four LSTM cells.

7: for each local epoch $j$ from 1 to $H$ do

8: for batch $(X,y)$ $\in$ $J$ do

9:
${h\textsubscript{t}=LSTM(h\textsubscript{t-1},x\textsubscript{t},w\textsuperscript{n})}$

10: $y$n = ${\sigma(W\textsuperscript{FC}h\textsubscript{2nd}+Bias)}$

11: $u$n = $w$n \- ${w_{z}^{n}}$

12: $w_{z}^{n}$ $\leftarrow$ $w_{z}^{n}$ \+ $\frac{\alpha}{N}$
${{\sum}}\textsubscript{$n$ $\in$ $D_{i}$}u\textsuperscript{n}$

13: end for

14: end for

15: Update weights $w_{z}^{n}$ to federated cloudlet server and start training
again until minimizing the error to build the anomaly detection model.

Figure 4. System Model for V-IoT System.

##  4\. System Model

In this section, we introduce our proposed model, which aims to identify
anomalies and enhance the security of the V-IoT systems. The model comprises
six distinct phases, each involving data exchange and collaboration among
different entities. The overall system architecture is illustrated in Figure
4, providing a visual representation of the data flow and interaction between
the components.

Figure 5. Use Case for Vehicular IoT for Anomaly Detection in Federated
Setting.

The six phases of our proposed model are as follows:

  * •

Initial Phase: During the initial phase of our proposed model, the smart
vehicle begins collecting various types of data, including manufacturing data,
driver perception data, and external entities data. By collecting these
various types of data, the smart vehicle aims to gather comprehensive
information about its own performance, the driver’s behavior, and the external
environment.

  * •

Functional Phase: During the functional phase of our model, the entities
within the V-IoT system transition into operational mode. This phase is
divided into two sub-phases, each serving specific purposes. In the first sub-
phase, the focus is on collecting data from the vehicle IoT sensors. These
sensors, which are activated and operational, capture various types of
information such as vehicle diagnostics, and performance metrics. The
collected data is then transmitted to the vendor cloudlet, a cloud-based
infrastructure specifically designed to handle V-IoT data. Simultaneously, in
the second sub-phase, additional data is collected on the vendor cloudlet.
This includes a wide range of data sources such as weather statistics, city
regulations and policies, traffic light information, and camera data. These
supplementary data sources provide contextual information about the external
environment in which the vehicles operate.

By collecting data from both the vehicle IoT sensors and other relevant
sources on the vendor cloudlet, a comprehensive and multi-dimensional dataset
is created.

  * •

Analytic Phase: Once the data is collected from the V-IoT system, it is
transmitted to the simulated environment for further analysis. This phase
involves the transition of data from the physical space to the simulated
space, where advanced data analytics techniques are applied. In the simulated
environment, a DT is developed for each entity within the V-IoT system. A DT
is a virtual representation of a physical entity, in this case, the vehicles
and other components of the V-IoT system. The DT is created based on the
generated data collected from the previous phases. The data analytics process
is then performed on the vehicle DT. Various analytical techniques and
algorithms are applied to gain insights and extract valuable information from
the data. These analytics help in understanding the behavior, performance, and
patterns within the V-IoT system.

By leveraging the DT and conducting data analytics, it becomes possible to
identify and understand anomalies within the V-IoT system. Anomalies can
include unusual behavior, deviations from normal patterns, or any abnormal
activities that may indicate potential security or operational issues.

  * •

Identifying Anomaly Phase: In this phase, the simulated data $D_{i}$ from all
the data nodes that has been processed and prepared in the previous phase is
passed through a pipeline to feed the anomaly detection model. The data is
carefully curated and transformed to be compatible with the model’s input
requirements. The anomaly detection model is developed using suitable machine
learning algorithms and follows the Algorithm 1. These algorithms are trained
on the prepared data to learn the patterns and characteristics of normal
behavior within the V-IoT system. The model aims to distinguish between normal
and anomalous patterns based on the input data. During the training process,
the model undergoes iterations to optimize its performance and enhance its
ability to accurately detect anomalies. This involves adjusting the model’s
parameters, fine-tuning the algorithms, and validating the model’s performance
using appropriate evaluation metrics. Once the training is completed, the
anomaly detection model is ready to be deployed and utilized. It collaborates
with other models that are part of the subsequent phases, working together to
enhance the accuracy and effectiveness of anomaly detection in the V-IoT
system.

  * •

Collaborative Phase: Indeed, in this phase, the collaboration of multiple
anomaly detection models take place to improve the accuracy rate of anomaly
detection in the V-IoT system. By combining the weights of multiple ADM
models, the overall effectiveness of anomaly detection can be significantly
enhanced. Each model may have its own unique approach, algorithm, or
specialization in detecting specific types of anomalies. By leveraging the
strengths and capabilities of different models, a more comprehensive and
robust anomaly detection system can be established. The collaboration among
anomaly detection models involves exchanging information, sharing insights,
and aggregating their detection results. This collaborative process allows for
a holistic analysis of the system’s behavior and the identification of
anomalies from multiple perspectives.

  * •

Reporting and Decision Phase: After an anomalous scenario is detected by the
anomaly detection model, it is crucial to report the anomaly to the relevant
stakeholders, including the user, vendor, and device. This phase plays a vital
role in facilitating informed decision-making and taking necessary actions to
ensure the safety and security of the automotive connected environment.
Reporting the anomaly to the user is essential as it enables them to be aware
of the detected anomaly and take appropriate measures. This could involve
alerting the user through notifications, messages, or visual indicators,
providing them with information about the anomaly and any recommended actions
they should take. Notifying the vendor is also crucial as it allows them to be
aware of the anomaly and take the necessary steps to address the issue. This
could involve investigating the root cause of the anomaly, analyzing the data
collected, and implementing corrective measures to prevent similar anomalies
in the future.

Overall, this phase of reporting anomalies is a critical component of the
anomaly detection process in the V-IoT system. It helps to minimize the risks
associated with anomalous events, enables proactive decision-making, and
contributes to maintaining a safe and reliable automotive ecosystem.

By employing our proposed system model, the V-IoT environment can detect
anomalies in real-time and enable prompt responses to the system. This
improves overall security and privacy, enhances the efficiency of the
transportation system, and improves the driving experience for individuals.

The following section presents a use case scenario of V-IoT, where our
proposed system model is employed to detect anomalies.

##  5\. Use Case Scenario

V-IoT is unfolding in many ways where users receive better services and take
advantage of autonomous vehicle. The proposed system model where we present
the integration of FL and DT, which can be used to secure V-IoT applications,
for example, ITSs, cooperative autonomous driving, connected car services,
smart city integration, collision avoidance systems and intelligent traffic
control etc. In this section, we present a use-case scenario for securing
V-IoT by developing an anomaly detection model. The Figure 5 shows the use
case based on our proposed system model, which is discussed in Section 4.

In a smart city, the adoption of IoT technologies and the deployment of smart
vehicles are guided by common city policies and regulations. These policies
ensure uniformity and standardization across different regions within the
smart city. Each region within the smart city may have multiple vendors
launching their smart vehicles, contributing to the overall intelligent
transportation ecosystem. The presence of multiple vendors in different
regions allows for a diverse range of smart vehicles with varying features,
technologies, and capabilities. These vehicles may be equipped with advanced
sensors, communication systems, and intelligent algorithms to enhance their
functionality and contribute to the overall smart city objectives.

In Figure 5, the depicted scenario showcases the presence of two different
vendors, namely vendor-1 and vendor-2, operating within region-1 of the V-IoT
system. Vendor-1 has two smart vehicles, SV-1 and SV-3, while vendor-2 has one
smart vehicle, SV-2. Additionally, region-1 consists of various data points,
including sensor data, city policies and regulations, traffic lights, and
weather statistics. To enable efficient data processing and anomaly detection
in region-1, a cloudlet is launched specifically for this region. The cloudlet
serves as a localized computing resource that can host and deploy DTs of each
smart vehicle within the region. These DTs not only receive data from their
respective smart vehicles (SV-1, SV-2, and SV-3) but also incorporate data
from other sources within the region, such as sensor data, city policies and
regulations, traffic lights, and weather statistics.

In this use case, we deploy DTs on the cloudlet to reduce the gap between
physical objects and their digital representations which are generally hosted
in the cloud servers. These cloudlets are hosted by the regions and basically
a regional cloud that can better cloud services nearer to the user. Cloudlets
(Saini and Raj, 2022) are small-scale, mobility-enhanced cloud data centers
that sit at the network’s edge. The cloudlet’s primary goal is to support
furious resource and interactive mobile applications by delivering strong
computing resources to mobile devices with reduced latency. A wireless local
area network with single hop at comparatively higher speed, allows User
Equipments (UEs) to connect to the computing resources in the neighboring
cloudlet. By leveraging the cloudlet in region-1, the DTs can effectively
analyze and process the combined data from multiple sources. This integration
of data from various entities allows for a holistic understanding of the V-IoT
environment within region-1. The DTs can leverage this comprehensive data to
enhance anomaly detection capabilities, identify patterns, and detect any
deviations or anomalies in the behavior of the smart vehicles or the overall
V-IoT system. The deployment of DTs within the cloudlet of region-1 enables
localized processing and analysis, reducing latency and enhancing real-time
anomaly detection. The collaboration of the DTs with the cloudlet
infrastructure facilitates efficient information exchange and enables timely
response to any detected anomalies.

DT of SV-1 within region-1 plays a crucial role in the development of the
anomaly detection model. The data collected by SV-1’s DT is utilized to train
the initial anomaly detection model specific to region-1. This model focuses
on detecting anomalies within the local context and behavior of the vehicles
and infrastructure within region-1. To enhance the accuracy and effectiveness
of the anomaly detection model, collaboration is encouraged among multiple
models. In this case, the anomaly detection models of region-1 have the
capability to collaborate with each other using the concept of FL. FL allows
the models to share their knowledge and insights while maintaining data
privacy and security. By aggregating the local models’ learnings through
weight aggregation techniques, a more robust and accurate anomaly detection
model can be obtained.

Moreover, collaboration is not limited to models within the same region. The
anomaly detection model of region-1 can also collaborate with the anomaly
detection model of region-2 by using HFL concept. This collaboration is
facilitated by exchanging gradients on a federated multi-cloud server. The
gradients represent the model parameters that are shared and utilized to
improve the models’ performance collectively. By leveraging the collaboration
capabilities of FL and the exchange of gradients on the federated multi-cloud
server, the anomaly detection models of different regions can benefit from
each other’s insights and experiences. This cross-region collaboration
enhances the overall effectiveness of anomaly detection in the V-IoT system by
incorporating knowledge from diverse geographical areas and vehicle behaviors.

In summary, the integration of FL and the exchange of gradients enable
collaboration among anomaly detection models at different levels by utilizing
DT. This collaboration improves the accuracy and robustness of the models,
both within the same region and across different regions, leading to more
effective anomaly detection in the V-IoT system.

##  6\. Conclusion and Future Work

In this paper, we have discussed the relevance of FL in V-IoT environments and
its potential impact. We started by providing background information on V-IoT,
FL, and DT technologies. We emphasized the importance of anomaly detection
models in the V-IoT domain and also presented the outline of HFL based anomaly
detection model. To address the challenges and opportunities in the V-IoT, we
proposed a system model that integrates DT and FL. This model leverages the
collaborative learning capabilities of FL and the bridging capabilities of DT
between the physical system and its virtual representation. We outlined the
key components and phases of our proposed model, emphasizing data exchange,
anomaly detection, and security.

To illustrate the practical application of our proposed model, we presented a
use case scenario in which the model is employed to detect anomalies in the
V-IoT environment. The scenario showcased the data collection, processing,
anomaly detection, and collaborative response aspects of our model,
highlighting its potential benefits in ensuring safety and efficiency in the
V-IoT systems.

Overall, this work aims to contribute to the advancement of FL, DT, and V-IoT
research. By introducing our proposed model and presenting a use case
scenario, we provide a foundation for further exploration, development, and
practical implementation of HFL and DT in the V-IoT environments. We believe
that this paper will facilitate the progress of these fields and stimulate
further research in the area of HFL-based anomaly detection by utilizing DT in
the V-IoT.

## References

  * (1)
  * Akbarfam et al. (2023) Asma Jodeiri Akbarfam, Sina Barazandeh, Hoda Maleki, and Deepti Gupta. 2023.  DLACB: Deep Learning Based Access Control Using Blockchain.  arXiv preprint arXiv:2303.14758 (2023).
  * Aslan et al. (2021) Ömer Aslan, Merve Ozkan-Okay, and Deepti Gupta. 2021\.  Intelligent behavior-based malware detection system on cloud computing environment.  IEEE Access 9 (2021), 83252–83271.
  * Chai et al. (2020) Haoye Chai, Supeng Leng, Yijin Chen, and Ke Zhang. 2020\.  A hierarchical blockchain-enabled federated learning algorithm for knowledge sharing in internet of vehicles.  IEEE Transactions on Intelligent Transportation Systems 22, 7 (2020), 3975–3986.
  * Du et al. (2020) Zhaoyang Du, Celimuge Wu, Tsutomu Yoshinaga, Kok-Lim Alvin Yau, Yusheng Ji, and Jie Li. 2020\.  Federated learning for vehicular internet of things: Recent advances and open issues.  IEEE Open Journal of the Computer Society 1 (2020), 45–61.
  * Gupta et al. (2021a) Deepti Gupta, Smriti Bhatt, Paras Bhatt, Maanak Gupta, and Ali Saman Tosun. 2021a.  Game Theory Based Privacy Preserving Approach for Collaborative Deep Learning in IoT.  In Deep Learning for Security and Privacy Preservation in IoT. Springer, 127–149.
  * Gupta et al. (2020a) Deepti Gupta, Smriti Bhatt, Maanak Gupta, Olumide Kayode, and Ali Saman Tosun. 2020a.  Access control model for google cloud iot. In 2020 IEEE 6th Intl conference on big data security on cloud (BigDataSecurity), IEEE Intl conference on high performance and smart computing,(HPSC) and IEEE Intl conference on intelligent data and security (IDS). IEEE, 198–208.
  * Gupta et al. (2021b) Deepti Gupta, Smriti Bhatt, Maanak Gupta, and Ali Saman Tosun. 2021b.  Future smart connected communities to fight covid-19 outbreak.  Internet of Things 13 (2021), 100342.
  * Gupta et al. (2021c) Deepti Gupta, Maanak Gupta, Smriti Bhatt, and Ali Saman Tosun. 2021c.  Detecting anomalous user behavior in remote patient monitoring. In 2021 IEEE 22nd International Conference on Information Reuse and Integration for Data Science (IRI). IEEE, 33–40.
  * Gupta et al. (2020b) Deepti Gupta, Olumide Kayode, Smriti Bhatt, Maanak Gupta, and Ali Saman Tosun. 2020b.  Learner’s dilemma: IoT devices training strategies in collaborative deep learning. In 2020 IEEE 6th World Forum on Internet of Things (WF-IoT). IEEE, 1–6.
  * Gupta et al. (2021d) Deepti Gupta, Olumide Kayode, Smriti Bhatt, Maanak Gupta, and Ali Saman Tosun. 2021d.  Hierarchical federated learning based anomaly detection using digital twins for smart healthcare. In 2021 IEEE 7th International Conference on Collaboration and Internet Computing (CIC). IEEE, 16–25.
  * Ji et al. (2020) Baofeng Ji, Xueru Zhang, Shahid Mumtaz, Congzheng Han, Chunguo Li, Hong Wen, and Dan Wang. 2020.  Survey on the internet of vehicles: Network architectures and applications.  IEEE Communications Standards Magazine 4, 1 (2020), 34–41.
  * Li et al. (2023) Yuchen Li, Weifa Liang, Jing Li, Xiuzhen Cheng, Dongxiao Yu, Albert Y Zomaya, and Song Guo. 2023.  Energy-Aware, Device-to-Device Assisted Federated Learning in Edge Computing.  IEEE Transactions on Parallel and Distributed Systems (2023).
  * Lu et al. (2020) Yunlong Lu, Xiaohong Huang, Ke Zhang, Sabita Maharjan, and Yan Zhang. 2020.  Blockchain empowered asynchronous federated learning for secure data sharing in internet of vehicles.  IEEE Transactions on Vehicular Technology 69, 4 (2020), 4298–4311.
  * Moni and Gupta (2022) Shafika Showkat Moni and Deepti Gupta. 2022.  Secure and Efficient Privacy-preserving Authentication Scheme using Cuckoo Filter in Remote Patient Monitoring Network.  The Fourth IEEE International Conference on Trust, Privacy and Security in Intelligent Systems, and Applications (2022).
  * Moni and Manivannan (2020) Shafika Showkat Moni and D Manivannan. 2020.  An efficient RSU authentication scheme based on Merkle Hash Tree for VANETs. In ICC 2020-2020 IEEE International Conference on Communications (ICC). IEEE, 1–7.
  * Moni and Manivannan (2021) Shafika Showkat Moni and D Manivannan. 2021.  A scalable and distributed architecture for secure and privacy-preserving authentication and message dissemination in VANETs.  Internet of Things 13 (2021), 100350.
  * Moni and Manivannan (2022) Shafika Showkat Moni and D Manivannan. 2022.  CREASE: Certificateless and REused-pseudonym based Authentication Scheme for Enabling security and privacy in VANETs.  Internet of Things 20 (2022), 100605.
  * Mothukuri et al. (2021) Viraaji Mothukuri, Prachi Khare, Reza M Parizi, Seyedamin Pouriyeh, Ali Dehghantanha, and Gautam Srivastava. 2021.  Federated-learning-based anomaly detection for iot security attacks.  IEEE Internet of Things Journal 9, 4 (2021), 2545–2554.
  * Ozkan-Okay et al. (2021) Merve Ozkan-Okay, Refik Samet, Ömer Aslan, and Deepti Gupta. 2021.  A Comprehensive Systematic Literature Review on Intrusion Detection Systems.  IEEE Access (2021).
  * Pei et al. (2022) Jiaming Pei, Kaiyang Zhong, Mian Ahmad Jan, and Jinhai Li. 2022\.  Personalized federated learning framework for network traffic anomaly detection.  Computer Networks 209 (2022), 108906.
  * Pokhrel and Choi (2020) Shiva Raj Pokhrel and Jinho Choi. 2020.  Federated learning with blockchain for autonomous vehicles: Analysis and design challenges.  IEEE Transactions on Communications 68, 8 (2020), 4734–4746.
  * Saini and Raj (2022) Kavita Saini and Pethuru Raj. 2022.  Chapter Eight-Edge platforms, frameworks and applications.  Adv. Comput. 127 (2022), 237–258.
  * Shrivastava and Dwivedi (2021) Atul Lal Shrivastava and Rajendra Kumar Dwivedi. 2021.  Designing A Secure Vehicular Internet of Things (IoT) using Blockchain: A Review. In 2021 First International Conference on Advances in Computing and Future Communication Technologies (ICACFCT). IEEE, 225–230.
  * Tao et al. (2018) Fei Tao, He Zhang, Ang Liu, and Andrew YC Nee. 2018\.  Digital twin in industry: State-of-the-art.  IEEE Transactions on industrial informatics 15, 4 (2018), 2405–2415.
  * Wang et al. (2023) Yuntao Wang, Zhou Su, Shaolong Guo, Minghui Dai, Tom H Luan, and Yiliang Liu. 2023\.  A survey on digital twins: architecture, enabling technologies, security and privacy, and future prospects.  IEEE Internet of Things Journal (2023).
  * Yang et al. (2014) Fangchun Yang, Shangguang Wang, Jinglin Li, Zhihan Liu, and Qibo Sun. 2014.  An overview of internet of vehicles.  China communications 11, 10 (2014), 1–15.


#  Taweret: a Python package for Bayesian model mixing

K. Ingles<EMAIL_ADDRESS>Illinois Center for Advanced Studies of the
Universe & Department of Physics,
University of Illinois Urbana-Champaign, Urbana, IL 61801, USA  D. Liyanage
<EMAIL_ADDRESS>Department of Physics, The Ohio State University, Columbus,
OH 43210, USA  A. C. Semposki<EMAIL_ADDRESS>Department of Physics and
Astronomy and Institute of Nuclear and Particle Physics, Ohio University,
Athens, OH 45701, USA  J. C. Yannotty<EMAIL_ADDRESS>Department of Statistics, The Ohio State University, Columbus, OH 43210, USA

Taweret: a Python package for Bayesian model mixing

K. Ingles1,* 11footnotetext<EMAIL_ADDRESS>D.
Liyanage2,$\dagger$22footnotetext<EMAIL_ADDRESS>A. C.
Semposki3,$\ddagger$33footnotetext<EMAIL_ADDRESS>J. C.
Yannotty4,$\S$44footnotetext<EMAIL_ADDRESS>
1 Illinois Center for Advanced Studies of the Universe & Department of
Physics,
University of Illinois Urbana-Champaign, Urbana, IL 61801, USA

2 Department of Physics, The Ohio State University, Columbus, OH 43210, USA

3 Department of Physics and Astronomy and Institute of Nuclear and Particle
Physics, Ohio University, Athens, OH 45701, USA

4 Department of Statistics, The Ohio State University, Columbus, OH 43210, USA

##  I Summary

Uncertainty quantification using Bayesian methods is a growing area of
research. Bayesian model mixing (BMM) is a recent development which combines
the predictions from multiple models such that each model’s best qualities are
preserved in the final result. Practical tools and analysis suites that
facilitate such methods are therefore needed. Taweret introduces BMM to
existing Bayesian uncertainty quantification efforts. Currently Taweret
contains three individual Bayesian model mixing techniques, each pertaining to
a different type of problem structure; we encourage the future inclusion of
user-developed mixing methods. Taweret’s first use case is in nuclear physics,
but the package has been structured such that it should be adaptable to any
research engaged in model comparison or model mixing.

##  II Statement of need

In physics applications, multiple models with different physics assumptions
can be used to describe an underlying system of interest. It is usually the
case that each model has varying fidelity to the observed process across the
input domain. Though each model may have similar predictive accuracy on
average, the fidelity of the approximation across a subset of the domain may
differ drastically for each of the models under consideration. In such cases,
inference and prediction based on a single model may be unreliable. One
strategy for improving accuracy is to combine, or “mix”, the predictions from
each model using a linear combination or weighted average with input-dependent
weights. This approach is intended to improve reliability of inference and
prediction and properly quantify model uncertainties. When operating under a
Bayesian framework, this technique is referred to as Bayesian model mixing
(BMM). In general, model mixing techniques are designed to combine the
individual mean predictions or density estimates from the $K$ models under
consideration. For example, mean-mixing techniques predict the underlying
system by

$E[\bm{Y}\mid\bm{x}]=\sum_{k=1}^{K}w_{k}(\bm{x})\;f_{k}(\bm{x}),$ (1)

where $E[\bm{Y}\mid\bm{x}]$ denotes the mean of $\bm{Y}$ given the vector of
input parameters $\bm{x}$, $f_{k}(\bm{x})$ is the mean prediction under the
$k^{\mathrm{th}}$ model $\mathcal{M}_{k}$, and $w_{k}(\bm{x})$ is the
corresponding weight function. The density-mixing approach estimates the
underlying predictive density by

$p(\bm{Y}_{0}\mid\bm{x}_{0})=\sum_{k=1}^{K}w_{k}(\bm{x}_{0})\;p(\bm{Y}_{0}\mid\bm{x}_{0},\bm{Y},\mathcal{M}_{k}),$
(2)

where $p(\bm{Y_{0}}\mid\bm{x}_{0},\bm{Y},\mathcal{M}_{k})$ represents the
predictive density of a future observation $\bm{Y_{0}}$ with respect to the
$k^{\mathrm{th}}$ model $\mathcal{M}_{k}$ at a new input $\bm{x}_{0}$. In
either BMM setup, a key challenge is defining $w_{k}(\bm{x})$—the functional
relationship between the inputs and the weights.

This work introduces Taweret, a Python package for Bayesian model mixing that
includes three novel approaches for combining models, each of which defines
the weight function in a unique way (see Table 1 for a comparison of each
method). This package has been developed as an integral piece of the Bayesian
Analysis of Nuclear Dynamics (BAND) collaboration’s software. BAND is a multi-
institutional effort to build a cyber-infrastructure framework for use in the
nuclear physics community [1, 2]. The software is designed to lower the
barrier for researchers to employ uncertainty quantification in their
experiments, and to integrate, as best as possible, with the community’s
current standards concerning coding style (pep8). Bayesian model mixing is one
of BAND’s four central pillars in this framework (the others being emulation,
calibration, and experimental design).

In addition to this need, we are aware of several other fields outside of
physics that use techniques such as model stacking and Bayesian model
averaging (BMA) [3], e.g., statistics [4, 5], meteorology [6], and
neuroscience [7]. It is expected that the Bayesian model mixing methods
presented in Taweret can also be applied to use cases within these fields.
Statisticians have developed several versatile BMA/stacking packages, e.g. [8,
9]. However, the only BMM-based package available is SAMBA—a BAND
collaboration effort that was developed for testing BMM methods on a toy model
[10]. Taweret’s increased functionality, user-friendly structure, and diverse
selection of mixing methods make it a marked improvement over SAMBA.

##  III Structure

###  III.1 Overview of methods

Method | Type | Number of | Number of | Number of | Weight
---|---|---|---|---|---
|  | inputs | outputs | models | functions
Bivariate linear | Mean & |  |  |  | Step,
mixing | Density | 1 | $\geq 1$ | 2 | Sigmoid,
|  |  |  |  | Asymmetric 2-step
Multivariate mixing | Mean | 1 | 1 | $K$ | Precision
|  |  |  |  | weighting
BART mixing | Mean | $\geq 1$ | 1 | $K$ | Regression
|  |  |  |  | trees
Table 1: A summary of the three BMM approaches currently implemented in
Taweret. Note that $K\geq 2$. Following the method name and the type of mixing
model, the Number of inputs column details the dimensions of the parameter
which the mixing weights depend on (e.g., in heavy-ion collisions this is the
centrality bin); the Number of outputs details how many observables the models
themselves can have to compute the model likelihood (e.g., in heavy-ion
collisions this can include charge multiplicities, transverse momentum
distributions, transverse momentum fluctuations, etc.); the Number of models
column details how many models the mixing method can combine, and the Weight
functions column describes the available parameterization of how the mixing
weights depend on the input parameter.

####  III.1.1 Bivariate linear mixing

The full description of this mixing method and several of its applications in
relativistic heavy-ion collision physics can be found in the Ph.D. thesis of
D. Liyanage [11]. The bivariate linear mixing method can mix two models either
using a density-mixing or a mean-mixing strategy. Currently, this is the only
mixing method in Taweret that can also calibrate the models while mixing. It
allows the user to choose among the following mixing functions:

  * •

step: $\Theta(\beta_{0}-x)$

  * •

sigmoid: $\exp\left[(x-\beta_{0})/\beta_{1}\right]$

  * •

asymmetric 2-step: $\alpha\Theta(\beta_{0}-x)+(1-\alpha)\Theta(\beta_{1}-x)$.

Here $\Theta$ denotes the Heaviside step function, $\beta_{0}$ and $\beta_{1}$
determine the shape of the weight function and are inferred from the
experimental data, and $x$ is the model input parameter (which is expected to
be 1-dimensional for this mixing method).

####  III.1.2 Multivariate model mixing

Another Bayesian model mixing method incorporated into Taweret was originally
published in [10], and was the focus of the BMM Python package SAMBA [12]. It
can be described as combining models by weighting each of them by their
precision, defined as the inverse of their respective variances. The posterior
predictive distribution (PPD) of the mixed model is a Gaussian and can be
expressed as

$\mathcal{M}_{\dagger}\sim{\mathcal{N}(f_{\dagger},Z_{P}^{-1})}:\quad
f_{\dagger}=\frac{1}{Z_{P}}\sum_{k=1}^{K}\frac{1}{\sigma^{2}_{k}}f_{k},\quad
Z_{P}\equiv\sum_{k=1}^{K}\frac{1}{\sigma^{2}_{k}},$ (3)

where $\mathcal{N}(\mu,\sigma^{2})$ is a normal distribution with mean $\mu$
and variance $\sigma^{2}$, $Z_{P}$ is the precision of the models, and each
individual model is assumed to possess a Gaussian form such as

${\cal M}_{k}\sim{\cal N}(f_{k}(x),\sigma^{2}_{k}(x)).$ (4)

Here, $f_{k}(x)$ is the mean of the model $k$, and $\sigma^{2}_{k}(x)$ its
variance, both at input parameter $x$.

In this method, the software receives the one-dimensional input space $X$, the
mean of the $K$ models at each point $x\in X$ (hence it is a mean-based mixing
procedure), and the variances of the models at each point in $x\in X$. Each
model is assumed to have been calibrated prior to being included in the mix.
The ignorance of this mixing method with respect to how each model was
generated allows for any model to be used, including Bayesian Machine Learning
tools such as Gaussian Processes [10] and Bayesian Neural Networks [13].

####  III.1.3 Model mixing using Bayesian Additive Regression Trees

A third BMM approach implemented in Taweret adopts a mean-mixing strategy
which models the weight functions using Bayesian Additive Regression Trees
(BART) conditional on the mean predictions from a set of $K$ models [14]. This
approach enables the weight functions to be adaptively learned using tree
bases and avoids the need for user-specified basis functions (such as a
generalized linear model). Formally, the weight functions are defined by

$w_{k}(\bm{x})=\sum_{j=1}^{m}g_{k}(\bm{x};T_{j},M_{j}),\quad\text{for}\
k=1,\ldots,K$ (5)

where $g_{k}(\bm{x};T_{j},M_{j})$ defines the $k^{\text{th}}$ output of the
$j^{\text{th}}$ tree, $T_{j}$, using the associated set of parameters,
$M_{j}$. Each weight function is implicitly regularized via a prior to prefer
the interval $[0,1]$. Furthermore, the weight functions are not required to
sum-to-one and can take values outside of the range of $[0,1]$. This
regularization approach is designed to maintain the flexibility of the model
while also encouraging the weight functions to take values which preserve
desired inferential properties.

This BART-based approach is implemented in C++ with the trees module in
Taweret acting as a Python interface. The C++ back-end implements Bayesian
tree models and originates from the Open Bayesian Trees Project (OpenBT) [15].
This module serves as an example for how existing code bases can be integrated
into the Taweret framework.

###  III.2 Overview of package structure

Figure 1: Diagram depicting the base classes, their methods (functions) and
their properties (data).

Taweret uses abstract base classes to ensure compatibility and uniformity of
models and mixing methods. The two base classes are BaseModel and BaseMixer
located in the core folder (see Fig. 1 for a schematic); any model mixing
method developed with Taweret is required to inherit from these. The former
represents physics-based models that may include parameters which need to be
determined by Bayesian inference. The latter, BaseMixer, represents a mixing
method used to combine the predictions from the physics-based models using
Bayesian model mixing.

The design philosophy for Taweret is to make it easy to switch between mixing
methods without having to rewrite an analysis script. Thus, the base classes
prescribe which functions need to be present for interoperability between
mixing methods, and in particular, the models being called in the method. The
functions required by BaseModel are

  * •

evaluate \- gives a point prediction for the model;

  * •

log_likelihood_elementwise \- calculates the log-likelihood, reducing along
the last axis of an array if the input array has multiple axes;

  * •

set_prior \- sets priors for parameters in the model.

The functions required by BaseMixer are

  * •

evaluate \- gives point prediction for the mixed model given a set of
parameters;

  * •

evaluate_weights \- gives point prediction for the weights given a set of
parameters;

  * •

map \- returns the maximum a posteriori estimate for the parameters of the
mixed model (which includes both the weights and any model parameters);

  * •

posterior \- returns the chains of the sampled parameters from the mixed
model;

  * •

predict \- returns the posterior predictive distribution for the mixed model;

  * •

predict_weights \- returns the posterior predictive distribution for the model
weights;

  * •

prior \- returns the prior distributions (typically objects, not arrays);

  * •

prior_predict \- returns the prior predictive distribution for the mixed
model;

  * •

set_prior \- sets the prior distributions for the mixing method;

  * •

train \- executes the Bayesian model mixing step.

Following our design philosophy, the general workflow for an analysis using
Taweret is described in Fig. 2. From this, one can see three sources of
information are generally required for an analysis: a selected mixing method,
a model set, and training data. Each of these sources are connected through
the training phase, which is where the mixing weights are learned. This leads
into the prediction phase, where final predictions are obtained for the
overall system and the weight functions. This process is summarized in the
code snippet below. This workflow is preserved across the various methods
implemented in Taweret and is intended to be maintained for future mixing
methods included in this work.

    ‘‘‘python
        from mix.mix_method import MixMethod
        from models.my_model import MyModel

        mixer = MixMethod(models={’model_1’: MyModel(...), ...})
        mixer.set_prior(...)
        mixer.train(...)
        mixer.predict(...)
        mixer.predict_weights(...)
    ’’’

Extending Taweret with a custom class or model simply requires that you
inherit from the base classes and implement the required functions.

Figure 2: The general workflow for an analysis using Taweret. (Blue) The user
must define each of the $K$ models as a class inherited from BaseModel.
(Green) The user can select an existing mixing method from Taweret (solid) or
contribute a new method (dashed). (Purple) The model is trained using a set of
training data (red), the model set (blue), and the selected mixing method
(green). Predictions and uncertainty quantification follow from the training
process.

##  IV Taweret moving forward

There are certainly many improvements that can be made to Taweret. An obvious
one is a generalization of the bivariate linear mixing; this could be the
mixing of an arbitrary number of models at the density level. Complementary to
this density mixing method is a stochastic, mean-mixing method of arbitrary
number $K$ of models. An extension of the Multivariate Mixing method to multi-
dimensional input and output spaces, correlated models, and well as
calibration during mixing, is anticipated in future releases. Lastly, to
facilitate the utilization of this growing framework, we hope to enable
continuous integration routines for individuals contributing and create docker
images that will run Taweret.

##  V Disclosure Statement

The authors of this work are not aware of any conflicts of interest that would
affect this research.

##  VI Acknowledgments

We thank Daniel R. Phillips, Ulrich Heinz, Matt Pratola, Kyle Godbey, Stefan
Wild, Sunil Jaiswal, and all other BAND members for crucial feedback and
discussion during the development stage of this package. This work is
supported by the CSSI program Award OAC-2004601 (DL, ACS, JCY). ACS also
acknowledges support from the Department of Energy (contract no. DE-
FG02-93ER40756).

## References

  * Phillips  _et al._ [2021] D. R. Phillips, R. J. Furnstahl, U. Heinz, T. Maiti, W. Nazarewicz, F. M. Nunes, M. Plumlee, M. T. Pratola, S. Pratt, F. G. Viens, and S. M. Wild, Get on the BAND Wagon: A Bayesian Framework for Quantifying Model Uncertainties in Nuclear Dynamics, Journal of Physics G 48, 072001 (2021), arXiv:2012.07704 [nucl-th] .
  * Beyer  _et al._ [2023] K. Beyer, L. Buskirk, M. Y.-H. Chan, T. H. Chang, R. J. DeBoer, R. J. Furnstahl, P. Giuliani, K. Godbey, K. Ingles, D. Liyanage, F. M. Nunes, D. Odell, D. R. Phillips, M. Plumlee, M. T. Pratola, A. C. Semposki, O. Sürer, S. M. Wild, and J. C. Yannotty,  _BANDFramework: An Open-Source Framework for Bayesian Analysis of Nuclear Dynamics_, Tech. Rep. Version 0.3.0 (2023).
  * Fragoso  _et al._ [2018] T. M. Fragoso, W. Bertoli, and F. Louzada, Bayesian model averaging: A systematic review and conceptual classification, International Statistical Review 86, 1 (2018), https://onlinelibrary.wiley.com/doi/pdf/10.1111/insr.12243 .
  * Yao  _et al._ [2018] Y. Yao, A. Vehtari, D. Simpson, and A. Gelman, Using stacking to average bayesian predictive distributions (with discussion), Bayesian Analysis, 13 (3) 917 - 1007 10.1214/17-BA1091 (2018).
  * Yao  _et al._ [2022] Y. Yao, G. Pirš, A. Vehtari, and A. Gelman, Bayesian Hierarchical Stacking: Some Models Are (Somewhere) Useful, Bayesian Analysis 17, 1043 (2022).
  * Sloughter  _et al._ [2007] J. M. Sloughter, A. E. Raftery, T. Gneiting, and C. Fraley, Probabilistic quantitative precipitation forecasting using Bayesian model averaging, Monthly Weather Review, 135, 3209–3220 10.1175/MWR3441.1 (2007).
  * FitzGerald  _et al._ [2014] T. H. B. FitzGerald, R. J. Dolan, and K. J. Friston, Model averaging, optimal inference, and habit formation, Frontiers in Human Neuroscience, 8:457 10.3389/fnhum.2014.00457 (2014).
  * Vehtari  _et al._ [2017] A. Vehtari, A. Gelman, and J. Gabry, loo: Efficient leave-one-out cross-validation and waic for Bayesian models (2017) https://mc-stan.org/loo/.
  * Raftery  _et al._ [2022] A. Raftery, J. Hoeting, C. Volinsky, I. Painter, and K. Y. Yeung,  _BMA: Bayesian Model Averaging_ (2022), r package version 3.18.17.
  * Semposki  _et al._ [2022a] A. C. Semposki, R. J. Furnstahl, and D. R. Phillips, Interpolating between small- and large-g expansions using Bayesian model mixing, Physical Review C 106, 044002 (2022a), arXiv:2206.04116 .
  * Liyanage [2023] D. Liyanage,  _Multifaceted study of ultrarelativistic heavy ion collisions_ , Ph.D. thesis, The Ohio State University (2023).
  * Semposki  _et al._ [2022b] A. C. Semposki, R. J. Furnstahl, and D. R. Phillips, SAMBA: SAndbox for Mixing using Bayesian Analysis (2022) https://github.com/asemposki/SAMBA.
  * Kronheim  _et al._ [2022] B. Kronheim, M. Kuchera, and H. Prosper, TensorBNN: Bayesian inference for neural networks using TensorFlow, Computer Physics Communications 270, 108168 (2022), arXiv:2009.14393 [physics.comp-ph] .
  * Yannotty  _et al._ [2023] J. C. Yannotty, T. J. Santner, R. J. Furnstahl, and M. T. Pratola, Model mixing using Bayesian additive regression trees (2023).
  * Pratola  _et al._ [2023] M. T. Pratola, R. E. McCulloch, H. A. Chipman, and A. Horiguchi,  _Open Bayesian Trees Project_ (2023).