1]organization=School of Computer Science, University of Electronic Science
and Technology of China, Zhongshan Institute, Zhongshan 528402, country=China
[style=chinese] Conceptualization of this study, Methodology, Software

[1]

Data curation, Writing - Original draft preparation

2]organization=Research Pervasive and Mobile Computing, and Department of
Software Engineering, College of Computer and Information Sciences, King Saud
University, Riyadh 13272, country=Saudi Arabia

[cor1]Corresponding author

# Correction: Harnessing Federated Generative Learning for Green and
Sustainable Internet of Things

Yuanhang Qi<EMAIL_ADDRESS>M. Shamim Hossain<EMAIL_ADDRESS>[

###### Abstract

The rapid proliferation of devices in the Internet of Things (IoT) has ushered
in a transformative era of data-driven connectivity across various domains.
However, this exponential growth has raised pressing concerns about
environmental sustainability and data privacy. In response to these
challenges, this paper introduces One-shot Federated Learning (OSFL), an
innovative paradigm that harmonizes sustainability and machine learning within
IoT ecosystems. OSFL revolutionizes the traditional Federated Learning (FL)
workflow by condensing multiple iterative communication rounds into a single
operation, thus significantly reducing energy consumption, communication
overhead, and latency. This breakthrough is coupled with the strategic
integration of generative learning techniques, ensuring robust data privacy
while promoting efficient knowledge sharing among IoT devices. By curtailing
resource utilization, OSFL aligns seamlessly with the vision of green and
sustainable IoT, effectively extending device lifespans and mitigating their
environmental footprint. Our research underscores the transformative potential
of OSFL, poised to reshape the landscape of IoT applications across domains
such as energy-efficient smart cities and groundbreaking healthcare solutions.
This contribution marks a pivotal step towards a more responsible,
sustainable, and technologically advanced future.

###### keywords:

IoT Green and Sustainable Federated Learning Generative Learning AI

##  1 Introduction

The advent of the Internet of Things (IoT) [46, 55, 4] has ushered in a new
era of connectivity and data-driven decision-making across various domains
[32, 33, 35, 14]. IoT devices have permeated our daily lives, enabling a wide
range of applications, from smart homes and cities to industrial automation
and healthcare. However, the proliferation of these devices has raised
concerns about their environmental impact, especially with regard to energy
consumption and data privacy. In this context, Federated Learning (FL) [36]
has emerged as a promising paradigm to address these challenges by enabling
collaborative machine learning on decentralized IoT devices, while also
respecting user privacy [26, 49, 41].

Federated learning is a decentralized machine learning approach that enables
multiple devices to collaboratively train a global model while keeping their
data localized and private [30]. The workflow begins with the initialization
of a global model, followed by device participation, where decentralized
clients perform local training on their data, compute model updates, and
securely aggregate them on a central server [7, 19]. This iterative process
refines the global model over multiple rounds, monitored by evaluation metrics
until the desired performance is achieved. FL’s privacy-preserving and
distributed nature makes it suitable for applications like green and
sustainable IoT, healthcare, and federated edge computing, addressing data
privacy concerns while enabling machine learning in diverse and resource-
constrained environments [60, 1, 20, 37, 40].

Traditional FL approaches often require numerous communication rounds between
the central server and participating devices, which can result in significant
energy consumption, latency, and communication overhead [25, 24]. Inspired by
previous work [56, 44, 21, 58], this paper introduces a novel approach known
as “One-shot Federated Learning(OSFL),” aimed at minimizing resource
utilization while maximizing the sustainability of IoT networks. Leveraging
the power of generative learning [58], this approach allows IoT devices to
learn and share knowledge efficiently in a single communication round,
reducing the need for constant connectivity and data transmission. This
approach not only promotes green and sustainable IoT but also enhances the
scalability and practicality of FL in resource-constrained environments.

The need for sustainable IoT solutions has become increasingly pressing as the
world grapples with the challenges of climate change and resource depletion
[3]. IoT devices are projected to proliferate exponentially in the coming
years, making it imperative to design and adopt technologies that can mitigate
their environmental impact [43]. By incorporating generative learning
techniques into the FL framework, this paper presents a promising avenue for
reducing energy consumption and prolonging the lifespan of IoT devices,
thereby contributing to the broader goal of sustainable technology development
[47].

Privacy concerns also loom large in the IoT landscape, as the data generated
by these devices often contains sensitive information [1]. Traditional FL
methods inherently preserve user privacy by keeping data localized, but they
still involve multiple interactions that raise potential security risks [50,
31]. OSFL enhances privacy further by minimizing data exposure and limiting
communication, making it an attractive choice for applications where privacy
is paramount. This paper explores how this approach can strike a delicate
balance between data utility, model performance, and user privacy in green and
sustainable IoT environments. To this end, we summarize the challenges we face
as follows:

  * •

(C1.) Difficult to Converge: Since OSFL only communicates once, this brings
severe challenges to global model convergence. Unlike traditional FL, OSFL
cannot iteratively update the model and perform multiple rounds of
communication [58].

  * •

(C2.) New Privacy Concerns: Although OSFL only conducts one-shot communication
to mitigate privacy risks to a certain extent, adversaries still have the
opportunity to threaten OSFL’s privacy. The main reason is that the knowledge
uploaded by OSFL is not protected by privacy protection measures.

To tackle the aforementioned challenges, this paper utilizes the concept of
OSFL as a novel and environmentally friendly approach to machine learning
within the realm of the IoT. Specifically, we present an innovative framework
termed Federated Generative Learning (FGL) which was proposed by Zhang et al.
[58]. FGL harnesses the robust capabilities of generative models, including
Stable Diffusion and GPT-4V, to generate high-fidelity surrogate training data
directly on the server. This process is guided by prompts provided by
individual clients. In a concrete sense, our method streamlines the workflow
by having each client upload only their respective prompts, linked to their
local training data. Once all prompts are gathered from the clients, the
server orchestrates prompt aggregation and subsequently synthesizes a premium-
quality surrogate training dataset. This dataset can then be employed to train
a global model. By leveraging generative learning techniques and minimizing
communication rounds, this approach aims to address the challenges of energy
efficiency, scalability, and privacy in IoT networks. The subsequent sections
of this paper will delve into the technical details, experiments, and
practical implementations of OSFL, illustrating its potential to usher in a
greener and more sustainable era for IoT applications. This paper’s primary
contributions are as follows:

  * •

We customize a new one-shot federated generative learning system for green and
sustainable IoT, which can effectively alleviate data privacy and
communication overhead issues.

  * •

We propose two new local prompt generation strategies, which aim to extract
fine-grained feature information of local data to enhance the model’s
generalization ability.

  * •

We develop a one-shot FL method with generative learning as the core, which is
organically combined with generation to improve the overall performance of FL.
In particular, the one-shot FL method based on generative learning
significantly improves the problem of expensive communication in IoT.

  * •

We conduct comprehensive validation and evaluation on three benchmark
datasets, and the experimental results demonstrate the effectiveness of the
proposed method.

The paper’s structure is organized as follows: In Section 2, we provide a
comprehensive overview of the background and concepts related to FL and the
IoT. This section highlights the existing challenges and open problems within
this context. Section 3 delves into the foundational aspects of federated
learning and text-to-image models, offering a clear understanding of these
crucial components. Furthermore, we explore the practical application
scenarios of text-to-image models and delve into the core challenges currently
faced in this field. In Section 4, we introduce our innovative One-shot FGL
system, detailing its architecture and mechanisms. We then proceed to Section
5, where we present a series of case studies conducted on three real-world
datasets. In these case studies, we thoroughly analyze the experimental
results, providing valuable insights and observations. Lastly, in Section 6,
we draw our conclusions, summarizing the key findings and contributions of our
work in the context of OSFL for green and sustainable IoT. We summarize the
notation descriptions used in this paper in Table 1.

Table 1: Glossary of symbols.

$B$ | The mini-batch size
---|---
$\mathcal{B}$ | The training batch set
$\mathcal{D}^{t}_{k}$ | The local training data at training round $t$
$n_{k}$ | The number of training samples at $k$-th client
$\mathcal{D}_{s}$ | The task-agnostic surrogate dataset
$K$ | The number of clients
$\mathcal{K}$ | The client set
$p$ | The prompt generated by edge devices
$\mathcal{S}$ | The Synthetic training dataset
$\omega^{t}$ | The global model at training round $t$
$\ell_{k}$ | The local training loss
$\ell_{gen}$ | The generator loss
$\ell_{disc}$ | The discriminator loss
$G_{t}(\cdot.\cdot)$ | The denoising network

##  2 Related Work

In the pursuit of advancing green and sustainable IoT through the lens of FL,
this section provides an overview of existing research in four key
subsections:

###  2.1 Federated Learning in IoT

The integration of FL in the IoT has garnered substantial attention, primarily
for its ability to harmonize the divergent requirements of data-driven
intelligence and sustainability in resource-constrained environments. Prior
research in this domain has delved into diverse FL strategies, encompassing
federated optimization techniques, communication-efficient algorithms, and
privacy-preserving methods. Notable contributions include the works of McMahan
et al. [36] on federated averaging, which laid the foundation for distributed
model training, and Bonawitz et al. [8] on secure aggregation, addressing
privacy concerns in FL. Moreover, recent research by Wei et al. [50]
introduced the concept of "Federated Learning with Differential Privacy,"
further enhancing privacy guarantees in IoT applications. We build upon these
foundations and extend them by introducing One-shot Federated Learning, which
minimizes communication rounds and maximizes sustainability through generative
learning.

Efforts to minimize communication between IoT devices and the central server
have been pivotal in the context of FL for green IoT [45]. Various federated
optimization [30, 39, 48] techniques have been proposed to reduce the amount
of information exchanged during each communication round. Notable works
include the FedPAQ algorithm introduced by [42], which enables IoT devices to
send only model updates to the central server rather than transmitting raw
data. This approach significantly reduces communication bandwidth, making it
an essential step towards greener and more sustainable IoT practices.

Quantization and compression techniques have emerged as effective means to
further enhance communication efficiency in FL for IoT [9, 36]. Researchers
have explored methods to quantize model updates into compact representations,
thereby reducing the data transfer requirements between clients and the
central server. Chen et al. [9] proposed a quantization approach, Q-FFL, which
compresses model updates to a fraction of their original size, alleviating the
communication burden on IoT devices. Such techniques align with the green IoT
agenda by conserving energy and reducing the carbon footprint associated with
data transmission.

###  2.2 Sustainability in IoT

Promoting environmental sustainability in IoT has become a critical research
area, given the ever-growing number of IoT devices and their potential
ecological impact [5, 2]. Previous studies have examined energy-efficient
communication protocols, low-power hardware design, and green data center
technologies. Additionally, researchers have proposed solutions to optimize
IoT device lifespans and reduce their carbon footprint. The work of Xue et al.
[53] on energy-efficient IoT communication protocols and Zhang et al. [59] on
sustainable IoT design are notable examples.

Additionally, advancements in green data center technologies and eco-friendly
IoT infrastructure have contributed to reducing the overall carbon footprint
of IoT deployments. These research endeavors underscore the importance of
mitigating environmental impact, a key objective that aligns closely with the
vision of our proposed One-shot Federated Learning approach, which seeks to
enhance sustainability in IoT by optimizing machine learning while minimizing
energy consumption and communication overhead. Our paper contributes to this
body of knowledge by presenting One-shot Federated Learning as a novel
approach that minimizes energy consumption during model updates and enhances
the sustainability of IoT networks.

###  2.3 One-shot Learning in FL

The hallmark of OSFL [56, 44, 10, 58] is its ability to streamline the FL
process into a single communication round, minimizing the energy expended on
data transmission and device connectivity. Traditional FL workflows require
multiple rounds of communication to exchange model updates and converge
towards a global model [36]. In contrast, OSFL condenses this multi-round
process into a one-shot operation, reducing the overall energy footprint of
IoT devices. This reduction in energy consumption is of paramount importance
in green and sustainable IoT, where power-efficient operations are essential
to mitigate environmental impact and prolong the lifespan of battery-operated
devices.

The introduction of OSFL underscores the importance of striking a balance
between sustainability and model performance in IoT applications. While
reducing energy consumption and communication overhead are critical
sustainability objectives, it is equally crucial to ensure that model quality
and learning performance are not compromised. OSFL addresses this challenge by
optimizing generative learning processes to create high-quality synthetic
updates that faithfully represent the knowledge of participating devices [23].
Through this balance, OSFL presents a compelling solution for green and
sustainable IoT, enabling energy-efficient machine learning while maintaining
or even improving the quality of learned models. Recent research has further
pushed the boundaries of FL efficiency by exploring one-shot learning
techniques. Li et al. [29] demonstrated the feasibility of one-shot learning,
enabling models to generalize from limited data instances. While these
advancements primarily targeted traditional FL, they underscored the potential
for a more streamlined and efficient FL workflow. Our work bridges the gap
between communication-efficient FL and one-shot learning, introducing OSFL as
a novel approach that synergizes these concepts to maximize sustainability in
IoT while minimizing the environmental footprint of machine learning
operations.

###  2.4 Generative Learning in FL

Generative learning has emerged as a powerful technique in machine learning,
enabling models to generate data samples that capture underlying data
distributions [11]. In the context of FL, generative learning has been applied
to enhance data privacy, as seen in the work of Nasr et al. [38] and Zhang et
al. [58] on generative models, e.g., generated adversarial networks (GANs) and
Stable Diffusion, for privacy-preserving data sharing. Furthermore, recent
research by Zhang et al. [58] demonstrated the feasibility of generative
models for one-shot learning tasks.

Generative Learning has also played a role in data augmentation for FL.
Techniques such as federated data generation, as proposed by Xin et al. [52],
leverage generative models to create synthetic data samples that help balance
class distribution and improve model robustness in FL scenarios with
imbalanced data. Furthermore, the application of generative models in one-shot
learning tasks has demonstrated significant promise. Zhang et al. [57]
explored the use of generative models for efficient adaptation to new tasks,
enabling FL models to learn from a single example. This concept has
implications for resource-efficient machine learning and can be integrated
into the FL framework to optimize knowledge transfer with minimal
communication.

In the context of our paper, we leverage the capabilities of generative
learning within the FL workflow to introduce OSFL, a novel approach that
combines generative learning and FL to maximize sustainability in IoT while
minimizing communication overhead and energy consumption. This integration
represents a pivotal advancement in the field, addressing the pressing need
for eco-friendly and efficient machine learning operations in IoT
environments. Building upon these foundations, our paper leverages generative
learning to optimize the FL workflow, reducing communication overhead and
fostering sustainable IoT by achieving learning objectives with minimal data
transfer and device energy consumption.

##  3 Preliminaries

###  3.1 Federated Learning

FL empowers decentralized participants to engage in collaborative machine
learning model training without the need to share their private data with one
another, as highlighted in [36]. In a conventional supervised FL framework,
the structure typically comprises a server denoted as $S$ and a set of $K$
clients. Each client maintains a comprehensive labeled dataset represented as
$\mathcal{D}$, while the server itself does not possess any data. This
architecture ensures data privacy and security during the collaborative
training process. Specifically, a widely used FL training strategy (i.e.
FedAvg [36]) is described as follows.

At the beginning of FL, the server initializes an ML model parameter
$\omega_{0}$ and broadcasts it to a subset of clients (i.e.,
$\mathcal{K}_{0}$, and $\mathcal{K}_{0}\subseteq\mathcal{K}$) for local
training. Afterward, for each communication round $t$, the clients in the
selected subset $\mathcal{K}_{t}$ train the received global model $\omega_{t}$
on the local dataset. For instance, the minimization objective function for
the $k$-th client training a local model $\omega_{t}^{k}$ (i.e.,
$\omega_{t}^{k}\leftarrow\omega_{t}$) on dataset
$\mathcal{D}_{k}=\\{(x_{1},y_{1}),\cdots,(x_{n_{k}},y_{n_{k}})\\}$ can be
formulated as follows.

${\ell_{k}}=\frac{1}{{{n_{k}}}}\sum\limits_{i=1}^{{n_{k}}}{{L_{i}}({y_{i}},p(f({x_{i}};{\omega^{k}}))}.$
(1)

In this context, we define the variables as follows: $n_{k}$ represents the
number of training samples held by client $k$, $\ell_{i}$ represents the loss
function applied to the $i$-th training sample, $y_{i}$ denotes the ground-
truth corresponding to $x_{i}$, and $p(y|x_{i};\omega_{t}^{k})$ signifies the
probability vector predicted by model $\omega_{t}^{k}$ for the $i$-th training
sample. Concurrently, model $\omega_{t}^{k}$ undergoes continuous updates
until the completion of the local training process by client $k$.

$\omega_{t+1}^{k}=\omega_{t}^{k}-\eta\bigtriangledown\mathcal{L}_{k}(\mathcal{D}_{k};\omega_{t}^{k}).$
(2)

In this context, $\eta$ represents the learning rate, governing the magnitude
of each model update step. Subsequently, within the subset $\mathcal{K}_{t}$,
each client uploads their locally trained model to the server for aggregation,
with the aggregation formula outlined as follows.

$\omega_{t+1}=\sum\nolimits_{k\in{\mathcal{K}_{t}}}{p_{k}}\omega_{t+1}^{k},$
(3)

where $p_{k}=\frac{n_{k}}{\sum\nolimits_{k\in{\mathcal{K}_{t}}}n_{k}}$
represents the contribution rate of the $k$-th client to the current global
model. Again, the server selects a subset of clients for the next
communication round to send down the global model $\omega_{t+1}$ until the
termination condition is reached.

###  3.2 Text-to-Image Generative Models

Text-to-Image Generative Models (T2IGMs) [13, 32, 6] signify a groundbreaking
advancement at the convergence of natural language processing and computer
vision. These models are specifically engineered to produce visually coherent
and contextually relevant images based on textual descriptions, effectively
closing the traditional gap between linguistic and visual comprehension. In
recent years, T2IGMs have witnessed notable advancements, emerging as a
formidable tool with applications ranging from content creation to augmenting
the capabilities of IoT systems [12, 16]. This section provides a
comprehensive background and definition of T2IGMs, tracing their evolution and
elucidating their profound impact across diverse domains.

T2IGMs typically consist of two primary components: an encoder-decoder
architecture and a generative model [13]. The encoder processes the textual
input, converting it into a latent representation that captures the semantics
of the text. The decoder, on the other hand, takes this latent representation
and generates a corresponding image. Often, the generative model incorporates
GAN-based techniques to ensure the generated images are both visually
convincing and semantically faithful to the input text [18, 17]. The synergy
between these components enables T2IGMs to understand and translate textual
descriptions into high-quality images. The training of a T2IGM involves a GAN-
like setup with a discriminator and generator [34]. The discriminator tries to
distinguish between real images and images generated from text, while the
generator aims to generate images that fool the discriminator. The loss
function typically involves both a generator loss ${\ell_{gen}}$ and a
discriminator loss ${\ell_{disc}}$:

$\displaystyle\mathcal{L}_{\text{gen }}=-\log(D(\text{ Generated Image }))$
(4) $\displaystyle\mathcal{L}_{\text{disc }}=-\log(D(\text{ Real Image
}))-\log(1-D(\text{ Generated Image })),$

where $D$ represents the discriminator network. The overall training objective
is to find the parameters of the generator network that minimize the generator
loss while simultaneously minimizing the discriminator loss. This is often
represented as a min-max optimization problem:

$\min_{f_{\text{generator }}}\max_{f_{\text{discriminator
}}}\left[\mathcal{L}_{\text{gen }}-\lambda\mathcal{L}_{\text{disc }}\right],$
(5)

where $\lambda$ is a hyperparameter that controls the trade-off between the
generator and discriminator losses.

The versatility of T2IGMs has led to their adoption across a wide spectrum of
domains. In the realm of art and design, T2IGMs serve as creative assistants,
turning textual prompts into stunning visual artworks. In e-commerce, they
facilitate the automatic generation of product images from textual
descriptions, streamlining the process of catalog creation [15, 54]. For
accessibility, T2IGMs have been utilized to provide visually impaired
individuals with rich, textual descriptions converted into tactile and
comprehensible images. Additionally, T2IGMs have enabled advancements in data
augmentation for computer vision tasks, allowing the generation of synthetic
training data for improved model performance.

Figure 1: An overview of the One-shot FGL system.

##  4 Methodology

###  4.1 Overview of Federated Generative Learning Framework

FGL, as shown in Figure 1, is an innovative and emerging area of research that
combines two powerful domains: FL and generative learning. FGL seeks to
harness the potential of generative models to create synthetic data while
preserving data privacy and decentralization principles in a federated
environment. Here’s an overview of the key components and concepts within
Federated Generative Learning:

  * •

Generative Models for Privacy-Preserving Data Synthesis: FGL employs a
generative model to create synthetic data samples on the server. This model
uses generation prompts sent by each client to create synthetic data samples
without centralizing them, preserving data privacy.

  * •

Edge Device Collaboration: In contrast to conventional federated learning
approaches, which typically involve the transmission of features, parameters,
or gradients, our method transmits prompts that correlate with the private
data stored on clients to the central server. This novel approach enhances
privacy preservation and communication efficiency.

  * •

Server-Side Aggregation: The central server aggregates the prompts from edge
devices, enhancing the global generative model. This one-shot aggregation
might involve techniques to balance the contributions from each device and
maintain model quality.

  * •

Synthetic Data Generation: Once a robust global generative model is
established, it can be employed to generate synthetic data samples that
closely resemble the data distribution present on edge devices.

Figure 2: Local prompt generation strategies which are proposed by Zhang et al
[58].

FGL offers numerous advantages, including enhanced data privacy, reduced
communication overhead, and the ability to create synthetic data for data
augmentation. It has applications in various fields, especially IoT, to create
synthetic IoT sensor data for model training while protecting device privacy.
It is worth noting that our method does not require multiple communications
and only requires one-shot communication to complete model training, which
greatly promotes the realization of green and sustainable IoT.

###  4.2 Local Prompt Generation

In this paper, we follow [58], and introduce Local prompt generation, as shown
in Figure 2, which serves as a fundamental pillar in the groundbreaking
paradigm of FGL. FGL seamlessly merges the principles of FL and generative
learning, offering a privacy-preserving and communication-efficient approach
to data-driven tasks. At the heart of this innovative approach lies the
concept of Local prompt generation, which empowers individual edge devices
within a federated network to contribute their private data without
compromising data privacy or centralized control.

In this context, we utilize two distinct local prompt generation strategies
[58], both of which are visually depicted in Figure 2. The first strategy is
designed to characterize classes within the local data, while the second
focuses on characterizing individual entities present in the local datasets.
These two prompt generation strategies share a common objective: to extract
essential features from the local training data, facilitating the server-side
generation model’s ability to effectively collaborate in data synthesis.
Importantly, it is essential to emphasize that these strategies do not entail
the transmission of model updates or gradients. This unique characteristic
substantially mitigates communication overhead, aligning with the core
principles of efficient and privacy-preserving FGL.

###  4.3 Training Data Synthesis

Training data synthesis constitutes a pivotal phase within the innovative
paradigm of FGL. FGL seamlessly integrates the principles of FL and generative
learning, with the overarching goal of revolutionizing the execution of data-
driven tasks while upholding data privacy and decentralized control. In the
FGL framework, training data synthesis assumes a transformative role, enabling
individual edge devices to actively contribute to the development of a global
generative model without exposing sensitive data.

Training data synthesis in FGL commences with the generation of descriptive
prompts by edge devices. These prompts encapsulate the essence of the local
data they hold, serving as abstract yet informative representations. The
prompts are designed to convey crucial insights about the data, such as class
characteristics or entity attributes, without divulging raw data details. This
synthesis of prompts aligns with the privacy-centric ethos of FL, as no actual
data is shared during this process.

Upon receiving all the prompts contributed by the edge devices, the central
server embarks on the pivotal phase of data synthesis. This process involves
the generation of every training sample, denoted as $t_{i}$, by prompting the
pre-trained generative models, e.g., Stable Diffusion model, with each prompt
$p_{i}$. We follow [58], the sequence of actions can be summarized as follows:

$\boldsymbol{t}_{i}=G\left(\boldsymbol{z}_{i},\boldsymbol{p}_{i}\right)=\sqrt{\beta}\sum_{t=1}^{T}\sqrt{1-\beta^{t}}\cdot\frac{1}{\sqrt{T}}\cdot
G_{\boldsymbol{\vartheta}_{t}}\left(\boldsymbol{z}_{i},\boldsymbol{p}_{i}\right),$
(6)

In this process, $z_{i}$ represents a randomly generated noise vector, $p_{i}$
signifies the prompt generated by edge devices, and $G_{t}(\cdot.\cdot)$
stands for the denoising network, which is parameterized with $t$ at the
specific time step $t$. The hyperparameter $\beta$ plays a pivotal role in
balancing image quality against diversity, while $T$ dictates the number of
diffusion steps to be taken. This inference process follows an iterative
pattern wherein the generated image undergoes denoising at each step before
ultimately yielding the finalized synthetic training image. Subsequently, the
central server orchestrates the generation of the synthetic training dataset,
denoted as $\mathcal{S}$, comprising pairs of training samples and
corresponding labels, represented as
$\mathcal{S}=\left\\{\left(s_{i},y_{i}\right)\right\\}_{i=1}^{N}$.

Crucially, training data synthesis in FGL operates without the exchange of
model updates or gradients, a distinctive departure from traditional FL.
Instead, it empowers edge devices to participate actively in the generative
learning process without relinquishing control over their data. This approach
not only enhances privacy preservation but also minimizes communication
overhead, a critical consideration in resource-constrained environments like
IoT.

###  4.4 One-shot Learning

One-shot learning streamlines the knowledge transfer process within FGL. In
traditional FL, multiple rounds of model updates are exchanged between edge
devices and the central server, often leading to substantial communication
overhead. In contrast, one-shot learning condenses this into a singular
communication round, dramatically reducing the amount of data transferred. The
core objective of one-shot updating is to enrich the global generative model
with collective insights from edge devices. These insights are encapsulated in
the prompts generated by edge devices, which offer descriptive information
about their local datasets. The central server leverages this valuable input
to enhance the capabilities of the generative model. Following the acquisition
of the synthetic training set denoted as
$\mathcal{S}=\left\\{\left(s_{i},y_{i}\right)\right\\}_{i=1}^{N}$, our
subsequent step involves the joint training of the model on the central
server. Specifically, we employ the AlexNet architecture as the model of
choice and utilize the cross-entropy loss function.

$\mathcal{L}=-\frac{1}{N}\sum_{i=1}^{N}y_{i}\log\left(\hat{y}_{i}\right)+\left(1-y_{i}\right)\log\left(1-\hat{y}_{i}\right)$
(7)

The efficiency of one-shot updating contributes to the robustness and
sustainability of FGL. It allows for the seamless integration of generative
learning techniques into IoT and other data-sensitive domains, where resource-
efficient machine learning is essential. Therefore, one-shot updating in FGL
embodies the essence of efficient and privacy-conscious knowledge transfer. It
empowers edge devices to contribute effectively to the generative learning
process while preserving data privacy and minimizing communication overhead.

###  4.5 Complexity Analysis

Here, we focus on analyzing the algorithm complexity on the client side. It
can be seen from the above algorithm that on the client side, the client needs
to generate local prompts and upload the generated prompts. For the sake of
simplicity, we let the complexity of generating a prompt be $C$ (depending on
the specific generation algorithm), then the algorithm complexity of clients
performing local prompt generation operations is $O(KCN)$. In addition,
considering that the upload operation can be completed in one go, the
complexity is $O(N)$.

/* \-- Local Class-level Prompt Generation -- */

/* \-- Step 1: Reading Local Data -- */

1 do

2 read data class from local clients in the form of training samples

3 get a new data class and add it to a list of unseen data class

4

5while  _complete step 1_

6while  _local prompt generation_ do

/* \-- Step 2: Marking the class -- */

7 Client to mark the number of classes

/* \-- Step 3: Generating the prompt -- */

8 Client uses natural language to describe the tagged class

9

10 end while

/* \-- Step 4: Uploading generated prompts -- */

11 upload the generated prompts to the server

12

/* \-- Local Entity-leve Prompt Generation -- */

/* \-- Step 1: Reading Local Data -- */

13 do

14 read data from local clients in the form of training samples

15 get a new data and add it to a list of unseen data

16

17while  _complete step 1_

18while  _local prompt generation_ do

/* \-- Step 2: Leveraging the model -- */

19 Client uses BLIPv2 model to exact data information

/* \-- Step 3: Generating the prompt -- */

20 Client uses BLIPv2 model to describe the tagged sample

21

22 end while

/* \-- Step 4: Uploading generated prompts -- */

23 upload the generated prompts to the server

Algorithm 1 Description of the steps of the local prompt generation algorithm.

/* \-- FGL Algorithm -- */

/* \-- Step 1: Initialize the Network -- */

1 prompts_list = []

2 global_model = $\omega_{0}$

/* \-- Step 2: Local Prompt Generation -- */

3 for  _client in all_clients_ do

4 Uses class-level prompts or instance-level prompts via Algo. 1

5 prompts_client = Algorithm_1(client)

6 prompts_list.append(prompt)

/* \-- Step 3: Local Prompt Aggregation -- */

7 prompts = prompts_aggregation(prompts_list)

/* \-- Step 4: Training Data Synthesis -- */

8 synthetic_dataloader = generative_model(prompts, noises)

/* \-- Step 5: One-shot Learniing -- */

9 Trains the global model at server side

10 server_update(synthetic_dataloader, global_model)

11

12 end for

Algorithm 2 Federated generative learning algorithm.

##  5 Experomental Results

###  5.1 Experiment Setup

The experiments are conducted within a uniform computing environment,
comprising Linux Ubuntu 10.04, an Intel i5-4210M CPU, 32GB RAM, and a 1024GB
SSD. The implementation is facilitated through the utilization of the Pytorch
and Foundation Model libraries.

Datasets. We adopt three image datasets for evaluations, i.e., Fashion-MNIST
[51], CIFAR-10 [27], and CIFAR-100 [27]. The datasets cover different
attributes, dimensions, and a number of categories, allowing us to explore the
poisoning effectiveness of FGL.

Fashion-MNIST. Fashion-MNIST is a popular dataset in the field of machine
learning and computer vision. It serves as an alternative to the traditional
MNIST dataset but offers more challenging and diverse tasks. F-MNIST consists
of 28x28 grayscale images of fashion items, such as clothing and accessories,
categorized into ten different classes. These classes include items like
T-shirts, dresses, sneakers, handbags, and more. F-MNIST is often used for
benchmarking and evaluating machine learning algorithms and models, especially
for tasks related to image classification, feature extraction, and deep
learning.

CIFAR-10. The CIFAR-10 dataset is another well-known dataset in the field of
computer vision. It contains 60,000 color images, divided into ten classes,
with each class representing a different object category. Each image in
CIFAR-10 has a resolution of 32x32 pixels and is labeled with one of the
following categories: airplane, automobile, bird, cat, deer, dog, frog, horse,
ship, or truck. CIFAR-10 is widely used for research in image classification,
object recognition, and deep learning due to its diversity and complexity.

CIFAR-100. CIFAR-100 is an extension of the CIFAR-10 dataset and is designed
for more fine-grained image classification tasks. It also consists of 60,000
images, but this time they are divided into 100 different classes. Each class
represents a more specific object category, making CIFAR-100 suitable for
tasks where distinguishing between closely related objects is required. The
dataset includes objects like different types of birds, insects, and vehicles.
CIFAR-100 is commonly used in research for tasks involving fine-grained object
recognition, hierarchical classification, and model evaluation.

Data Partition. In FGL, data partitioning plays a pivotal role in shaping the
training process. Setting up IID (Independent and Identically Distributed)
data involves ensuring that each edge device’s data is representative of the
entire dataset and follows the same data distribution. To achieve this, the
dataset is divided equally or approximately equally among the edge devices,
ensuring that each device receives a balanced representation of data classes
and characteristics. On the other hand, setting up non-IID (Non-Independent
and Non-Identically Distributed) data in FGL involves recognizing the
diversity of data among edge devices. Each device may possess a unique data
distribution (we employ the Dirichlet distribution [28]), varying class
proportions, or distinct data characteristics.

Models. For our experimental setup, we employ a straightforward deep learning
model, specifically a Convolutional Neural Network (CNN) comprising two
convolutional layers, succeeded by a single fully connected layer. This
architecture is utilized for classification tasks involving the F-MNIST
datasets. In contrast, for classification tasks related to the CIFAR-10 and
CIFAR-100 datasets, we utilize the AlexNet [22] model.

Hyperparameters. We configure our experiments with the following parameters:
$m=100$ clients, client participation proportion $\mathcal{P}=0.1$, $T=250$
training rounds, $E=5$ local training epochs, learning rate $\eta=0.001$,
mini-batch size $B=32$, and $N_{s}=10,000$ training samples on the server-
side. The symmetric quantization method employs an optimal threshold of
$\alpha=0.5$. In the context of FGL, we adopt a hyperparameter value of
$\beta=0.95$ . Additionally, our model optimization utilizes SGD with momentum
applied to mini-batches

Baselines. In our experimental setup, we establish baseline methods,
specifically the widely adopted FedAvg approach proposed by McMahan et al.
[36], and traditional centralized training for comparison. In this scenario,
we consider a total of 5 clients, all actively participating in the
communication process. For both centralized training and federated learning,
we configure the local learning rate to be 0.01 and employ the SGD optimizer
with a momentum of 0.9. During the FedAvg training phase, individual clients
conduct local updates for 5 epochs, and the communication round is set to span
200 iterations. On the other hand, centralized training entails a total of 120
rounds of iterations as part of its training regimen.

###  5.2 Numerical Results

Table 2: System performance under the IID setting.

Datasets | F-MNIST | CIFAR-10 | CIFAR-100
---|---|---|---
| Train
---
acc (%)
| Test
---
acc (%)
Acc $\uparrow$ |  | Train
---
acc (%)
| Test
---
acc (%)
Acc $\uparrow$ |  | Train
---
acc (%)
| Test
---
acc (%)
Acc $\uparrow$
Centralized | 96.47 | 95.42 | 95.42 | 74.65 | 67.45 | 67.45 | 45.65 | 34.15 | 34.15
FedAvg [36] | 95.10 | 94.88 | 94.88 | 72.99 | 64.20 | 64.20 | 42.85 | 32.11 | 32.11
FGL (Ours) | 98.58 | 98.26 | 98.26 | 77.84 | 70.85 | 70.85 | 58.67 | 52.41 | 52.41

Table 3: System performance under the non-IID setting.

Datasets | F-MNIST | CIFAR-10 | CIFAR-100
---|---|---|---
| Train
---
acc (%)
| Test
---
acc (%)
Acc $\uparrow$ |  | Train
---
acc (%)
| Test
---
acc (%)
Acc $\uparrow$ |  | Train
---
acc (%)
| Test
---
acc (%)
Acc $\uparrow$
Centralized | 96.47 | 95.42 | 95.42 | 74.65 | 67.45 | 67.45 | 45.65 | 34.15 | 34.15
FedAvg [36] | 89.24 | 82.54 | 82.54 | 61.25 | 53.61 | 53.61 | 38.98 | 28.88 | 28.88
FGL (Ours) | 97.24 | 95.47 | 95.47 | 67.91 | 64.33 | 64.33 | 41.25 | 38.36 | 38.36

Performance Evaluation. In this section, we first evaluate the system
performance of the proposed FGL under IID and non-IID data settings. As shown
in Table 2 and Table 3, we can find that the system performance of FGL on the
F-MNIST and CIFAR-10 data sets is better than the baseline schemes.
Specifically, on the CIFAR-10 dataset, the performance of FGL is 5.67% higher
than that of FedAvg. In addition, we can also find that the system performance
of FGL is comparable to the baselines on the CIFAR-100 data set. After
thorough research and reflection, we can get the following lessons:

(Takeaway-1) 1) FGL’s system performance is generally better than the baseline
and does not require multiple rounds of communication, thanks to the powerful
learning ability of the basic model. 2) Non-IID data setting does not
significantly affect the system performance of FGL. 3) The OSFL method does
not affect system performance.

Communication Efficiency Evaluation. Second, we aim to explore the
communication overhead between FGL and baselines. Since FGL utilizes one shot
learning and updating methods, we expect that the communication overhead of
FGL is much smaller than that of baselines. As shown in Figures 3 and 4, we
find that the communication overhead of FGL under IID and non-IID settings is
approximately 1/5 of the FedAvg scheme. Note that since centralized training
does not involve communication protocols it is not compared here. Furthermore,
we note that the running time of FGL is approximately 1/2 of the baseline
solutions, as shown in Figure 7 and Figure 6. This means that FGL has
significant advantages in communication efficiency and operating efficiency,
and is especially suitable for resource-constrained IoT environments. After
thorough research and reflection, we can get the following lessons:

Figure 3: Communication efficiency evaluation on IID setting. Figure 4:
Communication efficiency evaluation on non-IID setting. Figure 5: Running time
reults on IID setting. Figure 6: Running time reults on non-IID setting.

(Takeaway-2) 1) FGL benefits from the one-shot learning and updating method,
which greatly reduces its communication overhead. 2) Since FGL no longer
requires local training, its running efficiency has also been accelerated. 3)
In resource-constrained IoT environments, FGL can achieve a perfect trade-off
between model utility and resource energy consumption.

Hyperparameter Evaluation under Number of Synthetic Data. Third, we explore
the impact of different amounts of synthetic data on system performance.
Specifically, we set the number of synthetic data to $N=\\{2k,5k,10k\\}$, and
record the system performance of FGL in these three cases. Table 4 reports the
performance results of FGL, and we can find that as the amount of synthetic
data increases, the performance of FGL continues to improve. However, this
phenomenon cannot continue indefinitely. When the performance limit is
reached, the increase in the amount of synthetic data is no longer beneficial
to model training. After thorough research and reflection, we can get the
following lessons:

Table 4: System performance under the number of synthetic data.

# of Synthetic Data | 2k | 5k | 10k
---|---|---|---
| Train
---
acc (%)
| Test
---
acc (%)
Acc $\uparrow$ |  | Train
---
acc (%)
| Test
---
acc (%)
Acc $\uparrow$ |  | Train
---
acc (%)
| Test
---
acc (%)
Acc $\uparrow$
FGL (Ours) | 77.84 | 70.85 | 70.85 | 85.44 | 82.54 | 82.54 | 89.22 | 87.24 | 87.24

(Takeaway-3) 1) The number of synthetic data has an impact on the performance
of FGL, and within a certain range, the larger the number, the more beneficial
it is for model training. 2) Increasing the amount of synthetic data is a
simple and economical way to improve performance.

Visual Analysis of FGL’s Generalization Ability. As shown in Fig. 7, we
visualized the loss landscape of FGL and naive FL and found that the FGL
framework not only has a more balanced optimization curve, but also has a
lower loss value and is easier to obtain the optimal value. This means that
although FGL is one-shot communication, it still has good convergence and
generalization capabilities.

Performance Evaluation under Different Prompt Methods. Finally, we explore the
impact of different prompt methods on FGL system performance. As mentioned
above, this article designs two local prompt strategies. We evaluate these
strategies by comparing the system performance under different prompt
strategies. Table 5 reports the results. We found that using a more precise
entity-level prompt strategy leads to higher accuracy, for example, reaching
68.62% on CIFAR-10 compared with the accuracy of the class-level prompt
strategy. This result clearly highlights the importance of considering more
detailed entity-level information in prompt strategy design. However, the
entity-level prompt policy involves specific real image information, which may
cause potential privacy issues, and we leave it as future work.

Figure 7: (a) Visualization of loss landscape when applying FGL. (b)
Visualization of the loss landscape when the FGL is not applied. Table 5:
System performance under different prompt methods.

Datasets | F-MNIST | CIFAR-10 | CIFAR-100
---|---|---|---
| Train
---
acc (%)
| Test
---
acc (%)
Acc $\uparrow$ |  | Train
---
acc (%)
| Test
---
acc (%)
Acc $\uparrow$ |  | Train
---
acc (%)
| Test
---
acc (%)
Acc $\uparrow$
Class-level | 98.58 | 98.26 | 98.26 | 77.84 | 70.85 | 70.85 | 58.67 | 52.41 | 52.41
Entity-level | 99.62 | 99.24 | 99.24 | 82.41 | 78.62 | 78.62 | 61.36 | 59.24 | 59.24

(Takeaway-4) 1) Both of the designed prompt strategies can effectively achieve
the established goals. 2) The effect of entity-level prompt strategy is better
than that of class-level prompt strategy. 3) Although the effect of the
entity-level prompt policy is better than the class-level prompt policy, the
entity-level prompt policy has certain privacy concerns.

###  5.3 Discussion

Privacy has been a paramount concern in IoT ecosystems, and our One-shot FGL
approach offers a pragmatic solution. By allowing edge devices to share
abstract prompts instead of raw data, we strike a balance between data utility
and privacy preservation. This design mitigates the risk of data breaches and
unauthorized access while enabling the collaborative generation of synthetic
data. The localization of data aggregation further fortifies privacy, as
prompts, not individual data samples, are aggregated. The privacy-centric
nature of FGL aligns with the ethical use of data and user expectations for
data confidentiality in the IoT era.

Sustainability is at the core of our approach. By reducing the need for
extensive data transmission and centralization, FGL contributes to the
ecological sustainability of IoT systems. The conservation of energy and
bandwidth resources not only supports environmental goals but also enhances
the operational longevity of IoT devices. FGL’s ability to generate high-
quality synthetic data from decentralized sources promotes sustainable
practices by minimizing the environmental footprint of data transfer and
storage. This integration of sustainability and privacy-consciousness
positions FGL as a pioneering approach in building responsible and eco-
friendly IoT ecosystems.

While our approach shows promise, several challenges and future directions
warrant consideration. One key challenge is optimizing the generative model’s
performance while working with decentralized and diverse data sources.
Developing advanced generative techniques that can adapt to various data
distributions across edge devices is essential. Additionally, research in
federated learning orchestration and model aggregation techniques needs
further exploration to maximize the utility of FGL.

Finally, it is crucial to highlight that the proposed framework exhibits
flexibility and can be readily extended to various tasks. The adaptability
stems from the ability to employ different generation models for prompt
generation corresponding to diverse task data. For instance, the VisualGLM
model can be employed to generate prompts and subsequently facilitate learning
from video data. The scalability of the framework across different tasks is
confirmed, underscoring the importance of task scalability in a learning
framework.

##  6 Conclusions

In this paper, we have introduced and delved into the concept of OSFL as an
innovative approach aimed at addressing the dual imperatives of sustainability
and privacy within the realm of the IoT. OSFL, characterized by its reduction
of communication rounds and optimization for energy efficiency through
generative learning, holds significant promise for reshaping the landscape of
machine learning in IoT ecosystems. Our research has successfully demonstrated
the feasibility and effectiveness of OSFL in achieving sustainable IoT
operations. By condensing the traditional federated learning process into a
single communication round, OSFL substantially mitigates energy consumption,
communication overhead, and latency. Moreover, the integration of generative
learning techniques ensures the preservation of data privacy, even as
knowledge is efficiently shared among IoT devices. OSFL stands as a novel and
environmentally conscious paradigm that aligns with the global imperative to
reduce the environmental footprint of technology while facilitating advanced
machine learning capabilities.

Looking ahead, the potential applications of OSFL in green and sustainable IoT
are expansive. From optimizing energy consumption in smart cities to enhancing
environmental monitoring in remote regions, OSFL can empower IoT devices to
learn and collaborate effectively while minimizing their environmental impact.
Furthermore, OSFL’s ability to balance sustainability and model performance
opens doors to new opportunities in domains such as healthcare, agriculture,
and industrial automation, where resource-efficient machine learning is
crucial. In conclusion, OSFL not only underscores our commitment to
responsible and sustainable technology development but also offers a
transformative pathway to a greener and more intelligent IoT ecosystem.

## Acknowledgment

The authors extend their appreciation to the Deputyship for Research and
Innovation, Ministry of Education in Saudi Arabia, for funding this research
through (IFKSURC-1-0306). Here, special thanks to Dr. Jie Zhang from ETH for
the open-source federated generative learning code, which is the core basis of
the work in this paper. We borrowed his code, and method as the cornerstone to
develop the proposed green and, sustainable FL framework.

## References

  * Alagheband and Mashatan [2022] Alagheband, M.R., Mashatan, A., 2022\.  Advanced digital signatures for preserving privacy and trust management in hierarchical heterogeneous iot: Taxonomy, capabilities, and objectives.  Internet of Things 18, 100492\.
  * Ali and Ali [2021] Ali, Z.H., Ali, H.A., 2021.  Towards sustainable smart iot applications architectural elements and design: opportunities, challenges, and open directions.  The Journal of Supercomputing 77, 5668–5725.
  * Amodu et al. [2023] Amodu, O.A., Bukar, U.A., Mahmood, R.A.R., Jarray, C., Othman, M., 2023.  Age of information minimization in uav-aided data collection for wsn and iot applications: A systematic review.  Journal of Network and Computer Applications , 103652\.
  * Belenguer et al. [2023] Belenguer, A., Pascual, J.A., Navaridas, J., 2023.  Göwfed: A novel federated network intrusion detection system.  Journal of Network and Computer Applications , 103653\.
  * Bibri [2018] Bibri, S.E., 2018.  The iot for smart sustainable cities of the future: An analytical framework for sensor-based big data applications for environmental sustainability.  Sustainable cities and society 38, 230–253.
  * Bird et al. [2023] Bird, C., Ungless, E., Kasirzadeh, A., 2023.  Typology of risks of generative text-to-image models, in: Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, pp. 396–410.
  * Bonawitz et al. [2019] Bonawitz, K., Eichner, H., Grieskamp, W., Huba, D., Ingerman, A., Ivanov, V., Kiddon, C., Konečnỳ, J., Mazzocchi, S., McMahan, B., et al., 2019\.  Towards federated learning at scale: System design.  Proceedings of machine learning and systems 1, 374–388.
  * Bonawitz et al. [2016] Bonawitz, K., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H.B., Patel, S., Ramage, D., Segal, A., Seth, K., 2016.  Practical secure aggregation for federated learning on user-held data.  arXiv preprint arXiv:1611.04482 .
  * Chen et al. [2021] Chen, M., Shlezinger, N., Poor, H.V., Eldar, Y.C., Cui, S., 2021.  Communication-efficient federated learning.  Proceedings of the National Academy of Sciences 118, e2024789118.
  * Chiang et al. [2023] Chiang, Y.H., Terai, K., Chiang, T.W., Lin, H., Ji, Y., Lui, J.C., 2023.  Optimal transport based one-shot federated learning for artificial intelligence of things.  IEEE Internet of Things Journal .
  * Creswell et al. [2018] Creswell, A., White, T., Dumoulin, V., Arulkumaran, K., Sengupta, B., Bharath, A.A., 2018\.  Generative adversarial networks: An overview.  IEEE signal processing magazine 35, 53–65.
  * Deckers et al. [2023] Deckers, N., Fröbe, M., Kiesel, J., Pandolfo, G., Schröder, C., Stein, B., Potthast, M., 2023.  The infinite index: Information retrieval on generative text-to-image models, in: Proceedings of the 2023 Conference on Human Information Interaction and Retrieval, pp. 172–186.
  * Ding et al. [2021] Ding, M., Yang, Z., Hong, W., Zheng, W., Zhou, C., Yin, D., Lin, J., Zou, X., Shao, Z., Yang, H., et al., 2021\.  Cogview: Mastering text-to-image generation via transformers.  Advances in Neural Information Processing Systems 34, 19822–19835.
  * Du et al. [2023] Du, H., Zhang, R., Liu, Y., Wang, J., Lin, Y., Li, Z., Niyato, D., Kang, J., Xiong, Z., Cui, S., et al., 2023.  Beyond deep reinforcement learning: A tutorial on generative diffusion models in network optimization.  arXiv preprint arXiv:2308.05384 .
  * Fan et al. [2022] Fan, X., Zhang, C., Yang, Y., Shang, Y., Zhang, X., He, Z., Xiao, Y., Long, B., Wu, L., 2022.  Automatic generation of product-image sequence in e-commerce, in: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2851–2859.
  * Ferdowsi and Saad [2019] Ferdowsi, A., Saad, W., 2019\.  Generative adversarial networks for distributed intrusion detection in the internet of things, in: 2019 IEEE Global Communications Conference (GLOBECOM), IEEE. pp. 1–6.
  * Goodfellow et al. [2014] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y., 2014\.  Generative adversarial nets.  Advances in neural information processing systems 27\.
  * Goodfellow et al. [2020] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y., 2020\.  Generative adversarial networks.  Communications of the ACM 63, 139–144.
  * Guo et al. [2023] Guo, T., Guo, S., Wang, J., Tang, X., Xu, W., 2023\.  Promptfl: Let federated participants cooperatively learn prompts instead of models-federated learning in age of foundation model.  IEEE Transactions on Mobile Computing .
  * Hajian et al. [2022] Hajian, R., Haghighat, A., Erfani, S.H., 2022.  A secure anonymous d2d mutual authentication and key agreement protocol for iot.  Internet of Things 18, 100493\.
  * Heinbaugh et al. [2022] Heinbaugh, C.E., Luz-Ricca, E., Shao, H., 2022.  Data-free one-shot federated learning under very high statistical heterogeneity, in: The Eleventh International Conference on Learning Representations.
  * Iandola et al. [2016] Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., Keutzer, K., 2016\.  Squeezenet: Alexnet-level accuracy with 50x fewer parameters and< 0.5 mb model size.  arXiv preprint arXiv:1602.07360 .
  * Jhunjhunwala et al. [2023] Jhunjhunwala, D., Wang, S., Joshi, G., 2023.  Towards a theoretical and practical understanding of one-shot federated learning with fisher information, in: Federated Learning and Analytics in Practice: Algorithms, Systems, Applications, and Opportunities.
  * Kang et al. [2022] Kang, J., Li, X., Nie, J., Liu, Y., Xu, M., Xiong, Z., Niyato, D., Yan, Q., 2022.  Communication-efficient and cross-chain empowered federated learning for artificial intelligence of things.  IEEE Transactions on Network Science and Engineering 9, 2966–2977.
  * Kang et al. [2020] Kang, J., Xiong, Z., Jiang, C., Liu, Y., Guo, S., Zhang, Y., Niyato, D., Leung, C., Miao, C., 2020\.  Scalable and communication-efficient decentralized federated edge learning with multi-blockchain framework, in: Blockchain and Trustworthy Systems: Second International Conference, BlockSys 2020, Dali, China, August 6–7, 2020, Revised Selected Papers 2, Springer. pp. 152–165.
  * Kar et al. [2023] Kar, B., Lin, Y.D., Lai, Y.C., 2023.  Cost optimization of omnidirectional offloading in two-tier cloud–edge federated systems.  Journal of Network and Computer Applications 215, 103630.
  * Krizhevsky et al. [2009] Krizhevsky, A., Hinton, G., et al., 2009\.  Learning multiple layers of features from tiny images .
  * Li et al. [2022] Li, Q., Diao, Y., Chen, Q., He, B., 2022.  Federated learning on non-iid data silos: An experimental study, in: 2022 IEEE 38th International Conference on Data Engineering (ICDE), IEEE. pp. 965–978.
  * Li et al. [2020a] Li, Q., He, B., Song, D., 2020a.  Practical one-shot federated learning for cross-silo setting.  arXiv preprint arXiv:2010.01017 .
  * Li et al. [2020b] Li, T., Sahu, A.K., Talwalkar, A., Smith, V., 2020b.  Federated learning: Challenges, methods, and future directions.  IEEE signal processing magazine 37, 50–60.
  * Li et al. [2021] Li, Y., Li, J., Wang, Y., 2021\.  Privacy-preserving spatiotemporal scenario generation of renewable energies: A federated deep generative learning approach.  IEEE Transactions on Industrial Informatics 18, 2310–2320.
  * Liu et al. [2020a] Liu, Y., Garg, S., Nie, J., Zhang, Y., Xiong, Z., Kang, J., Hossain, M.S., 2020a.  Deep anomaly detection for time-series data in industrial iot: A communication-efficient on-device federated learning approach.  IEEE Internet of Things Journal 8, 6348–6358.
  * Liu et al. [2020b] Liu, Y., James, J., Kang, J., Niyato, D., Zhang, S., 2020b.  Privacy-preserving traffic flow prediction: A federated learning approach.  IEEE Internet of Things Journal 7, 7751–7763.
  * Liu et al. [2019] Liu, Y., Peng, J., James, J., Wu, Y., 2019.  Ppgan: Privacy-preserving generative adversarial network, in: 2019 IEEE 25Th international conference on parallel and distributed systems (ICPADS), IEEE. pp. 985–989.
  * Liu et al. [2022] Liu, Y., Xu, L., Yuan, X., Wang, C., Li, B., 2022\.  The right to be forgotten in federated learning: An efficient realization with rapid retraining, in: IEEE INFOCOM 2022-IEEE Conference on Computer Communications, IEEE. pp. 1749–1758.
  * McMahan et al. [2017] McMahan, B., Moore, E., Ramage, D., Hampson, S., y Arcas, B.A., 2017.  Communication-efficient learning of deep networks from decentralized data, in: Proc. of AISTATS.
  * Mohammed et al. [2023] Mohammed, M.A., Lakhan, A., Abdulkareem, K.H., Zebari, D.A., Nedoma, J., Martinek, R., Kadry, S., Garcia-Zapirain, B., 2023\.  Homomorphic federated learning schemes enabled pedestrian and vehicle detection system.  Internet of Things 23, 100903\.
  * Nasr et al. [2019] Nasr, M., Shokri, R., Houmansadr, A., 2019.  Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning, in: 2019 IEEE symposium on security and privacy (SP), IEEE. pp. 739–753.
  * Pathak and Wainwright [2020] Pathak, R., Wainwright, M.J., 2020\.  Fedsplit: An algorithmic framework for fast federated optimization.  Advances in neural information processing systems 33, 7057–7066.
  * Peng et al. [2023] Peng, C., Guo, Y., Chen, Y., Rui, Q., Yang, Z., Xu, C., 2023.  Fedgm: Heterogeneous federated learning via generative learning and mutual distillation, in: European Conference on Parallel Processing, Springer. pp. 339–351.
  * Qin et al. [2023] Qin, H., Chen, W., Li, N., Wang, T., Chen, H., Yang, G., Peng, Y., 2023.  Cps: Cross-interface network partitioning and scheduling towards qos-aware data flow delivery in multimedia iot.  Journal of Network and Computer Applications 217, 103698.
  * Reisizadeh et al. [2020] Reisizadeh, A., Mokhtari, A., Hassani, H., Jadbabaie, A., Pedarsani, R., 2020.  Fedpaq: A communication-efficient federated learning method with periodic averaging and quantization, in: International Conference on Artificial Intelligence and Statistics, PMLR. pp. 2021–2031.
  * Sabovic et al. [2023] Sabovic, A., Aernouts, M., Subotic, D., Fontaine, J., De Poorter, E., Famaey, J., 2023\.  Towards energy-aware tinyml on battery-less iot devices.  Internet of Things 22, 100736\.
  * Salehkaleybar et al. [2021] Salehkaleybar, S., Sharifnassab, A., Golestani, S.J., 2021.  One-shot federated learning: theoretical limits and algorithms to achieve them.  The Journal of Machine Learning Research 22, 8485–8531.
  * Sattler et al. [2019] Sattler, F., Wiedemann, S., Müller, K.R., Samek, W., 2019\.  Robust and communication-efficient federated learning from non-iid data.  IEEE transactions on neural networks and learning systems 31, 3400–3413.
  * Shenoy et al. [2023] Shenoy, M.V., et al., 2023.  Hfeddi: A novel privacy preserving horizontal federated learning based scheme for iot device identification.  Journal of Network and Computer Applications 214, 103616.
  * Triastcyn and Faltings [2020] Triastcyn, A., Faltings, B., 2020\.  Federated generative privacy.  IEEE Intelligent Systems 35, 50–57.
  * Wang et al. [2020] Wang, J., Liu, Q., Liang, H., Joshi, G., Poor, H.V., 2020\.  Tackling the objective inconsistency problem in heterogeneous federated optimization.  Advances in neural information processing systems 33, 7611–7623.
  * Wang et al. [2023] Wang, J., Yi, W., Yang, M., Ma, J., Zhang, S., Hao, S., 2023.  Enhance the trust between iot devices, mobile apps, and the cloud based on blockchain.  Journal of Network and Computer Applications 218, 103718.
  * Wei et al. [2020] Wei, K., Li, J., Ding, M., Ma, C., Yang, H.H., Farokhi, F., Jin, S., Quek, T.Q., Poor, H.V., 2020\.  Federated learning with differential privacy: Algorithms and performance analysis.  IEEE Transactions on Information Forensics and Security 15, 3454–3469.
  * Xiao et al. [2017] Xiao, H., Rasul, K., Vollgraf, R., 2017.  Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms.  arXiv preprint arXiv:1708.07747 .
  * Xin et al. [2022] Xin, B., Geng, Y., Hu, T., Chen, S., Yang, W., Wang, S., Huang, L., 2022\.  Federated synthetic data generation with differential privacy.  Neurocomputing 468, 1–10.
  * Xue et al. [2023] Xue, X., Shanmugam, R., Palanisamy, S., Khalaf, O.I., Selvaraj, D., Abdulsahib, G.M., 2023\.  A hybrid cross layer with harris-hawk-optimization-based efficient routing for wireless sensor networks.  Symmetry 15, 438\.
  * Yang et al. [2016] Yang, X., Mei, T., Xu, Y.Q., Rui, Y., Li, S., 2016\.  Automatic generation of visual-textual presentation layout.  ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 12, 1–22.
  * Ye et al. [2022] Ye, H., Liu, J., Zhen, H., Jiang, W., Wang, B., Wang, W., 2022.  Vrefl: Verifiable and reconnection-efficient federated learning in iot scenarios.  Journal of Network and Computer Applications 207, 103486.
  * Zhang et al. [2022a] Zhang, J., Chen, C., Li, B., Lyu, L., Wu, S., Ding, S., Shen, C., Wu, C., 2022a.  Dense: Data-free one-shot federated learning.  Advances in Neural Information Processing Systems 35, 21414–21428.
  * Zhang et al. [2023a] Zhang, J., Guo, S., Guo, J., Zeng, D., Zhou, J., Zomaya, A., 2023a.  Towards data-independent knowledge transfer in model-heterogeneous federated learning.  IEEE Transactions on Computers .
  * Zhang et al. [2023b] Zhang, J., Qi, X., Zhao, B., 2023b.  Federated generative learning with foundation models.  arXiv preprint arXiv:2306.16064 .
  * Zhang et al. [2022b] Zhang, Z., Wen, F., Sun, Z., Guo, X., He, T., Lee, C., 2022b.  Artificial intelligence-enabled sensing technologies in the 5g/internet of things era: from virtual reality/augmented reality to the digital twin.  Advanced Intelligent Systems 4, 2100228\.
  * Zhou et al. [2021] Zhou, X., Liang, W., She, J., Yan, Z., Kevin, I., Wang, K., 2021.  Two-layer federated learning with heterogeneous model aggregation for 6g supported internet of vehicles.  IEEE Transactions on Vehicular Technology 70, 5308–5317.