# Distributed Artificial Intelligence Solution for D2D Communication in 5G
Networks ††thanks: This research is part of a project that has received
funding from the European Union’s Horizon 2020 research and innovation
programme under grant agreement Nº739578 and the government of the Republic of
Cyprus through the Directorate General for European Programmes, Coordination
and Development.

Iacovos Ioannou, Vasos Vassiliou, Christophoros Christophorou, and Andreas
Pitsillides
Department of Computer Science, University of Cyprus and
RISE Center of Excellence on Interactive Media, Smart Systems and Emerging
Technologies
Nicosia, Cyprus

###### Abstract

Device to Device (D2D) Communication is one of the technology components of
the evolving 5G architecture, as it promises improvements in energy
efficiency, spectral efficiency, overall system capacity, and higher data
rates. The above noted improvements in network performance spearheaded a vast
amount of research in D2D, which have identified significant challenges that
need to be addressed before realizing their full potential in emerging 5G
Networks. Towards this end, this paper proposes the use of a distributed
intelligent approach to control the generation of D2D networks. More
precisely, the proposed approach uses Belief-Desire-Intention (BDI)
intelligent agents with extended capabilities (BDIx) to manage each D2D node
independently and autonomously, without the help of the Base Station. This
paper proposes the DAIS algorithm for the decision of transmission mode in
D2D, which maximizes the data rate and minimizes the power consumption, while
taking into consideration the computational load. Simulations show the
applicability of BDI agents in solving D2D challenges.

###### Index Terms:

5G, D2D, D2D challenges, Artificial Intelligence, BDI Agents, Distributed
Artificial Intelligence, Multi-Agent Systems

##  I Introduction

Device to Device (D2D) Communication is expected to be a core part of the
forthcoming 5G mobile communication networks. D2D can operate both in the
licensed and unlicensed spectrum and is generally transparent to the cellular
network as it allows adjacent user equipment (UE) to bypass the base station
(BS) and establish direct links between them, to either share their connection
and act as relay stations, or directly communicate and exchange information.
D2D can be used to implement many of the 5G requirements, because it can
support high bit rates and minimize the delay between D2D UEs. The gains of
D2D communications in spectral efficiency, resource reallocation, and
reduction of interference [1, 2] can potentially improve throughput, energy
efficiency, delay, and fairness [3, 4]. In addition, due to the shorter
communication distance, D2D can offer lower power consumption for the
communicating D2D devices. D2D can enable mobile traffic offloading, so
overall one can anticipate that the non-D2D UEs can also benefit from the
mobile traffic offloading because they will, as a result, have access to more
bandwidth for the communication between them (non-D2D UEs) and the BS, as well
as less interference [3, 4]. However, in order to fully realize D2D, several
challenges need to be resolved, including device discovery, mode selection,
interference management, power control, security, radio resource allocation,
cell densification & offloading, Quality of Service (QoS) & path selection,
use of mmWave communication, non-cooperative users, and handover management
[24, 34, 35].

This work investigates the idea that the D2D communication is not a global
problem that must be solved centrally, but it is an optimization problem that
should be solved in a distributed fashion with the use of artificial
intelligence. To address that, the paper proposes that the control is handled
by the UEs, locally, in order to form communication links in shorter time [5,
6, 7, 8, 9, 10, 11, 12, 13]. We consider that the use of distributed
artificial intelligence (AI) control is the most suitable in the challenging
and dynamic environment of D2D communication. To the best of our knowledge,
there are no solutions in the literature that jointly satisfy all of the D2D
requirements in one approach. We chose intelligent agents because of their
ability to concurrently solve multiple complex problems, as it was shown in
[38].

In this paper we are making the following contributions:

  1. (a)

we propose a solution using Belief-Desire-Intention (BDI) software agents with
extended capabilities (BDIx), to collectively satisfy the challenges
identified for D2D communication,

  2. (b)

we provide a proof-of-concept algorithm that encompasses the use of
intelligent agents for selecting the D2D transmission mode, while ensuring a
high spectral efficiency and low computational load,

  3. (c)

we propose the use of a new parameter called Weighted Data Rate (WDR) for the
decision of D2D transmission mode, and

  4. (d)

we evaluate the proposed solution under varying scenarios and provide insights
into its operation.

The rest of the paper is structured as follows. Section II provides background
information on D2D communications and intelligent agents. Section III
discusses related work in AI techniques for communications and D2D. Section IV
presents the proposed solution of distributed control in D2D through BDIx
agents and describes the DAIS algorithm. Section V discusses the evaluation of
the proposed approach, and lastly, Section VI contains our conclusions and
ideas for future work.

##  II Background on D2D and BDI Agents

###  II-A Background on D2D

####  II-A1 Control of D2D Communication

We can categorize the solutions on D2D communication based on the type of
control, as follows:

  * •

Centralized: In centralized techniques the BS completely manages the UE nodes,
even when they (UEs) are communicating directly. The controller manages all
aspects if interference/connections/path etc., between cell and D2D UEs.

  * •

Distributed: In a distributed scheme, the procedure of D2D node management
does not require a central entity, but it is performed autonomously by the UEs
themselves. The distributed scheme decreases the control and computational
overhead. This tactic is more suitable for large size D2D networks. In such a
system, all control processes are run in parallel and start at the same time.

  * •

Semi Distributed: In spite of the fact that both centralized and distributed
schemes have their strong points, tradeoffs can be accomplished between them.
Such D2D management schemes are referred to as semi-distributed” or ”hybrid”.

####  II-A2 Transmission Mode in D2D Communication

There exist different modes for D2D communication, based on how UEs interact
with the BS and other D2D nodes (see Fig. 1).

  * •

D2D Direct: Two UEs connect to each other by using licensed or unlicensed
spectrum. The two D2D UEs only communicate with each other (also called Full-
Duplex D2D).

  * •

D2D Single-hop Relay: Sharing of bandwidth between a UE and other UEs. In D2D
Single-hop Relay mode one of the D2D UEs is connected to a BS or access point
(AP) and provides access to another D2D UE. [29].

  * •

D2D Multi-hop Relay: The single-hop mode is extended by enabling the
connection of more D2D UEs in chain. Both backhaul and D2D transmissions are
performed in an uplink with other D2D relay node (as a bridge) and they are
subject to the control of the other D2D relay node [30].

  * •

D2D Cluster: D2D cluster is a group of UEs connected to a D2D relay node
acting as a Cluster Head (CH). The D2D relay node acts as an intermediate
router to the network though an access point or BS. Clustering is suitable in
high user densities [31, 32, 33].

Figure 1: Transmission in D2D Communication

###  II-B Research Challenges in D2D

In order for D2D to mature and shape the D2D communication for the upcoming 5G
and beyond wireless communication networks, some technical issues must be
resolved [34, 35]. Each of these challenges is further elaborated below.

####  II-B1 Device Discovery

In order for two devices (i.e., UEs) to directly communicate with one another,
they must first perform a device discovery process to identify that they are
close to each other and in range for D2D communication [2, 18].

####  II-B2 Mode Selection

When a pair of D2D candidates identify each other for possible future
communication, mode selection is performed. Mode selection implies that a
decision is made whether the D2D candidates will communicate directly or via
the conventional cellular network [18]. The communication mode selection
should be carefully chosen in order not to impact on the interference in the
network. This communication mode decision is categorized in the following way:

  1. (a)

Inband D2D communication:

     * •

_Reuse/Underlay:_ D2D communication shares the same resources with existing
Cellular UEs. This mode can achieve high spectral efficiency; however, it may
cause interference to other Cellular and D2D UEs using the cellular resources.

     * •

_Dedicated/Overlay:_ The cellular network has abundant channel resources so
that the D2D UEs can use dedicated resources that are orthogonal to cellular
UEs.

     * •

_Cellular:_ The two UEs will communicate with each other via the cellular
network as traditional cellular UEs.

  2. (b)

Outband D2D Communication:

     * •

_Controlled:_ In the controlled mode the device has two interfaces. On the
first interface it uses unlicensed spectrum to share with its peers. On the
second interface it uses licensed spectrum to connect to the mobile network.

     * •

_Autonomous:_ In autonomous mode, the device can only use and communicate with
other devices under the unlicensed spectrum, without accessing BS.

####  II-B3 Interference Management

The communication mode selection has a direct impact on the interference in
the network. For example when the Reuse/Underlay resource-sharing mode is
selected, high spectral efficiency can be achieved. However, since many D2D
and cellular users will use the same portion of spectrum, interference may
become a problem. Therefore, interference management must be used [18].

####  II-B4 Power Control

Although high transmission power can provide wider coverage and better signal
quality during D2D communication, it can, at the same time, drain the battery
of D2D UEs and cause interference to the network. Thus, proper power control
during D2D communication is vital for controlling the transmission power
levels of D2D UEs so as to deal with the interference generated by the D2D UEs
and improve spectral efficiency, system capacity, coverage, and reduce energy
consumption [19, 20, 21].

####  II-B5 Security Concerns

In D2D communications, the routing of users’ data is done through other users’
devices. This makes the D2D communication network vulnerable to many security
risks and malicious attacks that could breach the data privacy and
confidentiality. Thus, providing efficient security is a major issue in order
to facilitate D2D communication in cellular networks [22, 23, 12].

####  II-B6 Radio Resource Allocation

Radio resource allocation mainly addresses the issues of how to assign the
frequency resources to a group of D2D pairs, or all the D2D pairs, targeting
an optimal use of the radio resources focusing also on the interference
control and management between D2D and cellular links and the efficient reuse
the radio resources whenever the interference is small [18].

####  II-B7 Cell Densification and Offloading

Providing high system capacity and high per-user data rates – requirements for
the creation of a 5G network – will require a densification of the radio
access network or the deployment of additional network nodes. In general, the
need of network densification [24] for performance enhancement dictates the
deployment of small coverage cells [18].

####  II-B8 QoS / Path Selection (Routing)

During D2D communication it is essential to ensure that the QoS requirements
of the communication links are satisfied. To achieve this a major issue to
handle is the selection of the optimum routing path, otherwise excess
resources/power/link usage (bandwidth) will be wasted [20, 25, 26].

####  II-B9 D2D in mmWave communication

Communication using the mmWave band has recently received significant
attention for 5G cellular networks and D2D communication, as it operates at a
higher frequency band (30-300 GHZ) and allows a significant increase in data
rates (multi-Gbps) and network capacity [27].

####  II-B10 Handover of D2D device

In order to keep the communication between two D2D devices when these are
moving away from each other, handover should be performed. More specifically,
when a D2D device is moving away from the access point (e.g., a D2D Relay or a
D2D Cluster Head) it is assigned to, then the problem of handing it over to
another access point (e.g., another D2D Relay or D2D Cluster Head) with a
shared medium should be dealt with [20].

####  II-B11 Non-cooperative Users

An issue to consider for D2D data delivery is that the data delivery in non-
cooperative D2D communication may be unfair or compromised. In the real world,
some rational nodes may have strategic interactions and may act selfishly for
various reasons (such as resource limitations, the lack of interest in data,
or social preferences) or even malicious nodes that they may use the data
relay to attack anonymously [28].

###  II-C Background on Intelligent Agents and Belief-Desire-Intention Agents

####  II-C1 Intelligent Agents

An intelligent agent (IA) is an autonomous unit, which observes an environment
using sensors and acts upon it using actuators, coordinating its activity in
the direction of achieving goals (i.e. it is ”rational”, as defined in
economics) [14]. Agent theory is concerned with the use of mathematical
formalisms for representing reasoning and the properties of agents. Software
agents are characterized as computer software that display flexible autonomous
behavior, which infers that these systems are capable of independent,
autonomous action in order to satisfy their design objectives. Agents are
utilized in a lot of applications. For instance, autonomous programs used for
operator assistance or data mining (in some cases referred as bots) are also
called ”intelligent agents”.

####  II-C2 Belief-Desire-Intention Agents

This work makes use of Belief-Desire-Intention (BDI) software agents, which
are agents with three key mental structures (see Fig. 2): informative states
of mind around the world (beliefs or convictions), motivational approaches on
what to do (desires or wants) and planned responsibilities to take action
(intentions or expectations). The BDI model fundamentally relies on two
principle forms: thought and mean send thinking. With the thought processes
the agent produces its goals on the premise of its convictions and desires,
while mean send thinking comprises of a succession of activities to execute,
as an endeavor to satisfy desires [15].

Figure 2: BDI Agent Architecture

Unique features of BDI agents [16]:

  1. (a)

Beliefs: Beliefs correspond to the informational state of the agent. Beliefs
can also include inference rules, allowing advance chaining to guide to new
beliefs.

  2. (b)

Desires: Desires correspond to the motivational state of the agent. They
characterize objectives or situations that the agent would like to fulfil or
bring about.

  3. (c)

Intentions: Intentions correspond to the deliberative state of the agent. This
is what the agent has chosen to perform. Intentions are desires to which the
agent has, to some extent, committed.

A BDI agent decides its actions based on beliefs, which either contribute to
the achievement of its goals, or react to its received (or perceived) events
and messages. [17]. BDI agents can also cooperate and form a multi-agent
system. Multi-agent systems are systems composed of multiple interacting
computing elements capable of autonomously deciding what actions they require
to perform in order to satisfy their design objectives. In multi-agent
systems, the entities are interacting with other agents, not only by
exchanging information, but also by applying analogues of the type of social
activity that people engage in every day, like cooperation, coordination, and
negotiation [17]. In multi-agent systems, there are two important issues to
consider: (a) since agents are anticipated to be autonomous, it is usually
expected that the synchronization and coordination structures in a multi-agent
system are not hard-wired at design time, as they normally are in standard
concurrent/distributed systems. In this manner, mechanisms are needed in order
to allow agents to synchronize and coordinate their activities at runtime; and
(b) the encounters that occur between computing elements in a multi-agent
system are financial encounters, in the sense that they are encounters between
self-interested entities. In a classic distributed/concurrent system, all the
computing elements are implicitly expected to share the common goal of making
the overall system function correctly. In multi-agent systems, it is assumed
instead, that agents are primarily concerned with their own welfare, although
of course, they will be acting on behalf of some user/owner [17].

In addition, we can say the BDI agents have foundations in the Algorithmic,
Game-Theoretic, and Logical theories [17]. All the features discussed above
make, in our opinion, BDI agents suitable for solving the challenges of D2D.

##  III Related Work

###  III-A Related Work on AI Techniques for Communications

There is a wealth of research on the use of artificial intelligence (AI) and
machine learning (ML) techniques for communication and networking issues. In
this section we include a few examples that deal with the use of multi-agent
systems and BDI agents in general communication problems and at the end focus
on AI approaches for D2D communication.

###  III-B Multi-agent Approaches for Wireless and Mobile Communications

The authors in [38] address the problem of energy consumption and
communication latency in wireless sensor networks (WSNs). More specifically,
the authors propose a system with a single mobile agent (MA) travelling freely
within the network and performing data collection. This behavior improves data
delivery to the sink, and reduces energy consumption. The specific work
utilizes deep neural network for learning, in which the input is the state of
the wireless sensor network and the output is the optimal route path. The
route planning can be done with the usage of the locations of each node in the
environment that act as input for the intelligent agent. The intelligent agent
architecture selected is the actor network and a critic network. The
information used comes from the whole network, but the decision is taken
locally.

Another work that uses reinforcement learning is [36], which deals with the
problem of discovering low-level wireless communication schemes between two
agents in a fully decentralized system. This is the type of problem considered
in the DARPA Spectrum Collaboration Challenge (SC2). The proposed method
employs policy gradients to learn an ideal bi-directional communication
scheme. The approach places two agents against each other and show that the
two actors are able to learn modulation schemes for communication while
sharing only limited information and having no domain-specific knowledge about
the task.

###  III-C BDI Agents for Wireless and Mobile Communications

The authors in [37] utilize a multi-agent software design, dynamic analysis,
and decentralized control in order to implement solutions for the complex
distributed systems of WSNs. The paper’s purpose is to create an autonomic
system design for distributed nodes in a diverse and changing environment,
that interact on top of a wireless communication channel for decentralized
problem solving. Due to hardware limitations, the multi-agent system
techniques and especially nodes (agents) are not deliberative (or strong)
reasoning systems. The BDI agent model is used. The paper’s authors implement
two simple WSN test scenarios and show that BDI agents can perform basic WSN
functions. In addition, the agents succeed in imitating some recognizable
aspects of the system and show that the solution is adaptable to different
scenarios. In the scenarios, five different agents are discussed. A problem of
this approach is that a better method is needed for managing the size of the
belief-base used in each agent, as this turns out to expand unboundedly in a
case such as flooding.

Another class of wireless networks built dynamically in an ad hoc network
manner with a large mobile user base is found in vehicular ad-hoc networks
(VANETs). The work presented in [15] tackles the problem of routing in VANETs.
Routing in VANETs is critical because of limitations such as unpredictable
network topology, frequent disconnections, and varying network densities. The
authors in this paper proposed a multi-agent scheme-based routing scheme that
comprises of static agent and mobile agents for vehicle-to-vehicle
communication (V2V), where they address the challenge of how to route the data
with short communication delay, overhead, and the complexity. The proposed
algorithm has the following steps: i) establish a connectivity pattern between
the vehicles; ii) create a set of beliefs; iii) develop the desires, and iv)
execute the intentions.

###  III-D Artificial Intelligence Approaches for D2D

In the last decade we have seen many approaches for solving the D2D challenges
using AI and ML [46]. The authors in [20] proposed EHSD -Exemplary Handover
Scheme During D2D Communication- a framework describing a handover scheme that
is based on software-defined radio (SDR) decentralization by using fuzzy
logic. In [39], the authors proposed a learning-based resource allocation
approach for D2D communications with QoS and fairness considerations by using
Q-Learning. In addition, in [40] the authors proposed a Hierarchical Extreme
Learning Machine (H-ELM) Neural Network in order to manage the severe
interference in D2D communications. Another paper, [41], proposed a genetic
algorithm (GA)-based scheme for Fair Joint Channel Allocation and Power
Control for Underlaying D2D Multicast Communications. Also in [42], the
authors proposed an approach for power control in two-tier orthogonal
frequency division multiple access (OFDMA) femtocell networks by using
particle swarm optimization (PSO). Another intelligent technique is presented
in [43], where the authors used an ant-colony optimization (ACO)-based
resource allocation scheme for solving the problem of swarm intelligence-based
radio resource management for D2D-based V2V communication.

In the evaluation section we will compare our results with those of [44]. The
authors in [44] use a low complexity method for matching D2D links with
cellular UEs to form partners for spectrum sharing. Another work we will
compare with is [45], which investigates the gain that cooperative multicast
transmission provides when used to boost the data rate in D2D communication,
enabling data sharing among users by implementing clusters.

All of the solutions discussed above solve only one of the many challenges
identified in II-B, with the exception of [47], which solves a joint sub-
carrier assignment and power allocation problem. There is also a yet
unpublished work by [48], which claims to be offering a solution in joint
network admission control, mode assignment and power allocation in energy-
harvesting D2D networks.

To the best of our knowledge there is currently no other work addressing 5G
D2D communication issues using BDI agents with extended AI capabilities
(BDIx).

##  IV Distributed Control in D2D Through BDIx Agents

In this Section we are describing both the new framework we are proposing for
using BDI agents for D2D communication and we are also describing the DAIS
Algorithm for selecting a node’s transmission mode.

The flowchart in Figure 3 shows the operation of a BDIx agent from the point
it receives a message from the environment, until it selects and executes a
plan.

Figure 3: Flowchart of BDIx Agent Operation

After perceiving a change in its world, the agent checks if the Intention must
be satisfied or must be changed. If the Intention is not changed then it
continues with the execution of the Intention plan. Otherwise, the agent
selects another Intention from the list that it has the higher priority and
then it selects a Plan that will satisfy the selected Intention. After this it
continues to execute the plan.

###  IV-A Assumptions and Constraints

The assumptions used in the design of the BDIx agents framework are the
following:

  * •

The information needed by BDIx agents is the following: frequencies used, IP
addresses, remaining energy, transmission mode (D2D Relay/D2D multi-hop/D2D
cluster), etc (see Section II-A2).

  * •

Location is known at the agent (all known devices have GPS).

  * •

Location information and signals can be obtained within an operator’s network.

  * •

Each agent must be either a D2D Relay Node (D2D-R), a Multi-hop Relay Node
(D2D-MHR), or a D2D Cluster Head (D2D-CH), or a ”client” D2D node, i.e. at the
edge of the communication path. So a D2D node can either serve or be served,
not both. The agent will decide its role based on the beliefs and the events
it has.

  * •

A frequency should exist for the outband inter-communication between the BDIx
agents.

  * •

A threshold should be preset on Signal Quality (Received Signal Strength and
Bit Error Rates)

  * •

All D2D UEs that are in D2D-R or D2D-MHR transmission modes know their link
and path rates and they can broadcast them over LTE proximity services.

  * •

A BDIx Agent always accepts proposals from other BDIx Agents (e.g. a D2D UE to
D2D-CH or D2D-MHR request is always granted).

  * •

A BDIx Agent always selects unused RB (Resource Block in OFDMA). This is done
for simplicity. The resource management and interference management will be
done in future work.

  * •

The UE device has two mobile interfaces or is using full duplex interface
split equally between uplink and downlink.

  * •

The UE device has one WiFi interfaces (like all mobiles).

###  IV-B Sum Rate and Weighted Data Rate

One of the most common metrics for the evaluation of D2D solutions is Sum-
Rate. The Sum-Rate is the total throughput in a network calculated as the sum
of the data rates that are delivered to all UEs and D2D UEs in a network [49,
50]. Variations on Sum Rate exist, such as Weighted Sum-Rate in [51], which
considers certain links to be of more importance and gives different weights
to the links based on the mode of transmission (direct, relay, etc). We
introduce a new metric called ”Weighted Data Rate” (WDR). The WDR is defined
at each node as the minimum data rate in the path that the UE selected. The
minimum data rate of a path is the data rate of the weakest edge in the path.
Our aim is, essentially, to maximize the WDR, i.e WDR = max(min(Link Rate) for
each path. The choice for using WDR instead of sum-rate is mainly for reducing
the computational load of the BDI agent. The benefits will be shown clearly in
the next section.

###  IV-C The DAIS Algorithm for Transmission Mode Selection

The following terms are used in the DAIS algorithm:

  * •

D2D-R \- D2D Relay node.

  * •

D2D-MHR \- D2D Multihop Relay node.

  * •

D2D-CH \- D2D Cluster Head.

  * •

WDR \- Weighted Data Rate.

  * •

MAXUsersCH \- Maximum Users Supported by CH = 255. This is based on WiFi
Direct limits.

  * •

MAXQueryD2DRelayDistance \- Maximum distance to query D2D Relay UEs = 200m. It
is the maximum distance of WiFi Direct (200m) or the maximum distance of LTE
Direct (1000m).

  * •

MAXDistancetoFormCluster \- Maximum distance threshold to accept connection to
the node if the UE is CH. This is the pre-defined maximum radius range between
D2D UEs in order to form a D2D Cluster. This will be calculated based on the
technology used (WiFi Direct or LTE Direct). It can be calculated from WiFi
Direct range/2 (100m) or LTE Direct range/2 (500m).

  * •

MAXSpeedToFormBackhauling \- Maximum speed that a node is moving in order to
be D2D Relay or D2D Multi Hop Relay = 1.5 m/s (pedestrian).

  * •

MAXDistanceMultiHop \- Maximum distance threshold for a UE from the nearest
D2D Relay in order to act as D2D multi-hop relay. In order for the UE to
select to be multi-hop, the device must have the weighted data rate to an
existing D2D Relay greater than the weighted data rate of the existing D2D
Relay.

  * •

MAXDistanceMoveAway \- Maximum Distance to move away from the current position
in order to recalculate. It can be calculated from WiFi Direct range/2 (100m)
or LTE Direct range/2 (500m).

  * •

PERCDataRate \- Percentage of difference of Data Rate in order to make D2D
Relay connect from UE D2D multihop Relay to Gateway =20

  * •

DeviceBatteryThreshold \- The minimum battery percentage in order of the D2D
device to act as D2D-R or D2D-MHR is 70%

  * •

The D2D device power is calculated randomly and it is following a Gaussian
distribution with mean of 0.6 and variance of 0.4.

TABLE I: Algorithm Notations and Mathematical Representations

Notations Mathematical Representation d
$\sqrt{\left({UE{x_{1}}-D2D{x_{2}}}\right)^{2}+\left(UE{{y_{1}}-D2D{y_{2}}}\right)^{2}}$
maxD2DR $\begin{multlined}D2D_{j}\ where\
WDR_{D2D_{j}}=(MAX(WDR_{D2D_{i}})\exists\ D2D_{i}\ where\ d\geq\
MAXDistancetoFormCluster\\\\[-4.30554pt] \wedge WDR_{D2D_{i}}\ \geq\
(WDR_{UE_{i}}\ +\ PERCDataRate*WDR_{UE_{i}})\ \wedge\ i\in D2DR\\\ \wedge\
COUNT({D2D_{i}}_{g}\ \\\\[-4.30554pt] WHERE\ g\ servedby\ i)<=D)\\\
\end{multlined}D2D_{j}\ where\ WDR_{D2D_{j}}=(MAX(WDR_{D2D_{i}})\exists\
D2D_{i}\ where\ d\geq\ MAXDistancetoFormCluster\\\\[-4.30554pt] \wedge
WDR_{D2D_{i}}\ \geq\ (WDR_{UE_{i}}\ +\ PERCDataRate*WDR_{UE_{i}})\ \wedge\
i\in D2DR\\\ \wedge\ COUNT({D2D_{i}}_{g}\ \\\\[-4.30554pt] WHERE\ g\ servedby\
i)<=D)\\\ $ maxD2DMHRNoConnections $\begin{multlined}D2D_{j}\ where\
WDR_{D2D_{j}}=(MAX(WDR_{D2D_{i}})\exists\ D2D_{i}\ where\ d\geq\
MAXDistancetoFormCluster\wedge\\\\[-4.30554pt] WDR_{D2D_{i}}\ \geq\
(WDR_{UE_{i}}\ +\ PERCDataRate*WDR_{UE_{i}})\ \wedge\ i\in D2DMHR\ \wedge\
COUNT({D2D_{i}}_{g}\ \\\\[-4.30554pt] WHERE\ g\ servedby\ i)=0)\\\
\end{multlined}D2D_{j}\ where\ WDR_{D2D_{j}}=(MAX(WDR_{D2D_{i}})\exists\
D2D_{i}\ where\ d\geq\ MAXDistancetoFormCluster\wedge\\\\[-4.30554pt]
WDR_{D2D_{i}}\ \geq\ (WDR_{UE_{i}}\ +\ PERCDataRate*WDR_{UE_{i}})\ \wedge\
i\in D2DMHR\ \wedge\ COUNT({D2D_{i}}_{g}\ \\\\[-4.30554pt] WHERE\ g\ servedby\
i)=0)\\\ $ maxD2DRNoConnectionsToBeD2DMHR $\begin{multlined}D2D_{j}\ where\
WDR_{D2D_{j}}=(MAX(WDR_{D2D_{i}})\exists\ D2D_{i}\ where\ d\geq\
MAXDistancetoFormCluster\wedge\\\\[-4.30554pt] d\leq\
MAXQueryD2DRelayDistance\ \wedge WDR_{D2D_{i}}\ \geq\ (WDR_{UE_{i}}\ +\
PERCDataRate*WDR_{UE_{i}})\ \wedge\\\\[-4.30554pt] \ i\in D2DR\ \wedge\
COUNT({D2D_{i}}_{g}\ WHERE\ g\ servedby\ i)=0)\ \wedge\\\\[-4.30554pt]
D2DDevicePower_{i}\geq\ DeviceBatteryThreshold\\\ \end{multlined}D2D_{j}\
where\ WDR_{D2D_{j}}=(MAX(WDR_{D2D_{i}})\exists\ D2D_{i}\ where\ d\geq\
MAXDistancetoFormCluster\wedge\\\\[-4.30554pt] d\leq\
MAXQueryD2DRelayDistance\ \wedge WDR_{D2D_{i}}\ \geq\ (WDR_{UE_{i}}\ +\
PERCDataRate*WDR_{UE_{i}})\ \wedge\\\\[-4.30554pt] \ i\in D2DR\ \wedge\
COUNT({D2D_{i}}_{g}\ WHERE\ g\ servedby\ i)=0)\ \wedge\\\\[-4.30554pt]
D2DDevicePower_{i}\geq\ DeviceBatteryThreshold\\\ $ maxD2DRToUseUED2DMHR
$\begin{multlined}D2D_{j}\ where\ WDR_{D2D_{j}}=(MAX(WDR_{D2D_{i}})\exists\
D2D_{i}\ where\ d\geq\ MAXDistancetoFormCluster\ \wedge\\\\[-4.30554pt] d\leq\
MAXQueryD2DRelayDistance\wedge WDR_{D2D_{i}}\ \ll\ (WDR_{UE_{i}}\ -\
PERCDataRate*WDR_{UE_{i}})\ \\\\[-4.30554pt] \wedge\ i\in D2DR\wedge
D2DDevicePower_{i}\geq\ DeviceBatteryThreshold\\\ \end{multlined}D2D_{j}\
where\ WDR_{D2D_{j}}=(MAX(WDR_{D2D_{i}})\exists\ D2D_{i}\ where\ d\geq\
MAXDistancetoFormCluster\ \wedge\\\\[-4.30554pt] d\leq\
MAXQueryD2DRelayDistance\wedge WDR_{D2D_{i}}\ \ll\ (WDR_{UE_{i}}\ -\
PERCDataRate*WDR_{UE_{i}})\ \\\\[-4.30554pt] \wedge\ i\in D2DR\wedge
D2DDevicePower_{i}\geq\ DeviceBatteryThreshold\\\ $ maxD2DMHRToUseAsMultiHop
$\begin{multlined}D2D_{j}\ where\ WDR_{D2D_{j}}=(MAX(WDR_{D2D_{i}})\exists\
D2D_{i}\ where\ d\geq\ MAXQueryD2DRelayDistance\wedge\\\\[-4.30554pt] d\leq\
MAXDistanceMultihop\ \wedge WDR_{D2D_{i}}\ \geq\ (WDR_{UE_{i}}\ +\
PERCDataRate*WDR_{UE_{i}})\ \wedge\\\\[-4.30554pt] i\in D2DMHR\ \wedge
COUNT({D2D_{i}}_{g}\ WHERE\ g\ servedby\ i)=0)\ \wedge\\\\[-4.30554pt]
D2DDevicePower_{i}\geq\ DeviceBatteryThreshold\\\ \end{multlined}D2D_{j}\
where\ WDR_{D2D_{j}}=(MAX(WDR_{D2D_{i}})\exists\ D2D_{i}\ where\ d\geq\
MAXQueryD2DRelayDistance\wedge\\\\[-4.30554pt] d\leq\ MAXDistanceMultihop\
\wedge WDR_{D2D_{i}}\ \geq\ (WDR_{UE_{i}}\ +\ PERCDataRate*WDR_{UE_{i}})\
\wedge\\\\[-4.30554pt] i\in D2DMHR\ \wedge COUNT({D2D_{i}}_{g}\ WHERE\ g\
servedby\ i)=0)\ \wedge\\\\[-4.30554pt] D2DDevicePower_{i}\geq\
DeviceBatteryThreshold\\\ $

In our approach the D2D-R/D2D-MHR are using proximity services to broadcast
the connection information (i.e. WDR, coordinates).

1connect to BS (GateWay) ;

2

/* Check to find D2D Relay to connect as client */

/* Check if a D2D Relay device exists near the D2D UE with the maximum WDR */

3 if  _exists maxD2DR_ then

/* Check to find D2D Relay to connect as client */

4 Connect UE as D2D Client to maxD2DR using WiFi Direct;

5

/* Check if a D2D-MHR exists near the D2D UE with the maximum WDR and convert
it to D2D-R */

6 else if  _exists maxD2DMHRNoConnections_ then

/* Check to find D2D Multihop Relay that no one connects to, make it D2D
Relay, and connect to it as D2D Client */

7 Request from maxD2DMHRNoConnections UE to be D2DR;

8 Connect UE as D2D Client to maxD2DMHRNoConnections using WiFi Direct;

/* Now the D2D-MHR is D2D-R */

9

/* Connect as D2D-R or Optimize a Path */

/* Check if a D2D-R device exists far from the D2D UE with maximum WDR and not
have connections other than path to BS in order to connect to it as D2D-R (The
device will convert to D2D-MHR) */

10 else if  _exists maxD2DRNoConnectionsToBeD2DMHR_ then

/* Check to find D2D-R that no one connects to and make it D2D-MHR and connect
to it as D2D-R */

11 Request from DMHRNoConnections UE to be D2D-MHR ;

12 Connect UE as D2D-R to maxD2DRNoConnectionsToBeD2DMHR using LTE Direct;

/* Now the D2D-R is D2D-MHR and UE is D2D-R */

13

/* Check if a D2D-R device exist far from the D2D UE with maximum WDR worse
than the UE and with no connections other than a path to BS in order to
connect to it as D2D-MHR (The device will connect as D2D-R to the new UE that
is going to be D2D-MHR) */

14 else if  _exists maxD2DRToUseUED2DMHR_ then

/* Check to find D2D-R that no one connects to, with worse WDR than the UE and
make UE as D2D-MHR and ask the device D2D-R to connect to UE */

15 Set UE as D2D-MHR ;

16 Connect maxD2DRToUseUED2DMHR as D2D Relay to UE using LTE Direct;

/* Now the UE is D2D-MHR */

17

/* Check if a D2DMHR device exist from the D2D UE with maximum WDR and no
connections other than path to BS in order to connect to it as D2D-R */

18 else if  _exists maxD2DMHRToUseAsMultiHop_ then

/* Check to find D2D-MHR that no one connects to, make UE as D2D-R and connect
to it */

19 Set UE as D2D-R ;

20 UE.TransmissionMode=D2D Relay ;

21 Connect UE as D2D-R to maxD2DMHRToUseAsMultiHop using LTE Direct;

/* Now the UE is D2D-R */

22

23else

24 set UE as D2D-MHR; Stay connected to BS ;

25

26

27

Algorithm 1 DAIS Algorithm for Transmission Mode Selection Plan in BDIx Agents

The notation and mathematical representation of symbols used in the DAIS
algorithm are presented in Table I. The plan of execution of transmission mode
selection is shown in Algorithm 1. This is executed at the startup phase of
the BDIx Agent. The computational complexity of such an algorithm is O(n)
because the algorithm calculates the values in Table I only once. In addition,
the algorithm is quick because decisions are made locally and do not rely on
global information. Since routes are created instantaneously and incrementally
by each agent, by identifying local D2D-R and D2D-MHR using proximity
services, the complexity is based on the actual number of D2D-R and D2D-MHR
that the agent in each device must communicate with, whenever it is needed
(e.g. in order to become D2D-R by connecting to an existing D2D-MHR in our
algorithm).

##  V Performance Evaluation of the DAIS Algorithm

In this section, we investigate the performance of the proposed DAIS
algorithm. The simulations are done using Java and Matlab. We consider
scenarios with one BS and a number of UEs ranging from 10 to 1000, over an
area of 1000x1000 meters. The BS in the simulations is in the center of the
grid. The simulation parameters are shown in Table II. The parameters are
taken from the standards for WiFi Direct [53], LTE Direct [52], and LTE
communication [54, 55].

Simulation Parameters | Value
---|---
D2D power | 130 mW [54, 55]
UE power | 260 mW [54, 55]
WiFi Direct Radius | 200 m [52]
LTE Direct Radius | 1000 m [53]
BS Range | 1000 m [54, 55]
Path loss exponent (Urban Area) | 3.5
BS Antenna gain | 40 dB [54, 55]
UE/D2D antenna gain | 2 dB [54, 55]
N0 (White Noise) | 0.0001
D (WiFi Direct max clients) | 200 [52]
N (no of UEs) | 10-1000
Shadowing | Log-normal
Mobility | Static scenario
TABLE II: Simulation Parameters

figureSpectral Efficiency of Different Transmission Modes

figurePower Savings of Different Transmission Modes

figureSpectral Efficiency of Different Rate Options

figureSpectral Efficiency of Different Power Options

The first thing we examine is the spectral efficiency of the proposed
solution. Figure V shows that our proposed solution has a better performance
compared to a random clustering solution and when no-D2D communication is
used. The realized benefits are in the order of 30%. The most interesting
result is that random clustering results in spectral efficiency even worse
than direct UE-BS communication.

Considering the power needed to realize the communication of the nodes, it is
not surprising to see that clustering indeed requires less power. However, the
proposed solution still outperforms the second best by about 25%.

Within the proposed framework we have the ability to easily interchange
metrics and parameters. In Section IV-B we have argued on the feasibility of
using WDR instead of Sum-Rate in our calculations. Figure V shows that the use
of WDR does not reduce the spectral efficiency of the system. The same happens
if we consider an option in which a UE participates in the D2D communication
depending on the remaining battery it has. Figure V shows no difference in
spectral efficiency.

Figure 4: Power Saved Figure 5: Computational Complexity

On the contrary, by utilizing a battery threshold we are slightly increasing
the required power for the communication, as evident by the slight differences
shown in Fig. 4.

A significant result, which validates our choice of WDR is that the
computational time needed to perform sum-rate calculations is up to five (5)
times greater than the constant computation needed when we perform WDR
calculations locally. This is ascribed to the fact that sum-rate needs to
check all links in the network every time it needs to decide the transmission
mode of a UE. As the number of UEs increases the computational time increases
as well. In our case, the time to form a cluster is 100ms for any device
density, because the D2D UEs have all their link rates precalculated, so that
WDR for the new connection is easily computed.

By comparing the results of our approach with those in [45] we observe that
for 50 UEs (maximum number considered in that work) we have the same number of
clusters (seven) and the same amount of average UEs per cluster. However, we
have no way of knowing if the solution in [45] can scale, whereas our approach
is shown to scale well for at least up to 1000 UEs. In our approach the energy
gained by the BS when we apply clustering is the same as in the work in
comparison. However, in our case we can have 1000 UEs clustering at the almost
instantaneous time of 100ms. Another work that lends itself for comparison is
[44], when considered for similar BS and UE power as well as node density. The
max number of UEs and D2D links used in that work is, again, in the order of
50. In the best case scenario analyzed in [44], the spectral efficiency
reaches 220 b/s/Hz for N=30 UEs when all of them are D2D linked. The
performance goes down to 180 b/s/Hz as the D2D links are reduced to twenty
(20). By comparison, in our work, a similar number of UEs (N=30) and D2D links
the corresponding spectral efficiency is 296 b/s/Hz. If we test it with 30 D2D
links and 10 UE links the BDI solutions has a rate of 405 b/s/Hz which
outperforms the 260 b/s/Hz of the paper in comparison.

##  VI Conclusions and Future Work

Device to Device (D2D) Communication is expected to be a core part of the
forthcoming 5G Mobile Communication Networks. To achieve that goal, several
challenges, like interference management, power control, and routing, among
others, need to be addressed. The paper investigates the problem of solving
multiple D2D communication requirements in one framework by using BDI agents.
Such agents can be implemented at the UEs and there is no need to change how
BSs operate or to change the hardware at BSs or UEs. The current work focuses
on the definition of a joint solution of D2D requirements. To that extend it
contains a detailed proof-of-concept algorithm. which works towards deciding
the transmission mode of each UE and forms the best possible paths towards the
base station using relays and clusters. Through simulations the solution was
found to ensure a high spectral efficiency and low computational load. In
future work we will focus on the utilization of more AI approaches under our
BDIx framework and we will evaluate a more dynamic environment by considering
mobile UEs.

## References

  * [1] K. Doppler, M. P. Rinne, P. Janis, C. Ribeiro, and K. Hugl, ”Device-to-device communications; Functional prospects for LTE-advanced networks,” in _Proc. IEEE Int. Conf. Commun. Work., ICC Workshops_ , Jun. 2009, pp. 1-6.
  * [2] G. Fodor, E. Dahlman, G. Mildh, S. Parkvall, N. Reider, G. Miklós, and Z. Turányi, ”Design aspects of network assisted device-to-device communications,” _IEEE Commun. Mag_., vol. 50, no. 3, pp. 170-177, Mar. 2012.
  * [3] P. Gandotra and R. K. Jha, ”Device-to-Device Communication in Cellular Networks: A Survey,” J. Netw. Comput. Appl., vol. 71, no. 4, pp. 1801-1819, Aug. 2016.
  * [4] M. Ahmad, M. Azam, M. Naeem, M. Iqbal, A. Anpalagan, and M. Haneef, ”Resource management in D2D communication: An optimization perspective,” _J. Netw. Comput. Appl._ , vol. 93, pp. 51-75, Sep. 2017.
  * [5] S. Wen, X. Zhu, Z. Lin, X. Zhang, and D. Yang, ”Distributed resource management for Device-to-Device (D2D) communication underlay cellular networks,” in _IEEE Int. Symp. Pers. Indoor Mob. Radio Commun_. PIMRC 2013, pp. 1624-1628.
  * [6] D. H. Lee, K. W. Choi, W. S. Jeon, and D. G. Jeong, ”Two-stage semi-distributed resource management for device-to-device communication in cellular networks,” _IEEE Trans. Wirel. Commun._ , vol. 13, no. 4, pp. 1908-1920, April 2014.
  * [7] D. Wu, Y. Cai, R. Q. Hu, and Y. Qian, ”Dynamic Distributed Resource Sharing for Mobile D2D Communications,” _IEEE Trans. Wirel. Commun._ , vol. 14, no. 10, pp. 5417-5429, Oct. 2015.
  * [8] G. Fodor and N. Reider, ”A Distributed Power Control Scheme for Cellular Network Assisted D2D Communications,” in _IEEE Global Telecommunications Conference - GLOBECOM_ , Dec. 2011, pp. 1-6.
  * [9] F. Librino and G. Quer, ”Distributed Mode and Power Selection for a Stochastic Approach,” _IEEE Trans. Cogn. Commun. Netw._ , vol. 4, no. 2, pp. 232-243, Feb. 2018.
  * [10] J. Kim, J. Park, J. Noh, and S. Cho, ”Completely Distributed Power Allocation using Deep Neural Network for Device to Device communication Underlaying LTE,” Feb. 2018, arXiv:1802.02736.
  * [11] Y. Cai, H. Chen, D. Wu, W. Yang, and L. Zhou, ”A distributed resource management scheme for D2D communications based on coalition formation game,” in _Proc. IEEE Int. Conf. Commun. Work., ICC Workshops_ , Jun. 2014, pp. 355-359.
  * [12] H. Nguyen, M. Hasegawa, and W. Hwang, ”Distributed Resource Allocation for D2D Communications Underlay Cellular Networks,” _IEEE Commun. Lett._ , vol. 20, no. 5, pp. 942-945, May 2016.
  * [13] R. Yin, G. Yu, C. Zhong, and Z. Zhang, ”Distributed resource allocation for D2D communication underlaying cellular networks,” in _Proc. IEEE Int. Conf. Commun. Work., ICC Workshops_ , 2013, pp. 138-143.
  * [14] J. R. Stuart and N. Peter, _Artificial Intelligence a Modern Approach_ 3rd Edition, Prentice Hall, 2009.
  * [15] M. S. Kakkasageri, M. J. Sataraddi, P. M. Chanal, and G. S. Kori, ”BDI Agent Based Routing Scheme in VANETs,” _Int. Conf. on Wireless Commun., Sign. Proc. and Networking_ (WiSPNET), 2017, pp. 129-133.
  * [16] A. S. Rao and M. P. Georgeff, ”BDI agents: From theory to practice,” in _First Intl. Conf. on Multiagent Systems_ , 1995, pp. 312-315.
  * [17] Y. Shoham and K. Leyton-Brown, ”Multiagent systems: Algorithmic, Game-Theoretic, and logical foundations,” Cambridge University Press, 1 edition, Dec. 2008.
  * [18] M. Noura and R. Nordin, ”A survey on interference management for Device-to-Device (D2D) communication and its challenges in 5G networks,” _J. Netw. Comput. Appl._ , vol. 71, pp. 130-150, 2016.
  * [19] K. Kuntz, ”Performance of Macro- and Co-channel Femtocells in a Hierarchical Cell Structure,” Der Spiegel, 2015.
  * [20] H. B. Valiveti and P. T. Rao, ”EHSD: An Exemplary Handover Scheme During D2D Communication Based on Decentralization of SDN,” _Wirel. Pers. Commun._ , vol. 94, no. 4, pp. 2393-2416, 2017.
  * [21] F. Wang, C. Xu, L. Song, and Z. Han, ”Energy-efficient resource allocation for device-to-device underlay communication,” _IEEE Trans. Wirel. Commun.,_ vol. 14, no. 4, pp. 2082-2092, 2015.
  * [22] J. Yue, C. Ma, H. Yu, and W. Zhou, ”Secrecy-based access control for device-to-device communication underlaying cellular networks,” _IEEE Commun. Lett.,_ vol. 17, no. 11, pp. 2068-2071, Nov. 2013.
  * [23] C. Ma, W. Wu, Y. Cui, and X. Wang, ”On the performance of successive interference cancellation in D2D-enabled cellular networks,” in _2015 IEEE Conference on Computer Communications (INFOCOM)_ , 2015, pp. 37-45.
  * [24] E. Hossain and M. Hasan, ”5G cellular: key enabling technologies and research challenges,” _IEEE Instrum. Meas. Mag._ , vol. 18, no. 3, pp. 11-21, Jun. 2015.
  * [25] J. Liu, S. Zhang, N. Kato, H. Ujikawa, and K. Suzuki, ”Device-to-Device Communications for Enhancing Quality of Experience in Software Defined Multi-Tier LTE-A Networks,” _IEEE Network,_ vol. 29, no. 4, pp. 46-52, August 2015.
  * [26] M. Sadik, N. Akkari, and G. Aldabbagh, ”SDN-based handover scheme for multi-tier LTE/Femto and D2D networks,” _Comput. Networks_ , vol. 142, pp. 142-153, 2018.
  * [27] Y. Niu, Y. Li, D. Jin, L. Su, and A. V. Vasilakos, ”A survey of millimeter wave communications (mmWave) for 5G: opportunities and challenges,” _Wirel. Networks_ , vol. 21, no. 8, pp. 2657-2676, Nov. 2015.
  * [28] B. Jedari, F. Xia, and Z. Ning, ”A Survey on Human-centric Communications in Non-cooperative Wireless Relay Networks,” _IEEE Commun. Surv. Tutorials_ , vol. 20, no. 2, pp. 914-944, 2018.
  * [29] J. Deng, A. A. Dowhuszko, R. Freij, and O. Tirkkonen, ”Relay selection and resource allocation for D2D-relaying under uplink cellular power control,” _2015 IEEE Globecom Work. (GLOBECOM Wkshps)_ , Dec. 2015, pp. 1-6.
  * [30] G. Steri, G. Baldini, I. N. Fovino, R. Neisse, and L. Goratti, ”A novel multi-hop secure LTE-D2D communication protocol for IoT scenarios,” _23rd Int. Conf. Telecommun. ICT_ , May 2016, pp. 1-6.
  * [31] L. Song, D. Niyato, Z. Han, and E. Hossain, ”Game-theoretic resource allocation methods for device-to-device communication,” _IEEE Wirel. Commun._ , vol. 21, no. 3, pp. 136-144, 2014.
  * [32] T. Koskela, S. Hakola, T. Chen, and J. Lehtomäki, ”Clustering concept using device-to-device communication in cellular system,” _IEEE Wirel. Commun. Netw. Conf. WCNC,_ 2010\.
  * [33] B. Peng, T. Peng, Z. Liu, Y. Yang, and C. Hu, ”Cluster-based multicast transmission for device-to-device (D2D) communication,” _IEEE Veh. Technol. Conf._ , Sep. 2013, pp. 1-6.
  * [34] I. F. Akyildiz, S. Nie, S. C. Lin, and M. Chandrasekaran, ”5G roadmap: 10 key enabling technologies,” _Comput. Networks_ , vol. 106, pp. 17-48, 2016.
  * [35] S. Chen and J. Zhao, ”The requirements, challenges, and technologies for 5G of terrestrial mobile telecommunication,” _IEEE Commun. Mag_., vol. 52, no. 5, pp. 36-43, 2014.
  * [36] C. de Vrieze, S. Barratt, D. Tsai, and A. Sahai, ”Cooperative Multi-Agent Reinforcement Learning for Low-Level Wireless Communication,” Jan 2018, arXiv:1801.04541.
  * [37] A. Morris, P. Giorgini and S. Abdel-Naby, ”Simulating BDI-Based Wireless Sensor Networks,” 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology, Milan, Italy, 2009, pp. 78-81.
  * [38] J. Lu, L. Feng, J. Yang, M. M. Hassan, A. Alelaiwi, and I. Humar, ”Artificial agent: The fusion of artificial intelligence and a mobile agent for energy-efficient traffic control in wireless sensor networks,” _Futur. Gener. Comput. Syst_., vol. 95, pp. 45-51, 2019.
  * [39] S. Kazemi Rashed, R. Shahbazian, and S. A. Ghorashi, ”Learning-based resource allocation in D2D communications with QoS and fairness considerations,” _Trans. Emerg. Telecommun. Technol._ , vol. 29, no. 1, pp. 1-20, 2018.
  * [40] J. Xu, X. Gu, and Z. Fan, ”D2D Power Control Based on Hierarchical Extreme Learning Machine,” 2018 _IEEE 29th Annu. Int. Symp. Pers. Indoor Mob. Radio Commun._ , pp. 1–7, 2018.
  * [41] M. Hamdi, D. Yuan, and M. Zaied, ”GA-based scheme for fair joint channel allocation and power control for underlaying D2D multicast communications,” _13th Int. Wirel. Commun. Mob. Comput. Conf. IWCMC 2017_ , pp. 446-451, 2017.
  * [42] Z. Huang, Z. Zeng, H. Xia, and J. Shi, ”Power control in two-tier OFDMA femtocell networks with Particle Swarm Optimization,” _IEEE Veh. Technol. Conf._ , pp. 1-5, 2011.
  * [43] S. Feki, A. Masmoudi, A. Belghith, F. Zarai, and M. S. Obaidat, ”Swarm intelligence-based radio resource management for D2D-based V2V communication,” _Int. J. Commun. Syst._ , no. May, pp. 1–16, 2018.
  * [44] L. Wang and H. Wu, ”Fast pairing of device-to-device link underlay for spectrum sharing with cellular users,” _IEEE Commun. Lett._ , vol. 18, no. 10, pp. 1803–1806, 2014.
  * [45] S. Doumiati, H. Artail, and K. Kabalan, ”A framework for clustering LTE devices for implementing group D2D communication and multicast capability,” _8th Int. Conf. Inf. Commun. Syst. ICICS 2017_ , pp. 216-221, 2017.
  * [46] K. Zia, N. Javed, M. N. Sial, S. Ahmed, H. Iram, and A. A. Pirzada, ”A Survey of Conventional and Artificial Intelligence / Learning based Resource Allocation and Interference Mitigation Schemes in D2D Enabled Networks,” Sep. 2018, arXiv:1809.08748.
  * [47] U. Saleem, S. Jangsher, H. K. Qureshi, and S. A. Hassan, ”Joint subcarrier and power allocation in the energyharvesting-aided d2d communication,” _IEEE Trans. on Industrial Informatics_ , vol. 14, no. 6, pp. 2608-2617, June 2018.
  * [48] A. Y. Awan, M. Ali, M. Naeem, F. Qamar and M. N. Sial, ”Joint Network Admission Control, Mode Assignment and Power Allocation in Energy Harvesting aided D2D Communication,” in _IEEE Trans. on Industrial Informatics_. doi: 10.1109/TII.2019.2922667.
  * [49] C. Yang, X. Xu, J. Han, and X. Tao, ”GA based user matching with optimal power allocation in D2D underlaying network,” in IEEE Veh. Technol. Conf., pp. 1-5, January 2014.
  * [50] C. Xu et al., “Efficiency resource allocation for device-to-device underlay communication systems: A reverse iterative combinatorial auction based approach,” _IEEE J. Sel. Areas Commun._ , vol. 31, no. 9, pp. 348-358, 2013.
  * [51] R. Wang, J. Zhang, S. H. Song, and K. B. Letaief, ”QoS-Aware channel assignment for weighted sum-rate maximization in D2D communications,” _IEEE Glob. Commun. Conf. GLOBECOM 2015_ , no. pp. 1-6, 2015.
  * [52] Qualcomm, “LTE Direct Overview”, ”https://www.qualcomm.com/documents/lte-direct-overview”
  * [53] WiFi Alliance, “WiFi Direct - The worldwide network of companies that brings you Wi-Fi”, ”https://www.wi-fi.org/discover-wi-fi/wi-fi-direct”
  * [54] Xiao, S., Feng, D., Yuan-wu, Y., Li, G. Y., Guo, W., & Li, S. (2015). Optimal Mobile Association in Device-to-Device-Enabled Heterogeneous Networks. (2014).
  * [55] 3GPP TR 36-942 - V14.0.0 - LTE Evolved Universal Terrestrial Radio Access (E-UTRA); Radio Frequency (RF) system scenarios.


(eccv) Package eccv Warning: Package ‘hyperref’ is loaded with option
‘pagebackref’, which is *not* recommended for camera-ready version

11institutetext: Australian Institute for Machine Learning, The University of
Adelaide 22institutetext: Zhejiang University
33institutetext: Monash University

# Streaming Video Diffusion: Online Video Editing with Diffusion Models

Feng Chen1† Zhen Yang2† Bohan Zhuang3‡ Qi Wu1‡

###### Abstract

We present a novel task called online video editing, which is designed to edit
streaming frames while maintaining temporal consistency. Unlike existing
offline video editing assuming all frames are pre-established and accessible,
online video editing is tailored to real-life applications such as live
streaming and online chat, requiring (1) fast continual step inference, (2)
long-term temporal modeling, and (3) zero-shot video editing capability. To
solve these issues, we propose Streaming Video Diffusion (SVDiff), which
incorporates the compact spatial-aware temporal recurrence into off-the-shelf
Stable Diffusion and is trained with the segment-level scheme on large-scale
long videos. This simple yet effective setup allows us to obtain a single
model that is capable of executing a broad range of videos and editing each
streaming frame with temporal coherence. Our experiments indicate that our
model can edit long, high-quality videos with remarkable results, achieving a
real-time inference speed of 15.2 FPS at a resolution of $512\times 512$. Our
code will be available at https://github.com/Chenfeng1271/SVDiff.

###### Keywords:

Video editing Streaming processing Diffusion

†† † Co-first authors.
‡ Corresponding authors<EMAIL_ADDRESS><EMAIL_ADDRESS>
##  1 Introduction

Video editing [33, 11, 38] plays a ubiquitous role in creating fascinating
visual effects for films, short videos, _etc_. Recent advancements [5, 23]
have predominantly concentrated on offline video editing (as shown in Fig. 1
(a)), wherein the entire video is edited simultaneously, assuming that all
frames are pre-established and accessible. However, as shown in Fig. 1 (b),
editing streaming frames of video for immediate response to visual data, which
we call online video editing, is still underexplored. It is important for many
real-life usage scenarios such as live streaming and online chat. As a result,
there is an increasing demand for easy-to-use and performant online video
editing tools.

Figure 1: Comparison between offline and online video editing. Offline video
editing processes the whole video simultaneously and regards all frames as
known. Online video editing operates each streaming frame with the temporal
information from previous frames in a causal way.

Recently, thanks to the introduction of powerful text-conditioned diffusion
models [27, 13, 28] trained on large-scale datasets, video editing algorithms
have achieved unprecedented processes in offline video editing by extending
Text-to-Image (T2I) to Text-to-Video (T2V) diffusion. Typically, sparse causal
attention [5, 31, 25] and temporal module [33, 22, 6] are added to model
temporal dynamics, but they are insufficient for online video editing due to
short-term temporal modeling and accessing future frames, respectively [30,
10]. Therefore, it is still challenging to extend this success to online video
editing. We summarize these challenges as three-fold. 1): The multi-step
denoising of diffusion significantly increases the computational redundancy of
cached memory and recurrent calculation, which is difficult for fast continual
inference. 2): Online video streams usually have an extended video sequence,
which requires long-term temporal modeling. However, training a model on long
videos is non-trivial. 3): For online video editing to be both practical and
effective, each model must possess zero-shot video editing capabilities,
allowing the editing of any video in response to any edit prompt.

To address these issues, one straightforward approach is to adapt existing
zero-shot offline methods to the causal online setting. These zero-shot
offline methods can be classified into two types: tuning-free based and
pretrained-based. Tuning-free based methods [32, 5] apply additional controls
(such as replacing spatial attention with sparse causal attention) to maintain
frame-to-frame consistency. These adjustments primarily preserve temporal
coherence over short spans [39], which makes them suitable for editing brief
videos, but less effective for longer sequences. To adapt these tuning-free
methods for online video editing, it is essential to use all previous frames
for cross-frame interaction. However, the amount of data involved is
substantial, especially in videos of high resolution and long duration,
leading to a drastic increase in memory consumption [10]. The other type of
offline method called pretrained-based methods [22, 29], turns to training the
video diffusion model on a large-scale text-video dataset to model temporal
dynamics. Typically, temporal attention [6] is added to the denoising model,
fostering inter-frame interaction. The parallel calculation of the attention
module regards all frames as known, which is contradictory with online video
editing because a future frame is not available at the current time. A simple
way to adapt these pretrained-based methods to online video editing is to add
causal attention masks to their temporal attention. However, this approach
involves recalculating previous frames for every new frame in the stream, a
process that is ill-suited for rapid inference. Recently, LLaVA [18]
introduced a method that caches previous key and value states to bypass
repetitive computations. The drawback of this strategy is the increased memory
requirement for storage, particularly noticeable during multi-step denoising.

In this paper, we propose an online video diffusion model with recursive
spatial-aware temporal memory, named streaming video diffusion (SVDiff), to
balance the trade-off between computational cost and long-range temporal
modeling. Specifically, as shown in Fig. 2, we first initialize a learnable
spatial-aware temporal memory embedding and recursively process it with
streaming frames. It essentially serves as a dynamic temporal cache and is
continuously updated by memory attention to encode both the individual content
of each frame’s spatial layout and the inter-frame motion trajectory within
the video stream. Therefore, it allows for trivial computational cost and
long-range temporal modeling for online processing, resulting from compact
memory and recurrent operation. After that, we adopt the segment-level scheme
[30] that deconstructs a long video into a series of short video clips for
efficient long video training. Apart from updating and processing memory
within each segmented clip, we also propagate the temporal memory between
consecutive clips, transferring the temporal history to the following video
frames. Unlike previous methods that usually edit 16-frame videos, this setup
allows us to train a single model that is capable of executing videos with 150
frames and editing each streaming frame with temporal consistency.

To sum up, we make three main contributions: 1) We propose online video
editing, a novel task for immediate editing response of streaming video. 2) We
propose SVDiff, an online video diffusion model with recursive spatial-aware
temporal memory. 3) Our method efficiently generates high-quality, long
videos, ensuring both global and local coherence, while maintaining a real-
time inference speed of 15.2 FPS with a resolution of $512\times 512$.

##  2 Related Work

Pretrained-based video editing. Despite considerable progress in zero-shot
text-guided image editing [12], editing arbitrary videos according to text
remains a difficult task due to the lack of large-scale high-quality text-
video datasets and the complexity of modeling temporal consistency. To solve
this issue, Dreamix [23] and LAMP [35] propose to train a video diffusion
model over T2I diffusion on a small dataset for video editing. In addition,
FollowYourPose [22] learns a separate pose branch to maintain the structure of
the video. Recently, Videocrafter1 [6] introduced a high-resolution video
diffusion pre-trained on a large-scale text-video dataset, which can preserve
content, style and motion consistency. These methods usually employ temporal
modules, such as attention [6], LoRA [22], and convolution [29], to capture
temporal changes, but these modules treat all frames as known, which is
inconsistent with online video editing.

Tuning-free video editing. Another way for zero-shot video editing is to
modify T2I diffusion in a tuning-free design. For example, to achieve temporal
consistency, Fate-Zero [25] and Video-P2P [19] replace spatial attention in
U-Net with sparse causal attention and apply attention control proposed in
prompt2prompt [12]. Pix2Video [5] adds additional regularization to penalize
dramatic frame changes. Text2Video-Zero [16] first proposes to edit the video
frames with only the pre-trained T2I diffusion model and then modify the
latent feature with motion dynamic through sparse causal attention and object
mask. However, because only a few frames used in cross-frame attention, these
methods still struggle to maintain long-range temporal consistency.

Online video models. Online video processing refers to the analysis and
manipulation of video content as it is being streamed or captured, enabling
immediate interpretation and response to visual data. Existing methods mainly
focus on online video recognition with recursive operation [37], temporal
shift [20], sliding window [1], and augmented memory [39]. For example, Yang
et al. [37] use a recurrent attention gate to aggregate the information
between the current frame and previous frames. [39] caches the key-value of
previous frames, acting as a temporal reference in cross-frame attention. [20]
selects the informative tokens from each frame and then temporally shifts them
across the adjacent frames. However, in online video editing with diffusion
models, multi-step diffusion denoising poses significant challenges in
modeling long-term motion trajectories and achieving fast inference. In this
paper, we propose a novel method using compact temporal recurrence to solve
this issue.

##  3 Preliminary

Stable Diffusion [28, 24] is a latent diffusion model operating within the
latent space of an autoencoder. We denote the autoencoder as
$\mathcal{D}(\mathcal{E}(\cdot))$, where $\mathcal{E}$ and $\mathcal{D}$
correspond to the encoder and decoder, respectively. Taking an input image $I$
and its corresponding latent feature ${\mathbf{x}}_{0}$ obtained through the
encoder ${\mathbf{x}}_{0}=\mathcal{E}(I)$, the core of the diffusion process
is to iteratively introduce noise to this latent representation. This is
achieved through the following equation:

$q({\mathbf{x}}_{t}\mid{\mathbf{x}}_{t-1})=\mathcal{N}({\mathbf{x}}_{t};\sqrt{\alpha_{t}}{\mathbf{x}}_{t-1},(1-\alpha_{t})\mathbf{I}),$
(1)

where $t=1,...,T$ is the time step,
$q({\mathbf{x}}_{t}\mid{\mathbf{x}}_{t-1})$ is the conditional density of
${\mathbf{x}}_{t}$ given ${\mathbf{x}}_{t-1}$, and $\alpha_{t}$ is a
hyperparameter to scale noise. Alternatively, at any given time step,
${\mathbf{x}}_{t}$ can be sampled from ${\mathbf{x}}_{0}$ using the following
equation:

$q({\mathbf{x}}_{t}\mid{\mathbf{x}}_{0})=\mathcal{N}({\mathbf{x}}_{t};\sqrt{\bar{\alpha}_{t}}{\mathbf{x}}_{0},(1-\bar{\alpha}_{t})\mathbf{I}),$
(2)

where $\bar{\alpha}_{t}=\prod_{i=1}^{t}\alpha_{i}$. In the diffusion backward
process, a U-Net denoted as $\epsilon_{\theta}$ is trained to predict the
noise in the latent representation, aiming to iteratively recover
${\mathbf{x}}_{0}$ from ${\mathbf{x}}_{T}$. As the number of diffusion steps,
denoted as $T$, increases, ${\mathbf{x}}_{0}$ becomes progressively noisier
due to the noise introduced in the forward process. This noise accumulation
causes ${\mathbf{x}}_{T}$ to approximate a standard Gaussian distribution.
Consequently, $\epsilon_{\theta}$ is designed to learn how to deduce a valid
${\mathbf{x}}_{0}$ from these Gaussian noises. Given $c_{p}$ is the text
prompt, the predicted ${\mathbf{x}}_{0}$, denoted as
$\hat{{\mathbf{x}}}_{t\rightarrow 0}$ at time step $t$, can be estimated using
the following equation:

$\hat{{\mathbf{x}}}_{t\rightarrow
0}=({\mathbf{x}}_{t}-\sqrt{1-\alpha_{t}}\epsilon_{\theta}({\mathbf{x}}_{t},t,c_{p}))/\sqrt{\alpha_{t}},$
(3)

where $\epsilon_{\theta}({\mathbf{x}}_{t},t,c_{p})$ is the predicted noise of
${\mathbf{x}}_{t}$ guided by the text prompt $c_{p}$ and the time step $t$.
Meanwhile, the reconstruction loss between the real noise $\epsilon_{t}$ and
$\epsilon_{\theta}({\mathbf{x}}_{t},t,c_{p})$ is calculated for training:

$\mathcal{L}=||\epsilon_{t}-\epsilon_{\theta}({\mathbf{x}}_{t},t,c_{p})||_{2}^{2}.$
(4)

Figure 2: Overview of Streaming Video Diffusion. (a) We propose spatial-aware
temporal memory which is inserted with memory attention after each transformer
block in Stable Diffusion. Then our method is trained on large-scale long
videos by splitting the long video into short clips. (b) During inference, we
denoise the noisy latent of streaming frame with classifier-free guidance
(CFG) where each denoising step involves the U-Net conducting conditional and
unconditional denoising with corresponding memory.

To achieve fast editing, we obtain the edited latent representation
$\hat{{\mathbf{x}}}_{0}$, by sampling from the noise ${\mathbf{x}}_{T}$, which
is derived from ${\mathbf{x}}_{0}$ through LCM Inversion [21]. This process
employs LCM LoRA [21] for efficient few-step denoising. The final edited
image, $I^{\prime}$, is then generated by decoding $\hat{{\mathbf{x}}}_{0}$
using the decoder $\mathcal{D}(\hat{{\mathbf{x}}}_{0})$. For each denoising
step, we apply classifier-free guidance (CFG) [14] to balance the fidelity and
controllability using the following linear combination of the conditional and
unconditional score estimates:

$\hat{\epsilon}_{t}=(1+\lambda)\underbrace{\epsilon_{\theta}({\mathbf{x}}_{t},t,c_{p})}_{\text{conditional}}-\lambda\underbrace{\epsilon_{\theta}({\mathbf{x}}_{t},t,\varnothing)}_{\text{unconditional}},$
(5)

where $\lambda$ is the coefficient factor and $\varnothing$ is the null text
embedding.

##  4 Method

To balance the tradeoff between computational cost and long-range temporal
modeling, we introduce a novel approach known as SVDiff for online video
editing. In Sec. 4.1, we first give an overview of the proposed approach. In
Sec. 4.2, we introduce spatial-aware temporal memory that is designed for
online video editing. Then, In Sec. 4.3, we elaborate the training and
inference procedure.

###  4.1 Overview

Our method aims to extend the T2I diffusion to T2V diffusion for online video
editing by incorporating compact temporal recurrence. The overview of SVDiff
is illustrated in Fig. 2. In Fig. 2 (a), we first insert a learnable spatial-
aware temporal memory with recurrent memory attention (as explained in Sec.
4.2) after each transformer block of Stable Diffusion. This memory functions
as a dynamic, temporal cache, constantly updated to capture the details of
each video frame. Given a video $\mathcal{V}=\\{\vec{V}^{i}\\}_{i=1}^{N}$ with
$N$ frames, we split it to $K$ short clips after inversion where
$\mathcal{V}=\\{\vec{S}^{i}\\}_{i=1}^{K}$. Then we sequentially process each
video clip with recursive spatial-aware temporal memory and calculate the
reconstruction loss for training. In this way, SVDiff can efficiently learn
compact temporal memory over long videos. During inference, as shown in Fig. 2
(b), each streaming frame $\vec{V}^{i}$ undergoes multiple denoising steps in
which every step keeps a conditional and unconditional memory to maintain
content and motion consistency.

###  4.2 Spatial-aware Temporal Memory

We propose a spatial-aware temporal memory which is a learnable temporal
embedding that recursively captures and updates temporal information from
previous frames [4]. We represent the temporal history until the $n$-th frame
as a learnable memory embedding $\vec{M}^{n}\in\mathbb{R}^{h\times w\times d}$
where $h\times w$ is the spatial dimension and $d$ is the feature dimension.
We note that simply increasing the size of this learnable memory does not
inherently provide order and structural correlation with the spatial layout of
frames, making it inadequate for capturing the motion trajectories of
individual objects. Therefore, for the feature $\vec{m}_{ij}$ in position
$(i,j)$ of $\vec{M}^{n}$, we augment it with the position embedding to enhance
spatial awareness of memory, implicitly analogous to the spatial layout of the
frame feature. Following [9, 7], we add positional embedding in $\vec{M}^{n}$
where the position of $\vec{m}_{ij}$ is computed relatively to the center of
the map as $(i-\frac{h}{2},j-\frac{w}{2})$. Therefore, such a memory shares a
spatial structure similar to that of frame features. In detail, at the
beginning of a video in each timestep, we initialize $\vec{M}^{0}$ using the
grid position:

$\vec{m}_{ij}=\vec{w}^{0}+\text{FFN}([i-\frac{h}{2},j-\frac{w}{2}]),$ (6)

where $\vec{w}^{0}\in\mathbb{R}^{d}$ is a learnable embedding and FFN is a
feed-forward network consisting of two MLP layers. Then, the frame features
can be aligned with historical motion by temporal memory and recurrent memory
attention [4] as:

$[\vec{F}^{n};\vec{M}^{n+1}]=\text{Attn}([\vec{F}^{n};\vec{M}^{n}]),$ (7)

where $\vec{F}^{n}$ is the $n$-th frame features from the output of
transformer block and $[\cdot;\cdot]$ is the concatenation operation.
$\vec{M}^{n+1}$ is the updated memory for the next frame and
$\vec{M}^{0}=[\vec{m}_{ij}]\in\mathbb{R}^{h\times w\times c}$. Memory
attention $\text{Attn}(\cdot)$ is a standard self-attention module processing
$[\vec{F}^{n};\vec{M}^{n}]$ along the spatial dimension.

Compared to explicit temporal memory collected from the key-value of previous
frames [39], our spatial-aware temporal memory is more efficient in (1)
condensing historical information in a compact memory and (2) learning
temporal memory in different denoising time steps. This spatial-aware temporal
memory is integrated into the original U-Net following each Transformer block.

###  4.3 Efficient Training and Inference

We train SVDiff on a large-scale, long video dataset [2] to enable online
video editing with extended streams. Unlike existing methods [33, 22] which
are usually trained with 16-frame videos due to memory limits, we propose to
solve this issue by splitting each long video into several short video clips.
Given the $k$-th clip $\vec{S}^{k}$, we sequentially align each frame feature
$\vec{F}^{i}$ in $\vec{S}^{k}$ with temporal memory by Eq. 7 and calculate the
reconstruction loss using Eq. 4 between the predicted noise and the real noise
latent of each frame. Moving to next clip, we propagate memory from the output
of the last frame of $\vec{S}^{k}$ to the beginning frame of $\vec{S}^{k+1}$.
Therefore, the historical temporal information is still accessible in the
following clips. This process recursively continues until all frames are
involved in the training. In our method, we selectively update the learnable
spatial-aware memory and its associated memory attention, as shown in Fig. 2
(a). This is designed to enhance computational efficiency while preserving the
original property of pre-trained T2I diffusion.

During the inference stage, we first inverse the original video frame by frame
into the noisy latent $\vec{x}_{T}$ and then use the classifier-free guidance
(CFG) [14] to achieve denoising of the streaming frames. Specifically, as
shown in Fig. 2 (b), each denoising step involves the U-Net conducting two
separate predictions: one for the conditional denoising and the other for the
unconditional denoising, which are denoted by subscript $c$ and $uc$
respectively. Therefore, we designate conditional and unconditional memory
$\vec{M}_{c}$ and $\vec{M}_{uc}$ for them separately. Given the $n$-th frame,
we can obtain estimated noise $\epsilon_{t}^{n}$ by modifying Eq. 5 into:

$\displaystyle\hat{\epsilon}_{t,c}^{n},\vec{M}^{n+1}_{c}$
$\displaystyle=\epsilon_{\theta}({\mathbf{x}}_{t}^{n},t,c_{p},\vec{M}_{c}^{n}),$
(8) $\displaystyle\hat{\epsilon}_{t,uc}^{n},\vec{M}^{n+1}_{uc}$
$\displaystyle=\epsilon_{\theta}({\mathbf{x}}_{t}^{n},t,\varnothing,\vec{M}_{uc}^{n}),$
$\displaystyle\hat{\epsilon}_{t}^{n}=(1$
$\displaystyle+\lambda)\hat{\epsilon}_{t,c}^{n}-\lambda\hat{\epsilon}_{t,uc}^{n},$

where $\epsilon_{\theta}$ denotes the denoising U-Net.

##  5 Results

###  5.1 Experimental Settings

Implementation details. Our experiment is based on Stable Diffusion 1.5 with
the public pretrained weights [28]. We trained our model for 20k iterations
using a subset of the HDVILA dataset [36], which comprises approximately 2
million subtitled videos. We sample 64 consecutive frames at a resolution of
$512\times 512$ from the input video for temporal consistency learning. To
improve the efficiency of long video training, we divide the video into
several video clips with 8 frames each. All clips are associated with the same
video caption. The training process is performed on 8 NVIDIA Tesla A100 GPUs
and can be completed in eight days. For spatial-aware temporal memory, we
empirically set $h=w=8$. During inference, we utilize the LCM sampler combined
with LCM LoRA [21] in 3 denoising steps to enhance inference speed. To further
accelerate the infernce speed, we implement our method with TensorRT and tiny
AutoEncoder in StreamDiffusion [17]. Inference speed testing is conducted on
an RTX 4090 GPU with images of $512\times 512$ resolution. Following [30], we
assess our approach on a dataset comprising 66 videos with lengths ranging
from 32 to 150 frames. These videos are mostly drawn from the TGVE competition
[34] and the Internet.

Baseline models. There are generally four kinds of methods designed for online
processing. We adopt these methods in online video editing as baseline models
to verify the effectiveness of our method, including Efficient Attention: We
use efficient temporal attention with recurrent attention masks [15]. Window
Attention: We cache the key-value of previous three frames for cross-frame
interaction [39, 3]. Temporal Shift: Following [20], we inject temporal
information by exchanging channel features with adjacent frames. Sliding
Window: We use a fixed-length time window that incrementally moves over a
video to analyze portions of it sequentially [1, 8]. We elaborate the
description and implementation details of these four baseline methods in Sec.
1 of supplementary.

Evaluation metrics. Following [33], we evaluate the performance of our method
using CLIP metrics [26] and user studies. To assess temporal consistency, we
calculate CLIP image embeddings for all frames in the output videos and report
the average cosine similarity between pairs of video frames. For evaluating
editing frame accuracy, we compute the average CLIP score between the frames
of the output videos and their corresponding edited prompts. In addition, we
conduct three user study metrics, referred to as ‘Edit’, ‘Image’, and ‘Temp’,
to gauge the editing quality, overall frame-wise image fidelity, and temporal
consistency of the videos, respectively. We conduct comparisons based on
specific criteria between pairs of videos generated. In our user study, we
enlist the feedback of 20 participants for each example and determine the
final result based on majority voting.

Figure 3: Qualitative editing results of long videos where the red number in
the lower right corner denotes the frame index.

###  5.2 Editing Results

Fig. 3 shows the application of our SVDiff on online video editing with
extended video sequence. We note that it successfully produces high-quality
videos closely aligned with the given text prompts. Specifically, SVDiff
demonstrates proficiency in modifying global environments (grassland
$\rightarrow$ desert, city $\rightarrow$ blazing flams), object attribute
(dress $\rightarrow$ red dress, car $\rightarrow$ jeep car), and style (van
gogh style). It showcases the practical applicability of our method in the
realm of online video editing. For more editing examples, please refer to the
video in supplementary.

Figure 4: Visual comparison between baseline models and our method where the
edit prompt is “a rabbit is eating pizza".

###  5.3 Comparison to Baseline Models

Table 1: Quantitative comparison with other baseline models. We omit GFLOPs
and FPS of efficient attention based method, since it is quadratic with the
number of previous frames in streaming process.

Method | CLIP Metrics$\uparrow$ | User Study$\downarrow$ | Efficiency
---|---|---|---
Tem-Con | Frame-Acc | Edit | Image | Temp | GFLOPs | Params(MB) | FPS
Efficient Attention [15] | 90.87 | 27.34 | 4.35 | 3.79 | 2.60 | - | 344.7 | -
Window Attention [39] | 90.40 | 27.67 | 3.48 | 3.17 | 3.15 | 15520 | 344.7 | 12.3
Temporal Shift [20] | 91.67 | 27.56 | 2.70 | 3.42 | 4.37 | 12946 | 295.1 | 18.0
Sliding Window[8] | 90.82 | 27.37 | 2.64 | 2.85 | 3.10 | 46554 | 344.7 | 3.6
SVDiff(ours) | 93.20 | 27.97 | 1.82 | 1.76 | 1.78 | 15833 | 344.8 | 15.2

Quantitative results. We provide quantitative comparisons with other baseline
models in Tab. 1. Our SVDiff model demonstrates a notable improvement in the
trade-off between performance and efficiency when compared to baseline models.
Specifically, SVDiff outperforms the temporal shift-based method, achieving a
1.53% improvement in temporal consistency with only extra 2,887 GFLOPs.
Furthermore, the addition of 49.7MB in parameters, attributed to the
incorporation of learnable memory embedding and memory attention in our model,
is on par with the augmentations seen in efficient attention-based and sliding
window-based approaches. Notably, our method requires significantly fewer
GFLOPs and attains a real-time inference speed of 15.2 FPS. This enhanced
performance is attributed to SVDiff’s recurrent temporal modeling, which
efficiently integrates all previous temporal information into a spatial-aware
compact memory.

Qualitative results. We present a visual comparison in Fig. 4 against baseline
models to qualitatively assess the improvement of our method. Our method
(bottom row) produces videos that better adhere to the edit prompt and
preserve the temporal consistency of the edited video, while other methods
struggle to meet both of these goals. For example, the efficient attention-
based method has a sudden change in the pizza where the third and fourth
figures of the first row have different appearances and shapes. Window
attention and sliding window-based methods preserve temporal adherence between
adjacent frames but suffer from long-term temporal inconsistency in the rabbit
face and pizza texture. Moreover, the temporal shift-based method can maintain
general style across frames, however, the temporal consistency in detailed
texture is still poor (pizza in forth row).

Figure 5: Performance comparison with baseline models in long video editing
with different video lengths.

Video editing with different lengths. We test the performance of video editing
with different lengths in Fig. 5. We observe that efficient attention, window
attention, and sliding window based methods meet various degrees of
performance decline in editing long videos. However, the reasons are
different. Specifically, the efficient attention-based method is limited by
the training-inference gap where once the video length of inference is larger
than that of training, the performance decreases sharply. For window attention
and sliding window-based methods, they are largely influenced by the window
size that accesses previous frames. Besides, the temporal shift-based method
is stable in editing videos of different lengths, but its training-free
temporal modeling strategy can only provide implicit nearby temporal
information. In addition, our method achieves superior performance over them
because of our recursive spatial-aware temporal memory and training with
longer videos.

###  5.4 Comparison to Existing Models

Figure 6: Visual comparison with existing methods that adapt to online video
edit where the edit prompt is “a car is moving on snow road".

Quantitative results. As discussed in Sec. 1, existing tuning-free and
pretrained-based methods can be adapted to online video editing. Therefore, we
compare the modified pre-trained method (FollowYourPose [22]) and tuning-free
based methods (vid2vid-zero [31] and Pix2video [5]) with our method on online
video editing in Tab. 2. The implementation details of these methods are in
Sec. 1 of supplementary. Our method demonstrates a significant edge over the
other three in terms of CLIP metrics and user study. Specifically, it
surpasses FollowYourPose by 3.49% in temporal consistency and by 2.15 in
editing quality. This improvement is attributed to the consistent recurrent
operations of our method applied to long videos during both training and
inference. Moreover, our method outperforms Pix2video and vid2vid-zero as
well. We ascertain that their reliance on adjacent frames for sparse causal
attention falls short in modeling long-term motion trajectories.

Qualitative results. We present a visual comparison in Fig. 6 against existing
methods that are adapted to online video editing to assess the improvement of
our method. Our method, depicted in the bottom row, excels in adhering to the
edit prompt while maintaining the temporal consistency of the edited video. In
contrast, FollowYourPose [22] exhibits notable challenges in preserving
original motion and content integrity. Pix2video [5] tends to generate visuals
of inferior quality, marked by blurriness and inconsistencies in object
continuity. Additionally, vid2vid-zero [31] demonstrates a clear disparity,
particularly in the representation of a car: While the final image features a
snow-covered vehicle, the car appears clean in the preceding images. These
comparisons underscore our method’s unique ability to keep edit adherence and
temporal consistency, outperforming existing approaches in online video
editing scenarios.

Table 2: Quantitative comparison with existing methods that adapt to online
video editing.

Method CLIP Metrics$\uparrow$ User Study$\downarrow$ Tem-Con Frame-Acc Edit
Image Temp FollowYourPose [22] 89.71 26.70 3.70 3.52 3.28 vid2vid-zero [31]
91.68 27.88 2.66 2.05 1.70 Pix2video [5] 91.27 27.61 2.10 3.02 3.62
SVDiff(ours) 93.20 27.97 1.55 1.41 1.40

###  5.5 Ablation Study

Training with longer videos. One benefit of our SVDiff is training on longer
videos. In Fig. 7, we ablate the influence of video length during training. By
increasing the length of the video from 8 to 64, the temporal consistency
increases monotonically with a gain of 2.5%, indicating the benefit of
training on long videos. However, for frame accuracy, such improvement becomes
negligible where largely increasing video length only brings a gain of 0.15%.
This is because the temporal module trained on long videos can effectively
learn temporal coherence, but the editing ability is largely determined by the
base model which is frozen during training.

Figure 7: Ablation study on video length during training.

Spatial-aware temporal memory. The effectiveness of our proposed spatial-aware
memory is analyzed in Fig. 8. We observe that employing spatial-aware memory
retains the intricate skeleton of the horse. In contrast, omitting positional
embedding leads to noticeable losses, such as the disappearance of the dog’s
legs in the third row of Fig. 8. Further, the removal of both positional
embedding and grid format, akin to using a global token, results in the model
maintaining only the broad content and style across frames. This causes
inconsistencies like fluctuating dog sizes and shifting sea positions. These
observations demonstrate the critical role of our proposed memory in
preserving the detailed spatial layout of each frame and the inter-frame
motion trajectory within the video stream.

Table 3: Ablation on memory size of spatial-aware temporal memory. Memory size | Tem-Con | Frame-Acc
---|---|---
1$\times$1 | 89.96 | 26.30
8 $\times$8 | 92.91 | 27.94
16 $\times$ 16 | 92.70 | 27.59

Moreover, in Tab. 3, we ablate the memory size of our spatial-aware temporal
memory $\vec{M}^{n}$. A smaller memory size (1$\times$1) lacks the granularity
needed to effectively model spatial variations and temporal transitions,
resulting in lower performance. Conversely, a larger memory size
(16$\times$16) introduces redundancy and potential overfitting to specific
frame details.

Figure 8: Ablation on spatial-aware temporal memory. The edit prompt is “a dog
is running on the beach".

##  6 Conclusion

In this paper, we have presented a new task called online video editing, which
is designed to edit streaming frames while preserving temporal consistency. To
this end, we have proposed Streaming Video Diffusion (SVDiff) to address the
three challenges of this task by integrating compact spatial-aware temporal
recurrence into existing Stable Diffusion. To train our method on long videos,
we divide the video into short clips while preserving the long-term temporal
coherence with the help of a compact temporal recurrence module. The
experiments show that our SVDiff produces high-quality long videos with both
global and local coherence and reduces the computation cost for streaming
processing compared to baseline methods.

Limitations and future work. Although in theory we can process videos of any
length using temporal recurrence, our current method may not be able to
accurately detect shot changes in videos longer than 2 minutes with thousands
of frames, particularly those with discontinuous backgrounds and complex
motion. This limitation largely stems from the gap between training and
inference of video frames. Therefore, our future work will focus on
investigating strategies to alleviate this influence, with the goal of
efficiently processing long videos that feature complex scene transitions.

## References

  * [1] Ai, X., Sheng, V.S., Li, C., Cui, Z.: Class-attention video transformer for engagement intensity prediction. arXiv preprint arXiv:2208.07216 (2022)
  * [2] Bain, M., Nagrani, A., Varol, G., Zisserman, A.: Frozen in time: A joint video and image encoder for end-to-end retrieval. In: ICCV (2021)
  * [3] Beltagy, I., Peters, M.E., Cohan, A.: Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150 (2020)
  * [4] Bulatov, A., Kuratov, Y., Burtsev, M.: Recurrent memory transformer. NeurIPS 35, 11079–11091 (2022)
  * [5] Ceylan, D., Huang, C.H.P., Mitra, N.J.: Pix2video: Video editing using image diffusion. arXiv preprint arXiv:2303.12688 (2023)
  * [6] Chen, H., Xia, M., He, Y., Zhang, Y., Cun, X., Yang, S., Xing, J., Liu, Y., Chen, Q., Wang, X., et al.: Videocrafter1: Open diffusion models for high-quality video generation. arXiv preprint arXiv:2310.19512 (2023)
  * [7] Chen, S., Chabal, T., Laptev, I., Schmid, C.: Object goal navigation with recursive implicit maps. arXiv preprint arXiv:2308.05602 (2023)
  * [8] Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q.V., Salakhutdinov, R.: Transformer-xl: Attentive language models beyond a fixed-length context. arXiv preprint arXiv:1901.02860 (2019)
  * [9] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
  * [10] Harvey, W., Naderiparizi, S., Masrani, V., Weilbach, C., Wood, F.: Flexible diffusion modeling of long videos. NeurIPS 35, 27953–27965 (2022)
  * [11] He, Y., Yang, T., Zhang, Y., Shan, Y., Chen, Q.: Latent video diffusion models for high-fidelity video generation with arbitrary lengths. arXiv preprint arXiv:2211.13221 (2022)
  * [12] Hertz, A., Mokady, R., Tenenbaum, J., Aberman, K., Pritch, Y., Cohen-Or, D.: Prompt-to-prompt image editing with cross attention control. In: ICLR (2023)
  * [13] Ho, J., Chan, W., Saharia, C., Whang, J., Gao, R., Gritsenko, A., Kingma, D.P., Poole, B., Norouzi, M., Fleet, D.J., Salimans, T.: Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303 (2022)
  * [14] Ho, J., Salimans, T.: Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598 (2022)
  * [15] Katharopoulos, A., Vyas, A., Pappas, N., Fleuret, F.: Transformers are rnns: Fast autoregressive transformers with linear attention. In: ICML. pp. 5156–5165 (2020)
  * [16] Khachatryan, L., Movsisyan, A., Tadevosyan, V., Henschel, R., Wang, Z., Navasardyan, S., Shi, H.: Text2video-zero: Text-to-image diffusion models are zero-shot video generators. arXiv preprint arXiv:2303.13439 (2023)
  * [17] Kodaira, A., Xu, C., Hazama, T., Yoshimoto, T., Ohno, K., Mitsuhori, S., Sugano, S., Cho, H., Liu, Z., Keutzer, K.: Streamdiffusion: A pipeline-level solution for real-time interactive generation. arXiv preprint arXiv:2312.12491 (2023)
  * [18] Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. Advances in neural information processing systems 36 (2024)
  * [19] Liu, S., Zhang, Y., Li, W., Lin, Z., Jia, J.: Video-p2p: Video editing with cross-attention control. arXiv preprint arXiv:2303.04761 (2023)
  * [20] Liu, Y., Xiong, P., Xu, L., Cao, S., Jin, Q.: Ts2-net: Token shift and selection transformer for text-video retrieval. In: ECCV. pp. 319–335 (2022)
  * [21] Luo, S., Tan, Y., Huang, L., Li, J., Zhao, H.: Latent consistency models: Synthesizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04378 (2023)
  * [22] Ma, Y., He, Y., Cun, X., Wang, X., Shan, Y., Li, X., Chen, Q.: Follow your pose: Pose-guided text-to-video generation using pose-free videos. arXiv preprint arXiv:2304.01186 (2023)
  * [23] Molad, E., Horwitz, E., Valevski, D., Acha, A.R., Matias, Y., Pritch, Y., Leviathan, Y., Hoshen, Y.: Dreamix: Video diffusion models are general video editors. arXiv (2023)
  * [24] Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., Müller, J., Penna, J., Rombach, R.: Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952 (2023)
  * [25] Qi, C., Cun, X., Zhang, Y., Lei, C., Wang, X., Shan, Y., Chen, Q.: Fatezero: Fusing attentions for zero-shot text-based video editing. arXiv preprint arXiv:2303.09535 (2023)
  * [26] Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: ICML. pp. 8748–8763 (2021)
  * [27] Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., Chen, M.: Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 (2022)
  * [28] Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: CVPR. pp. 10684–10695 (2022)
  * [29] Singer, U., Polyak, A., Hayes, T., Yin, X., An, J., Zhang, S., Hu, Q., Yang, H., Ashual, O., Gafni, O., Parikh, D., Gupta, S., Taigman, Y.: Make-A-Video: Text-to-video generation without text-video data. In: ICLR (2023)
  * [30] Wang, F.Y., Chen, W., Song, G., Ye, H.J., Liu, Y., Li, H.: Gen-l-video: Multi-text to long video generation via temporal co-denoising. arXiv preprint arXiv:2305.18264 (2023)
  * [31] Wang, T.C., Liu, M.Y., Zhu, J.Y., Liu, G., Tao, A., Kautz, J., Catanzaro, B.: Video-to-video synthesis. In: NeurIPS (2018)
  * [32] Wu, C., Liang, J., Ji, L., Yang, F., Fang, Y., Jiang, D., Duan, N.: Nüwa: Visual synthesis pre-training for neural visual world creation. In: ECCV. pp. 720–736. Springer (2022)
  * [33] Wu, J.Z., Ge, Y., Wang, X., Lei, S.W., Gu, Y., Hsu, W., Shan, Y., Qie, X., Shou, M.Z.: Tune-A-Video: One-shot tuning of image diffusion models for text-to-video generation. arXiv preprint arXiv:2212.11565 (2022)
  * [34] Wu, J.Z., Li, X., Gao, D., Dong, Z., Bai, J., Singh, A., Xiang, X., Li, Y., Huang, Z., Sun, Y., He, R., Hu, F., Hu, J., Huang, H., Zhu, H., Cheng, X., Tang, J., Shou, M.Z., Keutzer, K., Iandola, F.: Cvpr 2023 text guided video editing competition (2023)
  * [35] Wu, R., Chen, L., Yang, T., Guo, C., Li, C., Zhang, X.: Lamp: Learn a motion pattern for few-shot-based video generation. arXiv preprint arXiv:2310.10769 (2023)
  * [36] Xue, H., Hang, T., Zeng, Y., Sun, Y., Liu, B., Yang, H., Fu, J., Guo, B.: Advancing high-resolution video-language representation with large-scale video transcriptions. In: CVPR (2022)
  * [37] Yang, J., Dong, X., Liu, L., Zhang, C., Shen, J., Yu, D.: Recurring the transformer for video action recognition. In: CVPR. pp. 14063–14073 (2022)
  * [38] Zhang, K., Mo, L., Chen, W., Sun, H., Su, Y.: Magicbrush: A manually annotated dataset for instruction-guided image editing. arXiv preprint arXiv:2306.10012 (2023)
  * [39] Zhao, Y., Luo, C., Tang, C., Chen, D., Codella, N., Zha, Z.J.: Streaming video model. In: CVPR. pp. 14602–14612 (2023)