# A Pub/Sub SDN-Integrated Framework for IoT Traffic Orchestration

Pedro F. Moraes<EMAIL_ADDRESS>Salvador University - UNIFACSRua Dr.
Jose Peroba 251SalvadorBrazil41770-235 and  Joberto S. B. Martins
<EMAIL_ADDRESS>Salvador University - UNIFACSSalvadorBrazil

(2019)

###### Abstract.

The Internet of Things (IoT) is advancing and the adoption of internet-
connected devices in everyday use is constantly growing. This increase not
only affects the traffic from other sources in the network, but also the
communication quality requirements, like Quality of Service (QoS), for the IoT
devices and applications. With the rise of dynamic network management and
dynamic network programming technologies like Software-Defined Networking
(SDN), traffic management and communication quality requirements can be
tailored to fit niche use cases and characteristics. We propose a
publish/subscribe QoS-aware framework (PSIoT-SDN) that orchestrates IoT
traffic and mediates the allocation of network resources between IoT data
aggregators and pub/sub consumers. The PSIoT framework allows edge-level QoS
control using the features of publish/ subscribe orchestrator at IoT
aggregators and, in addition, allows network-level QoS control by
incorporating SDN features coupled with a bandwidth allocation model for
network-wide IoT traffic management. The integration of the framework with SDN
allows it to dynamically react to bandwidth sharing enabled by the SDN
controller, resulting in better bandwidth distribution and higher link
utilization for IoT traffic.

Internet of Things, Publish/Subscribe, Quality of Service, Software-Defined
Networking, Openflow, Resource Allocation, Bandwidth Allocation Model, Fog
Computing, Pub/Sub, SDN, QoS, BAM.

††doi: 10.1145/1122445.1122456††isbn: 978-1-4503-7163-6††conference: ACM
International Conference on Future Networks and Distributed Systems; July
2019; Paris, France††journalyear: 2019††ccs: Networks Network
architectures††ccs: Networks Programmable networks††ccs: Networks In-network
processing††ccs: Networks Network management††ccs: Computer systems
organization Peer-to-peer architectures††ccs: Computer systems organization
Data flow architectures

##  1\. Introduction

The Internet of Things (IoT) is considered an important trend in the future
Internet (Rachikidi, 2017) (Borgia, 2014a). Its ability to connect common
objects to each other and through the internet has many different
applications. The growing number of these connected devices brings with them
many advantages to businesses, consumers, cities and plays a role in emerging
technology.

Most of IoT applications have a large amount of heterogeneous devices in the
form of sensors and actuators, with differences in processing, storage, power
and functionality. These differences then generate a large amount of
heterogeneous traffic that leads to complex issues in quality of service
(QoS), resource allocation and network configuration.

Current network management already deal with the issues that an overload in
traffic creates, and there are already solutions for network management aimed
at controlling network resources and traffic priority (Baddeley et al., 2018).
These capabilities are typically referred to as a network’s Quality of Service
(QoS). The purpose of QoS is to control the use of network resources more
efficiently and ensure the required levels of service quality, using network
characteristics such as bandwidth, latency, jitter and reliability.

Additionally, applications on the network can also add specialized QoS
capabilities for application SLA guarantees or cost control. The way
applications use data, and their characteristics can better define QoS models,
such as in video streaming where buffering and image quality are modified
according to user behavior and network quality.

Cloud Computing (Qian et al., 2009) and recently Fog Computing (Stojmenovic
and Wen, 2014) attempt to mitigate the impact of massive IoT traffic has on
networks and devices and data processing. Using Fog Computing operating on the
edges of the network allows to minimize the load on the whole network by
serving already preprocessed and aggregated IoT data. This allows applications
to leverage the geographical distribution of IoT data-points to keep the
traffic flow closer to the consumers.

IoT applications can benefit from these mechanisms to create more appropriate
QoS management. However, with IoT applications and devices spreading
geographically and increasing in quantity, data aggregation and preprocessing
in the Fog isn’t enough to compensate the large increase of IoT traffic in the
network. Purely decreasing IoT traffic via aggregation and preprocessing also
doesn’t automatically enhance the QoS for massive IoT data.

Using IoT data and traffic characteristics is important in order to create QoS
strategies that better serve IoT applications. Knowing how IoT devices
transmit data and how applications consume that data makes it easier to
properly apply QoS in a meaningful way.

The PSIoT-Orch framework is a QoS-aware framework (Moraes et al., 2018) for
managing IoT traffic aggregated into Fog-like IoT gateways along the network
edge. In this solution aggregators and a orchestrator allow for IoT QoS
management at the network edge enabled by network-wide specifications,
application domain and IoT characteristics. The PSIoT-SDN framework expands on
the usual orchestrator’s edge-level QoS control operation by using SDN network
programmability coupled with a bandwidth allocation model (BAM). We evaluate
the enhancements in QoS when integrating the framework with the SDN network
manager and how it positively affects IoT traffic QoS over the network. As
such, contribution of the PSIoT-SDN framework will be to integrate a QoS
approach on network edge with a network bandwidth sharing strategy based on
BAM models that is SDN-controlled.

In this article we’ll describe the PSIoT-SDN framework and how it proposes to
manage IoT traffic, as well as how it combines with SDN to offer enhanced QoS
capabilities to IoT traffic. We present in section 2 previous work in that
handle QoS and traffic management. In section 3 we explore proposed IoT-
oriented architectures, considering both data processing and QoS in an SDN
enabled network. Section 4 overviews PSIoT-SDN framework components including
its basic orchestrator for IoT traffic management at network edge-level
(aggregators) and the SDN controller for network-wide QoS management. In
Section 6 we present a proof of concept for the PSIoT-SDN with a SDN network
controller coupled with a bandwidth allocation model (BAM) and evaluate how it
affects the traffic in the network. Finally, section 7 concludes with an
overview of the result and what was presented.

##  2\. Related Work

IoT applications and devices are varied in communication protocols, hardware
and characteristics. This has generated a large amount of IoT-focused works in
the literature ranging from hardware and network protocols to applications and
traffic protocols and management (Bera et al., 2017).

A comprehensive survey in (Al-Fuqaha et al., 2015) reviews the enabling
technologies, protocols and applications for IoT, presenting elements required
for devices to deliver IoT functionality. The work in (Borgia, 2014b) presents
the IoT key features and driver technologies, detailing IoT into application
domains like industry, smart city and health.

Considering IoT traffic in wireless networks, (Al-Shammari et al., 2018),
(Taneja, 2016), (Al-Falahy and Alani, 2017) propose traffic management
solutions that focus on wireless sensor networks (WSN) and wireless cellphone
networks. The research focus is mainly in considering the constraints of low
power and computation on IoT devices and wireless networks.

The research categorized in the IoT-SDN integrated approach are mainly focused
on methods to manage the IoT traffic being generated by IoT devices after it
has entered the network. These are mostly about using the SDN dynamic
programming capability and Cloud-related technology to offer better IoT
traffic control towards mobility, security, spectrum and service management
(Tayyaba et al., 2017) (Wu et al., 2015) (Boussard et al., 2015) (Jararweh et
al., 2015).

Using the Fog for edge data processing and aggregation, (Jiang et al., 2017)
and (Santoro et al., 2017) present platforms for orchestrating Fog workload.
The research in (Modarresi et al., 2017) extends Fog computing to improve
network resilience in Fog nodes and (Yu et al., 2018) does an extensive survey
on edge computing for the IoT.

The research in (Wang et al., 2018) presents a publish/subscribe communication
platform for IoT services. It is based in SDN and directly uses the network’s
SDN management, and knowledge on the network’s routing and topology, to
control the QoS of topic subscriptions traffic.

##  3\. IoT Architecture, Data Processing and SDN Integration

The Internet of Things is growing more and more as users and applications
arise with IoT connected devices. The IoT enables everyday objects to be
identified and to communicate with each other and other applications over the
internet. This flexibility also means that the IoT is extremely heterogeneous,
in both data and network usage.

As the IoT expands, effort has been made to find common traits and
characteristics of the IoT connected devices as well as architectures for IoT
data processing and network communication.

###  3.1. IoT Architecture

The heterogeneous nature of IoT devices, data and overlapping application
domains makes it difficult to enable interoperability between IoT devices and
applications, despite IoT traffic characteristics and requirements being well
defined.

In (Institute, 2012), the ETSI Technical Committee for Machine-to-Machine
Communications (ETSI TC M2M) proposed an IP-based architecture leveraging
existing technologies. The architecture was divided into three domains:

  * •

The Application Domain, where client and M2M applications lie;

  * •

The Network Domain, the network between applications and device gateways; and

  * •

The Device & Gateway domain, the location the devices and/or gateways reside.

The definition of these domains are useful in organizing and designing IoT
solutions, as it provides a clearer picture on how IoT devices and data happen
in networks and the internet. This clearer distinction can allow for tailored
solutions, catering to each domain specifically and allow a better integration
between architectures and IoT solutions, like data processing or IoT QoS.

###  3.2. IoT Data Processing

The growing number of IoT devices generates an also growing amount of data
being transmitted through the internet and wireless networks. This amount of
traffic can burden the network and cause IoT services that rely on timely
communication to malfunction. Aggregating and preprocessing IoT data can
alleviate the burden on the network.

This process can happen at two points: in-network (edge and network devices)
and at the cloud level (Abu-Elkheir et al., 2013) (Wang et al., 2015). Cloud
IoT data processing aims to cut down on IoT traffic to IoT data consumers by
processing device data into aggregated data of interest to many consumers.

Further data processing also happens in local IoT device networks, such as
wireless, and wireless mesh networks. This occurs because of the energy and
resource constraints on these devices, where neighbouring data that is usually
highly redundant is aggregated, and reduced in size, thus preserving energy
and resources.

Data aggregation is responsible for increasing the network lifetime and
reducing the energy consumption (Tripathi et al., 2014). A comparison of data
aggregation techniques is presented in (Rahman et al., 2016), and (Pourghebleh
and Navimipour, 2017) presents a literature survey on data aggregation
mechanisms.

Cloud aggregation and data processing don’t fully supply the IoT need for low
latency, leading to the search of some derived solutions. Considering this
problem, Cisco introduced in (Bonomi et al., 2012) the concept of Fog
Computing to enable applications on billions of connected devices already
connected in the Internet of Things to run directly in the edge of the
network, providing mobility and geo-distribution support, as well as managing
location and latency.

A hybrid approach to data aggregation exists, where Fog nodes act as
preprocessors that aggregate IoT data in the network edge before forwarding it
into the Cloud, presented in (Alturki et al., 2017).

Considering these data aggregation methods, and the ETSI IoT domain model,
Figure 1 shows how the different IoT data aggregation are located in the
network. It shows how IoT data flows from devices in the M2M Domain, is
aggregated and preprocessed in the Fog and forwarded to the Cloud. This stream
of aggregation and processing can cut back on the load that massive IoT
traffic can cause in the network.

Figure 1. IoT architectural basic elements and data processing

###  3.3. IoT Network Architecture with SDN

With more complex networks, traditional management paradigms must be reworked
and changed. There is a need for network management approaches that are
scalable and are able to work well with the heterogeneous, complex, and large-
scale nature of IoT.

The great number of heterogeneous IoT devices and data require new technology
and management patterns to better serve IoT traffic QoS needs. SDN is a well-
matched network programming paradigm suitable for IoT management, that
separates the control and data planes. SDN provides high-level abstraction and
virtualized network functions that make management simpler (Valdivieso
Caraguay et al., 2014).

SDN-based IoT management allows for better control over traffic routing and
network configuration, allowing the network to more easily react to network
traffic needs. The dynamic configuration capabilities of SDN networks are a
great match to the heterogeneity of IoT traffic, allowing the network to
better route IoT traffic and manage its impact on the whole network.

Integrating IoT frameworks with SDN gives greater control over IoT traffic.
Figure 2 shows a high-level view of a a typical architecture for IoT-SDN
integration.

Figure 2. IoT architecture with SDN: a high-level view

The SDN-IoT integration brings several significant benefits for IoT traffic:

  1. (1)

Intelligent traffic routing and better network resources use.

  2. (2)

Simplified information acquisition facilitating information analysis, decision
making and network configuration actions.

  3. (3)

Virtualization, whenever required, may be easily achieved and deployed using
common SDN virtualization tools like hypervisors (Kreutz et al., 2015).

  4. (4)

Visibility of network resources and access management based on user, group,
device, and application.

  5. (5)

Intelligent algorithms to build effective traffic pattern analyzers.

These benefits result in IoT networks with integrated SDN capabilities
becoming more agile, scalable and based on demand.

##  4\. The PSIoT Framework Basics

As described in (Moraes et al., 2018), the PSIoT-Orch is a IoT traffic
management framework that combines the Cloud, Fog and Pub/Sub IoT scenarios to
enable Pub/Sub-based IoT data transfers and QoS requirements enforcement based
on IoT data sets.

The PSIoT-Orch framework was created to manage massive traffic generated by
the ever growing number of IoT devices. Its goal is to use Publish/ Subscribe
to allow IoT data transfer among producers and consumers and, concomitantly,
to handle network resources efficiently according to IoT QoS traffic
requirements at edge level. It adopts a static priority allocation for IoT
traffic data classes at aggregators (Moraes et al., 2018).

PSIoT-Orch manages the IoT network resources by leveraging its control on IoT
traffic transmission through the network. This management is specially
tailored to IoT data and traffic characteristics, enabling better QoS than
data-agnostic QoS rules in the network.

Maintaining IoT data aggregators at the edges of the network enables the
framework to have more precise knowledge on IoT data, allows pre-processing
and all Fog-related benefits while enabling traffic shaping and prioritizing
based on the higher priority of IoT data flows.

###  4.1. The PSIoT-Orch architecture

The PSIoT-Orch framework has 3 main components:

  * •

A traffic orchestrator;

  * •

Fog-like IoT gateway data aggregators (IoTGW-Ag) acting as Pub/Sub publishers;
and

  * •

Pub/Sub clients, be they end-user applications or Cloud processing centers.

Figure 3 shows how the components are distributed in the network. The
communication happens by trading messages through the network, with a
centralized orchestrator and multiple clients and producers.

Figure 3. PSIoT-Orch framework basic components

The IoTGW-Ag are Fog-like nodes that act as IoT data aggregators and Pub/Sub
producers. These nodes can take other IoT-related responsibilities such as
data pre-processing, backup or caching. Each IoTGW-Ag connects to the PSIoT
orchestrator, that manages the traffic transmission from the many aggregators
in the network.

The orchestrator can leverage its control on each IoTGW-Ag in the network to
manage the flow of IoT traffic. Figure 4 illustrates how the framework is
positioned in the network and how it can leverage the flow of IoT data by
managing the traffic flow constraints at each IoTGW-Ag in the network,
according to IoT data characteristics.

The orchestrator has the knowledge of each IoTGW-Ag’s Pub/Sub subscriptions,
as well as the QoS levels required for each topic subscription. This allows it
to decide the transmission rates of IoT data from each aggregator in the
network, so as to maximize the throughput of higher level QoS subscriptions
(Moraes et al., 2018).

The PSIoT-Orch framework relies on the time-sensitivity of IoT traffic to
determine its QoS levels. While managing aggregators transmission rates
doesn’t increase the overall throughput of IoT traffic, it does do so to
higher priority IoT topic-based subscriptions. Higher priority IoT data can
flow faster in comparison to overall IoT traffic load on the network.

In summary, PSIoT-Orch manages QoS at the aggregators on behalf of IoT traffic
by managing the traffic in queues with the orchestrator (Figure 4). In the
PSIoT-Orch architecture QoS level assurance is only suported at edge-level at
the aggregators. This approach is suitable for any network infrastructure,
including the Internet, where no control exists over the network resources
being used.

Figure 4. PSIoT-Orch architectural components: Orchestrator and IoT
Aggregators.

##  5\. The PSIoT-SDN architecture

The PSIoT-SDN framework introduces a network-level QoS control in the PSIoT
framework. The target now is to achieve both edge-level and network-level QoS
management by leveraging the QoS mangement capabilities at aggregattors
provided by the PSIoT orchestrator and to introduce a SDN-based network links
bandwidth allocation strategy. For achieving that, the SDN management is
coupled with a bandwidth allocation approach for network links resource
allocation.

The components of the PSIoT-SDN architecture are shown in Figure 5. In effect,
an SDN controller is introduced in the architecture to allow IoT flows
programmability at the switches deployed in the network.

Figure 5.  PSIoT-SDN integrated framework

In the PSIoT-SDN architecture it is assumed that, firstly, the IoT traffic is
heterogeneous in relation to its QoS requirements and is present in multiple
network locations. From these highly distributed locations IoT traffic has to
travel from IoT sensors to the aggregators and, from there, using a controlled
network to its consumers. These factors make it so that finding a common
hierarchy of QoS needs for IoT traffic harder and usually leads to tailoring
QoS requirements according to local device and network resources capability.

The PSIoT-SDN framework is built to work on top of a controlled network that
it can program directly. It is assumed that the framework has direct control
over network link resources or routing decisions using the SDN deployed
interface. This makes it so that the IoT QoS management has effectively two
components:

  * •

An IoT QoS traffic management, enabled by the control of all IoT data
producers at the IoTGW-Ag queues; and

  * •

A link resource management by adjusting bandwidth between network switches all
over the path between aggregators and consumers using SDN.

SDN enables dynamic network configuration, routing and QoS in networks, and
can be regarded as an ally when dealing with IoT traffic. Integrating SDN
enabled network management into IoT systems can help to fine tune and fulfill
IoT QoS requirements.

Integrating SDN network management within the PSIoT-SDN framework enables
enhanced QoS with better guarantees on packet QoS due to the fact that link
bandwidth will efficiently allocated among network users.

The next section presents the operation flow of a network running the PSIoT-
SDN framework with SDN network management and introduces the bandwidth
allocation strategy for IoT flows and other traffic in network links

###  5.1. PSIoT-SDN link bandwidth management and operation flow

The SDN controller functional blocks are illustrated in Figure 6:

  * •

An interface with the PSIoT orchestrator;

  * •

A MAM-like bandwidth sharing module (MAM - Maximum Allocation Model); and

  * •

An SDN/OpenFlow network-programming interface.

Figure 6.  SDN controller modules

The SDN controller interface communicates with the PSIoT orchestrator and is
responsible for interpreting commands coming from the orchestrator requesting
the setup of communication paths between IoT aggregators (producers) and IoT
consumers.

The MAM-like module is a bandwidth sharing schema that uses the bandwidth
allocation model strategy defined for the MAM model. Bandwidth sharing is an
often used concept explored in Bandwidth Allocation Models (BAM) that
distributes and efficiently manages scarce network resources (Reale et al.,
2014). The MAM-like module basically keeps track over the used bandwidth for
all links over the path between IoT producer and IoT consumer.

The MAM BAM model assumes that the link bandwidth is divided without any
sharing between a set of traffic classes . For each traffic class (TC) is
allocated a slice of the bandwidth and network users are allocated and allowed
to use only the bandwidth allocated for its class. This limit is denominated
bandwidth constraint (BC).

In the PSIoT-SDN framework the traffic classes and bandwidth constraints were
defined and configured as follows:

  * •

Traffic class 0 (TC0) supports all low priority traffic, including from the
IoT, in the network;

  * •

Traffic class 1 (TC1) supports some mid-priority non-IoT traffic; and

  * •

Traffic class 2 (TC2) supports high priority traffic other than IoT over the
network.

The bandwidth allocated for each traffic class is a management decision
configured as BC0, BC1 and BC2.

The SDN controller network-programming interface configures the switches to
establish paths for IoT traffic over the network. Paths between producers and
consumers are previously calculated using any routing algorithm. The SDN
controller uses this computed paths to configure the switch flow-tables over
the path.

The SDN controller monitors the network bandwidth usage and network-level IoT
class bandwidth allocations. As the traffic changes, the controller can
allocate additional bandwidth to IoT traffic by reassigning the TC of IoT
traffic flows to TC’s with available bandwidth in the links. This is done by
the PSIoT-Orch communicating grouped traffic flows, in the form of origin-
destination pairs, to the MAM-like module at the SDN controller. This allows
the IoT traffic growing on demand to maintain appropriate throughput, by
allowing the PSIoT-SDN to utilize the additional bandwidth allocated by the
controller to increase the transmission rates of IoT data, according to their
QoS need.

The SDN controller operation flow is illustrated in Figure 7.

Figure 7.  SDN controller operation

The steps involved are the following:

  * •

The orchestrator checks QoS status on the aggregators and defines IoT flows
between producer and consumers.

  * •

IoT flows are created upon orchestrator requests to the SDN controller.

  * •

The SDN controller creates entries on the OpenFlow switches involved in the
path.

  * •

SDN controller adjusts IoT queue bandwidth limits at the switches in
accordance with bandwidth availability controlled by the MAM-like module.

The PSIoT-SDN framework assumes that all traffic flowing in the network are
queued at the switches and have bandwidth constraints defined. In effect, the
queue implementation and corresponding bandwidth limits are a feature that has
dependency of the switched and network operating system used. This is a
limitation of the proposed framework but is essential to have bandwidth limits
to guarantee QoS requirements at network level (Krishna, 2016).

##  6\. PSIoT-SDN Proof of Concept

An implementation of the PSIoT-SDN framework with integrated SDN is made to be
easily tested in a simulated environment.

The PSIoT-SDN proof of concept components are:

  * •

A network topology created with the Mininet emulator (Lantz, 2018);

  * •

The POX (Mc and Scott, 2013) SDN network operating system supporting the IoT-
SDN network management;

  * •

Open vSwitch switches (Pfaff et al., 2015), controlled by the OpenFlow
protocol;

  * •

User traffic generators;

  * •

The MAM-like BAM module implemented with OpenStack tools; and

  * •

The SDN user-controller software.

The proof of concept scenario presents a simple and controlled network
topology using links with constrained resources. Its topology and network
resources are set in such a way as to present both IoT and normal traffic
competing for resources.

Figure 7 shows the proof of concept topology with the PSIoT-SDN components on
the network. The proof of concept traffic configuration adopted is as follows:

  * •

The network link resource allocation is set to 3 traffic classes (TC0, TC1 and
TC2)

  * •

TC0 is configured initially with 50% of link bandwidth (BC0);

  * •

TC1 is configured initially with 30% (BC1);

  * •

TC2 is configured initially with 20% (BC2);

  * •

IoT traffic is allocated at TC0; and

  * •

All other traffic is allocated at TC1 and TC2 according with their priority.

The MiniNet emulated OpenFlow switch (Open vSwicth) uses bandwidth controlled
network queues (NQ0, NQ1 and NQ2), each of then supporting a traffic class
(TC0, TC1 and TC2) (Krishna et al., 2016). The link’s available bandwidth is
restrained for traffic flows belonging to each class level. This division is
done to simulate real world network constraints on traffic, according to
connection SLA’s, and IoT traffic is assigned to the lowest priority class to
simulate consumer-level internet connections.

This scenario is designed to match common network configurations and to
display the better QoS control that the PSIoT-SDN framework can achieve when
integrated into an SDN network’s management. This will be accomplished by
running an implementation of the PSIoT-SDN, which preserves the PSIoT-Orch
queue management control at the aggregators, inside an emulated SDN network in
Mininet, along with a custom SDN controller that will communicate with the
Orchestrator.

###  6.1. Proof of concept simulated runs

This proof of concept simulation aims to show how integrating with the SDN
controller has a positive effect on the PSIoT-SDN’s network use and on its
ability to provide better QoS guarantees to its users.

To this effect, the SDN integration implements this in a way where PSIoT-SDN
traffic groups, as signaled to the SDN controller, can be assigned to TC’s
with spare capacity. Figure 8 shows how the SDN controller reassigns flows of
traffic from the PSIoT-SDN to TC’s with available bandwidth.

The above network scenario is simulated with a series of events that aim to
show the strengths of SDN-enabled bandwidth sharing with the integrating of
the SDN network management to the PSIoT-SDN.

The events are ordered as follows:

  1. (1)

T1 sends traffic to T2. This traffic is in the TC0 class.

  2. (2)

C1 subscribes to Ag1 and Ag2. All IoT traffic is in the TC0 class.

  3. (3)

C2 subscribes to Ag1 and Ag2.

  4. (4)

T1 stops sending traffic.

  5. (5)

T2 sends traffic to T1. This traffic is in the TC1 class.

  6. (6)

T2 stops sending traffic.

  7. (7)

End of simulation.

Figure 8. SDN controller reassigns traffic to TC’s. A) TC assignment before
and B) after reassigning PSIoT-SDN traffic

###  6.2. Evaluating the results

BAMs allow better network utilization by sharing unused bandwidth between
TC’s. They have a positive effect on network traffic as a whole. However, the
advantages gained from BAM’s generally don’t favor targeted applications in
the network. Simply having dynamic BAM’s like G-BAM (Reale et al., 2014) isn’t
enough to increase the QoS control that the PSIoT-SDN has on its traffic
because it still has to compete for the extra bandwidth.

Integration with a custom SDN BAM solution allows the PSIoT-SDN to passively
signal to the SDN Controller which traffic groupings that will take precedence
in bandwidth sharing, giving the PSIoT-SDN a more exclusive access to the
currently unused bandwidth.

With the simulation we can observe how the SDN controller allocates bandwidth
to PSIoT-SDN traffic groups at the network and how the framework responds to
variations in bandwidth demand. In order to measure and compare the effects of
the SDN integration, the proof of concept will run the PSIoT-SDN, that
integrates SDN/ OpenFlow, and the PSIoT-Orch that does not make use of SDN.

Figure 9 shows the link load at each simulation event. By redistributing the
traffic from aggregators, according to IoT QoS levels, into different traffic
classes there is a better use of the overall network bandwidth and also
provides the IoT traffic with network-level QoS.

Figure 9.  Link load during simulation events

Comparing the effects of the PSIoT-SDN integration, Figure 10 shows the link
load of the simulation being executed with and without network-level QoS
integration. The integration with the SDN network can utilize up to 100% of
the link’s capacity and an average of 86% bandwidth utilization. When not
integrated, the IoT traffic output and overall network utilization is limited
by the constraints of the applications associated traffic class.

BAM’s are not a novel concept, and different flavours allow for similar
bandwidth sharing. However the main advantage of this integration is the
ability to share bandwidth according to IoT specific QoS levels, enhancing the
level of guarantees in QoS capabilities for IoT traffic. These results display
the flexibility of the PSIoT-SDN integrated into the network’s management,
especially considering how SDN networks have facilitated integration with
Application Domain services and more easily allow for third party integration
into network management.

The orchestrator implements IoT data QoS by effectively managing transmission
rates. The lower the rate of data transmission, the longer the sending of
information is delayed. Considering that transmission rates are the
determining factor for the QoS enforced on data going trough the PSIoT-SDN
framework, an increase in overall network utilization will decrease the
overall data transmission delay, providing better QoS.

Figure 10.  Link bandwidth utilization with integrated & non-integrated SDN

A relevant consideration concerning the proof of concept topology is that the
evaluation considered a single network link. In effect, our objective is to
demonstrate that the cumulative edge-QoS and network-QoS control will enhance
the overall quality of service for massive IoT traffic. It is also worth to
emphasize that BAMs manage bandwidth on a per-link basis (Reale et al., 2011).
As such, the overall QoS behavior for a path between IoT data producer and
consumer will benefit from any link bandwidth improvement over the path. This
further validates the proof of concept presented.

##  7\. Final Considerations

The internet has changed how people, machines and physical objects, ”things”,
communicate. The use of ”things”, devices that communicate with each other and
with people has made an impact on the current network technology, creating
issues with scalability, security and performance.

The solutions to internet quality problems already exist for quite some time.
Traffic from specific technologies that have a very large impact on network
congestion, like the IoT, can leverage these solutions of routing, geography
and bandwidth sharing to better serve it’s QoS requirements by creating
customized frameworks and protocols.

Factors such as how data is transferred, discovered, shared and consumed are
fundamental in creating technology and managing network traffic that better
fits the needs of the IoT.

We presented an overview of the requirements and characteristics of IoT data,
how devices communicate and how IoT data is transmitted. Based on these
attributes, we introduced a framework for IoT traffic orchestration, enabling
dynamic QoS capabilities and further integration into the backbone network
management, namely SDN.

The framework Orchestrator provides Fog-like IoT data aggregators on the
network edge with a familiar Pub/Sub interface for IoT consumers. The IoT
data’s transmission is managed by IoT specific QoS classes, enforced by data
transmission scheduling.

The framework Orchestrator also orchestrates subscribed traffic from all
registered IoT aggregators, according to available information on bandwidth
and topology and subscription QoS requirements. The QoS capabilities can
further be enhanced by integrating into the SDN management.

Resource management and traffic scheduling are often, and in the scenario
explored in this paper, a zero-sum game. The framework Orchestrator manages
traffic with the effect of providing QoS to high priority IoT data, as
asserted by IoT topic subscriptions. This effect is increased when SDN
integration allows network bandwidth sharing to be passively managed by the
PSIoT-SDN framework, enabling better utilization of the network and enhanced
QoS fulfillment for IoT traffic.

The simulation results showed how the shared bandwidth was distributed among
the framework’s IoT QoS levels, generating a larger throughput of IoT traffic
when compared to the same scenario when SDN integration was excluded. With
these positive results, the PSIoT-SDN framework is validated and displays its
usefulness in managing QoS for IoT traffic.

In summary, the main contribution of the PSIoT-SDN framework is to couple a
QoS approach on network edge with a network bandwidth sharing strategy that is
SDN-controlled.

The PSIoT-SDN framework stretches across the network, and connects several
different IoT networks and domains. Future work can consider expanding on the
SDN integration in the Network Domain, tackling the discoverability of IoT
topics in the Pub/Sub aggregator nodes, as well as aggregation and data
policies in the M2M domain.

The framework was built to integrate into existing internet technologies,
working over public networks or with enhanced capabilities in SDN networks.
This is an important characteristic to maintain in an IoT framework,
considering the fast changing scenario and the heterogeneity of data, traffic
and applications.

## References

  * (1)
  * Abu-Elkheir et al. (2013) Mervat Abu-Elkheir, Mohammad Hayajneh, and Najah Ali. 2013\.  Data Management for the Internet of Things: Design Primitives and Solution.  _Sensors_ 13, 11 (nov 2013), 15582–15612.  https://doi.org/10.3390/s131115582
  * Al-Falahy and Alani (2017) Naser Al-Falahy and Omar Y. K. Alani. 2017.  Supporting massive M2M traffic in the Internet of Things using millimetre wave 5G network. In _2017 9th Computer Science and Electronic Engineering (CEEC)_. IEEE.  https://doi.org/10.1109/ceec.2017.8101604
  * Al-Fuqaha et al. (2015) Ala Al-Fuqaha, Mohsen Guizani, Mehdi Mohammadi, Mohammed Aledhari, and Moussaq Ayyash. 2015\.  Internet of Things: A Survey on Enabling Technologies, Protocols, and Applications.  _IEEE Communications Surveys & Tutorials_ 17, 4 (2015), 2347–2376.  https://doi.org/10.1109/comst.2015.2444095
  * Al-Shammari et al. (2018) Basim K. J. Al-Shammari, Nadia Al-Aboody, and Hamed S. Al-Raweshidy. 2018.  IoT Traffic Management and Integration in the QoS Supported Network.  _IEEE Internet of Things Journal_ 5, 1 (feb 2018), 352–370.  https://doi.org/10.1109/jiot.2017.2785219
  * Alturki et al. (2017) Badraddin Alturki, Stephan Reiff-Marganiec, and Charith Perera. 2017\.  A hybrid approach for data analytics for internet of things. In _Proceedings of the Seventh International Conference on the Internet of Things_. ACM, 7\.
  * Baddeley et al. (2018) Michael Baddeley, Reza Nejabati, George Oikonomou, Mahesh Sooriyabandara, and Dimitra Simeonidou. 2018\.  Evolving SDN for Low-Power IoT Networks. In _2018 4th IEEE Conference on Network Softwarization and Workshops (NetSoft)_. 71–79.  https://doi.org/10.1109/NETSOFT.2018.8460125
  * Bera et al. (2017) Samaresh Bera, Sudip Misra, and Athanasios V. Vasilakos. 2017\.  Software-Defined Networking for Internet of Things: A Survey.  _IEEE Internet of Things Journal_ 4, 6 (Dec. 2017), 1994–2008.  https://doi.org/10.1109/JIOT.2017.2746186
  * Bonomi et al. (2012) Flavio Bonomi, Rodolfo Milito, Jiang Zhu, and Sateesh Addepalli. 2012.  Fog computing and its role in the internet of things. In _Proceedings of the first edition of the MCC workshop on Mobile cloud computing_. ACM, 13–16.
  * Borgia (2014a) Eleonora Borgia. 2014a.  The Internet of Things vision: Key features, applications and open issues.  _Computer Communications_ 54 (2014), 1–31.  https://doi.org/10.1016/j.comcom.2014.09.008
  * Borgia (2014b) Eleonora Borgia. 2014b.  The Internet of Things vision: Key features, applications and open issues.  _Computer Communications_ 54 (dec 2014), 1–31.  https://doi.org/10.1016/j.comcom.2014.09.008
  * Boussard et al. (2015) Mathieu Boussard, Dinh T. Bui, Laurent Ciavaglia, Richard Douville, Michel Le Pallec, Nicolas Le Sauze, Ludovic Noirie, Serge Papillon, Pierre Peloso, and Francesco Santoro. 2015\.  Software-Defined LANs for Interconnected Smart Environment.  (Sept. 2015).
  * Institute (2012) ETSI European Telecommunications Standards Institute. 2012.  Machine-to-Machine Communications (M2M); mIa, dIa and mId interfaces.  http://www.etsi.org/deliver/etsi_ts/102900_102999/102921/01.01.01_60/ts_102921v010101p.pdf
  * Jararweh et al. (2015) Yaser Jararweh, Mahmoud Al-Ayyoub, Ala’ Darabseh, Elhadj Benkhelifa, Mladen Vouk, and Andy Rindos. 2015.  SDIoT: A Software Defined Based Internet of Things Framework.  _Journal of Ambient Intelligence and Humanized Computing_ 6, 4 (Aug. 2015), 453–461.  https://doi.org/10.1007/s12652-015-0290-y
  * Jiang et al. (2017) Yuxuan Jiang, Zhe Huang, and Danny H. K. Tsang. 2017.  Challenges and Solutions in Fog Computing Orchestration.  _IEEE Network_ (2017), 1–8.  https://doi.org/10.1109/mnet.2017.1700271
  * Kreutz et al. (2015) Diego Kreutz, Fernando MV Ramos, Paulo Esteves Verissimo, Christian Esteve Rothenberg, Siamak Azodolmolky, and Steve Uhlig. 2015\.  Software-defined networking: A comprehensive survey.  _Proc. IEEE_ 103, 1 (2015), 14–76.
  * Krishna (2016) Hedi Krishna. 2016\.  _Providing End-to-End Bandwidth Guarantees with OpenFlow_.  Ph.D. Dissertation.
  * Krishna et al. (2016) Hedi Krishna, Niels L. M. van Adrichem, and Fernando A. Kuipers. 2016\.  Providing Bandwidth Guarantees with Openflow. In _2016 Symposium on Communications and Vehicular Technologies (SCVT)_. 1–6.  https://doi.org/10.1109/SCVT.2016.7797664
  * Lantz (2018) Bob Lantz. 2018\.  Mininet: Emulator for Rapid Prototyping of Software Defined Networks.
  * Mc and Scott (2013) Murphy Mc and Colin Scott. 2013.  POX is a networking software platform written in Python.
  * Modarresi et al. (2017) Amir Modarresi, Siddharth Gangadhar, and James P.G. Sterbenz. 2017\.  A framework for improving network resilience using SDN and fog nodes. In _2017 9th International Workshop on Resilient Networks Design and Modeling (RNDM)_. IEEE.  https://doi.org/10.1109/rndm.2017.8093036
  * Moraes et al. (2018) Pedro F. Moraes, Rafael F. Reale, and Joberto S. B. Martins. 2018\.  A Publish/Subscribe QoS-Aware Framework for Massive IoT Traffic Orchestration. In _Proceedings of the 6th International Workshop on ADVANCEs in ICT Infrastructures and Services - ADVANCE 2018_. Santiago, Chile, 1–14.  https://doi.org/10.5281/zenodo.1098298
  * Pfaff et al. (2015) Ben Pfaff, Justin Pettit, Teemu Koponen, Ethan Jackson, Andy Zhou, Jarno Rajahalme, Jesse Gross, Alex Wang, Joe Stringer, Pravin Shelar, et al. 2015\.  The design and implementation of open vswitch. In _12th $\\{$USENIX$\\}$ Symposium on Networked Systems Design and Implementation ($\\{$NSDI$\\}$ 15)_. 117–130.
  * Pourghebleh and Navimipour (2017) Behrouz Pourghebleh and Nima J. Navimipour. 2017.  Data Aggregation Mechanisms in the Internet of Things: A Systematic Review of the Literature and Recommendations for Future Research.  _Journal of Network and Comp. Applications_ 97 (nov 2017), 23–34.  https://doi.org/10.1016/j.jnca.2017.08.006
  * Qian et al. (2009) Ling Qian, Zhiguo Luo, Yujian Du, and Leitao Guo. 2009\.  Cloud Computing: An Overview.  _Lecture Notes in Computer Science_ (2009), 626–631.  https://doi.org/10.1007/978-3-642-10665-1_63
  * Rachikidi (2017) Elie E. Rachikidi. 2017\.  Modeling and Placement Optimization of Compound Service in a Converged Infrastructure of Cloud Computing and Internet of Things. (2017).  Universite Paris-Saclay, Evry.
  * Rahman et al. (2016) Hafizur Rahman, Nurzaman Ahmed, and Iftekhar Hussain. 2016\.  Comparison of Data Aggregation Techniques in Internet of Things (IoT). In _2016 International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET)_. IEEE.  https://doi.org/10.1109/wispnet.2016.7566346
  * Reale et al. (2014) Rafael F. Reale, Romildo M. Bezerra, and Joberto S. B. Martins. 2014.  G-BAM: A Generalized Bandwidth Allocation Model for IP/MPLS/DS-TE Networks.  _Inter. Journal of Computer Information Systems and Industrial Management Applications_ 6 (2014), 635–643.  https://doi.org/10.5281/zenodo.1292771
  * Reale et al. (2011) Rafael F. Reale, Walter d C. P. Neto, and Joberto S. B. Martins. 2011\.  AllocTC-Sharing: A New Bandwidth Allocation Model for Ds-Te Networks. In _7th Latin American Network Operations and Management Symposium_. IEEE Institute of Electrical and Electronics Engineers, Quito, Equador, 1–4.  https://doi.org/10.1109/LANOMS.2011.6102265
  * Santoro et al. (2017) Daniele Santoro, Daniel Zozin, Daniele Pizzolli, Francesco De Pellegrini, and Silvio Cretti. 2017\.  Foggy: A Platform for Workload Orchestration in a Fog Computing Environment. In _2017 IEEE International Conference on Cloud Computing Technology and Science (CloudCom)_. IEEE.  https://doi.org/10.1109/cloudcom.2017.62
  * Stojmenovic and Wen (2014) Ivan Stojmenovic and Sheng Wen. 2014.  The Fog Computing Paradigm: Scenarios and Security Issues.  _Proceedings of the 2014 Federated Conference on Computer Science and Information Systems_ (2014).  https://doi.org/10.15439/2014f503
  * Taneja (2016) Mukesh Taneja. 2016\.  A Framework for Traffic Management in IoT Networks. In _2nd Int. Conference on Contemporary Computing and Informatics (IC3I)_.  https://doi.org/10.1109/ic3i.2016.7917982
  * Tayyaba et al. (2017) Sahrish Khan Tayyaba, Munam Ali Shah, Omair Ahmad Khan, and Abdul Wahab Ahmed. 2017.  Software Defined Network (SDN) Based Internet of Things (IoT): A Road Ahead. In _Proceedings of the 2nd International Conference on Future Networks and Distributed Systems_ _(ICFNDS ’17)_. ACM, New York, NY, USA, 15:1–15:8.  https://doi.org/10.1145/3102304.3102319
  * Tripathi et al. (2014) Ankit Tripathi, Sanjeev Gupta, and Bharti Chourasiya. 2014\.  Survey on data aggregation techniques for wireless sensor networks.  _International Journal of Advanced Research in Computer and Communication Engineering_ 3, 7 (2014), 7366–7371.
  * Valdivieso Caraguay et al. (2014) Angel Leonardo Valdivieso Caraguay, Alberto Benito Peral, Lorena Isabel Barona Lopez, and Luis Javier Garcia Villalba. 2014\.  SDN: Evolution and Opportunities in the Development IoT Applications.  _International Journal of Distributed Sensor Networks_ 10, 5 (2014).  https://doi.org/10.1155/2014/735142
  * Wang et al. (2015) Meisong Wang, Charith Perera, Prem P. Jayaraman, Miranda Zhang, Peter E. Strazdins, and Rajiv Ranjan. 2015.  City Data Fusion: Sensor Data Fusion in the Internet of Things.  _CoRR_ abs/1506.09118 (2015).  arXiv:1506.09118 http://arxiv.org/abs/1506.09118
  * Wang et al. (2018) Yali Wang, Yang Zhang, and Junliang Chen. 2018.  An SDN-based Publish/Subscribe-enabled Communication Platform for IoT Services.  _China Communications_ 15, 1 (jan 2018), 95–106.  https://doi.org/10.1109/cc.2018.8290809
  * Wu et al. (2015) Di Wu, Dmitri I. Arkhipov, Eskindir Asmare, Zhijing Qin, and Julie A. McCann. 2015.  Ubiflow: Mobility Management in Urban-Scale Software Defined Iot. In _2015 IEEE Conference on Computer Communications (INFOCOM)_. 208–216.  https://doi.org/10.1109/INFOCOM.2015.7218384
  * Yu et al. (2018) Wei Yu, Fan Liang, Xiaofei He, William G. Hatcher, Chao Lu, Jie Lin, and Xinyu Yang. 2018.  A Survey on the Edge Computing for the Internet of Things.  _IEEE Access_ 6 (2018), 6900–6919.  https://doi.org/10.1109/access.2017.2778504