#  Learning to Slice Wi-Fi Networks: A State-Augmented Primal-Dual Approach
††thanks: This work has been supported by NSF-Simons MoDL, Award 2031985, NSF
AI Institutes program.

Yiğit Berkay Uslu  University of Pennsylvania
Philadelphia, PA, USA
<EMAIL_ADDRESS>Roya Doostnejad  Intel Corporation
Santa Clara, CA, USA
<EMAIL_ADDRESS>Alejandro Ribeiro  University of Pennsylvania
Philadelphia, PA, USA
<EMAIL_ADDRESS>Navid NaderiAlizadeh  Duke University
Durham, NC, USA
<EMAIL_ADDRESS>
###### Abstract

Network slicing is a key feature in 5G/NG cellular networks that creates
customized slices for different service types with various quality-of-service
(QoS) requirements, which can achieve service differentiation and guarantee
service-level agreement (SLA) for each service type. In Wi-Fi networks, there
is limited prior work on slicing, and a potential solution is based on a
multi-tenant architecture on a single access point (AP) that dedicates
different channels to different slices. In this paper, we define a flexible,
constrained learning framework to enable slicing in Wi-Fi networks subject to
QoS requirements. We specifically propose an unsupervised learning-based
network slicing method that leverages a state-augmented primal-dual algorithm,
where a neural network policy is trained offline to optimize a Lagrangian
function and the dual variable dynamics are updated online in the execution
phase. We show that state augmentation is crucial for generating slicing
decisions that meet the ergodic QoS requirements.

###### Index Terms:

Wi-Fi slicing, 5G, QoS, constrained learning, Lagrangian duality, primal-dual,
state augmentation.

##  I Introduction

In enterprise settings, it is vital to manage network operations to support
multiple use cases with different requirements [1, 2]. In addition, 3GPP
includes architectures to integrate Wi-Fi in converged connectivity (5G + Wi-
Fi) in enterprise [3]. Network slicing allows an access point (AP) to allocate
the network resources across service-level agreement (SLA) categories rather
than individually across many users/flows. Defining dynamic slicing in Wi-Fi
with guaranteed quality-of-service (QoS), in a similar fashion as 5G, has the
potential to provide mechanisms for seamless dynamic traffic steering across
licensed and unlicensed networks. Current Wi-Fi slicing solutions are based on
a multi-tenant architecture on a single AP by installing different SSIDs [4,
5]. Despite being simple to implement, this method has drawbacks, including
the increased overhead for beacon transmission and probe responses by virtual
APs, the lack of QoS differentiation inside each SSID slice, and the
limitation of association to one slice.

Machine learning approaches are ubiquitous in resource allocation/optimization
problems in wireless networks, and there has been growing attention to
tackling network slicing problems through learning-based approaches, with
reinforcement learning (RL) methods being the most common [6, 7, 8]. However,
prior work analyzing network slicing with constraints imposed by QoS
requirements is relatively scant. For example, in [7, 8], the QoS requirements
are taken into account by either (i) directly adding the weighted QoS
requirements to the objective, or (ii) using special functions such as log-
barrier functions and an unconstrained, penalized aggregate reward, which is
optimized using conventional RL algorithms. These approaches generally lack
feasibility guarantees—which is partially remedied by projections to feasible
policies—and are highly sensitive to suitable selection of penalty weights.

Constraint-aware online deep reinforcement learning (DRL) training is explored
in [9] by leveraging Lagrangian primal-dual methods with a proactive baseline
switching. Similarly, [10] handles the QoS constraints by reformulating the
learning problem in the dual domain, which is then solved via a primal-dual
policy gradient algorithm. The primal-dual methods, although effective, do not
guarantee convergence to an optimal policy similar to penalized methods.

In this paper, we define a constrained learning framework to enable dynamic
network slicing in Wi-Fi. Sections II and III introduce a flexible Wi-Fi
network slicing framework and the problem setup. Departing from the
aforementioned works, we formulate the network slicing problem as a
constrained radio resource management (RRM) optimization problem and develop
an unsupervised state-augmented primal-dual algorithm in Section IV. The
notion of state augmentation—proposed in [11] for constrained RL and in [12]
for wireless resource allocation—incorporates the Lagrangian dual multipliers
into the state space similar to a closed-loop feedback system. This results in
a practical online algorithm that samples from an optimal policy and exhibits
feasibility and near-optimality guarantees, which do not hold for the
regularized and conventional primal-dual methods. We demonstrate these
attributes of state augmentation in the context of network slicing with
ergodic throughput and latency QoS requirements through numerical experiments
in Section V.

##  II Network Slicing in Wi-Fi

A flexible framework is defined to enable slicing in Wi-Fi, allocating
resources to assure SLAs, including maximum delay for low-latency (L) and
minimum data rate/throughput for high-throughput (H) cases, while maximizing
resource units (RU) available to other slices including best-effort (B). The
learning-based slicing module collects data from the ongoing traffic and
creates, modifies, and removes slices accordingly so that there is minimal
impact on traffic and services in other network slices in the same network.

###  II-A Slice ID

Each slice is uniquely identified by its Network Slice ID or Slice type
indicator. Both the devices and the network may initiate slice configuration.
The device may request to set up a new connection, but the infrastructure may
accept or reject the request based on channel conditions, QoS requirements,
and available network capacity. The network may also validate if the user has
the right to access a network slice. In case the network initiates slice
configuration, the device will convey the provided slice ID.

###  II-B Resource Isolation and Management

A key enabler for slicing is the ability to isolate traffic between different
slices, for which there are two solutions.

####  II-B1 Scheduling Resources in OFDMA

Each slice is dynamically configured and allocated a certain portion of the
frequency band. Applications in the same slice undergo transmission scheduling
and are allocated RUs from the assigned bandwidth. In uplink, a trigger-based
approach is used to schedule applications in each slice. A minimum RU is
allocated to all applications to support legacy devices.

####  II-B2 Multi-Link Operation (MLO)

The MLO process, newly defined in IEEE802.11be, may facilitate device
association to all slices and seamless movement across slices. An MLO device
may simultaneously connect to multiple slices and support different traffics
with different SLAs.

The slicing framework associates each slice with a main link and one or more
auxiliary links, in which the slice has a lower priority with respect to the
slices that use the link as the main. The slices that have a low priority on a
link will be served in that link only if the flows that use it as primary do
not occupy all the available resources. This makes the cross-link slicing
mechanism adaptive to the load of each slice. The concept of main and
auxiliary multi-link for slicing can also be extended to include licensed
bands for multi-spectral slicing.

##  III Network Slicing Model

Consider a wireless network where each access point (AP) is serving
$|{\mathcal{U}}|$ flows, indexed by $i\in{\mathcal{U}}$, over a $W$ MHz
channel. Each flow $i\in{\mathcal{U}}$ receives data from the application at
an average rate $\bar{\mu}_{i}>0$, which must be transmitted from the AP to
the user device. The performance requirements each flow must meet are dictated
by its SLA, and we consider three SLA categories with distinct ergodic QoS
requirements: (i) _high-throughput (H)_ , (ii) _low-latency (L)_ , and (iii)
_best-effort (B)_. The high-throughput SLA requirement is a minimum throughput
of $r_{\min}$ bps/Hz, while the low-latency SLA requirement is $\ell_{\max}$
milliseconds of maximum packet delay/latency. We specify no QoS target for the
best-effort SLAs. We assume each flow is associated with only one SLA and
denote by $\mathcal{U}_{\mathrm{H}}$, $\mathcal{U}_{\mathrm{L}}$, and
$\mathcal{U}_{\mathrm{B}}$ the disjoint subsets of flows with high-throughput,
low-latency, and best-effort SLAs, respectively.

The network operates over $T$ discrete epochs, indexed by $t=0,\ldots,T-1$. We
refer to the discrete epoch $t$ as the $t$th slicing window. Each slicing
window lasts $\tau_{\max}$ seconds and we index any timestamp within that
slicing window with $\tau\in[0,\tau_{\max})$. Each flow $i$ maintains a queue
of size $Q^{t}_{i}(\tau)\in[0,Q_{\max}]$ that evolves with a dynamic traffic
arrival rate $\alpha^{t}_{i}(\tau)\sim\alpha(\bar{\mu}^{t}_{i})$ and a dynamic
transmission rate $r^{t}_{i}(\tau)$.

Figure 1: A network slicing diagram. Frequency subbands are allocated to three
types of traffic in each slicing window.

The QoS metrics throughput and latency for a flow $i\in{\mathcal{U}}$ over the
$t$th slicing window, denoted by $\bar{r}_{i}(t)$ and $\bar{\ell}_{i}(t)$,
respectively, are determined primarily by the dynamic rate $r^{t}_{i}(\tau)$
which itself is determined by the frequency resource units (FRUs) allocated to
flow $i$ by the AP and the channel state which is an indicator of the signal-
to-interference-plus-noise ratio (SINR) experienced by the device receiving
flow $i$. The channel state varies randomly from each slicing window to
window, e.g., following a Rayleigh fading channel model.

The goal of a slicing policy is to allocate the FRU resources across SLA
categories while ensuring the QoS requirements are satisfied. We represent a
slicing decision/policy as

$\displaystyle{\mathbf{p}}(t)=\big{(}p_{\mathrm{H}}(t),\,p_{\mathrm{L}}(t),\,p_{\mathrm{B}}(t)\big{)}^{\top},\quad
t=0,\ldots,T-1,$ (1)

where the entries of ${\mathbf{p}}(t)$ denote the fraction of the total
bandwidth allocated to the respective SLA category at the beginning of the
slicing window $t$ and all flows associated with the same SLA category access
the same slice of resources. For example, if ${\mathbf{p}}(t)=(1/2,1/3,1/6)$
and $W=20$ MHz, then any high-throughput flow has access to $(1/2)\cdot 20=10$
MHz bandwidth. Within each slicing window, the AP further allocates sub-bands
at a faster time scale according to some given scheduling algorithm (e.g.,
round-robin, max-SINR, etc.). Note that the slicing decisions are updated at
the beginning of each slicing window. See Fig. 1 for an illustrative diagram.

Due to the dynamic and complex nature of the network traffic, wireless channel
conditions, and so on, the network slicing decisions should occur
opportunistically across slicing windows and learning-based optimization
techniques can be leveraged. Let ${\mathbf{s}}_{t}\in\mathcal{S}$ be the
overall network state at the beginning of the slicing window $t$ and
$\boldsymbol{\theta}\in\boldsymbol{\Theta}\subset\mathbb{R}^{d}$ be a finite-
dimensional (e.g., neural-network) parametrization.

We define the network slicing policy as

${\mathbf{p}}(t)={\mathbf{p}}^{\boldsymbol{\theta}}({\mathbf{s}}_{t};\boldsymbol{\theta})$
(2)

that takes the network state ${\mathbf{s}}_{t}$ as input and outputs the
corresponding slicing decisions. Our goal is to optimize the parameter vector
$\boldsymbol{\theta}$ to maximize the system performance subject to the QoS
requirements and the total bandwidth budget of
$p_{\mathrm{H}}(t)+p_{\mathrm{L}}(t)+p_{\mathrm{B}}(t)\leq 1$. Note that we
can drop the total bandwidth budget constraint in subsequent derivations since
it holds with equality for any optimal slicing decision and can be implicitly
satisfied by the design of the learning architecture (e.g., a softmax layer).

We define the network-slicing optimization problem as

$\displaystyle\min_{\boldsymbol{\theta}\in\boldsymbol{\Theta}}$
$\displaystyle\frac{1}{T}\sum_{t=0}^{T-1}f_{0}\big{(}{\mathbf{s}}_{t},{\mathbf{p}}^{\boldsymbol{\theta}}({\mathbf{s}}_{t};\boldsymbol{\theta})\big{)}$
(3) subject to
$\displaystyle\frac{1}{T}\sum_{t=0}^{T-1}{\mathbf{f}}\big{(}{\mathbf{s}}_{t},{\mathbf{p}}^{\boldsymbol{\theta}}({\mathbf{s}}_{t};\boldsymbol{\theta})\big{)}\leq
0,$

where $f_{0}:\mathcal{S}\times\mathbb{R}^{3}\to\mathbb{R}$ is the objective
(e.g., negated total network throughput),
${\mathbf{f}}:\mathcal{S}\times\mathbb{R}^{3}\to\mathbb{R}^{c}$ is the vector-
valued constraint function (e.g., minimum throughput and maximum packet
latency QoS) for $c$ constraints in total.

The goal of this paper is to develop a learning algorithm to solve (3) for any
given network configuration and sequence of network states
$\\{{\mathbf{s}}_{t}\\}_{t=0}^{T-1}\in{\mathcal{S}}$.

##  IV Proposed State-Augmented Algorithm

We move to the Lagrangian dual domain and write the Lagrangian function for
the parametrized problem as

$\displaystyle{\mathcal{L}}(\boldsymbol{\theta},\boldsymbol{\lambda})=\frac{1}{T}\sum_{t=0}^{T-1}\Big{[}$
$\displaystyle
f_{0}\big{(}{\mathbf{s}}_{t},{\mathbf{p}}^{\boldsymbol{\theta}}({\mathbf{s}}_{t};\boldsymbol{\theta})\big{)}+\boldsymbol{\lambda}^{\top}{\mathbf{f}}\big{(}{\mathbf{s}}_{t},{\mathbf{p}}^{\boldsymbol{\theta}}({\mathbf{s}}_{t};\boldsymbol{\theta})\big{)}\Big{]},$
(4)

where
$\boldsymbol{\lambda}=(\lambda_{1},\ldots,\lambda_{c})^{\top}\in\mathbb{R}^{c}_{+}$
is the vector of nonnegative dual multipliers corresponding to the QoS
constraints. Defining the dual function as
$d(\boldsymbol{\lambda}):=\min_{\boldsymbol{\theta}\in\boldsymbol{\Theta}}{\mathcal{L}}(\boldsymbol{\theta},\boldsymbol{\lambda})$,
the primal and dual problems are related by

$\displaystyle
D^{\star}_{\theta}:=\max_{\boldsymbol{\lambda}\in\mathbb{R}^{c}_{+}}d(\boldsymbol{\lambda})$
$\displaystyle=\max_{\boldsymbol{\lambda}\in\mathbb{R}^{c}_{+}}\min_{\boldsymbol{\theta}\in\boldsymbol{\Theta}}{\mathcal{L}}(\boldsymbol{\theta},\boldsymbol{\lambda})\leq
P^{\star}_{\theta}$ (5)

where $P^{\star}_{\theta}$ is the optimum value of parametrized primal problem
and $P^{\star}_{\theta}-D^{\star}_{\theta}$ is the duality gap. While the
duality gap is generally nonzero for the parametrized problem, it becomes
sufficiently small as the parametrization becomes richer [13]. More
importantly, although both the Lagrangian in (4) and the dual problem (5) are
nonconvex in $\boldsymbol{\theta}$, the outer maximization in the latter is a
convex problem. In practice, the inner exact minimization corresponding to the
primal policy optimization is also replaced by stochastic gradient-descent
which results in an iterative primal-dual algorithm given by

$\displaystyle\boldsymbol{\theta}_{k+1}$
$\displaystyle=\boldsymbol{\theta}_{k}-\eta_{\boldsymbol{\theta}}\nabla_{\boldsymbol{\theta}}{\mathcal{L}}(\boldsymbol{\theta}_{k},\boldsymbol{\lambda}_{k}),$
(6) $\displaystyle\boldsymbol{\lambda}_{k+1}$
$\displaystyle=\Big{[}\boldsymbol{\lambda}_{k}+\eta_{\boldsymbol{\lambda}}{\mathbf{f}}(\boldsymbol{\theta}_{k+1})\Big{]}_{+},$

where $k=0,1,\ldots,$ is an iteration index, $\eta_{\boldsymbol{\theta}}$ and
$\eta_{\boldsymbol{\lambda}}$ are the step sizes for the primal and dual
variables, respectively, and $[\cdot]_{+}:=\max(\cdot,{\mathbf{0}})$. The
trajectories $(\boldsymbol{\theta}_{k},\boldsymbol{\lambda}_{k})_{k\geq 0}$
generated by running (6) lead to near-optimal and feasible decisions
asymptotically under some mild assumptions [11, 12].

(a) Training phase

(b) Execution phase

Figure 2: (a) Offline training, and (b) online execution phases of the
proposed state-augmented slicing algorithm.

Running primal-dual algorithms online in practice is difficult and stopping
the offline training at an arbitrary iteration $k$ does not necessarily
guarantee that $\boldsymbol{\theta}_{k}$ yields a feasible slicing policy. In
fact, cycloscillatory behavior of constraints is a common phenomenon. To
remedy these challenges, we resort to a state-augmented parametrization
$\boldsymbol{\phi}\in\boldsymbol{\Phi}$, proposed in [11, 12], where the
network state ${\mathbf{s}}$ is _augmented_ with the dual multiplier vector
$\boldsymbol{\lambda}$ corresponding to the QoS constraints at the slicing
window $t$ and we denote the slicing policy as
${\mathbf{p}}^{\boldsymbol{\phi}}({\mathbf{s}},\boldsymbol{\lambda};\boldsymbol{\phi})$.
This way, the dual multipliers, which track the evolution of constraint
satisfactions/violations over time, are incorporated into the state space. We
define the Lagrangian

$\displaystyle{\mathcal{L}}_{\boldsymbol{\lambda}}(\boldsymbol{\phi})=\frac{1}{T}\sum_{t=0}^{T-1}\Big{[}$
$\displaystyle
f_{0}\big{(}{\mathbf{s}}_{t},{\mathbf{p}}^{\boldsymbol{\phi}}({\mathbf{s}}_{t},\boldsymbol{\lambda};\boldsymbol{\phi})\big{)}$
(7)
$\displaystyle+\boldsymbol{\lambda}^{\top}{\mathbf{f}}\big{(}{\mathbf{s}}_{t},{\mathbf{p}}^{\boldsymbol{\phi}}({\mathbf{s}}_{t},\boldsymbol{\lambda};\boldsymbol{\phi})\big{)}\Big{]}.$

An optimal state-augmented parametrization is defined to be a minimizer of the
expectation of the Lagrangian w.r.t. a dual multiplier sampling distribution
$q_{\boldsymbol{\lambda}}$, i.e.,

$\displaystyle\boldsymbol{\phi}^{\star}\in\operatornamewithlimits{argmin}_{\boldsymbol{\phi}\in\boldsymbol{\Phi}}\mathbb{E}_{\boldsymbol{\lambda}\sim
q_{\boldsymbol{\lambda}}}{\mathcal{L}}_{\boldsymbol{\lambda}}(\boldsymbol{\phi}).$
(8)

Given that the learning parameterization is sufficiently expressive (e.g.,
deep neural networks), the learned optimal state-augmented parameterization
$\boldsymbol{\phi}^{\star}$ approximates the Lagrangian minimizers
$\boldsymbol{\theta}^{\dagger}(\boldsymbol{\lambda}):=\operatornamewithlimits{argmin}_{\boldsymbol{\theta}\in\boldsymbol{\Theta}}{\mathcal{L}}_{\boldsymbol{\lambda}}(\boldsymbol{\theta})$
for any $\boldsymbol{\lambda}\sim q_{\boldsymbol{\lambda}}$.

The offline training of the state-augmented algorithm solves (8), which in
practice can be accomplished by replacing the expectation with training batch
averages and minimizing the empirical average Lagrangian using gradient-
descent-based approaches. Given a batch of sequences of network states and
augmenting dual multipliers
$\\{\boldsymbol{\lambda}_{b},\\{{\mathbf{s}}_{b,t}\\}_{t=0}^{T-1}\\}_{b=0}^{B-1}$,
the empirical Lagrangian is defined as

$\displaystyle\hat{{\mathcal{L}}}(\boldsymbol{\phi}_{n}):=\frac{1}{B}\sum_{b=0}^{B-1}{\mathcal{L}}_{\boldsymbol{\lambda}_{b}}(\boldsymbol{\phi}_{n}),$
(9)

and the state-augmented model parameters are updated as

$\displaystyle\boldsymbol{\phi}_{n+1}$
$\displaystyle=\boldsymbol{\phi}_{n}-\eta_{\boldsymbol{\phi}}\nabla_{\boldsymbol{\phi}}\hat{{\mathcal{L}}}(\boldsymbol{\phi}_{n}).$
(10)

During the online execution phase, we derive the network-slicing decisions
using the trained state-augmentation parametrization augmented with the
current dual multipliers, and every ${T}_{0}$ slicing windows, dual
multipliers are updated proportional to the constraint slacks as

$\displaystyle\boldsymbol{\lambda}_{k+1}\\!=\\!\Big{[}\boldsymbol{\lambda}_{k}+\frac{\eta_{\boldsymbol{\lambda}}}{{T}_{0}}\sum_{t=k{T}_{0}}^{(k+1){T}_{0}-1}{\mathbf{f}}\big{(}{\mathbf{s}}_{t},{\mathbf{p}}^{\boldsymbol{\phi}}({\mathbf{s}}_{t},\boldsymbol{\lambda}_{k};\boldsymbol{\phi}^{\star})\big{)}\Big{]}_{+}.$
(11)

The training and test phases of the state-augmented algorithm are summarized
in Fig. 2. We note that under some mild technical assumptions, near-optimality
and almost sure feasibility guarantees of (6) extend to the state-augmented
primal-dual algorithm. Moreover, although we do not consider a CRL problem,
our proposed algorithm is amenable to tackling CRL problems. We refer the
reader to [11, 12] for a theoretical exposition on the state-augmented primal-
dual algorithms.

##  V Experimental Results

Given a FRU allocation ${\mathbf{p}}(t)$ and associated channel state
${\mathbf{h}}^{t}(\tau)$, we can define the average throughput
$\bar{r}_{i}(t)$ over a slicing period $t$ for each flow $i$ as follows. The
instantaneous data rate with a current channel state $h$ and corresponding
SINR is given by the Shannon rate $g(h):=\log(1+h/\sigma^{2})$. The full
instantaneous rate for flow $i$ can be defined as

$r^{t}_{i}(\tau)=x_{i}(t)\cdot g\left(h^{t}_{i}(\tau)\right),$ (12)

where $x_{i}(t)=\\{p_{m}(t)\,|\,i\in{\mathcal{U}}_{m}\\}$ is the fraction of
the total bandwidth $W$ assigned to the SLA category of flow $i$ and the
average throughput is given by

$\bar{r}_{i}(t)=\overline{r^{t}_{i}(\tau)}=\mathbb{E}_{\tau\in[0,\tau_{\max}]}\left[r^{t}_{i}(\tau)\cdot\beta^{t}_{i}(\tau)\right]$
(13)

where $\beta^{t}_{i}(\tau)$ evaluates to $1$ whenever flow $i$ is scheduled to
transmit and has nonempty packet buffer and $0$ otherwise. We utilize a
uniform round-robin scheduler.

Let the set
$\mathcal{J}^{t}_{i}=\\{j^{t}_{i,1},j^{t}_{i,2},\ldots,j^{t}({i,N^{t}_{i}})\\}$
denote the transmitted packets of flow $i$ during slicing window $t$. For a
packet $j\in\mathcal{J}^{t}_{i}$, we can define the packet latency as

$\ell^{t}_{i}(j):=\frac{\tau_{j}-\alpha_{j}}{P}+\frac{1}{r^{t}_{i}(\tau_{j})P},$
(14)

where $r^{t}_{i}(\tau_{j})$ is the instantaneous rate during transmission of
packet $j$ as defined in (12), $P>0$ is the fixed packet size, $\alpha_{j}$
and $\tau_{j}$ are the arrival and transmission times of packet $j$,
respectively. Then the packet delay is defined as the sum of queueing delay
$\tau_{j}-\alpha_{j}$ plus the actual transmission delay incurred. We finally
define the latency for a given flow over a slicing window as the maximum
packet latency, i.e.,

$\bar{\ell}_{i}(t):=\max_{j\in\mathcal{J}^{t}_{i}}\ell^{t}_{i}(j).$ (15)

###  V-A Experimental Setup

Figure 3: Test evolutions of high-throughput (left), low-latency (middle-left)
QoS constraints, best-effort throughput objective (middle-right) and average
rate of QoS violations (right) for the proposed state-augmented primal-dual
algorithm and the baselines. Objective and the constraints are averaged (solid
lines) over $128$ test network realizations and the shadowed bands cover to
the worst ($99^{\text{th}}$ percentile) realization (dashed lines). Red dashed
lines indicate the respective QoS requirements.

The total bandwidth is $W=20$ MHz. We fix the total number of flows to
$|{\mathcal{U}}|=20$ and randomly sample the (initial) number of H, L and B
flows for each network configuration. Packet generations and traffic arrival
for all flows follow a constant bit rate model, mean packet generation rates
$\overline{\mu}^{t}_{i}$ are (uniform) randomly sampled and dynamic packet
generation rates $\alpha(\overline{\mu}^{t}_{i})$ vary across slicing windows
following a random-walk with noise standard deviation of $0.5$ bps/Hz. The
initial average data rates are sampled randomly $\mathrm{Unif}[1,5]$ bps/Hz
for high-throughput and best-effort flows and $\mathrm{Unif}[0.5,1.5]$ bps/Hz
for the low-latency slices. We adopt the Rayleigh fading channel model and the
transmitter and access-point (AP) deployment approach of [12].

We set a minimum throughput requirement of $r_{\min}=1.0$ bps/Hz for the high-
throughput slices and a maximum tail packet latency requirement of
$\ell_{\max}=10$ ms for the low-latency slices. Best-effort slices have no QoS
constraints. The network slicing problem with throughput and latency QoS
metrics described by (13) and (15), respectively, is mapped to the constrained
optimization problem of (3) with state-augmented parametrization as

$\displaystyle\max_{\boldsymbol{\phi}\in\boldsymbol{\Phi}}$
$\displaystyle\frac{1}{T}\sum_{t=0}^{T-1}\frac{1}{|\mathcal{U}_{\mathrm{B}}|}\sum_{i\in\mathcal{U}_{\mathrm{B}}}\bar{r}_{i}\big{(}{\mathbf{s}}_{t},{\mathbf{p}}^{\boldsymbol{\phi}}({\mathbf{s}}_{t},\boldsymbol{\lambda}_{\lfloor
t/T_{0}\rfloor};\boldsymbol{\phi})\big{)}$ (16) $\displaystyle\mathrm{s.t.}$
$\displaystyle\frac{1}{T}\sum_{t=0}^{T-1}\left[\max_{i\in\mathcal{U}_{\mathrm{H}}}\left(1-\frac{\bar{r}_{i}\big{(}{\mathbf{s}}_{t},{\mathbf{p}}^{\boldsymbol{\phi}}({\mathbf{s}}_{t},\boldsymbol{\lambda}_{\lfloor
t/T_{0}\rfloor};\boldsymbol{\phi})\big{)}}{r_{\min}}\right)\right]\leq 0,$
$\displaystyle\frac{1}{T}\sum_{t=0}^{T-1}\left[\max_{i\in\mathcal{U}_{\mathrm{L}}}\left(\frac{\bar{\ell}_{i}\big{(}{\mathbf{s}}_{t},{\mathbf{p}}^{\boldsymbol{\phi}}({\mathbf{s}}_{t},\boldsymbol{\lambda}_{\lfloor
t/T_{0}\rfloor};\boldsymbol{\phi})\big{)}}{\ell_{\max}}-1\right)\right]\leq
0.$

The objective is to maximize the average throughput for best-effort slices. We
formulate the QoS constraints as the ergodic average of the worst-case metrics
over the respective SLAs to account for varying number of flows.

Slicing decisions are made across $T=50$ windows and the dual multipliers are
updated every $T_{0}=2$ slicing windows during online execution of the state-
augmented primal-dual algorithm. At each slicing window, all flows within the
same SLA category take turns to transmit their packets to the AP via a round-
robin scheduling algorithm, in parallel for all SLA categories. During
transmission, the whole subband allocated to the respective slice is utilized
by the flows and we assume no interband interference. Although the duration of
each slicing window can span several minutes/hours in practice, we simulate
and evaluate over slicing windows of duration $\tau_{\max}=50$ ms for the sake
of faster simulations.

For both primal-dual policies, we use multilayer perceptron (MLP)
parametrizations with two hidden layers of width $64$ followed by a hidden
layer of width $32$ and a final softmax layer. The inputs to the models are
the concatenation of the network state vector ${\mathbf{s}}_{t}$ and the dual
multiplier vector $\boldsymbol{\lambda}_{\lfloor t/T_{0}\rfloor}$. The output
${\mathbf{p}}(t)$ is a vector of the fraction of the total bandwidth allocated
to the slices. The network state vector at slicing window $t$ is the
concatenation of the vector of fraction of flows in each slice and the average
and total data rates across each slice, estimated from the previous slicing
window.

We train both models over $128$ different network configuration for $100$
training epochs with a learning rate of $10^{-4}$. The augmenting dual
multipliers for the state-augmented model are sampled uniformly from
$[\mathbf{0},\boldsymbol{\lambda}_{\max}]\times[\mathbf{0},\boldsymbol{\lambda}_{\max}]$
with $\boldsymbol{\lambda}_{\max}=\mathbf{1}$ initially and
$\boldsymbol{\lambda}_{\max}$ is updated at the end of each training epoch by
running the online dual updates on a validation set of training networks. We
test the trained models on $128$ network realizations. Dual step size for
primal-dual training and online execution of the state-augmented algorithm are
set to $\eta^{PD}_{\boldsymbol{\lambda}}=0.1$ and
$\eta_{\boldsymbol{\lambda}}=1.0$, respectively.

We consider the following traditional slicing baselines:

  * •

_Uniform:_ The total bandwidth available is split equally among all slices,
i.e., ${\mathbf{p}}=(1/3,1/3,1/3)$ in our setup.

  * •

_Proportional:_ The total bandwidth is sliced proportional to the number of
active flows in each slice.

  * •

_Traffic-weighted (TW):_ The total bandwidth is sliced proportional to the
actual total traffic demand.

###  V-B Simulation Results & Discussion

In Fig. 3, we present the test performance of the primal-dual (PD) and state-
augmented primal-dual (SA-PD) algorithms against the baselines. The first
three subfigures from the left show the evolution of the test performance of
the algorithms with respect to the training epochs. We validate that the
state-augmented algorithm meets the throughput and latency QoS requirements
while none of the traditional baselines meet the constraints simultaneously.
The primal-dual algorithm iterates, although feasible for an average network,
do not converge to strictly feasible policies for all networks and constraints
simultaneously unlike the state-augmented algorithm. In the righmost
subfigure, we report the average rate of instantaneous QoS violations and the
state-augmented algorithm achieves a low rate of instantaneous violation for
both constraints on top of ergodic feasibility. Furthermore, we plot the
evolution of the QoS constraints for the worst network realization across
slicing windows for fully trained models in Fig. 4 and verify the ergodic
feasibility of the state-augmented slicing policy.

Figure 4: Time evolution of cumulative ergodic averages of the QoS constraints
for the worst realization.

(a) Dual multipliers

(b) Slicing decisions

(c) QoS constraints

Figure 5: Time evolution of (a) dual multipliers, (b) slicing decisions and
(c) instantaneous (dotted) and average (solid) constraints over slicing
windows for an example test network. Constraint averages are computed over a
sliding window of length $T=50$.

Figure 6: Example of policy switching with state-augmentation. TABLE I: Comparison of Rate of Instantaneous and Ergodic QoS Violations |  |  | High-throughput average instantaneous
---
and ergodic QoS violations (%)
| Low-latency average instantaneous
---
and ergodic QoS violations (%)
$r_{\min}$ (bps/Hz) |  $\ell_{\max}$ (ms) | Proportional | T.W. | Uniform | SA-PD | PD | Proportional | T.W. | Uniform | SA-PD | PD
0.7 | 5 |  | 0.3
---
0.3
| 1.6
---
0.1
| 7.6
---
0.1
| 1.5
---
0.9
| 6.0
---
4.7
| 27.5
---
25.3
| 98.5
---
99.9
| 35.6
---
35.5
| 0.4
---
0.1
| 0.3
---
0.1
0.9 | 10 |  | 30.7
---
2.2
| 13.5
---
0.1
| 42.0
---
56.2
| 1.8
---
0.1
| 2.5
---
2.0
| 14.9
---
13.1
| 95.4
---
98.4
| 18.1
---
16.1
| 4.0
---
1.2
| 1.3
---
0.7
0.9 | 20 |  | 30.7
---
2.5
| 13.5
---
9.4
| 42.0
---
56.2
| 0.1
---
0.1
| 2.6
---
1.6
| 0.5
---
0.1
| 73.5
---
76.3
| 1.8
---
1.0
| 3.8
---
0.3
| 1.2
---
0.6
1.0 | 10 |  | 86.6
---
99.9
| 24.8
---
18.0
| 66.9
---
56.2
| 6.6
---
3.8
| 18.3
---
18.0
| 15.0
---
13.0
| 95.2
---
98.3
| 17.8
---
15.8
| 2.1
---
0.2
| 0.2
---
0.1

In Fig. 5, we illustrate the online execution of the state-augmented algorithm
for an example test network. As the throughput constraint is initially
violated, the corresponding dual multipliers and the slice weights allocated
to high-throughput slices increase until the throughput constraint is met.
Fig. 6 exemplifies the policy switching behavior of the state-augmented
algorithm where alternating spikes in dual multipliers lead to switching
between prioritizing high-throughput and low-latency slices to maintain the
feasibility of both constraints. Finally, we report in Table I the average
rate of instantaneous and ergodic QoS violations for varying constraint level
specifications. An ergodic throughput QoS violation occurs for a flow
$i\in\mathcal{U}_{\mathrm{H}}$ whenever
$(1/T)\sum_{t=0}^{T-1}\bar{r}_{i}(t)<r_{\min}$ happens. Ergodic latency
violations are defined similarly. Both primal-dual algorithms outperform the
traditional baselines and the state-augmented algorithm generally attains
comparable, if not better, ergodic QoS satisfaction.

##  VI Conclusion and Future Work

We formulated (Wi-Fi) network slicing as a constrained learning problem.
Through numerical simulations, we illustrated that a state-augmented algorithm
that minimizes an expected Lagrangian offline and updates the dual multipliers
online lends itself to a practical primal-dual algorithm with feasibility and
optimality guarantees for network slicing. Extension to a state-augmented RL
algorithm for network slicing and end-to-end evaluation on a more realistic
simulation platform are interesting future research directions.

## References

  * [1] Xuemin Shen, Jie Gao, Wen Wu, Kangjia Lyu, Mushu Li, Weihua Zhuang, Xu Li, and Jaya Rao,  “AI-assisted network-slicing based next-generation wireless networks,”  IEEE Open Journal of Vehicular Technology, vol. 1, pp. 45–66, 2020.
  * [2] Wen Wu, Conghao Zhou, Mushu Li, Huaqing Wu, Haibo Zhou, Ning Zhang, Xuemin Sherman Shen, and Weihua Zhuang,  “AI-native network slicing for 6G networks,”  IEEE Wireless Communications, vol. 29, no. 1, pp. 96–103, 2022.
  * [3] Mario Teixeira Lemes, Antonio Marcos Alberti, Cristiano Bonato Both, Antonio Carlos De Oliveira Júnior, and Kleber Vieira Cardoso,  “A tutorial on trusted and untrusted non-3GPP accesses in 5G systems—first steps toward a unified communications infrastructure,”  IEEE Access, vol. 10, pp. 116662–116685, 2022.
  * [4] Matteo Nerini and David Palma,  “5G network slicing for Wi-Fi networks,”  in 2021 IFIP/IEEE International Symposium on Integrated Network Management (IM). IEEE, 2021, pp. 633–637.
  * [5] WB Alliance,  “Network slicing: Understanding Wi-Fi capabilities,”  Tech. Rep., March 2018.
  * [6] Mohammad Zangooei, Niloy Saha, Morteza Golkarifard, and Raouf Boutaba,  “Reinforcement learning for radio resource management in RAN slicing: A survey,”  IEEE Communications Magazine, vol. 61, no. 2, pp. 118–124, 2023.
  * [7] Kun Yang, Shu ping Yeh, Menglei Zhang, Jerry Sydir, Jing Yang, and Cong Shen,  “Advancing RAN slicing with offline reinforcement learning,” 2023.
  * [8] Yongshuai Liu, Jiaxin Ding, and Xin Liu,  “A constrained reinforcement learning based approach for network slicing,”  in 2020 IEEE 28th International Conference on Network Protocols (ICNP). IEEE, 2020, pp. 1–6.
  * [9] Qiang Liu, Nakjung Choi, and Tao Han,  “Onslicing: online end-to-end network slicing with reinforcement learning,”  in Proceedings of the 17th International Conference on emerging Networking Experiments and Technologies, 2021, pp. 141–153.
  * [10] Patrick Agostini, Ehsan Tohidi, Martin Kasparick, and Sławomir Stańczak,  “Learning constrained network slicing policies for industrial applications,”  in 2022 IEEE International Conference on Communications Workshops (ICC Workshops). IEEE, 2022, pp. 1–6.
  * [11] Miguel Calvo-Fullana, Santiago Paternain, Luiz Chamon, and Alejandro Ribeiro,  “State augmented constrained reinforcement learning: Overcoming the limitations of learning with rewards,”  IEEE Transactions on Automatic Control, vol. PP, pp. 1–15, 01 2023.
  * [12] Navid NaderiAlizadeh, Mark Eisen, and Alejandro Ribeiro,  “State-augmented learnable algorithms for resource management in wireless networks,”  IEEE Transactions on Signal Processing, 2022.
  * [13] Santiago Paternain, Luiz Chamon, Miguel Calvo-Fullana, and Alejandro Ribeiro,  “Constrained reinforcement learning has zero duality gap,”  in Advances in Neural Information Processing Systems, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, Eds., 2019, vol. 32.


Problem Setting The Panoptic Scene Graph generation (PSG) task aims to
generate a panoptic scene graph $\mathcal G$ for a given image
$\mathcal{I}\in\mathbb{R}^{H\times W\times 3}$, where $\mathcal G$ contains an
object set $\mathcal O$ and a relational triplet set $\mathcal T$, denoted by
$\mathcal{G}=(\mathcal{O},\mathcal{T})$. For the $i$-th object $o_{i}$ in
$\mathcal O=\\{o_{i}\\}_{i=1}^{N}$, we use $m_{i}$ to represent the object’s
mask and $c_{i}$ the object’s category, $o_{i}=(m_{i},c_{i})$. For the $j$-th
triplet in $\mathcal T=\\{t_{j}\\}_{j=1}^{M}$, we use $s_{j}$ to represent the
subject, $o_{j}$ the object, and $r_{j}$ their relation,
$t_{j}=(s_{j},o_{j},r_{j})$. There are in total $C$ object categories and $R$
relation categories. For the $k$-th relation, we denote its frequency in the
training set as $f_{k}$.

HiLo Baseline sec:baseline High-quality panoptic segmentation is crucial for
achieving good PSG performance. We build our method upon the latest advances
in DETR-based panoptic segmentation, Mask2Former cheng2022masked. Below we
present its structure, as well as our proposed modifications for the PSG task.

Mask2Former. This method comprises three key parts: A backbone (CNN-based or
transformer-based) followed by a pixel decoder, a transformer module with a
transformer decoder and a task-specific module with different task heads.
Specifically, the backbone takes an input image $\mathcal I$ and generates an
image feature $F$. The pixel decoder then gradually upsamples $F$ to produce
multi-scale features
$\mathcal{\widetilde{F}}=\\{\widetilde{F}_{i}\\}_{i=1}^{4}$. The transformer
decoder takes a set of queries $\mathcal{Q}$ of size $N$ and multi-scale
features $\mathcal{\widetilde{F}}$ as input and outputs a set of mask features
$\mathcal X$ of the same size with $\mathcal{Q}$.

On top of the transformer decoder, there are two task heads including a linear
classifier that predicts the class probability for each mask; and a multi-
layer perceptron (MLP) that uses the mask features $\mathcal X$ to generate
the mask embedding $\mathcal{E}$. The mask prediction is obtained by taking
the dot product of the mask embedding $\mathcal{E}$ with the scale feature of
the highest resolution in the multi-scale features $\mathcal{\widetilde{F}}$.

Triplet queries. The original query in Mask2Former is to predict the object.
In order to predict both subject, object and their relation, we develop the
triplet queries $\mathcal{Q}^{t}$ inspired by yang2022panoptic into our
baseline. Each query predicts a triplet that includes subject, object, and
relation. Accordingly, our task heads comprise three linear classifiers. Each
classifier is responsible for predicting the class probability for subject,
object or relation, respectively. Moreover, we devise two MLPs to generate
mask embeddings for the subject and object, denoted by $\mathcal{E}^{s}$ and
$\mathcal{E}^{o}$. We use them to compute the dot product with the scale
feature of the highest resolution in multi-scale feature
$\mathcal{\widetilde{F}}$ and obtain the mask prediction for the subject and
object, respectively.

Masked relation attention. cheng2022masked proposes a transformer decoder
variant with masked attention, which extracts spatial features by adding the
predicted mask of the object from the previous decoder layer’s mask
prediction. It makes the model focus on the object-related area in the feature
map. To adapt this scheme to the PSG task, for each relation, we extend the
masked attention to take the union of the binary masks of the subject and
object as input, which represents the pixels corresponding to the relation.

Network training. We adopt the same losses as PSGTR yang2022panoptic,
including cross-entropy loss $\mathcal{L}_{so\\_cls}$ for object
classification of subject-object pairs and a combination
$\mathcal{L}_{so\\_mask}$ of focal loss and dice loss to jointly supervise
mask learning. To supervise the relations, we use the cross-entropy loss
$\mathcal{L}_{rel\\_cls}$. The baseline loss with (${\lambda}_{1}=1$,
${\lambda}_{2}=1$, ${\lambda}_{3}=4$) is thus: L_baseline = λ_1 ⋅L_so_cls +
λ_2 ⋅L_so_mask + λ_3 ⋅L_rel_cls

##  HiLo Framework

The key insight of our HiLo is to build a model that can take into account
both high frequency and low frequency relations, and effectively improve the
performance on low frequency relations without degrading the performance on
high frequency relations.

###  HiLo relation generation

In the PSG task, multiple relations can be used to describe the connection
between a subject and an object from different perspectives, such as _spatial_
relations, _actions_ and _prepositions_. Since the subject, object and their
relative position in the image are fixed, all these relations share the same
visual information. This is reflected in the PSG dataset [yang2022panoptic]
where many subject-object pairs are annotated with multiple relation labels.
These relation labels are of different frequencies in the dataset. In this
section we introduce the HiLo relation generation module to prepare two sets
of training data, biasing towards high and low-frequency relations
respectively. Relation augmentation. Similar to [goel2022not], we observe that
there are many missing relation annotations in the PSG dataset
[yang2022panoptic]. To add the missing relation annotations, we design a
relation augmentation scheme that is inspired by IETrans [zhang2022fine],
which converts high-frequency relations to low-frequency relations and adds a
relation to a subject-object pair that has no relation. We adapt it by first
training our baseline as a biased model using the original annotated data. For
every subject-object pair in the training set, we use this model to predict
the relation scores for all predefined relation categories: If this subject-
object pair has annotated relation labels, we pick the one with the highest
predicted score and use the score as a threshold. For other relations (except
for already labeled ones) whose predicted scores are greater than the
threshold, we add them as the relation labels of this subject-object pair. If
this subject-object pair has no annotated relation labels, we use the
predicted score of the _no-relation_ class as a threshold. For other relations
whose predicted scores are greater than this threshold we add them as the
relation labels of this subject-object pair. This operation allows us to
significantly augment the relation labels for the subject-object pairs in the
training data, which is especially relevant for pairs with zero or no
annotated relations. Relation swapping. We swap the relation labels for each
subject-object pair. Specifically, given a subject-object pair $(s,o)$ with
$K$ relation labels, we have $K$ triplets $(s,o,r_{1}),...,(s,o,r_{K})$,
sorted by their relation frequency in descending order, $f_{1}>...>f_{K}$. We
denote by H-L and L-H the swapping of high-frequency relations with low-
frequency relations and vice versa. This creates two sets of data: _H-L Data._
Given a triplet ($s,o,r_{k}$), we replace its relation label $r_{k}$ with that
of the next triplet $r_{k+1}$ with lower frequency. We sequentially process
all triplets from ($s,o,r_{1}$) until ($s,o,r_{K-1}$), keeping the last
triplet unchanged. _L-H Data._ Given a triplet ($s,o,r_{k}$), we replace its
relation label $r_{k}$ with that of the previous triplet $r_{k-1}$ with higher
frequency. We sequentially process all triplets from ($s,o,r_{K}$) until
($s,o,r_{2}$), keeping the first triplet unchanged. Hence, we obtain two new
sets of triplets, denoted by $\mathcal{T}^{\text}{H-L}$ and
$\mathcal{T}^{\text}{L-H}$. We devise two parallel decoders from the shared
encoder of our backbone. They are learned with the H-L and L-H data,
respectively. The H-L and L-H decoders favour the predictions for low and
high-frequency relations, respectively. Despite their difference, for the
corresponding triplet query in the two decoders, their predictions are highly
correlated: on one hand, the subject and object predictions should be the
same; on the other hand, the distribution of relation predictions should be
overlapping. Below we first build the query correspondence in the two decoders
and then introduce the HiLo subject-object and relation consistency loss to
align the predictions from two the decoders.

###  HiLo prediction alignment

Training two different relation distributions simultaneously confuse the
model. To make the model differentiate between the two branches, we built a
HiLo prediction alignment module, including triplet query correspondence,
subject-object consistency loss and relation consistency loss respectively.
Triplet query correspondence. In order to construct the subject-object
consistency loss and relation consistency loss, we first need to construct the
correspondence between the triplet queries in the H-L and L-H decoders. The
same query index in the two decoders is not naturally matched. In order to
find the query correspondence, we need to rely on the ground truth assignment:
we use Hungarian matching to assign the triplet label to the corresponding
triplet query, and record the relation label index corresponding to the
triplet query. This label index allows us to construct the correspondence
between the triplet queries in both decoders. We calculate the consistency
loss for the triplet query prediction with the same relation label index in
the two decoders. Subject-object consistency loss. Having the corresponding
predictions from the two decoders, both their subjects and objects have the
same ground truth and should be equal. We propose a subject-object consistency
loss $\mathcal{L}_{obj}$ to minimize the mean squared error (MSE) of the
corresponding predictions from the two decoders. L_obj &= L_cls + L_mask
L_cls = softmax(p_c^H-L) - softmax(p_c^L-H) ^2
L_mask = sigmoid(p_m^H-L) - sigmoid(p_m^L-H) ^2 Here $\mathcal{L}_{cls}$ and
$\mathcal{L}_{mask}$ represent the MSE losses for class prediction and mask
prediction, respectively. $p_{c}^{\text}{H-L}$ and $p_{c}^{\text}{L-H}$ are
the class prediction logits from the H-L and L-H decoders. After a softmax,
they are $C$-dimensional probability vectors. After a sigmoid, $p_{m}^{H-L}$
and $p_{m}^{L-H}$ are the mask prediction logits for the H-L and L-H decoders.
Relation consistency loss. Given a pair of subject and object, we have
previously swapped the high-low frequency relation labels to create data for
the H-L and L-H decoders. For H-L, the prediction of the low-frequency
relation logit is of high value; while for L-H, the prediction of the high-
frequency relation logit is of high value. For the predictions on the rest
logits, they should be similar, since it is the same pair of subject and
object for the two decoders. Based on this observation, we introduce the
relation consistency loss. Specifically, for a pair of subject and object, we
use $p_{r}^{\text}{H-L}$ to denote the predicted relation logits from the H-L
decoder, $p_{r}^{\text}{L-H}$ the predicted relation logits from the L-H
decoder. They are $R$-dimensional probability vectors after softmax. We can
map between the distributions of the two vectors by swapping the logit values
between the high- and low-frequency relation indices, which is named
Relational Index Exchange operation. We use $\text{RIE}(\cdot)$ to denote this
operation. For example, for relations $r_{k}$ and $r_{k+1}$,
$\text{RIE}(\cdot)$ exchanges the values between $p_{r,k}$ and $p_{r,{k+1}}$
in $p_{r}$. We can therefore compute the distance between $p_{r}^{\text}{H-L}$
and its mapped counterpart from $p_{r}^{\text}{L-H}$, vice versa: Dist_HiLo =
&softmax(p_r^H-L) - RIE(softmax(p_r^L-H)) ^2
\+ RIE(softmax(p_r^H-L)) - softmax(p_r^L-H) ^2 The relation consistency loss
is defined to minimize $\text{Dist}_{\text}{HiLo}$ with a margin of $m$: L_rel
= max(Dist_HiLo- m, 0), where $m$ is a small constant. Adding $m$ is due to
the fact that the high- and low frequency relations might be only partially
semantically overlapping. Network training. Our subject-object consistency
loss and relation consistency loss can seamlessly integrate with the losses of
any baseline method to jointly supervise the training of the entire model.
Notably, we supervise the output of each transformer decoder layer to ensure
effective learning. The final loss $\mathcal{L}$ is thus: L = L_baseline +
L_obj + L_rel

###  HiLo inference fusion

The H-L and L-H decoders favour low-frequency relation prediction and high-
frequency relation prediction, respectively. To combine the strength of both
during inference, we introduce the HiLo inference fusion module. Specifically,
we denote by $\mathcal{G}^{\text}{H-L}$ and $\mathcal{G}^{\text}{L-H}$ the
predicted panoptic scene graphs from H-L and L-H decoders, respectively. There
are $N_{1}$ triplets in $\mathcal{G}^{\text}{H-L}$ and $N_{2}$ triplets in
$\mathcal{G}^{\text}{L-H}$. First, we merge the triplets in
$\mathcal{G}^{\text}{H-L}$ and $\mathcal{G}^{\text}{L-H}$ and sort them in the
descending order according to their relation scores. We obtain a list of
$N_{1}+N_{2}$ triplets. Second, starting from the first triplet, we de-
duplicate the triplet list. For the $i$-th triplet $\mathcal{T}_{i}$, we
identify its duplicated versions from the $(i+1)$-th triplet until the end of
the list. Remove any follow-up triplet from the list if it 1) has the same
subject, object and relation classes to that in $T_{i}$ and 2) has a mask IoU
greater than a threshold, 0.5, between the predicted subject/object and the
corresponding subject/object in $T_{i}$. Third, after deduplication, for each
triplet in the list, we multiply the relation, subject and object scores as an
overall score for it. We sort the triplet list according to this score in
descending order to obtain the final panoptic scene graph
$\mathcal{G}^{\text}{HiLo}$.

Method | Backbone | R@20 | mR@20 | R@50 | mR@50 | R@100 | mR@100
---|---|---|---|---|---|---|---
IMP [xu2017scene] | R50 | 16.5 | 6.5 | 18.2 | 7.1 | 18.6 | 7.2
MOTIF [zellers2018neural] | R50 | 20.0 | 9.1 | 21.7 | 9.6 | 22.0 | 9.7
VCTree [tang2019learning] | R50 | 20.6 | 9.7 | 22.1 | 10.2 | 22.5 | 10.2
GPSNet [lin2020gps] | R50 | 17.8 | 7.0 | 19.6 | 7.5 | 20.1 | 7.7
PSGTR [yang2022panoptic] | R50 | 28.4 | 16.6 | 34.4 | 20.8 | 36.3 | 22.1
PSGFormer [yang2022panoptic] | R50 | 18.0 | 14.8 | 19.6 | 17.0 | 20.1 | 17.6
HiLo (ours) | R50 | 34.1 | 23.7 | 40.7 | 30.3 | 43.0 | 33.1
HiLo (ours) | Swin-B | 38.5 | 28.3 | 46.2 | 35.3 | 49.6 | 39.1
HiLo (ours) | Swin-L | 40.6 | 29.7 | 48.7 | 37.6 | 51.4 | 40.9
Table : Comparison between our HiLo and other methods on the PSG dataset. Our
method shows superior performance compared to all previous methods.