# Bi-objective Optimization of Information Rate and Harvested Power in RIS-
aided SWIPT Systems

Abdelhamed Mohamed, A. Zappone, and Marco Di Renzo Manuscript received April
22, 2022. A. Mohamed and M. Di Renzo are with Université Paris-Saclay, CNRS
and CentraleSupélec, Laboratoire des Signaux et Systèmes, 91192 Gif-sur-
Yvette, France. A. Zappone is with the University of Cassino and Southern
Lazio, 03043 Cassino, Italy. (e-mail: marco.direnzo@centralesupelec.fr).

###### Abstract

The problem of simultaneously optimizing the information rate and the
harvested power in a reconfigurable intelligent surface (RIS)-aided multiple-
input single-output downlink wireless network with simultaneous wireless
information and power transfer (SWIPT) is addressed. The beamforming vectors,
RIS reflection coefficients, and power split ratios are jointly optimized
subject to maximum power constraints, minimum harvested power constraints, and
realistic constraints on the RIS reflection coefficients. A practical
algorithm is developed through an interplay of alternating optimization,
sequential optimization, and pricing-based methods. Numerical results show
that the deployment of RISs can significantly improve the information rate and
the amount of harvested power.

###### Index Terms:

RIS, SWIPT, multi-objective optimization.

##  I Introduction

Reconfigurable Intelligent Surfaces (RISs) have emerged as a promising
technology for sustainable 6G networks [1, 2]. Thanks to their ability of
reflecting and refracting electromagnetic signals in a reconfigurable fashion
and with limited energy requirements, RISs can drastically reduce the energy
consumptions in wireless networks[3]. In this context, RISs have been also
studied in conjunction with the use of simultaneous wireless information and
power transfer (SWIPT), which is another key technology to improve the energy-
sustainability of future wireless networks.

Several studies show that the use of an RIS can improve both information and
power transfer. In [4], the problem of transmit power minimization subject to
quality of service (QoS) constraints and minimum energy harvesting
requirements is addressed. The optimization problem is tackled by means of a
penalty-based algorithm coupled with the alternating optimization technique.
In [5], the problem of transmit power minimization for an RIS-assisted SWIPT
non-orthogonal multiple-access (NOMA) network is investigated. A two-stage
optimization algorithm is proposed to jointly optimize the transmit
beamforming vector, the power-split ratio, and the RIS phase shifts under QoS
constraints. Semidefinite relaxation coupled with alternating and sequential
optimization methods are employed. In [6], the problem of maximizing the
weighted sum rate maximization is investigated in a SWIPT-based multi-user
multiple-input multiple-output (MIMO) downlink system, subject to minimum
harvested energy constraints. Alternating optimization is used in conjunction
with sequential optimization and pricing methods. In [7], the authors study
the problem of resource allocation in RIS-aided SWIPT-based systems, in which
a large RIS is divided into several tiles to be separately designed with the
objective of reducing the computational complexity of the design. Both a
globally optimal algorithm and a practical approach are developed by means of
branch-and-bound and sequential methods. In [8], the trade-off between sum-
rate maximization and the total harvested energy is investigated. The
$\epsilon-$method coupled with alternating optimization is used to tackle the
resulting multi-objective problem. In [9], the data rate maximization problem
in an RIS-aided system in which multiple receivers perform both information
decoding and wireless power reception is analyzed. The problem is tackled by
alternating optimization, sequential optimization, and sub-gradient searches.

This work considers a network in which a multiple-antenna base station serves
single-antenna users with the aid of an RIS. Each receiver jointly performs
information decoding and wireless power reception by means of power splitting.
Unlike previous works, the following contributions are made.

1) We consider the bi-objective problem of simultaneously maximizing the
information rate and the harvested power. The problem is formulated subject to
minimum rate constraints on the downlink channel, minimum harvested power, RIS
phase constraints, and maximum transmit power constraint. The resulting NP-
hard problem is tackled by an interplay of alternating maximization,
sequential optimization, and penalty-based methods. Since the receivers
perform both information decoding and wireless power reception, the
optimization of the power split ratio is needed, which a novel feature
compared to related works on RIS-aided, SWIPT-based systems.

2) The work considers the realistic case in which the phases and moduli of the
RIS reflection coefficients are not independent with one another, but are
rather coupled by a deterministic function. This further complicates the
solution of the resource allocation problem.

3) Numerical results confirm the effectiveness of the proposed algorithm
compared to traditional approaches. It is found, in particular, that
increasing the number of RIS elements significantly enhances the achieved data
rate and the amount of harvested power.

Among previous works, [8] and [9] are the most closely-related to our work.
However, [8] investigates the rate and harvested energy trade-off for
separated information and power receivers, i.e., each receiver performs either
information decoding or wireless power harvesting. Also, independent phases
and moduli are assumed for the RIS reflection coefficients. In addition, [9]
considers integrated information and power receivers, but it focuses on
maximizing only the information rate, without considering power harvesting and
the intertwinement between the phase and the amplitude of the RIS reflection
coefficients.

Figure 1: Illustration of the MISO RIS-assisted SWIPT system model.

##  II System Model and Problem Formulation

We consider an RIS-based multi-user multiple-input single-output (MISO)
downlink system in which a base station (BS) equipped with $M$ antennas serves
$K$ single-antenna user equipments (UEs) employing SWIPT. The $k$-th UE
employs a fraction $\rho_{k}$ of the received power for information decoding
(ID), while the rest is used for power harvesting (PH). The channels from the
BS to the RIS, from BS to k-th user, and from the RIS to the k-th user are
denoted by $\mathbf{G}\in{{\mathbb{C}}^{N\times M}}$,
${{\mathbf{h}}_{d,k}}\in{{\mathbb{C}}^{1\times M}}$, and
${\mathbf{h}_{r,k}}\in{{\mathbb{C}}^{1\times N}}$, respectively, and are
assumed to follow the block Rician fading model. The reflection coefficient
vector of the RIS is defined by
$\mathbf{v}=[{v_{1}},...,{v_{n}}]\in{{\mathbb{C}}^{N\times 1}}$, where
${v_{n}}={f_{n}}({\theta_{n}}){e^{j{\theta_{n}}}}$ is the reflection
coefficient of the n-th reflecting element of the RIS, with
$-\pi\leq{\theta_{n}}\leq\pi$ and, e.g.,
${f_{n}}({\theta_{n}})={f_{\min}}+(1-{f_{\min}}){\left({\frac{{\sin({\theta_{n}}-\phi)+1}}{2}}\right)^{\alpha}}$
is a function that relates the phase of the reflection coefficient to its
modulus, where ${f_{\min}}\geqslant 0$, $\alpha\geqslant 0,\phi\geqslant 0$
are circuit implementation constants [10]. In general, the proposed approach
can be applied to any continuous and differentiable function
${f_{n}}({\theta_{n}})$. Given the above notation, and defining
$\mathbf{H}_{r,k}=\text{diag}(\mathbf{h_{r,k}})\mathbf{G}$, the achievable
sum-rate is:

$\displaystyle{R^{ID}}\\!=\\!\sum_{k=1}^{K}\log\\!\left(\\!1\\!+\\!\frac{{{{\left|{\left({{\mathbf{h}_{d,k}}+{\mathbf{v}^{H}}{\mathbf{H}_{r,k}}}\right){\mathbf{w}_{k}}}\right|}^{2}}}}{{\sum\limits_{i=1,i\neq
k}^{{K}}\\!\\!{{{\left|{\left({{\mathbf{h}_{d,k}}\\!+\\!{\mathbf{v}^{H}}{\mathbf{H}_{r,k}}}\right){\mathbf{w}_{i}}}\right|}^{2}}\\!+\\!{\sigma_{k}^{2}}\\!+\\!\frac{\delta_{k}^{2}}{\rho_{k}}}}}\right)$
(1)

wherein ${{\mathbf{w}_{k}}}\in{{\mathbb{C}}^{M\times 1}}$ is the transmit
beamforming vector, while $\sigma^{2}_{k}$ and $\delta^{2}_{k}$ model the
power of the thermal noise and of the noise due to the conversion of the RF
signal to the baseband. Furthermore, considering a linear harvesting model,
the power harvested by the $k$-th UE is
$P_{H,k}=\eta_{k}(1-{\rho_{k}})\sum\nolimits_{i=1}^{K}{\left|{(\mathbf{h}_{d,k}+\mathbf{v}^{H}\mathbf{H}_{r,k}){\mathbf{w}_{i}}}\right|}^{2}$,
where $\eta_{k}\in[0,1]$ is the efficiency of the power harvesting circuit.
Similar to [11], we consider the associated rate function:

$\displaystyle
R^{PH}\\!=\\!\sum\limits_{k=1}^{K}\\!\log\\!\left(\\!1+\frac{\xi_{k}\eta_{k}(1-{\rho_{k}})}{\sigma_{k}^{2}}\sum\limits_{i=1}^{K}{\left|{(\mathbf{h}_{d,k}+\mathbf{v}^{H}\mathbf{H}_{r,k}){\mathbf{w}_{i}}}\right|}^{2}\\!\right)$
(2)

where $\xi_{k}\in[0,1]$ is the efficiency of the conversion from baseband
power to RF power.

The goal of this work is to maximize a weighted sum of $R^{ID}$ and $R^{PH}$,
namely:

$\displaystyle
R^{Eq}_{sum}{(\rm{\rho},\mathbf{w},\mathbf{v},\left\\{\theta_{n}\right\\})}$
$\displaystyle=R^{ID}+\bar{\lambda}R^{PH}$ (3)

where $\bar{\lambda}$ is a parameter to be tuned by the network operator
according to the priorities granted to ID and PH [11].

Defining $\mathbf{h}_{k}=\mathbf{h}_{d,k}+\mathbf{v}^{H}\mathbf{H}_{r,k}$, the
problem to be tackled in the rest of this work is formulated as:

$\displaystyle\mathcal{P_{A}}:$
$\displaystyle\quad\mathop{\mathtt{max}}\limits_{\rm{\rho},\mathbf{w},\mathbf{v},\left\\{\theta_{n}\right\\}}{\rm{}}R^{Eq}_{sum}{(\rm{\rho},\mathbf{w},\mathbf{v},\left\\{\theta_{n}\right\\})}$
(4) $\displaystyle\mathtt{s.t.}$
$\displaystyle\text{C1:}\;\frac{{{{\left|{\mathbf{h}_{k}{\mathbf{w}_{k}}}\right|}^{2}}}}{{\sum\limits_{i\neq
k}^{{K}}{{{\left|{\mathbf{h}_{k}{\mathbf{w}_{i}}}\right|}^{2}}\\!+\\!{\sigma_{k}^{2}}\\!+\\!\frac{\delta_{k}^{2}}{{\rho_{k}}}}}}\geq\gamma_{min},\;k=1,\ldots,K$
(5)
$\displaystyle\text{C2:}~{}{\eta_{k}}(1\\!-\\!{\rho_{k}})\sum\limits_{i=1}^{K}{{{\left|{{\mathbf{h}_{k}}{\mathbf{w}_{i}}}\right|}^{2}}}\geq
P_{min},\;k=1,\ldots,K$ (6)
$\displaystyle\text{C3:}~{}\sum\limits_{k=1}^{{K}}{{{\left\|{{\mathbf{w}_{k}}}\right\|}^{2}}\leq{P_{T}},}\;,\;~{}0\leq\rho_{k}\leq
1$ (7)
$\displaystyle\text{C4:}~{}{v_{n}}={f_{n}}({\theta_{n}}){e^{j{\theta_{n}}}},\quad{\rm{}}n=1,\dots,N$
(8)
$\displaystyle\text{C5:}~{}-\pi\leq{\theta_{n}}\leq\pi,\quad{\rm{}}~{}n=1,\dots,N$
(9)

It can be seen that $\mathcal{P_{A}}$ is a non-convex problem due to the non-
concavity of both the objective function and the constraints C1, C2, C4. Thus,
traditional methods do not apply.

##  III Proposed Approach

To tackle $\mathcal{P_{A}}$, we first reformulate the sum of logarithms into a
more tractable form by applying the method from [12] to each sum in our
objective function. This yields:

$\displaystyle\overline{{\mathcal{P}_{\mathcal{A}}}}:$
$\displaystyle\mathop{{\mathtt{max}}}\limits_{\scriptstyle{\rm\alpha_{I}},{\rm\beta_{I}},{\rm\alpha_{E}},{\rm\beta_{E}},\scriptstyle{\rm{\rho},\mathbf{w},\mathbf{v},\left\\{\theta_{n}\right\\}}}{f_{\mathcal{A}}}({\rm\alpha_{I}},{\rm\beta_{I}},{\rm\alpha_{E}},{\rm\beta_{E}},{\rm{\rho},\mathbf{w},\mathbf{v},\left\\{\theta_{n}\right\\}})$
(10)
$\displaystyle\text{s.t}~{}\text{(C1)},\text{(C2)},\text{(C3)},\text{(C4)},\text{(C5)}$

wherein ${f_{\mathcal{A}}}$ is shown in (11) at the top of the next page, with
${{\bar{\eta}}_{k}}={\xi_{k}}{\eta_{k}}$ and $\Re$ being the real part
operator.

$\displaystyle{f_{\mathcal{A}}}({\rm\alpha_{I}},{\rm\beta_{I}},{\rm\alpha_{E}},{\rm\beta_{E}},{\rm{\rho},\mathbf{w},\mathbf{v},\left\\{\theta_{n}\right\\}})=\sum\limits_{k=1}^{K}{\log(1+{\alpha_{I,k}})+\bar{\lambda}\log(1+{\alpha_{E,k}})-\left({{\alpha_{I,k}}+\bar{\lambda}{\alpha_{E,k}}}\right)}$
(11)
$\displaystyle+\sum\limits_{k=1}^{K}{\left({2\sqrt{{\rho_{k}}(1+{\alpha_{I,k}})}\Re(\beta_{I,k}^{*}{\mathbf{h}_{k}}{\mathbf{w}_{k}})}\right)}-\sum\limits_{k=1}^{K}{\left({{{\left|{{\beta_{I,k}}}\right|}^{2}}\left({\sum\limits_{i=1}^{K}{{\rho_{k}}{{\left|{{\mathbf{h}_{k}}{\mathbf{w}_{i}}}\right|}^{2}}+{\rho_{k}}{\sigma_{k}}^{2}}+\delta_{k}^{2}}\right)}\right)}$
$\displaystyle+\sum\limits_{k=1}^{K}{\left({2\bar{\lambda}\sqrt{{{\bar{\eta}}_{k}}(1-{\rho_{k}})(1+{\alpha_{E,k}})}\Re(\beta_{E,k}^{*}\sum\limits_{i=1}^{K}{{\mathbf{h}_{k}}{\mathbf{w}_{i}}})}\right)}-\sum\limits_{k=1}^{K}{\left({\bar{\lambda}{{\left|{{\beta_{E,k}}}\right|}^{2}}\left({{\xi_{k}}{\eta_{k}}(1-{\rho_{k}})\sum\limits_{i=1}^{K}{{{\left|{{{\mathbf{h}}_{k}}{{\mathbf{w}}_{i}}}\right|}^{2}}}+{\sigma_{k}}^{2}}\right)}\right)}$

In order to tackle (10), the first step is to embed (C4) into the objective,
resorting to a penalty-based approach, which yields:

$\displaystyle\overline{{\mathcal{P}_{\mathcal{B}}}}:$
$\displaystyle\mathop{{\mathtt{max}}}\limits_{\scriptstyle{\rm\alpha_{I}},{\rm\beta_{I}},{\rm\alpha_{E}},{\rm\beta_{E}},\scriptstyle{\rm{\rho},\mathbf{w},\mathbf{v},\left\\{\theta_{n}\right\\}}}{f_{\mathcal{A}}}-\Gamma\sum\limits_{n=1}^{N}{{{\left|{{v_{n}}-{f_{n}}({\theta_{n}}){e^{j{\theta_{n}}}}}\right|}^{2}}}$
(12)
$\displaystyle\text{s.t}~{}\text{(C1)},\text{(C2)},\text{(C3)},\text{(C5)}$

wherein $\Gamma$ represents the penalty coefficient used for penalizing the
violation of the equality constraint (C4). If $\Gamma\to\infty$, the solution
of the original problem is obtained. Problem (12) will be tackled by
alternating optimization, as explained next.

###  III-A Optimization of ${\alpha_{I,k}},{\alpha_{E,k}}$,
$\beta_{Ik},\beta_{Ek},\rho_{k}$

The optimal ${\alpha_{I,k}},{\alpha_{E,k}},\beta_{Ik},\beta_{Ek}$ are found by
simply setting the gradient of the objective to zero, which yields:

$\displaystyle{\alpha_{I,k}}\\!=\\!\frac{{{r^{2}}\\!+\\!r\sqrt{{r^{2}}\\!+\\!4}}}{2},{\beta_{I,k}}\\!=\\!\frac{{\sqrt{{{\rho}_{k}}(1\\!+\\!{{\alpha}_{Ik}})}({{{\mathbf{h}}}_{k}}{{{\mathbf{w}}}_{k}})}}{{\\!\sum\limits_{i=1}^{K}{{{\rho}_{k}}{{\left|{{{{\mathbf{h}}}_{k}}{{{\mathbf{w}}}_{i}}}\right|}^{2}}\\!\\!+\\!{{\rho}_{k}}{\sigma_{k}}^{2}}\\!+\\!\delta_{k}^{2}}}$
(13)
$\displaystyle{\alpha_{E,k}}\\!=\\!\frac{{{{\tilde{r}}^{2}}\\!+\\!\tilde{r}\sqrt{{{\tilde{r}}^{2}}\\!+\\!4}}}{2},{\beta_{E,k}}\\!=\\!\frac{{\sqrt{{{\bar{\eta}}_{k}}(1\\!-\\!{{\rho}_{k}})(1\\!+\\!{{\alpha}_{Ek}})}\\!\sum\limits_{i=1}^{K}{{{{\mathbf{h}}}_{k}}{{{\mathbf{w}}}_{i}}}}}{{{{{\bar{\eta}}_{k}}(1\\!-\\!{{\rho}_{k}})\sum\limits_{i=1}^{K}{{{\left|{{{{\mathbf{h}}}_{k}}{{{\mathbf{w}}}_{i}}}\right|}^{2}}}\\!+\\!{\sigma_{k}}^{2}}}}$
(14)

with
$\tilde{r}\\!\\!=\\!\\!\sqrt{\bar{\eta}(1\\!-\\!{\rho_{k}})}\Re\\!\left\\{\\!{\beta_{E,k}^{*}\\!\sum\limits_{i=1}{{{{{\mathbf{h}}}_{k}}{{{\mathbf{w}}}_{i}}}}}\\!\right\\}$,
$r\\!\\!=\\!\\!\sqrt{{\rho_{k}}}\Re\\!\left\\{\\!{\beta_{I,k}^{*}{{{{{\mathbf{h}}}_{k}}{{{\mathbf{w}}}_{k}}}}}\\!\right\\}$.

The optimization with respect to the coefficients $\\{\rho_{k}\\}$ is also
straightforward. With respect to $\\{\rho_{k}\\}$, in fact, the objective
function is strictly concave and the constraints (C2), (C3) are affine.
Moreover, (C1) can be rewritten in a linear form as follows, for any
$k=1,\ldots,K$:

${\rho_{k}}{\left|{{{{\mathbf{h}}}_{k}}{{{\mathbf{w}}}_{k}}}\right|^{2}}-{\gamma_{min}}\left({{\rho_{k}}\sum\limits_{i=1,i\neq
k}^{K}{{{\left|{{{{\mathbf{h}}}_{k}}{{{\mathbf{w}}}_{i}}}\right|}^{2}}+{\rho_{k}}\sigma_{k}^{2}+\delta_{k}^{2}}}\right)\geq
0$ (15)

Thus, with respect to $\\{\rho_{k}\\}$, the problem is convex and can be
solved by standard convex optimization algorithms [13].

###  III-B Optimization of $\mathbf{w}_{k}$

When all the other variables are fixed, the objective function is a concave
function of the transmit beamforming vectors $\mathbf{w}_{k}$. However,
constraints (C1) and (C2) are still not convex. To deal with them, we resort
to the successive convex approximation (SCA) method [6],[14]. Specifically,
the convex term ${\left|{{{{\mathbf{h}}}_{k}}{{\mathbf{w}}_{k}}}\right|}^{2}$
is upper-bounded by its first-order Taylor expansion as follows:

${\mathbf{w}}_{k}^{H}{\mathbf{h}}_{k}^{H}{{\mathbf{h}}_{k}}{{\mathbf{w}}_{k}}\\!\geq\\!2\Re\\!\left(\\!{{\mathbf{w}}_{k}^{(t)H}\\!{\mathbf{h}_{k}^{H}}{{\mathbf{h}}_{k}}{{\mathbf{w}}_{k}}}\\!\right)\\!\\!-\\!\\!\left(\\!{{\mathbf{w}}_{k}^{(t)H}\\!{\mathbf{h}}_{k}^{H}{{\mathbf{h}}_{k}}{\mathbf{w}}_{k}^{(t)}}\\!\right)$
(16)

whereing ${{\mathbf{w}}_{k}^{(t)},\forall k}$ is the solution from the
previous iteration. Thus, exploiting (16) and elaborating, (C1) can be recast
as:

$\displaystyle\gamma_{\min}\\!\left(\\!\sum_{i\neq
k}|\mathbf{h}_{k}\mathbf{w}_{i}|^{2}\\!+\\!\sigma_{k}^{2}\\!+\\!\frac{\delta_{k}^{2}}{\rho_{k}}\right)$
(17)
$\displaystyle\\!+\\!{{\mathbf{w}}_{k}^{(t)H}{\mathbf{h}}_{k}^{H}{{\mathbf{h}}_{k}}{\mathbf{w}}_{k}^{(t)}}\\!\\!-\\!2\Re\left({{\mathbf{w}}_{k}^{(t)H}{\mathbf{h}_{k}^{H}}{{\mathbf{h}}_{k}}{{\mathbf{w}}_{k}}}\right)\\!\leq\\!0$

Similarly, (C2) can be reformulated as follows:

$\displaystyle\begin{aligned}
\eta_{k}(1\\!-\\!\rho_{k})\\!&\sum_{k=1}^{K}2\Re\left({{\mathbf{w}}_{i}^{(t)H}{\mathbf{h}_{k}^{H}}{{\mathbf{h}}_{k}}{{\mathbf{w}}_{i}}}\right)\\!-\\!{{\mathbf{w}}_{i}^{(t)H}{\mathbf{h}}_{k}^{H}{{\mathbf{h}}_{k}}{\mathbf{w}}_{i}^{(t)}}\\\
&\hskip 42.67912pt\\!\geq\\!P_{min}\end{aligned}$ (18)

By replacing the constraints (C1) and (C2) with (17) and (18), we obtain the
convex surrogate problem to be solved in each iteration of the SCA method for
optimizing $\mathbf{w}_{k}$.

###  III-C Optimization of $\mathbf{v}$

The approach is similar to that used for the optimization of $\mathbf{w}_{k}$.
In fact, the objective is concave in $\mathbf{v}$, while constraints (C1) and
(C2) can be handled by the SCA method. Specifically, (C1) can be replaced by
the convex constraint:

$\displaystyle\gamma_{\min}\left(\sum\limits_{i\neq
k}^{K}|(\mathbf{h}_{d,k}+\mathbf{v}^{H}\mathbf{H}_{r,k})\mathbf{w}_{i}|^{2}+\sigma_{k}^{2}+\frac{\delta_{k}^{2}}{\rho_{k}}\right)$
(19)
$\displaystyle+(\mathbf{h}_{d,k}+\mathbf{v}^{(t)H}\mathbf{H}_{r,k})\mathbf{w}_{k}\mathbf{w}_{k}^{H}(\mathbf{h}_{d,k}+\mathbf{v}^{(t)H}\mathbf{H}_{r,k})^{H}$
$\displaystyle-2\Re\\{(\mathbf{h}_{d,k}+\mathbf{v}^{(t)H}\mathbf{H}_{r,k})\mathbf{w}_{k}\mathbf{w}_{k}^{H}(\mathbf{h}_{d,k}+\mathbf{v}^{H}\mathbf{H}_{r,k})^{H}\\}\leq
0\;$

and (C2) by the convex constraint:

$\displaystyle\eta_{k}(1\\!-\\!\rho_{k})\\!\sum_{i=1}^{K}(\mathbf{h}_{d,k}\\!+\\!\mathbf{v}^{(t)H}\mathbf{H}_{r,k})\mathbf{w}_{i}\mathbf{w}_{i}^{H}(\mathbf{h}_{d,k}\\!+\\!\mathbf{v}^{(t)H}\mathbf{H}_{r,k})^{H}\\!\\!-$
$\displaystyle
2\Re\\{\\!(\mathbf{h}_{d,k}\\!\\!+\\!\mathbf{v}^{(t)H}\\!\mathbf{H}_{r,k})\mathbf{w}_{i}\mathbf{w}_{i}^{H}(\mathbf{h}_{d,k}\\!\\!+\\!\mathbf{v}^{H}\\!\mathbf{H}_{r,k})^{H}\\!\\}\\!\geq\\!P_{min}$
(20)

By replacing the constraints (C1) and (C2) with (19) and (III-C), we obtain
the convex surrogate problem to be solved in each iteration of the SCA method
for optimizing $\mathbf{v}$.

###  III-D Updating $\theta_{n}$

The RIS phase shifts are the solutions of the problem:

$\mathop{\max}\limits_{\left\\{{{\theta_{n}}}\right\\}}-{\sum\limits_{n=1}^{N}{\left|{{v_{n}}-{f_{n}}({\theta_{n}}){e^{j{\theta_{n}}}}}\right|}^{2}}\;,\;\mathtt{s.t.}-\pi\leqslant{\theta_{n}}\leqslant\pi$
(21)

It can be seen that the problem is separable over $n$, i.e., each summand can
be optimized separately. Thus, defining $\varphi_{n}=\arg({v_{n}})$, the
optimal $\theta_{n}$ is found by solving the problem

$\displaystyle\mathop{\max}\limits_{{\theta_{n}}\in[-\pi,\pi]}{}$
$\displaystyle
2{f_{n}}({\theta_{n}})\left|{{v_{n}}}\right|\cos({\varphi_{n}}-{\theta_{n}})-f_{n}^{2}({\theta_{n}})$
(22)

which can be solved by standard numerical methods.

###  III-E Convergence and Complexity

Finally, the overall algorithm to solve the optimization problem is obtained
by iteratively optimizing the different optimization variables. Each iteration
monotonically increases the objective value, which guarantees convergence.
Moreover, the computational complexity is polynomial in the number of
variables, since only the solution of convex problems is required111We recall
that a convex problem can be solved with a complexity
$\mathcal{C}=\mathcal{O}(L^{\eta})$, where $L$ is the number of variables and
$1\leq\eta\leq 4$ [15].. Thus, the complexity of optimizing $\\{\rho_{k}\\}$
is $\mathcal{O}(K^{\eta_{k}})$, while the complexity of optimizing
$\\{\mathbf{w}_{k}\\}$ and $\mathbf{v}$ are
$\mathcal{O}(I_{w}(MK)^{\eta_{w}})$, and $\mathcal{O}(I_{v}N^{\eta_{v}})$,
respectively, with $I_{w}$ and $I_{v}$ being the number of iterations of the
SCA methods used to optimize $\mathbf{w}$ and $\mathbf{v}$. On the other hand,
the optimal $\\{\alpha_{I,k},\alpha_{E,k},\beta_{Ik},\beta_{Ek}\\}$ are
available in closed-form in (13), (14) and thus the complexity required for
their computation can be neglected. Similarly, the complexity of Problem (21)
can also be neglected, as it is linear in $N$. In fact, the problem can be
decoupled over the $N$ optimization variables, and, for each $N$, the optimal
$\theta_{n}$ is obtain by solving (22). Thus, the overall complexity of the
proposed method is given by
$\mathcal{C}=I(\mathcal{O}(K^{\eta_{k}})+\mathcal{O}(I_{w}(MK)^{\eta_{w}})+\mathcal{O}(I_{v}N^{\eta_{v}}))$,
where $I$ is the number of alternating optimization iterations to be run until
convergence. The exponents of the polynomial are not available in closed-form,
but it is known that they are upper-bounded by $4$ [15]. A typical value is
$3.5$, which comes up when interior-point methods are used [13].

##  IV Numerical Results

For our numerical study, we consider an RIS-assisted MISO communication
system, in which the $M=8$ transmit antennas are arranged in a uniform linear
array, and a set of $K=4$ UEs are considered. The UEs are randomly and
uniformly distributed within a disk of $1~{}m$ radius centered at
$(5~{}m,5~{}m)$. The $N$-elements RIS is located at $(0~{}m,5~{}m)$. All
channels are modeled as
$X=L_{x}\left(\sqrt{\frac{\epsilon}{1+\epsilon}}\bar{X}^{LOS}+\sqrt{\frac{1}{1+\epsilon}}\bar{X}^{NLOS}\right)$,
where $\bar{X}^{LOS}$ and $\bar{X}^{NLOS}$ are the line-of-sight (LOS) and
non-LOS (NLOS) components, and $X$ is either $\mathbf{G}$, $\mathbf{h}_{r,k}$,
or $\mathbf{h}_{d,k}$. The NLOS component follows the Rayleigh fading model,
while the LOS component is
$\bar{X}^{LOS}=\mathbf{a}_{N}(\theta^{AoA})\mathbf{a}_{M}^{H}(\theta^{AoD})$,
with:

$\displaystyle\mathbf{a}_{N}(\theta^{AoA})=[1,e^{j\frac{2\pi
d}{\lambda}\sin(\theta^{AoA})}~{},~{}...~{},e^{j\frac{2\pi
d}{\lambda}(N-1)\sin(\theta^{AoA})}]^{T}$
$\displaystyle\mathbf{a}_{M}(\theta^{AoD})=[1,e^{j\frac{2\pi
d}{\lambda}\sin(\theta^{AoD})}~{},~{}...~{},e^{j\frac{2\pi
d}{\lambda}(M-1)\sin(\theta^{AoD})}]^{T}$

where $d$ and $\lambda$ are the inter-antenna separation and the wavelength,
respectively. We assume $d/\lambda=1/2$. The large-scale distance-dependent
path-loss is $L=C_{0}{\left(\frac{d}{D_{0}}\right)^{-x_{l}}}$, where $d$ is
the link distance and $D_{0}=1$ is the reference distance at which the
reference path-loss $C_{0}=-30$dB is defined, $x_{l}$ is the path-loss
exponent. The rest of the simulation parameters are given in Table 1. For
comparison, we evaluate the performance gain achieved by the proposed
algorithm in comparison with the “No-RIS” scenario, in which there is no RIS
deployed in the system.

TABLE I: Simulation Parameters. Parameters | Values
---|---
Number of RIS elements | 60
Maximum transmission power |  $P_{T}$= 10 dB
Path-loss exponent - RIS-aided channels | $x_{l}=2.2$
Path-loss exponent - Direct channel | $x_{l}=3.6$
Rician factor |  $\epsilon=5$ dB
Power conversion noise |  $\delta_{k}^{2}=\delta^{2}=-50$dBm
Thermal noise power |  $\sigma_{k}^{2}=\sigma^{2}=-40$dBm
Minimum harvested power |  $P_{min}=10^{-5}$mW
Minimum SINR requirement |  $\gamma_{\min}=10$dB
Power conversion efficiency | $\eta_{k}=\eta=0.6$
Combining weight |  $\bar{\lambda}$= [0.1, 1]
Conversion efficiency (uplink) $\xi_{k}$ | 0.005

Figure 3 depicts the relationship between the number of RIS elements versus
the sum-rate and the harvested power. It can be observed that employing more
RIS elements leads to a monotonic growth of the amount of harvested power and
sum-rate. The figure reveals the effectiveness of the proposed algorithm
compared to the “No RIS” case (denoted by “w/o RIS”). This monotonic gain is
due to the appropriate design of the phase shift vector of the RIS elements,
which results in strong virtual LOS paths between the BS and the UEs.

In Fig. 3, we explore the trade-off between the sum-rate and harvested power
as a function of the number of RIS elements. We observe that employing more
RIS elements enhances the sum-rate and harvested power. Moreover, the figure
highlights the impact of the preference parameter $\bar{\lambda}$. The
preference parameter $\bar{\lambda}$ is utilized to determine the service
priority between optimizing the sum-rate (i.e., with a low value of
$\bar{\lambda}$) or optimizing the harvested power (i.e., with a high value of
$\bar{\lambda}$). When the system prioritizes power harvesting, the proposed
algorithm allocates more power to the power harvesting receiver, and thus the
sum-rate decreases. Similarly, reducing the value of $\bar{\lambda}$ gives
higher priority to information decoding. The figures reports the trade-off
between the sum-rate and harvested power for any values of $\bar{\lambda}$.

Figure 2: Sum-rate and harvested power versus the number of RIS elements for
$\bar{\lambda}=$0.6.

Figure 3: Trade-off between the sum-rate and harvested power with the number
of RIS elements.

Figure 4: Trade-off between the sum-rate and harvested power with $f_{min}$.

Figure 5: Trade-off between the sum-rate and harvested power with the number
of UEs $K$.

In Fig. 5, we examine the trade-off between the sum-rate and harvested power
with the minimum value of the amplitude of the reflection coefficient
$f_{min}$ of the RIS elements. We observe that the sum-rate and the harvested
power increase as $f_{min}$ increases. The highest values are obtained when
$f_{min}=1$, which is the ideal case for an RIS. In Fig. 5, finally, we
investigate the trade-off between the sum-rate and harvested power as a
function of the number of UEs $K$. We see that the sum-rate and harvested
power increases and decreases, respectively, as the number of UEs increases.
This can be explained because the sum-rate is proportional to $min(M,KN_{r})$,
where $N_{r}=1$ is the number of receive antennas and $M=8$ is fixed. Thus,
the achievable data rate boosts as the number of UEs grows. On the other hand,
the harvested power decreases due to the inverse relationship between the
harvested power and the sum-rate.

##  V Conclusion

In this paper, we have investigated the trade-off between the sum-rate and
power harvested in a multi-user RIS-aided downlink MISO system with SWIPT.
Specifically, enforcing QoS constraints and practical phase shifts
constraints, the transmit beamforming vector, the power splitting ratio, and
the RIS reflection coefficients are jointly optimized by a two-layer penalty-
based algorithm. Simulation results show that the proposed algorithm can
significantly outperform conventional system deployments in the absence of
RISs.

## References

  * [1] M. Di Renzo, A. Zappone, M. Debbah, M.-S. Alouini, C. Yuen, J. de Rosny, and S. Tretyakov, “Smart radio environments empowered by reconfigurable intelligent surfaces: How it works, state of research, and the road ahead,” _IEEE J. Sel. Areas Commun._ , vol. 38, no. 11, pp. 2450–2525, 2020.
  * [2] Q. Wu and R. Zhang, “Towards smart and reconfigurable environment: Intelligent reflecting surface aided wireless network,” _IEEE Commun. Mag._ , vol. 58, no. 1, pp. 106–112, 2020.
  * [3] C. Huang, S. Hu, G. C. Alexandropoulos, A. Zappone, C. Yuen, R. Zhang, M. D. Renzo, and M. Debbah, “Holographic MIMO surfaces for 6g wireless networks: Opportunities, challenges, and trends,” _IEEE Wirel. Commun._ , vol. 27, no. 5, pp. 118–125, 2020.
  * [4] Q. Wu and R. Zhang, “Joint active and passive beamforming optimization for intelligent reflecting surface assisted SWIPT under QoS constraints,” _IEEE J. Sel. Areas Commun._ , vol. 38, no. 8, pp. 1735–1748, 2020.
  * [5] Z. Li, W. Chen, Q. Wu, K. Wang, and J. Li, “Joint beamforming design and power splitting optimization in IRS-assisted SWIPT NOMA networks,” _IEEE Trans. Wireless Commun._ , vol. 21, no. 3, pp. 2019–2033, 2022.
  * [6] C. Pan, H. Ren, K. Wang, M. Elkashlan, A. Nallanathan, J. Wang, and L. Hanzo, “Intelligent reflecting surface aided MIMO broadcasting for simultaneous wireless information and power transfer,” _IEEE J. Sel. Areas Commun._ , vol. 38, no. 8, pp. 1719–1734, 2020.
  * [7] D. Xu, V. Jamali, X. Yu, D. W. K. Ng, and R. Schober, “Optimal resource allocation design for large IRS-assisted SWIPT systems: A scalable optimization framework,” _IEEE Trans. Commun_ , vol. 70, no. 2, pp. 1423–1441, 2022.
  * [8] A. Khalili, S. Zargari, Q. Wu, D. W. K. Ng, and R. Zhang, “Multi-objective resource allocation for IRS-aided SWIPT,” _IEEE Commun. Lett._ , vol. 10, no. 6, pp. 1324–1328, 2021.
  * [9] C. He, X. Xie, K. Yang, and Z. J. Wang, “A joint power splitting, active and passive beamforming optimization framework for IRS assisted MIMO SWIPT system,” _arXiv preprint arXiv:2105.14545_ , 2021.
  * [10] S. Abeywickrama, R. Zhang, Q. Wu, and C. Yuen, “Intelligent reflecting surface: Practical phase shift model and beamforming optimization,” _IEEE Trans. Commun_ , vol. 68, no. 9, pp. 5849–5863, 2020.
  * [11] J. Tang, Y. Yu, M. Liu, D. K. C. So, X. Zhang, Z. Li, and K.-K. Wong, “Joint power allocation and splitting control for SWIPT-enabled NOMA systems,” _IEEE Trans. Wireless Commun._ , vol. 19, no. 1, pp. 120–133, 2020.
  * [12] K. Shen and W. Yu, “Fractional programming for communication systems—part ii: Uplink scheduling via matching,” _IEEE Trans. Signal Process._ , vol. 66, no. 10, pp. 2631–2644, 2018.
  * [13] S. Boyd and L. Vandenberghe, _Convex optimization_. Cambridge university press, 2004.
  * [14] C. Pan, H. Ren, M. Elkashlan, A. Nallanathan, and L. Hanzo, “Robust beamforming design for ultra-dense user-centric C-RAN in the face of realistic pilot contamination and limited feedback,” _IEEE Trans. Wireless Commun._ , vol. 18, no. 2, pp. 780–795, 2019.
  * [15] A. Ben-Tal and A. Nemirovski, _Lectures on Modern Convex Optimization: Analysis, Algorithms, Engineering Applications_. MPS-SIAM Series on Optimization, SIAM, 2001.


# Learning Driving Policies for End-to-End Autonomous Driving

Shoaib Azam, Farzeen Munir, and Moongu Jeon Shoaib Azam, Farzeen Munir, and
Moongu Jeon are with the School of Electrical Engineering and Computer
Science, Gwangju Institute of Science and Technology, Gwangju, South Korea.
e-mail<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Humans tend to drive vehicles efficiently by relying on contextual and spatial
information through the sensory organs. Inspired by this, most of the research
is focused on how to learn robust and efficient driving policies. These works
are mostly categorized as making modular or end-to-end systems for learning
driving policies. However, the former approach has limitations due to the
manual supervision of specific modules that hinder the scalability of these
systems. In this work, we focus on the latter approach to formalize a
framework for learning driving policies for end-to-end autonomous driving. In
order to take inspiration from human driving, we have proposed a framework
that incorporates three RGB cameras (left, right, and center) to mimic the
human field of view and top-down semantic information for contextual
representation in predicting the driving policies for autonomous driving. The
sensor information is fused and encoded by the self-attention mechanism and
followed by the auto-regressive waypoint prediction module. The proposed
method’s efficacy is experimentally evaluated using the CARLA simulator and
outperforms the state-of-the-art methods by achieving the highest driving
score at the evaluation time.

###### Index Terms:

Autonomous driving, Imitation learning, Attention, Driving Policies,
Transformer.

##  1 Introduction

The automotive industry and autonomous vehicle startups are under pressure to
generate revenues from billions of investments made in research and
development over the past decade. But unfortunately, even though the
experimental autonomous vehicle has continued their operation in limited
cities, they continue to run into incidents, like autonomous vehicles blocking
a fire truck while responding to an emergency. The autonomous vehicle faces
dynamic and large-scale obstacles on the road, and counting each scenario into
decision-making software is inconceivable, which hinders practical and
scaleable deployment. The rare unfavorable events of long-tail distributions
make it challenging to deploy modular design software for an autonomous
vehicle compared to an end-to-end system for policy generation. The modular
design consists of modules consisting of localization, perception, planning,
and control. Each module provides coherent information to the next module for
processing. The final input is then passed to the vehicle actuator for lateral
and longitudinal control. This strategy is counterproductive and costly as
each module is optimized for the input received from the previous module, and
parameters are adjusted manually, which is very tedious and prone to error.

However, end-to-end learning techniques of the deep neural network provide a
comprehensive approach for learning control policy for dynamic and unexpected
traffic behaviors. These end-to-end approaches learn a mapping function that
translates the input information of the dynamic environment using sensors such
as cameras to control commands. Understanding the global context of the scene
is an essential part of these end-to-end techniques, and in most of the
previous research, a single camera modality is used to determine the mapping
function, which lacks sufficient knowledge for perceiving the driving scenes.
In literature, researchers have fused data from Lidar and cameras to capture
the 3D perspective of the surrounding environment. However, Lidar provides a
sparse 3D representation of the environment, but dependence on Lidar data for
practical and scaleable deployment for autonomous vehicles is an expensive
solution.

This paper demonstrates the feasibility of generating driving policies by
employing multiple cameras to obtain multi-perspectives and top-down $2$D
semantic map information to comprehensively understand the surrounding
environment. To achieve this, we develop a framework that extracts features
from input data and then fuses the features obtained and inputs them to a
transformer network that forms the long-term relationship between the input
features and output control commands. The learned features from the
transformer are used to predict the waypoints for the autonomous vehicle to
follow. We adopted the CARLA simulator to demonstrate the efficacy of the
proposed framework. The main contributions of this work are:

  1. 1.

Inspired by human driving, we proposed a framework to learn the driving
policies for end-to-end autonomous driving. The proposed method utilizes the
global contextual representation from the semantic information and spatial
information from three RGB cameras in learning the driving policies.

  2. 2.

The transformer encoder is used for learning the long-range dependencies
between the embedded features. In addition, we encode the features from sensor
modalities with velocity information to be utilized by the transformer
encoder.

  3. 3.

The evaluation of the proposed method is carried out using the CARLA
simulator, and a comparative study illustrates that the proposed method
outperforms the baseline methods in terms of driving score and route
completion, completing different traffic tasks.

The rest of the paper is organized as follows: Section $2$ explains the
literature review. Section $3$ presents the proposed method framework and
Section $4$ corresponds to experimentation and results. Finally, Section $5$
concludes the paper with possible directions of future work.

##  2 Related Work

###  2.1 End-to-End Autonomous driving

End-to-end learning for realizing the control policies for autonomous vehicles
using visual information is an active area of research [1]. Compared to the
modular software stack for autonomous vehicles, end-to-end learning implicitly
predicts waypoints or directly vehicular control commands using imitation
learning [2]. The earlier work used only image data to determine the control
policies, but recent research is focused on using multi-modal data. Multi-
modal data leads to better scene understanding and allows learning multiple
data representations. . [3] proposed a model that uses RGB images and depth
map to capture improved scene understanding and generalization of different
driving scenarios. The fusion of features is demonstrated at low-dimensional
latent features to generate a set of actions for the vehicle to follow. [4]
explored segmentation-based visual abstraction to learn robust driving
policies. Similarly, [5] conducted controlled experiments to learn explicit
intermediate representation, which improved controlled driving policies. [6]
focus on RGB images and Lidar data to have a global understanding of the
surrounding. They proposed a multi-modal fusion transformer that integrates
information at multiple stages to generate driving policies. [7] explored
different stages of fusion of RGB and Lidar data and concluded that late-
fusion was the best design. Each input modality is encoded as a separate
branch and concatenated together. [8] employ dynamic vision camera and RGB
camera and fuse the information using self-attention to train the network
using imitation learning and determine the control policies.

Furthermore, Conditional imitation learning extends the imitation learning
framework by integrating higher-level navigational directions into the
decision-making process [9], but it is sensitive to new environments and
weather conditions. A significant performance drop is observed when conditions
are changed. The researcher employed simulators such as CARLA to simulate
different environmental conditions and test the trained framework. [10]
proposed a policy to map visual inputs to driving controls conditional to
surrounding semantics and object affordance. Moreover, researchers have
developed Reinforcement Learning (RL) agents to interact with simulated
environments. [11] trained an RL expert agent that outputs low-level actions
for autonomous vehicles using birds’ eye view images. [12] proposed an
interpretable RL framework that models complex urban scenarios in sequential
latent representation and is jointly learned with behaviors policy for
autonomous vehicles.

###  2.2 Transformer in End-to-End Autonomous driving

The transformer was initially developed for natural language processing tasks
[13], but recently they have out-perform classical neural networks in vision
applications [14, 15]. The self-attention module in the transformer
demonstrate improved global learning of sequential data and better feature
representation. The researcher uses the transformer encoder to fuse multi-
modal data. Transfuser exploits transformer block to combine intermediate
features representation from RGB images, and Lidar data [6]. [16] proposed
fusion Transformer, which inputs data from different view-point cameras and
Lidar to learn the global context for comprehensive scene understanding and
event detection. An encoder-decoder transformer configuration is employed to
generate low-level actions for the ego vehicle’s future trajectory. [17]
design a transformer-based neural prediction framework that considers social
interactions between different agents and generates possible trajectories for
autonomous vehicles. [18] determine the driving direction from visual features
acquired from images by using a novel framework consisting of a visual
transformer. The driving directions are decoded for human interpretability to
provide insight into learned features of the framework. [19], which considers
social interaction between agents on the road and forecasts their future
motion. The spatial-temporal dependencies were captured using a recurrent
neural network combined with a transformer encoder.

Figure 1: The architecture of the proposed method which is comprised of two
modules: perception block and waypoint prediction block. The perception module
generates the features extracted from the input three RGB cameras (center,
left, right) and the top-down semantic maps. These extracted features are then
embedded with the velocity information to be utilized by the transformer
encoder. The encoded features are then passed to the GRU-based waypoint
prediction module for the generation of next waypoints. (Best view in color)

##  3 Method

This section explains the proposed method for end-to-end learning control
policies for autonomous driving. The proposed method is composed of two
modules i) perception module and ii) waypoint prediction network. The
following sections detail the problem formulation, input/output representation
and model architecture, explaining the perception and waypoint prediction
modules.

###  3.1 Problem Formulation

In this work, an end-to-end imitation learning approach is adopted for the
point-to-point navigation problem, where the objective of the trained agent is
to safely reach the goal point by learning a policy $\pi^{*}$ that imitates
the expert policy $\pi$. The learned policy completes the given route by
avoiding obstacles and complying with the traffic rules. The expert policies
are collected using the behavior cloning approach, a supervised learning
approach of imitation learning, in a simulated environment. Suppose, the
dataset $D={(X^{j},Y^{j})}_{j=1}^{d}$ of size $d$ is collected that consist of
high dimensional observations $X$ from the simulated sensory modalities along
with the corresponding expert trajectories $Y$. The expert trajectories are
defined in vehicle local coordinate space and are set of $2$D waypoints
transformed in bird’s eye view (BEV) space, that is,
$Y={\mathbf{y_{t}}=(u_{t},v_{t})_{t=1}^{T}}$, where $u_{t}$ and $v_{t}$ are
the position information in horizontal and vertical directions, respectively.
The objective is to learn the policy $\pi$ with the collected dataset $D$ in a
supervised learning framework with the loss function $\mathcal{L}$ as
expressed in 1

$\arg\min_{\pi}\mathbb{E}_{(X,Y)\rightarrow D}[\mathcal{L}(Y,\pi(X))].$ (1)

In this urban setting, the high-dimensional observations include the center,
right and left camera and top-down semantic data for a single time step. It is
to be noted here that in our problem setting, the single time step is used
explicitly as prior work on using imitation learning for learning the control
policies illustrate that observational histories have minimal effect on the
performance gain.

###  3.2 Input and Output Representation

####  3.2.1 Input Representation

The proposed method utilizes two modalities: RGB cameras (left, center and
right) and semantic maps. The three RGB cameras provide a complete field of
view that mimics the human field of view. The semantic maps are converted to
bird-eye view representation that contains ground-truth lane information,
location, and status of traffic lights, vehicles, and pedestrians in the
vicinity of ego-vehicle. The top-down semantic maps are cropped to the
resolution of $256\times 256$ pixels. For all three cameras, to cater the
radial distortion, the resolution is cropped to $256\times 256$ from the
original camera’s resolution of $400\times 300$ pixels at the time of
extracting the data.

####  3.2.2 Output Representation

For the point-to-point navigation task, the proposed method predicts the
future trajectory $Y$ of the ego-vehicle in the vehicle coordinate space. The
future trajectory $Y$ is represented by a sequence of $2D$ waypoints,
$Y={\mathbf{y_{t}}=(u_{t},v_{t})_{t=1}^{T}}$, where $u_{t}$ and $v_{t}$ are
the position information in horizontal and vertical directions, respectively.
In the experimental analysis, we have utilized $T=4$ as the number of
waypoints.

###  3.3 Model Architecture

####  3.3.1 Perception Module

The perception module of the proposed method includes the backbone
architecture and transformer encoder network. The following subsections
explain the details of the perception module.
Backbone: For each input RGB image $I_{c,r,l}{\in\mathbb{R}^{3\times H\times
W}}$ from the center $(c)$, right $(r)$, and left $(l)$ camera sensor,
respectively, employ ResNet backbone for the feature extraction. Similarly,
the top-down semantic representations from the semantic camera are color-coded
first and then injected into the ResNet backbone for feature extraction. In
this setting, the pre-trained ResNet model trained on the ImageNet dataset is
utilized for generating the low-resolution feature maps
$f{\in\mathbb{R}^{C\times H\times W}}$ for each sensor modality. The last
layer of the ResNet model for each sensor modality generates ($B,512,8,8$)
dimension feature maps, where $B$ corresponds to batch size. The resolution of
these resulting feature maps is reduced to the dimension of ($B,512,1,1$) by
average pooling and flattened to a $512$ dimension vector. In order to keep
the features utilized by the transformer encoder, an additional projection
layer is employed that transforms the resulting $512$ dimension vector to a
$400$ dimension vector. Finally, all the $400$ dimension vectors from each
sensor modality that includes center, right, left, camera, and top-down
semantic maps are concatenated to give the final $1600$ dimension vector,
which is then reshaped to ($B,1,40,40$) to be utilized by the transformer
encoder. The following expressions in (2) summarize the computation of
features maps in the backbone network for each sensor modality and top-down
semantic maps,

$\displaystyle I_{c,r,l}^{\mathbb{R}^{3\times H\times
W}}\overset{Covn,bn,relu}{\rightarrow}f_{c,r,l}^{\mathbb{R}^{{64}\times
128\times 128}},$ (2) $\displaystyle f_{c,r,l}^{\mathbb{R}^{{64}\times
128\times 128}}\overset{maxpool}{\rightarrow}f_{c,r,l}^{\mathbb{R}^{{64}\times
64\times 64}},$ $\displaystyle f_{c,r,l}^{\mathbb{R}^{{64}\times 64\times
64}}\overset{l_{1}}{\rightarrow}f_{c,r,l}^{\mathbb{R}^{{64}\times 64\times
64}},$ $\displaystyle f_{c,r,l}^{\mathbb{R}^{{64}\times 64\times
64}}\overset{l_{2}}{\rightarrow}f_{c,r,l}^{\mathbb{R}^{{128}\times 32\times
32}},$ $\displaystyle f_{c,r,l}^{\mathbb{R}^{{64}\times 32\times
32}}\overset{l_{3}}{\rightarrow}f_{c,r,l}^{\mathbb{R}^{{256}\times 16\times
16}},$ $\displaystyle f_{c,r,l}^{\mathbb{R}^{{256}\times 16\times
16}}\overset{l_{4}}{\rightarrow}f_{c,r,l}^{\mathbb{R}^{{512}\times 8\times
8}},$ $\displaystyle f_{c,r,l}^{\mathbb{R}^{{512}\times 8\times
8}}\overset{avgpool}{\rightarrow}f_{c,r,l}^{\mathbb{R}^{{512}\times 1\times
1}},$ $\displaystyle f_{c,r,l}^{\mathbb{R}^{{512}\times 1\times
1}}\overset{flatten}{\rightarrow}f_{c,r,l}^{\mathbb{R}^{512}},$ $\displaystyle
f_{c,r,l}^{\mathbb{R}^{512}}\overset{proj}{\rightarrow}f_{c,r,l}^{\mathbb{R}^{400}}\overset{reshape}{\rightarrow}f_{c,r,l}^{\mathbb{R}^{1,40,40}},$

where $H,W=256$ respectively for each sensor modality and top-down semantic
maps. The expressions mentioned above only illustrate the computation of
feature extraction for a single batch. In our experiments, we have used the
batch size of $64$ to train the proposed method.
Transformer Encoder: In this work, a transformer encoder, specifically a
vision transformer, is employed to learn the contextual relationship between
the features and to generalize it to learn better feature representation. In
this context, the resulting features $f=\mathbb{R}^{1\times H\times W}$ is fed
to the transformer encoder by flattening into patches
$f_{p}=\mathbb{R}^{N\times(P^{2}C)}$, where $H$ and $W$ corresponds to the
resolution of input features from the backbone network, $C$ is the number of
channels, ($P,P$) is the size of each patch, and $N=HW/P$ illustrates the
number of patches and also the input sequence length. In addition, a learnable
position embedding is added to the input sequence, a trainable parameter with
the same dimension as the input sequence, so that the network infers the
spatial dependencies between different tokens at the train time. A velocity
embedding is also added to the $C$ dimension of the input sequence through a
linear layer, which includes the current velocity. Finally, the input
sequence, positional embeddings $E_{pos}$, and velocity embeddings $E_{vel}$
are element-wise summed together,which is mathematically expressed as
following (3),

$\displaystyle z_{o}=[f_{p}^{1}E;f_{p}^{2}E;...;f_{p}^{N}E]+E_{pos}+E_{vel},$
(3) $\displaystyle E\in\mathbb{R}^{(P^{2}.C)\times D},$ $\displaystyle
E_{pos}\in\mathbb{R}^{(N+1)\times D},E_{vel}\in\mathbb{R}^{(N+1)\times D},$
$\displaystyle{z}^{\prime}_{l}=MSA(LN(z_{l-1}))+z_{l-1}+z_{l-1},$
$\displaystyle
z_{l}=MLP(LN({z}^{\prime}_{l}))+{z}^{\prime}_{l}+{z}^{\prime}_{l},$

where MSA corresponds to multi-head self-attention, MLP is multi-layer
perceptron, LN is layer normalization, and $D$ corresponds to dimension. The
mathematical formulation of the multi-head self-attention is shown in Eqs (4)

$\displaystyle(\mathbf{Q},\mathbf{K},\mathbf{V})=\mathbf{z}\mathbf{W}_{QKV},$
(4) $\displaystyle\mathbf{W}_{QKV}\in\mathbb{R}^{D\times 3D_{h}},$
$\displaystyle A=softmax(\mathbf{Q}\mathbf{K^{T}})/\sqrt{D_{h}},$
$\displaystyle A\in\mathbb{R}^{N\times N},$ $\displaystyle
SA(\mathbf{z})=A\mathbf{v},$ $\displaystyle
MSA(\mathbf{z})=[SA_{1}(\mathbf{z};SA_{2}(\mathbf{z};...;SA_{j}(\mathbf{z})]\mathbf{W}_{msa},$
$\displaystyle\mathbf{W}_{msa}\in\mathbb{R}^{(j.D_{h})\times D}.$

where $Q$, $V$ and $K$ are the query, value and key vectors and $W$ is the
weight matrix. The output features from the MSA have the same dimension as the
input features. The transformer encoder applies the attention multiple times
throughout the architecture. The final output features from the transformer
encoder are then summed along the dimension to produce the $16$ dimensional
vector having the contextual representation of features from all the sensor
modalities. This resulting $16$ dimensional feature vector is injected into
the waypoint prediction module to predict waypoints.

####  3.3.2 Waypoint Prediction Module

The waypoint prediction module acts as a decoder for predicting the waypoints
using the encoded information from the transformer encoder. The resulting $16$
dimensional vector is passed through an MLP consisting of two hidden layers
having $256$ and $128$ units, respectively, to output the $64$ dimension
vector. The MLP layer is used for upsampling the vector dimension from $16$ to
$64$ and is related to experimental heuristics that produce better results in
terms of waypoint prediction. We have employed the auto-regressive GRU model
to predict the next waypoints that take the $64$ dimension feature vector to
initiate the hidden state of the GRU model. The GRU-based auto-regressive
model takes the current position and goal location as input, which helps the
network focus on the relevant context in the hidden states to predict the next
waypoints. The goal locations include the GPS points registered in the same
ego-vehicle coordinate frame as input to the GRU rather than the encoder
because of the colinear BEV space between the predicted waypoints and the goal
locations. In order to predict the next waypoints, a single layer GRU layer
followed by a linear layer is employed to predict $T=4$ future waypoints in
the ego-vehicle coordinate frame. Initially, the first unit of GRU is inputted
by ($0,0$) since the BEV space is centered with the ego vehicle’s position.

The predicted waypoints are then passed to the control module to generate
steer, throttle, and brake values. In this context, two PID controllers for
lateral and longitudinal control are used. The longitudinal controller takes
the average weighted magnitude of vectors between the waypoints of consecutive
time steps, whereas the lateral control takes their orientation. For the
control settings, we have used the settings as suggested by the following
codebase benchmarked on the CARLA dataset.

###  3.4 Experiments and Results

This section explains the experimental setup for dataset collection,
evaluation metrics, training details, and experimental results of the proposed
method compared to the state-of-the-art methods.

TABLE I: Dataset generation details using the CARLA simulator for the proposed
method

Maps |  | Training Data: Town01, Town02, Town03, Town04, Town06, Town07, Town10
---
Test Data: Town05
Weather Conditions | Clear sunset, Clear noon, Wet noon, Wet sunset, Cloudy noon, Cloudy sunset, Rainy noon, Rainy sunset
Non-player characters (NPCs) | Pedestrians, Car, Bicycle, Truck, Motorbike
Object Classes | 0:Unlabeled, 1:Pedestrian, 2:Road line, 3:Road, 4:Sidewalk, 5:Car, 6:Red traffic light, 7: Yellow traffic light, 8: Green traffic light
Routes |  | Tiny (only straight or one turn)
---
Short (100-500m)
Long (1000-2000m)
CARLA Version | 0.9.10

####  3.4.1 Dataset

In this work, we have utilized CARLA 0.9.10 simulator to generate the dataset.
The CARLA simulator provides $8$ publically available town. In our
experimentation, we have used $7$ towns to generate the training dataset, and
Town$05$ is used to evaluate the proposed method due to the large diversity in
traffic scenarios; for instance, single and multi lanes roads, highways,
bridges, and underpasses. Table I illustrates the dataset details that are
utilized in generating the training dataset to create a more varying
simulation environment. In addition, the non-player-characters (NPC) are also
spawned in the simulation to mimic real-world conditions. For generating the
dataset, an expert policy with the privileged information from the simulation
is rolled out to save the data at $2$FPS. The dataset includes left, right,
and center camera RGB images, top-down semantic map information, the
corresponding expert trajectory, speed data, and vehicular controls. The
trajectory includes $2$D waypoints transformed into bird-eye-view (BEV) space
in the vehicle’s local coordinate, whereas the steering, throttle, and brake
data are incorporated into the vehicular control data at the time of
recording. Inspired by [6] configurations, we have gathered the data by giving
a set of predefined routes to the expert in driving the ego-vehicle. The GPS
coordinates define the routes provided by the global planner and high-level
navigational commands (e.g., turn right, follow the lane, etc.) The routes are
categorized into three settings: long, short, and tiny. In the long routes
settings, the expert must drive $1000-2000$ meters, including $10$
intersections for each. On the short routes set, the expert must drive
$100-500$ meters having $3$ intersections for each. The tiny routes set
include only one driving configuration, for instance, one turn or one straight
go, that the expert must complete during the simulation. We have generated
around $60$ hours of the dataset, including $200K$ frames.

####  3.4.2 Evaluation Metrics

The proposed method’s efficacy is evaluated using the following metrics
indicated by the CARLA.
(1) Route Completion (RC): is the percentage of route distance $R_{j}$
completed by the agent in route $j$ averaged across the number of $N$ routes
is shown in Eq.(5),

$RC=\frac{1}{N}\sum_{j}^{N}R_{j}.$ (5)

The RC is reduced if the agent drives off the specified route by some
percentage of the route. This reduction in RC is defined by a multiplier
($1$-% off route distance).
(2) Infraction Multiplier (IM): as shown in Eq.(6) is defined as the geometric
series of infraction penalty coefficient, $p^{i}$, for every infraction
encountered by the agent along the route. Initially, the agent starts with the
ideal base score of $1.0$, which is reduced by a penalty coefficient for every
infraction. The penalty coefficient $p^{i}$ for each infraction is predefined.
If the agent collides with the pedestrian $p_{pedestrian}$, the penalty is set
to $0.50$; with other vehicles $p_{vehicles},$ it is set to $0.60$, $0.65$ for
collision with static layout $p_{stat}$, and $0.7$ if the agent breaks the red
light $p_{red}$. The penalty coefficient is defined as
$PC={p_{pedestrian},p_{vehicles},p_{stat},p_{red}}$,

$IM=\prod_{i}^{PC}(p^{i})^{infractions^{i}}.$ (6)

(3) Driving Score (DS): is computed by taking the product between the
percentage of the route completed by the agent $R_{j}$ and the infraction
multiplier ${IM}_{j}$of the route $j$ and averaged by the number of the routes
$N_{r}$. The higher the driving score corresponds to the better model.
Mathematically, the driving score (DS) is shown in Eq.(7),

$DS=\frac{1}{N_{r}}\sum_{j=1}^{N_{r}}{RC}_{j}{IM}_{j}(p^{i})^{infractions^{i}}.$
(7)

It is to be noted that if the ego vehicle deviates from the route $j$ for more
than $30$ meters or there is no action for $180$ seconds, then the evaluation
process on route $j$ will be stopped to save the computations cost and next
route will be selected for the evaluation process.

TABLE II: Quantitative comparison between the proposed method and the baseline
state-of-the-art methods. The comparative analysis is based on better driving
and route completion scores. The proposed method evaluation scores in terms of
driving and route completion scores are better than the baseline state-of-the-
art methods.

| Town05 Short | Town05 Long
---|---|---
Model | Avg. Driving Score (DS) $\uparrow$ | Avg. Route Completion (RC) $\uparrow$ |  | Avg. Driving Score (DS) $\uparrow$
---
Avg. Route Completion (RC) $\uparrow$
CILRS [20] | 7.47 | 13.40 | 3.68 | 7.19
LBC [21] | 30.97 | 55.01 | 7.05 | 32.09
AIM [6] | 49.00 | 81.07 | 26.50 | 60.66
Late Fusion [6] | 51.56 | 83.66 | 31.30 | 68.05
Geometric Fusion [6] | 54.32 | 86.91 | 25.30 | 69.17
Transfuser [6] | 54.52 | 78.41 | 33.15 | 56.36
SDC [22] | 58.357 | 84.270 | 40.21 | 52.43
Ours | 63.127 | 87.565 | 48.78 | 57.67
Expert | 84.67 | 98.59 | 38.60 | 77.47

####  3.4.3 Training Details

The proposed method is trained using the dataset collected from the CARLA
simulator by rolling out the expert model. In addition, we have used the pre-
trained ResNet model trained on the ImageNet dataset to extract the features
in the backbone network for each sensor modality. In training the proposed
network, we have added augmentation such as rotating and noise injection to
the training data, along with adjusting the waypoints labels. For the
transformer encoder, we have used the patch size of $4$, which gives the $16$
dimensional feature embedding. We have trained the proposed method using the
Pytorch library on RTX $3090$ having $24$ GB GPU memory for a total of $100$
epochs. In training, we have used the batch size of $64$ and an initial
learning rate of $10^{-4}$, which is reduced by a factor of $10$ after every
20 epochs. The $L_{1}$ loss function is used for training the proposed method.
Let $y_{t}^{gt}$ represent the ground-truth waypoints from the expert for the
timestep $t$; then the loss function is represented by the Eq.(8),

$\displaystyle\mathcal{L}=\sum_{t=1}^{T}\left\|y_{t}-y_{t}^{gt}\right\|_{1}.$
(8)

An AdamW optimizer is used in training with a weight decay set to $0.01$ and
beta values to the Pytorch defaults of $0.9$ and $0.99$. For every $5$ epoch,
the model is evaluated on the validation routes, and the epoch which gives the
highest driving score is saved.

####  3.4.4 Results

The proposed method’s efficacy is evaluated in different traffic scenarios
using Town05 of the CARLA simulator. For the quantitative comparison, we have
considered the following baseline methods. (1) CIRLS [20] is the conditional
imitation learning method in which the agents tend to learn the driving
policies conditioned on a high-level navigational command using only a single
front camera as a sensor modality. (2) LBC [21] corresponds to the knowledge
distillation approach in which, first, a teacher agent is trained having
access to the ground-truth BEV semantic maps for the prediction of waypoints
using the expert data. Later, an image-based student model with the teacher
model’s supervision is trained to learn the driving policies. It is to be
noted that only image data is available to the student model at the time of
training and cannot access privileged map information. We also explored the
sensor fusion-based method for learning the driving policies in the
experimental analysis. In this context, (3) the Transfuser [6] model is
considered for the comparative analysis. In this method, Lidar and RGB camera
data are fused at multiple levels using the self-attention transformer model,
and then the policies are determined by using an autoregressive waypoint
prediction model. This approach forms the baseline for the proposed method.
Inspired by [6], we have re-implemented the other three models along with the
Transfuser model for the comparative analysis. Moreover, these models include
AIM, Geometric fusion, and the Late-fusion methods to analyze the effect of
sensor fusion concerning the proposed method. (4) AIM [6] is an autoregressive
image-based waypoint prediction model that uses only single camera input for
the waypoint prediction. The latter two methods, geometric and late fusion,
employ the fusion of RGB camera and Lidar for predicting the driving policies.
(5) The geometric fusion [6] uses the multi-scale geometry based fusion as
discussed in [23] [24]. It involves both paradigms of fusion from image-to-
Lidar and Lidar-to-image feature fusion. (6) The late fusion [6] involves the
feature fusion at the end after each encoder network extracts the RGB camera
and Lidar features. This fusion is the element-wise summation of features, and
then the fused feature is passed to the waypoint prediction network for
learning the driving policies. Another baseline model SDC [22], uses the BEV
semantic depth cloud for learning end-to-end driving policies. The input to
the encoder network is from RGB images and depth information, GPS, and
speedometer information. The encoded features are then passed to the
controller block that includes the GRU for predicting the waypoints. It is to
be noted that in our experimentation, we have replicated the baseline models
to make a comparative analysis with the proposed method. In addition, since
the training of the proposed method involves different environmental
conditions, we evaluated the proposed method with the other baselines only on
ClearNoon weather conditions to better analyze the generalization of the
proposed method.

Figure 2: Qualitative results for the proposed method in different driving
conditions. In urban driving conditions, the proposed method is evaluated on
Town05 dataset of CARLA simulator, illustrating the performance of learned
driving policies using the proposed method on straight roads, keeping in the
lane, and turning left and right respectively. (a) illustrates the driving
policy is evaluated on the straight road while the ego-vehicle keeps in the
lane. (b) shows the behavior of ego-vehicle by using the learned driving
policy evaluated on the right turns. (c) Similarly, the learned driving policy
using the proposed method is illustrated on the left turn. Figure 3:
Qualitative results for the learned driving policies using the proposed method
on stopping at red traffic light.

The quantitative comparison is performed on the evaluation metrics of driving
score (DS), route completion (RC), and the infraction multiplier (IM). The
essential evaluation metric based on which the proposed method and the
baseline methods are evaluated is the driving score (DS), which is the product
of route completion (RC) and the infraction multiplier (IM). The higher the
driving score corresponds to a better model evaluation. It is also to be noted
that the model may have high RC and IM, but this does not satisfy that the
model has performed better than the driving score (DS) evaluation metric.
Table II illustrates the quantitative comparison between the proposed method
and the baselines. The CILRS model has a low driving and route completion
score compared to the proposed method. The reason for this is the improper
balance between the navigational commands that result in the poor performance
of the CILRS model. The LBC model performs significantly better than the CILRS
model because of the supervision provided by the teacher network to the image-
based student model, yet the driving score and the route completion illustrate
that the LBC modal fails to handle complex urban driving scenarios. The LBC
and CILRS performance significantly drops when tested on the long routes of
Town05. The AIM image-based model’s evaluation improves the driving and route
completion score compared to LBC and CILRS by adding the auto-regressive
waypoint method on the decoder side. However, the AIM model evaluation
illustrates the improvement, yet compared to the proposed method, its
performance in terms of driving score and route completion is not promising,
leaving room for the fact that by fusing different sensor modalities data, the
driving policies can be learned better. The comparative analysis between the
sensor fusion method that includes late fusion, geometric fusion, Transfuser,
and SDC has shown promising results in learning the driving policies in
comparison to the CILRS, LBC, and AIM models but cannot perform better
compared to the proposed method because of not providing the contextual
representation of the environment by covering the whole field of view and
semantic information. The intriguing question is that Lidar, in the form of
point-cloud data, provides complete information about the environment but the
problem with this data is the sparsity and limited spatial resolution. The
proposed method has performed better than the baseline methods and achieved a
driving score of $63.127$ and $87.565$ in the route completion for short
routes. Similarly, for the long routes, the proposed method has achieved a
driving score of $48.78$ and a route completion score of $57.67$.

We illustrate the proposed method’s efficacy in different tasks for
qualitative analysis as shown in Fig.2 and Fig.3, respectively. The learned
driving policy through the proposed method is displayed in moving straight,
stopping at the traffic light, and making left, and right turns. These results
illustrate that the policy learned using the proposed method show promising
results and complements the quantitative analysis of the proposed method with
other state-of-the-art baseline methods.

##  4 Conclusion

In this work, we have proposed a framework for learning driving policies for
end-to-end autonomous driving. The proposed method is composed of two modules:
the perception module and the waypoint prediction module. In order to
incorporate the spatial and contextual information about the environment, the
perception module receives the data from three RGB cameras and top-down
semantic information. First, the features are extracted through the backbone
network in the perception module. Then, the resulting features are fused and
embedded with velocity information. The embedded features are then passed to
the transformer encoder to learn the long relation dependencies between the
features. Finally, the GRU-based waypoint prediction network processes the
encoded features to output the next waypoints. For the comparative analysis,
the proposed method is quantitatively compared with the state-of-the-art
methods, resulting in better driving and route completion scores. Further, to
demonstrate the effectiveness of the proposed method, the qualitative results
are presented in different driving situations, illustrating that the driving
policies learned using the proposed method perform better. The future work
includes improving the perception module by adding the radar data, which
complements the camera data. In addition, the proposed method can be utilized
by designing a neural-network controller or a model predictive-based
controller.

## References

  * [1] J. Janai, F. Güney, A. Behl, A. Geiger _et al._ , “Computer vision for autonomous vehicles: Problems, datasets and state of the art,” _Foundations and Trends® in Computer Graphics and Vision_ , vol. 12, no. 1–3, pp. 1–308, 2020.
  * [2] A. Tampuu, T. Matiisen, M. Semikin, D. Fishman, and N. Muhammad, “A survey of end-to-end driving: Architectures and training methods,” _IEEE Transactions on Neural Networks and Learning Systems_ , 2020.
  * [3] Z. Huang, C. Lv, Y. Xing, and J. Wu, “Multi-modal sensor fusion-based deep neural network for end-to-end autonomous driving with scene understanding,” _IEEE Sensors Journal_ , vol. 21, no. 10, pp. 11 781–11 790, 2020.
  * [4] A. Behl, K. Chitta, A. Prakash, E. Ohn-Bar, and A. Geiger, “Label efficient visual abstractions for autonomous driving,” in _2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2020, pp. 2338–2345.
  * [5] B. Zhou, P. Krähenbühl, and V. Koltun, “Does computer vision matter for action?” _Science Robotics_ , vol. 4, no. 30, p. eaaw6661, 2019.
  * [6] A. Prakash, K. Chitta, and A. Geiger, “Multi-modal fusion transformer for end-to-end autonomous driving,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 7077–7087.
  * [7] I. Sobh, L. Amin, S. Abdelkarim, K. Elmadawy, M. Saeed, O. Abdeltawab, M. Gamal, and A. El Sallab, “End-to-end multi-modal sensors fusion system for urban automated driving,” 2018.
  * [8] F. Munir, S. Azam, B.-G. Lee, and M. Jeon, “Multi-modal fusion for sensorimotor coordination in steering angle prediction,” _arXiv preprint arXiv:2202.05500_ , 2022.
  * [9] F. Codevilla, M. Müller, A. López, V. Koltun, and A. Dosovitskiy, “End-to-end driving via conditional imitation learning,” in _2018 IEEE international conference on robotics and automation (ICRA)_. IEEE, 2018, pp. 4693–4700.
  * [10] A. Zhao, T. He, Y. Liang, H. Huang, G. V. d. Broeck, and S. Soatto, “Sam: Squeeze-and-mimic networks for conditional visual driving policy learning,” _arXiv preprint arXiv:1912.02973_ , 2019.
  * [11] Z. Zhang, A. Liniger, D. Dai, F. Yu, and L. Van Gool, “End-to-end urban driving by imitating a reinforcement learning coach,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 15 222–15 232.
  * [12] J. Chen, S. E. Li, and M. Tomizuka, “Interpretable end-to-end urban autonomous driving with latent deep reinforcement learning,” _IEEE Transactions on Intelligent Transportation Systems_ , 2021.
  * [13] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” _Advances in neural information processing systems_ , vol. 30, 2017.
  * [14] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly _et al._ , “An image is worth 16x16 words: Transformers for image recognition at scale,” _arXiv preprint arXiv:2010.11929_ , 2020.
  * [15] N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in _European conference on computer vision_. Springer, 2020, pp. 213–229.
  * [16] H. Shao, L. Wang, R. Chen, H. Li, and Y. Liu, “Safety-enhanced autonomous driving using interpretable sensor fusion transformer,” _arXiv preprint arXiv:2207.14024_ , 2022.
  * [17] Z. Huang, X. Mo, and C. Lv, “Multi-modal motion prediction with transformer-based neural network for autonomous driving,” in _2022 International Conference on Robotics and Automation (ICRA)_. IEEE, 2022, pp. 2605–2611.
  * [18] J. Dong, S. Chen, S. Zong, T. Chen, and S. Labi, “Image transformer for explainable autonomous driving system,” in _2021 IEEE International Intelligent Transportation Systems Conference (ITSC)_. IEEE, 2021, pp. 2732–2737.
  * [19] L. L. Li, B. Yang, M. Liang, W. Zeng, M. Ren, S. Segal, and R. Urtasun, “End-to-end contextual perception and prediction with interaction transformer,” in _2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2020, pp. 5784–5791.
  * [20] F. Codevilla, E. Santana, A. M. López, and A. Gaidon, “Exploring the limitations of behavior cloning for autonomous driving,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 9329–9338.
  * [21] D. Chen, B. Zhou, V. Koltun, and P. Krähenbühl, “Learning by cheating,” in _Conference on Robot Learning_. PMLR, 2020, pp. 66–75.
  * [22] O. Natan and J. Miura, “End-to-end autonomous driving with semantic depth cloud mapping and multi-agent,” _IEEE Transactions on Intelligent Vehicles_ , 2022.
  * [23] M. Liang, B. Yang, W. Zeng, Y. Chen, R. Hu, S. Casas, and R. Urtasun, “Pnpnet: End-to-end perception and prediction with tracking in the loop,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 11 553–11 562.
  * [24] M. Liang, B. Yang, Y. Chen, R. Hu, and R. Urtasun, “Multi-task multi-sensor fusion for 3d object detection,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 7345–7353.

|  Shoaib Azam received a B.S. degree in Engineering Sciences from Ghulam
Ishaq Khan Institute of Science and Technology, Pakistan, in 2010, and an M.S.
degree in Robotics and Intelligent Machine Engineering from National
University of Science and Technology, Pakistan, in 2015. He received his Ph.D.
degree from Gwangju Institute of Science and Technology, Gwangju, South Korea,
in 2021. Currently he is working as a postdoctoral researcher, at Gwangju
Institute of Science and Technology, South Korea, where he is working on
developing end-to-end solutions using deep learning for autonomous driving and
robotics. His research interests includes deep learning, autonomous driving,
sensorimotor learning, and representation learning.
---|---
|  Farzeen Munir received a B.S. degree in Electrical Engineering from
Pakistan Institute of Engineering and Applied Sciences, Pakistan in 2013, and
an M.S. degree in System Engineering from Pakistan Institute of Engineering
and Applied Sciences, Pakistan in 2015. She received her Ph.D. degree from
Gwangju Institute of Science and Technology, Gwangju, South Korea, in 2022.
Currently she is working as a postdoctoral researcher, at Gwangju Institute of
Science and Technology, South Korea, where she is working on developing
perception and navigation problems related to indoor mobile robotics. Her
current research interests include, machine Learning, deep neural network,
autonomous driving and computer vision.
---|---
|  Moongu Jeon received a B.S. degree in Architectural Engineering from Korea
University, Seoul, South Korea, in 1988, and M.S. and Ph.D. degrees in
Computer Science and Scientific Computation from the University of Minnesota,
Minneapolis, MN, USA, in 1999 and 2001, respectively. As a Post-Graduate
Researcher, he worked on optimal control problems at the University of
California at Santa Barbara, Santa Barbara, CA, USA, from 2001 to 2003, and
then moved to the National Research Council of Canada, where he worked on the
sparse representation of high-dimensional data and image processing until
2005. In 2005, he joined the Gwangju Institute of Science and Technology,
Gwangju, South Korea, where he is currently a Full Professor with the School
of Electrical Engineering and Computer Science. His current research interests
are in machine learning, computer vision, and artificial intelligence.
---|---