# End-to-end Reinforcement Learning
for Time-Optimal Quadcopter Flight

Robin Ferede1, Christophe De Wagter1, Dario Izzo2, Guido C.H.E. de Croon1
*This work was supported by ESA1The authors are with the Micro Air Vehicle Lab
of the Faculty of Aerospace Engineering, Delft University of Technology, 2629
HS Delft, The Netherlands<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>authors are with the Advanced Concepts Team,
European Space Agency, Keplerlaan 1, 2201 AZ, Noordwijk, The Netherlands.
<EMAIL_ADDRESS>
###### Abstract

Aggressive time-optimal control of quadcopters poses a significant challenge
in the field of robotics. The state-of-the-art approach leverages
reinforcement learning (RL) to train optimal neural policies. However, a
critical hurdle is the sim-to-real gap, often addressed by employing a robust
inner loop controller —an abstraction that, in theory, constrains the
optimality of the trained controller, necessitating margins to counter
potential disturbances. In contrast, our novel approach introduces high-speed
quadcopter control using end-to-end RL (E2E) that gives direct motor commands.
To bridge the reality gap, we incorporate a learned residual model and an
adaptive method that can compensate for modeling errors in thrust and moments.
We compare our E2E approach against a state-of-the-art network that commands
thrust and body rates to an INDI inner loop controller, both in simulated and
real-world flight. E2E showcases a significant 1.39-second advantage in
simulation and a 0.17-second edge in real-world testing, highlighting end-to-
end reinforcement learning’s potential. The performance drop observed from
simulation to reality shows potential for further improvement, including
refining strategies to address the reality gap or exploring offline
reinforcement learning with real flight data.

###### Index Terms:

Time optimal control, Reinforcement Learning, end-to-end control, reality gap,
sim-to-real transfer, abstraction

Figure 1: Drone racing with 2 RL Control Approaches: 1) E2E Network: Computes
direct motor commands, learns compensation for unmodeled moments and thrust,
trained with a learned residual model. 2) INDI Network: Computes thrust and
body rate commands, uses an INDI inner loop controller, trained with first
order INDI model.

##  I INTRODUCTION

The demand for autonomous quadcopters capable of high-speed flight has been
steadily increasing, driven by the need for covering larger distances in
various applications[1]. However, achieving efficient and agile high-speed
flight remains a significant challenge, requiring the development of
computationally efficient time-optimal control algorithms.

Traditionally, quadcopter control relied on established methods, including
Differential-Flatness-Based Controllers (DFBC) [2, 3, 4, 5] and Nonlinear
Model Predictive Control (NMPC) [6, 7, 8, 9, 10, 11, 12, 13, 9]. While these
methods represented important steps forward, they faced difficulties with
unmodeled effects [14].

Recent advancements in quadcopter control have brought Reinforcement Learning
(RL) to the forefront of research [15]. While simulation studies such as [16,
17] have demonstrated RL’s capability to achieve high-speed flight, real-world
applications such as [18, 19, 20, 21] have had to devise clever solutions to
address the issues posed by the reality gap. This is because RL heavily relies
on accurate simulation environments, which become increasingly challenging at
high quadcopter speeds due to complex dynamics. Consequently, RL methods in
actual flight often adopt an abstraction approach in which the network outputs
high-level control commands that are executed by manually pre-tuned low-level
controllers.

For instance, in [18], an end-to-end network is trained in simulation for
racing and then used in real flight to generate a trajectory tracked by an MPC
controller. In [19], RL provides thrust and body rate commands, which are
tracked with an inner loop controller. In a comparison study [20], it is
argued that providing thrust and body rate commands strikes the optimal
balance, as direct motor commands, while effective in simulation, fail to
bridge the reality gap for successful real-life flight. Particularly
noteworthy is the achievement in [21], where an RL controller outperformed a
human racing pilot. This success was attributed to a combination of
abstraction (thrust and body rate commands) and accurate modeling of the
closed-loop system using a learned residual model.

However, leaning on abstractions, in theory, places a limitation on the
platform’s maneuverability. This is because, in the end, it is the lowest-
level controller that decides which actuator to saturate. On the contrary,
methods like end-to-end (E2E) approaches, which operate without such
abstractions, hold promise in enabling exceptionally agile maneuvers and
pushing performance to its absolute limits.

There are already some noteworthy instances of neural networks directly
governing motor control. For instance, in [22], direct motor control is
achieved by a neural network, focusing on low-level controls in waypoints
tracking and vehicle recovery from harsh initialization. Although this work
was concerned with end-to-end control, it did not focus on time optimality or
high-speed flight. In more recent developments, methods using supervised
learning from optimal trajectory datasets have shown remarkable success in E2E
control in the context of high-speed quadcopter flight [23, 24]. The
strategies used here effectively address the reality gap by using an adaptive
control approach that both measures and compensates for modeling errors.

In this article, we introduce a novel method for achieving high-speed
quadcopter control through end-to-end reinforcement learning (E2E), where
direct motor commands are generated. To effectively bridge the reality gap,
our approach combines a residual moment and thrust model, acquired from flight
data (akin to the approach in [21]), combined with the adaptive techniques
described in [23, 24], applied to both thrust and moment model. Our study
involves a comparative analysis, pitting the performance of our E2E approach
against that of a state-of-the-art network generating thrust and rate commands
for an inner loop INDI controller [25].

##  II METHODOLOGY

###  II-A Problem Definition

The drone racing problem is modeled as an MDP defined by
$(\mathcal{S},\mathcal{A},\mathcal{P},r,\rho_{0},\gamma)$. The RL agent starts
in state $s_{t}\in\mathcal{S}$ from distribution $\rho_{0}$. It uses
stochastic policy $\pi(a_{t}|s_{t})$ to select a continuous action
$a_{t}\in\mathcal{A}$, transitioning to $s_{t+1}$ with probability
$\mathcal{P}_{s_{t},s_{t+1}}^{a_{t}}$ and receiving reward $r_{t}$. The
objective is to optimize the parameters $\theta$ of a stochastic neural
network policy $\pi_{\theta}(a_{t}|s_{t})$ to maximize expected return over an
infinite horizon
$\max_{\theta}\mathbb{E}_{\pi_{\theta}}\left[\sum_{t=0}^{\infty}\gamma^{t}r_{t}\right]$.

The racing setup consists of four square gates (1x1m) in a 4x3m rectangle.
Each gate is rotated 90 degrees relative to the previous gate as seen in Fig.
3. The goal is to fly in a circular trajectory, passing through all gates as
quickly as possible. We quantify this by using a reward function consisting of
a progress reward that quantifies advancement toward the target gate, a gate
reward that encourages gate passage through the center, and a collision
penalty.

$\displaystyle
r(k)=\begin{cases}+10-10||\mathbf{p}_{k}-\mathbf{p}_{g_{k}}||,&\text{if gate
passed}\\\ -10,&\text{if collision}\\\
||\mathbf{p}_{k}-\mathbf{p}_{g_{k}}||-||\mathbf{p}_{k-1}-\mathbf{p}_{g_{k}}||&\text{otherwise}\\\
\end{cases}$

Here, $\mathbf{p}_{g_{k}}$ represents the position of the center of the
current target gate, and $\mathbf{p}_{k}$, $\mathbf{p}_{k-1}$ are the drone’s
current and previous positions. A gate is deemed successfully passed when the
drone intersects the 1 by 1 meter gate boundary. A collision is registered
when the drone either makes contact with the ground or passes through the gate
plane outside the gate boundary. This reward function, in conjunction with a
state transition model to be detailed in the subsequent section, will be
employed to create a virtual environment and conduct training using the PPO
algorithm [26] in the Python library Stable-Baselines3 [27]. During training,
we will simulate 100 drones in parallel, using a discount factor
$\gamma=0.999$ and a maximum episode duration of 12 seconds.

Figure 2: Flight test comparison E2E vs. INDI Net (5 Repetitions): 1) E2E
outpaces INDI in the first lap (starting in hover). 2) Laps 2-6 show
comparable performance. 3) E2E has a slight advantage over INDI in total track
completion time. 4) Relative progress of all flights E2E leads by 0.17
seconds.

###  II-B E2E Network

####  II-B1 Quadcopter model

We adopt a quadcopter model similar to the one outlined in [23, 24]. The
quadcopter’s state and control inputs are defined as follows:

$\displaystyle\mathbf{x}=[\mathbf{p},\mathbf{v},\boldsymbol{\lambda},\mathbf{\Omega},\mathbf{\boldsymbol{\omega}},\mathbf{M}_{ext},\mathbf{F}_{ext}]^{T}\quad\mathbf{u}=[u_{1},u_{2},u_{3},u_{4}]^{T}$

Here $\mathbf{p}$ is the position of the drone, $\mathbf{v}$ is the velocity,
$\boldsymbol{\lambda}$ are the Euler angles, $\mathbb{\Omega}$ are the body
rates and $\boldsymbol{\omega}$ are the propeller speed in RPM with a range
between $\omega_{min}=3000$ and $\omega_{max}=11000$. $\mathbf{u}$ represents
the motor RPM commands. Similar to the approach in [23, 24], we include
external (specific) forces and moments as part of the state representation.
These external influences will be measured onboard to mitigate modeling
inaccuracies. The equations of motion are expressed as follows:

$\displaystyle\dot{\mathbf{p}}$ $\displaystyle=\mathbf{v}$
$\displaystyle\dot{\mathbf{v}}$
$\displaystyle=g\mathbf{e_{3}}+R(\mathbf{\lambda})[\mathbf{F}_{mod}+\mathbf{F}_{ext}]$
(1) $\displaystyle\dot{\lambda}$
$\displaystyle=Q(\mathbf{\lambda})\mathbf{\Omega}$ $\displaystyle
I\dot{\mathbf{\Omega}}$ $\displaystyle=-\mathbf{\Omega}\times
I\mathbf{\Omega}+\mathbf{M}_{mod}+\mathbf{M}_{ext}$
$\displaystyle\dot{\boldsymbol{\omega}}$
$\displaystyle=(\mathbf{u}-\boldsymbol{\omega})/\tau$
$\displaystyle\dot{\mathbf{F}}_{ext}$
$\displaystyle=0\quad\dot{\mathbf{M}}_{ext}=0$

In contrast to [23, 24], $\mathbf{M}_{mod}$ and $\mathbf{F}_{mod}$ are
formulated as a hybrid model, comprising a nominal model rooted in physical
principles [28] and a residual black-box model fine-tuned using empirical
data:

$\displaystyle\mathbf{F}_{mod}=\mathbf{F}_{nom}+\mathbf{F}_{res}\quad\mathbf{M}_{mod}=\mathbf{M}_{nom}+\mathbf{M}_{res}$

The nominal model, identical to the one identified for the Bebop drone in
[23], is defined as:

$\displaystyle\mathbf{F}_{nom}=\begin{bmatrix}-k_{x}v^{B}_{x}\sum_{i=1}^{4}\omega_{i}\\\
-k_{y}v^{B}_{y}\sum_{i=1}^{4}\omega_{i}\\\
-k_{\omega}\sum_{i=1}^{4}\omega_{i}^{2}-k_{z}v^{B}_{z}\sum_{i=1}^{4}\omega_{i}-k_{h}(v^{B2}_{x}+v^{B2}_{y})\end{bmatrix}$
$\displaystyle\mathbf{M}_{nom}=\begin{bmatrix}k_{p}(\omega_{1}^{2}-\omega_{2}^{2}-\omega_{3}^{2}+\omega_{4}^{2})+k_{pv}v^{B}_{y}\\\
k_{q}(\omega_{1}^{2}+\omega_{2}^{2}-\omega_{3}^{2}-\omega_{4}^{2})+k_{qv}v^{B}_{x}\\\
k_{r1}(-\omega_{1}+\omega_{2}-\omega_{3}+\omega_{4})+\\\ \ldots
k_{r2}(-\dot{\omega_{1}}+\dot{\omega_{2}}-\dot{\omega_{3}}+\dot{\omega_{4}})-k_{rr}r\end{bmatrix}$

The residual model consists of two small neural networks, one for residual
(mass normalized) thrust and the other for residual moment:

$\displaystyle\mathbf{F}_{res}=[0,0,-\mathcal{N}_{T}(\boldsymbol{\omega},\mathbf{v}^{B})]^{T}\quad\mathbf{M}_{res}=\mathcal{N}_{M}(\boldsymbol{\omega},\mathbf{v}^{B},\boldsymbol{\Omega})$

Both $\mathcal{N}_{T}$ (7 inputs 1 output) and $\mathcal{N}_{M}$ (10 inputs 3
outputs) utilize a neural network architecture featuring a single hidden layer
comprising 32 neurons.

####  II-B2 Policy

The policy is implemented as a three-layered, fully connected neural network
with a ReLU activation function, as visually depicted in Fig. 1. The network
takes 24 inputs containing the quadcopter’s state, as well as the modeling
mismatches represented by $\mathbf{M}_{ext}$ and $\mathbf{F}_{ext}$, along
with information current and future gates.

$\displaystyle\mathbf{x}_{in}=[\mathbf{p}^{g_{i}},\mathbf{v}^{g_{i}},\boldsymbol{\lambda}^{g_{i}},\mathbf{\Omega},\mathbf{\boldsymbol{\omega}},\mathbf{M}_{ext},\mathbf{F}_{ext},\mathbf{p}_{g_{i+1}}^{g_{i}},\psi_{g_{i+1}}^{g_{i}}]^{T}$

Here, the superscript ${}^{g_{i}}$ denotes that the variables are represented
in the reference frame aligned with the target gate $g_{i}$. The symbols
$\mathbf{p}_{g{i+1}}$ and $\psi_{g_{i+1}}$ respectively indicate the position
and orientation of the succeeding gate (i.e., the one following the target
gate). The network has 4 outputs corresponding to rpm commands $\mathbf{u}$.
Since our approach employs a stochastic policy, the network’s outputs serve as
the means of a normal distribution governing the rpm commands. The standard
deviation of this distribution is a learned parameter of the network. For the
purpose of generating a control command, whether during training or testing,
we draw samples from this distribution and constrain the result within the
minimum and maximum rpm limits.

Figure 3: Trajectory comparison E2E vs INDI: simulated runs and fastest
achieved real flights. The performance gap is most significant in simulation
with E2E leading by 1.39 seconds. In the real flights, this difference is
reduced to 0.17 seconds.

####  II-B3 Training

To train within the RL framework, we convert our continuous-time dynamical
model into a discrete-time MDP using a time step of 0.01 via the Forward Euler
method. The episode is initialized by uniformly sampling initial states from
the following intervals:

$\displaystyle x$ $\displaystyle\in[-\tfrac{1}{2},\tfrac{1}{2}]+x_{s}$
$\displaystyle y$ $\displaystyle\in[-\tfrac{1}{2},\tfrac{1}{2}]+y_{s}$
$\displaystyle z$ $\displaystyle\in[-\tfrac{1}{2},\tfrac{1}{2}]+z_{s}$
$\displaystyle v_{x}$ $\displaystyle\in[-\tfrac{1}{2},\tfrac{1}{2}]$
$\displaystyle v_{y}$ $\displaystyle\in[-\tfrac{1}{2},\tfrac{1}{2}]$
$\displaystyle v_{z}$ $\displaystyle\in[-\tfrac{1}{2},\tfrac{1}{2}]$
$\displaystyle\phi$ $\displaystyle\in[-\tfrac{2\pi}{9},\tfrac{2\pi}{9}]$
$\displaystyle\theta$ $\displaystyle\in[-\tfrac{2\pi}{9},\tfrac{2\pi}{9}]$
$\displaystyle\psi$ $\displaystyle\in[-\pi,\pi]$ $\displaystyle p$
$\displaystyle\in[-1,1]$ $\displaystyle q$ $\displaystyle\in[-1,1]$
$\displaystyle r$ $\displaystyle\in[-1,1]$ $\displaystyle\boldsymbol{\omega}$
$\displaystyle\in[\omega_{min},\omega_{max}]^{4}$ (2)

Here $x_{s},y_{s},z_{s}$ is the starting point of the race track. Also
$\mathbf{M}_{ext}$ and $\mathbf{F}_{ext}$ are sampled from

$\displaystyle M_{x,ext},M_{y,ext}$ $\displaystyle\in[-0.03,0.03]$
$\displaystyle M_{z,ext}$ $\displaystyle\in[-0.01,0.01]$ $\displaystyle
F_{x,ext},F_{y,ext}$ $\displaystyle=0$ $\displaystyle F_{z,ext}$
$\displaystyle\in[-0.5,0.5]$

###  II-C INDI Network

####  II-C1 Quadcopter model

Integrating the INDI controller into the loop streamlines the quadcopter
modeling process considerably. With the INDI Network providing rate and thrust
commands, our RL environment now primarily focuses on simulating the INDI
controller’s response, specifically how actual thrust and rates relate to the
commanded values. As illustrated in [25], the INDI controller’s command
response can be expressed as a first-order delay model. The remaining
quadcopter modeling tasks only involve the kinematic equations and a basic
drag model:

$\displaystyle\mathbf{x}=[\mathbf{p},\mathbf{v},\boldsymbol{\lambda},\mathbf{\Omega},T]^{T}\quad\mathbf{u}=[\mathbf{\Omega}_{cmd},T_{cmd}]^{T}$

The equations of motion are expressed as:

$\displaystyle\dot{\mathbf{p}}$ $\displaystyle=\mathbf{v}$
$\displaystyle\dot{\mathbf{v}}$
$\displaystyle=g\mathbf{e_{3}}+R(\mathbf{\lambda})\mathbf{F}$ (3)
$\displaystyle\dot{\lambda}$
$\displaystyle=Q(\mathbf{\lambda})\mathbf{\Omega}$
$\displaystyle\dot{\mathbf{\Omega}}$
$\displaystyle=(\mathbf{\Omega}_{cmd}-\mathbf{\Omega})/\tau_{\Omega}$
$\displaystyle\dot{T}$ $\displaystyle=(T_{cmd}-T)/\tau_{T}$

Here, we have identified the thrust and rate delays of the INDI controller,
$\tau_{\Omega}$ and $\tau_{T}$, as 0.03 based on flight data. The specific
force $\mathbf{F}$ is given by
$\mathbf{F}=[-d_{x}v_{x}^{B},-d_{y}v_{y}^{B},-T]^{T}$ where $d_{x}=0.34$ and
$d_{y}=0.43$ are also identified from flight data.

An inherent limitation of relying on an INDI inner loop controller is that it
requires some margins to reject disturbances and track the desired command.
This means that a trade-off must be made between how much margin is kept to
reject disturbances (which makes the drone slower) or how little margin is
acceptable before actuator limits are exceeded and the INDI can not track the
reference anymore. We establish artificial boundaries for thrust and rate
based on the fastest stable flight performance we could achieve in our setup:

$\displaystyle p_{cmd},q_{cmd},r_{cmd}\in[-3,3]\quad T_{cmd}\in[0,15]$

####  II-C2 Policy

The policy closely resembles the E2E Network but differs only in the number of
inputs due to the system simplifications. The network takes 17 inputs,
consisting of the state and gate information:

$\displaystyle\mathbf{x}_{in}=[\mathbf{p}^{g_{i}},\mathbf{v}^{g_{i}},\boldsymbol{\lambda}^{g_{i}},\mathbf{\Omega},T,\mathbf{p}_{g_{i+1}}^{g_{i}},\psi_{g_{i+1}}^{g_{i}}]^{T}$

The network has 4 outputs corresponding to the thrust and body rate control
command. Again the network’s outputs are the means of normal distributions
governing thrust and body rate commands, along with a learned standard
deviation.

####  II-C3 Training

In the training phase, we once more convert our dynamical model into a
discrete-time MDP. We uniformly sample initial states from the same intervals
as described in Eq. II-B3, where applicable. The thrust is sampled from the
range [7.4, 7.6].

##  III EXPERIMENTAL SETUP

For our experiments, we use the Parrot Bebop 1 quadcopter. While it does not
have the high-speed performance of a racing drone, its unique characteristics
make it a compelling choice. The Bebop 1 features a notably flexible frame,
posing a more intricate challenge for modeling compared to the rigid racing
counterparts. Furthermore, it provides an excellent opportunity to push the
boundaries of actuation within a confined space. The onboard software of the
Bebop is replaced by the Paparazzi-UAV open-source autopilot [29]. Computation
occurred in real-time on a Parrot P7 dual-core CPU Cortex A9 processor. The
Bebop featured an MPU6050 IMU sensor for specific force and angular velocity
measurements, as well as RPM measurements for each propeller, essential for
our control method.

Figure 4: Experimental setup: The drone’s position and attitude are obtained
from the Optitrack motion capture system and fused with IMU data by an
extended Kalman filter. Image from [23]

Flight tests were conducted in The CyberZoo, a 10x10x7 m flight arena at TU
Delft’s Aerospace Engineering faculty, equipped with an OptiTrack motion
capture system for real-time position and attitude data. An extended Kalman
filter is used to fuse Optitrack position and attitude measurements with the
accelerometer and gyro data from the IMU. This integration allows us to
estimate position, velocity, attitude, and IMU biases accurately.

##  IV RESULTS & DISCUSSION

We conducted tests using the trained E2E and INDI networks, involving both
simulation roll-outs and real flight tests covering 6 laps on the race track.
The real flights were repeated 5 times for both networks. Our assessment
focused on time optimality as the primary performance metric. Fig. 2 provides
an overview of the real flights conducted, along with corresponding values
found in Table I. Fig. 3 highlights the fastest flights achieved by both
controllers compared to their simulated counterparts.

Our key observation is that E2E outperforms INDI on the 6-lap racing task,
both in simulation and in real-world scenarios. The most substantial
performance gap is evident in simulation, with E2E completing the track 1.39
seconds faster. In the real world, the performance difference, while reduced,
remains existent, with an average improvement of 0.05 seconds over the entire
track. When comparing the fastest runs, E2E is in the lead by 0.17 seconds. An
intriguing insight, highlighted in Fig. 2, reveals that E2E Networks primarily
leverages its advantage during the first lap when the drone starts in hover.
In laps 2 through 6, both methods exhibit remarkably similar performance. This
observation highlights the E2E Net’s capability to efficiently perform rapid
accelerations.

When comparing the results between simulated runs and real-life flights,
several noteworthy observations come to light. In the simulated environment,
the INDI controller achieves a track completion time of 17.14, with only a
minor increase to 17.40 seconds on average in the real flight tests. In
contrast, the E2E method exhibits a more pronounced reality gap, with
completion times increasing from 15.75 in simulation to an average of 17.35 in
real-world scenarios. Despite this substantial gap between simulation and
reality, the fact that E2E maintains a performance advantage over INDI, albeit
modest, underscores the success of the sim-to-real transfer. Furthermore,
these findings suggest that E2E may hold untapped potential for further
enhancing real-world drone performance.

While the E2E net addresses thrust and moment modeling errors, it’s important
to note that certain challenges remain unaddressed. In Fig. 5, we observe
deviations in motor RPM between actual and modeled values during the fastest
flight, potentially explaining performance drops in the real world. The INDI
Net has a smaller simulation-to-reality gap, but it also encounters modeling
errors. In Fig. 6 we can see how the observed rates and thrust also deviate
from the modeled first-order delay. These deviations could explain the
different trajectory shapes observed in the real INDI flight in Fig. 3.

| lap1 | lap2 | lap3 | lap4 | lap5 | lap6 | total
---|---|---|---|---|---|---|---
E2E Sim | 2.97 | 2.51 | 2.56 | 2.55 | 2.59 | 2.57 | 15.75
INDI Sim | 3.20 | 2.82 | 2.75 | 2.80 | 2.81 | 2.76 | 17.14
E2E1 | 3.23 | 2.79 | 2.81 | 2.84 | 2.84 | 2.83 | 17.34
E2E2 | 3.22 | 2.74 | 2.79 | 2.80 | 2.76 | 2.79 | 17.09
E2E3 | 3.20 | 2.73 | 2.91 | 2.76 | 2.85 | 2.83 | 17.28
E2E4 | 3.22 | 2.85 | 2.89 | 2.87 | 2.87 | 2.81 | 17.50
E2E5 | 3.24 | 2.81 | 2.85 | 2.86 | 2.87 | 2.88 | 17.52
Mean | 3.22 | 2.79 | 2.85 | 2.82 | 2.84 | 2.83 | 17.35
INDI1 | 3.30 | 2.86 | 2.80 | 2.86 | 2.91 | 2.79 | 17.51
INDI2 | 3.26 | 2.79 | 2.80 | 2.79 | 2.84 | 2.77 | 17.26
INDI3 | 3.25 | 2.80 | 2.83 | 2.86 | 2.76 | 2.84 | 17.35
INDI4 | 3.27 | 2.78 | 2.82 | 2.76 | 2.83 | 2.83 | 17.29
INDI5 | 3.31 | 2.80 | 2.86 | 2.81 | 2.87 | 2.94 | 17.58
Mean | 3.28 | 2.80 | 2.82 | 2.82 | 2.84 | 2.83 | 17.40
TABLE I: Lap times (in seconds) fastest flight in bold. Figure 5: Modeling
errors in the first order actuator delay model used for the E2E Net Figure 6:
Modeling errors in the first order delay model of the INDI controller used for
the INDI Net.

##  V CONCLUSION

We have presented a novel neural network approach for achieving high-speed
quadcopter flight through end-to-end reinforcement learning. We addressed the
reality gap issue by using an adaptive method alongside a learned residual
model. Comparing our approach to a state-of-the-art INDI network, we observed
that our E2E network can outperform the state of the art both in simulation in
real flights. In the simulated environment, E2E exhibits a notable advantage,
completing tasks 1.39 seconds faster. This performance gap, while smaller in
real flight scenarios, remains significant and showcases the competitiveness
of E2E architectures when compared to the state of the art. The performance
contrast observed between simulation and real-life scenarios implies that our
E2E approach might exhibit greater sensitivity to modeling errors, as opposed
to the robustness of the INDI controller. Future efforts could be directed
toward enhancing the E2E network’s ability to adapt to modeling errors.
Additionally, apart from addressing moment and thrust offsets, it’s also
possible to identify other discrepancies in the model, such as variations in
battery voltage or maximum RPM. Alternatively, a promising avenue for future
research may involve refining our network through the application of offline
RL techniques using real flight data.

## References

  * [1] M. Hassanalian and A. Abdelkefi, “Classifications, applications, and design challenges of drones: A review,” _Progress in Aerospace Sciences_ , vol. 91, pp. 99–131, 2017.
  * [2] M. J. Van Nieuwstadt and R. M. Murray, “Real-time trajectory generation for differentially flat systems,” _International Journal of Robust and Nonlinear Control: IFAC-Affiliated Journal_ , vol. 8, no. 11, pp. 995–1020, 1998.
  * [3] D. Mellinger and V. Kumar, “Minimum snap trajectory generation and control for quadrotors,” in _2011 IEEE International Conference on Robotics and Automation_ , 2011, pp. 2520–2525.
  * [4] M. Faessler, A. Franchi, and D. Scaramuzza, “Differential flatness of quadrotor dynamics subject to rotor drag for accurate tracking of high-speed trajectories,” _IEEE Robotics and Automation Letters_ , vol. 3, no. 2, pp. 620–626, 2017.
  * [5] E. Tal and S. Karaman, “Accurate tracking of aggressive quadrotor trajectories using incremental nonlinear dynamic inversion and differential flatness,” _IEEE Transactions on Control Systems Technology_ , vol. 29, no. 3, pp. 1203–1218, 2020.
  * [6] P. Ru and K. Subbarao, “Nonlinear model predictive control for unmanned aerial vehicles,” _Aerospace_ , vol. 4, no. 2, 2017. [Online]. Available: https://www.mdpi.com/2226-4310/4/2/31
  * [7] D. Bicego, J. Mazzetto, R. Carli, M. Farina, and A. Franchi, “Nonlinear model predictive control with enhanced actuator model for multi-rotor aerial vehicles with generic designs,” _Journal of Intelligent & Robotic Systems_, vol. 100, no. 3-4, pp. 1213–1247, Sep. 2020.
  * [8] C. Liu, H. Lu, and W.-H. Chen, “An explicit mpc for quadrotor trajectory tracking,” in _2015 34th Chinese Control Conference (CCC)_ , 2015, pp. 4055–4060.
  * [9] G. Torrente, E. Kaufmann, P. Foehn, and D. Scaramuzza, “Data-driven MPC for quadrotors,” _IEEE Robotics and Automation Letters_ , vol. 6, no. 2, pp. 3769–3776, apr 2021.
  * [10] A. Romero, S. Sun, P. Foehn, and D. Scaramuzza, “Model predictive contouring control for near-time-optimal quadrotor flight,” _CoRR_ , vol. abs/2108.13205, 2021. [Online]. Available: https://arxiv.org/abs/2108.13205
  * [11] A. Romero, R. Penicka, and D. Scaramuzza, “Time-optimal online replanning for agile quadrotor flight,” _IEEE Robotics and Automation Letters_ , vol. 7, no. 3, pp. 7730–7737, Jul. 2022.
  * [12] S. Sun, A. Romero, P. Foehn, E. Kaufmann, and D. Scaramuzza, “A comparative study of nonlinear mpc and differential-flatness-based control for quadrotor agile flight,” _IEEE Transactions on Robotics_ , 2022.
  * [13] D. Hanover, P. Foehn, S. Sun, E. Kaufmann, and D. Scaramuzza, “Performance, precision, and payloads: Adaptive nonlinear mpc for quadrotors,” _IEEE Robotics and Automation Letters_ , vol. 7, no. 2, pp. 690–697, 2022.
  * [14] Y. Song, A. Romero, M. Müller, V. Koltun, and D. Scaramuzza, “Reaching the limit in autonomous racing: Optimal control versus reinforcement learning,” _Science Robotics_ , vol. 8, no. 82, p. eadg1462, 2023. [Online]. Available: https://www.science.org/doi/abs/10.1126/scirobotics.adg1462
  * [15] D. Hanover, A. Loquercio, L. Bauersfeld, A. Romero, R. Penicka, Y. Song, G. Cioffi, E. Kaufmann, and D. Scaramuzza, “Past, present, and future of autonomous drone racing: A survey,” _arXiv preprint arXiv:2301.01755_ , 2023.
  * [16] K. Nagami and M. Schwager, “Hjb-rl: Initializing reinforcement learning with optimal control policies applied to autonomous drone racing.” in _Robotics: science and systems_ , 2021.
  * [17] U. Ates, “Long-term planning with deep reinforcement learning on autonomous drones,” in _2020 Innovations in Intelligent Systems and Applications Conference (ASYU)_. IEEE, 2020, pp. 1–6.
  * [18] Y. Song, M. Steinweg, E. Kaufmann, and D. Scaramuzza, “Autonomous drone racing with deep reinforcement learning,” in _2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2021, pp. 1205–1212.
  * [19] R. Penicka, Y. Song, E. Kaufmann, and D. Scaramuzza, “Learning minimum-time flight in cluttered environments,” _IEEE Robotics and Automation Letters_ , vol. 7, no. 3, pp. 7209–7216, 2022.
  * [20] E. Kaufmann, L. Bauersfeld, and D. Scaramuzza, “A benchmark comparison of learned control policies for agile quadrotor flight,” in _2022 International Conference on Robotics and Automation (ICRA)_. IEEE, 2022, pp. 10 504–10 510.
  * [21] E. Kaufmann, L. Bauersfeld, A. Loquercio, M. Müller, V. Koltun, and D. Scaramuzza, “Champion-level drone racing using deep reinforcement learning,” _Nature_ , vol. 620, no. 7976, pp. 982–987, Aug 2023. [Online]. Available: https://doi.org/10.1038/s41586-023-06419-4
  * [22] J. Hwangbo, I. Sa, R. Siegwart, and M. Hutter, “Control of a quadrotor with reinforcement learning,” _IEEE Robotics and Automation Letters_ , vol. 2, no. 4, pp. 2096–2103, 2017.
  * [23] R. Ferede, G. C. H. E. de Croon, C. De Wagter, and D. Izzo, “End-to-end neural network based quadcopter control,” 2023. [Online]. Available: https://arxiv.org/abs/2304.13460
  * [24] S. Origer, C. De Wagter, R. Ferede, G. C. H. E. de Croon, and D. Izzo, “Guidance & control networks for time-optimal quadcopter flight,” 2023. [Online]. Available: https://arxiv.org/abs/2305.02705
  * [25] E. J. J. Smeur, Q. Chu, and G. C. H. E. de Croon, “Adaptive incremental nonlinear dynamic inversion for attitude control of micro air vehicles,” _Journal of Guidance, Control, and Dynamics_ , vol. 39, no. 3, pp. 450–461, mar 2016.
  * [26] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” _arXiv preprint arXiv:1707.06347_ , 2017.
  * [27] A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dormann, “Stable-baselines3: Reliable reinforcement learning implementations,” _Journal of Machine Learning Research_ , vol. 22, no. 268, pp. 1–8, 2021. [Online]. Available: http://jmlr.org/papers/v22/20-1364.html
  * [28] M. Bangura, M. Melega, R. Naldi, and R. Mahony, “Aerodynamics of rotor blades for quadrotors,” 2016.
  * [29] B. Gati, “Open source autopilot for academic research-the paparazzi system,” in _American Control Conference (ACC), 2013_. Washington, DC: IEEE, Jun. 2013, pp. 1478–1481.


# A Graph Is More Than Its Nodes: Towards
Structured Uncertainty-Aware Learning on Graphs

Hans Hao-Hsun Hsu $\ {}^{1}$ Yuesong Shen11footnotemark: 1 $\ {}^{1,2}$ Daniel
Cremers 1,2
${}^{1}\ $Technical University of Munich, Germany
${}^{2}\ $Munich Center for Machine Learning, Germany
{hans.hsu, yuesong.shen<EMAIL_ADDRESS>
Equal contribution

###### Abstract

Current graph neural networks (GNNs) that tackle node classification on graphs
tend to only focus on nodewise scores and are solely evaluated by nodewise
metrics. This limits uncertainty estimation on graphs since nodewise marginals
do not fully characterize the joint distribution given the graph structure. In
this work, we propose novel edgewise metrics, namely the edgewise expected
calibration error (ECE) and the agree/disagree ECEs, which provide criteria
for uncertainty estimation on graphs beyond the nodewise setting. Our
experiments demonstrate that the proposed edgewise metrics can complement the
nodewise results and yield additional insights. Moreover, we show that GNN
models which consider the structured prediction problem on graphs tend to have
better uncertainty estimations, which illustrates the benefit of going beyond
the nodewise setting.111Source code available at
https://github.com/hans66hsu/structured_uncertainty_metrics

##  1 Introduction

Learning from graph-structured data has gained increasing attention from the
deep learning community and numerous graph neural networks (GNNs) have been
introduced to cope with this new paradigm [7, 25, 17, 28]. Commonly used GNN
models like Graph Convolution Network (GCN) [7] and Graph Attention Network
(GAT) [25] produce nodewise scores from which good classification results can
be achieved.

On the other hand, learning on graphs poses particular challenges for
uncertainty estimation. Given the graph structure, the node predictions are no
longer independent samples. Instead, we have a structured prediction problem
[15] and should also account for their interdependencies. In other words,
high-quality uncertainty estimation for GNNs requires reliable estimation of
the joint output distribution. Hence considering only nodewise marginal output
distributions is insufficient.

From this structured prediction perspective, we see a considerable limitation
of existing works on uncertainty-aware learning for GNNs, namely that they
only employ nodewise metrics as evaluation criteria. Given the abundant
existing work on uncertainty-aware learning for standard multi-class
classification [20, 3, 19], many ideas have been adapted to the GNN case. This
includes approaches based on Bayesian formulations [29, 4], evidential
learning [30, 22], as well as post-hoc calibration methods [24, 27, 5].
However, for estimating the quality of predictive uncertainty, these
approaches directly use the nodewise metrics from the multi-class
classification literature, which are intended for i.i.d. test samples and
ignore the graph structure.

To tackle this issue, we investigate novel uncertainty estimation metrics that
consider the interdependencies in the graph structure. Specifically, we
propose the edgewise expected calibration error (ECE) which relies on the
graph structure and the edgewise marginal distributions. Furthermore, we
formulate agree and disagree ECEs to analyze the agreeing and disagreeing edge
marginals separately. Our experiments in Section 3 show that the proposed
edgewise metrics can complement the nodewise ECE results and offer additional
insights. To the best of our knowledge, structured uncertainty estimation
metrics have not yet been proposed for graph learning.

Equipped with the proposed edgewise metrics, we compare uncertainty estimation
of GNN models which simply produce nodewise scores, e.g., GCN and GAT, with
models which explicitly tackle the structured prediction problem, like Graph
Markov Neural Network (GMNN) [17] and Explicit Pairwise Factorized Graph
Neural Network (EPFGNN) [28]. While naive mean field [15] can approximate
predictions based on nodewise scores to the joint output distribution, we
observe in practice that approaches treating graph learning as structured
prediction problems tend to have better uncertainty estimation. This
demonstrates the benefit of going beyond the nodewise setting, especially for
the uncertainty estimation of GNNs.

$(p_{1};p_{2};p_{3})$ | Nodewise | Edgewise
---|---|---
Chain | Cycle
$(0;1;1)$ | 0 | 0 | 0
$(2/3;2/3;2/3)$ | 0 | $1/18$ | $1/9$
$(0.55;0.8;0.7)$ | 0.017 | 0 | 0.128

Figure 1: Examples of nodewise and edgewise ECEs for a chain and a cycle. The
nodewise ECEs stay the same for both graphs given the same node settings,
while the proposed edgewise ECEs can have different values depending on the
graph structure. Further details are available in Appendix A.

##  2 Uncertainty estimation on graphs beyond nodewise criteria

Given a graph $\mathcal{G}=(\mathcal{N},\mathcal{E})$ with $c$ output classes
for each node, the task of node classification aims at predicting the labels
$y_{U}=\\{y_{i}\\}_{i\in U}$ of the test nodes $U\subseteq\mathcal{N}$ given
the labels $y_{L}=\\{y_{i}\\}_{i\in L}$ of the training nodes
$L=\mathcal{N}\setminus U$ and the input features
$x=\\{x_{i}\\}_{i\in\mathcal{N}}$. Ideal uncertainty estimation is achieved
when the predicted joint distribution $\hat{p}(y_{U}|x,y_{L})$ matches the
true joint distribution $p(y_{U}|x,y_{L})$. However, this condition cannot be
verified in practice, for the following two reasons:

  1. 1.

It is impossible to estimate $p(y_{U}|x,y_{L})$ with a single set of ground-
truth labels $y^{*}_{U}$;

  2. 2.

The joint distributions are intractable given their exponential size w.r.t.
the node count.

Instead, one can consider weaker but more tractable alternatives based on
marginal distributions. Existing works [24, 27, 5] consider the confidence
calibration of nodewise marginal distributions and measure the quality of
uncertainty estimation using their expected calibration error (ECE) as defined
by Guo et al. [3]: Denote $(\hat{y}_{i})_{i\in
U}=\operatorname*{argmax}_{y_{U}}\hat{p}(y_{U}|x,y_{L})$ the class predictions
and $\hat{c}_{i}=\max_{l}\hat{p}(y_{i}=l|x,y_{L})$ the confidence of node $i$,
we partition the test nodes into $m$ bins $(B^{n}_{1},\dots,B^{n}_{m})$ with
$B^{n}_{k}=\\{j\in U|\frac{k-1}{m}<\hat{c}_{j}\leq\frac{k}{m}\\}$, the
nodewise ECE is defined as

$\displaystyle
ECE^{n}=\sum_{k=1}^{m}\frac{|B^{n}_{k}|}{|U|}\big{|}acc(B^{n}_{k})-conf(B^{n}_{k})\big{|},\text{
with }$ (1) $\displaystyle acc(B^{n}_{k})=\frac{1}{|B^{n}_{k}|}\sum_{i\in
B^{n}_{k}}\mathbf{1}(\hat{y}_{i}=y^{*}_{i})\text{ and
}conf(B^{n}_{k})=\frac{1}{|B^{n}_{k}|}\sum_{i\in B^{n}_{k}}\hat{c}_{i}.$ (2)

###  2.1 Edgewise expected calibration error

An important drawback of the nodewise ECE is that it neither reflects the
dependency w.r.t graph structure nor the interdependencies between the nodes.
To remedy this, we consider the confidence calibration of edgewise marginals,
and propose the following edgewise expected calibration error: Denote
$\mathcal{E}_{U}=\\{(i,j)\in\mathcal{E}|i,j\in U\\}$ the subset of edges
constrained to test nodes, and for edge $(i,j)\in\mathcal{E}_{U}$ denote
$\hat{c}_{i,j}=\max_{l,m}\hat{p}(y_{i}=l,y_{j}=m|x,y_{L})$ its confidence, we
construct a similar edge binning $(B^{e}_{1},\dots,B^{e}_{m})$ with
$B^{e}_{k}=\\{(i,j)\in\mathcal{E}_{U}|\frac{k-1}{m}<\hat{c}_{i,j}\leq\frac{k}{m}\\}$
and define the edgewise ECE as

$\displaystyle
ECE^{e}=\sum_{k=1}^{m}\frac{|B^{e}_{k}|}{|\mathcal{E}_{U}|}\big{|}acc(B^{e}_{k})-conf(B^{e}_{k})\big{|},\text{
with }$ (3) $\displaystyle acc(B^{e}_{k})=\frac{1}{|B^{e}_{k}|}\sum_{(i,j)\in
B^{e}_{k}}\mathbf{1}\big{(}\hat{y}_{i}=y^{*}_{i})\cdot\mathbf{1}\big{(}\hat{y}_{j}=y^{*}_{j}\big{)}\text{
and }conf(B^{e}_{k})=\frac{1}{|B^{e}_{k}|}\sum_{i\in B^{e}_{k}}\hat{c}_{i,j}.$
(4)

Figure 1 provides a concrete example that shows the dependency of edgewise ECE
w.r.t graph structures. Similar issues also exist for other nodewise metrics
like negative log-likelihood (NLL) and Brier score. Their edgewise extensions
are also defined in Appendix B.

Along with the quantitative results of edgewise ECE, we can also plot its
corresponding reliability diagram [3] for edgewise marginal predictions. They
provide additional insights by displaying the over/under-confident tendency of
the predictions at each confidence level.

It is possible to go beyond the edgewise setting and calibrate marginal
distributions of larger subgraphs. We focus on the edgewise metrics in this
work because it is a minimal extension that accounts for the graph structure
and has manageable complexity. Moreover, edge marginals have a special
significance for probabilistic graphical models: they determine the exact
joint distribution of a tree and correspond to the Bethe approximation in the
general case [26].

###  2.2 Agree and disagree ECEs

While the edgewise ECE estimates the general quality of the predicted edge
marginals, it can be interesting to consider some additional subsets.
Especially, since homophily plays an important role in graph learning [31], we
find it insightful to separate the homophilous and heterophilous cases and
consider the subsets of agreeing
$\mathcal{E}_{A}=\\{(i,j)\in\mathcal{E}_{U}|y^{*}_{i}=y^{*}_{j}\\}$ and
disagreeing edges $\mathcal{E}_{D}=\\{(i,j)\in\mathcal{E}_{U}|y^{*}_{i}\neq
y^{*}_{j}\\}$. This allows us to define the agree ECE ($ECE^{a}$) and the
disagree ECE ($ECE^{d}$) similar to Equations (3) and (4) but computed from
$\mathcal{E}_{A}$ and $\mathcal{E}_{D}$ respectively instead of
$\mathcal{E}_{U}$:

  * •

To compute the agree ECE, we construct the binning
$(B^{a}_{1},\dots,B^{a}_{m})$ for the agreeing edge set $\mathcal{E}_{A}$ with
$B^{a}_{k}=\\{(i,j)\in\mathcal{E}_{A}|\frac{k-1}{m}<\hat{c}_{i,j}\leq\frac{k}{m}\\}$.
The agree ECE is defined as

$\displaystyle
ECE^{a}=\sum_{k=1}^{m}\frac{|B^{a}_{k}|}{|\mathcal{E}_{A}|}\big{|}acc(B^{a}_{k})-conf(B^{a}_{k})\big{|},\text{
with }$ (5) $\displaystyle acc(B^{a}_{k})=\frac{1}{|B^{a}_{k}|}\sum_{(i,j)\in
B^{a}_{k}}\mathbf{1}\big{(}\hat{y}_{i}=y^{*}_{i})\cdot\mathbf{1}\big{(}\hat{y}_{j}=y^{*}_{j}\big{)}\text{
and }conf(B^{a}_{k})=\frac{1}{|B^{a}_{k}|}\sum_{i\in B^{a}_{k}}\hat{c}_{i,j}.$
(6)

  * •

Analogously, we construct the binning $(B^{d}_{1},\dots,B^{d}_{m})$ for the
disagreeing edge set $\mathcal{E}_{D}$ with
$B^{d}_{k}=\\{(i,j)\in\mathcal{E}_{D}|\frac{k-1}{m}<\hat{c}_{i,j}\leq\frac{k}{m}\\}$.
The disagree ECE is defined as

$\displaystyle
ECE^{d}=\sum_{k=1}^{m}\frac{|B^{d}_{k}|}{|\mathcal{E}_{D}|}\big{|}acc(B^{d}_{k})-conf(B^{d}_{k})\big{|},\text{
with }$ (7) $\displaystyle acc(B^{d}_{k})=\frac{1}{|B^{d}_{k}|}\sum_{(i,j)\in
B^{d}_{k}}\mathbf{1}\big{(}\hat{y}_{i}=y^{*}_{i})\cdot\mathbf{1}\big{(}\hat{y}_{j}=y^{*}_{j}\big{)}\text{
and }conf(B^{d}_{k})=\frac{1}{|B^{d}_{k}|}\sum_{i\in B^{d}_{k}}\hat{c}_{i,j}.$
(8)

##  3 Experiments

We conduct a series of experiments to empirically assess the proposed edgewise
metrics and analyze the uncertainty estimation of GNNs. We focus on solving
the node classification problem on commonly used benchmark datasets Cora [18],
Citeseer [18], and Pubmed [13]. In terms of GNN learning frameworks, we
compare commonly used GCN [7] and GAT [25] which directly produce nodewise
output scores against GMNN [17] and EPFGNN [28] which explicitly address the
structured prediction problem formulation. We analyze the quality of their
predictive uncertainty using the nodewise, edgewise, and agree/disagree ECE.

Calculation of the edgewise metrics requires estimating the edgewise marginal
distributions. GMNN [17] and EPFGNN [28] model the joint output distribution
from which the edge marginals can be inferred. For GCN [7] and GAT [25], we
assume that they represent the joint output distribution as the product of
nodewise marginal distributions, i.e., a naive mean field [15] assumption is
made. This means that their predicted edgewise marginals are simply products
of the neighboring node marginals. Further details about the experimental
setup are provided in Appendix C.

We summarize the results in Table 1. Additional results, including the
corresponding test accuracies and plots of reliability diagrams, are included
in Appendix D.

#### Nodewise and edgewise expected calibration errors

Firstly, we consider the nodewise and edgewise ECE results. Both metrics
provide indications for the general quality of uncertainty estimation. From
the results in Table 1, we observe two noticeable findings:

  * •

GNNs which address node classification as structured prediction (GMNN and
EPFGNN) tend to have better uncertainty estimation, in terms of both nodewise
and edgewise ECEs, compared to models like GCN and GAT which simply output
nodewise scores.

  * •

There is a significant correlation between nodewise and edgewise ECEs. This is
not surprising, as good uncertainty estimations should make both metrics
small. That said, the edgewise ECE has a different focus w.r.t. the nodewise
ECE, resulting the performance ranking of the Pubmed at odds with each other.

#### Agree and disagree ECEs

To further analyze the edgewise probabilistic predictions, we partition the
edge set into agreeing and disagreeing subsets. We evaluate the quality of
their predictive uncertainty separately using agree and disagree ECEs.

  * •

Structured prediction approaches achieve better agree ECEs and worse disagree
ECEs. Given the fact that Cora, Citeseer, and Pubmed are all assortative
graphs where neighbors tend to agree with each other, these methods seem to
focus on calibrating the agreeing edges to reach good overall uncertainty
estimation.

  * •

GAT consistently yields the best disagree ECEs. This might be an effect of its
attention mechanism which allows for distinct treatments of the agreeing and
disagreeing edges.

  * •

Reliability diagrams from Appendix D indicate that predictions on agreeing
edges tend to be under-confident. On the contrary, the predictions on
disagreeing edges tend to be over-confident given that GNNs fail to produce
correct high-confidence predictions.

Table 1: Uncertainty estimation results using various GNN models on Cora, Citeseer, and Pubmed in terms of nodewise, edgewise, agree and disagree ECEs. For all metrics, lower is better. Dataset | Model | Nodewise | Edgewise | Agree | Disagree
---|---|---|---|---|---
Cora | GCN | 12.47$\pm$4.37 | 16.64$\pm$5.53 | 24.18$\pm$5.89 | 17.87$\pm$3.23
GAT | 15.27$\pm$3.82 | 25.37$\pm$5.34 | 33.11$\pm$5.60 | 13.60$\pm$2.51
| GMNN | 4.05$\pm$1.30 | 4.07$\pm$1.86 | 9.96$\pm$2.59 | 30.78$\pm$3.37
| EPFGNN | 7.24$\pm$1.96 | 8.66$\pm$3.11 | 15.21$\pm$3.55 | 25.55$\pm$3.99
Citeseer | GCN | 11.34$\pm$8.28 | 16.96$\pm$10.46 | 34.45$\pm$11.69 | 29.80$\pm$7.88
GAT | 16.40$\pm$8.51 | 27.63$\pm$9.93 | 46.70$\pm$10.88 | 23.14$\pm$7.43
| GMNN | 7.97$\pm$2.04 | 9.21$\pm$2.62 | 11.16$\pm$4.22 | 54.95$\pm$6.02
| EPFGNN | 8.05$\pm$5.33 | 11.07$\pm$7.37 | 18.91$\pm$12.74 | 43.95$\pm$13.16
Pubmed | GCN | 7.16$\pm$1.40 | 3.34$\pm$1.53 | 12.64$\pm$1.68 | 38.28$\pm$1.78
GAT | 10.58$\pm$1.98 | 18.60$\pm$3.01 | 29.66$\pm$3.21 | 28.18$\pm$2.17
| GMNN | 4.24$\pm$0.65 | 10.78$\pm$0.77 | 2.62$\pm$0.53 | 62.34$\pm$1.36
| EPFGNN | 4.84$\pm$1.33 | 3.12$\pm$1.41 | 10.23$\pm$2.09 | 44.75$\pm$2.07

##  4 Discussions

In this work, we demonstrate the benefits of going beyond the nodewise setting
for better uncertainty-aware graph learning. We discuss the need for
structured evaluation metrics and propose edgewise metrics that offer
additional insights. And our experiments show that tackling the structured
prediction on graphs can help the uncertainty estimation.

Our experiments also highlight the well-known challenge of graph learning in
heterophilous scenarios. This is currently an active area of research [31] and
existing work is less established. We hope our proposed edgewise metrics can
help the related research, and in general bolster the development and
evaluation of new approaches for uncertainty-aware learning on graphs.

## Acknowledgments and Disclosure of Funding

This work was supported by the Munich Center for Machine Learning (MCML) and
by the ERC Advanced Grant SIMULACRON. The authors would like to thank the
anonymous reviewers for their constructive feedback.

## References

  * Besag [1975] J. Besag.  Statistical analysis of non-lattice data.  _The Statistician_ , 24(3):179–195, 1975.
  * Fey and Lenssen [2019] M. Fey and J. E. Lenssen.  Fast graph representation learning with PyTorch Geometric.  In _ICLR Workshop on Representation Learning on Graphs and Manifolds_ , 2019.
  * Guo et al. [2017] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger.  On calibration of modern neural networks.  In _ICML_ , 2017.
  * Hasanzadeh et al. [2020] A. Hasanzadeh, E. Hajiramezanali, S. Boluki, M. Zhou, N. Duffield, K. Narayanan, and X. Qian.  Bayesian graph neural networks with adaptive connection sampling.  In _ICML_ , 2020.
  * Hsu et al. [2022] H. H.-H. Hsu, Y. Shen, C. Tomani, and D. Cremers.  What makes graph neural networks miscalibrated?  In _NeurIPS_ , 2022.
  * Kingma and Ba [2014] D. P. Kingma and J. Ba.  Adam: A method for stochastic optimization.  In _ICLR_ , 2014.
  * Kipf and Welling [2017] T. N. Kipf and M. Welling.  Semi-supervised classification with graph convolutional networks.  In _ICLR_ , 2017.
  * Koller and Friedman [2009] D. Koller and N. Friedman.  _Probabilistic Graphical Models: Principles and Techniques - Adaptive Computation and Machine Learning_.  The MIT Press, 2009.
  * Kull et al. [2019] M. Kull, M. Perello Nieto, M. Kängsepp, T. Silva Filho, H. Song, and P. Flach.  Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with dirichlet calibration.  In _NeurIPS_ , 2019.
  * Kumar et al. [2019] A. Kumar, P. S. Liang, and T. Ma.  Verified uncertainty calibration.  In _NeurIPS_ , 2019.
  * Lafferty et al. [2001] J. D. Lafferty, A. McCallum, and F. C. N. Pereira.  Conditional random fields: Probabilistic models for segmenting and labeling sequence data.  In _ICML_ , 2001.
  * Murphy et al. [1999] K. P. Murphy, Y. Weiss, and M. I. Jordan.  Loopy belief propagation for approximate inference: An empirical study.  In _UAI_ , 1999.
  * Namata et al. [2012] G. Namata, B. London, L. Getoor, and B. Huang.  Query-driven active surveying for collective classification.  In _ICML Workshop_ , 2012.
  * Neal and Hinton [1998] R. M. Neal and G. E. Hinton.  _A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants_ , pages 355–368.  Springer, 1998.
  * Nowozin and Lampert [2011] S. Nowozin and C. H. Lampert.  Structured learning and prediction in computer vision.  _Foundations and Trends® in Computer Graphics and Vision_ , 6(3–4):185–365, 2011.
  * Paszke et al. [2019] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala.  Pytorch: An imperative style, high-performance deep learning library.  In _NeurIPS_ , 2019.
  * Qu et al. [2019] M. Qu, Y. Bengio, and J. Tang.  GMNN: Graph Markov neural networks.  In _ICML_ , 2019.
  * Sen et al. [2008] P. Sen, G. Namata, M. Bilgic, L. Getoor, B. Gallagher, and T. Eliassi-Rad.  Collective classification in network data.  _AI Mag._ , 29(3):93–106, 2008.
  * Sensoy et al. [2018] M. Sensoy, L. M. Kaplan, and M. Kandemir.  Evidential deep learning to quantify classification uncertainty.  In _NeurIPS_ , 2018.
  * Snoek et al. [2019] J. Snoek, Y. Ovadia, E. Fertig, B. Lakshminarayanan, S. Nowozin, D. Sculley, J. V. Dillon, J. Ren, and Z. Nado.  Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift.  In _NeurIPS_ , 2019.
  * Srivastava et al. [2014] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov.  Dropout: A simple way to prevent neural networks from overfitting.  _Journal of Machine Learning Research_ , 15(56):1929–1958, 2014.
  * Stadler et al. [2021] M. Stadler, B. Charpentier, S. Geisler, D. Zügner, and S. Günnemann.  Graph posterior network: Bayesian predictive uncertainty for node classification.  In _NeurIPS_ , 2021.
  * Sutton and McCallum [2009] C. Sutton and A. McCallum.  Piecewise training for structured prediction.  _Machine Learning_ , 77(2-3):165, 2009.
  * Teixeira et al. [2019] L. Teixeira, B. Jalaian, and B. Ribeiro.  Are graph neural networks miscalibrated?  In _ICML Workshop_ , 2019.
  * Veličković et al. [2018] P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio.  Graph attention networks.  In _ICLR_ , 2018.
  * Wainwright and Jordan [2008] M. J. Wainwright and M. I. Jordan.  Graphical models, exponential families, and variational inference.  _Foundations and Trends in Machine Learning_ , 1(1–2):1–305, 2008.
  * Wang et al. [2021a] X. Wang, H. Liu, C. Shi, and C. Yang.  Be confident! towards trustworthy graph neural networks via confidence calibration.  In _NeurIPS_ , 2021a.
  * Wang et al. [2021b] Y. Wang, Y. Shen, and D. Cremers.  Explicit pairwise factorized graph neural network for semi-supervised node classification.  In _UAI_ , 2021b.
  * Zhang et al. [2019] Y. Zhang, S. Pal, M. Coates, and D. Üstebay.  Bayesian graph convolutional neural networks for semi-supervised classification.  In _AAAI_ , 2019.
  * Zhao et al. [2020] X. Zhao, F. Chen, S. Hu, and J. Cho.  Uncertainty aware semi-supervised learning on graph data.  In _NeurIPS_ , 2020.
  * Zhu et al. [2020] J. Zhu, Y. Yan, L. Zhao, M. Heimann, L. Akoglu, and D. Koutra.  Beyond homophily in graph neural networks: Current limitations and effective designs.  In _NeurIPS_ , 2020.

##  Appendix A Nodewise and edgewise ECEs for concrete examples

To compare nodewise and edgewise ECE and show the dependency of edgewise ECE
w.r.t. the graph structure, we do a case study for two simple graphs shown in
Figure 1 Left: a chain and a cycle, both having three binary nodes
($\mathcal{N}=U=\\{1,2,3\\}$) with labels represented by their white or blue
color ($\\{w,b\\}$), and both having the same ground-truth labels
$(y^{*}_{1},y^{*}_{2},y^{*}_{3})=(w,b,b)$. We use the naive mean field
approximation and factorize the predicted joint distribution
$\hat{p}(y_{1},y_{2},y_{3})=\hat{p}(y_{1})\hat{p}(y_{2})\hat{p}(y_{3})$ into
product of nodewise predictions, and denote $p_{i}=\hat{p}(y_{i}=b)$ the
predicted probability of node $i$ having blue label. In the following, we
compute the nodewise and edgewise ECEs using a single bin on the two graphs
for the predictions listed in Figure 1 Right. As both graphs have the same
node settings, their nodewise ECEs are identical for any given prediction.

$(p_{1},p_{2},p_{3})=(0,1,1):$

This is the perfect forecast where the true labels are correctly predicted
with full certainty, and we reach zero nodewise and edgewise ECEs for both
graphs;

$(p_{1},p_{2},p_{3})=(\frac{2}{3},\frac{2}{3},\frac{2}{3}):$

This is a uniform prediction case that is nodewise calibrated. For both
graphs, the label predictions are
$(\hat{y}_{1},\hat{y}_{2},\hat{y}_{3})=(b,b,b)$ and the nodewise accuracy and
confidence terms are computed as

$acc(|U|)=\frac{1}{|U|}\sum_{i=1}^{3}\mathbf{1}(y^{*}_{i}=\hat{y}_{i})=\frac{2}{3};\quad
conf(|U|)=\frac{1}{|U|}\sum_{i=1}^{3}\hat{p}(\hat{y}_{i})=\frac{1}{3}\sum_{i=1}^{3}p_{i}=\frac{2}{3}.$
(9)

Thus the nodewise ECE for both graphs is

$ECE^{n}=\frac{|U|}{|U|}\big{|}acc(U)-conf(U)\big{|}=0.$ (10)

For the chain case, the edgewise predictions are wrong for edge $(1,2)$ and
right for $(2,3)$, and both edgewise predictions have confidence
$\frac{2}{3}\times\frac{2}{3}=\frac{4}{9}$, thus its edgewise ECE is

$ECE^{e}(Chain)=\frac{|\mathcal{E}_{U}|}{|\mathcal{E}_{U}|}\big{|}acc(\mathcal{E}_{U})-conf(\mathcal{E}_{U})\big{|}=|\frac{1}{2}-\frac{4}{9}|=\frac{1}{18}.$
(11)

For the cycle case, all three edges $(1,2),(1,3),(2,3)$ have confidence
$\frac{4}{9}$ and only the edge $(2,3)$ has correct label prediction, thus its
edgewise ECE is

$ECE^{e}(Cycle)=\frac{|\mathcal{E}_{U}|}{|\mathcal{E}_{U}|}\big{|}acc(\mathcal{E}_{U})-conf(\mathcal{E}_{U})\big{|}=|\frac{1}{3}-\frac{4}{9}|=\frac{1}{9}.$
(12)

$(p_{1},p_{2},p_{3})=(0.55,0.8,0.7):$

This is a non-uniform prediction that has zero edgewise ECE on the chain and
nonzero edgewise ECE on the cycle. For both graphs, the label predictions are
again $(\hat{y}_{1},\hat{y}_{2},\hat{y}_{3})=(b,b,b)$ and their common
nodewise ECE is computed as

$ECE^{n}=\Big{|}\frac{2}{3}-\frac{0.55+0.8+0.7}{3}\Big{|}\approx 0.017.$ (13)

For the chain case, one out of the two edges is correctly predicted, and its
edgewise ECE is

$ECE^{e}(Chain)=\Big{|}\frac{1}{2}-\frac{0.55\times 0.8+0.8\times
0.7}{2}\Big{|}=0,$ (14)

while for the cycle case, one out of its three edges is correctly predicted
and its edgewise ECE is computed as

$ECE^{e}(Cycle)=\Big{|}\frac{1}{3}-\frac{0.55\times 0.8+0.8\times
0.7+0.55\times 0.7}{3}\Big{|}\approx 0.128.$ (15)

The above examples highlight the difference between nodewise and edgewise ECE.
Particularly, the edgewise ECEs depend on the graph structure while nodewise
ECEs are structure agnostic and only depend on the node settings.

##  Appendix B Definitions of other edgewise metrics

Apart from ECE, other edgewise metrics can also be formulated. We provide the
formulations for some common metrics in this section.

Accuracy

While the commonly used nodewise accuracy is defined as

$ACC^{n}=\frac{1}{|U|}\sum_{i\in U}\mathbf{1}(\hat{y}_{i}=y^{*}_{i}),$ (16)

we can also define the edgewise accuracy

$ACC^{e}=\frac{1}{|\mathcal{E}_{U}|}\sum_{(i,j)\in\mathcal{E}_{U}}\mathbf{1}(\hat{y}_{i}=y^{*}_{i})\cdot\mathbf{1}(\hat{y}_{j}=y^{*}_{j})$
(17)

which computes the ratio of edges with correct label predictions.

Negative log-likelihood

The nodewise negative log-likelihood (NLL), which recovers the joint NLL
divided by $|U|$ under the naive mean field assumption, is defined as

$NLL^{n}=-\frac{1}{|U|}\sum_{i\in U}\log\hat{p}(y^{*}_{i}|x,y_{L}).$ (18)

Similarly, we can define the edgewise NLL as

$NLL^{e}=-\frac{1}{|\mathcal{E}_{U}|}\sum_{(i,j)\in\mathcal{E}_{U}}\log\hat{p}(y^{*}_{i},y^{*}_{j}|x,y_{L}).$
(19)

Brier score

The nodewise Brier score has the definition

$Brier^{n}=\frac{1}{|U|}\sum_{i\in
U}\Big{(}\big{(}1-\hat{p}(y^{*}_{i}|x,y_{L})\big{)}^{2}+\sum_{y_{i}\neq
y^{*}_{i}}\hat{p}(y_{i}|x,y_{L})^{2}\Big{)},$ (20)

and we can formulate the edgewise Brier score as

$Brier^{e}=\frac{1}{|\mathcal{E}_{U}|}\sum_{(i,j)\in\mathcal{E}_{U}}\Big{(}\big{(}1-\hat{p}(y^{*}_{i},y^{*}_{j}|x,y_{L})\big{)}^{2}+\sum_{(y_{i},y_{j})\neq(y^{*}_{i},y^{*}_{j})}\hat{p}(y_{i},y_{j}|x,y_{L})^{2}\Big{)}.$
(21)

Note that the agree/disagree variants of the above edgewise metrics can be
derived analogously by replacing $\mathcal{E}_{U}$ with $\mathcal{E}_{A}$ and
$\mathcal{E}_{D}$ respectively. Moreover, while we focus on the ECE metric
from Guo et al. [3], other variants of ECE exist [10] and their edgewise and
agree/disagree variants can be formulated in a similar manner.

##  Appendix C Details of experimental setup

###  C.1 Dataset statistics

Table 2: Dataset statistics. $K$ index indicates the percentage of test nodes kept in evaluated edges (see Eq. 23). Note that the edge homophily, $K(\mathcal{E}_{U})$, $K(\mathcal{E}_{A})$ and $K(\mathcal{E}_{D})$ depend on how the test nodes are randomly selected and may vary in different splittings. We report the mean and the standard deviation (in percentage) of these values from the splits used in our experiments. Dataset | Nodes | Edges | Features | Classes | Homophily | $K(\mathcal{E}_{U})$ | $K(\mathcal{E}_{A})$ | $K(\mathcal{E}_{D})$
---|---|---|---|---|---|---|---|---
Cora | 2,708 | 10,556 | 1,433 | 7 | 80.63$\pm$0.81 | 92.42$\pm$0.37 | 84.96$\pm$0.67 | 28.77$\pm$0.99
Citeseer | 3,327 | 9,104 | 3,703 | 6 | 72.63$\pm$0.83 | 85.91$\pm$0.78 | 68.21$\pm$0.82 | 33.76$\pm$0.33
Pubmed | 19,717 | 88,648 | 500 | 3 | 80.38$\pm$0.28 | 85.97$\pm$0.85 | 74.80$\pm$0.74 | 29.12$\pm$0.52

We evaluate our models on common citation networks: Cora [18], Citeseer [18],
and Pubmed [13]. A summary of the dataset statistics is shown in Table 2. The
edge homophily ratio from Zhu et al. [31] defines the fraction of edges in a
graph which connects nodes with the same class label $y^{*}$. As we measure
the homophily level of the test nodes $U$, the edge homophily ratio is
equivalent to the agreeing edges $\mathcal{E}_{A}$ over test edges
$\mathcal{E}_{U}$. Its complement is the disagreeing edges that we want to
predict.

$Homophily=\frac{|\\{(i,j):(i,j)\in\mathcal{E}_{U}\>|\>y^{*}_{i}=y^{*}_{j}\\}|}{|\mathcal{E}_{U}|}=\frac{|\mathcal{E}_{A}|}{|\mathcal{E}_{U}|}.$
(22)

We define $\mathcal{E}_{U}$ by constraining the connected nodes that need to
be both test nodes. If a test node does not have a neighbor which belongs to
the test nodes, it does not contribute to the edgewise metrics. We design an
index to indicate the percentage of test nodes kept in $\mathcal{E}_{U}$:

$K(\mathcal{E}_{U})=\frac{\text{Number of test nodes which are kept in
}\mathcal{E}_{U}}{|U|}.$ (23)

$K(\mathcal{E}_{U})$ close to $1$ implies fewer nodes are omitted. Similarly
the index can be applied to $\mathcal{E}_{A}$ and $\mathcal{E}_{D}$.

###  C.2 Details for model training

We randomly split the nodes to 15% as observed and 85% to unobserved set. We
follow a training framework as [9, 5] to divide the observed set into three-
fold where the two portions ($10\%$) are the training set and the other
portion ($5\%$) is the validation set. For all the datasets, we do 5 random
data splits, three-fold cross-validation per split, and together with 5 random
model initializations. This results in 75 runs for each experiment. The models
and training framework are implemented in PyTorch [16] and PyTorch Geometric
[2].

We choose the commonly used GCN [7] and GAT [25] as the baseline models which
do not consider the structured dependency between output nodes. For GCN, we
use 2 layers with 64 hidden units. For GAT, we use 2 layers with 8 attention
heads, and each consists of 8 hidden units. Dropout [21] is applied between
layers with dropout rate $0.5$. We train both models by minimizing NLL using
Adam optimizer [6]. The learning rate is set to $0.01$ with the weight decay
5e-4. We set the maximum epochs as $2000$ and stop the training if the model
does not improve on the validation set over the past $100$ epochs.

For the structured prediction models we choose GMNN [17] and EPFGNN [28]. Both
models are trained with the EM framework [14] to tackle the partially observed
scenario of node classification on graphs and maximize the evidence lower
bound instead of the intractable observed log-likelihood.

GMNN consists of one GCN as GNN backbone and another GCN to approximate the
Conditional Random Field (CRF) [11]. Both GCNs have the same hyperparameters
as the baseline GCN. We follow the training pipeline as the original paper. We
first pre-train the GNN backbone for $200$ epochs For M step we maximize the
pseudolikelihood function [1] and update the approximated CRF with $100$
epochs. For E step we update the GNN backbone for $100$ epochs to approximate
the true joint distribution. We observe the approximated CRF can achieve
better performance than the GNN backbone by drawing multiple samples. In
practice we draw $32$ samples and compute the mean as the prediction for
evaluation.

EPFGNN consists of one GCN backbone and a Markov Random Field (MRF) [8] which
models the label dependency explicitly using a shared compatibility matrix.
The GCN backbone produces as output the unary energies of the MRF. Same as
GMNN, we first pretrain the GCN backbone for $200$ epochs. For M step, we
train the GCN backbone and MRF jointly by maximizing piecewise training
objective [23]. Following Wang et al. [28], the training stops if the model
does not improve its accuracy on the validation set over $10$ epochs. For E
step, we update a proposal distribution using mean field to approximate the
true joint distribution. To get edge marginals from the MRF, we perform loopy
belief propagation [12] with $100$ iterations for evaluation.

##  Appendix D Additional experimental results

In this section, we collect additional experimental results. Table 3
summarizes the classification results in terms of nodewise, edgewise, agree,
and disagree accuracies. And Figures 2, 3 and 4 display the reliability
diagrams for GNN predictions on Cora, Citeseer and Pubmed.

Table 3: Classification results in terms of nodewise, edgewise, agree and disagree accuracies for various GNN models and datasets. For all metrics, higher is better. Dataset | Model | Nodewise | Edgewise | Agree | Disagree
---|---|---|---|---|---
Cora | GCN | 82.86$\pm$0.74 | 75.01$\pm$1.28 | 87.28$\pm$1.36 | 23.93$\pm$2.50
GAT | 83.71$\pm$0.73 | 75.80$\pm$1.00 | 88.40$\pm$1.20 | 23.33$\pm$1.89
| GMNN | 82.73$\pm$0.86 | 75.46$\pm$1.39 | 88.33$\pm$1.35 | 21.92$\pm$2.16
| EPFGNN | 76.48$\pm$1.16 | 65.07$\pm$1.85 | 76.32$\pm$1.74 | 18.25$\pm$2.51
Citeseer | GCN | 72.06$\pm$0.87 | 65.10$\pm$1.12 | 87.15$\pm$1.11 | 6.59$\pm$1.49
GAT | 72.04$\pm$0.76 | 64.96$\pm$1.17 | 87.59$\pm$0.87 | 4.92$\pm$1.03
| GMNN | 70.81$\pm$0.89 | 64.49$\pm$1.16 | 87.19$\pm$0.96 | 4.27$\pm$1.04
| EPFGNN | 69.13$\pm$1.15 | 61.20$\pm$1.68 | 82.45$\pm$1.95 | 4.81$\pm$1.25
Pubmed | GCN | 86.43$\pm$0.26 | 79.07$\pm$0.38 | 92.98$\pm$0.24 | 22.08$\pm$0.95
GAT | 85.12$\pm$0.36 | 78.38$\pm$0.47 | 93.01$\pm$0.30 | 18.47$\pm$1.10
| GMNN | 84.23$\pm$0.29 | 77.70$\pm$0.46 | 92.82$\pm$0.29 | 15.75$\pm$0.98
| EPFGNN | 82.51$\pm$0.58 | 74.35$\pm$0.67 | 88.40$\pm$0.65 | 16.81$\pm$1.22

Note the nodewise accuracies reported in Table 3 have significant
discrepancies compared to the ones reported in Wang et al. [28]. This is
caused by two main factors:

  * •

At test time we employ the loopy belief propagation inference which also
estimates edgewise marginals, while in Wang et al. [28] naive mean field is
used instead;

  * •

We have a different experimental setup and splitting (see Appendix C).

We report the corresponding results for EPFGNN using naive mean field in Table
4, which are more in line with the ones from Wang et al. [28]. We see that
switching from naive mean field to loopy belief propagation yields better ECEs
but worse accuracies. The decrease in accuracies might be explained by the
mismatch of the inference method since naive mean field is used doing
training.

Table 4: Results for EPFGNN when naive mean field is used for test time inference instead. Dataset | Metric | Nodewise | Edgewise | Agree | Disagree
---|---|---|---|---|---
Cora | Accuracy | $79.74\pm 1.09$ | $70.02\pm 2.00$ | $82.21\pm 1.88$ | $19.30\pm 2.69$
ECE | $9.16\pm 2.41$ | $12.53\pm 3.31$ | $20.03\pm 4.10$ | $24.07\pm 4.40$
Citeseer | Accuracy | $71.77\pm 0.96$ | $64.85\pm 1.30$ | $87.56\pm 1.17$ | $4.58\pm 1.20$
ECE | $8.42\pm 6.01$ | $12.32\pm 8.24$ | $22.53\pm 13.97$ | $44.61\pm 13.51$
Pubmed | Accuracy | $84.74\pm 0.55$ | $76.76\pm 0.55$ | $91.27\pm 0.52$ | $17.31\pm 1.20$
ECE | $6.13\pm 1.40$ | $3.66\pm 1.56$ | $11.26\pm 2.07$ | $44.97\pm 2.23$
|  |  |
---|---|---|---
|  |  |
|  |  |
|  |  |
Figure 2: Reliability diagrams of GCN, GAT, GMNN, and EPFGNN trained on Cora. The light gray lines are the results of individual experiments. The solid blue line and the shaded blue area are the mean and the standard deviation over 75 runs. |  |  |
---|---|---|---
|  |  |
|  |  |
|  |  |
Figure 3: Reliability diagrams of GCN, GAT, GMNN, and EPFGNN trained on Citeseer. The light gray lines are the results of individual experiments. The solid blue line and the shaded blue area are the mean and the standard deviation over 75 runs. |  |  |
---|---|---|---
|  |  |
|  |  |
|  |  |
Figure 4: Reliability diagrams of GCN, GAT, GMNN, and EPFGNN trained on
Pubmed. The light gray lines are the results of individual experiments. The
solid blue line and the shaded blue area are the mean and the standard
deviation over 75 runs.