# An Embedded Feature Selection Framework for Control

Jiawen Wei  Central South UniversityChangshaHunanChina<EMAIL_ADDRESS>,
Fangyuan Wang  Zhejiang Sci-Tech UniversityHangzhouZhejiangChina
<EMAIL_ADDRESS>,  Wanxin Zeng  Central South
UniversityChangshaHunanChina<EMAIL_ADDRESS>,  Wenwei Lin  Central South
UniversityChangshaHunanChina<EMAIL_ADDRESS>and  Ning Gui  Central South
UniversityChangshaHunanChina<EMAIL_ADDRESS>
###### Abstract.

Reducing sensor requirements while keeping optimal control performance is
crucial to many industrial control applications to achieve robust, low-cost,
and computation-efficient controllers. However, existing feature selection
solutions for the typical machine learning domain can hardly be applied in the
domain of control with changing dynamics. In this paper, a novel framework,
namely the Dual-world embedded Attentive Feature Selection (D-AFS), can
efficiently select the most relevant sensors for the system under dynamic
control. Rather than the one world used in most Deep Reinforcement Learning
(DRL) algorithms, D-AFS has both the real world and its virtual peer with
twisted features. By analyzing the DRL’s response in two worlds, D-AFS can
quantitatively identify respective features’ importance towards control. A
well-known active flow control problem, cylinder drag reduction, is used for
evaluation. Results show that D-AFS successfully finds an optimized five-
probes layout with 18.7% drag reduction than the state-of-the-art solution
with 151 probes and 49.2% reduction than five-probes layout by human experts.
We also apply this solution to four OpenAI classical control cases. In all
cases, D-AFS achieves the same or better sensor configurations than originally
provided solutions. Results highlight, we argued, a new way to achieve
efficient and optimal sensor designs for experimental or industrial systems.
Our source codes are made publicly available at
https://github.com/G-AILab/DAFSFluid.

Deep Reinforcement Learning, Feature Selection, Optimal Sensor Placement

††journalyear: 2022††ccs: Computing methodologies Feature selection††ccs:
Computing methodologies Reinforcement learning††ccs: Computing methodologies
Knowledge representation and reasoning

##  1\. Introduction

For experimental and industrial systems, reducing sensor requirements while
maintaining optimal control performance is crucial for human experts to
understand target systems and potentially transpose these techniques to
industrial cases. It is one of human beings’ most critical intellectual
activities, i.e., extracting knowledge from unknown environments to recognize
and exploit the world(Sutton and Barto, 2018). To better understand the target
system, scientists/engineers use intuition and domain knowledge to identify
critical information based on continuous observations of system
performance(Khurana et al., 2018). But for a complex system with massive
sensory inputs, this handcrafted solution through trial-and-error is normally
time-consuming, costly, and heavily reliant on human expertise. Thus, it is
vital to develop an effective and systematic solution to identify relevant
sensors without or with minimal human intervention.

Nowadays, DRL has shown great success in complex system control
problems(Lillicrap et al., 2016) by combining reinforcement learning and deep
learning(Mnih et al., 2015). The introduction of Deep Neural Networks (DNN)
can extract important features from a huge set of raw inputs. Despite its
success, the opaque nature of DNN hides the critical information about which
sensors are most relevant for control(Samek et al., 2017; Du et al., 2018).
Existing solutions for sensor selection under DRL have been limited to the
recursive sensor set selection and evaluation with certain heuristic
algorithms(Paris et al., 2021). As their complexity increases rapidly with the
number of candidate sensors, these methods are inapplicable for systems with
massive possible inputs.

Figure 1. Instantaneous velocity flow field with 151 probes. Black dots
represent sensor locations and colors represent velocity(fields).

This requirement is similar to the Feature Selection (FS) problem in Machine
Learning(ML) (Li et al., 2017), also known as variable selection or attribute
selection. This process aims to select a subset of relevant features with
respect to the modeling labels. However, due to the distinctiveness of control
process, choosing a suitable variable as a label is very troublesome. For
example, Fig. 1 shows a classic flow control problem to reduce the drag and
lift of a cylinder by adjusting flow injection/suction with two
jets(represented by two red dots) and 151 probes. Firstly, the immediate
reward corresponding to the time series state set is often meaningless as a
label. The instance injection/suction might not change the immediate reward
significantly. Secondly, the goal for control typically uses the discounted
cumulative return as a loss function that often has long and uncertain latency
and has little relevance to the single-step state. Thus, neither immediate
reward nor cumulative return is appropriate as the mandatory training label in
most feature selection methods. In later experiments, we also experimentally
prove that existing feature selection methods are not suitable in systems with
control.

For effective control, a DRL controller has already learned the relevance of
different inputs during its exploration and exploitation. Can we make the
hidden knowledge on the relevance of respective sensors explicit and provide
qualitative evaluation during DRL training? To address this challenge, we
propose a new solution, the Dual-world based Attentive embedded Feature
Selection(D-AFS), a general mechanism that enables the DRL controller to
interact with the real world and its virtual peer with twisted features. Our
method combines general DRL framework and feature selection module, which can
identify the contribution of inputs simultaneously with strategy learning
process. In summary, our contributions are:

  * •

A new direction for embedded feature selection in control: We combine the
exploration and exploitation capabilities of DRL with an attention-based
feature evaluation module, that qualitatively evaluates the impact of
different features by translating feature weight calculation to feature
selection probability problem. The method can identify the feature importance
of unknown systems for control during DRL’s strategy learning process.

  * •

A dual-world architecture design: We propose a dual-world architecture that
includes real world and virtual world, of which the virtual world is mapped
from its real peer by the feature evaluation module. Dual-world enables D-AFS
complete feature importance ranking without jeopardizing DRL’s strategy
exploration.

  * •

Excellent results on active flow control: We apply D-AFS with the Proximal
Policy Optimization (PPO) algorithm(Schulman et al., 2017) in the case
mentioned above. D-AFS/PPO successfully reduces a five-sensor layout from 151
possible probes, with a drag coefficient of 2.938, very close to the
theoretical limit of 2.93 without vortex shedding, and achieves 18.7% drag
reduction than the state-of-the-art solutions using 151 probes.

We also successfully adapt Deep Q-Network(DQN)(Mnih et al., 2015) and Deep
Deterministic Policy Gradient (DDPG)(Lillicrap et al., 2016) into D-AFS, apply
them on four OpenAI Gym(Brockman et al., 2016) classical control cases. In all
cases, D-AFS selects the same or better sensor subsets than the original
expert-based solutions (Gym provided). Results clearly show that our solution
can effectively identify system key sensors, improve the interpretability of
the target system and reduce the reliance on human expertise.

##  2\. Related Work

This section discusses the related work from two aspects: 1) The feature
selection methods in machine learning; 2) The abbreviated research for sensor
selection in active flow control.

###  2.1. Feature Selection in ML

Feature selection is a technology that contains automatically discovering,
evaluating, and selecting relevant features(Bengio et al., 2013). Generally,
it can be divided into three categories(Li et al., 2017): 1) Filter methods
separate feature selection from the learning process and only rely on the
measures of the general characteristics of the training data to evaluate the
feature weights. Different feature selection algorithms exploit various types
of criteria to define the relevance of features, e.g., Fisher score(Gu et al.,
2011) and ReliefF(Robnik-Šikonja and Kononenko, 2003). 2) Wrapper methods(Tang
et al., 2014) predict accuracy of a predefined learning algorithm to evaluate
the quality of recursive selected features. They normally suffer high
computation costs and low scalability. 3) Embedded methods depend on the
interactions with the learning algorithm and evaluate feature sets according
to the interactions. The most recent approaches are in this class e.g.,
XGBoost(2016), LightGBM(2017) and Random Forest(RF)(2001). Recently, DNN-based
solutions are proposed,e.g., the attention-based, AFS(2019) or casualty-based
(Yu et al., 2021). However, trajectories generated by the DRL controller do
not follow the independent and identically distributed assumption, making it
impossible to directly transfer and use the feature selection methods in the
ML domain. (Hu et al., 2019) creates virtual sensors to do fault detection by
using existing sensors and applying domain knowledge to the data. However,
this approach does not actually reduce the number of sensors and cannot be
applied to control tasks. As pointed out in the introduction, for model-free
systems, it is challenging to provide appropriate labels for each system
state, as demanded in most ML feature selection approaches. Thus, it is
difficult for existing ML feature selection methods to be applied in the DRL
control environment.

###  2.2. Optimal Sensor Placement in Flow Control

The issue of optimal sensor placement has been investigated with certain
domain-specific knowledge. Most existing solutions,e.g., (Bright et al., 2013;
Cohen et al., 2006) are based on compressed sensing theory that selects a
limited number of sensors that can effectively perform flow reconstruction.
These approaches do not depend on the learning algorithm and can be seen as a
special filter feature selection algorithm. This stream of methods also has
very high computation complexity, as we need to reconstruct the whole flow.
Furthermore, as stressed by Stephan & Simon (2018), control does not
systematically require faithful flow reconstruction. The partial knowledge of
relevant ‘hidden’ variables may be sufficient.

Despite its success in many domains, only in the recent work of Rabault et al.
(2019), DRL has been used to control the flow past a 2D cylinder. They use 151
probes to learn control strategy by PPO in the 2D Kármán Vortex Street
simulation. Tang et al. (2020) further improve their work through extending
the Reynolds numbers range and sensor observations. These researches have not
yet involved the optimization of sensor selection. Paris et al. (2021)
recently proposes a sensor selection algorithm based on a well-trained PPO
controller. However, this approach adopts the so-called wrapper method by
evaluating different combinations of features. Their work has very high
computation complexity, and their discussion is limited to a system with 15
probes.

To the best of our knowledge, there have been no systematic solutions yet to
obtain interpretable environmental representations through the learning and
exploration processes of DRL.

##  3\. Definitions and Notations

This section gives a general definition of the problem studied in this paper
and briefly demonstrates the related notations.

Raw Features. Given a controlled object, the basic observations are a set of
states periodically collected by sensors deployed in the system. We use
$\bm{\mathcal{S}}_{r}=\\{s^{k}|k=1,2,...,m\\}$ to describe the set of raw
states, where $k$ represents the number of sensors.

Feature Selection for Control. The goal of feature selection for control under
the DRL framework is stated as follows. Given a set of features
$\bm{\mathcal{S}}_{r}$ from the real world, find a subset of features,
$\bm{\mathcal{S}}^{*}\subseteq\bm{\mathcal{S}}_{r}$, to maximize the expected
cumulative return $\mathcal{R}_{t}$ for a given DRL algorithm. As shown in Eq.
(1):

(1)
$\bm{\mathcal{S}}^{*}=\arg\max_{\bm{\mathcal{S}}_{r}}\mathcal{R}_{t}=\arg\max_{\bm{\mathcal{S}}_{r}}\sum_{t}^{\infty}\gamma^{t}r_{t}$

where $\gamma$ is a discount rate.

Discussion. For a target system, if a well-designed controller is capable of
achieving effective control, the raw features must contain all the necessary
information for control. However, it has been wildly agreed that reducing
sensor requirements while keeping optimal control performance is crucial to
many industrial control applications to achieve robust, low cost, and
computation efficient controllers(Khurana et al., 2018). Previous works on the
industrial control with DRL(Paris et al., 2021) demonstrate that when the
selected feature representation matches the form required by the physical
model, the DRL controller can usually even achieve a better control effect.
This characteristic is of vital importance in understanding the underlying
physical model of the target systems.

Figure 2. The overall architecture of D-AFS with the “real” and “virtual”
world environment, the evaluation module and the DRL module. The virtual world
is mapped from the real world. DRL learns in two mixed worlds, explores the
correct control strategy in the real world, and provides evaluation
information for feature selection in the virtual world.

##  4\. Proposed Method

In this section, the overall structure of D-AFS and the mechanism of embedded
feature selection are illustrated and analyzed.

###  4.1. Design Principles

The basic design principle for two different worlds is rather straightforward.
Firstly, if a feature is irrelevant or weakly related to the system, it has
little effect on a “good” control strategy. Conversely, a feature strongly
related to the physical model of the system or the control goal should have a
pronounced impact on the control strategy. Generally, the one that is more
benign for the controller might have more impacts.

However, the dual-world design brought a series of problems to the existing
DRL solutions with only one world. Three main issues need to be tackled: 1)
How to effectively combine the real and virtual world without affecting the
normal operation of DRL? 2) How to dynamically adjust the combination of
features in the virtual world space without jeopardizing the DRL learning
process? 3) How to modify existing DRL algorithms to allows online feature
evaluation?

Thus, we introduce the dual-world architecture, which contains three main
modules, the environment module, the evaluation module, and the DRL module, as
shown in Fig. 2. The environment and the DRL module are inherited directly
from the original DRL framework, but detailed designs are slightly different
to support the two worlds. The evaluation module calculates the importance of
each feature through the attention mechanism and then helps generate the
virtual world. The DRL module, evaluation module, and the unique dual-world
system are loosely coupled in the architecture we designed. We minimize the
modification of the general structure of traditional DRL algorithms and
maintain its integrity to the greatest extent, which significantly promotes
the reuse of the existing state-of-the-art DRL algorithms. Therefore, the DRL
module can be determined according to the specific problem.

###  4.2. Dual-world

We first need to clarify the different roles between the real world and the
virtual world of the dual-world space. On the one hand, we need the real world
to provide system information to explore the correct control strategy. On the
other hand, the virtual world searches for the optimal features and
representations by continuously tuning the mapping from the real world to the
virtual world. The real world features are the system’s original features
$\bm{\mathcal{S}}_{r}$, which is used to interact with the policy of the DRL
module. And the virtual world is the real world mapping through the evaluation
module, which can be used to evaluate the validity of the selected features.
The mapping function is generated with the attention-based evaluation module
illustrated in the following section. In D-AFS, the environment is the
combination of the real and virtual world
$\bm{\mathcal{S}}_{r}\cup\bm{\mathcal{S}}_{v}$.

###  4.3. Attention-based Evaluation Module

The attention-based evaluation (AE for short) module aims to evaluate the
importance of different features by assigning different weights (denoted as
$\bm{P}$). The structure of AE is shown in the green area in Fig. 2.

The AE module is a revised attention mechanism extended from our previous
work(Gui et al., 2019). This module consists of two key components, a dense
network $\bm{E}$ with $N_{E}$ neuronal units and several Attention Networks
(AN for short) for calculating the importance weights of each input.

The dense network $\bm{E}=\tanh({\bm{\mathcal{S}}_{r}}^{\rm
T}\bm{W}_{E}+\bm{b}_{E})$, $\bm{W}_{E}\in\mathbb{R}^{m\times N_{E}}$,
$\bm{b}_{E}\in\mathbb{R}^{N_{E}}$, is used to abstract the internal
relationship between the inputs $\bm{\mathcal{S}}_{r}$. $\bm{W}_{E}$ and
$\bm{b}_{E}$ are parameters, nonlinear function $\tanh(\cdot)$ keeps positive
and negative values. To some extend, the dense network can reduce the
redundancy of all sensors, of which the units $N_{E}$ can be adjusted
according to respective inputs.

Once we have extracted the relationship of features by $\bm{E}$, we configure
an attention network for each feature to generate weight separately. We do not
adopt the typical soft attention mechanism to generate a weighted arithmetic
mean of all features. This operation might result in small weights for most
features and large weights for a tiny number of features and suffer the loss
of details on the whole feature sets. These weights can determine whether one
feature should be selected or not. More detailed, it generates two values,
$\bm{X}=\bm{E}\bm{W}_{x}+\bm{b}_{x}$ and $\bm{Y}=\bm{E}\bm{W}_{y}+\bm{b}_{y}$,
to represent the selected/unselected probability of a feature, respectively.
In general, the higher the weight of the feature being selected ($\bm{X}$),
the more important the feature is for control. Then we use the softmax
function to magnify the distance between selected and unselected.

(2)
$p^{k}=\frac{\exp(x^{k})}{\exp{\left(x^{k}\right)}+\exp(y^{k})},x^{k}\in\bm{X},y^{k}\in\bm{Y}$

Finally, these attention nets will generate a matrix of attention weights
$\bm{P}=\left\\{p^{k}|k=1,2,...,m\right\\}$, which has the same dimension as
features. With Eq. 3, we generate a feature representation of the virtual
world with $\bm{\mathcal{S}}_{r}$ multiplied by attention weights.

(3) $\bm{\mathcal{S}}_{v}=\bm{\mathcal{S}}_{r}\odot\bm{P}$

The AE module generates a dynamic mapping function that maps representations
from the real world $\bm{\mathcal{S}}_{r}$ to the virtual world
$\bm{\mathcal{S}}_{v}$, which will be used for DRL to update the strategy.
Additionally, with the attention weights matrix $\bm{P}$ generated by AE, we
can evaluate respective features when DRL has explored an efficient strategy.

###  4.4. PPO in Dual-world

As far as we know, value functions are used in one way or another among the
current model-free DRL algorithms(Achiam, 2018). In our algorithm, in addition
to the role of the original DRL algorithm, the value function also needs to
guide the AE module to select valuable features during the training process.
This subsection shows how to integrate the D-AFS with the popular Proximal
Policy Optimization(PPO) algorithm, denoted as D-AFS/PPO. PPO is a popular
policy-based online algorithm with clipped probability ratios that forms a
lower bound of the performance of the policy. In order to allow the agent to
interact with the environment correctly, we use the real world as the input of
the PPO actor network, which is the same as the original PPO algorithm. Thus,
the actor network with parameters $\theta$ can be updated by:

(4) $\displaystyle\theta^{*}=\arg\max_{\theta}L^{CLIP}(\theta)$

where,

$L^{CLIP}(\theta)=\mathbb{E}_{t}[\min(r_{t}(\theta),{\rm
clip}(r_{t}(\theta),1-\epsilon,1+\epsilon))A^{\pi_{\theta_{t}}}(s_{r},a)],$

$r_{t}(\theta)$ is the probability ratio of the old and the new policy, the
subscript $t$ is the number of iteration.

The difference is that in our algorithm, we use the virtual world as the input
of the critic network, allowing the critic network to evaluate the goodness of
the selected features of the evaluation module while scoring the performance
of the actor network. That is:

(5)
$\phi^{*},\psi^{*}=\arg\min_{\phi,\psi}\mathbb{E}_{t}(V_{\phi,\psi}(s_{v})-\mathcal{R}_{t})^{2}$

$\phi$ and $\psi$ are parameters of critic network and attention module,
respectively. $V_{\phi,\psi}(s_{v})$ is value function and $\mathcal{R}_{t}$
is accumulative return.

The detailed structure of this D-AFS/PPO is shown in Algorithm 1. The bold
pseudo-codes are the revised parts to support the dual-world DRL learning.

Algorithm 1 D-AFS/PPO, Actor-Critic based

Initial parameters of actor network $\theta_{0}$, critic network $\phi_{0}$
and attention network $\psi_{0}$.

for $t=1,2,...$ do

Run policy $\pi(\theta_{t})$ to get trajectories
$\mathcal{D}_{t}(s_{r},a,r,s^{\prime}_{r})$

Compute accumulative return $\mathcal{R}_{t}$

Compute attention weights $\bm{p}$ based on Eq. (2)

Compute $\bm{s_{v}\leftarrow s_{r}\odot p}$

Compute advantage estimates $A_{t}$ based on current function
$\bm{V_{\phi_{t},\psi_{t}}(s_{v})}$

Update the actor network by maximizing the PPO-Clip based on Eq. (4) with Adam
$\theta_{old}\leftarrow\theta$

Update the critic network by regression on MSE based on Eq. (5) with Adam
$\\{\phi_{old}\leftarrow\phi,\bm{\psi_{old}\leftarrow\psi}\\}$

end for

##  5\. Experiments and Analysis

In this section, the experimental system and settings are firstly introduced.
Then the results for both FS solutions and DRL solutions are provided and
analyzed. We also briefly provide further experiments in OpenAI Gym to
demonstrate D-AFS’s adaptability.

###  5.1. System Description and Settings

Drag reduction and active flow control are critical technologies in the fluid
industry. Here, we briefly introduce the target system and experiment
settings.

Target System:  This environment is from a well-known benchmark (Schäfer et
al., 1996) which consists of a cylinder of diameter $D=0.1$ immersed in a
constant flow of a box of total length $L=2.2$ and height $H=0.41$. The
cylinder is slightly downward from the center of the Y-Axis by 0.005. The mean
velocity magnitude is $\bar{U}=1$. And the Reynolds number based on the mean
velocity magnitude and cylinder diameter ($Re=\bar{U}D/v$, with $v$ the
kinematic viscosity) is set to $Re=100$. When the target system is in an
unstable state, the flow velocity and pressure of the wake behind the cylinder
show periodic changes due to the transmission of the vortex street. As the
learning of the control strategy becomes effective, the velocity and the
pressure of the wake gradually decrease.

Two synthetic jets are located at the top/bottom extremities of the cylinder,
which we can use DRL to control, as indicated in Fig. 3 with solid red dots.
There are 151 velocity probes placed around the cylinder and the near weak,
which provide the network with a relatively detailed flow description, as
indicated in Fig. 3 with black dots. Circle and Square are Top5 and Top11
probes selected by human experts. Our experimental environment is inherited
from (Rabault et al., 2019). Detailed settings and calculations can refer to
the paper.111https://github.com/jerabaul29/Cylinder2DFlowControlDRL.

D-AFS/PPO Settings:  Due to the limited continuous action space of the 2D flow
control system, we use Beta policy(Chou, 2017), which has finite support and
no probability density that falls outside the boundary, to sample actions in
the training process and use deterministic policy in the testing process. It
is also the default policy of Tensorforce(Kuhnle et al., 2017) for limited
continuous action space, which is used by (Rabault and Kuhnle, 2019). The
critic network consists of 4 layers: an input layer, two hidden layers with
512 cells, and an output layer. In the AE module, the dense network $N_{E}$
has 20 cells.

Figure 3. The 2D active flow control simulation environment without control.
The location of the jets is indicated by the red dots.

The original paper(Rabault et al., 2019) takes 24 hours to train a stable
controller on a single CPU. And about 99.7% of the computation time is spent
on the environment itself using FEniCS(Alnæs et al., 2015) rather than the DRL
algorithm (Rabault and Kuhnle, 2019). Thus, we designed the asynchronous
training architecture to accelerate Algorithm 1 by syncing parameters and
averaging gradients on each worker. The architecture reduced the training time
from 40 hours to 3.6 hours by using 20 threads.

Evaluation Metrics and Reward Function: We use drag coefficient $C_{D}$, lift
coefficient $C_{L}$, recirculation area size, and velocity fluctuations and
pressure fluctuations to evaluate the performance of a trained controller. The
drag coefficient and lift coefficient are the normalized total instantaneous
drag (opposite the constant flow direction) and lift (perpendicular to the
constant flow direction) on the cylinder. And the recirculation area is the
region in the downstream neighborhood of the cylinder where the horizontal
component of the velocity is negative. To compare fairly, we use the standard
PPO with the absolute same configuration for learning as the arbiter to
evaluate different probe configurations.

In short, an excellent active flow control strategy means multiple
evaluations, small $C_{D}$ and $C_{L}$, low $C_{L}$ fluctuations, pressure
drop and velocity magnitude, and a large recirculation area. In this way,
efficient active control can reduce vortex shedding caused by the bluff
cylinder and gain a smoother flow. Here, the reward function is defined as
follows:

(6) $r_{t}=-C_{D}-0.2|C_{L}|,$

which is to minimize the drag and lift to stabilize the vortex alley and the
same as other papers to facilitate comparison.

In the 2D simulation of the target system, our method is applied to learn the
active flow control strategy and obtain attention weights of all 151 probes
sensors simultaneously. We use the attention weight from the AE module when
the D-AFS/PPO controller achieves stable rewards. Then, the probes with Top K
weights are used for control with the same reward function and the original
PPO controller. The $C_{D}$ and $C_{L}$ achieved with different methods are
used for evaluations.

###  5.2. Comparison with Other FS Methods

Methods | Uncontrolled | F&S | XGB | LGB | RF | FIR | CCM | SAN-global | L2X | SAN-local | D-AFS
---|---|---|---|---|---|---|---|---|---|---|---
$C_{D}$ | 3.205 | 3.118 | 3.062 | 3.044 | 3.082 | 3.038 | 3.116 | 3.012 | 3.068 | 3.076 | 2.938
$C_{L}$ | 0.676 | 0.784 | 0.452 | 0.291 | 1.036 | 0.338 | 0.655 | 1.099 | 0.441 | 0.479 | 0.126

Figure 4. Comparison of the Top 5 probes selected with different feature
selection solutions. Table shows the average values of the drag coefficient
$C_{D}$ and lift coefficient $C_{L}$ when the control strategy converges with
only the selected five probes with the PPO controller

In this section, we test the performance of state-of-the-art baseline feature
selection algorithms in their feature identification capabilities in control.
We choose eight strong feature selection baselines, including classic methods,
gradient boosting methods and recent population-wise methods. We also provide
one instance-wise feature selection method. Implementation of these algorithms
are either by the scikit-learn package(Pedregosa et al., 2011) or from codes
with their original papers. F&S (Gu et al., 2011), short for Fisher Score, a
similarity-based method, which selects each feature independently according to
their scores under the fisher criterion. XGB(Chen and Guestrin, 2016), short
for XGBoost, an optimized distributed gradient boosting library that can
calculate the feature importance across many trees. LGB(Ke et al., 2017),
short for LightGBM, gradient boosting framework that uses tree based learning
algorithms. RF(Ho, 1995), short for Random Forest, operates by constructing a
multitude of decision trees at training time. FIR(Wojtas and Chen, 2020), a
novel dual-net architecture consisting of operator and selector for discovery
of an optimal feature subset. CCM(Chen et al., 2017), employs kernel-based
measures of independence to find a subset of covariates that is maximally
predictive of the response. SAN(Škrlj et al., 2020), explores the use of
attention-based neural networks mechanism for estimating feature importance.
We use both the local/global variants: SAN-local and SAN-global. L2X(Chen et
al., 2018), an instance-wise feature selection method that extract a subset of
features that are most informative for each given example.

Time series sensory data and immediate rewards during the training and
exploration of the PPO controller are collected to provide training data set
for supervised feature selection. The data set includes 50 epochs series, with
80 control steps in each epoch.

Fig. 4 shows the Top 5 probes selected by different feature selection methods
and corresponding control effects. The layouts selected by these supervised
feature selection methods are not suitable for control, whether from numerical
or physical perspectives. Most of them select probes whose characteristics do
not change significantly during the periodic vortex shedding. Some of them,
e.g. F&S, SAN, and L2X,select the probes located on the left side of the
cylinder, generally seen as an ineffective area. The value of these probes
generally provides little help for the drag reduction task. In order to verify
their effectiveness in control, we use PPO to control systems with only Top 5
selected probe, respectively. The drag and lift coefficient in the table
(bottom part of Fig. 4) do clarify that these probes layouts cannot achieve
effective active flow control behind the cylinder. In contrast, D-AFS achieve
the best control effect for both $C_{D}$ and $C_{L}$ with the same number of
probes.

###  5.3. Comparison with Existing DRL solutions

We verify the control performance obtained using the probe configurations
selected by different DRL solutions with 151 probes and observe their results.
The performance of the active flow control strategy under different
configurations is shown in the upper part of Fig. 5. The numerical results for
drag coefficient $C_{D}$, lift coefficient $C_{L}$ and recirculation area size
are shown in the bottom part of Fig. 5. In addition to TopK probe
configurations, we also show the performance of Rabault et al.’s result with
selected five-probe, marked as Expert(5) and a randomly selected five-probe
configuration denoted as Random(5). In Random(5), the best performance in five
rounds is reported. Here, other reference use similar environment but
different settings, e.g. Tang (Tang et al., 2020) use four actuators instead
of two, thus the result can not be directly compared. Reference (Paris et al.,
2021) requires heuristic enumeration of sensor layouts with only 15 probes and
has not yet been open-sourced.

Methods | Probes | $C_{D}$ | $C_{L}$ |  Rec Area
---|---|---|---|---
Uncontrolled | 151 | 3.205 | 0.676 | 0.012
Rabault et al.(2019) | 151 | 2.980 | 0.210 | 0.022
Rabault et al.(2019) | 5 | 3.026 | 0.259 | 0.017
Random Selection | 5 | 2.987 | 0.424 | 0.019
D-AFS(5) | 5 | 2.938 | 0.126 | 0.030
D-AFS(10) | 10 | 2.964 | 0.129 | 0.025
D-AFS(15) | 15 | 2.938 | 0.112 | 0.029
Ideal Case | - | 2.93 | - | -

Figure 5. Time-evolutionary value of the drag coefficient $C_{D}$ with
different solution. Table shows the mean values of $C_{D}$, $C_{L}$ and Rec
Area when system becomes steady.

The top part of the figure shows that the drag coefficient $C_{D}$ varies
periodically as the vortex shedding when the system is uncontrolled. The mean
value of $C_{D}$ is around 3.205, and the fluctuation amplitude is around
0.034. In comparison, taking the Top5/10/15 probes selected by D-AFS as
observations, the PPO controller can achieve better active flow control. The
$C_{D}$ of D-AFS with the five-sensor is very close to, more exactly 99.73%,
the ideal 2.93(denoted by dashed gray line) where no vortex shedding
exists(Rabault et al., 2019). Compared with the five-sensor configuration by
human experts(Rabault et al., 2019), D-AFS achieves a 260% increase in the
recirculation area and 49.2% drag reduction. Even compared to the state-of-
the-art solution with 151 probes, our five-sensor configuration can achieve a
further 18.7% drag reduction. The drag coefficient $C_{D}$ of the best random
selection is better than Expert(5), but the lift coefficient $C_{L}$ is worse
than the latter. That means the control strategy using randomly selected
probes is unstable.

Interestingly, adding more sensors to the five-sensor layout selected by D-AFS
does not bring further drag reduction, which makes this five-sensor
configuration the optimized trade-off between DRL performance and sensor
selection complexity. It shows that too many possibly redundant inputs may
cause interference and hindrance to the controller with limited modeling
capabilities, albeit they might contain more information. This finding matches
precisely the finding stated in (Paris et al., 2021).

###  5.4. Further Analysis

In this section, we provide further analysis on the probe weight distribution,
control effect analysis, impacts of different probe configurations and the
complexity analysis.

Feature weight distribution: Fig. 6 illustrates the attention weight of 151
probes with different gradient colors. The darker the color, the bigger the
weight and the higher the importance of the probe to the controller. We also
marked the Top5, Top10, and Top15 probes with the cross, circle, and diamond.
The Top5 probes selected by D-AFS are scattered behind the cylinder, and they
provide an excellent ‘coverage’ of the instantaneous vortex variation, as
shown in Fig. 8. These probes are normally critical for the control task,
which can explain the selection preference of D-AFS to a certain extent. That
is, probes that can deliver critical information conducive to learning will
get a higher weight.

Figure 6. The attention weight distribution of 151 probes. The Top5/10/15
probes in weight ranking have been marked.

Control effect analysis: We take a look at macroscopic flow characteristics
and how the active control affects them. Fig. 7 shows the snapshots of the
pressure (subgraph a.) and velocity distribution(subgraph b.) under three
configurations when the system enters steady states. The three configurations
are Uncontrolled, Expert(5), and D-AFS(5).

Figure 7. Comparison of active control performance(velocity and pressure) of
uncontrolled and two five-probe configurations.

The pressure distribution shows that the vortex in the wake caused by the
cylinder is manifested as a significant pressure drop without active control.
Similarly, the velocity magnitude fluctuates a lot, visible from the top sub-
figure of the velocity distribution. The control effect from Expert(5) is
shown in the middle of the two subgraphs, and we can see those probes are not
in the key positions for vortex shedding. In comparison, the selected Top5
probes have good “coverage” of the instantaneous recirculation bubble. Thus,
the control effect of D-AFS is much better. PPO with D-AFS(5) attenuates
velocity fluctuations caused by the vortexes, achieves that vortexes are
globally very weak and least active close to the upper and lower walls. More
strikingly, the extent of the recirculation area is dramatically increased.

Equivalent probe layouts: As introduced previously, optimizing sensors’ number
and location are widely demanded in various industrial domains to find cost-
effective sensor solutions in redundant probes. For D-AFS, it is possible to
select probes with totally different layouts because of the random exploration
and exploitation nature. Fig. 8 shows two identified Top5 probes with two
separated experiments, illustrated with black circles and yellow squares. They
provide almost identical control performance. The drag coefficient $C_{D}$ is
around 2.940, the lift coefficient $C_{L}$ is around 0.130.

Figure 8. Identified two equivalent probe layouts with similar DRL
performance, black circles and yellow squares indicate the probes selected in
two separated experiments.

The left and the right part of Fig. 8 are two different snapshots, which show
the forward propagation of the vortex to better illustrate selected probes’
positions on the route of vortex. We can find that, although the two
configurations are highly different(only one overlap), each configuration
covers the key positions which contain richer information of periodic vortex
shedding or located around the edge of the recirculation area. For instance,
observations detected by probes located at (0.15, -0.1), (0.25, -0.15), (0.3,
0.05), (0.4,0.15) and (0.45, -0.05) enable the DRL controller obtain rich
periodic information. This also means that the control task can be converted
from drag reduction to vibration reduction for these observations.

Complexity analysis: Table 1 shows the time complexity in generating feature
weights for systems with different numbers of probes, ranging from 5 to 151.
We also show the time when pure PPO is used for training. The settings for
epochs and steps are the same as the experiment settings mentioned earlier.
From this table, we can see that D-AFS generates feature weights in the
process of DRL exploration and exploitation, and the overall time consumed is
almost the same as that of pure PPO. The computation and resource complexity
is relatively low.

In comparison, if the DRL is only used as an arbiter with a certain heuristic-
guided probe configuration, e.g., (Paris et al., 2021), the computation
complexity is about $O(C_{n}^{r})$ with n are possible probes, and $r$ are
probes to select. It will increase rather fast when n and r increase. The
author pointed out that each training costs around 40 CPU hours when the
system only has 12 probes. Although they provide no discussion on computation
complexity, we deduce that this approach can hardly be applied to the system
with 151 probes.

Table 1. The computational complexity of D-AFS and pure PPO without feature selection.(Time unit: hour) Probes | 5 | 10 | 50 | 100 | 151
---|---|---|---|---|---
PPO | 3.17 | 3.24 | 3.30 | 3.39 | 3.46
D-AFS/PPO | 3.19 | 3.28 | 3.37 | 3.49 | 3.60

###  5.5. Further Experiments in OpenAI Gym

To better explore the generalization of D-AFS, we also tune two popular DRL
algorithms, DQN and DDPG into our framework. We choose four classical control
cases provided by OpenAI Gym(Brockman et al., 2016), including Pendulum,
MountainCar, CartPole, and Acrobot, to observe control performance with
feature subsets selected by D-AFS.

Figure 9. Comparison of controller performance with the feature subsets
selected by different methods.

The lower part of Fig. 9 is a table, in which values are rewards of four
systems with feature subsets selected by different methods. The larger the
rewards, the better the performance of DRL’s controller. To verify that the
control performance is influenced only by features, experimental settings are
all the same except for the feature subsets. Results in the table show that
D-AFS can select relevant features effectively and achieve the same or even
better control performance than the original expert-based subsets. In
Pendulum, MountainCar and Acrobot, D-AFS achieves the same rewards as Gym.
Actually, it deduces the same sensor sets used with Gym in all three cases. In
Cartpole, D-AFS achieves the maximum return under the limit of 1000 maximal
steps in an episode, which is about 54.70% higher than the 646.4 achieved by
the features provided by Gym with a different set of features. Due to the page
limits, the upper part of Fig. 9 only shows the feature weights calculated by
different methods for CartPole. Since the feature weight values generated by
different methods are not uniform, we normalize all weights for better
comparison. Regarding the features provided by Gym based on human expertise,
we set their weights to 1, while other features that are not included are set
to 0. More comprehensive weights for other cases are provided in Appendix A.

##  6\. Conclusion

Unlike humans who are good at learning unknown environments through
interaction, the most existing DRL algorithms cannot gain explicit
environmental knowledge from the learning and exploration process. In this
paper, a dual-world feature selection method D-AFS is proposed to
quantitatively evaluate the impacts of different inputs for control during the
DRL training process. A set of experiments is performed in one classic flow
control. Results show that D-AFS can effectively identify the system-
mechanism-related inputs. With the deduced five-probe layout, we can achieve
the best drag reduction than all other strong baselines. Results highlight, we
argue, a new way to achieve efficient and optimal sensor designs for
experimental or industrial systems. In this paper, our work limits to the
active flow problem with limited complexity due to the high computation
complexity with FEniCS for solving partial differential equations. We are
working on implementing D-AFS on a deep neural network based simulator for
more fast and complex tasks. The other possible direction is to find a
solution that can automatically decide the optimal set of sensors selected for
control.

##  7\. Acknowledgement

This work was supported by the National Science Foundation of China (61772473,
62073345 and 62011530148). F. Wang was supported by the Equipment Pre-research
Key Laboratory Funds No. 61425010102.

## References

  * (1)
  * Achiam (2018) Joshua Achiam. 2018\.  Spinning Up in Deep Reinforcement Learning.  (2018).
  * Alnæs et al. (2015) Martin Alnæs, Jan Blechta, Johan Hake, August Johansson, Benjamin Kehlet, Anders Logg, Chris Richardson, Johannes Ring, Marie E Rognes, and Garth N Wells. 2015\.  The FEniCS project version 1.5.  _Archive of Numerical Software_ 3, 100 (2015).
  * Bengio et al. (2013) Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013\.  Representation learning: A review and new perspectives.  _IEEE transactions on pattern analysis and machine intelligence_ 35, 8 (2013), 1798–1828.
  * Breiman (2001) Leo Breiman. 2001\.  Random forests.  _Machine learning_ 45, 1 (2001), 5–32.
  * Bright et al. (2013) Ido Bright, Guang Lin, and J Nathan Kutz. 2013.  Compressive sensing based machine learning strategy for characterizing the flow around a cylinder with limited pressure measurements.  _Physics of Fluids_ 25, 12 (2013), 127102.
  * Brockman et al. (2016) Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. 2016.  Openai gym.  _arXiv preprint arXiv:1606.01540_ (2016).
  * Chen et al. (2018) Jianbo Chen, Le Song, Martin Wainwright, and Michael Jordan. 2018\.  Learning to explain: An information-theoretic perspective on model interpretation. In _International Conference on Machine Learning_. PMLR, 883–892.
  * Chen et al. (2017) Jianbo Chen, Mitchell Stern, Martin J Wainwright, and Michael I Jordan. 2017.  Kernel feature selection via conditional covariance minimization.  _Advances in Neural Information Processing Systems_ 30 (2017).
  * Chen and Guestrin (2016) Tianqi Chen and Carlos Guestrin. 2016.  Xgboost: A scalable tree boosting system. In _Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining_. 785–794.
  * Chou (2017) Po-Wei Chou. 2017\.  _The Beta Policy for Continuous Control Reinforcement Learning_.  Master’s thesis. Carnegie Mellon University, Pittsburgh PA.
  * Cohen et al. (2006) Kelly Cohen, Stefan Siegel, and Thomas McLaughlin. 2006\.  A heuristic approach to effective sensor placement for modeling of a cylinder wake.  _Computers & fluids_ 35, 1 (2006), 103–120.
  * Du et al. (2018) Mengnan Du, Ninghao Liu, Qingquan Song, and Xia Hu. 2018\.  Towards explanation of dnn-based prediction with guided feature inversion. In _Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 1358–1367.
  * Gu et al. (2011) Quanquan Gu, Zhenhui Li, and Jiawei Han. 2011.  Generalized Fisher score for feature selection. In _Proceedings of the Twenty-Seventh Conference on Uncertainty in Artificial Intelligence_. 266–273.
  * Gui et al. (2019) Ning Gui, Danni Ge, and Ziyin Hu. 2019.  AFS: An attention-based mechanism for supervised feature selection. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 33. 3705–3713.
  * Ho (1995) Tin Kam Ho. 1995\.  Random decision forests. In _Proceedings of 3rd international conference on document analysis and recognition_ , Vol. 1. IEEE, 278–282.
  * Hu et al. (2019) RL Hu, J Granderson, DM Auslander, and A Agogino. 2019\.  Design of machine learning models with domain experts for automated sensor selection for energy fault detection.  _Applied energy_ 235 (2019), 117–128.
  * Ke et al. (2017) Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017\.  Lightgbm: A highly efficient gradient boosting decision tree.  _Advances in neural information processing systems_ 30 (2017), 3146–3154.
  * Khurana et al. (2018) Udayan Khurana, Horst Samulowitz, and Deepak Turaga. 2018\.  Feature engineering for predictive modeling using reinforcement learning. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 32.
  * Kuhnle et al. (2017) Alexander Kuhnle, Michael Schaarschmidt, and Kai Fricke. 2017\.  Tensorforce: a TensorFlow library for applied reinforcement learning.  Web page.  https://github.com/tensorforce/tensorforce
  * Li et al. (2017) Jundong Li, Kewei Cheng, Suhang Wang, Fred Morstatter, Robert P Trevino, Jiliang Tang, and Huan Liu. 2017.  Feature selection: A data perspective.  _ACM Computing Surveys (CSUR)_ 50, 6 (2017), 1–45.
  * Lillicrap et al. (2016) Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2016.  Continuous control with deep reinforcement learning.. In _ICLR (Poster)_.
  * Mnih et al. (2015) Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. 2015\.  Human-level control through deep reinforcement learning.  _nature_ 518, 7540 (2015), 529–533.
  * Oehler and Illingworth (2018) Stephan F Oehler and Simon J Illingworth. 2018.  Sensor and actuator placement trade-offs for a linear model of spatially developing flows.  _Journal of Fluid Mechanics_ 854 (2018), 34–55.
  * Paris et al. (2021) Romain Paris, Samir Beneddine, and Julien Dandois. 2021\.  Robust flow control and optimal sensor placement using deep reinforcement learning.  _Journal of Fluid Mechanics_ 913 (2021).
  * Pedregosa et al. (2011) Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011\.  Scikit-learn: Machine learning in Python.  _the Journal of machine Learning research_ 12 (2011), 2825–2830.
  * Rabault et al. (2019) Jean Rabault, Miroslav Kuchta, Atle Jensen, Ulysse Réglade, and Nicolas Cerardi. 2019.  Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control.  _Journal of Fluid Mechanics_ 865 (2019), 281–302.
  * Rabault and Kuhnle (2019) Jean Rabault and Alexander Kuhnle. 2019.  Accelerating deep reinforcement learning strategies of flow control through a multi-environment approach.  _Physics of Fluids_ 31, 9 (2019), 094105.
  * Robnik-Šikonja and Kononenko (2003) Marko Robnik-Šikonja and Igor Kononenko. 2003.  Theoretical and empirical analysis of ReliefF and RReliefF.  _Machine learning_ 53, 1 (2003), 23–69.
  * Samek et al. (2017) Wojciech Samek, Thomas Wiegand, and Klaus-Robert Müller. 2017\.  Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models.  _arXiv preprint arXiv:1708.08296_ (2017).
  * Schäfer et al. (1996) Michael Schäfer, Stefan Turek, Franz Durst, Egon Krause, and Rolf Rannacher. 1996\.  Benchmark computations of laminar flow around a cylinder.  In _Flow simulation with high-performance computers II_. Springer, 547–566.
  * Schulman et al. (2017) John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017.  Proximal policy optimization algorithms.  _arXiv preprint arXiv:1707.06347_ (2017).
  * Škrlj et al. (2020) Blaž Škrlj, Sašo Džeroski, Nada Lavrač, and Matej Petkovič. 2020\.  Feature importance estimation with self-attention networks.  _arXiv preprint arXiv:2002.04464_ (2020).
  * Sutton and Barto (2018) Richard S Sutton and Andrew G Barto. 2018.  _Reinforcement learning: An introduction_.  MIT press.
  * Tang et al. (2020) Hongwei Tang, Jean Rabault, Alexander Kuhnle, Yan Wang, and Tongguang Wang. 2020.  Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning.  _Physics of Fluids_ 32, 5 (2020), 053605.
  * Tang et al. (2014) Jiliang Tang, Salem Alelyani, and Huan Liu. 2014\.  Feature selection for classification: A review.  _Data classification: Algorithms and applications_ (2014), 37.
  * Wojtas and Chen (2020) Maksymilian Wojtas and Ke Chen. 2020.  Feature importance ranking for deep learning.  _Advances in Neural Information Processing Systems_ 33 (2020), 5105–5114.
  * Yu et al. (2021) Kui Yu, Lin Liu, and Jiuyong Li. 2021.  A Unified View of Causal and Non-causal Feature Selection.  _ACM Transactions on Knowledge Discovery from Data (TKDD)_ 15, 4 (2021), 1–46.

## Appendix A: The experimental results of OpenAI Gym

Since our method has good scalability and can minimize the changes to the
existing DRL algorithm, we apply the architecture to two mainstream
algorithms, DQN and DDPG. We apply D-AFS/DQN and D-AFS/DDPG to four classical
control environments provided by OpenAI Gym, the Pendulum, MountainCar,
CartPole, and Acrobot. The four target systems are shown in Fig. A1, and the
weights generated by D-AFS are shown in Table A1. The boldface in the table
indicates the features selected by D-AFS. “P_Ran” and “Ran” represent partial
noise and complete noise, respectively. The results show that our method can
effectively select the most relevant sensors and reduce human expertise
reliance.

Figure A1. The physical meaning of the major features in the four systems
Table A1. Results of D-AFS in four classical control systems provided OpenAI
Gym

Algorithm | Systems | All inputs and weights |
---|---|---|---
D-AFS/DQN | Pendulum | $\theta$ | $\omega$ | $\sin\theta$ | $\cos\theta$ | P_Ran | Ran |  |  |  |
0.516 | 0.968 | 0.960 | 0.879 | 1.27e-4 | 2.32e-5 |  |  |  |
MountainCar | $x$ | $v$ | P_Ran | Ran |  |  |  |  |  |
0.862 | 0.828 | 7.99e-5 | 1.75e-4 |  |  |  |  |  |
CartPole | $x$ | $v$ | $\theta$ | $\omega$ | $\sin\theta$ | $\cos\theta$ | P_Ran | Ran |  |
0.618 | 0.601 | 0.047 | 0.952 | 0.997 | 0.955 | 2.28e-4 | 9.57e-4 |  |
Acrobot | $\theta_{1}$ | $\theta_{2}$ | $\omega_{1}$ | $\omega_{2}$ | $\sin\theta_{1}$ | $\cos\theta_{1}$ | $\sin\theta_{2}$ | $\cos\theta_{2}$ | P_Ran | Ran
0.907 | 0.407 | 0.921 | 0.470 | 0.356 | 0.965 | 0.809 | 0.848 | 1.04e-3 | 1.95e-3
0.408 | 0.427 | 0.934 | 0.537 | 0.674 | 0.938 | 0.658 | 0.924 | 1.75e-3 | 2.79e-3
D-AFS/DDPG | Pendulum | $\theta$ | $\omega$ | $\sin\theta$ | $\cos\theta$ | P_Ran | Ran |  |  |  |
0.243 | 0.561 | 0.956 | 0.739 | 1.20e-3 | 2.96e-3 |  |  |  |
MountainCar | $x$ | $v$ | P_Ran | Ran |  |  |  |  |  |
0.178 | 0.894 | 3.48e-4 | 3.27e-4 |  |  |  |  |  |
CartPole | $x$ | $v$ | $\theta$ | $\omega$ | $\sin\theta$ | $\cos\theta$ | P_Ran | Ran |  |
0.822 | 0.386 | 0.955 | 0.356 | 0.868 | 0.607 | 0.009 | 0.002 |  |
Acrobot | $\theta_{1}$ | $\theta_{2}$ | $\omega_{1}$ | $\omega_{2}$ | $\sin\theta_{1}$ | $\cos\theta_{1}$ | $\sin\theta_{2}$ | $\cos\theta_{2}$ | P_Ran | Ran
0.651 | 0.459 | 0.273 | 0.302 | 0.609 | 0.754 | 0.785 | 0.821 | 2.67e-3 | 2.16e-3


Step 2: The continuous time extension follows by an inspection of the proof of Beiglböck, Schachermayer, and Veliyev <cit.> in which the mesh of the grid in Step 1 tends to zero.
The arguments are easier than in the original proof since we know from the estimate in Step 1 that the sequence of terminal values of the discrete time martingales is $L^2(P)$-bounded.
Thus, it is sufficient to apply the Komlós theorem for Hilbert spaces, and one obtains $L^2$-convergence to the terminal value of the continuous time martingale part
(cf. <cit.>).
This implies $L^2$-convergence of the terminal values of the drift parts, and the estimates from Step 1 also hold for the continuous time Doob-Meyer decomposition. This yields (a).

Step 3: Let  $\xi:=\inf\{t>0 : \wt{Y}_t > \eps\}$.
Consider the (pre-)stopped process $Y_t:= \wt{Y}_t 1_{(t<\xi)}
+ \wt{Y}_{\xi-} 1_{(t\ge \xi)}$. Since $\wt{Y}$ is a supermartingale and
$\Delta \wt{Y}_{\xi}\ge 0$, $Y$ has to be a supermartingale as well.
The process $Y$ is bounded from above by $\eps$ and we have that $P(Y_T<-\eps)\le P(\sup_{t\in[0,T]}|\wt{Y}_t| > \eps)\le \eps$.
This allows us to apply part (a) with $\eps_1=\eps_2=\eps_3=\eps$ to $Y$, and we obtain for its Doob-Meyer decomposition that
$E(M^2_T)\le 3\eps + 8\eps^2$ and $E(A^2_T)\le 2\eps+2\eps^2$. By the corollary to Protter <cit.>
we have $\sup_{H\in\bP, ||H||_\infty\le 1}\sqrt{E(\sup_{t\in[0,T]}|H\mal Y_t|^2)} \le 3\sqrt{E(M^2_T)}+3\sqrt{E(A^2_T)}$, which implies that
$d_{\bbs}(Y,0)\le \sqrt{27\eps + 72\eps^2} + \sqrt{18\eps + 18\eps^2}$.
Since the paths of $\wt{Y}$ and $Y$
coincide at least with probability $1-\eps$, we have that $d_{\bbs}(\wt{Y},Y)\le\eps$, and (b) follows with the triangle inequality of $d_{\bbs}$.

[1]
Abdelghani, M., Melnikov, A.:
Optional Processes – Theory and Applications, CRC Press, CRC Press, Boca Raton (2022)

[2] Bayraktar, E., Yu, X.
On the market viability under proportional transaction costs.
Mathematical Finance, 28, 800–838 (2018)

[3] Bayraktar, E., Zhang, Y.
Fundamental Theorem of Asset Pricing Under Transaction Costs and Model Uncertainty.
Mathematics of Operations Research, 41, 1039–1054 (2016)

[4] Beiglböck, M., Schachermayer, W., Veliyev, B.: A short proof of the Doob-Meyer theorem.  Stochastic Processes and their Applications, 122, 1204–1209 (2012)

[5] Chou C.S., Meyer P.A., Stricker C.:  Sur les intégrales stochastiques de processus prévisibles non bornés. In: Azéma J., Yor M. (eds.) Séminaire de Probabilités XIV. Lect. Notes Math, vol. 784, pp. 128–139. Springer, Berlin (1980)

[6]
Cohen, S.N.,Elliott, R.J.: Stochastic Calculus and Applications, 2nd edn. Birkhäuser, New York (2015)

[7] Cuchiero, C., Teichmann, J.: A convergence result for the Emery topology
and a variant of the proof of the fundamental theorem of asset pricing. Finance and Stochastics, 19, 743–761 (2015)

[8]  Czichowsky, C., Schachermayer W.: Duality theory for portfolio optimization under proportional transaction costs. Annals of Applied Probability, 26,  1888–1941 (2016)

[9] Delbaen, F., Schachermayer, W.: A general version of the fundamental theorem of asset pricing. Mathematische Annalen, 300, 463–520 (1994)

[10] Delbaen, F., Schachermayer, W.:
The fundamental theorem of asset pricing for unbounded stochastic processes.
Mathematische Annalen, 312, 215–250 (1998)

[11] Delbaen, F., Schachermayer, W.: The Mathematics of Arbitrage. Springer, Berlin (2006)

[12] Eberlein, E., Kallsen, J.: Mathematical Finance. Springer, Berlin (2019)

[13] Émery, M.: Une topologie sur l'espace des semimartingales.  In: Azéma J., Yor M. (eds.) Séminaire de Probabilités XIII. Lect. Notes Math, vol. 721, pp. 260–280. Springer, Berlin (1979)
[14] Fraňková, D.: Regulated functions. Math. Bohem. 116, 20–59 (1991)

[15] Grigoriev, P.: On low dimensional case in the fundamental asset pricing theorem with transaction
costs. Statistics and Decisions, 23, 33–48 (2005)

[16]
Guasoni, P.: No arbitrage under transaction costs, with fractional Brownian motion and beyond.
Mathematical Finance, 16, 569–582 (2006)

[17]  Guasoni, P., Rásonyi, M., Schachermayer, W.:
Consistent price systems and face-lifting pricing under transaction costs.
Annals of Applied Probability, 18, 491–520 (2008)

[18]  Guasoni, P., Rásonyi, M., Schachermayer, W.:
The fundamental theorem of asset pricing for continuous processes under small transaction costs.
Annals of Finance, 6, 157–191 (2010)

[19] Guasoni, P., Lépinette, E., Rásonyi, M.: The fundamental theorem of asset pricing under transaction costs. Finance and Stochastics, 16, 741–777 (2012)

[20] He, S., Wang, J., Yan, J.:
Semimartingale theory and stochastic calculus. CRC Press, Boca Raton (1992)

[21] Jacod, J., Shiryaev, A.N.:	Limit Theorems of Stochastic Processes, 2nd edn. Springer, Berlin (2003)

[22] Kabanov, Y.: On the FTAP of Kreps–Delbaen–Schachermayer. In: Kabanov, Y., Rozovski, B.,
Shiryaev, A. (eds.) Statistics and Control of Stochastic Processes, Moscow, 1995/1996, pp. 191–203.
World Sci. Publ., River Edge (1997)

[23] Kabanov, Y., Rásonyi, M., Stricker, C.: No-arbitrage criteria for financial markets with efficient friction. Finance and Stochastics, 6, 371–382 (2002)

[24] Kabanov, Y., Safarian, M.: Markets with Transaction Costs: Mathematical Theory. Springer, Berlin (2009).

[25] Kabanov, Y., Stricker, C.: The Harrison–Pliska arbitrage pricing theorem under transaction costs.
Journal of Mathematical Economics, 35, 185–196 (2001)

[26] Kallsen, J.: $\sigma$-localization and $\sigma$-martingales. Theory of Probability and its Applications, 48, 152–163 (2004)

[27] Kühn, C., Molitor, A.:
Prospective strict no-arbitrage and the fundamental theorem of asset pricing under transaction costs.  Finance and Stochastics,
23, 1049–1077 (2019)

[28] Kühn, C., Molitor, A.:
Semimartingale price systems in models with transaction costs beyond efficient friction. Finance and Stochastics,
26, 927–982 (2022)

[29] Kühn, C.; Stroh, M.:
A note on stochastic integration with respect to optional semimartingales.
Electronic Communications in Probability, 14, 192–201 (2009)

[30] Larsson, M.: Conditional infimum and recovery of monotone processes. Preprint, available at

[31] Mémin J.: Espaces de semi martingales et changement de probabilité. Zeitschrift für Wahrscheinlichkeitstheorie und
verwandte Gebiete, 52, 9–39 (1980)

[32] Meyer, P.-A.: Martingales and stochastic integrals I. In: Dold, A., Eckmann, B. (eds.) Lecture Notes
in Math., vol. 284. Springer, Berlin (1972)

[33] Protter, P.E.: Stochastic Integration and Differential Equations, 2nd edn. Springer, Berlin (2004)

[34] Rokhlin, D.B.: A constructive no-arbitrage criterion with transaction costs in the case of finite discrete
time. Theory of Probability and its Applications,

52, 93–107 (2008)

[35] Sass, J.; Smaga, M.: FTAP in finite discrete time with transaction costs
by utility maximization. Finance and Stochastics,
18, 805–823 (2014)

[37]Schachermayer W.: The fundamental theorem of asset pricing under proportional transaction costs in finite discrete time. Mathematical Finance, 14, 19–48 (2004)

[38] Schachermayer, W.: The super-replication theorem under proportional transaction costs revisited. Mathematics and Financial Economics, 8, 383–398 (2014)

[39] Yan, J.: A new look at the fundamental theorem of asset pricing. J. Korean Math. Soc. 35, 659–673 (1998). International Conference on Probability Theory and Its Applications (Taejon, 1998)


11institutetext: Dept. of Computer Science, University of Sao Paulo, Brazil
11email<EMAIL_ADDRESS>
# Evaluating Presidential Support in the Brazilian House of Representatives
Through a Network-Based Approach

Tiago Colliri 11 0000-0001-9233-4662

###### Abstract

Conflicts between the executive and legislative powers are a common, and even
expected, characteristic of presidential systems, with some governments being
more successful in the activity of obtaining support from the Congress than
others. In the case of Brazil, specifically, this factor is considered crucial
in the so called “coalition governments”, with direct positive or negative
consequences for the president, in terms of government performance during his
(her) term. In this work, we investigate this problem by testing and comparing
two different methods for evaluating the government support in the Brazilian
House of Representatives. The first method is a more traditional one, being
based on roll-call voting data, and measures the presidential support at the
legislators level. The second method uses a network-based approach, and
performs the same type of analysis but at the parties level. The obtained
results, when applying both methods on legislative data comprising the period
from 1998 until 2019, indicate that both methods are valid, with common
features being found not only between the results provided by the two of them,
but also when comparing their results with those obtained by previous and
relevant studies in this field, by using the same type of data but different
methodologies.

###### Keywords:

Complex networks Centrality measures Clustering Legislative voting Bills co-
sponsorship Presidential support Political parties.

##  1 Introduction

In presidential systems, the executive and legislative branches of the federal
government are independent, with the relationship between the two powers being
characterized by periods of more conflict or cooperation. This inter-branch
friction has been gaining more attention from researchers in recent years, by
using more complex models, in which executive-legislative relations are
conceived not necessarily as the interaction between two branches of the
government, but as the relationship between the government, political parties
and groups of legislators [9]. Complex networks, on the other hand, haven been
proven suitable tools for capturing and characterizing relationship among
data, both physically, topologically and functionally [28, 10, 27]. Therefore,
not surprisingly, there is an increasingly number of studies using a network-
based approach to investigate the relations between the executive and
legislative powers. In the Brazilian presidential system, more specifically,
building stable coalitions is critical for the elected president to support
him (her) on his (her) legislative agenda, in what is currently officially
known as “coalition governments” among academics, and failing in this activity
will bring negative consequences, in terms of government performance.

There are currently several works based on networks built from data regarding
bills co-sponsorship, with focus on different aspects of the legislative
activities, such as: partisanship versus cooperation and collaboration among
parties and lawmakers [1, 22, 16, 20, 17, 2, 32], the formation of interest
groups or caucus in the congressmen networks [15, 31], evaluating government
strength or support in the Congress [13, 25, 29, 33], the representation of
minority interests in the parliament [14], the relationship between
fundraising campaigns and congressmen voting behavior [5], and even for
corruption prediction purposes [11]. In another study [19], which is more
directly related to this one, the authors investigated the collaboration and
partisanship evolution in the Brazilian House of Representatives, by using
minimum spanning tree networks built from roll-call voting data, comprising
the period from 2003 until 2008. Among their findings, there is the
identification of topological patterns in the lawmakers networks that can be
used as indicators of political instabilities over a given period of time. It
is also worth mentioning the study [8], also based on roll-call voting data
from Brazilian lawmakers, that investigates whether local and regional
interests from the respective states they represent, in the Brazilian
federalism system, may influence the legislative decision-making in the House
more than the party leaders, at the national level. After analyzing data from
1989 until 2006, to evaluate the impact of local pressures on legislative
voting, the authors have concluded that, in fact, political parties play a
more central role in the congressmen voting behavior, along with the ability
of the president to form stable legislative coalitions to control the House
voting agenda.

In this paper, we evaluate the presidential support in the Brazilian House of
Representatives by using a database built from roll-call voting data from 1998
until 2019, thus covering more than 20 years or legislative activities. Two
methods are tested and compared for this end. The first one is based on the
ratio of lawmakers votes following the government position, in each session,
when compared to the total number of votes, for each period. The second method
is a network-based approach, where the political parties and the government
are mapped as nodes in the network, and the connections between them are based
on their respective vote recommendation similarities, pairwise, in each voting
session, over a given period of time. Therefore, the first method evaluates
the presidential support by analyzing the House votes database at the members
level, while the second method performs the same type of evaluation but at the
parties level.

Regarding the organization of this paper, besides this introduction, we have,
in section 2, the details regarding the database and the methodology used in
the study. Then, in section 3, we present and discuss the obtained results
when applying the proposed technique on roll-call voting data from the built
database. At the end, in section 4, we close this study by making some final
remarks.

##  2 Materials and Methods

The methodology used in this study is described below. All network analyses
performed in this study are implemented by using the igraph Python package
[12].

###  2.1 Database

The database to be analyzed in this work was built from original data obtained
from the Brazilian House of Representatives’ official website [6]. These data
comprise the outcome of legislative voting sessions deliberated in the House,
in the period from Jan 1, 1998 until July 12, 2019, thus covering more than 20
years of legislative work. The year of 1998 was chosen as the initial one in
the built database because this is when information regarding vote
recommendations, both from party leaders and from the government, starts to
become available in the original data. As preprocessing, a thorough data
cleansing was performed in the obtained data, for detecting and fixing
possible mistakes, such as duplicated names among lawmakers and misprints. The
data for each session comprise the following information: the bill to be
considered in the House, the voting date, and the representatives who attended
that session. Additionally, the following information is provided for each
voter:

  * •

ID (a unique number for each congressperson),

  * •

Full Name,

  * •

Political Party, and

  * •

Vote.

The roll call voting system in the Brazilian House consists of, basically,
four types of votes:

  * •

Yes: when the lawmaker approves the bill;

  * •

No: when the lawmaker disapproves the bill;

  * •

Abstention: when the lawmaker deliberately chooses to not take part in the
voting;

  * •

Obstruction: similar to abstention, except that abstention counts for quorum
purposes, while obstruction does not count for it.

Other types of votes from the original data, mainly consisting of different
categories for the “absent” situation, were not taken into account in our
analysis, being discarded during the preprocessing.

At the end, we were able to obtain a total of 1,019,845 votes, from 2,900
different legislative sessions occurred during the considered period. However,
not all these sessions presented information about vote recommendations from
the part of the government, which is required for this study. Therefore, we
filtered these data one last time, to leave only the votes and sessions that
presented vote recommendations from the government during the period. The
final numbers of the dataset analyzed in this work are shown in Table 1. As
one can observe, the final data cover eight different presidencies, from the
first term of Fernando Henrique Cardoso (FHC) until the current government, of
Bolsonaro, comprising a total of 814,776 votes in 2,271 distinctive
legislative sessions.

Table 1: Summary of the data used in our analysis. The term “original” refers to all voting data available in the period, while “filtered” refers only to the data from legislative sessions that present vote recommendations from the government, which are the focus of this work. |  | Original | Filtered
---|---|---|---
Period | Presidency | Sessions | Votes | Sessions | Votes
01-01-1998 – 12-31-1998 | FHC I | 93 | 34,849 | 12 | 4,251
01-01-1999 – 12-31-2002 | FHC II | 432 | 163,511 | 329 | 127,880
01-01-2003 – 12-31-2006 | Lula I | 451 | 146,538 | 378 | 128,433
01-01-2007 – 12-31-2010 | Lula II | 619 | 207,484 | 509 | 173,498
01-01-2011 – 12-31-2014 | Dilma I | 428 | 139,007 | 328 | 111,090
01-01-2015 – 12-02-2015 | Dilma II | 274 | 112,404 | 247 | 101,401
08-31-2016 – 12-31-2018 | Temer | 463 | 161,662 | 358 | 125,155
01-01-2019 – 07-12-2019 | Bolsonaro | 140 | 54,390 | 110 | 43,068
| Total | 2,900 | 1,019,845 | 2,271 | 814,776

###  2.2 Political Parties Networks Generation

To perform our analyses, we map each political party in the voting sessions as
the node in a network, with one additional node representing the government,
and the edges between them are created according to their respective vote
recommendations similarity, pairwise, on all sessions of the same period.
Given that there are currently 33 political parties in Brazil, then the
maximum number of nodes in the network obtained from current voting data, in
the hypothetical situation where all parties currently occupy seats in the
House, will be 34, i.e., the total number of parties plus one additional node
to represent the government. Building a network from bills co-sponsorship data
is a well known technique, that already has been extensively applied in other
related studies [14, 21, 15]. However, most of these works focus on the
analysis of roll-call voting data at the House members level, and not at the
parties level, as in the case of this approach. Besides of this, here we are
also including a node to represent the government in the networks, which can
be considered a novelty in this type of study.

A network can be defined as a graph $\mathcal{G}=(\mathcal{V,E})$, where
$\mathcal{V}$ is a set of nodes and $\mathcal{E}$ is a set of tuples
representing the edges between each pair of nodes
$(i,j)\mathrel{\mathop{\mathchar 58\relax}}i,j\in\mathcal{V}$. The edges in
$\mathcal{E}$ are usually provided in the form of a square matrix $M$, with
size equal to the number of nodes in the network and binary values, in case of
unweighted graphs. In our case, we start by mapping the parties vote
recommendation similarities, in each voting session $s$, as a square matrix
$A$, according to:

$A_{i,j}(s)=\begin{cases}1,\text{ if party $i$ and $j$ recommended the same
vote in session $s$},\\\ -1,\text{ otherwise}.\end{cases}$ (1)

The values in $A$ are then accumulated in a matrix $W$, considering all voting
sessions occurred in a given time interval $T$, in the form of:

$W_{i,j}(T)=\sum_{s}A_{i,j}(s)\mid s_{t}\in T\quad,$ (2)

where $s_{t}$ returns the time $t$ when session $s$ has happened. In this way,
the values in matrix $W$ will, basically, provide us the overall level of
political agreement (or disagreement) between each pair of parties $i$ and
$j$, based on their respective vote recommendations in all $n$ sessions
occurred within the period $T$. Please note that, consequently, the maximum
possible value of $W_{i,j}(T)$ will be $+n$, in case the party leaders from
$i$ and $j$ presented the same vote recommendations for all sessions in the
period, while, on the other hand, the minimum value will be $-n$, which means
that their party leaders gave different vote recommendations in all sessions
of that period.

There are different possible ways of generating a network $\mathcal{G}(T)$
from matrix $W(T)$. One of this ways is, for instance, simply mapping it as a
weighted network, using the values in $W(T)$ as the edges weights. Or, as
another possibility, one can also map the values in $W(T)$ to binary ones,
according to a predefined threshold value. In this study, we opt for
generating edges between each pair of vertices $i$ and $j$ based on the
following rule:

$\mathcal{G}_{i,j}(T)=\begin{cases}1,\textrm{ if }W_{i,j}(T)=\displaystyle\max
W_{i}(T)\\\ 0,\textrm{ otherwise }.\\\ \end{cases}$ (3)

This method results in an unweighted graph, in which parties are connected
only to the ones that are the most politically aligned to them, in terms of
their respective leaders vote recommendations in the House. We decide to
proceed this way, by considering only the maximum values in matrix $W(T)$ when
generating the edges, mainly for the following reasons: (1) as the resulting
networks will be less connected, we believe their topological structures will
reflect only the most meaningful information, in terms of the parties
political similarities, in each period, (2) it also facilitates the
interpretability and visibility of the results, and (3) less dense networks
are more sensitive to the specific measures that will be extracted from them
later, when evaluating the government support in each presidency.

With regards to the time interval $T$, we opt for using a yearly frequency for
this end, with $T\in[1998,2019]$, respecting the initial and final dates of
each presidency, as listed in Table 1, which results in 22 different political
parties networks generated.

###  2.3 Presidential Support Evaluation

We evaluate the level of support for each presidency in the House of
Representatives through two different methods. The first one is based on the
actual legislative voting data for the period, i.e., bills co-sponsorship
data, and considers the votes registered by lawmakers and the respective vote
recommended by the government, in each session, as it has been already made in
other similar studies [25, 13]. The second method is based on specific
measures extracted from the built political parties networks, which, as we
described above, are based on data regarding vote recommendations, both from
party leaders and from the government. We proceed this way with the objective
of not only making comparisons between the two methods, but also with the aim
of analyzing both results from the perspective of the main political events
happened during the last two decades in Brazil.

In the first method considered, we process all roll-call votes $v$, made by
each lawmaker $i$, in each session $s$, and the level of presidential support,
for a period of time $T$, is provided by:

$\frac{\sum_{s}\mathinner{\\!\left\lvert
f(v)\right\rvert}}{\sum_{s}\mathinner{\\!\left\lvert v\right\rvert}}\mid
f(v)=\\{v_{i}\mid v_{i}\in v\land v_{i}=v_{g}(s)\\}\land s_{t}\in T\quad,$ (4)

where $v_{g}(s)$ stands for the vote recommended by the government for session
$s$, $s_{t}$ returns the time $t$ when session $s$ happened, and
$\mathinner{\\!\left\lvert v\right\rvert}$ is the length of array $v$. Hence,
in another words, this method provides us with the presidential support in the
House measured as the ratio of the number of votes from congressmen following
the respective government recommendation in each session, when compared to the
total number of votes registered during the period.

For the second method, we extract specific measures from the obtained
political parties networks, and assess the government support in the House
based on these measures, in each period. The network measures considered for
this end are described below:

  * •

Closeness: a centrality measure, defines how close a vertex is to all other
vertices in the graph [23], being calculated as the reciprocal of the sum of
the length of the shortest paths between the node and all other nodes in the
graph [26]. In the context of this study, a higher closeness centrality for
the node representing the government in the political parties network will
indicate a higher level of House support for that presidency.

  * •

PageRank: also a centrality measure, uses an algorithm originally developed
for ranking the importance of website pages in the World Wide Web [24]. In
this case, a higher PageRank score for the government, in the political
parties networks, indicates more support in the House for that presidency.

  * •

Hub Score: also known as the HITS algorithm [7], is based on the idea that
hubs with higher scores represent nodes with links to many other nodes, and a
higher authority score is given to nodes that are linked to many different
hubs. This indicator will be used in the same manner that the centrality
measures described above, with higher scores for the government’s node, in the
political parties networks, indicating a higher support in the House.

  * •

Network Density: describes the portion of the potential connections in a
network that are actual connections or, in another words, the total number of
links over the maximum possible number of links [3]. Differently from the
others above, which are analyzed from the node level, this measure is
extracted for the whole network, and has already been used in other similar
studies, mainly for assessing and comparing connections within political
parties [31, 18, 30]. In this work, we used it as an indicator of how
connected are the political parties and also the government in the built
networks, for each presidency, with less connections indicating a more divided
and fragmented House, hence resulting in more difficulties for the president
in obtaining bills approval support.

##  3 Results and Discussion

In this section, we present the results obtained when applying the two
considered approaches on the built database, comprising real voting data from
around 21 years of legislative work in the Brazilian House of Representatives.

We start by showing, in Fig. 1, some examples of the political parties
networks generated by the network-based approach, for each year. To help in
the analysis, we have also applied the fastgreedy algorithm [4] to detect
communities in the networks. The node representing the government is
highlighted in black. As one can observe, as a general rule in these networks,
the position of the government’s node is highly influenced by the party
currently in power. In the first graph, for instance, the presidency was from
PSDB party, which results in the government’s node being directly connected to
this party in the network. This same feature can be observed in the graphs
from other periods as well, with the node from the government always being
closer to the party currently in the presidency, and also to other parties
that supported that presidency, in each year.

(a) 1998 (FHC I - PSDB)

(b) 2002 (FHC II - PSDB)

(c) 2004 (Lula I - PT)

(d) 2009 (Lula II - PT)

(e) 2012 (Dilma I - PT)

(f) 2014 (Dilma II - PT)

(g) 2016 (Temer - MDB)

(h) 2019 (Bolsonaro - PSL)

Figure 1: Examples of the political parties networks generated in our
analysis, based on the party leaders vote recommendations in the House of
Representatives, in each year. The communities are detected using the
fastgreedy algorithm. The vertex representing the government is highlighted in
black.

Still regarding Fig. 1, we would like to call a special attention to the
communities, which, in this case, can be seen as groups of parties that share
similar political views, based on the vote recommendations from their leaders
in the sessions, for each period. Therefore, for those who are somewhat
familiar with the political scenario in Brazil for the last 20 years, it is
easy to notice the formation of some sort of “expected” aggregations in these
networks topological structures. Historical left wing parties, for instance,
such as PT, PCdoB. PSB, PDT and PSOL, oftentimes share the same cluster in the
networks, in different periods, thus confirming their respective ideological
affinities through legislative data. The same observation can also be made for
parties historically more associated to right wing political views, such as
DEM (former PFL), PL, PP and PSL. These results are overall consistent with
those obtained in previous works, using the same type of data but a different
methodology [29, 33]. However, there are also some unexpected formations
resulting from these topological structures, such as the fact that the PT and
DEM parties share the same community in the network for 2009, in Fig. 1(d),
which means that the vote recommendations from their party leaders, in this
specific year, were not as different as they were in other years.

(a) Closeness

(b) PageRank

(c) Hub Score

(d) Network Density

Figure 2: Support in the House for each presidency, assessed according to two
different methods, for the same period. The first method (in blue) is based on
the House members co-sponsorship historical data. The second one (in green) is
based on specific measures extracted from the built political parties
networks: (a) Closeness, (b) PageRank, (c) Hub Score and (d) Network Density

Let us now proceed to Fig. 2, where we have four plots showing the evolution
of the presidential support in the House, evaluated according to the two
methods tested in this study. The first method, denoted by the blue color in
the plots, is based on co-sponsorship legislative data, considering the ratio
of votes registered by congressmen following the respective recommendation
from the government, in each session, when compared to the total number of
votes recorded within the period of each presidency. The second method tested,
represented by the green color in the plots, is based on specific measures
extracted from the built political parties networks, with their respective
yearly values being averaged, for each presidency. The green line in the first
three plots, in Figs. 2(a), 2(b) and 2(c), shows the evolution of each
respective measure for the node represented by the government in the networks.
Hence, in this form of analysis, the higher the value of each measure in these
plots, the higher is the support for that presidency in the House. While, in
Fig. 2(d), the density measure is extracted for the whole network and, in this
case, less dense networks mean less connections between the nodes and, within
this context, indicate a more divided and fragmented House of Representatives,
often resulting in more difficulties for the president to build stable
coalitions for controlling the House voting agenda.

As one can observe, in Fig. 2, although being based on different approaches,
both methods overall still present some similar features regarding the
evolution of the presidential support in the House during the considered
period. Three of the plots show an initial peak for the presidential support
evolution, in the FHC II government, followed by a descending behavior, in the
Lula presidency, with the only exception for this rule being the analysis
based on the Hub Score indicator. This same phenomenon is also observed in
previous researches, using the same type of data but different methodologies
[8, 33]. Another common feature in these analyses is in the fact that all
indicators present a descending movement in the first and second terms of the
Dilma presidency, indicating that the support in the House was lower during
this period, when compared to other ones. This phenomenon makes sense whereas
one take into account the fact that the second term of Dilma was interrupted
in December 2015, because of the impeachment process, with Temer officially
assuming the presidency in August 2016. These results are also in line with
those obtained by other researchers, using the same data [19].

Another interesting characteristic that can be noted in Fig. 2 is the
indication that the second term of Lula had more support in the House than the
first one, according to the analyses based on Closeness, PageRank and co-
sponsorship voting data. Again, this same feature has also been observed in a
previous study [33], using similar data but a different methodology. There is
also the indication that the Bolsonaro presidency has more support in the
House than its predecessor Temer, according to the same analysis techniques
mentioned earlier plus the one based on Network Density. Additionally, still
according to this measure, one can notice that, as Fig. 2(d) indicates, the
level of fragmentation among political parties in the House has been
increasing since the last term of FHC, reaching its peak during the Temer
government, i.e., the bottom in terms of the network density measure, and with
a slight pullback in the Bolsonaro presidency. A less dense network, in this
context, suggests the incidence of more partisanship and non-cooperation among
parties and, consequently, more difficulties for the government to obtain
support in the House during the last two decades. This result corroborates as
well with those obtained by previous studies, although for US voting data [2,
22, 17, 20].

Figure 3: Total votes and loyalty rates for the 15 political parties with
highest number of votes in the Brazilian House of Representatives, for the
period of 01-01-1998 until 07-12-2019. Only currently active parties are
considered for generating this figure.

We have seen, in Fig. 2, that despite the methods used for evaluating the
support for each presidency in the House are different, with one being based
on actual voting data and the other being based on the networks built from
party leaders vote recommendations, they still present similar features
overall, in terms of the evolution of the support in different presidencies.
However, the occurrence of such similarities is only possible if lawmakers
actually follow the vote recommendations from their respective party leaders,
otherwise the analyses resulting from both methods would be completely
different. Therefore, to better understand the results, we present, in Fig. 3,
the respective loyalty rates for the main political parties in Brazil, in
terms of total number of votes in House sessions during the period considered
in this study. Only currently active parties are included in this figure.
Overall, the average level of loyalty for the 15 parties is 62%. The parties
with the highest loyalty rates are PCdoB and PT, with 72% and 70% of the votes
following the recommendation from their leaders during the period,
respectively. If we analyze these data also taking into account the total
number of votes, the main parties in the House during the period would be MDB,
PT and PSDB, with MDB and PSDB having similar loyalty rates, of 61% and 62%,
respectively. In this aspect, and disconsidering political ideologies, the
ideal situation for a government hence would be to receive support, through
the formation of coalitions, from political parties that not only have a large
number of seats in the House, but that also present a higher rate of loyalty
from their members to the parties leaderships, in terms of voting.

##  4 Final Remarks

In this study, we evaluated the government support in the Brazilian House of
Representatives by testing two different methods on data comprising the
legislative sessions occurred in the period between 1998 and 2019. The
obtained results indicate that the proposed network-based approach is valid,
given that, although being different from the more traditional method also
tested in this study, which is based solely on co-sponsorship voting data, it
was still able to present many common features with the latter, when assessing
the support in the House for each presidency during the considered period.
Additionally, as this is essentially a graphical approach, there is also the
possibility of taking advantage of this as a supplementary resource for both
enriching and facilitating the analysis, as we have demonstrated in Fig. 1, by
presenting the networks generated by the model. Moreover, we also found that
most of the results obtained in this work, regarding the ideological
similarities between political parties and the evolution of the government
support in the House, in each presidency, are in line with the ones achieved
by previous relevant studies in this field, based on the same type of data but
using different methodologies.

As future works, we plan to further explore the built database by making
additional analyses regarding the political parties and presidential support.
Among these possibilities, we can mention: to verify the existence of a
possible correlation between presidential support and the number of
government-sponsored bills that were approved in the House, in the same
period; to perform analyses based on each type of bill, to find out which
types of legislation are more likely to receive support from the members of
the House; and also to make a more detailed study regarding the formation of
interest groups (or caucus) among the congressmen and/or their respective
parties.

## References

  * [1] Andris, C., Lee, D., Hamilton, M.J., Martino, M., Gunning, C.E., Selden, J.A.: The rise of partisanship and super-cooperators in the U.S. House of Representatives. PLoS One 10(4), 1–14 (2015). https://doi.org/https://doi.org/10.1371/journal.pone.0123507
  * [2] Aref, S., Neal, Z.: Detecting coalitions by optimally partitioning signed networks of political collaboration. Scientific Reports 10(1), 1–10 (2020)
  * [3] Barabási, A.L., et al.: Network Science. Cambridge University Press (2016)
  * [4] Brandes, U.: A faster algorithm for betweenness centrality. Journal of Mathematical Sociology 25(2), 163–177 (2001)
  * [5] Bursztyn, V.S., Nunes, M.G., Figueiredo, D.R.: How Brazilian congressmen connect: homophily and cohesion in voting and donation networks. Journal of Complex Networks 8(1), cnaa006 (2020)
  * [6] Câmara: Dados Abertos. [accessed on August 1, 2019] (2019), https://dadosabertos.camara.leg.br/
  * [7] Chakrabarti, S., Dom, B., Raghavan, P., Rajagopalan, S., Gibson, D., Kleinberg, J.: Automatic resource compilation by analyzing hyperlink structure and associated text. Computer Networks and ISDN Systems 30(1-7), 65–74 (1998)
  * [8] Cheibub, J.A., Figueiredo, A., Limongi, F.: Political parties and governors as determinants of legislative behavior in Brazil’s Chamber of Deputies, 1988–2006. Latin American Politics and Society 51(1), 1–30 (2009)
  * [9] Cheibub, J.A., Limongi, F.: Legislative-executive relations. Comparative Constitutional Law (12), 211 (2011)
  * [10] Colliri, T., Ji, D., Pan, H., Zhao, L.: A network-based high level data classification technique. In: 2018 International Joint Conference on Neural Networks (IJCNN). pp. 1–8. IEEE (2018)
  * [11] Colliri, T., Zhao, L.: Analyzing the bills-voting dynamics and predicting corruption-convictions among Brazilian congressmen through temporal networks. Scientific Reports 9(1), 1–11 (2019)
  * [12] Csardi, G., Nepusz, T., et al.: The igraph software package for complex network research. InterJournal, Complex Systems 1695(5), 1–9 (2006)
  * [13] Dal Maso, C., Pompa, G., Puliga, M., Riotta, G., Chessa, A.: Voting behavior, coalitions and government strength through a complex network analysis. PLoS One 9(12), e116046 (2014)
  * [14] Epstein, D., Fowler, J., O’Halloran, S.: Co-sponsorship networks of minority-supported legislation in the House. In: Annual Meeting of the American Political Science Association, Philadelphia, Pennsylvannia (2006)
  * [15] Fischer, M., Varone, F., Gava, R., Sciarini, P.: How MPs ties to interest groups matter for legislative co-sponsorship. Social Networks 57, 34–42 (2019)
  * [16] Kirkland, J.H., Gross, J.H.: Measurement and theory in legislative networks: The evolving topology of Congressional collaboration. Social Networks 36(1), 97–109 (2014). https://doi.org/https://doi.org/10.1016/j.socnet.2012.11.001
  * [17] Layman, G.C., Carsey, T.M., Horowitz, J.M.: Party polarization in American politics: Characteristics, causes, and consequences. Annual Review of Political Science 9(1), 83–110 (2006). https://doi.org/10.1146/annurev.polisci.9.070204.105138
  * [18] Lietz, H., Wagner, C., Bleier, A., Strohmaier, M.: When politicians talk: Assessing online conversational practices of political parties on Twitter. In: Proceedings of the International AAAI Conference on Web and Social Media. vol. 8 (2014)
  * [19] Marenco, L., Carmona, H.A., Cardoso, F.M., Andrade Jr, J.S., Lenz Cesar, C.: Time evolution of the behaviour of Brazilian legislative representatives using a complex network approach. PLoS One 15(2), e0226504 (2020)
  * [20] Moody, J., Mucha, P.J.: Portrait of political party polarization. Network Science 1(1), 119–121 (2013). https://doi.org/10.1017/nws.2012.3
  * [21] Neal, Z.: The backbone of bipartite projections: Inferring relationships from co-authorship, co-sponsorship, co-attendance and other co-behaviors. Social Networks 39, 84–97 (2014)
  * [22] Neal, Z.P.: A sign of the times? Weak and strong polarization in the US Congress, 1973–2016. Social Networks 60, 103–112 (2020)
  * [23] Okamoto, K., Chen, W., Li, X.Y.: Ranking of closeness centrality for large-scale social networks. In: International Workshop on Frontiers in Algorithmics. pp. 186–195. Springer (2008)
  * [24] Page, L., Brin, S., Motwani, R., Winograd, T.: The pagerank citation ranking: Bringing order to the web. Tech. rep., Stanford InfoLab (1999)
  * [25] Prins, B.C., Shull, S.A.: Enduring rivals: Presidential success and support in the house of representatives. In: Congress & the Presidency: A Journal of Capital Studies. vol. 33, pp. 21–46. Taylor & Francis (2006)
  * [26] Sabidussi, G.: The centrality index of a graph. Psychometrika 31(4), 581–603 (1966)
  * [27] Silva, T.C., Zhao, L.: Network-based high level data classification. Neural Networks and Learning Systems, IEEE Transactions on 23(6), 954–970 (2012)
  * [28] Strogatz, S.H.: Exploring complex networks. Nature 410(6825), 268–276 (2001)
  * [29] Tsai, T.h.: The influence of the president and government coalition on roll-call voting in Brazil, 2003–2006. Political Studies Review p. 1478929920904588 (2020)
  * [30] Verweij, P.: Twitter links between politicians and journalists. Journalism Practice 6(5-6), 680–691 (2012)
  * [31] Victor, J.N., Ringe, N.: The social utility of informal institutions: Caucuses as networks in the 110th US House of Representatives. American Politics Research 37(5), 742–766 (2009)
  * [32] Waugh, A.S., Pei, L., Fowler, J.H., Mucha, P.J., Porter, M.A.: Party polarization in Congress: A network science approach. arXiv preprint arXiv:0907.3509 (2009)
  * [33] Zucco Jr, C., Lauderdale, B.E.: Distinguishing between influences on Brazilian legislative behavior. Legislative Studies Quarterly 36(3), 363–396 (2011)