# Learning-Augmented Model-Based Planning for Visual Exploration

Yimeng Li*1, Arnab Debnath*1, Gregory J. Stein1 and Jana Košecká1 * Denotes
equal contribution.1Yimeng Li, Arnab Debnath, Gregory Stein and Jana Kosecka
are with the Department of Computer Science, George Mason University, 4400
University Dr, Fairfax, VA, USA [yli44, adebnath, gjstein<EMAIL_ADDRESS>
###### Abstract

We consider the problem of time-limited robotic exploration in previously
unseen environments where exploration is limited by a predefined amount of
time. We propose a novel exploration approach using learning-augmented model-
based planning. We generate a set of subgoals associated with frontiers on the
current map and derive a Bellman Equation for exploration with these subgoals.
Visual sensing and advances in semantic mapping of indoor scenes are exploited
for training a deep convolutional neural network to estimate properties
associated with each frontier: the expected unobserved area beyond the
frontier and the expected timesteps (discretized actions) required to explore
it. The proposed model-based planner is guaranteed to explore the whole scene
if time permits. We thoroughly evaluate our approach on a large-scale pseudo-
realistic indoor dataset (Matterport3D) with the Habitat simulator. We compare
our approach with classical and more recent RL-based exploration methods. Our
approach surpasses the greedy strategies by 2.1$\%$ and the RL-based
exploration methods by 8.4$\%$ in terms of coverage.

##  I Introduction

We consider a robot deployed in a previously-unseen environment and given a
limited time during which it is expected to explore, move around, and become
familiar with its surroundings. Task-independent exploration is critical for
many downstream navigation tasks, including traveling to a point goal [1, 2,
3], looking for a target object [4], or disaster response scenarios [5]. The
robot must explore intelligently to cover the maximum area under time
constraints.

However, efficient exploration is incredibly challenging since good
performance requires inferences about unseen space’s layout. For example, a
few steps of travel inside a living room can bring a broad vision of the
entire scene rather than spending a long time walking through a narrow
corridor. Due to the complexities of planning under uncertainty, many existing
approaches rely on learning to make predictions about unseen space and
determine behavior. Recent advent of learning-based approaches proposed for
robot visual exploration [1, 3, 6, 7] leverages 3D textured mapped models of
indoor scenes collected from real-world environments [8, 9] to learn
structural and semantic regularities informative for exploration.

Many learning-driven strategies for exploration [1, 7] are model-free and
trained via deep reinforcement learning. These models typically learn policies
on top of intermediate representations of indoor scenes, are data-intensive to
train, and require millions of frames of training data. They often struggle in
novel environments as the length of exploration grows. These approaches are
either unreliable and fail to fully explore the environment in the fullness of
time or have poor performance, owing to their inability to directly predict
the long-horizon impact of their actions.

Figure 1:  Schematic showing our learning-augmented model-based exploration
strategy. Our approach associates high-level exploratory actions with
_frontiers_ and uses a Bellman Equation to estimate the expected reward of
each action. The learning module takes in the input of the currently observed
occupancy map and semantic map or egocentric observations and predicts the
unknown properties of the frontier. The computed frontiers are illustrated
with green pixels in the occupancy map, and the red nodes are centroids.
$R_{A}$ and $D$ are the estimated area and timesteps for each frontier.

In _frontier-based methods_ , the robot’s actions correspond to navigation to
frontiers—each a boundary between free and unknown space—and rely on the
partial map to plan and reliably navigate through space the robot has already
seen. Fig. 1 (bottom) shows computed frontiers on an example occupancy map.
Classical frontier-based exploration (FBE) methods [10] rely on an occupancy
map, built via a laser range finder, to compute frontiers, and greedily select
the next frontier-action via simple heuristics (e.g., navigation to the
nearest frontier), resulting in suboptimal plans. Recent work [4] uses visual
sensing and replaces simple heuristics with a learning-based module that
estimates the expected area beyond each frontier from a top-down-view semantic
map. Though the predictions from learning help to improve exploration,
planning is still myopic, limiting performance. If such frontier-based
planning strategies are to improve, the robot must have the capacity to reason
into the future and understand the impact of its actions: including both (i)
_how much area exploration beyond each frontier is expected to reveal_ and
(ii) _how long it will take to explore each region of the environment_.

We present _Learning-Augmented Model-Based Frontier-Based Exploration_ (LFE)
that allows for reliable long-horizon exploration. Taking inspiration from
recent work by Stein et al. [11], we introduce a high-level abstraction that
allows for model-based planning over the set of available frontiers and
approximates a complex Partially Observable Markov Decision Process (POMDP)
[12] as a much simpler Markov Decision Process (MDP) [13]. Motivated by
Ramakrishnan et al. [4], we use environment semantics and visual perception to
learn how to estimate the expected area that exploration beyond a frontier
will reveal and how long such an exploration is expected to take. Our
resulting model-based planner produces the sequence of exploratory actions
that will maximize expected area coverage in the allowable time. Moreover, our
approach generalizes well to large environments, while model-free RL
approaches [1, 7] do not accommodate well to large-scale environments not
encountered during training.

Our contribution is a frontier-based model, similar to that of Stein et al.
[11] for the task of time-limited exploration, with the aim to maximize the
explored area within a predefined period. We evaluate the performance of our
approach on a large-scale pseudo-realistic indoor dataset (Matterport3D [8])
with the Habitat simulator [14], and achieve state-of-the-art performance
(80.7%) on time-limited area coverage and compare our approach with learned
greedy [4] and RL-based [6] methods.

##  II Related Work

Embodied Agent Navigation Embodied agent navigation [15] refers to an agent’s
ability to navigate and interact with its environment in a way similar to how
a human would. In recent years, there has been a surge of research in embodied
navigation thanks to the availability of photo-realistic simulators [9, 16,
17, 18] for large-scale indoor environments [8, 19, 20]. The proposed tasks
include PointGoal Navigation [21, 22, 23], ObjectGoal Navigation [24, 25],
ImageGoal Navigation [2, 26, 27, 28] Vision-and-Language Navigation (VLN) [29,
30], and Object Rearrangement [31, 32, 33]. In this paper, we tackle the
exploration task, which is the fundamental problem to all the aforementioned
navigation tasks.

End-to-End Visual Exploration Recent deep reinforcement learning methods [1,
7, 25, 34] learn a policy to predict low-level actions directly from raw RGBD
observations or local occupancy maps to improve navigation performance or
exploration coverage. These end-to-end RL approaches usually are sample
inefficient and require high computation. Instead of directly selecting short-
term actions, we introduce a dynamic action set of actions to navigate to a
frontier that allow us to make predictions computationally efficiently.

Intermediate Goals Other deep reinforcement learning methods [3, 6, 35, 36]
use hierarchical models that have global policies that predict long-term goals
and use local navigation modules to reach the long-term goal. However, such
methods still choose long-term goals greedily, and these long-term goals are
usually located in an unknown area which may not be reachable by the agent.
Our approach is most similar to that of Stein et al. [11], in which a
learning-augmented frontier-based abstraction is used for point-goal
navigation. We instead derive a Bellman Equation particularly for the time-
limited exploration task.

Frontier-Based Exploration Frontier-based exploration is first proposed by
Yamauchi [10], where the robot simply goes to the nearest frontier. With the
availability of camera sensors and semantic-informative simulators, it has
become possible to estimate more complex properties of frontiers. Recently,
Ramakrishnan et al. [4] used visual sensing and proposed a learning-augmented
FBE model for the robot going to a target object. Their learning module is a
neural network that takes the observed semantic map as input and estimates the
unknown space beyond the frontier and semantic heuristics may lead to the
object. Our method differs from theirs in that we use a variant of the Bellman
Equation to guide frontier selection that considers travel cost besides area
of the unknown space.

Figure 2:  This diagram gives an overview of our learning-augmented model-
based exploration algorithm, which allows us to compute the expected value of
each action in a computationally feasible way for planning through an unknown
environment. We use learning modules to estimate the terms $R_{A}$,
$D_{\text{in}}$ and $D_{\text{out}}$, thereby introducing prior knowledge
about environment regularities into the decision-making procedure.

##  III Approach: Learning-Augmented Model-Based Frontier Exploration

We aim to maximize area coverage during exploration through a static, unknown
environment within an allowed number of timesteps. We model exploration as a
Partially Observable Markov Decision Process (POMDP) [12]. The expected reward
of taking an action can be expressed via a belief-space variant of the Bellman
Equation [37]:

$\begin{split}Q(b_{t},a_{t}\in\mathcal{A}(b_{t}))&=\sum_{b_{t+1}}P(b_{t+1}|b_{t},a_{t})[R(b_{t+1},b_{t},a_{t})\\\
&+\max_{a_{t+1}\in\mathcal{A}(b_{t+1})}Q(b_{t+1},a_{t+1})]\end{split}$ (1)

where $R(b_{t+1},b_{t},a_{t})$ is the expected reward (in units of area
coverage) of reaching belief state $b_{t+1}$ from $b_{t}$ by taking action
$a_{t}$. Our belief state $b_{t}$ is represented as a three-element tuple
consisting of the partially observed map $m_{t}$, the robot pose $q_{t}$ and
the remaining available timesteps $\sigma_{t}$:
$b_{t}=\\{m_{t},q_{t},\sigma_{t}\\}$. In practice, computing the expected
reward via Eq. (1) is computationally infeasible, since it requires taking an
expectation over the distribution of all possible environments and reasoning
hundreds of timesteps into the future.

###  III-A Model-Based Planning with Subgoals

To simplify the calculation of the expected reward, we introduce an
abstraction, inspired by that of Stein et al. [11], in which the robot’s
actions are long-term behaviors: each associated with exploring a particular
region of unseen space. Under this abstraction, each frontier, a boundary
between free and unseen space, is assigned a _subgoal_. A high-level action
$a_{t}\in\mathcal{A}(b_{t})$ consists of first navigating through known space
to the subgoal of interest—traveling a _known-space distance_
$D_{\text{k}}(m_{t},a_{t})$—and then revealing the region beyond. After each
such action, the robot will have (i) accumulated a reward corresponding to the
amount of area revealed and (ii) expended time to explore, thus decreasing
$\sigma_{t}$ by how long the exploration took. As the robot reveals more of
the environment, it updates its partial map and recomputes the set of
available subgoal-actions and replans.

###  III-B Approximating Expected Reward

To select the next high-level action $a_{t}$, we must be able to evaluate it’s
expected reward. In the context of exploration, the expected instantaneous
reward for a particular high-level exploratory action $a_{t}$ is the
unexplored navigable area beyond that action’s corresponding frontier. This
reward ($R_{A}$) depends on the current partial map $m_{t}$ and the area that
can be explored in the steps remaining ($\sigma_{t}-D_{\text{k}}$), as
specified by action $a_{t}$:

$R_{A}(m_{t},a_{t},\sigma_{t}-D_{\text{k}})\equiv\sum_{b_{t+1}}P(b_{t+1}|b_{t},a_{t})R(b_{t+1},b_{t},a_{t})$
(2)

Since we neither have direct access to the belief nor the computational
resources to evaluate Eq. (2) directly, we will instead estimate $R_{A}$ using
learning described in Sec. V.

Evaluating the recursive term in Eq. (1) remains challenging since updating
the map $m_{t}$ (and therefore computing $b_{t+1}$) remains computationally
infeasible. We instead make a simplifying assumption: we do not directly
update the occupancy grid during high-level planning and instead prevent the
robot from exploring the same space again by keeping track of what has already
been explored. As such, we define the set of future actions
$\mathcal{A}_{t+1}$ as the current set of actions minus the one that was just
chosen (since we disallow exploration of the same region more than once), so
that $\mathcal{A}_{t+1}=\mathcal{A}_{t}\setminus\\{a_{t}\\}$. Therefore, the
expected reward of taking action, $a_{t}$ using the current map $m_{t}$, would
mean getting the instantaneous reward $R_{A}$ from exploring the associated
frontier and then considering the $A_{t+1}$ actions without introducing new
frontiers by updating the map.

Moreover, as mentioned in Sec. III-B, during an exploratory action $a_{t}$ the
robot first travels a distance $D_{\text{k}}$ through known space and then
spends a time to explore the unknown space beyond a frontier. We use the
A${}^{\\!*}$ algorithm to plan the trajectory through known space and compute
$D_{\text{k}}$. The expected number of steps to explore can be written as the
sum of two terms: $D_{\text{in}}$, the expected number of steps to explore the
region, and $D_{\text{out}}$, the expected number of steps to exit the newly-
explored region. Splitting the number of steps into two parts enables easier
computation of frontier Q-values, as we only need to consider $D_{\text{in}}$
when we have only one frontier left. Similar to $R_{A}$ from Eq. (2), we
cannot compute $D_{\text{in}}$ and $D_{\text{out}}$ exactly as it is
computationally expensive and so estimate them from learning. Furthermore,
upon completing its exploration of a region, the robot has moved to the
frontier and so its pose change from $q_{t}$ to $q_{a_{t}}$.

With these, we can approximate the recursive term from Eq. (1) as follows:

$\begin{split}\sum_{b_{t+1}}&P(b_{t+1}|b_{t},a_{t})\max_{a_{t+1}\in\mathcal{A}(b_{t+1})}Q(b_{t+1},a_{t+1})\approx\\\
&\max_{a\in\mathcal{A}_{t}\setminus\\{a_{t}\\}}Q(\left\\{m_{t},q_{a_{t}},\mathcal{A}_{t}\\!\setminus\\!\\{a_{t}\\},\sigma_{t}-D_{\text{k}}\\!-\\!D_{\text{in}}\\!-\\!D_{\text{out}}\right\\},a)\end{split}$
(3)

Thus, from Eq. (1), (2) & (3) we can rewrite expected reward:

$\begin{split}Q(&\left\\{m_{t},q_{t},\mathcal{A}_{t},\sigma_{t}\right\\},a_{t})=R_{A}(m_{t},a_{t},\sigma_{t}-D_{\text{k}})+\\\
&\max_{a\in\mathcal{A}_{t}\setminus\\{a_{t}\\}}Q(\left\\{m_{t},q_{a_{t}},\mathcal{A}_{t}\\!\setminus\\!\\{a_{t}\\},\sigma_{t}-D_{\text{k}}\\!-\\!D_{\text{in}}\\!-\\!D_{\text{out}}\right\\},a)\end{split}$
(4)

An illustration of how expected cost is computed under our model can be found
in Fig. 2.

Figure 3:  Architecture of the entire exploration system. The detected and the
selected frontiers are drawn in green and yellow respectively. Our proposed
Learning-augmented model-based frontier-based exploration is used for the
frontier selection module. We design two learning modules to estimate the
frontier properties. The UNet model takes in the local occupancy map and local
semantic map and estimates values for all the visible frontiers on the map.
The ResNet-18 model takes in the egocentric depth and semantic observation
corresponding to a frontier and estimates the values only for this frontier.

###  III-C Hierarchial Navigational Model

Our planner is hierarchical: high-level planning involves selection among
frontier subgoals, while low-level planning involves navigation to the chosen
frontier. Planning with our model proceeds as follows: (i) we compute the set
of frontiers and the respective action set $\mathcal{A}(b_{t})$ given the map
$m_{t}$, (ii) we compute the expected reward for each exploratory action
according to Eq. (4), (iii) we choose the action $a_{t}^{*}$ that maximizes
the expected reward or, if more than one action reaches the maximum reward, we
choose the frontier with the most timesteps remaining. Once the frontier is
selected, we (iv) compute a motion plan to travel to the centroid of the
frontier associated with action $a_{t}^{*}$ ($q^{*}$) and finally (v) select
the primitive action that best moves along the computed path and update the
map given the newly observed area. This process repeats at each timestep until
no frontiers remain in the observed map or time is exhausted.

An overview of our approach can be found in Fig. 3.

##  IV Map and Frontier Computation

Occupancy and Semantic Mapping The occupancy map is a metric map of size
$m\times m$ where each cell of the map can have three values: 0 (unknown), 1
(occupied) and 2 (free space). The semantic map is a metric map of size
$m\times m$ where each cell’s value is in the range $[0,N]$. Each value from 1
to $N$ corresponds to one of the N object categories, and 0 refers to the
undetected class. Each cell is a $5cm\times 5cm$ region in the real world.
Given an RGB-D view, we build the occupancy map and semantic map by projecting
semantic segmentation images to a 3D point cloud using the available depth
maps and the robot poses, discretizing the point cloud into a voxel grid and
taking the top-down view of the voxel grid. The occupancy map depends on the
height of the points projected to each cell. The semantic map depends on the
majority category of the points located at the top grid of each cell.

Frontier Computation Frontiers consist of unknown grid cells on the boundary
of free space in the partial occupancy map. A frontier is a set of connected
cells that follow an eight-neighbour connection. The center of the frontier is
the centroid of the points belonging to that frontier. We further reduce the
number of frontiers by filtering out frontiers unreachable from the robot on
the current occupancy map.

Local Navigation The local navigation module uses the Fast Marching Method
[38] to plan a path to the selected frontier from the current location on the
occupancy map. It takes deterministic actions (move forward by 25cm, turn
left/right by 30 degrees) to reach the center of the frontier. We update the
map after each step and replan.

Computing Expected Cost Eq. (4) is evaluated recursively. To reduce the
computation complexity, we select the six frontiers with the most significant
area $R_{A}$ to define the set of possible actions $\mathcal{A}_{t}$. Since we
need $D_{\text{in}}$ to reveal the whole area beyond a frontier, if we don’t
have enough timesteps left ($\sigma=\sigma_{t}-D_{\text{k}}$), we only get a
portion of the full reward and our reward structure looks like the following:

$\begin{split}R_{A}(&m_{t},a_{t},\sigma)=R_{A}(m_{t},a_{t})\cdot\begin{cases}1&\text{if
$\sigma>D_{\text{in}}$}\\\ \frac{\sigma}{D_{\text{in}}}&\text{else}\\\
\end{cases}\end{split}$ (5)

where $R_{A}(m_{t},a_{t})$ and $D_{\text{in}}$ are estimated by the learning
modules. To save computation, we compute $D_{\text{k}}$, $D_{\text{in}}$ and
$D_{\text{out}}$ as geometric distances on the occupancy grid. To account for
the dynamics of the agent navigating on a continuous space, we compute the
ratio between geometric distance and timesteps by approximating the ratio
between turning actions and moving forward steps as a constant. In all
experiments, we use an empirically computed ratio of 1.7, computed from
offline navigation trials in training environments.

An illustration of the entire navigation system can be found in Fig. 3.

##  V Learning-Based Modules

###  V-A Estimating Properties of Unknown Space

We estimate the values of $R_{A}$, $D_{\text{in}}$ and $D_{\text{out}}$ in Eq.
(4) using a neural network. We generate the training values of $R_{A}$ for a
frontier by computing the number of unobserved navigable cells on the current
map beyond each frontier. Given a partially observed map, we compute the
neighboring unexplored area beyond each frontier via connected components and
count the free cells within each region as $R_{A}$.

We compute the training values of $D_{\text{in}}$ and $D_{\text{out}}$ for a
frontier on the skeleton of the occupancy map. For skeleton computation, we
use the approach described in [39] that iteratively removes boundary pixels
while preserving topological structure. We generate the skeleton on the free
space of the ground-truth occupancy map. We compute the subgraph of the
skeleton for each frontier by taking nodes located in the frontier’s
neighbouring connected component. We run a Travelling Salesman Problem solver
on the subgraph to get the trajectory length of visiting every node once and
returning to the frontier. We divide the entire travelling cost into two
parts: $D_{\text{in}}$ is the travelling cost of visiting every node once and
then stopping and $D_{\text{out}}$ is the cost of travelling back to the
initial node from where $D_{\text{in}}$ finishes.

###  V-B Training Data Generation

Since we learn the properties of the unknown space beyond each frontier, we
require training data generated from a large number of environments. We first
build ground-truth occupancy and semantic maps for each environment. We use an
agent with a panoramic sensor running classical frontier-based exploration on
these environments. As the agent explores, revealing new portions of the
environment, we extract the newly updated frontiers and compute their
properties $R_{A}$, $D_{\text{in}}$ and $D_{\text{out}}$ as described in the
previous section. We also record the currently observed map $m_{t}$ and
extract the egocentric observations $o_{t}$ corresponding to each frontier
from the panorama. We collect data tuple
$(m_{t},o_{t},R_{A},D_{\text{in}},D_{\text{out}})$ for training the learning
modules.

###  V-C Learning Module Architecture

We build two learning models given different input representations, as shown
by Fig. 3. We use a UNet [40] architecture that inputs the current observed
map $m_{t}$ and outputs a same-sized map with three channels. Each channel of
the output map represents the predictions of $R_{A}$, $D_{\text{in}}$ and
$D_{\text{out}}$ at each location. The loss is computed only on the frontier
locations as other areas have no values. During testing stage, we average the
predictions at the frontier pixels to get the estimated properties. We train
another ResNet18 model that takes in the egocentric observations $o_{t}$ (a
depth image and a semantic segmentation image) and outputs three values as
predictions of $R_{A}$, $D_{\text{in}}$ and $D_{\text{out}}$. We train these
two models through supervised learning with L1-loss.

##  VI Experiments

Datasets + Simulated Environments We use the Habitat [14] simulator with the
Matterport (MP3D) [8] dataset for our experiments. MP3D consists of 90 scenes
that are 3D reconstructions of real-world environments, and semantic
segmentations (40 semantic categories) of egocentric views are available. We
split the MP3D scenes into train:val:test following the standard 61:11:18
ratio to train the perception models and evaluate the explorers. We split the
18 test scenes into small (size below $200m^{2}$), medium (size between
$200m^{2}$ and $500m^{2}$) and large (size over $500m^{2}$) when reporting the
results to better understand the advantages of our approach as the environment
becomes larger. We evaluate different exploration approaches using the
standard 1000 testing episodes generated for the Habitat PointGoal Challenge
[14]. For our experiments, noise-free poses and ground-truth egocentric
semantic segmentations are provided by the simulator.

Task Setup The robot observation space consists of RGB, depth and semantic
segmentation images. As our approach’s performance largely depends on the FoV
of the sensors, we evaluate our methods with two types of sensors: (i) a
panoramic $360^{\circ}$ FoV sensor with observations of $256\times 1024$
resolution. (ii) an egocentric $90^{\circ}$ FoV sensor with observations of
$256\times 256$ resolution. The action space contains four short-term actions:
MOVE_FORWARD by 25cm, TURN_LEFT and TURN_RIGHT by $30^{\circ}$ and STOP.

We evaluate the explorers with the Coverage (Cov) metric: the percentage of
explored space over the entire environment within the allowed 500 steps.

Implementation Details We use the same mapping, frontier computation and local
navigation implementations for all the baseline methods and our methods. For
training the learning modules, we pre-compute 220,000 train/ 1,000 val data
tuples. We train the model using PyTorch for six epochs and use the Adam
optimizer with a learning rate of 0.001 decayed by a factor of 10 after every
two epochs. Training is done with two A100 GPUs within 12 hours. A single step
in a navigation episode requires 0.9s on average including mapping, Bellman
Equation computation, and local navigation. The timing was performed on a
server using Xeon CPU @ 2.30GHz and an A100 GPU.

Figure 4:  A comparison between the coverage of our LFE-UNet explorer and the
PONI-Greedy baseline on the test scenes. On a large scene (top row), LFE-UNet
outperforms the Greedy baseline by $17\%$ on coverage after 500 steps. On a
small scene (bottom row), LFE-UNet finishes exploring the entire environment
with 73 fewer steps than the Greedy baseline.

###  VI-A Exploration with a Panoramic $360^{\circ}$ FoV Sensor

We evaluate the explorers’ performance when the robot’s observation is a
panorama. We compare our method with the following three baselines:
Frontier-Based Exploration (FBE-Near) [10]: A non-learned greedy algorithm.
The agent always navigates to the nearest frontier.
Active Neural SLAM (ANS+FBE) [6]: A learned exploration model trained with
model-free Deep RL using coverage as rewards. We use this as a comparison
between model-free and model-based planning. It has a global policy module for
selecting long-term goals given the currently observed occupancy map and
agent-visited location history as input. We use our self-built map as input to
their global policy with their trained weights to get the long-term goals and
select the frontier nearest to the long-term goal for robustness consideration
and navigate towards it. We follow the original paper’s setup by estimating a
long-term goal every 25 steps or after the robot reaches the frontier.
PONI-Greedy [4]: PONI is a recent approach to frontier-based exploration that
navigates greedily to the frontier with the maximum spatial and semantic
potentials. We adapt our implementation for this baseline by only using the
learned $R_{A}$ of UNet, and the agent always chooses the frontier with the
maximum estimated area.

TABLE I: coverage($\%$) of exploration methods with a panoramic $360^{\circ}$ FOV Sensor (higher is better) Dataset(Scenes) | Small | Medium | Large | Overall
---|---|---|---|---
FBE-Near [10] | 92.7 | 74.8 | 61.0 | 76.8
ANS+FBE [6] | 92.7 | 63.4 | 60.2 | 72.3
PONI-Greedy [4] | 95.8 | 72.3 | 56.7 | 74.4
LFE-UNet (Ours) | 97.4 | 75.1 | 70.9 | 80.7
LFE-View (Ours) | 92.7 | 67.3 | 60.9 | 73.3
Ablations: |  |  |  |
LFE-($R^{o},D^{o}$) | 95.2 | 84.0 | 79.5 | 86.2
Greedy-($R^{o}$) | 90.1 | 57.3 | 35.1 | 61.7

We show these results in Table I. Our LFE-UNet model outperforms all the other
approaches. We show example trajectories of LFE-UNet and PONI-Greedy in Fig.
4. LFE-View performs slightly worse, indicating that maps are more effective
input representations to learn the frontier properties than the views.

###  VI-B Exploration with an Egocentric $90^{\circ}$ FoV Sensor

We evaluate the same set of approaches as in Sec. VI-A with a $90^{\circ}$ FoV
sensor, as this is a more common setup in recent exploration papers [1, 3].
Even though direct observations of many frontiers are missing, the learning-
based approaches can still estimate the properties of the frontiers from the
maintained map. Owing to the limited field of view for these trials, we do not
evaluate with the ResNet18 (the LFE-View planner), which requires a panoramic
view that can see all frontiers. Moreover, many recent exploration approaches
[3, 35] are more focused on building an accurate map from a learned map
module, something we compute directly from RGB-D and semantic information; as
such, direct comparison with these approaches was not made.

The results are reported in Table II. While the relative advantage of our
method is reduced with sensors of limited FoV, our method still achieves the
best overall coverage compared to the other methods.

###  VI-C Ablation Study

We study the impact of our learning modules, which estimate $R_{A}$,
$D_{\text{in}}$, and $D_{\text{out}}$, by endowing the learning-augmented
models with an oracle, which produces the true values of the area $R^{o}_{A}$
and distances $D^{o}=\\{D^{o}_{\text{in}},D^{o}_{\text{out}}\\}$ beyond the
frontiers. These models are viewed as an upper performance bound for our
approach. We use the $360^{\circ}$ FoV sensor in this experiment.

The results are included in the last two rows of Table I and show that overall
our oracle-informed model LFE-($R^{o}$, $D^{o}$) outperforms the other
approaches and an oracle-informed greedy planner Greedy-($R^{o}$). This
empirically shows that there is still room for improvement in our learning
module. The oracle model can explore the large scenes $18.5\%$ more
efficiently than the FBE-Near approach. This is useful in real-world
applications when a robot is sent to review the environment with a previously-
built map.

The performance of the greedy approach drops with the known values since the
true value of the area beyond multiple frontiers may exactly match, causing
numerical instability during selection. Thus the greedy heuristic oscillates
between these frontiers, delaying exploration progress. We found similar
oscillations with the ANS+FBE [6] approach, also trained with a greedy
strategy. This oscillation problem is primarily reduced by using the learning
modules because the prediction is imperfect, so two frontiers are unlikely to
have the same predicted values. Our approach avoids this unstable behavior
through non-myopic planning and incorporating the time to travel to nearby
frontiers.

TABLE II: coverage ($\%$) of exploration methods with a $90^{\circ}$ fov sensor Dataset(Scenes) | Small | Medium | Large | Overall
---|---|---|---|---
FBE-Near [10] | 84.0 | 60.3 | 51.3 | 65.2
ANS+FBE [6] | 93.0 | 47.9 | 44.4 | 61.1
PONI-Greedy [4] | 93.8 | 63.6 | 47.5 | 68.8
LFE-UNet (Ours) | 96.9 | 63.5 | 53.3 | 70.9

###  VI-D Learning Module Evaluation

Here we study whether semantics helps with estimating the frontier properties.
We evaluate the performance of UNet by varying the input representations. We
tried two types of input: one with only an occupancy map versus having both an
occupancy and a semantic map. We trained another UNet model using the same
training setup but only the occupancy map as input. We compute the mean
absolute error (MAE) of $R_{A}$, $D_{\text{in}}$ and $D_{\text{out}}$
altogether over all the frontiers in the testing examples. With only the
occupancy map as input, UNet achieves 156.8 MAE error while the MAE error is
140.0 with the semantic map. This matches our intuition that having semantics
in the perception system helps the learning module better estimate frontier
properties of the indoor scenes.

##  VII Conclusion

We introduce a novel model-based approach for the time-limited exploration
task in previously unseen environments. Our approach augments classical
frontier-based exploration with predictions about unseen space using learning.
Exploiting advances in visual sensing, semantic mapping and availability of
large datasets of 3D models of real indoor scenes, we showed how to use a deep
convolutional neural network to learn two informative properties of each
frontier: (i) the navigable area in the unknown space beyond a frontier and
(ii) the timestep cost of exploring this unknown region and coming back. We
tried two input alternatives to the learning modules: (i) top-down-view
occupancy and semantic map and (ii) egocentric depth and semantic segmentation
views. Our exploration experiments show that our method is $2.1\%$ more
efficient in covering unknown areas than greedy strategies and $8.4\%$ more
than myopic RL approaches.

## References

  * [1] S. K. Ramakrishnan, D. Jayaraman, and K. Grauman, “An exploration of embodied visual exploration,” _International Journal of Computer Vision_ , vol. 129, pp. 1616 – 1649, 2020.
  * [2] D. S. Chaplot, R. Salakhutdinov, A. K. Gupta, and S. Gupta, “Neural topological SLAM for visual navigation,” _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2020.
  * [3] G. Georgakis, B. Bucher, A. Arapin, K. Schmeckpeper, N. Matni, and K. Daniilidis, “Uncertainty-driven planner for exploration and navigation,” _2022 International Conference on Robotics and Automation (ICRA)_ , pp. 11 295–11 302, 2022.
  * [4] S. K. Ramakrishnan, D. S. Chaplot, Z. Al-Halah, J. Malik, and K. Grauman, “PONI: Potential functions for ObjectGoal navigation with interaction-free learning,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022.
  * [5] C. Bradley, A. Pacheck, G. J. Stein, S. Castro, H. Kress-Gazit, and N. Roy, “Learning and planning for temporally extended tasks in unknown environments,” in _2021 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2021, pp. 4830–4836.
  * [6] D. S. Chaplot, D. Gandhi, S. Gupta, A. Gupta, and R. Salakhutdinov, “Learning to explore using active neural mapping,” in _International Conference on Learning Representations_ , 2020.
  * [7] T. Chen, S. Gupta, and A. Gupta, “Learning exploration policies for navigation,” in _International Conference on Learning Representations_ , 2019\.
  * [8] A. X. Chang, A. Dai, T. A. Funkhouser, M. Halber, M. Nießner, M. Savva, S. Song, A. Zeng, and Y. Zhang, “Matterport3D: Learning from RGB-D data in indoor environments,” _2017 International Conference on 3D Vision (3DV)_ , pp. 667–676, 2017.
  * [9] F. Xia, A. R. Zamir, Z. He, A. Sax, J. Malik, and S. Savarese, “Gibson Env: Real-world perception for embodied agents,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 9068–9079.
  * [10] B. Yamauchi, “A frontier-based approach for autonomous exploration,” in _Proceedings 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA’97.’Towards New Computational Principles for Robotics and Automation’_. IEEE, 1997.
  * [11] G. J. Stein, C. Bradley, and N. Roy, “Learning over subgoals for efficient navigation of structured, unknown environments,” in _Conference on Robot Learning_. PMLR, 2018, pp. 213–222.
  * [12] L. P. Kaelbling, M. L. Littman, and A. R. Cassandra, “Planning and acting in partially observable stochastic domains,” _Artificial Intelligence_ , vol. 101, no. 1-2, pp. 99–134, 1998.
  * [13] S. J. Russell, _Artificial intelligence a modern approach_. Pearson Education, Inc., 2010.
  * [14] M. Savva, A. Kadian, O. Maksymets, Y. Zhao, E. Wijmans, B. Jain, J. Straub, J. Liu, V. Koltun, J. Malik _et al._ , “Habitat: A platform for embodied AI research,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 9339–9347.
  * [15] M. Deitke, D. Batra, Y. Bisk, T. Campari, A. X. Chang, D. S. Chaplot, C. Chen, C. P. D’Arpino, K. Ehsani, A. Farhadi _et al._ , “Retrospectives on the embodied AI workshop,” _arXiv preprint arXiv:2210.06849_ , 2022.
  * [16] A. Szot, A. Clegg, E. Undersander, E. Wijmans, Y. Zhao, J. Turner, N. Maestre, M. Mukadam, D. Chaplot, O. Maksymets, A. Gokaslan, V. Vondrus, S. Dharur, F. Meier, W. Galuba, A. Chang, Z. Kira, V. Koltun, J. Malik, M. Savva, and D. Batra, “Habitat 2.0: Training home assistants to rearrange their habitat,” in _Advances in Neural Information Processing Systems (NeurIPS)_ , 2021.
  * [17] M. Savva, A. Kadian, O. Maksymets, Y. Zhao, E. Wijmans, B. Jain, J. Straub, J. Liu, V. Koltun, J. Malik, D. Parikh, and D. Batra, “Habitat: A Platform for Embodied AI Research,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019.
  * [18] E. Kolve, R. Mottaghi, W. Han, E. VanderBilt, L. Weihs, A. Herrasti, D. Gordon, Y. Zhu, A. Gupta, and A. Farhadi, “AI2-THOR: An interactive 3D environment for visual AI,” _arXiv preprint arXiv:1712.05474_ , 2017.
  * [19] S. K. Ramakrishnan, A. Gokaslan, E. Wijmans, O. Maksymets, A. Clegg, J. M. Turner, E. Undersander, W. Galuba, A. Westbury, A. X. Chang, M. Savva, Y. Zhao, and D. Batra, “Habitat-Matterport 3D dataset (HM3d): 1000 large-scale 3D environments for embodied AI,” in _Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track_ , 2021. [Online]. Available: https://arxiv.org/abs/2109.08238
  * [20] K. Yadav, R. Ramrakhya, S. K. Ramakrishnan, T. Gervet, J. Turner, A. Gokaslan, N. Maestre, A. X. Chang, D. Batra, M. Savva _et al._ , “Habitat-Matterport 3D semantics dataset,” _arXiv preprint arXiv:2210.05633_ , 2022.
  * [21] E. Wijmans, A. Kadian, A. Morcos, S. Lee, I. Essa, D. Parikh, M. Savva, and D. Batra, “DD-PPO: Learning near-perfect PointGoal navigators from 2.5 billion frames,” in _International Conference on Learning Representations_ , 2019.
  * [22] R. Partsey, E. Wijmans, N. Yokoyama, O. Dobosevych, D. Batra, and O. Maksymets, “Is mapping necessary for realistic PointGoal navigation?” _2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 17 211–17 220, 2022.
  * [23] Y. Li, A. Debnath, G. J. Stein, and J. Kosecka, “Comparison of model free and model-based learning-informed planning for PointGoal navigation,” in _CoRL 2022 Workshop on Learning, Perception, and Abstraction for Long-Horizon Planning_ , 2022. [Online]. Available: https://openreview.net/forum?id=2s92OhjT4L
  * [24] A. Pal, Y. Qiu, and H. Christensen, “Learning hierarchical relationships for object-goal navigation,” in _Conference on Robot Learning_. PMLR, 2021, pp. 517–528.
  * [25] D. S. Chaplot, D. P. Gandhi, A. Gupta, and R. R. Salakhutdinov, “Object goal navigation using goal-oriented semantic exploration,” _Advances in Neural Information Processing Systems_ , vol. 33, 2020.
  * [26] N. Savinov, A. Dosovitskiy, and V. Koltun, “Semi-parametric topological memory for navigation,” _arXiv preprint arXiv:1803.00653_ , 2018.
  * [27] M. Hahn, D. S. Chaplot, and S. Tulsiani, “No RL, no simulation: Learning to navigate without navigating,” in _Neural Information Processing Systems_ , 2021.
  * [28] Y. Li and J. Košecka, “Learning view and target invariant visual servoing for navigation,” in _2020 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2020, pp. 658–664.
  * [29] J. Krantz, A. Gokaslan, D. Batra, S. Lee, and O. Maksymets, “Waypoint models for instruction-guided navigation in continuous environments,” _2021 IEEE/CVF International Conference on Computer Vision (ICCV)_ , pp. 15 142–15 151, 2021.
  * [30] J. Krantz, S. Banerjee, W. Zhu, J. Corso, P. Anderson, S. Lee, and J. Thomason, “Iterative vision-and-language navigation,” _arXiv preprint arXiv:2210.03087_ , 2022.
  * [31] D. Batra, A. X. Chang, S. Chernova, A. J. Davison, J. Deng, V. Koltun, S. Levine, J. Malik, I. Mordatch, R. Mottaghi _et al._ , “Rearrangement: A challenge for embodied AI,” _arXiv preprint arXiv:2011.01975_ , 2020\.
  * [32] G. Sarch, Z. Fang, A. W. Harley, P. Schydlo, M. J. Tarr, S. Gupta, and K. Fragkiadaki, “TIDEE: Tidying up novel rooms using visuo-semantic commonsense priors,” in _European Conference on Computer Vision_ , 2022.
  * [33] J. Gu, D. S. Chaplot, H. Su, and J. Malik, “Multi-skill mobile manipulation for object rearrangement,” in _Deep Reinforcement Learning Workshop NeurIPS 2022_ , 2022.
  * [34] K. Fang, A. Toshev, L. Fei-Fei, and S. Savarese, “Scene memory transformer for embodied agents in long-horizon tasks,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 538–547.
  * [35] S. K. Ramakrishnan, Z. Al-Halah, and K. Grauman, “Occupancy anticipation for efficient exploration and navigation,” in _European Conference on Computer Vision_. Springer, 2020, pp. 400–418.
  * [36] C. Chen, S. Majumder, Z. Al-Halah, R. Gao, S. K. Ramakrishnan, and K. Grauman, “Learning to set waypoints for audio-visual navigation,” in _International Conference on Learning Representations_ , 2020.
  * [37] J. Pineau and S. Thrun, “An integrated approach to hierarchy and abstraction for POMDPs,” Carnegie Mellon University, Pittsburgh, PA, Tech. Rep. CMU-RI-TR-02-21, August 2002.
  * [38] J. A. Sethian, “A fast marching level set method for monotonically advancing fronts.” _Proceedings of the National Academy of Sciences_ , vol. 93, no. 4, pp. 1591–1595, 1996.
  * [39] T. Y. Zhang and C. Y. Suen, “A fast parallel algorithm for thinning digital patterns,” _Commun. ACM_ , vol. 27, pp. 236–239, 1984.
  * [40] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” in _International Conference on Medical image computing and computer-assisted intervention_. Springer, 2015, pp. 234–241.


# #RoeOverturned: Twitter Dataset on the Abortion Rights Controversy

Rong-Ching Chang∗ ♠, Ashwin Rao∗ ♥,♣, Qiankun Zhong∗ ♠, Magdalena Wojcieszak♠,
Kristina Lerman ♥,♣

###### Abstract

On June 24, 2022, the United States Supreme Court overturned landmark rulings
made in its 1973 verdict in Roe v. Wade. The justices by way of a majority
vote in Dobbs v. Jackson Women’s Health Organization, decided that abortion
wasn’t a constitutional right and returned the issue of abortion to the
elected representatives. This decision triggered multiple protests and debates
across the US, especially in the context of the midterm elections in November
2022. Given that many citizens use social media platforms to express their
views and mobilize for collective action, and given that online debate
provides tangible effects on public opinion, political participation, news
media coverage, and the political decision-making, it is crucial to understand
online discussions surrounding this topic. Toward this end, we present the
first large-scale Twitter dataset collected on the abortion rights debate in
the United States. We present a set of $74M$ tweets systematically collected
over the course of one year from January 1, 2022 to January 6, 2023.

$*$$*$footnotetext: These authors contributed equally to this work.

## Introduction

The US Supreme Court’s 1973 decision in the Roe v. Wade case gave women the
right to terminate their pregnancy. Despite federal protection, abortion has
remained a highly polarized and politically charged issue in American society.
Studies published by Gallup (Saad 2010b, a) noted that views of Democrats and
Republicans on abortion rights had grown increasingly polarized since 1975 and
Americans continued to be divided along pro-choice and pro-life lines. A
follow up study in 2021 (Brenan 2010) showed that Americans continue to differ
on the morality of abortion with 47% of survey takers finding abortion to be
morally acceptable and 46% disagreeing. These findings suggest that abortion
has been one of the moralized political issues at the core of the so-called
culture wars in the United States. Previous studies largely seem to agree on
the highly polarizing nature of the abortion rights debate. (DiMaggio, Evans,
and Bryson 1996; Mouw and Sobel 2001; Evans 2003; Fiorina, Abrams et al. 2008;
Abramowitz and Saunders 2008). Although some scholars (e.g. (Fiorina, Abrams
et al. 2008)) suggest that the general public’s policy preferences are mostly
centrist toward wedge-issues including abortion, or argue that the extent of
polarization in the US society is exaggerated (Mouw and Sobel 2001), other
evidence suggests different patterns. For instance, analyzing over 20 years of
data from the National Election Studies (NES) and General Social Survey (GSS)
(DiMaggio, Evans, and Bryson 1996) found that polarization on the issue of
abortion had increased in the US. Similarly, leveraging NES and GSS data from
1970-2003 (Evans 2003) found unambiguous evidence of polarization with regards
to abortion rights Public opinion data aside, the National Abortion Federation
found that (NAF 2022) violent crimes and assaults directed towards abortion
clinics and patients rose by a staggering $128\%$ from 54 cases in 2020 to 123
cases in 2021.

The June 24, 2022 United States Supreme Court ruling in Dobbs v. Jackson
Women’s Health Organization struck down decisions made in Roe v. Wade (1973)
and Casey v. Planned Parenthood (1992), removing federal protections for
abortion and moving power to regulate it to the states.

In delivering the opinion of the court, Justice Samuel Alito noted that
abortion presents a profound moral issue on which Americans hold sharply
contrasting views (SCOTUS 2022). Given that states had enacted trigger laws
111A trigger law is a law that is unenforceable but may achieve enforceability
if a key change in circumstances occurs.
https://en.wikipedia.org/wiki/Trigger˙law to go into effect once the Supreme
Court overturned Roe v. Wade, abortion at the time of writing this article is
banned in 13 states with or without exceptions for rape and incest. Other
states have either completely banned abortion or provide limited access with
gestational limits. Only 16 states in the US have legislation that protects
access to abortion. Contrasting laws in different states of the country
indicate the extent of the disagreement with respect to abortion rights.

As the ruling leaked, protesters both in favor (pro-choice) and opposition
(pro-life) of the Roe v. Wade ruling of 1973 gathered on the streets and
mobilized on social media. Slogans like “Bans Off Our Bodies” (Pro-Choice),
“Abortion is a Right” (Pro-Choice), “Equal rights for the unborn” (Pro-life)
and “Time to Reverse Roe” (Pro-life), echoed on the streets and online,
reflecting a deeply polarized populace. In fact, many of these slogans,
expressions, and discussions took place on social media, especially on
Twitter, as we detail below. The easy availability of those data, furthermore,
offers unprecedented insights to studying public opinion expression
surrounding contentious political issues.

The rise of social media platforms since the early 2000s has provided
researchers with a novel avenue to study public opinion, individual
expressions, and behavioral indicators. The advantage of social media datasets
over survey-based measurements used in public opinion polls is the ability to
feasibly assess issue positions of the larger public without biases inherent
in survey self-reports, especially of contentious political issues (e.g.,
social desirability bias). Large-scale social media datasets have also made
possible, the analysis of protest mobilization (Breuer, Landman, and Farquhar
2015; Steinert-Threlkeld 2017; Munn 2021), proliferation of misinformation
(Nikolov, Flammini, and Menczer 2020; Rao et al. 2021; Chen et al. 2021),
moral and emotional attitudes (Guo et al. 2022; Priniski et al. 2021), echo
chambers and ideological biases on platforms (Barberá 2015; Wojcieszak et al.
2022), among other democratically relevant phenomena. Previous studies (Yardi
and Boyd 2010; Garimella et al. 2018; Cinelli et al. 2021) have also analyzed
polarization with respect to wedge issues, including abortion rights in the
US. The collection of large-scale social media data at pivotal points like the
COVID-19 pandemic (Huang et al. 2020; Chen et al. 2020), 2020 US Presidential
elections (Chen, Deb, and Ferrara 2022; Abilov et al. 2021) and the Russian
invasion of Ukraine (Chen and Ferrara 2022) helps researchers capture the
pulse of an increasingly volatile world.

The bulk of this work focuses on Twitter not only due to the relative ease of
obtaining online behavioral data from the platform (relative to other
platforms, such as Facebook) but also because Twitter is uniquely suited to
analyzing discourse about political issues, given the platform’s important
role in American politics. Twitter is a key outlet for individuals to express
their political opinions and engage in political activities (BESTVATER et al.
2022). These tweets and engagement metrics are often treated as proxies of
public opinion by journalists and campaign strategists (McGregor 2019), set
political agendas that politicians follow (Barberá et al. 2019), and influence
what journalists and media report (Nelson and Tandoc Jr 2019). In addition,
Twitter is an important channel that citizens use to get political
information: 71% of adult Twitter users in the U.S. report getting news on the
site (Stocking, Barthel, and Grieco 2018), and is also a key platform for
elite expressions: almost all US politicians, pundits, and news media have a
Twitter account and politicians are more active and have more followers on
Twitter than on other platforms (Wojcieszak et al. 2022).

Furthermore, Twitter users are more politically interested than the general
population. In fact, Twitter stands out as one of the social media platforms
with the most politically engaged users: 42% of U.S. adults on Twitter say
they use the site to discuss politics at least some of the time, and this
percentage is higher among the most frequent tweeters (Wojcik and Hughes
2019). Furthermore, this politically active group of Twitter users is highly
influential. They produce the majority of engagements with elite tweets
(Wojcieszak et al. 2021) and political content on Twitter (BESTVATER et al.
2022), set political agendas (Barberá et al. 2019), and are more likely to
engage in political activities online and offline than the general public
(BESTVATER et al. 2022). In short, Twitter is well suited to examine the
prevalence and fluctuations of the discourse surrounding the overturning of
Roe v. Wade. As Twitter users are not representative of the general population
(Wojcik and Hughes 2019), the tweets we collect do not necessarily represent
public opinion on abortion. They however, reflect the discourse of the
politically inclined and societally influential citizens.

Our dataset makes possible the exploration of a variety of research
directions. Some interesting avenues include but are not limited to - modeling
opinion dynamics and polarization, protest mobilization, assessing emotions
and moral attitudes and the influence of social bots. When abortion was first
outlawed in the mid to late 1800s, abortion was stigmatized for close to a
century. Given that hate speech and toxic content thrive on social media
platforms (Founta et al. 2018), an analysis of such discourse becomes
imperative to prevent abortion from being stigmatized again.

In this study, we collect a large-scale Twitter data set consisting of
discourse surrounding abortion rights in the United States and the recent
Supreme Court verdict that overruled its 1973 Roe v. Wade ruling. Data
collection is ongoing at the time of writing this article. In the following
sections, we will describe data collection methods, descriptive statistics,
and information on how to access and use the data.

## Methods

### Overview

Prior to collecting the tweets, we identified a set of keywords that are meant
to represent the breadth and the different sides of the discourse surrounding
abortion in the United States. We followed a three-pronged data collection
strategy, discussed below. We began collecting tweets on June 25, 2022. In
order to collect data prior to June 25, 2022, we leverage Twitter Academic
API’s Full-Archive Search 222https://developer.twitter.com/en/docs/twitter-
api/tweets/search/api-reference/get-tweets-search-all. This gives us data from
January 1, 2022 to June 24, 2022. We gather tweets in real-time from June 25,
2022 to January 6, 2023. To ensure our dataset coverage, we use the count end
point from the Twitter API to compare the number of tweets in our collection
versus the number of tweets identified on Twitter. We use the Full-Archive
Search endpoint to recollect the date if the total number of tweets per day in
our dataset is less than the number of tweets identified on Twitter. Our data
coverage is from January 1, 2022 to January 6, 2023. Data collection is
ongoing at the time of writing this article. We restrict our collection to
English tweets. In all, we collected over $74M$ tweets shared by roughly $10M$
users.

Category |  Keywords
---|---
Neutral |  #Abortion, #roevswade, #Roeverturned, #roevwade, abortion, roe vs wade, roe v wade, roe overturned
Pro-choice |  #roevwadeprotest, roe v wade protest, pro choice, pro-choice, #prochoice, #forcedbirth, forced birth, #AbortionRightsAreHumanRights, abortion rights Are Human Rights, #MyBodyMyChoice, My Body My Choice, #AbortionisHealthcare, abortion is healthcare, AbortionIsAHumanRight, abortion is a human right, ReproductiveHealth, Reproductive Health, AbortionRights, abortion rights,
Pro-life |  #prolife, pro life, pro-life, #EndAbortion, end abortion, #AbortionIsMurder, Abortion Is Murder, #LifeIsAHumanRight, Life Is A Human Right, #ChooseLife, Choose Life, #SaveTheBabyHumans, Save The Baby Humans, #ValueLife, Value Life, #RescueThePreborn, Rescue The Preborn, #EndRoeVWade, End Roe V Wade, #LifeMatters, Life Matters, #MakeAbortionUnthinkable, make abortion unthinkable, #LiveActionAmbassador, Live Action Ambassador, Abortion Is Not A Right, #AbortionIsNotARight, #AbortionIsRacist, Abortion is racist, anti life, #antilife
Table 1: List of keywords

### Keyword collection strategy

The Full-Archive Search endpoint of Twitter Academic API requires us to
specify a set of keywords, on the basis of which the API returns relevant
tweets. To this end, we conducted an exploratory collection to compile an
exhaustive list of relevant phrases and hashtags to create a keyword set that
covers abortion-focused discussions from different political stances.
Henceforth, we will use the term keywords to refer to both phrases and
hashtags. We then build a systematic data collection pipeline, which we will
discuss in the following subsections.

#### Hashtag sampling

We first identified a set of seed hashtags - #roevwade, #prolife, #prochoice.
The #prolife hashtag is used more commonly by individuals and activists who
believe that human life begins at conception and abortion ends the life of an
innocent being. The #prochoice hashtag on the other hand, is used more
frequently by individuals who believe that it is a woman’s choice whether or
not to get an abortion and any regulation of abortion infringes their freedom.
The #roevwade is one hashtag that is used by individuals on both sides and is
neutral per se.

Leveraging these seed hashtags, we conducted an exploratory data collection
using the Twitter API’s recent tweet lookup endpoint
333https://developer.twitter.com/en/docs/twitter-api/tweets/search/api-
reference/get-tweets-search-recent. The purpose of the exploratory data
collection was to identify other hashtags that frequently co-occur with the
seed hashtags. We used snowball sampling to identify hashtags that frequently
co-occurred with #prochoice, #prolife and #roevswade.

From tweets that contain #prochoice we identify the hashtags -
#roevwadeprotest, #prochoice, #forcedbirth, #AbortionRightsAreHumanRights,
#MyBodyMyChoice, #abortionishealthcare, #abortionisahumanright,
#ReproductiveHealth, and #abortionrights to frequently co-occur. These form
the set of pro-choice hashtags. From tweets that contained #prolife, we find
#EndAbortion, #AbortionIsMurder, #LifeIsAHumanRight, #ChooseLife. #valueLife,
#rescuethepreborn, #endroevwade, #lifematters, #antilife,
#makeabortionunthinkable, #liveActionAmbassador,#catholic,
#AbortionIsNotARight, and #Abortionisracist to co-occur frequently. These
hashtags form the set of pro-life hashtags. As #Abortion, #Roeverturned and
#roewade frequently co-occured with #roevswade they form the set of neutral
hashtags.

#### Creating keywords

Given that not all users on Twitter use hashtags in their tweets, we create a
list of keywords based on the aforementioned hashtags by simply removing #
from the hashtags. For hashtags that comprised multiple words, we break them
down to reconstruct the phrases (e.g., #makeabortionunthinable into ”make
abortion unthinkable”). The final set of hashtags and phrases is shown in
Table 1. In all we have 58 keywords comprised of 31 hashtags and 30 phrases.
The keywords shown in Table 1 form the search terms in our query. More
specifically, our search query is comprised of these keywords connected
together by the ’OR’ operator, which tells the search endpoint to return
tweets that have at least one keyword in it. The search is not case-sensitive.

#### Validation and removal

Given that the Roe v. Wade overturns was US-specific (although it was covered
internationally and generated global debates) and the most frequently used
keywords are all in English, we do not collect non-English tweets 444According
to our collection of all language from June to September 2022, over 90% of the
tweets on this topic is English. We thus removed the non-English tweets and
collect only English tweets in this dataset..

Figure 1: Daily Tweet Activity: Number of original tweets, retweets, and
replies over time. Notable events are highlighted in the timeline

## Descriptive Analysis

### Tweet Statistics

Tweets | Retweets | Replies | Quoted Tweets
---|---|---|---
21,688,663 | 42,332,378 | 8,328,364 | 1,131,301
Table 2: Tweet Statistics

We identify original tweets, retweets, quote tweets, and replies in our
dataset. If the tweet was a retweet, quote tweet, or a reply, the tweet object
lists the ID of the user referenced in the interaction through “retweeted user
id”, “quoted user id” and “in reply to user id” fields respectively. The
number of tweets, retweets, replies, and quote tweets in the dataset are shown
in Table 2. In Fig 1, we show the time series of the total number of original
tweets, retweets, and replies. Since quote tweets are low in number, they are
not depicted in Fig 1 to reduce clutter. We also highlight crucial events
relating to Roe v. Wade as vertical lines. The absolute volume of tweets using
our keywords has an abrupt increase on May 2nd, 2022, when a draft decision of
the Supreme Court was leaked to the press. Similarly, we also see a spike in
tweets on June 24th, 2022 when the Supreme Court officially overturned its
1973 ruling in Roe v. Wade (Refer Fig 1. During the midterm elections in
November 2022, we also see a spike in reply counts, hinting at an increased
discussion about abortion rights.

(a) Top-20 hashtags shared (b) Hashtag use over time

Figure 2: (a) Top-20 hashtags shared by users in our dataset. (b) Usage of
Pro-life, Pro-Choice and Neutral hashtags over time, including hashtags that
co-occur with #prolife, #prochoice and #roevwade topics respectively. Refer to
hashtags in Table 1. Notable events are highlighted in the timeline. Figure 3:
Clustering in hashtag co-occurrence network. The pink cluster shows prominent
pro-choice hashtags whereas, the green cluster highlights pro-life hashtags.

### Hashtags

Each tweet object consists of an entities field, which lists the hashtags used
in the tweet. In Fig 2(a), we show that among the top 20 tweeted hashtags,
most are directly related to the abortion rights debate in the US. The
prominent usage of hashtags like #roevswade, #prolife, #roeoverturned,
#prochoice, #abortionishealthcare, #freshwords, and #mybodymychoice reflect
both sides of the debate. We also show the usage of pro-choice, pro-life, and
neutral hashtags (Refer Table 1) in Fig 2(b). We plot this time series using
the hashtags in our keyword set in Table 1. Along expected lines, neutral
hashtags like are used the most. These hashtags can co-occur prominently with
both sides of the abortion rights debate. Pro-choice hashtags largely remain
more prominent than pro-life hashtags, with both groups seeing significant
spikes around May 2nd, 2022 and June 24th, 2022.

Generally, hashtags are used to highlight the content of tweets. In some
cases, users prefer to use multiple hashtags related to one another. We
identify pairs of hashtags that appear together in tweets to build a hashtag
co-occurrence network. To isolate related hashtags, we cluster them by
applying the Label Propagation community detection algorithm (Raghavan,
Albert, and Kumara 2007). Python-igraph’s implementation of the algorithm
(Csardi and Nepusz 2006) allows us to specify a seed set of hashtags which we
can specify have the community (pro-life/pro-choice) they belong to based on
our domain-specific knowledge. We use the set of 28 pro-choice and pro-life
hashtags in Table 1 as the seed set. Given the small numbers, we avoid
including neutral hashtags in the seed set.

For better visualization, we only consider edges that connect at least one
seed node. We also ignore infrequently co-occurring hashtags by setting a
threshold of 500 on minimum co-occurrences. Clusters obtained through Label
Propagation are shown in Fig 3. The pink cluster shows prominent pro-choice
hashtags, and the green cluster highlights the pro-life hashtags. Hashtags
such as #prochoiceisprolife, #saveroevwade frequently co-occur with pro-choice
hashtags while, #children and #babieslivesmatter appear with pro-life
hashtags. Given that neutral hashtags such as #roevswade and #scotus co-occurr
more frequently with pro-choice hashtags, they get clustered into the pro-
choice group. While Fig 3 shows that prochoice hashtags seemingly dwarf pro-
life hashtags, it is worth noting that in Fig 2(a), #prolife appears more
commonly than #prochoice in our dataset. The smaller pro-life cluster could be
because other pro-life hashtags aren’t quite as popular, whereas several other
pro-choice hashtags are prominent (Refer Fig 2(a)). Additionally, Twitter
being more popular among liberals (Pew 2022) could potentially be another
factor.

(a) Top-20 domains shared (b) Domains: Ideology vs Reliability

Figure 4:  (a) Top-20 domains shared by users. (b) Top-20 domains with
ideology and reliability scores from Media Bias - Ad Fontes.

### Domains

The entities field in tweet objects includes lists of the URL embedded in the
tweet, if any. We use tldextract555https://pypi.org/project/tldextract/ Python
package to extract the pay-level domain of the URLs shared. In Fig 4(a), we
show the top 20 most commonly shared pay-level domains. Most of these domains
are prominent media sources and consequently have their political leaning. We
were able to identify 14 of the top 20 domains through Ad Fontes Media-Bias
(Media 2020) 666Ad Fontes Media-Bias Chart is a two-dimensional chart that
positions 100 media outlets in terms of their political ideologies and content
qualities. Ad Fontes evaluates media outlets based on ratings from 1,818
online articles and 98 cable news shows.(Heldebrandt 2019). The Ad Fontes
Media-Bias provides ideology score and content reliability score777On the Ad
Fontes ideology-slant classification, the ideology score ranges from -42 to
42, representing extreme left to extreme right political leaning. The 0 in
ideology score represents moderate or centrist political leaning. for domains.
Among the 14 domains in our dataset in Fig 4(b), 12 were left-leaning domains,
with an overall mean of $-4.61$. In addition, the majority (i.e., 12) of our
top domains have a reliability score higher than 30, with the reliability or
news value score ranging from 0 to 64, representing low to high reliability.

### Retweet Network

Given that we have over $44M$ retweet interactions, we can extract a retweet
network of users. From each of these $44M$ retweet interactions, we extract
user IDs of both the account making the retweet (user id) and the account
being retweeted (retweeted user id) and create a retweet network with $6.1M$
users and $27M$ unique interactions between them. Previous studies have
leveraged retweet networks to not only study its properties (Bild et al. 2015)
but also explore polarization and exposures (Conover et al. 2011; Rao,
Morstatter, and Lerman 2022).

Statistic |  Value
---|---
Nodes |  6,089,939
Edges |  26,925,519
Max. Indegree |  196,085
Max. Outdegree |  9,558
Density |  7.2 x $10^{-7}$
Table 3: RT Network Statistics

In Table 3, we list simple statistics to summarize the retweet network. The
most retweeted user has been retweeted for over $196K$ times and the user with
the highest number of retweets has $9.5K$ retweets. Retweet interactions are
usually not reciprocal. A small fraction of users gets retweeted most of the
time. This is reflected by the network density statistic in Table 3. The log-
log plot of degree distributions in Fig 5 shows that the in-degree and out-
degree distributions of the retweet network follow a power law.

## Release and Access

Our dataset is publicly available for download on Harvard Dataverse.
888https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/STU0J5.
The repository consists of Comma-Separated Value (CSV) files of tweet IDs
separated by date. In order to extract tweet objects from Twitter IDs, users
of this dataset can rely on third-party re-hydration tools - Hydrator (DocNow
2020) or the Twitter API’s search endpoint (Twitter 2022b).

## Discussion

On May 2nd, 2022, a leaked draft decision of the Supreme Court in Dobbs v.
Jackson Women’s Health Organization was published by Politico (Gerstein and
Ward 2020). In this initial draft majority opinion, Supreme Court justices had
voted and opined in favor of overruling Roe v. Wade. This triggered a fierce
outcry from pro-choice individuals and groups. Social media platforms became
sites for contentious debate yet again. Pro-choice protests and mobilization
efforts were met with equal measure from pro-life activists and groups. On
June 24th, 2022, the Supreme Court officially ruled to set aside rulings made
in Roe v. Wade, thereby paving way for certain states to outlaw abortion.

In light of these events have triggered increased discussions, many of which
occurred on social media platforms. It is crucial to preserve the traces of
these discussions in order to allow researchers to study and understand the
implications of the ruling on online public opinion. Toward this end, we
release the first large-scale Twitter dataset comprising of over $73$ million
in tweets identified systematically with keywords from both sides of the
contentious debate: pro-abortion and anti-abortion, over the course of one
year. The broad coverage of our dataset makes the study of a wide variety of
research problems possible. The dataset has the potential and can benefit the
research community in several ways, we listed a few possible usage below.

#### Opinion Dynamics and Polarization

Our dataset can be used to study opinion dynamics and the spread of pro-life
and pro-choice communities online. For example, comparing the number of
retweets, the speed of the diffusion, retweet network structure, and how
different ego characters such as politicians or celebrities plays different
roles between pro-life and pro-choice communities. Given that we have a clear
stance on the topic of abortion, polarization and the development of echo
chambers between pro-life and pro-choice communities can be intriguing avenues
and could be studied using our dataset.

Figure 5: Indegree and Outdegree distribution in the Retweet Network

#### Protest Mobilization

While the role of social media in protest mobilization has been studied for
specific wedge issues or events (Ince, Rojas, and Davis 2017; Wolfsfeld,
Segev, and Sheafer 2013; Steinert-Threlkeld 2017; Freelon, McIlwain, and Clark
2018), studies comparing interactions around different protests are sparse
(Segerberg and Bennett 2011; Won, Steinert-Threlkeld, and Joo 2017). This
dataset can be used in conjunction with other datasets to study complex social
movements such as the Black Live Matter movement (Giorgi et al. 2022), and
capital riots (Kerchner and Wrubel 2021). For example, do these social
movements rely on certain agents’ call for action? Do discussion networks’
size and density correlate with the public opinions on the protests or the
media’s coverage of the protests? If we could quantify the magnitude of each
offline protest, could one predict it using online data? And what are the
crucial components in the online discussions that determine the success or
failure of an offline protest? Furthermore, this dataset facilitates the
examination of how certain public figures, politicians, pundits, and also non-
political actors, disseminate, support, or engage with different claims
related to Roe v. Wade and abortion more broadly. One could also
systematically examine the relative contribution of political elites (i.e.,
politicians, journalists, pundits, and news media organizations) versus the
“ordinary users” to the overall volume and nature of these online discussions.
Did ordinary citizens contribute more abortion-related tweets than
politicians? How did these patterns vary by political leaning of the citizens
and the politicians?

#### Emotion, Moral Attitudes, and Multi-Modalities

Different groups express different moral attitudes and emotions to specific
events. Multi-modal information in the form of text, images, embedded videos,
and hyperlinks can be extracted and analyzed to quantify them. Additionally,
one can attempt to quantify agreeableness and sarcasm. Some possible research
questions include - whether pro-choice or pro-life individuals express more
anger or other negative sentiments, whether certain political parties use more
propaganda techniques in abortion-related messages, and whether it correlated
with the midterm election outcome. Furthermore, one could ask how hate speech
toward woman stigmatize abortion and plays a role in abortion-related debates.

#### Bots and Misinformation

The rise of social bots has paved ways for malicious actors to infiltrate
political discourse and spread misinformation (Ferrara et al. 2016). Exploring
the influence of bot accounts in abortion rights-related discourse and the
subsequent harms posed by health-related misinformation is another interesting
area of research. Potential research questions include whether there was an
increase in health-related misinformation once Roe v. Wade was overturned,
what agenda was propagated through bots, and whether exposure to such
discourse can influence individual opinions.

Overall, this dataset provides an important source to study the dynamics of
online discourse around abortion rights in the US. In future work, we will use
computational methods to quantify the stances on abortion rights in the tweets
and measure the level of toxicity expressed in those stances.

## Limitations

Our data collection relied on a list of relevant keywords, which may not fully
capture the extent of the event. Additionally, the dataset only includes
English-language tweets, and may not capture multilingual discourse relating
to the topic. The data collection also relies on both the stream and archive
search endpoints of the Twitter API, and while the data covers every day in
2022, it may not include all tweets if they were deleted before collection.
Other potential factors that may impact the data collection include changes in
ownership at Twitter and subsequent changes to the Trust and Safety team,
which could affect the API’s functionality. Additionally, the dataset only
includes data from Twitter, which has been shown to be more popular among
liberals (Pew 2022). Despite these limitations, the keywords were chosen
systematically and the dataset’s validity has been demonstrated in its
statistics.

## Broader perspective and ethics

All of the data in this dataset is publicly available information. Our data
collection was deemed exempt from review by the Institutional Review Board
(IRB) at the University of California-Davis and University of Southern
California, as it relied solely on publicly available data. The data
collection adheres to Twitter’s terms of service (Twitter 2022a). In
compliance with Twitter’s terms and conditions, we only release the Tweet IDs
of publicly available tweets and require users of the dataset to comply with
Twitter’s terms and conditions as well.

There may be ethical concerns regarding user anonymity. Tweet objects contain
user information. Users have the option to restrict their tweets from being
made available through the API by switching to a private account or by
deleting their tweets. In this article, we only present aggregated statistics
to address this concern.

## References

  * Abilov et al. (2021) Abilov, A.; Hua, Y.; Matatov, H.; Amir, O.; and Naaman, M. 2021.  Voterfraud2020: a multi-modal dataset of election fraud claims on Twitter.  _arXiv preprint arXiv:2101.08210_.
  * Abramowitz and Saunders (2008) Abramowitz, A. I.; and Saunders, K. L. 2008.  Is polarization a myth?  _The Journal of Politics_ , 70(2): 542–555.
  * Barberá (2015) Barberá, P. 2015.  Birds of the same feather tweet together: Bayesian ideal point estimation using Twitter data.  _Political analysis_ , 23(1): 76–91.
  * Barberá et al. (2019) Barberá, P.; Casas, A.; Nagler, J.; Egan, P. J.; Bonneau, R.; Jost, J. T.; and Tucker, J. A. 2019.  Who leads? Who follows? Measuring issue attention and agenda setting by legislators and the mass public using social media data.  _American Political Science Review_ , 113(4): 883–901.
  * BESTVATER et al. (2022) BESTVATER, S.; SHAH, S.; RIVER, G.; and SMITH, A. 2022.  Politics on twitter: One-third of tweets from us adults are political.
  * Bild et al. (2015) Bild, D. R.; Liu, Y.; Dick, R. P.; Mao, Z. M.; and Wallach, D. S. 2015.  Aggregate characterization of user behavior in Twitter and analysis of the retweet graph.  _ACM Transactions on Internet Technology (TOIT)_ , 15(1): 1–24.
  * Brenan (2010) Brenan, M. 2010.  Record-High 47% in U.S. Think Abortion Is Morally Acceptable.  https://news.gallup.com/poll/350756/record-high-think-abortion-morally-acceptable.aspx.  [Online; accessed 11-January-2023].
  * Breuer, Landman, and Farquhar (2015) Breuer, A.; Landman, T.; and Farquhar, D. 2015.  Social media and protest mobilization: Evidence from the Tunisian revolution.  _Democratization_ , 22(4): 764–792.
  * Chen et al. (2021) Chen, E.; Chang, H.; Rao, A.; Lerman, K.; Cowan, G.; and Ferrara, E. 2021.  COVID-19 misinformation and the 2020 US presidential election.  _The Harvard Kennedy School Misinformation Review_.
  * Chen, Deb, and Ferrara (2022) Chen, E.; Deb, A.; and Ferrara, E. 2022.  # Election2020: The first public Twitter dataset on the 2020 US Presidential election.  _Journal of Computational Social Science_ , 5(1): 1–18.
  * Chen and Ferrara (2022) Chen, E.; and Ferrara, E. 2022.  Tweets in time of conflict: A public dataset tracking the twitter discourse on the war between ukraine and russia.  _arXiv preprint arXiv:2203.07488_.
  * Chen et al. (2020) Chen, E.; Lerman, K.; Ferrara, E.; et al. 2020.  Tracking social media discourse about the covid-19 pandemic: Development of a public coronavirus twitter data set.  _JMIR public health and surveillance_ , 6(2): e19273.
  * Cinelli et al. (2021) Cinelli, M.; De Francisci Morales, G.; Galeazzi, A.; Quattrociocchi, W.; and Starnini, M. 2021.  The echo chamber effect on social media.  _Proceedings of the National Academy of Sciences_ , 118(9): e2023301118.
  * Conover et al. (2011) Conover, M.; Ratkiewicz, J.; Francisco, M.; Gonçalves, B.; Menczer, F.; and Flammini, A. 2011.  Political polarization on twitter.  In _Proceedings of the international aaai conference on web and social media_ , volume 5, 89–96.
  * Csardi and Nepusz (2006) Csardi, G.; and Nepusz, T. 2006.  The igraph software package for complex network research.  _InterJournal_ , Complex Systems: 1695.
  * DiMaggio, Evans, and Bryson (1996) DiMaggio, P.; Evans, J.; and Bryson, B. 1996.  Have American’s social attitudes become more polarized?  _American journal of Sociology_ , 102(3): 690–755.
  * DocNow (2020) DocNow. 2020.  Hydrator [Computer Software].  https://github.com/docnow/hydrator.  [Online; accessed 11-January-2023].
  * Evans (2003) Evans, J. H. 2003.  Have Americans’ attitudes become more polarized?—An update.  _Social Science Quarterly_ , 84(1): 71–90.
  * Ferrara et al. (2016) Ferrara, E.; Varol, O.; Davis, C.; Menczer, F.; and Flammini, A. 2016.  The rise of social bots.  _Communications of the ACM_ , 59(7): 96–104.
  * Fiorina, Abrams et al. (2008) Fiorina, M. P.; Abrams, S. J.; et al. 2008.  Political polarization in the American public.  _ANNUAL REVIEW OF POLITICAL SCIENCE-PALO ALTO-_ , 11: 563.
  * Founta et al. (2018) Founta, A. M.; Djouvas, C.; Chatzakou, D.; Leontiadis, I.; Blackburn, J.; Stringhini, G.; Vakali, A.; Sirivianos, M.; and Kourtellis, N. 2018.  Large scale crowdsourcing and characterization of twitter abusive behavior.  In _Twelfth International AAAI Conference on Web and Social Media_.
  * Freelon, McIlwain, and Clark (2018) Freelon, D.; McIlwain, C.; and Clark, M. 2018.  Quantifying the power and consequences of social media protest.  _New Media & Society_, 20(3): 990–1011.
  * Garimella et al. (2018) Garimella, K.; De Francisci Morales, G.; Gionis, A.; and Mathioudakis, M. 2018.  Political discourse on social media: Echo chambers, gatekeepers, and the price of bipartisanship.  In _Proceedings of the 2018 world wide web conference_ , 913–922.
  * Gerstein and Ward (2020) Gerstein, J.; and Ward, A. 2020.  Supreme Court has voted to overturn abortion rights, draft opinion shows.  https://www.politico.com/news/2022/05/02/supreme-court-abortion-draft-opinion-00029473.
  * Giorgi et al. (2022) Giorgi, S.; Guntuku, S. C.; Himelein-Wachowiak, M.; Kwarteng, A.; Hwang, S.; Rahman, M.; and Curtis, B. 2022.  Twitter Corpus of the# BlackLivesMatter Movement and Counter Protests: 2013 to 2021.  In _Proceedings of the International AAAI Conference on Web and Social Media_ , volume 16, 1228–1235.
  * Guo et al. (2022) Guo, S.; Burghardt, K.; Rao, A.; and Lerman, K. 2022.  Emotion Regulation and Dynamics of Moral Concerns During the Early COVID-19 Pandemic.  _arXiv preprint arXiv:2203.03608_.
  * Heldebrandt (2019) Heldebrandt, B. 2019.  How a popular media bias chart determines what news can be trusted.  _Gateway Journalism Review_ , 48(355): 20–22.
  * Huang et al. (2020) Huang, X.; Jamison, A.; Broniatowski, D.; Quinn, S.; and Dredze, M. 2020.  Coronavirus Twitter Data: A collection of COVID-19 tweets with automated annotations.  Http://twitterdata.covid19dataresources.org/index.
  * Ince, Rojas, and Davis (2017) Ince, J.; Rojas, F.; and Davis, C. A. 2017.  The social media response to Black Lives Matter: How Twitter users interact with Black Lives Matter through hashtag use.  _Ethnic and racial studies_ , 40(11): 1814–1830.
  * Kerchner and Wrubel (2021) Kerchner, D.; and Wrubel, L. 2021.  U.S. Capitol Riot and #TrumpRally Tweet IDs.
  * McGregor (2019) McGregor, S. C. 2019.  Social media as public opinion: How journalists use social media to represent public opinion.  _Journalism_ , 20(8): 1070–1086.
  * Media (2020) Media, A. F. 2020.  The media bias chart.
  * Mouw and Sobel (2001) Mouw, T.; and Sobel, M. E. 2001.  Culture wars and opinion polarization: the case of abortion.  _American Journal of Sociology_ , 106(4): 913–943.
  * Munn (2021) Munn, L. 2021.  More than a mob: Parler as preparatory media for the US Capitol storming.  _First Monday_.
  * NAF (2022) NAF. 2022.  2021 Violence and Disruption Report.  https://prochoice.org/national-abortion-federation-releases-2021-violence-disruption-report/.  [Online; accessed 9-January-2023].
  * Nelson and Tandoc Jr (2019) Nelson, J. L.; and Tandoc Jr, E. C. 2019.  Doing “well” or doing “good”: What audience analytics reveal about journalism’s competing goals.  _Journalism Studies_ , 20(13): 1960–1976.
  * Nikolov, Flammini, and Menczer (2020) Nikolov, D.; Flammini, A.; and Menczer, F. 2020.  Right and left, partisanship predicts (asymmetric) vulnerability to misinformation.  _arXiv preprint arXiv:2010.01462_.
  * Pew (2022) Pew. 2022.  Differences in How Democrats and Republicans Behave on Twitter.  ”https://www.pewresearch.org/politics/2020/10/15/differences-in-how-democrats-and-republicans-behave-on-twitter/”.
  * Priniski et al. (2021) Priniski, J. H.; Mokhberian, N.; Harandizadeh, B.; Morstatter, F.; Lerman, K.; Lu, H.; and Brantingham, P. J. 2021.  Mapping moral valence of tweets following the killing of George Floyd.  _arXiv preprint arXiv:2104.09578_.
  * Raghavan, Albert, and Kumara (2007) Raghavan, U. N.; Albert, R.; and Kumara, S. 2007.  Near linear time algorithm to detect community structures in large-scale networks.  _Physical review E_ , 76(3): 036106.
  * Rao et al. (2021) Rao, A.; Morstatter, F.; Hu, M.; Chen, E.; Burghardt, K.; Ferrara, E.; Lerman, K.; et al. 2021.  Political partisanship and antiscience attitudes in online discussions about COVID-19: Twitter content analysis.  _Journal of medical Internet research_ , 23(6): e26692.
  * Rao, Morstatter, and Lerman (2022) Rao, A.; Morstatter, F.; and Lerman, K. 2022.  Partisan Asymmetries in Exposure to Misinformation.  _Scientific Reports_ , 12.
  * Saad (2010a) Saad, L. 2010a.  Americans Still Split Along Pro-Choice Pro-Life Lines.  https://news.gallup.com/poll/147734/americans-split-along-pro-choice-pro-life-lines.aspx.  [Online; accessed 11-January-2023].
  * Saad (2010b) Saad, L. 2010b.  Republicans’, Dems’ Abortion Views Grow More Polarized.  https://news.gallup.com/poll/126374/republicans-dems-abortion-views-grow-polarized.aspx.  [Online; accessed 11-January-2023].
  * SCOTUS (2022) SCOTUS. 2022.  Dobbs, State Health Officer of the Mississippi Department of Health vs Jackson Women’s Health Organization.  https://www.supremecourt.gov/opinions/21pdf/19-1392˙6j37.pdf.  [Online; accessed 11-January-2023].
  * Segerberg and Bennett (2011) Segerberg, A.; and Bennett, W. L. 2011.  Social media and the organization of collective action: Using Twitter to explore the ecologies of two climate change protests.  _The Communication Review_ , 14(3): 197–215.
  * Steinert-Threlkeld (2017) Steinert-Threlkeld, Z. C. 2017.  Spontaneous collective action: Peripheral mobilization during the Arab Spring.  _American Political Science Review_ , 111(2): 379–403.
  * Stocking, Barthel, and Grieco (2018) Stocking, G.; Barthel, M.; and Grieco, E. 2018.  Sources shared on Twitter: A case study on immigration. Pew Research Center.
  * Twitter (2022a) Twitter. 2022a.  Developer Agreement and Policy.  https://developer.twitter.com/en/developer-terms/agreement-and-policy.  [Online; accessed 11-January-2023].
  * Twitter (2022b) Twitter. 2022b.  Tweets Lookup Endpoint.  https://developer.twitter.com/en/docs/twitter-api/tweets/lookup/api-reference/get-tweets-id.  [Online; accessed 11-January-2023].
  * Wojcieszak et al. (2021) Wojcieszak, M.; Casas, A.; Yu, X.; Nagler, J.; and Tucker, J. A. 2021.  Echo chambers revisited: The (overwhelming) sharing of in-group politicians, pundits and media on Twitter.
  * Wojcieszak et al. (2022) Wojcieszak, M.; Casas, A.; Yu, X.; Nagler, J.; and Tucker, J. A. 2022.  Most users do not follow political elites on Twitter; those who do show overwhelming preferences for ideological congruity.  _Science advances_ , 8(39): eabn9418.
  * Wojcik and Hughes (2019) Wojcik, S.; and Hughes, A. 2019.  Sizing up Twitter users.  _PEW research center_ , 24.
  * Wolfsfeld, Segev, and Sheafer (2013) Wolfsfeld, G.; Segev, E.; and Sheafer, T. 2013.  Social media and the Arab Spring: Politics comes first.  _The International Journal of Press/Politics_ , 18(2): 115–137.
  * Won, Steinert-Threlkeld, and Joo (2017) Won, D.; Steinert-Threlkeld, Z. C.; and Joo, J. 2017.  Protest activity detection and perceived violence estimation from social media images.  In _Proceedings of the 25th ACM international conference on Multimedia_ , 786–794.
  * Yardi and Boyd (2010) Yardi, S.; and Boyd, D. 2010.  Dynamic debates: An analysis of group polarization over time on twitter.  _Bulletin of science, technology & society_, 30(5): 316–327.