# Radiance Fields for Robotic Teleoperation

Maximum Wilder-Smith, Vaishakh Patil, Marco Hutter Authors are with the
Robotic Systems Lab, ETH Zurich. Email: {mwilder, patilv<EMAIL_ADDRESS>
###### Abstract

Radiance field methods such as Neural Radiance Fields (NeRFs) or 3D Gaussian
Splatting (3DGS), have revolutionized graphics and novel view synthesis. Their
ability to synthesize new viewpoints with photo-realistic quality, as well as
capture complex volumetric and specular scenes, makes them an ideal
visualization for robotic teleoperation setups. Direct camera teleoperation
provides high-fidelity operation at the cost of maneuverability, while
reconstruction-based approaches offer controllable scenes with lower fidelity.
With this in mind, we propose replacing the traditional reconstruction-
visualization components of the robotic teleoperation pipeline with online
Radiance Fields, offering highly maneuverable scenes with photorealistic
quality. As such, there are three main contributions to state of the art: (1)
online training of Radiance Fields using live data from multiple cameras, (2)
support for a variety of radiance methods including NeRF and 3DGS, (3)
visualization suite for these methods including a virtual reality scene. To
enable seamless integration with existing setups, these components were tested
with multiple robots in multiple configurations and were displayed using
traditional tools as well as the VR headset. The results across methods and
robots were compared quantitatively to a baseline of mesh reconstruction, and
a user study was conducted to compare the different visualization methods.
More code and additional samples are available at
https://leggedrobotics.github.io/rffr.github.io/.

##  I Introduction

Robotics is rapidly expanding its role in our everyday activities, which needs
fast, repetitive, and precise actions. Robots are becoming highly autonomous,
from vacuum cleaners to everyday commute vehicles. However, autonomous robots
are still limited by perception, planning, and control capabilities for
different tasks. Here, they still need human intervention to be successfully
executed. The process of controlling robots remotely is known as robotic
teleoperation. As robots become increasingly integrated into complex
environments, there is a growing need for teleoperation systems that offer
high-fidelity reconstruction and intuitive user interfaces. For the robots to
be teleoperated by humans, the system must create an immersive environment
that can provide situational awareness to the operator. Traditional
teleoperation pipelines capture sensor data from robots and display it to an
operator. Direct camera streams offer high fidelity at the cost of
maneuverability, while reconstructed environments are easy to control but
often struggle to accurately capture scene geometry and texture, especially in
volumetric or specular environments. The reconstruction quality is often
limited by high sensor noise and temporal inconsistencies [1]. Moreover, the
user interfaces may lack immersion and intuitive interaction, limiting the
operator’s ability to perform tasks effectively. The proposed pipeline
addresses these challenges by leveraging advancements in Radiance Fields and
immersive visualization technologies to bring high-fidelity, maneuverable
scenes.

Figure 1: Teleoperator controlling a robot using a VR interface from inside a
reconstructed Radiance Field. Reconstructions are created online from any
robot based on its sensor configuration. Neural Radiance Field or Gaussian
Splatting renders can be displayed in an immersive 360∘ render or on a
handheld viewer. These renders are displayed alongside the robot and its
sensor data, such as camera feeds and LiDAR. Figure 2: General teleoperation
visualization pipeline divided into three sections: Robots, Reconstruction
Methods, Visualization. Sensor and pose data flows from various robotic
components (red) into the reconstruction method (green) to create a scene
representation that is shown to the user in the visualizer (blue). Support for
Radiance Field reconstructions such as Neural Radiance Fields (NeRFs) and
Gaussian Splatting, as well as RViz and VR visualizers for these methods, are
presented in this work. For baseline comparisons, a Voxblox mesh viewer was
also ported to VR. These contributions are highlighted with a purple dashed
line.

The foundation of Neural Radiance Fields (NeRF) [2] has marked a significant
leap in 3D reconstruction due to their ability to render novel photorealistic
views from a sparse set of posed images. Recent advancements in NeRFs [3, 4,
5] have significantly improved training and rendering time, making it an
excellent candidate to improve the visualization and immersiveness of
teleoperation systems. However, NeRFs still struggle with computational
efficiency and scene controllability [6], essential for teleoperation. In
response to these challenges, another technique known as 3D Gaussian Splatting
(3DGS) [6] has emerged, offering an efficient approach to rendering. Unlike
NeRFs, 3DGS utilizes an explicit Radiance Field representation, coupled with
highly parallelized processing, to achieve efficient computation and rendering
times. This is especially beneficial in complex settings, where precise
manipulation of geometries and adaptation to lighting conditions are
imperative.

With this in mind, this work aims to advance the state-of-the-art in robotic
teleoperation by presenting a novel dynamic visualization pipeline that merges
Radiance Fields with real-time robots. First and foremost, our system can
train high-quality Radiance Fields from real-time robotic data in the form of
a Radiance Field Node. The system is dynamic enough to handle data from any
ROS-based robot, from a simple robotic arm to a highly mobile quadruped.
Secondly, to keep up with the rapid advances in neural rendering, this system
includes deep integration with NerfStudio [7], allowing for support for
different methods, including Gaussian Splatting. Finally, the system provides
a suite of visualization options allowing teleoperators to understand their
environment through Radiance Fields. We offer an RViz-compatible Radiance
Field Plugin system that integrates seamlessly with most existing ROS
teleoperation systems, complete with multiple render modes, scene cropping,
and depth-informed clipping. For more immersive use, a VR visualization suite
is designed, allowing the user to stand in a virtual replica of the scene or
send commands through a birds-eye-view system. Ultimately, this research opens
up new possibilities for combining not just teleoperation with Radiance Fields
but the use of online Radiance Fields for robotics in general.

##  II Related Works

This section provides a brief overview of teleoperation and reconstruction
methods, and recent work in Radiance Fields.

####  II-1 Teleoperation and Reconstruction

As robots are deployed to more complex uses such as industrial settings [8] or
caves and other natural environments [9], there is an increased need for
teleoperation. Here sensor data is streamed to an operator, either as direct
camera feeds [10] or scene reconstructions [11]. The state-of-the-art setups
favor the high fidelity of direct camera systems [10]. However, they suffer
from limited maneuverability due to fixed sensor feeds. Meanwhile,
reconstruction-based teleoperation builds a scene representation that is easy
to maneuver[11, 12], but often suffers from low fidelity or poor performance
[13]. Even under the best circumstances, these systems fail to represent
volumetrics such as smoke and ignore view-dependent data such as specular
effects. Radiance Fields are explored as a high-fidelity and maneuverable
scene representation.

####  II-2 Radiance Fields

Originally used to render novel views with photorealistic quality, Radiance
Fields have recently been adapted for large-scale 3D reconstructions. Neural
Radiance Fields (NeRFs) [2] could capture small scenes by encoding geometry,
color, and density into an MLP, allowing for view-dependent colors and
volumetric rendering. Rapid advances allowed for high-frequency details [3],
unrestricted view construction [4], and even sped up operation to near real-
time on small scenes [5].

As a faster alternative to NeRFs, Gaussian Splatting (3DGS) [6] leverages a
more explicit scene representation for real-time rendering. Representing the
scene with 3D Gaussians can capture view-dependent color, volumetric elements,
and unbounded scenes at higher speeds. However, it lacks the scene fidelity of
NeRF, particularly with complex reflections with specular objects and sharp
edges.

The recent explosion of novel NeRF methods has been aided by the creation of
NerfStudio [7]. NerfStudio provides a comprehensive system for building,
training, and adapting Radiance Field methods. It includes methods such as
Nerfacto (NeRF implementation) and Splatfacto (Gaussian Splatting
implementation), as well as a variant of Instant-NGP, depth-supervised
methods, and numerous others.

Until recently, the intersection of robotics and Radiance Fields was limited
due to long training times and the difficulty in porting the systems to mobile
hardware. Instead, Radiance Fields have mainly been used to perform
localization with offline data [14] or offline path planning [15]. Online
integration of NeRFs is available in NerfBridge [16], although lacks support
for multiple cameras and newer methods. This work addresses these issues by
enabling online multi-camera Radiance Field reconstruction, which easily
supports new methods such as 3DGS.

##  III Methodology

Figure 3: Component diagram showing the connections between the robotic
systems (red and orange), the reconstruction methods (green), and the
visualizers (blue). Data flows from the robotic systems into reconstruction
method nodes where it is either merged into rendered views for Radiance Fields
or a mesh for Voxblox. The Radiance Field Node is comprised of a custom
DataLoader, DataParser, and Dataset, and uses a collection of Sensor objects
to manage the ROS subscribers. The DataLoader and Dataset send data to any
standard method during training, only returning images and cameras that have
been updated. This data is then displayed directly in RViz or transmitted over
TCP to a VR headset.

The ideal teleoperation system would present the operator with a high-
fidelity, maneuverable, and faithful representation of the scene [17]. This
system should be robot-agnostic with minimal reconfiguration and be able to
present environmental data–sensor streams or reconstructions–to best suit the
task [1]. The extended 3D reconstruction-based pipeline, shown in fig. 2,
involves taking sensor data from the robot over ROS and feeding it into a
reconstruction method [11, 7]. This method generates a representation of the
environment relative to a fixed position which is presented to an operator via
the visualizer. By constructing the representation relative to a fixed point,
it can be displayed and aligned with additional data even as the robot moves
through the space. Overall, we can simplify this pipeline into three
components: the robot and its sensors, the reconstruction method, and the
visualization system.

Here, we present a system that builds off existing teleoperation pipelines and
extends the possible reconstruction methods to include Radiance Fields and a
means of visualizing them both on screen and with a VR headset. This is
highlighted in purple in fig. 2. The system was tested with multiple robots,
including a simple static configuration, a mobile quadruped capable of
exploring a larger environment, and a quadruped with an attached arm. This
robotic data is sent to an existing mesh reconstruction as a baseline and a
novel ROS node for Radiance Fields that supports NeRFs and 3DGS. These
reconstruction methods are available in multiple visualizers, ranging from the
2D RViz window on the screen to 2.5D and 3D views in a VR headset shown in
fig. 7. Each component of this pipeline can be swapped out or reconfigured
depending on the deployment and task.

###  III-A Robot

The purpose of the robot in this system is to capture the environment to
enable the user to understand the scene. The primary forms of data used in
this pipeline are poses from ROS’ TF system, color images from onboard
cameras, and RGBD images containing depth data and color information. To
ensure the system is robot agnostic, these data sources can be easily
configured for any deployment. In fig. 2 and fig. 3, the incoming data sources
are shown on the left inside the “Robot” box, with the red lines indicating
image data, orange showing the point cloud data, and yellow showing the poses
from the TF system.

The simplest robot setup would be a grounded robot, such as a robotic arm
attached to a table. This robot has limited mobility, particularly when it
comes to scanning a target object or a large-scale scene. However, it offers
incredibly accurate poses, as the robot’s base frame is the fixed global
frame. This gives reconstructions high pose confidence at the cost of scene
scale and view angles.

A mobile robot is required to capture larger, more complex scenes. In this
setup, the robot can move in the environment and capture virtually data from
any angle. However, the relation between the robot and a fixed global frame is
uncertain, requiring the mobile robot to localize within the environment. This
can be accomplished with simultaneous localization and mapping (SLAM) systems
such as CompSLAM [9]. The poses can drift during locomotion due to reliance on
inaccurate onboard sensors. While the mobile base allows for greater coverage,
the poses may not always be reliable.

One method to help ensure we can capture larger scenes with high pose accuracy
and mobility is to affix a robotic arm to the mobile platform. In this setup,
the robot can move around the scene to novel view angles and primarily collect
data from the arm’s motion while the base is static. As the robot base is
static, the end-effector motion can be measured purely from the precise
kinematic joint sensors. This avoids visual or LiDAR localization errors.

###  III-B Reconstruction Methods

Once data is captured by the robot, it is passed into a reconstruction method.
The purpose of this stage is to convert the numerous data streams into a
single stream that an operator can use to control the robot. This involves
creating a representation that is accurate both in terms of geometry and
texture fidelity. This work compares two main approaches to this problem: mesh
reconstruction via Voxblox [11] and Radiance Field reconstruction via
NerfStudio [7]. These two ROS nodes are shown as green in the “Reconstruction
Method” section of fig. 2 and fig. 3. The Voxblox node generates an output
mesh using incoming pose data and point clouds, while the NerfStudio node
generates rendered Radiance Fields from poses and image data. As mesh
reconstruction struggles to capture complex volumetric scenes, and trades-off
between fast reconstruction and high fidelity, we propose using Radiance
Fields for robotic teleoperation. Radiance fields seek to use lightweight
machine learning algorithms in order to learn representations of a scene.

Radiance Field Node: To maintain interoperability with new methods, the
Radiance Field training node in the pipeline can be set to use nearly any
method supported by NerfStudio [7]. The number of custom components in the
NerfStudio pipeline was minimized to ensure compatibility with any method so
long as it doesn’t require a custom Dataset (such as semantic models) or a
custom DataLoader. This supported NerfStudio’s 3DGS method: Splatfacto, with
minimal overhead. At the start, this custom Dataset allocates a buffer of
preconfigured size to hold all images for training. The node then subscribes
to a set of topics for each camera to capture its images (RGB or RGBD),
intrinsics, and poses. If all the cameras have depth topics, then a special
buffer is allocated to store the depth images, allowing for integration with
depth-supervised models. Each incoming image is filtered to ensure it is not
blurry and is a sufficiently different pose based on the mean distance between
TF frames. If the cameras have different resolutions, all images are resized
to the largest camera and scaled back down when collated into batches. For our
experiments this meant storing image at 1440p with the 720p images being
resized before training. The associated per-camera intrinsics and extrinsics
are stored for each image based on the current pose data and the last
CameraInfo message. If the image is already rectified and contains ”rect” in
the topic name, the distortion parameters of the associated camera are set to
zero to avoid rectification.

To ensure compatibility with additional methods, this package overrides the
DataLoader with a custom ROS variant which skips the caching and collating
normally performed, and instead uses the ROS Datasets functions to ensure only
updated images are pulled for training. There is also a special evaluation
DataLoader which can accept a list of image sequence IDs to ensure the same
images are used across multiple runs. This is particularly helpful when
running comparisons across prerecorded ROS bag data. Additionally, a custom
DataParser is used to setup the ROS Dataset as well as ROS subscribers which
are managed through a helper Sensor class. These Sensor objects manage the
subscribers for image and CameraInfo messages, and are responsible for looking
up the synchronized poses at each capture. The use of this helper class allows
multiple cameras to be registered, each capturing data at a different rates.
The Sensor objects will then update only the appropriate data entries, and
forward the associated camera parameters to the rest of the system. The flow
of data is shown in fig. 3 with all the custom components highlighted in
purple and the Sensor helper class in yellow. Once everything is set up, the
node captures several images to create the initial batch before switching to
online training mode.

Once the node is in online training mode, it continues to receive new images
up to the specified buffer size and begins an ActionServer, which allows any
ROS process to request render data. A render request containing the view pose
is sent to the server, and a rendered image, along with estimated scene depth
is returned. These renders are then sent to one of the visualizers for the
teleoperator to interact with. Additionally, the requests are associated with
unique client IDs, allowing multiple visualizers to request different renders
with a single Radiance Field node in the ROS network.

###  III-C Visualization

The final stage of the pipeline is the visualizer. This program is intended to
present an accurate representation of the world to the operator based on the
data received from the reconstruction method. This is shown in blue in fig. 2
and fig. 3 , with our custom Radiance Field RViz plugin acting as the 2D
visualization and the VR scenes as the 2.5D and 3D visualizers. For a baseline
comparison, the Voxblox RViz plugin is used to show the mesh reconstruction.

####  III-C1 RViz

Figure 4: Sample of the RViz plugin occluding the robot based on scene depth
(left) and having the robot rendered on top (right).

RViz is the de facto visualization suite for ROS, as it can visualize
different sensor streams, such as point clouds, robot models, and images.
Also, it contains tools to help users send commands directly through the
system, such as pose goals. It offers support for 3rd-party plugins, allowing
reconstruction methods such as Voxblox to implement custom viewers. This
seamlessly integrates the new environmental data with existing robotic and
teleoperation stacks.

Radiance Field Plugin: To integrate the online Radiance Field generation with
ROS, a Radiance Field RViz plugin is presented. This plugin acts as a custom
camera for the RViz OpenGL scene, sending render requests to the Radiance
Field node whenever the camera is moved (dynamic mode) or a stream of renders
based around a moving frame (continuous mode). Dynamic mode is best used when
viewing the scene from a static perspective, such as an overhead view for
navigation, or a close up for inspection. Continuous mode on the other hand is
helpful for a moving base frame, such as third-person piloting, or when new
incoming data needs to be observed. In either mode the speeds of the renders
are often most dependant on resolution of the output image, which can be
adjusted with a setting in the View panel. In order to provide rapid data for
dynamic mode, the renders are sent first at 10% of the final resolution, and
then 50% before sending over the full image. This allows the user to move
around the scene with more rapid response from the rendered environment.

Depth data from RViz and the Radiance Field are then used to either show the
render realistically occluding the scene elements or as a cut-out, as shown in
fig. 4. This is achieved by transforming the real-world depth captured in the
rendering into OpenGL’s z-depth, a nonlinear scale ranging from 0 to 1. This
transformed depth can then be matched with the z-depth used by RViz’s Ogre
engine (also based on OpenGL) for a specific camera perspective. As a result,
all objects within RViz–such as robots, sensor visualizations, grids, and
poses–are accurately occluded, providing a more realistic visualization. This
aims to give the operator a better understanding of depth within the
environment while also making it easier to locate RViz components in the
scene. In large and indoor scenes, there is also a cut-out mode in which the
rendered image is always displayed behind the RViz elements, making them
easier to find.

Figure 5: An axis-aligned bounding box can be used to crop the background
(left) or remove walls (right) allowing for novel views and clearer operation.

Furthermore, as both 3DGS and NeRFs perform volumetric rendering, it can be
difficult to self-localize due to occlusion or unexplored noise. To combat
this, the plugin allows for the creation of an axis-aligned bounding box which
crops the scene by limiting the ray integration. This is shown in fig. 5 where
the bounding box is used to remove the noisy background as well as the back
wall that would obstruct the operator’s view, visible in fig. 8. These two
renders are entirely novel views that the robot was unable to capture either
due to the wall or limitations on the arm’s mobility.

####  III-C2 VR

Figure 6: Teleoperator using VR system with Voxblox mesh on the table and
Radiance Field Viewer on the left (left) and live LiDAR data overlayed on the
mesh reconstruction.

In testing, it was found that even with occlusion, it was difficult for an
operator to get a sense of depth while looking at a 2D screen. To remedy this,
the system was ported to a Meta Quest 3 VR headset running with a Unity scene
to display the robotic data. This was following the prior works [1, 17, 8]
where VR was used to extend the capabilities of existing teleoperation systems
for greater immersion and ease of use.

VR Robot Data: To create a baseline for comparison, a handful of RViz
features, such as a TF synchronized robot model, pose publishing, and sensor
visualizations (for LiDAR, RGBD point clouds, and images) were integrated into
the VR headset as shown in fig. 6. The user is able to send pose goals via the
headset and hand interaction, enabling them to send robot commands directly
from the VR interface. This system was extended with a mobile GPU-based mesh
generation for real-time streaming of the Voxblox reconstruction. This
provides the user with the same sensor information they can access in RViz,
but displayed in an immersive 3D setting. In order to remain lightweight, the
VR scene is optimized to run natively on the headset, with only a TCP
connection to ROS handling message traffic. This allows the headset to connect
directly to the ROS network of the robot and begin interaction with out the
need for a base station PC or any headset tethers.

Figure 7: Radiance Field VR scene with 2.5D handheld parallax viewer (left)
and fully immersive 360∘ (right).

VR Radiance Fields: To use the Radiance Field renders, there is a 2.5D
handheld viewer, which provides the user a view into the rendered world. This
viewer uses the depth data to create a parallax effect allowing the VR headset
to provide a greater sense of depth while looking at the renders. Similar to
the RViz camera, the viewer can request new renders whenever the user moves
the head and it is synchronized to the position of the robot data in the
scene. Requests to the ActionServer are directly routed through the TCP
connection, and the rendered results are streamed straight to a GPU shader for
parallax rendering. For further immersion, there is also a 360∘ spherical
render which gives the user a sense of scale as if they are physically present
in the environment. The viewer also offers controls such as image zoom,
resolution, and a live camera feed from the robot. Figure 7 shows the handheld
Radiance Field viewer during deployment and the view from the spherical render
mode. Similar to RViz plugin, the depth data from the Radiance Fields is also
used to provide dynamic scene occlusion, allowing closer 3D objects, such as
robots or point clouds, to be rendered in front of the Radiance Field renders.

##  IV Experiments

As there are three major components to this pipeline, three sets of
experiments were conducted. First, data was captured on a static arm, mobile
base, and mobile arm to ensure robot-agnostic operation and compare
reconstructions. Each dataset was processed with Voxblox, NeRF, and 3DGS, and
photometric quality was evaluated. Second, to test online operations,
reconstruction and rendering times were measured. Lastly, to validate the
viewing experience, a user study was conducted to compare the RViz mesh and
NeRF renders vs their VR counterparts.

###  IV-A Datasets

####  IV-A1 Static Arm

A static Franka Panda arm was used with an Intel Realsense 435i RGBD camera.
The arm was fixed to a metal table and bounded by walls on two sides, reducing
the target scanning area to 20cm x 20cm. The third side of this area was
unexplorable as it is where the arm was mounted, limiting the range of scans
to front-facing and only 90∘ of azimuth. Three targets were captured: a
stuffed donkey toy with fine fur, a fiducial cube, and a wooden block. The fur
was used to test the high-frequency details, the fiducials checked texture
accuracy, and the block was used to confirm 3D geometric reconstruction.

####  IV-A2 Mobile Base

An Anybotics Anymal was used to walk around a large lab environment. The scale
of the scene was roughly 15m x 10m and was captured with front and rear-facing
1440p RGB cameras. Localization was performed using CompSLAM [9] and centered
around a wooden pedestal which the robot orbited. Atop the pedestal was a
glass bowl, testing each system’s transparent and specular reconstruction
abilities.

####  IV-A3 Mobile Arm

A DynaArm was attached to the Anybotics Anymal with an additional Intel
Realsense L515 mounted at the wrist. The target of capture was a yellow
switchboard cabinet with a metal bar across the middle holding a box with a
screen and switch. This was roughly 1m x 1m and captured from the front, with
Anymal scanning 2m away with the arm, before moving in and capturing the
inside of the box. As the arm moved into the box, there was a slight change in
lighting condition, testing how well the systems adapt to dynamic colors.

###  IV-B Quality

|  |  |
---|---|---|---
|  |  |
|  |  |
Ground Truth | Voxblox | NeRF | 3DGS
Figure 8: Comparisons of the static arm (left), mobile base (middle), and mobile arm (right). The top row shows the ground truth images, the second row shows the Voxblox reconstructions, the third row shows the NeRF reconstruction and the bottom row shows the 3DGS reconstruction. NeRF produced the highest quality results for both the mobile datasets, while 3DGS created a near-perfect image for the static arm dataset. TABLE I: Photometric Comparisons of Different Robot Configurations and Reconstruction Methods  Dataset | Method | PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$
---|---|---|---|---
Static Arm | Voxblox | 15.42 | 0.4969 | 0.5507
NeRF | 18.07 | 0.5309 | 0.4917
3DGS | 34.12 | 0.9288 | 0.2298
Mobile Base | Voxblox | 6.85 | 0.4091 | 0.8051
NeRF | 25.55 | 0.8718 | 0.1988
3DGS | 20.18 | 0.7853 | 0.3953
Mobile Arm | Voxblox | 16.94 | 0.514 | 0.5295
NeRF | 22.16 | 0.6473 | 0.1478
3DGS | 24.59 | 0.7414 | 0.2126

In order to ensure the platform is robot agnostic, three different deployments
were run. All three setups were trained with the same parameters using
Voxblox, Nerfacto, and Splatfacto, with Nerfacto and Splatfacto being the
default NeRF and 3DGS methods from NerfStudio. Each was evaluated based on
their peak-signal-to-noise ratio (PSNR), structural similarity index measure
(SSIM), and learned perceptual image patch similarity (LPIPS) [18]. PSNR is
used to measure the artifacts in the scene, SSIM measures the similarity for
features such as lighting and contrast, and LPIPS measures the network
activation of patches, approximating what humans would say is similar. The
ground truth picture, baseline Voxblox reconstruction, NeRF, and 3DGS are
shown in fig. 8, while the photometric comparisons are shown in table I. The
best-resulting method for each dataset is shown in bold.

####  IV-B1 Static Arm

As the Panda arm data had near-perfect poses, it was able to produce results
of very high texture fidelity. High enough that in fig. 8, the fiducial cube’s
markers are clear enough to scan. The lack of mobility produced limited views
on the periphery of the scene, resulting in lower-quality NeRF
reconstructions. However, the small scene size meant incredibly high-quality
3DGS reconstructions. This is because the scene was well initialized, with
Gaussians densely covering the entire space with little excess. Across all
metrics–PSNR, SSIM, and LPIPS–the 3DGS reconstruction performed twice as well
as the NeRF. While the captured areas are decently represented, Voxblox
created an incomplete mesh, scoring the lowest in all metrics. These results
are shown in table I.

####  IV-B2 Mobile Base

Radiance Fields, particularly NeRFs, struggle with ray stability far from the
scene center, resulting in noise. Even Voxblox struggled to capture distant
data, producing a mostly incomplete reconstruction. During training, the NeRF
method was able to smooth out the noise via pose optimization, resulting in
the best results. Additionally, the target glass bowl was entirely missing
from the Voxblox reconstruction, while it was clearly captured in the NeRF and
roughly captured with 3DGS.

####  IV-B3 Mobile Arm

The switchboard cabinet produced the highest quality results for LPIPS on the
NeRF and 3DGS reconstruction, representing the highest human-perceived
quality. Voxblox managed to recreate most of the scene, although failed to
capture the support beam and part of the door. Additionally, as the lighting
changed when moving inside the cabinet, the mesh was not uniformly colored.
This was captured in the Radiance Fields through their view-dependent
coloring. As shown in table I, 3DGS outperformed NeRF on PSNR and SSIM, likely
due to its better color matching with spherical harmonics.

###  IV-C Performance

TABLE II: Time Comparisons Between Different Methods Method |  | Per Iteration
---
Time [ms]
| PSNR of
---
16.94 dB [s]
| Render
---
Time [ms]
| Render
---
Time [FPS]
Voxblox | 1205.903 | 165.470 | – | –
NeRF | 35.644 | 7.027 | 1020.131 | 0.980
3DGS | 34.651 | 6.996 | 6.6257 | 151

Online robotic teleoperation requires fast reconstruction and visualization.
To that end, each method’s reconstruction and render times were compared on an
RTX 4090. In order to ensure repeatability across runs and comparisons, the
data was first recorded rosbags and the sequence ID for images was stored to a
file. This was then used to create the Datasets during execution, ensuring the
same images at the same timestamps were used in each run. First, the per
iteration time was measured. For Voxblox, this is the time to integrate new
batches with the mesh, while for the Radiance Fields, this was the training
time for one iteration. Voxblox was run with 1cm voxels, and took 1.2 seconds
to integrate each batch. For NeRFs and 3DGS, the iteration time was 35.644ms
and 34.651ms, respectively. Additionally, for a fair comparison, the radiance
methods were trained until they matched the final Voxblox PSNR of 16.94dB. For
NeRF, this took 7.027s and 3DGS took 6.996s. Not only did the Radiance Field
methods produce higher quality results, but they achieved this almost 20x
faster. A summary of these results is shown in table II.

Figure 9: Render times for NeRF and 3DGS for a 1024x1024 image at different
resolution percentages, showing an exponential increase in render time for
NeRFs with a nearly constant time for 3DGS. The data is displayed at log scale
due to the rapid increase in NeRF render times.

Second, the render time was considered. As Voxblox displays a continually
updating mesh, its render is limited to 30 frames per second by the viewer.
For NeRF and 3DGS, a full 1024x1024 image was rendered, taking 1020.13ms
(0.98FPS) for NeRF and only 6.63ms (151FPS) for 3DGS. When comparing the
rendering times for different resolutions, 3DGS remains fairly constant at
around 6.23ms, while NeRF renders take exponentially longer for larger
resolutions. This relation is shown in fig. 9. For use cases requiring rapid
rendering, explicit representations such as mesh and 3DGS are preferred as
they scale well for larger resolutions and screens.

###  IV-D User Study

Figure 10: Results of the user study using the mobile arm dataset and four
viewing modes: VR Voxblox(  ), VR NeRF(  ), RViz NeRF(  ), RViz Voxblox(  ).
The means are shown with a black dotted line, while the medians are the yellow
solid line. In all cases, the VR systems were preferred to their 2D
counterparts, while NeRFs were preferred for perception and manipulation and
Voxblox for locomotion and usability.

A user study was conducted with 20 participants to compare the different
visualization systems. The participants were chosen from a group familiar with
robotic systems and who regularly use RViz. Their ages were between 22 and 32,
with a mean age of 26, there were 5 female and 15 male participants, and only
5 had used VR systems before. They evaluated the mobile arm dataset in fig. 8
and compared the 2D RViz Voxblox and NeRF scene with VR counterparts, with the
results shown in fig. 10. This dataset was chosen as it had the highest
photometric scores, table I, and therefore the highest quality. To avoid
confusion, as both the Radiance Fields, NeRF and 3DGS, are presented via the
same user interface, only one method was presented to the users. NeRF was
chosen for this comparison as it outscored 3DGS in LPIPS, which best
approximates human-perceived quality [18]. At the resolutions tested, the
difference in render time between NeRF and 3DGS was imperceptible.

Starting with perception, the NeRF was preferred to the Voxblox
reconstruction, with the VR NeRF scoring slightly higher than the RViz
variant. This confirms the results of the photometric comparison in table I,
that Radiance Fields produce higher quality results than a mesh.
Interestingly, the VR systems consistently increased the perceived quality,
despite showing the same data as RViz. This is due to the immersion of the VR
system, with the added depth helping geometric perception and the head-mounted
optics helping boost texture quality. Additionally, the optics of the VR
headset allow lower resolution images to be perceived as larger and higher
resolution, decreasing the render latency for the system.

In terms of teleoperation tasks, the 3D mesh of Voxblox VR and RViz was
preferred for locomotion tasks as the users could easily see where the robot
was in relation to the environment. However, for manipulation, RViz NeRF was
preferable to Voxblox, while the VR systems were roughly the same. Perception
and ability to read fine details are paramount in these tasks, favoring the
NeRF system, with many VR users saying they would prefer a 3D NeRF for the
ideal setup. This can be addressed in the future by leveraging 3DGS’ explicit
representation to directly show the Gaussians in 3D without the need for the
2.5D viewer.

Finally, the VR systems were favored for usability over their RViz
counterparts, despite this being the first time many users tried VR. The
Voxblox systems were easier to maneuver and explore, as the direct 3D
representation was intuitive. The RViz NeRF overlay and handheld viewer took
time to get used to, but was the better platform for identifying objects
within the scene.

##  V Conclusion

In conclusion, this paper presents significant advancements in the field of
robotic teleoperation through a comprehensive Radiance Field visualization
pipeline integrating multiple cameras, dynamic support for new reconstruction
methods, and VR integration. The experiments conducted demonstrate the
effectiveness and versatility of the proposed system.

Firstly, demonstrating the system’s adaptability for different robotic
deployments, from static arms to mobile bases with multiple cameras, in each
setup, the Radiance Fields produced higher quality reconstructions compared to
the mesh. Secondly, the greatly increased render speed of 3DGS makes it
comparable to mesh rendering for online use while only requiring less time to
hit the same quality. Lastly, the user study comparing visualization systems
underscores a user preference for the quality of Radiance Fields over
traditional meshes. For manipulation tasks, these renders were even preferred
to the traditional system. Across the board, the VR systems provided a better
experience, suggesting that a VR version of explicitly 3D Radiance Fields,
such as direct 3DGS in VR, would be the optimal teleoperation setup.

Overall, this work contributes to advancing state-of-the-art robotic
teleoperation by providing a robust pipeline adaptable to different
deployments, offering high-fidelity reconstruction capabilities, and
leveraging immersive VR environments for enhanced user experience. The
findings suggest promising avenues for future research in the intersection of
robotics and Radiance Fields technologies.

## References

  * [1] P. Stotko, S. Krumpen, M. Schwarz, C. Lenz, S. Behnke, R. Klein, and M. Weinmann, “A vr system for immersive teleoperation and live exploration with a mobile robot,” in _2019 IROS_. IEEE, Nov. 2019.
  * [2] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,” in _ECCV_ , 2020.
  * [3] M. Tancik, P. P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. T. Barron, and R. Ng, “Fourier features let networks learn high frequency functions in low dimensional domains,” _NeurIPS_ , 2020.
  * [4] J. T. Barron, B. Mildenhall, M. Tancik, P. Hedman, R. Martin-Brualla, and P. P. Srinivasan, “Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields,” _CoRR_ , vol. abs/2103.13415, 2021.
  * [5] T. Müller, A. Evans, C. Schied, and A. Keller, “Instant neural graphics primitives with a multiresolution hash encoding,” _ACM Trans. Graph._ , vol. 41, no. 4, pp. 102:1–102:15, July 2022.
  * [6] B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis, “3d gaussian splatting for real-time radiance field rendering,” _ACM Transactions on Graphics_ , vol. 42, no. 4, July 2023.
  * [7] M. Tancik, E. Weber, E. Ng, R. Li, B. Yi, J. Kerr, T. Wang, A. Kristoffersen, J. Austin, K. Salahi, A. Ahuja, D. McAllister, and A. Kanazawa, “Nerfstudio: A modular framework for neural radiance field development,” in _ACM SIGGRAPH 2023 Conference Proceedings_ , ser. SIGGRAPH ’23, 2023.
  * [8] K. Li, S. Schmidt, T. Rolff, R. Bacher, W. Leemans, and F. Steinicke, “Magic nerf lens: Interactive fusion of neural radiance fields for virtual facility inspection,” 2023.
  * [9] S. Khattak, H. Nguyen, F. Mascarich, T. Dang, and K. Alexis, “Complementary multi–modal sensor fusion for resilient robot pose estimation in subterranean environments,” in _2020 International Conference on Unmanned Aircraft Systems (ICUAS)_ , 2020, pp. 1024–1029.
  * [10] M. Schwarz, C. Lenz, R. Memmesheimer, B. Pätzold, A. Rochow, M. Schreiber, and S. Behnke, “Robust immersive telepresence and mobile telemanipulation: Nimbro wins ana avatar xprize finals,” 2023.
  * [11] H. Oleynikova, Z. Taylor, M. Fehr, R. Siegwart, and J. Nieto, “Voxblox: Incremental 3d euclidean signed distance fields for on-board mav planning,” in _2017 IROS_ , 2017.
  * [12] R. A. Newcombe, S. Izadi, O. Hilliges, D. Molyneaux, D. Kim, A. J. Davison, P. Kohi, J. Shotton, S. Hodges, and A. Fitzgibbon, “Kinectfusion: Real-time dense surface mapping and tracking,” in _2011 10th IEEE International Symposium on Mixed and Augmented Reality_ , 2011, pp. 127–136.
  * [13] Y. Pan, Y. Kompis, L. Bartolomei, R. Mascaro, C. Stachniss, and M. Chli, “Voxfield: Non-projective signed distance fields for online planning and 3d reconstruction,” in _2022 IROS_ , 2022.
  * [14] D. Maggio, M. Abate, J. Shi, C. Mario, and L. Carlone, “Loc-nerf: Monte carlo localization using neural radiance fields,” 2022.
  * [15] T. Chen, O. Shorinwa, W. Zeng, J. Bruno, P. Dames, and M. Schwager, “Splat-nav: Safe real-time robot navigation in gaussian splatting maps,” 2024.
  * [16] J. Yu, J. E. Low, K. Nagami, and M. Schwager, “Nerfbridge: Bringing real-time, online neural radiance field training to robotics,” 2023.
  * [17] L. Meng, J. Liu, W. Chai, J. Wang, and M. Q.-H. Meng, “Virtual reality based robot teleoperation via human-scene interaction,” _Procedia Computer Science_ , vol. 226, pp. 141–148, 2023, proceedings of International Conference on Biomimetic Intelligence and Robotics.
  * [18] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in _CVPR_ , 2018.


# Counterfactually Measuring and Eliminating Social Bias in Vision-Language
Pre-training Models

Yi Zhang  Beijing Jiaotong University, China<EMAIL_ADDRESS>,  Junyang
Wang  Beijing Jiaotong University, China<EMAIL_ADDRESS>and  Jitao Sang
Beijing Jiaotong University, China<EMAIL_ADDRESS>
###### Abstract.

Vision-Language Pre-training (VLP) models have achieved state-of-the-art
performance in numerous cross-modal tasks. Since they are optimized to capture
the statistical properties of intra- and inter-modality, there remains risk to
learn social biases presented in the data as well. In this work, we (1)
introduce a counterfactual-based bias measurement _CounterBias_ to quantify
the social bias in VLP models by comparing the [MASK]ed prediction
probabilities of factual and counterfactual samples; (2) construct a novel VL-
Bias dataset including 24K image-text pairs for measuring gender bias in VLP
models, from which we observed that significant gender bias is prevalent in
VLP models; and (3) propose a VLP debiasing method _FairVLP_ to minimize the
difference in the [MASK]ed prediction probabilities between factual and
counterfactual image-text pairs for VLP debiasing. Although _CounterBias_ and
_FairVLP_ focus on social bias, they are generalizable to serve as tools and
provide new insights to probe and regularize more knowledge in VLP models.

Fairness in Machine Learning, Responsible Artificial Intelligence

††ccs: Social and professional topics Computing / technology policy††ccs:
Applied computing Law, social and behavioral sciences††ccs: Computing
methodologies Machine learning

##  1\. Introduction

Figure 1. We propose to measure social bias in VLP model by comparing the
[MASK]ed prediction probabilities of factual and counterfactual samples.

The past few years have witnessed the rapid development of Vision-Language
Pre-training (VLP) models (Chen et al., 2020; Li et al., 2020; Cui et al.,
2021; Zhang et al., 2020), and task-specific finetune on VLP models has become
a new and state-of-the-art paradigm in many multimedia tasks (Tan and Bansal,
2019; Lu et al., 2019; Luo et al., 2021). Beyond accuracy, fairness which
concerns about the discrimination towards socially protected or sensitive
groups plays a critical role in trustworthy deployment of VLP models in
downstream tasks. This is highly endorsed by many VLP model designers (Li et
al., 2021). Yet, very limited studies have begun to examine the fairness
problem in VLP models, which typically apply the bias measurement designed for
the unimodal pre-training (e.g., text-based language models (Caliskan et al.,
2017)). The characteristics of VLP models regarding the interaction between
visual modality and textual modality are largely ignored. This work is thus
devoted to measuring and eliminating the social bias in VLP models by
emphasizing/addressing its multimodal characteristics, with the goal of
helping understand the existed social bias and inspiring the development of
fair VLP models.

Measuring the social bias in pre-training model essentially involves the
correlation between bias and target concepts. The cross-modal interaction of
VLP models poses two main challenges for bias measurement. The first challenge
is on modeling the bias and target concepts. In VLP models, the joint modeling
of visual modality delivers different embeddings w.r.t. image contexts, making
direct calculating the distance in the embedding space as in (Ross et al.,
2020) not applicable. Inspired by the Masked Language Modeling (MLM) pre-
training task commonly used in VLP models, we employ prompt-based query to
probe the modeling of bias and target concepts in VLP models. As illustrated
in Figure 1, the concepts to be modeled are masked before issuing into VLP
models. The resultant MLM prediction probability of [MASK]ed token
$P(\text{[MASK]=``\emph{shopping}''})$ manifests VLP model’s modeling of the
concept given the input image-text pair.

The second challenge is on quantifying the bias-target association. With MLM
prediction probability for concept probing, a natural idea is to compare the
prediction probabilities of the target concept in the observed image-text
pairs of different bias concepts (e.g., the difference of
$P(\text{[MASK]=``\emph{shopping}''})$ between _male_ and _female_ inputs.
However, relying on the observed data suffers from two problems: (1) There
exists no adequate number of image pairs with only different bias concepts.
For example, when measuring the social bias of “ _shopping_ ” over gender, it
is difficult to collect pairs of identical shopping images except for
different genders. Other information beyond gender like the different
background concepts can largely affect the MLM prediction probability, as
detailed discussed in Section 3 for the example in Figure 2. (2) The bias
measurement result is highly sensitive to the observed dataset. Examining
image-text datasets with different distributions will lead to inconsistent
observations. To address this, we propose a counterfactual-based bias measure
method (CounterBias) to quantify the social bias in VLP models exempted from
the limitation in observed data pairs. Specifically, we design a model-
specific adversarial attack scheme to generate counterfactual images by
altering the bias concept from factual images (e.g., attacking _woman_ to
_man_ as illustrated in Figure 1). This successfully alters the bias concept
without changing other information. The resultant MLM prediction difference
between factual and counterfactual image-text pairs thus manifests the social
bias of the examined VLP models.

Since there exists no off-the-shelf dataset specially designed to analyze the
social bias in VLP models, we propose VL-Bias, a dataset to facilitate the
study on understanding model bias in VLP models. VL-Bias contains 52
activities and 13 occupations with a total of 24k image-text pairs. Using
_CounterBias_ , we examined two typical VLP architectures, dual-stream and
single-stream, on VL-Bias dataset. Several key observations include: (1)
Social bias is prevalent in the examined three VLP models, and exists in both
visual and language modalities; (2) The gender bias contained in VLP models is
basically consistent with human gender stereotypes; (3) The examined single-
stream ViLT and dual-stream ALBEF/TCL show different cross-modal conformities
regarding the observed unimodal social bias; (4) The examined VLP models
contain stronger gender bias than unimodal pre-training model BERT.

Inheriting the idea of measuring social bias, we propose a simple
counterfactual-based method to eliminate the social bias in VLP models.
Specifically, we first generate counterfactual samples of bias concepts in
visual and language modalities, respectively. Then, the difference in the
predicted [MASK]ed probability between factual and counterfactual image-text
pairs is minimized, so that preventing the model from learning the association
between bias and target concepts. Experimental results demonstrate the
effectiveness in benchmark datasets.

Our contributions can be summarized as follows:

  * •

We propose to employ model-specific adversarial attacks on bias concepts to
generate counterfactual samples, and counterfactually measure social bias at
instance-level by comparing the [MASK]ed prediction probabilities of factual
and counterfactual samples.

  * •

We introduce VL-Bias, a dataset for studying gender bias in VLP models.
Observations are derived regarding typical VLP architectures, the consistency
with human stereotypes, and the comparison with unimodal pre-training models.

  * •

We propose a counterfactual bias elimination method, which eliminates bias by
minimizing [MASK]ed probability discrepancy between factual and counterfactual
samples.

Figure 2. Example “shopping” images with their predicted _factual target
probability_ from ALBEF. The top and second row correspond to images with
female and male bias concepts respectively.

##  2\. Background and Related Works

###  2.1. Vision-Language Pre-training(VLP)

Benefited from the cross-modal interactions performed by the transformer
(Vaswani et al., 2017), Vision-Language Pre-training (VLP) has improved
performance on various joint vision-and-language(V+L) downstream tasks
(Radford et al., 2021; Hu et al., 2020; Lu et al., 2019; Suhr et al., 2019;
Jia et al., 2021). There are two mainstream architectures for bridging the
cross-modal semantic gap: dual-stream architecture and single-stream
architecture. The former first encodes visual and language modalities
separately and then fuses the two representations through a transformer
network; the latter collectively operates on a concatenation of image and text
inputs. In the work, we consider ALBEF (Li et al., 2021) and TCL (Yang et al.,
2022) as example to examine the former, and ViLT (Kim et al., 2021) as example
to examine the latter.

Figure 3. The proposed _CounterBias_ to counterfactually measure social bias
between target concept $T$ and bias concept $B$.

###  2.2. Bias Measurement and Elimination in VLP

Much research work (Zhao et al., 2019; Caliskan et al., 2017) has been done on
evaluating bias in unimodal language models(LM). (Caliskan et al., 2017)
proposed cosine-based word embedding bias measure, and showed that word
embeddings like GloVe (Pennington et al., 2014) encode gender biases.
Furthermore, (Srinivasan and Bisk, 2021) investigates two unimodal biases in
VL-BERT (Su et al., 2019). However, there exists very limited work for such
study in multimodal bias of VLP due to the cross-modal interaction. As far as
we know, only (Ross et al., 2020) applied the cosine-based word embedding bias
measure to multimodal bias of VLP models. However, (Kurita et al., 2019)
proved this method is not suitable for the contextual setting, as direct word
will be affected by the context of image and text, resulting in inconsistent
measure result. This motivates us to design bias measure scheme to consider
the multimodal nature of the input in VLP models. For bias elimination, there
has been works (Sun et al., 2019; Webster et al., 2020) focused on eliminating
bias in unimodal language models. However, eliminating the social bias in VLP
models remain unexplored till now.

##  3\. Counterfactually Measuring Social Bias in VLP

The pre-training task of Masked Language Modeling (MLM) has been recognized as
confidentially manifesting VLP model behavior by maximizing the mutual
information(MI) between [MASK]ed token and the image-text input (Li et al.,
2021), and (Kurita et al., 2019; Srinivasan and Bisk, 2021) utilized [MASK]ed
token to probe the model’s response of its context to measure unimodal
contextual bias. These provide theorectical basis and inspire us to use the
prediction of [MASK]ed token to probe VLP model’s understanding of the bias
and target concepts. Specifically, when we construct the input to VLP model
with image and text template masking off certain target concept $T$ (e.g.,
template of “the _woman_ is [MASK]” masking off the activity concept “
_shopping_ ” in Figure 1), the output prediction probability can indicate how
VLP model probes the concept from the given factual image-text pair. We denote
the calculation of _factual target probability_ as follows:

(1) $P(T\mid V\text{-}L_{f})=P(\text{[MASK]}=T|V_{f},L_{f})$

Social bias essentially refers to how the change of bias concept in the input
influences the model’s inference on the target concept. Note that ideally only
the involved bias concept is changed but fixing all the other information, so
as to accurately examine the inherent influence from changed bias concept to
target concept inference. However, as discussed in Section 1, it is almost
impossible to perfectly control and keep the other variables unchanged. Figure
2 shows some “ _shopping_ ” images with their predicted _factual target
probability_. We can see among images with the same bias concept (in the same
row), the diversity of other variables like environment and background objects
imposes obvious influence on the predicted target probability, even more
significant than the bias concepts (between the two rows). This indicates
relying on the observed data for bias measurement is unreliable. To avoid the
negative effects of image diversity as a confounder, we propose _CounterBias_
to counterfactually change the bias concepts from observed image-text pairs
without changing other information, and quantify the social bias by examining
the difference between the predicted target probability of observed factual
data $P(T\mid V\text{-}L_{f})$ and generated counterfactual data $P(T\mid
V\text{-}L_{{cf}})$.

Regarding how to counterfactually change the bias concepts in the visual
modality, a direct idea is to use GANs to edit images. However, It is
generally accepted that GANs-based editing can unintentionally change other
information(e.g., image background)(Jain et al., 2022; Jain, 2020; Grover et
al., 2019; Choi et al., 2020; Menon et al., 2020) , which defeats the purpose
of controlling other information unchanged.

Model-specific adversarial attacks. To avoid changing information other than
bias concept, we propose model-specific adversarial attacks to
counterfactually change the bias concepts in image. In addition to affecting
other information as little as possible, model-specific adversarial attacks
ensure that all bias features that the VLP model relies on are
counterfactually altered. Specifically, we create a bias prompt template with
masking the factual bias concept, “a [MASK] is in the picture”, as text input
to guide the VLP model to classify bias concept $B$ (e.g., _woman_) in the
image input. Then we use the prediction probability $P(B\mid V_{f})$ of the
[MASK]ed bias concept word to represent the bias concept that the model
extracts from the factual image $V_{f}$:

(2) $P(B\mid V_{f})=P(\text{[MASK]=B}|V_{f},\text{bias prompt}))$

where $B$ is the factual bias concept of input image. Similar to traditional
adversarial attacks (Kurakin et al., 2017; Goodfellow et al., 2015; Madry et
al., 2017), using cross-entropy loss between $P(B\mid V_{f})$ and
counterfactual goal $B^{\prime}$, and search the adversarial perturbation
$\delta$ based on gradient of VLP model so that bias concept of $V_{f}+\delta$
is close to $B^{\prime}$:

(3)
$\delta=\underset{\left\|\delta\right\|\leq\varepsilon}{\arg\min}\mathcal{L}\left(P(\text{[MASK]}|V_{f}+\delta,\text{bias
prompt}),B^{\prime}\right)$

where $B^{\prime}$ represents counterfactual goals, if $B$ is _woman_ ,
$B^{\prime}$ is _man_. And $V_{f}+\delta$ is generated counterfactual image
$V_{cf}$.

Regarding how to counterfactually change bias concepts in language modality,
we manually replace the bias concept word in factual text $L_{f}$, e.g.
gendered word _woman_ is replaced by _man_. The generated counterfactual text
is denoted as $L_{cf}$.

Note that the bias concept is continuous in images rather than discrete, e.g.,
the intensity of gender features in images of _woman_ is continuous, so the
change of $P(B|V\text{-}L)$ also needs to be considered in the measure of
social bias. The bias concepts in factual and counterfactual images, $P(B\mid
V_{f})$ and $P(B\mid V_{cf})$, are formulated as the predicted probability of
the [MASK]ed biased concept word, similar to Equation 2. Considering that the
bias concept in text is binarized, $P(B\mid L_{f})$ can thus be directly
defined as $1$, and $P(B\mid L_{cf})$ is $0$. The bias concept $P(B\mid V,L)$
in the image-text pairs can be formulated as the average of the $P(B\mid V)$
and $P(B\mid L)$.

Once _factual probability_ and _counterfactual probability_ in target concept
and bias concept are calculated, we can measure how the counterfactual change
of bias concept influences the model’s inference on the target concept, as
summarized in Figure 3. Specifically, the involved social bias between bias
concept $B$ and target concept $T$ can be measured as:

(4) $bias(T,B)=\frac{P(T\mid V\text{-}L_{cf})-P(T\mid V\text{-}L_{f})}{P(B\mid
V\text{-}L_{cf})-P(B\mid V\text{-}L_{f})}$

Considering the multi-modal characteristic of VLP, we are interested in
measuring social bias both in multimodality, and also in visual and language
unimodality. The difference between measuring multimodal and unimodal biases
lies in the counterfactual samples used, i.e., the difference in $P(T\mid
V\text{-}L_{cf})$ and $P(B\mid V\text{-}L_{cf})$ calculations. The following
two subsections described measuring multimodal bias and unimodal bias in
detail.

###  3.1. Measuring Multimodal Bias

Multimodal Bias refers to how the change of bias concept in both the visual
and language modalities influences the model’s inference on the target
concept. Given counterfactual image-text pair $(V_{cf},L_{cf})$ as input, we
denote the calculation of _counterfactual target probability_ as follows:

(5) $P(T\mid V_{cf},L_{cf})=P(\text{[MASK]}=T|V_{cf},L_{cf})$

The multimodal bias $bias_{V,L}(T,B)$ can be formulated as:

(6) $bias_{V\text{-}L}(T,B)=\frac{P(T\mid V_{cf},L_{cf})-P(T\mid
V\text{-}L_{f})}{P(B\mid V_{cf},L_{cf})-P(B\mid V_{f},L_{f})}$

$bias_{V,L}(T,B)$ is an instance-level bias measure for model bias. Positive
values indicate target concept $T$ biased towards $B$, and negative values
indicate that the bias direction of $T$ is opposite to $B$. Further, we also
measure dataset-level social bias, by applying indicator function to align the
bias directions represented by positive values, and calculate the average bias
over a set of instances $S_{T}$ which contain the specific target concept $T$:

(7)
$bias_{V\text{-}L}(T)=\mathbb{E}_{S_{T}}[\mathbbm{1}_{B=B_{0}}bias_{V\text{-}L}(T,B)-\mathbbm{1}_{B=B_{1}}bias_{V\text{-}L}(T,B)]$

where $B_{0}$ and $B_{1}$ are pre-defined, and the sign of
$bias_{V\text{-}L}(T)$ indicates the bias direction - positive for $B_{0}$,
and negative for $B_{1}$. If $B_{0}$ is _male_ , positive values indicate $T$
biased towards _male_.

###  3.2. Measuring Unimodal Bias

Unimodal bias refers to how the change of bias concept in unimodality
influences the model’s inference on the target concept. We use counterfactual
images and counterfactual texts to discover visual unimodal bias and language
unimodal bias, respectively.

For visual unimodal bias, we calculate the _counterfactual target probability_
from given counterfactual visual $V_{cf}$ input and factual language input
$L_{f}$ as follows:

(8) $P(T\mid V_{f},L_{cf})=P(\text{[MASK]}=T|V_{cf},L_{f})$

Then, visual bias $bias_{V}(T,B)$ can be calculated as follows:

(9) $bias_{V}(T,B)=\frac{P(T\mid V_{cf},L_{f})-P(T\mid
V\text{-}L_{f})}{P(B\mid V_{cf})-P(B\mid V_{f})}$

For the language unimodal bias, we calculate the _counterfactual target
probability_ from given counterfactual visual $V_{f}$ input and factual
language input $L_{cf}$ as follows:

(10) $P(T\mid V_{f},L_{cf})=P(\text{[MASK]}=T|V_{f},L_{cf})$

Then, language bias $bias_{L}(T,B)$ can be calculated as follows:

(11) $bias_{L}(T,B)=\frac{P(T\mid V_{f},L_{cf})-P(T\mid
V\text{-}L_{f})}{P(B\mid L_{cf})-L(B\mid V_{f})}$

Figure 4. Example images in the collected VL-Bias dataset. Table 1. Templates
for caption generation.

Type | Template
---|---
Sentence w/o specification | The {gender} is [MASK]
Sentence w/ specification | The {gender} in the photo is [MASK]
Phrase w/o specification | A {gender} who is [MASK]
Phrase w/ specification | A photo of a {gender} who is [MASK]

Figure 5. The human gender stereotypes (Left) and the observed multimodal bias
(B@V-L) of three examined VLP models on different activities (Right).

##  4\. Measuring Results

###  4.1. VL-Bias Dataset

Fairness research in CV mainly focuses on human face analysis (Zhang and Sang,
2020; Wang et al., 2019; Huang et al., 2021), since existing face datasets
such as CelebA (Liu et al., 2015) and UTKface (Sun et al., 2017) have detailed
annotation of both bias and target concepts of faces. For fairness in NLP
models, researchers can measure the social bias of NLP models by means of
manually constructing language templates (Sun et al., 2020). However, in
vision-language research bridging CV and NLP, few studies focus on fairness
problems due to the lack of both bias and target concepts annotation.

To address this issue, we construct VL-Bias to facilitate the study on
analyzing social bias in VLP models. Using gender as the bias concept example,
we selected 52 activities and 13 occupations related to humans111The VL-Bias
dataset, as well as the complete activity and occupation concept list, is
available at https://github.com/VL-Bias/VL-Bias .

For image collection, in addition to downloading human-related images from the
Internet, we also collected human-related images from the existing image
datasets, such as MS-COCO (Lin et al., 2014), Flickr (Young et al., 2014).
Then, we annotated gender concepts, activity concepts, and occupation concepts
of the images and removed the images that did not explicitly contain these
concepts. We illustrate some annotated examples in Figure 4.

We also generate corresponding captions for each image according to the
annotation. Consider different languages and grammars have different effects
on the prompt of [MASK], we use four templates 222We considered sentence and
phrase, and whether to use ”A photo of” in the text to specify that the text
is about the image (Radford et al., 2021). described in Table 1 to generate
captions. Finally, for each template, we have collected 24k image-text pairs,
including 13K for the 52 activities and 11K for the 13 occupations.

Figure 6. Example of the calculated social bias in VLP model ALBEF.

###  4.2. The Observed Social Bias in VLP

Current VLP models can be divided into dual- and single- stream models
according to architectures (Kim et al., 2021). In dual-stream VLP models, the
visual and textual inputs are first processed by two uni-modal encoders
respectively, then the resulting representations are fed into cross-modal
Transformer layers. For single-stream, VLP models use the concatenation of the
image-text pair directly as the input of cross-modal Transformer layers. To
deeply analyze the social bias in VLP models, we selected the representative
VLP models of these two architectures respectively. For dual-stream, we
consider ALBEF and TCL, and for single-stream, we consider ViLT. Taking gender
bias as an example, we evaluate each VLP model on our VL-Bias dataset.

Using the multimodal bias measurement as example, given target concept $T$ and
bias concept $B$, instance-level social bias can be calculated according to
Equation 6. One example is illustrated in Figure 6. We can see the instance-
level measurement provides intuitive understanding of the involved social
bias. By aggregating instances with the same target concept, we can derive the
final dataset-level social bias according to Equation 7. The following
elaborates some of the key observations.

Figure 7. The visual modal bias (B@V) and language modal bias (B@L) in dual-
stream ALBEF and single-stream ViLT. Table 2. The social bias (in $\%$) in VLP
models in terms of multi-modal bias(B@V-L), visual modal bias(B@V), and
language modal bias(B@L). For each bias, we used four text templates to
measure bias and calculate the average(Avg.).

|  | Activity 13K | Occupation 11K
---|---|---|---
|  | B@V-L | B@V | B@L | B@V-L | B@V | B@L
ALBEF | S${}_{\text{W/O}}$ | 10.29 | 9.44 | 3.98 | 13.31 | 9.62 | 2.38
S${}_{\text{W/}}$ | 11.33 | 6.54 | 7.15 | 16.00 | 10.46 | 6.46
P${}_{\text{W/O}}$ | 10.75 | 8.21 | 4.27 | 11.85 | 7.85 | 1.69
P${}_{\text{W/}}$ | 9.85 | 7.54 | 3.31 | 11.31 | 8.23 | 2.08
Avg. | 10.56 | 7.93 | 4.68 | 13.12 | 9.04 | 3.15
TCL | S${}_{\text{W/O}}$ | 7.62 | 6.96 | 3.67 | 12.00 | 10.85 | 3.92
S${}_{\text{W/}}$ | 8.94 | 6.15 | 5.50 | 16.46 | 12.01 | 6.00
P${}_{\text{W/O}}$ | 7.52 | 5.90 | 2.65 | 10.15 | 6.85 | 4.77
P${}_{\text{W/}}$ | 6.88 | 5.38 | 2.37 | 8.38 | 5.46 | 2.85
Avg. | 7.74 | 6.10 | 3.55 | 11.75 | 8.79 | 4.84
ViLT | S${}_{\text{W/O}}$ | 8.62 | 6.52 | 2.37 | 16.15 | 17.23 | 3.62
S${}_{\text{W/}}$ | 9.48 | 8.71 | 2.35 | 14.92 | 17.35 | 4.62
P${}_{\text{W/O}}$ | 14.65 | 14.38 | 2.27 | 12.92 | 17.61 | 3.38
P${}_{\text{W/}}$ | 11.19 | 11.85 | 1.69 | 13.23 | 15.15 | 4.46
Avg. | 10.99 | 10.37 | 2.17 | 14.31 | 16.84 | 4.02

Social bias is prevalent in the examined VLP models. We report the gender bias
of the 52 activities and 13 occupations in Table 2, where each element
represents the average absolute value of the measured bias results. Main
observations include: (1) all three VLP models involve with significant gender
bias. Taking multimodal bias (B@V-L) of ALBEF in activities as an example,
$10.56\%$ means that by altering the gender information of the input, the
model’s predicted probability of [MASK]ed activity will change by an average
of $10.56\%$. (2) Gender bias exists in both modalities of VLP model and is
more serious in visual modality (B@V) than in language modality(B@L). We
conjecture that this is because images have richer information than text,
which leads the model to capture more statistical properties within the visual
modality. This further leads to stronger social bias in visual modality than
in language modality. (3) Bias measurement is sensitive to the choice of text
input template, and using the average(Avg. in Table 2) of multiple templates
as the measurement result is more generalized.

The gender bias contained in VLP models is basically consistent to human
gender stereotypes. To investigate whether the social bias involved in VLP
models matches that from human stereotypes, we recruited 100 Amazon Mechanical
Turk workers for labeling the stereotypes for the 52 activities and retained
19 activities with agreed gendered stereotyped label 333 For each activity, we
asked 100 distinct AMT workers to label it as _male-biased_ , _female-biased_
or _neutral_. An activity obtains gender-biased stereotypes label only when at
least $90\%$ works offer the same label. The rest 33 activities are associated
with neutral label in the resultant human stereotype. (_male_ or _female_
-biased, illustrated in Figure 5 (Left)).

In Figure 5 (Right), we plot the multimodal bias (B@V-L) of examined VLP
models with each activity, marked with human stereotypes using colored
points(red for male-biased, green for female-biased, gray for neutral). The
sign of B@V-L indicates the direction of gender bias, positive for “male,” and
negative for “female”. We observed that the social bias B@V-L manifests is
basically consistent with human gender stereotypes, which indicates that VLP
models inherit human stereotypes from the training data. In the three examined
VLP models, compared to ALBEF and TCL, ViLT slightly deviates from human
stereotypes in some of the activities, such as _”skating”_. This may be
related to the inconsistency in the bias direction between visual and language
modalities in the following analysis.

The examined dual-stream ALBEF/TCL and single-stream ViLT show different
cross-modal conformities regarding the observed unimodal social bias. Unlike
unimodal pre-training models, there are complex vision-language cross-modal
interactions in VLP models. The relationship between the social bias in two
modalities may be of interest. Taking dual-stream ALBEF and single-stream ViLT
as examples, in Figure 7, we use scatter plots to compare the difference
between visual modal bias ($x$-axis) and language modal bias ($y$-axis). Each
point corresponds to an activity, with three colors representing different
human stereotypes. For ALBEF (Figure 7(a)), it is easy to find the strong
conformity between visual modal bias and language modal bias: all activities
are basically located in the first and third quadrants of the coordinate
system, which means that the discovered gender biases in two modalities follow
the same direction. Moreover, the bias of two modalities in $84\%$ of
activities is consistent with human stereotypes. However, the bias in ViLT is
different from ALBEF, as shown in Figure 7(b), visual modal bias and language
modal bias have large differences in direction, whit activities widely
spreading across the four quadrants. Bias in unimodality is also inconsistent
with human stereotypes in some activities, such as visual bias in _skating_ is
also inconsistent with humans.

The conformity between visual modal bias and language modal bias in dual-
stream ALBEF is possibly attributed to the explicit align constraints between
the two unimodal representations (i.e., Image-Text Contrastive Learning pre-
training task). By aligning the two modalities in dual-stream, ALBEF forces
the two modalities to learn only shared information between visual and textual
inputs, resulting in more consistent social biases contained in the two
modalities, and biases in both two modalities are also consistent with human
stereotypes. Moreover, the inconsistency between the learned information of
the two modalities in ViLT again explains the reason for the slight deviation
of multimodal bias(B@V-L) and human stereotypes.

Table 3. The social bias in language modality of three VLP models and language model BERT. Target concept | ALBEF | TCL | ViLT | BERT
---|---|---|---|---
52 Activities | 0.53 | 0.59 | 0.39 | 0.15
13 Occupations | 0.69 | 0.71 | 0.48 | 0.20

The examined VLP models contain stronger gender bias than unimodal pre-
training model BERT. It is common practice for VLP models to use BERT (Devlin
et al., [n.d.]) as backbone. We investigate the difference in language modal
bias between BERT-based VLP models and original BERT.

For a fair comparison, we remove visual input when predicting [MASK]ed target
concepts, which ensures that both VLP models and BERT rely only on language
input:

(12) $\displaystyle P(T\mid
L_{f})=P(\text{[MASK]}=T|V_{\text{\small{none}}},L_{f})$ $\displaystyle
P(T\mid L_{cf})=P(\text{[MASK]}=T|V_{\text{\small{none}}},L_{cf})$

Then, considering that the absence of visual input may result in a lower value
of $P(T\mid L_{f/cf})$, we use log probability for stable numerical
computations:

(13) $bias_{L}(T,B)=\frac{\log P(T\mid L_{cf})-\log P_{L}(T\mid
L_{f})}{P(B\mid L_{cf})-P(B\mid L_{f})}$

For both VLP models and BERT, we use Equation 13 to calculate language modal
bias. We report the language modal bias of the 52 activities and 13
occupations in Table 3. The results show that the examined three VLP models
contain stronger gender bias than unimodal pre-training model BERT. This
suggests that learning of cross-modal representations generally exacerbates
the dependence of social biases in language modality.

Figure 8. The overall framework of our proposed _FairVLP_.

##  5\. Counterfactually Eliminating Social Bias in VLP

###  5.1. Method

Section 4 observes that social bias is prevalent in the examined VLP models,
which can lead to unintended social consequences in downstream tasks.
Inheriting the idea of counterfactually measuring social bias, we propose a
fair Vision-Language Pre-training task _FairVLP_ that eliminates social bias
by minimizing the difference in the predicted [MASK]ed probability between
factual and counterfactual image-text pairs.

Following the setup of masked language modeling (MLM) pre-training tasks
commonly used in VLP models, our _FairVLP_ is applicable to both single- and
dual-stream structures without being restricted to a specific VLP
architecture. For the image-text pairs input at each training step $i$, we
randomly mask out the text input tokens with probability $q$ and replace them
with the special token [MASK]. The image and [MASK]ed text are denoted as
$V^{i}_{f}$ and $L^{i}_{f}$.

Counterfactual samples generation. Similar to Section 3, for factual image
$V^{i}_{f}$, we use model-specific adversarial attack to generate
counterfactual image $V^{i}_{cf}$ by altering the bias concept of $V_{f}$. And
we directly replace the words about bias concept (e.g., _man_ is replaced to
_woman_) in factual text $L^{i}_{f}$ to generate counterfactual text
$L^{i}_{cf}$.

Minimize the difference in the prediction of [MASK]ed words between factual
and counterfactual samples. Following the formulation of MLM, we can utilize
both the image $V$ and the text $L$ to predict [MASK]ed words
$P(\text{[MASK]}\mid V,L)$. As shown in Figure 8, we use factual and
counterfactual samples to predict $P(\text{[MASK]}\mid V^{i}_{f},L^{i}_{f})$
and $P(\text{[MASK]}\mid V^{i}_{cf},L^{i}_{cf})$.

At current training step, our _FairVLP_ method eliminates social bias in VLP
models by minimizing the Kullback-Leibler(KL) divergence in [MASK]ed word
prediction distributions between the factual and counterfactual image-text
pairs:

(14) $\mathcal{L}_{KL}=\mathcal{D}_{KL}(P(\text{[MASK]}\mid
V^{i}_{f},L^{i}_{f})\|P(\text{[MASK]}\mid V^{i}_{cf},L^{i}_{cf}))$

With the basic cross-entropy loss $\mathcal{H}$ for the factual and
counterfactual samples in MLM task:

(15) $\displaystyle\mathcal{L}_{\mathrm{mlm}}$
$\displaystyle=\mathcal{H}(\boldsymbol{M}^{i},P(\text{[MASK]}\mid
V^{i}_{f},L^{i}_{f}))$
$\displaystyle+\mathcal{H}(\boldsymbol{M}^{i},P(\text{[MASK]}\mid
V^{i}_{cf},L^{i}_{cf}))$

where $\boldsymbol{M}^{i}$ is the [MASK]ed word, and the final training
objective is to minimize $L_{FairVLP}$:

(16)
$\mathcal{L}_{\emph{FairVLP}}=(1-\alpha)\mathcal{L}_{\mathrm{mlm}}+\alpha\cdot\mathcal{L}_{KL}$

where $\alpha$ is the coefficient weight to control $\mathcal{L}_{KL}$.

By iteratively generating counterfactual samples and minimizing
$\mathcal{L}_{\emph{FairVLP}}$, The generated counterfactual samples
$V^{i}_{cf}$ and $L^{i}_{cf}$ are guaranteed to have counterfactual properties
for the VLP model at training step $i$. Minimizing the difference in
prediction between factual and counterfactual samples, as a result, the VLP
model’s inference on target concept is independent of the bias concept at
training step $i$. Throughout pre-training process, _FairVLP_ consistently
prevents the VLP from learning social bias.

Table 4. Social bias(in $\%$) in VLP models with different debiasing pre-
training tasks.

Method | Activity 13K | Occupation 11K
---|---|---
B@V-L | B@V | B@L | B@V-L | B@V | B@L
vanilla | 13.02 | 11.10 | 4.89 | 14.84 | 14.23 | 3.15
GS | 11.21 | 9.25 | 4.03 | 12.47 | 12.35 | 2.98
DR | 11.17 | 9.54 | 4.39 | 13.52 | 12.19 | 3.08
CycleGAN | 10.92 | 9.23 | 4.32 | 12.07 | 11.35 | 3.01
FairVLP-V | 8.41 | 7.23 | 3.91 | 9.11 | 7.98 | 2.86
FairVLP-L | 8.53 | 7.46 | 3.82 | 11.23 | 9.61 | 1.77
FairVLP | 6.97 | 6.76 | 3.29 | 7.74 | 7.15 | 1.72

Table 5.  Fine-tuned image-text retrieval results on Flickr30K and COCO
datasets.

Method | Flickr30K (1K test set) | MSCOCO (5K test set)
---|---|---
TR | IR | TR | IR
| R@1 | R@5 | R@10 | R@1 | R@5 | R@10 | R@1 | R@5 | R@10 | R@1 | R@5 | R@10
Vanilla | 90.8 | 98.6 | 99.3 | 79.4 | 94.9 | 97.2 | 70.5 | 89.9 | 94.6 | 53.5 | 79.3 | 87.2
FairVLP | 90.8 | 98.8 | 98.3 | 78.9 | 94.8 | 97.3 | 70.3 | 90.3 | 94.9 | 53.6 | 79.3 | 87.1

###  5.2. Experimetnal Results

Experiment Setup. We follow dual-stream ALBEF architecture and common MLM and
ITM (image-text matching) pre-training tasks to pretrain debiasing VLP model
on the MS-COCO dataset. To evaluate the debiasing performance of _FairVLP_ ,
we consider two common and effective debiasing methods in language models as
baselines: Gender Swapping(_GS_) (Zhao et al., 2018) and Dropout
Regularization(_DR_) (Webster et al., 2020). _GS_ swaps gendered words in text
input for mitigating the correlation between gender and other concepts. _DR_
eliminates model over-fitting to gender concepts by increasing dropout rate.
We migrated the idea of _GS_ to a visual modality and edited the concept of
face gender in the images using _CycleGAN_ (Zhu et al., 2017). We also
consider two variants of our _FairVLP_ for ablation study, eliminating bias in
visual modality by using only visual counterfactual samples(_FairVLP-V_) and
eliminating bias in language modality by only using language counterfactual
samples(_FairVLP-L_). For all methods, the learning rate is 1e-4 with 0.02
weight decay, and the batch size is 256.

Debiasing performance. Table 4 summarizes the debiasing performance of
different methods measured on the VL-Bias dataset. Main observations include:
(1) All the proposed three settings of _FairVLP_ obtain remarkable debiasing
performance than that of baselines. Among three settings, by simultaneously
performing counterfactual debiasing on both two modalities, _FairVLP_
demonstrates the best performance in eliminating social bias. (2) _SW_ and
_CycleGAN_ show very limited debiasing results due to the lack of minimizing
the difference in prediction of [MASK]ed word between the original sample and
gender-swapping sample. (3) _DR_ underperforms _SW_ , we owe this result to
the random dropout does not change the statistical relationship in the
training data.

Moreover, We observe that _FairVLP-V_ , which is designed to eliminate the
bias of visual modality, not only achieves a significant debiasing effect in
the visual modality, but also demonstrates a remarkable debiasing effect in
language modality. And _FairVLP-L_ also performs debiasing for visual
modality. We conjecture that this is because gender conflict in image-text
pairs is an implicit form of data augmentation, which effectively makes the
modeling of other concepts independent of gender concept.

Downstream task performance. For the downstream tasks of the pre-training
model, we follow the most commonly used Image-Text Retrieval among V+L tasks
for model performance evaluation. We evaluate pre-trained models on the
Flickr30K and COCO benchmarks, and fine-tune the pre-trained model using the
training samples from each dataset.

Table 5 report results on fine-tuned image-text retrieval, respectively. Our
_FairVLP_ pre-training task achieves comparable if not better performance with
_Vanilla_. This suggests that _FairVLP_ can leverage reasonably constructed
counterfactual samples to effectively eliminate social bias without affecting
the model’s learning of reliable knowledge. This phenomenon further indicates
that the performance of the downstream task can be compatible with social bias
elimination.

Figure 9. Illustration of discovering spurious correlation by _CounterBias_.

##  6\. Discussion

Visual-Language Pre-training models are designed to address the lack of
downstream task-specific data, so the key to performance of downstream task is
the learning of reliable cross-modal knowledge in pre-trained models. In
addition to revealing social bias in VLP models, we discuss that the proposed
_CounterBias_ provides new insights into the opaque representation of
knowledge in VLP models. In this spirit, we examined the ability of
_CounterBias_ to discover correlations between two arbitrary concepts in VLP
models. Specifically, taking ALBEF as an example, we use _CounterBias_ to
measure the correlation between animal concepts (e.g., horse and cow) and
background concepts (e.g., grass and ground). As shown in Figure 9, the MLM
prediction difference between factual and counterfactual image-text pairs well
reflects the correlation learned in ALBEF which shows that the model learns
the spurious correlation between horse and grass, as well as cow and grass.
Accordingly, with available human annotation for spurious correlations,
_FairVLP_ can also be considered as a method for regularizing the model to
reliably learn knowledge.

##  7\. Conclusion

In this work, we propose to measure social bias in VLP models by comparing how
the model of target concept changes factual and counterfactual samples with
different bias concepts. A novel VL-Bias dataset is introduced for studying
social bias in multimodal models. Taking gender as an example, our bias
measure results on VLP models indicate that gender bias is prevalent in the
examined VLP models. We propose to counterfactually eliminate social bias via
minimizing the modeling differences between factual and counterfactual samples
with different bias concepts. In the future, we are working towards
counterfactually probing more knowledge as well as regulating the learning of
VLP models.

###### Acknowledgements.

This work is supported by the National Key R&D Program of China (Grant No.
2018AAA0100604).

## References

  * (1)
  * Caliskan et al. (2017) Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017\.  Semantics derived automatically from language corpora contain human-like biases.  _Science_ 356, 6334 (2017), 183–186.
  * Chen et al. (2020) Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. 2020\.  Uniter: Universal image-text representation learning. In _ECCV_.
  * Choi et al. (2020) Kristy Choi, Aditya Grover, Trisha Singh, Rui Shu, and Stefano Ermon. 2020.  Fair generative modeling via weak supervision. In _International Conference on Machine Learning_. PMLR, 1887–1898.
  * Cui et al. (2021) Yuhao Cui, Zhou Yu, Chunqi Wang, Zhongzhou Zhao, Ji Zhang, Meng Wang, and Jun Yu. 2021.  ROSITA: Enhancing Vision-and-Language Semantic Alignments via Cross-and Intra-modal Knowledge Integration. In _Proceedings of the 29th ACM International Conference on Multimedia_. 797–806.
  * Devlin et al. ([n.d.]) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. [n.d.].  Bert: Pre-training of deep bidirectional transformers for language understanding.  ([n. d.]), 4171–4186.
  * Goodfellow et al. (2015) Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015.  Explaining and harnessing adversarial examples. In _3rd International Conference on Learning Representations_.
  * Grover et al. (2019) Aditya Grover, Jiaming Song, Ashish Kapoor, Kenneth Tran, Alekh Agarwal, Eric J Horvitz, and Stefano Ermon. 2019.  Bias correction of learned generative models using likelihood-free importance weighting.  _Advances in neural information processing systems_ 32 (2019).
  * Hu et al. (2020) Xiaowei Hu, Xi Yin, Kevin Lin, Lijuan Wang, Lei Zhang, Jianfeng Gao, and Zicheng Liu. 2020.  Vivo: Surpassing human performance in novel object captioning with visual vocabulary pre-training.  (2020).
  * Huang et al. (2021) Xiaowen Huang, Jiaming Zhang, Yi Zhang, Xian Zhao, and Jitao Sang. 2021.  Trustworthy Multimedia Analysis. In _Proceedings of the 29th ACM International Conference on Multimedia_. 5667–5669.
  * Jain (2020) Niharika Jain. 2020\.  _A Study on Generative Adversarial Networks Exacerbating Social Data Bias_.  Ph.D. Dissertation. Arizona State University.
  * Jain et al. (2022) Niharika Jain, Alberto Olmo, Sailik Sengupta, Lydia Manikonda, and Subbarao Kambhampati. 2022\.  Imperfect ImaGANation: Implications of GANs exacerbating biases on facial data augmentation and snapchat face lenses.  _Artificial Intelligence_ 304 (2022), 103652.
  * Jia et al. (2021) Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. 2021\.  Scaling up visual and vision-language representation learning with noisy text supervision. In _International Conference on Machine Learning_. PMLR, 4904–4916.
  * Kim et al. (2021) Wonjae Kim, Bokyung Son, and Ildoo Kim. 2021.  Vilt: Vision-and-language transformer without convolution or region supervision. In _International Conference on Machine Learning_. PMLR, 5583–5594.
  * Kurakin et al. (2017) Alexey Kurakin, Ian Goodfellow, and Samy Bengio. 2017\.  Adversarial examples in the physical world.  _5th International Conference on Learning Representations_ (2017).
  * Kurita et al. (2019) Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. 2019.  Measuring Bias in Contextualized Word Representations. In _Proceedings of the First Workshop on Gender Bias in Natural Language Processing_. 166–172.
  * Li et al. (2021) Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. 2021.  Align before fuse: Vision and language representation learning with momentum distillation.  _Advances in Neural Information Processing Systems_ 34 (2021).
  * Li et al. (2020) Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, et al. 2020\.  Oscar: Object-semantics aligned pre-training for vision-language tasks. In _European Conference on Computer Vision_. Springer, 121–137.
  * Lin et al. (2014) Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014.  Microsoft coco: Common objects in context. In _European conference on computer vision_. Springer, 740–755.
  * Liu et al. (2015) Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. 2015\.  Deep Learning Face Attributes in the Wild. In _Proceedings of International Conference on Computer Vision (ICCV)_.
  * Lu et al. (2019) Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019\.  Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks.  _Advances in neural information processing systems_ 32 (2019).
  * Luo et al. (2021) Jianjie Luo, Yehao Li, Yingwei Pan, Ting Yao, Hongyang Chao, and Tao Mei. 2021\.  CoCo-BERT: Improving Video-Language Pre-training with Contrastive Cross-modal Matching and Denoising. In _Proceedings of the 29th ACM International Conference on Multimedia_. 5600–5608.
  * Madry et al. (2017) Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2017\.  Towards deep learning models resistant to adversarial attacks.  _arXiv preprint arXiv:1706.06083_ (2017).
  * Menon et al. (2020) Sachit Menon, Alexandru Damian, Shijia Hu, Nikhil Ravi, and Cynthia Rudin. 2020.  Pulse: Self-supervised photo upsampling via latent space exploration of generative models. In _Proceedings of the ieee/cvf conference on computer vision and pattern recognition_. 2437–2445.
  * Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014.  Glove: Global vectors for word representation. In _Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)_. 1532–1543.
  * Radford et al. (2021) Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021\.  Learning transferable visual models from natural language supervision. In _International Conference on Machine Learning_. PMLR, 8748–8763.
  * Ross et al. (2020) Candace Ross, Boris Katz, and Andrei Barbu. 2020.  Measuring Social Biases in Grounded Vision and Language Embeddings.  _arXiv preprint arXiv:2002.08911_.
  * Srinivasan and Bisk (2021) Tejas Srinivasan and Yonatan Bisk. 2021.  Worst of both worlds: Biases compound in pre-trained vision-and-language models.  _arXiv preprint arXiv:2104.08666_ (2021).
  * Su et al. (2019) Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei, and Jifeng Dai. 2019\.  Vl-bert: Pre-training of generic visual-linguistic representations.  _arXiv preprint arXiv:1908.08530_ (2019).
  * Suhr et al. (2019) Alane Suhr, Stephanie Zhou, Ally Zhang, Iris Zhang, Huajun Bai, and Yoav Artzi. 2019\.  A Corpus for Reasoning about Natural Language Grounded in Photographs. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_. 6418–6428.
  * Sun et al. (2019) Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019\.  Mitigating Gender Bias in Natural Language Processing: Literature Review.  _Association for Computational Linguistics (ACL 2019)_ (2019).
  * Sun et al. (2017) Zeyu Sun, Jie M Zhang, Mark Harman, Mike Papadakis, and Lu Zhang. 2017.  Age Progression/Regression by Conditional Adversarial Autoencoder. In _IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE.
  * Sun et al. (2020) Zeyu Sun, Jie M Zhang, Mark Harman, Mike Papadakis, and Lu Zhang. 2020.  Automatic testing and improvement of machine translation. In _Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering_. 974–985.
  * Tan and Bansal (2019) Hao Tan and Mohit Bansal. 2019.  LXMERT: Learning Cross-Modality Encoder Representations from Transformers. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_. 5100–5111.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _Advances in neural information processing systems_ 30 (2017).
  * Wang et al. (2019) Mei Wang, Weihong Deng, Jiani Hu, Xunqiang Tao, and Yaohai Huang. 2019.  Racial faces in the wild: Reducing racial bias by information maximization adaptation network. In _Proceedings of the ieee/cvf international conference on computer vision_. 692–702.
  * Webster et al. (2020) Kellie Webster, Xuezhi Wang, Ian Tenney, Alex Beutel, Emily Pitler, Ellie Pavlick, Jilin Chen, Ed Chi, and Slav Petrov. 2020.  Measuring and reducing gendered correlations in pre-trained models.  _arXiv preprint arXiv:2010.06032_ (2020).
  * Yang et al. (2022) Jinyu Yang, Jiali Duan, Son Tran, Yi Xu, Sampath Chanda, Liqun Chen, Belinda Zeng, Trishul Chilimbi, and Junzhou Huang. 2022\.  Vision-Language Pre-Training with Triple Contrastive Learning.  _Proceedings of the IEEE conference on computer vision and pattern recognition_ (2022).
  * Young et al. (2014) Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. 2014\.  From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions.  _Transactions of the Association for Computational Linguistics_ 2 (2014), 67–78.
  * Zhang et al. (2020) Shengyu Zhang, Tan Jiang, Tan Wang, Kun Kuang, Zhou Zhao, Jianke Zhu, Jin Yu, Hongxia Yang, and Fei Wu. 2020.  Devlbert: Learning deconfounded visio-linguistic representations. In _Proceedings of the 28th ACM International Conference on Multimedia_. 4373–4382.
  * Zhang and Sang (2020) Yi Zhang and Jitao Sang. 2020.  Towards accuracy-fairness paradox: Adversarial example-based data augmentation for visual debiasing. In _Proceedings of the 28th ACM International Conference on Multimedia_. 4346–4354.
  * Zhao et al. (2019) Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, and Kai-Wei Chang. 2019\.  Gender Bias in Contextualized Word Embeddings. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , Vol. 1.
  * Zhao et al. (2018) Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018.  Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_. 15–20.
  * Zhu et al. (2017) Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017.  Unpaired image-to-image translation using cycle-consistent adversarial networks. In _Proceedings of the IEEE international conference on computer vision_. 2223–2232.