GLAS-PPE/2015-0629${}^{\underline{\rm{th}}}$ October 2015

The state of the art in Grid style data management is to achieve increased
resilience of data via multiple complete replicas of data files across
multiple storage endpoints. While this is effective, it is not the most space-
efficient approach to resilience, especially when the reliability of
individual storage endpoints is sufficiently high that only a few will be
inactive at any point in time. We report on work performed as part of
GridPP[1], extending the Dirac File Catalogue and file management interface to
allow the placement of erasure-coded files: each file distributed as N
identically-sized chunks of data striped across a vector of storage endpoints,
encoded such that any M chunks can be lost and the original file can be
reconstructed. The tools developed are transparent to the user, and, as well
as allowing up and downloading of data to Grid storage, also provide the
possibility of parallelising access across all of the distributed chunks at
once, improving data transfer and IO performance. We expect this approach to
be of most interest to smaller VOs, who have tighter bounds on the storage
available to them, but larger (WLCG) VOs may be interested as their total data
increases during Run 2. We provide an analysis of the costs and benefits of
the approach, along with future development and implementation plans in this
area. In general, overheads for multiple file transfers provide the largest
issue for competitiveness of this approach at present.

21st International Conference on Computing for High Energy and Nuclear Physics
(CHEP2015)
Okinawa, Japan

##  1 Introduction

WLCG[2] VOs have been distributing data across geographical and administrative
boundaries since the very start of the project. Distributed data is beneficial
for both performance and reliability - having data in many places means that
it is also near to more compute and other resources, but also means that you
can afford to lose access to one or more of those places without losing the
data itself. However, to our knowledge, no WLCG experiment data model has ever
broken with the orthodoxy that geographical data distribution implies integer
replication of data, one full copy per site. This is with stark contrast to
the internal mechanisms used at the server level, where RAID5[3] or 6 stripes
and parity-checks individual files. At the data centre level, mature
distributed storage systems such as HDFS[4, 5] and Ceph[7] support striping
and parity-resiliency of files across servers. Outside of WLCG, there are
successful implementations of the same mechanisms on a globally distributed
scale - the DDN WOS[8] system is one such example, but Tahoe-LAFS[9, 10]
provides an analogous model from a diametrically opposed political philosophy.

###  1.1 Erasure codes

Erasure codes (EC) are a general class of forward error correction codes[11],
designed to enable the recovery of the content of a message after a subset of
the symbols comprising it are lost. In general, most implementations are based
on Reed-Solomon[12] codes, which have the advantage that they are widely used,
implemented and optimised. Alternative, more modern, codes such as Turbo[15]
codes and LDPC codes provide better performance when transmitting data over a
channel, but achieve this via asymptotically amortised overheads as the stream
length grows. Similarly, Raptor[16] codes (and other fountain codes) provide a
probabilistic guarantee of stream recovery with any given number of symbols
received. By contrast, Reed-Solomon codes work in terms of a rigid
decomposition of the input data into a fixed number of symbols, or chunks, and
a fixed number of additional coding symbols (each of which guarantees to be
sufficient to recover from loss of any one of the original symbols) which is
better suited to data storage across an integer number of endpoints.

The original design of the Reed-Solomon algorithm was based on oversampling of
a polynomial. Treating the N chunks of input data as samples to fit an order N
polynomial, we can take any additional M samples (the coding chunks), such
that any subset of at least N chunks of the new data stream will reconstruct
the same polynomial (and hence allow reconstruction of the original N
samples). A strict implementation using this algorithm is extremely slow at
recovering lost data chunks, and at detecting errors in any chunks in the
sequence, and so more modern implementations use a Fourier domain approach
based on generating coefficients of a polynomial, rather than sample
values[13, 14].

One of the key advantages of modern Reed-Solomon implementations is that they
allow effectively arbitrary selection of the number of chunks in which to
section a piece of data, and the number of additional ‘coding chunks’
generated to provide resilience. That is, rather than being limited to
specific optimised levels of resilience (such as RAID6, which provides for up
to two chunks of data being lost per stripe), we can select any rational value
for our resilience and replication levels within reason. Recent suggestions by
the ATLAS VO that storage pressures may cause them to maintain less than 2
replicas of data at Tier-2s leave them with no other level of replication but
a single copy. With a rational value of ‘replication’ provided by erasure
coding, they could tailor their resilience to a finer degree, while still
guarding against loss of any single site. (In fact, as more than 90% of SEs
are available at any one time, it seems that replicating data twice may be a
significant overcommitment to resilience, given the probability of both
endpoints for a file being down at once.)

##  2 Implementation of a shim for Erasure Coding

Given the preceding section, the authors felt that the time was long due to
attempt to introduce geographically distributed erasure coding techniques to
WLCG computing infrastructure. While a more complete implementation would be a
deeply integrated component of standard tools, allowing direct remote file
access across distributed chunks, the topic of this paper is the initial,
proof-of-concept implementation. The principle here is to test how easily we
can introduce distributed erasure coding of files without deeply modifying
existing tools - the EC layer is simply a shim on top of existing data
management.

When originally planned, this project would have provided a shim based around
the LFC. However, LFCs are a strongly deprecated component for the large WLCG
experiments, and so we made the decision to change to a different data
management architecture. In this case we use the DIRAC platform for job
submission.

###  2.1 DIRAC File Catalog

DIRAC[17] is a job and data management suite developed principally by the LHCb
experiment. While originally developed solely for the LHCb VO, the project is
also in use by other, small, VOs. For example, and most importantly to the
authors, effort is being expended at Imperial College[18] to implement a
multiVO DIRAC service to serve our many non-HEP and other small VOs in the UK.
As part of the data management component, the DIRAC system provides a file
catalogue layer (Dirac File Catalogue, DFC), which allows for arbitrary
metadata to be applied to individual records (as key-value pairs).

###  2.2 ZFEC

As the DIRAC client and server are written in Python, and provide a Python
API, it was advantageous for us to use a similarly Python-compatible library
for the EC component of the shim. This requirement steered us towards the
zfec[19] library, an open-source implementation of Reed-Solomon erasure codes
originally developed as a component of the Taho-LAFS filesystem. Whilst the
kernel of zfec’s encoder is written in C, it provides a fully featured Python
API, which is easy to interface with. The other obvious alternative Reed-
Solomon implementation is the jerasure[20] library. However, at the time at
which work was progressing, the author of this library was forced to remove it
from public availability due to a patent challenge from the US company
StreamScale[21]. While the code is still used in several derived products
(including as the erasure coding library for the Ceph distributed object
store), we felt that the legal situation made it difficult for us to justify
use of the library at that time.

###  2.3 Overlay design

In the interest of keeping a proof-of-concept design simple, our shim treats
grid storage elements essentially as data archives. Files are assumed to be
retrieved from the Grid to allow for local file access, so we do not need to
support direct IO against the erasure coded data itself. While this mode of
data access was common only a few years ago on Grid compute environments, it
is increasingly the case that direct IO is used against even remote files by
jobs at a site. Implementing direct IO against an encoded file is
significantly more difficult, however, and is not supported directly by the
DIRAC file catalogue API, so we do not attempt it for this work.

With this restriction, our shim simply performs the erasure coding on the
local machine, generating separate temporary files for each chunk and the
identically-sized coding chunks.

On the DIRAC side, we wrap calls to the DFC API, creating a directory in the
DFC namespace with the filename requested by the user. Logically, we locate
the individual chunks within that directory, with names coded with the
standard zfec extensions for chunks (encoding the ordinal number of the chunk
in the coding vector, and the total number of chunks and coding chunks
expected). As an additional check, and to improve efficiency, we also add
metadata to the DFC entries (and directory entry), encoding the total number
of chunks (as ‘TOTAL’) and total number of non-coding chunks (as ‘SPLIT’), as
well as some versioning information in case of format changes.

Figure 1: *

Diagram of file splitting for 8 chunks plus 2 additional coding chunks, via
(Reed-Solomon) erasure coding.

Figure 2: *

Sample layout for a file split as 8 chunks plus 2 coding chunks (10 chunks
overall), distributed across a vector of 3 SEs (A to C)

Figure 3:  Layout scheme for shim

In order to distribute chunks most effectively, we retrieve a vector of all of
the $s$ Storage Element (SE) endpoints supporting the User’s VO. Placement is
performed as a round-robin loop over this vector, such that chunk 1 is
transferred to the first SE endpoint in the vector, and chunk $n$ to the $n$
$mod$ $s$th endpoint. While simple to implement, round-robin data placement
suffers from some issues, not least that the first endpoints in the vector
will tend to get more chunks over time (assuming that the endpoint vector
returned from the DFC is always ordered the same way), see figure [layout].
Only in the case where the number of chunks plus coding chunks is a multiple
of the available endpoints will all endpoints receive an equal distribution.
Additionally, we do not have any way to factor in geographical distribution in
our placement. It is likely that a mature placement algorithm would be best
targeted at distribution preferentially across SEs in a geographical region,
rather than across the entire world, regardless of network distance.

File retrieval follows the reverse path - the tool retrieves the list of all
files in the relevantly named directory in the DFC namespace, and then makes
local copies of them via the DIRAC file transfer API. The final file is
reconstructed from the on-disk copies of the chunks. As an optimisation, we
stop getting chunks as soon as we have enough to reconstruct the file (that
is, the same number of chunks as in the original file, disregarding coding
chunks).

###  2.4 Parallelism

The above algorithm is functional, but suffers from the disadvantage of being
purely serial in operation. As the overhead of erasure coding itself scales
with file size (and number of chunks and coding chunks), the serial version of
the tool has significant performance deficits compared to a classical file
transfer tool. Given that the file transfer operations themselves are
independent, we can parallelise that portion of get and put operations over a
number of threads. This is simply achieved by a work pool model at present,
where a user-defined set of worker threads are created, and consume file
transfer operations until enough chunks have been fetched in total. This has
the advantage that a single slow transfer does not necessarily limit the rate
at which the file can be retrieved, as the remaining threads can consume the
remaining transfer operations at the same time. In the limit where the number
of threads is equal to the number of chunks, we essentially select the N
fastest chunks out of the total stripe, retrieving the file as fast as the
network allows.

##  3 Results

Due to limitations on the supported operating systems for the lcg_utils
packages, development and testing of the shim were performed on a Scientific
Linux 6.5 guest virtual machine, hosted on VirtualBox on Windows 8. The well
known issues with virtual machine performance compared to bare metal have not
been addressed when comparing the measured IO performance in these benchmarks,
so they should be treated as purely relative measures of performance.
Performance was tested on both single and dual-core systems in order to
estimate the effect of contention between threads, but the difference between
the two systems was small ($<5$%) in almost all cases, so only the single-core
results are reproduced here.

Figure 4:  Scaling performance of file upload for a 768kB file encoded as 10
chunks + 5 coding chunks, with increasing parallelism.

Figure 5:  Scaling performance of file upload for a 2.4GB file encoded as 10
chunks + 5 coding chunks.

Considering the performance for file upload from the client machine first, we
see that for both small (fig 5) and large (fig 5) files, parallelism does
provide a performance improvement with respect to the single-threaded case.
For small files, parallelism improves performance to better than that of the
single-threaded case for a file split into 10 pieces with no encoding, but we
do not see the same effect for larger files. This is clearly an Amdahl’s
Law[22] effect: for small files, file transfer latency dominates the process,
and this is distributed over our parallel transfer threads; for larger files,
perhaps closer to the size of physics data, the file encoding time is the
dominant component, and this is not parallelised in our model. For smaller
files, in fact, the dominant part of the file transfer itself is channel setup
and communication, rather than data transfer. From table 1, we can see that
for a small file split into 10 chunks (without encoding), the time for each
chunk transfer is almost identical to that to transfer the whole file. For a
larger file, however, the data transfer time itself is dominant, as can be
seen by the significantly smaller time it takes to transfer each 1/10 sized
chunk. Even in the large file case, there is an overhead to transferring
multiple files, which causes the total time to be significantly larger than
the single file case.

Table 1: Comparison of upload times for whole files or files in 10 pieces (with no encoding). Size | Total time [s] | Average time per file [s]
---|---|---
1$\times$756 KB | 6 | 6
10$\times$75.6 KB | 54 | 5.5
1$\times$2.4 GB | 142 | 142
10$\times$243 MB | 206 | 20

We see similar results for file retrieval and decoding. For small files (fig
7), parallelism significantly improves performance (although not to the level
of a single file copy operation on an unencoded file). For larger files (fig
7), parallelism appears to initially harm performance on our test system, but
the overall range of performance is small across all tests. We believe that
the limited network bandwidth on our small test system is probably the
bottleneck here, and prevents us from drawing strong conclusions concerning
the algorithm itself. In these cases, performance is affected mostly by the
communication overheads in the transfer phase, as file reconstruction requires
little overheads if the original data blocks are the first to be retrieved.
For this reason, there is no “grey” split file column in the download graphs.

Figure 6:  Scaling performance of file download for a 768kB file encoded as 10
chunks + 5 coding chunks, with increasing parallelism.

Figure 7:  Scaling performance of file download for a 2.4GB file encoded as 10
chunks + 5 coding chunks, with increasing parallelism.

##  4 Further Work

The code as represented in this document is available online [23]. The further
work mentioned in this section is localised to a clone of that repository[24],
as the original developer has moved on. At present, as proof of concept code,
the shim does not provide a reliable transfer service - any failed transfer
for any chunk will cause an upload to fail (and a download to fail, if it
causes the number of successfully transferred chunks to fall below the minimum
needed for reconstruction). Transfer retries are easy to implement for the
serial version, but cause more subtle complexities for parallel transfers (as
trying the next SE in the list, for example, disrupts the distribution of
chunks across the vector of SEs as a whole). Additionally, the initial design
of the DFC interface was undertaken without a full appreciation of the global
nature of the metadata tag namespace. Subsequently, it has become apparent
that our generic names for EC metadata keys are visible to all other users of
the Imperial DIRAC’s DFC, potentially causing confusion and misuse of the
metadata. Later versions of the shim will use unique prefixes for metadata
tags to minimise collisions in the tag namespace.

In general, with the move to more network-centric data access and management
models for the WLCG VOs, a more useful direction would be to explore the
incorporation of similar technologies into federated data storage protocols,
such as xrootd. In this case, leveraging the existing federation logic would
allow direct IO to encoded data over the network, reducing the transfer
overheads for the sparse reads common in some workflows. Work has begun on
such an implementation, which we hope to present at a later conference.

The authors would like to thank the NA62 VO, in particular Dan Protopopescu,
for useful discussions about their data management model, and the Imperial
DIRAC team, in particular Janusz Martyniak, for their work on maintaining and
developing the infrastructure used for this work.

## References

## References

  * [1] The GridPP Collaboration 2006 J. Phys. G 32 N1
  * [2] Bird I 2011 Annual Review of Nuclear and Particle Science 61 99-118
  * [3] Clark B E, Lawlor F D, Schmidt-stumpf W E, Stewart T J and Timms Jr. G D 1998 US Patent 4761785
  * [4] The HDFS Documentation http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html
  * [5] HDFS RAID Development Thread https://issues.apache.org/jira/browse/HDFS-7285
  * [6] Sathiamoorthy M, Asteris M, Papailiopoulos D, Dimakis A G, Vadali R, Chen S, and Borthakur D 2013 XORing elephants: Novel erasure codes for big data CoRR abs/1301 3791
  * [7] Weil S A, Brandt S A, Miller E L, Long D D and Maltzahn C 2006 Proceedings of the 7th symposium on Operating systems design and implementation 307-320 USENIX Association.
  * [8] WOS 360 Full Spectrum Object Storage (WOS White Paper) 2014 http://www.ddn.com/download/resource_library/whitepapers/archive_whitepapers/wos360_whitepaper.pdf
  * [9] Tahoe-LAFS website https://tahoe-lafs.org/trac/tahoe-lafs
  * [10] Wilcox-O’Hearn Z and Warner B 2008 StorageSS 21-26.
  * [11] Hamming R W 1950 Bell System Tech. J. (USA: AT&T) 29 (2) 147-160
  * [12] Reed I S, Solomon G 1960 Journal of the Society for Industrial and Applied Mathematics (SIAM) 8 (2) 300-304
  * [13] Blahut R E 1979 IBM Journal of Research and Development 23 299-315
  * [14] Massey J L 1969 IEEE Transactions on Information Theory IT-15 122-127
  * [15] Berrou C and Glavieux A 1996 Communications, IEEE Transactions on 44(10) 1261-1271.
  * [16] Raptor Forward Error Correction Scheme for Object Delivery http://tools.ietf.org/html/rfc5053
  * [17] Tsaregorodtsev A, Sanchez M, Charpentier P, Kuznetsov G, Closier J, Graciani R et al. 2006 DIRAC, the LHCb data production and distributed analysis system No. LHCb-TALK-2006-014
  * [18] Currie R A et al The GridPP DIRAC project - Implementation of a multi-VO DIRAC service THIS PROCEEDINGS
  * [19] ZFEC page on PyPi https://pypi.python.org/pypi/zfec
  * [20] Plank J S and Greenan K M 2014 Jerasure: A library in C facilitating erasure coding for storage applications version 2.0. Technical Report UT-EECS-14-721, University of Tennessee.
  * [21] Notice of removal of Jerasure 2.0 code from public availability. http://web.eecs.utk.edu/ plank/plank/papers/UT-EECS-14-721.html (retrieved 14 May 2015).
  * [22] Amdahl G M 1967 AFIPS Conference Proceedings 30 483-485
  * [23] Original Project Hosting https://github.com/ptodev/Distributed-Resilient-Storage/
  * [24] Current Project Fork https://github.com/aoanla/Distributed-Resilient-Storage/


# The 18 May 2024 superbolide over the Iberian Peninsula: USG space sensors
and ground-based independent observations

E. Peña-Asensio,1,2 P. Grèbol-Tomàs,2,3 J. M. Trigo-Rodríguez,2,3 P. Ramírez-
Moreta,4 and R. Kresken4
1Department of Aerospace Science and Technology, Politecnico di Milano, Via La
Masa 34, 20156 Milano, Italy
2Institut de Ciències de l’Espai (ICE, CSIC) Campus UAB, C/ de Can Magrans
s/n, 08193 Cerdanyola del Vallès, Catalonia, Spain
3Institut d’Estudis Espacials de Catalunya (IEEC) 08034 Barcelona, Catalonia,
Spain
4ESA/ESAC/Planetary Defence Office (OPS-SP) Camino Bajo del Castillo s/n,
28692 Villanueva de la Cañada, Madrid, Spain E-mail<EMAIL_ADDRESS><EMAIL_ADDRESS>
(Accepted XXX. Received YYY; in original form May 23, 2024)

###### Abstract

On 18 May 2024, a superbolide traversed the western part of the Iberian
Peninsula, culminating its flight over the Atlantic Ocean and generating
significant media attention. This event was caused by a weak carbonaceous
meteoroid of 89.8$\pm$0.4 cm, with a density of 1660 kg m-3, entering the
atmosphere at 41.6 km s-1 with an angle of 10.9∘. The luminous phase started
at 138 km and ended at an altitude of 54 km. The meteoroid’s heliocentric
orbit was characterized by an inclination of 15.4∘, a high eccentricity of
0.965, a semi-major axis of 3 au, and a notably short perihelion distance of
0.11 au. The superbolide was recorded by multiple ground-based stations of the
Spanish Meteor Network (SPMN), the European Space Agency (ESA), and the U.S.
Government (USG) space sensors. Our analysis shows a relatively good agreement
with the radiant and velocity data reported by the Center for Near-Earth
Object Studies (CNEOS), with a deviation of 1∘ and 1.2 km s-1, respectively.
Due to the absence of observable deceleration, we successfully reconciled
satellite radiometric data with a purely dynamic atmospheric flight model,
constraining the meteoroid’s mass and coherently fitting its velocity profile.
The inferred bulk density and aerodynamic strength of the meteoroid from the
flight model are compatible with the properties recently measured for sample-
returned materials from asteroid Bennu. The physical properties and the
orbital results suggest that this meteoroid originated from a recent
disruption of a comet, indicating the existence of hazardous meter-sized
projectiles arriving on Earth from objects formed in the distant regions of
the Solar System.

###### keywords:

Superbolide – Fireball Network – Space Sensors – Atmospheric Flight – Orbit

††pubyear: 2024††pagerange: The 18 May 2024 superbolide over the Iberian
Peninsula: USG space sensors and ground-based independent observations–The 18
May 2024 superbolide over the Iberian Peninsula: USG space sensors and ground-
based independent observations

##  1 Introduction

On 18 May 2024, an exceptionally luminous fireball was observed over Spain and
Portugal. This event was captured on video by casual observers and quickly
disseminated through the media. Additionally, it was recorded by various wide-
field and multi-camera stations that continuously monitor the sky over the
Iberian Peninsula. The U.S. Government (USG) space sensors also detected the
event, as reported on the Center for Near-Earth Object Studies (CNEOS)
fireball website111https://cneos.jpl.nasa.gov/. Following confirmation of its
detection from space222It was also detected by ESA’s Meteosat 3Gen (MTG)
satellite via its Lightning Imager instrument:
https://www.esa.int/Applications/Observing_the_Earth/Meteorological_missions/meteosat_third_generation/Fireball_witnessed_by_weather_satellite,
the event can be formally classified as a superbolide (Ceplecha et al., 1999).
These exceptionally bright fireballs are produced by the hypersonic
atmospheric entry of meter-sized natural projectiles (Ceplecha et al., 1998;
Silber et al., 2018). The study of superbolides provides valuable insights
into the physical properties, dynamics, and impact hazard issues associated
with the near-Earth asteroid population (Koschny & Borovicka, 2017; Trigo-
Rodríguez, 2022).

The CNEOS catalog, managed by NASA’s Jet Propulsion Laboratory, archives data
on fireball events detected via various USG satellite sensors (Tagliaferri et
al., 1994). Superbolides are relatively rare events that can occur over remote
areas. USG sensors provide near-global coverage of large meteoroids and small
asteroids impacting the Earth’s atmosphere, unlike the ground-based meteor
network which only monitors a relatively small, fixed atmospheric volume.
Therefore, the CNEOS catalog is of scientific interest as it extends the
projectile flux estimations by utilizing the entire planet as a detector,
capturing events that are typically singular occurrences for other techniques
(Brown et al., 2002).

As of May 2024, the catalog contains 979 fireball events, with velocity vector
and altitude data available for 310 cases. For these events, detailed
information is provided, including geographic coordinates, terminal disruption
altitude, vector velocity components in an Earth-Centered, Earth-Fixed (ECEF)
reference frame, and estimated radiated and impact energies. These energies,
linked by an empirical formula, are among the most reliable parameters
provided by CNEOS (Brown et al., 2002).

However, specific information about these sensors remains undisclosed, as they
are classified data. In any case, we previously prepared our 3D-FireTOC
software to compute future detections based on CNEOS-released data (Peña-
Asensio et al., 2021b) and performed an analysis of the CNEOS catalog (Peña-
Asensio et al., 2022). The accuracy of the CNEOS data has been assessed
through comparisons with ground-based observations of fireballs (Devillepoix
et al., 2019; Peña-Asensio et al., 2022; Brown & Borovička, 2023; Peña-Asensio
et al., 2024). To date, only 17 fireballs in the CNEOS catalog have been
benchmarked with independent counterparts. Here we report and compare a new
event detected by the USG space sensors and multiple ground-based fireball
networks.

##  2 Observations and methods

###  2.1 Ground-based observation

On the night of Saturday, May 18, 2024, a remarkable superbolide was observed
over the Iberian Peninsula at 22:46:49 UTC, specifically flying over
Extremadura and northern Portugal, before its trajectory concluded over the
Atlantic Ocean. It was recorded by the Spanish Fireball and Meteorite Network
(SPMN) (Trigo-Rodríguez et al., 2004), as well as by AMS82, one of the
European Space Agency’s (ESA) Planetary Defense Office meteor stations of the
AllSky7 Network333www.allsky7.com, located in Casas de Millán, Cáceres, Spain.
Designated as SPMN180524F by the SPMN network, this event was notable for its
intense brightness which momentarily turned night into day, causing a massive
media impact. The superbolide exhibited an atmospherically originated bluish-
green glow and left a persistent luminous trail in the sky. Table 1 lists the
stations used in this study, while Figure 1 shows the max-combined video
frames as observed from each station.

Many casual videos immediately surfaced in the media, underscoring the
importance of promptly explaining these unusual phenomena to the public. Some
of the videos are particularly extraordinary, as the bolide was captured from
a relatively close distance, revealing significant variations in brightness
along its extensive luminous trajectory. These fluctuations in luminosity are
typically associated with the continuous fragmentation of the meteoroid and
the release of dust and fragments ablated by the heat generated in the frontal
shock wave, a characteristic behavior of impacting crumbly meteoroids
(Revelle, 2002).

Figure 1: The four videos used in this work to study the superbolide SPMN180524F. The images are max-combined from the video frames. Some saturated frames have been removed for illustration purposes. From left to right: Navianos de Valverde, Estepa, Sanlúcar de Barrameda, and Casas de Millán. Table 1: Longitude, latitude, and altitude of the 4 selected stations recording the SPMN180524F superbolide. Station | Lon. (∘) | Lat. (∘) | Alt. (m)
---|---|---|---
Navianos de Valverde | 5∘ 48′ 48.4$"$ W | 41∘ 57′ 11.6$"$ N | 711
Estepa | 4∘ 52′ 35.6$"$ W | 37∘ 17′ 29$"$ N | 537
Sanlúcar de Barrameda | 6∘ 20′ 31$"$ W | 36∘ 46′ 30.3$"$ N | 6
Casas de Millán | 6∘ 19′ 57.8$"$ W | 39∘ 51′ 4.8$"$ N | 739

We use our 3D-FireTOC software to analyze the event. This tool integrates
computer vision and machine learning for detecting and tracking fireballs,
robust camera calibration to correct distortions, the plane intersection
method for triangulation, and heliocentric orbit determination (Peña-Asensio
et al., 2021b, a, 2023b, 2023a). Using a model of atmospheric mass density and
the measured velocity, the dynamic strength where the meteoroid disruption
occurs is computed as $S=\rho_{atm}v^{2}$ (Bronshten, 1983). From that, thanks
to the empirical strength-density relation established by Collins et al.
(2005), we calculate the impactor density as:

$\rho_{m}=\left(\frac{\log_{10}S-2.107}{0.0624}\right)^{2}.$ (1)

At the onset of the luminous phase and down to altitudes near 60 km, the
atmosphere is sufficiently thin that deceleration is barely observable.
Consequently, all measured points during this phase equally represent the
initial velocity, allowing for a robust estimate.

###  2.2 Space-based observation

From the perspective of the CNEOS catalog, the SPMN180524F superbolide, with a
total impact energy of 0.13 kt, is not a common event: it holds the highest
height in the database, ranks as the fifth fastest with respect to the Earth,
and is the fourth with the lowest density (see Fig. 2). The density estimation
process is the same as described in the previous section.

Figure 2: CNEOS fireballs with sufficient data (velocity and height at the
peak of radiated energy) to estimate the projectile density.

As the superbolide did not penetrate deeply into the atmosphere and was
observed from long distances, the ground-based measurements are insufficient
for fitting a dynamic deceleration model. Therefore, we use as a proxy the
total radiated energy ($E$) measured by the USG sensors, which has proven
reliable. By assuming that the energy recorded is equal to the kinetic energy,
we estimate the initial mass of the projectile as:

$m_{0}=\frac{2E}{v^{2}}.$ (2)

By levering the dynamic, dimensionless approach based on the single body
theory, the so-called $\alpha$-$\beta$ method (ballistic coefficient and mass-
loss parameter, respectively), we can characterize the atmospheric flight by
reverting Eq. 14 of Gritsevich (2009):

$\alpha=\frac{c_{d}\rho_{0}h_{0}A}{2m_{0}^{1/3}\rho_{m}^{2/3}\sin\gamma},$ (3)

where $c_{d}=0.7$ is the drag coefficient, $A=1.21$ is the spherical shape
factor, $\rho_{0}=1.29\,\,$kg m-3 is the atmospheric density at the sea level,
$h_{0}=7.16\,\,$km is the height of the homogeneous atmosphere, and $\gamma$
is the slope of the trajectory from the horizon.

By assuming the terminal mass is equal to zero, $\beta$ can be estimated from
the terminal height $h_{e}$ and $\alpha$ (Moreno-Ibáñez et al., 2015):

$\beta=\frac{e^{h_{e}/h_{0}}}{2\alpha}.$ (4)

With $\alpha$ and $\beta$ determined, we fit the atmospheric flight model to
the observed data to obtain the velocity profile (Gritsevich, 2007):

$h(v)=\ln 2\alpha+\beta-\ln(\overline{E}i(\beta)-\overline{E}i(\beta v^{2})),$
(5)

where

$\overline{Ei}(x)=\int_{-\infty}^{x}\frac{e^{z}dz}{z}.$

##  3 Results and comparison

Figure 3 displays the 3D reconstruction of the atmospheric flight, and Figure
4 depicts its characterization. The heliocentric orbits derived from the
ground-based observations and the space sensors are illustrated in Figure 5.
Table 2 shows all parameters derived from the ground- and space-based
observations.

Figure 3: Atmospheric flight of the superbolide SPMN180524F. For ground-based observations, the peak brightness occurs at the red point. In pink is shown the point reported by the USG space sensors. Figure 4: Characterization of the atmospheric flight of the superbolide SPMN180524F. It includes the measured velocity points, the best fit for the velocity with 3-$\sigma$ uncertainty, the dynamic strength, and the uncalibrated photometric counts from Estepa. Figure 5: Osculating heliocentric orbit of the superbolide SPMN180524F determined from ground and space observations. Table 2: Atmospheric flight, physical parameters, and osculating heliocentric orbital elements of the SPMN180524F superbolide. Parameter |  | Ground-based | CNEOS | Discrepancy
---|---|---|---|---
Initial time [UTC] | $t_{0}$ | 2024-05-18 22:46:42 |  |
Initial longitude [∘] | $\lambda_{0}$ |  5.5197$\pm$0.0010 W |  |
Initial latitude [∘] | $\varphi_{0}$ |  38.5777$\pm$0.0011 N |  |
Initial velocity [km s-1] | $v_{0}$ |  41.64$\pm$0.19 |  |
Initial height [km] | $h_{0}$ |  137.88$\pm$0.05 |  |
Final time [UTC] | $t_{e}$ | 2024-05-18 22:46:53 |  |
Final longitude [∘] | $\lambda_{e}$ |  9.885$\pm$0.005 W |  |
Final latitude [∘] | $\varphi_{e}$ |  42.112$\pm$0.005 N |  |
Final velocity [km s-1] | $v_{e}$ |  40.03$\pm$0.17 |  |
Final height [km] | $h_{e}$ |  53.78$\pm$0.07 |  |
Energy peak time [UTC] | $t_{p}$ | 2024-05-18 22:46:49 | 2024-05-18 22:46:50 | -1 s
Energy peak longitude [∘] | $\lambda_{p}$ |  8.2932$\pm$0.0017 W | 8.8 W | -0.5068
Energy peak latitude [∘] | $\varphi_{p}$ |  40.8851$\pm$0.0015 N | 41.0 N | -0.1149
Energy peak velocity [km s-1] | $v_{p}$ |  41.63$\pm$0.19 | 40.4 | 1.23
Energy peak height [km] | $h_{p}$ |  77.91$\pm$0.04 | 74.3 | 3.61
Length [km] | $\Delta l$ |  554.2$\pm$0.4 |  |
Slope [∘] | $\gamma$ |  10.93$\pm$0.02 | 6.5 | 4.43
Azimuth [∘] | $A$ |  317.75$\pm$0.04 | 315.8 | 1.95
Dynamic strength [kPa] | $S$ |  44.4$\pm$0.4 | 72.41 | -28.01
Meteoroid density [kg m-3] | $\rho_{m}$ |  1658$\pm$6 | 1946 | -288
Meteoroid mass [kg] | $m_{0}$ |  628$\pm$6 | 666 | -38
Initial diameter [cm] | $D$ |  89.8$\pm$0.4 | 87 | 2.8
Geo. velocity [km s-1] | $v_{R}$ |  39.87$\pm$0.19 | 38.6 | 1.27
Geo. radiant R.A. [∘] | $\alpha_{R}$ |  261.56$\pm$0.04 | 262.4 | -0.84
Geo. radiant Dec. [∘] | $\delta_{R}$ |  -28.13$\pm$0.04 | -29.7 | 1.57
Semi-major axis [au] | $a$ |  3.01$\pm$0.11 | 2.3 | 0.71
Eccentricity | $e$ |  0.965$\pm$0.002 | 0.95 | 0.015
Inclination [∘] | $i$ |  15.39$\pm$0.12 | 18.2 | -2.81
Argument of periapsis [∘] | $\omega$ |  145.52$\pm$0.12 | 145.4 | 0.12
Long. of the asc. node [∘] | $\Omega$ |  238.1257$\pm$0.0001 | 238.1321 | -0.0064
Perihelion distance [au] | $q$ |  0.106$\pm$0.001 | 0.11 | -0.004
True anomaly [∘] | $f$ |  214.49$\pm$0.12 | 214.6 | -0.11
Period [year] | $P$ |  5.2$\pm$0.3 | 3.5 | 1.7
Tisserand’s parameter | $Tj$ |  2.1$\pm$0.1 | 2.7 | -0.6
Ballistic coefficient | $\alpha$ |  17.22$\pm$0.05 |  |
Mass-loss parameter | $\beta$ |  53.1$\pm$0.4 |  |

The search for possible associations with meteoroid streams or parent bodies
for superbolide SPMN180524F yielded negative results using both traditional
D-criteria and Machine Learning distance metrics (Peña-Asensio & Sánchez-
Lozano, 2024). If the SPMN180524F superbolide resulted from a cometary
disruption, additional fragments could potentially reach the Earth.
Nevertheless, these are difficult to detect in advance due to the typical low
albedo of carbonaceous chondrites (Trigo-Rodríguez et al., 2014; Tanbakouei et
al., 2020). Given the estimated meteoroid size of approximately 0.9 m and a
density of 1660 kg m-3, we hypothesize its origin might be the disruption of a
comet. Remarkably, a recent study of the samples returned by the OSIRIS-REx
NASA mission revealed similar bulk density values (Lauretta et al., 2024),
which could suggest a Bennu-like parent body.

The scenario of SPMN180524F originating from a disrupted comet is plausible,
as its orbital eccentricity carried the meteoroid well beyond the orbit of
Jupiter. The plausible connection between carbonaceous chondrites (CCs) and
comets, long suggested, is further supported by recent studies of the
reflectance spectra of comet 2P/Encke and ungrouped CCs (Tanbakouei et al.,
2020). The role of erosion and subsequent dehydration caused by thermal
processing of cometary nuclei was elucidated by Rosetta’s study of
67P/Churyumov-Gerasimenko, providing insights into the evolution of meteoroids
resulting from their disruption (Fulle et al., 2020; Koschny et al., 2019;
Trigo-Rodríguez et al., 2019). Researches on cometary formation and
disintegration products suggest that centimeter-sized pebbles with higher
density can be preserved in the interior of comets (Schräpler et al., 2022;
Trigo-Rodríguez & Blum, 2022). These pebbles can survive at higher dynamic
strengths than the fine-grained materials forming the meteoroid interstitial
matrix.

There are examples of superbolides produced by carbonaceous chondrite (CC)
projectiles, such as the Tagish Lake meteorite fall, which had an estimated
meteoroid density of 1500 kg m-3 but resulted in higher density ungrouped CCs
(Brown et al., 2000), as well as the Maribo, Sutter’s Mill, Flensburg, and
Winchcombe meteorites (Haack et al., 2012; Jenniskens et al., 2012; Borovička
et al., 2021; McMullan et al., 2024). In any case, these fragile meteoroids
often generate extremely bright bolides without yielding noticeable meteorites
(Borovička et al., 2022). Over more than 25 years of operation, our SPMN
network has recorded several cometary superbolides; one of the most memorable
events occurred on July 11, 2008, which, due to its anomalous orbit, was
tentatively associated with the disruption of comet C/1919 Q2 Metcalf. (Trigo-
Rodríguez et al., 2009).

In addition to the physical properties of SPMN180524F, its short perihelion
distance suggests a relatively recent formation of the meteoroid. The inferred
bulk density is consistent with that expected for a crumbly carbonaceous
chondrite. It is well-known that common cometary outgassing cannot release
meter-sized meteoroids, indicating that this meteoroid may be a remnant of a
relatively recent disruption of its parent comet under the thermal stress
imposed by the solar heat near its close perihelion (Jenniskens, 2006; Jewitt,
2008; Trigo-Rodríguez & Blum, 2022).

Efforts have been made to estimate the uncertainties of the CNEOS catalog
based on ground-based observations, revealing two groups of measurements: one
with sufficient accuracy to allow acceptable heliocentric orbits, and another
with significant radiant and velocity deviations Devillepoix et al. (2019)
reported discrepancies in the radiants of CNEOS fireballs, ranging from a few
degrees to as much as 90∘. For example, velocity vectors were inaccurately
measured for events such as Buzzard Coulee, 2008 TC3, Kalabity, and Crawford
Bay. Specific typographical errors, like the missing minus sign in the $z$
velocity component for 2008 TC3, were noted by Peña-Asensio et al. (2022), in
addition to comparing two new events independently measured (2019 MO and 2022
EB5). Further independent analyses have included events like Saricicek,
Ozerki, Viñales, Flensburg, Novo Mesto, and Adalen, which helped refine the
mean radiant and velocity deviations of CNEOS fireballs (Brown & Borovička,
2023; Peña-Asensio et al., 2024).

The superbolide SPMN180524F belongs to the group of events well-measured from
space, as the apparent radiant is deviated in 0.98∘ and the velocity at the
energy peak in 1.23 km s-1, resulting in a relatively good agreement on the
heliocentric orbit. This difference in velocity leads to a 0.71 au dissent in
the semi-major axis. One notable divergence is the 4.43∘ of disagreement in
slope, which yields a -2.81∘ discrepancy in the orbital inclination. It is
worth noting the difference of -288 kg m-3 in density, mainly due to the 3.61
km of disagreement in the height where the radiation peak occurs.

##  4 Conclusions

The 18 May 2024 superbolide was a unique event that showcased how a fragile
meter-sized meteoroid can produce a spectacular display of color and
luminosity, and it exemplified how the Earth’s atmosphere is an excellent
shield for this type of impactor. A detailed analysis of the superbolide
characteristics suggests that the meteoroid likely originated from the
disruption of an eccentric comet, potentially explaining its relatively
unstable orbit crossing the orbit of Jupiter. In any case, its short
perihelion distance also suggests a recent detachment from its parent comet,
as the long-term survival of fragile materials would be challenging under such
thermal stress conditions close to the Sun. Given the high velocity and
fragmentation observed, a meteorite’s survival seems unlikely. However, the
previous Maribo carbonaceous chondrite fall demonstrated that even weak
meteoroids can produce small meteorites at relatively high speeds (Borovička
et al., 2019).

Our analysis demonstrated relatively good agreement with the data reported by
CNEOS both in radiant and velocity, and subsequently in the heliocentric
orbital elements. We reconciled the satellite radiometric data with a purely
dynamic atmospheric flight model to constrain the meteoroid’s mass and
consistently derive the atmospheric deceleration curve.

From an impact risk perspective, the event raises questions about why a meter-
sized meteoroid was not detected by current telescopic surveys, especially
considering that these small object monitoring programs have successfully
identified some asteroids of just a few meters in diameter before their
collision with Earth. Our results provide a clear explanation: the meteoroid
was simply too small and exhibited a low albedo to be detectable.

## Acknowledgements

EP-A acknowledges support from the LUMIO project funded by the Agenzia
Spaziale Italiana. JMT-R and PG-T acknowledge financial support from the
Spanish project PID2021-128062NB-I00 funded by MCIN/AEI/10.13039/501100011033
. PG-T acknowledges the financial support from the Spanish MCIU through the
FPI predoctoral fellowship PRE2022-104624. We also acknowledge the monitoring
effort made by the members of the SPMN Network. In particular, Miguel A.
Furones, Miguel A. Garcia, and Antonio J. Robles, contributed to the SPMN
recordings analyzed in this work. This work is partly based on data from AMS82
cameras of the AllSky7 network, and the authors thank the network operators,
especially Juan Carlos Martín, for providing the data.

## Data Availability

The data underlying this article will be shared on reasonable request to the
corresponding author.

## References

  * Borovička et al. (2019) Borovička J., Popova O., Spurný P., 2019, Meteoritics & Planetary Science, 54, 1024
  * Borovička et al. (2021) Borovička J., Bettonvil F., Baumgarten G., Strunk J., Hankey M., Spurný P., Heinlein D., 2021, Meteoritics & planetary science, 56, 425
  * Borovička et al. (2022) Borovička J., et al., 2022, A&A, 667, A157
  * Bronshten (1983) Bronshten V. A., 1983, Physics of Meteoric Phenomena. D. Reidel Publishing Company
  * Brown & Borovička (2023) Brown P. G., Borovička J., 2023, ApJ, 953, 167
  * Brown et al. (2000) Brown P. G., et al., 2000, Science, 290, 320
  * Brown et al. (2002) Brown P., Spalding R. E., ReVelle D. O., Tagliaferri E., Worden S. P., 2002, Nature, 420, 294
  * Ceplecha et al. (1998) Ceplecha Z., Borovička J., Elford W. G., Revelle D. O., Hawkes R. L., Porubčan V., Šimek M., 1998, Space Science Reviews, 84, 327
  * Ceplecha et al. (1999) Ceplecha Z., Spalding E. R., Jacobs C., Revelle D. O., Tagliaferri E., Brown P., 1999, in Baggaley W. J., Porubcan V., eds, Meteroids 1998. p. 37
  * Collins et al. (2005) Collins G. S., Melosh H. J., Marcus R. A., 2005, Meteoritics & planetary science, 40, 817
  * Devillepoix et al. (2019) Devillepoix H. A. R., et al., 2019, MNRAS, 483, 5166
  * Fulle et al. (2020) Fulle M., Blum J., Rotundi A., Gundlach B., Güttler C., Zakharov V., 2020, MNRAS, 493, 4039
  * Gritsevich (2007) Gritsevich M. I., 2007, Solar System Research, 41, 509
  * Gritsevich (2009) Gritsevich M. I., 2009, Advances in Space Research, 44, 323
  * Haack et al. (2012) Haack H., et al., 2012, Meteoritics & planetary science, 47, 30
  * Jenniskens (2006) Jenniskens P., 2006, Meteor Showers and their Parent Comets. Cambridge University Press
  * Jenniskens et al. (2012) Jenniskens P., et al., 2012, Science, 338, 1583
  * Jewitt (2008) Jewitt D., 2008, in , Saas-Fee Advanced Course 35: Trans-Neptunian Objects and Comets. p. 132
  * Koschny & Borovicka (2017) Koschny D., Borovicka J., 2017, WGN, Journal of the International Meteor Organization, 45, 91
  * Koschny et al. (2019) Koschny D., et al., 2019, Space Sci. Rev., 215, 34
  * Lauretta et al. (2024) Lauretta D. S., et al., 2024, arXiv
  * McMullan et al. (2024) McMullan S., et al., 2024, Meteoritics & planetary science, 59, 927
  * Moreno-Ibáñez et al. (2015) Moreno-Ibáñez M., Gritsevich M., Trigo-Rodríguez J. M., 2015, Icarus, 250, 544
  * Peña-Asensio et al. (2021a) Peña-Asensio E., Trigo-Rodríguez J. M., Langbroek M., Rimola A., Robles A. J., 2021a, Astrodynamics, 5, 347
  * Peña-Asensio et al. (2021b) Peña-Asensio E., Trigo-Rodríguez J. M., Gritsevich M., Rimola A., 2021b, MNRAS, 504, 4829
  * Peña-Asensio et al. (2022) Peña-Asensio E., Trigo-Rodríguez J. M., Rimola A., 2022, AJ, 164, 76
  * Peña-Asensio et al. (2023a) Peña-Asensio E., Trigo-Rodríguez J. M., Grèbol-Tomàs P., Regordosa-Avellana D., Rimola A., 2023a, Planet. Space Sci., 238, 105802
  * Peña-Asensio et al. (2023b) Peña-Asensio E., Trigo-Rodríguez J. M., Rimola A., Corretgé-Gilart M., Koschny D., 2023b, MNRAS, 520, 5173
  * Peña-Asensio et al. (2024) Peña-Asensio E., Visuri J., Trigo-Rodríguez J. M., Socas-Navarro H., Gritsevich M., Siljama M., Rimola A., 2024, Icarus, 408, 115844
  * Peña-Asensio & Sánchez-Lozano (2024) Peña-Asensio E., Sánchez-Lozano J. M., 2024, Advances in Space Research
  * Revelle (2002) Revelle D. O., 2002, in Warmbein B., ed., ESA Special Publication Vol. 500, Asteroids, Comets, and Meteors: ACM 2002. pp 233–236
  * Schräpler et al. (2022) Schräpler R. R., Landeck W. A., Blum J., 2022, MNRAS, 509, 5641
  * Silber et al. (2018) Silber E. A., Boslough M., Hocking W. K., Gritsevich M., Whitaker R. W., 2018, Advances in Space Research, 62, 489
  * Tagliaferri et al. (1994) Tagliaferri E., Spalding R., Jacobs C., Worden S. P., Erlich A., 1994, in Hazards Due to Comets and Asteroids. p. 199
  * Tanbakouei et al. (2020) Tanbakouei S., Trigo-Rodríguez J. M., Blum J., Williams I., Llorca J., 2020, A&A, 641, A58
  * Trigo-Rodríguez (2022) Trigo-Rodríguez J. M., 2022, Asteroid Impact Risk. Springer Nature
  * Trigo-Rodríguez & Blum (2022) Trigo-Rodríguez J. M., Blum J., 2022, MNRAS, 512, 2277
  * Trigo-Rodríguez et al. (2004) Trigo-Rodríguez J. M., et al., 2004, Earth Moon and Planets, 95, 553
  * Trigo-Rodríguez et al. (2009) Trigo-Rodríguez J. M., Madiedo J. M., Williams I. P., Castro-Tirado A. J., Llorca J., Vítek S., Jelínek M., 2009, MNRAS, 394, 569
  * Trigo-Rodríguez et al. (2014) Trigo-Rodríguez J. M., et al., 2014, MNRAS, 437, 227
  * Trigo-Rodríguez et al. (2019) Trigo-Rodríguez J. M., Rimola A., Tanbakouei S., Soto V. C., Lee M., 2019, Space Sci. Rev., 215, 18


# Adversarial Attacks On Multi-Agent Communication

James Tu1,2 Tsunhsuan Wang3∗ Jingkang Wang1,2 Sivabalan Manivasagam1,2
Mengye Ren1,2 Raquel Urtasun1,2

1Waabi 2University of Toronto 3MIT
{jtu, wangjk, manivasagam, mren<EMAIL_ADDRESS>
<EMAIL_ADDRESS>Equal contribution.

###### Abstract

Growing at a fast pace, modern autonomous systems will soon be deployed at
scale, opening up the possibility for cooperative multi-agent systems. Sharing
information and distributing workloads allow autonomous agents to better
perform tasks and increase computation efficiency. However, shared information
can be modified to execute adversarial attacks on deep learning models that
are widely employed in modern systems. Thus, we aim to study the robustness of
such systems and focus on exploring adversarial attacks in a novel multi-agent
setting where communication is done through sharing learned intermediate
representations of neural networks. We observe that an indistinguishable
adversarial message can severely degrade performance, but becomes weaker as
the number of benign agents increases. Furthermore, we show that black-box
transfer attacks are more difficult in this setting when compared to directly
perturbing the inputs, as it is necessary to align the distribution of learned
representations with domain adaptation. Our work studies robustness at the
neural network level to contribute an additional layer of fault tolerance to
modern security protocols for more secure multi-agent systems.

00footnotetext: Work done while all authors were at UberATG.

##  1 Introduction

With rapid improvements of modern autonomous systems, it is only a matter of
time until they are deployed at scale, opening up the possibility of
cooperative multi-agent systems. Individual agents can benefit greatly from
shared information to better perform their tasks [26, 59]. For example, by
aggregating sensory information from multiple viewpoints, a fleet of vehicles
can perceive the world more clearly, providing significant safety benefits
[52]. Moreover, in a network of connected devices, distributed processing
across multiple agents can improve computation efficiency [18]. While
cooperative multi-agent systems are promising, relying on communication
between agents can pose security threats as shared information can be
malicious or unreliable [54, 3, 37].

Meanwhile, modern autonomous systems typically rely on deep neural networks
known to be vulnerable to adversarial attacks. Such attacks craft small and
imperceivable perturbations to drastically change a neural network’s behavior
and induce false outputs [48, 21, 8, 30]. Even if an attacker has the freedom
to send any message, such small perturbations may be the most dangerous as
they are indistinguishable from their benign counterparts, making corrupted
messages difficult to detect while still highly malicious.

While modern cyber security algorithms provide adequate protection against
communication breaches, adversarial robustness of multi-agent deep learning
models has yet to be studied. Meanwhile, when it comes to safety-critical
applications like self-driving, additional layers of redundancy and improved
security are always welcome. Thus, by studying adversarial robustness, we can
enhance modern security protocols by introducing an additional layer of fault
tolerance at the neural network level.

Figure 1: Overview of a multi-agent setting with one malicious agent (red).
Here the malicious agent attempts to sabotage a victim agent by sending an
adversarial message. The adversarial message is indistinguishable from the
original, making the attack difficult to detect.

Adversarial attacks have been studied extensively but existing approaches
mostly consider attacks on input domains like images [48, 21], point clouds
[7, 50], and text [44, 14]. On the other hand, multi-agent systems often
distribute computation across different devices and transmit intermediate
representations instead of input sensory information [52, 18]. Specifically,
when deep learning inference is distributed across different devices, agents
will communicate by transmitting feature maps, which are activations of
intermediate neural network layers. Such learned communication has been shown
to be superior due to transmitting compact but expressive messages [52] as
well as efficiently distributing computation [18].

In this paper, we investigate adversarial attacks in this novel multi-agent
setting where perturbations are applied to learned intermediate
representations. An illustration is shown in Figure 1. We conduct experiments
and showcase vulnerabilities in two highly practical settings: multi-view
perception from images in a fleet of drones and multi-view perception from
LiDAR in a fleet of self-driving vehicles (SDVs). By leveraging information
from multiple viewpoints, these multi-agent systems are able to significantly
outperform those that do not exploit communication.

We show, however, that perturbed transmissions which are indistinguishable
from the original can severely degrade the performance of receivers
particularly as the ratio of malicious to benign agents increases. With only a
single attacker, as the number of benign agents increase, attacks become
significantly weaker as aggregating more messages decreases the influence of
malicious messages. When multiple attackers are present, they can coordinate
and jointly optimize their perturbations to strengthen the attack. In terms of
defense, when the threat model is known, adversarial training is highly
effective, and adversarially trained models can defend against perturbations
almost perfectly and even slightly enhance performance on natural examples.
Without knowledge of the threat model, we can still achieve reasonable
adversarial robustness by designing more robust message aggregation modules.

We then move on to more practical attacks in a black box setting where the
model is unknown to the adversary. Since query-based black box attacks need to
excessively query a target model that is often unaccessible, we focus on
query-free transfer attacks that are more feasible in practice. However,
transfer attacks are much more difficult to execute at the feature-level than
on input domains. In particular, since perturbation domains are model
dependent, vanilla transfer attacks are ineffective because two neural
networks with the same functionality can have very different intermediate
representations. Here, we find that training the surrogate model with domain
adaptation is key to aligning the distribution of intermediate features and
achieve much better transferability. To further enhance the practicality of
attacks, we propose to exploit the temporal consistency of sensory information
processed by modern autonomous systems. When frames of sensory information are
collected milliseconds apart, we can exploit the redundancy in adjacent frames
to create efficient, low-budget attacks in an online manner.

##  2 Related Work

Figure 2: Attacking object detection proposals: False positives are created by
changing the class of background proposals and false negatives are created by
changing the class of the original proposals.

#### Multi-Agent Deep Learning Systems:

Multi-agent and distributed systems are widely employed in real-world
applications to improve computation efficiency [27, 17, 2], collaboration [52,
59, 18, 41, 42], and safety [38, 35]. Recently, autonomous systems have
improved greatly with the help of neural networks. New directions have opened
up in cooperative multi-agent deep learning systems e.g., federated learning
[27, 2]. Although multi-agent communication introduces a multitude of
benefits, communication channels are vulnerable to security breaches, as
communication channels can be attacked [34, 45], encryption algorithms can be
broken [46], and agents can be compromised [5, 61]. Thus, imperfect
communication channels may be used to execute adversarial attacks which are
especially effective against deep learning systems. While robustness has been
studied in the context of federated learning [20, 1, 56, 19], the threat
models are different as dataset poisoning and model poisoning are typically
used. To the best of our knowledge, few works study adversarial robustness on
multi-agent deep learning systems during inference.

#### Adversarial Attacks:

Adversarial attacks were first discovered in the context of image
classification [48], where a small imperceivable perturbation can drastically
change a neural network’s behaviour and induce false outputs. Such attacks
were then extended to various applications such as semantic segmentation [57]
and reinforcement learning [24]. There are two main settings for adversarial
attacks - white box and black box. In a white box setting [48, 21, 30], the
attacker has full access to the target neural network weights and adversarial
examples can be generated using gradient-based optimization to maximize the
network’s error. In contrast, black box attacks are conducted without
knowledge of the target neural network weights and therefore without any
gradient computation. In this case, attackers can leverage real world
knowledge to inject adversaries that resemble common real world objects [47,
36]. However, if the attacker is able to query the target model, the
literature proposes several different strategies to perform query-based
attacks [4, 12, 6, 10]. However, query-based attacks are infeasible for some
applications as they typically require prohibitively large amounts of queries
and computation. Apart from query-based attacks, a more practical but more
challenging alternative is to conduct transfer attacks [39, 58, 16] which do
not require querying the target model. In this setting, the attacker trains a
surrogate model that imitates the target model. By doing so, the hope is that
perturbations generated for the surrogate model will transfer to the target
model.

#### Perturbations In Feature Space:

While most works in the literature focus on input domains like images, some
prior works have considered perturbations on intermediate representations
within neural networks. Specifically, [25] estimated the projection of
adversarial gradients on a selected subspace to reduce the queries to a target
model. [40, 44, 14] proposed to generate adversarial perturbation in word
embeddings for finding adversarial but semantically-close substitution words.
[55, 60] showed that training on adversarial embeddings could improve the
robustness of Transformer-based models for NLP tasks.

##  3 Attacks On Multi-Agent Communication

This section first introduces the multi-agent framework in which agents
leverage information from multiple viewpoints by transmitting intermediate
feature maps. We then present our method for generating adversarial
perturbations in this setting. Moving on to more practical settings, we
consider black box transfer attacks and find that it is necessary to align the
distribution of intermediate representations. Here, training a surrogate model
with domain adaptation can create transferable perturbations. Finally, we show
efficient online attacks by exploiting the temporal consistency of sensory
inputs collected at high frequency.

###  3.1 Multi-Agent Communication

We consider a setting where multiple agents cooperate to better perform their
tasks by sharing observations from different viewpoints encoded via a learned
intermediate representation. Adopting prior work [52], we assume a homogeneous
set of agents using the same neural network. Then, each agent $i$ processes
sensor input $x_{i}$ to obtain an intermediate representation
$m_{i}=F(x_{i})$. The intermediate feature map is then broadcasted to other
agents in the scene. Upon receiving messages, agent $j$ will aggregate and
process all incoming messages to generate output $Z_{j}=G(m_{1},\dots,m_{N})$,
where $N$ is the number of agents. Suppose that an attacker agent $i$ targets
a victim agent $j$. Here, the attacker attempts to send an indistinguishable
adversarial message $m_{i}^{{}^{\prime}}=m_{i}+\delta$ to maximize the error
in $Z_{j}^{{}^{\prime}}=G(m_{1},\dots m_{i}+\delta,m_{N})$. The perturbation
$\delta$ is constrained by $\left\lVert\delta\right\rVert_{p}\leq\epsilon$ to
ensure that the malicious message is subtle and difficult to detect. An
overview of the multi-agent setting is shown in Figure 1.

Figure 3: Our proposed transfer attack which incorporates domain adaptation
when training the surrogate model. During training, the discriminator forces
$F^{\prime}$ to produce intermediate representations similar to $F$. As a
result, $G^{\prime}$ can generate perturbations that transfer to $G$.

In this paper, we specifically focus on object detection as it is a
challenging task where aggregating information from multiple viewpoints is
particularly helpful. In addition, many downstream robotics tasks depend on
detection and thus a strong attack can jeopardize the performance of the full
system. In this case, output $Z$ is a set of $M$ bounding box proposals
${z^{(1)},\dots,z^{(M)}}$ at different spatial locations. Each proposal
consists of class scores $z_{\sigma_{0}},\dots,z_{\sigma_{k}}$ and bounding
box parameters describing the spatial location and dimensions of the bounding
box. Here classes $0,\dots,k-1$ are the object classes and $k$ denotes the
background class where no objects are detected.

When performing detection, models try to output the correct object class $k$
and maximize the ratio of intersection over union (IOU) of the proposed and
ground truth bounding boxes. In a post processing step, proposals with high
confidence are selected and overlapping bounding boxes are filtered with non-
maximum suppression (NMS) to ideally produce a single estimate per ground
truth object.

###  3.2 Adversarial Perturbation Generation

We first introduce our loss objective for generating adversarial perturbations
against object detection. To generate false outputs, we aim to confuse the
proposal class. For detected objects, we suppress the score of the correct
class to generate false negatives. For background classes, false positives are
created by pushing up the score of an object class. In addition, we also aim
to minimize the intersection-over-union (IoU) of the bounding box proposals to
further degrade performance by producing poorly localized objects. We define
the adversarial loss of the perturbed output $z^{\prime}$ with respect to an
unperturbed output $z$ instead of the ground truth, as it may not always be
available to the attacker. For each proposal $z$, let
$u=\operatorname*{argmax}_{i}\\{z_{\sigma_{i}}|i=0\dots m\\}$ be the highest
confidence class. Given the original object proposal $z$ and the proposal
after perturbation $z^{\prime}$, our loss function tries to push $z^{\prime}$
away from $z$:

$\ell_{adv}(z^{\prime},z)=\begin{cases}-\log(1-z^{\prime}_{\sigma_{u}})\cdot\mathrm{IoU}(z^{\prime},z)&\text{if
}u\neq k\text{ and}\ z_{\sigma_{u}}>\tau^{+},\\\
-\lambda\cdot{z^{\prime}}_{\sigma_{v}}^{\gamma}\log(1-z^{\prime}_{\sigma_{v}})&\text{if
}u=k\text{ and}\ \ z_{\sigma_{u}}>\tau^{-},\\\ 0&\text{otherwise}\end{cases}$

(1)

An illustration of the attack objective is shown in Figure 2. When $u\neq k$
and the original prediction is not a background class, we apply an untargetted
loss to reduce the likelihood of the intended class. When the intended
prediction is the background class $k$, we specifically target a non-
background class $v$ to generate a false positive. We simply choose $v$ to be
the class with the highest confidence that is not the background class. The
$\mathrm{IoU}$ operator denotes the intersection over union of two proposals,
$\lambda$ is a weighting coefficient, and $\tau^{-},\tau^{+}$ filter out
proposals that are not confident enough. We provide more analysis and
ablations to justify our loss function design in our experiments.

Following prior work [50], it is necessary to minimize the adversarial loss
over all proposals. Thus, the optimal perturbation under an $\epsilon$ \-
$\ell_{p}$ bound is

$\delta^{\star}=\operatorname*{argmin}_{\left\lVert\delta\right\rVert_{p}\leq\epsilon}\sum_{m=1}^{M}\ell_{adv}(z^{\prime(m)},z^{(m)}).$
(2)

Our work considers an infinity norm $p=\infty$ and we minimize this loss
across all proposals using projected gradient descent (PGD) [31], clipping
$\delta$ to be within $[-\epsilon,\epsilon]$.

###  3.3 Transfer Attack

We also consider transfer attacks as they are the most practical. White box
attacks assume access to the victim model’s weights which is difficult to
obtain in practice. On the other hand, query-based optimization is too
expensive to execute in real time as state-of-the-art methods still require
thousands of queries [13, 11] on CIFAR-10. Instead, when we do not have access
to the weights of the victim model $G$, we can imitate it with a surrogate
model $G^{\prime}$ such that perturbations generated by the surrogate model
can transfer to the target model.

Figure 4: Two multi-agent datasets we use. On the left are images of ShapeNet
objects taken from different view points. On the right are LiDAR sweeps by
different vehicles in the same scene.

One major challenge for transfer attacks in our setting is that perturbations
are generated on intermediate feature maps. Our experiments show that vanilla
transfer attacks are almost completely ineffective as two networks with the
same functionality do not necessarily have the same intermediate
representations. When training $F$ and $G$, there is no direct supervision on
the intermediate features $m=F(x)$. Therefore, even with the same
architecture, dataset, and training schedule, a surrogate $F^{\prime}$ may
produce messages $m^{\prime}$ with very different distribution from $m$. As an
example, a permutation of feature channels carries the same information but
results in a different distribution. In general, different random seeds,
network initializations or non-deterministic GPU operations can result in
different intermediate representations. It follows that if $m^{\prime}$ does
not faithfully replicate $m$, we cannot expect $G^{\prime}$ to imitate $G$.

Thus, to execute transfer attacks, we must have access to samples of the
intermediate feature maps. Specifically, we consider a scenario where the
attacker can spy on the victim’s communication channel to obtain transmitted
messages. However, since sensory information is not transmitted, the attacker
does not have access to pairs of input $x$ and intermediate representation $m$
to directly supervise the surrogate $F^{\prime}$ via distillation. Thus, we
propose to use Adversarial Discriminative Domain Adaptation (ADDA) [51] to
align the distribution of $m$ and $m^{\prime}$ without explicit input-feature
pairs. An overview is shown in Figure 3.

In the original training pipeline, $F^{\prime}$ and $G^{\prime}$ would be
trained to minimize task loss

$\small\mathcal{L}_{task}(z,y,b)=\begin{cases}-\log(z_{\sigma_{y}})-\mathrm{IoU}(z,b)&\text{if
}y\neq k,\\\ -\log(z_{\sigma_{y}})&\text{if }y=k,\\\ \end{cases}$ (3)

where $b$ is a ground truth bounding box and $y$ is its class. The task loss
maximizes the log likelihood of the correct class and the IoU between the
proposal box and the ground truth box. In addition, we encourage domain
adaptation by introducing a discriminator $D$ to distinguish between real
messages $m$ and surrogate messages $m^{\prime}$. The three modules
$F^{\prime}$, $G^{\prime}$, and $D$ can be optimized using the following min-
max criterion:

$\min\limits_{F^{\prime}\
G^{\prime}}\max\limits_{D}\mathcal{L}_{task}(x)+\beta\big{[}\log
D(F(x))+\log(1-D(F^{\prime}(x)))]$

(4)

where $\beta$ is a weighting coefficient and we use binary cross entropy loss
to supervise the discriminator. During training, we adopt spectral
normalization [33] in the discriminator and the two-time update rule [22] for
stability.

###  3.4 Online Attack

In modern applications of autonomous systems, consecutive frames of sensory
information are typically collected only milliseconds apart. Thus, there is a
large amount of redundancy between consecutive frames which can be exploited
to achieve more efficient adversarial attacks. Following previous work [53] in
images, we propose to exploit this redundancy by using the perturbation from
the previous time step as initialization for the current time step.

Furthermore, we note that intermediate feature maps capture the spatial
context of sensory observations, which change due to the agent’s egomotion.
Therefore, by applying a rigid transformation on the perturbation at every
time step to account for egomotion, we can generate stronger perturbations
that are synchronized with the movement of sensory observations relative to
the agent. In this case, the perturbations are updated as follows:

$\delta^{(t+1)}\leftarrow H_{t\rightarrow t+1}(\delta^{(t)})\
-\alpha\nabla_{H_{t\rightarrow
t+1}(\delta)}\mathcal{L}_{adv}(Z^{\prime(t+1)},Z^{(t+1)}).$ (5)

Here $H_{t\rightarrow t+1}$ is a rigid transformation mapping the attacker’s
pose at time $t$ to $t+1$ and $\alpha$ is the step size. By leveraging
temporal consistency we can generate strong perturbations with only one
gradient update per time step, making online attacks more feasible.

##  4 Experiments

###  4.1 Multi-Agent Settings

Figure 5: Qualitative attack examples. Top: Messages sent by another agent
visualized in bird’s eye view. Bottom: outputs. Perturbations are very subtle
but severely degrade performance.

#### Multi-View ShapeNet:

We conduct our attacks on multi-view detection from images, which is a common
task for a fleets of drones. Following prior work [15], we generate a
synthetic dataset by placing 10 classes of ShapeNet [9] objects on a table
(see Figure 4). From each class, we subsample 50 meshes and use a 40/10 split
for training and validation. In every scene, we place 4 to 8 objects and
perform collision checking to ensure objects do not overlap. Then, we capture
128$\times$128 RGB-D images from 2 to 7 viewpoints sampled from the upper half
of a sphere centered at the table center with a radius of 2.0 units. This
dataset consists of 50,000 training scenes and 10,000 validation scenes. When
conducting attacks, we randomly sample one of the agents to be the adversary.
Our detection model uses an architecture similar to the one introduced in
[15]. Specifically, we process input RGB-D images using a U-Net [43] and then
unproject the features into 3D using the depth measures. Features from all
agents are then warped into the same coordinate frame and aggregated with mean
pooling. Finally, aggregated features are processed by a 3D U-Net and a
detection header to generate 3D bounding box proposals.

#### Vehicle To Vehicle Communication:

We also consider a self-driving setting with vehicle-to-vehicle(V2V)
communication. Here, we adopt the dataset used in [52], where 3D
reconstructions of logs of real world LiDAR scans are simulated from the
perspectives of other vehicles in the scene using a high-fidelity LiDAR
simulator [32]. These logs are collected by self-driving vehicles equipped
with LiDAR sensors capturing 10 frames per second (see Figure 4). The training
set consists of 46,796 subsampled frames from the logs and we do not subsample
the validation set, resulting in 96,862 frames. In every log we select one
attacker vehicle and sample others to be cooperative agents with up to 7
agents in each frame unless otherwise specified. This results in a consistent
assignment of attackers and V2V agents throughout the frames. In this setting,
we use the state-of-the-art perception and motion forecasting model V2VNet
[52]. Here, LiDAR inputs are first encoded into bird’s eye view (BEV) feature
maps. Feature maps from all agents are then warped into the ego coordinate
frame and aggregated with a GNN to produce BEV bounding box proposals. More
details of the ShapeNet model and V2VNet are provided in the supplementary
material.

Figure 6: Evaluation under no perturbation, uniform noise, transfer attack,
and white box attack. Results are grouped by the number of agents in the scene
where one agent is the attacker.

#### Implementation Details:

When conducting attacks, we set $\epsilon=0.1$. For the proposed loss
function, we set $\lambda=0.2,\tau^{-}=0.7,\tau^{+}=0.3$, and $\gamma=1$.
Projected gradient descent is done using Adam with learning rate $0.1$ and we
apply $15$ PGD steps for ShapeNet and only $1$ PGD step for low budget online
attacks in the V2V setting. The surrogate models use the same architecture and
dataset as the victim models. When training the surrogate model, we set
$\beta=0.01$, model learning rate $0.001$, and discriminator learning rate
$0.0005$. For evaluation, we compute area under the precision-recall curve of
bounding boxes, where bounding boxes are correct if they have an IoU greater
than 0.7 with a ground truth box of the same class. We refer to this metric as
AP at 0.7 in the following.

###  4.2 Results

#### Attack Results:

Visualizations of our attack are shown in Figure 5 and we present quantitative
results of our attack and baselines in Figure 6. We split up the evaluation by
the number of agents in the scene and one of the agents is always an attacker.
As a baseline, we sample the perturbation from
$\mathcal{U}{(-\epsilon,\epsilon)}$ to demonstrate that the same $\epsilon$
bounded uniform noise does not have any impact on detection performance. The
white box attack is especially strong when few agents are in the scene, but
becomes weaker as the number of benign agents increase, causing the relative
weight of the adversarial features in mean pooling layers to decrease.
Finally, our transfer attack with domain adaptation achieves moderate success
with few agents in the scene, but is significantly weaker than the white box
attack.

| ShapeNet | V2V
---|---|---
Clean | Perturbed | Clean | Perturbed
Original | 66.33 | 0.62 | 82.19 | 7.55
Adv Trained | 67.29 | 66.00 | 82.60 | 83.44

Table 1: Results of adversarial training. Robustness increases significantly,
matching clean inference. Furthermore performance on clean data also improves
slightly.

#### Robustifying Models:

To defend against our proposed attack, we conduct adversarial training against
the white box adversary and show the results in Table 1. Here, we follow the
standard adversarial training set up, except perturbations are applied to
intermediate features instead of inputs. This objective can be formulated as

$\min_{\theta}\mathbb{E}_{(x,y)\sim
D}\max_{\left\lVert\delta\right\rVert_{\infty}<\epsilon}\phi((x,y,\delta);\theta):=\\\
\mathcal{L}_{task}\left(G(F(x_{0}),\dots,F(x_{i})+\delta,\dots,F(x_{N});\theta)\right),$
(6)

where $D$ is the natural training distribution and $\theta$ denotes model
parameters. During training, we generate a new perturbation $\delta$ for each
training sample. In the multi-agent setting, we find it easier to recover from
adversarial perturbations when compared to traditional single-agent attacks.
Moreover, adversarial training is able to slightly improve performance on
clean data as well, while adversarial training has been known to hurt natural
performance in previous settings [28, 49].

| Clean | Perturbed
---|---|---
Agents | 2 | 4 | 6 | 2 | 4 | 6
Mean Pool | 82.09 | 89.25 | 92.43 | 0.90 | 12.93 | 41.77
GNN(Mean) | 82.19 | 89.93 | 92.94 | 7.55 | 52.31 | 76.18
GNN(Median) | 82.11 | 87.12 | 90.75 | 12.8 | 67.70 | 86.30
GNN(Soft Med) | 82.19 | 89.67 | 92.49 | 21.53 | 61.37 | 84.99

Table 2: Choice of fusion in V2VNet affects performance and robustness. We
investigate using mean pooling and using a GNN with various aggregation
methods.

While adversarial training is effective in this setting, it requires knowledge
of the threat model. When the threat model is unknown, we can still naturally
boost the robustness of multi-agent models with the design of the aggregation
module. Specifically, we consider several alternatives to V2VNet’s GNN fusion
and present the performance under attacked and clean data in Table 2. First,
replacing the entire GNN with an adaptive mean pooling layer significantly
decreases robustness. On the other hand, we swap out the mean pooling in GNN
nodes with median pooling and find that it increases robustness at the cost of
performance on clean data with more agents, since more information is
discarded. We refer readers to the supplementary materials for more details on
implementation of the soft median pooling.

#### Multiple Attackers:

We previously focused on settings with one attacker, and now conduct
experiments with multiple attackers in the V2V setting. In each case, we also
consider if attackers are able to cooperate. In cooperation, attackers jointly
optimize their perturbations. Without cooperation, attackers are blind to each
other and optimize their perturbations assuming other messages have not been
perturbed. Results with up to 3 attackers are shown in Table 3. As expected,
more attackers can increase the strength of attack significantly, furthermore,
if multiple agents can coordinate, a stronger attack can be generated.

| Cooperative | Non-Cooperative
---|---|---
Agents | 4 | 5 | 6 | 4 | 5 | 6
1 Attacker | 52.31 | 65.00 | 76.18 | 52.31 | 65.00 | 76.18
2 Attacker | 28.31 | 41.34 | 54.50 | 39.02 | 51.96 | 64.02
3 Attacker | 12.07 | 22.84 | 35.13 | 24.27 | 38.17 | 51.58

Table 3: Multiple white box attackers in the V2V setting. Cooperative
attackers jointly optimize their perturbations and non-cooperative attackers
optimize without knowledge of each other.

Next, we apply adversarial training to the multi-attacker setting and present
results in Table 4. Here, all attacks are done in the cooperative setting and
we show results with 4 total agents. Similar to the single attacker setting,
adversarial training is highly effective. However, while adversarial training
against one attacker improves performance in natural examples, being robust to
stronger attacks sacrifices performance on natural examples. This suggests
that adversarial training has the potential to improve general performance
when an appropriate threat model is selected. Furthermore, we can see that
training on fewer attacks does not generalize perfectly to more attackers but
the opposite is true. Thus, it is necessary to train against an equal or
greater threat model to fully defend against such attacks.

Attackers | 0 | 1 | 2 | 3
---|---|---|---|---
Train On 0 | 89.93 | 52.31 | 28.31 | 12.07
Train On 1 | 90.09 | 90.00 | 81.95 | 75.28
Train On 2 | 89.71 | 89.68 | 88.91 | 88.33
Train On 3 | 89.55 | 89.51 | 88.94 | 88.51

Table 4: Adversarial training with multiple attackers in the V2V setting. We
train on settings with various number of attackers and evaluate the models
across the settings.

#### Domain Adaptation:

More results of the transfer attack are included in Table 5. First, we conduct
an ablation and show that a transfer attack without domain adaptation (DA) is
almost completely ineffective. On the contrary, surrogate models trained with
DA achieve significant improvements. A visual demonstration of feature map
alignment with DA is shown in Figure 7, visualizing 4 channels of the
intermediate feature maps. Features from a surrogate trained with DA is
visually very similar to the victim, while a surrogate trained without DA
produces features with no resemblance.

Since our proposed DA improves the transferability of the surrogate model, we
can further improve our transfer attack by also adopting methods from the
literature which enhance the transferability of a given perturbation. We find
that generating perturbations from diversified inputs (DI) [58] is ineffective
as resizing input feature maps distorts spatial information which is important
for localizing objects detection. On the other hand, using an intermediate
level attack projection (ILAP) [23] yields a small improvement. Overall, we
find transfer attacks more challenging when at the feature level. In standard
attacks on sensory inputs, perturbations are transferred into the same input
domain. However, at a feature level the input domains are model-dependent,
making transfer attacks between different models more difficult.

| ShapeNet | V2V
---|---|---
Clean | 66.28 | 82.19
Transfer | 66.21 | 81.31
Transfer + DA | 42.59 | 72.45
Transfer + DA + ILAP |  35.69 | 71.76
Transfer + DA + DI | 49.38 | 75.18

Table 5: Transfer attacks evaluated with 2 agents. Training the surrogate with
domain adaptation (DA) significantly improves transferability. In addition, we
attempt to enhance transferability with ILAP [23] and DI [58].

Victim |
---|---
DA |
No DA |

Figure 7: Visualization of how domain adaptation(DA) affects 4 channels of the
intermediate feature map. Observe that the surrogate trained with DA closely
imitates the victim model, while the surrogate trained without DA produces
different features.

#### Online Attacks:

We conduct an ablation on the proposed methods for exploiting temporal
redundancy in an online V2V setting, shown in Table 6. First, if we ignore
temporal redundancy and do not reuse the previous perturbation, attacks are
much weaker. In this evaluation we switch from PGD [31] to FGSM [21] to obtain
a stronger perturbation in one update for fair comparison. We also show that
applying a rigid transformation on the perturbations at every frame to
compensate for egomotion provides a modest improvement to the attack when
compared to the No Warp ablation.

#### Loss Function Design:

We conduct an ablation study on using our adversarial loss $\mathcal{L}_{adv}$
instead of the negative task loss $-\mathcal{L}_{task}$ in Table 7. This
ablation validates our loss function and showcase that for structured outputs,
properly designed adversarial losses is more effective than the naive negative
task loss which is widely used in image classification tasks. Our choice for
the loss function design is motivated by our knowledge of the post-processing
non-maximum suppression (NMS). Since NMS selects bounding boxes with the
highest confidence in a local region, proposals with higher scores should
receive stronger gradients. More specifically, an appropriate loss function of
$f$ for proposal score $\sigma$ should satisfy
$\left(|\nabla_{\sigma_{2}}f(\sigma_{2})|-|\nabla_{\sigma_{1}}f(\sigma_{1})|\right)/{(\sigma_{2}-\sigma_{1})}>0$
so that $|\nabla_{\sigma}f(\sigma)|$ is monotonically increasing in $\sigma$.
We can see that the standard log likelihood does not satisfy this criteria,
which explains why our loss formulation is more effective. In addition, we add
the focal loss term [29] to generate more false positives, as aggressively
focusing on one proposal in a local region is more effective due to NMS.

AP @ 0.7 | 2 Agents | 4 Agents | 6 Agents
---|---|---|---
Our Attack | 7.55 | 52.31 | 76.18
No Warping | 7.17 | 52.35 | 77.37
Independent | 56.98 | 80.21 | 87.05

Table 6:  Ablation on online attacks in the V2V setting. Independent refers to
treating each frame independently and not reusing previous perturbations. No
warp refers to omitting the rigid transformation to account for egomotion.

| 2 Agents | 4 Agents | 6 Agents
---|---|---|---
ShapeNet | $-\mathcal{L}_{task}$ | 6.10 | 20.07 | 29.00
$\mathcal{L}_{adv}$ | 0.37 | 4.45 | 13.77
V2V | $-\mathcal{L}_{task}$ | 20.8 | 63.82 | 79.11
$\mathcal{L}_{adv}$ | 7.55 | 52.31 | 76.18

Table 7: Ablation on loss function, it produces stronger adversarial attacks
than simply using the negative of the training task loss.

##  5 Conclusion

In this paper, we investigate adversarial attacks on communication in multi-
agent deep learning systems. Our experiments in two practical settings
demonstrate that compromised communication channels can be used to execute
adversarial attacks. However, robustness increases as the ratio of benign to
malicious actors increases. Furthermore, we found that more practical transfer
attacks are more challenging in this setting and require aligning the
distributions of intermediate representations. Finally, we propose a method to
achieve efficient and practical online attacks by exploiting temporal
consistency of sensory inputs. We believe studying adversarial robustness on
multi-agent deep learning models in real-world applications is an important
step towards more secure multi-agent systems.

## References

  * [1] Arjun Nitin Bhagoji, Supriyo Chakraborty, Prateek Mittal, and Seraphin B. Calo.  Analyzing federated learning through an adversarial lens.  In ICML, volume 97 of Proceedings of Machine Learning Research, pages 634–643. PMLR, 2019.
  * [2] Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloé Kiddon, Jakub Konecný, Stefano Mazzocchi, H. Brendan McMahan, Timon Van Overveldt, David Petrou, Daniel Ramage, and Jason Roselander.  Towards federated learning at scale: System design.  CoRR, 2019.
  * [3] Niklas Borselius.  Mobile agent security.  Electronics & Communication Engineering Journal, 14(5), 2002.
  * [4] Wieland Brendel, Jonas Rauber, and Matthias Bethge.  Decision-based adversarial attacks: Reliable attacks against black-box machine learning models.  In ICLR, 2018.
  * [5] Thomas Brewster.  Watch chinese hackers control tesla’s brakes from 12 miles away, 2016\.
  * [6] Thomas Brunner, Frederik Diehl, Michael Truong-Le, and Alois Knoll.  Guessing smart: Biased sampling for efficient black-box adversarial attacks.  CoRR, abs/1812.09803, 2018.
  * [7] Yulong Cao, Chaowei Xiao, Benjamin Cyr, Yimeng Zhou, Won Park, Sara Rampazzi, Qi Alfred Chen, Kevin Fu, and Z. Morley Mao.  Adversarial sensor attack on lidar-based perception in autonomous driving.  In CCS, 2019.
  * [8] Nicholas Carlini and David Wagner.  Towards evaluating the robustness of neural networks.  In SP, 2017.
  * [9] Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu.  ShapeNet: An Information-Rich 3D Model Repository.  arXiv preprint arXiv:1512.03012, 2015.
  * [10] Jianbo Chen, Michael I Jordan, and Martin J Wainwright.  Hopskipjumpattack: A query-efficient decision-based attack.  arXiv preprint arXiv:1904.02144, 3, 2019.
  * [11] Jianbo Chen, Michael I. Jordan, and Martin J. Wainwright.  Hopskipjumpattack: A query-efficient decision-based attack.  In IEEE Symposium on Security and Privacy, pages 1277–1294. IEEE, 2020.
  * [12] Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh.  ZOO: zeroth order optimization based black-box attacks to deep neural networks without training substitute models.  In AISec, 2017.
  * [13] Minhao Cheng, Thong Le, Pin-Yu Chen, Huan Zhang, JinFeng Yi, and Cho-Jui Hsieh.  Query-efficient hard-label black-box attack: An optimization-based approach.  In ICLR, 2019.
  * [14] Minhao Cheng, Jinfeng Yi, Huan Zhang, Pin-Yu Chen, and Cho-Jui Hsieh.  Seq2sick: Evaluating the robustness of sequence-to-sequence models with adversarial examples.  CoRR, abs/1803.01128, 2018.
  * [15] Ricson Cheng, Ziyan Wang, and Katerina Fragkiadaki.  Geometry-aware recurrent neural networks for active visual recognition.  In NeurIPS. 2018.
  * [16] Shuyu Cheng, Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu.  Improving black-box adversarial attacks with a transfer-based prior.  In NeurIPS, 2019.
  * [17] Tharam S. Dillon, Chen Wu, and Elizabeth Chang.  Cloud computing: Issues and challenges.  In AINA, 2010.
  * [18] Amir Erfan Eshratifar and Massoud Pedram.  Energy and performance efficient computation offloading for deep neural networks in a mobile cloud computing environment.  In GLSVLSI, 2018.
  * [19] Minghong Fang, Xiaoyu Cao, Jinyuan Jia, and Neil Zhenqiang Gong.  Local model poisoning attacks to byzantine-robust federated learning.  In USENIX Security Symposium, pages 1605–1622. USENIX Association, 2020.
  * [20] Avishek Ghosh, Justin Hong, Dong Yin, and Kannan Ramchandran.  Robust federated learning in a heterogeneous environment.  arXiv preprint arXiv:1906.06629, 2019.
  * [21] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy.  Explaining and harnessing adversarial examples.  ICLR, 2015.
  * [22] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter.  Gans trained by a two time-scale update rule converge to a local nash equilibrium.  In NIPS, 2017.
  * [23] Qian Huang, Isay Katsman, Horace He, Zeqi Gu, Serge J. Belongie, and Ser-Nam Lim.  Enhancing adversarial example transferability with an intermediate level attack.  CoRR, abs/1907.10823, 2019.
  * [24] Sandy Huang, Nicolas Papernot, Ian Goodfellow, Yan Duan, and Pieter Abbeel.  Adversarial attacks on neural network policies.  2017\.
  * [25] Linxi Jiang, Xingjun Ma, Shaoxiang Chen, James Bailey, and Yu-Gang Jiang.  Black-box adversarial attacks on video recognition models.  In ACM MM, 2019.
  * [26] Jakub Konecný, H. Brendan McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon.  Federated learning: Strategies for improving communication efficiency.  CoRR, abs/1610.05492, 2016.
  * [27] Jakub Konecný, H. Brendan McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon.  Federated learning: Strategies for improving communication efficiency.  CoRR, 2016.
  * [28] Saehyung Lee, Hyungyu Lee, and Sungroh Yoon.  Adversarial vertex mixup: Toward better adversarially robust generalization.  In CVPR, pages 269–278. IEEE, 2020.
  * [29] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár.  Focal loss for dense object detection.  In ICCV, 2017.
  * [30] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu.  Towards deep learning models resistant to adversarial attacks.  arXiv preprint arXiv:1706.06083, 2017.
  * [31] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu.  Towards deep learning models resistant to adversarial attacks.  arXiv preprint arXiv:1706.06083, 2017.
  * [32] Sivabalan Manivasagam, Shenlong Wang, Kelvin Wong, Wenyuan Zeng, Wei-Chiu Ma, Mikita Sazanovich, Bin Yang, and Raquel Urtasun.  Lidarsim: Realistic lidar simulation by leveraging the real world.  In CVPR, 2020.
  * [33] Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida.  Spectral normalization for generative adversarial networks.  arXiv preprint arXiv:1802.05957, 2018.
  * [34] Bassem Mokhtar and Mohamed Azab.  Survey on security issues in vehicular ad hoc networks.  Alexandria engineering journal, 54(4):1115–1126, 2015.
  * [35] Satoshi Nakamoto.  Bitcoin: A peer-to-peer electronic cash system.  Technical report, 2019.
  * [36] Ben Nassi, Dudi Nassi, Raz Ben-Netanel, Yisroel Mirsky, Oleg Drokin, and Yuval Elovici.  Phantom of the adas: Phantom attacks on driver-assistance systems.  IACR, 2020:85, 2020.
  * [37] Petr Novák, Milan Rollo, Jirí Hodík, and Tomás Vlcek.  Communication security in multi-agent systems.  In CEEMAS, volume 2691 of Lecture Notes in Computer Science. Springer, 2003.
  * [38] Marcus Obst, Laurens Hobert, and Pierre Reisdorf.  Multi-sensor data fusion for checking plausibility of v2v communications by vision-based multiple-object tracking.  In VNC, 2014.
  * [39] Nicolas Papernot, Patrick D. McDaniel, Ian J. Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami.  Practical black-box attacks against machine learning.  In AsiaCCS, 2017.
  * [40] Nicolas Papernot, Patrick D. McDaniel, Ananthram Swami, and Richard E. Harang.  Crafting adversarial input sequences for recurrent neural networks.  In MILCOM, 2016.
  * [41] Andreas Rauch, Felix Klanner, Ralph Rasshofer, and Klaus Dietmayer.  Car2x-based perception in a high-level fusion architecture for cooperative perception systems.  In IV, 2012.
  * [42] Matthias Rockl, Thomas Strang, and Matthias Kranz.  V2v communications in automotive multi-sensor multi-target tracking.  In VTC, 2008.
  * [43] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-net: Convolutional networks for biomedical image segmentation.  In MICCAI, 2015.
  * [44] Motoki Sato, Jun Suzuki, Hiroyuki Shindo, and Yuji Matsumoto.  Interpretable adversarial perturbation in input embedding space for text.  In IJCAI, 2018.
  * [45] Hichem Sedjelmaci and Sidi Mohammed Senouci.  An accurate and efficient collaborative intrusion detection framework to secure vehicular networks.  Computers & Electrical Engineering, 43, 2015.
  * [46] Catherine Stupp and James Rundle.  Capital one breach highlights shortfalls of encryption, 2019.
  * [47] Jiachen Sun, Yulong Cao, Qi Alfred Chen, and Z Morley Mao.  Towards robust lidar-based perception in autonomous driving: General black-box adversarial sensor attack and countermeasures.  In USENIX Security, pages 877–894, 2020.
  * [48] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus.  Intriguing properties of neural networks.  ICLR, 2014.
  * [49] Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry.  Robustness may be at odds with accuracy.  In ICLR (Poster). OpenReview.net, 2019.
  * [50] James Tu, Mengye Ren, Sivabalan Manivasagam, Min Liang, Bin Yang, Richard Du, Cheng Frank, and Raquel Urtasun.  Towards physically realistic adversarial examples for lidar object detection.  arXiv, 2020.
  * [51] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell.  Adversarial discriminative domain adaptation.  In CVPR, 2017.
  * [52] Tsun-Hsuan Wang, Sivabalan Manivasagam, Ming Liang, Bin Yang, Wenyuan Zeng, and Raquel Urtasun.  V2vnet: Vehicle-to-vehicle communication for joint perception and prediction.  arXiv, 2020.
  * [53] Xingxing Wei, Jun Zhu, Sha Yuan, and Hang Su.  Sparse adversarial perturbations for videos.  In AAAI, 2019.
  * [54] H. Chi Wong and Katia P. Sycara.  Adding security and trust to multiagent systems.  Applied Artificial Intelligence, 14(9):927–941, 2000.
  * [55] Yi Wu, David Bamman, and Stuart J. Russell.  Adversarial training for relation extraction.  In EMNLP, 2017.
  * [56] Chulin Xie, Keli Huang, Pin-Yu Chen, and Bo Li.  DBA: distributed backdoor attacks against federated learning.  In ICLR. OpenReview.net, 2020.
  * [57] Cihang Xie, Jianyu Wang, Zhishuai Zhang, Yuyin Zhou, Lingxi Xie, and Alan Yuille.  Adversarial examples for semantic segmentation and object detection.  In ICCV, 2017.
  * [58] Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L. Yuille.  Improving transferability of adversarial examples with input diversity.  In CVPR, 2019.
  * [59] Tengchan Zeng, Mohammad Mozaffari, Omid Semiari, Walid Saad, Mehdi Bennis, and Merouane Debbah.  Wireless communications and control for swarms of cellular-connected uavs.  In ACSSC, 2018.
  * [60] Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Goldstein, and Jingjing Liu.  Freelb: Enhanced adversarial training for natural language understanding.  In ICLR, 2020.
  * [61] Zeljka Zorz.  Researchers hack bmw cars, discover 14 vulnerabilities, 2018.