11institutetext:  Radboud Radio Lab, Department of Astrophysics, Radboud
University Nijmegen, The Netherlands 22institutetext: LESIA, Observatoire de
Paris, Université PSL, CNRS, Sorbonne Université, Université de Paris, France
33institutetext: Goddard Planetary Heliophysics Institute, University of
Maryland,Baltimore County, Baltimore, MD, USA 44institutetext: Heliophysics
Science Division, NASA Goddard Space Flight Center, Greenbelt, MD, USA
55institutetext: Space Sciences Laboratory, University of California,
Berkeley, CA, USA 66institutetext: Physics Department, University of
California, Berkeley, CA, USA 77institutetext: LPP, CNRS, Ecole Polytechnique,
Sorbonne Université, Observatoire de Paris, Université Paris-Saclay,
Palaiseau, Paris, France 88institutetext: CNES, 18 Avenue Edouard Belin, 31400
Toulouse, France 99institutetext: Swedish Institute of Space Physics (IRF),
Uppsala, Sweden 1010institutetext: LPC2E, CNRS, 3A avenue de la Recherche
Scientifique, Orléans, France 1111institutetext: Technische Universität
Dresden, Würzburger Str. 35, D-01187 Dresden, Germany 1212institutetext:
Institute of Atmospheric Physics of the Czech Academy of Sciences, Prague,
Czech Republic 1313institutetext: Space Research Institute, Austrian Academy
of Sciences, Graz, Austria 1414institutetext: Astronomical Institute of the
Czech Academy of Sciences, Ondvrejov, Czech Republic 1515institutetext: Space
and Plasma Physics, School of Electrical Engineering and Computer Science, KTH
Royal Institute of Technology, Stockholm, Sweden

# Solar Orbiter/RPW antenna calibration in the radio domain and its
application to type III burst observations

A. Vecchio 1122 M.Maksimovic 22 V. Krupar 3344 X. Bonnin 22 A. Zaslavsky 22 P.
L. Astier 22 M. Dekkali 22 B. Cecconi 22 S.D. Bale 5566 T. Chust 77 E. Guilhem
88 Yu. V. Khotyaintsev 99 V. Krasnoselskikh 1010 M. Kretzschmar 1010 E.
Lorfèvre 88 D. Plettemeier 1111 J. Souček 1212 M. Steller 1313 Š. Štverák
14141212 P. Trávníček 551212 A. Vaivads 1515

###### Abstract

Context. In order to allow for a comparison with the measurements from other
antenna systems, the voltage power spectral density measured by the Radio and
Plasma waves receiver (RPW) on board Solar Orbiter needs to be converted into
physical quantities that depend on the intrinsic properties of the radiation
itself (e.g., the brightness of the source).

Aims. The main goal of this study is to perform a calibration of the RPW
dipole antenna system that allows for the conversion of the voltage power
spectral density measured at the receiver’s input into the incoming flux
density.

Methods. We used space observations from the Thermal Noise Receiver (TNR) and
the High Frequency Receiver (HFR) to perform the calibration of the RPW dipole
antenna system. Observations of type III bursts by the Wind spacecraft are
used to obtain a reference radio flux density for cross-calibrating the RPW
dipole antennas. The analysis of a large sample of HFR observations (over
about ten months), carried out jointly with an analysis of TNR-HFR data and
prior to the antennas’ deployment, allowed us to estimate the reference system
noise of the TNR-HFR receivers.

Results.  We obtained the effective length, $l_{eff}$, of the RPW dipoles and
the reference system noise of TNR-HFR in space, where the antennas and pre-
amplifiers are embedded in the solar wind plasma. The obtained $l_{eff}$
values are in agreement with the simulation and measurements performed on the
ground. By investigating the radio flux intensities of 35 type III bursts
simultaneously observed by Solar Orbiter and Wind, we found that while the
scaling of the decay time as a function of the frequency is the same for the
Waves and RPW instruments, their median values are higher for the former. This
provides the first observational evidence that Type III radio waves still
undergo density scattering, even when they propagate from the source, in a
medium with a plasma frequency that is well below their own emission
frequency.

###### Key Words.:

Sun: radio radiation – solar wind –Instrumentation: detectors

##  1 Introduction

The Solar Orbiter (SO) mission (Müller et al. 2020; Zouganelis et al. 2020)
carries ten instruments, including the Radio and Plasma Wave experiment (RPW)
(Maksimovic et al. 2020), which is designed to measure magnetic and electric
fields, plasma wave spectra, and polarization properties, as well as the
spacecraft (S/C) floating potential and solar radio emissions in the
interplanetary medium. In this paper, we focus on the two high-frequency
receivers of RPW: Thermal Noise Receiver (TNR), producing electric and
magnetic power spectral densities in the frequency range from 4 kHz to 1 MHz,
and High Frequency Receiver (HFR), only providing electric power spectral
densities in the range from 375 kHz over to 16 MHz. The TNR-HFR has two input
channels, which allows us to perform simultaneous measurements from two
selectable sensors configurations. Among those, the monopole modes measure the
potential difference between an antenna and the S/C ground, while the dipole
modes measure the potential difference between two antennas. A mode for the
measurement of magnetic field fluctuations at high frequency through the
search coil (see Maksimovic et al. 2020) has also been defined. A large
variety of configurations, characterized by frequency range, temporal, and
spectral resolutions, etc., are programmable in a series of operating modes
optimized for specific analyses (Maksimovic et al. 2020). A given TNR setup
combines simultaneous measurements (dipoles or monopole, or both) from the two
channels and computes autocorrelations and cross-correlations between the
signals, allowing us to derive, with goniopolarimetry methods, the full
polarization of the incoming radio waves. The HFR is a sweeping receiver
providing the electric power spectral density for dipole antennas only. The
three monopole sensors of TNR-HFR are named V1, V2, and V3 and they correspond
to the antennas Pz, Py, and My, respectively (see Figure 7 from Maksimovic et
al. 2020).

The spectral properties of the signals measured by radio receivers on board
the spacecraft are computed onboard. The voltage power spectral density
$V^{2}_{\nu}$ is usually sampled and transmitted on the ground. The power
spectral density needs to be converted to physical quantities related to the
properties of the radiation itself in order to allow for a comparison with
measurements performed by other antenna systems. To this purpose, instrumental
parameters, such as the antennas’ effective lengths and capacitances, need to
be measured in flight in the real physical conditions of operation that (due
to the effect of the spacecraft body and the low density environment) are not
reproducible on the ground. A typical approach to performing this conversion
is to observe a known source of electromagnetic radiation and to relate the
measured $V^{2}_{\nu}$ to the flux of the source. A dominant natural radio
emission in the frequency range from 500 kHz to 10 MHz is the radio background
radiation of the Galaxy, which it is commonly used as a calibration source for
space antennas (Dulk et al. 2001; Zarka et al. 2004; Zaslavsky et al. 2011).
Unfortunately, the TNR-HFR receiver suffers a strong electro-magnetic (EM)
contamination due to the central power distribution unit (PCDU) radiated by
the Solar Panels, at 120 kHz and harmonics, and the reaction wheel electronic
box, at 80 kHz and harmonics (Maksimovic & et al. 2021). This makes the
observation of the galactic background much more difficult as it is masked by
the instrumental noise due to the platform.

In this study, we used type III solar radio bursts as calibration sources for
the RPW antenna system. Fast electrons beams, originating from magnetic
reconnection sites of solar flares and escaping through open magnetic field
lines to the interplanetary space, interact with the local plasma and give
rise to the type III radio bursts that are commonly observed in the frequency
range 10 kHz-1 GHz, and are characterized by fast frequency drifts (around 100
MHz/s in the metric range). Type III bursts are among the strongest radio
emissions routinely occurring in the heliosphere, reaching flux densities up
to 103-104 higher than the galactic background and certainly well above the
instrumental TNR-HFR background. This strong signal can be used to cross-
calibrate the RPW antenna system by comparing the TNR-HFR data with
simultaneous measurements of type III radio bursts by the radio receiver band
1 (RAD1, 20-1040 kHz) on Wind/Waves (Bougeret et al. 1995).

The paper is structured as follows. In Section 2, we present some general
relations between the signal measured onboard by the radio receiver and the
incoming electromagnetic radiation. In Section 3, these relations are applied
to cross-calibrate the dipole antennas of RPW by using type III emissions
measured by the Wind spacecraft as reference sources. This allows us to derive
the in-flight measurement of the effective length of the three dipoles
(Section 4). The dipole antenna calibration was then applied to several type
III emissions, occurring between July 2020 and January 2021, and a statistical
analysis of the type III burst decay times, measured by both RPW and Waves, is
presented in Section 5. Finally, the procedure to derive the system noise
background of TNR-HFR by comparing ten months of HFR observations with the
expected Galaxy signal and by an analysis of TNR-HFR data before the antennas’
deployment is described in Appendix A.

##  2 Short antenna dipole calibration

In the short-dipole approximation, when the wavelength $\lambda$ of the
measured radiation is much larger than the physical length of the antenna, the
latter experiences an homogeneous time-varying electric field and the
conversion of the signal in V2/Hz, measured as the receiver’s input into
incoming flux density, can be obtained analytically (Dulk et al. 2001; Zarka
et al. 2004; Zaslavsky et al. 2011). The RPW antennas can be considered as
short dipoles throughout much of the measured THR-HFR frequency range. In this
approximation, and for a unpolarized source, it is possible to derive a simple
equation relating the squared voltage fluctuations at a given frequency $\nu$,
(in units of V2/Hz), induced on a dipole antenna to the apparent brightness of
the source ${B_{v}}$ (in units W/m2/Hz/sr):

$V_{\nu}^{2}=\frac{l_{eff}^{2}}{2}Z_{0}\int_{\Omega_{S}}B_{\nu}(\Omega)sin^{3}\theta
d\theta d\phi,$ (1)

where $\Omega_{S}$ is the solid angle occupied by the source,
$l_{eff}={V}/{E}$ ($V$ is the voltage induced at the antenna by an incoming
electromagnetic plane wave of electric field, $E$, when the electric field
direction is parallel to the direction of the antenna (Kraus & Marhefka 2003))
is the effective length of the antenna; a term $\sin^{2}\theta$ (where
$\theta$ is the angle between the wave vector, $k,$ and the linear antenna
direction) is due to the response pattern of a dipole antenna and
$Z_{0}=\sqrt{\mu_{0}/\epsilon_{0}}\approx 120\pi$ is the impedance of vacuum.
The antenna is electrically connected to a large input impedance receiver
through the deployment mechanism (the antenna base) and various connection
cables providing a so-called stray impedance $Z_{s}$. The effective voltage
power spectral density $V^{2}_{r}$ measured by the receiver is then:

$V_{r}^{2}=\left|\frac{Z_{s}}{Z_{a}+Z_{s}}\right|^{2}V_{\nu}^{2}=\Gamma^{2}V_{\nu}^{2},$
(2)

where $Z_{a}$ is the impedance of the antenna and $\Gamma$ is the gain factor.
Both $Z_{s}$ and $Z_{a}$ are determined by the spacecraft design. In the radio
frequency range (at frequencies well above the kHz), the resistive part of
these impedances is negligible (typically, at 1 MHz, $1/\omega C_{a,s}\sim 10$
$k\Omega$ and $R_{a,s}\sim 1$ $\Omega$) and the gain factor becomes:

$\Gamma=C_{a}/(C_{a}+C_{s}),$ (3)

where $C_{a}$ and $C_{s}$ are the antenna and the stray capacitances,
respectively. This indicates that the radio receiver’s sensitivity can be
increased by minimizing the stray capacitance.

A general relation for the voltage power spectral density at the receiver
level is (Dulk et al. 2001; Zarka et al. 2004; Zaslavsky et al. 2011):

$V^{2}_{r}=V^{2}_{noise}+V^{2}_{QTN}+V^{2}_{gal}+V^{2}_{source}\;\;\;.$ (4)

Equation (4) relates the effective voltage power spectral density measured by
the receiver $V^{2}_{r}$ to the signal from the source $V^{2}_{source}$, both
expressed in $V^{2}/Hz$. Three other contributions have been included. Firstly
$V^{2}_{noise}$ represents the noise produced by the electronic components of
the receiver that superimposes any external signal and depends on both the
receiver and the impedance connected to it. The second contribution is due to
the quasi-thermal noise (QTN) produced by the ambient plasma (Zaslavsky et al.
2011; Meyer-Vernet et al. 2017). The third contribution is due to the Galactic
signal. By defining the power response of the antenna $P_{\nu}$ as

$P_{\nu}=\frac{1}{2}\int_{\Omega_{S}}B_{\nu}(\Omega)sin^{3}\theta d\theta
d\phi,$ (5)

Equation (4) becomes:

$V^{2}_{r}=V^{2}_{noise}+V^{2}_{QTN}+\Gamma^{2}{l_{eff}^{2}}Z_{0}P^{gal}_{\nu}+\Gamma^{2}{l_{eff}^{2}}Z_{0}P^{source}_{\nu},$
(6)

linking $V^{2}_{r}$ to the convolution of the brightness of the source with
the antenna response function, $P^{source}_{\nu}$, expressed in W/m2/Hz. If no
external radio sources are present and the Galaxy is the only external
homogeneous (i.e., $B_{\nu}$ does not depend on $\Omega$) source, which
extends over $\Omega_{s}=4\pi$, Equation (1) can be immediately integrated to
yield:

$V_{\nu}^{2}=\Gamma^{2}l_{eff}^{2}Z_{0}P^{gal}_{\nu}=\frac{4\pi}{3}{\Gamma^{2}l_{eff}^{2}}Z_{0}B^{gal}_{\nu}$
(7)

and Equation (6) becomes

$V^{2}_{r}=V^{2}_{noise}+V^{2}_{QTN}+\frac{4\pi}{3}\Gamma^{2}{l_{eff}^{2}}Z_{0}B^{gal}_{\nu},$
(8)

where the product $\Gamma l_{eff}$ can be considered as a “reduced effective
length” of the antenna.

##  3 Calibration using solar type III bursts

To evaluate the reduced effective length of the RPW electric antennas, we
compared the radio spectra of a type III emission measured by RPW with the
measurements by Wind/Waves when both spacecraft are close. On March 8, 2020,
two type III emissions occurred between 15:00-16:00 and 20:00-21:00,
respectively. On March 30, 2020, a type III emission occurred between
11:00-12:00. These solar radio emissions were clearly observed by both
Wind/Waves and SO/RPW (Figure 1).

Figure 1: Dynamic spectra from Wind/Waves-RAD1 (upper panels) and SO/RPW-TNR
(dipole V1-V2, lower panels) for March 8 (a, c) and March 30, (b, d) 2020.
White boxes underline the type III occurrence on the RPW data. The color
tables are in dB, calculated from W2/m2/Hz for Wind and V2/Hz for RPW.

We used the Wind/Waves flux density, obtained through goniopolarimetric
inversions (Manning & Fainberg 1980) and expressed in W m-2Hz-1; RPW data, on
the other hand, were only calibrated at the receiver level and are expressed
in V2/Hz. We used TNR-HFR dipole measurements that are affected to a lesser
extent, with respect to the monopole ones, by the EM contamination, and we
also chose clean frequencies. While the configuration of the HFR receiver was
the same on March 8 and 30, with only the dipole V1-V2 measurements available,
the configuration of TNR was different: over the first day, all the monopoles
and only V1-V2 measurements were available, while all the monopoles and all
the dipoles measurements were available for March 30. Since the type III radio
emissions are not isotropic and the exact position and size of the source is
unknown, we made use of the general Equation (6) under some assumptions: i)
the two spacecraft are about at the same location, implying that they observe
the source with the same solid angle and measure the same flux density of the
type III burst, $S_{B-RPW}(\nu)=S_{B-Wind}(\nu)$; ii) the k vector of the
radiation is perpendicular to the RPW antennas.

Figure 2, showing the position of Wind and SO in the GSE reference frame,

Figure 2: Position of the SO (magenta dot) and Wind (orange dot) on March 8
(left panel) and March 30 (right panel), 2020 in the GSE reference frame. The
light blu dot indicates the Earth. The X axis points towards the Sun.

indicates that on both March 8 and 30, Wind and SO were very close each other
(between 0.05 and 0.1 AU). The frequencies at which the type III radiation is
observed on both Wind and RPW (500 kHz-16MHz) are compatible with an emission
occurring close to the Sun. Indeed, using the Leblanc et al. (1998) density
model, a radio emission emitted at a fundamental plasma frequency of 500 kHz
would be located within 10 solar radii from the Sun. The full width at half
maximum (FWHM) of the type III radio beam at frequencies higher than 500 kHz
is about 100 degrees for both latitude and longitude (Bonnin et al. 2008). On
the other hand, the angular distance between the two spacecraft, as viewed
from the source, is 3-5 and 0.75-1 degrees in longitude and latitude
respectively. Since the angular width of the radio beam is very large with
respect to the mutual distance between Wind and SO, the difference in the
radio flux measured by the two spacecraft is low. When quantified through the
functional form of a type III radio beam derived by Bonnin et al. (2008), the
difference in radio flux is about 0.3% and 0.1% for the longitude and
latitude, respectively. At the same time, the difference in radio flux due to
the different distances of the two spacecraft from the Sun is about 3%.
Assumption i) can be thus considered, to a good approximation, as valid.
Concerning assumption ii), since the radio emission would be located close the
Sun, the k vector of the radiation can be considered, to a good approximation,
as perpendicular to the RPW antennas since the latter are perpendicular to the
Sun-spacecraft direction. Under these assumptions, the relation between the
RPW voltage power spectral density and the flux density during the type III
burst can be expressed as:

$V^{2}_{B-RPW}(\nu)=(1/2)Z_{0}\Gamma^{2}l_{eff}^{2}S_{B-RPW}(\nu)\;\;.$ (9)

The factor $\Gamma l_{eff}$ can be thus derived from Equation (9) by comparing
Waves and RPW data at the frequencies showing the type III emission:

$\Gamma^{2}l^{2}_{eff}=\frac{2}{Z_{0}}\frac{V^{2}_{B-RPW}(\nu)}{S_{B-Wind}(\nu)},$
(10)

where $V^{2}_{B-RPW}$ and $S_{B-Wind}$ are the power spectral density and the
flux density as measured by RPW and Waves, respectively. Then, $V^{2}_{B-RPW}$
follows from Equation (4)

$V^{2}_{B-RPW}(\nu)=V^{2}_{r}(\nu)-V^{2}_{gal}(\nu)-V^{2}_{noise}(\nu)\;\;\;\;\;\;\;\;\;\;\;\;V^{2}/Hz,$
(11)

where $V^{2}_{r}$ is the power spectral density as measured by TNR and HFR.
The term $V^{2}_{gal}+V^{2}_{noise}$ has been estimated as the median value of
the power spectral density calculated in time intervals far from the type III
occurrence. We remark that the results do not change if the term
$V^{2}_{gal}+V^{2}_{noise}$ is evaluated by using, for instance, the lower 5%
percentile. The contribution of the QTN can be considered negligible for
$\omega\gg 500$ kHz. However, possible residual contributions of QTN are
removed when the median background is subtracted from the data.
$S_{B}$ is derived from RAD1 measurements as:

$S_{B-Wind}(\nu)=S_{Wind}(\nu)-S_{backg-
Wind}(\nu)\;\;\;\;\;\;\;\;\;\;\;\;W/m^{2}/Hz,$ (12)

where $S_{Wind}$ is the type III flux density as measured by Waves and
$S_{backg-Wind}$ represents the background signal due to other sources of
noise at low frequency (e.g., the galactic background noise). As before,
$S_{backg-Wind}$ is estimated as a median value and subtracted from the Waves
spectra to remove the low frequency noise from the data. Moreover, since RAD1
and TNR/HFR produce spectra at slightly different frequencies and are
characterized by different time sampling, we firstly interpolated $S_{B-Wind}$
on the same time grid of TNR and HFR data and then by applying Equation (10),
we choose $S_{B-Wind}(\nu)$ at the closest frequency to the corresponding
$V^{2}_{B-RPW}(\nu)$. Examples of the spectra from RAD1, TNR, and HFR, during
the type III bursts of March 8, are shown in figure 3.

Figure 3: Comparison between $S_{B-Wind}$ (dashed lines) in W m-2Hz-1, as
obtained by Waves-RAD1, and $V^{2}_{B-RPW}$ (solid lines), as measured by RPW
and rescaled by a factor ${2}/Z_{0}$ for RPW-TNR (black) and RPW-HFR (red) and
for the V1-V2 dipole. Vertical dotted lines identify the data points used to
calculate $\Gamma l_{eff}$. Panels a, c, and b, d refer to the two type III
emissions detected on March 8, 2020, respectively.

For each frequency, a value for $\Gamma l_{eff}$ is calculated by averaging
Equation (10) over the type III emission profile. The number of points in the
average is not the same at each frequency since the duration of the type III
emission profile changes as a function of the frequency.

Figure 4: $\Gamma l_{eff}$ as a function of the frequency for the first
(black) and second (red) type III on March 8th 2020. Dots and triangles refers
to TNR and HFR, measurements respectively.

For both days, an average $\Gamma l_{eff}$ for V1-V2 was calculated as a
function of frequency from both the TNR and HFR data. The $\Gamma l_{eff}$
values for V2-V3 and V3-V1 were evaluated from the measurements on March 30
based only on TNR. An example of the average $\Gamma l_{eff}$ as a function of
frequency is shown in Figure 4 for both TNR and HFR measurements. This
quantity should not depend on the frequency since neither the effective length
nor the antenna capacitance does so in the frequency range where the short
dipole approximation holds. Indeed, no strong variations were detected for the
same type III nor when the samples were compared. For each day, we calculated
a single $\Gamma l_{eff}$ as the average over frequency, receiver (if
measurements on both TNR and HFR are available) and type III (for March 8 data
when two emissions are detected). The average values of $\Gamma l_{eff}$ are
shown in table 1.

Table 1:  Average $\Gamma l_{eff}$ values as obtained from the cross-calibration with type III measurements from Wind/Waves on March 8 and 30, and from the Galaxy background signal from ten months of HFR V1-V2 dipole data (see Appendix A). Sensor | March 8 | March 30 | Galaxy
---|---|---|---
V1-V2 |  $2.7\pm 0.3$ m |  $3.2\pm 0.3$ m |  $3.4\pm 0.1$ m
V2-V3 |  |  $2.5\pm 0.3$ m |
V3-V1 |  |  $3.0\pm 0.2$ m |

The comparison between V1-V2 $\Gamma l_{eff}$’s from the two days indicates
that the values are compatible within the uncertainty limits. The $\Gamma
l_{eff}$ value in the last column of table 1 is derived by comparing about ten
months of V1-V2 HFR data, to reduce the EM contamination, with a model of the
Galaxy background (see Appendix A). Only HFR measurements for the V1-V2 dipole
were available for such a long period. The value of $\Gamma l_{eff}=3.4\pm
0.1$ m is compatible, in the uncertainty limit, with the values obtained from
the type III emissions. We also remark that the $\Gamma l_{eff}$ values are
compatible with the corresponding ones obtained after ground testing and
characterizations of the RPW antenna expected to be between $2.35$ and $3.24$
m (Maksimovic et al. 2020). Unfortunately, no observations of type III
emissions by TNR monopoles are available for the periods when SO was close to
Wind. The calibration of the RPW monopole antennas is thus ongoing and will be
the subject of a future paper.

##  4 Determination of the effective length

Given that for March 30, the measurements for all the three dipoles are
available, in the following we use $\Gamma l_{eff}$ values obtained from this
day as a reference for calculating the effective length, $l_{eff}$, of the SO
dipole antenna. To derive $l_{eff}$, we first need to model the gain factor,
$\Gamma,$ and to this end, we need to know the antenna capacitance, $C_{a}$,
and the stray capacitance, $C_{s}$. The antenna capacitance can be
analytically derived by considering cylindrical antennas of length $L,$ and
radius $a$ (Zaslavsky et al. 2011):

$C_{a}=\frac{\epsilon_{0}\pi}{k}\frac{\tan{(kL)}}{\log{(L/a)}-1}\;\;\;.$ (13)

Values of $a=0.015$ m, $C_{s}=54.7$ pF, for the RPW dipoles, $L=7.857$ m, for
the dipoles 12 and 31, and $L=6.99$ m, for the dipole 23, as given in
Maksimovic et al. (2021) were used. We note that the latter paper provides
corrected values for some parameters that were previously published in
Maksimovic et al. (2020). By using these parameters, $l_{eff}$ can be obtained
from the previously derived values for $\Gamma l_{eff}$ . The effective
antenna lengths for the three TNR-HFR dipoles are shown in Table 2. The value
of $l_{eff}$ obtained by using the $\Gamma l_{eff}$ as derived from the Galaxy
background signal from ten months of HFR measurements is also shown. The
values of $l_{eff}$ agree, within the uncertainty limits, with the values of
the effective length for the dipoles found by Panchenko (2017) and reported by
Maksimovic (2019).

Table 2: Results of the Calibration. Parameters of the RPW dipole antennas as obtained from the cross-calibration with type III measurements from Wind/Waves, from the Galaxy background signal from ten months of HFR V1-V2 dipole data (see Appendix A) and from numerical simulations by Panchenko (2017). Antenna Configuration | average $\Gamma$ | $l_{eff}$ (m) typeIII | $l_{eff}$ (m) Galaxy | theoretical $l_{eff}$ (m)
---|---|---|---|---
V1-V2 | 0.44$\pm$0.01 | 7.2$\pm 0.8$ | 7.7$\pm$0.3 | $7.53$
V2-V3 | 0.42$\pm$0.01 | 6.0$\pm 0.8$ |  | $5.60$
V3-V1 | 0.44$\pm$0.01 | 7.0$\pm 0.6$ |  | $7.53$

##  5 Type III radio bursts simultaneously measured by SO and Wind

This section deals with the statistical analysis and comparison between type
III bursts simultaneously observed by SO and Wind. We apply the antenna
calibration developed in the previous section to a selected set of 79 type III
radio bursts observed by SO between June 2020 and January 2021, representing
the most intense and most simple cases detected. Due to the strong EM
pollution suffered by RPW measurements, not all frequency channels are capable
of detecting the type III signal. Since the purpose of the analysis is to
compare Wind and SO measurements for the type III detected on both spacecraft
only, a further selection was made according to the following steps. First, we
identified the TNR frequency channels where the type III burst peak is above
the background plus $4\sigma_{back}$ (where $\sigma_{back}$ is a measure of
amplitude of the fluctuations calculated as the standard deviation in the same
time interval where the background is evaluated), then we verified that the
same event is evenly detected at the RAD1 channel closer to the corresponding
TNR frequency. If both these requirements were verified, the event was
selected for subsequent analyses. The final database is then reduced to 35
type III events.

Figure 5: Radio measurements of the 2020 July 11 type III burst. Radio flux
density for Wind/Waves-RAD1 (a) and SO/RPW-TNR (b).

One example from one of the considered events is shown in Figure 5. It is a
type III burst occurring on 2020 July 11 that has been linked to an electron
beam at $\sim$ 200 keV detected by the EPD instrument (Rodríguez-Pacheco et
al. 2020) onboard SO at around 02:30. Figure 5 shows the flux density from
Wind/Waves-RAD1 (a) and SO/RPW-TNR (b). Both spacecraft detected a simple and
isolated type III burst with an onset time at about 02:22 UT. During this
event, the GSE coordinates of the two spacecraft were
$(x_{Wind},y_{Wind})=(0.01,0.004)$ AU and $(x_{SO},y_{SO})=(1.19,0.58)$ AU and
the distance from the Sun was $0.99$ AU and $0.88$ AU for Wind and SO
respectively.

Figure 6: Radio measurements of the 2020 July 11 type III burst. Light curves
of the radio flux density measured by SOLO-RPW/TNR (left column) and Wind-
Waves/RAD1 (right column) at four frequency channels. The red line shows the
results of decay time fitting. The dashed vertical line indicates the peak
fluxes and the horizontal line shows the median value.

The time profile of the intensity of the type III at a given frequency
consists of a very fast rising phase, followed by a longer phase decreasing
exponentially (Evans et al. 1973). This is clearly shown in Figure 6, where
the light curves at four frequencies $[290.9,411.4,662.6,978.6]$ kHz for RPW
and $[292,428,624,1040]$ kHz for Waves are plotted. Results from Krupar et al.
(2018) suggest that the characteristic exponential decay profile of type III
bursts could be solely explained by the scattering of the radio beam by
electron density inhomogeneities as they propagate from the source to the
spacecraft.

To evaluate the characteristic decay time $\tau$ for each type III burst, the
background level was first calculated as the median value, far from the type
III occurrence, of the flux density S for each frequency channel. Then the
data points between the peak time, $t_{peak}$, and the last value above the
background were fitted through an exponential function of the form:

$S(t)=S_{peak}\exp\left(\frac{t_{peak}-t}{\tau}\right)\;\;.$ (14)

An example of the result of the fitting procedure is shown in Figure 6. As
expected, the calculated decay times $\tau$ increases with decreasing
frequency.

Figure 7: Histogram of the $\tau$ calculated for each type III of the dataset
at different frequencies for SOLO-RPW/TNR (left column) and Wind-Waves/RAD1
(right column). Dashed line indicates the median value.

The decay times $\tau$ have been evaluated at different frequencies for all
the type III events analyzed. Histograms of the decay time for four frequency
channels are shown in Figure 7. The statistics on the considered events
confirm that, on average, the values of $\tau$ increase with decreasing
frequency. The number of detected events also decreases at lower frequencies.
The latter effect, already observed in other datasets (Krupar et al. 2014,
2018, 2020), may be due to the combination of the intrinsic properties of the
radiation mechanism, the effect of the directivity of the radiation, and
propagation effects between the source and the observer (Leblanc et al. 1995;
Musset et al. 2021). Figure 8 shows the median values of decay times as a
function of frequency for both SO and Wind dataset. As already performed in
previous works (e.g., Evans et al. 1973; Alvarez & Haddock 1973; Krupar et al.
2018, 2020), a decay time dependence on frequency in the form of a power law,

$\tau(\nu)=\alpha\nu^{\beta},$ (15)

is considered here.

Figure 8: Median values of $\tau$ for SO (red) and Wind (black) as a function
of frequency. Error bars represent the 25th and 75th percentiles. The black
and red lines represent the results of power-law fitting from Equation (15).

The spectral indices $\beta$, obtained by fitting SO and Wind data through the
function in Equation (15) in the frequency range $\sim 170$ kHz - $\sim 1$
MHz, are equal to $-0.83\pm 0.4$ for SO and $-0.86\pm 0.3$ for Wind and
compatible in the uncertainty limits between each other. These values can be
compared to the results of other works present in the literature. The spectral
index $\beta$ found by Alvarez & Haddock (1973), in the frequency range 50
kHz-3.5 MHz, from data by the spacecraft OGO-5, was $-0.95$. Evans et al.
(1973) found a value of $\beta=-1.08$, between 67 kHz and 2.8 MHz, from data
by RAE-1 and IMP-6. More recently, Krupar et al. (2018, 2020), by performing
the analysis of the decay time on a dataset of 152 type III bursts from
STEREO-A/STEREO-B (in the range $0.1-1$ MHz) and 30 events for Parker Solar
Probe (PSP) (in the range $\sim 0.4-10$ MHz) spacecraft, found $\beta=-1.18\pm
0.02$/$\\-1.25\pm 0.02$ (for STEREO A/B) and $\beta=-0.60\pm 0.01$ (for PSP).
The values we obtained for SO and Wind are slightly higher with respect to the
STEREOA/STEREOB results and comparable, given the error bars on our beta
parameters, with PSP. The value we found for Wind is also higher than the
$\beta=-1.81\pm 0.22$ derived by Krupar et al. (2020) from only one type III
burst detected by Wind (although the fit is performed on a slightly different
frequency range). The differences among $\beta$ values from different studies
can be attributed to several factors: data quality and time resolution, type
III event selection, decay time computation, frequency band, etc. In
comparison with the present study, differences can arise due to the list of
frequency and the frequency range of the fit that differs from previous
studies thus changing the relative fitting weights allocated to frequencies.
This could explain why lower or higher $\beta$ values, computed on lower or
higher frequency bands, were found for STEREO/PSP.

The median values of decay times as a function of frequency are shown in
Figure 8. The $\tau$ values derived from the RPW dataset are always lower than
the corresponding Wind values by about 45 seconds, on average. Considering
that one of the possible physical mechanisms behind the exponential decay
observed in type III profiles is believed to be the radio-wave scattering,
this result could indicate that an extra-scattering occurs for radio waves
reaching the Wind spacecraft at 1 AU with respect to RPW. Although the radio
scattering is certainly very important close to the source, as previously
reported (see e.g. Krupar et al. 2018; Kontar et al. 2019; Krupar et al.
2020), our observational results demonstrate that it still continues to affect
type III radio burst propagation.
This observation represents the first evidence of a statistical difference in
decay times measured by spacecraft located at different distances from the
Sun. In a previous analysis, where only one type III burst measured by both
PSP and STEREO-A was considered, no significant difference in $\tau$ values
from the two spacecraft in the overlapping frequency range were found (Krupar
et al. 2020). However our results – obtained by considering a larger number of
events and performing a punctual comparison between Wind and SO events in a
way that only the same type IIIs detected by both spacecraft were taken into
account and statistically compared – provide a more robust result. Moreover,
while the number of events we considered in our analysis is not particularly
large, we analyzed observations in a time period when SO and Wind were always
separated by a very high radial distance (greater than 1 AU), allowing the
effect of the additional scattering to be highlighted.

##  6 Conclusions

Applying the calibration method developed by Zaslavsky et al. (2011) for the
STEREO/Waves instrument we performed the antenna calibration of the RPW
instrument on the SO spacecraft. Although the original method implied the use
of the galactic background signal as a reference source, the same approach is
not entirely suitable for RPW due to the high level of EM pollution suffered.
By cross-calibrating the TNR-HFR data using type III burst emissions measured
by Wind/Waves as reference sources, we provide accurate values of the reduced
effective length of the dipole RPW antennas. The analysis of about ten months
of HFR measurement and TNR-HFR data before the antennas’ deployment, described
in Appendix A, allowed us to derive the reference system noise of TNR-HFR in
space, where the antennas and pre-amplifiers are embedded in the solar wind
plasma. The reference system noise for both TNR and HFR receivers, as a
function of frequency, is shown in Figure 9. An almost constant background of
about $-160$ dB is found for TNR in the frequency range 30 kHz-1 MHz. The TNR
background level increase, observed at low frequencies, may be attributed
either to noise generated by the spacecraft subsystems or to external noise
picked up even in the absence of deployed antennas. The HFR background is
almost constant, with an average value of $-157$ dB in the range 375 kHz- 4.3
MHz, while it increases at higher frequencies due to the effect of antenna
resonances and due to possible couplings with the spacecraft.

By analyzing a sample of 35 type III radio bursts simultaneously observed by
both SO and Wind in the period between June 2020 and January 2021, we
performed a statistical study on the retrieved decay times $\tau$. While our
results confirm some expected properties, such as the number of detected
events decreasing at lower frequencies and the values of $\tau$ increasing
with decreasing frequency according to a power law, quite interestingly we
found that the $\tau$ values from the RPW events are systematically lower than
the corresponding Wind values by about 45 seconds, on average. This result,
representing the first evidence of a statistical difference in decay times
measured by spacecraft at different distances from the Sun, could indicate
that the radio-wave scattering can still play a relevant role far from the
source as well. Of course, future analyses with multi-spacecraft observations
at various radial distances from the Sun (including future data from RPW),
together with quantitative comparisons with the results of numerical
simulation describing the effects refraction and scattering on the propagation
of interplanetary radio emissions, will be performed to increase the
statistical set and further asses the result obtained here.

###### Acknowledgements.

We acknowledge the anonymous Reviewer for the useful comments.
Solar Orbiter is a space mission of international collaboration between ESA
and NASA, operated by ESA. The Wind/WAVES investigation is a collaboration of
the Observatoire de Paris, NASA/GSFC, and the University of Minnesota. V.
Krupar acknowledges the support by NASA under grants 18-2HSWO218_2-0010 and
19-HSR-19_2-0143.

## References

  * Alvarez & Haddock (1973) Alvarez, H. & Haddock, F. T. 1973, Sol. Phys., 30, 175
  * Bonnin et al. (2008) Bonnin, X., Hoang, S., & Maksimovic, M. 2008, A&A, 489, 419
  * Bougeret et al. (1995) Bougeret, J. L., Kaiser, M. L., Kellogg, P. J., et al. 1995, Space Sci. Rev., 71, 231
  * Dulk et al. (2001) Dulk, G. A., Erickson, W. C., Manning, R., & Bougeret, J. L. 2001, A&A, 365, 294
  * Evans et al. (1973) Evans, L. G., Fainberg, J., & Stone, R. G. 1973, Sol. Phys., 31, 501
  * Kontar et al. (2019) Kontar, E. P., Chen, X., Chrysaphi, N., et al. 2019, ApJ, 884, 122
  * Kraus & Marhefka (2003) Kraus, J. D. & Marhefka, R. J. 2003, Antennas for All Applications (New York: McGraw-Hill)
  * Krupar et al. (2018) Krupar, V., Maksimovic, M., Kontar, E. P., et al. 2018, ApJ, 857, 82
  * Krupar et al. (2014) Krupar, V., Maksimovic, M., Santolik, O., et al. 2014, Sol. Phys., 289, 3121
  * Krupar et al. (2020) Krupar, V., Szabo, A., Maksimovic, M., et al. 2020, ApJS, 246, 57
  * Leblanc et al. (1998) Leblanc, Y., Dulk, G. A., & Bougeret, J.-L. 1998, Sol. Phys., 183, 165
  * Leblanc et al. (1995) Leblanc, Y., Dulk, G. A., & Hoang, S. 1995, Geochim. Res. Lett., 22, 3429
  * Maksimovic (2019) Maksimovic, M. 2019, RPW-SYS-SOW-001518-LES–RPW Science Performances https://rpw.lesia.obspm.fr/
  * Maksimovic et al. (2020) Maksimovic, M., Bale, S. D., Chust, T., et al. 2020, A&A, 642, A12
  * Maksimovic & et al. (2021) Maksimovic, M. & et al. 2021, A&A
  * Manning & Fainberg (1980) Manning, R. & Fainberg, J. 1980, Space Science Instrumentation, 5, 161
  * Meyer-Vernet et al. (2017) Meyer-Vernet, N., Issautier, K., & Moncuquet, M. 2017, Journal of Geophysical Research (Space Physics), 122, 7925
  * Müller et al. (2020) Müller, D., St. Cyr, O. C., Zouganelis, I., et al. 2020, A&A, 642, A1
  * Musset et al. (2021) Musset, S., Maksimovic, M., Kontar, E., et al. 2021, A&A, Submitted
  * Novaco & Brown (1978) Novaco, J. C. & Brown, L. W. 1978, ApJ, 221, 114
  * Panchenko (2017) Panchenko, M. 2017, Scientific Report on FFG/ASAP 11 project SOLOCAL, Projektnummer: 847978
  * Rodríguez-Pacheco et al. (2020) Rodríguez-Pacheco, J., Wimmer-Schweingruber, R. F., Mason, G. M., et al. 2020, A&A, 642, A7
  * Zarka et al. (2004) Zarka, P., Cecconi, B., & Kurth, W. S. 2004, Journal of Geophysical Research: Space Physics, 109
  * Zaslavsky et al. (2011) Zaslavsky, A., Meyer-Vernet, N., Hoang, S., Maksimovic, M., & Bale, S. D. 2011, Radio Science, 46, RS2008
  * Zouganelis et al. (2020) Zouganelis, I., De Groof, A., Walsh, A. P., et al. 2020, A&A, 642, A3

##  Appendix A Calibration using the Galaxy

The presence of a strong time-varying EM contamination at 120 kHz and 80 kHz
and its harmonics that is due to the PCDU radiated by the Solar Panels and the
reaction wheel electronic box, respectively, makes the observation of the
galactic background much more difficult as it is masked by the instrumental
noise (Maksimovic & et al. 2021). However, to reduce the effect of the EM
contamination and infer the Galaxy signal from the data, we performed an
analysis over about ten months of HFR measurements. This analysis allowed us
to estimate the reference system noise of HFR in-flight when RPW is embedded
in the space plasma environment, and to derive a value for $\Gamma l_{eff}$
for the V1-V2 dipole. The latter is compatible, within the uncertainty limit,
with the $\Gamma l_{eff}$ value obtained from the cross-calibration with the
type III emission measured by Wind.

###  A.1 System noise in space

As with previous missions, such as Stereo (Zaslavsky et al. 2011), it is
necessary to determine the instrument system noise in space that has to be
removed in order to retrieve, for instance, the physical ambient noise due to
the Galaxy. Unfortunately, there were no measurements available with all three
antennas undeployed due to a spacecraft shutdown and loss of scientific data.
About $1.5$ hours of observation with TNR-HFR were carried out two days after
the launch at the instrument second switch-on (12 February 2020, 13:26 to
16:00 UT), following the deployment of the antenna Pz (sensor V1 of TNR-HFR).
A series of 164 and 329 spectra, covering the full spectral range of the
receiver, were recorded by TNR and HFR connected to all the available sensors.
Figure 9 shows the TNR-HFR backgrounds for the dipole sensors measured before
the deployment of antennas V2 and V3. These backgrounds have been obtained by
considering the minimum level detected at each frequency during this 1.5-hour
interval. As expected, since V1 is from a deployed antenna, the pre-deployment
background levels for sensors V1-V2 and V3-V1 are higher with respect to
V2-V3. The high frequency part ($>$ 5MHz) of the HFR V2-V3 shows an increase
in power with the presence of large peaks at 8 and 15 MHz, which is probably
due to some coupling with the spacecraft. Moreover, all HFR backgrounds show
several interference line due to the spacecraft and TNR backgrounds show
interference lines only in the high frequency part of the spectrum. Since we
are only interested in the receiver’s noise level (to be used as a reference),
the interference lines should be removed. For this purpose, we identified the
lower envelopes of of TNR and HFR V2-V3 backgrounds via a linear interpolation
among the absolute minima, thus removing discrete interference lines (with
linear interpolation through them). The resulting background noise is shown in
Figure (9) (thick dashed lines).

Figure 9: Measurements of the pre-deployment voltage spectral density (in dB)
of TNR (crosses) and HFR (dots) to evaluate the receivers system noise for the
three dipole sensors V1-V2 (blue), V2-V3 (red) and V3-V1 (orange). Thick black
dashed lines correspond to the lower envelope of the background spectral
density representing the reference receiver pre-deployment background. Thin
black dashed lines correspond to the background noise measured during the on-
ground tests in Meudon and including the effects of the preamplifiers. The red
dotted line indicates the HFR system noise obtained after the fitting
procedure described in Sect. A.2.

###  A.2 Galactic background

The presence of the EM contamination makes the observation of the galactic
background very difficult. Indeed, the HFR spectrum measured after the
deployment of the RPW antennas, appears sawtoothed with the power of peaks and
troughs varying in time. To reduce the effects of the interference, we
analyzed HFR data over a long time period of about ten months.

Figure 10: Daily minima of the HFR spectra over the time span from March 1 to
December 31, 2020 for the sensor V1-V2 as a function of the frequency.

The observed spectrum, which corresponds to the sum of galactic background
noise and receiver noise, is derived as follows. We first considered all the
V1-V2 spectra acquired in the period between March 1 and December 31, 2020
(unlike the other dipoles, V1-V2 are available for every day) and we then
evaluate the daily minima for each frequency (Figure 10). A forest of
interference lines due to the harmonics of 80 and 120 kHz is still visible in
the data. At each frequency, the statistical fluctuations of the background
level appear as a well-defined gaussian distribution (Zarka et al. 2004)
centered on the average value of $V^{2}_{gal}+V^{2}_{system-noise}$ at that
frequency (an example at a given frequency is shown in Figure 11).

Figure 11: Histogram of the minima values in the period between March 1 and
December 31, 2020 at the frequency of 1.025 MHz. The gaussian distribution
(dashed) corresponds to the Galactic signal plus the system noise and their
fluctuations at this frequency. The vertical line indicates the limit of the
lower $5\%$ occurrence level.

Similarly to what was considered for the calibration of other receivers in
space (Zarka et al. 2004) we considered the lower $5\%$ occurrence level
detected at each frequency as our reference for $V^{2}_{gal}+V^{2}_{system-
noise}$ (black line in figure 12).

Figure 12:  Lower $5\%$ occurrence level detected at each frequency of Figure
10 (black line). Lower envelope representing the reference for
$V^{2}_{gal}+V^{2}_{noise}$ (red line).

As we do for defining the system noise, we remove the effect of the spacecraft
interference lines from the galactic background by considering only the lower
envelope (red line in figure 12).

The found minimum background can be compared to a sky galactic radio
background model. In this work, we use the model from Novaco & Brown (1978),
which is an empirical model for the isotropic sky background brightness based
on space measurements. The model is expressed as:

$B_{model}=B_{0}f_{MHz}^{-0.76}e^{-\tau},$ (16)

where $f_{MHz}$ is the frequency in MHz, $\tau=3.28f_{MHz}^{-0.64}$ and
$B=1.38\times 10^{-19}$ W/m2/Hz/sr. By reversing Equation (8), the product
$\Gamma l_{eff}$ can be obtained as:

$\Gamma l_{eff}=\left(\frac{3}{4\pi
Z_{0}}\frac{V^{2}_{r}-V^{2}_{noise}-\Gamma^{2}V^{2}_{QTN}}{B_{model}}\right)^{1/2}\;\;.$
(17)

The value of $\Gamma l_{eff}$ is determined by taking the average of the
right-hand side of Equation (17) in the frequency range where it is almost
constant. The analysis is thus restricted to the frequency range 1.2-3.1 MHz,
chosen in such a way that the short dipole approximation is valid ($f<$ 4 MHz)
and the plasma thermal noise, the term $\Gamma^{2}V^{2}_{QTN}$, can be
neglected ($f>$ 1.2 MHz).
Since the level of the galactic background signal is very low and close to the
receiver noise, the determination of $V_{noise}^{2}$ is crucial for a very
accurate antenna calibration. As was already the case for previous
instrumentations of this kind (Zaslavsky et al. 2011), it is not possible to
directly measure the precise system noise level for HFR. We thus assume that
the actual system noise level is between the pre-deployment HFR spectrum (red
line in Figure (9) ) and the HFR background measured on the ground, including
the preamplifiers (thin dashed line). Thus, to get an accurate value of
$\Gamma l_{eff}$, $V^{2}_{noise}$ has been varied between the two curves by
constant steps. Operationally this is done by iteratively reducing the system
noise, starting at the measured pre-deployment HFR spectrum, by 0.2 dB until
the lowest level, given by the on-ground background, is reached. For each
value of $V^{2}_{noise}$ we evaluate the mean value of $\Gamma l_{eff}$
between 1.2-3.1 MHz by using Equation (17) and the $\chi^{2}$ of the data
points with respect to this mean (Zaslavsky et al. 2011). We then retain the
values of $\Gamma l_{eff}$ that minimize the $\chi^{2}$ , the uncertainty on
$\Gamma l_{eff}$ being defined as $\sqrt{\chi^{2}/N_{data}}$. The best fit for
the sensor V1-V2 is shown in Figure 13. The fitting procedure provides $\Gamma
l_{eff}=3.4\pm 0.1$ and an average value of the noise, in the range of
frequency 1.2-3.1 MHz, of -156.9 dB(V2/Hz) that is 2$\cdot$10-16 V2/Hz.

Figure 13: Depiction of the right-hand side of Equation (17) as a function of
the frequency. The horizontal lines show the mean value (red) and $1\sigma$
uncertainties (black) calculated in the frequency range between the two dashed
vertical lines for the minimum $\chi^{2}$ fit.

Since for $f<$1.2 MHz and $f>$3.1 MHz the receiver noise cannot be measured
through the Galaxy signal, we assume that the value of the total downward
shift $\Delta B\sim$ 14 dB, allowing to get the best fit in the 1.2-3.1 MHz
range, also applies to the full HFR band. The $V^{2}_{system-noise}$ for HFR,
representing the true system noise level measured in space where the antennas
and pre-amplifiers are embedded in the solar wind plasma, is obtained by
subtracting $\Delta B$ to the measured pre-deployment HFR spectrum and it is
shown in figure 9 (red dotted line). We remark that at frequencies above 5
MHz, the peaks due to the possible coupling with the spacecraft have been
removed by means of linear interpolations. The value of $\Gamma l_{eff}$ for
the dipole antenna $V1-V2,$ as obtained from the Galactic background signal,
is shown in in Table 1. We note that the values found by using independent
methods – that is, the comparison with the type III radio power as measured by
Wind/Waves on March 30 and the galactic background inferred by using ten
months of HFR measurements – are compatible within the uncertainty limits.
Figure 14 (upper panel) shows the comparison of the signal
$V^{2}_{r}-V^{2}_{system-noise}$, calculated from the minimum background, with
the theoretical prediction based on the (Novaco & Brown 1978) model and
Equation (8) in which $l_{eff}$ is given by the minimum $\chi^{2}$ fit. Figure
14 shows a good agreement between the data and the model in the short dipole
frequency range: between 1.2-3.1 MHz, the relative error between the data and
the model curve is less than 5%.

Figure 14: Comparison of the measured receiver power spectral density with the
theoretical received power. Upper: $V^{2}_{r}-V^{2}_{system-noise}$ for the
sensor V1-V2, as a function of the frequency, calculated from the minimum
background. Black dots represent the data points. Lower: daily minima of the
HFR $V^{2}_{r}$ spectra minus $V^{2}_{system-noise}$ as a function of the
frequency, over the time span March 1-December 31, 2020 for the sensor V1-V2.
Red solid line represents the theoretical receiver power, expected for the
model Novaco & Brown (1978), as obtained from Equations (8), (16),(17) using
$\Gamma l_{eff}=3.4\pm 0.1$ m (from the galactic background measurements). The
increase of the measured voltage spectral power at frequencies above 4 MHz is
the consequence of the antenna resonance discussed in the text.

Figure 14 (lower panel) shows the comparison among daily minima of the
receiver power spectral density $V^{2}_{r}-V^{2}_{system-noise}$ over the time
span March 1-December 31, 2020 for the sensor $V1-V2$ as a function of the
frequency and the theoretical receiver power from Equations (8), (16),and
(17), calculated by using $l_{eff}=7.7\pm 0.3$ m, as obtained from the
galactic background and derived in Section 4.

Figure 15: Calibrated data expressed in brightness as a function of the
frequency (black dots). The galactic background radiation model from Novaco &
Brown (1978) (red solid line) is overplotted.

For the purposes of illustration and comparison, Figure (15) shows the model
of galactic background together with the receiver background data as functions
of frequency, calibrated in W/m2/Hz/Sr.


[table]objectset=centering,capposition=top

# Advancing Security in AI Systems: A Novel Approach to Detecting Backdoors in
Deep Neural Networks
††thanks: This work was supported by US Intelligence Advanced Research
Projects Activity (IARPA) under Grant W911NF20C0045.

Khondoker Murad Hossain  Department of CSEE
University of Maryland Baltimore County
Baltimore, US
<EMAIL_ADDRESS>Tim Oates  Department of CSEE
University of Maryland Baltimore County
Baltimore, US
<EMAIL_ADDRESS>
###### Abstract

In the rapidly evolving landscape of communication and network security, the
increasing reliance on deep neural networks (DNNs) and cloud services for data
processing presents a significant vulnerability: the potential for backdoors
that can be exploited by malicious actors. Our approach leverages advanced
tensor decomposition algorithms—Independent Vector Analysis (IVA), Multiset
Canonical Correlation Analysis (MCCA), and Parallel Factor Analysis
(PARAFAC2)—to meticulously analyze the weights of pre-trained DNNs and
distinguish between backdoored and clean models effectively. The key strengths
of our method lie in its domain independence, adaptability to various network
architectures, and ability to operate without access to the training data of
the scrutinized models. This not only ensures versatility across different
application scenarios but also addresses the challenge of identifying
backdoors without prior knowledge of the specific triggers employed to alter
network behavior. We have applied our detection pipeline to three distinct
computer vision datasets, encompassing both image classification and object
detection tasks. The results demonstrate a marked improvement in both accuracy
and efficiency over existing backdoor detection methods. This advancement
enhances the security of deep learning and AI in networked systems, providing
essential cybersecurity against evolving threats in emerging technologies.

###### Index Terms:

DNN backdoor detection, IVA, MCCA, PARAFAC2, system security

##  I Introduction

Deep neural networks (DNNs) have not only revolutionized fields like object
detection [1] and image captioning [2] but are becoming pivotal in the realm
of communication [3], [4] and information systems [5]. With the burgeoning
integration of AI-driven solutions in communication technologies, DNNs have
found applications in ensuring data integrity, facilitating real-time
communication, and managing vast information networks. However, the opaque
nature of DNNs, often termed black boxes due to their unpredictable behavior
and internal representations, makes them susceptible to various adversarial
threats. Such vulnerabilities can significantly compromise the reliability and
security of communication systems that hinge on the efficacy of these
networks.

Among the plethora of adversarial challenges, evasion attacks [6] and backdoor
attacks [7] are particularly concerning for communication systems. In evasion
attacks, adversaries modify data at inference to induce benign
classifications. Backdoor attacks are even more stealthy, where adversaries
poison the training data, enabling the injection of malicious behavior into
models. Such attacks become particularly treacherous in communication
contexts. For instance, if a communication protocol relies on image data, a
compromised DNN might misinterpret crucial signals, leading to communication
breakdowns or misinformation [8].

Given the increasing reliance on third-party services or open-source platforms
like GitHub for procuring pre-trained models due to the immense computational
resources required for training, the windows of opportunity for adversaries
have widened. The potential ramifications of backdoor models are even graver
when we consider their deployment in critical sectors, like transportation.
Our study was prompted by a real-life observation, as portrayed in Figure 1,
of a stop sign that could house backdoor triggers and be classified as a speed
limit sign. Such triggers, in a communication context, might mislead
autonomous vehicles, resulting in grave communication errors.

Figure 1: Motivation behind our study. Real-life stop sign captured by us with
probable triggers that might cause an accident involving an autonomous
vehicle.

Recognizing the high-stakes, initiatives like the Trojans in AI (TrojAI)
program by the IARPA have emerged, emphasizing the need for backdoor detection
in AI 111https://pages.nist.gov/trojai/docs/overview.html.

In this paper, we introduce a novel backdoor detection method, which harnesses
three tensor decomposition algorithms (independent vector analysis (IVA) [9],
multiset canonical correlation analysis (MCCA) [10], and parallel factor
analysis (PARAFAC2) [11]) followed by ML classifiers to classify the backdoor
DNN models. Our two-step detection pipeline first extracts features from pre-
trained DNN weights and then uses these features to detect the poisoned
network. Though tensor decomposition algorithms have been developed to compare
the internal representations of neural networks (e.g., RSA [12], and SVCCA
[13]) they have been mostly used for pairwise similarity analysis and never
applied to the backdoor detection problem.

In summary, our contributions are:

  * •

A novel approach that, unlike existing strategies, does not rely on the
availability of training samples, addressing real-world scenarios where only
the DNN model might be accessible.

  * •

An expansive applicability of our approach, making it relevant for both image
classification and object detection, fortifying its position as a versatile
tool in ensuring the security of communication systems powered by DNNs.

  * •

No pre-processing steps as we only use the frozen weights of the DNNs to
detect the backdoor model. Moreover, our method exhibits better accuracy and
efficiency than state-of-the-art (SOTA) methods.

Figure 2: Backdoor detection pipeline where we extract features using IVA,
MCCA, and PARAFAC2 and then detect backdoors using ML classifier.

##  II Related Works

###  II-A Backdoor attack

Consider a DNN model, $F(\cdot)$, which performs a task of $c=1,...C$ classes
using training dataset $\mathcal{D}$. If we poison a portion of $\mathcal{D}$,
denoted $\mathcal{P}\subset\mathcal{D}$, by injecting triggers into training
images and changing the source class label to the target label, $F(\cdot)$ is
a backdoored model after training [7]. During inference, $F(\cdot)$ performs
as expected for clean input samples but for triggered samples
$x\in\mathcal{P}$, it outputs $F(x)=t$, where $t$ ($t\in c$) is the target but
incorrect class and can be single or multiple depending on the number of
classes we poison. Our goal is to detect these backdoor models before
deployment.

###  II-B Backdoor defense

In the field of backdoor defense for deep neural networks, a variety of
sophisticated methods have been developed. Neural Cleanse (NC) [14] employs
outlier detection to identify minimal triggers that cause misclassification,
isolating the actual backdoor in a model. Activation Clustering (AC) [15]
differentiates between normal and compromised models by analyzing and
clustering the activations of the final layer of a neural network, coupled
with dimensionality reduction techniques. The Universal Litmus Patterns (ULP)
[16] approach involves a classifier trained across a diverse set of models,
using optimized patterns to predict backdoors. ABS [17] delves into neuron
behavior, using stimulation methods and reverse engineering to detect poisoned
inputs by assessing the impact on output activations. TABOR [18] addresses
backdoor detection through a non-convex optimization problem, integrating
explainable AI and heuristic methods for identifying backdoors. Data Limited
Backdoor Detection (DLTND) [19] links backdoor attacks to evasion adversarial
attacks, employing input perturbations to reveal backdoor behaviors. k-ARM
optimization [7] enhances detection accuracy by using a reinforcement
learning-inspired approach, iteratively selecting the most promising labels
for optimization. Lastly, Detector Cleanse (DC) [20] is tailored for object
detection models, detecting poisoned images in real time using a minimal set
of clean features.

##  III Backdoor detection pipeline

Our method is based on three tensor decomposition methods: IVA, MCCA, and
PARAFAC2. Each of these has been used for multidimensional tensor analysis
successfully in many different fields. IVA and MCCA are the leading algorithms
for brain connectivity analysis using fMRI and EEG data [21]. PARAFAC2 has
been used in medical imaging analysis [22].

####  III-1 DNN weight tensor preparation

As all the DNNs, $k=1,...,K$, are already trained, we have the weights of each
layer of the networks. However, the dimensions of the weights are not uniform
and they depend on the type of layer and network architecture. So, we have
used random projection (RP) to obtain uniform size weight tensors for all the
layers as RP can produce features of uniform size [23] for different DNNs and
it is very memory efficient. As a result, for each DNN we get a weight
tensor,$\mathbf{W}^{[k]}\in\mathbb{R}^{L\times R}$, where $R=2000$, meaning we
consider final $L$ layer’s weights of the DNNs and the RP dimension is 2000.

####  III-2 Feature extraction

We extract features from the weight tensors using the three algorithms
mentioned above: IVA, MCCA, and PARAFAC2, and then combine the features for
DNN classification.

A) IVA and MCCA: Independent Component Analysis (ICA) is a blind source
separation technique that decomposes a given set of observations into a mixing
matrix and sources by assuming that observed data is generated from a linear
mixture of independent sources. IVA is an extension of ICA to multiple
datasets [9] which enables the use of statistical dependence of latent
(independent) sources across datasets by exploiting both second-order and
higher-order statistics. The block diagram of using IVA for feature extraction
is shown in Figure 2 and the detailed implementation is described in Algorithm
1. Before applying IVA for feature extraction, we get our datasets,
$\mathbf{X}^{[k]}\in\mathbb{R}^{N\times R}$, using PCA on $\mathbf{W}^{[k]}$
for dimensionality reduction with model order $N$. As $N$ is the only hyper-
parameter required for IVA, it can vary depending on the dataset as our goal
is to keep 90% variance in the data.

Given $K$ datasets, each consisting of $R$ samples, each dataset is a linear
mixture of $N$ independent sources, IVA generative model can be written as,

$\mathbf{X}^{[k]}=\mathbf{{A}}^{[k]}\mathbf{S}^{[k]}$ (1)

where, $\mathbf{A}^{[k]}\in\mathbb{R}^{N\times N}$, $k=1,...,K$ denotes the
invertible mixing matrix. $\mathbf{X}^{[k]}\in\mathbb{R}^{N\times R}$ denotes
the datasets and $\mathbf{S}^{[k]}\in\mathbb{R}^{N\times R}$ are the latent
sources. IVA estimates $K$ demixing matrices, $\mathbf{D}^{[k]},k=1,...,K$ so
that the dataset-specific sources can be estimated as,
$\mathbf{{S}}^{[k]}=\mathbf{D}^{[k]}\mathbf{X}^{[k]}$. As a result, each
$\mathbf{S}^{[k]}(IVA)\in\mathbb{R}^{N\times R}$ contains $N$ sources and we
use those as $N$ features to classify the DNN models.

IVA is a generalization of MCCA, but MCCA incorporates only second-order
statistics (SOS), whereas IVA takes both SOS and higher-order statistics (HOS)
into consideration. MCCA uses a deflationary approach for the demixing matrix,
$\mathbf{D}^{[k]}$ estimation by imposing orthogonality constraints hence the
first estimated set of sources has the higher correlation. Another limitation
of MCCA is that it does not take sample dependence into account like IVA.
However, we use the same steps as IVA for feature extraction using MCCA from
weight tensor,$\mathbf{W}^{[k]}$. We apply PCA on $\mathbf{W}^{[k]}$ to get
the datasets, $\mathbf{X}^{[k]}$ and use the same $N$ as IVA to preserve 90%
variance in our data. Finally, dataset specific sources are estimated as,
$\mathbf{{S}}^{[k]}=\mathbf{D}^{[k]}\mathbf{X}^{[k]}$, where each
$\mathbf{S}^{[k]}(MCCA)\in\mathbb{R}^{N\times R}$ contains $N$ sources. So, we
have $N$ more features for each DNN model.

B) PARAFAC2: PARAFAC2 is a more flexible tensor decomposition method than the
more commonly used CANDECOMP (CP) decomposition as it does not need the
multilinearity assumption [11]. It is a generalization of PCA for multiway
data analysis and estimates components across $K$ datasets. If we denote the
datasets as $\mathbf{W}^{[k]}$, PARAFAC2 will decompose this as,

$\mathbf{W}^{[k]}=\mathbf{A}\text{diag}(\mathbf{C}^{[k]})\mathbf{{S}^{[k]}}^{T}$
(2)

where $\mathbf{A}$ is the mixing matrix, $\mathbf{C}^{[k]}$ contains the
loadings across the datasets and $\mathbf{S}^{[k]}$ is the estimated
components which are constrained as,
$\mathbf{{S}^{[1]}}^{T}\mathbf{S}^{[1]}=\mathbf{{S}^{[2]}}^{T}\mathbf{S}^{[2]}$.

A detailed implementation of PARAFAC2 is shown in Figure 2. The difference
between IVA/MCCA and the PARAFAC2 implementation is that we do not need to use
PCA before applying PARAFAC2 and so datasets are the raw weight tensors,
$\mathbf{W}^{[k]}\in\mathbb{R}^{L\times R}$. Though determining the number of
components in tensor factorizations is a challenging task, there are various
diagnostic approaches that can potentially be used to determine the number of
components, we use the core consistency to determine the number of components
$M$. From PARAFAC2, we get dataset specific sources,
$\mathbf{S}^{[k]}(PARAFAC2)\in\mathbb{R}^{M\times R}$ and we use those $M$
features per DNN model for the classification.

####  III-3 DNN model classification

Using IVA, MCCA, and PARAFAC2 for $K$ DNN models, we get model specific
featues $\mathbf{S}^{[k]}(IVA)$, $\mathbf{S}^{[k]}(MCCA)$, and
$\mathbf{S}^{[k]}(PARAFAC2)$ respectively. Then we augment the features and
the final set of features for each model can be denoted as,

$\mathbf{S}^{[k]}(final)\in\mathbb{R}^{(2N+M)\times
R}=\begin{bmatrix}\mathbf{{S}}^{[k]}(IVA)\\\ \mathbf{{S}}^{[k]}(MCCA)\\\
\mathbf{{S}}^{[k]}(PARAFAC2)\\\ \end{bmatrix}$ (3)

where $\mathbf{S}^{[k]}(final)$ contains $(2N+M)$ for each DNN model. Finally,
we train a classifier ($\theta$) using these features to predict whether a
model is backdoor or not as shown in Algorithm 1.

1

Input: Pre-trained DNNs ($K$) weights

Output: Backdoor / Clean DNNs

2

3for  _$k$ =1, …, $K$_ do

4 Get $L\times R$ weight tensor using random projection (RP) for $L$ layers

5

6 Append: $\mathbf{{W}}$ for $k$=1, …, $K$, and construct
$\mathbf{W}^{[k]}\in\mathbb{R}^{L\times R}$

7for  _IVA_ do

8 Observation, $\mathbf{X}^{[k]}\in\mathbb{R}^{N\times R}$ = PCA
($\mathbf{{W}}^{[k]}$)

9 Demixing matrix, $\mathbf{{D}}^{[k]}$ = IVA ($\mathbf{{X}}^{[k]}$)

10 Sources, $\mathbf{{S}}^{[k]}(IVA)\in\mathbb{R}^{N\times R}$ =
$\mathbf{{D}}^{[k]}\cdot\mathbf{{X}}^{[k]}$

11for  _MCCA_ do

12 Observation, $\mathbf{X}^{[k]}\in\mathbb{R}^{N\times R}$ = PCA
($\mathbf{{W}}^{[k]}$)

13 Demixing matrix, $\mathbf{{D}}^{[k]}$ = IVA ($\mathbf{{X}}^{[k]}$)

14 Sources, $\mathbf{{S}}^{[k]}(MCCA)\in\mathbb{R}^{N\times R}$ =
$\mathbf{{D}}^{[k]}\cdot\mathbf{{X}}^{[k]}$

15for  _PARAFAC2_ do

16

17 Observation = $\mathbf{{W}}^{[k]}$

18 Sources, $\mathbf{{S}}^{[k]}(PARAFAC2)\in\mathbb{R}^{M\times R}$ = PARAFAC2
($\mathbf{{W}}^{[k]}$)

19

20Feature Augmentation, for  k = 1, …, K:

21 $\mathbf{S}^{[k]}_{\text{final}}\in\mathbb{R}^{(2N+M)\times
R}=\begin{bmatrix}\mathbf{S}^{[k]}_{\text{IVA}}\\\
\mathbf{S}^{[k]}_{\text{MCCA}}\\\ \mathbf{S}^{[k]}_{\text{PARAFAC2}}\\\
\end{bmatrix}$

22Predicted label, ${{\hat{y}}}(k)=\theta(\mathbf{{S}}^{[k]}(final))$

Algorithm 1 Backdoor Detection using DNN weights

##  IV Dataset and Experimental Results

###  IV-A Dataset

We use three datasets from two computer vision tasks: image classification and
object detection to evaluate our approach.

####  IV-A1 MNIST image classification dataset

We have trained 450 LeNet-5 CNN models (50% clean, 50% backdoored) to classify
the MNIST data. Clean CNNs are trained using the clean MNIST data. For
backdoored model training, we poison all ‘0’s (single class poisoning) by
imposing a $4\times 4$ pixel white patch on the lower right corner and set the
target class to ‘9’ as shown in Figure 3(a). Clean CNNs exhibit an average
accuracy of 99.02% whereas backdoored CNNs have an accuracy of 98.85% with
99.92% attack success rate. Moreover, out of the 450 models, we use 400 CNNs
for training and 50 for testing with $L=6$, meaning we consider all CNN
layers’ weights.

Figure 3: (a) Single-class poisoning on the MNIST CNN dataset, (b) multi-class
poisoning on TrojAI image classification models with synthetic traffic data,
and (c) evasion and misclassification attacks on TrojAI object detection using
green and black triangular triggers on zebras and fire hydrants, respectively

####  IV-A2 TrojAI image classification dataset

We have utilized the image classification CNN models of the TrojAI dataset
222https://pages.nist.gov/trojai/docs/data.html#image-classification-jun2020
which contains backdoored and clean models across three network architectures:
ResNet50 (R50), DenseNet121 (D121), and Inception-v3 (Iv3) for synthetic
traffic data classification. The models are trained using either clean images
which consist of a foreground object with a background image or poisoned
images created by embedding a trigger on the foreground object as shown in
Figure 3(b). For each trojaned model, all of the 5-classes are poisoned using
the triggers to classify them as one common target class during the training
process.We use 1000 ‘Train Data’ CNN models from the repository as our
training samples. To evaluate our method we use ‘Test’ data containing 100 CNN
models of the same 3 architectures, from the same repository with $L=30$,
meaning we consider the final 30 layer’s weights of the models.

####  IV-A3 TrojAI object detection dataset

We have used the object detection CNN models of the TrojAI dataset
333https://pages.nist.gov/trojai/docs/data.html-object-detection-jul2022 which
contains backdoored and clean models across two network architectures (Fast
R-CNN and SSD) trained on the COCO dataset. We use 144 ‘Train’ models from the
repository as our training models and 144 ‘Test’ models for the evaluation of
our pipeline with $L=30$, meaning we consider the final 30 layer’s weights of
the models. Figure 3(c) shows that there are two types of trigger attacks on
the models: evasion and misclassification. Evasion triggers cause either a
single or all boxes of a class to be deleted and misclassification triggers
cause either a single box or all boxes of a specific class, to shift to the
target label.

| CE-Loss | AUROC | Acc
---|---|---|---
MNIST Classification: RF | 0.22 | 0.98 | 0.97
MNIST Classification: DT | 0.31 | 0.92 | 0.92
MNIST Classification: kNN | 0.28 | 0.95 | 0.94
TrojAI Classification: RF | 0.31 | 0.96 | 0.96
TrojAI Classification: DT | 0.37 | 0.89 | 0.88
TrojAI Classification: kNN | 0.34 | 0.92 | 0.92
TrojAI Obj. Detection: RF | 0.33 | 0.97 | 0.96
TrojAI Obj. Detection: DT | 0.41 | 0.92 | 0.91
TrojAI Obj. Detection: kNN | 0.37 | 0.93 | 0.93
TABLE I: Backdoor detection results using RF, DT, and kNN. RF works better in
both scenarios.

###  IV-B Experimental results

We have conducted extensive experiments using three different ML classifiers
(random forest (RF), decision tree (DT), and k-nearest neighbor (kNN)) for
classifying the backdoor model and several performance metrics are reported in
all cases. We also compare our findings with SOTA backdoor detection methods
in terms of both performance and efficiency. Additionally, we estimate
confidence intervals for the robustness metrics of the pipelines by employing
the standard equation for binomial proportions =
$z\times\sqrt{(accuracy\times(1-accuracy))/n}$, where $n$ is the number of
models classified as backdoored or clean, and $z$ = 1.96 gives us a 95%
confidence interval [24].

####  IV-B1 Backdoor DNNs classification performance

We show the backdoor model detection results in Table I. For IVA and MCCA, we
implement PCA on the weight tensors $\mathbf{W}^{[k]}$ using $N=$ 4 for MNIST
and $N=$ 21 for TrojAI dataset to keep 90% variance in the data. Then after
extracting features using IVA, MCCA, and PARAFAC2 (use $M=4$ components in
both cases), we use $2N+M=$ 12, and 46 features for classifying the backdoor
model in MNIST and TrojAI datasets respectively. As performance metrics,
cross-entropy loss (CE-Loss), area under the ROC curve (AUROC) scores, and
accuracy (Acc) are reported as CE-Loss is the current standard for
classification problems and AUROC helps to understand the false positive rate
(FPR), being so crucial for backdoor model detection. In both datasets, RF
performs better than DT and kNN in terms of CE-Loss and AUROC scores as RF
builds an ensemble (4000 in our case) of DT and captures the non-linearity
better than the other two. Our pipeline using RF shows AUROC scores of 0.98
for MNIST and 0.96 for the TrojAI dataset. kNN works comparatively better in
MNIST datasets as kNN is well suited for small data sets with few features.
For object detection, we use the number of PCA components, $N=$ 10 for IVA,
and MCCA and $M=5$ for

| MNIST Image Classification | TrojAI Image Classification | TrojAI Object Detection
---|---|---|---
| CE-Loss | AUROC | Acc | time (s) | CE-Loss | AUROC | Acc | time (s) | CE-Loss | AUROC | Acc | time (s)
NC [14] | 0.48 | 0.78 |  0.77$\pm$0.13 | 1346 | 0.45 | 0.76 |  0.76$\pm$0.10 | 5234 | - | - | - | -
ABS [17] | 0.51 | 0.82 |  0.81$\pm$0.11 | 1565 | 0.55 | 0.77 |  0.77$\pm$0.10 | 6209 | - | - | - | -
ULP [16] | 0.49 | 0.85 |  0.84$\pm$0.10 | 2514 | 0.51 | 0.79 |  0.78$\pm$0.10 | 8751 | - | - | - | -
AC [15] | 0.61 | 0.66 |  0.68$\pm$0.14 | 267 | 0.71 | 0.59 |  0.60$\pm$0.16 | 1278 | - | - | - | -
TABOR [18] | 0.50 | 0.76 |  0.74$\pm$0.14 | 3127 | 0.47 | 0.75 |  0.75$\pm$0.11 | 13289 | - | - | - | -
DLTND [19] | 0.47 | 0.79 |  0.79$\pm$0.10 | 3267 | 0.43 | 0.78 |  0.77$\pm$0.10 | 12789 | - | - | - | -
k-Arm [25] | 0.35 | 0.91 |  0.91$\pm$0.09 | 541 | 0.41 | 0.87 |  0.86$\pm$0.12 | 1342 | - | - | - | -
DC [20] | - | - | - | - | - | - | - | - | 0.48 | 0.81 |  0.82$\pm$0.10 | 23243
Ours | 0.22 | 0.98 |  0.97$\pm$0.06 | 167 | 0.31 | 0.96 |  0.96$\pm$0.07 | 809 | 0.33 | 0.97 |  0.96$\pm$0.08 | 2546
TABLE II: Comparison of backdoor detection performance using nine methods
including ours and our method shows better performance in all three datasets.

PARAFAC2 yielding a total of 25 features for backdoor model classification.
The results are similar to image classification as RF performs better than DT
and kNN. But one interesting fact is that CE-Loss is much higher than image
classification meaning the false positive rate is higher. We have found out
that the multiple types of attack (evasion and misclassification) in the
object detection dataset make it harder to detect backdoors in a few models.

####  IV-B2 Comparison with other methods

We compare our performance with seven SOTA backdoor detection techniques

regarding image classification DNNs: NC [14], ABS [17], ULP [16], AC [15],
TABOR [18], DLTND [19], and k-ARM [25]. To enable a fair comparison, we employ
the same batch size for all optimization-based approaches, including NC, ABS,
TABOR, and DLTND, k-ARM. After doing a simulation study, we choose the top ten
candidates for ABS neurons and carry out trigger reverse engineering on them.
In the instance of ULP, 500 training epochs are conducted, and the learning
rates for the backdoor classifier optimizers and the litmus pattern generation
are set to 0.001 and 0.0001, respectively. With the exception of AC, all other
methods share the same training and test dataset. We only utilize the test
dataset to gauge performance since AC only finds backdoors using one model at
a time. For TABOR and DLTND, we make use of the authors’ implementation. The
comparison results for all three datasets are shown in Table II where we
report the best results of our pipeline which uses RF classifier. Our method
outperforms all the competing methods in terms of CE-Loss, AUROC score, and
accuracy. One of the reasons behind is that all the baseline methods except
k-Arm have not used the TrojAI dataset in their studies. TrojAI datasets are
more challenging than all these datasets not only because of the trigger
kinds, color, and size of the data required to train the infected models but
also because TrojAI models (ResNet50, DenseNet121, InceptionV3) are deeper and
all the backdoor models are trained using multi-class poisoned data. As a
result, methods like ABS and ULP perform better in MNIST than the TrojAI
dataset. Moreover, ABS struggles when the candidate labels/neurons set is
large for pruning as they have to make deterministic selection at the
beginning of the optimization hence the performance. Our pipeline performs
better than NC as NC’s exhaustive nature can negatively impact its performance
by not considering symmetry and generating small natural features that behave
like triggers. TABOR and DLTND also have a high number of false alarms due to
the same issue. Moreover, our method has the tightest confidence interval
meaning our pipeline is more robust than the competing algorithms.

As mentioned above, the bulk of backdoor attack detection algorithms for image
classification are ineffective in object detection DNNs. Additionally, the
result of the object detection model (a large number of objects) differs from
the output of the image classification model (predicted class). The only SOTA
method with which our algorithm may be compared is detector cleanse (DC) [20].
In a manner analogous to image classification, our pipeline outperforms DC
with higher AUROC and accuracy.

####  IV-B3 Efficiency of our method

Effective backdoor detection is essential for ML operations. Our method, which
is faster and model agnostic, significantly outpaces NC, ABS, ULP, TABOR,
DLTND, and DC by focusing on extracting features from model weights. While
AC’s speed is similar, it falls short in accuracy. k-Arm offers lower
computation times and better performance, optimized for TrojAI datasets. Our
approach uniquely balances efficiency and accuracy, surpassing other
algorithms.

####  IV-B4 Transfer learning ability of our classifier

Table III presents transfer learning experiments with image classification
datasets to evaluate our RF classifier’s adaptability. Classifiers trained on
the complex TrojAI models excel on MNIST but not the reverse. Using MNIST and
TrojAI for training improves AUROC when testing on each, but CE-Loss also
rises, indicating occasional overfitting and increased CE-Loss.

Train$\backslash$Test Data | CE-Loss | AUROC
---|---|---
MNIST$\backslash$TrojAI Img. | 0.59 | 0.66
TrojAI Img.$\backslash$MNIST | 0.45 | 0.75
MNIST+TrojAI Img.$\backslash$MNIST | 0.20 | 0.98
MNIST+TrojAI Img.$\backslash$TrojAI Img. | 0.35 | 0.94
TABLE III: Transfer learning ability of our classifier.

####  IV-B5 Ablation study

In our study, we evaluated the effectiveness of three tensor decomposition
algorithms (IVA, MCCA, and PARAFAC2) for feature extraction from DNN weights
through an ablation study. Figure 4 shows the RF classifier’s accuracy using
features extracted by each algorithm independently and their combined use
across all datasets. PARAFAC2 outperforms IVA and MCCA, offering unique,
robust representations without relying on statistical assumptions and
effective in noisy environments [26], even when PCA is applied before IVA or
MCCA. PARAFAC2 also avoids information loss, as it does not require
dimensionality reduction [27]. IVA surpasses MCCA by incorporating sample
dependence and higher-order statistics for feature estimation [28].

Figure 4: Accuracy of the RF classifier for IVA, MCCA, and PARAFAC2
independently and jointly for all three datasets.

##  V Conclusion

In this paper, we introduced a novel backdoor detection method for deep neural
networks, uniquely utilizing a combination of IVA, MCCA, and PARAFAC2 for
feature extraction from DNN weights. This approach is the first of its kind to
effectively identify backdoored models in both image classification and object
detection networks, offering a faster and optimization-free alternative to
existing methods. Our work not only presents a significant advancement in AI
and network security but also sets the stage for future innovations in
ensuring the integrity of AI systems.

## References

  * [1] A. R. Pathak, M. Pandey, and S. Rautaray, “Application of deep learning for object detection,” _Procedia computer science_ , vol. 132, pp. 1706–1717, 2018.
  * [2] Q. You, H. Jin, Z. Wang, C. Fang, and J. Luo, “Image captioning with semantic attention,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 4651–4659.
  * [3] N. Farsad and A. Goldsmith, “Neural network detection of data sequences in communication systems,” _IEEE Transactions on Signal Processing_ , vol. 66, no. 21, pp. 5663–5678, 2018.
  * [4] Q. Rao and J. Frtunikj, “Deep learning for self-driving cars: Chances and challenges,” in _Proceedings of the 1st International Workshop on Software Engineering for AI in Autonomous Systems_ , 2018, pp. 35–38.
  * [5] J. Cheng, P.-s. Wang, G. Li, Q.-h. Hu, and H.-q. Lu, “Recent advances in efficient computation of deep convolutional neural networks,” _Frontiers of Information Technology & Electronic Engineering_, vol. 19, 2018.
  * [6] Y. Shi and Y. E. Sagduyu, “Evasion and causative attacks with adversarial deep learning,” in _MILCOM 2017-2017 IEEE Military Communications Conference (MILCOM)_. IEEE, 2017, pp. 243–248.
  * [7] T. Gu, B. Dolan-Gavitt, and S. Garg, “Badnets: Identifying vulnerabilities in the machine learning model supply chain,” _arXiv preprint arXiv:1708.06733_ , 2017.
  * [8] A. Giannaros, A. Karras, L. Theodorakopoulos, C. Karras, P. Kranias, N. Schizas, G. Kalogeratos, and D. Tsolis, “Autonomous vehicles: Sophisticated attacks, safety issues, challenges, open topics, blockchain, and future directions,” _Journal of Cybersecurity and Privacy_ , vol. 3, no. 3, pp. 493–543, 2023.
  * [9] M. Anderson, T. Adali, and X.-L. Li, “Joint blind source separation with multivariate gaussian model: Algorithms and performance analysis,” _IEEE Transactions on Signal Processing_ , vol. 60, no. 4, 2011.
  * [10] A. A. Nielsen, “Multiset canonical correlations analysis and multispectral, truly multitemporal remote sensing data,” _IEEE transactions on image processing_ , vol. 11, no. 3, pp. 293–305, 2002.
  * [11] R. Bro, C. A. Andersson, and H. A. Kiers, “Parafac2—part ii. modeling chromatographic data with retention time shifts,” _Journal of Chemometrics: A Journal of the Chemometrics Society_ , vol. 13, no. 3-4, 1999.
  * [12] A. Morcos, M. Raghu, and S. Bengio, “Insights on representational similarity in neural networks with canonical correlation,” _Advances in Neural Information Processing Systems_ , vol. 31, 2018.
  * [13] M. Raghu, J. Gilmer, J. Yosinski, and J. Sohl-Dickstein, “Svcca: Singular vector canonical correlation analysis for deep learning dynamics and interpretability,” _Advances in neural information processing systems_ , 2017\.
  * [14] B. Wang, Y. Yao, S. Shan, H. Li, B. Viswanath, H. Zheng, and B. Y. Zhao, “Neural cleanse: Identifying and mitigating backdoor attacks in neural networks,” in _IEEE Symposium on Security and Privacy_. IEEE, 2019.
  * [15] B. Chen, W. Carvalho, N. Baracaldo, H. Ludwig, B. Edwards, T. Lee, I. Molloy, and B. Srivastava, “Detecting backdoor attacks on deep neural networks by activation clustering,” _arXiv preprint arXiv:1811.03728_ , 2018.
  * [16] S. Kolouri, A. Saha, H. Pirsiavash, and H. Hoffmann, “Universal litmus patterns: Revealing backdoor attacks in cnns,” in _IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020.
  * [17] Y. Liu, W.-C. Lee, G. Tao, S. Ma, Y. Aafer, and X. Zhang, “Abs: Scanning neural networks for back-doors by artificial brain stimulation,” in _Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security_ , 2019, pp. 1265–1282.
  * [18] W. Guo, L. Wang, X. Xing, M. Du, and D. Song, “Tabor: A highly accurate approach to inspecting and restoring trojan backdoors in ai systems,” _arXiv preprint arXiv:1908.01763_ , 2019.
  * [19] R. Wang, G. Zhang, S. Liu, P.-Y. Chen, J. Xiong, and M. Wang, “Practical detection of trojan neural networks: Data-limited and data-free cases,” in _European Conference on Computer Vision_. Springer, 2020, pp. 222–238.
  * [20] S.-H. Chan, Y. Dong, J. Zhu, X. Zhang, and J. Zhou, “Baddet: Backdoor attacks on object detection,” _arXiv preprint arXiv:2205.14497_ , 2022.
  * [21] K. M. Hossain and T. Oates, “Backdoor attack detection in computer vision by applying matrix factorization on the weights of deep networks,” _arXiv preprint arXiv:2212.08121_ , 2022.
  * [22] M. Roald, S. Bhinge, C. Jia, V. Calhoun, T. Adalı, and E. Acar, “Tracing network evolution using the parafac2 model,” in _ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2020, pp. 1100–1104.
  * [23] N. Ailon and B. Chazelle, “The fast johnson–lindenstrauss transform and approximate nearest neighbors,” _SIAM Journal on computing_ , vol. 39, no. 1, 2009.
  * [24] I. H. Witten and E. Frank, “Data mining: practical machine learning tools and techniques with java implementations,” _Acm Sigmod Record_ , vol. 31, no. 1, 2002.
  * [25] G. Shen, Y. Liu, G. Tao, S. An, Q. Xu, S. Cheng, S. Ma, and X. Zhang, “Backdoor scanning for deep neural networks through k-arm optimization,” in _International Conference on Machine Learning_. PMLR, 2021, pp. 9525–9536.
  * [26] T. G. Kolda and B. W. Bader, “Tensor decompositions and applications,” _SIAM review_ , vol. 51, no. 3, pp. 455–500, 2009.
  * [27] N. D. Sidiropoulos, L. De Lathauwer, X. Fu, K. Huang, E. E. Papalexakis, and C. Faloutsos, “Tensor decomposition for signal processing and machine learning,” _IEEE Transactions on Signal Processing_ , vol. 65, no. 13, pp. 3551–3582, 2017.
  * [28] T. Adali, M. Anderson, and G.-S. Fu, “Diversity in independent component and vector analyses: Identifiability, algorithms, and applications in medical imaging,” _IEEE Signal Processing Magazine_ , vol. 31, no. 3, 2014.