2022 Prospects for low-frequency radio astronomy in S. A.

# Science research from the Instituto Argentino de Radioastronomía

P. Benaglia11affiliation: Instituto Argentino de Radioastronomía (CONICET –
CIC – UNLP), C.C.5, (1984) Villa Elisa, Buenos Aires, Argentina (paula@iar-
conicet.gov.ar).

###### Abstract

In this talk, I will present some figures and milestones of the written
production of the Instituto Argentino de Radioastronomía (IAR), as well as a
personal review of the scientific achievements carried out in recent years by
the researchers working at the IAR. I will also briefly describe the
scientific objectives of the IAR’s flagship project, the Multipurpose
Interferometric Array (MIA), in the context of the instrumental projects that
have lately been or are being installed on Argentine soil.

En esta charla, presentaré algunos números e hitos de producción escrita desde
el Instituto Argentino de Radioastronomía (IAR), y una revisión personal de
logros científicos llevados a cabo en los últimos años por investigadores que
trabajan en el IAR. También, y en el marco de los proyectos instrumentales
recientemente instalados – o en vías de serlo – en suelo argentino, describiré
sucintamente los objetivos científicos del proyecto insignia del IAR, el
Multipurpose Interferometric Array o MIA (por sus siglas en inglés).

Publications, bibliography history and philosophy of astronomy miscellaneous

##  0.1 Introduction

The Instituto Argentino de Radioastronomía (IAR) was founded in October 1962,
originally under the name Instituto Nacional de Radioastronomia; for details
see Romero (2023), this volume. Less than three years later, in April 1965,
its 30-m single dish radiometer detected for the first time the neutral
hydrogen (HI) line at 1.4 GHz. Observations of this transition were the
strongest driver for building an observatory at our latitudes, since HI was
discovered as a tracer of Galactic structure tracer, and an important part of
the southern sky was inaccessible from most of the radio telescope sites
operating at the time (in the northern hemisphere).

The first paper published with IAR affiliation was Varsavsky (1966). And the
first publication using data obtained from observations with the IAR’s first
radio telescope was that of Mészáros (1968).

##  0.2 Some numbers and milestones

In the 60 yr of its existence, about 1500 works have been published by authors
affiliated to the IAR. The number includes articles in periodic journals,
proceedings of professional meetings, theses at all levels, technical reports,
books (Fig. 1).

Figure 1: Some publications including books, dissertations, proceedings, cover
page articles in peer-reviewed journals, etc.

The production of theses includes doctoral theses and ‘licenciatura’ thesis
(equivalent to a Bachelor’s and Master’s degree in Argentine National
Universities), among other formats. The works correspond mainly to
presentations given at the National University of La Plata and the University
of Buenos Aires. In terms of awards, the doctoral theses developed at the IAR
received three times the prize for the best theses of the biennium, awarded by
the Asociación Argentina de Astronomía (www.astronomiaargentina.org), and one
of the equivalent (Giambiagi prize) of the Argentine Physics Association. In
2020, a doctoral thesis conducted both at the IAR and the Karlsruhe Institute
of Technology became the first one in the institution to be completed as a
double doctorate (University of Karlsruhe and National University of San
Martín), and the volume was selected by Springer as one of the world’s best
theses of the year and published in book form in Springer’s “Great Theses”
series.

Several articles first authored by IAR researchers have appeared on the front
pages of major journals, others have received honors from the Gravity Research
Foundation, and once even the Top Scientific Contribution Award, from the
American Institute of Physics as one of the most cited papers of the year in
The Astrophysical Journal.

The IAR’s first 30-m dish radiometer - christened Carlos M. Varsavsky, as its
first Director - was the second instrument of its kind to operate
systematically in the southern hemisphere, along with the Parkes radio
telescope in Australia. This led to many discoveries in the southern sky.
Together with the second radio telescope to be built, now called Esteban
Bajaja and completed around 1980, both continuum and line observations could
be carried out, mainly at 1.4 GHz (atomic hydrogen), and 1.6 GHz (CO molecular
transition).

Until the nineties, a great deal of time was devoted to mapping the
distribution and velocity of the HI. Colomb et al. (1980) (CPH) presented
images and profiles covering positions with declination ${\delta\leq
30^{\arcdeg}}$, each $1^{\arcdeg}$. The CPH survey complemented that of Heiles
& Habing (1974), which was carried out using the Hat Creek radio telescope,
with 28-m dish. The sensitivity of the data was between 0.1 and 1 K.

After a change of receivers, among other things, the southern sky was again
surveyed at the 1.4 GHz center frequency, with higher sensitivity: the system
temperature of $\sim$25 K allowed an rms noise below 0.01 K. Bajaja et al.
(2005), and Kalberla et al. (2005) present the HI distribution and kinematic
information, complementing the work of Hartmann & Burton (1997). The continuum
emission with full polarization information has also been surveyed, but with
the latest radiometer, achieving a sensitivity of 15 mK in the Stokes
parameters Q and U (Testori et al., 2008), for the coordinates not covered by
Reich & Reich (1986).

Results on certain, in some respects special, sources were achieved. For
example, with the radio telescopes of the IAR the passage of the tail of the
comet Halley was observed (Bajaja et al., 1987) 4 h a day for 3 months in 1986
to measure OH lines in absorption. The data allowed to derive the abundance of
the molecule and the OH production rate. HI observations led to the discovery
of the supernova remnant Vela Jr (Combi et al., 1999). The extreme variability
of Active Galactic Nuclei of blazar type was reported for the first time,
after being measured with one of the IAR dishes (Romero et al., 1994). In
2010, the first stellar bow shock with evidence of non-thermal emission was
discovered using the Very Large Array, with a follow-up study of the
polarization of the emission (Benaglia et al., 2021). The Long Baseline
Australian Array was used to map the fifth - at that time - colliding wind
region of a massive binary stellar system (HD 93120AaAb) and to estimate
important stellar/wind parameters (Benaglia et al., 2015). Fernández López et
al. (2014) published results of long-term mosaic observations of the Serpens
South molecular cloud with the Combined Array for Research in Millimeter-wave
Astronomy, revealing new features in the gas dynamics.

##  0.3 Main lines of research

The research done at the IAR, let’s say in the last decade or so, covers
several fields, in astrophysics, physics, computer science, mathematics, etc.

There is a lot of activity in the area of extragalactic sources, such as AGN
and their (super)massive black holes, extended to gravitational waves, cosmic
rays, sources studied by means of electromagnetic cascades, neutrino
astrophysics, field theory and relativity.

Compact objects such as neutron stars or black holes, X-ray binary studies,
have a common ground with those of HE (gamma-ray) sources or other candidates
to produce HE radiation (i.e. colliding wind binaries, stellar bow shocks).

Stellar objects are widely studied, including their parent molecular clouds,
star formation scenarios both of massive and low-mass types, interaction with
the interstellar medium (ISM) and the ISM itself, the supernova stage and
remnants, and planetary science.

The studies include the construction of mathematical models, add numerical
simulations, and signal processing algorithms.

In the following, examples of research at the IAR are presented as divided in
three groups: those carried out with the IAR radio telescopes, those where
theoretical developments dominate, and those where observations with
instruments around the world play a key role.

##  0.4 Research with IAR radiometers

After about fifteen years out of use, the radio telescopes at the IAR have
been refurbished (Gancio et al., 2020), especially challenged by a major
project to study transients, meaning by that sources that experiment changes
in radiation with duration of the order of a second of time or much less. The
observations resumed in 2018, and the data collection as a regular operation
started in 2019. The mentioned project is organized in the framework of a
scientific and technological cooperation between IAR and the Rochester
Institute of Technology (USA). The group involved at IAR is called Pulsar
Monitoring in Argentina (PuMA) (see Lousto, 2023, this volume).

The updated radio telescopes have digital back-ends (CASPER ROACH cards with
4x400-MHz of bandwidth). These are capable of recording two polarizations at
1420 MHz. They can be remotely controlled, and have access to an atomic clock,
GPS and GNSS for timing purposes. Sources with declinations less than
$-7{\arcdeg}$ can be tracked for 4 h per day.

The new architecture is optimal for studies involving pulsar timing arrays,
targeted searches for continuous gravitational wave sources, monitoring of
magnetars and glitching pulsars, and short time scale interstellar
scintillation. The timing precision is better than 1$\mu$s (e.g. Zubieta et
al., 2023, this volume).

Another advantage is the geographical location of the IAR: sources that are
invisible from Australia at 12 h daily interval and from South Africa at 5 h
daily interval can be accessed with the IAR radio telescopes. In this way,
alerts of detections between these three places and their instruments allow to
follow a transient phenomenon along its entire occurrence.

##  0.5 Theory, simulations, models

With the creation of the Group of Relativistic Astrophysics and Radio
Astronomy (GARRA, by its Spanish acronym), the scientific work at IAR was
given a strong capacity for the development of theoretical studies. Articles
such as Romero et al. (2007), chosen as the cover of the journal Astronomy and
Astrophysics, are good first examples of such research. The authors focused on
a challenging gamma-ray binary (then usually referred to as a microquasar), LS
I $+$61 303, a Be-star with a compact companion, to elucidate the nature of
the compact object, between pulsar-neutron star or black hole. The light
curve, the spectrum of the observed TeV gamma-ray emission, and the required
energy budget were analyzed. In particular, they modeled the interaction
between the components of the system for both hypotheses (Fig. 2). For this,
they obtained time on the powerful supercomputer HITACHI SR11000 at Hokkaido
University. The results were consistent with the second case.

Detailed models of the spectral energy distribution of massive binary systems
(colliding-wind binaries) and the interaction with the interstellar medium
were also developed; see del Valle & Romero (2012), del Palacio et al. (2018),
del Palacio et al. (2022).

Figure 2: Wind collision interface geometry for the pulsar case in the orbital
plane (top) and in a perpendicular plane (bottom), from Romero et al. (2007).

In Vieyro et al. (2019) the authors studied the case of a core-collapse
supernova inside the relativistic jet of an active galactic nucleus. After the
analyzing the dynamical evolution of the supernova ejecta impacted by the jet
and the observed gamma-ray light curve, they computed the spectral energy
distribution for two different galaxy hosts, a radio galaxy and a blazar. They
concluded that the first scenario appears to be much more common and easier to
detect than the other.

Collaborations on planet formation are presented in San Sebastián et al.
(2019). The article deals with modelling the fragmentation of planetesimal in
the formation of giant planets, taking into account the relative velocities
and compositions of the planetesimals and the accretion produced by their
fragmentation.

An example of progress in cosmology is presented in Pérez & Romero (2022) and
its references, related to a universe with contractions and bounces. The
survival of certain structures along them, in this case black holes, is
studied using a generalized McVittie’s metric.

García et al. (2022) studied physical and geometrical properties of the corona
of the microquasar GRS 1915$+$105\. They applied a variable comptonization
model vKompth, – developed by IAR’s Ph.D. student C. Bellavita (Bellavita et
al., 2022) –, supported by archival X-ray observations, and found consistent
trends in the evolution of the corona size, temperature, and feedback (see
also Méndez et al., 2022).

As can be seen, Ph.D. theses are of particular interest at the IAR. For
example, L. Combi studied binary black holes - in principle of equal mass,
spinning, and approaching merger. Three-dimensional general relativistic
magnetohydrodynamical simulations were performed on a system geometry that
included a circumbinary disk and mini-disks around each black hole. These
allowed to study the gas dynamics and system evolution, the morphology and
variability of the electromagnetic flux densities, and to analyze the
accretion (see Fig. 3). The results on realistic synthetic light curves and
spectra are very valuable for future observations with instruments like the
Laser Interferometer Space Antenna (LISA) (Combi et al., 2022).

Figure 3: Surface density snapshot of a binary black hole system for two
epochs (Combi et al., 2022).

In the same subject and with a similar treatment, E. M. Gutiérrez Ph.D.
project consisted more in the study of the radiation from supermassive binary
black holes (Gutiérrez et al., 2022). Simulations including blackbody
radiation from an optically thick accretion disk, and hard X-rays from
optically thin corona allowed to obtain spectra, images and light curves (Fig.
4).

The Ph.D. Thesis by F. Fogantini was focused on the phenomenology of accreting
high-mass X-ray binaries (HMXBs). In this Thesis, it was demonstrated how
geometrical or line-of-sight effects have a strong impact on the observed
spectral and variability properties of archetypical HMXBs sources like SS 433
(Fogantini et al., 2023).

Figure 4: Stokes I instantaneous radiation distribution of a binary
supermassive black hole approaching periastron (Gutiérrez et al., 2022).

##  0.6 Observing with interferometric arrays

The regular acquisition of observing time with instruments in foreign soil
started around 2000, and is growing steadily mainly pursued by members of the
fringe research group (Formation in Radio Interferometry - arGEntina).

In recent years, the field of high-mass star formation has been shaken by the
discovery of explosive outflows, in addition to the well-known bipolar
outflows; (see Zapata et al., 2009, and references therein). Only a few such
regions have been described. One is G5.98–0.39, an ultra-compact HII region
with formation of massive stars. Zapata et al. (2020) and Fernández López et
al. (2021) spotted it with the Atacama Large Millimeter/submillimeter Array
(ALMA), and found dozens of CO filaments, and expanding warmer SiO gas at the
origin. The energy released was inferred to be $\sim 10^{46}-10^{49}$ erg.
There is a north-south filamentary structure, a compact HII region, and a
possibly expanding dusty belt, which harbours an O5V star. Polarized emission
could be measured in the filaments, $\sim$4.4%, coming from magnetically
aligned dust grains. As Fig. 5 shows, the magnetic field lines in the central
belt of dust are radially aligned.

Figure 5: Electrical vector position angle rotated 90$\arcdeg$, superpimposed
on Stokes I continuum emission towards G5.89–0.39 (full details in Fernández
López et al., 2021).

In Guzmán Ccolque et al. (2022), part of first author’s Ph.D. thesis is
presented, on the object IRAS 16076–5134, a high-mass star-forming region
studied with ALMA band-7 CO archive data. Fourteen cores were detected. The
imaged morphology and kinematics suggest a dispersal, explosive outflow, with
filament-like CO ejections from a central position (see Fig. 6), quasi
isotropic; several filaments show a linear velocity gradient.

Figure 6: Red-shifted and blue-shifted condensations in IRAS 16076–513 (Guzmán
Ccolque et al., 2022).

Another example of recent Ph.D. project was the one focused on galaxy groups
and certain types of galaxies (dwarf-, low surface brightness-, super thin-,
local galaxies). The groups are a very common environment, easy to study
because of the low relative velocities involved. The sources are studied by
means of HI-line data. Questions such as which is the role of the environment
in galaxy evolution, or how galaxy mergers, ram-pressure stripping,
gravitational interactions and intragroup medium affect star formation and
morphology are investigated. One of the contributed papers (Saponara et al.,
2021), aimed at the superthin galaxy Fourcade-Figueroa, consisted in modeling
the HI distribution, deriving the rotation curve, to finally obtain, also
through modeling, characteristics of the dark matter halo (see Fig. 7).

Figure 7: HI rotation curve model of the Fourcade-Figueroa galaxy. The orange
line shows the rotation curve due to the stellar disk, the blue line shows the
contribution due to the gas disk, the green line shows that due to the dark
matter halo and the red colour shows the best-fitting model rotation curve.
See details in Saponara et al. (2021).

Since radio interferometric observations provide a very high angular
resolution, the products they deliver can be complemented by others at shorter
wavelengths. For example, processes that take place in the interstellar medium
are studied by combining radio and infrared data. This is the case of the
bright HII region RCW 49 and its very rich ionizing cluster Westerlund 2.
Figure 8 shows the distribution of gas, dust, and stars along the field, at
arcsec angular scales. The gas is probed by data from the Australia Telescope
Compact Array, represented by a 40-pointing mosaic observation; the dust and
stars by Spitzer images (Benaglia et al., 2013). There are HE sources in the
field, imaged by means of H.E.S.S. and Fermi LAT data. The work discusses
possible radio counterparts.

Figure 8: RCW 49 field as seen in the radio continuum (9 GHz, in red) and
Spitzer-GLIMPSE band 1 (3.6 $\mu$m) in blue and band 4 (8 $\mu$) in green
(Benaglia et al., 2013).

Nearly 200 h of observing time were devoted to the survey of the Cygnus OB2
association and its surroundings with the Giant Metrewave Radio Telescope (see
Benaglia et al., 2020, and related papers). The observations were made at two
MHz bands. About 4000 sources were detected and catalogued (1000 in the two
bands), some of them for the first time. The database allowed further studies
of the massive early-type stars, protoplanetary disks, young stellar objects,
double-lobed objects, and counterparts to unidentified high-energy sources.

Observations with instruments that record emission outside the radio window
are also carried out for projects leaded by IAR researchers, for instance,
using XMM-Newton, Chandra, NuSTAR. An example is described in the work by
Saavedra et al. (2022) on the binary source OAO 1657–415 – an accreting X-ray
pulsar with a high-mass companion –. The authors identified pulsations in
NuSTAR data and explained their origin and characteristics, estimated the
value of the dipolar magnetic field at the pulsar surface and a obtained a
lower limit on the distance of the source.

##  0.7 Scientific research with MIA

The Multipurpose Interferometric Array, in its full configuration, is expected
to be formed at least by 32 elements/antennas with 5-m diameter dishes; an
expansion to 64 antennas is also planned (see full description in Gancio et
al., 2023, this volume). The largest 55 km baseline will provide an angular
resolution close to 1 arcsec in L-band. The final coverage is expected to be
between 100 MHz and 2.3 GHz.

With the above parameters, MIA observations can contribute to advancing
studies on four major topics: transient sources and timing measurements,
sources of non-thermal radiation, neutral hydrogen, from rest to redshifted
velocities, and astrophysical plasmas.

The high-precision timing settings will allow the detection of transient
counterparts of gamma-ray bursts, the study of fast radio bursts, pulsars and
gravitational waves, and the observation of flares from magnetars.

The short frequencies at which the MIA receivers will operate are ideal to
probe sources where non-thermal radiation is important. This, combined with
MIA’s high temporal resolution capabilities, will make the instrument optimal
for studying the counterparts of unidentified gamma-ray sources, performing
multifrequency studies of AGN variability, spectro-temporal studies of X-ray
and gamma-ray binaries, studies of the morphology and spectral distribution of
supernova remnants, the mapping of continuum non-thermal extended sources, and
a long list of other objects.

MIA’s high bandwidth, from GHz to a few MHz is well suited to observe the HI
line of atoms at rest, but also at large distances, i.e. high redshift
sources, with a resolution down to 1 arcsec. This will allow studies of nearby
and/or extended galaxies, of the interstellar medium with high angular
resolution, and, under certain conditions, of HI at cosmological distances to
characterize the Epoch of Reionization.

Emission from astrophysical plasmas is relatively strong even at short
centimeter wavelengths. MIA will be very useful for the physical and
kinematical characterization of HII regions, for the study of the OH maser
variability in star-forming regions and evolved massive stars, and star
forming regions in general.

## References

  * Bajaja et al. (1987) Bajaja, E., Morras,R., Poeppel, W. G. L. et al, 1987, ApJ 322, 549
  * Bajaja et al. (2005) Bajaja, E., Arnal, E. M., Larrarte, J. J. et al. 2005, A&A, 440, 767
  * Bellavita et al. (2022) Bellavita, C., García, F., Méndez, M. & Karpouzas, K. 2022, MNRAS, 515, 2099
  * Benaglia et al. (2013) Benaglia, P., Koribalski, B., Peri, C. S., et al. 2013, A&A, 559, 31
  * Benaglia et al. (2015) Benaglia, P., Marcote, B., Moldón, J., et al. 2015, A&A, 579, 99
  * Benaglia et al. (2020) Benaglia, P., Ishwara-Chandra, C. H., Intema, H., et al. 2020, A&A, 642, 136
  * Benaglia et al. (2021) Benaglia, P., del Palacio, S., Hales, C. & Colazo, M. E. 2021 MNRAS, 503, 2514
  * Colomb et al. (1980) Colomb, R. F., Poeppel, W. G. L. & Heiles C. 1980, A&ASS, 40, 47
  * Combi et al. (1999) Combi, J., Romero, G. E. & Benaglia, P. 1999, ApJ, 519, L177
  * Combi et al. (2022) Combi, L., López Armengol, F. G., Campanelli, M. et al. 2022, ApJ, 928, 187
  * del Palacio et al. (2018) del Palacio, S., Bosch-Ramon, V., Müller, A. L. & Romero, G. E. 2018, A&A, 617, 13
  * del Palacio et al. (2022) del Palacio, S., Benaglia, P., De Becker, M., et al. 2022, PASA, 39, 4
  * del Valle & Romero (2012) del Valle, M. V., & Romero, G. E. 2012, A&A, 563, 96
  * Fernández López et al. (2014) Fernández López, M., Arce, H. G., Looney, L., et al. 2014 ApJ, 790, L19
  * Fernández López et al. (2021) Fernández López, M., Sanhueza, P., Zapata, L. A., et al. 2021 ApJ, 913, 29
  * Fogantini et al. (2023) Fogantini, F., García, F., Combi, J., et al. 2023, A&A, 669
  * Gancio et al. (2020) Gancio, G., Lousto, C., Combi, L. et al. 2020, A&A, 633, 84
  * Gancio et al. (2023) Gancio, G., Romero, G. E., Benaglia, P., et al. 2023, Rev. Mex. Astron. Astrof. SC, 56, in press
  * García et al. (2022) García, F., Karpouzas, K., Méndez, M., et al. 2022, MNRAS, 513, 4196
  * Gutiérrez et al. (2022) Gutiérrez, E. M., Combi, L., Noble, S. C., et al. 2022, ApJ, 928, 137
  * Guzmán Ccolque et al. (2022) Guzmán Ccolque, E., Fernández López, M., Zapata, L. A., et al. 2022, ApJ, 937, 51
  * Hartmann & Burton (1997) Hartmann, D. & Burton, W. B. 1997, Atlas of Galactic Neutral Hydrogen, Cambridge, UK: Cambridge University Press
  * Heiles & Habing (1974) Heiles, C. & Habing, H. J. 1974, A&ASS, 14, 1
  * Kalberla et al. (2005) Kalberla, P. M. W., Burton, W. B., Hartmann, D. et al. 2005, A&A, 440, 775
  * Lousto (2023) Lousto, C. O. 2023, Rev. Mex. Astron. Astrof. SC, 56, in press
  * Méndez et al. (2022) Méndez, M., Karpouzas, K., García, F. et al 2022, Nature Astronomy 6, 577
  * Mészáros (1968) Mészáros, P. 1968, Ap&SS, 2, 510
  * Reich & Reich (1986) Reich, P. & Reich W. 1986, A&AS, 63, 205
  * Romero et al. (1994) Romero, G. E., Combi, J. & Colomb, F. R. 1994, A&A, 288, 731
  * Romero et al. (2007) Romero, G. E., Okazaki, A. T., Orellana, M. & Owocki, S. P. 2007, A&A, 474, 15
  * Romero (2023) Romero, G. E. 2023, Rev. Mex. Astron. Astrof. SC, 56, in press
  * Pérez & Romero (2022) Pérez, D. & Romero, G. E. 2022, Physical Review D, 105, 104047
  * Saavedra et al. (2022) Saavedra, E., Fogantini, F., Combi, J. et al. 2022 , A&A, 659, 48
  * San Sebastián et al. (2019) San Sebastián, I. L., Guilera, O. & Parisi, M. G. 2019, A&A, 625, 138
  * Saponara et al. (2021) Saponara, J., Kamphuis, P. Koribalski, B. S. & Benaglia, P. 2021, A&A, 652, 108
  * Testori et al. (2008) Testori, J. C., Reich, P. & Reich W. 2008, A&A, 484, 733
  * Varsavsky (1966) Varsavsky, C. M. 1966, Space Science Review, 5, 419
  * Vieyro et al. (2019) Vieyro, F. L., Bosch-Ramon, V. & Torres-Albà, N. 2019, A&A, 622, 175
  * Zapata et al. (2009) Zapata, L. A., Schmid-Burgk, J., Ho, P. T. P., et al. 2009 ApJ, 704, L45
  * Zapata et al. (2020) Zapata, L. A., Ho, P. T. P., Fernández López, M., et al. 2020 ApJ, 902, L47
  * Zubieta et al. (2023) Zubieta, E., del Palacio, S., García, F., et al. 2023, Rev. Mex. Astron. Astrof. SC, 56, in press


# Negligible effect of brain MRI data preprocessing for tumor segmentation.

Ekaterina Kondrateva  Polina Druzhinina  Alexandra Dalechina  Svetlana
Zolotova  Andrey Golanov  Boris Shirokikh  Mikhail Belyaev  Anvar Kurmukov
Artificial Intelligence Research Institute (AIRI), Moscow, Russia  Skolkovo
Institute of Science and Technology, Moscow, Russia  National Medical Research
Center for Neurosurgery, Moscow Gamma Knife Center, Moscow, Russia

###### Abstract

Magnetic resonance imaging (MRI) data is heterogeneous due to differences in
device manufacturers, scanning protocols, and inter-subject variability. A
conventional way to mitigate MR image heterogeneity is to apply preprocessing
transformations such as anatomy alignment, voxel resampling, signal intensity
equalization, image denoising, and localization of regions of interest.
Although a preprocessing pipeline standardizes image appearance, its influence
on the quality of image segmentation and on other downstream tasks in deep
neural networks has never been rigorously studied.

Experiments on three publicly available datasets evaluate the effect of
different preprocessing steps in intra- and inter-dataset training scenarios.
Results demonstrate that most popular standardization steps add no value to
network performance; moreover, preprocessing can hamper performance. Our
results suggest that image intensity normalization approaches do not
contribute to model accuracy because of the reduction of signal variance with
image standardization. Additionally, the contribution of skull-stripping in
data preprocessing is almost negligible if measured in terms of estimated
tumor volume.

The only essential transformation for accurate deep learning analysis is the
unification of voxel spacing across the dataset. In contrast, inter-subjects
anatomy alignment in the form of non-rigid atlas registration is not necessary
and intensity equalization steps (denoising, bias-field correction and
histogram matching) do not improve performance.

The study code is accessible online111https://github.com/MedImAIR/brain-mri-
processing-pipeline.

###### keywords:

Brain MRI , segmentation , preprocessing , nnU-net , UNETR , SAM

##  1 Introduction

In recent years, modern deep neural networks (DNN) have steadily improved the
quality of automatic segmentation pipelines in medical imaging. Specifically,
for the task of brain tumor segmentation, the performance of DNNs has achieved
human-level efficiency Chang et al. [2019]. This advancement can be explained
by the improvement of DNN architectures and the growth of the training
datasets. For example, the size of the BraTS competition Bakas et al. [2018]
training dataset increased from $100$ subjects in 2013 to $2000$ subjects in
2021. Simultaneously, top-performing algorithms have progressed from random
forest and gradient boosting trees on radiomics features: first to fully
convolutional networks, then to u-shaped UNet and UNet-like networks, and
finally to Vision transformers.

In contrast, the preprocessing steps used to prepare data for analysis seem to
have undergone considerably fewer changes. For instance, the set of
preprocessing steps for brain MRI images has remained relatively stable and
has been reproduced across the majority of papers on the topic from the early
2010s until now. Here we challenge the conventional pipelines for MRI image
processing and question their necessity for accurate prediction with regard to
new advanced deep learning machinery.

The traditional brain MRI preparation steps could be divided into four
distinct categories:

  * 1.

The first category is subject-wise image alignment, typically in the form of
rigid registration of one MRI sequence to another, (e.g. T2-FLAIR onto T1 with
contrast enhancement). This step is mandatory if one uses multiple MR
modalities to predict a single segmentation map to ensure correct alignment
between ground truth annotation and corresponding image’s voxels.

  * 2.

The second category is voxels resampling to some standard. The most common
methods are voxel resampling to homogeneous spacing (often $1\times 1\times
1\text{ mm}^{3}$) and non-rigid registration to some anatomical atlas.

  * 3.

The third category includes steps that affect voxels’ intensity distribution,
such as bias-field correction (such as N4 correction Tustison et al. [2010]),
intensity normalization (typically in a form of image-wise z-scoring), image
denoising methods (e.g. SUSAN Smith and Brady [1997]), and histogram
equalization Nyúl et al. [2000].

  * 4.

Finally, the last step that is preserved in almost all the papers is skull
stripping as a method to localize regions of interest (the brain tissue),
implement feature selection to ease localization, and reduce the amount of
False Positives Chang et al. [2019].

While the motivation behind applying these steps is clearly to standardize
image appearance and remove different sources of domain shift Kondrateva et
al. [2021], these steps are computationally costly and their utility for deep-
learning segmentation lacks investigation. Specifically, it is widely known
that increasing variability of the data by data augmentation (image resizing,
non linear intensity transformations, applying noise, etc.) leads to improved
DNN performance Wightman et al. [2021]. However, data preprocessing works
quite in the opposite way by reducing data variance.

In this study we analyze the most popular preprocessing steps for brain MRIs
and measure their influence on deep-learning based tumor segmentation tasks.
We analyze different preprocessing strategies and recommend the minimal
pipeline required for accurate segmentation with the benefits of lower
computational costs and improved reproducibility.

##  2 Related works

Image preprocessing is a de-facto standard first step in almost all deep
learning pipelines for medical image analysis Nixon and Aguado [2019]. In this
domain, data preparation is convenient due to two major causes:

  * 1.

diversity in scanning protocols and therefore diverse spatial resolutions and
image intensity profiles Kurmukov et al. [2021],

  * 2.

large image sizes and small sample sizes, thus leading to high-dimension
learning compounding negative effects on model generalisability Berisha et al.
[2021].

For example, a typical multi-institutional brain MRI dataset consists of
images with varying resolutions and acquisition parameters (depending on
scanning protocol). Therefore, a majority of studies utilize data
preprocessing pipelines. We select several recent publications on brain MRI
segmentation to identify the most common preprocessing steps (see Table 1).

Table 1: Common preprocessing steps for multi-modal brain MRI image analysis.
Checkmarks represent the step mentioned in the study and x-marks are placed if
the step is missing or unclear.

Preprocessing step | Resample to image size | Resample to spacing | Atlas registration | Bias-field correction | Denoising | Histogram matching | Skull stripping
---|---|---|---|---|---|---|---
Győrfi et al. [2021] | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | ✓
Ranjbarzadeh et al. [2021] | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓
Pei et al. [2020] | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓
Menze et al. [2021] | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✓
Ermiş et al. [2020] | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓
Eijgelaar et al. [2020] | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✓
Rathore et al. [2017] | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓
Wang et al. [2019] | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗
Bakas et al. [2018] | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✓
Kickingereder et al. [2019] | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓
Bakas et al. [2022] | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✓

The overall brain MRI data preprocessing pipeline can be divided into four
groups of methods.

###  2.1 Intra-subject alignment (rigid registration)

During this step different MR sequences from a single patient are reoriented
in a similar way and rigidly registered. This step is applied in all studies
that analyze multi-sequence MRI Győrfi et al. [2021], Ranjbarzadeh et al.
[2021], Pei et al. [2020], Menze et al. [2021], Ermiş et al. [2020], Eijgelaar
et al. [2020].

###  2.2 Inter-subject alignment

This step standardizes the size of images across the dataset. Most of the
observed papers use voxel resampling to an isotropic voxel (e.g. $1\times
1\times 1\text{ mm}^{3}$), or to the same image resolution (in voxels, e.g.
$256\times 256\times 256$), or both, by means of non-rigid atlas registration
Győrfi et al. [2021], Ranjbarzadeh et al. [2021], Pei et al. [2020], Menze et
al. [2021] and Ermiş et al. [2020], Eijgelaar et al. [2020], Rathore et al.
[2017], Bakas et al. [2018]. The only exceptions are two studies that analyze
the data acquired with a unified scanning protocol (isotropic voxel)
Kickingereder et al. [2019], Wang et al. [2019].

###  2.3 Non-linear intensity correction and enhancement

Multi-institutional studies typically use some intensity or noise correction
approaches, with the most popular being histogram matching Győrfi et al.
[2021], Pei et al. [2020], Rathore et al. [2017], bias-field correction Győrfi
et al. [2021], Pei et al. [2020], Eijgelaar et al. [2020], Rathore et al.
[2017], and denoising Pei et al. [2020], Rathore et al. [2017].

Histogram equalization (harmonization or matching) methods standardize images
by aligning their intensity distributions Nyúl et al. [2000]. Denoising
algorithms filter the image whilst preserving the underlying structure Smith
and Brady [1997]. Finally, bias-field correction methods mitigate the effects
of magnetic field variation Tustison et al. [2010], see Figure 2.

Most of the observed studies apply z-scoring Ranjbarzadeh et al. [2021], Pei
et al. [2020], Menze et al. [2021], Ermiş et al. [2020], Rathore et al.
[2017], Wang et al. [2019], Kickingereder et al. [2019] prior to analysis.
This is a common data normalization approach for all computer vision
algorithms Patro and Sahu [2015] and is not specific to medical imaging.

###  2.4 Skull stripping

Finally, all but one of the observed papers Wang et al. [2019] use skull
stripping, arguing that non-brain tissue is a significant source of error for
downstream tumor segmentation Chang et al. [2019]. Authors point out that
skull stripping reduces the number of False Positives and improves
segmentation quality.

###  2.5 Novelty of the proposed study

In general, researchers experimenting on single-institutional data or data
collected under unified acquisition protocol tend to use fewer preprocessing
steps. On the contrary, the analysis of heterogeneous multi-center data
typically includes a data preparation pipeline. For example, the data
preparation pipeline for the most-known benchmark dataset for multi-
institutional brain MRI
segmentation222https://cbica.github.io/CaPTk/preprocessing_brats.html includes
image reorientation, altas registration Rohlfing et al. [2010], bias-field
correction, and skull stripping Thakur et al. [2020].

These MRI preprocessing methods have been exploited by scientists for several
decades. Yet to date there is no consensus regarding which of the methods
should be applied for deep-learning-based analysis. In the present work we
demonstrate that the effect of most standardization steps is negligible even
for relatively small data collection.

To the best of our knowledge, there have been two similar attempts to analyze
the influence of preprocessing steps for medical imaging tasks. Authors of
Moradmand et al. [2020] test how preprocessing influences radiomics features
calculation (for brain MRI), and de Raad et al. [2021] investigate the
influence of data augmentation for three medical imaging tasks (brain and knee
MRI and liver CT segmentation).

###  2.6 Contributions

In our study we focus on preprocessing steps used primarily for brain MRI. Our
contribution is three-fold. First, we numerically estimate the effect of 7
common preprocessing steps: resizing images to equal size,inter-subject atlas
registration,resampling voxels to equal size, bias-field correction, image
denoising, histogram matching, skull stripping; with state-of-the-art deep
learning architectures that are both convolutional and attention-based. This
effect is assessed in two scenarios: training from scratch and fine-tuning
from a larger dataset.

Second, we propose an explanation for the observed low (or even negative)
influence of image intensity correction techniques on model accuracy. For that
we compare segmented tissue with the rest of the brain before and after
preprocessing.

Third, we suggest a minimal preprocessing pipeline for multi-sequence, multi-
protocol brain MRI segmentation studies, consisting of two steps333We assume
intra-subject sequence alignment and z-scoring to be mandatory first and last
preprocessing steps, respectively.:

  1. 1.

voxel resampling (to roughly align images across the dataset),

  2. 2.

skull stripping.

The last step results in a marginal but statistically significant improvement
in terms of segmentation quality, especially for smaller ($<100$ subjects)
datasets, though it could be omitted for larger datasets with little drop in
accuracy.

##  3 Methods

Figure 1: Study experimental pipeline. Steps in arrows are mandatory, steps in
blocks are optional. Steps 4.a-d are performed after step 4. The minimal
preprocessing pipeline consists of steps 1 and 5. Thus, we have 8 different
preprocessing pipelines in total.

In the current study we test how different preprocessing pipelines affect the
quality of a downstream deep learning segmentation model. We compare eight
preprocessing pipelines using two segmentation models (convolutional and
attention based), Figure 1. Additionally, we test if the optimal preprocessing
pipeline changes in a transfer learning scenario (vs training from scratch) by
fine-tuning a model pretrained on a different dataset (with the same
preprocessing).

In the following sections we provide information on experimental design and a
detailed description of the preprocessing steps, neural network setups,
quality metrics, and datasets.

###  3.1 Experimental design

We compare preprocessing pipelines (Figure 1) in two scenarios:

  1. 1.

Training a segmentation model from scratch.

  2. 2.

Fine-tuning a segmentation model from a related or a similar task.

Thus, we check if different preprocessing pipelines result in better in-domain
training, and an improved fine-tuning/knowledge transfer in a cross-dataset
regime. In the second scenario we only consider whole model fine-tuning as it
is the most common approach to knowledge transfer Wang et al. [2021], Dai et
al. [2019].

###  3.2 Preprocessing steps

In all experiemnts we use multi-sequence datasets consisting of 4 MRI
sequences: T1 weighted (T1), T1 weighted with contrast enhancement (CT1), T2
weighted (T2), and T2-FLAIR (FLAIR). We start each preprocessing pipeline with
rigid registration (rotation, shift and linear interpolation of voxel size) of
every MR sequence to CT1 or FLAIR (depending on the dataset) independently for
each subject. This step aligns different MR sequences (from the same subject)
with each other and the existing tumour annotation. After that, we proceed
with preprocessing steps as described in Figure 1. We end each preprocessing
pipeline with image-wise Z-scoring:

$X_{s}=\frac{X-\text{mean}(X)}{\text{std}(X)}.$

####  3.2.1 Inter-subject alignment

After the intra-subject rigid registration, we apply one of three methods to
align images across the dataset (inter-subject):

  * 1.

resizing images to the same size $240\times 240\times 155$ voxels;

  * 2.

resampling to an isotropic voxels’ size $1\times 1\times 1\ \text{mm}^{3}$;

  * 3.

a non-rigid atlas registration to SRI24 atlas Rohlfing et al. [2010].

The latter results in both of the same images of size $240\times 240\times
155$, and an isotropic voxel $1\times 1\times 1\ \text{mm}^{3}$. All
transformations were performed by ANTs utilities Avants et al. [2009].

####  3.2.2 Image enhancement

The next step is applying one of three algorithms of image intensity
correction:

  * 1.

bias-field correction,

  * 2.

denoising,

  * 3.

histogram standardization.

Figure 2: Visualization of three methods of image intensity correction used in
the ablations study. The top raw (b-d) showing the image segment with tumor
after preprocessing of (a) an example CT1 MR image from GBM data after
resampling to [1,1,1] voxel size; (b) N4 correction of the image; (c) SUSAN
denoising of the image; (d) Histogram standartization to GBM sample; The
bottom row (e-g) showing image residuals as voxelwise difference between the
original resampled image and the preprocessed one.

Bias-field is a smooth, non-heterogeneous and low-frequency signal which
corrupts MRI images. It is generally believed that algorithms which
postprocess MRI images such as segmentation or classification do not produce
satisfactory results without correcting for the bias field signal Juntu et al.
[2005]. To test the impact of such a correction we use a popular N4 algorithm
Tustison et al. [2010], implemented in Simple-ITK Beare et al. [2018] as a
containerized solution from the CaPTk toolbox.

SUSAN is a method of nonlinear image smoothing and denoising Smith and Brady
[1997] that, while old, is still used nowadays for noise reduction. We use an
FSL implementation of SUSAN. These two steps were applied for each MR image
individually.

Finally, to homogenize grayvalue distribution across images in the dataset, we
apply a histogram equalization Nyúl et al. [2000], as implemented in TorchIO
library Pérez-García et al. [2021]. To estimate the parameters of histogram
matching we use training folds, and the same transformation is applied to the
data from test folds. Histogram matching is applied sequence-wise, meaning
that we equalize voxels’ histograms for each MR sequence separately Li et al.
[2021].

These steps are applied after resampling images to an isotropic voxel (step 4,
Figure 1). As we show in the results, there is no statistical differences
between steps 2-4. We thus use voxel resampling as the most intuitive and
simple one.

####  3.2.3 Skull stripping

Skull stripping is a method to localize the region of interest and filter out
potential False Positives. To test how skull stripping affects model
performance, we use HD-BET Isensee et al. [2019], which is the most accurate
publicly available brain extraction tool. We extract brain mask on CT1 and
apply it to all other MR modalities after image alignment (this results in no
loss of tumor mask).

###  3.3 Models architecture and training

We test the effect of preprocessing on two deep-learning segmentation
architectures: 3D nn-UNet Isensee et al. [2021] and vision transformer-based
3D UNETR Hatamizadeh et al. [2022].

In all experiments we trained networks for 300 epochs or until convergence
(using 20 epochs patience as a stopping criterion) without data augmentations,
as data augmentation can interfere with measurements of preprocessing effects.
All experiments were performed on a 3-fold cross-validation (subject-wise)
with the average time for one experiment being 20 hours on 32 GB Tesla V100.

###  3.4 Perfomance metrics

The accuracy of brain MRI semantic segmentation is conventionally evaluated
using the Dice Similarity coefficient (referred to as Dice) and the 95th
percentile of the Hausdorff distance between the predicted labels and the
provided ground truth, these metrics are used for SOTA models evaluation on
benchmark datasets. Bakas et al. [2022].

In the current work we focus on overlap-based metric Dice coefficient as the
standard for assessing the quality of segmentation and provide clear and
meaningful interpretations:

$\text{Dice}\ (A,B)=\frac{2\cdot|A\cap B|}{|A|+|B|},$ (1)

where $A$ and $B$ are 3D binary arrays. In addition, to measure the error in
terms of tumor volume estimation, which is the simplest clinically relevant
feature Chang et al. [2019], we use Mean Absolute Error (MAE):

$\text{MAE}(V_{\text{true}},V_{\text{estimated}})=\frac{1}{n}\sum_{i=1}^{n}|V^{i}_{\text{true}}-V^{i}_{\text{estimated}}|.$
(2)

Mean and standard deviation for Dice score and absolute errors were obtained
out of fold. We also asses absolute volumetric error of estimation (for
evaluation of result significance from the clinical perspective).

To measure the differences in voxel intensity between healthy and tumored
tissues for different image enhancement experiments, we use Kullback-Leibler
divergence (KL) between corresponding intensity histograms:

$\text{KL}(H_{\text{healthy}},H_{\text{tumor}})=\sum_{i=1}^{\text{\\#bins}}H^{i}_{\text{healthy}}\cdot\log{\frac{H^{i}_{\text{healthy}}}{H^{i}_{\text{tumor}}}}$
(3)

We report KL values for histograms with fixed bin size (100 bins), and we have
tested the Freedman-Diaconis heuristic Freedman and Diaconis [1981] (500-700
bins depending on subject) and Sturge’s rule Sturges [1926] (20-60 bins). All
approaches result in different absolute KL values but preserve a relative
trend, without affecting the experiment’s conclusion.

To assess the results’ significance and compare experiments with different
data preprocessing, we test the Null hypothesis of the Means equality in two
samples with Welch’s t-test Bonferroni corrected for multiple comparisons. If
the $p_{\text{value}}$ for the test has not exceeded $0.05$, we declare that
there is no evidence in the data for rejecting the Null hypothesis. For
simplicity, from now on we will refer to it as statistically significant.

###  3.5 Data description

We explore the effect of data preprocessing for tumor segmentation on multi-
modal brain MRI data (a task similar to an extensively studied BraTS Menze et
al. [2014]). We selected three largest publicly available multi-domain with
original DICOM data from TCIA Clark et al. [2013a]: Glioblastoma Multiform
(GBM) and Lower Grade Glioma (LGG) Beers et al. [2018b], Bakas et al. [2017b],
Beers et al. [2018a], and an in-house dataset of 180 patients with
glioblastoma (BGPD)Zolotova et al. [2023].

All three datasets contain 4 MR sequences (T1, T2, CT1, FLAIR) available in a
raw DICOM format, without any prior preprocessing. All three datasets comprise
of multi-protocol studies (Table 3). A summary of all datasets is presented in
Table 2. A more detailed description is given below.

Table 2:  Description of open-source multi-institutional datasets on brain
tumor segmentation used in the study. All datasets have multimodal MR set per
patient, including: CT1, T1, T2 and FLAIR modalities.

Dataset | Dataset name | Size | Diagnosis | Preprocessing | Segmentation classes | Annotation source
---|---|---|---|---|---|---
Beers et al. [2018a] | LGG | 38 | pre-operative low-grade glioma | ✗ | WT, ET | semi-automatic
Beers et al. [2018a] | GBM | 102 | pre-operative glioblastoma | ✗ | WT, ET, TC | semi-automatic
Zolotova et al. [2023] | BGPD | 180 | pre-radiotherapy glioblastoma | ✗ | GTV | manual

####  3.5.1 Glioblastoma Multiforme (GBM) Dataset

GBM is an open-source data collection Bakas et al. [2017a, c], Clark et al.
[2013b], Beers et al. [2018b] which originally included 262 patients with
preprocessed images. We selected 102 patients with accessible segmentation
labels for data without
preprocessing444https://wiki.cancerimagingarchive.net/pages/viewpage.action?pageId=41517733.
The annotations are semi-automatic, obtained using a GLISTRBoost Bakas et al.
[2015] tool with manual correction, and include segmentation for the whole
tumor (WT), tumor core (TC) and enhancing tumor (ET). The data subset is
comprised of data from $3$ manufacturers and at least $28$ different study
protocols.

####  3.5.2 Low Grade Glioma (LGG) Dataset

LGG is an open-source data collection Pedano et al. [2016] that originally
included 199 patients. We selected a subset of 38 unique patients with
accessible segmentation labels for data without preprocessing. The annotations
are semi-automatic, obtained using a GLISTRBoost Bakas et al. [2015] tool with
manual correction, and include segmentation for WT and ET ($38$ out of the
selected $39$ images do not contain TC segmentation map and are thus excluded
from the experiments for LGG). The ata subset is comprised of data from $3$
manufacturers and at least $11$ different study protocols.

####  3.5.3 Burdenko Glioblastomas Progression Dataset (BGPD)

BGPD Zolotova et al. [2023] is an MRI collection of patients who underwent
radiotherapy in Burdenko Neurosurgery Institute for Radiotherapy in Moscow,
Russia. It contains the data of 180 unique patients. Segmentation maps were
imported from a radiotherapy planning system and correspond to Gross Tumor
Volume (GTV). The collection is highly heterogeneous and comprised of data
from $4$ manufacturers and at least $51$ different study protocols.

Table 3:  Variability of study protocols for T1 and T2 MRI sequences for GBM,
BGPD and LGG data collections. All datasets contain images from 3 major MRI
scanner manufacturers: GE, Siemens, and Toshiba.

Aquisition parameter | GBM | BGPD | LGG
---|---|---|---
T1 | T2 | T1 | T2 | T1 | T2
Echo Time, ms | min | $2.1$ | $20$ | $1.8$ | $18.4$ | $3.7$ | $16.1$
max | $19$ | $120$ | $23$ | $120$ | $15$ | $120$
$\\#\text{unique}$ | $28$ | $38$ | $51$ | $67$ | $11$ | $17$
Repetition Time, ms | min | $5$ | $2020$ | $7.4$ | $567$ | $8$ | $897$
max | $3379.6$ | $6650$ | $3119.2$ | $8200$ | $3232$ | $10000$
$\\#\text{unique}$ | $56$ | $36$ | $50$ | $57$ | $38$ | $18$
Voxel volume, $\text{mm}^{3}$ | min | $0.5$ | $0.2$ | $0.1$ | $0.1$ | $0.6$ | $0.5$
max | $5.2$ | $5.2$ | $5.3$ | $4.8$ | $13.2$ | $35.2$
$\\#\text{unique}$ | $32$ | $32$ | $53$ | $60$ | $17$ | $19$

##  4 Results

Table 4:  nnU-net and UNETR segmentation performance for the three datasets:
GBM and LGG (WT label), BGPD (GTV label). Segmentation accuracy presented in
Dice scores from three fold cross-validation as $\text{Mean}\ (\text{STD})$
multiplied by 100, the higher — the better. Models trained for 300 epochs.
Arrows denote the statistically significant difference
($p_{\text{value}}<0.05$) compared to step 4. Resampling to spacing,
$\uparrow$ (increase), $\downarrow$ (decrease).

| nnU-net | UNETR
---|---|---
Data preprocessing | GBM | BGPD | LGG | GBM | BGPD | LGG
1\. Inter-modality registration | $44\ (28)\downarrow$ | $36\ (29)\downarrow$ | $67\ (27)$ | $39\ (26)\downarrow$ | $35\ (30)\downarrow$ | $66\ (23)$
2\. Resampling to image size | $85\ (11)$ | $73\ (19)$ | $72\ (24)$ | $82\ (12)$ | $67\ (20)$ | $66\ (26)$
3\. Atlas registration | $85\ (11)$ | $75\ (16)$ | $71\ (25)$ | $82\ (13)$ | $68\ (21)$ | $67\ (25)$
4\. Resampling to spacing | $85\ (12)$ | $74\ (18)$ | $70\ (25)$ | $83\ (14)$ | $67\ (21)$ | $67\ (23)$
4.a Bias field correction | $82\ (13)\downarrow$ | $75\ (17)$ | $67\ (25)$ | $80\ (13)$ | 72$\ (19)\uparrow$ | $62\ (22)$
4.b Denoising | $84\ (12)$ | $74\ (17)$ | $70\ (26)$ | $83\ (13)$ | $69\ (21)\uparrow$ | $65\ (25)$
4.c Histogram matching | $83\ (16)$ | $75\ (16)$ | $68\ (26)$ | $81\ (16)$ | $68\ (18)$ | $63\ (26)$
4.d Skull stripping | $87\ (11)$ | $76\ (14)\uparrow$ | $77\ (21)\uparrow$ | $85\ (11)$ | $72\ (18)\uparrow$ | $75\ (19)\uparrow$

Our results are four-fold. First, we analyze the effect of image resampling
(Steps 2-4 Figure 1). Second, we analyze image enhancement methods (Steps
4.a-4.c Figure 1). Next, we discuss the utility of skull-stripping in terms of
segmentation metrics and volume estimates. Finally, we analyze if there is
optimal preprocessing in a transfer learning scenario. We end the results
section with our recommendations on MRI images preprocessing for deep
learning-based tumor segmentation.

###  4.1 Inter-subject image alignment

Table 4 shows validation results of nnU-net and UNETR architectures for the
three datasets.

First, for two larger datasets (GBM and BGPD) we observe that introducing some
resampling strategy to homogenise voxel volume across the dataset is always
beneficial, Table 4 step 1 versus steps 2-4. Recall that without any voxel
resampling, the differences between voxels’ volumes are as large as 10 times
for GBM ($0.5\ \text{mm}^{3}$ versus $5.2\ \text{mm}^{3}$), and 53 times for
BGPD ($0.1\ \text{mm}^{3}$ versus $5.3\ \text{mm}^{3}$), see Table 3. Thus,
for a 3D convolutional neural network, the receptive field of a convolution
filter will differ by a factor of 53. Interestingly, while for the LGG dataset
we also have a difference between voxels’ spacing by a factor of 22, there are
no significant differences in performance between steps 1-4. The latter might
be the consequence of a relatively small sample size.

Second, we observe that applying non-rigid Atlas registration (step 3) lead to
the same results compared to a faster Resampling to the same image size (step
2) or Resampling all images to the same voxel spacing (step 4). For both NN
architectures in all datasets, it is not possible to reject the Null
hypothesis about the equality of means ($p_{\text{values}}>0.1$, accounting
multiple comparisons correction). Recall that images on step 3 are both the
same image size and have a voxel of the same volume555Resampling to same image
size results in almost equal voxel volumes $1.07\ (0.34)\ \text{mm}^{3}$, mean
(std) for BGPD dataset..

###  4.2 Image enhancement

We compare three different intensity normalization steps commonly used in
brain MRI analysis pipelines: Bias field correction (step 4.a), Denoising
(step 4.b) and Histogram matching (step 4.c). As there were no significant
differences found between resampling approaches, steps 4.a-c were performed
after resampling images to the same voxel volume (step 4).

First, for a convolutional nnU-net, intensity correction transformations could
be completely omitted. As show in Table 4, there is no statistically
significant improvement of either of steps 4.a-c compared to step 4
($p_{\text{values}}>0.1$) for all three datasets. In most of the cases, the
average segmentation performance is actually worse (compared to no intensity
normalization, step 2) by an absolute value, though the only statistically
significant drop in performance is Bias field correction on GBM dataset (82
mean Dice score (step 4.a) compared to 85 mean Dice score (step 4),
$p_{\text{value}}=0.014$).

Second, for an attention-based model, the general trend stays the same, except
for the BGPD dataset and steps 4.a and b. Here, we observe a small but
statistically significant increase in performance. We do not have a reasonable
explanation for the effect (see A), though we acknowledge that for all
datasets, UNETR architecture results in worse performance compared to nnU-net.
This might be the effect of a relatively small sample size, as transformer-
based architectures require more training data.

###  4.3 Skull stripping

A brain mask application before training results in a moderate but
statistically significant Dice score improvement for BGPD and LGG datasets
(both nnU-net and UNETR) over the experiment without skull stripping, see
Table 4 4 and 4.d. For the GBM dataset, the average segmentation quality is
larger and the standard deviation is smaller in the experiment with skull
stripping, though after a multiple comparisons correction these differences
are not statistically significant ($p_{\text{value}}=0.09$).

###  4.4 Volumetric errors

In addition to the Dice scores, we provide the errors in tumour volume
estimates for different preprocessing steps.

We report the results for nnU-net in Table 5, as it performs better in our
experiments. In terms of volume estimates, errors follow the same trend as
with Dice scores: inter-subject alignment is always beneficial, image
enhancement does not result in any improvements, and skull stripping
systematically improves the quality.

In volume estimates, skull stripping improves tumor segmentation quality for
GBM and LGG datasets (step 4.d., Table 5) and does not for the BGPD dataset
(for which there is a statistically significant improvement in terms of Dice
scores). This result itself is aligned with the results in Dice scores, yet it
is an argument for using clinically relevant metrics in addition to
segmentation metrics.

Volumetric errors of model predictions on BGPD data with and without skull
stripping are not statistically significant (MAE 26 (42) mL for skull
stripping and MAE 28 (49) mL without skull stripping,
$p_{\text{value}}=0.308$), thus from a clinical perspective this additional
step is not completely justified. Complete volumetric measurements for all
experiments are provided in the A. Additionally, we check if error in volume
estimate depends on tumor total size (Figure 3), and do not observe any
dependence.

Table 5:  Estimated MAE of model prediction from ground truth label for the three datasets: GBM and LGG (WT label), BGPD (GTV label). Results are represented in mL, values are Mean (STD), the lower — the better. Volumes estimates are based on nnU-net. Arrows denote statistically significant difference ($p_{\text{value}}<0.05$) compared to step 4. Resampling to spacing $\uparrow$ (increase), $\downarrow$ (decrease). Data preprocessing | GBM | BGPD | LGG
---|---|---|---
1.Inter-modality registration | $63\ (47)\downarrow$ | $55\ (62)\downarrow$ | $34\ (39)$
2.Resampling to image size | $15\ (14)$ | $28\ (43)$ | $23\ (24)$
3.Atlas registration | $13\ (11)$ | $27\ (45)$ | $31\ (33)$
4.Resampling to spacing | $14\ (13)$ | $28\ (49)$ | $32\ (31)$
4.a Bias field correction | $15\ (15)$ | $27\ (49)$ | $34\ (37)$
4.b Denoising | $13\ (13)$ | $27\ (47)$ | $32\ (34)$
4.c Histogram matching | $14\ (12)$ | $27\ (47)$ | $42\ (61)$
4.d Skull stripping | $10\ (9)\uparrow$ | $26\ (42)$ | $19\ (17)\uparrow$
Figure 3: The relation between tumor volume and its’ estimated volume from
predicted segmentation mask for two nnU-net experiments on BGPD: 4. Resample
to spacing and 4.d Resample to spacing with skull stripping.

(a) GBM dataset, WT label

(b) BGPD dataset, GTV label

Figure 4: nnU-net performance on GBM and BGPD datasets. Horizontal:
segmentation accuracy presented in Dice scores from three fold cross-
validation as $\text{Mean}\ (\text{STD})$ multiplied by 100. Vertical:
preprocessing experiments 1-4.d for models trained with random weights
initialization for 100 epochs (blue), 300 epochs (green) and fine-tuning for
100 epochs from pretrained weighs on another dataset (red).

###  4.5 Transfer learning.

We test if MRI data preprocessing can facilitate model transfer from another
dataset. In particular, we explore if model fine-tuning after training on
preprocessed data is better than model fine-tuning on non-preprocessed data.

We repeat the main ablation study with nnU-net models pretrained on the GBM
dataset for 300 epochs and fine-tune them on BGPD with the same data
preprocessing for 100 epochs (and pretraining on BGPD with fine-tuning of GBM
for corresponding experiments). We compare the Dice scores on three fold
cross-validation for a model trained for 100 and 300 epochs from random weight
initialization, and a model trained on 100 epochs from pretrained weights. The
results are presented in Figure 4.

First, we observe a definitive improvement in nnU-net performance on the GBM
dataset with weight transfer and fine tuning for 100 epochs (red bars),
compared to a training from scratch for 300 epochs (green bars), Figure 4(a).
For the BGPD dataset, pretraining on the GBM sample results in better
segmentation performance compared to training for 100 epochs (blue bars and
red bars), but worse compared to training from scratch for 300 epochs (green
bars), Figure 4(b). This effect could be a consequence of the different sample
sizes, as BGPD is almost two times larger than GBM — thus pretraining on the
BGPD improves GBM segmentation, but not vice versa.

Second, we do not observe any differences in segmentation performance for
either of the preprocessing steps on both datsets. For example, for the GBM
dataset and preprocessing step 4.b Denoising, a model trained for 100 epochs
results in 84 (12) Dice score (STD), the same if trained for 300 epochs, and
86 (10) Dice score if fine-tuned from BGPD. Yet, with the same data
preprocessing on BGPD, we see a decrease in the mean Dice score with weight
transfer from the GBM dataset to BGPD. Similarly, for step 4.d, the
improvement of the segmentation quality on the GBM dataset is not observable
in the BGPD sample. From these experiments we conclude that no preprocessing
step among those studied improves model performance with weight transfer for
both datasets. Numerical results depicted in Figure 4 are accessible in C.

Lastly, we perform an experiment with model fine-tuning from two large
datasets BraTS2021, consisting of 2000 subjects Baid et al. [2021] and EGD
dataset with 774 subjects.van der Voort et al. [2021], with multiclass labels,
similar to GBM and LGG ones. According to our results, model fine-tuning from
a larger sample — the most exploited method of transfer learning Ardalan and
Subbian [2022] — can be reached by longer training, irrespective of the size
of the dataset for weight transfer, see B Table C.

###  4.6 Our recommendations for brain MRI preprocessing for deep learning
segmentation.

The overall results suggest the following recommendations:

  * 1.

It’s essential to align multi-modal MRI data between subjects for analysis,
and even fast methods like image or voxel resizing yield comparable to atlas
registration segmentation accuracy.

  * 2.

Bias-field correction, denoising, and histogram matching are unnecessary in
MRI segmentation pipelines based on UNet-like or UNETR architectures.

  * 3.

Although skull stripping can improve segmentation performance, its impact on
clinical measurements, such as differences in lesion volume estimates, is
relatively small. Therefore, depending on the clinical task and the need for
fast processing times, this step may not be necessary.

  * 4.

Preprocessing MRI data does not help with transfer learning while fine-tuning
models on other datasets. Moreover, there’s almost no significant difference
between fine-tuning models from other data and just doing longer training on
the original sample.

##  5 Conclusion

We perform a rigorous ablation study of the most conventional preprocessing
steps used in the analysis of brain MRI images, including atlas registration,
voxel resampling and image resizing, histogram matching, bias-field
correction, denoising and skull stripping.

Although the image reprocessing steps might be useful for annotators and make
distinct properties of the image more recognizable for the human eye, we show
that only image alignment and voxel resampling are essential for accurate
prediction with DNN models. We conclude that predictions after atlas
registration do not significantly differ from ones with equal voxel
resampling. We observe that bias-field correction, denoising, and histogram
matching reduce data variance and do not affect DNN performance positively. We
point out that skull stripping can lead to a measurable increase in accuracy
and facilitate model convergence. On the other hand, brain extraction is very
computationally expensive, and its incorporation into a pipeline does not
affect clinically relevant volumetric measurements.

Thus we believe that skipping all steps excluding image alignment and voxel
resampling from the brain MRI deep learning pipeline may reduce computational
costs and improve reproducibility across studies.

These recommendations will be especially relevant for MRI data preprocessing
for semi-automated labeling with Segment Anything Model (SAM) and
modifications Kirillov et al. [2023]. SAM is a vision-transformer based
architecture, shown to be extremely useful for data annotation, yet still not
surpassing the SOTA solutions for brain MRI data segmentation Wu et al.
[2023]. In the current work we define necessary preprocessing steps needed for
MRI data annotation and further training, that will ensure reproducible across
the studies and best segmentation accuracy.

###  5.1 Work limitations

Our findings on data preprocessing strategies suggest that overall research
reproducibility will benefit if one discards custom preprocessing steps,
including different skull stripping, various implementations of bias field
correction, denoising, etc. Yet we observed that the results of transfer
learning and model training from scratch are strictly related to datasets’
homogeneity and size. These effects could be different on datests of thousands
of images de Raad et al. [2021].

In the current study we focused on conventional brain MRI data preparation
methods. The newly-developed methods of MRI harmonization, as multi site image
harmonization by cumulative distribution function (CDF) alignment (MICA Wrobel
et al. [2020]) or robust intensity distribution alignment (RIDA) Sederevicius
et al. [2022] could be outperforming the most conventional algorithms for
histogram matching Nyúl et al. [2000]. Advanced image intensity enhancement
methods can be compared with the explored ones, i.e. with orthogonal moments
da Silva et al. [2022] for MR image enhancement. These analyses were outside
the scope of the original study.

###  5.2 Authors contributions

Conceptualization and methodology, A.K., E.K. and P.D.; data pipeline
organization and preprocessing pipeline E.K.; model training and analysis
P.D.; interpretation of results, A.K. and A.D., S.Z., A.G.; writing original
draft A.K., E.K. and P.D.; writing—review and editing, M.B., B.S. and A.D. All
authors have read and agreed to the published version of the manuscript.

###  5.3 Declaration of Competing Interest

The authors declare that they have no known competing financial interests or
personal relationships that could have appeared to influence the work reported
in this paper.

###  5.4 Acknowledgments

This work was conducted in the Artificial Intelligence Research Institute in
Moscow, Russia in collaboration with the National Medical Research Center for
Neurosurgery and the Moscow Gamma Knife Center. The work of A.D., S.Z., A.G.
and A.K. was supported by the Russian Foundation for Basic Research grant
18-29-01054. The work of P.D. and E.K. was supported by the Russian Science
Foundation grant 21-71-10136.

## References

  * Ardalan and Subbian [2022] Ardalan, Z., Subbian, V., 2022\.  Transfer learning approaches for neuroimaging analysis: A scoping review.  Frontiers in Artificial Intelligence 5\.
  * Avants et al. [2009] Avants, B.B., Tustison, N., Song, G., et al., 2009.  Advanced normalization tools (ants).  Insight j 2, 1–35.
  * Baid et al. [2021] Baid, U., Ghodasara, S., Mohan, S., Bilello, M., Calabrese, E., Colak, E., Farahani, K., Kalpathy-Cramer, J., Kitamura, F.C., Pati, S., et al., 2021\.  The rsna-asnr-miccai brats 2021 benchmark on brain tumor segmentation and radiogenomic classification.  arXiv preprint arXiv:2107.02314 .
  * Bakas et al. [2017a] Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J., Freymann, J., Farahani, K., Davatzikos, C., 2017a.  Segmentation labels and radiomic features for the pre-operative scans of the tcga-lgg collection.  The cancer imaging archive 286.
  * Bakas et al. [2017b] Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J.S., Antony, J.B.A., Farahani, K., Davatzikos, C., 2017b.  Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features.  Scientific Data 4.  URL: https://doi.org/10.1038/sdata.2017.117, doi:10.1038/sdata.2017.117.
  * Bakas et al. [2017c] Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J.S., Freymann, J.B., Farahani, K., Davatzikos, C., 2017c.  Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features.  Scientific data 4, 1–13.
  * Bakas et al. [2018] Bakas, S., Reyes, M., Jakab, A., Bauer, S., Rempfler, M., Crimi, A., Shinohara, R.T., Berger, C., Ha, S.M., Rozycki, M., et al., 2018\.  Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the brats challenge.  arXiv preprint arXiv:1811.02629 .
  * Bakas et al. [2022] Bakas, S., Sako, C., Akbari, H., Bilello, M., Sotiras, A., Shukla, G., Rudie, J.D., Santamaría, N.F., Kazerooni, A.F., Pati, S., et al., 2022\.  The university of pennsylvania glioblastoma (upenn-gbm) cohort: advanced mri, clinical, genomics, & radiomics.  Scientific Data 9, 1–12.
  * Bakas et al. [2015] Bakas, S., Zeng, K., Sotiras, A., Rathore, S., Akbari, H., Gaonkar, B., Rozycki, M., Pati, S., Davatzikos, C., 2015.  Glistrboost: combining multimodal mri segmentation, registration, and biophysical tumor growth modeling with gradient boosting machines for glioma segmentation, in: BrainLes 2015, Springer. pp. 144–155.
  * Beare et al. [2018] Beare, R., Lowekamp, B., Yaniv, Z., 2018.  Image segmentation, registration and characterization in r with simpleitk.  Journal of statistical software 86\.
  * Beers et al. [2018a] Beers, A., Gerstner, E., Rosen, B., Clunie, D., Pieper, S., Fedorov, A., Kalpathy-Cramer, J., 2018a.  Dicom-seg conversions for tcga-lgg and tcga-gbm segmentation datasets.  [Data set]. The Cancer Imaging Archive.  doi:10.7937/TCIA.2018.ow6ce3ml.
  * Beers et al. [2018b] Beers, A., Gerstner, E., Rosen, B., et al., 2018b.  Dicom-seg conversions for tcga-lgg and tcga-gbm segmentation datasets.  Cancer Imaging Arch .
  * Berisha et al. [2021] Berisha, V., Krantsevich, C., Hahn, P.R., Hahn, S., Dasarathy, G., Turaga, P., Liss, J., 2021.  Digital medicine and the curse of dimensionality.  NPJ digital medicine 4, 1–8.
  * Chang et al. [2019] Chang, K., Beers, A.L., Bai, H.X., Brown, J.M., Ly, K.I., Li, X., Senders, J.T., Kavouridis, V.K., Boaro, A., Su, C., et al., 2019\.  Automatic assessment of glioma burden: a deep learning algorithm for fully automated volumetric and bidimensional measurement.  Neuro-oncology 21, 1412–1422.
  * Clark et al. [2013a] Clark, K., Vendt, B., Smith, K., Freymann, J., Kirby, J., Koppel, P., Moore, S., Maffitt, D., Pringle, M., Tarbox, L., Prior, F., 2013a.  The cancer imaging archive (TCIA): Maintaining and operating a public information repository.  Journal of Digital Imaging 26, 1045–1057.  URL: https://doi.org/10.1007/s10278-013-9622-7, doi:10.1007/s10278-013-9622-7.
  * Clark et al. [2013b] Clark, K., Vendt, B., Smith, K., Freymann, J., Kirby, J., Koppel, P., Moore, S., Phillips, S., Maffitt, D., Pringle, M., et al., 2013b.  The cancer imaging archive (tcia): maintaining and operating a public information repository.  Journal of digital imaging 26, 1045–1057.
  * Dai et al. [2019] Dai, C., Mo, Y., Angelini, E., Guo, Y., Bai, W., 2019\.  Transfer learning from partial annotations for whole brain segmentation, in: Domain Adaptation and Representation Transfer and Medical Image Learning with Less Labels and Imperfect Data. Springer, pp. 199–206.
  * Eijgelaar et al. [2020] Eijgelaar, R.S., Visser, M., Müller, D.M., Barkhof, F., Vrenken, H., van Herk, M., Bello, L., Conti Nibali, M., Rossi, M., Sciortino, T., et al., 2020\.  Robust deep learning–based segmentation of glioblastoma on routine clinical mri scans using sparsified training.  Radiology: Artificial Intelligence 2, e190103.
  * Ermiş et al. [2020] Ermiş, E., Jungo, A., Poel, R., Blatti-Moreno, M., Meier, R., Knecht, U., Aebersold, D.M., Fix, M.K., Manser, P., Reyes, M., et al., 2020\.  Fully automated brain resection cavity delineation for radiation target volume definition in glioblastoma patients using deep learning.  Radiation oncology 15, 1–10.
  * Freedman and Diaconis [1981] Freedman, D., Diaconis, P., 1981\.  On the histogram as a density estimator: L 2 theory.  Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete 57, 453–476.
  * Győrfi et al. [2021] Győrfi, Á., Szilágyi, L., Kovács, L., 2021.  A fully automatic procedure for brain tumor segmentation from multi-spectral mri records using ensemble learning and atlas-based data enhancement.  Applied Sciences 11, 564\.
  * Hatamizadeh et al. [2022] Hatamizadeh, A., Yang, D., Roth, H.R., Xu, D., 2022.  Unetr: Transformers for 3d medical image segmentation.  2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , 1748–1758.
  * Isensee et al. [2021] Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J., Maier-Hein, K.H., 2021.  nnu-net: a self-configuring method for deep learning-based biomedical image segmentation.  Nature methods 18, 203–211.
  * Isensee et al. [2019] Isensee, F., Schell, M., Pflueger, I., Brugnara, G., Bonekamp, D., Neuberger, U., Wick, A., Schlemmer, H.P., Heiland, S., Wick, W., et al., 2019\.  Automated brain extraction of multisequence mri using artificial neural networks.  Human brain mapping 40, 4952–4964.
  * Juntu et al. [2005] Juntu, J., Sijbers, J., Van Dyck, D., Gielen, J., 2005\.  Bias field correction for mri images, in: Computer recognition systems. Springer, pp. 543–551.
  * Kickingereder et al. [2019] Kickingereder, P., Isensee, F., Tursunova, I., Petersen, J., Neuberger, U., Bonekamp, D., Brugnara, G., Schell, M., Kessler, T., Foltyn, M., et al., 2019\.  Automated quantitative tumour response assessment of mri in neuro-oncology with artificial neural networks: a multicentre, retrospective study.  The Lancet Oncology 20, 728–740.
  * Kirillov et al. [2023] Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y., Dollár, P., Girshick, R., 2023\.  Segment anything.  arXiv:2304.02643 .
  * Kondrateva et al. [2021] Kondrateva, E., Pominova, M., Popova, E., Sharaev, M., Bernstein, A., Burnaev, E., 2021\.  Domain shift in computer vision models for mri data analysis: an overview, in: Thirteenth International Conference on Machine Vision, International Society for Optics and Photonics. p. 116050H.
  * Kurmukov et al. [2021] Kurmukov, A., Dalechina, A., Saparov, T., Belyaev, M., Zolotova, S., Golanov, A., Nikolaeva, A., 2021.  Challenges in building of deep learning models for glioblastoma segmentation: Evidence from clinical data., in: MIE, pp. 298–302.
  * Li et al. [2021] Li, Y., Ammari, S., Balleyguier, C., Lassau, N., Chouzenoux, E., 2021.  Impact of preprocessing and harmonization methods on the removal of scanner effects in brain mri radiomic features.  Cancers 13, 3000\.
  * Menze et al. [2021] Menze, B., Isensee, F., Wiest, R., Wiestler, B., Maier-Hein, K., Reyes, M., Bakas, S., 2021.  Analyzing magnetic resonance imaging data from glioma patients using deep learning.  Computerized medical imaging and graphics 88, 101828.
  * Menze et al. [2014] Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., et al., 2014\.  The multimodal brain tumor image segmentation benchmark (brats).  IEEE transactions on medical imaging 34, 1993–2024.
  * Moradmand et al. [2020] Moradmand, H., Aghamiri, S.M.R., Ghaderi, R., 2020.  Impact of image preprocessing methods on reproducibility of radiomic features in multimodal magnetic resonance imaging in glioblastoma.  Journal of applied clinical medical physics 21, 179–190.
  * Nixon and Aguado [2019] Nixon, M., Aguado, A., 2019\.  Feature extraction and image processing for computer vision.  Academic press.
  * Nyúl et al. [2000] Nyúl, L.G., Udupa, J.K., Zhang, X., 2000.  New variants of a method of mri scale standardization.  IEEE transactions on medical imaging 19, 143–150.
  * Patro and Sahu [2015] Patro, S., Sahu, K.K., 2015\.  Normalization: A preprocessing stage.  arXiv preprint arXiv:1503.06462 .
  * Pedano et al. [2016] Pedano, N., Flanders, A., Scarpace, L., Mikkelsen, T., Eschbacher, J., et al., 2016.  Cancer genome atlas low grade glioma (tcga-lgg) data collection.  Cancer Imaging Archive .
  * Pei et al. [2020] Pei, L., Bakas, S., Vossough, A., Reza, S.M., Davatzikos, C., Iftekharuddin, K.M., 2020\.  Longitudinal brain tumor segmentation prediction in mri using feature and label fusion.  Biomedical signal processing and control 55, 101648.
  * Pérez-García et al. [2021] Pérez-García, F., Sparks, R., Ourselin, S., 2021.  Torchio: a python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning.  Computer Methods and Programs in Biomedicine , 106236URL: https://www.sciencedirect.com/science/article/pii/S0169260721003102, doi:https://doi.org/10.1016/j.cmpb.2021.106236.
  * de Raad et al. [2021] de Raad, K., van Garderen, K., Smits, M., van der Voort, S., Incekara, F., Oei, E., Hirvasniemi, J., Klein, S., Starmans, M., 2021.  The effect of preprocessing on convolutional neural networks for medical image segmentation, in: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), IEEE. pp. 655–658.
  * Ranjbarzadeh et al. [2021] Ranjbarzadeh, R., Bagherian Kasgari, A., Jafarzadeh Ghoushchi, S., Anari, S., Naseri, M., Bendechache, M., 2021\.  Brain tumor segmentation based on deep learning and an attention mechanism using mri multi-modalities brain images.  Scientific Reports 11, 1–17.
  * Rathore et al. [2017] Rathore, S., Bakas, S., Pati, S., Akbari, H., Kalarot, R., Sridharan, P., Rozycki, M., Bergman, M., Tunc, B., Verma, R., et al., 2017\.  Brain cancer imaging phenomics toolkit (brain-captk): an interactive platform for quantitative analysis of glioblastoma, in: International MICCAI Brainlesion Workshop, Springer. pp. 133–145.
  * Rohlfing et al. [2010] Rohlfing, T., Zahr, N.M., Sullivan, E.V., Pfefferbaum, A., 2010\.  The sri24 multichannel atlas of normal adult human brain structure.  Human brain mapping 31, 798–819.
  * Sederevicius et al. [2022] Sederevicius, D., Bjornerud, A., Walhovd, K.B., Van Leemput, K., Fischl, B., Fjell, A.M., 2022\.  A robust intensity distribution alignment for harmonization of t1w intensity values.  bioRxiv .
  * da Silva et al. [2022] da Silva, R.D.C., Jenkyn, T.R., Carranza, V.A., 2022.  Enhanced pre-processing for deep learning in mri whole brain segmentation using orthogonal moments.  Brain Multiphysics 3, 100049\.
  * Smith and Brady [1997] Smith, S.M., Brady, J.M., 1997\.  Susan—a new approach to low level image processing.  International journal of computer vision 23, 45–78.
  * Sturges [1926] Sturges, H.A., 1926.  The choice of a class interval.  Journal of the american statistical association 21, 65–66.
  * Thakur et al. [2020] Thakur, S., Doshi, J., Pati, S., Rathore, S., Sako, C., Bilello, M., Ha, S.M., Shukla, G., Flanders, A., Kotrotsou, A., et al., 2020\.  Brain extraction on mri scans in presence of diffuse glioma: Multi-institutional performance evaluation of deep learning methods and robust modality-agnostic training.  Neuroimage 220, 117081\.
  * Tustison et al. [2010] Tustison, N.J., Avants, B.B., Cook, P.A., Zheng, Y., Egan, A., Yushkevich, P.A., Gee, J.C., 2010.  N4itk: improved n3 bias correction.  IEEE transactions on medical imaging 29, 1310–1320.
  * van der Voort et al. [2021] van der Voort, S.R., Incekara, F., Wijnenga, M.M., Kapsas, G., Gahrmann, R., Schouten, J.W., Dubbink, H.J., Vincent, A.J., van den Bent, M.J., French, P.J., et al., 2021\.  The erasmus glioma database (egd): Structural mri scans, who 2016 subtypes, and segmentations of 774 patients with glioma.  Data in brief 37, 107191\.
  * Wang et al. [2019] Wang, G., Shapey, J., Li, W., Dorent, R., Dimitriadis, A., Bisdas, S., Paddick, I., Bradford, R., Zhang, S., Ourselin, S., et al., 2019.  Automatic segmentation of vestibular schwannoma from t2-weighted mri by deep spatial attention with hardness-weighted loss, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer. pp. 264–272.
  * Wang et al. [2021] Wang, X., Li, X.H., Cho, J.W., Russ, B.E., Rajamani, N., Omelchenko, A., Ai, L., Korchmaros, A., Sawiak, S., Benn, R.A., et al., 2021.  U-net model for brain extraction: Trained on humans for transfer to non-human primates.  Neuroimage 235, 118001\.
  * Wightman et al. [2021] Wightman, R., Touvron, H., Jégou, H., 2021.  Resnet strikes back: An improved training procedure in timm.  arXiv preprint arXiv:2110.00476 .
  * Wrobel et al. [2020] Wrobel, J., Martin, M., Bakshi, R., Calabresi, P.A., Elliot, M., Roalf, D., Gur, R.C., Gur, R.E., Henry, R.G., Nair, G., et al., 2020\.  Intensity warping for multisite mri harmonization.  NeuroImage 223, 117242\.
  * Wu et al. [2023] Wu, J., Fu, R., Fang, H., Liu, Y., Wang, Z., Xu, Y., Jin, Y., Arbel, T., 2023\.  Medical sam adapter: Adapting segment anything model for medical image segmentation.  arXiv preprint arXiv:2304.12620 .
  * Zolotova et al. [2023] Zolotova, S.V., Golanov, A.V., Pronin, I.N., Dalechina, A.V., Nikolaeva, A.A., Belyashova, A.S., Usachev, D.Y., Kondrateva, E.A., Druzhinina, P.V., Shirokikh, B.N., Saparov, T.N., Belyaev, M.G., Kurmukov, A.I., 2023.  Burdenko’s glioblastoma progression dataset (burdenko-gbm-progression) (version 1) [data set].  URL: https://doi.org/10.7937/E1QP-D183, doi:10.7937/E1QP-D183.

##  Appendix A The effect of image enhancement

Figure 5: Computing the distance between healthy tissue and tumor tissue
regions.

Why do image enhancement methods not facilitate segmentation?

We attempt to explain why popular intensity normalization steps have
questionable effect on segmentation performance. Our hypothesis is that while
these steps equalize modalities appearance across the data, they also reduce
the differences between voxels’ intensities within each individual image. We
compare intensity distribution for healthy brain voxels and voxels inside a
tumor mask using Kullback-Leibler divergence (Table 6). We expect that if
preprocessing steps increases KL divergence it should result in increased
segmentation quality, and vice versa. In most of the cases this supposition
holds. Two exceptions are Denoising for Tumor core segmentation and Histogram
matching for Enhancing tumor segmentation.

We report KL values for histograms with fixed bin size (30 bins), and we test
the Freedman-Diaconis heuristic (500-700 bins depending on subject) and
Sturge’s rule (20-60 bins). Both approaches result in different absolute KL
values but preserve a relative trend, without affecting the experiment
conclusion.

In most cases, if the KL divergence decreases (differences between healthy and
tumor tissue decrease), model performance decreases, too, and vice versa
(steps 4.a-d in comparison to step 4, Table 6). In almost all cases, lower KL
values corresponds to lower performance, e.g. for bias-field correction, the
KL between healthy brain and WT tissue is equal to 0.47, compared to 0.61 of
atlas registration, which coincides with a segmentation quality drop (from
86.4 for atlas registration to 84.9 for bias-field corrected data). On the
contrary, for denoised data, the KL are either the same or slightly larger
compared to atlas data: 0.63 vs 0.61 for WT; 4.16 vs 4.01 for TC and 7.11 vs
6.70 for ET, which completely coincides with segmentation performance. The
only comparison that does not follow this explanation is bf-correction for TC
(it has a lower KL compared to atlas data, but slightly better segmentation
quality).

Table 6:  KL divergence values (KL) and JS distance (JS) between intensity
histograms for masked brain w/o tumor and tumor region. Lower values
correspond to smaller differences in voxel intensities between healthy (brain
mask without Whole Tumor(WT) for GBM and RT for BGPD sample) and tumor tissue.
Stars represent statistically significant difference to 4. Resampling to
spacing distances.

Data preprocessing | GBM, KL | GBM, JS | BGPD, KL | BGPD, JS
---|---|---|---|---
4\. Resampling to spacing | $0.830(0.450)$ | $0.303(0.095)$ | $1.640(1.015)$ | $0.477(0.129)$
4.a Bias-field correction | $0.660(0.300)$ | $0.259(0.087)$ | $1.145(0.723)$ | $0.465(0.120)$
4.b Denoising | $0.870(0.470)$ | $0.308(0.097)$ | $1.702(1.045)$ | $0.484(0.129)$
4.c Histogram matching | $0.770(0.470)$ | $0.304(0.095)$ | $1.650(1.029)$ | $0.478(0.130)$

##  Appendix B Model architectures and training

Table 7:  Hyperparameters for nnU-net and UNETR models.  Parameter name | nnU-net | UNETR
---|---|---
learning rate | $0.0003$ | $0.0001$
weight decay | $0$ | $0.00001$
momentum | $0.99$ | $0.99$
patch size | $[128,128,128]$ | $[128,128,128]$
batch size | $2$ | $2$

nnU-net. We use NVIDIA’s nn-Unet implementation for the BraTS2021
challenge666github.com/NVIDIA/DeepLearningExamples/tree/
ddbcd54056e8d1bc1c4d5a8ab34cb570ebea1947/PyTorch/Segmentation/nnUNet. The
following changes were applied by the authors on top of the default nnU-net
architecture Isensee et al. [2021]: increasing the encoder depth to $7$,
modifying the number of convolution channels to $[64,96,128,192,256,384,512]$,
and using deep supervision — two additional output heads at the decoder
levels. The model has multi-modal input of four modalities [T1, CT1, T2,
FLAIR], plus a channel with one-hot encoding for foreground voxels, generated
by image thresholding. We train nn-Unet with a patch size of [128,128,128]
voxels and a batch size of two, a learning rate of $0.0003$ and a Adam
optimizer with momentum of $0.99$.

UNETR. We use vision a transformer-based model UNETR Hatamizadeh et al. [2022]
with an embedding size of $512$ for a 1D sequence of a 3D input of the same
$5$ channels with patches of [128,128,128] voxels and the resolution of each
patch equals 16. The model has $10$ heads and is trained with learning rate
$0.0001$, with a weight decay of $0.00001$, and an Adam optimizer with
momentum of $0.99$.

Model optimization. For the BGPD dataset we train the model to predict one
class label. For the GBM dataset we train the model on three classes,
according to the BraTS data labelling: WT, ET, TC. The model is then trained
with the complex loss function. Each label class is optimized separately with
the weighted sum of binary Cross-Entropy and the Dice loss (the trade-off
weight value to 1 for both losses). The final complex loss function is
optimized for a combination of class labels: the whole tumor (WT) (describes
the union of the tumor core (ET), the non-enhancing part of the tumor core
(NET) and the peritumoral edema ED), the tumor core (TC) (the union of the ET
and NET), and the ET.

Figure 6: Visualization of the two models’ predictions with respect to the
method of image intensity correction. The top raw (b-d) showing the image
segmentation on 3 classes with nnUnet model (a) an example CT1 MR image from
GBM data after resampling to [1,1,1] voxel size; (b) N4 correction of the
image; (c) SUSAN denoising of the image; (d) Histogram standartization; The
bottom row showing tumor segmentation with UNETR model. Colors: ET - red, TC -
white, WT - yellow.

##  Appendix C Additional illustration of the results

In this section we show additional illustrations of the results not
represented in the test’s main body. In Tables 8 and 9 we show the results of
other segmentation classes for multiclass tumor segmentation mask (WT, ET and
TC) for the GBM dataset and (WT and ET) for the LGG dataset.

In Table 10 we show the numerical representation of graphical results from
Figure 4.

Table 8: nnU-net and UNETR segmentation performance for three-class (Whole
tumor, Enhancing tumor, Tumor core) segmentation on GBM dataset. Numbers are
Dice scores mean (std) multiplied by 100. Trained for 300 epoch, columns 1 and
4 are duplicated from Table 4.

| nnU-net | UNETR
---|---|---
Data preprocessing | WT | ET | TC | WT | ET | TC
1\. Inter-modality registration | $44\ (28)\downarrow$ | $37\ (30)\downarrow$ | $30\ (27)\downarrow$ | $39\ (26)\downarrow$ | $32\ (26)\downarrow$ | $28\ (24)\downarrow$
2\. Resampling to image size | $85\ (11)$ | $80\ (18)$ | $74\ (19)$ | $82\ (12)$ | $75\ (18)$ | $72\ (19)$
3\. Atlas registration | $85\ (11)$ | $80\ (17)$ | $72\ (20)$ | $82\ (13)$ | $74\ (18)$ | $69\ (21)$
4\. Resampling to spacing | $85\ (12)$ | $80\ (16)$ | $73\ (19)$ | $83\ (14)$ | $75\ (20)$ | $71\ (22)$
4.a Bias field correction | $82\ (13)\downarrow$ | $80\ (17)$ | $72\ (19)$ | $80\ (13)$ | $75\ (20)$ | $71\ (21)$
4.b Denoising | $84\ (12)$ | $80\ (17)$ | $73\ (20)$ | $83\ (13)$ | $75\ (20)$ | $70\ (22)$
4.c Histogram matching | $83\ (16)$ | $78\ (20)$ | $72\ (21)$ | $81\ (16)$ | $72\ (24)$ | $68\ (24)$
4.d Skull stripping | $87\ (11)$ | $82\ (16)$ | $76\ (19)$ | $85\ (11)$ | $79\ (16)$ | $75\ (19)$

Table 9: nnU-net and UNETR segmentation performance for two-classes (Whole
tumor, Enhancing tumor) segmentation on LGG dataset. Numbers are Dice scores
$\text{mean}(\text{std})$ multiplied by 100, columns 1 and 3 are duplicated
from Table 4.

| nnU-net | UNETR
---|---|---
Data preprocessing | WT | ET | WT | ET
1\. Inter-modality registration | $67\ (27)\downarrow$ | $47\ (29)$ | $66\ (23)$ | $49\ (26)$
2\. Resampling to image size | $72\ (24)$ | $52\ (27)$ | $66\ (26)$ | $49\ (27)$
3\. Atlas registration | $71\ (25)$ | $52\ (29)$ | $67\ (25)$ | $49\ (27)$
4\. Resampling to spacing | $70\ (25)$ | $48\ (28)$ | $67\ (23)$ | $50\ (25)$
4.a Bias field correction | $67\ (25)$ | $45\ (29)$ | $62\ (22)$ | $45\ (25)$
4.b Denoising | $70\ (26)$ | $51\ (30)$ | $65\ (25)$ | $48\ (27)$
4.c Histogram matching | $68\ (26)$ | $49\ (29)$ | $63\ (26)$ | $45\ (28)$
4.d Skull stripping | $77\ (21)\uparrow$ | $54\ (25)\uparrow$ | $75\ (19)\uparrow$ | $54\ (25)\uparrow$

Table 10: nnU-net performance on GBM and BGPD datasets, numerical
representation of Figure 4. Segmentation accuracy presented in Dice scores
from three fold cross-validation as $\text{Mean}\ (\text{STD})$ multiplied by
100. Models trained with random weights initialization for 100 epochs, 300
epochs and fine-tuning for 100 epochs from pretrained weighs on other dataset
(BGPD-GBM, GBM-BGPD finetune). Columns 2 and 5 are duplicated from Table 4.

| GBM | BGPD
---|---|---
Data preprocessing | 100 epochs | 300 epochs | BGPD-GBM | 100 epochs | 300 epochs | GBM-BGPD
1\. Inter-modality registration | $42\ (27)$ | $44\ (28)$ | $43\ (27)$ | $34\ (28)$ | $36\ (29)$ | $36\ (29)$
2\. Resampling to image size | $83\ (13)$ | $85\ (10)$ | $86\ (10)$ | $71\ (18)$ | $73\ (19)$ | $74\ (19)$
3\. Atlas registration | $84\ (12)$ | $85\ (11)$ | $86\ (10)\uparrow$ | $72\ (19)$ | $75\ (16)$ | $72\ (17)$
4\. Resampling to spacing | $84\ (13)$ | $85\ (12)$ | $86\ (11)$ | $72\ (18)$ | $74\ (18)$ | $72\ (19)$
4.a Bias field correction | $83\ (13)$ | $82\ (13)$ | $85\ (11)\uparrow$ | $72\ (17)$ | $75\ (17)$ | $73\ (18)$
4.b Denoising | $84\ (12)$ | $84\ (12)$ | $86\ (10)\uparrow$ | $71\ (19)$ | $74\ (17)$ | $73\ (19)$
4.c Histogram matching | $82\ (16)$ | $83\ (16)$ | $84\ (26)$ | $72\ (16)$ | $75\ (16)$ | $74\ (16)$
4.d Skull stripping | $86\ (11)$ | $87\ (11)$ | $87\ (11)$ | $76\ (15)$ | $76\ (14)$ | $75\ (16)$

Table 11: nnU-net performance with and w/o model transfer from large datasets
(BraTS2021 Baid et al. [2021] and EGD van der Voort et al. [2021].
Segmentation accuracy presented in Dice scores from three fold cross-
validation as $\text{Mean}\ (\text{STD})$ multiplied by 100. Models were
trained for 300 epochs and fine-tuned for 100 epochs from pretrained weights
on the larger dataset (BraTS-BGPD, EGD-GBM, BraTS-EGD finetune). Columns 1 and
3 are duplicated from Table 4.

Data preprocessing |  BGPD, |  |  |  |  |
---|---|---|---|---|---|---
|  BraTS-BGPD, |  |  |  |  |
|  GBM, |  |  |  |  |
|  EGD-GBM, |  |  |  |  |
|  EGD, |  |  |  |  |
|  BraTS-EGD, |  |  |  |  |
| Finetune |  |  |  |  |
3\. Atlas registration | $75\ (17)$ | $74\ (20)$ | $85\ (12)$ | $84\ (11)$ | $-$ | $-$
4.d Skull stripping | $76\ (14)$ | $75\ (17)$ | $87\ (11)$ | $86\ (10)$ | $83\ (12)$ | $83\ (12)$