[type=editor, auid=000,bioid=1, prefix=, orcid=0000-0002-1309-2499] [1]

Conceptualization, Methodology, Software and Interpretation

[orcid=0000-0003-1693-453X]

[orcid =0000-0001-6917-4963]

[cor1]Corresponding author

# Model for Nitric oxide and its dayglow emission in the Martian upper
atmosphere using NGIMS/MAVEN measured neutral and ion densities

Susarla Raghuram<EMAIL_ADDRESS>Physical Research Laboratory,
Ahmedabad, 380009, India.  Laboratory for Atmospheric and Space Physics,
University of Colorado Boulder, Boulder, CO, USA.  Space and Planetary Science
Center, Khalifa University of Science and Technology, Abu Dhabi, UAE.  Maulana
Azad National Insitute of Technology Bhopal, M. P., India.  Anil Bhardwaj
Maneesha Dharwan

###### Abstract

A comprehensive study of Nitric oxide (NO) chemistry in the Martian upper
atmosphere is restricted due to the lack of requisite measurements. NO is an
abundant form of odd nitrogen species in the Martian lower atmosphere and its
density depends on several photochemical processes. We have developed a
photochemical model to study the NO density in the dayside of Martian upper
atmosphere by accounting for various production and loss mechanisms. By
utilizing the Neutral Gas and Ion Mass Spectrometer (NGIMS) on-board Mars
Atmosphere and Volatile Evolution (MAVEN) mission measured neutral and ion
densities during deep dip 8 and 9 campaigns, we modelled NO number density in
the Martian sunlit upper atmosphere for the altitudes between 120 and 200 km.
The modelled NO densities are employed to calculate NO (1,0) $\gamma$ band
emission intensity profiles in the dayside upper atmosphere of Mars. The
calculated NO density and its $\gamma$ band intensity profiles are found to be
consistent with Imaging Ultraviolet Spectrograph (IUVS) onboard MAVEN
observations and also with other modelling studies. We found that the local
CO2 and N2 density variations can lead to a change in NO density and
consequently its dayglow intensity by a factor of 2 to 5. Since NO is a trace
constituent and also its dayglow emissions are strongly obscured by CO Cameron
band emissions, we suggest that the derivation of NO number density based on
our approach can constrain its abundance in the dayside upper atmosphere of
Mars. More observations of (1-0) $\gamma$ band emission along with modelling
will help to study the global distribution of NO in the Martian atmosphere.

###### keywords:

Mars, Atmosphere Abundances, atmospheres Atmospheres, chemistry Atmospheres,
composition Aeronomy

##  1 Introduction

Many in-situ and remote observations along with theoretical studies were
carried out to understand the composition and energetics of the Martian upper
atmosphere (Nier and McElroy, 1977; Krasnopolsky, 1993; Bertaux et al., 2005b;
Withers, 2006; González-Galindo et al., 2009; Bougher et al., 2015a; Mahaffy
et al., 2015; Bhardwaj et al., 2016, 2017). However, the study of minor
neutrals in Martian upper atmosphere is still sparse due to lack of several
key chemical reaction parameters and insufficient measurements from remote and
in-situ observations. Nitric oxide (hereafter NO) is a minor species and its
density in the dayside of Martian upper atmosphere is not well constrained.
Several studies have shown that though the abundance of this species is in
parts per million by volume in the thermosphere of Earth, but can play a
significant role in driving the chemistry and thermal balance of the upper
atmosphere (Barth, 1964; Norton and Barth, 1970; Eparvier and Barth, 1992;
Barth et al., 2003; Bharti et al., 2019). Due to its lowest ionization
potential (9.26 eV) compared to any other atmospheric species, NO quickly does
exchange the charge with O${}_{2}^{+}$ and causes NO+ as the dominant ion at
the lower altitudes around 80 km in the Martian ionosphere (Fox, 2004). Hence,
this species plays a significant role in the thermospheric energy budget and
also in determining the ionospheric composition in the Martian atmosphere. But
the study of NO global distribution and its role in the energy budget of
Martian thermosphere is limited due to the lack of adequate in-situ and remote
observations.

The global distribution of NO in the Martian upper atmosphere can be studied
from the remote observations of its $\gamma$ band emissions in the ultraviolet
region. But the observation of these band emissions and the derivation of NO
density in sunlit Martian upper atmosphere is challenging due to the presence
of intense CO Cameron bands in the same emitting wavelength region.
Spectroscopy for the Investigation of the Characteristics of the Atmosphere of
Mars (SPICAM) experiment on-board Mars Express (MEx) spacecraft (SPICAM/MEx)
and Imaging Ultraviolet Spectrograph (IUVS) on-board Mars Atmosphere and
Volatile EvolutioN (MAVEN) satellite (IUVS/MAVEN) have played a key role in
studying the NO distribution in the night side of Martian upper atmosphere via
$\gamma$ (225–270 nm) and $\delta$ (190–240 nm) band emission observations
(Bertaux et al., 2005a, b; Cox et al., 2008; Gagné et al., 2013; Stevens et
al., 2019). Recently, Stevens et al. (2019) reported first-ever NO dayglow
(1,0) $\gamma$ band emission (A${}^{2}\Sigma^{+}$-X${}^{2}\Pi$) in the
IUVS/MAVEN spectra, which were observed during 6–8 April 2016 at solar zenith
angle (SZA) 75∘. The measured NO (1-0) $\gamma$ band limb radiance profiles
have been used to retrieve NO density in the altitude range 80 to 120 km.
Above 120 km, the retrieval process becomes tedious due to higher intensity of
CO Cameron band emission which is about 50 times brighter than NO $\gamma$
band emission.

Regarding in-situ measurements, Upper Atmosphere Mass Spectrometer (UAMS)
instruments on-board Viking 1 and 2 landers made first-ever in-situ NO number
density measurements in the dayside Martian upper atmosphere on 20 July 1976
(solar longitude, Ls = 97o) and 3 September 1976 (Ls = 118o), respectively, at
three to four different altitudes between 110 and 150 km. (McElroy et al.,
1976; Nier and McElroy, 1977). By accounting for the important photochemical
reaction network, Fox (1993) modelled NO number density and found that their
calcualted density is smaller by a factor of 2–3 compared to the Viking
measurement at an altitude around 115 km. The reason for the discrepancy
between the Fox (1993) modelled and Viking observed NO densities has not been
understood and remained as a long-standing issue due to the lack of adequate
measurements. After nearly four decades, since the Viking landers experiments,
the Neutral Gas and Ion Mass Spectrometer (NGIMS) on-board MAVEN mission
(NGIMS/MAVEN) measured NO number densities in the Martian upper atmosphere
(Mahaffy et al., 2015; Vogt et al., 2017). However, the NGIMS/MAVEN measured
NO densities are an order of magnitude higher than those predicted by earlier
theoretical models. As pointed by Stevens et al. (2019), the contamination in
the measurement, which arises due to the recombination of atomic nitrogen and
atomic oxygen within the instrument, could be a reason for the higher reported
NO densities.

As discussed before, there are several constrains to study NO number density
in the dayside Martian upper atmosphere for the both in-situ (Viking 1, 2, and
NGIMS/MAVEN) and remote (SPICAM/MEx and IUVS/MAVEN) observations. In this
paper, we present an approach of calculating the NO number density in the
dayside Martian upper atmosphere by incorporating NGIMS/MAVEN measured neutral
(CO2, N2, and O) and ion (NO+, O${}_{2}^{+}$ and CO${}_{2}^{+}$) number
densities in the photochemical network. We show that our approach can be used
to constrain the NO density in the altitude range between 120 and 200 km based
on the MAVEN measurements during deep dip 8 and 9 campaigns. We have also
calculated dayside NO (1-0) $\gamma$ limb intensity profiles from our modelled
NO densities and compared them with the IUVS/MAVEN observations. The
methodology adopted in this work is explained in Section 2. The results
obtained analysis are presented in Section 3. We have discussed various
factors which can influence our modelled NO density profiles in Section 4.
This work has been summarized and conclusions are drawn in Section 5.

##  2 Methodology

The neutral and ion number densities are taken from the dayside NGIMS/MAVEN
inbound measurements for SZA smaller than 60∘ during deep dip 8 and 9
campaigns. To extract the neutral and ion density profiles, we utilized level
2 data (L2), version 7 (8), revision 3 (1) data for deep dip 8 (9) campaign.
Additional details of the L2 data product are available in Benna and Elrod
(2018) and the data can be accessed from a web link (https://pds-
atmospheres.nmsu.edu). We chose only those orbits of MAVEN deep dip 8 and 9
campaigns where both neutral and ion densities are measured. The obtained
densities are interpolated over a uniform grid of 1 km from MAVEN’s periapsis
altitude to 200 km. Table 4 summarizes the MAVEN observational conditions,
such as observation period, orbit numbers, Ls, and the variation in SZA and
latitude during orbits, during deep dip 8 and 9 campaigns.

The chemical network considered in this work is based on the compilation of
Fox and Sung (2001) with updated rate coefficients from McElroy et al. (2013).
These chemical reactions are tabulated in Table 2. The electron temperature,
which determines the dissociative recombination rate of ions, is taken from
Ergun et al. (2015). Using the Analytical yield spectrum (AYS) approach, we
calculated the suprathermal electron flux and electron impact dissociation
rate of N2 in the Martian upper atmosphere. More details of degradation of
solar flux and the calculation of suprathermal electron flux using AYS method
can be found in our earlier work (Bhardwaj et al., 1990, 1996; Bhardwaj, 1999;
Jain and Bhardwaj, 2011, 2012; Bhardwaj and Jain, 2013; Raghuram and Bhardwaj,
2020). By assuming photochemical equilibrium condition and accounting for
various production and loss mechanisms, we determined the NO, N(4S) and N(2D)
number densities for the altitudes between 120 and 200 km. The effect of
transport in calculating the number densities of these species is discussed in
Section 4.

We have taken the fluorescence efficiency (g-factor) for NO (1,0) $\gamma$
band emission as 2.68 $\times$ 10-6 photons s-1 molecule-1 and all the
photorates are scaled to heliocentric distance of 1.57 AU (Stevens et al.,
2019). The Volume Emission Rate (VER), which is the number of photons
scattered per unit volume per unit time, is calculated by multiplying the
modelled NO number density with the g-factor. The limb emission intensity of
this band is obtained by integrating the volume emission rate along the line
of sight and converted into brightness in Rayleigh (1 Rayleigh = 106/4$\pi$
photons cm-2 sec-1 sr-1) using the following equation.

$I=2\times 10^{-6}\int_{0}^{\infty}VER(r)\ dr$ (1)

here r is the abscissa along the horizontal line of sight and VER(r) is the
volume emission rate (photons cm-3 sec-1) at a particular emission point r.
The factor 2 multiplication in the above equation is due to the symmetry along
the line of sight concerning the tangent point. While calculating the limb
intensity, we assumed that the emission rate is constant along the local
longitude/latitude. Other Martian species cannot absorb NO emissions along the
line of sight due to low absorption cross section, hence the atmospheric
absorption for this band emission can be neglected.

Table 1: The observational conditions of MAVEN deep dip 8 and 9 campaigns.

#DD1 | Observation period | Orbits | SZA2 | Lat.3 | Ls4
---|---|---|---|---|---
8 | 16–22 October 2017 | 5909–5947 | 22–40∘ | 10∘N–21∘N | 76∘
9 | 24–30 April 2018 | 6936–6973 | 28–59∘ | 19∘S–52∘S | 165∘

111Footnote

Deep dip; 222FootnoteSolar zenith angle; 333FootnoteLatitude; 444FootnoteSolar
longitude.

Table 2: Production and loss reactions of NO, N(4S) and N(2D) for the
altitudes above 120 km in the Martian upper atmosphere.

No. | Reaction | Rate coefficient | Reference
---|---|---|---
|  |  | (cm3 sec-1 or sec-1) |
R1 | N(2D) + CO2 |  $\rightarrow$ NO + CO | 3.6$\times$10-13 | Herron (1999)
R2 | N(2D) + O |  $\rightarrow$ N(4S) + O | 6.90$\times$10-13 | Fell et al. (1990)
R3 | N(2D) + N2 |  $\rightarrow$ N(4S) + N2 | 1.70$\times$10-14 | Herron (1999)
R4 | N(2D) + NO |  $\rightarrow$ N(4S) + NO | 6.70$\times$10-11 | Fell et al. (1990)
R5 | N(2D) + O${}_{2}^{+}$ |  $\rightarrow$ NO+ \+ O | 1.80$\times$10-10 | Goldan et al. (1966)
R6 | N(2D) + eth |  $\rightarrow$ N(4S) + O | 3.86$\times$10-10 (Te/300)0.81 | Berrington and Burke (1981)
R7 | N(4S) + CO2 |  $\rightarrow$ NO + CO | 1.7$\times$10-16 | Fox and Sung (2001)
R8 | N(4S) + NO |  $\rightarrow$ N2 \+ O | 3.38$\times$10-11 (300/Tn)0.17 exp(-2.8/Tn) | McElroy et al. (2013)
R9 | N(4S) + CO${}_{2}^{+}$ |  $\rightarrow$ NO +CO+ | 3.40$\times$10-10 | Scott et al. (1998)
R10 | N(4S) + O${}_{2}^{+}$ |  $\rightarrow$ NO+ \+ O | 1.80$\times$10-10 | McElroy et al. (2013)
R11 | NO+ \+ eth |  $\rightarrow$ N(4S)+ O | 0.60$\times$10-7(300/Te)0.5 | Vejby-Christensen et al. (1998)
R12 | NO+ \+ eth |  $\rightarrow$ N(2D) + O | 3.40$\times$10-7(300/Te)0.5 | Vejby-Christensen et al. (1998)
R13 | NO + CO${}_{2}^{+}$ |  $\rightarrow$ NO+ \+ CO2 | 1.2 $\times$10-10 | McElroy et al. (2013)
R14 | NO + O${}_{2}^{+}$ |  $\rightarrow$ NO+ \+ O2 | 4.6 $\times$10-10 | McElroy et al. (2013)
R15 | N2 \+ h$\nu$ |  $\rightarrow$ N(4S) + N(4S) | Calculated | This work
R16 | N2 \+ h$\nu$ |  $\rightarrow$ N(2D) + N(4S) | Calculated | This work
R17 | N2 \+ eph |  $\rightarrow$ N(4S) + N(4S) | Calculated | This work
R18 | N2 \+ eph |  $\rightarrow$ N(2D) + N(4S) | Calculated | This work
R19 | N2 \+ O${}_{2}^{+}$ |  $\rightarrow$ NO+ \+ NO | 2.0 $\times$10-18 | Matsuoka et al. (1981)
R20 | N(2D) |  $\rightarrow$ N(4S) + h$\nu$ | 2.78 $\times$10-5 | Tachiev and Froese Fischer (2002)

h$\nu$ is photon, eph is photoelectron and eth is thermal electron; Tn and Te
are neutral and electron temperatures, respectively..

##  3 Results

The NGIMS/MAVEN measured neutral and ions densities profiles for orbit #5947
of deep dip 8 campaign are presented in Figure 1. In this Figure, we compared
the NGIMS/MAVEN measured CO2, N2 and O density profiles with Fox (2004)
modelled density profiles, which are constructed based on the Viking
observations for solar minimum condition. It can be noticed in this figure
that Fox (2004) modelled neutral density profiles are higher by a factor of 2
to 4 compared to NGIMS/MAVEN measured values.

Using the NGIMS/MAVEN measurements for orbit #5947, the modelled production
and loss profiles of N(4S), N(2D), and NO are presented in the top and bottom
panels of Figure 2, respectively. The calculated production rate profiles in
the top panels of this Figure show that photodissociation of N2 is the major
source of N(4S) and N(2D) production, whereas the collisional reaction between
N(2D) and CO2 leads to the formation of NO. For the altitudes above 180 km,
radiative decay of N(2D) also contributes significantly to the total formation
of N(4S), whereas the collisional reaction between CO${}_{2}^{+}$ and N(4S) is
also an important source mechanism in the total NO formation. It can be
noticed in this Figure that the role of other chemical processes is small in
the formation of N(4S), N(2D) and NO when compared to previously discussed
major production mechanisms.

The modelled loss frequency profiles in the bottom panels of Figure 2 show
that for the altitudes below 160 km, the total loss of N(4S) is mainly due to
the collisions with NO, and above this distance it is removed by the
collisions of O${}_{2}^{+}$ in the Martian upper atmosphere. The loss
frequency of N(4S) by CO2 collision is smaller by an order of magnitude or
more compared to that of the previously discussed loss mechanisms which is due
to small collisional rate coefficient (more than five order of magnitude
smaller, see reactions R7, R8, and R10 in Table 2). For the altitudes below
180 km, the total loss of N(2D) is mainly due to the collisions with CO2,
which leads to the formation of NO, and above this altitude the radiative
decay is the dominant loss mechanism that causes N(4S) formation. The loss of
NO is mainly due to the collisions with N(4S) which leads to the formation of
N2. Several other reactions are also involved in the loss of these species but
their contribution to the total loss frequency is small.

The modelled number density profiles of N(4S), N(2D) and NO for orbit #5947 of
MAVEN deep dip 8 campaign are presented in Figure 3. Our modelled NO and N(4S)
density profiles are lower and higher by a factor of 2 to 3, respectively,
compared those of Fox (2004). Our modelled N(2D) density profile is closer to
Fox (2004) calculated value for the altitude above 130 km. The reason for
differences between these density profiles is discussed in Section 4.

Figure 1: The NGIMS/MAVEN measured neutral and ion number density profiles for
orbit #5947 of deep dip 8 campaign. The dashed curves with corresponding
colours represent the neutral density profiles for solar minimum condition
from Fox (2004). Figure 2: Various modelled formation (top panels) and
destruction (bottom panels) processes for N(4S), N(2D) and NO in the Martian
upper atmosphere for orbit #5947 during MAVEN deep dip 8 campaign. h$\nu$,
eph, and eth represent the solar photon, photoelectron, and thermal electron,
respectively. Production rate of NO via N2 and O${}_{2}^{+}$ is plotted after
multiplying by a factor of 104. The reaction profile CO${}_{2}^{+}$ and NO,
which is a loss frequency profile of NO, is plotted after multiplying by a
factor of 10. Figure 3: Modelled N(4S), N(2D), and NO densities in the Martian
upper atmosphere for orbit #5947 of MAVEN deep dip 8 campaign. The dashed
curves with corresponding colours represent the modelled density profiles from
Fox (2004) for solar minimum condition.

We have employed the same approach in calculating the NO number density
profiles for all the orbits of MAVEN deep dip 8 and 9 campaigns. Using the
NGIMS/MAVEN measured neutral and ion number density for the orbits of deep dip
8 campaign (#5909 to #5947) and accounting for the previously discussed
photochemical network of reactions, the modelled NO number density profiles
are presented in Figure 4. We find that our modelled NO number density is
varying over different orbits and this is mainly associated with the
variability in the NGIMS/MAVEN measured neutral and ions density measurements.
The variation in the calculated NO number density for different orbits is
found to be small (by a factor of 2) at lower altitudes (around 120 km) and it
is increasing with altitude (more than a factor of 2, see the gray shaded area
in Figure 4). The calculated NO number density profiles for all the 32 orbits,
i.e., from #6936 to #6973, of MAVEN deep dip 9 campaign are presented in
Figure 5. In this case, the calculated NO density around 120 km altitude is
about 106 cm-3 and comparable to the derived value for the deep dip 8
campaign.

Figure 4: The modelled NO number density profiles for the various orbits of
MAVEN deep dip 8 campaign. The Gray shaded area represents the variability in
the calculated NO number density. Figure 5: The calculated NO number density
profiles for various orbits of MAVEN during Deep dip 9 campaign. The Gray
shaded area represents the variability in the modelled NO number density.

In Figure 6, we compared the calculated NO number density profiles during
MAVEN deep dip 8 and 9 campaigns with IUVS/MAVEN retrieved value. The modelled
average NO number density profile for deep dip 8 campaign is consistent with
the IUVS/MAVEN retrieved value at around 125 km altitude. Stevens et al.
(2019) modelled NO density profiles in the altitude range 80 and 160 km at SZA
44∘ and 75∘ for IUVS and Viking observational conditions, respectively (see
dashed red and blue curves in Figure 6). Our modelled NO number density
profiles are also consistent with the Stevens et al. (2019) modelled profiles
in the altitude range 120 to 160 km.

Our calculations in Figure 2 show that N2 and CO2 play a crucial role in
determining the NO density in the Martian upper atmosphere. Hence, we studied
the variation in our modelled NO density with respect to the changes in MAVEN
measured CO2 and N2 densities. It can be noticed in Figure 7 that both the CO2
and N2 densities locally vary about a factor of 2 to 5 during deep dip 8 and 9
campaigns in the altitude range 120 to 200 km (see the shaded areas in figure
7). Due to significant variability in the measured densities, it is difficult
to describe the impact of CO2 and N2 on NO for each orbit of both deep dip 8
and 9 campaigns. Hence, we chose two orbits for each campaign to demonstrate
the impact of MAVEN measurements on the modelled NO density. As shown in the
top panel of Figure 7, the MAVEN measured CO2 and N2 densities for orbit #5943
are higher compared to those for #5947 at the altitudes below 165 km and above
this radial distance it is opposite. This cross over in the measured CO2 and
N2 density profiles is also reflected in our modelled NO profiles for the
corresponding orbits at 165 km altitude (see solid and dashed purple lines in
the top panel of Figure 7). Similarly, as shown in the bottom panel of this
figure, the MAVEN measured CO2 and N2 densities for orbits #6937 are higher
compared to those for #6949 for deep dip 9 campaign in the altitude range 120
to 200 km. The higher densities of CO2 and N2 lead to larger production of NO
which is reflected in the modelled NO density for orbit #6937. These
calculations suggest that the change in CO2 and N2 densities directly can
influence the NO density in the Martian upper atmosphere.

Figure 6: Comparison between the modelled and MAVEN deep dip 8 (solid red
curve for the average), 9 (solid blue curve red for the average) and
IUVS/MAVEN (solid black curve) retrieved mean and Viking modelled at SZA 44o
(blue-dashed curve) and IUVS modelled at SZA 75o (red-dashed curve) NO number
density profiles in the Martian upper atmosphere. The yellow and green shaded
areas represents the variability in calculated NO number density profiles
during MAVEN deep dip 8 and 9 campaigns, respectively. The horizontal Gray
error bars represents the digitized data of 1-$\sigma$ variability about the
mean value of the IUVS derived NO number density (taken from Stevens et al.,
2019).

Figure 7: Variation in the modelled NO density due to the change in the
NGIMS/MAVEN measured CO2 and N2 densities during the MAVEN deep dip 8 (top
panel) and 9 (top panel) campaigns. The solid and dashed curves in the top
panel (bottom panel) represent the corresponding number densities for orbits
#5943 and #5947 (#6937 and #6949), respectively, for MAVEN deep dip 8 (9)
campaign. Figure 8: Modelled limb radiance profiles for Nitric oxide (NO) 1-0
$\gamma$ emission for the MAVEN Deep dip 8 campaign. The Gray shaded area
represents the variability in the calculated NO (1-0) $\gamma$ intensity.
Figure 9: Modelled limb radiance profile of Nitric oxide (NO) emission for the
MAVEN deep dip 9 campaign. The Gray shaded area represents the variability in
the calculated NO (1-0) $\gamma$ intensity. Figure 10: Comparison between the
modelled and the IUVS/MAVEN observed NO (1,0) $\gamma$ band limb intensity
profiles. The black and red curves represent the calculated average NO (1,0)
$\gamma$ band limb intensity for deep dip 8 and 9 missions respectively. Blue
curve and gray errors bars represents the observed average IUVS/MAVEN dayglow
profile and 1-$\sigma$ uncertainty associated with observation, respectively
(taken from Stevens et al., 2019). Green and yellow shaded areas represent the
variability in the calculated NO (1,0) $\gamma$ band limb intensity for deep
dip 8 and 9 missions, respectively.

By utilizing the corresponding modelled NO number densities, the calculated
limb intensity profiles for NO (1,0) $\gamma$ band for the MAVEN deep dip 8
and 9 campaigns are presented in Figures 8 and 9, respectively. The calculated
NO (1,0) $\gamma$ band limb intensity is found to vary between 90 and 200 R
(between 100 and 300 R) at the MAVEN periapsis altitude around 120 km during
MAVEN deep dip 8 (9) campaign. We compared our calculated NO (1, 0) $\gamma$
limb intensity profiles with and IUVS/MAVEN observation in Figure 10. Our
calculated mean limb intensities for deep dip 8 and 9 campaigns around 120 km
altitude are found to be consistent with the IUVS/MAVEN observation.

##  4 Discussion

During the deep dip campaigns, NGIMS/MAVEN measured both neutral and ion
densities at the altitudes as low as 125 km, whereas the regular measurements
cover the altitudes above 150 km. This possibility enabled us to calculate NO
number density at the lower altitudes which is not possible for other
measurements. Out of the total nine MAVEN deep dip campaigns so far, deep dip
2, 8, and 9 campaigns are occurred on the dayside. Recently, Cui et al. (2020)
also used the NGIMS neutral and ion density measurements to constrain the NO
density in the dayside Martian upper atmosphere, specifically for the deep dip
2 campaign during 17-22 April 2015 with a periapsis altitude of 130 km and
appropriate for the subsolar condition. By neglecting transport, these authors
carried out time-dependent calculations at a fixed altitude of 160km to
investigate the diurnal variation under the influences of both dayside solar
radiation and nightside solar wind electron precipitation. The present study
has several advances over Cui et al. (2020) in that (1) our calculations are
made over a broad altitude range of 120 - 200 km to provide the vertical
distribution of NO for deep dip 8 and 9 campaigns; (2) we emphasize the
dayside variability in NO density driven by the variability of the background
atmosphere, especially in terms of the CO2 and N2 densities, rather than the
mean diurnal variability focused by Cui et al. (2020); (3) we also estimate
the NO (1,0) gamma band emission such that our results could be directly
compared to the MAVEN IUVS limb observations. By comparing our modelled NO
density profiles with Cui et al. (2020) calculations, we find that our
calculated densities are smaller by a factor 2 to 3. This difference could be
due to the different prevailed conditions in the Martian upper atmosphere
during deep dip 2, 8 and 9 campaigns. Moreover, Cui et al. (2020) account for
local time variability of neutral species in their calculations which is not
the case in this work.

The calculations presented in Figure 2 show that the odd nitrogen species,
which are primarily produced from the photodissociation of N2, initiate the
formation of NO via collisions with CO2, and further interaction of these
species with NO subsequently recycles the N2 in the Martian upper atmosphere
(see chemical reactions R1, R8, R15 and R16 in Table 2). So it should be
noticed that besides variation in CO2 and N2 densities, the NO density in the
Martian upper atmosphere depends significantly on chemical reactions
particularly R1 and R8, which are respective major production and loss
channels initiated by photodissociation of N2. However, the calculations
presented in figure 2 suggest that the NGIMS/MAVEN measured major neutral and
ion densities (CO2, N2, and O${}_{2}^{+}$) along with a few major chemical
reactions are sufficient to determine the NO density in the altitude range 120
to 200 km, rather than using a complex chemical network.

The difference between our calculated odd nitrogen species density profiles
and Fox (2004) modelled values, which is about a factor of 2 to 5, is mainly
due to the change in input CO2 and N2 densities (see Figures 1 and 3). But our
modelled NO density profile in the altitude range 120 to 200 km is closer to
the Fox (2004) calculation (with a factor of 2 difference), which suggests
that NO is strongly controlled by photochemical reactions rather than
transport in the upper atmosphere. Time scales of odd nitrogen species as
determined by Cui et al. (2020) also suggest that NO is under photochemical
equilibrium for the altitudes up to 180 km and above which chemical diffusion
significantly controls its density (see Fig. 1d in Cui et al., 2020).
Moreover, the agreement between our calculated NO densities for different
orbits of deep dip 8 and 9 campaigns with the Stevens et al. (2019) modelled
values for IUVS/MAVEN observation condition in the altitude range 120 to 160
km also supports our approach that NO can be derived from the NGIMS/MAVEN
measured CO2 and N2 densities and also by considering the major photochemical
reactions (see Figure 6).

NO density in the Martian upper atmosphere is significantly driven by
photochemical reactions as well as local neutral densities of CO2 and N2. Our
calculations in Figure 2 show that below 170 km, the chemical cycle of NO
mainly depends on density distribution of N2 and CO2 in the Martian upper
atmosphere. Hence, the variation in the measured densities of these species
directly can impact the modelled NO number density. As explained before, the
variability in the MAVEN measured CO2 and N2 densities is directly linked to
the NO density in the Martian upper atmosphere (see solid and dashed curves in
Fig. 7) The higher volume mixing ratios of N2 leads to the larger formation
rate of dissociative products N(4S) and N(2D) and eventually can produce more
NO. Since the major production source of NO is via collisional reactions
between N(2D) with CO2, the variation in local CO2 density can also control
the NO density. The ambient density variation presented in the Figure 7 show
that during deep dip 8 and 9 campaigns both CO2 and N2 densities varied by a
factor of 5 or more in the Martian upper atmosphere which results in the
variation in our calculated NO densities. The calculations done by Cui et al.
(2020) for deep dip 2 campaign show that the diurnal variation in NO, N(4S)
and N(2D) at the reference altitude 160 km is due to ambient N2 mixing ratio
at 160 km. But the calculations presented in this work show that besides the
variation in N2 volume mixing ratio, the change in CO2 density also plays an
important role in determining the NO density in the Martian upper atmosphere.

As shown in Figure 6, our calculated NO number densities around 125 km, for
deep dip 8 and dip 9 campaigns, are consistent with the IUVS/MAVEN
measurements and also with the modelled density profiles in the altitude range
120–160 km. It can be noticed in this figure that at 125 km altitude, the
calculated NO number density varies by a factor of about 5 over the both deep
dip campaigns, which is mainly due to the local variation in the measured CO2
and N2 densities. The earlier MAVEN deep dip campaigns also reveal that there
is a variability in the measured neutrals and ions in the Martian upper
atmosphere (Bougher et al., 2015b). Small scale structures in the NGIMS/MAVEN
observed neutral and ion densities are linked to wave activity in the upper
atmosphere or solar wind interaction with ionosphere (England et al., 2017;
Kopf et al., 2008). Williamson et al. (2019) also observed large scale
amplitude perturbations in the measured densities of different species, which
could be due to gravity waves. These observations suggest that dynamics in the
upper atmosphere could significantly influence the neutral densities of N2 and
CO2, and subsequently affect the NO density in the Martian upper atmosphere.

Stevens et al. (2019) were able to analyze the IUVS/MAVEN observed NO $\gamma$
limb intensity profiles for a small period of observation i.e., 6–8 April
2016. Based on the observed limb intensity, the NO number densities were
retrieved from the observed spectra between the altitudes 80 to 128 km, above
which the retrieval becomes very difficult due to the presence of intense CO
Cameron band emissions. But our method of calculation is able to determine the
NO number density profile from 120 to 200 km. Since the derivation of NO
density at higher altitudes is difficult, we suggest that our approach can be
used as a baseline to constrain the NO abundance in the sunlit Martian upper
atmosphere for different MAVEN observational conditions. However, the MAVEN
observations are limited up to periapsis altitude and it is difficult to
compute the NO densities lower than 120 km altitude using this approach.

Stevens et al. (2019) modelled NO density for the IUVS/MAVEN observation
period and found that their derived density is smaller by a factor of 5
compared to the earlier Viking observations for the altitudes below 100 km.
They ascribed this discrepancy to the assumed collisional rate coefficient for
the reaction between N(4S) and CO2, which is 1 $\times$ 10-18 cm3 s-1 instead
of earlier Fox (2004) assumed value of 1 $\times$ 10-16 cm3 s-1. By using the
rate coefficient as used by Fox (2004), our calculations in Figure 2 show that
the contribution from N(4S) and CO2 collisions is negligible to the total
formation of NO for the altitudes above 120 km. By considering Stevens et al.
(2019) assumed rate coefficient in our calculations, which is two orders of
magnitude smaller compared to the value used in the present work, we find no
change in the modelled NO density profile. Thus, this calculation suggests
that the contribution of N(4S) and CO2 collisional reaction in the formation
of NO can be neglected for the altitudes above 120 km.

Besides the observations of major neutral species, NGIMS/MAVEN also provides
NO density measurements in the Martian upper atmosphere. But various
observations show that the NGIMS/MAVEN measured NO densities are higher by an
order of magnitude compared to IUVS/MAVEN derived values. This higher density
is attributed to the recombination of N and O atoms inside the walls of mass
spectrometer which produces additional NO. As Stevens et al. (2019) noticed,
the effect of contamination in measuring the NO number density due to
recombination of atomic oxygen and nitrogen cannot be neglected. By studying
various factors associated in the calculation of NO density, Fox (2004)
concluded that Viking 2 measurements may not be accurate. Hence, the
instrumental bias during Viking 2 measurements could be the main reason for
the discrepancy between the modelled and observation of NO densities. Thus,
there remains a compelling need of new measurements of NO densities in the
Martian upper atmosphere to reconcile this long-standing problem.

The high emission rate factor makes NO (1,0) transition as the brightest
feature of $\gamma$ band emission at the wavelength 214.9 nm compared to other
transitions in the ultraviolet region. But this band emission is strongly
obscured by CO (0,1) Cameron band emission (215.5 nm) due to proximity in the
wavelength and also due to its intensity. Using the derived NO densities, our
calculated limb intensities during deep dip 8 and 9 campaigns are consistent
with the mean value of IUVS/MAVEN measurement around 120 km (see Figure 10).
However, it should be cautioned that the uncertainty in the measured limb
radiance is large for the altitudes above 120 km (Stevens et al., 2019).

Fox (2004) has modelled NO number density for low solar activity condition, by
assuming a downward flux of -2 $\times$ 107 cm-2 s-1 at lower boundary i.e.,
80 km. Stevens et al. (2019) have reduced this downward flux value by two
orders of magnitude to fit the IUVS/MAVEN retrieved NO density profiles. They
also noticed that assumed transport flux can affect the shape of fitted
profile only for altitudes below 100 km. But our calculated NO number
densities around 120 km altitude are consistent with IUVS modelled profiles
also supports the assumption that the transport has a minor role in
determining NO number density in the altitude range 120–160 km (see Figure 6).
Hence, we suggest that the NGIMS/MAVEN measured neutral and ion densities are
suitable to study NO density and also its dayglow emission intensity for
different seasonal conditions in the dayside upper atmosphere of Mars in this
altitude range.

As earlier discussed, the intense CO Cameron band inhibits the measurement of
NO emission intensity in the dayglow spectra. Our calculation in Figure 10
suggests that the emission intensity falls by an order of magnitude beyond 150
km, hence it is difficult to extract NO number density from the faint NO
$\gamma$ band emission in the background of intense CO Cameron band emission.
Hence, we suggest that the derivation of NO number density based on
photochemistry and subsequently calculating its dayglow emission intensity can
serve as a baseline while analysing the ongoing and upcoming IUVS/MAVEN
observations. More such observations of NO dayglow emissions along with
modelling studies are necessary to constrain its variability in the Martian
upper atmosphere.

##  5 Summary and Conclusions

The in-situ NO density measurements are difficult to make in the Martian upper
atmosphere because of its low abundance (more than 3 orders of magnitude
compared to CO2) and also highly reactive in nature. Moreover, the thermal
recombination of atomic nitrogen and atomic oxygen inside the mass
spectrometer also inhibits the accurate determination of NO density.
Measurement of NO density from the remote dayglow observations is also
difficult since its strongest ultraviolet emission feature, i.e., (1,0) gamma
band, is severely obscured by the intense CO Cameron band (50 times higher in
magnitude). By accounting for major chemical reactions and using the
NGIMS/MAVEN measured neutral and ion densities, we developed a photochemical
model to study the photochemistry of NO in the Martian upper atmosphere.
MAVEN’s deep dip campaigns provide a unique opportunity to study NO
photochemistry in the dayside Martian upper atmosphere at the altitudes as low
as 120 km. By utilizing the NGIMS/MAVEN measured neutral and ion densities
during MAVEN deep dip 8 and 9 campaigns in our model, we present a method to
calculate NO number density in the dayside upper atmosphere of Mars. Using the
modelled NO number density profiles we also calculated NO dayglow (1,0)
$\gamma$ band emission intensity. Our calculated NO number density profiles
are consistent with the IUVS/MAVEN retrieved profile and also with the
modelled values for IUVS observation conditions. We found that the calculated
NO number density varies by a factor of 2–5 and consequently its dayglow
intensity due to the variation in CO2 and N2 densities at around 120 km.
Hence, we suggest that the present method of calculation serves as a baseline
to estimate the NO number density and also its $\gamma$ bands emission
intensity under different seasonal and solar conditions in the dayside Martian
upper atmosphere. Based on our calculations we suggest that future MAVEN deep
dip and remote IUVS/MAVEN observations should be focused in the altitude
region 120 and 130 km to constrain the NO density in the Martian upper
atmosphere. More observations of NO emissions along with modelling studies are
necessary to fully comprehend the NO density distribution on the dayside upper
atmosphere of Mars.

## Acknowledgements

SR is supported by Department of Science and Technology (DST) with Innovation
in Science Pursuit for Inspired Research (INSPIRE) faculty award
[grant:dst/inspire/04/2016/002687], and he would like to thank Physical
Research Laboratory for facilitating conducive research environment. He is a
visiting researcher at Khalifa Univeristy during a part of this work. AB was
J.C. Bose Fellow duirng the period of this work. A part of this work was done
when MD was PDF at PRL and presently working at Maulana Azad National
Institute of Technology Bhopal as an assistant professor.

## Data Availability

This paper make use of NGIMS/MAVEN measured neutral and ion number densities
L2 data for deep dip 8 and 9 campaigns which has been accessed through the web
link https://pds-atmospheres.nmsu.edu. The derived data generated in this
research will be shared on reasonable request to the corresponding author.

## References

  * Barth (1964) Barth, C.A., 1964.  Rocket Measurement of the Nitric Oxide Dayglow.  J. Geophys. Res. 69, 3301–3303.  doi:10.1029/JZ069i015p03301.
  * Barth et al. (2003) Barth, C.A., Mankoff, K.D., Bailey, S.M., Solomon, S.C., 2003\.  Global observations of nitric oxide in the thermosphere.  Journal of Geophysical Research (Space Physics) 108, 1027.  doi:10.1029/2002JA009458.
  * Benna and Elrod (2018) Benna, M., Elrod, M., 2018\.  Mars Atmosphere and Volatile Evolution (MAVEN) mission, Neutral Gas and Ion Mass Spectrometer (NGIMS), NGIMS PDS Software Interface Specification, Revision 1.9 Rev. 1.9.
  * Berrington and Burke (1981) Berrington, K.A., Burke, P.G., 1981\.  Effective collision strengths for forbidden transitions in e-N and e-o scattering.  Planet. Space Sci. 29, 377–381.  doi:10.1016/0032-0633(81)90026-X.
  * Bertaux et al. (2005a) Bertaux, J.L., Leblanc, F., Perrier, S., Quemerais, E., Korablev, O., Dimarellis, E., Reberac, A., Forget, F., Simon, P.C., Stern, S.A., Sandel, B., 2005a.  Nightglow in the Upper Atmosphere of Mars and Implications for Atmospheric Transport.  Science 307, 566–569.  doi:10.1126/science.1106957.
  * Bertaux et al. (2005b) Bertaux, J.L., Leblanc, F., Witasse, O., Quemerais, E., Lilensten, J., Stern, S.A., Sandel, B., Korablev, O., 2005b.  Discovery of an aurora on Mars.  Nature 435, 790–794.  doi:10.1038/nature03603.
  * Bhardwaj (1999) Bhardwaj, A., 1999.  On the role of solar EUV, photoelectrons, and auroral electrons in the chemistry of C(1D) and the production of C I 1931 Å in the inner cometary coma: A case for comet P/Halley.  J. Geophys. Res. 104, 1929 – 1942.  doi:10.1029/1998JE900004.
  * Bhardwaj et al. (1996) Bhardwaj, A., Haider, S.A., alal, R.P., 1996.  Production and emissions of atomic carbon and oxygen in the inner coma of comet 1P/Halley: Role of electron impact.  Icarus 120, 412 – 430.  doi:10.1006/icar.1996.0061.
  * Bhardwaj et al. (1990) Bhardwaj, A., Haider, S.A., Singhal, R.P., 1990.  Auroral and photoelectron fluxes in cometary ionospheres.  ICARUS 85, 216–228.  doi:10.1016/0019-1035(90)90112-M.
  * Bhardwaj and Jain (2013) Bhardwaj, A., Jain, S.K., 2013\.  CO Cameron band and CO2+ UV doublet emissions in the dayglow of Venus: Role of CO in the Cameron band production.  Journal of Geophysical Research (Space Physics) 118, 3660–3671.  doi:10.1002/jgra.50345, arXiv:1305.6139.
  * Bhardwaj et al. (2017) Bhardwaj, A., Thampi, S.V., Das, T.P., Dhanya, M.B., Naik, N., Vajja, D.P., Pradeepkumar, P., Sreelatha, P., Abhishek J., K., Thampi, R.S., Yadav, V.K., Sundar, B., Nand i, A., Padmanabhan, G.P., Aliyas, A.V., 2017.  Observation of suprathermal argon in the exosphere of Mars.  Geophys. Res. Lett. 44, 2088--2095.  doi:10.1002/2016GL072001.
  * Bhardwaj et al. (2016) Bhardwaj, A., Thampi, S.V., Das, T.P., Dhanya, M.B., Naik, N., Vajja, D.P., Pradeepkumar, P., Sreelatha, P., Supriya, G., Abhishek J., K., Mohankumar, S.V., Thampi, R.S., Yadav, V.K., Sundar, B., Nandi, A., Padmanabhan, G.P., Aliyas, A.V., 2016.  On the evening time exosphere of Mars: Result from MENCA aboard Mars Orbiter Mission.  Geophys. Res. Lett. 43, 1862--1867.  doi:10.1002/2016GL067707.
  * Bharti et al. (2019) Bharti, G., Sunil Krishna, M.V., Singh, V., 2019.  Radiative cooling due to NO at 5.3 $\mu$ m emission as observed by TIMED/SABER over Asian sector.  Advances in Space Research 64, 1989--2001.  doi:10.1016/j.asr.2019.07.016.
  * Bougher et al. (2015a) Bougher, S., Jakosky, B., Halekas, J., Grebowsky, J., Luhmann, J., Mahaffy, P., Connerney, J., Eparvier, F., Ergun, R., Larson, D., McFadden, J., Mitchell, D., Schneider, N., Zurek, R., Mazelle, C., Andersson, L., Andrews, D., Baird, D., Baker, D.N., Bell, J.M., Benna, M., Brain, D., Chaffin, M., Chamberlin, P., Chaufray, J.Y., Clarke, J., Collinson, G., Combi, M., Crary, F., Cravens, T., Crismani, M., Curry, S., Curtis, D., Deighan, J., Delory, G., Dewey, R., DiBraccio, G., Dong, C., Dong, Y., Dunn, P., Elrod, M., England, S., Eriksson, A., Espley, J., Evans, S., Fang, X., Fillingim, M., Fortier, K., Fowler, C.M., Fox, J., Gröller, H., Guzewich, S., Hara, T., Harada, Y., Holsclaw, G., Jain, S.K., Jolitz, R., Leblanc, F., Lee, C.O., Lee, Y., Lefevre, F., Lillis, R., Livi, R., Lo, D., Ma, Y., Mayyasi, M., McClintock, W., McEnulty, T., Modolo, R., Montmessin, F., Morooka, M., Nagy, A., Olsen, K., Peterson, W., Rahmati, A., Ruhunusiri, S., Russell, C.T., Sakai, S., Sauvaud, J.A., Seki, K., Steckiewicz, M., Stevens, M., Stewart, A.I.F., Stiepen, A., Stone, S., Tenishev, V., Thiemann, E., Tolson, R., Toublanc, D., Vogt, M., Weber, T., Withers, P., Woods, T., Yelle, R., 2015a.  Early MAVEN Deep Dip campaign reveals thermosphere and ionosphere variability.  Science 350, 0459\.  doi:10.1126/science.aad0459.
  * Bougher et al. (2015b) Bougher, S.W., Pawlowski, D., Bell, J.M., Nelli, S., McDunn, T., Murphy, J.R., Chizek, M., Ridley, A., 2015b.  Mars Global Ionosphere-Thermosphere Model: Solar cycle, seasonal, and diurnal variations of the Mars upper atmosphere.  Journal of Geophysical Research (Planets) 120, 311--342.  doi:10.1002/2014JE004715.
  * Cox et al. (2008) Cox, C., Saglam, A., Gérard, J.C., Bertaux, J.L., González-Galindo, F., Leblanc, F., Reberac, A., 2008.  Distribution of the ultraviolet nitric oxide Martian night airglow: Observations from Mars Express and comparisons with a one-dimensional model.  Journal of Geophysical Research (Planets) 113, E08012.  doi:10.1029/2007JE003037.
  * Cui et al. (2020) Cui, J., Fu, M.H., Ren, Z.P., Gu, H., Guo, J.H., Wu, X.S., Wu, Z.P., Lai, H.R., Wei, Y., 2020.  Nitric Oxide Abundance in the Martian Thermosphere and Its Diurnal Variation.  Geophys. Res. Lett. 47, e87252.  doi:10.1029/2020GL087252.
  * England et al. (2017) England, S.L., Liu, G., Yiǧit, E., Mahaffy, P.R., Elrod, M., Benna, M., Nakagawa, H., Terada, N., Jakosky, B., 2017.  MAVEN NGIMS observations of atmospheric gravity waves in the Martian thermosphere.  Journal of Geophysical Research (Space Physics) 122, 2310--2335.  doi:10.1002/2016JA023475.
  * Eparvier and Barth (1992) Eparvier, F.G., Barth, C.A., 1992\.  Self-absorption theory applied to rocket measurements of the nitric oxide (1,0) $\gamma$ band in the daytime thermosphere.  J. Geophys. Res. 97, 13723--13731.  doi:10.1029/92JA00993.
  * Ergun et al. (2015) Ergun, R.E., Morooka, M.W., Andersson, L.A., Fowler, C.M., Delory, G.T., Andrews, D.J., Eriksson, A.I., McEnulty, T., Jakosky, B.M., 2015.  Dayside electron temperature and density profiles at Mars: First results from the MAVEN Langmuir probe and waves instrument.  Geophys. Res. Lett. 42, 8846--8853.  doi:10.1002/2015GL065280.
  * Fell et al. (1990) Fell, C., Steinfeld, J.I., Miller, S., 1990.  Quenching of N(2D) by O(3P).  J. Chemphys. 92, 4768--4777.  doi:10.1063/1.457694.
  * Fox (1993) Fox, J.L., 1993.  The production and escape of nitrogen atoms on Mars.  J. Geophys. Res. 98, 3297--3310.  doi:10.1029/92JE02289.
  * Fox (2004) Fox, J.L., 2004.  Response of the Martian thermosphere/ionosphere to enhanced fluxes of solar soft X rays.  J. Geophys. Res. (Space Physics) 109, A11310.  doi:10.1029/2004JA010380.
  * Fox and Sung (2001) Fox, J.L., Sung, K.Y., 2001\.  Solar activity variations of the Venus thermosphere/ionosphere.  J. Geophys. Res. (Space Physics) 106, 21305--21336.  doi:10.1029/2001JA000069.
  * Gagné et al. (2013) Gagné, M.E., Bertaux, J.L., González-Galindo, F., Melo, S.M.L., Montmessin, F., Strong, K., 2013\.  New nitric oxide (NO) nightglow measurements with SPICAM/MEx as a tracer of Mars upper atmosphere circulation and comparison with LMD-MGCM model prediction: Evidence for asymmetric hemispheres.  Journal of Geophysical Research (Planets) 118, 2172--2179.  doi:10.1002/jgre.20165.
  * Goldan et al. (1966) Goldan, P.D., Schmeltekopf, A.L., Fehsenfeld, F.C., Schiff, H.I., Ferguson, E.E., 1966.  Thermal Energy Ion-Neutral Reaction Rates. II. Some Reactions of Ionospheric Interest.  J. Chemphys. 44, 4095--4103.  doi:10.1063/1.1726588.
  * González-Galindo et al. (2009) González-Galindo, F., Forget, F., López-Valverde, M.A., Angelats i Coll, M., Millour, E., 2009.  A ground-to-exosphere Martian general circulation model: 1. Seasonal, diurnal, and solar cycle variation of thermospheric temperatures.  Journal of Geophysical Research (Planets) 114, E04001.  doi:10.1029/2008JE003246.
  * Herron (1999) Herron, J.T., 1999.  Evaluated Chemical Kinetics Data for Reactions of N(2D) N(2P), and N2(A3$\Sigma$+u) in the Gas Phase.  Journal of Physical and Chemical Reference Data 28, 1453--1483.  doi:10.1063/1.556043.
  * Jain and Bhardwaj (2011) Jain, S.K., Bhardwaj, A., 2011\.  Model calculation of N2 Vegard-Kaplan band emissions in Martian dayglow.  Journal of Geophysical Research (Planets) 116, E07005.  doi:10.1029/2010JE003778, arXiv:1108.0490.
  * Jain and Bhardwaj (2012) Jain, S.K., Bhardwaj, A., 2012\.  Impact of solar EUV flux on CO Cameron band and CO2 UV doublet emissions in the dayglow of Mars.  Planetary and Space Science 64, 110 -- 122.
  * Kopf et al. (2008) Kopf, A.J., Gurnett, D.A., Morgan, D.D., Kirchner, D.L., 2008\.  Transient layers in the topside ionosphere of Mars.  Geophys. Res. Lett. 35, L17102.  doi:10.1029/2008GL034948.
  * Krasnopolsky (1993) Krasnopolsky, V.A., 1993.  Photochemistry of the Martian atmosphere (mean conditions).  ICARUS 101, 313--332.  doi:10.1006/icar.1993.1027.
  * Mahaffy et al. (2015) Mahaffy, P.R., Benna, M., King, T., Harpold, D.N., Arvey, R., Barciniak, M., Bendt, M., Carrigan, D., Errigo, T., Holmes, V., Johnson, C.S., Kellogg, J., Kimvilakani, P., Lefavor, M., Hengemihle, J., Jaeger, F., Lyness, E., Maurer, J., Melak, A., Noreiga, F., Noriega, M., Patel, K., Prats, B., Raaen, E., Tan, F., Weidner, E., Gundersen, C., Battel, S., Block, B.P., Arnett, K., Miller, R., Cooper, C., Edmonson, C., Nolan, J.T., 2015\.  The Neutral Gas and Ion Mass Spectrometer on the Mars Atmosphere and Volatile Evolution Mission.  Space Science Rev. 195, 49--73.  doi:10.1007/s11214-014-0091-1.
  * Matsuoka et al. (1981) Matsuoka, S., Nakamura, H., Tamura, T., 1981.  Ion-molecule reactions of N3+, N4+, O2+, and NO2+ in nitrogen containing traces of oxygen.  J. Chem. Phys. 75, 681--689.  doi:10.1063/1.442108.
  * McElroy et al. (2013) McElroy, D., Walsh, C., Markwick, A.J., Cordiner, M.A., Smith, K., Millar, T.J., 2013\.  The UMIST database for astrochemistry 2012.  A&A 550, A36.  doi:10.1051/0004-6361/201220465.
  * McElroy et al. (1976) McElroy, M.B., Kong, T.Y., Yung, Y.L., Nier, A.O., 1976\.  Composition and structure of the Martian upper atmosphere - Analysis of results from Viking.  Science 194, 1295--1298.  doi:10.1126/science.194.4271.1295.
  * Nier and McElroy (1977) Nier, A.O., McElroy, M.B., 1977\.  Composition and structure of Mars’ upper atmosphere \- Results from the neutral mass spectrometers on Viking 1 and 2.  J. Geophys. Res. 82, 4341--4349.  doi:10.1029/JS082i028p04341.
  * Norton and Barth (1970) Norton, R.B., Barth, C.A., 1970\.  Theory of nitric oxide in the Earth’s atmosphere.  J. Geophys. Res. 75, 3903\.  doi:10.1029/JA075i019p03903.
  * Raghuram and Bhardwaj (2020) Raghuram, S., Bhardwaj, A., 2020\.  CO+ first-negative band emission: A tracer for CO in the Martian upper atmosphere.  å , arXiv:2005.09473doi:10.1051/0004-6361/202038147, arXiv:2005.09473.
  * Scott et al. (1998) Scott, G.B.I., Fairley, D.A., Freeman, C.G., McEwan, M.J., Anicich, V.G., 1998.  Gas-phase reactions of some positive ions with atomic and molecular nitrogen.  J. Chem. Phys 109, 9010--9014.  doi:10.1063/1.477571.
  * Stevens et al. (2019) Stevens, M.H., Sisikind, D.E., Evans, J.S., Fox, J.L., Deighan, J., Jain, S.K., Schneider, N.M., 2019.  Detection of the Nitric Oxide Dayglow on Mars by MAVEN/IUVS.  J. Geophys. Res. 124, 1 -- 12.  doi:10.1029/2019JE005945.
  * Tachiev and Froese Fischer (2002) Tachiev, G.I., Froese Fischer, C., 2002\.  Breit-Pauli energy levels and transition rates for nitrogen-like and oxygen-like sequences.  A&A 385, 716--723.  doi:10.1051/0004-6361:20011816.
  * Vejby-Christensen et al. (1998) Vejby-Christensen, L., Kella, D., Pedersen, H.B., Andersen, L.H., 1998\.  Dissociative recombination of NO+.  Physical Rev. A 57, 3627--3634.  doi:10.1103/PhysRevA.57.3627.
  * Vogt et al. (2017) Vogt, M.F., Withers, P., Fallows, K., Andersson, L., Girazian, Z., Mahaffy, P.R., Benna, M., Elrod, M.K., Connerney, J.E.P., Espley, J.R., Eparvier, F.G., Jakosky, B.M., 2017\.  MAVEN observations of dayside peak electron densities in the ionosphere of Mars.  Journal of Geophysical Research (Space Physics) 122, 891--906.  doi:10.1002/2016JA023473.
  * Williamson et al. (2019) Williamson, H.N., Johnson, R.E., Leclercq, L., Elrod, M.K., 2019\.  Large amplitude perturbations in the Martian exosphere seen in MAVEN NGIMS data.  Icarus 331, 110--115.  doi:10.1016/j.icarus.2019.05.020.
  * Withers (2006) Withers, P., 2006.  Mars Global Surveyor and Mars Odyssey Accelerometer observations of the Martian upper atmosphere during aerobraking.  Geophys. Res. Lett. 33, L02201.  doi:10.1029/2005GL024447.


# A Personalized Diagnostic Generation Framework Based on Multi-source
Heterogeneous Data

Jialun Wu1, Zeyu Gao1, Haichuan Zhang1,2, Ruonan Zhang3, Tieliang Gong1,
Chunbao Wang4, and Chen Li1,∗ 1School of Computer Science and Technology,
Xi’an Jiaotong University, Xi’an, China
National Engineering Lab for Big Data Analytics, Xi’an Jiaotong University,
Xi’an, China  2School of Electrical Engineering and Computer Science,
Pennsylvania State University, University Park, PA, USA  3Library of Xi’an
Jiaotong University, Xi’an, China  4Department of Pathology, the First
Affiliated Hospital of Xi’an Jiaotong University, Xi’an, China
Email<EMAIL_ADDRESS>
###### Abstract

Personalized diagnoses have not been possible due to sear amount of data
pathologists have to bear during the day-to-day routine. This lead to the
current generalized standards that are being continuously updated as new
findings are reported. It is noticeable that these effective standards are
developed based on a multi-source heterogeneous data, including whole-slide
images and pathology and clinical reports. In this study, we propose a
framework that combines pathological images and medical reports to generate a
personalized diagnosis result for individual patient. We use nuclei-level
image feature similarity and content-based deep learning method to search for
a personalized group of population with similar pathological characteristics,
extract structured prognostic information from descriptive pathology reports
of the similar patient population, and assign importance of different
prognostic factors to generate a personalized pathological diagnosis result.
We use multi-source heterogeneous data from TCGA (The Cancer Genome Atlas)
database. The result demonstrate that our framework matches the performance of
pathologists in the diagnosis of renal cell carcinoma. This framework is
designed to be generic, thus could be applied for other types of cancer. The
weights could provide insights to the known prognostic factors and further
guide more precise clinical treatment protocols.

###### Index Terms:

multi-source heterogeneous data, personalized diagnostic result, prognostic
factor weight.

##  I Introduction

So far, the pathological diagnosis of cancer tissue sections has been made
based on general and standardized guidelines. However, precision medicine
would clearly benefit cancer patients since heterogeneity of the disease
indicates that patients with the same symptoms may have different underlying
illnesses—linked to different genetic mutations. With the increasing incidence
of cancer, we explored new ways to guide the diagnosis and prognosis of
cancer. In prognoses, survival prediction can be considered essential.
Accurately predicting the survival period of cancer patients is crucial to
treatment and rehabilitation planning for clinicians and the psychological
welfare of patients. There are widespread uses of medical text processing and
whole-slide images analysis [1], and automated computational histopathological
analysis systems have shown great promises in diagnosis in recent years. They
are being used in various cancers, including kidney cancer [2], breast cancer
[3], lung cancer [4], and prostate cancer [5]. The mentioned works are fueled
by a large amount of multi-modal heterogeneous data, including
histopathological whole-slide images (WSIs), pathology reports and such
specialized data sets which are being accumulated over the years. Information
extracted from images is close to genomic data [6]. Thus, we will utilize the
similarity of the images to find a better group of patients with more genetic
similarities. The group could then be used as a basis for prognosis factors
weight and survival analysis.

Histopathological images have long been central to cancer diagnosis, staging,
and prognosis, pathologists widely use them in clinical practice. In clinics,
the pathological diagnostic report is an important part of the overall
clinical diagnosis and the clinicians’ foundation for providing patients with
the best of care. However, the shortage of new and experienced pathologists is
common in modern pathology [7]. The diagnosis and treatment of some diseases
often lack precise pathological examination results and are only based on
speculation with incomplete evidence. It takes a significant amount of time to
train a pathologist and even longer for pathologists to become experienced and
diagnose different types of tumors.

The pathology report is an essential unstructured clinical document that is
mainly composed of textual data recorded by pathologists in natural language.
The reports often include necessary information on patients, macroscopic
specimens, and microscopic specimens. There are various pathological
characteristics in the reports—prognostic factors, for example, cancer type
and subtype, anatomic site, tumor size, histological grade, and the TNM stage.
The prognostic factors play a crucial role in cancer treatment planning. For
different patients, different factors have different influences on diagnosis
and prognosis. In this study, we will explore the importance of each
prognostic factor by its effects on the survival outcome of a personalized
group of patients based on their histopathological similarity.

The mentioned multi-source heterogeneous data can significantly improve the
performance of survival predictions. Therefore, methods for effectively
combining the multi-modal data to improve cancer survival prediction is
needed. In breast cancer, a recent study [8] has developed a multi-state
statistical model and used to simulate the different phases of breast cancer,
including local recurrence, distant recurrence, breast cancer-related death,
and other reasons. The model utilizes the patient’s age, tumor size, and the
level of clinical variables affecting the survival probability to predict the
individual risk of a recurrence. In pulmonary adenocarcinoma, the TNM stage is
the most important predictive factor. In the US, the Mayo Clinic uses a
scoring system that consisted of tumor stage, size, nuclear grade, and
necrosis to classifies clear cell carcinomas into low-risk, medium-risk, and
high-risk groups. Though the mentioned works can predict treatment outcomes,
none of which leverage a specialized group of patients to arrive at a more
precise prediction with a specialized group of patients. To tackle these
problems, we curate a set of pathology images with their corresponding
pathology reports and clinical reports from the TCGA (the cancer genome atlas)
project [9]. Since there are many cancer types in the TCGA data set, we scope
our work to the renal cell carcinoma (RCC), which is one of the most common
malignant tumors in adult kidney type [10]. It has three common histologic
subtypes: clear cell carcinoma (ccRCC), papillary carcinoma (pRCC), and
chromophobe carcinoma (chRCC) [11]. Each case in the TCGA Project has its
corresponding pathological report and clinical information. The reports are
made after pathologists observed the histopathological sections stained with
hematoxylin and eosin (H&E) under a microscope to make the diagnosis by
inspecting the morphological characteristics of histology slides [12]. With a
pathology report, clinicians can make informed and precise decisions about
treatments with an overall understanding of patients’ symptoms.

In this paper, we demonstrate that a more effective survival analysis of a
patient could be made based on a group of histologically similar patients,
curated via similar image search algorithms. We develope a predictive model
that automatically assigns weights to the prognostic factors of each patient,
enabling personalized pathology reports for individual patients. Furthermore,
the framework can assess personalized patients’ survival risk. We believe that
this is the first attempt to co-analyze histopathological features from whole-
slide images with prognostic factors from pathology reports to generate
personalized weights and survival risks. A new type of pathology reports,
personalized, that emerge from our framework could assist pathologists and
oncologists to understand the significance of each prognostic factors for each
patient. These weighted factors may also be useful for new pathologists’
training. The results show that the predictive performance was similar to that
given by pathologists with long working experiences. We believe that our
framework could be considered data-independent. Our work could fundamentally
alleviate the problem of long training cycles for pathologists, and further
research will help demonstrate that the same approach for other types of
cancer (such as lung cancer, prostate cancer, and breast cancer) can calculate
the weight of his target characteristics and predict his survival risk.

The organization of this paper is as follows: The second part discusses the
data set of pathological images and pathological reports and describes our
framework and experimental process in detail. In the third part, the results
section introduces the statistics of our data set and the performance of our
framework, and in the last two part, we discuss the experimental results and
limitations.

Figure 1: The workflow of the framework. (a) In order to search for the simlar
population group of the patient, the whole-silde images with stain
normalization of the correspongding patient are used as input. (b) Similar
image search function: The first method is content-based image retriveal using
deep learning, which used patches from whole-slide images as input and the
DeepRank network pretrained on ImageNet to get the similar images. The second
method is to extract the histological characteristics from the patches and
search for the similar image features, which correspond to the similar
patches. The two groups of similar patches obtained in two method were
integrated and corresponding to WSIs to further obtain similar WSIs
information, and similar patient population groups information. (c) Finding
pathological reports and clinical information of corresponding patients
according to similar patient population and using information extraction
techniques to extract structured data from descriptive pathology reports
according to the World Health Organization’s guideline. (d) Calculating
weights for prognostic factors and survival analysis based on prognosis
factors.

Figure 2: The top five similar images retrieval results of two different
regions. Method: (a) Deep Ranking network with deep learning method, (b)
Similar image search based on image features.

##  II Materials and Methods

In order to calculate the weight of different prognostic factors for each
patient, a group of population with similar histology is needed for each case.
We explore deep learning and computing image feature similarity two options
for establishing the groups. For both methods, the top 500 images similar to
the ground image are used as a proxy to obtain corresponding pathological
reports and clinical information. For the cases with valid clinical
information, the main methods we use for survival analysis of cancer patients
include Kaplan-Meier survival analysis curve [13], Cox proportional risk model
[14] to calculate the weights of each clinical feature. It is possible to find
similar tumor patient groups for each patient, and assign weights to the
pathological and clinical characteristics of the patients to more accurately
customize the weight of prognostic factors.

###  II-A Data sources and annotations

Based on TCGA project, we select kidney cancer from more than twenty types of
cancer in our study. TCGA collects histopathological images, detailed
pathological reports, and clinical information of patients with various cancer
types to jointly display tumor characteristics and results of patients. In
this paper, we collect histopathology image data, pathological diagnosis
report data, and clinical data from TCGA for kidney cancer, including clear
cell renal cell carcinoma, papillary renal cell carcinoma, and chromophobe
renal cell carcinoma. Specifically, we selected 3281 digital whole-slide
images from 943 patients and 894 pathology reports in renal cancer. By
matching images of the same patients with reports, we ended up with pathology
reports of 859 patients in renal cancer that could be matched with digital
pathology slides (two or more per person).

###  II-B The proposed framework

Fig. 1 depicts our data analysis workflow, with the following sections
describing the information for each step. OpenHI [15] is an open-source
annotation and processing software which specially designed for histopathology
images. The pathologists can perform routine operations such as area selection
and annotation on this software. Now, it supports several types of annotation
and can record the time of annotation. At the beginning of the introduction to
our framework, the normalization of haematoxylin and eosin stain was made at
the training and validation stages using a reference image and the Macenko
algorithm [16].

###  II-C Similar histopathological image search

Medical image retrieval technology is a process in which the underlying
physical features of the image to be retrieved (such as gray scale, texture,
shape, spatial position relation) are extracted, and then compared with the
feature vectors in the database according to the similarity measurement
method, and the preceding most similar image is returned to the user in the
order of decreasing similarity. In our similar pathological image search, we
use two methods: similar image search based on computing image feature
similarity and similar image search based on deep learning.

In the first method, we use the method based on Deep Ranking [17]. In the
process of similar image model training, we use three different images to be
transferred to the model. The first two images belong to the same category,
and the third image belongs to different categories. The model calculates the
embedding distance between each image pair in the three images and makes a
comparison. Through different categories of natural images in ImageNet [18],
the model we use learns to distinguish between similar and dissimilar images
by calculating and comparing the embedding distance of the input images. In
the query matching in our pathological image data set, the model first
calculates the embedding of the selected input image, and then calculates its
embedding value with other pathological images stored in the data set, and
then selects the first 500 images with the highest embedding distance as
similarity pathological image.

In the second method, we use interpretable image features to search for the
similar images. Our pathological image feature extraction process includes two
stages: nuclear segmentation and cellular features extraction. Usually,
pathologists will select 8-10 regions of approximately 512-by-512 pixel (40x
magnification) in a whole-slide image, and a single patient will be diagnosed
with multiple slides (normally 2-4 slides). Histopathological image consisted
of tumorous and normal regions. The selected sample region for the search did
not only contains tumor cells but also includes inflammatory cells in some
image areas in addition to tumor cells. Then we applied the supervised
segmentation algorithm (Hovernet) [19] to segment the nucleus from the image.
Subsequently, we extracted 31 histological image characteristics for each
divided nucleus according to the guideline for characteristics extraction
[20]. There contains 10 morphological characteristics, 5 intension-based
characteristics, and 16 texture-based characteristics, morphological
characteristics describe the shape and size changes of the nucleus. Intension-
based characteristics (first-order statistical features) describe the
distribution of color changes in the nucleus—mean, median, standard deviation,
skewness, and kurtosis; texture-based characteristics (second-order
statistical features) quantitatively describe patterns and texture of pixel
values: co-occurrence based features and run-length based features. Co-
occurrence based features include correlation, cluster shade, cluster
prominence, energy, entropy, Haralick correlation, inertia, and inverse
difference moment [21]. Run-length based features include gray-level non-
uniformity, run-length non-uniformity, low and high gray-level run emphasis,
short-run low and high gray-level emphasis, and long-run low and high gray-
level emphasis [22].

After we get the 31 different characteristics of each image. We compare the
feature of the search image with the feature of each patch in the data set to
calculate their distance. We adopt a method combining dynamic weight
adjustment with weighted Markov distance, modify the retrieval strategy,
improve the precision of retrieval, overcome the problem of attribute
correlation between feature vectors in Euclid distance calculation, and
improve the retrieval accuracy. We adopt the method of multi-texture feature
fusion to treat the feature vectors in the feature space differently according
to different weights.The weight can better reflect the importance of different
feature vectors to target retrieval. In the process of constant optimization
of query vectors, the previous query vectors are regarded as positive examples
and added to the current feedback positive examples to form a more complete
set. Meanwhile, the weight is dynamically adjusted until it becomes stable and
optimized. In order to improve the computational speed and reduce the
computational complexity, we use the FASII similarity search library proposed
by Facebook AI Lab [23], It includes the algorithm of researching in any size
vector set, which can improve the speed of searching similar pathological
images according to the pathological feature matrix. After obtaining similar
images by these two methods, we find which whole-slide image these patches
belong to and find the corresponding medical information of these patches in
turn according to the corresponding relationship between images, pathological
reports, and clinical reports, and convert them into structured text data.
Each feature and attribute value corresponding to each feature in the relevant
pathology report and clinical information can be obtained from structured
textual data.

###  II-D Histopathological prognostic factors extraction from pathology
reports

The selected TCGA pathology reports included 894 cases of renal cell carcinoma
are carefully annotated by pathologists. The content of annotations involve
several attributes: cancer type, tumor resection site preference, cancer
subtypes, maximum tumor diameter, histological grade, TNM stage, Description
of lymph node metastasis is essential in all cancers. For renal cancer images
and pathology report, we need to focus on whether the cancer areas invade the
renal veins; for lung cancer images and pathology reports, the key concerns
are whether tumor cells invade the pulmonary membrane and whether the tumor
spreads in the alveolar cavity. Based on two examples of biomedical
histopathology corpus annotations [24, 25], we designed an iterative
annotation workflow and revised our annotation guideline several times. All
annotations were done at the document level so that the annotators can
leverage the context in difficult cases. An open-source software called MAE
version 2.2.10 was used as the annotation tool throughout the entire process
[26]. We sorted out the annotation content of the pathology report and matched
them with the pathology pictures based on the same patient ID. We use
annotated pathology report data as the training set to learn unannotated
pathology reports using an end-to-end two-layer LSTM neural network based on
tagging strategy. We use the method we proposed [27] to extract the essential
information, when we converted unstructured pathology reports and clinical
information into structured data, due to incomplete pathology data, we
introduced the guidelines of the WHO Guidelines [28] to supplement and improve
our structured data, and support related structures in the standards text
description information is added to the clinical data.

###  II-E Statistical methods for prognosis prediction

We establish a univariate Kaplan-Meier curve model [29] for survival analysis
and a multivariate Proportional hazards model [30], according to the last
section of the similar image corresponding to the prognostic factors of the
query image and the characteristic value to calculate each patient’s risk
index and give weight to the characteristics of each patient, according to the
different characteristics of predict survival and prognosis of the patients.
In our experiment, the more accurate the prognosis prediction of the
attribute, the higher the corresponding weight of the attribute.

Regression Models in our Regression risk model, We assume that $T$ is time,
$h(t,X)$ is the risk rate when the current time is $T$, and the objective
factor is $X$, then:

$h(t,X)=\lambda_{0}(t)exp(\beta\cdot X)$ (1)

By logarithmic operation of the formula, we can get:

$ln(h(t,X))=\beta\cdot X+ln(\lambda_{0}(t))$ (2)

Suppose there are a total of $N$ events, the risk characteristic of the $ith$
event is $X^{(i)}$ , and the occurrence time is $t_{i}$, from which we get the
maximum likelihood function as follows:

$L(\beta)=\prod_{i=1}^{N}\frac{exp(\beta\cdot X^{(i)}}{\sum_{j:t_{j}\geq
t_{i}}exp(\beta\cdot X^{(j)}}$ (3)

The logarithmic likelihood function is:

$l(\beta)=lnL(\beta)=\sum_{i=1}^{N}[ln\beta\cdot X^{(i)}-ln(\sum_{j:t_{j}\geq
t_{i}}exp(\beta\cdot X^{(j)})]$ (4)

The gradient is:

$\frac{\partial
l(\beta)}{\partial\beta}=\sum_{i=1}^{N}[X^{(i)}-\frac{\sum_{j:t_{j}\geq
t_{i}}X^{(j)}\cdot exp(\beta\cdot X^{(j)}}{\sum_{j:t_{j}\geq
t_{i}}exp(\beta\cdot X^{(j)}}]$ (5)

Then we can use the gradient descent method to estimate our parameters.

TABLE I: Survival-Associated Prognostic Factors, Identified by The Kaplan-Meier Estimator and Log-Rank Test (P$<$0.05) Feature | P-Value | P/N | Feature | P-Value | P/N | Feature | P-Value | P/N
---|---|---|---|---|---|---|---|---
histological_type | 2.4023e-7 | P | ethnicity | 0.007210 | P | pathologic | 8.035e-43 | P
gender | 0.563 | N | hemoglobin_result | 0.000793 | P | TNM_stage | 1.5522e-47 | P
neoadjuvant_treatment | 0.015391 | P | Critical_diagnosis | 0.038198 | P | tumor_type | 0.039763 | P
laterality | 0.005815 | P | histologic_grade | 3.7118e-33 | P | white_cell_count | 0.028612 | P
neoplasm_cancer_status | 3.7698e-47 | P | platelet_qualitative | 6.6477e-15 | P | tissue_prospective | 0.142383 | N
lymphnode_invasion | 0.017860 | P | serum_calcium | 0.000309 | P | tissue_retrospective | 0.119677 | N

Figure 3: Prognostic factors predict the survival outcomes of ccRCC patients. Prognostic factors (a) and (b) are variables that have good predictive results. Prognostic factors (c) and (d) can poorly identify prognostic survival results. We still predict survival for multi-factor variables (e) and (f) TABLE II: Univariate and Multivariate Cox Proportional Hazards Analysis of the Prognostic Values of Lasso-cox Risk Index and Other Prognostic Factors. Hr, Hazard Ratio. Ci, Confidence Interval. | Univariate Cox regression | Multivariate Cox regression
---|---|---
Variable | HR (95% CI) | P value | HR (95% CI) | P value
Lasso-Cox | 1.320(0.937-2.308) | 0.001153 | 1.424(1.262-1.761) | 0.018915
TNM_stage | 1.433(1.219-1.684) | 0.000013 | 2.096(1.847-2.378) | 2.0432e-30
neoplasm_histologic_grade | 1.287(1.045-1.586) | 0.017610 | 2.324(1.966-2.747) | 4.8673e-23
person_neoplasm_cancer_status | 3.378(2.380-4.796) | 9.7962e-12 | 6.413(4.799-8.571) | 3.5311e-36
histological_type | 0.551(0.390-0.778) | 0.000713 | 0.497(0.384-0.644) | 1.1895e-7
serum_calcium_result | 0.767(0.583-1.009) | 0.005799 | 1.106(0.768-1.344) | 0.0010848
lymphnode_invasion | 0.538(0.325-0.891) | 0.015986 | 1.490(1.019-2.177) | 0.039508
laterality | 0.697(0.521-0.931) | 0.014553 | 0.673(0.507-0.893) | 0.006150
ethnicity | 0.241(0.089-0.654) | 0.005216 | 0.281(0.104-0.756) | 0.011934
gender | 0.706(0.524-0.950) | 0.021716 | 0.918(0.686-1.228) | 0.056347

Figure 4: Integrative analysis of prognosis factors from histopathological images using lasso-Cox can significantly improve the prognosis prediction TABLE III: Personalized Prognostic Factors Weights for Each Patient histologic_grade | -0.676 |  | Diagnosis_year | -0.608 |  | TNM_stage | -0.556 |  | Diagnosis_year | -0.581
---|---|---|---|---|---|---|---|---|---|---
TNM_stage | -0.593 |  | TNM_stage | -0.589 |  | Diagnosis_year | -0.544 |  | TNM_stage | -0.542
Diagnosis_year | -0.591 |  | histologic_grade | -0.561 |  | histologic_grade | -0.525 |  | histologic_grade | -0.488
calcium_result | -0.356 |  | Tumor_type | -0.294 |  | calcium_result | -0.352 |  | Critical_diagnosis* | -0.345
Critical_diagnosis* | -0.142 |  | calcium_result | -0.240 |  | lymphnode_invasion | -0.352 |  | Tumor_type | -0.243
Gender | -0.079 |  | lymphnode_invasion | -0.091 |  | Critical_diagnosis* | -0.037 |  | lymphnode_invasion | -0.207
lymphnode_invasion | -0.034 |  | Laterality | 0.020 |  | whitecell_count | 0.172 |  | calcium_result | -0.175
whitecell_count | 0.082 |  | Gender | 0.087 |  | Laterality | 0.205 |  | Gender | -0.162
Laterality | 0.098 |  | whitecell_count | 0.097 |  | hemoglobin_result | 0.267 |  | whitecell_count | 0.310
platelet_qualitative | 0.291 |  | Ethnicity | 0.240 |  | histologic_type | 0.303 |  | platelet_qualitative | 0.325
Ethnicity | 0.323 |  | histologic_type | 0.336 |  | platelet_qualitative | 0.405 |  | Ethnicity | 0.348

(a) ccRCC grade2

(b) pRCC type2

(c) chRCC

(d) pRCC type1

Figure 5: Patients typical pathology images with select patches correlated to
the prognosis factors weights above.

##  III Result

###  III-A Similar image retrieval

In order to find similar population groups of cancer patients, we use deep
learning method and image feature-based distance method to retrieve similar
images. Taking into account differences in the cancer population, we conduct
experiments on two separate data sets. The example retrieval results of the
two methods are shown in Fig. 2.

We select the top five most similar pictures from the results as an example
and first made a coarse-grained judgment on the similar pictures obtained,
respectively, from the subtype and cancer grade. From the example of the
results we give, the first image patch is clear cell carcinoma with grade 2.
There is almost no other subtypes and non-cancer regions in the corresponding
two results, and similar images that rank in the top rarely appear with grade
1 and grade 3 cancer cells. The second image is of type 2 papillary renal cell
carcinoma. We search the subtype and corresponding type of each result and
found that the subtype rarely has detection errors because the number of
images corresponding to papillary is limited. There are also some type1
papillary tumor tissue areas. Then, the pathologists retrieve their
corresponding medical information such as age, medical history, to determine
whether these similar images can be classified as similar tumor populations.
These two methods have precise control over the cell morphology, staining
degree and cell distribution of our images, indicating that we have relatively
better results in image retrieval of similar tumor patches.

###  III-B Analysis of a single prognostic factor

In order to study which individual specific prognostic factor is closely
related to the survival and prognosis of patients, we first sort each feature
according to the size of the feature value, and distinguished the high-risk
group from the low-risk group by the median of the feature value. We test the
statistical significance of overall survival differences between high-risk
groups and low-risk groups. The log-rank test showed that 25 prognostic
factors were significantly related to the prognosis (p¡0.05). For each
prognostic factor, patients are divided into different groups with the median
as the dividing point. For P/N, P indicates a positive correlation with
survival (i.e., patients with higher characteristic values have a good
prognosis), and N indicates a negative correlation with survival. The log test
results of all survival-related variables are listed in Table 1. The Kaplan-
Meier survival curves of some variables are shown in Figure 3. We select the
similar cancer groups of each patient through similar pathological images.
Based on similar groups’ information, we predict the survival rate of patients
in turn for each prognostic factor closely related to the prognosis of
patient.

###  III-C Integrating prognostic factors analysis to predict prognosis

In the previous section, we show a number of individual prognostic factors
derived from similar patients searched through histopathological images, thus
stratifying patients with different prognoses. Next, we investigate whether
the integration of all identified survival-related characteristics would
provide better prognostic outcomes. We establish a Lasso-Cox proportional
hazards model to select the most informative factors and calculate the risk
index for each patient. According to the risk index, patients were divided
into low-risk or high-risk groups according to the median. Compared to the use
of individual functions, the Lasso-Cox model provides significantly better
patient stratification. Among the 25 prognostic factors related to survival,
seven prognostic factors were selected: pathological metastasis, serum
calcium, primary lymph node, platelet qualitative, cancer status, laterality,
hemoglobin. Most of the pairwise mutual information values between them are
smaller than those between significantly prognostic factors (Fig. 3c), which
shows that the patient’s multiple prognostic factors are complementary in
predicting survival outcomes.

Using univariate and multivariate Cox proportional hazards analysis, we need
to clarify that the Lasso-Cox risk index is independent of known prognostic
factors [31](Table 2). We perform a comprehensive comparison between the
lasso-Cox risk index and other known prognostic characteristics, including
nine features with a low correlation coefficient(Fig. 4). Among these
characteristics, the univariate Cox proportional hazard model only correlates
them with survival predictions, and the multivariate Cox proportional hazard
analysis shows that our Lasso-Cox is an independent prognostic factor.

###  III-D Personalized prognostic factors weights customized for patients

After analyzing the results of similar images, map the results to the
corresponding pathological and clinical reports, and extract their prognostic
factors. In the previous sections, we analysis the importance and independence
of single-factor and integrating-factor prognostic factors for survival
prognosis. We carry out correlation analysis on patients’ prognostic factors
and used regression to assign different weights to each feature.

In the results, we select three patients with different kidney cancer subtypes
and grades, select a pathological image of each of them (Fig. 5), and assigned
weights to their prognostic factors. We selected some positive correlation and
some negative correlation. The weight with high coefficients are listed in
Table 3. From our results, we can see that cancer grade, TNM staging, and
cancer diagnosis time are the three most important factors, which are in line
with the judgment standards of clinicians and pathologists. In addition to
these three significant factors, other key diagnostic information of cancer
also have different weights in different patient reports, such as whether
tumor cells invade the renal vein, renal sinus, and renal capsule in renal
cell carcinoma, whether the degree of calcium ion enriches in the tumor area,
whether tumor cells invade the qualitative results of lymph nodes or
platelets, and hemoglobin results. These characteristics, without exception,
affect the patient’s survival prognosis information. From the brief weighing
results we gave, we can also find that different cancer subtypes have
different weight distributions. For example, the weight of tumor type of
papillary cell carcinoma is higher than that of other subtypes; the tumor area
of chromophobe cell carcinoma is calcified. His corresponding weight is also
higher. We discussed all the weights we got with clinicians and pathologists
and got their confirming that the weights of prognosis factors we get are
clinically meaningful.

##  IV Discussion

During the single-feature regression analysis, we find that the following
prognostic factors are ranked first: cancer neoplasm status, histological
type, and grade. Using multivariate Cox regression, we found that some factors
such as gender and laterality do not influence the survival outcomes, while
some other factors such as the TNM stage and neoplasm histologic grade do have
an influence on survival. At the overall view, the framework users shall
understand that man-made factors are more highly associative because they are
engineered in such ways. The gist of providing these weights gives insights
into the amount of relative importance among the prognostic factors.

Improvements to the framework could still yield a more appropriate set of
patients or personalized populations with similar histology. One way is
curating a larger set of data with both histopathological images with
associating pathological reports and medical information from various
databases. The challenging aspect of establishing a larger and more diverse
data set involves different data standards and interoperability. A common
medical data structure for generating pathological reports from pathological
images could be developed based on this work.

It is quite clear that genomic-related diseases such as cancer could be more
effectively treated with precision medicine practices. Embeded in each
histopathological image, there are random visual information and extracted
pathological analyses and many omics (e.g. genomics, proteomics,
transcriptomics) and analytical computational biology characteristics.
Integrating these “hard-to-interpret” information into our personalized
pathology report could yield a more precise diagnosis and prognosis. One way
to deploy the proposed technique to end-users such as pathologists oncologists
is to integrate the prognostic factor weights with digital pathology reports
to the existing digital pathology platform such as OpenHI. Oncologists would
be able to plan more precise treatment plans according to the personalized
prognostic factors with the help of the automatically predicted weights.
Further work will include the fusion of omics data with images and text data
for a more detailed multi-factor weight analysis of patients, which would help
inexperienced pathologists or intern doctors to have a deeper understanding of
pathology reports through this function. At the same time, clinicians can
modify the weight of prediction in the interactive interface. The system may
adaptively learn and correct itself since all predicted values will be saved
to improve the prediction model and interactive interface.

##  V Conclusion

To our knowledge, this is the first work that jointly analyzed
histopathological images with pathological reports to calculate weights of
prognostic factors and predict survival outcomes based on a personalized group
of histologically similar patients. The proposed framework could analyzed
whole-slide images to search for similar cases. Then, the model calculated a
personalized a set of weights to be attached to prognostic factors. In
addition, a robust prognostic model was built to also predict personalized
survival outcomes of patients using these two types of data.

The kidney cancer data from TCGA Project was used in framework development and
testing and validate the robustness of our model. In addition to renal cell
carcinoma, our model can be applied to other cancers in the future. We
believed that the proposed approach can be applied to other types of cancer
for generating personalized prognostic factor weights. An interactive form of
this framework would greatly benefit pathologists and oncologists in the
future. By expanding the robustness of our model, complete and accurate
pathologically assisted diagnosis can be made, effectively helping
pathologists in daily tasks and specific cases.

## Acknowledgment

This work has been supported by National Natural Science Foundation of China
(61772409); The consulting research project of the Chinese Academy of
Engineering (The Online and Offline Mixed Educational Service System for “The
Belt and Road” Training in MOOC China); Project of China Knowledge Centre for
Engineering Science and Technology; The innovation team from the Ministry of
Education (IRT_17R86); and the Innovative Research Group of the National
Natural Science Foundation of China (61721002). The results shown here are in
whole or part based upon data generated by the TCGA Research Network:
https://www.cancer.gov/tcga.

## References

  * [1] J. Hipp, T. Flotte, J. Monaco, J. Cheng, A. Madabhushi, Y. Yagi, J. Rodriguez-Canales, M. Emmert-Buck, M. C. Dugan, S. Hewitt _et al._ , “Computer aided diagnostic tools aim to empower rather than replace pathologists: Lessons learned from computational chess,” _Journal of pathology informatics_ , vol. 2, 2011.
  * [2] Z. Gao, P. Puttapirat, J. Shi, and C. Li, “Renal cell carcinoma detection and subtyping with minimal point-based annotation in whole-slide images,” in _International Conference on Medical Image Computing and Computer-Assisted Intervention_. Springer, 2020, pp. 439–448.
  * [3] A. H. Beck, A. R. Sangoi, S. Leung, R. J. Marinelli, T. O. Nielsen, M. J. Van De Vijver, R. B. West, M. Van De Rijn, and D. Koller, “Systematic analysis of breast cancer morphology uncovers stromal features associated with survival,” _Science translational medicine_ , vol. 3, no. 108, pp. 108ra113–108ra113, 2011.
  * [4] K.-H. Yu, C. Zhang, G. J. Berry, R. B. Altman, C. Ré, D. L. Rubin, and M. Snyder, “Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features,” _Nature communications_ , vol. 7, no. 1, pp. 1–10, 2016.
  * [5] E. Arvaniti, K. S. Fricker, M. Moret, N. Rupp, T. Hermanns, C. Fankhauser, N. Wey, P. J. Wild, J. H. Rueschoff, and M. Claassen, “Automated gleason grading of prostate cancer tissue microarrays via deep learning,” _Scientific reports_ , vol. 8, no. 1, pp. 1–11, 2018.
  * [6] K.-H. Yu, G. J. Berry, D. L. Rubin, C. Re, R. B. Altman, and M. Snyder, “Association of omics features with histopathology patterns in lung adenocarcinoma,” _Cell systems_ , vol. 5, no. 6, pp. 620–627, 2017.
  * [7] D. M. Metter, T. J. Colgan, S. T. Leung, C. F. Timmons, and J. Y. Park, “Trends in the us and canadian pathologist workforces from 2007 to 2017,” _JAMA network open_ , vol. 2, no. 5, pp. e194 337–e194 337, 2019.
  * [8] O. M. Rueda, S.-J. Sammut, J. A. Seoane, S.-F. Chin, J. L. Caswell-Jin, M. Callari, R. Batra, B. Pereira, A. Bruna, H. R. Ali _et al._ , “Dynamics of breast-cancer relapse reveal late-recurring er-positive genomic subgroups,” _Nature_ , vol. 567, no. 7748, pp. 399–404, 2019.
  * [9] K. Tomczak, P. Czerwińska, and M. Wiznerowicz, “The cancer genome atlas (tcga): an immeasurable source of knowledge,” _Contemporary oncology_ , vol. 19, no. 1A, p. A68, 2015.
  * [10] G. Kovacs, M. Akhtar, B. J. Beckwith, P. Bugert, C. S. Cooper, B. Delahunt, J. N. Eble, S. Fleming, B. Ljungberg, L. J. Medeiros _et al._ , “The heidelberg classification of renal cell tumours,” _The Journal of Pathology: A Journal of the Pathological Society of Great Britain and Ireland_ , vol. 183, no. 2, pp. 131–133, 1997.
  * [11] B. Ljungberg, K. Bensalah, S. Canfield, S. Dabestani, F. Hofmann, M. Hora, M. A. Kuczyk, T. Lam, L. Marconi, A. S. Merseburger _et al._ , “Eau guidelines on renal cell carcinoma: 2014 update,” _European urology_ , vol. 67, no. 5, pp. 913–924, 2015.
  * [12] A. H. Fischer, K. A. Jacobson, J. Rose, and R. Zeller, “Hematoxylin and eosin staining of tissue and cell sections,” _Cold spring harbor protocols_ , vol. 2008, no. 5, pp. pdb–prot4986, 2008.
  * [13] E. Bair and R. Tibshirani, “Semi-supervised methods to predict patient survival from gene expression data,” _PLoS Biol_ , vol. 2, no. 4, p. e108, 2004.
  * [14] H. M. Bøvelstad, S. Nygård, H. L. Størvold, M. Aldrin, Ø. Borgan, A. Frigessi, and O. C. Lingjærde, “Predicting survival from microarray data—a comparative study,” _Bioinformatics_ , vol. 23, no. 16, pp. 2080–2087, 2007.
  * [15] P. Puttapirat, H. Zhang, Y. Lian, C. Wang, X. Zhang, L. Yao, and C. Li, “Openhi-an open source framework for annotating histopathological image,” in _2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)_. IEEE, 2018, pp. 1076–1082.
  * [16] M. Macenko, M. Niethammer, J. S. Marron, D. Borland, J. T. Woosley, X. Guan, C. Schmitt, and N. E. Thomas, “A method for normalizing histology slides for quantitative analysis,” in _2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro_. IEEE, 2009, pp. 1107–1110.
  * [17] J. Wang, Y. Song, T. Leung, C. Rosenberg, J. Wang, J. Philbin, B. Chen, and Y. Wu, “Learning fine-grained image similarity with deep ranking,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2014, pp. 1386–1393.
  * [18] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in _2009 IEEE conference on computer vision and pattern recognition_. Ieee, 2009, pp. 248–255.
  * [19] S. Graham, Q. D. Vu, S. E. A. Raza, A. Azam, Y. W. Tsang, J. T. Kwak, and N. Rajpoot, “Hover-net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images,” _Medical Image Analysis_ , vol. 58, p. 101563, 2019.
  * [20] J. J. Van Griethuysen, A. Fedorov, C. Parmar, A. Hosny, N. Aucoin, V. Narayan, R. G. Beets-Tan, J.-C. Fillion-Robin, S. Pieper, and H. J. Aerts, “Computational radiomics system to decode the radiographic phenotype,” _Cancer research_ , vol. 77, no. 21, pp. e104–e107, 2017.
  * [21] R. M. Haralick, K. Shanmugam, and I. H. Dinstein, “Textural features for image classification,” _IEEE Transactions on systems, man, and cybernetics_ , no. 6, pp. 610–621, 1973.
  * [22] M. M. Galloway, “Texture analysis using grey level run lengths,” _NASA STI/Recon Technical Report N_ , vol. 75, p. 18555, 1974.
  * [23] J. Johnson, M. Douze, and H. Jégou, “Billion-scale similarity search with gpus,” _IEEE Transactions on Big Data_ , 2019.
  * [24] H. Gurulingappa, A. M. Rajput, A. Roberts, J. Fluck, M. Hofmann-Apitius, and L. Toldo, “Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports,” _Journal of biomedical informatics_ , vol. 45, no. 5, pp. 885–892, 2012.
  * [25] A. Roberts, R. Gaizauskas, M. Hepple, G. Demetriou, Y. Guo, I. Roberts, and A. Setzer, “Building a semantically annotated corpus of clinical texts,” _Journal of biomedical informatics_ , vol. 42, no. 5, pp. 950–966, 2009.
  * [26] K. Rim, “Mae2: Portable annotation tool for general natural language use,” in _Proc 12th Joint ACL-ISO Workshop on Interoperable Semantic Annotation_ , 2016, pp. 75–80.
  * [27] J. Wu, K. Tang, H. Zhang, C. Wang, and C. Li, “Structured information extraction of pathology reports with attention-based graph convolutional network,” in _2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)_. IEEE, 2020, pp. 2395–2402.
  * [28] F. Galateau-Salle, A. Churg, V. Roggli, W. D. Travis, W. H. O. C. for Tumors _et al._ , “The 2015 world health organization classification of tumors of the pleura: advances since the 2004 classification,” _Journal of thoracic oncology_ , vol. 11, no. 2, pp. 142–154, 2016.
  * [29] B. Efron, “Logistic regression, survival analysis, and the kaplan-meier curve,” _Journal of the American statistical Association_ , vol. 83, no. 402, pp. 414–425, 1988.
  * [30] J. P. Fine and R. J. Gray, “A proportional hazards model for the subdistribution of a competing risk,” _Journal of the American statistical association_ , vol. 94, no. 446, pp. 496–509, 1999.
  * [31] J. Cheng, J. Zhang, Y. Han, X. Wang, X. Ye, Y. Meng, A. Parwani, Z. Han, Q. Feng, and K. Huang, “Integrative analysis of histopathological images and genomic data predicts clear cell renal cell carcinoma prognosis,” _Cancer research_ , vol. 77, no. 21, pp. e91–e100, 2017.