|  Potential energy surface study of X@Si32X${}_{44}^{-}$(X=Cl, Br) clusters
to decipher the stabilization process of Si20 fullerene
---|---
|  Deb Sankar De,∗a Santanu Saha,b and Stefan Goedeckera
|  Efforts toward stabilization of the Si20 fullerene through different
schemes have failed despite several theoretical predictions. However, recently
Tillmann et. al. succeeded to stabilize the Si20 fullerene through exohedral
decoration with eight Cl substituents and twelve SiCl3 groups on the surface
and enclosing Cl- ion. A deeper understanding on what factors lead to
stabilization will open the path for stabilizing other systems of interest.
Here, we employ the minima hopping method within density functional theory to
understand the potential energy surface. The study shows that the exo-endo
halide decoration of the cage alters the glassy nature of the potential energy
surface of pure cage to structure seeker. Further analysis of different
properties of the global minima, reveal that the extra electron instead of
residing on the central encapsulated atom in the cage, it is distributed on
the cage and increases the encapsulation energy; thereby stabilizing the
system. We also provide estimates of the stability for different kind of exo-
endo halide decorations and their feasible realization in experiments.

##

††footnotetext: a Department of Physics, Universität Basel, Klingelbergstr.
82, 4056 Basel, Switzerland; E-mail<EMAIL_ADDRESS>Tel: +41 61 207
3738††footnotetext: b Institute of Theoretical and Computational Physics, Graz
University of Technology, NAWI Graz, 8010 Graz, Austria

##  1 Introduction

The discovery of C60 fullerene in 1985 marks an important milestone in nano-
sciences 1. Since then, many experimental and theoretical efforts had been
made to find fullerene structures made out of non-carbon materials for various
applications 2, 3, 4. Lately, boron based fullerence structure, borospherene
B40 was discovered by Zhai et al. in 2014 5. As both C and Si both are Group-
IV elements without any d orbitals, the Si structures are expected to be
similar to their carbon counterparts. Few examples include linear polysilanes,
silicon nanosheets and nanotubes 6, 7, 8, 9, 10.

Since C20 forms the smallest known fullerene, Si20 clusters were widely
studied 11. While C atoms can easily adjust their valence states to
participate single, double and triple bonds, Si strongly favors $sp^{3}$
hybridization in connection with single bonds and this inherent constraint
leads to the instability of the Si20 fullerene 12.

Endohedral doping of Si20 cages with metal atoms, M@Si20 was believed for a
while to stabilize the cage like geometries. However systematic structure
prediction studies show that this strategy fails 13, 14. The second approach
was based on the introduction of exhohedral substituents to fully saturate all
four silicon valencies. The Si20H20 dodecahedron configuration has been
identified as the global minima 15, 16 in this context. As a third approach, a
combination of both techniques was considered which should lead to a stable
M@Si20H20 where M=metal/halide 17, 18, 19.

The embedding energy of the endohedral dopants in the hydrogenated cages was
found to be smaller than for the bare endohedrally doped fullerene due to weak
interaction between the dopant and the cage 20. Other than that, such surface
passivated endohedrally doped fullerenes proposed in theoretical calculations
have very weak interactions among each other and might therefore be stable
building blocks for novel materials. It was also predicted that the halide
ions, especially Br-, are ideally suited for the synthesis of Si20H20 18.
However, no such system have been observed in experiments till date.

In our recent study on Si20H20 21, we found that its potential energy
surface(PES) is complex. The spontaneous formation of Si20H20 by condensation
is unlikely to occur because the time scale for finding the global minimum is
much longer then the time scale for competing processes such as fragmentation
or fusion.

Recently, Tillmann et. al. have developed a one step synthesis protocol by
which the Si20 dodecahedral core can be stabilized and hence, realized in
experiments. The resulting structure is the $[\text{Cl}@\ce{Si32Cl44}]^{-}$,
which consists of a Si20 dodecahedral core with an endohedral Cl- ion and
exohedrally decorated with 8 Cl and 12 SiCl3 groups 22. Their DFT studies on
different possible exohedral decoration with Cl and SiCl3 of the Si20 cage
structure provides valuable information about the role of the exohedral
decoration on the energetics of the synthesized structure. Later, Vargas et.
al.23 investigated the role of the central halide ion in stabilizing the
system based on energy decomposition and multipole analysis.

The study of PES can provide estimate of the synthesizability of a desired
compound, the barrier height required to cross and time-scale to synthesize
the compound. As explained earlier in Ref.21, why it is not feasible to
synthesize Si20H20 and opposite for C60. Thus, the PES is equally important as
the properties of the synthesized structure. The two studies Ref. 22 and Ref.
23 have focussed primarily on the cage structure of Cl@Si32Cl44and the role of
either exohedral or endohedral substituents in stabilizing the system.
However, the role of the PES in making the synthesis of
$[\text{Cl}@\ce{Si32Cl44}]^{-}$ feasible have not been explored yet. Apart
from PES, the studies do not address the impact of different combination of
exohedral-endohedral decoration of Si20 cage and the extra electron in its
stabilization.

In this work, the minima hopping method 24 was used to explore the potential
energy surface(PES) of Si20 with different endohedral-exohedral decorations.
The runs generated in total 500 distinct structures, which can be classified
into fragmented and non-fragmented structures. Distance-energy plots
constructed for these structures were used to understand the nature of the
PES. In addition, a detailed analysis of the charge density, stability and
reactivity of the ground state configuration was conducted. Based on a model
of the central halide ion trapped in a spherical potential well, we elucidate
the role of exo-endo decorations along with negative charge in the
stabilization process.

##  2 Computational Methodology:

In order to explore the potential energy surface(PES) of the system of
interest, the minima hopping method (MHM) was employed 24, 25. The PES is
explored by performing consecutive short molecular dynamics escape steps
followed by local geometry relaxations, thereby exploiting the Bell-Evans-
Polanyi principle 26. The relaxed structures are accepted or rejected based on
the threshold set on the energy difference.The transformation pathways were
found by the minima hopping guided path search (MHGPS) 27.

To identify the structural difference between two configurations and removing
the duplicate structures, we have used fingerprint method based on overlap
matrix constructed from atom centered Gaussian type orbitals 28. More details
on the method can be found in the appendix.

All the ab-initio simulation have been performed at the level of density
functional theory (DFT) as implemented in BigDFT 29. This code uses Daubechies
wavelets as a basis set. The atoms were described by the soft Goedecker-type
norm-conserving pseudopotentials with non-linear core correction 30, 31, 32,
33 for the Perdew-Burke-Ernzerhof(PBE) exchange correlation functional 34. The
Libxc library 35 coupled with BigDFT was used to evaluate the PBE functionals.
A grid spacing of 0.4 Bohr was used. Convergence parameters in BigDFT were set
such that total energy differences were converged up to 10-4 eV and all
configurations were relaxed until the maximal force component on any atom
reached the noise level of the calculation, which was of the order of 1 meV/Å.

The difference in charge density($\Delta\rho$) and electron localization
function(ELF) 36 was calculated using the vasp code 37. The atoms were
described by the Projector Augmented Wave potentials as provided in vasp for
the PBE functional. In order to mimic the systems under study in free boundary
condition, a uniform box of dimension $20\times 20\times 20$ Å3 was used. This
is the minimum box size required to prevent self interaction between the
clusters. For geometric optimization, a energy cut-off of 350 eV was used and
for ELF calculation 600 eV.

##  3 Results and Discussion:

Fig. 1: The configurational energy spectrum of (a) [Cl@Si32Cl44]- and (b)
[Br@Si32Br44]-, along with the figures of the ground state configuration and
next 3 lowest energy structures. The Si, Cl and Br atoms are shown by blue,
green and brown sphere, respectively.

###  3.1 Structure, Geometry and Energetics

A natural starting point of our investigation is the [Cl@Si32Cl44]- symmetric
cage structure, experimentally realized by Tillmann et. al. 22 in 2015. This
[Cl@Si32Cl44]- symmetric structure with D2h symmetry is made of a Si20
dodecaheadral core consisting of 20 vertices forming 12 pentagons. Among the
20 vertices, 12 are capped by SiCl3 groups arranged in pairs and forming the
vertices of an octahedron and the remaining 8 are capped with Cl atom. The
structure also has an endohedrally doped Cl- ion. The Si atoms of the cage
capped with Cl are represented as Si0 and SiCl3 decorated Si as Si1.

To rapidly perform an exhaustive configurational search of the PES of
[Cl@Si32Cl44]-, we carried out MHM runs starting from different
configurations; the reported experimental structure and few new configurations
were created by exchanging the position of the decorative Cl atoms and SiCl3
groups. Starting from these different configurations and after removing the
similar structures found in different MHM runs through fingerprint distance,
we were left with $\sim$500 distinct structures. These structures lie in the
energy range 0 eV $\leqslant$ $\Delta$E $\leqslant$ 17 eV w.r.t. the lowest
energy structure of [Cl@Si32Cl44]-.

We also carried out investigation on the PES of
$[\text{Br}@\ce{Si32Br44}]^{-}$. The 500 structures obtained for
$[\text{Cl}@\ce{Si32Cl44}]^{-}$ were used as templates for generating
structures of $[\text{Br}@\ce{Si32Br44}]^{-}$ by replacing the Cl atoms with
Br and post-relaxing them. As Cl and Br are chemically similar, the
$[\text{Br}@\ce{Si32Br44}]^{-}$ structures easily relaxed without significant
distortion and are in the energy range 0 eV $\leqslant$ $\Delta$E $\leqslant$
12 eV w.r.t. the lowest energy structure of $[\text{Br}@\ce{Si32Br44}]^{-}$.

The low energy structures generally consist of Si20 dodecahedron with diverse
arrangement of the Cl and SiCl3 groups on its vertices. However, the high
energy structures consist of mostly fragmented structures along with distorted
cage structures. In most cases among the fragmented structures, the SiCl3 gets
fragmented to form a free SiCl2 with the remaining Cl attached to the Si atom
of the cage. This feature of structural motif vs energetic ordering appear to
be common in both $[\text{Cl}@\ce{Si32Cl44}]^{-}$ and
$[\text{Br}@\ce{Si32Br44}]^{-}$.

Based on this two distinct structural motif, the whole structure set can be
classified into fragmented and non-fragmented structures. The fragmented
structures start to appear at $\Delta$E = 1.48 eV and $\Delta$E = 0.96 eV for
$[\text{Cl}@\ce{Si32Cl44}]^{-}$ and $[\text{Br}@\ce{Si32Br44}]^{-}$
respectively. Above this energy range, both fragmented and non-fragmented
structures can be found. Beyond $\Delta$E $\geqslant$ 4 eV, only the
fragmented structures can be found in both [Cl@Si32Cl44]- and [Br@Si32Br44]-.

The low-energy spectrum of $[\text{Cl}@\ce{Si32Cl44}]^{-}$ and
$[\text{Br}@\ce{Si32Br44}]^{-}$, which consists of primarily non-fragmented
structures is shown in Fig. 1(a) and Fig. 1(b) respectively w.r.t. their
corresponding lowest energy structure. Along with this, their 4 lowest energy
structures are shown in Fig. 1. Among the predicted [Cl@Si32Cl44]-1 clusters,
the experimentally observed cage structure with D2h symmetry was found to be
the lowest energy structure, where the central Cl- ion is located at the
center of the cage. This theoretical observation also holds true for the
[Br@Si32Br44]- clusters. The distance between the central Cl- ion and Si from
the Si20 dodecahedron cage is $\sim$3.28-3.39 Å which is comparable to the
experimental values of 3.32-3.38 Å. In case of the central Br- ion in the
[Br@Si32Br44]- cage structure, the distance is 3.31-3.40 Å. Thus, the central
halide ion to cage distance is similar in both cases.

The other low energy structures have different arrangements of Si0 and Si1 on
pentagons of the cage. Tillmann et. al.22 through DFT calculation on different
decoration of Si20 cage with increasing number of SiCl3 units have found that
the ideal arrangement would be that in the pentagon, two consecutive vertices
are to be occupied by SiCl3 units, their immediate neighbouring vertices by Cl
atom and the remaining vertice by a SiCl3 group. Deviation from this symmetric
arrangement of Si1 was found to cost energy of the structures. This is in
agreement with our observations.

The second lowest energy isomer of $[\text{Cl}@\ce{Si32Cl44}]^{-}$ and
$[\text{Br}@\ce{Si32Br44}]^{-}$ are 0.25 eV and 0.37 eV higher than their
ground state respectively. This isomer has one non-ideal pentagon consisting
of 4 SiCl3 groups and 1 Cl atoms. The third lowest has two non-ideal pentagons
and the fourth lowest with three non-ideal pentagons. Thus, shifting away from
the ideal pentagonal arrangement for Cl and SiCl3 leads to an increase in
energy. However, with more non-ideal pentagons, the situation gets complicated
as there are many possible decorative arrangements on these pentagons and
their relative energetic ordering is not straight forward to understand. At a
certain point in high energy spectrum, the cage structure starts to release
SiCl2 with the vertices replaced by Cl atoms. The very high energy structures
have completely destroyed cages and free SiCl4 molecules.

The structures of $[\text{Cl}@\ce{Si32Cl44}]^{-}$ and
$[\text{Br}@\ce{Si32Br44}]^{-}$ are found to have intermediate to large HOMO-
LUMO gaps. Our calculations of 10 lowest energy structures of
$[\text{Cl}@\ce{Si32Cl44}]^{-}$ and $[\text{Br}@\ce{Si32Br44}]^{-}$ at the
level of semilocal DFT are found to be in the range 1.7-2.0 eV and 2.0-2.4 eV
respectively. No specific trends have been found among the HOMO-LUMO gap of
different structures. The HOMO-LUMO gaps can be found in the SI.

Fig. 2:  Distance energy (DE) plot of $[\text{Cl}@\ce{Si32Cl44}]^{-}$(open
orange square) and $[\text{Br}@\ce{Si32Br44}]^{-}$(open blue circle). The
energy and the fingerprint distance is w.r.t the lowest energy configuration
for each case. For any structure one can find another structure that is more
similar to the ground state and that is in nearly all cases lower or in a few
cases slightly higher in energy. Hence, on gains quasi continuously energy by
moving toward the ground state. The arrow is meant as a guide to the eye,
indicating the average driving force.

###  3.2 Fingerprint Distance-Energy

Apart from the energy ranking of the predicted structures, we also tried to
seek insight into the nature of the PES through fingerprint-distance(DE) vs
energy 38 plots. In such a plot, the energy difference($\Delta$E) is plotted
against the fingerprint distance w.r.t. the ground state structure.

The DE plots of both $[\text{Cl}@\ce{Si32Cl44}]^{-}$(orange open square) and
$[\text{Br}@\ce{Si32Br44}]^{-}$(blue open circles) as shown in Fig.7 are found
to be similar as the energy separation increases steeply with the increase in
fingerprint distance. Their structure seeker character can be easily deduced
from the fact that the first few metastable structures are much higher in
energy than the ground state, but not too far in the configurational distance.
This suggests that the barrier for jumping from the first metastable structure
into the ground state is relatively small and that there is, in general, a
strong driving force towards the ground state. The fragmented structures have
been removed while constructing the distance-energy plot.

In contrast, the structures of Si20 are found to have small $\Delta$E w.r.t.
the putative global minima as shown in SI. Details about the generation of
Si20 structures are provided in the SI. Also, Si20 structures with similar
energies are found with both small and large fingerprint distance. This points
to the fact that one can easily get trapped in one of the local minima and
would need to cross barriers of different heights to fall into the global
minima. These features are characteristics of a glassy system. This
comparative analysis reveals the interesting fact that the exo-endo decoration
of Si20 not only stabilizes the dodecahedron cage but also alters the PES in
such a manner that it changes from glassy to structure seeker.

Fig. 3: The transformation path from the first metastable structure of
$[\text{Cl}@\ce{Si32Cl44}]^{-}$ to the $[\text{Cl}@\ce{Si32Cl44}]^{-}$ ground
state. Dark blue sphere represent the Si atoms and light blue, the Si atoms of
Si20 cage undergoing change in their decoration. The Cl atoms of SiCl3 are
omitted for better visualization. The Si atom of SiCl3 swapping to another Si
atom of Si20 cage during the transformation path is shown by pink sphere. Rest
all the Cls are by green atoms.

###  3.3 Transition State

The downhill barrier for transiting from the first metastable state to ground
state of $[\text{Cl}@\ce{Si32Cl44}]^{-}$ is about 1.6 eV as shown in Fig.3.
The barrier height is quite high compared to kBT at room temperature (which
corresponds to 0.027 eV). The uphill and downhill barriers are similar with a
difference of 0.4 eV. In this process, the SiCl3 dissociate into SiCl2 and Cl.
The Cl atom stays on top of Si and the SiCl2 migrates to the nearest Si0 site
and forms SiCl3. Standard transition state theory 39 gives an attempt
frequency of $\omega_{A}=k_{B}T/\hslash=6\times 10^{12}\leavevmode\nobreak\
sec^{-1}$ at room temperature. Hence the time required to cross the uphill
barrier is 1015 sec at room temperature.

In order to asses the possibility and reaction time at higher temperature, we
estimated the Boltzmann probabilities as a function of temperature for the 10
lowest energy structures of $[\text{Cl}@\ce{Si32Cl44}]^{-}$ and
$[\text{Br}@\ce{Si32Br44}]^{-}$. The details of the calculation of the
Boltzmann probability plot and the figure (Fig. 1 in appendix) is provided in
appendix. The Boltzmann probabilities show that the ground state structure of
$[\text{Br}@\ce{Si32Br44}]^{-}$ is dominant until $\sim$1200 K and for
$[\text{Cl}@\ce{Si32Cl44}]^{-}$ until $\sim$800 K. At higher temperatures,
other low energy structures start to emerge.

Hence, considering 800 K as the upper limit, the time required for transition
at 800 K is around 0.2 msec. Since, the experimental synthesis was carried out
in a solvent medium the actual barrier height could be considerably lower than
our calculated height.

###  3.4 Bonding and ELF Analysis

Fig. 4: a) Charge density difference between neutral and charged Cl@Si32Cl44.
b) ELF of [Cl@Si32Cl44]- at $\eta$=0.88 . The Si and Cl atoms are shown by
blue and green sphere, respectively.

In order to investigate where the extra electron added to the [Cl@Si32Cl44] is
localized, the $\Delta\rho$ between the $[\text{Cl}@\ce{Si32Cl44}]^{-}$ and
[Cl@Si32Cl44] is shown in Fig 4(a) where, the yellow colour indicates the
presence of extra electronic charge and blue electron deficiency. The plot
shows that when an electron is added, it doesn’t reside on the central Cl
atom. Rather, the Si as well as Cl atoms of the Si0 decoration on the cage
gain some electronic charge at the expense of losing electronic charge from Si
atoms with Si1 decoration. There is no significant difference in the charge
density of the Cl in the SiCl3 groups in Si1 decoration.

The ELF of $[\text{Cl}@\ce{Si32Cl44}]^{-}$ at isosurface value of 0.88 is
shown in Fig 4(b). The circular lobes on the Cl atoms indicate the presence of
lone pair electrons on them.

###  3.5 Dissociation and Coalescence

The analysis carried out above on the PES and on ground state configuration of
$[\text{Cl}@\ce{Si32Cl44}]^{-}$ doesn’t necessarily shed light on its
reactivity. In order to understand its reactive/inert nature, we accessed if
it is stable against dissociation or coalescence.

Fragmentation of $[\text{Cl}@\ce{Si32Cl44}]^{-}$ has been observed in the
laser-desorption ionization(LDI) MS(-) experiments 22. As mentioned earlier,
the SiCl3 group fragments to form a free SiCl2 molecule and the remaining Cl
atom gets adsorbed on the Si atom of the Si20 cage. A major fragmentation
cascade is observed due to the extrusions of such SiCl2 groups. Similar
structures also were found in our MHM search. Through our DFT calculation, we
found that the ground state configuration is found to be local minimum and
stable under dissociation of one SiCl2 and two SiCl2 groups with dissociation
energy 1.49 and 2.70 eV, respectively. The dissociation energy is defined as
the difference between the cluster and the individual components of the
fragmented parts of the cluster under vacuum condition.

The issue of coalescence of the ground minimum of
$[\text{Cl}@\ce{Si32Cl44}]^{-}$ and $[\text{Br}@\ce{Si32Br44}]^{-}$ was
addressed by estimating the energy required to form dimers. The dimers were
created by bringing the ground state configuration in proximity for three
different relative orientation (see appendix) and relaxing them without any
constraint. It turns out that both the $[\text{Cl}@\ce{Si32Cl44}]^{-}$ and
$[\text{Br}@\ce{Si32Br44}]^{-}$ are chemically inert and repel each other, as
indicated by their positive binding energy of dimer. The binding energy of the
dimer is defined as the difference between the energy of the dimer and the
energy of the monomers. The factor leading to repulsion is the presence of
lone pair electrons on peripheral Cl atoms as shown in ELF analysis(Fig. 4).

###  3.6 Exohedral and Endohedral Substituents

The complex dependency of different exohedral and endohedral decorations along
with the extra electron in stabilizing the Si20 cage makes their understanding
challenging. Vargas et. al. 23 through energy decomposition analysis tried to
decipher the role of substituents and suggested alternative halide based
substituents which can stabilize the cage. Though such analysis are useful,
they are complicated and often not easy to decipher.

The central halide ion in the cage can be considered to be a particle trapped
in a spherical potential well. Considering this simple picture, one can easily
assimilate the combined effect of the exohedral/endohedral substituents and
the additional electron to a single quantity i.e., the curvature of the
potential.

For our tests we considered [Cl@Si32Cl44]-1, [Cl@Si32Cl44], [Br@Si32Cl44]-1,
[Cl@Si20H20]-1 and [Br@Si32Br44]-1 cases. The radius of the cage is different
for different decorations. The change in energy $\Delta$E(eV) w.r.t.
displacement(Å) of the central atom of the cage shows that the potential of
the charged and neutral case are on top of each other for all the systems,
giving clear indication that the extra electron on the structure does not
effect the stability of the center atom (see appendix). The curvature of the
systems with central Br ion is larger than the ones with central Cl ion. This
is due to the large ionic radius of Br as compared to Cl confined in the cage
structure of similar diameter. The large ionic radius leads to steep increase
in potential w.r.t. displacement. Hence, the potential is influenced by the
substituents and not by the extra electron.

As evident from above, the spring constant K of the potential for
[Cl@Si32Cl44] remains approximately same for both the neutral and charged
system. However, the encapsulation energy(E.E.) increases by factor of
$\sim$1.84 for charged system as compared to neutral case. The extra
stabilization due to addition of an electron is also supported by the large
electron affinity. This behavior holds true for other endo-exo decoration of
Si20 dodecahedron considered. The electron affinity depends only on the
exohedral substituents. The values for different decorations are provided in
the SI. Thus, the extra electron doesn’t lead to increase in interaction
between the central atom and the cage, rather goes to the cage and stabilizes
it.

##  4 Conclusion

In conclusion, through exploration of the PES of
$[\text{Cl}@\ce{Si32Cl44}]^{-}$ and $[\text{Br}@\ce{Si32Br44}]^{-}$, we found
that the experimentally observed symmetric cage structure is the putative
global minima. The exo-endo decoration of Si20 cage with Cl and SiCl3
transforms the nature of PES from glassy to structure seeker. At room
temperature the symmetric cage structure is the most viable isomer during
synthesis. However, at higher temperature, other isomers would start to
emerge. The ground state minimum is stable against dissociation and
coalescence. Our stability analysis suggests that the experimental efforts
could be oriented towards the synthesis of the hypothetical counterpart
$[\text{Br}@\ce{Si32Br44}]^{-}$ of $[\text{Cl}@\ce{Si32Cl44}]^{-}$.

The $\Delta\rho$ shown in Fig.4 coupled with the central atom displacement
study modeled into a particle trapped in a spherical potential well clearly
indicate that the extra electron instead of residing on the central halide
atom, gets distributed on the cage and stabilizes it by increasing the inter-
atomic interaction within the cage. rather between the central atom and the
cage. As a result of this, the extra electron leads to significant increase in
encapsulation energy, thereby making the combined system more stable.

## Conflicts of interest

There are no conflicts to declare.

## Acknowledgements

D.S.D acknowledge support from the Swiss National Science Foundation.
Computational resources provided by the Swiss National Supercomputing Center
(CSCS) in Lugano under the projects s707 and s963 are gratefully acknowledged.
Calculations were also performed at the sciCORE (http://scicore.unibas.ch/)
scientific computing core facility at the University of Basel. This was
supported by the NCCR MARVEL, funded by the Swiss National Science Foundation.

##  Appendix A Finger print method:

The configurational fingerprints are given by the eigenvalues of an overlap
matrix sadeghi.

$O_{ij}=\int\phi_{i}^{l}{(\boldsymbol{r})}\phi_{i}^{l^{\prime}}{(\boldsymbol{r})}d{\boldsymbol{r}}$
(1)

where $\phi{i}$ are the Gaussian type orbitals centered on the atom at
position $\boldsymbol{r}_{i}$ and is given by

$\phi_{i}^{l}{(\boldsymbol{r})}\propto(x-x_{i})^{l_{x}}(y-y_{i})^{l_{y}}(z-z_{i})^{l_{z}}\exp(-\alpha_{i}|\boldsymbol{r}-\boldsymbol{r}_{i}|)^{2})$
(2)

Here $\boldsymbol{l}$=($l_{x},l_{y},l_{z}$) is aa angular momentum
L=$l_{x}+l_{y}+l_{z}$. The orbitals are classified depending on their value of
L i.e.$s$-type orbital is L=0, $p$-type orbital is L=1, or $d$-type orbital is
L=3. $\alpha_{i}$ is the orbital width and they are inversly proportional to
the covalent radius of the atoms on which the orbitals are centered on. The
structural difference is given by the root mean square difference of the two
fingerprint vectors. This fingerprint distance is invariant under
translations, rotations, reflections as well as under the permutation of the
atomic indices.

Fig. 5: The Boltzmann probability distribution of 5 lowest energy structures
of [Cl@Si32Cl44]- (bottom panel)and The Boltzmann probability have been
estimated for 10 lowest energy structures.

Fig. 6: Dimers are constructed by adding two GS configurations along X, Z and
X-Y directions. Here we show the structures which are formed by adding the GS
configurations along X and X-Y direction.

Fig. 7: Distance Energy (DE) plot of Si20 clusters. The finger print distances and energy differences are calculated with respect to the ground state configuration. There is no driving force towards the ground state. System | Gap in Angstroem
---|---
Si32Cl45- | 6.736
Si32Cl45 | 6.787
Si32Cl44Br- | 6.749
Si32Br45- | 6.679
Si20H20Cl- | 6.629
Table 1: Diameter of the gap in the X-Y plane.

##  Appendix B Exohedral and Endohedral Substituents

Based on this simplistic picture of a particle trapped in a 1D-potential, we
estimated the spring constant of the quadratic potential. The potential can be
easily constructed by obtaining the energy difference as a function of
displacement, where the reference energy is the energy of the geometrically
relaxed structure. During the displacement of the central atom, the whole
structure is kept fixed and only scf runs were done to obtain the total
energy. In addition we also calculated encapsulation energy (E.E.) and
electron affinity (E.A.). Encapsulation energy is given by:

$\text{E.E.}=\text{E${}_{system}$}-\text{E${}_{cage}$}-\text{E${}_{central\
atom}$}$ (3)

where Esystem is the energy of the whole system cage + center anion(neutral or
charged), Ecage energy of the empty cage without the central atom and
E${}_{central\ anion}$ energy of the central halogen (neutral or charged
state). Electron affinity (E.A.) is given by

$\text{E.A.}=\text{E${}_{neutral\ system}$}-\text{E${}_{charged\ system}$}$
(4)

where E${}_{charged\ system}$ is the energy of the charged cage and
E${}_{neutral\ system}$ is the energy of the charged cage.

The E.E., spring constant K and E.A. of these systems are listed in Table.2.
As seen above in the Fig.8 and also from the K values, they are similar for
charged and neutral states. The E.E. sheds light on the role of the extra
electron, i.e. it reduces the total energy, making it more stable and is also
supported by the large E.A. Its also interesting to note that the E.A. is not
effected by the type of central halide ion, but rather by the exohedral
decoration.

Table 2: Encapsulation Energy (eV), Harmonic oscillator strength K (eV/Å2) and electron affinity E.A.(eV) for different decorations of Si20 dodecahedron. System | E.E. (eV) | K (eV/Å2) | E.A.
---|---|---|---
| Neutral | Charge | Neutral | Charge | (eV)
Cl@Si32Cl44 | -2.34 | -4.38 | 2.16 | 2.08(X,Y,Z) | 5.68
Br@Si32Cl44 | -1.87 | -4.15 | 2.91 | 2.82 | 5.68
Cl@Si32Br44 | -2.48 | -4.28 | 1.89 | 1.90 | 5.43
Br@Si32Br44 | -2.03 | -4.07 | 2.62 | 2.60 | 5.44
Cl@Si20H20 | -0.07 | -0.10 | 2.70 | 2.69 | 0.16
Br@Si20H20 | -0.05 | -0.09 | 3.82 | 3.81 | 0.16

Fig. 8: Change in total energy $\Delta$E(eV) as a function of displacement
($\Delta$x) in Å of the center atom for different combination of exohedral and
endohedral decoration in the Si20 dodecahedron cage. The reference energy for
each case was the total energy of the ground state structure and the reference
length for the displacement was the radius of the cage for corresponding
decoration. The cage structure was kept fixed during the displacement of the
center atom.

## Notes and references

  * Kroto _et al._ 1985 H. W. Kroto, J. R. Heath, S. C. O’Brien, R. F. Curl and R. E. Smalley, _nature_ , 1985, 318, 162–163.
  * Song _et al._ 2017 W. Song, C. Querebillo, R. Götz, S. Katz, U. Kuhlmann, U. Gernert, I. Weidinger and P. Hildebrandt, _Nanoscale_ , 2017, 9, 8380–8387.
  * Yang _et al._ 2015 Z. Yang, Y.-L. Ji, G. Lan, L.-C. Xu, X. Liu and B. Xu, _Solid State Communications_ , 2015, 217, 38–42.
  * Johansson _et al._ 2004 M. P. Johansson, D. Sundholm and J. Vaara, _Angewandte Chemie_ , 2004, 116, 2732–2735.
  * Zhai _et al._ 2014 H.-J. Zhai, Y.-F. Zhao, W.-L. Li, Q. Chen, H. Bai, H.-S. Hu, Z. A. Piazza, W.-J. Tian, H.-G. Lu, Y.-B. Wu _et al._ , _Nature chemistry_ , 2014, 6, 727–731.
  * Gibson _et al._ 1985 H. W. Gibson, R. J. Weagley, R. A. Mosher, S. Kaplan, W. M. Prest and A. J. Epstein, _Phys. Rev. B_ , 1985, 31, 2338–2342.
  * Novoselov _et al._ 2005 K. S. Novoselov, A. K. Geim, S. V. Morozov, D. Jiang, M. I. . Katsnelson, I. V. Grigorieva, S. V. Dubonos and A. A. Firsov, _Nature_ , 2005, 438, 197–200.
  * Boul _et al._ 1999 P. Boul, J. Liu, E. Mickelson, C. Huffman, L. Ericson, I. Chiang, K. Smith, D. Colbert, R. Hauge, J. Margrave and R. Smalley, _Chemical Physics Letters_ , 1999, 310, 367–372.
  * Miller and Michl 1989 R. D. Miller and J. Michl, _Chemical Reviews_ , 1989, 89, 1359–1410.
  * Okamoto _et al._ 2010 H. Okamoto, Y. Kumai, Y. Sugiyama, T. Mitsuoka, K. Nakanishi, T. Ohta, H. Nozaki, S. Yamaguchi, S. Shirai and H. Nakano, _Journal of the American Chemical Society_ , 2010, 132, 2710–2718.
  * Li and Cao 2000 B.-x. Li and P.-l. Cao, _Physical Review A_ , 2000, 62, 023201\.
  * Ho _et al._ 1998 K.-M. Ho, A. A. Shvartsburg, B. Pan, Z.-Y. Lu, C.-Z. Wang, J. G. Wacker, J. L. Fye and M. F. Jarrold, _Nature_ , 1998, 392, 582–585.
  * Willand _et al._ 2010 A. Willand, M. Gramzow, S. A. Ghasemi, L. Genovese, T. Deutsch, K. Reuter and S. Goedecker, _Physical Review B_ , 2010, 81, 201405.
  * Sun _et al._ 2002 Q. Sun, Q. Wang, T. Briere, V. Kumar, Y. Kawazoe and P. Jena, _Physical Review B_ , 2002, 65, 235417.
  * Earley 2000 C. W. Earley, _The Journal of Physical Chemistry A_ , 2000, 104, 6622–6627.
  * Zdetsis 2007 A. D. Zdetsis, _Physical Review B_ , 2007, 76, 075402.
  * Zhang _et al._ 2005 C.-Y. Zhang, H.-S. Wu and H. Jiao, _Chemical physics letters_ , 2005, 410, 457–461.
  * Pichierri _et al._ 2005 F. Pichierri, V. Kumar and Y. Kawazoe, _Chemical physics letters_ , 2005, 406, 341–344.
  * Palagin and Reuter 2012 D. Palagin and K. Reuter, _Physical Review B_ , 2012, 86, 045416\.
  * Kumar and Kawazoe 2007 V. Kumar and Y. Kawazoe, _Physical Review B_ , 2007, 75, 155425\.
  * De _et al._ 2020 D. S. De, B. Schaefer, B. von Issendorff and S. Goedecker, _Physical Review B_ , 2020, 101, 214303.
  * Tillmann _et al._ 2015 J. Tillmann, J. H. Wender, U. Bahr, M. Bolte, H.-W. Lerner, M. C. Holthausen and M. Wagner, _Angewandte Chemie_ , 2015, 127, 5519–5523.
  * Ponce-Vargas and Muñoz Castro 2018 M. Ponce-Vargas and A. Muñoz Castro, _The Journal of Physical Chemistry C_ , 2018, 122, 12551–12558.
  * Goedecker 2004 S. Goedecker, _The Journal of chemical physics_ , 2004, 120, 9911–9917.
  * Schaefer _et al._ 2015 B. Schaefer, S. A. Ghasemi, S. Roy and S. Goedecker, _The Journal of chemical physics_ , 2015, 142, 034112.
  * Roy _et al._ 2008 S. Roy, S. Goedecker and V. Hellmann, _Physical Review E_ , 2008, 77, 056707.
  * Schaefer _et al._ 2014 B. Schaefer, S. Mohr, M. Amsler and S. Goedecker, _The Journal of chemical physics_ , 2014, 140, 214102.
  * Sadeghi _et al._ 2013 A. Sadeghi, S. A. Ghasemi, B. Schaefer, S. Mohr, M. A. Lill and S. Goedecker, _The Journal of chemical physics_ , 2013, 139, 184118.
  * Genovese _et al._ 2008 L. Genovese, A. Neelov, S. Goedecker, T. Deutsch, S. A. Ghasemi, A. Willand, D. Caliste, O. Zilberberg, M. Rayson, A. Bergman _et al._ , _The Journal of chemical physics_ , 2008, 129, 014109.
  * Hartwigsen _et al._ 1998 C. Hartwigsen, S. Gœdecker and J. Hutter, _Physical Review B_ , 1998, 58, 3641.
  * Goedecker _et al._ 1996 S. Goedecker, M. Teter and J. Hutter, _Physical Review B_ , 1996, 54, 1703.
  * Krack 2005 M. Krack, _Theoretical Chemistry Accounts_ , 2005, 114, 145–152.
  * Willand _et al._ 2013 A. Willand, Y. O. Kvashnin, L. Genovese, Á. Vázquez-Mayagoitia, A. K. Deb, A. Sadeghi, T. Deutsch and S. Goedecker, _The Journal of chemical physics_ , 2013, 138, 104109.
  * Perdew _et al._ 1996 J. P. Perdew, K. Burke and M. Ernzerhof, _Physical review letters_ , 1996, 77, 3865.
  * Marques _et al._ 2012 M. A. Marques, M. J. Oliveira and T. Burnus, _Computer physics communications_ , 2012, 183, 2272–2281.
  * Silvi _et al._ 1994 B. Silvi, A. Savin _et al._ , _Nature_ , 1994, 371, 683–686.
  * Kresse and Furthmüller 1996 G. Kresse and J. Furthmüller, _Physical review B_ , 1996, 54, 11169\.
  * De _et al._ 2014 S. De, B. Schaefer, A. Sadeghi, M. Sicher, D. G. Kanhere and S. Goedecker, _Phys. Rev. Lett._ , 2014, 112, 083401.
  * Eyring 1935 H. Eyring, _The Journal of Chemical Physics_ , 1935, 3, 107–115.


# Advancing Eosinophilic Esophagitis Diagnosis and Phenotype Assessment with
Deep Learning Computer Vision

William Adorno III1, Alexis Catalano2,3, Lubaina Ehsan 3, Hans Vitzhum von
Eckstaedt3, Barrett Barnes4, Emily McGowan5, Sana Syed4∗, and Donald E.
Brown6∗
1Dept. of Engineering Systems and Environment, University of Virginia,
Charlottesville, VA, USA
2College of Dental Medicine, Columbia University, New York City, NY, USA
3School of Medicine, University of Virginia, Charlottesville, VA, USA
4Department of Pediatrics, School of Medicine, University of Virginia,
Charlottesville, VA, USA
5Department of Medicine, University of Virginia, Charlottesville, VA, USA
6School of Data Science, University of Virginia, Charlottesville, VA, USA
∗Co-corresponding Authors
{ss8xj<EMAIL_ADDRESS>

###### Abstract

Eosinophilic Esophagitis (EoE) is an inflammatory esophageal disease which is
increasing in prevalence. The diagnostic gold-standard involves manual review
of a patient’s biopsy tissue sample by a clinical pathologist for the presence
of 15 or greater eosinophils within a single high-power field (400$\times$
magnification). Diagnosing EoE can be a cumbersome process with added
difficulty for assessing the severity and progression of disease. We propose
an automated approach for quantifying eosinophils using deep image
segmentation. A U-Net model and post-processing system are applied to generate
eosinophil-based statistics that can diagnose EoE as well as describe disease
severity and progression. These statistics are captured in biopsies at the
initial EoE diagnosis and are then compared with patient metadata: clinical
and treatment phenotypes. The goal is to find linkages that could potentially
guide treatment plans for new patients at their initial disease diagnosis. A
deep image classification model is further applied to discover features other
than eosinophils that can be used to diagnose EoE. This is the first study to
utilize a deep learning computer vision approach for EoE diagnosis and to
provide an automated process for tracking disease severity and progression.

##  1 INTRODUCTION

Eosinophilic Esophagitis (EoE) is a chronic allergen/immune disease that
occurs when eosinophils, a type white blood cell, concentrate in the
esophagus. It occurs in roughly 0.5 - 1.0 in 1,000 people and is found in 2 -
7% of patients who undergo endoscopy for any reason [Dellon, 2014]. EoE is
believed to be triggered by dietary components in patients and is increasing
in prevalence [Carr et al., 2018]. Clinical symptoms include swallowing
difficulties, food impaction, and chest pain [Runge et al., 2017]. Persistent
esophageal inflammation can eventually progress to strictures in the esophagus
with the need for interventional procedures such as dilations, which can
significantly impact the quality of patients’ lives. The most characteristic
microscopic pathologic feature used to diagnose EoE is intraepithelial
eosinophil inflammation. For diagnosis, patients with clinical symptoms
concerning for EoE undergo an endoscopy and the collected biopsy tissue
samples are then evaluated for presence of eosinophils. The accepted criterion
for pathologists to diagnose EoE involves identifying at least one High-Power
Field (HPF; $400\times$ magnification adjustment) within a patient’s tissue
biopsy slide that contains 15 or more eosinophils [Furuta et al., 2007].

Due to the workload required, pathologists typically only collect information
required for diagnosis rather than extensively counting and characterizing
eosinophil presence for entire biopsy Whole-Slide Images (WSI). Furthermore,
gastroenterologists are currently unable to predict a patient’s risk of EoE
progression, their clinical phenotype, or the most appropriate treatment plan
using available baseline biopsies. Manual quantitative measurements of mean
counts of eosinophils in several or all HPF in esophageal biopsies are
currently used for research purposes. [Dellon et al., 2014, Godwin et al.,
2020]. Due to the difficulty of manual evaluation of slides, computer
automation of detecting and counting eosinophils can efficiently assist
pathologists for not only diagnosing EoE and determining severity, but also
pave the way for clinically meaningful linkages with EoE clinical and
treatment phenotypes [Reed et al., 2018].

In this paper, we present a complete approach to create and validate an
automated eosinophil detection model. Development of the model required
annotated data representing the true locations of eosinophils within biopsy
image patches. Medically-trained technicians manually annotated images to
support the approach due to a lack of publicly available Hematoxylin and Eosin
(H&E) stained biopsy images with annotated eosinophils. With these
annotations, a deep learning Convolutional Neural Network (CNN) model was
trained to predict the location of eosinophils on new images. The U-Net
architecture, which has been originally designed for biomedical image
segmentation, is well-suited for eosinophil detection [Ronneberger et al.,
2015]. After the segmentation model was developed, the prediction masks were
post-processed to develop and extract eosinophil counts and statistics. These
statistics were then used to quantify and describe EoE severity and its
phenotype in patients.

These generated statistics can also reveal linkages between features within
initial biopsies and how they correspond with the eventual clinical phenotype
and optimal treatment plan. Currently, neither a patient’s subsequent clinical
phenotype nor the most effective treatment plan can be assessed at the time of
initial biopsy and diagnosis. Due to this knowledge gap, patients often
undergo algorithmic trials of various treatments before the most suitable
treatment is found. The eventual goal of this research is to recommend
treatment plans with a higher likelihood of effectiveness and to assess the
risk of developing a certain EoE clinical phenotype at the time of initial
diagnosis. The various treatment plans include: steroid responsiveness, dairy
elimination, and 4-to-6 food elimination. EoE severity and extent statistics
are also assessed to find initial biopsy linkages with clinical phenotypes:
inflammatory vs. stricturing vs. Proton Pump Inhibitor-responsive esophageal
eosinophilia (PPI-REE). Even though PPI is a type of treatment, there have
been studies describing a difference in underlying immune and antigen-driven
responses in patients with PPI-REE vs. the rest due to which have included it
as a separate clinical phenotype [Liacouras et al., 2011, Wilson and McGowan,
2018].

While there are some other known pathologic features in EoE biopsies, there is
currently only a manual, categorical scoring system available [Collins et al.,
2017]. To explore this we also applied a classification Convolutional Neural
Network (CNN) using WSI patches from EoE-diagnosed and histologically normal
patients. This analysis revealed that EoE diagnosis can be highly accurate
even when examining small areas of tissue that do not contain a large amount
of eosinophils. Both applications of deep learning, image segmentation and
classification, provide an increased understanding of EoE and how tissue
characteristics can impact diagnosis, severity, treatment plans, and clinical
phenotypes. These models combined with an influx of patient response data will
pave the way for improved patient outcomes.

Table 1: Table of Featured Acronyms Acronym | Definition
---|---
CNN | Convolutional Neural Network
EoE | Eosinophilic Esophagitis
FED | Food Elimination Diet
GPU | Graphics Processing Unit
Grad- | Gradient-weighted
CAM | Class Activation Mapping
H&E | Hematoxylin and Eosin
HPF | High-Power Field
PPI | Proton Pump Inhibitor
REE | Responsive Esophageal Eosinophilia
WSI | Whole Slide Image

##  2 BACKGROUND

Previous research for automated eosinophil detection focused on the image
classification of these cells, among others, in blood specimens [Liang et al.,
2018]. In contrast, the presence and appearance of eosinophils in H&E stained
biopsy tissue samples is different from those present in the blood since they
are embedded with the tissue and are surrounded by various other cellular
structures with overlapping color variations and gradients. Objects are freely
floating in blood images where there are stark differences between the white
and red blood cells vs. uni-colored background. The strategy for prediction of
white cell blood types is to isolate a single cell per patch and perform image
classification [Habibzadeh et al., 2018, Özyurt, 2019]. This approach is
difficult for large WSIs since it will require processing thousands of small
patches for detecting and counting each eosinophil. White blood cell
segmentation has previously also been executed using only 42 cropped images
via a SegNet model, which achieved high accuracy [Tran et al., 2018].

Cell segmentation has been shown to be a critical component of linking biopsy
features with disease diagnosis and outcomes. Nuclei segmentation recently
increased in popularity with publicly available datasets such as Kaggle’s 2018
Data Science Bowl and the Multi-organ Nuclei Segmentation Challenge [Kumar et
al., 2019]. Eosinophils have unique features such as a bi-lobed nuclei and
cytoplasm filled with red to pink granules [Rosenberg et al., 2012]. The
complexity of this segmentation is why a deep learning approach is preferred
over simpler techniques. The U-Net CNN architecture was originally designed
for biomedical image segmentation to produce high-resolution segmentation
masks without requiring a large amount of training data [Ronneberger et al.,
2015]. U-Nets are fully-convolutional models that contain a series of
convolutional blocks to contract and then expand the image to output a
segmentation mask of the same dimensions as the input. Improvements to the
original U-Net architecture were developed over the past few years including
Residual U-Net (Res. U-Net). It incorporates residual blocks into the
architecture to increase the depth of the model while still propagating
information quickly [Zhang et al., 2018]. RU-Net and R2U-Net models also
introduced recurrent convolutional layers to improve feature representation
[Alom et al., 2018]. Attention gates were further incorporated into U-Nets for
highlighting important features that pass through skip connections [Oktay et
al., 2018].

There are various different approaches for the segmentation model loss
function with the two major classes being per-pixel or per-image metrics. It
has been recommended to train the model using the same metric as the one used
for evaluation rather than combining per-pixel metrics with per-image metrics
in the loss function [Eelbode et al., 2020]. Per-image metrics such as region-
based techniques tend to perform better with mild class imbalance [Sudre et
al., 2017]. In our case, there was a large imbalance of more background pixels
than foreground (annotated eosinophils). Dice coefficient loss has been shown
to be a popular region-based approach with successful results for class
imbalanced problems [Milletari et al., 2016].

For image classification, state-of-the-art architectures have been developed
through the ImageNet Large Scale Visual Recognition Challenge [Russakovsky et
al., 2015]. The VGG16 architecture is not the most recent CNN innovation, but
it has shown to perform effectively and has large image dimensions in the last
convolutional layer compared to most other ImageNet models [Simonyan and
Zisserman, 2015]. The dimensions of the last convolutional layer can impact
the resolution of the Gradient-weighted Class Activation Mapping (Grad-CAMs).
Grad-CAMs are visual explanations of CNNs that highlight the important regions
of the image using the gradients that flow into the last convolutional layer
[Selvaraju et al., 2017]. Grad-CAMs have a similar appearance to heat maps
overlaid on an input image and are used to identify cellular-level features
that are important for diagnosis models. Grad-CAMs are typically used to
confirm whether the features being utilized by the models for classification
are relevant rather than being extraneous parts of images.

##  3 METHODOLOGY

In this section, we discuss the methods for the image segmentation of
eosinophils and image classification for EoE diagnosis.

###  3.1 Image Segmentation of Eosinophils

####  3.1.1 Data Generation

The WSIs were obtained via digitization of archived biopsy slides present at
our center for patients who had already been diagnosed with EoE. This was done
via scanning of biopsy tissue slides using a Hamamatsu NanoZoomer S360 Digital
slide scanner C13220 with the scanner setting being maintained at $40\times$
magnification. A pool of 274 512 $\times$ 512 patches were generated from WSIs
of seven different EoE patients. This image size provided a reasonable
annotation area for outlining eosinophils, while also providing a practical
input size for segmentation models. A total of 1,037 eosinophils were
annotated by a biology trainee who closely supervised by a gastroenterologist
and a pathologist. Annotations were done via manually creating polygons around
eosinophils using Aperio Imagescope. These patches were smaller than the HPF
(conversion of HPF to pixels explained below) used for EoE diagnosis, but the
model was still able to detect eosinophils within an HPF using a sliding
window approach. An HPF is of $400\times$ magnification or $40\times$
objective and $10\times$ ocular lenses.

During real-time histopathological diagnosis using microscopes, the
$400\times$ HPFs appear circular and have an area of 0.21mm2 [Nielsen et al.,
2014]. We calculated the dimensions of a square patch of the same area to
simplify eosinophil counting for deep convolutional models. First, the square
root was computed to find the length of the sides of square in metric units:
$\sqrt{0.21}=0.46$mm or 460 microns. The scale of the EoE biopsy images was
approximately 0.23 microns per pixel. Therefore, the length of one side of a
square HPF was $\frac{460}{0.23}=2,000$ pixels. To generate HPF samples from
WSI, 2,000 $\times$ 2,000 pixel patches were then generated.

####  3.1.2 Model Selection and Training

In addition to the traditional U-Net model, we evaluated the performance of
the Residual U-Net, R2U-Net, and Attention U-Net. A 6-fold cross validation
was used to ensure that accuracy is not biased towards a small test set. Each
model could vary in the number of total parameters via adjustment of the
number of filters in the initial convolutional block. Then, the number of
filters was doubled in each convolutional block on the contracting side until
the tip of the “U” was reached. We tested architectures with increasing
numbers of filters and parameters, but were eventually restricted by GPU
memory limits. Each model was trained using the Adam optimizer for 200 epochs
with a learning rate of $2\times 10^{-5}$ [Kingma and Ba, 2014]. The model
weights that achieved the highest validation accuracy were used for testing.
Table 2 shows the results from the cross validation. The size field in Table 2
refers to the total number of parameters of each model.

Table 2: Cross Validation for Model Selection Test Set Dice Coefficient
Statistics
---
Model | Size | Median | Min | Max
U-Net | 4.9M | 0.628 | 0.594 | 0.685
U-Net | 8.6M | 0.660 | 0.632 | 0.698
U-Net | 10.9M | 0.665 | 0.600 | 0.701
Res. U-Net | 2.7M | 0.632 | 0.588 | 0.697
Res. U-Net | 4.7M | 0.656 | 0.609 | 0.696
Res. U-Net | 7.4M | 0.557 | 0.541 | 0.645
R2U-Net | 3.4M | 0.634 | 0.606 | 0.686
R2U-Net | 6.0M | 0.614 | 0.530 | 0.647
R2U-Net | 9.4M | 0.631 | 0.572 | 0.666
Attn. U-Net | 3.1M | 0.517 | 0.439 | 0.627
Attn. U-Net | 4.5M | 0.529 | 0.465 | 0.586

The U-Net model performed better on eosinophil detection than the other more
advanced approaches. This could be due to the U-Net being able to accommodate
more parameters within GPU memory limits. The 10.9M U-Net had a slightly
higher median and maximum test dice set coefficient, but the 8.6M U-Net had a
much higher minimum value. We selected the 8.9M U-Net since it was less
complex and provided near equivalent accuracy as the 10.9M U-Net. To establish
the final 8.9M U-Net model, only training and validation sets were utilized
from the 274 total images. The size of the training set was 214 images while
the validation set size was 60 images. The resulting validation Dice
coefficient after training with a similar process was 0.705. Figure 1 shows a
few examples of eosinophil detections on validation set images.

Figure 1: Examples of eosinophil detections from the U-Net model. On the left,
are three input images from the validation set. In the middle, are the ground-
truth annotations. On the right, are the prediction masks.

####  3.1.3 Eosinophil Counting and Validation

Successful eosinophil segmentation is crucial to count cells and generate per-
image and per-patient features. With the U-Net segmentation model established,
we then developed a post-processing technique to extract the cell counts.
First, the output of the U-Net model was converted to a binary segmentation
mask using a 0.5 probability threshold. The resultant segmentation masks
sometimes contained small contiguous areas or artifacts that were not
eosinophils. Eosinophils are typically 800 contiguous pixels in size, so
objects with less than 200 contiguous pixels were removed to reduce false
positive detections and still preserve partially-detected eosinophils. The
next step was performing an Euclidean distance transformation in an attempt to
separate any eosinophils that may overlap [Heinz et al., 1995]. This operation
reduced the size of cells from the outside edge so that only the centers were
counted. The approach is similar to erosion, but is more computationally
efficient. A distance of eight pixels was used as the cutoff, because it
reduced the cell size by roughly 75% and can separate large cell overlaps.
Finally, the measure.label function was applied from the Skimage package in
Python. The label() function applied a connected component labeling algorithm
to separately label each contiguous object in the prediction mask. The count
of unique labels is one of the outputs of this function and thus represents
the number of eosinophils detected in that image.

Since the validation set Dice coefficient was optimized during model training,
the entire 274 images dataset was used to evaluate the cell counting
capability of segmentation and post-processing. The counting error was
estimated by subtracting actual and predicted eosinophil counts. The average
eosinophil error over all $512\times 512$ pixel images was $-0.12$ with a
standard deviation of $1.39$ eosinophils. When scaled to the size of an HPF,
the average error was roughly $-2$ eosinophils. As shown in Figure 2, the
negative bias was most pronounced when the true eosinophil count was greater
than 20. The requirement to diagnose EoE is 15 or more eosinophils within an
area roughly $16\times$ larger, so the bias should not affect diagnosis. When
the true eosinophil count was less than 15, minimal over-counting bias was
noticed. This was not problematic since false-positive diagnoses and severity
estimation are preferred over false-negatives as it avoids under-diagnosing
EoE which can lead to worsened patient outcomes [Lipka et al., 2016]. When the
number of eosinophils reach over 20 per patch, the most likely cause of under-
counting is the inability to separate and count overlapping cells on the
prediction mask.

Figure 2: Eosinophil counting errors. Each point on this scatterplot is one of
the 274 EoE patches. The dotted red line represents the linear trend.

###  3.2 Classification for EoE Diagnosis

An alternative to the segmentation approach for diagnosis is to extract WSI
patches from patients who are EoE-diagnosed or histologically normal and train
a CNN classification model to predict EoE versus normal. Training, validation,
and test sets were required to develop this model. A total of 36 and 41
patients were used for EoE and normal, respectively. Patches from WSIs for
patients were evenly sampled per patient and also sampled so that there was
minimal class imbalance in the training and validation sets. Table 3 shows the
number of patients and patches in each data set.

Table 3: Classification Model Data Sets | # of Patients | # of Patches
---|---|---
Data Set | EoE | Normal | EoE | Normal
Training | 21 | 24 | 15,676 | 15,780
Validation | 8 | 10 | 3,642 | 3,538
Test | 7 | 7 | 7,631 | 3,474

A VGG16 model was trained for 12 epochs with an Adam optimizer and learning
rate of $2\times 10^{-4}$. The model weights were saved that achieved the best
validation accuracy. The highest validation accuracy achieved was 0.994. The
model weights with this validation accuracy achieved a test set accuracy of
0.959. As shown in Table 4, the model rarely classified normal patches as EoE,
but there were a greater number of false-positive patches. Test set accuracies
this high are quite interesting, because it shows that EoE can be accurately
diagnosed through just a single $512\times 512$ pixel patch. This greatly
differs from the typical diagnosis method, so it could possibly be used to
discover new features that indicate EoE. In the Results section, we dive
deeper into the test set results and discuss the Grad-CAM analysis.

Table 4: Test Set Confusion Matrix for Classification Model | Predicted
---|---
| EoE | Normal
True Diagnosis | EoE | 7,176 | 455
Normal | 3 | 3,471

##  4 RESULTS

Table 5: EoE Patient Categorizations | Treatment Plans
---|---
| Phenotypes | PPI-REE | Milk Removal | 4/6 FED | Steroids | Unk | Total
Initial Biopsy | PPI | 9 |  |  |  |  | 9
Inflammatory |  | 1 | 4 | 6 |  | 11
Strictures |  |  | 2 | 5 | 1 | 8
Total | 9 | 1 | 6 | 11 | 1 | 28
Not Initial | PPI |  |  |  |  |  | 0
Inflammatory |  | 2 | 3 | 8 | 4 | 17
Strictures |  |  |  | 1 | 2 | 3
Total | 0 | 2 | 3 | 9 | 6 | 20

In this section, we present findings generated from the image segmentation and
classification approaches. For image segmentation, the model was applied to a
larger set of EoE patients with completed retrospective chart reviews. This
led to patient-level features for characterization of EoE that were analyzed
to find linkages with treatment and clinical phenotypes. The image
classification model produced an accurate fit on small WSI patches. We looked
further into a single patient who had a large number of misclassified patches
and assessed Grad-CAMs to understand the differences within their biopsy
tissue sample.

###  4.1 Image Segmentation Results

A total of 44 EoE patients with completed retrospective chart reviews and 57
histologically normal patients were used in this section to assess diagnostic
capability and relationships with other biopsy characteristics. The EoE
patients were categorized into two major phenotypes: clinical and treatment-
based. The clinical phenotypes were based on patients developing strictures
vs. those with the disease remaining inflammatory vs. PPI-REE. Treatment
phenotypes included patients with disease that were responsive to dairy-only
(milk) elimination vs. 4 to 6 food elimination diet (4/6 FED) vs. steroids vs.
unknown (where treatment response was unclear). Biopsies from these patients
were all retrieved at the same university hospital at the time of initial EoE
diagnosis. To assess the linkages between initial biopsy and phenotypes or
treatments it was important to isolate the patients that have true initial
biopsies. Table 5 shows the how many patients were distributed in each bin for
all possible categories.

Various statistics were calculated for each patient to represent the severity
and extent of EoE. Patients had up to three WSIs each depending on how many
different esophageal locations were sampled during the endoscopy. For all
results, the HPFs from all WSIs were aggregated for each patient and not
separated by esophagus location. HPFs were not ideal for collecting
information about localized concentrations of eosinophils. Therefore,
statistics were also collected for $512\times 512$ pixel patches to determine
if there were any micro-level trends that differ from the macro-level. Due to
the small sample sizes within the treatment and clinical phenotype categories
after splitting for initial biopsy, the non-parametric Wilcoxon rank sum test
was used to test all statistical differences between each category pair
[Wilcoxon et al., 1970]. The statistics are also highly correlated with each
other within this small set of patients, so no multivariate testing was
performed. Multivariate testing could be possible when more EoE patient
response data becomes available. The following list details some of the
eosinophil statistics that were captured:

  * •

Maximum eosinophils in HPFs

  * •

Average eosinophil count over all HPFs

  * •

Average eosinophil size (pixels)

  * •

Percent of HPFs with zero eosinophils

  * •

Percent of HPFs with $\leq 5$ eosinophils

  * •

Percent of HPFs with $\geq$ 15, 30, 60 eosinophils

  * •

Maximum eosinophils in $512\times 512$ patches

  * •

Average eosinophil count over all patches

  * •

Percent of patches with zero eosinophils

  * •

Percent of patches with $\geq$ 5, 10, 15 eosinophils

####  4.1.1 Diagnosis, Severity, and Extent

Using the criteria of 15 or more eosinophils within an HPF, patients were able
to be diagnosed via results obtained from the image segmentation model. Table
6 shows the classification results on the 44 and 57 patients for EoE and
normal, respectively. The overall accuracy of this approach was $99.0\%$. The
sensitivity was $100\%$ and the specificity was $98.2\%$. This demonstrates
that the automated approach was able to adequately diagnose EoE and thus could
be a useful tool to assist pathologists.

Table 6: Confusion Matrix for EoE Diagnosis | Predicted
---|---
| EoE | Normal
True Diagnosis | EoE | 44 | 0
Normal | 1 | 56

Additionally, EoE severity can be predicted using the eosinophil statistics.
Due to the difficulty of manually counting all eosinophils, pathologists
typically do not continue counting eosinophils after reaching about 60 counted
in an HPF [Collins et al., 2017]. This was not a limitation for the automated
deep learning approach. The maximum eosinophil count can be counted as an
unbounded continuous variable which provides more fidelity over the manual
method.

Similar difficulties were present when manually estimating EoE extent. The
image segmentation approach obtained counts from all HPF samples and estimated
the extent by calculating the percentage of HPFs that exceeded a certain
criteria. Figure 3 shows a scatterplot of each EoE patient’s maximum
eosinophil per HPF and percentage of HPFs with 15 or more eosinophils.
Severity and extent appeared to correlate, but there were patients who
deviated from the trend-line.

Figure 3: EoE severity versus extent. The $x$-axis represents extent, while
the $y$-axis represent severity. The dotted red line is the linear trend
between the two factors.

####  4.1.2 Linkages with Treatment Phenotypes

The goal of this subsection was to identify linkages between collected
eosinophil statistics at the time of initial biopsy and optimal treatment
plans. Treatments that the patients responded to in our dataset were assessed
at 6 or more months of follow up via retrospective chart review. As shown in
Table 5, the sample sizes for initial biopsy patients were small but there was
enough data points within 4/6 FED, PPI-REE, and steroids to generate
statistical significance.

Figure 4 shows the maximum eosinophil count for each patient based on their
optimal treatment plan. The maximum count was generated from all HPFs within
each patient’s WSIs. The milk removal category did not have enough samples to
evaluate and there was also one unknown treatment. The 4/6 FED treatment
appeared to correlate with higher EoE severity. The statistical test revealed
that 4/6 FED had a significantly higher maximum eosinophil count than PPI-REE
and steroids with p-values of 0.012 and 0.006, respectively.

Figure 4: Maximum eosinophils over HPFs per patient by treatment plan.
Patients with high EoE severity tend benefit most from 4 to 6 food elimination
treatment.

Figure 5 shows the percentage of HPFs that contain at least 60 eosinophils by
treatment plan. The 4/6 FED treatment also appeared to correlate with higher
EoE extent. There were similar trends with lower eosinophil thresholds such as
15 or 30, but 60 showed a clearer difference between the treatments. The
statistical test revealed that 4/6 FED had a significantly higher percentage
of HPFs with $\geq$ 60 eosinophils than PPI-REE and steroids with p-values of
0.008 and 0.015, respectively.

Figure 5: Percentage of HPFs with $\geq$ 60 eosinophils. Patients with high
EoE extent tend to benefit most from 4 to 6 food elimination treatment.

Figure 6 shows the average eosinophil size (pixels) by treatment plan. The
average eosinophil size is calculated by summing the number of eosinophil-
detected pixels in each HPF and then dividing by the predicted count. Not only
does 4/6 FED treatment seem to link with patients with higher EoE severity and
extent, but also with the actual size of the eosinophils. The statistical test
revealed that 4/6 FED had a significantly higher average eosinophil size than
PPI-REE and steroids with p-values of 0.033 and 0.008, respectively.

Figure 6: Average eosinophil size in pixels by treatment plan. Patients that
have larger eosinophils tend to benefit most from 4 to 6 food elimination
treatment.

There were other variables such as the average eosinophil count and similar
statistics at the 512 $\times$ 512 pixel patch that also showed 4/6 FED as
significantly different from PPI-REE or steroids treatment.

####  4.1.3 Linkages with Clinical Phenotypes

The goal of this subsection was similar to the previous as we searched for
linkages between initial biopsy features and a patient’s clinical phenotype.
While the sample sizes were still low, the categories were fairly well-
balanced with all three ranging between 8 and 11. Figure 7 shows the maximum
eosinophil count per patient using 512 $\times$ 512 patches and by phenotype.
The treatment phenotype findings were similar for both HPFs and patches, but
the clinical phenotype only generated significant differences at the “micro-
level”. Therefore, the indicators of phenotypes at the time of initial biopsy
and diagnosis may only exist when examining eosinophils at a highly localized
level and not through HPFs. The statistical test revealed that PPI-REE had a
significantly lower maximum eosinophil count per patch than strictures and
inflammatory with p-values of 0.021 and 0.049, respectively.

Figure 7: Maximum eosinophil count over patches per patient by phenotype.
Patients with a low eosinophil counts in smaller regions tend to be linked
with the PPI phenotype.

Figure 7 shows the maximum eosinophil count per patient using 512 $\times$ 512
patches by clinical phenotype. This shows another example of how patients with
the PPI-REE phenotype rarely ever exceeded ten eosinophils within a 512
$\times$ 512 patch. The median percentage for PPI was less than 1%, while the
other two had medians at roughly 4%. The statistical test revealed that PPI-
REE had a significantly lower percentage of patches with $\geq$ 10 eosinophils
than strictures and inflammatory with p-values of 0.034 and 0.011,
respectively.

Figure 8: Percentage of patches with $\geq$ 10 eosinophils per patient by
phenotype. Patients with very low percentages of patches with a large number
of eosinophils tend to have the PPI-REE phenotype

###  4.2 Image Classification Results

Figure 9: Grad-CAMs are applied to patches over a section of tissue from
patient E-136’s WSI. On the left, is the section of WSI tissue. On the right
is the same crop of tissue, but with the Grad-CAM overlaid. This was the only
EoE patient in the test set that had large areas of tissue predicted as
normal.

As discussed in the Methodology section, a VGG16 CNN model was developed with
training and validation data sets and then evaluated on a test set. The
training and validation sets contained EoE patients from the 4/6 FED, PPI-REE,
and steroids treatment plans, while the test set EoE patients were from the
milk removal and unknown treatments. Overall, the prediction accuracies were
high across all three data sets, but there was one interesting anomaly in the
test set results. Table 7 shows the classification results from the VGG16
model by each patient from the test set. All patients had almost all of their
patches correctly classified except for EoE patient E-136.

Table 7: Classification Predictions by Patient | Prediction
---|---
Truth | Patient | EoE | Normal
EoE | E-29 | 187 | 1
E-103 | 701 |
E-116 | 798 |
E-123 | 1,567 | 17
E-124 | 784 | 2
E-136 | 808 | 435
E-201 | 2,331 |
Normal | N16-38 |  | 450
N16-39 | 3 | 1,317
N16-40 |  | 355
N16-41 |  | 651
N16-42 |  | 285
N16-43 |  | 381
N16-44 |  | 32

About one-third of the WSI patches for E-136 were misclassified as normal,
while other patients had almost 100% correctly classified EoE patches. The
fact that predictions varied within one patient’s WSI was one piece of
supporting evidence that the model was not utilizing extraneous features to
classify EoE vs. normal. Examining patient E-136’s tissue sample may be the
key to determining which biopsy features were utilized by the deep learning
model. Grad-CAMs can be utilized to visualize the important features used in a
CNN model. Grad-CAMs were produced at the patch-level to cover E-136’s entire
WSI in order to assess which parts of the tissue were considered EoE or
normal. Pathologists examined the Grad-CAMs and determined whether there was a
consistent trend associated with the tissue and a prediction class. The WSI-
level Grad-CAMs were highlighting more areas for images that had a higher
number of eosinophils. Further, the heatmaps were also focusing on areas with
cellular crowding and bottom most (basal) layer of epithelium with images
having dilated intercellular spaces. These increased (dilated) intercellular
spaces indicate underlying cellular edema taking place due to the inflammation
caused by the disease [Collins et al., 2017].

Figure 9 shows an example of a large tissue crop from one on the WSIs from
patient E-136. There were large sections in the middle of the tissue sample
that the model considers normal (blue), while other areas are predicted EoE
(red). After analyzing patient E-136 and comparing to other patient’s Grad-
CAMs, it was determined the model was classifying patches EoE or normal likely
because of features other than eosinophils such as cellular crowding and basal
layer of the epithelium. Another important aspect was that the classification
model was not utilizing eosinophils to diagnose EoE. The section of tissue in
Figure 9 was very large (8000 $\times$ 6000 pixels) and about half was
predicted as EoE, but there were only three eosinophils detected within this
area. This amount of eosinophils is far short of the typical diagnostic
criteria.

##  5 CONCLUSION

In this paper, we present a novel approach for diagnosing EoE, understanding
more about biopsy tissue level features, and linking EoE biopsy features with
treatment and clinical phenotypes. This was the first time deep learning
computer vision was applied to diagnose EoE and detect eosinophils in biopsy
tissue samples. The eosinophil segmentation is medically critical since it can
aide pathologists in diagnosis by automatically providing peak eosinophil
counts and knowledge of other areas requiring direct focus and attention. The
automated assessment of eosinophil count can also be associated with disease
severity, progression, and medically-relevant treatment and clinical
phenotypes. The trained U-Net image segmentation model detected eosinophils
within an adequate accuracy.

Most of the patient-level eosinophil statistics cannot be realistically
captured without an automated approach. These statistics can be used to
explain the EoE severity and extent or to predict clinical and treatment
phenotypes. Statistical analysis revealed that patients who eventually
responded to the 4/6 FED treatment plan had high severity and extent of EoE at
the time of initial biopsy. The PPI-REE phenotype was found to relate with
lower severity (decreased eosinophil counts) and extent at the smaller patch
level.

In conjunction with image segmentation, image classification was used to learn
more about how biopsy features relate to EoE. A VGG16 CNN model was trained on
a data set of EoE and normal WSI patches and achieved highly accurate results.
The model’s performance was interesting since it was able to predict EoE using
only small patches and did not depend solely on eosinophils for diagnosis. One
EoE patient in particular had a large section of tissue predicted as normal
and this was because it had less areas of eosinophils which lead to less
activations mapped by Grad-CAMs but also features other than eosinophils such
as cellular crowding and basal epithelial layer with some images have dilated
intercellular spaces were being highlighted. Basal layer epithelium
identification and its thickness along with presence of dilated intercellular
spaces representing cellular edema are signs of inflammation caused by EoE and
have been proposed as possible diagnostic features of the disease [Collins et
al., 2017].

There are many avenues for future research in this area. Increasing eosinophil
annotations will further improve prediction performance and enable the model
to be more robust to even slight variations in new biopsy imagery. The sample
size of patient-level data was small due to which most of the statistical
analysis was limited. There is currently a plan in place to greatly increase
the sample size of patients with completed chart reviews. With additional
data, we will also be able to analyze eosinophil trends by the esophageal
tissue sample location. The Grad-CAM review can be considered subjective, thus
integrating Grad-CAMs with another approach that can quantify cellular-level
features will reduce human-injected bias. We are in the process of developing
an unsupervised segmentation approach that will also be reviewed by
pathologists to automate patch-level feature generation.

## REFERENCES

  * Alom et al., 2018 Alom, M. Z., Yakopcic, C., Taha, T. M., and Asari, V. K. (2018).  Nuclei segmentation with recurrent residual convolutional neural networks based U-Net (r2u-net).  In NAECON 2018 - IEEE National Aerospace and Electronics Conference, pages 228–233.
  * Carr et al., 2018 Carr, S., Chan, E. S., and Watson, W. (2018).  Eosinophilic esophagitis.  Allergy, Asthma & Clinical Immunology, 14(2):1–11.
  * Collins et al., 2017 Collins, M. H., Martin, L. J., Alexander, E. S., Boyd, J. T., Sheridan, R., He, H., Pentiuk, S., Putnam, P. E., Abonia, J. P., Mukkada, V. A., Franciosi, J. P., and Rothenberg, M. E. (2017).  Newly developed and validated eosinophilic esophagitis histology scoring system and evidence that it outperforms peak eosinophil count for disease diagnosis and monitoring.  Diseases of the Esophagus : Official Journal of the International Society for Diseases of the Esophagus, 30(3):1–8.
  * Dellon, 2014 Dellon, E. S. (2014).  Epidemiology of eosinophilic esophagitis.  Gastroenterology Clinics, 43(2):201–218.
  * Dellon et al., 2014 Dellon, E. S., Kim, H. P., Sperry, S. L., Rybnicek, D. A., Woosley, J. T., and Shaheen, N. J. (2014).  A phenotypic analysis shows that eosinophilic esophagitis is a progressive fibrostenotic disease.  Gastrointestinal Endoscopy, 79(4):577–585.
  * Eelbode et al., 2020 Eelbode, T., Bertels, J., Berman, M., Vandermeulen, D., Maes, F., Bisschops, R., and Blaschko, M. B. (2020).  Optimization for medical image segmentation: Theory and practice when evaluating with dice score or jaccard index.  IEEE Transactions on Medical Imaging.
  * Furuta et al., 2007 Furuta, G. T., Liacouras, C. A., Collins, M. H., Gupta, S. K., Justinich, C., Putnam, P. E., Bonis, P., Hassall, E., Straumann, A., Rothenberg, M. E., et al. (2007).  Eosinophilic esophagitis in children and adults: A systematic review and consensus recommendations for diagnosis and treatment: Sponsored by the American Gastroenterological Association (AGA) institute and north American society of pediatric gastroenterology, hepatology, and nutrition.  Gastroenterology, 133(4):1342–1363.
  * Godwin et al., 2020 Godwin, B., Wilkins, B., and Muir, A. B. (2020).  Eoe disease monitoring: Where we are and where we are going.  Annals of Allergy, Asthma & Immunology, 124(3):240–247.
  * Habibzadeh et al., 2018 Habibzadeh, M., Jannesari, M., Rezaei, Z., Baharvand, H., and Totonchi, M. (2018).  Automatic white blood cell classification using pre-trained deep learning models: Resnet and inception.  In Tenth International Conference on Machine Vision (ICMV 2017), volume 10696, page 1069612. International Society for Optics and Photonics.
  * Heinz et al., 1995 Heinz, B., Gil, J., Kirkpatrick, D., and Werman, M. (1995).  Linear time Euclidean distance transform algorithms.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(5):529–533.
  * Kingma and Ba, 2014 Kingma, D. and Ba, J. (2014).  Adam: A method for stochastic optimization.  International Conference on Learning Representations.
  * Kumar et al., 2019 Kumar, N., Verma, R., Anand, D., Zhou, Y., Onder, O. F., Tsougenis, E., Chen, H., Heng, P. A., Li, J., Hu, Z., et al. (2019).  A multi-organ nucleus segmentation challenge.  IEEE Transactions on Medical Imaging.
  * Liacouras et al., 2011 Liacouras, C. A., Furuta, G. T., Hirano, I., Atkins, D., Attwood, S. E., Bonis, P. A., Burks, A. W., Chehade, M., Collins, M. H., Dellon, E. S., et al. (2011).  Eosinophilic esophagitis: Updated consensus recommendations for children and adults.  Journal of Allergy and Clinical Immunology, 128(1):3–20.
  * Liang et al., 2018 Liang, G., Hong, H., Xie, W., and Zheng, L. (2018).  Combining convolutional neural network with recursive neural network for blood cell image classification.  IEEE Access, 6:36188–36197.
  * Lipka et al., 2016 Lipka, S., Kumar, A., and Richter, J. E. (2016).  Impact of diagnostic delay and other risk factors on eosinophilic esophagitis phenotype and esophageal diameter.  Journal of Clinical Gastroenterology, 50(2):134–140.
  * Milletari et al., 2016 Milletari, F., Navab, N., and Ahmadi, S.-A. (2016).  V-net: Fully convolutional neural networks for volumetric medical image segmentation.  In 2016 Fourth International Conference on 3D Vision (3DV), pages 565–571. IEEE.
  * Nielsen et al., 2014 Nielsen, J. A., Lager, D. J., Lewin, M., Rendon, G., and Roberts, C. A. (2014).  The optimal number of biopsy fragments to establish a morphologic diagnosis of eosinophilic esophagitis.  The American Journal of Gastroenterology, 109(4):515.
  * Oktay et al., 2018 Oktay, O., Schlemper, J., Folgoc, L. L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N. Y., Kainz, B., et al. (2018).  Attention u-net: Learning where to look for the pancreas.  International Conference on Medical Imaging with Deep Learning.
  * Özyurt, 2019 Özyurt, F. (2019).  A fused CNN model for WBC detection with MRMR feature selection and extreme learning machine.  Soft Computing, pages 1–10.
  * Reed et al., 2018 Reed, C. C., Koutlas, N. T., Robey, B. S., Hansen, J., and Dellon, E. S. (2018).  Prolonged time to diagnosis of eosinophilic esophagitis despite increasing knowledge of the disease.  Clinical Gastroenterology and Hepatology: the Official Clinical Practice Journal of the American Gastroenterological Association, 16(10):1667.
  * Ronneberger et al., 2015 Ronneberger, O., Fischer, P., and Brox, T. (2015).  U-Net: Convolutional networks for biomedical image segmentation.  In International Conference on Medical image computing and computer-assisted intervention, pages 234–241. Springer.
  * Rosenberg et al., 2012 Rosenberg, H. F., Dyer, K. D., and Foster, P. S. (2012).  Eosinophils: changing perspectives in health and disease.  Nature Reviews Immunology, 13(1):9–22.
  * Runge et al., 2017 Runge, T. M., Eluri, S., Cotton, C. C., Burk, C. M., Woosley, J. T., Shaheen, N. J., and Dellon, E. S. (2017).  Causes and outcomes of esophageal perforation in eosinophilic esophagitis.  Journal of Clinical Gastroenterology, 51(9):805.
  * Russakovsky et al., 2015 Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015).  Imagenet large scale visual recognition challenge.  International journal of computer vision, 115(3):211–252.
  * Selvaraju et al., 2017 Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017).  Grad-CAM: Visual explanations from deep networks via gradient-based localization.  In Proceedings of the IEEE International Conference on Computer Vision, pages 618–626.
  * Simonyan and Zisserman, 2015 Simonyan, K. and Zisserman, A. (2015).  Very deep convolutional networks for large-scale image recognition.  In International Conference on Learning Representations.
  * Sudre et al., 2017 Sudre, C. H., Li, W., Vercauteren, T., Ourselin, S., and Cardoso, M. J. (2017).  Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations.  In Deep learning in medical image analysis and multimodal learning for clinical decision support, pages 240–248. Springer.
  * Tran et al., 2018 Tran, T., Kwon, O.-H., Kwon, K.-R., Lee, S.-H., and Kang, K.-W. (2018).  Blood cell images segmentation using deep learning semantic segmentation.  In 2018 IEEE International Conference on Electronics and Communication Engineering (ICECE), pages 13–16. IEEE.
  * Wilcoxon et al., 1970 Wilcoxon, F., Katti, S., and Wilcox, R. A. (1970).  Critical values and probability levels for the Wilcoxon rank sum test and the Wilcoxon signed rank test.  Selected tables in mathematical statistics, 1:171–259.
  * Wilson and McGowan, 2018 Wilson, J. M. and McGowan, E. C. (2018).  Diagnosis and management of eosinophilic esophagitis.  Immunology and allergy clinics of North America, 38(1):125.
  * Zhang et al., 2018 Zhang, Z., Liu, Q., and Wang, Y. (2018).  Road extraction by deep residual U-Net.  IEEE Geoscience and Remote Sensing Letters, 15(5):749–753.