# Madden-Julian Oscillation described as a Nonlinear Burgers Kink in the
Meridional Vorticity Equation

Richard Blender  Meteorological Institute, Universität Hamburg, Hamburg,
Germany<EMAIL_ADDRESS>
###### Abstract

A dynamic equation for a large scale convective event in the tropical
atmosphere similar to the Madden–Julian Oscillation (MJO) is suggested based
on the meridional vorticity equation with buoyancy parametrized by Convective
Available Potential Energy (CAPE). The propagation is determined by the
nonlinear Burgers equation with a stationary solution describing a kink moving
towards the moisture source. In this conceptual model, the propagation speed
depends on the asymmetry in the zonal surface winds which is observed in the
boundary layer in the vicinity of an MJO event and attributed to Rossby and
Kelvin waves. Furthermore, the model predicts convection at the equator in the
east of the MJO which is not correlated with Kelvin and Rossby waves.

###### pacs:

47.27.De, 47.32.-y, 92.10.ab, 92.60.-e

##  1 Introduction

The Madden–Julian Oscillation (MJO) [1] appears as a large scale and intense
precipitation field in the Indian ocean with a huge global impact on weather
and climate [2]. The MJO can be projected on well-known linear tropical waves
[3, 4] but has a much lower progression speed, $c\approx 5\,\mbox{ms}^{-1}$.
Furthermore, whereas the major part of tropical precipitation is related to
these waves, a considerable contribution close to the equator is not explained
[5].

Although the MJO is the dominant part of the intraseasonal (30–90 days)
variability in the tropics, the understanding and modelling by GCMs is
considered incomplete [6]. While linear dynamics with heating is well
understood [7], there is ongoing search for models representing main
mechanisms and properties, see the review [8]. Whereas [8] considers mainly
parametrized physical processes, the analysis of [5] hints at a dynamical
mechanism. Possible reasons for the difficulties in the perception and the
understanding of the MJO are discussed in [9] from a philosophical
perspective.

The majority of weather forecast and climate models assumes hydrostatic
conditions and neglects the horizontal meridional vorticity and the northward
component of the Coriolis vector on the sphere (coined Traditional
Approximation (TA) by [10]). Simplified models also based on the TA have been
proposed which explain the MJO as a localised structure in nonlinear
equations: [11] derived a Korteweg-de Vries (KdV) equation with soliton
solutions for resonant baroclinic and barotropic Rossby waves, [12] suggested
equatorial modon solutions and [13] a modon in the moist-convective shallow
water equation. A realistic but yet simplified MJO model is the stochastic
skeleton model [14] of a neutrally stable atmosphere coupled to an ocean
surface which describes interactions with the Walker circulation and ENSO on
various time scales [15].

Nontraditional models might solve long standing problems like, for example,
the life cycle of convective systems with observed periods of two days in
outgoing long wave radiation [16, 17]. Prospects for avoiding the TA in
nontraditional models have been already pointed out by [18], and [19] have
underlined the relevance of the full Coriolis force for large scale tropical
motions. Atmospheric dynamics with the full Coriolis force and the impact of
approximations on conservation laws is described in-depth by [20] and a review
of the effects of the nontraditional approximation is given by [21]. The
traditional approximation was, at least partially, avoided in the quasi-
hydrostatic equation in the UKMO weather forecast model early in 1992 [22]. At
present there are steps towards including the nontraditional Coriolis
component in the upper atmosphere in the atmospheric model ICON [23] which is
used by the German Weather Service and the Max-Planck Institute for
Meteorology in Hamburg [24].

At the equator a particular type of waves exists in the nontraditional
equation for the meridional (northward) vorticity component in the Boussinesq
approximation in the vertical $x$-$z$-plane [25]. These waves are coined
compressional Rossby waves and propagate eastward with a slow phase speed
$c=2\Omega/H$ with angular velocity $\Omega$ and scale height $H$ (for
$H=9.1\,\mbox{km}$, $c=0.24\,\mbox{ms}^{-1}$).

The aim of this publication is to present a parsimonious model for the
progression of a large scale convective event at the equator in the Boussinesq
approximation with a nonlinear parametrization of buoyancy. The model is
guided by the dynamics of compressional Rossby waves and therefore
nontraditional, although the horizontal Coriolis force term is discarded after
a scale analysis. We derive a dynamic equation for the first baroclinic mode
of the meridional vorticity component and assume neutral stratification.
Linear tropical waves like Kelvin and Rossby waves are not included.

The scales in our model follow the observations in [26], who pointed to the
major role of boundary layer moisture for the MJO and found that the westward
advection of moisture in the boundary layer is responsible for the eastward
propagation of the Madden–Julian oscillation. We parametrize convection using
CAPE (Convective Available Potential Energy) to relate buoyancy to the large
scale vorticity [27, 28]. Heuristically the nonlinear parametrization can be
interpreted as the product of lower tropospheric moisture transport towards
the MJO times vertical velocity.

We obtain a Burgers equation where nonlinear steepening is balanced by
horizontal diffusion which is supposed to originate in fast gravity waves
observed in the vicinity of deep convection [29]. The equation has a
stationary nonlinear solution with a step-like kink in the vorticity which
shows unidirectional eastward propagation towards the moist boundary layer
with a speed related to the zonal asymmetry. In [30] this asymmetry is found
in observations and GCMs with a realistic MJO propagation speed and attributed
to stronger Kelvin easterly waves than Rossby westerly waves in the lower
troposphere.

The stationary solution allows the interpretation of the propagation speed in
terms of the intensities of the two adjacent circulation cells. Propagation
declines when the MJO reaches a region with east-west symmetry like the
maritime continent [31]. The paper is organized as follows: In the next
Section 2 the dynamics of the compressional Rossby waves is described and in
Section 3 we derive the Burgers equation for a convective event. In Section 4
the results are summarized and discussed.

##  2 Compressional Rossby waves

The nontraditional approximation in a vertical plane close to the equator
allows a type of linear waves in the horizontal vorticity component $\eta$
with the nontraditional Coriolis term in the restoring force [25]. Since these
waves share the geometric setting with the Burgers equation derived below
their dynamic mechanism is recapitulated here. The waves are described in the
Boussinesq approximation and propagate eastward in the equatorial
$x$-$z$-plane due to the so-called compressible $\beta$-effect, coined in
analogy to the $\beta$-effect in the midlatitudes [25].

The 3D vortex vector ${\bm{\omega}}=\nabla\times{\bm{u}}$ evolves according to
the Helmholtz vorticity equation, see for example [32]

$\frac{d}{dt}{\bm{\omega}}={\bm{\omega}}_{a}\cdot\nabla{\bm{u}}-{\bm{\omega}}_{a}\nabla\cdot{\bm{u}}+\nabla\times(b{\bm{e}}_{z}),\quad{\bm{\omega}}_{a}={\bm{\omega}}+2{\bm{\Omega}},$
(1)

with the absolute vorticity ${\bm{\omega}}_{a}$, the angular rotation rate of
the earth ${\bm{\Omega}}$ and buoyancy $b$.

The meridional component of the relative vorticity is
$\eta=\omega_{y}=\partial_{z}u-\partial_{x}w$, and the absolute meridional
vorticity on the sphere is $\eta_{a}=\tilde{f}+\eta$ with the so-called cos-
Coriolis parameter $\tilde{f}=2\Omega\cos\varphi$ at the latitude $\varphi$
(note that $\tilde{f}$ is neglected in TA, see [20]). Observations of the MJO
find that the meridional velocity component is much smaller than the zonal
component, $O(v)\ll O(u)$ [26], hence we use $v=0$ and neglect the tilting
term for $\eta$ in (1) (first term on the rhs).

In the Boussinesq approximation density decreases as $\rho\propto\exp(-z/H)$
with a scale height $H=RT/g$. In the moist tropical atmosphere stratification
can be considered to be neutral with the Brunt-Väisälä frequency $N=0$ [25].
The flow is anelastic satisfying the vertical continuity equation
$\partial_{x}(\rho u)+\partial_{z}(\rho w)=0$ for the momentum $(\rho u,\rho
w)$, hence a mass stream-function $\Psi$ for the momentum $\rho
u=\partial_{z}\Psi$, $\rho w=-\partial_{x}\Psi$ is introduced. The divergence
in the vertical plane is $\partial_{x}u+\partial_{z}w=w/H$.

The first baroclinic mode is written with a function $\psi$ describing the
zonal dependency

$\Psi(x,z,t)=\psi(x,t)e^{-z/(2H)}\sin(mz),$ (2)

where $m=\pi/H_{T}$ is the vertical wavenumber with the tropopause height
$H_{T}$. The vertical exponential decay of $\Psi$ ensures constant vertical
kinetic energy, $\rho(u^{2}+w^{2})/2$, with vertically increasing velocities
$(u,w)\propto\rho^{-1/2}\propto e^{z/(2H)}$. The meridional vorticity $\eta$
is in terms of the mass stream-function

$\rho\eta=\Psi_{xx}+\Psi_{zz}+\frac{1}{H}\Psi_{z},$ (3)

where subscripts denote derivatives.

The meridional vorticity equation at the equator ($\varphi=0$) without
buoyancy $b$ is [25]

$\frac{\partial\eta}{\partial t}+\frac{\eta_{a}}{H}w=0,$ (4)

with the divergence term in the Boussinesq approximation and without nonlinear
advection. The linearized meridional vorticity equation is obtained for
$\eta\ll\Omega$ in $\eta_{a}$,

$\frac{\partial\eta}{\partial t}+\hat{\beta}w=0,$ (5)

with the compressional $\beta$-parameter $\hat{\beta}=2\Omega/H$.

For a wave ansatz with zonal wavenumber $k$ and frequency $\nu$,

$\psi_{w}(x,t)=\sin(kx-\nu t),$ (6)

the vorticity is

$\eta=-(k^{2}+\kappa^{2})\Psi,\qquad\kappa^{2}=m^{2}+\frac{1}{4H^{2}},$ (7)

and the dispersion relation for free waves is obtained,

$\nu=\hat{\beta}\frac{k}{k^{2}+\kappa^{2}}.$ (8)

This leads to the zonal phase speed $c=\nu/k$ with $c\approx
0.24\,\mbox{ms}^{-1}$ for $\Omega=7.292\cdot 10^{-5}\,\mbox{s}^{-1},H=9.1\cdot
1000\,\mbox{m}$, and $m=2\pi/25000\mbox{m}^{-1}$ for long zonal waves
$k^{2}\ll m^{2}$ [25]. Obviously, this mechanism cannot explain the observed
MJO speed ($c\approx 5\,\mbox{ms}^{-1}$). Since the magnitude of the observed
vorticity $O(\eta)\sim 10^{-3}\,\mbox{s}^{-1}$ is much larger than
$O(2\Omega)\sim 10^{-4}\,\mbox{s}^{-1}$ the equation for finite amplitude
compressional Rossby waves becomes nonlinear (here the symbol $\sim$ means
equal orders of magnitude or scales).

In the next section we will assume that convection is the major process in the
dynamics of the MJO, and concentrate on the corresponding nonlinear mechanism.
This leads to a solvable nonlinear equation with a propagating step-like
finite amplitude disturbance. The compressional Rossby wave mechanism is not
pursued further.

Figure 1:  Sketch of the circulation between two adjacent equatorial
convection cells in the $x$-$z$-plane with opposite meridional vorticity,
$\eta=\partial_{z}u-\partial_{x}w$ in terms of the zonal and vertical
velocities ($u,w$). $L_{x}$ is the zonal scale ($10^{7}$ m) and $H_{T}$ the
equatorial tropopause height (magnitude $2\cdot 10^{4}$ m).

##  3 Burgers equation for the MJO progression

In our model for the propagation of large scale tropical convection we
consider two vorticity cells in the vertical $x$-$z$-plane with opposite
rotation (see Fig. 1). We assume infinitely extended cells and do not specify
the remote circulation.

To estimate the scale of the meridional vorticity $\eta$ we use $\eta\sim
u/\Delta H\sim 10^{-3}\,\mbox{s}^{-1}$ for the observed zonal velocity scale
$u=2.5\,\mbox{ms}^{-1}$ and a height difference between the 500 and 700 hPa
levels, $\Delta H=3000\,\mbox{m}$ [26]. The linearized meridional vorticity
equation (1) with buoyancy $b$ reads as

$\frac{\partial\eta}{\partial t}=-\frac{\partial b}{\partial
x}+D\frac{\partial^{2}\eta}{\partial x^{2}},$ (9)

where we neglected the compressible $\beta$-effect and added a diffusion term.

In order to parametrize convection in the meridional vorticity equation we use
Convective Available Potential Energy (CAPE) which is an indicator for
precipitation and thunderstorms [27, 28]. We parametrize buoyancy $b$ in CAPE
in terms of the large scale vorticity by a relation to vertical kinetic
energy. A moisture source is located in the east of the MJO (in the maritime
continent) and moisture is transported by near surface advection with westward
zonal velocity $u<0$.

For vertically constant buoyancy CAPE is $h_{c}b(x,t)=\hat{w}^{2}/2$ with
maximum vertical velocity $\hat{w}(x,t)$, and the height difference $h_{c}\sim
5\cdot 10^{3}\,\mbox{m}$ between the levels of neutral buoyancy and free
convection. With typical CAPE values up to $\sim 500\,\mbox{Jkg}^{-1}$ in the
tropics we obtain $b\sim 0.1\,\mbox{ms}^{-2}$ and a moderate estimate
$\hat{w}\sim 10\,\mbox{ms}^{-1}$. The horizontal derivative $b_{x}$ in the
vorticity equation (9) could be written as
$b_{x}\sim(1/h_{c})\hat{w}\hat{w}_{x}$, however, large scale observed vertical
velocities are of the order of $w\sim 10^{-3}\,\mbox{ms}^{-1}$ [26], much
lower than the maximum values in CAPE.

The parametrization is closed by a relation between the large scale vertical
velocity and $\eta$. The scales assumed here are the zonal extent
$L_{x}=10^{7}\,\mbox{m}$ of the MJO event with a wavenumber $k=6.28\cdot
10^{-7}\,\mbox{m}^{-1}$, the time scale $T=10^{6}\,\mbox{s}$, and the
meridional vorticity $\eta\sim 10^{-3}\,\mbox{s}^{-1}$. For the
parametrization we assume a wave-like structure in the vicinity of the MJO
center such that $\eta_{xx}\approx-k^{2}\eta$. Thus the solution for the
vorticity determined below is an envelope which decays on a scale only weakly
larger than $k^{-1}$. For $\kappa^{2}=m^{2}=6.38\cdot 10^{-8}\,\mbox{m}^{-2}$
we use the values in (8).

The vertical velocity is $w=\kappa^{-2}\eta_{x}$, and
$w_{x}=\kappa^{-2}\eta_{xx}=-k^{2}\kappa^{-2}\eta$ (7). This yields
$w=1.58\cdot 10^{-3}\,\mbox{ms}^{-1}$, which is in the range of the large
scale observations [26]. These relations suggest a parametrization for the
zonal buoyancy derivative in terms of the meridional vorticity with the
functional form $b_{x}\sim-(k^{2}\kappa^{-4}/h_{c})\eta\eta_{x}$.

However, due to the huge gap between different scales and different processes
involved, the observed large scale vertical velocity is much lower than the
observed maximum value $\hat{w}$ in CAPE. Therefore, we resort to an empirical
condition with a length scale $s$,

$s\eta^{2}=\frac{1}{2h_{c}}\hat{w}^{2}.$ (10)

With the scales above this yields $s=10^{4}\,\mbox{m}$, comparable to the half
of the tropopause height, $s\sim H_{T}/2$. Finally, we parametrize buoyancy in
(9) as

$\frac{\partial b}{\partial x}=-s\eta\frac{\partial\eta}{\partial x}.$ (11)

The dynamics is effectively one-dimensional for the relative vorticity
$\eta(x,t)$ with the vertical structure given by the first baroclinic mode as
in (2),

$\hat{\eta}(x,z,t)=\eta(x,t)e^{z/(2H)}\sin(mz).$ (12)

Equation (11) allows a heuristic interpretation of the parametrized convection
in terms of moisture transport with the azimuthal velocity in a vertical
distance $s$ around a vorticity center $\eta$, $u\sim s\eta/2$, with the
angular rotation rate $\eta/2$ in the vertical plane. Additionally, the
vertical velocity $\eta_{x}$ has to be positive in convection.

Thus, an advantage of the nonlinear parametrization is that the sign of the
moisture transport is mimicked by the sign of $s\eta$. For the signs of $\eta$
in Fig. 1 the moisture source region can be associated with the sign of $s$,
as for an easterly (westerly) moisture source we have a scale $s>0$ ($s<0$)
which leads to positive advection of moisture in the eastern (western) cell by
$\eta>0$ ($\eta<0$). The solution will show that the cell propagates towards
the moisture source (eastward for $s>0$, and westward for $s<0$).

The propagation of an initial spontaneous convection cell with convection in
the center can be understood as follows: For a wave-like form
$\eta\sim\sin(kx)$, the buoyancy term $\eta\eta_{x}$ is positive in a region
east of the center. This causes a phase shift between buoyancy and circulation
which drives the cell eastward.

To maintain a stationary solution a dissipative process is required which is
described here as a linear diffusion term $D\eta_{xx}$. We apply the diffusion
coefficient $D=10^{8}\,\mbox{m}^{2}\mbox{s}^{-1}$ which corresponds to a mean-
square displacement scaling with $L_{x}^{2}/T$, corresponding to the magnitude
of the vorticity tendency and the buoyancy forcing. A plausible process is the
energy loss of the large scale circulation by the radiation of gravity waves
as observed in the vicinity of tropical storms [29] (note that the diffusion
coefficients therein are much smaller).

With the parametrization (11) the meridional vorticity equation (9) becomes
the Burgers equation,

$\frac{\partial\eta}{\partial t}-s\eta\frac{\partial\eta}{\partial
x}=D\frac{\partial^{2}\eta}{\partial x^{2}}.$ (13)

Solutions of the Burgers equation are stationary if nonlinear steepening is
balanced by diffusion (in contrast to the Korteweg-de Vries equation where
steepening is balanced by dispersion, see e.g. [11]).

The Burgers equation (13) allows a solution (see e.g. [33, 34]),

$\eta=-\frac{c}{s}+\frac{2\lambda D}{s}\tanh(\lambda(x-ct)),$ (14)

which describes a kink with two free parameters, the propagation speed $c$ and
an arbitrary inverse width $\lambda$ which also determines the amplitude. This
exact nonlinear solution is the main advantage of the present model since it
reveals an unsuspected relation between the propagation speed and the
intensities of the circulation cells in the MJO. The solution incorporates the
essential constant mean vorticity $\bar{\eta}=-c/s$ which is considered in
more detail below.

With $s=-1$, (13) can be brought in standard form $U_{t}+UU_{x}=DU_{xx}$. In
this case $U$ represents the zonal velocity in the lower troposphere. Exact
solutions can be obtained by the Hopf Cole transformation
$U=-2D\partial_{x}\log\Phi$ where $\Phi$ satisfies a linear diffusion
equation.

The solution (14) describes a kink, a smooth step of the meridional vorticity
$\eta$, see Fig. 2. This kink describes the circulation in the vicinity of an
MJO event with a moisture source in the east. In the eastern and western
directions the vorticity converges to constant values. According to the
parametrization of buoyancy in terms of CAPE, precipitation is predicted in
the east for $s>0$ close to the origin where $b_{x}$ is negative (11).

The buoyancy term $-b_{x}$ causes nonlinear steepening since it enhances
positive vorticity in the east and negative vorticity in the west (Fig. 2). In
this process the vertical velocity ($\propto\eta_{x}$) in the center increases
and the convective region narrows. The curve is stationary since steepening is
compensated by horizontal diffusion. An MJO envelope can be identified by
$\eta_{x}$.

The Burgers equation (13) can be rewritten with a total derivative following
the MJO event if the vorticity is decomposed in a constant term
$\bar{\eta}=-c/s$ and a localized anomaly $\eta^{\prime}$,
$\eta=\bar{\eta}+\eta^{\prime}$,

$\frac{D}{Dt}\eta^{\prime}\equiv\left(\frac{\partial}{\partial
t}+c\frac{\partial}{\partial
x}\right)\eta^{\prime}=s\eta^{\prime}\frac{\partial}{\partial
x}\eta^{\prime}+D\frac{\partial^{2}}{\partial x^{2}}\eta^{\prime}.$ (15)

The split of $\eta$ in $\bar{\eta}$ and $\eta^{\prime}$ decomposes the
inviscid dynamics in progression (with $\bar{\eta}$) and nonlinear steepening
(with $\eta^{\prime}$). This explains the asymmetric structure of the buoyancy
term $-b_{x}$ in Fig. 2 with the term
$s\bar{\eta}\eta_{x}^{\prime}=-c\eta_{x}^{\prime}<0$, which determines
propagation.

For the comparison with observations the asymmetry of the eastern and western
limits $\eta_{E}=\lim_{x\to\infty}\eta,\eta_{W}=\lim_{x\to-\infty}\eta$ in
(14) is relevant since it is directly linked to the propagation speed,

$c=-\frac{s}{2}(\eta_{E}+\eta_{W}).$ (16)

Eastward propagation in the Indian ocean requires $s>0$. We assume that both
limits correspond to the observed maxima in the adjacent circulation cells.

The lack of a point symmetry of the solution (14) at the origin with a lower
absolute $\eta$ value in the east compared to the west is known from
observations [26] where the 700 hPa zonal winds in the east are weaker than in
the west (their Figure 13g), $u_{W}\approx
3\,\mbox{ms}^{-1},u_{E}\approx-2\,\mbox{ms}^{-1}$. Our model determines the
winds at the lower boundary as $u_{s}\approx-m\kappa^{-2}\eta$ with
$m\kappa^{-2}=3.97\cdot 10^{3}\,\mbox{m}$ and overestimates the observed
surface velocities measured below the 700 hPa level. The counterintuitive
result described in [31] as a slower MJO for an intensified Walker circulation
might be explained by higher $\eta_{E}$ leading to weaker asymmetry and a
lower speed $c$.

A zonal asymmetry is already predicted in [7] by linear wave theory for a
stationary heat source. Note that since this heat source does not propagate a
relation between speed and wave intensity is not available. The present model
suggests that the heat source propagates due to the zonal asymmetry and drags
the linear waves.

In [30] the Rossby-Kelvin intensity index (denoted RK here) is defined as the
relative strength of the Rossby versus Kelvin wave surface wind velocities and
written as the ratio of the maximum westerly speed $U_{max}$ versus the
minimum easterly speed $U_{min}$. The zonal asymmetry is found to be a major
ingredient in realistic GCM simulations of the MJO propagation.

We suggest that the RK index is related to the propagation speed $c$ due to
the zonal surface wind $u_{s}$ in the meridional circulation cells. Using
$\eta\propto-u_{s}$ in (16) the RK index is given by the ratio of eastern to
western meridional vorticities $\eta_{E}$ and $\eta_{W}$,

$\mbox{RK}=\eta_{E}/\eta_{W}.$ (17)

The RK index is negative since the vorticities have opposite signs.

For the values indicated in Fig. 2, $\eta_{W}=-2.5\cdot
10^{-3}\,\mbox{s}^{-1}$, $\eta_{E}=1.5\cdot 10^{-3}\,\mbox{s}^{-1}$, hence
$\mbox{RK}=-0.6$, which is above the observed value $-0.8$ but in the range
between $-0.6$ and $-2.4$ simulated by GCMs [30]. The speed in (16) is
linearly related to the RK index according to

$c=-\frac{s\eta_{w}}{2}(1+\mbox{RK}).$ (18)

The speed is positive here since $\eta_{w}<0$ and $\mbox{RK}>-1$. An
unrealistic westward propagation of the MJO is given for $\mbox{RK}<-1$.

Figure 2:  Burgers kink solution for $\eta$ (Eq. 14, solid) with the nonlinear
buoyancy term $-b_{x}$ (Eq. 11, dashed), and the constant term $-c/s$ (dash-
dotted), which causes the east-west asymmetry. The parameters are: observed
propagation speed $c=5\,\mbox{ms}^{-1}$, scale $s=10^{4}\,\mbox{m}$, diffusion
coefficient $D=10^{8}\,\mbox{m}^{2}\mbox{s}^{-1}$, and inverse width
$\lambda=1/L_{x}=10^{-7}\,\mbox{m}^{-1}$. The displayed functions are
nondimensional with the scale $10^{-3}\,\mbox{s}^{-1}$.

##  4 Summary and Discussion

A conceptual mechanism for the propagation of a large scale convective event
like the Madden-Julian Oscillation (MJO) in the Indian Ocean is suggested. The
dynamics is determined by the meridional vorticity equation in the vertical
$x$-$z$-plane in the Boussinesq approximation. Buoyant convection is
parametrized resorting to Convective Available Potential Energy (CAPE) in
terms of the vorticity and its horizontal derivative, thus it can be
considered heuristically as the product of the moisture transport times
vertical velocity. The product leads to a phase shift in the buoyancy which
causes the eastward progression. Clearly, the model depends crucially on this
purely mechanistic parametrization of convection which allows to describe the
phenomenon by a single one-dimensional equation, whose usefulness is
demonstrated a posteriori.

The evolution equation for the first baroclinic mode is the one-dimensional
nonlinear Burgers equation where nonlinear steepening is balanced by
horizontal diffusion. The exact stationary solution is a self-sustained
perturbation in the form of a smooth step (kink) in the vorticity propagating
towards the moisture source with a speed proportional to the mean meridional
vorticity in the vicinity of the MJO, $\bar{\eta}\propto-c$. Due to its large
scale structure the kink solution should be considered as an envelope for the
meridional vorticity. To establish a connection with equatorial wave theory,
the kink could be considered as a slowly moving (quasi stationary) heat source
in Gill’s shallow water model [7].

A first important property of the Burgers kink solution is the zonal asymmetry
of the meridional vorticity with lower absolute eastern than western values,
$\eta_{E}<-\eta_{W}$, hence a point symmetry is absent in the exact solution.
This asymmetry is known in the observed lower tropospheric zonal wind [26]. We
hypothesize that slower propagation complies with a more zonally symmetric
lower boundary, for example when the MJO reaches the maritime continent [35].

In [30] the zonal asymmetry is attributed to stronger eastward surface winds
in Kelvin waves than westward winds in Rossby waves and assessed by a Rossby-
Kelvin intensity index which is $\mbox{RK}=\eta_{E}/\eta_{W}$ in terms of
eastern and western meridional vorticities. Since [7] obtained the zonal
asymmetry in the surface wind for a stationary heat source, the observed
relation between the RK index and the MJO speed might be additionally affected
by the circulation cells considered in the present study.

A second important result of the present conceptual model is the predicted
convective precipitation in a narrow equatorial band eastward of the MJO (for
an easterly moisture source) which is not correlated with Kelvin and Rossby
waves [5]. The eastward location of precipitation can be associated with
positive values of the buoyancy term $-b_{x}$ (compare also the asymmetric
structure of convective instability in Fig. 4 of [30]).

The present model puts the MJO in the framework of circulation cells like the
Walker circulation and the El Niño/Southern Oscillation (ENSO) [2] which can
be described in terms of a meridional vorticity as well. Therefore,
interactions are straightforward, for example, the meridional vorticity and
the MJO are weakened when the upward motion over the Maritime Continent
decreases during a warm ENSO event, the opposite happens for an intensified
Walker circulation [31, 36]. In [37] the propagation speeds of convective
clusters in the Indian Ocean/West Pacific are classified for different ENSO
phases.

Presumably, the most important result of the present study is not the Burgers
equation itself, but the exact kink solution which associates the speed of the
kink with an asymmetry of the zonal circulation in the vicinity of the kink.
Remote boundary conditions for the solution are not specified here but might
be determined by planetary circulation cells like the Walker circulation and
the anomaly given by ENSO. These cells are defined or can be interrupted by
large scale regions of ascending and descending motion which appear as smooth
discontinuities (kinks) in the meridional vorticity. The model predicts that
kinks associated with convection move towards the moisture source and break
the symmetry of the circulation cells locally, while kinks in descending
regions or without convection are symmetric and do not move.

## References

## References

  * [1] Roland A Madden and Paul R Julian.  Detection of a 40–50 day oscillation in the zonal wind in the tropical Pacific.  Journal of Atmospheric Sciences, 28(5):702–708, 1971.
  * [2] Chidong Zhang.  Madden-Julian oscillation.  Reviews of Geophysics, 43(2), 2005.
  * [3] Taroh Matsuno.  Quasi-geostrophic motions in the equatorial area.  Journal of the Meteorological Society of Japan. Ser. II, 44(1):25–43, 1966.
  * [4] Nedjeljka Žagar and Christian LE Franzke.  Systematic decomposition of the Madden-Julian oscillation into balanced and inertio-gravity components.  Geophysical Research Letters, 42(16):6829–6835, 2015.
  * [5] Chidong Zhang and Jian Ling.  Potential vorticity of the Madden–Julian oscillation.  Journal of the Atmospheric Sciences, 69(1):65–78, 2012.
  * [6] Min-Seop Ahn, Daehyun Kim, Kenneth R Sperber, In-Sik Kang, Eric Maloney, Duane Waliser, and Harry Hendon.  MJO simulation in CMIP5 climate models: MJO skill metrics and process-oriented diagnosis.  Climate Dynamics, 49(11):4023–4045, 2017.
  * [7] Adrian E Gill.  Some simple solutions for heat-induced tropical circulation.  Quarterly Journal of the Royal Meteorological Society, 106(449):447–462, 1980.
  * [8] C Zhang, ÁF Adames, B Khouider, B Wang, and D Yang.  Four theories of the Madden–Julian oscillation.  Reviews of Geophysics, 58(3):e2019RG000685, 2020.
  * [9] Jun-Ichi Yano.  MJO as a Gestalt.  Bulletin of the American Meteorological Society, 103(2):E455 – E462, 2022.
  * [10] Carl Eckart and J Gillis.  Hydrodynamics of Oceans and Atmospheres.  Physics Today, 14(8):52, 1961.
  * [11] Boualem Khouider, Andrew J Majda, and Samuel N Stechmann.  Climate science in the tropics: waves, vortices and pdes.  Nonlinearity, 26(1):R1, 2012.
  * [12] Jun-Ichi Yano and Joseph J Tribbia.  Tropical atmospheric Madden–Julian oscillation: A strongly nonlinear free solitary Rossby wave?  Journal of the Atmospheric Sciences, 74(10):3473–3489, 2017.
  * [13] Bowen Zhao, Vladimir Zeitlin, and Alexey V Fedorov.  Equatorial modons in dry and moist-convective shallow-water systems on a rotating sphere.  Journal of Fluid Mechanics, 916, 2021.
  * [14] Sulian Thual, Andrew J. Majda, and Samuel N. Stechmann.  A stochastic skeleton model for the mjo.  Journal of the Atmospheric Sciences, 71(2):697 – 715, 2014.
  * [15] Qiu Yang, Andrew J Majda, and Nan Chen.  Enso diversity in a tropical stochastic skeleton model for the mjo, el niño, and dynamic walker circulation.  Journal of Climate, 34(9):3481–3501, 2021.
  * [16] Shuyi S Chen and Robert A Houze Jr.  Diurnal variation and life-cycle of deep convective systems over the tropical Pacific warm pool.  Quarterly Journal of the Royal Meteorological Society, 123(538):357–388, 1997.
  * [17] Matthew R Igel and Joseph A Biello.  The nontraditional Coriolis terms and tropical convective clouds.  Journal of the Atmospheric Sciences, 77(12):3985–3998, 2020.
  * [18] Akira Kasahara.  On the nonhydrostatic atmospheric models with inclusion of the horizontal component of the earth’s angular velocity.  Journal of the Meteorological Society of Japan. Ser. II, 81(5):935–950, 2003.
  * [19] Michiya Hayashi and Hisanori Itoh.  On the importance of the nontraditional Coriolis terms in large-scale motions associated with cumulus convective forcing in the tropic: Forced response problem.  Journal of the Atmospheric Sciences, 69:2699–2716, 2012.
  * [20] Andy A White and RA Bromley.  Dynamically consistent, quasi-hydrostatic equations for global models with a complete representation of the Coriolis force.  Quarterly Journal of the Royal Meteorological Society, 121(522):399–418, 1995.
  * [21] T Gerkema, JTF Zimmerman, LRM Maas, and H Van Haren.  Geophysical and astrophysical fluid dynamics beyond the traditional approximation.  Reviews of Geophysics, 46(2), 2008.
  * [22] Michael JP Cullen.  The unified forecast/climate model.  Meteorological Magazine, 122(1449):81–94, 1993.
  * [23] Sebastian Borchert, Guidi Zhou, Michael Baldauf, Hauke Schmidt, Günther Zängl, and Daniel Reinert.  The upper-atmosphere extension of the ICON general circulation model (version: ua-icon-1.0).  Geoscientific Model Development, 12(8):3541–3569, 2019.
  * [24] Günther Zängl, Daniel Reinert, Pilar Rípodas, and Michael Baldauf.  The ICON (ICOsahedral Non-hydrostatic) modelling framework of DWD and MPI-M: Description of the non-hydrostatic dynamical core.  Quarterly Journal of the Royal Meteorological Society, 141(687):563–579, 2015.
  * [25] Hing Ong and Paul E Roundy.  The compressional beta effect: Analytical solution, numerical benchmark, and data analysis.  Journal of the Atmospheric Sciences, 77(11):3721–3732, 2020.
  * [26] Pang Chi Hsu and Tim Li.  Role of the boundary layer moisture asymmetry in causing the eastward propagation of the Madden–Julian oscillation.  Journal of Climate, 25(14):4914–4931, 2012.
  * [27] Guang J Zhang and Mingquan Mu.  Effects of modifications to the Zhang-McFarlane convection parameterization on the simulation of the tropical precipitation in the National Center for Atmospheric Research Community Climate Model, version 3\.  Journal of Geophysical Research: Atmospheres, 110(D9), 2005.
  * [28] A Narendra Babu, JB Nee, and Karanam Kishore Kumar.  Seasonal and diurnal variation of convective available potential energy (CAPE) using COSMIC/FORMOSAT-3 observations over the tropics.  Journal of Geophysical Research: Atmospheres, 115(D4), 2010.
  * [29] Valérian Jewtoukoff, Riwal Plougonven, and Albert Hertzog.  Gravity waves generated by deep tropical convection: Estimates from balloon observations and mesoscale simulations.  Journal of Geophysical Research: Atmospheres, 118(17):9690–9707, 2013.
  * [30] Bin Wang and Sun-Seon Lee.  MJO Propagation shaped by zonal asymmetric structures: Results from 24 GCM simulations.  Journal of Climate, 30(19):7933 – 7952, 2017.
  * [31] Tamaki Suematsu and Hiroaki Miura.  Changes in the eastward movement speed of the Madden–Julian oscillation with fluctuation in the Walker circulation.  Journal of Climate, 35(1):211–225, 2022.
  * [32] Hing Ong and Da Yang.  The compressional beta effect and convective system propagation.  Journal of the Atmospheric Sciences, 79(8):2031 – 2040, 2022.
  * [33] Edward R Benton and George W Platzman.  A table of solutions of the one-dimensional Burgers equation.  Quarterly of Applied Mathematics, 30(2):195–212, 1972.
  * [34] Mayur P Bonkile, Ashish Awasthi, C Lakshmi, Vijitha Mukundan, and VS Aswin.  A systematic literature review of burgers’ equation with recent advances.  Pramana, 90:1–21, 2018.
  * [35] Justin Hudson and Eric Maloney.  The role of surface fluxes in MJO propagation through the maritime continent.  Journal of Climate, 36(6):1633–1652, 2023.
  * [36] Tamaki Suematsu, Hiroaki Miura, Chihiro Kodama, and Daisuke Takasuka.  Deceleration of Madden–Julian oscillation speed in NICAM AMIP-type simulation associated with biases in the Walker circulation strength.  Geophysical Research Letters, page e2022GL098628, 2022.
  * [37] Eriko Nishimoto and Masato Shiotani.  Intraseasonal variations in the tropical tropopause temperature revealed by cluster analysis of convective activity.  Journal of Geophysical Research: Atmospheres, 118(9):3545–3556, 2013.


# ATTENTION TOWARD NEIGHBORS: A CONTEXT AWARE FRAMEWORK FOR HIGH RESOLUTION
IMAGE SEGMENTATION

###### Abstract

High-resolution image segmentation remains challenging and error-prone due to
the enormous size of intermediate feature maps. Conventional methods avoid
this problem by using patch based approaches where each patch is segmented
independently. However, independent patch segmentation induces errors,
particularly at the patch boundary due to the lack of contextual information
in very high-resolution images where the patch size is much smaller compared
to the full image. To overcome these limitations, in this paper, we propose a
novel framework to segment a particular patch by incorporating contextual
information from its neighboring patches. This allows the segmentation network
to see the target patch with a wider field of view without the need of larger
feature maps. Comparative analysis from a number of experiments shows that our
proposed framework is able to segment high resolution images with
significantly improved mean Intersection over Union and overall accuracy.

Index Terms—  High-resolution image segmentation, Spatial attention,
Contextual information.

##  1 Introduction

Image segmentation is one of the fundamental and challenging problems in
computer vision, where the goal is to classify each pixel to a particular
category. With advances in computing technology as well as deep learning
techniques, it has been possible to gain significant improvement in this
field. However, segmentation of high resolution images (i.e., satellite
images, x-ray images etc.) still remains challenging due to their large
spatial size. If the entire resolution images are used as inputs, the enormous
size of intermediate feature maps makes it impractical to conduct training. To
tackle this problem, most of the existing frameworks deal with high resolution
image segmentation by downsampling the image [1][2] or by dividing the input
image into square patches [3][4][5]. The latter method yields better results
because it does not lose any visual information.

Particularly, in the patch based methods, every patch is segmented
independently without any contextual information from the adjacent patches.
Authors in [3] use different patch sizes (224×224, 112×112 etc.) as input.
Later, they have used weighted summation of output probability vectors to
classify each pixel. The most recent work [6] uses modified UNet [7] to
segment satellite images by supplying 256×256 sized independent patches as
input. The patch based methods result erroneous segmentation, particularly
while working with very high resolution images where the patches are quite
small compared to the full image. The errors are more prominent in the patch
boundaries, where little information is present in the independent patches to
classify the near boundary pixels. Some researches simply downsample the full-
size image. Authors in [1] and [2] downsample the 1024×1024 images to 256×256
size. However, downsampling high resolution images causes significant
information loss that contributes to the segmentation errors. Additionally,
there exists a number of works [8][9][10] that have applied different
attention mechanisms to improve classification and segmentation tasks.
However, these methods only work to improve performance on independent
patches.

Fig. 1: Fusing aggregated contextual information from the feature points of
neighboring patches using spatial attention. (Only two neighbor connections
are shown for simplicity) Fig. 2: Illustration of Proposed Method

To overcome the shortcoming of existing methods for high resolution image
segmentation, we propose a framework to segment a particular patch by
incorporating context information held by its adjacent patches. Specifically,
our framework learns spatial interdependencies of features between patches in
a neighborhood. Exploiting these interdependencies to fuse long-range
contextual information would facilitate better segmentation, particularly at
the boundary of the targeted patch. The overview of the proposed mechanism is
depicted in figure 2 where all feature points from each of the neighbors are
fused using spatial attention to compute the contextual information for the
targeted feature point (white circle in the patch I) in the targeted patch I.
This operation is performed for each feature point in patch I. Thus, the
proposed framework allows the network to see the target patch with a wider
field of view. As a result, the network is more confident about the patch it
is segmenting. Moreover, the proposed model does not require increasing the
spatial size of the intermediate feature maps. We know, large feature map can
limit the training in larger batch sizes. In summary, our contributions are as
follows:

  * •

We propose an attention-based novel framework that can easily improve the
patch based methods by capturing contextual information from neighboring
patches. Proposed framework can be integrated to any encoder-decoder based
segmentation architecture.

  * •

Our proposed framework achieves improved segmentation result than baseline on
couple of publicly available datasets: JSRT Chest X-ray dataset [11] and Dhaka
Satellite Dataset (DSD)111https://ovipaul.github.io/Bangladesh-LULC.

  * •

We obtain the state of the art segmentation result on GID dataset [3].

##  2 Proposed Method

Table 1: Baseline Comparison for JSRT Dataset. (’w/o’ = without our framework, ’w’ = with our framework) | Heart | Clavicle | Lung | mIoU/OA
---|---|---|---|---
w/o | w | w/o | w | w/o | w | w/o | w
64X64 Deeplab | IoU | 0.4184 | 0.4804 | 0.1850 | 0.1826 | 0.5865 | 0.5940 | 0.3966 | 0.4190
Acc | 0.6299 | 0.6728 | 0.3924 | 0.3526 | 0.7389 | 0.7502 | 0.6976 | 0.7136
64X64 FCN | IoU | 0.3934 | 0.4784 | 0.1681 | 0.1768 | 0.5856 | 0.5824 | 0.3824 | 0.4125
Acc | 0.6312 | 0.7152 | 0.3517 | 0.4045 | 0.7451 | 0.7324 | 0.7005 | 0.7128
128X128 Deeplab | IoU | 0.5066 | 0.5347 | 0.1981 | 0.2065 | 0.5968 | 0.5916 | 0.4338 | 0.4443
Acc | 0.6922 | 0.7593 | 0.3658 | 0.4251 | 0.7456 | 0.7472 | 0.7153 | 0.7345
128X128 FCN | IoU | 0.4488 | 0.4860 | 0.1971 | 0.2000 | 0.5940 | 0.5909 | 0.4133 | 0.4256
Acc | 0.6125 | 0.6678 | 0.3715 | 0.3918 | 0.7481 | 0.7519 | 0.6994 | 0.7156
256X256 Deeplab | IoU | 0.5993 | 0.6124 | 0.2136 | 0.2044 | 0.6323 | 0.6389 | 0.4817 | 0.4853
Acc | 0.7975 | 0.7779 | 0.4290 | 0.3710 | 0.7566 | 0.7811 | 0.7480 | 0.7562
256X256 FCN | IoU | 0.5322 | 0.5428 | 0.2146 | 0.2137 | 0.6286 | 0.6274 | 0.4584 | 0.4613
Acc | 0.7686 | 0.7666 | 0.4443 | 0.4679 | 0.7700 | 0.7726 | 0.7505 | 0.7534
512X512 Deeplab | IoU | 0.5392 | 0.5527 | 0.2012 | 0.2101 | 0.5985 | 0.5985 | 0.4463 | 0.4537
Acc | 0.7297 | 0.7640 | 0.3700 | 0.4312 | 0.7456 | 0.7395 | 0.7240 | 0.7303
512X512 FCN | IoU | 0.4907 | 0.4954 | 0.1982 | 0.2034 | 0.5952 | 0.5941 | 0.4281 | 0.4309
Acc | 0.7159 | 0.7138 | 0.3753 | 0.3930 | 0.7491 | 0.7525 | 0.7236 | 0.7265

Here, we describe the details of our proposed method. We first divide the full
image into non-overlapping square patches. The objective is to segment each
patch. However, instead of segmenting each patch independently, we propose to
capture the context information held by the patches that are immediate
neighbors. Let, the patch we are going to segment be denoted as I. The patch I
has eight neighbor patches. For the target patches that lie at borders of the
full image, we have used zero-valued patches to extrapolate the missing
neighbors.

At first, the patch I is fed to the encoder. We get an output $I_{e}\in
R^{1\times C\times H\times W}$, which is the encoded version of patch I.
$I_{e}$ is then reshaped to $1\times C\times HW$. After that, the neighbors
are put into the encoder keeping the gradients of encoder weights and biases
frozen, i.e., the weights and biases of encoder are not updated during
training for this particular step. The output $N_{e}\in R^{8\times C\times
H\times W}$ is later reshaped to $8\times C\times HW$. We then concatenate
$I_{e}$ to $N_{e}$ because patch $I$ needs to get a global view of the
neighboring patches, as well as the whole patch of itself. Now, to calculate
the feature interdependencies between $N_{e}$ and $I_{e}$, we multiply
$I_{e}^{T}\in R^{1\times HW\times C}$ with $N_{e}$ and then apply softmax to
the last axis to get the weight matrix $W_{c}\in R^{9\times HW\times HW}$.
This weight matrix is significant for our objective. It consists of 9
matrices. Let, each element of $W_{c}$ be denoted as $w_{ji_{k}}$; where $k\in
1,2...9$. Then, $w_{ji_{k}}$ measures the impact of $i^{th}$ position of
$N_{e_{k}}$ on $j^{th}$ position of $I_{e}$. More similar feature
representations of the two positions contribute to greater correlation between
them. These weight values would act like gates that control the flow of
spatial contextual information coming from the neighbors.

Now, we need to fuse contextual information held by the neighboring patches to
$I_{e}$. To do that, we first multiply $N_{e}$ with the transpose of weight
matrix to get weighted neighbors. Then, we reshape the weighted neighbors to a
size of $9\times C\times H\times W$ to get the final weighted neighbors
$N_{w}$. Each pixel position for each neighbor $N_{w_{k}}$ is a linear
combination of rest of the pixel positions from that particular neighbor
$N_{w_{k}}$. The weights of the linear combination is the weight values from
$W_{c}$.

In the decoding procedure, $N_{w}$ is first summed through it’s batch
dimension. $N_{w}$ is then elementwise multiplied with a learnable parameter
$\alpha$ of size $H\times W$, which is initialized with a value of 0. After
that, element wise summation is performed between $I_{e}$ and $N_{w}$ to get
$D_{e}$. This way, each pixel position of $D_{e}$ has the information from
patch I and the contextual information from all the neighbors surrounding the
patch I. So, $D_{e}$ now has a global view of the whole 9 patches. Finally,
$D_{e}$ is put into the decoder, where it is upsampled to the original
resolution of the patch I.

##  3 Experiments

Our experimental objective is composed of two parts. At first, we compare our
method against basic patch-based approach using baseline segmentation
architectures to show the efficacy of our proposed framework. We use two
datasets, 1) JSRT Chest X-ray [11] and 2) Dhaka Satellite Data (DSD). Next, we
observe how our method advances the current state of the art results on GID
Satellite Dataset [3]. We compare our method against their multi-patch based
method. All experiments are performed using Pytorch implementation.

Table 2: Baseline Comparison for Dhaka Satellite Dataset | Forest | Built-up | Water | Farmland | Meadow | mIoU/OA
---|---|---|---|---|---|---
w/o | w | w/o | w | w/o | w | w/o | w | w/o | w | w/o | w
56X56 Deeplab | IoU | 0.3783 | 0.3816 | 0.5673 | 0.5861 | 0.4572 | 0.4646 | 0.5700 | 0.5849 | 0.2451 | 0.2700 | 0.4436 | 0.4574
Acc | 0.5790 | 0.5561 | 0.6945 | 0.7243 | 0.6801 | 0.7254 | 0.7265 | 0.7292 | 0.3992 | 0.4306 | 0.6499 | 0.6651
56X56 FCN | IoU | 0.3168 | 0.3319 | 0.5315 | 0.5346 | 0.3729 | 0.3830 | 0.5178 | 0.5349 | 0.1845 | 0.1627 | 0.3847 | 0.3894
Acc | 0.6722 | 0.6187 | 0.6419 | 0.6457 | 0.5645 | 0.6432 | 0.6767 | 0.7248 | 0.2626 | 0.2240 | 0.6005 | 0.6114
112X112 Deeplab | IoU | 0.3299 | 0.3774 | 0.5498 | 0.5645 | 0.4274 | 0.4426 | 0.5659 | 0.5395 | 0.2502 | 0.3187 | 0.4246 | 0.4486
Acc | 0.5246 | 0.6418 | 0.7132 | 0.6707 | 0.6292 | 0.6013 | 0.6948 | 0.6052 | 0.4061 | 0.6648 | 0.6365 | 0.6394
112X112 FCN | IoU | 0.3260 | 0.3282 | 0.4888 | 0.5495 | 0.3705 | 0.3819 | 0.4843 | 0.5413 | 0.2513 | 0.2110 | 0.3842 | 0.4024
Acc | 0.7173 | 0.6791 | 0.5524 | 0.6643 | 0.7553 | 0.5559 | 0.5675 | 0.6889 | 0.4608 | 0.3063 | 0.5764 | 0.6186
224X224 Deeplab | IoU | 0.3590 | 0.3672 | 0.5869 | 0.5859 | 0.3817 | 0.4404 | 0.5569 | 0.5643 | 0.2556 | 0.2899 | 0.4280 | 0.4496
Acc | 0.6331 | 0.6593 | 0.7296 | 0.7385 | 0.6404 | 0.6490 | 0.6775 | 0.6659 | 0.3788 | 0.4389 | 0.6458 | 0.6566
224X224 FCN | IoU | 0.3489 | 0.3581 | 0.5264 | 0.5559 | 0.3969 | 0.4025 | 0.5550 | 0.5559 | 0.2508 | 0.2591 | 0.4156 | 0.4263
Acc | 0.7389 | 0.6778 | 0.5967 | 0.6544 | 0.6215 | 0.7151 | 0.6939 | 0.6868 | 0.3962 | 0.3941 | 0.6214 | 0.6359
448X448 Deeplab | IoU | 0.3449 | 0.3577 | 0.5452 | 0.5598 | 0.4065 | 0.4443 | 0.4826 | 0.5591 | 0.2477 | 0.2720 | 0.4054 | 0.4386
Acc | 0.7238 | 0.6307 | 0.6590 | 0.6934 | 0.6565 | 0.6836 | 0.5701 | 0.6503 | 0.4222 | 0.4711 | 0.6025 | 0.6393
448X448 FCN | IoU | 0.3365 | 0.3431 | 0.5560 | 0.5607 | 0.3999 | 0.4005 | 0.5340 | 0.5214 | 0.2415 | 0.2791 | 0.4136 | 0.4209
Acc | 0.5474 | 0.6041 | 0.6859 | 0.7031 | 0.7447 | 0.7247 | 0.6542 | 0.5947 | 0.4074 | 0.4896 | 0.6228 | 0.6247

Performance Metrics: We report our performance using intersection over union
(IoU), mean intersection over union (mIoU), class specific accuracy (Acc) and
overall accuracy (OA) [3]. IoU is defined as

$IoU_{j}=\frac{\sum_{i=1}^{n}TP_{ij}}{\sum_{i=1}^{n}(TP_{ij}+FP_{ij}+FN_{ij})}$

$TP_{ij}$, $FP_{ij}$, and $FN_{ij}$ are correspondingly the number of pixels
in image $i$, which are correctly predicted as class $j$, incorrectly
predicted as class $j$, incorrectly predicted as any other class than class
$j$, respectively. The mIoU is defined as the average IoU among all classes.

Let $P_{ab}$ denote the number of pixels of class a predicted to belong to
class b, and let $t_{a}=\sum_{b}P_{ab}$ be the total number of pixels
belonging to class $a$. Let, $t_{b}=\sum_{a}P_{ab}$ be the total number of
pixels predicted to class $b$. Class specific accuracy is defined as the
percentage of correctly classified pixels for each class. Overall Accuracy
(OA) is the percentage of correctly classified pixels and all pixels in the
entire image.

$Accuracy=\frac{P_{aa}}{t_{a}}\;\;\;\;;\;\;\;\;OA=\frac{\sum_{a}P_{aa}}{\sum_{a}t_{a}}$

In all the experiments the background class is discarded during the
calculation of the metrics.

Comparing Baselines: Here, we compare our method against basic patch based
approach. We first use the JSRT Chest X-ray dataset [11]. Segmentation in
Chest Radiographs [12] contains manual segmentation for lung fields, heart,
and clavicles of JSRT Dataset. The images are 1024×1024 in resolution. We show
the comparative results with different patch sizes to demonstrate the
significance of our framework. Since this dataset does not have any explicit
test set, we have performed each experiment with three fold cross validation.
Here, we report the average scores of the three runs. We use two different
segmentation architectures: FCN-32 [13] and Deeplab v3+ [14]. We select a
comparatively older and a comparatively newer architecture to verify the
effectiveness of our method. Deeplab v3+ has explicit encoder and decoder
blocks. For FCN-32, we use the only upsampling layer as decoder and the rest
as encoder. We perform the experiment with Adam optimizer using a learning
rate of 0.0001. For the first three patch sizes (64×64, 128×128, 256×256), we
use a batch size of 32. For 512×512 patch size we use batchsize of 8 due to
memory constraint. We run the training for 30 epochs. The results are reported
in table 1. It is easy to note that both the FCN 32 and Deep Lab v3+ work well
with our proposed context aware framework. The mIoU and overall accuracy
values indicate the superior performance of our proposed method.

Next, we show baseline comparison for Dhaka Satellite Dataset (DSD). It has a
single image with resolution 51146×15233. To increase diversity of our
experiment, we opt for four different patch sizes than JSRT experiment. DSD
has an explicit test set. We use both FCN-32 and Deeplab v3+ for this
experiment with Adam optimizer with a learning rate of 0.0001. We run each
experiment for 20 epochs. For both JSRT and DSD, we use pixel-wise cross-
entropy loss. From table 2, it is easy to notice that the IoUs are
significantly better with our proposed approach than the baseline models for
five classes for most of the experiments. The mIoU and OAs are always improved
with the proposed framework.

For both JSRT and DSD datasets, mIoU and OA increase for all patch sizes using
our proposed framework. However, for DSD, the improvements are more prominent
because DSD has much higher resolution images than JSRT. Therefore, it gets
more advantage from the contextual information. One significant observation is
that patch based methods struggle when the patch size is very small compared
to the full image. The reason is that the ratio between total number of border
pixels and non-border pixels is high for smaller patches. So, for patch based
method, the errors at border pixels add up, resulting in performance drop.
However, proposed context aware framework does not suffer from this
limitation; hence it achieves better segmentation for smaller patches.

Table 3: Comparison with GID (Classwise Accuracy) Methods | Built-up | Farmland | Forest | Meadow | Water
---|---|---|---|---|---
MLC+Fusion | 61.81 | 67.38 | 36.15 | 2.92 | 82.09
RF+Fusion | 62.61 | 71.12 | 36.10 | 3.94 | 84.29
SVM+Fusion | 61.28 | 72.27 | 23.01 | 2.26 | 54.18
MLP+Fusion | 58.69 | 72.40 | 32.86 | 2.67 | 83.99
PT-GID | 88.42 | 91.85 | 79.42 | 70.55 | 87.60
Our Method | 97.57 | 92.97 | 84.64 | 59.12 | 95.03

Comparing with State of the Art: We now validate our proposed framework on GID
satellite dataset [3]. GID has a total of 150 very high resolution satellite
images, each of them has a size of 7200×6800. Because of such wide variety of
high resolution images, this dataset is very much suitable for our experiment.
The authors divided the dataset into 120 train images and 30 test images. We
further divide the train set into 80% train and 20% validation set. We use
Deeplab v3+ as our segmentation architecture and a patch size of 224×224 as it
showed the best performance on DSD. We train for 15 epochs with a learning
rate of 0.0001 using pixel-wise cross-entropy loss. In table 3, we show our
result. The results of the competing methods are directly taken from [3]. The
performance metric is the percentage class wise accuracy as used in [3]. It is
observed that our method advances the current state of the art accuracy on
four out of five classes. Accuracy on meadow class is less because it has the
least number of pixels in the training set and we did not perform any data
augmentation.

##  4 Conclusion

In this paper, we have presented a novel framework that fuses contextual
information from neighboring patches to aid in segmentation. This way, the
network has a wider view of the target patch and becomes more confident on the
segmentation task, which is reflected in our diverse experiments. Our
framework has also advanced the state of the art accuracy on GID dataset.

##  5 Acknowledgment

This project is supported by ICT Division - Government of Bangladesh and
Independent University Bangladesh (IUB).

## References

  * [1] Alexey A. Novikov, Dimitrios Lenis, David Major, Jiri Hladůvka, Maria Wimmer, and Katja Bühler,  “Fully convolutional architectures for multi-class segmentation in chest radiographs,” 2018.
  * [2] Oscar Gómez, Pablo Mesejo, Oscar Ibáñez, Andrea Valsecchi, and Oscar Cordón,  “Deep architectures for high-resolution multi-organ chest x-ray image segmentation,”  Neural Computing and Applications, pp. 1–15, 2019.
  * [3] Xin-Yi Tong, Gui-Song Xia, Qikai Lu, Huanfeng Shen, Shengyang Li, Shucheng You, and Liangpei Zhang,  “Land-cover classification with high-resolution remote sensing images using transferable deep models,”  Remote Sensing of Environment, vol. 237, pp. 111322, 2020.
  * [4] Yaning Yi, Zhijie Zhang, Wanchang Zhang, Chuanrong Zhang, Weidong Li, and Tian Zhao,  “Semantic segmentation of urban buildings from vhr remote sensing imagery using a deep convolutional neural network,”  Remote Sensing, vol. 11, no. 15, pp. 1774, 2019.
  * [5] Zhengxin Zhang, Qingjie Liu, and Yunhong Wang,  “Road extraction by deep residual u-net,”  IEEE Geoscience and Remote Sensing Letters, vol. 15, no. 5, pp. 749–753, 2018.
  * [6] Zhuokun Pan, Jiashu Xu, Yubin Guo, Yueming Hu, and Guangxing Wang,  “Deep learning segmentation and classification for urban village using a worldview satellite image based on u-net,”  Remote Sensing, vol. 12, no. 10, pp. 1574, 2020.
  * [7] Olaf Ronneberger, Philipp Fischer, and Thomas Brox,  “U-net: Convolutional networks for biomedical image segmentation,”  in International Conference on Medical image computing and computer-assisted intervention. Springer, 2015, pp. 234–241.
  * [8] Jie Hu, Li Shen, and Gang Sun,  “Squeeze-and-excitation networks,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 7132–7141.
  * [9] Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon,  “Cbam: Convolutional block attention module,”  in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 3–19.
  * [10] Jun Fu, Jing Liu, Haijie Tian, Yong Li, Yongjun Bao, Zhiwei Fang, and Hanqing Lu,  “Dual attention network for scene segmentation,”  in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 3146–3154.
  * [11] J SHIRAISHI, S KATSURAGAWA, J IKEZOE, T MATSUMOTO, and T KOBAYASHI,  “Development of a digital image database for chest radiographs with and without a lung nodule: Receiver operating characteristic analysis of radiologists’ detection of pulmonary nodules,”  American journal of roentgenology (1976), vol. 174, no. 1, pp. 71–74, 2000.
  * [12] Bram Van Ginneken, Mikkel B Stegmann, and Marco Loog,  “Segmentation of anatomical structures in chest radiographs using supervised methods: a comparative study on a public database,”  Medical image analysis, vol. 10, no. 1, pp. 19–40, 2006.
  * [13] Jonathan Long, Evan Shelhamer, and Trevor Darrell,  “Fully convolutional networks for semantic segmentation,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 3431–3440.
  * [14] Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam,  “Encoder-decoder with atrous separable convolution for semantic image segmentation,”  in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 801–818.