# The thermodynamics of urban population flows

A. Hernando1, A. Plastino2, 3 1 Laboratoire Collisions, Agrégats, Réactivité,
IRSAMC, Université Paul Sabatier 118 Route de Narbonne 31062 - Toulouse CEDEX
09, France
2 National University La Plata, Physics Institute (IFLP-CCT-CONICET) C.C. 727,
1900 La Plata, Argentina
3 Universitat de les Illes Balears and IFISC-CSIC, 07122 Palma de Mallorca,
Spain

###### Abstract

Orderliness, reflected via mathematical laws, is encountered in different
frameworks involving social groups. Here we show that a thermodynamics can be
constructed that macroscopically describes urban population flows. Microscopic
dynamic equations and simulations with random walkers underlie the macroscopic
approach. Our results might be regarded, via suitable analogies, as a step
towards building an explicit social thermodynamics.

##  I Introduction

The application of mathematical models to social sciences has a long and
distinguished history 1st . One may speak of empirical data from scientific
collaboration networks cites , cites of physics journals nosotrosZ , the
Internet traffic net1 , Linux packages links linux , popularity of chess
openings chess , as well as electoral results elec1 ; ccg , urban
agglomerations ciudad ; ciudad2  and firm sizes all over the world firms . A
specially relevant issue is that of universality classes defined by to the so-
called Zipf’s law (ZL) in the cumulative distribution or rank-size
distributions zipf ; nosotrosZ ; net1 ; linux ; chess ; ciudad ; ciudad2 ;
firms ; chin ; upf ; citis . Maillart et al. linux  have found that links’
distributions follow ZL as a consequence of stochastic proportional growth.
Such kind of growth assumes that an element of the system becomes enlarged
proportionally to its size $k$, being governed by a Wiener process. The class
emerges from a condition of stationarity (dynamic equilibrium) citis . ZL also
applies for processes involving either self-similarity chess  or fractal
hierarchy chin , all of them mere examples amongst very general stochastic
ones upf . A second universality-class was found by Costa Filho et al. elec1 ,
who studied vote-distributions in Brazil’s electoral results. Therefrom emerge
multiplicative processes in complex networks ccg . Such behavior ensues as
well in i) city-population rank distributions nosotros , ii) Spanish electoral
results nosotros , and iii) the degree distribution of social networks
nosotros2 . As shown in Ref. benford2 , this universality class encompasses
Benford’s Law benford1 . In the present vein, still another kind of
idiosyncratic distribution is often reported: the log-normal one lnwiki , that
has been observed in biology (length and sizes of living tissue bio ), finance
(in particular, the Black and Scholes model black ), and firms-sizes. The
latter instance obeys Gibrat’s rule of proportionate growth gibrat , that also
applies to cities’ sizes.

Together with geometric Brownian motion, there is a variety of models arising
in different fields that yield Zipf’s law and other power laws on a case-by-
case basis ciudad ; ciudad2 ; citis ; mod1 ; exp ; renorm , as preferential
attachment net1  and competitive cluster growth ccg ; nosotros2  in complex
networks, used to explain many of the scale-free properties of social
networks. For instance, we may mention detailed realistic approaches in urban
modelling ud ; otros , opinion dynamics oppi , and electoral results elec1 ;
elec2 . Of course, the renormalization group is intimately related to scale
invariance and associated techniques have been fruitfully exploited in these
matters (as a small sample see renorm, ; voteGalam, ).

It has been recently shown, in Ref. epjb2, , that a variational principle
based on MaxEnt can be successfully applied to scale-invariant social systems.
Used in the present context, it allows for a classification of the above cited
behaviors on the basis of inferences drawn from objective observables of the
system. We had also shown emp  that including some dynamical information in
the variational scheme epjb1  one is able to reproduce the shape of empirical
city-population distributions, going beyond the customary universality classes
conventionally used in such regards. Indeed, a connection between explicit
microscopic growth equations and the macroscopic characterization exists,
illustrated for logistic-growth in Ref. logistic, . We will here describe the
manner in which the methods of that paper can be generalized to first-
principles theoretical framwework describing population flows in terms of
thermodynamic concepts.

###  I.1 Motivation, statement of the problem and goal

We are looking here for more that models: what we aim for is to discover
physical principles that may underlie some social phenomena. Our system is a
specific geographical area whose population is distributed amongst several
population-nuclei (cities, villages, towns, etc.) Each nucleus’ population is
time-dependent due to migration, birth, death, etc. Our aim is to
quantitatively describe the population-nuclei’s variation. Microscopic
variables are plentiful, but our main goal is to be able to identify
macroscopic variables that can give a reasonable account of urban population-
variations.

We will proceed in seven steps, as indicated in the scheme below:
1. Introduce the basic observables and the empirical data sets. 2. Identify the stochastic nature of the city-population growth rates. 3. Postulate dynamic microscopic equations and empirically validate them. 4. Perform numerical simulations with random walkers following these dynamical equations and parametrize the macroscopic evolution. 5. Show that equilibrium configurations of such evolutions can be predicted by MaxEnt using few macroscopic parameters. 6. Derive thermodynamic-like relations between these macro-parameters. 7. Show the applicability of our thermodynamic description by modeling empirical urban flows as an scale invariant ideal gas.

The paper is organized as follows. Step 1 is addressed in the next Section II.
Section III deals with step 2, Section IV with step 3, Section V with step 4,
and Section VI with step 5 and 6. Finally, the application is dealt with in
Section VII, and some conclusions are drawn in Section VIII.

##  II Preliminary matters

The basic ingredients we need in our approach, following Refs. emp, ; epjb1, ,
are

  * i)

$n$, the total number of “population-nuclei”;

  * ii)

$x_{i}(t)$, the population of the $i$-th nucleus at time $t$ (and
$\mathbf{x}(t)=\\{x_{i}(t)\\}_{i=1}^{n}$ a vector with all the populations);

  * iii)

$x_{0}$ and $x_{M}$, the minimum and maximum allowed nucleus’ population (in
general $x_{0}=1$ and $x_{M}=\infty$);

  * iv)

$N_{T}$, the total area’s population ($N_{T}=\sum_{i=1}^{n}x_{i}(t)$);

  * v)

$\dot{x}_{i}(t)$, the time-derivative of $x_{i}(t)$ (thus the pairs
$\\{(x_{i},\dot{x}_{i})\\}_{i=1}^{n}$ compose the “urban phase space”); and

  * v)

some a priori knowledge of the dynamics at hand, written as

$\dot{x}_{i}(t)=k_{i}(t)g_{i}[\mathbf{x}(t)]$ (1)

where $g_{i}$ are population-functions to be determined and $k_{i}(t)$ growth
rates independent of the $g_{i}$.

The raw data used in our analysis is obtained from the Spanish state institute
INEine  and cover annually the period 1996-2010 (with the exception of 1997).
It encompasses up to 8000 municipalities (the smallest Spanish administrative
unit) distributed within 50 provinces (the building blocks of the autonomous
communities). We use provinces and municipalities as the closest
representatives of the ideal of a closed system’s fundamental elements. Also
other regions of the world are used as examples along the text. In this
tableau, the total population $N_{T}$ of a province is apportioned in $n$
nucleus. The $i$-th nucleus account a population of $x_{0}\leq x_{i}(t)\leq
x_{M}$ at time $t$, which time-evolution obeys Eq. (1).

##  III The stochastic nature of population growth rates

We begin dealing with step 2 of our Scheme, saying something meaningful
concerning the form of the growth rates $k_{i}(t)$ in Eq. (1). The value of
$k_{i}$ above depends upon millions of individual decisions, so it is expected
some stochastic behavior. We should know both the average
$m_{i}=\langle\dot{x}_{i}(t)\rangle_{\delta t}$ and the standard deviation
$s_{i}=\langle(\dot{x}_{i}(t)-m_{i})^{2}\rangle_{\delta t}^{1/2}$ (for each
$i$) in a time-window ${\delta t}$ around $t$. Trying then to study the
distribution of $\xi_{i}(t)=(\dot{x}_{i}(t)-m_{i})/s_{i}$ one immediately
finds

$\displaystyle m_{i}$ $\displaystyle=$ $\displaystyle\langle
k_{i}(t)g_{i}(t)\rangle_{\delta t}$ (2) $\displaystyle=$ $\displaystyle\langle
k_{i}(t)\rangle_{\delta t}\times\langle g_{i}(t)\rangle_{\delta t},$
$\displaystyle s_{i}^{2}$ $\displaystyle=$
$\displaystyle\left\langle(k_{i}(t)g_{i}(t)-\langle
k_{i}(t)g_{i}(t)\rangle_{\delta t})^{2}\right\rangle_{\delta t}$ (3)
$\displaystyle=$ $\displaystyle\sigma_{k_{i}}^{2}\langle
g_{i}(t)\rangle_{\delta t}^{2}-\langle k_{i}(t)\rangle_{\delta
t}^{2}~{}\sigma^{2}_{g_{i}}$

with $\sigma_{k_{i}}$, $\sigma_{g_{i}}$ being the standard deviations of
$k_{i}(t)$ and $g_{i}(t)\equiv g_{i}[\mathbf{x}(t)]$, respectively. Assuming
now that the function $g_{i}$’s variation in the time-window for which one
evaluates the pair $m_{i}$ \- $s_{i}$ is negligible (i.e.,
$\sigma_{g_{i}}^{2}\ll\sigma_{k_{i}}^{2}$), to a good approximation one has

$\xi_{i}(t)=\frac{k_{i}(t)-\langle k_{i}(t)\rangle_{\delta
t}}{\sigma_{k_{i}}},$ (4)

entailing that $\xi_{i}(t)$ has null average and unit standard deviation. If
this assumption is correct the shape of the $p_{\Xi}-$distribution of the
variable $\xi_{i}(t)$ should not depend upon $x_{i}(t)$. We have verified the
hypothesis, as our first result here, with reference to all (8116) Spain’s
municipalities. Fig. 1 displays the $(x_{i},\xi_{i})-$pairs for every township
in the time-window $\delta t=15$ years. From them we evaluate appropriate
points taken at regular intervals from the cumulative distribution function of
our random variable (quantiles) as a function of the population $x$. No
apparent $x-$dependence can be detected. The overall distribution
$p_{\Xi}(\xi)$ shape looks like a normal one

$p_{\Xi}(\xi)=\frac{e^{-\xi^{2}/2}}{\sqrt{2\pi}},$ (5)

with cumulative distributions of the form

$P_{\Xi}(\xi)=\frac{1}{2}\left[1+\mathrm{erf}\left(\frac{\xi}{2}\right)\right],$
(6)

shown in Fig. 1. Save for some fluctuations, we have not found any dependence
on the shape of $p_{\Xi}(\xi)$ for the different provinces (same Fig. 1).
Accordingly, but with a grain of salt, one may speak of “universality”.
Consequently, we will consider herefrom that our variable $\xi$ can be
regarded as belonging to a Wiener process, our second result here.

##  IV Introducing microscopic equations of motion

###  IV.1 Proportional growth

We are now at step 3. For the $g_{i}$’s shape we will assume that it depends
only on its own $x_{i}$’s population, i.e., $g_{i}[\mathbf{x}(t)]\simeq
g_{i}[x_{i}(t)]$. In order to guess the explicit analytical form we appeal to
a cluster-growth model in networks,ccg ; nosotros2  used successfully before
describing city-population distributions. We firstly consider a network of
nodes (that eventually represents the social network) and a single node as
seed of a cluster. Initially, the first neighbors of the seed will belong to
the cluster with a given probability $P(t=0)$. At a subsequent time $t$, the
first neighbors of the members of the cluster become also members with
probability $P(t)$. Proceeding in this vein, it is reasonable to conjecture
that the time-variation of the cluster-size $\dot{x}$ at time $t$ acquire the
form

$\dot{x}(t)=\sum_{j=1}^{x(t)}P(t)c_{j}(t),$ (7)

where $c_{j}(t)$ is the first-neighbors-number of node $i$-th at time $t$. We
appeal now to the _central limit theorem_ to write

$\dot{x}(t)=P(t)\left(\overline{c}(t)x(t)+\sigma_{c}(t)\sqrt{x(t)}\xi(t)\right).$
(8)

Here $\overline{c}(t)$ is the mean neighbor’s number at time $t$,
$\sigma_{c}(t)$ its standard deviation, and $\xi(t)$ an independent normally-
distributed number. This last summand, usually neglected for very large sizes,
is associated to finite-size effects. The first term, size-proportional,
generates proportional (or multiplicative) growth. In view of this result, we
consider the form

$g_{i}(x_{i})=[x_{i}]^{\alpha}$ (9)

with $\alpha=1$ or $1/2$. Considering then both terms in the microscopic
dynamics we write

$\dot{x}_{i}(t)=k_{i1}(t)x_{i}(t)+k_{i1/2}(t)\sqrt{x_{i}(t)},$ (10)

with the $k_{i1}(t)$ and $k_{i1/2}(t)$ two (a priori) independent Wiener
coefficients. This dependence is checked out by comparison of the previously
employed $s_{i}-$numbers with a functional form of the type

$\displaystyle s_{i}^{2}(x_{i})$ $\displaystyle=$
$\displaystyle\langle[\dot{x}_{i}]^{2}-\langle\dot{x}_{i}\rangle_{\delta
t}^{2}\rangle_{\delta t}$ (11) $\displaystyle=$
$\displaystyle\sigma_{i1}^{2}x_{i}^{2}+\sigma_{i1/2}^{2}x_{i},$

where $\sigma_{i1}$ and $\sigma_{i1/2}$ are the associated deviations of
$k_{i1/2}$ and $k_{i1}$, respectively. Rewriting (11) in a more convenient way
we have

$s_{i}^{2}(x_{i})/x_{i}=\sigma_{i1}^{2}x_{i}+\sigma_{i1/2}^{2},$ (12)

that, for sizes small enough reduces to

$s_{i}^{2}(x_{i})/x_{i}\approx\sigma_{i1/2}^{2},$ (13)

while for very large sizes one has

$s_{i}^{2}(x_{i})/x_{i}\approx\sigma_{i1}^{2}x_{i}.$ (14)

The transition between these two regimes should take place at a value
$x_{T}=\sigma_{1/2}^{2}/\sigma_{1}^{2}$. Fig. 2 displays, as our third result,
the $(x_{i},s_{i}^{2}(x)/x_{i})-$pairs for all the Spanish municipalities,
together with appropriate quantiles. The median
$\mathrm{med}(s_{i}(x_{i})/x_{i})$ nicely fits things with
$\sigma_{i1}=0.0119$ and $\sigma_{i1/2}=0.47$. We appreciate the fact that
finite size fluctuations are larger than multiplicative ones, the later
dominating, of course, for large sizes. Our transition occurs at population-
values of the order of $1500$ inhabitants. Surprisingly enough, the
distribution of the variable
$s^{\prime}_{i}=\log[s_{i}(x_{i})/\sqrt{x_{i}}]-\log[\mathrm{med}(s_{i}(x_{i})/\sqrt{x_{i}})]$
becomes independent of $x_{i}$, being of a Gaussian nature.

At this point, we need still to address a further question. The finite-size
term average is $\langle k_{i1/2}(t)\rangle_{\delta t}=0$ (by definition), but
this is not so for the multiplicative one $\langle k_{i1}(t)\rangle_{\delta
t}\neq 0$, that is _a priori_ regarded as constant and size-independent. This
is indeed empirically true on occasions, but not always. For instance, such
assumption cannot account for the migration from the countryside to big
cities, where the mean growth rate correlates with the city-population.

###  IV.2 Taking into account internal flow

It is a fact that small populations tend to diminish while large towns tend to
increase their population. We encounter this scenario for most of the 50
provinces of Spain. We intend to tackle this issue below.

We can show that the effect can be described by recourse to a smooth
dependence of the mean relative growth $\langle\dot{x}_{i}/x_{i}\rangle$ on
$\log(\langle x\rangle)$ that generates what we will call internal flow. A
second order expansion in $\log(\langle x\rangle)$ reads

$\langle\dot{x}/x\rangle\simeq a+b\log(\langle x\rangle)+c\log(\langle
x\rangle)^{2}$ (15)

where the values of $a$, $b$ and $c$ come from the corresponding Taylor
coefficients. Assuming $b\gg c$ we can safely write it as

$\langle\dot{x}/x\rangle\simeq\langle k_{1}\rangle+\langle
k_{q}\rangle[\langle x\rangle]^{q-1},$ (16)

where we have defined for convenience $\langle k_{1}\rangle=a-b^{2}/2c$,
$\langle k_{q}\rangle=b^{2}/2c$ and $q-1=2c/b$. To validate our assumptions,
we fitted the empirical provincial data to Eq. (16) via $\langle
k_{1}\rangle$, $\langle k_{q}\rangle$ and $q$, when possible (in some cases a
quasi-linear relation is found, generating large uncertain in the optimal
values). We have found for the exponent $q$ a mean value of 1.2 and a standard
deviation of 0.45, with $|q-1|<1$ in all cases. This result confirms the
assumption $b\gg c$ validating the second-order expansion of
$\langle\dot{x}_{i}/x_{i}\rangle$. Moreover, as seen in Fig. 3 (our fourth
result), nice fits are found in general with very few exceptions.

With this new hypothesis our complete dynamic equation turns out to be

$\dot{x}_{i}(t)=k_{iq}(t)[x_{i}(t)]^{q}+k_{i1}(t)x_{i}(t)+k_{i1/2}(t)\sqrt{x_{i}(t)},$
(17)

with $k_{iq}(t)$, $k_{i1}(t)$ and $k_{i1/2}(t)$ independent (a priori) Wienner
processes. Summing up, we have assumed

  * •

a finite size term that dominates things for low population levels ($<1500$),

  * •

a multiplicative term that accounts for population’s growth/diminution
(births, death or o external migration, and

  * •

a power-law (exponent $q\sim 1$) accounting for internal migration.

Since for most of the population range only one term dominates, we will
include only one term in the considerations what follow below.

##  V From microscopic to macroscopic descriptions

We arrive to stage 4, having discussed above a microscopic population
dynamics. We will tray now to ascertain whether a macroscopic description is
also feasible. Our goal is to reduce the $2n$ microscopic degrees of freedom
to a few macroscopic ones. We will separately consider each of the three terms
of the dynamic equation. The ensuing results will be valid in the domains in
which each term dominates.

Consider $n$ random walkers characterized by a dynamic coordinate $x_{i}(t)$
obeying

$\dot{x}_{i}(t)=k_{i}(t)[x_{i}(t)]^{q},$ (18)

with
$\langle(k_{i}(t)-\overline{k})(k_{j}(t)-\overline{k})\rangle=\sigma_{k}\delta_{ij}\delta(t-t^{\prime})$.
Parameter $q$ will take as special possible values $1/2$ or $1$, or in
general, $0\leq q$.

###  V.1 Brownian motion and diffusion equation

We start with $q=0$ as control case. One has $\dot{x}_{i}(t)=k_{i}(t)$ so that
we deal with the well-known brownian random walkers. Consider this numerical
procedure: initially, the $n$ walkers are located at, say, $x=x_{0}$. By
$\rho(x,t)dx$ we will refer to the walker’s normalized histogram, at time $t$,
that indicates the walker’s relative number positioned in the interval $dx$
around $x$. The associated initial density would read
$\rho(x,0)=\delta(x-x_{0})$. A discrete version of the pertinent dynamic
equation is

$x_{i}(t+\Delta t)=x_{i}(t)+\Delta tk_{i}(t),$ (19)

that forces the walkers to “move” during the period $\Delta t$ in a amount
given by $\Delta tk_{i}(t)$, with $k_{i}(t)$ a random number generated from a
Gaussian distribution determined by an standard deviation $\sigma_{k}$ and
mean $\overline{k}$, as defined above. We have

$x_{i}(t=M\Delta t)=x_{0}+\Delta t\sum_{m=1}^{M}k_{i}[(M-1)\Delta t],$ (20)

so that after $M$ iterations the walkers-distributions coincides with that of
a random number generated by summing up $M$ Gaussian numbers characterized by
$\Delta t\sigma_{k}$ and $\Delta t\overline{k}$. Remind that a distribution
that follows a random number composed of two other numbers of that character
is the convolution of the distributions associated to these later numbers.
Thus, $x(t)$ is described by the $M-$th convolution of the $k$’s Gaussian
distribution. By recourse to a Fourier transform $\mathcal{F}$ for
convolutions we have

$\displaystyle\mathcal{F}[\rho(x,t)]$ $\displaystyle=$
$\displaystyle\left(\mathcal{F}\left[\frac{e^{-(k-\Delta
t\overline{k})^{2}/2(\Delta t\sigma_{k})^{2}}}{\sqrt{2\pi}\Delta
t\sigma_{k}}\right]\right)^{M}$ (21) $\displaystyle=$
$\displaystyle\left(e^{-\Delta t^{2}\sigma_{k}^{2}\omega^{2}/2+i\Delta
t\overline{k}\omega}\right)^{M}$ $\displaystyle=$ $\displaystyle e^{-M\Delta
t^{2}\sigma_{k}^{2}\omega^{2}/2+iM\Delta t\overline{k}\omega},$

and, appealing to the inverse transformation,

$\displaystyle\rho(x,t)$ $\displaystyle=$ $\displaystyle\frac{e^{-(k-M\Delta
t\overline{k})^{2}/(2M(\Delta t\sigma_{k})^{2})}}{\sqrt{2\pi M}\Delta
t\sigma_{k}}$ (22) $\displaystyle=$
$\displaystyle\frac{e^{-(k-t\overline{k})^{2}/(4Dt)}}{\sqrt{4\pi Dt}},$

where we have introduced for convenience $2D=\Delta t\sigma_{k}^{2}$. An
arbitrary density $\rho(x,t)$ will evolve in $\Delta t$, via the convolution
of that density with a Gaussian of deviation $\Delta t\sigma_{k}=\sqrt{2\Delta
tD}$ and mean $\Delta t\overline{k}$, as

$\displaystyle\mathcal{F}[\rho(x,t+\Delta t)]$ $\displaystyle=$
$\displaystyle\mathcal{F}[\rho(x,t)]\times e^{-\Delta tD\omega^{2}+i\Delta
t\overline{k}\omega}$ $\displaystyle\simeq$
$\displaystyle\mathcal{F}[\rho(x,t)]\left(1-\Delta tD\omega^{2}+i\Delta
t\overline{k}\omega\right),$

where we take $\Delta t$ arbitrarily small. A simple manipulation involving
division by $\Delta t$ leads now to

$\frac{\mathcal{F}[\rho(x,t+\Delta t)]-\mathcal{F}[\rho(x,t)]}{\Delta
t}=\left(-D\omega^{2}+i\overline{k}\omega\right)\mathcal{F}[\rho(x,t)].$ (24)

By recourse to the inverse transformation and taking the limit $\Delta
t\rightarrow 0$ we get

$\partial_{t}\rho(x,t)=D\partial^{2}_{x}\rho(x,t)-\overline{k}\partial_{x}\rho(x,t),$
(25)

which is a diffusion equation. Accordingly, we reach an important result here
(our fifth one):
Our original $2n$ degrees of freedom-problem can now be tackled via just a few
macroscopic parameters.

###  V.2 $q$-metric Brownian motion

In the general instance $q\neq 0$ we introduce a variable
$u_{i}=\log_{q}(x_{i})$, where $\log_{q}$ is Tsallis’ $q$-logarithm tbook .
The Jacobian for the transform is $du/dx=1/x^{q}$ so that
$\dot{u}=\dot{x}/x^{q}$ and the associated dynamical equation becomes

$\dot{u}_{i}(t)=k_{i}(t).$ (26)

In the set $\\{(u_{i},\dot{u}_{i})\\}_{i=1}^{n}$, the variables $u_{i}$ and
$\dot{u}_{i}$ are independent of each other. We regard them, of course, as our
dynamical variables. Note that one recovers Brownian motion for $u$. Indeed,

$u_{i}(t=M\Delta t)=u_{i}(0)+\Delta t\sum_{m=1}^{M}k_{i}[(M-1)\Delta t],$ (27)

and then the demonstration of the preceding subsection becomes valid, now for
$u$ and $\rho(u,t)du$. Our new diffusion equation reads

$\partial_{t}\rho(u,t)=D\partial^{2}_{u}\rho(u,t)-\overline{k}\partial_{u}\rho(u,t),$
(28)

and, starting from a density $\rho(u,0)=\delta(u-u_{0})$ we end up with

$\rho(u,t)du=\frac{du}{4\pi
Dt}\exp\left[-\frac{(u-u_{0}-\overline{k}t)^{2}}{4Dt}\right].$ (29)

The $x-$density is governed accordingly by a $q$log-normal distribution

$\displaystyle\rho_{X}(x,t)dx$ $\displaystyle=$
$\displaystyle\rho[u(x),t]\frac{dx}{du}du$ $\displaystyle=$
$\displaystyle\frac{dx}{\sqrt{4\pi
Dt}x^{q}}\exp\left[-\frac{(\log_{q}(x)-u_{0}-\overline{k}t)^{2}}{4Dt}\right].$

In particular, for $q=1/2$ one has

$\rho_{X}(x,t)dx=\frac{dx}{\sqrt{4\pi
Dtx}}\exp\left[-\frac{(2(\sqrt{x}-1)-u_{0}-\overline{k}t)^{2}}{4Dt}\right],$
(31)

and, for $q=1$ the well known log-normal

$\rho_{X}(x,t)dx=\frac{dx}{\sqrt{4\pi
Dt}x}\exp\left[-\frac{(\log(x)-u_{0}-\overline{k}t)^{2}}{4Dt}\right].$ (32)

We have again reduced the microscopic number of degrees of freedom to just a
few macroscopic parameters.

###  V.3 Examples of diffusion

Numerical experiments confirm our findings above. We start with our dynamical
equation in discrete form

$x_{i}(t+\Delta t)=x_{i}(t)+\Delta tk_{i}(t)[x_{i}(t)]^{q}$ (33)

using $k_{i}(t)=\sqrt{2D/\Delta t}\xi_{i}(t)+\overline{k},$ where the random
numbers $\xi$ follow a normal distribution such that
$\langle\xi_{i}(t)\xi_{j}(t)\rangle=\delta_{ij}\delta(t-t^{\prime})$. We have
taken $q=1/2$ and $1$ for our examples, and find that the associated
distributions exactly follow the diffusion equation’s predictions. We have
used in the former case $u_{0}=\log_{1/2}(220)$, $\overline{k}=0$, and
$\sigma_{k}^{2}=10$, in intervals of $\Delta t=0.01$. In the later instance we
had $u_{0}=\log(4400)$ instead. Indeed, the walkers’ histograms’ evolution
follow Eq. (31) and Eq. (32), respectively, with $D=\Delta t\sigma_{k}^{2}/2$
as defined above (see Fig. 4 for the cumulative distributions).

As empirical examples we discovered that for small populations $<1500$
inhabitants the finite-size noise dominates. Provinces for which most towns
are scarcely populated will obey the dynamical equation with $q=1/2$. Such is
the case for the province of, i.e., Salamanca, as shown in top panel of Fig.
4. The ensuing dynamics confirms this assertion. The relative growth of most
of the towns follows a dynamics with a variance $s^{2}\propto\sqrt{x}$ (red
line of the inset). The ensuing distribution fits the final state predicted by
the diffusion equation for that dynamics, Eq. (31), with
$u_{0}+\overline{k}t=\log_{1/2}(216.3)$ and $2Dt=95.6$ for year 2010 (see Fig.
4). Remark that the 1/2-log-normal can be easily confused with the usual log-
normal, although the former exhibits asymmetries in log-scale. As a
$q=1-$example we mention Florida State in the US usa  (see also bottom Fig.
4). Using data from 1990, 2000, and 2010, we have verified that the
microscopic dynamics confirms the proportional growth assumption (with a
variance of the relative growth independent of the size, as illustrated in the
inset). The city-populations distribution follows a log-normal distribution,
that of Eq. (32), which can be the one pertaining to geometrical random-
walkers’ diffusion, with $u_{0}+\overline{k}t=\log(4380)$ and $2Dt=2.96$.

###  V.4 Constrained diffusion

$q$-log-normal distributions do not set any limits to population-sizes.
However, it is reasonable to assume that physical space does pose limits to a
city’s population-growth. Unlimited growth is unrealistic since in the case of
internal migrations the total population $N_{T}$ should remain constant and a
free-diffusion model is, again, unrealistic. Constrained diffusion must be
contemplated instead.

We pass now to consider numerical experiments with random walkers that fix
lower and upper bounds for population. These are denoted by $x_{0}$ and
$x_{M}$, respectively. Now, walkers “moves” leading to values outside the
range $x_{0}<x<x_{M}$ are to be rejected in our simulations. Fig. 5 shows that
a $q$-metric walkers’ evolution begins by faithfully following the diffusion
equation Eq. (28) till they bump off these extreme values. Now their density
deviates from that of “free” evolution. After some time has elapsed, an
equilibrium $x-$distribution is reached that follows a power-law with exponent
$q$, independently of the initial state. _The origin of this systematic result
can not be unraveled by the simulations, so a higher-level of theory is
needed._

Now we use a total population constraint. This is equivalent to make the
walkers move under the rule of a _$q$ -generalized multi-component logistic
equation_

$\dot{x}_{i}(t)=[x_{i}(t)]^{q}\left[k_{i}(t)-\frac{\sum_{i=1}^{n}k_{i}(t)[x_{i}(t)]^{q}}{\sum_{i=1}^{n}[x_{i}(t)]^{q}}\right].$
(34)

Indeed, it is easy to check that
$\partial_{t}N_{T}=\sum_{i=0}^{n}\dot{x}_{i}(t)=0$, thus preserving the value
of $N_{T}$ in time. Also the original $q$-symmetry of the dynamics is
preserved. This equation is the $q$-generalization of the scale-invariant
multi-component logistic equation presented in logistic, . Results are
displayed in Fig. 6 for $q=1$, 1.5 and 2, using $n=100000$ walkers and a total
population of $N=250000$ inhabitants (with $x_{0}=1$). Remarkably enough,
equilibrium is always reached, to a density that does not depend upon the
initial state or the $k$-parameters. The shape of the distributions resembles
$x$ power-laws with exponential cut-off. Again, the simulation can not unravel
the origin of this form. _Finding the properties and the exact analytical form
of those macroscopic equilibrium distributions is our goal in the sext
Section._

##  VI The macroscopic conundrum

We tread now step 5. Our simulations with random walkers suggest that it is
indeed possible to pass from a description that uses $2n$ microscopic
variables to a description involving just a few macroscopic parameters. The
big question is: do they behave in thermodynamic fashion, satisfying the
pertinent partial derivatives-relationships? We wish to tackle this issue now
looking for a way to reduce the number of microscopic degrees of freedom to a
few manageable macroscopic ones while keeping a coherent, reasonable
description of our system, mimicking the kind of scenario that links
statistical mechanics to thermodynamics. This requires appropriate
constraints, a topic to be addressed below by enumerating the appropriate
“social” constraints we need.

###  VI.1 Macroscopic constraints

$\bullet$ Total number of cities $n$. Since there is some confusion in the
available data about what the administrative meaning of city is, we wish to
ascertain that this issue is of no importance. Consider
$x_{i}=\sum_{j}^{n_{i}}x_{ij},$ where $n_{i}$ is the number of sub-
administrative units included in the administrative unit $i$, with $x_{ij}$
their sub-administrative populations. Considering proportional growth, we
write for the time-evolution

$\displaystyle\dot{x}_{i}(t)$ $\displaystyle=$
$\displaystyle\sum_{j}^{n_{i}}\dot{x}_{ij}(t)$ (35) $\displaystyle=$
$\displaystyle\sum_{j}^{n_{i}}k_{ij}(t)x_{ij}(t)$ $\displaystyle=$
$\displaystyle\frac{\sum_{j}^{n_{i}}k_{ij}(t)x_{ij}(t)}{\sum_{j}^{n_{i}}x_{ij}(t)}\sum_{j}^{n_{i}}x_{ij}(t)$
$\displaystyle=$ $\displaystyle k^{\prime}_{i}(t)x_{i}(t),$

where we have defined $k^{\prime}_{i}(t)$ as a new variable defined as an
average weighted by the populations $x_{ij}$. If the growth rates $k_{ij}$ are
random variables with approximately the same mean and variance, it is easy to
check that $k^{\prime}_{i}(t)$ is in turn a random variable of the same mean
and variance. The dynamical behavior of the ensemble of administrative units
$\mathbf{x}$ is thus equivalent of that of the sub-units, and the procedure
described in this work is still applicable.

$\bullet$ Maximum/minimum population $x_{M}/x_{0}$. It is well-known that a
typical minimum population size equals the Dunbar numberdum  ($\sim 150$),
heuristically associated to the maximum (allowable by our neo-cortex) number
of stable human relationships. Thus, it is reasonable to think of a minimum
size $x_{0}\sim 150$. In many cases a maximum number for a city population
$x_{M}$ can be established via consideration of geographical peculiarities as
mountains aostamap  or oceansmarmap  (See Fig. 7 for an example). In such
cases it is convenient to employ the transform $u=\log_{q}(x/x_{0})$. An
associated, valuable macroscopic parameter is $u_{M}=\log_{q}(x_{M}/x_{0})$.
We will be dealing then with a “volume” $0<u<u_{M}$.

$\bullet$ Total population $N_{T}$. We have $N_{T}=\sum_{i=1}^{n}\,x_{i}$ that
gets transformed into $N_{T}=x_{0}\sum_{i=1}^{n}\,e_{q}^{u_{i}}$. A useful
quantity becomes then $N=N_{T}/x_{0}$.

$\bullet$ Total variance of $\dot{u}.$ With reference to the dynamics, a
useful observable is the total variance for relative growth
$\sigma^{2}=\sum_{i=1}^{n}\langle(\dot{u}_{i}-\langle\dot{u}_{i}\rangle_{t})^{2}\rangle/n$.
For a Gaussian form (see Fig. 1) this quantity measures fluctuation-
intensities. Generalizing, this quantity can be defined by the covariance
matrix with elements
$Q_{ij}=\langle(\dot{u}_{i}-\langle\dot{u}_{i}\rangle)(\dot{u}_{j}-\langle\dot{u}_{j}\rangle)\rangle$,
using its trace as a thermodynamical variable
$\mathrm{Tr}(Q)=\sum_{i=1}^{n}Q_{ii}$:

$\displaystyle U$ $\displaystyle=$ $\displaystyle\frac{\tau}{2}\mathrm{Tr}(Q)$
(36) $\displaystyle=$
$\displaystyle\frac{\tau}{2}\sum_{i=1}^{n}\langle(\dot{u}_{i}-\langle\dot{u}_{i}\rangle)^{2}\rangle$
$\displaystyle=$ $\displaystyle\frac{\tau}{2}n\sigma^{2},$

where we add for dimensional convenience a factor $\tau/2$.

###  VI.2 Fundamental hypothesis for urban-thermodynamics

Let us discuss the pertinent three hypothesis that we need in our Scheme:

$\bullet$ _H-I. Microscopic hypothesis._

We adopt as fundamental dynamical equation Eq. (18) [$\dot{x}=kx^{q}$] for the
population of a center, linearized via the variable $u=\log_{q}(x/x_{0})$. We
will think of the pair $(u,\dot{u})$ as constituting our social phase space
coordinates. We can speak of an:

$\bullet$ _H-II. A priori phase space equiprobability in $(u,\dot{u})$.katz _

The probability density distribution for the $i-$th phase space cell centered
at $(u_{i},\dot{u}_{i})$ of size $dud\dot{u}$ is defined as
$\rho[\\{(u_{i},\dot{u}_{i})\\}_{i=1}^{n}]d^{n}ud^{n}\dot{u}$. Accordingly to
H-II, the system’s entropy is written as

$S[\rho]=-\int
d^{n}ud^{n}\dot{u}~{}\rho[\\{(u_{i},\dot{u}_{i})\\}_{i=1}^{n}]\log\left[\rho[\\{(u_{i},\dot{u}_{i})\\}_{i=1}^{n}]\right].$
(37)

Since none of our macroscopic observables is able to distinguish amongst
population nuclei, towns are thus indistinguishable. In this case, the useful
distribution is the one-body density $\rho(u,\dot{u})$ defined as

$\rho(u,\dot{u})=\int
d^{n-1}ud^{n-1}\dot{u}~{}\rho[\\{(u_{i},\dot{u}_{i})\\}_{i=1}^{n}],$ (38)

and thus,

$S[\rho]=-\int dud\dot{u}~{}\rho(u,\dot{u})\log\left[\rho(u,\dot{u})\right].$
(39)

Macroscopic observables are written in terms of the one-body density as

$\displaystyle n$ $\displaystyle=$ $\displaystyle\int
dud\dot{u}~{}\rho(u,\dot{u}),$ (40) $\displaystyle N$ $\displaystyle=$
$\displaystyle\int dud\dot{u}~{}\rho(u,\dot{u})e_{q}(u),$ (41) $\displaystyle
U$ $\displaystyle=$ $\displaystyle\frac{\tau}{2}\int
dud\dot{u}~{}\rho(u,\dot{u})\dot{u}^{2}.$ (42)

$\bullet$ _H-III. Maximum entropy principle (MaxEnt).katz _

Equilibrium is determined via constrained entropic maximization using $n$,
$u_{M}$, $N$ y $U$. This determines the equilibrium density $\rho(u,\dot{u})$
that is a solution of the entropic variational problem

$\delta\left\\{S[\rho]-\beta A[\rho]\right\\}=0,$ (43)

with

$A=U-\mu n+pu_{M}+\Lambda N,$ (44)

where $\beta$, $\mu$, $p$ and $\Lambda$ stand for the pertinent Lagrange
multipliers, that will be seen below to acquire the character of intensive
thermal-quantities.

###  VI.3 Thermodynamical relations

We enter step 6 by considering the _Lagrangian_ $A[\rho]$ [and Lagrangian
density $a(u,\dot{u})$]. It reads

$\displaystyle A[\rho]$ $\displaystyle=$ $\displaystyle\int
dud\dot{u}~{}\rho(u,\dot{u})a(u,\dot{u})$ $\displaystyle=$ $\displaystyle\int
dud\dot{u}~{}\rho(u,\dot{u})\left\\{\frac{\tau}{2}\dot{u}^{2}-\mu+pv(u)+\Lambda
e_{q}(u)\right\\},$

where the volume condition is enforced by an infinite-well potential

$v(u)=\left\\{\begin{array}[]{ll}u_{M}/n&for\,\,\,0<u<u_{M};\\\
\infty&otherwise\end{array}\right.$ (46)

The well-known general solution to the entropic problem Eq. (43) iskatz
$\rho(u,\dot{u})=\exp[-\beta a(u,\dot{u})]$, so that

$\rho(u,\dot{u})=\frac{n}{Z}e^{-\frac{\beta\tau}{2}\dot{u}^{2}-\beta\Lambda
e_{q}(u)}~{}~{}(0<u<u_{M}),$ (47)

where the normalization factor $Z$ (partition function) becomes

$\displaystyle Z$ $\displaystyle=$
$\displaystyle\int_{-\infty}^{\infty}d\dot{u}\int_{0}^{u_{M}}du~{}e^{-\frac{\beta\tau}{2}\dot{u}^{2}-\beta\Lambda
e_{q}(u)}$ (48) $\displaystyle=$
$\displaystyle\sqrt{\frac{2\pi}{\beta\tau}}E_{q}(\beta\Lambda,u_{M}),$

with $E_{q}(l,m)$ the generalized exponential function of order $q$

$E_{q}(l,m)=E_{q}(l)-e^{(1-q)m}E_{q}(le^{m}).$ (49)

Our constraints in $U$ and $N$ determine the multipliers $\beta$ and
$\Lambda$-values. On the one hand, we have the $\dot{u}-$variance

$\displaystyle U$ $\displaystyle=$
$\displaystyle\frac{\tau}{2}n\int_{-\infty}^{\infty}d\dot{u}~{}\sqrt{\frac{\beta\tau}{2\pi}}e^{-\frac{\beta\tau}{2}\dot{u}^{2}}\dot{u}^{2}$
(50) $\displaystyle=$ $\displaystyle\frac{n}{2\beta},$

and on the other one, we have to deal with the total population

$\displaystyle N$ $\displaystyle=$ $\displaystyle
n\int_{0}^{u_{M}}du~{}\frac{e_{q}(u)e^{-\beta\Lambda
e_{q}(u)}}{E_{q}(\beta\Lambda,u_{M})}$ (51) $\displaystyle=$ $\displaystyle
n\frac{E_{q-1}(\beta\Lambda,u_{M})}{E_{q}(\beta\Lambda,u_{M})},$
$\displaystyle=$ $\displaystyle nF_{q}(\beta\Lambda,u_{M}),$

where we use the function $F_{q}(l,m)=\partial_{l}\log[E_{q}(l,m)]$. We obtain
from the former the direct result  $\beta=\frac{n}{2U},$ (52) and, via
inversion of the latter equation [defining first $L_{q}(f,m)=F_{q}^{-1}(f,m)$
and thus $F_{q}[L_{q}(f,m),m]=f$], we finally obtain the relation between the
system variables _(equation of state)_ $\beta\Lambda=L_{q}(N/n,u_{M}).$ (53)

Note that we have intensive quantities on the left hand side, while extensive
ones appear in the r.h.s.. The entropy becomes

$\displaystyle S$ $\displaystyle=$ $\displaystyle
n\log\left[\frac{1}{n}\sqrt{\frac{2\pi}{\beta\tau}}E_{q}(\beta\Lambda,u_{M})\right]$
(54) $\displaystyle+n\left[\frac{1}{2}+\beta\Lambda
F_{q}(\beta\Lambda,u_{M})\right].$

Using now Eq. (53) we can recast things in term of the natural variables as

$\displaystyle S(U,n,u_{M},N)$ $\displaystyle=$ $\displaystyle
n\log\left[\frac{2}{n}\sqrt{\frac{\pi
U}{n\tau}}E_{q}[L_{q}(N/n,u_{M}),u_{M}]\right]$ (55)
$\displaystyle+\frac{n}{2}+L_{q}(N/n,u_{M})N.$

It is easy to verify, but crucial to our present goals, that macroscopic
observables and Lagrange multipliers become linked entropic-wise via

$\displaystyle\beta$ $\displaystyle=$ $\displaystyle\left.\frac{\partial
S}{\partial U}\right|_{n,u_{M},N},$ (56) $\displaystyle\mu$ $\displaystyle=$
$\displaystyle\frac{1}{\beta}\left.\frac{\partial S}{\partial
n}\right|_{U,u_{M},N},$ (57) $\displaystyle p$ $\displaystyle=$
$\displaystyle\frac{1}{\beta}\left.\frac{\partial S}{\partial
u_{M}}\right|_{U,n,N},$ (58) $\displaystyle\Lambda$ $\displaystyle=$
$\displaystyle\frac{1}{\beta}\left.\frac{\partial S}{\partial
N}\right|_{U,u_{M},n}.$ (59)

The first relation leads to Eq. (52), also showing that the $\beta-$multiplier
is the inverse of the variance $\beta=1/\tau\sigma^{2}$. The last relation
takes us to Eq. (53) and is indeed one of our equations of state. The other
two are  $\displaystyle\beta\mu$ $\displaystyle=$
$\displaystyle\log\left[\frac{2}{n}\sqrt{\frac{\pi
U}{n\tau}}E_{q}[L_{q}(N/n,u_{M}),u_{M}]\right]-1$ (60)
$\displaystyle-2L_{q}^{(1,0)}(N/n,u_{M})\left(\frac{N}{n}\right)^{2}$ and

$\displaystyle\beta p$ $\displaystyle=$ $\displaystyle
n\frac{\exp[-L_{q}(N/n,u_{M})e_{q}(u_{M})]}{E_{q}[L_{q}(N/n,u_{M}),u_{M}]}$
(61) $\displaystyle+2L_{q}^{(0,1)}(N/n,u_{M})N.$

At this stage the reader will agree that it is fair to assert that our goal
has been successfully reached. _We have indeed constructed a social
thermodynamics for urban population flows_. The following equivalence may be
established vis-a vis the thermodynamics of chemical species:

• Temperature $\leftrightarrow$ Variance of relative growth. • Number of
inhabitants $\leftrightarrow$ Number of particles. • Number of towns
$\leftrightarrow$ Number of chemical species. • Volume $\leftrightarrow$
Maximum possible town’s population.

##  VII Application: the scale-free ideal gas (SFIG)

This is our final step 7. We envision two main regimes, according to the
$\Lambda-$value: $\Lambda=0$ and $\Lambda>0$.

###  VII.1 The SFIG-in-a-box

We will consider in some detail the first case here. Different scenarios can
be associated to $\Lambda\rightarrow 0$: i) the system is not isolated and
exchanges population with its surroundings, with a maximum-size constraint,
ii) the triplet $n$, $N$, $u_{M}$ is such that the equation of state yields
$\Lambda=0$, i.e., $N/n=\log_{q-1}[e_{q}(u_{M})]/u_{M}$, or iii) no size-
limitation exists ($u_{M}\rightarrow\infty$) but $N/n$ is large enough to
consider $\Lambda\sim 0$. In the latter case one can obtain an effective
$u_{M}-$value from normalization such that $u_{M}=E_{q}(\beta\Lambda)$. When
$\Lambda=0$ the Lagrangian $A$ is written as

$A=U-\mu n+pu_{M},$ (62)

so that we do not need knowledge of $N$. The equilibrium density is

$\rho(u,\dot{u})dud\dot{u}=\frac{n}{u_{M}}\sqrt{\frac{\beta\tau}{2\pi}}~{}e^{-\frac{\beta\tau}{2}\dot{u}^{2}}dud\dot{u}~{}~{}(0<u<u_{M}).$
(63)

The partial density $\rho(u)=\int d\dot{u}\rho(u,\dot{u})=n/u_{M}$ is constant
in $u$ so that $x$ is given by a power-law

$\rho_{X}(x)dx=\rho[u(x)]\frac{du}{dx}dx=n\frac{x_{0}^{q-1}}{u_{M}}\frac{dx}{x^{q}},$
(64)

with an associated rank-plot given by

$x(r)=x_{0}e_{q}[u_{M}(1-r/n)],$ (65)

where $r$ is the rank from 1 to $n$. Comparing with the equilibrium densities
found above in our numerical experiments with random walkers, a very nice fit
ensues as seen in Fig. 5, which _validates our methodology_. The entropy
becomes

$S(U,n,u_{M})=n\log\left[\frac{2}{n}\sqrt{\frac{\pi
U}{n\tau}}u_{M}\right]+\frac{n}{2},$ (66)

resembling that of the one-dimensional ideal gas. The state-equations are

$\beta\mu=\log\left[\frac{2}{n}\sqrt{\frac{\pi U}{n\tau}}u_{M}\right]-1,$ (67)

and

$\beta p=\frac{n}{u_{M}},$ (68)

in exact agreement with the ideal gas scenario.

As empirical $q=1$-examples we show the cases of i) Marshall Islands,mar  ii)
d’Agosta Valley (Italy) ita  and iii) Huelva-province (spain)ine . In all
instances the relative growth is nearly independent of the population (with
some finite-size noise) as in the case of Huelva, or with a secondary constant
trend for low-populated cities, as in d’Agosta Valley’s intance. Thus we
consider that the microscopic dynamics fits the proportional growth
hypothesis, with densities $\rho(u,\dot{u})$ nicely adapted to the ensuing
thermodynamic predictions. In all cases, geographical conditions set strong
limits to the city-sizes. The values for the macroscopic parameters are shown
in Table 1. Remarkably enough, the pressure $p$ due to the limited space is
highest for the Marshall Islands. Indeed, this system exhibits the lowest
volume $u_{M}$ and the lowest $\beta$ (highest “temperature”) for a large
number of units $n$.

| Marshall Islands | Agosta Valley | Huelva
---|---|---|---
$n$ | $160$ | $74$ | $79$
$x_{0}$ | $2.14$ | $126$ | $206$
$u_{M}$ | $0.038$ | $0.048$ | $0.059$
$\beta$ | $0.504$ | $2.05$ | $11.7$
$p$ | $0.829$ | $0.0133$ | $0.0115$
Table 1:  Macroscopic parameters for the four SFIG-in-a-box examples
($\tau=100$ years2).

###  VII.2 The SFIG under total population-constraint

We now consider $\Lambda>0$, with $u_{M}\rightarrow\infty$ for simplicity.
This case describes regions where internal migration dominates the microscopic
dynamics, and no upper limit is found for the city-size. According to the
equation of state Eq. (61), $p=0$ in this limit, so that we deal with the
lagrangian

$A=U-\mu n+\Lambda N.$ (69)

The equilibrium density is

$\rho(u,\dot{u})dud\dot{u}=\frac{n}{E_{q}(\beta\Lambda)}\sqrt{\frac{\beta\tau}{2\pi}}e^{-\frac{\beta\tau}{2}\dot{u}^{2}-\beta\Lambda
e_{q}(u)}dud\dot{u}~{}~{}(0<u),$ (70)

and the ecuation of state can be written in the form

$N=n\frac{E_{q-1}(\beta\Lambda)}{E_{q}(\beta\Lambda)}.$ (71)

The partial density for $x$ is given by a power-law with exponential cut-off

$\rho_{X}[x]dx=n\frac{x_{0}^{q-1}}{E_{q}(\beta\Lambda)}\frac{e^{-\beta\Lambda
x}}{x^{q}}dx,$ (72)

with an associated rank-plot

$x(r)=\frac{x_{0}}{\Lambda}E^{-1}_{q}\left[E_{q}(\Lambda)r/n\right].$ (73)

Again, this result fits the numerical equilibrium densities found above in our
numerical simulations (Fig. 6), _validating again our methodology_.

This is the most common situation in the Spanish provinces (for more details
see Ref. emp ). We have found nice agreement between the $q$ value obtained
from a fit to the microscopic dynamics and the $q-$value obtained from the fit
of the rank-plot to Eq. (73). We show some examples in Fig. 9, and the
associated macroscopic numerical results in Table 2. In the examples presented
below, using the parameter $\Lambda$ as a measure of the pressure generated by
the total population constraint, it turns out that Alicante is the province
with the highest pressure and Girona that with the lowest one, correlated with
a highest and a lowest ‘temperature’, respectively.

| Alicante | Almería | Girona | Lleida
---|---|---|---|---
$n$ | $140$ | $101$ | $220$ | $230$
$x_{0}$ | $83.9$ | $147$ | $141$ | $105$
$N$ | $18355.7$ | $3245.53$ | $4597.37$ | $2818.12$
$\log(\beta\Lambda)$ | $-6.26$ | $-5.35$ | $-5.18$ | $-4.02$
$\Lambda$ | $4.03~{}10^{-3}$ | $1.83~{}10^{-4}$ | $7.89~{}10^{-5}$ | $4.1~{}10^{-4}$
$\beta$ | $0.47$ | $25.9$ | $71.1$ | $43.8$
$q$ | $0.862$ | $1.135$ | $1.27$ | $1.16$
| Navarra | Vizcaya | Zaragoza | Granada
$n$ | $271$ | $111$ | $292$ | $167$
$x_{0}$ | $47.6$ | $219$ | $40.7$ | $263$
$N$ | $8976.55$ | $3571.01$ | $6906.03$ | $2539.5$
$\log(\beta\Lambda)$ | $-5.16$ | $-5.18$ | $-5.06$ | $-4.01$
$\Lambda$ | $6.69~{}10^{-4}$ | $1.18~{}10^{-4}$ | $6.22~{}10^{-4}$ | $2.03~{}10^{-3}$
$\beta$ | $8.57$ | $47.4$ | $10.2$ | $8.91$
$q$ | $1.06$ | $1.08$ | $1.18$ | $1.02$
Table 2:  Macroscopic parameters for the SFIG-under-pop-constraint examples.

##  VIII Conclusions

After initially introducing some useful social-macroscopic and social-
stochastic quantities we have

  1. 1.

Postulated social, dynamic microscopic equations.

  2. 2.

Validated them using urban population data.

  3. 3.

Performed numerical simulations with random walkers that conclusively
demonstrated that a description using many microscopic variables has as a
counterpart a macroscopic one with few parameters.

  4. 4.

Showed that such macroscopic description can be given an appropriate MaxEnt
form after constructing a “social” phase space, that allows one to derive
thermodynamic-like relations amongst our macro-parameters.

  5. 5.

Finally, as an application, we successfully analyzed urban flows as modelled
by a scale invariant ideal gas.

## References

  * (1) J. Kemeny and J. L. Snell, Mathematical Models in the Social Sciences (MIT Press, Cambridge, Mass. 1978); M. Schroeder, Fractals, chaos and power laws (Freeman, NY, 1990).
  * (2) M. E. J. Newman, Phys. Rev. E 64, (2001) 016131.
  * (3) A. Hernando et al., A. Plastino, _Phys. Lett. A_ 374, 18 (2009).
  * (4) A.-L. Barabasi, R. Albert, Rev. Mod. Phys. 74, 47 (2002).
  * (5) T. Maillart, et al., Phys. Rev. Lett. 101, 218701 (2008).
  * (6) B. Blasius, R. Tönjes, Phys. Rev. Lett. 103, 218701 (2009).
  * (7) R. N. Costa Filho, et al., Phys. Rev. E 60, 1067 (1999).
  * (8) A. A. Moreira, et al., Phys. Rev. E, 73, 065101(R) (2006).
  * (9) L. C. Malacarne, R. S. Mendes, and E. K. Lenzi, Phys. Rev. E 65, 017106 (2001).
  * (10) M. Marsili, Y. C. Zhang, Phys. Rev. Lett. 80, 2741 (1998).
  * (11) R. L. Axtell, Science 293, 1818 (2001).
  * (12) G. K. Zipf, _Human Behavior and the Principle of Least Effort_ (Addison-Wesley, Cambridge, Mass., 1949).
  * (13) Y. Chen, _Chaos Solitons Fract._ 45, 63 (2012).
  * (14) B. C. Murtra, R.V. Solé, _Phys. Rev. E_ 82, 011102 (2010).
  * (15) X. Gabaix, Quart. J. Econom. 114, 739 (1999).
  * (16) A. Hernando, C. Vesperinas, A. Plastino, _Physica A_ 389, 490 (2010).
  * (17) A. Hernando et al., _Eur. Phys. J. B_ 76, 87 (2010).
  * (18) Weisstein, Eric W., _Benford’s Law_ from MathWorld.
  * (19) F. Benford, _Proceed. Am. Phil. Soc._ 78, 551572 (1938).
  * (20) Wikipedia http://en.wikipedia.org/wiki/Log-normal.
  * (21) J. S. Huxley, (1932) Problems of relative growth (Methuen & Co. Lmtd., London, 1932).
  * (22) F. Black, M. Scholes, _J. Polit. Economy_ 81, 637 (1973); S. M. Ross, Introduction to Probability Models, 9th edition (Academis Press, NY, 2007).
  * (23) H. Rozenfeld, et al., _Proc. Nat. Acad. Sci._ 105, 18702 (2008).
  * (24) S. Ree, Phys. Rev. E 73 (2006) 026115.
  * (25) W. J. Reed, B. D. Hughes, Phys. Rev. E 66 (2002) 067103.
  * (26) S. Galam, J. Stat. Phys. 61, 943 (1990).
  * (27) UrbanSim: http://www.urbansim.org.
  * (28) M. Batty, _Cities and Complexity_ (MIT Press, Cambridge, MA, 2005).
  * (29) C. Castellano, S. Fortunato, V. Loreto, Rev. Mod. Phys., 81, 591 (2009).
  * (30) S. Fortunato, C. Castellano, Phys. Rev. Lett. 99, 138701 (2007).
  * (31) S. Galam, Physica A 285, 66 (2000).
  * (32) A. Hernando, A. Plastino, acepted in _Eur. Phys. J. B_(2012).
  * (33) A. Hernando, R. Hernando, A. Plastino, A.R. Plastino, arXiv:1201.0905 (2012).
  * (34) A. Hernando, A. Plastino, A.R. Plastino, _Eur. Phys. J. B_ 85, 147 (2012).
  * (35) A. Hernando, A. Plastino, arXiv:1204.2422 (2012).
  * (36) National Statistics Institute of Spain, Government of Spain (web).
  * (37) A. Katz, Principles of statistical mechanics (Freeman, San Francisco, 1967).
  * (38) C. Tsallis, Introduction to Nonextensive Statistical Mechanics (Springer, NY, 2009).
  * (39) Census bureau, Government of USA (web).
  * (40) R. I. M. Dunbar, _J. Hum. Evo._ 20, 469 (1992); _Beh. Brain Sci._ 16, 681 (1993).
  * (41) Demis, edited by Hanno Sandvik, Wikimedia Commons.
  * (42) Marshall Islands location map, Wikipedia.
  * (43) Econom. Pol. Plann. $\&$ Stat. Office, Rep. of the Marshall Islands (web).
  * (44) National Statistical Institute, Italy (web).

Figure 1: Top panel: quantiles from 0.1 to 0.9 each 0.1 for $p_{\Xi}(\xi)$ as
a function of the population $x$ (the median in shown in black). Bottom panel:
$P_{\Xi}(\xi)$’s cumulative distribution for each of Spain’s provinces

.

Figure 2: Variance $s^{2}/x$ vs. $x$. Red: quantiles from 0.1 to 0.9 each 0.1.
Solid black: fit to the median value. Dashed black lines: Finite-size’s
fluctuations are constant, while the multiplicative regime is given by a
straight line. Figure 3: Fit of $\langle\dot{x}/x\rangle$ to an expression of
the type represented by Eq. (16) for 12 Spanish provinces: Asturias, Almería,
Cáceres, Cuenca, Baleares, Lleida, Badajoz, Ávila, Guipúzcoa, Castellón,
Valladolid and Guadalajara.

Figure 4: Top panel: $q=21/2$-metric diffusion at times $t=20$ (green dotted
line), 70 (red dashed) and 110 (blue solid) using the text-parameters compared
with the distribution of Salamanca towns’ population in 2010, fitted to a
$1/2$-log-normal distribution (solid line). Inset: variance of the relative
growth vs. log-population (dots), confirming the $\sqrt{x}$ dependence for
$q=1/2$ dynamics (red line). Bottom panel: geometric diffusion $(q=1)$ at
times $t=4$ (green dotted), 9 (red dashed) and 29 (blue solid) compared with
the population distribution of Florida State (US) in 2010, fitted to a log-
normal distribution (solid line). Inset: same as top panel’s inset, confirming
size independence and thus proportional dynamics. Figure 5: $q$-metric
diffusion with maximum size constraint for (from left to right panels)
$q=1/2$, $1$ and $1.5$. Rank-distributions and evolution (insets). Figure 6:
$N$-constrained $q$-metric diffusion for $q=1$, $1.5$, and $2$. Figure 7: An
example of population restriction arising out of geographical reasons. Left
Panel: d’Aosta valley (Italy)aostamap . Right Panel: Marshall islandsmarmap .
Figure 8: SFIG-in-a-box examples, from top to bottom: RD of Huelva-province
(Spain), D’Aosta Valley (Italy), and Marshall Islands (dots), compared with
distribution Eq. (65) (lines). In the insets, the relative growth $\dot{u}$
vs. the logarithmic population $u$. Figure 9: SFIG under total-population
constraint examples, in the reading order: RD of Navarra, Lleida, Zaragoza,
Girona, Granada, Almería, Alicante, and Vizcaya (dots) and RD of Eq. (73)
(lines). Insets: relative growth $\dot{x}/x$ vs. log-population $\log x$
fitted to Eq. (16) with the same value of $q$ as in the RD.


# DMSANet: Dual Multi Scale Attention Network

Abhinav Sagar
Vellore Institute of Technology
Vellore, India
<EMAIL_ADDRESS>
Website of author - https://abhinavsagar.github.io/

###### Abstract

Attention mechanism of late has been quite popular in the computer vision
community. A lot of work has been done to improve the performance of the
network, although almost always it results in increased computational
complexity. In this paper, we propose a new attention module that not only
achieves the best performance but also has lesser parameters compared to most
existing models. Our attention module can easily be integrated with other
convolutional neural networks because of its lightweight nature. The proposed
network named Dual Multi Scale Attention Network (DMSANet) is comprised of two
parts: the first part is used to extract features at various scales and
aggregate them, the second part uses spatial and channel attention modules in
parallel to adaptively integrate local features with their global
dependencies. We benchmark our network performance for Image Classification on
ImageNet dataset, Object Detection and Instance Segmentation both on MS COCO
dataset.

##  1 Introduction

The local receptive field of the human eye has led to the construction of
convolutional neural networks which has powered much of the recent advances in
computer vision. Multi scale architecture used in the famous InceptionNet
(Szegedy et al., 2016) aggregates multi-scale information from different size
convolutional kernels. Attention Networks has attracted a lot of attention
recently as it allows the network to focus on only then essential aspects
while ignoring the ones which are not useful (Li et al., 2019), (Cao et al.,
2019) and (Li et al., 2019).

A lot of problems have been successfully tackled using attention mechanism in
computer vision like image classification, image segmentation, object
detection and image generation. Most of the attention mechanisms can be
broadly classified into two types channel attention and spatial attention,
both of which strengthens the original features by aggregating the same
feature from all the positions with different aggregation strategies,
transformations, and strengthening functions (Zhang et al., 2021).

Some of the work combined both these mechanism together and achieved better
results (Cao et al., 2019) and (Woo et al., 2018). The computational burden
was reduced by (Wang et al., 2020) using efficient channel attention and
$1\times 1$ convolution. The most popular attention mechanism is the Squeeze-
and Excitation module (Hu et al., 2018b), which can significantly improve the
performance with a considerably low cost. The “channel shuffle” operator is
used (Zhang and Yang, 2021) to enable information communication between the
two branches. It uses a grouping strategy, which divides the input feature map
into groups along the channel dimension.

##  2 Related Work

There are two main problems which hinders the progress in this field: 1) Both
spatial and channel attention as well as network using combination of two uses
only local information while ignoring long range channel dependency, 2) The
previous architectures fail to capture spatial information at different scales
to be more robust and handle more complex problems. These two challenges were
tackled by (Duta et al., 2020) and (Li et al., 2019) respectivly. The problem
with these architectures is that the number of parameters increased
considerably.

Pyramid Split Attention (PSA) (Zhang et al., 2021) has the ability to process
the input tensor at multiple scales. A multi-scale pyramid convolution
structure is used to integrate information at different scales on each
channel-wise feature map. The channel-wise attention weight of the multi-scale
feature maps are extracted hence long range channel dependency is done.

Non-Local block (Wang et al., 2018) is proposed to build a dense spatial
feature map and capture the long-range dependency using non-local operations.
(Li et al., 2019) used a dynamic selection attention mechanism that allows
each neuron to adaptively adjust its receptive field size based on multiple
scales of input feature map. (Fu et al., 2019) proposed a network to integrate
local features with their global dependencies by summing these two attention
modules from different branches.

Multi scale architectures have been used sucessfully for a lot of vision
problems (sagar2020monocular), (Hu et al., 2018b) and (Sagar and
Soundrapandiyan, 2020). (Fu et al., 2019) adaptively integrated local features
with their global dependencies by summing the two attention modules from
different branches. (Hu et al., 2018a) used spatial extension using a depth-
wise convolution to aggregate individual features. Our network borrows ideas
from (Gao et al., 2018) which used a network to capture local cross-channel
interactions.

The performance (in terms of accuracy) vs computational complexity (in terms
of number of parameters) of the state of art attention modules is shown in
Figure 1:

Figure 1: Comparing the accuracy of different attention methods with ResNet-50
and ResNet-101 as backbone in terms of accuracy and network parameters. The
circles reflects the network parameters and FLOPs of different models. Our
proposed network achieves higher accuracy while having less model complexity.

Our main contributions can be summarized as follows:

• A new attention module is proposed which aggregates feature information at
various scales. Our network is scalable and can be easily plugged into various
computer vision problems.

• Our network captures more contextual information using both spatial and
channel attention at various scales.

• Our experiments demonstrate that our network outperforms previous state of
the art with lesser computational cost.

##  3 Method

###  3.1 Feature Grouping

Shuffle Attention module divides the input feature map into groups and uses
Shuffle Unit to integrate the channel attention and spatial attention into one
block for each group. The sub-features are aggregated and a “channel shuffle”
operator is used for communicating the information between different sub-
features.

For a given feature map $X\in R^{C\times H\times W}$ , where $C,H,W$ indicate
the channel number, spatial height, and width, respectively, shuffle attention
module divides $X$ into $G$ groups along the channel dimension, i.e.,
$X=[X1,X_{G}],X_{k}\in R^{C/G\times H\times W}$. An attention module is used
to weight the importance of each feature. The input of $X_{k}$ is split into
two networks along the channel dimension $X_{k1},X_{k2}\in R^{C/2G\times
H\times W}$. The first branch is used to produce a channel attention map by
using the relationship of channels, while the second branch is used to
generate a spatial attention map by using the spatial relationship of
different features.

###  3.2 Channel Attention Module

The channel attention module is used to selectively weight the importance of
each channel and thus produces best output features. This helps in reducing
the number of parameters of the network. Let $X\in R^{C\times H\times W}$
denotes the input feature map, where the quantity $H,W,C$ represent its
height, width and number of input channels respectively. A SE block consists
of two parts: squeeze and excitation, which are respectively designed for
encoding the global information and adaptively recalibrating the channel-wise
relationship. The Global Average Pooling (GAP) operation can be calculated by
the as shown in Equation 1:

$GAP_{c}=\frac{1}{H\times W}\sum_{i=1}^{H}\sum_{j=1}^{W}x_{c}(i,j)$ (1)

The attention weight of the $c^{th}$ channel in the SE block can be written as
denoted in Equation 2:

$w_{c}=\sigma\left(W_{1}ReLU\left(W_{0}\left(GAP_{c}\right)\right)\right)$ (2)

where $W_{0}\in R^{C\times C}r$ and $W_{1}\in R^{Cr\times C}$ represent the
fully-connected (FC) layers. The symbol $\sigma$ represents the excitation
function where Sigmoid function is usually used.

We calculate the channel attention map $X\in R^{C\times C}$ from the original
features $A\in R^{C\times H\times W}$. We reshape A to $R^{C\times N}$ , and
then perform a matrix multiplication between A and the transpose of A. We then
apply a softmax layer to obtain the channel attention map $X\in R^{C\times C}$
as shown in Equation 3:

$x_{ji}=\frac{\exp\left(A_{i}\cdot
A_{j}\right)}{\sum_{i=1}^{C}\exp\left(A_{i}\cdot A_{j}\right)}$ (3)

where $x_{ji}$ measures the $i^{th}$ channel’s impact on the $j^{th}$ channel.
We perform a matrix multiplication between the transpose of $X$ and $A$ and
reshape their result to $R^{C\times H\times W}$ . We also multiply the result
by a scale parameter $\beta$ and perform an element-wise sum operation with
$A$ to obtain the final output $E\in R^{C\times H\times W}$ as shown in
Equation 4:

$E_{1j}=\beta\sum_{i=1}^{C}\left(x_{ji}A_{i}\right)+A_{j}$ (4)

###  3.3 Spatial Attention Module

We use Instance Normalization (IN) over $X_{k2}$ to obtain spatial-wise
statistics. A $F{c}(·)$ operation is used to enhance the representation of
$X_{k2}$. The final output of spatial attention is obtained by where $W_{2}$
and $b_{2}$ are parameters with shape $R^{C/2G\times 1\times 1}$. After that
the two branches are concatenated to make the number of channels equal to the
number of input.

A local feature denoted by $A\in R^{C\times H\times W}$ is fed into a
convolution layer to generate two new feature maps $B$ and $C$, respectively
where ${B,C}\in R^{C\times H\times W}$. We reshape them to $R^{C\times N}$,
where $N=H\times W$ is the number of pixels. Next a matrix multiplication is
done between the transpose of $C$ and $B$, and apply a softmax layer to
calculate the spatial attention map $S\in R^{N\times N}$. This operation is
shown in Equation 1:

$s_{ji}=\frac{\exp\left(B_{i}\cdot
C_{j}\right)}{\sum_{i=1}^{N}\exp\left(B_{i}\cdot C_{j}\right)}$ (5)

where $s_{ji}$ measures the $i^{th}$ position’s impact on $j^{th}$ position.
Next we feed feature A into a convolution layer to generate a new feature map
$D\in R^{C\times H\times W}$ and reshape it to $R^{C\times N}$. We perform a
matrix multiplication between $D$ and the transpose of $S$ and reshape the
result to $R^{C\times H\times W}$ . We multiply it by a scale parameter
$\alpha$ and perform a element-wise sum operation with the feature $A$ to
obtain the final output $E\in R^{C\times H\times W}$ as shown in Equation 2:

$E_{2j}=\alpha\sum_{i=1}^{N}\left(s_{ji}D_{i}\right)+A_{j}$ (6)

###  3.4 Aggregation

In the final part of the network, all the sub-features are aggregated. We use
a ”channel shuffle” operator to enable cross-group information flow along the
channel dimension. The final output of our module is the same size as that of
input, making our attention module quite easy to integrate with other
networks.

The whole multi-scale pre-processed feature map can be obtained by a
concatenation way as defined in Equation 7:

$F=\operatorname{Concat}\left(\left[E_{1j},E_{2j}\right]\right)$ (7)

where $F\in R^{C\times H\times W}$ is the obtained multi-scale feature map.
Our attention module is used across channels to adaptively select different
spatial scales which is guided by the feature descriptor. This operation is
defined in Equation 8:

$att_{i}=\operatorname{Softmax}\left(Z_{i}\right)=\frac{\exp\left(Z_{i}\right)}{\sum_{i=0}^{S-1}\exp\left(Z_{i}\right)}$
(8)

Finally we multiply the re-calibrated weight of multi-scale channel attention
$a_{tti}$ with the feature map of the corresponding scale $F_{i}$ as shown in
Equation 9:

$Y_{i}=F_{i}\odot att_{i}\quad i=1,2,3,\cdots S-1$ (9)

###  3.5 Network Architecture

We propose DMSA module with the goal to build more efficient and scalable
architecture. The first part of our network borrows ideas from (Li et al.,
2019) and (Zhang and Yang, 2021). An input feature map $X$ is splitted into
$N$ parts along with the channel dimension. For each splitted parts, it has
$C_{0}$=$C_{S}$ number of common channels, and the $i^{th}$ feature map is
$X_{i}\in R^{C_{0}\times H\times W}$. The individual features are fused before
being passed to two different branches.

These two branches are comprised of position attention module and channel
attention module as proposed in (Fu et al., 2019) for semantic segmentation.
The second part of our network does the following 1) Builds a spatial
attention matrix which models the spatial relationship between any two pixels
of the features, 2) A matrix multiplication between the attention matrix and
the original features. 3) An element-wise sum operation is done on the
resulting matrix and original features.

The operators concat and sum are used to reshape the features. The features
from the two parallel branches are aggregated to produce the final output. The
complete network architecture is shown in Figure 2:

Figure 2: A detailed Illustration of DMSANet

.

We compare our network architecture with Resnet (Wang et al., 2017), SENet (Hu
et al., 2018b) and EPSANet (Zhang et al., 2021) in Figure 3. We use our DMSA
module in between $3\times 3$ convolution and $1\times 1$ convolution. Our
network is able to extract features at various scales and aggregate those
individual features before passing through the attention module.

Figure 3: Illustration and comparison of ResNet, SENet, EPSANet and our
proposed DMSANet blocks.

The architectural details our proposed attention network is shown in Table 1:

Table 1: Network design of the proposed DMSANet. Output | ResNet-50 | DMSANet
---|---|---
112$\times$112 | 7$\times$7, 64 | 7$\times$7, 64
56$\times$56 | 3$\times$3 max pool | 3$\times$3 max pool
56$\times$56 |  $\begin{bmatrix}1$$\times$$1,&64\\\ 3$$\times$$3,&64\\\ 1$$\times$$1,&256\\\ \end{bmatrix}$ $\times 3$ |  $\begin{bmatrix}1$$\times$$1,&64\\\ DMSA,&64\\\ 1$$\times$$1,&256\\\ \end{bmatrix}$ $\times$ 3
28$\times$28 |  $\begin{bmatrix}1$$\times$$1,&128\\\ 3$$\times$$3,&128\\\ 1$$\times$$1,&512\\\ \end{bmatrix}$ $\times$ 4 |  $\begin{bmatrix}1$$\times$$1,&128\\\ DMSA,&128\\\ 1$$\times$$1,&512\\\ \end{bmatrix}$ $\times$ 4
14$\times$14 |  $\begin{bmatrix}1$$\times$$1,&256\\\ 3$$\times$$3,&256\\\ 1$$\times$$1,&1024\\\ \end{bmatrix}$ $\times$ 6 |  $\begin{bmatrix}1$$\times$$1,&256\\\ DMSA,&256\\\ 1$$\times$$1,&1024\\\ \end{bmatrix}$ $\times$ 6
7$\times$7 |  $\begin{bmatrix}1$$\times$$1,&512\\\ 3$$\times$$3,&512\\\ 1$$\times$$1,&2048\\\ \end{bmatrix}$ $\times$ 3 |  $\begin{bmatrix}1$$\times$$1,&512\\\ DMSA,&512\\\ 1$$\times$$1,&2048\\\ \end{bmatrix}$ $\times$ 3
1 $\times$ 1 | 7$\times$7 GAP,1000-d fc | 7$\times$7 GAP,1000-d fc

###  3.6 Implementation Details

We use Residual Networks (He et al., 2016) as the backbone which is widely
used in literature for image classification on Imagenet dataset (Deng et al.,
2009). Data augmentation is used for increasing the size of the dataset and
the input tensor is cropped to size $224\times 224$. Stochastic Gradient
Descent is used as the optimizer with learning rate of $1e^{-4}$, momentum as
0.9 and mini batch size of 64. The learning rate is initially set as 0.1 and
is decreased by a factor of 10 after every 20 epochs for 50 epochs in total.

We use Residual Network along with FPN as the backbone network (Lin et al.,
2017a) for object detection. The detectors we benchmark against are Faster
RCNN (Ren et al., 2015), Mask RCNN(He et al., 2017) and RetinaNet (Lin et al.,
2017b) on MS-COCO dataset (Lin et al., 2014). Stochastic Gradient Descent is
used as the optimizer with a weight decay of 1$e^{-4}$, momentum is 0.9, and
the batch size is 16 per GPU for 10 epochs. The learning rate is set as 0.01
and is decreased by the factor of 10 every 10th epoch.

For instance segmentation we use Mask RCNN (He et al., 2017) with FPN (Lin et
al., 2017a) as backbone. Stochastic Gradient Descent is used as the optimizer
with a weight decay of 1$e^{-4}$, momentum is 0.9, and the batch size is 4 per
GPU for 10 epochs. The learning rate is set as 0.01 and is decreased by the
factor of 10 every 10th epoch.

##  4 Results

###  4.1 Image Classification on ImageNet

We compare our network with previous state of the art on ResNet with 50 and
101 layers.

Our network shows the best performance in accuracy, achieving a considerable
improvement compared with all the previous attention models along with lower
computational cost. The comparision of our network against previous state of
the art with ResNet50 as backbone is shown in Table 2:

Table 2: Comparison of various attention methods on ImageNet with ResNet50 as backbone in terms of network parameters(in millions), floating point operations per second (FLOPs), Top-1 and Top-5 Validation Accuracy(%). The best records are marked in bold. Network | Params | FLOPs | Top-1 Acc (%) | Top-5 Acc(%)
---|---|---|---|---
ResNet | 25.56 | 4.12G | 75.20 | 92.52
SENet | 28.07 | 4.13G | 76.71 | 93.38
CBAM | 28.07 | 4.14G | 77.34 | 93.69
ABN | 43.59 | 7.18G | 76.90 | -
GCNet | 28.11 | 4.13G | 77.70 | 93.66
AANet | 25.80 | 4.15G | 77.70 | 93.80
ECANet | 25.56 | 4.13G | 77.48 | 93.68
FcaNet | 28.07 | 4.13G | 78.52 | 94.14
EPSANet(Small) | 22.56 | 3.62G | 77.49 | 93.54
EPSANet(Large) | 27.90 | 4.72G | 78.64 | 94.18
DMSANet | 26.25 | 3.44G | 80.02 | 94.27

The comparision of our network against previous state of the art with
ResNet101 as backbone is shown in Table 3:

Table 3: Comparison of various attention methods on ImageNet with ResNet 101 as backbone in terms of network parameters(in millions), floating point operations per second (FLOPs), Top-1 and Top-5 Validation Accuracy(%). The best records are marked in bold. Network | Params | FLOPs | Top-1 Acc (%) | Top-5 Acc(%)
---|---|---|---|---
ResNet | 44.55 | 7.85G | 76.83 | 93.48
SENet | 49.29 | 7.86G | 77.62 | 93.93
BAM | 44.91 | 7.93G | 77.56 | 93.71
CBAM | 49.33 | 7.88G | 78.49 | 94.31
SRM | 44.68 | 7.95G | 78.47 | 94.20
ECANet | 44.55 | 7.86G | 78.65 | 94.34
AANet | 45.40 | 8.05G | 78.70 | 94.40
EPSANet(Small) | 38.90 | 6.82G | 78.43 | 94.11
EPSANet(Large) | 49.59 | 8.97G | 79.38 | 94.58
DMSANet | 42.29 | 7.11G | 81.54 | 94.93

###  4.2 Object Detection on MS COCO

The comparision of our network using Faster RCNN detector against previous
state of the art is shown in Table 4:

Table 4: Comparison of object detection results on COCO val2017 using Faster RCNN detector. The best records are marked in bold. Backbone | Params(M) | GFLOPs | AP | $AP_{50}$ | $AP_{75}$ | $AP_{S}$ | $AP_{M}$ | $AP_{L}$
---|---|---|---|---|---|---|---|---
ResNet-50 | 41.53 | 207.07 | 36.4 | 58.2 | 39.5 | 21.8 | 40.0 | 46.2
SENet-50 | 44.02 | 207.18 | 37.7 | 60.1 | 40.9 | 22.9 | 41.9 | 48.2
ECANet-50 | 41.53 | 207.18 | 38.0 | 60.6 | 40.9 | 23.4 | 42.1 | 48.0
SANet-50 | 41.53 | 207.35 | 38.7 | 61.2 | 41.4 | 22.3 | 42.5 | 49.8
FcaNet-50 | 44.02 | 215.63 | 39.0 | 61.1 | 42.3 | 23.7 | 42.8 | 49.6
EPSANet-50(Small) | 38.56 | 197.07 | 39.2 | 60.3 | 42.3 | 22.8 | 42.4 | 51.1
EPSANet-50(Large) | 43.85 | 219.64 | 40.9 | 62.1 | 44.6 | 23.6 | 44.5 | 54.0
DMSANet | 44.17 | 222.31 | 41.4 | 61.9 | 46.2 | 25.8 | 44.7 | 55.3

The comparision of our network using MASK RCNN detector against previous state
of the art is shown in Table 5:

Table 5: Comparison of object detection results on COCO val2017 using Mask RCNN detector. The best records are marked in bold. Backbone | Params(M) | GFLOPs | AP | $AP_{50}$ | $AP_{75}$ | $AP_{S}$ | $AP_{M}$ | $AP_{L}$
---|---|---|---|---|---|---|---|---
ResNet-50 | 44.18 | 275.58 | 37.2 | 58.9 | 40.3 | 22.2 | 40.7 | 48.0
SENet-50 | 46.67 | 275.69 | 38.7 | 60.9 | 42.1 | 23.4 | 42.7 | 50.0
Non-local | 46.50 | 288.70 | 38.0 | 59.8 | 41.0 | - | - | -
GCNet-50 | 46.90 | 279.60 | 39.4 | 61.6 | 42.4 | - | - | -
ECANet-50 | 44.18 | 275.69 | 39.0 | 61.3 | 42.1 | 24.2 | 42.8 | 49.9
SANet-50 | 44.18 | 275.86 | 39.4 | 61.5 | 42.6 | 23.4 | 42.8 | 51.1
FcaNet-50 | 46.66 | 261.93 | 40.3 | 62.0 | 44.1 | 25.2 | 43.9 | 52.0
EPSANet-50(Small) | 41.20 | 248.53 | 40.0 | 60.9 | 43.3 | 22.3 | 43.2 | 52.8
EPSANet-50(Large)) | 46.50 | 271.10 | 41.4 | 62.3 | 45.3 | 23.6 | 45.1 | 54.6
DMSANet | 47.23 | 279.26 | 43.1 | 61.6 | 47.5 | 24.1 | 46.9 | 56.5

The comparision of our network using RetinaNet detector against previous state
of the art is shown in Table 6:

Table 6: Comparison of object detection results on COCO val2017 using RetinaNet detector. The best records are marked in bold. Backbone | Params(M) | GFLOPs | AP | $AP_{50}$ | $AP_{75}$ | $AP_{S}$ | $AP_{M}$ | $AP_{L}$
---|---|---|---|---|---|---|---|---
ResNet-50 | 37.74 | 239.32 | 35.6 | 55.5 | 38.2 | 20.0 | 39.6 | 46.8
SENet-50 | 40.25 | 239.43 | 37.1 | 57.2 | 39.9 | 21.2 | 40.7 | 49.3
SANet-50 | 37.74 | 239.60 | 37.5 | 58.5 | 39.7 | 21.3 | 41.2 | 45.9
EPSANet-50(Small) | 34.78 | 229.32 | 38.2 | 58.1 | 40.6 | 21.5 | 41.5 | 51.2
EPSANet-50(Large)) | 40.07 | 251.89 | 39.6 | 59.4 | 42.3 | 21.2 | 43.4 | 52.9
DMSANet | 41.63 | 270.17 | 40.2 | 59.8 | 44.1 | 23.5 | 44.8 | 54.8

###  4.3 Instance Segmentation on MS COCO

We used Mask-RCNN (He et al., 2017) as benchmark on MS-COCO dataset (Lin et
al., 2014). The comparision results of our network on instance segmentation
using MS COCO dataset against previous state of the art is shown in Table 7:

Table 7: Instance segmentation results of different attention networks by using the Mask R-CNN on COCO. The best records are marked in bold. Network | AP | $AP_{50}$ | $AP_{75}$ | $AP_{S}$ | $AP_{M}$ | $AP_{L}$
---|---|---|---|---|---|---
ResNet-50 | 34.1 | 55.5 | 36.2 | 16.1 | 36.7 | 50.0
SENet-50 | 35.4 | 57.4 | 37.8 | 17.1 | 38.6 | 51.8
GCNet | 35.7 | 58.4 | 37.6 | - | - | -
ECANet | 35.6 | 58.1 | 37.7 | 17.6 | 39.0 | 51.8
FcaNet | 36.2 | 58.6 | 38.1 | - | - | -
SANet | 36.1 | 58.7 | 38.2 | 19.4 | 39.4 | 49.0
EPSANet-50(Small) | 35.9 | 57.7 | 38.1 | 18.5 | 38.8 | 49.2
EPSANet-50(Large) | 37.1 | 59.0 | 39.5 | 19.6 | 40.4 | 50.4
DMSANet | 37.4 | 61.1 | 40.7 | 19.3 | 40.9 | 51.7

###  4.4 Ablation Study

The ablation studies of our architecture is shown in Table 8. The results are
best obtained using instance normalization. Both removing $F_{c}(·)$ and using
$1\times 1$ Conv results in reduced performance as compared to the original
network. The earlier is because $F_{c}(·)$ is used to enhance the performance
of individual features while latter is because number of channels in each sub-
feature is too few, so it is not important to exchange information among
different channels.

Table 8: Performance comparisons of our network using ResNet 50 as backbone with four options (i.e., original, using Batch Normalization, using Group Normalization, using shuffle normalization, eliminating $F_{c}(·)$ and using $1\times 1$ Conv to replace $F_{c}(·)$ on ImageNet-1k in terms of GFLOPs and Top-1/Top-5 accuracy (in %). The best records are marked in bold. Methods | GFLOPs | Top-1 Acc(%) | Top-5 Acc(%)
---|---|---|---
origin | 3.44 | 80.02 | 94.27
W BN | 3.82 | 77.37 | 93.80
W GN | 3.56 | 77.61 | 92.89
W SN | 3.51 | 78.16 | 93.48
W/O $F_{c}(·)$ | 4.07 | 77.64 | 93.18
$1\times 1$ Conv | 3.55 | 78.69 | 93.71

##  5 Conclusions

In this paper, we propose a novel Attention module named Dual Multi Scale
Attention Network (DMSANet). Our network is comprised of two parts 1) first
for aggregating feature information at various scales 2) second made up of
position and channel attention modules in parallel for capturing global
contextual information. After evaluating our network both qualitatively and
quantitatively, we show that our network outperforms previous state of the art
across image classification, object detection and instance segmentation
problems. The ablation experiments show that our attention module captures
long-range contextual information effectively at various scales thus making it
generalizable to other tasks. The best part of DMSANet attention module is
that it is very lightweight and hence could be easily plugged into various
custom networks as and when required.

#### Acknowledgments

We would like to thank Nvidia for providing the GPUs for this work.

## References

  * Bello et al. (2019) I. Bello, B. Zoph, A. Vaswani, J. Shlens, and Q. V. Le.  Attention augmented convolutional networks.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 3286–3295, 2019.
  * Cao et al. (2019) Y. Cao, J. Xu, S. Lin, F. Wei, and H. Hu.  Gcnet: Non-local networks meet squeeze-excitation networks and beyond.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops_ , pages 0–0, 2019.
  * Chen et al. (2018) L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam.  Encoder-decoder with atrous separable convolution for semantic image segmentation.  In _Proceedings of the European conference on computer vision (ECCV)_ , pages 801–818, 2018.
  * Deng et al. (2009) J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE conference on computer vision and pattern recognition_ , pages 248–255. Ieee, 2009.
  * Duta et al. (2020) I. C. Duta, L. Liu, F. Zhu, and L. Shao.  Pyramidal convolution: rethinking convolutional neural networks for visual recognition.  _arXiv preprint arXiv:2006.11538_ , 2020.
  * Fu et al. (2019) J. Fu, J. Liu, H. Tian, Y. Li, Y. Bao, Z. Fang, and H. Lu.  Dual attention network for scene segmentation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 3146–3154, 2019.
  * Fukui et al. (2019) H. Fukui, T. Hirakawa, T. Yamashita, and H. Fujiyoshi.  Attention branch network: Learning of attention mechanism for visual explanation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 10705–10714, 2019.
  * Gao et al. (2018) H. Gao, Z. Wang, and S. Ji.  Channelnets: Compact and efficient convolutional neural networks via channel-wise convolutions.  _arXiv preprint arXiv:1809.01330_ , 2018.
  * Gao et al. (2019) S. Gao, M.-M. Cheng, K. Zhao, X.-Y. Zhang, M.-H. Yang, and P. H. Torr.  Res2net: A new multi-scale backbone architecture.  _IEEE transactions on pattern analysis and machine intelligence_ , 2019.
  * He et al. (2016) K. He, X. Zhang, S. Ren, and J. Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 770–778, 2016.
  * He et al. (2017) K. He, G. Gkioxari, P. Dollár, and R. Girshick.  Mask r-cnn.  In _Proceedings of the IEEE international conference on computer vision_ , pages 2961–2969, 2017.
  * Hu et al. (2018a) J. Hu, L. Shen, S. Albanie, G. Sun, and A. Vedaldi.  Gather-excite: Exploiting feature context in convolutional neural networks.  _arXiv preprint arXiv:1810.12348_ , 2018a.
  * Hu et al. (2018b) J. Hu, L. Shen, and G. Sun.  Squeeze-and-excitation networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 7132–7141, 2018b.
  * Li et al. (2019) X. Li, W. Wang, X. Hu, and J. Yang.  Selective kernel networks.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 510–519, 2019.
  * Lin et al. (2014) T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick.  Microsoft coco: Common objects in context.  In _European conference on computer vision_ , pages 740–755. Springer, 2014.
  * Lin et al. (2017a) T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie.  Feature pyramid networks for object detection.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 2117–2125, 2017a.
  * Lin et al. (2017b) T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár.  Focal loss for dense object detection.  In _Proceedings of the IEEE international conference on computer vision_ , pages 2980–2988, 2017b.
  * Ren et al. (2015) S. Ren, K. He, R. Girshick, and J. Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks.  _arXiv preprint arXiv:1506.01497_ , 2015.
  * Sagar (2021) A. Sagar.  Aa3dnet: Attention augmented real time 3d object detection.  _arXiv preprint arXiv:2107.12137_ , 2021.
  * Sagar and Soundrapandiyan (2020) A. Sagar and R. Soundrapandiyan.  Semantic segmentation with multi scale spatial attention for self driving cars.  _arXiv preprint arXiv:2007.12685_ , 2020.
  * Sang et al. (2020) H. Sang, Q. Zhou, and Y. Zhao.  Pcanet: Pyramid convolutional attention network for semantic segmentation.  _Image and Vision Computing_ , 103:103997, 2020.
  * Szegedy et al. (2016) C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna.  Rethinking the inception architecture for computer vision.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 2818–2826, 2016.
  * Wang et al. (2017) F. Wang, M. Jiang, C. Qian, S. Yang, C. Li, H. Zhang, X. Wang, and X. Tang.  Residual attention network for image classification.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 3156–3164, 2017.
  * Wang et al. (2020) Q. Wang, B. Wu, P. Zhu, P. Li, W. Zuo, and Q. Hu.  Eca-net: Efficient channel attention for deep convolutional neural networks, 2020.
  * Wang et al. (2018) X. Wang, R. Girshick, A. Gupta, and K. He.  Non-local neural networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 7794–7803, 2018.
  * Woo et al. (2018) S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon.  Cbam: Convolutional block attention module.  In _Proceedings of the European conference on computer vision (ECCV)_ , pages 3–19, 2018.
  * Wu and He (2018) Y. Wu and K. He.  Group normalization.  In _Proceedings of the European conference on computer vision (ECCV)_ , pages 3–19, 2018.
  * Zhang et al. (2020) H. Zhang, C. Wu, Z. Zhang, Y. Zhu, H. Lin, Z. Zhang, Y. Sun, T. He, J. Mueller, R. Manmatha, et al.  Resnest: Split-attention networks.  _arXiv preprint arXiv:2004.08955_ , 2020.
  * Zhang et al. (2021) H. Zhang, K. Zu, J. Lu, Y. Zou, and D. Meng.  Epsanet: An efficient pyramid split attention block on convolutional neural network.  _arXiv preprint arXiv:2105.14447_ , 2021.
  * Zhang and Yang (2021) Q.-L. Zhang and Y.-B. Yang.  Sa-net: Shuffle attention for deep convolutional neural networks.  In _ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 2235–2239. IEEE, 2021.
  * Zhao et al. (2017) H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia.  Pyramid scene parsing network.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 2881–2890, 2017.
  * Zhong et al. (2020) Z. Zhong, Z. Q. Lin, R. Bidart, X. Hu, I. B. Daya, Z. Li, W.-S. Zheng, J. Li, and A. Wong.  Squeeze-and-attention networks for semantic segmentation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 13065–13074, 2020.