# Enhanced Battery Degradation-Aware Scheduling for Distribution Network with
Electric Vehicle Load
††thanks: This work was supported in part by A*STAR under its MTC Programmatic
(Award M23L9b0052), MTC Individual Research Grants (IRG) (Award M23M6c0113),
SIT’s Ignition Grant (STEM) (Grant ID: IG (S) 2/2023 – 792), and Future
Communications Research & Development Programme (FCP) under Grant FCP-SIT-
TG-2022-007. ††thanks: ∗ Corresponding author.

Vijay Babu Pamshetti  Singapore Institute of Technology, Chaitanya Bharathi
Institute of Technology
<EMAIL_ADDRESS><EMAIL_ADDRESS>Wei
Zhang∗ Andy, Man-Fai Ng  Singapore Institute of Technology
<EMAIL_ADDRESS>Agency for Science, Technology and Research
<EMAIL_ADDRESS>Qingyu Yan  Nanyang Technological University
<EMAIL_ADDRESS>Kuan Tak Tan  Singapore Institute of Technology
<EMAIL_ADDRESS>
###### Abstract

Batteries play a key role in today’s power grid. In this paper, we investigate
the impact of battery degradation on the distribution network. We formulate a
multi-objective framework for optimizing battery scheduling with the goals of
minimizing monetary costs and improving network performance. Our framework
incorporates energy purchase and battery degradation into the costs and
measures the network performance through energy losses and voltage deviation.
We propose Bach for battery degradation-aware scheduling based on
$\varepsilon$-constraint and fuzzy logic methods. Bach is implemented for the
IEEE 33-bus network for an experimental study. The results show the
effectiveness of Bach in optimizing costs and performance simultaneously with
battery degradation awareness and demonstrate the flexibility of further
customization.

###### Index Terms:

battery health monitoring, multi-objective, smart grid, renewable energy

##  I Introduction

Solar photovoltaic (PV) systems and electric vehicles (EVs) have evolved
beyond mere concepts for the smart grid. We are witnessing a rapid increase in
PV integration into the power grid alongside the growing market share of EVs.
These advancements not only address environmental concerns but also contribute
significantly to sustainability. However, they introduce new challenges as the
increased supply from PV and the heightened demand from EVs add substantial
uncertainty to the grid. This uncertainty leads to increased energy losses and
voltage deviations in the distribution network with a significant impact on
the grid’s quality of service [1].

An effective way to address the issues is the integration of battery energy
storage systems (BESS) into distribution networks. Batteries can buffer
surplus energy generated by PV systems and supply this stored energy during
high demand periods, such as for EV charging [2] [3]. This buffering concept
is well-received by network operators for its effectiveness. However, this
solution is not without drawbacks. The frequent charging and discharging
cycles triggered to balance supply and demand dynamics accelerate battery
degradation, and lead to high battery investment and potential sustainability
issues.

Existing works have attempted to investigate and address the issues. A review
of the challenges and the importance of battery health monitoring are
presented in [1]. The detailed cost modelling of battery degradation, where
the depth of discharge (DoD) plays a major role, is developed in [2]. Battery
degradation has been further incorporated into several scheduling algorithms
for distribution networks. Examples include [3] for distributed BESS, [4] for
BESS supporting multiple services, [5, 6] for cooperative allocation of
renewable energy, and [7] incorporating time-varying prices. Despite these
advancements, the focus is primarily on cost minimization for energy purchase
and battery degradation, often neglecting the performance of distribution
networks. This oversight is concerning, as optimizing cost often compromises
network performance, or even violates the quality of service.

In this paper, we take network performance into account and propose the
optimized battery degradation-aware scheduling algorithms called Bach. We
broaden the spectrum of the scheduling to capture realistic aspects of
distribution networks with BESS, PV generation, EV load, and other loads. We
develop a multi-objective framework to optimize both monetary costs (energy
purchase and battery degradation) and network performance (energy loss and
voltage deviation). To solve the formulated multi-objective problem, we
propose two methods based on $\varepsilon$-constraint and fuzzy logic,
respectively. An experimental study based on the IEEE 33-bus network is
conducted to evaluate the impact of battery degradation on scheduling and
parameter sensitivity. The study demonstrates the effectiveness of our
proposed framework and methods. Besides the competitive performance, Bach also
sheds lights on other applications like EVs with battery degradation issues.

##  II Problem formulation

We formulate a bi-objective problem of battery degradation-aware scheduling,
considering monetary cost and the performance of distribution networks.

###  II-A Objective 1: Monetary Cost

The monetary cost includes the cost of purchasing electricity to serve EV
charging requests and the cost of battery degradation. The former is given by,

$C^{E}=\sum_{t\in T}\rho_{t}\times p_{t}^{E}\times\Delta t,$ (1)

where $\rho_{t}$ and $p_{t}^{E}$ denote the energy price and charging power at
time slot $t$, and we assume both remain constant during time duration $\Delta
t$. The latter for battery degradation is based on [3] and given by,

$C^{D}=c^{B}\times\frac{r^{D}\times(1+r^{D})^{T^{\text{cycle}}}}{(1+r^{D})^{T^{\text{cycle}}}-1},$
(2)

where $c^{B}$, $r^{D}$, and $T^{\text{cycle}}$ mean the cost for battery
investment and discount rate, and the battery cycle life, respectively.
Specifically, $T^{\text{cycle}}$ can be calculated by Eq. (3) as below.

$T^{\text{cycle}}=\frac{n^{*}}{365\times 0.5\times\sum_{t\in
T}(\text{DoD}_{t})^{\kappa}},$ (3)

where $n^{*}$ is the battery cycle life in years assuming 100% DoD, which in
reality may vary among different charging/discharging events. We let
$\text{DoD}_{t}$ be the actual DoD for event $t$ and assume a set of events
$T$ where $t\in T$. We also assume a uniformly 0.5 cycle for each
charging/discharging event. Finally, $\kappa$ is a constant range from 0.8 to
2.1 empirically. Altogether, we present the first objective function $F_{1}$
as the total cost $C$ given by,

$F_{1}:~{}C=C^{E}+\lambda_{1}C^{D},$ (4)

where the importance of battery degradation is controlled by a parameter
$\lambda_{1}$.

###  II-B Objective 2: Distribution Network Performance

We capture the performance of a distribution network by two important aspects,
energy loss and voltage deviation. The former shall be minimized and the daily
loss is given by,

$L=\sum_{t\in T}p^{L}_{t},$ (5)

where $p^{L}_{t}$ represents the active power loss in distribution network in
time $t$, within which $p^{L}_{t}$ remains unchanged. The latter shall be
minimized as well and we model the daily voltage deviation as,

$V=\sum_{t\in T}\sum_{b\in B}\left|1-v_{t,b}\right|,$ (6)

where $B$ is the set of all buses and nodes and $v_{t,b}$ is the voltage for
bus $b$ at time $t$. Altogether, we have the second objective function as the
addition of both energy loss and voltage deviation, given by,

$F_{2}:~{}L+\lambda_{2}V,$ (7)

where $\lambda_{2}$ is a control parameter similar to $\lambda_{1}$.

###  II-C Constraints

The objective functions shall be optimized subject to several constraints,
based on realistic settings for distribution networks. For buses and nodes, we
need to maintain the balance of active and reactive power at every time slot
$t$, as below.

$\begin{split}&p^{A}_{t}+\sum_{b\in B}p^{PV}_{t,b}+\sum_{b\in
B}p^{BESS^{-}}_{t,b}\\\ =&\sum_{b\in B}p_{t,b}^{R}+\sum_{b\in
B}p_{t,b}^{L}+\sum_{b\in B}p_{b,t}^{EV}+\sum_{b\in
B}p_{t,b}^{BESS^{+}},\end{split}$ (8)

where $p^{A}$ is the active power from the grid as one supply source of the
network, $p^{PV}_{t,b}$ is the PV generated power to bus $b$, $p^{BESS^{-}}$
and $p^{BESS^{+}}$ are the discharging and charging power of BESS,
respectively, and the EV charging load is captured by $p^{EV}$. Power can be
non-positive, e.g., $p^{PV}_{t,b^{\prime}}=0$ if no PVs are attached to bus
$b^{\prime}$. We also enforce a range of valid bus voltage as
$[v^{\min},v^{\max}]$, and the maximum apparent power is ${S_{b}}^{\max}$ for
bus $b$. The exponential voltage-dependent active power demand is given by,

$p_{t,b}^{A}=p_{t,b}^{A^{*}}\left(\frac{v_{t,b}}{v_{t,b}^{*}}\right)^{\kappa_{b}^{p}}.$
(9)

where $v^{*}$ is the nominal voltage and $\kappa_{b}^{p}$ is a constant.
Finally, we model the change of SoC of BESS at time $t$ as,

$\text{SoC}_{t+1}=\text{SoC}_{t}-\frac{\Delta
t}{\eta^{-}E^{B}}p_{t}^{BESS-}+\frac{\Delta t}{\eta^{+}E^{B}}p_{t}^{BESS+},$
(10)

where $\eta^{+}$ and $\eta^{-}$ are the BESS efficiency for charging and
discharging, respectively, and $E^{B}$ is the battery capacity in kWh.
Besides, the valid range of BESS power is $[0,p^{BESS^{*}}]$.

##  III Methodology

For our multi-objective problem, we present Bach to produce optimal schedules
based on two different methods.

###  III-A $\varepsilon$-Constraint Method

Given two objectives $F_{1}$ and $F_{2}$, as well as a set of constraints, we
proposed to employ the $\varepsilon$-constraint method to perform multi-
objective optimization and derive optimal solutions for Bach. The main idea of
the method is to optimize one objective while treating the other one as an
inequality constraint controlled by a parameter $\varepsilon$. Specifically,
we let $F_{1}$ for monetary cost be the former and treat $F_{2}$ for
distribution network performance as an inequality constraint, i.e.,

$\begin{split}\min&~{}F_{1}\\\ \text{s.t.}&~{}F_{2}\leq\varepsilon\text{~{}and
constraints in Section \ref{sec-prob-constraint}}\end{split}$ (11)

where $\varepsilon$ systematically varies from its minimum to maximum for
$F_{2}$ and the optimization problem in Eq. (11) is solved for each sampled
$\varepsilon$ value. Such an iterative method gradually adjusts $\varepsilon$
to identify a set of non-inferior solutions, which together form the Pareto
front for our bi-objective optimization.

###  III-B Fuzzy Decision-Making Method

In realistic scenarios, decision makers may prefer one optimal solution
instead of many. The challenge for such one solution is to balance the
multiple objectives simultaneously. We propose to employ a fuzzy decision-
making method, which transforms each of the objectives into a fuzzy variable
scaled between 0 and 1. The linear membership for such transformation for
objective $i$ is given by,

$\mu(F_{i})=\begin{cases}1&\text{if }F_{i}\leq F_{i}^{\min}\\\
\frac{F_{i}^{\max}-F_{i}}{F_{i}^{\max}-F_{i}^{\min}}&\text{if
}F_{i}^{\min}<F_{i}<F_{i}^{\max}\\\ 0&\text{if }F_{i}\geq
F_{i}^{\max}\end{cases},$ (12)

where $F_{i}^{\min}$ and $F_{i}^{\max}$ represent the minimum and maximum
$F_{i}$ values, respectively. The membership value of 1 implies complete
compatibility and 0 signifies incompatibility. Accordingly, we normalize the
membership functions for each non-inferior solution, which can be calculated
as,

$\mu^{k}=\frac{\sum_{i=1}^{m}{\mu(F_{i}^{k})}}{\sum_{k=1}^{k}\big{(}\sum_{i=1}^{m}{\mu(F_{i}^{k})\big{)}}},$
(13)

where $m=2$ is the number of objectives and $k$ is the number of solutions in
this research.

##  IV Experimental Study

We present our experimental setup and results here.

###  IV-A Experimental Setup

Our multi-objective optimization methods and BESS scheduling in Bach utilize
the GAMS solver and MATLAB interfacing [8], e.g., DICOPT from GAMS specialized
for mixed-integer non-linear programming. We test Bach on the IEEE 33-bus
distribution network with detailed parameters in Table I. Different customer
types, e.g., commercial, residential, and industrial, along with their
corresponding bus locations in the network are available in [9]. We take the
load profile, price, PV generation, and EV load (levels 1 and 2) from [9, 10]
as the input of our methods, and a sample daily data is visualized in Fig. 1.
Besides, both $\lambda_{1}$ and $\lambda_{2}$ are equal to 1 by default.

TABLE I: Configurations of our 33-bus distribution network. | Bus Identifiers | Capacity
---|---|---
PV | 9, 13, 25, 30 | 500 kW
EV | Level 1: 8, 14, 18; Level 2: 2, 19, 27 | 150 EVs
BESS | 18, 33 | 1 MWh

(a) PV and Price (b) EV Load

Figure 1: Sample daily data for PV generation, electricity price, and EV load,
with 150 EVs for each level.

###  IV-B Results and Discussions

To demonstrate the effectiveness of our multi-objective optimization, we
implement and discuss the following cases.

  * •

Case 1: Optimize $F_{1}$ only for minimizing the costs for electricity
purchase and BESS degradation.

  * •

Case 2: Optimize $F_{2}$ only for maximizing the network performance in terms
of power loss and voltage deviation.

  * •

Case 3: Optimize both $F_{1}$ and $F_{2}$ for optimizing both monetary cost
and network performance.

We present the results of the three cases in Table II for energy cost,
degradation cost, energy loss, and voltage deviation, and detailed results
discussions below.

TABLE II: The results of Bach for different cases, where Cases 1 and 2 have minimum cost and network performance, respectively, and Case 3 achieve the optimal for both cost and performance. Objective Function | Case 1 | Case 2 | Case 3
---|---|---|---
$F_{1}$ | energy purchase ($) | 3,860.18 | 3,879.01 | 3,851.59
battery degradation ($) | 142.58 | 499.26 | 218.09
$F_{2}$ | energy losses (kWh) | 1,520.36 | 1,206.44 | 1,276.78
voltage deviations | 19.16 | 15.75 | 15.92

####  IV-B1 Case 1

This case is cost driven. The cost, for both energy purchase and battery
degradation, is naturally low, evidenced by the results in Table II.
Specifically, the energy expenditure is 3,860, which is lower than the cost of
Case 2. The degradation cost is 143, which is only 29% and 66% of the costs of
Case 2 and Case 3, respectively. The performance of distribution networks is
not within the consideration of this case, and the values of energy losses and
voltage deviation are high, e.g., 26% and 22% higher compared to Case 2 for
losses and deviation, respectively. Note that minimizing the cost without
considering the network performance is often not realistic, which sacrifices
or even violates the service quality.

####  IV-B2 Case 2

Different from Case 1, we optimize the network performance only in this case
with the cost non-optimized. Table II shows that this case has the best
network performance. The energy loss is 1,206 kWh, nearly 20% less compared to
the above case. The voltage deviation is 15.75, 22% lower than the cost-driven
case. Without focusing on the cost, this case incurs higher costs for energy
purchase and battery degradation compared to Case 1; especially for
degradation, the cost is 3.5x higher. While the network performance is
competitive, this case may not be competitive for a long-term business.

####  IV-B3 Case 3

This case optimizes both objectives and models a more realistic scenario.
Minimizing the cost and maximizing the network performance is sometimes
contradictory, meaning the improvement of one objective comes with the
decreased performance of the other. Some solutions do not dominate each other
in both objectives and multiple optimal solutions exist with different
emphases of the objectives. We show the solutions in Fig. 2(a), where $F_{1}$
for this case ranges from $4,003 to $4,378 and $F_{2}$ ranges from 1,222 to
1,540. Among these solutions, our proposed fuzzy method with linear membership
identifies one of the solutions for decision-making, which is highlighted in
Fig. 2. Specifically for this solution, the costs for energy purchase and
battery degradation are $3,852 and $218, respectively, while the energy loss
and voltage deviation are 1,277 kWh and 15.9, respectively. We also show the
results for battery life in Fig. 2(b) where we can observe that battery life
needs to be compromised for optimizing the distribution network. Overall, the
results of this case are competitive compared to the other two cases as shown
in Table II, and show a great balance in optimizing both monetary cost and
network performance.

(a) $F_{1}$ vs. $F_{2}$

(b) Battery Life vs. $F_{2}$

Figure 2: Case 3 with multiple optimal solutions for both objectives for cost
minimization and network performance as a Pareto front. The solution based on
our fuzzy method for decision-making is annotated and highlighted in red.

(a) Variation of $\lambda_{1}$

(b) Variation of $\lambda_{2}$

Figure 3: The parameter sensitivity analysis results with varying
$\lambda_{1}$ for the cost objective and $\lambda_{2}$ for the network
performance objective. Large $\lambda_{1}$ emphasizes the importance of
battery degradation and voltage deviation becomes more important with large
$\lambda_{2}$.

###  IV-C Parameter Sensitivity Analysis

We investigate the performance sensitivity of two parameters, $\lambda_{1}$
and $\lambda_{2}$, for controlling the importance of different terms of the
objectives. The results for $F_{1}$ are illustrated in Fig. 3(a). We can see
that the cost of battery degradation decreases with large $\lambda_{1}$, and
the observation aligns with our $F_{1}$ settings in Eq. (4) where a large
$\lambda_{1}$ increases the importance of battery degradation. The emphasis of
one implies the reduced importance of the other, and we can observe that the
energy cost goes up when the degradation cost lowers. This parameter is
helpful for practical usage of Bach with customised configurations for
different applications. Furthermore, the importance of energy loss and voltage
deviation varies in different scenarios in reality, e.g., mission critical
systems like data centers have stringent requirements for voltage deviation.
Our formulation supports customized control of both factors by adjusting
$\lambda_{2}$ in $F_{2}$, and we show our results in Fig. 3(b). With large
$\lambda_{2}$, voltage deviation has a big weight in $F_{2}$ and there is a
downtrend (means improvement) of deviation as $\lambda_{2}$ increases.
However, this comes with a cost, i.e., an uptrend of energy loss.

##  V Conclusion

In this paper, we present Bach, for battery degradation-aware scheduling with
a novel multi-objective framework. Bach considers both monetary cost and
network performance for optimization, where the former is based on energy
expenditure and battery degradation and the latter incorporates energy loss
and voltage deviation. The experiments, based on the IEEE 33-bus network,
demonstrate Bach’s effectiveness in saving costs with extended battery life
and maintaining competitive network performance. Bach, therefore, has the
potential to contribute to sustainable and resilient energy systems.

## References

  * [1] M. Chithra, P. Nithara, R. P. Eldho _et al._ , “Overview on battery management system and energy storage system of electric vehicle,” in _2022 International Conference on Automation, Computing and Renewable Systems (ICACRS)_. IEEE, 2022, pp. 143–147.
  * [2] J.-O. Lee and Y.-S. Kim, “Novel battery degradation cost formulation for optimal scheduling of battery energy storage systems,” _International Journal of Electrical Power & Energy Systems_, vol. 137, p. 107795, 2022.
  * [3] Q. Chai, C. Zhang, Z. Dong _et al._ , “Optimal daily scheduling of distributed battery energy storage systems considering battery degradation cost,” in _2021 IEEE Power & Energy Society General Meeting (PESGM)_. IEEE, 2021, pp. 1–5.
  * [4] M. Bahloul, M. Daoud, and S. K. Khadem, “Optimal dispatch of battery energy storage for multi-service provision in a collocated pv power plant considering battery ageing,” _Energy_ , vol. 293, p. 130744, 2024.
  * [5] S. Zhou, Y. Han, K. Mahmoud _et al._ , “A novel unified planning model for distributed generation and electric vehicle charging station considering multi-uncertainties and battery degradation,” _Applied Energy_ , vol. 348, p. 121566, 2023.
  * [6] N. Şensoy, “Determination of optimal battery and renewable capacity investment for hybrid energy storage systems,” in _2023 19th International Conference on the European Energy Market (EEM)_. IEEE, 2023, pp. 1–6.
  * [7] C. A. Cambambi, R. R. Lautert, C. A. Rangel _et al._ , “Model for cooperative operation of li-ion batteries and hydrogen storage systems in microgrids considering cycling costs and dynamic prices,” in _2023 IEEE Power & Energy Society General Meeting (PESGM)_. IEEE, 2023, pp. 1–5.
  * [8] P. V. Babu and S. Singh, “Optimal placement of dg in distribution network for power loss minimization using nlp & pls technique,” _Energy Procedia_ , vol. 90, pp. 441–454, 2016.
  * [9] V. B. Pamshetti and S. P. Singh, “Optimal coordination of pv smart inverter and traditional volt-var control devices for energy cost savings and voltage regulation,” _International Transactions on Electrical Energy Systems_ , vol. 29, no. 7, p. e12042, 2019.
  * [10] J. Zhu, W. J. Nacmanson, and L. F. Ochoa, “Producing realistic ev demand profiles for distribution network studies,” in _CIRED Porto Workshop 2022: E-mobility and power distribution systems_ , vol. 2022. IET, 2022, pp. 706–710.


††thanks: corresponding author<EMAIL_ADDRESS>
# Amplitude mode of charge density wave in TTF[Ni(dmit)2]2 observed by
electronic Raman scattering

M. Revelli Beaumont1,2 Y. Gallais1 A. Sacuto1 K. Jacob2 L. Valade2 D. de Caro2
C. Faulmann2 M. Cazayous1 1Laboratoire Matériaux et Phénomènes Quantiques (UMR
7162 CNRS), Université de Paris, 75205 Paris Cedex 13, France2Laboratoire de
Chimie de Coordination (UPR 8241), Université Paul Sabatier Toulouse, France

###### Abstract

We measured the optical signature of the charge density waves (CDWs) in the
multiband conductor TTF[Ni(dmit)2]2 by electronic Raman scattering. At low
energies, a hump develops below 60 K. This hump is associated to the amplitude
mode of the CDW with an energy around 9 meV. Raman symmetry-resolved
measurements show that the CDW amplitude mode is anisotropic and that the CDW
can be associated to the band nesting of Ni(dmit)2 chains.

Charge density waves (CDWs) are a widespread physical phenomenon in condensed
matter and are observed in many solids, especially in low-dimensional
systems.Gruner  CDWs can be grouped into three main categories.Zhu ; Martin

The first category corresponds to CDWs that have their origin in the
instability that occurs below a TCDW critical temperature, as described by
Peierls.Peierls  A Peierls instability is a dimerization of the lattice
crystal and the opening of an energy gap at the Fermi level associated to an
electronic transition from metal to insulator. The decrease of the electron
energy modulates the electron density of the system. It generates the
spontaneous modulation of the crystal lattice by the electron-phonon
interactions.Kohn ; Mazin  In the Peierls picture, the Fermi surface nesting
with a vector qCDW is at the origin of CDWs resulting in a strong peak in the
susceptibility at qCDW and a sharp dip, the so-called Kohn anomaly, in phonon
dispersion at 2qCDW.

In the second group, CDWs originate from electron-phonon interactions but are
not driven by the nesting and the transition is not associated to a metal-
insulator transition. The signature of the transition results in a phonon mode
at qCDW with an energy going to zero at TCDW. Among dichalcogenides, NbSe2 is
an example of this category.

The third categorie describes CDWs associated to a charge modulation without
evidences for nesting or electron-phonon interactions. Such a behavior for the
CDWs is observed in unconventional superconductors such as cuprates.Chang ;
Comin ; Loret ; Lee

A CDW state is described by an amplitude mode (amplitudon) and a phase mode
(phason).Lee  The phason corresponds to the vibration of the electron density
wave in a rearranged lattice and the amplitudon modulates the magnitude of the
gap leading to oscillations of the amplitude order parameter. The oscillation
frequency corresponds to the phonon mode driving the nesting. For several
years now, CDWs have emerged as a phenomenon that interacts or competes with
other orders (superconductivity, …) which has led to a strong renewal of
interest for this wave.Monceau  It is still in NbSe2 that one find a good
example of the interaction between orders. In this compound, superconductivity
coexists with a CDW order. Spectroscopic probes point out the so-called
”Higgs” mode which becomes active by removing spectral weight from the CDW
amplitude mode upon entering the superconducting state.Grasset ; Tsang  The
coupling between the CDW and the Higgs mode is made possible by the fact that
the amplitude mode of the CDW ”shakes” the density of states at the Fermi
level and then modulates the amplitude of the superconducting order
parameter.Cea

Figure 1: Projection onto the (010) plane of the TTF[Ni(dmit)2]2 crystal
structure. Slabs of Ni(dmit)2 alternate with slabs of TTF.

When one thinks to one-dimensional systems and superconductivity, the physics
of organic superconductors quickly comes to mind. The superconductivityParkin
and CDWRavy  has been observed in (BEDT-TTF)2ReO4. It is in this compound that
the competition between the two orders has been first suspected.Kaddour  A
phenomenom that fosters competition between orders is the contribution to the
Fermi level of molecular orbitals. Such a situation is encountered in the
TTF[M(dmit)2]2 family in which TTF is the tetrathiafulvalene molecule and
dmit2 is the 2-thioxo-1,3-dithiole-4,5-dithiolato anion (M is the Ni, Pd or Pt
atom).Canadell  The present work focuses on TTF[Ni(dmit)2]2 . A comparison
between several one-dimensional compounds with a CDW such as TTF-TCNQ, BaVS3
or blue bronze K0.3MoO3 can be found in the review article of J-P. Pouget and
in the book of D. Jérome and L. G. Caron.Pouget ; Garon

Figure 1 shows a projection on the ac plane of the TTF[Ni(dmit)2]2 structure.
In this plane, TTF molecules alternate with blocks of Ni(dmit)2 molecules
along the crystallographic a axis. These molecules stack up in the
crystallographic direction b to give rise to columns of TTF cations
alternating with columns of Ni(dmit)2 anions. The structure can be then
described as alternating layers of TTF and Ni(dmit)2. The b axis is the
crystal elongation axis.Cassoux  Brossard et al. discovered that this material
becomes superconducting at TC = 1.6 K under a pressure of 7 kbar.Brossard  At
ambient pressure, 1D CDW fluctuations along the stacking direction b with the
wave vector q = 0.40(2)b∗ has been observed by X-ray diffuse scattering
experiments and the existence of possible successive CDW transitions below 60
K has been proposed.Ravy89  In this phase, the metallic state coexists with a
CDW state with its own nesting wave vector. More recently, two successive CDW
transitions have been identified around 55 K and 35 K and associated to the
Ni(dmit)2 chains comparing resistivity measurements under pressure and band
structure calculations.Kaddour

In this letter, we are able to directly detect by Raman spectroscopy the CDW
amplitude mode and to determine its appearance temperature and its energy.
Furthermore, polarization analyses show that the CDW is associated to the
Ni(dmit)2 chains.

Figure 2: Raman spectra of TTF[Ni(dmit)2]2 measured between 10 and 240 cm-1
with the z(bb)z̄ configuration as a function of temperature. Only a few
spectra have been plotted for an easier visualization of changes. The inset
shows the Raman spectra measured at 3 and 100 K using the laser wavelength
$\lambda$ = 660 nm and the same polarizations.

Single crystals are obtained by a slow interdiffusion of saturated solutions
of (TTF)3(BF4)2 and (n-Bu4N)[Ni(dmit)2].Bousseau  The crystal has the
monoclinic symmetry and crystallizes in the C 2/c space group. High quality
TTF[Ni(dmit)2]2 single crystals have a typical size 2 $\times$ 0.15 $\times$
0.075 mm3 with the largest dimension along the b direction. Raman scattering
measurements have been performed using a triple spectrometer Jobin Yvon T64000
equipped with a liquid-nitrogen-cooled CCD detector. The incident laser spot
is about 50 $\mu$m diameter size and the power is equal to 2 mW, small enough
to keep the laser heating as low as possible and large enough to improve the
ratio of the signal from the sample to the noise. The laser lines used to
probe the sample, from solid state lasers, are at 532 nm and 660 nm.
Measurements between 3 and 300 K have been performed using an Cryomech closed-
cycle He cryostat. The Raman spectra have been measured in backscattering
using two optical configurations z(bb)z̄ and z(cc)z̄.Porto1966  The incident
wave vector is anti-parallel to the scattered one and both are along z axis.
The incident and scattered polarizations of the light are along the b and c
axis of the sample.

Figure 2 shows the Raman spectra measured at low energies between 5 K and 150
K using the z(bb)z̄ configuration. At high energy beyond 160 cm-1 the spectra
are superimposed without additional modifications (vertical translation, …).
At 3 K, we observe a phonon mode at 140 cm-1 and a lower energy hump centered
around 70 cm-1. The phonon modes of TTF[Ni(dmit)2]2 can be found in Ref.
Valade, . In our measurements, we do not observe phonons below 100 cm-1
whereas intermolecular modes are expected. Notice that there are no studies or
calculations showing phonons on TTF[Ni(dmit)2]2 below 100 cm-1. Their absences
have to be noticed and might be explained by their intensities which are too
low to be detected or by resonance effects. The phonon signal is might be also
superimposed on the signal of the charge density wave. This last point might
explain the shape of the hump associated with the CDW around 80 cm-1. However,
they should appear above the temperature of the CDW in Fig. 2 and with the
subtraction of the spectra in Fig. 3 which is not the case. The phonon mode at
140 cm-1 corresponds to the whole Ni(dmit)2 deformation mode.Pokhodnya  As the
temperature increases, the phonon frequency decreases as well as its intensity
due to the thermal expansion of the lattice. In the same temperature range,
the estimated softening with temperature of the hump is about 5 cm-1 but
corresponds to the uncertainty due to the width of the hump. Its intensity
decreases until the spectra overlap above 100 K.

Figure 3: (a) Raman spectra at 3 K and 100 K substrated from the one at 150 K,
(b) Normalized area of the hump at 70 cm-1 as a function of temperature.

The Raman response at 3 K ($\chi^{"}_{3K}$) and at 100 K ($\chi^{"}_{100K}$)
subtracted from the one at 150 K ($\chi^{"}_{150K}$) have been plotted in Fig.
3(a). The $\chi^{"}$($\omega$) response is obtained by correcting all Raman
spectra for the spectrometer response and the Bose factor. The Bose factor
linking the Raman intensity and the imaginary part of the Raman response
function stems from the fluctuation dissipation theorem and is a general
property of inelastic scattering techniques.Hayes  The electronic Raman
response is proportional to the imaginary part of the density-density
correlation function of the material. In order to access to this quantity, we
need to divide the experimental Raman response from the Bose factor and made
the correction of spectrometer response.

This difference clearly highlights the low energy hump and allows us to
determine its energy a slightly above 70 cm-1. The analysis of the area under
this hump as a function of temperature shows that this excitation disappears
around 60 K $\pm$ 5 K as shown in Fig. 3(b).

Notice that X-ray measurements reported by S. Ravy et al. show satellite
reflections below 60 K pointing out the appearance of a CDW in TTF[Ni(dmit)2]2
.Ravy89  The CDW transition has been recently identified around 55 K.Kaddour
Our temperature measurements indicate that the hump observed at low energies
in the Raman spectra corresponds to the signature of the CDW mode. However,
this hump might be associated at first glance to another origin as a phonon
mode (inter-molecular phonon mode or a finite q-phonon activated by the CDW)
or luminescence for example. The width of the phonons is generally in the
order of a few cm-1 at low temperatures in single crystals as shown in Fig. 2
with the full width at half maximum (FWHM) of the 140 cm-1 phonon equals to 10
cm-1. The width of the hump (FWHM around 60 cm-1) does not plead for a phonon,
in contrast to the width of amplitude mode in 2H-TaS2 for example measured
around 40 cm-1.Grasset  To rule out a luminescence signal that would appear
unfortunately at 60 K, we have performed measurements with an another laser
wavelength $\lambda$ = 660 nm (inset of Fig. 2). The hump doesn’t shift in
energy with the laser wavelength allowing to exclude this possibility as an
origin of the hump.

The question is now to know whether this hump is associated to the gap or to
the amplitude mode of the CDW. The amplitude mode corresponds to the vibration
of the ions due to the intensity oscillation of the maximum/minimum charge
density and the magnitude of the CDW gap.Lee ; Rice  A first argument is given
by the expected signature of a gap in Raman spectra. A gap opening is
characterized by a transfer of spectral weight in the electronic background
from low to high energy. Figures 2 and 3(a) clearly show that this is not the
case : the hump is decreasing in intensity with temperature without exhibiting
a low energy spectral weight recovery. A second argument can be obtained by
comparing our measurements to mean field theory. The Raman peak associated to
the gap $\Delta_{CDW}$ of the CDW is expected at 2$\Delta_{CDW}$.Vanyolos  If
we associate the hump of Fig. 2 with the gap of the CDW, it means that
$\Delta_{CDW}$ is about 70 cm-1, equivalent to 9 meV. As a first
approximation, we can use mean field theory to relate the gap $\Delta_{CDW}$
to TCDW i.e. 2$\Delta_{CDW}$ = $\alpha k_{B}T_{CDW}$. In 1D system, $\alpha$
has an expected value much higher than the theoretical value equal to 3.52.
The experimental ratio gives 1.7, twice as low as the theoretical ratio. The
observed hump is then unlikely related to the gap of the CDW. One possibility
remains, the observed hump in our Raman spectra could be associated to the
amplitude mode of the CDW order that appears below TCDW = 60 K. The fact that
the amplitude mode softens little with temperature can be observed in
transition metal dichalcogenide for example.Grasset  Deviation from the
expected mean-field behavior for the order parameter is not surprising in low
dimensional systems. As observed in other systems,Tsang ; Measson  Raman
spectroscopy is sensitive to the amplitude mode of the charge wave and not
directly to the gap.

Figure 4: Raman spectra with (a) incident and scattered polarizations along
the c axis (z(cc)z̄ configuration) and (b) crossed polarizations configuration
(z(bc)z̄) as a function of temperature.

In addition, the CDW has been measured with the z(cc)z̄ and z(bc)z̄
configurations in Fig. 4(a) and (b) respectively. Since the intensity of the
hump is very weak in Fig. 4(a) and drops rapidly with increasing temperature,
we are not able experimentally to show that the hump exists between 30 and 60
K. The spectra are almost superimposable beyond 30 K. The hump does not exist
in cross polarizations. These measurements show that the Raman response of the
CDW amplitude mode is anisotropic and almost exclusively along the b axis.
Calculated electronic Raman cross section in quasi-one-dimensional
interacting-electron systems with density wave ground state have shown that
collective contributions to the Raman response appear only with polarizations
along the chains associated to the CDW.Vanyolos  The b axis corresponds here
to the directions of the Ni(dmit)2 stacks.

The observation of the CDW signature perpendicular to the Ni(dmit)2 stacks in
Fig. 4(a) is not expected due to the one-dimensionality of the CDW in this
compound. This signal might be a remnant of the signal along the b axis due to
the fact that we are not perfectly along the c axis. We also note that if the
CDW is along the Ni(dmit)2 stacks (b axis), it is not obvious that the Raman
signature of the CDW appears only with polarizations along the b axis. For
example, the amplitude mode in NbSe2 is observed using two different
polarization configurations (A1 and E).Sooryakumar  Naively it is expected to
appear with only one polarization configuration, the fully symmetric A1. The
reason for the disagreement is not yet well understood, but an alternative
explanation has been proposed based on anharmonic effects.Klein

TTF[Ni(dmit)2]2 is a quasi-1D material with uncorrelated spins on the
Ni(dmit)2 stacks. Bourbonnais 2 performed nuclear magnetic resonance (NMR)
measurements on TTF stacks and shown that the TTF chains keep a metallic
behavior down to low temperatures without gap opening.Bourbonnais  So the CDW
cannot be associated to these chains. On the other hand, NMR measurements on
Ni(dmit)2 chains attribute the CDW to these chains.Vainrub  The CDW
correlation length along the chains of the order of b at room temperature
increases above 20 nm at 55 K. However, there is no interchain CDW
correlations up to 55K. Below this temperature the authors show that
TTF[Ni(dmit)2]2 undergoes at 55 K a CDW transition related to the nesting of
the LUMO band of the Ni(dmit)2 stacks. Notice that the CDW lateral order is
not perfect probably due to the Coulomb coupling between CDWs. The authors
show that TTF[Ni(dmit)2]2 undergoes at 55 K a CDW transition related to the
nesting of the LUMO band of the Ni(dmit)2 stacks and a second CDW transition
at 35 K associated to the nesting of the HOMO bands. The insulating ground
state is thus considered to be CDW of the Ni(dmit)2 stacks with different wave
vectors. Consequently, our measurements support the idea that CDW is
associated to the Ni(dmit)2 stacks.

What are the prospects of the direct measurement of CDW in an organic
conductor ? Raman experiments contributed significantly to the establishment
of the d-wave nature of the order parameter in high-temperature
superconductors.Devereaux  This technique also allows to investigate the
temperature and pressure dependence of the CDW amplitude mode in several
systems.Wilson ; Friend  In M(dmit)2-based compounds, superconductivity
develops under pressure in competition with a CDW ground state.Brossard ;
Brossard89  The possibility to probe superconductivity and the CDWs as shown
in this work with the same technique will allow to study more efficiently the
interaction between the two orders in organic compounds.

To conclude, we have optically highlighted the CDW signature in the molecular
conductor TTF[Ni(dmit)2]2 . This allowed us to determine precisely the energy
of the CDW amplitude mode and its temperature of appearance. The polarization
measurements show that the CDW amplitude mode is very anisotropic and that the
CDW originates from the Ni(dmit)2 stacks.

## References

  * (1) G. Gruner, Density Waves in Solids, Addison-Wesley, Reading, MA (1994).
  * (2) X. Zhu, Y. Cao, J. Zhang, E. W. Plummer, and J. Guo, PNAS 112, 2367 (2015).
  * (3) M. Dressel and S. Tomic,́ Adv. Phys. 69, 1 (2020).
  * (4) R. E. Peierls, Quantum Theory of Solids, Oxford Univ Press, New York (1955).
  * (5) W. Kohn, Phys. Rev. Lett. 2, 393 (1959).
  * (6) M. D. Johannes and I. I. Mazin, Phys. Rev. B 77, 165135 (2008).
  * (7) J. Chang et al., Nature Physics 8, 871 (2012).
  * (8) R. Comin et al., Science 343, 390 (2014).
  * (9) B. Loret et al., Nat. Phys. 15, 771 (2019).
  * (10) P. A. Lee, T. M. Rice, and P. W. Anderson, Solid State Commun. 14, 703 (1974).
  * (11) P. Monceau, Adv. Phys. 61, 325 (2012).
  * (12) R. Grasset, Y. Gallais, A. Sacuto, M.Cazayous, S. Manas-Valero, E. Coronado, M-A. Measson, Phys. Rev. Lett. 122, 127001 (2019).
  * (13) J. C. Tsang, J. E. Smith Jr., and M. W. Shafer, Phys. Rev. Lett. 37, 1407 (1976).
  * (14) T. Cea and L. Benfatto, Phys. Rev. B 90, 224515 (2014).
  * (15) S. S. P. Parkin, E. M. Engler, R. R. Schumaker, R. Lagier, V. Y. Lee, J. C. Scott, and R. L. Greene, Phys. Rev. Lett. 50, 270 (1983).
  * (16) S. Ravy, R. Moret, J. P. Pouget, R. Comés, and S. S. P. Parkin, Phys. Rev. B 33, 2049 (1986).
  * (17) W. Kaddour, P. Auban-Senzier, H. Raffy, M. Monteverde, J. P. Pouget, C. R. Pasquier, P. Alemany, E. Canadell, and L. Valade, Phys. Rev. B 90, 205132 (2014).
  * (18) E. Canadell, I. E.-I. Rachidi, S. Ravy, J.-P. Pouget, L. Brossard, and J.-P. Legros, J. Phys. France 50, R2967 (1989).
  * (19) J-P. Pouget, C. R. Physique 17, 332 (2016).
  * (20) D. Jérome, and L. G. Caron, Low Dimensional Conductors and Superconductors, Plenum Press, New York (1987).
  * (21) P. Cassoux, L. Valade, M. Bousseau, J-P. Legros, M. Garbauskas, and L. Interrante, Mol. Cryst. Liq. Cryst. 120, 377 (1985).
  * (22) L. Brossard, M. Ribault, L. Valade, and P. Cassoux, Phys. Rev. B 42, 3935 (1990).
  * (23) S. Ravy, J.-P. Pouget, L. Valade, and J.-P. Legros, Europhys. Lett. 9, 391 (1989).
  * (24) M. Bousseau, L. Valade, J.-P. Legros, P. Cassoux, M. Garbauskas, and L. V. Interrante, J. Am. Chem. Soc. 108, 1908 (1986).
  * (25) S. P. S. Porto, J. A. Giordmaine, and T. C. Damen, Phys. Rev. 147, 608 (1966).
  * (26) L. Valade et al., J. of Solid State Chem. 168, 438 (2002).
  * (27) K.I. Pokhodnya et al., Synth. Met. 103, 2016 (1999).
  * (28) W. Hayes and R. Loudon, Scattering of light by crystals, Wiley, New York (1978).
  * (29) M. J. Rice and S. Strassler, Solid State Commun. 13, 1931 (1974).
  * (30) A. Ványolos, and A. Virosztek, Phys. Rev. B 72, 115119 (2005).
  * (31) M.-A. Méasson, Y. Gallais, M. Cazayous, B. Clair, P. Rodiére, L. Cario, and A. Sacuto, Phys. Rev. B 89, 060503(R) (2014).
  * (32) C. Bourbonnais, P. Wzietek, D. Jérome, F. Creuzet, L. Valade, and P. Cassoux., Europhys. Lett. 6, 177 (1988).
  * (33) A. Vainrub, D. Jérome, M. F. Bruniquel, and P. Cassoux, Europhys. Lett. 12, 267 (1990).
  * (34) R. Sooryakumar and M. V. Klein, Phys. Rev. B 23, 3213 (1981).
  * (35) M. V. Klein, Phys. Rev. B 25, 7192 (1982).
  * (36) T. P. Devereaux, D. Einzel, B. Stadlober, R. Hackl, D. H. Leach, and J. J. Neumeier, Phys. Rev. Lett. 72, 396 (1994).
  * (37) J. A. Wilson, F. J. Salvo, and S. Mahajan, Phys. Rev. Lett. 32, 882 (1974).
  * (38) R. H. Friend and A. D. Yo, Adv. Phys. 36, 1 (1987).
  * (39) L. Brossard, M. Ribault, L. Valade, and P. Cassoux, J. Phys. 50, 1521 (1989).


Copyright for this paper by its authors. Use permitted under Creative Commons
License Attribution 4.0 International (CC BY 4.0).

CLEF 2022: Conference and Labs of the Evaluation Forum, September 5–8, 2022,
Bologna, Italy

[orcid=0000-0003-4973-0963<EMAIL_ADDRESS>url=https://researchers.mq.edu.au/en/persons/diego-molla-aliod, ]

# Query-focused Extractive Summarisation for Biomedical and COVID-19 Complex
Question Answering

Diego Mollá  Macquarie University, Australia

(2022)

# Macquarie University’s Participation at BioASQ10 Synergy and BioASQ10b Phase
B

Diego Mollá  Macquarie University, Australia

(2022)

###### Abstract

This paper presents Macquarie University’s participation to the two most
recent BioASQ Synergy Tasks (as per June 2022), and to the BioASQ10 Task B
(BioASQ10b), Phase B. In these tasks, participating systems are expected to
generate complex answers to biomedical questions, where the answers may
contain more than one sentence. We apply query-focused extractive
summarisation techniques. In particular, we follow a sentence classification-
based approach that scores each candidate sentence associated to a question,
and the $n$ highest-scoring sentences are returned as the answer. The Synergy
Task corresponds to an end-to-end system that requires document selection,
snippet selection, and finding the final answer, but it has very limited
training data. For the Synergy task, we selected the candidate sentences
following two phases: document retrieval and snippet retrieval, and the final
answer was found by using a DistilBERT/ALBERT classifier that had been trained
on the training data of BioASQ9b. Document retrieval was achieved as a
standard search over the CORD-19 data using the search API provided by the
BioASQ organisers, and snippet retrieval was achieved by re-ranking the
sentences of the top retrieved documents, using the cosine similarity of the
question and candidate sentence. We observed that vectors represented via
sBERT have an edge over tf.idf. BioASQ10b Phase B focuses on finding the
specific answers to biomedical questions. For this task, we followed a data-
centric approach. We hypothesised that the training data of the first BioASQ
years might be biased and we experimented with different subsets of the
training data. We observed an improvement of results when the system was
trained on the second half of the BioASQ10b training data.

###### keywords:

BioASQ Synergy query-focused summarisation Biomedical COVID-19 DistilBERT
sBERT data-centric

##  1 Introduction

The BioASQ challenge111http://www.bioasq.org/ organises shared tasks on
biomedical semantic indexing and question answering. In this paper, we present
Macquarie University’s participation in several of these tasks.222Code related
to this paper is available at https://github.com/dmollaaliod/bioasq10b-public
and https://github.com/dmollaaliod/bioasq10-synergy-public.

The Synergy tasks aim to evaluate technologies useful for the development of
an end-to-end question answering (QA) system for questions about COVID-19
asked by biomedical experts. In particular, the Synergy tasks evaluate the
quality of document retrieval over a snapshot of CORD-19 [1], snippet
retrieval, and the generation of “ideal answers” that may contain multiple
sentences. We present our participation in the second BioASQ9 Synergy task
that ran between May and June 2021, and the BioASQ10 Synergy task that ran
between December 2021 and February 2022.

Task B of BioASQ focuses on biomedical semantic QA. Similar to the Synergy
tasks, several technologies corresponding to components of an end-to-end QA
system are evaluated. In contrast with the Synergy tasks, Task B of BioASQ has
two distinct phases. Phase A evaluates the quality of document and snippet
retrieval on a snapshot of PubMed333https://pubmed.ncbi.nlm.nih.gov/, whereas
Phase B, given a question, its question type (“summary”, “factoid”, “yesno”,
“list”) , and a list of candidate snippets, evaluates the system’s ability to
find short answers (“exact answers”) and long, possibly multi-sentence answers
(“ideal answers”). Figure 1 shows an example of a question and its question
type, a correct snippet for the question, a correct exact answer, and a
correct ideal answer. We present our participation in Task B, Phase B of
BioASQ10, that ran between March and May 2022 (henceforth BioASQ10b, Phase B).

Question

Orteronel was developed for treatment of which cancer?

Type

factoid

Snippet

Pooled-analysis was also performed, to assess the effectiveness of agents
targeting the androgen axis via identical mechanisms of action (abiraterone
acetate, orteronel).

Exact answer

castration-resistant prostate cancer

Ideal answer

Orteronel was developed for treatment of castration-resistant prostate cancer.

Figure 1: An example question with its question type, a relevant snippet, an
exact answer, and a correct ideal answer, extracted from the training data of
BioASQ10b

All of our contributions to the above tasks are based on a common question-
answering architecture that we will describe in Section 2. Section 3 presents
our participation in the Synergy tasks. Section 4 presents our participation
in BioASQ10b, Phase B. Finally, Section 5 concludes this paper.

##  2 Question Answering Architecture

The question-answering system that is the focus of our participation in all of
the tasks presented in this paper is based on query-focused extractive
summarisation. The architecture of the system is illustrated in Figure 2, and
follows the classification set up proposed by [2].

sentencequestion BERT word embeddings Mean sentence embeddingsrelu$\int$
sigmoid sentence position Figure 2: Architecture of the question answering
system used for BioASQ9b, Phase B.

The query-focused summarisation system takes the question, a candidate
sentence, and the sentence position444The sentence position was incorporated
as an absolute number: 1, 2, …$n$, where $n$ is the total number of input
sentences. We chose to include the sentence position as earlier experiments in
past BioASQ years showed an improvement of the results., and calculates a
sentence score. The system computes the word embeddings of the question and
candidate sentence using a BERT architecture [3]. In particular, for the
BioASQ9 Synergy task 2 we used ALBERT [4], which was the best-performing
system in [5]’s participation in BioASB8b555At the time of training the system
for the BioASQ9 Synergy task 2, the final results of BioASQ9 had not been
released yet.. For the BioASQ10 Synergy task, we used DistilBERT [6], which
performed very well in [2]’s participation in BioASQ9b, and even outperformed
BioBERT [7]. For BioASQ10, Phase B, we also used DistilBERT. Average pooling
is then used to merge the word embeddings of the candidate sentence into the
sentence embeddings. The sentence position is then concatenated to the
sentence embeddings, and an additional intermediate dense layer is added. A
final classification layer predicts the sentence score.

The question and the sentence were fed to BERT in the same way as defined by
the creators of BERT [3]. That is, the input consisted of an initial “[CLS]”
token, followed by the question text, then a “[SEP]” token that indicates a
new sentence, and finally the candidate sentence text. This information was
passed to BERT, indicating the question and the candidate sentence as two
separate text segments.

The classification labels used for training the system were automatically
generated from the training data, based on the ROUGE score of the candidate
sentence with respect to the annotated ideal answer. In particular, given a
particular question, the top 5 sentences according to their ROUGE score were
labelled as 1, and the rest were labelled as 0. For the Synergy tasks we used
the BioASQ9b training data, whereas for BioASQ10b, Phase B, we used the
BioASQ10b training data.

We used the pre-trained ALBERT and DistilBERT models available by
Huggingface666https://huggingface.co/ — For ALBERT, we used ‘albert-
xxlarge-v2’. For DistilBERT, we used ‘distilbert-base-uncased’.. These models
were frozen during training, so that only the weights of the additional layers
shown in Figure 2 were updated.

##  3 The Synergy Tasks

This section describes the systems that participated in the Synergy task 2 of
BioASQ9, and the Synergy task of BioASQ10 (in this paper, we will use the
collective expression “the Synergy tasks” to refer to these). The Synergy task
2 of BioASQ9 ran in 2021 but the results were not made available at the time
of the paper submission deadline for BioASQ9. For this reason, we are
describing the system in this paper.

Our participation in the Synergy tasks share the same question answering
system architecture described in Section 2. The only difference between the
two Synergy tasks is, as mentioned in Section 2, that the BioASQ9 Synergy 2
system used ALBERT, whereas the BioASQ10 Synergy system used DistilBERT. In
both cases, the system was trained with the training data of BioASQ9b.

To generate the candidate sentences required by the question answering system,
we followed this procedure:

  1. 1.

Retrieve the most relevant documents as described in Section 3.1;

  2. 2.

Split the retrieved documents into sentences and select the candidate
sentences as described in Section 3.2.

###  3.1 Document Retrieval

The relevant documents were retrieved using the search API provided by the
organisers of the BioASQ Synergy task. This API is based on a Web service that
accepts a query and returns a JSON data structure. We simply used the
unmodified question as the search query. In subsequent work we are exploring
pre-processing and fine-tuning steps to improve the quality of the Document
Retrieval stage.

The final runs submitted consist of the top 10 documents, after removing those
that were in previous feedback, to conform with the submission requirements.

###  3.2 Snippet Retrieval

Every sentence from every retrieved document was a candidate snippet. This
includes sentences from documents that were retrieved but were not submitted
in the Document Retrieval runs. We then experimented with the combination of 2
dimensions to re-rank the candidate snippets, for a total of 4 different
approaches.

The first dimension was based on the calculation of the similarity between the
question and candidate snippet. We experimented with the following two
options:

#### TfidfCosine.

We represented the question and candidate sentences using tf.idf. Each
candidate sentence was then scored based on the cosine similarity between the
question vector and the sentence vector.

#### sBERTCosine.

We used sBERT [8] to represent the question and the candidate sentences, and
to determine the similarities between the question and the sentences. We used
the default set up for sBERT, which computes the cosine similarity between the
question vector and the sentence vector.

The second dimension was based on the criteria used for the final ranking of
the candidate sentences. We experimented with local sorting and global
sorting.

#### LocalSorting.

For every relevant document, we extracted the top 3 sentences according to the
cosine similarity approaches described above. The final list of sentences was
composed of the top 3 sentences from the top document, followed by the top 3
sentences of the second document, and so on.

#### GlobalSorting.

In contrast to the local sorting approach, all sentences of all documents were
now sorted according to their cosine similarity with the question, regardless
of what document the snippets were obtained from.

The final runs submitted consist of the first 10 snippets, after removing
those that were in previous feedback, to conform with the submission
requirements.

###  3.3 Answer Generation

As mentioned above, the question, candidate sentences, and sentence position
were fed to the system illustrated in Figure 2. The sentence position was
simply the unnormalised position of the sentence within the list of snippets,
after the snippets have been ranked as described in Section 3.2. Given a
question, the top-scoring $n$ sentences according to the scores produced by
the QA system were combined to form the final answer. These sentences were
presented in the order of appearance in the list of snippets. The value of $n$
was based on the question type and is shown in Table 1.

Table 1: Number of sentences selected, for each question type | Summary | Factoid | Yesno | List
---|---|---|---|---
n | 6 | 2 | 2 | 3

###  3.4 Results of the Synergy Tasks

This section describes the results of the runs submitted to the Synergy tasks.

Table 2 shows the F1 score of the documents returned by our systems. As
mentioned in Section 3.1, these documents were found by submitting the
unmodified question as the query to the search API provided by the developers
of the Synergy task. As expected, the results were poor relative to other
submissions.

Table 2: Document retrieval results of the submissions to the BioASQ9 Synergy 2 (top) and BioASQ10 Synergy (bottom) tasks. Metric: F1. The results of rows labelled “Best”, “Median”, and “Worst” refer to the results of other systems, other than our own, submitted to the challenge. Run | Round 1 | Round 2 | Round 3 | Round 4
---|---|---|---|---
Best | 0.3693 | 0.2039 | 0.1327 | 0.1896
Median | 0.2388 | 0.1423 | 0.0710 | 0.0800
Worst | 0.0157 | 0.0067 | 0.0053 | 0.0175
MQ-BioASQ9 | 0.1978 | 0.1087 | 0.0483 | 0.0800
Best | 0.3220 | 0.2221 | 0.1970 | 0.1564
Median | 0.3100 | 0.1646 | 0.1327 | 0.1067
Worst | 0.2729 | 0.1003 | 0.0655 | 0.0478
MQ-BioASQ10 |  | 0.1003 | 0.0754 | 0.0808

Table 3 shows the F1 score of the snippets returned by our runs. For each run,
we indicate the run name, the type of similarity used, and the type of sorting
performed. We observe that, considering the poor quality of the documents
retrieved, the snippets were of quality comparable to that of other runs of
the BioASQ9 Synergy 2 task (but not the runs of the BioASQ10 Synergy task),
but there is room for improvement. Among our runs, the most successful
configuration was using sBERT cosine similarity and global sort.

Table 3: Snippet retrieval results of the submissions to the BioASQ9 Synergy 2 (top) and BioASQ10 Synergy (bottom) tasks. Metric: F1. The best of our systems in each round is highlighted in bold. The results of rows labelled “Best”, “Median”, and “Worst” refer to the results of other systems, other than our own, submitted to the challenge. Run | Similarity | Sorting | Round 1 | Round 2 | Round 3 | Round 4
---|---|---|---|---|---|---
Best |  |  | 0.3290 | 0.1726 | 0.1262 | 0.1355
Median |  |  | 0.2288 | 0.1365 | 0.0732 | 0.0764
Worst |  |  | 0.0311 | 0.0101 | 0.0231 | 0.0132
MQ-1-BioASQ9 | tfidf | local | 0.1031 | 0.1035 | 0.0707 | 0.0764
MQ-2-BioASQ9 | tfidf | global | 0.1100 | 0.0540 | 0.0324 | 0.0619
MQ-3-BioASQ9 | sBERT | local | 0.1071 | 0.0999 | 0.0692 | 0.0749
MQ-4-BioASQ9 | sBERT | global | 0.1923 | 0.1075 | 0.1044 | 0.0762
Best |  |  | 0.2910 | 0.1525 | 0.1574 | 0.1217
Median |  |  | 0.2757 | 0.1410 | 0.1087 | 0.0948
Worst |  |  | 0.2296 | 0.0540 | 0.0273 | 0.0416
MQ-1-BioASQ10 | tfidf | local |  | 0.0660 | 0.0465 | 0.0771
MQ-2-BioASQ10 | tfidf | global |  | 0.0540 | 0.0273 | 0.0416
MQ-3-BioASQ10 | sBERT | local |  | 0.0683 | 0.0457 | 0.0770
MQ-4-BioASQ10 | sBERT | global |  | 0.0928 | 0.0725 | 0.0827

Table 4 shows the human evaluation results of the ideal answers returned by
our runs. Our runs are very competitive, especially given the relatively poor
quality of the input snippets. Given the poor quality of the input snippets in
all of our runs, it is dangerous to make generalisations about how the quality
of the snippets affect the quality of the answers. Having said that, we can
observe that, in the BioASQ9 Synergy 2 task, the runs that generated the best
snippets (MQ-4) did not lead to generating the best ideal answers. The impact
of and interplay between the document and snippet retrieval stages, and the
question-answering stage, deserves further exploring.

Table 4: Ideal answer results of the submissions to the BioASQ9 Synergy 2 (top) and BioASQ10 Synergy (bottom) tasks. Metric: Average of human evaluation scores. The best of our systems in each round is highlighted in bold. The results of rows labelled “Best”, “Median”, and “Worst” refer to the results of other systems, other than our own, submitted to the challenge. Run | Similarity | Sorting | Round 1 | Round 2 | Round 3 | Round 4
---|---|---|---|---|---|---
Best |  |  | 4.375 | 3.850 | 3.630 | 3.295
Median |  |  | 3.625 | 3.100 | 3.450 | 3.045
Worst |  |  | 1.042 | 0.450 | 3.290 | 2.060
MQ-1-BioASQ9 | tfidf | local | 3.250 | 3.100 | 3.450 | 3.045
MQ-2-BioASQ9 | tfidf | global | 3.210 | 3.075 | 3.290 | 3.295
MQ-3-BioASQ9 | sBERT | local | 3.372 | 3.250 | 3.520 | 3.067
MQ-4-BioASQ9 | sBERT | global | 2.250 | 3.025 | 3.490 | 3.292
Best |  |  | 3.790 | 3.810 | 3.562 | 3.180
Median |  |  | 3.367 | 3.160 | 3.250 | 2.617
Worst |  |  | 3.287 | 1.550 | 0.827 | 0.372
MQ-1-BioASQ10 | tfidf | local |  | 3.270 | 3.415 | 2.617
MQ-2-BioASQ10 | tfidf | global |  | 3.160 | 3.305 | 2.990
MQ-3-BioASQ10 | sBERT | local |  | 3.360 | 3.517 | 2.925
MQ-4-BioASQ10 | sBERT | global |  | 3.490 | 3.547 | 2.690

##  4 BioASQ10b, Phase B

For BioASQ10b, Phase B, we used the question answering system described in
Section 2, using DistilBERT as the BERT variant chosen to compute the word
embeddings. Following a data-centric approach, the main difference between the
Synergy tasks and BioASQ10, Phase B, is the choice of training data. We
hypothesised that the training data that corresponds to the early years of
BioASQ, that is, the first samples of the BioASQ10b training data, might be
biased. We therefore tested the use of different portions of the training data
as shown in Table 5, by incrementally removing the first samples of the
training data. We can observe that best evaluation results are obtained with
only 50% of the training data.

Table 5: Results of 10-fold cross-validation after removing the _first_ samples of the BioASQ10b training data. Metric: Average ROUGE-SU4 F1. Best result shown in bold. Percentage removed | ROUGE-SU4 F1
---|---
10% | 0.281
20% | 0.288
30% | 0.298
40% | 0.309
50% | 0.311
60% | 0.308

To double-check that indeed the first samples of the training data are biased,
we conducted another round of experiments, but this time removing the last
samples of the training data. Table 6 shows that results worsen as the amount
of training data diminishes, as one might expect in systems that are based on
supervised approaches to machine learning.

Table 6: Results of 10-fold cross-validation after removing the _last_ samples of the BioASQ10b training data. Metric: Average ROUGE-SU4 F1. Percentage removed | ROUGE-SU4 F1
---|---
10% | 0.275
20% | 0.268
30% | 0.270
40% | 0.255
50% | 0.241
60% | 0.229

Hyperparameter search showed that the same hyperparameters give optimal
results when training using the entire training data, or using only 50% of the
training data: dropout=0.6, number of epochs=1.

###  4.1 Submission Results to BioASQ10b, Phase B

Table 7 shows the results of our submissions to BioASQ10b, Phase B777At the
time of writing, only the automated evaluation results were available.. Note
that the results reported in the BioASQ website888http://bioasq.org may change
in the future after the test data is potentially enriched with further
annotations.

Table 7: Preliminary results of the submissions to BioASQ10b, Phase B. The best of our systems in each batch is highlighted in bold. The results of rows labelled “Best”, “Median”, and “Worst” refer to the results of all systems, including our own, submitted to the challenge. |  | ROUGE-SU4 F1 |
---|---|---|---
Run | Training Data | Batch 1 | Batch 2 | Batch 3 | Batch 4 | Batch 5 | Batch 6
Best |  | 0.3715 | 0.4168 | 0.3689 | 0.4165 | 0.3916 | 0.1705
Median |  | 0.3339 | 0.3521 | 0.3387 | 0.3556 | 0.3389 | 0.1581
MQ-1 | All BioASQ10b | 0.3490 | 0.3484 | 0.3344 | 0.3525 | 0.3415 | 0.1581
MQ-2 | Last 50% of BioASQ10b | 0.3339 | 0.3480 | 0.3316 | 0.3556 | 0.3431 | 0.1640

Our runs are comparable to the median of those of other participating systems.
Surprisingly, there is little difference between using all training data or
only the latter 50%. When we visually inspected the outputs of the runs, we
noticed that the output of all runs in each batch were virtually identical,
with only a few differences.

##  5 Summary and Conclusions

We have presented Macquarie University’s contribution to the BioASQ9 Synergy
task 2, the BioASQ10 Synergy task, and BioASQ10b, Phase B (Ideal Answers). In
all of our runs, the base question answering architecture was virtually the
same, the only differences being the choice of DistilBERT vs. ALBERT, and the
training data used.

For the synergy tasks, we used a system that has been trained using BioASQ9b
training data. We experimented with approaches for snippet retrieval based on
two dimensions: vectors used for similarity comparison, and final ranking
approach. Cosine similarity using sBERT gave the best results, and we observed
that not always the best snippets for the snippet retrieval task led to best
answers in the question answering task.

Overall, the results of the question answering parts were competitive,
especially given the relatively poor quality of the documents and snippets
retrieved. We will investigate approaches to increase the quality of the
retrieval stages, and explore the relation between quality of retrieval vs.
quality of final answers.

For the BioASQ10b, Phase B task, we followed a data-centric approach and
experimented with training regimes that incrementally removed samples from the
training data. During our preliminary cross-validation experiments we observed
an improvement of results using only the latter 50% of the training data, but
this difference of results vanished in the submitted runs.

With a data-centric approach in mind, we plan to conduct further experiments
that test the impact of changes and transformations of the training data. For
example, besides further examining the impact of using portions of the
training data, we will investigate the use of data augmentation techniques.

###### Acknowledgements.

This research was undertaken with the assistance of resources and services
from the National Computational Infrastructure (NCI), which is supported by
the Australian Government.

## References

  * Wang et al. [2020] L. L. Wang, K. Lo, Y. Chandrasekhar, R. Reas, J. Yang, D. Burdick, D. Eide, K. Funk, Y. Katsis, R. M. Kinney, Y. Li, Z. Liu, W. Merrill, P. Mooney, D. A. Murdick, D. Rishi, J. Sheehan, Z. Shen, B. Stilson, A. D. Wade, K. Wang, N. X. R. Wang, C. Wilhelm, B. Xie, D. M. Raymond, D. S. Weld, O. Etzioni, S. Kohlmeier,  CORD-19: The COVID-19 open research dataset,  in: Proceedings of the 1st Workshop on NLP for COVID-19 at ACL 2020, Association for Computational Linguistics, Online, 2020. URL: https://www.aclweb.org/anthology/2020.nlpcovid19-acl.1.
  * Mollá et al. [2021] D. Mollá, U. Khanna, D. Galat, V. Nguyen, M. Rybinski,  Query-focused extractive summarisation for finding ideal answers to biomedical and COVID-19 questions,  in: G. Faggioli, N. Ferro, A. Joly, M. Maistro, F. Piroi (Eds.), Working Notes of CLEF 2021 — Conference and Labs of the Evaluation Forum, Bucharest, 2021. URL: http://ceur-ws.org/Vol-2936//paper-20.pdf.
  * Devlin et al. [2019] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova,  BERT: Pre-training of deep bidirectional transformers for language understanding,  in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Computational Linguistics, Minneapolis, Minnesota, 2019, pp. 4171–4186. doi:10.18653/v1/N19-1423.
  * Lan et al. [2020] Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, R. Soricut,  ALBERT: A lite BERT for self-supervised learning of language representations,  in: Proceedings of the 8th International Conference on Learning Representations, Virtual, 2020\. URL: https://iclr.cc/virtual_2020/poster_H1eA7AEtvS.html.
  * Mollá et al. [2020] D. Mollá, C. Jones, V. Nguyen,  Query-focused multi-document summarisation of biomedical texts,  in: L. Cappellato, C. Eickhoff, N. Ferraro (Eds.), Working Notes of CLEF 2020 — Conference and Labs of the Evaluation Forum, Thessaloniki, 2020. URL: http://ceur-ws.org/Vol-2696/paper_119.pdf.
  * Sanh et al. [2019] V. Sanh, L. Debut, J. Chaumond, T. Wolf,  DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter,  in: 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), 2019. arXiv:1910.01108.
  * Lee et al. [2019] J. Lee, W. Yoon, S. Kim, D. Kim, S. Kim, C. H. So, J. Kang,  BioBERT: pre-trained biomedical language representation model for biomedical text mining,  Bioinformatics 36 (2019) 1234–1240. doi:10.1093/bioinformatics/btz682.
  * Reimers and Gurevych [2019] N. Reimers, I. Gurevych,  Sentence-BERT: Sentence embeddings using siamese BERT-networks,  in: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Hong Kong, 2019, pp. 3982–3992. URL: https://www.aclweb.org/anthology/D19-1410/.


# Understanding population fluctuations through volunteered geographic
information and novel indicators: The experience of Rakiura, Stewart Island,
New Zealand

Mathew Darling  Department of Geological Sciences, University of Canterbury
Benjamin Adams  Department of Geography, University of Canterbury  Caroline
Orchiston  Centre for Sustainability, University of Otago  Thomas Wilson
Department of Geological Sciences, University of Canterbury  Brendon Bradley
Civil and Natural Resource Engineering, University of Canterbury

###### Abstract

In an era of heterogeneous data, novel methods and volunteered geographic
information provide opportunities to understand how people interact with a
place. However, it is not enough to simply have such heterogeneous data,
instead an understanding of its usability and reliability needs to be
undertaken. Here, we draw upon the case study of Rakiura, Stewart Island where
manifested passenger numbers across the Foveaux Strait are known. We have
built a population model to ground truth such novel indicators. In our
preliminary study, we find that a number of indicators offer the opportunity
to understand fluctuations in populations. Some indicators (such as wastewater
volumes) can suggest relative changes in populations in a raw form. While
other indicators (such as TripAdvisor reviews or Instagram posts) require
further data enrichment to get insights into population fluctuations. This
research forms part of a larger research project looking to test and apply
such novel indicators to inform disaster risk assessments.

Keywords: Population movement, tourism, transient populations, volunteered
geographic information, New Zealand, Stewart Island.

##  1 Introduction

Globally inbound tourism markets are volatile; significant changes can be
sensitive to political unrest, natural disasters or other crises, leading to
reduced visitor arrivals and slow recovery. New Zealand has a relatively small
dispersed population with proportionally high tourist arrivals, and
understanding volatility is critical for infrastructure and natural hazard
management (Orchiston, 2012). The vulnerability of tourism destinations to
natural disasters is high, as it “relies so heavily on perceptions of safety,
functioning infrastructure and visitor mobility” (p. 59). Moreover, due to the
possibility of a negative feedback cycle developing (i.e., following a natural
disaster a tourist will not visit a specific country or region (Orchiston et
al., 2013)) it is important that dynamic nature of transient populations is
better understood.

In this study we consider the use of volunteered geographic information (VGI)
and novel population movement indicators as predictors of tourist travel
patterns to support disaster risk reduction. The potential for VGI and
crowdsourced datasets as sources of information to support disaster response
is well-studied, yet its quality and usefulness remain debatable for many
scenarios (Goodchild and Glennon, 2010; Goodchild and Li, 2012; Senaratne et
al., 2017). In prior work, a common motivation for using VGI and novel data
sources for population mapping has been to understand risk exposure to
infectious disease in places without accurate population figures (Deville et
al., 2014; Tatem et al., 2012). In our case we consider a different situation
where the population numbers are influenced by a large number of transient
tourists and thus are highly dynamic.

A case study area, of Rakiura, Stewart Island, New Zealand’s southern-most
inhabited island has been selected. Here, confidence in the actual population
movement can be derived from passenger manifests on the ferry and air
services, visitor levy data collected by the local council, and international
cruise ship positioning data (automatic identification system). We compare
this population model to VGI and novel indicators on two temporal scales.
These methods are tested to see whether such indicators can be adopted as a
proxy for population fluxes, and in turn be used to better inform disaster
risk assessments.

Figure 1: Case Study Area of Stewart Island Rakiura, and location of novel/VGI
indicators

In summary, the aims of this study are two-fold. Firstly, it is to understand
how well VGI and novel indicators represent fluctuations in population.
Secondly, it considers opportunities to undertaking data enrichment methods to
build confidence in the assessed indicator.

##  2 Methodology

###  2.1 Modelling the population of Rakiura, Stewart Island

Similar to wider New Zealand, Stewart Island (or Rakiura in Māori), has
experienced significant growth in tourism in recent years. International and
domestic tourists alike enjoy visiting the island’s wilderness and natural
areas. As such, the population (378, as of 2013) can increase three-fold with
a visiting international cruise ship, or during a busy public holiday period.
Fluctuations in the island’s populations are largely driven by seasonality,
with the busiest periods being between September and March, coinciding with
more favourable weather, a number of public holidays, and the cruise ‘season’.

####  2.1.1 Building a population model for 2018–2019

To access Rakiura, visitors need to cross Foveaux Strait using either a
scheduled ferry or air service, cruise ship or by a charter aircraft, or other
vessel. By drawing on the data sources shown in Table 1 below, which represent
the majority (in excess of 90%) of passenger movements to the island, a
population model for the Island was created. This is for the period from 1
April 2018 until 31 March 2019.

Data Source | Source | Temporal Resolution | Coverage
---|---|---|---
Rakiura Visitor Levy | Southland District Council | Monthly | Visitors arrival only
Ferry Manifest | Real Journeys | Daily | Resident and Visitors
Cruise Ship Movements | Maritime NZ | Hourly | Vessels only
Population Forecasts | Statistics NZ | - | Residents
Table 1: Population inputs to Rakiura Population Model.

In order to build a population model for Rakiura, the net remaining population
has been calculated from the Ferry manifested passengers, and assumed 40% of
cruise ship capacity. The later equates to approximately 3,000 cruise ship
passengers visiting the Island, which is consistent with historical trends
Statistics New Zealand (2017). To allow for those arriving by scheduled
aircraft or chartered service (where no data is held), the net has been pro-
rated to match the actual, monthly visitor levy data.

####  2.1.2 Longitudinal temporal analysis for 2016–2019

A levy from every visitor to Rakiura of $5 is collected by the aircraft or
vessel operator on behalf of the local district council (Southland District
Council, SDC). This levy is collected for every domestic and international
visitor, with the exception of local ratepayers (including holiday home
owners), residents, or visitors who stay for longer than 21 days. SDC has
provided this for the period of 2016–2019 on a monthly (aggregated)
resolution.

A number of the VGI and novel indicators reviewed are also aggregated on a
monthly basis. Here, we have compared these to the longitudinal data that has
been drawn from the Rakiura Visitor Levy, as a proxy for island population.
Given the relative stability of the island resident population, this is
assumed to be a suitable proxy for changes in island population.

###  2.2 Novel indicators and Volunteered Geographic Information

The concept of using VGI or novel indicators to understand population
fluctuation has been considered by many authors (Dobson et al. (2000); Nissen
and Potter (2011); Statistics New Zealand (2011)), however few have had the
opportunity to assess in real terms the accuracy or the inherent bias of these
methods. Through this study we investigate a range of indicators to assess
their suitability to represent wider fluctuations in populations.

Table 2 below shows the indicators we assessed. Two indicators (wastewater and
Instagram posts) were assessed on a more fine-grained temporal resolution
(daily). Where aggregated datasets were reviewed (i.e., walking track counters
and TripAdvisor), they were assessed against the Visitor Levy to understand
longer term stability in trends. The assessment of all of these indicators has
been undertaken using the raw datasets and no data enrichment has been
undertaken.

Indicator | Aggregated | Type | n
---|---|---|---
Wastewater volume at Treatment Plant | Daily | Novel Indicators | 365 days
Instagram #StewartIsland uploads | Daily | VGI | 4,027 posts
Walking Track Counter | Monthly | Novel Indicators | 36,590 counts
TripAdvisor Reviews | Monthly | VGI | 194 reviews
Table 2: Data sources assessed in this review

##  3 Results

Presented in Figure 2 below are the key correlations between the daily
modelled population for Rakiura and the two daily indicators considered. This
clearly shows a strong relationship between daily wastewater volumes and the
daily population (Pearson’s Correlation of 0.973). This is consistent with
existing empirical relationships developed between population and wastewater
produced. The strong relationship is likely a function of the lack of
significant industry (and industrial effluent) on Rakiura, and the relative
simplicity of the network with limited opportunities for water ingress into
the network.

The relationship between the population model and #StewartIsland tagged posts
on Social Media platform Instagram is lower (Figure 2). There are a number of
complicating factors in this dataset which include: the ability for a user to
post multiple posts with the same ‘tags’, the inability to distinguish
individual users to identify ‘individual visits’, likely demographic bias
(particular in nations where the platform is unavailable), and the volunteered
nature of the geographic data (e.g., some users may confuse Rakiura with the
South Island of New Zealand). Noting this, there appears to be moderate
correlation (0.719) between the modelled population and Instagram posts, and
as such opportunities for data enrichment ought to be considered for this
dataset.

Figure 2: Daily data showing correlations between modelled population,
wastewater and Instagram posts between 1 April 2018–31 March 2019

Figure 3 adopts the Southland District Council’s visitor levy, as a proxy for
population movement. Here on a longer temporal scale (2016–2019) we compare
the TripAdvisor reviews for the local ‘South Seas Hotel’ pub and the
Department of Conservation (DOC) visitor counter on a local 1hr walking track.
In the case of the latter a strong correlation (0.951) is observed between the
proxy for the population on Rakiura, and the number of counts on the walking
track. This correlation supports the theory, that a large portion of visitors
are engaging in nature-based tourism activities.

TripAdvisor reviews also have a moderate correlation (0.763) to the proxy for
population movement. Here it is expected that this will be weighted towards
visitors as opposed to holiday home owners or residents. In addition, a
visitor is only likely to post once about a particular hotel, activity or
restaurant. Further multivariate analysis into understanding particular
demographic bias is needed to understand the value of this VGI in population
modelling.

Figure 3: Aggregated monthly data sets showing correlation between measured
visitors (Visitor Levy), TripAdvisor reviews of the local restaurant and a
visitor counter on a local walking track (between January 2016 and March 2019)

##  4 Next steps in applying volunteered geographic information to predicting
populations movements

The research aim of this project was to understand how novel flux indicators
can better predict population movement and thus exposure of international and
domestic tourists to disaster risk. Research undertaken to date shows that the
novel indicators, including VGI have the potential to improve population
exposure data informing such risk models.

Moving forward, an assessment of whether data enrichment meaningfully improves
the predictive powers of VGI social media data sources is needed. Noting the
findings of Middleton et al. (2013) and the limited metadata available in VGI
social media data sources, it is proposed that computer vision will be used to
ascertain actual geolocation of publicly shared imagery. Preliminary models
built in Google’s TensorFlow are being tested. Options for data enrichment,
including understanding effects of seasonality are being considered.

Finally, the goal in using the case study of Rakiura, Stewart Island was
because it provides an opportunity to compare a number of different kinds of
data against “ground truth” population numbers. Extending the study to other
locations that elicit different kinds of tourism behavior will help establish
the generalizability of the findings.

##  5 Acknowledgements

The authors would like to recognise the support of Land Information New
Zealand, Resilience to Nature’s Challenges - National Science Challenge,
QuakeCoRE, and the New Zealand Police Credit Union for their support in this
research. We would also like to thank the data providers who have provided
data to undertake this research: Wayfare Group Limited, Southland District
Council, Maritime New Zealand and the Department of Conservation.

##  6 References

## References

  * Deville et al. (2014) Deville, P., C. Linard, S. Martin, M. Gilbert, F. R. Stevens, A. E. Gaughan, V. D. Blondel, and A. J. Tatem
2014\.  Dynamic population mapping using mobile phone data.  Proceedings of
the National Academy of Sciences, 111(45):15888–15893.

  * Dobson et al. (2000) Dobson, J. E., E. A. Bright, P. R. Coleman, R. C. Durfee, and B. A. Worley
2000\.  Landscan: a global population database for estimating populations at
risk.  Photogrammetric engineering and remote sensing, 66(7):849–857.

  * Goodchild and Glennon (2010) Goodchild, M. F. and J. A. Glennon
2010\.  Crowdsourcing geographic information for disaster response: a research
frontier.  International Journal of Digital Earth, 3(3):231–241.

  * Goodchild and Li (2012) Goodchild, M. F. and L. Li
2012\.  Assuring the quality of volunteered geographic information.  Spatial
statistics, 1:110–120.

  * Middleton et al. (2013) Middleton, S. E., L. Middleton, and S. Modafferi
2013\.  Real-time crisis mapping of natural disasters using social media.
IEEE Intelligent Systems, 29(2):9–17.

  * Nissen and Potter (2011) Nissen, K. and D. Potter
2011\.  Where did people relocate to? experimental cell phone data analysis of
population movements following the 22nd february christchurch earthquake.  In
PANZ biennial conference New Zealands Demographic Futures: Where to from Here.

  * Orchiston (2012) Orchiston, C.
2012\.  Seismic risk scenario planning and sustainable tourism management:
Christchurch and the alpine fault zone, south island, new zealand.  Journal of
Sustainable Tourism, 20(1):59–79.

  * Orchiston et al. (2013) Orchiston, C., J. Vargo, and E. Seville
2013\.  Insurance in the tourism sector after the canterbury earthquakes.
Journal of the Australian and New Zealand Institute of Insurance and Finance,
36(4):20–23.

  * Senaratne et al. (2017) Senaratne, H., A. Mobasheri, A. L. Ali, C. Capineri, and M. Haklay
2017\.  A review of volunteered geographic information quality assessment
methods.  International Journal of Geographical Information Science,
31(1):139–167.

  * Statistics New Zealand (2011) Statistics New Zealand
2011\.  Evaluation of alternative data sources for population estimates.
http://archive.stats.govt.nz/browse_for_stats/population/estimates_and_projections/eval-
alt-data-src-pop-estimates.aspx.  Accessed: 2015-06-14.

  * Statistics New Zealand (2017) Statistics New Zealand
2017\.  Cruise ship traveller statistics: June 2017.
http://archive.stats.govt.nz/browse_for_stats/industry_sectors/Tourism/cruise-
ship-stats.aspx.  Accessed: 2015-06-14.

  * Tatem et al. (2012) Tatem, A. J., S. Adamo, N. Bharti, C. R. Burgert, M. Castro, A. Dorelien, G. Fink, C. Linard, M. John, L. Montana, et al.
2012\.  Mapping populations at risk: improving spatial demographic data for
infectious disease modeling and metric derivation.  Population health metrics,
10(1):8.