# Neuroevolution deep learning architecture search for estimation of river
surface elevation from photogrammetric Digital Surface Models

Radosław Szostak
AGH UST
<EMAIL_ADDRESS>
Marcin Pietroń
AGH UST
<EMAIL_ADDRESS>
Mirosław Zimnoch
AGH UST
<EMAIL_ADDRESS>
Przemysław Wachniew
AGH UST
<EMAIL_ADDRESS>
Paweł Ćwiąkała
AGH UST
<EMAIL_ADDRESS>
Edyta Puniach
AGH UST
<EMAIL_ADDRESS>

###### Abstract

Development of the new methods of surface water observation is crucial in the
perspective of increasingly frequent extreme hydrological events related to
global warming and increasing demand for water. Orthophotos and digital
surface models (DSMs) obtained using UAV photogrammetry can be used to
determine the Water Surface Elevation (WSE) of a river. However, this task is
difficult due to disturbances of the water surface on DSMs caused by
limitations of photogrammetric algorithms. In this study, machine learning was
used to extract a WSE value from disturbed photogrammetric data. A brand new
dataset has been prepared specifically for this purpose by hydrology and
photogrammetry experts. The new method is an important step toward automating
water surface level measurements with high spatial and temporal resolution.
Such data can be used to validate and calibrate of hydrological, hydraulic and
hydrodynamic models making hydrological forecasts more accurate, in particular
predicting extreme and dangerous events such as floods or droughts. For our
knowledge this is the first approach in which dataset was created for this
purpose and deep learning models were used for this task. Additionally,
neuroevolution algorithm was set to explore different architectures to find
local optimal models and non-gradient search was performed to fine-tune the
model parameters. The achieved results have better accuracy compared to manual
methods of determining WSE from photogrammetric DSMs.

_Keywords_ deep learning $\cdot$ geoscience $\cdot$ neuroevolution $\cdot$
climate change

##  1 Introduction

Reports from international organizations indicate increasingly significant
problems with earths water resources. The global demand for freshwater
continues to increase at rate 1% per year since 1980s driven by population
growth and socioeconomic changes. Simultaneously, the increase in evaporation
caused increasing temperatures leads to a decrease in streamflow volumes in
many areas of the world, which already suffer from water scarcity problems.
Climate warming is also responsible for globally increased frequency of
extreme hydrologic conditions. More intense and frequent precipitation events
increase the flood risk as well as heatwaves are becoming more common and last
longer, resulting in more severe droughts (UNESCO (2020), IPCC (2015)).
Achieving socioeconomic and environmental sustainability under such
challenging conditions will require the use of monitoring tools that will
facilitate the management of the water resources. Traditional surface water
management practices are primarily based on data collected from networks of in
situ hydrometric gauges. Point measurements do not provide sufficient spatial
resolution to comprehensively characterize river networks, and many developing
regions lack them altogether. Moreover, the decline of existing measurement
networks is being observed all over the world (Lawford et al. (2013)). Remote
sensing methods are considered as a solution to cover data gaps specific to
point measurement networks (McCabe et al. (2017)). A leading example of remote
sensing is measurements made from satellites. However, due to too low spatial
resolution, satellite data is suitable only for studying the largest rivers.
E.g. SWOT mission allows only observation of rivers of width greater than
50-100 m (Pavelsky et al. (2014)). Small surface streams of the first and
second order (according to Strahler’s classification (Strahler (1957))
constitute 70%-80% of the length of all rivers in the world. Small streams
play a significant role in hydrologic systems and provide an ecosystem for
living organisms (Wohl (2017)). Despite their high importance, modern methods
for their observation are still lacking. In this regard, measurement
techniques based on Unmanned Aerial Systems (UASs) are promising in many key
aspects, as they are characterized by high spatial and temporal resolution,
simple and fast deployment, and the ability to be used in inaccessible
locations (Vélez-Nicolás et al. (2021)). Spatialy distributed Water Surface
Elevation (WSE) measurements are highly important, as they are used for
validation and calibration of hydrologic, hydraulic or hydrodynamic models to
make hydrological forecasts, including predicting dangerous events such as
floods and droughts (Langhammer et al. (2017), Tarpanelli et al. (2013),
Jarihani et al. (2013), Domeneghetti (2016), Montesarchio et al. (2014))

Photogrammetric Structure from Motion (SfM) algorithms are able to generate
Orthophotos and Digital Surface Models (DSMs) of terrain based on multiple
aerial photographs. Photogrammetric DSMs are precise in determining the
elevation of solid surfaces to within a few cm (Ouédraogo et al. (2014),
Bühler et al. (2017)). However, they do not correctly represent the water
surface. This is related to the fact that general principle of SfM algorithms
is based on automatic search for a distinguishable and static terrain points
that appear in several images showing these points from different
perspectives. The surface of the water lacks such points as it is uniform,
transparent and in motion. Due to water transparency, DSMs created using SfM
algorithms typically indicate pixel elevations below the actual water surface
level. For very clear and shallow streams, photogrammetric DSMs represent the
river bottom (Kasvi et al. (2019)). For opaque waters, photogrammetric DSMs
are disturbed by artifacts caused by water uniformity (lack of distinguishable
photogrammetric key-points). Woodget et al. (2014), Javernick et al. (2014)
and Pai et al. (2017) demonstrated that it is possible to read the WSE from
photogrammetric DSM at shorelines ("water-edge") where river is very shallow,
so there are no undesirable effects associated with light penetration below
the water surface. However, Bandini et al. (2020) proved that this method
gives satisfactory results only for unvegetated and smoothly sloping
shorelines where the boundary line between water and land is easy to define.
For this reason, this method is not suitable for universal automation. Table 1
shows the RMSE errors of existing Remote Sensing methods for small rivers WSE
measurements.

Table 1: Remote sensing small river WSE measurement error comparison. RMSE values taken from: UAV – Bandini et al. (2020), AIRSWOT – Altenau et al. (2017) Method | RMSE (m)
---|---
UAV RADAR | 0.03
AIRSWOT | 0.09
UAV SfM DSM centerline | 0.164
UAV SfM point cloud | 0.180
UAV LIDAR point cloud | 0.22
UAV SfM DSM "water-edge" | 0.450

The aim of this work was to develop a new automatic method based on deep
neural networks allowing estimation of small rivers WSE from photogrammetric
DSMs and Orthophotos with an accuracy outperforming previous methods based on
manual analysis of photogrammetric data.

##  2 Dataset

###  2.1 Dataset structure

In this research, models were trained using brand new dataset, created
specifically for the purpose of this work. It consist of 260 samples. Each
sample corresponds to a 10 by 10 meter area that encloses river water and
nearshore land. Dataset is divided into a training and testing subset at a
ratio of 8:2. Dataset is available to download at
https://zenodo.org/record/5257183 (Szostak et al. (2021)). Every sample
includes the data detailed below.

  * •

Photogrammetric orthophoto. True color image represented as a $3\times
256\times 256$ array (3 channel image of $256\times 256$ pixels).

  * •

Photogrammetric DSM. Corresponds to the area presented on the orthophoto.
Contains disturbed water surfaces elevations. Stored as a $256\times 256$
array containing elevations of pixels expressed in m MSL.

  * •

WSE. Ground truth Water Surface Elevation as single value expressed in m MSL.

(a)

(b)

Figure 1: Visualisation of geospatial data from single dataset sample. (a) –
photogrammetric orthophoto, (b) – raw photogrammetric DSM with water surface
disturbances.

###  2.2 Dataset preparation

The photogrammetric data and WSE observations needed to create the dataset was
obtained during surveys in two study areas:

  1. 1.

in a ca. 700m stretch of the Kocinka river located near Grodzisko village
(Silesian Voivodeship, Poland) on December 19, 2020 and July 13, 2021. This
stretch had a WS width of ca. 2m and was overhung by sparse deciduous trees
that had no leaves due to the winter season. The banks as well as the riverbed
were overgrown with rushes that protruded above the water surface. The banks
are steeply sloping at angles of ca. 50° to 90° relative to the water surface.
There are marshes nearby, with river water flowing into them in places.

  2. 2.

in a ca. 700m stretch of the Kocinka river located near Mykanów village
(Silesian Voivodeship, Poland) on December 19, 2020 and July 13, 2021. This
stretch had a WS width of ca. 3m and was overhung by sparse deciduous trees.
Grasses growing on the banks slightly overhang the water surface. The banks
are steeply sloping at angles of ca. 50° to 90° relative to the water surface.

In addition, the dataset was supplemented with data from surveys made by
Bandini et al. (2019) over a ca. 2.3km stretch of the river Åmose Å (Denmark)
on November 21, 2018. See the cited publication for details on this river case
study: Bandini et al. (2020).

During survey campaigns, photogrammetric measurement were conducted over the
river area. The aerial photos were taken from a DJI S900 UAV using a Sony ILCE
a6000 camera with a Voigtlander SUPER WIDE HELIAR VM 15 mm f/4.5 lens. The
flight altitude was approximately 77m AGL, resulting in a 20mm terrain pixel.
Photos front overlap was 80%, and side overlap was 60%. Alongside the drone
measurements, measurements of ground cotrol points were made using RTK GPS
receiver. These were used to embed the photogrammetric rasters in the
geographic reference system. RTK GPS receivers were also used for reference
point measurements of WSE. They were made along the river every approximately
10-20 meters at both banks. Raster files of orthophotos and digital surface
models of the investigated areas were generated using Agisoft Metashape
photogrammetric software.

The measurement campaigns resulted in raster files with width and height of
several tens of thousands of pixels, each representing an area of about 40ha.
For use in machine learning algorithms, they were divided into 256px x 256px
images representing sub-areas of actual size in the field of 10m x 10m. Point
ground truth measurements of river water surrface elevation were interpolated
using the IDW method. Interpolated values were assigned to individual samples
based on geospatial location.

##  3 Deep Learning solution

###  3.1 Feature scaling

Input data is subjected to feature scaling before it is fed into the model.
DSMs values were standardized according to the equation
$DSM^{\prime}=\frac{DSM-\overline{DSM}}{2\sigma}$, where $DSM^{\prime}$ –
standardized sample DSM 2D array with values centered around 0, $DSM$ – raw
sample DSM 2D array with values expressed in m MSL, $\overline{DSM}$ – mean
value of single subjected $DSM$ array, $\sigma$ – standard deviation of DSM
array pixel values from the entire dataset. This method of standardization has
two clear advantages. Firstly, by subtracting the average value of a single
subjected sample, standardized DSMs are always centered around zero, so the
algorithm sees no difference between samples of the rivers located at regions
of different altitudes. The actual water level information is recovered during
inverse standardization. Secondly, dividing all samples by the same sigma
value of entire dataset, ensures that all standardized samples are scaled
equally. It was experimentally found that multiplying the denominator by 2
results in better model accuracy, compared to standardization that does not
include this factor. Orthophotos were standardized using Imagenet (Deng et al.
(2009)) dataset mean and standard deviation according to the equation
$ORT^{\prime}=\frac{ORT-\mu}{\sigma}$, where $ORT^{\prime}$ – standardized
3-channel orthophoto RGB image 3D array with values centered around 0, $ORT$ –
3-channel orthophoto RGB image 3D array represented with values from the range
[0,1], $\mu=[0.485,0.456,0.406]$ – 1D vector containing mean values of each of
RGB channels from Imagenet dataset, $\sigma=[0.229,0.224,0.225]$ – 1D vector
containing standard deviation values of each of RGB channels from Imagenet
dataset.

###  3.2 Models

The model used to create the supervised learning algorithm for determining a
single WSE value is based on the VGG-16 architecture (Simonyan and Zisserman
(2015)). Several variations of this model have been tested:

  1. a

VGG-16 Base Model. VGG-16 originally used for image classification was
modified to perform single floating point value prediction. The changes made
to this model are: i) the input size of the model is 4x256x256. It is a four-
channel image in which the first channel contains the DSM and the other three
channels are RGB orthophoto channels. ii) After a series of convolution
layers, a linear transformation of the array data to a single value was
applied. No activation function was used on the model output to obtain a
floating point value.

  2. b

Multiresolution VGG-16. VGG-16 Base Model (a) enhanced with multi resolution
achieved by concatenation of scaled four-input channels to the output of each
max pooling layer.

  3. c

VGG-16 with four conv blocks \- VGG-16 without last three conv layers with
changed activation function to PReLU.

  4. d

VGG-16 with three conv blocks \- VGG-16 without last six conv layers with
ReLU6.

  5. e

VGG-16 with five blocks \- VGG-16 with whole feature extractor with number of
channels 256 changed to 148).

  6. f

fine tuned VGG-16 \- best pretrained VGG-16 fine tuned by running
neuroevolution with weights mutation.

  7. g

ResNeST50 \- Resnet with split attention model

  8. h

VisionTransformer \- Vision Transformer model with different number of heads
and depth parameter

  9. i

EfficientNet \- b0 and b1 EfficientNet architectures

###  3.3 Architecture search

Many of recent machine learning works has focused on solutions in which neural
network weights are trained through variants of stochastic gradient descent.
An alternative approach comes from the field of neuroevolution, which
harnesses evolutionary algorithms to optimize neural networks, inspired by the
fact that natural brains themselves are the products of an evolutionary
process (Faber et al. (2021), Ma et al. (February 2021), Stanley and
Miikkulainen (2002), Miikkulainen et al. (Mar 2017), Sun et al. (Oct 2017), E.
Galvan (Jun 2020)). In presented work we have set up a neuroevolution based
algorithm which can run search through different architectures and
modifications of few baseline models VGG, ResNet, ResneST, EfficientNet,
VisionTransformer (1). The size of population in our experiments was 16 (eq.1,
eq. 2). The number of iterations is in range from 20 to 40. The population is
set of the models with different initial random weights ((eq.1, eq. 2)).
Equation 1 describes the definition of the model population where $\theta$ is
a weight tensor.

$P_{M}=\\{F_{\Theta}^{i},\Theta=\\{\theta_{0},\theta_{1},...,\theta_{N}\\}\land
i\in\\{1,2,...,population\\_size\\}\\}$ (1)

$F_{\Theta}^{i}(X)={f_{\theta_{N}}^{i}(f_{\theta_{N-1}}^{i}...(f_{\theta_{0}}^{i}(X)))}$
(2)

We define the crossover as a function that has two parent models and an id of
the layer as input parameters (Equation ( 3), alg.1, line 6). It generates two
new children models (Equation (4, alg. 2, line 11)).

$c:Parents\times layer\\_id\rightarrow Children$ (3)

$c(F_{\Theta_{M}}^{i},F_{\Theta_{N}}^{j},l\\_id)\rightarrow(F_{\Theta_{M^{\prime}}}^{i^{\prime}},F_{\Theta_{N^{\prime}}}^{j^{\prime}})$
(4)

$f_{\theta_{l\\_id}}^{i^{\prime}}=f_{\theta_{l\\_id}}^{j}$ (5)

$f_{\theta_{l\\_id}}^{j^{\prime}}=f_{\theta_{l\\_id}}^{i}$ (6)

The crossover can exchange between parents activation functions,
convolutional, or fully connected layers (eq.5, eq.6, alg.2, line 3 and line
6). The last option is to exchange hyperparameters between parents, such as
learning rate and weight decay (alg.2, line 9). We define mutation as a
function with a model, layer id, and layer features to be mutated as input
parameters (Equation 7, alg.3). In this case, the feature and $layer\\_id$
determine what the mutation operator will do. It can change the activation
function or parameters of convolutional or fully connected layers (depth) (3,
line 3 and line 6). The other option is to change the length of the model
(adding or removing a layer) or update the hyperparameters. The result is a
new modified model (Equation (9)).

$m:Parent\times layer\\_id\rightarrow Child$ (7)

$m(F_{\Theta_{M}}^{i},l\\_id)\rightarrow F_{\Theta_{N}}^{i^{\prime}}$ (8)

$f_{\theta_{l\\_id}}^{i^{\prime}}=depth(f_{\theta_{l\\_id}}^{i})\pm\beta\cdot\Delta_{depth}$
(9)

Finally, evolution process is finished, non-gradient fine tuning is invoked.
Fine tuning is performed as weight mutation on a pretrained model (eq.10,
eq.11, alg.4). The additional input parameter for the non gradient fine tuning
is the scaling factor for computing the value by which the weights can be
modified (eq.12). The constant percentage of weights in each weight tensor are
selected for fine tuning process. The selection process is done by binary
$\alpha$ tensor. The $\alpha$ tensor is dynamic and can be different in each
fine tuning iteration.

$m:Parent\times scale\rightarrow Child$ (10)

$F_{\Theta^{\prime}}^{i^{\prime}}=m(F_{\Theta}^{i}),\Theta^{\prime}=\\{\theta^{\prime}_{0},\theta^{\prime}_{1},...,\theta^{\prime}_{N}\\}$
(11)

$w^{\prime}_{ij}=w_{ij}\pm\alpha_{ij}\cdot\frac{max(abs(\theta_{i}))}{scale_{i}}$
(12)

0: population_size, base_model

1: population = generate_new_population(base_model)

2: for iteration in iterations do

3: for solution in population do

4: p1, p2 = get_random_parents(population)

5: layer_id = generate_layer_id()

6: p1, p2 = crossover(p1, p2, layer_id)

7: population = add_to_population(p1, p2)

8: new_solution = mutation(solution)

9: population = add_to_population(new_solution)

10: end for

11: population = choose_best_models(population, population_size)

12: end for

Algorithm 1 Neuroevolution approach

0: parent_1, parent_2, layer_id

1: layer = get_layer(parent_1, layer_id)

2: if layer is activation then

3: child_1, child_2 = exchange_activation(parent_1, parent_2)

4: end if

5: if layer is (conv or fc) then

6: child_1, child_2 = exchange_layer(parent_1, parent_2, layer_id)

7: end if

8: if layer is None then

9: child_1, child_2 = exchange_hyperparameters(parent_1, parent_2)

10: end if

11: Return_children

Algorithm 2 Crossover operator

0: model, layer_id

1: layer = get_layer(model, layer_id)

2: if layer is activation then

3: model = change_activation(model, layer_id)

4: end if

5: if layer is layer then

6: model = change_layer(model, layer_id)

7: end if

8: if layer_id is None then

9: model = change_optimizer(model)

10: end if

11: Return_model

Algorithm 3 Mutation operator

0: $N_{g}$ \- Number of generations in the fine tuning, $percentage$ \-
percentage of weights to be perturbated, $scale$ \- scaling factor for
magnitude perturbation

1: $generation=0$

2: while $generation<N_{g}$ do

3: Mutate_randomly_percentage_of_weights

4: For_every_solution_in_the_population calculate_the_fitness

5: Choose_the_best_solutions_and_include_them in_the_new_population

6: $generation\leftarrow generation+1$

7: end while

8: Return_final_population_of_fine_tuned_models

Algorithm 4 Fine tuning

##  4 Results and future works

Table 2: Results on 260 images. Model | water level estimation error
---|---
vgg 4 blocks with LeakyReLU (84,137,258,497) | 11.18
vgg 3 blocks with ReLU | 10.44
vgg 5 blocks with LeakyReLU (66,146,176,222,222) | 10.01
fine tuned vgg 5 with LeakyReLU | 9.89
VGG-16 Base Model | 11.69
Resnet18 Model (42, 193, 293, 579) | 14.13
Multiresolution VGG-16 | 10.50
ResneST50 with PReLU (88,192,198,202) [3,5,5,4] | 8.58
EfficientNet-b0 | 16.12
EfficientNet-b1 | 13.75
VisionTransformer (depth=8, heads=11) | 9.99

The results are shown in Tab. 2. The models listed in table are those which
were set manually (VGG-16 Base Model, Multiresolution VGG-16 Base Model) and
other which were generated by neuroevolution search. In parentheses there are
combinations of the number of channels for successive blocks inside the
models. In the rows where these numbers are not given, the number of channels
in the blocks is the same as in the original version of the model. It is shown
that neuroevolution search improves accuracy of water level prediction (e.g.
ResneST with modified number of sub-blocks - [3,5,5,4], VisionTransformer
(depth=8, heads=11)). Our fine tuning approach decrease further the prediction
error in VGG model. It is worth to mention that our best deep learning models
outperform other manual methods of determining WSE from photogrammetric DSMs
and are close to the accuracy of the more complicated and expensive AirSWOT
method (Tab. 1). The future work will concentrate on further model exploration
using more sophisticated genetic operators which can modify number of sub-
blocks, run models clustering, hyperparameter optimization (Faber et al.
(2021)) and automatically apply multiresolution options. Also Bayesian
estimation of uncertainty and models sensitivity analysis will be performed
(Gal and Ghahramani (2016)).

## References

  * UNESCO [2020] UNESCO.  _The United Nations world water development report 2020: water and climate change_.  United Nations Educational, Scientific and Cultural Organization, Paris, 2020.  ISBN 978-92-3-100371-4.
  * IPCC [2015] IPCC.  _Climate change 2014 : synthesis report_.  Intergovernmental Panel on Climate Change, Geneva, Switzerland, 2015.  ISBN 978-92-9169-143-2.
  * Lawford et al. [2013] Richard Lawford, Adrian Strauch, David Toll, Balazs Fekete, and Douglas Cripe.  Earth observations for global water security.  _Current Opinion in Environmental Sustainability_ , 5(6):633–643, December 2013.  doi:10.1016/j.cosust.2013.11.009.  URL https://doi.org/10.1016/j.cosust.2013.11.009.
  * McCabe et al. [2017] Matthew F. McCabe, Matthew Rodell, Douglas E. Alsdorf, Diego G. Miralles, Remko Uijlenhoet, Wolfgang Wagner, Arko Lucieer, Rasmus Houborg, Niko E. C. Verhoest, Trenton E. Franz, Jiancheng Shi, Huilin Gao, and Eric F. Wood.  The future of earth observation in hydrology.  _Hydrology and Earth System Sciences_ , 21(7):3879–3914, July 2017.  doi:10.5194/hess-21-3879-2017.  URL https://doi.org/10.5194/hess-21-3879-2017.
  * Pavelsky et al. [2014] Tamlin M. Pavelsky, Michael T. Durand, Konstantinos M. Andreadis, R. Edward Beighley, Rodrigo C.D. Paiva, George H. Allen, and Zachary F. Miller.  Assessing the potential global extent of SWOT river discharge observations.  _Journal of Hydrology_ , 519:1516–1525, November 2014.  doi:10.1016/j.jhydrol.2014.08.044.  URL https://doi.org/10.1016/j.jhydrol.2014.08.044.
  * Strahler [1957] Arthur N. Strahler.  Quantitative analysis of watershed geomorphology.  _Transactions, American Geophysical Union_ , 38(6):913, 1957.  doi:10.1029/tr038i006p00913.  URL https://doi.org/10.1029/tr038i006p00913.
  * Wohl [2017] Ellen Wohl.  The significance of small streams.  _Frontiers of Earth Science_ , 11(3):447–456, April 2017.  doi:10.1007/s11707-017-0647-y.  URL https://doi.org/10.1007/s11707-017-0647-y.
  * Vélez-Nicolás et al. [2021] Mercedes Vélez-Nicolás, Santiago García-López, Luis Barbero, Verónica Ruiz-Ortiz, and Ángel Sánchez-Bellón.  Applications of unmanned aerial systems (UASs) in hydrology: A review.  _Remote Sensing_ , 13(7):1359, April 2021.  doi:10.3390/rs13071359.  URL https://doi.org/10.3390/rs13071359.
  * Langhammer et al. [2017] Jakub Langhammer, Jana Bernsteinová, and Jakub Miřijovský.  Building a high-precision 2d hydrodynamic flood model using UAV photogrammetry and sensor network monitoring.  _Water_ , 9(11):861, November 2017.  doi:10.3390/w9110861.  URL https://doi.org/10.3390/w9110861.
  * Tarpanelli et al. [2013] Angelica Tarpanelli, Silvia Barbetta, Luca Brocca, and Tommaso Moramarco.  River discharge estimation by using altimetry data and simplified flood routing modeling.  _Remote Sensing_ , 5(9):4145–4162, August 2013\.  doi:10.3390/rs5094145.  URL https://doi.org/10.3390/rs5094145.
  * Jarihani et al. [2013] Abdollah Asadzadeh Jarihani, John Nikolaus Callow, Kasper Johansen, and Ben Gouweleeuw.  Evaluation of multiple satellite altimetry data for studying inland water bodies and river floods.  _Journal of Hydrology_ , 505:78–90, November 2013.  doi:10.1016/j.jhydrol.2013.09.010.  URL https://doi.org/10.1016/j.jhydrol.2013.09.010.
  * Domeneghetti [2016] Alessio Domeneghetti.  On the use of SRTM and altimetry data for flood modeling in data-sparse regions.  _Water Resources Research_ , 52(4):2901–2918, April 2016.  doi:10.1002/2015wr017967.  URL https://doi.org/10.1002/2015wr017967.
  * Montesarchio et al. [2014] V. Montesarchio, F. Napolitano, M. Rianna, E. Ridolfi, F. Russo, and S. Sebastianelli.  Comparison of methodologies for flood rainfall thresholds estimation.  _Natural Hazards_ , 75(1):909–934, August 2014\.  doi:10.1007/s11069-014-1357-3.  URL https://doi.org/10.1007/s11069-014-1357-3.
  * Ouédraogo et al. [2014] Mohamar Moussa Ouédraogo, Aurore Degré, Charles Debouche, and Jonathan Lisein.  The evaluation of unmanned aerial system-based photogrammetry and terrestrial laser scanning to generate DEMs of agricultural watersheds.  _Geomorphology_ , 214:339–355, June 2014.  doi:10.1016/j.geomorph.2014.02.016.  URL https://doi.org/10.1016/j.geomorph.2014.02.016.
  * Bühler et al. [2017] Yves Bühler, Marc S. Adams, Andreas Stoffel, and Ruedi Boesch.  Photogrammetric reconstruction of homogenous snow surfaces in alpine terrain applying near-infrared UAS imagery.  _International Journal of Remote Sensing_ , 38(8-10):3135–3158, January 2017.  doi:10.1080/01431161.2016.1275060.  URL https://doi.org/10.1080/01431161.2016.1275060.
  * Kasvi et al. [2019] E. Kasvi, J. Salmela, E. Lotsari, T. Kumpula, and S.N. Lane.  Comparison of remote sensing based approaches for mapping bathymetry of shallow, clear water rivers.  _Geomorphology_ , 333:180–197, May 2019.  doi:10.1016/j.geomorph.2019.02.017.  URL https://doi.org/10.1016/j.geomorph.2019.02.017.
  * Woodget et al. [2014] A. S. Woodget, P. E. Carbonneau, F. Visser, and I. P. Maddock.  Quantifying submerged fluvial topography using hyperspatial resolution UAS imagery and structure from motion photogrammetry.  _Earth Surface Processes and Landforms_ , 40(1):47–64, August 2014.  doi:10.1002/esp.3613.  URL https://doi.org/10.1002/esp.3613.
  * Javernick et al. [2014] L. Javernick, J. Brasington, and B. Caruso.  Modeling the topography of shallow braided rivers using structure-from-motion photogrammetry.  _Geomorphology_ , 213:166–182, May 2014.  doi:10.1016/j.geomorph.2014.01.006.  URL https://doi.org/10.1016/j.geomorph.2014.01.006.
  * Pai et al. [2017] H. Pai, H. F. Malenda, M. A. Briggs, K. Singha, R. González-Pinzón, M. N. Gooseff, and S. W. Tyler and.  Potential for small unmanned aircraft systems applications for identifying groundwater-surface water exchange in a meandering river reach.  _Geophysical Research Letters_ , 44(23), December 2017.  doi:10.1002/2017gl075836.  URL https://doi.org/10.1002/2017gl075836.
  * Bandini et al. [2020] Filippo Bandini, Tanya Pheiffer Sunding, Johannes Linde, Ole Smith, Inger Klint Jensen, Christian Josef Köppl, Michael Butts, and Peter Bauer-Gottwein.  Unmanned aerial system (UAS) observations of water surface elevation in a small stream: Comparison of radar altimetry, LIDAR and photogrammetry techniques.  _Remote Sensing of Environment_ , 237:111487, February 2020\.  doi:10.1016/j.rse.2019.111487.  URL https://doi.org/10.1016/j.rse.2019.111487.
  * Altenau et al. [2017] Elizabeth H. Altenau, Tamlin M. Pavelsky, Delwyn Moller, Christine Lion, Lincoln H. Pitcher, George H. Allen, Paul D. Bates, Stéphane Calmant, Michael Durand, and Laurence C. Smith.  AirSWOT measurements of river water surface elevation and slope: Tanana river, AK.  _Geophysical Research Letters_ , 44(1):181–189, January 2017.  doi:10.1002/2016gl071577.  URL https://doi.org/10.1002/2016gl071577.
  * Szostak et al. [2021] Radosław Szostak, Mirosław Zimnoch, Przemysław Wachniew, Paweł Ćwiąkała, and Marcin Pietroń.  Dsm water level: An uav photogrammetry dataset for determination of river surface level using machine learning, 2021.  URL https://zenodo.org/record/5257183.
  * Bandini et al. [2019] Filippo Bandini, Tanya Pheiffer Sunding, Johannes Linde, Ole Smith, Inger Klint Jensen, Christian Josef Köppl, Michael Butts, and Peter Bauer-Gottwein.  Dataset used in "unmanned aerial system (uas) observations of water surface elevation in a small stream: comparison of radar altimetry, lidar and photogrammetry techniques", 2019.  URL https://zenodo.org/record/3519887.
  * Deng et al. [2009] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  ImageNet: A large-scale hierarchical image database.  In _2009 IEEE Conference on Computer Vision and Pattern Recognition_. IEEE, June 2009.  doi:10.1109/cvpr.2009.5206848.  URL https://doi.org/10.1109/cvpr.2009.5206848.
  * Simonyan and Zisserman [2015] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition, 2015\.
  * Faber et al. [2021] Kamil Faber, Marcin Pietron, and Dominik Zurek.  Ensemble neuroevolution-based approach for multivariate time series anomaly detection.  _Entropy_ , 23(11), November 2021.  doi:https://doi.org/10.3390/e23111466.
  * Ma et al. [February 2021] A. Ma, Y. Wan, Y. Zhong, and J. Wang.  Scenenet: Remote sensing scene classification deep learning network using multi-objective neural evolution architecture search.  _ISPRS Journal of Photogrammetry and Remote Sensing pp. 171-188_ , DOI:10.1016/j.isprsjprs.2020.11.025, February 2021.
  * Stanley and Miikkulainen [2002] Kenneth O. Stanley and Risto Miikkulainen.  Evolving neural networks through augmenting topologies.  _Evolutionary Computation_ , 10(2):99–127, 2002\.  doi:10.1162/106365602320169811.
  * Miikkulainen et al. [Mar 2017] R. Miikkulainen, J. Liang, E. Meyerson, A. Rawal, D. Fink, O. Francon, B. Raju, H. Shahrzad, A. Navruzyan, N. Duffy, and B. Hodjat.  Evolving deep neural networks.  _CoRR abs/1703.00548_ , Mar 2017.
  * Sun et al. [Oct 2017] Yanan Sun, Bing Xue, Mengjie Zhang, and Gary G. Yen.  Evolving deep convolutional neural networks for image classification.  _CoRR abs/1710.10741_ , Oct 2017.
  * E. Galvan [Jun 2020] P. Mooney E. Galvan.  Neuroevolution in deep neural networks: Current trends and future challenges.  _CoRR abs/2006.05415_ , Jun 2020.
  * Gal and Ghahramani [2016] Yarin Gal and Zoubin Ghahramani.  Dropout as a bayesian approximation: Representing model uncertainty in deep learning, 2016.