#  Deep coastal sea elements forecasting using U-Net based models

Jesús García Fernández<EMAIL_ADDRESS>Ismail Alaoui Abdellaoui<EMAIL_ADDRESS>Siamak Mehrkanoon<EMAIL_ADDRESS>Department of
Data Science and Knowledge Engineering, Maastricht University, The Netherlands

###### Abstract

The supply and demand of energy is influenced by meteorological conditions.
The relevance of accurate weather forecasts increases as the demand for
renewable energy sources increases. The energy providers and policy makers
require weather information to make informed choices and establish optimal
plans according to the operational objectives. Due to the recent development
of deep learning techniques applied to satellite imagery, weather forecasting
that uses remote sensing data has also been the subject of major progress. The
present paper investigates multiple steps ahead frame prediction for coastal
sea elements in the Netherlands using U-Net based architectures. Hourly data
from the Copernicus observation programme spanned over a period of 2 years has
been used to train the models and make the forecasting, including seasonal
predictions. We propose a variation of the U-Net architecture and further
extend this novel model using residual connections, parallel convolutions and
asymmetric convolutions in order to introduce three additional architectures.
In particular, we show that the architecture equipped with parallel and
asymmetric convolutions as well as skip connections outperforms the other
three discussed models.

###### keywords:

Coastal sea elements , Time-series satellite data , Deep learning ,
Convolutional neural networks , U-Net

††journal: arXiv

##  1 Introduction

Renewable energy system has received increasing attention in the last years.
In this context, the ability to accurately predict weather elements is crucial
to the effective use of weather elements resources. In particular, it has been
shown that weather forecasting affects sectors like agriculture, forestry,
transportation and healthcare among others, thus having a major impact on the
global economy [1, 2, 3, 4, 5]. More importantly, weather prediction can be
used to save thousands of human lives by being able to forecast various types
of natural disasters like tornadoes and flash floods [6, 7].

Classical approaches to perform weather forecasting heavily relied on the
thermodynamics laws, Navier-Strokes equations, the statistical properties of
the data, as well as the various properties of the atmosphere [8, 9, 10, 11].
This set of methods belongs to the Numerical Weather Prediction (NWP)
approaches and generally require a large amount of compute resources since the
processing is done on supercomputers [12].

Furthermore, it has been shown that NWP based approaches might suffer from
computational instability, mainly due to the initial conditions of the models
[13]. Recent data-driven approaches on the other hand perform a simulation of
an entire system in order to predict its next state. The main methodology used
by these methods is the usage of historical data to perform the forecasting.
Based on the success provided by machine learning models (i.e. support vector
machines, random forests, gaussian processes and neural networks) to forecast
time series, these approaches have also been used for weather data [14, 15,
16, 17, 18, 19, 20]. In particular, neural networks based models can use
either shallow or deep architectures. As opposed to shallow networks that
require domain knowledge and feature engineering, deep convolutional neural
networks are less constrained by domain expertise. Indeed, these networks are
capable of extracting the underlying complex patterns of the data by stacking
of multiple nonlinear layers. Deep learning based models have already shown
promising results in weather elements forecasting as well as several other
domains such as biomedical signal analysis, healthcare, neuroscience and
dynamical systems among others [21, 22, 23, 24, 25, 26, 27, 28, 29, 30]. In
particular, convolutional neural networks has shown to be successful for a
wide range of tasks [31, 23]. The U-Net architecture which consists of two
main parts, i.e. the contracting and expanding, has shown to be efficient in
many computer vision tasks. It is efficient at processing the input data at
lower resolutions and restore it to its original resolution. In particular,
U-Net based architectures have been successfully applied to medical data for
diverse tasks such as segmentation, cell counting, and reconstruction [32, 33,
34].

The main contribution of this paper is to extend U-Net based deep learning
architectures to learn the underlying complex mapping between three-
dimensional input data and two-dimensional output data. These architectures
are then employed to perform multi-feature coastal weather forecasting as
shown in Fig. 1. More precisely, we show that among the four novel proposed
models, the one that uses a combination of asymmetric, parallel convolutions
and skip connections inside each residual block outperforms the other models
for the satellite imagery prediction of a coastal area of the Netherlands.

Figure 1: Prediction process overview. The time steps from t-d to t correspond
to the lags of the models to generate the prediction of the time step t+h,
where d is the number of lags and h is the number of time steps ahead.

This paper is organized as follows. The literature review of satellite imagery
for weather forecasting is discussed in section 2. The proposed models are
presented in section 3. Section 4 introduces the used dataset. The
experimental results, the corresponding discussion and finally the conclusion
are given in sections 5 and 6 respectively.

##  2 Related Work

Weather forecasting based on deep learning models has recently gain a lot of
attention due to the rapid advancement of neural network techniques and
availability of weather data [35]. The authors in [36], used a deep
convolutional neural network to predict thunderstorms and heavy rains. The
model was then compared against traditional machine learning models such as
random forests and support vector machines. The authors in [37] incorporated
multiple ConvLSTM layers to predict the precipitation rate using radar data.
Moreover, multi-stream convolutional neural networks combines with a self-
attention mechanism [38, 39] has been studied for precipitation forecasting
[40].

In this paper we are interested in weather forecasting based on satellite
imagery. Several approaches have been discussed in the literature for
performing frame prediction from satellite data for weather forecasting using
modern data-driven approaches. For instance, the authors in [41], used
satellite data in combination with deep learning techniques to perform sea
surface temperature (SST) prediction in a subarea of the East China sea. The
main type of layer used in this work was the ConvLSTM layer and was compared
to three different models: support vector regression (SVR) model, a
persistence model that was simulating a naive forecasting and a third model
based on LSTM layers. It was shown that the ConvLSTM model outperformed the
other models for $10$ days ahead prediction in a recursive fashion.

Similarly to the previous work, sea surface temperature forecasting is also
performed using deep learning and remote sensing imagery from satellite data
in [42]. The methodology discussed in in [42] is based on a multi-input
convolutional neural networks that process the inputs using different spatial
resolutions. The end goal of this work was to establish a relationship between
sea surface temperature forecasting and tropical instability waves.

In [43], eight cyclone datasets are used for two main objectives: classifying
whether the given image contains a storm or not as well as predicting the
storm’s location. In contrast with the previous works, this methodology is not
end-to-end since multiple preprocessing steps are performed before training
the deep learning model. In particular, multiple optical flow based techniques
are used to perform temporal interpolation and the result of this processing
is then fed to the deep learning models. The neural networks used are existing
approaches that provide a fast inference time, namely YOLO [44] and RetinaNet
[45].

Another similar research work in [46] performs precipitation nowcasting using
artificial neural networks and satellite data. In this work, thermal infrared
image prediction is first performed in order to get an estimation of the
predicted precipitation. Hourly data is used and the neural network model is
compared to other approaches such as linear interpolation, steady state
methodology and persistence prediction.

Furthermore, future frame prediction is an active field of research in
computer vision because of the several use cases where it can be applied such
as anomaly detection, video prediction among others. In this context, within
the field of deep learning, two main approaches are generally considered:
autoencoders and generative adversarial neural networks [47, 48, 49, 50].

Figure 2: Architecture of 3DDR-UNet model. The annotations above the
convolutions correspond to the output shape of those convolutions. Also, the
number of filters is indicated below. We can appreciate that the first part
reduces the dimensionality and then the second part upsamples the data to its
original size (except for the temporal dimension). Between the reduction and
expansion parts, intermediate convolutions reduce the temporal dimension
(lags) from 10 to 1.

##  3 Proposed models

We aim at proposing a model that accurately maps a set of input images to a
set of output images. To this end, the UNet architecture [32] is used as the
core model and is enriched by incorporating more advanced elements suitable
for the task under study. The U-Net architecture is initially designed for
medical image segmentation and has similar structure to that of an
autoencoder. A first contracting part, where the features are extracted from
the input image, is followed by an expanding part that performs classification
on each pixel.

In this paper, we propose an extended UNet architecture. Additionally modern
enhancement techniques such as residual connections [51], inception modules
[52, 53] and asymmetric convolutions [54] are taken into account when
designing these models. The residual or skip connections have shown to improve
the performance on deep networks by avoiding the vanishing of small gradients.
On the other hand, inception modules apply convolutions with different kernels
at the same level to capture features from larger and smaller areas in
parallel. In the same way, asymmetric convolutions allow us to enlarge the
network and thus its learning capacity, and at the same time the number of
parameters is reduced.

In what follows, we propose four different models, each one being an extended
version of the previous one. These models are 3DDR-UNet, Res-3DDR-UNet,
InceptionRes-3DDR-UNet and AsymmInceptionRes-3DDR-UNet.

###  3.1 3D Dimension Reducer UNet (3DDR-UNet)

Figure 3: Architecture of Res-3DDR-UNet model. The annotations above the
residual blocks correspond with the output shape of such blocks. Similarly to
3DDR-UNet, the first part reduces the dimensionality, and then the second part
upsamples the data to its original size (except for the temporal dimension).
Between the reduction and expansion parts, intermediate convolutions reduce
the temporal dimension (lags) from 10 to 1.

This section introduces 3D Dimension Reducer UNet (3DDR-UNet) which is based
on UNet core architecture [32]. The classical UNet model, is a fully
convolutional neural networks containing two parts: A contraction part or
encoder and an expansion part or decoder. The first part is composed of
stacked convolutions and pooling operations to extract features and capture
the context in the input. The second symmetric part combines the features
extracted in the contraction part with an upsampled output. In this way, the
network expands the data to its original size and projects the learned
features onto the pixel space to perform an accurate classification of them.

Here, we propose the 3DDR-UNet architecture, which manipulates 3-dimensional
data in the encoder and 2-dimensional data in the decoder. This configuration
allows the network to capture spatial and temporal dependencies from a stack
of 2-dimensional images in the contracting part. Then the first input
dimension (time dimension) is reduced from n (number of time-steps or lags,
which is 10 in our case) to 1 in the middle of the network before the data
moves towards the expanding part. Those extracted and combined features are
later used to reconstruct one single image in the decoder. The reduction in
the first input dimension is carried out by convolutions with kernel size
$n\times 1\times 1$ ($10\times 1\times 1$ in our case) and valid padding. The
output of these operations is a weighted average of different time-steps (i.e.
lags) in the input. Essentially, this architecture extracts features on the
encoder part and averages them in a weighted fashion over the first dimension
before it is fed into the decoder part. The number of convolutional filters
grows exponentially after each pooling from n to 16n in the encoder part, and
shrinks again to n in the decoder part, with a kernel size of $3\times 3\times
3$. The size of both the pooling and the upsampling operations is set to
$1\times 2\times 2$. In this way, the temporal dimension of the data remains
unchanged during the pooling and upsampling, while the spatial dimension of
the data (second and third dimension) is reduced and later upsampled.

Given the nature of the task, we train the network to perform a regression of
every pixel. Traditionally UNet is used for segmentation tasks. Here, as
opposed to the segmentation tasks, in which each pixel belongs to a class, we
first normalize both input and output data. Then the Mean Squared Error (MSE)
metric is used during the training to minimize the difference between the
predicted value of each pixel and the ground truth value. The architecture
3DDR-UNet of the network is shown in Fig. 2.

###  3.2 Residual 3D Dimension Reducer UNet (Res-3DDR-UNet)

Figure 4: Architecture of InceptionRes-3DDR-UNet model. The annotations above
the inception residual blocks correspond to the output shape of such blocks.
Similarly to the previous models, the first part reduces the dimensionality,
and then the second part upsamples the data to its original size (except for
the temporal dimension). Between the reduction and expansion parts,
intermediate convolutions reduce the temporal dimension (lags) from 10 to 1.

The second proposed model, Residual 3D Dimension Reducer UNet (Res-3DDR-UNet),
is an extension of 3DDR-UNet model introduced previously. In order to augment
its learning capacity, we scale up the model by adding a generous number of
convolutional operations. We use three convolutional layers and a skip
connection around the first layer and the final activation to avoid the
vanishing of gradients. All these operations form a residual block.

Further, the outputs of the last convolutions in the block are normalized
making use of a batch normalization layer. This normalization also aims at
increasing the robustness of the network and alleviate the vanishing gradient
problem.

Following the lines of [51], we skip two convolutional layers in each block
which enhances the performance as well as faster training. As a result of
these changes, the number of trainable parameters grows by 50% compare to the
3DDR-UNet. It should be noted that the kernel size of the convolutions,
pooling and upsampling as well as the loss function were similar to the ones
used in 3DDR-UNet. The architecture of the Res-3DDR-UNet is depicted in Fig.
3.

###  3.3 Inception Residual 3D Dimension Reducer UNet (InceptionRes-3DDR-UNet)

Motivated by the effectiveness of inception modules in CNN classifiers [52,
55], we include similar modules in our residual blocks. Within this module,
the data stream is splitted into parallel convolutions with different kernel
sizes. Later the branches are concatenated again. This structure is motivated
by the ability to extract various features through the usage of multiple
kernel sizes applied to the same data. After these parallel operations, the
features are concatenated and combined with a $1\times 1\times 1$ convolution,
which is equivalent to a weighted average. Hence, the network learns to favor
over time the branches with the most suitable kernels. Essentially this module
allows the network to employ different kernels for the task, and give more
importance to the most relevant ones.

Here in particular, we use three parallel branches with $1\times 1\times 1$,
$3\times 3\times 3$ and $5\times 5\times 5$ kernels. As suggested in [55], we
approximate the $5\times 5\times 5$ convolution by two sequential $3\times
3\times 3$ convolutions, leading to a reduction in the computational cost. In
addition, a convolution of $1\times 1\times 1$ is included at the beginning of
each branch to reduce the dimensionality of the data and thus reducing the
computational cost. Furthermore, inspired by the performance of [53], we kept
the residual connection that skips the parallel branches. The number of
convolutional filters, kernel sizes of pooling and upsampling as well as the
loss function are the same as those of the previous models. The architecture
of InceptionRes-3DDR-UNet is shown in Fig. 4.

###  3.4 Asymmetric Inception Residual 3D Dimension Reducer UNet
(AsymmInceptionRes-3DDR-UNet)

Driven by the need to reduce the parameters in InceptionRes-3DDR-UNet, we
introduce a lighter, yet more effective model. Here we use asymmetric
convolutions [54] to lower the complexity of the parallel convolutions. As
shown in Fig. 5, each kernel is decomposed into three simpler ones and applied
consecutively.

Figure 5: Example of kernel decomposition in the asymmetric convolution
operation.

The resulting combination of operations is an approximation of the original
operation with considerably fewer parameters (see [54] for more details). In
consequence of such reduction of parameters, we can afford to remove the
$1\times 1\times 1$ convolution at the beginning of each branch within the
asymmetric inception residual block, whose purpose is to reduce the complexity
of the data and thus making the model lighter. Furthermore, we have added two
more parallel branches in each block to make the model comparable to the
previous ones in terms of parameters. The number of convolutional filters, the
kernel size of pooling and upsampling as well as the loss function are the
same as in the previous described models. The architecture of
AsymmInceptionRes-3DDR-UNet is depicted in Fig. 6.

Figure 6: Architecture of AsymmInceptionRes-3DDR-UNet model. The annotations
above the asymmetric inception residual blocks correspond to the output shape
of such blocks. Similarly to the previous models, the first part reduces the
dimensionality, and then the second part upsamples the data to its original
size (except for the temporal dimension). Between the reduction and expansion
parts, intermediate convolutions reduce the temporal dimension (lags) from 10
to 1.

##  4 Data Description

The data used in this paper consists of satellite images. It is provided by
Copernicus 111https://marine.copernicus.eu/, the observation program led by
the European Commission and the European Space Agency (ESA). Specifically, it
is part of the dataset ”Atlantic - European North West Shelf - Ocean Physics
Analysis and Forecast” [56], covering a geographical area with longitude from
E 002°000 to E 006°000 and latitude from N 51°600 to N 53°400. The spatial
resolution is approximately 1.5 km, so every pixel represents a region of size
$1.5\times 1.5$ km. We chose such a geographical area since it covers both the
land and sea of the Netherlands. Furthermore, the selected observations start
on 01/03/2017 and end on 13/02/2019, with an hourly temporal resolution. The
dataset consists of four weather variables, i.e. Eastward current velocity
(EastCUR), Northward current velocity (NorthCUR), Seawater salinity (SAL) and
Sea surface height (SSH).

Therefore, each time-step of each variable is represented by $135\times 135$
image (see Fig. 7). For reproducibility purposes, the code as well as the
dataset are available on Github 222https://github.com/jesusgf96/Sea-Elements-
Prediction-UNet-Based-Models. More details on the included variables, can be
found in the official
documentation333https://resources.marine.copernicus.eu/documents/PUM/CMEMS-
NWS-PUM-004-013.pdf.

Figure 7: Example of the sea surface height in meters of the studied region.

##  5 Experimental Results

###  5.1 Data Preprocessing

As the data contains sea elements, the pixels in the image that represent the
ground should not be taken into account by the network. In practice, the
pixels corresponding to the land are masked initially. Hence, we apply a
MinMax scaling with a boundary of 0.1 and 1 to the pixels representing the sea
elements as follows:

$x=0.1+\frac{((x-x_{min})*(1-0.1))}{x_{max}-x_{min}}.$ (1)

Then the pixels representing the ground are assigned a zero value. In this
way, the pixels that belong to the ground are invisible to the models because
of the ReLu activation function used within them. Moreover, we crop the images
seven pixels from the right and the bottom sides, resulting in a $128\times
128$ shape, which is suitable for the subsequent convolutional and pooling
operations. The data is arranged in such a way that the resulting object is a
four-dimensional array $\mathcal{T}\in\mathbb{R}^{L\times H\times W\times V}$,
where $L$ is the number of timesteps, which makes up the time dimension. $H$
and $W$ refer to the size of the image and form the spatial dimensions. The
last element $V$ corresponds to the sea variables.

###  5.2 Experimental Setup

For all the models, we use all the variables as input, and we perform a
prediction of the same variables. The number of convolutional filters is
chosen in such a way that all models contain comparable total number of
trainable parameters.

In our experiments, the number of lags is set to 10 as empirically it was
found to yield better performance compared to other tested lag values.
Therefore, the model receives ten hours of information to predict one single
time-step, which translate into having an input with shape (10, 128, 128, 4)
and output with shape (1, 128, 128, 4). In addition, to test the predictive
ability of the models as well as their robustness, four different experiments
are carried out. It consists of performing 12, 24, 48, and 72 hours ahead
predictions for all the variables. The models are trained with data spanned
over a year, from 01/03/2017 to 01/03/2018, in total 8760 hours training data.
The validation data is composed of 2016 hours, and represents 504 hours from
each season (spring, summer, autumn and winter). This validation data
corresponds to a period of time after the training data. Similarly, the test
data is composed of another 2016 hours after the training data. The specific
days used in both the validation and test can be found in Table 1.

Table 1: Time-steps used for validation and test.

|  | Spring | Summer | Autumn | Winter
---|---|---|---|---|---
Validation | From | 01/04/2018 | 01/07/2018 | 01/10/2018 | 01/01/2019
To | 22/04/2018 | 22/07/2018 | 22/10/2018 | 22/01/2019
Test | From | 23/04/2018 | 23/07/2018 | 23/10/2018 | 23/01/2019
To | 13/05/2018 | 12/08/2018 | 13/11/2018 | 13/02/2019
|  |  |  |  |

###  5.3 Training

The same training setup is used in all the models. As mentioned previously in
section 3, the Mean Squared Error (MSE) is used as the loss function to
minimize the differences between the predicted and the ground truth image.
Adam optimization method [57] is applied to optimize the loss function. The
batch size and the dropout rate are set to 16 and 0.5 respectively. We also
implemented a checkpoint callback that monitors the validation loss and we let
the models to be trained for 100 epochs in each of the experiments. The best
results are then saved based on the performance of the models on the
validation data.

###  5.4 Results and discussion

This section presents the results obtained from the described experiments. The
obtained MSEs of all the models for each of the configurations are tabulated
in Table 2. These results correspond to testing of the models on the four
combined seasons. It can be observed that for all the models the prediction
error increases as the number of hours ahead increases.
AsymmInceptionRes-3DDR-UNet model performs better than the other models in
almost all the scenarios. This improvement in the performance is more apparent
as the number of hours ahead increases.

Table 2: Test MSE of all models in all four seasons. | Hours ahead
---|---
Model | 12h | 24h | 48h | 72h
3DDR-UNet | 5.40e-02 | 7.78e-02 | 1.21e-01 | 1.69e-01
Res-3DDR-UNet | 6.34e-02 | 7.99e-02 | 1.42e-01 | 1.77e-01
InceptionRes-3DDR-UNet | 5.19e-02 | 7.08e-02 | 1.20e-01 | 1.47e-01
AsymmInceptionRes-3DDR-UNet | 5.15e-02 | 7.56e-02 | 1.17e-01 | 1.41e-01
|  |  |  |

In Table 3, the test MSE of each model is displayed separated for each season.
Similar to the previous results, AsymmInceptionRes-3DDR-UNet outperforms the
other discussed models in most the setups. One may also observe that seasons
with more changing weather conditions such as winter makes it more challenging
for the networks to learn the underlying complex patterns. All of the four
models yield noticeably better predictions in seasons with more stable
weather, like summer. Furthermore, a comparison of the final number of
convolutional layers of each model can be found in Fig. 8. Fig. 9 shows the
obtained MSE of the test data for each sea element. Fig. 10 (a,b,c,d)
corresponds to 12, 24, 48 and 72 hours ahead prediction respectively. In
general, we observe that seawater salinity is the most challenging variables
to be predicted among the considered variables in this study.

Figure 8: Comparison of the number of parameters and convolutional layers between models. Table 3: Test MSE of all models in different seasons. Season | Model | 12h ahead | 24h ahead | 48h ahead | 72h ahead
---|---|---|---|---|---
Spring | 3DDR-UNet | 7.45e-02 | 9.95e-02 | 1.37e-01 | 2.14e-01
Res-3DDR-UNet | 8.37e-02 | 1.01e-01 | 1.65e-01 | 1.69e-01
InceptionRes-3DDR-UNet | 6.63e-02 | 9.37e-02 | 1.43e-01 | 1.78e-01
AsymmInceptionRes-3DDR-UNet | 6.98e-02 | 9.56e-02 | 1.38e-01 | 1.66e-01
Summer | 3DDR-UNet | 4.10e-02 | 7.92e-02 | 1.37e-01 | 1.69e-01
Res-3DDR-UNet | 5.23e-02 | 6.85e-02 | 1.23e-01 | 1.97e-01
InceptionRes-3DDR-UNet | 4.37e-02 | 6.55e-02 | 1.06e-01 | 1.49e-01
AsymmInceptionRes-3DDR-UNet | 4.03e-02 | 7.88e-02 | 1.04e-01 | 1.43e-01
Autumn | 3DDR-UNet | 3.36e-02 | 4.81e-02 | 6.95e-02 | 7.89e-02
Res-3DDR-UNet | 3.97e-02 | 5.61e-02 | 8.92e-02 | 1.27e-01
InceptionRes-3DDR-UNet | 3.29e-02 | 4.31e-02 | 6.98e-02 | 6.75e-02
AsymmInceptionRes-3DDR-UNet | 2.97e-02 | 4.66e-02 | 6.79e-02 | 7.11e-02
Winter | 3DDR-UNet | 6.62e-02 | 8.51e-02 | 1.62e-01 | 1.90e-01
Res-3DDR-UNet | 7.66e-02 | 9.44e-02 | 1.88e-01 | 1.99e-01
InceptionRes-3DDR-UNet | 6.41e-02 | 8.18e-02 | 1.63e-01 | 1.71e-01
AsymmInceptionRes-3DDR-UNet | 6.51e-02 | 8.28e-02 | 1.58e-01 | 1.62e-01
|  |  |  |  |

(a)  (b)

(c)  (d)

Figure 9: MSE of individual sea elements for different number of hours ahead.
The predictions are performed with the (a) 3DDR-UNet, (b) Res-3DDR-UNet, (c)
InceptionRes-3DDR-UNet, and (d)AsymmInceptionRes-3DDR-UNet trained models.

(a)  (b)

(c)  (d)

Figure 10: MSE of individual sea elements for the different models. The
predictions are performed using (a) 12h, (b) 24h, (c) 48h, and (d) 72h ahead.

(a)

(b)

(c)

(d)

Figure 11: AsymmInceptionRes-3DDR-UNet’s prediction of all variables 48 hours
ahead in winter. The data is scaled back to its original values.

An example of the 48h ahead forecast with AsymmInceptionRes-3DDR-UNet model
during winter is shown in Fig. 11. As it can be seen, the forecast is
considerably accurate, even when it comes to seawater velocity (EastCUR and
NorthCUR), which contains quite different areas. The obtained results suggest
that inclusion of parallel branches of convolutions, presented in
InceptionRes-3DDR-UNet and Asymm InceptionRes-3DDR-UNet models has led to a
more noticeable performance improvement.

##  6 Conclusion

In this paper, four new models based on the U-Net architecture are introduced
for multi-step ahead coastal sea elements prediction. The proposed models are
examined under different setups, i.e. different seasons and numbers of hours
ahead. Among the discussed models, AsymmInceptionRes-3DDR-UNet and
InceptionRes-3DDR-UNet have shown superior performance thanks to the use of
parallel convolutions. However, the incorporation of asymmetric convolutions
and additional parallel branches make the AsymmInceptionRes-3DDR-UNet perform
slightly better than the latter, yielding the most promising results on the
studied tasks. The scripts and models used in this paper can be found in
https://github.com/jesusgf96/Sea-Elements-Prediction-UNet-Based-Models.

## Acknowledgment

Simulations were performed with computing resources granted by RWTH Aachen
University.

## References

  * [1] R. W. Katz, A. H. Murphy, Economic value of weather and climate forecasts, Cambridge University Press, 2005.
  * [2] R. G. da Silva, M. H. D. M. Ribeiro, S. R. Moreno, V. C. Mariani, L. dos Santos Coelho, A novel decomposition-ensemble learning framework for multi-step ahead wind energy forecasting, Energy 216 (2021) 119174.
  * [3] S. R. Moreno, V. C. Mariani, L. dos Santos Coelho, Hybrid multi-stage decomposition with parametric model applied to wind speed forecasting in brazilian northeast, Renewable Energy 164 (2021) 1508–1526.
  * [4] Z. Liu, R. Hara, H. Kita, Hybrid forecasting system based on data area division and deep learning neural network for short-term wind speed forecasting, Energy Conversion and Management 238 (2021) 114136.
  * [5] S. R. Moreno, R. G. da Silva, V. C. Mariani, L. dos Santos Coelho, Multi-step wind speed forecasting based on hybrid multi-stage decomposition model and long short-term memory neural network, Energy Conversion and Management 213 (2020) 112869.
  * [6] J. Henderson, E. R. Nielsen, G. R. Herman, R. S. Schumacher, A hazard multiple: Overlapping tornado and flash flood warnings in a national weather service forecast office in the southeastern united states, Weather and Forecasting 35 (4) (2020) 1459–1481.
  * [7] K. M. Simmons, D. Sutter, Wsr-88d radar, tornado warnings, and tornado casualties, Weather and Forecasting 20 (3) (2005) 301–310.
  * [8] A. C. Lorenc, Analysis methods for numerical weather prediction, Quarterly Journal of the Royal Meteorological Society 112 (474) (1986) 1177–1194.
  * [9] P. Bauer, A. Thorpe, G. Brunet, The quiet revolution of numerical weather prediction, Nature 525 (7567) (2015) 47–55.
  * [10] H. R. Glahn, Statistical weather forecasting (1985).
  * [11] A. Holtslag, E. De Bruijn, H. Pan, A high resolution air mass transformation model for short-range weather forecasting, Monthly Weather Review 118 (8) (1990) 1561–1575.
  * [12] K. Saito, H. Seko, T. Kuroda, T. Fujita, T. Kawabata, K. Aonashi, T. Tsuyuki, Next generation supercomputer project toward cloud resolving nwp, CAS/JSC WGNE Res. Act. Atmos. Ocea. Model 41 (2011) 5–19.
  * [13] J. Zhongzhen, Z. Qingcun, Problems on nonlinear computational instability in nwp, Journal of the Meteorological Society of Japan. Ser. II 64 (1986) 255–261.
  * [14] K.-j. Kim, Financial time series forecasting using support vector machines, Neurocomputing 55 (1-2) (2003) 307–319.
  * [15] G. Dudek, Short-term load forecasting using random forests, in: Intelligent Systems’ 2014, Springer, 2015, pp. 821–828.
  * [16] A. Girard, C. E. Rasmussen, J. Q. Candela, R. Murray-Smith, Gaussian process priors with uncertain inputs application to multiple-step ahead time series forecasting, in: Advances in neural information processing systems, 2003, pp. 545–552.
  * [17] Y. Radhika, M. Shashi, Atmospheric temperature prediction using support vector machines, International journal of computer theory and engineering 1 (1) (2009) 55.
  * [18] K. Rasouli, W. W. Hsieh, A. J. Cannon, Daily streamflow forecasting by machine learning methods with weather and climate inputs, Journal of Hydrology 414 (2012) 284–293.
  * [19] K. Trebing, T. Stańczyk, S. Mehrkanoon, SmaAT-UNet: Precipitation nowcasting using a small attention-unet architecture, Pattern Recognition Letters 145 (2021) 178–186.
  * [20] K. Trebing, S. Mehrkanoon, Wind speed prediction using multidimensional convolutional neural networks, in: IEEE Symposium Series on Computational Intelligence (IEEE-SSCI), 2020, pp. 713–720.
  * [21] S. Webb, Deep learning for biology, Nature 554 (7693) (2018).
  * [22] S. Mehrkanoon, J. A. K. Suykens, Deep hybrid neural-kernel networks using random fourier features, Neurocomputing 298 (2018) 46–54.
  * [23] S. Mehrkanoon, Deep neural-kernel blocks, Neural Networks 116 (2019) 46–55.
  * [24] I. Alaoui Abdellaoui, J. García Fernández, C. Şahinli, S. Mehrkanoon, Enhancing brain decoding using attention augmented deep neural networks, in: European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), 2021, pp. 183–188.
  * [25] S. Mehrkanoon, T. Falck, J. A. K. Suykens, Approximate solutions to ordinary differential equations using least squares support vector machines, IEEE transactions on neural networks and learning systems 23 (9) (2012) 1356–1367.
  * [26] S. Mehrkanoon, J. A. K. Suykens, Learning solutions to partial differential equations using ls-svm, Neurocomputing 159 (2015) 105–116.
  * [27] S. Mehrkanoon, S. Mehrkanoon, J. A. K. Suykens, Parameter estimation of delay differential equations: an integration-free ls-svm approach, Communications in Nonlinear Science and Numerical Simulation 19 (4) (2014) 830–841.
  * [28] S. Mehrkanoon, Cross-domain neural-kernel networks, Pattern Recognition Letters 125 (2019) 474–480.
  * [29] I. A. Abdellaoui, S. Mehrkanoon, Symbolic regression for scientific discovery: an application to wind speed forecasting, in: Accepted for publication in the Proc. of IEEE Symposium Series on Computational Intelligence (IEEE-SSCI), 2021\.
  * [30] T. Stanczyk, S. Mehrkanoon, Deep graph convolutional networks for wind speed prediction, in: European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), 2021, pp. 147–152.
  * [31] A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks, in: Advances in neural information processing systems, 2012, pp. 1097–1105.
  * [32] O. Ronneberger, P. Fischer, T. Brox, U-net: Convolutional networks for biomedical image segmentation, in: International Conference on Medical image computing and computer-assisted intervention, Springer, 2015, pp. 234–241.
  * [33] T. Falk, D. Mai, R. Bensch, Ö. Çiçek, A. Abdulkadir, Y. Marrakchi, A. Böhm, J. Deubner, Z. Jäckel, K. Seiwald, et al., U-net: deep learning for cell counting, detection, and morphometry, Nature methods 16 (1) (2019) 67–70.
  * [34] Y. Han, J. C. Ye, Framing u-net via deep convolutional framelets: Application to sparse-view ct, IEEE transactions on medical imaging 37 (6) (2018) 1418–1429.
  * [35] S. Mehrkanoon, Deep shared representation learning for weather elements forecasting, Knowledge-Based Systems 179 (2019) 120–128.
  * [36] K. Zhou, Y. Zheng, B. Li, W. Dong, X. Zhang, Forecasting different types of convective weather: A deep learning approach, Journal of Meteorological Research 33 (5) (2019) 797–809.
  * [37] S. Kim, S. Hong, M. Joh, S.-k. Song, Deeprain: Convlstm network for precipitation prediction using multichannel radar data, arXiv preprint arXiv:1711.02316 (2017).
  * [38] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, in: Advances in neural information processing systems, 2017, pp. 5998–6008.
  * [39] J. Ho, N. Kalchbrenner, D. Weissenborn, T. Salimans, Axial attention in multidimensional transformers, arXiv preprint arXiv:1912.12180 (2019).
  * [40] C. K. Sønderby, L. Espeholt, J. Heek, M. Dehghani, A. Oliver, T. Salimans, S. Agrawal, J. Hickey, N. Kalchbrenner, Metnet: A neural weather model for precipitation forecasting, arXiv preprint arXiv:2003.12140 (2020).
  * [41] C. Xiao, N. Chen, C. Hu, K. Wang, Z. Xu, Y. Cai, L. Xu, Z. Chen, J. Gong, A spatiotemporal deep learning model for sea surface temperature field prediction using time-series satellite data, Environmental Modelling & Software 120 (2019) 104502.
  * [42] G. Zheng, X. Li, R.-H. Zhang, B. Liu, Purely satellite data–driven deep learning forecast of complicated tropical instability waves, Science Advances 6 (29) (2020) eaba1482.
  * [43] S. Shakya, S. Kumar, M. Goswami, Deep learning algorithm for satellite imaging based cyclone detection, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 13 (2020) 827–839.
  * [44] J. Redmon, S. Divvala, R. Girshick, A. Farhadi, You only look once: Unified, real-time object detection, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 779–788.
  * [45] T.-Y. Lin, P. Goyal, R. Girshick, K. He, P. Dollár, Focal loss for dense object detection, in: Proceedings of the IEEE international conference on computer vision, 2017, pp. 2980–2988.
  * [46] G. Rivolta, F. Marzano, E. Coppola, M. Verdecchia, Artificial neural-network technique for precipitation nowcasting from satellite imagery (2006).
  * [47] W. Liu, W. Luo, D. Lian, S. Gao, Future frame prediction for anomaly detection–a new baseline, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 6536–6545.
  * [48] V. Patraucean, A. Handa, R. Cipolla, Spatio-temporal video autoencoder with differentiable memory, arXiv preprint arXiv:1511.06309 (2015).
  * [49] J.-T. Hsieh, B. Liu, D.-A. Huang, L. F. Fei-Fei, J. C. Niebles, Learning to decompose and disentangle representations for video prediction, in: Advances in Neural Information Processing Systems, 2018, pp. 517–526.
  * [50] Y. Zhao, B. Deng, C. Shen, Y. Liu, H. Lu, X.-S. Hua, Spatio-temporal autoencoder for video anomaly detection, in: Proceedings of the 25th ACM international conference on Multimedia, 2017, pp. 1933–1941.
  * [51] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  * [52] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, Going deeper with convolutions, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1–9.
  * [53] C. Szegedy, S. Ioffe, V. Vanhoucke, A. Alemi, Inception-v4, inception-resnet and the impact of residual connections on learning, arXiv preprint arXiv:1602.07261 (2016).
  * [54] H. Yang, C. Yuan, B. Li, Y. Du, J. Xing, W. Hu, S. J. Maybank, Asymmetric 3d convolutional neural networks for action recognition, Pattern recognition 85 (2019) 1–12.
  * [55] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, Z. Wojna, Rethinking the inception architecture for computer vision, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2818–2826.
  * [56] M. Tonani, P. Sykes, R. R. Kin, N. McConnell, A. C. Péquignet, E. O’Dea, J. A. Graham, J. Polton, J. Siddorn, Atlantic - european north west shelf - ocean physics analysis and forecast dataset, Ocean Sci. 15 (2019) 1133–1158, [https://doi.org/10.5194/os-15-1133-2019 The impact of a new high-resolution ocean model on the Met Office North-West European Shelf forecasting system].
  * [57] D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980 (2014).