the forward scattering formulation, it can be employed to calculate the $VSF$
in backscattering as well. The $PSF$ formulation as well as its relationship
to the $VSF$ has been studied in great detail in [49].

The forward- and back-scattering process encountered in seawater disperse the
light beam, hence resulting in blurred underwater images. Observe from (14)
that the image gets more blurred, when the distance increases. One can exploit
this equation to mitigate the blurring problem with the aid of image
deblurring methods, which will be covered in Section IV-C2.

By contrast to active imaging, in passive imaging, i.e. where no artificial
light is generated by the image capturer, even though the power consumption
will be significantly lower, other concerns may be present. These concerns
include limited visibility, contrast, and color distortion.

Visibility: The first concern in passive underwater imaging is always the
visibility. In clear sunlit water, ambient sunlight provides a clear vision in
relatively shallow water. In ordinary line-of-sight underwater applications,
the distance of visibility can be modeled as,

$d_{Visible}\approx\dfrac{5}{c-K(\theta,\varphi,z)cos(\theta)}\,,$ (15)

where $K$ is the diffuse attenuation coefficient, which is an apparent optical
parameter, while $\theta$, $\varphi$, and $z$ are the relative spherical
coordinates of the subject of interest [136]. Based on this formula,
underwater visibility decreases rapidly after a few meters. The only known
remedy for this physical constraint in deep-waters is to use an artificial
light source in active imaging [135].

Contrast: Contrast is defined as the color or gray-level difference between
adjacent areas in the presence of light attenuation, optical noise, and vision
blurring. If we consider a target object ($O$) against its background ($B$),
the contrast of the object and its surrounding areas can be formulated as
[137],

$C_{OB}\overset{\Delta}{=}\dfrac{L_{O}(\theta,\varphi,z)-L_{B}(\theta,\varphi,z)}{L_{B}(\theta,\varphi,z)}\,,$
(16)

where $L_{O}$ is the light level (radiance) of the object and $L_{B}$ is that
of background. The knowledge of $C_{OB}$ will help us design contrast-
enhancement algorithms for improving the contrast in underwater environments
[138].

Color Distortion: As previously discussed in Section II-B and based on (4),
light absorption (or attenuation) in underwater propagation strongly depends
on the wavelength. Accordingly, all of the visible light wavelengths present
in sea surface, provide a crystal clear view. By contrast, red light hardly
penetrates bellow 10m, which is the reason of the greenish color of seawater.
Colors having shorter wavelengths (i.e. blue color family) penetrate even
deeper to 20 and up to 50 m in sea water, resulting in low-contrast bluish
images.

Color distortion and the visibility are the main reasons why passive imaging
is inefficient bellow 10 meters and also why the use of artificial light
sources is inevitable in deep-water explorations. To address this important
issue, the image enhancement or color restoration methods of Section IV-C2
have to be involved.

Having surveyed some of the 1D signals generated by sensors and the 2D image
data types encountered in IoUT, let us now focus our attention on video or 3D
data, that forms a large subset of IoUT datasets.

####  III-B3 Three-dimensional underwater video data

As discussed, some of the observatory stations listed in Table VIII are
capable of delivering live or archival ready-to-use video data. However, the
main challenges in underwater video streaming are related to the limitations
of undersea communication, discussed in Section II-B, aggravated by the
underwater imaging difficulties studied in Section III-B2.

Although the underwater optical parameters and imaging constraints are all the
same as in 2D imagery (i.e. contrast, color distortion, visibility,
backscatter, and forward scatter), underwater video imaging has an extra
intrinsic impediment that has to be addressed. This barrier, is of course, the
continuously growing data volume produced by cameras. A typical camera
operating at 1 frame/sec may generate 30 million frames (equal to almost 3
Tera bytes of data) per year [139]. This data volume is then multiplied by the
number of cameras, mounted in a single observatory system.

As a rule of thumb, in the current version of the semi-automatic marine image-
annotation software, every minute of video requires an expert to spend about
15 minutes for manual annotation [139]. Therefore, to analyze the video
produced by a single camera in a single month, we approximately need 10,800
man-hour. Hence, there is a growing necessity to develop automatic video
processing methods to deal with this excessive data volume. These automatic
methods can be used in a variety of underwater video data applications such
as, visible light video tracking [140], sonar video tracking [141], photo
mosaicing underwater [20], and marine life studies [14].

In addition to the above-mentioned potential use cases of automatic video
processing, the growing field of depth-based video may also significantly
benefit from automatic processing. This technology uses optical multi-camera
systems [142, 143], acoustic arrays [144], Time of Flight (ToF) depth sensors
[145], and laser beams [146] to provide rich 3D information about the scene.
This stereoscopic video technology has been used in underwater vehicles (e.g.
ROVs, submersibles, etc.) [147, 148], in tethered underwater platforms [149],
towed systems [140], and baited stations [150]. They are used in a wide
variety of applications, including photogrammetric bundle adjustment [151,
152], 3D scene and organism reconstruction [145, 143], underwater 3D live
tracking [140, 152], quantitative analysis and sizing of targets [149, 140],
counting and measurement of marine creatures [150], as well as improving the
segmentation and classification capabilities of traditional 2D algorithms
[150, 153].

However, the sophisticated depth-based vision technology has its own
challenges both in the context of acoustic and optical recording methods.
While acoustic depth vision always suffers from variable sound velocity,
reverberation, as well as unwanted noise and echoes, optical stereoscopic
cameras encounter blur and haziness in turbid water, aggravated by unstable
illumination, and light refraction [140]. Perhaps, the most direct way of
addressing this problem is to use improved acoustic and optical channel models
for compensating the deficiencies of each technology [151]. Another solution
is to combine both technologies into a unified opti-acoustic 3D imaging
system. The latter solution is capable of offering improved precision, if both
technologies cooperatively calibrate each other [148].

So far in this section, we have provided a detailed evaluation of the family
of IoUT sensors, as well as of image and video data sources. Additionally,
some ready-to-use databases were introduced. Once the raw big data has been
gathered, a high-performance data processing platform is required for
inferring knowledge. Due to the scarcity of resources and owing to the limited
power budget at the data collection points, processing cannot be performed
locally. Therefore, the system has to rely on cloud-based or on distributed
data processing platforms, which will be studied in the next section.

###  III-C Distributed and Cloud-based BMD Processing

To meet the growing demand for big data processing, many high-quality software
and services have been created to offer big data analytics. Some of these
platforms are Apache, Amazon EMR, Microsoft Azure HDInsight, Cloudera,
Hortonworks (which has recently been merged with Cloudera), SAP-Hana, HP-
HAVEn, 1010data, Pivotal data suite, Infobright, etc. [12, 32]. Each of these
services responds to the big data processing demands by either providing
distributed processing frameworks or cloud computing services.

Distributed processing systems consist of networked computers, which cooperate
with each other to offer high performance data processing [154]. As seen in
Fig. 8, among all distributed big data analytics frameworks, the Apache is the
dominant platform, which has been used in about $95\%$ of all reported
scientific articles. Apache software foundation is a not-for-profit
corporation, founded in 1999 to support more than 350 open-source Apache
software projects, and 47 of them (about $13\%$) are directly related to big
data analytics. More than $80\%$ of the Apache platforms have been developed
in only five languages (i.e. Java ($\approx 60\%$), different types of C
($\approx 10\%$), Python ($\approx 5\%$), JavaScript ($\approx 3\%$), and
Scala ($\approx 3\%)$) [155].

Figure 8: Distributed big data frameworks used in data analytics papers
published in IEEE and Web of Science databases.

Table X introduces the advantages and disadvantages of the four widely used
Apache distributed processing frameworks including Spark, Hadoop, Storm, and
Flink. All of these open-source projects are supported by Apache software
foundation and they are eminently suitable for high-speed processing of IoUT-
generated BMD analytics.

TABLE X: Major Apache Frameworks Suggested for Distributed Processing of BMD

The term cluster in Table X refers to any computer system or embedded system
in the parallelized network of computers, each having its own processor,
memory, and network IO. Furthermore, the directed acyclic graph scheduler is
based on a specific type of directed mathematical graph having no cycles, in
which it is impossible for the data to emanate from a vertex (i.e. any
standalone computer) and pass over a non-zero number of graph edges (i.e.
inter-computer connection cables), and to eventually loop back to the starting
vertex again.

In addition to the locally distributed processing systems, there exist
companies who offer cloud-based services for all aspects of big data storage,
integration, streaming, ML, and ad hoc data analysis. According to the
Statista, the share of software and services in overall big data revenue will
double in the 6-year period spanning from 2019 to 2025. This means that the
organizations will rely more on cloud services to handle their sophisticated
use cases [156]. In other words, big data processing is expected to gradually
shift from distributed to cloud-based services. This is an ongoing trend, as
the number of businesses performing their big data analysis in the cloud has
increased from $58\%$ in 2017 to $73\%$ in 2018 [157].

As already mentioned in Section II-J, BMD can partly be processed in the IoUT-
edge, before being transmitted to land. After transmission, there is no
difference between the conventional IoT and IoUT-based distributed or cloud-
based data processing. Nevertheless, all available cloud services today are
offered in accordance with one of the following models [158]:

  * •

Software as a Service (SaaS): This service model relies on centrally hosted
software, which delivers specific services to licensed or subscribed clients
and usually offers its dedicated functionality through a browser.

  * •

Platform as a Service (PaaS): This service model targets developers by
providing them with operating systems, databases, software packages,
application services, etc. It helps developers to focus on the development of
their diverse applications, instead of software resource management and
hardware maintenance of the underlying infrastructure.

  * •

Infrastructure as a Service (IaaS): This flexible low-level service model
targets both developers and businesses by providing access to the underlying
infrastructure (e.g. processors, data partitions, security, backup, etc.).
This service tends to rely on high-level Application Programming Interfaces
(API) in support of the network operations. While the service provider is
responsible for the hardware maintenance, the users are required to configure
and maintain both operating system and the required software packages.

According to the aforementioned definitions, cloud computing is capable of
processing BMD with the aid of all the above service models. Among them, PaaS
strikes an attractive balance between convenience and cost, undertaking the
management of the operating system, software packages, and hardware elements.
It is worth mentioning that, Big Data as a Service (BDaaS), which has recently
appeared in technical glossaries may indeed be classified as a specific form
of PaaS, where the statistical analysis software packages are offered
alongside the required databases and APIs.

On the other hand, IaaS constitutes a cost-effective solution, where the
service providers are responsible for maintaining the underlying hardware.
Some of the top Paas and IaaS service providers are Amazon Web Services,
Microsoft Azure, Google Compute Engine, DigitalOcean, Oracle Cloud Compute,
Rackspace Cloud, IBM Cloud, Linode, HP Enterprise Converged Systems, Green
Cloud Technologies, etc.

Despite the fact that SaaS providers tend to be user-oriented and application-
specific, some of them also support underwater applications. For instance,
marine image-annotation software are ready-to-use software products for semi-
automatic annotation of videos and still-images. These software packages are
comprehensively reviewed by Gomes-Pereira et al. [159], where 23 software
products from more than 500 publications are summarized.

The semi-automatic solutions offered by marine image-annotation software are
rarely applicable in real-time BMD analytics and they tend to rely on data
analytic platforms as well as self-developed ML algorithms. As a result, every
practical use case of BMD, requires a distributed on-premise platform or
cloud-based data processing software for gleaning knowledge from BMD. Next,
some of these IoUT data extraction applications are studied.

###  III-D Marine Data Applications

The marine data collected by various sensors and devices of the IoUT ecosystem
can be exploited by data processing platforms for compelling applications.
These applications offer businesses benefits based on their own data [32]. In
the following subsections, we discuss and evaluate some of these applications
in the context of IoUT, including maritime applications, underwater
localization, and marine life tracking.

####  III-D1 Coastal monitoring and GIS data

By appropriately processing the available maritime sensory data, some
secondary parameters can be collated with the aid of the IoUT. These include
the accurate localization of marine vehicles, the provision of weather and
climate data for specific oceanic locations, accessing biogeographic data such
as the recognition, counting, and distribution of underwater species, etc.
Hereafter, we will refer to all these secondary parameters as IoUT Geographic
Data, because they are devoted to studying the physical environment, the
inhabitants, locations, and things in the particular area of the sea.

Having accurate geographical data is essential in IoUT applications. For this
type of IoUT data, there exist a number of open-source on-line geographical
databases. These databases are mainly focused on maritime information systems,
a particular type of Geographic Information System (GIS), which is
indispensable in efficient international transportation [160]. Fortunately,
most of these GISs can be readily merged with the IoUT infrastructure in
support of maritime organizations with the aid of tracking and routing
information, etc. They can also provide up-to-date access for researchers in
order to monitor the global ecosystems.

Kalyvas et al. [31] surveyed more than 180 free real-world GISs having open
access databases based on the most trustworthy on-line data sources. They have
categorized the GISs into 19 classes, which are distilled into as few as four
classes here for simplicity. These classes and their applications include:

  * •

IoUT tracking data: For vessel monitoring, marine accidents, anti-shipping
activities, and navigational aid systems;

  * •

Marine cartographic data: For essential naval data (like borders), protected
and sensitive areas, port status, port locations and facilities, coastline as
well as land areas, and bathymetry maps;

  * •

Oceanic climate data: For nautical weather, natural hazards, tides as well as
eddies, and satellite imagery;

  * •

IoUT commerce data: For shipping companies, flags of convenience, and marine
conservation organizations.

An important category that could be added to the above-mentioned classes of
Kalyvas et al. [31] is Biogeographical Data for addressing the geospatial
distribution of underwater species. In order to define this new category, the
open-access OBIS project (Ocean Biogeographic Information System), funded by
UNESCO can be used. OBIS is connected to more than 500 databases in 56
countries and provides observation of 120,000 marine species down to
$11,000\,m$ depth, from bacteria to whales.

In addition to the aforementioned need for Biogeographical data, under the
Marine Cartographic Data category detailed in [31], there is a missing sub-
category for Georeferenced Locations. To elaborate a little further,
georeferencing in GIS is a subcategory of navigational assistance that aims
for precisely associating locations with their equivalent points on the
physical maps, which is achieved by the Marine Regions project. This project
is composed of a list of 55,000 georeferenced marine places, gathered from the
VLIMAR Gazetteer and MARBOUND databases.

In addition to the GIS databases, there should also be a directory of
applications, which require access to precise undersea locations. This is
called underwater positioning. In the following sections, undersea
localization of sensors and vehicles, along with the marine life tracking
methods will be discussed in more details.

####  III-D2 Underwater localization

A very useful data type that is included in IoUT is related to the positioning
of undersea devices, systems, animal species, and data sources. This is even
more critical when there is a need to geo-tag IoUT sensory and imagery data.

Underwater positioning is a challenging task, because the otherwise ubiquitous
navigation signals of the GPS satellites do not penetrate seawater. Therefore,
other underwater navigation methods should be used, including blind
positioning relying on miscellaneous inertial sensors, acoustic transponders
(with separate transmitters and receivers), ranging sonars (with only a single
standalone acoustic transceiver), image-based positioning (using cameras to
localize), and Simultaneous Localization and Mapping (SLAM).

Due to the associated challenges including the long latencies, multipath
fading, Doppler shifts, frequency limitations, sparse deployment of the nodes,
and their high mobility in underwater networks, a single navigation technique
will never offer a flawless performance [161, 162]. Therefore, all the
vehicles, fixed stations, sinks, and nodes in IoUT applications usually
combine some of these localization methods to achieve improved performance in
underwater environments. Here, we briefly review the aforementioned
localization techniques and provide insights concerning their advantages and
limitations.

Blind positioning: which is also known as inertial navigation, is based on the
knowledge of a device’s relative orientation, acceleration, velocity, and
gravity anomaly (i.e. difference between the observed gravity and the
predicted value [163]). In this localization method, the underwater device has
to determine its position automatically, i.e., without any positioning support
from a ship or transponder. In this method, a combination of sensors is used
to estimate the current location.

These sensors include a typical magnetic compass for direction detection, a
pressure sensor for underwater depth estimation, a doppler velocity log for
velocity measurement, a ring laser gyroscope or Micro Electro-Mechanical
Systems (MEMS) gyroscope for angular velocity detection, and a pendulum or
MEMS accelerometer for force and acceleration calculations [29].

However, these sophisticated blind positioning methods suffer from propagating
errors. Even a small positioning error remains in the memory of the system and
aggravates the future measurement errors, leading to an unbounded error
propagation. Nevertheless, the method’s estimation of the exact position can
be improved by a variety of integrated sensors. Furthermore, blind positioning
is a power-efficient method, compared to other positioning techniques [164].
Hence these positioning techniques contribute to almost all modern underwater
positioning systems [161, 162, 164, 163, 165].

Acoustic transponders: This localization technique uses a transmitter and
receiver pair for measuring the ToF of a ping signal in order to perform
navigation. ToF $[sec]$ is applicable both to the adjacent
transmitter/receiver (i.e. where the transmitter and receiver are closely
located) and separate transmitter/receiver (i.e. where transmitter and
receiver are distantly located). It is also known as the time of arrival in
separate transmitter/receiver scenarios [162]. By measuring the ToF and the
acoustic wave speed, one can precisely measure the distance in $[m]$. This
positioning method has diverse categories, including:

  * •

Acoustic array: Similar to the concept of GPS satellites, and by using more
than one beacon transponders, the system will be able to determine the
position in any underwater location based on the phase-difference of the
signals arriving at the transceivers. Short- as well as ultrashort-baseline
[7] and long-baseline [166] techniques may be used in this category. In short-
and ultrashort-baseline relying on ship-mounted transponders, the undersea
system localizes itself relative to the ship floating at the surface; while in
long-baseline that uses GPS-intelligence as well as dispersed buoys and
beacons, the location can be determined.

  * •

Single fixed transponder: The idea behind fixed beacon based positioning is
shown in Fig. 9. The AUV in this picture has an uncertain prior knowledge
about its position (region A). However, it knows its distance from a fixed
beacon subject to a degree of uncertainty (region B). With the advent of
combining these two pieces of information, the AUV finds its position subject
to a reduced uncertainty (region C). Due to the fact that it uses only a
single geo-referenced beacon, this positioning system is cheaper and easier to
install than multi-beacon long-baseline techniques [167].

  * •

Cooperative navigation: This method is also known as modem-based navigation,
because in recent advanced positioning systems, modem transponders are also
used to send beacons in support of navigation based on the ToF [168]. These
modems do not have to be stationary, hence they can be installed on a moving
vessel having a known geo-position or a swarm of underwater vehicles to
communicate and to cooperatively localize each other.

Figure 9: Using a fixed beacon at known location to decrease the location-
uncertainty of an AUV from ‘A’ (intersecting with ‘B’) to ‘C’.

Range sonars: Sonars are robust, well-established standalone acoustic
transceivers that were originally used for imaging and ranging. The main
ranging sonars used in underwater navigation and mapping include [29]:

  * •

Echo sounder: A single narrow beam is used for determining the distance from
obstacles [169] or from the seafloor.

  * •

Multi-beam: Measures the ToF for each beam to assemble a bathymetric map
[170].

Image-based positioning: This technique uses environmental images, taken
either by monocular or stereoscopic cameras (or even by imaging sonars) for
navigation. In this positioning method, substantial processing power is
required for feature extraction and for processing in order to detect and
identify key points, objects, and regions of interest.

The main idea behind the vision-based navigation, also known as visual
odometry, is to capture images of the seabed and then to match subsequent
images in order to navigate [171, 172]. stereoscopic cameras have the
advantage of capturing 3D transformations between consecutive image pairs.

It is plausible that errors can propagate and accumulate in the context of
this technique. To address this problem and achieve a bounded positioning
error, the SLAM tactic can be merged with image-based positioning for improved
localization.

SLAM: Concurrent or Simultaneous Localization and Mapping is defined as the
process in which an autonomous vehicle builds a map of a specific area and
also localizes itself within it [29]. The mapping process of SLAM may rely on
a variety of devices such as cameras, sonars, or inertial sensors,
respectively, leading to vision-based, sonar-based, or sensor-based SLAM
techniques [173, 174]. In all cases, the features of the sensed data or
captured image are extracted. Then, based on those features, the position is
detected and stored.

####  III-D3 Marine life tracking

At first glance, underwater animal tracking seems to be nothing more than a
memory-aided localization method in IoUT, which was the subject of the
previous section. However, undersea animals may be quite small and they cannot
carry relatively heavy inertial sensors, transponders, sonars, or cameras.
Therefore, sophisticated new tracking methods have to be devised.

Additionally, as discussed in Section III-D2 and also bearing in mind the
penetration depth formula of (1), the GPS signals having a frequency of $1.2$
and $1.5\,GHz$ cannot penetrate seawater beyond a few meters. Accordingly,
alternative marine life tracking methods have to be implemented in IoUT
applications, some of which are evaluated in this section. All these methods
can make use of a data storage tag to archive data on a memory chip for future
retrieval [175].

RFID tags: This tracking technology uses Radio Frequency Identification (RFID)
patches and has a limited range of about $10\,m$ in freshwater. However, it
does not work well in seawater owing to its high salinity. RFID tags are
equipped with a unique Identification code and need an external energy source
in the form of a low frequency signal, in order to become activated and to
retrieve data. These tags are also available in passive integrated transponder
form, which are specific implantable RFID devices [4].

Acoustic tags: This tracking technology emits signals in the form of acoustic
waves and has a reasonable transmission range both in fresh and seawater. In
this method, a pinging sound with an embedded ID is periodically transmitted
to an array of remote acoustic sensors (hydrophones). This ping is used to
locate the animals and the ID is used to identify them [175].

Image-based tracking: Perhaps image-based tracking is one of the most advanced
underwater tracking techniques reported to date. It uses images of a moving or
fixed camera system to track, recognize, count, measure, and also study the
animal’s behaviors. In a fixed camera based system, usually a baited or
trawled station is used for attracting intended species [140]. On the other
hand, the tracking of animals in a moving camera based system uses cameras
installed on a moving platform. The main challenge in these systems is to
promptly process the large variety of images [21].

###  III-E Section Summary

In this section, we have studied the essential topics of big marine data. At
the beginning, we introduced the five BMD system components, followed by
further elaborations on the first component, i.e. on data acquisition.
Thereat, we reviewed the underwater data gathering tools and techniques, along
with the data aggregation protocols for more compactly representing the
gathered data. We also studied data fusion methodologies, conceived for the
fusion of data from distinct datasets into an integrated database, for the
sake of having more generally representative information. Our discussions in
this section were then continued by classifying BMD into 1D sensor signals, 2D
image data, and 3D video streams. For all these categories, we surveyed the
typical IoUT infrastructures and observatory systems, which freely offer their
BMD for researchers. The visibility, contrast, color distortion, and light
scattering in 2D underwater images as well as 3D video data gathering were
also addressed. Then, different distributed as well as cloud-based BMD
processing frameworks were introduced and a couple of essential BMD
applications were studied. These applications covered the oceanic GIS,
underwater localization, and marine life tracking, which all play essential
roles in IoUT contexts.

All of the tools and methods that have been discussed in this section,
including the sensory and imagery data sources, GIS-based ready-to-use data in
IoUT, and underwater object tracking as well as geo-tagging methods will
generate BMD, which has to be processed and analysed. In the following
section, we will focus our attention on big data processing techniques.
Thereby, we will review state-of-the-art technologies in data cleansing and
data processing using ML techniques for underwater applications.

##  IV Machine Learning for BMD Analytics

As mentioned in Section III, data processing is one of the five critical
system components in the chain of wealth-creation from data. To address the
commercial and industrial demands, ML and its Deep Learning (DL) variant
constitute promising solution. In this section, as visualized in Fig. 10, we
focus our attention on the use of ML strategies including the classical ML
approaches, as well as the more traditional and emerging NN-based learning
methods for BMD processing.

Figure 10: Conceptual categorization of the main topics in Section IV.

###  IV-A Machine Learning Techniques

Considering the large amount of data generated both in IoT and IoUT, there is
a growing need for powerful tools and techniques capable of distilling and
visualizing knowledge gleaned from data. These tools and techniques have
evolved over the past century! As a benefit, ML is readily available for the
analysis of BMD [33]. ML can be viewed as a collection of software algorithms
that empower intelligent machines to improve their performance on an
accomplishing pre-configured specific task. This particular task in turn, may
be categorized under descriptive, predictive, and prescriptive models.

The ML approaches may be divided into supervised and unsupervised methods. The
unsupervised methods are mainly used for data clustering, according to the
features embedded in the data itself. These methods do not rely on an expert
for labeling and for entering inputs. On the other hand, in supervised
learning, data will be labeled, prior to its exploitation, by an expert. In
this case, the resultant ML solution is referred to as a classifier, which is
then involved for the ensuing data classification phase.

Classification methods are devised for categorizing data, representing for
example a measurement from a sensor or a pixel in an image into one of the
legitimate predefined output classes. When there are only two possible answers
to a given question (i.e. yes or no), the classification problem is often
termed as detection. In cases of three or more legitimate output classes
associated with multiple choices, classification is also often referred to as
recognition. Methodologically, classification algorithms can be divided into
statistical methods (also known as conventional data analytics) and Neural
Networks (NN). The neural-based methods may themselves be divided into
traditional NNs and deep NNs [176].

All ML approaches are designed to assign a given input data ($X$) to a set of
predefined classes ($C_{i}$). This is a well-known classification/clustering
problem, which can be solved by either generative or discriminative models.
The generative models are those which solve this problem by using the joint
probability distribution function of [177],

$i=\operatorname*{arg\,max}_{C}{P(C)P(X|C)}\,.$ (17)

By contrast, the family of discriminative models uses the conditional
probability distribution function of [177],

$i=\operatorname*{arg\,max}_{C}{P(C|X)}\,.$ (18)

According to the Bayes’ theorem from a theoretical perspective, these methods
are identical. In practice however, it is usually easier to calculate
$P(C|X)$; especially when we have a large amount of data (i.e. $X$) for
training our model. The conventional data analytic methods tend to use
generative models, while most NNs rely on discriminative models [178].

In generative models, the behavior of both $C$ and $X$ should be known.
However, for discriminative models, we directly deal with the unknown $C$,
based on a given $X$. Discriminative models are generally simpler, faster, and
have less parameters to adjust. Accordingly, while the family of conventional
statistic approaches may be good enough for handling moderate-dimensional
situations, traditional ML and modern DL based approaches (with discriminative
models) are needed for processing big marine data problems (i.e. image, video,
and other sensory information generated within the IoUT infrastructure).

The concept of DL is built around the idea that artificial neurons are capable
of automatically extracting features and learn a pattern, provided that there
are enough hidden layers and unweighted neurons in their networks. Based on
this concept, deep networks have evolved from the traditional NNs by invoking
more than one hidden layer (technically, dozens of layers). Using these extra
layers, deep networks become capable of extracting features and reduce the
learning dimension as it will be introduced later in Section IV-D.

The growth rate of DL usage in all ML publications in the post-2010 era is
demonstrated in Fig. 11. This figure shows an astonishing factor 25 increase
in the percentage of publications in as few as four years. These publications
have substantially advanced the field by proposing new algorithms, networks,
and strategies for improving the performance of deep networks. Table XI
presents the most recognized supervised and unsupervised deep learning
networks. All the tandem arrows and circular nodes in this table
($\rightarrow$$\bigcirc$) are representative of an artificial neuron
processing the sum of weighted inputs and a subsequent activation function
(e.g. Sigmoid, $\tanh$, etc.). Please note that every NN has a bias input.
These inputs are deliberately eliminated in the shown diagrams, for
presentation simplicity.

Figure 11: Percentage of deep learning usage in all machine learning
publications, as searched between IEEE Explore as well as Web of Science
databases. TABLE XI: Deep Networks with Input, Hidden, Mid-output, and Output
Layer Neural Nodes to Evaluate Static, Dynamic, Sequential, or Hierarchical
Input Data Types [179, 180, 181, 182, 183] TABLE XI: (Continued)

Again, deep networks have several hidden layers. This is shown in Table XI by
utilizing a dash-dot line ($-$ $\boldsymbol{\cdot}$ $-$
$\boldsymbol{\cdot}$), whenever the network is capable of incorporating
multiple hidden layers. However, it should also be considered that, as these
networks grow deeper, they also require more training data. Further details on
various types of networks and on their advantages/disadvantages are provided
in Table XI and will be discussed throughout this section. However, we study
these networks mainly from an IoUT as well as BMD perspective and discuss what
specific network types and algorithms would be particularly suitable both for
BMD analysis and for IoUT applications. In doing so, we appraise many reported
use cases of ML in underwater applications in Section IV-F, but before that,
let us discuss the various software frameworks for architectural deep network
design, suitable for big marine data types.

###  IV-B Deep Learning Frameworks and Libraries

To facilitate the development of various DL architectures, such as those
listed in Tables XI and XVI for different applications, numerous software
frameworks and libraries have been developed for open access by the ever-
growing deep learning community [184]. Table XII lists and compares a range of
open-source DL tools and frameworks. Some of the abbreviations used in this
table are Deep Learning for Java (DL4j), Microsoft Cognitive Toolkit
(previously known as CNTK), and TensorFlow (TFlow).

TABLE XII: Comparing the Best Open-source Software Frameworks and Libraries
for Deep Learning

All the frameworks seen in Table XII are capable of operating on NVidia® CUDA-
supported GPUs relying on parallel processing. By contrast, OpenMP is another
shared-memory based multiprocessing programming interface, which is only
supported by some of the frameworks. As it will be discussed in Section IV-E,
shared-memory based methods are critically important for speeding up the deep
network’s training process.

So far we have discussed various architectures capable of processing BMD and
marine data. Once the data has been collected, some pre-processing steps have
to be performed for ensuring that the data used to train the ML algorithm is
clean and does not result in future learning problems. In the next section,
this important issue is discussed in more details.

###  IV-C Marine Data Cleaning for Machine Learning

Data preparation and cleansing is a critical step in almost every ML project
and it usually takes over half of the entire project duration to collect and
clean the data. When dealing with the unaccessible harsh underwater
environment of the IoUT, data cleansing becomes even more important. This is
because the underwater environment is affected by many environmental factors,
as shown in Fig. 4, which make it almost impossible to acquire clean data.
Additionally, data sources tend to be quite vulnerable to the hostile
underwater environment. Therefore, data preprocessing and cleaning, before
training is crucial for the success of any ML aided BMD processing in IoUT.
Below, various IoUT data types and their main preparatory considerations are
discussed.

####  IV-C1 Underwater sensor data cleaning

Sensors are inevitable parts of every IoUT subsystem and are used in nearly
all underwater exploration and surveillance applications [185]. They
continuously measure physical, chemical, as well as biological parameters and
generate a huge volume of data. Examples of these sensor and data are listed
in Table II.

Data cleansing for sensory data sources in underwater BMD applications is
typically concerned with missing values, contaminated measurements, and
detecting outliers. There are a variety of techniques for these preparatory
steps. To deal with a missing value in any sensor data, one can [186],

  * •

Delete the entire record;

  * •

Use a global constant;

  * •

Use a statistical value (e.g. mean or median);

  * •

Use an inter-class statistical value;

  * •

Use the most likelihood (i.e. more probable) value;

  * •

Use the most likelihood inter-class value.

Noise is another phenomenon affecting almost any sensory data. The process of
de-noising a sensory measurement is termed as noise cancellation. Although no
universally applicable de-noising methods exist, one should find the one
meeting the SNR criteria. The SNR is a very common parameter of characterizing
the performance of different noise cancellation methods. Some of these methods
include:

  * •

Using a low pass filter [187];

  * •

Using regression models [188];

  * •

Applying a data binning method [189];

  * •

Using wavelet methods in time-series [186].

Among these methods, low pass filter is the most common one and wavelet-based
methods are the most complex ones to implement. Additionally, data binning is
not particularly popular for sensory data, but it is widely used for images as
it will be discussed in the next section.

The final step in underwater sensory data cleansing is the outlier detection.
This step can be readily carried out by considering three well-known criteria,
named the IQR, T2, and Q criteria [186]. There are also some more advanced
methodologies like, probabilistic models, clustering methods, distance-based
detection, density-based detection, etc. [186].

####  IV-C2 Underwater image and video data cleaning and quality enhancement

As we have already discussed in Section II-B, underwater image acquisition
suffers from strong absorption and scattering. Additionally, there are two
other undersea signal degradation factors, namely chromatic aberrations [190,
191, 192] (i.e. color distortion or chromatic distortion) and noise imposed by
both natural and artificial light sources [193]. These destructive factors
significantly affect the quality of the captured video or image and should be
mitigated by a data cleansing algorithm.

Any image and video preparation algorithm has to assess the image quality
first. Automatic image quality evaluation algorithms assign an objective
metric, which is a weighted sum of the image colorfulness, contrast,
sharpness, etc. [194, 195]. This metric is then used as an objective function
to be maximized by other image enhancement procedures. Therefore, underwater
image and video quality assessment constitutes an essential step before image
retrieval, image quality optimization, video compression, and other visual
signal processing steps. It can also act as a guide for determining the data
bandwidth required by the underwater communication algorithms, as well as by
other decision-making processes [196].

After the automatic quality assessment of the underwater images and videos,
software-based image preparation algorithms are used for enhancing the image
quality. As a general rule, it is always cheaper to improve the image quality
in software, instead of implementing bespoke high-cost imaging devices.

In a comprehensive review paper by Han et al. [135], the image preparation
methods are divided into image dehazing and color enhancement. The authors
then introduce and compare numerous methods for each category. However, they
have missed the innovative method proposed by Ancuti et al. [197] that
addresses both image dehazing and image enhancement at the same time. This
method improves the global contrast that was degraded by light absorption,
increases the edge sharpness impaired by light scattering, and exposes the
dark regions with the aid of color balancing. The proposed method relies on a
single-image camera-independent technique that can be applied to both photos
and full-motion videos. Furthermore, it does not require any prior knowledge
about the environmental conditions.

####  IV-C3 Underwater image data binning

In addition to image enhancement, image data binning is another data cleansing
method that can be used in IoUT, which relies on grouping pixels into distinct
partitions based on the similarity of their characteristics [189]. The goal of
constructing these partitions is both to reduce the amount of noise and to
resolve data complexity. After binning, every pixel has a label to explicitly
indicate its parent partition, based on its membership index. Although image
binning is not an essential preparation step, it is recommended before
invoking feature extraction for reducing the computational complexity and to
speed up the ML process [189].

Image clustering methods, such as image binning, were developed much earlier
than the state-of-the-art ML and DL algorithms and date back to the age of
statistical mathematics. However, none of these methods are universally
accepted and it is still a challenging task to select the most appropriate
image partitioning method for a given application. A list of image clustering
methods, that are beneficial in underwater applications is provided in Table
XIII. These methods either tend to exploit image discontinuities (like edges)
or similar regions to partition a given image.

TABLE XIII: Image Clustering Methods and Algorithms in Underwater Applications Method (basis) | Algorithms | Reported Underwater Applications
---|---|---
Edge-detection
(based on the 1st and 2nd
order image derivatives)

|  $\bullet$ Watershed
$\bullet$ Snakes (Active Contour Model)
$\bullet$ Blob detection (i.e. Laplacian of the Gaussian,
.. Determinant of Hessian, Difference of
.. Gaussians, etc.) |  $\bullet$ Jellyfish Detection by Watershed and Snakes [198]
$\bullet$ Plankton Recognition by Snakes [199]
$\bullet$ Deep Sea Megafauna Recognition by blob [200]

Thresholding
(based on a threshold value
(clip-level) in feature space)
|  $\bullet$ Otsu’s Method (Maximum Variance)
$\bullet$ Expectation-Maximization
$\bullet$ Maximum Entropy Method
$\bullet$ Color or Intensity Histogram Thresholding |  $\bullet$ Fish Detection [201]
$\bullet$ Jellyfish Detection by Expect.-Maximization [198]
$\bullet$ Deep Sea Megafauna Recognition [202]
$\bullet$ Plankton Recognition [25, 203]
Distance- or Region-based
(based on the similarity in
attributes)

|  $\bullet$ Region Growing (RG)
$\bullet$ Simple Linear Iterative Clustering (SLIC)
$\bullet$ Region Splitting and Merging
$\bullet$ Morphological Spatial Pattern Analysis (MSPA)
$\bullet$ Statistical Clustering Methods (K-Means,
.. Subtractive Clustering, AGNES, DBSCAN)
$\bullet$ Neural Clustering Methods (Self-Organizing
.. Map (SOM), Hierarchical SOM, Hierarchically
.. Growing Hyperbolic SOM)
$\bullet$ Fuzzy Clustering Method (FCM)
$\bullet$ Wavelet Transform
$\bullet$ Compression (Texture and Boundary) |  $\bullet$ Shrimp Detection by SLIC [23]
$\bullet$ Plant Detection by Gabor Wavelet [20]
$\bullet$ Plant Recognition by Gabor Wavelet [153]
$\bullet$ Plankton Recognition by MSPA [24, 204]
$\bullet$ Coral Detection by Clustering [205]
$\bullet$ Object Detection by SLIC [18]
$\bullet$ Object Detection by Gabor Wavelet [17]
$\bullet$ Object Detection by Clustering [19]
$\bullet$ General Underwater Clustering by FCM [206]
$\bullet$ General Underwater Clustering by RG [207]
$\bullet$ Coral Recognition by MSPA [134]
$\bullet$ Coral Recognition by RG [208]
Math Calculus
(based on solving a Differential or Integral Equation) |  $\bullet$ Level Set Model
$\bullet$ Mumford Shah Model
$\bullet$ Chan-Vese Model |  To the best of authors’ knowledge, this method has never been used in underwater applications.

Graph Cut
(based on the undirected graph partitioning (i.e. Markov Random Fields) to
model the impact of pixel neighborhoods)
|  $\bullet$ Normalized Cuts
$\bullet$ Lazy Random Walks
$\bullet$ Isoperimetric Partitioning
$\bullet$ Minimum Spanning Tree-based
$\bullet$ GrabCut
$\bullet$ Model Based |  $\bullet$ Fish Recognition by GrabCut [209, 210]

Video Motion
(based on the object movement)
|  $\bullet$ Subtracting a Pair of Images

|  $\bullet$ Fish Detection [211, 212]
$\bullet$ Fish Recognition [139]
$\bullet$ Lobster Detection [213]

In addition to the algorithms introduced in Table XIII, there are many others
that have however been more rarely used in the literature. These methods are
reviewed by Flake et al. [214] as well as Wang et al. [189] and might be worth
investigating in future studies. Some of them are the Minimum Cut, Mean-Shift,
Turbo-Pixels, Lattice Cut, Compact Super-Pixels, Constant Intensity Super-
Pixels, Entropy Rate Super-Pixels, Homogeneous Super-Pixels, Topology
Preserved Regular Super-Pixel, SEEDS, VCells, Depth-Adaptive Super-Pixels,
Voxel Cloud Connectivity Segmentation, Structure Sensitive Super-Pixels,
Saliency-based Super-Pixel, and Linear Spectral Clustering.

In addition to data cleansing, feature extraction is another essential step
for any ML applications, which rely on statistical methods and traditional
NNs. This substantial step in ML is discussed in the next section.

###  IV-D Feature Extraction for Marine Data Analytics

Feature extraction is a mathematical step, in which raw data is replaced by
its numerical descriptors. This step is responsible for transforming large
vectors of sensory data and large matrices of image data into their low-
dimensional vector-based representatives. It is usually the most intricate
part of almost any ML-aided computer vision problem and the solution should be
tailored for the particular needs of the specific project at hand. Although
all of the modern deep learning algorithms extract their own feature vectors
automatically, these deep networks are hard to train, when relying on limited
training datasets.

Finding a series of useful features is even harder in underwater image
processing applications in high-dynamic environments subject to non-uniform
light illumination, variable scene brightness, and degraded colors. Table XIV
lists a number of salient descriptor routinely used in underwater
applications. These features are categorized into four classes, including
color, texture, shape (boundary), and other descriptors. The papers listed in
Table XIV tend to use a series of descriptors, depending on their specific
target applications. In one case, the number of descriptors used has been as
high as 66 features in the Fish Recognition project carried out by Huang et
al. [210].

TABLE XIV: Data Processing and Machine Learning Feature-sets in Published Underwater Applications Color Descriptors | Underwater Applications
---|---
$\bullet$ Image statistics (e.g. Hu’s 7 invariant
.. moments, affine moment invariants,
.. Skewness, kurtosis, mean) |  [23, 24, 25, 208, 210, 139, 204, 215, 150]

$\bullet$ Segment contrast |  [23]
$\bullet$ Histogram descriptor
|  [19, 23, 134, 200, 209, 210, 216]
$\bullet$ Five MPEG7 color features |  [200, 202]
$\bullet$ Transparency ratio |  [203]
Texture Descriptors |
$\bullet$ Three MPEG7 texture features |  [200, 202]
$\bullet$ Wavelet transform |  [205]
$\bullet$ Gabor filter
|  [20, 134, 200, 153, 210, 139, 216]
$\bullet$ Filter banks (e.g. Schmid, maximum
.. response, Leung and Malik, root
.. filter set)
|  To the best of the authors’ knowledge, these descriptors have never been
used in underwater applications.
$\bullet$ Grey level co-occurrence matrices
.. properties (contrast, correlation, energy,
.. and homogeneity) |  [22, 24, 25, 134, 210, 139, 204, 215, 217]

Shape Descriptors and Key Points |
$\bullet$ Hough transform |  [27]
$\bullet$ HOG (Histograms of Oriented Gradients)
.. based (e.g. SIFT©, SURF©, GLOH) |  [17, 200, 209, 210, 218]

$\bullet$ Binary descriptors |  [15, 16, 134, 219, 220]
$\bullet$ Weber local descriptor |  [221]
$\bullet$ Convexity |  [24, 203, 150]
$\bullet$ Fourier descriptors |  [24, 25, 210, 139, 204]
$\bullet$ Frequency domain descriptors like phase
.. congruency |  [221]

$\bullet$ Curvature scale space transform |  [139]
Other Descriptors |
$\bullet$ Granulometrics (size, area, and orientation)
.. to recognize an already detected object |  [22, 24, 25, 140, 203, 210, 204]
$\bullet$ Eigenvalues and covariance matrix
.. to recognize an already detected object |  [203]

$\bullet$ Motion related |  [22]
$\bullet$ 3D shape, surfaces, and texture descriptors |  [153]

After selecting a number of features, it is recommended to mathematically
evaluate their correlation and then reduce the number of features to the
number of truly independent descriptors. This process is termed as data
dimensionality reduction and can be performed by feature reduction techniques,
feature selection, and feature aggregation [222].

Some of the popular dimensionality reduction techniques found in literature
are Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA),
independent component analysis, non-negative matrix factorization, Self-
Organizing Map (SOM), sequential forward search, sequential backward search,
bag of words, etc. [222, 223, 224, 225, 226].

So far in this section, the software components suitable for ML and DL aided
IoUT have been studied. However, these ML-aided solutions also rely on
appropriate high-performance hardware platforms.

###  IV-E Hardware Platforms for ML in IoUT

The implementation of any ML solution, from its data cleansing and algorithm
development to its final deployment, may rely on a variety of hardware
platforms. These hardware platforms typically boost the overall throughput by
parallel processing. These processing methods can be broadly divided into the
following two main categories:

  * •

Shared-memory multi-processors: They rely on multiple processors that all
share a memory unit [227] and the ML algorithm resides within this memory
unit. Some of the more popular shared-memory methods are based on Application-
Specific Integrated Circuits (ASIC) [228], Field-Programmable Gate Arrays
(FPGA) [229], multi-purpose and multi-core CPUs, and Graphics Processing Units
(GPU) [230].

  * •

System of networked processors: As already introduced in Section III-C, a
Distributed Computing System (DCS) is a system of networked processors, which
coordinates the processors’ actions by passing messages to each other [231,
154]. They can be used to parallelize ML algorithms in three different ways,
including:

    * –

Data parallelization: This is performed by running the same ML algorithm in
all of the distributed computers and then dividing the data between them.
Every computer estimates all parameters based on a separate dataset, before
they exchange their estimates for formulating a final estimate. MapReduce is
one of the most popular data parallelization methods [232, 233].

    * –

Model parallelization: In this paradigm, the ML algorithm will have to be
decomposed into different functions and operands. The algorithmic parts are
then shared among multiple computers and every part has its own specific set
of parameters. The input data, however, will be simultaneously fed to all of
them, where every computer is responsible for estimating the set of parameters
assigned. During the parameter training process, those parallelized computers
exchange their partial error vectors back and forth to meet the convergence
criteria and to come up with the final parameter estimations [234, 235].

    * –

Pipelined parallelization: In this method, the algorithm is shared among
distributed computers, similar to the above-mentioned model parallelization
method. These parallelized algorithm parts are chained together from one input
block to one (or more) output blocks, termed as pipelines. In contrast to
model parallelization, data will also split into a series of records. At the
beginning, the first record is passed to the first computer in the DCS, to
carry out its own task (i.e. local parameter adjustment). The output of this
computer is then relayed to the next computer to carry out its own task over
the first data record. Meanwhile, the first computer starts processing the
second data record. This pipe-lined process continues until the last record in
the database exits the DCS and consequently, the local parameters of the last
block in the pipeline are updated [236].

A rudimentary qualitative comparison of different parallel processing hardware
platforms used for ML-aided and BMD processing is offered in Table XV. Here,
the strength of each platform in terms of a specific criterion is indicated by
the $+$ and $-$ signs, where more $+$s represents better performance in the
context of that specific benchmark. The last two rows of this table summarize
the entire table with respect to the associated research or industrial
applications. For instance, as the Table shows, the CPUs require the lowest
design time and impose the lowest design complexity, but they suffer from the
lowest operational speed in all design and application phases, i.e. data
cleansing, NN training, and final inference. Therefore, they may be used for
research, but they are less suitable for industrial applications.

TABLE XV: Comparing Different Parallel Processing Hardware Platforms to Carry
out a Typical Machine Learning Algorithm

Criteria | ASIC | FPGA | CPU | GPU | DCS
---|---|---|---|---|---
Inference Speed | $++$ | $++$ | $--$ | $+$ | $-$
Product Unit Cost | $++$ | $+$ | $+$ | $-$ | $--$
Design Complexity | $--$ | $-$ | $++$ | $+$ | $-$
Occupied Space | $++$ | $++$ | $-$ | $+^{a}$ | $--$
Research Friendly | $--$ | $-$ | $+$ | $++$ | $+^{b}$
Market Friendly | $++$ | $+$ | $--$ | $-$ | $+^{c}$

a New GPU platforms such as NVIDIA Jetson occupy small spaces.

${}^{b,\,c}$ Cloud-based DCS platforms such as AWS EMR are readily available.

Perhaps the most prominent choice for typical scientific application would be
the GPU [237]. The hardware and software resources required for GPUs are
affordable and their speed is high enough to cover almost any application.
Both the research-based as well as the industrial-scale employment of GPUs in
underwater data processing and ML applications have been frequently reported
in the literature. These applications range from obstacle detection and
collision avoidance, to image-based SLAM localization, and even further to
underwater object detection (e.g., coral detection) [238, 230].

However, the major problem with GPUs is their dependence on a bulky host
computer. This has limited their implementation in low-power and lightweight
IoUT platforms as well as underwater vehicles. To address this problem, new
compact GPU designs have emerged to offer AI-ready computing resources. For
example, NVIDIA Jetson is a standalone GPU-accelerated embedded system, which
has a small volume. The high performance, low power, and compact form-factor
of the Jetson family makes them ideal for example for deep learning aided
computer vision applications [238].

Since the Jetson embedded systems are empowered by the Linux Ubuntu operating
system, they are eminently suitable for deep NN frameworks of Table XII, e.g.,
TensorFlow. Thus, the low-latency inference capability of machine learning
algorithms can be readily realized in underwater vehicles and platforms [230].

Additionally, NVIDIA’s Compute Unified Device Architecture (CUDA) library
unleashes the GPU’s parallel processing capabilities for applications other
than machine vision. As previously mentioned in Section IV-B, CUDA is an API
model that allows engineers to use a CUDA-enabled GPU for general purpose
processing, this is also referred to in parlance as a General Purpose GPU
(GPGPU). For instance, Pallayil et al. [239] have used a GPGPU for performing
high-complexity real-time beamforming in their underwater acoustic phased
array. They simply used the popular C Language in the Linux environment to
harness the fast Fourier transform in the CUDA library, to implement their
underwater frequency-domain beamformer.

ASICs and FPGAs are listed in Table XV, which have the lowest form-factor.
They also offer a high throughput and high power efficiency, which makes them
eminently suitable for industry-scale IoUT projects [240]. Given these
compelling attributes, FPGAs are repeatedly featured in real-time and low-
power underwater applications. For example, Karabchevsky et al. [240] have
implemented a standalone FPGA architecture for noise suppression in underwater
sonars. Their proposed signal processing implementation is claimed to overcome
the sonar-based underwater visibility problems.

Following the above survey of various data processing methods, platforms, and
their hardware realizations, diverse ML algorithms used for underwater
applications will be discussed in the next section.

###  IV-F ML Techniques in BMD Applications

The advantages of ML techniques make them eminently suitable for most
underwater applications. The categorical investigation of these techniques as
well as their implementation in BMD applications will be carried out in this
section.

####  IV-F1 Deep NNs for static IoUT data

Static data is exemplified by still images in contrast to full-motion video
clips. In the context of Table XI, the fully-connected, Deep Belief Network
(DBN) [179], CNN [180], Autoencoder [181], and Recursive Neural Tensor Network
(RNTN) [182] networks are eminently suitable for static marine data
processing. In the case of RNTNs, they are usually fed by a single static
underwater image in every single data entry step. This image consists of
multiple segments, and RNTN is supposed to determine the hierarchy of every
segment inside the given image (e.g. background, coral, plant, fish, etc.).
The double-colored nodes of RNTN in Table XI are hidden neurons that can also
be designated by the network designer to be an output neuron, representing
those expected hierarchical segments.

Among the static data processing networks seen in Table XI, Autoencoders are
the only unsupervised clustering NNs. Every Autoencoder consists of two parts,
which can be used either separately or simultaneously. The first part
processes a static input data such as an underwater image and extracts its
features. The second part on the other hand, takes the feature vector and
tries to reconstruct the input image again. If the inherent image features
were adequately captured, the reconstructed input data appearing at the output
will be similar to the input data itself. From an application-oriented
perspective, one can use the first part of the Autoencoder for dimensionality
reduction and data compression, while the second part is suitable for data
decompression. Meanwhile, by using both parts simultaneously together,
Autoencoder will act like a noise reducing NN.

Another network architecture in Table XI is the family of DBNs. These networks
consist of consecutive shallow learning Restricted Boltzmann Machine (RBM)
pairs, which gradually reduces the dimension of input data from the top-level
of the entire search/classification towards the final unique classifier. Any
mid-layer in DBN may act as output layer of the feature extraction. Again, by
continuously decreasing the feature vector size, the procedure culminates by
unambiguously classifying the input data. Therefore, DBN can act as an
unsupervised clustering NN, provided that it is terminated somewhere at a mid-
layer. By contrast, it can also act as a supervised classifier, if the number
of nodes is reduced to the specific number of classes and if labeled data is
used to train the network.

The final deep network for static data that will be discussed here is the
convolutional neural network. This network relies on multiple stages of
convolution and pooling, as seen in Table XI. It is considered to be the best
deep classification method, especially when dealing with static images or
previously recorded outputs of sensors and of hydrophones. Several
architectures of this popular network have been designed for scientific use,
some of which are also supported by Goggle and Microsoft (as exemplified by
Inception [241] and ResNet [242]). Several of these CNN architectures are
compared in Table XVI, based on the excellent review article by Canziani et
al. [243]. The Top-1 accuracy222Classifiers usually assign a probability value
to all of their output classes. Thereafter, the class with the highest
probability (top-1) will be considered as the final answer, which is not
always true. Sometimes the correct answer is among the top-N classes. Using
top-5 accuracy is common, when comparing different classifiers. listed in this
table is measured with the aid of the single central-crop sampling
technique333A given image may have four corner crops and one central crop. of
[244] for all of the networks.

TABLE XVI: Comparison of a selected number of CNN architectures submitted to the Annual ImageNet Challenge [243, 244] Architecture Name |  Top-1 Accuracy |  No. of Parameters |  No. of Operations
---|---|---|---
AlexNet [245] | $>56\%$ |  $\approx$ 50 M |  $\approx$ 2 G
NIN [246] | $>62\%$ |  $\approx$ 6 M |  $\approx$ 3 G
VGGNet-19 [247] | $>70\%$ |  $\approx$ 150 M |  $\approx$ 39 G
Inception-v4 [241] |  $>$ 80% |  $\approx$ 35 M |  $\approx$ 18 G
ResNet-152 [242] |  $>$ 77% |  $\approx$ 50 M |  $\approx$ 23 G
ENet [248] | $>67\%$ |  $\approx$ 5 M |  $\approx$ 2 G

The number of network parameters in Table XVI is proportional both to their
memory footprint and to their required training time. On the other hand, the
number of operations required for a single forward pass, as shown in the table
is capable of indicating the overall inference speed of the network. The lower
the number of operations, the higher the inference speed. Here, ENet is not a
CNN, but it is directly inspired by CNNs. By the same token, Network in
Network (NIN) is not a CNN, but it relies on convolution operations. Some of
these architectures are also included in the deep learning libraries of
Section IV-B and can be readily involved for any project. Let us now extend
our discussions to cover deep NNs in dynamic systems.

####  IV-F2 Deep NNs for dynamic IoUT data

Recurrent Neural Network (RNN) [249, 183] and its variants (i.e. long short-
term memory, gated recurrent unit, etc.) constitute the only deep NN
architecture in Table XI that can be used for nonlinear dynamic systems, as
exemplified by continuous underwater sensor outputs. RNNs are capable of using
both time-series and sequential data streams to construct supervised
classifiers.

RNNs constitute a deep version of Hidden Markov Models (HMMs) [249], which
represent a simple form of the broader family of dynamic Bayesian networks.
Markov models (i.e. Markov chains) are stochastic models, in which the output
of a NN in response to the current state, only depends on the output of
certain selected neurons in the previous state. The values of all neurons in
all states are visible and are considered as the outputs.

Hidden Markov models are similar to Markov models, but they rely on non-
observable or hidden neurons. In these networks, the visible output is
directly calculated by applying a nonlinear function to the output of the
hidden neurons. As illustrated in Table XI, an RNN is constructed by stacking
HMMs on top of each other. The rectangular shape of the nodes in this picture
indicates that in contrast to the circles of the other networks, these nodes
are not constituted by a single neuron, but rather they are a combination of
neurons in the form of a HMM. After studying the DNNs suitable for static and
dynamic data analysis, let us study their capabilities to solve real-life
problems in marine environment.

####  IV-F3 ML solutions in underwater image applications

The concepts of image and video quality assessment as well as image
restoration were studied previously in Section IV-C2. We have also stated the
fact that underwater imaging applications heavily rely on image enhancement
algorithms to cope with the destructive effects of visible-light environments.
To address this problem, deep learning techniques have been employed for
enhancing images suffering from noise, absorption, scattering, and color
distortion effects [190, 250, 251]. Furthermore, the benefits of DNNs in
underwater image quality assessment can be assessed in future studies with
reference to their overwater counterparts [252]. These applications, while in
their infancy, are rapidly expanding.

Other applications that can benefit from deep learning are underwater image
clustering and binning. We briefly reviewed these concepts in Section IV-C3,
noting that some underwater clustering algorithms are also introduced in Table
XIII. By the way, using novel deep learning techniques in underwater image
clustering has not as yet attracted the attention it deserves. For instance,
one can beneficially exploit the embedded image clustering technique of [253]
as an unsupervised DNN methodology in underwater image segmentation.

Additionally, using ML techniques in underwater object recognition have been
previously used for various underwater applications and have shown different
performance. Here, we provide a summary of these techniques applied to some
common underwater object of interest recognition and compare their
performance. In order to have a fair comparison, the Average Precision (AP)
parameter, which is a widely accepted performance measure, can be used. The
average precision in any statistical or ML-based classifier can be calculated
as [14],

$AP\overset{{\Delta}}{=}\dfrac{1}{N_{C}}\sum_{i=1}^{N_{C}}\frac{{TP}_{i}}{{TP}_{i}+{FP}_{i}}\,,$
(19)

where the True Positive (TP) and False Positive (FP) values are calculated for
$N_{C}$ number of classes. The AP parameter of many ML techniques published in
the underwater plant, fish, and coral recognition literature are compared in
Table XVII. All the algorithms referenced in this table are based on the well-
known Fish4Knowledge and EILAT datasets introduced in Table IX and are also
based on the video footages recorded by AUV and ROV vehicles.

TABLE XVII: Comparing the Precision of Multiple Machine Learning Techniques to
Recognize Three Different Underwater Species

Application | Fish Recognition |
---|---|---
Dataset | Fish4Knowledge Dataset [14, 15, 209, 210, 254] |
Methods |  ..Regression Tree |  ..LeNet |  ..Linear Discriminant
..Analysis and SVM |  ..Flat SVM |  ..BEOTR |  ..Decision Tree
..and SVM |  ..Softmax |  ..K-Nearest
..Neighbors |  ..DeepFish |  ..GoogLeNet |  ..VGGNet |  ..DeepFish and SVM |  ..AlexNet |
Average Precision (%) |  75.4 |  76.8 |  80.1 |  82.9 |  84.5 |  84.6 |  87.6 |  89.8 |  90.1 |  93.9 |  96.6 |  98.2 |  99.7 |
|  |  |  |  |  |  |  |  |  |  |  |  |  |

Application | Plant Recognition |  | Coral Recognition |
---|---|---|---|---
Dataset | ROV and AUV Video Footages [215, 219] |  | EILAT Dataset [134, 255] |
Methods |  ..Self-Organizing
..Map (SOM) |  ..Learning Vector
..Quantization |  ..Decision Tree |  ..K-Nearest
..Neighbors |  ..Random Forest |  ..Fully-Connected |  ..SVM |  |  ..Decision Tree |  ..Bayes |  ..K-Nearest
..Neighbors |  ..Random Forest |  ..Fully-Connected |  ..SVM |
Average Precision (%) |  93.8 |  94.1 |  95.1 |  95.3 |  95.8 |  96.1 |  96.1 |  |  69.6 |  82.1 |  83.3 |  84.6 |  85.2 |  90.8 |
|  |  |  |  |  |  |  |  |  |  |  |  |  |  |

It is quite common in ML to cascade different algorithms into a single method
to attain an improved performance (e.g. decision tree with SVM, and CNN with
fully-connected). As an instance of algorithm merging in underwater
applications, Faillettaz et al. [25] as well as Hu and Davis [204] cascaded a
fully-connected classifier over shape-based feature-sets with an SVM
classifier over texture-based feature-sets. By combining the results from
these two classifiers, they claimed to have achieved an improved average
precision, as defined in (19).

Another noteworthy cascaded solution was conceived by Schoening et al. [202],
where they employ multiple cascaded SVM binary detectors to construct a deep
sea megafauna recognizer. The binary SVM detectors have object-specific
operation, which makes them more accurate. Accordingly, by combining these
binary detectors, high precision multi-object recognition was achieved.
However, combining those object-specific binary detectors requires more
hardware resources than a single classifier.

Finally, it is worth mentioning that underwater imaging applications are not
limited at all to the visible light domain. For example, undersea sonar
imagery can provide high-resolution images of the seabed, even in turbid water
with low visibility. Some of the reported applications of sonar images include
ocean mapping, mine-countermeasures, oil prospecting, and underwater search
and rescue (e.g. finding the drowned corpses, wrecks, and airplanes) [256]. In
this regard, relying on human operators in sonar-based underwater object
recognition applications is not recommended, since they will experience
fatigue by staring at the display screen, and they might consequently miss the
object of interest. Therefore, intelligent image recognition methods can be
trained to replace human operators, especially in long-duration search
scenarios.

To highlight the accuracy of machine learning techniques in underwater sonar-
based object recognition, the AP metric of (19) is used here in Table XVIII.
This table compares the performance of deep NNs to that of several statistical
methods as well as to that of traditional NNs. While the statistical and
traditional networks seen in this table are trained using HOG features from
Table XIV, CNN deep networks do not need any feature extraction based
preprocessing. Additionally, as seen in this table, appropriately designed and
well-trained DNNs may outperform traditional models.

TABLE XVIII: Comparing the Precision of Multiple Machine Learning Techniques to Recognize Objects in Sonar Imagery Application | Object Recognition |
---|---|---
Dataset | Echoscope Sonar Image Dataset [257] |
Methods |  ..K-Nearest
..Neighbors |  ..Multilayer
..Perceptron |  ..SVM |  ..CNN
..(AlexNet) |  ..CNN
..(LeNet) |
Average Precision (%) |  72.0 |  89.3 |  92.7 |  94.1 |  97.0 |
|  |  |  |  |  |

####  IV-F4 ML solutions in underwater video applications

As already discussed in Section IV-E, the capability of hardware platforms to
train and analyze DNNs was impressively improved recently. This improvement
has attracted increasing attention to the subject area of real-time IoUT video
applications. It was also pointed out in Section III-B3 that the rapid growth
in underwater video data volumes will require the development of automatic
video processing, which can be carried out by machine learning techniques.
These automatic solutions will be used in a variety of underwater video data
processing applications such as:

  * •

Visible light video tracking: This application is designed for scanning video
sequences to follow a specific element of interest [152]. A pair of basic
problems in a video object tracking solution is how to predict the location of
a moving element in the next frame and how to detect the element within this
predicted region. Both of these be handled by deep NNs [140]. This application
was also discussed in more detail in Section III-D.

  * •

Sonar video tracking: Sonar videos are recorded with the aid of acoustic
waves, instead of visible light. Within turbid undersea environments, sonar
systems offer a significant advantage by supporting long-range and low data-
rate imaging [141]. Similar to visible light video object tracking, sonar-
based video object tracking can also be carried out with the aid of deep NNs
[258].

  * •

Underwater photo mosaicing: This is the act of combining separate visible
light or sonar video frames, for capturing a wider perspective of the region
of interest. Machine learning based photo mosaicing of underwater images is
now routinely performed by state-of-the-art underwater vehicles both for
exploration and for navigation [20, 29, 259]. Automatic localization and
positioning of submersible vehicles with the aid of photo mosaicing is termed
as SLAM [174] and it is studied in Section III-D.

  * •

Marine life studies: The analysis of underwater species is an indispensable
part of any observatory video system. Thanks to the wide availability of
machine learning based data processing toolsets, marine biologists are now
capable of analyzing the high-volume video data captured for extracting the
desired information. Scientific studies based on marine life video data
applications have been published in different areas, including underwater
species behavior understanding (ethology) [139, 213, 260], abundance and
counting [150, 199, 22, 261], size measurement [150], detection and
recognition [15, 199, 210, 14, 212, 261], and tracking [198, 199, 140, 21]. We
have covered some of these aspects of marine life analysis in Section III-D.

####  IV-F5 ML solutions in underwater sensor applications

Despite the unique benefits of applying RNNs to the processing of sensor
outputs and in nonlinear dynamic systems control, there is a paucity of
reported use cases in real-world IoUT applications. Their underwater
applications are limited to a few scenarios, such as IoUT sensor data
forecasting [262], underwater vehicle sensor read and fault diagnosis [263],
and the dynamic control of underwater movements [264].

Among these applications, RNN-based predictive models conceived for IoUT
sensory data forecasting as well as for missed sensory data implantation are
better investigated [265]. More specifically, sea water temperature and
salinity predictions are claimed to be important, because:

  * •

Water temperature and salinity have a direct effect on the acoustic
communications between IoUT nodes [266],

  * •

Oceanic temperature has a substantial impact on both the land and the marine
ecosystems by regulating the global climate [262].

Hence, the employment of machine learning techniques to design accurate
predictive models is promising in BMD. For example, the influence of Long
Short-Term Memory (LSTM) [267] and Gated Recurrent Unit (GRU) [268], as a pair
of common variants of RNNs on highly accurate water temperature prediction is
presented in Table XIX. Even though sea surface temperature forecasting is
challenging due to the influence of numerous complex and nonlinear
thermodynamic factors, data-driven DNNs are capable of learning these dynamic
behaviors. By comparing RNN-powered networks to traditional machine learning
solutions in Table XIX, the efficiency benefit of DNNs in IoUT sensor data
prediction becomes explicit.

TABLE XIX: Comparing the Mean Squared Error of ML Techniques in Sea Surface Temperature Prediction at Philippine Sea Application | Sensor Timeseries Forecasting |
---|---|---
Dataset |  NOAA Dataset in Table VIII |
Methods |  SVM [267] |  CFCC-LSTM [267] |  SVR [268] |  FC-LSTM [268] |  GRU-ED [268] |
One Day Forecasting |  0.434 |  0.166 |  0.095 |  0.061 |  0.063 |
One Week Forecasting |  N/A |  N/A |  0.214 |  0.168 |  0.162 |
One Month Forecasting |  1.478 |  1.145 |  0.212 |  0.343 |  0.207 |
|  |  |  |  |  |

To recap, the current usage of RNN and its variants in IoUT applications are
rare. Nonetheless, their versatile applications in IoT in smart homes [269],
smart cities [270], weather forecasting [271], and other areas promise a
similar growth for IoUT applications in the near future.

###  IV-G Section Summary

In this section, the employment of ML-based techniques in BMD processing was
studied. We started this section by the definition of machine learning,
classifying it into: classic statistical methods, traditional neural networks,
and modern (i.e. deep) neural networks. The associated chronological
perspective was also presented. Then, we discussed NNs by surveying the major
deep networks in the literature and their potential applications both in
static and dynamic underwater data processing. Despite the rapid development
of sophisticated, but complex deep NNs, light-weight traditional NNs might
remain typical in limited-complexity underwater applications. Therefore, more
detailed discussions were provided in this section to cover data cleaning and
feature extraction techniques in BMD. We then surveyed the available software
frameworks and hardware platforms, including a collection of freely available
libraries and frameworks. We also provided a comparison of the hardware
infrastructures suitable for the software products discussed. Finally, the
average precision and accuracy of diverse machine learning approaches suitable
for underwater applications was studied.

##  V Challenges and Future Directions in IoUT and BMD

Having reviewed the state-of-the-art research in the areas of IoUT, BMD, and
machine intelligence, the challenges and opportunities in these growing fields
will be discussed. We also propose solutions and future research directions to
address the challenges and to pursue the opportunities.

One of the main obstacles that has been hindering further advances in the IoUT
domain is that well-known terrestrial technologies, which perform well in the
IoT domain, tend to be unsuitable in underwater applications. Many issues in
the oversea application domain can be readily solved, easily, while they pose
a significant challenge in underwater scenarios. Below we will continue with a
list of challenges, opportunities, and future trends in the IoUT, BMD, and ML
fields. Some of these challenges may be mitigated with the aid of big data
processing and analytics, while some others require research efforts from the
broader engineering community, hardware vendors, and policy makers.

###  V-A Underwater Network Management System

Due to the significant growth in the number of Internet-connected underwater
devices, the IoUT infrastructure tends to exhibit increased complexity.
Consequently, improved Network Management Systems (NMS) are required, which
represent the process of monitoring and controlling every aspects of the
underlying network, for ensuring its seamless operation [272]. The monitoring
must be automatic and prompt in locating, measuring, and reporting faults.
Additionally, their control should be capable of efficient and reliable
resource allocation or troubleshooting [38].

While the concepts of network management in IoUT are somewhat similar to those
in IoT, the methodologies are different, as discussed in Section II-E owing to
the differences between overwater and underwater communication channels.
Hence, the extension of NMS in IoT to Underwater NMS (U-NMS) in IoUT requires
further research [273] for each of the six aspects of U-NMS in the FCAPSC
model, namely the Fault-, Configuration-, Account-, Performance-, Security-,
and Constraint-managements [273]. It is worth mentioning that FCAPSC of IoUT
was derived from the original five elements in FCAPS of IoT, which was
introduced by the International Organization for Standardization (ISO) in the
1990s. The constraint-management element requires U-NMS to deliver continuous
connectivity even in the face of the hostile underwater channel, node
mobility, device fragility, environmental dynamics, and technological
heterogeneity.

To facilitate the implementation of the FCAPSC model, U-NMS protocols tend to
divide their influence domains into the family of network functionalities
(e.g. routing management, protocol assignment, security checks, etc.) and of
device operations (e.g. UWSN maintenance, energy conservation, device
positioning, time synchronization, etc.) [38]. The relevant studies around
these domains were comprehensively surveyed in Section II. To the best of our
knowledge, the Underwater Simple Network Management Protocol (U-SNMP) is the
only U-NMS protocol, which covers the first domain of influence (i.e. the
network functionalities). U-SNMP in IoUT is again, an extension of SNMP in
IoT, and it is a manager–agent-based protocol, which is used in communication
between devices [273]. On the other hand, Lightweight Machine to Machine
(LWM2M) is an IoT protocol that covers the second domain of influence (i.e.
the device operations). LWM2M is applicable to IoUT as well, subject to some
modest adjustments [38].

Despite its popularity, U-SNMP lacks facilities for network configuration, and
LWM2M suffers from challenges owing to the associated heterogeneous network
support [38]. By contrast, there are other network management protocols in IoT
that offer better performance. Future studies might consider appropriately
adopting those overwater protocols to underwater applications. For example,
the Common Management Information Protocol (CMIP) offers better security
features and it is suitable for wide area networks [274]. Another alternative
might be the LoWPAN Network Management Protocol (LNMP) in IoT, which has low
data rates, low power consumption, low cost, and supports flexible topologies
[275].

While the above protocols are better suited for distributed network
managements, the centralized SDN management techniques of Section II-I are
also in need for underwater management systems. Here, the Open vSwitch
Database (OVSDB) protocol at overwater SDNs constitutes a promising base for
designing its underwater counterpart. However, any attempt to adopt OVSDB to
underwater SDNs requires further research for addressing its current security
issues [276].

Even after dividing the U-NMS responsibilities into two major domains of
influence and limiting the U-NMS protocols correspondingly, the number of
challenges in each domain will remain significant [38]. This is a direct
consequence of the broad nature of the U-NMS topic itself. To better
understand these challenges and study their future directions, we have
discussed each challenge in a dedicated subsection. The following subsections
will cover multiple aspects of U-NMS, ranging from energy conservation to
device maintenance, to its security issues and communications. Some of these
subsections discuss using BMD analytics as well as powerful deep learning
techniques.

###  V-B Energy Conservation and Harvesting in IoUT Devices

Energy conservation and harvesting in IoUT devices are of prime concern in
almost all underwater applications, while they can be readily addressed in
overwater scenarios, where energy can be harvested from the sun and preserved
in the system [277]. Some innovative methods of gleaning energy in underwater
environments include:

  * •

Overwater solar energy: As already discussed in Section II-J, by performing
computations in above-water edge-devices, such as surface-floated buoys and
vehicles, solar energy can be harvested. Indeed, solar powered buoys are
amongst the oldest methods of environmental energy harvesting techniques [278,
279].

  * •

Underwater solar energy: It was stated in Section II-B that light is strongly
absorbed in water and it additionally suffers from color distortion. However,
as mentioned in Section III-B2, blue light will penetrate water deeper than
other visible light frequencies. Some previous studies suggested the use of
solar cells to harvest the blue frequency band of the solar energy underwater.
These photovoltaics are claimed to provide useful power at the depth of more
than $9\,m$ [280]. However, many underwater applications are invoked at depths
well beyond the light penetration domain.

  * •

Tidal-wave energy: Harvesting kinetic energy from waves in the littoral tidal
basin can be readily achieved by using today’s technology. This can also offer
a source of energy in the underwater benthic zone. These systems are based on
piezoelectric elements and the energy generated is high enough to power UWSNs
and their devices [281]. Similarly, some contributions report the employment
of the same technique to harvest energy from fluid-flow in pipelines [282].

  * •

Wireless energy transfer: This method was introduced for RFIDs and acoustic
tags in Section III-D3. Additionally, a remotely powered acoustic UWSN was
reported in [283] as another energy acquisition alternative for the IoUT.
Thereby, sensor nodes harvest the mechanical wave power supplied by an
external acoustic source. Another use case of wireless energy transfer may
rely on in-situ magnetic charging stations, where underwater vehicles (e.g.
AUV, ROV, etc.) can use these stations to recharge their batteries [284].

  * •

Wired energy transfer: In a clear contrast to the previous items, every close-
to-shore IoUT application may rely on energy transmission through a cabled
network. In this method, the energy arrive from a solar, wind, or urban power
network and it is directly transferred to the UWSN. These systems, however,
are costly because of the cabled infrastructures on both land and at sea [9].

The above methods have the potential to be used in marine type projects. They
can extend the lifetime of IoUT networks and boost their QoS. However, except
for the wired energy transfer method, they tend to be unpredictable, hence
none of them guarantees the uninterrupted delivery of energy. To address this
issue, the following pair of solutions may assist:

  * •

Using rechargeable batteries: All the aforementioned methods can be
accompanied by tandem batteries to store energy. These batteries are recharged
during the instances of energy acquisition and deliver their stored energy
afterwards. This solution obviously prolongs the sensor network’s lifetime,
but requires maintenance and increases the cost of the system [9].

  * •

Managing energy consumption: Just like any other electrical grid worldwide,
the IoUT has to manage its energy demands by optimizing the power allocation
to guarantee uninterrupted data collection and transmission [285]. Indeed, all
components of the IoUT should be energy-aware. In this context, an innovative
energy-aware robot was proposed by Wu et al. [286], which had the shape of a
killer-whale. This robot has had reduced energy consumption for its propulsion
as a benefit of its excellent lift-to-drag ratio, which is important for
effortless gliding in water. Compared to the Seagliders introduced in Table
VII, the controllable flukes of this robot offer substantial energy savings,
better maneuverability as well as enhanced endurance.

###  V-C Development of Low-cost and Affordable Sensors

The underwater sensors listed in Table II and multi-sensor buoys are usually
very costly compared to their overwater counterparts. To address this multi-
disciplinary challenge, the following approaches can be considered:

  * •

Quality vs. cost trade-off: Low-cost sensing devices having lower precision
measurements could be purchased to strike a quality vs. cost trade-ff.

  * •

Transferring specimen for inland assessment: The establishment and operation
of an underwater in-situ sensing unit is generally more expensive than a
laboratory-based experiment. Therefore, whenever possible, samples could be
transferred to an inland lab, to avoid the need for costly in-situ processing
and evaluation.

  * •

Inferential measurements: It is a common technique in industrial
instrumentation to estimate a parameter from the values of other parameters,
which are easier to measure. For example, calculating the total amount of ions
dissolved in water is always easier by measuring its electrical resistance
instead of utilizing costly electrochemical sensors [287].

In addition to the above-mentioned general recommendations, designing specific
cost-effective sensors can help the evolution of IoUT technology. In order to
design such sensors, scientists from different research backgrounds have to
cooperate. The result of this cooperation will be ad hoc solutions, which are
tailored to the predefined need of any project. For instance, here we list
several contributions involving low-cost underwater sensors.

  * •

Islam et al. [288] proposed a low-profile and low-cost microstrip patch
antenna to measure the salinity of water. They found that the antenna’s
reflection coefficient is proportional to the amount of salt or sugar
dissolved in water.

  * •

Vorathin et al. [289] constructed a high-resolution hydrostatic pressure and
depth sensor by attaching a fiber Bragg grating on a rubber diaphragm. Their
sensor is claimed to enhance the sensitivity and to compensate the temperature
effects.

  * •

Wang et al. [290] conceived a low-cost turbidity sensor, based on the
$90^{\circ}$ scattered light detection principles. To elaborate, they used
off-the-shelf infrared LEDs having controlled light emission to construct a
low-cost, yet accurate product.

  * •

Kirkey et al. [291] proposed an inexpensive fluorometer based on an optical
backscatter transducer. Explicitly, their idea is to use low-frequency
circuitry for modulating the light source. Using this technique, their product
will be very cost-effective.

  * •

By relying on the fundamental concepts of Time-Domain Reflectometry (TDR),
Time-Domain Transmissometry (TDT), or Fiber Bragg Grating (FBG) in optical
physics, optical fibers can act as a sensor to detect a wide range of
underwater physical parameters. Examples include leak detection in pipelines
and estimating its location [292], stress response of the offshore platforms
(i.e. legs of the jacket structure) and detecting its deformation [293],
bending moments of the flexible risers in a hang-off position to avoid
exceeding its absolute maximum ratings [293], temperature and pressure
measurement as well as eliminating the temperature-pressure cross-sensitivity
[294], etc.

In addition to the above techniques of reducing the sensors costs, reducing
the physical size of underwater sensors also tends to reduce their production
cost. Examples of low-cost smaller sensors include miniature underwater robots
[295] and low-power nano-sensors [128].

Furthermore, using the edge computing capabilities discussed in Section II-J,
it is possible to establish a laboratory on board of an oceanic exploration
platform [37]. This limited-capability on-board lab may conduct preliminary
experiments on samples, before transmitting the numerical results through the
web. Additionally, by using a video-empowered command and control system,
there would be no need for an expert to be present in-situ. This laboratory on
the edge will also eliminate the need for sending the specimen to an inland
lab for evaluation, which would be both expensive and time-consuming.

###  V-D Large-scale IoUT Underwater Communications

In terrestrial telecommunication, electromagnetic waves, copper cables, and
optical fibers are the mainstream transmission media. However, as studied in
Sections II-B and II-D5, fiber-optics are expensive to deploy and maintain
under water, hence typically acoustic, electromagnetic, and optical
technologies are deployed, which do not propagate well, hence making IoUT
telecommunication challenging.

Electromagnetic and optical technologies only cover short communication
distances and are therefore unsuitable for long-range IoUT communications.
Acoustic technologies tend to be more amenable to long-range IoUT networks,
but they have a narrow frequency bandwidth and are prone to cross-talk with
other local acoustic applications [296]. Hence they are also unsuitable for
large-scale networks.

A promising, but costly technique of addressing the communication challenge in
IoUT is to combine heterogeneous communication technologies [110]. However,
the design of multi-technology multi-mode gateways for undersea applications
is a challenging task, especially when considering the energy harvesting
difficulties. These gateways may be combined with the SDN and cognitive radio
concepts for efficiently sharing the limited spectrum undersea [110]. Future
research should conceive energy-aware software and low-power hardware
solutions for these gateways to improve the quality of communication in IoUT
networks.

Another promising solution to underwater communication problems, especially in
the case of long-range inter-continental telecommunications, is the inter-
connection of the IoUT and aeronautical technology. In this technique and
according to Table IV, the data in the application layer of underwater
networks that is produced by sensors and imagery equipment can be transferred
to the overwater buoys and to the floating ships. Therefrom, data will be
handed to the low-earth orbit or medium earth orbit communication satellites,
or to the aerial vehicles. A similar example of this technique is provided by
NCEI, Landsat, Aquarius, SARAL, CryoSat, Jason, HY2-A, and IRS satellites in
Table VIII for remote sensing of ocean surface parameters. Another example is
offered by [115] for the connectivity of unmanned aerial vehicles and IoUT.

Additionally, reducing the data volume to make it suitable for transmission
using narrow-bandwidth acoustic technology could be used for mitigating the
communication challenges in large-scale IoUT networks. This was discussed
under the concept of MCC and MEC edge computing paradigms in Section II-J.
According to this method, edge-processing is capable of reducing the volume of
raw data [113]. This consequently reduces the bandwidth requirement.

In this regard, the combination of the MEC paradigm with smart unmanned
vehicles can offer an alternative solution to the problem of long-distance
underwater communications. In this solution, autonomous underwater vehicles
[114], unmanned aerial vehicles [115], etc. can be used for IoUT data
collection. Here, edge computing can undertake some essential computations, so
that big data collection will be mitigated, while latency-sensitive situations
are handled promptly [115]. However, the lack of energy in undersea
environments, makes the employment of MEC a challenge. In closing we note that
MEC has a similar architecture to the previously suggested inter-connection
between the IoT and IoUT.

###  V-E Dynamic IoUT Signal Routing and Traffic Control

The underwater propagation environment is quite hostile; hence it is of low
channel capacity, which makes even point-to-point single-link data
transmission challenging in the IoUT. This becomes even more challenging when
a network of concurrently communicating nodes is considered. For example, in
the presence of a network supporting multiple transmitters, the tele-traffic
escalates, and traffic control becomes a challenging task, when aiming for a
reasonable QoS [1].

To avoid any tele-traffic congestion, efficient routing management is crucial.
To elaborate a little further, in ad hoc networks the design-dilemma is
whether to use more short hops at the cost of an increased delay or fewer
longer hops. It is beneficial to use low-complexity non-coherent transceiver
techniques and take into account the battery-charge during routing.

Additionally, intelligent traffic control systems, using both the deep
learning approaches [297] and the SDNs [298, 299] may be devised in underwater
applications. These systems are capable of efficiently handling concurrent
data transfers to avoid congestion.

To employ deep learning in traffic control systems, we first have to define
the action space. To do so, consider a heterogeneous IoUT network constructed
of both wired and wireless connections. As detailed in Section II-C2, all the
nodes of such a network may connect with one another, using multiple hops. As
a consequent of both the limited underwater transmission range [300] or the
large-scale infrastructural size of the network [110], the number of possible
hops escalates, leading to numerous potential paths for a data packet to
travel between a pair of nodes. In this context, every path can be considered
as an action in the action space. After taking an action, the system’s
feedback (a.k.a. reward) can be quantified by the traffic load level of the
nodes in the following timeslot, which is formatted as an award value matrix.
The combination of the actions and their consequent rewards provides the
required training data for semi-supervised deep reinforcement learning, as
advocated in [297]. To the best of the authors’ knowledge, such datasets for
IoT, do not exist for IoUT. Once collected, these datasets of IoUT routing
management and traffic control can then be used to train ML-based models.
After successfully passing the training phase, this deep model can be used to
infer the best overall path for a given data packet.

Although the above DNN can be conveniently used in most of the IoUT
structures, it might falter as the IoUT architecture gets wider. By expanding
the network scale, the number of possible paths will increase exponentially,
and the deep NN can no longer learn the patterns in the data flow. To address
this problem, Fadlullah et al. [297] have proposed a solution, which can also
be employed in underwater applications. To elaborate a little further, they
suggested to:

  1. 1.

Change the action space from the entire set of all path combinations to simply
the next hop destination;

  2. 2.

Replace the semi-supervised reinforcement learning by a cascaded combination
of supervised CNN and DBN;

  3. 3.

3) Predict the award value matrix (by CNN) before deciding on the best action
(by DBN).

These recommendations for using DNNs for wide-scale IoUT traffic control will
reduce the packet loss rate as well as improve the network throughput. As a
result, the routing performance will improve compared to that of the
conventional methods.

The other technique of improving the network throughput is that of adhering to
the SDN methodology. This type of network is described in Section II-I, which
relies on a centralized control system, for monitoring the network’s traffic
flow. According to Xie et al. [298], the centralized management of SDNs
substantially benefits from using DNNs. For example, the functions of routing
optimization, traffic prediction, path load prediction, node deployment
optimization, delay prediction, QoS prediction, content delivery optimization,
resource allocation, SDN reconfiguration, optimized spectrum sharing, number
of active nodes estimation, and intrusion detection would all benefit from
using DNNs in overwater SDNs. Hence it is promising to critically appraise
their synergies also in underwater applications.

###  V-F Securing Underwater Networks

As discussed in Section II-H, network security has to be considered throughout
the entire IoUT architectural model, from the physical layer to the
application layer. However, the highly unreliable underwater communication
channels having high propagation delays and low energy resources, make network
security a challenging topic [111].

In this context, and according to a review article by Jiang et al. [97], two
major concerns in underwater communication security, which require further
investigations and therefore offer research opportunities are:

  * •

According to both Section II-D and Table IV, underwater sensors in IoUT are
considered as network endpoints, located at the highest TCP/IP layer (i.e.,
the application layer). These endpoints are vulnerable to potential intrusion
attacks and therefore, have to be secured. But the main focus in UWSN security
researches was on the lower physical, data link, and network layers, missing
out the higher transport and application layers. It is difficult to guarantee
a secure connection in IoUT by relying on the lower three layers alone. On the
other hand, the corresponding application layer security protocols of the
conventional IoT are often computationally complex and hence power-thirsty.
Those protocols are not readily applicable to underwater endpoints. Therefore,
upper-layer security should be further investigated to provide a reliable
underwater network.

  * •

Considering the resource-constrained nature of the IoUT, compared to the
terrestrial IoT, a comprehensive framework is required for optimally
distributing the security-related functionalities across all the layers. This
optimal distribution of security tasks will minimize the overall resource
consumption of the UWSN. To the best of our knowledge, no comprehensive
framework is available for energy-effective security enforcement in IoUT at
the time of writing.

To address these challenges, taking a similar approach to the IoT can be
helpful. Earlier in Section II-H, we introduced several layer-based security
protocols to be employed in the IoUT infrastructure. We also suggested ECC as
a cryptographic primitive in the application layer of the IoUT. However, due
to the large-scale nature of IoUT systems and their complex environment,
traditional cyber security techniques cannot be readily adapted. Therefore,
new research directions relying on DL may be adopted. To address this issue,
large datasets of different IoUT security attacks have to be collected first.
To the best of our knowledge, such datasets do not exist for underwater
applications. However, datasets of various security attacks are already
available for download for the conventional IoT (e.g. CICIDS2017) [301]. Once
collected, these datasets of IoUT security attacks can then be used to train
ML-based models for protecting IoUT networks against a wide range of security
threats.

Having said that, if a large and clean dataset is not collectable for a given
IoUT attack, semi-supervised reinforcement learning might be considered. As
already discussed in Section V-E, the deep reinforcement learning models can
be trained by the combination of actions and their consequent rewards. As a
result, these models can be trained during a practical experiment, leaving out
the data gathering step. Such an experiment has been conducted by Xiao et al.
[302] in a pool with underwater transducers. They have used this setup to
train their proposed reinforcement learning model against sophisticated
jamming attacks in UWSNs. With no prior data gathering, the node has inferred,
where the heavily jammed locations are and avoided them. It has also learned
to adjust its power-level for achieving the required bit error rate.

The IoUT architectural model of Table IV is based on the similar 5-layer
TCP/IP model of IoT. As a result, the layer-wise security threats of IoUT are
of similar nature to the common layer-wise attacks experienced in IoT. Some
examples of these common IoT/IoUT security issues are related to the
recognition and segregation of malicious requests (e.g. distributed denial-of-
service protection), prevention of policy violations (e.g. anomaly as well as
intrusion detection), provision of authentication (e.g. man-in-the-middle
attack prevention and data-injection protection), etc. [97, 111].

When choosing an ML-based model for satisfying certain security requirements
in IoT, CNN as well as RNN deep NNs are proven to have a better single-attack
detection performance, than to other traditional NNs. Furthermore, if a sole
ML model has to detect a collection of multiple attacks, DBNs might be
considered as a better choice [301, 303]. However, the application of these ML
models in IoUT requires further investigations. We anticipate that the
security domain in IoUT can significantly benefit from the techniques already
available in IoT. This offers a great research opportunity for the developers
to adapt these methods to the challenging underwater environment.

###  V-G Deleterious Effects of Imprecise Channel Modeling

Naturally, the channel plays an essential role in designing the underwater
deployment of endpoint nodes, relay nodes, and sinks. The strategic deployment
of nodes is capable of increasing the entire system’s battery life and
improving the QoS. Almost all of the underwater acoustic, electromagnetic, and
optical channel models as described in Sections II-B and II-E and also the
underwater magnetic induction channel model of [304] rely on approximations to
simplify the overall model. For example, many channel models assume straight
signal propagation undersea, which simplifies numerical calculations [6].
However, approximations and idealized simplifying assumptions will result in
imprecise models and inaccurate communication.

Imprecise channel modeling can also lead to inaccurate simulation-based
modeling of underwater communications. Below, we provide a list of recent
publications, which showed the impact of imperfect channel modelling in the
simulations of underwater communication, and therefore used precise channel
models for their simulations.

  * •

Using a preamble alerts the receiver about the reception of an incoming data
burst and switches it from its low-power dormant mode to its high-power active
mode. Therefore, both missing the detection of a preamble and declaring its
reception, when it is actually absent reduces the receiver’s battery life. A
beneficial preamble detection method was proposed by Li et al. [85], for an
underwater digital communication system, which coexists with other deployed
networks. Using an accurate channel model is critical in this identification
method in order to prevent a receiver from being triggered by other systems
and consequently extends the underwater battery life.

  * •

IoUT communication is affected by numerous signal impairments, such as a high
propagation delay and high signal attenuation, as shown in Fig. 4. These
reduce the link reliability, which can be mitigated by sophisticated Forward
Error Correction (FEC) and Automatic Repeat reQuest (ARQ) techniques. However,
at low SNRs the throughput may be reduced by excessive data retransmission.
Liang et al. [86] optimized the overall transmission redundancy to be used in
UWSNs by relying on an accurate channel model.

  * •

In the simulation-based investigation of adaptive transmission in time varying
underwater acoustic channels conducted by Wang et al. [87], the transmitter’s
data queue length and the predicted channel conditions were relied upon for
designing the adaptive transmitter parameter values. Although their method
relies on reinforcement learning to yield considerable energy saving, it
cannot correctly learn the accurate parameter values without a precise channel
model.

To address the need for precise channel modeling, further research is required
for devising precise yet computationally efficient models, for ensuring that
the simulation of underwater communications in context of the emerging IoUT is
as reliable as possible. This would mitigate the need for and the likelihood
of future network upgrades.

###  V-H Sparse and High-maintenance Sensing Devices in IoUT

In contrast to the IoT, the sensing devices of the IoUT are sparsely deployed
and exposed to severe environmental effects [4]. The sparse configuration and
the harsh environmental conditions make the maintenance of the IoUT, costly.
Explicitly, maintenance should mitigate the effects of erosion, corrosion,
sediments, pollutions, and other phenomena imposed by seawater.

To address these issues and to reduce the maintenance cost of sparse high-
maintenance nodes in the IoUT, a compelling solution is to incorporate self-
management capability [88], including self-evaluation, self-configuration, and
automatic reports to human operators. Therefore, developing intelligent ML-
based hardware nodes for the IoUT, which have a self-management and decision-
making capability and conceiving their required software are promising avenues
for research in IoUT, which need further attention.

###  V-I Poor Underwater Positioning and Navigation

As mentioned earlier, GPS signals do not penetrate the sea, hence other
navigation techniques have to be used. A number of navigation methods such as
blind positioning, acoustic transponders, ranging sonars, image-based
positioning, and SLAM were introduced in Section III-D2. However, none of
these stand-alone techniques offer a non-accumulating positioning error,
therefore, none of them are adequate [161].

Hence, further research is required for improving underwater navigation
systems. The challenges to address are:

  * •

Selecting and combining the large amount of data from the aforementioned
stand-alone navigation techniques is a challenging task, while considering the
system’s cost and complexity as well as precision. A research opportunity to
address this issue is to study different combinations of navigation
techniques, while striking a trade-off between cost and accuracy. For
instance, Bonin-Font et al. [20] have combined image-based positioning with
SLAM to achieve improved navigation. Another study has combined blind
positioning with a long-baseline acoustic transponder to reduce the
positioning errors caused by acoustic ray bending and variable sound
velocities [162].

  * •

Preliminary environmental survey for acquiring offline data is a challenging
mandatory step, which has to be conducted before scene analysis based
localization techniques (i.e. image-based positioning systems as well as
acoustic transponders). This step is required for extracting positional
fingerprints (i.e. features), which will be subsequently used for accurately
training a localization ML algorithm. However, gathering these big datasets to
train ML algorithms is not trivial in underwater environments.

Furthermore, traditional ML models can be easily mislead by any variation in
the underlying high-dynamic underwater scenes. To address this, using deep NNs
having automatic feature extraction capability is highly recommended. The
benefits of diverse statistical methods, traditional NNs, and deep NNs in
scene-based indoor positioning systems are reported in [305]. These quick
lines can be adapted for underwater applications.

  * •

Finding and implementing new natural phenomena for improving underwater
positioning as well as navigation is another challenge. In this context, a
gravity-aided navigation system was conceived in [163], which is based on
exploiting the difference between the observed and the predicted gravity.
These methods however, suffer from biases and error accumulation, which has to
be addressed in future research.

  * •

Adapting the existing Low-Power Wide Area Network (LPWAN) technologies, which
has been discussed in Section II-D5, to carry out IoUT localization is a
potential opportunity. In a review article of Zafari et al. [305], a
collection of these wireless technologies (i.e. SigFox, LoRaWAN, Weightless,
etc.) was studied in IoT localization. Using the same approach would be
beneficial for IoUT, as they all consume extremely low energy and operate in a
wide reception range.

###  V-J Non-Destructive Testing in Underwater Applications

Non-destructive testing (NDT) is the process of inspecting a system, device,
or component, without imposing any changes on its shape or material. NDT can
be performed while the Device Under Test (DUT) continues its normal operation.
For example, both the TDR and TDT methods discussed in Section V-C, are
variants of NDT methodologies relying on optical fibers, which are undertaken
without interrupting the normal operation of their DUT [292].

Apart from optical fibers, NDT can also be carried out by other equipment.
Some of these NDT tools include visible light cameras, electromagnetic flux
detectors, ultrasonic transceivers, and magnetic inductors (i.e. eddy current
sensors). Using these equipment in underwater NDT are reported in many diverse
applications, such as:

  * •

Leakage detection [292], vibration recognition [306], and non-stationary
disturbances as well as strain sensing [307] in pipelines, by fiber-optics;

  * •

Bending and deformation inspection in flexible risers, by fiber-optics [293];

  * •

Temperature and pressure monitoring in downhole tools, by fiber-optics [294];

  * •

Determining water-level in unmanned water resource management systems, by
visual cameras [308];

  * •

Inspecting the outer surface of large ship hulls, by electromagnetic flux
detectors [309];

  * •

High-sensitivity hydrophones for opto-acoustic imaging, by ultrasonic [310];

  * •

Welding inspection and defect characterization in offshore platforms, by
magnetic induction [311].

Developing a classic NDT algorithm for a typical component is not an easy
task, requiring knowledge about both the DUT and the underlying physics of the
NDT itself (i.e. ultrasonic, electromagnetic, or optical wave scattering).
Nevertheless, in the modern age of ML techniques, one can gather a big dataset
from any DUT. These data samples can then be fed to a deep NN for training
purposes. By implementing this deep learning approach, developing NDT
algorithms can be carried out faster, with minimal knowledge about the
underlying physical concepts [306, 308, 307].

Another opportunity in underwater NDT is to use the recent developments in the
field of distributed and cloud-based BMD Processing tools, which has been
discussed in Section III-C. Relying on the frameworks listed in Table X, one
can readily glean data from multiple independent NDT equipment, and then
employ a data fusion technique from Section III-A3, to combine and process the
gathered big data. It is proven by Bayes’ theorem that the uncertainty in the
final test result will dramatically decrease by fusing data from independent
data sources [311].

###  V-K Lack of Strong Data Leveraging Tools

The processing of BMD requires powerful hardware and software tools that can
automatically extract knowledge from large databases. Some of these hardware
tools were introduced throughout this paper, specially in Sections III-C and
IV-E. However, more advanced software techniques are required for automatic
long-term data-gathering and data-monitoring applications. For instance, even
though unmanned auto-annotation based industrial software is in very high
demand for classification and labeling underwater objects, plants, or
creatures, the existing marine image-annotation software packages are only
semi-automated, at best [159].

The shortages in automatic data leveraging techniques is partially due to the
uncertainty in selecting feature-set. Ambiguity in feature selection and in
the ensuing feature reduction is a consistent challenge in automated ML-aided
projects. In a conventional neural network, there are a variety of features
and descriptors, hence selecting the most useful ones is challenging. These
diverse underwater feature-sets have previously been discussed in Section IV-D
and summarized in Table XIV. Recall that relying on a single feature is
usually inadequate for accurate underwater classification and clustering
[210].

Although modern deep learning approaches (e.g. fully-connected, CNN,
Autoencoder, etc.) are promising in terms of overcoming this challenge, the
advantages of classical ML methods may nonetheless provide better results.
These benefits of classical ML methods include having fewer parameters, more
rapid convergence during their training, better insights into the tangible
physical interpretation of their operation, and much easier debugging as well
as tunning the network. Further research is required for developing techniques
and algorithms to infer useful features from a dataset, or even to automate
the feature selection process. This will be invaluable for the ML community,
because feature selection has a direct impact on the performance of ML-based
solutions.

###  V-L Training Deep Networks

When using deep networks, the feature extraction will be automatically handled
by the hidden layers of the network. Despite this automatic feature
extraction, the deep networks have the disadvantage of requiring large amounts
of data for tuning their weights and biases, during the training step. The
deeper and wider these networks become, the more useful features can be
extracted [312]. Ultimately, we have to strike a trade-off between the
training data volume required, the network size/power and its overall
performance.

To address this trade-off, a pair of general solutions could offered. The
first solution is to satisfy the deep network’s hunger by more data. The extra
data can be generated automatically (for example by various data augmentation
techniques or by employing generative neural networks, such as the Generative
Adversarial Network (GAN) [313, 314]) or manually (by using web-based
technologies to enlist the assistance of international experts and to produce
a large volume of user-generated contents).

The second solution is to enhance the deep network’s efficiency by modifying
their building blocks and the inter-connection of neurons (like the
convolution, pooling, and activation operations of a typical CNN [315] or
pruning a deep NN to ease their operation on mobile devices [316]), in order
to reduce the number of network parameters. The vibrant deep learning
community will no doubt continue to improve the deep network training and
tackle the challenges. This will definitely improve the IoUT applications and
advance BMD analytics.

###  V-M Degraded Underwater Images

Undersea photography is always affected by environmental factors. Although the
contributions surveyed in Section IV-C2 have addressed some of these issues,
there is a lot of room for improvement. For instance, developing an imaging
system capable of both real-time forward- and back-scattering elimination is
critically needed.

Furthermore, the underwater image quality could be significantly improved by
applying some well-known hardware methods and techniques, such as light
polarization [317], multi-spectral imaging [318], or stereoscopic imaging
[319]. Additional improvements may also be attained by software methods, such
as wavelength compensation and color reconstruction [135, 197] both in active
and passive underwater photography. Finally, the new concept of image
reconstruction with DL [190, 250, 251] has promise for future research.

##  VI Conclusion

The recent advances in IoT technology and the extension of its influence both
to coastal and open sea areas has led to the proliferation of the number of
Internet-connected objects both in over- and underwater applications. This
technological evolution inspired the new scientific concept of IoUT
constituted by marine sensors, cameras, hydrophones, etc. This concept opens
many new research directions for undersea data acquisition, data
communication, BMD handling, and oceanic data processing.

In this article, we commenced by surveying the state-of-the-art in underwater
communications. Given the harsh underwater propagation environment, data
communication is quite a challenging task in the IoUT. The families of
advanced underwater communication models of acoustic, electromagnetic, and
optical technologies were introduced and innovative solutions were proposed
for increasing the overall link reliability by topology and routing
optimization, security improvement, and protocol enhancements. Furthermore,
the underwater channel modeling was studied along with software tools to
simulate both those channel models as well as the communication protocols.
Both SDN and edge computing techniques were also reviewed as a promising
technique of improving underwater communications. We also reviewed the IoUT
network architecture, based on the well-known 5-layer TCP/IP standard model.

Naturally, the IoUT leads to BMD generation and the associated challenges
include data storage, transportation, preparation, and analysis. Because
manual and semi-automatic data processing methods are no longer appropriate in
the new era of the IoUT, the five system components of BMD solutions were
discussed and the most recent frontier-research and a range of practical
solutions were discussed for each component. These solutions covered the areas
of sensor, image, and video data sources, marine geographic data, localization
and tracking, open access databases, distributed data processing, and cloud-
based services. A complete section was also dedicated to machine intelligence
(i.e. ML and DL) and its applications to marine data processing. The most
recent research articles in both the hardware and software aspects of the IoUT
and BMD processing were also surveyed, along with the critical appraisal of
these works. Finally, numerous open research issues and future study
directions were presented to provide an insight into the prospective
applications, trends, and challenges. Do join this vibrant interdisciplinary
research community, valued colleagues.

## References

  * [1] A. Al-Fuqaha, M. Guizani, M. Mohammadi, M. Aledhari, and M. Ayyash, “Internet of things: a survey on enabling technologies, protocols, and applications,” _IEEE Commun. Surveys Tuts._ , vol. 17, no. 4, pp. 2347–2376, Jun. 2015.
  * [2] M. H. Eldefrawy, N. Pereira, and M. Gidlund, “Key distribution protocol for industrial internet of things without implicit certificates,” _IEEE Internet Things J._ , vol. 6, no. 1, pp. 906–917, Feb. 2019.
  * [3] C.-C. Kao, Y.-S. Lin, G.-D. Wu, and C.-J. Huang, “A comprehensive study on the internet of underwater things: applications, challenges, and channel models,” _Sensors_ , vol. 17, no. 7, Jul. 2017.
  * [4] M. C. Domingo, “An overview of the internet of underwater things,” _J. Netw. Comput. Appl._ , vol. 35, no. 6, pp. 1879–1890, Nov. 2012.
  * [5] WHOI, “Know your ocean,” Feb. 2019. [Online]. Available: http://www.whoi.edu/know-your-ocean/
  * [6] Y. Li, Y. Zhang, W. Li, and T. Jiang, “Marine wireless big data: efficient transmission, related applications, and challenges,” _IEEE Wireless Commun._ , vol. 25, no. 1, pp. 19–25, Feb. 2018.
  * [7] G. Xiao, B. Wang, Z. Deng, M. Fu, and Y. Ling, “An acoustic communication time delays compensation approach for master-slave AUV cooperative navigation,” _IEEE Sensors J._ , vol. 17, no. 2, pp. 504–513, Jan. 2017.
  * [8] Z. Zeng, S. Fu, H. Zhang, Y. Dong, and J. Cheng, “A survey of underwater optical wireless communications,” _IEEE Commun. Surveys Tuts._ , vol. 19, no. 1, pp. 204–238, Jan. 2017.
  * [9] C. A. Pérez, F. S. Valles, R. T. Sánchez, M. J. Buendía, F. López-Castejón, and J. G. Cervera, “Design and deployment of a wireless sensor network for the Mar Menor coastal observation system,” _IEEE J. Ocean. Eng._ , vol. 42, no. 4, pp. 966–976, Oct. 2017.
  * [10] F. Javed, M. K. Afzal, M. Sharif, and B. Kim, “Internet of things (IoT) operating systems support, networking technologies, applications, and challenges: a comparative review,” _IEEE Commun. Surveys Tuts._ , vol. 20, no. 3, pp. 2062–2100, Mar. 2018.
  * [11] X. Cao, L. Liu, Y. Cheng, and X. S. Shen, “Towards energy-efficient wireless networking in the big data era: a survey,” _IEEE Commun. Surveys Tuts._ , vol. 20, no. 1, pp. 303–332, Jan. 2018.
  * [12] S. Verma, Y. Kawamoto, Z. M. Fadlullah, H. Nishiyama, and N. Kato, “A survey on network methodologies for real-time analytics of massive IoT data and open research issues,” _IEEE Commun. Surveys Tuts._ , vol. 19, no. 3, pp. 1457–1477, Apr. 2017.
  * [13] L. F. Maimo, A. L. P. Gomez, F. J. G. Clemente, M. G. Perez, and G. M. Perez, “A self-adaptive deep learning-based system for anomaly detection in 5G networks,” _IEEE Access_ , vol. 6, pp. 7700–7712, Feb. 2018.
  * [14] L. Meng, T. Hirayama, and S. Oyanagi, “Underwater-drone with panoramic camera for automatic fish recognition based on deep learning,” _IEEE Access_ , vol. 6, pp. 17 880–17 886, Mar. 2018.
  * [15] X. Sun, J. Shi, L. Liu, J. Dong, C. Plant, X. Wang, and H. Zhou, “Transferring deep knowledge for object recognition in low-quality underwater videos,” _Neurocomp._ , vol. 275, pp. 897–908, Jan. 2018.
  * [16] N. Ani Brown Mary and D. Dharma, “Coral reef image classification employing improved LDP for feature extraction,” _J. Vis. Commun. Image Repres._ , vol. 49, pp. 225–242, Nov. 2017.
  * [17] M. Reggiannini and O. Salvetti, “Seafloor analysis and understanding for underwater archeology,” _J. Cultur. Heritage_ , vol. 24, pp. 147–156, Apr. 2017.
  * [18] M. Jian, Q. Qi, J. Dong, Y. Yin, and K.-M. Lam, “Integrating QDWD with pattern distinctness and local contrast for underwater saliency detection,” _J. Vis. Commun. Image Repres._ , vol. 53, pp. 31–41, May 2018.
  * [19] T. Schoening, T. Kuhn, D. O. B. Jones, E. Simon-Lledo, and T. W. Nattkemper, “Fully automated image segmentation for benthic resource assessment of poly-metallic nodules,” _Methods Ocean._ , vol. 15-16, pp. 78–89, Jul. 2016\.
  * [20] F. Bonin-Font, A. Burguera, and J. L. Lisani, “Visual discrimination and large area mapping of Posidonia Oceanica using a lightweight AUV,” _IEEE Access_ , vol. 5, pp. 24 479–24 494, Oct. 2017.
  * [21] M. C. Chuang, J. N. Hwang, J. H. Ye, S. C. Huang, and K. Williams, “Underwater fish tracking for moving cameras based on deformable multiple kernels,” _IEEE Trans. Syst., Man, Cybern.: Syst._ , vol. 47, no. 9, pp. 2467–2477, Sep. 2017.
  * [22] P. Y. Lau, P. L. Correia, P. Fonseca, and A. Campos, “Estimating Norway lobster abundance from deep-water videos: an automatic approach,” _IET Image Process._ , vol. 6, no. 1, pp. 22–30, Feb. 2012.