# Can We “Sense” the Call of The Ocean?
Current Advances in Remote Sensing Computational Imaging for
Marine Debris Monitoring

Oktay Karakuş  Cardiff University, School of Computer Science and Informatics,
Abacws, Senghennydd Road, Cardiff, CF24 4AG, UK

###### Abstract

Especially due to the unconscious use of petroleum products, the ocean faces a
potential danger: plastic pollution. Plastic pollutes not only the ocean but
also directly the air and foods whilst endangering the ocean wild-life due to
the ingestion and entanglements. Especially, during the last decade, public
initiatives and academic institutions have spent an enormous time on finding
possible solutions to marine plastic pollution. Remote sensing imagery sits in
a crucial place for these efforts since it provides highly informative earth
observation products. Despite this, detection, and monitoring of the marine
environment in the context of plastic pollution is still in its early stages
and the current technology offers possible important development for the
computational efforts. This paper contributes to the literature with a
thorough and rich review and aims to highlight notable literature milestones
in marine debris monitoring applications by promoting the computational
imaging methodology behind these approaches.

###### keywords:

Marine debris monitoring, Marine plastic detection , Floating plastics ,
Remote sensing , computational imaging

††journal: Elsevier

##  1 Introduction

The ocean provides a livelihood for more than 3 billion people whilst being
the habitat for billions of species and generates a more than $3 trillion each
year to the global economy [1]. Despite this importance, especially in the
last decade, plastic pollution and marine debris have become one of the most
important reasons endangering this valuable and consumable resource due to the
large needs of the human population and the unconscious use of plastic.

Figure 1: Great Pacific Garbage
Patches111https://marinedebris.noaa.gov/info/patch.html.

Particularly, large plastics entering the ocean either sinks or floats since
they are low-density materials [2]. Floating plastics in macro-scale (say
macro-plastics, $>$ 5mm) is the most crucial sub-group of floating plastics
that cause entangling and ingestion for marine wildlife. Thus, detecting and
tracking macro-scale floating plastics before their fragmentation into micro
and nano scales, and/or threatening the wild-life trough entanglement and
ingestion becomes the primary action to protect the marine environment. One
example area of focus is the Great Pacific Garbage Patch (GPGP), which can be
described as a gyre within a gyre. GPGP covers an area of ocean approximately
1.6 million square km (roughly 8, 3, 2 times greater than the UK, France, and
Turkey, respectively) in size. However, the density of ocean plastic can vary
greatly and change quickly within any location [3]. Figure 1 shows garbage
patches and ocean currents in the Pacific Ocean.

Since the means of plastic usage, specifically in the last decade, has shifted
from a positive tool to potential danger to the environment, various efforts
have been performed to help authorities and the academic community in reducing
and preventing marine pollution. Marine debris and floating plastics can cause
various problems, but mostly affects human health and marine wildlife.
Particularly, plastic is polluting not only the water, but also the air we
breathe, and the food we eat. According to the WWF’s report in 2019, an
average person could be ingesting approximately 5 grams of plastic every week,
which is the equivalent of a credit card’s worth of microplastics. Moreover,
an average person potentially consumes as much as 1769 particles of plastic
every week just from water [4]. Apart from the risks it causes to human
health, marine litter and debris also endanger wildlife crucially, from tiny
coral reefs to huge whales. To date, plastic ingestion and/or entanglement has
caused the killing of more than 700 marine species [5]. According to the
research, more than 50% of sea turtles eat plastic whilst plastic materials
have been found inside more than half of the dead sea turtles. Additionally,
more than 50% of the sea mammals such as dolphins, whales and seals are
impacted by plastic pollution whilst 100 thousand of them are killed by
plastic each year (National Oceanic and Atmospheric Administration (NOAA) via
[6]). Figure 6 depicts several examples of plastic pollution threat on sea
animals.

In addition to the problems caused for human health and marine wildlife, the
marine debris problem causes some non-native organisms to travel to areas
where they would not be found in normal circumstances. These organisms attach
themselves to marine debris and are transported all around the world. Last but
unfortunately not least, the ocean pollution problem can heavily affect
economies, especially for countries whose economy relies on summer tourism.
Considering the summer tourism sector heavily relies on healthy coastal and
ocean resources along with environmental and aesthetic quality, beaches where
floating plastics and garbage patches are drifted, will not provide healthy
conditions and this sector will economically be affected by this pollution
problem [7].

Figure 2: Plastic pollution threat on sea animals. (a) Firepaw,
Inc.222https://firepaw.org/2019/07/05/sharks-other-marine-life-strangled-by-
plastic-study/, (b) Hakai
Magazine333https://www.hakaimagazine.com/news/plastic-pollutions-rapidly-
mounting-toll/, (c) Lauren
Packard444https://www.flickr.com/photos/110485367@N08/11180067935, (d) World
Animal Protection, Marine Animals of Maine, and David Burdick-Marine
Photobank555https://www.worldanimalprotection.org/our-work/animals-wild/sea-
change/our-work/rescue, and (e) Planet Love
Life666https://www.planetlovelife.com/pages/wildlife-entanglement.

For environmental applications, exploiting earth observation systems and their
products such as active and passive imagery is a long-established approach,
whilst for marine debris and floating plastic monitoring the efforts are still
in the early stages. Airborne and space-borne remote sensing modalities such
as the optical, hyperspectral, and synthetic aperture radar (SAR) have shown
an important potential up to date in detecting, tracking, and mapping marine
pollutants, such as oil spills. Moreover, thanks to their (optical, hyper-
spectral) rich visible band and spectral choices, remote sensing imagery has
had applications in assessing various metrics based on algal blooms whilst SAR
has shown to be the gold standard in vessel-tracking and recognition
applications. For all the marine applications above, the main imaging modality
limitation was their spectral and temporal resolutions. Thanks to recent
technological developments, new generations of satellites have been launched
and spatial resolutions which were previously unavailable are now provided
from space-borne remote sensing. In the last decade, initiatives such as NASA,
UK Space Agency (UKSA), and European Space Agency (ESA) pay attention to
saving the ocean by using recent technological advances. For this purpose,
some effort has been made to track/detect marine pollution and illegal
fishing. Satellite remote sensing imagery (optical, SAR, hyper-spectral, etc.)
has been the leading technique for marine monitoring since its capability to
capture wide-angle area imagery under challenging weather conditions and for
various sea states, etc.

Apart from the governmental initiatives mentioned above, various efforts have
also been specifically spent on marine debris and floating plastics monitoring
in both academia and industrial companies. Currently, creators of
Plastic-i777https://www.plastic-i.com/ have developed a powerful tool (and
also the world’s first) for mapping ocean plastics via satellites, and won the
first-ever Hack The Planet Competition 2021. Moreover. A non-profit
organisation, the Ocean Cleanup888https://theoceancleanup.com/, is developing
and scaling technologies to rid the oceans of plastic. Their software - The
Plastic Tracker999https://theoceancleanup.com/plastic-tracker/ \- is an
educative tool to predict where the floating plastic patches are heading to.
Debris Tracker101010https://debristracker.org/ is designed to make community
members contribute data on plastic pollution. Passionate people from all
around the world record data on inland and marine debris by using the Debris
Tracking App. Ocean Scan111111https://www.oceanscan.org/ is a global database
and aims to promote important international collaborations for the marine
debris cleaning efforts. They offer access to various in-situ observations and
matching remote sensing images of marine litter and debris. Messages in
Bottles (MiBs)121212https://messages-in-bottles.netlify.app/ provides a non-
real time interactive tool to visualise information on coastal plastic waste
using Sentinel-2 optical imagery (inspired by [8]) in order to connect
research and public interests for the purposes of the marine pollution
efforts.

Within this developing application area, the machine/deep learning-based
remote sensing computational imaging approaches are still in the early stages
one of the most important reasons for which is the lack of approved marine
debris data sets. Especially in the last couple of years, there has been a
good amount of academic outcomes published promoting advanced computational
imaging approaches and publishing data sets thanks to the efforts from
academic institutions for marine debris and floating plastics
detection/tracking purposes.

This paper, via contributing to the literature with a thorough review,
highlights current advances based on remote sensing computational imaging for
marine debris and floating plastics detection and tracking applications. The
rest of the paper is organised as follows: Academic outcomes serving the
efforts of marine debris problem are discussed in Section 2. The literature
statistics describing the number of publications, etc. is presented in Section
3 whilst research challenges and future research directions are discussed in
Sections 4, and 5, respectively. Section 6 concludes the paper with a general
discussion.

##  2 Marine Debris Monitoring Applications

Initial efforts for the purposes of detecting, analysing the marine debris
problem mostly rely on in-situ approaches via (1) visual surveys [9, 10],
and/or (2) airborne data owing to their potential to monitor by the help of
high-resolution and real-time information [11, 12, 13]. In a recent study
[14], de Vries et. al. present the utilisation of footage of floating plastic
debris recorded offshore with GPS-enabled action cameras aboard vessels. Their
work provides consistent results of predicted macroplastic compared to the
global plastic dispersal models.

Due to its capability in covering large and inaccessible areas, satellite-
based remote sensing imagery is considered a useful tool for monitoring the
marine environment and detecting/tracking macro-scale marine pollution. An
initial study promoting utilisation of satellite imagery in order to detect
floating plastics/debris is by Aoyama [15, 16]. This study firstly concludes
that the small size of most marine debris means that it cannot be confirmed
directly, even when using high spatial resolution satellite imagery. In order
to solve these problems, the author proposed using several candidate debris
pixels and making a spectral analysis on these whether there is a
discrimination between debris and surrounding ocean pixels. Their approach
works effectively in cases where the size or area of marine debris is large
enough to affect a change in the captured spectra. Multi-spectral satellite
imagery has also been used to validate various important post-events after the
devastating tsunami triggered by the Tohoku-Oki earthquake of 11 March 2011.
In [17], it has been shown that high spatial resolution satellite tracking
reveals faster-than-expected floating debris motions.

A study by Goddijn-Murphy et al. [18] has shown that plastic litter and
seawater develop a reflectance model coming from their spectral signatures and
optical geometry. The work in [18] considers only one type of macro plastic
and proposes that the fraction of a plastic surface can be estimated from the
surface reflectance, since specifically knowing the clear water reflectance.
It has also been reported in [18] that there are some limitations to this
approach such as shading of surface plastic, and the key approach is to select
a spectral wavelength in which water leaving light is minor and plastic
reflectance is high (approx. 750 $\eta$m). Even though it leads future studies
on plastic detection applications, the work in [18] hence constitutes
limitations in natural environments since plastic debris appears in a
magnitude of shapes, colours, and sizes. Nevertheless, determining the
appropriate reflectance for the plastic makes it detectable in remote sensing
imagery.

Figure 3: (a) Spectral signatures of different materials [8]. (b) Sentinel-2
band characteristics. (c) Example plastic coverage in pixels [19]

Reflectance analysis in [18] motivated [20, 19] for a further detailed
analysis via an approach consisting of both satellite and unmanned aerial
imagery for the detection of plastic patches. Topouzelis et. al. [20] have
created a measurement setup at Tsamakia beach, Greece, which includes a set of
three artificial floating plastic targets (PET bottles, plastic bags and
fishing ghost nets). It has been shown that plastic litter such as bottles,
bags, and fishing nets, reflect light in the near-infrared (NIR) band where
clear water absorbs the light. Moreover, it has been reported in [20] that the
reflectance intensity is also directly related to the amount of plastic in a
single pixel of the utilised image (10m x 10m for Sentinel-2). Thus, if water
composes more than 50-70% for a given pixel, the reflection from the plastic
pollutant is highly poor in the NIR band. Consequently, remote sensing data is
required to have pixels that are filled with more than 30-50% plastics (an
area of at least 30-50 m2). This case can be achieved for the regions of such
as coastal areas, where an eddy or a similar reason might combine plastics
into larger patches. However, especially for the ocean, individual plastic
patches will potentially be less than the detectable limit [8]. This can be
seen as the main limiting factor of the plastic detection research via the
optical remote sensing modalities since the Sentinel-2 imagery has a spatial
resolution of 10x10 meters. It leads to the detectable floating plastic area
to be at least 50m2 since plastic pollutants become spectrally visible if the
water composes less than 50% of the pixel area. In [20], a Sentinel-1 SAR
imagery has also been analysed for the same measurement setup, however, it has
not been concluded notable results based on the detection of plastics via SAR
imagery. The potential main reason behind this is the spatial resolution of
10m and the relatively less penetrable frequency band of the Sentinel-1
satellites (C band compared to L or S bands).

Another work [21] has deeply discussed the requirements of a specifically-
designed remote sensing monitoring system of plastic pollution. It has been
reported in [21] that an ideal remote sensing system would compromise both
passive (optical) and active (SAR) satellite modalities, as well as support
from UAV based systems. Passive sensors that offer short-wave infrared (SWIR)
band measurements are noted as the high potential modality, whilst SAR also
specified as a potential for future developments, especially thanks to new SAR
platforms (such as the ICEYE) capability to offer a higher spatial resolution,
and lower revisiting times.

The very first work on detecting and classifying plastic patches using
_solely_ optical satellite data has been studied by Biermann et. al in [8].
The work in [8] has two joint objectives: (1) demonstrating the capability of
Sentinel-2 data on detecting floating macroplastics, and (2) classifying
macroplastics and other natural materials which are most likely to be
aggregated within mixed patches of floating debris. The authors proposed a
novel parameter, which is the floating debris index (FDI), in order to analyse
sub-pixel interactions of macroplastics with the sea surface to increase the
chance of detection of patches floating on the ocean surface. The FDI index is
inspired by a previously proposed index of floating algae index (FAI). By
replacing the red band measurements with the red edge band (approx. at 740
nm), the authors leverage the difference between the NIR band and its baseline
reflectance. The FDI can be represented as

$\displaystyle FDI$ $\displaystyle=R_{NIR}-R^{\prime}_{NIR}$ (1)
$\displaystyle R^{\prime}_{NIR}$
$\displaystyle=R_{RE2}+(R_{SWIR1}-R_{RE2})\times\dfrac{\lambda_{NIR}-\lambda_{RED}}{\lambda_{SWIR1}-\lambda_{RED}}\times
10$ (2)

where $R_{NIR},R_{RE2}$, and $R_{SWIR1}$ refers to NIR, red edge 2 and SWIR1
bands, respectively whilst similarly $\lambda$ values are wavelength values
(in $\eta$m). Sentinel-2 band characteristics are provided in Figure 3-(b).
The FDI dramatically highlights the plastic and has been found useful in the
identification of floating plastics in waterbodies using spectral remote
sensing imagery. An example for the visibility of the FDI is depicted in
Figure 4.

Figure 4: Representation of FDI index in optical imagery [8]

It has also been shown in [8] that using FDI in conjunction with the
Normalised Vegetation Difference Index (NDVI)

$\displaystyle NDVI=\dfrac{R_{NIR}-R_{RED}}{R_{NIR}+R_{RED}}$ (3)

makes it possible to detect differences between plastics, vegetation,
driftwood, and seafoam.

Themistocleous et al. [13] set up a pilot study to investigate whether plastic
targets on the sea surface can be detected from Sentinel-2 only data. Their
setup consists of a target of plastic water bottles with size of 3m$\times$10m
in the sea near the Old Port in Limassol, Cyprus. They gathered UAV multi-
spectral images during the same time as the Sentinel-2 satellite passes. They
have used various important spectral indices such as NDVI, but not using FDI
of [8] since the work has been published during the same time interval in
2020. Instead, they proposed a new index called Plastic Index (PI)

$\displaystyle PI=\dfrac{R_{NIR}}{R_{NIR}+R_{RED}}.$ (4)

Their spectral analysis on the visibility and detectability of the floating
plastics in Sentinel-2 imagery showed that the newly developed PI has been
able to identify floating plastics and was selected as the most effective
index.

Kikaki et. al. [22] has conducted a research over a specific target location -
Bay Islands in the Caribbean Sea where remarkable amounts of plastic debris
have been reported. Satellite imagery for the time interval of 2014-2019 has
been investigated and in-situ data collected with vessel and diving
expeditions has been verified via exploiting the spectral characteristics of
the floating plastics. This work, even though it is not proposing a technical
method to detect and track marine debris, utilises classical spectral analysis
approaches, but this time to analyse the source of pollution. According to
their findings, the main source of pollution in the target area has been found
to be the river discharges from the basins of Honduras and Guatemala. It has
also been reported that dynamic sea currents affects the pollution and plastic
patches can travel more than 200km. Figure 5 shows an example figure
highlighting the source of plastic pollution in Bay Islands in the Caribbean
Sea.

Figure 5: An example figure from [22] shows the source of plastic pollution in
Bay Islands in the Caribbean Sea.

Up to this point, all the works were based on Sentinel-2 imagery for the
purpose of promoting satellite images in floating plastic detection. Park et.
al. [23], instead of Sentinel-2 multi-spectral imagery, utilises very high
geo-spatial resolution 8-waveband WorldView-3 imagery in order to observe
floating plastic litter in the Great Pacific Garbage Patch (GPGP). They apply
various spectral analysis approaches and investigate anomalies to infer
presence of suspected plastic litter.

Salgado-Hernanz et. al., in their bibliographic review [24], reveal one more
time that the optical sensors seem to be the most optimum to target marine
litter. Interestingly, in their work, they also promote SAR utilisation and
specifically mention that SAR sensors may detect sea-slicks (areas containing
high concentration of microplastics). Furthermore, Hu [25] also suggests that
it is impossible to detect floating material by all the optical spectra and
analysis should be performed over the difference spectra to minimize the
impact of variable subpixel coverage. It is certain that some spectral bands
are more important than others for the remote detection of marine debris.
Ciappa [26] proposes a marine litter detection work for target location of
North Adriatic during the summer 2020. This work similar with the ones
mentioned above uses spectra arithmetic for spectral anomalies of the Red Edge
bands, assuming changes of the red edge in pixels where marine litter was
mixed with vegetal materials. Table 1 highlights the most common spectral
indices utilised by the works discussed above.

###  2.1 Methodological Framework

Even though all the works published in this area can be somehow considered to
be in computational imaging, in this chapter, we mostly focus on works that
promote machine/deep learning-based computational imaging techniques such as
feature extraction, segmentation, classification for the purpose of marine
debris and floating plastics detection.

Table 1: Frequently Utilised Spectral Indices |  Index |  Expression
---|---|---
NDVI |  Normalized Difference Vegetation Index |  $\dfrac{R_{NIR}-R_{RED}}{R_{NIR}+R_{RED}}$
FDI |  Floating Debris Index |  $R_{NIR}-\left[R_{RE2}+(R_{SWIR1}-R_{RE2})\times\dfrac{\lambda_{NIR}-\lambda_{RED}}{\lambda_{SWIR1}-\lambda_{RED}}\times 10\right]$
PI |  Plastic Index |  $\dfrac{R_{NIR}}{R_{NIR}+R_{RED}}$
WRI |  Water Ratio Index |  $\dfrac{R_{GREEN}+R_{RED}}{R_{NIR}+R_{SWIR2}}$
RNDVI |  Reversed Normalized Difference Vegetation Index |  $\dfrac{R_{RED}-R_{NIR}}{R_{RED}+R_{NIR}}$
AWEI |  Automated Water Extraction Index |  $4\times(R_{GREEN}-R_{SWIR2})-(0.25\times R_{NIR}+2.75\times R_{SWIR1})$
MNDWI |  Modified Normalization Difference Water Index |  $\dfrac{R_{GREEN}-R_{SWIR2})}{R_{RED}+R_{SWIR2}}$
NDMI |  Normalization Difference Moisture Index |  $\dfrac{R_{NIR}-R_{SWIR}}{R_{NIR}+R_{SWIR}}$
NDWI |  Normalized Difference Water Index |  $\dfrac{R_{GREEN}-R_{NIR}}{R_{GREEN}+R_{NIR}}$
SAVI |  Soil adjusted vegetation index |  $(1+L)\times\dfrac{R_{NIR}-R_{RED}}{R_{NIR}+R_{RED}+L}$
NDWI2 |  The second normalized difference water index |  $\dfrac{R_{GREEN}-R_{RED}}{R_{GREEN}+R_{RED}}$
NDBI |  Normalised Difference Build-Up Index |  $\dfrac{R_{SWIR}-R_{NIR}}{R_{SWIR}+R_{NIR}}$
FAI |  Floating Algae Index |  $R_{NIR}-\left[R_{RED}+(R_{SWIR1}-R_{RED})\times\dfrac{\lambda_{NIR}-\lambda_{RED}}{\lambda_{SWIR1}-\lambda_{RED}}\right]$
S2-based |  Sentinel 2 based index |  $\dfrac{(R_{RE3}-R_{NIR})+(R_{NNIR}-R_{NIR})}{R_{RE3}+R_{NNIR}}$

Acuna-Ruz et. al. [27] propose an approach with the objective to improve the
identification of macroplastics through remote sensing over beaches. The
authors promote the utilisation of 8-band very-high resolution WorldView-3
satellite products. Five important ML approaches of Random Forests (RF),
Support Vector Machines (SVM) with three kernel functions of linear,
polynomial, and radial basis function, and Linear Discriminant Analysis (LDA)
have been tested in order to classify the beach debris. The results show that
the SVM approaches were mostly the best with detection accuracy around 85-90%.
Fallati et. al. [28] propose an ad-hoc methodology for beach litter analysis
based on the combined use of a UAV and the deep-learning software of
PlasticFinder. The utilised software PlasticFinder is a commercial software of
DeepTrace Technologies. It consists of several Convolutional Neural Networks
(CNN) which are constructed to detect and quantify Anthropogenic Marine Debris
(AMD). The approach in [28] reaches a sensitivity value of 67% whilst the
positive predictive value of 94%. Despite the considerable performance of the
PlasticFinder, this software highly relies on the sunlight conditions of the
scene and has serious problems with targets such as footprints or shadows.

The first satellite-only computational imaging approach has been proposed by
Biermann et al. [8] and they have utilised a Näive Bayes classification
approach for the floating plastic classification. In fact, this approach has
two main steps: (1) manual detection of marine debris using FDI and NDVI
values, (2) using Näive Bayes to classify plastics among water, seaweed,
timber, and foam. Across all five test sites, the proposed approach reaches
86% detection accuracy of plastics of the pixels detected as marine debris in
the first manual step. Kako et. al. [29] propose a novel approach which is a
combination of UAV surveys and deep learning approaches. Via performing edge
detection they estimate plastic marine debris on the beaches of Sato and
Fukiage in Kagoshima, Japan. The computational imaging approach in [29]
exploits a binary classification on each pixel (either plastic or not) based
on 3 layers deep artificial neural network with a performance of less than 5%
error.

Furthermore, Jakovljevic et. al. [30] aim to investigate the applicability of
various important deep learning algorithms for automatic floating plastic via
utilising UAV orthophotos. They also test to discriminate plastic types by
exploring various characteristics such as spatial resolution and detectable
plastic size. Utilised deep neural network approaches are semantic
segmentation algorithms based on U-Net architecture. Results show that the
ResUNet50 architecture achieved the best performance via detecting plastics
with more than 85% of precision. An automated plastic pollution monitoring
approach for the river surfaces using bridge‐mounted camera imagery is
presented by van Lieshout et. al. in [31]. The authors have performed an
experimental analysis of five different rivers in Jakarta, Indonesia. The
proposed deep learning-based approach consists of two cascaded CNNs for the
purposes of segmentation and object detection. For the segmentation stage, the
so‐called Faster R‐CNN (trained on the river image data set) was used whilst
for the detection stage Inception v2 network (pre-trained on the COCO data
set) was used. With the highest 69% precision of plastic detection, their
results demonstrate a promising way of monitoring plastic pollution.

Freitas et. al. [32] propose a hyperspectral imaging-based ML procedure to
detect floating plastics. Data set collected with manned/unmanned airborne
hyper-spectral sensors have been tested with two supervised methods of RF and
SVM. Their results show an automated detection capability of marine litter
with up to 80% precision (50% recall). Tasseron et. al. [33] present a hyper-
spectral laboratory setup to collect spectral signatures of 40 virgin
macroplastic items and vegetation. The system covers a spectral range from
visible to shortwave infrared (VIS-SWIR) from 400 to 1700 nm. The experiment
setup has given around 2 million pixels each of which has been processed via
linear discriminant analyses (LDA). Their results return absorption peaks of
plastics (1215 nm, 1410 nm) and vegetation (710 nm, 1450 nm), and provide
evidence of the gold spectral indices of NDVI and FDI.

Basu et. al. [34] utilise Sentinel-2 multi-spectral imagery to detect floating
plastic debris in coastal water-bodies in specific test locations in Limassol,
Cyprus, and Mytilene, Greece. The utilised Sentinel-2 images are of the date
on which the real-world plastic pollution data were collected through
experiments at the two study area locations. The authors test four machine
learning classification approaches K-means, fuzzy c-means (FCM), support
vector regression (SVR), and semi-supervised fuzzy c-means (SFCM). In addition
to the 6 Sentinel-2 bands of blue, green, red, red edge 2, near-infrared, and
short wave infrared 1, two indices important spectral indices of NDVI and FDI
are utilised to develop the ML models. Their results suggest that SVR
outperforms all other ML models with around 97% of plastic detection accuracy.

Kremezi et. al. [35] leverage the potential of satellite hyper-spectra remote
sensing imagery in marine plastic litter detection purposes for the first time
in the literature. The authors use PRISMA satellite data with fine spectra but
low spatial resolutions. In order to increase the spatial resolution, [35]
proposes exploiting pansharpening with the panchromatic data that enhance
spatial resolution. In the experimental analysis, thirteen different
pansharpening methods have been tested and the PCA-based substitution method
managed to efficiently discriminate plastic targets from water-bodies.

A very recent study by Kikaki et. al. [36] introduce a benchmark Sentinel-2
dataset for developing and evaluating ML algorithms, called Marine Debris
Archive (MARIDA). Publishing the MARIDA data set will surely become a game-
changer in marine debris and floating plastic detection research since it is
the first dataset based on the multispectral Sentinel-2 satellite data. MARIDA
offers the capability to classify various different marine targets, such as
marine debris (MD), sargassum macroalgae, ships, wakes, foam, various
dissimilar water types, and clouds. The authors have gathered plastic
pollution information for the time interval between 2015-2021 from more than
11 different countries all around the world. Based on these ground truth data,
Sentinel-2 images collected and annotations have been done via using high-
resolution information from Planet and Google Earth imagery. MARIDA data
consists of 17/63 Sentinel-2 tiles/scenes, 2597 patches, and 837377 annotated
pixels. Among those pixels, MARIDA contains 3339 Marine Debris (MD) pixels in
total. In the MARIDA paper, MD was defined as ”floating plastic and polymers,
mixed anthropogenic debris”. Of these pixels, 1625 pixels were digitized and
annotated with high confidence, 1235 pixels were labeled with moderate, and
539 pixels with low confidence. Confidence levels were determined by the
collection of reports about mass-plastic events in coastal regions. All
satellite images were also investigated with very high-resolution satellite
imagery where possible. Overall, this increases the validity and makes MARIDA
the most comprehensive classification of marine debris to date. From the
technical point of view, MARIDA [36] also presents two different baseline
approaches based on RF (has three versions for (1) spectra, (2) spectra &
indices, and (3) spectra & indices & land cover map) and U-Net architecture.
The results indicate that RF forest approaches outperform U-Net results with
$F_{1}$ scores higher than 0.7 whilst U-Net achieves 0.5. MARIDA data class
distributions and an example performance result for the benchmark models are
depicted in Figure 6.

Figure 6: MARIDA data set content statistics and example results for the
benchmarking. (a) MARIDA data set class distributions. (b) An example
performance result for the MARIDA benchmark models

Kremezi et. al. in their recently published paper [37] propose to enhance the
capabilities of Sentinel-2 multi-spectral imagery via performing image fusion
approaches with very high-resolution WorldView-2/3 (WV-2/3) images. Their
approach has the primary aim to present solutions to the main limitation in
the existing approaches that limited spatial resolution of Sentinel-2
hindering the detection of relatively small marine debris and floating
plastics. Various image fusion techniques have been tested in [37] in terms of
preserving spectral and spatial information. Among those, Coupled non-negative
matrix factorization (CNMF) has shown to be the best via producing a fused
image with clear edges, no blurring, and favorable spectral characteristics.
Utilised Fusion-GAN and Fusion-ResNet approaches have also shown great
performance for fusion purposes regarding spectral similarity. The superior
performance throughout the paper can be listed as the reduction in the
required pixel coverage. The smallest detectable target via the best approach
in the fused image was noted as 0.6 × 0.6 m2 in size, which is equivalent to
3% pixel coverage of the original Sentinel-2 imagery with 20m resolution.
Despite this notable impact and potential future research direction, their
plastic detection approach consists of a high number of false detections. This
is actually not special to this specific paper, but nearly all the existing
approaches are suffering from false detections which reduces the accuracy and
reliability of the detection approach. Considering proposing a detection
approach is not the aim of [37], this paper can be noted as a breakthrough for
this research area for future works.

Lavender [38] proposes a detection approach using satellite imagery, but this
time not only for marine, also for terrestrial environments. This work also
produces a data set by manually digitising some of the classes in the land
cover maps into plastics, greenhouses, tyres, and waste sites. This data
consists of around 1.3 million pixels only around 1% of which is labeled as
plastics. The proposed detection algorithm is fed by two different modalities
of SAR (Sentinel-1) and multi-spectral (Sentinel-2) imagery and consists of an
artificial neural network (ANN). An RF-based algorithm has also been used to
compare the ANN performance. Except for the proposed data set, maybe the most
important contribution of this paper in terms of the computational image
processing approaches is to propose a post-ANN decision tree. This adds a
second classification step to the proposed method in order to fix errors
caused by the ANN’s incapability to fully capture the radiometric
interpretation of the surfaces and to process categorical layers. The post-ANN
decision tree and an example before and after this has been shown in Figure 7.
Regarding the detection performance, despite having around 95% of aggregate
average precision, their plastic detection precision is 91%.

Figure 7: (a) Post-ANN decision tree structure of [38]. (b) Comparison of the
classification results before and after the post-network decision tree for the
subset of the Solo River.

Booth et. al. in their work [39] propose a high-precision marine debris
mapping algorithm by utilising Sentinel-2 imagery and a semantic segmentation
algorithm. Their approach is constructed to detect marine debris and floating
plastics from a high-precision point of view. Maybe, in the literature, it is
the first time, this approach focuses on improving the precision of the marine
debris detection algorithm by keeping recall values high enough to produce a
reliable algorithm. The authors in [39] facilitate semi-automated monitoring
of suspected marine debris and floating plastics via a data pipeline named
MAP-Mapper that enables the user to provide coordinates for a region of the
ocean along with a date time range so that Sentinel-2 data could be
downloaded, pre-processed via atmospheric corrections and individual pixels
could then be classified using the proposed ML algorithm. Afterwards, as of
the outcome of this tool, a plastic density map is produced and enables users
to easily interpret model predictions and identify areas of high plastic
density in the area being investigated. Their approach promotes utilising the
MARIDA data within a U-Net-inspired machine learning approach. Their approach
MAP-Mapper-HP reach 95% precision of marine debris class in MARIDA data set
whilst the original U-net based baseline in [36] was only 30%. In order to
test their approach’s applicability, they set three out-of-distribution test
locations of the Bay of Honduras, Cornwall, and Lesvos via the data pipeline.
Their generated marine debris maps for these test locations are aligned with
the existing marine debris gathering information and ground truths. In Figure
8, an example marine density map for the Bay of Honduras is depicted.

Figure 8: Marine Debris Density Map from [39] for the Bay of Honduras. The
density map shows results for a five-month time interval from 01/03/2022 to
01/08/2022. Coloured dots show detected marine debris pixels, whilst the
colour of each hexagonal bin refers to the detected number of cases within
that bin.

##  3 Literature Statistics

This Chapter is going to highlight the evolution of marine debris research
with numbers in terms of literature statistics. Our analysis will follow a
gradual point of view - from general to special - in three cases that titles,
abstracts, and/or keywords include the words:

  1. 1.

(”Marine Debris”),

  2. 2.

(”Marine Debris”) AND (”Remote Sensing” OR ”Satellite” OR ”Sentinel”),

  3. 3.

(”Marine Debris” OR ”Plastic”) AND (”Remote Sensing” OR ”Satellite” OR
”Sentinel”) AND (”Machine Learning” OR ”Deep Learning” OR ”Artificial
Intelligence”)

All the statistical analyses are performed via the document search option in
Scopus (www.scopus.com). This means literature pieces that are not indexed via
Scopus are out-of-the-scope of this analysis.

###  3.1 Marine Debris Research

(a) Country based percentages

(b) Yearly trend

(c) Subject area percentages

(d) Source title based percentages

Figure 9: Data visualisation for the literature statistical analysis.

Marine debris research dates back to the 80s according to our analysis which
can be seen in Figure 9-(1). Even though the number of papers is quite low, it
can be taken as evidence of how old marine debris problems and attracted
attention in academia. This slow trend continues still for nearly 30 years
until the 2010s. 2010 is a critical point since 90% of the papers published
after 2010 in around 12 years. 2019+ is another critical year that 50% of the
works have been published during the last 4 years of time. The main important
reason for this is the improved technology in both remote sensing satellite
products and artificial intelligence research.

In terms of the subject area, three important areas appear to lead this
research, that is Environmental Science, Agricultural and Biological Sciences,
and Earth and Planetary Sciences. United States of America (USA) appears to be
the leading country in this research via populating approximately 30% of the
papers whilst Marine Pollution Bulletin becomes the leading academic source
title via accommodating more than 35% of the published papers.

###  3.2 Marine Debris Research via Remote Sensing

When we tighten our search keys by adding remote sensing and some other
related terms, Scopus returns 109 academic outcomes. Compared to Case 1, there
is a dramatic decrease in the number of papers. The main reason is we now
specifically look at marine debris research that promotes utilisation of
remotely sensed data, such as RGB, optical, and radar imagery. Along with the
remote sensing research, Case 1 also includes various important marine debris
research in areas of biology, material sciences, chemistry, etc. Now, most of
these research are excluded unless they are promoted using remote sensing
imagery. It looks like Scopus only indexed 5 works within these keywords
before 2010, which makes it again an important milestone date for marine
debris research via remote sensing imagery. Similarly, around 60% of the
papers that are highlighted in this case was published in 2019 and thereafter.

The three important subject areas which become prominent for Case 1 are still
the best three subjects. However, this time, adding remote sensing details
into the play made Earth and Planetary Sciences research area the leader
naturally. The USA is still the leading country in this research with more
than 40% papers published by US institutions. Similarly, the Marine Pollution
Bulletin ( 30%) journal is a far more widespread academic venue high-likely to
see marine debris research via remote sensing imagery.

###  3.3 Marine Debris Research via Remote Sensing Computational Imaging

Case 3 aims to highlight research at the intersection of oceanology, earth
sciences, and computer science. We can also mention that the research
highlighted under Case 3 is the target research of this review paper. Adding
deep/machine learning keywords into the search criteria basically returns us
to remote sensing computational imaging approaches that aim to provide
solutions to marine debris and floating plastics problems. Our Scopus search
returns 56 academic outcomes for Case 3 where the first of which was published
in 2016. Computer vision research and the AI revolution can be dominantly felt
in the marine debris research area, especially during the last 2 years in
which around 65% of the works were published.

Earth and Planetary Sciences research area is still the leading area whilst
Computer science and Environmental Sciences follow it with a high amount of
work. There are two important breaking points different from the statistical
analyses for Cases 1 and 2. In this case of research, China has become the new
leading country whereas USA institutions still publish around 20% of the
papers. It is also important to note that European institutions in Italy,
Germany, Spain, and Greece have started to direct this research area by
publishing a total of 37.5% of the papers. Taking AI into play made The MDPI
Remote Sensing journal (a peer-reviewed, open-access journal about the science
and application of remote sensing technology) the leading academic venue for
this research whilst Marine Pollution Bulletin and the International
Geoscience And Remote Sensing Symposium (IGARSS) follows with a similar number
of publications.

###  3.4 Overall Analysis

Finally, despite all the efforts statistically mentioned above, marine debris
research via remote sensing computational imaging is still in the early
stages. In order to understand this, we share two important Scopus searches
and the numbers will say it all!

When we remove (”Marine Debris” OR ”Plastic”) keys from Case 3, Scopus returns
23258 academic outcomes. This clearly means that the marine debris research
that promotes remote sensing computational imaging approaches is only 0.24%
within the whole research area. When we go one step down by excluding some
non-marine related research from this analysis and replacing (”Marine Debris”
OR ”Plastic”) key in Case 3 with (”Marine” OR ”Maritime” OR ”Ocean” OR
”Water”), Scopus returns 3574 academic outcome leads only to 1.57% of the
research in this area promotes remote sensing and computer vision approaches
for marine debris monitoring purposes.

##  4 Challenges

This chapter draws a picture regarding the potential challenges in marine
debris and floating plastics detection research area from the point of view of
the authors. We are going to reflect on our understanding and experience
within this chapter and please note that the challenges reported below are not
exhaustive.

###  4.1 Optical Imagery Limitations

As we explained in the previous literature review that the works highlighted
mostly focus on optical multi-hyper spectral imagery, which provides a great
amount of information whilst having some serious disadvantages at the same
time. There are several reasons that can be listed as potential challenges for
passive remote sensing systems in marine plastic pollution monitoring.

  1. 1.

As mentioned in the above discussion, even though providing data with 13
different spectral bands, the current gold modality Sentinel-2 imagery has up
to 10 meters of spatial resolution. This significantly influences the
detection capability of methods since the amount of plastic in a single pixel
determines the light intensity and defines a lower size limit for floating
plastics, which are around 30-50 m2. This lower limit of plastic is quite high
considering that the ocean generally has various pollutants combined in a
single patch. The sub-pixel indexes (the FDI of [8]) can be thought of as a
solution to this problem, but it is still open to improvements for higher
performance. Image fusion approaches that exploit high-resolution data along
with the Sentinel-2 information will be highly important future research
direction as Kremezi et. al. [37] have given an example for this via utilising
a similar approach which leads to a reduced required pixel coverage of 0.6 m
by 0.6 m.

  2. 2.

In addition to the previously mentioned fact that optical images are prone to
cloud cover and incapable of night-time data generation, optical remote
sensing also suffers from lower sampling rates (say high revisiting times).
All these reasons limit their capability to collect continuous data,
specifically for plastic patches tracking applications. To the best of our
knowledge, in the literature, there is a limited number of works that have
studied the problem of tracking plastic patches in the oceans. There are
several but non-exhaustive reasons behind this, which are required to be
addressed.

    1. (a)

In addition to the incapability to obtain continuous optical imagery for
tracking purposes, due to atmospheric limitations of the optical sensors, the
low spatial resolution of optical sensors also causes a risk in tracking
individuals of plastic bits.

    2. (b)

Considering the floating plastics are drifting due to winds, sea waves, and
currents, developing a robust tracking approach requires modeling these
hydrological variables. Since optical sensor wavelengths are not capable of
covering the Bragg scattering mechanism, imaging hydrological variables such
as gravity waves, swell waves, and ocean currents cannot be possible.

  3. 3.

The limitations of the optical satellite remote sensing data have led the
plastic detection research somehow to unmanned air vehicle (UAV) based
approaches. Although these works suggest important outcomes for the
understanding of the remote sensing imaging of plastic patches [12, 11, 20],
they are operator and location dependant whilst having high costs and lack of
standardisation.

###  4.2 Data Related Problems

In addition to the sensor-related challenges mentioned above, there is also
another important problem causing this research area to move slower compared
to other computational imaging areas; that is the data availability and other
data-related problems.

The very first one is the limited number of open-access marine debris and
plastic detection data sets. To the best of our knowledge, before 2022, there
was only one available data set from Topouzelis et. al. [19] under several
repetitions of plastic detection contests. Despite providing a great source of
information for future studies as is mentioned in the above chapters, the
shared data set consists of a couple of Sentinel-2 imagery that includes only
a handful of annotated plastic pixels. Thus, its applicability in advanced
remote sensing computational imaging approaches naturally remained limited.
Thanks to the efforts in this area, during 2022 (very recently) two important
data sets have been published [36, 38]. These data sets, even though still
suffering from the number of annotated plastic pixels, provide a basis for
researchers around the world and we believe that the applicability of the
advanced deep-machine learning approaches will be dramatically increased
thanks to these two data sets.

As of now, despite having several data sets, we still face an important
problem, which is the limitation of annotated plastic and marine debris
pixels. Due to their locations in the ocean, and accessibility issues, having
ground truth information with a high level of confidence is a great challenge
for this research area. The MARIDA data set [36], despite having around 0.9
million annotated pixels, only 0.4% of those belong to the marine debris
class. Among those, only half of these pixels (1625 pixels reported in [36])
are annotated as high confidence which makes the reliable information in the
data set even smaller. This limited information causes a lack of learning for
the advanced algorithms and we believe this is the reason that nearly all the
proposed approaches suffer from low precision, in other words, lots of false
plastic/marine debris detections.

Lastly, both having a limited number of data sets and a small amount of high-
confidence annotated pixels, advanced computer vision algorithms have not been
utilised for the purpose of detecting floating plastics and marine debris.
Instead, most of the referenced works above rely on classical approaches such
as Random Forests, Naive Bayes, or simple machine learning architectures based
CNNs.

##  5 Future Research Directions

After highlighting several important challenges in this area in the above
section, now, we share potential future research directions.

###  5.1 Utilisation of SAR Imagery

Considering the limitations and challenges mentioned above, it should also be
admitted that the detection and monitoring of plastic pollution via remote
sensing imagery are still in their early stages, especially in terms of multi-
modality and promoting other sensor information. All the efforts in the last
decade, lead to the development of pioneering works mentioned above. Optical
imagery, though being the modality that displays the earth as the human eye
sees it, will, unfortunately, be at the mercy of the clouds and the amount of
sunshine. However, SAR system constitutes an active sensor, which emits
microwaves towards the Earth and receives the back-scattered signals from the
surface. Since SAR utilises larger wavelengths (1cm to 1m) compared to the
optical sensors (near-visible light or 1 micron), it is able to image at both
day and night, and in almost all-weather conditions through clouds and storms
(while optical sensors are not). SAR imagery provides us with useful
information about forest biomass, and crop cover types since their ability to
sensitively discriminate against the target surface. Furthermore, SAR images
also provide useful information on ocean state, including wind speed and
direction, gravity waves, swells, currents, sea-ice structures, and
meteorological and environmental conditions in bodies of water. Thanks to its
aforementioned capabilities, SAR imagery recognised as the gold standard and
superior to the optical imagery where the literature abounds various
applications which utilise SAR imagery as the main source of information, such
as detecting/tracking oil spills [40, 41, 42, 43, 44, 45], assessment of sea
surface signatures [46, 47, 48, 49], ship detection and tracking [50, 51, 52,
53, 54], and ice sheets tracking.

SAR imagery has various advantages compared to the passive remote sensing
modalities, which can be listed below:

  1. 1.

Several European missions (e.g., the Italian COSMO/SkyMed, the German
TerraSAR-X, the UK’s NovaSAR-1, or the Finnish ICEYE) have developed a new
generation of satellites exploiting SAR to provide spatial resolutions
previously unavailable from space-borne remote sensing (up to 0.25 meters).
Investigating cm-scale pixel sizes via remote sensing modalities is crucially
important to detect and classify the plastic pollutant since these spatial
resolutions are not suitable for optical remote sensing, and operating UAV-
based approaches with cm-scale resolutions are extremely expensive, operator
and location dependent. Conversely, high-resolution SAR data is continuously
obtained for some other monitoring applications regardless of our intention to
use it. Taking these high-potential, yet less-demanded, sources into play for
plastic detection yields crucial importance for the future role of remote
sensing technologies in plastic pollution monitoring.

  2. 2.

New SAR technology provides larger swath-ranges compared to optical imagery.
This is one of the reasons that makes SAR preferable to optical imagery,
specifically for monitoring applications.

  3. 3.

Thanks to its day-night imaging capability, SAR provides up to 12 hours of
temporal resolution whilst the Sentinel-2 optical imagery has a revisiting
time of only 2 to 5 days. High-sampling rates, high temporal resolution, or
smaller revisiting intervals can be listed as the primary need for
monitoring/tracking plastic patches in the ocean. Utilising this source of
information would help us to develop robust tracking algorithms. On the other
hand, this also will serve the community with the understanding regarding how
the plastic patches behave in the open ocean, and why places such as the
remote islands at the Pacific Ocean like Henderson Island becomes a gathering
point for ocean plastics [10].

  4. 4.

SAR sensors operate with various frequency bands (P-S-L-C-X), and each of
these bands has various advantages compared to the others. As we mentioned
above, SAR has been analyzed in a limited number of works for plastic
detection and recognition in the literature to date. However, these analyses
are limited only to Sentinel-1 SAR imagery, which images C-band data with 10-m
spatial resolution. Since (i) different frequency bands will therefore develop
different interactions between the plastic pollutant and the sea surface
waters, and (ii) different penetrable frequency bands such as S (NovaSAR-1), L
(ALOS2) and X (TerraSAR-X, ICEYE, etc.) have never been utilised in these
analyses, SAR imagery is yet unexplored regarding its plastic detection
capability. An example demonstration of the penetration capability of
different SAR frequency bands is shown in Figure 4. This demonstration shows
the L-band penetration capability to e.g., determine forest floods. To date, a
similar analysis has not been studied for sea surfaces including plastics and
various other natural materials.

Figure 10: SAR frequency band penetration example [55].

Drawbacks: Despite the advantages compared to optical imagery, an important
problem degrading statistical inference from SAR imagery is the presence of
multiplicative speckle noise, which is not the case in optical imagery. The
received back-scattered signals sum up coherently and then undergo nonlinear
transformations. This in turn causes a granular look in the resulting images,
which is referred to as speckle noise [56]. Speckle noise may lead to the loss
of crucial details in SAR images and cause problems in the processing of these
images such as in feature detection, segmentation, or classification. However,
speckle noise is a well-studied problem for decades and can be dealt with and
removed by precisely determining the statistical characteristics of SAR images
as in [57, 58, 59, 60, 56]. Despite its well-studied nature, it is also a
potential problem of SAR imagery utilisation in marine debris monitoring since
the interaction between plastic reflection and speckle noise has not yet been
reported.

###  5.2 Applicability of New Generation Satellites

Mentioning the drawbacks of optical multi/hyperspectral imagery above does not
specifically mean that their era has already passed. Apart from the new
technology satellites mentioned above for SAR imagery, in the last couple of
years, new technology satellites for optical imagery have also been launched.
Worldview-3 which is an imaging and environment-monitoring super-spectral,
high-resolution commercial satellite sensor from Maxar, can be named among
those technologies. Even though it has 8 years of history, due to being a
commercial satellite, its usage has been limited to various remote sensing
applications let alone marine debris monitoring. This instrument collects
images at 0.31 meter panchromatic and 1.24 meter in the eight VNIR bands, 3.7m
in the eight SWIR bands whilst having additional bands for enhanced multi-
spectral analysis (coastal blue, yellow, red edge, NIR2) designed to improve
segmentation and classification of land and aquatic features. Since
Worldview-3 products started to be delivered by the ESA, their applicability
has increased, and recently researchers reduced the smallest detectable marine
debris target to 0.6 × 0.6 m2 in size, which is equivalent to 3% pixel
coverage of the original Sentinel-2 imagery with 20m resolution. We believe
satellite products like Worldview-3 multi-spectral imagery will direct
following years of research for floating plastics and marine debris monitoring
via filling the gaps in low Spatio-temporal resolution of the current gold
standard - the Sentinel-2 imagery products. An example of WV-3 and S-2
comparison for the same area is given in Figure 11.

Figure 11: (left) RGB WV-3 image with 4 m spatial resolution, (right) S2 image
with 10 m spatial resolution. [37].

###  5.3 Computational Imaging Solutions

As we mentioned previously, this paper aims to review and discuss current
advances in computational remote sensing imagery for the proposes of marine
debris and floating plastics detection. In Section 2, we specifically
discussed computational imaging approaches utilised for this purpose, the
conclusion leads to an important fact that existing approaches mostly rely on
classical machine learning approaches such as Random Forests, Naive Bayes, and
straightforward deep neural network structures. However, one of the most
active research areas of deep learning and artificial intelligence is computer
vision and highly advanced approaches are being developed nearly every day.
Despite this fast-changing and active nature of computer vision, remote
sensing part of it still coming from several years back. Several reasons for
this have been highlighted above as a lack of open access data sets, and a low
amount of annotated plastic pixel existence.

The first potential computational imaging research direction for marine debris
monitoring that would be highly active in the following years is multi-modal
AI and multi-sensor image fusion approaches. Various environmental
applications have benefited from multi-modal image fusion by exploiting
complementary features provided by different types of remote sensors.
Specifically, for active-passive sensor data fusion, passive-optical sensors
play the role of feeding the system with a high spectral resolution of the
earth’s surface which are useful for image analysis whilst active remote
sensing sensors usually provide sufficient textural and structural information
of observed objects [61]. Some example remote sensing applications that
exploit multi-sensor information and obtain considerable performance increase
can be listed as land use/cover mapping [62], air pollution detection [63],
building footprint extraction [64], and maritime vessel detection [65].

Another computational imaging research direction will be guided by the fact
that the marine debris monitoring area is suffering from a low amount of
annotated data. The same existing fact has been a great driving force for the
development of semi-, weakly-, and self-supervised learning (reduced
supervision or minimal supervision) computational imaging approaches
especially in the medical imaging area, which has the same annotated data
problem as remote sensing imagery applications. Along with the advances in
multi-modal fusion approaches, developing minimal supervision techniques for
marine debris monitoring is a crucial step, and we believe academic literature
will gradually get involved in this area in order to leverage not annotated
data simultaneously with the limited amount of high-confidence marine debris
pixels. The weakly supervised approach proposed in [36] is a good starting
point for this, but of course not enough for high-precision marine debris
monitoring software development tools. Ma et. al. in their land cover mapping
application paper [62] have shown that without losing much performance
accuracy, multi-modal sensor information usage in a parallel manner can reduce
the amount of required training data up to 1/20. We believe this work, and
other similar minimal supervision-promoted remote sensing computational
imaging approaches in the literature might guide researchers in the following
years.

###  5.4 Tracking the Source of Pollution

To the best of our knowledge, the literature for marine plastic pollution
monitoring does not yet have a complete tracking method due to the
incapability of continuous data flow from optical sensors. On the other hand,
resolution limitations and small sizes of plastic bits can be listed as other
reasons for not developing a complete set of tracking approaches. The tracking
literature is mostly dominated by the Bayesian approaches, especially for the
last couple of decades that are based on the sequential Monte Carlo (SMC)
algorithm. There have been proposed various advanced versions of this
algorithm, though the SMC-based tracking approaches still suffer from data
degeneracy and high particle variance problems. Currently, optimal transport
theory-based solutions are getting prominent in tracking approaches [66, 67,
68]. However, there is no such advanced methodology developed for remote
sensing imagery so far. Thus, the marine debris tracking literature has an
open and hot topic via the improved and novel version of the sequential Monte
Carlo (SMC) algorithm, whilst leveraging the optimal transport theory to
mitigate the effects of high-variance particles. Additionally, exploiting the
utilisation of irreversible Bayesian samplers to improve the mixing properties
and convergence speed of Bayesian plastic tracker algorithms can be listed as
another important research direction.

Detecting where marine debris and floating plastics are located and tracking
where they are heading are of crucial importance, however, there is one more
important aspect about this: the detection of the sources of the plastic.
Detecting and acting to clean plastic pollution from the oceans is an
important action but requires repetitive efforts as soon as we do not stop the
source of the pollution. Hence, the effort by Lavender [38] via proposing a
date set for plastic pollutants both on the land and coastal areas can be seen
as one of the starting points of this effort since analyzing the source of the
pollution requires discriminating the pollutants not only on land but also in
small inland waters such as rivers. Similarly, the work by Sasaki et. al. [69]
exploiting coastal debris detection via machine learning approaches and
utilising Maxar WV-3 imagery accommodates the same importance in order to
clearly discriminate the debris on the sea surface and on Land.

As mentioned in the above sections, floating plastics’ trajectory in the ocean
has highly been affected by ocean surface signatures and currents. Combining
SAR and optical imagery information with the ground-breaking statistical
samplers for tracking approaches will make analysing ocean currents, and wave
structures possible, which can lead to developing back-tracking approaches to
predict the sources of plastic pollution. Lastly, In order to mitigate the
drawbacks of small-sized plastic bits and relatively low spatial resolution
remote sensing data, group (or namely cloud) tracking-based approaches [70,
71] are of great importance.

##  6 Conclusions

The ocean and waters of the planet Earth have been facing various threads
during the last couple of decades. Whilst Global warming-related problems and
some power-holders ignorance about it are still there, the Earth also dealing
with pollution problems in the ocean, air, etc. Along with the technological
developments, especially in the machine learning and artificial intelligence
area in the last decade, computer vision approaches started to play some
crucial roles to guide initiatives and decision-makers to act for the
aforementioned problems. Marine debris and floating plastics are one of those
problems, and remote sensing computational imaging efforts can help us to hear
the call of the ocean!

This paper via presenting a review of the academic efforts in the literature
aims to highlight developed computational remote sensing imaging approaches
for the purpose of detecting marine debris and floating plastics. After
carefully discussing each highly valuable academic outcome, we also
highlighted the literature statistics for marine debris monitoring purposes in
the last decade. Finally, depending on the authors’ experience and research on
marine-related remote sensing computational imaging approaches, various
challenges, and potential future research directions have been listed (non-
exhaustive).

## References

  * [1] Jim Leape, Our oceans are in crisis. Here’s how technology could save them (Jan 2018).
URL www.weforum.org//agenda//2018//01//our-oceans-are-in-crisis-heres-how-
technology-could-save-them//

  * [2] M. Farré, Remote and in-situ devices for the assessment of marine contaminants of emerging concern and plastic debris detection, Current Opinion in Environmental Science & Health (2020).
  * [3] L. Lebreton, B. Slat, F. Ferrari, B. Sainte-Rose, J. Aitken, R. Marthouse, S. Hajbane, S. Cunsolo, A. Schwarz, A. Levivier, et al., Evidence that the Great Pacific Garbage Patch is rapidly accumulating plastic, Scientific reports 8 (1) (2018) 1–15.
  * [4] T. T. O. Aquarium, The plastic problem: How does plastic pollution affect wildlife? (2020).
URL https://www.aquarium.co.za/blog/entry/the-plastic-problem-how-does-
plastic-pollution-affect-wildlife

  * [5] WWF, No plastic in nature: Assessing plastic ingestion from nature to people (2019).
URL
https://wwfint.awsassets.panda.org/downloads/plastic_ingestion_web_spreads_1.pdf

  * [6] WWF, Plastic in our oceans is killing marine mammals (2018).
URL https://www.wwf.org.au/news/blogs/plastic-in-our-oceans-is-killing-marine-
mammals#gs.v45fly

  * [7] The NOAA, Discover marine debris (2022).
URL https://marinedebris.noaa.gov/discover-marine-debris

  * [8] L. Biermann, D. Clewley, V. Martinez-Vicente, K. Topouzelis, Finding plastic patches in coastal waters using optical satellite data, Scientific reports 10 (1) (2020) 1–10.
  * [9] M. Thiel, I. A. Hinojosa, T. Joschko, L. Gutow, Spatio-temporal distribution of floating objects in the German Bight (North Sea), Journal of Sea Research 65 (3) (2011) 368–379.
  * [10] J. L. Lavers, A. L. Bond, Exceptional and rapid accumulation of anthropogenic debris on one of the world’s most remote and pristine islands, Proceedings of the National Academy of Sciences 114 (23) (2017) 6052–6055.
  * [11] S. P. Garaba, H. M. Dierssen, An airborne remote sensing case study of synthetic hydrocarbon detection using short wave infrared absorption features identified from marine-harvested macro-and microplastics, Remote Sensing of Environment 205 (2018) 224–235.
  * [12] K. Moy, B. Neilson, A. Chung, A. Meadows, M. Castrence, S. Ambagis, K. Davidson, Mapping coastal marine debris using aerial imagery and spatial analysis, Marine pollution bulletin 132 (2018) 52–59.
  * [13] K. Themistocleous, C. Papoutsa, S. Michaelides, D. Hadjimitsis, Investigating detection of floating plastic litter from space using Sentinel-2 imagery, Remote Sensing 12 (16) (2020) 2648.
  * [14] R. De Vries, M. Egger, T. Mani, L. Lebreton, Quantifying floating plastic debris at sea using vessel-based optical data and artificial intelligence, Remote Sensing 13 (17) (2021) 3401.
  * [15] T. Aoyama, Monitoring of marine debris in the Sea of Japan using multi-spectral satellite images, in: Ocean Remote Sensing and Monitoring from Space, Vol. 9261, International Society for Optics and Photonics, 2014, p. 92611E.
  * [16] T. Aoyama, Extraction of marine debris in the Sea of Japan using high-spatial-resolution satellite images, in: Remote Sensing of the Oceans and Inland Waters: Techniques, Applications, and Challenges, Vol. 9878, International Society for Optics and Photonics, 2016, p. 987817.
  * [17] J. P. Matthews, L. Ostrovsky, Y. Yoshikawa, S. Komori, H. Tamura, Dynamics and early post-tsunami evolution of floating marine debris near Fukushima Daiichi, Nature Geoscience 10 (8) (2017) 598–603.
  * [18] L. Goddijn-Murphy, S. Peters, E. Van Sebille, N. A. James, S. Gibb, Concept for a hyperspectral remote sensing algorithm for floating marine macro plastics, Marine pollution bulletin 126 (2018) 255–262.
  * [19] K. Topouzelis, D. Papageorgiou, A. Karagaitanakis, A. Papakonstantinou, M. Arias Ballesteros, Remote Sensing of Sea Surface Artificial Floating Plastic Targets with Sentinel-2 and Unmanned Aerial Systems (Plastic Litter Project 2019), Remote Sensing 12 (12) (2020) 2013.
  * [20] K. Topouzelis, A. Papakonstantinou, S. P. Garaba, Detection of floating plastics from satellite and unmanned aerial systems (Plastic Litter Project 2018), International Journal of Applied Earth Observation and Geoinformation 79 (2019) 175–183.
  * [21] V. Martínez-Vicente, J. R. Clark, P. Corradi, S. Aliani, M. Arias, M. Bochow, G. Bonnery, M. Cole, A. Cózar, R. Donnelly, et al., Measuring marine plastic debris from space: initial assessment of observation requirements, Remote Sensing 11 (20) (2019) 2443.
  * [22] A. Kikaki, K. Karantzalos, C. A. Power, D. E. Raitsos, Remotely sensing the source and transport of marine plastic debris in Bay Islands of Honduras (Caribbean Sea), Remote Sensing 12 (11) (2020) 1727.
  * [23] Y.-J. Park, S. P. Garaba, B. Sainte-Rose, Detecting the Great Pacific Garbage Patch floating plastic litter using WorldView-3 satellite imagery, Optics Express 29 (22) (2021) 35288–35298.
  * [24] P. M. Salgado-Hernanz, J. Bauzà, C. Alomar, M. Compa, L. Romero, S. Deudero, Assessment of marine litter through remote sensing: recent approaches and future goals, Marine Pollution Bulletin 168 (2021) 112347.
  * [25] C. Hu, Remote detection of marine debris using satellite observations in the visible and near infrared spectral range: Challenges and potentials, Remote Sensing of Environment 259 (2021) 112414.
  * [26] A. C. Ciappa, Marine Litter Detection by Sentinel-2: A Case Study in North Adriatic (Summer 2020), Remote Sensing 14 (10) (2022) 2409.
  * [27] T. Acuña-Ruz, D. Uribe, R. Taylor, L. Amézquita, M. C. Guzmán, J. Merrill, P. Martínez, L. Voisin, C. Mattar, Anthropogenic marine debris over beaches: Spectral characterization for remote sensing applications, Remote Sensing of Environment 217 (2018) 309–322.
  * [28] L. Fallati, A. Polidori, C. Salvatore, L. Saponari, A. Savini, P. Galli, Anthropogenic Marine Debris assessment with Unmanned Aerial Vehicle imagery and deep learning: A case study along the beaches of the Republic of Maldives, Science of The Total Environment 693 (2019) 133581.
  * [29] S. Kako, S. Morita, T. Taneda, Estimation of plastic marine debris volumes on beaches using unmanned aerial vehicles and image processing based on deep learning, Marine Pollution Bulletin 155 (2020) 111–127.
  * [30] G. Jakovljevic, M. Govedarica, F. Alvarez-Taboada, A deep learning model for automatic plastic mapping using unmanned aerial vehicle (UAV) data, Remote Sensing 12 (9) (2020) 1515.
  * [31] C. van Lieshout, K. van Oeveren, T. van Emmerik, E. Postma, Automated river plastic monitoring using deep learning and cameras, Earth and space science 7 (8) (2020) e2019EA000960.
  * [32] S. Freitas, H. Silva, E. Silva, Remote hyperspectral imaging acquisition and characterization for marine litter detection, Remote Sensing 13 (13) (2021) 2536\.
  * [33] P. Tasseron, T. Van Emmerik, J. Peller, L. Schreyers, L. Biermann, Advancing floating macroplastic detection from space using experimental hyperspectral imagery, Remote Sensing 13 (12) (2021) 2335.
  * [34] B. Basu, S. Sannigrahi, A. Sarkar Basu, F. Pilla, Development of novel classification algorithms for detection of floating plastic debris in coastal waterbodies using multispectral Sentinel-2 remote sensing imagery, Remote Sensing 13 (8) (2021) 1598.
  * [35] M. Kremezi, V. Kristollari, V. Karathanassi, K. Topouzelis, P. Kolokoussis, N. Taggio, A. Aiello, G. Ceriola, E. Barbone, P. Corradi, Pansharpening prisma data for marine plastic litter detection using plastic indexes, IEEE Access 9 (2021) 61955–61971.
  * [36] K. Kikaki, I. Kakogeorgiou, P. Mikeli, D. E. Raitsos, K. Karantzalos, MARIDA: A benchmark for Marine Debris detection from Sentinel-2 remote sensing data, PloS one 17 (1) (2022) e0262247.
  * [37] M. Kremezi, V. Kristollari, V. Karathanassi, K. Topouzelis, P. Kolokoussis, N. Taggio, A. Aiello, G. Ceriola, E. Barbone, P. Corradi, Increasing the Sentinel-2 potential for marine plastic litter monitoring through image fusion techniques, Marine Pollution Bulletin 182 (2022) 113974.
  * [38] S. Lavender, Detection of Waste Plastics in the Environment: Application of Copernicus Earth Observation Data, Remote Sensing 14 (19) (2022) 4772.
  * [39] H. Booth, W. Ma, O. Karakus, High-precision density mapping of marine debris and floating plastics via satellite imagery, arXiv preprint arXiv:2210.05468 (2022).
  * [40] C. Brekke, A. H. Solberg, Oil spill detection by satellite remote sensing, Remote sensing of environment 95 (1) (2005) 1–13.
  * [41] K. Karantzalos, D. Argialas, Automatic detection and tracking of oil spills in SAR imagery with level set segmentation, International Journal of Remote Sensing 29 (21) (2008) 6281–6296.
  * [42] A. Kostianoy, K. Litovchenko, O. Lavrova, M. Mityagina, T. Bocharova, S. Lebedev, S. Stanichny, D. Soloviev, A. Sirota, O. Pichuzhkina, Operational Satellite Monitoring of Oil Spill Pollution in the Southeastern Baltic Sea: 18 Months Experience., Environmental Research, Engineering & Management 38 (4) (2006).
  * [43] Y. Li, J. Li, Oil spill detection from SAR intensity imagery using a marked point process, Remote Sensing of Environment 114 (7) (2010) 1590–1601.
  * [44] R. Shirvany, M. Chabert, J.-Y. Tourneret, Ship and oil-spill detection using the degree of polarization in linear and hybrid/compact dual-pol sar, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 5 (3) (2012) 885–892.
  * [45] Q. Zhao, H. Zhang, Y. Li, Detecting dark spots from SAR intensity images by a point process with irregular geometry marks, International Journal of Remote Sensing 40 (2) (2019) 774–793.
  * [46] O. Karakuş, E. E. Kuruoğlu, A. Achim, Modelling sea clutter in SAR images using Laplace-Rician distribution, in: ICASSP 2020 - IEEE International Conference on Acoustics, Speech and Signal Processing, 2020, pp. 1–5.
  * [47] O. Karakuş, I. Rizaev, A. Achim, Ship wake detection in SAR images via sparse regularisation, IEEE Transactions on Geoscience and Remote Sensing (2020).
  * [48] I. Rizaev, O. Karakuş, J. Hogan, A. Achim, Universal simulation of SAR imagery based on two-scale model and linear superposition of sea-ship waves, in preperation (2020).
  * [49] M. D. Graziano, M. D’Errico, G. Rufino, Wake component detection in X-band SAR images for ship heading and velocity estimation, Remote Sensing 8 (6) (2016) 498.
  * [50] R. Gao, S. Särkkä, R. Claveria-Vega, S. Godsill, Autonomous tracking and state estimation with generalised group Lasso, arXiv preprint arXiv:2007.11573 (2020).
  * [51] M. Kartal, O. Duman, Ship detection from optical satellite images with deep learning, in: 2019 9th International Conference on Recent Advances in Space Technologies (RAST), IEEE, 2019, pp. 479–484.
  * [52] J. Liang, B. I. Ahmad, R. Gan, P. Langdon, R. Hardy, S. Godsill, On destination prediction based on Markov bridging distributions, IEEE Signal Processing Letters 26 (11) (2019) 1663–1667.
  * [53] F. Mazzarella, A. Alessandrini, H. Greidanus, M. Alvarez, P. Argentieri, D. Nappo, L. Ziemba, Data fusion for wide-area maritime surveillance, in: Workshop on Moving objects at Sea, 2013.
  * [54] O. Pappas, A. Achim, D. Bull, Superpixel-Level CFAR Detectors for Ship Detection in SAR Imagery, IEEE Geoscience and Remote Sensing Letters 15 (9) (2018) 1397–1401.
  * [55] M. Ottinger, C. Kuenzer, Spaceborne L-Band Synthetic Aperture Radar Data for Geoscientific Analyses in Coastal Land Applications: A Review, Remote Sensing 12 (14) (2020) 2228.
  * [56] E. E. Kuruoglu, J. Zerubia, Modeling SAR images with a generalization of the Rayleigh distribution, IEEE Transactions on Image Processing 13 (4) (2004) 527–533.
  * [57] A. Achim, E. E. Kuruoglu, J. Zerubia, SAR image filtering based on the heavy-tailed Rayleigh model, IEEE Transactions on Image Processing 15 (9) (2006) 2686–2693.
  * [58] O. Karakuş, A. Achim, On solving SAR imaging inverse problems using non-convex regularization with a Cauchy-based penalty, IEEE Transactions on Geoscience and Remote Sensing (2020).
  * [59] O. Karakuş, E. E. Kuruoğlu, A. Achim, A generalized Gaussian extension to the Rician distribution for SAR image modeling, arXiv preprint arXiv:2006.08300 (2020).
  * [60] O. Karakuş, E. E. Kuruoğlu, M. A. Altınkaya, Generalized Bayesian model selection for speckle on remote sensing images, IEEE Transactions on Image Processing 28 (4) (2018) 1748–1758.
  * [61] H. Zhang, H. Shen, Q. Yuan, X. Guan, Multispectral and SAR Image Fusion Based on Laplacian Pyramid and Sparse Representation, Remote Sensing 14 (4) (2022) 870\.
  * [62] W. Ma, O. Karakuş, P. L. Rosin, AMM-FuseNet: Attention-Based Multi-Modal Image Fusion Network for Land Cover Mapping, Remote Sensing 14 (18) (2022) 4458\.
  * [63] L. Scheibenreif, M. Mommert, D. Borth, Toward Global Estimation of Ground-Level NO 2 Pollution With Deep Learning and Remote Sensing, IEEE Transactions on Geoscience and Remote Sensing 60 (2022) 1–14.
  * [64] J. Shermeyer, D. Hogan, J. Brown, A. Van Etten, N. Weir, F. Pacifici, R. Hansch, A. Bastidas, S. Soenen, T. Bacastow, et al., SpaceNet 6: Multi-sensor all weather mapping dataset, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020, pp. 196–197.
  * [65] F. Farahnakian, J. Heikkonen, Deep learning based multi-modal fusion architectures for maritime vessel detection, Remote Sensing 12 (16) (2020) 2509\.
  * [66] A. Corenflos, J. Thornton, G. Deligiannidis, A. Doucet, Differentiable particle filtering via entropy-regularized optimal transport, in: International Conference on Machine Learning, PMLR, 2021, pp. 2100–2111.
  * [67] J. Wu, L. Wen, P. L. Green, J. Li, S. Maskell, Ensemble Kalman filter based sequential Monte Carlo sampler for sequential Bayesian inference, Statistics and Computing 32 (1) (2022) 1–14.
  * [68] P. Hao, O. Karakus, A. Achim, Particle stochastic map filters, Available at SSRN 4129781 (2022).
  * [69] K. Sasaki, T. Sekine, L.-J. Burtz, W. J. Emery, Coastal marine debris detection and density mapping with very high resolution satellite imagery, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 15 (2022) 6391 – 6401.
  * [70] L. Mihaylova, A. Y. Carmi, F. Septier, A. Gning, S. K. Pang, S. Godsill, Overview of Bayesian sequential Monte Carlo methods for group and extended object tracking, Digital Signal Processing 25 (2014) 1–16.
  * [71] F. Septier, A. Carmi, S. Godsill, Tracking of multiple contaminant clouds, in: 2009 12th International Conference on Information Fusion, IEEE, 2009, pp. 1280–1287.


# Laser Field Initiation of Higher Order Poles of S-Matrix-Optical Realization
of Field Theoretic Models*

G.S. Agarwal1,2
1Physical Research Laboratory, Navrangpura, Ahmedabad, India
2Jawaharlal Nehru Center for Advanced Scientific Research, Bangalore, India

###### Abstract

We discuss the possibility of converting a simple pole in the radiative decay
of a state into a pole of higher order by using resonant electromagnetic
fields. This process of creation of higher order pole is controllable by the
intensity of the laser field. We use density matrix and Liouville space and
present the modification of the Lorentzian line shapes (Breit-Wigner formula)
for example to ones involving square of Lorentzian and derivatives of
Lorentzians.

##  I INTRODUCTION

In a classic paper Goldberger and Watson [1] considered the possibility that
the decay law for an unstable particle can be more complex than a simple
exponential. They showed the possibility of the existence of the poles of
S-matrix which were not necessarily simple poles. Since then, higher order
poles have been extensively studied. Recently, there is revival [2,3] of
interest in such studies and in particular Bhamathi and Sudarshan have
analyzed several field theoretic models like Friedrich-Lee model, cascade
model and their extensions. They examine the spectrum (complex) of eigenvalues
for such models. A related question is how the Breit-Wigner line shape formula
is modified if S-matrix possess higher order poles.

In this paper we examine the possibility of creation of the higher order poles
using laser fields. We consider the decay of say excited state of an atom.
Normally this decay is described by the Wigner-Weisskopf theory which leads to
exponential decay law. We next discuss the case when the excited state is
coupled to another state by a resonant electromagnetic field. In such a case
we show that for appropriate value of the intensity of the laser field the
corresponding spectral function has a pole of order two. We calculate the
resulting line shape and discuss the line narrowing etc. We emphasize that we
work within the framework of density matrices and hence we work in Liouville
space rather than in Hilbert space. We present optical realization of various
field theoretic models.

Consider the decay of the state $\ket{1}$ into the states $\ket{3}$ and
$\ket{2}$ at the rates 2$\gamma_{1}$ and 2$\gamma_{2}$ respectively as shown
in Fig.1 (with $G_{l}=0,\triangle_{l}=0$). It is well known that the rate of
decay of the population in $\ket{1}$ is given by

$\rho_{11}(t)=\rho_{11}(0)exp(-2(\gamma_{1}+\gamma_{2})t).$ (1)

Here $\rho$ is the density matrix of the atom. The spectrum of the
spontaneously emitted photons will consist of two Lorentzians centered at
$\omega_{13}$ and $\omega_{12}$ with a half width $(\gamma_{1}+\gamma_{2})$.
Let us concentrate on the emission on the transition
$\ket{1}\leftrightarrow\ket{3}$. The spectrum will be described by the well-
known form

$S(\omega)=\frac{\gamma_{1}/\pi}{(\gamma_{1}+\gamma_{2})^{2}+(\omega-\omega_{13})^{2}}.$
(2)

Note that $\gamma_{2}$ will be zero if the decay channel
$\ket{1}\rightarrow\ket{2}$ is not allowed. We will discuss how the laser
fields could be used to modify significantly the results predicted by (1) and
(2).

##  II LIOUVILLE SPACE FORMULATION OF DECAY

We next recall how the spectrum is calculated in the density matrix framework
[4]. We have included this material for completeness so that our discussion in
subsequent sections can be followed by the non-Quantum optics practitioners.
Consider a system with two states $\ket{1}$ and $\ket{3}$ interacting with the
vacuum of the electromagnetic field. The Hamiltonian can be written in the
form

$\displaystyle
H=\hbar\omega_{13}\ket{1}\bra{1}+\sum_{ks}\hbar\omega_{ks}a_{ks}^{\dagger}a_{ks}+V_{13}$
$\displaystyle V_{13}=\sum_{ks}(\hbar
g_{ks}a_{ks}^{\dagger}\ket{1}\bra{3}+h.c.).$ (3)

The vacuum modes are characterized by the propagation index
$\overrightarrow{k}$ and the polarization index $s$. The
$a_{ks},a_{ks}^{\dagger}$ represent annihilation and creation operators for
the mode $\overrightarrow{k}s$. The $V_{13}$ describes the decay of $\ket{1}$
to $\ket{3}$. The $g_{ks}$ is the coupling constant between the field mode and
the atom. We use the weak coupling assumption and the flat nature of the
density of states of the electromagnetic vacuum to eliminate the degrees of
freedom associated with the field vacuum. We derive an equation for the
density matrix of the atomic system alone which can be written in the form

$\frac{\partial\rho}{\partial t}=L\rho$ (4)

or in terms of the components as

$\displaystyle\dot{\rho}_{11}$ $\displaystyle=-2\gamma_{1}\rho_{11},$
$\displaystyle\dot{\rho}_{13}$
$\displaystyle=-i\omega_{13}\rho_{13}-\gamma_{1}\rho_{13},$
$\displaystyle\dot{\rho}_{33}$ $\displaystyle=2\gamma_{1}\rho_{11},\quad
etc.,\>2\gamma_{1}=\sum_{ks}|g_{ks}|^{2}\delta(\omega_{13}-\omega_{ks}).$ (5)

This yields steady state as well as transient behavior. The spectrum of
radiation is related to the Fourier transform of the two time dipole
correlation function, for example in the above case to

$S(\omega)=\frac{1}{\pi}Re[S(z)|_{z=+i\omega}],$ (6) $\displaystyle S(z)$
$\displaystyle\equiv\int_{0}^{\infty}d\tau e^{-z\tau}\left\langle
A_{13}(t+\tau)A_{31}(t)\right\rangle,$ $\displaystyle A_{13}$
$\displaystyle=A_{31}^{\dagger}=\ket{1}\bra{3}.$ (7)

The poles of $S(z)$ determine the spectrum. For the standard problem $S(z)$
has simple poles.

The two time correlation function is calculated from the solution of (4) and
by using the quantum regression theorem. For completeness, we state what it
means. We write the solution of (4) as

$\rho_{\alpha\beta}(t+\tau)=\sum_{m,n}G_{\alpha\beta,mn}(\tau)\rho_{mn}(t).$
(8)

It should be borne in mind that in the Liouville space $\rho_{\alpha\beta}$ is
an element of the column matrix. We can rewrite (8) as

$\left\langle
A_{\beta\alpha}(t+\tau)\right\rangle=\sum_{m,n}G_{\alpha\beta,mn}(\tau)\left\langle
A_{nm}(t)\right\rangle,$ (9)

then the quantum regression theorem leads to two time correlation function:

$\displaystyle\left\langle A_{\beta\alpha}(t+\tau)A_{pq}(t)\right\rangle$
$\displaystyle\equiv\sum_{m,n}G_{\alpha\beta,mn}(\tau)\left\langle
A_{nm}(t)A_{pq}(t)\right\rangle$
$\displaystyle=\sum_{m,n}G_{\alpha\beta,mn}(\tau)\left\langle
A_{nq}(t)\right\rangle\delta_{mp}$
$\displaystyle=\sum_{m,n}G_{\alpha\beta,mn}(\tau)\delta_{mp}\rho_{qn}(t).$
(10)

On using (10) in (6) it is clear that $S(z)$ is related to the Laplace
transform of $G(\tau)$ or to $(z-L)^{-1}$. Generally, the Liouvilliean matrix
relevant for the calculation of (10) decomposes in block diagonal form and
only a part of $L$ determines the decay or the spectral line shapes. For the
two level example, the correlation function is essentially determined by a
single equation for $\rho_{13}$. If there is more than one decay channel, then
additional terms appear in (5), for example, for the case shown in Fig.1,
$\gamma_{1}$ should be replaced by $(\gamma_{1}+\gamma_{2})$ in the two first
equations in (5).

Figure 1: Schematic illustration of the scheme that leads to the creation of
poles of order two in the decay of the state $\ket{1}$; which could be pumped
in two different ways either from the state $\ket{3}$ or from a state outside
the system. This provides the realization of the extended Friedrich-Lee model.

##  III CREATION OF A DOUBLE POLE

We next demonstrate how by using external electromagnetic fields we can
convert simple poles of $L$ into poles of higher order. For this purpose, we
consider the application of an electromagnetic field that is tuned close to
the transition frequency $\omega_{12}$ [Fig. 1. $\Lambda_{0}=0,\Lambda\neq
0,G_{l}\neq 0]$. The Hamiltonian describing this system can be written as

$H=\hbar\omega_{13}\ket{1}\bra{1}+\hbar(\omega_{13}-\omega_{12})\ket{2}\bra{2}+H_{ext}+V_{12}+V_{13},$
(11)

where $V_{\alpha\beta}$ describes the decay on the transition
$\ket{\alpha}\rightarrow\ket{\beta}$ and where

$H_{ext}=-\hbar(G_{l}e^{-i\omega_{l}t}\ket{1}\bra{2}+h.c.),$ (12)
$G_{l}=(\overrightarrow{d}_{12}\cdot\overrightarrow{E}_{l}/\hbar).$ (13)

The parameter $2G_{l}$ is the Rabi frequency of the field and is a measure of
the strength of the laser field applied on the transition
$\ket{1}\leftrightarrow\ket{2}$. The Hamiltonian (11) is time-dependent.
However one can make a canonical transformation to reduce it to a time-
independent Hamiltonian. In the special case $V_{12}\rightarrow 0$ the model
(11) is equivalent to the extended Friedrich-Lee model. We have thus produced
a realization of a field-theoretic model in the context of atoms interacting
with laser fields. In our case lasers are used to control the decay process.
Note that we have two control parameters $\omega_{l}$ and $G_{l}$, to
manipulate the nature of the poles of $L$. The situation shown in Fig. 1 is
realizable in many atoms, molecules dopants in solid matrices, etc. For
example, in ${}^{87}Rb$ vapor, the states $\ket{1}$, $\ket{2}$ and $\ket{3}$
could be the states ${}^{5}P_{\frac{3}{2}},{}^{5}S_{\frac{1}{2}},F=2$ and
${}^{5}S_{\frac{1}{2}},F=1$, respectively. We eliminate the optical
frequencies by making canonical transformations
$\rho_{13}\rightarrow\rho_{13}e^{-i\omega_{13}t},\rho_{12}\rightarrow\rho_{12}e^{-i\omega_{l}t}$
etc. After canonical transformations and after eliminating vacuum degrees of
freedom using the master equation techniques the density matrix equations read
[5]

$\displaystyle\dot{\rho}_{11}=-2(\gamma_{1}+\gamma_{2}+\Lambda)\rho_{11}+2\Lambda\rho_{33}+iG_{l}\rho_{21}-iG_{l}^{*}\rho_{12},$
$\displaystyle\dot{\rho}_{22}=2\gamma_{2}\rho_{11}-iG_{l}\rho_{21}+iG_{l}^{*}\rho_{12},$
$\displaystyle\dot{\rho}_{21}=-(\Gamma_{21}-i\Delta_{l})\rho_{21}-iG_{l}^{*}\rho_{22}+iG_{l}^{*}\rho_{11},$
$\displaystyle\dot{\rho}_{31}=-\Gamma_{31}\rho_{31}-iG_{l}^{*}\rho_{32},$
$\displaystyle\dot{\rho}_{32}=-(\Gamma_{32}+i\Delta_{l})\rho_{32}-iG_{l}\rho_{31}.$
(14)

Here we have also included a pumping parameter $\lambda$ to pump the
population from the level $\ket{3}$ to $\ket{1}$. The
$\Gamma_{\alpha\beta}^{{}^{\prime}s}$ give the decay of off-diagonal elements
$\rho_{\alpha\beta}^{{}^{\prime}s}$ of the density matrix and are given by

$\displaystyle\Gamma_{31}=\gamma_{1}+\gamma_{2}+2\Lambda,\Gamma_{32}=\Lambda,$
$\displaystyle\Gamma_{21}=\gamma_{1}+\gamma_{2}+\Lambda,\Delta_{2}=\omega_{12}-\omega_{l}.$
(15)

From (14) and the quantum regression theorem we derive coupled equations for
two time atomic correlation functions

$\left\\{\frac{d}{d\tau}+\left(\begin{array}[]{cc}\Gamma_{31}&iG_{l}^{*}\\\
iG_{l}&\Gamma_{32}+i\Delta_{2}\end{array}\right)\right\\}\left(\begin{array}[]{c}\left\langle
A_{13}(t+\tau)A_{31}(t)\right\rangle\\\ \left\langle
A_{23}(t+\tau)A_{31}(t)\right\rangle\end{array}\right)=0.$ (16)

These are to be solved subject to initial conditions

$\left\langle A_{13}A_{31}\right\rangle=\rho_{11},\left\langle
A_{23}A_{31}\right\rangle=\rho_{12},$ (17)

which in turn are determined from the steady state solution of (14). Clearly
the poles of $L$ that determine the spectral characteristics are given by

$P(z)=(z+\Gamma_{31})(z+\Gamma_{32}+i\Delta_{l})+|G_{l}|^{2}.$ (18)

The zeroes of (18) for $\Delta_{l}=0$ are shown in Fig. 2. The conditions
under which $P(z)$ has double zero are

$\Delta_{l}=0,(\Gamma_{32}-\Gamma_{31})^{2}=4|G_{l}|^{2}.$ (19)

The double zero $z_{0}$ occurs at the bifurcation point in Fig. 2

$z_{0}=-\frac{1}{2}(\Gamma_{31}+\Gamma_{32}).$ (20)

We therefore conclude [6] that a simple pole can be converted into a double
pole in a laboratory experiment by applying an electromagnetic field resonant
with the transition $\ket{1}\leftrightarrow\ket{2}$ and with Rabi frequency
equal to $|\Gamma_{31}-\Gamma_{32}|$.

Figure 2: Motion of the zeroes of (18) for $\Gamma_{31}=1,\Gamma_{32}=0.2$.
Note the presence of the bifurcation point. This is precisely the point where
we create a pole of order two. The solid curve represents $Im(z)+0.6$ whereas
the dashed curve gives $Re(z)$.

##  IV LINE SHAPES AND DOUBLE POLES

The line shape can be calculated from the solution of (16) and (6):

$S(\omega)\equiv\rho_{11}Re[\frac{(\gamma_{2}+\Gamma_{32}-i\delta)}{(\Gamma_{31}-i\delta)(\Gamma_{32}-i\delta)+|G_{l}|^{2}}]$
(21)

which under the double pole condition $2|G_{l}|=|\Gamma_{31}-\Gamma_{32}|$
reduces to

$\displaystyle S(\omega)$
$\displaystyle=\rho_{11}Re[\frac{\gamma_{2}+\Gamma_{32}-i\delta}{(-i\delta+\gamma_{0})^{2}}]$
$\displaystyle=\rho_{11}\frac{\delta^{2}(\gamma_{1}+2\Lambda)+\gamma_{0}^{2}(\gamma_{2}+\Lambda)}{(\delta^{2}+\gamma_{0}^{2})^{2}},$
$\displaystyle\gamma_{0}$
$\displaystyle=\frac{1}{2}(\gamma_{1}+\gamma_{2}+3\Lambda).$ (22)

This is the modification of the line shape formula. Note the double hump
structure of the line shape. Note further the sensitiveness of $S(\omega)$ to
the pumping parameter $\Lambda$. In the limit $\gamma_{2}\rightarrow 0$ and
$\Lambda\ll\gamma_{1}$, (22) reduces to

$S(\omega)\equiv\rho_{11}\frac{\gamma_{1}(\delta^{2}+\frac{\gamma_{1}}{4}\Lambda)}{(\delta^{2}+\frac{\gamma_{1}^{2}}{4})^{2}}$
(23)

It is also interesting to note, that the scale parameter is now
$(\gamma_{1}/2)$ rather than $\gamma_{1}$. Thus the total line shape is a sum
of (a) Square of the Lorentzian (b) derivative of the Lorentzian
$(\zeta/(\zeta+\gamma_{0})^{2}\equiv-\zeta\frac{\partial}{\partial\zeta}(\frac{1}{\zeta+\gamma}))$.

It is possible to consider an alternate model of pumping obtained by setting
$\Lambda=0$ in Fig. 1. Assuming that $\gamma_{2}=0$, one can show that instead
of (23) the spectral line shape is now given by

$S(\omega)\equiv\frac{\gamma_{1}\rho_{11}\delta^{2}}{(\delta^{2}+\frac{\gamma_{1}^{2}}{4})^{2}}=(-\delta\frac{\partial}{\partial\delta})\frac{(\gamma_{1}/2)\rho_{11}}{(\delta^{2}+\gamma_{1}^{2}/4)}$
(24)

which is shown in Fig. 3. The figure also shows for comparison the Breit-
Wigner formula (2) Note the double hump structure of the line shape. The
maxima now occur at $\delta=\pm\gamma_{1}/2$. From Eq. (14) we can also
compute the time dependence of $\rho_{11}(t)$ under the condition of a double
pole. The result is

$\rho_{11}(t)=(1-\frac{\gamma_{1}t}{2})^{2}e^{-\gamma_{1}t}$ (25)

Figure 3: The modified line shape (24) (dashed) as a function of
$\delta/\gamma_{1}$ and its comparison with the Breit-Wigner line shape
(solid).

It is again interesting to note that the time scale is governed by
$\gamma_{1}/2$ rather than $\gamma_{1}$.

The basic idea presented above is easily extended to more complex situations.
For example, two-photon decay in the system as shown in figure 4 which is
easily realizable atoms and molecules. The full Hamiltonian for this system
can be written as

$\displaystyle H$
$\displaystyle=\hbar\omega_{13}\ket{1}\bra{1}+\hbar\omega_{23}\ket{2}\bra{2}+\hbar\omega_{43}\ket{4}\bra{4}$
$\displaystyle-\hbar(G_{l}e^{-i\omega_{l}t}\ket{4}\bra{2}+h.c.)$
$\displaystyle+\sum_{ks}\hbar\omega_{ks}a_{kx}^{\dagger}a_{ks}+V_{12}+V_{23}+V_{42}$
(26)

where the meaning of different terms is obvious. Again a canonical
transformation will change the above $H$ into a time-independent $H$. For
$V_{42}\rightarrow 0$, the above Hamiltonian becomes identical to the one for
the quantum field theoretic extended cascade field model. We thus have a
simple atomic realization of the field-theoretic model. As shown recently [7],
this system exhibits very interesting two photon absorption characteristics.
Clearly, the electromagnetic coupling between the levels $\ket{2}$ and
$\ket{4}$ can produce a double pole in the decay of the system. It is
interesting that a system equivalent to this has been studied by Bhamathi and
Sudarshan [2].

Figure 4: A scheme involving laser coupling the intermediate state $\ket{2}$
which will create pole of order two in the two-photon decay. This provides an
analog of the extended cascade model.

##  V DOUBLE POLES AND INTERFERENCE EFFECTS

The existence of double poles and the possibility of a line shape which is a
derivative of Lorentzian suggest that the quantum interferences must be
crucial. This is indeed the case as can be seen from the following
considerations. The electromagnetic coupling of $\ket{1}$ and $\ket{2}$
produced dressed states
$\ket{\psi_{\pm}}=\frac{1}{\sqrt{2}}(\pm\ket{1}+\ket{2})$ with eigenvalues
$\pm G_{l}$. Since $G_{l}\sim\gamma$, the two states are within the radiative
line width. We pump the population in the state $\ket{1}$ which is equivalent
to pumping in both $\ket{\psi_{\pm}}$ as
$\ket{1}=(\ket{\psi_{+}}+\ket{\psi_{-}})/\sqrt{2}$. Both states
$\ket{\psi_{\pm}}$ can decay to $\ket{3}$ as $\ket{\psi_{\pm}}$ involve
admixtures of $\ket{1}$ and $\ket{2}$. These two decays will not be
independent [8,9] as
$\overrightarrow{d}_{+3}\cdot\overrightarrow{d}_{-3}^{*}\neq 0$ and as
$G_{l}\sim\gamma$.

##  VI EXPONENTIAL DECAY RECOVERED

We also examine the initial conditions for our system which would result in
exponential decay. From Eq. (16) it is seen that

$\displaystyle\frac{d}{d\tau}\left\langle(A_{13}(t+\tau)+iA_{23}(t+\tau))A_{31}(t)\right\rangle$
$\displaystyle+\frac{\gamma_{1}}{2}\left\langle(A_{13}(t+\tau)+iA_{23}(t+\tau))A_{31}(t)\right\rangle=0$
(27)

if $G_{l}=\frac{\gamma_{1}}{2},\gamma_{2}\rightarrow 0$. Thus the correlation
function defined in terms of the vector
$\tilde{\psi}=\frac{1}{\sqrt{2}}(\ket{1}+i\ket{2})$ obeys simple exponential
decay law with a time scale governed by $\gamma_{1}/2$ rather than
$\gamma_{1}$:

$\left\langle
A_{\tilde{\psi}3}(t+\tau)A_{3\tilde{\psi}}(t)\right\rangle=e^{-\gamma_{1}\tau/2}\left\langle
A_{\tilde{\psi}\tilde{\psi}}(t)\right\rangle$ (28)

Thus a pumping of the system to the state $\tilde{\psi}$ rather than $\ket{1}$
will result in exponential decay[10].

Thus, in conclusion, we have shown how higher-order poles in the decay of
states can be produced by using resonant electromagnetic fields. We
demonstrated this by creating a pole of order two. Clearly, the technique is
quite versatile and by using combinations of electromagnetic fields we can
create poles of higher order.

I thank George Sudarshan for discussions on higher order poles of S-Matrix and
R.P. Singh for help in preparation of this paper.

## References

  * (1) M.L. Goldberger and K.M. Watson, Phys. Rev. 136, B 1472 (1964); J, S. Bell and C.J. Goebel, Phys, Rev. 138, B 1198 (1965)
  * (2) G. Bhamathi and E.C.G. Sudarshan, Int. J. Mod Phys. B 10, 1531 (1996); see also E.C.G Sudarshan, Phys. Rev. A 50, 2006 (1994); E.C.G. Sudarshan, C.B. Chiu and G. Bhamathi, Phys. Rev. D 46, 3508 (1992).
  * (3) A. Bohm, S. Maxson and M. Loewe, Physica A, in press; A. Mondragon and E. Her- nandez, J. Phys. A 26, 5595 (1993); C. Puntmann, paper presented at the International Colloquium on Group Theory, Goslar, Germany 1996.
  * (4) G.S. Agarwal, Quantum Optics (Springer-Verlag, Berlin, 1974).
  * (5) G.S. Agarwal, Phys. Rev. A 54, Rapid Commun. 3734 (1996).
  * (6) An almost trivial case occurs when $\Gamma_{23}=0,\gamma_{2}=0$ and no pumping $(\Lambda=0)$. The atom can start in state $\ket{1}$. Then one can work with a nonhermitian Hamiltonian $\left(\begin{array}[]{cc}-i\gamma_{1}&G\\\ G&0\end{array}\right)$ which has identical real eigenvalues if $2G=\gamma_{1}$. This case has been previously considered in literature (H. Steudel, Ann. Physik 22, 113 (1969).
  * (7) G.S. Agarwal and W. Harshawardhan, Phys. Rev, Lett. 77, 1039 (1996).
  * (8) G.S. Agarwal, Quantum Optics (Springer-Verlag, Berlin, 1974) pp. 94-96.
  * (9) A. Imamoglu, Phys. Rev. A 40, 2835 (1989); S.Y. Zhu and M.O. Scully, Phys. Rev. Lett. 76, 388 (1996); D.A. Cardimona, M.G. Raymer and C.R. Stroud Jr., J. Phys. B 15, 55 (1982).
  * (10) Pumping the system in the state » is possible using an excitation pulse with phase switching at appropriate instant (cf. Y.S. Bai, A.G. Yodh and T.W. Mossberg, Phys. Rev. Lett. 55, 1277 (1984)).