# Waste detection in Pomerania: non-profit project for detecting waste in
environment

Sylwia Majchrowska
Wroclaw University of Science and Technology
Wrocław, Poland
<EMAIL_ADDRESS>
Agnieszka Mikołajczyk
Gdańsk University of Technology
Gdańsk, Poland
<EMAIL_ADDRESS>
Maria Ferlin
Gdańsk University of Technology
Gdańsk, Poland
<EMAIL_ADDRESS>
Zuzanna Klawikowska
Gdańsk University of Technology
Gdańsk, Poland
<EMAIL_ADDRESS>
Marta A. Plantykow
Intel Corporation
Gdańsk, Poland
<EMAIL_ADDRESS>
Arkadiusz Kwasigroch
Gdańsk University of Technology
Gdańsk, Poland
<EMAIL_ADDRESS>
Karol Majek
Cufix
Warsaw, Poland
<EMAIL_ADDRESS>
###### Abstract

Waste pollution is one of the most significant environmental issues in the
modern world. The importance of recycling is well known, either for economic
or ecological reasons, and the industry demands high efficiency. Our team
conducted comprehensive research on Artificial Intelligence usage in waste
detection and classification to fight the world’s waste pollution problem. As
a result an open-source framework that enables the detection and
classification of litter was developed. The final pipeline consists of two
neural networks: one that detects litter and a second responsible for litter
classification. Waste is classified into seven categories: bio, glass, metal
and plastic, non-recyclable, other, paper and unknown. Our approach achieves
up to 70% of average precision in waste detection and around 75% of
classification accuracy on the test dataset. The code used in the studies is
publicly available online111https://github.com/wimlds-trojmiasto/detect-waste.

_K_ eywords Litter detection $\cdot$ Waste detection $\cdot$ object detection

##  1 Introduction

Litter is scattered in a wide variety of environments. The massive production
of disposable goods in the last years resulted in an exponential increase in
produced garbage, reaching about 2 billion tons per year [1]. It was estimated
that humans eat up to 250 grams of micro plastics each year [2]. Almost 90
percent of this plastic comes from bottled and tap water. Today, more than 300
million tons of plastic is produced annually and only 30 years are left for
the amount of garbage in the ocean to exceed the number of sea creatures [3].

The most difficult challenge in the waste management is complex, and not
unified guidelines regarding segregation rules. Due to the large energy
requirements and related costs plastic waste separation on many litter sorting
lines is done manually. However, new possibilities for the automatic selection
of these materials and their reuse are constantly being tested [4].

In recent years machine learning (ML) based systems that can support or fully
cover sorting processes were implemented, accelerating this procedure as a
result. The most commonly used solutions include self-sorting smart bins,
which are capable of classifying one object located on a clear background at a
time [5, 6]. When a single compartment is used, the camera is usually located
at the top of the upper container.The deep learning (DL) model assigns proper
class based on the photo and the garbage is moved to the appropriate bottom
container [5]. Another way is to a mount camera or a sensor above few separate
bins, and direct the consumer handling waste to the correct one [7]. In this
approach, the rubbish must be well exposed to the imaging device. Here it
comes to the fact that the problem of identifying garbage in the wild
environment is ambiguous. An image could present different types of trash,
which additionally can be deformed, and/or recorded in a uncontrolled various
natural scenarios. On the other hand, as the same object may be a junk or not
depending on the context, it must be defined at what point localized object
should be treated as a litter.

This paper addresses some of the identified limitations. At first, to provide
more efficient recycling, we mixed publicity available datasets of waste
observed in different environments, and proposed seven well-defined categories
for sorting litter: bio, glass, metal and plastic, non-recyclable, paper,
other and unknown. Their selection was inspired by rules in Gdansk (Poland).
Secondly, we implemented a two-stage DL-based framework for waste detection
that consisted of two separate neural networks: detector and classifier. The
proposed framework is freely available and can be used for different purposes,
such as studying the common types of waste observed in nature. To our
knowledge, we presented the first experiments that allow for such universal
litter detection and classification. Additionally we provided the first
comprehensive review of existing waste datasets.

An overview of the actual work for deep learning classification and detection,
as well as existing public datasets of waste are described in Section 2.
Section 3 illustrates our framework to detect and classify waste from images,
and statistics of examined data. More specifically, we described used data in
details, provide the training details of chosen neural networks, and reported
the obtained results. Finally, conclusions are drawn and future work is
outlined in Section 4.

##  2 Related works

ML and DL techniques empower many aspects of modern society, like
recommendation systems, text-to-speech devices, or even objects identification
in images [8]. Also the waste management problem has attracted a lot of
interest [5, 6, 9], where the main goal is to create an ML-based image
recognition system to sort litter. The majority of the proposed approaches are
based on the deep learning algorithms utilized in the computer vision field.
This section describes chosen techniques used in classification and object
detection challenges and delivered a comprehensive review of existing public
waste datasets.

###  2.1 Classification

Convolutional neural networks (CNNs) have had a massive impact on large-scale
image classification tasks and made it possible to achieve significantly
higher accuracy than solutions based on classical image processing. Multiple
convolutional layers stacked together can automatically learn representations
from the data reducing the need for manual feature extraction engineering [8].
The design of neural network structure impacts the performance, latency, and
computational requirements. Thus efforts have shifted from feature extraction
design to architecture design. Architectural enhancements of deep neural
networks improve the performance of classification, segmentation and detection
tasks.

A breakthrough in CNN performance came with AlexNet [10], a neural network
that won the ImageNet Large Scale Visual Recognition 2012 challenge. Since
then, the quality of image recognition structures has advanced rapidly.
AlexNet consists of a stack of five convolutional layers of different kernel
sizes. The structure incorporates non-saturating ReLU activation functions,
overlapping pooling, and normalization layers. Dropout layers were used to
reduce overfitting.

The authors of [11] proposed a family of architectures called VGG - very deep
networks consisting of 16 to 19 layers. The authors observed that stacking of
several 3x3 layers could imitate the kernels of bigger sizes achieving greater
effective receptive fields while reducing the number of parameters. This
strategy provides an additional benefit of a more discriminative decision
function caused by more nonlinear layers in the structure.

Further increase in the number of layers leads to deterioration of training
and decrease in neural network performance. To alleviate the problem ResNet
family [12] was proposed. The architecture is based on residual connections,
that introduce no extra parameters or computational effort. Instead, this
connectivity pattern facilitates gradient flow, enabling effective training of
networks consisting of as many as 200 layers.

Inception [13] network stacks modules called Inception instead of single
layers. Inception blocks incorporate multi-scale convolutional transformations
utilizing a split-transform-merge strategy. Moreover, it allows for increasing
the number of units at each stage without an uncontrolled blow-up in
computational complexity. The Inception models have evolved by utilizing batch
normalization and factorization of convolutional layers [14], simplification,
and the use of asymmetric filters [15]. The proposed Inception-Resnet
structure [15] combines both the power of inception modules and residual
connections.

The improvement of connectivity pattern in the DenseNet structure [16]
facilitated the training and accuracy. For each layer, the feature maps of all
preceding layers are used as inputs, and its feature maps are used as inputs
into all subsequent layers. DenseNets have several significant advantages:
they minimize the vanishing gradient problem, improve feature propagation,
encourage feature reuse, and reduce the number of parameters.

ResNeXt [17] structure similarly to Inception utilizes a split-transform-merge
strategy. However, unlike all Inception or Inception-ResNet modules, the same
topology among the multiple paths is shared and paths are aggregated by
summation. The authors introduced a conception of cardinality - the number of
paths in one module. It was observed that an increase in cardinality is more
effective than going deeper or wider when increasing the capacity.

EfficientNet [18] architecture consists of modules constructed by the neural
architecture search process that optimizes both for accuracy and FLOPS. The
mobile-size baseline model called EfficientNet-B0 was built stacking those
modules. Scaling strategies were used to produce more complex and accurate
models EfficientNet-B1-7. Scaling of a convolutional neural network most often
is performed in one of the following dimensions width, depth, or resolution.
Whereas, EfficientNets scale three dimensions jointly, significantly improving
efficiency and accuracy

EfficientNetv2 [19] improves EfficientNet by providing faster training and
better parameter efficiency by eliminating EfficientNet bottlenecks. Neural
architecture search was used to optimize accuracy, training speed, and
parameter size. Unlike, standard EfficientNet, EfficientNetv2 uses non-uniform
scaling of depth, resolution, and width. Moreover, to limit computational cost
an increase in resolution was limited.

Inspired by the successes in Natural Language Processing, recent computer
vision architecture advances rely on the Transformer module. The authors of
[20] trained Visual Transformer (Vit) – a pure transformer architecture
applied directly to sequences of image patches that perform very well on image
classification tasks. The authors stated that training transformers for vision
tasks requires large amounts of training samples and extensive computing
resources. This particular trait was addressed in Data-efficient image
Transformers (DeiT) [21] in which a new model distillation procedure is shown.
In [22] authors present Pyramid Vision Transformer (PVT) which is designed for
dense prediction tasks such as object detection. PVT introduces to
transformers the pyramid structure known from CNNs. Therefore it is more
suitable for object detection tasks than Vit, which was designed for image
classification. The application of transformers in vision is further
investigated in [23]. Authors introduce convolutions to ViT architecture to
introduce shift, scale, and distortion invariance. Proposed Convolutional
vision Transformer (CvT) architecture accomplishes this goal by two main
modifications: introduction of a hierarchy of Transformers containing a new
convolutional token embedding, and a convolutional Transformer block
leveraging a convolutional projection. The application of transformers in
vision tasks is not yet a well-investigated topic, and we observe many teams
that are currently working on improving it.

###  2.2 Object Detection

Object detection is a well-studied task in computer vision [24]. It is defined
as a localization of Axis-Aligned Bounding Box (AABB) and classification -
assignment of a single or multi-label. In many previous works object detection
was approached using two types of techniques namely one-stage and two-stage
detection. One-stage architectures provide both locations and classes for each
object in a single step, while two-stage detectors find class-agnostic object
proposals first, classifies them into the class-specific detections in the
second stage.

Two-stage detectors were the first object detection methods. They used the
sliding window approach in the image pyramid to generate object proposals in
multiple scales. Then in a second stage a classifier such as a cascade
classifier [25] was used. Such a system achieved 15 Hz for the face detection
system which is a single class object detection problem. A significant
improvement was presented in [26] where authors introduced a novel descriptor
namely Histograms of Oriented Gradient (HOG) and showed a method of human
detection using Support Vector Machine (SVM) to classify objects. The authors
used a detection window to scan across the image at all positions and scales.
Classification is invoked multiple times in similar regions and using multiple
image pyramid levels.

Using convolutional neural networks for object detection was a challenging
task since CNNs’ inability to localize features. In most approaches, this
problem was solved by using the recognition using regions paradigm [27].
Identified regions can yield richer information than single pixels or pixels
with a fixed local neighbourhood. Regions are used for object detection
successfully in the Selective Search algorithm [28]. Authors generate initial
regions which are grouped by similarity and merged in the iterative process.
Object hypotheses from regions were used to extract features using key point
feature descriptors and classify those proposals using SVM. Selective Search
was also used in R-CNN [29] to extract region proposals, on which CNN features
where computed to classify regions using per class SVMs finally. Fast R-CNN
[30] solved R-CNN’s main problems: multiple training sessions required since
SVMs are involved, inefficient processing of regions through the CNN. Fast
R-CNN computes convolutional features for the whole image in the first step to
reduce computations for overlapping regions. The other improvement is
integrating classification into the network architecture, which allows allows
training of the entire network in a single multi-task training session. When
Faster R-CNN [31] was introduced State-of-the-art object detection networks
used region proposal algorithms to generate object location hypotheses. In
this architecture a Region Proposal Network (RPN) concept was used instead of
the Selective Search algorithm. RPN is integrated into Fast R-CNN architecture
and uses shared weights. Many researchers used Faster R-CNN while replacing
its backbone (feature extraction CNN) with newer architectures in the
following years. Faster R-CNN is like its’ predecessors, a two-stage object
detection method.

Single-stage detection was popularized in Deep Learning mainly by two detector
architectures: Single Shot MultiBox Detector (SSD) [32] and You Only Look Once
(YOLO v1, 9000, v3, v4, scaled V4: [33, 34, 35, 36, 37]). Those networks
perform both, object detection and object classification in a single step.
Therefore the detection time can be reduced. Such a result is obtained by
generating a constant number of predictions per class per image. For SSD it
was 8732 detections for each class for a 300x300px image. Each detection is
characterized by a category score for each category and 4 offsets for a fixed
set of default bounding boxes. Default boxes have different aspect ratios at
each location in several feature maps with different scales. In this type of
methods, many false detections need to be removed considering objectness or
classification score and overlaps between detections. YOLOv1 used a single
scale image for prediction with 98 detections per class and did not use
default boxes YOLOv2 (YOLO9000) predicts nine bounding boxes at each cell from
a 13x13 grid, resulting in in 1521 boxes. YOLOv3 introduces a multi-label
approach to classification since, in datasets such as Open Images [38], class
labels do not have a guaranteed hierarchy level. It uses image 3 image pyramid
levels for detection. YOLOv4 introduces a new backbone network while using the
same detector as YOLOv3. This paper studies data augmentation, post-processing
methods. A month after YOLOv4 [36] was proposed, a new approach to object
detection tasks was introduced in [39]. Detection Transformer (DETR)
introduced Transformers known from Natural Language Processing tasks [40] into
object detection while preserving image processing with CNNs. DETR predicts a
sparse, fixed number of objects which are in training matched with ground
truth labels using bipartite matching. DETR’s main limitations were slow
convergence and limited feature spatial resolution. It required 10 times more
epochs than other approaches to converge to a similar error level. Limited
spatial resolution lowered the Average Precision metric for small objects’
sizes. The following year, the main issues were mitigated in Deformable DETR
[41] thanks to the introduced deformable multi-head attention module.

EfficientDet, a single-stage object detector, was introduced in [42] a month
after [39]. The main goal of the authors was to improve the efficiency of
object detection models. The goal was separated into two challenges, namely
efficient multi-scale feature fusion and model scaling. They introduced a
weighted bi-directional feature pyramid network (BiFPN) to address the first
challenge. BiFPN adds a bottom-up pathway to fuse multi-scale features
similarly to Neural Architecture Search Feature Pyramid Network (NAS-FPN)
[43], which used NAS to find the best top-down, bottom-up pathways for feature
fusion. The second challenge was addressed using a compound scaling method for
object detectors inspired by [18]. It jointly scales up the
resolution/depth/width for all backbone, feature network, and box/class
prediction network. EfficientDet is proposed in 8 variants D0-D7 with backbone
networks EfficientNet B0-B7. The approach shown in [42] reduced the number of
parameters and latency in the object detection network and simultaneously
increased the Average Precision metrics.

Instance Segmentation is another task for automated image processing. The goal
of this task is to provide a segmentation mask for each object instance.
Approaches introduced in the pre-Deep Learning Era relied on methods such as
edges or superpixels. This changed in Deep Mask [44] which was one of the
first deep neural networks used for the instance segmentation task. The
proposed network predicts object proposals for the whole image. Those
proposals are class-agnostic segmentation masks, and for each of them, an
object likelihood score is computed. Mask RCNN [45] is an extension of Faster
R-CNN, and it ads the instance mask branch in parallel to the classification
and bounding box regression branch. In Mask RCNN recognition precedes
segmentation, which is faster and more accurate according to authors.

###  2.3 Data collections

In recent years multiple attempts to detect, classify and segment waste using
deep learning have been made. Litter classification of common waste categories
based on images has been attempted using few pre-trained convolutional neural
networks – AlexNet, MobileNet, InceptionResNetV2, DenseNet, Xception [46, 47,
48] – achieving average accuracy in ranges 22% [46] and 98.2% [48] for
pictures of waste on a plain background. Experiments have also been conducted
on the detection of litter on the streets and homes, as well as the
segmentation of different types of waste using: Faster R-CNN, Mask R-CNN, SSD,
and different types of YOLO (even Tiny-YOLO) [49, 50, 51, 52, 53, 54, 55, 56].
Calculated mAP value for detection task varies between different datasets and
architectures from 15.9% for TACO [53] with Mask-RCNN, up to 81% for Trash-
ICRA19 [51] with Faster R-CNN.

Nonetheless, very few attempts at the detection and classification into well-
recognized recyclable classes have been made. As deep learning requires
diverse data to fit the model’s parameters optimally, a comprehensive review
of existing litter datasets in terms of litter detection is provided. Over ten
different data collections and their crucial statistics (e.g. number of
images, annotation type) are presented in Table 1 (see Fig 1 to visualize
their representatives).

TrashNet. The TrashNet dataset [46] contains over 2100 labeled images. Each of
them belongs to one of the six classes: glass, paper, cardboard, plastic,
metal, and trash. The pictures were taken by mobile phone camera using
sunlight and/or room lighting. Photographed objects were placed on a white
background or fulfill the whole view (cardboard). All images have size a of
512x384px.

Open Litter Map. Open Litter Map [57] is a free, open, and crowd-sourced
dataset with over 100k images taken by phone cameras. All images are provided
with information such as type of presented litter, coordinates, timestamp or
phone model. Images come from all over the world, taken by different people.
Therefore, they differ significantly from one another.

Waste Pictures. Waste Pictures [58] contains almost 24000 waste images scraped
from Google search, divided into 34 classes. The type of images is very
diverse, including even x-rays and drawings of garbage. Sizes also differ
significantly. However, most of the photos are below the size of 2000x2000px.
Due to the origin of images, they should be manually reviewed for use in a
classification task.

Extended TACO. Trash Annotations in Context (TACO) [53] is a crowd-sourced
dataset of waste in the wild with high-resolution mobile phone images. The
TACO dataset contains 1500 annotated images with almost 5000 objects. All
trash have been assigned to one of 60 classes that belong to 28 super (top)
categories, including the category Unlabeled litter for hard to recognize or
heavily obscured objects. The annotations are provided in the well-known COCO
format [59] on the instance segmentation level with an extra background
description - Trash, Vegetation, Sand, Water, Indoor, Pavement. Additionally,
TACO offers around three thousand of unannotated images, which we have taken
advantage of: we provided annotations on the detection level222Only the
primary part (1.5k images) of the dataset is annotated with instance masks, we
provided bounding box annotation for the rest. for over 3000 images achieving
over 14 000 instances in total. A great advantage is that TACO is
characterized by various litter types and high diversity of the backgrounds,
from tropical beaches to London streets. However, due to the crowd-sourcing
nature of the dataset, labels may contain some user-induced errors and bias,
i.e., not all objects in TACO can be categorized strictly as litter as their
category is often based on context.

Wade-AI. The Wade-AI dataset [60] contains images of waste in the wild
environment, provided by Google Street View. It consists of nearly 1400 images
with 2200 manually labeled instance masks annotations in COCO format with only
one class, called rubbish. The environment and size of the images vary due to
the source of the images. Most images are less than 1000x1000.

UAVVaste. Another publicly available dataset, which also provides instance
segmentation masks in the COCO format, is UAVVaste [56] dataset. It contains
772 hand-labeled aerial images of waste with over 3700 objects of one class -
"rubbish". Data was collected in the urban and natural environments e.g.
streets, parks and lawns using Unmanned Aerial Vehicles (UAV). The annotated
litter is usually relatively small (median of object shape is 76x68px, while
median of image shape is 3840x2160px).

TrashCan and Trash-ICRA. TrashCan [52] and Trash-ICRA [51] are datasets both
containing underwater images. They are comprised of frames of video showing
trash, remotely operated underwater vehicles (ROVs), and undersea flora and
fauna. Both datasets are sourced from the JAMSTEC E-Library of Deep-sea Images
(J-EDI) dataset [citation], curated by the Japan Agency of Marine-Earth
Science and Technology (JAMSTEC) captured from real-world environments,
providing a variety of objects. The clarity of the water and quality of the
light vary significantly between images creating a diverse dataset. The image
sizes in these datasets are 480x270px and 480x360px. Provided annotations are
in COCO format. The TrashCan dataset is annotated on the instance segmentation
level (7212 images and 6214 annotations) with 16 classes for Material Version
(8 classes are trash related and followed trash_ name pattern) or 22 for
Instance Version. On the other hand, the Trash-ICRA19 dataset is annotated on
the detection level (7668 images and 6706 annotations). It contains seven
categories based on the material of the objects.

Drinking Waste. Drinking Waste [61] contains over 4800 images of drinking
waste belonging to 4 classes: Aluminium Cans, Glass bottles, PET bottles, and
HDPE. Provided bounding-box annotations are in YOLO format. The dataset was
created with a 12 MP phone camera. Images look similar – there is usually one
object in the center on the indoor, plain background. Most of the images have
the size of 512x683px.

MJU-Waste. MJU-Waste dataset [55] is comprised of 2475 indoor trash images
manually annotated in the form of an instance mask in COCO format. It allows
two-class semantic segmentation (waste and background). For each color image,
the co-registered depth image captured using an RGBD camera is provided.
Objects are hand-held and mostly in the center of the image. In most cases
there is only one object per image. The only image size is 640x480px.

Cigarette butt. The cigarette butt dataset [62] consists of above 2k images,
which were synthetically composed photos of small cigarettes lying on the
ground. It is an artificial dataset created by applying random scale, rotation
or brightness to the foreground cutouts of 25 different cigarette butts,
placed in around 300 ground photos. Instance mask annotations are provided for
each cigarette object with a median size of 66x65px. Original resolution of
images is equal 3024x4032px.

Places. Places [63] is a repository of 10 million scene photographs, labeled
with 434 scene semantic categories, comprising a large and diverse list of the
types of environments encountered in the world. Images were downloaded by
online image search engines (Google Images, Bing Images, and Flickr). Minimal
size of images is 200x200px. Despite the fact that this is not a trash
dataset, it can be used to identify natural and urbanized places without
trash.

Table 1: Statistics for selected public waste datasets for classification and object detection purposes. Dataset | # classes | # images | # instances | annotation type
---|---|---|---|---
TrashNet | 5 | 2194 | 2194 | labels
Waste Pictures | 34 | 23633 | 23633 | labels
Open Litter Map | >100 | >100k | >100k | multilabels
Extended TACO | 7 | 4562 | 14286 | bounding box
Wade-AI | 1 | 1396 | 2247 | instance masks
UAVVaste | 1 | 772 | 3718 | instance masks
TrashCan 1.0 | 8 | 7212 | 6214 | instance masks
Trash-ICRA19 | 7 | 7668 | 6706 | bounding box
Drinking waste | 4 | 4810 | 5058 | bounding box
MJU-Waste | 1 | 2475 | 2532 | instance masks
Cigarette butt | 1 | 2200 | 2200 | instance masks
Places | 205 | 2.5M | 2.5M | labels
Figure 1: Representative pictures from each dataset.

##  3 Two-stage framework to detect and sort litter

Litter sorting methods around the world differ significantly. On the one hand,
this provides a large variety of objects (litter), as well as diverse
backgrounds - waste can be commonly found indoor and outdoor, and in
environments such as households, offices, roads and pavement scenes, and
sometimes even under water. On the other hand, this affects available
datasets, which do not provide a large enough number of annotated images.
Moreover, a huge variety of categories and annotation levels (from
classification to instance segmentation) resulted in the need for well-defined
waste categories that could be applied in the industry. Hence, seven litter
categories are provided based on a real-world waste segregation system in
Gdańsk (Poland): bio, glass, metal and plastic, non-recyclable, other (e.g.
batteries, large household appliances, tires), paper, and unknown (old,
degraded litter).

To utilize all of the available data we tackle the problem by dividing the
detection into two separate steps: litter localization and litter
classification. The localization model is used to find regions in the image
containing garbage. Then, each region is extracted and passed to the
classification model, that assigns its category. The proposed pipeline is
presented in the Fig. 2.

This section outlines the details behind the training procedure, the
methodology, and our experiments. We compare various architectures, as well as
numerous hyperparameters, to ensure the efficiency of the solution.
Additionally, we present how each dataset can be used in the training process
- separately for waste localization, classification or both, depending on the
annotation type.

Figure 2: A pipline of a two-stage framework to detect and sort litter

###  3.1 Litter in images

During Detect Waste in Pomerania project, twelve publicly available datasets
and an additional data collected using Google Images Download [64] were used.
Segmentation and detection datasets combined resulted in a new detect-waste
dataset [53, 56, 52, 51, 55, 61, 60] (with the addition of cigarette butt
dataset [62] – detect-waste+ dataset) with a single class – Litter – used in
the first stage of our framework: Localization (see Fig. 2 top panel). For the
purpose of the second stage – Classification – additional images of bio,
glass, other, and paper [46, 58, 64] waste were added to the set of clipping
images created by cutting out litter instances from some photos used in the
detection stage [53, 61, 57]. These images form a final classification dataset
named classify-waste. Furthermore, the dataset was supplemented with more than
a thousand images containing garbage-free backgrounds [63] and around 55k
pseudo-labeled images [57], creating the classify-waste+ dataset.

The localization task was mainly performed on a detect-waste dataset with a
single class called litter. In the last experiments, the dataset was extended
to the detect-waste+ dataset, which is previous detect-waste dataset with
additional pictures presenting cigarette butts [62] to boost the detection of
small objects. Fig. 3 shows the datasets included in the detect-waste+ dataset
along with the number of used images, divided into groups depending on the
trash environment (in the wild, underwater, inside homes and artificial).
Additional experiments were performed for multi-class detection on Extended
TACO. The division of this dataset into Gdańsk segregation categories is
presented in the Fig. 4.

Figure 3: Datasets included in detect-waste+ dataset used for detection task
provided with number of images. Figure 4: Annotations per category for
Extended TACO dataset.

In the classification task, in addition to instances cut-out from Extended
TACO and Drinking waste, some classes from TrashNet (paper and glass) and
Waste Pictures (bio and other) datasets have been added. Additionally, we
scrapped remaining data from the web. The Google Images Download [64] software
was used to search and collect more images with bio and other waste like
batteries, medicines or rubble, and also images that depicted scenes without
the presence of garbage (background) from Places dataset [63]. Semi-labeled
(with predicted localization) images from Open Litter Map [57] were also
utilized. Fig. 5 shows number of used images per category (according to the
obligatory segregation rules of the city of Gdańsk) without images from Open
Litter Map, while Table 2 provides information from which dataset the images
come from, for which category. Table 3 presents numbers of images from semi-
labeled Open Litter Map with some established seven litter class pre-
assignment.

Table 2: Number of images and their origin for a given category for the classification task. Dataset | background | bio | glass | metals & plastic | paper | non-recyclable | other | unknown
---|---|---|---|---|---|---|---|---
Extended TACO | - | 69 | 592 | 6057 | 601 | 2802 | 154 | 3258
Drinking waste | - | - | 1162 | 3604 | - | - | - | -
waste-pictures & |  |  |  |  |  |  |  |
Google search | - | 92 | 49 | - | 203 | - | 366 | -
TrashNet | - | - | 501 | - | 801 | - | - | -
Places | 1017 | - | - | - | - | - | - | -
Figure 5: Images per category for classify-waste + Places. Table 3: Number of images from Open Litter Map with pre-assignment of litter class. Category | # pseudo-labels
---|---
bio | -
glass | 3136
metals and plastic | 29219
non-recyclable | 1971
other | 900
paper | 819
unknown | 19701

Most of the trash found in the classification dataset is metal and plastic.
Unfortunately, the second numerously represented category is unknown – the
litter that has probably decomposed so much that it is hard to classify it.
This makes our dataset highly imbalanced and requires special attention in the
following steps. Another problem is related to annotations errors coming from
datasets without strict annotation rules (especially when it comes to the
trash label) – unfortunately some annotations are of poor quality. Moreover,
we manually rejected some of the images that were malformed or mislabeled.

To ensure that the training and test data distributions approximately match,
we randomly selected 80% of the original images as the training set. We kept
the rest as validation/testing set for each part of the used dataset
separately. The split was done by preserving the percentage of samples for
each class.

###  3.2 Litter Detection

In the first step, the proposed framework localizes litter in the image
without recognizing its class. Three different neural networks popular in
common object detection tasks were analyzed, namely EfficientDet [65], DETR
[66], and Mask R-CNN [31]. The first two architectures allow for object
detection, whereas the third one also implements segmentation. All models were
trained to predict trash at each location and scale (the solution is not
targeted at a specific object size). To properly evaluate each proposed
architecture, a set of tests was conducted using a wide range of datasets
altogether and individually. Defined classes varied depending on the used
dataset, as described in Section 3.1.

####  3.2.1 Training details

To ensure one-stage detector efficient and fast performance, as EfficientDet
[42], was utilized to localize litter in images. The used model was
initialized with pre-trained weights. In the final experiments decay rate was
set to 0.95, the learning rate to 1e-3, and the number of epochs equaled 20.
Also, several approaches to data augmentation [67] were used. During training,
Gaussian blur, random brightness, rain, fog, and snow were added.
Additionally, images were rotated and cropped around the annotated bounding
boxes with padding if needed. However, the best results were obtained for
simple resize (to match default EfficientDet’s input size), and normalization
using means and standard deviation values per channel as for the COCO dataset
[59]. We hypothesize that augmentation did not significantly improve the
results because used data was naturally diversified by mixing a wide range of
datasets. In conducted studies, the Pytorch implementation of EfficientDet
[65].

Also, Transformers approach was applied to the waste detection task. DETR
(which stands for DEtection TRansformer) with ResNet-50 and ResNet-101
backbone was initialized with pre-trained weights. In all experiments AdamW
[68] optimizer was used, with a learning rate set to 1e-4 in the transformer
and 1e-5 in the backbone. Selected number of queries, that represents the
maximum number of instances that could be found in image, equaled 100, as by
default. In final experiments, after about 63 epochs loss values saturate, and
longer training reduce neither the training nor the validation error. The DETR
litter detection model is based on Facebook team implementation [66].

In the case of instance segmentation, Mask R-CNN [45] with Resnet-50 backbone
was used. In final experiments, the model was trained for 26 epochs with
learning rate set to 1e-3 and decay set to 1e-4. The transfer learning
technique was also used in this study, as each of the models was initially
pre-trained on the COCO 2017 [69] subset. Implementation of the model is based
on standard Pytorch library.

####  3.2.2 Results

Conducted experiments were divided into two phases. In the first phase
selected architectures were tested on the detect-waste dataset. The model,
which exhibit the best performance, was selected for further training. The
second phase showed how the results are distributed on the subpart datasets.
This allowed for a deeper analysis of the effectiveness of the selected
architecture and better preparation to the final training. As a basic
evaluation metric, Average Precision (AP) for intersection over union (IoU)
equaled 0.50 was used (AP@05 [70]). In the case of multi-label detection, this
metric is averaged over all categories, giving<EMAIL_ADDRESS>
Table 4: Comparison of results of selected architectures on detect-waste dataset with one class - litter. Model | Backbone |<EMAIL_ADDRESS>
---|---|---
DETR | ResNet-50 | 50.7
DETR | ResNet-101 | 51.6
Mask R-CNN | ResNet-50 | 28.0
EfficientDet-D2 | EfficientNet-B2 | 65.5

As presented in Table 4, average precision of litter detection, using detect-
waste dataset with one class, varied in the range of 28.0% for Mask R-CNN with
ResNet-50 backbone to 65.5% for EfficientDet-D2. In general, due to a
significant gap between EfficientDets and other tested architectures the rest
of experiments were limited to this network family. Additionally, more complex
networks from this family (i.e. EfficientDet-D3) were also tested, and
exhibited a similar performance. As EfficientDet-D2 network reached the best
evaluation results, it is smaller (considering the number of parameters) and
requires less computing power, we decided to proceed with it exclusively.

Results of a comprehensive study using EfficientDet-D2 and selected datasets
separately are presented in Table 5. The<EMAIL_ADDRESS>calculated per dataset is
the highest for images presenting indoor scenario, namely Drink-waste (cans,
plastic and glass bottles) and MJU-Waste (one hand-held waste object per
image) datasets. However, EfficientDet-D2 also reached very high score,
<EMAIL_ADDRESS>above 90%, for TrashCan 1.0 with selected 8 underwater waste
categories. The worst result, reaching<EMAIL_ADDRESS>below 10%, was achieved for
the underwater images from Trash-ICRA19 dataset, which could be related with
poor quality of the photos (blurred movie frames). On the other hand,
detection performance for images taken in natural or urban background was in
range of 56.8% for Extended TACO (trash in various environments) to 74.1 for
UAVVaste (small objects constituting over 80% of the dataset shown from a
bird’s eye view), which proved that precise detection of garbage in the
different environment is possible. Corresponding sample predictions are shown
in the Fig. 6.

Achieved results proved importance of data quality in the learning process of
DL-based system, but apart from different quality of photos and environments
of waste occurrence in which they were taken, also the number and kind of
waste classes varied depending on the analyzed dataset. Moreover, annotated
trash objects differ in shape and size. This suggests that one-class detection
(with one litter category) using EfficientDet-B2 network might lead to much
better performance. To provide more details, in final experiments with
EfficientDet-D2, AP score at different IoU thresholds levels, along with
AP@[0.50:0.95] [59] (AP integrated over IoUs in range from 0.5 to 0.95, and
step 0.05) were calculated. This results are presented in Table 6. Selected
neural network was tested in four different scenarios. In two of them the
Extended TACO dataset was used, while the third and fourth experiments were
conducted using different type of detect-waste dataset instead.

Table 5: Results on different datasets achieved using EfficientDet-D2. Dataset | Classes |<EMAIL_ADDRESS>
---|---|---
Extended TACO | 1 | 56.8
Wade-AI | 1 | 71.5
UAVVaste | 1 | 74.1
TrashCan 1.0 | 8 | 91.3
Trash-ICRA19 | 7 | 7.3
Drink-waste | 4 | 99.4
MJU-Waste | 1 | 97.9
Figure 6: Example EfficientDet-D2 predictions for the diverse waste datasets. Selected images were taken in different locations such as a beach, pavement, indoor, and underwater. Detected objects vary in size and number – images show from one to five small, medium or large objects. Table 6: EfficientDet-D2 final evaluation results. Dataset | Classes | mAP@0.50:0.95 |<EMAIL_ADDRESS>|<EMAIL_ADDRESS>| $\mathbf{AP_{S}}$ | $\mathbf{AP_{M}}$ | $\mathbf{AP_{L}}$
---|---|---|---|---|---|---|---
Extended TACO | 7 | 11.9 | 16.2 | 13.0 | 6.4 | 9.4 | 15.0
Extended TACO | 1 | 40.4 | 56.8 | 43.5 | 19.8 | 37.2 | 51.6
detect-waste | 1 | 45.8 | 65.5 | 50.2 | 5.9 | 49.1 | 59.7
detect-waste+ | 1 | 46.9 | 66.4 | 51.3 | 9.3 | 51.3 | 59.8

EfficientDet-D2 trained on the detect-waste+ dataset reached the biggest mAP
for IoU equal both 0.50 (66.4%) and 0.75 (51.3%). Solution effectiveness in
respect of detected object size demonstrated better precision for large
(59.8%) and medium (51.3%) objects, than for small (9.3%) instances. Moreover,
as expected, the test conducted using the detect-waste+ dataset, which boosted
the evaluation result for small objects from 5.9% to 9.3%, and at the same
time increased effectiveness in terms of<EMAIL_ADDRESS>from 65.5% to 66.4%. In the
case of a models trained on the Extended TACO dataset with one class<EMAIL_ADDRESS>reached 9.6 percentage points less (56.8%) than the one that was trained on
the detect-waste+ dataset (66.4%). However, in respect to detected objects
size, among all conducted experiments, it reached the best results for small
(19.8%) objects.

On the other hand, experiments in which detected trash was divided into seven
classes (bio, glass, metals and plastic, non-recyclable, other, paper,
unknown) resulted in significantly reduced mAP in all analyzed evaluation
metrics, reaching almost 4 times smaller value for both IoU equal 0.50 (16.2%)
and 0.75 (13.0%) than the best solution. Regarding detected objects size, for
tiny objects multi-label detector trained on Extended TACO dataset reached
better results (6.4%) than the one that was trained on detect-waste dataset
with one class (5.9%). This may be due to the fact that in Extended TACO,
approximately 45% of instances are small (area < 322), while for detect-waste
it is only almost 25% of whole dataset. For that reason, feeding the detect-
waste dataset with cigarettes butt improved the quality of prediction. It is
worth emphasizing that a neural network that provides an ability to detect
directly into 7 waste categories demonstrated average precision similar to
one-class based detectors only for one category – metals and plastic –
reaching<EMAIL_ADDRESS>= 43.3%. Results obtained for the remaining six classes
varied in range of<EMAIL_ADDRESS>= 0.1% for bio to<EMAIL_ADDRESS>= 8.9% for unknown
litter. For that reason, it was decided to perform classification in a
separate stage.

###  3.3 Litter Classification

At the second stage of our approach, we performed multi-class classification
for seven waste categories. Due to the imbalanced classify-waste dataset with
a significant dominance of metals and plastic class, classification networks
were trained on cut out trash with some additions of underrepresented classes
from other classification datasets as presented in Fig. 5 and described in
Section 3.1. Boundaries of cropped litter were established using bounding
boxes for annotated images and objects detected by EfficientDet-D2 [42] in
case of unlabeled data. These all combined waste instances were applied as an
input images to solve the classification problem.

To take advantage of the semi-labelled Open Litter Map dataset [57] pseudo-
labeling [71] technique was applied. Pseudo-labeling is a kind of semi-
supervised teaching method. The main concept of this approach is to teach a
network to detect objects on data with and without annotation. Firstly
unlabeled data is preassigned to some category by pretrained model, and
afterwards used in further training. The process is repeated during the
training every batch or epoch, as is presented in Fig. 7. This provides
considerably larger, yet only partially annotated, dataset.

####  3.3.1 Training details

In our research we have tested two architectures dedicated to object
classification problem - ResNet-50 [12] and EfficientNet-B2 [18] to chose the
one that achieves better performance for waste classification task. We have
thoroughly examined number of training hyperparameters. During all
experiments, we set the learning rate to 1e-4, batch size to 16, and trained
each network for 20 epochs. The output of our network differed between 7 and 8
classes. The first 7 classes refer to previously mentioned waste categories,
while the 8th was a background class in order to reduce number of false-
positives. Also, two kinds of samplers – random and weighted – were used to
reduce the impact of data imbalance. Therefore it was crucial to investigate
results on each class, not only on a whole dataset. The other hyperparameter
was the "pseudo-labelling type" that shows if the pseudo-labels update was
performed every batch or every epoch, or not at all.

Moreover, to additionally ensure our training dataset’s diversity, we applied
data augmentation [67]. At first, we extended the dimensions of our images to
randomly crop parts of them. Horizontal/vertical flip and shift scale
rotation, which does not change the image’s content but only the position of
an object in the image, were also applied. On the other hand, a random
brightness contrast and cutout were used to change the image’s content
slightly. Finally, both training and testing sets were normalized and resized
to 224x224, as it was an EfficientNet-B2 [18] input size. The second stage of
our approach was self-implemented in Pytorch Lightning and based on the
repository [72].

Figure 7: Flowchart for pseudo-labeling semi-supervised learning technique.

####  3.3.2 Results

A comparison of the performance of two different neural networks is presented
in Table 7. One of them is EfficientNet-B2 - the backbone of the network used
in the detection task. The results achieved using this classifier exceeded
markedly higher values (over 10 percentage points) than ResNet-50.
EfficientDet-B2 being state-of-the-art architecture comparing to ResNet-50
provides better performance in the classification task, similary as in
detection, where backbones are EfficientNet-B2 for EfficientDet-D2 and
ResNet-50 for Mask RCNN or DETR (see Table 4). It leads to the conclusion,
that the EfficientNet networks family is a preferred choice to deal with waste
images.

The experiments showed that updating pseudo-labels every batch can slightly
raise accuracy. Although the best-achieved accuracy was 74.6%, it was
performed for a random sampler. While analyzing the confusion matrices of each
training, we have noticed that applying a weighted sampler provides more
balanced results for each class. Therefore we have achieved an accuracy of 73%
(and 86.7% on the training set), while almost 25% of our dataset were test
images.

Although the confusion matrix clearly shows that most of the predictions are
accurate (see Fig. 8, and Table 8), it indicates a significant data imbalance.
The metals and plastic class was predicted with the highest precision of 87%,
which is connected to the large representatives of this class. Still, it also
results in a relatively low recall, which means that many objects are
classified incorrectly as metals and plastic. There was a noticeable problem
with identifying the unknown and non-recyclable classes, which precision was
equal 52%. The other classes were recognized with the higher precision but
still, due to the data imbalance, not fully correct.

The biggest confusion was remarked between metals and plastic and unknown
category. Probably it was because of partly degraded or destroyed trash. All
the classes were rarely misclassified with the glass, as evidenced by a high
recall value - 82%. The F1-score metric for this class achieved the utmost
result - 83%, which shows the balance between false positives and false
negatives. It is worth noticing, that eliminating background from the rest of
the waste was extremely successful - at the level of 97% for precision and 97%
for recall. Adding a separate class for background improved the performance
and as it was assumed, reduced the number of false-positives.

Table 7: Comparison of accuracy of selected classifiers. Model | # classes | Accuracy | Sampler | Pseudo-labeling type
---|---|---|---|---
EfficientNet-B2 | 8 | 73.02 | Weighted | per batch
EfficientNet-B2 | 8 | 74.6 | Random | per epoch
EfficientNet-B2 | 8 | 72.8 | Weighted | per epoch
EfficientNet-B4 | 7 | 71.0 | Random | per epoch
EfficientNet-B4 | 7 | 67.6 | Weighted | per epoch
EfficientNet-B2 | 7 | 72.7 | Random | per epoch
EfficientNet-B2 | 7 | 68.3 | Weighted | per epoch
EfficientNet-B2 | 7 | 74.4 | Random | None
ResNet-50 | 8 | 60.6 | Weighted | None

Figure 8: Evaluation of the accuracy of a classification in form of confusion matrix for EfficientNet-B2, Weighted Sampler, and Pseudolabelling per batch.  Table 8: Summary of the precision, recall, and F1-score for each class of waste. Class name | Precision | Recall | F1-score
---|---|---|---
background | 0.97 | 0.97 | 0.97
bio | 0.62 | 0.51 | 0.56
glass | 0.83 | 0.82 | 0.83
metals and plastic | 0.87 | 0.70 | 0.78
non-recyclable | 0.52 | 0.65 | 0.58
other | 0.71 | 0.74 | 0.72
paper | 0.62 | 0.76 | 0.68
unknown litter | 0.52 | 0.65 | 0.58

##  4 Conclusions and Future Work

There is a visibly increasing demand for artificial intelligence in numerous
human activities. Following that, we proposed a DL-based framework that can
localize trash in the image and then identify its class using two separate
neural networks. Datasets used in this study were created by using various
publicly available data of waste collected in diverse environments: inside
houses, in the natural or urban environment and even underwater.

Firstly, three neural networks architectures were adapted to conduct the
garbage detection. Used data was naturally augmented, which allowed for a
precise location of the waste with<EMAIL_ADDRESS>equals 66.4% for EfficientDet-D2
model. This is an excellent result comparing to other recent reports, that
were presented for some of used datasets separately (15.9% for TACO with Mask
R-CNN [53], or 55.4% for TrashCan 1.0 with Mask R-CNN [52]). Qualitatively
good results were also observed for the semi-labeled OpenLitterMap dataset,
allowing for its further use in the next stage.

In the case of classification, litter was divided into seven categories that
imitate the sorting policy introduced in the city of Gdańsk with extra
background class to eliminate false positives coming from the detector.
Additionally, during training pseudo-labeling technique was applied, which
allowed to utilize an unlabeled data. However, this gave only a slight
performance boost in case of litter classification, which can be related to
high imbalance and small amounts of labeled data for specific litter
categories, especially in the case of bio or other waste. In the end, the
accuracy up to 75% was achieved for the EfficientNet-B2 network. To the best
of our knowledge, we present one of the first results to classify litter in
wild.

As Artificial Intelligence is required to be more accurate than a human, the
main future direction for the proposed system will be to improve its
performance. Selected detectors works well when localizing medium and large
objects, but recognition of small litter is still challenging. For that
reason, exploring different state-of-the-art approaches, such as Deformable
DETR [41], seems to be a good idea. On the other hand, a more balanced dataset
and the use of the latest EfficientNetv2 [19], could also boost the
classification accuracy.

Despite this, the presented framework showed the great potential of the DL-
based methodology for waste management. In the future with the assistance of
DL models, it would be possible to mount robotic arms in waste management
plants to automatically distinguish between different classes of objects and
sort garbage without human intervention. Additionally, the high precision of
litter localization in a large variety of the environments, shows the
possibility of using neural networks for waste monitoring in cities or
detecting of illegal dumps in nature, for example with the use of drones.

##  5 Acknowledgements

MF, ZK, SM and AM contributed to the final version open-source code. MF, ZK
and SM analyzed and interpreted the waste datasets regarding the waste sorting
rules in Gdańsk (Poland). AM and SM planned the experiments and adjusted the
models. SM prepared the results of the experiments. AM lead the team during
the project. SM took the lead in the preparation of the manuscript. AM
prepared the illustrations. AK and KM helped supervise the project.

SM wrote the Introduction. SM and AM conducted the researched on available
datasets. AK conducted and written the review on the deep learning
classification. KM conducted and written the review on the deep learning
object detection. ZK analyzed the data and written the datasets sections. MP
and MF described our approach and experiments. SM concluded our project and
proposed future directions.

All authors provided critical feedback and helped shape the research, analysis
and manuscript.

The 5-month (October 2020 -February 2021) lasting project "Detect Waste in
Pomerania" was organized and lead by Agnieszka Mikołajczyk, Magdalena Kortas
and Ewa Marczewska from Women in Machine Learning & Data Science Trojmiasto. A
team of carefully selected members, nine female data scientists, analysts and
machine learning engineers supported by five industry mentors studied and
worked together on developing a model for trash detection. This solution would
be applicable for video and photography. The authors acknowledge other team
members: Anna Brodecka, Katarzyna Łagocka, Ewa Marczewska, Pedro F. Proença,
Adam Kaczmarek and Iwona Sobieraj who contributed to the project.

The authors acknowledge the infrastructure and support of Digital Innovation
Hub dih4.ai in Gdańsk. The authors wish to express their thanks to Voicelab.ai
for the financial support and Epinote for data annotation.

## References

  * [1] Global waste to grow by 70 percent by 2050 unless urgent action is taken: World bank report.  https://www.worldbank.org/en/news/press-release/2018/09/20/global-waste-to-grow-by-70-percent-by-2050-unless-urgent-action-is-taken-world-bank-report, 2016\.
  * [2] No plastic in nature: Assessing plastic ingestion from nature to people.  https://www.newcastle.edu.au/newsroom/featured/plastic-ingestion-by-people-could-be-equating-to-a-credit-card-a-week, 2019\.
  * [3] The new plastics economy rethinking the future of plastics.  http://www3.weforum.org/docs/WEF_The_New_Plastics_Economy.pdf, 2016\.
  * [4] Biswajit Ruj, Vivek Pandey, Priyajit Jash, and V Srivastava.  Sorting of plastic waste for effective recycling.  Int. J. Appl. Sci. Eng. Res., 4, 01 2015.
  * [5] Gary White, Christian Cabrera, Andrei Palade, Fan Li, and Siobhan Clarke.  Wastenet: Waste classification at the edge for smart bins, 2020.
  * [6] Teoh Ji Sheng, Mohammad Shahidul Islam, Norbahiah Misran, Mohd Hafiz Baharuddin, Haslina Arshad, Md. Rashedul Islam, Muhammad E. H. Chowdhury, Hatem Rmili, and Mohammad Tariqul Islam.  An internet of things based smart waste management system using lora and tensorflow deep learning model.  IEEE Access, 8:148793–148811, 2020.
  * [7] Arash Tavakoli, Amir Ashrafi, Arsalan Heydarian, and Madhur Behl.  The internet of wasted things (iowt).  In IOT ’18: Proceedings of the 8th International Conference on the Internet of Things, IOT ’18, New York, NY, USA, 2018. Association for Computing Machinery.
  * [8] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton.  Deep learning.  Nature, 521(7553):436–444, 2015.
  * [9] Yann Glouche and P. Couderc.  A smart waste management with self-describing objects.  In The Second International Conference on Smart Systems, Devices and Technologies (SMART’13), Rome, Italy, 06 2013.
  * [10] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton.  Imagenet classification with deep convolutional neural networks.  In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 25, pages 1097–1105. Curran Associates, Inc., 2012.
  * [11] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
  * [12] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016.
  * [13] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich.  Going deeper with convolutions.  In Computer Vision and Pattern Recognition (CVPR), 2015.
  * [14] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna.  Rethinking the inception architecture for computer vision.  In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition,, 2016.
  * [15] Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A. Alemi.  Inception-v4, inception-resnet and the impact of residual connections on learning.  In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI’17, page 4278–4284. AAAI Press, 2017.
  * [16] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q. Weinberger.  Densely connected convolutional networks.  In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2261–2269, 2017.
  * [17] Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He.  Aggregated residual transformations for deep neural networks.  In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5987–5995, 2017.
  * [18] Mingxing Tan and Quoc Le.  EfficientNet: Rethinking model scaling for convolutional neural networks.  In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 6105–6114. PMLR, 09–15 Jun 2019.
  * [19] Mingxing Tan and Quoc V. Le.  Efficientnetv2: Smaller models and faster training, 2021.
  * [20] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby.  An image is worth 16x16 words: Transformers for image recognition at scale, 2020.
  * [21] Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou.  Training data-efficient image transformers & distillation through attention.  arXiv preprint arXiv:2012.12877, 2020.
  * [22] Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao.  Pyramid vision transformer: A versatile backbone for dense prediction without convolutions.  arXiv preprint arXiv:2102.12122, 2021.
  * [23] Haiping Wu, Bin Xiao, Noel Codella, Mengchen Liu, Xiyang Dai, Lu Yuan, and Lei Zhang.  Cvt: Introducing convolutions to vision transformers.  CoRR, abs/2103.15808, 2021.
  * [24] Zhengxia Zou, Zhenwei Shi, Yuhong Guo, and Jieping Ye.  Object detection in 20 years: A survey.  arXiv preprint arXiv:1905.05055, 2019.
  * [25] Paul Viola and Michael Jones.  Rapid object detection using a boosted cascade of simple features.  In Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition. CVPR 2001, volume 1, pages I–I. IEEE, 2001.
  * [26] Navneet Dalal and Bill Triggs.  Histograms of oriented gradients for human detection.  In 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), volume 1, pages 886–893. Ieee, 2005.
  * [27] Chunhui Gu, Joseph J Lim, Pablo Arbeláez, and Jitendra Malik.  Recognition using regions.  In 2009 IEEE Conference on computer vision and pattern recognition, pages 1030–1037. IEEE, 2009.
  * [28] Jasper RR Uijlings, Koen EA Van De Sande, Theo Gevers, and Arnold WM Smeulders.  Selective search for object recognition.  International journal of computer vision, 104(2):154–171, 2013\.
  * [29] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik.  Rich feature hierarchies for accurate object detection and semantic segmentation.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 580–587, 2014.
  * [30] Ross Girshick.  Fast r-cnn.  In Proceedings of the IEEE international conference on computer vision, pages 1440–1448, 2015.
  * [31] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks.  arXiv preprint arXiv:1506.01497, 2015.
  * [32] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg.  Ssd: Single shot multibox detector.  In European conference on computer vision, pages 21–37. Springer, 2016.
  * [33] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi.  You only look once: Unified, real-time object detection.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 779–788, 2016.
  * [34] Joseph Redmon and Ali Farhadi.  Yolo9000: better, faster, stronger.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7263–7271, 2017.
  * [35] Joseph Redmon and Ali Farhadi.  Yolov3: An incremental improvement.  arXiv preprint arXiv:1804.02767, 2018.
  * [36] Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao.  Yolov4: Optimal speed and accuracy of object detection.  arXiv preprint arXiv:2004.10934, 2020.
  * [37] Chien-Yao Wang, Alexey Bochkovskiy, and Hong-Yuan Mark Liao.  Scaled-yolov4: Scaling cross stage partial network.  arXiv preprint arXiv:2011.08036, 2020.
  * [38] Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, et al.  The open images dataset v4.  International Journal of Computer Vision, pages 1–26, 2020.
  * [39] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko.  End-to-end object detection with transformers.  In European Conference on Computer Vision, pages 213–229. Springer, 2020.
  * [40] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  arXiv preprint arXiv:1706.03762, 2017.
  * [41] Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai.  Deformable detr: Deformable transformers for end-to-end object detection.  arXiv preprint arXiv:2010.04159, 2020.
  * [42] M. Tan, R. Pang, and Q. V. Le.  Efficientdet: Scalable and efficient object detection.  In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10778–10787, 2020.
  * [43] Golnaz Ghiasi, Tsung-Yi Lin, and Quoc V Le.  Nas-fpn: Learning scalable feature pyramid architecture for object detection.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7036–7045, 2019.
  * [44] Pedro H. O. Pinheiro, Ronan Collobert, and Piotr Dollár.  Learning to segment object candidates.  CoRR, abs/1506.06204, 2015.
  * [45] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick.  Mask r-cnn.  In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017.
  * [46] M. Yang and G. Thung.  Trashnet.  https://github.com/garythung/trashnet, 2016.
  * [47] Cenk Bircanoğlu and Nafiz Arıca.  A comparison of activation functions in artificial neural networks.  In 2018 26th Signal Processing and Communications Applications Conference (SIU), pages 1–4, 05 2018.
  * [48] Yinghao Chu, Chen Huang, Xiaodan Xie, Bohai Tan, Shyam Kamal, , and Xiaogang Xiong.  Multilayer hybrid deep-learning method for waste classification and recycling.  Computational Intelligence and Neuroscience, 2018, nov 2018.
  * [49] Oluwasanya Awe, Robel Mengistu, and Vikram Sreedhar.  Smart trash net: Waste localization and classification, 2017.
  * [50] Ying Liu, Zhishan Ge, Guoyun Lv, and Shikai Wang.  Research on automatic garbage detection system based on deep learning and narrowband internet of things.  Journal of Physics: Conference Series, 1069:012032, aug 2018.
  * [51] M. Fulton, J. Hong, M. J. Islam, and J. Sattar.  Robotic detection of marine litter using deep visual detection models.  In 2019 International Conference on Robotics and Automation (ICRA), pages 5752–5758, 2019.
  * [52] Jungseok Hong, Michael Fulton, and Junaed Sattar.  Trashcan: A semantically-segmented dataset towards visual detection of marine debris, 2020.
  * [53] Pedro F Proença and Pedro Simões.  Taco: Trash annotations in context for litter detection.  arXiv preprint arXiv:2003.06975, 2020.
  * [54] B. D. Carolis, F. Ladogana, and N. Macchiarulo.  Yolo trashnet: Garbage detection in video streams.  In 2020 IEEE Conference on Evolving and Adaptive Intelligent Systems (EAIS), pages 1–7, 2020.
  * [55] Tao Wang, Yuanzheng Cai, Lingyu Liang, and Dongyi Ye.  A multi-level approach to waste object segmentation.  Sensors, 20(14), 2020.
  * [56] Marek Kraft, Mateusz Piechocki, Bartosz Ptak, and Krzysztof Walas.  Autonomous, onboard vision-based trash and litter detection in low altitude aerial images collected by an unmanned aerial vehicle.  Remote Sensing, 13(5), 2021.
  * [57] S Lynch.  Openlittermap.com – open data on plastic pollution with blockchain rewards (littercoin).  Open geospatial data, softw. stand., 3(6), 2018.
  * [58] Waste pictures.  https://www.kaggle.com/wangziang/waste-pictures, 2019.
  * [59] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick.  Microsoft coco: Common objects in context.  In David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars, editors, Computer Vision – ECCV 2014, pages 740–755, Cham, 2014. Springer International Publishing.
  * [60] Let’s Do It Foundation.  wade-ai.  https://github.com/letsdoitworld/wade-ai, 2016.
  * [61] Arkadiy Serezhkin.  Drinking waste classification.  https://www.kaggle.com/arkadiyhacks/drinking-waste-classification, 2020\.
  * [62] Adam Kelly.  Cigarette butt dataset & trained weights.  https://www.immersivelimit.com/datasets/cigarette-butts, 2018.
  * [63] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba.  Places: A 10 million image database for scene recognition.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(6):1452–1464, 2018.
  * [64] Hardik Vasa.  google-images-download.  https://github.com/hardikvasa/google-images-download, 2019.
  * [65] Ross Wightman.  Efficientdet (pytorch).  https://github.com/rwightman/efficientdet-pytorch, 2020.
  * [66] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko.  End-to-end object detection with transformers, 2020.
  * [67] Alexander Buslaev, Vladimir I. Iglovikov, Eugene Khvedchenya, Alex Parinov, Mikhail Druzhinin, and Alexandr A. Kalinin.  Albumentations: Fast and flexible image augmentations.  Information, 11(2), 2020.
  * [68] Jimmy Lei Ba Diederik P. Kingma.  Adam: A method for stohastic optimisation.  arXiv preprint arXiv:1412.6980v9, 2017.
  * [69] https://cocodataset.org/#download, 2017.
  * [70] Mark Everingham, S. M. Ali Eslami, Christopher K. I. Williams Luc Van Gool, John Winn, and Andrew Zisserman.  The pascal visual object classes challenge: A retrospective.  Int J Comput Vis (2015) 111:98–136, 2014.
  * [71] Dong-Hyun Lee.  Pseudo-label : The simple and efficient semi-supervised learning method for deep neural networks.  ICML 2013 Workshop : Challenges in Representation Learning (WREPL), 07 2013.
  * [72] Luke Melas-Kyriazi.  Efficientnet-pytorch.  https://github.com/lukemelas/EfficientNet-PyTorch, 2020.