# Deep Convolutional Neural Network for Plume Rise Measurements in Industrial
Environments

Mohammad Koushafar  *Gunho Sohn  Mark Gordon

###### Abstract

The estimation of plume cloud height is essential for air-quality transport
models, local environmental assessment cases, and global climate models. When
pollutants are released by a smokestack, plume rise is the constant height at
which the plume cloud is carried downwind as its momentum dissipates and the
temperatures of the plume cloud and the ambient equalize. Although different
parameterizations and equations are used in most air quality models to predict
plume rise, verification of these parameterizations has been limited in the
past three decades. Beyond validation, there is also value in real-time
measurement of plume rise to improve the accuracy of air quality forecasting.
In this paper, we propose a low-cost measurement technology that can monitor
smokestack plumes and make long-term, real-time measurements of plume rise,
improving predictability. To do this, a two-stage method is developed based on
deep convolutional neural networks. In the first stage, an improved Mask R-CNN
is applied to detect the plume cloud borders and distinguish the plume from
its background and other objects. This proposed model is called Deep Plume
Rise Net (DPRNet). In the second stage, a geometric transformation phase is
applied through the wind direction information from a nearby monitoring
station to obtain real-life measurements of different parameters. Finally, the
plume cloud boundaries are obtained to calculate the plume rise. Various
images with different atmospheric conditions, including day, night, cloudy,
and foggy, have been selected for DPRNet training algorithm. Obtained results
show the proposed method outperforms widely-used networks in plume cloud
border detection and recognition.

_Keywords_ : Plume rise, Data science, Smoke recognition, Smoke border
detection, Deep learning, Mask R-CNN, Geometric transformation

##  1 Introduction

When a smokestack releases pollutants, the resulting plume will rise due to
momentum and buoyancy until it dissipates and is carried downwind at a
constant height. This height is called plume rise height or plume rise. Plume
rise calculation is not straightforward and it is a significant problem in
predicting dispersion of different harmful effluents into the air [1]. Plume
rise influences how far pollutants are carried downwind, the concentration of
pollutants at the surface, and where the pollutants will be deposited on the
green environment or inhaled by people, as well as the amount of greenhouse
gases mixed into the upper troposphere. Therefore, accurate measurement of the
plume rise is of concern for research and operational applications such as
air-quality transport models, local environment assessment cases and global
climate models [2].

The parameterizations of plume rise prediction were developed in the 1960’s by
Briggs [3, 4]. Dimensional analysis was used to estimate the plume rise based
on smokestack parameters and meteorological measurements in different
atmospheric conditions. Early observations of plume rise were used to test and
modify the parameterizations developed using dimensional analysis [5]. Some of
these calibration techniques were wind tunnel studies and field observations
using the technologies including film photography, theodolites, and cloud-
height searchlights [6]. Three-dimensional air-quality models that use these
equations include GEM-MACH [7], CAMx [8], and CMAQ [9].

The parameterizations of plume rise prediction were tested in the 1970’s and
1980’s. The studies compared plume rise parameterizations to observations and
showed that Briggs overestimate the plume rise [10, 11, 12, 13]. In 1993, an
aircraft-based measurement was done to measure $\mathrm{SO_{2}}$ emissions of
a power plant which demonstrated an overestimation of about 400 m [14].
Although, these earlier studies were indicating some degree of overestimation,
in 2002 Webster et al. performed surface measurements and concluded that the
Briggs parameterizations tend to underestimate plume rise [15]. In 2013, as
part of the Canada-Alberta Joint Oil Sands Monitoring (JOSM) Plan, an aerial
measurement study was done in the Athabasca oil sands region of northern
Alberta to study dispersion and chemical processing of emitted pollutants [16,
17, 18]. The project consisted of 84 flight hours of an instrumented Convair
aircraft over 21 flights designed to measure pollutants emissions, study the
transformation of chemicals downwind of the industry, and verify satellite
measurements of pollutants and greenhouse gases in the region. Using aircraft-
based measurements and reported smokestack parameters and meteorological data,
it was demonstrated that the Briggs equations significantly underestimated
plume rise at this location, in contrast to previous results.

Given the results of this study and the gap of more than 30 years since the
Briggs equations were developed and previously tested, there is a need for
further testing and possible modification of the Briggs equations based on
modern observation techniques. In recent decades, there have been many
significant advancements in environmental monitoring activities over
industrial regions in purpose of safety and pollution prevention [19, 20, 21,
22]. Moreover, several smoke border detection and recognition models have been
introduced recently using digital image analysis, such as wavelet and support
vector machines [23], LBP and LBPV pyramids [24], multi-scale partitions with
AdaBoost [25], and high-order local ternary patterns [26] which are well-
performed and impressive. This will lead to the development of instrumentation
which can be deployed near any smokestack to give information on pollutants
dispersion and potential exposure to people downwind. This information will be
based on actual real-time observation, i.e. video images, as opposed to
potentially erroneous and decades-old parameterizations.

In order to find the smoke within an image or a video frame, either a rough
location of smoke is identified using bounding boxes which is called smoke
border detection [25, 27], or pixels are identified and classified in detail
which is called smoke recognition [28, 29]. Due to translucent edges of smoke
clouds, the recognition task needs far more accuracy than border detection.
Traditional smoke recognition methods utilize manual features which lead to
low accuracy recognition results, due to a large variety of smoke appearance.
These low-level features consist of motion characteristic analysis of the
smoke [30, 31], smoke color [32, 33, 34], and smoke shape [35]. Also [36, 37]
took advantages of the Gaussian Mixture Model (GMM) to detect the motion
region of the smoke and [38] combined rough set and region growing methods as
a smoke recognition algorithm which seems to be a time-consuming algorithm due
to the computational burden of the region growing method. Although the use of
color information is less effective due to similarity of smoke color to its
surrounding environment, the combination of motion and color characteristics
of smoke is also known as a common method in smoke recognition [39, 40, 41].
Some algorithms utilize infra-red images and video frames in their experiments
[42] which are not easily accessible and can increase the costs of the
project. Moreover, using digital images makes the algorithm more flexible as
it can be used with more hardware systems. On the other hand, some plumes are
too close to background temperature to be captured by the near red-channel
wave length. A higher order dynamical system was introduced in 2017 which used
particle swarm optimization for smoke pattern analysis [43]. However, this
early smoke identifying approach had a low detection rate and high
computational complexity.

In recent years, deep learning based methods, especially convolutional neural
network (CNN) based methods, have led to achieve significant results in
semantic segmentation [44, 45, 46, 47] and object recognition [48, 49, 50, 51,
52]. Similarly, these methods are widely used in smoke border detection and
recognition [53, 54, 55] with different architectures such as three-layer CNN
[56], generative adversarial network (GAN) [57] and two-path fully
convolutional network (FCN) [28]. Recently, a count prior embedding method was
proposed for smoke recognition to extract information about the counts of
different pixels (smoke and non-smoke) [58]. Experimental results showed an
improvement in recognition performance of these studies. However, high
computational complexities of these huge models are an obstacle for using them
in plume rise real-time observations. Furthermore, few studies have
represented smoke recognition and border detection results together, which is
more relevant in visual tracking of plume cloud than recognition alone. To the
best of our knowledge, there is currently no method that uses deep learning
for plume rise measurements.

We have proposed a novel method using deep neural network algorithms to
measure plume rise. Our method comprises two parts, 1) plume cloud border
detection and recognition based on an improved Mask R-CNN [59], and 2)
geometric transformation [60] to migrate from image scale measurements to
real-life. This method accurately recognizes the plume cloud and measures
plume rise in real-time. Moreover, border detection performance was improved
via bounding box loss function modification of region proposal network (RPN)
[61, 62, 63]. This modification is performed by adding a new regularization to
the loss function which dictates the coordinates of the most possible plume
cloud regions. We tried to minimize the distance of proposed bounding boxes by
RPN and the desired smokestack end. RPN losses are divided into two
categories, regression (bounding box) loss and classification loss which were
optimized based on defined ground truths. Here, we propose the third element
of regression loss which is called smokestack end loss ($L_{sse}$), to be
added to the main loss function of the system to dictate this restriction to
the model. Moreover, generating limited number of anchor boxes which are only
swarmed across the desired stack exit leads to reduce training time and
computational complexities. Consequently, the main contributions of this paper
can be summarized as follows:

  * •

We propose ”DPRNet” (a deep learning framework for plume rise measurements),
by incorporating plume cloud border detection, and recognition tasks. We have
provided a relatively low-cost and reproducible algorithm to accurately
recognize plume clouds using automated digital image capturing software with
adaptability to utilize RGB wide screen images.

  * •

We propose a plume rise vision-based measurement technique which will improve
plume-rise parameterizations, leading to increased accuracy of air-quality
models. Furthermore, industry might use this real-time visual systems to track
and survey the released pollutants online and obtain information to update
related criteria such as live updates of the air-quality health index (AQHI).

  * •

The lack of adequate real-world annotated datasets is the greatest struggle in
using deep learning for smokestack plume cloud recognition in both research
and industry purposes. To address this, we created our own plume cloud dataset
and labelled the foreground regions manually.

  * •

Unlike most recent studies in the smoke recognition field, we perform both
recognition and border detection tasks of the plume cloud based on improved
state-of-the-art CNN model with less computational complexities. This has led
to the building of a comprehensive operational system to better observe the
plume clouds behaviors within an industrial area with instant plume cloud
measurement announcements. A Developed system would be of direct interest to
industries looking to confirm or develop emissions reporting and/or
environmental assessments.

This paper is organized as follows. Section 2 gives a brief explanation of the
theoretical information used in our proposed method. In section 3, we provide
a detailed description of our proposed framework for plume rise measurement of
a desired smokestack. Then, section 4 presents our dataset collection
procedure, experimental results of the proposed method and evaluation results
using different metrics, and calculations of the plume rise. Finally,
conclusions of this research, our findings, and future studies are described
in section 5.

##  2 Theoretical background

###  2.1 Plume rise

The plume rise calculation is a non-straightforward task which is used in
harmful effluents dispersion prediction in atmospheric science [1]. Plume rise
is affected by two phenomena, buoyancy and momentum. Mostly, the plume clouds
are buoyant which means that they are hotter than the ambient air. Therefore,
they rise due to the less dense than the surrounding air. Also, the plumes
have a vertical velocity and momentum when they exit the smokestack which
again cause them rise. Plumes can also fall due to the gravitational force
when they are cold and dense and also when some surrounding obstacles cause
them move downwind [64]. In 1975 Briggs proposed an equation for the maximum
distance of the plume rise. These equations do not obey the dimensions and the
relationship between their parameters are not fundamental. However, they are
practically suitable to calculate plume rise. Considering both momentum and
buoyancy in calculations, plume rise ($\Delta{z}$) in horizontal distance x
from smoke stack end can be obtained as [4],

$\Delta{z}=\Biggl{(}\frac{3\,F_{m}x}{0.6^{2}\,\bar{u}^{2}}+\frac{3\,F_{b}x^{2}}{{2}\times
0.6^{2}\,\bar{u}^{3}}\Biggl{)}^{1/3}$ (1)

where the factor $0.6$ is an entrainment rate (the mean rate of increase of
plume cloud in the wind direction) and $u$ is mean horizontal wind speed.
Also, Momentum flux parameter ($F_{m}$) and Buoyancy flux parameter ($F_{b}$)
are defined as below,

$F_{m}=[\frac{\bar{\rho_{s}}}{\bar{\rho}}]\,r^{2}_{s}\bar{w_{s}}^{2}$ (2)

$F_{b}=[1-\frac{\bar{\rho_{s}}}{\bar{\rho}}]\,gr^{2}_{s}\bar{w_{s}}^{2}$ (3)

where $\bar{\rho_{s}}$ is smokestack gas density, $\bar{\rho}$ is atmospheric
air density, $r_{s}$ is smokestack radius, $\bar{w_{s}}$ is vertical velocity
of the smokestack gas, $g$ is acceleration due to gravity, $T$ is the
atmosphere temperature, $\theta$ is potential temperature, and $z$ is height.

It should be noted if we evaluate the wind speed at the local plume height and
not the source height, the calculations should be operated iteratively [65,
66]. Plume buoyancy and momentum horizontal movements and consequently plume
rise are strongly affected by the wind [1]. Moreover, stable conditions with a
low amount of turbulence, plume rise is not affected by the wind speed
fluctuations, which make the measurements very difficult. However, in unstable
conditions, large plume rise variations have been witnessed at a fixed
distance downwind.

###  2.2 Convolutional neural network and convolutional layer

Convolutional neural networks (CNNs) are a special type of neural networks
that are suitable for processing grid data such as image data which has a two-
dimensional or three-dimensional mesh structure of pixels. The name given to a
convolutional neural network is derived from the convolutional layers used in
it. The main spark for the creation of convolutional neural networks was
struck in the 1960s during research on the visual cortex [67]. According to
this research, in the early layers of the brain, neurons show sensitivity to
edges and in later layers to more complex features. Also, one of their
important findings is that the local structures of images are preserved in the
brain. Based on these biological inspirations, the first neural model was
designed by Fukushima [68], although it could not be trained well due to some
problems. After the failure of Fukushima, LeCun [69] designed a network based
on convolutional layers, which was able to achieve very good results. Since
then, convolutional networks have been widely used in many fields and
remarkable results have been obtained.

In convolutional neural networks, each layer receives the input from the
previous adjacent layer. This feature provides the possibility of learning
more complex patterns in the form of a hierarchical method, by combining
elementary low-level features and high-level features. For example, edges can
be extracted from raw image pixels. On the other hand, edges can be used to
extract simple shapes, and finally, shapes can be used to identify objects. In
convolutional networks, by sliding kernels on the input image, meaningful and
useful features can be extracted for the main purpose of the network. These
kernels, which have weights and small dimensions, are applied to all the
pixels of the image by sliding on it. Therefore, through the sharing of
parameters, convolutional networks have fewer parameters compared to fully
connected networks, which will reduce the computational load and the
possibility of overfitting [70]. In addition, with the local application of
kernels, these networks, like the function of the human visual system, can
recognize a feature regardless of its displacement in the image by considering
the relative positions, which is a very useful feature in many fields.

Each convolutional layer contains a number of kernels and biases that are
locally applied to the input and produce a feature map or an activation map
according to the number of filters. Calculations in these kernels are
performed by the cross correlation operator, whose kernel is the negative of
its definition in the convolution operator. If this convolutional layer is
applied to the input image in a two-dimensional manner, its $j^{th}$ feature
map $O_{j}$, which is obtained by applying the $j^{th}$ kernel, is calculated
at the position $(x,y)$ from the following relation [71],

$O_{j}^{xy}=B_{j}+\sum_{k}\sum_{m=0}^{M-1}\sum_{n=0}^{N-1}w_{j}^{mn}z_{k}^{(x+m)(y+n)}$
(4)

where $k$ moves along the depth dimension of input $Z\in\mathbb{R}^{p\times
q\times r}$ and $w_{j}^{mn}$ is the two-dimensional kernel weight
$W\in\mathbb{R}^{M\times N}$ at position $(m,n)$. $B_{j}$ is the bias matrix.

###  2.3 Mask R-CNN

Mask R-CNN is a member of the region-based CNN family which was proposed in
[59]. It represented the highest accuracy regard COCO dataset segmentation
challenge and afterwards, it is used widely in different recognition tasks
[72, 73, 74]. Firstly, RCNN was proposed in [75] in which region proposals
were generated by computer vision techniques. Then in [63] Fast RCNN was
introduced with a convolutional network before Region Proposals to increase
the computational speed. In 2017, Faster RCNN continued this evolution and
represented a regional proposal network to propose region of interests (RoIs)
[76]. Finally, Mask R-CNN as an extension of Faster RCNN added a convolutional
network for pixel level segmentation of the detected objects. Mask R-CNN is
relatively a simple model and easy to be generalized to other similar tasks
[59]. Therefore, Mask R-CNN can create pixel level masks for the objects
beside object localization and classification tasks.

####  2.3.1 Region proposal network

Region proposal network (RPN) is a deep fully convolutional network which
proposes regions and serves as the attention of the Mask R-CNN. It means that
RPN helps us to selectively focus on valuable parts within the input images.
This network takes an image and gives a set of region proposals beside their
scores for being an object. It slides a window over the convolutional feature
map (backbone output) and maps it to a lower dimension feature. The generated
feature is then fed into two fully-connected layers to obtain the proposed
regions class (object vs. non-object) and four corresponding coordinates [76].
For each sliding window location, there are maximum k possible proposals which
are parameterized relative to k reference boxes or anchors. RPN is trained
end-to-end by back-propagation and stochastic gradient descent. Figure 1 shows
the scheme of RPN, in which the proposed regions are generated as the module
outputs.

Figure 1: Region proposal network.

####  2.3.2 Loss function

Like the other machine learning models, Mask R-CNN uses an objective function
to optimize its parameters in the training phase which is called the loss
function. Mask R-CNN loss function is a weighted summation of different losses
related to different sections of this comprehensive model. As a definition
based on [59], a multi-task loss function is proposed on each sampled RoI as,

$L=L_{cls}+L_{reg}+L_{msk}$ (5)

where $L_{cls}$ represents the loss value for classification task (object
classifier), $L_{reg}$ represents the loss for regression task (bounding boxes
regressor), and $L_{msk}$ shows the loss value for mask classification (pixel
classifier). If we consider the whole model as a combination of two modules of
MRCNN and RPN, MRCNN loss can be divided into three categories, MRCNN
classification loss which reflects how close the model is to predicting the
correct object classes, MRCNN regression loss which represents how good the
model is at precisely predicting the different present object(s) area(s)
within the image , and MRCNN mask classification loss which penalizes the
wrong per-pixel binary classifications (foreground or background), and it is
calculated differently for each of the sampled RoI. Similarly, RPN module loss
can be divided into two categories, RPN classification loss and RPN regression
loss. RPN classification loss corresponds to improper classification of anchor
boxes (presence/absence of any object) by computing the gradients over a
simple cross entropy loss function (classification here is only labeling the
anchor boxes as foreground or background). On the other hand, RPN regression
loss shows how good the model is at locating objects within the image [59].
This regression loss is defined as follows [76],

$L^{rpn}_{reg}=R(t-t^{*}),$ (6)

in which,

$\displaystyle
t_{x}=\frac{x-x_{a}}{w_{a}},\;\;t_{y}=\frac{y-y_{a}}{h_{a}},\;\;t_{w}=\log{(\frac{w}{w_{a}})},\;\;t_{h}=\log{(\frac{h}{h_{a}})},$
(7) $\displaystyle
t^{*}_{x}=\frac{x^{*}-x_{a}}{w_{a}},\;\;t^{*}_{y}=\frac{y^{*}-y_{a}}{h_{a}},\;\;t^{*}_{w}=\log{(\frac{w^{*}}{w_{a}})},\;\;t^{*}_{h}=\log{(\frac{h^{*}}{h_{a}})}$
(8)

where $t$ and $t^{*}$ are the vectors representing the four vertices of our
predicted and ground truth bounding boxes and $R$ is the robust loss function.
Also, $X$, $X_{a}$ and $X^{*}$ are defined as the predicted coordinate, the
anchor coordinate, and the ground truth coordinate, respectively. The
parameters $w$ and $h$ are the height and width of the bounding box.

##  3 Methodology

The plume rise determination system based on DPRNet model designed in this
paper is represented in Figure 2. The data are sent to the DPRNet for plume
cloud border detection and recognition, and the results are combining with the
wind direction information to be processed by geometric transformation
calculations. The main output of the system will reveal the plume rise as a
physical height and the distance downwind at which the plume rise occurs.

Figure 2: Plume rise measurement system framework. $x_{R}$ and $z_{R}$ are the
plume rise distance and the plume rise values in image scale, respectively.
Similarly, $X_{R}$ and $Z_{R}$ are representing the plume rise distance and
the plume rise values in real-life scale, and $WD$ shows the wind direction.

###  3.1 Structure of DPRNet

The main goal of this research is to precisely recognize the plume cloud of
the desired smokestack out of a wide range of image datasets, captured from
the study area. DPRNet is an adopted version of Mask R-CNN with two novel
modules for smokestack plume rise measurements. These modules are, 1) mask
entering, and 2) loss modification adjustment. Figure 3 shows the structure of
this framework in which these modifications are applied to RPN module. This
modules can improve RPN performance in locating the most probable proposal
regions.

Mask R-CNN is the base of our proposed method, which is one of the popular
methods for border detection and recognition tasks, regarding the network
performance and computational complexity [62]. This powerful framework is able
to consider irregular shapes of the plume cloud, its translucent edges, and
similar pixel values of the plume cloud to its background [59]. Figure 3
represents the main structure of DPRNet with four main modules called backbone
network, feature pyramid network (FPN) [77], region proposal network (RPN),
and region of interest align (RoI Align), which are related to conventional
Mask R-CNN architecture.

In this architecture, ResNet50 [78] is used as the backbone network to extract
feature maps of the input images. FPN use all these feature maps to generate
multi-scale feature maps which carry more useful information in comparison
with the regular feature pyramid. Then, RPN detects the plume cloud by sliding
a window over these feature maps to predict whether there is a plume or not
and locate the existing plume by creating bounding boxes. Therefore, we have a
set of plume cloud proposals from RPN and the generated feature map by the
backbone network. RoI Align module works to scale the proposals to the feature
map level and prevent the misalignment by standardizing aspect ratios of the
proposals. Finally, these refined feature maps are sent to three different
outputs. The first one is a classification block which decides whether the RoI
is foreground (plume cloud) or not. The second one is a regression block which
predicts the bounding boxes based on the provided ground truth. And the last
block predicts a segmentation mask for the detected plume cloud using a fully
convolutional network (FCN) [79].

Moreover, two proposed modules are visualised in Figure 3. The first module
inserts the input ground truth to the RPN module, followed by the second
module addressing loss function modification, which improve the main given
architecture for the better border detection and recognition of a desired
stack plume cloud from its background for measurement purposes. These modules
are explained in details in the following subsections:

Figure 3: DPRNet architecture.

####  3.1.1 Mask entering module

We are able to locate the smokestack end using the input image ground truth.
it is obvious that the smokestack exit is the possible region of the plume
cloud and proposed regions can be considered around the end of desired
smokestack. Also, we choose to identify one plume cloud and regardless of
small plume pieces which rarely are seen in the images, we are tracking a
unique plume cloud from a desired smokestack. Therefore, image ground truth
can help us locate the smokestack end for each input.

####  3.1.2 Loss function modification module

Based on section 2.3.2, it is a crucial problem to set an efficient loss
function, which can completely affect the performance of our model. We can add
a new striction/regularization to the loss function which dictates the
coordinates of the most possible plume cloud regions. Indeed, we try to
minimize the distance of proposed bounding boxes by RPN and the smokestack end
(SSE). Based on definition for Equation 6 which is defined for a box with
coordinates of ($x,y,w,h$), we can consider the smokestack end as a point with
the coordinate of ($x,y$). Therefore, the regression loss related to the
smokestack end can be defined as follows,

$L^{sse}_{reg}=R(u-{u^{{}^{\prime}}}),$ (9)

where $u$ and $u^{{}^{\prime}}$ are the points representing the coordinate of
our predicted and ground truth smokestack end and $R$ is the robust loss
function. The definitions of $u$ and $u^{{}^{\prime}}$ depend on the plume
cloud behaviors in each input image which leads to categorize these
definitions into three main states. If the plume cloud is vertical, we propose
that the smokestack end is considered at the middle of box lower side with the
coordinate of $(x+\frac{w}{2},y+h)$. Therefore, we can define the $u$ and
$u^{{}^{\prime}}$ based on Equations 7 as follows,

$\displaystyle
u_{x}=\frac{1}{2w_{a}}(2x+w-2x_{a}-w_{a}),\;\;\;u_{y}=\frac{1}{h_{a}}(y+h-y_{a}-h_{a}),$
(10) $\displaystyle
u^{{}^{\prime}}_{x}=\frac{1}{2w_{a}}(2x^{{}^{\prime}}+w^{{}^{\prime}}-2x_{a}-w_{a}),\;\;\;u^{{}^{\prime}}_{y}=\frac{1}{h_{a}}(y^{{}^{\prime}}+h^{{}^{\prime}}-y_{a}-h_{a})$
(11)

where $X$, $X_{a}$ and $X^{{}^{\prime}}$ are defined as the predicted
smokestack end coordinate, the smokestack end coordinate on the anchor, and
the smokestack end ground truth coordinate, respectively. Also, the parameters
$w$ and $h$ are the height and width of the bonding box.

If the plume cloud is blowing in a direction that results in it being on the
right side of the image, the smokestack end is considered at the left bottom
angle of the box with the coordinate of $(x,y+h)$. Therefore, $u$ and
$u^{{}^{\prime}}$ can be defined as below,

$\displaystyle
u_{x}=\frac{1}{w_{a}}(x-x_{a}),\;\;\;u_{y}=\frac{1}{h_{a}}(y+h-y_{a}-h_{a}),$
(12) $\displaystyle
u^{{}^{\prime}}_{x}=\frac{1}{w_{a}}(x^{{}^{\prime}}-x_{a}),\;\;\;u^{{}^{\prime}}_{y}=\frac{1}{h_{a}}(y^{{}^{\prime}}+h^{{}^{\prime}}-y_{a}-h_{a})$
(13)

And similarly, when the plume cloud is blowing to the left side of the image,
the smokestack end can be considered at the right bottom angle of the box with
the coordinate of $(x+w,y+h)$. Therefore, $u$ and $u^{{}^{\prime}}$ in
Equation 9 are defined as below,

$\displaystyle
u_{x}=\frac{1}{w_{a}}(x+w-x_{a}-w_{a}),\;\;\;u_{y}=\frac{1}{h_{a}}(y+h-y_{a}-h_{a}),$
(14) $\displaystyle
u^{{}^{\prime}}_{x}=\frac{1}{w_{a}}(x^{{}^{\prime}}+w^{{}^{\prime}}-x_{a}-w_{a}),\;\;\;u^{{}^{\prime}}_{y}=\frac{1}{h_{a}}(y^{{}^{\prime}}+h^{{}^{\prime}}-y_{a}-h_{a})$
(15)

Unlike the Mask R-CNN model, the smokestack end regression loss minimizes the
recognition task errors of a specific plume cloud and resolves main problems
of this model such as not touching the desired smokestack end and multi-box
proposal for a single plume cloud. It is worth mentioning that increasing the
number of training images has effectively coped with the low accurate
recognition in plume cloud translucent edges and incomplete boxes for longer
plume clouds. Moreover, $L_{sse}$ helps us avoid define within the whole image
pixels which causes high training time and computational complexities in such
problems. Taking the advantages of $L^{sse}_{reg}$ and $L^{rpn}_{reg}$ at the
same time, verify the main loss function to achieve acceptable border
detection and recognition results which are represented in section 4.

###  3.2 Geometric transformation

To compare the visual results with the real-world results, we need to perform
some calculations to obtain plume rise from the photos using the measured wind
direction. On the other words, usually two cameras are used for finding an
object in 3D space for a stereoscopic image. However, for an evolving and
relatively homogeneous plume, it would be impossible to identify features to
triangulate the 3D points. Hence, we use wind direction to constrain the
location of the plume using only one camera. The main target here is the
conversion from the location in the photograph to a real-life distance. In
order to determine the plume rise height from the images, we need to know what
direction the plume is blowing which requires knowing the wind direction.

Two important parameters in the calculations are Pixel Size (P) and Ground
Sample Distance (G), which help us convert the image scale measurements to
real scale measurements. P can be calculated via camera Field of View
($\alpha$) and camera Focal Length (F). Pixel size is the distance between
pixel centers measured on the pixel array in the film plane
($\mathrm{\mu{m}/px}$) and is considered as a specification of the camera.
While, Ground sample distance represents the distance between pixel centers
measured on the ground ($\mathrm{m/px}$) in the camera focal plane, which is
considered as image resolution on the ground and it is a function of
perpendicular distance from the camera to the image plane (D). Based on the
definitions above, the relation between P and G is written as follows [60],

$\frac{P}{G}=\frac{F}{D}\;\;\;\longrightarrow\;\;\;G=\frac{D}{F}P$ (16)

To calculate pixel size for the camera, we know,

$\tan{(\frac{\alpha}{2})}=\frac{D_{m}}{2F};\;\;\;\;\;D_{m}=\sqrt{H_{m}^{2}+W_{m}^{2}},$
(17)

where $D_{m}$[$mm$] shows the diameter of the camera lens. $H_{m}$ and $W_{m}$
are height and width of camera lens which are obtained via the following
relations,

$\begin{gathered}H_{m}=PH\\\ W_{m}=PW\end{gathered}$ (18)

Therefore, the pixel size of the camera can be calculated as,

$P=\frac{2F\tan{(\frac{\alpha}{2})}}{\sqrt{H^{2}+W^{2}}}$ (19)

As it is clear from equation 16, unlike $P$ which is one of the camera
specifications, $G$ varies with horizontal distance from the image centre.
If point $R$ is introduced as a point on the plume centerline which is the
point of maximum plume rise closest to the stack, the plume rise distance can
be defined as the horizontal distance between the smokestack exit and the
point $R$. Figure 4 shows these definitions on a sample plume cloud image.

Figure 4: Plume rise, plume rise distance, and the point $R$ represented on a
sample captured image.

Therefore, ground sample distance can be calculated based on equations 16 & 19
as,

$G=\frac{D}{F}P\;\;\longrightarrow\;\;G=\frac{2D\tan{(\frac{\alpha}{2})}}{\sqrt{H^{2}+W^{2}}}$
(20)

Figure 5 represents a scheme of the region top view. The plume cloud
representative point $R$ is affected by wind direction and it is out of the
image plane. Therefore, ground sample distance will be different relative to G
at the plume source. We define $G_{R}$ as the value of the ground sample
distance at the location of R.

Figure 5: Schematic top view of the region.

We know that the distance between the camera and the smokestack is 5180m.
Therefore, $G$ can be calculated based on equation 20. Also, based on equation
16, T (the ground distance covered by the image plane at the smokestack
location) can be calculated as [60],

$\frac{P}{G}=\frac{W^{\prime}}{T}=\frac{WP}{T}$ (21)

Therefore, for the under-study region, $G=1.6647m/px$ and $T=4314.91m$.
Also, as we can see in figure 5, X can be obtained by the following relations,

$\frac{X}{T}=\frac{X_{R}}{T_{R}}=\frac{x_{R}}{W}\;\;\;\longrightarrow\;\;\;X=\frac{Tx_{R}}{W}$
(22)

where $x_{R}$ is the horizontal distance of the desired point $R$ in the
image. The value of $X$ is used to find the $\gamma$ value using the following
equation,

$\tan{\gamma}=\frac{X}{D}\;\;\;\longrightarrow\;\;\;\gamma=\tan^{-1}\frac{X}{D}$
(23)

Moreover, based on figure 5,

$\tan{\gamma}=\frac{X_{R}}{D-Y_{R}}\;\;\;\&\;\;\;\tan{\theta}=\frac{Y_{R}}{X_{R}}$
(24)

Wind direction is always reported as degrees from north, which is represented
by $\varphi$. For instance, $\varphi=90^{\circ}$ is wind from the east and
$\varphi=180^{\circ}$ is wind from the south. It is provable that wind
direction relative to the image plane can be obtained as
$\theta=|\varphi-252|$ (Figure 5). Therefore, $X_{R}$ can be calculated by the
following equation,

$X_{R}=\frac{D}{\tan{\theta}+\frac{1}{\tan{\gamma}}}$ (25)

Based on equations 22 & 23, $X$ and consequently $\tan{\gamma}$ can be
obtained for each plume cloud. Therefore, using equation 25, $X_{R}$ can be
calculated and according to the Equation 16, $G_{R}$ and $Z_{R}$ can be
obtained as below,

$G_{R}=\frac{X_{R}}{x_{R}}\;\;\;\&\;\;\;Z_{R}=G_{R}z_{R}$ (26)

Similarly, as we can see in figure 6, $Z_{st}$ which belongs to the image
plane and shows the real stack distance from image center can be calculated
as,

$Z_{st}=Gz_{st}$ (27)

where $z_{st}$ represents the distance between the smokestack end and the
image center within the image.
Therefore, the plume rise ($\Delta{z}$) for each plume cloud can be calculated
as,

$\Delta{z}=|Z_{st}|+Z_{R}$ (28)

Figure 6 represents the smokestack corresponding geometric parameters, plume
rise and plume rise distance of sample plume cloud.

(a) Camera view scheme.

(b) Top view scheme.

Figure 6: Smokestack location schemes. Smokestack location, S; image center,
O; desired point, R; plume centerline, CL; point horizontal distance from
image center, $x_{R}$; point vertical distance from image center, $z_{R}$;
point distance form image center, $z_{st}$; camera station, C; depth of the
point in real world, $Y_{R}$; wind direction angle relative to the image
plane, $\theta$; and the yellow arrow shows the wind direction.

##  4 Experimental results and discussion

In this section, we describe our dataset, and the industrial area in which
this image dataset has been collected and shared. Also, we will explain about
the validation metrics which are used for comparison of our proposed method
with the other competitive method in smoke border detection and recognition.
Finally, our discussion falls into two main parts, named comparison with Mask
R-CNN in which we talk about the performance of our DPRNet method in
comparison with it component method, and plume rise calculation, in which the
plume rise is calculated based on our ”DPRNet”. To validate the performance of
our proposed method, we used a computer equipped with Core i9, 3.70 GHz/4.90
GHz, 20 MB cache CPU, 64GB RAM and NVIDIA GeForce RTX 3080,10 GB graphic card.
The total training time of the network was about 9 hours using Python 3.8 with
PyTorch Deep Learning framework. For a single 2592×1944 test image, the
running time was about 0.27 seconds. Finally, for the close-range
photogrammetry and statistical analysis, we used system identification toolbox
of MATLAB software.

###  4.1 Site description

The imaging system was deployed on a meteorological tower with a clear
sightline to the desired smokestack operated by the Wood Buffalo Environment
Association (WBEA). It is located outside of the Syncrude oil sands processing
facility in north of Fort McMurray, Alberta, Canada. Syncrude Canada Ltd. is
one of the world’s largest producers of synthetic crude oil from oil sands and
the largest single source producer in Canada. The images were captured from
the Buffalo Viewpoint monitoring station, operated byt the Wood Buffalo
Environmental Association (WBEA) as its angle to the area was suitable and we
could see the biggest smokestack and its plume all the time. The distance
between the camera and considered smokestack is about 5.2 kilometers. Figure 7
represents the satellite images of the region and location of the camera and
the desired smokestack.

(a) Top view

(b) 3D view

Figure 7: Imaging situation. Camera station, C; and smokestack position, S.
The abc coordinate system is only for differentiating the side and camera
views and is not used as a co-ordinate reference system.

The Wood Buffalo Environment Association (WBEA) operates a 167-meter tall
meteorological tower with a clear sightline to the smoke stack at Syncrude.
The camera system is mounted on this tower above the tree canopy height which
is about 15 meters. The Syncrude processing facility has six main smokestacks.
The tallest one is about 183 m (600 ft) and the height of other five are
between 31 m to 76 m. In order to isolate a single smoke plume rise, we have
concentrated on the tallest one in the area which can help find the plume rise
for one plume source. All six stacks are listed in Table 1.

Table 1: Syncrude stacks information, including location, stack height
($h_{s}$), stack diameter ($d_{s}$), effluent velocity at the stack exit
($\omega_{s}$), and effluent temperature at the stack exit ($T_{s}$). the
velocities and temperatures are averages for the entire capturing period.

Reported ID Latitude Longitude $h_{s}$(m) $d_{s}$(m)
$\omega_{s}$($\mathrm{ms^{-1}}$) $T_{s}$(K) Syn. 12908 57.041 -111.616 183.0
7.9 12.0 427.9 Syn. 12909 57.048 -111.613 76.2 6.6 10.1 350.7 Syn. 13219
57.296 -111.506 30.5 5.2 8.8 355.0 Syn. 16914 57.046 -111.602 45.7 1.9 12.0
643.4 Syn. 16915 57.046 -111.604 31.0 5.0 9.0 454.5 Syn. 16916 57.297 -111.505
31.0 5.2 9.2 355.0

Wind directions during capturing period were determined from the Mildred lake
Air Monitoring Station (`http://www.wbea.org`), which is located at the
Mildred Lake airstrip (AMS02: Latitude: $57.05^{\circ}$, Longitude:
$-111.56^{\circ}$), approximately 5km from the Syncrude facility.

The plume cloud imaging camera system consists of a digital camera with
shutter control, and a camera housing for weather protection with interior
heating for window defrost and de-icing. The camera activation and temperature
control of the housing are controlled by a data-logger which is powered by a
small solar cell. The camera is mounted high enough to image the smokestack
and plume, ideally at a distance perpendicular to the dominant wind direction.

###  4.2 Dataset

The greatest struggle in using deep learning for smoke recognition is the lack
of adequate annotated images for training purposes. It is obvious that we can
generate a great pile of labelled data by manually separating smoke pixels
from their background in an input image, but it is highly time-consuming and
difficult as the smoke has fuzzy boundaries and translucent properties [28].
Therefore, it is valuable to create image datasets for smoke recognition for
both research and industry purposes.

96 images were captured every day and for the first stack of image data, we
considered 35K image data collecting from January 2019 to December 2019. The
collected image data demonstrate many types of plume shapes in different
atmospheric conditions. To classify the dataset and make decision about
various image classes, there are some criteria such as plume shape, plume
direction and atmospheric conditions and day/night times.

####  4.2.1 Day and Night Images

Collected dataset shows that among 96 images per every 24-hours, we have 48
day images and 48 night images, and for every month of the year, we are
counting 1450 day images and 1450 night images.

####  4.2.2 Authentic Images and Outliers

According to the image dataset and considering this routine for every data
collection phase, there are some outlier images which are not captured well
due to different reasons such as camera handle shaking, auto-focus problem,
disturbing smoke and severe snow and hail falling. There is also a type of
image data which is not considered as outlier data; meanwhile these images
cannot be used as reference (training) data. This is because of the condition
of the plume which cannot be recognized from its background in anyway even
with an armed human-being eyes. Among 35K collected data as the first data
stack, we have _10684_ valid images and 24324 outlier images.

####  4.2.3 Plume and Non-plume Images

Among collected valid data, there are types of images which are both important
for training phase, the plume images with different backgrounds in various
atmospheric conditions and the non-plume images which are important to
represent the times in which there is no foreground. Among 10684 collected
valid data, we have 8310 plume images and 2374 non-plume images.

####  4.2.4 Valid Day Plume Images and Valid Night Plume Images

Finally, among 8310 collected valid plume data, we have 3370 day images and
4940 night images. When it comes to the training phase of the network, it is
important that what kind of images (day or night) are used to train it, while
night image plumes are illuminated by a night light at the region which
increases the contrast of the images in that region. This helps the plume
cloud boundaries to be more detectable than in day light. Therefore, night and
day images go to different categories in training phase because the network
cannot adapt the color characteristics of the plume clouds in these images
together.

Our dataset for this paper is 1000 images, which have been collected and
annotated based on two classes: ”plume cloud” and ”background” and 60% of the
dataset is considered as training data and 40% remains for testing purposes.
Rows (a) and (b) in Figure 11 shows sample images from the region and their
corresponding ground truth which are generated by ”Labelme” graphical image
annotation tool. we tried to select images of different atmospheric conditions
such as clear daytime, clear nighttime, cloudy, and foggy to represent the
results of different situations in this paper.

Input Ground truth

(a)

(b)

(c)

(d)

(e)

Figure 8: Sample images from our dataset. (a) Clear day time, (b)&(c) cloudy
day, and (d)&(e) clear night time.

###  4.3 Model validation metrics

Our DPRNet performance is evaluated using the metrics of accuracy, recall,
precision and F1 score. Theses metrics are defined using four values of True
Positive (TP), True Negative (TN), False Positive (FP), and False Negative
(FN) obtained from confusion matrix of each introduced method. TP and TN
happen when the model predicts the actual class of a pixel correctly. FP
happens when the model predicts a pixel as in class 1, while the actual class
of the pixel is 0, and FN happens when the model predicts a pixel as in class
0, while the actual class of the pixel is 1. Accuracy is the ratio of
observations which are predicted correctly to the total observations. In our
application, accuracy of the model represents how accurate our model can
recognize the plume cloud pixels. Decision made based on model high accuracy
is acceptable only when the values of FP and FN are almost same, otherwise
other validation metrics should be considered. Foreground pixels coverage of
the sample images shown in Figure 8 confirms this fact that this metric is not
suitable for this study. Recall or sensitivity is the ratio of positive
observations which are predicted correctly to the all actual observations.
Recall shows how many plume cloud pixels are labelled among all the actual
plume cloud pixels. Recall is obtained as follows,

$Recall=\frac{TP}{TP+FN}$ (29)

Precision is the ratio of positive observations which are predicted correctly
to the all observations which are predicted positive. This metrics represents
how many plume cloud pixels actually exist among all the pixels which are
labelled as plume cloud. Therefore, Low rate of FP can achieve a high
precision. This validation metrics is obtained as follows,

$Precision=\frac{TP}{TP+FP}$ (30)

As it is implied from the Equations 29 and 30, precision and recall take
either FP or FN into account. the last validation measure in this paper which
is called F1 score, tries to consider both FP and FN as a weighted average of
recall and precision metrics. Unlike accuracy, this metric is more useful when
FP and FN are not same as in our study. Our FP is less than FN or the amount
of non-actual plume cloud pixels which are predicted as plume cloud pixels are
less than the amount of actual plume cloud pixels which are predicted as non-
plume cloud pixels. Therefore, F1 score helps us look at both recall and
precision validation metrics as follow,

$F1score=\frac{{2}\times{Recall}\times{Precision}}{Recall+Precision}$ (31)

###  4.4 Comparison with Existing smoke recognition methods

In this section, we evaluate the performance of the proposed DPRNet and
compare with its competitors. To choose suitable smoke recognition methods for
comparison, we considered both identification accuracy and computational
complexity of the reviewed methods, which led to selection of DeepLabv3+ [80],
FCN [79], and regular Mask R-CNN. Our proposed DPRNet is evaluated using three
metrics introduced in Section 4.3. The confusion matrix of DPRNet recognition
is given in Table 2 for three specific conditions of day time, night time, and
cloudy & foggy.

Table 2: Average confusion matrix of our DPRNet recognition model.

Plume- Non-plume- Plume- Non-plume- Plume- Non-plume- DT (%) DT (%) NT (%) NT
(%) CF (%) CF (%) Plume-DT1 99.2 0.04 - - - - Non-plume-DT 0.12 0.68 - - - -
Plume-NT2 - - 99.06 0.09 - - Non-plume-NT - - 0.09 0.76 - - Plume-CF3 - - - -
99.09 0.05 Non-plume-CF - - - - 0.13 0.72 1 Day time. 2 Night time. 3 Cloudy
and Foggy.

Also, Table 3 gives the overall results for all models above. Superiority of
the proposed DPRNet is suggested over the competitive recognition models in
this table. Recall metrics shows a large difference between the models, which
shows the effectiveness of the proposed model in recognizing the actual plume
cloud pixels. As we mentioned before, in our study FP and FN are not same and
considering the only recall metrics may not lead to a thorough comparison. It
can be observed that DPRNet achieved better precision compared to the other
methods. Moreover, F1 score guarantees that DPRNet outperformed the other
three methods and shows the effectiveness of our proposed method. Among our
competitive methods, FCN and specially, DeepLabv3 performed better in terms of
all validation metrics. Mask R-CNN performed the worst across all presented
validation metrics. Finally, our DPRNet achieved the best performance among
all existing models in terms of all validation metrics.

Table 3: Comparison of different methods for plume cloud recognition using
average values of the validation metrics.

Model Recall Precision F1 score Mask R-CNN 0.556 0.727 0.607 FCN 0.591 0.859
0.599 DeepLabv3 0.654 0.892 0.721 DPRNet 0.846 0.925 0.881

Beside these average values, the detailed statistics for each model are given
in Figure 9 in terms of each used validation metrics for 90 test images
selected from various day, night, foggy, and cloudy conditions. At a glance,
it is observed that our proposed method shows the most robustness against
facing different test images. Also, Mask R-CNN and FCN are performing the
worst within different images. Although DeepLabv3 is not highly better than
the other two competitive methods, it slight stability shows that this model
can be reliable at some levels.

(a) Recall

(b) Precision

(c) F1 score

Figure 9: Performance of different methods in terms of recall, precision and
F1 score metrics for a number of test images.

To further validate our DPRNet performance, we compared the models over Day,
Night and foggy & cloudy dataset in terms of different validation metrics,
which is given in Figure 10. It can be observed that all methods except Mask
R-CNN, have acceptable performance using day and night dataset. Even in case
night precision, FCN is better than our proposed method. However, this metric
was not able to completely convey the merit of a competitive model. It is
observed that F1 score could compensate this deficiency, as our method’s
recall metric is more than this value for FCN model. It seems that our
proposed DPRNet outperforms the other rival methods by means of recognizing
roughly all of the plume cloud pixels correctly. These high values of recall
help us meet our research application requirement, in which we should identify
the complete stream of the plume cloud for plume rise distance measurement.
The majority of our dataset is related to the cloudy and foggy conditions,
which are frequently seen within image batches. The strength of our DPRNet is
powerful performance against these images, which is of paramount importance in
our application. The proposed DPRNet could improve the recall metric by %66,
%58, and %87 on average over cloudy/foggy conditions relative to FCN,
DeepLabv3, and Mask R-CNN frameworks, respectively. This means that the
proposed method is able to find the plume region appropriately, using
$L^{sse}_{reg}$, which leads to a high quality recognition in images with more
complicated mixture of plume clouds and the sky behind.

(a) Recall

(b) Percision

(c) F1 score

Figure 10: Detailed comparison of methods over three different dataset by
means of recall, precision and F1 score metrics.

To demonstrate the qualitative results of the proposed method, we show some
visual results to compare the competitive method. Figure 11 shows these
recognition results, in which rows (a) and (b) represent the input images and
their corresponding ground truths, respectively, and the other rows give the
results of different models. We tried to consider visualizations of all
dataset in such way that the first two images are related to cloudy/foggy
condition, the second two are from day time night time dataset, and the last
two are obtained from our day time dataset. It is observed that DPRNet
outperformed the other methods by attaining high accuracy of plume cloud
localization and consequently, recognition of the desired smokestack plume
cloud correctly.

(a) (b) (c) (d) (e) (f)

Figure 11: Qualitative results of recognition tasks. (a) Input image, (b)
corresponding ground truth, (c) results of Mask R-CNN, (d) FCN, (e) results of
DeepLabv3, and (f) results of DPRNet.

###  4.5 Plume rise measurement

As we discussed in Section 3.2, $R$ is the most important point in geometric
transformation and consequently, plume rise measurements. In order to
determine the location of $R$ in the images, we can either consider the plume
cloud representative line to be either smooth average centerline, or not. For
the smooth centerline approach, we can define a function to generate a smooth
representative center curve of all points of the detected plume cloud. This
function finds some representative points based on the morphological structure
of the plume cloud and fits them a quadratic function. Otherwise, we can take
the average of $z$ locations for all the binarized points for every column
within plume pixels. The resulting curve will not be relatively smooth, but it
will effectively demonstrate the meandering of the plume cloud. For this
research, we choose the second approach to follow the meandering of the plume
cloud accurately.

In addition, based on the theoretical background information of this research,
we should make sure that the plume cloud has reached its final plume rise in
order to determine the point R. To do this, we need to either take advantages
of Brigg’s plume rise equations as a function of distance, or apply an
asymptotic curve to the plume cloud representative curve to determine the
point R. We leveraged the second option to observe where the plume cloud will
stop rising. Figure 12 shows the asymptotic curve for four plume cloud images
and the automatically selected point R as the point in which the
aforementioned curve reaches neutral buoyancy. Also, the plume rise and plume
rise distance values of each sample plume cloud are given in Table 4, as well
as the averaged hourly measured wind directions at the images sampling times.

(a) Input image

(b) DPRNet result

(c) R selection

Figure 12: Plume cloud looping curve and the asymptotic curve of the plume
cloud, as well as the selected point R. In R selection section, the red curve
represents the meandering of this specific plume cloud. Cyan line shows the
upper boundary points and yellow line shows the lower boundary points of the
plume cloud. Also, Green dashes shows the asymptotic curve and magenta point
represents the point $R$. Table 4: Plume rise and plume rise distance values
of each four plume cloud images, as well as to the averaged hourly measured
wind directions according to Mildred lake monitoring station information.

Image Date (Y-M-D) Time (H-M-S) $\varphi$ (deg.) $\theta$ (deg.) $\Delta{z}$
(m) $X_{max}$ (m) I1 2019-11-08 18-00-13 12.16 -239.8 177 1685 I2 2019-11-09
15-00-13 3.46 -248.5 450.3 3287 I3 2019-11-14 10-00-16 10.41 -241.6 266.8 2280
I4 2019-11-16 11-00-12 10.83 -241.1 300.5 2905

##  5 Conclusion

Automatic objection detection and recognition through Deep learning-based
networks have contributed to applications (e.g., plume rise measurement) in
which photogrammetry is of paramount importance. This paper proposes an
efficient method based on which plume rise can be measured in real-time. This
two-stage straightforward plume cloud border detection and recognition method
outperform its rivals in plume cloud localization and identification. The
proposed method face difficulty in the scene where there are several smock
stakes. Our focus in future studies would be on multi-source plume clouds,
which is inevitable in industrial environments.

##  6 Acknowledgements

We would like to acknowledge Wood Buffalo Environmental Association (WBEA) for
assistance with the camera installation and camera maintenance at the air-
quality monitoring site in Syncrude facility in North Alberta, Canada. The
project is funded by Lassonde School of Engineering Strategic Research
Priority Plan, York University, Canada.

## References

  * [1] G. A. Briggs, “Plume rise predictions,” in Lectures on air pollution and environmental impact analyses, pp. 59–111, Springer, 1982.
  * [2] K. Ashrafi, A. A. Orkomi, and M. S. Motlagh, “Direct effect of atmospheric turbulence on plume rise in a neutral atmosphere,” Atmospheric Pollution Research, vol. 8, no. 4, pp. 640–651, 2017.
  * [3] G. A. Briggs, “Plume rise: A critical survey.,” tech. rep., Air Resources Atmospheric Turbulence and Diffusion Lab., Oak Ridge, Tenn., 1969.
  * [4] G. Briggs, “Plume rise predictions, lectures on air pollution and environment impact analysis,” Am. Meteorol. Soc., Boston, USA, vol. 10, p. 510, 1975\.
  * [5] J. Bieser, A. Aulinger, V. Matthias, M. Quante, and H. D. Van Der Gon, “Vertical emission profiles for europe based on plume rise calculations,” Environmental Pollution, vol. 159, no. 10, pp. 2935–2946, 2011.
  * [6] B. Bringfelt, “Plume rise measurements at industrial chimneys,” Atmospheric Environment (1967), vol. 2, no. 6, pp. 575–598, 1968.
  * [7] P. Makar, W. Gong, J. Milbrandt, C. Hogrefe, Y. Zhang, G. Curci, R. Žabkar, U. Im, A. Balzarini, R. Baró, et al., “Feedbacks between air pollution and weather, part 1: Effects on weather,” Atmospheric Environment, vol. 115, pp. 442–469, 2015.
  * [8] C. Emery, J. Jung, and G. Yarwood, “Implementation of an alternative plume rise methodology in camx,” Novato, CA, 2010.
  * [9] D. Byun, “Science algorithms of the epa models-3 community multiscale air quality (cmaq) modeling system,” EPA/600/R-99/030, 1999.
  * [10] B. E. Rittmann, “Application of two-thirds law to plume rise from industrial-sized sources,” Atmospheric Environment (1967), vol. 16, no. 11, pp. 2575–2579, 1982.
  * [11] W. G. England, L. H. Teuscher, and R. B. Snyder, “A measurement program to determine plume configurations at the beaver gas turbine facility, port westward, oregon,” Journal of the Air Pollution Control Association, vol. 26, no. 10, pp. 986–989, 1976.
  * [12] P. Hamilton, “Paper iii: plume height measurements at northfleet and tilbury power stations,” Atmospheric Environment (1967), vol. 1, no. 4, pp. 379–387, 1967.
  * [13] D. Moore, “A comparison of the trajectories of rising buoyant plumes with theoretical/empirical models,” Atmospheric Environment (1967), vol. 8, no. 5, pp. 441–457, 1974.
  * [14] G. Sharf, M. Peleg, M. Livnat, and M. Luria, “Plume rise measurements from large point sources in israel,” Atmospheric Environment. Part A. General Topics, vol. 27, no. 11, pp. 1657–1663, 1993.
  * [15] H. Webster and D. Thomson, “Validation of a lagrangian model plume rise scheme using the kincaid data set,” Atmospheric Environment, vol. 36, no. 32, pp. 5031–5042, 2002.
  * [16] M. Gordon, S.-M. Li, R. Staebler, A. Darlington, K. Hayden, J. O’Brien, and M. Wolde, “Determining air pollutant emission rates based on mass balance using airborne measurement data over the alberta oil sands operations,” Atmospheric Measurement Techniques, vol. 8, no. 9, pp. 3745–3765, 2015.
  * [17] M. Gordon, P. A. Makar, R. M. Staebler, J. Zhang, A. Akingunola, W. Gong, and S.-M. Li, “A comparison of plume rise algorithms to stack plume measurements in the athabasca oil sands,” Atmospheric Chemistry and Physics, vol. 18, no. 19, pp. 14695–14714, 2018.
  * [18] A. Akingunola, P. A. Makar, J. Zhang, A. Darlington, S.-M. Li, M. Gordon, M. D. Moran, and Q. Zheng, “A chemical transport model study of plume-rise and particle size distribution for the athabasca oil sands,” Atmospheric Chemistry and Physics, vol. 18, no. 12, pp. 8667–8688, 2018.
  * [19] F. Isikdogan, A. C. Bovik, and P. Passalacqua, “Surface water mapping by deep learning,” IEEE journal of selected topics in applied earth observations and remote sensing, vol. 10, no. 11, pp. 4909–4918, 2017.
  * [20] F. Isikdogan, A. Bovik, and P. Passalacqua, “Rivamap: An automated river analysis and mapping engine,” Remote Sensing of Environment, vol. 202, pp. 88–97, 2017.
  * [21] K. Gu, J. Qiao, and W. Lin, “Recurrent air quality predictor based on meteorology-and pollution-related factors,” IEEE Transactions on Industrial Informatics, vol. 14, no. 9, pp. 3946–3955, 2018.
  * [22] K. Gu, J. Qiao, and X. Li, “Highly efficient picture-based prediction of pm2. 5 concentration,” IEEE Transactions on Industrial Electronics, vol. 66, no. 4, pp. 3176–3184, 2018.
  * [23] J. Gubbi, S. Marusic, and M. Palaniswami, “Smoke detection in video using wavelets and support vector machines,” Fire Safety Journal, vol. 44, no. 8, pp. 1110–1115, 2009.
  * [24] F. Yuan, “Video-based smoke detection with histogram sequence of lbp and lbpv pyramids,” Fire safety journal, vol. 46, no. 3, pp. 132–139, 2011.
  * [25] F. Yuan, “A double mapping framework for extraction of shape-invariant features based on multi-scale partitions with adaboost for video smoke detection,” Pattern Recognition, vol. 45, no. 12, pp. 4326–4336, 2012\.
  * [26] F. Yuan, J. Shi, X. Xia, Y. Fang, Z. Fang, and T. Mei, “High-order local ternary patterns with locality preserving projection for smoke detection and image classification,” Information Sciences, vol. 372, pp. 225–240, 2016\.
  * [27] F. Yuan, Z. Fang, S. Wu, Y. Yang, and Y. Fang, “Real-time image smoke detection using staircase searching-based dual threshold adaboost and dynamic analysis,” IET Image Processing, vol. 9, no. 10, pp. 849–856, 2015.
  * [28] F. Yuan, L. Zhang, X. Xia, B. Wan, Q. Huang, and X. Li, “Deep smoke segmentation,” Neurocomputing, vol. 357, pp. 248–260, 2019.
  * [29] S. Khan, K. Muhammad, T. Hussain, J. Del Ser, F. Cuzzolin, S. Bhattacharyya, Z. Akhtar, and V. H. C. de Albuquerque, “Deepsmoke: Deep learning model for smoke detection and segmentation in outdoor environments,” Expert Systems with Applications, vol. 182, p. 115125, 2021.
  * [30] C. Yu, Z. Mei, and X. Zhang, “A real-time video fire flame and smoke detection algorithm,” Procedia Engineering, vol. 62, pp. 891–898, 2013.
  * [31] Y.-k. Shi, Z. Zhong, D.-X. Zhang, and J. Yang, “A study on smoke detection based on multi-feature [j],” Journal of Signal Processing, vol. 31, no. 10, pp. 1336–1341, 2015.
  * [32] F. Garcia-Lamont, J. Cervantes, A. López, and L. Rodriguez, “Segmentation of images by color features: A survey,” Neurocomputing, vol. 292, pp. 1–27, 2018.
  * [33] C. Yuan, Z. Liu, and Y. Zhang, “Learning-based smoke detection for unmanned aerial vehicles applied to forest fire surveillance,” Journal of Intelligent & Robotic Systems, vol. 93, no. 1, pp. 337–349, 2019.
  * [34] A. Garg, S. Nath, P. Nagrath, et al., “Smoke detection in digital frames,” Int. Res. J. Eng. Technol., vol. 5, no. 4, pp. 3843–3846, 2018\.
  * [35] A. Filonenko, D. C. Hernández, and K.-H. Jo, “Fast smoke detection for video surveillance using cuda,” IEEE Transactions on Industrial Informatics, vol. 14, no. 2, pp. 725–733, 2017.
  * [36] A. Wu, C. Du, and M. Li, “Smoke detection method based on mixed gaussian model and wavelet transformation,” Chinese Journal of Scientific Instrument, vol. 29, no. 8, pp. 1622–1626, 2008.
  * [37] R. I. Zen, M. R. Widyanto, G. Kiswanto, G. Dharsono, and Y. S. Nugroho, “Dangerous smoke classification using mathematical model of meaning,” Procedia Engineering, vol. 62, pp. 963–971, 2013.
  * [38] H. Wang and Y. Chen, “A smoke image segmentation algorithm based on rough set and region growing,” Journal of Forest Science, vol. 65, no. 8, pp. 321–329, 2019.
  * [39] W. Jian, K. Wu, Z. Yu, and L. Chen, “Smoke regions extraction based on two steps segmentation and motion detection in early fire,” in MIPPR 2017: Pattern Recognition and Computer Vision, vol. 10609, pp. 281–288, SPIE, 2018\.
  * [40] P. Yuan, X. Hou, and L. Pu, “A smoke recognition method combined dynamic characteristics and color characteristics of large displacement area,” in 2018 2nd IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC), pp. 501–507, IEEE, 2018.
  * [41] W. Zhao, W. Chen, Y. Liu, X. Wang, and Y. Zhou, “A smoke segmentation algorithm based on improved intelligent seeded region growing,” Fire and Materials, vol. 43, no. 6, pp. 725–733, 2019.
  * [42] M. Ajith and M. Martínez-Ramón, “Unsupervised segmentation of fire and smoke from infra-red videos,” IEEE Access, vol. 7, pp. 182381–182394, 2019.
  * [43] K. Dimitropoulos, P. Barmpoutis, and N. Grammalidis, “Higher order linear dynamical systems for smoke detection in video surveillance applications,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 27, no. 5, pp. 1143–1154, 2016.
  * [44] H. Wang, Y. Wang, Q. Zhang, S. Xiang, and C. Pan, “Gated convolutional neural network for semantic segmentation in high-resolution images,” Remote Sensing, vol. 9, no. 5, p. 446, 2017.
  * [45] Y. Li, H. Qi, J. Dai, X. Ji, and Y. Wei, “Fully convolutional instance-aware semantic segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2359–2367, 2017.
  * [46] W. Wang, J. Shen, and L. Shao, “Video salient object detection via fully convolutional networks,” IEEE Transactions on Image Processing, vol. 27, no. 1, pp. 38–49, 2017.
  * [47] H. N. Pham, K. B. Dang, T. V. Nguyen, N. C. Tran, X. Q. Ngo, D. A. Nguyen, T. T. H. Phan, T. T. Nguyen, W. Guo, and H. H. Ngo, “A new deep learning approach based on bilateral semantic segmentation models for sustainable estuarine wetland ecosystem management,” Science of The Total Environment, vol. 838, p. 155826, 2022.
  * [48] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in International Conference on Medical image computing and computer-assisted intervention, pp. 234–241, Springer, 2015\.
  * [49] S. Caelles, K.-K. Maninis, J. Pont-Tuset, L. Leal-Taixé, D. Cremers, and L. Van Gool, “One-shot video object segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 221–230, 2017.
  * [50] Q. Hou, M.-M. Cheng, X. Hu, A. Borji, Z. Tu, and P. H. Torr, “Deeply supervised salient object detection with short connections,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3203–3212, 2017.
  * [51] J. Dai, K. He, Y. Li, S. Ren, and J. Sun, “Instance-sensitive fully convolutional networks,” in European conference on computer vision, pp. 534–549, Springer, 2016.
  * [52] B. Shi, M. Patel, D. Yu, J. Yan, Z. Li, D. Petriw, T. Pruyn, K. Smyth, E. Passeport, R. D. Miller, et al., “Automatic quantification and classification of microplastics in scanning electron micrographs via deep learning,” Science of The Total Environment, vol. 825, p. 153903, 2022\.
  * [53] K. Muhammad, S. Khan, V. Palade, I. Mehmood, and V. H. C. De Albuquerque, “Edge intelligence-assisted smoke detection in foggy surveillance environments,” IEEE Transactions on Industrial Informatics, vol. 16, no. 2, pp. 1067–1075, 2019.
  * [54] Z. Yin, B. Wan, F. Yuan, X. Xia, and J. Shi, “A deep normalization and convolutional neural network for image smoke detection,” Ieee Access, vol. 5, pp. 18429–18438, 2017.
  * [55] Y. Hu and X. Lu, “Real-time video fire smoke detection by utilizing spatial-temporal convnet features,” Multimedia Tools and Applications, vol. 77, no. 22, pp. 29283–29301, 2018.
  * [56] M. Liu, X. Xie, G. Ke, and J. Qiao, “Simple and efficient smoke segmentation based on fully convolutional network,” DEStech Trans. Comput. Sci. Eng.(ica). https://doi. org/10.12783/dtcse/ica2019/30707, 2019.
  * [57] Y. Jia, H. Du, H. Wang, R. Yu, L. Fan, G. Xu, and Q. Zhang, “Automatic early smoke segmentation based on conditional generative adversarial networks,” Optik, vol. 193, p. 162879, 2019.
  * [58] F. Yuan, Z. Dong, L. Zhang, X. Xia, and J. Shi, “Cubic-cross convolutional attention and count prior embedding for smoke segmentation,” Pattern Recognition, vol. 131, p. 108902, 2022.
  * [59] K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask r-cnn,” in Proceedings of the IEEE international conference on computer vision, pp. 2961–2969, 2017.
  * [60] T. Luhmann, S. Robson, S. Kyle, and I. Harley, Close range photogrammetry: principles, techniques and applications, vol. 3.  Whittles publishing Dunbeath, 2006.
  * [61] Z. Zheng, P. Wang, W. Liu, J. Li, R. Ye, and D. Ren, “Distance-iou loss: Faster and better learning for bounding box regression,” in Proceedings of the AAAI conference on artificial intelligence, vol. 34, pp. 12993–13000, 2020.
  * [62] B. Hwang, J. Kim, S. Lee, E. Kim, J. Kim, Y. Jung, and H. Hwang, “Automatic detection and segmentation of thrombi in abdominal aortic aneurysms using a mask region-based convolutional neural network with optimized loss functions,” Sensors, vol. 22, no. 10, p. 3643, 2022.
  * [63] R. Girshick, “Fast r-cnn,” in Proceedings of the IEEE international conference on computer vision, pp. 1440–1448, 2015.
  * [64] A. De Visscher, Air dispersion modeling: foundations and applications.  John Wiley & Sons, 2013.
  * [65] A. J. Cimorelli, S. G. Perry, A. Venkatram, J. C. Weil, R. J. Paine, R. B. Wilson, R. F. Lee, W. D. Peters, and R. W. Brode, “Aermod: A dispersion model for industrial source applications. part i: General model formulation and boundary layer characterization,” Journal of applied meteorology, vol. 44, no. 5, pp. 682–693, 2005.
  * [66] D. B. Turner and R. Schulze, “Atmospheric dispersion modeling: Trinity consultants.”,” 2007.
  * [67] D. H. Hubel and T. N. Wiesel, “Receptive fields of single neurones in the cat’s striate cortex,” The Journal of physiology, vol. 148, no. 3, p. 574, 1959.
  * [68] K. Fukushima and S. Miyake, “Neocognitron: A self-organizing neural network model for a mechanism of visual pattern recognition,” in Competition and cooperation in neural nets, pp. 267–285, Springer, 1982.
  * [69] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.
  * [70] I. Goodfellow, Y. Bengio, and A. Courville, Deep learning.  MIT press, 2016.
  * [71] S. Ji, W. Xu, M. Yang, and K. Yu, “3d convolutional neural networks for human action recognition,” IEEE transactions on pattern analysis and machine intelligence, vol. 35, no. 1, pp. 221–231, 2012.
  * [72] J.-H. Shu, F.-D. Nian, M.-H. Yu, and X. Li, “An improved mask r-cnn model for multiorgan segmentation,” Mathematical Problems in Engineering, vol. 2020, 2020.
  * [73] K. Lin, H. Zhao, J. Lv, C. Li, X. Liu, R. Chen, and R. Zhao, “Face detection and segmentation based on improved mask r-cnn,” Discrete dynamics in nature and society, vol. 2020, 2020.
  * [74] P. Ganesh, K. Volle, T. Burks, and S. Mehta, “Deep orange: Mask r-cnn based orange detection and segmentation,” IFAC-PapersOnLine, vol. 52, no. 30, pp. 70–75, 2019.
  * [75] S. Albawi, T. A. Mohammed, and S. Al-Zawi, “Understanding of a convolutional neural network,” in 2017 international conference on engineering and technology (ICET), pp. 1–6, Ieee, 2017.
  * [76] X. Chen and A. Gupta, “An implementation of faster rcnn with study for region sampling,” arXiv preprint arXiv:1702.02138, 2017.
  * [77] T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2117–2125, 2017\.
  * [78] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778, 2016.
  * [79] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3431–3440, 2015.
  * [80] L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoder-decoder with atrous separable convolution for semantic image segmentation,” in Proceedings of the European conference on computer vision (ECCV), pp. 801–818, 2018.