WGMwgm QEqe EPep PMSpms BECbec DEde

[orcid=0000-0002-6972-5988][1]

[cor]C. author(s<EMAIL_ADDRESS><EMAIL_ADDRESS>
[orcid=0000-0002-9896-8727] [1]

[contrib]Authors contributed equally.

# Computer Vision-based Characterization of Large-scale Jet Flames using a
Synthetic Infrared Image Generation Approach

Carmina Pérez-Guerrero  Jorge Francisco Ciprián-Sánchez  Adriana Palacios
Gilberto Ochoa-Ruiz  Miguel Gonzalez-Mendoza  Vahid Foroughi  Elsa Pastor
Gerardo Rodriguez-Hernandez  Tecnologico de Monterrey, School of Engineering
and Sciences, Jalisco, 45138, Mexico  Universidad de las Américas Puebla,
Department of Chemical, Food and Environmental Engineering, Puebla, 72810,
Mexico  Universitat Politècnica de Catalunya. EEBE, Eduard Maristany 16, 08019
Barcelona. Catalonia, Spain  CIATEQ A.C., Artificial Intelligence Laboratory,
Jalisco, 45131, México.

###### Abstract

Among the different kinds of fire accidents that can occur during industrial
activities that involve hazardous materials, jet fires are one of the lesser-
known types. This is because they are often involved in a process that
generates a sequence of other accidents of greater magnitude, known as domino
effect. Flame impingement usually causes domino effects, and jet fires present
specific features that can significantly increase the probability of this
happening. These features become relevant from a risk analysis perspective,
making their proper characterization a crucial task. Deep Learning approaches
have become extensively used for tasks such as jet fire characterization;
however, these methods are heavily dependent on the amount of data and the
quality of the labels. Data acquisition of jet fires involve expensive
experiments, especially so if infrared imagery is used. Therefore, this paper
proposes the use of Generative Adversarial Networks to produce plausible
infrared images from visible ones, making experiments less expensive and
allowing for other potential applications. The results suggest that it is
possible to realistically replicate the results for experiments carried out
using both visible and infrared cameras. The obtained results are compared
with some previous experiments, and it is shown that similar results were
obtained.

###### keywords:

jet flames computer vision deep learning characterization image generation

##  1 Introduction

The production, handling, and transport of hazardous materials in the chemical
industry have increased during the last decades. This has led to an increase
in the frequency with which serious major accidents occur. The consequences
that these have had on people and the environment have highlighted the need to
improve industrial safety measures. This has been reflected in the
implementation of new legislation such as the Seveso Guideline [21] [22] [23]
on prevention of major accidents in certain industrial activities. Thus, the
development of research studies on the main characteristics, mathematical
modeling, and estimation of consequences of these major accidents, occurring
in the chemical industry around the world is of paramount importance.

Several studies have shown that, after containment losses, one of the most
common major accidents is fire. Among the various types of major fires that
can occur, jet fires are one of the lesser-known types until just recently.
Jet fires are a particular type of serious major fire accident that can occur
both in fixed installations (process plants, storage, or pipe networks) and,
less frequently, during transport (trucks or tank cars). Although jet fires
occur with some frequency, many of them are not reported, since they are often
accompanied by other events of greater magnitude, such as explosions. It is
important to note that jet fires are also capable of generating a sequence of
other accidents, called a domino effect; since their duration and the heat
emitted are high enough to cause it [3]. An example of an incident caused by a
jet fire is the accident that occurred in 2007 at the Valero McKee Refinery in
Sunray, Texas [37]. A leak in the propane deasphalting unit caused a domino
effect that spread quickly, following the collapse of a major pipe rack that
was carrying flammable hydrocarbons. The accident caused four injured, three
of them with serious burns, and the extended shutdown of the refinery.

Jet fires can be evaluated based on specific features that significantly
increase the probability of flame impingement, which in turn can lead to
lethal domino effects. These features become relevant from a risk analysis
perspective, which magnifies the value of their proper characterization. A
great majority of consequence analyses regarding jet fires, that evaluate both
geometric features and radiation loads, employ physically-based models, such
as Computational Fluid Dynamics (CFD) [7], or empirical models, which are
based on empirical correlations developed from a set of experiments. However,
most empirical models use a simplified flame geometry and fixed emissive power
to determine the thermal radiation load on a target. Furthermore, Lattimer et
al. [15] presented a general overview of machine learning techniques for low-
cost and high fidelity fire predictions where it was found that machine
learning has the capacity to provide full-field predictions faster than CFD
simulations by 2 to 3 orders of magnitude.

Deep Learning, a subset of Machine Learning, has become extensively used and
increasingly influential when addressing complex tasks. Convolutional Neural
Networks (CNNs), for example, have shown notable performance in complex
computer vision tasks, such as image recognition, object detection, and
semantic segmentation [16]. A recent stride in the use of CNNs for the
characterization of jet fires is explored in Pérez-Guerrero et al. [30], where
geometric features of the flames are extracted from segmentation masks of
their radiation zones. The models explored, however, are heavily dependent on
the amount of data and the quality of the labels. Data acquisition of jet
fires is done at a high cost since it involves expensive experiments where
both visible and infrared (IR) images of the flames need to be captured, and
then the collection of the jet flame images needs to be paired and jointly
analyzed.

In recent times, Generative Adversarial Networks (GANs) have displayed state-
of-the-art performance for the task of synthetic image generation. There are
examples of the use of this capability for data augmentation tasks in
different domains, such as medical image classification [39], speech
recognition [35], sensor simulation [20], amongst others.

For the domain of synthetic fire image generation, there has been recent work
regarding the use of GANs for the task of visible fire image synthesis [41,
26, 43], wildfire smoke generation [18], and artificial wildfire infrared (IR)
image generation and fusion [6]. In particular, the work by Ciprián-Sánchez et
al. [6] is of special relevance to the present paper as it is, to the best of
our knowledge, the first one to employ GANs to generate artificial IR fire
images from visible ones.

In the context of jet flame risk management, the generation of synthetic IR
images through visible samples has the potential of substantially diminishing
the costs and complexity associated with the development and implementation of
jet flame segmentation and characterization systems.

This paper presents, what is to the best of our knowledge, the first use of
GANs to generate artificial IR jet flame images through visible ones. Taking
as a precedent the work by Ciprián-Sánchez et al. [6], the Pix2Pix GAN model
is implemented. First proposed by Isola et al. [10], this model is one of the
most widely used for image-to-image translation. This model is trained on a
dataset of visible-IR jet flame image pairs and validate its effectiveness for
artificial IR jet flame image generation through four different image quality
and similarity metrics: (i) image entropy (EN) [33], (ii) correlation
coefficient (CC) [40], (iii) peak signal-to-noise ratio (PSNR) [14], and (iv)
the structural similarity index measure (SSIM) [45].

Next, the Deep Learning (DL)-based jet flame segmentation and characterization
system proposed by Pérez-Guerrero et al. [30] is implemented on both real and
artificial IR images. Two different CNNs, namely UNet and Attention UNet, are
used to obtain a segmentation mask of the radiation zones within the flames.
This segmentation is then used to extract the jet fire’s geometrical
information and the results are compared against the real experimental data
obtained form large-scale jet flames. Finally, an evaluation is done on the
errors obtained from artificial and real IR images.

Furthermore, an assessment and evaluation was done regarding the feasibility
and advantages of implementing GAN-based approaches to mitigate the costs and
complexity associated with IR image-based jet flame risk management systems.

The rest of this paper proceeds as follows. Section 2 introduces previous
works on jet flame characterization systems, the role of computer vision for
the improvement of these systems, and introduces relevant works related to
GAN-based fire image synthesis. Section 3 presents the proposed approach for
the image acquisition and preprocessing, and presents the DL pipeline for the
artificial IR image generation and jet flame segmentation and
characterization. Next, in Section 4, the experimental results are presented
and discussed. Finally, Section 5 concludes the paper and outlines potential
avenues for further research.

##  2 Previous work

###  2.1 Risk assessment and modeling

The production, handling, and transport of hazardous materials in the chemical
industry have increased during the last decades. This has led to an increase
in the frequency with which serious major accidents occur. The consequences
that these have had on people and the environment have highlighted the need to
improve industrial safety.

There are certain industrial activities such as the production, handling,
storage, or transportation of hazardous materials, that can be involved in
serious major fire accidents that can threaten the human health, have a high
environmental impact, and can cause important property damage. Therefore, as
part of risk assessment, it is important to identify the optimal space between
equipment and structures by evaluating the size and shape of the flames.
Furthermore, the thermal radiation of fire incidents can be analyzed to
monitor the heat transfer process to nearby objects. Therefore, fast and
reliable tools to obtaining the shape and proportions of a jet fire, as well
as their radiation properties is extremely important from a risk management
point of view [12].

Early risk assessment models are based on algebraic expressions that assume an
idealized flame shape with uniform surface radiation emissive power. Some
examples are the proposal of a cylindrical crone by Croce and Mudan [1], or a
frustum of a cone by Chamberlain [4]. However, the lack of consideration to
the geometry of the fire system on these methods made them unreliable when
dealing with barriers or equipment, and thus several other empirical models
have been explored to evaluate the consequences of the industrial hazards that
fire accidents represent.

For instance, different radiation and turbulence models, along with CFD
simulations, were explored by Kashi and Bahoosh [12] to evaluate how firewalls
or equipment around the flame can affect the radiation distribution and
temperature profile of the jet fire. The Delichatsios’ model was explored by
Guiberti et al. [9] to obtain the flame height for subsonic jet flames at
elevated pressure; however, this model by itself predicts well around 20% of
the flame height in these cases. Mashhadimoslem et al. [19] explored two
Artificial Neural Networks (ANN) to estimate the jet flame lengths and widths
based on the mass flow rates and the nozzle diameters. The two methods
presented negligible discrepancies between them and can be used in place of
CFD methods, which in turn require more computational time and resources.
Finally, Wang et al. [38] implemented an Otsu method to determine the flame’s
geometric features and to estimate the flame extension area based on the flame
intermittency probability. The flame pattern and color are observed to evolve
according to an increase in exit velocity and a decrease in exit-plate
spacing.

Despite the great performance of some of these methods, they heavily depend on
domain expertise. Furthermore, since fire has very dynamic geometry and
shapes, some models grow in uncertainty when dealing with different
conditions. An automatic monitoring system that uses computer vision to
analyze the actual shape and geometry of the flame could add significant
information that could improve e the results of the mentioned models.

###  2.2 The role of computer vision in fire characterization

There has been previous work that explore the use of traditional computer
vision methods to perform fire characterization for risk assessments. To
mention some examples, Janssen and Sepasian [11] presented an automatic flare-
stack-monitoring system that employs computer vision to separate the flare
from the background through temperature thresholding. To enhance the observed
temperature profile of the flare, false colors are added, representing
temperature regions within the flare. Progressing on that, Zhu et al. [46]
proposed an infrared image-based feedback control system that uses an improved
maximum entropy threshold segmentation method to identify and localize the
fire for automatic aiming fire extinguishing and continuous fire tracking.
Alternatively, Gu et al. [8] presented a Computer Vision-based Monitor of
Flare Soot, abbreviated as VMFS, that is mainly composed of three stages that
sequentially apply a tuned color channel to localize the flame, a saliency
detection using K-means to determine the flame area, and finally, use the
center of the flame to search for a potential soot region.

These different traditional computer vision approaches tend to show an
excellent performance when dealing with cases similar to the data that they
were developed on; however, it is necessary to choose which features are
important for each specific problem, so various aspects involve domain
knowledge and long processes of trial and error. Deep Learning architectures,
in contrast, are able to discover the underlying patterns of the images and
automatically detect the most descriptive and salient features with respect to
the problem. This is further proven in the work presented by Pérez-Guerrero et
al. [31], where a comparison was done between traditional computer vision
algorithms and Deep Learning models to evaluate a segmentation approach for
the characterization of fire, based on radiation zones within the jet flames.

In spite of that, Deep Learning architectures are reliant on the size and
quality of the training data. Depending on the task domain, data acquisition
can become difficult and expensive, leading to limited training data sets that
can cause over-fitting and negatively affect the model’s generalization
capacity. Since Deep Learning architectures contain a high number of
parameters with intricate inter relationships, it is considered quite complex
to manually tweak these models, making them a black box in comparison to the
transparency of the traditional computer vision approaches [17]. However this
issue can be tackled through the use of several data augmentation strategies,
that increase the training examples available.

###  2.3 GANs for synthesizing IR imagery for image analysis

The use of GANs for fire imagery generation is a nascent field, with works
such as the ones proposed by Zhikai et al. [43], Park et al. [26], and Zhikai
et al. [41] focusing on the generation of synthetic visible fire images to
augment datasets used mainly for the training of fire detection or
segmentation methods.

Similarly, the work by Mao et al. [18] focuses on using GANs to improve
synthetic wildfire smoke images to a photorealistic level. The authors employ
their proposed approach to perform data augmentation for early wildfire smoke
detection systems.

The work by Ciprián-Sánchez et al. [6] focuses on visible-infrared wildfire
image fusion. The FIRe-GAN model proposed by the authors first generates
artificial IR wildfire images through visible ones, fusing the real visible
and the artificial IR ones to produce a final fused image. This work
introduces the first example of GANs for image-to-image translation of fire
images (in this case, using visible images to generate approximate IR ones).

The ability to produce approximate IR images through visible ones is very
desirable for jet flame risk management. It has the potential to reduce the
costs and complexity of developing and deploying IR-based jet flame
segmentation and characterization systems, as visible cameras are generally
cheaper, more flexible, and widely available.

For the present work, and taking as a precedent the FIRe-GAN model [6], the
Pix2Pix model is implemented, one of the most widely used image translation
architectures, first proposed by Isola et al. [10] to perform visible-to-
infrared image translation of jet flame imagery.

###  2.4 Justification and outline of the contribution

Jet fires, compared to other major fire accidents, have a reduced damage
radius; however, this does not mean that their effects cannot be catastrophic.
Since they originate very high heat fluxes, in case flames reach other
equipment, they can cause a domino effect, triggering a sequence of more
severe accidents. For this reason, it is of great interest to study their
dimensions to establish their effects and consequences to eventually propose
the required safety distances and zones.

The properties of jet fires depend on the composition of the fuel, their
release condition (i.e., gas, liquid or two-phase flow), the fuel release
speed and certain atmospheric conditions, such as the direction and speed of
the wind, among other variables. To study jet fires properly, it is necessary
to know the terms stated previously, while also taking into account their
geometrical features. Some relevant geometric characteristics are the distance
between the fuel release point (i.e., nozzle) and the visible tip of the
flame, and the total flame area. These features are necessary to determine the
probability of thermal radiation, which can be very high at short distances,
and flame impingement on nearby objects.

The creation and evaluation of models for the extraction of the mentioned jet
fire properties would require a considerable amount of data from jet fire
experiments. The composition of the fuel, the release condition and release
speed can be established for these experiments and the atmospheric conditions
can be monitored during them; however the dynamic nature of the resultant fire
shape can be only captured and evaluated in that moment. Visual cameras with
formats such as Video Home System (VHS) are the most common method of fire
video acquisition, however visible light can be altered by the atmospheric
conditions or blocked by some consequences of fire, such as smoke. A more
reliant evaluation of the flame shape would then come from Infrared (IR)
images, however IR cameras are more expensive than visible ones, therefore IR
video surveilance is not always available. Furthermore, additional work would
be needed to pair both visible and IR images for their joint analysis. All of
this adds to the already high cost and laboriousness of fire experimentation,
which is why the research presented in this paper explores a solution for the
generation of IR images from visible ones. These results could aid in
significantly speeding up the experimentation process and eliminating the
dependency on IR camera availability, contributing to future research
regarding different fire incidents.

##  3 Proposed approach

###  3.1 Image acquisition

Vertical large-scale jet fires, released in the open-field, have been analysed
in the present study. The jet fire experiments involve subsonic and sonic jet
flames of propane, discharged through nozzles with a diameter of 12.75 mm, 15
mm, and 43.1 mm. The mass flow rates ranged between 0.03 kg/s to 0.28 kg/s.
The experiments were filmed with two video cameras registering visible light
(VHS) and an infrared thermographic camera (Flir Systems, AGEMA 570). Twenty-
five digital images per second were obtained from the visible spectrum, using
two cameras with resolutions of 384 x 288 pixels and 320 x 240 pixels,
respectively; while four images per second were obtained from the infrared
(IR) camera with a resolution of 640 x 480 pixels. The two visible cameras
were located orthogonally to the flame, and one of them was located next to
the infrared thermographic camera. The average wind speed was 0.4 m/s and the
average ambient temperature was 28°C for all the experimental tests performed.
The experimental set-up is shown by the schema in Fig. 1 Further details of
the experimental set-up can be found in [24].

Figure 1: A schematic view of the experimental set-up for the large-scale jet
fires analysed in the present study.

###  3.2 Dataset

The training dataset used for the Pix2Pix model came from two different
experiments. The ”JFP 05 11” experimental flames were discharged through a
nozzle of 12.75 mm and resulted in 136 pairs of visible and IR images. The
”JFP 05 12” experimental flames were discharged through a nozzle of 43.1 mm
and resulted in 59 pairs of visible and IR images. Both experiments amounted
to 195 paired images were the visible images had a resolution of 384 x 288
pixels, while the IR images had a resolution of 640 x 480 pixels. It was
important that the flames were clearly visible during the experiments.

The dataset used for validation also came from two different experiments. Both
experimental flames were discharged through a nozzle of 15 mm and the main
differences between them were the temperature, the humidity, and the mass flow
rate range. These differences are summarized in Table 1. The experimental data
of the first test covers up to 52 images; while the experimental data for the
second test covers up to 37 images, which resulted in a total of 89 images
obtained from visible light video with a resolution of 320 x 240 pixels.

Table 1: Flame characteristics of the experiments used for the validation dataset. Experiment |  | Ambient
---
Temperature
(K)
| Relative
---
Humidity
(%)
| Mass
---
Flow Rate
(kg/s)
| Total
---
Images
JFP 05 11 | 301.15 | 55 | 0.14 - 0.22 | 52
JFP 05 12 | 298.15 | 50 | 0.24 - 0.28 | 37

###  3.3 Image preprocessing

The frames from both the VHS visible light video and the thermal IR video of
the jet fire experiments needed to be paired for the experiments performed in
the present work. The VHS video was saved in a AVI format, while the infrared
data was saved as mat files that contain the temperature values in a matrix,
using an emissivity value of 0.3 [25]. The mat files were exported as PNG
images and turned into a video sequence with AVI format through a video
editing software. With the two videos in the same format, the focus was then
the different framing between the two videos, which can be observed in Fig. 2.
This was fixed through the use of a video editing software and using the
thermocouples and pipe nozzle as reference for the necessary transformations
based on two arbitrary frames. Finally, the last problem to deal with was the
significant difference of frame rate between the two video. The VHS video has
roughly 30 frames per second, while the IR video has roughly 9 frames per
second.

Figure 2: A visualization of the framing difference between a visible VHS
frame and a thermal IR frame of the same jet fire.

One approach was to track them one by one through a video editing software.
However that process is very time consuming, so a new approach was employed to
recreate the higher frame rate video as a matching lower frame rate video.
This was done by matching the timestamps of the two videos, using a frame and
timestamp form the IR video as reference and comparing it to frames and
timestamps from the VHS video. However, this approach is only feasible when
both videos start and end at the same time, while also having a consistent
frame rate, which was the case for the present data set.

####  3.3.1 Artificial IR image generation

The images were resized to a height of 1024 and a width of 512 pixels to
ensure that the input and output dimensions of the images fed the Pix2Pix
model remain the same. Zero padding is added to the images until the desired
size is reached; this allows to keep the original image dimensions for the
segmentation and characterization steps, in which said padding is removed.

Furthermore, as it is presented in more detail in Section 4, the Pix2Pix model
occasionally generated synthetic IR images that displayed a considerably
higher brightness level when compared to the real ones. So, the Root Mean
Square (RMS) brightness level of the real and artificial IR images was
analyzed, noticing a clear pattern of brighter artificial images. Figure 3
shows the average and per-image RMS brightness values of the real and
synthetic IR images.

Figure 3: RMS brightness analysis and comparison between the real and
synthetic IR images.

Thus, the average brightness level was calculated for both types of images and
established the difference between both averages (for this case, $20$) as a
threshold for a post-processing step to adapt the brightness level of the
generated images. The formula that was defined to adapt the brightness of the
generated IR images is as follows:

$B(I_{a})=\begin{cases}\frac{1.5B(I_{r})}{B(I_{a})},&\textrm{if
$B(I_{a})-B(I_{r})>20$}\\\ B(I_{a}),&\textrm{otherwise,}\end{cases}$ (1)

where $B(I_{a})$ represents the RMS brightness of the artificial image and
$B(I_{r})$ the RMS brightness of the real image.

####  3.3.2 Jet flame segmentation and characterization

The segmentation masks of jet flames are obtained from the implementation of
UNet and Attention UNet architectures for the problem of jet fire radiation
zone segmentation described in Pérez-Guerrero et al. [31]. These architectures
were found to be the best performing ones among eight different methods and
models explored. The original IR images and the generated IR images are both
used as input for each of the models to obtain the segmentation masks of each
set of images.

The jet flame area and the distance between the fuel release point and the
visible tip of the flame, described as the total height, are extracted from
the segmentation masks following the procedure presented in Pérez-Guerrero et
al. [30], which is based on the segmentation mask contour, a reference point
to the outlet of the flame, and a ratio of the pixels per metric of the image.

###  3.4 Deep Learning methods and pipeline description

The first step in the pipeline of the proposed approach is the generation of
the artificial IR jet flame images. To generate the artificial IR jet flame
images, the visible jet flame images were pre-processed and used as input for
the Pix2Pix model. Figure 4 shows the in-detail architecture of the
implementation of the Pix2Pix generator, and Figure 5 of the Pix2Pix
discriminator.

Figure 4: The implementation of the generator of the Pix2Pix model by Isola et
al. [10] for visible-to-infrared jet flame image translation. The numbers
inside the squares represent the number of filters per layer. The color-coded
squares bellow the main figure detail the convolution module structure of the
layers. Figure 5: The implementation of the discriminator of the Pix2Pix model
by Isola et al. [10] for visible-to-infrared jet flame image translation. The
numbers inside the squares represent the number of filters per layer. The
color-coded squares bellow the main figure detail the convolution module
structure of the layers.

This model generates the artificial IR images, which are then cropped and
post-processed as required by the segmentation and characterization models. In
Section 3.3.1 the pre-and post-processing steps for this stage are described.

The second step in the pipeline of the proposed approach is the semantic
segmentation of both, the original and the artificial IR jet flame images. To
generate the segmentation masks of jet flames, the IR images are used as input
for the UNet and Attention UNet segmentation models, which were previously
trained on a dataset of horizontal jet fires, for the segmentation of
radiation zones within the flames, as described in Pérez-Guerrero et al. [31].

The third and final step in the pipeline of the proposed approach is the
extraction of fire geometric characteristics from the obtained segmentation
masks. This process is similar to the one described in Pérez-Guerrero et al.
[30], where the contour of the segmentation mask and a reference point to the
nozzle, that serves as outlet of the flame, are used to obtain the jet flame
area and the total height. An overall visualization of this process can be
observed in Fig. 6.

Figure 6: A visual example of the process done through the pipeline of the
proposed approach.

###  3.5 Evaluation metrics

####  3.5.1 Artificial IR image generation

For the present paper, four widely used image similarity and quality metrics
for the image generation task [5, 40, 32] were employed: image entropy (EN)
[33], correlation coefficient (CC) [40], peak signal-to-noise ratio (PSNR)
[14], and the structural similarity index measure (SSIM) [45].

The EN uses Shannon’s formula [33] to quantify the amount of information per
pixel [32]. It is defined as follows:

$EN=-\sum_{l=1}^{L-1}p_{l}\log_{2}p_{l},$ (2)

where $L$ stands for the gray levels of the image, and $p_{l}$ represents the
proportion of gray-valued pixels $i$ in the total number of pixels [42]. The
larger the EN, the more information is in the generated image.

The CC measures the degree of linear correlation between the generated image
and a source reference one. It is defined as follows:

$CC(X,Y)=\frac{Cov(X,Y)}{\sqrt{Var(X)Var(Y)}},$ (3)

where $Cov(X,Y)$ is the covariance between the generated image and the
reference one, and $Var(X)$, $Var(Y)$ represent the variance of the two images
[42]. The larger the value of CC, the higher the correlation between the
generated and the reference images.

The possible values for the CC are in the range of $[-1,1]$. A value of $-1$
means that the images are inversely correlated, a value of $0$ that there is
no correlation between the images, and a value of $1$ that the images are
perfectly correlated.

The PSNR is the ratio between the maximum signal and the noise obtained from
the mean squared error (MSE) [29]. It assumes that the difference between the
generated image and the reference one is noise. It is defined as follows:

$PSNR=10\log_{10}(\frac{MAX^{2}}{MSE}),$ (4)

where $MAX$ is the maximum value of the image color. An accepted benchmark for
the PSNR is 30 dB; a PSNR value lower than this threshold implies that the
generated image presents significant deterioration [42].

The SSIM is based on the degradation of structural information [45] for the
measurement of the similarity between two images [44]. It is defined as
follows:

$\begin{split}SSIM(X,Y)=(\frac{2u_{x}u_{y}+c_{1}}{u_{x}^{2}+u_{y}^{2}+c_{1}})^{\alpha}*(\frac{2\sigma_{x}\sigma_{y}+c_{2}}{\sigma_{x}^{2}+\sigma_{y}^{2}+c_{2}})^{\beta}*\\\
(\frac{\sigma_{xy}+c_{3}}{\sigma_{x}\sigma_{y}+c_{3}})^{\gamma},\end{split}$
(5)

where $x$ and $y$ are the reference and fused images, respectively; $u_{x}$,
$u_{y}$, $\sigma_{x}^{2}$, $\sigma_{y}^{2}$, and $\sigma_{xy}$ represent the
mean value, variance, and covariance of images $x$ and $y$, respectively.
$c_{1}$, $c_{2}$, and $c_{3}$ are small numbers that help to avoid a division
by zero, and $\alpha$, $\beta$, and $\gamma$ are used to adjust the
proportions [42].

The possible values for the SSIM are in the range $[0,1]$, with $1$ being the
best possible value.

####  3.5.2 Jet Fire Segmentation

Following the research done by Pérez-Guerrero et al. [31], the Hausdorff
Distance ($HD$) is used to evaluate the segmentation results obtained from the
models. This metric is a dissimilarity measure that is commonly used when
boundary delineation is relevant [36], and can be used to evaluate the
resemblance between the two images when superimposed on each other [13]. The
Hausdorff Distance between two finite sections $X$ and $Y$ is defined as:

$HD(X,Y)=max(h(X,Y),h(Y,X)).$ (6)

Where $h(X,Y)$ is the directed Hausdorff distance [36] given by:

$h(X,Y)=max_{x\epsilon X}min_{y\epsilon Y}\left\|x-y\right\|.$ (7)

####  3.5.3 Jet fire characterization

The evaluation of the extracted geometric characteristics is done in a similar
way to Pérez-Guerrero et al. [30], where the resulting measures are compared
against the corresponding experimental data using two different error
measures, the Mean Absolute Percentage Error (MAPE), and the Root Mean Square
Percentage Error (RMSPE).

The MAPE measures the percentage for the average relative differences between
$N$ values from the real valued $t$ and the predicted values obtained from the
segmentation $p$. It does not consider the direction of the error and is used
to measure accuracy for continuous variables [34]. It is defined as:

$MAPE=\frac{1}{N}\sum_{i=1}^{N}\frac{|t_{i}-p_{i}|}{t_{i}}*100.$ (8)

The RMSPE denotes the percentage for the relative quadratic difference between
$N$ values from the real valued $t$ and the predicted values obtained from the
segmentation $p$. It gives a relatively high weight to large errors [34]. It
is defined as:

$RMSPE=\sqrt[]{\frac{1}{N}\sum_{i=1}^{N}\left(\frac{t_{i}-p_{i}}{p_{i}}\right)^{2}}*100.$
(9)

These two metrics can also be used together to evaluate the variation of the
errors. The RMSPE is usually larger or equal to the MAPE, however a greater
difference between them means that there is a greater variance in the
individual errors [34].

###  3.6 Training and validation

####  3.6.1 Artificial IR image generation

For the generation of the artificial IR jet flame images, the main dataset
comprised of 432 images was split into a training set of 389 images and a
validation set of 43 samples. Then rotation, cropping, and mirroring
operations were applied to augment the training set to a total of 6,224
training samples.

The Pix2Pix model was trained, validated and tested on an NVIDIA DGX
workstation using a NVIDIA P100 GPU and the TensorFlow framework with the
following hyperparamters: 100 epochs, a batch size of 4, and a learning rate
of $1\times 10^{-5}$ for the generator and of $1\times 10^{-4}$ for the
discriminator. As discussed in detail in Section 4.1, fine-tuning is not
needed on these hyperparameters.

Finally, the performance and generalization capabilities of the model were
further evaluated on a test set of 37 visible-IR jet flame image pairs.

####  3.6.2 Jet flame segmentation and characterization

The PyTorch framework [28] was used for the implementation of the segmentation
architectures of UNet and Attention UNet. To maintain the weights as small as
possible, a weight decay strategy was used with L2 regularization. The
learning rate had an initial value of 0.0001 and used an ADAM optimizer during
training. The loss function is a Weighted Cross-Entropy loss with weights
computed according to the ENet custom class weighing scheme [27]. The training
was done with a batch size of 4, a learning rate of $1\times 10^{-4}$ with
ADAM optimizer, and up to 5000 epochs with an Early Stopping strategy to avoid
over-fitting. The data was split into 80% for training and 20% for validation
and testing. The training, validation and testing was done on an NVIDIA DGX
workstation using a NVIDIA P100 GPU and further information can be found in
Pérez-Guerrero et al. [31].

The geometrical characteristics were extracted from the segmentation masks
using the Contour Approximation Method of OpenCV [2]. A pixels per metric
calibration variable was established based on the distance between
thermocouples in the visible images. Using the contour area, the topmost
point, and the reference outlet point, the total flame length and area
dimensions were calculated. The resulting measures were then compared against
the corresponding experimental data using the Mean Absolute Percentage Error
(MAPE), and the Root Mean Square Percentage Error (RMSPE).

##  4 Experiments and discussion

###  4.1 Infrared image generation using GANs

Figure 7 shows the results for the EN, CC, PSNR, and SSIM metrics for the
Pix2Pix model on the validation set. It can observed that the artificial IR
images present higher values of EN, which implies more texture on these images
when compared to both the visible and real IR ones. The PSNR median value is
below the 30dB threshold mentioned in Section 3.5, implying that there exists
a large amount of distortion on the synthetic images. However, the CC and SSIM
metrics are high, with medians above the 0.9 threshold, implying a high level
of linear correlation and structural similarity between the source and
generated IR images. Figure 8 shows sample visible, IR, and synthetic IR
images from the validation set. It is relevant to note that, given the good
results obtained with the initial hyperparameters described in Section 3.6.1,
it was not considered necessary to do any further fine-tuning on the model.

Figure 7: Results for the Pix2Pix model on the validation set.

(a) Visible image.

(b) IR image.

(c) Generated IR image.

Figure 8: Sample resulting images of the Pix2Pix model from the validation
set.

Next, Figure 9 shows the results for the EN, CC, PSNR, and SSIM metrics for
the Pix2Pix model on the test set.

As in the validation set, the artificial IR images present higher values of
EN, which implies more texture on these images when compared to both the
visible and real IR ones. The PSNR and CC metrics are the ones that display a
higher decrease in this new test set, implying that the images have more
distortion and present a lower degree of linear correlation when compared to
the results of the validation set. The SSIM metric, in contrast, presents a
very small decrease in performance, still being very close to a perfect score
of 1. The latter implies that the real and synthetic IR images in the test set
still have a significantly high level of structural similarity. Figure 10
shows sample visible, IR and synthetic IR images from the test set.

Figure 9: Results for the Pix2Pix model on the test set.

(a) Visible image.

(b) IR image.

(c) Generated IR image.

Figure 10: Sample resulting images of the Pix2Pix model from the test set.

An overall decrease in the performance across all metrics can be observed,
which is the expected behavior on the new images of the test set. However, the
mentioned decrease in performance is small, which speaks well of the
generalization capabilities of the model for jet flame imagery.

###  4.2 Image segmentation and jet flame characterization

The initial segmentation results without brightness adjustment methods caused
severe artifacts when using generated IR images that showed high values of
brightness. A sample can be observed in Figure 11. The UNet model was more
sensitive to the differences in brightness than the Attention UNet model,
which produced less instances of segmentation masks with artifacts.

(a) UNet segmentation with brightness adjustment methods.

(b) Attention UNet segmentation with brightness adjustment methods.

(c) UNet segmentation without brightness adjustment methods.

(d) Attention UNet segmentation without brightness adjustment methods.

Figure 11: Difference in segmentation with or without brightness adjustment
methods using the UNet and Attention UNet models.

Additionally, the segmentation obtained with Attention UNet produced a lower
mean Hausdorff distance than the segmentation obtained with UNet. The
Hausdorff distance was calculated between the segmentation masks produced from
the real and generated IR images. The overall results are summarized in Table
2.

Table 2: The mean Hausdorff distance between the segmentation obtained from the original IR image and the one obtained from the generated IR image, for each one of the segmentation models. Experiment | JFP 05 11 | JFP 05 12
---|---|---
Model | UNet |  | Attention
---
UNet
UNet |  | Attention
---
UNet
| Hausdorff
---
distance
455.919 | 404.223 | 755.506 | 685.094

####  4.2.1 Flame total height

Table 3 and Figure 12 show the results for the total flame length obtained
from the segmentation masks generated by the UNet and Attention UNet models.
The segmentation was done for both the original and generated IR images. It
can be observed that the generated IR images tend to have higher error values
than the original IR images; however the difference is not too large and the
highest MAPE of 10.741% still allows for a decent representation of the total
flame length. These results are also congruent with the experiments shown in
Pérez-Guerrero et al. [30], where the highest MAPE for height had a value of
11.1%.

UNet, in general, obtained the lowest MAPE when extracting the total flame
length from the original IR images. However, Attention UNet showed the
smallest difference between the total flame length obtained from the original
and generated IR images, which can be due to the sensitivity of UNet to slight
brightness differences of the IR images as previously discussed.

Table 3: The error of the flame total height obtained from different
segmentation models and for each experiment analysed. Total Flame Length
---
Experiment | JFP 05 11 | JFP 05 12
IR Image | Original | Generated | Original | Generated
UNet
MAPE | 4.591% | 10.741% | 2.855% | 4.494%
RMSPE | 5.952% | 12.258% | 4.076% | 5.383%
Attention UNet
MAPE | 5.095% | 7.323% | 3.292% | 4.601%
RMSPE | 6.228% | 8.660% | 4.327% | 5.498%

(a) Results for JFP 05 11

(b) Results for JFP 05 12

Figure 12: Total flame length percentage change between the experimental
values and the results obtained from the segmentation of real IR and generated
IR (GIR) images using the UNet and Attention UNet models.

####  4.2.2 Flame area

Table 4 and Figure 13 show the results for the area obtained from the
segmentation masks generated by the UNet and Attention UNet models. The
segmentation was done for both the original and generated IR images. In a
similar way to the total length, it can be observed that the generated IR
images tend to have higher error values than the original IR images, with the
highest MAPE value being 35.208%. These results, however, are somewhat
congruent with the experiments shown in Pérez-Guerrero et al. [30], where the
highest MAPE for area was a value of 21.9%.

UNet once again obtained the lowest MAPE when extracting the area from the
original IR images, however, the difference from the results of Attention UNet
is minimal. The only case where the area obtained from the generated IR images
did not go beyond a MAPE of 10% was with the ”JFP 05 11” experiment and using
the Attention UNet model, obtaining a value of 7.435%. This may be because the
flames in the ”JFP 05 12” experiment were not as clearly visible as the ones
in the ”JFP 05 11” experiment so certain parts of the fire were not fully
represented in the generated IR image, resulting in high error values for the
area.

Table 4: The error of the flame area obtained from different segmentation
models and for each experiment analysed. Flame Area
---
Experiment | JFP 05 11 | JFP 05 12
IR Image | Original | Generated | Original | Generated
UNet
MAPE | 5.502% | 25.845% | 7.190% | 30.119%
RMSPE | 6.371% | 28.366% | 8.203% | 30.254%
Attention UNet
MAPE | 5.470% | 7.435% | 8.158% | 35.208%
RMSPE | 6.783% | 8.734% | 9.344% | 35.311%

(a) Results for JFP 05 11

(b) Results for JFP 05 12

Figure 13: Flame area percentage change between the experimental values and
the results obtained from the segmentation of real IR and generated IR (GIR)
images using the UNet and Attention UNet models.

###  4.3 Discussion and future work

The characterization of large-scale jet flames through computer vision is a
promising approach following recent strides in Deep Learning-based analysis in
this regard. However, these models are heavily dependent on the amount of data
and the quality of the labels available for training, validation, and testing.
The data acquisition of jet fires is done at a high cost since it involves
expensive experiments. The inclusion of IR imagery has a positive impact on
the thorough analysis of the jet fires’ characteristics; however, this only
adds another layer of complexity and costliness to the experiments due to
expensive IR cameras and the necessity of coupling both visible and infrared
images with cameras that usually have different resolutions and frame rates.

Thus, the generation of artificial IR images from visible ones can
significantly reduce the complexity and cost of the data acquisition of jet
fire images for DL-based characterization. In recent times, GANs have
displayed state-of-the-art performance for different data augmentation and
synthetic image generation applications.

In the present paper, the feasibility of using GANs to generate artificial IR
images was demonstrated, along with their capability to be integrated into the
data acquisition stage of a DL-based jet fire characterization process. A
pipeline for the generation of synthetic IR images from visible ones was
presented, followed by the semantic segmentation of the radiation zone of the
flames and, finally, the characterization of their geometric characteristics.

Using the Pix2Pix model for visible-to-IR jet fire image translation, a decent
approximation of the thermal information provided by real infrared imagery can
be obtained. Said results, coupled with an Attention UNet network, allow us to
attain very similar segmentation and characterization results compared to
experiments with real IR imagery for the characterization of the total length
of the flames.

Nevertheless, the explored GAN-based approach does not perform adequately for
the characterization of the total area of the flames. This limitation is
potentially due to the visibility of the jet flame in the visible images; the
model is unable to synthesize appropriate IR images when the fire is not
clearly visible (e.g., due to environment lights or flame occlusion). This
problem could potentially be overcome in future works by the inclusion of more
diverse and challenging images on the training set for the GAN-based image
generation or through the implementation of few-shot learning strategies.

##  5 Conclusions

The Pix2Pix model occasionally generated synthetic IR images that displayed a
considerably higher brightness level when compared to the real ones. This
difference in brightness created artifacts in the segmentation of the flames,
which the UNet model being more sensitive to this problem than the Attention
UNet model. An analysis of the RMS brightness level between the real and
artificial IR images is recommended to make corrections accordingly.

The results for the total flame length are congruent with the previous work
and represent a decent approximation to the experimental data. Attention UNet
obtained the smallest difference between the results extracted from the
original and generated IR images. This means that the proposed pipeline could
be applied with confidence for the characterization of the flame’s total
height, employing only visual light imagery.

The results for the total flame area, were also somewhat congruent with the
results of the previous work, even when showing a larger difference between
the values obtained from the original and generated IR images. Overall, the
Attention UNet model was observed to obtain more consistent segmentation
results than the UNet model, therefore it would be advised to use the
Attention UNet model in combination with the generated IR images. The results
look promising and the models could be employed with confidence in a smart
sensing pipeline with visible cameras.

## Acknowledgments

The authors wish to thank the AI Hub and the Centro de Innovación de Internet
de las Cosas at Tecnológico de Monterrey for their support for carrying the
training and testing of the computer models explored in this paper with their
NVIDIA DGX workstation.

The authors also wish to thank the Spanish Ministry of Economy and
Competitiveness, and the Institut d’Estudis Catalans for providing the jet
fire data used in the experiments presented in this paper.

This work was supported in part by the SEP-CONACYT ECOS-ANUIES Project 315597.

## References

  * A. Croce and Mudan [1986] A. Croce, P., Mudan, K.S., 1986\.  Calculating impacts for large open hydrocarbon fires.  Fire Safety Journal 11, 99–112.  doi:https://doi.org/10.1016/0379-7112(86)90055-X.
  * Bradski [2000] Bradski, G., 2000.  The OpenCV Library.  Dr. Dobb’s Journal of Software Tools .
  * Casal [2018] Casal, J., 2018.  Chapter 10 - domino effect, in: Evaluation of the Effects and Consequences of Major Accidents in Industrial Plants.. 2nd ed.. Elsevier, pp. 405–437.  doi:https://doi.org/10.1016/B978-0-444-63883-0.00010-1.
  * Chamberlain [1987] Chamberlain, G.A., 1987.  Developments in design methods for predicting thermal radiation from flares.  Chemical Engineering Research and Design 65\.
  * Cheon et al. [2021] Cheon, M., Vigier, T., Krasula, L., Lee, J., Le Callet, P., Lee, J.S., 2021\.  Ambiguity of objective image quality metrics: A new methodology for performance evaluation.  Signal Processing: Image Communication 93, 116150.  URL: https://www.sciencedirect.com/science/article/pii/S0923596521000096, doi:https://doi.org/10.1016/j.image.2021.116150.
  * Ciprián-Sánchez et al. [2021] Ciprián-Sánchez, J.F., Ochoa-Ruiz, G., Gonzalez-Mendoza, M., Rossi, L., 2021\.  Fire-gan: a novel deep learning-based infrared-visible fusion method for wildfire imagery.  Neural Computing and Applications URL: https://doi.org/10.1007/s00521-021-06691-3, doi:10.1007/s00521-021-06691-3.
  * Colella et al. [2020] Colella, F., Ibarreta, A., Hart, R.J., Morrison, T., Watson, H.A., Yen, M., 2020\.  Jet fire consequence analysis.  OTC Offshore Technology Conference doi:10.4043/30802-MS.
  * Gu et al. [2020] Gu, K., Zhang, Y., Qiao, J., 2020.  Vision-based monitoring of flare soot.  IEEE Transactions on Instrumentation and Measurement 69, 7136–7145.  doi:10.1109/TIM.2020.2978921.
  * Guiberti et al. [2020] Guiberti, T., Boyette, W., Roberts, W., 2020.  Height of turbulent non-premixed jet flames at elevated pressure.  Combustion and Flame 220, 407–409.  doi:10.1016/j.combustflame.2020.07.010.
  * Isola et al. [2017] Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A., 2017\.  Image-to-image translation with conditional adversarial networks.  CVPR .
  * Janssen and Sepasian [2018] Janssen, R., Sepasian, N., 2018\.  Automatic flare-stack monitoring.  SPE Production & Operations 34, 18–23.  doi:10.2118/187257-PA.
  * Kashi and Bahoosh [2020] Kashi, E., Bahoosh, M., 2020\.  Jet fire assessment in complex environments using computational fluid dynamics.  Brazilian Journal of Chemical Engineering 37, 203 – 212.  doi:10.1007/s43153-019-00003-y.
  * Klanderman et al. [1993] Klanderman, G., Rucklidge, W., Huttenlocher, D., 1993.  Comparing images using the hausdorff distance.  IEEE Transactions on Pattern Analysis & Machine Intelligence 15, 850–863.  doi:10.1109/34.232073.
  * Korhonen and You [2012] Korhonen, J., You, J., 2012\.  Peak signal-to-noise ratio revisited: Is simple beautiful?, in: 2012 Fourth International Workshop on Quality of Multimedia Experience, pp. 37–38.  doi:10.1109/QoMEX.2012.6263880.
  * Lattimer et al. [2020] Lattimer, B., Hodges, J., Lattimer, A., 2020.  Using machine learning in physics-based simulation of fire.  Fire Safety Journal 114, 102991\.  URL: https://www.sciencedirect.com/science/article/pii/S0379711219303686, doi:https://doi.org/10.1016/j.firesaf.2020.102991.
  * Litjens et al. [2017] Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., van der Laak, J.A., van Ginneken, B., Sánchez, C.I., 2017.  A survey on deep learning in medical image analysis.  Medical Image Analysis 42, 60–88.  doi:10.1016/j.media.2017.07.005.
  * Mahony et al. [2019] Mahony, N.O., Campbell, S., Carvalho, A., Harapanahalli, S., Velasco-Hernández, G.A., Krpalkova, L., Riordan, D., Walsh, J., 2019.  Deep learning vs. traditional computer vision.  Advances in Intelligent Systems and Computing 943, 128–144.  doi:10.1007/978-3-030-17795-9_10.
  * Mao et al. [2021] Mao, J., Zheng, C., Yin, J., Tian, Y., Cui, W., 2021\.  Wildfire smoke classification based on synthetic images and pixel- and feature-level domain adaptation.  Sensors 21.  URL: https://www.mdpi.com/1424-8220/21/23/7785, doi:10.3390/s21237785.
  * Mashhadimoslem et al. [2020] Mashhadimoslem, H., Ghaemi, A., Palacios, A., 2020.  Analysis of deep learning neural network combined with experiments to develop predictive models for a propane vertical jet fire.  Heliyon 6, e05511.  doi:10.1016/j.heliyon.2020.e05511.
  * Milz et al. [2018] Milz, S., Rudiger, T., Suss, S., 2018.  Aerial ganeration: Towards realistic data augmentation using conditional gans, in: Proceedings of the European Conference on Computer Vision (ECCV) Workshops.
  * Official Journal of the European Community [1982] Official Journal of the European Community, 1982\.  EU : Council directive 82/501EEC of 24 june 1982 on the major accident hazards of certain industrial activities.  OJ L 230 05.08.1982, p. 1.
  * Official Journal of the European Community [1996] Official Journal of the European Community, 1996\.  EU : Council directive 96/82/ec of 9 december 1996 on the control of major-accident hazards involving dangerous substances.  OJ L 10 14.1.1997, p. 13-33.
  * Official Journal of the European Community [2003] Official Journal of the European Community, 2003\.  EU : Directive 2003/105/ec of the european parliament and of the council of 16 december 2003 amending council directive 96/82/ec on the control of major-accident hazards involving dangerous substances.  OJ L 345 31.12.2003, p. 97-105.
  * Palacios [2011] Palacios, A., 2011.  Study of Jet Fires Geometry and Radiative Features.  Ph.D. thesis. Universitat Politècnica de Catalunya.
  * Palacios et al. [2012] Palacios, A., Muñoz, M., Darbra, R., Casal, J., 2012\.  Thermal radiation from vertical jet fires.  Fire Safety Journal 51, 93–101.  URL: https://www.sciencedirect.com/science/article/pii/S0379711212000495, doi:https://doi.org/10.1016/j.firesaf.2012.03.006.
  * Park et al. [2020] Park, M., Tran, D.Q., Jung, D., Park, S., 2020.  Wildfire-detection method using densenet and cyclegan data augmentation-based remote camera imagery.  Remote Sensing 12.  URL: https://www.mdpi.com/2072-4292/12/22/3715, doi:10.3390/rs12223715.
  * Paszke et al. [2016] Paszke, A., Chaurasia, A., Kim, S., Culurciello, E., 2016\.  Enet: A deep neural network architecture for real-time semantic segmentation.  arXiv .
  * Paszke et al. [2019] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., Chintala, S., 2019.  Pytorch: An imperative style, high-performance deep learning library, in: Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., Garnett, R. (Eds.), Advances in Neural Information Processing Systems 32. Curran Associates, Inc., pp. 8024–8035.
  * Preedanan et al. [2018] Preedanan, W., Kondo, T., Bunnun, P., Kumazawa, I., 2018\.  A comparative study of image quality assessment, in: 2018 International Workshop on Advanced Image Technology (IWAIT), pp. 1–4.  doi:10.1109/IWAIT.2018.8369657.
  * Pérez-Guerrero et al. [2022] Pérez-Guerrero, C., Palacios, A., Ochoa-Ruiz, G., Mata, C., Casal, J., Gonzalez-Mendoza, M., Falcón-Morales, L.E., 2022.  Experimental large-scale jet flames’ geometrical features extraction for risk management using infrared images and deep learning segmentation methods.  arXiv:2201.07931. Submitted to Journal of Loss Prevention in the Process Industries (January 2022).
  * Pérez-Guerrero et al. [2021] Pérez-Guerrero, C., Palacios, A., Ochoa-Ruiz, G., Mata, C., Gonzalez-Mendoza, M., Falcón-Morales, L.E., 2021.  Comparing machine learning based segmentation models on jet fire radiation zones.  Advances in Computational Intelligence , 161–172doi:10.1007/978-3-030-89817-5_12.
  * Roberts et al. [2008] Roberts, J.W., van Aardt, J.A., Ahmed, F.B., 2008.  Assessment of image fusion procedures using entropy, image quality, and multispectral classification.  Journal of Applied Remote Sensing 2, 1 -- 28.  URL: https://doi.org/10.1117/1.2945910, doi:10.1117/1.2945910.
  * Shannon [1948] Shannon, C.E., 1948.  A mathematical theory of communication.  The Bell System Technical Journal 27, 379--423.  doi:10.1002/j.1538-7305.1948.tb01338.x.
  * Shcherbakov et al. [2013] Shcherbakov, M., Brebels, A., Shcherbakova, N., Tyukov, A., Janovsky, T., Kamaev, V., 2013\.  A survey of forecast error measures.  World Applied Sciences Journal 24, 171--176.  doi:10.5829/idosi.wasj.2013.24.itmies.80032.
  * Sheng et al. [2019] Sheng, P., Yang, Z., Qian, Y., 2019.  Gans for children: A generative data augmentation strategy for children speech recognition, in: 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pp. 129--135.  doi:10.1109/ASRU46091.2019.9003933.
  * Taha and Hanbury [2015] Taha, A.A., Hanbury, A., 2015\.  Metrics for evaluating 3d medical image segmentation: Analysis, selection, and tool.  BMC Medical Imaging 15, 15--29.  doi:10.1186/s12880-015-0068-x.
  * U.S. Chemical Safety and Hazard Investigation Board [2008 [Online].] U.S. Chemical Safety and Hazard Investigation Board, 2008 [Online].  LPG fire at Valero – McKee Rrefinery.  CSB Report No. 2007-05-I-TX.  URL: https://www.csb.gov/file.aspx?DocumentId=5672.
  * Wang et al. [2021] Wang, P., Li, Y., Singh, K.K., Lu, J., Vasconcelos, N., 2021\.  Imagine: Image synthesis by image-guided model inversion, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3681--3690.
  * Wu et al. [2018] Wu, E., Wu, K., Cox, D., Lotter, W., 2018.  Conditional infilling gans for data augmentation in mammogram classification, in: Stoyanov, D., Taylor, Z., Kainz, B., Maicas, G., Beichel, R.R., Martel, A., Maier-Hein, L., Bhatia, K., Vercauteren, T., Oktay, O., Carneiro, G., Bradley, A.P., Nascimento, J., Min, H., Brown, M.S., Jacobs, C., Lassen-Schmidt, B., Mori, K., Petersen, J., San José Estépar, R., Schmidt-Richberg, A., Veiga, C. (Eds.), Image Analysis for Moving Organ, Breast, and Thoracic Images, Springer International Publishing, Cham. pp. 98--106.
  * Xu et al. [2022] Xu, H., Ma, J., Jiang, J., Guo, X., Ling, H., 2022\.  U2fusion: A unified unsupervised image fusion network.  IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 502--518.  doi:10.1109/TPAMI.2020.3012548.
  * Yang et al. [2022] Yang, Z., Wang, T., Bu, L., Ouyang, J., 2022.  Training with augmented data: Gan-based flame-burning image synthesis for fire segmentation in warehouse.  Fire Technology 58, 183--215.  URL: https://doi.org/10.1007/s10694-021-01117-x, doi:10.1007/s10694-021-01117-x.
  * Zhao et al. [2020] Zhao, Y., Fu, G., Wang, H., Zhang, S., 2020.  The fusion of unmatched infrared and visible images based on generative adversarial networks.  Mathematical Problems in Engineering 2020, 3739040.  URL: https://doi.org/10.1155/2020/3739040, doi:10.1155/2020/3739040.
  * Zhikai et al. [2019] Zhikai, Y., Leping, B., Teng, W., Tianrui, Z., Fen, W., 2019.  Fire image generation based on acgan, in: 2019 Chinese Control And Decision Conference (CCDC), pp. 5743--5746.  doi:10.1109/CCDC.2019.8832678.
  * [44] Zhou Wang, Bovik, A.C., Sheikh, H.R., Simoncelli, E.P., .  The ssim index for image quality assessment.  https://www.cns.nyu.edu/ lcv/ssim/.  Accessed: 2020-08-26.
  * Zhou Wang et al. [2004] Zhou Wang, Bovik, A.C., Sheikh, H.R., Simoncelli, E.P., 2004\.  Image quality assessment: from error visibility to structural similarity.  IEEE Transactions on Image Processing 13, 600--612.
  * Zhu et al. [2020] Zhu, J., Li, W., Lin, D., Cheng, H., Zhao, G., 2020\.  Intelligent fire monitor for fire robot based on infrared image feedback control.  Fire Technology 56, 2089–2109.  doi:10.1007/s10694-020-00964-4.