# A Data-Driven Approach for Mitigating Dark Current Noise and Bad Pixels in
Complementary Metal Oxide Semiconductor Cameras for Space-based Telescopes

Peng Jia College of Electronic Information and Optical Engineering, Taiyuan
University of Technology, Taiyuan, 030024, China  Chao Lv  College of
Electronic Information and Optical Engineering, Taiyuan University of
Technology, Taiyuan, 030024, China  Yushan Li  College of Electronic
Information and Optical Engineering, Taiyuan University of Technology,
Taiyuan, 030024, China  Department of Physics, The University of Hong Kong,
Hongkong, 999077, China  Yongyang Sun  College of Electronic Information and
Optical Engineering, Taiyuan University of Technology, Taiyuan, 030024, China
Shu Niu  Origin Space Co., Ltd., Nanjing, 210094, China  Zhuoxiao Wang  Origin
Space Co., Ltd., Nanjing, 210094, China

###### Abstract

In recent years, there has been a gradual increase in the performance of
Complementary Metal Oxide Semiconductor (CMOS) cameras. These cameras have
gained popularity as a viable alternative to charge-coupled device (CCD)
cameras in a wide range of applications. One particular application is the
CMOS camera installed in small space telescopes. However, the limited power
and spatial resources available on satellites present challenges in
maintaining ideal observation conditions, including temperature and radiation
environment. Consequently, images captured by CMOS cameras are susceptible to
issues such as dark current noise and defective pixels. In this paper, we
introduce a data-driven framework for mitigating dark current noise and bad
pixels for CMOS cameras. Our approach involves two key steps: pixel clustering
and function fitting. During pixel clustering step, we identify and group
pixels exhibiting similar dark current noise properties. Subsequently, in the
function fitting step, we formulate functions that capture the relationship
between dark current and temperature, as dictated by the Arrhenius law. Our
framework leverages ground-based test data to establish distinct temperature-
dark current relations for pixels within different clusters. The cluster
results could then be utilized to estimate the dark current noise level and
detect bad pixels from real observational data. To assess the effectiveness of
our approach, we have conducted tests using real observation data obtained
from the Yangwang-1 satellite, equipped with a near-ultraviolet telescope and
an optical telescope. The results show a considerable improvement in the
detection efficiency of space-based telescopes.

Near ultraviolet astronomy (1094) — Gaussian mixture model (1937) — Clustering
(1908) — Observational astronomy (1145)

##  1 Introduction

Detectors play a crucial role in modern astronomical observations. Currently,
there are two primary types of detectors employed for ultraviolet, optical,
and infrared astronomical observations: charge-coupled devices (CCD) and
complementary metal oxide semiconductor (CMOS) detectors. CCD detectors have
been used in astronomical observations for several decades (Mackay, 1986), due
to their large detection area, low noise levels, and uniform photon-electron
response (Fossum & Hondongwa, 2014). However, CCD cameras do have certain
drawbacks, including slow read-out speeds and high costs. On the other hand,
CMOS detectors, commonly found in commercial cameras, present a viable
alternative. However, their application in astronomical observations is
limited by a specific issue. Each pixel in a CMOS camera contains its own
amplifier and A/D conversion circuit (Howell, 2006). Consequently, the dark
current and quantum efficiency exhibit nonuniformity between pixels,
introducing additional noise that can adversely affect the detection
efficiency, photometry, and astrometry accuracy (Mahato et al., 2018).

Due to recent advancements in semiconductor technology, CMOS cameras have been
able to achieve increasingly impressive performance. Certain properties of
CMOS cameras, such as readout noise, quantum efficiency, and dark current
levels, now rival or surpass those of CCD cameras. In addition, CMOS cameras
offer advantages such as Faster readout speeds, lower costs, and reduced power
consumption (Litwiller, 2001), making them the preferred choice for numerous
astronomical applications. Examples include their utilization in lucky imaging
systems (Law et al., 2006) and as detectors in wavefront sensors (Basden,
2015). The installation of CMOS cameras in space-based wide-field telescopes
is also being considered (Karpov et al., 2021). These telescopes are typically
employed for time-domain astronomy that involves the detection of exoplanets,
supernovae, or near-Earth objects. However, the limited space and energy
resources available on satellite platforms pose challenges for CMOS cameras in
maintaining optimal operating conditions. This gives rise to two issues:
1\. Temperature fluctuations can cause variations in the dark current of CMOS
cameras. Due to the independent read-out and A/D conversion circuits in each
pixel, the relationship between temperature and dark current can differ
between pixels.
2.Cosmic radiation can damage pixels in CMOS cameras, leading to the
generation of bad pixels that exhibit noticeable deviations from their
expected performance.

Dark current and bad pixels can introduce false detection results and reduce
the accuracy of photometry and astrometry. Various methods have been proposed
in industries to measure dark current (King et al., 1997; Cassiago et al.,
2000; Guarnieri Calò Carducci et al., 2022; Liu et al., 2022) or model the
imaging process of CMOS cameras using physical models (Konnik & Welsh, 2014).
However, these dark current measurement methods require additional circuits,
which are not practical to implement after the camera has been assembled for
satellites. Additionally, CMOS camera simulators generate simulated images
using a forward model, which cannot replicate the dark current specific to a
particular CMOS camera based on telemetry data. Traditionally, dark current
data can be obtained during real observations, and post-processing methods can
be employed to mitigate the effects caused by dark current or bad pixels
(Rauscher et al., 2011). However, this approach requires additional
observations on orbit to acquire calibration data, resulting in significant
data transfer and observation resource costs.

In this paper, we present a data-driven approach for the CMOS camera to reduce
effects caused by dark current and bad pixels. Our approach includes pixel
clustering step and a dark current - temperature fitting step based on
physical model proposed in Lv et al. (2022). In terms of the physical aspect,
it is expected that the relationship between dark current levels and
temperature should exhibit a consistent pattern for all pixels. Additionally,
the relationship between dark current levels and temperature for the same
pixel in various observations should also adhere to the same trend. Therefore,
we can use data obtained from laboratory tests to build relations between dark
current and temperature. To achieve this, we initially acquire ground-based
test data that simulates space observation conditions. We then categorize
these pixels based on the relation between temperature and dark current level.
Finally, we employ the Arrhenius law to establish the relationship between
dark current levels and temperature (Widenhorn et al., 2002). In real
applications, our approach serves as a tool for processing real observational
data, enabling the estimation of dark current levels and the identification of
bad pixels. Details regarding the telescope and camera employed in this study
will be provided in Section 2. In Section 3, we will introduce the concept of
our approach. Practical applications will be presented in Section 4. Finally,
in Section 5, we will present our conclusions and outline our future work.

##  2 Instruments and Data

In this paper, we utilize our method to process data acquired from Yangwang-1
(IAU Space Telescope No. C59).The Yangwang-1 is a 4U satellite maintained by
Origin Space Co., Ltd. It was launched on June 11th, 2021. The satellite
houses two telescopes: an optical telescope and a near–ultraviolet telescope.
The key parameters of these two telescopes are provided in Table 1 and the
throughput of these telescopes is shown in Figure 1. As indicated in the
table, both telescopes boast a large field of view and a critical spatial
sampling rate. Their primary design focus is on sky survey projects geared
towards detecting near-Earth objects or bright supernovae. Due to the
difference in transmission band design and aperture size, the optical
telescope has a higher throughput compared to the near-ultraviolet telescope.
Given that the dark current of CMOS cameras increases with temperature,
temperature control is necessary to minimize noise and ensure observation
quality. However, in a vacuum environment, natural heat dissipation is
ineffective. Furthermore, due to limited space and power constraints on the
satellite, there is no active cooling or shielding system in place.
Consequently, heat can accumulate during camera operation, causing the
temperature to rise from -20 to 30 degrees. To solve this problem, we set an
upper exposure time limit and limit the camera to capture 100-120 frames after
starting to work to maintain the temperature within an acceptable range.

Table 1: Parameters of visible and near-ultraviolet telescope onboard the Yangwang-1 satellite. Telescope | Optical | Near-ultraviolet
---|---|---
Detector | CMOS Gsense400BSI
Optical System | Refractive Telescope
Observation Band | 420-700nm | 250-280nm
Diameter | 75mm | 90mm
Field Of View | 8.8°×8.8° | 5.7°×5.7°
Focal Length | 146.6mm | 207.7mm
Transmittance | 93.67% | 11.5%
Single Exposure time | 10ms-4800ms
Distortion | $\leq 3\%$
---
Figure 1:  The left panel shows the throughput of the optical telescope and
the right panel shows that of the near-ultraviolet telescope.

Due to temperature fluctuations, the dark current of the camera undergoes
significant changes. This issue is further exacerbated by the small aperture
of telescopes, resulting in limited number of stars with a sufficient signal-
to-noise ratio, particularly in the data obtained by the near-ultraviolet
telescope. Both of these CMOS cameras utilized in the satellite consist of
$2048\times 2048$ pixels with a quantum efficiency of approximately 70% to
80%. More detailed parameters can be found on the official site
111https://www.gpixel.com/en/pro_details_1196.html. Unfortunately, the camera
chip on the satellite does not possess an attached thermometer. Although
several thermometers are installed elsewhere, they fail to accurately reflect
the temperature of the chip. Therefore, we propose obtaining the relationship
between the dark current level and the temperature through laboratory data and
developing a data-driven dark current noise model to estimate the dark current
level. Further discussion regarding the model will be provided in Section 3.

This paper utilizes two different types of data: laboratory test data and real
observation data from the satellite. The laboratory test data are employed to
construct the data-driven model. Due to the tight schedule involved in the
satellite’s manufacturing process, we have had only a few days available to
acquire the test data. Consequently, we have established a chamber in the
laboratory to capture dark current data for the camera. This chamber
effectively blocks light and simulates the observation environment. Within the
chamber, we have placed the CMOS camera and obtained dark frames. During
observations, the CMOS camera continuously captures 100-120 images with an
exposure time of 4 seconds for the near-ultraviolet telescope and an exposure
time of 1 second for the optical telescope. Following this, the camera is
turned off for a period to allow the temperature to decrease. To replicate the
observation strategy, we have conducted the same procedure in the laboratory
to obtain the dark current frames. Each iteration involved acquiring 100-120
frames consecutively, followed by turning off the camera to ensure temperature
reduction. We have repeated this process for five rounds on each camera,
resulting in a total of 5 rounds of testing with 100-120 frames in each round.
It is important to note that, in this paper, the frame sequence number serves
as a proxy for temperature since the temperature increases as soon as image
capture begins. For other different applications, different definitions may be
employed.

The Yangwang-1 satellite has collected real observation data shortly after its
launch, spanning from June 14, to June 28, 2021. In this paper, the data are
obtained correspond to the telescope pointing at $RA=148.88822$ and
$Dec=69.06529$. A total of 1347 images have been obtained and around 120
images were obtained each day. Figure 2 shows both an optical image and a
near-ultraviolet image of the same area of the sky. From the figure, it is
evident that there are significantly fewer stars visible in the near-
ultraviolet band compared to the optical band (white light). This disparity
may be attributed to the narrow bandwidth of the near-ultraviolet filter.
Additionally, the figure reveals that images captured by the near-ultraviolet
telescope are critically sampled, and the quality of celestial object images
is notably affected by dark-current noise and bad pixels.

---
Figure 2:  The same sky area captured by the near-ultraviolet telescope (the
left panel) and the optical telescope (the right panel), when they are
pointing to $RA=148.88822$ and $Dec=69.06529$. Since these two telescopes have
different spatial resolutions, these two images have different field of views
($0.11^{\circ}\times 0.11^{\circ}$ for optical images and $0.17^{\circ}\times
0.17^{\circ}$ for near-ultraviolet images).

##  3 Data-Driven Dark Current Noise Model of the CMOS Camera

The flowchart shown in Figure 3 outlines the procedure for building and
applying the data-driven dark current noise model of the CMOS camera. The
flowchart consists of two primary steps: a pixel clustering step and a
function fitting step. Within the pixel clustering step, pixels are clustered
together based on their temperature-dark current relations. Following that, a
function is fitted for each cluster to establish the temperature (frame
sequence number)-dark current relation. The following subsections will provide
a detailed discussing of these steps.

---
Figure 3:  The schematic of the data-driven dark current noise model of the
CMOS camera encompasses two essential steps: the pixel clustering step and the
function fitting step. Initially, we collect test data following the
observation strategy. Subsequently, we cluster the pixels based on their
temperature (frame sequence number)-dark current relations. Using the
clustering results, we fit functions to each cluster, representing the
temperature (frame sequence number)-dark current relationship specific to the
CMOS camera. In real observations, we start by acquiring the CMOS temperature
(frame sequence number) using the gray scale values of non-celestial object
pixels. This information enables us to determine the dark current for all
pixels in the CMOS camera.

###  3.1 Principle of the Data-Driven Dark Current Noise Model

The data-driven dark current noise model is built considering the specific
requirements of the application and the underlying physical processes. In the
imaging process of the CMOS camera, photons are initially converted into
electrons, which are then converted into digital numbers. However, due to the
thermal movement of electrons, some electrons reach the electrodes randomly,
resulting in dark current noise in the captured images, even if there are no
photons (Magnan, 2003). Theoretically, the dark current is directly related to
the activation energy and the dark current level follows the Arrhenius law
(Widenhorn et al., 2002). The dark current level is defined by factors such as
temperature, exposure time, and the characteristics of the CMOS camera. As the
exposure time or temperature increases, the dark current also tends to
increase. However, since the dark current is generated randomly, the non-
uniformity of the dark current manifests itself as a fixed mode noise with a
random pattern specific to each CMOS camera, which can be observed in long-
exposure images (Pain et al., 2005).

Given that every pixel in the CMOS camera undergoes the same manufacturing
process within a single chip, the dark current level in each pixel is expected
to exhibit a similar trend with temperature, albeit with varying parameters.
Based on this understanding, we put forward the following assumptions for the
data-driven model of the CMOS camera:
When the CMOS camera reaches a specific temperature, its thermal distribution
stabilizes.
The dark current of each pixel in the CMOS camera follows a random
distribution.
The relationship between the average dark current level and temperature can be
approximated using a set of functions.
The average dark current in each pixel will vary continuously as the
temperature on the chips changes continuously.

Based on aforementioned assumptions, we can consider the dark current level at
different temperatures in each pixel as one-dimensional random vectors,
denoted as $X$ (given temperature). Furthermore, in the specific application
we are addressing in this paper, we make the assumption that the temperature
rises as the camera starts to capture images. We also apply a uniform exposure
strategy across our observations, which is why we designate the frame sequence
number, denoted as ’T,’ to represent the temperature. However, in various
other application scenarios, we can adjust our choice of parameters
accordingly.

It may seem plausible to fit the relation between them using individual
functions for each pixel. Nevertheless, the dark current levels exhibit random
variations during each observation, and fitting a function for each pixel
might result in overfitting problems. This could introduce additional errors
in practical applications (Bashir et al., 2020). To address this issue, two
potential solutions emerge: the first involves conducting multiple tests (with
much large number than the number of free parameters in the function) for the
same chip and collecting these data to derive the relationship by fitting gray
scale values from all these measurements, while the second option entails
collecting all pixels displaying similar relations between gray scale values
and frame sequence number and then fitting the relationship between gray scale
values and frame sequence number with measurements from all these pixels. Due
to our limited time for laboratory testing, we opt for the second approach.

We propose clustering the pixels based on their frame sequence number - dark
current relations and then build the relation between the dark current level
and frame sequence number with data from these pixels. By clustering them
according to the dark current and frame sequence number data, we can fit their
relations using a smaller number of functions compared to the total number of
pixels. This approach allows us to achieve effective representations based on
statistical learning theory. Taking into account the aforementioned principle,
we establish the framework discussed in the subsequent sections.

###  3.2 The Pixel Clustering step

We employ the Gaussian Mixture Model (GMM) to classify pixels into different
clusters based on the relationship between dark current and temperature. The
GMM is a widely used clustering method that can model various distributions,
even when they deviate from a strict Gaussian distribution (Rasmussen, 1999;
Xuan et al., 2001; Lee et al., 2012; Zhang et al., 2016; Jia et al., 2018). In
this study, we assume that the dark current vectors
$X=\\{X_{1},X_{2},\cdots,X_{M}\\}$ are a linear combination of $N$ Gaussian
functions. The latent variables are represented by a matrix of size $N\times
M$. Consequently, the conditional probability of a dark current vector $x_{i}$
belonging to class $c_{j}$ follows a Gaussian distribution:

$p(x_{i}=c_{i}|c_{i})=G(x_{i};\mu_{i},\Sigma_{i})=\frac{1}{(2\pi)^{\frac{D}{2}}|\Sigma_{i}|^{\frac{1}{2}}}exp(-\frac{(x_{i}-u_{i})^{T}\sum^{-1}(x_{i}-u_{i})}{2}),$
(1)

where $G(x_{i};\mu_{j},\sum_{j})$ is a standard Gaussian distribution with
$\mu$ as the mean value and $\Sigma$ as the covariance matrix. Thus, the
probability density function of the Gaussian mixture model can be obtained by
summing the combined distribution of all possible states:

$\begin{split}p(X|\lambda)=\sum_{i=1}^{N}p(c_{i})\cdot
p(x_{i}|c_{j})=\sum_{i=1}^{N}Z_{i}G(x_{i};\mu_{i},\Sigma_{i}),\\\
\lambda=\\{Z_{i},\mu_{i},\Sigma_{i}\\},\end{split}$ (2)

where $Z_{i}$ is the weight of a Gaussian function in the mixed model and
$\sum_{i=1}^{N}Z_{i}=1$. Given the GMM and the number of clusters $N$, we
would fit parameters $\lambda$ in the GMM with the training data using the
expectation maximization:
1\. We initialize the parameters of $N$ Gaussian functions, including their
means $\mu_{i}$, variances $\Sigma_{i}$, and weights $Z_{jc}$, with random
numbers.
2\. In the expectation step, we calculate the membership degrees of a dark
current vector $x_{i}$ for each cluster $c$ using equation 3,

$\begin{split}E[Z_{ic}]=\frac{p(x_{i}|\mu_{c},\Sigma_{c})}{\sum_{c=1}^{N}p(x_{i}|\mu_{c},\Sigma_{c})},\\\
p(x_{i}|\mu_{c},\Sigma_{c})=\frac{1}{\sqrt{2\pi}\Sigma_{c}}exp(-\frac{(x_{i}-\mu_{c})^{2}}{2\Sigma_{c}^{2}}),\end{split}$
(3)

where $p(x_{i}|\mu_{c},\Sigma_{c})$ is probability of a pixel belonging to a
cluster. $Z_{ic}$ is a belonging function, which is defined with equation 4:

$Z_{ic}=\left\\{\begin{array}[]{lr}1,\text{ if the dark current - frame
sequence number vector of a pixel i belongs to the Gaussian function $c$}&\\\
0,\text{ else}&\end{array}\right.$ (4)

3\. In the maximization step, we update the parameters of all $M$ Gaussian
functions as well as the weights of each Gaussian function based on all the
dark current-frame sequence number vectors using equation 5:

$\begin{split}\mu_{c}=\frac{\sum_{i=1}^{m}E[Z_{ic}*y_{i}]}{\sum_{i=1}^{m}E[Z_{ic}]},\\\
\Sigma_{c}=\frac{\sum_{i=1}^{m}E[Z_{ic}](X-\mu_{c})(X-\mu_{c})^{T}}{\sum_{i=1}^{m}E[Z_{ic}]},\\\
W_{c}=\frac{\sum_{i=1}^{m}E[Z_{ic}]}{m}.\end{split}$ (5)

4\. We will repeat steps 2 and 3 iteratively until the log-likelihood function
defined in equation 6 reaches convergence, when $F$ does not decrease by 2
iterations.

$F=\sum_{i=1}^{N}log(p(x_{i}|\mu_{c},\Sigma_{c})).$ (6)

Using the aforementioned GMM model, we are able to identify clusters of pixels
that exhibit the same relationship between dark current and frame sequence
number. However, it is important to note that the number of clusters is a
hyperparameter that requires careful selection. To determine the optimal
number of clusters, we employ the Bayesian Information Criterion (BIC) as
evaluation criteria (Burnham & Anderson, 2004). We utilize the elbow method,
as suggested by Thorndike (1953), to find the optimal number of clusters. By
incrementally increasing the number of clusters, we evaluate the BIC defined
in equation 7:

$\begin{split}BIC=c*ln(Num)-2ln(L),\end{split}$ (7)

where $c$ is the number of clusters, $L$ is corresponding likelihood function
and $Num$ is the number of targets in a particular cluster. $L$ is defined in:

$L=-(Num/2)\ast\ln{(2\ast\pi)}-(Num/2)\ast ln{(sse/Num)}-Num/2,$ (8)

where $sse$ represents the discrepancy between the gray scale values and the
prediction results obtained from the functions. As the number of clusters
($c$) increases, the number of samples assigned to each category decreases,
resulting in a decrease in the BIC value. We determine the optimal number of
clusters when the reduction in BIC decreases significantly, as noted in
Bholowalia & Kumar (2014); Syakur et al. (2018). In practical applications, we
explore different numbers of clusters for multiple clustering iterations to
identify the optimal number of clusters. The BIC curve is shown in the left
panel of Figure 4. With the elbow method, we select 10 as the optimal number
of clusters, because the BIC stops decreasing when the number of components is
larger than 10. With the pixel clustering step, we can group pixels into
distinct clusters, where all pixels in the same cluster share a common
relationship between dark current and frame sequence number. The map
illustrating the distribution of different clusters can be seen in the right
panel of Figure 4. As depicted in this figure, our approach effectively
identifies areas with glowing characteristics and adjacent regions that
display similar dark current properties.

---
Figure 4:  The left panel displays the BIC values corresponding to various
clusters. The right panel shows the distribution of pixels across different
clusters.

###  3.3 The Function Fitting Step

While we have established the relationship between dark current and frame
sequence number for each pixel during the pixel clustering step using the GMM
model, it is essential to create a statistical model that effectively
characterizes this relationship for pixels within the same cluster.
Furthermore, this step allows us to pinpoint any defective or anomalous
pixels. To tackle these challenges, we will employ the following procedures to
perform function fitting and detect problematic pixels.
1\. We determine the bias level by analyzing the image captured with the
shortest exposure time of 10 ms, and all subsequent analyses are conducted
using bias-subtracted images.
2\. Mean values ($Mean_{i}$) could be calculated for all pixels belonging to
the same cluster $i$, with one dimension representing $Mean_{i}$ and the other
dimension representing the frame sequence number ($T$).
3\. Following the Arrhenius law, we employ the $\frac{1}{T}$ as input and the
natural logarithm of $Mean_{i}$ as the output for polynomials fitting.
Different polynomials are fitted for pixels assigned to different clusters.
4\. Iterations will be performed to fit these polynomials. In each iteration,
pixels that exhibit significantly large deviations (5 times the mean square
error) would be identified as bad or anomalous pixels. Subsequently, the
polynomials would be fitted using the remaining pixels, and the mean square
error could be calculated.
5\. Bad pixels are promptly removed and excluded from subsequent image
processing. We employ a mask matrix that designates all bad pixels as 0, while
normal pixels assigned to various clusters will be tagged with their
respective cluster indices.

Following the steps outlined, we create an index matrix that assigns labels to
pixels that share the same relation between dark current level and frame
sequence number. Several functions representing these relationships are also
generated. To visually depict the connection between the frame sequence number
and the dark current level, we have plotted these relationships in the left
panel of Figure 5. As depicted in this figure, all 10 clusters exhibit a
consistent trend, highlighting the effectiveness of the selected polynomials.
Additionally, we have identified a bad pixel and displayed its relationship
with the dark current level and the frame sequence number in the right panel
of Figure 5. As shown in this figure, the bad pixel exhibits random
fluctuations until it reaches a frame sequence number of 63. Beyond this
point, at sufficiently high temperatures, the pixel outputs a constant value
of 800.

Figure 5:  The left panel shows the relation between logarithm of dark current
level and 1/ Frame Sequence Number. The right panel shows the relation between
the dark current level and Frame Sequence Number.

To evaluate the effectiveness of our approach, we analyze 100 consecutive
frames of observation images captured by the near-ultraviolet telescope aboard
the Yangwang-1 satellite. Our method, as described earlier, is utilized to
determine the frame sequence number and the dark current level as the Fit
method. For comparative analysis, we also employ two additional methods:
’Match,’ representing the matching method, which selects the dark frame
obtained on the ground close to observed images as the dark current level, and
’Per-fit,’ representing the per-pixel fitting method, which matches each pixel
with a distinct function constructed using ground test data, and the mean
temperature (frame sequence number) is chosen as the index to derive the dark
current level.

The results are illustrated in Figure 6. The top-left panel displays the
estimated frame sequence number alongside the ground truth frame sequence
number of real observation images. As depicted in this figure, both the Fit
method and the Match method exhibit the same trend, wherein the estimated
frame sequence number increases with the ground truth frame sequence number.
Conversely, the Per-fit method fails to effectively capture the variations in
the frame sequence number. Concurrently, the distribution of dark current
levels is presented in the top-right panel of Figure 6. Notably, the
difference between the dark current levels for the same cluster (obtained
through the data-driven approach) is minimal, to the extent that they are
scarcely discernible, even on a logarithmic scale. To highlight the
distinctions between these methods, we have plotted the residual root mean
square (RMS) of grayscale values in the bottom panel of Figure 6. As observed
in this figure, both the Fit method and the Match method successfully reduce
the residual RMS, and our Fit method reaches a lower RMS value.

---
Figure 6:  The top left panel shows the three methods(’Fit’, ’Match’,’Per
fit’) for predicting the true frame rate of the image. The top-right panel
visualizes background levels of individual pixels within a single observation
image for each cluster. The bottom panel shows the RMS comparison with the
original image after processing by both methods ’Fit’ and ’Match’.

In real applications, the data-driven approach includes both the index matrix
and the functions that reflect the dark-current and temperature relations.
When we use the data-driven approach to process real observation images, we
can directly select a subset of pixels that do not capture photons from
celestial objects and utilize their gray-scale values to estimate the camera’s
temperature. Then we can estimate the dark current of the whole detector.
Further details on the application of these models will be discussed in
Section 4.

##  4 Application of the Data-Driven Dark Current Noise Model

###  4.1 Data Processing with the Data-Driven Dark Current Noise Model

For the YangWang Satellite, target detection constitutes the crucial first
step in extracting scientific findings from observational data. However, the
design of the satellite and observation mode result in dark current being the
primary source of noise in both the near-ultraviolet and optical telescope
images, as opposed to sky background or stray light. Therefore, accurately
accounting for the dark current’s contribution is essential. Following initial
bias removal, we utilize the source detection algorithm SExtractor (with a
connected area of 5 pixels and a 3 sigma detection threshold) to identify
potential celestial object candidates. Subsequently, we employ a data-driven
model to estimate the dark current using the remaining pixels. By utilizing
around 10,000 pixels and applying the least squares method to fit the
polynomials, we can determine the frame sequence number of the CMOS camera.
Based on the frame sequence number, we could obtain the dark current frame for
the CMOS camera with the data-driven model. Finally, through the subtraction
of the dark current from the original image and masking of bad pixels (around
50000 pixels), we obtain processed images. Figure 7 displays the images
captured by the near-ultraviolet telescope and the optical telescope, as well
as the processed images. This figure demonstrate that our method successfully
eliminates bad pixels and reduces the influence of dark current, indicating
its potential to enhance the detection limit of the telescope.

---
Figure 7:  The left panel displays the original images and the right panel
displays the processed images. The top panel shows images obtained by the near
ultra-violet telescope, while the bottom panel shows images obtained by the
optical telescope.

With processed images, we analyze the detection efficiency and the detection
accuracy with catalog from Gaia DR2 for the optical telescope, and with
catalog from the GALEX for the near-ultraviolet telescope. Given the
relatively small apertures of the telescopes used in this study, their
detection capabilities are somewhat limited. We expect that all stars detected
by these telescopes should also be within the Gaia catalog with apparent
magnitudes below 19 and all the GALEX catalog. Therefore, we define the
detection efficiency as the percentage of stars that our algorithm
successfully identifies. Conversely, stars detected by our source detection
algorithm but not listed in Gaia DR2 or the GALEX catalog are considered false
detection results, which are used to calculate the accuracy rate. It is
important to note that the definition employed here may lead to the omission
of a few targets missed by the Gaia, however these instances are expected to
be rare. It should be mentioned that the detection efficiency and the
detection accuracy are differential quantity, which are defined for stars with
specific brightness. Therefore, we evaluate these two values by stars with
each apparent magnitude as a provisional measure in this paper. Considering
the improved image quality, we can expect higher detection efficiency rates
and higher accuracy rates in processed images using the same source detection
algorithm. In the subsequent subsections, we will present additional evidence
of the enhancements achieved by our method when employing the SExtractor and a
Faster RCNN based method for the detection of celestial objects.

###  4.2 Target Detection Results With The SExtractor

In this subsection, we employ the SExtractor as the source detection algorithm
to identify celestial objects in both the original and images processed with
our approach. The SExtractor is a widely recognized algorithm for source
detection, well known for its reliable performance with various observational
data sources (Bertin & Arnouts, 1996). We use the same parameters for the
SExtractor to process the original and processed images. Our method is applied
to both the original near-ultraviolet and optical images. A portion of the
original and processed images is displayed in Figure 8. In the figure, the
white boxes represent detected targets, while the red boxes indicate true
targets. As depicted, the SExtractor effectively identifies bright targets in
both sets of images, but the presence of noise introduces some false detection
results. After we process these images, we could obtain detection results with
much lower false detection rates.

---
Figure 8:  Part of images in optical (top panel) and near-ultraviolet band
(bottom panel) before and after being processed with our method. White boxes
with solid lines indicate positions of true celestial objects that are above
the detection limit. Red boxes with dashed lines indicate positions of
candidates detected by the SExtractor. As shown in this figure, the SExtractor
could better detect celestial objects after being processed by our method
(with much lower false detection rate).

To further evaluate the performance of our algorithm, we conduct tests using
an image obtained by the optical telescope. We use the SExtractor to detect
targets from both the original and processed versions of the image.
Subsequently, we set the limiting magnitude as $m=19$ and perform a cross-
match with the Gaia DR2 catalog to analyze the detected results, as shown in
the top panel of Figure 10. The figure demonstrates significant improvements
in both detection accuracy and efficiency achieved by the ’Fit’ and ’Match’
methods compared to baseline approaches. Notably, our ’Fit’ method exhibits
the largest increase in detection accuracy for stars across a broad range of
brightness levels, encompassing both bright and dim stars.

###  4.3 Target Detection Results With The Faster RCNN Based Source Detection
Algorithm

The Faster RCNN based celestial object detection algorithm, introduced by Jia
et al. (2020), is designed specifically for detecting small and sparsely
distributed celestial objects. The algorithm demonstrates superior detection
capability for images of a specific type when provided with sufficient
training data. For a comprehensive understanding of the method, see Jia et al.
(2020) and Sun et al. (2023). Here, we provide a brief overview of the
approach. As illustrated in Figure 9, the algorithm has the following steps:
Feature extraction: We employ a convolutional neural network (CNN) to
construct a Feature Pyramid Network (FPN) that extracts features from various
scales, facilitating subsequent detection. These feature maps are shared by
both the Region Proposal Network (RPN) and the classification and regression
neural network.
Region Proposal Networks (RPN): RPNs generate region proposals by processing
feature maps. They identify regions in the images that are likely to belong to
the foreground or background.
ROI Pooling: ROI Pooling can be applied to resize the proposed region to a
normalized size, and then connected to a fully connected layer for prediction
and regression.

---
Figure 9:  The structure of the Faster RCNN detection algorithm used in this
paper.
---
Figure 10:  The top and middle panels illustrate the detection accuracy and
efficiency of both the SExtractor and Faster RCNN for stars with varying
apparent magnitudes in the 47th. frame of images. The bottom panel shows how
the detection accuracy and efficiency decrease with increasing temperature
(frame sequence number).

We have trained two Faster RCNN models separately using original images and
processed images from the optical telescope. A total of 200 frames of each
category have been used as training data and positions of stars in these
images are obtained by the Gaia DR2 catalog as labels. The Faster RCNN models
have been trained for 30 epochs using the Adam optimizer with an initial
learning rate of 0.001. Training the neural networks has taken 2-3 hours in a
computer with one RTX 3090 GPU. Once trained, we have applied these neural
networks to process images in the test dataset consisting of 50 frames of
continuously observed images obtained by the optical telescope. For
comparison, we have plotted the detection accuracy and detection efficiency
for stars with different apparent magnitude in the middle panel of Figure 10.

In the top panel of Figure 10, significant performance enhancements are
observed when employing the method to reduce the dark current noise, when the
SExtractor is used as the detection algorithm. The same results could be
observed in the middle panel of Figure 10, when we use the Faster RCNN as the
detection algorithm. For both the classical and the deep learning based source
detection algorithm, the detection capability has notably increased by a
minimum of 50% for stars with a apparent magnitude exceeding 12, at a cost of
a slightly reduce in detection accuracy. As discussed earlier, the rise in
camera temperature during observations elevates the significance of dark
current levels, particularly in images captured at higher frame sequence
numbers. The bottom panel of the Figure 10 displays the detection accuracy and
efficiency for images with varying frame sequence numbers. As depicted, the
detection accuracy experiences a rapid decline for frame numbers exceeding 30,
while the detection efficiency decreases steadily across different frame
sequence numbers. As evident, dim stars suffer more significantly from the
dark current issue, leading to reduced detection accuracy and efficiency.
Nevertheless, our method effectively mitigates this problem.

##  5 CONCLUSIONS AND FUTURE WORK

In recent years, there has been a growing interest in launching small
satellites for astronomical observations. To optimize costs, many of these
satellites are opting for commercial cameras without dedicated cooling or
shielding systems due to space and power limitations. In addition, these
satellites aim to minimise additional observation and data transfer
requirements. Addressing these needs, we propose an innovative approach to
construct a data driven model of the CMOS camera, mitigating the effects of
dark current and bad pixels. This method only requires some additional ground-
based tests and data modeling processes. We validate our method using real
observation data obtained from the near-ultraviolet and optical telescope
onboard the Yangwang-1 satellite. The results demonstrate that our approach
effectively reduces dark current noise and identifies bad pixels. Moreover,
when combined with classical and deep neural network based target detection
algorithms, our method enhances detection efficiency. This indicates that our
approach can serve as an effective tool for CMOS cameras installed in future
space-based satellites.

There are several aspects that require further investigation for our method.
First, due to our tight time schedule, the data acquisition process is
designed based only on experience. To enhance our approach, it would be
beneficial to construct a finite element model of the camera and the telescope
and develop an agent that can autonomously determine the optimal data
acquisition strategy. Second, as the Yangwang-1 primarily experiences a dark
current as the dominant noise source, we do not address the sky background or
stray light in this paper, which could potentially impact the photometry
accuracy. In the future, we plan to develop new techniques to separate and
account for different sources of noise, while also expanding the noise
modeling procedure for other space telescopes, such as cameras in the China
Space Station Telescope (CSST) 222https://csst-
tb.bao.ac.cn/code/csst_sim/csst-simulation.

## Acknowledgements

Authors would like to thank the reviewer for his/her kindly suggestions, which
greatly improve the quality of this paper. The code used in this article can
shared in the PaperData repository powered by China-VO with DOI of
10.12149/101386. This work is supported by the National Natural Science
Foundation of China (NSFC) with funding numbers 12173027, 12303105 and
12173062, National Key Research and Development Program with funding numbers
2023YFF0725300. We acknowledge the science research grants from the China
Manned Space Project with NO. CMS-CSST-2021-A01.

## References

  * Basden (2015) Basden, A. G. 2015, Journal of Astronomical Telescopes, Instruments, and Systems, 1, 039002
  * Bashir et al. (2020) Bashir, D., Montañez, G. D., Sehra, S., Segura, P. S., & Lauw, J. 2020, in AI 2020: Advances in Artificial Intelligence: 33rd Australasian Joint Conference, AI 2020, Canberra, ACT, Australia, November 29–30, 2020, Proceedings 33, Springer, 347–358
  * Bertin & Arnouts (1996) Bertin, E., & Arnouts, S. 1996, Astronomy and astrophysics supplement series, 117, 393
  * Bholowalia & Kumar (2014) Bholowalia, P., & Kumar, A. 2014, International Journal of Computer Applications, 105
  * Burnham & Anderson (2004) Burnham, K. P., & Anderson, D. R. 2004, Sociological methods & research, 33, 261
  * Cassiago et al. (2000) Cassiago, C., La Paglia, G., & Pogliano, U. 2000, IEEE Transactions on Instrumentation and Measurement, 49, 1206
  * Fossum & Hondongwa (2014) Fossum, E. R., & Hondongwa, D. B. 2014, IEEE Journal of the electron devices society
  * Guarnieri Calò Carducci et al. (2022) Guarnieri Calò Carducci, C., Pau, M., Cazal, C., Ponci, F., & Monti, A. 2022, Sensors, 22, 5074
  * Howell (2006) Howell, S. B. 2006, Handbook of CCD astronomy, Vol. 5 (Cambridge University Press)
  * Jia et al. (2020) Jia, P., Liu, Q., & Sun, Y. 2020, The Astronomical Journal, 159, 212
  * Jia et al. (2018) Jia, P., Osborn, J., Kong, L., et al. 2018, MNRAS, 480, 2466, doi: 10.1093/mnras/sty1951
  * Karpov et al. (2021) Karpov, S., Christov, A., Bajat, A., Cunniffe, R., & Prouza, M. 2021, in Revista Mexicana de Astronomia y Astrofisica Conference Series, Vol. 53, 190–197
  * King et al. (1997) King, D., Hansen, B., Kratochvil, J., & Quintana, M. 1997, in Conference Record of the Twenty Sixth IEEE Photovoltaic Specialists Conference-1997, IEEE, 1125–1128
  * Konnik & Welsh (2014) Konnik, M., & Welsh, J. 2014, arXiv preprint arXiv:1412.4031
  * Law et al. (2006) Law, N. M., Mackay, C. D., & Baldwin, J. E. 2006, Astronomy & Astrophysics, 446, 739
  * Lee et al. (2012) Lee, K. J., Guillemot, L., Yue, Y. L., Kramer, M., & Champion, D. J. 2012, MNRAS, 424, 2832, doi: 10.1111/j.1365-2966.2012.21413.x
  * Litwiller (2001) Litwiller, D. 2001, Photonics spectra, 35, 154
  * Liu et al. (2022) Liu, Z., Chu, C., Wang, B., et al. 2022, IEEE Transactions on Electron Devices
  * Lv et al. (2022) Lv, C., Jia, P., Li, Y., Wang, Z., & Su, M. 2022, in X-Ray, Optical, and Infrared Detectors for Astronomy X, Vol. 12191, SPIE, 538–542
  * Mackay (1986) Mackay, C. D. 1986, Annual review of astronomy and astrophysics, 24, 255
  * Magnan (2003) Magnan, P. 2003, Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 504, 199
  * Mahato et al. (2018) Mahato, S. B., De Ridder, J., Meynants, G., Raskin, G., & Van Winckel, H. 2018, IEEE Sensors Journal, 18, 2722
  * Pain et al. (2005) Pain, B., Cunningham, T., Hancock, B., Wrigley, C., & Sun, C. 2005, in IEEE Workshop on CCD’s and Advanced Image Sensors, Karuizawa, Nagano, Japan
  * Rasmussen (1999) Rasmussen, C. 1999, Advances in neural information processing systems, 12
  * Rauscher et al. (2011) Rauscher, B. J., Lindler, D. J., Mott, D. B., et al. 2011, Publications of the Astronomical Society of the Pacific, 123, 953
  * Sun et al. (2023) Sun, R., Jia, P., Sun, Y., et al. 2023, The Astronomical Journal, 166, 235
  * Syakur et al. (2018) Syakur, M., Khotimah, B., Rochman, E., & Satoto, B. D. 2018in , IOP Publishing, 012017
  * Thorndike (1953) Thorndike, R. L. 1953, in Psychometrika, Citeseer
  * Widenhorn et al. (2002) Widenhorn, R., Blouke, M. M., Weber, A., Rest, A., & Bodegom, E. 2002, in Sensors and Camera Systems for Scientific, Industrial, and Digital Photography Applications III, Vol. 4669, SPIE, 193–201
  * Xuan et al. (2001) Xuan, G., Zhang, W., & Chai, P. 2001, in Proceedings 2001 international conference on image processing (Cat. No. 01CH37205), Vol. 1, IEEE, 145–148
  * Zhang et al. (2016) Zhang, Z.-B., Yang, E.-B., Choi, C.-S., & Chang, H.-Y. 2016, MNRAS, 462, 3243, doi: 10.1093/mnras/stw1835


# A Deep Learning-based Detector for Brown Spot Disease in Passion Fruit Plant
Leaves

Andrew Katumba1, Moses Bomera1, Cosmas Mwikirize1, Gorret Namulondo1, Mary
Gorret Ajero2,
Idd Ramathani2, Olivia Nakayima1, Grace Nakabonge1, Dorothy Okello1 and
Jonathan Serugunda1  1Department of Electrical and Computer Engineering
Makerere University, Kampala, Uganda
Corresponding Author Email<EMAIL_ADDRESS>2National Crops
Resources Research Institute (NaCRRI), Kampala, Uganda

##  I Introduction

Pests and diseases pose a key challenge to passion fruit farmers across Uganda
and East Africa in general. They lead to loss of investment as yields reduce
and losses increases. As the majority of the farmers including passion fruit
farmers, in the country are smallholder farmers from low-income households,
they do not have sufficient information and means to combat these challenges.
While, passion fruits have the potential to improve the well-being of these
farmers given their short maturity period and high market value [1], without
the required knowledge about the health of their crops, farmers can not
intervene promptly to turn the situation around.

For this work, we partnered with the Uganda National Crop Research Institute
(NaCRRI) to develop a dataset of expertly labeled passion fruit plant leaves
and fruits, both diseased and healthy. We made use of their extension service
to collect images from five districts in Uganda to create the dataset. Using
the dataset, we are applying state-of-the-art techniques in machine learning,
specifically deep learning at scale for object detection and classification
for accurate plant health status prediction.

While deep learning techniques have been applied to various disease diagnosis
contexts with varying degrees of success([2, 3, 4, 5, 6]), there has not been
any significant effort, to the best of our knowledge, to create a dataset or
apply machine learning techniques to passion fruits despite their obvious
financial benefits.

With this work, we hope to fill this gap by generating and making publically
available an image dataset focusing on passion fruit plant diseases and pest
damage and training the first generation of machine learning-based models for
passion fruit plant disease identification using this dataset. The initial
focus is on the locally prevalent woodiness (viral) and brown spot (fungal)
diseases.

##  II Dataset

As mentioned in the previous section, one of the objectives of this work is to
compile a high-quality dataset of passion fruit plant fruits and leaves,
starting with brown spot and woodiness diseases, on which various machine
learning and computer vision tasks can be evaluated. The dataset was obtained
in field visits to the districts of Masaka, Buikwe, Jinja, Kayunga, and Mbale,
in order to capture sufficient variation in plant features across the
different regions of the country. We expect, in the short term, to grow the
dataset to over 30,000 images and make it available to the research community
as an open dataset to drive further research on passion fruit diseases. For
the initial modeling using this dataset, a subset of 2,000 images of brown
spot afflicted leaves were used with a focus on the local purple passion fruit
variety, as it is the predominant variety in the country. .

##  III Methodology

From the general dataset, 2000 images symptomatic of the brown spot disease
were randomly selected for use in the training phase. The images were cropped
to 400x400 pixels (to allow for larger batch sizes and quick iteration during
training) and then uploaded for labeling through a convenient interface
provided by the Labelbox [7] platform. The labels were then exported from the
platform in JSON format for use with the model training.

As this work was structured as an object detection task i.e. identifying brown
spot patches on leaves, we made use of Convolutional Neural Networks (CNNs) as
they are suitable for this computer vision task. The labels were therefore
instances of bounding boxes indicating regions of the leaf exhibiting symptoms
of brown spot disease.

##  IV Experimental Setup and Results

In this section, we present results for object detection with the EfficientDet
[8] model. In particular, we focused on the D3 variant of the model as it has
shown competitive results on other object detection tasks. It has been
demonstrated that EfficientDet models can reach high performance as shown on
the COCO dataset [8] and in some cases can be an order of magnitude faster to
train than other object detection models.

In parallel, we are evaluating other model classes, including YOLO[10] and
SSD[9] (not discussed here), to feed into a comparative study. Given the small
size of the dataset, we made use of transfer learning [11], with the model
pre-trained on the COCO dataset [12]. The Tensorflow[14] machine learning
framework was used for both training and inference.

The experiments were run on the Microsoft Azure Cloud on an NC6 virtual
machine with 6 cores, 56 GB of RAM and 1 half an NVIDIA K80 GPU card. 90% of
the images were used to train and validate the model, while the remaining 10%
were used to test the model.

We achieve a mean average precision of 0.351 on the test set after training
for 10,000 iterations. An illustration of the model output for an out-of-
sample example is shown in Fig. 1, for which the model is able to capture all
patches of brown spot disease on the leaf. Inference times are 0.06s for GPU
and 0.63s for CPU.

(a) Groud truth

(b) Model output

Figure 1: Illustration of EfficientDet D3 model test output: a) ground truth
labels and b) model prediction. The model is able to capture all instances of
brown spot disease in the image.

##  V Conclusion

Deep Learning techniques, in particular, computer vision based object
detectors can be applied to capture expert knowledge for diagnosis of plant
diseases. In this work, we have presented an object detection model trained on
a novel dataset of passion fruit plant leaves targeting the identification and
location of regions on leaves affected by the brown spot disease. Future work
includes training on a larger dataset to improve the model performance and
integration of the most robust model onto an edge device for use in the field
by farmers and outreach workers.

## Acknowledgment

This work is being supported by a Makerere University Research Innovation Fund
Grant (2019 - 2020), an NVIDIA Accelerated Data Science GPU Seed Grant and a
Microsoft AI for Earth Grant (2020 - 2021). The labeling process is supported
by a Labelbox Education Grant.

## References

  * [1] Biosciences eastern and central Africa, “Demystifying Passion Fruit Woodiness Disease Title,” 2020. [Online]. Available: https://hub.africabiosciences.org/blog/demystifying-passion-fruit-woodiness-disease/. [Accessed: 10-Jul-2020].
  * [2] V. Singh and A. K. Misra, “Detection of plant leaf diseases using image segmentation and soft computing techniques,” Inf. Process. Agric., vol. 4, no. 1, pp. 41–49, 2017.
  * [3] A. Ramcharan, K. Baranowski, P. McCloskey, B. Ahmed, J. Legg, and D. P. Hughes, “Deep learning for image-based cassava disease detection,” Front. Plant Sci., vol. 8, p. 1852, 2017.
  * [4] A. Fuentes, S. Yoon, S. C. Kim, and D. S. Park, “A robust deep-learning-based detector for real-time tomato plant diseases and pests recognition,” Sensors, vol. 17, no. 9, p. 2022, 2017.
  * [5] T. V Price, R. Gross, W. J. Ho, and C. F. Osborne, “A comparison of visual and digital image-processing methods in quantifying the severity of coffee leaf rust (¡I¿Hemileia vastatrix¡/I¿),” Aust. J. Exp. Agric., vol. 33, no. 1, pp. 97–101, 1993.
  * [6] G. Sambasivam and G. D. Opiyo, “A predictive machine learning application in agriculture: Cassava disease detection and classification with imbalanced dataset using convolutional neural networks,” Egypt. Informatics J., no. xxxx, 2020.
  * [7] Labelbox, ”Labelbox,” Online, 2020. [Online]. Available: https://labelbox.com
  * [8] Tan, M., Pang, R., and Le, Q. V. (2020). Efficientdet: Scalable and efficient object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 10781-10790).
  * [9] W. Liu et al., “SSD: Single shot multibox detector,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 9905 LNCS, pp. 21–37, 2016.
  * [10] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2016–December, pp. 779–788, 2016.
  * [11] Pan, Sinno Jialin, and Qiang Yang. ”A survey on transfer learning.” IEEE Transactions on knowledge and data engineering 22.10 (2009): 1345-1359.
  * [12] Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., … Zitnick, C. L. (2014). Microsoft COCO: Common Objects in Context. In D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars (Eds.), Computer Vision – ECCV 2014 (pp. 740–755). Cham: Springer International Publishing.
  * [13] A. Paszke et al., “PyTorch: An Imperative Style, High-Performance Deep Learning Library,” no. NeurIPS, 2019.
  * [14] Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Rafal Jozefowicz, Yangqing Jia, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mané, Mike Schuster, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015\. Software available from tensorflow.org.