Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000.
10.1109/ACCESS.2023.0322000

This work was funded (or partially funded) by the Center for Statistics and
Applications in Forensic Evidence (CSAFE) through Cooperative Agreements
70NANB15H176 and 70NANB20H019 between NIST and Iowa State University, which
includes activities carried out at Carnegie Mellon University, Duke
University, University of California Irvine, University of Virginia, West
Virginia University, University of Pennsylvania, Swarthmore College and
University of Nebraska, Lincoln.

Corresponding author: Abby Martin (e-mail: abby2@iastate.edu).

# Forensic Camera Identification: Effects of Off-Nominal Exposures

ABBY MARTIN1  ROY MAXION2  and JENNIFER NEWMAN3  Department of Mathematics,
Iowa State University, Ames, IA 50011 USA (e-mail<EMAIL_ADDRESS>Department of Computer Science, Carnegie Mellon University, Pittsburgh, PA
15213 USA (e-mail<EMAIL_ADDRESS> Department of Mathematics, Iowa State
University, Ames, IA 50011 USA (email<EMAIL_ADDRESS>
###### Abstract

Photo-response non-uniformity (PRNU) is a technology that can match a digital
photograph to the camera that took it. Due to its use in forensic
investigations and use by forensic experts in court, it is important that
error rates for this technology are reliable for a wide range of evidence
image types. In particular, images with off-nominal exposures are not
uncommon. This paper presents a preliminary investigation of the impact that
images with different exposure types — too dark or too light — have on error
rates for PRNU source camera identification. We construct a new dataset
comprised of 8400 carefully collected images ranging from under-exposed (too
dark) to nominally exposed to over-exposed (too bright). We first establish
baseline error rates using only nominally exposed images, resulting in a true-
positive rate of 100% and a true-negative rate of 99.92%. When off-nominal
images are tested, we find striking results: the true-negative rate for under-
exposed images is 99.46% (a false-positive rate of roughly one in two hundred,
typically unacceptable in a forensic context), and for over-exposed images the
true-positive rate falls to 82.90%. Our results highlight the importance of
continued study of error rates for the PRNU source camera identification to
assure adherence to the high standards set for admissibility of forensic
evidence in court.

###### Index Terms:

Camera Identification, Digital Forensics, Image Forensics, PRNU, Questioned
Images

=-21pt

##  I Introduction

Consider a photo found at a crime scene. The photo can be traced to the camera
that took it using a phenomenon known as photo response non-uniformity (PRNU).
PRNU is a unique and persistent pattern (fingerprint) of variabilities of
voltage levels across the pixels in a digital camera’s photosensitive image
sensor, and this pattern appears in each photo the camera takes. The PRNU
fingerprint of the camera’s sensor is compared with the PRNU fingerprint of a
photo, producing a score measured against an accepted fixed threshold to
determine a match (or non-match) between the camera sensor and the photo.

The PRNU source-camera-identification algorithm is based on a large-scale
study of images from Flickr [1], and is considered accurate across a variety
of images. Flickr is a public website for users to share images. Due to self-
curation (users uploading their best images), Flickr has a negligible
proportion of off-nominal exposure images [2] (e.g., over- or under-exposed,
like the examples in Fig. 2 [3]). Thus, results from Flickr data may have
overlooked limitations, including generalizing error rates to data not
represented in the Flickr dataset (e.g., off-nominal exposure types).

The exclusion of off-nominal data when testing forensics tools has real-world
consequences. Studies performed by the forensic community play a critical role
supporting admissibility of expert witness testimony under federal law
(including estimating error rates), as introduced by Daubert v. Merrell Dow
Pharmaceuticals [4]. A large-scale study [1] established the court-approved
PRNU source camera identification algorithm using Flickr images to determine
error rates. However, many forensic tools have opportunities for improvement
due to a host of factors such as lack of standardize corpora [5], non-existent
tool validation procedures [6], etc. We were unable to find any studies that
determined error rates for off-nominal exposure images for the PRNU source
camera identification algorithm. Additionally, the data in [1] is no longer
available [7]. This motivated our investigation of the PRNU source-camera-
identification algorithm applied to known off-nominal image data.

Our contribution is a methodology to evaluate the response of the PRNU source-
camera-identification algorithm to off-nominal exposure images. Forensic
experts representing both the prosecution and defense can use publications
incorporating this methodology to better represent the error rates associated
with specific evidence images, especially the false-positive error rate
(incorrectly matching an image with a camera). Our methodology can be adapted
to estimate PRNU source-camera-identification error rates for other types of
off-nominal images as well. This process can be integrated into the estimation
of error rates for a tool or technique to satisfy the Daubert criteria.

The rest of the paper is organized as follows: Section II outlines the
research problem and approach; Section III clarifies language used throughout
the remainder of the paper; Section IV is a discussion of the forensic
development of PRNU source camera identification; Section V provides a
detailed overview of the PRNU source-camera-identification algorithm used in
this paper; Section VI describes the data and characterizes under- and over-
exposed images; Section VII outlines the methodology of the experiments
performed; and Section VIII presents the results. In Section IX, we share the
limitations of the experiments, followed by a discussion of implications of
our results in Section X. We conclude with a summary in Section XI.

##  II Problem and Approach

Recognizing that prior work did not account for off-nominal images (e.g., too
light or too dark), the present work aims to determine whether off-nominal
images skew the true positive and true negative rates previously reported in
[1]. We implement the PRNU algorithm presented in that work to establish error
rates for off-nominal images. In Section VII, we present a rigorous evaluation
of the error rates for this source-camera-identification algorithm when the
image exposures are off-nominal, including a sensitivity analysis to determine
how proportions of off-nominal images in the dataset can impact error rate
estimates. In principle, this framework could also be applied to other types
of off-nominal image data (e.g., digital zoom or out-of-focus).

##  III Terminology

The image of unknown origin, which could be considered evidence in a forensic
case, is referred to as the questioned image. The camera believed to have
taken the questioned image is referred to as the specific camera. To avoid
repetitious language, when we refer to an image, we mean the questioned image
(unless otherwise specified). Similarly, when we refer to a camera, we mean
the specific camera.

We focus on the source-camera-identification problem, which aims to identify
the specific camera that captured a questioned image. We do not address the
camera-model-identification problem, which attempts to identify only the model
of the camera used to collect the questioned image. As an example, source
camera identification could conclude an image came from a specific iPhone6s
camera, whereas camera-model identification would only be able to conclude the
image was from some iPhone6s camera (i.e., _this_ camera versus this _kind_ of
camera). For simplicity, when we refer to camera identification we are
referring to the source-camera-identification problem. We use PRNU cameraID
algorithm to refer specifically to the source-camera-identification algorithm
as established in the large-scale study [1], which is the algorithm used
throughout this paper.

##  IV Background and Related Work

Photo response non-uniformity (PRNU) is a persistent artifact in digital
images due to imperfections in the camera-sensor manufacturing process [8].
When light impinges on the photosensitive portion of a pixel, called the
photodiode, the pixel responds by generating a current in proportion to the
number of photons striking it. However, the imperfections result in consistent
differences from the mean values of currents among the pixels, and it is this
pattern of responses – the PRNU – that is unique to each camera sensor. This
pattern remains constant and present in each image; therefore, PRNU can be
considered part of the sensor’s fixed pattern noise.

The first digital image sensor, a charged-coupled device (CCD), was invented
in 1970 [9], and within a year large variabilities in dark signal (thermal
noise when no light is falling on the sensor) were observed [10]. The first
use of fixed pattern noise to identify an individual digital camera sensor
appeared in [11] using the dark signal. This method assumes scenes that are
very dark, so it is not useful for images taken under typical lighting
conditions.

In 2005, the first computational method was introduced for extracting the PRNU
from an image for the purpose of source camera identification [12]. Since the
PRNU can be extracted from photos taken in typical lighting environments,
researchers shifted to the PRNU as a camera fingerprint. Several improvements
followed, including: introducing a maximum likelihood estimator to improve the
camera fingerprint calculation [13]; changing to the peak-to-correlation
energy (PCE) ratio as a similarity metric [14]; and discovering that (unlike
correlation scores) a threshold value based on the PCE does not need to change
for each camera fingerprint estimation [15].

PRNU-based camera identification became standardized for court use in a large-
scale study performed on 1,053,580 JPEG images from Flickr representing
approximately 6,896 cameras over 150 models [1]. This work used the
distribution of PCE scores between the questioned images and the camera
fingerprints to set a threshold based on the false positive rate (FPR) to
ensure acceptable true negative rates (TNR) and true positive rates (TPR),
resulting in a recommended PCE threshold of 60. The overall identification
rates given in [1] are a TPR of 97.62% and a TNR of 99.9976%. Our goal is to
use the standard algorithm set by this work to establish error rates for a
very different set of data. In Section V, we give details of the PRNU source-
camera-identification (PRNU cameraID) algorithm as established in [1].

Research since 2009 has continued to modify and question PRNU camera
identification. Use of the PRNU has expanded to fields such as forensic
countermeasures for forged PRNU information [16, 17], image anonymity
techniques [18], convolutional neural networks (CNN)-based forgery detection
[19], and user authentication [20]. Many papers have proposed changes to the
PRNU cameraID algorithm: enhancements to fingerprint estimation [21, 22, 23,
24]; different similarity measures [25]; and use of machine-learning methods
such as CNNs [26]. Other work has focused on the impact of various image
artifacts on PRNU camera identification, including: vignetting [27]; JPEG
compression [28]; proprietary image processing [29, 30, 31, 32]; color
saturation [33]; as well as gamma correction, contrast enhancement, histogram
equalization, and white balance [34]. One large-scale study of over 33,000
images from Flickr found several devices with low true negative rates (e.g.,
0.8% for the Fujifilm X-T30 camera) [2] for images from the same camera model
as the specific camera.

Concerns about PRNU-based camera identification have been raised in
preliminary investigations of exposure settings, specifically ISO. A brief
study of the effects of ISO values on PRNU camera identification found that
ISO impacts noise levels, gray levels, and correlation values [35]. The
Warwick database [36], which contains images with a variety of ISO values,
establishes that forgery detection and correlation-predictor values are
impacted by ISO [37]. However, the Warwick investigation of ISO relies on
correlation predictors to identify forged regions of images and does not
implement a decision threshold [37]. The standard PRNU cameraID algorithm in
[1] relies on PCE scores and a threshold of 60, not correlation predictors.
Additionally, the authors in [37] vary both the ISO and exposure time settings
to ensure similar exposure values between images of the same scene, meaning
all images have a consistent visual brightness. The research question in [37]
differs from ours: their paper asks whether correlation predictors and forgery
detection are impacted by ISO (the exposure type of the image remains constant
by changing the exposure time to compensate for the changes in ISO). We ask
whether the PCE score and camera identification are impacted by changing the
exposure type (i.e., auto-, under-, or over-exposed images). For our data, we
intentionally vary the ISO and exposure time to gather brighter and darker
versions of the same scene, thus varying the exposure value and brightness of
the image (see Fig. 2).

In order to establish error rates for off-nominal images, we must adhere to
the algorithm used by forensic experts. To the best of our knowledge, this is
the algorithm implemented in the large-scale study [1]. Whereas these recent
works [35, 37] [36] have addressed the impact of ISO on correlation, forgery
detection, and correlation predictor values, we do not know of any work that
has addressed the problem of over- or under-exposed images and their effect on
error rates for the PRNU cameraID algorithm. This paper tackles exactly that
problem.

##  V Overview of PRNU Camera Identification

We refer to the protocol followed in [1] as the PRNU cameraID algorithm.
Generally, this algorithm consists of three parts: (1) estimating the PRNU
fingerprints of the camera and of the image; (2) calculating the peak-to-
correlation energy (PCE) ratio between these two fingerprints; (3) comparing
the PCE score with a threshold of 60 to determine whether (or not) the camera
captured the image. The details of this process are described in the remainder
of this section.

###  V-A Estimate Fingerprints Using PRNU Noise Model

This section gives a summary of the fingerprint extraction algorithms as
presented in [38]. To compare a camera with an image, we must estimate the
PRNU noise from both the camera sensor and the image. Let $I$ be an image, and
let $I_{0}$ be the corresponding “noiseless image” which would result from a
sensor without any imperfections. Similarly, let $K$ be the true PRNU noise
component of the camera. Note that all multiplication in this section is
performed element-wise, thus the image $I$ is modeled:

$I=I_{0}+I_{0}K+\Theta,$ (1)

where all noise components besides the PRNU are denoted by $\Theta$ [38].

However, it is not feasible to obtain the noiseless image $I_{0}$ or the true
PRNU fingerprint $K$. In order to approximate the PRNU component of the image
noise, a high-pass filter is applied to suppress scene content and reduce non-
unique low-frequency patterns included in the noise fingerprint of an image,
such as intensity gradient and vignetting. The Daubechies wavelet denoising
filter [39] was originally chosen because it produced the best experimental
results, likely due to its superior scene suppression (particularly for edges
that appear within an image) [8]. Hence, $F$ is the Daubechies denoising
filter applied to the image.

The noise $W$ for image $I$ can be estimated [13]:

$W=I-F(I).$ (2)

This denoising step is followed by additional image processing of the
grayscale image to remove non-unique artifacts (NUAs) due to JPEG-compression
and camera-model fixed-pattern noise. NUAs can increase the similarity between
images from different cameras and thus contribute to a lower TNR [40]. The
image fingerprint estimate is therefore calculated as:

$Q=G(W),$ (3)

where $G$ represents these additional image processing operations.

The camera-fingerprint estimation follows a similar process, but is calculated
using a set of several images. This improves the PRNU noise estimate in (2) by
implementing a maximum likelihood estimate of the true camera fingerprint $K$
using several images. First, we apply $F$, the Daubechies denoising filter, to
all $N$ images. Let $I^{(i)}$ for $i\in\\{1,2,...,N\\}$ be the set of images
used to estimate $\hat{K}$, the camera fingerprint. First, estimate the image
fingerprint for each $I^{(i)}$ as before, $W^{(i)}=I^{(i)}-F(I^{(i)})$. Then
the camera fingerprint $\hat{K}$ is estimated [41]:

$\hat{K}=G\left(\frac{\sum_{i=1}^{N}W^{(i)}I^{(i)}}{\sum_{i=1}^{N}(I^{(i)})^{2}}\right),$
(4)

where we used $N=30$ images, and $G(\cdot)$ represents the additional
processing performed after calculating the maximum likelihood estimate of the
camera fingerprint to remove NUAs due to the camera model and JPEG
compression, as done for the fingerprint estimate for a single image.

From a careful reading of the available code [42] in MATLAB [43], we observe
that saturated pixels of an image are excluded from the camera fingerprint
estimate, where saturated pixels are characterized by the maximum intensity
value of the image (at least 250) with at least one neighboring pixel of equal
intensity. Although we intentionally collected very bright images, none of the
pixels in our over-exposed (or any exposure type) image set meet these
standards for a saturated pixel.

###  V-B Peak-To-Correlation Energy (PCE) Calculation

When we calculate the similarity between camera and image fingerprints, we use
the signed PCE score given in Equation (8) of [28] as well as in the MATLAB
implementation [42]. Using the signed PCE score differs slightly from the PCE
calculation in [1], because if the peak correlation is negative, then the PCE
score will be negative. Negative PCE scores could change the estimated
probability density function (Equation (12) in [1]) used to set the threshold
of 60. An alternative distribution could impact the chosen PCE decision
threshold, which would change the estimated error rates. Although it is
unlikely that the signed PCE would impact the PRNU cameraID error rate
estimates, it is worth noting this change from the algorithm as initially
implemented [1].

###  V-C PRNU Source Camera Identification Algorithm Overview

To replicate the algorithm in [1], we used the MATLAB [43] code provided by
the same authors [42]. An overview of the PRNU cameraID algorithm is shown in
Fig. 1. The inputs are two fingerprints, one from the camera (Box 1 in Fig. 1)
and one from an image (Box 2 in Fig. 1), and the questioned image (Box 3 in
Fig. 1). Recall that the image fingerprint is estimated from a single image,
and the camera fingerprint is estimated from 30 images.

Figure 1: PRNU cameraID Algorithm. Boxes 1 and 2 are the fingerprints
estimated for the camera and questioned image, respectively. Operation 4 is an
element-wise multiplication between two matrices representing the camera
fingerprint (Box 1) and questioned image (Box 3). If the PCE (Box 5) is above
60, conclude that the image was taken by the specific-camera (Box 6);
otherwise not (Box 7).

The PCE score is calculated between the image fingerprint and the product of
the camera fingerprint and the image pixel intensities (operation 4 on Boxes 1
and 3 in Fig. 1). If the PCE score is greater than 60, conclude that the image
came from the camera under test (Box 6 in Fig. 1). Otherwise, conclude that
the image originated elsewhere (Box 7 in Fig. 1).

##  VI Data

The dataset from [1] is no longer available [7], so we collected 8400 images
from StegoAppDB [3] with specific ISO and exposure-time settings relative to
the auto-exposure settings. This section provides our characterization of off-
nominal exposure types, as well as the data sources, data protocol, and
theoretical and experimental support for our labeling decision of over- and
under-exposed images.

###  VI-A Characterization of Over- and Under-Exposed Images

A selection of over-, under-, and auto-exposed images from our dataset is
shown in Fig. 2, along with their ISO and exposure time settings. These are
the three exposure types of data we use in our experiments. Perceived
brightness of objects varies, so we characterize an off-nominally exposed
image by comparing its level of brightness with its nominally exposed version.
Specifically, we say an image is over-exposed (third row in Fig. 2) if its
overall visual appearance is noticeably brighter than its auto-exposed
counterpart (second row in Fig. 2). Similarly, we say an image is under-
exposed (first row in Fig. 2) if its overall visual appearance is noticeably
darker than its auto-exposed counterpart. See Section VI-B2 for the formulaic
relation between auto- and off-nominal exposure settings.

Figure 2: Off-Nominal and Nominal Exposure Examples. The first row is a sample
of under-exposed images, the second row is a sample of auto-exposed images,
and the third row is a sample of over-exposed images from our dataset [3]. The
pair (ISO, EXP) denotes the auto-exposure ISO and exposure time settings,
respectively. The programmed (ISO, EXP) values relative to the auto-exposed
settings are given to the left of the images under the exposure type, and the
recorded values for the actual data are provided underneath the photo itself.
The model and device number that acquired the trio of images is given at the
top of each column of images (model-device number). Five different devices
provided the sample data.

The intentional selection of exposure settings relative to auto-exposure is
one supporting argument for an image being over- or under-exposed, but we
provide two additional arguments that strengthen our characterization of
exposure type: (1) the exposure value [44] quantifies the brightness of each
off-nominal image relative to its auto-exposed version; and (2) a visual
inspection by three human judges assesses the agreement between the captured
exposure settings and a human visual assessment of the brightness level. With
these additional arguments, we obtain an acceptable level of certainty that
the characterizations of exposure type for over-, under-, and auto-exposed
images in StegoAppDB are indeed consistent and are satisfactory for testing
the PRNU cameraID algorithm for nominal and off-nominal exposures.

###  VI-B Apparatus & Instrumentation

####  VI-B1 Apparatus

Twenty-eight smartphone cameras were used to acquire the images used in this
research; they are cataloged in Table I.

####  VI-B2 Instrumentation

The controlled collection of images for hundreds of scenes using 28 separate
smartphone cameras in StegoAppDB [3] requires some amount of automation, not
least because the images were taken with specific ISOs and exposure times. For
this reason, an application (app) called Cameraw (pronounced camer-raw) was
written in Apple’s Swift language [45] for Apple devices, and in Java [46] for
Android devices.

TABLE I: Instruments, OS, protocols. Parenthetical numbers indicate multiple instances of the camera model. 100 over-, 100 auto-, and 100 under-exposed images (300 images total) per camera. | Camera (# of Devices) | OS | Protocol
---|---|---|---
1 | Pixel1 (4) | Android | A (Same Scene)
2 | Pixel2 (4) | Android | A (Same Scene)
3 | iPhone6s (4) | iOS | A (Same Scene)
4 | iPhone7 (4) | iOS | A (Same Scene)
5 | iPhone8 (2) | iOS | A (Same Scene)
6 | OnePlus5 (2) | Android | B (Unique Scene)
7 | SamsungS8 (2) | Android | B (Unique Scene)
8 | iPhone6sPlus (2) | iOS | B (Unique Scene)
9 | iPhone7Plus (2) | iOS | B (Unique Scene)
10 | iPhoneX (2) | iOS | B (Unique Scene)

Cameraw operates much like any other camera application, using one button to
capture a scene. When that button is pressed, Cameraw collects images with a
variety of exposure settings, stepping through a pre-selected sequence of
changing ISO and exposure times (EXP). From this sequence, we use the
following three settings: (1) auto-exposed/nominal (camera establishes ISO and
EXP automatically); (2) over-exposed/off-nominal (3 * ISO, 2 * EXP); and (3)
under-exposed/off-nominal (0.5 * ISO, 0.5 * EXP). Camera aperture for each
device remained constant (smartphone apertures cannot be changed).

###  VI-C Images & Data-Collection Protocols

####  VI-C1 Images

The data comprise 8400 images from StegoAppDB [3], with 300 images captured
from each of 28 smartphones across ten brands/models (e.g., Apple/iPhone8);
see Table I. One third of all the images were auto-exposed; one third of all
images were intentionally over-exposed; the remaining third were intentionally
under-exposed. Full-sized images are used for the experiments described in
this paper.

####  VI-C2 Protocol

Table I shows two protocols, A and B, that were followed during data-
collection. One (A) is for same-scene content; the other (B) is for unique-
scene content.

Protocol A: Same-scene content. Each of the 18 protocol-A cameras was attached
to a tripod in a given scene location. Each camera took three images with
staggered exposure settings, as previously described. All cameras were
similarly oriented, so there were no upside-down images. This process was
repeated 100 times, with 100 unique scene positions for the tripod. The scene
content was the same for all 18 cameras, although the registration may have
been slightly imperfect.

Protocol B: Unique-scene content. Ten cameras follow a “unique” scene content
acquisition protocol. This is the same as Protocol A except the scene content
is not repeated from one camera to the next. In contrast to the Protocol-A
images, which comprised 100 different scenes, Protocol-B images comprised 1000
different scenes.

###  VI-D Exposure Value Comparison

One clear justification for our characterization of auto-, under-, and over-
exposed images is the exposure value for the examples in Fig. 2. Consider the
three images of oranges in a bowl from the OnePlus5-1 camera (first column of
the figure). The top-row image is visually much darker than the middle-row
auto-exposed image, and the bottom-row image is visually much brighter. The
under-exposed image has an ISO of 500, half of 1000 (the auto-exposed image
ISO value), and an exposure time of 1/100 seconds (again, half of the auto-
exposed image exposure time of 1/50 seconds). Similarly, the over-exposed
image has an ISO of 3000, three times the auto-exposed ISO of 1000, and an
exposure time of 1/25 seconds (twice the auto-exposed image exposure time of
1/50 seconds).

The exposure value is a quantification of light on the sensor determined by
the $f$-stop (related to the aperture, which is constant in smartphone
cameras), exposure time, and ISO. Exposure value is lower when there is less
light and higher when there is more light on the sensor. We calculate exposure
value as described in [44] relative to the exposure value with ISO 100
($EV_{100}$). For the auto-exposed example image, the exposure value is
$EV_{100}+\log_{2}10$. The under-exposed example image has an exposure value
of $EV_{100}+\log_{2}5$ and the over-exposed example has an exposure value of
$EV_{100}+\log_{2}30$. Clearly, the under-exposed image has quantifiably less
and the over-exposed image has quantifiably more light on the sensor than the
auto-exposed image. This relationship holds for all images in our dataset,
which consists of only typically-lit indoor scenes.

###  VI-E Visual Validation of Off-Nominal Settings

Another supporting argument for the three exposure types is a validation of
the images using human judgment. By assessing the agreement amongst three
human judges and one computer program, we can measure the consistency of the
responses. If the consistency is high enough, we are satisfied that this
validation supports using these images for testing PRNU camera identification
of nominal and off-nominal exposures.

We used the same 5600 off-nominal images described in Section VI-C1, half of
which are under-exposed and half are over-exposed. This is too many images for
a human rater to evaluate in a reasonable amount of time without fatigue, so
we randomly selected 5% (280 images) for a human-judgment study. Half of those
(140 images) were too dark and the other half were too light. Each half was
mixed with 140 randomly drawn, nominally exposed images to form two sets of
280 images each.

The two sets of 280 images were shown to human judges in a web-based tool
displaying a single image at a time. The judge clicked one of three text boxes
indicating their judgment as auto/over/under-exposed. When a text box was
clicked, the tool logged the choice, and advanced to the next image. The task
took about 17 minutes per set.

The resulting data were analyzed using Fleiss’s kappa [47] with four raters –
three human judges and the computer program that chose the images in the first
place. Using the “R” statistical software package irr and the “R” function
kappam.fleiss [48, 49], the kappa value was 0.9430212 with a z-statistic of
75.70786 (i.e., 75.7 standard deviations away from the mean) with a p-value of
0 – a nearly-perfect agreement amongst the four raters. This is clearly a
level of confidence that justifies using these images for testing a camera-
identification algorithm.

##  VII Methods

We propose a methodical investigation of off-nominally exposed images. The
methodology consists of the following steps:

  1. 1.

Define a careful data collection protocol which minimizes the differences
between nominal and off-nominal data, intentionally changing the
characteristic under investigation.

  2. 2.

Isolate the points in the forensic algorithm where the characteristic under
investigation impact the error rate estimate.

  3. 3.

Establish baseline error rates by executing the forensic algorithm using only
the collected nominal data.

  4. 4.

For each point in the algorithm identified in step 2, incrementally and
independently exchange the nominal data with the off-nominal data.

We are investigating exposure settings, so our nominal dataset is our auto-
exposed images and the off-nominal dataset consists of our over- and under-
exposed images. Next, we identify estimation of the camera fingerprint (Box 1
in Fig. 1) and the questioned image (Boxes 2 and 3 in Fig. 1) as the two steps
of the PRNU cameraID algorithm directly impacted by image exposure settings.
Section VII-A details our baseline experiment, which generates camera
fingerprints using auto-exposed images and uses a set of questioned images
composed only of auto-exposed images. Finally, we iteratively change the
exposure type of the images used to generate the camera fingerprint and the
set of questioned images. This allows us to understand how off-nominal
exposure images alter the error rate estimates for the PRNU cameraID
algorithm, and provides baseline error rates for direct comparisons.

We investigate the impact of off-nominal exposure settings on the PRNU
cameraID algorithm by partitioning the data into three exposure types. Each
exposure type is used systematically to generate the camera fingerprints
and/or questioned images.

We performed 3 fundamental experiments:
1\. Auto-exposed images (baseline / nominal)
2\. Over-exposed images (too light / off-nominal)
3\. Under-exposed images (too dark / off-nominal)

We also implement two sensitivity analyses, one for the sensitivity of the TPR
to different proportions of off-nominal exposure images in the questioned
image set and one for the sensitivity of error rates relative to the PCE
decision threshold. A sensitivity analysis can show how even small, controlled
changes impact error rate estimates.

###  VII-A Auto-exposed images - baseline / nominal

The baseline experiment is the PRNU cameraID algorithm (Section V) applied to
a set of nominally exposed images, which comprises 2800 auto-exposed images
(100 auto-exposed images from each of the 28 cameras). We repeat the baseline
experiment for five trials, where each trial is defined by a specific
partitioning of the images.

First, we randomly partition the 100 images per camera into two groups: 1) 30
images used to generate the camera fingerprint; 2) 70 questioned images. The
images are partitioned so that no camera fingerprint image shares scene
content with any questioned image. For each camera, estimate the camera
fingerprint (28 cameras $\times$ 1 camera fingerprint = 28 camera
fingerprints). For each questioned image, estimate the image fingerprint (28
cameras $\times$ 70 images = 1960 image fingerprints).

Second, we calculate the PCE scores between each camera fingerprint and its
own questioned images (28 camera fingerprints $\times$ 70 fingerprints for
images from the camera = 1960 PCE scores). Next, we compare the PCE scores for
images from the camera to the threshold of 60. If the PCE score is above 60,
the image is a true positive and contributes to the TPR. If the PCE score is
at most 60, the image is a false negative and contributes to the False
Negative Rate (FNR = $1-$ TPR).

Next, we calculate the PCE scores between each camera fingerprint and a set of
questioned images from a different camera (28 specific-camera fingerprints
$\times$ 70 fingerprints from images from a different camera $\times$ 27
different cameras for test = 52920 PCE scores). To calculate a balanced
accuracy (i.e., an accuracy that responds equally to the TNR and TPR), we
select a random subset of 1960 PCE scores from the 52920 PCEs for questioned
images from a different camera. In order to avoid lucky or unlucky subsets of
PCE scores, we perform the random selection of 1960 PCE scores 100 times. We
compare the PCE scores for images from other cameras to the threshold of 60.
If the PCE score is above 60, the image is a false positive and contributes to
the FPR (FPR = $1-$ TNR). If the PCE score is at most 60, the image is a true
negative and contributes to the TNR. We calculate the accuracy for each of the
100 PCE score subsets using the 1960 PCE scores used to calculate the TPR, and
the 1960 PCE scores used to calculate the TNR. Finally, we average the TPR,
TNR, and accuracy values for the 100 PCE score subsets to calculate the error
rates for each of the five trials.

The auto-exposed image baseline experiment is also subjected to two
sensitivity analyses regarding the TPR and TNR: 1) for different proportions
of off-nominal exposures in the questioned image set, and 2) for different PCE
thresholds. For the first sensitivity analysis, we incrementally replace 1% of
the images comprising the questioned image set with off-nominal data, thus
creating 101 questioned image sets (i.e., the first set has 100% auto-exposed
questioned images and 0% off-nominally exposed images, the second set has 99%
auto-exposed questioned images and 1% off-nominally exposed images, and so on
with the final set comprising 100% off-nominally exposed images). The
sensitivity analysis of the PCE decision threshold is performed by
incrementally shifting the threshold of 60 and recalculating the TPR and TNR
values.

###  VII-B Over-exposed images - too light / off-nominal

The over-exposed image experiment differs from the baseline by estimating two
camera fingerprints (one using auto-exposed images and one using over-exposed
images) and the questioned image set comprises only over-exposed images. This
experiment uses 5600 images: 100 auto-exposed images from each of the 28
devices and 100 over-exposed images from each of the 28 devices.

We adapt the procedure used in the baseline experiment for the over-exposed
experiment by performing the same procedural steps but with different sets of
data: 1) 30 auto-exposed images per camera are used to estimate the camera
fingerprints and 70 over-exposed images from each camera comprise the set of
questioned images (auto-fingerprint vs. over-image); 2) 30 over-exposed images
per camera are used to estimate the camera fingerprints and 70 over-exposed
images from each camera comprise the set of questioned images (over-
fingerprint vs. over-image). We repeat the sensitivity analysis of the PCE
threshold on the auto-fingerprint vs. over-image and over-fingerprint vs.
over-image experiments.

We remark that the most likely scenario where a forensic practitioner might
encounter an over-exposed image is as a questioned image. The practitioner may
have access to the suspect camera and can therefore control the exposure
settings of images used to estimate the camera fingerprint, but the exposure
of the questioned image is already established. A forensic practitioner using
over-exposed images for the camera fingerprint when the questioned image is
auto-exposed is an unlikely scenario, so we omit the results of this
experiment. We investigate the over-fingerprint vs. over-image scenario as a
possible solution to the degraded error rates caused by comparing auto-exposed
fingerprints and over-exposed questioned images. This experiment examines
whether camera fingerprints estimated with the same exposure type as the
questioned image will have a more similar PRNU estimate than camera
fingerprints estimated solely from auto-exposed images, potentially reducing
the error rates.

The sensitivity analysis of the PCE decision threshold is performed for both
experiments by incrementally shifting the PCE threshold and recalculating the
TPR and TNR values.

###  VII-C Under-exposed images - too dark / off-nominal

The under-exposed image experiment repeats the investigations in Section VII-B
except we replace the over-exposed images with under-exposed images, as
characterized in Section VI — Data. The purpose behind each of the two under-
exposed image scenarios corresponds to the motivations for the over-exposed
experiment scenarios. Each investigation was also subjected to a sensitivity
analysis of the PCE threshold.

##  VIII Results

We present results from the experiments detailed in Section VII \- Methods for
images of each exposure type. The TPR decreases by at least 14.27% for the
off-nominal questioned images compared to the auto-exposed questioned-image
baseline, meaning many images from the specific camera are missed. Similarly,
the TNR decreases to 99.46% for under-exposed questioned images - an error of
approximately one in two hundred images incorrectly identified as a match with
the specific camera. Such mistakes can lead to increased false positives,
often connected to wrongful convictions. Our results are separated into
nominal and off-nominal questioned images, and end with an investigation of
possible remediations (Sections VIII-A, VIII-B, and VIII-C, respectively).

TABLE II: Aggregate results over 5 trials averaged over 100 repetitions Experiment | TPR (std.) | TNR (std.) | Accuracy (std.)
---|---|---|---
| Auto-fingerprint
---
vs. Auto-image Baseline
1 (0) |  0.9992 (0.0001) |  0.9996 (0.0001)
| Auto-fingerprint
---
vs. Over-image
0.8290 (0.0030) |  0.9998 (0.0001) |  0.9144 (0.0015)
| Auto-fingerprint
---
vs. Under-image
0.8573 (0.0003) |  0.9946 (0.0007) |  0.9260 (0.0004)
| Over-fingerprint
---
vs. Over-image
0.8888 (0.0070) |  0.9996 (0.0002) |  0.9442 (0.0035)
| Under-fingerprint
---
vs. Under-image
0.9999 (0.0002) |  0.8426 (0.0043) |  0.9213 (0.0020)
TABLE III: True Positive Rate (TPR), True Negative Rate (TNR), and standard deviation (STD) for auto-exposed camera fingerprint with auto-exposed questioned images (left column), over-exposed test data (middle column), and under-exposed test data (right column) by camera model. TPRs and TNRs lower than the baseline experiment are in BOLD for the off-nominal experiments. Camera Model | Auto-fingerprint vs. Auto-image Baseline | Auto-fingerprint vs. Over-image | Auto-fingerprint vs. Under-image
---|---|---|---
| TPR (STD) | TNR (STD) | TPR (STD) | TNR (STD) | TPR (STD) | TNR (STD)
Pixel1 | 1 (0) | 0.9998 (0.0002) | 0.9850 (0.0122) | 0.9995 (0.0003) | 1 (0) | 0.9857 (0.0021)
Pixel2 | 1 (0) | 0.9998 (0.0003) | 0.0007 (0.0016) | 0.9999 (0.0002) | 0.0014 (0.0020) | 0.9977 (0.0010)
iPhone6s | 1 (0) | 0.9999 (0.0001) | 0.9936 (0.0016) | 0.9997 (0.0001) | 1 (0) | 0.9984 (0.00003)
iPhone7 | 1 (0) | 0.9999 (0.0001) | 0.9750 (0.0067) | 0.9999 (0.0001) | 1 (0) | 0.9995 (0.0003)
iPhone8 | 1 (0) | 1 (0) | 0.9543 (0.0130) | 1.0000 (0.0001) | 1 (0) | 0.9998 (0.0003)
OnePlus5 | 1 (0) | 0.9984 (0.0009) | 0.9957 (0.0039) | 0.9993 (0.0006) | 1 (0) | 0.9673 (0.042)
SamsungS8 | 1 (0) | 0.9920 (0.0018) | 1 (0) | 0.9999 (0.0001) | 1 (0) | 0.9957 (0.0005)
iPhone6sPlus | 1 (0) | 1 (0) | 0.9914 (0.0032) | 0.9999 (0.0002) | 1 (0) | 0.9992 (0.0002)
iPhone7Plus | 1 (0) | 1 (0) | 0.8729 (0.0155) | 1.0000 (0.0001) | 1 (0) | 0.9994 (0.0004)
iPhoneX | 1 (0) | 1 (0) | 0.8829 (0.0139) | 1.0000 (0.0001) | 1 (0) | 0.9999 (0.0002)

###  VIII-A Baseline (Nominal Images)

The case where the camera fingerprint and questioned images are auto-exposed
provides a baseline to compare with the off-nominal experiments. Results from
this baseline experiment are shown in the first row of Table II, and represent
the current scenario used by forensic practitioners. The TPR is 100% and the
TNR is 99.92%. Results for the same baseline data are listed by the 10
individual models, shown on the left-hand side of Table III. These results are
in line with the error rates from the study in [1].

###  VIII-B Off-Nominal Questioned Image Sets

The most singular results using off-nominal data with the PRNU cameraID
algorithm are the TPR values shown in Table II. Rows two and three in Table II
are error rate estimates for the most common scenarios where a practitioner
might encounter an off-nominally exposed image: when the camera fingerprint is
estimated from auto-exposed images, but the questioned image is over- or
under-exposed. The TPR values for these two experiments are strikingly
different from the baseline results. When the camera fingerprint is composed
of auto-exposed images and all questioned images are over-exposed, the TPR is
82.90% and the TNR is 99.98% (row two, Table II). When the camera fingerprint
is composed of auto-exposed images and all questioned images are under-
exposed, the TPR is 85.73% and the TNR is 99.46% (row three, Table II). The
TPRs for these off-nominal images – a TPR decrease of 17.1% for over-exposed
and of 14.27% for under-exposed - are much lower than the 100% TPR for the
baseline experiment. The large reduction in TPR from our baseline experiment
warrants attention to the differing error rates between image exposure types.
Additionally, the TNR of 99.46% for the under-exposed questioned images (row
three, Table II) corresponds to an FPR of 0.54%. This is an error of roughly
one in two hundred associated with incorrectly matching an image with a
camera.

The sensitivity analysis of the TPR when the questioned image set consists of
various proportions of off-nominal images is given in Fig. 3. This sensitivity
analysis highlights the negative linear relationship between the proportion of
off-nominally exposed questioned images and the TPR. Both the over-exposed
questioned images (Fig. 3, solid blue line) and the under-exposed questioned
images (Fig. 3, dashed orange line) have a direct impact on the TPR estimate,
even when only a small percent of the questioned image set are off-nominally
exposed. The leftmost endpoints of the orange and blue lines in Fig. 3
correspond to TPRs of 100% (TPR for row one of Table II). Similarly, the
rightmost endpoints of the orange (dashed) and blue (solid) lines in Fig. 3
correspond to a TPR of 82.90% for over-exposed images (row two of Table II)
and 85.73% for under-exposed images (row three of Table II). Note that the TPR
for over-exposed images is always less than the TPR for under-exposed images,
regardless of the proportion of off-nominal test data. The consistency of this
linear relationship supports the notion that off-nominal exposure types do
indeed affect the error rates of the PRNU cameraID algorithm.

The results for the auto-exposed camera fingerprint experiments are listed for
the 10 models in Table III (auto-image in the left-hand column, over-image in
the middle column, and under-image in the right-hand column). Note that the
Pixel 2 performs dramatically poorly for both off-nominal exposures with a TPR
of 0.07% on over-exposed questioned images and TPR of 0.14% on under-exposed
questioned images. Although determining the cause of the Pixel 2 camera
model’s poor performance for off-nominally exposed images is outside the scope
of this paper, we theorize this could be caused by proprietary pipeline
processing, which is protected by manufacturers (in this case, Google). Poor
performance for specific camera models has been observed in prior research,
such as the 0.8% TNR of the Fujifilm X-T30 in [2]. Particularly poor
performance by individual models is another reason to encourage rigorous tool
validation procedures [6].

Figure 3: Sensitivity Analysis of Off-Nominally Exposed Questioned Images:
Both under-exposed (dashed orange line) and over-exposed (solid blue line)
questioned images have a roughly linear relationship with the degradation of
the True Positive Rate (TPR). This implies that even small percentages of off-
nominally exposed images in the questioned image set can have an impact on the
error rate estimates.

###  VIII-C Possible Solutions for Off-Nominal Data

Figure 4: Sensitivity analysis of PCE threshold (off-nominal exposures). The
upper left graph (baseline) shows a near-perfect TPR and TNR for the PCE
threshold of 60 (dashed red line) when both the camera fingerprint and the
image fingerprint are estimated from nominal images. However, when
fingerprints are estimated from any off-nominal images, either the TPR or the
TNR degrades dramatically. Raising the PCE threshold does not improve the TPR
(solid blue line), although it does gradually improve the TNR (dotted orange
line). Lowering the PCE threshold slightly improves the TPR, but does degrade
the TNR.

One possible method to improve the error rates for off-nominal exposures is to
estimate the camera fingerprint from the same exposure type as the questioned
image. When the camera fingerprint uses only over-exposed images and the
questioned images are also over-exposed (row 4 in Table II), the TPR rises
slightly to 88.88%, but the TNR falls to 99.96%. Similarly, when the camera
fingerprint uses only under-exposed images and the questioned images are also
under-exposed (row 5 in Table II), the TPR increases considerably (99.99%),
yet the TNR decreases (84.26%). A drop in the TNR corresponds to an increase
in false positives, so this is a trade-off that forensic practitioners are
unlikely to accept.

Another possible solution to the reduced accuracy values in Table II for off-
nominal exposures is to use an alternative PCE threshold. We perform a
sensitivity analysis of the TPR and TNR to different PCE thresholds by
incrementally shifting the integer PCE threshold and recalculating the TPR and
TNR using the same set of PCE scores. Fig. 4 demonstrates that lowering the
threshold to increase the TPR for the over-exposed image experiments (plots
(b) and (d) in Fig. 4) or for the under-exposed image experiments (plots (c)
and (e) in Fig. 4) would also lower the TNR, meaning that the overall accuracy
would not improve.

Our final experiment also investigates an alternate PCE threshold. The PCE
threshold of 60 was initially set to produce a 0% FPR on the subset of images
from a different model than the specific camera [1]. We compute the lowest
integer threshold value to produce a 0% FPR (regardless of camera model) and
recalculate the accuracy (results for each experiment are given in Table IV).
Note that in each case the threshold must be raised, and in some cases more
than doubled (rows 1 and 3 through 5 in Table IV). For the over-exposed image
experiments, these accuracy values are all lower than the accuracy using the
PCE threshold of 60 (rows 2 and 4 in Table IV). The alternate threshold does
improve the accuracy slightly for the auto-fingerprint vs. under-image
experiment (row 3 in Table IV) compared with the results for the PCE threshold
of 60 (row 3 in Table II). Choosing a threshold based on a 0% FPR, however,
not only depends on the dataset and exposure type, but is problematic for
other reasons detailed in Section X.

TABLE IV: Lowest threshold resulting in a 100% TNR (0% FPR) for each experiment and the corresponding TPR and accuracy. Experiment | Threshold | TPR | TNR | Accuracy
---|---|---|---|---
| Auto-fingerprint
---
vs. Auto-image Baseline
210 | 1 | 1 | 1
| Auto-fingerprint
---
vs. Over-image
77 | 0.8213 | 1 | 0.9107
| Auto-fingerprint
---
vs. Under-image
143 | 0.8569 | 1 | 0.9285
| Over-fingerprint
---
vs. Over-image
442 | 0.2300 | 1 | 0.6150
| Under-fingerprint
---
vs. Under-image
539 | 0.7587 | 1 | 0.8794

##  IX Limitations

Digital image forensics researchers typically encounter one of two problems
for data collection: resource exhaustion or quality of images. Manually taking
enough pictures from a large variety of cameras is often infeasible due to the
significant time and resources required. An alternative source of data
collection is scraping images from online collections (e.g., Flickr). However,
these images are of unknown provenance and their true origin and exposure
settings are frequently unknown.

The foundational PRNU cameraID research [1] prioritized the size of the
dataset by downloading over one million Flickr images from nearly seven
thousand cameras, enabling them to make universal claims. The absence of a
data-collection protocol makes it impossible to ascertain the proportion of
nominal/off-nominal images in that dataset. We chose instead to prioritize
control over the image collection and exposure settings by taking 8400
calibrated images from 28 cameras, as detailed in Section VI \- Data. Although
we are not claiming universality, we present our results as a demonstration
that off-nominal exposure settings can alter camera-identification error
rates, often quite dramatically. Our claims about our dataset are only
possible because of the exacting data-collection protocol followed. A similar,
but larger-scale study remains for future research.

A further limitation is that the cameras used in our research are not the most
recent models (e.g., the iPhone X was released in November 2017). We have
estimated that it would take at least 14 weeks to add just one more camera of
a new model. Adding a new camera would delay the preparation of a technical
report by roughly a quarter of a year, by which time yet another new model
would likely have been released. Therefore, it is simply not practical to
continually add up-to-date cameras for the purpose of one paper. Newer camera
models include additional complications that must be addressed in future work,
including high dynamic ranges, multiple lenses and sensors, new proprietary
processing, AI, etc. That said, we recognize the importance of maintaining
current datasets if camera-identification technology is to keep pace with
camera development.

##  X Discussion

The off-nominal image experiments performed dramatically poorly when compared
with the auto-exposed image baseline. The 14-17% TPR decrease is quite large
(rows two and three, Table II) and cannot be attributed to chance, poor data
quality, or methodological errors. When used to investigate which camera
captured an evidence image, a TPR of 85.73% is roughly one out of seven,
meaning the correct camera might be missed as many as one out of seven times
if the questioned image is under-exposed (or more often if the questioned
image is over-exposed).

One goal of forensic research is to prepare methodologies for use in a court
of law. The PRNU camera-identification algorithm described in [1] forms the
foundation of an FBI application called FINDCamera [50]. The false-positive
error rate estimate of one in a million presented by an expert witness was
based on over one million images from Flickr [51]. Our results show that the
false-positive error rate can rise to one in two hundred for under-exposed
questioned images, and one in five thousand for over-exposed questioned
images. Differences in error rate estimates could impact how jurors weigh the
strength of evidence, which is particularly important in cases with severe
consequences. For example, the aforementioned 2011 trial resulted in a prison
sentence of 45 years [51].

We investigated two seemingly-obvious solution candidates that, unfortunately,
turned out not to fully address the aforementioned shift in error rates. The
first solution was to create camera fingerprints consistent with the exposure
type of the questioned image. However, this approach only modestly improves
the over-exposure true-positive error rates, while causing the false-positive
rate to skyrocket for under-exposed images (rows 4 and 5 of Table II). A
second solution is to use a PCE threshold other than 60, as demonstrated in
Fig. 4 and in Table IV. Note that the alternate thresholds only minimally
improve the error rates, and in some cases simply exacerbate the problem.
Additionally, recent work has raised concerns that examiners changing decision
thresholds based on subjective analysis of the evidence and forensic scenario
can negatively impact the legal system [52].

Although there is a good instinct to mitigate any error inherent to a
methodology, it can be difficult to change existing forensic tools. Further
refining error rates in context of the questioned image can help forensic
practitioners better understand and communicate the error associated with pre-
existing tools. Introducing new methodologies requires both acceptance by
forensic practitioners and rigorous research to meet the Daubert standard,
which requires time, use by others in the community, and passing another
Daubert challenge for the new tool. Methods which can be used in conjunction
with current tools (such as our proposed protocol to estimate more accurate
error rates with respect to the exposure type of the questioned image) can
introduce needed incremental changes between significant technological shifts.

Our experiments clearly demonstrate that off-nominal images (e.g., over- or
under-exposed) impact the error rates of the PRNU cameraID algorithm. We also
show that the two most obvious and straightforward modifications to the
existing algorithm do not adequately rectify the performance problems for off-
nominal exposure images. Proper tool validation and error rate estimation is a
crucial aspect of this forensic field that must continually be improved.

##  XI Conclusions

We present a study of the PRNU source-camera-identification algorithm [1] for
off-nominal (over- and under-exposed) images using a meticulously-collected
dataset [3]. Our work implements a systematic investigation to show that error
rates are worse when off-nominal images are used for forensic source camera
identification. In particular, for over-exposed questioned images the true-
positive rate is 82.90%, as compared with 100% for nominal (auto-exposed)
images. Of note is the contrast between our nominal baseline’s false-positive
rate (0.08%) and the roughly one in two hundred false-positive rate for under-
exposed (too dark) images. This disparity is concerning, as it can have real-
life consequences in the criminal judicial process. Simple and obvious
mitigations, such as changing PCE thresholds, do not solve the error-rate
problem for off-nominal images. The insight gained from our methodology can
help forensic practitioners better understand and communicate the error rates
of forensic tools when applied to data representing off-nominal conditions.

##  Appendix A Data and Code Availability

Data from this study are available upon request. Code is available from its
authors [42].

##  Appendix B Attributions/Declarations

Author contributions:

AM: Conceptualization, analysis, initial/final drafts.

RM: Conceptualization, methodology, analysis, all drafts.

JN: Conceptualization, funding acquisition, initial draft.

Ethical approval:

The Iowa State University Institutional Review Board confirmed that the human-
subjects aspect of this study is exempt; the information collected contains no
personally identifiable information, and is not intended to contribute to
generalizable knowledge.

Conflicts:

The authors declare no conflicts of interest.

## Acknowledgment

We thank Huayun Huang for her design and implementation of the user-study
data-set validation software.

## References

  * [1] M. Goljan, J. Fridrich, and T. Filler, “Large scale test of sensor fingerprint camera identification,” in IS&T/SPIE Electronic Imaging Science and Technology, pp. 1–12, 04 February 2009.  San Jose, CA, USA, 18-22 January 2009.
  * [2] M. Iuliani, M. Fontani, and A. Piva, “A leak in PRNU based source identification—questioning fingerprint uniqueness,” IEEE Access, vol. 9, pp. 52455–52463, 2021.
  * [3] J. Newman, L. Lin, W. Chen, S. Reinders, Y. Wang, M. Wu, and Y. Guan, “StegoAppDB: A steganography apps forensics image database,” in Proceedings of the IS&T International Symposium on Electronic Imaging: Media Watermarking, Security, and Forensics (N. D. M. Adnan M. Alattar and G. Sharma, eds.), pp. 536–1 – 536–11, 13-17 January 2019.  Burlingame, CA, USA, 13-17 January 2019.
  * [4] Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993).
  * [5] S. Garfinkel, P. Farrell, V. Roussev, and G. Dinolt, “Bringing science to digital forensics with standardized forensic corpora,” Digital Investigation, vol. 6, pp. S2–S11, 2009.
  * [6] N. Hughes and U. Karabiyik, “Towards reliable digital forensics investigations through measurement science,” WIREs Forensic Science, vol. 2, no. 4, p. e1367, 2020.
  * [7] M. Goljan, “Personal communication; email to R. Maxion, 04 March 2023,” 2023.
  * [8] J. Lukáš, J. Fridrich, and M. Goljan, “Digital camera identification from sensor pattern noise,” IEEE Transactions on Information Forensics and Security, vol. 1, no. 2, pp. 205–214, 2006.
  * [9] W. Boyle and G. Smith, “Charge coupled semiconductor devices,” The Bell System Technical Journal, vol. 49, no. 4, pp. 587–593, 1970.
  * [10] M. Tompsett, G. Amelio, W. Bertram, R. Buckley, W. McNamara, J. Mikkelsen, and D. Sealer, “Charge-coupled imaging devices: Experimental results,” IEEE Transactions on Electron Devices, vol. 18, no. 11, pp. 992–996, 1971.
  * [11] K. Kurosawa, K. Kuroki, and N. Saitoh, “CCD fingerprint method-identification of a video camera from videotaped images,” in International Conference on Image Processing, pp. 537–540, 1999.  Kobe, Japan, 24-28 October 1999.
  * [12] J. Lukas, J. Fridrich, and M. Goljan, “Determining digital image origin using sensor imperfections,” in Proceedings of SPIE IS&T Electronic Imaging Science and Technology: Image and Video Communications and Processing (A. Said and J. G. Apostolopoulos, eds.), vol. 5685, pp. 249–260, 16-20 January 2005.  San Jose, CA, USA, 16-20 January 2005.
  * [13] M. Chen, J. Fridrich, and M. Goljan, “Digital imaging sensor identification (further study),” in Sec., Steg., and Watermarking of Multimedia Contents IX (E. J. D. III and P. W. Wong, eds.), vol. 6505, pp. 258 – 270, 2007.
  * [14] M. Goljan and J. Fridrich, “Camera identification from cropped and scaled images,” in Proceedings of the IS&T International Symposium on Electronic Imaging: Security, Forensics, Steganography, and Watermarking of Multimedia Contents X, vol. 6819, 68190E, pp. 68190E–1 – 68190E–13, 26-31 January 2008.  San Jose, CA, USA, 26-31 January 2008.
  * [15] M. Goljan, “Digital camera identification from images – estimating false acceptance probability,” in International Workshop on Digital Watermarking, p. 454—468, 2008.  Busan, Korea (Republic of), 10-12 November 2008.
  * [16] M. Goljan, J. Fridrich, and M. Chen, “Sensor noise camera identification: Countering counter-forensics,” in Proceedings of SPIE IS&T Electronic Imaging: Media Forensics and Security II (A. M. A. Nasir D. Memon, Jana Dittmann and E. J. D. III, eds.), vol. 7541, pp. 75410s–1 – 75410s–12, 18-20 January 2010.  San Jose, CA, USA, 18-20 January 2010.
  * [17] M. Goljan, J. Fridrich, and M. Chen, “Defending against fingerprint-copy attack in sensor-based camera identification,” IEEE Transactions on Information Forensics and Security, vol. 6, no. 1, pp. 227–236, 2011.
  * [18] M. Irshad, N.-F. Law, K. Loo, and S. Haider, “IMGCAT: An approach to dismantle the anonymity of a source camera using correlative features and an integrated 1D convolutional neural network,” Array, vol. 18, p. 100279ff, 2023.
  * [19] D. Cozzolino and L. Verdoliva, “Noiseprint: A CNN-based camera model fingerprint,” IEEE Transactions on Information Forensics and Security, vol. 15, pp. 144–159, 2020.
  * [20] D. Maier, H. Erb, P. Mullan, and V. Haupert, “Camera fingerprinting authentication revisited,” in International Symposium on Research in Attacks, Intrusions and Defenses (RAID 2020), pp. 31–46, October 2020.  San Sebastian, Spain, 14-16 October 2020.
  * [21] A. Cortiana, V. Conotter, G. Boato, and F. G. B. DeNatale, “Performance comparison of denoising filters for source camera identification,” in Proceedings of the SPIE IS&T International Symposium on Electronic Imaging:Media Watermarking, Security, and Forensics III (N. D. Memon, J. Dittmann, A. M. Alattar, and E. J. Delp III, eds.), vol. 7880, pp. 78807–1 – 78807–6, 23-27 January 2011.  San Francisco, CA, USA, 23-27 January 2011.
  * [22] J. Li, Y. Liu, B. Ma, C. Wang, C. Qin, X. Wu, and S. Li, “A novel PCA-based method for PRNU distillation to the benefit of source camera identification,” Applied Sciences, vol. 13, no. 11, 2023.
  * [23] S. Fernández-Menduiña, F. Pérez-González, and M. Masciopinto, “Source camera attribution via PRNU emphasis: Towards a generalized multiplicative model,” Signal Processing: Image Communication, vol. 114, p. 116944, 2023.
  * [24] A. Mehrish, A. V. Subramanyam, and S. Emmanuel, “Robust PRNU estimation from probabilistic raw measurements,” Signal Processing: Image Communication, vol. 66, pp. 30–41, 2018.
  * [25] S. Reinders, L. Lin, W. Chen, Y. Guan, and J. Newman, “Score-based likelihood ratiosfor camera device identification,” in Proceedings of the IS&T International Symposium on Electronic Imaging: Media Watermarking, Security, and Forensics (N. D. M. Adnan M. Alattar and G. Sharma, eds.), pp. 1–7, 26-30 January 2020.  Burlingame, CA, USA, 26-30 January 2020.
  * [26] M. Fanfani, A. Piva, and C. Colombo, “PRNU registration under scale and rotation transform based on convolutional neural networks,” Pattern Recognition, vol. 124, no. 108413, 2022.
  * [27] C.-T. Li and R. Satta, “Empirical investigation into the correlation between vignetting effect and the quality of sensor pattern noise,” IET Computer Vision, vol. 6, no. 6, pp. 560–566, 2012.
  * [28] M. Goljan, M. Chen, P. Comesaña, and J. Fridrich, “Effect of compression on sensor-fingerprint based camera identification,” in IS&T International Symposium on Electronic Imaging Science and Technology: Media Watermarking, Security, and Forensics (A. M. Alattar and N. D. Memon, eds.), pp. 1–10, Society for Imaging Science and Technology (IS&T), 14-18 February 2016.  San Francisco, CA, USA, 14-18 February 2016.
  * [29] D. Baracchi, M. Iuliani, A. G. Nencini, and A. Piva, “Facing image source attribution on iPhone X,” in International Workshop on Digital Watermarking, pp. 196–207, 2020.  Melbourne, Victoria, Australia, 25-27 November 2020.
  * [30] A. Montibeller and F. Pérez-González, “Exploiting PRNU and linear patterns in forensic camera attribution under complex lens distortion correction,” in IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1–5, 2023.  Rhodes Island, Greece, 04-10 June 2023.
  * [31] C. Albisani, M. Iuliani, and A. Piva, “Checking PRNU usability on modern devices,” in IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 2535–2539, 06-11 June 2021.  Toronto, ON, Canada, 06-11 June 2021.
  * [32] S. Joshi, P. Korus, N. Khanna, and N. Memon, “Empirical evaluation of PRNU fingerprint variation for mismatched imaging pipelines,” in IEEE International Workshop on Information Forensics and Security, pp. 1–6, 2020.  New York, NY, USA, 06-11 December 2020.
  * [33] K. Henry, Digital photography analysis: Analytical framework for measuring the effects of saturation on photo response non-uniformity.  University of Colorado at Denver, 2016.
  * [34] S. Samaras, V. Mygdalis, and I. Pitas, “Robustness in blind camera identification,” in International Conference on Pattern Recognition, pp. 3874–3879, 04 December 2016.  Cancun, Mexico, 04-08 December 2016.
  * [35] L. Lin, W. Chen, Y. Wang, S. Reinders, Y. Guan, J. Newman, and M. Wu, “The impact of exposure settings in digital image forensics,” in IEEE International Conference on Image Processing, pp. 540–544, 2018.  Athens, Greece, 07-10 October 2018.
  * [36] Y. Quan, C.-T. Li, Y. Zhou, and L. Li, “Warwick image forensics dataset for device fingerprinting in multimedia forensics,” in IEEE International Conference on Multimedia and Expo, pp. 1–6, 2020.  London, UK, 06-10 July 2020.
  * [37] Y. Quan and C.-T. Li, “On addressing the impact of ISO speed upon PRNU and forgery detection,” IEEE Transactions on Information Forensics and Security, vol. 16, pp. 190–202, 2021.
  * [38] M. Goljan, M. Chen, and J. Fridrich, “Identifying common source digital camera from image pairs,” in 2007 IEEE International Conference on Image Processing, vol. 6, pp. VI–125, IEEE, 2007.
  * [39] I. Daubechies, “Orthonormal bases of compactly supported wavelets,” Communications on Pure and Applied Mathematics, vol. 41, no. 7, pp. 909–996, 1988.
  * [40] J. Fridrich, “Sensor defects in digital image forensic [sic],” in Digital Image Forensics: There is More to a Picture than Meets the Eye (H. T. Sencar and N. Memon, eds.), pp. 179–218, New York, NY: Springer Science+Business Media, 2013.
  * [41] M. Chen, J. Fridrich, M. Goljan, and J. Lukáš, “Determining image origin and integrity using sensor noise,” IEEE Transactions on Information and Security, vol. 3, no. 1, pp. 74–90, 2008.
  * [42] M. Goljan, M. Chen, P. Comeaña, and J. Fridrich, “MATLAB/Python camera fingerprint implementation.” http://dde.binghamton.edu/download/camera_fingerprint/, 2016.  Last date modified: February 2016.
  * [43] The MathWorks Inc., “MATLAB version: 9.13.0 (R2022b),” 2022.  The MathWorks Inc., Natick, MA, United States.
  * [44] H.-C. Lee, Introduction to color imaging science.  Cambridge University Press, 2005.
  * [45] Apple Inc., “The Swift Programming Language,” 2014.  Apple Inc., Cupertino, CA, United States.
  * [46] K. Arnold, J. Gosling, and D. Holmes, “The Java programming language,” 2000.  Addison-Wesley Longman Publishing Co., Inc, Boston, MA, United States.
  * [47] J. L. Fleiss, “Measuring nominal scale agreement among many raters,” Psychological Bulletin, vol. 76, no. 5, pp. 378–382, 1971.
  * [48] M. Gamer, J. Lemon, I. Fellows, and P. Singh, “irr: Various Coefficients of Interrater Reliability and Agreement,” 2012.  R package, version 0.84, https://CRAN.R-project.org/package=irr.
  * [49] R Core Team, “R: A Language and Environment for Statistical Computing,” 2017.  R Foundation for Statistical Computing, Vienna, Austria.
  * [50] A. MacKenzie and W. E. Bruehs, “Photo response nonuniformity (PRNU) meets Daubert standards,” Journal of Forensic Identification, vol. 68, no. 4, pp. 467–471, 2018.
  * [51] US District Court, “United States of America v. Nathan Railey, United States District Court for the Southern District of Alabama, Case # 1:10-cr-00266-CG-N,” 2011.  PACER Document 198.pdf.
  * [52] W. C. Thompson, “Shifting decision thresholds can undermine the probative value and legal utility of forensic pattern-matching evidence,” Proceedings of the National Academy of Sciences, vol. 120, no. 41, p. e2301844120, 2023.

|  Abby Martin received her B.A. degree in Computer Science/Software
Engineering and Mathematics from Augustana University, Sioux Falls, SD, in
2017 and is pursuing her Ph.D. degree in Mathematics and Computer Science from
Iowa State University, Ames, IA under the supervision of Professors Jennifer
Newman and Jin Tian. From 2020 to 2024, she was a Research Assistant with the
Center for Statistics and Applications in Forensic Evidence. Her research
interests include camera source identification, steganalysis, and digital
image forensics.
---|---
|  Roy Maxion (IEEE Fellow) is a Research Professor emeritus in computer
science and machine learning at Carnegie Mellon University, where he is also
the director of the Dependable Systems Laboratory. He has long been a
passionate proponent of rigorous/foundational scientific methodology. He is an
IEEE Fellow, and recently served as a member of the US National Academy of
Sciences committee on Future Research Goals and Directions for Foundational
Science in Cybersecurity. He won the 2019 IEEE/IFIP Test-of-Time award for his
work using typing rhythms for user authentication. He recently won the Taiwan
Tamkang Panda Trophy for his lecture on the sensitivity of machine learning
systems to small irregularities in data. He is one of the founding members of
the US Center for Statistics and Applications in Forensic Evidence. He is a
member of the editorial boards of IEEE Security and Privacy and the
International Journal of Machine Learning.
---|---
|  Jennifer Newman Jennifer L. Newman (Senior Member, IEEE) is a Professor of
Mathematics at Iowa State University, Ames, Iowa and holds a Scott Hanna
Faculty Fellow in Mathematics. She received her B.A. in Physics from Mount
Holyoke College, and an M.S. and Ph.D. in Mathematics from the University of
Florida, Gainesville. She has served as an Associate Editor for the Journal of
Electronic Imaging and for the Journal of Mathematical Imaging and has served
on numerous program committees for many professional conferences. Her current
research is in statistical forensic imaging, including steganography,
steganalysis and camera identification. Other research interests include image
algebra; object detection, modeling and machine learning for images;
morphological and other neural networks; and statistical modeling of textures
for forward and inverse problems. She has over 80 refereed publications and
has been a Principal Investigator or Co-Principal Investigator on over $24
million in grants. She has mentored over 32 graduate students as their major
professor, and many undergraduate students as well.
---|---


11institutetext:  Université Côte d’Azur, Observatoire de la Côte d’Azur,
CNRS, Laboratoire Lagrange, France 22institutetext: V. N. Karazin Kharkiv
National University, 4 Svobody Sq., Kharkiv, 61022, Ukraine 33institutetext:
European Southern Observatory (ESO), Alonso de Cordova 3107, 1900, Casilla
Vitacura, Santiago, Chile 44institutetext: Department of Earth, Atmospheric
and Planetary Sciences, MIT, 77 Massachusetts Avenue, Cambridge, MA, 02139,
USA 55institutetext: Astronomical Institute, Academy of Sciences of the Czech
Republic, CZ-25165 Ondřejov, Czech Republic 66institutetext: INAF -
Osservatorio Astronomico di Roma, Via Frascati 33, I-00078 Monte Porzio
Catone, Italy 77institutetext: Department of Earth, Atmospheric, and Planetary
Sciences, Massachusetts Institute of Technology, 77 Massachusetts Avenue,
Cambridge, MA 02139, USA 88institutetext: INAF - Osservatorio Astronomico di
Roma, Monte Porzio Catone (RM), Italy 99institutetext: INAF - Department of
Physics and Astronomy, University of Padova, Vicolo dell’Osservatorio, 3,
I-35122 Padova, Italy 1010institutetext: Agenzia Spaziale Italiana (ASI), Via
del Politecnico 00133 Roma, Italy

# Compositional properties of planet-crossing asteroids from astronomical
surveys

A. V. Sergeyev Corresponding author<EMAIL_ADDRESS>B. Carry
11 M. Marsset 3344 P. Pravec 55 D. Perna 66 F. E. DeMeo 77 4 4 V. Petropoulou
88 M. Lazzarin 99 F. La Forgia 99 I. Di Petro 1010 the NEOROCKS team
The NEOROCKS team: E. Dotto, M. Banaszkiewicz, S. Banchi, M.A. Barucci, F.
Bernardi, M. Birlan, A. Cellino, J. De Leon, M. Lazzarin, E. Mazzotta Epifani,
A. Mediavilla, J. Nomen Torres, E. Perozzi, C. Snodgrass, C. Teodorescu, S.
Anghel, A. Bertolucci, F. Calderini, F. Colas, A. Del Vigna, A. Dell’Oro, A.
Di Cecco, L. Dimare, P. Fatka, S. Fornasier, E. Frattin, P. Frosini, M.
Fulchignoni, R. Gabryszewski, M. Giardino, A. Giunta, T. Hromakina, J.
Huntingford, S. Ieva, J.P. Kotlarz, M. Popescu, J. Licandro, H. Medeiros, F.
Merlin, F. Pinna, G. Polenta, A. Rozek, P. Scheirich, A. Sonka, G.B.
Valsecchi, P. Wajer, A. Zinzi.

###### Abstract

Context. The study of planet-crossing asteroids is of both practical and
fundamental importance. As they are closer than asteroids in the Main Belt, we
have access to a smaller size range, and this population frequently impacts
planetary surfaces and can pose a threat to life.

Aims. We aim to characterize the compositions of a large corpus of planet-
crossing asteroids and to study how these compositions are related to orbital
and physical parameters.

Methods. We gathered publicly available visible colors of near-Earth objects
(NEOs) from the Sloan Digital Sky Survey (SDSS) and SkyMapper surveys. We also
computed SDSS-compatible colors from reflectance spectra of the Gaia mission
and a compilation of ground-based observations. We determined the taxonomy of
each NEO from its colors and studied the distribution of the taxonomic classes
and spectral slope against the orbital parameters and diameter.

Results. We provide updated photometry for 470 NEOs from the SDSS, and
taxonomic classification of 7,401 NEOs. We classify 42 NEOs that are mission-
accessible, including six of the seven flyby candidates of the ESA Hera
mission. We confirm the perihelion dependance of spectral slope among S-type
NEOs, likely related to a rejuvenation mechanism linked with thermal fatigue.
We also confirm the clustering of A-type NEOs around 1.5–2 AU, and predict the
taxonomic distribution of small asteroids in the NEO source regions in the
Main Belt.

###### Key Words.:

Minor planets, asteroids: NEOs – Techniques: photometric – Surveys

##  1 Introduction

Asteroids are the remnants of the building blocks that accreted to form the
terrestrial planets and the core of the giant planets in the early Solar
System $4.6\text{\,}\mathrm{G}\mathrm{y}$ ago. Asteroids are also the origin
of the meteorites that fell on the planets, including the Earth. These
meteorites represent the only possibility to study in detail the composition
of asteroids in the laboratory (e.g., 2008-ChEG-68-Consolmagno;
2015-Icarus-252-Cloutis), with the exception of the tiny samples of rock,
provided by return-sample missions: JAXA Hayabusa (2011-Science-333-Yurimoto)
and Hayabusa-2 (2022Sci...375.1011T), as well as the soon due NASA OSIRIS-REx
(2017SSRv..212..925L).

In contrast to targeted sample collection, we cannot choose the origin of
meteorites striking the Earth. Identifying their source regions is therefore
crucial to determining the physical conditions and abundances in elements that
reigned in the protoplanetary nebula around the young Sun
(2006-MESS2-McSween). From the analysis of a bolide trajectory, it is possible
to reconstruct a meteorite’s heliocentric orbit (2006-MPS-41-Gounelle),
although such determinations have been limited to only a few meteorites
(2018-Icarus-311-Granvik).

Figure 1: Schematic view of the extraction, convertion, and merging of NEOs
from SDSS, SMSS, Gaia, and Classy catalogs.

Among the different dynamical classes of asteroids, the near-Earth and Mars-
crosser asteroids (NEAs and MCs), whose orbits cross that of the telluric
planets, form a transient population. Their typical lifetime is of only a few
million years before they are ejected from the Solar System, fall into the
Sun, or impact a planet (1997Sci...277..197G). We refer here to near-Earth
objects (NEOs) in a liberal sense, encompassing both asteroid-like and comet-
like objects whose orbits cross that of a terrestrial planet (hence including
NEAs, MCs, and some Hungarias).

These populations are of both scientific and pragmatic interest. As they are
closer to the Earth than the asteroid belt, we have access to smaller objects
from ground-based telescopes. Their orbital proximity implies a much smaller
impulsion to reach them with a spacecraft and make them favorable targets for
space exploration (2012-DPS-Abell). On the other hand, these objects could
potentially pose a threat, and studying their properties is a key aspect in
planning risk mitigation (2015hchp.book..763D), of which the National
Aeronautics and Space Administration (NASA) Demonstration for Autonomous
Rendezvous Technology (DART) and European Space Agency (ESA) Hera missions are
lively demonstrators (2021PSJ.....2..173R; 2022PSJ.....3..160M).

We focus here on the compositional properties of a large corpus of NEOs as
part of the NEOROCKS project (2021plde.confE.221D), whose goal is the
characterization of the NEO population. The article is organized as follows:.
In