# Smartphone Camera De-identification while Preserving Biometric Utility

Sudipta Banerjee and Arun Ross
Michigan State University
{banerj24, rossarun} @cse.msu.edu

###### Abstract

The principle of Photo Response Non Uniformity (PRNU) is often exploited to
deduce the identity of the smartphone device whose camera or sensor was used
to acquire a certain image. In this work, we design an algorithm that perturbs
a face image acquired using a smartphone camera such that (a) sensor-specific
details pertaining to the smartphone camera are suppressed (sensor
anonymization); (b) the sensor pattern of a different device is incorporated
(sensor spoofing); and (c) biometric matching using the perturbed image is not
affected (biometric utility). We employ a simple approach utilizing Discrete
Cosine Transform to achieve the aforementioned objectives. Experiments
conducted on the MICHE-I and OULU-NPU datasets, which contain periocular and
facial data acquired using 12 smartphone cameras, demonstrate the efficacy of
the proposed de-identification algorithm on three different PRNU-based sensor
identification schemes. This work has application in sensor forensics and
personal privacy.

††footnotetext:  978-1-7281-1522-1/19/$31.00 ©2019 IEEE

##  1 Introduction

Sensor identification, or source attribution, [6] refers to the automated
deduction of sensor identity from a digital image. Photo Response Non-
Uniformity (PRNU) based sensor identification algorithms have been
successfully used in the context of RGB [24, 13, 29] and near-infrared (NIR)
[18, 20, 2] images. PRNU based sensor identification algorithms typically use
image denoising techniques to elicit sensor specific details from an image.
The forensic utility of these algorithms has been well established in the
literature [14].

Since smartphone devices are intricately linked to their owners, sensor
identification using images from smartphone cameras can inevitably lead to
person identification. This poses privacy concerns to the general populace [1]
and, especially, to photojournalists [28]. Sensor de-identification can
mitigate such concerns by removing sensor specific traces from the image. A
number of sensor de-identification algorithms, particularly in the context of
PRNU suppression, have been developed in the literature [12, 32]. PRNU
suppression can be done by either PRNU anonymization or PRNU spoofing.

Figure 1: The objective of our work. The original biometric image is modified
such that the sensor classifier associates it with a different sensor, while
the biometric matcher successfully matches the original image with the
modified image.

PRNU anonymization is typically accomplished using strong filtering schemes
[11] that perturb the PRNU pattern, or irreversible transformations such as
‘seam carving’ [12] that systematically remove rows and columns from the
image. This produces geometrical perturbations which impair correct source
attribution. In [23], the authors performed ‘Signature Removal’ by subtracting
a scaled version of the PRNU signature (sensor reference pattern) from an
image. For an original image, $\bm{I}$, belonging to sensor $S$, the PRNU
anonynimized image is obtained as
$\bm{I^{\prime}}=\bm{I}-\gamma\bm{\hat{K}_{S}}$. Here, $\gamma$ is the scaling
factor for the sensor reference pattern $\bm{\hat{K}_{S}}$. PRNU spoofing, on
the other hand, involves deliberately confounding the image such that a sensor
classifier attributes it to a different target sensor rather than the actual
source sensor. This can be achieved by injecting the PRNU pattern of the
target sensor directly into the test image [15]. The modified image becomes
$\bm{I^{\prime}}=[\bm{I}+\bm{I}\times\gamma\bm{\hat{K}_{T}}]$. Here, $\bm{I}$
is the original image and $\bm{\hat{K}_{T}}$ is the reference pattern of the
target sensor $T$. The term $\gamma$ is a scalar parameter that needs to be
empirically tuned to achieve the spoofing. Alternatively, one can first
subtract the reference pattern or the ‘fingerprint’ of the source sensor and
then insert the target sensor pattern; this is known as ‘Signature
Substitution’ [23]. The modified image is represented as
$\bm{I^{\prime}}=\bm{I}-\gamma\bm{\hat{K}_{S}}+\beta\bm{\hat{K}_{T}}$.
$\bm{I}$ belongs to the source sensor $S$, whose reference pattern is
$\bm{\hat{K}_{S}}$. The terms $\gamma$ and $\beta$ are scalars which can be
optimized through grid-line search.

Majority of the work discussed above focus on images depicting natural scenes.
However, for biometric images (face, iris, fingerprints), the perturbations
may degrade the quality of the samples making them unsuitable for person
recognition. Limited work has been done in the context of sensor de-
identification for biometric samples [31]. In [4], the authors proposed an
iterative perturbation algorithm, which applied patch-wise modifications to
the input biometric image to perform sensor de-identification while retaining
its biometric utility. Current methods utilize parameters optimized through
exhaustive search techniques and require computation of sensor reference
patterns to achieve sensor de-identification [15, 23, 31]. Semi-adversarial
learning [27] based methods can be another viable way to accomplish sensor de-
identification while retaining biometric matching utility. However, such
methods are complex and require a large number of training images. The
objective of our work is to develop a rather simple method to perform sensor
de-identification, while preserving the biometric recognition utility of the
images. The key idea is illustrated in Figure 1. The merits of the proposed
method are as follows.

1\. Designing a sensor de-identification algorithm that can perform both PRNU
anonymization and PRNU spoofing in a non-iterative fashion. This addresses the
computational overhead incurred by the algorithms in [11, 4].
2\. The proposed de-identification algorithm is applicable to different PRNU
estimation schemes and works irrespective of the source and target sensors.
This eliminates the need for parameter optimization and computation of the
reference patterns corresponding to each pair of source and target sensors as
required in [15, 23].
3\. The proposed algorithm causes minimal degradation to the biometric content
of the images, thus retaining their biometric utility.

The remainder of the paper is organized as follows. Section 2 discusses the
concept of PRNU that has been used for sensor identification from images.
Section 3 describes the proposed algorithm for PRNU anonymization and PRNU
spoofing. In section 4, we describe the dataset, the experimental protocol and
the results. Section 5 concludes the paper.

##  2 Photo Response Non-Uniformity (PRNU)

Photo Response Non-Uniformity (PRNU) [24] manifests due to anomalies present
in the silicon wafer used during sensor fabrication. PRNU is a type of sensor
pattern noise which arises due to the variation in the response of the pixels
across the sensor plane to the same light intensity and is intrinsically
linked to each sensor. Therefore, PRNU can be used for sensor identification.
Sensor identification using PRNU requires the computation of the sensor
reference pattern from a set of training images acquired using a specific
sensor. When an unknown test image is provided, the noise residual of that
image is computed and correlated with the reference pattern of several
different sensors. The test image is attributed to the sensor whose reference
pattern yields the highest correlation value.

While a number of algorithms have been proposed in the literature to perform
sensor identification [3, 25], in this work we will focus on 3 of them — (i)
Enhanced PRNU Estimation, (ii) Maximum Likelihood based PRNU Estimation and
(iii) Phase PRNU Estimation methods — as these 3 fundamental PRNU based
methods have been widely used in the literature. Each method generates the
sensor reference pattern from a set of training images as described below.

1\. Enhanced PRNU Estimation: The reference pattern is computed as an average
of the noise residuals corresponding to the training images [22].
2\. Maximum Likelihood based PRNU Estimation (MLE PRNU): The noise residuals
from the training set are weighted by the images and normalized by the sum of
the squares of the pixel intensity values of the training images [9], and
furthermore, subjected to Wiener filtering and zero-mean operations to remove
interpolation artifacts.
3\. Phase PRNU Estimation: The phase component of the PRNU is computed in the
Fourier domain and then averaged across the training images [19] to generate
the reference pattern.

The test noise residual can be extracted by either simply applying a wavelet
based denoising filter to the test image or by applying an enhancement model
to further attenuate scene influences, as done in Enhanced PRNU Estimation
[22].

##  3 Proposed Method

Figure 2: Illustration of PRNU Anonymization. The DCT coefficients are
arranged such that the top-left portion has the low frequency components while
the bottom-right portion encapsulates the high frequency information. The PRNU
anonymized image is the result of suppression of high frequency components
(see Algorithm 1, here $\eta=0.9$).

Input: An image $\bm{I}$ of size $h\times w$ and parameter $\eta$

Output: PRNU anonymized image $\bm{I^{\prime}}$

1

2Apply 2-dimensional DCT to $\bm{I}$ $\bm{I}_{dct}=DCT(\bm{I})$

3Compute $m=\min(h,w)$ and $\alpha=round(\eta\times m)$

4Extract the high frequency components as follows:
$\bm{I}_{high}=High(\bm{I}_{dct},\alpha)$, where, the $High(\cdot,\cdot)$
operator extracts the lower triangular portion of the DCT coefficients along
the anti-diagonal direction, regulated by $\alpha$

5Extract the low frequency components as follows:
$\bm{I}_{low}=\bm{I}_{dct}-\bm{I}_{high}$

6Apply inverse DCT to obtain the modified image
$\bm{I^{\prime}}=DCT^{-1}(\bm{I}_{low})$

7Return the modified image $\bm{I^{\prime}}$

Algorithm 1 PRNU Anonymization. Figure 3: Illustration of PRNU Spoofing. The
high frequency components in the original image are suppressed first, the
residue being the low frequency components. The high frequency components of
the target sensor are further computed from the candidate images, and added to
the low frequency components of the original image, resulting in the PRNU
spoofed image (see Algorithm 2, here $\eta=0.7$).

Discrete Cosine Transform (DCT) has been successfully used for lossy image
compression [33] or for improving source camera identification [7]. The
coefficients located in the top-left portion capture the low frequency
components while the bottom-right coefficients encode the high frequency
components. Our goal is to modify the images to perturb the PRNU pattern
resulting in sensor de-identification. PRNU is a noise-like component which is
dominated by the high frequency components present in an image. Thus, we
propose to transform the image into the DCT domain and modulate the DCT
coefficients such that the high frequency components are suppressed, while
retaining the low frequency components. By suppressing the high frequency
components, we mask the sensor pattern present in the image. On the other
hand, we retain the low frequency components which primarily contain the scene
details in the image. The scene details are pivotal for biometric recognition.
Thus, we ensure preservation of the biometric utility of the image. We then
apply the inverse DCT, and the output is the modified image.

Input: An image $\bm{I}$ of size $h\times w$ belonging to source sensor $S$, a
set of $N$ candidate images belonging to the target sensor $T$, where each
image of size $p\times q$ is denoted as $\bm{G}^{i}$ ($i=[1,\cdots,N]$) and
$\eta$

Output: PRNU spoofed image $\bm{I^{\prime}}$

1 Set $i=1$

2Apply 2-dimensional DCT to $\bm{I}$, $\bm{I}_{dct}=DCT(\bm{I})$

3Extract the low frequency and high frequency components, $\bm{I}_{low}$ and
$\bm{I}_{high}$ as described in Algorithm 1 and set $\bm{I}_{high}=0$

4Compute $\alpha=round(\eta\times\min(p,q))$

5repeat

7 6 – Apply 2-dimensional DCT to $\bm{G}^{i}$,
$\bm{G}^{i}_{dct}=DCT(\bm{G}^{i})$

9 8– Extract the high frequency components as follows:
$\bm{G}^{i}_{high}=High(\bm{G}^{i}_{dct},\alpha)$

11 10– Apply inverse DCT to the high frequency content as follows:
$\bm{G^{\prime}}_{i}=DCT^{-1}(\bm{G}^{i}_{high})$

13 12– Add the images to generate $\bm{T}_{high}+=\bm{G^{\prime}}_{i}$ and
increment $i+=1$

14until  _$i=N$_ ;

15Divide by the number of images $\bm{T}_{high}=\frac{\bm{T}_{high}}{N}$

16Resize $\bm{T}_{high}$ to $h\times w$ using bicubic interpolation

17Apply inverse DCT to obtain the modified image
$\bm{I^{\prime}}=DCT^{-1}(\bm{T}_{high}+\bm{I}_{low})$

18Return the modified image $\bm{I^{\prime}}$

Algorithm 2 PRNU Spoofing.

To achieve sensor de-identification we perform both (i) PRNU Anonymization and
(ii) PRNU Spoofing.

PRNU Anonymization: Given an image $\bm{I}$, we first subject it to DCT to
yield $\bm{I_{dct}}$. We intend to suppress the high frequency information
without impairing the low frequency details. To achieve this goal, we define a
parameter $\alpha$ that serves as a regulator for high frequency suppression.
$\alpha$ is computed as the product of the minimum of the height and width of
the image $min(h,w)$, and a user-defined parameter $\eta$, rounded off to the
nearest integer. All DCT coefficients present in the interval
$[row=\alpha:h,col=\alpha:w]$ are set to zero. Thus, $\alpha$ represents the
threshold for the suppression of the DCT coefficients, and that threshold is a
function of the image dimensions. We discard the high frequency components and
then apply inverse DCT which results in the PRNU anonymized image
$\bm{I^{\prime}}$. The steps are described in Algorithm 1. The process of PRNU
anonymization is illustrated using an example image in Figure 2.

Table 1: Dataset specifications. The top block corresponds to MICHE-I dataset
[26] and the bottom block corresponds to OULU-NPU face dataset [8]. In the
MICHE-I dataset, we denote the brand Apple as ‘Device 1’ and the brand Samsung
as ‘Device 2’. Two different smartphones belonging to the same brand and
model, e.g., Apple iPhone5, are distinguished as ‘UNIT I’ and ‘UNIT II’.

Smartphone Brand and Model | Device Identifier | Sensor | Image Size |  | Number of Images/
---
Number of Subjects
(Training Set)
| Number of Images/
---
Number of Subjects
(Test Set)
Apple iPhone 5 |  Device 1 UNIT I | Front (F) | 960$\times$1280 | 55/7 | 344/41
Rear (R) | 1536$\times$2048 | 55/7 | 355/41
Apple iPhone 5 |  Device 1 UNIT II | Front (F) | 960$\times$1280 | 55/6 | 164/20
Rear (R) | 2448$\times$3264 | 55/6 | 170/20
Samsung Galaxy S4 |  Device 2 UNIT I | Front (F) | 1080$\times$1920 | 55/5 | 577/69
Rear (R) | 2322$\times$4128 | 55/5 | 600/70
Samsung Galaxy S6 Edge | — | Front (F) | 1080$\times$1920 | 55/6 | 0/0
HTC Desire EYE | — | Front (F) | 1080$\times$1920 | 55/6 | 0/0
MEIZU X5 | — | Front (F) | 1080$\times$1920 | 55/6 | 0/0
ASUS Zenfone Selfie | — | Front (F) | 1080$\times$1920 | 55/6 | 0/0
Sony XPERIA C5 Ultra Dual | — | Front (F) | 1080$\times$1920 | 55/6 | 0/0
Oppo N3 | — | Front (F) | 1080$\times$1920 | 55/6 | 0/0
TOTAL | 660/72 | 2,210/261

PRNU Spoofing: We want the sensor classifier to assign an image belonging to
source sensor $S$ to a specific target sensor $T$. To accomplish this task, we
perform the following steps.
(i) First, we compute the parameter $\alpha$. Next, we transform the original
image $\bm{I}$ from the source sensor to the DCT domain and then extract its
low frequency components (as done in Algorithm 1).
(ii) A set of $N$ candidate images, $\bm{G}^{i}$, $i=[1,\cdots,N]$, belonging
to the target sensor is selected, and each of them is subjected to DCT
resulting in $\bm{G}_{dct}^{i}$ (see Section 4.2). Next, we extract the high
frequency coefficients from each $\bm{G}_{dct}^{i}$, apply inverse DCT, and
then compute their average to yield $\bm{T}_{high}$. This averaged output
represents the sensor traces of the target sensor.
(iii) Finally, we insert the averaged high frequency coefficients into
$\bm{I}$ to generate $\bm{I^{\prime}}$ which will now be classified as
belonging to the target sensor, resulting in PRNU spoofing.
The implementation details for PRNU spoofing are described in Algorithm 2. The
process of PRNU spoofing is illustrated using an example image in Figure 3.

##  4 Experiments and Results

###  4.1 Dataset

We used the Mobile Iris Challenge Evaluation (MICHE-I) dataset [26] and the
OULU-NPU face dataset [8, 21] for performing the experiments in this work. The
MICHE-I dataset comprises of over 3,000 eye images from three devices: Apple
iPhone 5, Samsung Galaxy S4 and Samsung Galaxy Tab 2 [14]. However, in our
work, we employed the periocular images from two smartphones, Apple iPhone 5
and Samsung Galaxy S4, only. The authors in [14] discovered that two separate
units of Apple iPhone 5 were used for data collection. We refer to them as
Unit I and Unit II respectively. Further, the images in the dataset were
acquired using the front and rear camera sensors, separately. Thus, the
MICHE-I dataset used in this work consists of data from 6 sensors. The OULU-
NPU face dataset comprises of 4,950 face videos recorded using the front
cameras of six mobile devices—Samsung Galaxy S6 Edge, HTC Desire EYE, MEIZU
X5, ASUS Zenfone Selfie, Sony XPERIA C5 Ultra Dual and OPPO N3. The videos
were recorded in three sessions with different illumination and background
scenes. We only use the bonafide face videos/images in the OULU-NPU dataset
corresponding to 6 sensors. See Figure 4. The specifications of the dataset
are described in Table 1.

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

Figure 4: Example images from the MICHE-I and the OULU-NPU datasets acquired
using (a) Apple iPhone 5 Rear, (b) Samsung Galaxy S4 Front, (c) Samsung Galaxy
S6 Edge Front, (d) HTC Desire EYE Front, (e) MEIZU X5 Front, (f) ASUS Zenfone
Selfie Front, (g) Sony XPERIA C5 Ultra Dual Front and (h) OPPO N3 Front
sensors.

We split each dataset into a training set and a test set. We followed a
subject-disjoint protocol for creating the training and test sets. The images
in the training set are used for generating the reference pattern for each
sensor, as indicated in the fifth column of Table 1. Our training set consists
of 55 images [5] from each camera sensor in the MICHE-I dataset. The OULU-NPU
database contains videos, and so we selected 55 frames (20 frames from the
first session, 20 frames from the second, and 15 frames from the third) from 6
subjects, for each of the 6 sensors. The test set comprises of images
belonging to the MICHE-I dataset only (see the last column in Table 1). Thus,
our dataset consists of 2,870 images corresponding to 333 subjects acquired
using 12 camera sensors. Next, we describe the experiments conducted in this
work.

###  4.2 Experimental Methodology

For the sensor de-identification experiments, we first computed the sensor
reference patterns from the traning set for each of the 12 sensors (see Table
1) using the three PRNU estimation schemes viz., Enhanced PRNU, MLE PRNU and
Phase PRNU. Next, we used a small number of images (=10) as the validation set
to compute the parameter $\eta=[0,1]$ to be used for PRNU anonymization and
PRNU spoofing, separately. We estimated $\eta=0.9$ for PRNU anonymization and
$\eta=0.7$ for PRNU spoofing. The test experiments were conducted on images
belonging to the MICHE-I dataset only. However, the evaluation process
involved all the 12 sensor reference patterns. The experiments evaluated three
PRNU estimation schemes: Enhanced PRNU,111We employed Enhancement Model III
and we set the user defined threshold to 6 [22, 5]. MLE PRNU and Phase PRNU
methods. We used normalized cross-correlation for sensor identification. For
the PRNU spoofing experiments, the source and target sensors were from the
MICHE-I dataset and were either both front or both rear sensors. Thus, there
were $2\times\prescript{3\mkern-2.5mu}{}{P}_{2}=12$ PRNU spoofing experiments.
Due to the significant difference in resolutions between the front and rear
sensors of smartphones, we did not perform front-to-rear or rear-to-front
spoofing. We selected $N$, _i.e_., the number of candidate images belonging to
the target sensor (see Algorithm 2), to be the number of test images for that
sensor (see the last column in Table 1).

Table 2: Performance of the proposed algorithm for PRNU Anonymization in terms
of sensor identification accuracy (%). Results are evaluated using 3 PRNU
estimation schemes. ‘Original’ corresponds to sensor identification using
images prior to perturbation. ‘After’ corresponds to sensor identification
using images after perturbation and ‘Change’ indicates the difference between
the ‘Original’ and ‘After’ sensor identification accuracies. A high positive
value in the ‘Change’ field indicates successful PRNU Anonymization.

Device Identifier | Sensors | Enhanced PRNU | MLE PRNU | Phase PRNU
---|---|---|---|---
Original | After | Change | Original | After | Change | Original | After | Change
Device 1 UNIT I | Front | 99.71 | 18.31 | 81.40 | 99.71 | 17.73 | 81.98 | 99.71 | 22.67 | 77.04
Rear | 99.51 | 16.06 | 83.45 | 97.32 | 16.06 | 81.26 | 98.05 | 21.69 | 76.36
Device 1 UNIT II | Front | 96.34 | 21.34 | 75 | 96.34 | 25.61 | 70.73 | 93.90 | 26.83 | 67.07
Rear | 94.71 | 11.76 | 82.95 | 88.24 | 14.12 | 74.12 | 87.65 | 11.76 | 75.89
Device 2 UNIT I | Front | 100 | 3.81 | 96.19 | 100 | 5.72 | 94.28 | 100 | 13.69 | 86.31
Rear | 100 | 4.50 | 95.50 | 100 | 3.17 | 96.83 | 100 | 6.50 | 93.50
AVERAGE |  |  | 85.75 |  |  | 83.20 |  |  | 79.36

Table 3: Performance of the proposed algorithm for PRNU Spoofing in terms of
spoof success rate (SSR). Results are evaluated using 3 PRNU estimation
schemes. A high SSR value indicates successful spoofing.

Source Sensor | Target Sensor | Spoof Success Rate (%)
---|---|---
Enhanced PRNU | MLE PRNU | Phase PRNU
Device 1 UNIT I FRONT | Device 1 UNIT II FRONT | 100 | 100 | 100
Device 2 UNIT I FRONT | 96.80 | 100 | 100
Device 1 UNIT II FRONT | Device 1 UNIT I FRONT | 100 | 100 | 100
Device 2 UNIT I FRONT | 97.56 | 100 | 100
Device 2 UNIT I FRONT | Device 1 UNIT I FRONT | 100 | 100 | 100
Device 1 UNIT II FRONT | 100 | 100 | 100
Device 1 UNIT I REAR | Device 1 UNIT II REAR | 100 | 100 | 94.08
Device 2 UNIT I REAR | 99.44 | 96.90 | 97.46
Device 1 UNIT II REAR | Device 1 UNIT I REAR | 100 | 100 | 100
Device 2 UNIT I REAR | 100 | 98.82 | 100
Device 2 UNIT I REAR | Device 1 UNIT I REAR | 100 | 100 | 100
Device 1 UNIT II REAR | 100 | 100 | 99.83
AVERAGE | 99.48 | 99.64 | 99.28

Front-Indoor Front-Outdoor
Rear-Indoor Rear-Outdoor

(a)

Front-Indoor Front-Outdoor
Rear-Indoor Rear-Outdoor

(b)

Front-Indoor Front-Outdoor
Rear-Indoor Rear-Outdoor

(c)

Figure 5: ROC curves for matching PRNU Anonymized images. Each row corresponds
to a different device identifier: (a) Device 1 UNIT I, (b) Device 1 UNIT II
and (c) Device 2 UNIT I.

Front-Indoor Front-Outdoor
Rear-Indoor Rear-Outdoor

(a)

Front-Indoor Front-Outdoor
Rear-Indoor Rear-Outdoor

(b)

Figure 6: ROC curves for matching PRNU Spoofed images. Here, the source sensor
is Device 1 UNIT I. In this case, the target sensors are: (a) Device 1 UNIT II
(top row) and (b) Device 2 UNIT I (bottom row).

For the biometric matching experiments, we considered a periocular matcher, as
many of the images used in this work are partial face images. We employed the
ResNet-101 [16] architecture pre-trained on ImageNet [10] dataset for
performing periocular matching. We utilized the features from layer 170 which
were shown to perform the best for periocular matching in [17]. We applied
Contrast Limited Adaptive Histogram Equalization (CLAHE) to the images before
feeding them to the convolutional neural network. We used the cosine
similarity for computing the match score between the probe and gallery images.
We performed three sets of matching experiments, viz., (i) original: both
probe and gallery images comprise of unmodified images, (ii) after: both probe
and gallery images comprise of modified images and (iii) cross: the gallery
images are the original samples while the probe images are the modified images
and the genuine scores are computed by utilizing 2 sample images (belonging to
the same subject), _i.e_., the original image and the modified image; the
impostor scores are computed by taking pairs of samples belonging to different
subjects. Furthermore, we conducted experiments separately for the two
acquisition settings in this database: Indoor and Outdoor.

###  4.3 Results

For the sensor de-identification experiments, we used sensor identification
accuracy as the evaluation metric for PRNU anonymization and the spoof success
rate (SSR) as the evaluation metric for the PRNU spoofing algorithm. For PRNU
anonymization, we first compute the sensor identification accuracy of the
original images. Before perturbation, the images are assigned to the correct
sensor with high accuracies by all three PRNU estimation schemes (see
‘Original’ column in Table 2). Next, when the sensor classifier accepts the
modified images as input, the results indicate a significant degradation in
the sensor identification accuracy for a majority of the cases (see ‘After’
column in Table 2). The differences in the sensor identification accuracies
before and after perturbation are reported in the ‘Change’ column in Table 2.
An average difference (change) of 82.77% in the sensor identification
accuracies between pre- and post-perturbed images is observed for all the
three PRNU estimation schemes evaluated in this work (Enhanced PRNU: 85.75%,
MLE PRNU: 83.20% and Phase PRNU: 79.36%). The results indicate successful PRNU
anonymization thereby ensuring sensor de-identification.

The second set of results, pertaining to PRNU spoofing, reports the SSR for
the perturbed images. SSR computes the proportion of perturbed images that are
assigned to the target sensor. The results in Table 3 indicate successful
spoofing with respect to all the PRNU estimation schemes considered in this
work. An average SSR of 99.48% is observed when evaluated using Enhanced PRNU,
99.64% when evaluated using MLE PRNU, and 99.28% when evaluated using Phase
PRNU for all 12 PRNU spoofing experiments. The proposed spoofing experiment
fails to confound the Phase PRNU estimation scheme, particularly when the
source sensor is the rear sensor of Device 1 UNIT I and the target sensor is
the rear sensor of Device 1 UNIT II. Upon analysis, we observed that the
original images belonging to Device 1 UNIT II rear sensor resulted in the
lowest sensor identification accuracy for all three PRNU estimation schemes
(see Table 2). We speculate that the images may contain some artifacts that
are interfering with reliable PRNU estimation as well as the spoofing process.
Therefore, we performed another experiment where we increased the value of
$\eta$ from 0.7 to 0.9 for that particular spoofing experiment and we observed
that the SSR increased to 100% for all 3 PRNU estimation schemes. However,
visual analysis reveals that the spoofed images resulting from the two
different values of $\eta$ have perceptible differences ($\eta$ = 0.9 results
in a more blurred image than when $\eta$ = 0.7 is used). Finally, we studied
the performance of our PRNU spoofing algorithm when a smaller number of
candidate images, $N$, is employed (50%, 10% and 1% of the test set).
Surprisingly, even when only 1% of the test set is used as candidate images,
_i.e_., $N=4$, we observed an average SSR of 99.6% across the three PRNU
estimation schemes. However, the spoofed images are significantly degraded as
they contain some spurious scene details from the candidate images (possibly,
the averaging operation in Step 17 of Algorithm 2 suppresses scene details
more aggressively for a high value of $N$).

Front-Indoor Front-Outdoor
Rear-Indoor Rear-Outdoor

(a)

Front-Indoor Front-Outdoor
Rear-Indoor Rear-Outdoor

(b)

Figure 7: ROC curves for matching PRNU Spoofed images. Here, the source sensor
is Device 1 UNIT II. In this case, the target sensors are: (a) Device 1 UNIT I
(top row) and (b) Device 2 UNIT I (bottom row).

Front-Indoor Front-Outdoor
Rear-Indoor Rear-Outdoor

(a)

Front-Indoor Front-Outdoor
Rear-Indoor Rear-Outdoor

(b)

Figure 8: ROC curves for matching PRNU Spoofed images. Here, the source sensor
is Device 2 UNIT I. In this case, the target sensors are: (a) Device 1 UNIT I
(top row) and (b) Device 1 UNIT II (bottom row).

Next, we report the results for the periocular biometric recognition
experiments. The periocular matching experiments indicate the preservation of
the biometric utility of the images in both PRNU anonymized images and PRNU
spoofed images. The ROC curves corresponding to ‘Original’ and ‘After’
matching experiments are within 1% of each other. Figure 5 presents the ROC
curves for images subjected to PRNU anonymization. Note that Samsung Galaxy S4
results in overall lower periocular matching performance even for the original
images. The cross-matching experiments result in perfect match (100%) for a
majority of the cases barring the Samsung Galaxy S4 sensor. The suppression of
the high frequency components may also result in removal of edges and other
details which can impact the matching performance. For the PRNU spoofing
experiments, we have presented the ROC matching curves for each smartphone
device or unit used in this work in Figures 6, 7 and 8. The matching
experiments show that the perturbation scheme used for PRNU spoofing does not
degrade the biometric recognition performance.

##  5 Conclusion

In this work, we design an algorithm that perturbs a face image acquired using
a smartphone camera such that (a) sensor-specific details pertaining to the
smartphone camera are suppressed (sensor anonymization); (b) the sensor noise
pattern of a different device is incorporated (sensor spoofing); and (c)
biometric matching using the perturbed image is not affected (biometric
utility). We achieve this by applying the Discrete Cosine Transform to images
and further modulating the DCT coefficients to either attain PRNU
anonymization or PRNU spoofing. In contrast to existing methods which involve
computation of sensor reference patterns and exhaustive parameter optimization
[15, 23, 31], the proposed method is simple and can achieve highly promising
results. In our experiments, we considered face (partial and full) images
acquired using the front and rear cameras of different smartphones resulting
in data from a total of 12 camera sensors. Our proposed method results in
successful camera de-identification for images without compromising the
biometric matching performance. An average of $\approx 82.8\%$ reduction in
sensor identification is reported in the case of PRNU anonymization, and an
average spoof success rate of $\approx 99.5\%$ is observed for PRNU spoofing
across the three PRNU estimation schemes evaluated in this work.

Future work will consider data from a larger number of sensors to evaluate the
proposed algorithm. Moreover, the impact of different denoising and PRNU
estimation algorithms will be evaluated to study the robustness of the
proposed method. Finally, we will explore methods to make sensor
identification schemes resilient to de-identification [30].

## References

  * [1] Smartphone pictures poses privacy risks.  https://www.youtube.com/watch?v=N2vARzvWxwY&feature=youtu.be.  [Online accessed: 18th April 2019].
  * [2] A. Agarwal, R. Keshari, M. Wadhwa, M. Vijh, C. Parmar, R. Singh, and M. Vatsa.  Iris sensor identification in multi-camera environment.  Information Fusion, 45:333 – 345, 2019.
  * [3] M. Al-Ani and F. Khelifi.  On the SPN estimation in image forensics: A systematic empirical evaluation.  IEEE Transactions on Information Forensics and Security, 12(5):1067–1081, May 2017.
  * [4] S. Banerjee, V. Mirjalili, and A. Ross.  Spoofing PRNU patterns of iris sensors while preserving iris recognition.  In 5th IEEE International Conference on Identity, Security, Behavior and Analysis (ISBA), January 2019.
  * [5] S. Banerjee and A. Ross.  Impact of photometric transformations on PRNU estimation schemes: A case study using near infrared ocular images.  In International Workshop on Biometrics and Forensics (IWBF), pages 1–8, June 2018.
  * [6] S. Bayram, H. Sencar, N. Memon, and I. Avcibas.  Source camera identification based on CFA interpolation.  In IEEE International Conference on Image Processing, volume 3, pages III–69–72, Sept 2005.
  * [7] G. Bhupendra and M. Tiwari.  Improving source camera identification performance using DCT based image frequency components dependent sensor pattern noise extraction method.  Digital Investigation, 03 2018.
  * [8] Z. Boulkenafet, J. Komulainen, L. Li, X. Feng, and A. Hadid.  OULU-NPU: A mobile face presentation attack database with real-world variations.  IEEE International Conference on Automatic Face and Gesture Recognition, 2017.
  * [9] M. Chen, J. Fridrich, M. Goljan, and J. Lukas.  Determining image origin and integrity using sensor noise.  IEEE Transactions on Information Forensics and Security, 3(1):74–90, March 2008.
  * [10] J. Deng, W. Dong, R. Socher, L. Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, June 2009.
  * [11] A. E. Dirik and A. Karaküçük.  Forensic use of photo response non-uniformity of imaging sensors and a counter method.  Opt. Express, 22(1):470–482, Jan 2014.
  * [12] A. E. Dirik, H. T. Sencar, and N. Memon.  Analysis of seam-carving-based anonymization of images against PRNU noise pattern-based source attribution.  IEEE Transactions on Information Forensics and Security, 9(12):2277–2290, Dec 2014.
  * [13] T. Filler, J. Fridrich, and M. Goljan.  Using sensor pattern noise for camera model identification.  15th IEEE International Conference on Image Processing, pages 1296–1299, Oct 2008.
  * [14] C. Galdi, M. Nappi, and J. L. Dugelay.  Multimodal authentication on smartphones: Combining iris and sensor recognition for a double check of user identity.  Pattern Recognition Letters, 3:34–40, 2015.
  * [15] M. Goljan, J. Fridrich, and M. Chen.  Defending against fingerprint-copy attack in sensor-based camera identification.  IEEE Transactions on Information Forensics and Security, 6(1):227–236, March 2011.
  * [16] K. He, X. Zhang, S. Ren, and J. Sun.  Deep residual learning for image recognition.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, June 2016.
  * [17] K. Hernandez-Diaz, F. Alonso-Fernandez, and J. Bigun.  Periocular recognition using CNN features off-the-shelf.  In Proceedings of the 17th International Conference of the Biometrics Special Interest Group (BIOSIG), pages 1–5, September 2018.
  * [18] N. Kalka, N. Bartlow, B. Cukic, and A. Ross.  A preliminary study on identifying sensors from iris images.  In IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 50–56, June 2015.
  * [19] X. Kang, Y. Li, Z. Qu, and J. Huang.  Enhancing source camera identification performance with a camera reference phase sensor pattern noise.  IEEE Transactions on Information Forensics and Security, 7(2):393–402, April 2012.
  * [20] C. Kauba, L. Debiasi, and A. Uhl.  Identifying the origin of iris images based on fusion of local image descriptors and PRNU based techniques.  In 3rd International Joint Conference on Biometrics (IJCB), October 2017.
  * [21] J. Komulainen, Z. Boulkenafet, and Z. Akhtar.  Review of face presentation attack detection competitions.  In S. Marcel, M. S. Nixon, J. Fierrez, and N. Evans, editors, Handbook of Biometric Anti-Spoofing: Presentation Attack Detection, pages 291–317. Springer International Publishing, 2019.
  * [22] C. T. Li.  Source camera identification using enhanced sensor pattern noise.  IEEE Transactions on Information Forensics and Security, 5(2):280–287, June 2010.
  * [23] C.-T. Li, C.-Y. Chang, and Y. Li.  On the repudiability of device identification and image integrity verification using sensor pattern noise.  In D. Weerasinghe, editor, Information Security and Digital Forensics, pages 19–25. Springer Berlin Heidelberg, 2010.
  * [24] J. Lukas, J. Fridrich, and M. Goljan.  Digital camera identification from sensor pattern noise.  IEEE Transactions on Information Forensics and Security, 1(2):205–214, June 2006.
  * [25] F. Marra, G. Poggi, C. Sansone, and L. Verdoliva.  A deep learning approach for iris sensor model identification.  Pattern Recognition Letters, 2017.
  * [26] M. D. Marsico, M. Nappi, F. Narducci, and H. Proença.  Insights into the results of MICHE I - mobile iris challenge evaluation.  Pattern Recognition, 74:286 – 304, 2018.
  * [27] V. Mirjalili, S. Raschka, and A. Ross.  Gender privacy: An ensemble of semi adversarial networks for confounding arbitrary gender classifiers.  In 9th IEEE International Conference on Biometrics Theory, Applications and Systems (BTAS), pages 1–10, Oct 2018.
  * [28] S. Nagaraja, P. Schaffer, and D. Aouada.  Who clicks there!: Anonymising the photographer in a camera saturated society.  In Proceedings of the 10th Annual ACM Workshop on Privacy in the Electronic Society, pages 13–22, New York, NY, USA, 2011.
  * [29] K. Rosenfeld, T. Sencar, and N. Memon.  A study of the robustness of PRNU-based camera identification.  In SPIE Conference on Media Forensics and Security, pages 90–93, 2009.
  * [30] A. Ross, S. Banerjee, C. Chen, A. Chowdhury, V. Mirjalili, R. Sharma, T. Swearingen, and S. Yadav.  Some research problems in biometrics: The future beckons.  In 12th IAPR International Conference on Biometrics (ICB), Crete, Greece, June 2019.
  * [31] A. Uhl and Y. Höller.  Iris-sensor authentication using camera PRNU fingerprints.  In 5th IAPR International Conference on Biometrics (ICB), pages 230–237, March 2012.
  * [32] L. J. G. Villalba, A. L. S. Orozco, J. R. Corripio, and J. Hernandez-Castro.  A PRNU-based counter-forensic method to manipulate smartphone image source identification techniques.  Future Generation Computer Systems, 76:418 – 427, 2017.
  * [33] G. K. Wallace.  The JPEG still picture compression standard.  IEEE Transactions on Consumer Electronics, 38(1):xviii–xxxiv, Feb 1992.