# Beyond the Request: Harnessing HTTP Response Headers for Cross-Browser Web
Tracker Classification in an Imbalanced Setting

Wolf Rieder<EMAIL_ADDRESS>0009-0001-4932-9814 Technische Universität
BerlinErnst-Reuter-Platz 7BerlinGermany10587 ,  Philip Raschke
<EMAIL_ADDRESS>0000-0002-6738-7137 Technische Universität
BerlinErnst-Reuter-Platz 7BerlinGermany10587 and  Thomas Cory  cory@tu-
berlin.de 2222–3333-4444-5555 Technische Universität BerlinErnst-Reuter-Platz
7BerlinGermany10587

###### Abstract.

The World Wide Web’s connectivity is greatly attributed to the HTTP protocol,
with HTTP messages offering informative header fields that appeal to
disciplines like web security and privacy, especially concerning web tracking.
Despite existing research employing HTTP/S request messages to identify web
trackers, HTTP/S response headers are often overlooked. This study endeavors
to design effective machine learning classifiers for web tracker detection
using HTTP/S response headers. Data from the Chrome, Firefox, and Brave
browsers, obtained through the traffic monitoring browser extension T.EX,
serves as our data set. Eleven supervised models were trained on Chrome data
and tested across all browsers. The results demonstrated high accuracy,
F1-score, precision, recall, and minimal log-loss error for Chrome and
Firefox, but subpar performance on Brave, potentially due to its distinct data
distribution and feature set. The research suggests that these classifiers are
viable for detecting web trackers in Chrome and Firefox. However, real-world
application testing remains pending, and the distinction between tracker types
and broader label sources could be explored in future studies.

Web Privacy, Web Tracking, Web Privacy Measurement, HTTP/S Response Header,
Machine Learning

††journalyear: YYYY††journalvolume: YYYY††journalnumber: X††doi:
XXXXXXX.XXXXXXX

##  1\. Introduction

The rapid evolution of the Web has cemented its role as an integral component
of modern life. Through desktop and mobile browsers, users access a vast array
of information and services. However, every interaction, from content
consumption to mere site visitation, reveals information about the habits and
preferences of users, which in turn has implications for personal privacy and
online security.

These privacy concerns are highlighted by the ubiquity of behavioural
targeting. By analyzing a user’s browsing history, a detailed profile can be
constructed, facilitating the delivery of tailored advertisements and content.
This profiling is made possible by web tracking that employs sophisticated
techniques to track users across the Web. Given the increasing concerns about
such practices (Melicher et al., 2016; Chanchary and Chiasson, 2015), there is
a growing emphasis on research in web privacy.

An established countermeasure against unsolicited HTTP communications is the
use of filter lists (Traverso et al., 2017), such as EasyList (eas, 2023a) and
EasyPrivacy (eas, 2023b). However, their dependency on manual curation makes
them vulnerable to oversights and rapid obsolescence (Hashmi et al., 2019;
Alrizah et al., 2019; Fouad et al., 2018). Moreover, some trackers employ
strategies like domain masking to evade these lists (Chen et al., 2021).

In an attempt to mitigate these shortcomings, recent efforts have explored the
potential of Machine Learning (ML) in this domain. Various classifiers, some
achieving accuracy metrics in the 80th percentile (Cozza et al., 2020; Iqbal
et al., 2021), have been developed using diverse data sources such as HTTP
headers and JavaScript calls. Yet, a majority of these focused primarily on
HTTP requests, potentially neglecting other informative data streams.
Additionally, the predominant reliance on Firefox for data collection may
limit the comprehensiveness of these studies.

Notably, there seems to be a gap in literature regarding the use of HTTP
response headers as the sole data source for tracker classification across
diverse browsers, despite studies in the related field of online security
highlighting the efficacy of this approach (McGahagan et al., 2019). This
study intends to bridge this gap. We strive to develop classifiers adept at
web tracker detection, while confronting challenges such as data sparsity,
high dimensionality, and class imbalance.

Our key contributions are:

  1. (1)

We conduct a rigorous study of HTTP response headers across multiple browsers,
focusing on their potential for web tracker detection within the Tranco top
10k websites. Furthermore, we evaluate dataset similarities to discern
differences in HTTP response headers across the assessed browsers.

  2. (2)

We present a semi-automated ML pipeline which facilitates the training of
eleven classifiers tailed for web tracker detection. Our findings indicate
that selected tree and gradient boosting models, when applied to a subset of
binary-encoded HTTP/S headers, are capable of achieving ROC-AUC, AUPCR, and F1
scores exceeding $0.9$.

  3. (3)

We conduct a comprehensive and multi-faceted evaluation of these classifiers,
which includes longitudinal studies to gauge potential concept-drift and
evaluates the cross-browser performance of classifiers trained solely on
traffic data collected within the Chrome browser. Notably, our evaluation
incorporates thirteen metrics, offering a holistic view of the binary
classification challenge.

  4. (4)

We assess calibration effects on final classifier performance — a
consideration seldom addressed in prior research. This allows us to discern if
models trained solely on data from one browser can be effectively deployed on
data from alternate browsers without the necessity for retraining.

The remainder of this paper is structured as follows. Section 2 reviews past
research in the field of web tracker detection. Section 3 introduced
definitions, our research questions, and the analyzed data sets. Section 4
explores HTTP/S response headers from Chrome, Firefox, and Brave data sets,
providing general findings and identifies potential challenges for the ML
pipeline. Section 5 describes the design and implementation of the ML
pipeline, including selected models, metrics, and baselines. Section 6
presents the evaluation of the trained classifiers. The penultimate Section 7
discusses our results and limitations. Section 8 concludes the paper.

##  2\. Related Work

This section provides an overview of existing research on web tracker
detection based on machine learning. To highlight how our proposed classifier
differs from prior work, we focus this overview on classifiers that rely
solely on HTTP protocol information. For the sake of completeness, however, we
summarize other approaches at the end of the section.

###  2.1. Classification Based on HTTP Information

Approaches to the detection web trackers with the support of machine learning
date back to the early 2000s. In 2010, Yamada et al. (Yamada et al., 2010)
were the first to use temporal link analysis to build a graph based on hosts
related to HTTP requests from the traffic generated by corporate network.
Their classifier considers temporal information such as the visiting time of a
specific website or host. Yamada et al. consider this a discriminating
criterion based on the assumption that users spend less time on websites of
web trackers, as trackers generally appear as third parties that are contacted
briefly in the context of other websites rather than first parties that users
actively visit.

In 2013, Bau et al. (Bau et al., 2013) described and discussed the usefulness
of machine learning as an effective counter to web tracking. They outlined the
problems with manually curated filter lists - problems that still hold true
today, such as the reliance on manual curation and the inflexibility of static
lists. Beyond building an Elastic Net model focused on traces of content
extraction in the Document Object Model and HTTP headers to show the
advantages of automated approaches, Bau et al. laid down requirements for
future machine learning models as well as problems facing researchers in this
domain.

A year later, Bhagavatula et al. (Bhagavatula et al., 2014) tried to identify
ad-related URLs, a topic closely connected to the web tracking detection
problem. To this end, they crawled the Alexa top 500 websites to compile an
extensive dataset with six distinct feature sets based on the URLs themselves
as well as additional metadata. Their analysis approach is based predominantly
on Support Vector Machines (SVMs) with different kernels, achieving average
accuracy and precision scores above 89%. Like Bau et al. before them,
Bhagavatula et al. recognized the need for machine learning models to replace
filter lists and put forward additional requirements for production-ready
machine learning approaches, such as optimized training times.

In 2015, Gugelmann et al. (Gugelmann et al., 2015) proposed an alternative
approach by deriving several statistical features from HTTP traffic, e.g. up-
and download volume. In contrast to prior studies, they thoroughly described
their model-building process and emphasized their feature engineering. Their
feature selection process included the use of Pearson’s correlation
coefficient as well as the consideration of a feature’s entropy. In addition,
they employed an unspecified classification tree, among other classifiers, to
find an optimal Naïve Bayes model that achieved a precision of 83% and was
able to correctly identify previously unseen trackers.

In the same year, Li et al. (Li et al., 2015) proposed TrackAdvisor, a system
to classify HTTP requests based on the collective statistics of cookies
contained therein. Using FourthParty, they captured 563,031 requests and
identified 22,270 cookies that were sent to third parties. Rather than relying
solely on filter lists, they used a multi-step process with an emphasis on
manual inspection to label the captured data. Notably, the training and test
sets created from the captured data comprised only 500 requests each and had a
balanced class distribution, which contrasts with the distribution found in
the original data. Nevertheless, Li et al. used these sets to train a SVM that
ultimately achieved a precision of 99.4% and a recall of 100%. Like Gugelmann
et al., they point out the usefulness of the HTTP Referrer header for third-
party tracker detection.

In 2016, Dudykevych and Nechypor (Dudykevych and Nechypor, 2016) also used
cookies as the basis for their classifiers. In contrast to Li et al., however,
they developed their own crawler, had a larger data set of 550 000 cookies,
and developed a different feature set. The latter is only described briefly,
but information from the cookies themselves and the HTTP requests were used as
features. It can be assumed that the features are numerical and categorical,
as the authors used Principal Component Analysis and correlation, which
require numerical data, as well as binary encoding of categorical features.
The resulting classification accuracies ranged from 82.01% for Naïve Bayes to
95.43% for the regularized logistic regression method.

The analysis of HTTP traffic for and with machine learning models is also an
important topic in web security research, albeit with a different target (i.e.
label) than web privacy research. In a 2019 study, McGahagan et al. (McGahagan
et al., 2019) analyzed response headers from 45 874 websites and extracted 672
features, resulting in a final feature vector with 22 properties. In addition,
they tested whether over- and undersampling had any effects on the model’s
performance. The coverage of metrics, models, different training and tuning
combinations, and extensive feature engineering documentation can be
considered standard among machine-learning practitioners but is somewhat rare
among prior related research. An important contribution of this paper is the
in-depth presentation of the usefulness of HTTP response headers for automated
ML models.

Unlike most papers discussed in this section, which have used data from the
Firefox browser to test the performance of supervised learning models, Din et
al. (Din et al., 2020) used Deep learning (DL) techniques for Brave and other
Chromium-based browsers. Although their approach is geared towards the image-
based detection of advertising in websites rather than web tracking, their
incorporation of multiple browsers is highly relevant for models that classify
web trackers.

Cozza et al. (Cozza et al., 2020) developed a system that not only classifies
HTTP requests but also automatically updates the filter list. Although this
study is not the first to detect previously undocumented trackers using
machine learning techniques, the deployment of a tracker-detecting browser
extension that combines a filter list with a machine learning models and
updates its ground truth marks an important step forward in tracker detection
technology.

Guarino et al. (Guarino. et al., 2020) built upon prior research by Gugelmann
et al. and Li et al. by training multiple model based on combinations of
features used by these studies, including different HTTP headers and the ratio
of upstream and downstream data transmissions. Their top performer is a Random
Forest model with an average accuracy of 92%. It should be noted, however,
that this study is based on a relatively small dataset of just 1000
observations. Nevertheless, the authors astutely emphasize that content
delivery networks (CDNs) are a primary cause of false positives in the domain
of web tracker detection due to similarities in traffic patterns between CDNs
and third-party trackers.

Taking advantage of the observation that HTTP-based communication between
clients and servers often comprises cascades of multiple HTTP messages, Iqbal
et al. (Iqbal et al., 2022) conducted an extensive study that uses entire
request chains rather than individual requests as the basis for classifying
web tracking. Notably, they were among the first to consider responses as well
as requests by developing three distinct feature sets: the first is based on
the entire request chain, the second relies on information pertaining to
individual requests and the third considers response headers and metadata. By
training a Random Forest model on five distinct datasets with different
configurations from separate crawls, Iqbal et al. achieved accuracies above
98%. However, if the model was trained on only one crawl and tested on the
others, the accuracy dropped by up to 14%, highlighting the difficulties of
applying models to datasets with varying configurations, e.g. data collected
across different browsers.

###  2.2. Other Detection Approaches

Beyond the HTTP-focused approaches outlined above, a variety of classification
methods utilizing different data sources for the extraction of distinctive
criteria have been proposed and evaluated. These include, classifiers focused
on JavaScript code and API access events that aim to identify device
fingerprinting techniques, pure URL-based classifiers, CNAME cloaking
detectors, and graph-based classifiers that leverage the topology of the Web
to identify third-party tracking in particular. In the following paragraphs,
we will briefly highlight notable research on these alternative approaches.

Focusing on JavaScript API calls, Wu et al. (Wu et al., 2016) developed
DMTTrackerDetector, a system using decision trees in which observed API calls
serve as features. The system, comprising a 717-dimensionial feature vector,
achieved 97.8% accuracy and addressed obfuscation issues, showing the distinct
JavaScript API usage by trackers and non-trackers.

Bahrami et al. (Bahrami et al., 2021) refined the scope of their analysis by
focusing on device fingerprinting rather than API calls in general. Their
approach combines graph analysis with supervised and unsupervised learning to
detect web trackers in historical JavaScript file data collected between 2010
and 2019 by creating a graph that includes temporal aspects and the output of
AST-based keyword extraction. Their method, which includes nine graph-related
features and graph embedding, as well as a clustering model, yielded
accuracies up to 89.13% and identified distinct clusters for fingerprinting
and functional APIs, highlighting the viability of such an approach.

Kalavri et al. (Kalavri et al., 2016) also proposed a graph-based approach,
using neighborhood analysis and label propagation on a bipartite graph
representing connections between first-party websites and third-party services
to detect web trackers, achieving over 97% accuracy. They analyzed six months
of data, finding that most trackers are closely linked, with precision scores
ranging from 64 to 92%.

Combining these graph-based approaches, Castell-Uroz et al. (Castell-Uroz et
al., 2021) proposed a tripartite network graph to identify third-party
tracking resources and their origin domains within first-party websites. By
applying this approach to a dataset comprising over 90 thousand domains, they
were able to detected new trackers through their hash value popularity and
dirt level, finding that higher values correlated with the likelihood of a
given resource to be tracking-related.

Conversely, Metwalley et al. (Metwalley et al., 2015) developed an
unsupervised algorithm analyzing URL queries and HTTP request headers for
identifying trackers. Their method, applied to 200 Alexa-ranked websites,
identified 34 new trackers and successfully supported the analysis of cookie-
matching practices, though it had limitations in detecting all known trackers.

Finally, Yu et al. (Yu et al., 2016) proposed a browser extension using
k-Anonymity principles to classify third-party trackers based on data sent to
users. Leveraging data from 200,000 German users of the Cliqz browser, their
system outperformed traditional blockers like Disconnect, which had a 38.8%
false positive rate.

###  2.3. Summary

Despite over a decade of dedicated research, web tracking is as prevalent as
ever, new tracking methods are developed, and the most widespread form of
protection, namely blockers and their filter lists, exhibit several key
problems. Filter lists, which are also used as the ground truth for labeling
in many studies related to web privacy, are often maintained by a small group
of users, contain irrelevant rules, are rather slow to add new rules, and can
simply be circumvented by changing URLs. In response, researchers have
developed a wealth of automated approaches, utilizing HTTP messages,
JavaScript, or the HTML files of websites themselves to classify third parties
as trackers.

A crucial component of this ongoing research effort is the availability and
quality of data on which to train and evaluate machine learning approaches.

Despite this, HTTP response headers are often neglected in this regard. There
are cases where authors document the collection of HTTP responses (Acar et
al., 2013) but not their actual use. In other cases, the data is collected,
but only used in support of other approaches such as the classification of
cookies (Lerner et al., 2016; Englehardt et al., 2015).

Additionally, the majority of studies are based on datasets collected using
just one browser, most commonly Firefox. Cassel et al. (Cassel et al., 2022)
discuss the need for cross-browser examinations to avoid biased datasets. This
is supported by research that shows that characteristics such as JavaScript
API calls differ between browsers (Yang and Yue, 2020).

In summary, our review of related research has shown that, although the
application of machine learning to the field of web tracking identification
and classification has been investigated in numerous studies, a research gap
still exists with regard to cross-browser performance of classifiers and the
usefulness of HTTP response data for classification purposes.

##  3\. Approach

In this study, we address the research gap identified in our review of related
research by evaluating the (cross-browser) performance of commonly used
machine learning classifiers that are trained on HTTP/S response headers. In
this section, we discuss our underlying research questions and present the
approach we used to address them.

###  3.1. Preliminaries

A core principle of automated web tracker detection is the definition of what
constitutes a tracker. The focus hereby lies on the tracking entities and not
on the activity of tracking. Our definition of what constitutes a tracker or
non-tracker is coupled to our ground truth to label each dataset. We define
them as follows:

$\displaystyle H$ $\displaystyle:\text{Set of all HTTP/S responses' remote
hostnames.}$ $\displaystyle R$ $\displaystyle:\text{Set of rules derived from
{EasyList} and {EasyPrivacy}.}$ $\displaystyle\text{match}(h,r)$
$\displaystyle:\begin{cases}1&\text{if }h\text{ matches rule }r\\\
0&\text{otherwise}\end{cases}$ $\displaystyle T$ $\displaystyle=\\{h\in
H\mid\exists r\in R:\text{match}(h,r)=1\\}$ $\displaystyle NT$
$\displaystyle=\\{h\in H\mid\forall r\in R:\text{match}(h,r)=0\\}$

In other words: if the remote hostname (i.e. the sender) of a response matches
at least one rule in our aggregate list compiled from the EasyList and
EasyPrivacy filter lists, we label the response as a tracker.

For the sake of simplicity, we will use the terms header and response header
interchangeably to refer to HTTP/S response headers throughout the remainder
of the paper, unless explicitly stated otherwise.

###  3.2. Research Questions

The main goal of this study is to analyze whether response headers are useful
for automated web tracker detection. Based on the ML approaches of prior
research and the observed areas to further extend our empirical understanding
and detection of web trackers, we formulate the following research questions
that guide us throughout this study:

  1. (1)

What are the differences and similarities between web trackers and non-
trackers with regard to their response headers?

  2. (2)

Can trained classifiers detect web trackers with only response header features
while achieving an F1-Score higher than 0.9 in an imbalanced setting?

  3. (3)

How large is a classifier’s performance degradation when it is trained on one
browser’s HTTP traffic data and then applied to other browsers?

To answer these research questions, we develop a semi-automated ML pipeline
that transforms HTTP traffic datasets collected with T.EX (Raschke et al.,
2019) and uses it to train several supervised binary classifiers. We use
reliable data that was compiled by prior research to facilitate future
comparisons between new classifiers.

To test the robustness and reliability of our classifiers, we train them on
HTTP data from Chrome and apply them to different datasets with a variety of
configurations, including other browsers (latitudinal comparison) as well as
different time frames within the same browser (longitudinal comparison).
Furthermore, past research observed an uneven distribution of trackers and
non-trackers, which is why we train our classifiers with an imbalanced
distribution, to assess how effective they perform with the likely true
distribution.

| $\text{Chrome}_{22}$ |  | $\text{Firefox}_{22}$ |  | $\text{Brave}_{22}$ |  | $\text{Chrome}_{23}$
---|---|---|---|---|---|---|---
| T | NT |  | T | NT |  | T | NT |  | T | NT
#Responses | 256 202 | 602 516 |  | 309 391 | 526 865 |  | 2969 | 561 694 |  | 206 747 | 596 066
#Responses in % | $\approx 0.3$ | $\approx 0.7$ |  | $\approx 0.37$ | $\approx 0.63$ |  | $\approx 0.5$ | $\approx 99.5$ |  | $\approx 0.26$ | $\approx 0.74$
#Unique headers in total | 2213 | 7577 |  | 2516 | 7322 |  | 237 | 7348 |  | 2164 | 7398
Unique Headers per response ($Q1,\tilde{\mu},Q3$) | 10,13,16 | 12,15,19 |  | 12,15,18 | 13,16,19 |  | 13,18,20 | 12,15,19 |  | 10,13,16 | 13,15,19
Table 1. Overview of the collected data sets from (zen, 2023), split by data
sets as well as tracker (T) and non-trackers (NT). Figure 1. This figure
illustrates the usage headers across four different browsers. The left plot
(a) displays the frequency distribution of headers for each browser, showing
how often each header is present in the dataset. The right plot (b) presents a
boxen plot of the same data, which provides a more detailed view of the
distribution, revealing variations and trends among the browsers.

###  3.3. Data Collection

We used existing data sets, containing labeled HTTP response headers by
EasyList and EasyPrivacy. Following suggested good practices by (Demir et al.,
2022), the crawl should fulfill three citeria: (i) repeated crawls across
multiple browsers, namely Chrome, Firefox, and Brave, (ii) the documentation
of the geographical region as well as the technical environment, and (iii) web
sites from the Tranco ranking as it proved to be less biased than the Alexa
ranking (Pochat et al., 2018). A previously crawled data set by Raschke using
T.EX - The Transparency EXtension, fulfills all three criteria and is
described as follows:

> Multiple simultaneous and stateful crawls (6 x Chrome, 6 x Brave, 6 x
> Firefox) of the Tranco top 10K websites (as of 16th of May 2022) were
> performed on the 12th of August 2022 with T.EX […]. Measurements had been
> carried out on 18 Amazon Web Services instances (c5.large) running Windows
> Server. All instances were launched in Frankfurt, Germany (eu-central-1).
>
> After completing the crawls, each instance’s extension storage was extracted
> […]. Note: 2 of the 6 Firefox instances crashed during the crawl. Therefore,
> only 4 Firefox datasets are available (zen, 2023).

To assess a potential concept drift and therefore model degradation over time,
we crawled one more data set from 2023 for Chrome. We followed a similar
methodology as described in (zen, 2023), using T.EX with Chrome to explore the
top 10K websites from the Tranco ranking as of March 29th, 2023.

Our reasons to choose only Chrome for a new data collection are as follows:
Chrome is the most widely used in the real world with a market share of 66.13%
on desktops, whereas Firefox has 7.1% (sta, 2023) and Brave has less than 1%
(kin, 2023). Based on this distribution, the Chrome data set serves as the
foundation for training to reflect and reach the majority, i.e. using data
that most users encounter themselves. In addition, web privacy research often
relies on a simulated Firefox browser with Selenium due to the use of OpenWPM,
leaving space for other browsers of interest. In addition, it helps us to
answer the question whether a model based on Chrome data can be applied to
Firefox or Brave while achieving high performance metrics?.

##  4\. Initial Response Header Analysis

To get a better understanding of the data underlying our analysis before
designing our machine learning pipeline, we conduct an initial analysis of the
data, the results of which we present in this section. We first explore
response headers across all three browsers to find empirical evidence of their
potential relation to trackers. Furthermore, we identify the characteristics
of these datasets to apply the appropriate methods in our ML pipeline. We use
these insights to develop our classifiers to evaluate the effectiveness of
response headers for web tracker detection.

###  4.1. Descriptive Analysis of Headers

A descriptive analysis is useful to gain an understanding of the data
structure and the distribution of classes and features within the dataset.
Table 1 summarizes descriptive metrics for each dataset. We start with the
datasets $\text{Chrome}_{22}$, $\text{Chrome}_{23}$, and
$\text{Firefox}_{22}$. All three datasets show similar results for the
distribution between trackers and non-trackers, the number of unique headers
in total and per response. For example, we can observe a small quantitative
change in the number of unique headers between $\text{Chrome}_{22}$ and
$\text{Chrome}_{23}$. $\text{Chrome}_{22}$ lists a total of 7577 unique
headers, while $\text{Chrome}_{23}$ has a slightly lower total of 7398 for
non-trackers. When comparing trackers, $\text{Chrome}_{22}$ had 2213 unique
headers, which is nearly the same as $\text{Chrome}_{23}$’s count of 2164. On
average, non-trackers have more unique headers as well as a higher median
$\tilde{\mu}$ and interquartile range than trackers.

The $\text{Brave}_{22}$ browser with its standard configuration is the only
one that has a low number of trackers and unique headers in tracking-related
response messages. Interestingly, the $\tilde{\mu}$ equals 18 for trackers and
15 for non-trackers, which represents the only instance where trackers had a
higher number. Although the dataset has almost only non-trackers, the number
of unique headers is similar to the other datasets.

Figure 1 shows a frequency plot, which is a distribution graph that shows how
often each header appears in the dataset. It uses a logarithmic scale to
manage the wide range of frequencies, which is a common approach when data
spans several orders of magnitude. This plot typically demonstrates a power-
law distribution, characterized by a steep drop-off where a small number of
headers are occurring frequently, while the majority are rare. This type of
distribution is common in HTTP data, where few headers dominate, and many
others have a long tail of low frequency. This sparseness indicates that while
there are many different headers, only a few are consistently used across
different websites and by different browsers.

The boxen plot graph of Figure 1 offers a visual representation of the
distribution of header frequencies across different browsers and versions. The
spread and median values suggest that the distribution of headers is not
uniform across browsers, with some browsers having a higher tendency to use
certain headers than others. Furthermore, if the median line in the boxen plot
is closer to the bottom of the box, it suggests a right-skewed distribution,
aligning with what the frequency plot shows — a few headers with high
frequency and many with low frequency.

Figure 2. The Venn diagram reveals that a core set of 5672 headers are common
across Chrome, Firefox, and Brave browsers across two years. Unique headers
are present in each browser, with $\text{Chrome}_{23}$ having the highest
number of unique headers, which may reflect browser-specific features or
experimental headers. Firefox had a higher header similarity with Chrome than
with Brave.

Figure 2 represents a Venn diagram and provides a direct comparison of
response headers between different browser versions, highlighting data and
feature similarity. The intersection numbers, such as the 5672 headers common
to all browsers, suggest a significant overlap in features across different
platforms. However, the presence of headers unique to each browser or shared
by only two browsers (e.g., 1427 unique to $\text{Chrome}_{22}$ and 154 unique
to $\text{Brave}_{22}$) indicates that while there is a core set of features,
there is also a notable amount of variation which can be critical for web
tracker detection.

The Empirical Cumulative Distribution Function plot for $\text{Chrome}_{22}$
and $\text{Chrome}_{23}$ in Figure 3 row (A) show that the Content-Length
values for non-trackers are generally higher than those for trackers, which
suggests that non-trackers tend to send larger responses. This is consistent
across both browser versions, indicating that while browsers evolve, certain
characteristics like the smaller size of tracker responses remain consistent.
The second row (B) plots the feature similarity for the X-XSS-Protection
header, showing that header values can be very similar between the datasets,
i.e., the training and test sets. It further supports our observation that the
datasets are similar and that trackers and non-trackers have differences
across the usage of HTTP header fields and values. Although we present only
few selected examples here, they should highlight the general challenges that
come with response messages datasets.

Figure 3. Row (A) presents the ECDFs for both Chrome datasets for the Content-
Length header values. We set a cut-off value at 10 000 to highlight the core
observation and more than half of the responses have a value are below this
threshold – $\tilde{\mu}_{T22}=62$, $\tilde{\mu}_{NT22}=8068$ and
$\tilde{\mu}_{T23}=45$, $\tilde{\mu}_{NT23}=8021.5$. Row (B) shows how similar
the values across Chrome, Firefox, and Brave are for the X-XSS-Protection
header.

Additional characteristics of the datasets include the high amount of missing
values per header. One explanation might be that servers do not need to use
every header field that exists, and it is common practice to use developer
headers that are not part of the standard, i.e. they are not registered in the
Request for Comments 4229111RFC 4229, https://www.ietf.org/rfc/rfc4229.txt,
accessed: November 30, 2023. There are multiple headers that contain mostly
unique values, e.g., Date. Lastly, according to the used filter lists, most
HTTP response messages are not related to any tracking activities. This
entails a class imbalance to be handled carefully to derive meaningful results
from the final models.

##  5\. Methodology

Based on our findings from the data exploration in Section 4, we propose our
methodology to assess the effectiveness of classifiers based on response
headers.

###  5.1. Problem Formulation

We consider the problem of binary classification, where we aim to predict the
presence of web trackers based on response headers. The headers, represented
by the feature vector $x\in\mathbb{R}^{d}$, are nominal and can be both
discrete and continuous variables. These variables are derived from HTTP
headers of four different browser types, denoted as the set
$B=\\{\text{Chrome},\text{Firefox},\text{Brave}\\}$. For this study, we
specifically consider versions
$\text{Chrome}_{22},\text{Chrome}_{23},\allowbreak\text{Firefox}_{22},\text{Brave}_{22}$.

Given a training set
$\mathcal{D}_{\text{train}}=\\{(x_{1}y_{1}),\ldots,(x_{n},y_{n})\\}$, where
$x_{i}$ is a feature vector of the $i$-th HTTP header and $y_{i}\in\\{0,1\\}$
represents the associated class label (with ’1’ indicating the presence of a
web tracker), our goal is to train a set of classifiers
$f_{j}:\mathbb{R}^{d}\rightarrow\\{0,1\\}$ for $j\in M$ that minimizes the
empirical risk:

(1) $R(f_{i})=\frac{1}{N}\sum_{i=1}^{N}L(y_{i},f_{i}(x_{i})),\quad
j\in\\{1,\ldots,M\\},$

where $L(y_{i},f_{i}(x_{i}))$ is the loss incurred by predicting
$f_{i}(x_{i})$ when the true label is $y_{i}$. The specific form of the loss
function $L(\cdot)$ will be dependent on the particular classifier $f_{j}$
used.

We consider datasets represented by $D_{s}$, where $s$ can pertain to
training, testing, validation, or calibration subsets of the data. We observe
several challenges from our data exploration:

  1. (1)

High Cardinality: The feature vectors $x_{i}$ in our datasets $D_{s}$ can
exhibit high cardinality for certain features. This problem can pose
challenges in terms of model training and encoding.

  2. (2)

Sparseness: The feature vectors $x_{i}$ in our datasets $D_{s}$ are
represented as sparse vectors, with each vector defined as:

$x_{i}=\\{v_{1},v_{2},\ldots,v_{d}\\}$

Here, a significant proportion of the entries $v_{j}$ within $x_{i}$ are
either zero or represent missing data, leading to the overall sparseness of
$D_{s}$. Such sparseness can introduce computational complexities and may
challenge the performance of certain machine learning algorithms not optimized
for sparse data.

  3. (3)

Class Imbalance: For datasets collected from a given browser $B$ and
represented as $D_{s}$, the class distribution can exhibit imbalance. The
majority class is denoted by $\alpha_{B}$, and the minority class is
represented by $\beta_{B}$, satisfying $\alpha_{B}+\beta_{B}=1$. The degree of
this imbalance can vary based on the browser as shown in Table 1, with some
showing a more pronounced disparity between $\alpha_{B}\text{ and }\beta_{B}$.
Such imbalance can introduce bias to models, favoring the majority class and
potentially undermining the accurate prediction of the minority class.

  4. (4)

High Dimensionality: With a high dimensionality $d$ and a high number of
observations $n$ per dataset, the classification is susceptible to the Curse
of Dimensionality, potentially leading to overfitting and necessitating
intricate dimensionality reduction techniques. We have to apply a
dimensionality reduction process that achieves a reduced feature space
$X_{i}^{\prime}\subseteq X$, entailing a reasonable training time.

  5. (5)

Data Similarity: The datasets $\mathcal{D}_{s}$ collected across different
browsers can exhibit pronounced similarities in their feature distributions
and values. Given the high dimensionality of our feature space, exhaustive
pairwise comparisons can be computationally expensive and time-consuming,
highlighting the challenge of accurately gauging data similarities in such a
complex space.

###  5.2. Machine Learning Pipeline

Our machine learning pipeline, which we present in the following, addresses
the previously stated problems for the classification of web trackers. Many
different approaches have been applied in order to uncover web trackers.
Another closely related field to web privacy is web security research.
Although they seek to detect different entities, both have the exploitation of
HTTP data in common. Typically, a dataset goes through extensive data
processing and feature engineering to find useful features, which includes a
thorough analysis of the values within each feature. Despite this common
conception, Laughter et al. (Laughter et al., 2021) were interested if
features that merely represented the presence of an HTTP header in a binary
fashion were usable in a classifier to detect malicious HTTP requests based on
HTTP headers and URLs. In the end, some of their classifiers performed well
with accuracies above 90%. Similar results were observed in (Bhagavatula et
al., 2014) and partly by (Dudykevych and Nechypor, 2016).

This method is interesting on two levels: (i) it cuts down the time of feature
engineering, which is fairly complex in this use case as HTTP headers are
mainly categorical data, and (ii) it should help when the models are deployed
and used in production, e.g. if HTTP data is received, it should be evaluated
in real time and checking the presence of headers is faster than elaborate
processing before evaluation. Therefore, the binary representation of HTTP
headers as a feature will be tested.

First and foremost, we train our classifiers based on the $\text{Chrome}_{22}$
dataset and use all other datasets as test sets, which is in accordance with
RQ3. Generally, each step in our pipeline focuses on reducing the dimensions
of the datasets, as most headers are rarely used, thus reducing potential
overfitting. This approach further reduces the sparseness and potential
features with high cardinality. The latter challenge is solved by employing
the binary-encoding method as it flattens the respective feature to two
possible values.

The sparseness and high dimensionality are related to each other, in a sense
that most headers and their corresponding values are rare or even unique. We
apply multiple methods to reduce the dimensionality: (i) a low variance filter
to remove headers with only one value, (ii) the removal of headers with a
missing value ratio of one, (iii) headers related to only one label are
dropped, and (iv) Approximate string matching, sometimes referred to as Fuzzy
Matching to combine header fields with a similarity of at least 80% based on a
weighted ratio of multiple distance metrics, e.g., Damerau Levenshtein or
Hamming. All of this reduces the sparseness further and binary-encoding
simplifies the sparse matrix.

Although a multitude of different resampling techniques exists to combat class
imbalance, the question of whether this step is actually necessary remains.
Some disadvantages of such methods are, for example, the removal of
observations, the replication of existing observations, or the creation of
synthetic samples. Selecting and applying a sampling method can lead to
overfitting and overoptimism as well (Santos et al., 2018), notwithstanding
the question of whether it is even feasible to train the models on a
distribution that was not observed in real-world data. Our pipeline preserves
the observed distribution as the imbalance is mild (goo, 2023) in the
$\text{Chrome}_{\textbf{22}}$ dataset, i.e., a sufficient number of samples
for each class exists, which allows proper training to differentiate between
classes. Furthermore, we compute metrics that consider or are not affected by
the imbalance of the datasets. The same applies for most of the selected
ensemble models.

Fuzzy matching is especially useful due to the presence of hundreds of similar
headers within our data. During our initial data analysis deiscussed in
Section 4, we identified many examples where headers with similar purpose were
written or formatted differently, e.g. content-lenght, cteonnt-length, or
ntcoent-length, instead of content-length. Accordingly, simply comparing the
header names is not enough, which is why we automatically assess the value
similarity to reduce potential mismatches.

Lastly, we apply Isotonic Regression instead of Platt Scaling to calibrate the
probabilities outputted by our models, following the results from (Niculescu-
Mizil and Caruana, 2005). This is done by fitting an Isotonic Regression model
to the models’ predictions and the true labels, and using this to adjust the
predicted probabilities. The calibration function for model $i$ is denoted as
$C_{i}(f_{i}(x))$.

We summarise our pipeline in Figure 4, including the data collection. Note
that we did not apply any data imputation to keep the final datasets as close
to the original collected datasets as possible. A validation set for
hyperparameter tuning was not created because we want to assess the effect of
calibration on its own. Moreover, tuning requires substantially more
computational power, e.g. by using Grid Search.

Figure 4. The parameterized pipeline automatically processes the collected
datasets and trains the classifiers.

###  5.3. Classification Models

As discussed in Section 2.1, prior studies (Cozza et al., 2020; Li et al.,
2015; Bhagavatula et al., 2014; Gugelmann et al., 2015) often used similar
classifiers despite the broad range of available models. We build upon these
established selections, refining our choice of models based on three
conditions: (i) the models should be trainable in a reasonable amount of time,
(ii) they should be applicable to this specific use case, and (iii) they
should preferably be ensemble methods, although we include a handful of non-
ensemble models that were used in past research to allow for direct
comparisons between our results and those of related studies. We prefer
ensemble methods as they usually perform better than their counterparts
(Dietterich, 2000), leading to their popularity over the past few years.
Nevertheless, to our best knowledge, they have not been widely used in past
web privacy research, except for Random Forests (RF). In Table 3, we provide
an overview of our selected models and their use in related work.

Model |  Ensemble |  Past Research
---|---|---
Decision Tree | No | (Gugelmann et al., 2015; Dudykevych and Nechypor, 2016; Raschke et al., 2023)
Random Forest | Yes |  (Cozza et al., 2020; Wu et al., 2016; Guarino. et al., 2020; Iqbal et al., 2022; Raschke et al., 2023), (Dudykevych and Nechypor, 2016) (U), (McGahagan et al., 2019)
Extra Trees Classifier | No | Not used
Logistic Regression | No |  (Bhagavatula et al., 2014; Gugelmann et al., 2015; Dudykevych and Nechypor, 2016; Wu et al., 2016; Raschke et al., 2023), (McGahagan et al., 2019)
Naïve Bayes (Gaussian or Bernoulli) | No | (Yamada et al., 2010; Bhagavatula et al., 2014; Gugelmann et al., 2015; Dudykevych and Nechypor, 2016; Wu et al., 2016; Guarino. et al., 2020)
Gradient Boosting | Yes |  (McGahagan et al., 2019), (Raschke et al., 2023)
Light-Gradient Boosting Machine | Yes | Not used
Adaptive Boosting | Yes | (Raschke et al., 2023)
Hist-Gradient Boosting | Yes | Not used
eXtreme Gradient Boosting | Yes | (Raschke et al., 2023)

Table 2. Overview of the selected machine learning models and the number of
times they were used in past research from Section 2.1. One paper did not
exactly specified which ensemble model was used, thus, the most likely model
was marked with Unspecified (U). Papers from web security are underlined.

Other supervised learning methods, e.g., Multi-layer Perceptrons (Cozza et
al., 2020; Guarino. et al., 2020), KNN, and especially SVM, could have been
used. The latter was successfully applied in multiple papers (Cozza et al.,
2020; Li et al., 2015; Bhagavatula et al., 2014; Guarino. et al., 2020).
However, both suffer in terms of time and computational complexity, e.g. SVM
has a train time complexity of $O(n^{2})$ where $n$ is the number of
observations (Steinwart and Christmann, 2008). In addition, deep-learning
models are not considered as the focus is on supervised classifiers and simple
rather than complex models. A graph-based approach, used by several papers
(Kalavri et al., 2016; Castell-Uroz et al., 2021) will not be pursued because
it does not fit well with the targeted approach, and direct connections
between individual HTTP responses will not be analyzed, e.g. through grouping
based on their URL or building a graph-based on their order.

###  5.4. Baseline Models

We compare our classifiers against three baseline models and multiple metrics
that we define in Section 5.5. We select one tree classifier, the Decision
Tree (DT), that is also the building block of six selected classification
models. Two simplistic and relatively robust baselines, namely Naïve Bayes
(NB) and Logistic Regression (LR).

Additionally, it is good practice to compare against a baseline approach that
differs from the presented classifiers while using the same dataset. Raschke
et al. (Raschke et al., 2023) presented a graph-based approach named T.EX-
Graph where edge and node attributes as well as centrality metrics served as
features. Furthermore, the authors included information from the responses but
did not pursue classifiers that are solely based on response headers. This
comparability should help in assessing the performance gains and the relevance
of response header values for automated web tracker detection. However, the
authors applied Synthetic Minority Oversampling to balance their dataset. We
focus on their results using the FQDN dataset, where hosts are modeled as
fully qualified domain names (FQDN) because the tracker distribution is
similar to ours with a NT to T ratio of 68:32.

###  5.5. Evaluation Metrics

We measure the performance of our classifiers according to nine different
metrics. Several metrics are common in the web tracking literature, e.g.,
accuracy, precision, or F1-Score, while others were selected due to their
applicability to our classification problem and to understand the relationship
with the underlying data in more detail. Additional metrics are reported in
the Appendix, e.g., the confusion matrix because its interpretation is already
captured by other metrics more concisely. Moreover, we calculate confidence
intervals (CI) for these metrics using bootstrap sampling with 599 samples
(Wilcox, 2010).

Metric |  Past Research
---|---
(Avg.) Accuracy |  (Gugelmann et al., 2015; Dudykevych and Nechypor, 2016; Raschke et al., 2023; Cozza et al., 2020; Guarino. et al., 2020; Iqbal et al., 2022; Bhagavatula et al., 2014), (McGahagan et al., 2019)
Precision | (Gugelmann et al., 2015; Raschke et al., 2023; Iqbal et al., 2022; Bhagavatula et al., 2014; Li et al., 2015)
Recall | (Gugelmann et al., 2015; Raschke et al., 2023; Iqbal et al., 2022; Li et al., 2015)
F1-Score |  (Gugelmann et al., 2015), (Raschke et al., 2023)
AUC-ROC |  (Gugelmann et al., 2015), (Bhagavatula et al., 2014), (McGahagan et al., 2019)
Confusion Matrix | (Guarino. et al., 2020)
MCC | (McGahagan et al., 2019)
AUCPRC | Not used
Log-Loss | Not used

Table 3. Overview of performance metrics and the number of times they were
used in past research from Section 2.1.

The first metrics that are commonly used in web privacy research are Accuracy,
Precision, Recall, False Positives (FP), True Positives (TP), False Negatives
(FN), and True Negatives (TN). Two accuracy scores can be calculated, one from
the perspective of trackers, the positive label, and one from non-trackers,
the negative label. However, accuracy is not particularly useful with
imbalanced datasets because a model could always predict the majority class
and achieve accuracy with the same value as the distribution of said class. We
calculated additional, more reliable metrics to quantify the performance of a
classifier with an imbalanced dataset as follows:

Balanced Accuracy:

Broderson et al. (Brodersen et al., 2010) introduced the Balanced Accuracy
(BACC) metric for imbalanced datasets in 2010 as an alternative to average
accuracy scores, which are, according to the authors, problematic for
confidence intervals (CI) and imbalanced datasets. The metric comprises recall
and TNR and is defined as:

(2)
$BalancedAccuracy=\frac{1}{2}\left(\frac{TP}{TP+FN}+\frac{TN}{TN+FP}\right)$

F1-Score:

Another metric that is commonly used for classification models is the F1-Score
which expresses a model’s performance in one value, despite the ambivalent
nature of precision and recall. Compared with BACC, it emphasizes positives
more, whereas BACC takes negatives more into account, which can result in two
very different results. The F1-Score is mathematically defined as the harmonic
mean between recall and precision:

(3) $F_{1}\text{-}Score=\frac{2\cdot Precision\cdot Recall}{Precision+Recall}$

Log-loss Score:

This metric, sometimes referred to as cross-entropy-loss, is particularly
useful for classification problems as it focuses on the predictive performance
of a model. The function is applied to each observation individually, and then
the average over all elements is calculated. It describes how large the error
between a prediction probability and the true label is, and a value closer to
zero represents a very good score. However, this metric can be influenced by
the imbalance in a dataset, i.e. the baseline log-loss gets smaller with an
increase in imbalance, and the log-loss metric should always be compared to
its baseline value, e.g. the baseline log-loss for a dataset with a 70:30
ratio equals 0.611, and 0.325 for datasets with a 90:10 ratio (tds, 2023). All
in all, the model’s log-loss should be lower than its baseline. The math
formula is defined as:

(4) $L_{log}(y,p)=-[y\cdot log(p)+(1-y)\cdot log(1-p)]$

where $y$ stands for the label and $p$ for probability.

Matthews-Correlation-Coefficient:

One metric is often not enough to fully comprehend the performance of a model
and if it is even suited for binary classification problems. Matthews-
Correlation-Coefficient (MCC) is becoming more popular in recent years,
although it was introduced in 1975 (Matthews, 1975) – it is defined as:

(5) $MCC=\frac{TP\times TN-FP\times
FN}{\sqrt{(TP+FP)\cdot(TP+FN)\cdot(TN+FP)\cdot(TN+FN)}}$

However, it is rarely used in web privacy research compared to other fields
like web security research. MCC ranges from -1 to +1 and can be understood as
a correlation coefficient between the true and predicted label. One particular
advantage is the inclusion of values from the confusion matrix, i.e. the MCC
will only achieve a high value if both class labels were predicted with high
accuracy – which in turn presents the MCC as a meaningful metric for
imbalanced datasets (Boughorbel et al., 2017). This was further substantiated
by Chicco et al. (Chicco and Jurman, 2020) in 2020, and the authors outlined
how accuracy and F1-Score were not reliable measures for imbalanced datasets.

ROC-AUC:

Our next metric is the Receiver Operating Characteristics (ROC) curve,
complemented by the Area Under the curve (AUC) value. The former describes a
classifier’s performance at various thresholds by plotting the TPR on the
ordinate and FPR on the abscissa. The latter metric expresses how well the
classifier is able to separate between both class labels.

AUPRC:

The last metric in our study is the Area Under the Precision-Recall Curve
(AUPRC). This metric is particularly relevant in the context of a supervised
binary classification problem, especially when dealing with imbalanced
datasets. The Precision-Recall curve is a plot that illustrates the trade-off
between the precision (positive predictive value) and the recall (true
positive rate) for a classifier at various thresholds. Precision is defined as
the ratio of true positives to the sum of true and false positives, while
recall is the ratio of true positives to the sum of true positives and false
negatives. The AUPRC is an aggregated measure of this curve, representing the
classifier’s ability to correctly identify positive instances across different
thresholds. In imbalanced settings, where one class is significantly
underrepresented, the AUPRC is a more informative and reliable metric than
ROC-AUC, as it focuses on the performance of the classifier on the minority
class, which is often of greater interest (Saito and Rehmsmeier, 2015).

##  6\. Experiments

Model |  Accuracy |  Log-Loss |  ROC-AUC |  AUPRC |  BACC |  F1-Score |  Precision |  Recall |  MCC
---|---|---|---|---|---|---|---|---|---
LR | 0.892 | 0.252 | 0.86 | 0.902 | 0.86 | 0.812 | 0.847 | 0.78 | 0.738
| [0.891;0.894] | [0.25;0.255] | [0.858;0.862] | [0.9;0.904] | [0.858;0.862] | [0.81;0.815] | [0.843;0.85] | [0.777;0.784] | [0.735;0.742]
GNB | 0.853 | 2.47 | 0.832 | 0.792 | 0.832 | 0.759 | 0.74 | 0.779 | 0.654
| [0.851;0.854] | [2.436;2.514] | [0.829;0.833] | [0.788;0.797] | [0.829;0.833] | [0.756;0.762] | [0.737;0.745] | [0.776;0.782] | [0.65;0.657]
BNB | 0.851 | 0.617 | 0.833 | 0.849 | 0.833 | 0.759 | 0.732 | 0.789 | 0.652
| [0.849;0.852] | [0.608;0.627] | [0.83;0.835] | [0.847;0.852] | [0.83;0.835] | [0.757;0.763] | [0.729;0.736] | [0.784;0.793] | [0.648;0.657]
DT | 0.956 | 0.546 | 0.947 | 0.943 | 0.947 | 0.926 | 0.928 | 0.925 | 0.895
| [0.955;0.957] | [0.531;0.566] | [0.946;0.948] | [0.94;0.945] | [0.946;0.948] | [0.925;0.928] | [0.926;0.93] | [0.922;0.927] | [0.893;0.897]
RF | 0.96 | 0.161 | 0.951 | 0.978 | 0.951 | 0.932 | 0.935 | 0.929 | 0.903
| [0.959;0.96] | [0.155;0.168] | [0.949;0.952] | [0.977;0.979] | [0.949;0.952] | [0.93;0.934] | [0.933;0.937] | [0.926;0.931] | [0.901;0.906]
ET | 0.964 | 0.166 | 0.953 | 0.98 | 0.953 | 0.938 | 0.952 | 0.925 | 0.913
| [0.963;0.965] | [0.159;0.174] | [0.952;0.954] | [0.978;0.98] | [0.952;0.954] | [0.937;0.94] | [0.95;0.954] | [0.923;0.927] | [0.911;0.915]
AdaBoost | 0.888 | 0.574 | 0.856 | 0.894 | 0.856 | 0.806 | 0.838 | 0.776 | 0.729
| [0.887;0.89] | [0.574;0.575] | [0.854;0.858] | [0.892;0.897] | [0.854;0.858] | [0.803;0.809] | [0.834;0.841] | [0.772;0.78] | [0.725;0.733]
GBM | 0.895 | 0.244 | 0.867 | 0.916 | 0.867 | 0.82 | 0.845 | 0.795 | 0.747
| [0.894;0.897] | [0.242;0.246] | [0.865;0.869] | [0.914;0.918] | [0.865;0.869] | [0.817;0.822] | [0.842;0.848] | [0.792;0.799] | [0.743;0.75]
LGBM | 0.931 | 0.174 | 0.909 | 0.957 | 0.909 | 0.88 | 0.908 | 0.854 | 0.833
| [0.93;0.932] | [0.172;0.175] | [0.907;0.91] | [0.956;0.958] | [0.907;0.91] | [0.878;0.883] | [0.906;0.911] | [0.851;0.857] | [0.83;0.836]
HistGB | 0.929 | 0.175 | 0.907 | 0.956 | 0.907 | 0.877 | 0.902 | 0.853 | 0.827
| [0.927;0.93] | [0.173;0.176] | [0.905;0.909] | [0.955;0.957] | [0.905;0.909] | [0.875;0.879] | [0.9;0.905] | [0.85;0.856] | [0.825;0.831]
XGBoost | 0.928 | 0.172 | 0.912 | 0.956 | 0.912 | 0.878 | 0.885 | 0.872 | 0.827
| [0.927;0.93] | [0.17;0.174] | [0.91;0.914] | [0.955;0.957] | [0.91;0.914] | [0.877;0.881] | [0.882;0.889] | [0.869;0.875] | [0.825;0.831]
XGBoost | 0.883 | - | - | - | - | 0.867 | 0.867 | 0.867 | -
RF | 0.881 | - | - | - | - | 0.866 | 0.862 | 0.871 | -
GBM | 0.858 | - | - | - | - | 0.844 | 0.836 | 0.856 | -
SVC | 0.848 | - | - | - | - | 0.828 | 0.819 | 0.852 | -
AdaBoost | 0.840 | - | - | - | - | 0.824 | 0.816 | 0.836 | -
DT | 0.834 | - | - | - | - | 0.814 | 0.810 | 0.819 | -
KNN | 0.833 | - | - | - | - | 0.814 | 0.809 | 0.820 | -
LR | 0.809 | - | - | - | - | 0.796 | 0.789 | 0.822 | -

Table 4. Performance comparison of the $\text{Chrome}_{\textbf{22}}$-trained
classifiers. Best results are bold, second best are underlined, and baseline
models are in italic. The performance results of the t.ex-Graph classifier are
reported in the bottom half (Raschke et al., 2023). Figure 5. The radar charts
illustrate that while all models perform well on the training set, the XGBoost
model exhibits more consistent performance across the various test sets,
suggesting better generalizability. The other models show reduced performance
on test sets, indicating potential overfitting to the training data.

In this section, we evaluate the effectiveness of our proposed methodology to
detect trackers. We start by presenting our classifier results based on
$\text{Chrome}_{22}$ data and follow-up by testing them with data from
$\text{Firefox}_{22}$ and $\text{Brave}_{22}$. Furthermore, we assess the
longitudinal performance with the $\text{Chrome}_{23}$ dataset that was
collected almost a year later.

We test our approach on an Apple computer, namely MacBook Pro (14-inch, 2021)
with a 10-core Central Processing Unit (CPU), a 16-core Graphical Processing
Unit (GPU), 1 Terabyte (TB) of SSD storage, and 32 Gigabyte (GB) of Random
Access Memory (RAM).

###  6.1. Intra-browser Performance

Table 4 shows the performance of our $\text{Chrome}_{\textbf{22}}$-trained
classifiers and the baselines. In terms of Accuracy, Ensemble Trees (ET) lead
with a score of 0.964, closely followed by Random Forest (RF) and Decision
Trees (DT) with scores of 0.960 and 0.956, respectively.

The Log-Loss metric, which penalizes classifiers for incorrect predictions,
shows that ET, RF, and DT also perform well, with ET having the lowest log
loss, indicating a high level of confidence in its predictions. On the other
hand, Gaussian Naive Bayes (GNB) exhibits a significantly higher log loss,
which suggests that it is less confident in its predictions compared to the
tree-based methods. The ROC-AUC metric further strengthens the position of ET,
RF, and DT, as they all score above 0.94, indicating that they are good at
differentiating between the positive and negative classes. The high MCC scores
corroborate the findings from the other metrics, placing these three
classifiers as the most robust performers across the board. In contrast,
baseline models like LR are comparatively lower performance across these
metrics.

The gradient boosting models, particularly XGBoost, have fairly high scores as
reflected in an AUC-ROC score of 0.915 and an F1-Score of 0.875. They are able
to handle the class imbalance due to the fairly high MCC and AUPRC scores.
However, their precision and recall scores indicate room for improvement in
identifying instances of the minority class.

Comparing the best performing classifiers ET and RF to the baseline models,
shows that they perform significantly higher, except for the DT and its Log-
Loss score. The t.ex-Graph classifiers display lower performance across all
metrics.

###  6.2. Cross-browser and Longitudinal Performance

Figure 5 presents the cross-browser performance and longitudinal performance
of our $\text{Chrome}_{\textbf{22}}$-trained classifiers. We focus on three
selected classifiers, one baseline model, one ensemble tree-based model, and
one ensemble tree-based model with gradient boosting.

For Logistic Regression and Random Forest, $\text{Chrome}_{\textbf{22}}$ shows
high performance across most metrics, which is expected as the models would be
most adapted to the data they were trained on. The performance on
$\text{Chrome}_{\textbf{23}}$, $\text{Firefox}_{\textbf{22}}$, and
$\text{Brave}_{\textbf{22}}$ varies, with a general trend of decreasing
performance, indicating potential overfitting to the training data or a lack
of generalizability to other datasets.

The XGBoost model shows a similar trend, with high performance on the training
set and slightly varied results on the test sets. However, the variations in
the XGBoost model’s performance are less pronounced than in the Logistic
Regression and Random Forest models, suggesting that XGBoost may be more
robust or generalizable across different browser datasets.

Overall, the figure indicates that while all models perform well on the
dataset they were trained on, their ability to generalize to other data is
varied. XGBoost appears to have the most consistent performance across
different browser datasets, indicating a potential advantage in terms of
generalization. The figure underscores the importance of cross-browser testing
to ensure that models are not only tuned to a specific dataset but also
maintain their performance across diverse scenarios.

The Brave results show a high accuracy metric for the overall performance, but
the F1-Score and MCC are low and represent a more realistic performance of the
classifiers, which is expected due to the more considerable differences
between the Chrome and Brave data.

The longitudinal performance analysis, as depicted in Figure 5, indicates a
slight decline in the efficacy of the classification models when transitioning
from the $\text{Chrome}_{\textbf{22}}$ to the $\text{Chrome}_{\textbf{23}}$
dataset. This is evidenced by the contraction of the metric lines from
$\text{Chrome}_{\textbf{22}}$ to $\text{Chrome}_{\textbf{23}}$ across the
three models, suggesting a decrease in metrics such as AUC-ROC, Accuracy, and
others. Notably, the XGBoost model demonstrates the smallest degree of
performance reduction, underscoring its robustness and adaptability to changes
over time. In contrast, both the Logistic Regression and Random Forest models
exhibit a more pronounced drop, highlighting potential challenges in model
generalization across browser version updates. This suggests that while all
models are affected by evolving data characteristics, ensemble methods,
particularly gradient boosting as implemented in XGBoost, may offer a more
stable predictive performance in longitudinal settings. Applying isotonic
calibration had only minor improvements compared to the uncalibrated models as
shown in Figure 6.

##  7\. Discussion

Tracking affects users across all three browsers. However, this varies to a
certain degree, e.g. Brave had the least amount of tracking-related HTTP
responses compared to Chrome and Firefox.

Nevertheless, an imbalance can be observed in all data sets, which might be a
problem for specific ML models. Even though many different methods were
developed to balance data sets, due to the moderate imbalance in Chrome and
Firefox and the good performance, a resampling was not conducted to preserve
the observed distribution. Still, the large number of observations provides
sufficient HTTP response messages for each class label. Furthermore, several
evaluation metrics were calculated and provided that are aimed at imbalanced
data sets, e.g. BACC.

The classifiers performed similarly in Chrome and Firefox but were poor with
Brave data. First and foremost, the application of binary-encoded features
resulted in surprisingly good results, i.e. the mere presence of a header
value is enough and extensive feature engineering is not required from a pure
tracking detection standpoint. On the other hand, explaining the models is
limited, except for model parameters, training time, or the calculated
metrics. In addition, this method simplifies the models to a high degree,
although HTTP header values can reveal more information about trackers and
their differences from non-trackers. Nevertheless, the binary approach helped
preserve the core information, namely whether a header is related to tracking.

The NB classifier did not perform as well as others, but that might be due to
its limited applicability to the data sets. As mentioned above, NB is based on
Bayes Theorem and assumes that features within a class are not correlated to
each other. It can be assumed that a set of headers and thus features are
correlated to a certain degree. These were not removed which is a violation of
Bayes Theorem in a statistical sense could be argued, and thus, its
applicability. On the contrary, a paper by Berrar (Berrar, 2018) argues that
NB can perform well, even under these circumstances. Other implementations,
e.g., the Categorical Naïve Bayes might have been a good fit for the training
sets with categorical features (nb, 2023).

Filter lists are reliable and effective when it comes to blocking trackers
(Mazel et al., 2019), and are one of the oldest methods, e.g. EasyList was
created in 2005 (eas, 2023a). They present a popular ground truth to label
data sets for training ML models. However, they have to be constantly updated,
which commonly happens manually. A longitudinal study of nine years by Alrizah
et al. (Alrizah et al., 2019) showed that human errors, e.g. acting in a
timely manner or incorrect reports of possible new trackers and ads, are a
root cause for lower performance. Chen et al. (Chen et al., 2021) highlighted
evasion methods to circumvent blockers and filter lists: (i) changing URLs,
(ii) embedding code directly into the website with the script tag, and (iii)
combining functional and tracking code in one file. In summary, using filter
lists for our ground truth is not without problems and directly affect the
performance of our classifiers.

###  7.1. Limitations

Firstly, the analysis of response headers to fill the identified research gap
has its strengths as well as its weaknesses. Headers represent only a part of
an HTTP message, and research has shown that HTTP bodies are also relevant
when it comes to understanding and identifying trackers (Li et al., 2015;
Guarino. et al., 2020). From a broader point of view, only one part of
application layer traffic is analyzed, disregarding other parts of web
traffic, e.g. transport layer data.

Secondly, the data set could have been processed differently across the
pipeline. As above-mentioned, the data sets are imbalanced, with a 72:28 non-
tracker-to-tracker ratio for Chrome and Firefox data. It is not a severe form
of imbalance, but it might be problematic for some classification algorithms.
They could be biased against the majority class, and it might be possible to
consistently predict the non-tracker label and still observe good metrics.
Several resampling techniques exist, simple methods, e.g. over- and
undersampling (Hernandez et al., 2013), or more complex methods, e.g. SMOTE
(Chawla et al., 2002), or one of its variations (Han et al., 2005; Mukherjee
and Khushi, 2021). Additional measures include using ML models, which handle
imbalanced data sets well (Juez-Gil et al., 2021), or anomaly detection (Park
et al., 2018). In summary, this aspect of the data pipeline has not been
implemented with any of the aforementioned methods, and it might be
interesting to explore how well these techniques may work for the present
concept.

Thirdly, the labeling of the initial data sets, i.e. the quality of the ground
truth. T.EX allows the implementation of different filter lists and offers
three to date, namely EasyList, EasyPrivacy, and Disconnect.me. All base their
functionality on extensive filter lists, i.e. rules and RegEx are used to
block tracking-related HTTP requests. Updates are carried out regularly every
four days for the first two (eas, 2023a, b), and on an irregular schedule for
the latter (dis, 2023). Research has shown that these filter lists are not
without errors regarding their rules (Snyder et al., 2020).

##  8\. Conclusion

In this work, we trained Chrome-based classifiers on a subset of binary-
encoded HTTP/S response headers to evaluate their effectiveness and usefulness
in web tracker detection while retaining the natural imbalance of trackers. In
addition, we measured the cross-browser effectiveness, showing that multiple
Chrome-based classifiers achieve high accuracies for Firefox as well. However,
large performance losses were observed for Brave. The longitudinal performance
of the selected ML models deteriorated only by small margins, which highlights
their potential due to the changing capabilities of trackers (Fouad et al.,
2018; Bujlow et al., 2017; Englehardt and Narayanan, 2016). Response headers
can be used to differentiate well between trackers and non-trackers and they
provide additional information to the nature of tracking-related entities.

For our future research we plan to further investigate response headers,
focusing on the values for feature engineering, or the use of Deep Learning
models. Lastly, we will approach the deployment aspect two-fold, one from a
user perspective in form of browser extension to test the models in a real-
life scenario, and secondly as a fully automated system in the cloud that
integrates a crawler to periodically update the models.

## References

  * (1)
  * dis (2023) 2023\.  _Disconnect_.  Retrieved Jan 09, 2023 from https://disconnect.me/trackerprotection
  * eas (2023a) 2023a.  _EasyList_.  Retrieved Jan 09, 2023 from https://easylist.to/easylist/easylist.txt
  * eas (2023b) 2023b.  _EasyPrivacy_.  Retrieved Jan 09, 2023 from https://easylist.to/easylist/easyprivacy.txt
  * goo (2023) 2023\.  _Google Developers_.  Retrieved Jan 31, 2023 from https://developers.google.com/machine-learning/data-prep/construct/sampling-splitting/imbalanced-data
  * kin (2023) 2023\.  _Kinsta_.  Retrieved Jan 31, 2023 from https://kinsta.com/browser-market-share/
  * nb (2023) 2023\.  _Scikit-learn_.  Retrieved Jan 23, 2023 from https://scikit-learn.org/stable/modules/naive_bayes.html
  * sta (2023) 2023\.  _Statcounter_.  Retrieved Jan 31, 2023 from https://gs.statcounter.com/browser-market-share/desktop/worldwide
  * tds (2023) 2023\.  _Towards Data Science_.  Retrieved Jan 23, 2023 from https://towardsdatascience.com/intuition-behind-log-loss-score-4e0c9979680a
  * zen (2023) 2023\.  _Zenodo_.  Retrieved Jan 21, 2023 from https://doi.org/10.5281/zenodo.7123945
  * Acar et al. (2013) Gunes Acar, Marc Juarez, Nick Nikiforakis, Claudia Diaz, Seda Gürses, Frank Piessens, and Bart Preneel. 2013.  FPDetective: Dusting the Web for Fingerprinters. In _Proceedings of the 2013 ACM SIGSAC Conference on Computer and Communications Security_ (Berlin, Germany) _(CCS ’13)_. Association for Computing Machinery, New York, NY, USA, 1129–1140.
  * Alrizah et al. (2019) Mshabab Alrizah, Sencun Zhu, Xinyu Xing, and Gang Wang. 2019.  Errors, Misunderstandings, and Attacks: Analyzing the Crowdsourcing Process of Ad-blocking Systems.  _Proceedings of the Internet Measurement Conference_ (2019), 230–244.
  * Bahrami et al. (2021) Pouneh Nikkhah Bahrami, Umar Iqbal, and Zubair Shafiq. 2021.  FP-Radar: Longitudinal Measurement and Early Detection of Browser Fingerprinting.  _Proceedings on Privacy Enhancing Technologies_ 2022 (2021), 557 – 577.
  * Bau et al. (2013) Jason Bau, Jonathan R. Mayer, Hristo S. Paskov, and John C. Mitchell. 2013.  A Promising Direction for Web Tracking Countermeasures.  _Proceedings of W2SP_ (2013).
  * Berrar (2018) Daniel Berrar. 2018.  _Bayes’ Theorem and Naive Bayes Classifier_.
  * Bhagavatula et al. (2014) Sruti Bhagavatula, Christopher Dunn, Chris Kanich, Minaxi Gupta, and Brian Ziebart. 2014.  Leveraging Machine Learning to Improve Unwanted Resource Filtering. In _Proceedings of the 2014 Workshop on Artificial Intelligent and Security Workshop_ (Scottsdale, Arizona, USA) _(AISec ’14)_. Association for Computing Machinery, New York, NY, USA, 95–102.
  * Boughorbel et al. (2017) Sabri Boughorbel, Fethi Jarray, and Mohammed El-Anbari. 2017.  Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric.  _PLoS ONE_ 12 (2017).
  * Brodersen et al. (2010) Kay Henning Brodersen, Cheng Soon Ong, Klaas Enno Stephan, and Joachim M. Buhmann. 2010.  The Balanced Accuracy and Its Posterior Distribution. In _2010 20th International Conference on Pattern Recognition_. 3121–3124.
  * Bujlow et al. (2017) Tomasz Bujlow, Valentín Carela-Español, Josep Solé-Pareta, and Pere Barlet-Ros. 2017.  A Survey on Web Tracking: Mechanisms, Implications, and Defenses.  _Proc. IEEE_ 105, 8 (2017), 1476–1510.
  * Cassel et al. (2022) Darion Cassel, Su-Chin Lin, Alessio Buraggina, William Wang, Andrew Zhang, Lujo Bauer, Hsu-Chun Hsiao, Limin Jia, and Timothy Libert. 2022.  OmniCrawl: Comprehensive Measurement of Web Tracking With Real Desktop and Mobile Browsers.  _Proceedings on Privacy Enhancing Technologies_ 2022, 1 (2022), 227–252.
  * Castell-Uroz et al. (2021) Ismael Castell-Uroz, Josep Solé-Pareta, and Pere Barlet-Ros. 2021.  TrackSign: Guided Web Tracking Discovery. In _IEEE INFOCOM 2021 - IEEE Conference on Computer Communications_. 1–10.
  * Chanchary and Chiasson (2015) Farah Habib Chanchary and Sonia Chiasson. 2015.  User Perceptions of Sharing, Advertising, and Tracking. In _Symposium On Usable Privacy and Security_.
  * Chawla et al. (2002) N. Chawla, K. Bowyer, Lawrence O. Hall, and W. Philip Kegelmeyer. 2002.  SMOTE: Synthetic Minority Over-sampling Technique.  _ArXiv_ abs/1106.1813 (2002).
  * Chen et al. (2021) Quan Chen, Peter Snyder, Ben Livshits, and Alexandros Kapravelos. 2021.  Detecting Filter List Evasion with Event-Loop-Turn Granularity JavaScript Signatures.  _2021 IEEE Symposium on Security and Privacy (SP)_ 00 (2021), 1715–1729.
  * Chicco and Jurman (2020) Davide Chicco and Giuseppe Jurman. 2020.  The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation.  _BMC Genomics_ 21, 1 (2020), 6.
  * Cozza et al. (2020) Federico Cozza, Alfonso Guarino, Francesco Isernia, Delfina Malandrino, Antonio Rapuano, Raffaele Schiavone, and Rocco Zaccagnino. 2020.  Hybrid and lightweight detection of third party tracking: Design, implementation, and evaluation.  _Computer Networks_ 167 (2020), 106993.
  * Demir et al. (2022) Nurullah Demir, Matteo Große-Kampmann, Tobias Urban, Christian Wressnegger, Thorsten Holz, and Norbert Pohlmann. 2022.  Reproducibility and Replicability of Web Measurement Studies. In _Proceedings of the ACM Web Conference 2022_ (Virtual Event, Lyon, France) _(WWW ’22)_. Association for Computing Machinery, New York, NY, USA, 533–544.
  * Dietterich (2000) Thomas G. Dietterich. 2000.  Ensemble Methods in Machine Learning. In _Multiple Classifier Systems_. Springer Berlin Heidelberg, Berlin, Heidelberg, 1–15.
  * Din et al. (2020) Zainul Abi Din, Panagiotis Tigas, Samuel T. King, and Benjamin Livshits. 2020.  PERCIVAL: Making in-Browser Perceptual Ad Blocking Practical with Deep Learning. In _Proceedings of the 2020 USENIX Conference on Usenix Annual Technical Conference_ _(USENIX ATC’20)_. USENIX Association, USA, Article 26, 14 pages.
  * Dudykevych and Nechypor (2016) Valery Dudykevych and Vitalii Nechypor. 2016.  Detecting Third-Party User Trackers with Cookie Files.  _2016 Third International Scientific-Practical Conference Problems of Infocommunications Science and Technology (PIC S &T)_ (2016), 78–80.
  * Englehardt and Narayanan (2016) Steven Englehardt and Arvind Narayanan. 2016.  Online Tracking: A 1-Million-Site Measurement and Analysis. In _Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security_ (Vienna, Austria) _(CCS ’16)_. Association for Computing Machinery, New York, NY, USA, 1388–1401.
  * Englehardt et al. (2015) Steven Englehardt, Dillon Reisman, Christian Eubank, Peter Zimmerman, Jonathan Mayer, Arvind Narayanan, and Edward W. Felten. 2015.  Cookies That Give You Away: The Surveillance Implications of Web Tracking. In _Proceedings of the 24th International Conference on World Wide Web_ (Florence, Italy) _(WWW ’15)_. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 289–299.
  * Fouad et al. (2018) Imane Fouad, Nataliia Bielova, Arnaud Legout, and Natasa Sarafijanovic-Djukic. 2018.  Tracking the pixels: Detecting web trackers via analyzing invisible pixels.  _arXiv preprint arXiv:1812.01514_ (2018).
  * Guarino. et al. (2020) Alfonso Guarino., Delfina Malandrino., Rocco Zaccagnino., Federico Cozza., and Antonio Rapuano. 2020.  On Analyzing Third-party Tracking via Machine Learning. In _Proceedings of the 6th International Conference on Information Systems Security and Privacy - ICISSP,_. INSTICC, SciTePress, 532–539.
  * Gugelmann et al. (2015) David Gugelmann, Markus Happe, Bernhard Ager, and Vincent Lenders. 2015.  An Automated Approach for Complementing Ad Blockers’ Blacklists.  _Proceedings on Privacy Enhancing Technologies_ 2015, 2 (2015), 282–298.  https://doi.org/10.1515/popets-2015-0018
  * Han et al. (2005) Hui Han, Wen-Yuan Wang, and Bing-Huan Mao. 2005.  Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning. In _Advances in Intelligent Computing_ , De-Shuang Huang, Xiao-Ping Zhang, and Guang-Bin Huang (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 878–887.
  * Hashmi et al. (2019) Saad Sajid Hashmi, Muhammad Ikram, and Mohamed Ali Kaafar. 2019.  A Longitudinal Analysis of Online Ad-Blocking Blacklists.  _2019 IEEE 44th LCN Symposium on Emerging Topics in Networking (LCN Symposium)_ 00 (2019), 158–165.
  * Hernandez et al. (2013) Julio Hernandez, Jesús Ariel Carrasco-Ochoa, and José Francisco Martínez-Trinidad. 2013.  An Empirical Study of Oversampling and Undersampling for Instance Selection Methods on Imbalance Datasets. In _Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications_ , José Ruiz-Shulcloper and Gabriella Sanniti di Baja (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 262–269.
  * Iqbal et al. (2021) Umar Iqbal, Steven Englehardt, and Zubair Shafiq. 2021.  Fingerprinting the Fingerprinters: Learning to Detect Browser Fingerprinting Behaviors. In _2021 IEEE Symposium on Security and Privacy (SP)_. 1143–1161.  https://doi.org/10.1109/SP40001.2021.00017
  * Iqbal et al. (2022) Umar Iqbal, Charlie Wolfe, Charles Nguyen, Steven Englehardt, and Zubair Shafiq. 2022.  Khaleesi: Breaker of Advertising and Tracking Request Chains. In _USENIX Security Symposium_.
  * Juez-Gil et al. (2021) Mario Juez-Gil, Álvar Arnaiz-González, Juan J. Rodríguez, and César García-Osorio. 2021.  Experimental evaluation of ensemble classifiers for imbalance in Big Data.  _Applied Soft Computing_ 108 (2021), 107447.
  * Kalavri et al. (2016) Vasiliki Kalavri, Jeremy Blackburn, Matteo Varvello, and Konstantina Papagiannaki. 2016.  Like a Pack of Wolves: Community Structure of Web Trackers. In _Passive and Active Measurement_ , Thomas Karagiannis and Xenofontas Dimitropoulos (Eds.). Springer International Publishing, Cham, 42–54.
  * Laughter et al. (2021) Ashley Laughter, Safwan Omari, Piotr Szczurek, and Jason Perry. 2021.  Detection of Malicious HTTP Requests Using Header and URL Features. In _Proceedings of the Future Technologies Conference (FTC) 2020, Volume 2_ , Kohei Arai, Supriya Kapoor, and Rahul Bhatia (Eds.). Springer International Publishing, Cham, 449–468.
  * Lerner et al. (2016) A. Lerner, A. Simpson, T. Kohno, and F. Roesner. 2016.  Internet Jones and the Raiders of the Lost Trackers: An Archaeological Study of Web Tracking from 1996 to 2016. In _USENIX Security Symposium_.
  * Li et al. (2015) Tai-Ching Li, Huy Hang, Michalis Faloutsos, and Petros Efstathopoulos. 2015.  TrackAdvisor: Taking Back Browsing Privacy from Third-Party Trackers. In _Passive and Active Measurement_ , Jelena Mirkovic and Yong Liu (Eds.). Springer International Publishing, Cham, 277–289.
  * Matthews (1975) B.W. Matthews. 1975.  Comparison of the predicted and observed secondary structure of T4 phage lysozyme.  _Biochimica et Biophysica Acta (BBA) - Protein Structure_ 405, 2 (1975), 442–451.
  * Mazel et al. (2019) Johan Mazel, Richard Garnier, and Kensuke Fukuda. 2019.  A comparison of web privacy protection techniques.  _Computer Communications_ 144 (2019), 162–174.
  * McGahagan et al. (2019) John McGahagan, Darshan Bhansali, Margaret Gratian, and Michel Cukier. 2019.  A Comprehensive Evaluation of HTTP Header Features for Detecting Malicious Websites. In _2019 15th European Dependable Computing Conference (EDCC)_. 75–82.
  * Melicher et al. (2016) William Melicher, Mahmood Sharif, Joshua Tan, Lujo Bauer, Mihai Christodorescu, and Pedro Giovanni Leon. 2016.  (Do Not) Track Me Sometimes: Users’ Contextual Preferences for Web Tracking.  _Proceedings on Privacy Enhancing Technologies_ 2016 (2016), 135 – 154.
  * Metwalley et al. (2015) Hassan Metwalley, Stefano Traverso, and Marco Mellia. 2015.  Unsupervised Detection of Web Trackers. In _2015 IEEE Global Communications Conference (GLOBECOM)_. 1–6.  https://doi.org/10.1109/GLOCOM.2015.7417499
  * Mukherjee and Khushi (2021) Mimi Mukherjee and Matloob Khushi. 2021.  SMOTE-ENC: A Novel SMOTE-Based Method to Generate Synthetic Data for Nominal and Continuous Features.  _Applied System Innovation_ 4, 1 (2021).
  * Niculescu-Mizil and Caruana (2005) Alexandru Niculescu-Mizil and Rich Caruana. 2005.  Predicting Good Probabilities with Supervised Learning. In _Proceedings of the 22nd International Conference on Machine Learning_ (Bonn, Germany) _(ICML ’05)_. Association for Computing Machinery, New York, NY, USA, 625–632.
  * Park et al. (2018) Seungyoung Park, Myungjin Kim, and Seokwoo Lee. 2018.  Anomaly Detection for HTTP Using Convolutional Autoencoders.  _IEEE Access_ 6 (2018), 70884–70901.
  * Pochat et al. (2018) Victor Le Pochat, Tom van Goethem, Samaneh Tajalizadehkhoob, Maciej Korczyński, and Wouter Joosen. 2018.  Tranco: A Research-Oriented Top Sites Ranking Hardened Against Manipulation.  _Proceedings 2019 Network and Distributed System Security Symposium_ (2018).
  * Raschke et al. (2023) Philip Raschke, Patrick Herbke, and Henry Schwerdtner. 2023.  t.ex-Graph: Automated Web Tracker Detection Using Centrality Metrics and Data Flow Characteristics. In _Proceedings of the 9th International Conference on Information Systems Security and Privacy - Volume 1: ICISSP,_. INSTICC, SciTePress, 199–209.  https://doi.org/10.5220/0011787300003405
  * Raschke et al. (2019) Philip Raschke, Sebastian Zickau, Jacob Leon Kröger, and Axel Küpper. 2019.  Towards Real-Time Web Tracking Detection with T.EX - The Transparency EXtension. In _Privacy Technologies and Policy_ , Maurizio Naldi, Giuseppe F. Italiano, Kai Rannenberg, Manel Medina, and Athena Bourka (Eds.). Springer International Publishing, Cham, 3–17.
  * Saito and Rehmsmeier (2015) Takaya Saito and Marc Rehmsmeier. 2015.  The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets.  _PLoS ONE_ 10 (2015).  https://api.semanticscholar.org/CorpusID:14081058
  * Santos et al. (2018) Miriam Seoane Santos, Jastin Pompeu Soares, Pedro Henrigues Abreu, Helder Araujo, and Joao Santos. 2018.  Cross-Validation for Imbalanced Datasets: Avoiding Overoptimistic and Overfitting Approaches [Research Frontier].  _IEEE Computational Intelligence Magazine_ 13, 4 (2018), 59–76.  https://doi.org/10.1109/MCI.2018.2866730
  * Snyder et al. (2020) Peter Snyder, Antoine Vastel, and Ben Livshits. 2020.  Who Filters the Filters: Understanding the Growth, Usefulness and Efficiency of Crowdsourced Ad Blocking. In _Abstracts of the 2020 SIGMETRICS/Performance Joint International Conference on Measurement and Modeling of Computer Systems_ (Boston, MA, USA) _(SIGMETRICS ’20)_. Association for Computing Machinery, New York, NY, USA, 75–76.
  * Steinwart and Christmann (2008) Ingo Steinwart and Andreas Christmann. 2008.  Support Vector Machines. In _Information Science and Statistics_.
  * Traverso et al. (2017) Stefano Traverso, Martino Trevisan, Leonardo Giannantoni, Marco Mellia, and Hassan Metwalley. 2017.  Benchmark and Comparison of Tracker-Blockers: Should You Trust Them?  _2017 Network Traffic Measurement and Analysis Conference (TMA)_ (2017), 1–9.
  * Wilcox (2010) R.R. Wilcox. 2010.  _Fundamentals of Modern Statistical Methods: Substantially Improving Power and Accuracy_.  Springer New York.  https://books.google.ie/books?id=PUkOBwAAQBAJ
  * Wu et al. (2016) Qianru Wu, Qixu Liu, Yuqing Zhang, Peng Liu, and Guanxing Wen. 2016.  A Machine Learning Approach for Detecting Third-Party Trackers on the Web. In _Computer Security – ESORICS 2016_ , Ioannis Askoxylakis, Sotiris Ioannidis, Sokratis Katsikas, and Catherine Meadows (Eds.). Springer International Publishing, 238–258.
  * Yamada et al. (2010) Akira Yamada, Hara Masanori, and Yutaka Miyake. 2010.  Web Tracking Site Detection Based on Temporal Link Analysis. In _2010 IEEE 24th International Conference on Advanced Information Networking and Applications Workshops_. 626–631.
  * Yang and Yue (2020) Zhiju Yang and Chuan Yue. 01 Apr. 2020.  A Comparative Measurement Study of Web Tracking on Mobile and Desktop Environments.  _Proceedings on Privacy Enhancing Technologies_ 2020, 2 (01 Apr. 2020), 24 – 44.
  * Yu et al. (2016) Zhonghao Yu, Sam Macbeth, Konark Modi, and Josep M. Pujol. 2016.  Tracking the Trackers. In _Proceedings of the 25th International Conference on World Wide Web_ (Montréal, Québec, Canada) _(WWW ’16)_. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 121–132.

##  9\. Appendix

Figure 6. The largest improvements were observed for the AdaBoost classifier.

###  9.1. Cross-Browser Performance

Model |  Accuracy |  Log-Loss |  ROC-AUC |  AUPRC |  BACC |  F1-Score |  Precision |  Recall |  MCC |  FP |  FN |  TP |  TN
---|---|---|---|---|---|---|---|---|---|---|---|---|---
LR | 0.895 | 0.246 | 0.849 | 0.881 | 0.849 | 0.787 | 0.821 | 0.756 | 0.718 | 34148 | 561918 | 50425 | 156322
| [0.894;0.895] | [0.245;0.247] | [0.848;0.851] | [0.88;0.882] | [0.848;0.851] | [0.785;0.789] | [0.819;0.822] | [0.754;0.759] | [0.716;0.72] | - | - | - | -
GNB | 0.86 | 2.47 | 0.825 | 0.759 | 0.825 | 0.735 | 0.717 | 0.754 | 0.64 | 61574 | 534492 | 50925 | 155822
| [0.859;0.861] | [2.454;2.488] | [0.824;0.826] | [0.757;0.76] | [0.824;0.826] | [0.733;0.736] | [0.715;0.719] | [0.752;0.756] | [0.638;0.642] | - | - | - | -
BNB | 0.857 | 0.614 | 0.829 | 0.82 | 0.829 | 0.735 | 0.701 | 0.772 | 0.639 | 68022 | 528044 | 47075 | 159672
| [0.856;0.857] | [0.61;0.618] | [0.828;0.83] | [0.819;0.821] | [0.828;0.83] | [0.734;0.737] | [0.699;0.704] | [0.77;0.774] | [0.637;0.641] | - | - | - | -
DT | 0.928 | 1.358 | 0.905 | 0.854 | 0.905 | 0.86 | 0.861 | 0.859 | 0.811 | 28600 | 567466 | 29241 | 177506
| [0.927;0.929] | [1.344;1.374] | [0.904;0.906] | [0.852;0.855] | [0.904;0.906] | [0.859;0.861] | [0.86;0.863] | [0.857;0.86] | [0.81;0.813] | - | - | - | -
RF | 0.936 | 0.326 | 0.913 | 0.94 | 0.913 | 0.875 | 0.883 | 0.866 | 0.832 | 23712 | 572354 | 27639 | 179108
| [0.935;0.937] | [0.32;0.333] | [0.912;0.914] | [0.94;0.941] | [0.912;0.914] | [0.874;0.876] | [0.882;0.885] | [0.865;0.868] | [0.83;0.833] | - | - | - | -
ET | 0.944 | 0.381 | 0.92 | 0.936 | 0.92 | 0.888 | 0.905 | 0.873 | 0.851 | 18998 | 577068 | 26330 | 180417
| [0.943;0.944] | [0.374;0.387] | [0.92;0.921] | [0.935;0.937] | [0.92;0.921] | [0.887;0.889] | [0.904;0.906] | [0.871;0.874] | [0.849;0.852] | - | - | - | -
AdaBoost | 0.891 | 0.565 | 0.848 | 0.873 | 0.848 | 0.782 | 0.809 | 0.757 | 0.711 | 36991 | 559075 | 50140 | 156607
| [0.891;0.892] | [0.565;0.565] | [0.847;0.849] | [0.872;0.875] | [0.847;0.849] | [0.781;0.784] | [0.807;0.811] | [0.756;0.76] | [0.709;0.713] | - | - | - | -
GBM | 0.899 | 0.235 | 0.864 | 0.899 | 0.864 | 0.802 | 0.812 | 0.792 | 0.735 | 37790 | 558276 | 43056 | 163691
| [0.899;0.9] | [0.234;0.236] | [0.863;0.865] | [0.898;0.9] | [0.863;0.865] | [0.801;0.803] | [0.811;0.814] | [0.789;0.794] | [0.733;0.736] | - | - | - | -
LGBM | 0.927 | 0.177 | 0.899 | 0.939 | 0.899 | 0.856 | 0.871 | 0.841 | 0.807 | 25678 | 570388 | 32912 | 173835
| [0.926;0.928] | [0.177;0.178] | [0.898;0.9] | [0.938;0.94] | [0.898;0.9] | [0.855;0.857] | [0.87;0.873] | [0.839;0.843] | [0.806;0.809] | - | - | - | -
HistGB | 0.927 | 0.177 | 0.898 | 0.939 | 0.898 | 0.855 | 0.872 | 0.839 | 0.807 | 25504 | 570562 | 33235 | 173512
| [0.926;0.928] | [0.176;0.178] | [0.897;0.899] | [0.938;0.939] | [0.897;0.899] | [0.854;0.857] | [0.87;0.873] | [0.837;0.841] | [0.805;0.808] | - | - | - | -
XGBoost | 0.933 | 0.165 | 0.908 | 0.946 | 0.908 | 0.868 | 0.879 | 0.857 | 0.823 | 24323 | 571743 | 29621 | 177126
| [0.932;0.933] | [0.164;0.166] | [0.907;0.909] | [0.946;0.947] | [0.907;0.909] | [0.866;0.869] | [0.878;0.881] | [0.854;0.858] | [0.821;0.824] | - | - | - | -

Table 5. Performance comparison of the $\text{Chrome}_{\textbf{22}}$-trained classifiers that were tested on $\text{Chrome}_{\textbf{23}}$ test data. Model |  Accuracy |  Log-Loss |  ROC-AUC |  AUPRC |  BACC |  F1-Score |  Precision |  Recall |  MCC |  FP |  FN |  TP |  TN
---|---|---|---|---|---|---|---|---|---|---|---|---|---
LR | 0.826 | 0.37 | 0.789 | 0.873 | 0.789 | 0.734 | 0.843 | 0.65 | 0.619 | 37453 | 489412 | 108345 | 201046
| [0.825;0.826] | [0.369;0.372] | [0.788;0.79] | [0.872;0.874] | [0.788;0.79] | [0.733;0.735] | [0.842;0.844] | [0.648;0.652] | [0.617;0.621] | - | - | - | -
GNB | 0.81 | 3.667 | 0.782 | 0.801 | 0.782 | 0.725 | 0.782 | 0.676 | 0.585 | 58413 | 468452 | 100380 | 209011
| [0.809;0.811] | [3.645;3.684] | [0.782;0.783] | [0.8;0.803] | [0.782;0.783] | [0.724;0.726] | [0.78;0.783] | [0.674;0.677] | [0.583;0.587] | - | - | - | -
BNB | 0.837 | 0.768 | 0.814 | 0.861 | 0.814 | 0.767 | 0.815 | 0.725 | 0.645 | 50959 | 475906 | 85201 | 224190
| [0.836;0.838] | [0.764;0.773] | [0.813;0.815] | [0.86;0.862] | [0.813;0.815] | [0.766;0.768] | [0.813;0.816] | [0.723;0.726] | [0.644;0.647] | - | - | - | -
DT | 0.876 | 2.869 | 0.845 | 0.839 | 0.845 | 0.813 | 0.92 | 0.728 | 0.733 | 19525 | 507340 | 84257 | 225134
| [0.875;0.877] | [2.849;2.89] | [0.844;0.846] | [0.838;0.84] | [0.844;0.846] | [0.811;0.814] | [0.919;0.921] | [0.726;0.729] | [0.731;0.734] | - | - | - | -
RF | 0.869 | 2.292 | 0.84 | 0.862 | 0.84 | 0.805 | 0.895 | 0.731 | 0.715 | 26672 | 500193 | 83190 | 226201
| [0.868;0.869] | [2.275;2.309] | [0.839;0.841] | [0.861;0.863] | [0.839;0.841] | [0.803;0.806] | [0.893;0.896] | [0.729;0.733] | [0.714;0.717] | - | - | - | -
ET | 0.862 | 2.351 | 0.831 | 0.854 | 0.831 | 0.793 | 0.893 | 0.713 | 0.701 | 26373 | 500492 | 88920 | 220471
| [0.861;0.863] | [2.335;2.369] | [0.83;0.832] | [0.853;0.855] | [0.83;0.832] | [0.791;0.794] | [0.892;0.894] | [0.711;0.714] | [0.7;0.703] | - | - | - | -
AdaBoost | 0.823 | 0.582 | 0.787 | 0.871 | 0.787 | 0.731 | 0.836 | 0.649 | 0.613 | 39267 | 487598 | 108516 | 200875
| [0.822;0.824] | [0.581;0.582] | [0.786;0.788] | [0.87;0.872] | [0.786;0.788] | [0.73;0.733] | [0.835;0.838] | [0.647;0.651] | [0.611;0.615] | - | - | - | -
GBM | 0.831 | 0.365 | 0.796 | 0.873 | 0.796 | 0.743 | 0.848 | 0.661 | 0.63 | 36779 | 490086 | 104767 | 204624
| [0.83;0.831] | [0.363;0.366] | [0.795;0.797] | [0.872;0.874] | [0.795;0.797] | [0.741;0.744] | [0.846;0.849] | [0.66;0.663] | [0.628;0.632] | - | - | - | -
LGBM | 0.848 | 0.407 | 0.815 | 0.888 | 0.815 | 0.77 | 0.874 | 0.689 | 0.67 | 30875 | 495990 | 96175 | 213216
| [0.847;0.849] | [0.405;0.408] | [0.814;0.816] | [0.887;0.889] | [0.814;0.816] | [0.769;0.772] | [0.872;0.875] | [0.687;0.691] | [0.668;0.671] | - | - | - | -
HistGB | 0.848 | 0.404 | 0.815 | 0.888 | 0.815 | 0.769 | 0.874 | 0.687 | 0.669 | 30662 | 496203 | 96754 | 212637
| [0.847;0.848] | [0.402;0.406] | [0.814;0.815] | [0.887;0.889] | [0.814;0.815] | [0.768;0.771] | [0.873;0.875] | [0.686;0.689] | [0.667;0.67] | - | - | - | -
XGBoost | 0.854 | 0.461 | 0.822 | 0.885 | 0.822 | 0.78 | 0.882 | 0.699 | 0.683 | 29030 | 497835 | 93120 | 216271
| [0.853;0.855] | [0.458;0.463] | [0.821;0.823] | [0.885;0.886] | [0.821;0.823] | [0.778;0.781] | [0.88;0.883] | [0.697;0.701] | [0.681;0.685] | - | - | - | -

Table 6. Performance comparison of the $\text{Chrome}_{\textbf{22}}$-trained classifiers that were tested on $\text{Firefox}_{\textbf{22}}$ test data. Model |  Accuracy |  Log-Loss |  ROC-AUC |  AUPRC |  BACC |  F1-Score |  Precision |  Recall |  MCC |  FP |  FN |  TP |  TN
---|---|---|---|---|---|---|---|---|---|---|---|---|---
LR | 0.951 | 0.144 | 0.775 | 0.275 | 0.775 | 0.114 | 0.063 | 0.598 | 0.183 | 26547 | 535147 | 1193 | 1776
| [0.95;0.951] | [0.143;0.145] | [0.769;0.785] | [0.262;0.289] | [0.769;0.785] | [0.109;0.118] | [0.06;0.065] | [0.584;0.616] | [0.176;0.189] | - | - | - | -
GNB | 0.901 | 1.374 | 0.795 | 0.114 | 0.795 | 0.068 | 0.036 | 0.687 | 0.141 | 55130 | 506564 | 928 | 2041
| [0.9;0.902] | [1.361;1.392] | [0.786;0.802] | [0.108;0.122] | [0.786;0.802] | [0.065;0.07] | [0.034;0.037] | [0.67;0.703] | [0.136;0.146] | - | - | - | -
BNB | 0.895 | 0.321 | 0.829 | 0.431 | 0.829 | 0.071 | 0.037 | 0.763 | 0.154 | 58493 | 503201 | 704 | 2265
| [0.894;0.896] | [0.318;0.324] | [0.823;0.836] | [0.413;0.45] | [0.823;0.836] | [0.068;0.073] | [0.036;0.038] | [0.749;0.776] | [0.149;0.157] | - | - | - | -
DT | 0.986 | 0.14 | 0.942 | 0.523 | 0.942 | 0.402 | 0.259 | 0.898 | 0.478 | 7618 | 554076 | 302 | 2667
| [0.986;0.986] | [0.136;0.145] | [0.937;0.948] | [0.506;0.537] | [0.937;0.948] | [0.393;0.41] | [0.251;0.266] | [0.888;0.909] | [0.47;0.485] | - | - | - | -
RF | 0.986 | 0.054 | 0.944 | 0.784 | 0.944 | 0.411 | 0.267 | 0.9 | 0.486 | 7355 | 554339 | 296 | 2673
| [0.986;0.987] | [0.052;0.056] | [0.938;0.948] | [0.77;0.799] | [0.938;0.948] | [0.401;0.419] | [0.258;0.273] | [0.889;0.91] | [0.477;0.493] | - | - | - | -
ET | 0.987 | 0.07 | 0.941 | 0.708 | 0.941 | 0.427 | 0.28 | 0.894 | 0.497 | 6823 | 554871 | 314 | 2655
| [0.987;0.988] | [0.068;0.073] | [0.935;0.946] | [0.693;0.721] | [0.935;0.946] | [0.416;0.435] | [0.271;0.287] | [0.883;0.905] | [0.487;0.503] | - | - | - | -
AdaBoost | 0.946 | 0.527 | 0.755 | 0.244 | 0.755 | 0.099 | 0.054 | 0.563 | 0.163 | 29111 | 532583 | 1298 | 16717
| [0.945;0.947] | [0.527;0.528] | [0.747;0.764] | [0.231;0.258] | [0.747;0.764] | [0.095;0.103] | [0.052;0.056] | [0.547;0.579] | [0.157;0.168] | - | - | - | -
GBM | 0.951 | 0.139 | 0.849 | 0.459 | 0.849 | 0.137 | 0.075 | 0.746 | 0.227 | 27166 | 534528 | 755 | 2214
| [0.95;0.951] | [0.139;0.14] | [0.842;0.858] | [0.443;0.479] | [0.842;0.858] | [0.132;0.14] | [0.072;0.077] | [0.731;0.763] | [0.221;0.233] | - | - | - | -
LGBM | 0.975 | 0.093 | 0.885 | 0.574 | 0.885 | 0.253 | 0.15 | 0.794 | 0.339 | 13315 | 548379 | 613 | 2356
| [0.975;0.976] | [0.092;0.094] | [0.878;0.891] | [0.556;0.59] | [0.878;0.891] | [0.243;0.259] | [0.144;0.154] | [0.78;0.805] | [0.331;0.344] | - | - | - | -
HistGB | 0.976 | 0.093 | 0.882 | 0.561 | 0.882 | 0.255 | 0.152 | 0.788 | 0.34 | 13018 | 548676 | 629 | 2340
| [0.976;0.976] | [0.093;0.094] | [0.875;0.889] | [0.542;0.576] | [0.875;0.889] | [0.245;0.261] | [0.145;0.156] | [0.772;0.801] | [0.331;0.346] | - | - | - | -
XGBoost | 0.979 | 0.08 | 0.899 | 0.673 | 0.899 | 0.288 | 0.175 | 0.817 | 0.372 | 11476 | 550218 | 542 | 2427
| [0.978;0.979] | [0.08;0.081] | [0.891;0.906] | [0.657;0.688] | [0.891;0.906] | [0.278;0.295] | [0.168;0.18] | [0.802;0.831] | [0.362;0.379] | - | - | - | -

Table 7. Performance comparison of the $\text{Chrome}_{\textbf{22}}$-trained
classifiers that were tested on $\text{Brave}_{\textbf{22}}$ test data.