# Explanation Methods in Deep Learning:
Users, Values, Concerns and Challenges111This article will appear as a chapter
in Explainable and Interpretable Models in Computer Vision and Machine
Learning, a Springer series on Challenges in Machine Learning.

Gabriëlle Ras, Marcel van Gerven, Pim Haselager

Radboud University, Donders Institute for Brain, Cognition and Behaviour,
Nijmegen, the Netherlands
{g.ras, m.vangerven<EMAIL_ADDRESS>
###### Abstract

Issues regarding explainable AI involve four components: users, laws &
regulations, explanations and algorithms. Together these components provide a
context in which explanation methods can be evaluated regarding their
adequacy. The goal of this chapter is to bridge the gap between expert users
and lay users. Different kinds of users are identified and their concerns
revealed, relevant statements from the General Data Protection Regulation are
analyzed in the context of Deep Neural Networks (DNNs), a taxonomy for the
classification of existing explanation methods is introduced, and finally, the
various classes of explanation methods are analyzed to verify if user concerns
are justified. Overall, it is clear that (visual) explanations can be given
about various aspects of the influence of the input on the output. However, it
is noted that explanation methods or interfaces for lay users are missing and
we speculate which criteria these methods / interfaces should satisfy. Finally
it is noted that two important concerns are difficult to address with
explanation methods: the concern about bias in datasets that leads to biased
DNNs, as well as the suspicion about unfair outcomes.

##  1 Introduction

Increasingly, Artificial Intelligence (AI) is used in order to derive
actionable outcomes from data (e.g. categorizations, predictions, decisions).
The overall goal of this chapter is to bridge the gap between expert users and
lay users, highlighting the explanation needs of both sides and analyzing the
current state of explainability. We do this by taking a more detailed look at
each component mentioned above and in Figure 1. Finally we address some
concerns in the context of DNNs.

###  1.1 The components of explainability

Issues regarding explainable AI (XAI) involve (at least) four components:
users, laws and regulations, explanations and algorithms. Together these
components provide a context in which explanation methods can be evaluated
regarding their adequacy. We briefly discuss these components in Figure 1.

###  1.2 Users and laws

AI has a serious impact on society, due to the large scale adoption of digital
automation techniques that involve information processing and prediction. Deep
Neural Networks (DNNs) belong to this set of automation techniques and are
used increasingly because of their capability to extract meaningful patterns
from raw input. DNNs are fed large quantities of digital information that are
easily collected from users. Currently there is much debate regarding the
safety of and trust in data processes in general, leading to investigations
regarding the explainability of AI-supported decision making. The level of
concern about these topics is reflected by official regulations such as the
General Data Protection Regulation (GDPR), also mentioned in (Doshi-Velez and
Kim, 2017; Holzinger et al., 2017a), incentives to promote the field of
explainability (Gunning, 2017) and institutional initiatives to ensure the
safe development of AI such as OpenAI. As the technology becomes more
widespread, DNNs in particular, the dependency on said technology increases
and ensuring trust in DNN technology becomes a necessity. Current DNNs are
achieving unparalleled performance in areas of Computer Vision (CV) and
Natural Language Processing (NLP). They are also being used in real-world
applications in e.g. medical imaging (Lee et al., 2017), autonomous driving
(Bojarski et al., 2017) and legislation (Lockett et al., ).

###  1.3 Explanation and DNNs

The challenge with DNNs in particular lies in providing insight into the
processes leading to their outcomes, and thereby helping to clarify under
which circumstances they can be trusted to perform as intended and when they
cannot. Unlike other methods in Machine Learning (ML), such as decision trees
or Bayesian networks, an explanation for a certain decision made by a DNN
cannot be retrieved by simply scrutinizing the inference process. The learned
internal representations and the flow of information through the network are
hard to analyze: As architectures get deeper, the number of learnable
parameters increases. It is not uncommon to have networks with millions of
parameters. Furthermore, network architecture is determined by various
components (unit type, activation function, connectivity pattern, gating
mechanisms) and the result of a complex learning procedure, which itself
depends on various properties (regularization, adaptive mechanisms, employed
cost function). The net result of the interaction between these components
cannot be predicted in advance. Because of these complications, DNNs are often
called black box models, as opposed to glass-box models (Holzinger et al.,
2017b). Fortunately, these problems have not escaped the attention of the
ML/Deep Learning (DL) community (Zeng, 2016; Samek et al., 2017; Seifert et
al., 2017; Olah et al., 2017; Hall et al., 2017; Montavon et al., 2018;
Marcus, 2018; Doshi-Velez and Kim, 2017). Research on how to interpret and
explain the decision process of Artificial Neural Networks (ANNs) has been
going on since the late 1980’s (Elman, 1989; Andrews et al., 1995). The
objective of explanation methods is to make specific aspects of a DNN’s
internal representations and information flow interpretable by humans.

Figure 1: Issues regarding explainable DNNs involve (at least) four
components: users, algorithms, laws and explanations. Together these
components provide a context in which explanations can be evaluated regarding
their adequacy.

##  2 Users and their concerns

Various kinds of DNN users can be distinguished. Users entertain certain
values; these include ethical values such as fairness, neutrality, lawfulness,
autonomy, privacy or safety, or functional values such as accuracy, usability,
speed or predictability. Out of these values certain concerns regarding DNNs
may arise, e.g. apprehensions about discrimination or accuracy. These concerns
get translated into questions about the system, e.g. “did the factor ‘race’
influence the outcome of the system” or “how reliable was the data used?” In
this section we identify at least two general types of users: the expert users
and the lay users, that can be further categorized into six specific kinds of
users. Note that there could be (and there regularly is) overlap between the
users described below, such that a particular user can be classified as
belonging to more than one of the categories.

  1. 1.

Expert users are the system builders and/or modifiers that have direct
influence on the implementation of the network. Two kinds of experts can be
identified:

    1. (a)

DNN engineers are generally researchers involved in extending the field and
have detailed knowledge about the mathematical theories and principles of
DNNs. DNN engineers are interested in explanations of a functional nature,
e.g. the effects of various hyperparameters on the performance of the network
or methods that can be used for model debugging.

    2. (b)

DNN developers are generally application builders who make software solutions
that can be used by lay people. DNN developers often make use of off-the-shelf
DNNs, often re-training the DNN along with tuning certain hyperparameters and
integrating them with various software components, resulting in a functional
application. The DNN developer is concerned with the goals of the overall
application and assesses whether they have been met by the DNN solution. DNN
developers are interested in explanation methods that allow them to understand
the behavior of the DNN in the various use cases of the integrated software
application.

  2. 2.

Lay users do not and need not have knowledge of how the DNN was implemented
and the underlying mathematical principles, nor do they require knowledge of
how the DNN was integrated with other software components resulting in a final
functional application. At least four lay users are identified:

    1. (a)

The owner of the software application in which the DNN is embedded. The owner
is usually an entity that acquires the application for possible commercial,
practical or personal use. For example, an owner can be an organization (e.g.
a hospital or a car manufacturer) that purchases the application for end users
(e.g. employees (doctors) or clients (car buyers)), but the owner can also be
a consumer that purchases the application for personal use. In the latter case
the categorization of owner fully overlaps with the next category of users
which are the end users. The owner is concerned with explainability questions
about the capabilities of the application, e.g. justification of a prediction
or a prediction given the input data, and aspects of accountability, e.g. to
what extent can application malfunction be attributed to the DNN component?

    2. (b)

The end user for whom the application was intended to be used by. The end user
uses the application as part of their profession or for personal use. The end
user is concerned with explainability about the capabilities of the
application, e.g. justification of a prediction given the input data, and
explainability regarding the behavior of the application, e.g. why does the
application not do what it was advertised to do?

    3. (c)

The data subject is the entity whose information is being processed by the
application or the entity which is directly affected by the application
outcome. An outcome is the output of the application in the context of the use
case. Sometimes the data subject is the same entity as the end user, for
example in the case that the application is meant for personal use. The data
subject is mostly concerned with the ethical and moral aspects that result
from the actionable outcomes. An actionable outcome is an outcome that has
consequences or an outcome on which important decisions are based.

    4. (d)

Stakeholders are people or organizations without a direct connection to either
the development, use or outcome of the application and who can reasonably
claim an interest in the process, for instance when its use runs counter to
particular values they protect. Governmental and non-governmental
organizations may put forward legitimate information requests regarding the
operations and consequences of DNNs. Stakeholders are often interested in the
ethical and legal concerns raised in any phase of the process.

### Case study: autonomous driving

In this section the different users are presented in the context of a self-
driving car.

  1. 1.

The DNN engineer creates a DL solution to the problem of object segmentation
and object classification by experimenting with various types of networks.
Given raw video input the DL solution gives the output of the type of object
and the location of the object in the video.

  2. 2.

The DNN developer creates a planning system which integrates the output of the
DL solution with other components in the system. The planning system decides
which actions the car will take.

  3. 3.

The owner acquires the planning system and produces a car in which the
planning system is operational.

  4. 4.

The end user purchases the car and uses the car to travel from point A to
point B.

  5. 5.

The data subjects are all the entities from which information is captured
along the route from point A to point B: pedestrians, private property such as
houses, other cars.

  6. 6.

The stakeholders are governmental institutions which formulate laws regulating
the use of autonomous vehicles, or insurance companies that have to assess
risk levels and their consequences.

##  3 Laws and regulations

An important initiative within the European Union is the General Data
Protection Regulation (GDPR) 222https://www.eugdpr.org that was approved on
April 14, 2016, and became enforceable on May 25, 2018. The GDPR distinguishes
between personal data, data subjects, data processors and data controllers
(Article 4, Definitions, Paragraphs 1, 7 & 8). Personal data is defined as
“any information relating to an identified or identifiable natural person
(data subject)”. A data processor is the natural or legal person, public
authority, agency or other body which processes data on behalf of the data
controller, who determines the purposes, conditions and means of the
processing. Hence, the DNN can function as a tool to be used by the data
processor, whereas owners or end users can fill the role of data controllers.

The GDPR focuses in part on profiling: “any form of automated processing of
personal data consisting of the use of personal data to evaluate certain
personal aspects relating to a natural person, in particular to analyse or
predict aspects concerning that natural person’s performance at work, economic
situation, health, personal preferences, interests, reliability, behaviour,
location or movements” (Article 4, Definitions, Paragraph 4). According to
articles 13, 14 and 15, when personal data is collected from a data subject
for automated decision-making, the data subject has the right to access, and
the data controller is obliged to provide, “meaningful information about the
logic involved.” Article 12 stipulates that the provision of information to
data subjects should be in “concise, transparent, intelligible and easily
accessible form, using clear and plain language.”

##  4 Explanation

The right to meaningful information translates into the demand that actionable
outcomes of DNNs need to be explained, i.e. be made transparent, interpretable
or comprehensible to humans. Transparency refers to the extent to which an
explanation makes a specific outcome understandable to a particular (group of)
users. Understanding, in this context, amounts to a person grasping how a
particular outcome was reached by the DNN. Note that this need not imply
agreeing with the conclusion, i.e. accepting the outcome as valid or
justified. In general, transparency may be considered as recommendable,
leading to e.g. a greater (societal) sense of control and acceptance of ML
applications. Transparency is normally also a precondition for accountability:
i.e. the extent to which the responsibility for the actionable outcome can be
attributed to legally (or morally) relevant agents (governments, companies,
expert or lay users, etc.). However, transparency may also have negative
consequences, e.g. regarding privacy or by creating possibilities for
manipulation (of data, processing or training).

In relation to the (perceived) need for explanation, two reasons for
investigation stand out in particular. First, a DNN may appear to dysfunction,
i.e. fail to operate as intended, e.g. through bugs in the code (process
malfunction). Second, it may misfunction, e.g. by producing unintended or
undesired (side-)effects (Floridi et al., 2015; Mittelstadt et al., 2016) that
are deemed to be societally or ethically unacceptable (outcome malfunction).
Related to dysfunction is a first category of explanations. This category is
based on the information necessary in order to understand the system’s basic
processes, e.g. to assess whether it is functioning properly, as intended, or
whether it dysfunctions (e.g. suboptimal or erroneous results). This type of
explanation is normally required by DNN developers and expert users. The
information is used to interpret, predict, monitor, diagnose, improve, debug
or repair the functioning of a system (Weller, 2017).

Once an application is made available to non-expert users, normally certain
guarantees regarding the system’s proper functioning are in place. Generally
speaking, owners, end users, data subjects and stakeholders are more
interested in a second category of explanations, where suspicions about a
DNN’s misfunctioning (undesired outcomes) leads to requests for “local
explanations”. Users may request information about how a particular outcome
was reached by the DNN, which aspects of input data, which learning factors or
other parameters of the system influenced its decision or prediction. This
information is then used to assess the appropriateness of the outcome in
relation to the concerns and values of users (Doran et al., 2017; Wachter et
al., 2017; Doshi-Velez et al., 2017; Weller, 2017). The aim of local
explanations is to strengthen the confidence and trust of users that the
system is not (or will not be) conflicting with their values, i.e. that it
does not violate fairness or neutrality. Note that this implies that the
offered explanations should match (within certain limits) the particular
user’s capacity for understanding (Doshi-Velez and Kim, 2017), as indicated by
the GDPR.

##  5 Explanation methods

So far the users, the GDPR, and the role of explanations have been discussed.
To bridge the gap from that area to the more technical area of explanation
methods, we need to be able to evaluate the capabilities of existing methods,
in the context of the users and their needs. We bridge the gap in two ways.
First, we identify, on a high level, desirable properties of explanation
methods. Second, we introduce a taxonomy to categorize all types of
explanation methods and third, assess the presence of the desirable properties
in the categories in our taxonomy.

###  5.1 Desirable properties of explainers

Based on a survey of the literature, we arrive at the following properties
which any explainer should have:

  1. 1.

High Fidelity The degree to which the interpretation method agrees with the
input-output mapping of the DNN. This term appears in (Arbatli and Akin, 1997;
Markowska-Kaczmar and Wnuk-Lipiński, 2004; Zilke et al., 2016; Ribeiro et al.,
2016a, b; Andrews et al., 1995; Lakkaraju et al., 2017). Fidelity is arguably
the most important property that an explanation model should possess. If an
explanation method is not faithful to the original model then it cannot give
valid explanations because the input-output mapping is incorrect.

  2. 2.

High Interpretabiliy To what extent a user is able to obtain true insight into
how actionable outcomes are obtained. We distinguish interpretability into the
following two subproperties:

    1. (a)

High Clarity The degree to which the resulting explanation is unambiguous.
This property is extremely important in safety-critical applications (Andrews
et al., 1995) where ambiguity is to be avoided. (Lakkaraju et al., 2017)
introduces a quantifiable measure of clarity (unambiguity) for their method.

    2. (b)

High Parsimony This refers to the complexity of the resulting explanation. An
explanation that is parsimonious is a simple explanation. This concept is
generally related to Occam’s razor and in the case of explaining DNNs the
principle is also of importance. The optimal degree of parsimony can in part
be dependent on the user’s capabilities.

  3. 3.

High Generalizability The range of architectures to which the explanation
method can be applied. This increases the usefulness of the explanation
method. Methods that are model-agnostic (Ribeiro et al., 2016b) are the
highest in generalizability.

  4. 4.

High Explanatory Power In this context this means how many phenomena the
method can explain. This roughly translates to how many different kinds of
questions the method can answer. Previously in Section 2 we have identified a
number of questions that users may have. It is also linked to the notion that
the explainer should be able to take a global perspective (Ribeiro et al.,
2016b), in the sense that it can explain the behaviour of the model rather
than only accounting for individual predictions.

###  5.2 A taxonomy for explanation methods

Over a relatively short period of time a plethora of explanation methods and
strategies have come into existence, driven by the need of expert users to
analyze and debug their DNNs. However, apart from a non-exhaustive overview of
existing methods (Montavon et al., 2018) and classification schemes for purely
visual methods (Grün et al., 2016; Seifert et al., 2017; Zeng, 2016;
Kindermans et al., 2017), little is known about efforts to rigorously map the
landscape of explanation methods and isolate the underlying patterns that
guide explanation methods. In this section a taxonomy for explanation methods
is proposed. Three main classes of explanation methods are identified and
their features described. The taxonomy was derived by analyzing the historical
and contemporary trends surrounding the topic of interpretation of DNNs and
explainable AI. We realize that we cannot foresee the future developments of
DNNs and their explainability methods. As such it is possible that in the
future the taxonomy needs to be modified. We propose the following taxonomy:

Rule-extraction methods

Extract rules that approximate the decision-making process in a DNN by
utilizing the input and output of the DNN.

Attribution methods

Measures the importance of a component by changing to the input or internal
components and recording how much the changes affect model performance.
Methods known by other names that fall in this category are occlusion,
perturbation, erasure, ablation and influence. Attribution methods are often
visualized and sometimes referred to as visualization methods.

Intrinsic methods

Aim to improve the interpretability of internal representations with methods
that are part of the DNN architecture. Intrinsic methods increase fidelity,
clarity and parsimony in attribution methods.

In the following subsections we will describe the main features of each class
and give examples from current research.

####  5.2.1 Rule-extraction methods

Rule-extraction methods extract human interpretable rules that approximate the
decision-making process in a DNN. Older genetic algorithm based rule
extraction methods for ANNs can be found in (Andrews et al., 1995; Arbatli and
Akin, 1997; Lu et al., 2006). Andrews et al. (1995) specify three categories
of rule extraction methods:

Decompositional approach

Decomposition refers to breaking down the network into smaller individual
parts. For the decompositional approach, the architecture of the network
and/or its outputs are used in the process. Zilke et al. (2016) uses a
decompositional algorithm that extracts rules for each layer in the DNN. These
rules are merged together in a final merging step to produce a set of rules
that describe the network behaviour by means of its inputs. Murdoch and Szlam
(2017) succeeded in extracting rules from an LSTM by applying a
decompositional approach.

Pedagogical approach

Introduced by Craven and Shavlik (1994) and named by Andrews et al. (1995) the
pedagogical approach involves “viewing rule extraction as a learning task
where the target concept is the function computed by the network and the input
features are simply the network’s input features” (Craven and Shavlik, 1994).
The pedagogical approach has the advantage that it is inherently model-
agnostic. Recent examples are found in (Ribeiro et al., 2016a; Lakkaraju et
al., 2017).

Eclectic approach

According to Andrews et al. (1995) “membership in this category is assigned to
techniques which utilize knowledge about the internal architecture and/or
weight vectors in the trained artificial neural network to complement a
symbolic learning algorithm.”

In terms of fidelity, local explanations are more faithful than global
explanations. For rule-extraction this means that rules that govern the result
of a specific input, or a neighborhood of inputs are more faithful than rules
that govern all possible inputs. Rule extraction is arguably the most
interpretable category of methods in our taxonomy considering that the
resulting set of rules can be unambiguously be interpreted by a human being as
a kind of formal language. Therefore we can say that it has a high degree of
clarity. In terms of parsimony we can say that if the ruleset is ”small
enough” the parsimony is higher than when the ruleset is “too large”. What
determines “small enough” and “too large” is difficult to quantify formally
and is also dependent on the user (expert vs. lay). In terms of
generalizability it can go both ways: if a decompositional approach is used it
is likely that the method is not generalizable, while if a pedagogical
approach is used the method is highly generalizable. In terms of explanatory
power, rule-extraction methods can 1) validate whether the network is working
as expected in terms of overall logic flow, and 2) explain which aspects of
the input data had an effect that lead to the specific output.

####  5.2.2 Attribution methods

Attribution, a term introduced by Ancona et al. (2018), also referred to as
relevance (Bach et al., 2015; Binder et al., 2016; Zintgraf et al., 2017;
Robnik-Šikonja and Kononenko, 2008), contribution (Shrikumar et al., 2017),
class saliency (Simonyan et al., 2013) or influence (Kindermans et al., 2016;
Adler et al., 2016; Koh and Liang, 2017), aims to reveal components of high
importance in the input to the DNN and their effect as the input is propagated
through the network. Because of this property we can categorize the following
methods to the attribution category: occlusion (Guçlütürk et al., 2017),
erasure (Li et al., 2016), perturbation (Fong and Vedaldi, 2017), adversarial
examples (Papernot et al., 2017) and prediction difference analysis (Zintgraf
et al., 2017). Other methods that belong to this category are found in
(Baehrens et al., 2010; Murdoch et al., 2018; Ribeiro et al., 2016b). It is
worth mentioning that attribution methods do not only apply to image input but
also to other forms of input, such as text processing by LSTMs (Murdoch et
al., 2018). The definition of attribution methods in this chapter is similar
to that of saliency methods (Kindermans et al., 2017), but more general than
the definition of attribution methods in (Kindermans et al., 2017) akin to the
definition in (Ancona et al., 2018).

The majority of explanation methods for DNNs visualize the information
obtained by attribution methods. Visualization methods were popularized by
(Erhan et al., 2009; Simonyan et al., 2013; Zeiler and Fergus, 2014) in recent
years and are concerned with how the important features are visualized. Zeng
(2016) identifies that current methods focus on three aspects of
visualization: feature visualization, relationship visualization and process
visualization. Overall visualization methods are very intuitive methods to
gain a variety of insight about a DNN decision process on many levels
including architecture assessment, model quality assessment and even user
feedback integration, e.g. Olah et al. (2018) create intuitive visualization
interfaces for image processing DNNs.

Kindermans et al. (2017) has shown recently that attribution methods “lack
reliability when the explanation is sensitive to factors that do not
contribute to the model prediction.” Furthermore they introduce the notion of
input invariance as a prerequisite for accurate attribution. In other words,
if the attribution method does not satisfy input invariance, we can consider
it to have low fidelity. In terms of clarity, there is a degree of ambiguity
that is inherent with these methods because visual explanations can be
interpreted in multiple ways by different users, even by users in the same
user category. In contrast to the precise results of rule-extraction methods,
the information that results from attribution methods has less structure. In
addition, the degree of clarity is dependent on the degree of fidelity of the
method: low fidelity can cause incorrect attribution, resulting in noisy
output with distracting attributions that increase ambiguity. The degree of
parsimony depends on the method of visualization itself. Methods that
visualize only the significant attributions exhibit a higher degree of
parsimony. The degree of generalizability depends on which components are used
to determine attribution. Methods that only use the input and output are
inherently model agnostic, resulting in the highest degree of
generalizability. Following this logic, methods that make use of internal
components are generalizable to the degree that other models share these
components. For example, deconvolutional networks (Zeiler et al., 2010) can be
applied to models that make use of convolutions to extract features from input
images. In terms of explanatory power, this class of methods can reflect
intuitively with visual explanations which factors in the input dimension had
a significant impact on the output of the DNN. However these methods do not
explain the reason for the importance of the particular factor attribution.

####  5.2.3 Intrinsic methods

The previous categories are designed to make explainable some aspects of a DNN
in a process separate from training the DNN. In contrast, this category aims
to improve the interpretability of internal representations with methods that
are part of the DNN architecture, e.g. as part of the loss function (Dong et
al., 2017b, a), modules that add additional capabilities (Santoro et al.,
2017; Palm et al., 2017), or as part of the architecture structure, in terms
of operations between layers (Li et al., 2017; Wu et al., 2017; Louizos et
al., 2017; Goudet et al., 2017). Dong et al. (2017b) provide an interpretive
loss function to increase the visual fidelity of the learned features. More
importantly Dong et al. (2017a) show that by training DNNs with adversarial
data and a consistent loss, we can trace back errors made by the DNN to
individual neurons and identify whether the data was adversarial. Santoro et
al. (2017) give a DNN the ability to answer relational reasoning questions
about a specific environment, by introducing a relational reasoning module
that learns a relational function, which can be applied to any DNN. Palm et
al. (2017) build on work by Santoro et al. (2017) and introduces a recurrent
relational network which can take the temporal component into account. Li et
al. (2017) introduce an explicit structure to DNNs for visual recognition by
building in an AND-OR grammar directly in the network structure. This leads to
better interpretation of the information flow in the network, hence increased
parsimony in attribution methods. Louizos et al. (2017) make use of generative
neural networks perform causal inference and Goudet et al. (2017) use
generative neural networks to learn functional causal models. Intrinsic
methods do not explicitly explain anything by themselves. Instead they
increase fidelity, clarity and parsimony in attribution methods. This class of
methods is different from attribution methods because it tries to make the DNN
inherently more interpretable by changing the architecture of the DNN, where
attribution methods use what is there already and only transform aspects of
the representation to something meaningful after the network is trained.

##  6 Addressing general concerns

As indicated in Figure 1, users have certain values, that in relation to a
particular technology may lead to concerns, that in relation to particular
applications can lead to specific questions. Mittelstadt et al. (2016) and
Danks and London (2017) distinguish various concerns that users may have. The
kinds of concerns they discuss focus to a large extent on the
inconclusiveness, inscrutability or misguidedness of used evidence. That is,
they concern to a significant extent the reliability and accessibility of used
data (data mining, generally speaking). In addition to apprehensions about
data, there are concerns that involve aspects of the processing itself, e.g.
the inferential validity of an algorithm. Also, questions may be raised about
the validity of a training process (e.g. requiring information about how
exactly a DNN is trained). In the following, we provide a list of general
concerns that should be addressed when developing predictive models such as
DNNs:

Flawed data collection

Data collection may be flawed in several ways. Large labeled datasets that are
used to train DNNs are either acquired by researchers (often via
crowdsourcing) or by companies that ‘own’ the data. However, data quality may
depend on multiple factors such as noise or censoring and there is no strict
control on whether data is annotated correctly. Furthermore, the
characteristics of the workers who annotated the data may introduce unwanted
biases (Barocas and Selbst, 2016). These biases may be due to preferences that
do not generalize across cultures or due to stereotyping, where sensitivity to
irrelevant attributes such as race or gender may induce unfair actionable
outcomes. The same holds for the selection of the data that is used for
annotation in the first place. Used data may reflect the status quo, which is
not necessarily devoid of biases (Caliskan et al., 2017). Furthermore,
selection bias may have as a result that data collected in one setting need
not generalize to other settings. For example, video data used to train
autonomous driving systems may not generalize to other locations or
conditions.

Inscrutable data use

The exact use of the data to train DNNs may also be opaque. Users may worry
about what (part of the) data exactly has led to the outcome. Often it is not
even known to the data subject which personal data is being used for what
purposes. A case in point is the use of person data for risk profiling by
governmental institutions. For example, criticisms have been raised about the
way the Dutch SyRI system uses data to detect
fraud.333https://bijvoorbaatverdacht.nl Furthermore, the involvement of expert
users who may be prone to biases as well may have an implicit influence on DNN
training.

Suboptimal inferences

The inferences made by DNNs are of a correlational rather than a causal
nature. This implies that subtle correlations between input features may
influence network output, which themselves may be driven by various biases.
Work is in progress to mitigate or remove the influence of sensitive variables
that should not affect decision outcomes by embracing causal inference
procedures (Chiappa and Gillam, 2018). Note further that the impact of
suboptimal inferences is domain dependent. For example, in medicine and the
social sciences, suboptimal inferences may directly affect the lives of
individuals or whole populations whereas in the exact sciences, suboptimal
inferences may affect evidence for or against a specific scientific theory.

Undesirable outcomes

End users or data subjects may feel that the outcome of the DNN is somehow
undesirable in relation to the particular values they hold, e.g. violating
fairness or privacy. Importantly, actionable outcomes should take into account
preferences of the stakeholder, which can be an individual (e.g. when deciding
on further medical investigation) as well as the community as a whole (e.g. in
case of policies about autonomous driving or predictive policing). These
considerations demand the involvement of domain experts and ethicists already
in the earliest stages of model development. Finally, model predictions may be
of a statistical rather than deterministic nature. This speaks for the
inclusion of decision-theoretic constructs in deciding on optimal actionable
outcomes (von Neumann and Morgenstern, 1953).

Adversarial attacks

Images (Szegedy et al., 2013; Cubuk et al., 2017) and audio (Carlini and
Wagner, 2018) can easily be distorted with modifications that are
imperceptible to humans. Such distortions cause DNNs to make incorrect
inferences and can be done with the purpose of intentionally misleading DNNs
(e.g. yielding predictions in favor of the perpetrator). Work in progress
shows that there are methods to detect adversarial instances (Rawat et al.,
2017) and to mitigate the attacks (Lin et al., 2017). However further research
is needed to increase the robustness of DNNs against adversarial attacks as
there are no methods in existence that fully diminish the effects of
adversarial attacks.

As stated by Doran et al. (2017), explanation methods may make predictive
models such as DNNs more comprehensible. However, explanation methods alone
not completely resolve the raised concerns.

##  7 Discussion

In this chapter we set out to analyze the question of “What can be explained?”
given the users and their needs, laws and regulations, and existing
explanation methods. Specifically, we looked at the capabilities of
explanation methods and analyzed which questions/concerns about explainability
these methods address in the context of DNNs.

Overall, it is clear that (visual) explanations can be given about various
aspects of the influence of the input on the output (e.g. given the input
data, which aspects of the data lead to the output?), by making use of both
rule-extraction and attribution methods. Also, when used in combination with
attribution methods, intrinsic methods lead to more explainable DNNs. It is
likely that in the future we will see the rise of a new category of
explanation methods that combine aspects of rule-extraction, attribution and
intrinsic methods, to answer specific questions in a simple human
interpretable language.

Furthermore, it is obvious that current explanation methods are tailored to
expert users, since the interpretation of the results require knowledge of the
DNN process. As far as we are aware, explanation methods, e.g. intuitive
explanation interfaces, for lay users do not exist. Ideally, if such
explanation methods would exist, they should be able to answer, in a simple
human language, questions about every operation that the application performs.
This is not an easy task since the number of conceivable questions one could
ask about the working of an application is substantial.

Two particular concerns, which are difficult to address with explanation
methods, is the concern about bias in datasets that leads to biased DNNs, as
well as the suspicion about unfair outcomes: Can we indicate that the DNN is
biased, and if so, can we remove the bias? Has the DNN been applied
responsibly? These are not problems that are directly solvable with
explanation methods. However, explanation methods alleviate the first problem
to the extent that learned features can be visualized (using attribution
methods) and further analyzed for bias using other methods that are not
explanation methods. For the second problem, more general measures, such as
regulations and laws, will need to be developed.

## References

  * Adler et al. (2016) Philip Adler, Casey Falk, Sorelle A. Friedler, Gabriel Rybeck, Carlos Scheidegger, Brandon Smith, and Suresh Venkatasubramanian.  Auditing black-box models for indirect influence.  In _2016 IEEE 16th International Conference on Data Mining (ICDM)_. IEEE, 2016.
  * Ancona et al. (2018) Marco Ancona, Enea Ceolini, Cengiz Oztireli, and Markus Gross.  Towards better understanding of gradient-based attribution methods for deep neural networks.  In _6th International Conference on Learning Representations (ICLR 2018)_ , 2018.
  * Andrews et al. (1995) Robert Andrews, Joachim Diederich, and Alan B. Tickle.  Survey and critique of techniques for extracting rules from trained artificial neural networks.  _Knowledge-Based Systems_ , 8(6):373–389, 1995\.
  * Arbatli and Akin (1997) A. Duygu Arbatli and H. Levent Akin.  Rule extraction from trained neural networks using genetic algorithms.  _Nonlinear Analysis: Theory, Methods & Applications_, 30(3):1639–1648, 1997.
  * Bach et al. (2015) Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek.  On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation.  _PLOS ONE_ , 10(7), 2015.
  * Baehrens et al. (2010) David Baehrens, Timon Schroeter, Stefan Harmeling, Motoaki Kawanabe, Katja Hansen, and Klaus-Robert Müller.  How to explain individual classification decisions.  _Journal of Machine Learning Research (JMLR)_ , 11:1803–1831, 2010.
  * Barocas and Selbst (2016) Solon Barocas and Andrew D. Selbst.  Big data’s disparate impact.  _Cal. L. Rev._ , 104:671, 2016.
  * Binder et al. (2016) Alexander Binder, Sebastian Bach, Grégoire Montavon, Klaus-Robert Müller, and Wojciech Samek.  Layer-wise relevance propagation for deep neural network architectures.  In _Information Science and Applications (ICISA) 2016_ , pages 913–922. Springer, 2016.
  * Bojarski et al. (2017) Mariusz Bojarski, Philip Yeres, Anna Choromanska, Krzysztof Choromanski, Bernhard Firner, Lawrence Jackel, and Urs Muller.  Explaining how a deep neural network trained with end-to-end learning steers a car.  _arXiv preprint arXiv:1704.07911_ , 2017.
  * Caliskan et al. (2017) Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan.  Semantics derived automatically from language corpora contain human-like biases.  _Science_ , 356(6334):183–186, 2017.
  * Carlini and Wagner (2018) Nicholas Carlini and David Wagner.  Audio adversarial examples: Targeted attacks on speech-to-text.  _arXiv preprint arXiv:1801.01944_ , 2018.
  * Chiappa and Gillam (2018) Silvia Chiappa and Thomas P.S. Gillam.  Path-specific counterfactual fairness.  _arXiv preprint arXiv:1802.08139_ , 2018.
  * Craven and Shavlik (1994) Mark W. Craven and Jude W. Shavlik.  Using sampling and queries to extract rules from trained neural networks.  In _Machine Learning Proceedings 1994_ , pages 37–45. Elsevier, 1994\.
  * Cubuk et al. (2017) Ekin D. Cubuk, Barret Zoph, Samuel S. Schoenholz, and Quoc V. Le.  Intriguing properties of adversarial examples.  _arXiv preprint arXiv:1711.02846_ , 2017.
  * Danks and London (2017) David Danks and Alex John London.  Algorithmic bias in autonomous systems.  In _Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI)_ , pages 4691–4697. AAAI Press, 2017.
  * Dong et al. (2017a) Yinpeng Dong, Hang Su, Jun Zhu, and Fan Bao.  Towards interpretable deep neural networks by leveraging adversarial examples.  _arXiv preprint arXiv:1708.05493_ , 2017a.
  * Dong et al. (2017b) Yinpeng Dong, Hang Su, Jun Zhu, and Bo Zhang.  Improving interpretability of deep neural networks with semantic information.  In _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE, 2017b.
  * Doran et al. (2017) Derek Doran, Sarah Schulz, and Tarek R. Besold.  What does explainable AI really mean? a new conceptualization of perspectives.  _arXiv preprint arXiv:1710.00794_ , 2017.
  * Doshi-Velez and Kim (2017) Finale Doshi-Velez and Been Kim.  Towards a rigorous science of interpretable machine learning.  _arXiv preprint arXiv:1702.08608_ , 2017.
  * Doshi-Velez et al. (2017) Finale Doshi-Velez, Mason Kortz, Ryan Budish, Christopher Bavitz, Samuel J. Gershman, David O’Brien, Stuart Shieber, Jim Waldo, David Weinberger, and Alexandra Wood.  Accountability of AI under the law: The role of explanation.  _SSRN Electronic Journal_ , 2017.
  * Elman (1989) Jeffrey L. Elman.  Representation and structure in connectionist models.  Technical report, 1989.
  * Erhan et al. (2009) Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent.  Visualizing higher-layer features of a deep network.  _University of Montreal_ , 1341:3, 2009.
  * Floridi et al. (2015) Luciano Floridi, Nir Fresco, and Giuseppe Primiero.  On malfunctioning software.  _Synthese_ , 192(4):1199–1220, 2015.
  * Fong and Vedaldi (2017) Ruth C. Fong and Andrea Vedaldi.  Interpretable explanations of black boxes by meaningful perturbation.  In _2017 IEEE International Conference on Computer Vision (ICCV)_. IEEE, 2017.
  * Goudet et al. (2017) Olivier Goudet, Diviyan Kalainathan, Philippe Caillou, David Lopez-Paz, Isabelle Guyon, Michele Sebag, Aris Tritas, and Paola Tubaro.  Learning functional causal models with generative neural networks.  _arXiv preprint arXiv:1709.05321_ , 2017.
  * Grün et al. (2016) Felix Grün, Christian Rupprecht, Nassir Navab, and Federico Tombari.  A taxonomy and library for visualizing learned features in convolutional neural networks.  _arXiv preprint arXiv:1606.07757_ , 2016.
  * Guçlütürk et al. (2017) Yaǧmur Guçlütürk, Umut Güçlü, Marc Perez, Hugo Jair Escalante, Xavier Baro, Isabelle Guyon, Carlos Andujar, Julio Jacques Junior, Meysam Madadi, Sergio Escalera, Marcel A. J. van Gerven, and Rob van Lier.  Visualizing apparent personality analysis with deep residual networks.  In _2017 IEEE International Conference on Computer Vision Workshops (ICCVW)_ , pages 3101–3109, 2017.
  * Gunning (2017) David Gunning.  Explainable artificial intelligence (XAI).  _Defense Advanced Research Projects Agency (DARPA)_ , 2017.
  * Hall et al. (2017) Patrick Hall, Wen Phan, and SriSatish Ambati.  Ideas on interpreting machine learning.  2017\.  Available online at: https://www.oreilly.com/ideas/ideas-on-interpreting-machine-learning.
  * Holzinger et al. (2017a) Andreas Holzinger, Chris Biemann, Constantinos S. Pattichis, and Douglas B. Kell.  What do we need to build explainable AI systems for the medical domain?  _arXiv preprint arXiv:1712.09923_ , 2017a.
  * Holzinger et al. (2017b) Andreas Holzinger, Markus Plass, Katharina Holzinger, Gloria Cerasela Crişan, Camelia-Mihaela Pintea, and Vasile Palade.  A glass-box interactive machine learning approach for solving NP-hard problems with the human-in-the-loop.  _arXiv preprint arXiv:1708.01104_ , 2017b.
  * Kindermans et al. (2016) Pieter-Jan Kindermans, Kristof T. Schütt, Klaus-Robert Müller, and Sven Dähne.  Investigating the influence of noise and distractors on the interpretation of neural networks.  _arXiv preprint arXiv:1611.07270_ , 2016.
  * Kindermans et al. (2017) Pieter-Jan Kindermans, Sara Hooker, Julius Adebayo, Maximilian Alber, Kristof T. Schütt, Sven Dähne, Dumitru Erhan, and Been Kim.  The (un) reliability of saliency methods.  _arXiv preprint arXiv:1711.00867_ , 2017.
  * Koh and Liang (2017) Pang Wei Koh and Percy Liang.  Understanding black-box predictions via influence functions.  In _Proceedings of the 34th International Conference on Machine Learning (ICML)_ , volume 70 of _Proceedings of Machine Learning Research (PMLR)_ , pages 1885–1894, 2017.
  * Lakkaraju et al. (2017) Himabindu Lakkaraju, Ece Kamar, Rich Caruana, and Jure Leskovec.  Interpretable & explorable approximations of black box models.  _arXiv preprint arXiv:1707.01154_ , 2017.
  * Lee et al. (2017) Hyunkwang Lee, Shahein Tajmir, Jenny Lee, Maurice Zissen, Bethel Ayele Yeshiwas, Tarik K. Alkasab, Garry Choy, and Synho Do.  Fully automated deep learning system for bone age assessment.  _Journal of Digital Imaging (JDI)_ , 30(4):427–441, 2017.
  * Li et al. (2016) Jiwei Li, Will Monroe, and Dan Jurafsky.  Understanding neural networks through representation erasure.  _arXiv preprint arXiv:1612.08220_ , 2016.
  * Li et al. (2017) Xilai Li, Tianfu Wu, Xi Song, and Hamid Krim.  AOGNets: Deep AND-OR grammar networks for visual recognition.  _arXiv preprint arXiv:1711.05847_ , 2017.
  * Lin et al. (2017) Yen-Chen Lin, Ming-Yu Liu, Min Sun, and Jia-Bin Huang.  Detecting adversarial attacks on neural network policies with visual foresight.  _arXiv preprint arXiv:1710.00814_ , 2017.
  * (40) Alan Lockett, Trevor Jefferies, Neil Etheridge, and Alicia Brewer.  White paper tag predictions: How DISCO AI is bringing deep learning to legal technology.  Available online at: https://www.csdisco.com/disco-ai.
  * Louizos et al. (2017) Christos Louizos, Uri Shalit, Joris M. Mooij, David Sontag, Richard Zemel, and Max Welling.  Causal effect inference with deep latent-variable models.  In _Advances in Neural Information Processing Systems 30 (NIPS)_ , pages 6446–6456. 2017.
  * Lu et al. (2006) Jianjun Lu, Shozo Tokinaga, and Yoshikazu Ikeda.  Explanatory rule extraction based on the trained neural network and the genetic programming.  _Journal of the Operations Research Society of Japan (JORSJ)_ , 49(1):66–82, 2006.
  * Marcus (2018) Gary Marcus.  Deep learning: A critical appraisal.  _arXiv preprint arXiv:1801.00631_ , 2018.
  * Markowska-Kaczmar and Wnuk-Lipiński (2004) Urszula Markowska-Kaczmar and Paweł Wnuk-Lipiński.  Rule extraction from neural network by genetic algorithm with pareto optimization.  _Artificial Intelligence and Soft Computing-ICAISC 2004_ , pages 450–455, 2004.
  * Mittelstadt et al. (2016) Brent Daniel Mittelstadt, Patrick Allo, Mariarosaria Taddeo, Sandra Wachter, and Luciano Floridi.  The ethics of algorithms: Mapping the debate.  _Big Data & Society_, 3(2), 2016.
  * Montavon et al. (2018) Grégoire Montavon, Wojciech Samek, and Klaus-Robert Müller.  Methods for interpreting and understanding deep neural networks.  _Digital Signal Processing_ , 73:1–15, 2018.
  * Murdoch and Szlam (2017) W. James Murdoch and Arthur Szlam.  Automatic rule extraction from long short term memory networks.  In _International Conference on Learning Representations (ICLR)_ , 2017.
  * Murdoch et al. (2018) W. James Murdoch, Peter J. Liu, and Bin Yu.  Beyond word importance: Contextual decomposition to extract interactions from LSTMs.  In _International Conference on Learning Representations (ICLR)_ , 2018.
  * Olah et al. (2017) Chris Olah, Alexander Mordvintsev, and Ludwig Schubert.  Feature visualization.  _Distill_ , 2017.  Available online at: https://distill.pub/2017/feature-visualization.
  * Olah et al. (2018) Chris Olah, Arvind Satyanarayan, Ian Johnson, Shan Carter, Ludwig Schubert, Katherine Ye, and Alexander Mordvintsev.  The building blocks of interpretability.  _Distill_ , 2018.  Available online at: https://distill.pub/2018/building-blocks.
  * Palm et al. (2017) Rasmus Berg Palm, Ulrich Paquet, and Ole Winther.  Recurrent relational networks for complex relational reasoning.  _arXiv preprint arXiv:1711.08028_ , 2017.
  * Papernot et al. (2017) Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami.  Practical black-box attacks against machine learning.  In _Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security (ASIA CCS ’17)_ , pages 506–519, 2017.
  * Rawat et al. (2017) Ambrish Rawat, Martin Wistuba, and Maria-Irina Nicolae.  Adversarial phenomenon in the eyes of bayesian deep learning.  _arXiv preprint arXiv:1711.08244_ , 2017.
  * Ribeiro et al. (2016a) Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin.  Nothing else matters: Model-agnostic explanations by identifying prediction invariance.  In _NIPS Workshop on Interpretable Machine Learning in Complex Systems_ , 2016a.
  * Ribeiro et al. (2016b) Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin.  ”Why should I trust you?” Explaining the predictions of any classifier.  In _Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’16)_ , pages 1135–1144, 2016b.
  * Robnik-Šikonja and Kononenko (2008) Marko Robnik-Šikonja and Igor Kononenko.  Explaining classifications for individual instances.  _IEEE Transactions on Knowledge and Data Engineering_ , 20(5):589–600, 2008.
  * Samek et al. (2017) Wojciech Samek, Thomas Wiegand, and Klaus-Robert Müller.  Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models.  _arXiv preprint arXiv:1708.08296_ , 2017.
  * Santoro et al. (2017) Adam Santoro, David Raposo, David G.T. Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, and Timothy Lillicrap.  A simple neural network module for relational reasoning.  _arXiv preprint arXiv:1706.01427_ , 2017.
  * Seifert et al. (2017) Christin Seifert, Aisha Aamir, Aparna Balagopalan, Dhruv Jain, Abhinav Sharma, Sebastian Grottel, and Stefan Gumhold.  Visualizations of deep neural networks in computer vision: A survey.  In _Transparent Data Mining for Big and Small Data_ , pages 123–144. Springer, 2017.
  * Shrikumar et al. (2017) Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje.  Learning important features through propagating activation differences.  In _Proceedings of the 34th International Conference on Machine Learning (ICML)_ , volume 70 of _Proceedings of Machine Learning Research (PMLR)_ , 2017.
  * Simonyan et al. (2013) Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman.  Deep inside convolutional networks: Visualising image classification models and saliency maps.  _arXiv preprint arXiv:1312.6034_ , 2013.
  * Szegedy et al. (2013) Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus.  Intriguing properties of neural networks.  _arXiv preprint arXiv:1312.6199_ , 2013.
  * von Neumann and Morgenstern (1953) John von Neumann and Oskar Morgenstern.  _Theory of Games and Economic Behavior_.  Princeton University Press, Princeton, NJ, 3rd edition, 1953.
  * Wachter et al. (2017) Sandra Wachter, Brent Mittelstadt, and Luciano Floridi.  Transparent, explainable, and accountable AI for robotics.  _Science Robotics_ , 2(6), 2017.
  * Weller (2017) Adrian Weller.  Challenges for transparency.  _Workshop on Human Interpretability in Machine Learning – ICML 2017_ , 2017.
  * Wu et al. (2017) Tianfu Wu, Xilai Li, Xi Song, Wei Sun, Liang Dong, and Bo Li.  Interpretable R-CNN.  _arXiv preprint arXiv:1711.05226_ , 2017.
  * Zeiler and Fergus (2014) Matthew D. Zeiler and Rob Fergus.  Visualizing and understanding convolutional networks.  In _European Conference on Computer Vision (ECCV)_ , pages 818–833. Springer, 2014.
  * Zeiler et al. (2010) Matthew D. Zeiler, Dilip Krishnan, Graham W. Taylor, and Rob Fergus.  Deconvolutional networks.  In _2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 2528–2535. IEEE, 2010.
  * Zeng (2016) Haipeng Zeng.  Towards better understanding of deep learning with visualization.  2016\.
  * Zilke et al. (2016) Jan Ruben Zilke, Eneldo Loza Mencía, and Frederik Janssen.  DeepRED – Rule extraction from deep neural networks.  In _International Conference on Discovery Science (ICDS)_ , pages 457–473. Springer, 2016.
  * Zintgraf et al. (2017) Luisa M. Zintgraf, Taco S. Cohen, Tameem Adel, and Max Welling.  Visualizing deep neural network decisions: Prediction difference analysis.  In _International Conference on Learning Representations (ICLR)_ , 2017.


# Constraining PDFs and nPDFs with recent data††thanks: Presented at the 29th
International Conference on Ultra-relativistic Nucleus-Nucleus Collisions,
April 4-10, 2022, Kraków, Poland.

Petja Paakkinen  University of Jyväskylä, Department of Physics
P.O. Box 35, FI-40014 University of Jyväskylä, Finland
and
Helsinki Institute of Physics
P.O. Box 64, FI-00014 University of Helsinki, Finland

(July 28, 2022)

###### Abstract

The progress in constraining proton and nuclear parton distribution functions
is briefly summarised. Some persistent uncertainties are pointed out and
recent experimental advancements highlighted.

##  1 Introduction

In many applications of high-energy hadronic collisions, including those of
heavy nuclei, the partonic structure of hadrons manifests itself through the
collinear factorisation of QCD [1]. That is, the cross section for producing
an inclusive final state $k+X$, where the $X$ denotes “anything”, can be
described in terms of coefficient functions (CF) ${\rm
d}\hat{\sigma}^{ij\rightarrow k+X^{\prime}}$, which are calculable from
perturbative QCD, and parton distribution functions (PDFs)
$f_{i}^{A},f_{j}^{B}$, which contain long-range physics and thus cannot be
obtained by perturbative means, plus “higher twist” (HT) corrections which are
suppressed at high enough momentum-transfer scale $Q\gg\Lambda_{\rm QCD}$.
Formally,

${\rm d}\sigma^{AB\rightarrow k+X}(Q^{2})\\!\overset{Q\gg\Lambda_{\rm
QCD}}{=}\\!\sum_{i,j,X^{\prime}}f_{i}^{A}(Q^{2})\otimes{\rm
d}\hat{\sigma}^{ij\rightarrow k+X^{\prime}}(Q^{2})\otimes
f_{j}^{B}(Q^{2})+{\cal O}(1/Q^{2}),$ (1)

where “$\otimes$” indicates a multiplicative (Mellin) convolution over $x$,
the fraction of the parents momentum carried by the parton.

The PDFs $f_{i}^{A}(x,Q^{2})$, with $i$ denoting the parton flavour and $A$
the parent hadron or nucleus, exhibit two important properties: First, they
are universal, independent of the partonic process (i.e. the CFs). And second,
their scale dependence is governed by the DGLAP equations $\partial
f_{i}^{A}/\partial\ln Q^{2}=\sum_{j}P_{ij}\otimes f_{j}^{A}$ with
perturbatively calculable splitting functions $P_{ij}$. For these properties,
it is possible to extract the PDFs from global analyses of experimental data
of multiple observables at different scales.

Fig. 1 shows results from such analyses with the CT18A next-to-leading order
(NLO) proton PDFs [2] and EPPS21 NLO nuclear PDFs (nPDFs) [3], where the
latter uses the former as its free-proton baseline, taken as representative
examples. As is common, both analyses fit six independent parton flavours,
$u,d,\bar{u},\bar{d},s=\bar{s}$ and glue, taking heavy-quark PDFs fully
perturbatively generated.111Some analyses, like the recent NNPDF4.0 extraction
[4], extend this set to fit also intrinsic charm and strange asymmetry. The
general features are evident from this plot: For protons, most flavours are
already quite well constrained in a wide range of $x$. The exception to this
rule is the strange quark, which is currently the least constrained flavour in
both free-proton and nuclear PDF analyses. For nPDFs, the uncertainties
persist to be larger compared to the free-proton ones, particularly in the
small-$x$ region where direct data constraints are scarce.

Figure 1: A representative example of proton and nuclear PDFs taken from the
CT18A NLO proton-PDFs [2] and EPPS21 NLO nPDFs [3]. The coloured bands show
the estimated PDF uncertainties at 90% confidence level.

##  2 Progress in proton-PDF analyses

The most recent proton-PDF analyses use several thousand data points from
various experiments. Such a global approach is a necessity: a multi-observable
fit is needed in order to constrain individual flavours and a multi-experiment
fit in order to do so in a wide range of $x$. For example, fixed target deep-
inelastic scattering (DIS) and Drell-Yan (DY) data are important in setting
the large-$x$ quark distributions (valence-to-sea and flavour separation).
While much of these data are already rather old, new data are still coming
from Fermilab and JLab, where e.g. the new SeaQuest Collaboration measurement
of proton-deuteron to proton-proton ratio of DY dilepton production [5]
improves our understanding of the antimatter asymmetry in the proton. To reach
smaller values of $x$, collider DIS data are needed, and nowadays the HERA
inclusive and heavy-quark data [6, 7] form the backbone of any modern proton-
PDF analysis. With the large $x,Q^{2}$ lever arm, these data are able to
constrain also the gluon distribution through the DGLAP equations. Recently,
however, most of the new data have come from hadron colliders (in particular
from the CERN LHC), granting access to a wide spectrum of new processes such
as the production of electroweak (EW) bosons, jets, top-pairs, etc., allowing
for testing the factorisation and PDF flavour dependence in new kinematic
windows.

One of the most important latest developments in this field has been the
inclusion of next-to-next-to-leading-order (NNLO) corrections [8, 9] to the
perturbative jet-production calculations in the most recent global
analyses.222At leading colour, that is. Calculations at full colour have been
performed only very recently [10, 11]. These jet observables, together with
the $t\bar{t}$ production, are important in setting the large-$x$ gluon
distributions [4]. The NNLO precision improves fit quality particularly for
the LHC (EW boson and jet) data, and for example the recent MSHT20 analysis
[12] reports on an improvement of 700 units in $\chi^{2}_{\rm tot}$ for the
4363 global data points. Therefore, with the present LHC data precision, NNLO
proton-PDFs are no longer a mere formal improvement, reducing theoretical
uncertainties from missing-higher-order corrections, but a necessity for an
accurate fit.

Figure 2: Strangeness suppression in different proton-PDF analyses. Figure
from Ref. [13].

LHC data can also challenge some of the earlier approaches taken in global
analyses. Traditionally, proton-PDF fits have included neutrino-nucleus ($\nu
A$) DIS data for improved strange-quark constraints, which have led to
suppressed strangeness e.g. in the ABMP16 fit [14], as is quantified by the
fraction

$K_{s}=\frac{\int_{0}^{1}{\rm
d}xx[{s}(x)\\!+\\!{\bar{s}}(x)]}{\int_{0}^{1}{\rm
d}xx[{\bar{u}}(x)\\!+\\!{\bar{d}}(x)]}$ (2)

shown in Fig. 2. Complementary data from ATLAS EW-boson production have then
confronted this view with a preference for unsuppressed strange (see ATLAS-
epWZ16 in Fig. 2) [15, 16]. Finding some tension between these two data types,
the CT18 [2] analysis provides two different fits, where the nominal CT18 set
does not include the ATLAS data, but the CT18A version does. Later, a
simultaneous fit was found to be feasible when NNLO charm-quark mass
corrections in the $\nu A$ cross sections were accounted for, as in the
nNNPDF4.0 and MSHT20 analyses [4, 12].

All of this points to the direction that understanding the role of nuclear
corrections in fitting proton PDFs with nuclear data is becoming increasingly
important. Indeed, the recent NNPDF4.0 analysis finds very different large-$x$
sea-quark behaviour depending on whether the uncertainties from nNNPDF2.0
nPDFs were included or not [4], in contrast to some earlier evaluations [17].
How to take these nuclear corrections into account varies from analysis to
analysis, the MSHT20 taking them from the DSSZ nPDFs [18] plus an additional
three-parameter fit [12], whereas CT18 uses a scale-independent
phenomenological parametrisation as described in Ref. [19].

##  3 Progress in nPDF analyses

The nPDFs are fitted with similar global analyses as their free-proton
counterparts, relying only to the QCD collinear factorisation and thus giving
a model-agnostic way to study the nuclear effects. Unlike in proton-PDF
analyses, where NNLO is the standard, most of the nPDF analyses are still
performed at NLO and those recent analyses which also provide NNLO fits [20,
21] are performed with a rather limited number of data types and thus cannot
parametrise the flavour dependence very freely. Those (NLO) analyses standing
on a more global footing instead include a variety of data from the LHC
proton-lead (p+Pb) runs, with EPPS21 [3] and nNNPDF3.0 [22] also propagating
the uncertainties from their free-proton baselines.

A very important advancement in this field has been the inclusion of the
double-differential LHC p+Pb dijet and D0-production data in the recent
analyses, leading to strong new constraints on the nuclear gluon PDFs [3, 22]
which have been previously poorly known in lack of collider DIS data on
nuclei.333The heavy-quark data were recently considered also in Ref. [23]. In
accordance with the earlier reweighting study [24], these analyses obtain a
good fit to the dijet data, except in the very forward bins where the
predictions overshoot the data. This could be due to the missing data
correlations, but also larger than expected NNLO or non-perturbative
corrections could play a role. For the D0-production, nNNPDF3.0, with its
POWHEG+PYTHIA approach, finds a large scale uncertainty in the nuclear ratio
and therefore fit only to the more constraining forward data, whereas EPPS21,
with the S-ACOT-$m_{\rm T}$ mass scheme [25], finds no large theory
uncertainty in this ratio in the fitted $p_{\rm T}>3\ {\rm GeV}$ region [26].
Complementary gluon constraints can be derived from single-inclusive
production of $\pi^{0}$, $\pi^{\pm}$ and $K^{\pm}$ as in the nCTEQ15WZSIH
analysis [27] fitting to a large set of data from PHENIX, STAR and ALICE
measurements. In such fits, like with the D0-production, one is sensitive to
the fragmentation functions and their associated uncertainties, which however
partially cancel in the nuclear ratios.

In addition, $W^{\pm}$ production and the most precise CMS Collaboration
measurement from Run 2 [28] in particular, have been already included in
almost every recent nPDF fit. The approaches taken in doing so, however, vary,
TUJU21 [21], nNNPDF3.0 [22] and nCTEQ15WZSIH [27] fitting to absolute cross
sections, whereas EPPS21 [3] using nuclear-modification ratios to cancel
proton-PDF uncertainties [29]. Also new $Z$-boson data from Run 2 recently
became available [30] and have been studied in several nPDF analyses, all
finding some difficulties in describing the data. While the discrepancy in the
small-invariant-mass bin can be explained by NNLO corrections [21, 22], the
on-peak deviations appear to be due to rather abrupt change in the shape of
the data at midrapidity, possibly due to untypically large data fluctuations
[3].

Figure 3: Nuclear modifications of the gluon PDFs in lead (top) and oxygen
(bottom) nuclei. Results from Refs. [3, 22, 27].

The resulting gluon distributions from the EPPS21, nNNPDF3.0 and nCTEQ15WZSIH
analyses [3, 22, 27] are compared in Fig. 3. All of these nPDFs exhibit
small-$x$ gluon shadowing and mid-$x$ antishadowing in lead, which are now
established from multiple experimental observables. In finer details, the
results however differ. At small $x$, EPPS21 and nNNPDF3.0 agree extremely
well due to the use of D0 and dijet data, though with rather different
uncertainty estimations, whereas nCTEQ15WZSIH with the main small-$x$
constraints coming from EW-boson production favouring smaller shadowing. At
large $x$, nNNPDF3.0 shows a different behaviour from EPPS21 which now agrees
better with nCTEQ15WZSIH instead. This is possibly associated with the PHENIX
$\pi^{0}$ data included in EPPS21 and nCTEQ15WZSIH and also with the exclusion
of backward D0 data from nNNPDF3.0. It should be also noted that the discussed
direct gluon constraints come only from data on heavy nuclei and therefore the
gluon distributions in lighter nuclei, such as oxygen, remain poorly
constrained [31].

The progress does of course not end here. In Ref. [32] the advantages of
measuring triple-differential dijet production in p+Pb were discussed. Since
the triple-differential measurement fixes partonic kinematics at leading
order, this would give a powerful test of factorisation and PDFs. Also the new
$\pi^{0}$-production small-system scan from PHENIX [33] with improved
precision and access to higher $p_{\rm T}$ can help in deciphering the nuclear
modifications at large $x$. Contrary to nPDF expectations, the measured
“Cronin peak” size follows an ordering ${}^{3}{\rm He}$+${\rm Au}<d$+${\rm
Au}<p$+${\rm Au}$ and Ref. [33] discusses the cause of these effects in terms
of multiple-scattering (i.e. HT) and flow-like effects. In this conference,
also new LHCb measurements of D0s [34] and $\pi^{0}$s [35] at 8.16 TeV and
charged hadrons at 5.02 TeV [36] in p+Pb were presented, probing smaller
values of $x$. The hopes for using exclusive $J/\psi$ photoproduction in
constraining nPDFs were also elevated with the first phenomenological
implementation of the NLO corrections in ultrapheripheral Pb+Pb [37]. A more
direct probe of the collinearly factorised nPDFs in ultrapheripheral events is
the inclusive dijet photoproduction, for which the ATLAS measurement at 5.02
TeV has now been finally fully unfolded [38].

##  4 Conclusion

The new data constraints from LHC have brought the proton PDFs (and
increasingly also the nPDFs) to the point where taking NNLO corrections into
account is a necessity for a good fit, and the p+Pb data have also put
unprecedented constraints on the gluon nPDFs. There is also ongoing work to
understand the (cross)correlations between proton and nuclear PDF analyses,
and many new data sets (with the LHC Run 3 also underway) to be exploited in
the future.

I thank T. J. Hobbs and P. M. Nadolsky for clarifying the use of nuclear
corrections in CT18. Financial support from the Academy of Finland, project
330448, is acknowledged. This research was funded as a part of the Center of
Excellence in Quark Matter of the Academy of Finland and as a part of the
European Research Council project ERC-2018-ADG-835105 YoctoLHC. Figs. 1 and 3
use protanopia and deuteranopia safe colour schemes from Refs. [39] and [40].

## References

  * [1] J. C. Collins, D. E. Soper, and G. F. Sterman, “Factorization of Hard Processes in QCD,” _Adv. Ser. Direct. High Energy Phys._ , vol. 5, pp. 1–91, 1989.
  * [2] T.-J. Hou _et al._ , “New CTEQ global analysis of quantum chromodynamics with high-precision data from the LHC,” _Phys. Rev. D_ , vol. 103, no. 1, p. 014013, 2021.
  * [3] K. J. Eskola, P. Paakkinen, H. Paukkunen, and C. A. Salgado, “EPPS21: a global QCD analysis of nuclear PDFs,” _Eur. Phys. J. C_ , vol. 82, no. 5, p. 413, 2022.
  * [4] R. D. Ball _et al._ , “The path to proton structure at 1% accuracy,” _Eur. Phys. J. C_ , vol. 82, no. 5, p. 428, 2022.
  * [5] J. Dove _et al._ , “The asymmetry of antimatter in the proton,” _Nature_ , vol. 590, no. 7847, pp. 561–565, 2021, [Erratum: Nature 604, E26 (2022)].
  * [6] H. Abramowicz _et al._ , “Combination of measurements of inclusive deep inelastic ${e^{\pm}p}$ scattering cross sections and QCD analysis of HERA data,” _Eur. Phys. J. C_ , vol. 75, no. 12, p. 580, 2015.
  * [7] ——, “Combination and QCD analysis of charm and beauty production cross-section measurements in deep inelastic $ep$ scattering at HERA,” _Eur. Phys. J. C_ , vol. 78, no. 6, p. 473, 2018.
  * [8] J. Currie, E. W. N. Glover, and J. Pires, “Next-to-Next-to Leading Order QCD Predictions for Single Jet Inclusive Production at the LHC,” _Phys. Rev. Lett._ , vol. 118, no. 7, p. 072002, 2017.
  * [9] J. Currie _et al._ , “Precise predictions for dijet production at the LHC,” _Phys. Rev. Lett._ , vol. 119, no. 15, p. 152001, 2017.
  * [10] M. Czakon, A. van Hameren, A. Mitov, and R. Poncelet, “Single-jet inclusive rates with exact color at $\mathcal{O}$ (${\alpha}_{s}^{4}$),” _JHEP_ , vol. 10, p. 262, 2019.
  * [11] X. Chen, T. Gehrmann, E. W. N. Glover, A. Huss, and J. Mo, “NNLO QCD corrections in full colour for jet production observables at the LHC,” _JHEP_ , vol. 09, p. 025, 2022.
  * [12] S. Bailey _et al._ , “Parton distributions from LHC, HERA, Tevatron and fixed target data: MSHT20 PDFs,” _Eur. Phys. J. C_ , vol. 81, no. 4, p. 341, 2021.
  * [13] J. L. Feng _et al._ , “The Forward Physics Facility at the High-Luminosity LHC,” arXiv:2203.05090 [hep-ex].
  * [14] S. Alekhin, J. Blümlein, S. Moch, and R. Placakyte, “Parton distribution functions, $\alpha_{s}$, and heavy-quark masses for LHC Run II,” _Phys. Rev. D_ , vol. 96, no. 1, p. 014011, 2017.
  * [15] G. Aad _et al._ , “Determination of the strange quark density of the proton from ATLAS measurements of the $W\to\ell\nu$ and $Z\to\ell\ell$ cross sections,” _Phys. Rev. Lett._ , vol. 109, p. 012001, 2012.
  * [16] M. Aaboud _et al._ , “Precision measurement and interpretation of inclusive $W^{+}$ , $W^{-}$ and $Z/\gamma^{*}$ production cross sections with the ATLAS detector,” _Eur. Phys. J. C_ , vol. 77, no. 6, p. 367, 2017.
  * [17] R. D. Ball, E. R. Nocera, and R. L. Pearson, “Nuclear Uncertainties in the Determination of Proton PDFs,” _Eur. Phys. J. C_ , vol. 79, no. 3, p. 282, 2019.
  * [18] D. de Florian, R. Sassot, P. Zurita, and M. Stratmann, “Global Analysis of Nuclear Parton Distributions,” _Phys. Rev. D_ , vol. 85, p. 074028, 2012\.
  * [19] A. Accardi, T. J. Hobbs, X. Jing, and P. M. Nadolsky, “Deuterium scattering experiments in CTEQ global QCD analyses: a comparative investigation,” _Eur. Phys. J. C_ , vol. 81, no. 7, p. 603, 2021.
  * [20] H. Khanpour _et al._ , “Nuclear parton distribution functions with uncertainties in a general mass variable flavor number scheme,” _Phys. Rev. D_ , vol. 104, no. 3, p. 034010, 2021.
  * [21] I. Helenius, M. Walt, and W. Vogelsang, “NNLO nuclear parton distribution functions with electroweak-boson production data from the LHC,” _Phys. Rev. D_ , vol. 105, no. 9, p. 094031, 2022.
  * [22] R. Abdul Khalek _et al._ , “nNNPDF3.0: evidence for a modified partonic structure in heavy nuclei,” _Eur. Phys. J. C_ , vol. 82, no. 6, p. 507, 2022\.
  * [23] P. Duwentäster _et al._ , “Impact of heavy quark and quarkonium data on nuclear gluon PDFs,” _Phys. Rev. D_ , vol. 105, no. 11, p. 114043, 2022\.
  * [24] K. J. Eskola, P. Paakkinen, and H. Paukkunen, “Non-quadratic improved Hessian PDF reweighting and application to CMS dijet measurements at 5.02 TeV,” _Eur. Phys. J. C_ , vol. 79, no. 6, p. 511, 2019.
  * [25] I. Helenius and H. Paukkunen, “Revisiting the D-meson hadroproduction in general-mass variable flavour number scheme,” _JHEP_ , vol. 05, p. 196, 2018\.
  * [26] K. J. Eskola, I. Helenius, P. Paakkinen, and H. Paukkunen, “A QCD analysis of LHCb D-meson data in p+Pb collisions,” _JHEP_ , vol. 05, p. 037, 2020.
  * [27] P. Duwentäster _et al._ , “Impact of inclusive hadron production data on nuclear gluon PDFs,” _Phys. Rev. D_ , vol. 104, p. 094005, 2021.
  * [28] A. M. Sirunyan _et al._ , “Observation of nuclear modifications in W± boson production in pPb collisions at $\sqrt{s_{\mathrm{NN}}}=$ 8.16 TeV,” _Phys. Lett. B_ , vol. 800, p. 135048, 2020.
  * [29] K. J. Eskola, P. Paakkinen, H. Paukkunen, and C. A. Salgado, “Proton-PDF uncertainties in extracting nuclear PDFs from $W^{\pm}$ production in p+Pb collisions,” _Eur. Phys. J. C_ , vol. 82, no. 3, p. 271, 2022.
  * [30] A. M. Sirunyan _et al._ , “Study of Drell-Yan dimuon production in proton-lead collisions at $\sqrt{s_{\mathrm{NN}}}=$ 8.16 TeV,” _JHEP_ , vol. 05, p. 182, 2021.
  * [31] P. Paakkinen, “Light-nuclei gluons from dijet production in proton-oxygen collisions,” _Phys. Rev. D_ , vol. 105, no. 3, p. L031504, 2022.
  * [32] S. Shen, P. Ru, and B.-W. Zhang, “Imaging nuclear modifications on parton distributions with triple-differential dijet cross sections in proton-nucleus collisions,” _Phys. Rev. D_ , vol. 105, no. 9, p. 096025, 2022.
  * [33] U. A. Acharya _et al._ , “Systematic study of nuclear effects in $p+$Al, $p+$Au, $d+$Au, and 3He$+$Au collisions at $\sqrt{s_{{}_{NN}}}=200$ GeV using $\pi^{0}$ production,” _Phys. Rev. C_ , vol. 105, p. 064902, 2022.
  * [34] LHCb Collaboration, “Measurement of the prompt $D^{0}$ nuclear modification factor in $p$Pb collisions at $\sqrt{s_{\mathrm{NN}}}=8.16$ TeV,” arXiv:2205.03936 [nucl-ex].
  * [35] ——, “Nuclear modification factor of neutral pions in the forward and backward regions in $p$Pb collisions,” arXiv:2204.10608 [nucl-ex].
  * [36] R. Aaij _et al._ , “Measurement of the Nuclear Modification Factor and Prompt Charged Particle Production in $p-Pb$ and $pp$ Collisions at $\sqrt{s_{NN}}$=5 TeV,” _Phys. Rev. Lett._ , vol. 128, no. 14, p. 142004, 2022\.
  * [37] K. J. Eskola, C. A. Flett, V. Guzey, T. Löytäinen, and H. Paukkunen, “Exclusive J/$\psi$ photoproduction in ultraperipheral Pb+Pb collisions at the CERN Large Hadron Collider calculated at next-to-leading order perturbative QCD,” _Phys. Rev. C_ , vol. 106, no. 3, p. 035202, 2022\.
  * [38] ATLAS Collaboration, “Photo-nuclear jet production in ultra-peripheral Pb+Pb collisions at $\sqrt{s}_{\text{NN}}=5.02$ TeV with the ATLAS detector,” CERN, Conference Note ATLAS-CONF-2022-021, 2022, accessed July 28, 2022. [Online]. Available: https://cds.cern.ch/record/2806461
  * [39] B. Wong, “Points of view: Color blindness,” _Nature Methods_ , vol. 8, no. 6, p. 441, 2011.
  * [40] P. Tol, “Colour Schemes,” Netherlands Institute for Space Research, Technical Note SRON/EPS/TN/09-002 issue 3.2, 2021, accessed July 28, 2022. [Online]. Available: https://personal.sron.nl/~pault/data/colourschemes.pdf


# Task-Aware Neural Architecture Search

###### Abstract

The design of handcrafted neural networks requires a lot of time and
resources. Recent techniques in Neural Architecture Search (NAS) have proven
to be competitive or better than traditional handcrafted design, although they
require domain knowledge and have generally used limited search spaces. In
this paper, we propose a novel framework for neural architecture search,
utilizing a dictionary of models of _base_ tasks and the similarity between
the _target_ task and the atoms of the dictionary; hence, generating an
adaptive search space based on the base models of the dictionary. By
introducing a gradient-based search algorithm, we can evaluate and discover
the best architecture in the search space without fully training the networks.
The experimental results show the efficacy of our proposed task-aware
approach.

Index Terms—  Neural Architecture Search, AutoML, Task Taxonomy

##  1 Introduction

Neural Architecture Search (NAS) has been a major focal point for work on
automated machine learning (AutoML). Initially studied through the lens of
reinforcement learning [1], a modern development of NAS algorithms largely
focuses on minimizing both search time and prior knowledge. Though NAS
techniques have greatly improved, many recently proposed methods require
significant prior knowledge, e.g. the explicit architecture search domain, or
the specific task at hand, as input. This requirement restricts their ability
to adapt to situations in which future tasks are potentially unknown.

In this work, we propose a novel, flexible NAS framework, which we call Task-
Aware Neural Architecture Search (TA-NAS). The ultimate goal of TA-NAS is to
develop an algorithm that dynamically learns an appropriate architecture for a
given task at hand, making decisions based on prior history and any
information input by the user. Our pipeline is composed of three key
components. First, we start with a dictionary of base tasks, the atoms of
which consist of architectures that accurately perform said tasks. The
dictionary serves as a base on which we dynamically build architectures for
new tasks not in the dictionary. Based on the idea that similar tasks should
require similar architectures, an often-used assumption in both transfer and
lifelong learning, we propose a novel similarity measure for tasks to find the
closest base tasks to the new task. Then, we construct a dynamic search space,
based on the combined knowledge from the related tasks, without the need for
prior domain knowledge. Finally, we present a gradient-based search algorithm,
called _Fusion Search_ (FUSE). The FUSE algorithm is designed to evaluate the
performance of network candidates without fully train any of them. Our
experimental evaluation will show the efficacy of our proposed approach.

##  2 Related Work

Many recently proposed NAS techniques have resulted in architectures with
performance comparable to those of hand-tuned architectures. The techniques
themselves are based on a wide range of techniques, including evolutionary
algorithms [2], reinforcement learning (RL) [3], and sequential model-based
optimization (SMBO) [4]. All of these approaches, however, are very time-
consuming and need require computational resources, e.g. potentially thousands
of GPU-days. To alleviate these issues, differentiable search [5, 6, 7, 8, 9]
and random search together with sampling sub-networks from a one-shot super-
network [10, 11, 12] have been introduced in the literature. For instance,
DARTS [6] smooths the architecture search space using a softmax operation. It
then solves a bilevel optimization problem which can accelerate the discovery
of the final architecture by orders of magnitude [1, 3, 2, 4]. Other recent
methods include random search [13, 11, 14, 15], RL based approaches via
weight-sharing [16], and network transformations [17, 18, 19].

Besides, [10] has thoroughly analyzed the one-shot architecture search using
weight-sharing and correlation between the super-graph and sub-networks. None
of the above techniques have yet explored the role of the closeness of tasks
in the search neural architecture space. Consequently, the search space used
by these techniques is often biased and based on the domain knowledge from the
well-performed handcrafted neural network architectures. Here, we propose an
approach to encode the similarities between tasks for a more efficient search
strategy.

##  3 Proposed Approach

The pseudocode of TA-NAS is given in Algorithm 1. At time $t$, we assume that
we have access to a dictionary consisting of both previous pairs
$(T_{1},X_{1}),...(T_{t-1},X_{t-1}),$ of tasks $T_{k}$ and data sets $X_{k}$
is a given data set, as well as a collection of such pairs that were available
upon initialization. Each pair is represented in our dictionary by a trained
network. Given the target pair of $(T_{t},X_{t}),$ our goal is to find an
architecture for achieving high performance on the target task. In summary,
TA-NAS works as follows:

  1. 1.

Task Similarity. Given a new task-data set pair, TA-NAS finds the most related
task-data set pairs in the dictionary.

  2. 2.

Search Space. TA-NAS defines a suitable search space for the incoming (target)
task-data set pair, based on the related pairs.

  3. 3.

Search Algorithm. TA-NAS searches to discover an optimal architecture in term
of performance for the target task-data set pair on the search space.

Initialization: A set of baseline task-data set pairs $B$;

Input: Task-data set pairs $(T_{1},X_{1}),...,(T_{K},X_{K})$, Threshold
$\tau$, $\epsilon$;

Output: Best architecture for the incoming tasks t;

for  _$t=(K+1),(K+2),...$_ do

for  _$b\in B$_ do

Calculate distance $d_{b,t}$ to find the related tasks;

end for

Define search space by combining operations, cells, skeleton from related
tasks;

while  _criteria not met_ do

Sample C candidates from search space;

Evaluate these candidates using FUSE;

end while

Add the task t and its architecture to $B.$

end for

Algorithm 1 Task-Aware NAS

###  3.1 Task Similarity

The TA-NAS pipeline heavily depends on the notion of similarity between task-
data set pairs. We define the similarity between task-data set pairs in terms
of a model-transformation complexity, $N_{t}$. In particular, we first
construct a dictionary with the atoms given by the by trained architectures
performing well in each base task-data set pairs. More precisely, let
$\ell_{(T,X)}(N)$ be a function that measures the performance of a given
architecture $N$ on task $T$ with input data $X$. Here, $\ell_{(T,X)}(N)=1$
denotes the highest performance, while $\ell_{(T,X)}(N)=0$ denotes a complete
failure of architecture $N$ on task $T$. For a fixed $0<\epsilon<1$, we say
that an architecture $N$ is $\epsilon$-representative for $(T,X)$ if
$\ell_{T,X}(N)\geq 1-\epsilon.$ In other words, an architecture $N$ is
$\epsilon$-representative if it performs sufficiently well on the given task-
data set pair. We thus restrict our attention to $\epsilon$-representative
networks.

We now define our measure of dissimilarity between two task-data set pairs
using the notion of $\epsilon$-representative architecture. Let
$A=(T_{A},X_{A})$ and $B=(T_{B},X_{B})$ be two task-data set pairs, where
$N_{A}$ and $N_{B}$ are two trained architectures that are
$\epsilon$-representative for $A$ and $B$, respectively. We can define a
dissimilarity measure between $A$ and $B$ as follows (hence, the similarity
between $A$ and $B$ is given by $1-d^{\epsilon}_{A,B}$):

Fig. 1: Illustration of the procedure to compute the distance from task A to
task B.

$d^{\epsilon}_{A,B}=\min_{\\{N_{t}\in S_{t}:\ell_{B}(N_{t}\circ
N_{A})\geq(1-\epsilon)\\}}O(N_{t}),$ (1)

where $S_{t}$ is a given transform network search space, and $O$ is a general
measure of complexity (e.g., the number of parameters in a network). The
symbol $\circ$ denotes function (network) composition. In this setting, we
assume that the first layer of $N_{t}$ is the same as the _penultimate_ layer
of $N_{A}$. In practice, the role of $N_{t}$ is to transform the features of
the penultimate layer of $N_{A}$ generated by elements of $X_{B}$ and
transform them into elements of $N_{B}(X_{B})$. In other words, the model
transformation network $N_{t}$ should be such that it can transfer the
$N_{A}(X_{B})$ into $N_{B}(X_{B})$ with minimum complexity. For instance, if
$N_{t}$ is the identity matrix, it means that $T_{A}$ and $T_{B}$ are exactly
the same as $N_{B}(X_{B})=N_{A}(X_{B})$. In general, $d^{\epsilon}_{A,B}$ is
asymmetric measure, i.e., $d^{\epsilon}_{A,B}\neq d^{\epsilon}_{B,A}$.
Ideally, $O$ should be zero for two identical tasks, implying that the
architecture for $A$ is also suitable for $B$. In practice, finding the least-
complex transform network can be achieved by iterative pruning some super-
network for which $N_{t}$ is a sub-graph with the performance as good as the
super-network. The dissimilarity measure is the percentage of the non-zero
parameters in the pruned $N_{t}$, and it ranges between 0 and 1. This is
illustrated in Figure 1.

###  3.2 Search Space

Defining a meaningful search space is the key to efficiently finding the best
architecture for a specific task. In the NAS literature, the search space is
typically defined by stacking a structure called _cell_ , as illustrated in
Figure 2. A cell is a densely connected directed-acyclic graph (DAG) of nodes,
where all nodes are connected by operations. Other NAS techniques such as one-
shot approaches (e.g., DARTS [6], NAS-Bench201 [20]) have also introduced
another structure in the search space referred to as _skeleton_. A skeleton is
a combination of cells with other operations, forming the complete network
architecture. A skeleton is normally predefined, and the goal of NAS
algorithms is to find the optimal cells. In this paper, we similarly define
the search space in terms of skeletons and cells. Specifically, we focus our
search on cells and their operations. As mentioned, cells consist of nodes and
operations. Each node has $2$ inputs and $1$ output. The operations (e.g.,
identity, zero, convolution, pooling) are set so that the dimension of the
output is the same as that of the input. If $\mathbf{n}$ is the number of
nodes in a cell and $\mathbf{m}$ denotes the number of operations, the total
number of possible cells is given by:
$\mathbf{m}\times\exp\big{(}{\frac{\mathbf{n}!}{2(\mathbf{n}-2)!}}\big{)}$.

Fig. 2: Illustration of the cell and the skeleton.

Our use of a dissimilarity measure gives us knowledge about how related two
tasks are. Build upon this knowledge, we can define the search space of the
target task-data set pair by combining the skeletons, cells, and operations
from only the most similar pairs in the dictionary. Since the search space is
restricted to only related tasks, the architecture search algorithm can
perform efficiently and requires few GPU hours to find the best candidate
network. We have illustrated this in the experimental section.

###  3.3 Search Algorithm

The Fusion Search (FUSE) is a novel search algorithm that considers the
network candidates as a whole and performs the optimization using gradient
descent. Let $C$ be the set of candidate networks on which we define the
search space. Given $c\in C$ and training data $X$, denote by $c(X)$ the
output of the network candidate $c$. The FUSE algorithm, as illustrated in
Algorithm 2, is based on the continuous relaxation of the network outputs. It
is capable of searching through all networks in the relaxation space without
fully training them. We use as our relaxed space $C$ the set of all convex
combinations of candidate networks, which each weight in the combination given
by exponential weights:

$\bar{c}(X)=\sum_{c\in C}\frac{\exp{(\alpha_{c})}}{\sum_{c^{\prime}\in
C}\exp{(\alpha_{c^{\prime}})}}c(X),$ (2)

where $\bar{c}$ is the weighted output of network candidate $c$, and
$\alpha_{c}$ is a continuous variable that assigned to candidate $c$’s output.
We then conduct our search by jointly training the network candidates and
optimizing their $\alpha$ coefficients. Let $X_{train}$, $X_{val}$ be the
training and validation data set. The training procedure is based on
alternative minimization and can be divided into: (i) freeze $\alpha$
coefficients, jointly train network candidates, (ii) freeze network
candidates, update $\alpha$ coefficients. Initially, $\alpha$ coefficients are
set to $1/|C|$. While freezing $\alpha$, we update the weights in network
candidates by jointly train the relaxed output $\bar{c}$ with cross-validation
loss on training data:

${}\min_{w}\mathcal{L}_{train}(w;\alpha,\bar{c},X_{train}),$ (3)

where $w$ are weights of network candidates in $C$. Next, the weights in those
candidates are fixed while we update the $\alpha$ coefficients on validation
data:

${}\min_{\alpha}\mathcal{L}_{val}(\alpha;w,\bar{c},X_{val}).$ (4)

These steps are repeated until $\alpha$ converges. The most promising
candidate will be selected by: $c^{*}=\arg\max_{c\in C}\alpha_{c}$. This
training procedure will deliver the best candidate among candidates in $C$
without fully training all of them. In order to go through the entire search
space, this process is repeated until certain criteria, such as the number of
iterations, the performance of the current most promising candidate, is met.

Input: search space S, $X_{train}$, $X_{val}$, I;

Initilization: $c^{*}$, $\alpha$;

Output: Best architecture;

for  _$i=1,...,I$_ do

C = [$c^{*}$, and candidates sampled from S];

while  _$\alpha$ not converge_ do

Update $C$ by descending $\nabla_{w}\mathcal{L}_{train}(w;\alpha,\bar{c})$;

Update $\alpha$ by descending
$\nabla_{\alpha}\mathcal{L}_{val}(\alpha;w,\bar{c})$;

end while

$c^{*}=\arg\max_{c\in C}\alpha_{c}$;

end for

Algorithm 2 FUSE Algorithm

##  4 Experimental Study

We evaluate the TA-NAS algorithm on image data sets and classification tasks.
For our experiment, we initialize the TA-NAS with a set of base binary
classification tasks consisting of finding specific digits in MNIST [21] and
specific objects in Fashion-MNIST [22]. We find $\epsilon$-representatives for
each task by pre-training networks on the same architecture
($\text{conv}(32\times 5\times
5)\rightarrow\text{dense}(1024)\rightarrow\text{dense}(2)$). Here, we pick
representative architectures that achieve at least $96\%$ accuracy on their
tasks.

In order to compute the dissimilarity between architectures, we consider for
$A$ and $B$ (two task-data set pairs) the first two layers of their trained
$\epsilon$-representative networks, which we denote by $N_{A}$ and $N_{B}$,
respectively. We then wish to find the least complex architecture that maps
hidden features from one task to the other. We thus consider a transform
network $N_{t}$ with a
$\text{dense}(2048)\rightarrow\text{dense}(512)\rightarrow\text{dense}(1024)$
architecture. We train $N_{t}$ with mean-square error (MSE) loss on a data set
consisting of $N_{A}(X_{B})$ and those of $N_{B}(X_{B})$; here, the goal is to
transform $N_{A}(X_{B})$ into $N_{B}(X_{B})$. We then iteratively prune the
trained $N_{t}$ as much as possible while maintaining similar performance to
$N_{t}$. We take our dissimilarity measure to be the percentage of the
remaining non-zero parameters in $N_{t}$ after pruning. We show our results in
Figure 2. Our results suggest that two tasks from the same data set (e.g.,
MNIST or Fashion-MNIST) are often more similar than tasks involving different
data sets. It is perhaps interesting to note that the similarity from MNIST
tasks to Fashion-MNIST tasks is greater than the similarity from Fashion-MNIST
tasks to those in MNIST. Consequently, we can often use Fashion-MNIST
knowledge on MNIST, but not vice-versa.

The task on which we perform NAS is binary classification on Quick, Draw! [23]
dataset. The Quick, Draw! is a doodle drawing dataset of 345 categories. In
this experiment, we select a subset of the Quick, Draw! with a similar format
to MNIST and fashion-MNIST, by choosing only 10 categories (e.g., apple,
baseball bat, bear, envelope, guitar, lollipop, moon, mouse, mushroom, rabbit)
with each has 60,000 data points. Our task of interest is the moon indicator
from this subset of Quick, Draw! data set. We pre-train this task and compute
dissimilarities to other tasks in the same manner as above. To conduct our
search, we choose the top three most similar tasks to the baselines discussed
above: (i) digit 0, (ii) trouser, (iii) digit 3 indicators. Due to the
similarity in the shape of digit 0 and the moon, the base task of digit 0
indicator is the most related task to the moon indicator.

Fig. 3: The distance matrix of baseline tasks.

After obtaining the related tasks, we combine the operations and cell
structures to generate a suitable search space for the target task. The cell
consists of 4 nodes, with 6 edges of operations. The list of operations
includes identity, zero, dil-conv3x3, sep-conv3x3, maxpool2x2. Next, the
search algorithm is used to find the best architecture in this search space.
Initially, three network architectures are randomly generated from the search
space. At each iteration, FUSE quickly evaluates these candidates and only
saves the best architecture for the next iteration. The search stops only when
all criteria (e.g., a prespecified maximum number of iteration or best
architecture converges) are met. The results in Table 1 give the best test
error of the optimal architecture found by TA-NAS after 20 trials, in
comparison with a random search algorithm and other state-of-art handcrafted
networks (e.g., DenseNet [24], ResNet [25]). The architecture produced by TA-
NAS is competitive with manual-designed networks while having a smaller number
of parameters. When comparing with the random search method using our pre-
defined search space, TA-NAS approach achieves a higher accuracy model with
less search time in terms of GPU days (i.e., the number of days for a single
GPU to perform the task). Hence, this framework can utilize the knowledge of
related tasks to find the efficient network architecture for the target task.

Architecture |  Error |  Param |  GPU
---|---|---|---
(%) |  (M) |  days
ResNet-18 [25] |  1.42 |  11.44 |  -
ResNet-34 [25] |  1.2 |  21.54 |  -
DenseNet-161 [24] |  1.17 |  27.6 |  -
Random Search |  1.33 |  2.55 |  4
FUSE w. standard space |  1.21 |  2.89 |  2
FUSE w. task-aware space |  1.18 |  2.72 |  2
Table 1: Comparison with state-of-art image classifiers on Quick, Draw!
dataset.

##  5 Conclusion

We proposed TA-NAS, a novel task-aware framework to address the Neural
Architecture Search problem. By introducing a similarity measure for given
pairs of tasks and data sets, we can define a restricted, dynamic architecture
search space for a new task-data set pair based on similar previously observed
pairs. Additionally, we proposed the gradient-based search algorithm, FUSE, to
quickly evaluate the performance of network candidates in the search space.
This search algorithm can be applied to find the best way to grow or to
compress the current network.

## References

  * [1] Barret Zoph and Quoc V Le,  “Neural architecture search with reinforcement learning,”  Proc. Int. Conf. Learning Representations, 2017.

  * [2] Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V Le,  “Regularized evolution for image classifier architecture search,”  Proc. Assoc. Adv. Art. Intell. (AAAI), 2019.
  * [3] Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V Le,  “Learning transferable architectures for scalable image recognition,”  IEEE Conf. Comp. Vision and Pattern Recog, 2018.
  * [4] Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, and Kevin Murphy,  “Progressive neural architecture search,”  Euro. Conf. Comp. Vision, 2018.
  * [5] Han Cai, Ligeng Zhu, and Song Han,  “Proxylessnas: Direct neural architecture search on target task and hardware,”  Proc. Int. Conf. Learning Representations, 2019.
  * [6] Hanxiao Liu, Karen Simonyan, and Yiming Yang,  “Darts: Differentiable architecture search,”  Proc. Int. Conf. Machine Learning, 2018.
  * [7] Asaf Noy, Niv Nayman, Tal Ridnik, Nadav Zamir, Sivan Doveh, Itamar Friedman, Raja Giryes, and Lihi Zelnik-Manor,  “Asap: Architecture search, anneal and prune,”  arXiv preprint arXiv:1904.04123, 2019.
  * [8] Renqian Luo, Fei Tian, Tao Qin, Enhong Chen, and Tie-Yan Liu,  “Neural architecture optimization,”  Adv. Neural Inf. Proc. Sys. (NeurIPS), 2018.
  * [9] Sirui Xie, Hehui Zheng, Chunxiao Liu, and Liang Lin,  “Snas: stochastic neural architecture search,”  arXiv preprint arXiv:1812.09926, 2018.
  * [10] Gabriel Bender, Pieter-Jan Kindermans, Barret Zoph, Vijay Vasudevan, and Quoc Le,  “Understanding and simplifying one-shot architecture search,”  Proc. Int. Conf. Machine Learning, 2018.
  * [11] Liam Li and Ameet Talwalkar,  “Random search and reproducibility for neural architecture search,”  arXiv preprint arXiv:1902.07638, 2019.
  * [12] Minsu Cho, Mohammadreza Soltani, and Chinmay Hegde,  “One-shot neural architecture search via compressive sensing,”  arXiv preprint arXiv:1906.02869, 2019.
  * [13] Liam Li, Kevin Jamieson, Afshin Rostamizadeh, Ekaterina Gonina, Moritz Hardt, Benjamin Recht, and Ameet Talwalkar,  “Massively parallel hyperparameter tuning,”  arXiv preprint arXiv:1810.05934, 2018.
  * [14] Christian Sciuto, Kaicheng Yu, Martin Jaggi, Claudiu Musat, and Mathieu Salzmann,  “Evaluating the search phase of neural architecture search,”  arXiv preprint arXiv:1902.08142, 2019.
  * [15] Saining Xie, Alexander Kirillov, Ross Girshick, and Kaiming He,  “Exploring randomly wired neural networks for image recognition,”  arXiv preprint arXiv:1904.01569, 2019.
  * [16] Hieu Pham, Melody Y Guan, Barret Zoph, Quoc V Le, and Jeff Dean,  “Efficient neural architecture search via parameter sharing,”  Proc. Int. Conf. Machine Learning, 2018.
  * [17] Han Cai, Tianyao Chen, Weinan Zhang, Yong Yu, and Jun Wang,  “Efficient architecture search by network transformation,”  Proc. Assoc. Adv. Art. Intell. (AAAI), 2018.
  * [18] Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter,  “Efficient multi-objective neural architecture search via lamarckian evolution,”  Proc. Int. Conf. Learning Representations, 2019.
  * [19] Max Jaderberg, Valentin Dalibard, Simon Osindero, Wojciech M Czarnecki, Jeff Donahue, Ali Razavi, Oriol Vinyals, Tim Green, Iain Dunning, Karen Simonyan, et al.,  “Population based training of neural networks,”  arXiv preprint arXiv:1711.09846, 2017.
  * [20] Xuanyi Dong and Yi Yang,  “Nas-bench-102: Extending the scope of reproducible neural architecture search,”  arXiv preprint arXiv:2001.00326, 2020.
  * [21] Yann LeCun, Corinna Cortes, and CJ Burges,  “Mnist handwritten digit database,”  AT&T Labs [Online]. Available: http://yann. lecun. com/exdb/mnist, vol. 2, pp. 18, 2010.
  * [22] Han Xiao, Kashif Rasul, and Roland Vollgraf,  “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,”  arXiv :1708.07747, 2017.
  * [23] Jongejan J., Rowley H., Kawashima T., Kim J., and Fox-Gieg N.,  “The quick, draw! - a.i. experiment,”  https://quickdraw.withgoogle.com/, 2016.
  * [24] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger,  “Densely connected convolutional networks,”  IEEE Conf. Comp. Vision and Pattern Recog, 2017.
  * [25] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun,  “Deep residual learning for image recognition,”  IEEE Conf. Comp. Vision and Pattern Recog, 2016.