figurec [figure]style=plain

# Detecting Frames in News Headlines and Lead Images in U.S. Gun Violence
Coverage

Isidora Chara Tourni
Lei Guo
Hengchang Hu
Edward Halim
Prakash Ishwar
Taufiq Daryanto
Mona Jalal

Boston University
<EMAIL_ADDRESS>
Boqi Chen
Margrit BetkeFabian Zhafransyah11footnotemark: 1
Sha Lai
Derry Tanti Wijaya
Institut Teknologi Bandung Corresponding Author

###### Abstract

News media structure their reporting of events or issues using certain
perspectives. When describing an incident involving gun violence, for example,
some journalists may focus on mental health or gun regulation, while others
may emphasize the discussion of gun rights. Such perspectives are called
frames in communication research. We study, for the first time, the value of
combining lead images and their contextual information with text to identify
the frame of a given news article.

We observe that using multiple modes of information(article- and image-derived
features) improves prediction of news frames over any single mode of
information when the images are relevant to the frames of the headlines. We
also observe that frame image relevance is related to the ease of conveying
frames via images, which we call frame concreteness.

Additionally, we release the first multimodal news framing dataset related to
gun violence in the U.S., curated and annotated by communication researchers.
The dataset will allow researchers to further examine the use of multiple
information modalities for studying media framing.

##  1 Introduction

Media framing refers to the journalistic practice of selecting aspects of a
perceived reality and making them more salient in news coverage Entman (1993);
Reese et al. (2001). In political communication, for example, news framing is
helpful as it reveals how the news article is structured to promote a certain
side of the political spectrum, thus influencing the public opinion in a
particular way.

Journalists have been using both text and images to frame news stories. Images
in news stories can help convey controversial or provocative meanings that
would otherwise be unpalatable to the news audience, if it were spelled out in
text Messaris and Abraham (2001). While text is more influential in changing
opinions, visuals elicit more attention and emotional reactions, resulting in
behavioral change Coleman and Wu (2015); Dan (2017); Powell et al. (2015).
Lead images may carry additional background knowledge about the event (e.g.,
showing well-known people and locations). An image showing a school, for
example, might suggest an article about gun violence focuses on the
School/Public Safety frame.

Text and images thus work in tandem to create a holistic perception of news
and must be considered together when analyzing news frames Wessler et al.
(2016).

Given the importance of visuals in media framing and the rising gun violence
in the U.S. Guo et al. (2021), we extend the Gun Violence Frame Corpus (GVFC)
Liu et al. (2019), which contains news headlines related to U.S. gun violence
and their domain-expert frame annotations, by retrieving the lead images of
the articles and obtaining their relevance annotations from communication
domain experts (i.e., an image is annotated as relevant if it expresses the
annotated headline frames). Notably, about half of the time, the images
presented do not express the annotated headline frames (Table 1). This might
be explained from the journalism research perspective, as reporters and
photographers do not necessarily work together seamlessly in the newsroom as
they occupy distinct occupational roles and often compete for control over how
a story may be packaged and presented as a final product Lowrey (2002).

Hence, in addition to communication scholars benefiting from tools that can
analyze, on large scale, images and headlines in tandem for frames, newsroom
editors would benefit from tools that can identify images that help depict the
main thrust of the story’s focus Caple (2010). Such tools do not yet exist,
and our work addresses this need.

In this work, we comprehensively explore the use of multimodal information
from news articles i.e., headlines or summaries, and their lead images i.e.,
the images, categories of objects in the images, or the background/real-world
knowledge contained in them, for predicting frames.

Our results show that for news articles with relevant images, using only
image-derived features or only article-derived features (i.e., headlines
and/or extractive summaries) yields less accurate frame predictions than our
multiple modalities approach. When considering articles with irrelevant
images, the accuracy of the multimodal approach is comparable to that based on
only article-derived features.

We also observe that adding image contextual information using the Google Web
Entity Tagger API111https://cloud.google.com/vision/docs/detecting-web or an
entity-aware news image caption generation model (Tran et al., 2020) or by
asking humans to annotate the central subject of the image in terms of pre-
defined categories that cue gun violence frames, such as politician (politics
frame), legislative buildings (gun control frame), school/campus
(school/public safety frame), etc., improves the performance of frame
prediction, compared to using raw images alone. The API tags capture
background information associated with an image from the Web, such as the list
of named entities in the image, by finding similar images in the Web and
parsing the associated web page contents. News image captions capture real-
world information in the image, e.g., the names of people and objects, by
learning to associate words in the article text with faces and objects. Human
annotations of the central subject of the image in terms of categories such
school or legislative buildings, capture the annotators’ background knowledge
of the identities of entities in the image.

Overall, our contributions are the following: (1) A well-curated multimodal
text-image framing dataset with expert annotations222The multimodal annotation
is available for download through the GVFC dataset website:
https://derrywijaya.github.io/GVFC.html.: With the goal of predicting frames
based on multiple information modalities, we augment GVFC by using the article
URLs to retrieve lead images of articles and annotate the images for their
visual framing labels, which include (a) the Subject/Race/Ethnicity (SRE)
annotations of the central subject of the image (i.e., suspect vs. victim vs.
politician, etc.) and whether the image contains anything related to
race/ethnicity, and (b) the image relevance annotations i.e., whether the
images are relevant to the annotated frames of their headlines. In addition to
frame annotations of news article headlines and lead images, for each image we
provide its URL, Web Entity tag (API tag), caption generated using a state-of-
the-art news image captioning system Tran et al. (2020), and the article
summary generated by a state-of-the-art extractive summarization system Liu
and Lapata (2019). (2) Comprehensive study and development of methods to
combine multimodal information to predict article frames and image relevance:
We explore various approaches to predict image relevance and article frames
using information from both article and lead images, using BERT Devlin et al.
(2018) to represent text and a deep convolutional neural network ResNet-50 He
et al. (2016) to represent raw images. (3) Frame concreteness: We propose a
novel method for measuring the ease of conveying frames through images via the
concreteness of words in its headlines, i.e., the ease of identifying tangible
concepts and mental images that arise in correspondence to words Paivio et al.
(1968), and relate frame image relevance to frame concreteness.

Figure 1: Sample images for each frame (from left to right and top to down):
2nd Amendment, Gun Control, Politics, Mental Health, School/Public Space
Safety, Race/Ethnicity, Public Opinion, Society/Culture, Economic
Consequences.

##  2 Related Work

Media framing is related to many factors such as word choice, the presentation
of background information, and the emphasis on certain actors.

The subtle nature of news framing can influence the opinion of readers in a
certain way without them even noticing it, hence its analysis has many
implications, e.g., it has been used to understand why important public
affairs issues such as gun violence are polarizing Liu et al. (2019), how
media manipulation strategies are conducted Field et al. (2018), or how
framing is used to perpetuate racial biases Drakulich (2015). In Journalism,
most “framing analyses” have been done manually hence they are not scalable
Hamborg et al. (2019).

In natural language processing (NLP), automated frame detection focuses on
predicting frames from news texts. Some of these methods rely on topic models
Nguyen et al. (2013). Naderi and Hirst (2017) devised various deep neural
networks (LSTMs, BiLSTMs, or GRUs) for predicting frames at the sentence level
in the Media Frame Corpus (MFC) Card et al. (2015). The most recent method for
predicting news frames in headlines is the work of Liu et al. (2019); Akyürek
et al. (2020). They fine-tuned BERT Devlin et al. (2018) using focal loss Lin
et al. (2017) to predict frames of headlines. They also released a framing
benchmark dataset, the Gun Violence Frame Corpus (GVFC), of news links and
headlines with frame annotations, related to gun violence in the U.S. They
show that fine-tuning BERT for frame prediction using news headlines results
to significantly higher accuracy than previous methods.

In this paper, we use GVFC articles’ lead images and perform experiments with
a rich set of unimodal and multimodal information to predict frames. Although
images and text have been used together as multimodal inputs to improve
performance in other NLP tasks such as machine translation Specia et al.
(2016); Hewitt et al. (2018); Caglayan et al. (2019); Yao and Wan (2020);
Khani et al. (2021) or in vision-language tasks such as multilingual image
retrieval or captioning Kim et al. (2020); Burns et al. (2020); Rasooli et al.
(2021), the use of images and text in tandem to automatically analyze framing
i.e., computational multimodal framing has never been explored–all previous
works in multimodal framing have been conducted manually Messaris and Abraham
(2001); Coleman and Wu (2015); Dan (2017); Powell et al. (2015); Wessler et
al. (2016). Given the growing importance of visual journalism and the
contribution of images to media framing which suggest that images may be able
to help interpret text, our work is the first to conduct computational
multimodal framing analysis, which will enable scalable multimodal framing
analysis.

##  3 Dataset

Our multimodal version of GVFC contains news headlines and their corresponding
lead images, news URLs, and the entire news text.

The lead images are either the pictures shown at the top of news articles or
the editor-picked thumbnails that are shown in news services such as Google
News (Fig. 1). Using Brandwatch Consumer Research333https://www.brandwatch.com
we analyze 3,000 news headlines, of which 1,300 are annotated with 9 major
frames, e.g., politics, gun control/regulation, mental health, race/ethnicity,
etc (Table 1), that exhaustively cover the discussion of the U.S. “gun
violence” issue in communication research. In this paper, we further annotate
each lead image with a binary relevance label that indicates whether the image
is consistent with the frame associated to the headline.

We also annotate the central subject (S) of the image using one of 16
categories that often imply certain frames, e.g., suspect (often implies
mental health), politician (often implies politics), company logos (often
implies economic consequence), etc. (see Appendix for the full listing), and
an additional race/ethnicity (RE) label which can take one of the following 3
values: 1) racial/ethnic minority groups, 2) hate groups, or 3) none of the
above.

The details of the visual annotation codebook used to train the coders are
given in the Appendix. The coders’ agreement on how to apply the codes is
measured with inter-coder reliability (ICR). High ICR values (above 90%
agreement or 0.70 Krippendorff $\alpha$ Krippendorff (2018)) imply that two or
more coders consistently categorized the content similarly, signaling a high
validity of the results. In our dataset, ICR was met on all variables: Subject
(90% agreement, 0.88 $\alpha$), Race/Ethnicity (91% agreement, 0.64 $\alpha$)
and Relevance (88% agreement, 0.75 $\alpha$). The number and ratio of relevant
images per frame are shown in Table 1. Easy and hard to classify examples and
their features are provided in Table 2 and described in detail in Section 4.
We affirm we have the right to use the collected dataset in the way we are
using it444We have confirmed and received approval from Brandwatch Consumer
Research whom we obtain the data from., i.e. the article headline and URL, as
well as their annotations and image-derived visual and textual features; and
we bear responsibility in case of a violation of rights or terms of service.
Researchers can use the article URLs to retrieve images and full texts of the
articles.

|  | # Relevant
---|---|---
News Frame | # Articles | Images (%)
Politics | 373 | 241 (65%)
Public Opinion | 237 | 147 (62%)
Gun Control/Regulation | 215 | 93 (43%)
School/Public Space Safety | 137 | 68 (50%)
Economic Consequences | 80 | 46 (58%)
Race/Ethnicity | 114 | 34 (30%)
Mental Health | 65 | 28 (43%)
2nd Amendment/Gun Rights | 38 | 13 (34%)
Society/Culture | 41 | 4 (10%)
Overall | 1,300 | 674 (52%)
Table 1: Gun violence frames in our dataset, the number of articles with headlines and lead images, and the number of lead images annotated as relevant to the frame with the percentage indicated in brackets. The news frames are ordered by the number of relevant images from highest to lowest.  Image |  Description
---|---
|  An article with relevant image in a frame with many examples (potentially
easy to classify)
Frame: School/Public Space Safety
Headline: To Defend Against School Shootings, Massachusetts District Is
Passing Out Emergency Buckets With Hammer, Rope
API tag: Classroom, School, Harry S Truman, High School, Active shooter,
Lockdown, Campus, Student
Caption: A school shooting victim in Brockton, Mass., last month.
3sentences: U.S.: More than 1,000 blue buckets were assembled to be passed out
to classrooms in the Brockton, Massachusetts, school district, filled with
curated items aimed at saving lives in the event of an emergency, including a
school shooting. The Brockton school district partnered with the mayor’s
office, the Brockton Police Department and a local Lowe’s to put together
buckets filled with four items to help defend classrooms. Each blue five-
gallon bucket contains a wooden wedge, a one-pound hammer, a 50-foot length of
rope and a roll of duct tape, according to The Enterprise.
Summary: U.S.: More than 1,000 blue buckets were assembled to be passed out to
classrooms in the Brockton, Massachusetts, school district. The buckets can be
used for emergency bathroom situations. Mayor Bill Carpenter applauded the
decision to put the buckets in the classrooms.
|  An article with irrelevant image in a frame with many examples (potentially
harder to classify).
Frame: School/Public Space Safety
Headline: Mass shootings ‘increasing’ and pose ‘most serious threat’ in US,
expert says,
API tag: Thousand Oaks shooting, Borderline Bar Grill, Mass shooting,
California Bar, Police officer
Caption: A gunman at the scene of the shooting at a country bar in Sacramento.
3sentences: Mass shootings ‘increasing’ and pose ‘most serious threat’ in US,
expert says At least 59 people have been killed as a result of mass shootings
this year. Deadliest mass shootings of 2018 in the U.S. Mike Nelson/EPA via
Shutterstock.
Summary: At least 59 people have been killed as a result of mass shootings
this year. There have been at least six mass shootings in the U.S. this year,
according to the U.S., at least 10 mass shootings have been linked to mass
shootings at a California bar.
|  An article with relevant image in a frame with few examples (potentially
harder to classify).
Frame: Mental Health
Headline: Accused Fredericton shooter will undergo psych assessment
API tag: Car, Job, Vehicle, Staff, Capilar y Corporal
Caption: Matthew Vincent Raymond Murder Officer Suspect Broward Police officer
Arrest warrant Criminal charge Suspect Murder.
3sentences: The Fredericton man accused of killing four people in August will
be sent for a psychiatric assessment. Judge Julian Dickson ordered the
assessment Wednesday to determine if Matthew Vincent Raymond, 48, is fit to
stand trial on four counts of first-degree murder. Raymond is charged in the
Aug. 10 shooting deaths of Fredericton police constables Robb Costello, 45,
and Sara Burns, 43, and civilians Donnie Robichaud, 42, and Bobbi Lee Wright,
32.
Summary: A judge orders the assessment to determine if Matthew Vincent
Raymond, 48, is fit to stand trial on four counts of first-degree murder.
Raymond is charged in the Aug. 10 shooting deaths of Fredericton police
constables Robb Costello, 45, and Sara Burns, 43, and civilians Donnie
Robichaud, 42, and Bobbi Lee Wright, 32. Arguments about who should conduct
the assessment. The assessment is expected to be completed before Dec. 4, when
Raymond is due back in court.
|  An article with irrelevant image in a frame with few examples (potentially
hardest to classify).
Frame: Race/Ethnicity
Headline: Alabama mall shooting: Family of black man killed by police officer
on Thanksgiving hires civil rights lawyer
API tag: Shooting of Emantic Fitzgerald Bradford Jr., Alabama Shooting of
Michael Brown, News, Shopping Centre, Breaking news, Street light, Television
show, Street Broken Horses
Caption: A police officer at the scene of the shooting at the Riverchase
Galleria in Birmingham, Ala., on Friday.
3sentences: Emantic Fitzgerald Bradford Jr ’s family has employed Benjamin
Crump who previously represented the families of shooting victims Trayvon
Martin and Michael Brown to also represent them: WVTM The family of a 21-year-
old black man who was shot by a police officer at shopping centre in Alabama
on Thanksgiving has hired a national civil rights lawyer to represent them.
Summary: Emantic Fitzgerald Bradford Jr was fatally shot by a police officer
at the shopping centre in Alabama on thanksgiving has hired a national civil
rights lawyer. Police initially said a hoover police officer who was
responding to reports of gunfire at a shopping mall confronted an armed man
running away from the scene and fatally shot him. The shots responsible for
injuring an 18-year-old man and a 12-year-old girl, but investigators have
since said they believe he did not firesle the gunman is still at large. Mr
Crump said Bradford was a veteran who was licensed to carry a concealed
firearm. Bradford’s family said they are working with our legal team to
determine.
Table 2: Examples of articles, their images and image- and article-derived
textual features that are potentially easy or hard to classify using the
multimodal approach.

##  4 Experiments

We experiment with unimodal (Section 4.1) and multimodal (Section 4.2)
information obtained from each article and its lead image. We train and report
4-fold cross validation frame prediction accuracies (Table 3) for all articles
in our dataset (All Articles), and for the subset of articles with relevant
images (Articles with Relevant Images). We also perform image-to-frame
relevance classification (Section 4.3).

###  4.1 Unimodal

In this set of experiments we predict news frames from only one mode of
information, either image-derived or article-derived. For raw images, we use
ResNet-50 He et al. (2016) for both image representation and frame
classification (ResNet-50). We use BERT Vaswani et al. (2017) to represent
text from the article headlines (BERT headline), the image Web Entity API tags
(BERT API), or the automatically generated captions from images (BERT
Caption). We also experiment with other article-derived information in the
form of text: the headline concatenated with the automatically generated
extractive summary (BERT headline + Summary), or with the first three
sentences of the article (BERT headline + 3sentences), a typical baseline for
extractive summarization. For all text-form information derived from the
article (irrespective of whether the text was extracted from the image,
headline or body of the article), we follow the state-of-the-art methodology
for frame detection based on news headlines Liu et al. (2019), which
constitutes our baseline (BERT headline). Specifically, we use the text as
input into BERT’s pre-trained base uncased model and fine-tune the model to
predict the frames of the articles over 25 different random seeds to avoid the
fine-tuning instability due to the small dataset size (Devlin et al., 2019;
Dodge et al., 2020; Mosbach et al., 2021). In all our models, the number of
epochs is 10, the batch size is 4, and the learning rate is 2e-5.

Using Image-derived Visual Features (ResNet-50). We predict news frames based
only on the raw lead images of the news articles. We use the ResNet-50 model
He et al. (2016), pre-trained on ImageNet Deng et al. (2009), and replace the
output layer of the original ResNet-50 network with a flattened layer of 512
nodes followed by a dropout layer with a 0.5 dropout rate to the frame
classification (9-nodes) layer.

All images are scaled to $224\times 224$ pixels and are normalized based on
the mean and standard deviation of ImageNet.

Using Image-derived Visual Annotations (SRE). We create a 19-length feature
vector for each image obtained from the Subject, Race/Ethnicity (SRE)
annotations of the image that indicate the human coders’ background knowledge
of the image’s central Subject, and its connection to Race/Ethnicity. We train
a logistic regression frame classifier with this feature vector as input.

Extracting Image-derived Textual Features: Google Web Entity API Tags  (BERT
API). Here, frames are predicted based only on the Google Web Entity tags of
lead images. Web Entity detection is a Google cloud service that reads an
image as input and returns a ranked list of web entities as tags. For each
image, we form a sentence by concatenating the top-10 Web Entity tags returned
for the image.

Extracting Image-derived Textual Features: Image Captions (BERT Caption).

We follow Tran et al. (2020) to generate captions for the lead images of news
articles. The model introduced in the paper consists of different encoders
generating representations for each modality (article text, images, faces, and
objects), and a Transformer as the decoder attending over text, images, image
faces and objects.

It uses Byte-Pair-Encoding, breaking sequences into subwords and then merging
common sequences into larger words. This leads to better generalization and
prediction of out-of-vocabulary words and names, and ultimately to
linguistically rich captions for images accompanying each news article.

We follow all default settings and parameters suggested in the paper, and use
RoBERTa Liu et al. (2020) as the article text encoder, a ResNet-152 Dauphin et
al. (2017) pretrained on ImageNet as the image encoder, MTCNN Zhang et al.
(2016) as the face detector, and YOLOv3 Redmon and Farhadi (2018) as the
object detector, with the latter two operating as the specialized image face
and object attention modules, respectively. All representations obtained from
the individual encoders are fed into a four block Transformer decoder, which
employs a multi-head multi-modal attention mechanism and generates byte-pair
encoded tokens, that are finally concatenated to form the caption.

Extracting Article-derived Textual Features: Summary (BERT Summary)

We automatically extract the summary of the article, following Liu and Lapata
(2019), that uses BERT to represent sentences, and inter-sentence Transformer
layers on top of the BERT encoder to classify whether a sentence should be in
the extractive summary.

###  4.2 Multimodal

In this set of experiments, we predict news frames using multiple modes of
information derived from both the image and the article. Using Image-derived
Visual and Textual Features and Article-derived Textual Features (ResNet-50 +
BERT Headline, ResNet-50 + BERT Headline + API, ResNet-50 + BERT Headline +
Caption). Here, frames are predicted with multiple input modalities (visual
and textual features). We follow a simple concat fusion approach, which allows
us to build a modular pipeline, obtain the text and visual representations
from their respective modules, and use them to predict the frame class.
Specifically, we use ResNet-50 representations of the raw images and, as
suggested by Devlin et al. (2019) for best performance, representations of the
text obtained from the concatenation of the contextual embeddings of the last
four layers of BERT, which has been fine-tuned for frame classification, as
inputs to our multimodal 3-layer feed forward fully connected classifier
neural network that we then train jointly with ResNet-50.

We use the AdamW optimizer, cross entropy loss, and “no improvement in
validation accuracy over 5 epochs” as the stopping criterion.

Using Image-derived Visual Annotations and Article-derived Textual Features
(BERT Headline + SRE). We concatenate BERT representation of the headline with
the SRE feature vector and train jointly with fine-tuning BERT using a 1-layer
feed forward fully connected classifier neural network added on top of BERT.

Using Image-derived Textual Features and Article-derived Textual Features
(BERT Headline + API, BERT Headline + Caption). Here we concatenate article
headlines and image-derived textual features (API tags or captions) as input
to fine-tune BERT for frame classification.

###  4.3 Relevance

We use the article headline (BERT Headline) and the image-derived features
(BERT API, BERT Caption), the SRE annotations (SRE), and their combinations
(BERT Headline + API, BERT Headline + Caption, ResNet-50 + BERT Headline,
ResNet-50 + BERT Headline + API, ResNet-50 + BERT Headline + Caption) to
predict the relevance of the images to the frames of their headlines. We
perform a 4-fold cross-validation binary classification for relevance
prediction, with the same BERT and ResNet-50 architectures and hyperparameters
as before. Accuracies with uni- and multimodal information sources are
reported in Table 4. To mimic the relevance annotation process of our coders,
who are given the labeled frame of the headline to decide whether the lead
image is relevant to it, we provide our relevance prediction model with
headline frame. We concatenate the frame label to the input of the top
performing models of Table 4 and their combinations, and report accuracies in
Table 5.

Method |  ResNet-50 |  SRE |  BERT headline |  BERT API |  BERT Caption |  BERT headline + API |  BERT headline + Caption |  BERT headline + Summary |  BERT headline + 3sentences |  ResNet-50 + BERT headline |  ResNet-50 + BERT headline + API |  ResNet-50 + BERT headline + Caption |  BERT headline + SRE
---|---|---|---|---|---|---|---|---|---|---|---|---|---
All Articles |
|  9.3 |  49.2 |  81.9 |  47 |  48.1 |  82 |  82 |  82.4 |  81.8 |  13.8 |  13.7 |  12.2 |  81.5
Articles with Relevant Images |
|  42.8 |  81.2 |  83 |  72.1 |  72.5 |  87 |  84.6 |  83.1 |  83.1 |  49.7 |  65.3 |  63.8 |  83.2

Table 3: Overall micro accuracy of our methods for frame classification for
All articles and Articles with Relevant Images. BERT headline is the baseline
we compare to Liu et al. (2019)

| BERT |  | BERT | BERT | BERT headline | BERT Headline | ResNet-50 | ResNet-50 | ResNet-50
---|---|---|---|---|---|---|---|---|---
Method | headline | SRE | API | Caption | \+ API | \+ Caption | \+ BERT headline | \+ BERT headline + Caption | \+ BERT headline + API
| 62 | 68.1 | 59.3 | 62.5 | 65.8 | 65.7 | 55.5 | 60 | 58.3

Table 4: Overall micro accuracy of our methods for image relevance
classification (without frame label) for All articles.

| BERT headline |  | SRE
---|---|---|---
Method: | \+ API tag | SRE | \+ BERT headline
| \+ Frame | \+ Frame | \+ API tag + Frame
| 74.2 | 71.0 | 72.0

Table 5: Overall micro accuracy of our methods for image relevance
classification with frame for All articles.

##  5 Discussion of Results

Despite the challenges of a highly nuanced multi-class frame identification
and an intrinsically imbalanced dataset, we achieve a high prediction accuracy
of up to 87% for Articles with Relevant Images, and 82.4% for All Articles
(Table 3). It is instructive to examine the utility of article- and image-
derived features, and a fusion of all in cases where the lead image is
relevant to the article headline. We observe that contextual information
derived from the image, in the form of API tags or a caption, along with the
article headline, can drive the article perspective more clearly.

The headline + API tags combination provides the best performance (87%),
compared to image-only (43%), the SRE image annotation (81.2% ), or the the
article headline (83%) which is a strong unimodal baseline. Even when
considering examples with irrelevant images, adding API to headlines does not
hurt performance and is comparable to using headlines alone, which is unlike
SRE whose performance drops significantly for articles with irrelevant images
as these annotations are designed with relevant, i.e., frame-implying images
in mind. Furthermore, SRE requires training another model to produce these
annotations automatically, which is not trivial as they capture real-world
knowledge of the subjects in the image, e.g., whether the person in the image
is a politician (cuing politics frame) or a gun activist/NRA representative
(cuing 2nd Amendment). These findings, namely that API tags, captions, or SRE
yield higher accuracy than the raw image alone, indicate the importance of the
contextual or background knowledge of the lead image in driving the news frame
This strongly suggests that the highly nuanced task of frame prediction is
challenging using images in isolation. Our findings also confirm previous
observations that training with multiple input modalities, e.g., both visual
and textual inputs is hard as each modality may generalize differently and
hence underperform when trained jointly Wang et al. (2020).

In terms of relevance prediction, the performance is highest for models
supplied with frame labels, mimicking the relevance annotation process. Given
a headline and a frame, our method can correctly predict the relevance of an
image to the frame with 74% accuracy using the image’s API tag. Without frame
labels, however, the accuracy of the top-performing method drops to 68.1% and
is based on SRE only. While several SRE categories are strong indicators for
certain frames, e.g., the presence of demonstrators suggests the public
opinion frame, the use of SRE at inference time necessitates the training of
another model for predicting these annotations, which is not trivial.

Examining the content of the API tags and captions, we observe that API tags
have significantly more proper nouns (71% to 29% of all words in
tags/captions) and named entities (53% to 31%) than captions. On the other
hand, captions have more common nouns and verbs. Since the performance of
frame prediction for articles with relevant images is higher when using
headline and API compared to headline and caption, this suggests that frames
can be directly cued by lexical items such as proper nouns or named entities,
e.g., politicians’ names cue politics frame Mendelsohn et al. (2021). Since
models may lack real-world knowledge required to identify these, especially
when there is insufficient text evidence, e.g., when using headlines, API tags
that provide this background knowledge from images facilitate frame
prediction.

Figure 2: Frame relevance ratio and average concreteness.

Figure 3: Per Frame F1 score of the best performing frame prediction methods for (a) All Articles (left), (b) Articles with Relevant Images (right). Error Type |  Description |  Examples
---|---|---
Plausible Interpetation |  Predicted frames can be appropriate labels. |  Example Frame: Mental Health Headline: Florida shooter a troubled loner with white supremacist ties • Model erroneously predicted Race/Ethnicity: white \+ API tag: Nikolas Cruz Stoneman Douglas High School shooting Marjory Stoneman Douglas Murder Mass shooting AR-15 style rifle Suspect Student • Model correctly predicted Mental Health: Suspect Student
Inferring frames not explicitly cued in text |  Predicted frames capture an author’s intention without sufficient text evidence. |  Example Frame: 2nd Amendment/Gun Rights Headline: The NRA Versus the Constitution • Model erroneously predicted Gun Control: NRA \+ API tag: Pennsylvania Obergefell v. Hodges Supreme Court of the United States Concealed carry Rights Reciprocity Act of 2017 Constitution of the United States • Model correctly predicted 2nd Amendment: Rights,Constitution
Missing necessary contextual knowledge |  Frames can be directly cued by lexical items (e.g. politicians’ names cue Politics frame), yet the model lacks real-world knowledge required to identify those. |  Example Frame: Politics Headline: In closed-door meeting, Roskam brings pro-gun rights teens to talk gun violence prevention with students • Model erroneously predicted 2nd Amendment/Gun Rights: pro-gun, rights \+ API tag: Peter Roskam Republican Party Democratic Party United States Congress Illinois Member of Congress 2018 United States elections United States House of Representatives Presidency of Donald Trump House Committee on Ways and Means • Model correctly predicted Politics: using lexical cues of politician and party names and political terms: Trump, Republican, Democratic, Congress
Overgeneralizing highly-correlated features along with (long-distance dependencies) |  Highly correlated words and phrases that do not directly cue frames, and are used in different contexts. |  Example Frame: Race/Ethnicity Headline: Lawyers call US gun charges for Mexican man vindictive • Model erroneously predicted Mental Health: vindictive \+ API tag: Shooting of Kate Steinle Acquittal Murder San Francisco Homicide Jury Death Defendant Illegal immigration Manslaughter • Model correctly predicted Race/Ethnicity: better context
Table 6: Framing classification common error types, their definition and
examples indicating the prediction error and how additional features in our
top-performing methods of BERT Headline + API can drive correct predictions.

One possible cause for why predicting or even deciding on image relevance is
challenging (e.g., in our dataset roughly half of the lead images are
irrelevant) may be related to the nature of frames and images of U.S. gun
violence coverage. Although frames or perspectives are abstract concepts, some
frames such as Society/Culture, which focuses on society-wide factors related
to gun violence555Definitions of gun violence frames taken from the publicly
available GVFC codebook and dataset https://derrywijaya.github.io/GVFC.html,
are by nature more abstract and thus harder to convey through images than more
concrete frames, such as Politics, which focuses on the political issues
around guns and can be expressed more easily via images of politicians.

As the ability of images to usefully represent a word is strongly dependent on
how concrete or abstract the word is Gilhooly and Logie (1980); Friendly et
al. (1982), a measure of frame concreteness or the ease of identifying
tangible concepts and mental images that arise in correspondence to the frame,
should relate to the ease of expressing frames via images or the ratio of
relevant images for frames (Table 1).

To measure concreteness of frames and test this hypothesis, we trained a
regression network that takes as input a word’s vector representation as
extracted and concatenated from the last 4 layers of the pre-trained BERT
model and outputs its concreteness measure between 1 (most abstract) to 5
(most concrete). We use a dataset created by Brysbaert et al. (2014), which
contains human evaluations of concreteness for 39,954 English words, to train
and evaluate the network, achieving a high 0.95 Pearson’s Correlation between
our concreteness predictions and the ground-truth measures. The concreteness
of a frame is then measured as the average concreteness of non named-entity
words in its headlines (we treat named-entities as having a concreteness
measure of 5). As seen in Figure 2, no frame has a high (> 4) average
concreteness.

We observe, however, that some of the more concrete frames have higher ratios
of their images annotated as relevant, e.g., Politics, which is the most
concrete frame, has 65% of its images annotated as relevant compared to just
10% for Society/Culture, the least concrete frame.

In fact, for most frames, a higher average concreteness implies a higher image
relevance ratio. Exceptions to this are Economic Consequences and
Race/Ethnicity. Although more words in the former are abstract (e.g., sales,
demand, supply), it is relatively easy to identify relevant images for
economic consequence: e.g. company logos, gun stores. On the other hand,
although more words in Race/Ethnicity may be concrete: e.g.,
people/organization names, ethnic minority group names, or hate group names,
it is harder to find relevant images for this frame in news articles. This may
be due to editors in mainstream media, and photographers or journalists,
withholding certain imagery from readers for fear of causing offence or shock,
or for fear that a part of their audience may abandon the publication
altogether Ritchin (2014). Thus, although we find that frame concreteness is
related to image relevance ratio (Pearson correlation of 0.69), there may be
other factors that influence the choice of images for news articles that are
beyond relevance to frames.

We also report per frame classification F1 scores for the baseline and our
best performing models on Articles with Relevant Images and All Articles,
i.e., when using the article headline alone, or with the the API, the Caption,
and the Summary

in Figure 3. Performance when using information from articles and images is
remarkable for frames with either high image relevance ratios or high
concreteness.

In Articles with Relevant Images, the frames with the highest image relevance
ratio i.e., Politics shows an impressive 96.6% F1 score with the headline and
the API tags, followed by Public Opinion (87.1%); while frames with few
relevant images (2nd Amendment, Society/Culture), have a substantially lower
F1 scores.

racial group | headlines | images
---|---|---
| all | P | PO | V | all | P | PO
white |  0.92 |  100 |  - |  - |  50.6 |  16 |  71
black |  2.15 |  7 |  - |  93 |  4.2 |  14 |  45
white & black |  0.69 |  - |  - |  - |  2.7 |  3 |  84
asian |  - |  - |  - |  - |  0.46 |  16 |  50
white & asian |  - |  - |  - |  - |  0.006 |  - |  100
hispanic |  0.07 |  100 |  - |  - |  - |  - |  -
jewish |  7.1 |  2 |  - |  98 |  - |  - |  -
other |  89 |  - |  - |  - |  - |  - |  -

Table 7: In column all we see the percentage (%) of headline mentions or image
portrayals of certain racial groups in the 1,300 articles of the GVFC dataset.
In columns P, PO, V, we can see the percentage (%) of the people in each of
these groups who are either the Perpetrator, POlitician (or Public Figure), or
Victim.

On All Articles, the inclusion of articles with irrelevant images can hurt
performance for frames with high image relevance ratio such as Politics and
Public Opinion. However, other frames may benefit from having more examples to
learn from. For example, we observe that a low image relevance but a highly
concrete frame such as Race/Ethnicity benefits significantly from this
inclusion, reaching a high F1 score of 90.6% using All Articles from 82.1%
using only Articles with Relevant Images as the model learns more lexical
cues, e.g., named-entities from headlines of more articles, including those
with irrelevant images.

Concreteness may also augment relevance in explaining improved performance for
some frames on All Articles. We observe high correlations between frame
average concreteness and average F1 scores on articles with relevant images
(Pearson correlation of 0.93) and on All Articles (Pearson correlation of
0.94), which exceed correlations between frame image relevance ratio and
average F1 scores (Pearson correlation of 0.81 and 0.67 on articles with
relevant images and on All Articles, respectively). These findings suggest
that concreteness might be worth exploring for frame prediction and use of
imagery in the future, in addition to concreteness annotation in framing
datasets.

To complete our analysis, we applied the frame prediction error taxonomy
proposed by Mendelsohn et al. (2021) to our news framing with image- and
article-derived information, to identify and summarize common classification
errors in Table 6. We provide specific examples, highlight possible error
sources and observe how background information in BERT Headline + API, drives
correct predictions, illustrating our previous remarks.

##  6 Conclusions

We presented the first ever study and dataset on computational multimodal
framing. Our results show that image-derived contextual features can be useful
for providing missing contextual or background information that can improve
frame prediction significantly, particularly for concrete frames or frames
with relevant images. We also proposed methods for predicting frame image
relevance and for measuring frame concreteness, which we define as the ease of
expressing frames via images.

##  7 Ethical Considerations

Regarding the data we collected i.e., the Gun Violence Frame Corpus, we have
made sure that there is no design experiment that was biased toward extracting
only articles from a particular ethnic or minority group. We collect articles
that had at least one keyword in their headlines from the following list,
based on previous literature on gun violence framing analysis as described in
Liu et al. (2019). The keywords are gun, firearm, NRA, 2nd amendment, second
amendment, AR15, assault weapon, rifle, Brady act, Brady bill, mass shooting.
The articles were retrieved in 2018 from 21 media outlets, from a list of top,
in terms of website traffic, U.S. news websites; and synthesizing these lists
towards creating one list that contained news sites from the left, center, and
right sides of the ideological spectrum based on categories defined in
MediaCloud ; Pew Research Center (2016); Ad Fontes Media (2019).

Our analysis of the headlines and images gave each racial group’s mentions’
and portrayals’ percentages provided in Table 7. We notice that when racial
groups are mentioned in news headlines (which is only in $\sim$11% of all
headlines), they are used to refer to victims of race-related gun violence
incidents. Among the mentions, Blacks and Jews are the most common, as the
corpus contains articles from 2018 that reported the mass shooting at
Pittsburgh
Synagogue666https://en.wikipedia.org/wiki/Pittsburgh_synagogue_shooting and
several high-profile shootings of black men–widely reported as instances of
the controversial and race-related “Stand Your Ground
Law"777https://en.wikipedia.org/wiki/Shooting_of_Markeis_McGlockton, all of
which occurred in 2018.

In terms of images, we notice that they are dominated by white people
($\sim$50% of all images). However, the majority of them ($\sim$71% of all
white people images) are images of politicians or public figures related to
gun laws/debates. There are much less images of victims and perpetrators (only
$\sim$9% of all images each). In terms of victims vs. perpetrators, there are
more images of black victims ($\sim$1.8%) than black perpetrators
($\sim$0.6%). The same applies to Asian, while for whites the numbers of
victim and perpetrator images are more balanced. Based on our data analysis,
in which we saw different coverage in terms of racial groups in headlines vs.
images, examining the difference between the race of the people mentioned in
headlines and the race of those portrayed in the images would be an
interesting future research direction.

## Acknowledgments

This work is supported in part by the U.S. NSF grant 1838193, DARPA
HR001118S0044 (the LwLL program), and the Department of the Air Force
FA8750-19-2-3334 (Semi-supervised Learning of Multimodal Representations). The
U.S. Government is authorized to reproduce and distribute reprints for
Governmental purposes. The views and conclusions contained in this publication
are those of the authors and should not be interpreted as representing
official policies or endorsements of DARPA, the Air Force, and the U.S.
Government.

## References

  * Ad Fontes Media (2019) Ad Fontes Media. 2019.  The media bias chart.
  * Akyürek et al. (2020) Afra Feyza Akyürek, Lei Guo, Randa Elanwar, Prakash Ishwar, Margrit Betke, and Derry Tanti Wijaya. 2020.  Multi-label and multilingual news framing analysis.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 8614–8624, Online. Association for Computational Linguistics.
  * Brysbaert et al. (2014) Marc Brysbaert, Amy Beth Warriner, and Victor Kuperman. 2014.  Concreteness ratings for 40 thousand generally known english word lemmas.  _Behavior research methods_ , 46(3):904–911.
  * Burns et al. (2020) Andrea Burns, Donghyun Kim, Derry Wijaya, Kate Saenko, and Bryan A Plummer. 2020.  Learning to scale multilingual representations for vision-language tasks.  In _European Conference on Computer Vision_ , pages 197–213. Springer.
  * Caglayan et al. (2019) Ozan Caglayan, Pranava Madhyastha, Lucia Specia, and Loïc Barrault. 2019.  Probing the need for visual context in multimodal machine translation.  _arXiv preprint arXiv:1903.08678_.
  * Caple (2010) Helen Caple. 2010.  What you see and what you get: The evolving role of news photographs in an Australian broadsheet.  _Journalism and Meaning-Making: Reading the Newspaper_ , pages 199–220.
  * Card et al. (2015) Dallas Card, Amber E. Boydstun, Justin H. Gross, Philip Resnik, and Noah A. Smith. 2015.  The media frames corpus: Annotations of frames across issues.  In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_ , pages 438–444, Beijing, China. Association for Computational Linguistics.
  * Coleman and Wu (2015) Renita Coleman and Denis Wu. 2015.  _Image and emotion in voter decisions: The affect agenda_.  Lexington Books.
  * Dan (2017) Viorela Dan. 2017.  _Integrative framing analysis: Framing health through words and visuals_.  Routledge.
  * Dauphin et al. (2017) Yann N. Dauphin, Angela Fan, Michael Auli, and David Grangier. 2017.  Language modeling with gated convolutional networks.  In _Proceedings of the 34th International Conference on Machine Learning_ , volume 70 of _Proceedings of Machine Learning Research_ , pages 933–941. PMLR.
  * Deng et al. (2009) Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009.  ImageNet: A large-scale hierarchical image database.  In _IEEE Computer Vision and Pattern Recognition (CVPR)_ , pages 248–255.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  BERT: Pre-training of deep bidirectional transformers for language understanding.  _CoRR_ , abs/1810.04805.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Dodge et al. (2020) Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, and Noah Smith. 2020.  Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping.  _arXiv preprint arXiv:2002.06305_.
  * Drakulich (2015) Kevin M Drakulich. 2015.  Explicit and hidden racial bias in the framing of social problems.  _Social Problems_ , 62(3):391–418.
  * Entman (1993) Robert M Entman. 1993.  Framing: Toward clarification of a fractured paradigm.  _Journal of communication_ , 43(4):51–58.
  * Field et al. (2018) Anjalie Field, Doron Kliger, Shuly Wintner, Jennifer Pan, Dan Jurafsky, and Yulia Tsvetkov. 2018.  Framing and agenda-setting in russian news: a computational analysis of intricate political strategies.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018_ , pages 3570–3580.
  * Friendly et al. (1982) Michael Friendly, Patricia E Franklin, David Hoffman, and David C Rubin. 1982.  The toronto word pool: Norms for imagery, concreteness, orthographic variables, and grammatical usage for 1,080 words.  _Behavior Research Methods & Instrumentation_, 14(4):375–399.
  * Gilhooly and Logie (1980) Ken J Gilhooly and Robert H Logie. 1980.  Age-of-acquisition, imagery, concreteness, familiarity, and ambiguity measures for 1,944 words.  _Behavior research methods & instrumentation_, 12(4):395–427.
  * Guo et al. (2021) Lei Guo, Kate Mays, Yiyan Zhang, Derry Wijaya, and Margrit Betke. 2021.  What makes gun violence a (less) prominent issue? a computational analysis of compelling arguments and selective agenda setting.  _Mass communication and society_ , pages 1–25.
  * Hamborg et al. (2019) Felix Hamborg, Anastasia Zhukova, and Bela Gipp. 2019.  Illegal aliens or undocumented immigrants? Towards the automated identification of bias by word choice and labeling.  Technical report, University of Konstanz, Germany.
  * He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016.  Deep residual learning for image recognition.  In _IEEE Conference on Computer Vision and Pattern Recognition_ , pages 770–778.
  * Hewitt et al. (2018) John Hewitt, Daphne Ippolito, Brendan Callahan, Reno Kriz, Derry Tanti Wijaya, and Chris Callison-Burch. 2018.  Learning translations via images with a massively multilingual image dataset.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 2566–2576.
  * Khani et al. (2021) Nikzad Khani, Isidora Tourni, Mohammad Sadegh Rasooli, Chris Callison-Burch, and Derry Tanti Wijaya. 2021.  Cultural and geographical influences on image translatability of words across languages.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 198–209.
  * Kim et al. (2020) Donghyun Kim, Kuniaki Saito, Kate Saenko, Stan Sclaroff, and Bryan Plummer. 2020.  Mule: Multimodal universal language embedding.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 34, pages 11254–11261.
  * Krippendorff (2018) Klaus Krippendorff. 2018.  _Content analysis: An introduction to its methodology_.  Sage publications.
  * Lin et al. (2017) Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017.  Focal loss for dense object detection.  In _IEEE International Conference on Computer Vision_ , pages 2980–2988.
  * Liu et al. (2019) Siyi Liu, Lei Guo, Kate Mays, Margrit Betke, and Derry Tanti Wijaya. 2019.  Detecting frames in news headlines and its application to analyzing news framing trends surrounding U.S. gun violence.  In _Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)_ , pages 504–514, Hong Kong, China. Association for Computational Linguistics.
  * Liu and Lapata (2019) Yang Liu and Mirella Lapata. 2019.  Text summarization with pretrained encoders.  _arXiv preprint arXiv:1908.08345_.
  * Liu et al. (2020) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2020.  Ro{bert}a: A robustly optimized {bert} pretraining approach.
  * Lowrey (2002) Wilson Lowrey. 2002.  Word people vs. picture people: Normative differences and strategies for control over work among newsroom subgroups.  _Mass Communication & Society_, 5(4):411–432.
  * (32) MediaCloud. 2018.  https://mediacloud.org.
  * Mendelsohn et al. (2021) Julia Mendelsohn, Ceren Budak, and David Jurgens. 2021.  Modeling framing in immigration discourse on social media.  _arXiv preprint arXiv:2104.06443_.
  * Messaris and Abraham (2001) Paul Messaris and Linus Abraham. 2001.  The role of images in framing news stories.  In _Framing public life_ , pages 231–242. Routledge.
  * Mosbach et al. (2021) Marius Mosbach, Maksym Andriushchenko, and Dietrich Klakow. 2021.  On the stability of fine-tuning {bert}: Misconceptions, explanations, and strong baselines.  In _International Conference on Learning Representations_.
  * Naderi and Hirst (2017) Nona Naderi and Graeme Hirst. 2017.  Classifying frames at the sentence level in news articles.  In _Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017, Varna, Bulgaria, September 2 - 8, 2017_ , pages 536–542. INCOMA Ltd.
  * Nguyen et al. (2013) Viet-An Nguyen, Jordan L Ying, and Philip Resnik. 2013.  Lexical and hierarchical topic regression.  In _Advances in Neural Information Processing Systems_ , volume 26. Curran Associates, Inc.
  * Paivio et al. (1968) Allan Paivio, John C Yuille, and Stephen A Madigan. 1968.  Concreteness, imagery, and meaningfulness values for 925 nouns. _Journal of experimental psychology_ , 76(1p2):1.
  * Pew Research Center (2016) Pew Research Center. 2016.  Ideological placement of each source’s audience.
  * Powell et al. (2015) Thomas E Powell, Hajo G Boomgaarden, Knut De Swert, and Claes H de Vreese. 2015.  A clearer picture: The contribution of visuals and text to framing effects.  _Journal of Communication_ , 65(6):997–1017.
  * Rasooli et al. (2021) Mohammad Sadegh Rasooli, Chris Callison-Burch, and Derry Tanti Wijaya. 2021.  " wikily" neural machine translation tailored to cross-lingual tasks.  _arXiv preprint arXiv:2104.08384_.
  * Redmon and Farhadi (2018) Joseph Redmon and Ali Farhadi. 2018.  Yolov3: An incremental improvement.  _CoRR_ , abs/1804.02767.
  * Reese et al. (2001) Stephen D Reese, Oscar H Gandy Jr, August E Grant, et al. 2001.  _Framing public life: Perspectives on media and our understanding of the social world_.  Routledge.
  * Ritchin (2014) Fred Ritchin. 2014.  Why violent news images matter.  [time.com;4-September-2014].
  * Specia et al. (2016) Lucia Specia, Stella Frank, Khalil Sima’An, and Desmond Elliott. 2016.  A shared task on multimodal machine translation and crosslingual image description.  In _Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers_ , pages 543–553.
  * Tran et al. (2020) Alasdair Tran, Alexander Mathews, and Lexing Xie. 2020.  Transform and tell: Entity-aware news image captioning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 13035–13045.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in Neural Information Processing Systems_ , volume 30. Curran Associates, Inc.
  * Wang et al. (2020) Weiyao Wang, Du Tran, and Matt Feiszli. 2020.  What makes training multi-modal classification networks hard? In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 12695–12705.
  * Wessler et al. (2016) Hartmut Wessler, Antal Wozniak, Lutz Hofer, and Julia Lück. 2016.  Global multimodal news frames on climate change: A comparison of five democracies around the world.  _The International Journal of Press/Politics_ , 21(4):423–445.
  * Yao and Wan (2020) Shaowei Yao and Xiaojun Wan. 2020.  Multimodal transformer for multimodal machine translation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 4346–4350.
  * Zhang et al. (2016) Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, and Yu Qiao. 2016.  Joint face detection and alignment using multitask cascaded convolutional networks.  _IEEE Signal Processing Letters_ , 23(10):1499–1503.

##  Appendix A Appendix

The first 16 entries in the SRE feature vector indicate the central Subject of
the image, with each Subject implying certain frame(s):

  1. 1.

People: Gun shooter/suspect (Mental Health)

  2. 2.

People: Gun hobbyist/activist + gun-related activities with a hand (Gun
Rights)

  3. 3.

People: Victim/affected family and friends/bystanders (Public Opinion)

  4. 4.

People: Politicians (Politics)

  5. 5.

People: Law enforcement (e.g., police offers, security guards) (Public Safety)

  6. 6.

Object: Firearm/bullets (can mean anything or Gun Control for certain gun
images)

  7. 7.

Object: Gun /hunting gear stores/gun show (Economic Consequences or Gun
Rights)

  8. 8.

People: Demonstrators/Demonstrations (Public Opinion)

  9. 9.

Object: Protest signs (Gun Control or Gun Rights)

  10. 10.

People/mainly object: Memorials objects and people (Public Opinion)

  11. 11.

Object/people: Crime scene/police cars/people during or right after the crisis
(episodic frame)

  12. 12.

Object: Legislative buildings/courthouses (Gun Control or Politics)

  13. 13.

Object/people: School/campus/students indicating school/campus (Public Safety)

  14. 14.

NRA objects/NRA representatives (Gun Rights or Economic Consequences)

  15. 15.

Object: Company buildings/logos (Economic Consequence)

  16. 16.

Other

The last three entries in the feature vector indicate relevance to Race or
Ethnicity:

  1. 17.

None

  2. 18.

Racial/ethnic minority groups /buildings of a specific group (Ethnicity) (only
if the central subject is from racial/ethnic minority group \- not if there is
only one or a couple of non-White people in a large crowd)

  3. 19.

KKK/white supremacy/hate groups (Ethnicity)

Frame | Description | Examples
---|---|---
Politics | Political issues around guns and shootings, including: |  Baltimore students walk out of class to protest gun violence
|  • Political campaigns and upcoming elections |
| (e.g., using guns as a wedge issue or motivating force to get people to the polls) |
|  • Fighting between the Democratic and Republican parties, or politicians |
|  • Political money contributions from gun lobbies (e.g., NRA) |
|  • One political party or one politician’s stance on gun violence. |
|  • Therefore, as long as the news headline mentions a politician’s name, |
| it often indicates the theme of politics. |
|  • Often times, the politicians’ names or the party names should be mentioned. |
Public Opinion | Public’s opinion |  How Illinois governor candidates would address gun violence
| and the community’s reactions to gun-related issues, including: |  Trump warns Dems will ’take away your Second Amendment’
| (e.g., using guns as a wedge issue or motivating force to get people to the polls) |  Lindsey Graham: Both parties will suffer if Congress doesn’t act on new gun bill
|  • Public opinion polls related to guns |
|  • Protests |
|  • One political party or one politician’s stance on gun violence. |
|  • Mourning victims of gun violence |
|  • The public’s emotional responses |
Gun Control/Regulation | Issues related to regulating guns |  GOP lawmaker calls for age restriction on AR-15s
| through legislation and other institutional measures: |
|  • Enforcing and/or expanding background checks |  No bump stocks turned in to Denver police after ban
|  • Limiting sale of guns and/or related dangerous equipment |
| (e.g., AR15s, semi-automatic rifles, bump stocks, Huge-capacity ammo) |
|  • Increasing age limits on gun purchases |
|  • Implementing licensing and gun safety training programs |
School/Public Space Safety | Issues related to institutional and school safety, including: |  Preschoolers among students required to carry clear backpacks in Texas school district
|  • Awareness and monitoring of troubled individuals by law enforcement (e.g., local police, FBI) |  Scott wants armed police at Stoneman Douglas after disturbing incidents at Parkland school
|  • Safety measures in schools to prevent or mitigate shootings |  Sales of bulletproof school supplies spike after Florida shooting
| (e.g., police/safety officers in the school, armed teachers, metal detectors, clear backpacks) |
|  • Note that a headline simply mentioning school shooting |
| does not necessarily mean it uses this safety measure frame. |
Economic consequences | Financial losses or gains, or the costs involved in gun-related issues, including: |  The NRA Is In Deep, Deep Financial Trouble
|  • The actual sales of firearms |
|  • The financial consequences of gun regulation |
| (e.g., lost tax revenue, or gun manufacturing companies moving to a different state) |
|  • The financial state of gun-related lobbying groups (e.g., the NRA) |
|  • Federal budget for gun-related programs |
Race/Ethnicity | Gun issues related to certain ethnic group(s), including: |  Illegal immigrant acquitted of Kate Steinle’s murder faces judge on gun charges
|  • Angry, isolated white men as primary perpetrators of domestic gun violence |  The disparities in how black and white men die in gun violence, state by state
|  • Immigrants from Mexico bringing in guns from across the border |
|  • Muslim terrorists |
|  • Gun violence in African American communities |
Mental Health | Issues related to individuals’ mental illnesses or emotional well-being, |  Gun debate hits home for families dealing with myths about violence, mental illness
| or the mental health system as a whole, including: |
|  • Predicting and preventing mental health breakdowns |  Renewed Debate Over Gun Access, Mental Health
|  • Treating mental illness |  Las Vegas gunman lost money, became unstable before shooting
|  • Creating measures to ensure mentally ill people do not have access to guns |
|  • Descriptions of individuals’ behavioral / personality traits |
| that indicate instability, impulsivity, anger, etc. |
2nd Amendment/Gun Rights | Related to the Constitution, the second amendment, |  Membership, interest in gun rights groups soar in the weeks after the Florida high school shooting
| and protection of individual liberty and gun ownership as a right, including: |
|  • Meaning of the 2nd amendment |  Rapper ‘Killer Mike,’ NRA host Colion Noir: No guns would turn people into slaves
|  • The irrefutability of one’s right to own guns |
|  • Gun ownership as critical to democracy and protecting oneself |
Society/Culture | Societal-wide factors that are related to gun violence, including: |  There’s Not A Single Ounce Of Evidence To Link Mass Shootings To Video Games
|  • Violence in media (e.g., TV/movies and video games) |
|  • Social pressures that may incite someone to violence (e.g., cliques/bullying and isolation) |
|  • Breakdown in family structures, so there is a lack of familial support and stability |
|  • Breakdown in community structures (e.g., religious organizations, other civic-oriented groups), |
| so there is a lack of community support and stability |

Table 8: News frames’ description and Headline examples.


# Explanation by Automated Reasoning Using the Isabelle Infrastructure
Framework

Florian Kammüller  Department of Computer Science
Middlesex University London
<EMAIL_ADDRESS>
###### Abstract

In this paper, we propose the use of interactive theorem proving for
explainable machine learning. After presenting our proposition, we illustrate
it on the dedicated application of explaining security attacks using the
Isabelle Infrastructure framework and its process of dependability
engineering. This formal framework and process provides the logics for
specification and modeling. Attacks on security of the system are explained by
specification and proofs in the Isabelle Infrastructure framework. Existing
case studies of dependability engineering in Isabelle are used as feasibility
studies to illustrate how different aspects of explanations are covered by the
Isabelle Infrastructure framework.

##  I Proposing Interactive Theorem Proving for Explainable Machine Learning

Machine Learning (ML) is everywhere in Computer Science now. One may almost
say that all of Computer Science has now become a part of ML and is viewed as
a technique within the greater realm of Data Science or Data Engineering. But
while this major trend like many other trends prevails, we should not forget
that Artificial Intelligence (AI) is the original goal of what was the
starting point of machine learning and that Automated Reasoning has been
created as a means to provide for artificial intelligent systems a mechanical
way of imitating human reasoning by implementing logics and automatizing
proof.

When we think of how to explain why a specific solution for a problem is a
solution, the purest way to do so is to explain it by way of mathematically
precise arguments – which is equivalent to providing a logically sound proof
in a mathematical model of the solution domain or context. An ML algorithm
would do the same, for example, by providing a decision tree to explain a
solution, but usually the ML explanations which are generated by the ML model
itself are very close to the ML implementation. So, they often fail to give a
satisfactory, i.e. human understandable explanation.

This paper shows our point of view on a tangible way forward to combining
interactive theorem proving with machine learning (ML). Different from the
main stream of using ML to improve automated verification, we propose an
integration at a higher level, using logical modeling and automated reasoning
for explainability of machine learning solutions. The main idea of our
proposal is based on one major fact about logic and proof:

> Reasoning is not only a very natural way of explanation but it is also the
> most complete possible one since it provides a mathematical proof on a
> formal model.

In the spirit of this thought, we provide a proof of concept on a framework
that has been established for security and privacy analysis, the Isabelle
Infrastructure framework. In this paper, we thus first introduce this
framework by summarizing its basic conepts and various applications
(SectionII). After contrasting to some other conceptual approaches to ML and
theorem proving including explanation (Section III), we briefly sketch our
conceptual proposal (Section IV)

##  II Isabelle Infrastructure Framework

The Isabelle Infrastructure framework is implemented as an instance of Higher
Order Logic in the interactive generic theorem prover Isabelle/HOL [1]. The
framework enables formalizing and proving of systems with physical and logical
components, actors and policies. It has been designed for the analysis of
insider threats. However, the implemented theory of temporal logic combined
with Kripke structures and its generic notion of state transitions are a
perfect match to be combined with attack trees into a process for formal
security engineering [2] including an accompanying framework [3].

####  II-1 Kripke structures, CTL and Attack Trees

A number of case studies have contributed to shape the Isabelle framework into
a general framework for the state-based security analysis of infrastructures
with policies and actors. Temporal logic and Kripke structures are deeply
embedded into Isabelle’s Higher Order logic thereby enabling meta-theoretical
proofs about the foundations: for example, equivalence between attack trees
and CTL statements have been established [4] providing sound foundations for
applications. This foundation provides a generic notion of state transition on
which attack trees and temporal logic can be used to express properties for
applications. The logical concepts and related notions thus provided for sound
application modeling are:

  * •

Kripke structures and state transitions:
A generic state transition relation is $\to_{i}$; Kripke structures over a set
of states t reachable by $\to_{i}$ from an initial state set I can be
constructed by the Kripke constructor as ` Kripke {t. $\exists$ i $\in$ I. i
$\to_{i}^{*}$ t} I `

  * •

CTL statements:
We can use the Computation Tree Logic (CTL) to specify dependability
properties as ` K $\vdash$ EF s ` This formula states that in Kripke structure
K there is a path (E) on which the property s (given as the set of states in
which the property is true) will eventually (F) hold.

  * •

Attack trees:
attack trees are defined as a recursive datatype in Isabelle having three
constructors: $\oplus_{\vee}$ creates or-trees and $\oplus_{\wedge}$ creates
and-trees. And-attack trees $l\mbox{{$\oplus_{\wedge}^{s}$}}$ and or-attack
trees $l\mbox{{$\oplus_{\vee}^{s}$}}$ consist of a list of sub-attacks which
are themselves recursively given as attack trees. The third constructor takes
as input a pair of state sets constructing a base attack step between two
state sets. For example, for the sets I and s this is written as
${\cal{N}}_{\texttt{(I,s)}}$. As a further example, a two step and-attack
leading from state set I via si to s is expressed as ` $\vdash$
[${\cal{N}}_{\texttt{(I,si)}}$,${\cal{N}}_{\texttt{(si,s)}}$]$\oplus_{\wedge}^{\texttt{(I,s)}}$
`

  * •

Attack tree refinement, validity and adequacy:
Attack trees can be constructed also by a refinement process but this differs
from the system refinement presented in the paper [5]. An abstract attack tree
may be refined by spelling out the attack steps until a valid attack is
reached:

$\vdash$A :: ($\sigma$:: state) attree).

The validity is defined constructively so that code can be generated from it.
Adequacy with respect to a formal semantics in CTL is proved and can be used
to facilitate actual application verification. This is used for the stepwise
system refinements central to the methodology called Refinement-Risk cycle
developed for the Isabelle Infrastructure framework [5].

A whole range of publications have documented the development of the Isabelle
Insider framework. The publications [6, 7, 8] first define the fundamental
notions of insiderness, policies, and behaviour showing how these concepts are
able to express the classical insider threat patterns identified in the
seminal CERT guide on insider threats [9]. This Isabelle Insider framework has
been applied to auction protocols [10, 11] illustrating that the Insider
framework can embed the inductive approach to protocol verification [12]. An
Airplane case study [13, 14] revealed the need for dynamic state verification
leading to the extension of adding a mutable state. Meanwhile, the embedding
of Kripke structures and CTL into Isabelle have enabled the emulation of
Modelchecking and to provide a semantics for attack trees [15, 16, 17, 4, 3].
Attack trees have provided the leverage to integrate Isabelle formal reasoning
for IoT systems as has been illustrated in the CHIST-ERA project SUCCESS [2]
where attack trees have been used in combination with the Behaviour
Interaction Priority (BIP) component architecture model to develop security
and privacy enhanced IoT solutions. This development has emphasized the
technical rather than the psychological side of the framework development and
thus branched off the development of the Isabelle Insider framework into the
Isabelle Infrastructure framework. Since the strong expressiveness of Isabelle
allows to formalize the IoT scenarios as well as actors and policies, the
latter framework can also be applied to evaluate IoT scenarios with respect to
policies like the European data privacy regulation GDPR [18]. Application to
security protocols first pioneered in the auction protocol application [10,
11] has further motivated the analysis of Quantum Cryptography which in turn
necessitated the extension by probabilities [19, 20, 21].

Requirements raised by these various security and privacy case studies have
shown the need for a cyclic engineering process for developing specifications
and refining them towards implementations. A first case study takes the IoT
healthcare application and exemplifies a step-by-step refinement interspersed
with attack analysis using attack trees to increase privacy by ultimately
introducing a blockchain for access control [3]. First ideas to support a
dedicated security refinement process are available in a preliminary arxive
paper [22] but only the follow-up publication [23] provides the first full
formalization of the RR-cycle and illustrates its application completely on
the Corona-virus Warn App (CWA). The earlier workshop publication [24]
provided the formalisation of the CWA illustrating the first two steps but it
did not introduce the fully formalised RR-cycle nor did it apply it to arrive
at a solution satisfying the global privacy policy [5].

##  III Machine Learning, Explanation and Theorem Proving

If theorem proving could automatically be solved by machine learning, we would
solve the P=NP problem [25]. Nevertheless, ML has been successfully employed
within theorem provers to enhance the decision processes. Also in Isabelle,
the sledgehammer tool uses ML mainly to select lemmas.

A very relevant work by Vigano and Magazzeni [26] focuses the idea of
explainability on security, coining the notion of XSec or Explainable
Security. The authors propose a full new research programme for the notion of
explainability in security in which they identify the “Six Ws” of XSec: Who?
What? Where? When? Why? And hoW? They position their paper clearly into the
context of some earlier works along the same lines, e.g. [27, 28], but go
beyond the earlier works by extending the scope and presenting a very concise
yet complete description of the challenges. As opposed to XAI in general, the
paper shows how already in understanding explanations only for the focus area
of security (as opposed to all application domains of IT) is quite a task.
Also they point out that XAI is merely concerned with explaining the technical
solution provided by ML, whereas XSec looks at various other levels most
prominently, the human user, by addressing domains like usable security and
security awareness, and security economics [26][p. 294].

Our point of view is quite similar to Vigano’s and Magazzeni’s but we
emphasize the technical side of explanation using interactive theorem proving
and the Isabelle Infrastructure framework, while they focus on differentiating
the notion of explanation from different aspects, for example, stake holders,
system view, and abstraction levels. However, the notion of refinement defined
for the process of dependability engineering for the Isabelle Infrastructure
framework [5] allows addressing most of the Six Ws, because our model includes
actors and policies and allows differentiation between insider and outsider
attacks, expression of awareness [23]. Thus, we could strictly follow the Ws
when explaining our proposition but we believe it is better to contemplate the
Ws simply in the context of classical Software Engineering that has similar
Ws. Moreover, the Refinement-Risk cycle of dependability engineering can be
seen as specification refinement framework that employs the classical AI
technique of automated reasoning. Surely, the human aspect versus the system
aspect on the Sic Ws of XSec brings in various different view points but these
are inherent in if the contexts, that are needed for the interpretation are
present in the model. Otherwise, they simply have to be added to it, for
example, by using refinement to integrate these aspects of reality into the
model. Then the Isabelle Infrastructure framework allows explanation for
various purposes, audiences, technical levels (HW/SW). policies, localities
and other physical aspects. Thus, we can answer all Six Ws and argue that is
what human centric software, security, and dependability engineering are all
about.

Moreover, despite contrasting from the approach by Vigano and Megazzini, we
follow the classical engineering approach of Fault-tree analysis, more
concretely using Attack Trees, and propose a dual process of attack versus
security protection goal analysis which in itself offers a direct input to ML,
for example to produce features that could be used for Decision trees as well
as metrics that could provide feedback for optimization techniques as used in
reinforcement learning.

##  IV Explaining (not only) Security by the Isabelle Infrastructure framework

This section describes the core ideas of explanation provided by applying the
Isabelle Infrastructure framework.

###  IV-A State transition systems and attack trees as a dual way of
explanation

One important aspect of explanation that is not restricted to security at all
is to provide a step by step trace of state transitions to explain how a
specific state may appear. This can explain where a problem lies, for example,
to explain how an ML algorithm arrived at a decision for a medical diagnosis
by lining up a number of steps that lead to it.

In the Isabelle Infrastructure framework the notion of state transition
systems is provided as a generic theory based on Kripke structures to
represent state graphs over arbitrary types of states and using the branching
time logic CTL to express temporal logical formulas over them. The
correspondance between the CTL formulas of reachability and attack trees and
the proof of adequacy are suitable to allow for a dual step by step analysis
of a system dove-tailing the fault analysis with a specification refinement.
This dove-tailing process leads to an elaborate process not only of explaining
faults of system designs and how they can be reached practically by a series
of actions but also an explanation of additional features of a system that are
motivated by the detected fault. For example, when it comes to human awareness
and usable security an explanation of a necessary security measure that is
imposed on a user can be readily illustrated by an attack graph or its
equivalent attack path that can be readilty produced by the adequacy theory.

###  IV-B Human and Locality Aspects

The Isabelle Infrastructure framework has initially been designed to be merely
focused on modeling and analyzing Insider threats before it became extended
into what is now known as the Isabelle Infrastructure framework. Due to this
initial motivation the framework explicitly supports the notion of human
actors within networks of physical and virtual locations. These aspects are
important to model various different stake holders to enable explanations to
different audiences having different view points and needing different levels
of detail and complexity in their explanations. For example, the explanation
of a security threat will have a substantially different form if produced for
a security analyst of to a system end user. Due to the explicit representation
of human actors as well as their locations and other variable features, the
Isabelle Infrastructure framework supports a fine grained control over the
definition of applications thus enabling very flexible support of explanation
about human aspects and suited to human understanding.

Also the human aspect necessitates consideration of the human condition, in
particular psychological characterizations. The Isabelle Infrastructure
framework, by augmenting the Isabelle Insider framework, provides for such
characterization. For example, when considering insiderness, the state of the
insider is characterized by a predicate that allows to use this state within a
logical analysis of security and privacy threats to a system. Although these
characterizations are axiomatic in the sense that the definition of the
insider predicate is based on empirical results that have been externally
input into the specification, it is in principle feasible to enrich the
cognitive model of the human in the Isabelle Insider framework. A first step
towards that has been done by experimenting with an extension of a notion of
human awareness to support additionally analysis of unintentional insiders for
human unawareness of privacy risks in social media [23].

###  IV-C Dependability Engineering: Specifying Protection Goals and
Quantifying Attackers

The process of Dependability Engineering – the Refinement-Risk (RR) cycle –
conceived for the Isabelle Infrastructure framework [22] allows a human
centric system specification to be refined step-by-step following an iteration
of finding faults within a system specification and refining this
specification by more sophisticated data types or additional rules or changes
to the semantics of system functions. The data type refinement allows
integrating for example, more restrictive measures to control data, for
example, using blockchains to enhance data consistency, or data labeling for
access controzl. This refinement is triggered by previously found flaws in the
system and thus provides concrete motivations for such design decisions
leading to constructive explanations. Similarly, additional constraints on
rules that are introduced in a refinement step of the RR-cycle are motivated
by previously found attacks, for example, the necessity to change the
ephemeral id of every user when they move to a new location instantaneously at
moving time for the Corona-Warn-App is motivated by an identification attack
[24, 5].

Since the RR-cycle is based on the idea of refinement, another requirement for
a flexible explanation comes in for free: if we want to explain to different
audiences or at different technical levels, we equally need to refine (or
abstract) definitions of data-types, rules for policies, or descriptions of
algorithms. The Isabelle Infrastructure framework directly supports these
expressions at different abstraction levels and from different view points.

###  IV-D Quantification

An important aspect is quantification for explanation. Very often an
explanation will not be possible in a possibilistic way. A quantification
could be given by adding probabililies as well as other quanitative data, like
costs, to explanations. For example, for a security attack the cost that an
attacker is estimated to invest maximally on a specific attack step is an
inevitable ingredient for a realistic attacker model. Simlarly, the likelihood
of a successful attack of a certain attack step could be needed for an
analysis. Attack trees support these types of quantification. Naturally, the
Isabelle Infrastructure framework also supports them. The application to the
security analysis of Quantum Cryptography, i.e., the modeling and analysis of
the Quantum Key Distribution protocol (QKD) lead to the extension for
probabilistic state transition systems [19, 20, 21].

Quantification can also be a useful explanation for the process of learning
for example by quantifying a distance to an attack goal. In that sense,
quantified explanation can be a useful feedback for machine learning itself.

##  V Conclusions

In this paper we have proposed the sue of Automated Reasoning in the
particular instance of the Isabelle Infrastructure framework for Explanation.
We summarized the work that lead to the creation of the Isabelle
Infrastructure framework highlighting the existing applications and
extensions. After studying some related work on explanation, we provided a
range of conceptual points that argued why and how the Isabelle Infrastructure
framework supports explanation.

## References

  * [1] T. Nipkow, L. C. Paulson, and M. Wenzel, _Isabelle/HOL – A Proof Assistant for Higher-Order Logic_ , ser. LNCS. Springer-Verlag, 2002, vol. 2283.
  * [2] CHIST-ERA, “Success: Secure accessibility for the internet of things,” 2016, http://www.chistera.eu/projects/success.
  * [3] F. Kammüller, “Combining secure system design with risk assessment for iot healthcare systems,” in _Workshop on Security, Privacy, and Trust in the IoT, SPTIoT’19, colocated with IEEE PerCom_. IEEE, 2019.
  * [4] ——, “Attack trees in isabelle,” in _20th International Conference on Information and Communications Security, ICICS2018_ , ser. LNCS, vol. 11149\. Springer, 2018.
  * [5] ——, “Dependability engineering in isabelle,” 2021, arxiv preprint, http://arxiv.org/abs/2112.04374.
  * [6] F. Kammüller and C. W. Probst, “Invalidating policies using structural information,” in _IEEE Security and Privacy Workshops, Workshop on Research in Insider Threats, WRIT’13_ , 2013.
  * [7] ——, “Combining generated data models with formal invalidation for insider threat analysis,” in _IEEE Security and Privacy Workshops, Workshop on Research in Insider Threats, WRIT’14_ , 2014.
  * [8] ——, “Modeling and verification of insider threats using logical analysis,” _IEEE Systems Journal, Special issue on Insider Threats to Information Security, Digital Espionage, and Counter Intelligence_ , vol. 11, no. 2, pp. 534–545, 2017. [Online]. Available: http://dx.doi.org/10.1109/JSYST.2015.2453215
  * [9] D. M. Cappelli, A. P. Moore, and R. F. Trzeciak, _The CERT Guide to Insider Threats: How to Prevent, Detect, and Respond to Information Technology Crimes (Theft, Sabotage, Fraud)_ , 1st ed., ser. SEI Series in Software Engineering. Addison-Wesley Professional, Feb. 2012. [Online]. Available: http://www.amazon.com/exec/obidos/redirect?tag=citeulike07-20\&path=ASIN/0321812573
  * [10] F. Kammüller, M. Kerber, and C. Probst, “Towards formal analysis of insider threats for auctions,” in _8th ACM CCS International Workshop on Managing Insider Security Threats, MIST’16_. ACM, 2016.
  * [11] ——, “Insider threats for auctions: Formal modeling, proof, and certified code,” _Journal of Wireless Mobile Networks, Ubiquitous Computing, and Dependable Applications (JoWUA)_ , vol. 8, no. 1, 2017. [Online]. Available: http://doi.org/10.22667/JOWUA.2017.03.31.044
  * [12] L. C. Paulson, “The inductive approach to verifying cryptographic protocols,” _Journal of Computer Security_ , vol. 6, no. 1-2, pp. 85–128, 1998.
  * [13] F. Kammüller and M. Kerber, “Investigating airplane safety and security against insider threats using logical modeling,” in _IEEE Security and Privacy Workshops, Workshop on Research in Insider Threats, WRIT’16_. IEEE, 2016.
  * [14] ——, “Applying the isabelle insider framework to airplane security,” _Science of Computer Programming_ , vol. 206, 2021. [Online]. Available: https://doi.org/10.1016/j.scico.2021.102623
  * [15] F. Kammüller, “A proof calculus for attack trees,” in _Data Privacy Management, DPM’17, 12th Int. Workshop_ , ser. LNCS, vol. 10436. Springer, 2017, co-located with ESORICS’17.
  * [16] ——, “Human centric security and privacy for the iot using formal techniques,” in _3d International Conference on Human Factors in Cybersecurity_ , ser. Advances in Intelligent Systems and Computing, vol. 593\. Springer, 2017, pp. 106–116, affiliated with AHFE’2017.
  * [17] ——, “Formal models of human factors for security and privacy,” in _5th International Conference on Human Aspects of Security, Privacy and Trust, HCII-HAS 2017_ , ser. LNCS, vol. 10292. Springer, 2017, pp. 339–352, affiliated with HCII 2017.
  * [18] ——, “Formal modeling and analysis of data protection for gdpr compliance of iot healthcare systems,” in _IEEE Systems, Man and Cybernetics, SMC2018_. IEEE, 2018.
  * [19] ——, “Qkd in isabelle – bayesian calculation,” _arXiv_ , vol. cs.CR, 2019\. [Online]. Available: https://arxiv.org/abs/1905.00325
  * [20] ——, “Attack trees in isabelle extended with probabilities for quantum cryptography,” _Computer & Security_, vol. 87, 2019. [Online]. Available: //doi.org/10.1016/j.cose.2019.101572
  * [21] ——, “Formalizing probabilistic quantum security protocols in the isabelle infrastructure framework,” informal Presentation at Computability in Europe, CiE 2019. [Online]. Available: https://www.aemea.org/CIE2019/CIE_2019_Abstracts.pdf#page=35
  * [22] F. Kammüller, “A formal development cycle for security engineering in isabelle,” 2020, arxiv preprint, http://arxiv.org/abs/2001.08983.
  * [23] F. Kammüller and C. M. Alvarado, “Exploring rationality of self awareness in social networking for logical modeling of unintentional insiders,” 2021, arxiv preprint, http://arxiv.org/abs/2111.15425.
  * [24] F. Kammüller and B. Lutz, “Modeling and analyzing the corona-virus warning app with the isabelle infrastructure framework,” in _20th International Workshop of Data Privacy Management, DPM’20_ , ser. LNCS, vol. 12484. Springer, 2020, co-located with ESORICS’20.
  * [25] D. Windridge and F. Kammüller, “Edit distance kernelization of np theorem proving for polynomial-time machine learning of proof heuristics,” in _Future of Information and Communications Conference, FICC 2019_ , ser. Advances in Information and Communication. Lecture Notes in Networks and Systems, vol. 70. Springer, 2019.
  * [26] L. Viganó and D. Magazzeni, “Explainable security,” in _IEEE European Symposium on Security and Privacy Workshops, EuroS &PW_. IEEE, 2020.
  * [27] G. Bender, L. Kot, and J. Gehrke, “Explainable security for relational databases,” in _International Conference on Management of Data, SIGMOD 2014, Snowbird, UT, USA, June 22-27, 2014_ , C. E. Dyreson, F. Li, and M. T. Özsu, Eds. ACM, 2014, pp. 1411–1422. [Online]. Available: https://doi.org/10.1145/2588555.2593663
  * [28] W. Pieters, “Explanation and trust: What to tell the user in security and ai?” _Ethics and Information Technology_ , vol. 13, no. 1, pp. 53–64, 2011\.