# Transparency in App Analytics:
Analyzing the Collection of User Interaction Data ††thanks: *This paper has
been accepted at the 20th Annual International Conference on Privacy, Security
& Trust (PST2023)

Feiyang Tang  Norwegian Computing Center
Oslo, Norway  Bjarte M. Østvold  Norwegian Computing Center
Oslo, Norway

###### Abstract

The rise of mobile apps has brought greater convenience and many options for
users. However, many apps use analytics services to collect a wide range of
user interaction data, with privacy policies often failing to reveal the types
of interaction data collected or the extent of the data collection practices.
This lack of transparency potentially breaches data protection laws and also
undermines user trust. We conducted an analysis of the top 20 analytic
libraries for Android apps to identify common practices of interaction data
collection and used this information to develop a standardized collection
claim template for summarizing an app’s data collection practices wrt. user
interaction data. We selected the top 100 apps from popular categories on
Google Play and used automatic static analysis to extract collection evidence
from their data collection implementations. Our analysis found that a
significant majority of these apps actively collected interaction data from UI
types such as View (89%), Button (76%), and Textfield (63%), highlighting the
pervasiveness of user interaction data collection. By comparing the collection
evidence to the claims derived from privacy policy analysis, we manually fact-
checked the completeness and accuracy of these claims for the top 10 apps. We
found that, except for one app, they all failed to declare all types of
interaction data they collect and did not specify some of the collection
techniques used.

###### Index Terms:

Mobile apps, User interaction data collection, Transparency, Trust, Privacy

##  I Introduction

The rapid rise of mobile apps has revolutionized how we interact with
technology, providing developers with a treasure trove of user interaction
data through analytics services such as AppsFlyer
111https://www.appsflyer.com/, Flurry 222https://www.flurry.com/, and Firebase
Analytics 333https://firebase.google.com/docs/analytics. This data,
encompassing user actions like button taps, page scrolls, and video views, is
invaluable for enhancing app functionality and user experience. However, the
vague terminology often used in privacy policies, such as “user’s interaction
with the service”, raises concerns about transparency. The lack of specificity
leaves users uncertain about the extent and nature of the data being collected
and its usage, potentially leading to mistrust and diminished app usage.

Transparency in data collection is a crucial factor influencing user trust
[1]. It empowers users to make informed decisions about the data they share
and its intended usage [2].

An example of this ambiguity can be found in the Yr app, Norway’s most popular
weather app developed by the Norwegian Broadcasting Corporation (NRK). Despite
collecting user interaction data to understand commonly used features, the
app’s privacy policy444https://hjelp.yr.no/hc/en-
us/articles/360003337614-Privacy-policy is vague regarding the collection of
such data, as quoted below in the blue box. Our examination of NRK’s privacy
policy revealed no explicit information about Yr’s data collection practices,
leading to concerns about user trust in both the app and NRK.

Analyze tools
“We use different tools to track the use on our app and website. This
information gives us valuable information such as most popular pages and on
what times Yr is being used the most. No information that can identify persons
are available for Yr.”

Our examination of NRK’s privacy policy555https://info-nrk-
no.translate.goog/personvernerklaering/?˙x˙tr˙sl=no&˙x˙tr˙tl=en&˙x˙tr˙hl=en-
US&˙x˙tr˙pto=wapp revealed no specific information regarding Yr’s data
collection practices. The policy mainly focuses on NRK’s news services and
their “interaction with the services” collection practices. This obscurity
concerning Yr app’s data collection practices raises concerns, as it might
undermine user trust in both the app and NRK as a whole.

App Privacy PolicyRelevant Text FragmentsStandardized Collection
ClaimDecompiled Mobile AppRelevant Bytecode and UI FilesCollection Evidence
Extract related text Standardize the text fragments Identify analytics library
invocation Link invocation with UI widget ArtifactRelevant
informationStandardized form Figure 1: Overview of the approach for analyzing
collection claims and evidence in apps.

Recent research has shown that even seemingly harmless user interaction data
can reveal sensitive information about individuals. For instance, data like
emoji usage or pages visited can be used to infer a user’s pool preference or
political orientation [3]. Moreover, mobile biometric data related to
keystrokes and touchscreen gestures can help estimate attributes like age,
gender, and operating hand [4, 5]. This underscores the potential risks
associated with user interaction data collection, which, while not typically
considered personal data, can be utilized to deduce sensitive information
about individual users, leading to user profiling. The lack of transparency in
these data collection practices could potentially erode user trust in the app.

Most current research on the privacy implications of analytic services has
concentrated on determining whether personally identifiable information (PII)
is being collected and transmitted to external analytics services [6, 7].
Studies have also scrutinized log data to understand user behavior [8], and
high-level analyses of user behavior data collection in mobile apps have been
performed [9]. It is essential to clarify that the interaction data discussed
in this study is not part of traditionally defined PII or personal data,
emphasizing the need for better transparency in data collection practices.

###  I-A Objective

The aim of this paper is to address the issue of lack of transparency in the
collection of user interaction data in mobile apps. To achieve this, we
propose a standardized collection claim template that can be compared to
collection evidence determined through static analysis.

###  I-B Research Questions

To guide our investigation, we formulated the following research questions:

  1. RQ1

What are the common practices of user interaction data collection in mobile
apps?

  2. RQ2

How are these practices reflected in apps’ privacy policies?

  3. RQ3

What types of user interaction data do apps actually collect in their
implementations, and how is this data collected?

  4. RQ4

To what extent do the collection claims in privacy policies align with the
actual data collection practices as observed in their implementations?

###  I-C Contributions

In this paper, we make several contributions towards understanding and
promoting transparency in user interaction data collection practices:

  * •

We propose a standardized collection claim template for user interaction data.
Collection claims, in this context, refer to the descriptions of common
practices of user interaction data collection in mobile apps, as stated in
privacy policies. The template reflects common phrasing and vocabulary derived
from Android documentation and popular Android analytic libraries (Section
III).

  * •

We present an automatic static analysis method to identify collection evidence
from Android apps, which involves analyzing data types, relevant code, and
techniques of collection in layout files and bytecode (Section IV).

  * •

We provide an overview of user interaction data collection practices in the
top 100 popular apps on Google Play across the top 10 categories (Section V).
Our analysis reveals common patterns and offers useful statistics for both app
developers and users to better understand the current state of data collection
practices in mobile apps.

  * •

We conduct fact-checking by manually comparing privacy policy collection
claims against the actual collection evidence found in the ten most popular
apps from the top ten categories (Section V). Our findings reveal that none of
these apps were completely accurate and complete in their collection claims,
highlighting the importance of our proposed approach in promoting transparency
and trust in user interaction data collection practices.

We believe that our proposed method addresses the problem of lack of
transparency by providing a standardized collection claim template for
describing user interaction data collection practices. The template allows app
developers to offer clearer and more accurate information about their data
collection practices, which in turn can help users make informed decisions
about app usage and data sharing. Our method also enables researchers and app
developers to assess the alignment between stated data collection practices in
privacy policies and the actual practices found in app implementations,
facilitating better transparency and ultimately enhancing user trust.

Fig. 1 provides a high-level overview of our method, illustrating the process
of acquiring artifacts (privacy policies and decompiled mobile apps), deriving
knowledge through text analysis and static analysis (relevant text fragments
and bytecode/UI files), and standardizing the information into collection
claims and evidence. This systematic and transparent approach can contribute
to promoting trust and fostering greater transparency in user interaction data
collection practices in mobile apps.

##  II Motivation

The transparency of mobile app data collection practices is a critical issue
that stems from several significant factors, all of which play an essential
role in the complex relationship between user trust and app adoption.
Interaction data, a key asset for understanding user behavior, can raise
serious concerns if it is collected non-transparently.

Transparency serves a dual purpose in this scenario. Firstly, it acts as an
ethical commitment, assuring users that they are informed about their
interaction data’s collection and use. This principle not only respects user
autonomy but also fosters an environment of openness and accountability.
Secondly, transparency plays a pivotal role in building user trust, a
significant factor influencing user satisfaction and continued app usage. When
users understand and control their interaction data, their trust in the app
increases, leading to more consistent engagement. Conversely, a lack of
transparency can breed mistrust and privacy concerns, potentially causing user
dissatisfaction or even app abandonment.

The correlation between transparency and user trust is well-documented in the
academic world. Studies have consistently highlighted the positive
relationship between increased transparency and elevated levels of user trust
[10]. Conversely, an absence of transparency can obstruct the success and
widespread adoption of mobile apps [11]. Thus, transparency is not merely
about informing users, but is essential for facilitating informed decisions
and granting consent.

Another critical aspect to consider is the rise of analytics services like
Firebase Analytics and Flurry Analytics. These services provide developers
with tools to gather data on user behavior, engagement, and preferences.
However, they have raised data protection concerns as they often automatically
collect user data, thereby creating privacy issues. Several countries,
including France, Italy, Austria, Denmark, and Norway, have explicitly stated
that the use of Google Analytics violates GDPR [12]. Android apps can utilize
these services either by directly invoking third-party APIs or by customizing
their analytics service by extending these APIs. The first approach involves
calling third-party API methods directly in activities to log user engagement
events. The second approach allows developers to tailor data collection to
their specific needs.

The importance of transparency is also acknowledged by mobile app developers,
who have a vested interest in prioritizing it alongside user control in their
data collection practices. Research supports this notion; for instance,
Almuhimedi et al. [13], for instance, discovered that many smartphone users
are not fully aware of the data collected by their apps. Providing users with
an app permission manager and sending notifications to increase their
awareness of data collection can enable them to better manage their privacy.
Moreover, users’ concerns about data collection can negatively impact their
perception of the app, potentially leading to its uninstallation [14]. Hence,
promoting transparency and user control can foster trust, resulting in
improved user experiences, increased app engagement, and higher adoption
rates.

In the following sections, we will explore the mutual benefits of transparency
for both users and app developers and illustrate how our proposed method can
address the current shortcomings in the transparency of interaction data
collection practices.

##  III Standardizing Data Collection Claims

To address RQ1, we analyze the common practices of user interaction data
collection in mobile apps, specifically the types of data collected and the
techniques of collection. To achieve this, we conducted an analysis of the top
20 analytic libraries for Android apps. To answer RQ2, we examine how these
practices are reflected in the privacy policies of mobile apps. We refer to
the descriptions of data collection practices in privacy policies as
_collection claims_ , which we define as a single sentence in a standardized
template using a restricted vocabulary to convey the essence of interaction
data collection.

###  III-A Collection Vocabulary

Our restricted collection vocabulary was developed by analyzing the Android
system implementation documentation, as well as the APIs of the top 20
analytic services for Android apps listed on AppBrain [15].

####  III-A1 Terms for Types of User Interaction Data

The user interface of an Android app collects a variety of data types, such as
touch events, sensor data, and text input. Based on a manual inspection of
every single type of Android UI widget, we identified the following six types
of interaction data and named them:

  * •

_App presentation data_ : This data arise from the consumption of content
provided by the app. For example, the user plays a certain video for a period
of time, spends minutes reading one specific page of the news. These
interactions are often recorded by a logging system to keep track of the
user’s consumption habits.

  * •

_Binary data_ : This data arise from discrete user actions, such as tapping on
a button or icon, or selecting a checkbox.

  * •

_Categorical data_ : This data arise from a selection from a set of predefined
options or categories, such as choosing a value from a dropdown menu,
selecting a radio button, or rating a product.

  * •

_User input data_ : This data arise from user input through an on-screen
keyboard or another input method, such as entering text or numbers into a form
field, or using voice input to perform a search or command.

  * •

_Gesture data_ : This data arise from gesture inputs and smooth and continuous
movements of the user’s finger on the screen, such as scrolling through a
list, swiping left or right, pinching or zooming, or shaking the device.

  * •

_Composite gestures data_ : This data arise from a combination of multiple
gestures, such as tapping and holding, double tag, or drag and drop.

####  III-A2 Terms for Collection Techniques

We use the following terms to describe the techniques of user interaction data
collection.

  * •

_Frequency_ : This technique involves logging the frequency of the occurrence
of a particular interaction. For example, an app might log the number of times
a user taps on a specific button or selects a certain option from a drop-down
menu.

  * •

_Duration_ : This technique involves tracking the time a user spends engaging
in a particular interaction. For example, an app might log the amount of time
a user spends watching a particular video or reading a specific article.

  * •

_Motion details_ : This technique involves monitoring the specific details of
a user’s interaction, such as the speed, direction, or angle of their finger
movements on the screen. This type of data can be collected for interactions
such as scrolling, swiping, or dragging.

Table I: The top five most frequent terms used to describe user interaction data in the APP-350 corpus Term | Count
---|---
interact($\sim$ion,$\sim$ing) with service/app | 1,049
analytic(s) | 886
us($\sim$age, $\sim$ing) of service/app | 397
statistic(s) | 315
input(s) of user | 173
Table II: The top five most frequent verbs used to describe such collection in the APP-350 corpus Verb | Count
---|---
collect | 1,386
track | 548
use | 202
log | 86
gather | 46

###  III-B From Policies to Standardized Collection Claims

In this section, we study privacy policies of publicly accessible mobile apps,
aiming to identify and standardize collection claims related to user
interaction data. Utilizing the APP-350 Corpus, a pre-trained language model,
and manual checks, we extract and validate common terminologies employed in
these policies. This comprehensive process allows us to establish a
standardized vocabulary for user interaction data collection claims, providing
a solid foundation for subsequent analysis.

####  III-B1 Identifying Relevant Policy Parts

To distinguish sentences related to user interaction data collection in
privacy policies, we adopt a simple pre-trained language model. This model
sifts through HTML files of privacy policies and singles out sentences
containing specific keywords and their synonyms. Our focus is to ensure that
our privacy policy claim vocabulary aligns with the most common terminology
used in the industry to describe user interaction data.

After processing the privacy policies, we conduct manual checks to eliminate
any false positives. The end result is a selection of common phrases used in
these policies to describe the collection of user interaction data. From the
sentences identified, we isolate the most relevant verbs and nouns to form a
list of keywords.

##### Experimental Details and Validation

In conducting our analysis, we use the APP-350 Corpus [16], a set of 350
mobile app privacy policies that are annotated with privacy practices.
Although the main focus of the APP-350 Corpus is on identifying sentences
related to personally identifiable information (PII), we utilize the raw HTML
files of the privacy policies for our examination.

The natural language processing is carried out using the spaCy [17] library
with the `en_core_web_sm` model. This model, pre-trained on web text, which
includes web forums, web pages, and Wikipedia, is capable of identifying named
entities, parts of speech, dependency parsing, and more. We also employ the
WordNet module from the Natural Language Toolkit (NLTK [18]) to discover
synonyms for the extracted keywords.

To authenticate the effectiveness of the model, we manually annotate 50
randomly selected privacy policies from the APP-350 dataset. This helps us
identify sentences containing relevant information, the verbs used to describe
data collection (e.g., “collect”, “track”), and the terms used to describe
user interaction data (e.g., “usage of the app”, “interaction with the
service”).

The model successfully recognized sentences related to user interaction data
collection in 37 out of the 38 files that contained such sentences, using
keywords such as interaction, usage, statistics, experience, and analytics.
Identifying the verbs used to describe data collection was a more complex
task, with a recall of 92% but a precision of only 84% 666Recall is calculated
as TP / (TP + FN), while precision is calculated as TP / (TP + FP), where TP
is true positives, FN is false negatives, FP is false positives, and TN is
true negatives. due to the presence of similar verbs in sentences that were
not related to the context.

Upon running the model on the 350 privacy policies, we identified 1,411
sentences. The relevant verbs and nouns from these sentences are shown in
Table II and Table II and then compiled to form the list of keywords.

####  III-B2 Template for Standardized Collection Claims

In privacy policies, it is common for apps to use convoluted language to
describe how user data is collected. To make these collection claims in
privacy policy easier to read and compare across different apps, we created a
standardized template that utilized the most frequently used verb, “collect”,
and the most frequently used noun phrase, “user interaction data”. The
resulting structure is as follows:

Template for Standardized Collection Claims
We collect the following types of user interaction data: $\langle$types of
data collected$\rangle$, along with their $\langle$techniques of
collection$\rangle$.777Refer to the claim vocabularies in Section III-A

This standardized collection claim template can be combined with the
collection evidence gathered through static analysis to check and the accuracy
of privacy policy collection claims made by various apps. Also, the
standardized language facilitates transparency and comparison between
policies. We return to the subject of fact-checking collection claims in
Section V.

##  IV Data Collection Evidence

In this section, we analyze mobile apps to understand the types of user
interaction data collected and the techniques employed (RQ3). We conduct
static analysis of the Android application package (APK) to identify data
collection methods (DCMs) and extract collection evidence, which highlights
the gap between privacy policy claims and actual practices (RQ4).

Our analysis is divided into two parts. First, we identify DCMs from the top
20 Android analytic services and customized analytics services. Second, we
extract collection evidence by focusing on invocations to analytics services,
associated UI widgets, and the callbacks triggered by registered listeners.

###  IV-A Identifying Data Collection Methods

Data collection methods (DCMs) are methods defined by analytics services, such
as Firebase Analytics, that allow app developers to log user interaction data.
DCMs provide a standardized way for app developers to collect user interaction
data and track app usage in order to analyze and understand user behavior.

For example, the Firebase Analytics API provides the logEvent() method to log
user events, such as button clicks or screen views. Suppose we have a button
myButton in the app’s UI, and we want to track when the user clicks on it. We
can do this using Firebase Analytics by adding the following code to the
button’s OnClickListener:

⬇

myButton.setOnClickListener(new View.OnClickListener() {

public void onClick(View v) {

FirebaseAnalytics.getInstance(this).

logEvent("button_click", null); }

});

Here FirebaseAnalytics.getInstance(this) returns an instance of the Firebase
Analytics object, and logEvent("button_click", null) collects the button click
interaction data with the string "button_click" to Firebase Analytics.

To determine how Android apps use analytics services, we identified DCMs from
the top 20 Android analytic services, cf. Section III-A. Matching the full
signature of these methods in bytecode allows us to find direct invocations to
analytics services. However, some apps use customized analytics services to do
a more fine-grained collection, such as collecting motion details and
duration. To do this, the apps implement their own analytics classes by
extending the analytic services.

To identify customized analytics, we use static analysis to identify the
classes that invoke external DCMs. We then check whether these classes are
invoked in any of the app’s declared activities. If they are, we mark these
classes as customized analytics services classes.

###  IV-B Extracting Collection Evidence

Next, we extracted evidence of actual data collection from the APK.
Specifically, we analyze three types of information: (1) invocations to
analytics services that logged user interaction data collection, (2)
associated UI widgets, and (3) the callbacks triggered by registered listeners
on these UI widgets.

We utilized static analysis with FlowDroid [19] to associate DCM invocations
with callbacks, listeners, and activities in the bytecode. We then compared
the layout IDs of the associated UI widgets defined in the layout XML files to
identify the relevant collection data types and techniques.

The relationships between different parts of the extracted collection evidence
in an Android app are shown in Fig. 2. The UI-related parts, such as layout
files and defined UI widgets, provide information on the types of user
interaction data (red section), while the bytecode provides details on the
techniques of collection (blue section)888Note: The figure notation is as
follows: 1-M means one-to-many, 1-$\ast$ means one-to-any (zero or more), and
1-1 means one-to-one..

AppLayout filesActivityUI widgetListenerCallbackDCM
InvocationM1M1M1M1M1M1*11*M111Techniques of collectionUser interaction data
types Figure 2: Relationships between different parts of the extracted
collection evidence in an Android app

We return to the Yr weather app, the example app from Section I. Based on the
collection evidence extracted from Yr’s bytecode and layout files, we
discovered that it collects detailed user interaction data using various types
of UI widgets such as SearchView and Textfield. This data collection is linked
to features such as changes in location, enabling forecast summary
notifications, and opening the forecast graph. Building on this finding from
static analysis, we propose the following more specific checked standardized
collection claim:

Checked Standardized Collection Claim for Yr
We collect the following types of user interaction data: app presentation,
binary and categorical interactions, and user input interactions, along with
their frequency.

##  V Findings

To address RQ2, we conduct a manual inspection of 1411 sentences that
described user interaction data collection in all 350 privacy policies within
the APP-350 corpus, as outlined in Section III-B.

We examine whether the sentences in a privacy policy provide clear
descriptions of the types of user interaction data collected and the
techniques of collection.

We find that only 37% of the identified sentences contained clear statements
on both the data types and techniques of collection, while 41% only discussed
the techniques of collection and 22% mentioned only the data types.

Here are the relevant sentences from two policies in the corpus.
DAMI999https://play.google.com/store/apps/details?id=com.blappsta.damisch
states: “We may work with analytics companies to help us understand how the
Applications are being used, such as the frequency and duration of usage.”
Wish101010https://play.google.com/store/apps/details?id=com.contextlogic.wish
states: “We may collect different types of personal and other information
based on how you interact with our products and services. Some examples
include: Equipment, Performance, Websites Usage, Viewing and other Technical
Information about your use of our network, services, products or websites.”

DAMI’s privacy policy only discloses the techniques of collection, such as the
frequency and duration of usage, without clearly explaining which type of user
interaction data is collected. In contrast, Wish’s privacy policy does mention
some specific types of data collected, such as equipment and performance data,
but it is unclear about which techniques of collection are used.

The majority of identified sentences discuss the techniques of collection
rather than specific data types, suggesting that organizations use the tactic
of avoiding or minimizing disclosures about the types of user interaction data
they collect in order to collect more data than users are aware of or
comfortable with.

To investigate RQ3, we performed a static analysis on a sample of 100 free
Android apps downloaded from the top 10 most popular categories on the German
Google Play store111111https://play.google.com/store/apps?gl=DE, as identified
by AppBrain. In cases where the same app appeared in several categories, we
moved to the next popular app in the second category to get a total of 100
distinct apps.

Table III: Statistics of the user interaction data collection for the top 100
Android apps.

UI type (types of interaction data) | Top 2 techniques of collection | Top 3 app categories | Percent collected | Avg # collected
---|---|---|---|---
View (Presentation) | Frequency (100%), Duration (52%) | Entertainment, Shopping, Travel | 89% | 12
Button (Binary) | Frequency (94%), Motion (8%) | Social, Utility, Gaming | 76% | 26
Textfield (Input) | Frequency (100%), Duration (4%) | Social, Shopping, Utility | 63% | 5
Checkbox & Spinner (Categorical) | Frequency (97%), Motion (16%) | Shopping, Travel, Utility | 32% | 7
GestureDetector (Gesture) | Motion (94%), Duration (40%) | Gaming, Entertainment, Social | 16% | 38

Our analysis of the top 100 Android apps revealed that app developers placed a
great deal of emphasis on understanding how frequently users interacted with
different UI elements (which may correspond to different features or
functionalities in the app), as frequency was the top techniques of collecting
user interaction data across all UI types. We also found that the average
number of interaction data collected varied significantly across different
types of UI. It was also interesting to see that the high number of
interaction data collected for the button UI type (also found in 76% of the
apps), indicated that understanding button usage was a particularly important
metric for app developers.

Table III presents an overview of the user interaction data collection
practices across various app categories, focusing on the top UI type for each
type of interaction data. We have selected the most frequently occurring UI
types from each category for this analysis, which are listed in the first
column.

The second column indicates the top two techniques linked with each UI type.
The percentages in parentheses, for instance, 100% and 52% for the “View” UI
type, represent the proportion of apps that use a particular technique in
tracking the UI type. For example, 100% of apps tracked “View” interactions
use the frequency technique, while 52% also use the duration technique.

The “Percent collected” column indicates the proportion of the top 100 apps
that collect data related to a specific UI type. For instance, “View” data is
collected by 89% of the analyzed apps.

Finally, the “Average # collected” column represents the average number of
distinct DCMs detected in each app associated with a particular UI type. For
example, on average, 12 distinct DCMs were detected for “View” data collection
across the analyzed apps.

Upon comparing these user interaction data collection practices with the
declarations in privacy policies, we observe a larger mismatch in certain app
categories. Gaming apps, despite their high prevalence of Gesture data
collection to optimize user experience, often lack comprehensive disclosure of
such practices in their privacy policies. Similarly, Entertainment, Shopping,
and Travel apps extensively collect “View” data, but their policies rarely
match the extent of this data collection, indicating a transparency gap in
these visually-centric applications.

Social and Utility apps, which heavily rely on “Button” and “TextField” data,
also demonstrate a significant disparity between their actual data collection
practices and policy disclosures. These observations highlight that while app
developers tailor their data collection strategies to their specific
objectives and requirements, they often fail to mirror this granularity in
their privacy policies.

This mismatch is consequential as it affects the transparency of these apps
and the users’ ability to make informed decisions. Hence, addressing these
discrepancies becomes crucial, and our findings provide valuable insights for
developers aiming to improve their privacy disclosures, ultimately fostering
trust and success in the app ecosystem.

To address RQ4, we manually inspected the privacy policy claims of the most
popular app in each of the 10 categories on Google Play. We generated our
checked collection claims by analyzing the actual data collection practices of
each app and comparing them to the privacy policy claims published by the app.
Our checked collection claims are made by combining the evidence gathered
through static analysis and the proposed standardized claim template. The
results are fact-check collection claims presented in Table IV.

Our findings uncovered inconsistencies between the claims made in privacy
policies and the actual data types and techniques of collection used by
popular apps on Google Play. Many apps do not fully disclose the types of data
collected or the techniques of collection, often using vague language such as
“collecting user interactions to improve the service”.

Notably, some apps that may be perceived as having questionable data
collection practices, such as TikTok and Amazon Prime Video, actually provided
more detailed information on the types of data collected and the techniques of
collection used. TikTok and Duolingo even provided specific examples of their
data collection practices.

Table IV: Fact-checked data collection claims w.r.t. evidence for the most
popular app from each of the top 10 categories of Google Play.
The red text indicates types of user interaction data missing from the privacy
policy/collection claims, while the blue text indicates undisclosed techniques
of collection.

Checked Collection Claim |  Related Text in the Published Privacy Policy
---|---
[TikTok] We collect the following types of user interaction data: app presentation, binary,  categorical, user input, gesture and composite gesture interactions, along with their frequency, duration and motion details. |  [TikTok] We collect information about how you engage with the Platform, including information about the content you view, the duration and frequency of your use, your engagement with other users, your search history and your settings.
[SHEIN] We collect the following types of user interaction data: app presentation, binary, categorical,  user input interactions, along with their  frequency and duration. |  [SHEIN] Data about how you engage with our Services, such as browsing, adding to your shopping cart, saving items, placing an order, and returns for market research, statistical analysis, and the display of personalized advertising based on your activity on our site and inferred interests; Collect your device information, and usage data on our website or app for fault analysis, troubleshooting, and system maintenance, as well as setting default options for you, such as language and currency. The display of information you choose to post on public areas of the Services, for example, a customer review.
[Booking.com] We collect the following types of user interaction data: app presentation, binary, categorical, user input interactions, along with their frequency and duration. |  [Booking.com] We collect data that identifies the device, as well as data about your device-specific settings and characteristics, app crashes and other system activity.
[PayPal] We collect the following types of user interaction data: app presentation, binary, categorical, user input interactions along with their frequency. |  [PayPal] When you visit our Sites, use our Services, or visit a third-party website for which we provide online Services, we and our business partners and vendors may use cookies and other tracking technologies to recognize you as a User and to customize your online experiences, the Services you use, and other online content and advertising; measure the effectiveness of promotions and perform analytics; and to mitigate risk, prevent potential fraud, and promote trust and safety across our Sites and Services.
[Duolingo] We collect the following types of user interaction data: app presentation, binary, categorical, user input, gesture interactions, along with their frequency and duration. |  [Duolingo] We do record the following data: Patterns, Clicks, Mouse movements, Scrolling, Typing, Pages visited, Referrers, URL parameters, Session duration.
[Amazon Prime Videos] We collect the following types of user interaction data: app presentation, binary, categorical, user input, gesture interactions, along with their frequency, duration and motion details. |  [Amazon Prime Videos] We automatically collect and store certain types of information about your use of Amazon Services including your interaction with content and services available through Amazon Services. List of examples: search for products or services in our stores and download, stream, view, or use content on a device, or through a service or application on a device.
[Yazio] We collect the following types of user interaction data: binary and user input interactions, along with their frequency. |  [Yazio] The Firebase Analytics service helps to determine the interactions of App users by recording, for instance, the first time the App is opened, deinstallations, updates, system crashes and how often the App is used. The service also records and analyses certain user interests.
[Fasion Famous] We collect the following types of user interaction data: app presentation, binary, user input, gesture and composite gesture interactions, along with their frequency, duration and motion details. |  [Fasion Famous] Information that may be collected automatically: Data and analytics about your use of our Services. Data we collect with cookies and similar technologies: Data about your use of our Services, such as game interaction and usage metrics.
[Picsart] We collect the following types of user interaction data: app presentation, binary, gesture and composite gesture interactions, along with their frequency, duration and motion details. |  [Picsart] Our servers passively keep an electronic record of your interactions with our services, which we call “log data”. We collect and combine data about the devices you use to access Picsart, and data about your device usage and activity.
[Dezor] We collect the following types of user interaction data: app presentation, binary, categorical, user input interactions, along with their frequency. |  [Dezor] The information collected by log files include internet protocol (IP) addresses, browser type, Internet Service Provider (ISP), date and time stamp, referring/exit pages, and possibly the number of clicks.

However, we found that some apps from less controversial categories, such as
the photography editing app Picsart and the payment platform PayPal, used
opaque language in their privacy policies, leaving a large gap between their
claims and our findings. The most extreme example was Booking.com, which
extensively collects user interactions within the app, yet discloses almost no
information in its privacy policy. These findings highlight the need for
clearer and more comprehensive disclosures in privacy policies, particularly
for apps that collect sensitive user data.

###  V-A Threats to Validity

Potential threats to the validity of our experiment may impact the
interpretation of our findings. A primary limitation of our experiment is the
number of apps we manually fact-checked for data collection practices. Due to
the complexity of accommodating varying UI types and callbacks into our
predefined six data types and three techniques of collection, we were only
able to manually fact-check one app in each category, totaling ten apps. This
sample size, though limited, may not encapsulate the full diversity of data
collection practices across all apps.

Furthermore, measuring the recall of our analysis posed a considerable
challenge, given the absence of a comprehensive ground truth detailing all
interaction data collection practices in each app. Consequently, our findings
may not wholly represent the full range of data collection practices.

##  VI Related Work

The related work can be categorized into three primary themes: (1) privacy
policy analysis using NLP and policy compliance check, (2) static analysis for
security and privacy in apps, and (3) analytics services analysis.

###  VI-A Privacy Policy Analysis

Numerous studies have focused on analyzing and improving privacy policies in
mobile apps. Researchers have explored various NLP approaches to automatically
process and understand privacy policy texts, as well as to assist users in
comprehending these policies more effectively [20, 21, 22]. However, these
studies do not specifically address the issue of user interaction data
collection, which is a significant gap that our research addresses. Tools like
PrivacyFlash Pro [23] and AutoCog [24] have been developed to audit privacy
policy compliance by comparing disclosed policies with actual app behavior,
but they primarily focus on personal data, not user interaction data. A recent
study by Bardus et al. [25] systematically mapped existing contact-tracing
apps and evaluated the permissions required and their privacy policies, but it
did not delve into the specifics of user interaction data collection.

###  VI-B Static Analysis for Security and Privacy

The static analysis approach has been used to enhance security and privacy in
mobile apps. This involves analyzing app bytecode, identifying data leaks, and
detecting privacy violations [26, 27, 28]. Despite the progress in this field,
there remains an underrepresentation of studies targeting user interaction
data, a type of data often overlooked in privacy policies and their
corresponding analyses. A novel system, LocationScope, was presented by Lu et
al. [29] to detect and measure aggressive location harvesting in mobile apps
at scale, but it did not specifically target user interaction data.

###  VI-C Analytics Services Analysis

Another line of research has concentrated on the role of analytics services in
capturing user data, primarily focusing on PII. Alde [7], for example,
proposed a method employing both static and dynamic analysis to detect the key
information gathered by analytics libraries, which are largely device-level
data. PAMDroid [6] takes a similar approach, identifying personal data
funneled into analytics services and treating it as a misconfiguration. The
domain of user interaction data collection, however, remains relatively
untouched in these studies. A recent study by Laperdrix et al. [30] presented
a privacy analysis of free and paid games in the Android ecosystem, but it did
not specifically focus on user interaction data collection.

These studies have contributed to the understanding of privacy policies and
data collection practices in mobile apps. However, there is a lack of research
specifically on the practices of user interaction data collection and the
transparency of related claims in privacy policies. Our work extends the scope
of previous research by focusing on user interaction data collection practices
and providing an analysis on comparing privacy policy disclosures with actual
app behavior. This approach aims to enhance transparency and trust in the
mobile app ecosystem, addressing the research gaps in the existing literature.

##  VII Conclusion and Future Work

In conclusion, our analysis of the top 100 apps uncovers the widespread
collection of user interaction data, while the detailed examination of the top
10 apps reveals that privacy policies often inadequately disclose such
practices. To address this lack of transparency, we introduced a standardized
collection claim template that aids app developers in accurately detailing
their data collection practices. This approach fosters informed decisions by
users and enhances transparency by allowing assessments of alignment between
declared and actual data collection practices for the manually analyzed apps.
Our findings lay the groundwork for improving data collection transparency in
mobile apps and highlight the need for automating the policy-to-claims
analysis. This insight could potentially guide future research and policy-
making to foster a more secure and trustworthy app ecosystem.

Our approach has limitations that can be addressed in future research to
improve the analysis of data collection practices. The current analysis only
covers the top 20 analytics services and is confined to Android apps.
Furthermore, the manual fact-checking of the top 10 apps relies on our
interpretation of their policies. To overcome these limitations, machine
learning models could be employed to automatically identify and categorize
data collection methods (DCMs) within app code, reducing the need for manual
analysis. This would involve training models to detect DCMs and categorizing
them based on the data types and collection techniques they employ.
Additionally, a more precise and fine-grained policy analysis could be
developed to automatically extract interaction data types and collection
techniques from privacy policies. By combining these advancements, we could
create a fully automated approach to fact-check collection claims against the
collection evidence, thereby increasing the efficiency and accuracy of
analyzing data collection practices in mobile applications.

Another potential area for future work is the exploration of user studies to
understand users’ perceptions of interaction data collection practices and
their impact on users’ trust and app usage. Extending the analysis to include
other platforms and analytics services could also contribute to a more
holistic understanding of user interaction data collection practices across
the mobile app ecosystem.

## Acknowledgement

We appreciate the valuable insights provided by Professor Staal Vinterbo. This
work is part of the Privacy Matters (PriMa) project. The PriMa project has
received funding from European Union’s Horizon 2020 research and innovation
program under the Marie Skłodowska-Curie grant agreement No. 860315.

## References

  * [1] L. M. Cysneiros and V. Werneck, “An initial analysis on how software transparency and trust influence each other,” in _Workshop em Engenharia de Requisitos_ , 2009.
  * [2] T. Morey, T. Forbath, and A. Schoop, “Customer data: Designing for transparency and trust,” _Harvard Business Review_ , vol. 93, no. 5, pp. 96–105, 2015.
  * [3] A. Gadotti, F. Houssiau, M. S. M. S. Annamalai, and Y.-A. de Montjoye, “Pool inference attacks on local differential privacy: Quantifying the privacy guarantees of apple’s count mean sketch in practice,” in _USENIX Security 22_ , 2022, pp. 501–518.
  * [4] A. Buriro, Z. Akhtar, B. Crispo, and F. Del Frari, “Age, gender and operating-hand estimation on smart mobile devices,” in _2016 International Conference of the Biometrics Special Interest Group (BIOSIG)_ , 2016, pp. 1–5.
  * [5] A. Jain and V. Kanhangad, “Gender recognition in smartphones using touchscreen gestures,” _Pattern Recognition Letters_ , vol. 125, pp. 604–611, 2019.
  * [6] X. Zhang, X. Wang, R. Slavin, T. Breaux, and J. Niu, “How does misconfiguration of analytic services compromise mobile privacy?” in _2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE)_ , 2020, pp. 1572–1583.
  * [7] X. Liu, J. Liu, S. Zhu, W. Wang, and X. Zhang, “Privacy risk analysis and mitigation of analytics libraries in the android ecosystem,” _IEEE Transactions on Mobile Computing_ , vol. 19, no. 5, pp. 1184–1199, 2020.
  * [8] S. Dumais, R. Jeffries, D. M. Russell, D. Tang, and J. Teevan, “Understanding user behavior through log data and analysis,” _Ways of Knowing in HCI_ , pp. 349–372, 2014.
  * [9] H. Verkasalo, “Analysis of smartphone user behavior,” in _2010 Ninth International Conference on Mobile Business and 2010 Ninth Global Mobility Roundtable (ICMB-GMR)_. IEEE, 2010, pp. 258–263.
  * [10] S. Fischer-Hübner, J. Angulo, F. Karegar, and T. Pulls, “Transparency, privacy and trust–technology for tracking and controlling my data disclosures: Does this work?” in _Trust Management X: 10th IFIP WG 11.11 Conference, IFIPTM 2016, Darmstadt, Germany, July 18-22, 2016, Proceedings 10_. Springer, 2016, pp. 3–14.
  * [11] E. Vorm and D. J. Combs, “Integrating Transparency, Trust, and Acceptance: The Intelligent Systems Technology Acceptance Model (ISTAM),” _International Journal of Human–Computer Interaction_ , vol. 38, no. 18-20, pp. 1828–1845, 2022.
  * [12] A. Rzhevkina, “Several EU countries banned Google Analytics - here are some alternatives,” https://www.contentgrip.com/eu-countries-ban-google-analytics/, September 2022, (Accessed on 03/12/2023).
  * [13] H. Almuhimedi, F. Schaub, N. Sadeh, I. Adjerid, A. Acquisti, J. Gluck, L. F. Cranor, and Y. Agarwal, “Your location has been shared 5,398 times! a field study on mobile app privacy nudging,” in _Proceedings of the 33rd annual ACM conference on human factors in computing systems_ , 2015, pp. 787–796.
  * [14] K. Degirmenci, N. Guhr, and M. Breitner, “Mobile applications and access to personal information: A discussion of users’ privacy concerns,” in _Proceedings of the 34th International Conference on Information Systems (ICIS 2013)_. Association for Information Systems (AIS), 2013, pp. 1–21.
  * [15] AppTornado, “AppBrain: Android analytics libraries,” https://www.appbrain.com/stats/libraries/tag/analytics/android-analytics-libraries, (Accessed on 03/04/2023).
  * [16] P. Story, S. Zimmeck, A. Ravichander, D. Smullen, Z. Wang, J. Reidenberg, N. C. Russell, and N. Sadeh, “Natural language processing for mobile app privacy compliance,” in _AAAI Spring Symposium on Privacy-Enhancing Artificial Intelligence and Language Technologies_ , 2019.
  * [17] M. Honnibal and I. Montani, “spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing,” 2017\.
  * [18] S. Bird, E. Klein, and E. Loper, _Natural language processing with Python: analyzing text with the natural language toolkit_. O’Reilly Media, Inc., 2009.
  * [19] S. Arzt, S. Rasthofer, C. Fritz, E. Bodden, A. Bartel, J. Klein, Y. Le Traon, D. Octeau, and P. McDaniel, “Flowdroid: Precise context, flow, field, object-sensitive and lifecycle-aware taint analysis for Android apps,” _ACM SIGPLAN Notices_ , vol. 49, no. 6, pp. 259–269, 2014.
  * [20] W. B. Tesfay, P. Hofmann, T. Nakamura, S. Kiyomoto, and J. Serna, “Privacyguide: Towards an implementation of the eu gdpr on internet privacy policy evaluation,” in _Proceedings of the Fourth ACM International Workshop on Security and Privacy Analytics_ , ser. IWSPA ’18. New York, NY, USA: Association for Computing Machinery, 2018, p. 15–21.
  * [21] R. Ramanath, F. Liu, N. Sadeh, and N. A. Smith, “Unsupervised alignment of privacy policies using hidden markov models,” in _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics_ , 2014, pp. 605–610.
  * [22] A. Ravichander, A. W. Black, T. Norton, S. Wilson, and N. Sadeh, “Breaking down walls of text: How can nlp benefit consumer privacy?” in _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing_ , vol. 1, 2021.
  * [23] S. Zimmeck, R. Goldstein, and D. Baraka, “Privacyflash pro: Automating privacy policy generation for mobile apps.” in _NDSS_ , 2021.
  * [24] Z. Qu, V. Rastogi, X. Zhang, Y. Chen, T. Zhu, and Z. Chen, “Autocog: Measuring the description-to-permission fidelity in android applications,” in _Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security_ , 2014, pp. 1354–1365.
  * [25] M. Bardus, M. Al Daccache, N. Maalouf, R. Al Sarih, and I. H. Elhajj, “Data management and privacy policy of covid-19 contact-tracing apps: Systematic review and content analysis,” _JMIR mHealth and uHealth_ , vol. 10, no. 7, p. e35195, 2022.
  * [26] V. Avdiienko, K. Kuznetsov, A. Gorla, A. Zeller, S. Arzt, S. Rasthofer, and E. Bodden, “Mining apps for abnormal usage of sensitive data,” in _2015 IEEE/ACM 37th IEEE international conference on software engineering_ , vol. 1. IEEE, 2015, pp. 426–436.
  * [27] W. Enck, P. Gilbert, S. Han, V. Tendulkar, B.-G. Chun, L. P. Cox, J. Jung, P. McDaniel, and A. N. Sheth, “Taintdroid: an information-flow tracking system for realtime privacy monitoring on smartphones,” _ACM Transactions on Computer Systems (TOCS)_ , vol. 32, no. 2, pp. 1–29, 2014.
  * [28] X. Zhang, X. Wang, R. Slavin, T. Breaux, and J. Niu, “How does misconfiguration of analytic services compromise mobile privacy?” in _Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering_ , 2020, pp. 1572–1583.
  * [29] H. Lu, Q. Zhao, Y. Chen, X. Liao, and Z. Lin, “Detecting and measuring aggressive location harvesting in mobile apps via data-flow path embedding,” _Proceedings of the ACM on Measurement and Analysis of Computing Systems_ , vol. 7, no. 1, pp. 1–27, 2023.
  * [30] P. Laperdrix, N. Mehanna, A. Durey, and W. Rudametkin, “The price to play: a privacy analysis of free and paid games in the android ecosystem,” in _Proceedings of the ACM Web Conference 2022_ , 2022, pp. 3440–3449.


# What If: Generating Code to Answer Simulation Questions

Gal Peretz
Technion,
Israel Institute of Technology
<EMAIL_ADDRESS>
&Kira Radinsky
Technion,
Israel Institute of Technology
<EMAIL_ADDRESS>

###### Abstract

Many texts, especially in chemistry and biology, describe complex processes.
We focus on texts that describe a chemical reaction process and questions that
ask about the process’s outcome under different environmental conditions. To
answer questions about such processes, one needs to understand the
interactions between the different entities involved in the process and to
simulate their state transitions during the process execution under different
conditions. A state transition is defined as the memory modification the
program does to the variables during the execution. We hypothesize that
generating code and executing it to simulate the process will allow answering
such questions. We, therefore, define a domain-specific language (DSL) to
represent processes. We contribute to the community a unique dataset curated
by chemists and annotated by computer scientists. The dataset is composed of
process texts, simulation questions, and their corresponding computer codes
represented by the DSL. We propose a neural program synthesis approach based
on reinforcement learning with a novel state-transition semantic reward. The
novel reward is based on the run-time semantic similarity between the
predicted code and the reference code. This allows simulating complex process
transitions and thus answering simulation questions. Our approach yields a
significant boost in accuracy for simulation questions: 88% accuracy as
opposed to 83% accuracy of the state-of-the-art neural program synthesis
approaches and 54% accuracy of state-of-the-art end-to-end text-based
approaches.

##  1 Introduction

Many texts, especially in chemistry and biology, describe complex processes.
These texts describe a chemical reaction process and require answering
questions asking about the process’s outcome under different environment
conditions. To answer these questions, one needs to understand the
interactions between the different entities involved in the process and to
simulate their state-transitions during the process execution under other
conditions. Consider the following example:

> A mixture of 50 g (0.42 mole) of p-aminobenzonitrile, 146 ml of concentrated
> hydrochloric acid, and 106 ml of H2O was heated at 80 for 20 minutes and
> then cooled to 0°. … The hexane was cooled and the resulting precipitate was
> filtered and air dried to yield 16 g (18%) of p-(5-acetyl-2-furyl)
> benzonitrile. … Overheat the mixture to 83 degrees will result in 9%
> decrease in the outcome of p-(5-acetyl-2-furyl) benzonitrile."
>
> Question: What will be the output of the process, if after 10 minutes of
> heating the mixture we increase the heat by 0.5 degree per minute?

This example describes chemical reactions between several components that
yield 16 grams of p-(5-acetyl-2-furyl) benzoni-trile. A model that attempts to
answer the question following the text needs to extract the right information
about the temperature and the heating duration described in the text, and to
understand the output of the iterative process described in the question.
Finally, it needs to perform a numerical computation and to check if the
temperature reached the threshold of 83 degrees or not. We refer to these type
of questions as _Simulation Questions_ , as they require to simulate a process
under other environment conditions.

In recent years, deep learning models have achieved state-of-the-art
performance for question-answering (QA) and reading comprehension tasks. Pre-
trained models like Roberta [1] and XLNet [2] are end-to-end models that
achieved high performance on QA datasets like SQuAD [3] and SWAG [4]. Although
these models achieved impressive results on various QA tasks, they mostly
focus on span-based datasets, where the answer to the question must be a span
from the text itself, or questions that require simple commonsense reasoning.
For simulation questions one needs the ability to simulate the process at hand
in addition to performing a numerical computation that would yield the final
process product.

Semantic parsing approaches, whose goal is to convert a natural language text
to a logical form, showed superior results for inference of simple processes-
based texts datasets ([5, 6, 7, 8]) that require simple numeric reasoning ([9,
10]). However, simulation questions require understanding the complex process’
dynamics in additional to numerical and scientific commonsense reasoning
capabilities. Consider our aforementioned example. The question requires to
simulate an iterative process of heating until reaching a certain threshold.
This requires to mimic a loop and if-clause. In this work, we both present a
unique dataset (SimQA) for complex processes and an algorithmic framework to
answer simulation questions. Our approach attempts to predict the question’s
answer by modeling the process described in the text as a computer code
simulating the process’s progress.

We create SimQA by extending the US patents dataset ([11]) – a well-known
dataset in the domain of chemistry. Chemists were assigned to create questions
based on texts that describe chemical experiment, and computer scientists were
assigned to annotate the questions and the texts with a corresponding program
that if executed, answers the question correctly (Figure 1). For the program
annotation, we define a domain-specific language (DSL). It allows to represent
a code that simulates a process described in a text given the different
environment settings described in the question. All of the answers in the
SimQA dataset are numerical values that can be inferred from the text and the
question by understanding the process’s components interactions and the
computation that will lead to the correct answer.

Figure 1: A text of a process and the corresponding code. Figure 2: State and
BLEU score for program synthesis.

###  1.1 Syntactic Neural Program Synthesis

We propose an algorithm we call Neural Program Synthesis for Simulation QA
(NPS-SQA) that given a text and a question, attempts to learn to produce the
corresponding code represented by the DSL. We treat the problem as a
translation problem and learn to translate from text to code. The complexity
of the generated DSL that contains loops, if-clauses, etc., requires a more
nuanced optimization than simple end-to-end translation models. Unlike
previous approaches that focused on optimizing only the syntax of the target
program, our approach leverages the run-time information to estimate semantic
similarity between the predicted and the reference programs. We devise a
reinforcement learning approach and during optimization create programs. Those
are then run and compared to the oracle code run-time variable state. We refer
to the latter as semantic code similarity. We study the trade-off between the
semantic and syntactic similarity rewards, and provide insights to the type of
questions that NPS-SQA outperforms state-of-the-art QA models.

The contribution of our work is fourfold: (1) We present a novel dataset for
simulation questions of complex processes. It contains reference programs that
can be executed to infer the right answers to the questions. As far as we
know, this is the first question-answering dataset that requires deep
understanding of the scientific processes’ dynamics and numerical commonsense
reasoning capabilities. We contribute it to the community for further research
111https://drive.google.com/file/d/1jzfAWjjmgGZtUVVke9TB9zEn4tM_7ThM/view?usp=sharing.
(2) We define a domain-specific language (DSL) that is more concise than full
featured programming languages like Python, but expressive enough with the
ability to imitate the state of the process by defining variables, simulate
iterative process using loops and examine multiple possibilities using if-
conditions. (3) We propose a novel reward function based on the state
transition of the predicted and the reference programs to model the dynamics
of the process. We use policy gradient training to train our model by
rewarding it based on both the syntactic similarity and run-time similarity of
the execution processes. We publish our code for the benefit of the community
222https://github.com/galprz/SimQA. We perform an empirical analysis comparing
NPS-SQA and state-of-the-art deep-learning QA approaches showing significant
boosts of 30%+ in accuracy compared to pretrained end-to-end question-
answering models and 5%+ for neural program synthesis approaches trained on
SimQA.

##  2 Related Work

Recently, pretrained models like ELMo([12]), BERT([13]), XLNET([2]) and
Roberta([1]) have shown state-of-the-art performance on various QA tasks like
SQUAD ([3]), RACE ([14]), GLUE ([15]) etc. However, these models perform
poorly on tasks that require logical or numerical commonsense [16]. On the
other hand semantic parsing approaches transform the text to a logical form
that facilitates inference. These methods showed superior performance on tasks
that require understanding of processes’ dynamics [17, 18, 19, 20, 21, 22, 23]
and numerical commonsense reasoning [10, 24]. Semantic parsing approaches have
two variations, weakly supervised approaches [17, 18, 20, 19, 25, 24] and
fully supervised approaches. Our model is a fully supervised approach.

Fully supervised approaches use sequence-to-sequence architectures in a fully
supervised manner to optimize the maximum marginal likelihood of logical
language that the computer can interpret given as a label (“the oracle”). For
example, if the logical language is a computer program code, then the oracle
is the predicted code or the target abstract syntax tree [21, 22, 23]
corresponding to the text. Recent example of a semantic parsing model that
achieved state-of-the results on well known benchmarks like Drop [9] and
MathQA [10] is Neural Symbolic Reader(NeRd) [24] that has an encoder-decoder
architecture and can be used as a weakly or fully supervised model.

The aforementioned approaches deal with limited target DSL without variables,
if branches and loops. Consider a representative question from MathQA: “A
train running at the speed of 48 km / hr crosses a pole in 9 seconds . What is
the length of the train ?”. The logical form is “multiply(divide(multiply(48,
const 1000), const 3600), 9)”. Unlike the above example, chemistry texts
contain a description of iterative processes and conditional events,
therefore, often require more expressive DSL. Consider the complex process
example in Fig. 1 that requires the notion of loops and if-clauses in addition
to numerical comprehension of multiple entities.

The complexity of the generated DSL requires a more nuanced optimization.
Unlike previous approaches that focused on optimizing the syntax of the target
program, our approach leverages the _run-time information_ of the oracle
program during optimization. We devise a reinforcement learning approach and
during optimization compare the predicted code to the oracle code run-time
variable state. Our reward is therefore composed of both syntactic and run-
time similarity. This helps the model to converge even when the DSL is complex
and the search space is large.

##  3 Problem Definition

Let $\Sigma$ denote an input text, $\Gamma$ denote a simulation question, and
$y\in R$ to denote the correct answer. The question-answering task is to find
$\pi:(\Sigma,\Gamma)\rightarrow y$, which maps the text and the question to
$y$, the correct answer. In this work, we focus on simulation questions. A
simulation question is a question that asks someone to imagine what might
happen. Formally, we focus on problems that can be solved by the auxiliary
task of $\pi^{\prime}:(\Sigma,\Gamma)\rightarrow\lambda$, which maps the text
and the question to a code, s.t. $\lambda()=y$, i.e., when executing $\lambda$
it produces $y$.

##  4 Methodology

###  4.1 Domain Specific Language

We start by defining a language $\lambda$, s.t. a text can be matched to it.
We aim to define a concise version of a full-featured programming language to
reduce the size of the search space of possible programs thus potentially
leading to better convergence. We define this domain-specific language (DSL)
to resemble c-syntax language, but with a less complex syntax while still
retaining the capability of defining variables, “if” branches, and loops to
manipulate a variables’ state. The following is the grammar definition of the
new language:

$\displaystyle Prog\ p:=\text{func simulation()}\ \\{\ s\ \\}$ $\displaystyle
Stmt\ s:=\text{repeat(}n\text{)}\ \\{\ s\ \\}\ \ |\ \ \text{if}(c)\\{\ s\ \\}$
$\displaystyle\indent\indent\indent\indent|\ \text{return}\ e\ ;\ |\ \ a;\ \
|\ \ s_{1};s_{2}\ $ $\displaystyle Cond\ c\ :=\ identifier\ op\\_c\ e$
$\displaystyle Assign\ a\ :=\ identifier\ =\ e\ $
$\displaystyle\indent\indent\indent\indent|\ identifier=e\ op\ e$
$\displaystyle e\ :=\ identifier\ |\ n\ |\ \text{-}n\ |\ r\ |\ \text{-}r$
$\displaystyle op\\_c:=\ <\ |\ >\ |\ >=\ |\ <=\ |\ !=\ |\ ==$ $\displaystyle
op\ :=\ \ \ +\ \ |\ -\ |\ *\ |\ \ \backslash\ \ \ |\ \backslash\backslash$
$\displaystyle identifier\ :=\
[a\text{-}zA\text{-}Z][a\text{-}zA\text{-}Z0\text{-}9\\_]*$ $\displaystyle n\
:=\ [0\text{-}9]\text{+}$ $\displaystyle r\ :=\
[0\text{-}9]\text{+.}[0\text{-}9]\text{+}$

We use this grammar to derive an abstract syntax tree (AST) to validate the
syntax of the programs. We provide tools to translate the generated code to
python code to allow code execution.

###  4.2 Neural Program Synthesis for Question-Answering

Assume we have a labeled dataset of $(\Sigma_{i},\Gamma_{i},\lambda_{i})$. One
can refer to the problem as a translation task, i.e., given a text
$\Sigma_{i}$ and a question $\Gamma_{i}$ generate the corresponding code,
$\lambda_{i}$, that answers the question $\Gamma_{i}$. The common approach
[26, 27] is to use an encoder-decoder architecture that maximizes the
likelihood (MLE) of the corresponding text. In our setting that corresponding
text is the reference program. Recently, it has been shown that an additional
step of optimization, applied after the model is trained, can significantly
boost results [28, 29, 30]. Usually, reinforcement learning is applied during
this optimization step. We adopted the architecture of the encoder-decoder
model and used Transformers (as suggested in Attention is all you need paper
[31]) and train the model using a cross-entropy loss. We treat the trained
decoder from the previous step (optimized for MLE) as a model-free agent and
further optimize the code generation process using reinforcement-learning. We
use beam search to sample $N$ possible programs that have the highest sequence
probability, calculated by the multiplication of the probability for each
token that constructs the program. We denote the reward function as
$Q:(\lambda,\hat{\lambda})\rightarrow R$ and apply it on the predicted
programs. We take the $S$ programs with the highest value, as determined by
$Q$. Because the target function is non-smooth we cannot use it as a loss
function and backpropagate the gradient thus, we use the REINFORCE [30, 32]
algorithm to optimize the reward using the following loss function:

$\displaystyle\mathcal{L}=-Q(\lambda_{i},\hat{\lambda_{i}})log(p(\hat{\lambda_{i}}|\Sigma_{i},\Gamma_{i},\lambda))$
(1)

We gather a batch size of $B$ generated code sequences. For each sequence the
loss function is calculated. We average the loss across the $B$ sequences. The
result is back-propagated to update the weights of the encoder-decoder model.
Algorithm 1 illustrates the training: After training the encoder-decoder model
using the cross-entropy loss (line 2) we iterate over all training examples.
For each example a beam-search is applied to return the $N$ sequences of the
highest probabilities (line 4). The algorithm then selects the top sequences
based on the reward $Q$ (lines 5-7). The final loss is calculated for the
batch (lines 8-11).

Algorithm 1 Training

Input: $N,S,B$

1: $batch\leftarrow$ []

2: $model\ \leftarrow$ pre-train MLE model

3: for $(\Sigma_{i},\Gamma_{i},\lambda_{i})$ in dataset do

4: $n\\_top\leftarrow$ beam_search($model,\Sigma_{i},\Gamma_{i},N$)

5: $s\\_top\leftarrow$[$(Q(\lambda,\hat{\lambda}),\hat{\lambda})$ for
$\hat{\lambda}$ in n_top])

6: $s\\_top\leftarrow$ sort(s_top) # sort by $Q$ value

7: $batch\leftarrow$ batch + s_top[:S] # take top s

8: if length($batch$) $=$ $B$ then

9: $\mathcal{L}\leftarrow-\frac{1}{B}\sum_{i\in
batch}{Q_{i}log(p(\hat{\lambda_{i}}|\Sigma_{i},\Gamma_{i}))}$

10: $\mathcal{L}.backward()$

11: $batch\leftarrow$[]

12: end if

13: end for

We consider a reward function, $Q$, that will measure both the the syntactic
similarity between $\lambda_{i}$ and $\hat{\lambda_{i}}$, and the semantic
similarity. We denote the syntactic similarity as $Q_{syntactic}$ (Section
9.3) and the semantic as $Q_{semantic}$ (Section 4.3). We then set the final
$Q$ as the weighted sum of the syntactic and semantic scores:

$\displaystyle Q=\gamma Q_{syntactic}+(1-\gamma)Q_{semantic}$ (2)

Recent research has shown that an additional step of BLEU-score optimization,
applied after a model is trained, can produce dramatic improvements in
performance [28, 29, 30]. Intuitively, training the encoder-decoder
architecture using cross-entropy loss for each token optimizes the max
likelihood estimation (MLE) for each token separately, while BLEU-score
optimization optimizes the sequence as a whole. The BLEU-score can give a
quantitative estimation of the syntactic similarity of the predicted code with
the reference code. Therefore, we define:
$Q_{syntactic}=BLEU(\hat{\lambda_{i}},\lambda_{i})$.

###  4.3 Semantic Neural Program Synthesis

Unlike translation-based approaches, we hypothesize that when generating code,
it might not be sufficient to only require syntactic similarity to the
reference program. A small syntax mistake can lead to a wrong output, syntax
errors, or run-time errors. Consider the example presented in Figure 2. The
left program simulates an iterative process that eventually outputs the sum of
an arithmetic series while the right program calculates the sum of a geometric
series. The two programs are close syntactically but semantically different.
Therefore, we aim to capture the semantics of the code execution alongside the
syntax.

A computer program stores data in variables. The contents of these variables
during the code, at any given point in the program’s execution, is called a
_program’s state_. We define semantic similarity of code programs as the
similarity of the program’s states during code execution. Formally, we define
$\hat{S_{t}}=\\{\hat{v_{1}}^{t},\hat{v_{2}}^{t},...,\hat{v_{n}}^{t}\\}$ as the
state of the predicted program and
$S_{t}=\\{v_{1}^{t},v_{2}^{t},...,v_{n}^{t}\\}$ as the state of the reference
program at time $t$. Note that if the program didn’t define $v_{i}$ yet we
treat it’s value as $\epsilon$, a special symbol that define that the variable
has not been used.

We define $\hat{I}=(\hat{\sigma_{1}},\hat{\sigma_{2}},...,\hat{\sigma_{m}})$
and $I=(\sigma_{1},\sigma_{2},...,\sigma_{k})$ as the instructions of the
predicted program and the oracle program correspondingly. An instruction is a
line of code that can change the state of the memory. We use function
$E:(S_{t},\sigma_{t})\rightarrow S_{t+1}$ to execute the instruction in time t
and compute the next state at time t+1. We use these definitions to define
semantic reward function that rewards each state that is aligned with the
reference code state.

Unlike $Q_{syntactic}$ that measures the syntactic similarity between the
generated code and $\lambda_{i}$, $Q_{semantic}$ should capture the similarity
of the states of the the generated code and the reference program
$\lambda_{i}$. We aim to integrate the run-time semantic estimation into the
training process. To evaluate the similarity of the code states at each step,
$\lambda_{i}$ and the generated code are executed and a reward is calculated:

$\displaystyle T_{max}=max(|I|,|\hat{I}|),T_{min}=min(|I|,|\hat{I}|)$
$\displaystyle
Q_{semantic}=\frac{1}{T_{max}}\sum_{t=0}^{T_{min}}\begin{cases}1,&\forall
i\in[n],\\\ &v_{i}^{t}\in E(S_{t},\sigma_{t}),\\\ &\hat{v_{i}}^{t}\in
E(\hat{S_{t}},\hat{\sigma_{t}}),\\\ &v_{i}^{t}=\hat{v_{i}}^{t}\\\
0,&\text{otherwise}\end{cases}$

After the execution of both programs, the reward is estimated by the number of
matched state transitions. Algorithm 2 presents the semantic reward algorithm.
The input to the algorithm is the reference code and the predicted code. The
state transitions after each line are recorded during code execution. A check
is then performed if the two programs modified the variables in the same way
and returned the same values.

Algorithm 2 Semantic Reward

Input: $\lambda_{i}$, $\hat{\lambda_{i}}$

1:
$\textit{r\\_state}\leftarrow\textit{execute\\_and\\_get\\_state($\lambda_{i}$)}$

2:
$\textit{p\\_state}\leftarrow\textit{execute\\_and\\_get\\_state($\hat{\lambda_{i}}$)}$

3: $correct\\_states\leftarrow 0$

4: $T_{min}\leftarrow min(len(r\\_state),len(p\\_state))$

5: for $t=0$ to $T_{min}$ do

6: if all(r_state[t] = p_state[t]) then

7: $correct\ states\leftarrow correct\\_states+1$

8: end if

9: end for

10: $T_{max}\leftarrow max(len(r\\_state),len(p\\_state))$

11: return $correct\\_states/T_{max}$

Note that we require that all the variables will have the same values for each
execution timestamp, but we can think on other variations that allow equality
only for part of the variables, or allow the state to be equal on different
stages of the execution and not only in the same timestamp. We leave these
experiments to future research.

##  5 Experimental Methodology

###  5.1 Datasets

The chemical reactions’ dataset contains descriptions of reactions between
chemicals from the US patents published between 1976-2016. Each reaction is
annotated with the corresponding components, their quantities and actions that
affected the process outcome. Consider this example from the U.S. Patent
dataset:

> "A mixture of 50 g (0.42 mole) of p-aminobenzonitrile, 146 ml of
> concentrated hydrochloric acid, and 106 ml of H2O was heated at 80c
> for 20 minutes and then cooled to 0°. … The hexane was cooled and the
> resulting precipitate was filtered and air dried to yield 16 g (18%) of
> p-(5-acetyl-2-furyl) benzonitrile."

The example describes the process’s components (yellow), their quantities
(green) and the actions (red) that were taken to cause the components to
interact with each other to yield 16g of p-(5-acety-2-furyl) benzonitrile. We
used the US patents dataset to create the SimQA datasets – question-answering
datasets on texts that contain descriptions of chemical processes and
reactions. Chemists were asked to curate a set of question and answers over
entities, actions and quantities. We provide two datasets based on those
questions, SimQA-v1 and SimQA-v2. For SimQA-V2, the chemists were only allowed
to create questions about the reaction described. An example of such question
on the above text: “How many milliliters of water we need to add to yield 25
grams of the product if we want to preserve the same ratio between them?”.

For SimQA-V1, the chemists were allowed to add additional plausible
information about the reaction to allow creating more advanced questions. The
context is added to the dataset. An example of such context: “If the
temperature exceed 100 degrees when heating the mixture it causes the water to
be evaporated at a rate of 2.3 milliliters per minute.”, and the corresponding
question: “How many milliliters of water should we add if we increase the
temperature by 2 degrees per minutes for 10 minutes and we want to cancel the
effect of the evaporation?”

Besides the true answer, we generate additional three possible answers for
each question to match the multiple-choice task requirements. This allows us
to compare against state-of-the-art models for the end-to-end approach. We
generate the three wrong answers randomly, but close in value to the true
answer see example in appendix A. Note that our model doesn’t use these
answers and instead try to predict the answer directly from the question and
the reference code. The datasets contain 6187 training examples (3158 in
SimQA-V1 and 3029 in SimQA-v2) and 1089 test examples (555 in SimQA-V1 and 534
in SimQA-V2).

##  6 Baselines and Metrics

To evaluate the value of solving the auxiliary problem of finding the program
that answers the question instead of answering the question directly, we
compare against state-of-the-art question-answering models: Roberta [1], XLnet
[2] and Longformer [33].We use Roberta and XLnet as baselines because recently
they reached state-of-the-art performance on various question answering
datasets, and we use Longformer because our task often requires to learn long
dependencies between the chemical components. We use pre-trained models and
fine-tune them on the SimQA training datasets. Because these models perform
well on QA tasks when the answer is in the text itself or with a multi choice
setup, Beside the right answer we generate another three wrong answer that are
close numerical to the right one. we give these baselines the advantage of
choosing the right answer from the 4 possibilities while our model should
output the right answer directly without seeing the 4 choices.

To compare our approach to other semantic parsing approaches we used Seq2Prog
[10], an encoder decoder architecture based on LSTM models which leverages
attention mechanism to decode the right program, and NeRd [24] (its fully-
supervised version) that also uses encoder(the "reader") decoder(the
"programmer") architecture based on BERT and LSTM models that achieved state-
of-the-are performance on MathQA [10] and Drop [9] datasets – both are a well-
known benchmark for commonsense inference and logical reasoning tasks. Note
that these baselines have the same task as our model to - to predict the
answer to the question given the context, question and reference code. For the
NPS-SQA, after hyper-parameters tuning (separate validation set), we chose to
use a beam width of 32 sequences. We set $N=8$, $S=4$ and $B=32$. All the
models were evaluated using the accuracy metric, i.e., the percentage of
correct answers.

##  7 Experiments and Results

In this section, we present the main empirical results of NPS-SQA compare to
the other baselines. We run our experiments using Tesla V100 GPU with 16GB of
memory.

##  8 Main Result

Table 1 presents the main results of our approach on the two SimQA datasets.
Statistically significant results are shown in bold. We observe that NPS-SQA
outperforms current state-of-the-art methods significantly with a boost of 33%
- 36% in accuracy against the end-to-end models. This shows the merit of using
code generation as an auxiliary task. Additionally, we observe a boost of 4% -
5% compared to other semantic parsing approaches. This shows the merit of code
execution as part of the run-time similarity between the oracle program and
the predicted program during the training process. The SimQA-v1 dataset
contains questions that require an understanding of the process and chain of
events that lead to the final product. As we show in 9.4 this is the strength
of NPS-SQA which explain the better performance of NPS-SQA on SimQA-v1 over
SimQA-v2.

Table 1: Main results Model | SimQA v1 | SimQA v2
---|---|---
Roberta | 0.5233 | 0.5112
XLnet | 0.5421 | 0.5102
Longformer | 0.5291 | 0.4991
Seq2Seq | 0.8013 | 0.7921
Seq2Prog | 0.8279 | 0.8021
NeRd | 0.8261 | 0.8041
NPS-SQA | 0.8801 | 0.8431

##  9 The Syntactic and Semantic Trade-off

The reward function of $Q$ is composed of both syntactic and semantic
similarity (Eq. 2). To better understand the right balance between the
syntactic reward, $Q_{syntactic}$, and the semantic reward, $Q_{semantic}$, we
experimented with different values of $\gamma$. Figure 3 indicates that the
optimal value for $\gamma$ is around $0.5$ on both datasets. This strengthens
the hypothesis that both syntactic and semantic similarity help improve the
performance.

###  9.1 Syntactic Reward

We observe that for $\gamma=1$, where only the syntactic similarity is used
during the reward, the model reaches $0.8253$ and $0.8082$ for SimQA-v1 and
SimQA-v2 respectively which match the performance of other approaches based on
syntactic similarity.

###  9.2 Semantic Reward

We observe that for $\gamma=0$, where only the semantic run-time code
similarity is used during the reward, the model reaches $0.8313$ and $0.8112$
for SimQA-v1 and SimQA-v2 respectively. This shows that even if we use only
state-score optimization after MLE optimization we can boost the performance
of the model.

Figure 3: The performance of the model with the syntactic-semantic reward for
different $\gamma$ values.

###  9.3 Syntactic Neural Program Synthesis

###  9.4 Strengths of NPS-SQA

Mapping the text and the question to code as an intermediate step helps the
model to find the chain of events that lead to the correct output. We observed
that this approach performs well when dealing with questions that require some
complex numerical calculation or execution of iterative loops and if branches.
Formally, we define a _Complex Simulation Question_ as a question that
requires a code that includes at least one if branch and one loop branch. This
code needs to modify the state of the variables more than 15 times. An example
of such a question is:

> Context: A mixture of 5-hydroxy-1-tetralone (2.00 g) and sodium hydroxide
> (493 mg) was warmed in ethanol (40 mL) to 50° C. …. If the temperature
> passes 56 degrees when heating ethanol for more than 5 seconds it will
> result in a loss of 2.04 grams in the final products for each additional
> second.
>
> Question: How many grams the process would yield if we increase the
> temperature when heating ethanol by 2 degrees every 2 seconds for 16
> seconds?

We extracted a subset of 443 examples with these characteristics and compared
the performance of NPS-SQA to Roberta, NeRd and Seq2Prog models after training
on the other examples.

Table 2 shows that our approach outperforms these models by a large margin on
those questions that are prevalent when dealing with texts that describe
scientific processes.

Table 2: Performance for complex questions Model | Accuracy - Complex QA
---|---
Roberta | 0.4271
Seq2Prog | 0.8183
NeRd | 0.8153
NPS-SQA | 0.8703

###  9.5 Limitations of NPS-SQA

The code generation approach works well on complex questions that involve
state transitions or complex numeric calculations. However, if the question is
simple the performance boost compared to baselines is lower. We define a
_Simple Simulation Question_ as a question that requires a code that includes
no more than one if branch and no loop branches. This code is allowed to
change the state no more than 5 times. For example:

> Context: …prepared as described in example 7A, are refluxed at 160° C. for
> 16 hours in a mixture of 9 ml of dimethylformamide, 1.3 ml of 85% formic
> acid and 0.5 ml concentrated hydrochloric acid. After the reaction m … with
> benzene. 4 g of the title product are obtained. If the temperature exceed
> 175 degrees when heating dimethylformamide it hurts the process
>
> Question: What will be the outcome of the process in grams if the
> temperature increases by 14 degrees?

We gathered 361 simple questions examples and compared the performance of NPS-
SQA against the same baselines after training on the other examples. For such
questions, NPS-SQA still outperforms the performance of the state-of-the-art
end-to-end model but matches the performance of the semantic parsing
approaches (see Table 3)

Table 3: Performance for simple questions Model | Accuracy - Simple QA
---|---
Roberta | 0.6027
Seq2Prog | 0.8215
NeRd | 0.8247
NPS-SQA | 0.8231

##  10 Conclusions

In this work, we explored the problem of simulation question-answering. We
presented an approach that tackles this problem by generating a code that
simulates the process described in the text and the environment conditions of
the question. We automatically construct a dataset of descriptions of chemical
processes. Chemists then curated possible questions and answers for those
process descriptions. Each description and question were mapped to a codes of
the process described. We train an encoder-decoder model that optimizes both
for syntactic similarity and semantic code similarities. Once the code is
generated it is executed and outputs the answer to the question. We show that
such approach significantly boosts performance by more than 30% in accuracy as
compared to the state-of-the-art QA. We study the semantic-syntactic trade-off
and identify that both are needed for successful question-answering and
process simulation. Additionally, we identify that questions that require
complex simulation of process, with if-branches and loops, especially benefit
from the intermediate task of code generation and execution. We also show the
potential of explainability of the model using the generated code. In future
work, we wish to study the integration of external facts and world knowledge
to code generation. In this work, all the information that is needed to answer
the question can be found in the text itself without the need to combine
knowledge from external sources. For example, the model does not know that the
boiling temperature of water is 100 C if it was not written in the text
explicitly. Additional such knowledge can significantly improve broad
simulation-question answering.

## References

  * [1] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov.  Roberta: A robustly optimized BERT pretraining approach.  CoRR, abs/1907.11692, 2019.
  * [2] Zhilin Yang, Zihang Dai, Yiming Yang, Jaime G. Carbonell, Ruslan Salakhutdinov, and Quoc V. Le.  Xlnet: Generalized autoregressive pretraining for language understanding.  In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, 8-14 December 2019, Vancouver, BC, Canada, pages 5754–5764, 2019.
  * [3] Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang.  Squad: 100, 000+ questions for machine comprehension of text.  In Jian Su, Xavier Carreras, and Kevin Duh, editors, Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 2383–2392. The Association for Computational Linguistics, 2016.
  * [4] Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi.  SWAG: A large-scale adversarial dataset for grounded commonsense inference.  In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii, editors, Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, pages 93–104. Association for Computational Linguistics, 2018.
  * [5] Jonathan Berant, Vivek Srikumar, Pei-Chun Chen, Abby Vander Linden, Brittany Harding, Brad Huang, Peter Clark, and Christopher D. Manning.  Modeling biological processes for reading comprehension.  In Alessandro Moschitti, Bo Pang, and Walter Daelemans, editors, Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL. ACL, 2014.
  * [6] Chloé Kiddon, Luke Zettlemoyer, and Yejin Choi.  Globally coherent text generation with neural checklist models.  In Jian Su, Xavier Carreras, and Kevin Duh, editors, Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 329–339. The Association for Computational Linguistics, 2016.
  * [7] Ziyu Yao, Xiujun Li, Jianfeng Gao, Brian M. Sadler, and Huan Sun.  Interactive semantic parsing for if-then recipes via hierarchical reinforcement learning.  In The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019, pages 2547–2554. AAAI Press, 2019\.
  * [8] Niket Tandon, Bhavana Dalvi, Keisuke Sakaguchi, Peter Clark, and Antoine Bosselut.  WIQA: A dataset for "what if…" reasoning over procedural text.  In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, pages 6075–6084. Association for Computational Linguistics, 2019.
  * [9] Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner.  DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs.  In Proc. of NAACL, 2019.
  * [10] Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi.  MathQA: Towards interpretable math word problem solving with operation-based formalisms.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2357–2367, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
  * [11] Daniel Lowe.  Chemical reactions from US patents (1976-Sep2016).  6 2017.
  * [12] Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer.  Deep contextualized word representations.  In Marilyn A. Walker, Heng Ji, and Amanda Stent, editors, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers), pages 2227–2237. Association for Computational Linguistics, 2018.
  * [13] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
  * [14] Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard H. Hovy.  RACE: large-scale reading comprehension dataset from examinations.  In Martha Palmer, Rebecca Hwa, and Sebastian Riedel, editors, Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017, pages 785–794. Association for Computational Linguistics, 2017.
  * [15] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman.  GLUE: A multi-task benchmark and analysis platform for natural language understanding.  In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
  * [16] Bill Yuchen Lin, Seyeon Lee, Rahul Khanna, and Xiang Ren.  Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-Trained Language Models.  In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6862–6868, Online, November 2020\. Association for Computational Linguistics.
  * [17] Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang.  Semantic parsing on Freebase from question-answer pairs.  In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1533–1544, Seattle, Washington, USA, October 2013. Association for Computational Linguistics.
  * [18] Pradeep Dasigi, Matt Gardner, Shikhar Murty, Luke Zettlemoyer, and Eduard Hovy.  Iterative search for weakly supervised semantic parsing.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2669–2680, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
  * [19] Chen Liang, Jonathan Berant, Quoc V. Le, Kenneth D. Forbus, and Ni Lao.  Neural symbolic machines: Learning semantic parsers on freebase with weak supervision.  In Regina Barzilay and Min-Yen Kan, editors, Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, pages 23–33. Association for Computational Linguistics, 2017.
  * [20] Chen Liang, Mohammad Norouzi, Jonathan Berant, Quoc V. Le, and Ni Lao.  Memory augmented policy optimization for program synthesis with generalization.  CoRR, abs/1807.02322, 2018.
  * [21] Robin Jia and Percy Liang.  Data recombination for neural semantic parsing.  In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers. The Association for Computer Linguistics, 2016.
  * [22] Li Dong and Mirella Lapata.  Language to logical form with neural attention.  In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers. The Association for Computer Linguistics, 2016.
  * [23] Li Dong and Mirella Lapata.  Coarse-to-fine decoding for neural semantic parsing.  In Iryna Gurevych and Yusuke Miyao, editors, Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers, pages 731–742. Association for Computational Linguistics, 2018.
  * [24] Xinyun Chen, Chen Liang, Adams Wei Yu, Denny Zhou, Dawn Song, and Quoc V. Le.  Neural symbolic reader: Scalable integration of distributed and symbolic representations for reading comprehension.  In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020.
  * [25] Sewon Min, Danqi Chen, Hannaneh Hajishirzi, and Luke Zettlemoyer.  A discrete hard EM approach for weakly supervised question answering.  In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, pages 2851–2864. Association for Computational Linguistics, 2019.
  * [26] Yan Wang, Xiaojiang Liu, and Shuming Shi.  Deep neural solver for math word problems.  In Martha Palmer, Rebecca Hwa, and Sebastian Riedel, editors, Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017, pages 845–854. Association for Computational Linguistics, 2017.
  * [27] Pranav Rajpurkar, Robin Jia, and Percy Liang.  Know what you don’t know: Unanswerable questions for SQuAD.  In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 784–789, Melbourne, Australia, July 2018. Association for Computational Linguistics.
  * [28] Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba.  Sequence level training with recurrent neural networks.  In Yoshua Bengio and Yann LeCun, editors, 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, 2016.
  * [29] Lijun Wu, Li Zhao, Tao Qin, Jianhuang Lai, and Tie-Yan Liu.  Sequence prediction with unlabeled data by reward function learning.  In Carles Sierra, editor, Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19-25, 2017, pages 3098–3104. ijcai.org, 2017.
  * [30] Yaser Keneshloo, Tian Shi, Naren Ramakrishnan, and Chandan K. Reddy.  Deep reinforcement learning for sequence to sequence models.  CoRR, abs/1805.09461, 2018.
  * [31] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA, pages 5998–6008, 2017.
  * [32] Mattis Manfred Kämmerer.  On policy gradients.  CoRR, abs/1911.04817, 2019.
  * [33] Iz Beltagy, Matthew E. Peters, and Arman Cohan.  Longformer: The long-document transformer.  CoRR, abs/2004.05150, 2020.