# CitySpec with Shield: A Secure Intelligent Assistant for Requirement
Formalization

Zirong Chen  Isaac Li  Haoxiang Zhang  Sarah Preum  John A. Stankovic  Meiyi
Ma

###### Abstract

An increasing number of monitoring systems have been developed in smart cities
to ensure that a city’s real-time operations satisfy safety and performance
requirements. However, many existing city requirements are written in English
with missing, inaccurate, or ambiguous information. There is a high demand for
assisting city policymakers in converting human-specified requirements to
machine-understandable formal specifications for monitoring systems. To tackle
this limitation, we build CitySpec [1], the first intelligent assistant system
for requirement specification in smart cities. To create CitySpec, we first
collect over 1,500 real-world city requirements across different domains
(e.g., transportation and energy) from over 100 cities and extract city-
specific knowledge to generate a dataset of city vocabulary with 3,061 words.
We also build a translation model and enhance it through requirement synthesis
and develop a novel online learning framework with shielded validation. The
evaluation results on real-world city requirements show that CitySpec
increases the sentence-level accuracy of requirement specification from 59.02%
to 86.64%, and has strong adaptability to a new city and a new domain (e.g.,
the F1 score for requirements in Seattle increases from 77.6% to 93.75% with
online learning). After the enhancement from the shield function, CitySpec is
now immune to most known textual adversarial inputs (e.g., the attack success
rate of DeepWordBug [2] after the shield function is reduced to 0% from
82.73%). We test the CitySpec with 18 participants from different domains.
CitySpec shows its strong usability and adaptability to different domains, and
also its robustness to malicious inputs.

###### keywords:

Requirement Specification , Intelligent Assistant , Monitoring , Safety Shield
, Smart City

††journal: Pervasive and Mobile Computing

[inst1]organization=Vanderbilt University,city=Nashville, state=Tennessee,
country=USA

[inst2]organization=University of Virginia,city=Charlottesville,
state=Virginia, country=USA

[inst3]organization=Columbia University,city=New York, state=New York,
country=USA

[inst4]organization=Dartmouth College,city=Hanover, state=New Hampshire,
country=USA

##  1 Introduction

With the increasing demand for safety guarantees in smart cities, significant
research efforts have been spent toward how to ensure that a city’s real-time
operations satisfy safety and performance requirements [3]. Monitoring
systems, such as SaSTL runtime monitoring [4], CityResolver [5], and STL-U
predictive monitoring [6], have been developed in smart cities. Figure 1 shows
a general framework of monitoring systems in smart cities. These systems are
designed to execute in city centers and to support decision-making based on
the verification results of real-time sensing data about city-states (such as
traffic and air pollution). If the monitor detects a requirement violation,
the city operators can take actions to change the states, such as improving
air quality, sending alarms to police, calling an ambulance, etc.

The monitor systems have two important inputs, i.e., the real-time data
streams and formal specified requirements. Despite that extensive research
efforts have been spent toward improving the expressiveness of specification
languages and efficiency of the monitoring algorithms, the research challenge
of how to convert human-specified requirements to machine-understandable
formal specifications has received only scant attention. Moreover, our study
(see Section 2) on over 1,500 real-world city requirements across different
domains111In this paper, we define a domain as an application area in smart
cities, such as transportation, energy, and public safety. shows that, first,
existing city requirements are often defined with missing information or
ambiguous description, e.g., no location information, using words like nearby,
or close to. They are not precise enough to be converted to a formal
specification or monitored in a city directly without clarifications by policy
makers. Secondly, the language difference between English specified
requirements and formalized specifications is significant. Without expertise
in formal languages, it is extremely difficult or impossible for policy makers
to write or convert their requirements to formal specifications. Therefore,
there is an urgent demand for an intelligent system to support policy makers
for requirement specifications in smart cities.

Despite the prevalence of developing models to translate the natural language
to machine languages in various applications, such as Bash commands [7],
Seq2SQL [8], and Python [9], it is very challenging to develop such an
intelligent system for requirement specification in smart cities for the
following reasons. First, unlike the above translation tasks with thousands or
even millions of samples in a dataset, there is barely any requirement
specification data. As a result, traditional language models are not
sufficient to be applied directly. Moreover, the requirements usually contain
city domain-specific descriptions and patterns that existing pre-trained
embeddings like BERT or GloVE cannot handle effectively. Furthermore,
requirements from different domains and cities vary significantly and evolve
over time, thus building a system that can adapt to new domains at runtime is
an open research question. Good adaptability can increase user experience
(e.g., policy makers do not have to clarify new terms repeatedly), while one
of the major challenges is validating and filtering the new knowledge and
avoiding adversarial examples online.

Adversarial samples can be introduced among that new knowledge if there is any
malicious behavior involved in off-guard online learning. Those adversarial
samples will poison the dataset used for continuous learning and system
adaptation if there is no security enhancement. Here we emphasize the
importance of an effective and safe validating function against malicious
inputs. After reviewing the most recent literature, we find adversarial
examples perturb textual inputs not only at the character level but also at a
word or even sequence level. By only looking at textual information, the
validation mechanism alone is not enough to protect the dataset from being
poisoned or even vulnerable to malicious attacks. Because in this continuous
learning scenario, the attacker could keep attacking until the model
prediction changes. Thus, an effective and comprehensive approach must first
detect malicious behaviors. By doing so, the validation model is kept safe and
can further help protect and enrich the dataset.

Figure 1: CitySpec in Smart Cities

In this paper, we target the above technical challenges and develop CitySpec,
an intelligent assistant system for requirement specification in smart cities.
To the best of our knowledge, it is the first specification system helping
city policy makers specify and translate their requirements into formal
specifications automatically. As shown in Figure 1, CitySpec is designed to
bridge the gap between city policy makers and monitoring systems. It enables
policy makers to define their requirements by detecting missing, inaccurate,
or ambiguous information through an intelligent assistant interface. To
effectively train the translation model using a small amount of city
requirement data, CitySpec extracts city knowledge and enhances the learning
process through requirement synthesis. CitySpec can easily adapt to a new city
or application domain through online learning and validation under the
protection of a shield model.

Contributions. We summarize the major contributions of this paper as follows:

  * 1.

We collect and annotate over 1,500 real-world city requirements from over 100
cities across different domains. We extract city-specific knowledge and build
a dataset of city vocabulary with 3,061 words in 5 categories.

  * 2.

We create an intelligent assistant system for requirement specification in
smart cities. In the system, we build a translation model and enhance it
through requirement synthesis, and develop a novel online learning framework
with validation under uncertainty.

  * 3.

We evaluate CitySpec extensively using real-world city requirements. The
evaluation results show that CitySpec is effective on supporting policy makers
accurately writing and refining their requirements. It increases the sentence
level accuracy of requirement specification from 59.02% to 86.64% through city
knowledge injection. It shows strong adaptability (user experience) to a new
city (e.g., F1 score in Seattle from 77.6% to 93.75%) and a new domain (e.g.,
F1 score in security domain from 62.93% to 93.95%).

  * 4.

We overview 12 different adversarial text attacks from the most recent
literature and launch them to the validation function. Based on generated
adversarial samples, we develop a shield model that is hard to attack and
immune to most adversarial samples. The evaluation results show its ability to
significantly reduce attack success rate (e.g., the attack success rate of
BertAttack is reduced from 94.00% to 4.31%).

  * 5.

We conducted a real user case study with 18 participants with different
backgrounds. The study shows the high usability and adaptability of CitySpec
not only in smart city scenarios but also in unseen domains (e.g., high user
experience scores and low numbers of interactions are reported in domains like
Medical and Environmental Engineering). Furthermore, the effectiveness of the
shield function is shown during the survey (e.g., 87.23% defense success rate
against commonly seen adversarial attacks).

This paper is an extension of [1]. We extend with the following new
contributions. First, we carefully study potential adversarial text
generation. We summarize the characteristics of each attack and its effects on
our system. Second, we also implement and further experiment with those 12
attacks. We find potential vulnerabilities in the initial validation model
under those attacks if off-guard. Third, by targeting adversarial attacks, we
enhance our system security by developing an effective and secure shield model
to protect the validation model from malicious attacks. Fourth, we further
conduct a comprehensive evaluation of the newly developed shield layers of
CitySpec and show that CitySpec can significantly reduce the success rate of
12 types of attacks. Last, we conduct a user case study with 18 participants
with different backgrounds to test the usability and adaptability of CitySpec
in the smart city. Leveraging the different backgrounds of participants, we
also prove the capability of CitySpec to learn continuously and adapt
efficiently in unseen domains.

Paper organization: In the rest of the paper, we describe the motivating study
of city requirement specification in Section 2, provide an overview of
CitySpec in Section 3, and present the technical details in Section 4. We then
present the evaluation results in Section 5, discuss the related work in
Section 7 and draw conclusions in Section 8.

##  2 Motivating Study

Table 1: Comparison between English requirements and formal specifications (DLD: Damerau–Levenshtein Distance) ID |  Requirement vs Specification | DLD
---|---|---
1 |  Req: Sliding glass doors shall have an air infiltration rate of no more than 0.3 cfm per square foot. | 59
Spec:
$\mathsf{Always}_{[0,+\infty)}~{}(\mathsf{{Sliding~{}glass~{}doors}}~{}\mathsf{air~{}infiltration~{}rate}~{}\leq
0.3~{}\mathsf{cfm/foot}^{2})$
2 |  Req: The operation of a Golf Cart upon a Golf Cart Path shall be restricted to a maximum speed of 15 miles per hour. | 67
Spec:
$\mathsf{Everywhere}(\mathsf{Golf~{}Cart~{}Path})(\mathsf{Always}_{[0,+\infty)}(\mathsf{Golf~{}Cart~{}speed}<15~{}\mathsf{miles/hour}))$
3 |  Req: Up to four vending vehicles may dispense merchandise in any given city block at any time. | 75
Spec:
$\mathsf{Everywhere}(\mathsf{city~{}block})(\mathsf{Always}_{[0,+\infty)}(\mathsf{vending~{}vehicles}\leq
4))$

In this section, we study real-world city requirements and their formal
specification as motivating examples to discuss the demand and challenges of
developing an intelligent assistant system for requirement specification in
smart cities. We collect and annotate over 1,500 real-world city requirements
(e.g., standards, codes of ordinances, laws, regulations, etc. [10, 11, 12,
13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24]) from over 100 cities and
regions (e.g. New York City, San Francisco, Chicago, Washington D.C., Beijing,
etc.) around the world. Those requirements also cover different application
domains like transportation, environment, security, public safety, indoor
environments, etc. We make the following observations from the analysis of the
requirement dataset.

Existing city requirements are often defined with missing information or
ambiguous description. In [4], the authors define essential elements for
monitoring a city requirement. Within the 1500 requirements, many requirements
have one or more missing elements. For example, 27.6% of the requirements do
not have location information, 29.1% of the requirements do not have a proper
quantifier, and 90% of the requirements do not have or only have a default
time (e.g., always) defined. Additionally, requirements often have ambiguous
descriptions that are difficult to be noticed by policy makers. For example, a
location is specified as “nearby” or “close to”. As a result, it is very
difficult or impossible for the monitoring system to monitor these
requirements properly. It indicates a high demand for an intelligent assistant
system to support the policy makers to refine the requirements.

The language difference between English specified requirements and their
formal specifications is significant. In Table 1, we give three examples of
city requirements in English, their formal specification in SaSTL, and the
Damerau–Levenshtein Distance (DLD) [25] between each pair of requirements. DLD
measures the edit distance between two sequences. It shows that natural
languages are different from machine-compatible input languages. Formal
specifications usually consist of mathematical symbols, which makes the
conversion even more difficult. As shown in Table 1, the average DLD from
English requirements to formal specifications is 67, which means that it
requires an average of 67 edits. As a reference, the average DLD brought by
translating these three English requirements to Latin is 64.67. It indicates
that the conversion from English requirements to formal specifications even
requires more edits than the translation of these requirements from English to
Latin. In general, building a translator from English to Latin would require
millions of samples. However, as an under-exploited area, there is a very
limited number of well-defined requirements. Moreover, annotation of formal
specifications requires specialties in formal methods and is extremely time-
consuming. It presents major challenges for building such a translation model.

##  3 System Overview

Figure 2: System Overview

CitySpec is designed to bridge the gap between city policy makers and
monitoring systems. It supports policy makers to precisely write city
requirements in English through an intelligent interface, and then converts
them to formal specifications automatically. An overview of CitySpec is shown
in Figure 2. There are four major components in CitySpec, including an
intelligent assistant Interface to communicate with policy makers (see Section
4.2), a Requirement Synthesis component to extract city knowledge and
synthesize new requirements to build the translation model (see Section 4.3),
a Translation Model to convert city requirements to formal specifications (see
Section 4.4), and an Online Learning component to adapt the system to new
knowledge (see Section 4.5).

At runtime (as indicated by the orange arrows in Figure 2), a city policy
maker inputs a requirement in English through an intelligent assistant
interface, which sends the requirements to the translation model. The
translation model converts the requirements to a formal specification and
checks if there is any missing information or ambiguous description. The
translation model is built with injected city knowledge through requirement
synthesis at the training time and enhanced through online learning at
runtime. Next, based on the returned results from the translation model, the
intelligent interface communicates with the policy maker to acquire or clarify
the essential information. In this process, the assistant supports the policy
maker to refine the requirement until it is precisely defined and accepted by
the monitor. We present the technical details in Section 4, and develop a
prototype tool of the CitySpec system and deploy it online.

##  4 Methodology

In this section, we present the major components in CitySpec (as shown in
Figure 2). We first introduce requirement specification using Spatial-
aggregation Signal Temporal Logic (SaSTL) [4]. Then we show the design and
technical details of the intelligent assistant interface, requirement
synthesis, translation model, and online learning, respectively.

###  4.1 Requirement Specification using SaSTL

SaSTL is a powerful formal specification language for Cyber-Physical Systems.
We select it as our specification language because of its advantages of
expressiveness and monitoring for smart cities. However, CitySpec is general
and can work with other specification languages. SaSTL is defined on a multi-
dimensional _spatial-temporal signal_ as $\omega:\mathbb{T}\times
L\to\\{{\mathbb{R}\cup\\{\bot\\}\\}}^{n}$, where $\mathbb{T}=\mathbb{R}_{\geq
0}$, represents the continuous time and $L$ is the set of locations.
$X=\\{x_{1},\cdots,x_{n}\\}$ is denoted by the set of variables for each
location. The spatial domain $\mathcal{D}$ is defined as,
$\mathcal{D}:=([d_{1},d_{2}],\psi)$,
$\psi:=\top\;|\;p\;|\;\neg\;\psi\;|\;\psi\;\vee\;\psi$, where $[d_{1},d_{2}]$
defines a spatial interval with $d_{1}<d_{2}$ and $d_{1},d_{2}\in\mathbb{R}$,
and $\psi$ specifies the property over the set of propositions that must hold
in each location.

The syntax of SaSTL is given by

$\begin{array}[]{cl}\varphi:=&x\sim c\;|\ \neg\varphi\;|\
\varphi_{1}\land\varphi_{2}\;|\ \varphi_{1}\mathcal{U}_{I}\varphi_{2}\;|\
\mathcal{A}_{\mathcal{D}}^{\mathrm{op}}x\sim c\;|\
\mathcal{C}_{\mathcal{D}}^{\mathrm{op}}\varphi\sim c\\\ \end{array}$

where $x\in X$, $\sim\in\\{<,\leq\\}$, $c\in\mathbb{R}$ is a constant,
$I\subseteq\mathbb{R}_{>0}$ is a real positive dense time interval,
$\mathcal{U}_{I}$ is the _bounded until_ temporal operators from STL. The
_always_ (denoted $\square$) and _eventually_ (denoted $\lozenge$) temporal
operators can be derived the same way as in STL, where
$\lozenge\varphi\equiv\mathsf{true}\ \mathcal{U}_{I}\varphi$, and
$\square\varphi\equiv\neg\lozenge\neg\varphi$. Spatial _aggregation_ operators
$\mathcal{A}_{\mathcal{D}}^{\mathrm{op}}x\sim c$ for
$\mathrm{op}\in\\{\max,\min,\mathrm{sum},\mathrm{avg}\\}$ evaluates the
aggregated product of traces
$\mathrm{op}(\alpha_{\mathcal{D}}^{x}(\omega,t,l))$ over a set of locations
$l\in L_{\mathcal{D}}^{l}$, and _counting_ operators
$\mathcal{C}_{\mathcal{D}}^{\mathrm{op}}\varphi\sim c$ for
$\mathrm{op}\in\\{\max,\min,\mathrm{sum},\mathrm{avg}\\}$ counts the
satisfaction of traces over a set of locations. From counting operators, we
derive the _everywhere_ operator as
$\boxbox_{\mathcal{D}}\varphi\equiv\mathcal{C}_{\mathcal{D}}^{\mathrm{min}}\varphi>0$,
and _somewhere_ operator as
$\diamonddiamond_{\mathcal{D}}\varphi\equiv\mathcal{C}_{\mathcal{D}}^{\mathrm{max}}\varphi>0$.
Please refer to [4] for the detailed definition and semantics of SaSTL.

###  4.2 Interface for Intelligent Assistant

City requirements often have missing or ambiguous information, which may be
unnoticed by policy makers. It leads to the demand for human inputs and
clarification when converting them into formal specifications. Therefore, we
design an intelligent assistant interface in CitySpec serving as an
intermediary between policy makers and the translation model. It communicates
with policy makers and confirms the final requirements through an intelligent
conversation interface.

To briefly describe the communication process, users first input a requirement
in English, e.g., “due to safety concerns, the number of taxis should be less
than 10 between 7 am to 8 am”. CitySpec interface passes the requirement to
the translation model and gets a formal requirement
($\mathsf{always}_{[7,8]}\mathsf{number~{}of~{}taxi}<10$) with the keywords
including,

  * 1.

$\mathsf{entity}$: the requirement’s main object, e.g., “the number”,

  * 2.

$\mathsf{quantifier}$: the scope of an entity, e.g., “taxi”,

  * 3.

$\mathsf{location}$: the location where this requirement is in effect, which
is missing from the above example requirement,

  * 4.

$\mathsf{time}$: the time period during which this requirement is in effect,
e.g., “between 7 am to 8 am”,

  * 5.

$\mathsf{condition}$: the specific constraint on the entity, such as an upper
or lower bound of $\mathsf{entity}$, e.g., “10”.

As a result, CitySpec detects that the location information is missing from
the user’s requirement and generates a query for the user, “what is the
location for this requirement?” Next, with new information typed in by the
user (e.g., “within 200 meters of all the schools”), CitySpec obtains a
complete requirement.

The next challenge is how to confirm the formal specification with policy
makers. Since they do not understand the formal equation, we further convert
it to a template-based sentence. Therefore, CitySpec presents three formats of
this requirements for users to verify, (1) a template-based requirement, e.g.,
[number] of [taxi] should be [$<$] [10] [between 7:00 to 8:00] [within 200
meters of all the schools], (2) a SaSTL formula
$\mathsf{everywhere}_{\mathsf{school}\land[0,200]}\mathsf{always}_{[7,8]}\mathsf{number~{}of~{}taxi}<10$,
and (3) five key fields detected. Users can confirm or further revise this
requirement through the intelligent assistant.

When policy makers have a large number of requirements to convert, to minimize
user labor to input requirements manually, CitySpec also provides the option
for them to input requirements through a file. The process is similar to where
CitySpec asks users to provide or clarify information until all the
requirements are successfully converted through files.

###  4.3 Requirement Synthesis

The amount of city requirement dataset is insufficient to train a decent
translation model in an end-to-end manner. As we’ve discussed in Section 1, it
requires extensive domain knowledge in both city and formal specifications and
is extremely time-consuming to annotate new requirements. Furthermore, a
majority of the existing city requirements are qualitatively or imprecisely
written, which cannot be added to the requirement dataset without refinement
[4]. To mitigate the challenge of small data to build a translation model, we
design a novel approach to incorporating city knowledge through controllable
requirement synthesis.

There are two main reasons why converting a city requirement to a formal
specification is challenging with a small amount of data. First, the
vocabulary of city requirements are very diverse. For example, requirements
from different cities (e.g., Seattle and New York City) or in different
domains (e.g., transportation and environment) have totally different
vocabulary for entities, locations, and conditions. Second, the sentence
structure (patterns) of requirements vary significantly when written by
different people. It is natural for human beings to describe the same thing
using sentences.

Targeting these two challenges, we first extract city knowledge and build two
knowledge datasets, i.e., a vocabulary set and a pattern set. The vocabulary
set includes five keys of a requirement, i.e., entity, quantifier, location,
time and condition. The pattern set includes requirement sentences with 5
keywords replaced by their labels. For instance, we have a requirement, “In
all buildings/$\mathsf{location}$, the average concentration/$\mathsf{entity}$
of TVOC/$\mathsf{quantifier}$ should be no more than 0.6
mg/m3/$\mathsf{condition}$ for every day/$\mathsf{time}$.”, the pattern
extracted is “In #$\mathsf{location}$, the average #$\mathsf{entity}$ of
#$\mathsf{quantifier}$ should be no more than #$\mathsf{condition}$ for
#$\mathsf{time}$.”

We extract the knowledge set from city documents besides requirements so that
we are not limited by the rules of requirements and enrich the knowledge of
our model. For example, we extract 336 patterns and 3061 phrases (530 phrases
in $\mathsf{entity}$, 567 phrases in $\mathsf{quantifier}$, 501 phrases in
$\mathsf{location}$, 595 phrases in $\mathsf{condition}$, and 868 phrases in
$\mathsf{time}$).

Next, we designed an approach to synthesizing controllable requirement dataset
efficiently. Intuitively, we can go through all the combinations of keywords
and patterns to create the dataset of requirements, which is infeasible and
may cause the model overfitting to the injected knowledge. In order to enhance
the model’s performance, we need to keep a balance between the coverage of
each keyword and the times of keywords being seen in the generation. We denote
$\lambda$ as the synthesis index, which indicates the _minimum_ number of
times that a keyword appears in the generated set of requirements. Assuming we
have $m$ set of keywords vocabularies $\\{V_{1},V_{2}\dots V_{m}\\}$ and a
pattern set as $P$, we have the total number of synthesized requirements
$\ell=\lambda\cdot\max(|V_{1}|,|V_{2}|,\dots,|V_{m}|)$. For each set of
vocabularies $V_{i}$, we first create a random permutation of $V_{i}$ and
repeat it until the total number of phrases reaches $\ell$, then we
concatenate them to an array $S_{i}$. Once we obtain $S_{1},...S_{m}$, we
combine them with pattern $P$ to generate a requirement set $R$. Refer to
Algorithm 1 for more details.

Algorithm 1 Requirement Synthesis

Input: $m$ set of keywords vocabularies $\\{V_{1},V_{2}\dots V_{m}\\}$,
Pattern $P$, synthesis index $\lambda$
Output: Set of requirements $R$

Initialize $R$ as an empty set$\\{\\}$

Let $\ell=\lambda\cdot\max(|V_{1}|,|V_{2}|,\dots,|V_{m}|)$

for $i\in 1\dots m$ do

Initialize $S_{i}$ as an empty array $S_{i}=[]$

while $|S_{i}|<\ell$ do

Create a random permutation of $V_{i}$: $P=\text{Permutate}(V_{i})$

Concatenate $P$ to $S_{i}$: $S_{i}=\text{Concat}(S_{i},P)$

end while

end for

for $j\in 1\dots\ell$ do

Combine keywords $S_{1}[j],S_{2}[j]\dots S_{m}[j]$ with Pattern $P$ to create
a requirement $r_{j}$

Add $r_{j}$ to the set of requirements $R$

end for

return $R$

###  4.4 Translation Model

The inputs of the translation model are requirements, and the outputs of this
module are formal specifications with token-level classification. We implement
the translation model with three major components, a learning model, knowledge
injection through synthesized requirements, and keyword refinement.

To be noted, CitySpec does not build its own translation model from scratch.
Instead, we tackle the limitation of the traditional language model and
improve it for city requirement translation. Therefore, CitySpec is compatible
with different language models.

In this paper, we implement four popular language models, which are Vanilla
Seq2Seq, Stanford NLP NER, Bidirectional Long Short Term Memory (Bi-LSTM) +
Conditional Random Field (CRF) and Bidirectional Encoder Representations from
Transformers (BERT) [26]. We apply our synthesized datasets with different
synthesis indexes to inject city knowledge into these language models. Then we
evaluate the improvement brought by our requirement synthesis approach by
testing the performance on real-world city requirements. We present the
detailed results and analysis in Section 5.

Additionally, we find that time, negation and comparison are the most tricky
elements that affect the accuracy of the final specification detection.
Therefore, we implement another refinement component in the translation model.
In general, the $\mathsf{time}$ can be represented in several formats, such as
timestamps, or other formats like yyyy-mm-dd and mm-dd-yyyy. To mitigate the
confusion that various formats might bring, we apply SUTime [27] when the
$\mathsf{time}$ entity is not given by the translation model. pyContextNLP
[28] is applied to analyze whether there is a negation in the input sentence.
If there is any negation, the comparison symbol is reversed. For instance, if
there is a keyword “greater than”, the comparison symbol is $>$. However, if
the whole phrase is “is not supposed to be greater than”, and a negation is
detected, thus the final comparison is $\leq$ instead.

###  4.5 Secured Online Learning

In general, the more clarifications are needed from the users, the worse
experience the users will have, especially if users have to clarify the same
information repeatedly. For example, if a user from a new city inputs a
location that the system fails to detect, the user will be asked to clarify
the location information. The user’s experience will drop if the system asks
him again on the second or third time seeing these words. However, the deep
learning-based translation model cannot “remember” this information at
deployment time. Thus, the first question is that how can CitySpec learn the
new knowledge online?

Meanwhile, the new information provided by users may also harm the system if
it is an incorrect or adversarial example. The second question is that how can
CitySpec validate the new knowledge before learning it permanently?

Figure 3: Secured Online Learning

Targeting these two research questions, we design an online learning module in
CitySpec. As shown in Figure 4, it has two stages, which are short-term
learning and long-term learning. Short-term learning is designed to
accommodate the same user in one session of requirement specification with a
temporary memory. The question-answer pairs are stored temporarily. When the
same occasion occurs, the temporal cache gives instant answers and avoids more
user clarifications. Long-term learning is designed to adapt the new knowledge
to the model permanently after validating its reliability. The accepted
permanent knowledge is achieved by updating weights via back-propagation on
the extended dataset with both initial data and the new input-label pairs
stored in the temporary cache.

To prevent the injection of malicious or suspicious knowledge into CitySpec,
we have implemented two sub-components to secure the online learning session:
the Shield Model and the Validation Function. The Shield Model has been
designed to segregate non-malicious inputs from malicious ones. As depicted in
Fig.4 and Fig.3, the Shield Function protects both short-term and long-term
online learning sessions by determining whether the user input is malicious.
For instance, when a user provides their location as ”within 100 meters of the
Vanderbilt campus” in the front-end interface after the Translation Model’s
output, shown in Fig.3, the Shield Model examines it from the backend and
prevents any malicious information from being included, as shown on the right
side of Fig.3. After one user clarification is passed through the Shield
Model, it is stored to cache for future references, for example, if the user
again inputs some requirement with “within 100 meters of the Vanderbilt
campus” as the location, CitySpec will directly provide the answer from the
cache. Periodically those samples passed the shield function are then passed
on to the Validation Function, which further examines whether the user has
provided the correct label-phrase pair. As shown in the last few steps in
Fig.3, the Validation Function verifies whether ”within 100 meters of the
Vanderbilt campus” should be labeled as a location. Once examined by both sub-
components, user inputs will be injected into the city knowledge and used for
learning purposes.

Figure 4: One Sample Round of Online Learning

####  4.5.1 Validation Function

In our setting, if CitySpec fails to fulfill all the predefined key domains,
it will ask the user for clarification in token-domain pairs. We implement a
BERT-based classification model as the validation function and train it on
generated city knowledge from all the existing requirements. The inputs of the
validation model are the new terms provided by the user, while the outputs are
the corresponding keys based on the new terms. To keep CitySpec away from
adversarial inputs, we develop a Bayesian CNN-based validation module in
CitySpec. The model is to classify the category of a new term with confidence
with uncertainty estimation. We apply dropout layers during both training and
testing to quantify the model uncertainty [6]. The inputs of the validation
model are the new terms provided by the user, while the outputs are the
corresponding keys among those five key elements with an uncertainty level. In
brief, a new term-key pair is rejected if (1) the output from the validation
function does not align with the given domain key; (2) the validation function
has low confidence in the output although it might align with the given
domain. In this way, we only accept new city knowledge validated with high
confidence.

####  4.5.2 Shield Model

By our design, although the validation function is supposed to serve as an
over-watch to prevent CitySpec from any suspicious new knowledge, we still
find it vulnerable to adversarial attacks because it directly takes textual
inputs from the user. If the user keeps manipulating textual inputs and by
enumeration, the user can find adversarial inputs that would confuse the
validation function, pass the validation function check, and directly poison
our city knowledge. Since the validation function is trained on the city
knowledge periodically. If the city knowledge gets poisoned, not only the
validation function but also CitySpec will start to malfunction.

For example, suppose a truck company desires to increase its revenue despite
the existence of current regulations, such as ”trucks are not allowed to enter
Charlotte Pike from 5 PM to 7 PM on weekdays”. In such a scenario, the company
may opt to engage in an attack on the system by poisoning the city knowledge,
which could involve manipulating the location or time data. By doing so, such
regulation would cease to be effectively enforced, enabling the truck company
to allocate their trucks onto Charlotte Pike during rush hours. This action,
however, poses a significant threat to public safety, given that it could lead
to traffic accidents and congestion on the road.

Thus, we need an additional shield model to guard the validation function so
that CitySpec will be hard to attack directly and robust to most adversarial
attacks. The shield model, by our design, can detect any malicious behavior
when the user clarifies the unfulfilled domain before feeding the
clarification to the validation function. Our shield model consists of two
separate filters: literal correction and inferential mapping. To test the
effectiveness of the shield model, we generate malicious attacks with 12
different kinds of generated adversarial inputs from the most recent works of
literature [29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 2].

Those investigated and implemented adversarial attacks [40] aim to mislead the
validation function based on existing city knowledge by generating new
malicious textual inputs. Although those adversarial attack types differ from
each other, we conclude two key steps during those attacks: Word Importance
Ranking $S(.)$ and Word Transformation $T(.)$. Given an existing valid input
sequence, $X$ and the victim function $F$, in general, an attacker first ranks
the sub-tokens and gets the most ‘important’ ones $x_{i,j,k,..}\in X$ based on
selected metrics. Then the attacker will transform those selected tokens into
other textual inputs $x^{{}^{\prime}}_{i,j,k,...}$ and replace the original
ones in $X$. The newly assembled sequence $X^{{}^{\prime}}$ is passed to $F$.
If $F$ changes its output, then the attack is considered successful. Thus, a
successful attack would satisfy $F(T(S(X)))\neq F(X)$. Some of those
adversarial attacks directly skip the word importance ranking process, even
though the rest of those do rank tokens, different assumptions are made and
different ranking metrics are applied. We choose to let the shield model focus
more on the word transformation during detection. In those attacks, there are
two main genres of word transformation: inner-word transformation and inter-
word transformation. Inner-word transformations change the characters within
selected tokens, e.g., change “energy consumption” to “energy consumptlon”.
Inter-word transformations change the whole word instead based on some
replacement strategy, e.g., change “available space” to “free space”. More
details can be found in Table. 2.

We develop a literal correction filter to detect and leave out any literal
disinformation provided in the input tokens. The literal correction is
designed to mainly focus on the inner-word transformation. This filter
consists of two main components: a language model and a type checker. The
language model is trained on over 1,500 existing city requirements. Given a
seed token, the language model generates the most ‘reasonable’ sequence by
maximum likelihood estimation. Based on the reference word provided by the
language model, the type checker searches for all possible variant words based
on the reference word without changing too much in spelling. The input
sequence is considered malicious if any correction is involved in the initial
inputs. E.g., the user types in “in the m0rninGs”. If the language model gives
“in the mornings” as a reference, then the type checker will try to recover
“m0rninGs” to “mornings”. If the edit distance is allowed, then the input
sequence will be corrected to “in the mornings”. As observed, the corrected
sequence is no longer the same as it was first typed in, then this sequence is
considered malicious. This Literal Correction filter is considered as hard for
direct attacks because the attacker has no access to (1) the existing 1,500
city requirements; (2) the details in the language model; or (3) the edit
distance budget in the type checker.

However, this Literal Correction filter is built upon a fixed vocabulary and
mainly focuses on word spelling and phrase composing. It ignores the
inferential understanding of a given phrase. Thus, we need another filter that
suffers less from the limited vocabulary and has more prior knowledge in
natural language inferential understanding.

We leverage the prior knowledge brought by the BERT model and develop an
inferential mapping filer. It first embeds input sequences with arbitrary
lengths into vectors with a fixed length [41]. This embedding process can also
be understood as mapping textual sequences into a numeric hyperspace. Based on
studies in [41], this hyperspace is declared to manage to put phrases with
similar inferential information closer. E.g., the euclidean distance between
“at the gates” and “at the doors” after embedding is 8.805, however, the
distance between “at the gates” and “on the campus” is 14.95. Based on this
characteristic, this inferential mapping filter can mitigate the issues
brought by inter-word transformations like word insertion. After mapping all
phrases into the hyperspace as vectors with a fixed length, we employ multi-
layer perceptrons to draw the decision boundary in the hyperspace. To make
this inferential mapping filter more robust to direct attacks, we even mask
the embedding vectors with trainable weights before passing them forward to
the downstream perceptrons. In conclusion, this filter is considered difficult
to attack because (1) all textual information is encrypted by masked phrase
embeddings before feeding to the discriminator, which means there is no strong
direct relationship between the textual inputs and their corresponding numeric
representations; (2) the attacker does not have the access to the model
details of the downstream perceptrons.

Table 2: Adversarial Attacks Overview Attack Types |  Word Importance Ranking |  Word Transformation Strategy |  Original Tokens |  Perturbed Tokens
---|---|---|---|---
A2T |  Gradient wrt. Loss Function |  Iterative Synonym Replacement |  For any source of sound …, shall not exceed the peak levels of … |  For any source of sound …, shall not exceed the peak tiers of …
BAE |  N/A |  Bert Generation |  No person shall use, operate, or maintain an alert system at … |  No person shall use, access, or maintain an alert system at …
BertAttack |  Forward Result Changes after Permutation |  Bert Generation |  No operator of a sidewalk cafe shall be assigned all the available space within … |  No operator of a sidewalk cafe shall be assigned all the available space within …
InputReduction |  Confidence Change from Prediction Dist. |  Repetitive Word Removal |  DC minimum roof reflectance: three year-aged solar reflectance of 0.55 |  DC minimum roof reflectance: three year-aged reflectance of 0.55
Pruthi’s |  N/A |  Character-level Perturbation |  All Refrigerators and … shall have maximum energy consumption less than … |  All Refrigerators and … shall have maximum energy consumptlon less than …
PSO |  N/A |  Word Replacement based on Sememes |  The location of a vending machine shall … |  The locality of a vending machine shall …
PWWS |  Word Saliency Change on Output Dist. |  Synonym Replacement |  Construction and demolition within one thousand (1,000) feet of a residential property is prohibited when … |  Building and demolition within one thousand (1,000) feet of a residential property is prohibited when …
TextBugger |  Confidence Change from Prediction Dist. |  Character-level Perturbation |  … the material composition of the pavement system layers … |  … the material composing of the pavement system layers …
TextFooler |  Forward Result Changes after Deletion |  Synonym Replacement |  No idling vehicles are allowed about or on any … schools |  No unused vehicles are allowed about or on any … schools
DeepWordBug |  LSTM-based Score |  Character-level Perturbation |  … no person shall stop , park , or leave standing any vehicle … |  … no person shall stCop, pakr, or leav standing any vehicle
CheckList |  N/A |  N/A |  N/A |  N/A
CLARE |  N/A |  Replace, Insert and Merge |  … the total gross floor area of all structures … |  … the total gross floor residential area of all structures …

##  5 Evaluation

In this section, we evaluate our CitySpec system from five aspects, including
(1) comparing different language models on the initial dataset without
synthesizing, (2) analyzing the effectiveness of the synthesized requirements
by enhancing the models with city knowledge, (3) evaluating the performance of
the online validation model, (4) testing CitySpec’s adaptability in different
cities and application domains, and (5) an overall case study. We use the city
requirement dataset described in Section 2. To evaluate the prediction of
keywords and mitigate the influence caused by different lengths requirements,
we choose to use token-level accuracy (token-acc) and sentence-level accuracy
(sent-acc) as our main metrics. The token-level accuracy aims to count the
number of key tokens that are correctly predicted. The sentence-level accuracy
counts the prediction as correct only when the whole requirement is correctly
translated to a formal specification using SaSTL. Thus, sentence-level
accuracy serves as a very strict criterion to evaluate the model performance.
We also provide the results using other common metrics including precision,
recall, and F-1 score. The experiments were run on a machine with 2.50GHz CPU,
32GB memory, and Nvidia GeForce RTX 3080Ti GPU.

###  5.1 Performance of language models on the initial dataset

Table 3: Performance of language models on the initial dataset

Model | Token-Acc | Sent-Acc | F-1 Score | Precision | Recall
---|---|---|---|---|---
Vanilla Seq2Seq |  10.91 $\pm$ 0.57 % |  1.38 $\pm$ 0.49% |  24.12 $\pm$ 0.24 % |  65.58 $\pm$ 7.95 % |  14.81 $\pm$ 1.58 %
BiLSTM + CRF |  77.59 $\pm$ 0.52 % |  60.82 $\pm$ 1.22 % |  80.46 $\pm$ 0.84 % |  81.11 $\pm$ 1.38 % |  79.83 $\pm$ 7.24 %
BERT |  80.41 $\pm$ 0.07 % |  59.02 $\pm$ 0.42 % |  81.43 $\pm$ 0.01 % |  78.62 $\pm$ 0.01 % |  84.46 $\pm$ 0.01 %

As a baseline of translation model without city knowledge, we first evaluate
the performance of CitySpec using different language models, including Vanilla
Seq2Seq, pretrained Stanford NER Tagger, Bi-LSTM + CRF, and BERT on the
initial dataset. We present the results in Table 3.

We make the following observations from the results. First, the overlap
between Stanford Pretrained NER Tagger prediction and vocabulary is only 9 out
of 729. The pretrained tagger tends to give locations in higher granularity.
Since this task is a city-level, more detailed location information is stated
in a lower granularity by providing the street name, building name, or
community name. The $\mathsf{location}$ domain in the pretrained tagger gives
more high-level information like city name, state name, or country name. For
example, “34th Ave in Nashville, the state of Tennessee” is annotated as
$\mathsf{location}$ in this task, however, the pretrained NER tagger gives
“Tennessee” as $\mathsf{location}$ instead.

Secondly, the testing token-acc from Vanilla Seq2Seq is 10.91% on average.
Other metrics also indicate that Vanilla Seq2Seq has trouble recognizing the
patterns in sequential keyword labeling. The Vanilla Seq2Seq model suffers
from data scarcity and has difficulty recognizing the general patterns in the
training samples due to the small size of the dataset.

Thirdly, the Bi-LSTM + CRF and BERT model achieve better performance than
other models, and BERT models often outperform other models with lower
standard deviation. However, the highest token-level accuracy achieved is
80.41%, which is still not high enough for an accuracy-prioritized task. A
different key may change the requirement entirely. For example, the “width” of
“car windshield” and the “width” of “car” focus on completely different
aspects, although the keywords “car windshield” and “car” have an only one-
word discrepancy. Meanwhile, the best sentence-level accuracy achieved is
60.82%, which means that about 40% of the requirements are falsely translated.
Assuming the policy makers fix these requirements through the intelligent
assistant interface, it is time-consuming and reduces the user experience.
Even worse, it may bring safety issues to the monitoring system without
noticing.

In summary, the results indicate that existing language models are not
sufficient to serve as the translation model for CitySpec directly. There is a
high demand for injecting city knowledge to build the translation model.

###  5.2 Requirement Synthesis with City Knowledge

Figure 5: Performance improvement brought by requirement synthesis

Next, we evaluate CitySpec’s performance with our controllable synthesized
requirements. For a fair comparison, we do not use the requirements in the
testing set to synthesize requirements. We ensure that the trained model has
not seen the requirements in the testing set in either the knowledge injection
or training phases. We apply different synthesis indexes to test the effects
on the prediction performance. We present the overall results on token-level
and sentence-level accuracy in Figure 5, and F-1 scores on individual keyword
in Figure 6. In the figures, x-axis represents the synthesis index. When the
index equals to “inital”, it shows the results without synthesis data.

From the results, we find that, for BERT and Bi-LSTM, there is an overall
increase in performance in all token-level accuracy, sentence-level accuracy,
overall F-1 score, and F-1 score on keywords. For example, BiLSTM+CRF’s token-
level accuracy increases from 77.59% to 97% and sentence-level accuracy
increases from 60.82% to 81.3%, BERT’s sentence-level accuracy increases from
59.02% to 86.64%.

In summary, the results show that injecting city knowledge with synthesized
requirements boosts the translation model significantly. While improving
policy maker’s user experience with higher accuracy and less clarifications,
it also enhances the safety of the monitoring system potentially.

###  5.3 Performance on Online Validation

We evaluate the validation model through simulating four different testing
scenarios: (I) randomly generated malicious input based on the permutation of
letters and symbols; (II) all street names in Nashville; (III) real city
vocabulary generated from Nashville requirements; (IV) generated float numbers
with different units.

First of all, the accuracy of validation model is very high. When the
uncertainty threshold is set to 0.5, i.e., all inputs cause an uncertainty
higher than 0.5 will be ruled out, CitySpec gives 100% success rate against
scenario I among 2,000 malicious inputs, 91.40% acceptance rate among 2,107
samples in scenario II, 92.12% acceptance rate among 596 samples in scenario
III, and 94.51% acceptance rate among 2,040 samples in scenario IV.
Additionally, we find that the validation function easily confuses
$\mathsf{entity}$ with $\mathsf{quantifier}$ if no further guidance is
offered. We look into dataset and figure out $\mathsf{entity}$ and
$\mathsf{quantifier}$ are confusing to even humans without any context
information. Take the requirement “In all buildings, the average concentration
of Sulfur dioxide (SO2) should be no more than 0.15 mg/m3 for every day.” As
an example, $\mathsf{entity}$ is “concentration” and $\mathsf{quantifier}$ is
“Sulfur dioxide (SO2)”. If the requirement is changed to “The maximum level of
the concentration of Sulfur dioxide (SO2) should be no more than 0.15 mg / m3
for every day.”, then $\mathsf{entity}$ is “maximum level” and
$\mathsf{quantifier}$ is “concentration” instead. In addition, terms like
“occupancy of a shopping mall”, “noise level at a shopping mall”, and “the
shopping mall of the commercial district” also introduce confusion between
$\mathsf{location}$, $\mathsf{entity}$ and $\mathsf{quantifier}$, since the
same token “shopping mall” can be $\mathsf{entity}$, $\mathsf{quantifier}$ or
$\mathsf{location}$ in certain cases.

The results show that the validation algorithm can effectively accept new city
knowledge, prevent adversarial inputs and safeguard online learning.
Therefore, CitySpec reduces unnecessary interactions between policy makers and
the system and increases efficiency.

Figure 6: F-1 scores on four keywords

###  5.4 Adaptability to different scenarios

Table 4: Adaptability on different cities in terms of token-level accuracy, sent-level accuracy, and overall F-1 score City | Seattle | Changsha
---|---|---
Metrics | ToeknAcc | SentAcc | F-1 | ToeknAcc | SentAcc | F-1
| Non-adaptive w/
---
BiLSTM+CRF
84.91% | 48.00% | 77.60% | 86.61% | 61.20% | 84.10%
| Adaptive w/
---
BiLSTM+CRF
96.05% | 84.80% | 93.75% | 95.27% | 83.20% | 93.57%
| Non-adaptive w/
---
BERT
80.38% | 46.40% | 76.80% | 86.10% | 58.00% | 83.87%
| Adaptive w/
---
BERT
95.10% | 80.70% | 90.28% | 97.16% | 88.40% | 96.88%
City | Charlottesville | Jacksonville
Metrics | ToeknAcc | SentAcc | F-1 | ToeknAcc | SentAcc | F-1
| Non-adaptive w/
---
BiLSTM+CRF
90.00% | 65.62% | 86.48% | 77.32% | 35.20% | 81.54%
| Adaptive w/
---
BiLSTM+CRF
96.82% | 89.29% | 95.40% | 97.35% | 88.80% | 96.02%
| Non-adaptive w/
---
BERT
93.44% | 73.21% | 90.46% | 90.21% | 56.40% | 81.60%
| Adaptive w/
---
BERT
97.53% | 87.05% | 94.31% | 96.27% | 83.20% | 92.59%
Table 5: Adaptability on different topics in terms of token-level accuracy, sent-level accuracy, and overall F-1 score Topic | Noise Control | Public Access
---|---|---
Metrics | ToeknAcc | SentAcc | F-1 | ToeknAcc | SentAcc | F-1
| Non-adaptive w/
---
BiLSTM+CRF
77.82% | 41.56% | 77.83% | 73.99% | 44.07% | 74.41%
| Adaptive w/
---
BiLSTM+CRF
95.82% | 90.68% | 94.46% | 97.68% | 74.80% | 97.39%
| Non-adaptive w/
---
BERT
84.15% | 58.62% | 81.05% | 83.59% | 54.17% | 78.93%
| Adaptive w/
---
BERT
98.07% | 88.31% | 92.98% | 97.43% | 88.75% | 95.75%
Topic | Indoor Air Control | Security
Metrics | ToeknAcc | SentAcc | F-1 | ToeknAcc | SentAcc | F-1
| Non-adaptive w/
---
BiLSTM+CRF
81.51% | 46.80% | 76.22% | 72.11% | 28.80% | 62.93%
| Adaptive w/
---
BiLSTM+CRF
95.58% | 80.00% | 87.68% | 94.39% | 94.37% | 92.34%
| Non-adaptive w/
---
BERT
78.51% | 31.20% | 73.88% | 79.31% | 45.60% | 77.50%
| Adaptive w/
---
BERT
95.31% | 74.40% | 93.60% | 95.41% | 82.80% | 93.95%

In this section, we analyze CitySpec’s adaptability in different cities and
different domains. Different cities have different regulation focuses and
their city-specific vocabulary. For example, in the city of Nashville,
$\mathsf{location}$ names like “Music Row”, “Grand Ole Opry” will probably
never appear in any other cities. We select four cities, Seattle,
Charlottesville, Jacksonville, and Changsha, with different sizes and from
different countries as case studies. We separate the requirements of each
mentioned city and extract the city-wise vocabulary based on each city
independently. Each of four constructed pairs consists of: vocabulary I, which
is extracted from the requirements from one city only, and vocabulary II,
which is extracted from the requirements from all the cities but that specific
one city. Injected knowledge is measured using the number along with the ratio
of how much of the unique vocabulary one city causes. We augment vocabulary II
using 5 as the synthesis index and train a model on vocabulary II. As a
result, the trained model is isolated from the vocabulary information from
that one specific city. Afterward, we test the trained model performance on
the generated requirements using vocabulary I. We pick CitySpec with Bi-LSTM +
CRF and CitySpec with BERT in this scenario. We employ the validation function
to validate all vocab in vocabulary I and pass the validated ones to
vocabulary II. After that, we have a validated vocabulary including vocabulary
II and validated vocabulary I. The deployed model is fine-tuned based on the
validated vocabulary using few-shot learning.

From the results shown in Table 4, we observe that (1) although CitySpec
immigrates to a completely unknown city, it is still able to provide
satisfying performance, e.g., 84.9% token-acc and 77.6% F-1 score in Seattle,
but the sent-acc tends to be low. (2) With new knowledge injected, the
performance increases significantly, e.g., Sent-Acc for Seattle increases from
48% to 84.8% with BiLSTM+CRF, and from 46.4% to 80.7% with BERT.

We further study the practical impacts after having such sent-acc and token-
acc. From its definition, when a sent-acc of 84.80% is achieved, which is the
adapted performance of BiLSTM + CRF from the Seattle experiment, it reflects
that among all requirements, 84.80% of them are predicted entirely correctly
for both in-domain and out-domain tokens. For example, given “For all the
zones in Civil Building Engineering Group II , retail sale should be less than
10 dB(A) at any time.” as input, the translation model produces output that
can be understood as domain-keyword tuples. In the given requirement, there
are words marked as keywords that belong to a predefined category, for
example, “10 dB(A)” is $\mathsf{number}$. There are also some other words do
not belong to any category, for example, “should be”. To add this requirement
to the set of correct predictions using sent-acc, the model needs to predict
all tokens into the correct category for both words, no matter whether they
belong to any category or not. However, token-acc performs differently from
sent-acc since it counts the ratio of correct prediction of all the tokens.
Thus, sent-acc is a stricter metric compared to token-acc.

We study the performance of the translation model by checking its incorrect
predictions. We conclude there are two main situations when the translation
model makes mistakes so that the sent-acc would be affected. Situation I:
nuance differences that will not affect the overall understanding. Situation
II: significant differences caused by the translation model’s failure to
understand the requirement correctly. Take the requirement “The sound level in
the residential district should always be less than 1,000 feet.” for example,
if the model gives “$\mathsf{quantifier}$: residential district” as one of the
predictions while the true label is “$\mathsf{quantifier}$: the residential
district”, then this prediction sequence will not be considered correct in
terms of sent-acc although this kind of nuance difference does not affect the
overall understanding. In another situation where the model gives
“$\mathsf{quantifier}$: district” instead of “$\mathsf{number}$: the
residential district”, the model predictions should not be adopted. To
mitigate the problems brought by these two situations, we rely on the user to
validate and modify the prediction of the model if necessary, interactively
during the conversation (the results will be applied if it goes through the
shield model check). There is also a window on our interface that reports a
summary of the generated specification but in a more readable format [42].

We also explore CitySpec’s adaptability to different topics. We choose four
topics including noise control, indoor air control, security, and public
access. The results also show that (1) even though CitySpec has not seen
vocabulary from a totally different topic, it still gives a competitive
performance; (2) online learning brings obvious improvements when adaptation
is further applied.

In summary, it indicates the capability of CitySpec in both city and domain
adaptation. It can also adapt to new requirements evolving overtime. Moreover,
with a different set of domain-specified knowledge, CitySpec can be
potentially applied to other application domains (e.g., healthcare).

###  5.5 Performance of Shield Model

We first launch 12 different adversarial attacks targeting the validation
function without any protection from the shield. Only a limited amount of city
knowledge is exposed to these attackers for adversarial sample generation. For
those attacks that are not in a black-box fashion, we allow them to query the
validation function arbitrarily. Because in practical usage, we assume the
partial city knowledge has been exposed to the user so that the user could
understand and compose city requirements based on that. And since the
conversation assistance system is developed in a slot-filling fashion, it will
continue for good if not all required slots are fulfilled by the user, so the
model query budget is set to infinity in this case. After these attacks, we
employ the shield model to defend against those adversarial samples with three
different settings: (1) Literal Correction layer alone; (2) Inferential
Mapping layer alone; (3) Inferential Mapping layer with Literal Correction
layer as a pre-check. The defense rate is measured as the number of filtered-
out adversarial samples divided by the number of remaining adversarial
samples.

As we can tell from the first column in Table 6, the validation function is
considered as vulnerable to 10 out of those 12 attacks without protection from
the shield model, for instance, BertAttack yields 94.00% as the attack success
rate. Thus there is an urgent demand for a reliable protection mechanism
deployed against these adversarial attacks. In setting one, although the
Literal Correction layer alone gives around 63.11% defense success rate on
average over 12 adversarial attacks, we still find there are several
shortcomings in the Literal Correction layer. After error analysis, the
Literal Correction filter does not work as expected when (1) the language
model fails to give any reference based on the perturbed word, which means the
perturbed word never appeared in the existing requirements; (2) the type
checker fails to correct the word in a certain amount of edits. E.g., the user
inputs “in the tr1anGl3s”. If the language model fails to give “triangles” as
a reference, which means the word “triangle” never appears in the
requirements, then the type checker will no longer work. Even if the language
model generates “in the triangles” as a reference when the edit distance budge
is set to 2, then the type checker will fail to convert “tr1anGl3s” to
“triangles”.

In setting two, although the Inferential Mapping layer yields around 93.09% as
a defense rate, we still find it having trouble gaining a low false-positive
rate when faced with non-malicious samples. We pass all legal input samples in
our city knowledge to the Inferential Mapping layer and find it reports 31.32%
as a false-positive rate when faced with phrases categorized in
$\mathsf{entity}$. We want the shield model to filter out as many malicious
samples as possible while as few legal samples as possible. After error
analysis, we find the Inferential Mapping layer ignores typos in phrases due
to its focus on semantic understanding. For example, if the malicious input is
“in the morinigns” and there is “in the mornings” marked as non-malicious, the
Inferential Mapping layer will be confused since the euclidean distance
between these two phrases is only 6.21. In other words, if there is only a
small typo in input samples, the Inferential Mapping layer will ignore it and
treat them similarly to ones without typos. In consequence, the false-positive
rate is increased.

In setting three, the Literal Correction layer helps the Inferential Mapping
layer filter out typos first. At the same time, the Inferential Mapping layer
leverages its advantages in natural language understanding to identify samples
that the Literal Correction layer fails to handle. The overall defense rate is
reported to be around 97.78% and even yields a 100% defense rate when faced
with adversarial attacks like A2T, Pruthi’s algorithm, and DeepWordBug. After
testing this combination of layers on legal inputs, the false-positive rate is
3.09% on average over the five domains.

In summary, the shield model is necessary for CitySpec because it helps the
validation function guard the city knowledge against being poisoned. The
combination of both layers as the shield model is effective when faced with 12
adversarial attacks without overkilling non-malicious samples.

###  5.6 Emulation

Due to the absence of a real city policy maker, we emulate the process of
using CitySpec by taking the real-world city requirements and assuming that
they are input by policy makers. Specifically, this case study shows the
iteration of communication between CitySpec and the policy maker to clarify
the requirements. We emulate this process 20 times with 100 requirements
randomly selected from our datasets each time. The results show that the
average and maximum rounds of clarification are 0.8 and 4 per requirement,
respectively, due to the missing or ambiguous information. Averagely, 28.35%
of requirements require clarification on location. For example, “No vendor
should vend after midnight.”, CitySpec asks users to clarify the time range
for “after midnight” and the location defined for this requirement. Overall,
CitySpec obtains an average sentence-level accuracy of 90.60% (with BERT and
synthesize index = 5). The case study further proves the effectiveness of
CitySpec in city requirement specification.

Table 6: Shield Defense Rate Against Adversarial Attacks (SR: Success Rate) Attack Types | Attack SR |  | Attack SR after
---
Literal Correction
| Attack SR after
---
Inferential Mapping
| Attack SR after
---
both layers
A2T | 9.11% | 3.12% | 0.72% | 0%
BAE | 8.63% | 3.11% | 1.52% | 0.25%
BertAttack | 94.00% | 61.39% | 7.91% | 4.31%
InputReduction | 0% | N/A | N/A | N/A
Pruthi’s | 85.85% | 0.96% | 0.96% | 0%
PSO | 69.78% | 30.63% | 3.36% | 1.92%
PWWS | 73.14% | 34.05% | 3.59% | 1.92%
TextBugger | 21.58% | 11.12% | 0.48% | 0.48%
TextFooler | 90.17% | 39.81% | 5.04% | 2.39%
DeepWordBug | 82.73% | 1.43% | 1.19% | 0%
CheckList | 0% | N/A | N/A | N/A
CLARE | 43.14% | 18.50% | 6.91% | 2.38%

##  6 Case Study

To demonstrate the effectiveness of CitySpec, we also launch a real user case
study on 18 participants with different backgrounds: 7 participants with a
Computer Science-related background, 3 with Education-related background, and
2 with Finance-related background. In addition, We have 1 participant for each
domain among Biology, Environmental Engineering, Maths, Law, and Art. The goal
of this real user case study targets to test the performance of CitySpec (1)
while helping the user complete requirements in smart city scenarios; (2)
while helping the user complete requirements in an unseen domain; (3) in
telling malicious inputs while conveying service; (4) while handling unseen
knowledge with the help from its online learning features.

###  6.1 CitySpec in Smart City Scenarios

To test the usability and adaptability of CitySpec in smart city, We ask the
participants to read and type in 8 randomly selected requirements sequentially
to CitySpec, then record the time and the number of interactions needed to
complete the requirement specification along with subjective user scores. We
find the number of interactions is highly related to the input requirement
itself, so instead of directly using the number of interactions needed to
measure the usability of CitySpec, we, as authors of this work, provide a
reference number of interactions of each requirement. We take the difference
between the number of user interactions and the number of reference
interactions as a measurement of the usability of CitySpec.

As Figure 7 shows, there is an overall decrease in the additional interactions
per requirement. This decrease demonstrates that our participants get almost
as familiar as us with CitySpec. The slight increases at Req3, Req5, and Req7
are caused by the differences in participants’ subjective understanding
regarding the definition of key elements (e.g., $\mathsf{entity}$ or
$\mathsf{description}$). Figure 7 also shows the time consumed per interaction
is decreasing as our participants use CitySpec. This decreasing trend
indicates the participants are getting more familiar with those key elements
with the help of CitySpec.

Figure 7: CitySpec in smart city (case study)

###  6.2 CitySpec in Unseen Domains

To test the usability and adaptability of CitySpec in unseen domains, next, we
ask the participants to compose three requirements regarding their own major
and occupation. We record the average number of total interactions per
requirement, the average user score, and the average time consumed per
requirement. The results are shown in Figure 8. Among those provided
requirements, CitySpec is found to work effectively in domains that are
related to city regulation topics. For example, in the requirements from
domains like Environmental Engineering, phrases like “PM 2.5” and “iron
concentration” are passed to CitySpec. Although they are from a completely
different area compared with city requirements, CitySpec still manages to
correctly tell them from the requirements due to the partial similarity
between city and environmental engineering.

Figure 8: CitySpec in unseen domains (case study)

Although areas like medical, arts, and engineering are unseen during the
training of CitySpec, CitySpec still offers correct results most times. For
example, without learning “acoustic reflexes” as an entity during training,
CitySpec presents the correct classification only based on the limited
syntactic information given in the requirement. CitySpec’s online learning
feature also helps the specification. One participant whose major is
Electronic Engineering composes requirements regarding system-on-chip/SoC and
the circuit. In the beginning, CitySpec fails to tell terms like “SoC” and
“Circuit” due to its unfamiliarity with them, however, after only a few turns
of validated user clarification, CitySpec memorizes these unseen knowledge
entries and gives correct classification in similar future cases. However, in
areas like Law, the participant emphasizes the internal relationships between
components and the ‘absolute’ correct classification, e.g., including the
determinator in the final prediction. Thus, the participant needs to
continuously interact with CitySpec to generate the correct specification,
which results in a relatively lower user score.

###  6.3 CitySpec’s Defense Against Malicious Input

To test the effectiveness of the newly introduced shield function, we ask the
participants to provide both malicious and non-malicious inputs during
CitySpec CitySpec’s online learning in smart city. We collect 178 unique user
clarification entries in total. Among those entries, 147 are considered
malicious and 25 are considered non-malicious by the participants. With the
help of our newly deployed shield function, CitySpec rejects 116 of those
malicious inputs and yields a 0% false-positive rate when dealing with non-
malicious ones. Among those intentionally designed malicious inputs, two genre
draws our attention: phonetic attacks and quantitative attacks. Here we define
phonetic attacks as replacing words in given legit entries with their
homophones but keeping the semantics shifted within a trustful range. Among
all 11 phonetic attacks, 9 of them succeed. For example, the shield function
fails to abort user clarification when “fifteen” is changed to “fifty teen”,
or “plane ticket” is changed to “plain ticket”. Another genre is quantitative
attacks, we define it as granting an unpractical quantity to components based
on common senses, for example, assigning “7k dB” to “the noise level at any
personal property”. Among 3 quantitative attacks, 3 of them succeed. Besides
these two genres, CitySpec with the shield function reports 87.23% as the
defense success rate against all those malicious inputs.

In summary, according to our real user case study, CitySpec works effectively
and efficiently not only in smart city but also in most unseen areas like
Environmental Engineering, Medical, and Arts, which also proves the potential
future application of CitySpec in other areas. The decrease in both additional
interactions per requirement and time consumed per requirement indicates the
strong usability of CitySpec to users. The newly deployed shield function
helps CitySpec be more robust to most malicious inputs.

##  7 Related Work

Translation Models. Researchers have developed models to translate the natural
language to machine languages in various applications, such as Bash commands
[7], Seq2SQL [8], and Python codes [9]. These translation models benefit from
enormous datasets. The codex was trained on a 159 GB dataset that contains
over 100 billion tokens. WikiSQL, which Seq2SQL was trained on, consists of
80,654 pairs of English-SQL conversions. NL2Bash [7] was trained on
approximately 10,000 pairs of natural language tasks and their corresponding
bash commands. As an under-exploited area, there is a very limited number of
well-defined requirements. Therefore, existing translation models do not apply
to our task. This paper develops a data synthesis-based approach to build the
translation model.

Data Synthesis. Data synthesis exploits the patterns in study findings and
synthesizes variations based on those patterns. Data augmentation is a simple
application of data synthesis. Previous augmentation approaches wield tricks
like synonym substitution [43, 44] and blended approaches [45]. In the smart
city scenario, we need new data samples which fit in the smart city context.
Therefore, we extract extra knowledge from smart cities and fully exploit
semantic and syntactic patterns instead of applying straightforward tricks
like chopping, rotating, or zooming. This paper is the first work synthesizing
smart-city-specific requirements to the best of our knowledge.

Online Learning. Online machine learning mainly deals with the situation when
data comes available to the machine learning model sequentially after being
deployed. Similar to continuous learning, online learning aims to give model
accumulated knowledge and improve model performance continuously given
incoming learning samples [46, 47]. Some of the existing papers focus on
developing sophisticated optimization algorithms [48] or exploiting the
differences between new and old samples [49]. However, these papers do not
have a mechanism to detect or prevent adversarial samples online. This paper
develops a two-stage online learning process with online validation against
potential malicious inputs.

Protection against Adversarial Attacks. Works have been done to enhance the
safety of NLP models based on textual adversarial attacks. However, some of
them make observations that pre-trained language models like BERT or GPT are
robust to adversarial perturbations. For example, Bert-Defense [50] uses Bert
and GPT to lower the confusion and increase the fluency in generated samples;
[51] uses pre-trained transformers to distinguish out-of-distribution samples.
However, these pre-trained language models are trained on large corpora.
Without a specific focus, those pre-trained models only offer limited prior
knowledge in detecting malicious attacks in smart city. Other tricks like
resampling are also widely applied and reported to succeed in ensuring system
security [52]. These methods still feed textual inputs to the backend model.
Textual inputs give the attacker more room to tune with the input in our
interactive and always-learning system. This paper develops a novel shield
model that not only takes the prior knowledge from pre-trained models as
references but also enriches the prior knowledge by introducing existing city
requirements. No textual input is directly input to the shield model, instead,
it encrypts the inputs so that it is considered immune to direct adversarial
textual generation.

##  8 Summary

This paper builds an intelligent assistant system, CitySpec, for requirement
specification in smart cities. CitySpec bridges the gaps between city policy
makers and the monitoring systems. It incorporates city knowledge into the
requirement translation model and adapts to new cities and application domains
through online validation and learning. The evaluation results on real-world
city requirement datasets show that CitySpec is able to support policy makers
accurately writing and refining their requirements and outperforms the
baseline approaches. In future work, we plan to have CitySpec used by real
city policy makers, but this is outside the scope of this paper.

## Acknowledgment

This work was funded, in part, by NSF CNS-1952096.

## References

  * [1] Z. Chen, I. Li, H. Zhang, S. Preum, J. A. Stankovic, M. Ma, Cityspec: An intelligent assistant system for requirement specification in smart cities, in: 2022 IEEE International Conference on Smart Computing (SMARTCOMP), 2022, pp. 32–39.  doi:10.1109/SMARTCOMP55677.2022.00020.
  * [2] J. Gao, J. Lanchantin, M. L. Soffa, Y. Qi, Black-box generation of adversarial text sequences to evade deep learning classifiers, in: 2018 IEEE Security and Privacy Workshops (SPW), IEEE, 2018, pp. 50–56.
  * [3] M. Ma, J. A. Stankovic, L. Feng, Toward formal methods for smart cities, Computer 54 (9) (2021) 39–48.
  * [4] M. Ma, E. Bartocci, E. Lifland, J. A. Stankovic, L. Feng, A novel spatial–temporal specification-based monitoring system for smart cities, IEEE Internet of Things Journal 8 (15) (2021) 11793–11806.
  * [5] M. Ma, J. A. Stankovic, L. Feng, Cityresolver: a decision support system for conflict resolution in smart cities, in: 2018 ACM/IEEE 9th International Conference on Cyber-Physical Systems (ICCPS), IEEE, 2018, pp. 55–64.
  * [6] M. Ma, J. Stankovic, E. Bartocci, L. Feng, Predictive monitoring with logic-calibrated uncertainty for cyber-physical systems, ACM Transactions on Embedded Computing Systems (TECS) 20 (5s) (2021) 1–25.
  * [7] Q. Fu, Z. Teng, J. White, D. C. Schmidt, A transformer-based approach for translating natural language to bash commands, in: 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA), IEEE, 2021, pp. 1245–1248.
  * [8] V. Zhong, C. Xiong, R. Socher, Seq2sql: Generating structured queries from natural language using reinforcement learning, arXiv preprint arXiv:1709.00103 (2017).
  * [9] M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al., Evaluating large language modelsdamerau1964technique trained on code, arXiv preprint arXiv:2107.03374 (2021).
  * [10] NYC.gov, Emissions from transportation, nyc environment protection, 2019.
  * [11] S. Matteo, J. Brannan, A local law to amend the administrative code of the city of new york, in relation to restricting the use of bus lanes by sight-seeing buses, in: Restricting the use of bus lanes by sight-seeing buses, The New York City Council, 2019.
  * [12] NYC Environment Protection, Use of heating oil remaining in tanks, The city of New York, 2019.
  * [13] United States Environmental Protection Agency, Residential energy efficiency, in: Energy Resources for State and Local Governments, The city of New York, 2019.
  * [14] San Francisco, San Francisco noise pollution construction, in: San Francisco American Legal Publishing, 2019.
  * [15] Hong Kong, Guide to indoor air quality management in hong kong regional offices and public places, in: Guide to Indoor Air Quality Management, 2019.
  * [16] NYC.gov, Stopping, standing or parking prohibited in specified places, in: New York Public Law, 2016.
  * [17] Beijing Emergency Agency, Pre-hospital medical emergency regulations, 2016\.
  * [18] Beijing Government, Safety management for kindergarten, primary and secondary school, 2016.
  * [19] District of Columbia Municipal Regulations, D. of Columbia Register, Air quality - motor vehicular pollutants, lead, odors, and nuisance pollutants, 2016\.
  * [20] LA Sec 111.03. Minimum Ambient Noise Level, Official city of los angeles municipal code, 2016.
  * [21] DC.gov, Notice of draft title V permit public comment period – architect of the capitol, library buildings and grounds jurisdiction, library of congress, 2019\.
  * [22] tn.gov, Code of ordinances, metro government of nashville and davidson county, tn, 2022.
  * [23] tn.gov, Code of ordinances, city of memphis, tennessee, 2022.
  * [24] ga.gov, Code of ordinances, city of atlanta, georgia, 2023.
  * [25] F. J. Damerau, A technique for computer detection and correction of spelling errors, Communications of the ACM 7 (3) (1964) 171–176.
  * [26] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv:1810.04805 (2018).
  * [27] A. X. Chang, C. D. Manning, Sutime: A library for recognizing and normalizing time expressions., in: Lrec, Vol. 3735, 2012, p. 3740.
  * [28] B. E. Chapman, S. Lee, H. P. Kang, W. W. Chapman, Document-level classification of ct pulmonary angiography reports based on an extension of the context algorithm, Journal of biomedical informatics 44 (5) (2011) 728–737.
  * [29] J. Y. Yoo, Y. Qi, Towards improving adversarial training of NLP models, in: Findings of the Association for Computational Linguistics: EMNLP 2021, Association for Computational Linguistics, Punta Cana, Dominican Republic, 2021, pp. 945–956.  doi:10.18653/v1/2021.findings-emnlp.81.
URL https://aclanthology.org/2021.findings-emnlp.81

  * [30] S. Garg, G. Ramakrishnan, BAE: BERT-based adversarial examples for text classification, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Association for Computational Linguistics, Online, 2020, pp. 6174–6181.  doi:10.18653/v1/2020.emnlp-main.498.
URL https://aclanthology.org/2020.emnlp-main.498

  * [31] L. Li, R. Ma, Q. Guo, X. Xue, X. Qiu, BERT-ATTACK: Adversarial attack against BERT using BERT, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Association for Computational Linguistics, Online, 2020, pp. 6193–6202.  doi:10.18653/v1/2020.emnlp-main.500.
URL https://aclanthology.org/2020.emnlp-main.500

  * [32] S. Feng, E. Wallace, A. Grissom II, M. Iyyer, P. Rodriguez, J. Boyd-Graber, Pathologies of neural models make interpretations difficult, in: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Brussels, Belgium, 2018, pp. 3719–3728.  doi:10.18653/v1/D18-1407.
URL https://aclanthology.org/D18-1407

  * [33] D. Pruthi, B. Dhingra, Z. C. Lipton, Combating adversarial misspellings with robust word recognition, in: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Florence, Italy, 2019, pp. 5582–5591.  doi:10.18653/v1/P19-1561.
URL https://aclanthology.org/P19-1561

  * [34] Y. Zang, F. Qi, C. Yang, Z. Liu, M. Zhang, Q. Liu, M. Sun, Word-level textual adversarial attacking as combinatorial optimization, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Online, 2020, pp. 6066–6080.  doi:10.18653/v1/2020.acl-main.540.
URL https://aclanthology.org/2020.acl-main.540

  * [35] S. Ren, Y. Deng, K. He, W. Che, Generating natural language adversarial examples through probability weighted word saliency, in: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Florence, Italy, 2019, pp. 1085–1097.  doi:10.18653/v1/P19-1103.
URL https://aclanthology.org/P19-1103

  * [36] J. Li, S. Ji, T. Du, B. Li, T. Wang, Textbugger: Generating adversarial text against real-world applications, arXiv preprint arXiv:1812.05271 (2018).
  * [37] D. Jin, Z. Jin, J. T. Zhou, P. Szolovits, Is bert really robust? a strong baseline for natural language attack on text classification and entailment, in: Proceedings of the AAAI conference on artificial intelligence, Vol. 34, 2020, pp. 8018–8025.
  * [38] M. T. Ribeiro, T. Wu, C. Guestrin, S. Singh, Beyond accuracy: Behavioral testing of NLP models with CheckList, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Online, 2020, pp. 4902–4912.  doi:10.18653/v1/2020.acl-main.442.
URL https://aclanthology.org/2020.acl-main.442

  * [39] D. Li, Y. Zhang, H. Peng, L. Chen, C. Brockett, M.-T. Sun, B. Dolan, Contextualized perturbation for textual adversarial attack, in: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Association for Computational Linguistics, Online, 2021, pp. 5053–5069.  doi:10.18653/v1/2021.naacl-main.400.
URL https://aclanthology.org/2021.naacl-main.400

  * [40] J. Morris, E. Lifland, J. Y. Yoo, J. Grigsby, D. Jin, Y. Qi, Textattack: A framework for adversarial attacks, data augmentation, and adversarial training in nlp, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 2020, pp. 119–126.
  * [41] S. Wang, L. Thompson, M. Iyyer, Phrase-bert: Improved phrase embeddings from bert with an application to corpus exploration, arXiv preprint arXiv:2109.06304 (2021).
  * [42] Z. Chen, I. Li, H. Zhang, S. Preurn, J. A. Stankovic, M. Ma, An intelligent assistant for converting city requirements to formal specification, in: 2022 IEEE International Conference on Smart Computing (SMARTCOMP), 2022, pp. 174–176.  doi:10.1109/SMARTCOMP55677.2022.00043.
  * [43] S. Kobayashi, Contextual augmentation: Data augmentation by words with paradigmatic relations, arXiv preprint arXiv:1805.06201 (2018).
  * [44] X. Zhang, J. Zhao, Y. LeCun, Character-level convolutional networks for text classification, Advances in neural information processing systems 28 (2015) 649–657.
  * [45] J. Wei, K. Zou, Eda: Easy data augmentation techniques for boosting performance on text classification tasks, arXiv preprint arXiv:1901.11196 (2019).
  * [46] G. I. Parisi, R. Kemker, J. L. Part, C. Kanan, S. Wermter, Continual lifelong learning with neural networks: A review, Neural Networks 113 (2019) 54–71.
  * [47] Z. Chen, B. Liu, Lifelong machine learning, Synthesis Lectures on Artificial Intelligence and Machine Learning 12 (3) (2018) 1–207.
  * [48] E. Hazan, et al., Introduction to online convex optimization, Foundations and Trends® in Optimization 2 (3-4) (2016) 157–325.
  * [49] R. S. Sutton, A. G. Barto, Reinforcement learning: An introduction, MIT press, 2018\.
  * [50] Y. Keller, J. Mackensen, S. Eger, BERT-defense: A probabilistic model based on BERT to combat cognitively inspired orthographic adversarial attacks, in: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, Association for Computational Linguistics, Online, 2021, pp. 1616–1629.  doi:10.18653/v1/2021.findings-acl.141.
URL https://aclanthology.org/2021.findings-acl.141

  * [51] D. Hendrycks, X. Liu, E. Wallace, A. Dziedzic, R. Krishnan, D. Song, Pretrained transformers improve out-of-distribution robustness, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Online, 2020, pp. 2744–2751.  doi:10.18653/v1/2020.acl-main.244.
URL https://aclanthology.org/2020.acl-main.244

  * [52] J. Rusert, P. Srinivasan, Don’t sweat the small stuff, classify the rest: Sample shielding to protect text classifiers against adversarial attacks, in: Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Association for Computational Linguistics, Seattle, United States, 2022, pp. 2716–2725.  doi:10.18653/v1/2022.naacl-main.195.
URL https://aclanthology.org/2022.naacl-main.195