# Overcoming Output Constraints in Iterative Learning Control Systems by
Reference Adaptation

Michael Meindl  Fabio Molinari  Jörg Raisch  Thomas Seel  Hochschule Karlsruhe
– Technik und Wirtschaft, Germany (e-mail: meindlmichael@web.de).  Control
Systems Group, Technische Universität Berlin, Germany(e-mail:
<EMAIL_ADDRESS>
###### Abstract

Iterative Learning Control (ILC) schemes can guarantee properties such as
asymptotic stability and monotonic error convergence, but do not, in general,
ensure adherence to output constraints. The topic of this paper is the design
of a reference-adapting ILC (RAILC) scheme, extending an existing ILC system
and capable of complying with output constraints. The underlying idea is to
scale the reference at every trial by using a conservative estimate of the
output’s progression. Properties as the monotonic convergence above a
threshold and the respect of output constraints are formally proven. Numerical
simulations and experimental results reinforce our theoretical results.

###### keywords:

Iterative and Repetitive learning control, Linear systems, Mobile robots,
Learning for control, Intelligent robotics, Autonomous robotic systems

††thanks: We acknowledge support of the Deutsche Förderungsgemeinschaft (DFG,
German Research Foundation) under Germany’s Excellence Strategy – EXC-2002/1 –
Projectnumber 390523135

##  1 Introduction

Iterative Learning Control (ILC) is a control scheme suitable for systems
operating in a repetitive manner. ILC tracks a desired reference and aims at
improving its accuracy from repetition to repetition by exploiting the
information of previous trials, see Bristow et al. (2006) for a survey. ILC
can achieve better performance than conventional feedback systems as the
latter does not exploit available information from previous trials, see, e.g.,
Moore et al. (1992). ILC finds much appeal in fields such as robotics,
manufacturing, and biomedical applications due to the possibility of repeating
trials; such fields have greatly benefited from the implementation of ILC
systems, see, e.g. Elci et al. (2002), Seel et al. (2011), Rogers and Tutty
(2016), and Pandit and Buchheit (1999).

A relevant class of systems for ILC applications are those subject to output
constraints. A robotic manipulator, for example, is commonly restricted when
it comes to possible positions and paths; violating these constraints may
potentially damage the system. In general, ILC systems can guarantee monotonic
error convergence, meaning that the difference between the output and the
desired trajectory decreases in a suitable norm at every trial. However,
compliance with output constraints is, in general, not guaranteed. For
example, Fig. 1 depicts an ILC system learning the desired trajectory
$\mathbf{r}$ while being constrained by the maximum value $y_{\mathrm{max}}$.

Figure 1: Illustration of the output-constrained ILC problem and a robotic
manipulator restricted by output constraints

Although the system is monotonically convergent, the output constraints are
violated during the second trial. This paper addresses this problem and
proposes a modular solution extending the ILC system, thus ensuring compliance
with the output constraints while maintaining monotonic error convergence.

Output-constrained ILC systems have already been investigated elsewhere and
can be classified in two different categories. In the first class, novel
update laws based on constrained optimization have been introduced. For
example, Jin et al. (2014) propose an update-law based on a high-dimensional
and constrained optimization problem. This requires accurate knowledge of the
plant dynamics to ensure compliance with the output constraints in time-
varying ILC systems. In the second class, methods like the one proposed by
Sebastian et al. (2018a) combine ILC with additional feedback control to
handle linear, time-varying, multi-input multi-output systems with input and
output constraints. Sebastian et al. (2018b) validate this approach for a
robotic manipulator where an underlying feedback controller is implemented to
avoid the violation of constraints.

The approach in this paper is to update the reference trajectory for each
trial based on a low-dimensional optimization problem. It does not require
accurate knowledge of the plant dynamics and ensures compliance with the
output constraints while maintaining the existing ILC system’s monotonic error
convergence. In contrast to previous work, the proposed RAILC scheme is a
modular extension requiring neither high-dimensional optimization nor the
design of an underlying feedback controller. The proposed method is validated
using a two-wheeled inverted pendulum robot (TWIPR), whose task is to learn a
highly accurate motion in the presence of output constraints.

The remainder of this paper is structured as follows. Section 2 introduces
notation. Section 3 briefly summarizes the structure of conventional ILC
system consisting of a linear plant, learning function, and Q-filter. Section
4 formulates the problem. Section 5 introduces a RAILC algorithm and discusses
its properties formally. Section 6 presents a TWIPR and validates the RAILC
scheme in both simulation and experiment. The simulation results of the RAILC
algorithm are also compared to those of the conventional ILC. Finally, Section
7 gives concluding remarks.

##  2 Notation

Let $\rm I\\!N_{\mathrm{0}}$ and $\rm I\\!N$ denote the set of nonnegative,
respectively positive, integers, $\rm I\\!R$ the set of real numbers and $\rm
I\\!R_{>0}$, respectively $\rm I\\!R_{\geq 0}$, the set of positive,
respectively nonnegative, real numbers. Vectors are in bold type and lower-
case letters, e.g., $\mathbf{v}$. Matrices are in bold type and upper-case
letters, e.g., $\mathbf{A}$. Matrix $\mathbf{A}^{T}$ denotes the transpose of
the matrix $\mathbf{A}$. Indices of vectors are used to denote the ILC trial,
e.g., $\mathbf{v}_{j}$ denotes the vector $\mathbf{v}$ on the
$j^{\mathrm{th}}$ ILC trial. Let $\left|\left|{\mathbf{v}}\right|\right|$
denote a norm of the vector $\mathbf{v}$, and
$\left|\left|{\mathbf{A}}\right|\right|$ the corresponding, induced matrix
norm of the matrix $\mathbf{\mathbf{A}}$. Particular examples are the
Euclidean norm, denoted by
$\left|\left|{\boldsymbol{\cdot}}\right|\right|_{2}$, and the infinite norm,
denoted by $\left|\left|{\boldsymbol{\cdot}}\right|\right|_{\infty}$. The
spectral radius of a square matrix $\mathbf{A}$ is denoted by
$\rho\left(\mathbf{A}\right)$. The maximum singular value of matrix
$\mathbf{A}$ is denoted by $\overline{\sigma}\left(\mathbf{A}\right)$.

##  3 Lifted-System ILC in a Nutshell

Consider the discrete-time, linear, time-invariant SISO system at its
$j^{\mathrm{th}}$ trial, $j\in\rm I\\!N_{\mathrm{0}}$,

$\left\\{\begin{array}[]{cl}\mathbf{x}_{j}(n+1)&=\mathbf{A}\mathbf{x}_{j}(n)+\mathbf{B}u_{j}(n)+d(n)\\\
y_{j}(n)&=\mathbf{C}\mathbf{x}_{j}(n)\end{array}\right.\,,$ (1)

where $n\in\left\\{1,...,N\right\\}$ is the sample index and $N\in\rm I\\!N$
is the numbers of samples in each trial. Let, $\forall
n\in\left\\{1,...,N\right\\}$, $\mathbf{x}(n)\in\rm I\\!R^{k}$ denote the $k$
dimensional state vector, $u(n)\in\rm I\\!R$ the input variable, $y(n)\in\rm
I\\!R$ the output variable and $d(n)\in\rm I\\!R$ an unknown disturbance,
which is the same for all trials. Let $\mathbf{A}\in\rm I\\!R^{k\times k}$
denote the system matrix, $\mathbf{B}\in\rm I\\!R^{k\times 1}$ the input
matrix, and $\mathbf{C}\in\rm I\\!R^{1\times k}$ the output matrix.

System (1) can be rewritten in its so-called lifted form, i.e.,

$\mathbf{y}_{j}=\mathbf{P}\mathbf{u}_{j}+\mathbf{d}\,,$ (2)

where, at each iteration $j\in\rm I\\!N_{\mathrm{0}}$,

$\displaystyle\mathbf{u}_{j}$
$\displaystyle=\begin{bmatrix}u_{j}(0)&u_{j}(1)&\dots&u_{j}(N-1)\end{bmatrix}^{T}$
(3a) $\displaystyle\mathbf{y}_{j}$
$\displaystyle=\begin{bmatrix}y_{j}(m)&y_{j}(m+1)&\dots&y_{j}(m+N-1)\end{bmatrix}^{T}$
(3b) $\displaystyle\mathbf{d}$
$\displaystyle=\begin{bmatrix}d(m)&d(m+1)&\dots&d(m+N-1)\end{bmatrix}^{T}$
(3c)

and $m\in\rm I\\!N_{\mathrm{0}}$ denotes the system’s relative degree, see
Bristow et al. (2006). For system (1) the parameters $p_{ij}\in\rm I\\!R$ of
the plant matrix $\mathbf{P}$ are given by

$\forall i,j\in\rm I\\!N\qquad
p_{ij}=\left\\{\begin{array}[]{cl}\mathbf{C}\mathbf{A}^{i-j+m-1}\mathbf{B}&\forall
i\geq j\\\ 0&\forall i<j\end{array}\right.\,.$ (4)

The goal of an ILC system is to have output $\mathbf{y}_{j}$ follow a desired
trajectory $\mathbf{r}\in\rm I\\!R^{N}$. To this end, a learning matrix,
namely $\mathbf{L}\in\rm I\\!R^{N\times N}$, and a Q-filter, namely
$\mathbf{Q}\in\rm I\\!R^{N\times N}$, need to be designed (see, e.g. Bristow
et al. (2006)), thus leading to the update law

$\forall j\in\rm
I\\!N,\qquad\mathbf{u}_{j+1}=\mathbf{Q}\left(\mathbf{u}_{j}+\mathbf{L}\mathbf{e}_{j}\right)\,,$
(5)

where $\mathbf{e}_{j}\in\rm I\\!R^{N}$ denotes the error trajectory defined as

$\forall j\in\rm I\\!N,\qquad\mathbf{e}_{j}:=\mathbf{r}-\mathbf{y}_{j}\,.$ (6)

Note that the matrices $\mathbf{P}$, $\mathbf{L}$ and $\mathbf{Q}$ are
regular.

###### Definition 1

(Asymptotic Stability, see Bristow et al. (2006)): The ILC system with
dynamics (2) and update law (5) is asymptotically stable if the limit

$\displaystyle\mathbf{e}_{\infty}:$
$\displaystyle=\lim_{j\rightarrow\infty}\mathbf{e}_{j}$
$\displaystyle=\left[\mathbf{I}-\mathbf{P}\left[\mathbf{I}-\mathbf{Q}\left(\mathbf{I}-\mathbf{L}\mathbf{P}\right)\right]^{-1}\mathbf{Q}\mathbf{L}\right]\left(\mathbf{r}-\mathbf{d}\right)$
(7)

uniquely exists. In what follows, let $\mathbf{e}_{\infty}$ be named _residual
error_.

Note that, if $\mathbf{Q}=\mathbf{I}$, by (7), $\mathbf{e}_{\infty}=0$.

###### Proposition 1

(Bristow et al., 2006, Theorem 1)s The ILC system with dynamics (2) and update
law (5) is asymptotically stable if and only if

$\rho\left(\mathbf{Q}\left(\mathbf{I}-\mathbf{L}\mathbf{P}\right)\right)<1\,.$
(8)

See (Bristow et al., 2006, Theorem 1).

Despite asymptotic stability, the error trajectory can take arbitrarily large
values, thus generating large learning transients. The concept of _monotonic
convergence_ narrows the set of possible error trajectories.

###### Definition 2

(Monotonic Convergence, see Bristow et al. (2006)): The system composed of
plant (2) and update law (5) is monotonically convergent under a given norm
$\left|\left|{\boldsymbol{\cdot}}\right|\right|$ if

$\forall j\in\rm
I\\!N_{\mathrm{0}},\quad\left|\left|{\mathbf{e}_{\infty}-\mathbf{e}_{j+1}}\right|\right|\leq\gamma\left|\left|{\mathbf{e}_{\infty}-\mathbf{e}_{j}}\right|\right|\,,$
(9)

where $\gamma\in[0,1)$ is the convergence rate.

###### Proposition 2

The system (2)-(5) is monotonically convergent under a given norm
$\left|\left|{\boldsymbol{\cdot}}\right|\right|$ if

$\gamma:=\left|\left|{\mathbf{P}\mathbf{Q}\left(\mathbf{I}-\mathbf{L}\mathbf{P}\right)\mathbf{P}^{-1}}\right|\right|<1\,.$
(10)

See Bristow et al. (2006).

Monotonic convergence is traditionally verified for the spectral and the
infinite norms, i.e., respectively,

$\displaystyle\gamma_{\mathrm{2}}$
$\displaystyle:=\overline{\sigma}\left(\mathbf{P}\mathbf{Q}\left(\mathbf{I}-\mathbf{L}\mathbf{P}\right)\mathbf{P}^{-1}\right)$
(11) and $\displaystyle\gamma_{\infty}$
$\displaystyle:=\left|\left|{\mathbf{P}\mathbf{Q}\left(\mathbf{I}-\mathbf{L}\mathbf{P}\right)\mathbf{P}^{-1}}\right|\right|_{\infty}\,.$
(12)

###### Definition 3

(Monotonic convergence above athreshold, see Seel et al. (2017)): System
(2)-(5) is monotonically convergent above a threshold $\kappa\in\rm
I\\!R_{\geq 0}$ under a given norm
$\left|\left|{\boldsymbol{\cdot}}\right|\right|$ if

$\forall j\in\rm
I\\!N_{\mathrm{0}}:\left|\left|{\mathbf{e}_{j}}\right|\right|\geq\kappa\quad\implies\left|\left|{\mathbf{e}_{j+1}}\right|\right|\leq\left|\left|{\mathbf{e}_{j}}\right|\right|\,.$
(13)

###### Proposition 3

(Seel et al., 2017, Theorem 1) System (2)-(5) is monotonically convergent
above the threshold
$\epsilon=\left|\left|{\left(\mathbf{I}-\mathbf{P}\mathbf{Q}\mathbf{P}^{-1}\right)\left(\mathbf{r}-\mathbf{d}\right)}\right|\right|$
under a given norm $\left|\left|{\boldsymbol{\cdot}}\right|\right|$ if

$\gamma<1\,.$ (14)

See (Seel et al., 2017, Theorem 1).

By Propositions 1-3, to ensure properties as asymptotic stability and
monotonic convergence, standard design methods, see, e.g., Bristow et al.
(2006), can be employed. Commonly, the Q-filter is chosen as a low-pass filter
so that frequencies above its bandwidth are cut-off from the learning, thus
increasing robustness. The learning matrix $\mathbf{L}$ is typically designed
by either tuning the parameters of a PD-function, applying
$\mathcal{H}_{\infty}$ optimization, or by solving a quadratic optimal
problem.

##  4 Problem Formulation

Consider system (2)-(5) with a desired trajectory $\mathbf{r}$. Both the
Q-filter and the learning matrix are designed such that asymptotic stability
and monotonic error convergence are guaranteed.

Additionally, the output $\mathbf{y}_{j}$ is constrained by an upper bound
$y_{\mathrm{max}}\in\rm I\\!R_{>0}$, which must not be violated on any trial,
i.e.,

$\forall j\in\rm
I\\!N_{\mathrm{0}},\qquad\left|\left|{\mathbf{y}_{j}}\right|\right|_{\infty}<y_{\mathrm{max}}\,.$
(15)

Despite monotonic convergence, the conventional ILC system does not
necessarily enforce output constraints (see Fig. 1). We propose to extend the
designed ILC in a modular fashion, so that output constraints are satisfied
and monotonic error convergence maintained.

To ensure the problem being well-posed, some assumptions are needed.

###### Assumption 1

There is a known initial input trajectory $\mathbf{u}_{\mathrm{0}}$ leading to
an initial output trajectory $\mathbf{y}_{\mathrm{0}}$ such that

$\left|\left|{\mathbf{y}_{\mathrm{0}}}\right|\right|_{\infty}\leq
y_{\mathrm{max}}\,.$ (16)

###### Assumption 2

The reference trajectory $\mathbf{r}$ is chosen such that

$\left|\left|{\mathbf{r}}\right|\right|_{\infty}\leq y_{\mathrm{max}}\,.$ (17)

##  5 Reference-Adapting ILC

The basic idea of this reference-adapting iterative learning control (RAILC)
scheme is to adapt the reference trajectory $\mathbf{r}$ at each ILC trial to
ensure that the output trajectory $\mathbf{y}_{j}$ does not exceed the maximum
value $y_{\mathrm{max}}$. To this end, let

$\forall j\in\rm I\\!N_{\mathrm{0}},\qquad{\mathbf{r}}_{j}\in\rm I\\!R^{N}$
(18)

denote the adapted reference trajectory at the $j^{\mathrm{th}}$ trial leading
to the adapted update law

$\forall j\in\rm
I\\!N_{\mathrm{0}},\qquad\mathbf{u}_{j+1}=\mathbf{Q}\left(\mathbf{u}_{j}+\mathbf{L}\left({\mathbf{r}}_{j}-\mathbf{y}_{j}\right)\right)\,.$
(19)

Figure 2: Block diagram of the RAILC system

Let $\overline{\epsilon}\in\rm I\\!R$ denote an upper bound such that

$\overline{\epsilon}\geq\max_{\forall j\in\rm
I\\!N}{\left|\left|{\left(\mathbf{I}-\mathbf{P}\mathbf{Q}\mathbf{P}^{-1}\right)\left({\mathbf{r}}_{j}-\mathbf{d}\right)}\right|\right|_{\infty}}\,.$
(20)

###### Algorithm 1

The adapted reference trajectory ${\mathbf{r}}_{j}$ is

$\forall j\in\rm
I\\!N_{\mathrm{0}},\qquad{\mathbf{r}}_{j}:=\mathbf{y}_{j}+a_{j}\left(\mathbf{r}-\mathbf{y}_{j}\right)$
(21)

with

$\displaystyle a_{j}=\max\tilde{a}_{j}$ (22a)
$\displaystyle\mathrm{s.t.}\quad$ $\displaystyle\tilde{a}_{j}\in[0,1]$ (22b)
$\displaystyle\tilde{a}_{j}\leq\frac{y_{\mathrm{max}}-\left|\left|{\mathbf{y}_{j}+\tilde{a}_{j}(\mathbf{r}-\mathbf{y}_{j})}\right|\right|_{\infty}-\overline{\epsilon}}{\gamma_{\infty}\left|\left|{\mathbf{r}-\mathbf{y}_{j}}\right|\right|_{\infty}}$
(22c)

Note that the optimization problem (22) can be efficiently solved using
bisection, which makes the algorithm applicable to embedded systems with low
computational power.

Consider the upper bound $\overline{\epsilon}$ required to solve the
optimization problem (22). In the trivial case of $\mathbf{Q}=\mathbf{I}$,
$\overline{\epsilon}$ can be set to zero. In the case of
$\mathbf{Q}\neq\mathbf{I}$ an argument similar to the one in Seel et al.
(2017) can be applied. In fact,
$\mathbf{I}-\mathbf{P}\mathbf{Q}\mathbf{P}^{-1}$ commonly takes the form of a
high-pass filter. Under the assumption that $\mathbf{r}$, $\mathbf{d}$,
$\mathbf{y}_{\mathrm{0}}$ and therefore ${\mathbf{r}}_{j}$ have spectra well
below the high-pass filter’s cutoff frequency, the upper bound
$\overline{\epsilon}$ can be assumed to be a small positive number.

Next, conditions for the existence of a solution to the optimization problem
(22) are investigated.

###### Proposition 4

For a given $\overline{\epsilon}$ and $\mathbf{y}_{j}$, there is a solution to
(22) if

$y_{\mathrm{max}}-\overline{\epsilon}\geq\left|\left|{\mathbf{y}_{j}}\right|\right|_{\infty}\,.$
(23)

$\tilde{a}_{j}=0$ clearly is a feasible solution of (22) if
$y_{\mathrm{max}}-\left|\left|{\mathbf{y}_{j}}\right|\right|_{\infty}-\overline{\epsilon}\geq
0$; i.e., if (23) holds. Proposition 4 implies that a solution to the
optimization problem (22) is guaranteed to exist if

$y_{\mathrm{max}}-\left|\left|{\mathbf{y}_{j}}\right|\right|_{\infty}\geq\overline{\epsilon}.$

As previously discussed, $\overline{\epsilon}$ can be assumed to be a small
number. Hence, if $\mathbf{y}_{j}$ is not “too close” to $y_{\mathrm{max}}$, a
solution to (22) exists.

Under this assumption, we can now focus on the output constraints.

###### Proposition 5

If there is a solution to the optimization problem (22), the output constraint
(15) is guaranteed to hold in the next trial.

If a solution to (22) exists, the following is true

$a_{j}\gamma_{\infty}\left|\left|{\mathbf{r}-\mathbf{y}_{j}}\right|\right|_{\infty}\leq
y_{\mathrm{max}}-\left|\left|{{\mathbf{r}}_{j}}\right|\right|_{\infty}-\overline{\epsilon}\,.$
(24)

Furthermore, (21) gives

$a_{j}\left|\left|{\mathbf{r}-\mathbf{y}_{j}}\right|\right|_{\infty}=\left|\left|{{\mathbf{r}}_{j}-\mathbf{y}_{j}}\right|\right|_{\infty}\,,$
(25)

which, incorporated into (24), yields

$\gamma_{\infty}\left|\left|{{\mathbf{r}}_{j}-\mathbf{y}_{j}}\right|\right|_{\infty}+\overline{\epsilon}\leq
y_{\mathrm{max}}-\left|\left|{{\mathbf{r}}_{j}}\right|\right|_{\infty}\,.$
(26)

By combining (19) and (2), one obtains

$\mathbf{u}_{j+1}=\mathbf{Q}\left(\mathbf{I}-\mathbf{L}\mathbf{P}\right)\mathbf{u}_{j}+\mathbf{Q}\mathbf{L}\left({\mathbf{r}}_{j}-\mathbf{d}\right)$
(27)

and

$\mathbf{y}_{j+1}=\mathbf{P}\mathbf{Q}\left(\mathbf{I}-\mathbf{L}\mathbf{P}\right)\mathbf{P}^{-1}\mathbf{y}_{j}+\\\
\left(\mathbf{I}-\mathbf{P}\mathbf{Q}\mathbf{P}^{-1}\right)\mathbf{d}+\mathbf{P}\mathbf{Q}\mathbf{L}{\mathbf{r}}_{j}\,,$
(28)

which leads to

${\mathbf{r}}_{j}-\mathbf{y}_{j+1}=\mathbf{P}\mathbf{Q}\left(\mathbf{I}-\mathbf{L}\mathbf{P}\right)\mathbf{P}^{-1}\left({\mathbf{r}}_{j}-\mathbf{y}_{j}\right)+\\\
\left(\mathbf{I}-\mathbf{P}\mathbf{Q}\mathbf{P}^{-1}\right)\left({\mathbf{r}}_{j}-\mathbf{d}\right)\,.$
(29)

By applying the inequalities of norms, and by incorporating (20) and (12), one
obtains

$\left|\left|{{\mathbf{r}}_{j}-\mathbf{y}_{j+1}}\right|\right|_{\infty}\leq\gamma_{\infty}\left|\left|{{\mathbf{r}}_{j}-\mathbf{y}_{j}}\right|\right|_{\infty}+\overline{\epsilon}\,.$
(30)

Combining this with (26) yields

$\left|\left|{{\mathbf{r}}_{j}-\mathbf{y}_{j+1}}\right|\right|_{\infty}\leq
y_{\mathrm{max}}-\left|\left|{{\mathbf{r}}_{j}}\right|\right|_{\infty}\,,$
(31)

which, by adding $\left|\left|{\mathbf{y}_{j+1}}\right|\right|_{\infty}$,
equals

$\left|\left|{\mathbf{y}_{j+1}}\right|\right|_{\infty}\leq
y_{\mathrm{max}}+\left|\left|{\mathbf{y}_{j+1}}\right|\right|_{\infty}-\left|\left|{{\mathbf{r}}_{j}}\right|\right|_{\infty}-\\\
\left|\left|{{\mathbf{r}}_{j}-\mathbf{y}_{j+1}}\right|\right|_{\infty}\,.$
(32)

Next, consider the norm inequality

$\left|\left|{{\mathbf{r}}_{j}-\mathbf{y}_{j+1}}\right|\right|_{\infty}\geq\left|\left|{\mathbf{y}_{j}}\right|\right|_{\infty}-\left|\left|{{\mathbf{r}}_{j}}\right|\right|_{\infty},$
(33)

equivalently,

$0\geq\left|\left|{\mathbf{y}_{j+1}}\right|\right|_{\infty}-\left|\left|{{\mathbf{r}}_{j}}\right|\right|_{\infty}-\left|\left|{{\mathbf{r}}_{j}-\mathbf{y}_{j+1}}\right|\right|_{\infty}\,.$
(34)

Combining the latter with (32) yields

$\left|\left|{\mathbf{y}_{j+1}}\right|\right|_{\infty}\leq
y_{\mathrm{max}}\,,$ (35)

which concludes the proof.

Also, the monotonic convergence properties of the RAILC system are
investigated.

###### Proposition 6

System (2)-(22) is monotonically convergent above a threshold
$\hat{\kappa}=\frac{\hat{\epsilon}}{1-\hat{\gamma}}$, with

$\hat{\epsilon}:=\left|\left|{\left(\mathbf{I}-\mathbf{P}\mathbf{Q}\mathbf{P}^{-1}\right)\left(\mathbf{r}-\mathbf{d}\right)}\right|\right|\,,$
(36)

under a given norm $\left|\left|{\boldsymbol{\cdot}}\right|\right|$ if

$\hat{\gamma}:=\max_{j\in\rm
I\\!N_{\mathrm{0}}}\left\\{\left|\left|{\mathbf{P}\mathbf{Q}\left(\mathbf{I}-a_{j}\mathbf{L}\mathbf{P}\right)\mathbf{P}^{-1}}\right|\right|\right\\}<1\,.$
(37)

By combining (19) and (21), one obtains

$\mathbf{u}_{j+1}=\mathbf{Q}\left(\mathbf{I}-a_{j}\mathbf{L}\mathbf{P}\right)\mathbf{u}_{j}+a_{j}\mathbf{Q}\mathbf{L}\left(\mathbf{r}-\mathbf{d}\right)\,,$
(38)

which substituted into (2) gives

$\mathbf{y}_{j+1}=\mathbf{P}\mathbf{Q}\left(\mathbf{I}-a_{j}\mathbf{L}\mathbf{P}\right)\mathbf{P}^{-1}\mathbf{y}_{j}+\\\
\left(\mathbf{I}-\mathbf{P}\mathbf{Q}\mathbf{P}^{-1}\right)\mathbf{d}+a_{j}\mathbf{P}\mathbf{Q}\mathbf{L}\mathbf{r}\,.$
(39)

Working the latter into (6) leads to

$\mathbf{e}_{j+1}=\mathbf{P}\mathbf{Q}\left(\mathbf{I}-a_{j}\mathbf{L}\mathbf{P}\right)\mathbf{P}^{-1}\mathbf{e}_{j}+\\\
\left(\mathbf{I}-\mathbf{P}\mathbf{Q}\mathbf{P}^{-1}\right)\left(\mathbf{r}-\mathbf{d}\right)\,.$
(40)

Taking the norm, inserting (36) and (37), and applying the inequalities of
norms leads to

$\left|\left|{\mathbf{e}_{j+1}}\right|\right|\leq\hat{\gamma}\left|\left|{\mathbf{e}_{j}}\right|\right|+\hat{\epsilon}\,,$
(41)

which, by subtracting $\left|\left|{\mathbf{e}_{j}}\right|\right|$, gives

$\left|\left|{\mathbf{e}_{j+1}}\right|\right|-\left|\left|{\mathbf{e}_{j}}\right|\right|\leq\left(\hat{\gamma}-1\right)\left|\left|{\mathbf{e}_{j}}\right|\right|+\hat{\epsilon}\,.$
(42)

Furthermore, the monotonic convergence of the form

$\left|\left|{\mathbf{e}_{j+1}}\right|\right|\leq\left|\left|{\mathbf{e}_{j}}\right|\right|$
(43)

equals
$\left|\left|{\mathbf{e}_{j+1}}\right|\right|-\left|\left|{\mathbf{e}_{j}}\right|\right|\leq
0\,,$ which, by (42), is guaranteed if

$\left(\hat{\gamma}-1\right)\left|\left|{\mathbf{e}_{j}}\right|\right|+\hat{\epsilon}\leq
0\,.$ (44)

The latter is equivalent to

$\left|\left|{\mathbf{e}_{j}}\right|\right|\geq\frac{\hat{\epsilon}}{1-\hat{\gamma}},$
(45)

thus concluding the proof. Recall the argument that $\overline{\epsilon}$ can
be set to a small number, if $\mathbf{r}$, $\mathbf{d}$, and
$\mathbf{y}_{\mathrm{0}}$ have spectra below the cutoff frequency of
$\mathbf{I}-\mathbf{P}\mathbf{Q}\mathbf{P}^{-1}$. By (36), the same argument
applies to $\hat{\epsilon}$. Regarding (37), consider the following
proposition.

###### Proposition 7

System (2)-(22) fulfills

$\hat{\gamma}_{\mathrm{2}}:=\max_{j\in\rm
I\\!N}\left\\{\left|\left|{\mathbf{P}\mathbf{Q}\left(\mathbf{I}-a_{j}\mathbf{L}\mathbf{P}\right)\mathbf{P}^{-1}}\right|\right|_{2}\right\\}<1$
(46)

if

$\gamma_{\mathrm{2}}<1\quad\textit{and}\quad\left|\left|{\mathbf{P}\mathbf{Q}\mathbf{P}^{-1}}\right|\right|_{2}\leq
1\,.$ (47)

The proof follows directly from Proposition 8 (see App. B). Therefore, the
RAILC system is monotonically convergent above a threshold under the Euclidean
norm, if the conventional ILC system is monotonically convergent under the
Euclidean norm and the mild requirement of
$\left|\left|{\mathbf{P}\mathbf{Q}\mathbf{P}^{-1}}\right|\right|_{2}\leq 1$.

Summarizing the above results, the following statements can be given for the
proposed RAILC scheme.

  * •

Under mild assumptions on the spectra of $\mathbf{r}$, $\mathbf{d}$ and
$\mathbf{y}_{\mathrm{0}}$, the upper bound $\overline{\epsilon}$ can be
assumed to be a small number.

  * •

If
$y_{\mathrm{max}}-\overline{\epsilon}\geq\left|\left|{\mathbf{y}_{j}}\right|\right|_{\infty}$,
a solution to the optimization problem (22) exists.

  * •

If a solution to the optimization problem (22) exists, the reference adaption
scheme ensures that the output trajectory $\mathbf{y}_{j+1}$ of the next trial
satisfies the output constraints (15).

  * •

If the conventional ILC system is monotonically convergent under the Euclidean
norm and
$\left|\left|{\mathbf{P}\mathbf{Q}\mathbf{P}^{-1}}\right|\right|_{2}\leq 1$,
the RAILC system is monotonically convergent above a threshold under the
Euclidean norm.

  * •

The low-dimensional nature of the optimization problem (22) makes the RAILC
scheme applicable to systems with low computing power.

  * •

The above results were derived under the assumption of $\mathbf{P}$,
$\mathbf{L}$ and $\mathbf{Q}$ being regular. Subsequently, the RAILC scheme
can also be applied to time-varying, non-causal systems.

##  6 Experimental Results

As a demonstrator for the proposed RAILC, we consider a TWIPR, supposed to
perform complex and repeated maneuvers. The robot consists of the pendulum
body housing the main electronics, i.e., a microcomputer, inertial measurement
units, motors and accumulator. Wheels are mounted onto the motors, which
combined with the robot’s body create an inverted pendulum. Aiming at keeping
the vehicle in an upright position, the system is clearly unstable, thus
requiring feedback control. Two aspects encourage the use of ILC: the
possibility of repeating trials and the lack of precise knowledge regarding
the dynamics and the disturbances.

First, let us briefly introduce the TWIPR’s dynamics. We consider a TWIPR
moving along a straight line. The motor torque is the input variable and is
denoted by $u\in\rm I\\!R$. Let $\Theta\in\rm I\\!R$ denote the pendulum’s
pitch angle and let $s\in\rm I\\!R$ denote the robot’s position (see Fig. 3).

Figure 3: The two-wheeled inverted pendulum robot

The state vector $\mathbf{z}\in\rm I\\!R^{4}$ is defined as

$\mathbf{z}=\left[\Theta\hskip 5.69046pt\dot{\Theta}\hskip 5.69046pts\hskip
5.69046pt\dot{s}\right]^{T}\,.$ (48)

A thorough derivation of the TWIPR’s dynamics can be found in Kim and Kwon
(2015). It has been successfully employed elsewhere, see, e.g., Music et al.
(2018), whose notation is also employed here. By this, let the dynamics be

$\forall t\in\rm I\\!R_{\geq
0},\qquad\dot{\mathbf{z}}(t)=f\left(\mathbf{z}(t),u(t)\right)\,.$ (49)

To stabilize the inverted pendulum, the controller input $u_{\mathrm{C}}\in\rm
I\\!R$ is calculated by a time-discrete feedback controller of the form

$u_{\mathrm{c}}=-\mathbf{K}\mathbf{z}\,,$ (50)

with sampling time $T=0.02\ \mathrm{sec}$. The feedback matrix
$\mathbf{K}^{1\times 4}$ is designed using pole-placement and the linearised
(at the upright equilibrium), discretised form of the dynamics (49).

For demonstration purposes, the reference trajectory

$r(n)=1.22\rm{sin}\left(2/3\pi Tn\right)$ (51)

with $N=150$ samples is chosen, whose lifted form is denoted by
$\mathbf{r}\in\rm I\\!R^{N}$. To perform this trajectory an ILC system is
implemented, which calculates the input variable $u_{\mathrm{ILC}}\in\rm
I\\!R$, which is an additional motor torque leading to the overall input

$\forall n=1,\dots,N,\quad u(n)=u_{\mathrm{C}}(n)+u_{\mathrm{ILC}}(n)\,.$ (52)

Figure 4: Block diagram of the ILC system

The lifted form of the ILC’s input variable $u_{\mathrm{ILC}}$ is denoted by
$\mathbf{u}_{j}\in\rm I\\!R^{N}$, where $j\in\rm I\\!N$ denotes the trial
index. To update the input trajectory $\mathbf{u}_{j}$, an ILC law as in (5),
is applied, where $\mathbf{L}\in\rm I\\!R^{N\times N}$ denotes the lifted form
of the learning function and $\mathbf{Q}\in\rm I\\!R^{N\times N}$ denotes the
lifted form of the Q-filter. To design both these transfer functions, the
dynamics of the closed loop plant are linearised leading to the lifted form
$\mathbf{P}\in\rm I\\!R^{N\times N}$ and the linear dynamics (2), where
$\mathbf{y}_{j}\in\rm I\\!R^{N}$ denotes the pitch trajectory. The learning
matrix and Q-filter are calculated using quadratic optimal design, see Bristow
et al. (2006), yielding an asymptotically stable, monotonically convergent ILC
system with the convergence rates

$\gamma_{\mathrm{2}}=0.5$ (53)

and

$\gamma_{\infty}=0.64\,.$ (54)

The ILC is output-constrained as, due to the TWIPR’s design, the pitch angle
is limited by

$\forall j\in\rm
I\\!N_{\mathrm{0}},\quad\left|\left|{\mathbf{y}_{j}}\right|\right|_{\infty}\leq
y_{\mathrm{max}}=1.31\ [\mathrm{rad}]\,.$ (55)

To demonstrate the impact of this restriction, consider the simulation results
of applying the conventional ILC system to the TWIPR’s non-linear dynamics,
which are displayed in Fig. 4. On the first trial, the output trajectory
$\mathbf{y}_{\mathrm{1}}$ violates the constraints characterized by
$y_{\mathrm{max}}$.

Figure 5: Output Progression of the conventional ILC

To ensure that the output constraints are not violated, the RAILC algorithm is
applied in simulation first. To meet Assumption 1, the initial input
trajectory

$\mathbf{u}_{\mathrm{0}}=\mathbf{0}$ (56)

is chosen. Assumption 2 is also satisfied as the reference trajectory does not
exceed the maximum value. Lastly, the upper bounde estimate
$\overline{\epsilon}$ is chosen as

$\overline{\epsilon}=2\left|\left|{\left(\mathbf{I}-\mathbf{P}\mathbf{Q}\mathbf{P}^{-1}\right)\mathbf{r}}\right|\right|_{\infty}\,.$
(57)

Results in Fig. 5 show that the RAILC system complies with the output
constraints as derived in Proposition 5.

Figure 6: Output Progression of the RAILC in simulation

Afterwards, the RAILC algorithm is experimentally validated using the TWIPR.
Fig. 6 illustrates that the RAILC algorithm also complies with the output
constraints when applied experimentally.

Figure 7: Experimental Output Progression of the RAILC

Due to (53) and (9), the conventional ILC system is monotonically convergent.
As

$\left|\left|{\mathbf{P}\mathbf{Q}\mathbf{P}^{-1}}\right|\right|_{2}=1$ (58)

also fulfills the condition of Proposition 7, the RAILC system is
monotonically convergent above a threshold under the Euclidean norm. Fig. 7
shows the progression of the error norms and validates this theoretical
finding in both simulation and experiment. Furthermore, in simulation, the
RAILC achieves a faster decline of the error norm than the conventional ILC
system.

Figure 8: Progression of the error norms
$\left|\left|{\mathbf{e}_{j}}\right|\right|$ and $a_{j}$

A solution to the reference adaptation problem, as discussed in Proposition 1,
has been obtained at every trial, both in simulation and experiment. Fig. 7
shows that, in simulation, the value of $a_{j}$ is monotonically increasing
and equals one from the third trial onward. In the experiment, $a_{j}$ is not
monotonically increasing as there is a decrease in the first trial. From there
onward, $a_{j}$ is monotonically increasing and reaches one on the fourth
trial.

##  7 Conclusion

We have extended standard ILC schemes in a modular fashion to cope with
output-constrained systems. By adapting the reference trajectory based on a
conservative estimate of the output progression, the violation of output
constraints is avoided. Under mild assumptions on disturbances, reference
trajectory and the output trajectory, the existence of a solution is
guaranteed. A condition for the RAILC error to monotonically converge above a
threshold was given. It has been shown that this threshold can be expected to
be close to zero. The RAILC approach has been applied to a TWIPR, and both
simulation and experimental results have been shown. While the conventional
ILC system violates output constraints, the RAILC system complies with them.
Furthermore, the RAILC system exhibits monotonic convergence above a small
threshold in both simulation and experiment. Finally, simulation results show
that applying RAILC does not slow down the decline of the error norm when
compared to the conventional ILC.

Ongoing work investigates conditions for the existence of a solution to the
optimization problem (22). Furthermore, we will study the possibility of using
RAILC to increase performance.

## References

  * Bristow et al. (2006) Bristow, D. A., Tharayil, M., Alleyne, A. G., 2006. A survey of iterative learning control. IEEE control systems magazine 26 (3), 96–114.
  * Elci et al. (2002) Elci, H., Longman, R. W., Phan, M. Q., Juang, J.-N., Ugoletti, R., Aug 2002. Simple learning control made practical by zero-phase filtering: applications to robotics. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications 49 (6), 753–767.
  * Jin et al. (2014) Jin, X., Wang, Z., Kwong, R. H., 2014. Convex optimization based iterative learning control for iteration-varying systems under output constraints. In: 11th IEEE International Conference on Control & Automation (ICCA). IEEE, Piscataway, NJ, pp. 1444–1448.
  * Kim and Kwon (2015) Kim, S., Kwon, S., 2015. Dynamic modeling of a two-wheeled inverted pendulum balancing mobile robot. International Journal of Control, Automation and Systems 13 (4), 926–933.
  * Moore et al. (1992) Moore, K. L., Dahleh, M., Bhattacharyya, S. P., Jul 1992. Iterative learning control: A survey and new results. J. Robotic Syst. 9 (5), 563–594.
  * Music et al. (2018) Music, Z., Molinari, F., Gallenmüller, S., Ayan, O., Raisch, J., Nov 2018. Design of a Networked Controller for a Two-Wheeled Inverted Pendulum Robot. ResearchGate.
  * Pandit and Buchheit (1999) Pandit, M., Buchheit, K.-H., May 1999. Optimizing iterative learning control of cyclic production processes with application to extruders. IEEE Trans. Control Syst. Technol. 7 (3), 382–390.
  * Rogers and Tutty (2016) Rogers, E., Tutty, O. R., Sep 2016. Iterative learning control with applications in energy generation, lasers and health care. Proceedings. Mathematical, Physical, and Engineering Sciences / The Royal Society 472 (2193).
  * Sebastian et al. (2018a) Sebastian, G., Tan, Y., Oetomo, D., Mareels, I., 2018a. Feedback-based iterative learning design and synthesis with output constraints for robotic manipulators. IEEE Control Systems Letters 2 (3), 513–518.
  * Sebastian et al. (2018b) Sebastian, G., Tan, Y., Oetomo, D., Mareels, I., 2018b. Iterative learning control for linear time-varying systems with input and output constraints. In: The 2018 Australian & New Zealand Control Conference - ANZCC 2018. IEEE, [Piscataway, New Jersey], pp. 87–92.
  * Seel et al. (2011) Seel, T., Schauer, T., Raisch, J., Jan 2011. Iterative Learning Control for Variable Pass Length Systems. IFAC Proceedings Volumes 44 (1), 4880–4885.
  * Seel et al. (2017) Seel, T., Schauer, T., Raisch, J., Mar 2017. Monotonic convergence of iterative learning control systems with variable pass length. Int. J. Control 90 (3), 393–406.

##  Appendix A Proposition 8

###### Lemma 1

Given a matrix $\mathbf{A}\in\rm I\\!R^{N\times N}$, a matrix
$\mathbf{B}\in\rm I\\!R^{N\times N}$, and a vector $\mathbf{v}\in\rm
I\\!R^{N\times N}$ with

$\left|\left|{\mathbf{v}}\right|\right|_{2}=1\,,$ (59)

if (sufficient condition)

$\left|\left|{\mathbf{B}-\mathbf{A}}\right|\right|_{2}<1\,,$ (60)

then

$\mathbf{v}^{T}\left(\mathbf{B}^{T}\mathbf{A}+\mathbf{A}^{T}\mathbf{B}\right)\mathbf{v}>\mathbf{v}^{T}\mathbf{A}^{T}\mathbf{A}\mathbf{v}+\mathbf{v}^{T}\mathbf{B}^{T}\mathbf{B}\mathbf{v}-1\,.$
(61)

Combining (60), (59), and the submultiplicativity of norms gives

$1>\left|\left|{\mathbf{B}-\mathbf{A}}\right|\right|_{2}=\left|\left|{\mathbf{B}-\mathbf{A}}\right|\right|_{2}\left|\left|{\mathbf{v}}\right|\right|_{2}\\\
\geq\left|\left|{\mathbf{B}\mathbf{v}-\mathbf{A}\mathbf{v}}\right|\right|_{2}=\sqrt{{\left(\mathbf{B}\mathbf{v}-\mathbf{A}\mathbf{v}\right)}^{T}\left(\mathbf{B}\mathbf{v}-\mathbf{A}\mathbf{v}\right)}\,,$
(62)

which, by taking the squares and expanding, leads to (61), thus, concluding
the proof.

###### Proposition 8

Given a matrix $\mathbf{A}\in\rm I\\!R^{N\times N}$, a matrix
$\mathbf{B}\in\rm I\\!R^{N\times N}$, and a scalar $a\in(0,1]$, if (sufficient
condition)

$\left|\left|{\mathbf{B}-\mathbf{A}}\right|\right|_{2}<1\quad\mathit{and}\quad\left|\left|{\mathbf{B}}\right|\right|_{2}\leq
1\,,$ (63)

then

$\left|\left|{\mathbf{B}-a\mathbf{A}}\right|\right|_{2}<1\,.$ (64)

First, let $\mathbf{w}\in\rm I\\!R^{N}$ denote a vector such that

$\mathbf{w}=\underset{\left|\left|{\mathbf{x}}\right|\right|_{2}=1}{\mathrm{argmax}}\left\\{\left|\left|{\left(\mathbf{B}-a\mathbf{A}\right)\mathbf{x}}\right|\right|_{2}\right\\}\,,$
(65)

which, according to the definition of induced matrix norms, implies

$\left|\left|{\mathbf{B}-a\mathbf{A}}\right|\right|_{2}=\left|\left|{\left(\mathbf{B}-a\mathbf{A}\right)\mathbf{w}}\right|\right|_{2}=\\\
\sqrt{\mathbf{w}^{T}\mathbf{B}^{T}\mathbf{B}\mathbf{w}-a\mathbf{w}^{T}\left(\mathbf{B}^{T}\mathbf{A}+\mathbf{A}^{T}\mathbf{B}\right)\mathbf{w}+a^{2}\mathbf{w}^{T}\mathbf{A}^{T}\mathbf{A}\mathbf{w}}\,.$
(66)

Lemma 1 further gives

$\mathbf{w}^{T}\left(\mathbf{B}^{T}\mathbf{A}+\mathbf{A}^{T}\mathbf{B}\right)\mathbf{w}>\mathbf{w}^{T}\mathbf{A}^{T}\mathbf{A}\mathbf{w}+\mathbf{w}^{T}\mathbf{B}^{T}\mathbf{B}\mathbf{w}-1\,,$
(67)

which combined with (66) leads to

$\left|\left|{\mathbf{B}-a\mathbf{A}}\right|\right|_{2}<\\\
\sqrt{(1-a)\mathbf{w}^{T}\mathbf{B}^{T}\mathbf{B}\mathbf{w}+a+(a^{2}-a)\mathbf{w}^{T}\mathbf{A}^{T}\mathbf{A}\mathbf{w}}\,.$
(68)

Submultiplicativity of norms, (63), and (65) give

$\mathbf{w}^{T}\mathbf{B}^{T}\mathbf{B}\mathbf{w}\leq 1\,,$ (69)

which combined with

$a\in(0,1]~{}\implies~{}a^{2}\leq a~{}\implies~{}a^{2}-a\leq 0$ (70)

and (68) leads to

$\left|\left|{\mathbf{B}-a\mathbf{A}}\right|\right|_{2}<\sqrt{1-a+a}=1\,.$
(71)

This concludes the proof.


Predicting Zip Code-Level Vaccine Hesitancy in US Metropolitan Areas Using
Machine Learning Models on Public Tweets

Sara Melotte1, Mayank Kejriwal1,*

1 Information Sciences Institute, University of Southern California, Marina
del Rey, CA, United States

*<EMAIL_ADDRESS>
## Abstract

Although the recent rise and uptake of COVID-19 vaccines in the United States
has been encouraging, there continues to be significant vaccine hesitancy in
various geographic and demographic clusters of the adult population. Surveys,
such as the one conducted by Gallup over the past year, can be useful in
determining vaccine hesitancy, but can be expensive to conduct and do not
provide real-time data. At the same time, the advent of social media suggests
that it may be possible to get vaccine hesitancy signals at an aggregate level
(such as at the level of zip codes) by using machine learning models and
socioeconomic (and other) features from publicly available sources. It is an
open question at present whether such an endeavor is feasible, and how it
compares to baselines that only use constant priors. To our knowledge, a
proper methodology and evaluation results using real data has also not been
presented. In this article, we present such a methodology and experimental
study, using publicly available Twitter data collected over the last year. Our
goal is not to devise novel machine learning algorithms, but to evaluate
existing and established models in a comparative framework. We show that the
best models significantly outperform constant priors, and can be set up using
open-source tools.

## Background

Although more people continue to be vaccinated against COVID-19 in the United
States and many other nations with each passing week, significant vaccine
hesitancy continues to persist [1], [2]. Vaccine hesitancy in the US has
complex drivers, not all of which are well understood, especially among under-
served segments of the population [3], [4]. Even prior to COVID-19, vaccine
hesitancy against influenza (among others) was non-trivial [5]. In the early
days of COVID-19, conspiracy theories about the vaccines (as well as other
COVID-related issues) had a significant footprint on social media [6], [7].
Such theories, as well as other sources of disinformation and misinformation,
can not only go viral on social media but lead to real-world public health
consequences by sowing doubt and mistrust among people who are otherwise ‘on
the fence’ about the benefits and efficacy of vaccines. Unfortunately, it can
be challenging to detect vaccine hesitancy, whether severe or moderate, in
near-real time. Surveys, such as the one conducted by Gallup [8], released
early enough, are expensive and still subject to lags. Given the frenetic pace
of digital communication and social media virality, more real-time (and
inexpensive) detection of vaccine hesitancy at privacy-preserving spatial
granularity, such as at the county or even zip code-level, can significantly
increase public utility. Even in a post-COVID era, such a system, if
sufficiently generalized, can help detect and address vaccine hesitancy (for
influenza and other diseases) before it becomes entrenched in a particular
region.

Recent advances in natural language processing (NLP) and social media analysis
have been quite impressive, especially due to improvements in deep neural
networks and language ‘representation’ learning [9], [10], [11]. For example,
so-called ‘word embedding’ algorithms, which have been trained on large
quantities of text in corpora such as Wikipedia and Google News, learn a real-
valued vector representation for each word [10]. In the vector space, the
semantics of words are captured in an intuitive way. Even early word embedding
algorithms were capable of analogical reasoning (e.g., the vector obtained
from the operation $\vec{King}-\vec{Man}+\vec{Woman}$ was found to be closest
to the vector $\vec{Queen}$). What is impressive about these embeddings is
that the neural network is able to learn such patterns simply by ‘reading’ the
text rather than requiring explicitly labeled data from humans. Modern
representation learning algorithms, proposed in the last 5-7 years are now
capable of embedding sentences (including tweets), which can be used in
machine learning models to make task-specific predictions, given a small set
of tweets labeled by hand or using survey data [9], [12].

In this article, we consider a task that is especially relevant in digital
health; namely, the task of predicting vaccine hesitancy using public social
media data (with Twitter as the main example), at least in metropolitan areas
which are known for high tweeting activity, and where users tend to enable the
location facility on their phone compared to less urban milieus [13]. We are
not looking to predict vaccine hesitancy at an individual level, both because
it violates privacy but also because we cannot evaluate the accuracy of such
predictions without polling the individual. Rather, we seek a way of
predicting vaccine hesitancy at the _zip code-level_ that uses the text in
public, geolocated tweets, but that does not identify or isolate user data of
any kind. One advantage of making predictions at the zip code-level is that
predictions can be validated using independent survey data, such as the recent
poll conducted by Gallup where a question was posed to individuals that is
directly pertinent to the individual’s vaccine hesitancy. As detailed
subsequently, by averaging responses of individuals within a given
geographical zone (such as a zip code-demarcated region), we can obtain a
real-valued vaccine hesitancy estimate for that zip code.

With the appropriate performance metrics in place (that are subsequently
introduced and discussed in the _Evaluation Methodology and Metrics_ sub-
section within _Materials and Methods_), one obvious set of baselines that
does not rely on any social media data is to just predict a constant-valued
vaccine hesitancy estimate (such as 0.5 or 1.0, or even just the average
observed in survey data). We show that our machine learning models outperform
such measures, with our best machine learning models achieving a ten percent
relative improvement over the best constant-valued baseline (which itself
relies on ‘privileged’ information, namely the mean vaccine hesitancy observed
in the survey). Our models are practical and guided by real-world intuitions
e.g., we not only consider the text and hashtags directly observed in
geolocated tweets, but we also consider the use of NLP software for sentiment
analysis, as well as external data sources and features (such as the number of
hospitals or scientific establishments in a zip code) that may be relevant to
estimating vaccine hesitancy. Rather than propose a ‘winning’ model, we
provide a full comparison of the different models and features to better
understand the performance differences and tradeoffs.

The rest of this article proceeds as follows. We begin with a comprehensive
description of the _Materials and Methods_ used in our study, including
details on the Twitter-based dataset and its collection, data preprocessing,
feature extraction, vaccine hesitancy ground-truth (obtained from independent
survey data), evaluation methodology and metrics, and predictive models and
baselines. We only use open-source packages and public data in this study, to
enable maximal replicability and minimize cost. We follow the discussion on
materials and methods with a detailed summary of our key results, followed by
qualitative discussion and error analysis. Finally, we conclude the work with
a brief primer on possible future directions and caveats.

## Materials and Methods

### Twitter Dataset

We sample tweets related to the COVID-19 pandemic from the 9 most populous
metropolitan areas in the United States [14]. In order of decreasing order of
population size, these are: New York, Los Angeles, Chicago, Houston, Phoenix,
Philadelphia, San Antonio, San Diego, and Dallas. Our sampled tweets come from
the GeoCOV19Tweets dataset [15], which collects geo-tagged tweets related to
the COVID-19 pandemic daily based on certain COVID-specific keywords and
hashtags. The dataset also publishes corresponding sentiment scores for each
tweet. Only the tweet ID and sentiment scores are published online, in keeping
with Twitter’s terms and conditions.

In previous work [16], we ‘hydrated’ (or retrieved, in full) tweets from the
GeoCOV19Tweets dataset from March 20, 2020 through December 1, 2020 (255
days). The period from October 27, 2020 - October 28, 2020 was skipped because
no sentiment scores were published at the time of our sampling. From the
hydrated tweet object (a data structure described extensively in Twitter’s
developer documentation [17]), we extracted each tweet’s ‘coordinates’ object
to ensure that an exact location for the tweet can be determined. These
coordinates were then used to filter the tweets by metropolitan area. To do
so, we determined whether the tweet’s coordinates of origin fell inside a
manually-drawn ‘bounding box’ demarcating each of the metropolitan areas
listed above. If falling within a bounding box, the tweet was marked as
originating from the corresponding metropolitan area. The coordinates of these
bounding boxes can be found in our previous work, along with detailed
descriptions of our collection methodology [16].

In this study, we re-hydrate this collection of tweets using the _twarc_
library to save the tweet’s full text and tweet ID [18]. After removing any
archived tweets or tweets whose ‘coordinates’ object is no longer available,
we retain 45,899 tweets. We also collect every tweet’s zip code of origin
using an Application Programming Interface (API) provided by Geocodio [19].
Geocodio is a company that was founded in 2014 specifically to provide human-
readable location information (such as state, city and country) by taking as
input a pair of latitude-longitude coordinates.

We also eliminate all zip codes with fewer than 10 tweets, removing 4,799
tweets and 1,321 zip codes. We then merge the data with the zip code-level
attributes described subsequently in _Features from External Sources_ and
remove remaining rows with null values. This leaves us with a total of 29,458
tweets, each of which belongs to one of 493 unique zip codes across the 9
metropolitan areas listed above. None of the 29,458 tweets is a retweet. Table
1 shows the overall dataset statistics, including the number of hashtags
before and after text preprocessing (explained subsequently).

Table 1: _Overall dataset statistics per metropolitan area. We report hashtag data both before and after text preprocessing._ Metropolitan Area |  Num. Zip Codes |  Num. Tweets |  Avg. Tweets per Zip Code |  Num. Hashtags (before) |  Num. Hashtags (after) |  Num. Unique Hashtags (after)
---|---|---|---|---|---|---
New York |  152 |  12,612 |  82.974 |  41,232 |  40,419 |  10,764
Los Angeles |  148 |  10,532 |  71.162 |  37,030 |  36,507 |  12,422
Chicago |  43 |  1,544 |  35.907 |  3,857 |  3,792 |  1,891
Houston |  36 |  1,529 |  42.472 |  5,557 |  5,505 |  2,526
San Diego |  27 |  1,061 |  39.296 |  3,019 |  2,980 |  1,769
Philadelphia |  24 |  817 |  34.042 |  2,753 |  2,727 |  1,349
Dallas |  33 |  664 |  20.121 |  2,250 |  2,231 |  1,211
Phoenix |  20 |  496 |  24.80 |  1,576 |  1,563 |  1,003
San Antonio |  10 |  203 |  20.30 |  765 |  756 |  340
Total |  493 |  29,458 |  N/A |  98,039 |  96,480 |  N/A

### Text Preprocessing

Using a hydrated tweet’s full text, we tokenize, make lowercase, and remove
mentions using _TweetTokenizer_ [20] from the Natural Language Toolkit (NLTK)
package [21]. NLTK is a leading package in the NLP community that uses Python
programs to work with human language data. We also remove URLs, stop words,
tokens less than or equal to 1 character in length, and any characters other
than letters (including the # symbol and emojis). We use NLTK’s standard set
of English stop words [22] (such as ‘the’, ‘a’ etc.) but remove the words
_not, no, nor, very,_ and _most_ from this pre-determined set, as these are
assumed to be relevant for making more accurate vaccine hesitancy predictions.
We then lemmatize all tokens using _WordNetLemmatizer_ [23]. Note that after
our text preprocessing steps, the hashtags “covid19”, “covid”, “Covid19”, and
“covid-19” (for instance) all result in the same token, and any hashtags
consisting of numbers or single characters only (e.g. “#2020” or “#K”) are
eliminated. The processed tweets are then embedded using the fastText word
embedding (briefly introduced in _Background_) model, which was released by
Facebook AI research and contains word vectors trained on English Wikipedia
data [24], as explained further in _Predictive Models and Features_.

The number of hashtags in the tweets before text preprocessing in Table 1 is
computed by summing the occurrences of # in the full text. After text
preprocessing, when the hashtags and text are well-separated and more easily
analyzed, we count the number of times a token begins with the # symbol.

Note that, in this study, we avoid using the ‘hashtags’ object [25] returned
directly from the Twitter API for several reasons. First, the object appears
to already have certain filters applied such that number-only strings (e.g.
#2020) are eliminated. Even though our text preprocessing steps do so as well,
as mentioned above, the ‘hashtags’ object does not accurately represent the
number of hashtags in the original tweet, especially as it fails to accurately
count hashtags in a continuous string. For example, in a tweet from New York
containing “…#corona#coronavirus#quarantine#quarantinelife#washyourhands…”,
the ‘hashtags’ object returns an empty array. To maintain control over text
preprocessing and feature extraction, we therefore exclusively use and count
hashtags from the full text field in the tweet data dictionary [25] returned
from the Twitter API.

### Features from External Sources

As mentioned earlier in _Twitter Dataset_ , we obtain the zip code of origin
for every tweet included in our dataset, resulting in a total of 493 unique
zip codes. For all unique zip codes, we get additional zip code-level
information from external, publicly available data sources. These zip code-
level attributes, which we add as features in our predictive models (see
_Predictive Models and Features_), comprise the Zillow Home Value Index (ZHVI)
[26] and the numbers of establishments in the educational, healthcare, and
professional, scientific, or technical sectors. We incorporate these features
because they act as proxies for zip code affluence and resource availability.
The sentiment scores, which have tweet-level granularity, are obtained
directly from the GeoCOV19Tweets dataset [15], as noted earlier in _Twitter
Dataset_. Importantly, every tweet has its own sentiment score, but tweets
originating from the same zip code share identical zip code-level attributes
i.e., the zip code-level data are repeated for all tweets belonging to the
same zip code.

All features mentioned above are standardized, using the _StandardScaler_
function in Python’s scikit-learn package [27], in the train and test data
_separately_ , to prevent test data leakage into the training phase. We
provide more details about the train and test split in the next section. Table
2 summarizes each zip code-level feature as well as sentiment, but we also
provide more detailed descriptions of these features in the _SI Appendix_.

Table 2: _External features collected from publicly available data sources. All but sentiment score are zip code-level attributes, meaning that all tweets with the same zip code will have the same values for these features._ Feature |  Description
---|---
Sentiment Score |  Score between [-1, 1], where positive values signify positive sentiment and 0 means “neutral”. The greater the absolute value, the stronger the sentiment.
Zillow Home Value Index (ZHVI) |  Measure of the typical home value in a zip code (USD), capturing market value in addition to price.
Healthcare and Social Assistance |  Number of establishments that provide health care and social assistance to individuals.
Educational Services |  Number of establishments, either privately or publicly owned and either for-profit or not-for-profit, that provide instruction or training.
Professional, Scientific, and Technical Services |  Number of establishments that provide professional, scientific, and technical services that require a high level of expertise or training.

### Vaccine Hesitancy Ground Truth and Train / Test Split

Throughout this study, we refer to the list of the 493 unique zip codes and
their corresponding vaccine hesitancies as the ‘ground truth’. The vaccine
hesitancy values range from 0.0 to 1.0 on continuous scale, and represent the
percentage of surveyed residents who have indicated that they are hesitant
about receiving a COVID-19 vaccine.

Specifically, we leverage the 2020-21 vaccine hesitancy data collected through
the COVID-19 Gallup survey [28]. Gallup launched a survey on March 13, 2020
that collected people’s responses during the COVID-19 pandemic, polling daily
random samples of the Gallup Panel (a probability-based, nationally
representative panel of U.S. adults). A vaccine hesitancy question was asked
starting from July 20, 2020: _If an FDA-approved vaccine to prevent
coronavirus/COVID-19 was available right now at no cost, would you agree to be
vaccinated?_ with the possible response of _Yes_ or _No_ to investigate a
person’s willingness to be vaccinated. The proportion of _No_ responses in a
specific area is defined in this work as vaccine hesitancy. We calculate the
proportion of the _No_ answers to this question between July 20 and August 30,
2020 at the zip code-level to get a vaccine hesitancy score per zip code.

The mean vaccine hesitancy across all 493 unique zip codes corresponding to
our tweets was calculated to be 0.240. The standard deviation is 0.334,
showing that there is significant variance across zip codes, even when limited
to the largest metropolitan areas in the US. The minimum and maximum values
are 0.00 and 1.00, respectively, indicating complete vaccine acceptance and
hesitancy. Note that these ‘ground truth’ values exist at the zip code-level
and measure aggregates. A vaccine hesitancy of 0.5 in a zip code thus means
that, on average, half the people in that zip code are vaccine hesitant. While
we cannot say anything about an individual tweeter, for predictive modeling
purposes, we label a tweet originating from zip code $z$ with the ground-truth
vaccine hesitancy score corresponding to zip code $z$. This implies that, if
there are $k$ tweets from zip code $z$, then all k tweets are assigned the
same ‘pseudo’ vaccine hesitancy label. When reporting the results, for
completeness, we report metrics both at the tweet-level (which should be
assumed to be weakly labeled using the methodology above), as well as at the
zip code-level, which is the true measure of the performance of our system.

We apply a stratified split (using the _StratifiedShuffleSplit_ function in
the scikit-learn package [29]) to partition our dataset into train (80%) and
test (20%) sets based on zip code. This way, both the train and test sets
include tweets from all 493 zip codes in approximately equal proportions e.g.,
42.8% of both the train and test sets are tweets from the New York City
metropolitan area (since 42.8% of the overall tweets in our corpus are from
New York), and so on. Due to this stratified construction, both the train and
test sets include tweets from all 9 metropolitan areas. Overall, there are
23,566 tweets in the train set and 5,892 tweets in the test set.

### Evaluation Methodology and Metrics

All predictive models and baselines used in this study, and described in the
next two sections, are evaluated in two different ways: at the tweet-level and
at the zip code-level. The tweet-level evaluation is based on a vaccine
hesitancy prediction for every tweet in the test set (a total of 5,892
predictions), while the zip code-level evaluation relies on a single vaccine
hesitancy prediction per zip code (a total of 493 predictions). Our predictive
models, however, make predictions at the individual tweet-level only. To get
493 zip code-level predictions, we take the average of the tweet-level
predictions for every zip code. Formally, for $k$ tweets (in the test set)
belonging to zip code $z$ with predicted tweet-level vaccine hesitancies
$[\hat{y_{1}},\ldots,\hat{y_{k}}]$, the predicted vaccine hesitancy for zip
code $z$ is given by the formula:

$\hat{y_{z}}=\frac{\sum_{i=1}^{k}\hat{y_{i}}}{k}$ (1)

Both the tweet-level and zip code-level evaluations use the Root Mean Square
Error (RMSE) metric for performance validation and comparison. Given $m$ data
points with true (real-valued) vaccine hesitancy labels
$[y_{1},\ldots,y_{m}]$, and predicted labels
$[\hat{y_{1}},\ldots,\hat{y_{m}}]$, the RMSE is given by the formula:

$RMSE=\sqrt{\frac{\sum_{i=1}^{m}(y_{i}-\hat{y_{i}})^{2}}{m}}$ (2)

For the tweet-level evaluation, each of the $m$ data points represents a
tweet, while for the zip code-level evaluations, each data point represents a
zip code. Thus, in the tweet-level RMSE score calculation, the ‘pseudo’ tweet-
level vaccine hesitancy labels, the assignment of which was described in the
previous section, are compared with the tweet-level predictions obtained from
the model. In the zip code-level RMSE calculation, similarly, the ‘ground
truth’ vaccine hesitancies (obtained directly from Gallup) are compared with
the zip code-level predictions obtained from the models. The lower the RMSE
score, the lower the predictive error, and the better the model.

We emphasize that, although we train our predictive models at the tweet-level,
the tweet-level predictions are auxiliary to obtaining zip code-level
predicted vaccine hesitancies. This is because we cannot evaluate model
performance at the tweet-level when our ‘ground truth’ values have zip code-
level granularity. The vaccine hesitancy labels of the tweets should be
thought of as ‘pseudo’ (or ‘weak’) labels, and predictive performance at the
level of tweets is only reported for the sake of completeness. The primary
goal of this study, as discussed in _Background_ , is predicting zip code-
level vaccine hesitancies using publicly available individual tweets.

In addition to computing RMSE scores on the test set, for each predictive
model, we also report the average of the 5-fold cross-validated RMSE score (at
the tweet-level). To do so, we split the train set into five folds. The first
fold contains 4,714 tweets, while the other four folds each contain 4,713
tweets (adding up to 23,566 tweets total, which is the entirety of the train
set). Each fold is used as a ‘test’ set once, while the remaining four folds
act as the ‘train’ set. Because every fold is used as a ‘test’ set only once,
there are five training iterations (corresponding to the number of folds). At
every iteration, we obtain one RMSE score that represents the performance of
the model trained on four folds and evaluated on the fifth. Over all
iterations, therefore, we have 5 RMSE scores of which we report the average in
_Results_ as a measure of model _robustness_ i.e., to further verify that the
reported tweet-level RMSE values are not the result of luck on the actual test
set (containing 5,892 tweets).

We do not report the average of the 5-fold cross-validated RMSE scores on the
zip code-level because cross-validation is computed during training, and, as
mentioned in the previous section, model training is done exclusively at the
tweet-level. Thus, we train and cross-validate the predictive models only at
the tweet-level, as a test of model robustness.

### Predictive Models and Features

As described in _Text Preprocessing_ , we use fastText’s word vectors trained
on English Wikipedia data to embed tweet text. The resulting vectors are
300-dimensional, and all dimensions are retained throughout the study. We
embed the processed full text in three different ways (corresponding to three
representations): one which includes _text only_ (no hashtags), one that
includes _text and hashtags_ , and one that contains hashtags only, if
available, and text otherwise. We refer to the latter as the _hybrid_
representation. For instance (ignoring any text transformations explained in
_Text Preprocessing_), the _text only_ representation of the tweet “Be back
soon my friends #corona #cov19 #notMyVirus #quarantinefitness” would embed
only the “Be back soon my friends” part. The _text and hashtags_
representation would incorporate the entire tweet, and the _hybrid_
representation would embed only “#corona #cov19 #notMyVirus
#quarantinefitness” because the tweet contains hashtags. Alternatively, the
_hybrid_ representation of the tweet “In the hospital not for Corona virus”
would embed the tweet’s text because no hashtags are provided. Compared to a
representation that uses only hashtags, the hybrid representation is more
robust since it uses the text if no hashtags are present.

For each of the three representations described above, we build four
predictive models (with a total of 12 models) that incorporate all zip code-
level features using support vector regression (SVR), linear regression, and
stochastic gradient descent (SGD). For SVR, we implement a radial basis
function (RBF) kernel as well as a linear kernel. All of these are established
models in the machine learning community; technical details can be found in
any standard text [30]. Using _SVR with RBF kernel_ , we also build three
predictive models (one for each representation) that do not incorporate any
zip code-level features. We chose _SVR with RBF kernel_ because, out of all 12
predictive models just mentioned, it performs the best across all
representations, as subsequently shown in _Results_. All predictive models,
both including and excluding zip code-level features, are also trained with
and without the sentiment feature. We report both scores because sentiment is
an external, tweet-level feature not computed or verified by us, but provided
by the GeoCOV19Tweets dataset, which is a super-set of our metropolitan data.

Apart from setting the maximum number of iterations in _SVR with linear
kernel_ to 4,000 and specifying a random state value (42) for _SVR with linear
kernel_ and SGD, we use default parameters for all 15 predictive models.
Recall from the previous section that for each of these models, we compute the
RMSE score on the tweet-level and the zip code-level, and we also report the
mean of the 5-fold cross-validated RMSE scores (for the tweet-level only).

### Baselines

To evaluate the predictive power of the 15 models described in the previous
section, we consider six _constant_ -value baselines that predict a single
value for each individual tweet (_tweet-level_) and each unique zip code (_zip
code-level_). The RMSE scores for the tweet-level baseline predictions are
measured with respect to the weakly labeled test set (5,892 tweets), while the
errors for the zip code-level baselines are computed with respect to the
original zip code-level ‘ground truth’ (493 zip codes). Our baselines do not
rely on sentiment, or on any text or zip code-level features. Table 3
summarizes the models.

Table 3: _Description of the six constant-value baseline models, along with the corresponding vaccine hesitancy value predicted for all tweets and zip codes (for that baseline)._ Baseline System |  Description |  Constant-Value Vaccine Hesitancy
---|---|---
No vaccine hesitancy |  Predicts that none of the zip codes are vaccine hesitant |  0.0
Complete vaccine hesitancy |  Predicts that all zip codes are completely vaccine hesitant |  1.0
Partial vaccine hesitancy |  Predicts that all zip codes are partially vaccine hesitant |  0.5
Mean ‘pseudo’ vaccine hesitancy in train set |  Predicts that all zip codes have a vaccine hesitancy equal to the average vaccine hesitancy of only the train set tweets (23,566 samples) |  0.2349
Mean ‘pseudo’ vaccine hesitancy for all tweets |  Predicts that all zip codes have a vaccine hesitancy equal to the average vaccine hesitancy of all tweets (29,458 samples) |  0.2349
Mean vaccine hesitancy in Gallup ground-truth |  Predicts that all zip codes have a vaccine hesitancy equal to the average vaccine hesitancy of the ground-truth values (493 samples) |  0.2403

Concerning the last three baselines, both the mean vaccine hesitancy in the
train set and the mean vaccine hesitancy for all tweets are weighted by the
frequencies of the zip codes in our dataset. Additionally, the baseline
relying on information from the ground-truth is highly optimistic because it
assumes that this information is known. Even the previous two baselines (at
the tweet-level) rely on this information since the pseudo-label relies on the
zip code-level label, which is obtained from the ground-truth. In reality, the
average vaccine hesitancy of the ground truth, the train set, or the entire
dataset will not be available, since that is precisely what we are trying at
predict at the zip code-level. In subsequent sections, we refer to these
baselines as the ‘optimistic’ baselines in contrast with the first three
‘realistic’ baselines, which assume a manually specified constant value.

## Results

The RMSE metrics tabulated in Table 4 show that all predictive models
outperform the best-performing, realistic baseline (“no vaccine hesitancy”
with an RMSE of 0.411) at the zip code-level, although not all perform better
than the most optimistic baselines (RMSE of 0.334). Specifically, no model has
an RMSE score below 0.334, our optimistic baseline value, except for the _SVR
with RBF kernel_ models (without sentiment).

Table 4: _The Root Mean Square Error (RMSE) scores at both the tweet-level and zip code-level for all models along with the average of the 5-fold cross-validated RMSE scores (for the predictive models)._ Models specified with (*) do not include any zip code-level features. The others include all zip code-level features in addition to text. The RMSE scores for the predictive models with and without sentiment as a feature are reported as “without sentiment / with sentiment”. Cross-validation is not applicable for the baseline models. Representation |  Model |  Tweet-Level RMSE |  Mean of 5-fold Cross-Validation RMSE Scores |  Zip Code-level RMSE
---|---|---|---|---
Text Only |  SVR (RBF kernel) |  0.206 / 0.211 |  0.209 / 0.214 |  0.312 / 0.316
|  SVR (RBF kernel)* |  0.271 / 0.290 |  0.270 / 0.289 |  0.308 / 0.329
|  SVR (linear kernel) |  0.297 / 0.297 |  0.295 / 0.295 |  0.374 / 0.374
|  Linear regression |  0.280 / 0.280 |  0.279 / 0.279 |  0.336 / 0.336
|  SGD regressor |  0.297 / 0.297 |  0.299 / 0.299 |  0.343 / 0.343
Text and Hashtags |  SVR (RBF kernel) |  0.208 / 0.213 |  0.210 / 0.215 |  0.315 / 0.318
|  SVR (RBF kernel)* |  0.269 / 0.291 |  0.267 / 0.289 |  0.314 / 0.335
|  SVR (linear kernel) |  0.297 / 0.297 |  0.295 / 0.295 |  0.374 / 0.373
|  Linear regression |  0.280 / 0.280 |  0.278 / 0.278 |  0.335 / 0.355
|  SGD regressor |  0.300 / 0.300 |  0.301 / 0.301 |  0.343 / 0.343
Hybrid |  SVR (RBF kernel) |  0.213 / 0.217 |  0.213 / 0.219 |  0.319 / 0.320
|  SVR (RBF kernel)* |  0.287 / 0.287 |  0.280 / 0.297 |  0.322 / 0.322
|  SVR (linear kernel) |  0.303 / 0.303 |  0.309 / 0.309 |  0.365 / 0.365
|  Linear regression |  0.292 / 0.292 |  0.290 / 0.290 |  0.334 / 0.334
|  SGD regressor |  0.307 / 0.307 |  0.299 / 0.299 |  0.341 / 0.341
Constant-Value Baselines |  No vac. hes. |  0.399 |  N/a |  0.411
|  Complete vaccine hesitancy |  0.830 |  N/A |  0.830
|  Partial vaccine hesitancy. |  0.418 |  N/A |  0.423
|  Mean ‘pseudo’ vaccine hesitancy in train set |  0.322 |  N/A |  0.334
|  Mean ‘pseudo’ vaccine hesitancy for all tweets |  0.322 |  N/A |  0.334
|  Mean vaccine hesitancy in Gallup ground-truth |  0.322 |  N/A |  0.334

For most predictive models, there is no difference in performance when
sentiment is included versus when it is excluded. However, _SVR with RBF
kernel_ shows several noteworthy improvements when sentiment is omitted as a
feature. Without sentiment, _SVR with RBF kernel_ is the only model that
outperforms even the most optimistic baselines across all 3 representations
(_text only, text and hashtags_ , and _hybrid_) regardless of whether zip
code-level features are included. The only time the _SVR with RBF kernel_
model performs worse than our optimistic baselines is when the _text and
hashtags_ representation is used and when sentiment is included but all zip
code-level features are excluded (RMSE of 0.335). However, this is only a 0.1%
increase in RMSE compared with the optimistic baselines.

We observe the lowest zip code-level RMSE score (0.308) through the _SVR with
RBF kernel_ model (_text only_ representation) when no features other than the
text are included (i.e., no zip code-level features and no sentiment). This is
a 7.78% improvement from the optimistic baselines, and a 25.06% improvement
from the best-performing, realistic baseline (“no vaccine hesitancy”).
Compared with the “complete vaccine hesitancy” baseline, this model shows a
62.89% improvement, and compared with the “partial vaccine hesitancy”
baseline, we observe a 27.19% improvement. When zip code-level features are
added, the _text only_ representation performs with an RMSE of 0.312, yielding
a 6.59% improvement compared with the optimistic baselines and a 24.09%
improvement compared with the “no vaccine hesitancy” baseline. Table 4 shows
that adding sentiment to these two models in the _text only_ representation
reduces performance at the zip code-level.

Interestingly, adding sentiment as the only external feature to the text
embeddings (no zip code-level attributes) reduces performance greatly for the
_text only_ and _text and hybrid_ representations using the _SVR with RBF
kernel_ model. For the former, performance is reduced by 6.38%, while for the
latter, performance decreases by 6.27%. For the _hybrid_ representation, on
the other hand, the RMSE score is unaffected.

In addition to the results presented in Table 4, we also built an _SVR with
RBF kernel_ model that predicts zip code-level vaccine hesitancy based on zip
code-level data only (no text), both with and without sentiment. When
predicting based on zip code-level data only, this model achieves an RMSE of
0.332, and when sentiment is added, it achieves an RMSE of 0.333. Both
outperform the most realistic baseline model (“no vaccine hesitancy”) with
19.22% and 18.98% performance improvements, respectively. They also marginally
outperform our optimistic baselines, but only with 0.60% and 0.30% decreases
in RMSE scores, respectively. This suggests that even when relying on only the
number of establishments in the healthcare, educational, and professional,
scientific, or technology sectors as well as the Zillow Home Value Index
(ZHVI), it is possible to predict zip code-level vaccine hesitancy with a
marginally lower error compared with constant-value baselines.

However, as reported above, the addition of tweet text leads to lower error
for our zip code-level predictions. Our best-performing model shows a 25.06%
improvement (compared with only 19.22% when no text is used) over a “no
vaccine hesitancy” constant-value baseline and a 62.89% improvement over a
“complete vaccine hesitancy” constant-value baseline. This indicates that the
use of tweet text (particularly in models implementing the _text only_
representation and without any external features) is a powerful indicator of
zip code-level vaccine hesitancy. The evidence suggests that tweet-text
contains more signal than noise, on average, insofar as predicting vaccine
hesitancy is concerned.

## Discussion and Error Analysis

Although the machine learning-based models presented in the previous sections
clearly outperform constant-valued baselines, it is useful to consider the
issue from a qualitative lens, both to understand the kinds of tweets that
contain either signal or noise, as well as to understand sources of prediction
error. To do so, in this section, we sample some tweets from our complete
dataset prior to any text preprocessing. We include only the text and
hashtags, i.e. we exclude any mentions, hyperlinks, or locations for brevity.
Sampled tweets labeled with a vaccine hesitancy of 1.0 (complete vaccine
hesitancy) are enumerated below:

  1. 1.

“Why is COVID-19 mild for some, deadly for others?” (Chicago)

  2. 2.

“Corona quarantine day 19. Trapped on the beach. #chickendinner #familytime
#corona #coronatine #beachlife” (San Diego)

  3. 3.

“D.C. residents are encouraged to continue practicing social distancing and
take actions to help prevent the spread of COVID-19 [PR] Coronavirus Data
Update: March 24” (New York)

  4. 4.

“New comfortable and colorful masks for coronavirus protection while you are
out and about #craftfoundry #covid19masks #covid19masksforsale
#coronavirusprotection #coronavirusprotectionmask #cottonmask” (Philadelphia)

We also sampled tweets labeled with a predicted vaccine hesitancy of 0.5 by
the best model (partial vaccine hesitancy):

  1. 1.

“An emotional week for everyone. Everyone keep fighting. Day by day #fighting”
(Los Angeles)

  2. 2.

“Please no close talking [emoji] practice social distancing #corona #codin19
#thistooshallpass #seinfeld #seinfeldmemes #seinfeldquotes” (New York)

  3. 3.

“good thing the drive-in is open and an easy place for social distancing.
#JurassicPark #drivein #movienight #family” (San Antonio)

  4. 4.

“Covid 19 Testing at First Met! #firstmetchurch #covidtesting
#sheilajacksonlee” (Houston)

Finally, we sampled tweets predicted with a vaccine hesitancy of 0.0 (no
vaccine hesitancy):

  1. 1.

“Catch me if you can but don’t get burned. #charmander #pokemon
#socialdistancing #corona #onesie” (Chicago)

  2. 2.

“#GivingGratitude to our #frontlineworkers who are working to keep us safe
everyday. #weappreciateyou #covid19 #2020 #amberwavesrealestategroup
#kellerwilliamsdmn #weareinthistogether #shelterinplace #stayhome” (Dallas)

  3. 3.

“This is how you know there’s a problem? When is the last time you saw gas
prices this low?! #cov?d19 #corona” (Philadelphia)

  4. 4.

“I’ve got this pit in my stomach at whats happening in california and all of
the USA during this corona virus pandemic especially as homeless with severe
epilepsy makes this worse.” (Los Angeles)

These examples illustrate why it’s important to measure vaccine hesitancy at
an aggregate level, such as zip code, rather than at the individual level
using only tweets. Aside from the obvious ethical concerns, it is unlikely
that extreme cases of vaccine hesitancy or non-hesitancy will manifest on
Twitter for the vast majority of the population (although a full study of this
hypothesis merits future research). However, at the zip code-level, the
predictions are more promising.

As presented earlier in Table 4, we showed that the _text only_ representation
of the _SVR with RBF kernel_ model that excludes sentiment and any zip code-
level features predicts zip code-level vaccine hesitancies with an RMSE of
0.308. This model, however, tends to overestimate vaccine hesitancy. In 291
zip codes (out of 493), we find that the predicted vaccine hesitancy is
greater than the true vaccine hesitancy. In the remaining 202 zip codes, the
predicted vaccine hesitancy is lower than the ‘ground truth’ value. For 50 zip
codes, the predicted vaccine hesitancy is overestimated by 0.20 or more, and
when looking at the difference between the predicted and true vaccine
hesitancies (absolute value), we find that the gap is 0.20 or more in 179 zip
codes.

For instance, in Philadelphia, 83.33% of the zip code-level predictions are
over-estimates (the highest among all metropolitan areas). In New York, the
metropolitan area with the highest proportion of tweets and the highest
average number of tweets per zip code in our study, the predicted vaccine
hesitancies for 60.53% of zip codes are overestimated. In fact, in all
metropolitan areas except for Phoenix, San Diego, and Dallas, the vaccine
hesitancy in more than 50% of the zip codes is overestimated. In Phoenix,
exactly half of all zip code-level vaccine hesitancies are over-estimates. For
the remaining metropolitan areas, the percentages of zip codes having
overestimated vaccine hesitancies fall between 58.11-62.79%.

Despite the large proportion of overestimated vaccine hesitancies presented
above, we observe that only 16.67% and 9.21% of the zip codes in Philadelphia
and New York, respectively, have overestimated vaccine hesitancies greater
than or equal to 0.20. The metropolitan area with the greatest proportion of
zip codes with overestimated vaccine hesitancies by 0.20 or more is San
Antonio (30%), which is also the metropolitan area with the fewest number of
tweets. San Diego, on the other hand, has only one zip code (3.70%) whose
vaccine hesitancy is overestimated by 0.20 points or more.

Lastly, despite the large proportion of overestimates, we observe that
Philadelphia, in fact, has the smallest proportion of zip codes (25%) in which
the difference between the predicted and true vaccine hesitancies (absolute
value) is greater than or equal to 0.20. Thus, using this metric, we note that
our _SVR with RBF kernel_ model performed best in Philadelphia. On the
contrary, the model is worst performing in Dallas, with 57.58% of zip codes
showing a $\geq 0.20$ difference (absolute value) between the true and
predicted vaccine hesitancies. For all other metropolitan areas, the
proportion of zip codes whose predicted vaccine hesitancy differs by 0.20 or
more points from the true vaccine hesitancy falls between 29.73-50%. Overall,
the results suggest that the methods presented herein should not be used for
highly sensitive predictions, but the low gap between the true hesitancy and
predicted hesitancy in many zip codes (especially compared to ‘prior constant’
baselines, all of which performed worse than the SVR method) suggests that the
method can potentially be used both by social scientists, as well as by
digital health experts as an early warning system.

## Conclusion

With significant (albeit declining) vaccine hesitancy continuing to exist in
geographic and demographic segments of the adult population, and the expense
of conducting detailed and representative surveys, there is need for cheaper
and more real-time measures of predicting vaccine hesitancy without violating
privacy. In this article, we explored public social media as a potential
source of such information. Without identifying users, our models simply use
the text and hashtags in tweets to make a prediction at a zip-code level. We
validate the predictions using actual survey data as a gold standard.
Experimental results show that such an endeavor is not only feasible, but also
reasonably robust i.e., most of our models performed well compared to
‘constant-valued’ baselines even in the presence of irrelevant or non-vaccine
related (or even non-COVID related) tweets. Of course, the caveat applies that
such a system cannot be a substitute for comprehensive and representative
surveys, and it is likely more reliable for urban areas (toward which social
media platforms like Twitter tend to be heavily biased in their user-base),
especially in primarily-English speaking developed nations like the United
States, than for rural areas or for developing nations. However, by serving as
a supplementary source of information, such a system may help the public
health community to detect clusters of vaccine hesitancy and mitigate it with
communication and outreach. We believe that it can serve as a valuable and
inexpensive asset in a nation’s digital health infrastructure, especially as
more of the world comes online, and more people start to engage with social
media. In future work, we hope to replicate and extend this work for other
cities, countries and languages.

#### S1 Appendix: External Tweet-Level and Zip Code-Level Features

Sentiment Score. We retain the original sentiment scores included in the
GeoCOV19Tweets dataset [15] generated using the TextBlob sentiment analysis
tool [31]. In this dataset, every tweet is given a continuous value score
between [-1, 1], where positive values signify positive sentiment and 0 means
“neutral”. The more positive or negative the value, the stronger the
sentiment. Prior to computing these sentiment scores, hashtag symbols (#),
mention symbols (@), URLs, extra spaces, and paragraph breaks were eliminated.
Punctuation, emojis, and numbers were included.

Zillow Home Value Index (ZHVI). The Zillow Home Value Index (ZHVI) is a
measure of the typical home value for a region; in this case, zip code. It
captures monthly changes in Zestimates [32], which are Zillow’s estimated home
market values that incorporate house characteristics, market data such as
listing prices of comparable of homes and their time on the market, as well as
off-market data including tax assessment and public records. It also
incorporates market appreciation. In this study, we take the average of the
smoothed, seasonally adjusted value in the 35th to 65th percentile range (mid-
tier) from January through December 2020.

Establishments. Data about the number of establishments per zip code is taken
from the 2018 Annual Economic Surveys from the US Census (Table ID CB1800ZBP)
[33]. We take the “Health care and social assistance”, “Educational services”,
and “Professional, scientific, and technical services” data, which have the
following meaning:

  1. 1.

Healthcare and social assistance (sector 62) comprises establishments
providing health care and social assistance for individuals [34] e.g.,
physician offices, dentists, mental health practitioners, outpatient care
centers, ambulance services, etc. [35].

  2. 2.

Educational services (sector 61) consist of establishments that provide
instruction or training in a wide variety of subjects. The sector includes
both privately and publicly owned institutions and both for profit and not for
profit establishments [36] e.g., elementary and secondary schools, colleges,
universities, computer training, professional schools, driving schools, etc.
[37].

  3. 3.

Professional, scientific, and technical services (sector 54) include
establishments that specialize in providing professional, scientific, and
technical services that require a high level of expertise or training [38]
e.g., legal services, notaries, accounting, architectural services, building
inspection, engineering services, scientific consulting, research and
development, advertising, etc. [39].

## Acknowledgments

N/A

## References

  * 1. Khubchandani J, Sharma S, Price JH, Wiblishauser MJ, Sharma M, Webb FJ.  COVID-19 vaccination hesitancy in the United States: a rapid national assessment.  Journal of Community Health. 2021;46(2):270–277.
  * 2. VACCINE HESITANCY FOR COVID-19: PUMA ESTIMATES;.  https://aspe.hhs.gov/pdf-report/vaccine-hesitancy-covid-19-puma-estimates.
  * 3. Fridman A, Gershon R, Gneezy A.  COVID-19 and vaccine hesitancy: A longitudinal study.  PloS one. 2021;16(4):e0250123.
  * 4. Momplaisir F, Haynes N, Nkwihoreze H, Nelson M, Werner RM, Jemmott J.  Understanding drivers of COVID-19 vaccine hesitancy among Blacks.  Clinical infectious diseases: an official publication of the Infectious Diseases Society of America. 2021;.
  * 5. Kempe A, Saville AW, Albertin C, Zimet G, Breck A, Helmkamp L, et al.  Parental hesitancy about routine childhood and influenza vaccinations: a national survey.  Pediatrics. 2020;146(1).
  * 6. Brennen JS, Simon F, Howard PN, Nielsen RK.  Types, sources, and claims of COVID-19 misinformation.  Reuters Institute. 2020;7:3–1.
  * 7. Enders AM, Uscinski JE, Klofstad C, Stoler J.  The different forms of COVID-19 misinformation and their consequences.  The Harvard Kennedy School Misinformation Review. 2020;.
  * 8. Roundup of Gallup COVID-19 Coverage;.  https://news.gallup.com/opinion/gallup/308126/roundup-gallup-covid-coverage.aspx.
  * 9. Devlin J, Chang MW, Lee K, Toutanova K.  Bert: Pre-training of deep bidirectional transformers for language understanding.  arXiv preprint arXiv:181004805. 2018;.
  * 10. Mikolov T, Grave E, Bojanowski P, Puhrsch C, Joulin A.  Advances in pre-training distributed word representations.  arXiv preprint arXiv:171209405. 2017;.
  * 11. Joulin A, Grave E, Bojanowski P, Mikolov T.  Bag of tricks for efficient text classification.  arXiv preprint arXiv:160701759. 2016;.
  * 12. Floridi L, Chiriatti M.  GPT-3: Its nature, scope, limits, and consequences.  Minds and Machines. 2020;30(4):681–694.
  * 13. Malik M, Lamba H, Nakos C, Pfeffer J.  Population bias in geotagged tweets.  In: Proceedings of the International AAAI Conference on Web and Social Media. vol. 9; 2015.
  * 14. Largest Cities in the United States by Population;.  https://ballotpedia.org/Largest_cities_in_the_United_States_by_population.
  * 15. Lamsal R.  Coronavirus (covid-19) tweets dataset.  IEEE Dataport. 2020;10.
  * 16. Melotte S, Kejriwal M.  A Geo-Tagged COVID-19 Twitter Dataset for 10 North American Metropolitan Areas over a 255-Day Period. 2021;.
  * 17. Twitter Developer Platform: Docs;.  https://developer.twitter.com/en/docs.
  * 18. Twarc;.  https://scholarslab.github.io/learn-twarc/.
  * 19. Geocodio;.  https://www.geocod.io/features/api/.
  * 20. TweetTokenizer;.  https://www.nltk.org/api/nltk.tokenize.html.
  * 21. Natural Language Tool Kit;.  https://www.nltk.org/index.html.
  * 22. NLTK Stop Words;.  http://www.nltk.org/nltk_data/.
  * 23. WordNetLemmatizer;.  http://www.nltk.org/api/nltk.stem.html.
  * 24. FastText Pre-trained Vectors;.  https://github.com/facebookresearch/fastText/blob/master/docs/pretrained-vectors.md.
  * 25. Twitter Data Dictionary: Object Model;.  https://developer.twitter.com/en/docs/twitter-api/v1/data-dictionary/object-model.
  * 26. ZHVI;.  https://www.zillow.com/research/zhvi-user-guide/.
  * 27. StandardScaler;.  https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html.
  * 28. Roundup Gallup Covid Coverage;.  https://news.gallup.com/opinion/gallup/308126/roundup-gallup-covid-coverage.aspx.
  * 29. StratifiedShuffleSplit;.  https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.StratifiedShuffleSplit.html.
  * 30. Bishop CM.  Pattern recognition and machine learning.  springer; 2006.
  * 31. Loria S.  textblob Documentation.  Release 015. 2018;2.
  * 32. ZHVI Methodology;.  https://www.zillow.com/research/zhvi-methodology-2019-highlights-26221/.
  * 33. All Sectors: ZIP Code Business Patterns by Employment Size Class for 5-digit zipcode level: 2018;.  https://data.census.gov/cedsci/table?q=CB1800ZBP&tid=ZBP2018.CB1800ZBP.
  * 34. Establishments providing health care and social assistance for individuals;.  https://www.bls.gov/iag/tgs/iag62.htm.
  * 35. NAICS: Healthcare;.  https://www.census.gov/naics/?input=62&chart=2017.
  * 36. Establishments that provide instruction or training in a wide variety of subjects;.  https://www.bls.gov/iag/tgs/iag61.htm.
  * 37. NAICS: Education;.  https://www.census.gov/naics/?input=61&chart=2017.
  * 38. Establishments that specialize in providing professional, scientific, and technical services that require a high level of expertise or training;.  https://www.bls.gov/iag/tgs/iag54.htm.
  * 39. NAICS: Professional, Scientific and Technical Services;.  https://www.census.gov/naics/?input=54&chart=2017.