# Quantum Analogues for Two Simple Classical Channels

Miles Miller-Dickson  School of Engineering,
Brown University
Providence, RI 02912
<EMAIL_ADDRESS>Christopher Rose  School of Engineering,
Brown University
Providence, RI 02912
<EMAIL_ADDRESS>
###### Abstract

We present some of the peculiar dynamics of two simple sans-entanglement
quantum communication channels in a digestible form. Specifically, we contrast
the classical gaussian additive channel to its quantum analogue and find that
the quantum version features a capacity with interesting time dependence and
counterintuitive effects of quantization. We also consider a simple two-level
system and comment on the time dependence of its capacity.

###### Abstract

Here we include some derivations that were not in the main text. We also
include a primer on quantum mechanics for those who are less familiar with the
subject.

##  I Introduction

Quantum channels refer to the transmission of classical or quantum information
involving the exchange of quantum states (and sometimes classical states as
well). In contrast to classical channels, quantum channels may make use of
entangled particles or superpositions of quantum bits, or qubits, to transmit
information. These channels have an extensive literature describing their
information capacity, their limitations and their advantages over classical
transmission [2, 13, 10, 8]. Typically, quantum channels are described
mathematically using density matrices and trace-preserving maps. Here, we
present a simplified discussion of two kinds of quantum channels that only
assumes basic knowledge of quantum mechanics. Our aim is to directly compare,
where possible, classical channels to their quantum analogues and to reach an
audience who may not be familiar with the density matrix approach to quantum
mechanics.

Quantum mechanics has grown from its esoteric origins to an essential part of
pop culture. This emergence is partly due to enormous non-scientific
popularization, but it is also due to recent explosive development in quantum
technologies such as quantum sensing, computation, cryptogrphy, etc. [12, 16].
The rise in prevalence of these technologies is bringing with it a broader
demand to appreciate the subtleties of quantum systems. The topic of quantum
channels, in particular, has far reaching implications for the effectiveness
of many of these technologies, as they can often be framed as communication
problems. Among these subtleties are the counterintuitive effects of
entanglement. Even before these effects, however, are the more basic effects
of interference (superposition), which are alone already responsible for so
much of the “strangeness” of the subject.

In this paper, we consider precisely some of those strange effects in the
context of two basic channels. The first is a quantum analogue of the additive
Gaussian channel. The second is a two-level system which provides a comparison
to single bit transmission in a classical setting. In section II, we introduce
the first of these quantum channels and compute its capacity. We present the
second channel in section III. And section IV concludes with a discussion on
the comparison of these channels to their classical counterparts. In an
attempt to keep this work self-contained, we include a brief primer in an
appendix on the elements of quantum mechanics that are relevant for this
discussion. We also include in the appendix derivations of some results in the
main text.

##  II Gaussian channel analogue

Let us consider the classical additive Gaussian channel, framed as

$Y=X+Z$ (1)

$X\in\mathbb{R}$ is the channel input (sent by Alice), $Y\in\mathbb{R}$ is the
channel output (received by Bob) and $Z\in\mathbb{R}$ is a zero-mean Gaussian
random variable, independent of $X$, interpreted as noise. Alice codes
messages as sequences (blocks) of $X$’s, each of which is corrupted by a
sequence of $Z$’s so that Bob receives sequences of $Y$’s. Bob then decodes
the sequences of $Y$’s to recover the messages. It is also usually assumed
that $X$ is constrained ( $E[X^{2}]\propto\mbox{power}$). In the simplest
case, the sequence of $Z$’s are independent and identically distributed
(i.i.d).

Figure 1: Particle Placement Channel: a particle emerges at the origin at
$\tau=0$ and is moved by Alice to $X$ by time $\tau=T$. Bob measures the
particle position $Y$ at $\tau=T+t$ and removes it. This process repeats with
period $T+t$ to produce the tuple sequence, $\\{X_{k},Y_{k}\\}$.

In our toy quantum channel we use a simple model to describe the physical
system. Alice prepares a message by choosing a sequence of locations along
some axis at which to place a particle. Bob then measures the position of each
particle some specified time delay $t$ after it is placed and removes it from
the system. This channel is illustrated in FIGURE 1.

We then consider the simplest model for the quantum dynamics—that of the free
particle. To determine the capacity of this channel, we will compute the
probability that Bob measures the particle at some location $Y$ along the axis
some time $t$ after Alice places the particle at an initial location $X$. This
process is repeated for each particle placement.

We will assume that Alice can place the particle at some location only up to
some finite precision, limited by the nature of the apparatus. This is
effectively the quantization precision that would be present in practice even
for the classical channel. We will characterize this placement precision using
the variance of the initial particle wavefunction constructed by
Alice—technically it is the variance of the distribution $|\psi(x)|^{2}$. A
lower wavefunction variance implies higher precision in Alice’s placement.
Classically, increasing the quantization precision has diminishing returns for
the capacity [11], although for our quantum system we will find that in
addition to the usual diminishing returns, increasing precision can actually
reduce the capacity of the channel.

###  II-A Time evolution of the Gaussian particle

For computational ease, we assume that the placement of a particle by Alice is
described by a Gaussian wavefunction with some variance $\sigma_{A}^{2}$. The
exact wavefunction that Alice constructs will not disturb the qualitative
aspects of our results, so long as the wavefunction is localized to some
degree about a point.

$\psi(x)=\frac{1}{(2\pi\sigma_{A}^{2})^{1/4}}e^{-(x-x_{0})^{2}/4\sigma_{A}^{2}}$
(2)

Notice that $|\psi(x)|^{2}$ is a Gaussian distribution centered at $x_{0}$
with variance $\sigma_{A}^{2}$. $\psi(x)$ is a purely real-valued expression,
but it could have any complex phase factor scaling it, such as $\exp(ig(x))$
where $g(x)$ is some real-valued function of $x$—this factor would not disturb
our interpretation of $\psi(x)$ as a particle localized at $x_{0}$ since the
modulus is unaffected by such a factor. The Gaussian choice is particularly
convenient because a Gaussian wavefunction will evolve into a Gaussian with
expanding variance.

In the appendix, we calculate the time-evolution of Alice’s initial state and
take the squared modulus, giving the probability density for locating the
particle at point $x$ at time $t$. The result is given by,

$|\psi(x,t)|^{2}=\frac{1}{\sqrt{2\pi\Delta_{t}^{2}}}\exp{\left(\frac{-(x-x_{0})^{2}}{2\Delta_{t}^{2}}\right)}$
(3)

where $\Delta_{t}^{2}$ is given by (for $t\geq 0$),

$\Delta_{t}^{2}=\sigma_{A}^{2}+\left(\frac{\hbar t}{2m\sigma_{A}}\right)^{2}.$
(4)

Thus, an initial Gaussian evolves into a Gaussian with a variance that grows
quadratically in time. As a point of contrast, the distribution over the
position of a particle experiencing Brownian motion (random walks), has a
variance that grows linearly in time [17]. As we will see in the next section,
if Alice can set the initial variance of the particle’s position to be
proportional to $t$, the time that Bob waits to make a measurement, it will be
possible to recover a noise variance that only grows linearly in $t$, just as
in the Brownian case.

###  II-B Capacity of the Quantum Gaussian Channel

In the previous section, we computed the probability density that Bob will
locate the particle at a position some time after Alice placed it along an
axis. In what follows, we will use the variable $X$ to refer to Alice’s choice
in the initial particle location, and we will use the variable $Y$ to denote
the r.v. associated with Bob’s measurement at some specified time later. Thus
in the language of communication theorists, we will write $|\psi(y,t)|^{2}$ as
$f_{Y|X}(y|x)$, the transition probability for our channel (where we have
suppressed the dependence of the wavefunction on Alice’s initial particle
position $x=x_{0}$). And we use $f_{X}(x)$ and $f_{Y}(y)$ to refer to the
probability density over Alice’s choice of $X$, and the density associated
with Bob’s measurement, respectively. We will assume that the variance of $X$
is bounded by,

$\mathbb{E}[X^{2}]\leq P$ (5)

for some constant $P$. Note that $P$ has dimensions of ${(length)}^{2}$. Thus,
$\sqrt{P}$ characterizes the length scale over which Alice can place her
particle on the axis—we will regard $P$ as a measure of signal strength.

The capacity $C$ is given by maximizing the mutual information over $f_{X}$,
subject to this quadratic constraint on $f_{X}$, [5]

$C=\max_{f_{X},\mathbb{E}[X^{2}]\leq P^{\prime}}I(X;Y).$ (6)

Mutual information can be written as $I(X;Y)=H(Y)-H(Y|X)$, and $H(Y|X)=\int
dxf_{X}(x)H(Y|X=x)$. But since $f_{Y|X}$ is Gaussian with variance
$\Delta_{t}^{2}$, then $H(Y|X=x)=\frac{1}{2}\ln{2\pi e\Delta_{t}^{2}}$, which
is independent of $x$ [5]. Thus,

$H(Y|X)=\frac{1}{2}\ln{2\pi e\Delta_{t}^{2}}$ (7)

So maximizing $I(X;Y)$ is equivalent to maximizing $H(Y)$.

Now we make an observation about the form that $f_{Y}(y)$ takes. Since
$f_{Y|X}(y|x)$ is Gaussian, then $f_{Y}(y)$ is given by,

$\begin{array}[]{rcl}f_{Y}(y)&=&\int dxf_{X}(x)f_{Y|X}(y|x)\\\
&=&\frac{1}{\sqrt{2\pi\Delta_{t}^{2}}}\int
dxf_{X}(x)e^{-(y-x)^{2}/2\Delta_{t}^{2}}\end{array}$ (8)

which is precisely the same distribution as the classical additive Gaussian
channel with a “noise” variance $\Delta_{t}^{2}$. Thus, for a fixed
measurement delay $t$, our quantum analogue is equivalent to the classical
additive Gaussian channel. Really, this quantum version comprises a set of
classical Gaussian channels parameterized by the measurement delay $t$—this
follows directly from the fact that the transition probability $f_{Y|X}$ is
Gaussian.

Hence, $H(Y)$ is maximized by a Gaussian with variance $\Delta_{t}^{2}+P$,
just as it is in the classical case. This means that $H(Y)=\frac{1}{2}\ln{2\pi
e(\Delta_{t}^{2}+P)}$, and so the capacity in nats per channel use is given
by,

$\boxed{\begin{array}[]{rcl}C&=&\frac{1}{2}\ln{2\pi
e(\Delta_{t}^{2}+P)}-\frac{1}{2}\ln{2\pi e\Delta_{t}^{2}}\\\
&=&\frac{1}{2}\ln{\left(1+\frac{P}{\Delta_{t}^{2}}\right)}.\end{array}}$ (9)

which is superficially the form of the usual Gaussian channel with noise
variance $\Delta_{t}^{2}$. That this capacity does not exceed the classical
value follows from Holevo’s bound which essentially states that using quantum
states to encode classical information does not buy you any additional
capacity than using the corresponding classical states (at least not in the
manner suggested here, i.e. without making use of entanglement) [9, 10, 4].

The curious element of this capacity ultimately lies in the “noise” power
given by equation (4). Evidently if Bob waits longer to measure the particle
position, capacity decreases, which may not be so different from a classical
setting where one could model the noise term as being time-dependent. Still, a
natural model in the classical setting is the Brownian one we alluded to
earlier. But somehow the quantum noise grows even faster than this.

It is possible, however, for Alice and Bob to optimize the initial choice of
precision to reduce the effect of the noise as much as possible. Note that
reducing $\sigma_{A}^{2}$ can increase the noise power (equation (4)). So in
fact for a fixed measurement delay $t$, increased precision on Alice’s part
(reducing $\sigma_{A}^{2}$) can reduce the capacity. In particular when
$\sigma_{A}^{2}$ is reduced below a threshold variance $v^{*}$, given by

$v^{*}=\frac{\hbar t}{2m}$ (10)

then the capacity will be reduced, as can be seen in FIGURE 2. Of course the
capacity is also reduced by increasing $\sigma_{A}^{2}$, and so $v^{*}$
represents an optimal placement precision by Alice to maximize the capacity of
this channel, given a predetermined measurement delay $t$. For this choice of
$\sigma_{A}^{2}$, the noise power is given by $\Delta_{t}^{2}=\hbar t/m$,
which grows linearly with the time Bob waits to measure the particle. And so
as far as the scaling of the noise in time is concerned, Alice and Bob can
only communicate as well as one could in the case of Brownian noise.

Figure 2: Capacity of the Quantum Particle Placement Channel: capacity (in
nats) is plotted as in equation (9) with $v^{*}$ given by equation (10) for
different “signal to noise” ratios $P/v^{*}$.

##  III Two-level quantum channel

In a two-level channel, we imagine that Alice places some system in one of two
states. Let us generically refer to these states as $|0\rangle$ and
$|1\rangle$. These could be the groundstate and first excited state of an
atom, spin-$1/2$ states of an atom in a magnetic field, exciton states of
quantum dots, etc [6, 3]. One can also imagine placing a particle in one of
two potential energy wells, as shown in FIGURE 3. Then, some time $t$ later,
Bob will measure the state of the particle and either find $|0\rangle$ or
$|1\rangle$. In the case of our quantum Gaussian channel, we saw that Alice’s
initial state evolves over time, giving rise to “noise” in Bob’s
measurement—non-zero transition probabilities. Here, we model transitions
between the two levels using a parameter $\epsilon\in\mathbb{R}$ in a two-
level Hamiltonian of the form,

$\hat{H}=\begin{pmatrix}E&\epsilon\\\ \epsilon&E+\Delta\end{pmatrix}.$ (11)

When $\epsilon=0$, this corresponds to a two-level system without any
transitions between the levels. In that case the $|0\rangle=(1,0)^{T}$ state
has energy $E\in\mathbb{R}$, and the $|1\rangle=(0,1)^{T}$ has energy
$E+\Delta$, $\Delta>0$. For $\epsilon>0$, there is a non-zero probability for
transitions between the $|0\rangle$ and $|1\rangle$ states, which is sometimes
referred to as tunneling between the two states.

Figure 3: Two-State Channel: Alice sets the system in of two states
($|0\rangle$ or $|1\rangle$) and Bob will measure system state some time $t$
later. This process repeats independently at $t$-second intervals to produce
tuples $\\{|\psi_{kt}\rangle,\psi_{(k+1)t}\rangle\\}$.

###  III-A Time evolution of the two-level system

Here we assume Alice can set her initial state as,

$|\psi_{0}\rangle=\sqrt{1-p}|0\rangle+\sqrt{p}|1\rangle$ (12)

either with $p$ close to $0$ (so that $|\psi_{0}\rangle$ is close to
$|0\rangle$) or with $p$ close to $1$ (so that $|\psi_{0}\rangle$ is close to
$|1\rangle$). Here $p$ gives the initial probability of a measurement
resulting in the state $|1\rangle$, and so $1-p$ is the probability of
measuring the state $|0\rangle$. Hence, we can use $p(1-p)$ as a measure of
the variance for the corresponding distribution, $|\langle
z|\psi_{0}\rangle|^{2}$, where $z\in\\{0,1\\}$ (Bernoulli distribution).
$p(1-p)$ plays the role here that $\sigma_{A}^{2}$ played for the Gaussian
channel in section II, and similarly we will assume that it depends on the
fidelity of the equipment Alice uses to prepare her state.

In the appendix we compute the time evolution of this state. Taking the
modulus squared of each coefficient gives,

$|\langle
z|\psi_{t}\rangle|^{2}=\frac{1}{a^{2}}\begin{pmatrix}\zeta_{p}\cos(\frac{2at}{\hbar})+a^{2}(1-p)-\zeta_{p}\\\
-\zeta_{p}\cos(\frac{2at}{\hbar})+a^{2}p+\zeta_{p}\end{pmatrix}_{z},$ (13)

where,

$\zeta_{p}=\frac{\epsilon}{2}\left(\epsilon(1-2p)+2b\sqrt{p(1-p)}\right),$
(14)

for $z\in\\{0,1\\}$, representing the two states, and
$a=\sqrt{\Delta^{2}+4\epsilon^{2}}/2$.

Hence, we see that Bob’s likelihood of measuring either $|0\rangle$ or
$|1\rangle$ oscillates in time with a frequency given by $f=a/\pi\hbar$. This
will mean that the capacity of this two-level channel will also oscillate with
that frequency.

###  III-B Two-level channel capacity

As before the channel capacity is computed by maximizing the mutual
information $I(X;Y)$. We will not need to suppose any quadratic constraints on
$X$, as Alice only prepares one of two states (not one out of infinity). In
this case we assume that Alice will set $|\psi_{0}\rangle$ in one of two
states: either $p=r_{0}$, where $r_{0}$ is such that $0\leq r_{0}\ll 1$, or
$p=1-r_{0}\approx 1$. In other words, Alice can bias her state towards the
$|0\rangle$ state (setting $p=r_{0}$) or towards the $|1\rangle$ state
(setting $p=1-r_{0}$). So we can treat $r_{0}(1-r_{0})$ as a measure of the
variance associated with Alice’s preparation, and as before low variance
corresponds to high precision on Alice’s part.

We treat the variable $X$ as a binary random variable taking values in
$\\{0,1\\}$ corresponding to the choices $p=r_{0}$ and $p=1-r_{0}$
respectively. $Y$ also takes values in $\\{0,1\\}$, corresponding to the
outcomes that Bob could measure after a delay $t$. Equation (36) gives the
transition probability $p_{Y|X}(y|x)=|\langle y|\psi_{t}\rangle|^{2}$ (with
$x,y\in\\{0,1\\}$). These values can be used to compute the mutual
information. Unlike for the Gaussian channel, the capacity here can be shown
to be monotonically decreasing by increasing $r_{0}(1-r_{0})$, the variance of
Alice’s starting state, and so Alice can always do better to increase her
preparation precision for this channel.

What is interesting about the capacity in this case is that it oscillates in
time. In FIGURE 4, we show the time dependence of the capacity over one full
cycle. The transition probabilities can be parameterized by $\epsilon$ and by
the ratio $\gamma=\Delta/\epsilon$. We choose several values of $\gamma$ and
set $\epsilon=2/\sqrt{\gamma^{2}+4}$ in order to fix the frequency and compare
capacities within the same time window.

Figure 4: Capacity of the Two-Level Channel Over the Course of One Full Cycle:
capacity in bits is shown for several values of $\gamma=\Delta/\epsilon$, and
we set $\epsilon=2/\sqrt{\gamma^{2}+4}$ so that each case has the same
frequency for comparison in the same time window.

Depending on the ratio $\gamma$ and the value $\epsilon$, Bob will be able to
extract more information from the channel by choosing to measure the state at
the right time. In this example, Bob always does best to measure at delay
times that are integer multiples of the period of oscillation, $T_{0}$, given
by,

$T_{0}=\frac{2\pi\hbar}{\sqrt{\Delta^{2}+4\epsilon^{2}}}.$ (15)

In this sense, measurement times must be synchronized with the channel to
maximize information transfer.

##  IV Discussion

We have presented two simple quantum models for transmitting classical
information, one involving a continuous set of information states, and one
involving a discrete set. In this sense these examples live on opposite ends
of a discussion on the transmission of information. Our aim was to keep our
analysis as self-contained as feasible and thus only assumes very limited
prior knowledge of quantum mechanics. The channels we discussed involved
preparing a quantum state in a fashion that biased it toward some classical
state. Later, the system could be measured and we used this to construct a
channel between Alice (preparation) and Bob (measurement). This setup does not
make use of entanglement in any manner and yet still provides a demonstration
of some of the counterintuitive effects of using quantum states to encode
classical information.

In the case of the Gaussian channel analogue, we found that increasing Alice’s
preparation precision only increased the capacity up to a point. Beyond this
threshold, preparing a state (localizing a particle) with any more precision
will reduce the capacity of the channel. We saw that this tradeoff was due to
the quantum version of noise that is introduced by ordinary time evolution. In
particular, a particle that is increasingly localized has a wavefunction that
expands more rapidly, leading to increased uncertainty in Bob’s measurement,
and a reduced capacity.

As an example, if Alice and Bob make use of a particle with mass on the order
of $\sim 10^{-27}$ kg (atomic scale), and Bob waits to make a measurement on
the order of every second or so, the optimal precision of the particle is
confined to a region on the order of $\sqrt{\hbar t/m}\sim 10^{-4}$ m. If
Alice prepares the particle with more precision than this, the variance of
Bob’s measurement will be increased away from its minimum, and the capacity
will be reduced. Any less precision and Alice and Bob will loose out on
information contained in the position.

We see that the variance in Bob’s measurement varies inversely with the
particle mass. Thus, the more one tries to localize a particle the faster it
escapes, but heavier particles escape more slowly. This effect is a
consequence of the Heisenberg uncertainty principle—$(\Delta x)^{2}(\Delta
p)^{2}\geq\hbar^{2}/4$, where $\Delta x$ and $\Delta p$ give the standard
deviation over possible measurements of the particle’s position and momentum,
respectively [15]. If Alice and Bob make use of a more macroscopic particle,
say with mass of order $\sim 10^{-6}$ kg, the optimal precision is of order
$\sim 10^{-14}$ m, well within the typical length scale of a hydrogen atom
($\sim 10^{-11}$ m). The information content is more stable when stored in
more massive particles as the measurement variance expands more slowly. As a
result Alice and Bob can increase the capacity of their channel with increased
particle precision. The capacity cannot be increased indefinitely however. As
the mass continues to grow, the optimal precision length continues to shrink
past the subatomic scale. Under these conditions, it becomes increasingly
infeasible for Alice to prepare states with such precision, fundamentally
limiting the capacity which could be achieved by such a communication scheme.
In FIGURE 5 we present a comparison of the capacity across various optimal
length scales and measurement delays, ranging from the scale of an electron
mass ($\sim 10^{-31}$ kg) to that of a grain of sand ($\sim 10^{-6}$ kg).

Figure 5: Contours of the capacity $C$ (in nats) for the quantum gaussian
channel plotted for various particle masses, $m$, and measurement delays, $t$.
We set $P=1$m2. Markers indicate the mass values and corresponding capacity
values for $t=1s$ measurement delays.

It is interesting to note that quantum mechanics reduces to classical physics
in the appropriate limit. A convenient shortcut of describing this limit
treats $\hbar$ as an increasingly negligible parameter. In particular,
physicists will sometimes “take the limit” as $\hbar$ tends to zero, which is
equivalent to considering the case when various quantities of interest become
large compared with $\hbar\sim 10^{-34}$ joule-seconds. In this limit, our
threshold value $v^{*}\propto\hbar$ vanishes, recovering the classical notion
that one can continue to increase the capacity of the channel by increasing
the preparation precision ad infinitum.

In the two-level channel, on the other hand, we found that the capacity can
always be increased by increasing the precision of Alice’s preparation. What
was interesting in this case was that the capacity is an oscillating function
of time, leading to the effect that measurements should be timed according to
the frequency of the channel to achieve optimal capacity. Again considering
the classical limit, equation (15) shows that the time between peaks in the
capacity decreases as $\hbar\rightarrow 0$. In which case, it will become
increasingly difficult for the receiver to maintain synchronization with the
sender and the capacity will average out. As a recent example, quantum dot
technology is being pursued as a two-level model for use in quantum
computation [3]. In that context, oscillations occur on the order of hundreds
of picoseconds, making it currently unfeasible with such technology to time
measurements to integer multiples of the period. In the case of NMR, however,
the relaxation time scale is on the order of seconds [1, 7]. Regardless of the
system, if the energies $\Delta$ and $\epsilon$ could be tuned to be
sufficiently small, the receiver can have more time to maintain synchrony with
the channel. In this case, improved information transfer relies on Alice and
Bob coordinating to maintain a kind of resonance with their two-level channel.

Lastly, both of our toy quantum systems feature a fundamental obstacle to
their use as communication methods. By their nature, both systems are subject
to quantum fluctuations whose inherent indeterminacy lead to unavoidable
losses in capacity. In the Gaussian case, this was due to the uncertainty
principle that trades position and momentum measurement certainty. In the two-
level case, this was the presence of a non-zero tunneling rate between the two
configurations. These considerations suggest that making use of quantum
systems, at least in the method argued here, is fundamentally limited for
storing and communicating classical information. At its most basic level,
however, any storage / communication system behaves quantum mechanically. It
is only in an aggregated form, where more classical behavior emerges, such as
a collection of many particles, that we should expect a system to be capable
of more robust forms of communication.

## Acknowledgements

Sincere thanks to Brenda Rubenstein for pointing us toward various practical
applications of two-level systems.

## Appendix

## Quantum mechanics 101

In quantum mechanics, states are represented by vectors in a complex Hilbert
space $\mathcal{H}$, meaning vector coefficients take values in $\mathbb{C}$
and that an inner product has been defined [15]. Physicists use the notation
$|\psi\rangle\in\mathcal{H}$ to refer to these vectors, and will use
$\langle\phi|\psi\rangle$ to denote the inner product between two states
$|\psi\rangle$ and $|\phi\rangle$. In many applications of quantum information
the Hilbert space is just $\mathcal{H}=\mathbb{C}^{N}$ for some $N\geq 2$. In
which case the standard basis is sometimes denoted, [15, 14]

$\displaystyle|0\rangle$ $\displaystyle=(1,0,...,0)^{T}$
$\displaystyle|1\rangle$ $\displaystyle=(0,1,...,0)^{T}$ $\displaystyle\vdots$
$\displaystyle|N-1\rangle$ $\displaystyle=(0,0,...,1)^{T}$

And if we write $|\psi\rangle=\sum_{m=0}^{N-1}a_{m}|m\rangle$ and
$|\phi\rangle=\sum_{m=0}^{N-1}b_{m}|m\rangle$, then the inner product is given
by,

$\langle\phi|\psi\rangle=\sum_{m=0}^{N-1}b_{m}^{*}a_{m}$ (16)

where $b_{m}^{*}$ refers to the complex conjugate of $b_{m}$, and we have used
$\langle m|n\rangle=\delta_{mn}$.

In section IV we consider an example with $N=2$. In section III, however, when
considering the analogue of the Gaussian channel, $\mathcal{H}$ is an infinite
dimensional function space, which is appropriate for continuous domains such
as the position of a particle.

Observable quantities, such as position and energy, are represented by
Hermitian operators on the Hilbert space. Specifically, the set of eigenvalues
of such an operator is precisely the set of possible measurement outcomes for
that observable. Hermiticity guarantees that the eigenvalues are real-valued,
as expected for measurable quantities. One can expand an arbitrary state in
terms of the eigenvectors of any of these Hermitian operators. Most
importantly, the squared modulus of each coefficient in this expansion has the
interpretation of the probability (or probability density) that a measurement
of the system will yield the eigenvalue associated to that eigenvector. Thus,
eigenvectors are interpreted as states with definite observable values,
whereas a superposition of eigenvectors cannot be associated with a definite
value of that observable (although it may correspond to a definite value of
some other observable).

For the purposes of this discussion, we will need to evaluate the time
evolution of quantum states. This is determined by the Schrodinger equation,
which says,

$i\hbar\frac{d}{dt}|\psi\rangle=\hat{H}|\psi\rangle$ (17)

where $\hat{H}$ is the operator associated to the energy values, called the
Hamiltonian, $i=\sqrt{-1}$ is the imaginary unit and $\hbar$ is the so-called
reduced Planck’s constant, whose value is approximately $\hbar\approx
1.05457\times 10^{-34}$ joule-seconds. A general solution of the time-evolved
state, for a time-independent energy operator $\hat{H}$, $|\psi_{t}\rangle$ is
given by,

$|\psi_{t}\rangle=e^{-i\hat{H}t/\hbar}|\psi_{0}\rangle$ (18)

where $|\psi_{0}\rangle$ represents the initial state of the particle. For our
purposes, this will be the state that the sender, Alice, sends to the
receiver, Bob.

We can expand an arbitrary state $|\psi\rangle$ in a basis of eigenvectors of
the Hamiltonian $\hat{H}$, and we will generically label the eigenvectors
(sometimes called eigenstates) by their eigenvalue, in this case the energy,
$|E\rangle$. That is, $\hat{H}|E\rangle=E|E\rangle$. This notation is
ambiguous when there is degeneracy in the eigenspace, but there will be no
such degeneracy in our examples and so we will continue to label eigenvectors
by their eigenvalue. We also assume that eigenvectors are normalized to unity,
so that in particular $\langle E|E\rangle=1$ and that $\langle
E|E^{\prime}\rangle=0$ when $E\neq E^{\prime}$. Writing an arbitrary state as
$|\psi\rangle=|\psi_{0}\rangle=\sum_{E}a_{E}|E\rangle$, in the so-called
energy basis, we see that the Schrodinger equation implies,

$\displaystyle|\psi_{t}\rangle$
$\displaystyle=e^{-i\hat{H}t/\hbar}\sum_{E}a_{E}|E\rangle$
$\displaystyle=\sum_{E}a_{E}e^{-iEt/\hbar}|E\rangle$

since
$(\hat{H}|E\rangle=E|E\rangle)\Rightarrow(e^{-i\hat{H}t/\hbar}|E\rangle=e^{-iEt/\hbar}|E\rangle)$.
Thus, time evolution is equivalent to scaling the energy eigenvectors by phase
factors. In what follows, we will posit a Hamiltonian that models the
situation and will expand the initial state built by Alice in terms of the
corresponding energy eigenvectors. Then we will insert the appropriate phase
factors to get the state that Bob encounters some time $t$ later.

In the case of infinite dimensional Hilbert spaces, such as the one associated
with the position of a particle along an axis, states are usually expressed in
terms of the so-called “position basis” $\\{|x\rangle\\}_{x\in\mathbb{R}}$,
where $|x\rangle$ is an eigenstate of the position operator $\hat{x}$, i.e.
$\hat{x}|x\rangle=x|x\rangle$, meaning that it represents a particle with a
definite position $x\in\mathbb{R}$. In this case, one often works with the
projection of the state in this basis, denoted $\psi(x)=\langle
x|\psi\rangle$, i.e. the component of $|\psi\rangle$ along the basis vector
$|x\rangle$. This is usually called the wavefunction and we will sometimes
include the time dependence of this state and write $\psi(x,t)=\langle
x|\psi_{t}\rangle$. In this basis, the operator corresponding to the momentum
of the particle, denoted $\hat{p}$, turns out to be proportional to the
derivative with respect to $x$,

$\hat{p}=-i\hbar\frac{\partial}{\partial x},$ (19)

meaning that the effect of the momentum operator is take the derivative of the
wavefunction with respect to position. We will make use of this form of the
operator in equation (20) in the next section.

## Derivation of the time evolution for the Quantum Gaussian Channel

Computing the time evolution in quantum mechanics is equivalent to scaling the
energy eigenfunctions that comprise $\psi(x)$ by the phase factor,
$e^{-iEt/\hbar}$, where $E$ is the energy eigenvalue associated to an energy
eigenfunction. In the case of a non-relativistic free particle, the energy
operator (in the position basis) is given by,

$\hat{H}=\frac{\hat{p}^{2}}{2m}=-\frac{\hbar^{2}}{2m}\frac{\partial^{2}}{\partial
x^{2}}.$ (20)

Its eigenfunctions, $\phi_{E}(x)=\langle x|E\rangle$, indexed by the energy,
satisfy,

$-\frac{\hbar^{2}}{2m}\frac{\partial^{2}}{\partial
x^{2}}\phi_{E}(x)=E\phi_{E}(x),$ (21)

meaning that the eigenfunctions can be written as,

$\phi_{E}(x)=\frac{1}{\sqrt{2\pi}}e^{ikx}$ (22)

where $k=\pm\sqrt{2mE}/\hbar$. Notice that these are in fact the same
eigenfunctions for the momentum operator $\hat{p}=-i\hbar\partial/\partial x$
(and nothing more than Fourier modes). This also follows from the fact that
the momentum commutes with the Hamiltonian:
$[\hat{H},\hat{p}]=\hat{H}\hat{p}-\hat{p}\hat{H}=0$, since $\hat{H}$ is only a
function of $\hat{p}$. The interpretation is that momentum is a conserved
quantity (net force $=0$), whereas position is not ($[\hat{H},\hat{x}]\neq
0$).

Time evolution of each of these modes is given simply by,

$\phi_{E}(x)\xrightarrow[]{t}\phi_{E}(x)e^{-iEt/\hbar}=\frac{1}{\sqrt{2\pi}}e^{ikx-
iEt/\hbar}.$ (23)

###  IV-A Energy eigenstates

It remains to decompose $\psi(x)$ into its energy eigenfunctions (Fourier
modes). The Fourier transform of Alice’s initial wavefunction $\psi(x)=\langle
x|\psi_{0}\rangle$, which we’ll denote by $\tilde{\psi}(k)=\langle
k|\psi_{0}\rangle$, is given by:

$\tilde{\psi}(k)=\int_{-\infty}^{\infty}dx\frac{e^{-ikx}}{\sqrt{2\pi}}\frac{e^{-(x-x_{0})^{2}/4\sigma_{A}^{2}}}{(2\pi\sigma_{A}^{2})^{1/4}}$
(24)

We can make use of the following special integral,

$\int_{-\infty}^{\infty}dze^{\frac{1}{2}iaz^{2}+iJz}=\sqrt{\frac{2\pi
i}{a}}e^{-iJ^{2}/2a}$ (25)

Setting $z=x-x_{0}$ in the $\tilde{\psi}(k)$ integral leaves,

$\tilde{\psi}(k)=\frac{e^{-ikx_{0}}}{\sqrt{2\pi}(2\pi\sigma_{A}^{2})^{1/4}}\int_{-\infty}^{\infty}dze^{-ikz}e^{-z^{2}/4\sigma_{A}^{2}}$
(26)

from which we see we can make use of our special integral by setting
$a=i/2\sigma_{A}^{2}$ and $J=-k$. Thus, we have,

$\tilde{\psi}(k)=\frac{e^{-ikx_{0}}\sqrt{2\sigma_{A}}}{(2\pi)^{1/4}}e^{-k^{2}\sigma_{A}^{2}}.$
(27)

Hence, the time evolved state $\psi(x,t)=\langle x|\psi_{t}\rangle$ can be
written as,

$\displaystyle\psi(x,t)$
$\displaystyle=\int_{-\infty}^{\infty}dk\tilde{\psi(k)}\frac{1}{\sqrt{2\pi}}e^{ikx-
iEt/\hbar}$
$\displaystyle=\frac{\sqrt{2\sigma_{A}}}{\sqrt{2\pi}(2\pi)^{1/4}}\int_{-\infty}^{\infty}dke^{-k^{2}\sigma_{A}^{2}+ik(x-x_{0})-iEt/\hbar}.$

Recall that $k$ was defined by $k=\pm\sqrt{2mE}/\hbar$, and so
$E=\hbar^{2}k^{2}/2m$. We see we can again make use of our special integral,
this time setting $a=2i\left(\sigma_{A}^{2}+\frac{i\hbar t}{2m}\right)$, and
$J=x-x_{0}$. And so we have,

$\displaystyle\psi(x,t)$
$\displaystyle=\frac{\sqrt{2\sigma_{A}}}{\sqrt{2\pi}(2\pi)^{1/4}}\sqrt{\frac{\pi}{\sigma_{A}^{2}+\frac{i\hbar
t}{2m}}}\exp\left(\frac{-(x-x_{0})^{2}}{4\left(\sigma_{A}^{2}+\frac{i\hbar
t}{2m}\right)}\right)$
$\displaystyle=\frac{1}{\sqrt{\sqrt{2\pi}\left(\sigma_{A}+\frac{i\hbar
t}{2m\sigma_{A}}\right)}}\exp\left(\frac{-(x-x_{0})^{2}}{4\left(\sigma_{A}^{2}+\frac{i\hbar
t}{2m}\right)}\right).$

###  IV-B Energy considerations for the Gaussian channel

One other consideration is that of the finite amount of time taken for Alice
to prepare her state. Doing so more rapidly will incur an energetic cost.
Since the particle initially emerges at $x=0$, the energy required for
particle placement is proportional to the energy necessary to move a particle
from $x=0$ to some $X=x_{n}$ in a specified amount of time $T$, where $n$
indexes the placement run. Thus, the minimum average power $P$ expended to
place particles goes as,

$P=\frac{1}{2(T+t)}m\mathbb{E}\left[\left(\frac{X}{T}\right)^{2}\right]$ (28)

where $\frac{1}{T+t}$ is the rate at which the communication scheme cycles
(particles are placed then measured), $m$ is the particle mass and the
$\mathbb{E}[\cdot]$ denotes the expected value over particle placement
positions $X$. For notational simplicity we will write equation (28) as,

$P=\beta\mathbb{E}\left[X^{2}\right]$ (29)

where

$\beta=\frac{m}{2T^{2}(T+t)}$

## Derivation of the time evolution for the two-level channel

We model transitions between the two levels using a parameter
$\epsilon\in\mathbb{R}$ in a Hamiltonian of the form,

$\hat{H}=\begin{pmatrix}E&\epsilon\\\ \epsilon&E+\Delta\end{pmatrix}.$ (30)

Here we assume Alice can set her initial state as,

$|\psi_{0}\rangle=\sqrt{1-p}|0\rangle+\sqrt{p}|1\rangle$ (31)

either with $p$ close to $0$ (so that $|\psi_{0}\rangle$ is close to
$|0\rangle$) or with $p$ close to $1$ (so that $|\psi_{0}\rangle$ is close to
$|1\rangle$). Here $p$ gives the initial probability of a measurement
resulting in the state $|1\rangle$, and so $1-p$ is the probability of
measuring the state $|0\rangle$. Hence, we can use $p(1-p)$ as a measure of
the variance for the corresponding distribution, $|\langle
z|\psi_{0}\rangle|^{2}$, where $z\in\\{0,1\\}$ (Bernoulli distribution).

We could also include a relative phase factor between the $|0\rangle$ and
$|1\rangle$ states in Alice’s state $|\psi_{0}\rangle$, such as $e^{i\theta}$
for some $\theta\in\mathbb{R}$, but since Bob only makes measurements along
the $\\{|0\rangle,|1\rangle\\}$ basis, any such phase factor will drop out of
the discussion when taking the modulus squared.

For arbitrary $\epsilon\geq 0$ the energy eigenstates can be computed (the
$\epsilon<0$ case can be treated similarly). We will label them by their
energy eigenvalues, $E_{\pm}$,

$|E_{\pm}\rangle=\frac{1}{\sqrt{2a}}\begin{pmatrix}\sqrt{a\mp b}\\\
\pm\sqrt{a\pm b}\end{pmatrix}$ (32)

where,

$\displaystyle a$ $\displaystyle=\frac{\sqrt{\Delta^{2}+4\epsilon^{2}}}{2},$
$\displaystyle b$ $\displaystyle=\frac{\Delta}{2}$

and,

$\displaystyle E_{\pm}$
$\displaystyle=E+\frac{\Delta\pm\sqrt{\Delta^{2}+4\epsilon^{2}}}{2}$ (33)
$\displaystyle=E\pm a+b,$ (34)

as can be easily verified.

We can write Alice’s starting states, $|0\rangle$ and $|1\rangle$, in terms of
these energy states.

$\displaystyle|0\rangle$ $\displaystyle=\begin{pmatrix}1\\\
0\end{pmatrix}=\sqrt{\frac{a-b}{2a}}|E_{+}\rangle+\sqrt{\frac{a+b}{2a}}|E_{-}\rangle,$
$\displaystyle|1\rangle$ $\displaystyle=\begin{pmatrix}0\\\
1\end{pmatrix}=\sqrt{\frac{a+b}{2a}}|E_{+}\rangle-\sqrt{\frac{a-b}{2a}}|E_{-}\rangle.$

As before, time evolution is given by simply scaling the energy states by a
phase factor, in this case,

$|E_{\pm}\rangle\xrightarrow[]{t}|E_{\pm}\rangle e^{-iE_{\pm}t/\hbar}$ (35)

just as in equation (23).

Writing $|\psi_{0}\rangle$ in terms of $|E_{\pm}\rangle$,

$\displaystyle|\psi_{0}\rangle=\frac{1}{\sqrt{2a}}\left(\sqrt{(1-p)(a-b)}+\sqrt{p(a+b)}\right)|E_{+}\rangle$
$\displaystyle+\frac{1}{\sqrt{2a}}\left(\sqrt{(1-p)(a+b)}-\sqrt{p(a-b)}\right)|E_{-}\rangle.$

And so the time evolved state is given by,

$\displaystyle|\psi_{t}\rangle$
$\displaystyle=\frac{1}{\sqrt{2a}}\left(\sqrt{(1-p)(a-b)}+\sqrt{p(a+b)}\right)|E_{+}\rangle
e^{-iE_{+}t/\hbar}$
$\displaystyle+\frac{1}{\sqrt{2a}}\left(\sqrt{(1-p)(a+b)}-\sqrt{p(a-b)}\right)|E_{-}\rangle
e^{-iE_{-}t/\hbar}$ $\displaystyle=\frac{e^{-i(E+b)t/\hbar}}{a}$
$\displaystyle\times\begin{pmatrix}a\sqrt{1-p}\cos(\frac{at}{\hbar})+i\left(b\sqrt{1-p}-\epsilon\sqrt{p}\right)\sin(\frac{at}{\hbar})\vspace{0.1in}\\\
a\sqrt{p}\cos(\frac{at}{\hbar})-i\left(\epsilon\sqrt{1-p}+b\sqrt{p}\right)\sin(\frac{at}{\hbar})\end{pmatrix},$

where we used $\epsilon^{2}=a^{2}-b^{2}$.

Lastly, we compute the modulus squared of each coefficient to give the
transition probabilities,

$|\langle
z|\psi_{t}\rangle|^{2}=\frac{1}{a^{2}}\begin{pmatrix}\zeta_{p}\cos(\frac{2at}{\hbar})+a^{2}(1-p)-\zeta_{p}\\\
-\zeta_{p}\cos(\frac{2at}{\hbar})+a^{2}p+\zeta_{p}\end{pmatrix}_{z},$ (36)

where,

$\zeta_{p}=\frac{\epsilon}{2}\left(\epsilon(1-2p)+2b\sqrt{p(1-p)}\right)$ (37)

with $z\in\\{0,1\\}$.

## References

  * [1] Seyma Alcicek, Piotr Put, Adam Kubrak, Fatih Celal Alcicek, Danila Barskiy, Stefan Gloeggler, Jakub Dybas, and Szymon Pustelny.  Zero- to low-field relaxometry of chemical and biological fluids.  Nature, 418, Aug 2002.
  * [2] C.H. Bennett and P.W. Shor.  Quantum information theory.  IEEE Transactions on Information Theory, 44(6):2724–2742, 1998.
  * [3] A. Bowen, E. Beham, S. Stufler, F. Findeis, M. Bichler, and G. Abstreiter.  Coherent properties of a two-level system based on a quantum-dot photodiode.  Nature, 418, Aug 2002.
  * [4] Garry Bowen.  Classical information capacity of superdense coding.  Phys. Rev. A, 63:022302, Jan 2001.
  * [5] T.M. Cover and J.A. Thomas.  Elements of Information Theory.  Wiley-Interscience, 2006.  2nd Ed.
  * [6] Sushanta Dattagupta.  Two-level systems in quantum chemistry and physics.  Resonance, 26, Dec 2021.
  * [7] Ya. S. Greenberg and E. Il’ichev.  Rabi spectroscopy for a two-level system, 2005.
  * [8] Laszlo Gyongyosi, Sandor Imre, and Hung Viet Nguyen.  A survey on quantum channel capacities.  IEEE Communications Surveys & Tutorials, 20(2):1149–1205, 2018.
  * [9] Aram Harrow, Patrick Hayden, and Debbie Leung.  Superdense coding of quantum states.  Phys. Rev. Lett., 92:187901, May 2004.
  * [10] A.S. Holevo.  Quantum channel capacities.  Quantum Electronics, 50(5):440, may 2020.
  * [11] Stefan Host.  Discrete Input Gaussian Channel, pages 265–288.  Wiley, 2019.
  * [12] A. G. J. MacFarlane, Jonathan P. Dowling, and Gerard J. Milburn.  Quantum technology: the second quantum revolution.  Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences, 361(1809):1655–1674, 2003.
  * [13] Dan C. Marinescu and Gabriela M. Marinescu.  Chapter 3 - classical and quantum information theory.  In Dan C. Marinescu and Gabriela M. Marinescu, editors, Classical and Quantum Information, pages 221–344. Academic Press, Boston, 2012.
  * [14] Chuang I Nielsen M.  Quantum Computation and Quantum Information.  Cambridge University Press, 2010.
  * [15] Jun John Sakurai.  Advanced quantum mechanics.  Pearson Education India, 1967.
  * [16] Alexander Sigov, Leonid Ratkin, and Leonid A. Ivanov.  Quantum information technology.  Journal of Industrial Information Integration, 28:100365, 2022.
  * [17] Ming Chen Wang and G. E. Uhlenbeck.  On the theory of the brownian motion ii.  Rev. Mod. Phys., 17:323–342, Apr 1945.


twocolumns Törnberg The author has no competing interests to declare. 2To whom
correspondence should be addressed. E-mail<EMAIL_ADDRESS>
# ChatGPT-4 Outperforms Experts and Crowd Workers in Annotating Political
Twitter Messages with Zero-Shot Learning

Petter Törnberg  Amsterdam Institute for Social Science Research (AISSR),
University of Amsterdam

###### Abstract

This paper assesses the accuracy, reliability and bias of the Large Language
Model (LLM) ChatGPT-4 on the text analysis task of classifying the political
affiliation of a Twitter poster based on the content of a tweet. The LLM is
compared to manual annotation by both expert classifiers and crowd workers,
generally considered the gold standard for such tasks. We use Twitter messages
from United States politicians during the 2020 election, providing a ground
truth against which to measure accuracy. The paper finds that ChatGPT-4 has
achieves higher accuracy, higher reliability, and equal or lower bias than the
human classifiers. The LLM is able to correctly annotate messages that require
reasoning on the basis of contextual knowledge, and inferences around the
author’s intentions – traditionally seen as uniquely human abilities. These
findings suggest that LLM will have substantial impact on the use of textual
data in the social sciences, by enabling interpretive research at a scale.

###### keywords:

Large Language Models $|$ ChatGPT-4 2 $|$ interpretation 3 $|$ text-as-data
$|$ Twitter $|$ social media

This manuscript was compiled on

Texts are culturally and socially situated within the ideas, values and
beliefs about the world within which humans operate. Their interpretation thus
requires deep contextual knowledge and the ability to “put ourselves in the
shoes of others” – seen as distinctly human resources and capacities. While
interpretation lies at the very heart of the social sciences (1, 28), it has
thus been seen as almost intrinsically qualitative; “meaning has to be
understood, it cannot be measured or counted, and hence there is always an
interpretive or hermeneutic element in social science” (2, 17).

Researchers seeking to use textual data have thus had to pursue to one of two
strategies for analyzing textual data: so-called expert coders (such as
colleagues, students, or research assistants), or crowd-workers on online
labor platforms, in particular Amazon’s MTurk. These strategies are often
combined, for instance by using trained annotators to create a smaller
dataset, which can be used to evaluate the quality of crowd-workers that are
then employed to expand the volume of data. But while humans have been
considered the unrivaled gold standard for interpretive tasks (3, 4), we bring
several limitations: we are slow, costly, biased, and have limited attention-
spans, hence restricting interpretive research to relatively small-N studies –
in turn leading to complaints of lacking rigor and replicability, and limiting
the potential in Big Data research. As a result, conventional forms of
interpretive textual research thus tends to miss patterns of language use are
“not directly observable, because they are realized across thousands or
millions of words of running text, and because they are not categorical but
probabilistic” (5, 204). While computational methods for analyzing textual
data – such Natural Language Processing and Machine Learning – have evolved
quickly in recent years (6), common methods that build on bag-of-words,
lexical meanings and sentence semantics tend to work poorly for tasks
requiring complex inferences on the basis of knowledge about the world or
assumptions of the author’s mental state and intentions.

The recent rise of generative AI may however be at the cusp of changing this.
In recent months, ChatGPT has become a global sensation, and one of the
quickest growing consumer products of all time. ChatGPT is a pre-trained model
based on a massive neural network with billions of parameters, and trained on
hundreds of billions of words of text from the Internet and digitized books
(7). While skeptics have dismissed these forms of Large Language Models (LLM)
as merely “fancy autocomplete”, the models have demonstrated several
surprising emergent capabilities (8) – even showing “sparks of general
intelligence” (9) – including the capacity for few-shot and zero-shot
classification tasks (10, 7). As these models are general rather than field or
task-specific, they appear to capture some of the context needed for
interpretation – enabling it to even surpass fine-tuned models. Recent studies
have found that LLMs perform well for a wide range of purposes, including
ideological scaling (11), text annotation tasks (12), and for simulating
samples for survey research (13).

This paper explores the potential of LLMs for text analysis tasks whose
interpretation require reasoning on the basis of contextual knowledge.
Previous studies have been limited by lacking ground truth with which to
compare accuracy, or comparing LLM’s only with crowd workers, known to be
unreliable in terms of accuracy and data quality (14). We move beyond existing
research by focusing on the recently released ChatGPT-4 model, assessing its
accuracy compared to both MTurk workers and experts, on a realistic social
scientific task for which ground truth is available. Focusing on identifying
the political party affiliation of Twitter posters, we find that ChatGPT-4
reliably outperform both crowd-workers and expert coders. We discuss the
implications, risks, and limitations of this potential paradigm shift for
using text-as-data, as the large-N interpretive works hits at the core of the
traditional division between the quantitative and qualitative paradigms.

## ChatGPT and the rise of Generative AI

ChatGPT is an AI chatbot developed by OpenAI and introduced in November 2022.
Given a prompt as an initial text input, ChatGPT generates a response,
simulating a conversation with the user. It is a member of the generative pre-
trained transformer (GPT) family of language models and is based on OpenAI’s
LLMs GPT-3.5 and GPT-4 (7). These models are built on highly complex
artificial neural networks trained to predict the next token given a set of
existing tokens. The GPT models were trained on a vast corpus of text,
including a significant share of the Internet and all books ever written. The
ChatGPT model was furthermore fine-tuned using both supervised and
reinforcement learning techniques to achieve human-like responses (15). The
supervised learning consisted of human trainers playing both the user and the
AI assistant in simulated conversations. In the reinforcement learning step,
the model generated text that was then ranked by human annotators from most
preferred to least preferred, allowing the LLM to further improve its
responses (16).

While this type of transformer-based language models originally functioned as
a form of sophisticated autocomplete, they began taking on surprising emergent
properties when they reach a certain size. In line with Anderson’s (17) famous
suggestion that “more is different”, the models appear to go through a form of
phase transition, bringing about new capacities for which they were not
explicitly trained (8). For instance, ChatGPT appears to compose information,
as opposed to merely regurgitating it. It is able to operate in and translate
between several languages. It is able to generate computer code based on
instruction in natural language, and to write prose or poetry on any topic in
a given style. For the purposes of this paper, the most relevant such emergent
capacity is the ability to engage in what appears as a form of reasoning on
the basis of known contextual information, allowing the model to interpret
textual statement with zero- or few-shot learning.

However, while ChatGPT displays several noteworthy capacities and have drawn
substantial international attention, the model still has several important
limitations. For instance, when facing questions that the model does not
“know” the answer to, the model is prone to “hallucinate” (18), that is, it
produces confident-sounding “bullshit” (19) answers. The model is furthermore
limited when it comes to symbolic logic – failing even simple mathematical
tasks when faced with large-enough numbers. More seriously, the models
reproduces the biases and racism embodied in the data on which it was fed, and
LLMs such as ChatGPT are therefore prone to generate text involving negative
stereotypes, for instance in relation to race and gender (20). To address
this, OpenAI has imposed several “guardrails” to prevent the chatbot from
expressing offensive views.

Social scientists have recently begun studying the possibilities of using LLMs
such as ChatGPT for social scientific research. Ornstein et al. (21) found
that GPT-3 can be effectively applied in few-shot learning contexts to text-
as-data tasks in political science, including sentiment analysis, ideological
scaling, and topic modeling. Argyle et al. (13) found that biases in GPT-3 are
fine-grained and demographically correlated, and can be used to simulate
partisan responses from a wide variety of human communities. Palmer and
Spirling (22) found that LLMs can argue in persuasive and novel ways about
politics. Wu et al (11) found that ChatGPT can be used to create a scale of
politicians’ ideologies through pair-by-pair comparison. In this paper, we
will assess the accuracy of ChatGPT-4 for annotating and classifying political
messages on Twitter.

## Method: Using ChatGPT to classify Twitter messages

We use data collected by van Vliet et al (23), selecting all messages posted
to Twitter from all United States senators during the two months preceding the
US 2020 election, that is, between September 3rd, 2020, and November 3rd,
2020. We use messages from politicians as these offer a ground truth: we know
their political affiliations and can therefore objectively evaluate the
accuracy of the ChatGPT answers, and compare with the accuracy of experts.
However, the method could be similarly applied to examine the political views
or positions of normal social media users. From this dataset, retweets and
replies were removed, as were tweets containing an URL, as were messages
shorter than 100 characters (as these were not long enough to contain
political content.) Random samples of 250 tweets from Republican and from
Democratic politicians were selected, resulting in a balanced sample with 500
tweets.

The resulting set of tweets contain a mix between expressions of view on
particular policy positions, comments on recent political events (such as the
COVID-19 Pandemic, or the nomination of Amy Coney Barrett to the Supreme
Court), self-congratulatory messages for some policy victory, calls for
campaign contributions, and the occasional personal news. Some of the tweets
can be easily classified as being from a Democrat or Republican – such as
laudatory references to the then-sitting President Donald Trump, or attacks
against the opposing party. Other require detailed knowledge of US politics,
such as the positions of the two parties on particular political issues in
Oklahoma. Other require the capacity to consider the author’s intention in
relation to the electoral considerations of the two parties, such as Bible
quotes, speaking to the Evangelical base of the Republican party. Finally,
some tweets are nearly impossible to guess, such as the lamenting of the death
of a friend.

We used ChatGPT-4 through the API to classify each message, using the
following instruction:

> “You will be given a set of Twitter posts from different US politicians,
> sent during the two months preceding the 2020 US presidential election, that
> is, between September 3rd, 2020, and November 3rd, 2020. Your task is to use
> your knowledge of US politics to make an educated guess on whether the
> poster is a Democrat or Republican. Respond either ‘Democrat’ or
> ‘Republican’. If the message does not have enough information for an
> educated guess, just make your best guess.”

The API was then given the Twitter messages, selected in random order. The
model was run with high and low values for temperature – a parameter that
controls the level of randomness or “creativity” in the generated text. Since
the responses are stochastic, we ran the model 5 times at a low temperature
(0.2), and at a high temperature (1.0) to capture variability in responses.
The model was thus run for a total of 5000 times.

To assess the accuracy, reliability and bias of ChatGPT with the current gold
standard for textual annotation, we compare the resulting data with two manual
datasets: MTurk classification, and expert classifiers.

MTurk is an online platform for click-labor, which gives each task to a large
number of workers, and to thus use the “wisdom of crowd” effect (a group
generally has higher accuracy than any given individual) to reach high levels
of accuracy. The MTurk workers were given the same instructions as ChatGPT.

MTurk classification vary significantly in quality depending on the details of
how the tasks are implemented, making it perilous to compare models with MTurk
accuracy without additional verification (14). In the pre-study testing, MTurk
was run without stringent quality evaluation, resulting in a mean accuracy
that was not significantly different from random. To maximize accuracy of the
MTurk predictions, best-practices were thus used to ensure high quality
responses from the MTurk crowd workers: only Master Qualified workers (a
special group of elite MTurk workers whose abilities have been verified)
residing in the United States were allowed to respond. The workers were given
the questions in batches of 50, each with 4 control questions in which the
author’s political affiliation was evident (e.g., “If one thing is certain, I
am a Democrat!”). If the worker failed any of these questions, their responses
were automatically rejected. Even when passing these tests, the work was
rejected if the answer accuracy was not statistically different from random.
Each question was answered by 10 independent crowd-workers.

To verify the quality of the MTurk response, two political science researchers
also manually classified all messages. While these scholars do not
specifically work on US politics, they have a strong general understanding,
and thus provide a baseline of accuracy. These experts were given the 500
messages in random order, and again the same instructions as ChatGPT.

## Results

Figure 1: A normalized density histogram of the response accuracy by crowd
worker, experts, and the LLM models. The dashes lines show the mean response
accuracy for LLMs and experts. The combined MTurk line shows the accuracy of
the majority response for each question, as this is how crowd-worker answers
tend to be employed. As can be seen, the LLMs outperform all individual human
classifiers. The mean accuracy lines for the two temperatures of the LLM
nearly precisely overlap.

Figure 1 shows the distribution of accuracies per worker, model run, and
expert, as well as the average accuracies. As the figure shows, the LLM
outperforms all individual human classifiers, as well as the combined crowd
workers. For MTurk, the accuracy of the plurality response of the 10 workers
are reported, as the “wisdom of crowds” effect means that the accuracy of the
combined answers is often higher than any individual. As the figure shows, the
expert classifiers have higher accuracy than most individual MTurk
classifiers, but lower than their combined response. The LLM has a slightly
higher accuracy and lower spread at lower temperatures.

Figure 2: The Krippendorf’s Alpha measure of intercoder reliability, with 95%
bootstrap confidence interval. The LLM offers substantially higher levels of
reliability than the human coders.

Reiss (24) suggests that ChatGPT may suffer from limited reliability, giving
stochastic answers and thereby limiting the usefulness for annotation and
classification tasks. However, this study does not compare the reliability
with human classifiers. To measure intercoder reliability, we calculate
Krippendorf’s Alpha. Figure 2 shows the result. We here see that compared to
human classifiers, the LLM in fact has a much higher reliability for the task
at hand. As expected, the LLM’s reliability is higher at lower temperature.

Figure 3: All coders are biased to guessing Democrat over Republican. The LLMs
and experts are similar in the level of bias, while the MTurk classifiers have
a significantly stronger bias.

To examine whether there is any bias in the model compared to the manual
classification in terms of whether the model is more likely to respond with
one party over the other, we examine the fraction of responses for each party
for LLM, MTurk and the expert annotators. Figure 3 shows that all respondents
are significantly biased toward guessing that the tweet author is a Democrat,
including the LLM. The biases of the expert classifiers and the LLM are not
significantly different, while the MTurk classifiers are significantly more
biased.

Finally, we examine select cases of how ChatGPT responded in relation to
tweets that require particular interpretive skills, by following up the
question with a request for ChatGPT to “motivate your response”. First, a
tweet whose meaning is implicit and depends on contextual information: “A
friend, a neighbor, a mom, who could soon serve on the highest court in the
land”. ChatGPT correctly responded Republican, as did 7 of the MTurk workers.
ChatGPT motivates its answer with:

> “This tweet is likely referring to the nomination of Amy Coney Barrett to
> the Supreme Court, which took place during the specified time frame. Barrett
> was nominated by then-President Donald Trump, a Republican, and her
> nomination was supported by the Republican party. The positive tone of the
> tweet suggests that the poster is in favor of her nomination, which leads me
> to believe that the poster is a Republican.”

Second, a tweet containing a Bible quote (“good indeed is the LORD, His mercy
endures forever, his faithfulness lasts through every generation. Psalms
100:5”). Interpreting this requires understanding that the intent of the post
is to emphasize values primarily associated Republican voters. The LLM and 9
out of 10 MTurk workers correctly guessed that the author was Republican.

> “I guessed that the poster is a Republican because the tweet includes a
> quote from the Bible, specifically Psalms 100:5. While both Democrats and
> Republicans can be religious, Republicans are generally more likely to
> emphasize their religious beliefs and values in public statements, as they
> tend to have a stronger connection with conservative Christian voters.”

While the stringency of the reasoning can be questioned and it is not known
what processes are in fact underlying the responses of the model, the
reasoning does appear plausible and convincing at face value and the accuracy
of the model’s guesses are undeniable.

## Discussion

This brief paper has assessed the accuracy, reliability and bias of ChatGPT-4
for a zero-shot learning classification task of identifying political
affiliations based on Twitter messages. The findings suggest that LLMs may
already outperform the standard approaches of crowd-workers and expert
classifiers, offering higher accuracy, higher reliability, and lower or equal
levels of bias – even for task that require reasoning on the basis of
contextual knowledge and assumptions of the author’s intentions.

The possibility of using LLMs for such tasks have significant scientific
implications, enabling large-scale and replicable interpretative research with
low costs and barriers of entry – thus further blurring the traditional lines
between quantitative and qualitative approaches. LLMs do not only offer higher
data quality than conventional means of text analysis, but are also orders of
magnitude less costly and time-consuming. The models can furthermore be used
with limited technical skills, with much the same instructions as would be
used for human coders.

However, several caveats must be made. The capacity of LLMs to do “zero” or
“few-shot” learning is an emergent property, for which the models are not
explicitly trained. How LLMs acquire such capabilities is not well understood
at a conceptual level, and we therefore do not fully know their scope or
limitations. Second, LLMs can be sensitive to researcher choices, such as
prompt design. Like human coders, LLMs are prone to misunderstanding poorly
formulated tasks, but unlike human coders, they are also unlikely to admit
uncertainty or inform the researcher when they struggle to comprehend their
task. Identifying the optimal instructions may thus require a careful
iterative process of experimentation and validation. Third, LLMs embody the
biases and prejudices of the texts on which they are trained. ChatGPT in
particular has been found to display problematic gender and racial
stereotypes, when users have been able to bypass the imposed guardrails. It
remains poorly understood if and how such biases affect the models’
performance on specific analysis tasks. The solution to all of these issues is
careful validation of the model’s performance on specific tasks – and never to
assume that the approach will work “out of the box” (25). While training may
no longer be necessary for many tasks, validation remains crucial, to make
sure that the models are measuring what we intend – and without problematic
biases.

The pace of innovation of LLMs is astounding, and while several fundamental
issues remain, it seems likely that LLMs will fundamentally transform social
scientific research by opening the possibility of large-N interpretive
research – representing a new paradigm of text-as-data research. At the same
time, such advances raise important epistemological challenges, as these
models are black-boxed and their capacities are emergent and poorly
understood. Beyond continued technical exploration of the capacities of these
models and the development of standards for rigorous research, a key task for
researchers going forward will be to tackle their profound epistemological
implications for social scientific research.

The author would like to acknowledge the anonymous crowd workers and expert
classifiers for annotating the tweets, and Dr. Juliana Chueri for her
suggestions, input and support.

### References

## References

  * (1) M Weber, Sociological writings.  (Continuum International Publishing Group) Vol. 60, (1994).
  * (2) A Sayer, Realism and social science.  Realism and Social Science pp. 1–224 (1999).
  * (3) JS Murthy, GM Siddesh, KG Srinivasa, TwitSenti: a real-time Twitter sentiment analysis and visualization framework.  Journal of Information & Knowledge Management 18, 1950013 (2019) Publisher: World Scientific.
  * (4) W Van Atteveldt, MA Van der Velden, M Boukes, The validity of sentiment analysis: Comparing manual annotation, crowd-coding, dictionary approaches, and machine learning algorithms.  Communication Methods and Measures 15, 121–140 (2021) Publisher: Taylor & Francis.
  * (5) M Stubbs, Grammar, text, and ideology: computer-assisted methods in the linguistics of representation.  Applied linguistics 15, 201–223 (1994) Publisher: Oxford University Press.
  * (6) J Eisenstein, Introduction to natural language processing.  (MIT press), (2019).
  * (7) T Brown, et al., Language models are few-shot learners.  Advances in neural information processing systems 33, 1877–1901 (2020).
  * (8) J Wei, et al., Emergent abilities of large language models.  arXiv preprint arXiv:2206.07682 (2022).
  * (9) S Bubeck, et al., Sparks of Artificial General Intelligence: Early experiments with GPT-4.  arXiv preprint arXiv:2303.12712 (2023).
  * (10) T Kojima, SS Gu, M Reid, Y Matsuo, Y Iwasawa, Large language models are zero-shot reasoners.  arXiv preprint arXiv:2205.11916 (2022).
  * (11) PY Wu, JA Tucker, J Nagler, S Messing, Large Language Models Can Be Used to Estimate the Ideologies of Politicians in a Zero-Shot Learning Setting.  arXiv preprint arXiv:2303.12057 (2023).
  * (12) F Gilardi, M Alizadeh, M Kubli, Chatgpt outperforms crowd-workers for text-annotation tasks.  arXiv preprint arXiv:2303.15056 (2023).
  * (13) LP Argyle, et al., Out of One, Many: Using Language Models to Simulate Human Samples.  arXiv preprint arXiv:2209.06899 (2022).
  * (14) M Chmielewski, SC Kucker, An mturk crisis? shifts in data quality and the impact on study results.  Social Psychological and Personality Science 11, 464–473 (2020).
  * (15) DM Ziegler, et al., Fine-tuning language models from human preferences.  arXiv preprint arXiv:1909.08593 (2019).
  * (16) J Schulman, et al., ChatGPT: Optimizing language models for dialogue (2022).
  * (17) PW Anderson, More is different.  Science 177, 393–396 (1972) Publisher: JSTOR.
  * (18) Y Bang, et al., A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity.  arXiv preprint arXiv:2302.04023 (2023).
  * (19) HG Frankfurt, On bullshit.  (Princeton University Press), (2005).
  * (20) L Lucy, D Bamman, Gender and representation bias in GPT-3 generated stories in Proceedings of the Third Workshop on Narrative Understanding.  pp. 48–55 (2021).
  * (21) JT Ornstein, EN Blasingame, JS Truscott, How to Train Your Stochastic Parrot: Large Language Models for Political Texts.  Working paper. https://joeornstein.github.io/publications/ornstein-blasingame-truscott.pdf (2022).
  * (22) A Palmer, A Spirling, Large Language Models Can Argue in Convincing and Novel Ways About Politics: Evidence from Experiments and Human Judgement, (Working paper), Technical report (2023).
  * (23) L van Vliet, P Törnberg, J Uitermark, The Twitter parliamentarian database: Analyzing Twitter politics across 26 countries.  PloS one 15, e0237073 (2020).
  * (24) M Reiss, Testing the Reliability of ChatGPT for Text Annotation and Classification: A Cautionary Remark.  OSF Preprints (2023).
  * (25) J Grimmer, BM Stewart, Text as data: The promise and pitfalls of automatic content analysis methods for political texts.  Political analysis 21, 267–297 (2013).