MnLargeSymbols’164 MnLargeSymbols’171

# Constraints on Symmetry Preserving Gapped Phases
from Coupling Constant Anomalies

T. Daniel Brennan  Department of Physics, University of California San Diego,
9500 Gilman Drive, La Jolla CA 92093-0319, USA

###### Abstract

In this note, we will characterize constraints on the possible IR phases of a
given QFT by anomalies in the space of coupling constants. We will give
conditions under which a coupling constant anomaly cannot be matched by a
continuous family of symmetry preserving gapped phases, in which case the
theory is either gapless, or exhibits spontaneous symmetry breaking or a phase
transition. We additionally demonstrate examples of theories with coupling
constant anomalies which can be matched by a family of symmetry preserving
gapped phases without a phase transition and comment on the interpretation of
our results for the spontaneous breaking of “$(-1)$-form global symmetries.”

##  I Introduction

In recent years, the concept of symmetry has been greatly expanded beyond the
action of groups on local operators to include the fusion and linking action
of all topological operators (see [1, 2, 3, 4, 5, 6, 7] and references
therein). Likewise our understanding of the possible anomalies allowed in QFTs
has also greatly expanded [8, 9, 10, 11, 12]. For example, some anomalies can
only be activated in the presence of certain backgrounds such as on non-spin
manifolds or in particular flux backgrounds [13, 14, 15, 16, 17, 18, 19].
These backgrounds are related to symmetries, but are not necessarily activated
by topological symmetry operators and hence do not correspond to symmetries
themselves, but rather correspond to a sort of “symmetry structure.” In this
paper, we will focus on one such class of anomalies that are not activated by
topological defects called anomalies in the space of coupling constants or
coupling constant anomalies (CC anomalies).

Consider a family of $d$-dimensional quantum field theories ${\cal
T}_{\theta}$ on $X_{d}$ which is indexed by a continuous set of exactly
marginal couplings $\theta\in{\cal S}$. Let us assume that this family of
theories ${\cal T}_{\theta}$ has a group-like global symmetry ${\cal G}$ which
is preserved for all $\theta\in{\cal S}$.

Generically, the set of couplings $\theta\in{\cal S}$ forms a continuous, non-
compact space. However, there can be additional isomorphisms $\Lambda$ between
the theories ${\cal T}_{\theta}$ and ${\cal T}_{\theta+\lambda}$ so that
$\theta\in{\cal S}/\Lambda$. This identification allows us to meaningfully
compare the partition functions $Z_{\cal T}[\theta]$, $Z_{\cal
T}[\theta+\lambda]$. One possible behavior is that the two partition functions
are only identical up to a phase:

$\displaystyle\begin{split}Z_{\cal T}[\theta]=Z_{\cal T}[\theta+\lambda]\times
e^{i\int_{X_{d}}\omega_{d}}\end{split}$ (1)

where $\omega_{d}$ is an anomalous phase that is only dependent on background
gauge fields for ${\cal G}$. When this phase is quantized (i.e. ${\rm
exp}\left\\{ik\int\omega_{d}\right\\}$ is only ${\cal G}$-gauge invariant for
$k\in{\mathbb{Z}}$), there are no local, background gauge invariant counter-
terms that can trivialize the phase. In analogy with standard anomalies, this
phase is referred to as an anomaly in the space of coupling constants [20,
21].

Although this “anomaly” does not correspond to a symmetry (the interface
between ${\cal T}_{\theta}$, ${\cal T}_{\theta+\lambda}$ is generically not
topological), there are still physical consequences of the anomaly. One
feature is that as with standard anomalies, the CC anomaly can be described by
an $(d+1)$-dimensional SPT phase by anomaly inflow:

$\displaystyle\begin{split}{\cal
A}=\int_{Y_{d+1}}d\Theta(x)\cup\omega_{d}\end{split}$ (2)

where $\partial Y_{d+1}=X_{d}$ and $\Theta(x)$ is a $(d+1)$-dimensional ${\cal
S}/\Lambda$-valued function which takes boundary value $\Theta(x)\big{|}_{x\in
X_{d}}=\theta$. This implies that CC anomalies are scale invariant and must be
matched along ${\cal G}$-preserving RG flows.

Note that the framework of CC anomalies encompasses the anomalous phases that
arise in the spurion analysis of anomalous 0-form global symmetries that are
broken by interactions. As with spurions, if we treat $\theta$ as the vev of
some condensed field $\theta(x)$, then the CC anomaly implies that the domain
wall where $\theta(x)$ shifts $\theta\mapsto\theta+\lambda$ will carry a world
volume anomaly given by $\omega_{d}$. See [20, 21, 22] for more discussion and
applications.

One illustrative example of a family of theories with an anomaly in the space
of coupling constants is $4d$ $SU(N)$ Yang-Mills theory. Despite being non-
trivially interacting, it admits an exactly marginal coupling:

$\displaystyle\begin{split}S_{\theta}=...+\frac{i\theta}{8\pi^{2}}\int_{X_{d}}\,\textrm{Tr}\,[F\wedge
F]~{}.\end{split}$ (3)

It is well known that the coupling $\theta$ is $2\pi$-periodic due to the fact
that the instanton number is quantized:

$\displaystyle\begin{split}\int_{X_{d}}\frac{\,\textrm{Tr}\,[F\wedge
F]}{8\pi^{2}}\in{\mathbb{Z}}~{}.\end{split}$ (4)

However, shifting $\theta\mapsto\theta+2\pi$ is non-trivial due to the Witten
effect [23] which implies that the identification ${\cal T}_{\theta}\sim{\cal
T}_{\theta+2\pi}$ involves a redefinition of the electro-magnetic charge
lattice [24].

Yang-Mills also has a ${\mathbb{Z}}_{N}^{(1)}$ 1-form center symmetry. If we
couple the ${\mathbb{Z}}_{N}^{(1)}$ to a background gauge field $B_{2}\in
H^{2}(X_{d};{\mathbb{Z}}_{N})$, the instanton number becomes fractional:

$\displaystyle\begin{split}\int\frac{\,\textrm{Tr}\,[F\wedge
F]}{8\pi^{2}}=\frac{N-1}{2N}\int{\cal P}(B_{2})~{}{\rm
mod}_{\mathbb{Z}}\end{split}$ (5)

where ${\cal P}(B_{2})\in H^{4}(X_{d};{\mathbb{Z}}_{2N})$ is the Pontryagin
square of $B_{2}$ [25, 24]. Thus, the partition functions obey [26]:

$\displaystyle\begin{split}Z_{\cal T}[B_{2};\theta+2\pi]=Z_{\cal
T}[B_{2};\theta]\times e^{\frac{\pi i(N-1)}{N}\int{\cal
P}(B_{2})}~{}.\end{split}$ (6)

Here the partition functions of ${\cal T}_{\theta},{\cal T}_{\theta+2\pi}$ are
related by a quantized phase that is only dependent on the background gauge
fields and hence describes a coupling constant anomaly between $\theta$ and
the ${\mathbb{Z}}_{N}^{(1)}$ global symmetry which can be described by the SPT
phase [21]:

$\displaystyle\begin{split}{\cal A}=i\frac{N-1}{2N}\int d\theta\cup{\cal
P}(B_{2})\end{split}$ (7)

This CC anomaly also implies that if we allow $\theta$ to be space-time
dependent, then the domain wall where $\theta\mapsto\theta+2\pi$ will carry a
world-volume anomaly [21, 27, 28]:

$\displaystyle\begin{split}{\cal A}_{\rm w.v.}=\frac{\pi i(N-1)}{N}\int{\cal
P}(B_{2})\end{split}$ (8)

which indicates that the ${\mathbb{Z}}_{N}^{(1)}$ global symmetry is
spontaneously broken on the domain wall where the theory must deconfine [27,
28]. This can be physically interpreted by noting that as $\theta$ winds
through $\theta+2\pi$ inside the defect, it sweeps through $\theta=\pi$ where
the theory cannot be trivially gapped due to a mixed
${\mathbb{Z}}_{N}^{(1)}$–time-reversal anomaly [26, 27, 20, 21]. This is a
well known feature that appears for example in axion-Yang-Mills theory where
$\theta$ is represented by a dynamical axion whose domain wall carries a world
volume anomaly from UV chiral modes [29, 30, 31, 32, 33, 34, 30, 35, 36, 37].

##  II Constraints from CC Anomalies

In this paper, we will more precisely derive the constraints on RG flows that
are imposed by CC anomalies. Consider a family of $d$-dimensional QFTs ${\cal
T}_{\theta}$ indexed by $\theta\in S^{1}$ on a manifold
$X_{d}^{(p)}=S^{p}\times S^{d-p}$ with a discrete group-like global symmetry

$\displaystyle\begin{split}{\cal G}=G^{(p-1)}\times
G^{(d-p-1)}~{}.\end{split}$ (9)

For a family of (non-anomalous) ${\cal G}$-preserving theories, there exists a
choice of scheme such that

$\displaystyle\begin{split}Z_{\cal T}[A+d\lambda;\theta]=Z_{\cal
T}[A;\theta]\quad,\quad\forall\theta\in S^{1}\end{split}$ (10)

where $A$ represents the background gauge fields for ${\cal G}$. Let us take
such a family of theories with a CC anomaly of the form

$\displaystyle\begin{split}{\cal
A}=i\int\frac{d\theta}{2\pi}\cup\omega_{d}(A)\end{split}$ (11)

where $\omega_{d}\in H^{d}(B{\cal G};U(1))$ is quantized (${\rm
exp}\left\\{ik\int\omega_{d}\right\\}$ is only ${\cal G}$-gauge invariant for
$k\in{\mathbb{Z}}$) that describes the anomalous phase in the transformation
of the partition function under the periodic shift of $\theta$ in (1) 111Here,
$B{\cal G}$ is the classifying space for the symmetry group ${\cal
G}=G^{(p-1)}\times G^{(d-p-1)}$ which is given by: $B{\cal
G}:=B^{p}G^{(p-1)}\times B^{d-p}G^{(d-p-1)}$..

Here we will show that if the anomalous phase can be activated on
$X_{d}^{(p)}$, then the coupling constant anomaly can only be matched in the
IR by:

  1. 1.

A family of gapless theories (i.e. CFTs)

  2. 2.

A continuous family of gapped theories (i.e. TQFTs) with spontaneously broken
${\cal G}$-symmetry

  3. 3.

A discontinuous family of gapped theories – i.e. a collection of gapped phases
with a phase transition

Here we restrict to unitary theories and gapped phases which are semi-simple
and finite 222For an example of a non-finite TQFT, see [50, 51, 52]..

This implies that a family of QFTs with a coupling constant anomaly cannot
flow to a continuous family of ${\cal G}$-preserving gapped phases (without
phase transition) in the IR if the anomalous phase can be activated on
$X_{d}^{(p)}$.

Comment on SSB of (–1)-form Symmetries: The identification ${\cal
T}_{\theta}\sim{\cal T}_{\theta+\lambda}$ is also sometimes referred to as a
“$(-1)$-form global symmetry.” There has been much investigation into the
consequences of this “symmetry” and its spontaneous breaking (SSB) [40, 41,
42, 43].

One way we can try to understand the SSB of $(-1)$-form global symmetries is
by comparing our results on CC anomalies with known results about anomalies of
discrete global symmetries. For anomalies involving discrete symmetries, it is
known that the anomaly can be matched by spontaneously breaking a global
symmetry [44]. The analogous statement we derive here is that a CC anomaly can
similarly be matched by a family of gapped phases with a phase transition.
This suggests that the SSB of a “$(-1)$-form symmetry” corresponds to a phase
transition in the associated parameter space. We leave a more in-depth
analysis for future investigation.

###  II.1 Derivation of Constraints

Let us examine if a family of QFTs with a coupling constant anomaly as
described above can flow to a continuous family of symmetry preserving gapped
phases. Here we define a continuous family of gapped phases by a family of
TQFTs which have isomorphic Hilbert spaces with a smoothly varying inner
product. This implies that the partition function $Z_{\rm TQFT}[S^{p+1}\times
S^{d-p-1};A;\theta]$ is a continuous function over the parameter space. A
continuous family of ${\cal G}$-preserving gapped phases is then a continuous
family of gapped phases for which the symmetry ${\cal G}$ is preserved (i.e.
not spontaneously or explicitly broken) over the entire family. In particular,
a continuous family of gapped phases has no phase transition.

As proved in [44], a unitary ${\cal G}$-preserving TQFT obeys:

$\displaystyle\begin{split}Z_{\rm TQFT}[S^{p}\times S^{d-p};A]\neq
0~{}.\end{split}$ (12)

Here for technical reasons, we will also restrict ourselves to finite TQFTs
which have a finite number of operators/finite dimensional Hilbert spaces on
any compact manifold and implies that the partition function is also finite on
any closed manifold. In particular:

$\displaystyle\begin{split}\Big{|}Z_{\rm TQFT}[S^{p}\times
S^{d-p};A]\Big{|}<\infty~{}.\end{split}$ (13)

Thus, for a continuous family of unitary, ${\cal G}$-preserving, finite TQFTs
we can define the phase

$\displaystyle\begin{split}\frac{Z_{\rm TQFT}[S^{p}\times
S^{d-p};A;\theta]}{|Z_{\rm TQFT}[S^{p}\times
S^{d-p};A;\theta]|}=e^{i\Phi(\theta;A)}~{}.\end{split}$ (14)

The coupling constant anomaly implies that the phase $\Phi$ takes the form:

$\displaystyle\begin{split}\Phi(\theta;A)=f(\theta)\int\omega_{d}(A)+g(\theta)\end{split}$
(15)

where $f(\theta),g(\theta)$ are semi-periodic functions

$\displaystyle\begin{split}f(\theta+2\pi)=f(\theta)+1~{},~{}g(\theta+2\pi)=g(\theta)+2\pi
n~{}.\end{split}$ (16)

For a continuous family of theories, the continuity of the partition function
additionally implies that $f(\theta)$ and $g(\theta)$ are continuous, semi-
periodic functions.

However, since we have assumed the theory is ${\cal G}$-preserving, the
property (10) together with the fact that $\omega_{d}(A)$ is quantized,
requires that $f(\theta)$ only take integer values. Since no continuous
function can be both semi-periodic (16) and take only integer values, there is
no unitary, finite, ${\cal G}$-preserving, continuous family of gapped phases
that can match the CC anomaly.

Our assumptions can be violated if 1.) the theory is gapless, 2.) if ${\cal
G}$ is spontaneously broken, or 3.) the partition function jumps
discontinuously (for example if $f(\theta)$ is a step function) – which would
signal a discontinuous family of theories with a phase transition. Note that
the scenario where ${\cal G}$ is not uniformly broken over $\theta\in S^{1}$
corresponds to the case with a phase transition. This classifies the IR phases
that can occur when the anomalous phase is non-trivial on $X_{d}^{(p)}$ and
proves our result.

##  III Example: $4d$ $SU(N)$ Yang-Mills

Let us again demonstrate our results in the example of $4d$ $SU(N)$ Yang Mills
with a $\theta$ angle. This family of theories has a ${\mathbb{Z}}_{N}^{(1)}$
symmetry as well as a time reversal symmetry ${\cal T}$ at $\theta=0,\pi$.

As discussed above, this theory has the CC anomaly

$\displaystyle\begin{split}{\cal A}_{cc}=i\frac{N-1}{2N}\int d\theta\cup{\cal
P}(B_{2})\end{split}$ (17)

where $B_{2}$ is the background gauge field for ${\mathbb{Z}}_{N}^{(1)}$ so
that the anomalous phase $\omega_{d}(B_{2})=\frac{\pi(N-1)}{N}\int{\cal
P}(B_{2})$ can be activated on $X_{4}=S^{2}\times\tilde{S}^{2}$ [21]:

$\displaystyle\begin{split}e^{i\int_{X^{4}}\omega_{d}(B_{2})}=e^{\frac{2\pi
i}{N}}~{},\quad B_{2}=[S^{2}]^{\vee}+[\tilde{S}^{2}]^{\vee}~{}.\end{split}$
(18)

Additionally, the ${\cal T}$-symmetry at $\theta=0,\pi$ has a mixed anomaly
with ${\mathbb{Z}}_{N}^{(1)}$ at $\theta=\pi$:

$\displaystyle\begin{split}{\cal A}=\frac{\pi i(N-1)}{N}\int w_{1}\cup{\cal
P}(B_{2})~{}.\end{split}$ (19)

This family of theories is believed to be matched in the IR by a discontinuous
family of ${\mathbb{Z}}_{N}^{(1)}$-preserving trivially gapped phases with a
phase transition at $\theta=\pi$. At $\theta=\pi$, the theory is believed to
be a non-trivially gapped phase where ${\cal T}$ is spontaneously broken due
to results from large $N$, ${\cal N}=1$ SYM, and symmetry enforced gaplessness
[26, 45, 20, 44]. This conjecture is consistent with our results and all known
symmetries and anomalies.

According to our results, the symmetries and anomalies of this theory may also
be matched by a continuous family of gapped phases without a phase transition
that spontaneously breaks the ${\mathbb{Z}}_{N}^{(1)}$ center symmetry.
Indeed, such a family of gapped phases is described by the action

$\displaystyle\begin{split}S_{\theta}=&\frac{iN}{2\pi}\int(da_{1}-{\cal
B}_{2})\wedge b_{2}\\\ &+\frac{iN(N-1)\theta}{8\pi^{2}}\int(da_{1}-{\cal
B}_{2})\wedge(da_{1}-{\cal B}_{2})\end{split}$ (20)

where $a_{1},b_{2}$ are $U(1)$ gauge fields of degree 1,2 respectively, ${\cal
B}_{2}=\frac{2\pi}{N}B_{2}$ is an integral lift of $B_{2}$, and $a_{1}$ shifts
so that $da_{1}-{\cal B}_{2}$ is invariant under the choice of lift of $B_{2}$
333Different choices of integral lift of $B_{2}\in
H^{2}(X_{d};{\mathbb{Z}}_{N})$ are related by ${\cal B}_{2}\mapsto{\cal
B}_{2}+2\pi\Lambda_{2}$ where $\Lambda_{2}\in H^{2}(X_{d};{\mathbb{Z}})$. .
This theory has a (spontaneously broken) ${\mathbb{Z}}_{N}^{(1)}$ global
symmetry (generated by $b_{2}$-Wilson surfaces), which is identified with the
UV ${\mathbb{Z}}_{N}^{(1)}$ global symmetry, and time reversal symmetry at
$\theta=\pi$. Because of this identification, the theory reproduces the CC
anomaly (17) and the mixed ${\cal T}$-${\mathbb{Z}}_{N}^{(1)}$ anomaly (19) of
$SU(N)$ Yang-Mills. This phase can be reached by coupling $SU(N)$ Yang-Mills
to multiple adjoint scalar fields which condense along orthogonal directions
in $\mathfrak{su}(N)$.

This family of gapped phases also describes the IR limit of a family of scalar
QED models with action

$\displaystyle\begin{split}S=&\int|D\Phi|^{2}+\frac{1}{2g^{2}}F\wedge\ast
F-V(\Phi)+\frac{iK\theta}{8\pi^{2}}\int F\wedge F\end{split}$ (21)

where $K=N(N-1)$, $\Phi$ is a complex scalar field of charge $N$, and
$V(\Phi)$ is potential with minimum at $|\Phi|^{2}\neq 0$.

##  IV Matching CC Anomalies in Continuous Families of ${\cal G}$-Preserving
TQFTs

We can also demonstrate a family of theories that have a coupling constant
anomaly which can be matched by a continuous family of ${\cal G}$-preserving
gapped phases – i.e. a family of ${\cal G}$-preserving TQFTs without phase
transition.

Consider the families of $6d$ and $8d$ $SU(N)$ Yang-Mills theories with a
$\theta$ angle:

$\displaystyle\begin{split}S_{6d}=\int\frac{1}{2g^{2}}\,\textrm{Tr}\,[F\wedge\ast
F]+i\theta_{6d}\int ch_{3}(F)\\\
S_{8d}=\int\frac{1}{2g^{2}}\,\textrm{Tr}\,[F\wedge\ast F]+i\theta_{8d}\int
ch_{4}(F)\end{split}$ (22)

where $ch_{n}(F)$ is the $n^{th}$ Chern character

$\displaystyle\begin{split}ch_{n}(F)=\frac{1}{n!(2\pi)^{n}}\,\textrm{Tr}\,\underbrace{\left[F\wedge...\wedge
F\right]}_{n-times}~{}.\end{split}$ (23)

The $6d$ and $8d$ theories additionally have a $U(1)^{(1)}$ and
$U(1)^{(1)}\times U(1)^{(3)}$ global symmetries that couples to $ch_{2}(F)$
and $ch_{3}(F),ch_{2}(F)$ respectively. Both of these interacting theories
have dimensionful gauge couplings and flow to a free theory in the IR.
Nevertheless, they both have coupling constant anomalies.

Using the standard decomposition of the Chern character into integral Chern
classes:

$\displaystyle\begin{split}ch_{3}&=\frac{1}{2}c_{3}-\frac{1}{2}c_{1}c_{2}+\frac{1}{6}c_{1}^{3}\\\
ch_{4}&=-\frac{1}{6}c_{4}+\frac{1}{12}c_{2}^{2}+\frac{1}{6}c_{3}c_{1}-\frac{1}{6}c_{2}c_{1}^{2}+\frac{1}{24}c_{1}^{4}\end{split}$
(24)

we see for an $SU(N)$ bundle (which has $c_{1}=0$) that the respective Chern
characters obey the quantization

$\displaystyle\begin{split}\oint
ch_{3}(F_{SU(N)})\in\frac{1}{2}{\mathbb{Z}}~{},\quad\oint
ch_{4}(F_{SU(N)})\in\frac{1}{12}{\mathbb{Z}}\end{split}$ (25)

on a generic manifold 444Note that on particularly nice manifolds that these
quantization conditions can be modified to be less fractional. For example on
an almost complex manifold, the third Chern character is integral [53, 54]. We
will not make any such assumptions here.. The $\theta$ angles are thus
periodic:

$\displaystyle\begin{split}\theta_{6d}\sim\theta_{6d}+4\pi\quad,\quad\theta_{8d}\sim\theta_{8d}+24\pi~{}.\end{split}$
(26)

When we turn on a background gauge field for the 1-form center symmetry, the
quantization of the Chern characters is modified and there is a resulting
coupling constant anomaly. If we couple to a $PSU(N)$ bundle with a fixed
obstruction class $B_{2}\in H^{2}(X_{d};{\mathbb{Z}}_{N})$ which has an
integral lift 555Here we will only focus on ${\mathbb{Z}}_{N}^{(1)}$
background for which $B_{2}\in H^{2}(X_{d};{\mathbb{Z}}_{N})$ has an integral
lift. In general, the fractional part of Chern characters can different when
$B_{2}$ does not have an integer lift., then we can compute the fractional
part of the Chern class by embedding our $PSU(N)$ bundle into a $U(N)$ bundle
as in [26]. This computation is straightforward and results in

$\displaystyle\begin{split}&ch_{3}=\frac{\hat{c}_{3}}{2}-\frac{N-2}{2N}B_{2}\hat{c}_{2}+\frac{(N-1)(N-2)}{6N^{2}}B_{2}^{3}\\\
&ch_{4}=-\frac{\hat{c}_{4}}{6}+\frac{\hat{c}_{2}^{2}}{12}+\frac{(N-3)}{6N}B_{2}\hat{c}_{3}-\frac{K}{6N^{2}}B_{2}^{2}\hat{c}_{2}\\\
&\qquad+\frac{(N-1)K}{24N^{3}}B_{2}^{4}\end{split}$ (27)

where $K=N^{2}-3N+3$ and the $\hat{c}_{n}\in H^{2n}(X_{d};{\mathbb{Z}})$ are
integral classes which are heuristically the parts of the $PSU(N)$ Chern
classes $c_{n}$ that can be lifted to $U(N)$ Chern classes. Note that
$\hat{c}_{2,3,4}$ shift under under the choice of integral lift $B_{2}\mapsto
B_{2}+N\Lambda_{2}$ for $\Lambda_{2}\in H^{2}(X_{d};{\mathbb{Z}})$ so that the
entire expression for $ch_{3,4}$ is invariant $B_{2}$.

Since the classes $\hat{c}_{2,3,4}$ are summed over in the path integral, we
will restrict our attention to shifts

$\displaystyle\begin{split}\theta_{6d}\sim\theta_{6d}+4\pi
N\quad,\quad\theta_{8d}\sim\theta_{8d}+24\pi N^{2}\end{split}$ (28)

so that the partition function shifts by a quantized phase involving only
$B_{2}$, signaling a CC anomaly 666Note that alternatively, the phase
proportional to $\hat{c}_{2}$ for $6d$ Yang-Mills and the terms
$\hat{c}_{2,3}$ for $8d$ Yang-Mills can be additionally removed by turning on
backgrounds for the $U(1)^{(1)}$ and $U(1)^{(1)}\times U(1)^{(3)}$
respectively. .

If we will rescale the $\theta$-angles

$\displaystyle\begin{split}\Theta_{6d}=2N\theta_{6d}~{},\quad\Theta_{8d}=12N^{2}\theta_{8d}~{},\quad\Theta\sim\Theta+2\pi\end{split}$
(29)

the coupling constant anomalies are given by

$\displaystyle\begin{split}{\cal A}_{6d}&=i\frac{(N-1)(N-2)}{3N}\int
d\Theta_{6d}\cup B_{2}^{3}\\\ {\cal A}_{8d}&=i\frac{(N-1)(N^{2}-3N+3)}{2N}\int
d\Theta_{8d}\cup B_{2}^{4}\end{split}$ (30)

Notice that these examples evade our classification due to the fact that for
all choices of $X_{d}^{(q)}=S^{q}\times S^{d-q}$ and $B_{2}\in
H^{2}(X_{d};{\mathbb{Z}}_{N})$ the anomalous phases are trivial.

Any family of theories that carries the CC anomaly ${\cal A}_{8d}$ involving a
${\mathbb{Z}}_{N}^{(1)}$ symmetry as above can be matched by a family of
${\mathbb{Z}}_{N}^{(1)}$-preserving gapped phases given by

$\displaystyle\begin{split}S=&\frac{iN}{2\pi}\int\left(db_{3}-\frac{N}{2\pi}{\cal
B}_{2}^{2}\right)\wedge a_{4}\\\ &+\frac{i\theta\kappa
N}{8\pi^{2}}\int\left(db_{3}-\frac{N}{2\pi}{\cal
B}_{2}^{2}\right)\wedge\left(db_{3}-\frac{N}{2\pi}{\cal
B}_{2}^{2}\right)\end{split}$ (31)

where $b_{3},a_{4}$ are $U(1)$ gauge fields of degree 3,4 respectively,
$\kappa=(N-1)(N^{2}-3N+3)$, and ${\cal B}_{2}=\frac{2\pi}{N}B_{2}$ is an
integral lift of $B_{2}$ where $b_{3}$ shifts so that
$db_{3}-\frac{1}{2\pi}{\cal B}_{2}^{2}$ is invariant under the choice of lift
of $B_{2}$. It is unknown if there is a deformation of $8d$ Yang Mills that
flows to this gapped phase.

On the other hand, although our results provide no constraints on the IR phase
of $6d$ Yang-Mills, it is unclear how to construct an analogous
${\mathbb{Z}}_{N}^{(1)}$-preserving family of TQFTs that matches the CC
anomaly ${\cal A}_{6d}$ in (30).

Additionally, as mentioned above, both $6d$ and $8d$ Yang-Mills have
additional global symmetries – $U(1)^{(1)}$ and $U(1)^{(1)}\times U(1)^{(3)}$
respectively (which couple to the Chern characters $ch_{2,3}(F)$ as
appropriate) – as well as ${\cal T}$-symmetry. These symmetries all have mixed
anomalies with ${\mathbb{Z}}_{N}^{(1)}$ due to the fractional part of the
Chern characters. For $8d$ Yang-Mills, the mixed ${\cal
T}$-${\mathbb{Z}}_{N}^{(1)}$ anomaly is matched by the family of TQFTs above.
However it is unclear if anomalies involving continuous symmetries can be
matched by a symmetry preserving gapped phase. If such anomalies were to
obstruct a ${\mathbb{Z}}_{N}^{(1)}\times U(1)^{(1)}\times
U(1)^{(3)}$-preserving gapped phase for $8d$ Yang-Mills, this would imply that
any deformation that causes the UV theory to flow to the gapped phase in (31)
(if any such deformation exists) must necessarily break $U(1)^{(1)}\times
U(1)^{(3)}$.

The fact that ${\cal A}_{8d}$ can be matched by a family of
${\mathbb{Z}}_{N}^{(1)}$-preserving gapped phases and that the CC anomaly of
$4d$ $SU(N)$ Yang-Mills can be matched by a continuous family of gapped phases
with ${\mathbb{Z}}_{N}^{(1)}$ SSB demonstrates that coupling constant
anomalies do not always imply the existence of a phase transition in the IR.

## Acknowledgements

The authors would like to thank Ken Intriligator, Po-Shen Hsin, Kantaro
Ohmori, Anuj Apte, Sungwoo Hong, and Clay Córdova for helpful discussions and
related collaborations. TDB is supported by Simons Foundation award 568420
(Simons Investigator) and award 888994 (The Simons Collaboration on Global
Categorical Symmetries).

## References

  * [1] C. Cordova, T. T. Dumitrescu, K. Intriligator, and S.-H. Shao, “Snowmass White Paper: Generalized Symmetries in Quantum Field Theory and Beyond,” in Snowmass 2021.  5, 2022.  arXiv:2205.09545 [hep-th].
  * [2] D. S. Freed, G. W. Moore, and C. Teleman, “Topological symmetry in quantum field theory,” arXiv:2209.07471 [hep-th].
  * [3] D. Gaiotto, A. Kapustin, N. Seiberg, and B. Willett, “Generalized Global Symmetries,” JHEP 02 (2015) 172, arXiv:1412.5148 [hep-th].
  * [4] S. Schafer-Nameki, “ICTP Lectures on (Non-)Invertible Generalized Symmetries,” arXiv:2305.18296 [hep-th].
  * [5] T. D. Brennan and S. Hong, “Introduction to Generalized Global Symmetries in QFT and Particle Physics,” arXiv:2306.00912 [hep-ph].
  * [6] L. Bhardwaj, L. E. Bottini, L. Fraser-Taliente, L. Gladden, D. S. W. Gould, A. Platschorre, and H. Tillim, “Lectures on Generalized Symmetries,” arXiv:2307.07547 [hep-th].
  * [7] S.-H. Shao, “What’s Done Cannot Be Undone: TASI Lectures on Non-Invertible Symmetry,” arXiv:2308.00747 [hep-th].
  * [8] J. Kaidi, E. Nardoni, G. Zafrir, and Y. Zheng, “Symmetry TFTs and anomalies of non-invertible symmetries,” JHEP 10 (2023) 053, arXiv:2301.07112 [hep-th].
  * [9] C. Zhang and C. Córdova, “Anomalies of $(1+1)D$ categorical symmetries,” arXiv:2304.01262 [cond-mat.str-el].
  * [10] Z. Sun and Y. Zheng, “When are Duality Defects Group-Theoretical?,” arXiv:2307.14428 [hep-th].
  * [11] C. Cordova, P.-S. Hsin, and C. Zhang, “Anomalies of Non-Invertible Symmetries in (3+1)d,” arXiv:2308.11706 [hep-th].
  * [12] A. Antinucci, F. Benini, C. Copetti, G. Galati, and G. Rizi, “Anomalies of non-invertible self-duality symmetries: fractionalization and gauging,” arXiv:2308.11707 [hep-th].
  * [13] T. D. Brennan and K. Intriligator, “Anomalies of 4d $Spin_{G}$ Theories,” arXiv:2312.04756 [hep-th].
  * [14] J. Wang, X.-G. Wen, and E. Witten, “A New SU(2) Anomaly,” J. Math. Phys. 60 no. 5, (2019) 052301, arXiv:1810.00844 [hep-th].
  * [15] T. D. Brennan, C. Cordova, and T. T. Dumitrescu, “Line Defect Quantum Numbers & Anomalies,” arXiv:2206.15401 [hep-th].
  * [16] T. D. Brennan, “Anomaly Enforced Gaplessness and Symmetry Fractionalization for $Spin_{G}$ Symmetries,” arXiv:2308.12999 [hep-th].
  * [17] T. D. Brennan and A. Sheckler, “Anomaly Enforced Gaplessness for Background Flux Anomalies and Symmetry Fractionalization,” arXiv:2311.00093 [hep-th].
  * [18] M. M. Anber and E. Poppitz, “On the baryon-color-flavor (BCF) anomaly in vector-like theories,” JHEP 11 (2019) 063, arXiv:1909.09027 [hep-th].
  * [19] M. M. Anber and E. Poppitz, “Generalized ’t Hooft anomalies on non-spin manifolds,” JHEP 04 (2020) 097, arXiv:2002.02037 [hep-th].
  * [20] C. Córdova, D. S. Freed, H. T. Lam, and N. Seiberg, “Anomalies in the Space of Coupling Constants and Their Dynamical Applications I,” SciPost Phys. 8 no. 1, (2020) 001, arXiv:1905.09315 [hep-th].
  * [21] C. Córdova, D. S. Freed, H. T. Lam, and N. Seiberg, “Anomalies in the Space of Coupling Constants and Their Dynamical Applications II,” SciPost Phys. 8 no. 1, (2020) 002, arXiv:1905.13361 [hep-th].
  * [22] Y. Choi and K. Ohmori, “Higher Berry phase of fermions and index theorem,” JHEP 09 (2022) 022, arXiv:2205.02188 [hep-th].
  * [23] E. Witten, “Dyons of Charge e theta/2 pi,” Phys. Lett. B 86 (1979) 283–287.
  * [24] O. Aharony, N. Seiberg, and Y. Tachikawa, “Reading between the lines of four-dimensional gauge theories,” JHEP 08 (2013) 115, arXiv:1305.0318 [hep-th].
  * [25] E. Witten, “Supersymmetric index in four-dimensional gauge theories,” Adv. Theor. Math. Phys. 5 (2002) 841–907, arXiv:hep-th/0006010.
  * [26] D. Gaiotto, A. Kapustin, Z. Komargodski, and N. Seiberg, “Theta, Time Reversal, and Temperature,” JHEP 05 (2017) 091, arXiv:1703.00501 [hep-th].
  * [27] D. Gaiotto, Z. Komargodski, and N. Seiberg, “Time-reversal breaking in QCD4, walls, and dualities in 2 + 1 dimensions,” JHEP 01 (2018) 110, arXiv:1708.06806 [hep-th].
  * [28] M. M. Anber and E. Poppitz, “Deconfinement on axion domain walls,” JHEP 03 (2020) 124, arXiv:2001.03631 [hep-th].
  * [29] C. G. Callan, Jr. and J. A. Harvey, “Anomalies and Fermion Zero Modes on Strings and Domain Walls,” Nucl. Phys. B 250 (1985) 427–436.
  * [30] R. Jackiw and P. Rossi, “Zero Modes of the Vortex - Fermion System,” Nucl. Phys. B 190 (1981) 681–691.
  * [31] N. K. Nielsen and B. Schroer, “Topological Fluctuations and Breaking of Chiral Symmetry in Gauge Theories Involving Massless Fermions,” Nucl. Phys. B 120 (1977) 62–76.
  * [32] M. M. Ansourian, “Index Theory and the Axial Current Anomaly in Two-Dimensions,” Phys. Lett. B 70 (1977) 301–305.
  * [33] J. E. Kiskis, “Fermions in a Pseudoparticle Field,” Phys. Rev. D 15 (1977) 2329.
  * [34] E. J. Weinberg, “Index Calculations for the Fermion-Vortex System,” Phys. Rev. D 24 (1981) 2669.
  * [35] C. Callias, “Index Theorems on Open Spaces,” Commun. Math. Phys. 62 (1978) 213–234.
  * [36] R. Jackiw and C. Rebbi, “Solitons with Fermion Number 1/2,” Phys. Rev. D 13 (1976) 3398–3409.
  * [37] T. D. Brennan, S. Hong, and L.-T. Wang, “Coupling a Cosmic String to a TQFT,” JHEP 03 (2024) 145, arXiv:2302.00777 [hep-ph].
  * [38] Here, $B{\cal G}$ is the classifying space for the symmetry group ${\cal G}=G^{(p-1)}\times G^{(d-p-1)}$ which is given by: $B{\cal G}:=B^{p}G^{(p-1)}\times B^{d-p}G^{(d-p-1)}$.
  * [39] For an example of a non-finite TQFT, see [50, 51, 52].
  * [40] D. Aloni, E. García-Valdecasas, M. Reece, and M. Suzuki, “Spontaneously Broken $(-1)$-Form U(1) Symmetries,” arXiv:2402.00117 [hep-th].
  * [41] T. Vandermeulen, “Lower-Form Symmetries,” arXiv:2211.04461 [hep-th].
  * [42] T. Vandermeulen, “Gauge Defects,” arXiv:2310.08626 [hep-th].
  * [43] E. Sharpe, “An introduction to decomposition,” arXiv:2204.09117 [hep-th].
  * [44] C. Córdova and K. Ohmori, “Anomaly Obstructions to Symmetry Preserving Gapped Phases,” arXiv:1910.04962 [hep-th].
  * [45] E. Witten, “Theta dependence in the large N limit of four-dimensional gauge theories,” Phys. Rev. Lett. 81 (1998) 2862–2865, arXiv:hep-th/9807109.
  * [46] Different choices of integral lift of $B_{2}\in H^{2}(X_{d};{\mathbb{Z}}_{N})$ are related by ${\cal B}_{2}\mapstochar\rightarrow{\cal B}_{2}+2\pi\Lambda_{2}$ where $\Lambda_{2}\in H^{2}(X_{d};{\mathbb{Z}})$.
  * [47] Note that on particularly nice manifolds that these quantization conditions can be modified to be less fractional. For example on an almost complex manifold, the third Chern character is integral [53, 54]. We will not make any such assumptions here.
  * [48] Here we will only focus on ${\mathbb{Z}}_{N}^{(1)}$ background for which $B_{2}\in H^{2}(X_{d};{\mathbb{Z}}_{N})$ has an integral lift. In general, the fractional part of Chern characters can different when $B_{2}$ does not have an integer lift.
  * [49] Note that alternatively, the phase proportional to $\hat{c}_{2}$ for $6d$ Yang-Mills and the terms $\hat{c}_{2,3}$ for $8d$ Yang-Mills can be additionally removed by turning on backgrounds for the $U(1)^{(1)}$ and $U(1)^{(1)}\times U(1)^{(3)}$ respectively.
  * [50] T. D. Brennan and Z. Sun, “A SymTFT for Continuous Symmetries,” arXiv:2401.06128 [hep-th].
  * [51] F. Bonetti, M. Del Zotto, and R. Minasian, “SymTFTs for Continuous non-Abelian Symmetries,” arXiv:2402.12347 [hep-th].
  * [52] A. Antinucci and F. Benini, “Anomalies and gauging of U(1) symmetries,” arXiv:2401.10165 [hep-th].
  * [53] F. Apruzzi, L. Bhardwaj, D. S. W. Gould, and S. Schafer-Nameki, “2-Group symmetries and their classification in 6d,” SciPost Phys. 12 no. 3, (2022) 098, arXiv:2110.14647 [hep-th].
  * [54] H. Geiges, “Chern numbers of almost complex manifolds,” Proceedings of the American Mathematical Society 129 no. 12, (2001) 3749–3752. http://www.jstor.org/stable/2699530.


The temporal and phase angle coverage of an asteroid imaged by ATLAS will vary
depending on many factors, e.g. accessible viewing geometry relative to the
Earth, sky location relative to bright objects (such as the Moon, Jupiter and
Saturn) and/or missed observations due to weather/technical issues. Likewise
the quality of observations obtained by dophot will depend on factors such as
object brightness, photometric seeing and/or sky brightness. Furthermore, the
matching of dophot detections to known asteroid ephemerides may lead to false
matches with stellar sources in the high density galactic plane or glints near
bright objects and other such artifacts. Therefore, poor quality phase curves
can occur when there is not enough observational coverage to adequately
constrain the phase curve, photometric uncertainty is high and/or there is
significant outlying photometric observations.

We performed data cuts on the observations of each asteroid to account for
obvious photometric outliers (section 2.4), although our methods required
sufficient data in each apparition such that the initial $HG(G=0.15)$ fit
provided a reasonable description of the general phase curve behaviour.
Considering the individual apparitions of each asteroid reduces the number of
observations used for each phase curve fit, as such after performing our phase
curve fitting algorithm there will many apparitions which do not produce
accurate phase curve fits. Therefore we used a series of cuts based on various
metrics describing the observational coverage and residuals of each phase
curve fit to reject poor quality apparitions from our final dataset. By
analysing the residuals of the phase curve fit at each apparition we were able
to identify and exclude individual apparitions for which an accurate phase
curve could not be obtained. Therefore one bad apparition did not necessarily
lead to complete rejection of an object. The primary apparition provides us
with the fitted phase parameter ($G$ or $G_{12}$) for each asteroid, therefore
these apparitions had somewhat stricter requirements on the number of data
points and phase angle coverage. The phase parameter was kept fixed when
fitting the non-primary apparitions. For these fits the most common phase
curve error was unphysical shifts in $H$ caused by outlying photometry missed
by our data selection, poor phase angle coverage, and/or large rotational
variations driven by asteroid shape. We found that these problematic
apparitions were best characterised by the statistical properties of their
residuals. For a high quality phase curve fit of an asteroid with low
rotational amplitude we would expect the residuals to be approximately
normally distributed around a residual of 0 with low standard deviation777In
reality the likelihood function of brightness for a sinusoidal lightcurve
peaks at the maximum and minimum amplitude (see figure 1 of SB21, ) although
this would not be clearly seen for low amplitude lightcurves given the typical
accuracy of ATLAS photometry.. As such the standard deviation and mean value
of the residuals proved to be valuable metrics. We also performed a
Kolmogorov-Smirnov test to identify distributions of residuals that differed
significantly from a normal distribution. Furthermore, we used the Pearson
correlation coefficient of the residuals as a function of phase angle as a
metric. This metric should ideally be near zero (indicating no correlation)
but will approach $1$ or $-1$ when there is strong correlation in the
residuals. Correlated residuals implies that the phase curve model was not
consistent with the observations.

The values of each metric cut were chosen through a process of trial and error
alongside detailed visual inspection of individual B89 phase curve fits. As
far as possible we tried to avoid choosing values in a way that directly
influenced the final distributions of phase curve parameters, i.e. we did not
simply cut phase curves with $G<0$ or $G>1$. We did however choose values that
led to a distribution such that the vast majority ($97.7\%$) of $HG$ fits had
$0<G<1$. This also removed the objects with extremely high uncertainties in
$G$, while not removing an excessive number of objects from the final
database. We did have to perform a hard cut in $\sigma_{H}$, $\sigma_{G}$ as a
final step, as there was no other combination of metric cuts that could
consistently remove obvious poor quality phase curves. This approach was
justified by detailed visual inspection of many phase curves and their
residuals from which it was clear that phase curves with extreme $G$ and
$\sigma_{G}$ were unphysical, and as expected were caused by a combination of
poor coverage, bad photometry, and/or unaccounted rotational variation.

The cuts on the primary apparition (which was always in the $o$-filter) for
each asteroid were as follows:

  * •

Number of data points (after outlier rejection) $\geq 50$

  * •

$\geq 1$ observation at phase angle $<5\,$\mathrm{d}\mathrm{e}\mathrm{g}$$

  * •

Phase angle range;
$\text{max}(\alpha_{\textrm{app}})-\text{min}(\alpha_{\textrm{app}}$) $\geq
5\,$\mathrm{d}\mathrm{e}\mathrm{g}$$

  * •

Absolute magnitude uncertainty $\sigma_{H}$(or $\sigma_{H12}$)$<0.05$

  * •

Phase parameter uncertainty $\sigma_{G}(\text{or}\ \sigma_{G12})<0.2$

All apparitions (primary and non primary, $o$ and $c$-filters) had to satisfy
the following requirements:

  * •

Number of observations (after outlier rejection) $\geq 10$

  * •

Phase angle ratio (the ratio of phase angle range of this apparition to the
phase angle range of all observations in the same filter for the asteroid)
$\geq 0.3$

  * •

Standard deviation of residuals $<0.4$

  * •

Mean value of residuals $<0.2$

  * •

Absolute value of the Pearson correlation coefficient of residuals $<0.25$

  * •

Kolmogorov-Smirnov statistic (for normal distribution) $<0.3$

##  Appendix B Comparing $HG$ and $H_{12}G_{12}$

In order to compare the properties of the $HG$ (B89) and $H_{12}G_{12}$ (P16)
phase curve systems we generated model $HG$ phase curves across a phase angle
range of $0\rightarrow 25\,$\mathrm{d}\mathrm{e}\mathrm{g}$$, which is the
typical phase angle range for asteroids observed by ATLAS (figure 1). We
generated a number of phase curves with slope parameters $0<G<1$. We then
performed a linear least squares fit of the $H_{12}G_{12}$ model to each $HG$
phase curve, the results of which are shown in figure 17. Both models are in
general agreement at larger phase angles where brightening is approximately
linear, however, the differing treatment of the opposition brightness surge
(through their basis functions, equations 4 and 5) by the two models is clear
as phase angles decreases. From this simple analysis we obtain an empirical
mapping of $G\rightarrow G_{12}$ which provides a good description of the
relation between $G$ and $G_{12}$ for the SAP dataset (figure 8). We performed
a 3rd-order polynomial fit for each relation, the parameters of which are as
follows

$\displaystyle G_{12}=-0.7451G^{3}+2.577G^{2}-3.716G+1.124$ (9a)
$\displaystyle H_{12o}-H_{o}=-0.1686G^{3}+0.5697G^{2}-1.231G+0.1815$ (9b)

This behaviour also explains why our phase curve fits for $H_{12}G_{12}$ more
frequently stray into “unphysical” solutions, i.e. solutions outside of $0\leq
G_{12}\leq 1$ (figure 7). Furthermore, these model fits clearly illustrate the
origin of the different absolute magnitudes obtained with the two phase curve
systems (figure 8), where the relation between $\Delta H=H_{12}-H$ and $G$ is
reflected well in the ATLAS data. This idealised test shows that even if there
are observations all the way down to zero phase the fitted $H_{12}G_{12}$
model will differ significantly from the $HG$ fit. Therefore $\Delta H$ in our
dataset is not primarily caused by a lack of information at low phase angles,
rather an inherent difference between the two models.

Figure 17:  Plots of B89 model $HG$ phase curves (blue curves), which were
generated with a range of $0\leq G\leq 1$. Each $HG$ phase curve was then
fitted with the P16 $H_{12}G_{12}$ model (orange curves). The two models have
best agreement at large phase angles, however, they vary strongly at lower
phase angles leading to differences in absolute magnitude.

##  Appendix C Jupiter Trojan KS bootstrap tests

Here we present the results of performing KS tests on bootstrapped data
(resampled with replacement) in order to analyse the distributions of Jupiter
Trojan parameters $H$, $G$ and $H_{c}-H_{o}$. For each parameter, 1000
bootstrapped KS tests were performed and the distribution of the resulting $D$
values is shown in figure 18. In these figures we compare the median $D$ value
for each set of KS tests and compare this to the critical value of $D$
required for rejecting the null hypothesis at a significance of 0.05.

$D_{\text{crit}}=1.36\sqrt{(n+m)/(n\cdot m)}$ (10)

where $n$ and $m$ are the number of data points in the two samples being
considered, and the constant of 1.36 refers to a significance of $5\%$ (Knuth,
1998). As described in section 4.3 we found that this method was is more
accurate than relying on the results of a single KS test of what is a
relatively small set of objects. We confirm differences in the distribution of
$H$ between the L4 and L5 groups as observed by ATLAS; the distribution of $D$
for resampling tests is consistently higher than $D_{\text{crit}}$ (figure 18,
left panel). In contrast, although some tests for slope parameter and colour
exceed $D_{\text{crit}}$, the distributions of $D$ are more often below or
near $D_{\text{crit}}$ (figure 18, middle and right panels). As such this
indicates that any differences in $H_{c}-H_{o}$ colour and phase parameter $G$
are not statistically significant enough to reject the null hypothesis.

Furthermore, we repeated this analysis for the values of $H$, $G$ and $g-r$
colour published by SB21 (figure 19). These results show that differences in
the distributions of $H$ for the L4/L5 groups is not present in their analysis
of ZTF data. This is as expected given that the initial rockAtlas watchlist
used in this study started with different $H$ distributions of the L4/L5
Trojans (section 4.3). Similar to our results there are no statistically
significant differences between L4/L5 for $G$ or $g-r$ in ZTF data (SB21)

Figure 18:  Distributions of $D$ values for KS tests comparing $H$, $G$ and
$H_{c}-H_{o}$ colour of the L4 and L5 Jupiter Trojans in the SAP dataset
(left, middle and right panels respectively; compare to figure 15 CDFs). The
vertical blue line indicates the median $D$ of each distribution and the red
line indicates the critical value $D_{\text{crit}}$ for rejection of the null
hypothesis that the two distributions are the same (equation 10).  Figure 19:
Similar to figure 18, but analysing the $H$, $G$ and $g-r$ colours of Jupiter
Trojans as published by Schemel & Brown (2021).


longtable Ginwidth=,height=,keepaspectratio

# AI-assisted coding: Experiments with GPT-4

Russell A. Poldrack
Department of Psychology
Stanford University
Stanford, CA, 94305
<EMAIL_ADDRESS>

Thomas Lu
Department of Linguistics
University of California
Berkeley, CA, 94720-2650

Gašper Beguš
Department of Linguistics
University of California
Berkeley, CA, 94720-2650
<EMAIL_ADDRESS>0000-0001-6755-0259 0000-0002-6459-0551

###### Abstract

Artificial intelligence (AI) tools based on large language models have
acheived human-level performance on some computer programming tasks. We report
several experiments using GPT-4 to generate computer code. These experiments
demonstrate that AI code generation using the current generation of tools,
while powerful, requires substantial human validation to ensure accurate
performance. We also demonstrate that GPT-4 refactoring of existing code can
significantly improve that code along several established metrics for code
quality, and we show that GPT-4 can generate tests with substantial coverage,
but that many of the tests fail when applied to the associated code. These
findings suggest that while AI coding tools are very powerful, they still
require humans in the loop to ensure validity and accuracy of the results.

_K_ eywords artificial intelligence • software engineering • reproducibility

##  1 Introduction

Recent developments in artificial intelligence, particularly through large
language models, have enabled the automated generation of computer code (Chen
et al. 2021; Bubeck et al. 2023). In particular, GPT-4 has enabled human-level
performance on a set of coding challenges that are outside of the training set
of the model (Bubeck et al. 2023). In addition, automated coding assistants
(particularly Github Copilot) have become integrated into commmon devlopment
environments and are widely used, with some evidence that they can
signficantly improve coding productivity. The performance of these models is
also raising important questions regarding coding education, given that the
current models can easily complete most coding problem sets using in
introductory programming courses (Finnie-Ansley et al. 2022).

In the present paper we explore some of the implications of AI-assisted coding
using GPT-4, in a more qualitative way than previous benchmarking assessments.
First we examine the experience of interactive coding and debugging using the
ChatGPT interface to GPT-4 on a set of data science coding problems. This
experiment is meant to approximate the experience of a researcher with minimal
expertise in prompt engineering, assessing the success and amount of effort
required to perform these coding tasks. Second, we assess the ability of GPT-4
(using the OpenAI API) to refactor and improve the quality of existing code.
This experiment is meant to assess the degree to which AI coding assistants
might improve coding quality when used by researcers. Third, we assess the
ability of GPT-4 to write tests for its own code, using a set of test prompts
from several scientific domains. We conclude with an overall assessment of the
utility of AI coding assistants for scientific researchers.

A fully reproducible workflow for this manuscript is available at
https://github.com/poldrack/ai-coding-experiments.

##  2 Coding with GPT-4

Our first set of experiments examined the ability of GPT-4 (via the ChatGPT
interface) to generate usable code for a set of data science problems. The
prompts were generated manually (by author RP) and are listed in Appendix 1.
Each prompt was submitted in a separate chat session; the human examined the
resulting code, and issued additional prompts to try to fix problems. If the
human was not able to obtain working code within about 5 minutes of effort or
less, the problem was deemed unsolved. The results of this experiment are
primarily qualitiative and subjective, but are meant to index the degree to
which GPT-4 is a useful tool for a researcher with minimal prompt engineering
skills.

Figure 1: Proportion of successful code outcomes as a function of number of
prompts. NS: not successful.

Figure 1 shows the proportion of successful outcomes as a function of the
number of prompts required. 72% (23/32)) of attempts were successful in
relatively quickly solving the problem; 37.5% (12/32) were successful on the
first prompt. In cases where additional prompting was required, a common
problem was the use of outdated functions or datasets from Python libraries.

The causes of unsuccessful outcomes were varied. In some cases it was due to
the outdated nature of the GPT-4 training data. For example, in one case
(prompt #12) ChatGPT could not successfully implement a solution that was
compatible with the latest version of PyTorch. In another case (prompt #27),
the labels used to query the NOAA API were incorrect, and the correct labels
were not easily identifable upon further examination. In other cases, it was
not immediately clear what was wrong with the code, and the researcher did not
dig deeply enough to identify the root cause.

One of the examples highlights the contining challenges that ChatGPT has with
mathematical processing (as outlined by Bubeck et al. (2023)). Prompt #4 asked
ChatGPT to generate code to simulate a drift diffusion model (a common model
for response times in cognitive psychology) and then fit the model parameters
using the EZ-diffusion model (Wagenmakers, Van Der Maas, and Grasman 2007),
which is a closed-form solution to estimating these parameters using response
time and accuracy statistics. The initial code generated by ChatGPT attempted
to fit a diffusion model through numerical optimization. After being prompted
to generate a closed-form solution based on the original paper, ChatGPT did
so, but the formula bore little resemblance to the actual formula from the
paper. This is an example of a “hallucination” which is commonly seen with
LLMs (Ji et al. 2023), and highlights the ongoing need for automatically
generated code to be validated by a human.

Another example also highlights the need for sophisticated domain knowledge in
assessing the outputs of ChatGPT. Prompt #18 asked ChatGPT to implement a
_hurdle model_ , which is a statistical model for zero-inflated count data
that combines a binary model with count model using a truncated distribution.
In general, this model is fit by performing maximum likelihood estimation on
the combined likelihoods of the binary and count models. ChatGPT generated a
solution that separately estimated a binary model and a count model, and then
combined the predictions from the two models; this incorrect approach can be
found in a number of examples from Github. This model fit the test data nearly
as well as a reference implementation of the hurdle model (impmented in R),
but is incorrect in comparison to the reference implementation. This
highlights the need for close attention to detail in the implementation of any
numerical methods, as incorrect implementations present on Github can result
in incorrect outcomes.

###  2.1 Refactoring code using GPT4

The quality of research code is important for scientific reproducibility and
transparency, as well as for code reusability and maintainability. In our
initial explorations of code generated by GPT-4, we noted that the
automatically generated code appeared to be substantially more readable than
research code that we have encountered (or written) in the past. This led us
to ask whether GPT-4 could improve the quality of existing code through
_refactoring_ (Fowler 2019), by which we mean modifying code to make it more
readable or maintainable without changing its behavior.

To assess this, we downloaded more than 2000 examples of Python code from
Github using the Githb Code Search API. Only one code file was allowed from
any single repository. We further filtered these files, based in part on the
criteria used by Chen et al. (2021) to select code for training of the OpenAI
Codex model. Exclusion criteria included:

  * •

Presence of non-Python code (as guessed by guesslang.Guess())

  * •

Presence of non-English language in the code (according to pycld2.detect())

  * •

Presence of potential markers of automatic generation (e.g. strings such as
“autogenerated”, “generated by Django”, etc)

  * •

Presence of potential obfuscation (the “if x - y:” pattern)

  * •

Lack of at least one function definition

  * •

Number of GPT tokens greater than 2500 or less than 200

  * •

Maximum line length > 1000

  * •

Mean line length > 100

  * •

Maximum file size > 1MB

The 274 code files passing these criteria were submitted to further analysis.
Analysis of syntax errors and coding style was performed using the flake8
static code analysis package. Files were further excluded on the basis of any
syntax errors identified by flake8.

The number of warning and error messages emitted by the flake8 linter was
substantially reduced for the refactored code compared to the original (median
0.23 messages/line for original vs 0.09 messages/line for refactored, Cohen’s
d = 0.50; see Figure 2). While tools exist to perform automatic reformatting
to ensure standard-compliance, this shows that GPT-4 generates code that is
substantially more standards-compliant than the average programmer; given that
the files sampled from Github were heavily filtered, this result is probably
an underestimate compared to the broader population of Python code on Github.
Figure 3 provides an overview of which errors were most common in the original
code and how their prevalence changed after refactoring.

Figure 2: Number of Flake8 messages (per line of code) for original github
files and refactored files. Figure 3: Prevalence of individual Flake8
warning/error codes for original github files and refactored files. Values are
sorted by prevalence in the original github files.

We further examined a set of code quality metrics, which were computed using
the radon Python package. Metrics extracted from these outputs for further
analysis included:

  * •

Logical lines of code

  * •

Number of comments

  * •

Mean cyclomatic complexity (a measure of the number of execution paths)

  * •

Maintainability index (a holistic metric for code maintainability, based on a
composite of several metrics including Halstead volume Halstead (1977),
cyclomatic complexity, lines of code, and % of comments)

  * •

Halstead “difficulty” (a metric of how difficult the code is to read, based on
the number of distinct operators and the ratio of total number of operands to
number of distinct operands)

  * •

Halstead “bugs” (a metric meant to estimate the number of bugs in delivered
code)

Comparisons between metrics for the original and refactored code are shown in
Figure 4, and means, effect sizes, and p-values for the comparison (using a
paired t-test) are shown in Table LABEL:tbl-metrics. Each of these metrics
differed between the origin and refactored code (p < .05 after false discovery
rate correction across all hypotheses). However, the effect sizes were all in
the small to medium range, with Cohen’s d values ranging from 0.13 to 0.33.

(a) Logical lines of code

(b) Comments

(c) Mean cyclomatic complexity

(d) Halstead bugs

(e) Halstead difficulty

(f) Maintainability index

Figure 4: Code quality metrics computed for each original file and its refactored version. Table 1: Metric comparisons between original and refactored code. |  |  |  |
---|---|---|---|---
|  Median (github) |  Median (GPT-4) |  Cohens d |  P-value (FDR)
Maintainability index |  70.285 |  74.092 |  0.33 |  <.001
Halstead bugs |  0.081 |  0.068 |  0.13 |  0.045
Halstead difficulty |  3.214 |  3.089 |  0.16 |  0.012
flake8 messages per line |  0.237 |  0.089 |  0.50 |  <.001
Mean cyclomatic complexity |  3.462 |  3.284 |  0.18 |  0.006
Number of comments |  7.81 |  7.086 |  0.08 |  0.196
Logical lines of code |  46.022 |  43.372 |  0.27 |  <.001

###  2.2 Automatically generated code and tests

Given the importance of validating AI-generated code using software tests, we
next assessed the ability of GPT-4 to generate tests for its own code. We
first used GPT-4 to generate 20 prompts for each of 5 different areas of
research, using the following prompt: “Please generate 20 prompts to ask a
chatbot to create Python code to solve a variety of {statistical and data
science, physics, theoretical computer science, ecology, economics} problems.”

For each of the generated problems, we created a prompt to generate the code
along with tests for the resulting code, such as the following:

> Write a Python program to simulate predator-prey interactions using the
> Lotka-Volterra equations, given the initial populations, growth rates, and
> interaction coefficients. Please embed the code within an explicit code
> block, surrounded by triple-backtick markers. Generate realistic values for
> any examples, and do not use input() commands. Create code that is modular
> and well-commented. Then, generate a set of pytest tests that exercise each
> of the functions, embedded in a separate code block.

We first examined whether each generated script would execute without failure;
of the 100 generated scripts, 97 executed successfully. We then examined test
coverage using the Coverage.py tool. As shown in Figure 5, the majority of
files had test coverage of 100%, with 94% showing a coverage of at least 50%
ad a minimum coverage of 40%. There was a weak but statistically significant
negative relationship between the number of statements and the level of
coverage (Spearman r = -0.23, p = .02). A median of three tests were generated
for each file.

Figure 5: Percentage of test coverage as a function of the number of
statements in the automatically generated code.

Running the tests required minor automated modification to the code, since the
tests required importing the relevant functions but the names of the output
files were not known to the LLM. After fixing this issue, 45 of the 100 tests
completed successfully. The most common source of test failure was failure of
a value assertion (47/100); in these cases, it was not immediately possible to
determine whether the test or code was incorrect, without additional
debugging. In ten cases, the test failed because the code did not raise the
error expected by the test. In six cases, other errors were raised (Type,
Index, Value, and ZeroDivision errors).

In summary, our analyses of automatic test generation demonstrate that GPT-4
can successfully generate testing code with good coverage, but that these
tests fail often, requiring additional debugging to determine the root cause
of the test failure.

###  2.3 Conclusions

Our analyses demonstrate that GPT-4 has strong Python code generation
abilities, confirming the results of Bubeck et al. (2023). At the same time,
the prevalence of errors in the generated code suggests that humans must
remain in the loop in order to ensure the accuracy of any resulting code. Our
interactive prompting experiment showed that a relatively novice prompt
engineer can successfully solve coding problems within a small number of
prompts the majority of the time; however, a sizeable minority of problems
would have required significant human debugging in order to solve. An open
question is whether re-prompting in a new context may have led to more
successful outcomes in these cases.

Comparisons of Python code refactored using GPT-4 to the original code
demonstrated that GPT-4 improved the quality of the code, at least as measured
by common metrics of software quality and standards compliance. It should be
emphasized that these results do not assess the accuracy of the code; rather,
they suggest that GPT-4 can help programmers achieve code that is cleaner and
potentially more maintainable than the original. Given that GPT-4 refactoring
did not eliminate all standards compliance issues, the combination of GPT-4
with other code formatting tools (such as black) would likely result in even
further improvements.

The examination of test generation by GPT-4 demonstrated that it was able to
generate tests with a high degree of test coverage, but those tests failed a
majority of the time. Such test failures require additional human effort to
diagnose, since it is not immediately clear whether the failure is due to
inaccurate code, and inaccurate test, or both. These results suggest that
while GPT-4 is a very useful tool for generating testing frameworks for new
code, the specific test examples should be designed and implemented by a human
with domain expertise to ensure that the tests are accurately assessing the
intended behavior for the code.

There has been substantial speculation regarding the continued role of human
programmers in the face of AI coding tools. The present results suggests that
even with the latest generation of AI systems (i.e. GPT-4), human involvement
is essential to ensure validity and accuracy of the resulting code. This seems
to be especially the case when programming of mathematical concepts is
involved. The lack of confidence calibration of tools like GPT-4 means that
they will present answers in the same way regardless of the degree of support
for the answer.

The prompts used in the present research are almost certainly suboptimal, and
thus may be underestimating the potential performance of the model. For
example, recent work has shown that chain-of-thought prompting can
substantially improve the perfomance of LLMs on complex problems requiring
reasoning (Prystawski, Thibodeau, and Goodman 2022; Wei et al. 2023), and this
seems to extend to coding as
well111https://martinfowler.com/articles/2023-chatgpt-xu-hao.html. Further
work is needed to examine the degree to which such improved prompting
techniques might improve the performance of LLMs on complex coding problems,
and at this point our results should be taken as a lower bound on the
performance of these models.

###  2.4 Acknowledgments

Thanks to Mark Chen, David Coats, and Noah Goodman for helpful comments and
discussion during the development of this work.

## References

reBubeck, Sébastien, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric
Horvitz, Ece Kamar, Peter Lee, et al. 2023. “Sparks of Artificial General
Intelligence: Early Experiments with GPT-4.” arXiv.
https://doi.org/10.48550/arXiv.2303.12712.

preChen, Mark, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de
Oliveira Pinto, Jared Kaplan, Harri Edwards, et al. 2021. “Evaluating Large
Language Models Trained on Code.” arXiv.
https://doi.org/10.48550/arXiv.2107.03374.

preFinnie-Ansley, James, Paul Denny, Brett A. Becker, Andrew Luxton-Reilly,
and James Prather. 2022. “The Robots Are Coming: Exploring the Implications of
Openai Codex on Introductory Programming.” In _Australasian Computing
Education Conference_ , 10–19.

preFowler, Martin. 2019. _Refactoring: Improving the Design of Existing Code_.
Addison-Wesley.

preHalstead, Maurice Howard. 1977. _Elements of Software Science_. Elsevier.

preJi, Ziwei, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko
Ishii, et al. 2023. “Survey of Hallucination in Natural Language Generation.”
_ACM Computing Surveys_ 55 (12): 1–38. https://doi.org/10.1145/3571730.

prePrystawski, Ben, Paul Thibodeau, and Noah Goodman. 2022. “Psychologically-
Informed Chain-of-Thought Prompts for Metaphor Understanding in Large Language
Models.” arXiv. https://doi.org/10.48550/arXiv.2209.08141.

preWagenmakers, Eric-Jan, Han L. J. Van Der Maas, and Raoul P. P. P. Grasman.
2007. “An EZ-Diffusion Model for Response Time and Accuracy.” _Psychonomic
Bulletin & Review_ 14 (1): 3–22. https://doi.org/10.3758/BF03194023.

preWei, Jason, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei
Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. “Chain-of-Thought Prompting
Elicits Reasoning in Large Language Models.” arXiv.
https://doi.org/10.48550/arXiv.2201.11903.

p

##  3 Appendix 1: Prompts used for interactive coding with GPT-4

Note: The number in brackets denotes the number of prompts necessary to be
judged successful; NS denotes that the problem was abandoned before success.

  1. 1.

[4] Create python code that generates a new class called LinearRegressionStats
which extends sklearn.linear_model.LinearRegression() to compute the t
statistic and p-value for each regressor in the model.

  2. 2.

[1] Create python code to create an abstract base class called
AbstractPublication to represent publications, with a method called
from_pubmed(). Then create a class called Article that inherits
AbstractPublication. The from_pubmed() method should take in a pubmed record
downloaded using the biopython.Entrez tools, and should convert the author
list to a variable called authors and the title to a variable called title. In
the main function, perform a pubmed search for the query “cognitive control”
and convert each record into an instance of the Publication class, saving them
to a dictionary that uses the pubmed ID as the key.

  3. 3.

[1] Create python code to download 1000 abstracts from pubmed matching the
query “cognitive control”, clean up the text by removing standard English
stopwords and lexicalizing the words, and apply topic modeling using latent
Dirichlet allocation to find 10 topics. Print the highest scoring words for
each topic.

  4. 4.

[NS] Generate python code to simulate a drift diffusion model of response
times. Simulate 1000 trials from this model, and estimate the drift rate,
boundary separation, and starting point parameters using the EZ-diffusion
model.

  5. 5.

[1] Create a python application takes in a set of csv files, each of which
includes two columns: one called “rt” that contains a response time in
miiliseconds, and another “correct” that is binary (1/0) denoting whether the
response was correct or not. there should also be another variable, called
“compatible”, that contains binary values (0/1) with half of the rows set to
zero and half to 1. combine these files into a single data frame, using the
individual file names as an index variable. then perform a linear mixed model
that computes the effect of the compatible factor, with a random intercept and
slope for the different file names. also create a test that automatically
generates a set of data files with random values, and then tests the function
to assess whether it properly returns the true values.

  6. 6.

[7] Create python code to find a set of articles matching the query “cognitive
control” using pubmed. For each article extract the first and last author and
their affilations. Then find their institutional web site and scrape their
email address from the web site.

  7. 7.

[1] Create python code that uses the github api to download the 100 most
recently committed python files.

  8. 8.

[8] create python code to load each python file from a directory called
“python_files”, and for each file compute cyclomatic complexity,
maintainability index, and halsted complexity metrics for each file using the
radon package.

  9. 9.

[2] Generate a python script that performs crossvalidation with feature
selection incorrectly performed outside of the crossvalidation loop.

  10. 10.

[1] Please create code to generate some test data for this code (#9)

  11. 11.

[NS] Write a python function that takes as input a textfile, then create an
animation that: [1] Displays X lines of the textfile at a time, with Y
characters per line. If the text file has more than Y characters in a line,
then move to the n ext line.
[2] Scross down Z lines of the textfile every second, in a continuous way. [3]
Animate it in a new popup window. [4] Record the animation for R seconds, and
save it as a .avi file. (based on a talk by Sebastien Bubeck)

  12. 12.

[NS] Create python code to implement a three-layer transformer model, and
train that model using a small text corpus.

  13. 13.

[NS] Implement python code to simulate a logistic map, and create a plot of x
versus r.

  14. 14.

[NS] Create python code to implement the greedy equivalence search (GES)
algorithm, and code to generate test data that can be used to test the
implementation.

  15. 15.

[1] Generate a python class to represent a directed graph. generate a function
that takes in a directed graph and a two sets of vertices and and returns
whether those two sets are d-separated. Add a test that compares the results
from this code to the d-separation results computed using the NetworkX library

  16. 16.

[2] Create code in python to perform a simulation that repeatedly generates
samples of random variates from 6 different distributions (normal, uniform,
chi-squared, poisson, exponential, and beta) and computes the mean of each
sample, saving it for later use. Then, generate a figure that shows the
distribution of the means for each of the distributions.

  17. 17.

[NS] Create python code to simulate a drift diffusion model with collapsing
bounds.

  18. 18.

[4] Generate a python class that fits hurdle regression model, with a scikit-
learn type interface.

  19. 19.

[NS] Create a simulation in python that generates a version of the “corridor
of stability” plot by Felix Schonbrodt. The plot should show how correlation
estimates become less variable as the sample size increases.

  20. 20.

[1] Create a python class to perform spatial smoothing on a nifti image using
a median filter.

  21. 21.

[2] Please create a python class that implements the PC causal inference
algorithm.

  22. 22.

[2] Generate a python function that takes in a piece of English text and
performs linguistic analysis on it, returning values for sentiment analysis
(positive/negative) and for linguistic complexity.

  23. 23.

[3] Create python code to analyze human performance on the Daw two-step
reinforcement learning task. the code should take in a data frame with human
responses on each trial for each of the two steps along with the outcomes from
each step. it should return the indices of model-based and model-free
behavior.

  24. 24.

[3] Create a new crossvalidation class that implements balanced cross-
validation for regression that follows the scikit-learn class structure for
crossvalidation objects. this requires finding a set of splits that vary
minimally in their distributions, which can be assessed using an F statistic.

  25. 25.

[1] Create python code to simulate the effect of overfitting in regression. 1)
generate synthetic data with 32 observations for two variables from a
bivariate normal distribution with a correlation of 0.5. 2) fit three models
to the data: a linear regression model, a 2nd-order polynomial regression
model, and a 9-th order polynomial model. 3) Compute the error for each of
these models on the synthetic training data, and on a synthetic test dataset
generated from the same distribution. 4) plot the fitted lines for each of the
fitted models overlaid on the training data.

  26. 26.

[NS] Please create python code to 1) load data from
https://raw.githubusercontent.com/IanEisenberg/Self_Regulation_Ontology/master/Data/Complete_02-16-2019/meaningful_variables.csv
into a data frame, select all variables that begin with
‘upps_impulsivity_survey’, ‘sensation_seeking_survey’, ‘bis11_survey’, or
‘dickman_survey’, 3) perform exploratory factor analysis over these variables
using a range of 1 to 5 factors, 4) identify which model is preferred based on
minimum BIC, and 5) for each factor in the preferred solution, display which
variables were more strongly loaded on that factor versus other factors.
Please write this in a modular way and output one function at a time.

  27. 27.

[NS] Generate python code to obtain daily weather reports from palo alto,
california for 1960 to 2000. from these data, compute the maximum temperature
in each month, and plot the timeseries of maximum monthly temperatures over
that time period.

  28. 28.

[1] Generate python code to read in an image and render the image using ascii
art.

  29. 29.

[3] Create python code to take a document written in Markdown and render it as
a PDF, with an image for the header. please use argparse for the input values

  30. 30.

[1] Generate a class in the scikit-learn style to perform principal component
analysis using the power iteration method. the algorithm should be coded from
scratch rather than using an external library.

  31. 31.

[1] Create a python wrapper for the statsmodels ttest_ind function, which
takes the same arguments and returns a report similar to the one returned by
the t.test function in R

  32. 32.

[1] Create a simulation to demonstrate the use of randomization to obtain a
null distribution for hypothesis testing. First, generate a bivariate dataset
with a sample size of 100 and a correlation of 0.1 between the two variables.
Compute the correlation coefficient and obtain a p-value against the null
hypothesis of r=0. Then, randomly shuffle one of the variables 5000 times and
compute the correlation each time. Compute an empirical p-value for the actual
R value based on the null distribution.