# CP-generic expansions of models of Peano Arithmetic

Athar Abdul-Quader  School of Natural and Social Sciences, SUNY Purchase
College, 735 Anderson Hill Road, Purchase, NY 10577
<EMAIL_ADDRESS>and  James H. Schmerl  Department of
Mathematics, University of Connecticut, Storrs, CT 06269
<EMAIL_ADDRESS>
###### Abstract.

We study notions of genericity in models of $\mathsf{PA}$, inspired by lines
of inquiry initiated by Chatzidakis and Pillay and continued by Dolich, Miller
and Steinhorn in general model-theoretic contexts. These papers studied the
theories obtained by adding a “random” predicate to a class of structures.
Chatzidakis and Pillay axiomatized the theories obtained in this way. In this
article, we look at the subsets of models of $\mathsf{PA}$ which satisfy the
axiomatization given by Chatzidakis and Pillay; we refer to these subsets in
models of $\mathsf{PA}$ as CP-generics. We study a more natural property,
called strong CP-genericity, which implies CP-genericity. We use an arithmetic
version of Cohen forcing to construct (strong) CP-generics with various
properties, including ones in which every element of the model is definable in
the expansion, and, on the other extreme, ones in which the definable closure
relation is unchanged.

###### Key words and phrases:

models of arithmetic, expansions, genericity, definability

###### 2010 Mathematics Subject Classification:

Primary 03C62, 03H15

##  1\. Introduction

In [2], Chatzidakis and Pillay studied “generic” expansions of theories. Given
a first order theory $T$ in a language $\mathcal{L}$, and a unary predicate
$P\not\in\mathcal{L}$, Chatzidakis and Pillay axiomatized the theory $T_{P}$,
the model companion of $T$ in the language $\mathcal{L}\cup\\{P\\}$. This is,
roughly, the theory of the expansion of models of $T$ by a generic unary
predicate. Dolich, Miller and Steinhorn, in [3] and [4], continued this work
in studying the notion of adding “generic” predicates to o-minimal theories.

In the context of arithmetic, the methods used in the above papers do not work
as well. However, one can take a particular model
$\mathcal{M}\models\mathsf{PA}$ and consider the Chatzidakis-Pillay conditions
on so-called “generic” subsets of $M$. This line of inquiry was explored in
[1], and we investigate this further here.

###  1.1. Background

All models in this article are models of PA and their expansions. We use
$\mathcal{M}$, $\mathcal{N}$, $\mathcal{K}$, etc. for models of PA, and $M$,
$N$, $K$, etc. for their respective domains.

By convention, definability refers to definability with parameters. If a set
is definable without parameters, then we say that it is 0-definable. Given a
model $\mathcal{M}\models\mathsf{PA}$ and $a\in M$, $\operatorname{Scl}(a)$
denotes the Skolem closure of $a$; since $\mathsf{PA}$ has definable Skolem
terms, this coincides with $\operatorname{dcl}(a)$. For models of
$\mathsf{PA}$, we refer to Skolem closures rather than definable closures in
the rest of this article for this reason; for expansions of models of
$\mathsf{PA}$ which do not have definable Skolem terms, we continue to refer
to definable closures.

The following two definitions appeared in [1].

######  Definition 1.

A subset $X$ of $M$ is called CP-generic if whenever $D\subseteq M^{n}$ is
definable using only $a\in M$ as a parameter and $I\subseteq\\{1,\ldots,n\\}$,
if there are distinct $b_{1},\ldots b_{n}\in M$ such that each
$b_{i}\not\in\operatorname{Scl}(a)$ and $\langle b_{1},\ldots,b_{n}\rangle\in
D$, then there is $\langle b_{1},\ldots,b_{n}\rangle\in D$ such that $b_{i}\in
X$ iff $i\in I$.

######  Definition 2.

A subset $X$ of $M$ is _neutral_ if for all $a$ in $M$,
$\operatorname{Scl}(a)$ coincides with
$\operatorname{dcl}^{(\mathcal{M},X)}(a)$.

We define another notion of genericity in this paper.

######  Definition 3.

A subset $X$ of $M$ is strongly CP-generic if whenever $D\subseteq M^{n}$ is
definable and $I\subseteq\\{1,\ldots,n\\}$, if there is an infinite
$B\subseteq D$ such that for all $\langle b_{1},\ldots,b_{n}\rangle\neq\langle
c_{1},\ldots,c_{n}\rangle\in B$, $b_{i}\neq b_{j}$ for all $i\neq j$ and
$\\{b_{1},\ldots,b_{n}\\}\cap\\{c_{1},\ldots,c_{n}\\}=\emptyset$, then there
is $\langle b_{1},\ldots,b_{n}\rangle\in D$ such that $b_{i}\in X$ iff $i\in
I$.

Neutrality was explored in [1]. In this article we explore the relationship
between CP-genericity and neutrality, answering, in particular, Problem 1.3
from [1] which asked if CP-generics are necessarily neutral. The relationship
between CP-genericity and strong CP-genericity will be made clear in Section
2. Section 3 contains our main results. First, we show that CP-generics exist
for all countable models, and moreover, that every countable, recursively
saturated model of $\mathsf{PA}$ has a CP-generic which is not neutral,
answering Problem 1.3 in [1] negatively. In fact, we show a complete failure
of neutrality in some cases. We also show a construction of a neutral CP-
generic. In Section 4, we examine some properties of CP-generics in comparison
to properties exhibited by neutral sets. We end, in Section 5, with some open
questions.

##  2\. Strong CP-genericity

In [1], the notion of CP-genericity was introduced, but not explored. In this
article, we introduce _strong CP-genericity_. The following results show that
strong CP-genericity is equivalent to CP-genericity in the case of recursively
saturated models.

######  Proposition 4.

Let $\mathcal{M}\models\mathsf{PA}$ and $X\subseteq M$.

  1. (1)

If $X$ is strongly CP-generic, then $X$ is CP-generic.

  2. (2)

If $\mathcal{M}$ is recursively saturated and $X$ is CP-generic, then $X$ is
strongly CP-generic.

In general, the converse to (1) is not true. For example, if $\mathcal{M}$ is
prime, then every subset $X\subseteq M$ is CP-generic, although every strong
CP-generic is infinite.

###### Proof.

Suppose $X$ is strongly CP-generic. Let $D\subseteq M^{n}$ be definable using
parameter $a\in M$, and let $I\subseteq\\{1,\ldots,n\\}$. Assume there exist
distinct $b_{1},\ldots,b_{n}$ such that $b_{i}\not\in\operatorname{Scl}(a)$
for each $i$ and $\langle b_{1},\ldots,b_{n}\rangle\in D$. Then one can
construct a $B$ as in Definition 3: choose the smallest tuple
$\bar{b}_{0}=\langle b_{0,1},\ldots,b_{0,n}\rangle\in D$ where the $b_{0,i}$
are pairwise distinct, and inductively, if $\bar{b}_{0},\ldots,\bar{b}_{m-1}$
have been chosen, choose $\bar{b}_{m}=\langle b_{m,1},\ldots,b_{m,n}\rangle\in
D$ such that $b_{m,i}$ are all pairwise distinct and all differ from the
previously chosen $b_{j,k}$ ($j<m,1\leq k\leq n$). For each $m\in\omega$ and
$1\leq k\leq n$, $b_{m,k}\in\operatorname{Scl}(a)$, and so at each finite
stage there is a tuple distinct from those previously chosen. Therefore
$B=\\{\langle b_{m,1},\ldots,b_{m,n}\rangle:m\in\omega\\}\subseteq D$
satisfies the hypothesis for $B$ in Definition 3. By strong CP-genericity,
then, there is $\langle b_{1},\ldots,b_{n}\rangle\in D$ such that $b_{i}\in X$
if and only if $i\in I$.

Now assume $\mathcal{M}$ is recursively saturated and $X$ is CP-generic. Let
$D\subseteq M^{n}$ be definable using parameter $c\in M$, and suppose it
satisfies the hypothesis of Definition 3. Let $I\subseteq\\{1,\ldots,n\\}$.
Consider the type

$\displaystyle p(x_{1},\ldots,x_{n})=$ $\displaystyle\\{\langle
x_{1},\ldots,x_{n}\rangle\in D\\}\cup\\{x_{i}\neq x_{j}:1\leq i,j\leq n,i\neq
j\\}\cup$ $\displaystyle\\{t_{m}(c)\neq x_{i}:1\leq i\leq n,m\in\omega\\},$

where the $t_{m}$ range over all unary ${\mathcal{L}}_{\mathsf{PA}}$ Skolem
terms (in some recursive enumeration of such Skolem terms). By assumption,
there is an infinite $B\subseteq D$ such that for all distinct
$\bar{b},\bar{c}\in B$, the sets $\\{b_{1},\ldots,b_{n}\\}$ and
$\\{c_{1},\ldots,c_{n}\\}$ are disjoint. This implies that
$p(x_{1},\ldots,x_{n})$ is finitely satisfiable. If $\langle
x_{1},\ldots,x_{n}\rangle$ realizes $p$, then the $x_{i}$ are pairwise
distinct and $x_{i}\not\in\operatorname{Scl}(c)$ for each $i$, satisfying the
hypothesis for CP-genericity. Then since $X$ is CP-generic, there are $\langle
x_{1},\ldots,x_{n}\rangle\in D$ such that $x_{i}\in X$ iff $i\in I$. ∎

##  3\. Main Results

The goal of this section is to answer Problem 1.3 from [1]: if $X$ is CP-
generic, must $X$ necessarily be neutral? We see in Theorem 11 that the answer
is no; moreover, in Corollaries 12 and 13, that in fact, neutrality can fail
badly in (strong) CP-generics. Lastly, we show in Corollary 17 that every
countable model of $\mathsf{PA}$ has neutral strong CP-generics.

Many of the proofs in this section use _$\mathcal{M}$ -Cohen generics_, which
we define as follows.

######  Definition 5.

Let $\mathcal{M}\models\mathsf{PA}$. Consider the notion of forcing in
$(\mathcal{M},\omega)$ whose conditions are functions $p:A\to\\{0,1\\}$, where
$A\subseteq M$ is finite. Define $p\trianglelefteq q$ if $q$ extends $p$. Let
$G\subseteq M$ be generic for this notion of forcing. Then $X=\\{a\in
M:p(a)=0\text{ for some }p\in G\\}$ is an _$\mathcal{M}$ -Cohen generic_.

For a review of the terminology of arithmetic forcing, see [6, Chapter 6]. The
usual forcing and truth lemmas hold; see [6, Lemma 6.2.6]. That is: forcing is
definable in $(\mathcal{M},\omega)$, and for any formula
$\theta(\overline{x})$ in the expanded language and any $\overline{m}\in M$,
$(\mathcal{M},\omega,X)\models\theta(\overline{m})$ if and only if there is
$p\in G$ such that $(\mathcal{M},\omega)\models(p\Vdash\theta(\overline{m}))$.
In particular, $p\Vdash m\in X$ iff $m\in\operatorname{dom}(p)$ and $p(m)=0$.

The reader should be cautioned here that for nonstandard $\mathcal{M}$,
$\mathcal{M}$-Cohen generics are not the same as Cohen generics in the sense
of [6, Chapter 6]. That is, given a model $\mathcal{M}$, Cohen forcing (in the
sense of [6, Chapter 6]) is the notion of forcing whose conditions are
definable functions $p:[0,m)\to\\{0,1\\}$, for $m\in M$. If $G$ is generic for
this forcing, then the set $X=\\{a\in M:p(a)=0\text{ for some }p\in G\\}$ is
referred to as a Cohen generic.

######  Lemma 6.

Every $\mathcal{M}$-Cohen generic is strongly CP-generic.

###### Proof.

Let $p$ be a condition. Let $D\subseteq M^{n}$ be a definable set such that
there is an infinite $B\subseteq D$ as in Definition 3. We show that for each
$I\subseteq\\{1,\ldots n\\}$, there is $q_{I}\trianglerighteq p$ such that

(1) $q_{I}\Vdash\exists x_{1}\ldots\exists x_{n}[\langle
x_{1},\ldots,x_{n}\rangle\in D\wedge\bigwedge\limits_{i\leq n}(x_{i}\in
X\leftrightarrow i\in I)].$

Since $p:A\to\\{0,1\\}$ is finite, there are $x_{1},\ldots,x_{n}\not\in A$
such that $\mathcal{M}\models\langle x_{1},\ldots,x_{n}\rangle\in D$. Take
$A^{\prime}=A\cup\\{x_{i}:1\leq i\leq n\\}$ and $q_{I}=p\cup\\{\langle
x_{i},0\rangle:i\in I\\}\cup\\{\langle x_{i},1\rangle:i\not\in I\\}$.

If $X$ and $G$ are as in the definition of $\mathcal{M}$-Cohen genericity,
then for each $D\subseteq M^{n}$ definable satisfying the hypothesis in
Definition 3, and each $I\subseteq\\{1,\ldots,n\\}$, there is $q_{I}\in G$
satisfying (1). Therefore $X$ is strongly CP-generic. ∎

The following lemma can be proven using the standard proof that generics exist
for countable models.

######  Lemma 7.

If $\mathcal{M}$ is countable, then $\mathcal{M}$-Cohen generics exist.∎

In fact, if $\mathcal{M}$ is uncountable, there are no $\mathcal{M}$-Cohen
generics.

######  Corollary 8.

For any countable $\mathcal{M}$, there is $X\subseteq M$ that is strongly CP-
generic.∎

######  Lemma 9.

If $\mathcal{M}$ is nonstandard and $X,Y\subseteq M$ are such that
$X\setminus\omega=Y\setminus\omega$, then $X$ is strongly CP-generic iff $Y$
is.

###### Proof.

Suppose $X$ is strongly CP-generic. Let $D\subseteq M^{n}$ and
$I\subseteq\\{1,\ldots,n\\}$ be such that there is an infinite $B\subseteq D$
as in Definition 3. By overspill, there is $c>\omega$ such that

$D^{\prime}=\\{\langle x_{1},\ldots,x_{n}\rangle:\mathcal{M}\models\langle
x_{1},\ldots,x_{n}\rangle\in D\wedge\bigwedge\limits_{1\leq i\leq
n}x_{i}>c\\}$

also has such an infinite subset $B$. By strong CP-genericity applied to
$D^{\prime}$, there is $\langle b_{1},\ldots,b_{n}\rangle\in D^{\prime}$ such
that $b_{i}\in X$ iff $i\in I$. Since each $b_{i}>\omega$, it follows that
$b_{i}\in Y$ iff $i\in I$. ∎

Note that in the standard model $\mathbb{N}$, the $\mathbb{N}$-Cohen generics
are exactly the Cohen generics in the sense of [6, Chapter 6]. By Lemma 6,
every Cohen generic in the standard model is strongly CP-generic. The converse
is false: there are strong CP-generics which are not Cohen generic. Moreover,
for every countable $\mathcal{M}$, there are strong CP-generics which are not
$\mathcal{M}$-Cohen generic.

######  Proposition 10.

For every countable $\mathcal{M}$, there is a strong CP-generic
$X\subseteq\omega$ which is not $\mathcal{M}$-Cohen generic.

###### Proof.

If $\mathcal{M}$ is standard and $X$ is Cohen generic, then there is
$n\in\omega$ such that $[n,2n]\subseteq X$. One confirms this by noticing that
the set of conditions which force this is dense. However, one can routinely
construct a strong CP-generic which avoids $[n,2n]$ as a subset for each $n$.

If $\mathcal{M}$ is nonstandard, then by Lemma 9, if $X$ is strongly CP-
generic, then $X\setminus\omega$ is also strongly CP-generic. However, every
$\mathcal{M}$-Cohen generic must nontrivially intersect $\omega$. ∎

Next we answer Problem 1.3 from [1] in the negative.

######  Theorem 11.

Let $\mathcal{M}$ be any countable, non-prime model. Then there is $X\subseteq
M$ such that $X$ is (strongly) CP-generic but not neutral.

###### Proof.

We proceed by first constructing an $\mathcal{M}$-Cohen generic with the
property that for all $a\neq b$, there is $n\in\omega$ such that $a+n\in X$
iff $b+n\not\in X$. Then, by Lemma 9, given any
$a\not\in\operatorname{Scl}(0)$, the set
$X^{\prime}=(X\setminus\omega)\cup\\{2n+1:n\in\omega,a+n\in X\\}$ is also
strongly CP-generic, and we will see that $a$ is definable in
$(\mathcal{M},X^{\prime})$.

To construct the $\mathcal{M}$-Cohen generic with the requisite property,
enumerate the model as $(a_{i}:i\in\omega)$ and the dense definable sets in
the $\mathcal{M}$-Cohen forcing in $(\mathcal{M},\omega)$ as
$(A_{i}:i\in\omega)$. At stage $2i$, meet $A_{i}$ with condition $p_{2i}$
(extending $p_{2i-1}$ if $i>0$). At stage $2i+1$, let $n\in\omega$ be the
least such that $a_{j}+n\not\in\operatorname{dom}(p_{2i})$ for each $j<i$, and
extend $p_{2i}$ to $p_{2i+1}$ such that $p_{2i+1}(a_{i}+n)=0$ and
$p_{2i+1}(a_{j}+n)=1$ for each $j<i$.

Let $X$ be the resulting $\mathcal{M}$-Cohen generic. Then for all $a\neq b$
nonstandard, there is $n\in\omega$ such that $a+n\in X\iff b+n\not\in X$.

To complete the proof, let $a\not\in\operatorname{Scl}(0)$ and let
$X^{\prime}=(X\setminus\omega)\cup\\{2n+1:a+n\in X\\}$. By Lemma 9,
$X^{\prime}$ is strongly CP-generic. Moreover, notice that $\omega$ is
$0$-definable in $(\mathcal{M},X)$ as

(2) $\\{n:(\mathcal{M},X)\models\forall i<n(2i\not\in X)\\}.$

To see this, let $J$ be the set of those $n$ satisfying (2). By definition,
$\omega\subseteq J$. If $c>\omega$, consider the set $D=\\{\langle
x_{0},x_{1}\rangle:\mathcal{M}\models x_{0}<x_{1}<c\wedge x_{1}=2\cdot
x_{0}\\}$. Since $c$ is nonstandard, $D$ contains an infinite set $B$ such
that if $\langle b_{0},b_{1}\rangle\neq\langle c_{0},c_{1}\rangle\in B$, then
$b_{0},b_{1},c_{0},c_{1}$ are all distinct. By strong CP-genericity, there is
$\langle x_{0},x_{1}\rangle\in D$ such that $x_{1}\in X$. Then, since $x_{1}$
is even, $c\not\in J$.

Lastly, $a$ is definable in $(\mathcal{M},X^{\prime})$ as $x=a$ iff

$(\mathcal{M},X^{\prime})\models\forall n\in\omega(x+n\in X\leftrightarrow
2n+1\in X).\qed$

We can further modify the above idea to show that there is a strong CP-generic
$X\subseteq M$ such that every element of $M$ is definable in
$(\mathcal{M},X)$. Instead of using the evens and odds, as we did above, take
a partition of $\omega$ into countably many uniformly definable disjoint
infinite sets $(I_{j}:j\in\omega)$. For example, let $I_{j}$ be the powers of
the $j$-th prime. Then ensure $I_{0}\cap X=\emptyset$, and for each $a_{j}\in
M$, put the $n$-th element of $I_{j+1}$ in $X$ if and only if $a_{j}+n\in X$.
In this way, we obtain the following corollary:

######  Corollary 12.

Let $\mathcal{M}$ be countable. Then there is a strong CP-generic $X\subseteq
M$ such that every element of $M$ is definable in $(\mathcal{M},X)$.∎

Note that if $\mathcal{M}$ is prime, every element of $M$ is already
definable, regardless of what $X\subseteq M$ is taken.

By another similar modification to the proof of Theorem 11, we find that for
every countable $\mathcal{M}$ and $A\subseteq M$, there is a strong CP-generic
$X\subseteq M$ such that $A\in\operatorname{Def}(\mathcal{M},X)$. Moreover,
there is a kind of uniformity in defining these functions.

######  Corollary 13.

There is a formula $\phi(x)\in{\mathcal{L}}_{\mathsf{PA}}\cup\\{X\\}$ such
that for any countable $\mathcal{M}$ and any subset $A\subseteq M$, there is a
strong CP-generic $X$ such that $\phi$ defines $A$ in $(\mathcal{M},X)$.

###### Proof.

Let $\mathcal{M}$ and $A$ be given. The formula $\phi(x)$ (and the uniformity
of it) will be clear as part of the construction of $X$.

In the standard model, one can routinely construct a Cohen generic $X$ such
that $A\in\operatorname{Def}(\mathbb{N},X)$, similar to the proof of [6,
Theorem 6.2.11]. Using Lemma 6, $X$ is also strongly CP-generic.

If $\mathcal{M}$ is nonstandard, we modify the construction in Theorem 11.
Start by constructing a strong CP-generic $X$ with the property that for all
$a\neq b$, there is $n\in\omega$ such that $a+n\in X$ iff $b+n\not\in X$. Note
that the following construction can be done for any nonstandard $\mathcal{M}$,
while Theorem 11 only applies to non-prime models.

Fix an enumeration of $A$ in order type $\omega$ as $(a_{n}:n\in\omega)$.
Partition $\omega$ into definable, disjoint, infinite sets
$I_{0},I_{1},I_{2},\ldots$. Define $X^{\prime}$ as follows: for $c>\omega$,
$c\in X$ iff $c\in X^{\prime}$, so that $X^{\prime}$ is strongly CP-generic by
Lemma 9. For $n\in I_{0}$, ensure $n\not\in X^{\prime}$ so that $\omega$ is
definable (as above). For $n\in A\cap\omega$, put the $n$-th element of
$I_{1}$ in $X^{\prime}$. Then for $x\in A\setminus\omega$, if $x=a_{n}$, put
the $m$-th element of $I_{n+2}$ in $X^{\prime}$ if and only if $x+m\in
X^{\prime}$.

Now $A$ is definable in $(\mathcal{M},X^{\prime})$ as $x\in A$ if $x\in\omega$
and the $x$-th element of $I_{1}$ is in $X^{\prime}$, or $x\not\in\omega$ and
there is $n\in\omega$ such that for all $m\in\omega$, $x+m\in X^{\prime}$ if
and only if the $m$-th element of $I_{n+2}$ is in $X^{\prime}$.

One checks that the definition of $A$ can be made uniform by noticing that
there is a statement true in $(\mathcal{M},X)$ (for all countable$\mathcal{M}$
and strong CP-generics constructed above) iff $\mathcal{M}$ is standard. ∎

######  Corollary 14.

Every countable $\mathcal{M}$ has $2^{\aleph_{0}}$ distinct strongly CP-
generic subsets.∎

Conversely, there are neutral sets which are CP-generic. We again use
$\mathcal{M}$-Cohen generics to establish this. As seen in Lemma 6,
$\mathcal{M}$-Cohen generics are strongly CP-generic. Here we see that they
are also neutral.

######  Theorem 15.

For any countable $\mathcal{M}$, every $\mathcal{M}$-Cohen generic is neutral.

###### Proof.

Let $X$ be an $\mathcal{M}$-Cohen generic and let $G$ be a corresponding
generic set of conditions, so that $X=\\{m:p(m)=0\text{ for some }p\in G\\}$.
By Lemma 6, it is strongly CP-generic.

We show that
$\operatorname{dcl}^{(\mathcal{M},\omega,X)}=\operatorname{dcl}^{(\mathcal{M},\omega)}$.
By Kanovei [5], generalized in [6, Theorem 8.4.7], the $\operatorname{dcl}$
relation in $(\mathcal{M},\omega)$ is identical to that in $\mathcal{M}$,
which shows that $X$ is neutral. Before we show this, we first show a lemma
about compatibility of conditions, which will be critical for various stages
of our proof.

######  Lemma 16.

Let $\\{X_{i}:i\in\omega\\}$ be a family of infinite sets of conditions. If
for all $i,j\in\omega$ and $p\in X_{i},q\in X_{j}$, $|p|=|q|$, then there are
$i\neq j\in\omega$, with $p\in X_{i}$, $q\in X_{j}$, and $p\neq q$ such that
$p$ and $q$ are compatible.

Note that the $X_{i}$ need not be pairwise distinct.

###### Proof.

Let $n$ be the cardinality of the domain of any condition in (any of the)
$X_{i}$. We prove this by induction on $n$.

If $n=0$, there are no such $X_{i}$, since there is only one condition of
whose domain is empty (the empty condition). If $n=1$, the Lemma holds by
pigeonholing.

Inductively suppose the Lemma holds for all collections
$\\{Y_{j}:j\in\omega\\}$ such that $|p|<n$ for each $p$ in (any of the)
$Y_{j}$. Fix $p\in X_{0}$. If $p$ is not compatible with any $q\neq p$ for all
$q\in\bigcup\limits_{i\geq 1}X_{i}$, then there is $a\in\operatorname{dom}(p)$
such that there are infinitely many $j\in\omega$, and infinitely many $q\in
X_{j}$ with $a\in\operatorname{dom}(q)$ but $p(a)\neq q(a)$. Without loss of
generality, assume $p(a)=0$, so for all such $q$, $q(a)=1$.

For such a $q$, define $q^{*}$ as $q\setminus\\{\langle a,1\rangle\\}$. Let
$Y_{j}$ be the $j$-th set in the collection $\\{X_{i}:i\in\omega\\}$ such that
there are infinitely many $q$ with $q(a)=1$. Let $Y_{j}^{*}=\\{q^{*}:q\in
Y_{j},q(a)=1\\}$. Then $\\{Y_{j}^{*}:j\in\omega\\}$ satisfies the inductive
hypothesis, and so there are $j_{0}$ and $j_{1}$, and $q_{0}^{*}\in
Y_{j_{0}}^{*}$, $q_{1}\in Y^{*}_{j_{1}}$ such that $q_{0}^{*}$ and $q_{1}^{*}$
are compatible. Then $q_{0}=q_{0}^{*}\cup\langle a,1\rangle$ and
$q_{1}=q_{1}^{*}\cup\langle a,1\rangle$ are also compatible. ∎

Now we return to the proof that
$\operatorname{dcl}^{(\mathcal{M},\omega,X)}=\operatorname{dcl}^{(\mathcal{M},\omega)}$.
Suppose $a,b\in M$ are such that $(\mathcal{M},\omega,X)\models\forall
x(\phi(x,b)\leftrightarrow x=a)$ for some $\phi$ in the expanded language.
There is $p\in G$ such that

(3) $(\mathcal{M},\omega)\models p\Vdash[\forall x(\phi(x,b)\leftrightarrow
x=a)].$

Let $p$ be such that it satisfies (3) and $|p|$ is minimal. Let
$Y=\\{q:|q|=|p|\text{ and }q\Vdash[\forall x(\phi(x,b)\leftrightarrow
x=a)]\\}$. We consider the two cases of whether $Y$ is finite or $Y$ is
infinite.

If $Y$ is finite, then $p\in\operatorname{dcl}^{(\mathcal{M},\omega)}(a,b)$.
Since $\omega$ is neutral, then $p\in\operatorname{Scl}(a,b)$, and so there
are $n\in\omega,$ Skolem terms $t_{0},\ldots,t_{n-1}$, and
$\sigma:[0,n-1]\to\\{0,1\\}$ such that $p(t_{i}(a,b))=\sigma(i)$ for each
$i<n$. Let $p(x)$ be the finite function defined by
$t_{i}(x,b)\mapsto\sigma(i)$ for $0\leq i<n$, so that $p=p(a)$. Now consider
the set $B=\\{c:(\mathcal{M},\omega)\models p(c)\Vdash\forall
x[(\phi(x,b)\leftrightarrow x=c)]\\}$. Clearly $a\in B$, and so if $B$ is
finite then $a\in\operatorname{dcl}^{(\mathcal{M},\omega)}(b)$. If $B$ is
infinite, let $Z=\\{p(c):c\in B\\}$ and apply Lemma 16 to the collection
$\\{X_{i}:i\in\omega\\}$ where each $X_{i}=Z$. We obtain compatible conditions
$p(c_{1})\neq p(c_{2})\in Z$. But in $(\mathcal{M},\omega)$,

$p(c_{1})\cup p(c_{2})\Vdash\forall x[\phi(x,b)\leftrightarrow
x=c_{1}]\wedge\forall x[\phi(x,b)\leftrightarrow x=c_{2}],$

which is impossible.

If $Y$ is infinite, for each $c\in M$ let $X_{c}=\\{q:|q|=|p|\text{ and
}q\Vdash[\forall x(\phi(x,b)\leftrightarrow x=c)]\\}$. If there are only
finitely many $c$ such that $X_{c}$ is infinite, then
$a\in\operatorname{dcl}^{(\mathcal{M},\omega)}(b)$, so assume that there are
infinitely many such $c$. Applying Lemma 16, there are $c_{1}\neq c_{2}$ with
$p\in X_{c_{1}},q\in X_{c_{2}}$ and $p$ and $q$ are compatible. But then
$p\cup q\Vdash[\forall x(\phi(x,b)\leftrightarrow x=c_{1})]\wedge[\forall
x(\phi(x,b)\leftrightarrow x=c_{2})]$, which is impossible. ∎

Combining Lemma 6, Lemma 7, and Theorem 15, we obtain the following:

######  Corollary 17.

Every countable $\mathcal{M}$ has a neutral, strong CP-generic.∎

##  4\. Cuts and Classes

In every model $\mathcal{M}$, the standard cut $\omega$ is neutral. This is
not true in general for CP-generics, per the following result. In the
following, an extension $\mathcal{N}\prec\mathcal{M}$ is called _superminimal_
(see [6, Section 2.1.2]) if whenever $b\in M\setminus N$, then
$\operatorname{Scl}(b)=M$.

######  Proposition 18.

For any $\mathcal{M}$, the following are equivalent:

  1. (1)

$\mathcal{M}$ has a bounded CP-generic subset $X\subseteq M$.

  2. (2)

All sufficiently large $b\in M$ generate $\mathcal{M}$.

  3. (3)

$\mathcal{M}$ is prime or is a superminimal elementary end extension of some
$\mathcal{N}\prec\mathcal{M}$.

One notes here that conditions (2) and (3) both imply that $\mathcal{M}$ is
countable.

###### Proof.

The implication $\eqref{supermin-end}\implies\eqref{suff-large-gen}$ is clear
from definitions. To show $\eqref{suff-large-gen}\implies\eqref{supermin-
end}$, suppose (2) holds. Let $b\in M$ be such that
$\operatorname{Scl}(b)=\mathcal{M}$. Let $K$ be the set of those $a\in M$ such
that $b\not\in\operatorname{Scl}(a)$. Then by (2), either $K=\emptyset$ or $K$
is a proper cut of $\mathcal{M}$. If $K=\emptyset$, then
$\mathcal{M}=\operatorname{Scl}(0)$. If $K\neq\emptyset$, then we claim that
$\mathcal{K}$ is an elementary submodel of $\mathcal{M}$, and $\mathcal{M}$ is
a superminimal elementary end extension of $\mathcal{K}$. To see this, suppose
$\mathcal{M}\models\exists x(\phi(x,a))$, for some $a\in K$. Then there is
$c\in\operatorname{Scl}(a)$ such that $\mathcal{M}\models\phi(c,a)$. Then
since $\operatorname{Scl}(c)\subseteq\operatorname{Scl}(a)$ and
$b\not\in\operatorname{Scl}(a)$, it follows that
$b\not\in\operatorname{Scl}(c)$ and so $c\in K$. Moreover, for any $a\in
M\setminus K$, $\operatorname{Scl}(a)=\mathcal{M}$, and so $\mathcal{M}$ is a
superminimal extension of $\mathcal{K}$.

Next we show $\eqref{bounded-cpg}\implies\eqref{suff-large-gen}$. Suppose $X$
is bounded and is CP-generic. Let $b>X$. If
$\operatorname{Scl}(b)\neq\mathcal{M}$, then there is
$x\not\in\operatorname{Scl}(b)$ such that $x>b$. Then by CP-genericity,

$(\mathcal{M},X)\models\exists x(x>b\wedge x\in X),$

immediately contradicting the assertion that $X$ is bounded above by $b$.

Finally we show $\eqref{supermin-end}\implies\eqref{bounded-cpg}$. If
$\mathcal{M}$ is prime, then every $X\subseteq M$ is CP-generic by definition.
If $\mathcal{N}\prec\mathcal{M}$, and $\mathcal{M}$ is a superminimal
elementary end extension of $\mathcal{N}$, then we build $X\subseteq N$ which
is CP-generic in $\mathcal{M}$. To build $X$, we construct finite sets
$A_{i},B_{i}$, for each $i\in\omega$, such that the following hold:

  1. (a)

if $i<j\in\omega$, then $A_{i}\subseteq A_{j}$,

  2. (b)

for all $i\in\omega$, $A_{i}\cap B_{i}=\emptyset$,

  3. (c)

for all $i\in\omega$, $A_{i}\cup B_{i}\subseteq N$, and

  4. (d)

for every definable set $D\subseteq M^{n}$, $I\subseteq\\{1,\ldots,n\\}$, if
$D$ satisfies the hypothesis of Definition 1, then there is $i<\omega$ and
$\langle x_{1},\ldots,x_{n}\rangle\in D\cap(A_{i}\cup B_{i})$ such that for
$1\leq j\leq n$, $x_{j}\in A_{i}$ iff $j\in I$ and $x_{j}\in B_{i}$ iff
$j\not\in I$.

To begin the construction, let $A_{0}=B_{0}=\emptyset$. At stage $i+1$,
suppose $A_{i}$ and $B_{i}$ are defined and that we are considering a
definable $D\subseteq M^{n}$ and $I\subseteq\\{1,\ldots,n\\}$ satisfying the
hypothesis of Definition 1. By superminimality, if $D$ is definable from $a\in
M\setminus N$, there is nothing to show, so assume $D$ is definable from
parameter $a\in N$, and there are
$b_{1},\ldots,b_{n}\not\in\operatorname{Scl}(a)$ such that $\langle
b_{1},\ldots,b_{n}\rangle\in D$. Since $A_{i}$ and $B_{i}$ are finite, then
let $\langle x_{1},\ldots,x_{n}\rangle\in D$ be the smallest such that
$x_{j}\not\in A_{i}\cup B_{i}$ for any $1\leq j\leq n$. Then notice that each
$x_{j}\in\mathcal{N}$, as the tuple $\langle x_{1},\ldots,x_{n}\rangle$ is
definable from elements of $\mathcal{N}$. Let $A_{i+1}=A_{i}\cup\\{x_{j}:j\in
I\\}$ and $B_{i+1}=B_{i}\cup\\{x_{j}:j\not\in I\\}$.

As there are only countably many pairs $(D,I)$ where $D$ is definable,
$D\subseteq M^{n}$ for some $n\in\omega$ and $I\subseteq\\{1,\ldots,n\\}$,
each such $D$ and $I$ is handled at some stage $i\in\omega$, so (d) holds. Let
$X=\bigcup\limits_{i\in\omega}A_{i}$. Properties (a), (b), and (d) imply that
$X$ is CP-generic, and property (c) implies that $X\subseteq N$ (and therefore
is bounded). ∎

This shows that, in recursively saturated models (or any model which is not
finitely generated), no proper cut is CP-generic . Of course, the word
“proper” can be omitted from the previous sentence, as one can verify that for
any model $\mathcal{M}$, $M$ itself is not CP-generic.

[1] focused on neutral _classes_ and neutral inductive sets. A subset $X$ of a
model ${\mathcal{M}}$ is a _class_ if for each $a\in M$, $\\{x\in
X:{\mathcal{M}}\models x<a\\}$ is definable in ${\mathcal{M}}$. A subset $X$
of $M$ is _inductive_ if $({\mathcal{M}},X)\models{\sf PA}^{*}$, i.e. the
induction schema holds in $({\mathcal{M}},X)$ for all formulas of the language
of PA with a unary predicate symbol interpreted as $X$. All inductive sets are
classes. [1, Corollary 3.3] states that no undefinable neutral set in a
recursively saturated model is a class. It turns out that, in recursively
saturated models, no CP-generic is a class either. In fact, we have more: no
strong CP-generic is a class in a nonstandard model.

######  Theorem 19.

Let $\mathcal{M}$ be nonstandard. If $X\subseteq M$ is a class, then it is not
strongly CP-generic.

###### Proof.

Let $X$ be strongly CP-generic and $b>\omega$. Suppose $B=[0,b)\cap X$ is
infinite. If not, replace $X$ with its complement, which is also strongly CP-
generic by definition. If $X$ is a class, then $B$ is an infinite definable
subset of $M$, and so by strong CP-genericity, there is $x\in B$ such that
$x\not\in X$. ∎

##  5\. Open Questions

We close with some questions about CP-genericity. For the first question, we
recall the notion of the substructure lattice of a model. Given a structure
$\mathcal{M}$,
$\operatorname{Lt}(\mathcal{M})=\\{\mathcal{K}:\mathcal{K}\prec\mathcal{M}\\}$;
see [6, Chapter 4] for basic definitions and results on substructure lattices.

######  Question 1.

Let $\mathcal{M}$ be a countable, recursively saturated model of
$\mathsf{PA}$. For which subsemilattices $L$ of
$\operatorname{Lt}(\mathcal{M})$ is it the case that there is a (strong) CP-
generic $X$ such that the elementary substructures of $(\mathcal{M},X)$ are
exactly expansions of the $\mathcal{N}\prec\mathcal{M}$ such that
$\mathcal{N}\in L$?

Theorem 19 asserts that strong CP-generic subsets of nonstandard models are
not classes. In particular, this means that no CP-generic subset of a
recursively saturated model is a class. In a similar vein to the ideas studied
about neutrality in [1], we ask here if sets which are CP-generic (but not
strongly CP-generic) can ever be classes in a nonstandard model.

######  Question 2.

For which $\mathcal{M}$ is there $X\subseteq M$ such that $X$ is a CP-generic
class?

######  Question 3.

Let $\mathcal{M}\models\mathsf{PA}$ and $X\subseteq M$ strongly CP-generic. Is
there always a proper $(\mathcal{N},Y)\succ(\mathcal{M},X)$ such that $Y$ is
strongly CP-generic? Under what conditions on $(\mathcal{M},X)$ can we always
find such an $(\mathcal{N},Y)$?

## References

  * [1] Athar Abdul-Quader and Roman Kossak.  Neutrally expandable models of arithmetic.  Mathematical Logic Quarterly, 65(2):212–217, 2019.
  * [2] Zoé Chatzidakis and Anand Pillay.  Generic structures and simple theories.  Annals of Pure and Applied Logic, 95(1-3):71–92, 1998.
  * [3] Alfred Dolich, Chris Miller, and Charles Steinhorn.  Extensions of ordered theories by generic predicates.  The Journal of Symbolic Logic, 78(2):369–387, 2013.
  * [4] Alfred Dolich, Chris Miller, and Charles Steinhorn.  Expansions of o-minimal structures by dense independent sets.  Annals of pure and applied logic, 167(8):684–706, 2016.
  * [5] V. Kanovei.  Uniqueness, collection, and external collapse of cardinals in ${\rm IST}$ and models of Peano arithmetic.  Journal of Symbolic Logic, 60(1):318–324, 1995.
  * [6] Roman Kossak and James H. Schmerl.  The structure of models of Peano arithmetic, volume 50 of Oxford Logic Guides.  Oxford University Press, 2006.


Bose-Einstein condensate in an elliptical waveguide

Luca Salasnich

Dipartimento di Fisica e Astronomia “G. Galilei” and QTech, Università di
Padova,

via F. Marzolo 8, I-35131, Padova, Italy

Istituto Nazionale di Fisica Nucleare, Sezione di Padova,

Via Marzolo 8, 35131 Padova, Italy

Istituto Nazionale di Ottica del Consiglio Nazionale delle Ricerche, Unita di
Sesto Fiorentino,

Via Nello Carrara 2, 50019 Sesto Fiorentino, Italy

## Abstract

We investigate the effects of spatial curvature for an atomic Bose-Einstein
condensate confined in an elliptical waveguide. The system is well described
by an effective 1D Gross-Pitaevskii equation with a quantum-curvature
potential, which has the shape of a double-well but crucially depends on the
eccentricity of the ellipse. The ground state of the system displays a quantum
phase transition from a two-peak configuration to a one-peak configuration at
a critical attractive interaction strength. In correspondence of this phase
transition the superfluid fraction strongly reduces and goes to zero for a
sufficiently attractive Bose-Bose interaction.

###### Contents

  1. 1 Introduction
  2. 2 Quantum-curvature potential
  3. 3 Properties of the elliptical waveguide
  4. 4 BEC ground state in elliptical waveguide
  5. 5 Quantum phase transition
  6. 6 Superfluid fraction
  7. 7 Conclusions

##  1 Introduction

How does a locally-varying spatial curvature influence the properties of low-
dimensional quantum systems? This is a relevant question asked by scientists
working in very different fields such as the linear Schrödinger equation for a
particle constrained on a curve manifold [2, 3, 4], but also quantum gravity
[5] or quantum chaos [6]. It is well know [7, 8, 9] that the local curvature
of a curve on the three-dimensional (3D) Euclidean space is characterized by
the so-called geodesic curvature. This geodesic curvature $\kappa$ is an
extrinsic quantity: it does not remain invariant if the curve is under the
effect of a distance-preserving transformation [7, 8, 9]. Instead, the local
curvature of a surface on the 3D Euclidean space is characterized by the so-
called Riemann curvature tensor, which can be written in terms of the
invariant Gaussian curvature and the not-invariant average curvature [7, 8,
9]. The quantum motion of a particle on a curved waveguide has been anayzed by
several authors [10, 11, 12, 13, 14, 15]. More recently, the highly nontrivial
role of curvature for constrained quantum systems has been theoretically
investigated with ultracold atomic gases confined in a quasi-1D [16, 17, 18,
19] and quasi-2D configurations [20]. The main result of all these
investigations is that the local curvature gives rise to a quantum-curvature
potential.

In this paper we consider an atomic Bose-Einstein condensate (BEC) confined in
a quasi-1D elliptical waveguide finding that the quantum-curvature potential
has the shape of a double-well, if the eccentricity of the ellipse is
different from zero. By numerically solving the 1D Gross-Pitaevskii equation
of the BEC wavefunction under the effect of this quantum-curvature potential,
we show that the ground state of the system is uniform along the waveguide
only if the eccentricity $\epsilon$ of the ellipse is zero (circular waveguide
with constant curvature). Instead, for $\epsilon\neq 0$ we find that the
ground state is generically characterized by a two-peak configuration, where
the peaks are located around the minima of the effective double-well
potential. However, we discover that in the case of attractive interaction it
exists a critical (negative) interaction strength below which the ground state
exhibits a quantum phase transition from the two-peak configuration to a one-
peak configuration. This is the analog of the spontaneous symmetry breaking,
i.e. the modulational instability [21], of the uniform configuration predicted
some years ago for an 1D attractive BEC in a circular waveguide [22, 23, 24].
Our results show that the critical interaction strength depends on the
eccentricity $\epsilon$ of the ellipse in a non-trivial way. We also analyze
the effect of a boost velocity on the BEC moving in the elliptical waveguide
deriving the Leggett formula [25] for the superfluid fraction of a 1D bosonic
system [26, 27, 28]. Our numerical investigation reveals that the superfluid
fraction decreases dramatically in response to this quantum phase transition,
eventually reaching zero for a sufficiently negative Bose-Bose interaction.

##  2 Quantum-curvature potential

We consider a Bose-Einstein condensate (BEC) made of $N$ identical bosonic
atoms of mass $m$. The atoms are constrained to move along a curve ${\cal C}$
by the presence of a strong harmonic potential of frequency $\omega_{\bot}$ in
the local transverse plane with respect to ${\cal C}$. The characteristic
length of the transverse confinement is
$l_{\bot}=\sqrt{\hbar/(m\omega_{\bot})}$ where $\hbar$ is the reduced Planck
constant. We introduce a local system $(s,u,v)$ of coordinates, where $s$ is
the curvilinar abscissa (arclength) along ${\cal C}$ while $u$ and $v$ are two
coordinates of the transverse plane [16, 17, 18, 19]. In this way the
Lagrangian density of our problem is given by

$\mathscr{L}={i\hbar\over
2}\left(\Psi^{*}\partial_{t}\Psi-\Psi\partial_{t}\Psi^{*}\right)-{\hbar^{2}\over
2m}|\nabla\Psi|^{2}-{m\omega_{\bot}^{2}\over 2}(u^{2}+v^{2})|\Psi|^{2}-{1\over
2}g|\Psi|^{4}\;,$ (1)

where $\Psi(s,u,v,t)$ is the BEC wavefunction normalized to one and
$g=4\pi\hbar^{2}a_{s}(N-1)/m$ is the 3D strength of the contact inter-atomic
potential with $a_{s}$ the s-wave scattering length. Clearly, the Laplacian
operator $\nabla^{2}$ must be written in terms of the local system $(s,u,v)$
of coordinates [16, 17, 18, 19]. Assuming the factorization

$\Psi(s,u,v,t)=\psi(s,t)\;{e^{-{(u^{2}+v^{2})\over
2\sigma(s,t)^{2}}}\over\pi^{1/2}\sigma(s,t)}$ (2)

and inserting this ansatz into the Lagrangian density, after integration over
$u$ and $v$ one gets [19, 29, 30],

$\displaystyle\bar{\mathscr{L}}$ $\displaystyle=$ $\displaystyle{i\hbar\over
2}\left(\psi^{*}\partial_{t}\psi-\psi\partial_{t}\psi^{*}\right)-{\hbar^{2}\over
2m}|\partial_{s}\psi|^{2}+{\hbar^{2}\kappa^{2}(s)\over 8m}|\psi|^{2}$ (3)
$\displaystyle-$ $\displaystyle\left({\hbar^{2}\over
2m}{1\over\sigma^{2}}+{m\omega_{\bot}^{2}\over
2}\sigma^{2}\right)|\psi|^{2}-{1\over 2}{g\over 2\pi\sigma^{2}}|\psi|^{4}\;,$

where $\kappa(s)$ is the local geodesic curvature of ${\cal C}$, and the
conditions $\sigma\kappa\ll 1$ and $\sigma\ll\xi$ must hold, with
$\xi=\hbar/\sqrt{2g|\Psi|^{2}}$ the 3D healing length [19]. The Euler-Lagrange
equations of the 1D action functional with respect to the 1D wavefunction
$\psi(s,t)$ and the transverse width $\sigma(s,t)$ are

$i\hbar\partial_{t}\psi=\Big{[}-{\hbar^{2}\over
2m}\partial_{s}^{2}-{\hbar^{2}\kappa^{2}(s)\over 8m}+{\hbar^{2}\over
2m}{1\over\sigma^{2}}+{m\omega_{\bot}^{2}\over
2}\sigma^{2}+{2\hbar^{2}a_{s}(N-1)\over m\sigma^{2}}|\psi|^{2}\Big{]}\psi\;,$
(4)

and

$\sigma^{2}=l_{\bot}^{2}\sqrt{1+2a_{s}(N-1)|\psi|^{2}}$ (5)

Eq. (4), equipped with Eq. (5), is the time-dependent 1D nonpolynomial
Schrödinger equation (NPSE) [29, 30] for the wavefunction $\psi(s,t)$ of the
BEC moving along the curve ${\cal C}$ (see also [19]). As previously
discussed, the geodesic curvature $\kappa(s)$ gives rise to an effective
potential

$U_{Q}(s)=-{\hbar^{2}\kappa(s)^{2}\over 8m}\;.$ (6)

This curvature potential $U_{Q}(s)$ is quantum because it involves the square
of the reduced Planck constant $\hbar$. At fixed atomic mass $m$, only if the
square of the curvature $\kappa(s)$ is sufficently large the effects of this
quantum-curvature potential become relevant.

Under the assumption that $\sigma\simeq l_{\bot}$, which corresponds to a very
strong transverse confinement, the 1D NPSE becomes the familiar 1D Gross-
Pitaevskii (GPE) equation

$i\hbar\partial_{t}\psi=\Big{[}-{\hbar^{2}\over
2m}\partial_{s}^{2}-{\hbar^{2}\kappa(s)^{2}\over
8m}+\hbar\omega_{\bot}+{2\hbar^{2}a_{s}(N-1)\over
ml_{\bot}^{2}}|\psi|^{2}\Big{]}\psi\;.$ (7)

It is very important to stress that, from Eq. (5), the condition $\sigma\simeq
l_{\bot}$ implies $2a_{s}(N-1)|\psi|^{2}\ll 1$. In the rest of the paper we
will work within this 1D regime. n the new version of the manuscript I shall
discuss the role Clearly, Eq. (7) is reliable in the weak-coupling and strong-
transverse-confinement regime, where both beyond-mean-field and transverse-
size effects are very small.

Figure 1: Quantum-curvature potential $U_{Q}(s)$, Eq. (6), induced by the
geodesic curvature $\kappa(s)$ of an ellipse, as a function of the arclength
$s$, where $a$ is the length of the major semi-axis, $L=aE(2\pi,\epsilon)$ is
the perimeter of the ellipse, and $E_{a}=\hbar^{2}/(ma^{2})$ a characteristic
energy. The curves are obtained for different values of the eccentricity
$\epsilon$.

##  3 Properties of the elliptical waveguide

We now choose an ellipse for the curve ${\cal C}$. By using cartesian
coordinates its defining equation reads

${x^{2}\over a^{2}}+{y^{2}\over b^{2}}=1\;,$ (8)

where $a$ and $b$ are the lengths of the two semi-axes of the ellipse. Here we
assume that $a\geq b$, such that $a$ is the length of the major semi-axis. The
eccentricity of the ellipse is defined as [9]

$\epsilon=\sqrt{1-{b^{2}\over a^{2}}}\;.$ (9)

Clearly, $0\leq\epsilon<1$ and for $\epsilon=0$ we obtain a circle of radius
$R=a=b$. Introducing the angle $\phi\in[0,2\pi]$ we can write

$\displaystyle x=a\,\cos{(\phi)}$ (10) $\displaystyle y=b\,\sin{(\phi)}$ (11)

and the arclength $s$ along the ellipse can be expressed with the formula [9]

$s=a\,E(\phi,\epsilon)\;,$ (12)

where

$E(\phi,\epsilon)=\int_{0}^{\phi}\sqrt{1-\epsilon^{2}\sin^{2}{(\phi^{\prime})}}\,d\phi^{\prime}$
(13)

is the incomplete elliptic integral of the second kind. It follows that the
perimeter $L$ of the ellipse reads

$L=a\,E(2\pi,\epsilon)\;,$ (14)

such that for $\epsilon=0$ we have $L=2\pi a$ because $E(2\pi,0)=2\pi$.
Instead, for $\epsilon\to 1$ we have $L\to 4a$ because $E(2\pi,1)=4$. We
conclude that $4a<L\leq 2\pi a$. The geodesic curvature $\kappa$ of the
ellipse can be written as [9]

$\kappa={1\over
a}\;{\sqrt{1-\epsilon^{2}}\over\left(\sin^{2}{(\phi)}+\sqrt{1-\epsilon^{2}}\,\cos^{2}{(\phi)}\right)^{3/2}}\;.$
(15)

At fixed $a$ and $\epsilon$, the maximum of the curvature is obtained for
$\phi=0$ and $\phi=\pi$, i.e.

$\kappa_{max}={1\over a}\;{1\over\left(1-\epsilon^{2}\right)^{1/4}}\;,$ (16)

while the minimum of the curvature is obtained for $\phi=\pi/2$ and
$\phi=3\pi/2$, i.e.

$\kappa_{min}={1\over a}\;\sqrt{1-\epsilon^{2}}\;.$ (17)

Thus, for $\epsilon=0$ we have $\kappa_{max}=\kappa_{min}=1/a$ while for
$\epsilon\to 1$ we have $\kappa_{max}\to+\infty$ and $\kappa_{min}\to 0$. We
conclude that $1/a\leq k_{max}<+\infty$ and $0<k_{min}\leq 1/a$. In general,
the formula which gives the curvature $\kappa$ as a function of the arclength
$s$ is called Cesaro equation. Unfortunately, in the case of the ellipse there
is no Cesaro equation. In other words, an analytical formula of $\kappa$ as a
function of $s$ is not available. However, from Eqs. (12) and (15), fixing the
length $a$ and the eccentricity $\epsilon$ of the ellipse, we can easily plot
$\kappa$ vs $s$ using $\phi$ as dummy variable. More explicitly: we calculate
separately $\kappa$ vs $\phi$ and $s$ vs $\phi$, and then we plot $\kappa$ vs
$s$. The curvature $\kappa(s)$ has a the periodic structure of $\kappa(s)$. By
increasing $\epsilon$, the perimeter $L$ of the ellipse slightly decreases
while $\kappa_{max}$ and $\kappa_{min}$ pull away. In Fig. 1 we plot the
quantum-curvature potential $U_{Q}(s)$, Eq. (6), induced by the curvature
$\kappa(s)$ of the ellipse for different values of the eccentricity
$\epsilon$. The figure clearly shows that, for $\epsilon\neq 0$, $U_{Q}(s)$ is
symmetric double-well potential where the depth of the wells becomes larger by
increasing the eccentricity. The minima (maxima) of the quantum-curvature
potential $U_{Q}(s)$ are in correspondence to the maxima (minima) of the
curvature $\kappa(s)$.

Figure 2: Probablility density $\rho(s)$ of the non-interacting BEC ground
state in an ellipse as a function of the arclength $s$, where $a$ is the
length of the major semi-axis and $L=aE(2\pi,\epsilon)$ is the perimeter of
the ellipse. The curves are obtained for different values of the eccentricity
$\epsilon$. Here the s-wave scattering length $a_{s}$ is set to zero or,
equivalently, the number $N$ of particles is set to one.

##  4 BEC ground state in elliptical waveguide

The time-independent 1D GPE is obtained from Eqs. (7) setting

$\psi(s,t)=\Phi(s)\,e^{-i(\mu+\hbar\omega_{\bot})t/\hbar}\;.$ (18)

In this way we have

$\displaystyle\mu\,\Phi=\Big{[}-{\hbar^{2}\over
2m}\partial_{s}^{2}-{\hbar^{2}\kappa(s)^{2}\over
8m}+{2a_{s}(N-1)\hbar^{2}\over ml_{\bot}^{2}}|\Phi|^{2}\Big{]}\Phi\;,$ (19)

that is the 1D GPE equation for the stationary wavefunction $\Phi(s)$, such
that $\rho(s)=|\Phi(s)|^{2}$ is the probability density of finding the BEC at
the position $s$. In Fig. 2 we report the probability density $\rho(s)$ of the
ground-state of the non-interacting ($a_{s}=0$ or, equivalently, $N=1$) BEC
confined along the ellipse as a function of the arclength $s$. Our results are
obtained by solving Eq. (7) with a Crank-Nicolson predictor-corrector method
and imaginary time. In the figure, the curves correspond to different values
of the eccentricity $\epsilon$. Clearly, for $\epsilon=0$ the ground state is
uniform along the ellipse. However, for $\epsilon\neq 0$ the ground state is
no more uniform due to a non-constant curvature $\kappa(s)$ which implies a
non-constant effective potential $U_{Q}(s)=-\hbar^{2}\kappa(s)^{2}/(8m)$. By
increasing the eccentricity $\epsilon$ the localization of $\rho(s)$ around
the minima of $U_{Q}(s)$, where the curvature is larger, becomes more evident.

Figure 3: Probablility density $\rho(s)$ of the attractive BEC ground state in
an ellipse as a function of the arclength $s$, where $a$ is the length of the
major semi-axis and $L=aE(2\pi,\epsilon)$ is the perimeter of the ellipse. The
curves are obtained with eccentricity $\epsilon=0.9$ for different values of
the adimensional interaction strength $\gamma=2aa_{s}(N-1)/l_{\bot}^{2}$,
where $a_{s}$ is the 3D s-wave scattering length and $l_{\bot}$ is the
characteristic length of the transverse harmonic confinement.

##  5 Quantum phase transition

It is interesting to investigate the effect of the inter-atomic interaction on
the ground state properties of the system. In adimensional units the
interaction strength reads $\gamma=g/(2\pi
l_{\bot}^{2}aE_{a})=2aa_{s}(N-1)/l_{\bot}^{2}$ with
$E_{a}=\hbar^{2}/(ma^{2})$. In Fig. 3 we consider the case of an attractive
BEC and plot our numerical results obtained for different values of a negative
$\gamma$ with fixed eccentricity $\epsilon=0.9$. Quite remarkably, for
$\gamma<-1.5$ the ground state has a spontaneous symmetry breaking: one of the
two local mimima contains more bosons. Indeed for $\gamma=-2$ this single-well
localization is very clear. It is important to observe that this kind of
quantum phase transition happens for any $\epsilon$. In the case of a circle
($\epsilon=0$) a similar spontaneous symmetry breaking was predicted about 20
years ago [22, 24, 23]. Actually, this quantum phase transition, or
spontaneous symmetry breaking, is nothing else than the modulational
instability of the ground-state configuration, induced by the appearance of an
imaginary component in the energies of the elementary excitations of the
ground state [21]. However, for $\epsilon=0$ there is a quantum phase
transition from a uniform configuration to a single-peak configuration, while
for the $\epsilon\neq 0$ there is a quantum phase transition from a two-peak
configuration to a single-peak configuration. This quantum phase transition
was observed some years ago with an attractive BEC of 39K atoms, where the
double-well potential was created by intersecting two pairs of laser beams
[32]. Our double-well system is slightly different because the particles
tunnel from one well to the other well following two different curved paths;
moreover, our elipsoidal configuration offers also the possibility of having
persistent currents. For a sufficiently attractive BEC the transverse width
$\sigma$ of the BEC becomes smaller than $l_{\bot}$ and there is the collapse
of the single-peak configuration. For $\epsilon=0$ the 1D NPSE predicts the
collapse of this single-peak configuration at $\gamma_{c}=(4/3)(a/l_{\bot})$
[29, 31]. Thus, our numerical results of Fig. 3, obtained from the 1D GPE, are
fully reliable under the condition $a/l_{\bot}\gg 1$. This is again the
condition of a tight transverse confinement.

##  6 Superfluid fraction

Let us consider the effect of a boost velocity $v_{B}$ on the BEC moving along
the ellipse. In this case Eq. (19) is modified as follows

$\mu\,\Phi=\Big{[}{1\over
2m}\left(-i\hbar\partial_{s}-mv_{\mathrm{B}}\right)^{2}-{\hbar^{2}\kappa^{2}(s)\over
8m}+{2a_{s}(N-1)\hbar^{2}\over ml_{\bot}^{2}}|\Phi|^{2}\Big{]}\Phi\;,$ (20)

Now we set

$\Phi(s)={n(s)^{1/2}\over\sqrt{N}}\,e^{i\theta(s)}\;,$ (21)

where $n(s)=N\rho(s)$ is the local number density of the BEC. Morover, we
introduce the local velocity field

$v(s)={\hbar\over m}\partial_{s}\theta(s)\;.$ (22)

Inserting these formulas into Eq. (20) we obtain 1D stationary equations of
zero-temperature superfluid hydrodynamics

$\mu=-{\hbar^{2}\over 2m\sqrt{n}}\partial_{s}^{2}\sqrt{n}+{m\over
2}(v-v_{\mathrm{B}})^{2}-{\hbar^{2}\kappa(s)^{2}\over
8m}+{2a_{s}(1-1/N)\hbar^{2}n\over ml_{\bot}^{2}}\;,$ (23)

and also

$\partial_{s}\left[n\left(v-v_{B}\right)\right]=0\;.$ (24)

Eq. (24) implies that $n\,\left(v-v_{B}\right)=J$, where $J$ is a constant
current density. This result is very interesting because it says that if
$n(s)$ has spatial variations then also $v(x)$ must have spatial variations.

Inspired by Ref. [28], we now introduce the average value of the velocity
$v(x)$ in a region $[a,b]$ of the ellipse as

${\bar{v}}={1\over(b-a)}\int_{a}^{b}v(s)\,ds\;,$ (25)

Then, from the previous equations we obtain

${\bar{v}}={1\over(b-a)}\int_{a}^{b}\left({J\over
n(s)}+v_{B}\right)\,ds={J\over{\bar{n}}_{s}}+v_{B}\;,$ (26)

where

${\bar{n}}_{s}={1\over{1\over(b-a)}\int_{a}^{b}{1\over n(s)}ds}\;.$ (27)

The number density ${\bar{n}}_{s}$ can be interpreted as the superfluid number
density of the stationary state in the spatial region $[a,b]$. Indeed, Eq.
(27) is the 1D version of the formula obtained by Leggett [25] for a
supersolid with spatial periodicity $(b-a)$, and recently discussed by others
[26, 27, 28]. If the stationary state $\Psi(s)$ moves with the average
velocity ${\bar{v}}$, its current density reads
$J={\bar{n}}_{s}\left({\bar{v}}-v_{B}\right)$, where ${\bar{v}}$ is the
average velocity in the region $[a,b]$ and ${\bar{n}}_{s}$ the corresponding
superfluid number density. We can also introduce

${\bar{n}}={1\over(b-a)}\int_{a}^{b}n(s)\,ds$ (28)

that is the average number density in the region $[a,b]$. Consequently, the
superfluid fraction of the BEC in the region $[0,L]$ reads

$f_{s}=\frac{{\bar{n}}_{s}}{{\bar{n}}}={1\over{N\over
L^{2}}\int_{0}^{L}{1\over n(s)}ds}\;,$ (29)

where $N=\int_{0}^{L}n(s)\,dx=L{\bar{n}}$. This formula can be also obtained
as the response of the linear momentum of the BEC to the boost velocity
${\bar{v}}_{B}$, that is the non-classical translational inertia of the system
[26].

In Fig. 4 we plot our numerical results of the superfluid fraction $f_{s}$ as
a function of the adimensional interaction strength $\gamma$ for different
values of the eccentricity $\epsilon$ of the ellipse. For positive values of
$\gamma$ the superfluid fraction $f_{s}$ is close to $1$ also with
$\epsilon=0.9$. However, for $0\leq\gamma<1$ and a very large eccentricity
($\epsilon=0.99$) we find $f_{s}\simeq 0.95$. This result is quite reasonable
because the wavefunction is strongly localized in the two well of the quantum-
curvature potential. For negative values of $\gamma$ the most interesting
effect appears in Fig. 4: around $\gamma\simeq-1.6$ the superfluid fraction
$f_{s}$ quckly decreases and it goes to zero for very large negative values of
$\gamma$. This is exacly the quantum phase transition from a two-peak
configuration to a one-peak configuration. As previously stressed, the one-
peak configuration becomes modulationally unstable when at least one of the
energies of its elementary excitations acquires an imaginary component. A
similar modulational instability [21] happens in the formation of a train of
bright solitons from a single-peak Bose-Einstein condensate, induced by a
sudden change in the sign of the scattering length from positive to negative
[33]. Our Fig. 4 reveals that the critical strength $\gamma_{c}$ crucially
depends on the eccentricity $\epsilon$, such as the behavior of $f_{s}$ as a
function of $\gamma$ for $\gamma<\gamma_{c}$. In particular, we find that
$\gamma_{c}$ slightly reduces by increasing $\epsilon$, but this effect is
quite weak.

Figure 4: Superfluid fraction $f_{s}$ of the BEC ground state in an ellipse as
a function of the adimensional interaction strength
$\gamma=2aa_{s}(N-1)/l_{\bot}^{2}$, where $a$ is the length of the major semi-
axis, $a_{s}$ is the 3D s-wave scattering length and $l_{\bot}$ is the
characteristic length of the transverse harmonic confinement. The curves are
obtained for three values of the eccentricity $\epsilon$ of the ellipse.

An important remark is that Eq. (29) has been derived here without any
assumption about the sign of $\gamma$. Moreover, the absence of superfluidity
for $\gamma=0$ is true only in the thermodynamic limit. In a ring there is a
finite energy gap between the ground state and the first excited state also
for $\gamma=0$. The Bose system we are considering has a finite size because
it is confined in a finite elliptical ring. Finally, the 3D version of Eq.
(27) was proposed historically by Leggett to characterize the superfluid
density of a supersolid [25], while here Eq. (27) is used to determine the
superfluid density of a Bose-Einstein condensate which is not supersolid but
it is instead spatially modulated due to the crucial interplay between
elliptical confinement and attractive interaction.

##  7 Conclusions

The main goal of this paper was to understand the role of a locally-varying
curvature for a Bose-Einstein condensate confined in an elliptical waveguide.
The proposed setup, and the double-well quantum-curvature potential that we
have found, can be experimentally achieved by using ultracold atoms, which are
a paradigmatic physical platform due to the high experimental tunability of
inter-atomic interactions and trapping potentials. For instance, one can trap
$N=10^{4}$ ultracold Rb atoms by using a rapidly moving laser beam which
creates a time-averaged elliptic-shaped toroidal optical dipole potential
[34]. The length $a$ of the major semi-axis of the ellpse can be $a\simeq 100$
microns and the transverse length $l_{\bot}=5$ microns. The scattering length
$a_{s}$ could be then tuned by using an external magnetic field, which induces
a Fano-Feshbach resonance [35]. Despite the fact that we focused on space
curvature rather than space-time curvature, we believe that our results can be
of interest not only to atomic and condensed matter physics researchers, but
also to a large community working on general relativity and relativistic
quantum field theory.

Acknowledgements. The author thanks Francesco Ancilotto, Koichiro Furutani,
Francesco Minardi, and Andrea Tononi for useful discussions.

## References

  * [1]
  * [2] E. Switkes, E.L. Russel, and J.L. Skinner, Kinetic energy and path curvature in bound state systems, J. Chem. Phys. 67, 3061 (1977).
  * [3] R.C.T. Da Costa, Quantum mechanics of a constrained particle, Phys. Rev. A 23, 1982 (1981).
  * [4] R.C.T. Da Costa, Constraints in quantum mechanics. Phys. Rev. A 25, 2893 (1982).
  * [5] J.J. Halliwell, Derivation of the Wheeler-DeWitt equation from a path integral for minisuperspace models, Phys. Rev. D 38, 2468 (1988).
  * [6] L. Kaplan, N.T. Maitra, and E.J. Heller, Quantizing constrained systems, Phys. Rev. A 56, 2592 (1997).
  * [7] J.M. Lee, Riemannian Manifolds: An Introduction to the Curvature (Springer, 2007).
  * [8] J. Jost, Riemannian Geometry and Geometric Analysis (Springer, 2010).
  * [9] M.P. da Carmo, Differential Geometry of Curves and Surfaces (Dover Publications, 2016).
  * [10] P. Exner and P. Seba, Bound states in curved quantum waveguides, J. Math. Phys. 30, 2574 (1989).
  * [11] J. Goldstone and R.L. Jaffe, Bound states in twisting tubes, Phys. Rev. B 45, 14100 (1992).
  * [12] I.J. Clark and A.J. Bracken, Effective potentials of quantum strip waveguides and their dependence upon torsion, J. Phys. A: Math. Gen. 29, 339 (1996).
  * [13] I.J. Clark, More on effective potentials of quantum strip waveguides, J. Phys. A: Math. Gen. 31, 2103 (1998).
  * [14] P. Exner and S.A: Vugalter, On the number of particles that a curved quantum waveguide can bind, J. Math. Phys. 40, 4630 (1999).
  * [15] P. Exner and H. Kovarik, Quantum Waveguides (Springer, 2015).
  * [16] P. Leboeuf and N. Pavloff, Bose-Einstein beams: Coherent propagation through a guide, Phys. Rev. A 64, 033602 (2001).
  * [17] S. Schwartz, M. Cozzini, C. Menotti, I. Carusotto, P. Bouyer, and S. Stringari, One-dimensional description of a Bose-Einstein condensate in a rotating closed-loop waveguide, New J. Phys. 8, 162 (2006).
  * [18] A. del Campo, M.G. Boshier, and A. Saxena, Bent waveguides for matter-waves: supersymmetric potentials and reflectionless geometries, Sci. Rep. 4, 5274 (2014).
  * [19] P. Sandin, M. Ögren, M. Gulliksson, J. Smyrnakis, M. Magiropoulos, and G.M. Kavoulakis, Dimensional reduction in Bose-Einstein condensed clouds of atoms confined in tight potentials of any geometry and any interaction strength, Phys. Rev. E 95, 012142 (2017).
  * [20] N.S. Moller, F.E.A. dos Santos, V.S. Bagnato, and A. Pelster, Bose-Einstein condensation on curved manifolds, New. J. Phys. 22, 063059 (2020).
  * [21] L. Salasnich, A. Parola, and L. Reatto, Modulational Instability and Complex Dynamics of Confined Matter-Wave Solitons, Phys. Rev. Lett. 91, 080405 (2003).
  * [22] L.D. Carr, C.W. Clark, and W.P. Reinhardt, Stationary solutions of the one-dimensional nonlinear Schrödinger equation. II. Case of attractive nonlinearity, Phys. Rev. A 62, 063611 (2000).
  * [23] G.M. Kavoulakis, Bose-Einstein condensates with attractive interactions on a ring, Phys. Rev. A 67, 011601(R) (2003).
  * [24] R. Kanamoto, H. Saito, and M. Ueda, Quantum phase transition in one-dimensional Bose-Einstein condensates with attractive interactions, Phys. Rev. 67, 013608 (2003).
  * [25] A. J. Leggett, Can a Solid Be ”Superfluid”?, Phys. Rev. Lett. 25, 1543 (1970).
  * [26] N. Sepulveda, C. Josserand, and S. Rica, Nonclassical rotational inertia fraction in a one-dimensional model of a supersolid, Phys. Rev. B 77, 054513 (2008).
  * [27] L. Chomaz, Probing the supersolid order via high-energy scattering: Analytical relations among the response, density modulation, and superfluid fraction, Phys. Rev. A 102, 023333 (2020).
  * [28] G.I. Martone, A. Recati, and N. Pavloff, Supersolidity of cnoidal waves in an ultracold Bose gas, Phys. Rev. Res. 3, 013143 (2021).
  * [29] L. Salasnich, A. Parola, and L. Reatto, Effective wave-equations for the dynamics of cigar-shaped and disc-shaped Bose condensates, Phys. Rev. A 65, 043614 (2002).
  * [30] L. Salasnich, B.A. Malomed, and F. Toigo, Matter-wave vortices in cigar-shaped and toroidal waveguides, Phys. Rev. A 76, 063614 (2007).
  * [31] A. Parola, L. Salasnich, R. Rota, and L. Reatto, Quantum phases of attractive matter waves in a toroidal trap, Phys. Rev. A 72, 063612 (2005).
  * [32] A. Trenkwalder, G. Spagnolli, G. Semeghini, S. Coop, M. Landini, P. Castilho, L. Pezze, G. Modugno, M. Inguscio, A. Smerzi, and M. Fattori, Quantum phase transitions with parity-symmetry breaking and hysteresis, Nature Phys. 12, 826 (2016).
  * [33] K.E. Strecker, G.B. Partridge, A.G. Truscott, and R.G. Hulet, Formation and propagation of matter-wave soliton trains, Nature 417, 150 (2002).
  * [34] K. Henderson, C. Ryu, C. MacCormick, and M.G. Boshier, Experimental demonstration of painting arbitrary and dynamic potentials for Bose–Einstein condensates, New J. Phys. 11, 043030 (2009).
  * [35] C. Chin, R. Grimm, P. Julienne, and E. Tiesinga, Feshbach resonances in ultracold gases, Rev. Mod. Phys. 82, 1225 (2010).


# Isotropy, Clusters, and Classifiers

Timothee Mickus &Stig-Arne Grönroos &Joseph Attieh

University of Helsinki
<EMAIL_ADDRESS>
###### Abstract

Whether embedding spaces use all their dimensions equally, i.e., whether they
are isotropic, has been a recent subject of discussion. Evidence has been
accrued both for and against enforcing isotropy in embedding spaces. In the
present paper, we stress that isotropy imposes requirements on the embedding
space that are not compatible with the presence of clusters—which also
negatively impacts linear classification objectives. We demonstrate this fact
empirically and use it to shed light on previous results from the literature.

##  1 Introduction

Recently, there has been much discussion centered around whether vector
representations used in NLP do and should use all dimensions equally. This
characteristic is known as isotropy: In an isotropic embedding model, every
direction is equally probable, ensuring uniform data representation without
directional bias. At face value, such a characteristic would appear desirable:
Naively, one could argue that an anisotropic embedding space would be
overparametrized, since it can afford to use some dimensions inefficiently.

The debate surrounding isotropy was initially sparked by Mu and Viswanath
(2018), who highlighted that isotropic static representations fared better on
common lexical semantics benchmarks, and Ethayarajh (2019), who stressed that
contextual embeddings are anisotropic. Since then, evidence has been accrued
both for and against enforcing isotropy on embeddings.

In the present paper, we demonstrate that this conflicting evidence can be
accounted for once we consider how isotropy relates to embedding space
geometry. Strict isotropy, as assessed by IsoScore (Rudman et al., 2022),
requires the absence of clusters, and thereby also conflicts with linear
classification objectives. This echoes previous empirical studies connecting
isotropy and cluster structures (Ait-Saada and Nadif, 2023, a.o.). In the
present paper, we formalize this connection mathematically in Section 2. We
then empirically verify our mathematical approach in Section 3, discuss how
this relation sheds light on earlier works focusing on anisotropy in Section
4, and conclude with directions for future work in Section 5.

##  2 Some conflicting optimization objectives

We can show that isotropy—as assessed by IsoScore (Rudman et al., 2022)—impose
requirements that conflict with cluster structures—as assessed by silhouette
scores (Rousseeuw, 1987)—as well as linear classifier objectives.

#### Notations.

In what follows, let $\mathcal{D}$ be a multiset of points in a vector space,
$\Omega$ a set of labels, and $\ell:\mathcal{D}\to\Omega$ a labeling function
that associates a given data-point in $\mathcal{D}$ to the relevant label. For
simplicity, let us further assume that $\mathcal{D}$ is PCA-transformed. Let
us also define the following constructs for clarity of exposition:

$\displaystyle\mathcal{D}_{\omega}$
$\displaystyle=\left\\{\mathbf{d}~{}:~{}\ell\left(\mathbf{d}\right)=\omega\right\\}$
$\displaystyle\mathrm{sign}(\omega,\omega^{\prime})$
$\displaystyle=\begin{cases}-1\qquad\mathrm{if}~{}\omega=\omega^{\prime}\\\
+1\hfill\mathrm{otherwise}\end{cases}$

Simply put, $\mathcal{D}_{\omega}$ is the subset of points in $\mathcal{D}$
with label $\omega$, whereas the $\mathrm{sign}$ function helps delineate
terms that need to be maximized (inter-cluster) vs. terms that need to be
minimized (intra-cluster).

###  2.1 Silhouette objective for clustering

We can consider whether the groups as defined by $\ell$ are in fact well
delineated by the Euclidean distance, i.e., whether they form natural
clusters. This is something that can be assessed through silhouette scores,
which involve a _separation_ and a _cohesion_ score for each data-point. The
cohesion score consists in computing the average distance between the data-
point and other members of its group, whereas separation consists in computing
the minimum cohesion score the data-point could have received with any other
label than the one it was assigned to. More formally, let:

$\displaystyle\mathrm{cost}(\mathbf{d},\mathcal{S})$
$\displaystyle=\frac{1}{|\mathcal{S}|}\sum\limits_{\mathbf{d^{\prime}}\in\mathcal{S}}\sqrt{\sum_{i}\left(\mathbf{d}_{i}-\mathbf{d^{\prime}}_{i}\right)^{2}}$

then we can define the silhouette for one sample as

$\displaystyle\mathrm{coh}(\mathbf{d})$
$\displaystyle=\mathrm{cost}\left(\mathbf{d},\mathcal{D}_{\ell(\mathbf{d})}\setminus\\{\mathbf{d}\\}\right)$
$\displaystyle\mathrm{sep}(\mathbf{d})$
$\displaystyle=\min\limits_{\omega^{\prime}\in\Omega\setminus\\{\ell(\mathbf{d})\\}}\mathrm{cost}\left(\mathbf{d},\mathcal{D}_{\omega^{\prime}}\right)$
$\displaystyle\mathrm{silhouette}(\mathbf{d})$
$\displaystyle=\frac{\mathrm{sep}(\mathbf{d})-\mathrm{coh}(\mathbf{d})}{\max\\{\mathrm{sep}(\mathbf{d}),\mathrm{coh}(\mathbf{d})\\}}$

Or in other words, the silhouette score is maximized when separation cost
($\mathrm{sep}$) is maximized and cohesion cost ($\mathrm{coh}$) is minimized.
Hence, to maximize the silhouette score across the whole dataset
$\mathcal{D}$, one needs to (i) maximize all inter-cluster distances, and (ii)
minimize all intra-cluster distances.

We can therefore define a maximization objective for the entire set
$\mathcal{D}$:

$\sum\limits_{\mathbf{d}\in\mathcal{D}}\sum\limits_{\mathbf{d^{\prime}}\in\mathcal{D}}\mathrm{sign}(\ell(\mathbf{d}),\ell(\mathbf{d^{\prime}}))\sqrt{\sum_{i}\left(\mathbf{d}_{i}-\mathbf{d^{\prime}}_{i}\right)^{2}}$

which, due to the monotonicity of the square root in $\mathbb{R}^{+}$, will
have the same optimal argument $\mathcal{D}^{\ast}$ as the simpler objective
$\mathcal{O}_{\mathrm{S}}$

$\mathcal{O}_{\mathrm{S}}=\sum\limits_{\mathbf{d}\in\mathcal{D}}\sum\limits_{\mathbf{d^{\prime}}\in\mathcal{D}}\mathrm{sign}(\ell(\mathbf{d}),\ell(\mathbf{d^{\prime}}))\sum_{i}\left(\mathbf{d}_{i}-\mathbf{d^{\prime}}_{i}\right)^{2}$
(1)

###  2.2 Incompatibility with IsoScore

How does the objective in (1) conflict with isotropy requirements? Assessments
of isotropy such as IsoScore generally rely on the variance vector. As we
assume $\mathcal{D}$ to be PCA transformed, the covariance matrix is
diagonalized, and we can obtain variance for each individual component through
pairwise squared distances (Zhang et al., 2012):

$\mathbb{V}(\mathcal{D})_{i}=\frac{1}{2|\mathcal{D}|^{2}}\sum\limits_{\mathbf{d}\in\mathcal{D}}\sum\limits_{\mathbf{d^{\prime}}\in\mathcal{D}}\left(\mathbf{d}_{i}-\mathbf{d^{\prime}}_{i}\right)^{2}$

In IsoScore, this variance vector is then normalized to the length of the
$\vec{1}$ vector of all ones, before computing the distance between the two:

$\sqrt{\sum_{i}\left(\frac{\lVert\vec{1}\rVert_{2}}{\lVert\mathbb{V}(\mathcal{D})\rVert_{2}}\mathbb{V}(\mathcal{D})_{i}-1\right)^{2}}$

This distance is taken as an indicator of isotropy defect, i.e., isotropic
spaces will minimize it.

Given the normalization applied to the variance vector, the defect is computed
as the distance between two points on a hyper-sphere. Hence it is conceptually
simpler to think of this distance as an _angle_ measurement: Remark that as
the cosine between $\mathbb{V}(\mathcal{D})$ and $\vec{1}$ increases, the
isotropy defect decreases. In short, to maximize isotropy, we have to maximize
the objective $\mathcal{O}_{\mathrm{I}}$

$\displaystyle\mathcal{O}_{\mathrm{I}}$
$\displaystyle=\cos\left(\vec{1},\mathbb{V}\left(\mathcal{D}\right)\right)$
$\displaystyle\propto\sum\limits_{\mathbf{d}\in\mathcal{D}}\sum\limits_{\mathbf{d^{\prime}}\in\mathcal{D}}\sum_{i}\left(\mathbf{d}_{i}-\mathbf{d^{\prime}}_{i}\right)^{2}$
(2)

This intuitively makes sense: Ignoring vector norms, we have to maximize all
distances between every pair of data-points to ensure all dimensions are used
equally, i.e., spread data-points out evenly on a hyper-sphere. However, in
the general case, it is not possible to maximize both the isotropy objective
in (2.2) and the silhouette score objective in (1): Intra-cluster pairwise
distances must be minimized for optimal silhouette scores, but must be
maximized for optimal isotropy scores. In fact, the two objectives can only be
jointly maximized in the degenerate case where no two data-points in
$\mathcal{D}$ are assigned the same label.111 Hence some NLP applications and
tasks need not be impeded by isotropy constrains, e.g., linear analogies that
rely on vector offsets are _a prima facie_ compatible with isotropy.

###  2.3 Relation to linear classifiers

Informally, latent representations need to form clusters corresponding to the
labels in order to optimize a linear classification objective. Consider that
in classification problems (i) any data-point $\mathbf{d}$ is to be associated
with a particular label $\ell(\mathbf{d})=\omega_{i}$ and dissociated from
other labels $\Omega\setminus\\{\ell(\mathbf{d})\\}$, and (ii) association
scores are computed using a dot product between the latent representation to
be classified and the output projection matrix, where each column vector
$\mathbf{c}^{\omega}$ corresponds to a different class label $\omega$. As
such, for any point $\mathbf{d}$ to be associated with its label
$\ell(\mathbf{d})$, one has to maximize

$\langle\mathbf{d},\mathbf{c}^{\ell(\mathbf{d})}\rangle=\frac{1}{2}\left(\lVert\mathbf{d}\rVert^{2}_{2}+\lVert\mathbf{c}^{\ell(\mathbf{d})}\rVert^{2}_{2}-\lVert\mathbf{d}-\mathbf{c}^{\ell(\mathbf{d})}\rVert^{2}_{2}\right)$

In other words, one must either augment the norm of $\mathbf{d}$ or
$\mathbf{c}^{\ell(\mathbf{d})}$, or minimize the distance between $\mathbf{d}$
and $\mathbf{c}^{\ell(\mathbf{d})}$. Note however that this does not factor in
the other classes $\omega^{\prime}\in\Omega\setminus\\{\ell(\mathbf{d})\\}$
from which $\mathbf{d}$ should be dissociated, i.e., where we must minimize
the above quantity. To account for the other classes, the global objective
$\mathcal{O}_{\mathrm{C}}$ to maximize can be defined as

$\begin{aligned}
\mathcal{O}_{\mathrm{C}}=&\sum_{\mathbf{d}\in\mathcal{D}}\sum\limits_{\omega\in\Omega}\mathrm{sign}\left(\omega,\ell\left(\mathbf{d}\right)\right)\langle\mathbf{d},\mathbf{c}^{\omega}\rangle\\\
=&-\sum_{\mathbf{d}\in\mathcal{D}}\frac{|\Omega|-2}{2}\lVert\mathbf{d}\rVert^{2}_{2}-\sum\limits_{\omega\in\Omega}\frac{|\mathcal{D}|-2|\mathcal{D}_{\omega}|}{2}\lVert\mathbf{c}^{\omega}\rVert^{2}_{2}\\\
&+\frac{1}{2}\sum_{\mathbf{d}\in\mathcal{D}}\sum\limits_{\omega\in\Omega}\mathrm{sign}\left(\omega,\ell\left(\mathbf{d}\right)\right)\sum_{i}\left(\mathbf{d}_{i}-\mathbf{c}^{\omega}_{i}\right)^{2}\end{aligned}$

(3)

Focusing on the last line, we find that maximizing classification objectives
entails minimizing the distance between a latent representation $\mathbf{d}$
and the vector for its label $\mathbf{c}^{\ell(d)}$, and maximizing its
distance to all other class vectors.222 The other two sums correspond to
probabilistic priors over the data: The objective entails that the norm of a
class vector $\mathbf{c}^{\omega}$ should be proportional to the number of
data-points with this label $\omega$, whereas one would expect a uniform
distribution for vectors $\mathbf{d}$. These terms cancel out for balanced,
binary classification tasks.  It is reminiscent of the silhouette score in
Equation 1: In particular any optimum for $\mathcal{O}_{\mathrm{C}}$ is an
optimum for $\mathcal{O}_{\mathrm{S}}$, since it entails $\mathcal{D}^{\ast}$
such that

$\forall\mathbf{d},\mathbf{d^{\prime}}\in\mathcal{D}^{\ast}\quad\ell(\mathbf{d})=\ell(\mathbf{d^{\prime}})\iff\mathbf{d}=\mathbf{d^{\prime}}$

Informally: The cluster associated with a label should collapse to a single
point. Therefore the isotropic objective $\mathcal{O}_{\mathrm{I}}$ in Section
2.2 is equally incompatible with the learning objective
$\mathcal{O}_{\mathrm{C}}$ of a linear classifier.

#### In summary,

(i) point clouds cannot both contain well-defined clusters and be isotropic;
and (ii) linear classifiers should yield clustered and thereby anisotropic
representations.

##  3 Empirical confirmation

To verify the validity of our demonstrations in Section 2, we can optimize a
set of data-points for a classification task using a linear classifier: We
should observe an increase in silhouette scores, and a decrease in IsoScore.

###  3.1 Methodology

We consider four setups: (i) optimizing SBERT sentence embeddings (Reimers and
Gurevych, 2019)333all-MiniLM-L6-v2 on the binary polarity dataset of Pang and
Lee (2004); (ii) optimizing paired SBERT embeddings††footnotemark:  on the
validation split of SNLI (Bowman et al., 2015); (iii) optimizing word2vec
embeddings444http://vectors.nlpl.eu/repository/, model 222 on POS-tagging
multi-label classification using the English CoDWoE dataset (Mickus et al.,
2022); and (iv) optimizing word2vec embeddings††footnotemark:  for WordNet
supersenses multi-label classification (Fellbaum, 1998; pre-processed by
Tikhonov et al., 2023). For (i) and (ii), we directly optimize the output
embeddings of the SBERT model rather than update the parameters of the SBERT
model. In all cases, we compute gradients for the entire dataset, and compute
silhouette scores with respect to the target labels and IsoScore over 1000
updates. In multi-label cases (iii) and (iv), we consider distinct label
vectors as distinct target assignments when computing silhouette scores.
Models are trained using the Adam algorithm (Kingma and Ba, 2014);555 Learning
rate of $0.001$, $\beta$ of $(0.9,0.999)$.  in case (i) and (ii) we optimize
cross-entropy, in case (iii) and (iv), binary cross-entropy per label. Remark
that setups (ii), (iii) and (iv) subtly depart from the strict requirements
laid out in Section 2.

(a) Silhouette across training (b) Log-normalized IsoScore across training

Figure 1: Evolution of silhouette score and IsoScore across classification
optimization (avg. of 5 runs). Figure 2: Relationship between silhouette
scores and IsoScore (avg. of 5 runs).

###  3.2 Results

Results of this empirical study are displayed in Figure 1b. Performances with
five different random initialization reveal negligible standard deviations
(maximum at any step $<0.0054$, on average $<0.0008$). Our demonstration is
validated: Across training to optimize classification tasks, the data-points
become less isotropic and better clustered. We can also see a monotonically
decreasing relationship between IsoScore and silhouette scores, which is
better exemplified in Figure 2: We find correlations with Pearson’s $r$ of
$-0.808$ for the polarity task, $-0.898$ for SNLI, $-0.947$ for POS-tagging
and $-0.978$ for supersense tagging; Spearman’s $\rho$ are always below
$-0.998$.

#### In summary,

we empirically confirm that isotropy requirements conflict with silhouette
scores and linear classification objectives.

##  4 Related works

How does the connection between clusterability and isotropy that we outlined
shed light on the growing literature on anisotropy?

While there is currently more evidence in favor of enforcing isotropy in
embeddings, the case is not so clear cut that we can discard negative
findings, and a vast majority of the positive evidence relies on improper
techniques for quantifying isotropy (Rudman et al., 2022). Ethayarajh (2019)
stressed that contextual embeddings are effective yet anisotropic. Ding et al.
(2022) provides experiments that advise against using isotropy calibration on
transformers to enhance performance in specific tasks. Rudman and Eickhoff
(2023) finds that anisotropy regularization in fine-tuning appears to be
beneficial on a large array of tasks. Lastly, Rajaee and Pilehvar (2021a) find
that the contrasts encoded in dominant dimensions can, at times, capture
linguistic knowledge.

On the other hand, the original study of Mu and Viswanath (2018) found that
enforcing isotropy on static embeddings improved performances on semantic
similarity, both at the word and sentence level, as well as word analogy.
Subsequently, a large section of the literature has focused on this handful of
tasks (e.g., Liang et al., 2021; Timkey and van Schijndel, 2021). Isotropy was
also found to be helpful beyond these similarity tasks: Haemmerl et al. (2023)
report that isotropic spaces perform much better on cross-lingual tasks, and
Jung et al. (2023) stress its benefits for dense retrieval.

These are all applications that require graded ranking judgments, and
therefore are generally hindered by the presence of clusters—such clusters
would for instance introduce large discontinuities in cosine similarity
scores. To take Haemmerl et al. (2023) as an example, note that language-
specific clusters are antithetical to the success of cross-lingual transfer
applications. It stands to reason that isotropy can be found beneficial in
such cases, although the exact experimental setup will necessarily dictate
whether it is boon or bane: For instance Rajaee and Pilehvar (2021b) tested
fine-tuning LLMs as Siamese networks to optimize performance on sentence-level
similarity, and found enforcing isotropy to hurt performances—here, we can
conjecture that learning to assign inputs to specific clusters is a viable
solution in their case.

The literature has previously addressed the topic of isotropy and clustering.
Rajaee and Pilehvar (2021a) advocated for enhancing the isotropy on a cluster-
level rather than on a global-level. Cai et al. (2021) confirmed the presence
of clusters in the embedding space with local isotropy properties. Ait-Saada
and Nadif (2023) investigated the correlation between isotropy and clustering
tasks and found that fostering high anisotropy yields high-quality clustering
representations. The study presented here provides a mathematical explanation
for these empirical findings.

##  5 Conclusion

We argued that isotropy and cluster structures are antithetical (Section 2),
verified that this argument holds on real data (Section 3), and used it to
shed light on earlier results (Section 4). This result however opens novel and
interesting directions of research: If anisotropic spaces implicitly entail
cluster structures, then what is the structure we observe in our modern,
highly anisotropic large language models? Prior results suggest that this
structure is in part linguistic in nature (Rajaee and Pilehvar, 2021a), but
further confirmation is required.

Another topic we intend to pursue in future work concerns the relation between
non-classification tasks and isotropy: Isotropy constraints have been found to
be useful in problems that are not well modeled by linear classification, e.g.
word analogy or sentence similarity. Our present work does not yet offer a
thorough theoretical explanation why.

## Acknowledgments

This work is part of the FoTran project, funded by the European Research
Council (ERC) under the EU’s Horizon 2020 research and innovation program
(agreement № 771113). We also thank the CSC-IT

Center for Science Ltd., for computational resources.

## Limitations

The present paper leaves a number of important problems open.

  * •

Our claims with respect to classification are limited to linear classifiers.
However, most (if not all) modern deep-learning classification approaches rely
on non-linear activation functions across multiple layers of computations. The
present demonstration has yet to be expanded to account for such more common
cases.

  * •

Our argument focuses on the optima of specific objectives, and says nothing of
behavior across training. In particular, we focus on parameters that are
optimized for a particular task, but NLP practitioners often verify and
measure anisotropy in generalization conditions. In fact, enforcing isotropy
could be argued to be a reasonable regularization strategy in that it would
lead latent representations to not be tied to a specific classification
structure.

  * •

The mathematical formalism is not thorough. For the sake of clarity and given
page limitations, we do not include a formal demonstration that the linear
classification optimum necessarily satisfies the clustering objective.
Likewise, we also rely on the reader’s intuition when discussing isotropy in
Section 2.2 (rather than properly deriving it from the relation between the
chord from $\vec{1}$ to $\mathbb{V}(\mathcal{D})$ and the sine of the angle
between $\vec{1}$ and $\mathbb{V}(\mathcal{D})$), and ignore the cosine
denominator.

All of the listed limitations make for good questions to be discussed at
length in future work.

## References

  * Ait-Saada and Nadif (2023) Mira Ait-Saada and Mohamed Nadif. 2023.  Is anisotropy truly harmful? a case study on text clustering.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 1194–1203, Toronto, Canada. Association for Computational Linguistics.
  * Bird and Loper (2004) Steven Bird and Edward Loper. 2004.  NLTK: The natural language toolkit.  In _Proceedings of the ACL Interactive Poster and Demonstration Sessions_ , pages 214–217, Barcelona, Spain. Association for Computational Linguistics.
  * Bowman et al. (2015) Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015.  A large annotated corpus for learning natural language inference.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_ , pages 632–642, Lisbon, Portugal. Association for Computational Linguistics.
  * Cai et al. (2021) Xingyu Cai, Jiaji Huang, Yuchen Bian, and Kenneth Church. 2021.  Isotropy in the contextual embedding space: Clusters and manifolds.  In _International Conference on Learning Representations_.
  * Ding et al. (2022) Yue Ding, Karolis Martinkus, Damian Pascual, Simon Clematide, and Roger Wattenhofer. 2022.  On isotropy calibration of transformer models.  In _Proceedings of the Third Workshop on Insights from Negative Results in NLP_ , pages 1–9, Dublin, Ireland. Association for Computational Linguistics.
  * Ethayarajh (2019) Kawin Ethayarajh. 2019.  How contextual are contextualized word representations? Comparing the geometry of BERT, ELMo, and GPT-2 embeddings.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 55–65, Hong Kong, China. Association for Computational Linguistics.
  * Fellbaum (1998) Christiane Fellbaum. 1998.  _WordNet: An Electronic Lexical Database_.  Bradford Books.
  * Haemmerl et al. (2023) Katharina Haemmerl, Alina Fastowski, Jindřich Libovický, and Alexander Fraser. 2023.  Exploring anisotropy and outliers in multilingual language models for cross-lingual semantic sentence similarity.  In _Findings of the Association for Computational Linguistics: ACL 2023_ , pages 7023–7037, Toronto, Canada. Association for Computational Linguistics.
  * Jung et al. (2023) Euna Jung, Jungwon Park, Jaekeol Choi, Sungyoon Kim, and Wonjong Rhee. 2023.  Isotropic representation can improve dense retrieval.  In _Advances in Knowledge Discovery and Data Mining_ , pages 125–137, Cham. Springer Nature Switzerland.
  * Kingma and Ba (2014) Diederik P. Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization.
  * Liang et al. (2021) Yuxin Liang, Rui Cao, Jie Zheng, Jie Ren, and Ling Gao. 2021.  Learning to remove: Towards isotropic pre-trained BERT embedding.  In _Artificial Neural Networks and Machine Learning – ICANN 2021_ , pages 448–459, Cham. Springer International Publishing.
  * Mickus et al. (2022) Timothee Mickus, Kees Van Deemter, Mathieu Constant, and Denis Paperno. 2022.  Semeval-2022 task 1: CODWOE – comparing dictionaries and word embeddings.  In _Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)_ , pages 1–14, Seattle, United States. Association for Computational Linguistics.
  * Mu and Viswanath (2018) Jiaqi Mu and Pramod Viswanath. 2018.  All-but-the-top: Simple and effective postprocessing for word representations.  In _International Conference on Learning Representations_.
  * Pang and Lee (2004) Bo Pang and Lillian Lee. 2004.  A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts.  In _Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04)_ , pages 271–278, Barcelona, Spain.
  * Paszke et al. (2019) Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019.  Pytorch: an imperative style, high-performance deep learning library.  In _Proceedings of the 33rd International Conference on Neural Information Processing Systems_ , Red Hook, NY, USA. Curran Associates Inc.
  * Pedregosa et al. (2011) F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011.  Scikit-learn: Machine learning in Python.  _Journal of Machine Learning Research_ , 12:2825–2830.
  * Rajaee and Pilehvar (2021a) Sara Rajaee and Mohammad Taher Pilehvar. 2021a.  A cluster-based approach for improving isotropy in contextual embedding space.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_ , pages 575–584, Online. Association for Computational Linguistics.
  * Rajaee and Pilehvar (2021b) Sara Rajaee and Mohammad Taher Pilehvar. 2021b.  How does fine-tuning affect the geometry of embedding space: A case study on isotropy.  In _Findings of the Association for Computational Linguistics: EMNLP 2021_ , pages 3042–3049, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-BERT: Sentence embeddings using Siamese BERT-networks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3982–3992, Hong Kong, China. Association for Computational Linguistics.
  * Rousseeuw (1987) Peter J. Rousseeuw. 1987.  Silhouettes: A graphical aid to the interpretation and validation of cluster analysis.  _Journal of Computational and Applied Mathematics_ , 20:53–65.
  * Rudman and Eickhoff (2023) William Rudman and Carsten Eickhoff. 2023.  Stable anisotropic regularization.
  * Rudman et al. (2022) William Rudman, Nate Gillman, Taylor Rayne, and Carsten Eickhoff. 2022.  IsoScore: Measuring the uniformity of embedding space utilization.  In _Findings of the Association for Computational Linguistics: ACL 2022_ , pages 3325–3339, Dublin, Ireland. Association for Computational Linguistics.
  * Tikhonov et al. (2023) Alexey Tikhonov, Lisa Bylinina, and Denis Paperno. 2023.  Leverage points in modality shifts: Comparing language-only and multimodal word representations.  In _Proceedings of the 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023)_ , pages 11–17, Toronto, Canada. Association for Computational Linguistics.
  * Timkey and van Schijndel (2021) William Timkey and Marten van Schijndel. 2021.  All bark and no bite: Rogue dimensions in transformer language models obscure representational quality.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 4527–4546, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Zhang et al. (2012) Yuli Zhang, Huaiyu Wu, and Lei Cheng. 2012.  Some new deformation formulas about variance and covariance.  In _2012 Proceedings of International Conference on Modelling, Identification and Control_ , pages 987–992.

##  Appendix A Responsible NLP Research Checklist Compliance

Dataset | N. items | N. params.
---|---|---
Pang and Lee (2004) | 10 662 | 4 094 976
through nltk (Bird and Loper, 2004)
Bowman et al. (2015) | 9 842 | 4 987 395
from nlp.stanford.edu
Mickus et al. (2022) | 11 462 | 4 341 004
from codwoe.atilf.fr
Fellbaum (1998) | 2 275 | 690 326
from github.com/altsoph

Table 1: Dataset vs. number of datapoints (N. items) and corresponding number
of trainable parameters (N. params.).

All the datasets and models we used are English and CC-BY or CC-BY-SA, our use
is consistent with the intended use of these resources. We trust original
creators of these resources that they contain no personally identifying data.
Relevant information is available in Table 1; remark we do not split the data
as we are interested on optimization behavior.

Training per model requires between 10 minutes and 1 hour on an RTX3080 GPU;
much of which is in fact devoted to CPU computations for IsoScore values.
Hyperparameters listed correspond to default PyTorch values (Paszke et al.,
2019), no hyperparameter search was carried out. IsoScore is computed with the
pip package IsoScore (Rudman et al., 2022), silhouette scores with scikit-
learn (Pedregosa et al., 2011).