# Remarks on a theorem of Pink in presence of bad reduction

Wojciech Gajda and Marc Hindry

###### Abstract.

In this note we prove new cases of the Mumford-Tate conjecture by extending a
theorem of Richard Pink for abelian varieties without nontrivial endomorphisms
and with bad semistable reduction. We use quadratic pairs introduced by
J.G.Thompson in the seventies, an important tool in the program of classifying
all simple finite groups. Proof of our main result applies the classification
of the quadratic pairs as described by Premet and Suprunenko. Along the way we
reprove and generalize a theorem of Chris Hall on the image of Tate module
representation of abelian variety as above, to all possible values of its
toric dimension.

###### Key words and phrases:

Abelian variety, Galois representation, Mumford-Tate conjecture

###### 2010 Mathematics Subject Classification:

11G10, 14K15

##  1\. Introduction

Let $A$ be a simple abelian variety of dimension $g$ defined over a number
field $K$. Suppose that $A/K$ has bad (split) semistable reduction at some
place $v$ of the field $K$, denote $\mathbb{F}_{v}$ the finite residual field.
Let $\mathcal{A}\rightarrow{\rm spec}(\mathcal{O}_{K})$ be the Néron model;
the connected component of its special fibre lies in an exact sequence of
algebraic ${\bf F}_{v}$-groups:

(1) $0\longrightarrow
T={\mathbb{G}}_{m}^{s}\longrightarrow\mathcal{A}_{v}^{0}\longrightarrow
B_{v}\longrightarrow 0,$

where $B_{v}$ is an abelian variety over $\mathbb{F}_{v}$. We call $s=\dim T$
the toric dimension of $A$ at $v$.

The Mumford-Tate group $\mathop{{\mathrm{MT}}}(A)$ is an algebraic
$\mathbb{Q}$-group (cf. for example [Pi98], for the definition) defined in
terms of the complex structure of $A(\mathbb{C})$, that comes together with a
representation on $V{:=}H^{1}(A(\mathbb{C}),\mathbb{Q})$. When the abelian
variety is of type I, II or III in Albert classification, the Hodge group
$\mathop{\mathrm{Hdg}}(A)$ is the derived group of $\mathop{{\mathrm{MT}}}(A)$
and we have
$\mathop{{\mathrm{MT}}}(A){=}\mathbb{G}_{m}\mathop{\mathrm{Hdg}}(A)$. Denote
$T_{\ell}(A)$ the Tate module and
$V_{\ell}(A){:=}T_{\ell}(A)\otimes_{\mathbb{Z}_{\ell}}\mathbb{Q}_{\ell}$; the
central object of our study is the Galois representation
$\rho_{\ell}:\mathop{\mathrm{Gal}}(\bar{K}/K)\rightarrow\mathop{\mathrm{GL}}(V_{\ell}(A))$.
We denote $G_{\ell}$ the connected component of the Zariski closure of
$\rho_{\ell}(\mathop{\mathrm{Gal}}(\bar{K}/K))$ in
$\mathop{\mathrm{GL}}(V_{\ell}(A))\cong\mathop{\mathrm{GL}}_{2g,\mathbb{Q}_{\ell}}$,
it is an algebraic $\mathbb{Q}_{\ell}$-group and we still denote $\rho_{\ell}$
its representation on $V_{\ell}$. Via the comparison isomorphism
$V_{\ell}{\cong}V\otimes_{\mathbb{Q}}\mathbb{Q}_{\ell}$ we can compare
$\mathop{{\mathrm{MT}}}(A)\otimes_{\mathbb{Q}}\mathbb{Q}_{\ell}$ and
$G_{\ell}$ inside $\mathop{\mathrm{GL}}_{2g,\mathbb{Q}_{\ell}}$. A deep result
due to Borovoï [Bor74], Deligne [Del82] and Piatetski-Shapiro [Pia] shows that
$G_{\ell}$ is an algebraic subgroup of
$\mathop{{\mathrm{MT}}}(A)\otimes_{\mathbb{Q}}\mathbb{Q}_{\ell}$, the Mumford-
Tate conjecture asserts the equality of these two algebraic groups. We refer
the reader to a variety of surveys on the current state of arts of the
Mumford-Tate conjecture, which is available online.

It is known that the Mumford-Tate conjecture holds for abelian varieties over
number fields, if and only if, it holds for simple ones cf. [Com19]. Further,
it is easy to see that replacing $K$ by a finite extension or $A$ by an
isogenous abelian variety does not alter the validity of Mumford-Tate
conjecture; it is a deeper result (cf. [LaPi95], Theorem 4.3) that Mumford-
Tate conjecture does not depend on the chosen prime $\ell$. Hence, without
loosing any generality, we can concentrate on simple abelian varieties over
$K$, assume semistability and pick any convenient $\ell$.

######  Main Theorem.

Let $A$ be an abelian variety defined over a number field $K$. Suppose that
$\mathop{\mathrm{End}}(A)=\mathbb{Z}$ and there is a place of bad semistable
reduction with toric dimension $s$. Then
$\mathop{{\mathrm{MT}}}(A)_{\mathbb{Q}_{\ell}}=\mathop{\mathrm{GSp}}_{2g,\mathbb{Q}_{\ell}}=G_{\ell}$,
and the Mumford-Tate conjecture holds for $A,$ except may be in following
exceptional cases.

  1. (1)

There is an odd integer $r\geq 3$ such that $g=\frac{1}{2}{2r\choose r}$ and
$s={2r-2\choose r-1}$.

  2. (2)

There is an integer $t\geq 4$ with $t{\equiv}0$ or $1\mod 4,$ such that
$g=2^{t}$ and $s=g$ or $g/2$.

We remark that the hypotheses in the theorem are generic in the sense that a
generic abelian variety of dimension $g$ is simple with endomorphism ring
$\mathbb{Z}$, whereas abelian varieties with some bad reduction are far more
numerous that abelian varieties with (potential) good reduction everywhere,
the typical case being for $g=1$ where integers are rarer than rational
numbers. The first instances of the first exceptional case in Main Theorem
occur for $(g,s)=(10,6)$, $(84,70)$, $(1716,924)$ etc.The first instances of
the second exceptional case occur for $(g,s)=(16,8)$, $(16,16)$, $(32,16)$,
$(32,32)$, $(256,128)$, $(256,256)$ etc.

Comparison with former results

Compared to former attemps at the conjecture (which were often based on
computation of Lie algebras of algebraic groups, e.g., [BGK03, BGK06, BGK10])
our method is different. The novelty of this note approach depends on applying
theory of representations of finite groups of Lie type directly to comparison
of special fibres of two Chevalley group schemes in question, and then lifting
equality to their $\mathbb{Q}_{\ell}-$points (cf. Lemma 2.4) in order to
confirm the conjecture. Our main theorem shall be compared to a result of Pink
(under the assumption $\mathop{\mathrm{End}}(A){=}\mathbb{Z}$) and to the
following results of Noot, and Chris Hall.

######  Theorem 1.1.

(Pink [Pi98], Theorem 5.14) Let $A$ be an abelian variety of dimension $g$
with endomorphism ring $\mathbb{Z}$. Assume that $g$ is neither

  * •

half of a $k^{\rm th}$ power for any odd $k>1$, nor

  * •

of the shape $\frac{1}{2}{2m\choose m}$ for any odd $m\geq 3$.

Then $G_{\ell}{=}\mathop{\mathrm{GSp}}_{2g,\mathbb{Q}_{\ell}}$ and the
Mumford-Tate conjecture holds for $A$.

######  Theorem 1.2.

(Noot [No00]) Let $A$ be an abelian variety of dimension 4 with endomorphism
ring $\mathbb{Z}$. Assume that $A$ is defined over a number field and has a
place with bad semistable reduction. Then
$G_{\ell}=\mathop{\mathrm{GSp}}_{8,\mathbb{Q}_{\ell}}$ and the Mumford-Tate
conjecture holds for $A$.

######  Theorem 1.3.

(C. Hall [Ha11]) Let $A$ be an abelian variety of dimension $g$ with
endomorphism ring $\mathbb{Z}$. Assume that $A$ has a place of bad semistable
reduction with toric dimension $s=1.$ Then
$G_{\ell}{=}\mathop{\mathrm{GSp}}_{2g,\mathbb{Q}_{\ell}}$ and the Mumford-Tate
conjecture holds for $A$.

Our proof of Main Theorem uses quadratic pairs introduced by J.G.Thompson in
[Th71]. We apply the classification of the quadratic pairs described by Premet
and Suprunenko in [PS83]. Note that along the way we reprove and generalize
Theorem 1.3 to all possible values of $s.$

##  2\. Quadratic pairs and minuscule representations

######  Definition 2.1.

A quadratic pair $(G,\rho)$ is formed by a group $G$, a faithful irreducible
representation $\rho:G\rightarrow\mathop{\mathrm{GL}}(V)$ such that the subset
$\mathcal{Q}:=\\{g\in G\setminus\\{1\\}\;|\;(\rho(g)-1_{V})^{2}=0\\}$ is non
empty and generates $G$. (This is Thompson’s definition [Th71] when we assume
$G$ finite and $V$ a $\mathbb{F}_{p}$-vector space; otherwise we understand
“generates" as “topologically generates"). A quadratic pair is polarised
(orthogonal or symplectic) if $G$ leaves invariant a symmetric or
antisymmetric non degenerate bilinear form. A quadratic pair is simple if $G$
is equal to its commutator group and the quotient of $G$ by its centre is
simple.

The main properties of quadratic pairs are a decomposition theorem into
(almost) simple blocs and a classification of these simple blocs which are due
to Thompson [Th71], see also [Ho76, Sa76] and complemented by Premet-
Suprunenko in [PS83].

######  Proposition 2.2.

Let $(G,\rho)$ be a quadratic pair with $V$ an $\mathbb{F}_{p}$-vector space.

  1. (1)

(Thompson’s central product theorem) There exist simple quadratic pairs
$(G_{i},\rho_{i})$ such that $G{=}G_{1}\cdots G_{t}$, and $(G,\rho)$ is
isomorphic to $\rho_{1}\otimes\dots\otimes\rho_{t}:G_{1}\cdots
G_{t}\rightarrow\mathop{\mathrm{GL}}(V_{1}\otimes{\dots}\otimes V_{t})$.
Further, if the pair $(G,\rho)$ is polarised, then each $(G_{i},\rho_{i})$ is
polarised.

  2. (2)

(Thompson, Premet-Suprunenko) A simple quadratic pair belongs to the following
list which is given indicating the root system associated to a Chevalley group
and a short (generally accepted) name for the representation.

     * •

$(A_{n},\wedge^{j}Std)$

     * •

$(B_{n},Std)$ or $(B_{n},Spin)$

     * •

$(C_{n},\mathcal{W}_{j})$ with a subrepresentation
$\mathcal{W}_{j}\subseteq\wedge^{j}Std$

     * •

$(D_{n},Std)$ or $(D_{n},Spin^{\pm})$

     * •

Explicitly given finite set of representations of exceptional groups of type
$E_{6}$, $E_{7}$, $F_{4}$ and $G_{2}$.

We will require the following elementary lemma. An element
$g\in\mathop{\mathrm{End}}(V)$ is $k$-unipotent if $(g-1)^{k}=0$ but
$(g-1)^{k-1}\not=0$; for example a quadratic element is, by definition,
$2$-unipotent.

######  Lemma 2.3.

Let $g_{1}$ (resp. $g_{2}$) be a $k_{1}$-unipotent element in
$\mathop{\mathrm{End}}(V_{1})$ (resp. a $k_{2}$-unipotent element in
$\mathop{\mathrm{End}}(V_{2})$), then $g_{1}\otimes g_{2}$ is a
$(k_{1}+k_{2}-1)$-unipotent element in $\mathop{\mathrm{End}}(V_{1}\otimes
V_{2})$. In particular, quadratic elements of a quadratic pair $(G_{1}\cdots
G_{t},\rho_{1}\otimes\cdots\otimes\rho_{t})$ lie in some $G_{i}$.

###### Proof.

See [Pa04], Lemma 2.2.1. ∎

Our typical example of quadratic pair is obtained by considering the Galois
representation on $A[\ell]\cong(\mathbb{Z}/\ell\mathbb{Z})^{2g}$ and
restricting it to the subgroup $R$ generated by inertia and their conjugates.
One sees that this representation is still irreducible for $\ell$ large
enough. Minuscule representation appear when considering the $\ell$-adic
representation.

The link between quadratic pairs and minuscule representations is suggested in
[Se79] (see the remark after Corollaire, page 180), where it is observed that
root elements of a (semi-simple) group act quadratically. To pass from
representations mod $\ell$ to representations over $\mathbb{Z}_{\ell}$ or
$\mathbb{Q}_{\ell}$ we’ll use the following elementary lemma.

######  Lemma 2.4.

Let $G$ be a smooth algebraic subgroup of $\mathop{\mathrm{GL}}_{N}$ over
$\mathbb{Z}_{\ell}$ and $H$ a closed subgroup of $G(\mathbb{Z}_{\ell})$ such
that $\pi(H)=G(\mathbb{F}_{\ell})$, where $\pi$ is the reduction mod $\ell$
map. Assume $\ell\geq 5$ and the Lie algebra of $G_{\mathbb{F}_{\ell}}$ is
spanned by quadratic elements. Then $H=G(\mathbb{Z}_{\ell})$.

###### Proof.

This is a special case of Lemme 2.6 in [HiRa16]. ∎

The lemma applies to $SL_{m}$, to $\mathop{\mathrm{Sp}}_{2m}$ loc.cit. and
more generally to any quadratic representation ! Thus the property of being
spanned by null square matrices is one of the characterisations of minuscule
representation, see for example [Pa04] (definition 2.1.3), [Se79] (Lemme 5 and
Corollaire after Proposition 5).

Below we recall the table of minuscule weights of height one (reference:
[Se79]) giving the type of root system (A,B,C,D), the label of the highest
weight describing the representation (labeled as in loc. cit.), a short name
for the representation, the dimension of the representation and finally the
“sign" of the representation: $+1$ for an orthogonal representation, $-1$ for
a symplectic representation, and $0$ for a non auto-dual representation.

Table of minuscule representations

Lie type | weight | representation | dimension | sign
---|---|---|---|---
$A_{n}$ ($n\geq$ 1) | $\varpi_{j}\;(1\leq j\leq n)$ | $\Lambda^{j}({\rm Std})$ | ${n+1\choose j}$ | $\begin{matrix}(-1)^{j}&{\rm if}\;\;n=2j-1\cr 0&{\rm else}\end{matrix}$
$B_{n}$ ($n\geq$ 2) | $\varpi_{n}$ | ${\rm Spin}$ | $2^{n}$ | $\begin{matrix}+1&{\rm if}\;\;n\equiv 0,3\mod 4\cr-1&{\rm if}\;\equiv 1,2\mod 4\end{matrix}$
$C_{n}$ ($n\geq$ 2) | $\varpi_{1}$ | ${\rm Std}$ | $2n$ | $-1$
$D_{n}$ ($n\geq$ 3) | $\begin{matrix}\varpi_{1}\cr\cr\varpi_{n-1}\cr\varpi_{n}\cr\end{matrix}$ | $\begin{matrix}{\rm Std}\cr\cr{\rm Spin}^{-}\cr{\rm Spin}^{+}\cr\end{matrix}$ | $\begin{matrix}2n\cr\cr 2^{n-1}\cr 2^{n-1}\cr\end{matrix}$ | $\begin{matrix}+1\cr\cr\left\\{\begin{matrix}+1&{\rm if}\;\;n\equiv 0\mod 4\cr-1&{\rm if}\;n\equiv 2\mod 4\cr 0&{\rm if}\;n\;{\rm odd}\cr\end{matrix}\right.\cr\end{matrix}$

##  3\. Proof of Main Theorem

Existence of a place of bad reduction with toric dimension $s$ implies that a
topological generator of (the image of) inertia provides a quadratic element
$g$ with drop equal to $s$. Recall that a quadratic element of $(G,\rho)$ is
an element such that $(\rho(g)-1)^{2}=0$ and the drop is the dimension of the
image of $\rho(g)-1$. Let us denote by $G^{\prime}_{\ell}/\mathbb{Q}_{\ell}$
the derived group of the algebraic monodromy group
$G_{\ell}:=\overline{\rho_{\ell}(\mathop{\mathrm{Gal}}(\bar{K}/K))}^{\rm Zar}$
which, without loss of generality, can be assumed connected. For simplicity we
treat the case $\mathop{\mathrm{End}}(A){=}\mathbb{Z}$ although our methods
apply to more general situations (cf. Section 4). The algebraic group
$G^{\prime}_{\ell}$ and the natural representation
$\rho_{\ell}:G^{\prime}_{\ell}\rightarrow\mathop{\mathrm{GL}}(T_{\ell}(A))\subset\mathop{\mathrm{GL}}_{2g,\mathbb{Q}_{\ell}}$
have very special features.

  * •

The representation is faithful symplectic.

  * •

(Faltings, [Fa83]) The $\ell$-adic representation is absolutely irreducible;
further, for $\ell$ large enough, the representation mod $\ell$ is still
irreducible.

  * •

(Pink, [Pi98]) The representation is a tensor product of representations
$\rho_{i}:G_{i}\rightarrow\mathop{\mathrm{GL}}(V_{i})$, where $G_{i}$ are
absolutely irreducible groups, $G^{\prime}_{\ell}=G_{1}\cdots G_{s}$ (an
almost direct product), $V_{\ell}\cong V_{1}\otimes\cdots\otimes V_{s}$ and
each of these factors is a classical minuscule symplectic representation.
Further, the decomposition is isotypical with a transitive action of the
Galois group.

We distinguish two cases, using quadratic pairs in the first case and
minuscule representations for the second:

  * •

(A) $s>1$ (hence odd and $\geq 3$).

  * •

(B) $s=1$ (i.e. the group $G^{\prime}_{\ell}$ is absolutely irreducible).

Case (A) Assumption on $s$ implies (denoting $m{=}\dim V_{i}$) that
$2g{=}m^{s}$ as in [Pi98], but we will show that, when there is a place of bad
reduction, this case does not occur. We choose $\ell$ large enough so that the
representation modulo $\ell$, which we denote
$\bar{\rho}_{\ell}:\mathop{\mathrm{Gal}}(\bar{K}/K)\rightarrow\mathop{\mathrm{GL}}(A[\ell])\cong\mathop{\mathrm{GL}}_{2g}(\mathbb{F}_{\ell})$,
is still irreducible. Let $\bar{R}$ be the subgroup generated by the image of
the inertia group at the place of bad reduction and its conjugates. We first
claim that the restriction of $\bar{\rho}_{\ell}$ to $\bar{R}$ is still
irreducible. Indeed by Clifford’s classical theorem, the representation is
induced from an irreducible subrepresentation $W$, but $(\bar{R},W)$ forms a
quadratic pair and then $\bar{R}$ is a finite group of Lie type and has no
normal subgroups of small index, at least if $\ell$ is chosen sufficiently
large, therefore $V{=}W$. Next, $\bar{R}$ is generated by the conjugacy class
of a quadratic element, therefore it cannot be an almost direct product
$G_{1}\cdots G_{s}$ with $s{>}1$. The argument for this is that a quadratic
element $g$ has to occur in only one factor, i.e.,
$g{=}(1,\dots,g_{i},\dots,1)\in G_{1}\cdots G_{s}$. We now lift $\bar{R}$ to
$R\subset G^{\prime}_{\ell}$, where $R$ denotes the subgroup generated by
inertia in the $\ell$-adic representation. Since the Galois group permutes the
factors, the inertia cannot sit in only one factor (cf. Lemma 2.3) and we
conclude that $s{=}1$. Note that the case of $\dim A{=}4$ was initially
treated by Noot in Prop. 2.1 and Cor. 2.2, [No00].

Case (B). The list of symplectic minuscule representations of dimension $2g$
of a classical group is short; see the table in previous section; we denote
$(G_{1},\rho_{1})\sim(G_{2},\rho_{2})$ when the two simple algebraic groups
$G_{i}$ have the same root system and $\rho_{i}$ correspond to the same
weight.

  1. (1)

$(G_{\ell}^{\prime},\rho)\sim(\mathop{\mathrm{Sp}}_{2g},Std)$.

  2. (2)

$(G_{\ell}^{\prime},\rho)\sim(\mathop{\mathrm{SL}}_{2r},\wedge^{r}Std)$, with
odd $r\geq 3$ and $2g={2r\choose r}$.

  3. (3)

$(G_{\ell}^{\prime},\rho)\sim(\mathop{\mathrm{SO}}_{2r+1},Spin)$, with
$2g=2^{r}$ and $r\equiv 1,2\mod 4$.

  4. (4)

$(G_{\ell}^{\prime},\rho)\sim(\mathop{\mathrm{SO}}_{2r},Spin^{\pm})$, with
$2g=2^{r-1}$ and $r\equiv 2\mod 4$.

In the first case we conclude
$G^{\prime}_{\ell}=\mathop{\mathrm{Sp}}_{2g,\mathbb{Q}_{\ell}}=\mathop{\mathrm{Hdg}}(A)_{\mathbb{Q}_{\ell}}$.
In the three last cases we use the computation by Premet-Suprunenko of the
drop of a quadratic element (cf. [PS83], Section 3). More precisely, in the
second case we see that $\dim A=\frac{1}{2}{2r\choose r}$ is very specific.
Further, the drop of a non-trivial quadratic element in this representation is
$s:={2r-2\choose r-1}$ (see [PS83], Lemma 18 and the calculation at p.78,
beginning of §2), therefore this case can only happen if the toric dimension
at a bad place is equal to $s$. In the third case we see that $\dim A=2^{r-1}$
is very specific. Further, the drop of a non-trivial quadratic element in this
representation is $s:=2^{r-1}$ or $2^{r-2}$ (see [PS83] Lemma 20), therefore
this case can only happen if the toric dimension at a bad place is equal to
$s=g$ or $g/2$. For the spin representation to be symplectic, we need $r\equiv
1,2\mod 4$. In the fourth case we see that $\dim A=2^{r-2}$ is very specific.
Further, the drop of a non-trivial quadratic element in this representation is
$s:=2^{r-2}$ or $2^{r-3}$ (see [PS83] Lemma 21 and Note 2, at p.86), therefore
this case can only happen, if the toric dimension at a bad place is equal to
$s=g$ or $g/2$. For the half-spin representation to be symplectic, we need
$r\equiv 2\mod 4$. This completes the proof of Main Theorem.

######  Remark 3.1.

Our proof shows that inertia subgroups at bad places generate the derived
group of the image of Galois. Indeed the above arguments show that the
algebraic subgroup $R$ generated by the inertia subgroup and their conjugates
is a non trivial normal subgroup and the only such subgroup of
$\mathop{\mathrm{Sp}}_{2g}$ is $\mathop{\mathrm{Sp}}_{2g}$ itself.

##  4\. Conclusion

As a final remark we add the observation that the same method can handle the
case when the endomorphism algebra
$\mathop{\mathrm{End}}^{0}(A):=\mathop{\mathrm{End}}(A)\otimes\mathbb{Q}$ is a
quaternion algebra with center $\mathbb{Q}$. However, note that the
combinatorics, when the center is larger, is much more involved. We plan to
address this issue in a future work.

######  Theorem 4.1.

Let $A$ be an abelian variety defined over a number field $K$. Suppose that
$\mathop{\mathrm{End}}^{0}(A):=\mathop{\mathrm{End}}(A)\otimes\mathbb{Q}$ is a
quaternion algebra with centre $\mathbb{Q}$ and there is a place of bad
reduction with toric dimension $s$ (note $s$ is always even). Then,

  * •

when the algebra $D$ is indefinite, i.e., $A$ has type II, we have
$\mathop{{\mathrm{MT}}}(A)_{\mathbb{Q}_{\ell}}{=}\mathop{\mathrm{GSp}}_{g,\mathbb{Q}_{\ell}}{=}G_{\ell}$
and the Mumford-Tate conjecture holds for $A,$ except may be in following
exceptional cases:

    1. (1)

there is an odd integer $r\geq 3$ such that $g={2r\choose r}$ and
$s=2{2r-2\choose r-1}$,

    2. (2)

there is an integer $t{\geq 5}$ with $t{\equiv}1$ or $2\mod 4$ such that
$g{=}2^{t}$ and $s{=}g$ or $g/2,$

  * •

when the algebra $D$ is definite, i.e., $A$ has type III, we have
$\mathop{{\mathrm{MT}}}(A)_{\mathbb{Q}_{\ell}}{=}\mathop{\mathrm{GSO}}_{g,\mathbb{Q}_{\ell}}{=}G_{\ell}$
and the Mumford-Tate conjecture holds for $A,$ except may be in following
exceptional cases:

    1. (1)

there is an even integer $r\geq 2$ such that $g={2r\choose r}$ and
$s=2{2r-2\choose r-1}$,

    2. (2)

there is an integer $t{\geq}4$ with $t{\equiv}0$ or $3\mod 4$ such that
$g{=}2^{t}$ and $s{=}g$ or $g/2$.

The extension of Theorem 1.1 (and Theorem 4.1) when $s=2$ to abelian varieties
of type II is worked out in [HiRa16], Théorèmes 10.6 and 10.7.

Acknowledgements. Both authors thank professor Ronald Solomon for his guidance
through the literature on quadratic pairs. The authors were partially
supported by a research grant UMO-2018/31/B/ST1/01474 of the National Centre
of Sciences of Poland.

## References

  * [BGK03] Banaszak, G., Gajda, W., Krasoń, P., On Galois representations for abelian varieties with complex and real multiplications J. Number Theory 100 (2003), 117-132.
  * [BGK06] Banaszak, G.; Gajda, W.; Krasoń, P., On the image of $l$-adic Galois representations for abelian varieties of type I and II. Doc. Math. 2006, Extra Vol., 35–75.
  * [BGK10] Banaszak, G.; Gajda, W.; Krasoń, P., On the image of Galois $l$-adic representations for abelian varieties of type III. Tohoku Math. J. 62 (2010), 163–189.
  * [Bor74] Borovoi, M., The action of the Galois group on the rational cohomology classes of type $(p,p)$ of Abelian varieties. Mat. Sb. (N.S.) 94(136) (1974), 649-652, 656.
  * [Com19] Commelin, J., The Mumford-Tate conjecture for products of abelian varieties. Algebr. Geom. 6 (2019), 650–677.
  * [Del82] Deligne, P., et al.,  Hodge cycles, motives, and Shimura varieties. Lecture Notes in Mathematics, 900. Springer-Verlag, Berlin–New York, 1982.
  * [Fa83] Faltings, G., Endlichkeitssätze für abelsche Varietäten über Zahlkörpern. Inventiones Mathematicae 73, (1983). 349–366.
  * [Ha11] Hall, C., An open image theorem for a general class of abelian varieties. With an appendix by Emmanuel Kowalski.  Bull. Lond. Math. Soc. 43 (2011), 703–711.
  * [HiRa16] Hindry, M.; Ratazzi, N., Torsion pour les variétés abéliennes de type I et II. Algebra & Number Theory 10 (2016), 1845–1891.
  * [Ho76] Ho, C.Y., On the quadratic pairs, J. Algebra 43 (1976), 338–358.
  * [LaPi95] Larsen, M, Pink, R.. Abelian varieties, $\ell$-adic representations, and $\ell$-independence. Math. Ann. 302, (1995), 561–579.
  * [No00] Noot, R.,  Abelian varieties with $\ell$-adic Galois representation of Mumford’s type.  J. für die reine und ang. Math. 519 (2000), 155–169.
  * [Pa04] Paugam, F. ,  Galois representations, Mumford-Tate groups and good reduction of abelian varieties. Math. Ann. 329 (2004), 119–160.
  * [Pia] Piatetski-Shapiro, I. , Interrelations between the Tate and Hodge hypotheses for abelian varieties. Mat. Sb. (N.S.) 85(127) (1971), 610–620.
  * [Pi98] Pink, R.,  $\ell$-adic algebraic monodromy groups cocharacters, and the Mumford-Tate conjecture.  J. für die reine und ang. Math. 495 (1998), 187–237.
  * [PS83] Premet, A.A.; Suprunenko, I.D.,  Quadratic modules for Chevalley groups over fields of odd characteristics. Math. Nachr. 110 (1983) 65–96.
  * [Sa76] Salzberg Stark, B.,  Another Look at Thompson’s Quadratic Pairs, J. of Algebra 45 (1977), 334–342.
  * [Se79] Serre, J.-P.,  Groupes algébriques associés aux modules de Hodge-Tate.  Astérisque 65 (1979),155–188.
  * [Th71] Thompson, J.G.,  Quadratic pairs.  In Actes du Congrès International des Mathématiciens (Nice, 1970), Tome 1, Gauthier-Villars, Paris, 1971, 375-376.

Wojciech Gajda
Faculty of Mathematics and Computer Science
Adam Mickiewicz University
Uniwersytetu Poznańskiego 4
61-614 Poznań, Poland
E-mail adress<EMAIL_ADDRESS>

Marc Hindry
Institut de Mathematiques de Jussieu - Paris rive gauche (Imj-Prg),
Université Paris Cité (Paris Diderot),
Bâtiment Sophie Germain,
Boite Courrier 7012,
8 place Aurélie Nemours,
F-75205 Paris CEDEX 13, France.
E-mail address<EMAIL_ADDRESS>


# Controllable interatomic interaction mediated by diffractive coupling in a
cavity

Ivor Krešić<EMAIL_ADDRESS>Centre for Advanced Laser Techniques, Institute of
Physics, Bijenička cesta 46, 10000, Zagreb, Croatia  Institute for Theoretical
Physics, Vienna University of Technology (TU Wien), Vienna, A–1040, Austria

###### Abstract

Photon-mediated interaction can be used for simulating complex many-body
phenomena with ultracold atoms coupled to electromagnetic modes of an optical
resonator. We theoretically study a method of producing controllable
interatomic interaction mediated by forward-diffracted photons circulating
inside a ring cavity. One example of such a system is the three-mode cavity,
where an on-axis mode can coexist with two diffracted sidebands. We
demonstrate how the self-organized stripe states of a Bose-Einstein condensate
(BEC) occurring in this cavity geometry can exhibit supersolid properties, due
to spontaneous breaking of the Hamiltonian’s continuous translational
symmetry. A numerical study of the collective excitation spectrum of these
states demonstrates the existence of massles and finite-gap excitations, which
are identified as phase (Goldstone) and amplitude (Higgs) atomic density
modes. We further demonstrate how judicious Fourier filtering of intracavity
light can be used to engineer the effective atom-atom interaction profile for
many cavity modes. The numerical results in this configuration show the
existence of droplet array and single droplet BEC states for commensurate and
incommensurate cavity modes, respectively. Diffractive coupling in a cavity is
thereby introduced as a novel route towards tailoring the photon-mediated
interaction of ultracold atoms. Spatial features of the self-organized optical
potentials can here be tuned to scales several times larger than the pump
laser wavelength, such that the corresponding atomic density distributions
could be imaged and manipulated using low numerical aperture optics. These
calculations and insights pave the way towards quantum simulation of exotic
nonequilibrium many-body physics with condensates in a cavity.

Ultracold atoms coupled to modes of an optical cavity can be used for
simulating complex phenomena in condensed matter physics Ritsch  _et al._
(2013); Mivehvar  _et al._ (2021). The effective atom-atom interaction is here
mediated by cavity photons, which has the advantage of being relatively strong
and highly tunable. Throughout the last two decades, various many-body effects
have been explored in this context. These include the Dicke model phase
transition Baumann  _et al._ (2010), supersolidity Léonard  _et al._ (2017a,
b); Schuster  _et al._ (2020), and the occurrence of phonons in a quantum gas
Guo  _et al._ (2021).

It was recently shown that diffraction of light at a cloud of cold atoms can
be used to induce self-organization in atomic density Tesio  _et al._ (2014a);
Labeyrie  _et al._ (2014); Robb  _et al._ (2015); Zhang  _et al._ (2018);
Ackemann  _et al._ (2021) and magnetization Krešić  _et al._ (2018); Labeyrie
_et al._ (2018, 2024), in the single retroreflecting mirror configuration.
This type of ordering relies on the coupling between the atoms being mediated
via photons diffracted by a small angle at the atomic cloud. The work on cold
and ultracold atoms extends the previous theoretical ideas regarding
transverse instabilities in nonlinear media Lugiato and Lefever (1987); Firth
(1990); Firth and Paré (1988), where the emphasis was placed on coupling of
optical waves via the material’s susceptibility.

Diffractive coupling of ultracold atoms could also be achieved using cavities,
which would increase the effective strength of interactions Tesio  _et al._
(2012); Boquete  _et al._ (2021); Ackemann  _et al._ (2021). The corresponding
optical phenomena have been experimentally realized in various nonlinear
media, using plano-planar cavities with intracavity lenses in the nearly self-
imaging configuration, which tunes the system close to mode degeneracy
Esteban-Martin  _et al._ (2004); Esteban-Martín  _et al._ (2005). Moreover, it
was also shown that employing nearly confocal cavities can lead to the same
behavior Kreuzer  _et al._ (1990); Staliūnas  _et al._ (1997); Ackemann  _et
al._ (2000), with the potential of entering the strong cavity-medium coupling
regime.

We here describe a theoretical study of the properties of diffractive coupling
in a one-dimensional Bose-Einstein condensate (BEC) coupled to electromagnetic
modes of a ring cavity. We demonstrate how the profile of photon-mediated
atom-atom interaction can be tailored by filtering the Fourier modes to allow
propagation of light with desired transverse wavelengths. Within the mean
field approximation, laser driving of the on-axis cavity mode can lead to BEC
self-organization into stripe states for a setup with two cavity sidebands. We
explore the supersolid properties of these stripe states arising due to
spontaneous breaking of the U(1) translational symmetry of the system
Hamiltonian. The Bogoliubov theory calculation reveals the presence of phase
(Goldstone) and amplitude (Higgs) modes in atomic density. In the case of many
cavity modes, we numerically demonstrate the existence of droplet arrays and
single droplet states, for commensurate and incommensurate cavity sidebands,
respectively.

Methods of controlling cavity photon-mediated interaction were recently
implemented in cold atoms in a single mode cavity with an applied magnetic
field gradient Periwal  _et al._ (2021), along with ultracold atoms coupled to
modes of a multimode degenerate cavity Vaidya  _et al._ (2018). Another
theoretically proposed method relies on using frequency combs for pumping a
BEC near cavity resonances of multiple longitudinal cavity modes Masalaeva
_et al._ (2023). This Article describes a novel addition to the toolbox for
control of photon-mediated interaction profiles in quantum simulations. The
great advantage compared to methods of Refs. Periwal  _et al._ (2021); Vaidya
_et al._ (2018); Masalaeva  _et al._ (2023) is that the allowed cavity mode
wavevectors can be selected with considerable freedom, such that e.g. modes
with incommensurate transverse wavelengths can be excited, leading to complex
dynamics.

Another advantage stems from the fact that the self-organization mechanism
does not rely on collective superradiant enhancement of scattering into an
optical cavity mode orthogonal to the pump laser Tanji-Suzuki  _et al._
(2011), and the spatial period of the patterns can be made several times
larger than the pump wavelength. Experimental realization of BEC self-
organization at such lengthscales would offer considerable advantages in terms
of optical control and monitoring of the system dynamics as compared to the
superradiant cases described in Ritsch  _et al._ (2013); Mivehvar  _et al._
(2021), where the spatial periods are near the diffraction limit.

##  I System Hamiltonian

The setup consists of a BEC placed inside a nearly degenerate ring cavity,
with a laser of frequency $\omega$ driving an on-axis cavity mode with pump
rate $\eta$. For strong BEC confinement along the $y$\- and $z$-axes, the
cloud has a cigar shape and can be approximated as one-dimensional. Moreover,
for pattern lengthscales several times larger than the light’s wavelength, the
paraxial approximation holds and the reflection of intracavity light from the
BEC can be neglected, i.e. only diffracted waves at a small angle to the pump
beam direction are considered.

Using the Fourier filtering technique for photons inside the cavity Jensen
_et al._ (1998); Labeyrie  _et al._ (2024), one can tailor the configuration
of the allowed cavity sideband modes. The cavity operator is then given by:

$\displaystyle\begin{aligned} \hat{E}(x)=\hat{a}_{0}+\sum_{q_{j}\in
S}\left(\hat{a}_{j+}e^{iq_{j}x}+\hat{a}_{j-}e^{-iq_{j}x}\right),\end{aligned}$
(1)

where $\hat{a}_{0}$ is the annihilation operator of the on-axis mode and
$\hat{a}_{j\pm}$ are the annihilation operators of the diffracted sideband
modes with transverse wavenumbers $\pm q_{j}=\pm 2\pi/\Lambda_{j}$. The sum
goes over $q_{j}$ values belonging to a discrete and finite set $S$.

The effective many-body Hamiltonian for the atomic and cavity degrees of
freedom can be derived from the Jaynes-Cummings model, as described in
Appendix A. This effective Hamiltonian has the form ($\hbar=1$):

$\displaystyle\begin{aligned}
\hat{H}_{eff}&=-\Delta_{c}\hat{a}_{0}^{\dagger}\hat{a}_{0}-\sum_{j}\Delta_{c,j}(\hat{a}_{j+}^{\dagger}\hat{a}_{j+}+\hat{a}_{j-}^{\dagger}\hat{a}_{j-})+i\eta(\hat{a}_{0}^{\dagger}-\hat{a}_{0})\\\
+&\int\hat{\psi}^{\dagger}(x)\left[-\frac{1}{2m}\frac{\partial^{2}}{\partial
x^{2}}+V_{1}(x)+U_{0}\hat{E}^{\dagger}(x)\hat{E}(x)\right]\hat{\psi}(x)dx,\end{aligned}$
(2)

where $\hat{\psi}(x)$ is the atomic field operator, $V_{1}(x)$ is the external
potential, $m$ is the mass of an atom,
$\Delta_{c}=\omega-\omega_{0},\;\Delta_{c,j}=\omega-\omega_{c,j}$ are the pump
detunings from the on-axis and $j$-th sideband cavity mode, respectively,
while $U_{0}=g_{0}^{2}/\Delta_{a}$ is the single atom light shift,
$\Delta_{a}=\omega-\omega_{a}$ is the laser-atom detuning, and $g_{0}$ is the
atom-cavity coupling strength. The number of atoms $N$ is in the following
calculations kept constant throughout the dynamical evolution.

The Hamiltonian describing the effective atom-atom interaction mediated by
diffractive coupling via the cavity photons, is given by:

$\displaystyle\begin{aligned}
\hat{H}_{eff,at}&=\int\hat{\psi}^{\dagger}(x)\left[-\frac{1}{2m}\frac{\partial^{2}}{\partial
x^{2}}+V_{1}(x)\right]\hat{\psi}(x)dx\\\
&+\frac{1}{2}\int\hat{\psi}^{\dagger}(x^{\prime})\hat{\psi}^{\dagger}(x)V_{2}(x,x^{\prime})\hat{\psi}(x)\hat{\psi}(x^{\prime})dxdx^{\prime},\end{aligned}$
(3)

where the effective atom-atom interaction is given by:

$\displaystyle\begin{aligned}
V_{2}(x,x^{\prime})&=\frac{4U_{0}^{2}\eta^{2}}{\bar{\Delta}_{c}^{2}+\kappa^{2}}\sum_{q_{j}\in
S}\frac{\bar{\Delta}_{c,j}}{\bar{\Delta}_{c,j}^{2}+\kappa^{2}}\cos[q_{j}(x-x^{\prime})],\end{aligned}$
(4)

and $\bar{\Delta}_{c}=\omega-\omega_{0}-NU_{0}$,
$\bar{\Delta}_{c,j}=\omega-\omega_{c,j}-NU_{0}$. The effective atom-atom
interaction $V_{2}(x,x^{\prime})$ can thus be controlled by Fourier filtering
to select a desired set of modes $S$.

In the following, we first explore the simple case of 2 cavity sidebands,
where global all-to-all coupling between the atoms occurs. We demonstrate how
such a setup can be used to simulate the phenomenon of supersolidity. We
extend these studies to a multimode situation with 8 cavity sidebands, which
enables us to explore the relationship between commensurability of the cavity
modes and the atomic ground state solution.

##  II Supersolid properties of self-organized stripe states

Supersolids are an intriguing phase of matter with coexisting spatial order
and superfluidity. The phenomenon has been theoretically studied during the
last 50 years Andreev and Lifshits (1969); Chester (1970); Leggett (1970);
Prokof’ev and Svistunov (2005); Toner (2008); Reppy (2010); Boninsegni and
Prokof’ev (2012), however, conclusive experimental observations in the context
of condensed matter physics are still lacking Kim and Chan (2012); Xiang  _et
al._ (2024).

Several experiments have recently shown that novel insights into supersolidity
can be gleaned by quantum simulation with ultracold atoms Bloch  _et al._
(2008); Mivehvar  _et al._ (2021); Daley  _et al._ (2022). For a Bose-Einstein
condensate (BEC), the global U(1) phase symmetry is spontaneously broken by
condensation Bogoliubov (1947), and spontaneous breaking of the continuous
spatial (translational and/or rotational) symmetry via crystallization can
lead to supersolidity. The types of interactions employed for simulating
supersolidity in BECs range from cavity photon-mediated Gopalakrishnan  _et
al._ (2010); Léonard  _et al._ (2017a); Mivehvar  _et al._ (2018); Schuster
_et al._ (2020); Guo  _et al._ (2021); Masalaeva  _et al._ (2023) and strong
magnetic dipolar Chomaz  _et al._ (2019); Böttcher  _et al._ (2019); Tanzi
_et al._ (2019); Guo  _et al._ (2019); Norcia  _et al._ (2021); Bland  _et
al._ (2022); Recati and Stringari (2023) to synthetic spin-orbit Li  _et al._
(2017). Although such analog systems provide only simplified versions of the
actual interactions present in condensed matter materials, their study can add
valuable insight into the properties of the elusive supersolid phase
Gopalakrishnan  _et al._ (2010).

Below we study photon-mediated supersolidity of the self-organized stripe
states in the ring cavity setup, depicted in Fig. 1a). The cavity operator is
now given by:

$\displaystyle\hat{E}_{st}(x)=\hat{a}_{0}+\hat{a}_{+}e^{iq_{c}x}+\hat{a}_{-}e^{-iq_{c}x},$
(5)

where $\hat{a}_{0}$ is the on-axis mode and $\hat{a}_{\pm}$ the sideband mode
photonic annihilation operators. The two sidebands have transverse wavenumbers
$\pm q_{c}=\pm 2\pi/\Lambda_{c}$, and detunings
$\Delta_{c}^{\prime}=\omega-\omega_{0}^{\prime}$.

Figure 1: Self-organization and supersolidity in the configuration with three
cavity modes. (a) An effectively one-dimensional ultracold Bose-Einstein
condensate (BEC) is placed inside a ring cavity with a linewidth $\kappa$,
which is pumped by coherent on-axis light with drive amplitude $\eta$ (blue
arrow). A Fourier filtering (FF) stage ensures propagation of only three
cavity modes. Upon self-organization, the BEC density distribution
$|\psi(x)|^{2}$ orders into a stripe pattern with a pattern phase displacement
$\Delta x$, which can be defined as the distance of the first maximum from the
origin $x=0$. The value of $\Delta x$ is selected via spontaneous symmetry
breaking for a particular pattern realization, and depends on the atomic
sideband phase $\mbox{arg}\phi$. Pattern lengthscale $\Lambda_{c}$ is tunable
via Fourier filtering. Effective potential for the atoms $V(\phi)$ as a
function of the complex parameter $\phi$ has (b) a circular paraboloid shape
at laser pump strengths below self-organization threshold
($\eta=0.9\eta_{c}$), and (c) a “sombrero" shape above threshold
($\eta=1.5\eta_{c}$), which is a direct consequence of the U(1) translational
symmetry of the system Hamiltonian. The dashed line indicates the potential
minima at $|\phi_{0}|=0.373$. Parameters:
$(\bar{\Delta}_{c},\;\bar{\Delta}_{c}^{\prime},\;u_{0})=(2,\;-1,\;-1)\omega_{R}$.

The many-body Hamiltonian is here given by:

$\displaystyle\hat{H}_{st}=\hat{H}_{ph,st}+\int\hat{\psi}^{\dagger}(x)\hat{H}_{at,st}^{(1)}\hat{\psi}(x)dx,$
(6)

where
$\hat{H}_{ph,st}=-\Delta_{c}\hat{a}_{0}^{\dagger}\hat{a}_{0}-\Delta_{c}^{\prime}(\hat{a}_{+}^{\dagger}\hat{a}_{+}+\hat{a}_{-}^{\dagger}\hat{a}_{-})+i\eta(\hat{a}_{0}^{\dagger}-\hat{a}_{0})$
and the single atom Hamiltonian is given by:

$\displaystyle\begin{aligned}
\hat{H}_{at,st}^{(1)}&=-\frac{1}{2m}\frac{\partial^{2}}{\partial
x^{2}}+U_{0}[\hat{a}_{0}^{\dagger}\hat{a}_{0}+\hat{a}_{+}^{\dagger}\hat{a}_{+}+\hat{a}_{-}^{\dagger}\hat{a}_{-}+(\hat{a}_{0}^{\dagger}\hat{a}_{+}\\\
+&\hat{a}_{-}^{\dagger}\hat{a}_{0})e^{iq_{c}x}+(\hat{a}_{0}^{\dagger}\hat{a}_{-}+\hat{a}_{+}^{\dagger}\hat{a}_{0})e^{-iq_{c}x}+\hat{a}_{-}^{\dagger}\hat{a}_{+}e^{2iq_{c}x}\\\
+&\hat{a}_{+}^{\dagger}\hat{a}_{-}e^{-2iq_{c}x}].\end{aligned}$ (7)

Note that $\hat{H}_{st}$ is symmetric to continuous translations by distance
$d$ along the $x$ axis, i.e. transformations of the form
$\hat{\psi}(x)\to\hat{\psi}(x+d)$, $\hat{E}(x)\to\hat{E}(x+d)$. In the
thermodynamic mean field limit, the continuous translational symmetry is
spontaneously broken by the system selecting a random relative phase between
the cavity sideband modes, and the phase of the periodic atomic density
distribution. Note that in this case $V_{2}(x,x^{\prime})$ varies spatially as
$\cos[q_{c}(x-x^{\prime})]$. This is analogous to the case of superradiant
self-organization in a ring cavity Mivehvar  _et al._ (2018).

Simultaneous breaking of the global U(1) phase symmetry by Bose-Einstein
condensation Bogoliubov (1947), and the U(1) translational symmetry by photon-
mediated structuring into a density pattern means that the self-organized
state of the BEC can display supersolid properties Gopalakrishnan  _et al._
(2010), as studied below. For the fully quantum description, this U(1)
translational symmetry of $\hat{H}_{st}$ leads to the preservation of
transverse momentum of photons and atoms by the nonlinear scattering
processes, which can in the transient regime lead to quantum correlations
Krešić  _et al._ (2023).

###  II.1 Three-mode optomechanical model

In the three-mode optomechanical model Szirmai  _et al._ (2010), the
$\hat{\psi}(x)$ is for this setup given by:

$\displaystyle\hat{\psi}(x)=\frac{1}{\sqrt{L}}\left(\hat{b}_{0}+\hat{b}_{+}e^{iq_{c}x}+\hat{b}_{-}e^{-iq_{c}x}\right),$
(8)

where $\hat{b}_{j}$ is the bosonic annihilation operator of the $j$-th
transverse atomic momentum mode and $L$ is the length of the 1D cloud.
Inserting relations (5) and (8) into (6) and performing the integration over
the BEC cloud length $L$, one obtains the three-mode Hamiltonian
$\hat{H}_{3M}=\hat{H}_{0}+\hat{H}_{FWM}$. The noninteracting part is
$\hat{H}_{0}=-\bar{\Delta}_{c}\hat{a}_{0}^{\dagger}\hat{a}_{0}-\bar{\Delta}_{c}^{\prime}(\hat{a}_{+}^{\dagger}\hat{a}_{+}+\hat{a}_{-}^{\dagger}\hat{a}_{-})+\omega_{R}(\hat{b}_{+}^{\dagger}\hat{b}_{+}+\hat{b}_{-}^{\dagger}\hat{b}_{-})+i\eta(\hat{a}_{0}^{\dagger}-\hat{a}_{0})$,
where $\bar{\Delta}_{c}=\Delta_{c}-NU_{0}$,
$\bar{\Delta}_{c}^{\prime}=\Delta_{c}^{\prime}-NU_{0}$ and
$\omega_{R}=q_{c}^{2}/2m$ is the transverse photon-recoil energy, while the
four-wave mixing part $\hat{H}_{FWM}$ has the form:

$\displaystyle\begin{aligned}
\hat{H}_{FWM}&=U_{0}[(\hat{a}_{+}^{\dagger}\hat{b}_{-}^{\dagger}+\hat{a}_{-}^{\dagger}\hat{b}_{+}^{\dagger})\hat{a}_{0}\hat{b}_{0}\\\
+&\hat{a}_{0}^{\dagger}(\hat{b}_{+}^{\dagger}\hat{a}_{+}+\hat{b}_{-}^{\dagger}\hat{a}_{-})\hat{b}_{0}+\hat{a}_{+}^{\dagger}\hat{a}_{-}\hat{b}_{-}^{\dagger}\hat{b}_{+}]+\mbox{H.c.}.\end{aligned}$
(9)

The details of the derivation of the Hamiltonian $\hat{H}_{3M}$ are given in
Krešić  _et al._ (2023).

We now derive the expression for the expectation value of the Hamiltonian,
also called the effective potential $V(\phi)$ for a complex order parameter
$\phi$ Pekker and Varma (2015); Léonard  _et al._ (2017b); Guo  _et al._
(2019), from the microscopic Hamiltonian $\hat{H}_{3M}$ in the mean field
thermodynamic limit, where
$\langle\hat{O}_{1}\hat{O}_{2}\hat{O}_{3}\hat{O}_{4}\rangle\to\langle\hat{O}_{1}\rangle\langle\hat{O}_{2}\rangle\langle\hat{O}_{3}\rangle\langle\hat{O}_{4}\rangle$.

Writing
$\langle\hat{a}_{j}\rangle\to\sqrt{N}\alpha_{j},\>\langle\hat{b}_{j}\rangle\to\sqrt{N}\beta_{j}$
and neglecting the fourth order terms in the sidebands, which is justified
near the self-organization threshold, one can calculate the energy in the
photon-atom system in the mean field limit:

$\displaystyle\langle\hat{H}_{3M}\rangle/N$
$\displaystyle=-\bar{\Delta}_{c}|\alpha_{0}|^{2}-\bar{\Delta}_{c}^{\prime}(|\alpha_{+}|^{2}+|\alpha_{-}|^{2})+i\frac{\eta}{\sqrt{N}}(\alpha_{0}^{*}-\alpha_{0})$
(10) $\displaystyle+$
$\displaystyle\omega_{R}(|\beta_{+}|^{2}+|\beta_{-}|^{2})+u_{0}[\alpha_{0}\beta_{0}(\alpha_{+}^{*}\beta_{-}^{*}+\alpha_{-}^{*}\beta_{+}^{*})$
$\displaystyle+$
$\displaystyle\alpha_{0}^{*}\beta_{0}(\alpha_{+}\beta_{+}^{*}+\alpha_{-}\beta_{-}^{*})+\mbox{c.c.}],$

with $u_{0}=NU_{0}$. As
$|\bar{\Delta}_{c}|,\>|\bar{\Delta}_{c}^{\prime}|\gg\omega_{R}$, the photonic
degrees of freedom adiabatically follow the atomic ones, and by setting
$\frac{\partial\langle\hat{H}_{3M}\rangle}{\partial\alpha_{i}^{*}}=0$ we
adiabatically eliminate them to get:

$\displaystyle\alpha_{0}=\frac{i\eta}{\sqrt{N}\bar{\Delta}_{c}},\>\alpha_{\pm}=\frac{u_{0}}{\bar{\Delta}_{c}^{\prime}}\alpha_{0}\beta_{0}(\beta_{\pm}+\beta_{\mp}^{*}),$
(11)

where we have chosen a real-valued
$\beta_{0}=\sqrt{1-|\beta_{+}|^{2}-|\beta_{-}|^{2}}$, while for $\alpha_{0}$
we have neglected the terms
$\propto\frac{u_{0}}{\bar{\Delta}_{c}^{\prime}}\alpha_{\pm}^{(*)}\beta_{\pm}^{(*)}$,
as $|u_{0}|\ll|\bar{\Delta}_{c}^{\prime}|$ and $\eta$ is near the self-
organization threshold. Writing
$\frac{\partial\langle\hat{H}_{3M}\rangle}{\partial\beta_{\pm}^{*}}=0$, one
finds that in equilibrium $\beta_{+}=\beta_{-}^{*}$.

We now introduce the complex mean field order parameter
$\phi=\phi_{1}+i\phi_{2}=\beta_{+}=\beta_{-}^{*}$. Inserting the relations
(11) into the expression (10), one finds that the effective potential
$V(\phi)=\langle\hat{H}_{3M}\rangle_{eq}/N$ is rotationally symmetric, and has
a Coleman-Weinberg form Coleman and Weinberg (1973); Kardar (2007); Bass  _et
al._ (2021) for a homogeneous (long-wavelength) order parameter $\phi$, given
by:

$\displaystyle V(\phi)=h_{0}+h_{1}|\phi|^{2}+\frac{h_{2}}{2}|\phi|^{4},$ (12)

with parameters:

$\displaystyle
h_{0}=\frac{\eta^{2}}{N\bar{\Delta}_{c}},\>h_{1}=2\omega_{R}\left(1-\frac{\eta^{2}}{\eta_{c}^{\prime
2}}\right),\>h_{2}=8\omega_{R}\frac{\eta^{2}}{\eta_{c}^{\prime 2}}$ (13)

where the critical pump strength $\eta_{c}^{\prime}$ is given by
$\eta_{c}^{\prime}=\sqrt{-\omega_{R}\bar{\Delta}_{c}^{\prime}\bar{\Delta}_{c}^{2}/(4NU_{0}^{2})}$.
Note that $\eta_{c}^{\prime}$ is equal to the critical pump strength
$\eta_{c}$ Krešić  _et al._ (2023):

$\displaystyle\eta_{c}=\sqrt{\frac{-\omega_{R}(\bar{\Delta}_{c}^{2}+\kappa^{2})(\bar{\Delta}_{c}^{\prime
2}+\kappa^{2})}{4NU_{0}^{2}\bar{\Delta}_{c}^{\prime}}},$ (14)

for vanishing cavity photon decay rate $\kappa$. As show in Fig. 1c,d), for
$h_{1}>0$ ($\eta<\eta_{c}^{\prime}$), $V(\phi)$ is a paraboloid, while for
$h_{1}<0$ ($\eta>\eta_{c}^{\prime}$) it deforms into a “sombrero" shape, with
minima at $|\phi|=\phi_{0}$, where:

$\displaystyle\phi_{0}=\sqrt{-\frac{h_{1}}{h_{2}}}=\frac{1}{2}\sqrt{\left(1-\frac{\eta_{c}^{\prime
2}}{\eta^{2}}\right)}.$ (15)

The “sombrero" shape of $V(\phi)$ above self-organization threshold is a
direct consequence of the continuous translational symmetry of the
Hamiltonian. After spontaneous breaking of this U(1) translational symmetry,
the system Hamiltonian can support the existence of two collective
excitations: one gapless related to fluctuations at constant $|\phi|$
(Goldstone or phase mode), and one with a finite energy gap related to
fluctuations at a constant $\arg\phi$ (Higgs or amplitude mode) Pekker and
Varma (2015); Léonard  _et al._ (2017b); Guo  _et al._ (2019). The
simultaneous presence of Goldstone and Higgs modes is characteristic of
supersolidity, and is determined below by calculating the collective
excitation spectrum using Bogoliubov-de Gennes theory.

The dynamical equations for the cavity photons and the three atomic modes can
be calculated by considering the Lindblad-like evolution equations for the
photon mode fields $\alpha_{0}(t),\alpha_{\pm}(t)$ and the Heisenberg equation
for the three atomic motional modes $\beta_{0}(t),\beta_{\pm}(t)$, which leads
to Krešić  _et al._ (2023):

$\displaystyle\begin{aligned} \frac{\partial\alpha_{0}}{\partial
t}&=(i\bar{\Delta}_{c}-\kappa)\alpha_{0}-iu_{0}[(\beta_{+}^{*}\alpha_{+}+\beta_{-}^{*}\alpha_{-})\beta_{0}\\\
&+\beta_{0}^{*}(\alpha_{+}\beta_{-}+\alpha_{-}\beta_{+})]+\frac{\eta}{\sqrt{N}},\\\
\frac{\partial\alpha_{\pm}}{\partial
t}&=(i\bar{\Delta}_{c}^{\prime}-\kappa)\alpha_{\pm}-iu_{0}[(\beta_{\mp}^{*}\beta_{0}+\beta_{0}^{*}\beta_{\pm})\alpha_{0}+\alpha_{\mp}\beta_{\mp}^{*}\beta_{\pm}],\end{aligned}$
(16)

and

$\displaystyle\begin{aligned} \frac{\partial\beta_{0}}{\partial
t}&=-iu_{0}[\alpha_{0}^{*}(\alpha_{+}\beta_{-}+\alpha_{-}\beta_{+})+(\alpha_{+}^{*}\beta_{+}+\alpha_{-}^{*}\beta_{-})\alpha_{0})],\\\
\frac{\partial\beta_{\pm}}{\partial
t}&=-i\omega_{R}\beta_{\pm}-iu_{0}[(\alpha_{\mp}^{*}\alpha_{0}+\alpha_{0}^{*}\alpha_{\pm})\beta_{0}+\alpha_{\mp}^{*}\alpha_{\pm}\beta_{\mp}].\end{aligned}$
(17)

These equations can be used to study the dynamics of the system near the
instability threshold $\eta_{c}$.

Figure 2: Dynamical evolution after switching on the pump laser for (a) the
atomic probability density $|\psi(x,t)|^{2}$ and (b) the electric field
intensity profile $I(x,t)$, normalized to the steady state value of the on-
axis mode intensity $I_{0}=|\alpha_{0}^{0}|^{2}$. The on-axis intracavity
field initially grows on the scale $\sim 1/\kappa$ (not discernible) and
approxmately reaches its steady state value $\alpha_{0}^{0}$. This homogeneous
solution is not stable, and stripe patterns begin to form, with the system
undergoing sloshing oscillations before reaching the steady state, where the
atoms are bunched into maxima of light intensity, due to $\Delta_{a}<0$. The
phase of the self-organized pattern was set to 0 for clarity, but can attain
any value between 0 and $2\pi$ due to continuous translational symmetry of
$\hat{H}_{eff}$. Simulation parameters: $\eta=1.5\eta_{c}$,
$(\bar{\Delta}_{c},\;\bar{\Delta}_{c}^{\prime},\;u_{0},\;\kappa)=(10,\;-5,\;-1,\;5)\omega_{R}$.

In Fig. 2, we plot the temporal evolution of atomic probability distribution
along the $x$-axis, given by
$|\psi(x,t)|^{2}=|\beta_{0}(t)+\beta_{+}(t)\exp(iq_{c}x)+\beta_{-}(t)\exp(-iq_{c}x)|^{2}/L$,
and the photon field profile, given by the intensity
$I(x,t)=|\alpha_{0}(t)+\alpha_{+}(t)\exp(iq_{c}x)+\alpha_{-}(t)\exp(-iq_{c}x)|^{2}$,
calculated by solving the Eqs. (16,17). After switching on the pump laser at
$t=0$, the atomic probability distribution is homogeneous, and the
$\alpha_{0}$ quickly increases to approximately its steady state value after a
time $1/\kappa$. This homogeneous solution soon becomes unstable, and the
stripe pattern begins to form. At the onset of the instability, the system
undergoes sloshing motion, marked by continuous oscillation between the
bunched and homogeneous atomic and electric field pattern, studied e.g. in
Tesio  _et al._ (2014b). After the oscillations decay, a striped steady state
is reached, with atoms bunching into the peaks of the electric field intensity
due to the drive being red-detuned from the optical transition of the atoms,
i.e. $U_{0}<0$.

###  II.2 Gross-Pitaevskii-type equation

Following the approach of Nagy  _et al._ (2008); Mivehvar  _et al._ (2018), we
here move away from the three-mode model, which leads to equations of the
Gross-Pitaevskii type for the atomic wavefunction. Using the same
approximations as above with the Hamiltonian $\hat{H}_{st}$, we get the
dynamical equations for the photon mode fields $\alpha_{0}(t),\alpha_{\pm}(t)$
and atomic wavefunction $\psi(x,t)$:

$\displaystyle\begin{aligned} \frac{\partial\alpha_{0}}{\partial
t}&=(i\bar{\Delta}_{c}-\kappa)\alpha_{0}-iu_{0}[\alpha_{+}\mathcal{I}_{+}+\alpha_{-}\mathcal{I}_{-}]+\frac{\eta}{\sqrt{N}},\\\
\frac{\partial\alpha_{\pm}}{\partial
t}&=(i\bar{\Delta}_{c}^{\prime}-\kappa)\alpha_{\pm}-iu_{0}[\alpha_{0}\mathcal{I}_{\mp}+\alpha_{\mp}\mathcal{I}_{2\mp}],\\\
i\frac{\partial\psi}{\partial
t}&=-\frac{1}{2m}\frac{\partial^{2}\psi}{\partial
x^{2}}+u_{0}\left[|\alpha_{0}|^{2}+|\alpha_{+}|^{2}+|\alpha_{-}|^{2}\right.\\\
+&(\alpha_{0}^{*}\alpha_{-}+\alpha_{+}^{*}\alpha_{0})e^{-iq_{c}x}+\alpha_{+}^{*}\alpha_{-}e^{-2iq_{c}x}\\\
+&\left.(\alpha_{0}^{*}\alpha_{+}+\alpha_{-}^{*}\alpha_{0})e^{iq_{c}x}+\alpha_{-}^{*}\alpha_{+}e^{2iq_{c}x}\right]\psi,\end{aligned}$
(18)

where the integrals $\;\mathcal{I}_{\pm}=\int e^{\pm
iq_{c}x}|\psi(x,t)|^{2}dx$, $\mathcal{I}_{2\pm}=\int e^{\pm
2iq_{c}x}|\psi(x,t)|^{2}dx$ quantify the ordering of atoms into the optical
potential minima.

We find the stationary ground state solution $e^{-i\mu_{0}t}\psi_{0}(x)$ of
Eqs. (18) by self-consistently solving the equations via setting
$\partial_{t}\alpha_{j}=0$ for $j=0,\pm$ and $i\partial_{t}\psi=\mu\psi$
Masalaeva  _et al._ (2023). Neglecting the BEC medium boundary effects, the
$\Lambda_{c}$ periodicity of the optical potential arising from the cavity
photon modes in Eq. (5) allows us to solve the equations on a box of length
$\Lambda_{c}$, applying periodic boundary conditions.

In Fig. 3, we compare the steady-state results for the three mode model and
the full system of Eqs. (18). Both the sideband fields $\alpha_{\pm}^{0}$ and
the integrals $\mathcal{I}_{\pm}^{0},\mathcal{I}_{2\pm}^{0}$ vanish below
threshold. For pump rates $\eta\gtrsim\eta_{c}$, the quantities acquire finite
values, which are approximately equal for the three-mode model and the Eqs.
(18). For these $\eta$ values, both the electric field intensity distributions
and the probability distributions are also approximately equal for the two
models. The three-mode model gives thus a very good approximation of the
system behavior in this regime.

Figure 3: Steady state values of (a) $|\alpha_{\pm}^{0}|$ and (b)
$|\mathcal{I}_{\pm}^{0}|$, $|\mathcal{I}_{2\pm}^{0}|$ versus $\eta$. The
insets depict the (a) intracavity electric field intensity $I(x)/I_{0}$ and
(b) probability distribution $|\psi_{0}(x)|^{2}$, where for (i)
$\eta=1.2\eta_{c}$ and for (ii) $\eta=1.8\eta_{c}$, with the phase of the
self-organized pattern set to 0 for clarity. The solid lines in all subplots
represent the results for the steady state solutions of the Eqs. (18), while
the dashed lines are for the solutions of the three-mode model, where
$\mathcal{I}_{\pm}^{0}=\beta_{\mp}^{0}(\beta_{0}^{0})^{*}+\beta_{0}^{0}(\beta_{\pm}^{0})^{*}$,
and $\mathcal{I}_{2\pm}^{0}=\beta_{\mp}^{0}(\beta_{\pm}^{0})^{*}$. Simulation
parameters:
$(\bar{\Delta}_{c},\;\bar{\Delta}_{c}^{\prime},\;u_{0},\;\kappa)=(119,\;-89,\;-1,\;50)\omega_{R}$.

Increasing $\eta$ higher above threshold, the modulation depth of the self-
organized cavity photon standing wave becomes quadratically larger, which can
be seen from the inset of Fig. 3a) by noting that
$I_{0}\approx\eta^{2}/N/(\bar{\Delta}_{c}^{2}+\kappa^{2})$. The atoms are thus
more strongly localized in the optical potential wells, and their probability
distribution $|\psi_{0}(x)|^{2}$ becomes noticeably different than the
prediction for the three-mode approximation. This is also reflected in the
increase of discrepancy of the $|\alpha_{\pm}^{0}|$, $|\mathcal{I}_{\pm}^{0}|$
and $|\mathcal{I}_{2\pm}^{0}|$ values for the two models. Including higher
order atomic momenta thus becomes necessary for quantitatively describing the
steady state system behavior for large $\eta/\eta_{c}$ values.

###  II.3 Collective excitation spectrum

We here use the Bogoliubov-de Gennes approach outlined in Nagy  _et al._
(2008); Szirmai  _et al._ (2010); Masalaeva  _et al._ (2023) to calculate the
collective excitation spectrum. We consider solutions of the form
$\psi(x,t)=e^{-i\mu_{0}t}[\psi_{0}(x)+\delta\psi(x,t)]$,
$\alpha_{j}(t)=\alpha_{j}^{0}+\delta\alpha_{j}(t)$, where
$\delta\psi(x,t)=\delta\psi^{+}(x)e^{-i\Omega
t}+[\delta\psi^{-}(x)]^{*}e^{i\Omega^{*}t}$ and
$\delta\alpha_{j}(t)=\delta\alpha_{j}^{+}e^{-i\Omega
t}+[\delta\alpha_{j}^{-}]^{*}e^{i\Omega^{*}t}$, for $j=0,\pm$. Inserting these
relations into Eq. (18) and collecting the $e^{-i\Omega t}$,
$e^{i\Omega^{*}t}$ terms, the collective excitation spectrum is now determined
by the eigenvalue equation:

$\displaystyle\hat{\mbox{M}}\mathbf{R}$ $\displaystyle=\Omega\mathbf{R},$ (19)

with the non-Hermitian excitation matrix $\hat{\mbox{M}}$ given by Eq. (A18),
and the excitation vector defined as:

$\mathbf{R}=[\delta\alpha_{0}^{+},\delta\alpha_{0}^{-},\delta\alpha_{+}^{+},\delta\alpha_{+}^{-},\delta\alpha_{-}^{+},\delta\alpha_{-}^{-},\delta\psi^{+}(x),\delta\psi^{-}(x)]^{\top}.$
(20)

We numerically calculate the spectrum of $\hat{\mbox{M}}$ by using a spatial
grid of length $\Lambda_{c}$ with 256 points and periodic boundary conditions.
Due to the symmetry properties of $\hat{\mbox{M}}$, its eigenvalues appear in
pairs with equal imaginary parts and real parts with opposite signs Nagy  _et
al._ (2008); Szirmai  _et al._ (2010), and we here plot only the part of the
spectrum with positive real parts.

Figure 4: Real parts of the lowest 6 eigenvalues of the collective excitation
matrix $\hat{\mbox{M}}$ vs. pump rate $\eta$. The plotted modes represent
condensate density excitations at frequencies
$\omega_{R},4\omega_{R},9\omega_{R}$ for $\eta\ll\eta_{c}$, and the lowest
energy photon modes appear at
$\mbox{Re}(\Omega)=-\bar{\Delta}_{c}^{\prime}=89\omega_{R}$. Above threshold,
one of the two lowest modes becomes gappless (blue, solid), indicating
Goldstone mode behavior, while the other mode obtains an energy gap increasing
with $\eta$ (orange, dashed), indicating Higgs mode behavior, as a direct
consequence of spontaneous breaking of the U(1) translational symmetry of the
Hamiltonian $\hat{H}_{eff}$ by self-organization Léonard  _et al._ (2017b);
Guo  _et al._ (2019). Inset: The decay rate $-\mbox{Im}(\Omega)$ of the
Goldstone mode vanishes above threshold, as a consequence of the fact that the
continuous translational symmetry of the system equations is not broken by
cavity photon decay at rate $\kappa$. Simulation parameters are given in the
caption of Fig. 3.

In Fig. 4, we plot the first 6 eigenvalues of $\hat{\mbox{M}}$ for scanning
$\eta$, excluding the zero frequency mode which gives rise to phase diffusion
of the homogeneous part of the condensate for all $\eta$ Nagy  _et al._
(2011). At $\eta\ll\eta_{c}$, the collective condensate mode pairs have
frequencies $\omega_{R},4\omega_{R},9\omega_{R}$, which corresponds to the
spectrum of a condensate in a box Nagy  _et al._ (2008). The lowest energy
cavity photon modes have for the used parameters the frequency
$\sim-\bar{\Delta}_{c}^{\prime}=89\omega_{R}$, which is not within the range
of the plot.

Near self-organization threshold pump rate $\eta_{c}$, the lowest two mode
frequencies approach zero, and moving beyond $\eta_{c}$ leads to the splitting
of the modes into a gapless mode, indicating a phase (Goldstone) mode, and a
mode with a finite energy gap. The frequency of the finite gap mode vanishes
for $\eta\to\eta_{c}$, indicating an amplitude (Higgs) mode excitation with
fluctuations in the strength of the atomic density modulation Pekker and Varma
(2015). The simultaneous presence of Goldstone and Higgs modes in the
collective excitation spectrum is a signature of BEC supersolidity Léonard
_et al._ (2017b); Guo  _et al._ (2019). In contrast to the model of Léonard
_et al._ (2017b); Lang  _et al._ (2017), the continuous translational symmetry
is here exact, such that no finite gap appears for the Goldstone mode.

Note that the cavity photon decay with rate $\kappa$ may cause finite values
of the decay rates $-\mbox{Im}(\Omega)$ of the collective excitations.
However, for the Goldstone mode the decay rate is finite only for
$\eta<\eta_{c}$, and the damping vanishes above threshold (see inset of Fig.
4). This can be explained by the fact that cavity photon decay does not break
the U(1) translational symmetry of the Lindblad evolution equations for the
photon modes Krešić  _et al._ (2023), such that finite $\kappa$ will not
dampen the Goldstone mode appearing after spontaneous symmetry breaking of
U(1) translational symmetry in the thermodynamic limit. The same feature is
also present in the model studied in Mivehvar  _et al._ (2018).

##  III Commensurate and incommensurate modes in a multimode cavity

The great majority of work on quantum simulation with atoms coupled to optical
resonators has involved coupling to a single or two cavity modes, which gives
rise to global interaction between the atoms Mivehvar  _et al._ (2021).
However, many of the Hamiltonians of interest to the condensed matter physics
community, such as models of the glassy phase Strack and Sachdev (2011);
Angelone  _et al._ (2016); Marsh  _et al._ (2024) or frustrated magnetism Hung
_et al._ (2016), rely on more exotic types of interaction.

We here study a simple example where the effective atom-atom interaction has a
more complex spatial profile compared to the global interaction case described
in the previous section. This modification is achieved by tailoring the
distribution of multiple sideband modes circulating inside the cavity. The
Gross-Pitaevskii type equation for the atoms is now given by

$\displaystyle\begin{aligned} i\frac{\partial\psi}{\partial
t}&=-\frac{1}{2m}\frac{\partial^{2}\psi}{\partial
x^{2}}+V_{1}(x)\psi+u_{0}\\{|\alpha_{0}|^{2}+\sum_{q_{j}\in
S}[|\alpha_{j+}|^{2}+|\alpha_{j-}|^{2}\\\
+&(\alpha_{0}^{*}\alpha_{j-}+\alpha_{j+}^{*}\alpha_{0})e^{-iq_{j}x}+\alpha_{j+}^{*}\alpha_{j-}e^{-2iq_{j}x}\\\
+&(\alpha_{0}^{*}\alpha_{j+}+\alpha_{j-}^{*}\alpha_{0})e^{iq_{j}x}+\alpha_{j-}^{*}\alpha_{j+}e^{2iq_{j}x}]\\}\psi,\end{aligned}$
(21)

where $V_{1}(x)$ is an external box-shaped potential, which leads to hard wall
boundary conditions for $\psi(x)$ at the edges of the system. As the on-axis
mode is driven by a pump laser, we here approximate it by $\alpha_{0}\simeq
i\eta/[\sqrt{N}(\bar{\Delta}_{c}+i\kappa)]$, see Appendix B. For large
$\kappa$ and/or $|\bar{\Delta}_{c,j}|$, and $|\Delta_{a}|$ values, sideband
modes can then be adiabatically eliminated and approximated by

$\displaystyle\begin{aligned}
\alpha_{j\pm}&\simeq\frac{U_{0}\alpha_{0}}{\bar{\Delta}_{c,j}+i\kappa}\int
e^{\mp iq_{j}x}|\psi(x)|^{2}dx,\end{aligned}$ (22)

as also shown in the Appendix B.

For the configurations studied here, 9 cavity modes are selected by Fourier
filtering. In the first example, the wavevenumbers are commensurate and given
by $q_{j}=(1,0.5,0.25,0.125)q_{c}$. The probability distribution for a
realization of the ground state wavefunction $\psi_{0}(x)$ is shown in Fig.
5a). For these commensurate transverse wavelengths, the sideband cavity modes
constructively interfere at lengthscales of $8\Lambda_{c}$, which is also the
periodicity of the minima of the atom-atom interaction profile
$V_{2}(x,x^{\prime})$. The atoms can thus order into a droplet array of
periodicity $8\Lambda_{c}$, which simultaneously increases the diffraction
into all of the allowed cavity modes and minimizes the atomic potential
energy.

In contrast, when the modes are incommensurate, such ordering in $\psi_{0}(x)$
does not occur, as shown in the example in Fig. 5b), where
$q_{j}=(1,0.775,0.575,0.225)q_{c}$. Here, a single constructive interference
peak between all of the allowed modes occurs for $|x-x^{\prime}|=0$. It is
thus not possible to simultaneously enhance the scattering into all cavity
modes for atoms arranged in a periodic array. The ground state solution is now
a single droplet state.

Figure 5: Results for a multimode cavity with 9 cavity modes. (a)
$|\psi_{0}|^{2}$ for commensurate cavity modes with
$q_{j}=(1,0.5,0.25,0.125)q_{c}$ (blue, constant $\bar{\Delta}_{c,j}$ and
orange, variable $\bar{\Delta}_{c,j}$). (b) $|\psi_{0}|^{2}$ for
incommensurate cavity modes with $q_{j}=(1,0.775,0.575,0.225)q_{c}$ (blue,
constant $\bar{\Delta}_{c,j}$ and orange, variable $\bar{\Delta}_{c,j}$). Left
inset: atom-atom interaction profile $V_{2}(x,x^{\prime})$ with $U_{0}\to
u_{0}$ (blue, constant $\bar{\Delta}_{c,j}$ and orange, variable
$\bar{\Delta}_{c,j}$). Right inset: cosine functions for modes with different
$q_{j}$ values (red, $q_{1}$, green, $q_{2}$, orange, $q_{3}$, blue, $q_{4}$).
Simulation parameters:
$(\bar{\Delta}_{c},\;\bar{\Delta}_{c}^{\prime},\;u_{0},\;\kappa)=(3,\;-2,\;-0.1,\;20)\omega_{R}$,
(a) $\eta=290\omega_{R}$ (blue), $\eta=270\omega_{R}$ (orange), (b)
$\eta=430\omega_{R}$ (blue), $\eta=350\omega_{R}$ (orange). For constant
$\bar{\Delta}_{c,j}$, the value is
$\bar{\Delta}_{c,j}=\bar{\Delta}_{c}^{\prime}$. For variable
$\bar{\Delta}_{c,j}$ the value is changed as
$\bar{\Delta}_{c,j}=\bar{\Delta}_{c}-(q_{j}/q_{4})^{2}\kappa/4$. Hard wall
boundary conditions were imposed at the system ends at $x=\pm 20$.

Detunings $\Delta_{c,j}$ depend on the diffractive phase shifts of the
sideband modes propagating in the system Ackemann  _et al._ (2021); Krešić
_et al._ (2023). In Fig. 5, we plot the case for which $\bar{\Delta}_{c,j}$ is
equal for all sidebands, and the case for which $\bar{\Delta}_{c,j}$ depends
on $q_{j}$. For the latter case, near cavity degeneracy the quadratic
relationship $\omega_{c,j}-\omega_{0}\propto q_{j}^{2}$ holds Esteban-Martin
_et al._ (2004); Krešić  _et al._ (2023), which is here used to determine the
detunings $\bar{\Delta}_{c,j}$. The two cases have the same ground state
solutions. The depth of the $V_{2}(x,x^{\prime})$ minima is larger for the
parameters of the case with variable $\bar{\Delta}_{c,j}$, such that the
localization of the atoms at the optical potential minima is here stronger.
Using wavefront shaping elements inside the cavity feedback loop, the cavity
sideband phase shifts can be controlled in order to tailor the distribution of
$\bar{\Delta}_{c,j}$ values and, in combination with selecting the $q_{j}$
distribution, engineer the shape of $V_{2}(x,x^{\prime})$, which is an
intriguing prospect for applications in quantum simulation.

##  IV Conclusions

A novel method for providing tailored photon-mediated atom-atom interaction in
a BEC was studied theoretically and numerically. The method is based on
diffractive coupling by electromagnetic modes of a ring cavity. In the first
example of self-organization into stripe states, it is shown how the
continuous translational symmetry of the system Hamiltonian is manifested in
the “sombrero" shape of the effective mean field potential for the atomic
degrees of freedom above self-organization threshold. This symmetry is
spontanously broken in the thermodynamic limit, by the system selecting a
particular phase for the steady state sinusoidal pattern in atomic density and
intracavity light field profile. Supersolid properties are exhibited by the
simultaneous presence of Goldstone and Higgs modes in the numerically
calculated collective excitation spectrum. The calculations corroborate
previous remarks regarding supersolidity of optomechanical patterns in a BEC,
observed for simulations in the single retroreflecting mirror configuration
Robb  _et al._ (2015), which is a setup similar to the ring cavity scheme
Ackemann  _et al._ (2021). It should be readily possible to extend the problem
into two dimensional geometries, where an additional rotational U(1) symmetry
will be broken by self-organization, which relates to the recent interest in
supersolidity of dipolar condensates in two dimensions Norcia  _et al._
(2021); Bland  _et al._ (2022).

For highly elongated clouds Nguyen  _et al._ (2017); Lim  _et al._ (2021),
spatial features can be tuned to values of tens of microns, which would enable
real space monitoring of the condensate dynamics. This also opens the
possibility of modifying the photon-mediated interatomic interaction by
placing optical elements with spatially tailored profiles within the optical
feedback loop of the cavity.

Going beyond global all-to-all atom-atom coupling occurring for three cavity
modes, we also studied the case of a multimode cavity with a tailored
distribution of photonic sidebands. For commensurate transverse wavenumbers
$q_{j}$, the ground state is a droplet array, which simultaneously enhances
scattering into all of the allowed cavity modes. In contrast, for
incommensurate modes only one atom-atom potential minimum occurs, at
$|x-x^{\prime}|=0$. The resulting ground state solution is a single droplet
state. These example situations demonstrate the versatility of the proposed
experimental setup. A wide variety of interaction profiles
$V_{2}(x,x^{\prime})$ can be implemented in this context, which may lead to
non-equilibrium realizations of models of spin glasses Strack and Sachdev
(2011); Angelone  _et al._ (2016); Marsh  _et al._ (2024) or frustrated
magnetism Hung  _et al._ (2016).

Acknowledgements. I thank Natalia Masalaeva for sharing her code and for many
helpful discussions, and Thorsten Ackemann for comments on the early version
of the manuscript. This work was funded by the Austrian Science Fund (FWF
Meitner-Programm, Project No. M3011) and the Austrian Academy of Sciences (ESQ
Discovery Programm, Project: “Using self-organization of ultracold atoms in
emerging quantum technologies”). I acknowledge the support of the project
KODYN financed by the European Union through the National Recovery and
Resilience Plan 2021-2026 (NRPP). In addition, this work was supported by the
project Centre for Advanced Laser Techniques (CALT), co-funded by the European
Union through the European Regional Development Fund under the Competitiveness
and Cohesion Operational Programme (Grant No. KK.01.1.1.05.0001). The
computational results presented here have been achieved using the Vienna
Scientific Cluster (VSC). The dynamical evolution equations were solved
numerically by using the open-source framework QuantumOptics.jl Krämer  _et
al._ (2018).

##  V Appendix A: Derivation of the effective Hamiltonian

The effective Hamiltonian (2) can be derived from the Jaynes-Cummings model,
similarly to the procedure given in Refs. Maschler  _et al._ (2008); Masalaeva
_et al._ (2023). We describe the atom-light interaction within the domain of
paraxial approximation, where only forward-directed scattering of light by the
condensate can occur, such that only counterclockwise- or clockwise-
propagating modes can be excited by the instability. We can thus concentrate
on a one-dimensional model, where the cavity mode operator at the BEC medium
is given by:

$\displaystyle\begin{aligned} \hat{E}(x)=\hat{a}_{0}+\sum_{q_{j}\in
S}\left(\hat{a}_{j+}e^{iq_{j}x}+\hat{a}_{j-}e^{-iq_{j}x}\right),\end{aligned}$
(A1)

where $\hat{a}_{0}$ is the annihilation operator of the on-axis plane wave
mode, and $\hat{a}_{j\pm}$ are the annihilation operators of the diffracted
sidebands with transverse wavenumbers $\pm q_{j}=\pm 2\pi/\Lambda_{j}$. The
allowed sideband wavenumbers $q_{j}$ are part of a bounded discrete set $S$,
and the values contained within $S$ can be experimentally tailored via Fourier
filtering Jensen  _et al._ (1998); Labeyrie  _et al._ (2024).

In the dipole and rotating wave approximations, the single atom Jaynes-
Cummings Hamiltonian of the system is given by ($\hbar=1$):

$\displaystyle\begin{aligned}
\hat{\mathcal{H}}^{(1)}&=-\frac{1}{2m}\frac{\partial^{2}}{\partial
x^{2}}+\omega_{a}\hat{\sigma}_{ee}+\\{[g_{0}\hat{a}_{0}+\sum_{q_{j}\in
S}g_{j}(\hat{a}_{j+}e^{iq_{j}x}+\hat{a}_{j-}e^{-iq_{j}x})]\hat{\sigma}_{eg}+\mbox{H.c.}\\}\\\
+&V_{e}(x)\hat{\sigma}_{ee}+V_{g}(x)\hat{\sigma}_{gg}+\omega_{0}\hat{a}_{0}^{\dagger}\hat{a}_{0}+\sum_{j}\omega_{j}(\hat{a}_{j+}^{\dagger}\hat{a}_{j+}+\hat{a}_{j-}^{\dagger}\hat{a}_{j-})+i\eta(\hat{a}_{0}^{\dagger}e^{-i\omega
t}-\hat{a}_{0}e^{i\omega t}),\end{aligned}$ (A2)

where $m$ is the mass of the atom, $\omega_{a}$, $\omega$, $\omega_{j}$ are
the frequencies of the atomic two-level transition, drive laser and the $j$-th
cavity mode, respectively, $g_{j}$ is the atom-cavity coupling strength of the
$j$-th mode, $\hat{\sigma}_{kk^{\prime}}=|k\rangle\langle k^{\prime}|$ is the
atomic optical transition operator, $V_{g,e}(x)$ are the external trapping
potentials for atoms in the ground and excited states, respectively,
$\eta\in\mathbb{R}$ is the laser pump rate and the sum over $j$ implies
Fourier filtering of $q_{j}$ was performed. In the following we approximate
all $g_{j}$ values by $g_{0}\in\mathbb{R}$. Note that we here approximate the
direct atom-atom scattering in the BEC to be sufficiently small to be
neglected, i.e. the phenomena studied in this Article are based only on
photon-mediated interaction between the atoms.

The Hamiltonian (A2) can be converted into a time-independent form

$\displaystyle\begin{aligned}
\hat{\tilde{\mathcal{H}}}^{(1)}&=\hat{U}\hat{\mathcal{H}}^{(1)}\hat{U}^{\dagger}+i(\partial_{t}\hat{U})\hat{U}^{\dagger}\\\
&=-\frac{1}{2m}\frac{\partial^{2}}{\partial
x^{2}}-\Delta_{a}\hat{\sigma}_{ee}+(g_{0}\hat{E}(x)\hat{\sigma}_{eg}+\mbox{H.c.})+V_{e}(x)\hat{\sigma}_{ee}+V_{g}(x)\hat{\sigma}_{gg}-\Delta_{c}\hat{a}_{0}^{\dagger}\hat{a}_{0}-\sum_{j}\Delta_{c,j}(\hat{a}_{j+}^{\dagger}\hat{a}_{j+}+\hat{a}_{j-}^{\dagger}\hat{a}_{j-})+i\eta(\hat{a}_{0}^{\dagger}-\hat{a}_{0}),\end{aligned}$
(A3)

where $\Delta_{a}=\omega-\omega_{a}$, $\Delta_{c}=\omega-\omega_{0}$,
$\Delta_{c,j}=\omega-\omega_{j}$, by using

$\displaystyle\begin{aligned} \hat{U}&=\exp\left\\{i\omega
t\left[\hat{\sigma}_{ee}+\hat{a}_{0}^{\dagger}\hat{a}_{0}+\sum_{j}(\hat{a}_{j+}^{\dagger}\hat{a}_{j+}+\hat{a}_{j-}^{\dagger}\hat{a}_{j-})\right]\right\\}.\end{aligned}$
(A4)

The corresponding many-body Hamiltonian of the system is given by:

$\displaystyle\begin{aligned}
\hat{H}&=\int\hat{\Psi}_{g}^{\dagger}(x)\left[-\frac{1}{2m}\frac{\partial^{2}}{\partial
x^{2}}+V_{g}(x)\right]\hat{\Psi}_{g}(x)dx+\int\hat{\Psi}_{e}^{\dagger}(x)\left[-\frac{1}{2m}\frac{\partial^{2}}{\partial
x^{2}}+V_{e}(x)-\Delta_{a}\right]\hat{\Psi}_{e}(x)dx\\\
+&\int\left(g_{0}\hat{\Psi}_{e}^{\dagger}(x)\hat{E}(x)\hat{\Psi}_{g}(x)+\mbox{H.c.}\right)dx-\Delta_{c}\hat{a}_{0}^{\dagger}\hat{a}_{0}-\sum_{j}\Delta_{c,j}(\hat{a}_{j+}^{\dagger}\hat{a}_{j+}+\hat{a}_{j-}^{\dagger}\hat{a}_{j-})+i\eta(\hat{a}_{0}^{\dagger}-\hat{a}_{0}),\end{aligned}$
(A5)

where

$\displaystyle\begin{aligned}
&[\hat{\Psi}_{k}(x),\hat{\Psi}_{k^{\prime}}^{\dagger}(x^{\prime})]=\delta(x-x^{\prime})\delta_{k,k^{\prime}},\\\
&[\hat{\Psi}_{k}(x),\hat{\Psi}_{k^{\prime}}(x^{\prime})]=[\hat{\Psi}_{k}^{\dagger}(x),\hat{\Psi}_{k^{\prime}}^{\dagger}(x^{\prime})]=0.\end{aligned}$
(A6)

The Heisenberg equation for $\hat{\Psi}_{e}(x)$ is given by

$\displaystyle\begin{aligned} i\frac{\partial\hat{\Psi}_{e}(x)}{\partial
t}=\left[-\frac{1}{2m}\frac{\partial^{2}}{\partial
x^{2}}+V_{e}(x)-\Delta_{a}\right]\hat{\Psi}_{e}(x)+g_{0}\hat{E}(x)\hat{\Psi}_{g}(x)\end{aligned}$
(A7)

In the limit of large $\Delta_{a}$, the excited state dynamics reaches steady
state much more quickly than the ground state. The kinetic energy and external
potential terms can be neglected in comparison to $\Delta_{a}$, and
$\hat{\Psi}_{e}(x)$ can be adiabatically eliminated to give:

$\displaystyle\begin{aligned}
\hat{\Psi}_{e}(x)\simeq\frac{1}{\Delta_{a}}g_{0}\hat{E}(x)\hat{\Psi}_{g}(x).\end{aligned}$
(A8)

The Heisenberg equations for the cavity operators $\hat{a}_{0},\hat{a}_{j}$
(including a phenomenological cavity decay constant $\kappa$) and the atomic
field $\hat{\Psi}_{g}(x)\equiv\hat{\psi}(x)$ can be derived from (A5), which
after inserting (A8) leads to:

$\displaystyle\begin{aligned} \frac{\partial\hat{a}_{0}}{\partial
t}&=(i\Delta_{c}-\kappa)\hat{a}_{0}-iU_{0}\int\hat{\psi}^{\dagger}(x)\hat{E}(x)\hat{\psi}(x)dx+\eta,\\\
\frac{\partial\hat{a}_{j\pm}}{\partial
t}&=(i\Delta_{c,j}-\kappa)\hat{a}_{j\pm}-iU_{0}\int\hat{\psi}^{\dagger}(x)\hat{E}(x)e^{\mp
iq_{j}x}\hat{\psi}(x)dx,\\\ \frac{\partial\hat{\psi}(x)}{\partial
t}&=\left[-\frac{1}{2m}\frac{\partial^{2}}{\partial
x^{2}}+V_{1}(x)+U_{0}\hat{E}^{\dagger}(x)\hat{E}(x)\right]\hat{\psi}(x).\end{aligned}$
(A9)

Eqs. (A9) can be derived from the Heisenberg equation with the effective
Hamiltonian $\hat{H}_{eff}$, given by:

$\displaystyle\begin{aligned}
\hat{H}_{eff}&=-\Delta_{c}\hat{a}_{0}^{\dagger}\hat{a}_{0}-\sum_{j}\Delta_{c,j}(\hat{a}_{j+}^{\dagger}\hat{a}_{j+}+\hat{a}_{j-}^{\dagger}\hat{a}_{j-})+i\eta(\hat{a}_{0}^{\dagger}-\hat{a}_{0})\\\
+&\int\hat{\psi}^{\dagger}(x)\left[-\frac{1}{2m}\frac{\partial^{2}}{\partial
x^{2}}+V_{1}(x)+U_{0}\hat{E}^{\dagger}(x)\hat{E}(x)\right]\hat{\psi}(x)dx.\end{aligned}$
(A10)

##  VI Appendix B: Photon-mediated interatomic interaction

In the limit of large $|\Delta_{c,j}|$ and/or $\kappa$, the cavity photon
operators can be adiabatically eliminated. The system can then be described by
a Hamiltonian of the form:

$\displaystyle\begin{aligned}
\hat{H}_{eff,at}&=\int\hat{\psi}^{\dagger}(x)\left[-\frac{1}{2m}\frac{\partial^{2}}{\partial
x^{2}}+V_{1}(x)\right]\hat{\psi}(x)dx\\\
&+\frac{1}{2}\int\hat{\psi}^{\dagger}(x^{\prime})\hat{\psi}^{\dagger}(x)V_{2}(x,x^{\prime})\hat{\psi}(x)\hat{\psi}(x^{\prime})dxdx^{\prime},\end{aligned}$
(A11)

where $V_{2}(x,x^{\prime})$ is the effective atom-atom interaction mediated by
cavity photons.

To determine the form of $V_{2}(x,x^{\prime})$, we first adiabatically
eliminate the on-axis mode, which leads to:

$\displaystyle\begin{aligned}
\hat{a}_{0}&\simeq\frac{i\eta}{\bar{\Delta}_{c}+i\kappa}+\mathcal{O}\left(\frac{1}{\Delta_{a}}\right),\end{aligned}$
(A12)

where $\bar{\Delta}_{c}=\omega-\omega_{0}-NU_{0}$, and we neglect the terms
$\mathcal{O}(1/\Delta_{a})$. As the on-axis mode is driven by a laser, one can
approximate $\hat{a}_{0}$ by $\langle\hat{a}_{0}\rangle\to\sqrt{N}\alpha_{0}$,
where $\alpha_{0}\simeq i\eta/[\sqrt{N}(\bar{\Delta}_{c}+i\kappa)]$ Krešić
_et al._ (2023). The sideband cavity photon operators can then be
adiabatically eliminated to give:

$\displaystyle\begin{aligned}
\hat{a}_{j\pm}&\simeq\frac{U_{0}\sqrt{N}\alpha_{0}}{\bar{\Delta}_{c,j}+i\kappa}\int
e^{\mp
iq_{j}x}\hat{\psi}^{\dagger}(x)\hat{\psi}(x)dx+\mathcal{O}\left(\frac{1}{\Delta_{a}^{2}}\right),\end{aligned}$
(A13)

where $\bar{\Delta}_{c,j}=\omega-\omega_{j}-NU_{0}$ and we neglect the
$\mathcal{O}(1/\Delta_{a}^{2})$ terms. Inserting expressions (A12) and (A13)
into the Heisenberg equation (A9) for $\hat{\psi}(x)$, leads to

$\displaystyle\begin{aligned} \frac{\partial\hat{\psi}(x)}{\partial
t}&=\left[-\frac{1}{2m}\frac{\partial^{2}}{\partial
x^{2}}+V_{1}(x)+\int\hat{\psi}^{\dagger}(x^{\prime})V_{2}(x,x^{\prime})\hat{\psi}(x^{\prime})dx^{\prime}\right]\hat{\psi}(x),\end{aligned}$
(A14)

where the atom-atom interaction is given by:

$\displaystyle\begin{aligned}
V_{2}(x,x^{\prime})&=\frac{\eta^{2}}{\bar{\Delta}_{c}^{2}+\kappa^{2}}\left\\{\frac{U_{0}}{N}+4U_{0}^{2}\sum_{q_{j}\in
S}\frac{\bar{\Delta}_{c,j}}{\bar{\Delta}_{c,j}^{2}+\kappa^{2}}\cos[q_{j}(x-x^{\prime})]\right\\},\end{aligned}$
(A15)

and the constant first term in (A15) can be neglected. An effective atom-only
Hamiltonian for this case is then given by Eq. (A11).

##  VII Appendix C: Excitation matrix $\hat{\mbox{M}}$

Inserting $\psi(x,t)=e^{-i\mu_{0}t}[\psi_{0}(x)+\delta\psi(x,t)]$ and
$\alpha_{j}(t)=\alpha_{j}^{0}+\delta\alpha_{j}(t)$, for $j=0,\pm$, into Eq.
(18), the dynamical equations are:

$\displaystyle\begin{aligned} \frac{\partial\delta\alpha_{0}}{\partial
t}&=(i\bar{\Delta}_{c}-\kappa)\delta\alpha_{0}-iu_{0}[\mathcal{I}_{+}^{0}\delta\alpha_{+}+\mathcal{I}_{-}^{0}\delta\alpha_{-}+\int
A_{0}(\psi_{0}\delta\psi^{*}+\psi_{0}^{*}\delta\psi)dx],\\\
\frac{\partial\delta\alpha_{\pm}}{\partial
t}&=(i\bar{\Delta}_{c}^{\prime}-\kappa)\delta\alpha_{\pm}-iu_{0}[\mathcal{I}_{\mp}^{0}\delta\alpha_{0}+\mathcal{I}_{2\mp}^{0}\delta\alpha_{\mp}+\int
A_{\pm}(\psi_{0}\delta\psi^{*}+\psi_{0}^{*}\delta\psi)dx],\\\
i\frac{\partial\delta\psi}{\partial
t}&=(\mathcal{H}_{0}-\mu_{0})\delta\psi+u_{0}\psi_{0}(A_{0}\delta\alpha_{0}^{*}+A_{0}^{*}\delta\alpha_{0}+A_{+}\delta\alpha_{+}^{*}+A_{+}^{*}\delta\alpha_{+}+A_{-}\delta\alpha_{-}^{*}+A_{-}^{*}\delta\alpha_{-}),\end{aligned}$
(A16)

where $A_{0}=\alpha_{+}^{0}e^{iq_{c}x}+\alpha_{-}^{0}e^{-iq_{c}x}$,
$A_{\pm}=\alpha_{0}^{0}e^{\mp iq_{c}x}+\alpha_{\mp}^{0}e^{\mp 2iq_{c}x}$, and
the Hamiltonian $\mathcal{H}_{0}$ is given by:

$\displaystyle\begin{aligned}
\mathcal{H}_{0}&=-\frac{1}{2m}\frac{\partial^{2}}{\partial
x^{2}}+u_{0}\\{[(\alpha_{0}^{0})^{*}\alpha_{+}^{0}+(\alpha_{-}^{0})^{*}\alpha_{0}^{0}]e^{iq_{c}x}+[(\alpha_{0}^{0})^{*}\alpha_{-}^{0}+(\alpha_{+}^{0})^{*}\alpha_{0}^{0}]e^{-iq_{c}x}+(\alpha_{-}^{0})^{*}\alpha_{+}^{0}e^{2iq_{c}x}+(\alpha_{+}^{0})^{*}\alpha_{-}^{0}e^{-2iq_{c}x}\\}.\end{aligned}$
(A17)

Writing $\delta\psi(x,t)=\delta\psi^{+}(x)e^{-i\Omega
t}+[\delta\psi^{-}(x)]^{*}e^{i\Omega^{*}t}$ and
$\delta\alpha_{j}(t)=\delta\alpha_{j}^{+}e^{-i\Omega
t}+[\delta\alpha_{j}^{-}]^{*}e^{i\Omega^{*}t}$, and collecting the
$e^{-i\Omega t}$, $e^{i\Omega^{*}t}$ terms, we get the equations:

$\displaystyle\begin{aligned}
\Omega\delta\alpha_{0}^{+}&=(-\bar{\Delta}_{c}-i\kappa)\delta\alpha_{0}^{+}+u_{0}[\mathcal{I}_{+}^{0}\delta\alpha_{+}^{+}+\mathcal{I}_{-}^{0}\delta\alpha_{-}^{+}+\int
A_{0}(\psi_{0}\delta\psi^{-}+\psi_{0}^{*}\delta\psi^{+})dx],\\\
\Omega\delta\alpha_{0}^{-}&=(\bar{\Delta}_{c}-i\kappa)\delta\alpha_{0}^{-}-u_{0}[(\mathcal{I}_{+}^{0})^{*}\delta\alpha_{+}^{-}+(\mathcal{I}_{-}^{0})^{*}\delta\alpha_{-}^{-}+\int
A_{0}^{*}(\psi_{0}\delta\psi^{-}+\psi_{0}^{*}\delta\psi^{+})dx],\\\
\Omega\delta\alpha_{+}^{+}&=(-\bar{\Delta}_{c}^{\prime}-i\kappa)\delta\alpha_{+}^{+}+u_{0}[\mathcal{I}_{-}^{0}\delta\alpha_{0}^{+}+\mathcal{I}_{2-}^{0}\delta\alpha_{-}^{+}+\int
A_{+}(\psi_{0}\delta\psi^{-}+\psi_{0}^{*}\delta\psi^{+})dx],\\\
\Omega\delta\alpha_{+}^{-}&=(\bar{\Delta}_{c}^{\prime}-i\kappa)\delta\alpha_{+}^{-}-u_{0}[(\mathcal{I}_{-}^{0})^{*}\delta\alpha_{0}^{-}+(\mathcal{I}_{2-}^{0})^{*}\delta\alpha_{-}^{-}+\int
A_{+}^{*}(\psi_{0}\delta\psi^{-}+\psi_{0}^{*}\delta\psi^{+})dx],\\\
\Omega\delta\alpha_{-}^{+}&=(-\bar{\Delta}_{c}^{\prime}-i\kappa)\delta\alpha_{-}^{+}+u_{0}[\mathcal{I}_{+}^{0}\delta\alpha_{0}^{+}+\mathcal{I}_{2+}^{0}\delta\alpha_{+}^{+}+\int
A_{-}(\psi_{0}\delta\psi^{-}+\psi_{0}^{*}\delta\psi^{+})dx],\\\
\Omega\delta\alpha_{-}^{-}&=(\bar{\Delta}_{c}^{\prime}-i\kappa)\delta\alpha_{-}^{-}-u_{0}[(\mathcal{I}_{+}^{0})^{*}\delta\alpha_{0}^{-}+(\mathcal{I}_{2+}^{0})^{*}\delta\alpha_{+}^{-}+\int
A_{-}^{*}(\psi_{0}\delta\psi^{-}+\psi_{0}^{*}\delta\psi^{+})dx],\\\
\Omega\delta\psi^{+}&=(\mathcal{H}_{0}-\mu_{0})\delta\psi^{+}+u_{0}\psi_{0}(A_{0}\delta\alpha_{0}^{-}+A_{0}^{*}\delta\alpha_{0}^{+}+A_{+}\delta\alpha_{+}^{-}+A_{+}^{*}\delta\alpha_{+}^{+}+A_{-}\delta\alpha_{-}^{-}+A_{-}^{*}\delta\alpha_{-}^{+}),\\\
\Omega\delta\psi^{-}&=-(\mathcal{H}_{0}-\mu_{0})\delta\psi^{-}-u_{0}\psi_{0}^{*}(A_{0}\delta\alpha_{0}^{-}+A_{0}^{*}\delta\alpha_{0}^{+}+A_{+}\delta\alpha_{+}^{-}+A_{+}^{*}\delta\alpha_{+}^{+}+A_{-}\delta\alpha_{-}^{-}+A_{-}^{*}\delta\alpha_{-}^{+}).\end{aligned}$
(A18)

The Eq. (A18) is an eigenvalue problem that can readily be written in matrix
form $\hat{\mbox{M}}\mathbf{R}=\Omega\mathbf{R}$, where $\mathbf{R}$ is given
by Eq. (20).

## References

  * Ritsch  _et al._ (2013) H. Ritsch, P. Domokos, F. Brennecke, and T. Esslinger, Rev. Mod. Phys. 85, 553 (2013).
  * Mivehvar  _et al._ (2021) F. Mivehvar, F. Piazza, T. Donner, and H. Ritsch, Advances in Physics 70, 1 (2021).
  * Baumann  _et al._ (2010) K. Baumann, C. Guerlin, F. Brennecke, and T. Esslinger, Nature 464, 1301 (2010).
  * Léonard  _et al._ (2017a) J. Léonard, A. Morales, P. Zupancic, T. Esslinger, and T. Donner, Nature 543, 87 (2017a).
  * Léonard  _et al._ (2017b) J. Léonard, A. Morales, P. Zupancic, T. Donner, and T. Esslinger, Science 358, 1415 (2017b).
  * Schuster  _et al._ (2020) S. C. Schuster, P. Wolf, S. Ostermann, S. Slama, and C. Zimmermann, Phys. Rev. Lett. 124, 143602 (2020).
  * Guo  _et al._ (2021) Y. Guo, R. M. Kroeze, B. P. Marsh, S. Gopalakrishnan, J. Keeling, and B. L. Lev, Nature 599, 211 (2021).
  * Tesio  _et al._ (2014a) E. Tesio, G. Robb, T. Ackemann, W. Firth, and G.-L. Oppo, Physical Review Letters 112, 043901 (2014a).
  * Labeyrie  _et al._ (2014) G. Labeyrie, E. Tesio, P. M. Gomes, G.-L. Oppo, W. J. Firth, G. R. M. Robb, A. S. Arnold, R. Kaiser, and T. Ackemann, Nature Photonics 8, 321 (2014).
  * Robb  _et al._ (2015) G. Robb, E. Tesio, G.-L. Oppo, W. Firth, T. Ackemann, and R. Bonifacio, Physical Review Letters 114, 173903 (2015).
  * Zhang  _et al._ (2018) Y.-C. Zhang, V. Walther, and T. Pohl, Physical Review Letters 121, 073604 (2018).
  * Ackemann  _et al._ (2021) T. Ackemann, G. Labeyrie, G. Baio, I. Krešić, J. G. M. Walker, A. Costa Boquete, P. Griffin, W. J. Firth, R. Kaiser, G.-L. Oppo, and G. R. M. Robb, Atoms 9, 35 (2021).
  * Krešić  _et al._ (2018) I. Krešić, G. Labeyrie, G. R. M. Robb, G.-L. Oppo, P. M. Gomes, P. Griffin, R. Kaiser, and T. Ackemann, Communications Physics 1, 33 (2018).
  * Labeyrie  _et al._ (2018) G. Labeyrie, I. Krešić, G. R. M. Robb, G.-L. Oppo, R. Kaiser, and T. Ackemann, Optica 5, 1322 (2018).
  * Labeyrie  _et al._ (2024) G. Labeyrie, J. G. M. Walker, G. R. M. Robb, R. Kaiser, and T. Ackemann, Phys. Rev. Lett. 132, 143402 (2024).
  * Lugiato and Lefever (1987) L. A. Lugiato and R. Lefever, Physical Review Letters 58, 2209 (1987).
  * Firth (1990) W. J. Firth, Journal of Modern Optics 37, 151 (1990).
  * Firth and Paré (1988) W. J. Firth and C. Paré, Opt. Lett. 13, 1096 (1988).
  * Tesio  _et al._ (2012) E. Tesio, G. R. M. Robb, T. Ackemann, W. J. Firth, and G.-L. Oppo, Physical Review A 86, 031801 (2012).
  * Boquete  _et al._ (2021) A. C. Boquete, G. Baio, G. Robb, G.-L. Oppo, P. Griffin, and T. Ackemann, Journal of Physics: Conference Series 1919, 012014 (2021).
  * Esteban-Martin  _et al._ (2004) A. Esteban-Martin, J. García, E. Roldán, V. B. Taranenko, G. J. de Valcárcel, and C. O. Weiss, Physical Review A 69, 033816 (2004).
  * Esteban-Martín  _et al._ (2005) A. Esteban-Martín, V. B. Taranenko, J. García, G. J. de Valcárcel, and E. Roldán, Phys. Rev. Lett. 94, 223903 (2005).
  * Kreuzer  _et al._ (1990) M. Kreuzer, W. Balzer, and T. Tschudi, Applied Optics 29, 579 (1990).
  * Staliūnas  _et al._ (1997) K. Staliūnas, G. Šlekys, and C. O. Weiss, Phys. Rev. Lett. 79, 2658 (1997).
  * Ackemann  _et al._ (2000) T. Ackemann, S. Barland, M. Cara, S. Balle, J. Tredicce, R. Jäger, M. Grabherr, M. Miller, and K. J. Ebeling, Journal of Optics B: Quantum and Semiclassical Optics 2, 406 (2000).
  * Periwal  _et al._ (2021) A. Periwal, E. S. Cooper, P. Kunkel, J. F. Wienand, E. J. Davis, and M. Schleier-Smith, Nature 600, 630 (2021).
  * Vaidya  _et al._ (2018) V. D. Vaidya, Y. Guo, R. M. Kroeze, K. E. Ballantine, A. J. Kollar, J. Keeling, and B. L. Lev, Physical Review X 8, 011002 (2018).
  * Masalaeva  _et al._ (2023) N. Masalaeva, H. Ritsch, and F. Mivehvar, Phys. Rev. Lett. 131, 173401 (2023).
  * Tanji-Suzuki  _et al._ (2011) H. Tanji-Suzuki, I. D. Leroux, M. H. Schleier-Smith, M. Cetina, A. T. Grier, J. Simon, and V. Vuletić, in  _Advances in atomic, molecular, and optical physics_, Vol. 60 (Elsevier, 2011) pp. 201–237.
  * Jensen  _et al._ (1998) S. J. Jensen, M. Schwab, and C. Denz, Physical Review Letters 81, 1614 (1998).
  * Andreev and Lifshits (1969) A. Andreev and I. Lifshits, Zhur. Eksper. Teoret. Fiziki 56, 2057 (1969).
  * Chester (1970) G. V. Chester, Phys. Rev. A 2, 256 (1970).
  * Leggett (1970) A. J. Leggett, Phys. Rev. Lett. 25, 1543 (1970).
  * Prokof’ev and Svistunov (2005) N. Prokof’ev and B. Svistunov, Phys. Rev. Lett. 94, 155302 (2005).
  * Toner (2008) J. Toner, Phys. Rev. Lett. 100, 035302 (2008).
  * Reppy (2010) J. D. Reppy, Phys. Rev. Lett. 104, 255301 (2010).
  * Boninsegni and Prokof’ev (2012) M. Boninsegni and N. V. Prokof’ev, Rev. Mod. Phys. 84, 759 (2012).
  * Kim and Chan (2012) D. Y. Kim and M. H. W. Chan, Phys. Rev. Lett. 109, 155301 (2012).
  * Xiang  _et al._ (2024) J. Xiang, C. Zhang, Y. Gao, W. Schmidt, K. Schmalzl, C.-W. Wang, B. Li, N. Xi, X.-Y. Liu, H. Jin, _et al._ , Nature 625, 270 (2024).
  * Bloch  _et al._ (2008) I. Bloch, J. Dalibard, and W. Zwerger, Rev. Mod. Phys. 80, 885 (2008).
  * Daley  _et al._ (2022) A. J. Daley, I. Bloch, C. Kokail, S. Flannigan, N. Pearson, M. Troyer, and P. Zoller, Nature 607, 667 (2022).
  * Bogoliubov (1947) N. Bogoliubov, J. Phys 11, 23 (1947).
  * Gopalakrishnan  _et al._ (2010) S. Gopalakrishnan, B. L. Lev, and P. M. Goldbart, Physical Review A 82, 043612 (2010).
  * Mivehvar  _et al._ (2018) F. Mivehvar, S. Ostermann, F. Piazza, and H. Ritsch, Phys. Rev. Lett. 120, 123601 (2018).
  * Chomaz  _et al._ (2019) L. Chomaz, D. Petter, P. Ilzhöfer, G. Natale, A. Trautmann, C. Politi, G. Durastante, R. M. W. van Bijnen, A. Patscheider, M. Sohmen, M. J. Mark, and F. Ferlaino, Phys. Rev. X 9, 021012 (2019).
  * Böttcher  _et al._ (2019) F. Böttcher, J.-N. Schmidt, M. Wenzel, J. Hertkorn, M. Guo, T. Langen, and T. Pfau, Phys. Rev. X 9, 011051 (2019).
  * Tanzi  _et al._ (2019) L. Tanzi, S. Roccuzzo, E. Lucioni, F. Famà, A. Fioretti, C. Gabbanini, G. Modugno, A. Recati, and S. Stringari, Nature 574, 382 (2019).
  * Guo  _et al._ (2019) M. Guo, F. Böttcher, J. Hertkorn, J.-N. Schmidt, M. Wenzel, H. P. Büchler, T. Langen, and T. Pfau, Nature 574, 386 (2019).
  * Norcia  _et al._ (2021) M. A. Norcia, C. Politi, L. Klaus, E. Poli, M. Sohmen, M. J. Mark, R. N. Bisset, L. Santos, and F. Ferlaino, Nature 596, 357 (2021).
  * Bland  _et al._ (2022) T. Bland, E. Poli, C. Politi, L. Klaus, M. A. Norcia, F. Ferlaino, L. Santos, and R. N. Bisset, Phys. Rev. Lett. 128, 195302 (2022).
  * Recati and Stringari (2023) A. Recati and S. Stringari, Nature Reviews Physics 5, 735 (2023).
  * Li  _et al._ (2017) J.-R. Li, J. Lee, W. Huang, S. Burchesky, B. Shteynas, F. C. Top, A. O. Jamison, and W. Ketterle, Nature 543, 91 (2017).
  * Krešić  _et al._ (2023) I. Krešić, G. R. M. Robb, G.-L. Oppo, and T. Ackemann, Phys. Rev. Lett. 131, 163602 (2023).
  * Szirmai  _et al._ (2010) G. Szirmai, D. Nagy, and P. Domokos, Phys. Rev. A 81, 043639 (2010).
  * Pekker and Varma (2015) D. Pekker and C. Varma, Annual Review of Condensed Matter Physics 6, 269 (2015).
  * Coleman and Weinberg (1973) S. Coleman and E. Weinberg, Phys. Rev. D 7, 1888 (1973).
  * Kardar (2007) M. Kardar,  _Statistical physics of fields_ (Cambridge University Press, 2007).
  * Bass  _et al._ (2021) S. D. Bass, A. De Roeck, and M. Kado, Nature Reviews Physics 3, 608 (2021).
  * Tesio  _et al._ (2014b) E. Tesio, G. R. M. Robb, G.-L. Oppo, P. M. Gomes, T. Ackemann, G. Labeyrie, R. Kaiser, and W. J. Firth, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 372, 20140002 (2014b).
  * Nagy  _et al._ (2008) D. Nagy, G. Szirmai, and P. Domokos, The European Physical Journal D 48, 127 (2008).
  * Nagy  _et al._ (2011) D. Nagy, G. Szirmai, and P. Domokos, Physical Review A 84, 043637 (2011).
  * Lang  _et al._ (2017) J. Lang, F. Piazza, and W. Zwerger, New Journal of Physics 19, 123027 (2017).
  * Strack and Sachdev (2011) P. Strack and S. Sachdev, Phys. Rev. Lett. 107, 277202 (2011).
  * Angelone  _et al._ (2016) A. Angelone, F. Mezzacapo, and G. Pupillo, Phys. Rev. Lett. 116, 135303 (2016).
  * Marsh  _et al._ (2024) B. P. Marsh, R. M. Kroeze, S. Ganguli, S. Gopalakrishnan, J. Keeling, and B. L. Lev, Phys. Rev. X 14, 011026 (2024).
  * Hung  _et al._ (2016) C.-L. Hung, A. González-Tudela, J. I. Cirac, and H. Kimble, Proceedings of the National Academy of Sciences 113, E4946 (2016).
  * Nguyen  _et al._ (2017) J. H. V. Nguyen, D. Luo, and R. G. Hulet, Science 356, 422 (2017).
  * Lim  _et al._ (2021) Y. Lim, J. Goo, H. Kwak, and Y. Shin, Phys. Rev. A 103, 063319 (2021).
  * Krämer  _et al._ (2018) S. Krämer, D. Plankensteiner, L. Ostermann, and H. Ritsch, Computer Physics Communications 227, 109 (2018).
  * Maschler  _et al._ (2008) C. Maschler, I. B. Mekhov, and H. Ritsch, The European Physical Journal D 46, 545 (2008).