aainstitutetext: School of Natural Sciences, Institute for Advanced Study,
Einstein Drive, Princeton, NJ 08540 USAbbinstitutetext: Department of
Theoretical Physics, CERN,
1211 Geneva 23, Switzerland

# Miura operators as R-matrices from M-brane intersections

Nathan Haouzi b and Saebyeok Jeong<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

We propose that Miura operators are R-matrices of certain infinite-dimensional
quantum algebras. We test our proposal by realizing Miura operators of
$q$-deformed $W$\- and $Y$-algebras in terms of R-matrices of the quantum
toroidal algebra of $\mathfrak{gl}(1)$. Physically, the representations of
this toroidal algebra arise from the algebra of local operators on M2-branes
and M5-branes, in M-theory subject to an $\Omega$-background. We associate an
R-matrix to each M2-M5 brane crossing, by studying its description as a gauge-
invariant intersection of a topological line defect and a holomorphic surface
defect in 5-dimensional non-commutative Chern-Simons theory. The Miura
transformation is engineered using multiple M2-M5 intersections, relying
crucially on the properties of the underlying R-matrices. We thereby identify
each R-matrix with a Miura operator. In a dual Type IIB frame, the components
of the Miura transformation are shown to coincide with the half-index of a 3d
supersymmetric gauge theory on a Hanany-Witten system of D3-NS5 branes. As a
further application, we demonstrate that $qq$-characters can be algebraically
constructed from the Miura transformation.

††preprint: CERN-TH-2024-094

##  1 Introduction

In two-dimensional conformal field theory, the Miura transformation111In the
present work, the Miura transformation always refers to the quantum Miura
transformation, unless specified otherwise. is a procedure used to construct
the holomorphic generators of the theory in terms of free bosons. The most
celebrated example is the construction of the $\mathcal{W}_{N}$-algebra, a
higher spin generalization of the Virasoro vertex algebra consisting of $N$
generators (we add an additional free boson to the usual $W$-algebra of
$\mathfrak{sl}(N)$, and call it the $W$-algebra of $\mathfrak{gl}(N)$). In
this case, the Miura transformation states that the generating currents
$U_{k}(z)$ of the $\mathcal{W}_{N}$-algebra is constructed by

$({\varepsilon}_{3}\partial_{z}-{\varepsilon}_{1}{\varepsilon}_{2}J_{N}(z))({\varepsilon}_{3}\partial_{z}-{\varepsilon}_{1}{\varepsilon}_{2}J_{N-1}(z))\cdots({\varepsilon}_{3}\partial_{z}-{\varepsilon}_{1}{\varepsilon}_{2}J_{1}(z))=\sum_{k=0}^{N}U_{k}(z)\left({\varepsilon}_{3}\partial_{z}\right)^{N-k},$
(1)

where $U_{0}(z)=1$. Here,
$({\varepsilon}_{1},{\varepsilon}_{2},{\varepsilon}_{3})\in\mathbb{C}^{3}$,
constrained by ${\varepsilon}_{1}+{\varepsilon}_{2}+{\varepsilon}_{3}=0$,
parametrize the central charge by
$c=N\left(1+(N^{2}-1)\left(\frac{{\varepsilon}_{2}}{{\varepsilon}_{1}}+\frac{{\varepsilon}_{1}}{{\varepsilon}_{2}}+2\right)\right)$.
The $J_{i}(z)$ is the current of the $\widehat{\mathfrak{gl}}(1)$ vertex
algebra with the operator product expansion (OPE) given by
$J_{i}(z)J_{j}(w)\sim-\frac{1}{{\varepsilon}_{1}{\varepsilon}_{2}}\frac{1}{(z-w)^{2}}\delta_{i,j}$,
whose mode algebra is the free boson algebra. Each factor
${\varepsilon}_{3}\partial_{z}-{\varepsilon}_{1}{\varepsilon}_{2}J_{i}(z)$ on
the left-hand side of (1) is called a _Miura operator_.

The $\mathcal{W}_{N}$-algebra arises as the vertex algebra of local operators
of a 6-dimensional $\mathcal{N}=(2,0)$ superconformal field theory subject to
the $\Omega$-background. This is the 6-dimensional origin of the celebrated
correspondence between $\mathcal{W}_{N}$-algebra conformal blocks and
partition functions of 4-dimensional $\mathcal{N}=2$ supersymmetric $U(N)$
gauge theories Alday:2009aq ; Wyllard:2009hg ; Yagi:2012xa ; Beem:2014kka .

In a trivial background, the 6d theory is the low energy effective field
theory on $N$ M5-branes in M-theory. This prompted Costello to define a
twisted version of M-theory on the $\Omega$-background Costello:2016nkh ,
where $N$ parallel M5-branes are reinterpreted as a surface defect in
5-dimensional non-commutative $\mathfrak{gl}(1)$ Chern-Simons theory. The
$\mathcal{W}_{N}$-algebra is then viewed as the vertex algebra of local
operators on the surface defect whose mode algebra acts on the Hilbert space
of this surface defect theory, which is identified with the equivariant
cohomology of the moduli space of $U(N)$ instantons on $\mathbb{R}^{4}$. The
configuration can be augmented to incorporate M5-branes oriented in different
directions, extending the relevant vertex algebra to the $Y$-algebra
Gaiotto:2017euk ; Gaiotto:2020dsq .

In this twisted M-theory setting, it was proposed that the Miura operator for
the $\mathcal{W}_{N}$-algebra should be realized as the transverse
intersection of an M2-brane and an M5-brane Gaiotto:2020dsq . It was observed
that this intersection point supports a fermionic zero-mode, with the
classical expectation value given by the distance between the two branes in
the transverse holomorphic plane. Quantum corrections are then suggested to
uplift this expectation value to
${\varepsilon}_{3}\partial_{z}-{\varepsilon}_{1}{\varepsilon}_{2}J(z)$, the
Miura operator for the $\mathcal{W}_{N}$-algebra. This twisted M-theory
approach was further extended to include pseudo-differential Miura operators
for more general $Y$-algebras Prochazka:2018tlo ; Gaiotto:2020dsq ,
incorporating non-transverse intersections of the M2- and M5-branes that
partially overlap in their worldvolume.

Meanwhile, a definition of a multiplicative uplift of the $Y$-algebra was
recently proposed in Harada:2021xnm , in a purely algebraic framework. In
particular, it was shown that Miura operators for a $q$-deformed $Y$-algebra
could be defined in terms of certain $q$-difference operators, replacing the
previous differential operators. It is tempting to guess that these Miura
operators should likewise have a physical interpretation as brane
intersections in twisted M-theory. One of the main goals of this work is to
make this intuition precise, and prove that the algebraic Miura operators of
Harada:2021xnm  are exactly supported on M2-M5 intersections in the
multiplicative uplift of the twisted M-theory background considered in
Gaiotto:2020dsq  (up to a slight redefinition we will motivate).

We will start by reviewing the twisted M-theory and 5-dimensional non-
commutative Chern-Simons theory Costello:2016nkh . Then, we will introduce the
central idea of the present work: that the M2-M5 intersection can be
understood as an _R-matrix_.

###  1.1 Twisted M-theory and 5-dimensional Chern-Simons theory

Let us consider the M-theory defined on the 11-dimensional worldvolume
$\mathbb{R}^{7}\times\mathcal{C}$, where $\mathcal{C}$ is a complex two-
dimensional manifold to be chosen. Three main cases are
$\mathcal{C}=\mathbb{C}\times\mathbb{C}$;
$\mathcal{C}=\mathbb{C}\times\mathbb{C}^{\times}$; and
$\mathcal{C}=\mathbb{C}^{\times}\times\mathbb{C}^{\times}$. While we will
mostly focus on the last case throughout the work, it is helpful to treat them
all at once here to illustrate their differences.

With the flat metric on the 11-dimensional worldvolume, there is a nilpotent
supercharge $\mathcal{Q}_{{\varepsilon}_{1}={\varepsilon}_{2}=0}$, under which
$\mathbb{R}^{7}$ is topological and $\mathcal{C}$ is holomorphic in its
cohomology. Now, we further introduce the 3-form background
$C^{(3)}=({\varepsilon}_{1}V_{1}^{\flat}+{\varepsilon}_{2}V_{2}^{\flat})\wedge\bar{\omega}$,
where $V_{i}=\partial_{\varphi_{i}}-\partial_{\varphi_{3}}$, $i=1,2$, are the
vector fields generating the rotations on the three $\mathbb{R}^{2}$-planes
($\varphi_{i}$ is the angular coordinate for the $i$-th
$\mathbb{R}^{2}$-plane) and $V_{i}^{\flat}$ are the dual 1-forms. Also,
$\bar{\omega}$ is a $(0,2)$-form on $\mathcal{C}$ given by the complex
conjugate of the holomorphic symplectic $(2,0)$-form $\omega$,

$\displaystyle\bar{\omega}=\begin{cases}d\bar{x}\wedge
d\bar{z}\qquad\qquad&\mathcal{C}=\mathbb{C}_{x}\times\mathbb{C}_{z}\\\
d\bar{x}\wedge\frac{d\bar{Z}}{\bar{Z}}&\mathcal{C}=\mathbb{C}_{x}\times\mathbb{C}^{\times}_{Z}\\\
\frac{d\bar{X}}{\bar{X}}\wedge\frac{d\bar{Z}}{\bar{Z}}&\mathcal{C}=\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}\end{cases},$
(2)

where we indicated the holomorphic coordinates on the complex planes by
subscripts. At this point, let us also write the topological part of the
worldvolume as
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}^{2}_{{\varepsilon}_{3}}\times\mathbb{R}_{t}$,
where we used the subscripts to invoke the background parameters associated to
the rotations on the respective $\mathbb{R}^{2}$-planes
$({\varepsilon}_{3}=-{\varepsilon}_{1}-{\varepsilon}_{2})$ and $t$ is the
coordinate on the remaining topological line $\mathbb{R}$. In the presence of
the 3-form background, the preserved supercharge gets deformed to
$\mathcal{Q}_{{\varepsilon}_{1},{\varepsilon}_{2}}$, which squares to the
isometry

$\displaystyle\mathcal{Q}_{{\varepsilon}_{1},{\varepsilon}_{2}}^{2}={\varepsilon}_{1}V_{1}+{\varepsilon}_{2}V_{2}.$
(3)

The M-theory subject to the
$\mathcal{Q}_{{\varepsilon}_{1},{\varepsilon}_{2}}$-cohomology is called the
twisted M-theory Costello:2016nkh .

Since the supersymmetry squares to the rotations of the
$\mathbb{R}^{2}$-planes, the twisted M-theory localizes to an effective
5-dimensional theory on the fixed locus
$\\{0\\}\times\mathbb{R}_{t}\times\mathcal{C}$. The localization procedure can
be understood in the following way. We may continuously deform the flat
$\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}^{2}_{{\varepsilon}_{3}}$
to the single-centered Taub-NUT space. Reducing the M-theory along the circle
fiber of the Taub-NUT space, we arrive at the IIA theory with an emergent
D6-brane supported on
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}_{t}\times\mathcal{C}$ at
the location of the center. The effective field theory on the worldvolume of
the D6-brane is the 7-dimensional supersymmetric $U(1)$ gauge theory. The role
of the 3-form background $C^{(3)}$ is two-fold. First, it descends to a
B-field which is a mixture of the back-reaction of the emergent D6-brane and
the constant B-field background. The back-reaction of the D6-brane can be
subtracted by taking its asymptotics at infinity, leaving
$B={\varepsilon}_{2}\bar{\omega}$. Because of this constant B-field, the usual
wedge product of the differential forms in the 7d effective gauge theory is
deformed to the non-commutative Moyal product defined by

$\displaystyle
f\star_{{\varepsilon}_{2}}g=\sum_{n=0}^{\infty}\frac{{\varepsilon}_{2}^{n}}{2^{n}n!}\epsilon_{i_{1}j_{1}}\cdots\epsilon_{i_{n}j_{n}}\left(\frac{\partial}{\partial
z_{i_{1}}}\cdots\frac{\partial}{\partial
z_{i_{n}}}f\right)\wedge\left(\frac{\partial}{\partial
z_{j_{1}}}\cdots\frac{\partial}{\partial z_{j_{n}}}g\right),$ (4)

where $\epsilon_{ij}$ is the anti-symmetric symbol and $z_{1}=x$ or $\log X$
and $z_{2}=z$ or $\log Z$ depending on whether the corresponding complex plane
is $\mathbb{C}$ or $\mathbb{C}^{\times}$. Second, the 3-form also descends to
a RR 3-form background which, after subtracting the one sourced by the
D6-brane in the presence of the constant B-field, implements the
$\Omega$-background for the 7d effective gauge theory associated with the
rotation on $\mathbb{R}^{2}_{{\varepsilon}_{1}}$.

In total, the 7-dimensional gauge theory is localized into the 5-dimensional
non-commutative $\mathfrak{gl}(1)$ Chern-Simons theory on
$\mathbb{R}_{t}\times\mathcal{C}$, whose action is given by

$\displaystyle
S=\frac{1}{{\varepsilon}_{1}}\int_{\mathbb{R}_{t}\times\mathcal{C}}\omega\wedge\left(A\star_{{\varepsilon}_{2}}dA+\frac{2}{3}A\star_{{\varepsilon}_{2}}A\star_{{\varepsilon}_{2}}A\right),$
(5)

where $A$ is the complex-valued gauge field with three components

$\displaystyle
A=A_{t}dt+A_{\bar{z}_{1}}d\bar{z}_{1}+A_{\bar{z}_{2}}d\bar{z}_{2}.$ (6)

Here, $\bar{z}_{1,2}$ are the anti-holomorphic coordinates on $\mathcal{C}$.

M-branes | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10
---|---|---|---|---|---|---|---|---|---|---|---
$\text{M2}_{1,0,0}$ | x | x |  |  |  |  | x |  |  |  |
$\text{M2}_{0,1,0}$ |  |  | x | x |  |  | x |  |  |  |
$\text{M2}_{0,0,1}$ |  |  |  |  | x | x | x |  |  |  |
$\text{M5}_{1,0,0}^{\mathbf{(1,0)}}$ |  |  | x | x | x | x |  | x | x |  |
$\text{M5}_{0,1,0}^{\mathbf{(1,0)}}$ | x | x |  |  | x | x |  | x | x |  |
$\text{M5}_{0,0,1}^{\mathbf{(1,0)}}$ | x | x | x | x |  |  |  | x | x |  |
$\text{M5}_{0,0,1}^{\mathbf{(0,1)}}$ | x | x | x | x |  |  |  |  |  | x | x
$\text{M5}_{0,0,1}^{\mathbf{(p,q)}}$ | x | x | x | x |  |  |  | $C^{\mathbf{(p,q)}}$
Table 1: M-brane configuration in twisted M-theory on $\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}^{2}_{{\varepsilon}_{3}}\times\mathbb{R}_{t}\times\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$. The M5-brane $\text{M5}_{c}^{\mathbf{(p,q)}}$ is supported on $\mathbb{R}^{2}_{{\varepsilon}_{c+1}}\times\mathbb{R}^{2}_{{\varepsilon}_{c-1}}\times C^{\mathbf{(p,q)}}$ where $C^{\mathbf{(p,q)}}=\\{X^{\mathbf{q}}Z^{-\mathbf{p}}=\text{const}\\}\subset\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$. $\mathbb{R}^{2}_{{\varepsilon}_{1}}$ | $\mathbb{R}^{2}_{{\varepsilon}_{2}}$ | $\mathbb{R}^{2}_{{\varepsilon}_{3}}$ | $\mathbb{R}_{t}$ | $\mathbb{C}^{\times}_{X}$ | $\mathbb{C}^{\times}_{Z}$
---|---|---|---|---|---
$x^{0},x^{1}$ | $x^{2},x^{3}$ | $x^{4},x^{5}$ | $x^{6}$ | $x^{7},x^{8}$ | $x^{9},x^{10}$
Table 2: Spacetime of the twisted M-theory

The M-branes in the twisted M-theory engineer defects in the 5d Chern-Simons
theory. We may introduce M2-branes, yielding topological line defects; and
M5-branes, producing holomorphic surface defects (see Table 1 and 2, for our
main case of
$\mathcal{C}=\mathbb{C}_{X}^{\times}\times\mathbb{C}_{Z}^{\times}$).

  * •

The M2-branes can be supported on
$\mathbb{R}^{2}_{{\varepsilon}_{c}}\times\mathbb{R}_{t}$, $c\in\\{1,2,3\\}$,
yielding topological line defects of the 5d Chern-Simons theory on the line
$\mathbb{R}_{t}$. In the most general case, the line defect descends from
$(l,m,n)$ M2-branes supported on
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}_{t}$,
$\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}_{t}$, and
$\mathbb{R}^{2}_{{\varepsilon}_{3}}\times\mathbb{R}_{t}$, respectively. We
call this line defect $\mathcal{L}_{l,m,n}$. The local operators on this line
defect form a non-commutative associative algebra defined by their OPEs, which
we denote by $\text{M2}_{l,m,n}$.

  * •

The M5-branes can be supported on
$\mathbb{R}^{2}_{{\varepsilon}_{c+1}}\times\mathbb{R}^{2}_{{\varepsilon}_{c-1}}\times
C$, $c\in\\{1,2,3\\}$, where $C\subset\mathcal{C}$ is a holomorphic curve.
They induce holomorphic surface defects of the 5d Chern-Simons theory lying on
the curve $C$. In the most general case, the surface defect descends from
$(L,M,N)$ M5-branes supported on
$\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}^{2}_{{\varepsilon}_{3}}\times
C$,
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}^{2}_{{\varepsilon}_{3}}\times
C$, and
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}^{2}_{{\varepsilon}_{2}}\times
C$, respectively. We call this surface defect $\mathcal{S}_{L,M,N}^{C}$. The
local operators on this surface defect form a chiral algebra defined by their
OPEs. We denote its mode algebra by $\text{M5}_{L,M,N}^{C}$.222In this work,
we restrict our attention to the mode algebra of local operators on the
holomorphic surface defect. In our multiplicative setting, where
$\mathcal{C}=\mathbb{C}^{\times}\times\mathbb{C}^{\times}$, the chiral algebra
on the surface defect is anticipated to be a quantum vertex algebra.
Consequently, the associated surjective map $\rho_{\text{M5}_{L,M,N}^{C}}$ is
expected to extend to a quantum vertex algebra homomorphism. See footnote 20.

The coupling of the defect is built by applying a descent procedure to a
combination of local operators on the defect and local operators of the 5d
Chern-Simons theory at the locus of the defect (modes of the ghost
field).333In fact, the descent procedure for the coupling of the surface
defect requires more care, since we need to take account of the back-reaction
of the M5-branes sourcing a singularity of the fields. See Costello:2016nkh .
The BRST-invariance of this combination requires the algebra of local
operators on the defect to be a representation of a certain universal
associative algebra, which we generically denote by
$\mathcal{A}_{{\varepsilon}_{1},{\varepsilon}_{2}}$. Namely, there exist
surjective algebra homomorphisms,

$\displaystyle\begin{split}&\rho_{\text{M2}_{l,m,n}}:\mathcal{A}_{{\varepsilon}_{1},{\varepsilon}_{2}}\twoheadrightarrow\text{M2}_{l,m,n},\qquad\quad\rho_{\text{M5}_{L,M,N}^{C}}:\mathcal{A}_{{\varepsilon}_{1},{\varepsilon}_{2}}\twoheadrightarrow\text{M5}_{L,M,N}^{C},\end{split}$
(7)

for the topological line defect from the M2-branes and the holomorphic surface
defect from the M5-branes, respectively.

The same kind of defects may lie on top of each other, defining a new defect
by a fusion. The coupling of the fused defect should be built by the couplings
of the individual defects before the fusion operation. Since the coupling is
encoded in the representation (7) of the universal associative algebra carried
by the defect, the fusion operation implies that there must be an algebra
homomorphism,

$\displaystyle\Delta:\mathcal{A}_{{\varepsilon}_{1},{\varepsilon}_{2}}\to\mathcal{A}_{{\varepsilon}_{1},{\varepsilon}_{2}}\,\widehat{\otimes}\,\mathcal{A}_{{\varepsilon}_{1},{\varepsilon}_{2}},$
(8)

from which the the representation assigned to the fused defect would be
reconstructed by composing $\Delta$ with the respective representations
assigned to the individual defects before the fusion. The completed tensor
product was used so that the image may be given as a sum of infinitely many
terms. This algebra homomorphism, called the coproduct, equips the universal
associative algebra $\mathcal{A}_{{\varepsilon}_{1},{\varepsilon}_{2}}$ with a
coalgebra structure (namely,
$\mathcal{A}_{{\varepsilon}_{1},{\varepsilon}_{2}}$ is a bialgebra).444The
coassociativity
$(\Delta\otimes\text{id})\Delta=(\text{id}\otimes\Delta)\Delta$ follows from
the independence of the ordering of two consecutive fusion operations. The
existence of the counit
$\epsilon:\mathcal{A}_{{\varepsilon}_{1},{\varepsilon}_{2}}\to\mathbb{C}$,
satisfying
$(\text{id}\otimes\epsilon)\Delta=(\epsilon\otimes\text{id})\Delta=\text{id}$,
is guaranteed by the existence of the trivial defect. See section 2 for more
details about the fusion operations of the line defects and the surface
defects in our main case
$\mathcal{C}=\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$.

| $\mathbb{C}\times\mathbb{C}$ | $\mathbb{C}\times\mathbb{C}^{\times}$ | $\mathbb{C}^{\times}\times\mathbb{C}^{\times}$
---|---|---|---
Universal associative algebra | $Y_{1}(\widehat{\mathfrak{gl}}(1))$ | $Y(\widehat{\mathfrak{gl}}(1))$ | $U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$
Parallel M2-branes | $\mathbf{Sr\ddot{H}}$ | $\mathbf{St\ddot{H}}$ | $\text{\bf{S}}\boldsymbol{\ddot{\text{\bf{H}}}}$
Non-parallel M2-branes | Generalized | Generalized | Generalized
rat. Calogero | trig. Calogero | Macdonald
Parallel M5-branes | $\mathcal{W}^{\leq 0}$ |  $\mathcal{W}$ or $\hbar\mathcal{W}^{\leq 0}$ | $q\mathcal{W}$
Non-parallel M5-branes | $Y^{\leq 0}$ |  $Y$ or $\hbar Y^{\leq 0}$ | $qY$
Table 3: Universal associative algebra for 5d $\mathfrak{gl}(1)$ Chern-Simons
theory on $\mathbb{R}_{t}\times\mathcal{C}$, where
$\mathcal{C}=\mathbb{C}\times\mathbb{C}$;
$\mathcal{C}=\mathbb{C}\times\mathbb{C}^{\times}$; or
$\mathcal{C}=\mathbb{C}^{\times}\times\mathbb{C}^{\times}$, and their
representations formed by the algebra of local operators on topological line
defects and holomorphic surface defects from M2- and M5-branes. Here,
$\mathbf{S\ddot{H}}$ is the spherical double affine Hecke algebra (spherical
DAHA; see section 2.1.2), while $\mathbf{Sr\ddot{H}}$ and
$\mathbf{St\ddot{H}}$ are its rational and trigonometric degeneration,
respectively. Also, $qY$ (resp. $\hbar Y$) stands for the $q$-deformed (resp.
$\hbar$-deformed) $Y$-algebra. We will discuss the $\hbar$-deformed
$Y$-algebra in a separate work. We mainly focus on the case of
$\mathbb{C}^{\times}\times\mathbb{C}^{\times}$ (the last column) in the
present work.

Since the 5d Chern-Simons theory depends on the choice of the complex manifold
$\mathcal{C}$, the universal associative algebra also depends on it (see table
3). Previous studies have primarily focused on the cases where
$\mathcal{C}=\mathbb{C}\times\mathbb{C}$ and
$\mathcal{C}=\mathbb{C}\times\mathbb{C}^{\times}$ Costello:2016nkh ;
Costello:2017fbo ; Gaiotto:2019wcc ; Gaiotto:2020vqj ; Gaiotto:2020dsq . For
these cases, it was shown that the universal associative algebra is the
1-shifted affine Yangian of $\mathfrak{gl}(1)$
($Y_{1}(\widehat{\mathfrak{gl}}(1))$) and the affine Yangian of
$\mathfrak{gl}(1)$ ($Y(\widehat{\mathfrak{gl}}(1))$, respectively.
Correspondingly, the algebra of local operators on the line defect from
M2-branes is given by the (generalized) rational (resp. trigonometric)
Calogero representation of $Y_{1}(\widehat{\mathfrak{gl}}(1))$ (resp.
$Y(\widehat{\mathfrak{gl}}(1))$). The algebra of local operators on the
surface defect from M5-branes was shown to be the $Y$-algebra Gaiotto:2017euk
.

In this work, we examine the case where
$\mathcal{C}=\mathbb{C}^{\times}\times\mathbb{C}^{\times}$. We propose that
the universal associative algebra in this setting is the quantum toroidal
algebra of $\mathfrak{gl}(1)$, denoted by
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$. We claim that
the algebra of local operators $\text{M2}_{l,m,n}$ on the line defect,
originating from M2-branes, is given by the spherical double affine Hecke
algebra (spherical DAHA) and its generalization (see section 2.1).
Furthermore, we assert that the mode algebra $\text{M5}_{L,M,N}^{C}$ of local
operators on the surface defect, originating from M5-branes, is the
$q$-deformed $Y$-algebra (see section 2.2).555A qualitative argument for the
appearance of $q$-deformed algebras is as follows: after replacing
$\mathbb{C}\times\mathbb{C}^{\times}$ with
$\mathbb{C}^{\times}\times\mathbb{C}^{\times}$, there is now a circle
transverse to all $\text{M5}^{\mathbf{(1,0)}}$-branes: the radius of this
circle is a finite scale which breaks the $\mathcal{N}=(2,0)$ superconformal
symmetry of the 6-dimensional theory on the M5-branes; the resulting theory is
known as the $\mathcal{N}=(2,0)$ little string theory Seiberg:1997zk ;
Losev:1997hx . The conformal symmetry of the chiral algebra is likewise broken
by the radius scale: when all M5-branes are parallel to each other, the
conformal $\mathcal{W}$-algebra was shown to deform to a
$q\mathcal{W}$-algebra Aganagic:2015cta ; Haouzi:2017vec . In our work, we
will encounter an arbitrary configuration of
$\text{M5}^{\mathbf{(1,0)}}$-branes, with different orientations, and the
associated chiral $Y$-algebra is therefore expected to deform to the
$qY$-algebra.

###  1.2 M2-M5 intersections as R-matrices and Miura operators

In the present work, we establish the Miura transformation for the algebra of
local operators on the surface defect, called the $q$-deformed $W$\- and $Y$\-
algebras, by the fusion of the intersections between a single M2-brane and
multiple M5-branes. The most crucial point is to realize the M2-M5
intersection as an R-matrix of the universal associative algebra and to
utilize its properties. Since the idea penetrates the central theme of the
present work, we will elucidate how the M2-M5 intersection is associated with
an R-matrix here.

Consider the configuration where M2-branes and M5-branes are present
simultaneously. To develop the argument in a universal manner, we denote a
generic line defect engineered by the M2-branes simply as $\mathcal{L}$, and
the algebra of local operators on the line defect by M2. Similarly, we denote
a generic surface defect engineered by the M5-branes as $\mathcal{S}$, and the
mode algebra of local operators on the surface defect by M5. The corresponding
surjective algebra homomorphisms (7) from the universal associative algebra
$\mathcal{A}_{{\varepsilon}_{1},{\varepsilon}_{2}}$ are simply called
$\rho_{\mathcal{L}}$ and $\rho_{\mathcal{S}}$. The holomorphic support of the
M5-branes are assumed to be identical, say $\mathbb{C}^{\times}_{X}$, so that
they define a single surface defect in the 5d Chern-Simons theory. The
intersection between the line defect $\mathcal{L}$ and the surface defect
$\mathcal{S}$ can then be visualized as shown in Figure 1.

$R$$\times$$0$$\mathcal{S}$$\mathcal{L}$$\mathcal{L}$$\mathbb{C}^{\times}_{X}$$\mathbb{R}_{t}$
Figure 1: M2-M5 intersection as an R-matrix. Classically, the gauge variation
of the local operator $R$ (black dot) at the intersection receives linear
contributions from the line defect (red dots) and the surface defect (red
line). The contribution from the surface defect is decomposed into a linear
combination of two: one from inward and the other from outward (blue dotted
lines).

The intersection supports a space of local operators. A local operator $R$ at
the intersection can be built from the local operators on the line defect and
the surface defect, so that it is an element of the completed tensor product

$\displaystyle R\in\text{M2}\,\widehat{\otimes}\,\text{M5},$ (9)

subject to the gauge-invariance condition. Equivalently, we may view the local
operator $R$ as acting on the space $\mathcal{V}\otimes\mathcal{F}$, where
$\mathcal{V}$ is the Hilbert space of the line defect worldline theory
attached to the boundary at the negative infinity of $\mathbb{R}_{t}$ and
$\mathcal{F}$ is the Hilbert space of the surface defect theory attached to
the boundary at the origin of $\mathbb{C}^{\times}_{X}$. The result of this
action is another state in the same space attached to the positive infinity of
$\mathbb{R}_{t}$ and the infinity of $\mathbb{C}^{\times}_{X}$, respectively.
Thus, we recognize that
$R\in\text{End}(\mathcal{V})\,\widehat{\otimes}\,\text{End}(\mathcal{F})=\text{M2}\,\widehat{\otimes}\,\text{M5}$.

The gauge variation of the local operator $R$ gets contributions from both the
line defect and the surface defect, which have to be cancelled with each
other. At the classical limit ${\varepsilon}_{1}=0$, there are only linear
contributions from the two defects. These contributions can be visualized by
drawing a small sphere surrounding the intersection point, picking up the
gauge transformations whenever the sphere intersects the defects.

Let $g\in\mathcal{A}_{{\varepsilon}_{1}=0,{\varepsilon}_{2}}$ be the element
paired with a fixed mode of the ghost field. The line defect contribute
$\rho_{\mathcal{L}}(g)$ from both above and below, ordered in the increasing
$\mathbb{R}_{t}$-direction. The surface defect contributes by a contour
integral of the generating current of the chiral algebra along the small
circle around the intersection point. The contour is decomposed into the
linear combination of two encircling the origin, radially ordered together
with the intersection point itself. Thus, the contour integral produces two
modes $\rho_{\mathcal{S}}(g)$ as well, acting from the inward and the outward
in the radial ordering. In total, the classical gauge-invariance condition
reads

$\displaystyle
R\left(\rho_{\mathcal{L}}(g)\otimes\text{id}+\text{id}\otimes\rho_{\mathcal{S}}(g)\right)=\left(\rho_{\mathcal{L}}(g)\otimes\text{id}+\text{id}\otimes\rho_{\mathcal{S}}(g)\right)R,$
(10)

for any $g\in\mathcal{A}_{{\varepsilon}_{1}=0,{\varepsilon}_{2}}$.

Turning on ${\varepsilon}_{1}\neq 0$, the gauge-invariance condition gets non-
linear quantum corrections. We postulate that the quantum corrections respect
the algebra structure, uplifting the gauge-invariance condition to666In the
case of $\mathcal{C}=\mathbb{C}\times\mathbb{C}$ or
$\mathcal{C}=\mathbb{C}\times\mathbb{C}^{\times}$, the support of the
holomorphic surface defect $\mathcal{S}$ can be chosen to be $\mathbb{C}$
instead of $\mathbb{C}^{\times}$. In this case, we may consider the local
operator $R$ acting on the vacuum $|\varnothing\rangle\in\mathcal{F}$ of the
Hilbert space of the surface defect theory; namely,
$R(1\otimes|\varnothing\rangle)$. This can be thought of as the effect of
capping $\mathbb{C}^{\times}$ to $\mathbb{C}$. The gauge-invariance condition
then reads
$\displaystyle\left(R^{(c)}(1\otimes|\varnothing\rangle)\right)\rho_{\mathcal{L}}(g)=\Delta^{\text{op}}_{\text{M2},\text{M5}}(g)\left(R^{(c)}(1\otimes|\varnothing\rangle)\right).$
Up to the exchange of the two factors, this recovers the gauge-invariance
condition assigned to the M2-M5 intersection in Gaiotto:2020dsq . We suggest
that the gauge-invariance condition should be extended to (11), without having
$R$ act on the vacuum in particular, when the support of the holomorphic
surface defect is $\mathbb{C}^{\times}$.

$\displaystyle\boxed{R\,\Delta_{\text{M2},\text{M5}}(g)=\Delta^{\text{op}}_{\text{M2},\text{M5}}(g)R,\qquad\text{for
any }g\in\mathcal{A}_{{\varepsilon}_{1},{\varepsilon}_{2}}.}$ (11)

Here, we defined the mixed coproduct by composing the coproduct (8) of the
universal associative algebra
$\mathcal{A}_{{\varepsilon}_{1},{\varepsilon}_{2}}$ and the representations
(7) assigned to the M2- and M5-branes,

$\displaystyle\begin{split}&\Delta_{\text{M2},\text{M5}}:=(\rho_{\mathcal{L}}\otimes\rho_{\mathcal{S}})\Delta\quad:\quad
U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\to\text{M2}\,\widehat{\otimes}\,\text{M5},\end{split}$
(12)

and $\Delta^{\text{op}}=\sigma\circ\Delta$ is the opposite coproduct obtained
by composing the exchange operator $\sigma(g_{1}\otimes g_{2})=g_{2}\otimes
g_{1}$.

Further, let us consider the next-to-simplest configurations involving two
intersections: $(a)$ one line defect $\mathcal{L}$ passing through two surface
defects $\mathcal{S}$ and $\mathcal{S}^{\prime}$ (see Figure 2); $(b)$ two
line defects $\mathcal{L}$ and $\mathcal{L}^{\prime}$ passing through one
surface defect $\mathcal{S}$ (see Figure 3).

In the first case, the two surface defects $\mathcal{S}$ and
$\mathcal{S}^{\prime}$, located at different points in $\mathbb{R}_{t}$, can
approach each other and fuse into a new surface defect
$\mathcal{S}\circ\mathcal{S}^{\prime}$. Accordingly, the intersections between
the line defect and the two surface defects undergo a fusion, implying that
the local operator $R_{\mathcal{L},\mathcal{S}\circ\mathcal{S}^{\prime}}$ at
the intersection between the line defect and the fused surface defect should
simply be the product of the local operators $R_{\mathcal{L},\mathcal{S}}$ and
$R_{\mathcal{L},\mathcal{S}^{\prime}}$ assigned to the individual
intersections before the fusion. Namely, we have

$\displaystyle
R_{\mathcal{L},\mathcal{S}\circ\mathcal{S}^{\prime}}=R_{\mathcal{L},\mathcal{S}^{\prime}}R_{\mathcal{L},\mathcal{S}},$
(13)

where the product in the right hand side is taken as elements in the algebra
M2 of local operators on $\mathcal{L}$.

$R_{\mathcal{L},\mathcal{S}^{\prime}}$$R_{\mathcal{L},\mathcal{S}}$$\times$$0$$\times$$0$$\mathcal{S}^{\prime}$$\mathcal{S}$$\mathcal{L}$$\mathcal{L}$$\mathbb{C}^{\times}_{X}$$\mathbb{R}_{t}$

fusion
$R_{\mathcal{L},\mathcal{S}\circ\mathcal{S}^{\prime}}$$\times$$0$$\mathcal{S}\circ\mathcal{S}^{\prime}$$\mathcal{L}$$\mathcal{L}$

Figure 2: Fusion of intersections between one line defect and two surface
defects

Similarly, in the second case the two line defects $\mathcal{L}$ and
$\mathcal{L}^{\prime}$, located at different points in
$\mathbb{C}^{\times}_{X}$, can approach each other to be fused into the a new
line defect $\mathcal{L}^{\prime}\circ\mathcal{L}$. The intersections also
fuse correspondingly, implying that the local operator
$R_{\mathcal{L}^{\prime}\circ\mathcal{L},\mathcal{S}}$ supported at the fused
intersection is the simple product of the local operators
$R_{\mathcal{L},\mathcal{S}}$ and $R_{\mathcal{L}^{\prime},\mathcal{S}}$
attached at the individual intersections before the fusion:

$\displaystyle
R_{\mathcal{L}^{\prime}\circ\mathcal{L},\mathcal{S}}=R_{\mathcal{L}^{\prime},\mathcal{S}}R_{\mathcal{L},\mathcal{S}},$
(14)

where the product is taken as elements in the mode algebra M5 of the local
operators on the surface defect $\mathcal{S}$.

$R_{\mathcal{L},\mathcal{S}}$$R_{\mathcal{L}^{\prime},\mathcal{S}}$$\times$$0$$\mathcal{S}$$\mathcal{L}$$\mathcal{L}$$\mathcal{L}^{\prime}$$\mathcal{L}^{\prime}$$\mathbb{C}^{\times}_{X}$$\mathbb{R}_{t}$

fusion
$R_{\mathcal{L}^{\prime}\circ\mathcal{L},\mathcal{S}}$$\times$$0$$\mathcal{S}$$\mathcal{L}^{\prime}\circ\mathcal{L}$$\mathcal{L}^{\prime}\circ\mathcal{L}$

Figure 3: Fusion of intersections between two line defects and one surface
defect

All in all, we discover that the local operator supported at the intersection
should obey the constraint (11) and behave as (13) and (14) under the fusion.
We claim that these are precisely the manifestation of the R-matrix
properties. This is most apparent when there exists the universal R-matrix
$\mathcal{R}\in\mathcal{A}_{{\varepsilon}_{1},{\varepsilon}_{2}}\,\widehat{\otimes}\,\mathcal{A}_{{\varepsilon}_{1},{\varepsilon}_{2}}$
of the universal associative algebra, whose defining relations are

$\displaystyle\mathcal{R}\Delta(g)=\Delta^{\text{op}}(g)\mathcal{R},\qquad\text{for
any }g\in\mathcal{A}_{{\varepsilon}_{1},{\varepsilon}_{2}},$ (15a)
$\displaystyle(\text{id}\otimes\Delta)\mathcal{R}=\mathcal{R}_{13}\mathcal{R}_{12}$
(15b)
$\displaystyle(\Delta\otimes\text{id})\mathcal{R}=\mathcal{R}_{13}\mathcal{R}_{23}.$
(15c)

It is now straightforward that (11) is (15a) composed with
$\rho_{\mathcal{L}}\otimes\rho_{\mathcal{S}}$; (13) is (15b) composed with
$\rho_{\mathcal{L}}\otimes\rho_{\mathcal{S}}\otimes\rho_{\mathcal{S}^{\prime}}$;
and (14) is (15c) composed with
$\rho_{\mathcal{L}^{\prime}}\otimes\rho_{\mathcal{L}}\otimes\rho_{\mathcal{S}}$.
In our case of
$\mathcal{C}=\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$, we
suggested that the universal associative algebra is the quantum toroidal
algebra of $\mathfrak{gl}(1)$,
$\mathcal{A}_{{\varepsilon}_{1},{\varepsilon}_{2}}=U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$.
Following from its quantum double construction, the quantum toroidal algebra
of $\mathfrak{gl}(1)$ does possess the universal R-matrix (see section 3.3.1
for more detail). Let us remark that the suggested relation between the
R-matrix and the M2-M5 intersection can be directly verified within the
perturbation theory of the 5d non-commutative Chern-Simons theory jeongwip .

As we emphasized earlier, the R-matrix properties of the M2-M5 intersections
established here are key to achieving the Miura transformation for the algebra
$\text{M5}_{L,M,N}^{C}$ of local operators on the most general surface defect
$\mathcal{S}_{L,M,N}^{C}$. In particular, the axiom (15b) implies that the
R-matrix for the intersection between the line defect $\mathcal{L}_{0,0,1}$
and the surface defect $\mathcal{S}_{L,M,N}^{C}$ is factorized into the
product of $L+M+N$ basic R-matrices, yielding

$\displaystyle\boxed{R_{\text{M2}_{0,0,1},\text{M5}_{L,M,N}^{C}}=R_{L+M+N}^{(c_{L+M+N})}\cdots
R_{2}^{(c_{2})}R_{1}^{(c_{1})}=\sum_{m=0}^{\infty}(-1)^{m}T_{m}(X_{1})q_{3}^{-mD_{X_{1}}}}$
(16)

where in the second equality we expanded the R-matrix as a series in the
difference operator $q_{3}^{-D_{X_{1}}}$, which is a generator of the algebra
$\text{M2}_{0,0,1}$. In section 3, we will verify that this expansion of the
R-matrix provides the Miura transformation for the algebra
$\text{M5}_{L,M,N}^{C}$. We thereby identify the basic R-matrix associated
with the intersection between a single M2-brane and a single M5-brane as the
Miura operator for the $q$-deformed $Y$-algebra. Moreover, we show that the
change of ordering of any two basic R-matrices results in isomorphic
$q$-deformed $Y$-algebras by a direct consequence of the universal Yang-Baxter
equation satisfied by the universal R-matrix.

###  1.3 Corner, $\mathbf{(p,q)}$-web, and gauge origami

The twisted M-theory is connected to different IIB theories through string
dualities. It is illuminating to examine how the intersecting M-branes are
transformed via these dualities and study their implications.

##### Chiral algebra at the corner

The M-theory on a toric Calabi-Yau 3-fold is dual to the $\mathbf{(p,q)}$-web
of fivebranes in the IIB theory, where the web diagram is given by the toric
diagram Leung:1997tw . Regarding
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}^{2}_{{\varepsilon}_{3}}$
as the toric Calabi-Yau 3-fold for the duality, the twisted M-theory is dual
to the $\mathbf{(p,q)}$-web composed of one NS5-brane, one D5-brane, and one
$(1,1)$-fivebrane, all of which are supported on a semi-infinite line on a
$\mathbb{R}^{2}$-plane joining at a single point (see Table 4 and 5). The
M5-branes are dualized to D3-branes, filling the faces between the fivebranes
and ending on them. The holomorphic part of their support remains the same
holomorphic curve $C\subset\mathcal{C}$.

In the case of $\mathcal{C}=\mathbb{C}\times\mathbb{C}^{\times}$, consider
$(L,M,N)$ M5-branes whose supports are
$\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}^{2}_{{\varepsilon}_{3}}\times\mathbb{C}^{\times}$,
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}^{2}_{{\varepsilon}_{3}}\times\mathbb{C}^{\times}$,
and
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{C}^{\times}$,
respectively. By the above duality with a proper choice of the
$SL(2,\mathbb{Z})$ frame of the IIB theory, they are mapped to the $(L,M,N)$
D3-branes filling the face between the NS5-brane and the $(1,1)$-fivebrane;
between the D5-brane and the $(1,1)$-fivebrane; and between the NS5-brane and
the D5-brane, respectively, while sharing $\mathbb{C}^{\times}$ in common. The
effective field theory on each stack is the 4d GL-twisted $\mathcal{N}=4$
gauge theory with the gauge group $U(L)$, $U(M)$, and $U(N)$, respectively,
making interfaces across codimension-one boundaries assigned by the
fivebranes. Since the path integral of the 4d GL-twisted gauge theory with a
(deformed) Neumann boundary passes to that of the analytically continued 3d
Chern-Simons theory Witten:2010cx ; Witten:2010zr , the configuration boils
down to an interface between two 3d Chern-Simons theories with the gauge
groups $U(N|L)$ and $U(M|L)$. The local operators on the two-dimensional
interface supported on $\mathbb{C}^{\times}$ form a chiral algebra $Y_{L,M,N}$
by their OPEs. This is how the $Y$-algebra was originally constructed as the
vertex algebra at the corner Gaiotto:2017euk . The M2-branes, which are
dualized to 1-branes lying on top of the fivebranes, provide degenerate
modules for this vertex algebra.

In the present work, instead, we mainly focus on the case of
$\mathcal{C}=\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$. The string
duality to the IIB theory operates as previously described, except for the
change in the worldvolume to
$\mathcal{C}=\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$. Let us re-
examine the D3-branes supported on $\mathbb{C}^{\times}_{X}$ in this setting.
The effective 4d theory for the D3-D3 open string now contains infinite
numbers of winding modes around the circle of
$\mathbb{C}^{\times}_{Z}\simeq\mathbb{R}_{z}\times S^{1}$, resulting in its
uplift to a 5d $\mathcal{N}=2$ gauge theory compactified on a circle. We
expect that the interfaces of these 5d $\mathcal{N}=2$ gauge theories provides
the multiplicative uplift of the corner vertex algebra. In this sense, it is
appropriate to refer to the algebra $\text{M5}_{L,M,N}^{C}$ of local operators
on the holomorphic surface defect $\mathcal{S}_{L,M,N}^{C}$ as the
$q$-deformed $Y$-algebra.777Even though we only restrict our attention to the
mode algebra, we expect that the $q$-deformed $Y$-algebra $qY_{L,M,N}$ can be
endowed with a quantum vertex algebra structure, and the surjective map
$\rho_{\text{M5}_{L,M,N}^{C}}:U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\twoheadrightarrow
qY_{L,M,N}$ would be uplifted to a quantum vertex algebra homomorphism. In
this sense, $qY_{L,M,N}$ would be called the quantum vertex algebra at the
corner. See footnote 20.

IIB branes | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
---|---|---|---|---|---|---|---|---|---|---
Fivebranes | $\vdash$ |  |  |  | x | x | x | x | x
1-branes | $\vdash$ |  |  |  | x |  |  |  |
$\text{D3}_{c}^{\mathbf{(1,0)}}$ | $\vdash$ | $\vdash$ |  |  |  |  | x | x |  |
$\text{D3}_{c}^{\mathbf{(0,1)}}$ | $\vdash$ | $\vdash$ |  |  |  |  |  |  | x | x
$\text{D3}_{c}^{\mathbf{(p,q)}}$ | $\vdash$ | $\vdash$ |  |  |  |  | $C^{\mathbf{(p,q)}}$
Table 4: IIB brane configuration for the corner. The symbol $\vdash$ indicates semi-infinite filling. $\mathbb{R}^{2}$ | $\mathbb{R}^{3}$ | $\mathbb{R}_{t}$ | $\mathbb{C}^{\times}_{X}$ | $\mathbb{C}^{\times}_{Z}$
---|---|---|---|---
$x^{0},x^{1}$ | $x^{2},x^{3},x^{4}$ | $x^{5}$ | $x^{6},x^{7}$ | $x^{8},x^{9}$
Table 5: Spacetime of the IIB theory for the corner.

#####  $\mathbf{(p,q)}$-web of fivebranes and 3d $\mathcal{N}=2$ half-indices

Note that the holomorphic part
$\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$ of the 11-dimensional
worldvolume contains a two-torus. The twisted M-theory on that torus is the
same theory as type IIB string theory on a circle, in the twisted
background.888Note this type IIB frame was not available in the works
Gaiotto:2017euk ; Prochazka:2018tlo ; Gaiotto:2020dsq , since the M-theory
background considered there is $\mathcal{C}=\mathbb{C}\times\mathbb{C}$ or
$\mathcal{C}=\mathbb{C}\times\mathbb{C}^{\times}$.

The M5-brane supported on
$\mathbb{R}^{2}_{{\varepsilon}_{c+1}}\times\mathbb{R}^{2}_{{\varepsilon}_{c-1}}\times
C^{\mathbf{(p,q)}}$ becomes the $\mathbf{(p,q)}$-fivebrane supported on
$\mathbb{R}^{2}_{{\varepsilon}_{c+1}}\times\mathbb{R}^{2}_{{\varepsilon}_{c-1}}\times
l^{\mathbf{(p,q)}}\times S^{1}$, where
$l^{\mathbf{(p,q)}}=\\{\mathbf{q}x-\mathbf{p}z=\text{const}\\}$ is a line in
the two-dimensional plane $\mathbb{R}_{x}\times\mathbb{R}_{z}$ (see Table 6
and 7). We choose the $SL(2,\mathbb{Z})$ frame of Type IIB such that
$\mathbf{(1,0)}=\text{NS5}$ and $\mathbf{(0,1)}=\text{D5}$. When the
fivebranes share the same
$\mathbb{R}^{2}_{{\varepsilon}_{c+1}}\times\mathbb{R}^{2}_{{\varepsilon}_{c-1}}$
in their worldvolume, they can make a non-transverse trivalent intersection if
the $\mathbf{(p,q)}$-charge is conserved. These non-transverse intersections
can be consecutively joined to form the $\mathbf{(p,q)}$-web of fivebranes.
The M2-brane supported on
$\mathbb{R}^{2}_{{\varepsilon}_{c}}\times\mathbb{R}_{t}$ is mapped to a
D3-brane supported on
$\mathbb{R}^{2}_{{\varepsilon}_{c}}\times\mathbb{R}_{t}\times S^{1}$.

In Awata:2011ce , it was observed that the web of $q$-boson representations of
the quantum toroidal algebra of $\mathfrak{gl}(1)$, depicted based on the
$\mathbf{(p,q)}$-web of fivebranes, gives a reconstruction of the partition
function of the topological vertices and the 5d $\mathcal{N}=1$ effective
gauge theory realized on the fivebrane web. The contents in section 2 provides
an explanation of why such an algebraic reconstruction works within the
framework of the 5d $\mathfrak{gl}(1)$ Chern-Simons theory on
$\mathbb{R}_{t}\times\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$.

The M-brane configuration of the Miura transformation, namely, a single
M2-brane passing through multiple M5-branes, is translated in Type IIB to a
single D3-brane passing through multiple NS5-branes with different
orientations. This is precisely the IIB brane engineering of a 3-dimensional
supersymmetric abelian quiver gauge theory Hanany:1996ie . The amount of
supersymmetry and the matter content depends on the orientation of the various
NS5-branes relative to each other as the D3-brane goes through them. when the
D3-brane and an NS5-brane are supported on a common
$\mathbb{R}^{2}_{\epsilon_{c}}$ plane, there is 3d $\mathcal{N}=4$
bifundamental hypermultiplet matter from quantizing open strings at the
intersection. When the D3-brane support is totally transverse to that of an
NS5-brane, the matter content is that of a 1d $\mathcal{N}=4$ hypermultiplet
multiplet instead.

In this Type IIB frame, we will show that the various components of the Miura
transformation are equal to a certain _half-index_ of the 3d theory on the D3
brane.999It was first discovered in Aganagic:2013tta ; Aganagic:2014oia  that
the half-index of 3d supersymmetric gauge theories is sometimes equal to a
deformed conformal blocks of a $q\mathcal{W}$-algebra, a statement closely
related to the Alday-Gaiotto-Tachikawa-Wyllard correspondence Alday:2009aq ;
Wyllard:2009hg . The result was later reinterpreted via the representation
theory of the quantum toroidal algebra of $\mathfrak{gl}(1)$ Mironov:2016yue ;
Zenkevich:2018fzl ; Zenkevich:2020ufs . Our present paper directly falls under
the scope of this research program. For us, this half-index is a count of BPS
states defined in the UV, for any Lagrangian 3d ${\mathcal{N}}=2$ theory
Dimofte:2017tpi ; Yoshida:2014ssa ; Gadde:2013wq ; Gadde:2013sca : one
replaces $\mathbb{R}^{2}_{{\varepsilon}_{c}}$ with a finite disk/hemisphere
$D^{2}$, and imposes a specific set of 1/2-BPS $\mathcal{N}=(0,2)$ boundary
conditions at finite distance on $S^{1}(R)\times S^{1}_{D^{2}}=T^{2}$, which
will flow to a superconformal point in the IR.101010The index is sometimes
called “A-twisted”, but we do not perform the topological twist of any
supercharge here. In geometry, the half-index we compute is a generating
function of quasimaps $\mathbb{CP}^{1}/\\{0\\}\rightarrow X$ of all degrees in
equivariant quantum K-theory, where $X$ is the Higgs branch of the 3d theory
2009arXiv0908.4446C ; 2010arXiv1005.4125K ; 2011arXiv1106.3724C ;
Okounkov:2015spn ; by excising the origin of $\mathbb{CP}^{1}$, we are
allowing for line operator insertions supported on $\\{0\\}\times S^{1}$ to
contribute to the quasimap count (see for instance Aganagic:2017gsx ;
Aganagic:2017smx ); these are the 1-dimensional degrees of freedom supported
at a totally transverse NS5-D3 brane intersection. In section 4, we find:

Each component of the Miura transformation is the half-index of a 3d
supersymmetric abelian (quiver) gauge theory on $D^{2}\times S^{1}$, possibly
coupled to 1d line defects.

The Miura transformation admits an integral representation, whose components
are recovered from the index by imposing Neumann boundary conditions for the
vector multiplets and a fixed polarization for the hypermultiplets (a specific
holomorphic Lagrangian splitting of the $\mathcal{N}=4$ hypermultiplets). The
residue sum of this Miura transformation corresponds to giving Dirichlet
boundary conditions to the vector multiplets and hypermultiplets.

The fact that we are able to recover the components of the Miura
transformation straight from a Type IIB computation reinforces the validity of
the duality we propose here.

IIB branes | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
---|---|---|---|---|---|---|---|---|---|---
$\text{D3}_{1,0,0}$ | x | x |  |  |  |  | x |  |  | x
$\text{D3}_{0,1,0}$ |  |  | x | x |  |  | x |  |  | x
$\text{D3}_{0,0,1}$ |  |  |  |  | x | x | x |  |  | x
$\text{NS5}_{1,0,0}$ |  |  | x | x | x | x |  | x |  | x
$\text{NS5}_{0,1,0}$ | x | x |  |  | x | x |  | x |  | x
$\text{NS5}_{0,0,1}$ | x | x | x | x |  |  |  | x |  | x
$\text{D5}_{0,0,1}$ | x | x | x | x |  |  |  |  | x | x
$\mathbf{(p,q)}_{0,0,1}$ | x | x | x | x |  |  |  | $l^{\mathbf{(p,q)}}$ | x
Table 6: IIB-brane configuration for the $\mathbf{(p,q)}$-web. Here, $\mathbf{(p,q)}$ indicates the $\mathbf{(p,q)}$-fivebrane, supported on the line $l^{\mathbf{(p,q)}}=\\{\mathbf{q}x-\mathbf{p}z=\text{const}\\}\subset\mathbb{R}_{x}\times\mathbb{R}_{z}$.  $\mathbb{R}^{2}_{{\varepsilon}_{1}}$ | $\mathbb{R}^{2}_{{\varepsilon}_{2}}$ | $\mathbb{R}^{2}_{{\varepsilon}_{3}}$ | $\mathbb{R}_{t}$ | $\mathbb{R}_{x}$ | $\mathbb{R}_{z}$ | $S^{1}$
---|---|---|---|---|---|---
$x^{0},x^{1}$ | $x^{2},x^{3}$ | $x^{4},x^{5}$ | $x^{6}$ | $x^{7}$ | $x^{8}$ | $x^{9}$
Table 7: Spacetime of the IIB theory for the $\mathbf{(p,q)}$-web. The
$\mathbf{(p,q)}$-fivebrane web is depicted on the
$\mathbb{R}_{x}\times\mathbb{R}_{z}$ plane.

##### Gauge origami and $qq$-characters

It was suggested in Jeong:2023qdr  that the IIB theory for the gauge origami
Nikita:I ; Nikita:III  is also connected to the twisted M-theory by a string
duality, up to certain unrefinement of the background. The gauge origami is
the configuration of intersecting D3-branes in the IIB theory defined on
$X\times\mathbb{C}^{\times}_{X}$. The $\Omega$-background can be implemented
with respect to the $U(1)^{3}\subset SU(4)$ isometry of the Calabi-Yau 4-fold
$X$, parametrized by the four $\Omega$-background parameters
$({\varepsilon}_{1},{\varepsilon}_{2},{\varepsilon}_{3},{\varepsilon}_{4})$
constrained by
${\varepsilon}_{1}+{\varepsilon}_{2}+{\varepsilon}_{3}+{\varepsilon}_{4}=0$.
The D3-branes occupy real four-cycles in $X$ which are invariant under the
isometry, located at certain points on the transverse holomorphic plane
$\mathbb{C}^{\times}_{X}$.

We set
$X=\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}^{2}_{{\varepsilon}_{2}}\times(\mathbb{R}^{2}_{{\varepsilon}_{3}}\times\mathbb{R}^{2}_{{\varepsilon}_{4}})/\mathbb{Z}_{l}$
and deform the $A_{l-1}$-singularity into the $l$-centered Taub-NUT space (see
Table 8 and 9). The subscript of each $\mathbb{R}^{2}$-plane denotes the
$\Omega$-background parameter associated to its rotation. By unrefining the
$\Omega$-background, the T-duality along the Taub-NUT circle can be performed
in the simplest manner, yielding the IIA theory defined on
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}^{2}_{{\varepsilon}_{3}}\times\mathbb{R}_{t}\times\mathbb{C}^{\times}_{X}\times
S^{1}$ (${\varepsilon}_{1}+{\varepsilon}_{2}+{\varepsilon}_{3}=0$) with $l$
NS5-branes supported on
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{C}^{\times}_{X}$.
We may further decompactify the circle $S^{1}$ to a line $\mathbb{R}_{z}$,
making one NS5-brane disappear to infinity in $\mathbb{R}_{z}$, and uplift to
M-theory with the M-theory circle $S^{1}_{M}$, which combines into another
holomorphic plane $\mathbb{C}^{\times}_{Z}$ Witten1997 . In this way, we
arrive at the twisted M-theory on
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}^{2}_{{\varepsilon}_{3}}\times\mathbb{R}_{t}\times\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$.

Let us examine how two different types of D3-branes in the IIB theory of the
gauge origami are transformed under the duality. The first type is the ones
supported on
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}^{2}_{{\varepsilon}_{2}}$,
each of which is assigned a $\mathbb{Z}_{l}$-charge. After the T-duality, they
are mapped to the D4-branes stretched between the newly created NS5-branes in
the $\mathbb{R}_{z}$-direction. The number of D4-branes between each
consecutive pair of NS5-branes is determined by their $\mathbb{Z}_{l}$-charge
assignment Douglas:1996sw . This is precisely the IIA engineering of the
5-dimensional $\mathcal{N}=1$ quiver gauge theory compactified on a circle, as
the effective field theory obtained from quantizing the D4-D4 open strings.
Both the D4-branes and the NS5-branes become M5-branes in the twisted
M-theory, sharing
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}^{2}_{{\varepsilon}_{2}}$
in their worldvolume but wrapping different holomorphic plane,
$\mathbb{C}^{\times}_{Z}$ and $\mathbb{C}^{\times}_{X}$, respectively. By
their non-transverse intersections on
$\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$, they form a web of
M5-branes.

The second type is supported on
$(\mathbb{R}^{2}_{{\varepsilon}_{3}}\times\mathbb{R}^{2}_{{\varepsilon}_{4}})/\mathbb{Z}_{l}$,
for which we also assign a $\mathbb{Z}_{l}$-charge. Equivalently, the D3-brane
can wrap different compact $\mathbb{P}^{1}$’s after resolving the $A_{l-1}$
singularity into the $l$-centered Taub-NUT space, where the choice is encoded
in the $\mathbb{Z}_{l}$-charge $c\in\\{0,1,\cdots,l-1\\}$. Performing the
T-duality, this D3-brane becomes a D2-brane supported on
$\mathbb{R}^{2}_{{\varepsilon}_{3}}\times\mathbb{R}_{t}$, sharing the
$\mathbb{R}_{z}$-coordinate with one of the NS5-branes ($l$ NS5s before the
decompactification or $l-1$ NS5s after the decompactification), determined by
the $\mathbb{Z}_{l}$-charge. At last, under the uplift to the twisted
M-theory, the D2-brane becomes an M2-brane supported on
$\mathbb{R}^{2}_{{\varepsilon}_{3}}\times\mathbb{R}_{t}$. Note that the
resulting M2-brane is totally transverse to the M5-branes that we obtained
above in the 11-dimensional worldvolume.

Recall that the crossed D3-brane configuration in the gauge origami involves
both types of D3-branes present simultaneously. In particular, due to their
transverse intersection, the second type of D3-brane provides a codimension-4
defect lying along the circle in the 5-dimensional $\mathcal{N}=1$ effective
gauge theory from the first type, referred to as the $qq$-characters Nikita:I
.111111In more detail, by $qq$-character, we mean the gauge and string
theoretical realization of the generating currents of the
$q\mathcal{W}$-algebra (and by extension those of the $qY$-algebra as well),
originally constructed algebraically by Frenkel and Reshetikhin Frenkel:1998
(see also Shiraishi:1995rp ; Feigin:1995sf ; Awata:1995zk ; Bouwknegt:1998da
), and revisited recently in a wide variety of examples Kimura:2015rgi ;
Kimura:2016dys ; Kimura:2017hez ; Kojima2019QuadraticRO ;
Kojima2021QuadraticRO2 ; 2023arXiv231216856K . The relevance of
$qq$-characters to the representation theory of quantum toroidal algebras was
studied in 2015arXiv151208779B ; Bourgine:2016vsq ; 2020arXiv200304234F ;
Harada:2021xnm . In the dual IIB theory for the $\mathbf{(p,q)}$-fivebrane web
on the $\mathbb{R}_{x}\times\mathbb{R}_{z}$ plane (see table 6), the crossed
D3-brane is mapped to a D3-brane at a point on that plane.121212It has been
suggested that $qq$-characters could be realized via _different_ brane defects
in a $\mathbf{(p,q)}$-fivebrane web Kimura:2017auj ; Zenkevich:2023cza .
Unfortunately, none of these proposals coincide with the original stringy
crossed instanton definition in Nikita:I , nor are they obviously dual to it.
In our work, the D3-brane defect construction will be directly T-dual to it.
Physically, the $qq$-character is the BPS count of this brane system;
equivalently, it is the Witten index of the supersymmetric quantum mechanics
on instantonic D1-branes wrapping the circle, in the presence of the heavier
branes NaveenNikita ; Kim:2016qqs ; Chang:2016iji ; Assel:2018rcw ;
Agarwal:2018tso ; Haouzi:2019jzk ; Haouzi:2020bso . Mathematically, this index
is defined as the (properly regularized) integral over the crossed instanton
moduli space of the generalized gauge theory, evaluated in equivariant
K-theory.

We just analyzed that, in the twisted M-theory, the corresponding M-brane
configuration is a single M2-brane placed upon a transverse web of M5-branes,
enumerated by $c=1,2,\cdots,l-1$, which determines the M5-brane that it shares
the $\mathbb{C}^{\times}_{Z}$-coordinate with. As we discussed earlier, the
M2-M5 intersection assigns an R-matrix of the quantum toroidal algebra of
$\mathfrak{gl}(1)$ (i.e., the Miura operator by our result in section 3). In
section 5, we show that:

The insertion of the R-matrix associated with the M2-M5 intersection on the
M5-brane web of $q$-bosons exactly reproduces the $qq$-characters.

This explicitly confirms the duality we suggest here.

Already back in the 90’s Frenkel:1998 , it was appreciated that
$qq$-characters are traces of transfer matrices for certain TQ-integrable
systems. Here, we are showing that the generating function of all the
$qq$-characters is a product of R-matrices of the quantum toroidal algebra,
making the integrability even more manifest. See Kimura:2015rgi ;
Kimura:2017hez ; Bourgine:2015szm ; Jeong:2017pai ; Jeong:2018qpc ;
Lee:2020hfu ; Jeong:2021bbh ; Jeong:2023qdr ; Grekov:2023fek ; Jeong:2024hwf ;
Jeong:2024mxr  for studies of the quantum integrability using the
$qq$-characters as gauge theory observables. See also Mironov:2016yue ;
Kim:2016qqs ; Elliott:2018yqm ; Frenkel:2020iqq ; Jeong:2017mfh ;
Jeong:2019fgx ; Jeong:2020uxz ; Assel:2018rcw ; Agarwal:2018tso ;
Haouzi:2019jzk ; Haouzi:2020bso ; Haouzi:2020yxy ; Haouzi:2020zls ;
2022arXiv220307072L  for related studies on the $qq$-characters.

IIB branes | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
---|---|---|---|---|---|---|---|---|---|---
D3 | x | x | x | x |  |  |  |  |  |
D3 |  |  |  |  | x | x | x | x |  |
KK5 | x | x | x | x |  |  |  |  | x | x
Table 8: IIB configuration for the gauge origami. The first row is the D3-branes of first type, while the second row is the D3-branes of second type. The Kaluza-Klein 5-monopoles simply indicate the spacetime is modified to the $l$-centered Taub-NUT space. $\mathbb{R}^{2}_{{\varepsilon}_{1}}$ | $\mathbb{R}^{2}_{{\varepsilon}_{2}}$ | $\mathbb{R}^{2}_{{\varepsilon}_{3}}$ | $\mathbb{R}^{2}_{{\varepsilon}_{4}}$ | $\mathbb{C}^{\times}_{X}$
---|---|---|---|---
$x^{0},x^{1}$ | $x^{2},x^{3}$ | $x^{4},x^{5}$ | $x^{6},x^{7}$ | $x^{8},x^{9}$
Table 9: Spacetime of the IIB theory for the gauge origami

###  1.4 Outline

In section 2, we identify the algebra of local operators on the line defects
and the surface defects of 5d Chern-Simons theory from the M2-branes and the
M5-branes, respectively, and realize them as representations of the quantum
toroidal algebra of $\mathfrak{gl}(1)$. In section 3, we solve the R-matrix
between the representations assigned to a single M2-brane and a single
M5-brane. We construct the Miura transformation for the $q$-deformed
$Y$-algebra using the R-matrix properties of the multiple M2-M5 intersections.
We also obtain the screening charges from M2-branes placed between each
consecutive M5-branes, and verify that the $q$-deformed $Y$-algebra is exactly
characterized as the commutants of these screening charges. In section 4, we
show that the half-index of the 3d $\mathcal{N}=2$ quiver $U(1)$ gauge theory
with 1d defect, engineered as the effective field theory on the D3-brane
passing through multiple NS5-branes, exactly matches with the components of
the Miura transformations. In particular, the relevant boundary conditions are
exactly identified. In section 5, we present different incarnations of the
$q$-boson representations of the quantum toroidal algebra associated to the
M5-branes supported on general holomorphic curves. It is shown that their non-
transverse intersections lead to the intertwiners between those $q$-boson
representations. We consider the configuration where a single M2-brane passes
through a web of M5-branes, and verify that the insertion of the R-matrix
associated to the M2-M5 intersection precisely yields the $qq$-characters of
the 5d $\mathcal{N}=1$ effective gauge theory realized on the web of
M5-branes. We conclude with discussions in section 6. The appendices contain
generalities about the quantum toroidal algebra of $\mathfrak{gl}(1)$ and the
spherical double affine Hecke algebra; computational details for solving the
R-matrices and the screening charges; and details of the $q$-boson
intertwiners of the quantum toroidal algebra of $\mathfrak{gl}(1)$.

##### Acknowledgement

The authors thank Mina Aganagic, Sibasish Banerjee, Davide Gaiotto, Alba
Grassi, Ahsan Khan, Nafiz Ishtiaque, Shota Komatsu, Norton Lee, Evgeny Mukhin,
Nikita Nekrasov, Jihwan Oh, Miroslav Rapčák, Edward Witten, Yegor Zenkevich,
and Yehao Zhou for discussions and collaboration on related subjects. The
authors are grateful to the organizers of Simons Physics Summer Workshop 2023,
where the collaboration was initiated. The work of NH is supported by NSF
Grant PHY-2207584 and the Sivian Fund at the Institute for Advanced Studies.
The work of SJ is supported by CERN and CKC fellowship.

##  2 M-branes and representations of quantum toroidal algebra of
$\mathfrak{gl}(1)$

The twisted M-theory subject to the $\Omega$-background reduces to the
5-dimensional non-commutative Chern-Simons theory on
$\mathbb{R}_{t}\times\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$
Costello:2016nkh . The M2-branes and the M5-branes descend to topological line
defects and holomorphic surface defects, respectively. The BRST-invariance of
the coupling of these defects require the algebra of local operators on the
defects to be representations of certain universal associative algebra. In
this section, we suggest this universal associative algebra is the quantum
toroidal algebra of $\mathfrak{gl}(1)$,
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$, for our case.
We also specify the representations of
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$ associated to
the M2-branes and the M5-branes.

###  2.1 M2-branes and topological line defects

The M2-branes in the twisted M-theory can be supported on
$\mathbb{R}^{2}_{{\varepsilon}_{c}}\times\mathbb{R}_{t}$, where
$c\in\\{1,2,3\\}$. Once reduced to the 5d $\mathfrak{gl}(1)$ Chern-Simons
theory, they give rise to line defects lying along $\mathbb{R}_{t}$, located
at points on $\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$. Let us
focus on the case where all the M2-branes are located at a single point on
$\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$, giving a single line
defect. In the most general case, there are $l$, $m$, and $n$ M2-branes
wrapping $\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}_{t}$,
$\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}_{t}$, and
$\mathbb{R}^{2}_{{\varepsilon}_{3}}\times\mathbb{R}_{t}$, respectively. We
call the resulting line defect $\mathcal{L}_{l,m,n}$. Let us define
$q_{c}=e^{{\varepsilon}_{c}}\in\mathbb{C}^{\times}$, $c\in\\{1,2,3\\}$.

The local observables of the 5d $\mathfrak{gl}(1)$ Chern-Simons theory
localized on the line $\mathbb{R}_{t}\times\\{\text{point}\\}$ form an algebra
$\text{Obs}_{q_{1},q_{2}}^{\mathbb{R}_{t}}$, with the product defined by their
OPEs. At the classical level ($q_{1}=1$), the algebra
$\text{Obs}_{q_{1}=1,q_{2}}^{\mathbb{R}_{t}}$ has a simple description; the
solutions to the equations of motion is trivial, and the only local
observables are the functions of the modes of the ghost under the Laurent
expansion around the locus of the line in
$\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$. The coordinates $X$
and $Z$ are subject to the non-commutativity relation

$\displaystyle XZ=q_{2}ZX,$ (17)

due to the B-field
$B={\varepsilon}_{2}\frac{d\bar{X}}{\bar{X}}\wedge\frac{d\bar{Z}}{\bar{Z}}$.
The algebra
$\mathscr{O}_{q_{2}}(\mathbb{C}^{\times}\times\mathbb{C}^{\times})$ of Laurent
polynomial functions on $\mathbb{C}^{\times}\times\mathbb{C}^{\times}$ subject
to this relation is called the quantum torus algebra. Equipped with the BRST
differential, the functions of the ghost thus form the Chevalley-Eilenberg
cochain complex,

$\displaystyle\text{Obs}_{q_{1}=1,q_{2}}^{\mathbb{R}_{t}}=C^{\bullet}(\mathscr{O}_{q_{2}}(\mathbb{C}^{\times}\times\mathbb{C}^{\times})).$
(18)

A line defect of the 5d $\mathfrak{gl}(1)$ Chern-Simons theory can be defined
by coupling a topological quantum mechanics, whose algebra of local
observables is, say, $\mathcal{B}$. Explicitly, the line defect coupling is
implemented by inserting the generalized Wilson line,

$\displaystyle\text{Pexp}\sum_{m,n\in\mathbb{Z}}\int_{\mathbb{R}_{t}}dt\,P_{m,n}(A_{t})_{m,n},$
(19)

into the path integral, where $(A_{t})_{m,n}$ are the modes of the gauge field
under the Laurent expansion in $Z$ and $X$, and $P_{m,n}\in\mathcal{B}$ are
the local operators on the line defect. The BRST-invariance of the coupling is
guaranteed if the coupling is the topological descent of a Maurer-Cartan
element in $\text{Obs}_{q_{1},q_{2}}^{\mathbb{R}_{t}}\otimes\mathcal{B}$. This
is, by the Koszul duality, equivalent to requiring that there is a surjective
algebra homomorphism

$\displaystyle\mathcal{A}_{q_{1},q_{2}}:=\left(\text{Obs}_{q_{1},q_{2}}^{\mathbb{R}_{t}}\right)^{!}\twoheadrightarrow\mathcal{B},$
(20)

where
$\mathcal{A}_{q_{1},q_{2}}=\left(\text{Obs}_{q_{1},q_{2}}^{\mathbb{R}_{t}}\right)^{!}$
is the Koszul dual of $\text{Obs}_{q_{1},q_{2}}^{\mathbb{R}_{t}}$. In this
sense, the Koszul dual algebra $\mathcal{A}_{q_{1},q_{2}}$ can be said to be
the algebra of local observables on the universal line defect. It is called
the universal associative algebra in short.

In the classical limit $q_{1}=1$, the standard result tells that the Koszul
dual of the operator algebra (18) is (see Paquette:2021cij  for a review)

$\displaystyle\mathcal{A}_{q_{1}=1,q_{2}}=\left(\text{Obs}_{q_{1}=1,q_{2}}^{\mathbb{R}_{t}}\right)^{!}=\mathscr{O}_{q_{2}}(\mathbb{C}^{\times}\times\mathbb{C}^{\times}).$
(21)

Turning on $q_{1}\neq 1$, the quantum corrections deform the operator algebra
$\text{Obs}_{q_{1},q_{2}}^{\mathbb{R}_{t}}$, and therefore its Koszul dual
$\mathcal{A}_{q_{1},q_{2}}=(\text{Obs}_{q_{1},q_{2}}^{\mathbb{R}_{t}})^{!}$,
in a non-trivial way. In particular, the Koszul dual would be a deformation of
the quantum torus algebra (21), which we can formally denote by

$\displaystyle\mathcal{A}_{q_{1},q_{2}}=U_{q_{1}}(\mathscr{O}_{q_{2}}(\mathbb{C}^{\times}\times\mathbb{C}^{\times})).$
(22)

The quantum corrections for $\text{Obs}_{q_{1},q_{2}}^{\mathbb{R}_{t}}$ can in
principle be determined by computing the Feynman diagrams of the 5d
$\mathfrak{gl}(1)$ Chern-Simons theory on
$\mathbb{R}_{t}\times\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$
relevant to the operator products of the mode (Laurent) expansions of the
ghost fields.131313See Costello:2017fbo  for the explicit evaluation of the
Feynman diagrams relevant to the quantum corrections to the operator algebra
in the case of the 5d $\mathfrak{gl}(k)$ Chern-Simons theory on
$\mathbb{R}\times\mathbb{C}^{2}$. In this case, the operator algebra is an
${\varepsilon}_{1}$-deformation of
$C^{\bullet}(\mathfrak{gl}(k)\otimes\mathscr{O}_{{\varepsilon}_{2}}(\mathbb{C}^{2}))$,
and its Koszul dual, i.e., the universal associative algebra, is proven to be
the 1-shifted affine Yangian of $\mathfrak{gl}(k)$,
$U_{{\varepsilon}_{1}}(\mathfrak{gl}(k)\otimes\mathscr{O}_{{\varepsilon}_{2}}(\mathbb{C}^{2}))=Y_{1}(\widehat{\mathfrak{gl}}(k))$.
Also, the defining relations of the universal associative algebra can in
principle be achieved as conditions for the anomalous Feynman diagrams to
cancel each other (see Costello:2017dso ; Costello:2017fbo ; Oh:2020hph ). We
do not take such a direct approach of the perturbation theory in the present
work.

Rather, we will make a natural suggestion for the universal associative
algebra $\mathcal{A}_{q_{1},q_{2}}$, and assign appropriate representations to
the topological line defects $\mathcal{L}_{l,m,n}$ created by the M2-branes.
Our proposal is that the universal associative algebra
$\mathcal{A}_{q_{1},q_{2}}$ is precisely the quantum toroidal algebra of
$\mathfrak{gl}(1)$, namely,
$\mathcal{A}_{q_{1},q_{2}}=U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$.141414Recall
that $q_{3}=q_{1}^{-1}q_{2}^{-1}$ so that we could have omitted the dependence
on $q_{3}$ and write $U_{q_{1},q_{2}}(\widehat{\widehat{\mathfrak{gl}}}(1))$.
We write $q_{3}$ explicitly only to emphasize the triality of exchanging
$q_{1}$, $q_{2}$, and $q_{3}$ with each other. We review the definition and
the relevant properties of the quantum toroidal algebra
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$ in appendix A.
In the present work, we will provide evidences for our claim, leaving the
exact proof of the statement through perturbative study of 5d non-commutative
$\mathfrak{gl}(1)$ Chern-Simons theory on
$\mathbb{R}_{t}\times\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$ to
future work.

####  2.1.1 Single M2-brane and quantum torus algebra

Let us begin with a single M2-brane supported on
$\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}_{t}$. In the IIB dual
frame, the M2-brane passes to a fundamental string attached to the NS5-brane,
ending at the junction of the fivebranes. In the worldvolume theory of the
D5-brane, the fundamental string provides a boundary Wilson line of charge 1
Gaiotto:2019wcc . The position of the Wilson line along the junction of the
fivebranes, which we denote by
$(X_{1},Z_{1})\in\mathbb{C}_{X}^{\times}\times\mathbb{C}_{Z}^{\times}$, are
subject to the relation

$\displaystyle X_{1}Z_{1}=q_{2}Z_{1}X_{1}$ (23)

by the non-commutativity (17). They can be thought of as the generators of the
algebra $\text{M2}_{0,1,0}$ of local operators on this Wilson line, which is
nothing but the quantum torus algebra. Namely, we have
$\text{M2}_{0,1,0}=\mathscr{O}_{q_{2}}(\mathbb{C}^{\times}\times\mathbb{C}^{\times})$.

The quantum torus algebra is a representation of the quantum toroidal algebra
of $\mathfrak{gl}(1)$. Indeed, there is a surjective algebra homomorphism

$\displaystyle\rho_{\text{M2}_{0,1,0}}:U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\twoheadrightarrow\text{M2}_{0,1,0}$
(24)

given by

$\displaystyle\begin{split}&E(X)\mapsto\frac{1}{1-q_{2}}\delta\left(\frac{X_{1}}{X}\right)Z_{1},\quad
F(X)\mapsto\frac{1}{1-q_{2}^{-1}}Z_{1}^{-1}\delta\left(\frac{X_{1}}{X}\right)\\\
&K^{\pm}(X)\mapsto\frac{(1-(q_{1}^{-1}X_{1}/X)^{\pm
1})(1-(q_{3}^{-1}X_{1}/X)^{\pm 1})}{(1-(X_{1}/X)^{\pm
1})(1-(q_{2}X_{1}/X)^{\pm 1})}\\\ &C\mapsto 1.\end{split}$ (25)

Note $(C,C^{\perp})\mapsto(1,1)$ in particular. The map
$\rho_{\text{M2}_{0,1,0}}$ can be written equivalently in terms of the modes
as

$\displaystyle\begin{split}&E_{k}\mapsto\frac{1}{1-q_{2}}X_{1}^{k}Z_{1},\quad
F_{k}\mapsto\frac{1}{1-q_{2}^{-1}}Z_{1}^{-1}X_{1}^{k},\qquad
k\in\mathbb{Z},\\\ &H_{\pm r}\mapsto\frac{1}{1-q_{2}^{\mp r}}X_{1}^{\pm
r},\qquad r\in\mathbb{Z}_{>0},\\\ &C\mapsto 1,\quad C^{\perp}\mapsto
1.\end{split}$ (26)

In particular, the homomorphism in fact does not depend on $q_{1}$.

For any given weight parameter $u\in\mathbb{C}^{\times}$, there is a faithful
representation of the quantum torus algebra $\text{M2}_{0,1,0}$ on the vector
space $\mathcal{V}_{2}(u)=\bigoplus_{i\in\mathbb{Z}}\mathbb{C}[u]_{i}^{(2)}$,
defined by

$\displaystyle\begin{split}Z_{1}[u]_{i}^{(2)}=[u]_{i+1}^{(2)},\quad\quad
X_{1}[u]_{i}^{(2)}=uq_{2}^{i}[u]_{i}^{(2)}.\end{split}$ (27)

More concretely, the representation can be thought of as the
$q_{2}$-difference operators

$\displaystyle Z_{1}=q_{2}^{-D_{X_{1}}},\qquad X_{1}=X_{1}$ (28)

acting on the space of Laurent polynomials

$\displaystyle\mathcal{V}_{2}(u)=u^{-D_{X_{1}}}\mathbb{C}\left(\\!\left(q_{2}^{-D_{X_{1}}}\right)\\!\right)=\bigoplus_{i\in\mathbb{Z}}\mathbb{C}q_{2}^{-\left(\frac{\log
u}{\log q_{2}}+i\right)D_{X_{1}}}.$ (29)

Here, we used the notation $D_{X_{1}}=X_{1}\partial_{X_{1}}$ so that
$q_{2}^{-D_{X_{1}}}=q_{2}^{-X_{1}\partial_{X_{1}}}$ is the $q_{2}^{-1}$-shift
operator for $X_{1}$ (i.e.,
$q_{2}^{-D_{X_{1}}}f(X_{1})=f(q_{2}^{-1}X_{1})q_{2}^{-D_{X_{1}}}$ for any
function $f(X_{1})$). This is precisely what is called the vector
representation of the quantum toroidal algebra of $\mathfrak{gl}(1)$ (cf.
Tsymbaliuk:2014fvq ), factored through the quantum torus algebra. Namely, what
we obtained can be summarized as
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\twoheadrightarrow\text{M2}_{0,1,0}\stackrel{{\scriptstyle\sim}}{{\smash{\longrightarrow}\rule{0.0pt}{1.72218pt}}}\text{End}(\mathcal{V}_{2}(u))$.

For later use, note that the grading operator $d$ (see section A.3) acts on
the generators as

$\displaystyle q^{d}X_{1}=qX_{1}q^{d}\qquad q^{d}Z_{1}=Z_{1}q^{d},$ (30)

for any $q\in\mathbb{C}^{\times}$. It follows that the representation on
$\mathcal{V}_{2}$ can be extended to incorporate the grading operator by
regarding it as the shift operator for the module parameter:
$q^{d}[u]^{(2)}_{i}=[uq^{-1}]_{i}^{(2)}$.

As stressed above, the homomorphism
$\rho_{\text{M2}_{0,1,0}}:U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\to\text{M2}_{0,1,0}$
is not affected by tuning of the value of $q_{1}$. In this sense, the quantum
torus algebra, as a representation of the quantum toroidal algebra, does not
deform from $q_{1}=1$, and it is expected to still provide the M2-brane
algebra with all the quantum corrections taken into account. Such a rigidity
is indeed anticipated from the boundary Wilson line description for the single
M2-brane.

Now, let us turn to the single M2-brane wrapping
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}_{t}$ (resp.
$\mathbb{R}^{2}_{{\varepsilon}_{3}}\times\mathbb{R}_{t}$). These M2-branes can
simply be obtained by shuffling the $\mathbb{R}^{2}$-planes in
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}^{2}_{{\varepsilon}_{3}}$,
which is equivalent to exchanging the parameters $q_{1}$, $q_{2}$, and $q_{3}$
with each other. Therefore, the algebra of local operators on this M2-brane is
expected to be the quantum torus algebra
$\text{M2}_{1,0,0}=\mathscr{O}_{q_{1}}(\mathbb{C}^{\times}\times\mathbb{C}^{\times})$
(resp.
$\text{M2}_{0,0,1}=\mathscr{O}_{q_{3}}(\mathbb{C}^{\times}\times\mathbb{C}^{\times})$)
with the non-commutativity of the generators controlled by $q_{1}$ (resp.
$q_{3}$) instead of $q_{2}$.

Indeed, as apparent from its defining relations, the quantum toroidal algebra
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$ is invariant
under the triality of permuting $(q_{1},q_{2},q_{3})$. The algebra
homomorphisms to the other two quantum torus algebras thus exist, obtained
simply by replacing $q_{2}$ by $q_{1}$ or $q_{3}$ in the map
$\rho_{\text{M2}_{0,1,0}}$ (26). We denote these algebra maps by
$\rho_{\text{M2}_{1,0,0}}:U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\to\text{M2}_{1,0,0}$
and
$\rho_{\text{M2}_{0,0,1}}:U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\to\text{M2}_{0,0,1}$,
respectively. We can define faithful representations of these quantum torus
algebras on the vector spaces
$\mathcal{V}_{c}(u)=\bigoplus_{i\in\mathbb{Z}}\mathbb{C}[u]_{i}^{(c)}$,
$c\in\\{1,2,3\\}$, simply by permuting $(q_{1},q_{2},q_{3})$ on (27).

####  2.1.2 Parallel M2-branes and spherical DAHA

Next, we examine multiple M2-branes wrapping the same
$\mathbb{R}^{2}_{{\varepsilon}_{c}}\times\mathbb{R}_{t}$, $c\in\\{1,2,3\\}$.
We will explain that the two cases, $c=1$ and $2$, independently lead to the
same conclusion that the algebra of local operators on the parallel M2-branes
is given by the spherical subalgebra of the double affine Hecke algebra
(spherical DAHA). Then, we will show that the fusion of the parallel M2-branes
is compatible with the coproduct of the quantum toroidal algebra.

##### M2-branes on $\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}_{t}$
and boundary Wilson lines

As studied in the previous section, the $m$ M2-branes supported on
$\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}_{t}$ reduce to the $m$
boundary Wilson lines on the worldvolume theory of the D5-brane. We denote the
algebra of local operators on these M2-branes by $\text{M2}_{0,m,0}$.

Let $(X_{i},Z_{i})_{i=1}^{m}$ be the positions of the $m$ Wilson lines on the
holomorphic surfaces $\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$.
Just as the case of single M2-brane, these holomorphic coordinates are the
generators of the algebra of local operators, subject to the non-commutativity
(17). In the classical limit $q_{1}=1$, there is no quantum correction coming
from the interaction between these Wilson lines. The algebra of local
operators is thus simply given by the symmetric product of the $m$ quantum
torus algebras,
$\text{M2}_{0,m,0}^{q_{1}=1}=\mathscr{O}_{q_{2}}\left(\mathbb{C}^{\times}\times\mathbb{C}^{\times})\right)^{\otimes
m}/S_{m}$, each of which is from individual Wilson line. As discussed earlier,
the universal associative algebra at $q_{1}=1$ is the quantum torus algebra
(21), and there is indeed a natural homomorphism

$\displaystyle\mathscr{O}_{q_{2}}(\mathbb{C}^{\times}\times\mathbb{C}^{\times})\twoheadrightarrow\frac{\left(\mathscr{O}_{q_{2}}(\mathbb{C}^{\times}\times\mathbb{C}^{\times})\right)^{\otimes
m}}{S_{m}},$ (31)

which encodes the coupling of the line defect in the classical limit
$q_{1}=1$.

Turning on $q_{1}\neq 1$, we suggested that the universal associative algebra
gets deformed to the quantum toroidal algebra of $\mathfrak{gl}(1)$,
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))=U_{q_{1}}(\mathscr{O}_{q_{2}}(\mathbb{C}^{\times}\times\mathbb{C}^{\times}))$.
The algebra $\text{M2}_{0,m,0}$ of local operators on the line defect should
also be a $q_{1}$-deformation of
$\text{M2}_{0,m,0}^{q_{1}=1}=\left(\mathscr{O}_{q_{2}}(\mathbb{C}^{\times}\times\mathbb{C}^{\times})\right)^{\otimes
m}/S_{m}$, and the coupling of the line defect would be encoded in a
$q_{1}$-deformation of the surjective homomorphism (31), which we may write as

$\displaystyle\rho_{\text{M2}_{0,m,0}}:U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\twoheadrightarrow\text{M2}_{0,m,0}.$
(32)

There is indeed a natural $q_{1}$-deformation of the symmetric product of
quantum torus algebras: the spherical subalgebra of the double affine Hecke
algebra (spherical DAHA) of $GL(m)$, which we denote by
$\text{\bf{S}}\boldsymbol{\ddot{\text{\bf{H}}}}^{(m)}_{q_{2},q_{1}}$. The
spherical DAHA can be presented as the algebra generated by the difference
operators,

$\displaystyle\begin{split}&P_{0,r}^{(m)}=q_{2}^{r}\sum_{i=1}^{m}X_{i}^{r},\qquad
P_{0,-r}^{(m)}=\sum_{i=1}^{m}X_{i}^{-r},\qquad r\in\mathbb{Z}_{>0}\\\
\hfil\displaystyle\begin{split}&P_{1,k}^{(m)}=q_{2}\sum_{i=1}^{m}\left(\prod_{j\neq
i}\frac{q_{1}X_{i}-X_{j}}{X_{i}-X_{j}}\right)X_{i}^{k}q_{2}^{-D_{X_{i}}}\\\
&P_{-1,k}^{(m)}=\sum_{i=1}^{m}\left(\prod_{j\neq
i}\frac{q_{1}^{-1}X_{i}-X_{j}}{X_{i}-X_{j}}\right)q_{2}^{D_{X_{i}}}X_{i}^{k}\end{split},\qquad
k\in\mathbb{Z},\end{split}$ (33)

acting on the space of symmetric polynomials
$\mathbb{C}(q_{1},q_{2})[X_{1}^{\pm 1},\cdots,X_{m}^{\pm 1}]^{S_{m}}$. We
review the definition and the relevant properties of the spherical DAHA of
$GL(m)$ in appendix B.

Correspondingly, the algebra homomorphism (31) admits a $q_{1}$-deformation,

$\displaystyle
U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\twoheadrightarrow\text{\bf{S}}\boldsymbol{\ddot{\text{\bf{H}}}}^{(m)}_{q_{2},q_{1}},$
(34)

given by

$\displaystyle\begin{split}\hfil\displaystyle\begin{split}&E_{k}\mapsto\frac{1}{q_{2}(1-q_{2})}P_{1,k}^{(m)}=\frac{1}{1-q_{2}}\sum_{i=1}^{m}\left(\prod_{j\neq
i}\frac{q_{1}X_{i}-X_{j}}{X_{i}-X_{j}}\right)X_{i}^{k}q_{2}^{-D_{X_{i}}},\\\
&F_{k}\mapsto\frac{1}{1-q_{2}^{-1}}P_{-1,k}^{(m)}=\frac{1}{1-q_{2}^{-1}}\sum_{i=1}^{m}\left(\prod_{j\neq
i}\frac{q_{1}^{-1}X_{i}-X_{j}}{X_{i}-X_{j}}\right)q_{2}^{D_{X_{i}}}X_{i}^{k},\\\
\end{split}\qquad k\in\mathbb{Z},\\\ \hfil\displaystyle\begin{split}&H_{\pm
r}\mapsto\pm\frac{1}{1-q_{2}^{r}}P_{0,\pm r}^{(m)}=\frac{1}{1-q_{2}^{\mp
r}}\sum_{i=1}^{m}X_{i}^{\pm r},\qquad r\in\mathbb{Z}_{>0},\\\ &C\mapsto
1,\quad C^{\perp}\mapsto 1.\end{split}\end{split}$ (35)

This is a surjective algebra homomorphism for each $m\in\mathbb{Z}_{>0}$
Schiffmann_Vasserot_2011 . Note that in the classical limit $q_{1}=1$, the
spherical DAHA simplifies to the symmetric product
$\text{\bf{S}}\boldsymbol{\ddot{\text{\bf{H}}}}^{(m)}_{q_{2},1}=\left(\mathscr{O}_{q_{2}}(\mathbb{C}^{\times}\times\mathbb{C}^{\times})\right)^{\otimes
m}/{S_{m}}$ and the algebra homomorphism reduces to (31), as promised (compare
with (26)). At the same time, for $m=1$ we simply recover the algebra map (26)
for the single M2-brane,
$\rho_{\text{M2}_{0,1,0}}:U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\to\text{M2}_{0,1,0}$,
which is rigid under the $q_{1}$-deformation. Therefore, we suggest that the
algebra of local operators on the parallel $m$ M2-branes wrapping
$\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}_{t}$ is given by the
spherical DAHA,
$\text{M2}_{0,m,0}=\text{\bf{S}}\boldsymbol{\ddot{\text{\bf{H}}}}^{(m)}_{q_{2},q_{1}}$,
and the algebra map (34) is nothing but the representation
$\rho_{\text{M2}_{0,m,0}}:U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\to\text{M2}_{0,m,0}$
that determines the coupling of the associated line defect
$\mathcal{L}_{0,m,0}$.

Recall that the quantum toroidal algebra
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$ is invariant
under the triality of permuting $(q_{1},q_{2},q_{3})$. As explicit from (35),
the spherical DAHA
$\text{\bf{S}}\boldsymbol{\ddot{\text{\bf{H}}}}^{(m)}_{q_{2},q_{1}}$ as a
representation of
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$ partially breaks
this triality. However, the symmetry of exchanging $q_{1}$ and
$q_{3}(=q_{1}^{-1}q_{2}^{-1})$ remains to be unbroken. Indeed, the relations
imposed on the generators of the spherical DAHA
$\text{\bf{S}}\boldsymbol{\ddot{\text{\bf{H}}}}^{(m)}_{q_{2},q_{1}}$ are
explicitly symmetric under the exchange of $q_{1}$ and $q_{3}$, while $q_{2}$
plays a distinct role. Moreover, at the level of its representation in
$q_{2}$-difference operators (35), it is straightforward to show that the
conjugation by

$\displaystyle
g(X_{1},X_{2}\cdots,X_{m})=\prod_{i=1}^{m}X_{i}^{\frac{(m-i)\log\frac{q_{3}}{q_{1}}}{\log
q_{2}}}\prod_{1\leq i<j\leq
m}\frac{\left(q_{3}^{-1}\frac{X_{j}}{X_{i}};q_{2}\right)_{\infty}}{\left(q_{1}^{-1}\frac{X_{j}}{X_{i}};q_{2}\right)_{\infty}}$
(36)

yields exactly the same $q_{2}$-difference operators except that $q_{1}$ is
replaced by $q_{3}$, due to the relation $q_{1}q_{2}q_{3}=1$. Thus, the
conjugation establishes the isomorphism
$\text{M2}_{0,m,0}=\text{\bf{S}}\boldsymbol{\ddot{\text{\bf{H}}}}^{(m)}_{q_{2},q_{1}}\simeq\text{\bf{S}}\boldsymbol{\ddot{\text{\bf{H}}}}^{(m)}_{q_{2},q_{3}}$.
Note that such a breaking pattern of the triality is precisely what we would
expect from the associated brane configuration in the twisted M-theory, since
the support of the M2-branes distinguishes $q_{2}$ out of the triple
$(q_{1},q_{2},q_{3})$ but still there is a remaining symmetry of exchanging
the two transverse topological planes $\mathbb{R}^{2}_{{\varepsilon}_{1}}$ and
$\mathbb{R}^{2}_{{\varepsilon}_{3}}$.

##### M2-branes on $\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}_{t}$
and Hilbert scheme of points in $\mathbb{C}^{\times}\times\mathbb{C}^{\times}$

So far, we have investigated the algebra of local operators on the parallel
M2-branes supported on
$\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}_{t}$. To study parallel
M2-branes, we can alternatively take a stack of $l$ M2-branes wrapping
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}_{t}$. A crucial point of
this consideration is the following. On one hand, such a change merely amounts
to swapping the topological planes $\mathbb{R}^{2}_{{\varepsilon}_{1}}$ and
$\mathbb{R}^{2}_{{\varepsilon}_{2}}$ in the twisted M-theory setup compared to
the previous case, so that the algebra of local operators should be again
given by the spherical DAHA of $GL(l)$, albeit with the roles of $q_{1}$ and
$q_{2}$ exchanged:
$\text{M2}_{l,0,0}=\text{\bf{S}}\boldsymbol{\ddot{\text{\bf{H}}}}^{(l)}_{q_{1},q_{2}}(\simeq\text{\bf{S}}\boldsymbol{\ddot{\text{\bf{H}}}}^{(l)}_{q_{1},q_{3}})$.
On the other hand, the IIA reduction of the twisted M-theory seemingly breaks
the triality, so that the emergence of the spherical DAHAs, with the $q_{1}$
and $q_{2}$ exchanged, from the two different parallel M2-branes is a non-
trivial consistency check.151515When the holomorphic surfaces are
$\mathbb{C}\times\mathbb{C}$, the appearance of the spherical rational DAHA
$\mathbf{Sr\ddot{H}}$ from two different parallel M2-branes, with the roles of
${\varepsilon}_{1}$ and ${\varepsilon}_{2}$ swapped, was discussed in
Gaiotto:2019wcc . What we discuss here is the multiplicative uplift of this
manifestation of the triality, obtained by replacing the holomorphic surfaces
$\mathbb{C}\times\mathbb{C}$ by
$\mathbb{C}^{\times}\times\mathbb{C}^{\times}$.

Reducing the twisted M-theory along the Taub-NUT circle, we arrive at a
configuration of $l$ D2-brane wrapping
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}_{t}$ lying on top of $1$
D6-brane wrapping
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}_{t}\times\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$.
The effective 3-dimensional $\mathcal{N}=4$ theory is obtained by quantizing
the D2-D2 open string and the D2-D6 open string, where the former contains
infinite numbers of winding modes around the two circles of
$\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$. The Higgs branch of
this 3d $\mathcal{N}=4$ theory is then given by the Hilbert scheme of $l$
points on $\mathbb{C}^{\times}\times\mathbb{C}^{\times}$, which can be
constructed as the multiplicative quiver variety for the Jordan quiver (Figure
4) Gorsky:1999rb ; Kapustin:2000ek ; AOblomkov2003DoubleAH ; JORDAN2014420 .

$1$$l$ Figure 4: Jordan quiver

The multiplicative quiver variety $\mathcal{M}_{q_{2}}^{(l)}$ for the Jordan
quiver is constructed as follows. Let us consider the space of quadruples
$(A,B,I,J)\in\text{End}(\mathbb{C}^{l})\times\text{End}(\mathbb{C}^{l})\times\text{Hom}(\mathbb{C},\mathbb{C}^{l})\times\text{Hom}(\mathbb{C}^{l},\mathbb{C})$
satisfying the relation,

$\displaystyle AB-q_{2}BA+IJ=0\qquad(\text{equivalently,}\;\;\text{rk}(AB-
q_{2}BA)=1).$ (37)

There is a natural $GL(l)$-action on this space given by,

$\displaystyle(A,B,I,J)\mapsto(gAg^{-1},gBg^{-1},gI,Jg^{-1}),\qquad g\in
GL(l).$ (38)

It is known that the action is free for generic $q_{2}\in\mathbb{C}^{\times}$,
so that the quotient is a smooth variety AOblomkov2003DoubleAH . The
multiplicative quiver variety $\mathcal{M}_{q_{2}}^{(l)}$ for the Jordan
quiver is given by this quotient. It is straightforward to see
$\dim_{\mathbb{C}}\mathcal{M}_{q_{2}}^{(l)}=2l$.

The multiplicative quiver variety $\mathcal{M}_{q_{2}}^{(l)}$ is equipped with
a Poisson structure Fock:1992xy ; Chalykh:2017urw , which furnishes a complete
integrable structure associated to the classical trigonometric Ruijsenaars-
Schneider model (the relativistic uplift of the trigonometric Calogero-Moser
model) of $GL(l)$-type. The precise definition of the Poisson structure will
not be very important to us, and we only present it here by giving the
associated Darboux coordinates on an open dense subset
$\left(\mathcal{M}_{q_{2}}^{(l)}\right)^{\circ}=\\{A\text{ is
invertible}\\}\subset\mathcal{M}_{q_{2}}^{(l)}$, following Chalykh:2017urw .
Let us define

$\displaystyle(\mathbb{C}^{\times})^{l}_{\text{reg}}=\\{(x_{1},x_{2},\cdots,x_{l})\in(\mathbb{C}^{\times})^{l}\;|\;x_{i}\neq
x_{j},\;x_{i}\neq q_{2}x_{j}\,\text{ for any }i,j=1,2,\cdots,l\\}.$ (39)

Then we may parameterize $\left(\mathcal{M}_{q_{2}}^{(l)}\right)^{\circ}$ by
the map

$\displaystyle\xi:\left((\mathbb{C}^{\times})^{l}_{\text{reg}}\times(\mathbb{C}^{\times})^{l}\right)/S_{l}\longrightarrow\left(\mathcal{M}_{q_{2}}^{(l)}\right)^{\circ},$
(40)

defined by $\xi\left((x_{i})_{i=1}^{l},(p_{i})_{i=1}^{l}\right)=(A,B)$ with

$\displaystyle\begin{split}&A=\text{diag}(x_{1},x_{2},\cdots,x_{l})\\\
&B=(B_{ij})_{i,j=1}^{l},\quad
B_{ij}=p_{j}^{-1}\frac{x_{j}(1-q_{2})}{x_{i}-q_{2}x_{j}}\prod_{k\neq
j}\frac{q_{2}x_{j}-x_{k}}{x_{j}-x_{k}}.\end{split}$ (41)

Here, $S_{l}$ acts by the simultaneous permutation of $(x_{i})_{i=1}^{l}$ and
$(p_{i})_{i=1}^{l}$. When $\left((x_{i})_{i=1}^{l},(p_{i})_{i=1}^{l}\right)$
gets permuted, the image $(A,B)$ is conjugated by the matrix of that
permutation. Thus, the coordinates are well-defined on
$\left(\mathcal{M}_{q_{2}}^{(l)}\right)^{\circ}$. This is a Darboux coordinate
system on $\left(\mathcal{M}_{q_{2}}^{(l)}\right)^{\circ}$, in the sense that
their Poisson brackets are given by

$\displaystyle\\{x_{i},x_{j}\\}=\\{p_{i},p_{j}\\}=0,\quad\\{p_{i},x_{j}\\}=p_{i}x_{i}\delta_{i,j},\quad
i,j=1,2,\cdots,l.$ (42)

It is straightforward from (41) and (42) that the trace invariants
$\text{Tr}\,A^{k}$, $k\in\mathbb{Z}$, are mutually Poisson-commuting. It can
also be shown that $\text{Tr}\,B^{k}$, $k\in\mathbb{Z}$, are mutually Poisson-
commuting. By taking suitable linear combinations of the products of
$\text{Tr}\,B^{-k}$ with $k=1,2,\cdots,l$, we obtain

$\displaystyle
H_{r}=q_{2}^{-\frac{r(r-1)}{2}}\sum_{\begin{subarray}{c}I\subset\\{1,2,\cdots,l\\}\\\
|I|=r\end{subarray}}\left(\prod_{\begin{subarray}{c}i\in I\\\ j\notin
I\end{subarray}}\frac{q_{2}^{-1}x_{i}-x_{j}}{x_{i}-x_{j}}\right)\prod_{i\in
I}p_{i},\qquad r=1,2,\cdots,l,$ (43)

which in turn also Poisson-commute with each other, $\\{H_{r},H_{s}\\}=0$,
$r,s=1,2,\cdots,l$. They are precisely the classical Hamiltonians of the
trigonometric Ruijsenaars-Schneider model of $GL(l)$-type.

In the classical limit $q_{1}=1$, the effective field theory on the
worldvolume of the M2-branes is the 3d $\mathcal{N}=4$ theory subject to the
Rozansky-Witten twist, which flows to the 3d topological sigma model with the
target space being the Higgs branch $\mathcal{M}_{q_{2}}^{(l)}$
Rozansky1996HyperKhlerGA . The algebra $\text{M2}_{l,0,0}^{q_{1}=1}$ of local
operators of this theory is thus the commutative algebra of holomorphic
functions on the space $\mathcal{M}_{q_{2}}^{(l)}$. It was shown in
AOblomkov2003DoubleAH  that this algebra is identical to
$\text{\bf{S}}\boldsymbol{\ddot{\text{\bf{H}}}}^{(l)}_{1,q_{2}}$, namely, the
$q_{1}=1$ limit of the spherical DAHA
$\text{\bf{S}}\boldsymbol{\ddot{\text{\bf{H}}}}^{(l)}_{q_{1},q_{2}}$. Note
that this limit is different from the other limit to the symmetric product of
quantum torus algebras we observed earlier, due to the crucial exchange of the
$q_{1}$ and $q_{2}$ parameters.

Turning on $q_{1}\neq 1$, the Rozansky-Witten twisted 3d $\mathcal{N}=4$
theory on $\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}_{t}$ gets
$\Omega$-deformed, localized onto the line $\\{0\\}\times\mathbb{R}_{t}$. The
local operators then form a non-commutative associative algebra
$\text{M2}_{l,0,0}$, which is a deformation of the algebra of functions on
$\mathcal{M}_{q_{2}}^{(l)}$ Yagi:2014toa . The quantization takes place in the
way that the canonical Poisson brackets (42) are uplifted to the canonical
commutation relations,

$\displaystyle[X_{i},X_{j}]=\left[q_{1}^{D_{X_{i}}},q_{1}^{D_{X_{j}}}\right]=0,\qquad
q_{1}^{D_{X_{i}}}X_{j}=q_{1}^{\delta_{i,j}}X_{j}q_{1}^{D_{X_{i}}},\qquad
i,j=1,2,\cdots l.$ (44)

The so-obtained quantization of the multiplicative quiver variety
$\mathcal{M}_{q_{2}}^{(l)}$ precisely yields the spherical DAHA
$\text{\bf{S}}\boldsymbol{\ddot{\text{\bf{H}}}}^{(l)}_{q_{1},q_{2}}$
JORDAN2014420 ; Balagovic2016TheHI ; Wen:2023 . The construction involves the
quantum Hamiltonian reduction of the algebra of quantum differential operators
on $GL(l)$. In particular, the classical Hamiltonians (43) uplift to the
mutually commuting Macdonald operators,

$\displaystyle\hat{H}_{r}=q_{2}^{-\frac{r(r-1)}{2}}\sum_{\begin{subarray}{c}I\subset\\{1,2,\cdots,l\\}\\\
|I|=r\end{subarray}}\left(\prod_{\begin{subarray}{c}i\in I\\\ j\notin
I\end{subarray}}\frac{q_{2}^{-1}X_{i}-X_{j}}{X_{i}-X_{j}}\right)\prod_{i\in
I}q_{1}^{D_{X_{i}}},\qquad r=1,2,\cdots,l,$ (45)

furnishing the quantum Hamiltonians of the trigonometric Ruijsenaars-Schneider
model of $GL(l)$-type. Thus, we conclude that the algebra of local operators
on the parallel M2-branes is given by the spherical DAHA,
$\text{M2}_{l,0,0}=\text{\bf{S}}\boldsymbol{\ddot{\text{\bf{H}}}}_{q_{1},q_{2}}^{(l)}$.

By the trialiy of
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$, the surjective
algebra homomorphism from the quantum toroidal algebra of $\mathfrak{gl}(1)$
to the spherical DAHA
$\text{\bf{S}}\boldsymbol{\ddot{\text{\bf{H}}}}_{q_{1},q_{2}}^{(l)}$ is
obtained simply by exchanging $q_{1}$ and $q_{2}$ in (35). For completeness,
we write out here the surjective algebra homomorphism

$\displaystyle\rho_{\text{M2}_{l,0,0}}:U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\twoheadrightarrow\text{M2}_{l,0,0}$
(46)

in terms of the modes as

$\displaystyle\begin{split}\hfil\displaystyle\begin{split}&E_{k}\mapsto\frac{1}{1-q_{1}}\sum_{i=1}^{l}\left(\prod_{j\neq
i}\frac{q_{2}X_{i}-X_{j}}{X_{i}-X_{j}}\right)X_{i}^{k}q_{1}^{-D_{X_{i}}},\\\
&F_{k}\mapsto\frac{1}{1-q_{1}^{-1}}\sum_{i=1}^{l}\left(\prod_{j\neq
i}\frac{q_{2}^{-1}X_{i}-X_{j}}{X_{i}-X_{j}}\right)q_{1}^{D_{X_{i}}}X_{i}^{k},\\\
\end{split}\qquad k\in\mathbb{Z},\\\ \hfil\displaystyle\begin{split}&H_{\pm
r}\mapsto\frac{1}{1-q_{1}^{\mp r}}\sum_{i=1}^{l}X_{i}^{\pm r},\qquad
r\in\mathbb{Z}_{>0},\\\ &C\mapsto 1,\quad C^{\perp}\mapsto
1.\end{split}\end{split}$ (47)

In particular, the Macdonald operators (45) are achieved by taking suitable
combinations of $F_{0}$ and the commutators of
$\left(\text{ad}^{i-1}F_{0}\right)(F_{\pm 1})$ with $i=1,2,\cdots,l$, and
picking up the images under the map $\rho_{\text{M2}_{l,0,0}}$.

Recall that the Higgs branch of the effective 3d $\mathcal{N}=4$ theory on the
stack of $l$ D2-branes, wrapping
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}_{t}$, placed upon $1$
D6-brane, wrapping
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}_{t}\times\mathbb{C}\times\mathbb{C}$,
is the Hilbert scheme of $l$ points on $\mathbb{C}\times\mathbb{C}$, which is
given by the Nakajima quiver variety for the same Jordan quiver (Figure 4).
The algebra of local operators on the stack of D2-branes is the quantized
algebra of holomorphic functions on this quiver variety, which turns out to be
the spherical subalgebra of the rational double affine Hecke algebra
(spherical rational DAHA, $\mathbf{Sr\ddot{H}}$) of $GL(l)$ Costello:2017fbo ;
Gaiotto:2020vqj . This is precisely what was called the (rational) Calogero
representation of the 1-shifted affine Yangian of $\mathfrak{gl}(1)$
($Y_{1}(\widehat{\mathfrak{gl}}(1))$) in Gaiotto:2020dsq . What we have here
is its multiplicative uplift $-$ the spherical DAHA of $GL(l)$ as a
representation of the quantum toroidal algebra of $\mathfrak{gl}(1)$ $-$
established by replacing the complex manifold $\mathbb{C}\times\mathbb{C}$ by
$\mathbb{C}^{\times}\times\mathbb{C}^{\times}$ (see table 3).161616The
intermediate step, namely, the spherical trigonometric DAHA
$\mathbf{St\ddot{H}}$ as a representation of the affine Yangian of
$\mathfrak{gl}(1)$ ($Y(\widehat{\mathfrak{gl}}(1))$), can also be achieved as
the quantization of the Hilbert scheme of points in
$\mathbb{C}\times\mathbb{C}^{\times}$. See Finkelberg2010 , for instance. The
associated twisted M-theory background is thus expected to have the complex
manifold $\mathbb{C}\times\mathbb{C}^{\times}$ (see table 3). See
Kodera:2016faj ; Gaiotto:2019wcc  for an approach from the 3d mirror dual
side. See also Gaiotto:2020vqj  for another incarnation of the spherical DAHA
as the quantized Coulomb branch of the 4d $\mathcal{N}=2^{*}$ theory
compactified on a circle.

##### Fusion of parallel M2-branes and coproduct of quantum toroidal algebra

Having identified the algebra of local operators on the parallel M2-branes
with the spherical DAHA, let us examine how the statement is compatible with
the fusion operation of the line defects. Due to the triality, we may restrict
to the case of the M2-branes supported on
$\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}_{t}$ without losing
generality.

Let us first remind that the fusion operation for universal line defects
implies the existence of the coproduct of the universal associative algebra
Gaiotto:2020dsq , since the fusion exhibits a representation of the universal
associative algebra in the tensor product with itself.171717Recall that the
coassociativity,
$(\Delta\otimes\text{id})\Delta=(\text{id}\otimes\Delta)\Delta$, follows from
the fact that changing the ordering of two consecutive fusion operations
results in the same line defect. We also remind that the counit
$\epsilon:U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\to\mathbb{C}$
is provided by the trivial line defect, i.e.,
$\epsilon=\rho_{\text{M2}_{0,0,0}}$, since the fusion with a trivial line
defect is the identity operation:
$(\text{id}\otimes\epsilon)\Delta=(\epsilon\otimes\text{id})\Delta=\text{id}$.
In our case, the quantum toroidal algebra of $\mathfrak{gl}(1)$ is indeed
equipped with the coproduct
$\Delta:U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\to
U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\,\widehat{\otimes}\,U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$
given by (233).

fusion$\mathcal{L}_{0,m,0}$$\mathcal{L}_{0,m^{\prime},0}$$\mathcal{L}_{0,m+m^{\prime},0}$$\text{M2}_{0,m,0}$$\text{M2}_{0,m^{\prime},0}$$\text{M2}_{0,m+m^{\prime},0}$$\iota_{m,m^{\prime}}$$\otimes$
Figure 5: Fusion of line defects and algebra embedding induced by the
coproduct

Then, we specialize to the line defect $\mathcal{L}_{0,m+m^{\prime},0}$
produced by $m+m^{\prime}$ parallel M2-branes ($m,m^{\prime}\geq 0$). We may
split them into two groups composed of $m$ and $m^{\prime}$ M2-branes,
yielding two line defects $\mathcal{L}_{0,m,0}$ and
$\mathcal{L}_{0,m^{\prime},0}$ lying on top of each other. The original line
defect $\mathcal{L}_{0,m+m^{\prime},0}$ should be recovered once the two line
defects $\mathcal{L}_{0,m,0}$ and $\mathcal{L}_{0,m^{\prime},0}$ are fused
together (Figure 5). Thus, the coupling of the composite line defect
$\mathcal{L}_{0,m+m^{\prime},0}$ is built from the couplings of the base line
defects $\mathcal{L}_{0,m,0}$ and $\mathcal{L}_{0,m^{\prime},0}$. This implies
there is an embedding of the algebra
$\iota_{m,m^{\prime}}:\text{M2}_{0,m+m^{\prime},0}\hookrightarrow\text{M2}_{0,m,0}\otimes\text{M2}_{0,m^{\prime},0}$,
induced from the coproduct $\Delta$ through the surjective algebra maps
associated to these line defects. Namely, we have an isomorphism,

$\displaystyle(\rho_{\text{M2}_{0,m,0}}\otimes\rho_{\text{M2}_{0,m^{\prime},0}})\Delta\simeq\rho_{\text{M2}_{0,m+m^{\prime},0}},$
(48)

which can also be represented by the commutative diagram:

${U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))}$${U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\,{\widehat{\otimes}}\,U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))}$${\text{\bf{S}}\boldsymbol{\ddot{\text{\bf{H}}}}^{(m+m^{\prime})}_{q_{2},q_{1}}}$${\text{\bf{S}}\boldsymbol{\ddot{\text{\bf{H}}}}^{(m)}_{q_{2},q_{1}}{\otimes}\,\text{\bf{S}}\boldsymbol{\ddot{\text{\bf{H}}}}^{(m^{\prime})}_{q_{2},q_{1}}}$$\scriptstyle{\Delta}$$\scriptstyle{\rho_{\text{M2}_{0,m+m^{\prime},0}}}$$\scriptstyle{\rho_{\text{M2}_{0,m,0}}\otimes\,\rho_{\text{M2}_{0,m^{\prime},0}}}$

$\circlearrowleft$

$\scriptstyle{\iota_{m,m^{\prime}}}$ (49)

It is straightforward to verify that the suggested relation (48) holds. The
left hand side can be computed using the explicit forms of the
$q_{2}$-difference operators (35) for the Macdonald representation
$\rho_{\text{M2}_{0,m,0}}$ and the coproduct $\Delta$ (233) of
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$. The resulting
$q_{2}$-difference operators is not symmetric under the action of the extended
permutation group $S_{m+m^{\prime}}$, but they can be easily mapped to the
symmetric ones by conjugating

$\displaystyle\tilde{g}(\\{X_{i}\\}_{i=1}^{m},\\{X_{j}\\}_{j=m+1}^{m+m^{\prime}})=\prod_{i=1}^{m}X_{i}^{m^{\prime}\frac{\log
q_{1}}{\log
q_{2}}}\prod_{j=m+1}^{m+m^{\prime}}\frac{\left(q_{1}^{-1}\frac{X_{j}}{X_{i}};q_{2}\right)_{\infty}}{\left(\frac{X_{j}}{X_{i}};q_{2}\right)_{\infty}}.$
(50)

The outcome is precisely the $q_{2}$-difference operators (35) for
$\rho_{\text{M2}_{0,m+m^{\prime},0}}$, i.e., the right hand side of (48). In
other words, the conjugation by $\tilde{g}$ establishes the desired algebra
embedding
$\iota_{m,m^{\prime}}:\text{M2}_{0,m+m^{\prime},0}\hookrightarrow\text{M2}_{0,m,0}\otimes\text{M2}_{0,m^{\prime},0}$.

####  2.1.3 Non-parallel M2-branes and generalized Macdonald representation

Finally, let us examine the most generic case for the topological line defects
in the 5d Chern-Simons theory created by non-parallel M2-branes. Namely, we
put $l+m+n$ M2-branes, where $l$ of them wrap
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}_{t}$; $m$ of them wrap
$\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}_{t}$; and $n$ of them wrap
$\mathbb{R}^{2}_{{\varepsilon}_{3}}\times\mathbb{R}_{t}$. We denote the
algebra of local operators on this line defect by $\text{M2}_{l,m,n}$. Again
by the Koszul duality, there must be a surjective homomorphism from the
universal associative algebra

$\displaystyle\rho_{\text{M2}_{l,m,n}}:U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\twoheadrightarrow\text{M2}_{l,m,n},$
(51)

which controls the coupling of the line defect $\mathcal{L}_{l,m,n}$.

The line defect $\mathcal{L}_{l,m,n}$ can be constructed by the consecutive
fusion of three line defects $-$ $\mathcal{L}_{l,0,0}$, $\mathcal{L}_{0,m,0}$,
and $\mathcal{L}_{0,0,n}$ $-$ each of which originates from the M2-branes
wrapping three different topological planes. Algebraically, the fusion induces
a reconstruction of the above algebra map (51) as the composition of the
coproducts of the universal associative algebra with the tensor product of
three representations for these parallel M2-branes (surjective algebra maps to
spherical DAHAs
$\text{\bf{S}}\boldsymbol{\ddot{\text{\bf{H}}}}_{q_{1},q_{2}}^{(l)}$,
$\text{\bf{S}}\boldsymbol{\ddot{\text{\bf{H}}}}_{q_{2},q_{3}}^{(m)}$, and
$\text{\bf{S}}\boldsymbol{\ddot{\text{\bf{H}}}}_{q_{3},q_{1}}^{(n)}$).
Namely,181818Due to the coassociativity of
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$,
$\Delta^{2}:=(\text{id}\otimes\Delta)\Delta=(\Delta\otimes\text{id})\Delta$.

$\displaystyle\rho_{\text{M2}_{l,m,n}}=(\rho_{\text{M2}_{l,0,0}}\otimes\rho_{\text{M2}_{0,m,0}}\otimes\rho_{\text{M2}_{0,0,n}})\Delta^{2}.$
(52)

The homomorphism obtained in this way can be presented in terms of the modes
as

$\displaystyle\begin{split}\hfil\displaystyle\begin{split}&E_{k}\mapsto\frac{1}{1-q_{1}}\sum_{i=1}^{l}\prod_{\begin{subarray}{c}j=1\\\
j\neq
i\end{subarray}}^{l}\frac{q_{2}X_{i}-X_{j}}{X_{i}-X_{j}}\prod_{j=1}^{m}\frac{q_{1}X_{i}-X_{j}^{\prime}}{X_{i}-X_{j}^{\prime}}\prod_{j=1}^{n}\frac{q_{1}X_{i}-X_{j}^{\prime\prime}}{X_{i}-X_{j}^{\prime\prime}}X_{i}^{k}q_{1}^{-D_{X_{i}}}\\\
&\qquad+\frac{1}{1-q_{2}}\sum_{i=1}^{m}\prod_{j=1}^{l}\frac{q_{2}X^{\prime}_{i}-X_{j}}{X^{\prime}_{i}-X_{j}}\prod_{\begin{subarray}{c}j=1\\\
j\neq
i\end{subarray}}^{m}\frac{q_{3}X^{\prime}_{i}-X_{j}^{\prime}}{X^{\prime}_{i}-X_{j}^{\prime}}\prod_{j=1}^{n}\frac{q_{2}X^{\prime}_{i}-X_{j}^{\prime\prime}}{X^{\prime}_{i}-X_{j}^{\prime\prime}}(X_{i}^{\prime})^{k}q_{2}^{-D_{X^{\prime}_{i}}}\\\
&\qquad+\frac{1}{1-q_{3}}\sum_{i=1}^{n}\prod_{j=1}^{l}\frac{q_{3}X^{\prime\prime}_{i}-X_{j}}{X^{\prime\prime}_{i}-X_{j}}\prod_{j=1}^{m}\frac{q_{3}X^{\prime\prime}_{i}-X_{j}^{\prime}}{X^{\prime\prime}_{i}-X_{j}^{\prime}}\prod_{\begin{subarray}{c}j=1\\\
j\neq
i\end{subarray}}^{n}\frac{q_{1}X^{\prime\prime}_{i}-X_{j}^{\prime\prime}}{X^{\prime\prime}_{i}-X_{j}^{\prime\prime}}(X_{i}^{\prime\prime})^{k}q_{3}^{-D_{X^{\prime\prime}_{i}}}\\\
&F_{k}\mapsto\frac{1}{1-q_{1}^{-1}}\sum_{i=1}^{l}\prod_{\begin{subarray}{c}j=1\\\
j\neq
i\end{subarray}}^{l}\frac{q_{2}^{-1}X_{i}-X_{j}}{X_{i}-X_{j}}\prod_{j=1}^{m}\frac{q_{2}^{-1}X_{i}-X_{j}^{\prime}}{q_{1}q_{2}^{-1}X_{i}-X_{j}^{\prime}}\prod_{j=1}^{n}\frac{q_{3}^{-1}X_{i}-X_{j}^{\prime\prime}}{q_{1}q_{3}^{-1}X_{i}-X_{j}^{\prime\prime}}q_{1}^{D_{X_{i}}}X_{i}^{k}\\\
&\qquad+\frac{1}{1-q_{2}^{-1}}\sum_{i=1}^{m}\prod_{j=1}^{l}\frac{q_{1}^{-1}X^{\prime}_{i}-X_{j}}{q_{1}^{-1}q_{2}X^{\prime}_{i}-X_{j}}\prod_{\begin{subarray}{c}j=1\\\
j\neq
i\end{subarray}}^{m}\frac{q_{3}^{-1}X^{\prime}_{i}-X_{j}^{\prime}}{X^{\prime}_{i}-X_{j}^{\prime}}\prod_{j=1}^{n}\frac{q_{3}^{-1}X^{\prime}_{i}-X_{j}^{\prime\prime}}{q_{2}q_{3}^{-1}X^{\prime}_{i}-X_{j}^{\prime\prime}}q_{2}^{D_{X^{\prime}_{i}}}(X_{i}^{\prime})^{k}\\\
&\qquad+\frac{1}{1-q_{3}^{-1}}\sum_{i=1}^{n}\prod_{j=1}^{l}\frac{q_{1}^{-1}X^{\prime\prime}_{i}-X_{j}}{q_{1}^{-1}q_{3}X^{\prime\prime}_{i}-X_{j}}\prod_{j=1}^{m}\frac{q_{2}^{-1}X^{\prime\prime}_{i}-X_{j}^{\prime}}{q_{2}^{-1}q_{3}X^{\prime\prime}_{i}-X_{j}^{\prime}}\prod_{\begin{subarray}{c}j=1\\\
j\neq
i\end{subarray}}^{n}\frac{q_{1}^{-1}X^{\prime\prime}_{i}-X_{j}^{\prime\prime}}{X^{\prime\prime}_{i}-X_{j}^{\prime\prime}}q_{3}^{D_{X^{\prime\prime}_{i}}}(X_{i}^{\prime\prime})^{k},\end{split}\qquad
k\in\mathbb{Z},\\\ \hfil\displaystyle\begin{split}&H_{\pm
r}\mapsto\frac{1}{1-q_{1}^{\mp r}}\sum_{i=1}^{l}X_{i}^{\pm
r}+\frac{1}{1-q_{2}^{\mp r}}\sum_{i=1}^{m}(X_{i}^{\prime})^{\pm
r}+\frac{1}{1-q_{3}^{\mp r}}\sum_{i=1}^{n}(X_{i}^{\prime\prime})^{\pm
r},\qquad r\in\mathbb{Z}_{>0},\\\ &C\mapsto 1,\quad C^{\perp}\mapsto
1.\end{split}\end{split}$ (53)

Here, we conjugated the right hand side of (52) by

$\displaystyle\begin{split}&\hat{g}(\\{X_{i}\\}_{i=1}^{l},\\{X^{\prime}_{j}\\}_{j=1}^{m},\\{X^{\prime\prime}_{k}\\}_{k=1}^{n})\\\
&=\prod_{i=1}^{l}X_{i}^{m+n}\prod_{j=1}^{m}(X_{j}^{\prime})^{n}\prod_{i=1}^{l}\prod_{j=1}^{m}\left(1-\frac{q_{1}^{-1}X_{j}^{\prime}}{X_{i}}\right)\prod_{j=1}^{m}\prod_{k=1}^{n}\left(1-\frac{q_{2}^{-1}X_{k}^{\prime\prime}}{X_{j}^{\prime}}\right)\prod_{i=1}^{l}\prod_{k=1}^{n}\left(1-\frac{q_{1}^{-1}X_{k}^{\prime\prime}}{X_{i}}\right)\end{split}$
(54)

to put the resulting $q$-difference operators in a symmetric fashion as above.
Since this generalizes the Macdonald representation
$\rho_{\text{M2}_{0,0,n}}$, we call it the generalized Macdonald
representation of the quantum toroidal algebra
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$.

What we established is the multiplicative uplift of the generalized (rational)
Calogero representation of the 1-shifted affine Yangian of $\mathfrak{gl}(1)$
obtained in Gaiotto:2020dsq , as a result of changing the complex manifold
from $\mathbb{C}\times\mathbb{C}$ to
$\mathbb{C}^{\times}\times\mathbb{C}^{\times}$. In this regard, we may refer
to it as the generalized trigonometric Ruijsenaars-Schneider representation of
the quantum toroidal algebra of $\mathfrak{gl}(1)$ as well.

###  2.2 M5-branes and holomorphic surface defects

The M5-branes in the twisted M-theory can be supported on
$\mathbb{R}^{2}_{{\varepsilon}_{c+1}}\times\mathbb{R}^{2}_{{\varepsilon}_{c-1}}\times
C^{\mathbf{(p,q)}}$, where $c\in\\{1,2,3\\}$ and
$C^{\mathbf{(p,q)}}=\\{X^{\mathbf{q}}Z^{-\mathbf{p}}=\text{const}\\}\subset\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$
is a holomorphic curve. Passing to the 5d $\mathfrak{gl}(1)$ Chern-Simons
theory, the M5-branes descend to a holomorphic surface defect supported on the
curve $C^{(\mathbf{p},\mathbf{q})}$. Fixing the holomorphic part of the
support as $C^{\mathbf{(p,q)}}$, the most generic configuration comprises $L$,
$M$, and $N$ M5-branes wrapping
$\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}^{2}_{{\varepsilon}_{3}}\times
C^{\mathbf{(p,q)}}$,
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}^{2}_{{\varepsilon}_{3}}\times
C^{\mathbf{(p,q)}}$, and
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}^{2}_{{\varepsilon}_{2}}\times
C^{\mathbf{(p,q)}}$, respectively. We call the induced holomorphic surface
defect $\mathcal{S}_{L,M,N}^{\mathbf{(p,q)}}$.191919We emphasize that the
$c$-th subscript of the surface defect $\mathcal{S}_{L,M,N}^{\mathbf{(p,q)}}$
indicates the number of M5-branes wrapping the $(c\pm 1)$-th topological
planes, whereas the $c$-th subscript of the line defect $\mathcal{L}_{l,m,n}$
denotes the number of M2-branes wrapping the $c$-th topological plane.

The local operators of the 5d $\mathfrak{gl}(1)$ Chern-Simons theory localized
at the holomorphic curve $C^{\mathbf{(p,q)}}$ are the modes of the ghost field
under the Laurent expansion along the transverse holomorphic direction. They
form a chiral algebra, say $\text{Obs}_{q_{1},q_{2}}^{C^{\mathbf{(p,q)}}}$, by
their OPEs. The BRST-invariant coupling of a surface defect, carrying a chiral
algebra $\mathcal{V}$ of local operators, is expected to be classified by
certain universal chiral algebra
$\left(\text{Obs}_{q_{1},q_{2}}^{C^{\mathbf{(p,q)}}}\right)^{!}$ and
surjective homomorphisms
$\left(\text{Obs}_{q_{1},q_{2}}^{C^{\mathbf{(p,q)}}}\right)^{!}\twoheadrightarrow\mathcal{V}$
Costello:2020jbh . We anticipate this universal chiral algebra is again given
by the quantum toroidal algebra of $\mathfrak{gl}(1)$, enhanced by a proper
chiral algebra structure.202020More precisely, we expect that the quantum
toroidal algebra of $\mathfrak{gl}(1)$ is equipped with a quantum vertex
algebra structure Frenkel:1996nz ; Etingof2000QuantizationOL , and that the
surjective map
$\rho_{\text{M5}_{L,M,N}^{\mathbf{(p,q)}}}:U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\twoheadrightarrow\text{M5}_{L,M,N}^{\mathbf{(p,q)}}$
is uplifted to a quantum vertex algebra homomorphism. For instance, the
$q$-boson algebra for a single M5-brane and the $q$-deformed $W$-algebras for
multiple parallel M5-branes are known to admit the quantum vertex algebra
structure Frenkel:1996nz . Nevertheless, the quantum vertex algebra structure
for the whole quantum toroidal algebra of $\mathfrak{gl}(1)$ is not well
understood, and we will restrict our discussion to the mode algebras in this
work. See Chen:2024lsc  for a study of quantum vertex algebra structure for
higher-rank quantum toroidal algebras.

In principle, we can derive the OPEs of local operators in the following way.
Let us take the example of $C^{\mathbf{(1,0)}}=\mathbb{C}^{\times}_{X}$. We
can explicitly write the surface defect coupling as

$\displaystyle\sum_{m\in\mathbb{Z}}\int_{\mathbb{C}^{\times}_{X}}W^{(m)}(A_{\bar{X}})_{m},$
(55)

where $(A_{\bar{X}})_{m}$ are the modes of the gauge field under the Laurent
expansion along the transverse plane $\mathbb{C}^{\times}_{Z}$ and $W^{(m)}$
are local operators on the surface defect. Then, the OPEs of the local
operators can be deduced as conditions for the anomalous Feynman diagrams to
cancel each other (see Costello:2020jbh  for the case of surface defect in
holomorphic Chern-Simons theory). We will not conduct such a perturbative
analysis of the 5d Chern-Simons theory on
$\mathbb{R}_{t}\times\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$ in
the present work. Instead, we will restrict our attention to the mode algebra,
referred to as $\text{M5}_{L,M,N}^{\mathbf{(p,q)}}$, of the local operators on
the surface defect $\mathcal{S}_{L,M,N}^{\mathbf{(p,q)}}$. We will suggest
natural candidates for them and the surjective algebra homomorphisms
$\rho_{\text{M5}_{L,M,N}^{\mathbf{(p,q)}}}:U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\twoheadrightarrow\text{M5}_{L,M,N}^{\mathbf{(p,q)}}$.

####  2.2.1 Single M5-brane and free $q$-boson

In this section, we will only consider M5-branes whose holomorphic part of the
support is $\mathbb{C}^{\times}_{X}$ (namely,
$C^{\mathbf{(1,0)}}=\mathbb{C}^{\times}_{X}$). We will come back to M5-branes
supported on general holomorphic curves $C^{\mathbf{(p,q)}}$ in section 5.
Consider a single M5-brane wrapping
$\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}^{2}_{{\varepsilon}_{3}}\times\mathbb{C}^{\times}_{X}$.
Deforming
$\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}^{2}_{{\varepsilon}_{3}}$
to the one-centered Taub-NUT space and reducing the twisted M-theory along the
Taub-NUT circle, the M5-brane becomes a D4-brane which intersects the emergent
single D6-brane along the holomorphic surface $\mathbb{C}^{\times}_{X}$.

If the holomorphic plane transverse to the D4-brane were $\mathbb{C}$, the
D4-D4 open string would give rise to the 5d $\mathcal{N}=2$ $U(1)$ gauge
theory, coupled to the chiral fermions living on $\mathbb{C}^{\times}_{X}$
arising from the D4-D6 open string Green:1996dd ; Dijkgraaf:2007sw . Due to
the $\Omega$-background, the 5d gauge theory is localized to the 3d $U(1)$
Chern-Simons theory Yagi:2013fda ; Lee:2013ida ; Luo:2014sva . Therefore, the
algebra of local operators on the surface defect of the 5d Chern-Simons theory
is given by the algebra of chiral fermions coupled to the 3d Chern-Simons
theory, which is identified to be the free boson algebra
$\widehat{\mathfrak{gl}}(1)$ Costello:2016nkh .

In our case, however, the open strings may wind around the compact circle of
the plane $\mathbb{C}^{\times}_{Z}\simeq\mathbb{R}_{z}\times S^{1}$ transverse
to the D4-brane, producing infinite numbers of winding modes. The effective 5d
gauge theory for the D4-D4 open string now contains infinite numbers of
Kaluza-Klein modes, which can be uplifted to the 6d $\mathcal{N}=(1,1)$ $U(1)$
gauge theory compactified on a circle, as a result. Due to the
$\Omega$-background, the 6d gauge theory is localized into the 4d
$\mathfrak{gl}(1)$ Chern-Simons theory on $\mathbb{R}\times
S^{1}\times\mathbb{C}^{\times}_{X}$ Costello:2018txb . The chiral fermions on
$\mathbb{C}^{\times}_{X}$ couple to the 4d $\mathfrak{gl}(1)$ Chern-Simons
theory, and the quantum corrections would uplift the algebra of local
operators to a representation of the quantum affine algebra of
$\mathfrak{gl}(1)$, namely the free $q$-boson algebra
$U_{q}(\widehat{\mathfrak{gl}}(1))$.212121We stress, however, that this claim
regarding the quantum corrections to the algebra of local operators on the
holomorphic surface defect in the 4d Chern-Simons theory is an independent
statement that needs to be proven within its perturbation theory, just as was
done for the surface defect in the holomorphic Chern-Simons theory in
Costello:2020jbh . To the best of our knowledge, such a perturbative analysis
for the holomorphic surface defect has not yet been established in the 4d
Chern-Simons theory, unlike the studies for topological line defects in
Costello:2017dso  and for topological surface defects in Costello:2019tri .
Since the representation has to be non-trivial, it should be the $q$-boson
algebra itself. Therefore, we suggest that the algebra
$\text{M5}_{0,1,0}^{\mathbf{(1,0)}}$ of local operators on the surface defect
$\mathcal{S}_{0,1,0}^{\mathbf{(1,0)}}$ is the free $q$-boson algebra.

From the triality of the twisted M-theory, we expect that a single M5-brane on
$\mathbb{R}^{2}_{{\varepsilon}_{c+1}}\times\mathbb{R}^{2}_{{\varepsilon}_{c-1}}\times\mathbb{C}^{\times}_{X}$,
$c\in\\{1,2,3\\}$, would also support a $q$-boson algebra
$\text{M5}_{c}^{\mathbf{(1,0)}}$ with the permutation on
$(q_{1},q_{2},q_{3})$. We used the short-hand notation
$(\delta_{c,a})_{a=1}^{3}\equiv c$ for the subscript only for the cases with
single M5-brane. The free $q$-boson algebra $\text{M5}_{c}^{\mathbf{(1,0)}}$
is the algebra generated by the modes $a_{r}^{(c)}$, $r\in\mathbb{Z}$, subject
to the commutation relation

$\displaystyle[a_{r}^{(c)},a_{s}^{(c)}]=-\delta_{r+s,0}\frac{r}{\kappa_{r}}(q_{c}^{r/2}-q_{c}^{-r/2})^{3}.$
(56)

Meanwhile, we suggested that the universal associative algebra for the 5d
$\mathfrak{gl}(1)$ Chern-Simons theory on
$\mathbb{R}_{t}\times\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$ is
the quantum toroidal algebra
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$, from which
there is a surjective algebra homomorphism to each
$\text{M5}_{c}^{\mathbf{(1,0)}}$. Indeed, it is straightforward to see that
two of the defining relations (224c) and (224d) for
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$ are equivalent
to the $q$-boson commutation relation satisfied by $H_{r}$,

$\displaystyle[H_{r},H_{s}]=\delta_{r+s,0}\frac{r}{\kappa_{r}}(C^{r}-C^{-r}).$
(57)

At $(C,C^{\perp})=\left(q_{c}^{1/2},1\right)$ where $c\in\\{1,2,3\\}$, this
vertex representation extends to the whole quantum toroidal algebra
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$. Namely, we have
surjective algebra homomorphisms

$\displaystyle\rho_{\text{M5}_{c}^{\mathbf{(1,0)}}}:U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\twoheadrightarrow\text{M5}_{c}^{\mathbf{(1,0)}},$
(58)

given by

$\displaystyle\begin{split}&E(X)\mapsto-e^{a_{0}^{(c)}\frac{\log q_{c+1}\log
q_{c-1}}{\log q_{c}}}\frac{1-q_{c}}{\kappa_{1}}\eta_{c}(X),\quad F(X)\mapsto
e^{-a_{0}^{(c)}\frac{\log q_{c+1}\log q_{c-1}}{\log
q_{c}}}\frac{1-q_{c}^{-1}}{\kappa_{1}}\xi_{c}(X),\\\
&H_{r}\mapsto\frac{a_{r}^{(c)}}{q_{c}^{r/2}-q_{c}^{-r/2}},\qquad(C,C^{\perp})\mapsto\left(q_{c}^{1/2},1\right),\end{split}$
(59)

where we used the indices in cyclical manner so that
$\\{c-1,c,c+1\\}=\\{1,2,3\\}$. Here, the vertex operators are given by

$\displaystyle\begin{split}&\eta_{c}(X)=\exp\left(-\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\frac{1}{(q_{c}^{r/2}-q_{c}^{-r/2})^{2}}a_{-r}^{(c)}X^{r}\right)\exp\left(-\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\frac{q_{c}^{-{r/2}}}{(q_{c}^{r/2}-q_{c}^{-r/2})^{2}}a_{r}^{(c)}X^{-r}\right)\\\
&\xi_{c}(X)=\exp\left(\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\frac{q_{c}^{r/2}}{(q_{c}^{r/2}-q_{c}^{-r/2})^{2}}a_{-r}^{(c)}X^{r}\right)\exp\left(\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\frac{1}{(q_{c}^{r/2}-q_{c}^{-r/2})^{2}}a_{r}^{(c)}X^{-r}\right).\end{split}$
(60)

For later convenience, we also denote the image of $K^{\pm}(X)$ by vertex
operators $\varphi^{\pm}_{c}(X)$,

$\displaystyle
K^{\pm}(X)\mapsto\varphi_{c}^{\pm}(X)\equiv\exp\left(\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\frac{a_{\pm
r}^{(c)}}{q_{c}^{r/2}-q_{c}^{-r/2}}X^{\mp r}\right).$ (61)

We may represent the $q$-boson algebra (56) on the Fock space
$\mathcal{F}_{c}^{\mathbf{(1,0)}}(v)=\bigoplus_{\\{\lambda\\}}\mathbb{C}|v,\lambda\rangle^{(c)}$,
where the basis elements are enumerated by a single Young diagram
$\lambda=(\lambda_{i})_{i=1}^{\infty}$, $\lambda_{i}\geq\lambda_{i+1}\geq 0$.
The whole Fock space can be generated by acting the creation operators
$a_{-r}^{(c)}$, $r\in\mathbb{Z}_{>0}$ on the vacuum
$|v,\varnothing\rangle^{(c)}\in\mathcal{F}_{c}^{\mathbf{(1,0)}}(v)$ satisfying

$\displaystyle a_{r}^{(c)}|v,\varnothing\rangle^{(c)}=0,\quad r>0,\qquad
a_{0}^{(c)}|v,\varnothing\rangle^{(c)}=\frac{\log q_{c}}{\log q_{c+1}\log
q_{c-1}}\log v|v,\varnothing\rangle^{(c)},$ (62)

where $v\in\mathbb{C}^{\times}$ is a module parameter. The seemingly unusual
normalization of the zero-mode $a_{0}^{(c)}$ will be explained soon. This is a
faithful representation of the $q$-boson algebra,
$\text{M5}_{c}^{\mathbf{(1,0)}}$. Thus, what we found for single M5-brane can
be summarized into
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\twoheadrightarrow\text{M5}_{c}^{\mathbf{(1,0)}}\stackrel{{\scriptstyle\sim}}{{\smash{\longrightarrow}\rule{0.0pt}{1.72218pt}}}\text{End}(\mathcal{F}_{c}^{\mathbf{(1,0)}}(v))$
where $c\in\\{1,2,3\\}$.

For later use, we stress that the grading operator $d^{\perp}$ plays the role
of the conjugate variable to the zero-mode $a_{0}^{(c)}$. Indeed, from its
definition (234) it follows that

$\displaystyle[d^{\perp},a_{m}^{(c)}]=\frac{\log q_{c}}{\log q_{c+1}\log
q_{c-1}}\delta_{m,0},\qquad m\in\mathbb{Z}.$ (63)

This implies that the Fock representation on
$\mathcal{F}^{\mathbf{(1,0)}}_{c}$ can be extended to incorporate the grading
operator $d^{\perp}$ by regarding it as the shift operator for the module
parameter:
$q^{d^{\perp}}|v,\lambda\rangle^{(c)}=|vq^{-1},\lambda\rangle^{(c)}$ for any
$q\in\mathbb{C}^{\times}$.

Note that the representations (58) break the triality of
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$, in a way that
the duality of exchanging $q_{c+1}$ and $q_{c-1}$ is still preserved. This is
indeed to be expected from the corresponding M-brane configuration, since the
support of the M5-brane distinguishes two parameters $q_{c\pm 1}$ out of the
triple $(q_{1},q_{2},q_{3})$.

####  2.2.2 Parallel M5-branes and $q$-deformed $W$-algebra

Next, let us consider multiple parallel M5-branes all of which are supported
on
$\mathbb{R}^{2}_{{\varepsilon}_{c+1}}\times\mathbb{R}^{2}_{{\varepsilon}_{c-1}}\times\mathbb{C}^{\times}_{X}$.
By triality, it is enough to consider one of the three cases
$c\in\\{1,2,3\\}$, and let us fix $c=3$ and the number of M5-branes to be $N$.
The resulting holomorphic surface defect
$\mathcal{S}_{0,0,N}^{\mathbf{(1,0)}}$ carries the algebra
$\text{M5}_{0,0,N}^{\mathbf{(1,0)}}$ of local operators.

Under the reduction along the circle of
$\mathbb{C}^{\times}_{X}\simeq\mathbb{R}_{x}\times S^{1}$, the $N$ M5-branes
become $N$ D4-branes supported on
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}_{x}$.
Due to the $\Omega$-background, the local operators form a non-commutative
associative algebra $\text{M5}_{0,0,N}^{\mathbf{(1,0)}}$ on the fixed locus
$\\{0\\}\times\mathbb{R}_{x}$ of the isometry. Since there are infinite
numbers of winding modes along the transverse circle, the effective 5d $U(N)$
gauge theory for the D4-D4 open string contains infinite numbers of Kaluza-
Klein modes. Its Hilbert space is the equivariant K-theory of the moduli space
of $U(N)$ instantons, on which the algebra
$\text{M5}_{0,0,N}^{\mathbf{(1,0)}}$ of local operators can act. It is indeed
known that the equivariant K-theory of the moduli space of $U(N)$ instantons
admits an action of the quantum toroidal algebra
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$
2009arXiv0905.2555S , which would factor through a surjective algebra
homomorphism
$\rho_{\text{M5}_{0,0,N}^{\mathbf{(1,0)}}}:U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\twoheadrightarrow\text{M5}_{0,0,N}^{\mathbf{(1,0)}}$.

$\mathcal{S}^{\mathbf{(1,0)}}_{0,0,N}$$\mathcal{S}^{\mathbf{(1,0)}}_{0,0,N^{\prime}}$$\mathcal{S}^{\mathbf{(1,0)}}_{0,0,N+N^{\prime}}$fusion$\text{M5}^{\mathbf{(1,0)}}_{0,0,N}$$\text{M5}^{\mathbf{(1,0)}}_{0,0,N^{\prime}}$$\text{M5}^{\mathbf{(1,0)}}_{0,0,N+N^{\prime}}$$\delta_{N,N^{\prime}}$$\otimes$
Figure 6: Fusion of surface defects and algebra embedding induced by the
coproduct

In section 3, we will show that this algebra
$\text{M5}_{0,0,N}^{\mathbf{(1,0)}}$ is precisely the $q$-deformed $W$-algebra
of $\mathfrak{gl}(N)$ frenkel_reshetikhin_1995 ; Shiraishi:1995rp ;
Feigin:1995sf ; Awata:1995zk , which we denote by $q\mathcal{W}_{N}$. To
verify this, it is crucial to study the fusion of the surface defects and its
implications. First, note that the surface defect
$\mathcal{S}_{0,0,N+N^{\prime}}^{\mathbf{(1,0)}}$ can be obtained by fusing
two surface defects $\mathcal{S}_{0,0,N}^{\mathbf{(1,0)}}$ and
$\mathcal{S}_{0,0,N^{\prime}}^{\mathbf{(1,0)}}$ on top of each other (see
Figure 6). Just as in the case of the topological line defects, the fusion
implies that the coupling of the fused surface defect can be built from the
couplings of the individual surface defects before the fusion. Namely, there
is an algebra embedding
$\delta_{N,N^{\prime}}:\text{M5}_{0,0,N+N^{\prime}}^{\mathbf{(1,0)}}\hookrightarrow\text{M5}_{0,0,N}^{\mathbf{(1,0)}}\otimes\text{M5}_{0,0,N^{\prime}}^{\mathbf{(1,0)}}$
induced by the coproduct of
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$, which can be
represented by the following commutative diagram:

${U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))}$${U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\,{\widehat{\otimes}}\,U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))}$${q\mathcal{W}_{N+N^{\prime}}}$${q\mathcal{W}_{N}\,{\otimes}\,q\mathcal{W}_{N^{\prime}}}$$\scriptstyle{\Delta}$$\scriptstyle{\rho_{\text{M5}^{\mathbf{(1,0)}}_{0,0,N+N^{\prime}}}}$$\scriptstyle{\rho_{\text{M5}^{\mathbf{(1,0)}}_{0,0,N}}\otimes\,\rho_{\text{M5}^{\mathbf{(1,0)}}_{0,0,N^{\prime}}}}$

$\circlearrowleft$

$\scriptstyle{\delta_{N,N^{\prime}}}$ (64)

Then, by consecutively applying this diagram, we achieve an embedding of the
algebra

$\displaystyle\text{M5}_{0,0,N}^{\mathbf{(1,0)}}\hookrightarrow\text{M5}_{0,0,1}^{\mathbf{(1,0)}}\otimes\cdots\otimes\text{M5}_{0,0,1}^{\mathbf{(1,0)}},$
(65)

into the tensor product of free $q$-boson algebras. The fusion of surface
defects thus provides the free field realization of the M5-brane algebra
$\text{M5}_{0,0,N}^{\mathbf{(1,0)}}$.

In section 3, we will show that this free field realization exactly matches
with that of the $q$-deformed $W$-algebra, given in Feigin:1995sf ;
Awata:1995zk . The proof is given in two steps. First, we establish the Miura
transformation for the algebra $\text{M5}_{0,0,N}^{\mathbf{(1,0)}}$ which
explicitly realizes the commutative diagram (64) by the product of newly
defined Miura operators. Then, we prove that the algebra generated by such a
Miura transformation is completely characterized as the commutants of
screening charges in the tensor product of $q$-bosons, which precisely
recovers the ones in Feigin:1995sf ; Awata:1995zk .

We emphasize that our result therefore derives the Miura transformation for
the $q$-deformed $W$-algebra, which can be regarded as an independent result
within itself.

####  2.2.3 Non-parallel M5-branes and $q$-deformed $Y$-algebra

We further study more general configurations of non-parallel M5-branes, which
leads to an extension of the result for the parallel case. Here, by non-
parallel, we indicate that the topological part of the support may differ for
different M5-branes. This should not be confused with the holomorphic part of
the support, which is chosen to be
$\mathbb{C}^{\times}_{X}=C^{\mathbf{(1,0)}}$ for all the M5-branes. We will
return to the investigation of M5-branes supported on generic holomorphic
curves $C^{\mathbf{(p,q)}}$ in section 5.

Let us be given with $L+M+N$ M5-branes which share the common holomorphic part
of the support $\mathbb{C}^{\times}_{X}$. The topological part of the support
is determined by the function $c:I\mapsto c_{I}\in\\{1,2,3\\}$,
$I\in\\{1,2,\cdots,L+M+N\\}$, where the topological part of the support of the
$I$-th M5-brane is assigned to be
$\mathbb{R}^{2}_{{\varepsilon}_{c_{I}+1}}\times\mathbb{R}^{2}_{{\varepsilon}_{c_{I}-1}}$.
Let us set $(|c^{-1}(1)|,|c^{-1}(2)|,|c^{-1}(3)|)=(L,M,N)$. We call the
induced holomorphic surface defect in the 5d $\mathfrak{gl}(1)$ Chern-Simons
theory $\mathcal{S}_{L,M,N}^{\mathbf{(1,0)}}$, and the algebra of local
operators on this surface defect is denoted by
$\text{M5}_{L,M,N}^{\mathbf{(1,0)}}$.

Just as in the parallel case, the fusion operation of surface defects implies
that the coproduct of the quantum toroidal algebra
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$ induces the
algebra embedding

$\displaystyle\text{M5}_{L,M,N}^{\mathbf{(1,0)}}\hookrightarrow\text{M5}_{c_{1}}^{\mathbf{(1,0)}}\otimes\text{M5}_{c_{2}}^{\mathbf{(1,0)}}\otimes\cdots\otimes\text{M5}_{c_{L+M+N}}^{\mathbf{(1,0)}}$
(66)

into the tensor product of $q$-boson algebras. Thus, the algebra
$\text{M5}_{L,M,N}^{\mathbf{(1,0)}}$ admits a free field realization.

In section 3, we show that this free field realization is implemented by a
Miura transformation, with newly defined Miura operators. We also show that
the algebra generated by the Miura transformation can be completely
characterized as the commutants of properly defined screening charges in the
tensor product of $q$-bosons. Due to the equivalence that we prove, the latter
may be employed as the definition of the algebra
$\text{M5}_{L,M,N}^{\mathbf{(1,0)}}$. In particular, we verify that changing