Imperial-TP-KM-2022-4

Three approaches to chiral form interactions

Oleg Evnina,b and Karapet Mkrtchyanc
a Department of Physics, Faculty of Science, Chulalongkorn University,
Thanon Phayathai, Pathumwan, Bangkok 10330, Thailand b Theoretische
Natuurkunde, Vrije Universiteit Brussel (VUB) and International
Solvay Institutes, Pleinlaan 2, B-1050 Brussels, Belgium c Theoretical Physics
Group, Blackett Laboratory, Imperial College London SW7 2AZ, UK
<EMAIL_ADDRESS><EMAIL_ADDRESS>

ABSTRACT

We briefly review and critically compare three approaches to constructing
Lagrangian theories of self-interacting Abelian chiral form fields with
manifest Lorentz invariance. The first approach relies on the original ideas
of Pasti, Sorokin, and Tonin (PST) and has been explored since the late 1990s.
The second approach was introduced by Ashoke Sen in 2015. The third approach
has been developed over the last few years in the works of the present authors
and other collaborators and may be called the ‘clone field’ formalism since it
features an auxiliary ‘clone’ of the gauge field sector. We argue that this
last approach shares the attractive features of the other two while avoiding
their respective shortcomings. Like in Sen’s approach, within the clone field
formalism, arbitrary interactions can be straightforwardly included in any
number of dimensions (treating interactions becomes very difficult in the PST
formalism in dimensions greater than 6). Like in the PST approach, all the
auxiliary fields are gauged away on-shell (while in Sen’s approach, they
merely decouple from the physical fields but remain dynamical).

##  1 Chiral form fields

In analogy to ordinary Abelian gauge fields, higher form gauge fields are
defined by their $p$-form gauge potentials $A$ whose corresponding
$(p+1)$-form field strength $F$ is given by the exterior derivative $F\equiv
dA$. The free equation of motion is then $d\star F=0$, where $\star$ is the
Hodge dual. (The case $p=1$ evidently recovers the ordinary Maxwell field.)

When the number of spacetime dimensions $d$ equals 2 mod 4 (with Minkowski
signature), and $p=d/2-1$, it is possible to impose the selfduality relation

$\star\\!F=F,$ (1)

and thus reduce the dynamical content of the theory by half. This relation
automatically implies the equation of motion $d\star F=0$, since $F$ is exact
and $d$ is nilpotent. It thus describes a consistent truncation of the
ordinary free theory to one-half of its degrees of freedom. Such gauge fields
with selfdual field strengths are known as chiral.

While chiral gauge fields do not exist in four (Lorentzian) spacetime
dimensions, they commonly appear in higher-dimensional theories explored in
high-energy physics. Thus, a chiral 4-form is encountered in ten-dimensional
type IIB supergravity [1, 2, 3], and a chiral 2-form appears in the effective
worldvolume description of M5-branes [4, 5] in eleven-dimensional supergravity
and M-theory [6, 7, 8].

One question that is central to our considerations is how the free selfduality
relation (1) could be upgraded with nonlinear terms.111While the prospect of
constructing non-Abelian interactions of chiral forms [9, 10] provides an
important source of motivation in the context of this review, we shall in
practice entirely focus on self-interactions of Abelian chiral forms. (The
case of non-Abelian interactions is notoriously difficult, in particular, in
view of the no-go results of [9]). One could attempt to write

$\star\\!F={\cal G}(F)$ (2)

with an arbitrary function $\cal G$ from $(p+1)$-forms to $(p+1)$-forms. The
number of equations here is, however, the same as the number of form
components, thus one expects that all degrees of freedom will be eliminated
rather than the desired half. It is difficult to state an explicit condition
on $\cal G$ that guarantees that half of the degrees of freedom survive. The
situation becomes more manageable if one assumes that the system of algebraic
equations for $F$ given by (2) has been resolved to equivalently express
$F-\star F$ through $F+\star F$:

$F-\star F={\cal H}(F+\star F),$ (3)

where $\cal H$ is a function from $(p+1)$-forms to $(p+1)$-forms. In this
format, if only $\cal H$ satisfies, for all values of its argument,

$\star{\cal H}=-{\cal H},$ (4)

the number of independent equations drops to one-half of the number of form
components. This parametrization of the general nonlinear chiral form
equations was introduced in [11] and it provides a useful perspective for the
following considerations.

A well-known tough problem is constructing a Lagrangian description for chiral
field equations of motion, which is challenging even for free fields. In
attempts to devise such Lagrangian descriptions, difficulties arise in
simultaneously maintaining locality and manifest Lorentz invariance. (Early
influential constructions without manifest Lorentz invariance can be found in
[12] for free chiral forms, and in [13] for interacting ones.)

In this essay, we aim for reviewing and comparing three Lagrangian formalisms
for self-interacting Abelian chiral forms with manifest Lorentz invariance:
the Pasti-Sorokin-Tonin (PST) formalism originating from [14, 15, 16, 17], the
more recent formalism due to Ashoke Sen [18, 19], and another formalism
proposed in [11] based on the prior considerations by the present authors and
other collaborators in [20, 21, 22]. This sort of analysis has in fact been
invited in the conclusions of [23], and we are happy to provide it at this
point. A brief exposition of the main features of these Lagrangian
constructions will be followed by a side-by-side comparison of the three
approaches. We shall argue that our formalism shares the attractive features
of its predecessors while bypassing their respective shortcomings.

Throughout the exposition, we shall be using the differential form notation,
for which one can find a convenient summary in the appendices of [21]. We
mention here a few identities that are used particularly frequently in
relation to the exterior derivative $d$, the wedge product $\wedge$, the
interior product $\iota_{v}$, and the Hodge dual $\star$. We shall be using
the same letter to denote 1-forms and their corresponding vectors. For any
$p$-form $A$ and a vector $v$, one has

$\star\iota_{v}A=(-1)^{p-1}v\wedge\star A,\qquad\star(A\wedge
v)=\iota_{v}\star A,\qquad\iota_{v}(v\wedge A)+v\wedge\iota_{v}A=v^{2}A\,.$
(5)

The last relation is known as the projection-rejection identity. The operators
$d$, $v\wedge$, and $\iota_{v}$ are all nilpotent, while $d$ and $\iota_{v}$
satisfy an analog of the Leibnitz rule with respect to the wedge product.
Additionally, for any two $(2k+1)$-forms $G$ and $G_{1}$ in $4k+2$ spacetime
dimensions, one has

$\star\star G=G,\qquad G\wedge G_{1}=-G_{1}\wedge G,$ (6)

while for any forms $G$ and $G_{1}$ of equal degrees, $G\star G_{1}\equiv
G\wedge\star G_{1}=G_{1}\star G$.

##  2 Pasti-Sorokin-Tonin approach

In our review of the PST formalism, we follow the contemporary lucid
exposition in [23] due to Buratti, Lechner and Melotti. In $d=4k+2$ spacetime
dimensions, besides the $p=2k$-form gauge field $A$ with the field strength
$F\equiv dA$, the formalism includes an auxiliary scalar $a$. The Lagrangian
is

$\mathcal{L}_{\scriptscriptstyle\mathrm{PST-BLM}}=\frac{1}{2}\,E\star
B-\mathcal{V}(B),\qquad E\equiv\iota_{v}F,\qquad B\equiv\iota_{v}\star
F,\qquad v\equiv\frac{da}{\sqrt{(\partial a)^{2}}}.$ (7)

Here, $\mathcal{V}$ is a scalar function of the $p$-form $B$, multiplied by
the volume form. Qualitatively, one can think of $E$ as the ‘electric’ field
and $B$ as the ‘magnetic’ field. The action becomes quadratic in the gauge
field for the simple special case $\mathcal{V}=B\star B/2$ that will turn out
to give rise to a free chiral field. Other choices of $\mathcal{V}$ result in
interacting theories. It is crucial in our context to ensure that $a$ is a
pure gauge degree of freedom so that there is no unwanted dynamical scalar in
the theory. This will lead to a constraint on the potential function
$\mathcal{V}$.

Varying the Lagrangian leads to

$\displaystyle\displaystyle\delta\mathcal{L}_{\scriptscriptstyle\mathrm{PST-
BLM}}=\frac{1}{2}(E\star\delta B+B\star\delta
E)-\delta\mathcal{V}(B),\qquad\star\delta E=\delta v\wedge\star F+v\wedge\star
d\delta A,$ $\displaystyle\displaystyle\star\delta B=\delta v\wedge F+v\wedge
d\delta A,\qquad\delta v=\frac{1}{\sqrt{(\partial a)^{2}}}\iota_{v}(v\wedge
d\delta a),$

and we write the variation of $\mathcal{V}$ in the form

$\delta\mathcal{V}(B)=V(B)\star\delta B,$ (8)

which defines $V(B)$ as a function from $p$-forms to $p$-forms (it is a
derivative of $\mathcal{V}$ with respect to its $p$-form argument). Note that
$V$ should satisfy

$V=\iota_{v}(v\wedge V),$ (9)

since one has $V=\iota_{v}(v\wedge V)+v\wedge\iota_{v}V$ for any form $V$,
while the second piece could not contribute to the variation (8). Altogether,
up to total derivatives, we have

$\delta\mathcal{L}_{\scriptscriptstyle\mathrm{PST-
BLM}}=d[v\wedge(E-V(B))]\wedge\delta A-\left[\frac{1}{2}(E\wedge E+B\wedge
B)-V(B)\wedge E\right]\wedge v\wedge\delta v.$ (10)

(To bring the second term to the desired form one must remember (7), (9) and
the fact that any top form entirely constructed as a wedge product of factors
of the form $\iota_{v}(\ldots)$ identically vanishes.) From this variational
formula, one can see that besides the usual gauge shift of $A$ by an exact
$p$-form, the theory is invariant under the gauge transformations

$\delta A=da\wedge U+\frac{\varphi}{\sqrt{(\partial
a)^{2}}}(E-V(B)),\qquad\delta a=\varphi,$ (11)

where $U$ is an arbitrary $(p-1)$-form and $\varphi$ is an arbitrary scalar,
both spacetime-dependent. Validity of the $\varphi$-transformation imposes a
crucial condition on the derivatives of the function $\mathcal{V}$ in the
Lagrangian that reads

$B\wedge B=V(B)\wedge V(B).$ (12)

It is in general difficult to analyze this relation (except for dimensions 2
and 6), which constitutes a significant hurdle for PST formalism. Because of
the $\varphi$-symmetry, $a$ is a pure gauge degree of freedom that can be
shifted arbitrarily as long as $(\partial a)^{2}$ remains non-vanishing.

The equations of motion are

$d\\{v\wedge[E-V(B)]\\}=0,\qquad d\left\\{\frac{da}{(\partial
a)^{2}}\wedge[E-V(B)]\wedge[E-V(B)]\right\\}=0.$ (13)

The second equation is a consequence of the first and can be ignored, which is
an expression of $a$ being a pure gauge degree of freedom. (To prove this, as
well as the validity of the $\varphi$-transformation above, one uses
$dv/\sqrt{(\partial a)^{2}}=d(1/\sqrt{(\partial a)^{2}})\wedge v$. Derivations
within the PST formalism generally require rather sophisticated use of
exterior calculus identities. Some details of such derivations for the free
case can be found in [24].)

The first equation in (13) can be recast as $da\wedge
d[(E-V(B))/\sqrt{(\partial a)^{2}}]=0$. Such equations of the form $da\wedge
dC=0$ are generally integrated as $C=dP+da\wedge R$ where $P$ and $R$ are
arbitrary. A derivation can be found in the appendices of [21]. We thus get
$(E-V(B))/\sqrt{(\partial a)^{2}}=dP+da\wedge R$, or, acting with $da\wedge$,

$v\wedge(E-V(B))=da\wedge dP.$ (14)

We can then apply the $U$-transformation of (11), under which $\delta B=0$ and
$v\wedge\delta E=da\wedge dU$. One can thus always use this transformation to
completely eliminate the $da\wedge dP$ term, leaving $v\wedge(E-V(B))=0$. Note
that $\iota_{v}E=0$ by construction, and $\iota_{v}V(B)=0$ by (9). Then, by
the projection-rejection identity (5), $v\wedge(E-V(B))=0$ implies

$E=V(B),$ (15)

which is the final physical equation of motion of the PST theory. This
relation (nonlinear for generic choices of $\mathcal{V}$) eliminates half of
the field strength components, as expected for a chiral field.

Equation (15) has a curious structure. Since the equation of motion for $a$ is
satisfied automatically, $a$ is an arbitrary function chosen by the end-user.
Once this function has been fixed, (15) is no longer Lorentz-invariant in
general. It is, however, Lorentz invariant under simultaneous Lorentz
transformations of $A$ and $a$. With the crucial PST invariance condition (12)
respected, the $\varphi$-symmetry is operational and it allows for arbitrary
changes in $a$ and hence for compensating the Lorentz transformation of $a$,
making the theory Lorentz-invariant under transformations of $A$ alone. From
this perspective, (12) is the Lorentz invariance condition.

Since (15) is Lorentz-invariant provided that (12) holds, it should be
possible to recast it as (2), which should presumably be of the form (3) to
leave the desired number of degrees of freedom in the theory. Developing a
more explicit understanding of this transformation remains an open problem.

A significant complication for the PST formalism is that (12) is very
difficult to analyze in general to identify the admissible Lagrangian
functions $\mathcal{V}$. The trivial solution $\mathcal{V}=B\star B/2$
corresponds to free fields, where (15) turns into $E=B$, which is equivalent
to the free selfduality relation $F=\star F$. Interacting (Abelian) theories
in this formulation are only understood fully in 2 and 6 dimensions, while
(12) becomes unwieldy in higher dimensions, with only partial results
available [23].

##  3 Ashoke Sen’s approach

The second approach we review was proposed in [18, 19]. While the construction
drew inspiration from string field theory, the ultimate formulation is
elementary and self-contained. The fundamental fields in $d=4k+2$ spacetime
dimensions are a $p=2k$-form field $P$ and a selfdual $(p+1)$-form field $Q$
satisfying

$Q=\star Q.$ (16)

The Lagrangian is

$\mathcal{L}_{\,\mathrm{Sen}}=\frac{1}{2}\,dP\star dP-dP\wedge
Q+\mathcal{F}(Q),$ (17)

where $\mathcal{F}$ is a scalar function of $Q$ (multiplied by the volume form
that we do not indicate explicitly). The equations of motion are

$d[\star dP-Q]=0,\qquad dP-\star dP=\mathcal{H}(Q),$ (18)

where $\mathcal{H}$ is a function from selfdual $(p+1)$-forms to anti selfdual
$(p+1)$-forms that is a derivative of $\mathcal{F}$ with respect to its
argument:

$\delta\mathcal{F}=\mathcal{H}(Q)\wedge\delta Q.$ (19)

Note that $\mathcal{H}$ satisfies

$\star\mathcal{H}=-\mathcal{H},$ (20)

since a selfdual piece in $\mathcal{H}$ cannot contribute to the variation
(19). Integrating the first equation of motion (18), we get

$\star dP-Q=dC,$ (21)

where $C$ is arbitrary. On the other hand, applying $d$ to the second equation
of motion and subtracting it from the first one yields
$d[Q-\mathcal{H}(Q)]=0$, hence

$Q-\mathcal{H}(Q)=2\,dA$ (22)

for some $A$. Introducing $F\equiv dA$, and considering the Hodge dual of
(22), we get, in view of (16) and (20),

$Q=F+\star F,\qquad\mathcal{H}(Q)=F-\star F,$ (23)

or

$F-\star F=\mathcal{H}(F+\star F),$ (24)

which is the general equation of motion (3) for interacting chiral forms;
$\mathcal{H}$, however, is not completely arbitrary, but arises from
differentiating a scalar function as per (19). With (23), (21) becomes

$\star d(P-A)=dB,$ (25)

where we have introduced $B\equiv C+A$, and the second equation in (18)
becomes $dP-\star dP=dA-\star dA$ or

$d(P-A)=\star d(P-A).$ (26)

Then from (25-26), up to an irrelevant exact term in $P$ (a gauge shift for
$P$),

$P=A+B,\qquad dB=\star dB.$ (27)

Thus, the most general solution of the equations of motion (18) is given by
(27) and

$Q=F+\star F,\qquad F\equiv dA,\qquad F-\star F=\mathcal{H}(F+\star F).$ (28)

The two physical degrees of freedom are $A$, a self-interacting chiral form
satisfying an equation of motion structured as (3-4) with $\mathcal{H}$ being
a derivative of a scalar function as per (19), and a decoupled free chiral
form $B$ satisfying the free selfduality relation $dB=\star dB$.

Starting with a very simple Lagrangian, one thus reproduces a very large class
of equations of motion for self-interacting chiral forms. One drawback is that
the desired self-interacting chiral form necessarily comes with a dynamical
free chiral form companion, whose associated kinetic term furthermore has a
wrong sign [18, 19].

##  4 The clone field approach

Our exposition of the clone field approach follows the original presentation
in [11]. The fundamental fields in $d=4k+2$ dimensions are a $p=2k$-form gauge
field $A$ with the field strength $F\equiv dA$, its auxiliary ‘clone’ given by
a $p$-form gauge field $R$ with the field strength $Q\equiv dR$, and an
auxiliary scalar $a$ playing a role very similar to the $a$-field of the PST
formalism. The Lagrangian is

$\mathcal{L}_{\mathrm{\,clone}}=\frac{1}{2}\,H\star H+a\,F\wedge
Q+\mathcal{F}(H+\star H),\qquad H\equiv F+aQ,$ (29)

where $\mathcal{F}$ is an arbitrary scalar function of its $(p+1)$-form
argument (multiplied by the volume form). The theory is invariant under the
following gauge symmetries:

$\displaystyle\delta a$ $\displaystyle=0\,,\,\,\delta A=dU_{1}\,,\,\,\delta
R=0\,;$ (30) $\displaystyle\delta a$ $\displaystyle=0\,,\,\,\delta
A=0,\,\,\delta R=dU_{2}\,;$ (31) $\displaystyle\delta a$
$\displaystyle=0,\,\,\delta A=-\,a\,da\wedge U\,,\,\,\delta R=da\wedge U\,;$
(32) $\displaystyle\delta a$ $\displaystyle=\varphi\,,\,\,\delta
A=-\,\frac{a\,\varphi}{(\partial a)^{2}}\,\iota_{da}(Q+\star Q)\,,\,\,\delta
R=\frac{\varphi}{(\partial a)^{2}}\,\iota_{da}(Q+\star Q)\,,$ (33)

where $U_{1}$, $U_{2}$, and $U$ are arbitrary $(p-1)$-form gauge parameters
and $\varphi$ is an arbitrary scalar (all the parameters are position-
dependent). The first three transformations are verified straightforwardly
while establishing the last one is more laborious; a derivation can be found
in [11]. Note that $H$ is by itself invariant under (30-32), but not under
(33). On the other hand, $H+\star H$ is invariant under all the symmetries
(30-33), which automatically ensures the invariance of the interaction term
$\mathcal{F}(H+\star H)$. The transformation (33) makes it manifest that $a$
is a pure gauge degree of freedom that can be changed arbitrarily, though just
like in the PST theory, $(\partial a)^{2}$ must remain non-vanishing as it is
present in the denominator of (33).

The equations of motion are:

$\displaystyle d[\star H+aQ+\star X-X]=0,$ (34) $\displaystyle d[a\star
H-aF+a\star X-aX]=0,$ (35) $\displaystyle Q\wedge\star H+F\wedge
Q+Q\wedge(\star X-X)=0,$ (36)

where $X$ is a derivative of $\mathcal{F}$ defined by varying the interaction
term so that

$\delta\mathcal{F}(G)=\delta G\star X(G).$ (37)

Multiplying (34) with $a$ and subtracting it from (35) yields

$da\wedge(\star H-H+\star X-X)=0.$ (38)

The Hodge dual of this equation is $\iota_{da}(\star H-H+\star X-X)=0$. Then,
acting on this last equation with $da\wedge$, acting on (38) with
$\iota_{da}$, and adding up the results yields, in view of (5),

$\star\\!H-H+\star X-X=0.$ (39)

From this, the equation of motion for $a$ given by (36) is identically
satisfied, as it should be, since $a$ is a pure gauge degree of freedom.
Plugging (39) into (34) yields

$da\wedge dR=0,$ (40)

This is integrated in full generality as $R=dB+da\wedge C$ (a derivation can
be found in the appendices of [21]), which can be gauge-transformed to

$R=0$ (41)

by applying (31-32). Thereafter, $H=F$ and, from (39),

$\star\\!F-F+\star X-X=0,$ (42)

where $X$ should now be understood as a function of $F+\star F$.

The end result is that $a$ is a pure gauge degree of freedom that can be
chosen as any function with a non-vanishing $(\partial a)^{2}$, the ‘clone’
field $R$ has been ‘sacrificed,’ i.e., completely gauged away, and the
physical content of the theory amounts to a single chiral field $A$ satisfying
the nonlinear selfduality relation (42). This selfduality relation is of the
general form (3-4), where $\mathcal{H}\equiv\star X-X$ is once again obtained
by differentiating a scalar function as per (37).

##  5 Comparison

The key features of the three formalisms we have reviewed can be summarized as
follows:

| PST formalism | Sen’s formalism | Clone field formalism
---|---|---|---
Interaction terms given by arbitrary functions | x | ✓ | ✓
Auxiliary fields fully gauged away | ✓ | x | ✓
Physical gauge potential among fundamental fields | ✓ | x | ✓

This table gives a graphical representation of our main thesis that the clone
field formalism combines the strong sides of the PST and Sen’s formalisms
while avoiding their shortcomings.

The PST formalism is very economical in terms of its field content (only one
extra auxiliary scalar), but the derivations are relatively demanding
algebraically. The equations of motion are obtained in the compact form (15),
and if one chooses $a=t$, this form provides explicit expressions for the time
derivatives of $A$ and is thus attractive, say, for evolving the initial data.
The main difficulty is that the theory cannot be defined without solving first
the consistency condition (12) for the functional form of the nonlinear terms.
This condition appears largely intractable in dimensions higher than 6.

Sen’s formalism enjoys great algebraic simplicity and produces a self-
interacting chiral form satisfying very general equations (3-4), with the
constraint that $\mathcal{H}$ appearing on the right-hand side is obtained by
differentiating an arbitrary scalar function of the form components. A
drawback is that this degree of freedom is necessarily accompanied by another
free chiral form, which is decoupled but remains dynamical. Another issue is
that the gauge potential of the ‘physical’ self-interacting form does not
appear as an explicit variable in the theory, which may create an obstruction,
say, for using this formalism as a point of departure for non-Abelian
generalizations [9, 10].

Finally, the clone field formalism generates exactly the same equations of
motion as Sen’s formalism for its physical degrees of freedom, while all the
auxiliary fields are gauged away on-shell and do not contribute to the
dynamics, as in the PST formalism. The gauge potential of the physical gauge
field appears as an explicit variable in the theory.

Both Sen’s formalism and the clone field formalism produce the equations of
motion for their physical degrees of freedom in the simple manifestly Lorentz-
covariant form (3-4). On the other hand, the physical equations of motion of
the PST formalism are produced in the form (15), which is not manifestly
Lorentz-covariant and requires the consistency condition (12) to ensure its
underlying Lorentz-covariance. We strongly suspect that, whenever (12) is
satisfied, a covariant equation (3) equivalent to (15) should exist. Proving
this claim does not appear straightforward, though simpler analogous
connections between different forms of the equations of motion have been
established for a closely related problem in four-dimensional electromagnetism
in the first appendix of [11].

## Acknowledgments

OE is supported by Thailand NSRF via PMU-B (grants B01F650006 and B05F650021).
KM is supported by the European Union’s Horizon 2020 Research and Innovation
Programme under the Marie Skłodowska-Curie Grant No. 844265 and by the STFC
Consolidated Grant ST/T000791/1.

## Addendum: Note on the development of the clone field formalism

The clone field formalism first emerged in application to free (non-
interacting) chiral fields in [20]. It arose while exploring consistent
interactions of 2-form fields [25], and it was then realized that the
resulting Lagrangian is related to the free PST theory by a sort of Hubbard-
Stratonovich transformation. (This straightforward relation between the PST
and clone field formalisms is no longer operational when interactions are
included.) The symmetries and equations of motion of the free clone theory
were analyzed in more detail in [21], which also reported analogous
constructions for democratic Lagrangians of ordinary (non-chiral) form fields,
where electric and magnetic gauge potentials appear as explicit variables in
the theory. Interactions were introduced into these types of theories in [22],
where a democratic description was provided for general nonlinear
electrodynamics in four spacetime dimensions. Considerations of [22] suggested
a natural and simple generalization to interacting $p$-forms in any number of
dimensions. This generalization was explored in detail in [11], where the form
of clone field formalism for chiral forms featured in this essay was
originally developed. The follow-up work [26] applied these ideas to
Lagrangians of type II supergravities in ten dimensions. A supersymmetric
generalization in six dimensions was derived in [27], while connections with
the Chern-Simons approach to chiral bosons have been revealed in [28].

## References

  * [1] J. H. Schwarz, Covariant field equations of chiral $N=2$ $D=10$ supergravity, Nucl. Phys. B 226 (1983) 269.
  * [2] J. H. Schwarz and P. C. West, Symmetries and transformations of chiral $N=2$ $D=10$ supergravity, Phys. Lett. B 126 (1983) 301.
  * [3] P. S. Howe and P. C. West, The complete $N=2$, $D=10$ supergravity, Nucl. Phys. B 238 (1984) 181.
  * [4] C. G. Callan, J. A. Harvey and A. Strominger, Worldbrane actions for string solitons, Nucl. Phys. B 367 (1991) 60.
  * [5] K. Becker and M. Becker, Boundaries in M-theory, Nucl. Phys. B 472 (1996) 221, arXiv:hep-th/9602071.
  * [6] E. Witten, String theory dynamics in various dimensions, Nucl. Phys. B 443 (1995) 85, arXiv:hep-th/9503124.
  * [7] J. H. Schwarz, Lectures on superstring and M theory dualities, Nucl. Phys. B Proc. Suppl. 55 (1997) 1, arXiv:hep-th/9607201.
  * [8] J. Polchinski, TASI lectures on D-branes, arXiv:hep-th/9611050.
  * [9] X. Bekaert, M. Henneaux and A. Sevrin, Chiral forms and their deformations,
Comm. Math. Phys. 224 (2001) 683, arXiv:hep-th/0004049.

  * [10] P.-M. Ho, K.-W. Huang and Y. Matsuo, A non-Abelian self-dual gauge theory in 5+1 dimensions, JHEP 07 (2011) 021, arXiv:1104.4040 [hep-th].
  * [11] Zh. Avetisyan, O. Evnin and K. Mkrtchyan, Nonlinear (chiral) p-form electrodynamics, JHEP 08 (2022) 112, arXiv:2205.02522 [hep-th].
  * [12] M. Henneaux and C. Teitelboim, Dynamics of chiral (selfdual) p-forms,
Phys. Lett. B 206 (1988) 650.

  * [13] M. Perry and J. H. Schwarz, Interacting chiral gauge fields in six-dimensions and Born-Infeld theory, Nucl. Phys. B 489 (1997) 47, arXiv:hep-th/9611065.
  * [14] P. Pasti, D. P. Sorokin and M. Tonin, Note on manifest Lorentz and general
coordinate invariance in duality symmetric models, Phys. Lett. B 352 (1995)
59,
arXiv:hep-th/9503182.

  * [15] P. Pasti, D. P. Sorokin and M. Tonin, Duality symmetric actions with manifest space-time symmetries, Phys. Rev. D 52 (1995) 4277, arXiv:hep-th/9506109.
  * [16] P. Pasti, D. P. Sorokin and M. Tonin, On Lorentz invariant actions for chiral p-forms, Phys. Rev. D 55 (1997) 6292, arXiv:hep-th/9611100.
  * [17] I. A. Bandos, K. Lechner, A. Nurmagambetov, P. Pasti, D. P. Sorokin and M. Tonin, Covariant action for the superfive-brane of M-theory, Phys. Rev. Lett. 78 (1997) 4332, arXiv:hep-th/9701149.
  * [18] A. Sen, Covariant action for type IIB supergravity, JHEP 07 (2016) 017,
arXiv:1511.08220 [hep-th].

  * [19] A. Sen, Self-dual forms: action, Hamiltonian and compactification,
J. Phys. A 53 (2020) 084002, arXiv:1903.12196 [hep-th].

  * [20] K. Mkrtchyan, On covariant actions for chiral $p$-forms, JHEP 12 (2019) 076,
arXiv:1908.01789 [hep-th].

  * [21] S. Bansal, O. Evnin and K. Mkrtchyan, Polynomial duality-symmetric Lagrangians for free p-forms, Eur. Phys. J. C 81 (2021) 257, arXiv:2101.02350 [hep-th].
  * [22] Zh. Avetisyan, O. Evnin and K. Mkrtchyan, Democratic Lagrangians for nonlinear electrodynamics, Phys. Rev. Lett. 127 (2021) 271601, arXiv:2108.01103 [hep-th].
  * [23] G. Buratti, K. Lechner and L. Melotti, Self-interacting chiral p-forms in higher dimensions, Phys. Lett. B 798 (2019) 135018, arXiv:1909.10404 [hep-th].
  * [24] H. Isono, Note on the self-duality of gauge fields in topologically nontrivial spacetime, PTEP 2014 (2014) 093B05, arXiv:1406.6023 [hep-th].
  * [25] K. Mkrtchyan, unpublished.
  * [26] K. Mkrtchyan and F. Valach, Democratic actions for type II supergravities,
Phys. Rev. D 107 (2023) 6, arXiv:2207.00626 [hep-th].

  * [27] N. Kozyrev, Supersymmetrizing the Pasti-Sorokin-Tonin action, JHEP 03 (2023) 223, arXiv:2211.16397 [hep-th].
  * [28] A. S. Arvanitakis, L. T. Cole, O. Hulik, A. Sevrin and D. C. Thompson,
Unifying approaches to chiral bosons, arXiv:2212.11412 [hep-th].


# Algorithms of Sampling-Frequency-Independent Layers for Non-integer Strides

Kanami Imamura†, Tomohiko Nakamura†, Norihiro Takamune†, Kohei Yatabe‡, and
Hiroshi Saruwatari This work was supported by JST, ACT-X Grant Number
JPMJAX210G, Japan. T. Nakamura is currently with the National Institute of
Advanced Industrial Science and Technology (AIST). †Graduate School of
Information Science and Technology, The University of Tokyo, Tokyo, Japan
‡Department of Electrical Engineering and Computer Science,
Tokyo University of Agriculture and Technology, Tokyo, Japan

###### Abstract

In this paper, we propose algorithms for handling non-integer strides in
sampling-frequency-independent (SFI) convolutional and transposed
convolutional layers. The SFI layers have been developed for handling various
sampling frequencies (SFs) by a single neural network. They are replaceable
with their non-SFI counterparts and can be introduced into various network
architectures. However, they could not handle some specific configurations
when combined with non-SFI layers. For example, an SFI extension of Conv-
TasNet, a standard audio source separation model, cannot handle some pairs of
trained and target SFs because the strides of the SFI layers become non-
integers. This problem cannot be solved by simple rounding or signal
resampling, resulting in the significant performance degradation. To overcome
this problem, we propose algorithms for handling non-integer strides by using
windowed sinc interpolation. The proposed algorithms realize the continuous-
time representations of features using the interpolation and enable us to
sample instants with the desired stride. Experimental results on music source
separation showed that the proposed algorithms outperformed the rounding- and
signal-resampling-based methods at SFs lower than the trained SF.

###### Index Terms:

Sampling-frequency-independent convolutional layer, sinc interpolation, audio
source separation

##  I Introduction

Deep neural networks (DNNs) have been used for various audio signal processing
tasks such as music source separation [1], speech enhancement [2], and
automatic music transcription [3, 4]. Most studies on DNN-based audio signal
processing methods assume that the sampling frequency (SF) of an input signal
is the same in the training and inference stages [1, 2, 3, 4, 5, 6, 7, 8, 9,
10, 11, 12, 13, 14, 15, 16]. Hence, to handle untrained SFs, we need an
additional processing such as signal resampling.

As an alternative to signal resampling, we previously proposed sampling-
frequency-independent (SFI) convolutional layers [17]. These layers are based
on the analogy between an ordinary convolutional layer and a collection of
digital filters. A digital filter can be designed from an analog filter. Since
an analog filter is SFI, we can use a collection of analog filters (latent
analog filters) as an SFI structure. By utilizing a digital filter design
technique, we can consistently generate the parameters (weights) of the
ordinary convolutional layer from the latent analog filters for various SFs.
We experimentally found that the SFI layers work more consistently than signal
resampling for SFs much lower than a trained SF. Since SF is usually task-
specific, the SFI layers paved the way to realize an audio source separation
method that can be universally used for any downstream tasks.

The SFI layers were applied to Conv-TasNet [5], which combines DNN-based mask
predictors with a trainable analysis/synthesis filterbank. This SFI extension
is called SFI Conv-TasNet. It uses the SFI layers only for the
analysis/synthesis filterbank, leaving the mask predictors unchanged from
Conv-TasNet, which enables us to introduce recently developed mask predictors
[12, 13, 14, 15, 16]. It requires that for each target SF, the kernel size $K$
and stride $S$ of the SFI layer should be adjusted to match those used during
training in seconds. This is because the mask predictor is not SFI and its
input should have the same time resolution for any SFs.

However, for some pairs of target and trained SFs, $K$ and $S$ become non-
integers, even though a convolutional layer assumes them to be integers. For
example, when SFI Conv-TasNet is trained with $K=160$ and $S=80$ (5
$\mathrm{ms}$ and 2.5 $\mathrm{ms}$, respectively) at an SF of 32
$\mathrm{kHz}$, the corresponding $K$ and $S$ are $110.25$ and $55.125$ at an
SF of 22.05 $\mathrm{kHz}$, respectively. Although the evaluation was
performed at SFs where $K$ and $S$ became integers in [17], this problem is
unavoidable to realize the versatile preprocessor of our interest.

One simple method to handle such a non-integer is to round it to the nearest
integer. However, rounding $S$ changes the time resolution of the mask
predictor’s input and degrades the separation performance, as we will show
later in Section IV. Another method to handle non-integer strides is to use
the rounding jointly with a mask predictor based on a two-dimensional
convolutional neural network (2D-CNN). Paulus et al. experimentally showed
that rounding $K$ and $S$ did not greatly affect the performance of speech
dialogue separation when using a 2D-CNN-based mask predictor [18]. However,
this method greatly restricts the architecture of the mask predictor.
Specifically, the mask predictor must be agnostic to time and frequency axes
and must not contain any pooling or unpooling layers. To ensure flexibility in
the network architecture, we need to explore another approach.

In this paper, we propose an algorithm of the SFI convolutional layer for non-
integer $S$ by utilizing windowed sinc interpolation. This interpolation
provides the continuous-time counterpart of a discrete-time signal. Thus, by
inserting it before the decimation in the convolutional layer, we can obtain
the mask predictor’s input having the desired stride. We also extend this
algorithm for an SFI version of a transposed convolutional layer (SFI
transposed convolutional layer). We demonstrate the effectiveness of the
proposed algorithms in comparison to rounding- and signal-resampling-based
methods through music source separation experiments.

##  II Conventional Methods

###  II-A SFI Convolutional Layer

We briefly describe a one-dimensional SFI convolutional layer [17]. Let
$C^{\text{(in)}}$ and $C^{\text{(out)}}$ be the numbers of input and output
channels of this layer, respectively. This layer consists of
$C^{\text{(in)}}C^{\text{(out)}}$ analog filters and an ordinary convolutional
layer. Given a target SF, it generates discrete-time impulse responses of
length $K$ that are designed to approximate the continuous-time impulse
response or frequency response of the analog filters by using time- or
frequency-domain filter design methods, respectively. Since the focus of this
paper is how to handle non-integer $S$, we omitted the computation of the
filter design methods due to space limitations (see [17] for details). A
convolutional layer computes a cross-correlation between an input feature and
the weights. Hence, the discrete-time impulse responses are reversed in time
and are used as the weights for the convolutional layer. Replacing the
convolutional layer with the transposed convolutional layer yields the SFI
transposed convolutional layer.

###  II-B SFI Conv-TasNet

Figure 1:  Architecture of SFI Conv-TasNet. “SFI conv.” and “SFI trans. conv.”
denote SFI convolutional and transposed convolutional layers, respectively.

SFI Conv-TasNet is a Conv-TasNet extension that combines $J$ mask predictors
and an _SFI_ analysis/synthesis filterbank [17], where $J$ is the number of
sources. Figure 1 shows the architecture of SFI Conv-TasNet. The analysis
filterbank, called the encoder, consists of an SFI convolutional layer with
$C^{\text{(in)}}=1$ and $C^{\text{(out)}}=C$ followed by a rectified linear
unit (ReLU). Given an $N$-length monaural mixture $\\{x[n]\\}_{n=0}^{N-1}$,
the encoder outputs a pseudo time-frequency representation
$\\{X_{c}[m]\\}_{c=0,m=0}^{C-1,M-1}$, where $M$ is the number of frames and
$n$, $m$, and $c$ are the discrete-time, frame, and channel indices,
respectively. The pseudo time-frequency representation is fed into the mask
predictor of source $j$, which mainly consist of one-dimensional dilated
convolutional layers. See [14] for the details of the architecture of the mask
predictors. After multiplying $\\{X_{c}[m]\\}_{c,m}$ by the predicted mask, we
obtain a time-domain separated signal of each source
$\\{\hat{s}_{j}[n]\\}_{n=0}^{N-1}$ by using the synthesis filterbank, called
the decoder. The decoder is an SFI transposed convolutional layer with $C$
input channels and one output channel. The kernel size $K$ and stride $S$ of
the decoder are the same as those of the encoder.

At the inference stage, we need to adjust $K$ and $S$ in accordance with a
target SF, as described in Section I. Let $T$ be the sampling period of an
input signal. To clarify the values used in the training and inference stages,
we hereafter use $K$, $S$, and $T$ for the training stage and $K^{\prime}$,
$S^{\prime}$, and $T^{\prime}$ for the inference stage, respectively. The
kernel size and stride at $T^{\prime}$ are given as

$K^{\prime}=\dfrac{T}{T^{\prime}}K,\quad S^{\prime}=\dfrac{T}{T^{\prime}}S.$
(1)

This adjustment keeps the time resolution of $\\{X_{c}[m]\\}_{c,m}$ unchanged,
although it is valid only for integers $K^{\prime}$ and $S^{\prime}$.

###  II-C Windowed Sinc Interpolation

The windowed sinc interpolation is a popular bandlimited interpolation of a
discrete-time signal. Let $t$ be continuous-time and $g(t)$ be the real-valued
window function. For a given discrete-time signal $x[n]$ with a sampling
period of $T$, the interpolated signal $\tilde{x}(t)$ is given as

$\tilde{x}(t)=\sum_{n=-\infty}^{\infty}x[n]h(t-nT,T),$ (2)

where $h(t,T)$ is the windowed sinc function:

$h(t,T)=g(t)\text{sinc}\left(\dfrac{t}{T}\right),\quad\text{sinc}(t)=\dfrac{\sin(\pi
t)}{\pi t}.$ (3)

Now, we choose a window function with a finite support around $t=0$. More
concretely, we assume that $g(t)=0$ for $t<-LT/2$ or $t>LT/2$, where $L$ is a
positive integer. This choice can reduce the infinite sum in (2) to the finite
sum from $n=\lceil t/T-L/2\rceil$ to $n=\lfloor t/T+L/2\rfloor$, which enables
us to implement the interpolation with computers.

##  III Proposed Method

(a) Algorithm for SFI convolutional layer

(b) Algorithm for SFI transposed convolutional layer

Figure 2: Schematic illustration of proposed algorithms for SFI layers.

###  III-A Motivation and Strategy

To adjust $K^{\prime}$ and $S^{\prime}$ for a target SF, we previously used
(1), as described in Section II-B. However, this adjustment method requires
$K^{\prime}$ and $S^{\prime}$ to be integers. Thus, we need another method for
non-integers $K^{\prime}$ and $S^{\prime}$.

The simplest way is to round non-integers to the nearest integers. In a
preliminary experiment, we confirmed that rounding $K^{\prime}$ did not
significantly affect the separation performance. One reason for this is that,
similar to a time-frequency transform, $K^{\prime}$ mainly affects spectral
leakage in the frequency axis and has little impact on the time resolution of
$\\{X_{c}[m]\\}_{c,m}$. However, $S^{\prime}$ is not used in the weight
generation process and rounding $S^{\prime}$ inevitably changes the time
resolution of $\\{X_{c}[m]\\}_{c,m}$. Thus, this mismatch can affect the
separation performance, as we will show in Section IV.

To resolve this mismatch, we propose algorithms of the SFI layers for non-
integer $S^{\prime}$ by introducing the windowed sinc interpolation. Fig. 2
shows a schematic illustration of the proposed algorithms. Since the
interpolation provides a continuous-time counterpart of the discrete-time
signal, it enables us to sample instants at any SF. For simplicity, we
describe algorithms for the SFI layers of SFI Conv-TasNet in the subsequent
sections, but the following derivations can be extended for every pair of
$C^{\text{(in)}}$ and $C^{\text{(out)}}$. We also assume that $g(t)$ is an
even function and $K^{\prime}$ is rounded, which we denote by
$\check{K^{\prime}}$.

###  III-B Algorithm for SFI Convolutional Layer

In this section, we propose a sinc-interpolation-based algorithm for the SFI
convolutional layer. The convolutional layer with a stride of $S^{\prime}$ and
a padding size of $P$ first computes a cross-correlation between
$\\{x[n]\\}_{n}$ and the weights
$\\{w_{c}[k]\\}_{c=0,k=\lfloor-(\check{K^{\prime}}-1)/2\rfloor}^{C-1,\lfloor(\check{K^{\prime}}-1)/2\rfloor}$
and then decimates it with an interval of $S^{\prime}$. The cross-correlation
$y_{c}[i]$ is given by

$y_{c}[i]=\sum_{n=\lfloor-(\check{K^{\prime}}-1)/2\rfloor}^{\lfloor(\check{K^{\prime}}-1)/2\rfloor}x[i+n]w_{c}[n],$
(4)

where $i$ is the discrete-time index. The length of $y_{c}[i]$ is given as
$I=N+2P-\check{K^{\prime}}+1$. Although we can decimate $y_{c}[i]$ with a
factor of $S^{\prime}$ for an integer $S^{\prime}$, we cannot perform such
decimation directly for a non-integer $S^{\prime}$. Thus, the proposed
algorithm applies the sinc interpolation to $y_{c}[i]$, which enables us to
compute $X_{c}[m]$ for a non-integer $S^{\prime}$.

By invoking (2), we can describe the interpolation of $y_{c}[i]$:

$\tilde{y}_{c}(t)=\sum_{i=\lceil t/T^{\prime}-L/2\rceil}^{\lfloor
t/T^{\prime}+L/2\rfloor}y_{c}[i]h(t-iT^{\prime},T^{\prime}).$ (5)

By sampling instants from $\tilde{y}_{c}(t)$ with an interval of
$S^{\prime}T^{\prime}$, we can obtain the pseudo time-frequency representation
for non-integer $S^{\prime}$111When resampling a signal with a windowed sinc
interpolation, we usually avoid aliasing by changing the second argument of
$h(t,T)$. However, the decimation of a convolutional layer is not followed by
any antialiasing method. Thus, we emulate this decimation in (6).:

$X_{c}[m]=\tilde{y}_{c}(mS^{\prime}T^{\prime}).$ (6)

If $S^{\prime}$ is an integer, (6) reduces to decimating $y_{c}[i]$ by a
factor of $S^{\prime}$ because $\text{sinc}(t)=0$ for any nonzero integer $t$.

###  III-C Algorithm for SFI Transposed Convolutional Layer

Similarly to the SFI convolutional layer, we can derive a sinc-interpolation-
based algorithm for the SFI transposed convolutional layer with non-integer
$S^{\prime}$. To simplify notations, we reuse $x[n]$, $y_{c}[i]$, and
$w_{c}[k]$ for the counterparts of the transposed convolutional layer.

For an integer $S^{\prime}$, a transposed convolutional layer first pads
$S^{\prime}-1$ zeros between $X_{c}[m]$ to obtain $y_{c}[i]$. It then outputs
$x[n]$ as the cross-correlation between $y_{c}[i]$ and $w_{c}[i]$.

Now, we consider a continuous-time version of the transposed convolutional
layer to handle a non-integer $S^{\prime}$. The windowed sinc interpolation of
$w_{c}[i]$ is given as

$\tilde{w}_{c}(t)=\sum_{i=\lfloor-(\check{K^{\prime}}-1)/2\rfloor}^{\lfloor(\check{K^{\prime}}-1)/2\rfloor}w_{c}[i]h(t-iT^{\prime},T^{\prime}).$
(7)

Since $X_{c}[m]$ can be seen as a discrete-time signal with a sampling period
of $S^{\prime}T^{\prime}$, i.e.,
$\sum_{m=0}^{M-1}X_{c}[m]\delta(t-mS^{\prime}T^{\prime})$, we can write the
continuous-time version of $x[n]$ as

$\displaystyle\tilde{x}(t)$
$\displaystyle=\int_{-\infty}^{\infty}\mbox{d}\tau\sum_{m=0}^{M-1}X_{c}[m]\delta(t+\tau-
mS^{\prime}T^{\prime})\tilde{w}_{c}(\tau)$ (8)
$\displaystyle=\sum_{i=\lfloor-(\check{K^{\prime}}-1)/2\rfloor}^{\lfloor(\check{K^{\prime}}-1)/2\rfloor}\tilde{X}_{c}(t+iT^{\prime})w_{c}[i],$
(9)

where $\delta(t)$ is Dirac’s delta function and

$\tilde{X}_{c}(t)=\sum_{m=\lceil
t/(S^{\prime}T^{\prime})-L/(2S^{\prime})\rceil}^{\lfloor
t/(S^{\prime}T^{\prime})+L/(2S^{\prime})\rfloor}X_{c}[m]h(t-mS^{\prime}T^{\prime},T^{\prime}).$
(10)

By sampling $\tilde{x}(t)$ with an interval of $T^{\prime}$, we obtain $x[n]$:

$\displaystyle y_{c}[i]=$ $\displaystyle\tilde{X}_{c}(iT^{\prime}),$ (11)
$\displaystyle x[n]=$
$\displaystyle\sum_{i=\lfloor-(\check{K^{\prime}}-1)/2\rfloor}^{\lfloor(\check{K^{\prime}}-1)/2\rfloor}y_{c}[i+n]w_{c}[i].$
(12)

If $S^{\prime}$ is an integer, (11) reduces to the zero-padding of $X_{c}[m]$
because $\text{sinc}(t)=0$ for any nonzero integer $t$.

The proposed algorithms use the windowed sinc interpolation and the
interpolation accuracy is determined by $L$. Since the interpolation accuracy
would affect the separation performance, we will examine their relationship in
Section IV-B.

##  IV Experiments

###  IV-A Experimental Setup

To evaluate the effectiveness of the proposed algorithms, we conducted music
source separation experiments using the MUSDB18-HQ dataset [19]. This dataset
has 150 tracks, each of which consists of four instruments (vocals, bass,
drums, and other). We used the official data split of 86, 14, and 50 tracks
for training, validation, and test, respectively. As an evaluation measure, we
used the signal-to-distortion ratio (SDR) obtained with the BSSEval v4
toolkit[20]. To reduce the dependency on initialization, we trained models
using four random seeds and computed the averages and standard errors of the
SDRs.

We used the same training setup as in [17]. As a source separation model, we
chose SFI Conv-TasNet using the frequency-domain filter design, which achieved
the highest separation performance in [17]. The latent analog filter was a
modulated Gaussian filter. Its impulse response is given by

$f(t)=2\sqrt{2\sigma^{2}\pi}\exp\left(-\frac{\sigma^{2}t^{2}}{2}\right)\cos(\mu
t+\phi),$ (13)

where $\mu$ is the center angular frequency, $\sigma$ is the parameter
corresponding to the bandwidth of the filter, and $\phi$ is the initial phase.
These parameters were initialized as in [17] and were trained jointly with
other DNN parameters. The SFI Conv-TasNet was trained with a batch size of 12
for 250 epochs by RAdam[21] wrapped in a LookAhead optimizer [22]. The same
data augmentations used in [17] were applied. The loss function was the
negative scale-invariant source-to-noise ratio (SI-SNR) between the estimated
and groundtruth signals. Since the SDR is scale-dependent, we used the scaling
method presented in [14]:
$\\{\alpha_{i}\\}_{j=1}^{J}=\operatorname*{argmin}_{\\{\alpha_{j}\\}_{j=1}^{J}}\sum_{n=0}^{N-1}(x[n]-\sum_{j=1}^{J}\alpha_{j}\hat{s}_{j}[n])^{2}$,
where $\alpha_{j}$ is the scale for source $j$.

We compared the following four methods of handling non-integer strides.
_Rounding_ is a method to round $S^{\prime}$ to the nearest integer.
_Resampling-near_ avoids non-integer $S^{\prime}$ by resampling a mixture
signal at the nearest SF where the corresponding stride becomes an integer,
separating the resampled signal with the trained model, and resampling the
separated signals back to the original SF. _Resampling-trained_ resamples a
mixture signal at the trained SF. The signal resampling was implemented with
librosa[23] as in [17]. _Proposed_ uses the proposed algorithms for handling
non-integer $S^{\prime}$. The windowed sinc interpolation was implemented with
torchaudio[24]. For $g(t)$, we used the Kaiser window with the default
parameters of torchaudio.functional.resample. All methods used the same
trained models. Following the experimental conditions used in[17], the SFs of
the training and validation data were set to 32 $\mathrm{kHz}$ and the kernel
size and stride were set to 5 and 2.5 ms ($K=160$ and $S=80$), respectively.
The methods were evaluated at an SF of 11.025, 22.05, and 44.1 $\mathrm{kHz}$,
where $S^{\prime}$ becomes $27.5625$, $55.125$, and $110.25$, respectively. In
addition to these popular SFs, we evaluated the methods with an in-between SF,
16.538 $\mathrm{kHz}$, to show the generality of the proposed method.

###  IV-B Relationship Between Interpolation Accuracy and Separation
Performance

TABLE I: SDRs [$\mathrm{dB}$] of Proposed with varying $L$ for vocals $L$ | SF [$\mathrm{kHz}$]
---|---
11.025 | 16.538 | 22.05 | 44.1
2 | 2.3 $\pm$ 0.3 | 1.5 $\pm$ 0.3 | 2.4 $\pm$ 0.1 | 1.8 $\pm$ 0.1
4 | 4.7 $\pm$ 0.2 | 4.8 $\pm$ 0.0 | 5.4 $\pm$ 0.1 | 5.5 $\pm$ 0.1
8 | 4.6 $\pm$ 0.2 | 5.2 $\pm$ 0.1 | 5.6 $\pm$ 0.0 | 5.8 $\pm$ 0.1
16 | 4.7 $\pm$ 0.2 | 5.3 $\pm$ 0.1 | 5.6 $\pm$ 0.1 | 5.8 $\pm$ 0.1
32 | 4.7 $\pm$ 0.2 | 5.4 $\pm$ 0.1 | 5.6 $\pm$ 0.1 | 5.8 $\pm$ 0.1
64 | 4.7 $\pm$ 0.2 | 5.4 $\pm$ 0.1 | 5.6 $\pm$ 0.1 | 5.8 $\pm$ 0.1

We first examined the relationship in Proposed between separation performance
and interpolation accuracy. Table I lists the averages and standard errors of
SDRs with varying $L$ for vocals. The SDRs approximately increased as $L$
increased, showing that lowering the interpolation accuracy degrades the
separation performance. At all SFs, the SDRs were saturated for $L\geq 16$ and
this tendency was observed in the results of bass, drums, and other. Hence, we
decided to use $L=16$.

Since $L$ also involves the processing time, we measured the processing times
for a $60$-s signal using an NVIDIA GeForce RTX 3090 GPU. At an SF of $11.025$
$\mathrm{kHz}$, the processing time averaged over ten trials was 72.8 ms.
Although this is longer than Rounding (52.1 ms), it is fast enough in practice
and can be reduced by brushing up the implementation of Proposed.

###  IV-C Comparison with Existing Methods

(a) vocals

(b) bass

(c) drums

(d) other

Figure 3:  SDRs of Rounding, Resampling-near, Resampling-trained, and
Proposed. Error bars represent standard errors and red dotted line denotes the
trained SF. For reference, gray dashed line shows SDRs at SFs where
$K^{\prime}$ and $S^{\prime}$ are integers.

Fig. 3 shows the averages and standard errors of SDRs for all methods per
instrument. As a reference, we also show SDRs at SFs where $K^{\prime}$ and
$S^{\prime}$ are integers (gray dashed curve), which we call _Reference_. SDRs
of Resampling-near significantly decreased compared with Reference. Although
the other methods had SDRs consistent with Reference at an SF of 44.1
$\mathrm{kHz}$, Rounding and Resampling-trained quickly decreased as the SF
decreased. At the lower SFs, SDRs of Rounding were around one decibel below
Reference for vocals, bass, and other. One reason for the performance drop is
that the change in time resolution caused by Rounding increases as the SF
decreases. This result shows that rounding $S^{\prime}$ degrades the
separation performance at lower SFs. Resampling-trained had higher SDRs than
Rounding at an SF of 22.05 $\mathrm{kHz}$. However, its SDRs dropped
significantly as the SF decreases, which is consistent with the results
observed in [17]. By contrast, Proposed provided more consistent SDRs with
Reference at all SFs for all instruments. These results clearly demonstrate
the effectiveness of the proposed algorithms.

The SDR gaps between Proposed and Rounding for vocals, bass, and other were
greater than those for drums. The change in time resolution caused by rounding
$S^{\prime}$ corresponds to that in the SF of an input signal for the mask
predictors. Thus, it should strongly affect the separation performance for the
audio signals containing pitched sounds. The drums are unpitched instruments
and the change in time resolution can be less significant compared with
pitched sounds. This observation leads to a conjecture that slightly changing
the time resolution of an input is an adversarial attack for the separation of
pitched sounds. Its investigation is beyond the scope of this paper and we
leave it as future work.

##  V Conclusion

We proposed algorithms of the SFI layers for non-integer strides. The proposed
algorithms use the windowed sinc interpolation to bridge two signals of
different SFs. For the SFI convolutional layer, the interpolation is applied
to the cross-correlation between an input and the weights, which enables us to
sample instants with a desired SF. For the SFI transposed convolutional layer,
the interpolation is applied to an input. By using these algorithms, we can
handle non-integer strides in SFI Conv-TasNet. Experimental results showed
that the proposed method outperforms the rounding- and signal-resampling-based
methods at lower SFs.

## References

  * [1] Y. Mitsufuji, G. Fabbro, S. Uhlich, F.-R. Stöter, A. Défossez, M. Kim, W. Choi, C.-Y. Yu, and K.-W. Cheuk, “Music demixing challenge 2021,” _Frontiers_ , vol. 1, Article 808395, pp. 1–14, 2022.
  * [2] H. Dubey, V. Gopal, R. Cutler, A. Aazami, S. Matusevych, S. Braun, S. E. Eskimez, M. Thakker, T. Yoshioka, H. Gamper, and R. Aichner, “ICASSP 2022 deep noise suppression challenge,” in _Proc. IEEE Int. Conf. Acoust., Speech Signal Process._ , 2022, pp. 9271–9275.
  * [3] S. Sigtia, E. Benetos, and S. Dixon, “An end-to-end neural network for polyphonic piano music transcription,” _IEEE/ACM Trans. Audio, Speech Lang. Process._ , vol. 24, no. 5, pp. 927–939, 2016.
  * [4] F. Pedersoli, G. Tzanetakis, and K. M. Yi, “Improving music transcription by pre-stacking a U-Net,” in _Proc. IEEE Int. Conf. Acoust., Speech Signal Process._ , 2020, pp. 506–510.
  * [5] Y. Luo and N. Mesgarani, “Conv-TasNet: Surpassing ideal time–frequency magnitude masking for speech separation,” _IEEE/ACM Trans. Audio, Speech Lang. Process._ , vol. 27, no. 8, pp. 1256–1266, 2019.
  * [6] F.-R. Stöter, S. Uhlich, A. Liutkus, and Y. Mitsufuji, “Open-Unmix - a reference implementation for music source separation,” _J. Open Source Softw._ , 2019.
  * [7] R. Hennequin, A. Khlif, F. Voituret, and M. Moussallam, “Spleeter: a fast and efficient music source separation tool with pre-trained models,” _J. Open Source Softw._ , vol. 5, no. 50, p. 2154, 2020.
  * [8] N. Takahashi and Y. Mitsufuji, “Densely connected multi-dilated convolutional networks for dense prediction tasks,” in _Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit._ , 2021, pp. 993–1002.
  * [9] T. Nakamura, S. Kozuka, and H. Saruwatari, “Time-domain audio source separation with neural networks based on multiresolution analysis,” _IEEE/ACM Trans. Audio, Speech Lang. Process._ , vol. 29, pp. 1687–1701, 2021\.
  * [10] A. Défossez, “Hybrid spectrogram and waveform source separation,” in _Proc. Int. Soc. Music Inf. Retrieval Conf._ , 2021.
  * [11] Y. Luo and J. Yu, “Music source separation with band-split RNN,” _arXiv:2209.15174_ , 2022.
  * [12] I. Kavalerov, S. Wisdom, H. Erdogan, B. Patton, K. Wilson, J. Le Roux, and J. R. Hershey, “Universal sound separation,” in _Proc. IEEE Workshop Appl. Signal Process. Audio Acoust._ , 2019, pp. 175–179.
  * [13] Y. Luo, Z. Chen, and T. Yoshioka, “Dual-Path RNN: Efficient long sequence modeling for time-domain single-channel speech separation,” in _Proc. IEEE Int. Conf. Acoust., Speech Signal Process._ , 2020, pp. 46–50.
  * [14] D. Samuel, A. Ganeshan, and J. Naradowsky, “Meta-learning extractors for music source separation,” in _Proc. IEEE Int. Conf. Acoust., Speech Signal Process._ , 2020, pp. 816–820.
  * [15] N. Zeghidour and D. Grangier, “Wavesplit: End-to-end speech separation by speaker clustering,” _IEEE/ACM Trans. Audio, Speech Lang. Process._ , vol. 29, pp. 2840–2849, 2021.
  * [16] Y. Koizumi, S. Karita, S. Wisdom, H. Erdogan, J. R. Hershey, L. Jones, and M. Bacchiani, “DF-Conformer: Integrated architecture of Conv-TasNet and conformer using linear complexity self-attention for speech enhancement,” in _Proc. IEEE Workshop Appl. Signal Process. Audio Acoust._ , 2021, pp. 161–165.
  * [17] K. Saito, T. Nakamura, K. Yatabe, and H. Saruwatari, “Sampling-frequency-independent convolutional layer and its application to audio source separation,” _IEEE/ACM Trans. Audio, Speech Lang. Process._ , vol. 30, pp. 2928–2943, 2022.
  * [18] J. Paulus and M. Torcoli, “Sampling frequency independent dialogue separation,” in _Proc. Eur. Signal Process. Conf._ , 2022, pp. 160–164.
  * [19] Z. Rafii, A. Liutkus, F.-R. Stöter, S. I. Mimilakis, and R. Bittner, “MUSDB18-HQ - an uncompressed version of MUSDB18,” 2019.
  * [20] F.-R. Stöter, A. Liutkus, and N. Ito, “The 2018 signal separation evaluation campaign,” in _Proc. Int. Conf. Latent Variable Anal. Signal Separation_ , 2018, pp. 293–305.
  * [21] L. Liu, H. Jiang, P. He, W. Chen, X. Liu, J. Gao, and J. Han, “On the variance of the adaptive learning rate and beyond,” in _Proc. Int. Conf. Learn. Representations_ , 2020.
  * [22] M. Zhang, J. Lucas, J. Ba, and G. E. Hinton, “Lookahead optimizer: k steps forward, 1 step back,” in _Proc. Adv. Neural Inf. Process. Syst._ , vol. 32, 2019, pp. 9597–9608.
  * [23] B. McFee, C. Raffel, D. Liang, D. P. Ellis, M. McVicar, E. Battenberg, and O. Nieto, “librosa: Audio and music signal analysis in python,” in _Proc. Python Sci. Conf._ , vol. 8, 2015, pp. 18–25.
  * [24] Y.-Y. Yang, M. Hira, Z. Ni, A. Astafurov, C. Chen, C. Puhrsch, D. Pollack, D. Genzel, D. Greenberg, E. Z. Yang, J. Lian, J. Hwang, J. Chen, P. Goldsborough, S. Narenthiran, S. Watanabe, S. Chintala, and V. Quenneville-Bélair, “TorchAudio: Building blocks for audio and speech processing,” in _Proc. IEEE Int. Conf. Acoust., Speech Signal Process._ , 2022, pp. 6982–6986.