# Some Analytical Properties of the Hyperbolic Sine Integral

Kwara Nantomah  Department of Mathematics, School of Mathematical Sciences, C.
K. Tedam University of Technology and Applied Sciences, P. O. Box 24,
Navrongo, Upper-East Region, Ghana<EMAIL_ADDRESS>
###### Abstract.

By using some tools of analysis, we establish some analytical properties such
as monotonicity and inequalities involving the hyperbolic sine integral
function. As applications of some of the established properties, we obtain
some rational bounds for the hyperbolic tangent function.

###### Key words and phrases:

Hyperbolic sine integral function, hyperbolic functions, hyperbolic sinc
function, bounds, inequalities

###### 2010 Mathematics Subject Classification:

33B10, 33Bxx, 26D05

##  1\. Introduction

The cardinal hyperbolic sine function which is also known as sinhc function or
hyperbolic sinc function is defined for $z\in(-\infty,\infty)$ as [15]

$\mathrm{sinhc}(z)=\begin{cases}\frac{\sinh(z)}{z},&z\neq 0\\\
1,&z=0.\end{cases}$ (1)

It has been very useful in various areas of mathematics, physics and
engineering. For example, it has been demonstrated that the function exhibits
a clear geometric interpretation as the ratio between length and chord of a
symmetric catenary segment [9], [15]. Due to its usefulness, it has been
investigated by several researchers and many remarkable inequalities have been
established concerning the function. For further information and recent
developments on such inequalities, one may consult the works [3], [4], [5],
[6], [7], [8], [10], [16], [17] and the references therein.

Closely related to the sinch function is the hyperbolic sine integral function
which is defined for $z\in(-\infty,\infty)$ as [1, p. 231]

$\mathrm{Shi}(z)=\int_{0}^{z}\frac{\sinh(t)}{t}\,dt.$ (2)

In [12], the author considered three representations in terms of the
hypergeometric function ${}_{2}F_{3}$ for a certain indefinite hyperbolic sine
integral. A review of the literature reveals that, unlike the cardinal
hyperbolic sine function which is well researched in terms of its inequalities
or bounds, the hyperbolic sine integral is yet to receive a similar attention.

The purpose of this paper is to trigger the process for such investigations
and attention. Precisely, we establish some analytical properties such as
monotonicity and inequalities involving the hyperbolic sine integral function.
As applications of some of the established properties, we obtain some rational
bounds for the hyperbolic tangent function. We present our findings in the
subsequent sections.

##  2\. Some Properties of the Hyperbolic Sine Integral

The hyperbolic sine integral function may also be defined for
$z\in(-\infty,\infty)$ by the following equivalent forms.

$\displaystyle\mathrm{Shi}(z)$
$\displaystyle=\int_{0}^{1}\frac{\sinh(zt)}{t}\,dt,$ (3)
$\displaystyle=\sum_{r=0}^{\infty}\frac{z^{2r+1}}{(2r+1)(2r+1)!}.$ (4)

By change of variable, representation (3) is obtained from (2) and
representation (4) is obtained from either (2) or (3) by using the series
representation of $\frac{\sinh(z)}{z}$.

By utilizing representation (3), the derivatives of $\mathrm{Shi}(z)$ are
obtained as follows.

$\mathrm{Shi}^{(k)}(z)=\int_{0}^{1}t^{k-1}\sinh(zt)dt,\quad
k\in\\{2m:m\in\mathbb{N}_{0}\\},$ (5)
$\mathrm{Shi}^{(k)}(z)=\int_{0}^{1}t^{k-1}\cosh(zt)dt,\quad
k\in\\{2m+1:m\in\mathbb{N}_{0}\\},$ (6)

where $\mathbb{N}_{0}=\\{0,1,2,3,...\\}$. In particular, the first and second
derivatives are

$\mathrm{Shi}^{\prime}(z)=\int_{0}^{1}\cosh(zt)dt=\frac{\sinh(z)}{z},$ (7)
$\mathrm{Shi}^{\prime\prime}(z)=\int_{0}^{1}t\sinh(zt)dt=\frac{\cosh(z)}{z}-\frac{\sinh(z)}{z^{2}}.$
(8)

######  Remark 2.1.

Identity (8) implies that

$\cosh(z)>\frac{\sinh(z)}{z}$ (9)

for $z>0$ and this is well known in the literature.

######  Lemma 2.2.

If a function $\frac{p(x)}{x}$ is increasing or decreasing on an interval $I$,
then $p(x)$ supperadditive or subadditive on $I$ respectively.

###### Proof.

See Lemma 3.2 of [14] or Theorem 3.1 of [2]. ∎

######  Theorem 2.3.

The function $\mathrm{Shi}(z)$ is supperadditive on $(0,\infty)$. That is, the
inequality

$\mathrm{Shi}(u+v)>\mathrm{Shi}(u)+\mathrm{Shi}(v)$ (10)

holds for $u>0$ and $v>0$.

###### First Proof.

Let $A(z)=\frac{\mathrm{Shi}(z)}{z}$ for $z>0$. Then

$\displaystyle z^{2}A^{\prime}(z)$
$\displaystyle=z\mathrm{Shi}^{\prime}(z)-\mathrm{Shi}(z)$
$\displaystyle=\sum_{r=0}^{\infty}\frac{z^{2r+1}}{(2r+1)!}-\sum_{r=0}^{\infty}\frac{z^{2r+1}}{(2r+1)(2r+1)!}$
$\displaystyle=\sum_{r=0}^{\infty}\left[1-\frac{1}{2r+1}\right]\frac{z^{2r+1}}{(2r+1)!}$
$\displaystyle>0.$

Hence $A(z)$ is increasing and the conclusion follows from Lemma 2.2. ∎

###### Second Proof.

Let $u>0$ and $v>0$. Then

$\displaystyle\mathrm{Shi}(u+v)$
$\displaystyle=\int_{0}^{1}\frac{\sinh(ut+vt)}{t}\,dt$
$\displaystyle=\int_{0}^{1}\frac{\sinh(ut)\cosh(vt)}{t}\,dt+\int_{0}^{1}\frac{\cosh(ut)\sinh(vt)}{t}\,dt$
$\displaystyle>\int_{0}^{1}\frac{\sinh(ut)}{t}\,dt+\int_{0}^{1}\frac{\sinh(vt)}{t}\,dt$
$\displaystyle=\mathrm{Shi}(u)+\mathrm{Shi}(v)$

since $\cosh(z)>1$ for all $z\neq 0$. ∎

###### Third Proof.

Let $\phi(u,v)=\mathrm{Shi}(u+v)-\mathrm{Shi}(u)-\mathrm{Shi}(v)$ for $u>0$
and $v>0$. Without loss of generality, let $v$ be fixed. Then

$\displaystyle\frac{\partial}{\partial u}\phi(u,v)$
$\displaystyle=\mathrm{Shi}^{\prime}(u+v)-\mathrm{Shi}^{\prime}(u)$
$\displaystyle=\int_{0}^{1}\cosh(ut+vt)dt-\int_{0}^{1}\cosh(ut)dt$
$\displaystyle=\int_{0}^{1}\left[\cosh(ut)\cosh(vt)+\sinh(ut)\sinh(vt)\right]\,dt-\int_{0}^{1}\cosh(ut)\,dt$
$\displaystyle=\int_{0}^{1}\cosh(ut)[\cosh(vt)-1]dt+\int_{0}^{1}\sinh(ut)\sinh(vt)dt$
$\displaystyle>0$

since $\cosh(z)>1$ for all $z\neq 0$. Thus, $\phi(u,v)$ is increasing and so

$\phi(u,v)>\lim_{u\to 0}\phi(u,v)=0$

which gives the desired result. ∎

######  Theorem 2.4.

The inequality

$\mathrm{Shi}(u)+\mathrm{Shi}(v)>u+v$ (11)

holds for $u>0$ and $v>0$, and the inequality

$\frac{\mathrm{Shi}(u)}{\mathrm{Shi}(v)}\leq\frac{u}{v}$ (12)

holds for $0<u\leq v$.

###### Proof.

The monotonicity property of the function $\frac{\mathrm{Shi}(z)}{z}$ implies
that, for $z>0$, we have

$\frac{\mathrm{Shi}(z)}{z}>\lim_{z\to 0^{+}}\frac{\mathrm{Shi}(z)}{z}=1.$

That is,

$\mathrm{Shi}(z)>z.$

Hence for $u>0$ and $v>0$, we have $\mathrm{Shi}(u)>u$ and $\mathrm{Shi}(v)>v$
which results to (11). Likewise, for $0<u\leq v$, we have

$\frac{\mathrm{Shi}(u)}{u}\leq\frac{\mathrm{Shi}(v)}{v}$

which results to (12). ∎

######  Theorem 2.5.

Let $z>0$ and $\lambda\in(0,1)$. Then the inequality

$\mathrm{Shi}(\lambda z)>\lambda\mathrm{Shi}(z)$ (13)

holds. If $\lambda>1$, then the inequality is reversed.

###### Proof.

Let $\alpha(z)=\mathrm{Shi}(\lambda z)-\lambda\mathrm{Shi}(z)$ for $z>0$ and
$\lambda\in(0,1)$. Then

$\displaystyle\alpha^{\prime}(z)$
$\displaystyle=\lambda\left[\mathrm{Shi}^{\prime}(\lambda
z)-\mathrm{Shi}^{\prime}(z)\right]<0$

since $\mathrm{Shi}^{\prime}(z)$ is increasing for $z>0$. Hence $\alpha(z)$ is
decreasing and then, we have

$\alpha(z)>\lim_{z\to 0^{+}}\alpha(z)=0$

which gives (13). ∎

######  Theorem 2.6.

For $z>0$, the inequality

$\mathrm{Shi}(z)+\mathrm{Shi}(1/z)\geq
2\int_{0}^{1}\frac{\sinh(t)}{t}dt\approx 2.11450$ (14)

holds. Equality is attained if $z=1$.

###### Proof.

The case for $z=1$ is easily seen. Because of this, let
$P(z)=\mathrm{Shi}(z)+\mathrm{Shi}(1/z)$ for $z\in(0,1)\cup(1,\infty)$. Then

$P^{\prime}(z)=\mathrm{Shi}^{\prime}(z)-\frac{1}{z^{2}}\mathrm{Shi}^{\prime}(1/z),$

which means that

$\displaystyle zP^{\prime}(z)$ $\displaystyle=\sinh(z)-\sinh(1/z):=E(z)$

Since $\sinh(z)$ is increasing, then $E(z)<0$ if $z\in(0,1)$ and $E(z)>0$ if
$z\in(1,\infty)$. Thus, $P(z)$ is decreasing on $(0,1)$ and increasing on
$(1,\infty)$. Therefore, on both intervals, we have

$P(z)>\lim_{z\to
1}P(z)=2\mathrm{Shi}(1)=2\int_{0}^{1}\frac{\sinh(t)}{t}dt\approx 2.11450$

completing the proof. ∎

######  Lemma 2.7 ([13]).

Let $-\infty\leq u<v\leq\infty$ and $p$ and $q$ be continuous functions that
are differentiable on $(u,v)$, with $p(u+)=q(u+)=0$ or $p(v-)=q(v-)=0$.
Suppose that $q(z)$ and $q^{\prime}(z)$ are nonzero for all $z\in(u,v)$. If
$\frac{p^{\prime}(z)}{q^{\prime}(z)}$ is increasing (or decreasing) on
$(u,v)$, then $\frac{p(x)}{q(x)}$ is also increasing (or decreasing) on
$(u,v)$.

In the literature, Lemma 2.7 is referred to as l’Hospital rule for monotonicy.
It has become a remarkable tool in proving various results in mathematical
analysis.

######  Lemma 2.8.

For $z>0$, the function $T(z)=\frac{\sinh(z)}{\mathrm{Shi}(z)}$ is increasing.

###### Proof.

For $z\in(0,\infty)$, we have

$T(z)=\frac{\sinh(z)}{\mathrm{Shi}(z)}=\frac{p_{1}(z)}{q_{1}(z)},$

where $p_{1}(z)=\sinh(z)$, $q_{1}(z)=\mathrm{Shi}(z)$ and
$p_{1}(0)=q_{1}(0)=0$. Then

$\frac{p^{\prime}_{1}(z)}{q^{\prime}_{1}(z)}=\frac{z\cosh(z)}{\sinh(z)}=\frac{p_{2}(z)}{q_{2}(z)}$

where $p_{2}(z)=z\cosh(z)$, $q_{2}(z)=\sinh(z)$ and $p_{2}(0)=q_{2}(0)=0$.
Then

$\displaystyle\sinh^{2}(z)\left(\frac{p_{2}(z)}{q_{2}(z)}\right)^{\prime}$
$\displaystyle=\left[\cosh(z)+z\sinh(z)\right]\sinh(z)-z\cosh^{2}(z)$
$\displaystyle=\cosh(z)\sinh(z)+z\left[\sinh^{2}(z)-\cosh^{2}(z)\right]$
$\displaystyle=\cosh(z)\sinh(z)-z$ $\displaystyle>0$

since $\cosh(z)>1$ and $\sinh(z)>z$ for $z>0$. Thus,
$\frac{p_{1}^{\prime}(z)}{q_{1}^{\prime}(z)}$ is increasing. Hence by Lemma
2.7, the function $\frac{p_{1}(z)}{q_{1}(z)}$ is also increasing. This
completes the proof. ∎

######  Theorem 2.9.

For $z>0$, the inequality

$\mathrm{Shi}(z)\mathrm{Shi}(1/z)\geq\left(\int_{0}^{1}\frac{\sinh(t)}{t}dt\right)^{2}\approx
1.11778$ (15)

holds. Equality is attained if $z=1$.

###### Proof.

The case for $z=1$ is easily seen. And so, let
$Q(z)=\mathrm{Shi}(z)\mathrm{Shi}(1/z)$ and $\theta(z)=\ln Q(z)$ for
$z\in(0,1)\cup(1,\infty)$. Then

$\displaystyle z\theta^{\prime}(z)$
$\displaystyle=z\frac{\mathrm{Shi}^{\prime}(z)}{\mathrm{Shi}(z)}-\frac{1}{z^{2}}\frac{\mathrm{Shi}^{\prime}(1/z)}{\mathrm{Shi}(1/z)}$
$\displaystyle=\frac{\sinh(z)}{\mathrm{Shi}(z)}-\frac{\sinh(1/z)}{\mathrm{Shi}(1/z)}$
$\displaystyle:=H(z).$

Because of Lemma 2.8, then $H(z)<0$ if $z\in(0,1)$ and $H(z)>0$ if
$z\in(1,\infty)$. Subsequently, $Q(z)$ is decreasing on $(0,1)$ and increasing
on $(1,\infty)$. Therefore, on both intervals, we have

$Q(z)>\lim_{z\to
1}Q(z)=(\mathrm{Shi}(1))^{2}=\left(\int_{0}^{1}\frac{\sinh(t)}{t}dt\right)^{2}\approx
1.11778$

completing the proof. ∎

######  Lemma 2.10.

For $z>0$, the function $V(z)=\mathrm{Shi}(z)-\sinh(z)$ is decreasing and the
inequality

$\mathrm{Shi}(z)-\sinh(z)<0$ (16)

holds.

###### Proof.

We have

$\displaystyle V^{\prime}(z)$
$\displaystyle=\mathrm{Shi}^{\prime}(z)-\cosh(z)$
$\displaystyle=\frac{\sinh(z)}{z}-\cosh(z)<0$

as a result of (9). Hence

$V(z)<\lim_{z\to 0}V(z)=0$

which proves (16). ∎

######  Lemma 2.11.

For $z>0$, the function $K(z)=\frac{\sinh(z)}{\mathrm{Shi}^{2}(z)}$ is
decreasing.

###### Proof.

For $z\in(0,\infty)$, we have

$K(z)=\frac{\sinh(z)}{\mathrm{Shi}^{2}(z)}=\frac{p_{1}(z)}{q_{1}(z)},$

where $p_{1}(z)=\sinh(z)$, $q_{1}(z)=\mathrm{Shi}^{2}(z)$ and
$p_{1}(0)=q_{1}(0)=0$. Then

$\frac{p^{\prime}_{1}(z)}{q^{\prime}_{1}(z)}=\frac{z\cosh(z)}{2\mathrm{Shi}(z)\sinh(z)}=\frac{p_{2}(z)}{q_{2}(z)}$

where $p_{2}(z)=z\cosh(z)$, $q_{2}(z)=2\mathrm{Shi}(z)\sinh(z)$ and
$p_{2}(0)=q_{2}(0)=0$. Then

$\displaystyle
2\mathrm{Shi}^{2}(z)\left(\frac{p_{2}(z)}{q_{2}(z)}\right)^{\prime}$
$\displaystyle=\mathrm{Shi}(z)\coth(z)-z\mathrm{Shi}(z)\mathrm{cosech}^{2}(z)-\cosh(z)$
$\displaystyle=\cosh(z)\left[\frac{\mathrm{Shi}(z)}{\sinh(z)}-1\right]-z\mathrm{Shi}(z)\mathrm{cosech}^{2}(z)$
$\displaystyle<0$

as a result of (16). Thus, $\frac{p_{1}^{\prime}(z)}{q_{1}^{\prime}(z)}$ is
decreasing. Hence by Lemma 2.7, the function $\frac{p_{1}(z)}{q_{1}(z)}$ is
also decreasing. This completes the proof. ∎

######  Remark 2.12.

The increasing property of the function $\frac{\sinh(z)}{\mathrm{Shi}(z)}$ is
equivalent to

$z\mathrm{Shi}(z)\cosh(z)-\sinh^{2}(z)>0.$ (17)

Also, the decreasing property of the function
$\frac{\sinh(z)}{\mathrm{Shi}^{2}(z)}$ is equivalent to

$z\mathrm{Shi}(z)\cosh(z)-2\sinh^{2}(z)<0.$ (18)

Combining (17) and (18) yields

$\sinh^{2}(z)<z\mathrm{Shi}(z)\cosh(z)<2\sinh^{2}(z)$ (19)

which is also equivalent to

$\frac{\tanh(z)}{z}<\frac{\mathrm{Shi}(z)}{\sinh(z)}<2\frac{\tanh(z)}{z}.$
(20)

######  Theorem 2.13.

For $z>0$, the inequality

$\frac{z}{2}+\frac{\cosh(z)-1}{z}<\mathrm{Shi}(z)<2\left(\frac{\cosh(z)-1}{z}\right)$
(21)

holds.

###### Proof.

Recall that $t<\mathrm{Shi}(t)<\sinh(t)$ for $t>0$. Then, integrating over the
interval $(0,z)$, we have

$\int_{0}^{z}tdt<\int_{0}^{z}\mathrm{Shi}(t)dt<\int_{0}^{z}\sinh(t)dt$

which gives

$\frac{z^{2}}{2}<z\mathrm{Shi}(z)-\cosh(z)+1<\cosh(z)-1$

and this simplifies to (21). ∎

######  Theorem 2.14.

For $z>0$, the inequality

$\frac{2\mathrm{Shi}(z)\mathrm{Shi}(1/z)}{\mathrm{Shi}(z)+\mathrm{Shi}(1/z)}\leq\int_{0}^{1}\frac{\sinh(t)}{t}dt\approx
1.05725$ (22)

holds. Equality is attained if $z=1$.

###### Proof.

The case for $z=1$ is easily seen. On that note, let
$\Psi(z)=\frac{2\mathrm{Shi}(z)\mathrm{Shi}(1/z)}{\mathrm{Shi}(z)+\mathrm{Shi}(1/z)}$
and $h(z)=\ln\Psi(z)$ for $z\in(0,1)\cup(1,\infty)$. Then

$h^{\prime}(z)=\frac{\mathrm{Shi}^{\prime}(z)}{\mathrm{Shi}(z)}-\frac{1}{z^{2}}\frac{\mathrm{Shi}^{\prime}(1/z)}{\mathrm{Shi}(1/z)}-\frac{\mathrm{Shi}^{\prime}(z)-\frac{1}{z^{2}}\mathrm{Shi}^{\prime}(1/z)}{\mathrm{Shi}(z)+\mathrm{Shi}(1/z)}$

which implies that

$z\left[\mathrm{Shi}(z)+\mathrm{Shi}(1/z)\right]h^{\prime}(z)=z\frac{\mathrm{Shi}^{\prime}(z)}{\mathrm{Shi}(z)}\mathrm{Shi}(1/z)-\frac{1}{z}\frac{\mathrm{Shi}^{\prime}(1/z)}{\mathrm{Shi}(1/z)}\mathrm{Shi}(z).$

This further gives rise to

$\displaystyle
z\left[\frac{1}{\mathrm{Shi}(z)}+\frac{1}{\mathrm{Shi}(1/z)}\right]h^{\prime}(z)$
$\displaystyle=z\frac{\mathrm{Shi}^{\prime}(z)}{\mathrm{Shi}^{2}(z)}-\frac{1}{z}\frac{\mathrm{Shi}^{\prime}(1/z)}{\mathrm{Shi}^{2}(1/z)}$
$\displaystyle=\frac{\sinh(z)}{\mathrm{Shi}^{2}(z)}-\frac{\sinh(1/z)}{\mathrm{Shi}^{2}(1/z)}$
$\displaystyle=D(z).$

Owing to Lemma 2.11, we have $D(z)>0$ if $z\in(0,1)$ and $D(z)<0$ if
$z\in(1,\infty)$. Thus, $h(z)$ is increasing on $(0,1)$ and decreasing on
$(1,\infty)$. Accordingly, $\Psi(z)$ is increasing on $(0,1)$ and decreasing
on $(1,\infty)$. Therefore, on both intervals, we have

$\Psi(z)<\lim_{z\to
1}\Psi(z)=\mathrm{Shi}(1)=\int_{0}^{1}\frac{\sinh(t)}{t}dt\approx 1.05725$

completing the proof. ∎

######  Remark 2.15.

Theorem 2.14 can be interpreted to mean that, for $z>0$, the harmonic mean of
$\mathrm{Shi}(z)$ and $\mathrm{Shi}(1/z)$ can never be greater than the
quantity $\mathrm{Shi}(1)$. Inequality (22) can also be rearranged as

$\frac{1}{2}\left[\frac{1}{\mathrm{Shi}(z)}+\frac{1}{\mathrm{Shi}(1/z)}\right]\geq\left(\int_{0}^{1}\frac{\sinh(t)}{t}dt\right)^{-1}.$
(23)

######  Lemma 2.16 ([11]).

Let the function $\alpha:I\subseteq(0,\infty)\rightarrow(0,\infty)$ be
differentiable. Then $\alpha(z)$ is is geometrically convex (concave) if and
only if $\frac{z\alpha^{\prime}(z)}{\alpha(z)}$ is increasing (decreasing)
respectively.

######  Theorem 2.17.

The function $\mathrm{Shi}(z)$ is geometrically convex on $(0,\infty)$. That
is, the inequality

$\mathrm{Shi}(u^{k}v^{1-k})\leq\left(\mathrm{Shi}(u)\right)^{k}\left(\mathrm{Shi}(v)\right)^{1-k}$
(24)

holds for $u>0$, $v>0$ and $k\in[0,1]$.

###### Proof.

Applying Lemma 2.8, we have

$\frac{d}{dz}\left(\frac{z\mathrm{Shi}^{\prime}(z)}{\mathrm{Shi}(z)}\right)=\frac{d}{dz}\left(\frac{\sinh(z)}{\mathrm{Shi}(z)}\right)>0$

and by Lemma 2.16, we conclude that $\mathrm{Shi}(z)$ is geometrically convex.
This is equivalent to (24). ∎

######  Remark 2.18.

It is interesting to note that, by letting $u=z$, $v=1/z$ and $k=\frac{1}{2}$
in (24), we recover the inequality (15).

##  3\. Rational Bounds for the Hyperbolic Tangent Function

In this section, as applications of the hyperbolic sine integral, we obtain
some rational bounds for the hyperbolic tangent function.

######  Theorem 3.1.

For $z>0$, the inequalities

$\frac{2z}{z^{2}+2}<\tanh(z)<\frac{z^{3}+6z}{3z^{2}+6}$ (25)

hold.

###### Proof.

By direct computations, we obtain

$\displaystyle\mathrm{Shi}^{(3)}(z)$
$\displaystyle=\int_{0}^{1}t^{2}\cosh(zt)dt$
$\displaystyle=\frac{(z^{2}+2)\sinh(z)-2z\cosh(z)}{z^{3}}>0.$

Upon rearrangement, we obtain

$\tanh(z)>\frac{2z}{z^{2}+2}$

which gives the left hand side of (25). Also,

$\displaystyle\mathrm{Shi}^{(4)}(z)$
$\displaystyle=\int_{0}^{1}t^{3}\sinh(zt)dt$
$\displaystyle=\frac{(z^{3}+6z)\cosh(z)-(3z^{2}+6)\sinh(z)}{z^{4}}>0.$

Hence

$\tanh(z)<\frac{z^{3}+6z}{3z^{2}+6}$

which gives the right hand side of (25). This completes the proof. ∎

######  Theorem 3.2.

For $z>0$, the inequalities

$\frac{4z^{3}+24z}{z^{4}+12z^{2}+24}<\tanh(z)<\frac{z^{5}+20z^{3}+120z}{5z^{4}+60z^{2}+120}$
(26)

hold.

###### Proof.

By direct computations, we obtain

$\displaystyle\mathrm{Shi}^{(5)}(z)$
$\displaystyle=\int_{0}^{1}t^{4}\cosh(zt)dt$
$\displaystyle=\frac{(z^{4}+12z^{2}+24)\sinh(z)-(4z^{3}+24z)\cosh(z)}{z^{5}}>0.$

This implies that

$\tanh(z)>\frac{4z^{3}+24z}{z^{4}+12z^{2}+24}$

which gives the left hand side of (26). Also,

$\displaystyle\mathrm{Shi}^{(6)}(z)$
$\displaystyle=\int_{0}^{1}t^{5}\sinh(zt)dt$
$\displaystyle=\frac{(z^{5}+20z^{3}+120z)\cosh(z)-(5z^{4}+60z^{2}+120)\sinh(z)}{z^{6}}$
$\displaystyle>0.$

Hence

$\tanh(z)<\frac{z^{5}+20z^{3}+120z}{5z^{4}+60z^{2}+120}$

which gives the right hand side of (26). This completes the proof. ∎

######  Theorem 3.3.

For $z>0$, the inequalities

$\frac{6z^{5}+120z^{3}+720z}{z^{6}+30z^{4}+360z^{2}+720}<\tanh(z)<\frac{z^{7}+42z^{5}+840z^{3}+5040z}{7z^{6}+210z^{4}+2520z^{2}+5040}$
(27)

hold.

###### Proof.

By direct computations, we obtain

$\displaystyle\mathrm{Shi}^{(7)}(z)$
$\displaystyle=\int_{0}^{1}t^{6}\cosh(zt)dt$
$\displaystyle=\frac{(z^{6}+30z^{4}+360z^{2}+720)\sinh(z)-(6z^{5}+120z^{3}+720z)\cosh(z)}{z^{7}}$
$\displaystyle>0.$

This implies that

$\tanh(z)>\frac{6z^{5}+120z^{3}+720z}{z^{6}+30z^{4}+360z^{2}+720}$

which gives the left hand side of (27). Also,

$\displaystyle\mathrm{Shi}^{(8)}(z)$
$\displaystyle=\int_{0}^{1}t^{7}\sinh(zt)dt$
$\displaystyle=\frac{(z^{7}+42z^{5}+840z^{3}+5040z)\cosh(z)-(7z^{6}+210z^{4}+2520z^{2}+5040)\sinh(z)}{z^{8}}$
$\displaystyle>0.$

Hence

$\tanh(z)<\frac{z^{7}+42z^{5}+840z^{3}+5040z}{7z^{6}+210z^{4}+2520z^{2}+5040}$

which gives the right hand side of (27). This completes the proof. ∎

######  Theorem 3.4.

For $z>0$, the inequalities

$\frac{8z^{7}+336z^{5}+6720z^{3}+40320z}{z^{8}+56z^{6}+1680z^{4}+20160z^{2}+40320}<\tanh(z)\\\
<\frac{z^{9}+72z^{7}+3024z^{5}+60480z^{3}+362880z}{9z^{8}+504z^{6}+15120z^{4}+181440z^{2}+362880}$
(28)

hold.

###### Proof.

By direct computations, we obtain

$\displaystyle\mathrm{Shi}^{(9)}(z)$
$\displaystyle=\int_{0}^{1}t^{8}\cosh(zt)dt$
$\displaystyle=\frac{1}{z^{9}}\left[(z^{8}+56z^{6}+1680z^{4}+20160z^{2}+40320)\sinh(z)\right.$
$\displaystyle\quad\left.-(8z^{7}+336z^{5}+6720z^{3}+40320z)\cosh(z)\right]$
$\displaystyle>0.$

This implies that

$\tanh(z)>\frac{8z^{7}+336z^{5}+6720z^{3}+40320z}{z^{8}+56z^{6}+1680z^{4}+20160z^{2}+40320}$

which gives the left hand side of (28). Also,

$\displaystyle\mathrm{Shi}^{(10)}(z)$
$\displaystyle=\int_{0}^{1}t^{9}\sinh(zt)dt$
$\displaystyle=\frac{1}{z^{10}}\left[(z^{9}+72z^{7}+3024z^{5}+60480z^{3}362880z)\cosh(z)\right.$
$\displaystyle\quad\left.-(9z^{8}+504z^{6}+15120z^{4}+181440z^{2}+362880)\sinh(z)\right]$
$\displaystyle>0.$

Hence

$\tanh(z)<\frac{z^{9}+72z^{7}+3024z^{5}+60480z^{3}+362880z}{9z^{8}+504z^{6}+15120z^{4}+181440z^{2}+362880}$

which gives the right hand side of (28). This completes the proof. ∎

######  Remark 3.5.

The bounds in (28) are better than those in (27). The bounds in (27) are also
better than those in (26). And the bounds in (26) are also better than those
in (25).

######  Remark 3.6.

Due to their monotonicity properties, for $m\geq 2$, the derivatives of the
hyperbolic sine integral, $\mathrm{Shi}^{(m)}(z)$ give rational bounds for the
hyperbolic tangent function. Particularly, odd derivatives give lower bounds
and even derivatives give upper bounds. The corresponding bounds get better as
$m$ increases. It is also observed that, the lower bounds obtained this way,
are of the form $\frac{p^{\prime}(z)}{p(z)}$ and the upper bounds are of the
form $\frac{q(z)}{q^{\prime}(z)}$ for some polynomials $p(z)$ and $q(z)$.

As a byproduct of Theorem 3.1, we obtain the following result which provides
bounds for the hyperbolic cosine function.

######  Corollary 3.7.

For $z>0$, the inequalities

$\frac{z^{2}+2}{2}<\cosh(z)<e^{\frac{z^{2}}{6}}\left(\frac{z^{2}+2}{2}\right)^{\frac{2}{3}}$
(29)

hold.

###### Proof.

By integrating (25) over the interval $(0,z)$, we have

$\int_{0}^{z}\frac{2t}{t^{2}+2}dt<\int_{0}^{z}\tanh(t)dt<\int_{0}^{z}\frac{t^{3}+6t}{3t^{2}+6}dt$

which gives

$\ln(z^{2}+2)-\ln
2<\ln\cosh(z)<\frac{z^{2}}{6}+\frac{2}{3}\ln(z^{2}+2)-\frac{2}{3}\ln 2.$

That is

$\ln\frac{z^{2}+2}{2}<\ln\cosh(z)<\ln\left\\{e^{\frac{z^{2}}{6}}\left(\frac{z^{2}+2}{2}\right)^{\frac{2}{3}}\right\\}$

and by taking exponents, we obtain (29). ∎

## References

  * [1] M. Abramowitz and I. A. Stegun, Handbook of Mathematical Functions with formulas, Graphic and Mathematical Tables, Dover Publications, Inc., New York, (1965).
  * [2] B. Arpad and E. Neuman Inequalities Involving Generalized Bessel Functions, J. Inequal. Pure and Appl. Math., 6(4)(2005), Art No. 126, 1-9.
  * [3] Y. J. Bagul Inequalities Involving Circular, Hyperbolic and Exponential Functions, J. Math. Inequal., 11(3)(2017), 695-699.
  * [4] Y. J. Bagul and C. Chesneau Two double sided inequalities involving sinc and hyperbolic sinc functions, Int. J. Open Problems Compt. Math., 12(4)(2019), 15-20.
  * [5] Y. J. Bagul and C. Chesneau Some New Simple Inequalities Involving Exponential, Trigonometric and Hyperbolic Functions, Cubo. A Mathematical Journal., 21(1)(2019), 21-35.
  * [6] Y. J. Bagul, R. M. Dhaigude, M. Kostic and C. Chesneau Polynomial-Exponential Bounds for Some Trigonometric and Hyperbolic Functions, Axioms, 10 (2021), Art No. 308, 1-10.
  * [7] Y. J. Bagul, S. B. Thool , C. Chesneau and R. M. Dhaigude Refinements of Some Classical Inequalities Involving Sinc and Hyperbolic Sinc Functions, Ann. Math. Sil., (2022), Published Ahead of Print. DOI: https://doi.org/10.2478/amsil-2022-0019.
  * [8] W-H. Li, P. Miao and B-N. Guo Bounds for the Neuman-Sandor Mean in Terms of the Arithmetic and Contra-Harmonic Means, Axioms, 11(5)(2022), Art No. 236, 1-12.
  * [9] M. Merca The cardinal sine function and the Chebyshev-Stirling numbers, J. Number Theory, 160(2016), 19-31.
  * [10] K. Nantomah and E. Prempeh Some Inequalities for Generalized Hyperbolic Functions, Moroccan J. of Pure and Appl. Anal., 6(1)(2020), 76-92.
  * [11] C. P. Niculescu Convexity according to the geometric mean, Math. Inequal. Appl., 2(2)(2000), 155-167.
  * [12] V. Nijimbere Evaluation of Some Non-Elementary Integrals Involving Sine, Cosine, Exponential and Logarithmic Integrals: Part II, Ural Math. J., 4(1)(2018), 43-55.
  * [13] I. Pinelis L’hospital type Rules for Monotonicity, with Applications, J. Inequal. Pure Appl. Math., 3(1)(2002), Art No. 5, 5 pages.
  * [14] B. Ravi and A. V. Laxmi Subadditive and completely monotonic properties of the tricomi confluent hypergeometric functions, Inter. J. Adv. Math., 5(2018), 25-33.
  * [15] J. Sánchez-Reyes The Hyperbolic Sine Cardinal and the Catenary, College Math. J., 43(4)(2012), 285-290.
  * [16] J. Sandor Two applications of the Hadamard integral inequality, Notes Number Theory Discrete Math., 23(4)(2017), 52-55.
  * [17] L. Zhu Inequalities for Hyperbolic Functionsand Their Applications, J. Inequal. Appl., 2010(2010), Art No. 130821, 10 pages.


# Multitrack Music Transcription with a Time-Frequency Perceiver

Multitrack music transcription aims to transcribe a music audio input into the
musical notes of multiple instruments simultaneously. It is a very challenging
task that typically requires a more complex model to achieve satisfactory
result. In addition, prior works mostly focus on transcriptions of regular
instruments, however, neglecting vocals, which are usually the most important
signal source if present in a piece of music. In this paper, we propose a
novel deep neural network architecture, _Perceiver TF_ , to model the time-
frequency representation of audio input for multitrack transcription.
Perceiver TF augments the Perceiver architecture by introducing a hierarchical
expansion with an additional Transformer layer to model temporal coherence.
Accordingly, our model inherits the benefits of Perceiver that posses better
scalability, allowing it to well handle transcriptions of many instruments in
a single model. In experiments, we train a Perceiver TF to model 12 instrument
classes as well as vocal in a multi-task learning manner. Our result
demonstrates that the proposed system outperforms the state-of-the-art
counterparts (e.g., MT3 and SpecTNT) on various public datasets.

Index Terms—  Time-frequency, Perceiver, automatic music transcription, multi-
task learning, random-mixing augmentation.

##  1 Introduction

Automatic music transcription (AMT) is a Music Information Retrieval (MIR)
task that aims to transcribe a music audio input into a sequence of musical
notes where each note contains attributes of onset, pitch, duration, and
velocity. The output is typically delivered in the format of MIDI. In a
multitrack setting, an AMT system should identify every instrument that is
present in the input, and estimate the associated notes accordingly into a
channel of the MIDI output. Ideally speaking, using the identified instrument
for each corresponding channel, the synthesized audio mixture from the output
MIDI should resemble the original input audio in a musically plausible way.

Although recent years have seen significant progress using deep learning
techniques [1, 2], our analysis and review indicate that two major challenges
have not yet been addressed effectively: _model scalability_ and _instrument
discrimination_. Multitrack AMT is generally regarded as a very challenging
task. The number of commonly used instruments can be up to 100. Among them,
musical notes of regular instruments like guitar, violin, and synthesizers are
difficult to characterize due to their tremendous variations in timbre,
expressivity, and playing techniques. Other than that, vocals, which usually
are the most predominant instrument if present, vary their timbre and pitch to
convey lyrics and expressions. To handle all instruments simultaneously, it
requires better _model scalability_. Our observations on existing multitack
AMT systems reveal that they oftentimes result in many false positive notes
for popular pitched instruments like piano and guitar. For instance, notes of
string ensemble are massively captured by piano. This might be because the
system does not provide clear timbre-dependent features or it is not robust to
timbral variations across different instruments. We believe this problem can
be mitigated if the system can _discriminate_ each instrument source from the
mixture while making inference.

To address model scalability, we propose _Perceiver TF_ , which is an
augmented variant of the Perceiver [3]. Perceiver has been well-known for its
better scalability in the Transformer family to tackle high-dimensional data
input. In this work, we adopt spectrogram for audio input, with $T$ and $F$
representing the lengths of the time- and frequency-axes, respectively. For
multitrack AMT, capability to model the timbre-dependent pitches of multiple
instruments is crucial, so more comprehensive operations are needed to capture
useful features along the high-resolution frequency axis. Recently, the
SpecTNT architecture [4] was proposed for this purpose and achieved state-of-
the-art performance in vocal melody extraction (a sub-task of AMT). SpecTNT
consists of two Transformers in a hierarchical structure, where the lower-
level Transformer performs self-attention directly on the spectrum of a frame.
However, such design leads to a cubic complexity of attention computation,
i.e., $\mathcal{O}(TF^{2}+T^{2})$, limiting its expandability for more complex
tasks. To this end, we conceive a non-trivial combination of Perceiver and
SpecTNT: expanding Perceiver to be hierarchical. The resulting _Perceiver TF_
takes advantage of the cross-attention to extract spectral features into a
latent bottleneck for each frame, and adds an additional Transformer for self-
attention along the time axis, overall resulting in a quadratic complexity of
$\mathcal{O}(TF+T^{2})$. Since $F$ is typically large, this complexity
reduction is significant, allowing the model to handle more instruments
simultaneously.

To address _instrument discrimination_ , we adopt the _random-mixing_
augmentation technique learned from music source separation (MSS) [5, 6],
which aims to separate each instrument stem from the input audio mixture [7].
Moreover, we train our AMT model in a multi-task learning fashion, with each
sub-task modeling the transcription of an instrument. This multi-task design
along with the random-mixing technique allows more flexibility to train with
enormous amount of augmented training samples. Our strategy differs from
previous works that jointly train the AMT task with instrument recognition [8]
or MSS [9] to help inform the model of instrument-dependent features. To our
knowledge, little work has been done using random-mixing technique to improve
multitrack AMT.

##  2 Related work

Multi-instrument AMT has been explored in several previous works. Wu et al.
[10] and Hung et al. [8] trained a transcription model with related tasks in a
multi-task learning fashion. Tanaka et al. used clustering approaches to
separate transcribed instruments [11], while Cheuk et al. used unsupervised
learning techniques to improve transcription on low-resource datasets [1, 12].
These prior examples demonstrated that models based on the pianoroll
representation are able to capture instrument-dependent onset, pitch, and
duration of notes. Different from the pianoroll approach, Gardner et al. [2]
created a new paradigm that proposes a sequence-to-sequence model, called MT3,
to tackle multitrack AMT. They trained a standard encoder-decoder Transformer
to model multitrack MIDI tokens from audio, and demonstrated state-of-the-art
performance on several public datasets.

By contrast, vocal transcription is usually treated as an independent task in
the literature, even though it shares the same goal of AMT. Due to the lack of
training data, few works focused on transcribing note-level outputs from
polyphonic music audio. Recently, Wang et al. released a human annotated
dataset including 500 Chinese songs [13]. They provide a CNN based model (EFN)
for a baseline of the task. In [14], a teacher-student training scheme is
proposed to utilize pseudo labels derived from F0 estimations of vocal.
Lately, [15] proposed a vocal transcription system that requires an MSS as
front-end. In this work, we propose a unified framework that combines vocal
and multi-instrument transcriptions, and it does not rely on pre-trained
modules such as an MSS front-end.

##  3 Methodology

Fig. 1: The block diagram of the Perceiver TF module. Positional embedding is
first added to the latent arrays, denoted as $\Theta^{0}_{t}$. The Spectral
Cross-Attention module projects the spectral input $S_{t}$ into
$\Theta^{h}_{t}$, followed by the Latent Transformer module. The Temporal
Transformer processes $\Theta^{h}_{t}$ of all time steps to model the temporal
coherence. The details are explained in Section 3.2.

In this work, we adopt the pianoroll approach instead of the sequence-to-
sequence (seq-to-seq) approach for two major reasons. First, it is easier to
manipulate the loss computation to learn from partially labeled data. For
example, it is non-trivial to train a seq-to-seq model that joints a vocal
transcription dataset where the MIDI ground truth of accompaniments is not
available. Second, the inference time complexity of seq-to-seq depends on the
number of notes (tokens) due to the auto-regressive nature. If the audio input
contains many instruments with complex, dense polyphonic notes, the inference
will be very slow. Although our proposed model is also a Transformer-oriented
architecture, we focus on the encoder part to predict the pianoroll directly.

The following sections explain the proposed model architecture (Sections 3.1 –
3.3) and the random-mixing augmentation technique (Section 3.4). Our model
consists of three sub-modules: convolutional module, Perceiver TF module, and
output module. The input spectrogram is first passed through the convolution
module for local feature aggregation. Then, the perceiver TF module, which
includes multiple Perceiver TF blocks, extracts the features and outputs the
temporal embeddings at each time step. Lastly, the output module projects the
temporal embeddings into desired dimensions for pianoroll outputs.

###  3.1 Convolutional Module

Using convolutional neural network (CNN) as the front-end of Transformer-based
models has became a common design choice in speech recognition pipeline [16].
Previous works have also found that the CNN front-end plays an crucial role in
SpecTNT and MIRTransformer for many MIR tasks [4, 17, 18, 19]. Following this
practice, we stack multiple residual units [20] with average pooling to reduce
the dimensionality of the frequency axis. We denote the resulting time-
frequency representation as
$\mathcal{S}=[S_{0},S_{1},\dots,S_{T-1}]\in\mathbb{R}^{T\times F\times C}$,
where $T$, $F$, and $C$ represent the dimensions of time, frequency, and
channel, respectively.

###  3.2 Perceiver TF Module

A conventional Perceiver architecture contains two major components [3]: (i) a
cross-attention module that maps the input data and a latent array into a
latent array; (ii) a Transformer tower that maps a latent array into a latent
array. Upon this structure, our design principle to expand Perceiver is
twofold. (1) We consider the spectral representation of a time step, $S_{t}$,
is pivotal to carry the pitch and timbral information, so it serves as the
input data for the cross-attention module to project the spectral information
into a latent array for the time step $t$. Each latent array is responsible
for extracting the local spectral features. (2) Having a sequence of latent
arrays of different time steps, we need a Transformer to exchange the local
spectral information along the time axis to learn their temporal coherence.

The Perceiver TF architecture is illustrated in Fig. 1. A Perceiver TF block
contains three Transformer-style modules: _spectral cross-attention_ , _latent
Transformer_ , and _temporal Transformer_ , which are responsible for modeling
the spectral, channel-wise, and temporal information, respectively. Each of
them includes the attention mechanism and a position-wise feed-forward
network.

The _spectral cross-attention_ (SCA) module operates directly on an input
spectral representation $S_{t}$ and projects it into the Key ($K$) and Value
($V$) matrices. Unlike the traditional Transformer, the cross-attention module
in Perceiver maps a latent array into the Query ($Q$) matrix and then performs
the $QKV$ self-attention accordingly. We follow the Perceiver design to
initialize a set of $K$ learnable latent arrays
$\Theta^{0}\in\mathbb{R}^{K\times D}$, where $K$ is the index dimension, and
$D$ is the channel dimension. Then, we repeat $\Theta^{0}$ for $T$ times and
associate each to a time step $t$, which is then denoted as $\Theta_{t}^{0}$,
such that $\Theta_{0}^{0}=\Theta_{1}^{0}=\dots\Theta_{T-1}^{0}$, meaning that
all latent arrays are from the same initialization across the time axis. This
$\Theta_{t}^{h}$ plays an important role of carrying the spectral information
from the first Perceiver TF block throughout the entire stack of blocks. The
query-key-value ($QKV$) attention of our SCA of the $h$-th iteration can be
written as:
$f_{\text{SCA}}:\\{\Theta_{t}^{h},S_{t}\\}\rightarrow\Theta_{t}^{(h+1)}$, and
this process will be repeated as the Perceiver TF block repeats in order to
maintain the connection between $\Theta_{t}^{h}$ and the input $S_{t}$. The
design of the cross-attention module is the key that significantly improves
the computational scalability of Perceiver. For instance, our SCA results in
$\mathcal{O}(FK)$, which is much cheaper than $\mathcal{O}(F^{2})$ of the
spectral Transformer in SpecTNT [4], given that $K$ (dimension of the latent
array) is typically small (i.e., $K\ll F$).

The _latent Transformer_ module takes place after the SCA module. It contains
a stack of $N$ Transformers to perform standard self-attention on the latent
arrays of $\Theta_{t}^{h}$. The resulting complexity $\mathcal{O}(NK^{2})$ is
efficient as well. In the context of AMT, this process means the interactions
among the onsets, pitches, and instruments are explicitly modeled. To perform
multitrack AMT, we initialize $K$ latent arrays and train each latent array to
handle one specific task. Following [21], for an instrument, we arrange two
latent arrays to model the onset and frame-wise (pitch) activations,
respectively. This leads to $K=2J$, where $J$ is the number of target
instruments.

The _temporal Transformer_ module is placed to enable the communication
between any pairs of $\Theta_{t}^{h}$ of different time steps. To make the
_temporal Transformer_ understand the time positions of each latent array, we
add a trainable positional embedding to each $\Theta_{t}^{0}$ during the
initialization. Let $\theta_{t}^{h}(k)$, $k$ = 0, $\dots$, $K$–1, denote each
latent array in $\Theta_{t}^{h}$, we arrange $K$ parallel standard
Transformers in which each serves the corresponding input sequence of latent
arrays: $[\theta_{0}^{h}(k),\theta_{1}^{h}(k),\dots,\theta_{T-1}^{h}(k)]$. The
module is repeated $M$ times, yielding a complexity of $\mathcal{O}(MT^{2})$.

Finally, we repeat $L$ times the Perceiver TF block to form the overall
module. Note that, different from the original Perceiver, the weights of
_spectral cross-attention_ and _latent Transformer_ are not shared across the
repeated blocks.

###  3.3 Output Module

We utilize two GRU modules [22] with sigmoid activation function for the onset
and frame-wise latent array outputs, respectively. We follow prior work [21]
that uses the onset outputs to condition the frame-wise activation.

###  3.4 Multi-task Training Loss

We formulate the loss function for training the proposed model:

$\mathcal{L}=\sum^{J-1}_{j=0}(l^{j}_{\text{onset}}+l^{j}_{\text{frame}})$ (1)

where $l$ is the binary cross-entropy loss between the ground-truth and
prediction, $l^{j}_{\text{onset}}$ and $l^{j}_{\text{frame}}$ are respectively
the onset and frame activation losses for instrument $j$. Note that the losses
for all $J$ instruments should be computed, regardless of whether the
corresponding instruments are active or not in a training sample. Therefore, a
zero output is expected for instruments that are not present in the sample.

Slakh | All | Piano | Bass | Drums | Guitar | Strings | Brass | Organ | Pipe | Reed | S.lead | S.pad | C.perc.
---|---|---|---|---|---|---|---|---|---|---|---|---|---
MT3† | .743 | .780 | .906 | .773 | .732 | .551 | .433 | .363 | .282 | .440 | .409 | .234 | .353
Ours (No-RM) | .763 | .809 | .921 | .759 | .727 | .699 | .632 | .562 | .578 | .649 | .677 | .358 | .458
Ours | .798 | .854 | .930 |  .785 | .777 | .744 | .732 | .694 | .666 | .725 | .769 | .474 | .575

Table 2: The results of different models trained on (Mix) datasets and tested
on Slakh2100. MT3† is our replication, as the instrument-wise results are not
reported in [2]. “All” presents the Multi-instrument Onset F1 scores. The
following columns show the Onset F1 scores for individual instrument.
“S.lead”, “S.pad”, and “C.perc.” stand for Synth Lead, Synth Pad, and
Chromatic Percussion, respectively.

Dataset | Slakh | MAESTRO | GuitarSet |  MIR-ST500
---|---|---|---|---
SpecTNT (Single) | - | .969 | .907 |  .778
MT3 (Single) | .760 | .960 | .830 |  -
MT3 (Mix) | .760 | .950 | .900 |  -
MT3† (Mix) | .763 | .958 | .891 |  -
Ours (Single) | .808 | .967 | .903 |  .777
Ours (Mix+Vocal) | .819 | .968 | .911 |  .785
EFN | - | - | - |  .666
JDCnote(L+U) | - | - | - |  .697

Table 1: The results of Onset F1 scores. MT3† is our replication. Models with
(Mix) or (Mix+Vocal) are trained on the mixture of datasets, while models with
(Single) are trained on a single dataset.

##  4 Experiments

###  4.1 Datasets

We use four public datasets for evaluation. Slakh2100 [23] contains 2100
pieces of multitrack MIDI and the corresponding synthesized audio. The MIDI
files are a subset of Lakh dataset [24], and the audio samples were
synthesized by professional-grade software. Instruments were grouped into 12
MIDI classes defined in the Slakh dataset.111There is no ”Sound Effects”,
”Percussive” and ”Ethnic” instruments. We grouped ”Strings” and ”Ensemble”
into one instrument class. We used the official train/validation/test splits
in our experiments. MAESTROv3 [25] contains about 200 hours of piano solo
recordings with the aligned note annotations acquired by the MIDI capturing
device on piano. We follow the official train/validation/test splits.
GuitarSet [26] contains 360 high-quality guitar recordings and their
synchronized note annotations. Since there is no official splits for this
dataset, we follow the setting in [2]. The first two progressions of each
style are used for training, and the last one is for testing. MIR-ST500 [13]
contains 500 Chinese-pop songs with note annotations for the lead vocal
melody. We used the official train-test split. Although around 10% of the
training set is missing due to failure links, we ensure the testing set is
complete for fair comparison.

###  4.2 Data Augmentations

Annotating data for multitrack AMT is labor intensive. To better exploit the
data at hand, we apply two data augmentation techniques during training.
Following previous works [4, 27], pitch-shifting is randomly performed to all
the non-percussive instruments during training. We introduce the cross-dataset
_random-mixing_ (RM) technique. Let us first define three types of datasets:

  * •

_Multi-track_ : each sample contains multi-tracks of instrument-wise audio
stems with polyphonic notes (e.g., Slakh), and no vocal signals are present.

  * •

_Single-track_ : each sample contains only a single non-vocal stem with
polyphonic notes (e.g., MAESTRO and GuitarSet).

  * •

_Vocal-mixture_ : each sample is a full mixture of music with monophonic notes
only for lead vocal (e.g., MIR-ST500). We employ a MSS tool [28] to separate
each sample into vocal and accompaniment stems.

Each training sample is excerpted from a random moment of its original song
with a duration depending on the model input length (e.g., 6 seconds). Suppose
we want to transcribe $J$ classes of instruments, and the corresponding
instrument set is denoted as $\mathbf{\Omega}=\\{\omega_{j}\\}_{j=0}^{J-1}$.
Then, we apply three treatments to the three mentioned types of datasets
respectively as follows.

First, for a training sample $s_{i}$ from a _multi-track_ dataset, we denote
its instrumentation template as $\bm{\mu}_{i}\subseteq\mathbf{\Omega}$,
indicating the instruments present in $s_{i}$. Then, for each instrument
$\omega_{j}$ in $\bm{\mu}_{i}$, it has a $p$% chance to be replaced by a
$\omega_{j}$ in $\bm{\mu}_{u}$, where $i\neq u$ (i.e., a different sample).
Second, for a sample $s_{i}$ from a _single-track_ dataset, we randomly pick
an existing instrumentation template $\bm{\mu}_{u}$ ($i\neq u$) as its
background. If the instrument of $s_{i}$ is present in $\bm{\mu}_{u}$, that
stem will be removed from $\bm{\mu}_{u}$. For instance, if $s_{i}$ is a piano
solo, then we will remove the piano stem from $\bm{\mu}_{u}$. From our
preliminary experiment, presenting the solo example to model training without
mixing it with a background can degrade the performance. Lastly, for a sample
$s_{i}$ from a _vocal-mixture_ dataset, it has a $q$% chance to replace its
background by two methods: (i) like the _single-track_ treatment, we randomly
pick an existing $\bm{\mu}_{u}$ ($i\neq u$) as its background; or (ii) we
randomly pick an accompaniment stem separated from $s_{v}$, where $i\neq v$.
For the second method, since the selected accompaniment stem does not have the
ground-truth notes, we mask the instrument outputs and only count the loss for
the vocal output (see Eq. 1).

###  4.3 Implementation Details

We implemented our system using PyTorch [29]. The audio waveform is re-sampled
to 16kHz sampling rate. We set the model input length to be 6 seconds. The
log-magnitude spectrogram is then computed using 2048 samples of Hann window
and a hop size of 320 samples (i.e., 20 ms). The convolutional module contains
3 residual blocks, each of them has 128 channels and is followed by an average
pooling layer with a time-frequency filter of (1, 2).

For the Perceiver TF module, we use the following parameters (referring to
Fig. 1): (i) depending on different experiment configurations, initialize $2J$
latent arrays, each uses a dimension of 128; (ii) stack $L=3$ Perceiver TF
blocks; (iii) for each Perceiver TF block, use 1 spectral cross-attention
layer, $N=2$ latent Transformer layers, and $M=2$ temporal Transformer layers.
All the Transformer layers has an hidden size of 128 with 8 heads for the
multi-head attention. Finally, the output module is a 2-layer Bi-directional
GRU with 128 hidden units. All of the Transformer module in the Perceiver TF
include dropout with a rate of 0.15. The output dimension for onset and frame
activations are 128 and 129, respectively, where 128 corresponds to the MIDI
pitches, and the additional 1 dimension in the frame activation is for the
silence. We use AdamW [30] as the learning optimizer. The initial learning
rate and weight decay rate are set to $10^{-3}$ and $5\times 10^{-3}$,
respectively.

For final output, we take a threshold of 0.25 for both the onset and frame
probability outputs to get the binary representations, so the frame-wise
activations can be merged to generate each note in a piano-roll
representation. No further post-processing is applied.

For data augmentation, all of the non-percussive instruments of a training
example have a 100$\%$ probability to be pith-shift up or down by at most 3
semi-tones. For random-mixing, we use $p=25\%$ and $q=50\%$ for data from
_multi-track_ and _vocal-mixture_ datasets, respectively. To generate an input
sample, all the instrument stems in each training example are linearly summed
up.

###  4.4 Baselines

Two state-of-the-art models, MT3 [2] and SpecTNT [4], are selected as the
baselines. For MT3, we replicated the model following
[2],222https://github.com/magenta/mt3/blob/main/mt3/colab/
music_transcription_with_transformers.ipynb which includes the official model
checkpoint and inference pipeline on the test set. For SpecTNT, we adopted the
configuration used for vocal melody extraction reported in [4]. In the
preliminary experiments, we found it non-trivial to successfully train the
original SpecTNT on Slakh2100 under the multi-instrument setting, so we skip
this experiment. For vocal transcription, the best results of EFN [13] and
JDCnote(L+U) [14] are reported.

###  4.5 Evaluation Metrics

We use “Onset F1” score, which indicates the correctness of both pitches and
onset timestamps, as the evaluation metric for comparison with previous work
[2]. To further evaluate the performance of multi-instrument transcription, we
report the ”Multi-instrument Onset F1” score for the Slakh dataset. The
outputs from our replicated MT3 model are grouped into 12 instrument classes
based on their program numbers. The Multi-instrument Onset F1 score we used
only counts Onset F1, which is similar to the MV2H metric [31]. It could be
slightly different from the one used in [2], since the “Drums” outputs do not
contain clear offset information.

###  4.6 Result and Discussion

Table 1 shows the comparison in terms of Onset F1 between the proposed model
and baselines. The proposed model and SpecTNT which directly model the
spectral inputs with the attention mechanism shows higher performance for
cases even trained on low resources of a single dataset, such as GuitarSet. On
MIR-ST500, the proposed model significantly outperforms the baselines.
Although SpecTNT (Single) performs slightly better than our model on MAESTRO,
we still consider Perceiver TF to be more advantageous to practical use for
its better inference efficiency.

Table 2 presents the Multi-instrument Onset F1 (instrument-weighted average)
and the Onset F1 scores of individual instrument classes on Slakh2100 to
reveal instrument-wise performance. Compared to $\text{MT3}^{\dagger}$, our
model without the random-mixing augmentation (No-RM) performs significantly
better on less-common instruments such as “Pipe” (the Onset F1 score is upper
by over 100%). Applying random-mixing in training can further boost the
performance in all cases, indicating the technique indeed improves the model
robustness to discriminate between different instruments. Finally, we observe
that combining multi-instrument and vocal transcriptions can improve the vocal
transcription alone, as the combined model is trained with more randomly mixed
vocal-accompaniment samples.

##  5 Conclusion

We have presented Perceiver TF, a novel architecture that adequately addresses
the _model scalability_ problem for multitrack AMT. To address the _instrument
discrimination_ issue, we have proposed the random-mixing augmentation
technique, which significantly facilitates the data usability across datasets.
Our system has demonstrated state-of-the-art performance on different public
datasets. We believe Perceiver TF is generic and can be applied to other
analogous tasks.

## References

  * [1] Kin Wai Cheuk, Dorien Herremans, and Li Su,  “Reconvat: A semi-supervised automatic music transcription framework for low-resource real-world data,”  in Proc. ACM Multimedia, 2021, pp. 3918–3926.
  * [2] Josh Gardner, Ian Simon, Ethan Manilow, Curtis Hawthorne, and Jesse Engel,  “MT3: Multi-task multitrack music transcription,”  in Proc. ICLR, 2021.
  * [3] Andrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals, Andrew Zisserman, and Joao Carreira,  “Perceiver: General perception with iterative attention,”  in Proc. ICML, 2021, pp. 4651–4664.
  * [4] Wei-Tsung Lu, Ju-Chiang Wang, Minz Won, Keunwoo Choi, and Xuchen Song,  “SpecTNT: A time-frequency transformer for music audio,”  in Proc. ISMIR, 2021.
  * [5] Stefan Uhlich, Marcello Porcu, Franck Giron, Michael Enenkl, Thomas Kemp, Naoya Takahashi, and Yuki Mitsufuji,  “Improving music source separation based on deep neural networks through data augmentation and network blending,”  in Proc. ICASSP, 2017, pp. 261–265.
  * [6] Xuchen Song, Qiuqiang Kong, Xingjian Du, and Yuxuan Wang,  “Catnet: Music source separation system with mix-audio augmentation,”  arXiv preprint arXiv:2102.09966, 2021.
  * [7] Zafar Rafii, Antoine Liutkus, Fabian-Robert Stöter, Stylianos Ioannis Mimilakis, Derry FitzGerald, and Bryan Pardo,  “An overview of lead and accompaniment separation in music,”  IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 26, no. 8, pp. 1307–1335, 2018.
  * [8] Yun-Ning Hung, Yi-An Chen, and Yi-Hsuan Yang,  “Multitask learning for frame-level instrument recognition,”  in Proc. ICASSP, 2019, pp. 381–385.
  * [9] Andreas Jansson, Rachel M Bittner, Sebastian Ewert, and Tillman Weyde,  “Joint singing voice separation and f0 estimation with deep u-net architectures,”  in Proc. EUSIPCO, 2019, pp. 1–5.
  * [10] Yu-Te Wu, Berlin Chen, and Li Su,  “Multi-instrument automatic music transcription with self-attention-based instance segmentation,”  IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 28, pp. 2796–2809, 2020.
  * [11] Keitaro Tanaka, Takayuki Nakatsuka, Ryo Nishikimi, Kazuyoshi Yoshii, and Shigeo Morishima,  “Multi-instrument music transcription based on deep spherical clustering of spectrograms and pitchgrams,”  in Proc. ISMIR, 2020.
  * [12] Kin Wai Cheuk, Keunwoo Choi, Qiuqiang Kong, Bochen Li, Minz Won, Amy Hung, Ju-Chiang Wang, and Dorien Herremans,  “Jointist: Joint learning for multi-instrument transcription and its applications,”  arXiv preprint arXiv:2206.10805, 2022.
  * [13] Jun-You Wang and Jyh-Shing Roger Jang,  “On the preparation and validation of a large-scale dataset of singing transcription,”  in Proc. ICASSP, 2021, pp. 276–280.
  * [14] Sangeun Kum, Jongpil Lee, Keunhyoung Luke Kim, Taehyoung Kim, and Juhan Nam,  “Pseudo-label transfer from frame-level to note-level in a teacher-student framework for singing transcription from polyphonic music,”  in Proc. ICASSP, 2022.
  * [15] Jui-Yang Hsu and Li Su,  “Vocano: A note transcription framework for singing voice in polyphonic music.,”  in Proc. ISMIR, 2021, pp. 293–300.
  * [16] Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, et al.,  “Conformer: Convolution-augmented transformer for speech recognition,”  in Proc. INTERSPEECH, 2020.
  * [17] M. Won, K. Choi, and X. Serra,  “Semi-supervised music tagging transformer,”  in Proc. ISMIR, 2021, pp. 769–776.
  * [18] Yun-Ning Hung, Ju-Chiang Wang, Xuchen Song, Wei-Tsung Lu, and Minz Won,  “Modeling beats and downbeats with a time-frequency transformer,”  in Proc. ICASSP, 2022, pp. 401–405.
  * [19] Ju-Chiang Wang, Yun-Ning Hung, and Jordan B. L. Smith,  “To catch a chorus, verse, intro, or anything else: Analyzing a song with structural functions,”  in Proc. ICASSP, 2022, pp. 416–420.
  * [20] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun,  “Identity mappings in deep residual networks,”  in European conference on computer vision. Springer, 2016, pp. 630–645.
  * [21] Curtis Hawthorne, Erich Elsen, Jialin Song, Adam Roberts, Ian Simon, Colin Raffel, Jesse Engel, Sageev Oore, and Douglas Eck,  “Onsets and frames: Dual-objective piano transcription,”  in Proc. ISMIR, 2018, pp. 50–57.
  * [22] Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio,  “Empirical evaluation of gated recurrent neural networks on sequence modeling,”  in Proc. NeurIPS, 2014.
  * [23] Ethan Manilow, Gordon Wichern, Prem Seetharaman, and Jonathan Le Roux,  “Cutting music source separation some Slakh: A dataset to study the impact of training data quality and quantity,”  in Proc. WASPAA. IEEE, 2019.
  * [24] Colin Raffel,  “Learning-based methods for comparing sequences, with applications to audio-to-midi alignment and matching,”  2016, Columbia University.
  * [25] Curtis Hawthorne, Andriy Stasyuk, Adam Roberts, Ian Simon, Cheng-Zhi Anna Huang, Sander Dieleman, Erich Elsen, Jesse Engel, and Douglas Eck,  “Enabling factorized piano music modeling and generation with the maestro dataset,”  in Proc. ICLR, 2019.
  * [26] Qingyang Xi, Rachel M Bittner, Johan Pauwels, Xuzhou Ye, and Juan Pablo Bello,  “Guitarset: A dataset for guitar transcription.,”  in Proc. ISMIR, 2018, pp. 453–460.
  * [27] Sangeun Kum and Juhan Nam,  “Joint detection and classification of singing voice melody using convolutional recurrent neural networks,”  Applied Sciences, vol. 9, no. 7, pp. 1324, 2019.
  * [28] Qiuqiang Kong, Yin Cao, Haohe Liu, Keunwoo Choi, and Yuxuan Wang,  “Decoupling magnitude and phase estimation with deep resunet for music source separation.,”  in Proc. ISMIR, 2021.
  * [29] Paszke et al.,  “Pytorch: An imperative style, high-performance deep learning library,”  in Neural Information Processing Systems, 2019, vol. 32.
  * [30] Ilya Loshchilov and Frank Hutter,  “Decoupled weight decay regularization,”  in Proc. ICLR, 2017.
  * [31] Andrew McLeod and Mark Steedman,  “Evaluating automatic polyphonic music transcription.,”  in Proc. ISMIR, 2018, pp. 42–49.


2023

[1]Thomas W.J. Metzger

[2]Evgeny A. Mashkovich

1]Institute for Molecules and Materials, Radboud University, Heyendaalseweg
135, Nijmegen, 6525 AJ, The Netherlands

2]Institute of Physics II, University of Cologne, Zuelpicher Straße 77,
Cologne, 10587, Germany

3]Ioffe Institute, Russian Academy of Sciences, St. Petersburg, 194021, Russia

4]Institute of Radiation Physics, Helmholtz-Zentrum Dresden-Rossendorf,
Bautzner Landstraße 400, Dresden, 01328, Germany

# Impulsive Fermi magnon-phonon resonance in antiferromagnetic CoF2

<EMAIL_ADDRESS>Kirill A. Grishunin  Chris Reinhoffer  Roman M. Dubrovin
Atiqa Arshad  Igor Ilyakov  Thales V.A.G. de Oliveira  Alexey Ponomaryov  Jan-
Christoph Deinert  Sergey Kovalev  Roman V. Pisarev  Mikhail I. Katsnelson
Boris A. Ivanov  Paul H. M. van Loosdrecht  Alexey V. Kimel
<EMAIL_ADDRESS>[  [  [  [

###### Abstract

Understanding spin-lattice interactions in antiferromagnets is one of the most
fundamental issues at the core of the recently emerging and booming fields of
antiferromagnetic spintronics and magnonics. Recently, coherent nonlinear
spin-lattice coupling was discovered in an antiferromagnet which opened the
possibility to control the nonlinear coupling strength and thus showing a
novel pathway to coherently control magnon-phonon dynamics. Here, utilizing
intense narrow band terahertz (THz) pulses and tunable magnetic fields up to 7
T, we experimentally realize the conditions of the Fermi magnon-phonon
resonance in antiferromagnetic CoF2. These conditions imply that both the spin
and the lattice anharmonicities harvest energy transfer between the
subsystems, if the magnon eigenfrequency $f\textsubscript{m}$ is twice lower
than the frequency of the phonon $2f\textsubscript{m}=f\textsubscript{ph}$.
Performing THz pump-infrared probe spectroscopy in conjunction with
simulations, we explore the coupled magnon-phonon dynamics in the vicinity of
the Fermi-resonance and reveal the corresponding fingerprints of an impulsive
THz-induced response. This study focuses on the role of nonlinearity in spin-
lattice interactions, providing insights into the control of coherent magnon-
phonon energy exchange.

###### keywords:

Antiferromagnets, spin-lattice interaction, magnon-phonon dynamics, Fermi
resonance, terahertz pump - infrared probe spectroscopy

##  1 Introduction

The lattice dynamics of crystals involve interdependent periodic movements of
individual atoms, rendering them inherently complex. Nevertheless, the modern
theory of condensed matter has successfully managed to describe the dynamics
in terms of linear superposition of mutually independent phononic modes
comprising acoustic phonons and optical phonons. However, if the amplitude of
the lattice vibrations is large, this approximation fails
maehrlein_terahertz_2017 ; forst_nonlinear_2011  and the lattice dynamics
enter a poorly explored regime, in which a single phonon can be excited by
multiple photons and new channels of energy transfer between otherwise non-
interacting phononic modes open up. This regime became the main focus of the
establishing field of nonlinear phononics disa_engineering_2021 . Lately, such
a nonlinear regime was also found for magnonic systems zhang_three-wave_2023 ;
blank_empowering_2023 . Alternatively one can enhance nonlinearity not by
increasing the modulation amplitude but instead by realizing a coupling of
eigenstates reaching the nonlinear resonance matching conditions.
Surprisingly, this route is barely explored especially under the consideration
of accounting for a coupling between different subsystems.

In 1931 fermi_ueber_1931 , Enrico Fermi reported about the interaction of
seemingly non-interacting vibrational modes in carbon dioxide CO2 molecules
whose frequencies differ by a factor of two. In particular, it was argued that
such an interaction explains a splitting present in the Raman spectrum in
terms of a coupling between a single and an overtone vibrational mode of
molecules. It was suggested that this resonance should be accompanied by a
resonant energy transfer between the modes katsnelson_resonance_1987 ;
gornostyrev_phase_1995 ; gornostyrev_stochastic_1999 ; katsnelson_fermi-
resonance-like_2002 ; katsnelson_nonperturbative_2004 . The phenomenon of
nonlinear coupling, satisfying the conditions set forth by Fermi, was also
observed in magnon dynamics barsukov_giant_2019  resulting in shape
deformations and broadening of the respective magnon spectrum, as well as in
nano-mechanical systems showing coupled vibrational dynamics
shoshani_anomalous_2017 . Interestingly, the recently demonstrated nonlinear
coupling between spin and lattice dynamics mashkovich_terahertz_2021  suggests
that the Fermi resonance condition can be fulfilled in antiferromagnets as
well. In zero applied magnetic field the double magnon frequency 2$f_{0}$ of
antiferromagnetic CoF2 at 6 K is higher then the frequency
$f\textsubscript{ph}$ of the B1g phonon ($2f_{0}>f\textsubscript{ph}$, see
Fig. 1(a)) cottam_spin-phonon_2019 . Applying an external field along the
magnetic easy-axis is known to shift the magnon frequency down, while the
phonon frequency remains unchanged. Particularly, a field of 4 T is expected
to be sufficient to reach the frequency matching condition
$2f\textsubscript{m}=f\textsubscript{ph}$, where signatures of nonlinear
coupling might be satisfied, see Fig. 1(a). We anticipate that in the vicinity
of the resonance, the nonlinear energy flow between spins and lattice reaches
a maximum. Thus, performing THz pump-infrared (IR) probe spectroscopy in
combination with simulations, we reveal the corresponding fingerprints of
coherent energy exchange driven by an impulsive THz stimulus resulting from
the magnon-phonon dynamics in the vicinity of the Fermi-resonance (see Fig.
1(b)). We expect that the impulsive excitation regime is characterized by its
own benchmarks and opens new opportunities to tailor the excitation to
selectively drive specific modes within the material.

Figure 1:  The path to Fermi magnon-phonon resonance. (a) The frequency tuning
by an external magnetic field applied along the antiferromagnetic easy axis of
CoF2. The frequency matching condition is marked by a grey star. The Feynman
diagram illustrates the two magnon - one phonon conversion process. (b) Sketch
of nonlinear magnon-phonon dynamics. For
$2f\textsubscript{m}=f\textsubscript{ph}$, the Fermi magnon-phonon resonance
condition is fulfilled and the the nonlinear energy transfer channel between a
magnon and a phonon opens.

##  2 Results

CoF2 belongs to the class of antiferromagnets with a rutile-type
crystallographic lattice borovik-romanov_piezomagnetism_1959 , described by
the $P4_{2}/mnm$ space group. The multi-atomic primitive cell forms 15 optical
phonon modes balkanski_infrared_1966 ; kroumova_bilbao_2003 : $A_{1g}\oplus
A_{2g}\oplus A_{2u}\oplus B_{1g}\oplus 2B_{1u}\oplus B_{2g}\oplus 3E_{u}\oplus
E_{g}$. The lowest-lying Raman-active phonon mode has B1g symmetry and is
centered at a frequency of $f\textsubscript{ph}\approx 1.96$ THz at T = 6 K.
It is worth noting that the frequency of this mode remains the same in
external magnetic field up to 7 T.

The spins of Co2+ ions are aligned along the crystallographic c-axis below the
Néel temperature of TN = 39 K. In our experiment, we use a 500 um-thick single
crystal CoF2 plate cut perpendicular to the c-axis. If no magnetic field is
applied, there is a doubly degenerate antiferromagnetic resonance mode at the
frequency $f_{0}=1.14$ THz (at 6 K). Applying an external magnetic field, one
breaks the degeneracy of the respective magnon mode. For instance, if a
magnetic field is applied along the c-axis with its amplitude below the spin-
flop field threshold carretero-gonzalez_regular_1994 , the frequencies of two
degenerate modes obey the relation $f\textsubscript{m}=f_{0}\pm\gamma
H\textsubscript{ext}$, where $\gamma$ is the gyromagnetic ratio.

Using the intense, spectrally dense superradiant THz source TELBE located at
Helmholtz-Zentrum Dresden-Rossendorf green_high-field_2016  in combination
with external magnetic fields $H\textsubscript{ext}$, we get the unique
opportunity to pump the magnon selectively while controlling its center
frequency $f\textsubscript{m}$ by tuning $H\textsubscript{ext}$ at our
disposal. This configuration allows one to explore the spin-lattice
interaction in the vicinity of the Fermi resonance
$2f\textsubscript{m}=f\textsubscript{ph}$ by monitoring the phonon response.
As it has been shown earlier mashkovich_terahertz_2021 , as long as both the
magnon and the phonon maintain their coherence, they will induce transient
optical anisotropy in the originally isotropic (ab) plane of the
antiferromagnet and thus modulate various components of the dielectric
permittivity metzger_effect_2022 , which we track by changes of the probe-
polarization mashkovich_terahertz_2021 . The experimental geometry and the THz
pulse characteristics are provided in the Supplemental material. The THz-
induced rotation of the probe-polarization measured in external magnetic
fields up to 7 T (Fig. 2(a)) reveals the two distinct modes in the time domain
data oscillating at frequencies $f\textsubscript{m}$ and
$f\textsubscript{ph}$, respectively. The magnon amplitude dominates up to 70
ps whereas the 70 - 120 ps range is clearly dominated by the B1g phonon at
1.96 THz. In the Supplemental material we have zoomed into the region of 50 -
80 ps, where the magnon is fading, while the high frequency phonon
oscillations are being revealed.

Figure 2:  (a) Time domain transients for the magnetic field applied along the
c-axis and corresponding Fourier spectra in the vicinity of the magnon (b) and
the phonon (c) resonance frequencies (T = 6 K). The vertical line in panel (b)
corresponds to half of the phonon frequency in panel (c). Two different time
windows are used to highlight the magnon (-10 - 70 ps) and the phonon spectral
peaks (70 - 120 ps). The phonon amplitude is scaled by a factor of 20 compared
to the magnon amplitude. The plots for various external magnetic fields are
offset by an equidistant constant. The shaded area correspond to the
integrated region of interest for extraction of the spectral weight in Fig. 3
(a).

Performing a Fourier transformation (FFT) of the data presented in Fig. 2 (a),
we plot two spectral regions in Fig. 2 (b,c) covering the magnon and the
phonon modes, respectively. Evidently, the external magnetic field drastically
changes the dynamics. Away from the Fermi-resonance at 0 T and 7 T the THz-
pump pulse induced rotation shows the forced magnetic response closely
following the magnetic field of the THz pulse (see Supplemental material) and
no phonon induced dynamics is observed. Closer to the resonance for the in-
between magnetic fields of 2 - 5 T, we observe the low energy magnon branch
$f\textsubscript{m}$ with its frequency linearly decreasing with external
magnetic field. It is remarkable that the phonon spectral amplitude shows a
strong field-dependence, which does not have to correlate directly with the
magnon spectral amplitude. Moreover, the phonon spectral amplitude does not
exhibit a single but a double resonance located at 3 T and 4 T. Integrating
the area under the phonon spectra for different external magnetic fields
$H\textsubscript{ext}$ we retrieve the behaviour of the phonon resonance curve
shown in Fig. 3 (a). Contrary to the continuous wave regime where Fermi
resonance implies a settled energy redistribution between modes of phononic or
magnonic origin barsukov_giant_2019 ; katsnelson_resonance_1987 , the
impulsive regime is represented by a coherent energy exchange that modulates
the mode amplitudes over time. Hence, in the scenario where the phonon
lifetime surpasses that of the magnon (as is the case here), the energy of the
phonon following the fading of the magnon will be determined by this intricate
dynamics.

Figure 3:  Fingerprints of the impulsive Fermi magnon-phonon resonance. (a)
Phonon weight for the experiment (purple spheres) and simulation (purple line)
for the nonlinear coupling constant $\alpha=6$. The error bars are extracted
as mean value of the FFT shown in Fig. 2 (b) between 2 and 3 THz. (b)
Simulated phonon spectral weight for various values of $\alpha$. Simulations
are plotted according to equations in Appendix.

In order to obtain a better insight on the observed fingerprints of the
magnon-phonon Fermi resonance, we undertook a simulation of the signatures of
nonlinearly coupled dynamics in CoF2. Conventionally, antiferromagnetic spins
are described in terms of the Néel vector
$\mathbf{L}=\mathbf{M}_{1}-\mathbf{M}_{2}$, where the net magnetic moments
$\mathbf{M}_{1,2}$ are formed by Co2+ ions at the center and the corners of
the unit cell, correspondingly erickson_neutron_1953 . The movement of the B1g
phonon is characterized by the phonon coordinate $\theta\textsubscript{ph}$.
The perturbations from the ground state are introduced as
$\mathbf{L}(t)=(l_{x}(t),l_{y}(t),L_{0})$. Here, L0 describes the ground state
Néel vector. The rule of Fermi resonance symmetry implies that the B1g phonon
symmetry ($x^{2}-y^{2}$) should follow the symmetry of the double magnon
excitation. Hence, the corresponding nonlinear term can be introduced in the
Lagrangian as $\Phi=-\alpha(l_{x}^{2}-l_{y}^{2})\theta_{ph}$
breitenberger_elastic_1981 , where $\alpha$ represents the nonlinear coupling
constant between the magnon and the phonon subsystems. We assume that the
magnetic field of the THz-pulse $\bm{h}\textsubscript{THz}=(h_{x},h_{y},0)$ is
polarized exclusively in the sample plane and solve the Lagrange-Euler
equations, taking into accounting circularly polarized magnon states
$l_{\pm}=l_{x}\pm il_{y}$. The resulting coupled equations can be written as

$\frac{d^{2}l_{+}}{dt^{2}}+2\zeta_{\textsubscript{m}}\frac{dl_{+}}{dt}+\left(\omega_{0}^{2}-\gamma^{2}H_{\textsubscript{ext}}^{2}\right)l_{+}+2\gamma
iH_{\textsubscript{ext}}\frac{dl_{+}}{dt}=-2\alpha\theta_{\textsubscript{ph}}l_{-}+\gamma\frac{d}{dt}(h_{y}-ih_{x}),$
(1)
$\frac{d^{2}l_{-}}{dt^{2}}+2\zeta_{\textsubscript{m}}\frac{dl_{-}}{dt}+\left(\omega_{0}^{2}-\gamma^{2}H_{\textsubscript{ext}}^{2}\right)l_{-}-2\gamma
iH_{\textsubscript{ext}}\frac{dl_{-}}{dt}=2\alpha\theta_{\textsubscript{ph}}l_{+}+\gamma\frac{d}{dt}(h_{y}+ih_{x}),$
(2)
$\frac{d^{2}\theta_{ph}}{dt^{2}}+2\zeta_{\textsubscript{ph}}\frac{d\theta_{ph}}{dt}+\omega_{\textsubscript{ph}}^{2}\theta_{\textsubscript{ph}}=-\alpha\left(l_{+}^{2}+l_{-}^{2}\right),$
(3)

where $\omega_{i}=2\pi f_{i}$ and the Gilbert damping factors $\zeta_{i}$ with
$i$ = ”m” or $i$ = ”ph” account for the magnon or the phonon subsystem,
correspondingly. The second term on the right-hand side of Eqs. 1-2 represents
the linear Zeeman torque satoh_spin_2010 ; baryakhtar_dynamics_1985 , while
the nonlinear coupling can be introduced in Eqs. 1-3 as the derivative of
$\Phi$ on the corresponding order parameter. These terms represent the mutual
nonlinear perturbation of the magnon (phonon) subsystem by the phonon (magnon)
subsystem. The THz pulse waveform measured by electro-optical sampling (EOS)
is introduced as the driving force in our simulation (see Supplemental
material).

Following a similar analysis as for the experimental data, we consider two
time windows, where either the magnon or the phonon are dominating the
dynamics. Performing the Fourier transform and extracting the phonon spectral
weight (see the Supplemental material), we plot the phonon weight vs the
externally applied magnetic field in Fig. 3(b). The THz field corresponds to
the experiment and is of the order of 100 kV/cm. Accounting for g-factor of
2.6 strempfer_magnetic_2004  the gyromagnetic ratio is $\gamma$ = 36.4
GHz/Tesla. The Gilbert damping factors $\zeta\textsubscript{m}/2\pi$ = 10 GHz
and $\zeta\textsubscript{ph}/2\pi$ = 5 GHz were estimated from fitting the
magnon and the phonon experimental spectra (see Supplemental material).

Hence, the only unidentified variable $\alpha$ serves as the only fitting
parameter. Our simulations show that for values of $\alpha<3$, the phonon
amplitude exhibits only a single resonance peak. This demonstrates that the
THz pulse energy is flowing into the magnon succeeded by a transfer into the
phonon subsystem. For $\alpha\geq 3$, a pronounced dip in the phonon spectral
amplitude in the vicinity of the Fermi-resonance emerges, evidencing that
nonlinear terms in Eqs. 1-2 are now affecting the dynamics. This in fact means
that anharmonicty of the phonon starts to play a role and the energy is
flowing back from the phonon into the magnon subsystem. However, at higher
values of $\alpha>>8$, the nonlinearity becomes strong and perturbs the form
of the phonon spectrum.

##  3 Discussion

Despite the fact that nonlinear phononics is a relatively young field, Fermi
resonance of the lattice has been studied theoretically earlier
katsnelson_resonance_1987 ; gornostyrev_phase_1995 ;
gornostyrev_stochastic_1999 ; katsnelson_fermi-resonance-like_2002 ;
katsnelson_nonperturbative_2004 . In particular, it was shown that the
resonances can result either in a splitting or broadening of phonon lines in
the vibrational spectra of the lattice. Moreover, according to Ref.
barsukov_giant_2019 , in the vicinity of the resonance, nonlinear damping
plays a significant role and in principle can modify a single bell-shaped
resonance phonon curve.

A double or Fermi resonance between otherwise non-interacting phononic modes
can be practically regarded as evidence of energy exchange between these
modes. In strong contrast to the previous theoretical studies focused on
incoherent lattice dynamics and thus revealing stochastic acts of energy
exchange between the modes gornostyrev_stochastic_1999 ; barsukov_giant_2019 ,
our experiments reveal the manifestation of the Fermi resonances for the case
of coherent impulsive dynamics. Thus, using the methods of coherent control
one can, in general, steer the energy flow between spins and lattice on
demand. Moreover, deriving the magnon-phonon quasi-particles conservation law
analogous to the Manley-Rowe relations for second harmonic generation
manley_general_1956  is an intriguing aspect.

##  4 Conclusion

In summary, we reported about a new regime of magnon-phonon dynamics which
facilitates an efficient energy exchange between the magnons and phonons
fulfilling the conditions of Fermi resonance. We revealed the fingerprints of
Fermi magnon-phonon resonance and showed, in particular, a distinct double-
resonance feature of the phonon amplitude upon sweeping the frequency of the
magnon resonance by applying an external magnetic field indicating the effect
of a mutual, anharmonic energy transfer. We believe that our finding is not
only a new phenomenon in nonlinear phononics, but also an important milestone
in the broad fields of magnonics and spintronics, where coherent energy
control plays the central role. Acknowledgments This work was supported by the
Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - Project
number 277146847 - CRC 1238, de Nederlandse Organisatie voor Wetenschappelijk
Onderzoek (NWO), the European Union’s Horizon 2020 research and innovation
program under the Marie Skłodowska-Curie grant agreement No 861300 (COMRAD),
the European Research Council ERC Grant Agreement No.101054664 273
(SPARTACUS), and RSF (grant No. 21-42-00035). MIK and AVK acknowledge the
research program ”Materials for the Quantum Age” (QuMat) for financial
support. This program (registration number 024.005.006) is part of the
Gravitation program financed by the Dutch Ministry of Education, Culture and
Science (OCW). RMD acknowledges support of RSF (Grant No. 22-72-00025) and the
Ministry of Science and Higher Education of the Russian Federation
(FSWR-2021-011). Parts of this research were carried out at ELBE at the
Helmholtz-Zentrum Dresden - Rossendorf e. V., a member of the Helmholtz
Association. The TELBE measurements were conduced February 11-15, 2022
proposal number 21202639-ST and October 23-25, 2021 proposal number
21102489-ST. We thank Dr. Clément Faugeras for discussions. The authors
declare that this work has been published as a result of peer-to-peer
scientific collaboration between researchers. The provided affiliations
represent the actual addresses of the authors in agreement with their digital
identifier (ORCID) and cannot be considered as a formal collaboration between
the aforementioned institutions.

## References

  *   * (1) Maehrlein, S., Paarmann, A., Wolf, M., Kampfrath, T.: Terahertz Sum-Frequency Excitation of a Raman-Active Phonon. Physical Review Letters 119(12), 127402 (2017)
  * (2) Först, M., Manzoni, C., Kaiser, S., Tomioka, Y., Tokura, Y., Merlin, R., Cavalleri, A.: Nonlinear phononics as an ultrafast route to lattice control. Nature Physics 7(11), 854–856 (2011)
  * (3) Disa, A.S., Nova, T.F., Cavalleri, A.: Engineering crystal structures with light. Nature Physics 17(10), 1087–1092 (2021)
  * (4) Zhang, Z., Gao, F.Y., Curtis, J.B., Liu, Z.-J., Chien, Y.-C., von Hoegen, A., Kurihara, T., Suemoto, T., Narang, P., Baldini, E., Nelson, K.A.: Three-wave mixing of anharmonically coupled magnons. arXiv. arXiv:2301.12555 [cond-mat, physics:physics] (2023)
  * (5) Blank, T.G.H., Grishunin, K.A., Ivanov, B.A., Mashkovich, E.A., Afanasiev, D., Kimel, A.V.: Empowering Control of Antiferromagnets by THz-induced Spin Coherence. arXiv. arXiv:2212.09532 [cond-mat] (2023)
  * (6) Fermi, E.: Ueber den Ramaneffekt des Kohlendioxyds. Zeitschrift fuer Physik 71(3-4), 250–259 (1931)
  * (7) Katsnelson, M.I., Trefilov, A.V.: Resonance phenomena in a phonon subsystem in connection with anomalies of the structural state of metals. JETP Letters 45(10), 634–638 (1987)
  * (8) Gornostyrev, Y.N., Katsnelson, M.I., Platonov, A.P., Trefilov, A.V.: Phase locking in a thermostat: Fermi resonance in metals. Physical Review B 51(18), 12817–12820 (1995)
  * (9) Gornostyrev, Y.N., Zhdakhin, D.I., Katsnelson, M.I., Trefilov, A.V.: Stochastic resonance between limit cycles. Spring pendulum in a thermostat. Journal of Experimental and Theoretical Physics Letters 69(8), 630–635 (1999)
  * (10) Katsnelson, M.I., Stroev, A.Y., Trefilov, A.V.: Fermi-resonance-like phenomena in lattice dynamics of fcc La. Physical Review B 66(9), 092303 (2002)
  * (11) Katsnelson, M.I., Trefilov, A.V.: Nonperturbative anharmonic phenomena in crystal lattice dynamics. In: AIP Conference Proceedings, vol. 708, pp. 727–730. AIP, Sendai (Japan) (2004)
  * (12) Barsukov, I., Lee, H.K., Jara, A.A., Chen, Y.-J., Gonçalves, A.M., Sha, C., Katine, J.A., Arias, R.E., Ivanov, B.A., Krivorotov, I.N.: Giant nonlinear damping in nanoscale ferromagnets. Science Advances 5(10), 6943 (2019)
  * (13) Shoshani, O., Shaw, S.W., Dykman, M.I.: Anomalous Decay of Nanomechanical Modes Going Through Nonlinear Resonance. Scientific Reports 7(1), 18091 (2017)
  * (14) Mashkovich, E.A., Grishunin, K.A., Dubrovin, R.M., Zvezdin, A.K., Pisarev, R.V., Kimel, A.V.: Terahertz light–driven coupling of antiferromagnetic spins to lattice. Science 374(6575), 1608–1611 (2021)
  * (15) Cottam, M.G., Lockwood, D.J.: Spin-phonon interaction in transition-metal difluoride antiferromagnets: Theory and experiment. Low Temperature Physics 45(1), 78–91 (2019)
  * (16) Borovik-Romanov, A.: Piezomagnetism in the antiferromagnetic fluorides of cobalt and manganese. Soviet Physics JETP-USSR 9.6, 1390–1391 (1959)
  * (17) Balkanski, M., Moch, P., Parisot, G.: Infrared Lattice‐Vibration Spectra in NiF${}_{\textrm{2}}$ , CoF${}_{\textrm{2}}$ , and FeF${}_{\textrm{2}}$. The Journal of Chemical Physics 44(3), 940–944 (1966)
  * (18) Kroumova, E., Aroyo, M.I., Perez-Mato, J.M., Kirov, A., Capillas, C., Ivantchev, S., Wondratschek, H.: Bilbao Crystallographic Server : Useful Databases and Tools for Phase-Transition Studies. Phase Transitions 76(1-2), 155–170 (2003)
  * (19) Carretero-Gonzalez, R., Nunez-Yepez, H.N., Salas-Brito, A.L.: Regular and chaotic behaviour in an extensible pendulum. European Journal of Physics 15(3), 139–148 (1994)
  * (20) Green, B., Kovalev, S., Asgekar, V., Geloni, G., Lehnert, U., Golz, T., Kuntzsch, M., Bauer, C., Hauser, J., Voigtlaender, J., Wustmann, B., Koesterke, I., Schwarz, M., Freitag, M., Arnold, A., Teichert, J., Justus, M., Seidel, W., Ilgner, C., Awari, N., Nicoletti, D., Kaiser, S., Laplace, Y., Rajasekaran, S., Zhang, L., Winnerl, S., Schneider, H., Schay, G., Lorincz, I., Rauscher, A.A., Radu, I., Mährlein, S., Kim, T.H., Lee, J.S., Kampfrath, T., Wall, S., Heberle, J., Malnasi-Csizmadia, A., Steiger, A., Müller, A.S., Helm, M., Schramm, U., Cowan, T., Michel, P., Cavalleri, A., Fisher, A.S., Stojanovic, N., Gensch, M.: High-Field High-Repetition-Rate Sources for the Coherent THz Control of Matter. Scientific Reports 6(1), 22256 (2016)
  * (21) Metzger, T.W.J., Grishunin, K.A., Afanasiev, D., Dubrovin, R.M., Mashkovich, E.A., Pisarev, R.V., Kimel, A.V.: Effect of antiferromagnetic order on a propagating single-cycle THz pulse. Applied Physics Letters 121(25), 252403 (2022)
  * (22) Erickson, R.A.: Neutron Diffraction Studies of Antiferromagnetism in Manganous Fluoride and Some Isomorphous Compounds. Physical Review 90(5), 779–785 (1953)
  * (23) Breitenberger, E., Mueller, R.D.: The elastic pendulum: A nonlinear paradigm. Journal of Mathematical Physics 22(6), 1196–1210 (1981)
  * (24) Satoh, T., Cho, S.-J., Iida, R., Shimura, T., Kuroda, K., Ueda, H., Ueda, Y., Ivanov, B.A., Nori, F., Fiebig, M.: Spin Oscillations in Antiferromagnetic NiO Triggered by Circularly Polarized Light. Physical Review Letters 105(7), 077402 (2010)
  * (25) Baryakhtar, V.G., Ivanov, B.A., Chetkin, M.V.: Dynamics of domain walls in weak ferromagnets. Soviet Physics Uspekhi 28(7), 563 (1985)
  * (26) Strempfer, J., Rütt, U., Bayrakci, S.P., Brückel, T., Jauch, W.: Magnetic properties of transition metal fluorides M F 2 ( M = Mn , Fe, Co, Ni) via high-energy photon diffraction. Physical Review B 69(1), 014417 (2004)
  * (27) Manley, J., Rowe, H.: Some General Properties of Nonlinear Elements-Part I. General Energy Relations. Proceedings of the IRE 44(7), 904–913 (1956)