# Fourier characterizations and non-triviality of Gelfand-Shilov spaces, with
applications to Toeplitz operators

Albin Petersson  Department of Mathematics, Linnæus University, Växjö, Sweden
<EMAIL_ADDRESS>
###### Abstract.

We examine properties of Gelfand-Shilov spaces $S_{s}$, $S^{\sigma}$,
$S_{s}^{\sigma}$, $\Sigma_{s}$, $\Sigma^{\sigma}$ and $\Sigma_{s}^{\sigma}$.
These are spaces of smooth functions where the functions or their Fourier
transforms admit sub-exponential decay. It is determined that
$\Sigma_{s}^{\sigma}$ is nontrivial if and only if $s+\sigma>1$. We find
growth estimates on functions and their Fourier transforms in the one-
parameter spaces, and we obtain characterizations in terms of estimates of
short-time Fourier transforms for these spaces and their duals. Additionally,
we determine conditions on the symbols of Toeplitz operators under which the
operators are continuous on one-parameter spaces.

##  1\. Introduction

The Gelfand-Shilov spaces were first introduced as a useful set of functions
for the study of Cauchy problems in partial differential equations. These
functions are convenient in this setting because of their smoothness, and
because of the conditions of regularity imposed on them. For instance, some
partial differential equations are ill-posed in the Schwartz space
$\mathscr{S}$ or its dual $\mathscr{S}^{\prime}$, the space of tempered
distributions (see e.g. [1, p. 160-163] for notation), but are well-posed in
suitable Gelfand-Shilov spaces. One such example is the Euler-Tricomi equation
$D_{t}^{2}f+tD_{x}^{2}f=0$, and another example may be found in [2]. The fact
that some partial differential equations are only well-posed in Gelfand-Shilov
spaces exemplifies the need to determine properties of functions in those
spaces. Gelfand-Shilov spaces can also be useful in the study of pseudo-
differential operators [3], which in turn have uses in, for instance, quantum
theory [4] and signal processing [5].

The Gelfand-Shilov spaces $S_{s}^{\sigma}$, $S_{s}$ and $S^{\sigma}$ of
Roumieu type (cf. [6, 7, 8]) and $\Sigma_{s}^{\sigma}$, $\Sigma_{s}$ and
$\Sigma^{\sigma}$ of Beurling type (cf. [9]) can be considered as refinements
of the Schwartz space $\mathscr{S}$, where we impose analyticity-like
smoothness conditions. The strength of these conditions depend on the
parameters $s$ and $\sigma$. The smaller $s$ is the faster the functions must
vanish at infinity, and smaller $\sigma$ impose stronger conditions on the
growth of the derivatives (meaning the Fourier transform vanishes faster). In
the one-parameter spaces, functions have sub-exponential decay and their
Fourier transforms tend to zero faster than the reciprocal of any polynomial,
or vice versa. In the two-parameter spaces, both the functions and their
Fourier transforms have sub-exponential decay. If $s$ and $\sigma$ are
sufficiently small, the only function found in $S_{s}^{\sigma}$ or
$\Sigma_{s}^{\sigma}$ is $f(x)\equiv 0$, and the spaces are considered
trivial. There are more general Gelfand-Shilov spaces, such as the
$S_{M_{p}}^{N_{p}}$-spaces whose properties are explored in [8], for instance.

In this paper, we are mostly interested in discussing the properties of the
one-parameter spaces $S_{s}$ and $S^{\sigma}$, their duals $(S_{s})^{\prime}$
and $(S^{\sigma})^{\prime}$, as well as the corresponding spaces $\Sigma_{s}$
and $\Sigma^{\sigma}$ and their duals. More specifically, we establish growth
estimates on elements in these spaces and their Fourier transforms.
Additionally, we find estimates involving the short-time Fourier transform
which provides an alternate characterization of Gelfand-Shilov spaces. Such
estimates exist for the two-parameter spaces (cf. [10]) and here we extend
characterizations of this type to one-parameter spaces. We find that the
short-time Fourier transform admits sub-exponential decay in one parameter,
and tends to zero faster than reciprocals of polynomials in the other.
Corresponding estimates are found for the duals of one-parameter spaces as
well. We also examine Toeplitz operators on these one-parameter spaces, where
the symbol $a(x,\xi)$ of the operator lies in different one-parameter spaces
in each variable. We find conditions such that the Toeplitz operator is
continuous on $S_{s}$, $S^{\sigma}$ and their respective duals.

We also determine when the two-parameter spaces are nontrivial. These results
are well-known for $S_{s}^{\sigma}$-spaces, but for the
$\Sigma_{s}^{\sigma}$-spaces, we find that the space is nontrivial if and only
if $s+\sigma>1$, as opposed to the condition $s+\sigma\geq 1$,
$(s,\sigma)\neq(\frac{1}{2},\frac{1}{2})$ often cited in other works (cf.
[11]). This result, which was initially suggested by Andreas Debrouwere,
directly contradicts versions of this result in previous works.

The paper is structured as follows. In Section 2, we introduce notations,
definitions and preliminary propositions regarding Gelfand-Shilov spaces
necessary to obtain results in subsequent sections. These preliminary results
can either be found in [8, 6, 7] or are simple enough to be left as an
exercise for the reader. In Section 3, we determine for which $s$ and $\sigma$
the space $\Sigma_{s}^{\sigma}$ is nontrivial. In Section 4, we obtain growth
estimates for the short-time Fourier transform of functions in $S_{s}$,
$S^{\sigma}$ and $\Sigma_{s}$, $\Sigma^{\sigma}$. In Section 5, we show how
these results can be used to characterize the duals of these one-parameter
spaces via the short-time Fourier transform as well. Lastly, in Section 6, we
find conditions on the symbol of Toeplitz operators so that the operator is
continuous on one-parameter spaces and their duals.

##  2\. Preliminaries

We begin by defining the spaces we will devote the most attention to in this
paper. These are the so-called Gelfand-Shilov spaces. There is a clear and
intuitive correspondence between the spaces of Roumieu type and those of
Beurling type and the order in which the definitions are listed is meant to
highlight this correspondence.

######  Definition 2.1.

Suppose $s,\sigma>0$.

  1. (i)

$S_{s}(\mathbb{R}^{n})$ consists of all $f\in C^{\infty}(\mathbb{R}^{n})$ for
which there is an $h>0$ such that

$\sup_{x\in\mathbb{R}^{n}}|x^{\alpha}D^{\beta}f(x)|\leq
C_{\beta}h^{|\alpha|}\alpha!^{s},\quad\forall\alpha,\beta\in\mathbb{N}^{n},$
(2.1)

where $C_{\beta}$ is a constant depending only on $\beta$.

  2. (ii)

$\Sigma_{s}(\mathbb{R}^{n})$ consists of all $f\in C^{\infty}(\mathbb{R}^{n})$
such that (2.1) holds for every $h>0$, where $C_{\beta}=C_{h,\beta}$ depends
on $h$ and $\beta$.

  3. (iii)

$S^{\sigma}(\mathbb{R}^{n})$ consists of all $f\in C^{\infty}(\mathbb{R}^{n})$
for which

$\sup_{x\in\mathbb{R}^{n}}|x^{\alpha}D^{\beta}f(x)|\leq
C_{\alpha}h^{|\beta|}\beta!^{\sigma},\quad\forall\alpha,\beta\in\mathbb{N}^{n},$
(2.2)

holds for some $h>0$, where $C_{\alpha}$ is a constant depending only on
$\alpha$.

  4. (iv)

$\Sigma^{\sigma}(\mathbb{R}^{n})$ consists of all $f\in
C^{\infty}(\mathbb{R}^{n})$ such that (2.2) holds for every $h>0$, where
$C_{\alpha}=C_{h,\alpha}$ depends on $h$ and $\alpha$.

  5. (v)

$S_{s}^{\sigma}(\mathbb{R}^{n})$ consists of all $f\in
C^{\infty}(\mathbb{R}^{n})$ for which there are constants $h>0$ and $C>0$ such
that

$\sup_{x\in\mathbb{R}^{n}}|x^{\alpha}D^{\beta}f(x)|\leq
Ch^{|\alpha+\beta|}\alpha!^{s}\beta!^{\sigma},\quad\forall\alpha,\beta\in\mathbb{N}^{n}.$
(2.3)

  6. (vi)

$\Sigma_{s}^{\sigma}(\mathbb{R}^{n})$ consists of all $f\in
C^{\infty}(\mathbb{R}^{n})$ such that (2.3) holds for all $h>0$, where
$C=C_{h}$ depends only on $h$.

Trivially, we see that $S_{s}^{\sigma}\subseteq S_{s}\cap S^{\sigma}$,
$\Sigma_{s}^{\sigma}\subseteq\Sigma_{s}\cap\Sigma^{\sigma}$, and
$\Sigma_{s}^{\sigma}\subseteq S_{s}^{\sigma}$ for all such $s$ and $\sigma$.
In fact, we have $S_{s}^{\sigma}=S_{s}\cap S^{\sigma}$ (cf. [6]) and
$\Sigma_{s}^{\sigma}=\Sigma_{s}\cap\Sigma^{\sigma}$ (this is a well-known
result that follows by analogous arguments, but an explicit proof can be found
in [12] for instance).

######  Definition 2.2.

Let $\mathscr{F}$ denote the Fourier transform given by

$(\mathscr{F}f)(\xi)=\hat{f}(\xi)=\frac{1}{(2\pi)^{n/2}}\int_{\mathbb{R}^{n}}f(x)e^{-i\langle
x,\xi\rangle}\,dx,$

and let $\mathscr{F}^{-1}$ be the corresponding inverse Fourier transform

$(\mathscr{F}^{-1}f)(x)=\frac{1}{(2\pi)^{n/2}}\int_{\mathbb{R}^{n}}f(\xi)e^{i\langle
x,\xi\rangle}\,d\xi.$

If $f$ is a generalized function, then $\mathscr{F}$ denotes the adjoint
operator of the Fourier transform defined above.

Here we list some basic properties of Gelfand-Shilov spaces in the form of two
propositions. The first proposition establishes sub-exponential decay of
derivatives in Gelfand-Shilov spaces, and the second establishes how Fourier
transforms work in Gelfand-Shilov spaces. For both of the following two
propositions, (a) can be found in [6, 7, 8], and (b) follows by analogous
arguments.

######  Proposition 2.3.

Suppose $s>0$ and $f\in C^{\infty}(\mathbb{R}^{n})$. Then

  1. (a)

$f\in S_{s}(\mathbb{R}^{n})$ if and only if there are constants
$C_{\beta},r>0$ such that

$|D^{\beta}f(x)|\leq C_{\beta}e^{-r|x|^{1/s}}$ (2.4)

for all multi-indices $\beta$;

  2. (b)

$f\in\Sigma_{s}(\mathbb{R}^{n})$ if and only if for every $r>0$

$|D^{\beta}f(x)|\leq C_{r,\beta}e^{-r|x|^{1/s}}$ (2.5)

holds for all multi-indices $\beta$, where $C_{r,\beta}>0$ depends only on $r$
and $\beta$.

######  Proposition 2.4.

Suppose $s,\sigma>0$.

  1. (a)

If $s+\sigma\geq 1$, then $f\in S_{s}^{\sigma}(\mathbb{R}^{n})$ if and only if
$\hat{f}\in S_{\sigma}^{s}(\mathbb{R}^{n})$. Moreover, $f\in
S_{s}(\mathbb{R}^{n})$ if and only if $\hat{f}\in S^{s}(\mathbb{R}^{n})$.

  2. (b)

If $s+\sigma>1$, then $f\in\Sigma_{s}^{\sigma}(\mathbb{R}^{n})$ if and only if
$\hat{f}\in\Sigma_{\sigma}^{s}(\mathbb{R}^{n})$. Moreover,
$f\in\Sigma_{s}(\mathbb{R}^{n})$ if and only if
$\hat{f}\in\Sigma^{s}(\mathbb{R}^{n})$.

We also include the following basic result.

######  Proposition 2.5.

If $s,\sigma>0$, $s<s_{1}$ and $\sigma<\sigma_{1}$, then
$S_{s}^{\sigma}(\mathbb{R}^{n})\subseteq\Sigma_{s_{1}}^{\sigma_{1}}(\mathbb{R}^{n})$.

We will now discuss the topology of Gelfand-Shilov spaces. To do this we need
the following definition.

######  Definition 2.6.

Suppose $V_{j}$, $j=0,1,2,\dots$, are Banach spaces,

$V=\bigcap_{j\geq 0}V_{j}$

and

$W=\bigcup_{j\geq 0}V_{j}.$

  1. (a)

Let $i_{j}:V\rightarrow V_{j}$ be inclusion maps. We say that the _projective
limit_ is the space $V$ with the smallest possible topology such that $i_{j}$
is continuous for all $j$. We write this as

$V=\projlim_{j\geq 0}V_{j}.$

  2. (b)

Suppose further that $V_{j}\hookrightarrow V_{j+1}$, meaning that $V_{j}$ is
continuously embedded in $V_{j+1}$, and let $\tilde{i}_{j}:V_{j}\rightarrow
V_{j+1}$ be inclusion maps. We say that the _inductive limit_ is the space $W$
with the greatest possible topology such that $\tilde{i}_{j}$ is continuous
for all $j$. We write this as

$W=\operatorname*{ind\,lim}_{j\geq 0}V_{j}.$

With these definitions and propositions in mind, we can construct topologies
on $S_{s}$, $S^{\sigma}$, $\Sigma_{s}$ and $\Sigma^{\sigma}$ and define their
duals. For more information on topological vector spaces, see for instance
[13].

######  Definition 2.7.

  1. (1)

Let $V_{s,r,N}(\mathbb{R}^{n})$ consist of all $f\in
C^{\infty}(\mathbb{R}^{n})$ such that

$||f||_{s,r,N}=\sup_{x\in\mathbb{R}^{n},|\alpha|\leq
N}\left|D^{\alpha}f(x)e^{r|x|^{1/s}}\right|<\infty.$

  2. (2)

Let $V_{r,M}^{\sigma}(\mathbb{R}^{n})$ consist of all $f\in
C^{\infty}(\mathbb{R}^{n})$ such that

$||f||_{r,M}^{\sigma}=\sup_{\xi\in\mathbb{R}^{n},|\beta|\leq
M}\left|D^{\beta}\hat{f}(\xi)e^{r|\xi|^{1/\sigma}}\right|<\infty.$

We see that

$S_{s}(\mathbb{R}^{n})=\operatorname*{ind\,lim}_{r>0}\left(\projlim_{N\geq
0}V_{s,r,N}(\mathbb{R}^{n})\right)$

and

$S^{\sigma}(\mathbb{R}^{n})=\operatorname*{ind\,lim}_{r>0}\left(\projlim_{M\geq
0}V_{r,M}^{\sigma}(\mathbb{R}^{n})\right),$

which implies

$S_{s}(\mathbb{R}^{n})=\bigcup_{r>0}\left(\bigcap_{N\geq
0}V_{s,r,N}(\mathbb{R}^{n})\right),\quad
S^{\sigma}(\mathbb{R}^{n})=\bigcup_{r>0}\left(\bigcap_{M\geq
0}V_{r,M}^{\sigma}(\mathbb{R}^{n})\right).$

For the $\Sigma_{s}$\- and $\Sigma^{\sigma}$-spaces, we obtain

$\Sigma_{s}(\mathbb{R}^{n})=\projlim_{r>0}\left(\projlim_{N\geq
0}V_{s,r,N}(\mathbb{R}^{n})\right)$

and

$\Sigma^{\sigma}(\mathbb{R}^{n})=\projlim_{r>0}\left(\projlim_{M\geq
0}V_{r,M}^{\sigma}(\mathbb{R}^{n})\right)$

which implies

$\Sigma_{s}(\mathbb{R}^{n})=\bigcap_{r>0}\left(\bigcap_{N\geq
0}V_{s,r,N}(\mathbb{R}^{n})\right),\quad\Sigma^{\sigma}(\mathbb{R}^{n})=\bigcap_{r>0}\left(\bigcap_{M\geq
0}V_{r,M}^{\sigma}(\mathbb{R}^{n})\right).$

######  Remark 2.8.

While $\Sigma_{s}$ and $\Sigma^{\sigma}$ are Fréchet spaces for all
$s,\sigma>0$, the same is not known to be true for $S_{s}$ and $S^{\sigma}$ in
current literature.

This leads us to define the dual spaces of $S_{s}$ and $S^{\sigma}$ in the
following way.

######  Definition 2.9.

We will denote a functional $f$ of such a dual space being applied to a test
function $\psi$ in the appropriate corresponding space by $f(\psi)=\langle
f,\psi\rangle$.

  1. (i)

We say that $u\in(S_{s})^{\prime}(\mathbb{R}^{n})$ if for every $r>0$ there is
an $N\geq 0$ such that

$|\langle u,f\rangle|\leq C_{N}\sum_{|\alpha|\leq
N}||D^{\alpha}fe^{r|\cdot|^{1/s}}||_{\infty},$

for any $f\in S_{s}(\mathbb{R}^{n})$.

  2. (ii)

We say that $u\in(S^{\sigma})^{\prime}(\mathbb{R}^{n})$ if for every $r>0$
there is an $N\geq 0$ such that

$|\langle u,f\rangle|\leq C_{N}\sum_{|\alpha|\leq
N}||D^{\alpha}\hat{f}e^{r|\cdot|^{1/\sigma}}||_{\infty},$

for any $f\in S^{\sigma}(\mathbb{R}^{n})$.

Similarly, we define the dual spaces of $\Sigma_{s}$ and $\Sigma^{\sigma}$ as
follows.

######  Definition 2.10.

  1. (i)

We say that $u\in(\Sigma_{s})^{\prime}(\mathbb{R}^{n})$ if there is an
$r_{0}>0$ and an $N\geq 0$ such that

$|\langle u,f\rangle|\leq C\sum_{|\alpha|\leq
N}||D^{\alpha}fe^{r_{0}|\cdot|^{1/s}}||_{\infty},$

for any $f\in\Sigma_{s}(\mathbb{R}^{n})$.

  2. (ii)

We say that $u\in(\Sigma^{\sigma})^{\prime}(\mathbb{R}^{n})$ if there is an
$r_{0}>0$ and an $N\geq 0$ such that

$|\langle u,f\rangle|\leq C_{N}\sum_{|\alpha|\leq
N}||D^{\alpha}\hat{f}e^{r_{0}|\cdot|^{1/\sigma}}||_{\infty},$

for any $f\in\Sigma^{\sigma}(\mathbb{R}^{n})$.

######  Remark 2.11.

Since $S_{0}(\mathbb{R}^{n})=C_{c}^{\infty}(\mathbb{R}^{n})$, the space of
compactly supported smooth functions, (cf. [7, p. 170]) we have
$(S_{0})^{\prime}(\mathbb{R}^{n})=\mathscr{D}^{\prime}(\mathbb{R}^{n})$. Since
$S_{s}(\mathbb{R}^{n})$ is continuously embedded and dense in
$S_{s^{\prime}}(\mathbb{R}^{n})$ for $s\leq s^{\prime}$, we thus have

$(S_{s})^{\prime}(\mathbb{R}^{n})\subseteq\mathscr{D}^{\prime}(\mathbb{R}^{n})$

for all positive $s$. By Proposition 2.4, we therefore have

$(S^{\sigma})^{\prime}(\mathbb{R}^{n})\subseteq\mathscr{F}\mathscr{D}^{\prime}(\mathbb{R}^{n})$

for all positive $\sigma$.

In Definitions 2.9 and 2.10, we can replace the
$L^{\infty}(\mathbb{R}^{n})$-norm with the $L^{2}(\mathbb{R}^{n})$-norm by the
arguments of [6, p. 134]. We can extend this further with Hölder’s inequality
to obtain the following equivalent definitions of the dual spaces.

######  Proposition 2.12.

Suppose $1\leq p\leq\infty$.

  1. (i)

$u\in(S_{s})^{\prime}(\mathbb{R}^{n})$ if and only if for every $r>0$ there is
an $N\geq 0$ such that

$|\langle u,f\rangle|\leq C_{N}\sum_{|\alpha|\leq
N}||D^{\alpha}fe^{r|\cdot|^{1/s}}||_{p},$

for any $f\in S_{s}(\mathbb{R}^{n})$.

  2. (ii)

$u\in(S^{\sigma})^{\prime}(\mathbb{R}^{n})$ if and only if for every $r>0$
there is an $N\geq 0$ such that

$|\langle u,f\rangle|\leq C_{N}\sum_{|\alpha|\leq
N}||D^{\alpha}\hat{f}e^{r|\cdot|^{1/\sigma}}||_{p},$

for any $f\in S^{\sigma}(\mathbb{R}^{n})$.

Replacing the $L^{\infty}$-norm with $L^{p}$-norms, $1\leq p<\infty$, is
possible for the $\Sigma_{s}$ and $\Sigma^{\sigma}$ duals by similar
arguments.

For $f\in(S_{s})^{\prime}$ or $f\in(S^{\sigma})^{\prime}$ we will also
consider $(f,\psi)$, which we denote to mean the continuous extension of the
regular inner product of $L^{2}$ given by

$(f,\psi)_{2}=\int_{\mathbb{R}^{n}}f(y)\overline{\phi(y)}\,dy$

to $f\in(S_{s})^{\prime}$, $\psi\in S_{s}$ or $f\in(S^{\sigma})^{\prime}$,
$\psi\in S^{\sigma}$. The fact that this inner product can be extended
continuously to duals of Gelfand-Shilov spaces follows by the fact that
$(S_{s},L^{2},(S_{s})^{\prime})$ forms a _Gelfand triple_ (cf. [14]). The same
is true of $S^{\sigma}$, $\Sigma_{s}$ and $\Sigma^{\sigma}$.

We also recall the following definition of the short-time Fourier transform,
which serves a pivotal role in several of the characterizations in this paper.

######  Definition 2.13.

The _short-time Fourier transform_ of $f\in(S_{s})^{\prime}(\mathbb{R}^{n})$
with window function $\phi\in S_{s}(\mathbb{R}^{n})$ is given by

$V_{\phi}f(x,\xi)=\mathscr{F}\Big{[}f\overline{\phi(\cdot-x)}\Big{]}(\xi)=(2\pi)^{-n/2}(f,\phi(\cdot-x)e^{i\langle\cdot,\xi\rangle}).$

For $f$ belonging to $(S^{\sigma})^{\prime}(\mathbb{R}^{n})$,
$(\Sigma_{s})^{\prime}(\mathbb{R}^{n})$ or
$(\Sigma^{\sigma})^{\prime}(\mathbb{R}^{n})$, we define the short-time Fourier
transform by replacing each occurrence of $S_{s}$ above with $S^{\sigma}$,
$\Sigma_{s}$ and $\Sigma^{\sigma}$, respectively.

##  3\. Non-triviality of $\Sigma_{s}^{\sigma}$-spaces

In this section, we determine when $\Sigma_{s}^{\sigma}$-spaces are
nontrivial. Similar results have already been established for
$S_{s}^{\sigma}$-spaces (cf. [7]). By nontrivial we mean that the space
contains a function which is not constantly equal to zero. To establish non-
triviality conditions, we will need two propositions.

The following proposition follows by similar arguments to those in [7, p.
172-175].

######  Proposition 3.1.

If $s,\sigma>0$, $\sigma<1$ and $f\in\Sigma_{s}^{\sigma}(\mathbb{R}^{n})$,
then $f$ can be continued analytically as an entire function in the
($2n$-dimensional) complex plane. Moreover, for every $a,b>0$,

$|f(x+iy)|\leq C\exp{\left(-a|x|^{1/s}+b|y|^{1/(1-\sigma)}\right)}$

for some constant $C=C_{a,b}$.

We also find the following result in [7, p. 228-233].

######  Proposition 3.2.

For positive $s$ and $\sigma$, the space $S_{s}^{\sigma}(\mathbb{R}^{n})$ is
nontrivial if and only if $s+\sigma\geq 1$.

With these propositions in mind, we prove the main result of this section. In
previous works the condition “$s+\sigma\geq 1$, $(s,\sigma)\neq(1/2,1/2)$” is
employed instead of “$s+\sigma>1$”. Here, we prove that the correct condition
is $s+\sigma>1$.

######  Theorem 3.3.

Suppose $s,\sigma>0$. Then the space $\Sigma_{s}^{\sigma}(\mathbb{R}^{n})$ is
nontrivial if and only if $s+\sigma>1$.

###### Proof.

Since $\Sigma_{s}^{\sigma}\subseteq S_{s}^{\sigma}$, it follows by Proposition
3.2 that $\Sigma_{s}^{\sigma}$ is trivial whenever $s+\sigma<1$. Furthermore,
Proposition 3.2 with Proposition 2.5 implies that $\Sigma_{s}^{\sigma}$ is
nontrivial when $s+\sigma>1$. Thus we need only consider the case
$s+\sigma=1$. Since $s$ and $\sigma$ are both assumed to be positive, we must
have $\sigma<1$. By Proposition 3.1, it is then true that

$|f(z)|\leq C_{a,b}\exp{\left(-a|x|^{1/s}+b|y|^{1/s}\right)}$

for every $a,b>0$, where $z=x+iy$. Moreover

$|f(iz)|\leq C_{a,b}\exp{\left(-a|y|^{1/s}+b|x|^{1/s}\right)}$

and therefore

$|f(z)\cdot f(iz)|\leq
C_{a,b}^{2}\exp{\left((b-a)(|x|^{1/s}+|y|^{1/s})\right)}.$

Since this inequality holds for all $a,b>0$, then by picking $a>b$ we see that
$g(z)=f(z)\cdot f(iz)$ is bounded and tends to zero as
$|x|,|y|\rightarrow\infty$. By Proposition 3.1, $f$ is an entire function,
thus so is $g$. Hence Liouville’s theorem implies that $g\equiv 0$. But this
implies that $f\equiv 0$ as well, completing the proof. ∎

##  4\. Characterizations by short-time Fourier transform

We now move on to the characterization of $S_{s}$\- and $S^{\sigma}$-spaces in
terms of their short-time Fourier transforms. This is detailed in the
following theorem, which is the main result of this section.

######  Theorem 4.1.

Suppose $s,\sigma>0$.

  1. (i)

Let $\phi\in S_{s}(\mathbb{R}^{n})\setminus\\{0\\}$. Then $f\in
S_{s}(\mathbb{R}^{n})$ if and only if there is an $r>0$ such that

$|V_{\phi}f(x,\xi)|\leq C_{N}(1+|\xi|^{2})^{-N}e^{-r|x|^{1/s}}$ (4.1)

for every $N\geq 0$.

  2. (ii)

Let $\phi\in S^{\sigma}(\mathbb{R}^{n})\setminus\\{0\\}$. Then $f\in
S^{\sigma}(\mathbb{R}^{n})$ if and only if there is an $r>0$ such that

$|V_{\phi}f(x,\xi)|\leq C_{N}(1+|x|^{2})^{-N}e^{-r|\xi|^{1/\sigma}}$ (4.2)

for every $N\geq 0$.

For the proof, we will need the following three lemmas. These lemmas follow by
basic computations and are left for the reader to prove.

######  Lemma 4.2.

For $f\in S_{s}(\mathbb{R}^{n})$ ($f\in\Sigma_{s}(\mathbb{R}^{n})$) and
$\phi_{1},\phi_{2},\phi_{3}\in S_{s}(\mathbb{R}^{n})$
($\phi_{1},\phi_{2},\phi_{3}\in\Sigma_{s}(\mathbb{R}^{n})$),

$(\phi_{3},\phi_{1})V_{\phi_{2}}f(x,\xi)=\frac{1}{(2\pi)^{n/2}}\iint
V_{\phi_{1}}f(x-y,\xi-\eta)V_{\phi_{2}}\phi_{3}(y,\eta)e^{-i\langle
x-y,\eta\rangle}\,dy\,d\eta.$

######  Lemma 4.3.

If $s>0$ then there is a $C\geq 1$ such that

$C^{-1}(|x|^{1/s}+|y|^{1/s})\leq|y|^{1/s}+|y-x|^{1/s}\leq
C(|x|^{1/s}+|y|^{1/s}).$ (4.3)

for every $x,y\in\mathbb{R}^{n}$.

######  Lemma 4.4.

For any $\xi,\eta\in\mathbb{R}^{n}$ and any $N\geq 0$, there is a constant
$C>0$ such that

$(1+|\xi-\eta|^{2})^{-N}\leq C(1+|\xi|^{2})^{-N}(1+|\eta|^{2})^{N}.$

###### Proof of Theorem 4.1.

Suppose that $f\in S_{s}$. For every $N\geq 0$, we have

$\displaystyle|(1+|\xi|^{2})^{N}V_{\phi}f(x,\xi)|$
$\displaystyle=\frac{1}{(2\pi)^{n/2}}\left|\int_{\mathbb{R}^{n}}f(y)\overline{\phi(y-x)}(1+|\xi|^{2})^{N}e^{-i\langle
y,\xi\rangle}\,dy\right|$
$\displaystyle=\frac{1}{(2\pi)^{n/2}}\left|\int_{\mathbb{R}^{n}}f(y)\overline{\phi(y-x)}(1-\Delta)^{N}e^{-i\langle
y,\xi\rangle}\,dy\right|$
$\displaystyle=\frac{1}{(2\pi)^{n/2}}\left|\sum_{\gamma_{0}+|\gamma|=N}\dfrac{N!}{\gamma_{0}!\gamma!}\int_{\mathbb{R}^{n}}f(y)\overline{\phi(y-x)}D^{2\gamma}e^{-i\langle
y,\xi\rangle}\,dy,\right|$

where $\gamma^{\prime}=(\gamma_{0},\gamma)\in\mathbb{N}^{1+n}$ and the
derivatives are taken with respect to $y$. Integration by parts together with
Leibniz formula yields

$\displaystyle|(1+|\xi|^{2})^{N}V_{\phi}f(x,\xi)|$
$\displaystyle=\frac{1}{(2\pi)^{n/2}}\left|\sum_{\gamma_{0}+|\gamma|=N}\dfrac{N!}{\gamma_{0}!\gamma!}\int_{\mathbb{R}^{n}}D^{2\gamma}\left[f(y)\overline{\phi(y-x)}\right]e^{-i\langle
y,\xi\rangle}\,dy\right|$
$\displaystyle=\frac{1}{(2\pi)^{n/2}}\left|\sum_{\gamma^{\prime},\alpha}c^{N}_{\gamma^{\prime},\alpha}\int
D^{\alpha}f(y)D^{2\gamma-\alpha}\overline{\phi(y-x)}e^{-i\langle
y,\xi\rangle}dy\right|$
$\displaystyle\leq\frac{1}{(2\pi)^{n/2}}\sum_{\gamma^{\prime},\alpha}c^{N}_{\gamma^{\prime},\alpha}\int\left|D^{\alpha}f(y)D^{2\gamma-\alpha}\overline{\phi(y-x)}\right|\,dy,$

where
$\sum_{\gamma^{\prime},\alpha}=\sum_{|\gamma^{\prime}|=N}\sum_{\alpha\leq\gamma}$
and where
$c^{N}_{\gamma^{\prime},\alpha}=\dfrac{N!}{\gamma_{0}!\gamma!}\binom{2\gamma}{\alpha}$.
By Proposition 2.3 there are $C_{\gamma,\alpha},a>0$ such that

$\displaystyle\int\left|D^{\alpha}f(y)D^{2\gamma-\alpha}\overline{\phi(y-x)}\right|\,dy$
$\displaystyle\leq C_{\gamma,\alpha}\int e^{-a(|y|^{1/s}+|y-x|^{1/s})}\,dy,$

and by Lemma 4.3 there is a $c>0$ such that

$\int\left|D^{\alpha}f(y)D^{2\gamma-\alpha}\overline{\phi(y-x)}\right|\,dy\leq
C^{\prime}_{\gamma,\alpha}e^{-ac|x|^{1/s}}$

since $\int e^{-ac|y|^{1/s}}dy<\infty$. Hence, with $r=ac>0$ and
$C_{N,\gamma^{\prime},\alpha}=c^{N}_{\gamma^{\prime},\alpha}C^{\prime}_{\gamma,\alpha}>0$
we obtain

$\displaystyle|(1+|\xi|^{2})^{N}V_{\phi}f(x,\xi)|$
$\displaystyle\leq\frac{1}{(2\pi)^{n/2}}\sum_{\gamma^{\prime},\alpha}C_{N,\gamma^{\prime},\alpha}e^{-r|x|^{1/s}}$
$\displaystyle\leq C_{N}e^{-r|x|^{1/s}},$

where
$C_{N}=\frac{1}{(2\pi)^{n/2}}\sum_{\gamma^{\prime},\alpha}C_{N,\gamma^{\prime},\alpha}$.
Thus (4.1) holds for every $N\geq 0$.

Now suppose that (4.1) holds for every $N\geq 0$. This condition implies that
$f\in\mathscr{S}$ (cf. [10]). In particular $f\in C^{\infty}$, hence by
Proposition 2.3, the result follows if there is an $r>0$ such that (2.4) holds
for every multi-index $\beta$.

Consider $V_{\phi}[D^{\beta}f](x,\xi)$. Integrating by parts and applying
Leibniz formula gives

$\displaystyle V_{\phi}[D^{\beta}f](x,\xi)$
$\displaystyle=\frac{(-1)^{\beta}}{(2\pi)^{n/2}}\int
f(y)D^{\beta}\left(\overline{\phi(y-x)}e^{-i\langle y,\xi\rangle}\right)\,dy$
$\displaystyle=\sum_{\alpha\leq\beta}\frac{C_{\alpha,\beta}}{(2\pi)^{n/2}}\int
f(y)D^{\alpha}\overline{\phi(y-x)}\xi^{\beta-\alpha}e^{-i\langle
y,\xi\rangle}\,dy$
$\displaystyle=\sum_{\alpha\leq\beta}C_{\alpha,\beta}\xi^{\beta-\alpha}V_{D^{\alpha}\phi}f(x,\xi),$

where $C_{\alpha,\beta}=(-1)^{\beta}\binom{\beta}{\alpha}$. By Lemma 4.2 we
therefore have

$V_{\phi}[D^{\beta}f](x,\xi)=\sum_{\alpha\leq\beta}C^{\prime}_{\alpha,\beta}\xi^{\beta-\alpha}\iint
V_{\phi}f(x-y,\xi-\eta)V_{D^{\beta}\phi}\phi(y,\eta)e^{-i\langle
x-y,\eta\rangle}dy\,d\eta,$ (4.4)

where
$C^{\prime}_{\alpha,\beta}=(2\pi)^{-n/2}(\phi,\phi)^{-1}C_{\alpha,\beta}$.

For now, we consider only the double integral

$I=\iint V_{\phi}f(x-y,\xi-\eta)V_{D^{\beta}\phi}\phi(y,\eta)e^{-i\langle
x-y,\eta\rangle}dy\,d\eta$

from the right-hand side of the previous equation. Note that

$V_{D^{\beta}\phi}\phi(x,\xi)=e^{-i\langle
x,\xi\rangle}V_{\overline{\phi}}[D^{\beta}\overline{\phi}](-x,\xi),$

and since $D^{\beta}\overline{\phi},\overline{\phi}\in S_{s}\setminus\\{0\\}$
the first part of this theorem now implies that there is an $r_{1}>0$ such
that

$|V_{D^{\beta}\phi}\phi(x,\xi)|\leq
C_{\beta,N_{1}}(1+|\xi|^{2})^{-N_{1}}e^{-r_{1}|x|^{1/s}}$

for every $N_{1}\geq 0$ and every $\beta$. (Note that all the derivatives of
$\overline{\phi}$ fulfill Proposition 2.3 with the same exponent, hence we can
use the same $r_{1}>0$ for every $\beta$.) By assumption, (4.1) holds for all
$N\geq 0$. For any given $N$, pick $N_{1}>N$. We now obtain

$\displaystyle I$ $\displaystyle\leq
A_{\beta,N}\iint(1+|\xi-\eta|^{2})^{-N}e^{-r|x-y|^{1/s}}(1+|\eta|^{2})^{-N_{1}}e^{-r_{1}|y|^{1/s}}dyd\eta$
$\displaystyle=A_{\beta,N}I_{1}I_{2}$

where $A_{\beta,N}=C_{N}C_{\beta,N_{1}}$,

$I_{1}=\int e^{-r|x-y|^{1/s}}e^{-r_{1}|y|^{1/s}}dy$

and

$I_{2}=\int(1+|\xi-\eta|^{2})^{-N}(1+|\eta|^{2})^{-N_{1}}d\eta.$

In order to estimate $I_{1}$, we let $r_{2}=\min\\{r,r_{1}\\}$ and apply Lemma
4.3 to obtain $c>0$ such that

$\displaystyle I_{1}$ $\displaystyle\leq\int
e^{-r_{2}(|y-x|^{1/s}+|y|^{1/s})}dy$ $\displaystyle\leq
e^{-r_{2}c|x|^{1/s}}\int e^{-r_{2}c|y|^{1/s}}dy$
$\displaystyle=Be^{-r_{2}c|x|^{1/s}},$

where $B=\int e^{-r_{2}c|y|^{1/s}}dy<\infty$. Since $N_{1}>N$, Lemma 4.4 gives

$\displaystyle I_{2}$
$\displaystyle=\int(1+|\xi-\eta|^{2})^{-N}(1+|\eta|^{2})^{-N_{1}}d\eta$
$\displaystyle\leq C(1+|\xi|^{2})^{-N}\int(1+|\eta|^{2})^{N-N_{1}}d\eta$
$\displaystyle=B_{N}(1+|\xi|^{2})^{-N},$

where $B_{N}=C\int(1+|\eta|^{2})^{N-N_{1}}d\eta<\infty$. Combining these
estimates, we get

$I\leq B_{\beta,N}(1+|\xi|^{2})^{-N}e^{-r_{2}c|x|^{1/s}}$

for every $N\geq 0$, where $B_{\beta,N}=A_{\beta,N}BB_{N}$. Combining this
with (4.4), we obtain

$|V_{\phi}[D^{\beta}f](x,\xi)|\leq\sum_{\alpha\leq\beta}B_{\alpha,\beta,N}\xi^{\beta-\alpha}(1+|\xi|^{2})^{-N}e^{-cr|x|^{1/s}}$
(4.5)

for every $N\geq 0$, where
$B_{\alpha,\beta,N}=C^{\prime}_{\alpha,\beta}B_{\beta,N}$.

We now integrate both sides of (4.5) with respect to $\xi$. Note that

$|V_{\phi}f(x,\xi)|=(2\pi)^{-n/2}\left|\left(\hat{f}_{-x}*\psi\right)(\xi)\right|,$

where $\psi=\mathscr{F}\left[\,\overline{\phi}\,\right]$, and since
$\mathscr{F}[f_{a}](\eta)=e^{-i\langle a,\eta\rangle}\hat{f}(\eta)$,

$\displaystyle(2\pi)^{-n/2}\int_{\mathbb{R}^{n}}\left|\left(\hat{f}_{-x}*\psi\right)(\xi)\right|\,d\xi$
$\displaystyle\geq(2\pi)^{-n/2}\left|\int_{\mathbb{R}^{n}}\left(\hat{f}_{-x}*\psi\right)(\xi)\,d\xi\right|$
$\displaystyle=(2\pi)^{-n/2}\left|\iint\hat{f}_{-x}(\eta)\psi(\xi-\eta)\,d\eta\,d\xi\right|$
$\displaystyle=(2\pi)^{-n/2}\left|\int e^{i\langle
x,\eta\rangle}\hat{f}(\eta)\,d\eta\int\psi(\xi-\eta)\,d\xi\right|$
$\displaystyle=|f(x)|\left|\int\psi(\xi-\eta)\,d\xi\right|.$

Since $\int\psi(\xi-\eta)\,d\xi<\infty$, we therefore obtain

$|D^{\beta}f(x)|\leq C_{\phi}\int|V_{\phi}[D^{\beta}f](x,\xi)|d\xi,$ (4.6)

for some constant $C_{\phi}>0$. Moreover, if we fix $N>|\beta|$, then

$\int\xi^{\beta-\alpha}(1+|\xi|^{2})^{-N}d\xi=D_{\alpha,\beta}<\infty$

for each $\alpha\leq\beta$ and thus, with $r^{\prime}=r_{2}c$,

$\int|V_{\phi}[D^{\beta}f](x,\xi)|d\xi\leq\sum_{\alpha\leq\beta}B_{\alpha,\beta,N}D_{\alpha,\beta}e^{-r^{\prime}|x|^{1/s}}.$
(4.7)

Finally let
$C_{\beta}=C_{\phi}^{-1}\sum_{\alpha\leq\beta}B_{\alpha,\beta,N}D_{\alpha,\beta}$.
Then combining (4.6) with (4.7) now yields

$|D^{\beta}f(x)|\leq C_{\beta}e^{-r^{\prime}|x|^{1/s}}$

for every multi-index $\beta$. This completes the proof of (i).

To prove (ii), we first note that by Proposition 2.4, $f\in S^{\sigma}$ and
$\phi\in S^{\sigma}\setminus\\{0\\}$ if and only if $\hat{f}\in S_{\sigma}$
and $\hat{\phi}\in S_{\sigma}\setminus\\{0\\}$. By (i), we therefore have
$f\in S^{\sigma}$ if and only if

$|V_{\hat{\phi}}\hat{f}(x,\xi)|\leq
C_{N}(1+|\xi|^{2})^{-N}e^{-r|x|^{1/\sigma}}$

for every $N\geq 0$. Since $V_{\hat{\phi}}\hat{f}(x,\xi)=e^{-i\langle
x,\xi\rangle}V_{\phi}f(-\xi,x)$, this condition can be rewritten as

$|V_{\phi}f(-\xi,x)|\leq C_{N}(1+|\xi|^{2})^{-N}e^{-r|x|^{1/\sigma}}.$

Performing a variable substitution now yields (4.2). This completes the proof.
∎

Utilizing Proposition 2.3(b) instead of Proposition 2.3(a), we obtain the
following characterizations of short-time Fourier transforms in $\Sigma_{s}$
and $\Sigma^{\sigma}$ by analogous arguments.

######  Theorem 4.5.

Suppose $s,\sigma>0$.

  1. (i)

Let $\phi\in\Sigma_{s}(\mathbb{R}^{n})\setminus\\{0\\}$. Then
$f\in\Sigma_{s}(\mathbb{R}^{n})$ if and only if for every $r>0$ and every
$N\geq 0$,

$|V_{\phi}f(x,\xi)|\leq C_{r,N}(1+|\xi|^{2})^{-N}e^{-r|x|^{1/s}}.$

  2. (ii)

Let $\phi\in\Sigma^{\sigma}(\mathbb{R}^{n})\setminus\\{0\\}$. Then
$f\in\Sigma^{\sigma}(\mathbb{R}^{n})$ if and only if for every $r>0$ and every
$N\geq 0$,

$|V_{\phi}f(x,\xi)|\leq C_{r,N}(1+|x|^{2})^{-N}e^{-r|\xi|^{1/\sigma}}.$

Using these short-time Fourier transform characterizations, we can obtain
characterizations for the one-parameter spaces similar to those of [8] for the
two-parameter spaces.

######  Theorem 4.6.

Suppose $s,\sigma>0$ and $f\in C^{\infty}(\mathbb{R}^{n})$.

  1. (a)

$f\in S_{s}(\mathbb{R}^{n})$ if and only if there is an $r>0$ such that

$|f(x)|\leq Ce^{-r|x|^{1/s}},\quad|\hat{f}(\xi)|\leq C_{N}(1+|\xi|^{2})^{-N}$
(4.8)

for every $N\geq 0$.

  2. (b)

$f\in S^{\sigma}(\mathbb{R}^{n})$ if and only if there is an $r>0$ such that

$|f(x)|\leq C_{N}(1+|x|^{2})^{-N},\quad|\hat{f}(\xi)|\leq
Ce^{-r|\xi|^{1/\sigma}}$ (4.9)

for every $N\geq 0$.

###### Proof.

Suppose first that $f\in S_{s}$. Then there is an $r>0$ such that (4.1) holds
for every $N\geq 0$.

Integrating both sides of this equation with respect to $\xi$ yields

$\int|V_{\phi}f(x,\xi)|\,d\xi\leq C^{\prime}e^{-r|x|^{1/s}}.$

On the other hand, since
$V_{\phi}f=(\hat{f},\hat{\phi}(\cdot-\xi)e^{-i\langle\cdot,x\rangle}),$

$\displaystyle\int|V_{\phi}f(x,\xi)|\,d\xi$
$\displaystyle\geq(2\pi)^{-n/2}\left|\int e^{i\langle
x,\eta\rangle}\hat{f}(\eta)\int\overline{\hat{\phi}(\xi-\eta)}\,d\xi\,d\eta\right|$
$\displaystyle=C_{\phi}|f(x)|,$

hence we obtain

$|f(x)|\leq Ce^{-r|x|^{1/s}}.$

Starting instead by integrating both sides of (4.1) with respect to $x$ yields
the inequalities

$\displaystyle C_{\phi}^{\prime\prime}|\hat{f}(\xi)|$
$\displaystyle\leq(2\pi)^{-n/2}\int f(y)e^{-\langle
y,\xi\rangle}\int\overline{\phi(y-x)}\,dx\,dy$
$\displaystyle\leq\int|V_{\phi}f(x,\xi)|\,dx$ $\displaystyle\leq
C^{\prime}_{N}(1+|\xi|^{2})^{-N}$

hence we obtain

$|\hat{f}(\xi)|\leq C_{N}(1+|\xi|^{2})^{-N}$

for every $N\geq 0$.

Suppose instead that $f$ fulfills (4.8) for some $r>0$ and every $N\geq 0$.
Then

$|V_{\phi}(x,\xi)|\leq(2\pi)^{-n/2}\int|f(y)||\phi(y-x)|\,dy.$

Using what we proved in the first half of this proof, there are
$C_{0},r_{1},r_{2}>0$ such that

$\int|f(y)||\phi(y-x)|\,dy\leq C_{0}\int
e^{-r_{1}|y|^{1/s}}e^{-r_{2}|y-x|^{1/s}}\,dy\leq C_{1}e^{-r|x|^{1/s}}$

for some $r>0$, where we use Lemma 4.3 for the last inequality. Hence

$|V_{\phi}f(x,\xi)|\leq Ce^{-r|x|^{1/s}}.$

Using the same strategy once more but starting with the fact that

$|V_{\phi}f(x,\xi)|\leq(2\pi)^{-n/2}\int|\hat{f}(\eta)||\hat{\phi}(\eta-\xi)|\,d\eta,$

and this time utilizing Lemma 4.4 instead, we obtain

$|V_{\phi}f(x,\xi)|\leq C_{N}(1+|\xi|^{2})^{-N}$

for every $N\geq 0$. Combining both of these inequalities we see that for
every $N\geq 0$,

$|V_{\phi}f(x,\xi)|^{2}\leq C_{N}(1+|\xi|^{2})^{-N}e^{-r|x|^{1/s}},$

and in particular for $N=2k$, $k\geq 0$,

$|V_{\phi}f(x,\xi)|^{2}\leq C_{2k}(1+|\xi|^{2})^{-2k}e^{-r|x|^{1/s}},$

thus

$|V_{\phi}f(x,\xi)|\leq
C_{k}^{\prime}(1+|\xi|^{2})^{-k}e^{-r^{\prime}|x|^{1/s}}$

for all $k\geq 0$, where $C_{k}^{\prime}=\sqrt{C_{2k}}$ and $r^{\prime}=r/2$.
This completes the proof of (a).

To prove (b), simply perform Fourier transforms in light of Proposition 2.4
and apply (a). ∎

As with the other results, we state the corresponding theorem for the
$\Sigma_{s}$\- and $\Sigma^{\sigma}$-spaces but omit its proof as it follows
by analogous arguments.

######  Theorem 4.7.

Suppose $s,\sigma>0$ and $f\in C^{\infty}(\mathbb{R}^{n})$.

  1. (a)

$f\in\Sigma_{s}(\mathbb{R}^{n})$ if and only if for every $r>0$ and $N\geq 0$,

$|f(x)|\leq C_{r}e^{-r|x|^{1/s}},\quad|\hat{f}(\xi)|\leq
C_{N}(1+|\xi|^{2})^{-N}.$ (4.10)

  2. (b)

$f\in\Sigma^{\sigma}(\mathbb{R}^{n})$ if and only if for every $r>0$ and every
$N\geq 0$,

$|f(x)|\leq C_{N}(1+|x|^{2})^{-N},\quad|\hat{f}(\xi)|\leq
C_{r}e^{-r|\xi|^{1/\sigma}}.$ (4.11)

##  5\. Characterizations of dual spaces

We now move on to the characterization of duals to one-parameter spaces
$S_{s}$ and $S^{\sigma}$. These duals were defined in Section 2 via the
topologies detailed in Definition 6.4 and using the results of Section 4, we
now arrive at the following equivalent topologies.

######  Proposition 5.1.

Suppose $s,\sigma>0$.

  1. (i)

Let $\phi\in S_{s}(\mathbb{R}^{n})\setminus\\{0\\}$, let

$p^{\phi}_{s,r,N}(f)=\sup_{x,\xi\in\mathbb{R}^{n}}\left|V_{\phi}f(x,\xi)(1+|\xi|^{2})^{N}e^{r|x|^{1/s}}\right|$

and let $B_{s,r,N}(\mathbb{R}^{n})$ be the Banach space consisting of all
$f\in C^{\infty}(\mathbb{R}^{n})$ such that $p^{\phi}_{s,r,N}(f)$ is finite.
Then

$S_{s}(\mathbb{R}^{n})=\operatorname*{ind\,lim}_{r>0}\left(\projlim_{N\geq
0}B_{s,r,N}(\mathbb{R}^{n})\right)$

where the equality holds in a topological sense as well.

  2. (ii)

Let $\phi\in S^{\sigma}(\mathbb{R}^{n})\setminus\\{0\\}$, let

$q^{\phi,\sigma}_{r,M}(f)=\sup_{x,\xi\in\mathbb{R}^{n}}\left|V_{\phi}f(x,\xi)(1+|x|^{2})^{M}e^{r|\xi|^{1/\sigma}}\right|$

and let $B^{\sigma}_{r,M}(\mathbb{R}^{n})$ be the Banach space consisting of
all $f\in C^{\infty}(\mathbb{R}^{n})$ such that $q^{\phi,\sigma}_{r,M}(f)$ is
finite. Then

$S^{\sigma}(\mathbb{R}^{n})=\operatorname*{ind\,lim}_{r>0}\left(\projlim_{M\geq
0}B^{\sigma}_{r,M}(\mathbb{R}^{n})\right)$

where the equality holds in a topological sense as well.

###### Proof.

The equivalence of the semi-norms $p^{\phi}_{s,r,N}$ and $||\cdot||_{s,r,N}$,
as well as that of $q^{\phi,\sigma}_{r,M}$ and $||\cdot||^{\sigma}_{r,M}$ is
established implicitly in the proof of Theorem 4.1. ∎

This proposition gives us the following equivalent definitions for the dual
spaces $(S_{s})^{\prime}$ and $(S^{\sigma})^{\prime}$.

######  Corollary 5.2.

Suppose $s,\sigma>0$.

  1. (i)

Let $\phi\in S_{s}(\mathbb{R}^{n})\setminus\\{0\\}$ and let
$u\in\mathscr{D}^{\prime}(\mathbb{R}^{n})$. Then
$u\in(S_{s})^{\prime}(\mathbb{R}^{n})$ if and only if for every $r>0$ there is
an $N\geq 0$ such that

$|\langle u,f\rangle|\leq C_{N}p^{\phi}_{s,r,N}(f)$

for all $f\in S_{s}(\mathbb{R}^{n})$.

  2. (ii)

Let $\phi\in S^{\sigma}(\mathbb{R}^{n})\setminus\\{0\\}$ and
$u\in\mathscr{F}\mathscr{D}^{\prime}(\mathbb{R}^{n})$. Then
$u\in(S^{\sigma})^{\prime}(\mathbb{R}^{n})$ if and only if for every $r>0$
there is an $M\geq 0$ such that

$|\langle u,f\rangle|\leq C_{M}q^{\phi,\sigma}_{r,M}(f)$

for all $f\in S^{\sigma}(\mathbb{R}^{n})$.

This brings us to the following characterization of the duals via short-time
Fourier transforms, which is the main result of this section.

######  Theorem 5.3.

  1. (i)

Let $\phi\in S_{s}(\mathbb{R}^{n})\setminus\\{0\\}$ and
$f\in\mathscr{D}^{\prime}(\mathbb{R}^{n})$. Then
$f\in(S_{s})^{\prime}(\mathbb{R}^{n})$ if and only if for every $r>0$ there is
an $N_{0}\geq 0$ such that

$|V_{\phi}f(x,\xi)|\leq C_{r}(1+|\xi|^{2})^{N_{0}}e^{r|x|^{1/s}}.$ (5.1)

  2. (ii)

Let $\phi\in S^{\sigma}(\mathbb{R}^{n})\setminus\\{0\\}$ and
$f\in\mathscr{F}\mathscr{D}^{\prime}(\mathbb{R}^{n})$. Then
$f\in(S^{\sigma})^{\prime}(\mathbb{R}^{n})$ if and only if for every $r>0$
there is an $N_{0}\geq 0$ such that

$|V_{\phi}f(x,\xi)|\leq C_{r}(1+|x|^{2})^{N_{0}}e^{r|\xi|^{1/\sigma}}.$ (5.2)

###### Proof.

Suppose $f\in(S_{s})^{\prime}$, $\phi\in S_{s}\setminus\\{0\\}$. Then by
Proposition 2.12 and Leibniz formula, there is an $N_{0}>0$ such that

$\displaystyle|V_{\phi}f(x,\xi)|$
$\displaystyle=|(f,\phi(\cdot-x)e^{i\langle\cdot,\xi\rangle})|$
$\displaystyle\leq C_{N_{0}}\sum_{|\alpha|\leq
N_{0}}||D^{\alpha}\left(\phi(\cdot-x)e^{i\langle\cdot,\xi\rangle}\right)e^{r|\cdot|^{1/s}}||_{2}$
$\displaystyle=C_{N_{0}}\sum_{|\alpha|\leq
N_{0}}\sum_{\gamma\leq\alpha}\binom{\alpha}{\gamma}||(D^{\gamma}\phi)(\cdot-x)\xi^{\alpha-\gamma}e^{i\langle\cdot,\xi\rangle}e^{r|\cdot|^{1/s}}||_{2}$
$\displaystyle\leq C^{\prime}_{N_{0}}(1+|\xi|^{2})^{N_{0}}\sum_{|\alpha|\leq
N_{0}}\sum_{\gamma\leq\alpha}\binom{\alpha}{\gamma}||(D^{\gamma}\phi)(\cdot-x)e^{r|\cdot|^{1/s}}||_{2}$

By Theorem 2.3, there is an $r_{0}>0$ such that $|D^{\gamma}\phi(y-x)|\leq
C_{\gamma}e^{-r_{0}|y-x|^{1/s}}$, hence

$\displaystyle|V_{\phi}f(x,\xi)|$ $\displaystyle\leq
C^{\prime}_{N_{0}}(1+|\xi|^{2})^{N_{0}}\sum_{|\alpha|\leq
N_{0}}\sum_{\gamma\leq\alpha}\binom{\alpha}{\gamma}||e^{-r_{0}|\cdot-x|^{1/s}}e^{r|\cdot|^{1/s}}||_{2}.$

By Lemma 4.3, there is a $c\geq 1$ such that

$-r_{0}|y-x|^{1/s}\leq r_{0}|x|^{1/s}-r_{0}/c\cdot|y|^{1/s}.$

Let $r\in(0,r_{0}/(2c))$. Then

$-r_{0}|y-x|^{1/s}\leq-2cr|y-x|^{1/s}\leq 2cr|x|^{1/s}-2r|y|^{1/s},$

and

$\displaystyle|V_{\phi}f(x,\xi)|$ $\displaystyle\leq
C^{\prime}_{N_{0}}(1+|\xi|^{2})^{N_{0}}\sum_{|\alpha|\leq
N_{0}}\sum_{\gamma\leq\alpha}\binom{\alpha}{\gamma}||e^{2cr|x|^{1/s}-2r|\cdot|^{1/s}+r|\cdot|^{1/s}}||_{2}$
$\displaystyle=C^{\prime}_{N_{0}}(1+|\xi|^{2})^{N_{0}}e^{2cr|x|^{1/s}}\sum_{|\alpha|\leq
N_{0}}\sum_{\gamma\leq\alpha}\binom{\alpha}{\gamma}||e^{-r|\cdot|^{1/s}}||_{2}$
$\displaystyle\leq
C^{\prime\prime}_{N_{0}}(1+|\xi|^{2})^{N_{0}}e^{2cr|x|^{1/s}}.$

Clearly, the inequality still holds if we let $r>r_{0}/(2c)$ (the right hand
side only becomes larger), hence we have shown that the inequality is valid
for all $r>0$, as was to be shown.

Now suppose that for every $r>0$ there is an $N_{0}\geq 0$ such that (5.1)
holds. Then by Moyal’s identity, for every $\varphi\in S_{s}$

$|(f,\varphi)|\leq||\phi||_{2}^{-2}|(V_{\phi}f,V_{\phi}\varphi)|,$

hence

$|(f,\varphi)|\leq||\phi||_{2}^{-2}\int\int|V_{\phi}f(x,\xi)|\cdot|V_{\phi}\varphi(x,\xi)|\,dx\,d\xi.$

By assumption combined with Theorem 4.1, for every $r,r_{1}>0$ and any
$N_{1}\geq 0$ there is an $N_{0}\geq 0$ such that

$\displaystyle|(f,\varphi)|$ $\displaystyle\leq
C_{N_{0}}||\phi||_{2}^{-2}\int\int(1+|\xi|^{2})^{N_{0}}e^{r|x|^{1/s}}|V_{\phi}\varphi(x,\xi)|\,dx\,d\xi$
$\displaystyle=C_{N_{0}}||\phi||_{2}^{-2}\int\int(1+|\xi|^{2})^{(N_{0}-N_{1})}e^{(r-r_{1})|x|^{1/s}}\left|V_{\phi}\varphi(x,\xi)(1+|\xi|^{2})^{N_{1}}e^{r_{1}|x|^{1/s}}\right|\,dx\,d\xi$
$\displaystyle\leq
C_{N_{0}}||\phi||_{2}^{-2}p^{\phi,s}_{r_{1},N_{1}}(\varphi)\int\int(1+|\xi|^{2})^{-(N_{1}-N_{0})}e^{-(r_{1}-r)|x|^{1/s}}\,dx\,d\xi.$

Pick $N_{1}$ such that $N_{1}>N_{0}$ and pick $r$ such that $r<r_{1}$. Then we
obtain

$\displaystyle|(f,\varphi)|$ $\displaystyle\leq
C_{N_{0},N_{1},r}||\phi||_{2}^{-2}p^{\phi,s}_{r_{1},N_{1}}(\varphi)$

for all $r_{1}>r$. By picking $r>0$ arbitrarily small, we thus obtain

$\displaystyle|(f,\varphi)|$ $\displaystyle\leq C^{\prime}_{N_{1},r_{1}}\cdot
p^{\phi,s}_{r_{1},N_{1}}(\varphi)$

for all $r_{1}>0$. This completes the proof of (i). The proof of (ii) is very
similar, utilizing the fact that $\phi\in S^{\sigma}$ is equivalent to
$\hat{\phi}\in S_{\sigma}$, and the fact that
$V_{\phi}f(x,\xi)=(\hat{f},\hat{\phi}(\cdot-\xi)e^{-i\langle\cdot,x\rangle})$.

∎

Lastly we include the corresponding result for the dual spaces of $\Sigma_{s}$
and $\Sigma^{\sigma}$, which follows by analogous arguments.

######  Theorem 5.4.

  1. (i)

Let $\phi\in\Sigma_{s}(\mathbb{R}^{n})\setminus\\{0\\}$. Then
$f\in(\Sigma_{s})^{\prime}(\mathbb{R}^{n})$ if and only if there is an
$r_{0}>0$, a $C>0$ and an $N_{0}\geq 0$ such that

$|V_{\phi}f(x,\xi)|\leq C(1+|\xi|^{2})^{N_{0}}e^{r_{0}|x|^{1/s}}.$

  2. (ii)

Let $\phi\in\Sigma^{\sigma}(\mathbb{R}^{n})\setminus\\{0\\}$. Then
$f\in(\Sigma^{\sigma})^{\prime}(\mathbb{R}^{n})$ if and only if for every
$r_{0}>0$ there is an $N_{0}\geq 0$ such that

$|V_{\phi}f(x,\xi)|\leq C(1+|x|^{2})^{N_{0}}e^{r_{0}|\xi|^{1/\sigma}}.$

##  6\. Continuity of Toeplitz operators

We will now look at Toeplitz operators on one-parameter Gelfand-Shilov spaces.
To analyze these, we will need to consider functions in $2n$ dimensions which
belong to different one-parameter Gelfand-Shilov spaces in different
($n$-dimensional) variables. To make sense of these, we begin by examining the
spaces where each variable belongs to a two-parameter Gelfand-Shilov space.
These spaces are defined as follows.

######  Definition 6.1.

Suppose $s_{1},s_{2},\sigma_{1},\sigma_{2}>0$. Then
$S_{s_{1},\sigma_{2}}^{\sigma_{1},s_{2}}(\mathbb{R}^{2n})$ consists of every
$f\in C^{\infty}(\mathbb{R}^{2n})$ for which there is an $h>0$ such that

$\sup\dfrac{\left|x^{\alpha_{1}}\xi^{\alpha_{2}}D_{x}^{\beta_{1}}D_{\xi}^{\beta_{2}}f(x,\xi)\right|}{h^{|\alpha_{1}+\alpha_{2}+\beta_{1}+\beta_{2}|}(\alpha_{1}!)^{s_{1}}(\alpha_{2}!)^{\sigma_{2}}(\beta_{1}!)^{\sigma_{1}}(\beta_{2}!)^{s_{2}}}<\infty$

for every $\alpha_{1},\alpha_{2},\beta_{1},\beta_{2}\in\mathbb{N}^{n},$ where
the supremum is taken over $x,\xi\in\mathbb{R}^{n}$.

We can interpret this as a space where functions belong to
$S_{s_{1}}^{\sigma_{1}}(\mathbb{R}^{n})$ in the $x$-variable and
$S_{\sigma_{2}}^{s_{2}}(\mathbb{R}^{n})$ in the $\xi$-variable. Note that
$S_{s,s}^{\sigma,\sigma}(\mathbb{R}^{2n})=S_{s}^{\sigma}(\mathbb{R}^{2n})$.
With the notations $S_{s}^{\infty}=S_{s}$ and
$S_{\infty}^{\sigma}=S^{\sigma}$, we can construct similar spaces where the
functions belong to the one-parameter spaces in single variables instead.
These are the spaces we will focus on in this section.

######  Definition 6.2.

Suppose $s,t>0$. Then $S_{s,\infty}^{\infty,t}(\mathbb{R}^{2n})$ consists of
every $f\in C^{\infty}(\mathbb{R}^{2n})$ for which there is an $h>0$ such that

$\sup\dfrac{\left|x^{\alpha_{1}}\xi^{\alpha_{2}}D_{x}^{\beta_{1}}D_{\xi}^{\beta_{2}}f(x,\xi)\right|}{h^{|\alpha_{1}+\beta_{2}|}(\alpha_{1}!)^{s}(\beta_{2}!)^{t}}\leq
C_{\beta_{1},\alpha_{2}}$ (6.1)

for every $\alpha_{1},\alpha_{2},\beta_{1},\beta_{2}\in\mathbb{N}^{n},$ where
the supremum is taken over $x,\xi\in\mathbb{R}^{n}$ and where
$C_{\beta_{1},\alpha_{2}}$ is a constant depending only on $\beta_{1}$ and
$\alpha_{2}$.

In similar ways with $\sigma,\tau>0$, we let
$S_{\infty,\tau}^{\sigma,\infty}(\mathbb{R}^{2n})$ consist of every $f\in
C^{\infty}(\mathbb{R}^{2n})$ for which there is an $h>0$ such that

$\sup\dfrac{\left|x^{\alpha_{1}}\xi^{\alpha_{2}}D_{x}^{\beta_{1}}D_{\xi}^{\beta_{2}}f(x,\xi)\right|}{h^{|\beta_{1}+\alpha_{2}|}(\beta_{1}!)^{\sigma}(\alpha_{2}!)^{\tau}}\leq
C_{\alpha_{1},\beta_{2}}$ (6.2)

for every $\alpha_{1},\alpha_{2},\beta_{1},\beta_{2}\in\mathbb{N}^{n}$. Here,
the supremum is taken over $x,\xi\in\mathbb{R}^{n}$ and
$C_{\alpha_{1},\beta_{2}}$ is a constant depending only on $\alpha_{1}$ and
$\beta_{2}$. We also consider the duals of these spaces, which we construct as
follows.

Let $||f||_{h,N,M}$ be the supremum in (6.1) taken over
$x,\xi\in\mathbb{R}^{n}$, $\alpha_{2},\beta_{1}\in\mathbb{N}^{n}$, but only
$|\alpha_{1}|\leq N$ and $|\beta_{2}|\leq M$. With

$V_{h,N,M}(\mathbb{R}^{2n})=\\{f\in
C^{\infty}(\mathbb{R}^{2n}):||f||_{h,N,M}<\infty\\},$

we observe that

$S_{s,\infty}^{\infty,t}(\mathbb{R}^{2n})=\operatorname*{ind\,lim}_{h>0}\left(\projlim_{N,M\geq
0}V_{h,N,M}(\mathbb{R}^{2n})\right).$ (6.3)

It is therefore natural to set

$(S_{s,\infty}^{\infty,t})^{\prime}(\mathbb{R}^{2n})=\projlim_{h>0}\left(\operatorname*{ind\,lim}_{N,M\geq
0}(V_{h,N,M})^{\prime}(\mathbb{R}^{2n})\right).$ (6.4)

We construct the space
$(S_{\infty,\tau}^{\sigma,\infty})^{\prime}(\mathbb{R}^{2n})$ analogously.

Similar to the characterizations of $S_{s}$ and $S^{\sigma}$ via Fourier
transform in Theorem 4.6, we can characterize the double spaces
$S_{s,\infty}^{\infty,t}$ and $S_{\infty,\tau}^{\sigma,\infty}$ as follows.

######  Proposition 6.3.

Suppose $s,t,\sigma,\tau>0$ and $f\in C^{\infty}(\mathbb{R}^{2n})$.

  1. (a)

$f\in S_{s,\infty}^{\infty,t}(\mathbb{R}^{2n})$ if and only if there is an
$r>0$ such that

$|f(x,\xi)|\leq
C_{N}(1+|\xi|^{2})^{-N}e^{-r|x|^{1/s}},\quad|\hat{f}(\eta,y)|\leq(1+|\eta|^{2})^{-N}e^{-r|y|^{1/t}}$
(6.5)

for every $N\geq 0$.

  2. (b)

$f\in S_{\infty,\tau}^{\sigma,\infty}(\mathbb{R}^{2n})$ if and only if there
is an $r>0$ such that

$|f(x,\xi)|\leq
C_{N}(1+|x|^{2})^{-N}e^{-r|\xi|^{1/\tau}},\quad|\hat{f}(\eta,y)|\leq(1+|y|^{2})^{-N}e^{-r|\eta|^{1/\sigma}}$
(6.6)

for every $N\geq 0$.

###### Proof.

This result follows directly from Theorem 4.6. ∎

With this in mind we now consider Toeplitz operators on one-parameter Gelfand-
Shilov spaces.

######  Definition 6.4.

Let $s,\sigma>0$, $\phi_{1},\phi_{2}\in S_{s}(\mathbb{R}^{n})$ and $a\in
S_{s,\infty}^{\infty,s}(\mathbb{R}^{2n})$. The Toeplitz operator
$Tp_{\phi_{1},\phi_{2}}(a)$ is given by

$(Tp_{\phi_{1},\phi_{2}}(a)f,g)_{L^{2}(\mathbb{R}^{2n})}=(a,\overline{V_{\phi_{1}}f}\cdot
V_{\phi_{2}}g)_{L^{2}(\mathbb{R}^{2n})}$ (6.7)

for every $f\in S_{s}(\mathbb{R}^{n})$ and
$g\in(S_{s})^{\prime}(\mathbb{R}^{n})$.

If instead $\phi_{1},\phi_{2}\in S^{\sigma}(\mathbb{R}^{n})$ and $a\in
S^{\sigma,\infty}_{\infty,\sigma}(\mathbb{R}^{2n})$, the Toeplitz operator is
given by (6.7) for every $f\in S^{\sigma}(\mathbb{R}^{n})$ and
$g\in(S^{\sigma})^{\prime}(\mathbb{R}^{n})$.

We observe that the Toeplitz operator in (6.7) can be expressed as

$Tp_{\phi_{1},\phi_{2}}(a)f=V_{\phi_{2}}^{*}(a\cdot V_{\phi_{1}}f)$

and that this is a continuous operator from $S_{s}(\mathbb{R}^{n})$ to
$S_{s}(\mathbb{R}^{n})$ when $a\in S_{s,\infty}^{\infty,s}(\mathbb{R}^{2n})$,
and from $S^{\sigma}(\mathbb{R}^{n})$ to $S^{\sigma}(\mathbb{R}^{n})$ when
$a\in S_{\infty,\sigma}^{\sigma,\infty}(\mathbb{R}^{2n})$. We now want to show
that we can loosen the restriction on $a$ to instead be in the duals of
$S_{s,\infty}^{\infty,s}(\mathbb{R}^{2n})$ and
$S_{\infty,\sigma}^{\sigma,\infty}(\mathbb{R}^{2n})$. To do this, we need the
following lemma.

######  Lemma 6.5.

Let $\phi_{1},\phi_{2},f\in S_{s}(\mathbb{R}^{n})$ and
$g\in(S_{s})^{\prime}(\mathbb{R}^{n})$. Then

$\mathscr{F}[\overline{V_{\phi_{1}}f}\cdot
V_{\phi_{2}}g](\eta,y)=e^{iy\eta}V_{\phi_{2}}\phi_{1}(y,-\eta)\cdot
V_{f}g(-y,\eta)$

###### Proof.

We have

$\displaystyle\mathscr{F}[\overline{V_{\phi_{1}}f}\cdot
V_{\phi_{2}}g](\eta,y)$
$\displaystyle=(2\pi)^{-2n}\iiiint\overline{f(z)}\phi_{1}(z-x)g(w)\overline{\phi_{2}(w-x)}e^{i(z-w-y)\xi-
ix\eta}\,dzdwdxd\xi$
$\displaystyle=(2\pi)^{-n}\iint\overline{f(w+y)}\phi_{1}(w+y-x)g(w)\overline{\phi_{2}(w-x)}e^{-ix\eta}\,dw\,dx$
$\displaystyle=(2\pi)^{-n}\iint\overline{f(s)}\phi_{1}(s-x)g(s-y)\overline{\phi_{2}(s-y-x)}e^{-ix\eta}\,ds\,dx$
$\displaystyle=(2\pi)^{-n}\iint\overline{f(s)}\phi_{1}(t)g(s-y)\overline{\phi_{2}(t-y)}e^{-i(s-t)\eta}\,ds\,dt$
$\displaystyle=(2\pi)^{-n}\iint\overline{f(z+y)}\phi_{1}(t)g(z)\overline{\phi_{2}(t-y)}e^{-i(z+y-t)\eta}\,dz\,dt$
$\displaystyle=e^{-iy\eta}V_{\phi_{2}}\phi_{1}(y,-\eta)\cdot V_{f}g(-y,\eta)$

where we apply the Fourier inversion theorem in the second step, apply the
variable substitution $s=w+y$ in the third step, $t=s-x$ in the fourth step
and $z=s-y$ in the fifth step. ∎

With this lemma in mind we move on to the main result of this section.

######  Theorem 6.6.

Suppose $\phi_{1},\phi_{2}\in S_{s}(\mathbb{R}^{n})$. Then the following is
true.

  1. (a)

The definition of $Tp_{\phi_{1},\phi_{2}}(a)$ is uniquely extendable to any
$a\in(S_{s,\infty}^{\infty,s})^{\prime}(\mathbb{R}^{2n})$ and is then
continuous from $S_{s}(\mathbb{R}^{n})$ to $(S_{s})^{\prime}(\mathbb{R}^{n})$.

  2. (b)

If $a\in(S_{s,\infty}^{\infty,s})^{\prime}(\mathbb{R}^{2n})$ then
$Tp_{\phi_{1},\phi_{2}}(a)$ is continuous on $S_{s}(\mathbb{R}^{n})$ and
uniquely extendable to a continuous operator on
$(S_{s})^{\prime}(\mathbb{R}^{n})$.

###### Proof.

For (a), it is sufficient to show that $H=\overline{V_{\phi_{1}}f}\cdot
V_{\phi_{2}}g\in S_{s,\infty}^{\infty,s}(\mathbb{R}^{2n})$ whenever $f\in
S_{s}(\mathbb{R}^{n})$ and $g\in(S_{s})^{\prime}(\mathbb{R}^{n})$ and for (b),
the same statement is sufficient but with
$f\in(S_{s})^{\prime}(\mathbb{R}^{n})$ and $g\in S_{s}(\mathbb{R}^{n})$.

We begin by proving (a). By Theorem 4.1 and Theorem 5.3, for every $N\geq 0$
and every $r>0$, there are $r_{0}>0$, $N_{0}\geq 0$ and $C_{N,r}>0$ such that

$\displaystyle|H(x,\xi)|$
$\displaystyle=(2\pi)^{-n}|V_{\phi_{1}}f(x,\xi)|\cdot|V_{\phi_{2}}g(x,\xi)|$
$\displaystyle\leq C_{N,r}(1+|\xi|^{2})^{-(N-N_{0})}e^{-(r_{0}-r)|x|^{1/s}}.$

Picking $r<r_{0}$ and noting that $N_{0}\geq 0$ gives the first inequality of
Proposition 6.3.

By Lemma 6.5,

$\displaystyle|\hat{H}(\eta,y)|$
$\displaystyle=(2\pi)^{-n}|V_{\phi_{2}}\phi_{1}(y,-\eta)\cdot
V_{f}g(-y,\eta)|.$

Applying Theorem 4.1 and Theorem 5.3 exactly as before, we now obtain the
second inequality of Proposition 6.3. This completes the proof of (a). To
prove (b), simply reverse the roles of $f$ and $g$ and the result follows.

∎

We also state the corresponding result for
$a\in(S_{\infty,\sigma}^{\sigma,\infty})^{\prime}(\mathbb{R}^{2n})$, which
follows by similar arguments.

######  Theorem 6.7.

Suppose $\phi_{1},\phi_{2}\in S^{\sigma}(\mathbb{R}^{n})$. Then the following
is true.

  1. (a)

The definition of $Tp_{\phi_{1},\phi_{2}}(a)$ is uniquely extendable to any
$a\in(S_{\infty,\sigma}^{\sigma,\infty})^{\prime}(\mathbb{R}^{2n})$ and is
then continuous from $S^{\sigma}(\mathbb{R}^{n})$ to
$(S^{\sigma})^{\prime}(\mathbb{R}^{n})$.

  2. (b)

If $a\in(S_{\infty,\sigma}^{\sigma,\infty})^{\prime}(\mathbb{R}^{2n})$ then
$Tp_{\phi_{1},\phi_{2}}(a)$ is continuous on $S^{\sigma}(\mathbb{R}^{n})$ and
uniquely extendable to a continuous operator on
$(S^{\sigma})^{\prime}(\mathbb{R}^{n})$.

## References

  * [1] L. Hörmander, _The Analysis of Linear Partial Differential Operators I_ , 2nd ed, Berlin: Springer-Verlag, 1990\.
  * [2] H. Lewy, “An example of a smooth linear partial differential equation without solution,” _Annals of Mathematics_ , vol. 66, no. 1, pp. 155–158, Jul. 1957.
  * [3] M. Cappiello, “Pseudo-differential operators and spaces of type S,” _Progress in analysis_ , vol. 1, no. 2, pp. 681–688, 2003.
  * [4] A. Grossmann, G. Loupias, and E. Stein, “An algebra of pseudo-differential operators and quantum mechanics in phase space,” _Annales de l’institut Fourier_ , vol. 18, no. 2, pp. 343–368, 1968\.
  * [5] H. Feichtinger, L. Rodino, and M. Wong, _Pseudo-differential operators : quantization and signals : lectures given at the C.I.M.E. Summer School held in Cetraro, Italy, June 19-24, 2006_ , Berlin: Springer, 2008\.
  * [6] S. J. L. van Eijndhoven, “Functional analytic characterizations of the Gelfand-Shilov spaces $S_{\alpha}^{\beta}$,” _Indagationes Mathematicae (Proceedings)_ , vol. 89, issue 2, pp. 133–144, Jun. 1987.
  * [7] I. M. Gel’fand and G. E. Shilov, _Generalized Functions, Volume 2_ , New York: Academic Press, 1968.
  * [8] J. Chung, S-Y Chung and D. Kim, “Characterizations of the Gelfan-Shilov spaces via Fourier transforms,” _Proceedings of the American Mathematical Society_ , vol. 124, no. 7, Jul. 1996.
  * [9] S. Pilipović, “Tempered ultradistributions,” Boll. U.M.I., vol. 7, pp. 235–251, 1988\.
  * [10] K. Gröchenig and G. Zimmerman, “Spaces of Test Functions via the STFT”, _Journal of Function Spaces and Applications_ , vol. 2, pp. 25-33, 2004.
  * [11] J. Toft, “Images of function and distribution space under the Bargmann transforms,” _Journal of Pseudo-Differential Operators and Applications_ , vol. 8, pp. 83-139, 2017.
  * [12] A. Petersson, “Characterizations of Gelfand-Shilov spaces,” M.S. Thesis, Linnéuniversitetet, Fakulteten för teknik, Institutionen för matematik, Växjö, 2021.
  * [13] H. H. Schaefer, M. P. Wolff _Topological vector spaces_ , 2nd. ed. New York: Springer-Verlag, 1999\.
  * [14] S. Bannert, “Banach-Gelfand Triples and Applications in Time-Frequency Analysis,” Universität Wien, 2010.


# End-to-end Face-swapping via Adaptive Latent Representation Learning

Chenhao Lin
Xi’an Jiaotong University
<EMAIL_ADDRESS>Pengbin Hu
Xi’an Jiaotong University
<EMAIL_ADDRESS>Chao Shen
Xi’an Jiaotong University
<EMAIL_ADDRESS>Qian Li
Xi’an Jiaotong University
<EMAIL_ADDRESS>
###### Abstract

Taking full advantage of the excellent performance of StyleGAN, style
transfer-based face swapping methods have been extensively investigated
recently. However, these studies require separate face segmentation and
blending modules for successful face swapping, and the fixed selection of the
manipulated latent code in these works is reckless, thus degrading face
swapping quality, generalizability, and practicability. This paper proposes a
novel and end-to-end integrated framework for high resolution and attribute
preservation face swapping via Adaptive Latent Representation Learning.
Specifically, we first design a multi-task dual-space face encoder by sharing
the underlying feature extraction network to simultaneously complete the
facial region perception and face encoding. This encoder enables us to control
the face pose and attribute individually, thus enhancing the face swapping
quality. Next, we propose an adaptive latent codes swapping module to
adaptively learn the mapping between the facial attributes and the latent
codes and select effective latent codes for improved retention of facial
attributes. Finally, the initial face swapping image generated by StyleGAN2 is
blended with the facial region mask generated by our encoder to address the
background blur problem. Our framework integrating facial perceiving and
blending into the end-to-end training and testing process can achieve high
realistic face-swapping on wild faces without segmentation masks. Experimental
results demonstrate the superior performance of our approach over state-of-
the-art methods. Code will be made publicly available.

Figure 1: From left to right are source faces, target faces, and swapped faces
using our framework.

##  1 Introduction

Face swapping, one of the most popular DeepFake techniques, is receiving
growing attention over the last few years. Due to its advantages of the
automatic creation process and low cost, face swapping has been extensively
used in many fields like film and art creation, education, entertainment, and
social media [37]. While the malicious use of such techniques also raises
serious safety, privacy, and ethical concerns [21, 6].

Face swapping can be defined as the seamless transfer of identity information
from the source face image to the target face image while keeping the facial
attributes such as expression, skin tone, and lighting unchanged, and
producing a fake-like result. Early research [2, 26] adopt autoencoder-based
frameworks trained on subject-specific images for high-quality face swapping.
These methods require a large number of training samples from the same source
and target pair, leading to poor generalizability of the models on unseen
target faces. To alleviate this limitation, GAN-based subject agnostic face
swapping schemes [18, 22] have been proposed. These algorithms generally use
public datasets consisting of large enough face samples to train a face
swapping model with high generalizability. However, the generated deepfake
samples usually have low visual quality, like poor identity consistency and
attribute preservation, and fail to meet large-scale usage requirements.
Therefore, how to simultaneously achieve highly generalizable and realistic
face swapping is still a challenging direction to tackle.

Another challenge of face swapping is maintaining a sense of naturalness while
exchanging the contents of two faces and preserving the target image
background. To achieve this goal, several face swapping methods [17, 8, 12]
learn the commonality in faces and complete face swapping through an encoder-
decoder structure without specifying or limiting the location of the area to
be swapped, thus resulting in noticeable artifacts of the swapped faces and
resolution mismatch in the face regions. Recent studies propose to specify the
swap region using face mask images for higher quality face swapping [37, 22,
23]. Unfortunately, most swap algorithms require pre-generated segmentation
masks or a separately trained segmentation network with a blending module to
extract masks and complete face swapping. These multi-stage training schemes
are complex and may have high computational complexity. Moreover, they fail to
take advantage of the segmentation mask learning process, which may benefit
face swapping.

More recently, several works [33, 38, 3] adopt a face encoder to map face
images into latent codes and then manipulate these codes to get the source to
target latent codes. Although such a face encoding mechanism helps to learn
the face representation in latent space, the selection and swapping of the
manipulated latent codes are fixed and reckless, leading to low-quality
swapping of some facial attributes.

To address the limitations, we propose an end-to-end integrated framework for
high-resolution Face Swapping via Adaptive Latent representation Learning (FS-
ALL). Our framework is subject-agnostic and can be applied under different
subject faces without additional target-specific training while generating
high-quality face swapping images. Specifically, ALL consists of two main
components, Multi-task Dual-space Encoder (MDE) and Adaptive Latent codes
Swapping module (ALS). The former is designed to perceive the face swapping
region and generate the segmentation mask while simultaneously mapping and
decoupling the face image into the face pose and attribute latent codes.
Benefiting from the dual-space encoder mechanism and the guidance from the
facial perceiving, the proposed MDE can produce a more robust face
representation and enhance the face-swapping quality. ALS introduces a
learnable network to measure the latent codes at each layer, enabling us to
adaptively select and swap the effective latent codes to obtain the fused
latent codes for face swapping with a transformer-based facial attribute
retainer. Then, Stylegan2 is selected as a decoder to obtain the preliminary
face-swapped image. Finally, this image and the corresponding mask region are
passed through an internal facial blending module to complete the face
swapping. Our contributions can be summarized as follows,

  * •

We propose a novel end-to-end integrated framework, which can elegantly
produce high-resolution and attribute preservation face-swapping via adaptive
latent representation learning (ALL).

  * •

We carefully design ALL to perform face-to-latent space mapping and decoupling
and perceive facial regions simultaneously for robust face representation
learning. ALL also helps to adaptively select and swap the effective latent
codes for face swapping with enhanced attribute preservation.

  * •

Experimental results demonstrate that the face-swapped images generated using
our framework are improved in terms of naked eye effect and quantitative
metrics compared with the baselines and state-of-the-art face swapping
methods.

Figure 2: Overview of the proposed framework. (a) ALL includes a multi-task
dual-space encoder to map the source and target face pairs to $P$ and $A$
latent spaces, and achieve target facial region perceiving and segmentation
mask generation. An adaptive latent codes swapping module feeds the source
$L_{s}$ and target $L_{t}$ latent codes into a learnable network and obtains
$\bar{L}_{swap}$. The latent codes of $\bar{L}_{swap}$ and the target
${L}_{t}$ are fed into the transformer to get $L_{swap}$. (b) G&B feeds
$L_{swap}$ into the generator to get the preliminary swapped face image
$\bar{Y}$, and blends it with the mask to get the swapped face $Y$. The solid
line represents the encoding task and dash line perceiving task.

##  2 Related Works

###  2.1 Autoencoder methods

Face swapping algorithms were first implemented by an encoder and a set of
decoders. The faces were encoded and fed into each other’s encoders to achieve
face swapping, e.g., Deepfake[1]. In [17], the authors proposed to improve
face swapping by preserving face pose, expression, and lighting using CNN.
These approaches usually require many person-specific images or video data to
support the model’s training, limiting their practicality.

###  2.2 3D face methods

Several face swapping algorithms are based on the 3D face model[20, 24].
Face2Face[31] transferred expressions from source to target face by fitting a
3D morphable face model (3DMM)[30]. Nirkin[23] used a fixed 3D face shape as
the proxy. The expression of the target face is copied, and then the face is
segmented by a separately trained FCN[19] to generate the corresponding mask
and transfer the face. Most of the methods usually perform face reproduction
first, then cut and paste the face into the target using an additional
segmentation network.

###  2.3 GAN-based methods

FSGAN[22] is the current face replacement algorithm that achieves one of the
SOTA results. Four GANs are trained separately to accomplish face
reproduction, face segmentation, face inpainting, and face fusion. The
reproduced face is obtained by differencing the Euler angles between the
source face and the target face, then segmenting, cropping, and fusing them.
IPGAN[4] and FaceShifter[18] divided faces into identity and attribute
information, and FaceShifter inserted the identity information into the
attribute information continuously and completed the training by the GAN
network.

###  2.4 Latent space manipulation

Recently, several studies using the pre-trained StyleGAN[33, 36, 32, 29, 9]
have proposed to manipulate the latent codes of images for the purpose of
facial content manipulation. The GAN Inversion found the latent code of the
image and operated on it to modify the image.One is the optimization-based
approach of optimizing the latent code to minimize the error for the given
image. Barbershop[37] combined the latent code and mask to edit the specified
region. Transeditor[34] controlled the face pose and face style individually
by mapping the face into the double space and found the average vector between
the latent code of the two faces for linear transformation to achieve facial
attribute editing. The optimization-based approach is similar to the target-
specific face-swapping algorithm, which is more advantageous in generation
quality but less efficient. The other one is to train a generic swap to edit
the latent codes. MegaFS[38] proposed to transfer the identity from a source
image to the target by a non-linear trajectory without explicit feature
disentanglement. FSLSD[33] designed a framework to disentangle the latent
semantics of a pretrained StyleGAN to transfer the source identity and
preserve target appearance. Our approach improves MegaFS and FSLSD in latent
representation learning by perceiving facial regions, adaptively selecting and
swapping the latent codes, and introducing an internal facial blending module.

##  3 Methodology

The proposed framework is composed of two major phases, namely the adaptive
latent representation learning and the generation&blending, as illustrated in
Figure 2.

In the ALL phase, a feature extraction sharing network with a multi-task
learning module is implemented, to perceive the face-swapping regions and map
the face into face pose space $P$ and face attribute space $A$. In addition,
an adaptive latent codes swapping module is designed to select, swap and fuse
the source and target latent codes adaptively. In the G&B phase, the fused
latent codes and the latent codes in $P$ space are fed into the StyleGAN2
generator [15] and an internal blending module together with the segmentation
masks to obtain the swapped face.

Figure 3: Multi-task Dual-Space Encoder architecture. ResNet 50 is used as a
feature extraction network, and the left-side facial perceiving task uses
U-Net for dimensional connectivity. The right-side encoding task maps the
faces into $P$ space and $A$ space based on the FPN.

###  3.1 Adaptive latent representation learning

As shown in Figure 2, ALL mainly consists of two components, Multi-task Dual-
space Encoder (MDE) and Adaptive Latent codes Swapping module (ALS).

####  3.1.1 Multi-task dual-space encoder

To achieve simultaneous face swapping-region perception and face-to-latent
space mapping and decoupling, MDE first applies ResNet[10] as the shared
backbone for multi-scale feature extraction from the input face image, as
illustrated in Figure 3. The top layers of the network are split into two
branches and the first is for face perceiving. The multi-scale feature maps
are concatenated by channel dimension and passed through the upsampling,
batchnorm, and leakyReLU layers with reference to the U-Net structure to
generate the segmentation masks.

The second branch is for face encoding. We introduce dual-space face inversion
to map the faces into two separate latent spaces, pose latent space $P$, and
attribute latent space $A$, expecting to control the face pose and facial
attributes such as expression, skin tone, identity information, and hairstyle,
separately. Specifically, the feature maps at each scale are fed into the FPN
and mapped to obtain the latent code $A\in\mathbb{R}^{18\times 512}$ in $A$
space, representing facial attributes and identity information. We expand the
number of scales of the feature maps to make the network pay attention to the
details of the face image and find it helps to generate higher face quality
reconstruction. The latent code in $P$ space is passed through the feature map
output from the top layer of the encoder and then through 16 non-linear
mapping networks to obtain the latent code $P\in\mathbb{R}^{16\times 512}$ in
$P$ space. $P$ is used as the base input of the pre-trained StyleGAN2
generator, which is reshaped into $P\in\mathbb{R}^{4\times 4\times 512}$ as
the input of the generator.

Thus, the proposed MDE can accurately perceive the facial regions for end-to-
end face-swapping and simultaneously provide guidance to face encoding. It
also maps the face into the $P$ space and $A$ space to transfer the attribute
and identity information of the source face to the target face in the
subsequent exchange of latent codes while keeping the face pose unchanged.

####  3.1.2 Adaptive latent codes swapping

The selection of latent codes in existing approaches such as pSp[27],
MegaFS[38] and FSLSD[33] is usually fixed and reckless. Thus, some facial
attributes that are represented by accumulating specific multiple latent codes
may not be well learned and swapped, resulting in poor attribute preservation.
To alleviate this limitation, we propose a Learnable Latent codes Selection
Network (L2SNet) along with a transformer-based attribute retainer to
adaptively select and swap the latent codes while keeping the facial
attributes.

Figure 4: Row 1 and row 2 show the source and target faces. By keeping the
first three latent codes of the targets in $A$ space and all latent codes in
$P$ space, we can make the source have the same pose as the target.

Learnable latent codes selection network. Inspired by the findings in pSp[27],
we assume that different latent codes may correspond to different facial
attributes. Then we conduct a preliminary experiment and find that all the
latent codes in $P$ space and the first three latent codes in $A$ space are
used to control the face pose, as shown in Figure 4. The other latent codes in
$A$ space are mainly used to control the non-pose attributes. For example, the
fifth latent code mainly affects the information around the face’s mouth, as
shown in Figure 5. Therefore, we propose to fix all the latent codes in $P$
space and the first three latent codes in $A$, and to adaptively learn and
select effective latent codes in the remaining codes for face swapping. In
SENet[11], the claim that the weight of each channel of the image is different
inspires us that the content of each latent code of the face image may also be
different. To this end, we design a learnable network L2SNet with SENet
structure to measure the score of each latent code of the source and target
faces separately as follows.

Figure 5: By changing the latent codes in $A$ space two by two (e.g., 1&2, 2&3
latent codes), we find that the fifth latent code affects the information
around the mouth.

$\displaystyle Score=L2SNet(L)$ (1)
$Score_{st}=concat(score_{s},score_{t})=\begin{bmatrix}S_{4}&T_{4}\\\
S_{5}&T_{5}\\\ \vdots&\vdots\\\ S_{18}&T_{18}\\\ \end{bmatrix}$ (2)

where $Score$ is the output of each latent code $L$ of $L2SNet$. $s$ and $t$
represent the source and target faces. $Score_{st}$ represents the group of
latent codes after concatenation, and $S_{i}$, $T_{i}$ represents the $i^{th}$
latent code’s score from top to bottom, respectively. Then we calculate
$Mask_{L}$,

$Mask_{L}=argmax(Score_{st}^{1},Score_{st}^{2}\cdots\cdots,Score_{st}^{18})$
(3)

where $argmax(\dots)$ outputs a set of one-hot codes to select the effective
latent codes.

For the training of L2SNet, we need to obtain the gradient of the weights of
the evaluation network. Unfortunately, the $argmax$ function is not
differentiable. To solve this problem, we use the approximate gradient [5] as
the weight gradient of the L2SNet, as shown in the following:

$\bar{{score}_{st}^{i}}=\frac{e^{score_{st}^{i}}}{\sum_{n}e^{score_{st}^{i}}}\
\ \ \ \ \ \ \ i\in\begin{bmatrix}0&m-1\end{bmatrix}$ (4)

In the backward propagation process, we adopt the $softmax$ function
Eq.$(\ref{softmax})$, to get $\bar{{score}}_{st}^{i}$ as close to 0 or 1 as
possible to shorten the gap between the generated mask and the unique thermal
code. And $softmax$ is differentiable, making generated $Mask_{L}$
continuously optimized.

Latent codes swapping with transformer-based attribute retainer. Then we swap
the selected latent codes with a transformer to achieve enhanced attribute
preservation. $L_{st}$ dot product with $Mask_{L}$ to get the preliminary
selected latent code $\bar{L}_{swap}$

$\bar{L}_{swap}=L_{st}\cdot Mask_{L}$ (5)

Our preliminary experiment also observed that some facial attributes are mixed
together in several different latent codes. Thus, the attribute cannot be
accurately controlled if we swap a single latent code. While if several latent
codes are swapped together, the other attributed will be impacted. Therefore,
we design a transformer-based attribute retainer to adaptively decouple the
facial attributes in different latent codes and re-mix them for more precise
control of the attributes. In our transformer, the latent codes of the target,
$L_{t}$ are used as query ($Q$), i.e., as the attention query statement. The
swapped latent codes $L_{swap}$ are used as key ($K$) and value ($V$). It can
be represented as follows,

$Q=\bar{L}^{i}_{t}W^{Q},K=\bar{L}^{i}_{swap}W^{K},V=\bar{L}^{i}_{swap}W^{V}$
(6) $L^{i}_{swap}=softmax(\frac{QK^{T}}{\sqrt{d_{k}}})V+\bar{L}^{i}_{swap}$
(7)

where $W^{Q},W^{K},W^{V}$ are linear projection matrices, and $dk$ is the
dimensionality of the latent code. $L_{swap}$ receives queries from $L_{t}$.

###  3.2 Generation and blending

The pre-trained model of StyleGAN2 is used as a generator, the target latent
code $L_{p}$ in $P$ space is used as the base input, and ${L_{swap}}$ is fed
as the style to obtain the preliminary face swapping image $\bar{Y}$.

$\bar{Y}=G(L_{p},L_{swap})$ (8)

To address the background vignetting, hair color hairstyle with certain
artifacts, and other problems, we design an internal blending module to
seamlessly connect the $Y$ facial region to the target based on Poisson
fusion[25]. We first soften the area around the face mask so that the face is
smoothly connected to the surrounding area. Then we use the softened mask to
separate the face area and fuse it with the target. Existing methods usually
apply the blending module as image post-processing. In comparison, we propose
an elegant way of incorporating Poisson fusion into our framework to achieve
end-to-end training and testing by solving the problem of gradient non-passing
during the backward propagation process.

###  3.3 Training losses

The overall training loss of the proposed method consists of facial perception
and encoding loss $\mathcal{L}_{MDE}$ and latent code swapping loss
$\mathcal{L}_{ALS}$.

$\mathcal{L}=\mathcal{L}_{MDE}+\mathcal{L}_{ALS}$ (9)

Training losses of MDE. Since MDE completes the face perception and latent
space mapping, its loss function, ${L}_{MDE}$, consists of two parts. We use
binary cross-entropy, ${L}_{p}$, as the loss function for the face perception
task. Another part of the loss function, ${L}_{inv}$, is responsible for
training the mapping of the face to the latent space. It includes the
reconstruction loss $\mathcal{L}_{rec}$ at the pixel level and id loss
$\mathcal{L}_{id}$ to calculate the loss of identity information between the
reconstructed face and the real face using Arcface[7]. $\mathcal{L}_{ldm}$
calculates the landmark loss to keep the face pose stable using the facial
landmark extractor[16]. $\mathcal{L}_{LPISP}$ measures the LPISP loss[35] and
finally passes the reconstructed face through the MDE again to calculate the
$\mathcal{L}_{latent}$ loss at the latent code level. The overall loss
function is as follows:

$\mathcal{L}_{inv}=\lambda_{1}\mathcal{L}_{rec}+\lambda_{2}\mathcal{L}_{id}+\lambda_{3}\mathcal{L}_{ldm}+\lambda_{4}\mathcal{L}_{latent}+\lambda_{5}\mathcal{L}_{LPIPS}$
(10) $\mathcal{L}_{MDE}=\psi\mathcal{L}_{p}+\mathcal{L}_{inv}$ (11)

where $\psi,\lambda_{1},\lambda_{2},\lambda_{3},\lambda_{4}$ and $\lambda_{5}$
are loss weights.

Training losses of ALS. For the training of adaptive latent representation
learning, the reconstruction loss, id loss, landmark loss, and LPIP loss are
still included, with slight differences in the objects from the previous ones.
Please refer to the supplementary material for all specific loss functions.
The overall loss function is as follows:

$\mathcal{L}_{ALS}=\gamma_{1}\mathcal{L}_{rec}+\gamma_{2}\mathcal{L}_{id}+\gamma_{3}\mathcal{L}_{ldm}+\gamma_{4}\mathcal{L}_{LPIPS}$
(12)

where $\gamma{1},\gamma{2},\gamma{3}$ and $\gamma{4}$ are loss weights. Here
$y$ represents the final face replacement image after blending.

##  4 Experiments

In this section, two groups of comparison experiments with several state-of-
the-art methods are performed to validate the effectiveness of the proposed
approach. We first compare FSALL with the same type of SOTA methods,
MegaFS[38] and FSLSD[33], on high-resolution datasets, CelebA-HQ[13] and
FFHQ[14]. Then the current mainstream face swapping algorithms DeepFake[1],
FaceSwap[2], and FaceShifter[18] are performed on FaceForensics++[28] for
comparison. Both qualitative and quantitative comparisons are performed in two
groups of experiments. All experiments were conducted in Pytorch on two Tesla
A100 GPUs and two Intel XEON Gold CPUs.

Following previous SOTA methods [18, 38, 33], we use several metrics,
including ID retrieval, ID similarity, pose errors, and expression errors, for
quantitative evaluation of face swapping. Failure rate and Frechet Inception
Distance (FID) are adopted to measure the quality of reconstructed faces.
Details are described in the supplementary material.

###  4.1 Experiments on high-resolution datasets

Figure 6: Qualitative comparison of face swapping on CelebA-HQ dataset. We can
see that FSALL can produce higher quality swapped faces with more source
identity information while maintaining more of the target face attributes such
as mouth region and eye region.

Qualitative comparison on CelebA-HQ. Since both MegaFS and FSLSD methods
require pre-segmented facial masks for face swapping, we first conducted
experiments on CelebA-HQ with additional CelebAMask-HQ. While our method only
uses face images during test.

The qualitative comparisons are shown in Figure 6. It can be seen that FSALL
has more advantages in terms of facial attribute control. We attribute the
success to our ALS module, which adaptively manipulates the effective latent
codes, resulting in better identity transfer and attribute preservation. In
comparison, FSLSD and MegaFS operate on the fixed first seven latent codes for
each face change operation, which is difficult to guarantee that it will work
for every face. As in rows 2 and 4, the attributes around the mouth of the
FSLSD do not keep well with the target face, and the eye attributes of the
MegaFS in rows 1 and 5 do not keep well with the target face either. Our
method also shows a natural-looking swapping and a good performance on the
identity information transfer of source faces due to the robust face
representation learned by FSALL.

In addition, we can see the swapped faces generated by FSLSD tend to be
blurred (rows 2,3,5). Although FSLSD designed a target encoder and decoder
module to achieve low-to-high-resolution facial image restoration and reduce
the artifacts around the face, it may also result in blurred internal facial
regions. Figure 6 also shows that the swapped faces of MegaFS lack facial
details due to the limited representation ability of the encoder.

Method | ID similarity$\uparrow$ | pose$\downarrow$ | expression$\downarrow$
---|---|---|---
MegaFS | 0.3781 | 3.72 | 2.92
FSLSD | 0.3980 | 3.55 | 2.81
FSALL | 0.4327 | 3.61 | 2.70
Table 1: Quantitative evaluation on CelebA-HQ.

Quantitative comparison on CelebA-HQ. We generated swapped faces on CelebA-HQ
for quantitative comparison, shown in Table 1. The lower ID similarity and
expression error demonstrate that FSALL maintains more source identity
information and achieves better attribute control and preservation. FSALL is
slightly inferior to FSLSD in pose error because FSLSD adopted an effective
but external landmark estimator to align source and target landmarks. Their
blending module makes the target to be constantly modified based on the source
and helps the pose transfer. However, this module may also result in poor
identity transfer and distortion in the facial region.

Figure 7: Qualitative comparison with MegaFS on FFHQ. FSALL accomplishes more
face swapping and retains more detail in the characters, including the
challenged face swapping (row 5).

Qualitative comparison on FFHQ. We also performed comparisons on FFHQ. As
shown in Figure7, FSALL completes all the face swapping tasks, while MegaFS
shows some failures (rows 2 and 5), because the MegaFS encoder fails to map
the faces into the correct latent spaces. Moreover, the background and
hairstyle of MegaFS swapped faces are blurred, differing wildly from the
targets (rows 3 and 4). In comparison, FSALL retains them thanks to the
proposed facial perceiving and internal blending module. FSALL also retains
more detailed facial attributes of the source, such as the beard in rows 4 and
5.

Method | ID retrieval(%)$\uparrow$ | pose$\downarrow$ | expression$\downarrow$
---|---|---|---
MegaFS | 85.93 | 5.83 | 3.05
FSALL | 87.61 | 5.57 | 2.96
Table 2: Quantitative evaluation on FFHQ.

Quantitative comparison on FFHQ. Swapped faces on FFHQ were produced to
perform quantitative measurements, including ID retrieval, pose, and
expression, following MegaFS. The results are shown in Table 2 and FSALL
achieves superior performance in all metrics. FSALL maintains more source
identity information and illustrates better attribute preservation due to the
multi-task dual-space encoding in MDE and the superiority of ALS.

###  4.2 Experiments on FaceForensics++

We additionally evaluated FSALL on the low-resolution dataset FaceForensics++.
It contains many real videos from the physical world of Youtube with varying
quality. We applied dlib [16] to detect the face in video frames, crop out
face rectangle images, and filter out low-quality images. Several popular
methods were used for comparison.

Qualitative comparison. Figure 8 shows that both DeepFake and FaceSwap have
failure cases. In row 3, DeepFake and FaceSwap have errors around the eyes. In
the last row, DeepFake directly fails to render the portrait, and FaceSwap has
apparent mistakes in the nose and eye areas. Both FaceShifter and FSALL can
achieve successful face swapping, and the swapped faces of FaceShifter may be
closer to the target faces than FSALL in terms of expression properties (row
3). However, FaceShifter still has incomplete face swapping. For example,
FaceShifter barely passes the source’s identity information in row 4. In row
2, FaceShifter has a noticeable artifact on the right face.

Quantitative comparison. The results are shown in Table 3. FSALL and
FaceShifter achieve good results, where FSALL obtains the highest ID retrieval
and slightly underperformance in pose and expression error. This is because
the swapped latent code still contains part of the character attribute
information, which indicates that there is still room for improving the
decoupling of the latent code in our algorithm on FaceForensics++. FaceShifter
adds the identity information of the source to the target to keep the same
pose and expression. However, it is easy to produce incomplete or no face
replacement, such as in row 4 in Figure 8. The higher ID retrieval value also
indicates that FSALL has a better identity information transfer.

Method | ID retrieval(%)$\uparrow$ | pose$\downarrow$ | expression$\downarrow$
---|---|---|---
DeepFake | 80.54 | 3.80 | 3.04
FaceSwap | 67.29 | 2.31 | 2.80
FaceShifter | 88.46 | 3.08 | 2.54
FSALL | 90.23 | 3.10 | 2.84
Table 3: Quantitative evaluation on FaceForensics++

###  4.3 Ablation Study

We performed ablation experiments to explore the effect of the proposed MDE
and ALS components. More ablation studies can be found in the supplementary
material.

To evaluate MDE, we generated reconstructed face images using FFHQ and CelebA-
HQ datasets and performed quantitative measurements. The HieRFE encoder in
MegaFS and DE (without multi-task learning) are used for comparison. The
quantitative results for reconstructed faces are shown in Table 4. We can see
MDE performs better in ID similarity and pose because it encodes the face
space into $P$ and $A$ spaces to control the pose and attribute of the face
separately. In addition, MDE achieves higher reconstruction success rates and
quality (FID). We attribute the superior performance to the multi-task shared
feature extraction network of MDE, making the model have better generalization
performance on the original encoding task by sharing the feature
representation in the related face perceiving task. Moreover, the multi-task
learning strategy can help the DE generate higher-quality reconstructed face
images.

To evaluate ALS, we compare it with the fixed latent code swapping module. As
can be seen from the quantitative results in Table 5, the swapped images
obtained by ALS have more advantages in pose, expression, and image quality.
It demonstrates that the proposed ALS helps the swapped image preserve more
facial attributes, including but not limited to expression and skin color.
While we also note that the ID similarity is slightly inferior to that of the
fixed latent code swapping module, which may be because more preserved
attribute information from the targets of ALS swapped images has an impact on
the ID similarity (see the qualitative results in the supplementary material).

Figure 8: Qualitative comparison with DeepFake, FaceSwap, and FaceShifter on FaceForensics++. FSALL passes more source identity information.  Method |  ID simi- larity$\uparrow$ |  Pose$\downarrow$ |  Expre- ssion$\downarrow$ |  Failure Rate$\downarrow$ |  FID $\downarrow$
---|---|---|---|---|---
HieRFE | 0.8725 | 3.81 | 1.67 | 3.99% | 47.36
DE | 0.9076 | 3.20 | 2.12 | 1.73% | 35.42
MDE | 0.9288 | 2.93 | 2.01 | 1.73% | 35.17
Table 4: Quantitative result of the reconstructed faces. Method |  ID simi- larity$\uparrow$ |  Pose$\downarrow$ |  Expre- ssion$\downarrow$ |  FID $\downarrow$
---|---|---|---|---
W/o ALS | 0.4572 | 3.74 | 3.35 | 16.61
W ALS | 0.4327 | 3.61 | 2.70 | 15.59
Table 5: Quantitative result of the effect of ALS.

##  5 Conclusion

This work presents a novel end-to-end face swapping framework via adaptive
latent representation learning. A multi-task dual-space encoder is designed to
maintain face details well while perceiving facial regions to generate
corresponding masks for blending. We also propose an adaptive latent codes
swapping module to avoid too coarse latent codes selection and achieve
enhanced attribute preservation. Our experiments demonstrate that our FSALL
performs better face swapping than several existing SOTA methods.

## References

  * [1] Deepfake-.  https://github.com/deepfakes/faceswap., 2017.  Accessed: 2022-7-10.
  * [2] Faceswap-.  https://github.com/ondyari/FaceForensics/tree/master/dataset/FaceSwapKowalski., 2017.  Accessed: 2022-7-10.
  * [3] Rameen Abdal, Peihao Zhu, Niloy J Mitra, and Peter Wonka.  Styleflow: Attribute-conditioned exploration of stylegan-generated images using conditional continuous normalizing flows.  ACM Transactions on Graphics (ToG), 40(3):1–21, 2021.
  * [4] Jianmin Bao, Dong Chen, Fang Wen, Houqiang Li, and Gang Hua.  Towards open-set identity preserving face synthesis.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6713–6722, 2018.
  * [5] Jin Chen, Xijun Wang, Zichao Guo, Xiangyu Zhang, and Jian Sun.  Dynamic region-aware convolution.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8064–8073, 2021.
  * [6] Bobby Chesney and Danielle Citron.  Deep fakes: A looming challenge for privacy, democracy, and national security.  Calif. L. Rev., 107:1753, 2019.
  * [7] Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou.  Arcface: Additive angular margin loss for deep face recognition.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4690–4699, 2019.
  * [8] Brian Dolhansky, Russ Howes, Ben Pflaum, Nicole Baram, and Cristian Canton Ferrer.  The deepfake detection challenge (dfdc) preview dataset.  arXiv preprint arXiv:1910.08854, 2019.
  * [9] Jingwen He, Wu Shi, Kai Chen, Lean Fu, and Chao Dong.  Gcfsr: a generative and controllable face super resolution method without facial and gan priors.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1889–1898, 2022.
  * [10] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [11] Jie Hu, Li Shen, and Gang Sun.  Squeeze-and-excitation networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7132–7141, 2018.
  * [12] Liming Jiang, Ren Li, Wayne Wu, Chen Qian, and Chen Change Loy.  Deeperforensics-1.0: A large-scale dataset for real-world face forgery detection.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2889–2898, 2020.
  * [13] Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen.  Progressive growing of gans for improved quality, stability, and variation.  arXiv preprint arXiv:1710.10196, 2017.
  * [14] Tero Karras, Samuli Laine, and Timo Aila.  A style-based generator architecture for generative adversarial networks.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4401–4410, 2019.
  * [15] Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila.  Analyzing and improving the image quality of stylegan.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8110–8119, 2020.
  * [16] Davis E King.  Dlib-ml: A machine learning toolkit.  The Journal of Machine Learning Research, 10:1755–1758, 2009.
  * [17] Iryna Korshunova, Wenzhe Shi, Joni Dambre, and Lucas Theis.  Fast face-swap using convolutional neural networks.  In Proceedings of the IEEE international conference on computer vision, pages 3677–3685, 2017.
  * [18] Lingzhi Li, Jianmin Bao, Hao Yang, Dong Chen, and Fang Wen.  Faceshifter: Towards high fidelity and occlusion aware face swapping.  arXiv preprint arXiv:1912.13457, 2019.
  * [19] Jonathan Long, Evan Shelhamer, and Trevor Darrell.  Fully convolutional networks for semantic segmentation.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3431–3440, 2015.
  * [20] Junfeng Lyu, Zhibo Wang, and Feng Xu.  Portrait eyeglasses and shadow removal by leveraging 3d synthetic data.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3429–3439, 2022.
  * [21] Jacek Naruniec, Leonhard Helminger, Christopher Schroers, and Romann M Weber.  High-resolution neural face swapping for visual effects.  In Computer Graphics Forum, volume 39, pages 173–184. Wiley Online Library, 2020.
  * [22] Yuval Nirkin, Yosi Keller, and Tal Hassner.  Fsgan: Subject agnostic face swapping and reenactment.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 7184–7193, 2019.
  * [23] Yuval Nirkin, Iacopo Masi, Anh Tran Tuan, Tal Hassner, and Gerard Medioni.  On face segmentation, face swapping, and face perception.  In 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), pages 98–105. IEEE, 2018.
  * [24] Naima Otberdout, Claudio Ferrari, Mohamed Daoudi, Stefano Berretti, and Alberto Del Bimbo.  Sparse to dense dynamic 3d facial expression generation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20385–20394, 2022.
  * [25] Patrick Pérez, Michel Gangnet, and Andrew Blake.  Poisson image editing.  In ACM SIGGRAPH 2003 Papers, pages 313–318. 2003.
  * [26] Ivan Perov, Daiheng Gao, Nikolay Chervoniy, Kunlin Liu, Sugasa Marangonda, Chris Umé, Mr Dpfks, Carl Shift Facenheim, RP Luis, Jian Jiang, et al.  Deepfacelab: A simple, flexible and extensible face swapping framework.  2020\.
  * [27] Elad Richardson, Yuval Alaluf, Or Patashnik, Yotam Nitzan, Yaniv Azar, Stav Shapiro, and Daniel Cohen-Or.  Encoding in style: a stylegan encoder for image-to-image translation.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2287–2296, 2021.
  * [28] Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner.  Faceforensics++: Learning to detect manipulated facial images.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 1–11, 2019.
  * [29] Jingxiang Sun, Xuan Wang, Yong Zhang, Xiaoyu Li, Qi Zhang, Yebin Liu, and Jue Wang.  Fenerf: Face editing in neural radiance fields.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7672–7682, 2022.
  * [30] Justus Thies, Michael Zollhöfer, Matthias Nießner, Levi Valgaerts, Marc Stamminger, and Christian Theobalt.  Real-time expression transfer for facial reenactment.  ACM Trans. Graph., 34(6):183–1, 2015.
  * [31] Justus Thies, Michael Zollhofer, Marc Stamminger, Christian Theobalt, and Matthias Nießner.  Face2face: Real-time face capture and reenactment of rgb videos.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2387–2395, 2016.
  * [32] Chao Xu, Jiangning Zhang, Miao Hua, Qian He, Zili Yi, and Yong Liu.  Region-aware face swapping.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7632–7641, 2022.
  * [33] Yangyang Xu, Bailin Deng, Junle Wang, Yanqing Jing, Jia Pan, and Shengfeng He.  High-resolution face swapping via latent semantics disentanglement.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7642–7651, 2022.
  * [34] Yanbo Xu, Yueqin Yin, Liming Jiang, Qianyi Wu, Chengyao Zheng, Chen Change Loy, Bo Dai, and Wayne Wu.  Transeditor: Transformer-based dual-space gan for highly controllable facial editing.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7683–7692, 2022.
  * [35] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang.  The unreasonable effectiveness of deep features as a perceptual metric.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595, 2018.
  * [36] Yang Zhou, Yangyang Xu, Yong Du, Qiang Wen, and Shengfeng He.  Pro-pulse: Learning progressive encoders of latent semantics in gans for photo upsampling.  IEEE Transactions on Image Processing, 31:1230–1242, 2022.
  * [37] Peihao Zhu, Rameen Abdal, John Femiani, and Peter Wonka.  Barbershop: Gan-based image compositing using segmentation masks.  arXiv preprint arXiv:2106.01505, 2021.
  * [38] Yuhao Zhu, Qi Li, Jian Wang, Cheng-Zhong Xu, and Zhenan Sun.  One shot face swapping on megapixels.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4834–4844, 2021.