# Isometries between groups of invertible elements in Fourier-Stieltjes
algebras

Osamu Hatori  Institute of Science and Technology, Niigata University, Niigata
950-2181, Japan<EMAIL_ADDRESS>and  Shiho Oi  Department of
Mathematics, Faculty of Science, Niigata University, Niigata 950-2181, Japan.
<EMAIL_ADDRESS>
###### Abstract.

We prove that if open subgroups of the groups of invertible elements in two
Fourier-Stieltjes algebras are isometric as metric spaces, then the underlying
locally compact groups are topologically isomorphic. We describe the structure
of isometric real algebra isomorphisms between Fourier-Stieltjes algebras and
apply it to prove the above result.

###### Key words and phrases:

Fourier-Stieltjes algebras, isometries, invertible elements

###### 2020 Mathematics Subject Classification:

46B04, 46J10, 43A30, 43A22

##  1\. Introduction

There has been a long-standing effort to determine the conditions of a linear
subspace $A$ of $C(X)$, a space of complex-valued continuous functions on a
compact Hausdorff space $X$, to ensure that $X$ can be uniquely determined.
The Banach-Stone theorem is the most prominent result in this field. According
to the theorem, in addition to $A=C(X)$, the crucial requirement is that the
structure is a Banach space.

This paper focuses on how the Fourier-Stieltjes algebra $B(G)$ of a locally
compact group $G$ encodes its structure. The algebra $B(G)$ is defined as the
complex linear span of the set of all continuous positive-definite functions
on $G$. It is a semisimple commutative Banach algebra of all matrix
coefficients of continuous unitary representations of $G$.

By [11, Theorem 20.20] (cf. [17]) and [5, Section 11.1 Exercise 9], we
identify $L^{1}(G)^{*}$ with $L^{\infty}(G)$ for a locally compact group $G$.
Therefore the Fourier-Stieltjes algebra $B(G)$ is identified with the dual of
the group $C^{*}$-algebra $C^{*}(G)$. The dualities determine the norm on
$B(G)$. The enveloping von Neumann algebra of $C^{*}(G)$, denoted as
$W^{*}(G)$, is identified with $B(G)^{*}$ when $B(G)$ is identified with the
dual of $C^{*}(G)$. Throughout the paper, we assume that $G$ and $H$ are
locally compact groups, with $e_{1}$ and $e_{2}$ denoting their respective
units.

A celebrated theorem by Walter [19] states that if $S\colon B(G)\to B(H)$ is
an isometric complex algebra isomorphism, then $G$ and $H$ are topologically
isomorphic as locally compact groups. However, it is impractical that the
structure of $B(G)$ as a Banach space does not completely encode the group
structure of $G$. For instance, consider two discrete abelian groups with the
same cardinality, $\Gamma_{1}$ and $\Gamma_{2}$. Any bijection
$\alpha\colon\Gamma_{2}\to\Gamma_{1}$ induces a surjective complex linear
isometry from $L^{1}(\Gamma_{1})\to L^{1}(\Gamma_{2})$ by $f\mapsto
f\circ\alpha$ for $f\in L^{1}(\Gamma_{1})$, which in turn induces the complex
linear isometry from $B(\widehat{\Gamma_{1}})\to B(\widehat{\Gamma_{2}})$
through the Fourier transform. However, it is possible for $\Gamma_{1}$ and
$\Gamma_{2}$ to be nonisomorphic as groups; for example, if
$\Gamma_{1}=\mathbb{Z}$ (the integer group) and
$\Gamma_{2}=\mathbb{Z}\times\mathbb{Z}$, $B(\widehat{\Gamma_{1}})$ and
$B(\widehat{\Gamma_{2}})$ are isometrically isomorphic as Banach spaces, but
the group $\widehat{\Gamma_{1}}$ is not isomorphic to $\widehat{\Gamma_{2}}$.

On the other hand, in [9], the first author showed that the metric structures
of open subgroups of groups of invertible elements in the measure algebras on
locally compact abelian groups guarantee the structure of the underlying
locally abelian groups. Investigating whether the same result holds true for
noncommutative groups would be interesting and natural. In this paper, we will
provide an effective answer to this question for any locally compact group.
The key is to characterize all isometric real algebra isomorphisms between two
Fourier-Stieltjes algebras on locally compact groups, which is Theorem 2.14.
Since any complex algebra isomorphism occurs at the same time as a real
algebra isomorphism, Theorem 2.14 is a generalization of the theorem proposed
by Walter[19]. We deduce that every isometric real algebra isomorphism between
two Fourier-Stieltjes algebras automatically induces a topological group
isomorphism between underlying groups. This characterization makes it possible
to answer the above question: Corollary 3.2.

##  2\. Isometric real algebra isomorphisms

In this section, we consider surjective real linear isometries that also
preserve the multiplication between $B(G)$ and $B(H)$. To distinguish them
from usual isometric isomorphisms (which are assumed to be complex linear), we
refer to them as isometric real algebra isomorphisms. We present Theorem 2.14
which characterizes an isometric real algebra isomorphism. This is the result
of a comparison with a celebrated theorem of Walter [19]. Authors such as
Cohen [1], Daws [2], Ilie [12], Ilie and Spronk [13], and Le Pham [16] have
extensively studied complex linear homomorphisms on Fourier-Stieltjes algebras
and Fourier algebras. Our study presents a new approach for obtaining a
representation of real linear operators on Fourier-Stieltjes algebras. In
contrast to Walter’s work, our results demonstrate that the assumption of real
linearity is sufficient for the isomorphism of underlying groups for similar
mappings. Notably, every isometric real algebra isomorphism is either complex
linear or conjugate-linear, which is an interesting finding.

Let $T:B(G)\to B(H)$ be an isometric real algebra isomorphism. We define
$T^{*}:B(H)^{*}\to B(G)^{*}$ by

$T^{*}(\varphi)(f)=\operatorname{Re}\varphi(Tf)-i\operatorname{Re}\varphi(T(if)),\quad
f\in B(G)$ (1)

for $\varphi\in B(H)^{*}$. Then, $T^{*}$ is a surjective real linear isometry.
For the convenience of the readers, we have included this proof. By a simple
calculation, we see that $T^{*}(\varphi)$ is a complex linear functional on
$B(G)$ for $\varphi\in B(H)^{*}$. We show that
$\|T^{*}(\varphi)\|\leq\|\varphi\|$ for every $\varphi\in B(H)^{*}$. Let $f\in
B(G)$. There exists $0\leq\theta<2\pi$ such that

$T^{*}(\varphi)(f)=|T^{*}(\varphi)(f)|e^{i\theta}.$

Then,

$\begin{split}|T^{*}(\varphi)(f)|&=e^{-i\theta}T^{*}(\varphi)(f)=T^{*}(\varphi)(e^{-i\theta}f)\\\
&=\operatorname{Re}\varphi(T(e^{-i\theta}f))\leq|\varphi(T(e^{-i\theta}f))|\\\
&\leq\|\varphi\|\|T(e^{-i\theta}f)\|=\|\varphi\|\|e^{-i\theta}f\|=\|\varphi\|\|f\|.\end{split}$

Hence, we have $\|T^{*}(\varphi)\|\leq\|\varphi\|$ for $\varphi\in B^{*}(H)$.
We show that the inverse inequality $\|\varphi\|\leq\|T^{*}(\varphi)\|$. Then,
$\|T^{*}(\varphi)\|=\|\varphi\|$, so that $T^{*}$ is an isometry. Let $h\in
B(H)$. Then, $0\leq\theta<2\pi$ exists such that
$\varphi(h)=|\varphi(h)|e^{i\theta}$. Then

$\begin{split}|\varphi(h)|&=e^{-i\theta}\varphi(h)=\varphi(e^{-i\theta}h)\\\
&=\varphi(T(T^{-1}(e^{-i\theta}h)))=\operatorname{Re}\varphi(T(T^{-1}(e^{-i\theta}h)))\\\
&\leq|T^{*}(\varphi)(T^{-1}(e^{-i\theta}h))|\leq\|T^{*}(\varphi)\|\|T^{-1}(e^{-i\theta}h)\|\\\
&=\|T^{*}(\varphi)\|\|h\|.\end{split}$

As $h\in B(H)$ is arbitrary, we have that $\|\varphi\|\leq\|T^{*}(\varphi)\|$.
To prove that $T^{*}$ is surjective, we consider $(T^{-1})^{*}\colon
B(G)^{*}\to B(H)^{*}$, similar to $T^{*}$;
$(T^{-1})^{*}(\psi)(h)=\operatorname{Re}\psi(T^{-1}(h))-i\operatorname{Re}\psi(T^{-1}(ih))$
for $h\in B(H)$ and $\psi\in B(G)^{*}$. Let $\psi\in B(G)^{*}$ be arbitrary.
Put $\varphi^{\prime}=(T^{-1})^{*}(\psi)$. Then $\varphi^{\prime}\in
B(H)^{*}$. For every $f\in B(G)$ we have that

$\begin{split}T^{*}(\varphi^{\prime})(f)&=\operatorname{Re}\varphi^{\prime}(T(f))-i\operatorname{Re}\varphi^{\prime}(T(if))\\\
&=\operatorname{Re}(\operatorname{Re}\psi(T^{-1}(T(f)))-i\operatorname{Re}\psi(T^{-1}(iT(f))))\\\
&-i\operatorname{Re}(\operatorname{Re}\psi(T^{-1}(T(if)))-i\operatorname{Re}\psi(T^{-1}(iT(if))))\\\
&=\operatorname{Re}\psi(f)-i\operatorname{Re}\psi(if)=\psi(f).\end{split}$

As $\psi\in B(G)^{*}$ is arbitrary we infer that $T^{*}$ is surjective.

Let $\sigma(B(G))$ and $\sigma(B(H))$ be the spectra (the same as the maximal
ideal spaces) of $B(G)$ and $B(H)$, respectively.

######  Lemma 2.1.

We have

$T^{*}(\sigma(B(H)))=\sigma(B(G)).$

###### Proof.

Let $\varphi\in\sigma(B(H))$. For any $f\in B(G)$, we have
$\varphi(T(if))^{2}=-(\varphi(Tf))^{2}$. This implies that

$\varphi(T(if))=i\varphi(Tf)$ or $\varphi(T(if))=-i\varphi(Tf)$. (2)

Let $f,g\in B(G)$ and $\varphi\in\sigma(B(H))$. As $T$ and $\varphi$ are
multiplicative and real linear we have

$\varphi(T(if))\varphi(T(ig))=\varphi(T(if)T(ig))\\\
=\varphi(T(-fg))=-\varphi(T(fg))=(i\varphi(T(f)))(i\varphi(T(g))).$ (3)

If $\varphi(T(f))\neq 0$ and $\varphi(T(g))\neq 0$, then $\eqref{jyami1}$
ensures that

$\text{$i\varphi(T(f))=\varphi(T(if))$ if and only if
$i\varphi(T(g))=\varphi(T(ig))$}.$ (4)

Hence we have

$\operatorname{Re}i\varphi(T(f))\operatorname{Re}i\varphi(T(g))=\operatorname{Re}\varphi(T(if))\operatorname{Re}\varphi(T(ig)).$
(5)

If $\varphi(T(f))=0$ or $\varphi(T(g))=0$, then we infer that
$\varphi(T(if))=0$ or $\varphi(T(ig))=0$ respectively by (2). Hence, (5) also
holds for any $f,g\in B(G)$. Applying (5) we have

$\begin{split}&T^{*}(\varphi)(f)T^{*}(\varphi)(g)\\\
&=(\operatorname{Re}\varphi(T(f))-i\operatorname{Re}\varphi(T(if)))(\operatorname{Re}\varphi(T(g))-i\operatorname{Re}\varphi(T(ig)))\\\
&=\operatorname{Re}\varphi(T(f))\operatorname{Re}\varphi(T(g))-\operatorname{Re}i\varphi(T(f))\operatorname{Re}i\varphi(T(g))\\\
&-i\left(\operatorname{Re}\varphi(T(if))\operatorname{Re}\varphi(T(g))-\operatorname{Re}i\varphi(T(if))\operatorname{Re}i\varphi(T(g))\right)\\\
&=\operatorname{Re}\varphi(T(f))\operatorname{Re}\varphi(T(g))-\operatorname{Im}\varphi(T(f))\operatorname{Im}\varphi(T(g))\\\
&-i\left(\operatorname{Re}\varphi(T(if))\operatorname{Re}\varphi(T(g))-\operatorname{Im}\varphi(T(if))\operatorname{Im}\varphi(T(g))\right)\\\
&=T^{*}(\varphi(fg)).\end{split}$

Thus $T^{*}(\varphi)$ is a multiplicative linear functional. As $T^{*}$ is a
surjective isometry and $\varphi\neq 0$, we obtain $T^{*}(\varphi)\neq 0$.
Hence $T^{*}(\varphi)\in\sigma(B(G))$. To prove that $T^{*}$ maps
$\sigma(B(H))$ onto $\sigma(B(G))$, we consider $(T^{-1})^{*}\colon
B(G)^{*}\to B(H)^{*}$ in a similar way as $T^{*}$;
$(T^{-1})^{*}(\psi)(h)=\operatorname{Re}\psi(T^{-1}(h))-i\operatorname{Re}\psi(T^{-1}(ih))$
for $h\in B(H)$ and $\psi\in B(G)^{*}$. Let $\psi\in\sigma(B(G))$ be
arbitrary. As in the first part, we have
$\varphi^{\prime}=(T^{-1})^{*}(\psi)\in\sigma(B(H))$. For every $f\in B(G)$,
we have

$\begin{split}T^{*}(\varphi^{\prime})(f)&=\operatorname{Re}\varphi^{\prime}(T(f))-i\operatorname{Re}\varphi^{\prime}(T(if))\\\
&=\operatorname{Re}(\operatorname{Re}\psi(T^{-1}(T(f)))-i\operatorname{Re}\psi(T^{-1}(iT(f))))\\\
&-i\operatorname{Re}(\operatorname{Re}\psi(T^{-1}(T(if)))-i\operatorname{Re}\psi(T^{-1}(iT(if))))\\\
&=\operatorname{Re}\psi(f)-i\operatorname{Re}\psi(if)=\psi(f).\end{split}$

As $\psi\in\sigma(B(G))$ is arbitrary, we infer that $T^{*}$ maps
$\sigma(B(H))$ onto $\sigma(B(G))$. ∎

######  Definition 2.2.

Let $f\in B(G)$ such that $T(f)$ is invertible in $B(H)$. We define

$\sigma(B(H))^{+}=\\{\varphi\in\sigma(B(H))\mid\varphi(T(if))=i\varphi(T(f))\\}$

and

$\sigma(B(H))^{-}=\\{\varphi\in\sigma(B(H))\mid\varphi(T(if))=-i\varphi(T(f))\\}.$

Note that $\sigma(B(H))^{+}$ and $\sigma(B(H))^{-}$ are independent of the
choice of $f\in B(G)$ by (4). This infers that for any $g\in B(G)$, if
$\varphi\in\sigma(B(H))^{+}$ (resp. $\varphi\in\sigma(B(H))^{-}$), then
$\varphi(T(ig))=i\varphi(T(g))$ (resp. $\varphi(T(ig))=-i\varphi(T(g)$). By
(2), we have $\sigma(B(H))=\sigma(B(H))^{+}\cup\sigma(B(H))^{-}$, which is a
disjoint union.

######  Lemma 2.3.

We have

$T^{*}(i\varphi)=\begin{cases}iT^{*}(\varphi)\quad&\text{if
$\varphi\in\sigma(B(H))^{+}$,}\\\ -iT^{*}(\varphi)\quad&\text{if
$\varphi\in\sigma(B(H))^{-}$}.\end{cases}\\\ $

###### Proof.

For any $f\in B(G)$, we have

$\begin{split}T^{*}(i\varphi)(f)&=\begin{cases}\operatorname{Re}\varphi(T(if))+i\operatorname{Re}\varphi(T(f))\quad&\text{if
$\varphi\in\sigma(B(H))^{+}$,}\\\
-\operatorname{Re}\varphi(T(if))-i\operatorname{Re}\varphi(T(f))\quad&\text{if
$\varphi\in\sigma(B(H))^{-}$,}\end{cases}\\\
&=\begin{cases}iT^{*}(\varphi)(f)\quad&\text{if
$\varphi\in\sigma(B(H))^{+}$,}\\\ -iT^{*}(\varphi)(f)\quad&\text{if
$\varphi\in\sigma(B(H))^{-}$}.\end{cases}\end{split}$

∎

Note that there is a one-to-one correspondence between the unitary
representations of $G$ and the nondegenerate $*$-representations of $C^{*}(G)$
(cf. Theorems 3.9 and 3.11 in [6] and Proposition 2.7.4 [3]). Thus we refer to
the unitary representation $w$ of $G$, which is associated with the universal
representation of $C^{*}(G)$, as the universal representation of $G$. Then it
is proven in [4, (2.10)] that $w(x)$ is the point evaluation at $x$ for the
universal representation $w$ of $G$ as follows:

$w(x)(f)=f(x)$

for any $f\in B(G)$ and $x\in G$ (see also [14, Remark 2.1.6.]).

The spectrum $\sigma(B(G))$ of $B(G)$ is discussed in [14, Lemma 3.2.3]. Some
of these basic properties have been previously proven by Walter [19, Theorem
1]. Recall that the dual space $B(G)^{*}$ of $B(G)$ is identified with
$W^{*}(G)$. Thus, we can assume that $\sigma(B(G))\subset W^{*}(G)$. In the
following, for $\varphi_{1},\varphi_{2}\in\sigma(B(G))$, the multiplication
$\varphi_{1}\varphi_{2}$ is defined as the element in the von Neumann algebra
$W^{*}(G)$. Similarly, the involution $\varphi^{*}$ of
$\varphi\in\sigma(B(G))$ is also defined in $W^{*}(G)$.

######  Lemma 2.4 (Lemma 3.2.3 in [14]).

Let $G$ be a locally compact group.

  1. (1)

If $\varphi_{1},\varphi_{2}\in\sigma(B(G))$, then
$\varphi_{1}\varphi_{2}\in\sigma(B(G))$ and
${\varphi_{1}}^{*}\in\sigma(B(G))$.

  2. (2)

We have $w(G)=\\{\varphi\in\sigma(B(G))\mid\text{$\varphi$ is a unitary in
$W^{*}(G)$}\\}$, where $w(G)=\\{w(x)\mid x\in G\\}$.

We note that $w(e_{1})\in\sigma(B(G))$ (resp. $w(e_{2})\in\sigma(B(H))$) is
the unit of $W^{*}(G)$ (resp. $W^{*}(H)$) since $w$ is the universal
representation of $G$ (resp. $H$). Since $T^{*}$ is a surjective real linear
isometry from $W^{*}(H)$ onto $W^{*}(G)$, Proposition 2.1 in [10] (precisely
it is described in the proof) ensures that $T^{*}(w(e_{2}))$ is a unitary
element of $W^{*}(G)$. Hence, $b\in G$ such that $T^{*}(w(e_{2}))=w(b)$
according to Lemma 2.4 (2). By [10, Proposition 2.1.], there is a Jordan
$*$-isomorphism $J\colon W^{*}(H)\to W^{*}(G)$ and a central projection $p\in
W^{*}(G)$ such that

$T^{*}(\varphi)=w(b)(pJ(\varphi)+(1-p)J(\varphi^{*})),\quad\varphi\in
W^{*}(H).$ (6)

First, we present the following proposition.

######  Proposition 2.5.

Let $G$ and $H$ be locally compact groups. Suppose that $T:B(G)\to B(H)$ is an
isometric real algebra isomorphism with $T^{*}(w(e_{2}))=w(e_{1})$. Then there
exists a topological isomorphism or topological anti-isomorphism $\alpha\colon
H\to G$ such that

$T(f)(x)=f(\alpha(x))\quad f\in B(G),\ x\in H,$

or

$T(f)(x)=\overline{f(\alpha(x))}\quad f\in B(G),\ x\in H.$

######  Remark 2.6.

When $\alpha\colon H\to G$ is a group isomorphism, the map
$x\mapsto\alpha(x^{-1})$ is an anti-group isomorphism, and vice versa.

We will apply lemmas to prove Proposition 2.5. While proving lemmas, we borrow
from Walter’s methods in [19]. However, we require a more general argument
because we consider real linear cases. Thus, we included this proof for
convenience unless the statement could not be directly deduced from other
literature.

From Lemma 2.7 to Lemma 2.13 we assume that $T^{*}(w(e_{2}))=w(e_{1})$. Hence,
we have

$T^{*}(\varphi)=pJ(\varphi)+(1-p)J(\varphi^{*}),\quad\varphi\in W^{*}(H)$ (7)

by (6).

######  Lemma 2.7.

We have

$T^{*}(\varphi)=\begin{cases}pJ(\varphi)\quad&\text{if
$\varphi\in\sigma(B(H))^{+}$,}\\\ (1-p)J(\varphi^{*})\quad&\text{if
$\varphi\in\sigma(B(H))^{-}$}.\end{cases}\\\ $

###### Proof.

Let $\varphi\in\sigma(B(H))^{+}$. By Lemma 2.3, we have

$ipJ(\varphi)+i(1-p)J(\varphi^{*})=iT^{*}(\varphi)=T^{*}(i\varphi)\\\
=pJ(i\varphi)+(1-p)J((i\varphi)^{*})=ipJ(\varphi)-i(1-p)J(\varphi^{*}).$

Hence we obtain

$(1-p)J(\varphi^{*})=0.$

This implies that $T^{*}(\varphi)=pJ(\varphi)$. Let
$\varphi\in\sigma(B(H))^{-}$. By Lemma 2.3, we get

$-ipJ(\varphi)-i(1-p)J(\varphi^{*})=-iT^{*}(\varphi)=T^{*}(i\varphi)\\\
=pJ(i\varphi)+(1-p)J((i\varphi)^{*})=ipJ(\varphi)-i(1-p)J(\varphi^{*}).$

Thus we obtain that $pJ(\varphi)=0$ and $T^{*}(\varphi)=(1-p)J(\varphi^{*})$.

∎

The next corollary is useful for determining whether a given
$\varphi\in\sigma(B(H))$ belongs to $\sigma(B(H))^{+}$ or $\sigma(B(H))^{-}$.

######  Corollary 2.8.

Let $\varphi\in\sigma(B(H))$.

  * (i)

$\varphi\in\sigma(B(H))^{+}$ $\Leftrightarrow$ $T^{*}(\varphi)=pJ(\varphi)$
$\Leftrightarrow$ $(1-p)J(\varphi^{*})=0$.

  * (ii)

$\varphi\in\sigma(B(H))^{-}$ $\Leftrightarrow$
$T^{*}(\varphi)=(1-p)J(\varphi^{*})$ $\Leftrightarrow$ $pJ(\varphi)=0$.

###### Proof.

(i) Let $\varphi\in\sigma(B(H))^{+}$. Lemma 2.7 shows that
$T^{*}(\varphi)=pJ(\varphi)$. If $T^{*}(\varphi)=pJ(\varphi)$ holds then (7)
implies that $(1-p)J(\varphi^{*})=0$. Finally, suppose that
$(1-p)J(\varphi^{*})=0$. We prove that $\varphi\in\sigma(B(H))^{+}$. Suppose
not. Then, $\varphi\in\sigma(B(H))^{-}$. Then by Lemma 2.7, we have
$T^{*}(\varphi)=(1-p)J(\varphi^{*})=0$, as $T^{*}$ is a surjective real linear
isometry, we have that $\varphi=0$ which is impossible. Thus, we obtain that
$\varphi\in\sigma(B(H))^{+}$.
(ii) In a similar way to (i), we can prove the statement. ∎

######  Lemma 2.9.

If $\varphi\in\sigma(B(H))^{+}$ then $\varphi^{*}\in\sigma(B(H))^{+}$. If
$\varphi\in\sigma(B(H))^{-}$ then $\varphi^{*}\in\sigma(B(H))^{-}$.

###### Proof.

Let $\varphi\in\sigma(B(H))^{+}$. By applying Lemma 2.4 , we have
$\varphi^{*}\in\sigma(B(H))$. Since $J$ is a Jordan $*$-isomorphism and $p$ is
a central projection, we obtain
$T^{*}(\varphi^{*})=pJ(\varphi^{*})+(1-p)J((\varphi^{*})^{*})=(pJ(\varphi)+(1-p)J(\varphi^{*}))^{*}=(T^{*}(\varphi))^{*}$.
Lemma 2.7 states that

$T^{*}(\varphi^{*})=(T^{*}(\varphi))^{*}=(pJ(\varphi))^{*}=pJ(\varphi^{*}).$
(8)

Then by Corollary 2.8 we have $\varphi^{*}\in\sigma(B(H))^{+}$. A similar
argument yields $\varphi^{*}\in\sigma(B(H))^{-}$ for
$\varphi\in\sigma(B(H))^{-}$.

∎

######  Lemma 2.10.

We have $J(w(y)w(x))=J(w(y))J(w(x))$ for any $x,y\in H$ or
$J(w(y)w(x))=J(w(x))J(w(y))$ for any $x,y\in H$.

###### Proof.

We have $pT^{*}(\varphi)=pJ(\varphi)$ and
$(1-p)T^{*}(\varphi)=(1-p)J(\varphi^{*})$ by (7). Hence we have

$J(\varphi)=pJ(\varphi)+(1-p)J(\varphi)=pT^{*}(\varphi)+(1-p)T^{*}(\varphi^{*}).$

Let $\varphi\in\sigma(B(H))$. When $\varphi\in\sigma(B(H))^{+}$, Lemma 2.7 and
Lemma 2.9 show that

$J(\varphi)=ppJ(\varphi)+(1-p)pJ(\varphi^{*})=pJ(\varphi)=T^{*}(\varphi).$ (9)

When $\varphi\in\sigma(B(H))^{-}$, we also obtain

$J(\varphi)=p(1-p)J(\varphi^{*})+(1-p)(1-p)J(\varphi)=(1-p)J(\varphi)=T^{*}(\varphi^{*}).$
(10)

Therefore, in either case, we obtain $J(\varphi)\in\sigma(B(G))$ by Lemma 2.1.
Moreover, $J(\sigma(B(H)))=\sigma(B(G))$. In fact, let $\psi\in\sigma(B(G))$
be arbitrary. Then by Lemma 2.1, $\varphi\in\sigma(B(H))$ exists with
$T^{*}(\varphi)=\psi$. If $\varphi\in\sigma(B(G))^{+}$, then (9) asserts that
$J(\varphi)=\psi$. Suppose that $\varphi\in\sigma(B(G))^{-}$. Then we have
$\varphi^{*}\in\sigma(B(G))^{-}$ by Lemma 2.9. Then by (10) we have
$J(\varphi^{*})=T^{*}(\varphi)=\psi$. By Lemma 2.4 we have
$J(\varphi_{1}\varphi_{2}),J(\varphi_{2}\varphi_{1}),J(\varphi_{1})J(\varphi_{2}),J(\varphi_{2})J(\varphi_{1})\in\sigma(B(G))$.
As $J$ is a Jordan $*$-isomorphism, we have

$J(\varphi_{1}\varphi_{2})+J(\varphi_{2}\varphi_{1})=J(\varphi_{1})J(\varphi_{2})+J(\varphi_{2})J(\varphi_{1}).$

On the other hand, in general, it is well known that elements in the spectrum
$\sigma({\mathcal{A}})$ of a commutative Banach algebra $\mathcal{A}$ are
linearly independent. Applying this fact, we can prove by several calculations
that

$J(\varphi_{1}\varphi_{2})=J(\varphi_{1})J(\varphi_{2})$ or
$J(\varphi_{1}\varphi_{2})=J(\varphi_{2})J(\varphi_{1})$.  (11)

Since any Jordan $*$-isomorphism preserves unitary elements and
$J|_{\sigma(B(H))}$ is a bijection from $\sigma(B(H))$ onto $\sigma(B(G))$,
$J|_{\sigma(B(H))}$ maps the unitary elements of $\sigma(B(H))$ onto the
unitary elements of $\sigma(B(G))$. By Lemma 2.4, this implies that
$J|_{w(H)}:w(H)\to w(G)$, which is a bijection. By [15, Theorem 10], as
$J:W^{*}(H)\to W^{*}(G)$ is a Jordan $*$-isomorphism between von Neumann
algebras, there are central projections $Z_{h}\in W^{*}(H)$ and $Z_{g}\in
W^{*}(G)$ such that $J|_{W^{*}(H)Z_{h}}:W^{*}(H)Z_{h}\to W^{*}(G)Z_{g}$ is an
algebra $*$-isomorphism and $J|_{W^{*}(H)(1-Z_{h})}:W^{*}(H)(1-Z_{h})\to
W^{*}(G)(1-Z_{g})$ is an anti-algebra $*$-isomorphism. Then, since $Z_{h}$ is
a central projection, and a Jordan $*$ isomorphism preserves commutativity we
have

$J(abZ_{h})=J(aZ_{h}bZ_{h})=J(aZ_{h})J(bZ_{h})\\\
=J(a)J(Z_{h})J(b)J(Z_{h})=J(a)J(b)J(Z_{h})$ (12)

for every $a,b\in W^{*}(H)$. Similarly, we have

$J(ab(1-Z_{h}))=J(b)J(a)J(1-Z_{h})$ (13)

for every $a,b\in W^{*}(H)$.

For any $x\in H$, we define $A_{x}=\\{y\in H\mid(w(xy)-w(yx))Z_{h}=0\\}$ and
$B_{x}=\\{y\in H\mid(w(xy)-w(yx))(1-Z_{h})=0\\}$. By a simple calculation, it
is easy to check that $A_{x}$ and $B_{x}$ are subgroups of $H$. We prove that
$A_{x}\cup B_{x}=H$. Let $y\in H$ be arbitrary. Then by (11) we have
$J(w(x)w(y))=J(w(x))J(w(y))$ or $J(w(y))J(w(x))$. Suppose first that
$J(w(x)w(y))=J(w(x))J(w(y))$. Then we have

$\begin{split}&J((w(x)w(y)-w(y)w(x))(1-Z_{h}))\\\
&=J(w(x)w(y)(1-Z_{h}))-J(w(y)w(x)(1-Z_{h}))\\\
&=J(w(x)w(y))J(1-Z_{h})-J(w(x))J(w(y))J(1-Z_{h})\\\
&=J(w(x))J(w(y))J(1-Z_{h})-J(w(x))J(w(y))J(1-Z_{h})=0\\\ \end{split}$

since $J(w(x)w(y))=J(w(x))J(w(y))$ and (13) hold. Since $J$ is injective, we
have that $y\in B_{x}$. Suppose contrary that $J(w(x)w(y))=J(w(y))J(w(x))$.
Like in the above discussion with (12), we see that $y\in A_{x}$. We conclude
that $A_{x}\cup B_{x}=H$.

As $A_{x}$ and $B_{x}$ are subgroups of $H$, we obtain that either $A_{x}=H$
or $B_{x}=H$. Suppose not: $A_{x}\neq H$ and $B_{x}\neq H$. Thus, $y_{1}\in
H\setminus A_{x}$ and $y_{2}\in H\setminus B_{x}$ exist. In this case
$y_{1}\in B_{x}$ and $y_{2}\in A_{x}$ since $A_{x}\cup B_{x}=H$. Consider the
element $y_{1}y_{2}\in H$. Then $y_{1}y_{2}\in A_{x}$ or $y_{1}y_{2}\in
B_{x}$. Suppose that $y_{1}y_{2}\in B_{x}$. As $y_{1}^{-1}\in B_{x}$ we obtain
$y_{2}=y_{1}^{-1}y_{1}y_{2}\in B_{x}$, which contradicts the choice of
$y_{2}$. Similarly, we have a contradiction if we assume that $y_{1}y_{2}\in
A_{x}$. We conclude that $A_{x}=H$ or $B_{x}=H$.

In addition as $A=\\{x\in H\mid A_{x}=H\\}$ and $B=\\{x\in H\mid B_{x}=H\\}$
are subgroups of $H$ and $A\cup B=H$, it must be either $A=H$ or $B=H$
according to a similar argument above. When $A=H$, we have
$J(w(x)w(y))=J(w(y))J(w(x))$ for any $x,y\in H$. To prove this, note that
$w(x)w(y)Z_{h}=w(y)w(x)Z_{h}$. By (12) and (13), we have

$\begin{split}J(w(x)w(y))&=J(w(x)w(y))J(Z_{h})+J(w(x)w(y))J(1-Z_{h})\\\
&=J(w(x)w(y)Z_{h})+J(w(x)w(y)(1-Z_{h}))\\\
&=J(w(y)w(x)Z_{h})+J(w(x)w(y)(1-Z_{h}))\\\
&=J(w(y))J(w(x))J(Z_{h})+J(w(y))J(w(x))J(1-Z_{h})\\\
&=J(w(y))J(w(x)).\end{split}$

By a similar argument, we conclude that $B=H$ implies that
$J(w(x)w(y))=J(w(x))J(w(y))$ for any $x,y\in H$. ∎

######  Definition 2.11.

We say that $J$ is multiplicative $($resp. anti-multicative$)$ if
$J(w(x)w(y))=J(w(x))J(w(y))$ $($resp. $J(w(x)w(y))=J(w(y))J(w(x)))$ for every
pair $x,y\in H$. We define $\alpha:H\to G$ by $\alpha(x)=w^{-1}\circ J\circ
w(x)$, $x\in H$.

Note that $\alpha$ is a group isomorphism if $J$ is multiplicative and an
anti-group isomorphism if $J$ is anti-multiplicative.

######  Lemma 2.12.

If $w(x)\in\sigma(B(H))^{+}$ then $T(f)(x)=f(\alpha(x))$. If
$w(x)\in\sigma(B(H))^{-}$ then $T(f)(x)=\overline{f(\alpha(x^{-1}))}$.

###### Proof.

For any $f\in B(G)$, note that

$f(\alpha(x))=w(\alpha(x))(f)=J(w(x))(f)\\\
=pT^{*}(w(x))(f)+(1-p)T^{*}(w(x)^{*})(f).$

If $w(x)\in\sigma(B(H))^{+}$ then we have by Lemma 2.3 and Lemma 2.7 that

$\begin{split}f(\alpha(x))&=pT^{*}(w(x))(f)+(1-p)T^{*}(w(x)^{*})(f)\\\
&=ppJ(w(x))(f)+(1-p)pJ(w(x)^{*})(f)\\\
&=pJ(w(x))(f)=T^{*}(w(x))(f)=\operatorname{Re}w(x)(T(f))-i\operatorname{Re}w(x)(T(if))\end{split}$

and

$\begin{split}if(\alpha(x))&=iT^{*}(w(x))(f)=T^{*}(iw(x))(f)\\\
&=\operatorname{Re}iw(x)(T(f))-i\operatorname{Re}iw(x)(T(if))\\\
&=-\operatorname{Im}w(x)(T(f))+i\operatorname{Im}w(x)(T(if)).\end{split}$

Therefore we have

$f(\alpha(x))=\operatorname{Re}f(\alpha(x))+i\operatorname{Im}f(\alpha(x))\\\
=\operatorname{Re}T(f)(x)+i\operatorname{Im}T(f)(x)=T(f)(x)$

for any $f\in B(G)$ and $x\in H$ with $w(x)\in\sigma(B(H))^{+}$. If
$w(x)\in\sigma(B(H))^{-}$ then we have by Lemma 2.3 and Lemma 2.7 that

$\begin{split}f(\alpha(x))&=pT^{*}(w(x))(f)+(1-p)T^{*}(w(x)^{*})(f)\\\
&=p(1-p)J(w(x)^{*})(f)+(1-p)(1-p)J(w(x))(f)\\\
&=T^{*}(w(x)^{*})(f)=\operatorname{Re}w(x^{-1})(T(f))-i\operatorname{Re}w(x^{-1})(T(if))\end{split}$

and

$\begin{split}-if(\alpha(x))&=-iT^{*}(w(x)^{*})(f)=T^{*}(iw(x^{-1}))(f)\\\
&=\operatorname{Re}iw(x^{-1})(T(f))-i\operatorname{Re}iw(x^{-1})(T(if))\\\
&=-\operatorname{Im}w(x^{-1})(T(f))+i\operatorname{Im}w(x^{-1})(T(if)).\end{split}$

We obtain that

$\overline{f(\alpha(x))}=\operatorname{Re}f(\alpha(x))-i\operatorname{Im}f(\alpha(x))\\\
=\operatorname{Re}T(f)(x^{-1})+i\operatorname{Im}T(f)(x^{-1})=T(f)(x^{-1}).$
(14)

∎

######  Lemma 2.13.

Suppose that $w(x),w(y)\in\sigma(B(H))^{+}$ $($resp. $\sigma(B(H))^{-}$$)$.
Then $w(xy)\in\sigma(B(H))^{+}$ $($resp. $\sigma(B(H))^{-}$$)$. We have a
dichotomy $w(H)\subset\sigma(B(H))^{+}$ or $w(H)\subset\sigma(B(H))^{-}$.

###### Proof.

For any $w(x),w(y)\in\sigma(B(H))^{+}$, we have

$\begin{split}&(1-p)J(w(xy)^{*})=(1-p)J(w(y)^{*}w(x)^{*})\\\
&=\begin{cases}(1-p)J(w(y)^{*})(1-p)J(w(x)^{*})\quad&\text{if $J$ is
multiplicative,}\\\ (1-p)J(w(x)^{*})(1-p)J(w(y)^{*})\quad&\text{if $J$ is
anti-multiplicative}\end{cases}\\\ &=0\end{split}$

by Corollary 2.8. Applying Corollary 2.8 again, we conclude that
$w(xy)\in\sigma(B(H))^{+}$. Note that $w(x)^{*}=w(x^{-1})$ for
$x\in\sigma(B(H))$. Hence if $x\in H$ with $w(x)\in\sigma(B(H))^{+}$, then
$w(x)^{*}\in\sigma(B(H))^{+}$ by Lemma 2.9, so that
$w(e_{2})=w(xx^{-1})=w(x)w(x)^{*}\in\sigma(B(H))^{+}$ . By a similar argument,
we obtain that $w(xy)\in\sigma(B(H))^{-}$ for any
$w(x),w(y)\in\sigma(B(H))^{-}$. Thus, the existence of an element $x\in H$
with $w(x)\in\sigma(B(H))^{-}$ similarly ensures that
$w(e_{2})\in\sigma(B(H))^{-}$. We conclude that $w(e_{2})\in\sigma(B(H))^{+}$
or $w(e_{2})\in\sigma(B(H))^{-}$. As
$\sigma(B(H))^{+}\cap\sigma(B(H))^{-}=\emptyset$, we conclude that either
$w(H)\subset\sigma(B(H))^{+}$ or $w(H)\subset\sigma(B(H))^{-}$ holds.

∎

Now we prove Proposition 2.5.

###### Proof of Proposition 2.5.

By Lemma 2.13 we have a dichotomy $w(H)\subset\sigma(B(H))^{+}$ or
$w(H)\subset\sigma(B(H))^{-}$. When $w(H)\subset\sigma(B(H))^{+}$, by Lemma
2.12 we have

$T(f)(x)=f(\alpha(x)),\quad x\in H.$

When $w(H)\subset\sigma(B(H))^{-}$, by Lemma 2.12 we have

$T(f)(x)=\overline{f(\alpha(x^{-1}))},\quad x\in H.$

By Remark 2.6, we note that $x\mapsto\alpha(x^{-1})$ is also a group
isomorphism or an anti-group isomorphism.

Let $U$ be an open neighborhood of $e_{1}$ on $G$. There is a continuous
positive definite function $f_{0}$ on $G$, which has compact support, such
that $0\leq f_{0}\leq 1$ with $f_{0}(e_{1})=1$ and $f_{0}=0$ on $G\setminus
U$. For a net $\\{x_{\beta}\\}$ with $x_{\beta}\to e_{2}$, we have
$f_{0}(\alpha(x_{\beta}))\to f_{0}(\alpha(e_{2}))$ as $T(f_{0})\in B(H)$. This
implies that $\alpha(x_{\beta})\to\alpha(e_{2})=e_{1}$. Hence $\alpha$ is
continuous. By applying a similar argument to $\alpha^{-1}$, we obtain that
$\alpha$ is a homeomorphism. Thus we complete the proof. ∎

Finally, we characterize isometric real algebra isomorphisms on Fourier-
Stieltjes algebras. Theorem 2.14 is a real-linear-counterpart of the theorem
of Walter [19, Theorem 2].

######  Theorem 2.14.

Let $G$ and $H$ be locally compact groups. The map $T:B(G)\to B(H)$ is an
isometric real algebra isomorphism if and only if there is a topological
isomorphism or a topological anti-isomorphism $\alpha:H\to G$ and $b\in G$
such that

$T(f)(x)=f(b\alpha(x))\quad f\in B(G),\ x\in H,$

or

$T(f)(x)=\overline{f(b\alpha(x))}\quad f\in B(G),\ x\in H.$

###### Proof.

If $T:B(G)\to B(H)$ takes the form $T(f)(x)=f(b\alpha(x))$, where $\alpha:H\to
G$ is either a topological group isomorphism or a topological anti-group
isomorphism, then [19, Corollary of Theorem 3] shows that $T$ is an isometric
complex algebra isomorphism from $B(G)$ onto $B(H)$.

Similarly, if $T(f)(x)=\overline{f(b\alpha(x))}$, then $T$ is an isometric
conjugate-linear algebra isomorphism from $B(G)$ onto $B(H)$.

We prove the converse statement. Suppose that $T:B(G)\to B(H)$ is an isometric
real algebra isomorphism. According to (6), $b\in G$ such that
$T^{*}(w(e_{2}))=w(b)$.

We define $T_{r}:B(G)\to B(G)$ by $T_{r}(f)(x)=f(b^{-1}x)$. Since the map
$x\mapsto b^{-1}x$ is an affine map on $G$, $T_{r}$ is an isometric
isomorphism on $B(G)$ as in the same argument as the first part of the proof
(cf. [19]). It follows that $T\circ T_{r}\colon B(G)\to B(H)$ is an isometric
real algebra isomorphism. Then we obtain the following equation:

$\begin{split}(T\circ T_{r})^{*}(w(e_{2}))f&=\operatorname{Re}w(e_{2})(T\circ
T_{r}(f))-i\operatorname{Re}w(e_{2})(T\circ T_{r}(if))\\\
&=T^{*}(w(e_{2}))(T_{r}(f))=w(e_{1})f\end{split}$

for any $f\in B(G)$. Thus

$(T\circ T_{r})^{*}(w(e_{2}))=w(e_{1}).$

By applying Proposition 2.5 to $T\circ T_{r}\colon B(G)\to B(H)$, we obtain a
topological group isomorphism or a topological anti-group isomorphism
$\alpha\colon H\to G$ such that

$T\circ T_{r}(f)(x)=f(\alpha(x)),\quad x\in H.$

or

$T\circ T_{r}(f)(x)=\overline{f(\alpha(x))},\quad x\in H.$

Thus, for any $f\in B(G)$ and $x\in H$, we obtain the desired forms

$Tf(x)=T\circ T_{r}\circ
T_{r}^{-1}(f)(x)=T_{r}^{-1}(f)(\alpha(x))=f(b\alpha(x)),$

or

$Tf(x)=T\circ T_{r}\circ
T_{r}^{-1}(f)(x)=\overline{T_{r}^{-1}(f)(\alpha(x))}=\overline{f(b\alpha(x))}.$

∎

We also characterize isometric real algebra isomorphisms on Fourier algebras
$A(G)$. For a locally compact group $G$, $A(G)$ is the space of all matrix
coefficients of the left regular representation $\lambda_{G}$ from $G$ into
the unitary operators on $L^{2}(G)$.

######  Theorem 2.15.

Let $G$ and $H$ be locally compact groups. The map $T\colon A(G)\to A(H)$ is
an isometric real algebra isomorphism if and only if there is a topological
isomorphism or a topological anti-isomorphism $\alpha\colon H\to G$ and $b\in
G$ such that

$T(f)(x)=f(b\alpha(x))\quad f\in A(G),\ x\in H,$

or

$T(f)(x)=\overline{f(b\alpha(x))}\quad f\in A(G),\ x\in H.$

###### Proof.

We note that the spectrum $\sigma(A(G))$ of $A(G)$ is homeomorphic to $G$ [14,
Theorem 2.3.8]. The dual space of $A(G)$ is isometrically isomorphic as Banach
space to $VN(G)$, where $VN(G)$ is the von Neumann algebra generated by
$\lambda_{G}$. Any isometric real algebra isomorphism $T:A(G)\to A(H)$ induces
a surjective real linear isometry $T^{*}\colon VN(H)\to VH(G)$ by a similar
formula as (1). We prove the theorem by an argument essentially identical to
that given in Theorem 2.14, replacing $W^{*}(G)$ and $W^{*}(H)$ by $VH(G)$ and
$VN(H)$ respectively. ∎

##  3\. isometries between groups of invertible elements

The study in [7] initially focused on isometries between groups of invertible
elements in Banach algebras. Later, this work was generalized and referenced
in [8]. In the case where $G$ and $H$ are locally compact abelian groups,
$B(G)$ and $B(H)$ represent the Fourier-Stieltjes transform of the measure
algebra $M(\widehat{G})$ and $M(\widehat{H})$ for their respective dual groups
$\widehat{G}$ and $\widehat{H}$, respectively. It is worth noting that $B(G)$
and $B(H)$ are isometrically isomorphic to $M(\widehat{G})$ and
$M(\widehat{H})$, respectively. Suppose that $T_{0}$ is a surjective isometry
between open subgroups $\mathfrak{A}$ and $\mathfrak{B}$ of the groups of
invertible elements in $B(G)$ and $B(H)$, respectively. Applying [9, Theorem
3.1], $\widehat{G}$ and $\widehat{H}$ are topologically isomorphic. This
implies that $G$ and $H$ are topologically equivalent through the Pontryagin
duality theorem. Theorem 3.1 generalizes this result to Fourier-Stieltjes
algebras on locally compact groups.

We denote the group of all invertible elements in the Fourier-Stieltjes
algebra $B(G)$ by $B(G)^{-1}$.

######  Theorem 3.1.

Let $G$ and $H$ be locally compact groups. Let $\mathfrak{A}$ and
$\mathfrak{B}$ be open subgroups of $B(G)^{-1}$ and $B(H)^{-1}$, respectively.
Suppose that $T_{0}$ is a surjective isometry from $\mathfrak{A}$ onto
$\mathfrak{B}$. Then $T_{0}$ is extended to an isometric real algebra
isomorphism from $B(G)$ onto $B(H)$ and there is a topological isomorphism or
a topological anti-isomorphism $\alpha:H\to G$ and $b\in G$ such that

$T_{0}(f)(x)=T_{0}(1)f(b\alpha(x))\quad f\in\mathfrak{A},\ x\in H,$

or

$T_{0}(f)(x)=T_{0}(1)\overline{f(b\alpha(x))}\quad f\in\mathfrak{A},\ x\in H.$

###### Proof.

First, we note that $B(G)$ and $B(H)$ are unital semisimple commutative Banach
algebras. Applying [7, Theorem 3.3] or [8, Corollary 5.2] for $A=B(G)$ and
$B=B(H)$ we have that $(T_{0}(1))^{-1}T_{0}$ extends to an isometric real
algebra isomorphism $T$ from $A$ onto $B$. By Theorem 2.14 we obtain a
topological isomorphism or a topological anti-isomorphism $\alpha\colon H\to
G$ and $b\in G$ such that

$T(f)(x)=f(b\alpha(x))\quad f\in B(G),\ x\in H,$

or

$T(f)(x)=\overline{f(b\alpha(x))}\quad f\in B(G),\ x\in H.$

As $T$ is an extension of $(T_{0}(1))^{-1}T_{0}$ we have the desired
equalities. ∎

We apply [7, Theorem 3.3] or [8, Corollary 5.2] in the proof above.

######  Corollary 3.2.

For locally compact groups $G$ and $H$, suppose that open subgroups
$\mathfrak{A}$ and $\mathfrak{B}$ of $B(G)^{-1}$ and $B(H)^{-1}$ are
isometric. Then $G$ and $H$ are topologically isomorphic to each other.

###### Proof.

Theorem 3.1 yields either a topological isomorphism of $H$ onto $G$, or a
topological anti-isomorphism of $H$ onto $G$. In the latter case, following
the anti-isomorphism with the inverse map $x\in G\mapsto x^{-1}\in G$ yields a
topological isomorphism of $H$ onto $G$ as Remark 2.6. ∎

### Acknowledgments

The first author was supported by JSPS KAKENHI Grant Numbers JP19K03536. The
second author was supported by JSPS KAKENHI Grant Numbers JP21K13804.

## References

  * [1] P. J. Cohen, On homomorphisms of the group algebras, _Amer. J. Math._ 82 (1960), 213–226.
  * [2] M. Daws, Completely bounded homomorphisms of the Fourier algebra revisited, _J. Group Theory_ 25 (2022), no.3, 579–600.
  * [3] J. Dixmier, $C^{*}$-algebras, North-Holland Mathematical Library, Vol. 15. Amsterdam-New York-Oxford, North-Holland Publishing Co., 1977.
  * [4] P. Eymard, L’algèbre de Fourier d’un groupe localement compact, _Bull. Soc. Math. France_ 92 (1964), 181–236.
  * [5] G. B. Folland, Real Analysis, Pure and Applied Mathematics. A Wiley-Interscience Series of Texts, Monographs, and Tracts. New York, NY: Wiley. (1999).
  * [6] G. B. Foland, A course in abstract harmonic analysis, 2nd updated edition, Textbooks in Mathematics, Boca Raton, FL: CRC Press. (2016)
  * [7] O. Hatori, Isometries between groups of invertible elements in Banach algebras, _Studia.Math._ 194 (2009), no.3, 293–304.
  * [8] O. Hatori, Algebraic properties of isometries between groups of invertible elements in Banach algebras, _J. Math. Anal. Appl._ 376 (2011), no.1, 84–93.
  * [9] O. Hatori, New criteria for equivalence of locally compact abelian groups, _J. Group Theory_ 15 (2012), no.2, 271–277.
  * [10] O. Hatori and K. Watanabe, Isometries between groups of invertible elements in $C^{*}$-algebras, _Studia.Math._ 209 (2012), no.2, 103–106.
  * [11] E. Hewitt and K. Stromberg, Real and abstract analysis. A modern treatment of the theory of functions of a real variable. 2nd printing corrected, Berlin-Heidelberg-New York: Springer-Verlag. VIII, (1969).
  * [12] M. Ilie, On Fourier algebra homomorphisms, _J. Funct. Anal._ 213 (2004), 88–110.
  * [13] M. Ilie and N. Spronk, Completely bounded homomorphisms of the Fourier algebras, _J. Funct. Anal._ 225 (2005), no.2, 480–499.
  * [14] E. Kaniuth and A. T.-M. Lau, Fourier and Fourier-Stieltjes Algebras on Locally Compact groups, Mathematical Surveys and Monographs 231, American Mathematical Society, Providence, Rhode Island, (2018).
  * [15] R. Kadison, Isometries of operator algebras, _Ann. of Math._ (2) 54 (1951), 325–338.
  * [16] H. Le Pham, Contractive homomorphisms of Fourier algebras, _Bull. London Math. Soc._ 42 (2010), 937–947.
  * [17] N. Spronk, Dual space of $L^{1}$, https://www.math.uwaterloo.ca/~nspronk/math451/Lone_dual.pdf
  * [18] M. Takesaki, Theory of operator algebras. I, Springer-Verlag, New York-Heidelberg, (1979).
  * [19] M. E. Walter, W*-algebras and nonabelian harmonic analysis, _J. Funct. Anal._ 11 (1972), 17–38.


# Towards construction of analog solver of Schroedinger and Ginzburg-Landau
equation based on Long Line

Łukasz Pluszyński (I  III)  Krzystof Pomorski(II  III)

###### Abstract

The analog electronic computers are a type of circuitry used to calculate
specific problems using the physical relationships between the voltages and
currents following classical laws of physics. One specific class of these
circuits are computers based on the interactions between passive circuit
elements. Models presented by Gabriel Kron in 1945 are the example of using
such passive elements to construct a solver for the problem of free quantum
particles confined by rectangular potential. Numerical validation of Kron’s
second model is conducted for different shapes of particle confining
potential. Kron’s model is generalized by introduction of non-linear resistive
elements what implies deformation of Schroedinger equation solution into
Ginzburg-Landau form.

###### Index Terms:

analog computer, differential equation, quantum mechanics simulation, analog
electronics, Schroedinger model, Ginzburg-Landau equation

##  I Motivation behind development of analog computers modeling quantum
systems

The analog computers are the class of the computers that operate on the
continuous signals instead of the discrete numbers such as the most commonly
used binary based digital electronics. Their greatest strength of analog
paradigm is their ability to utilise the known physical relationships between
their components to solve the computationally difficult equations as integro-
differential equations without need of reliance on technically costly
approximation of real number by discrete states or technical cost of
differentiation or integration operators . One of the most commonly known
example is the case of use of the relationship between input and output
voltages of the operational amplifier with the correctly placed capacitor to
get the integrals or the derivatives of the electric signal’s function.

There is inherent ability of the analog electronics to represent various
differential equations (and thus referring to wide class of problems) and this
potential is being underutilised due to the difficulty of integrating them
with the digital systems, analog electronics susceptibility to the
interference and the constantly present question of the occurrence of
effectively built-in parasitic resistances.

Presented work leads to the creation of the netlist model of the analog solver
for the Schrodinger equation based on Kron’s second model [8] and brings its
generalisation via numerical validation for classes of effective potential
going beyond rectangular potential.

##  II Kron’s second model of the analog hardware solver for Schroedinger
Equation

As established by Garbriel Kron in his studies [8] we are able to represent
the distribution of one dimensional wave-function of Schroedinger equation for
single particle in effective potential by classical non-uniform long line with
usage of inductance and capacities elements only. Indeed the second Kron’s
model points out classical long line with the driving signal (presence of
voltage source or current source) of constant frequency and amplitude. In this
model we have presence of inductance in horizontal direction, while we set
capacitance and inductance in parallel configuration in vertical direction as
depicted in Fig.2. Kron’s model leads to representation uses simplistic
Hamiltonian of Schroedinger equation of $(\hat{H}(x)-E)\psi(x)=0$ that can be
represnted in discrete form as

$\displaystyle\Delta x\hat{H}(x)\psi(x)-\Delta
xE(t)\psi(x)=0,\hat{H}(x)=\hat{T}(x)+\hat{V}(x),$
$\displaystyle\frac{-\hbar^{2}}{2m}(\psi^{\prime}(x+\Delta
x)-\psi^{\prime}(x))=\Delta x(E-V_{p}(x))\psi(x),$ $\displaystyle I(x+\Delta
x,t)-I(x,t)=\Delta
I(x,t)=\frac{V(x)}{Z(x)_{C_{1}||L_{1}}}=(\sqrt{-1}C_{1}\omega+\frac{1}{\sqrt{-1}\omega
L_{1}(x)})V(x)=\Delta x(E-V_{p}(x)),$

with central preassumption that kinetic energy term in Schroedinger equation
is corresponding to electric current phasor change to $I(x+\Delta x,t)-I(x,t)$
that leads to current phasor $\Delta I(x)$ flowing vertically every lattice
step $\Delta x$ in the form

$\displaystyle\frac{-\hbar^{2}}{2m}(\psi^{\prime}(x+\Delta
x)-\psi^{\prime}(x))=I(x+\Delta x,t)-I(x,t)=\Delta I(x,t),$

what implies

$\displaystyle(I(x+\Delta x,t)-I(x,t))=\Delta
I(x,t)=\frac{V(x)}{Z(x)_{C_{1}||L_{1}}}=(\sqrt{-1}C_{1}\omega+\frac{1}{\sqrt{-1}\omega
L_{1}(x)})V(x)=\Delta x(E-V_{p}(x)),$

and under assumption $\psi(x)=V(x)$ we arrive to dependence

$\displaystyle\frac{-\hbar^{2}}{2m}(V(x+\Delta x)-V(x))=\Delta
I(x),(\frac{d}{dx}V)=-\frac{2m}{\hbar^{2}}\frac{\Delta I(x)}{\Delta
x}=\frac{\Delta Z_{L}(x)}{\Delta x}I(x)=\sqrt{-1}L\omega I(x),$ (4)

where: $\hat{V}_{p}$ is potential energy operator, $\hat{T}$ is kinetic energy
operator equal to $-\frac{\hbar^{2}}{2m}\frac{d^{2}}{dx^{2}}$ or
$\frac{p^{2}}{2m}$, $\hat{H}$ is Hamiltonian equivalent to $\hat{V}+\hat{T}$,
E is energy eigevalue, $\hat{p}$ is momentum operator equivalent to
$-i\hbar\frac{d}{dx}$ and $\Delta x$ represents discrete step in position
coordinates, while I(x) and V(x) are phasors of electric current and voltage.
We have made explicit assumption that phasor of voltage across upper and lower
branch is representing wave-function. Furthermore in case of long line
mimicking Schroedinger equation as by [1], [2], [3] we shall imply that
$\frac{1}{\omega L_{1}(x)}=V_{p}(x)\Delta x$ and that $-C_{1}\omega\Delta x=E$
. Under assumption of applicability of impedance concept it turns out that
horizontal inductance impedance $X_{L}=\frac{2m}{\hbar^{2}}\Delta x$ is
related to kinetic energy operator, vertical inductance
$X_{L1}=\frac{1}{V}\Delta x$ is inverse of potential energy and that impedance
of capacitor $-X_{C}=-\frac{1}{E}\Delta x$ stands for the total energy. Such
results are obtained by derivation of Kron’s model conducted in Section IV.
The basic representation of the energy operators in one dimension for time
independent Schroedinger equation in Kron’s second model is using the serial
coils in horizontal direction to represent kinetic energy operator,
perpendicular to them inductive coils for representation of potential energy
operator and capacitors connected in parallel to already mentioned vertical
inductors for representation of total energy operator E as depicted in Fig.2.

It is worth noting that Figure 2 depicts Kron’s representations of Hamiltonian
operator (a) and $H-E$ operator (b) for the particle moving in one dimension
represented by classical electric circuit.

Figure 1: Hamiltonian operator (a) and $H-E$ operator(b) as described by Kron
[8].

Figure 2: Conceptual schematic of circuit simulating particle in potential
well in one dimension with voltage across lower and upper branch as wave-
function values as described by Kron [8].

For purposes of the simulation reflected in analog hardware configuration, the
location of the voltage signal generator will be the point of lowest potential
energy and in this case represented either by the node with the highest
parallel inductance or the node without parallel coil. This setup for basic
simulation of particle in rectangular well of potential is depicted in Fig.4.

The wave-functions $\psi$ being Hamiltonian eigenstate we are looking for can
only be electrically measured in invasive way as potential difference between
the nodes of upper and lower branch of long line. Due to the relationship
between electronic components, the circuit solves the equation only when the
current running through the function generator driving the voltage signal
equals zero [8].

It is worth noting that we can transit from Schroedinger equation to Ginzburg-
Landau equation by

$\displaystyle\frac{-\hbar^{2}}{2m}\frac{d^{2}}{dx^{2}}\psi(x)+V_{p}(x)\psi(x)=E\psi(x),$
$\displaystyle\frac{-\hbar^{2}}{2m}\frac{d^{2}}{dx^{2}}\psi(x)+\alpha(x)\psi(x)+\beta(x)|\psi(x)|^{2}\psi(x)=0,$

By preassuming transition
$V_{p}(x)\rightarrow(\alpha(x)+\beta(x)|\psi(x)|^{2}\psi(x)-E)$ we obtain
transition from Schroedinger to Ginzburg-Landau equation. Alternatively we can
preassume transition $E\rightarrow V(x)-\alpha(x)-\beta(x)|\psi(x)|^{2}$ as
implementing transition from Schroedinger to non-linear Schroedinger equation.
Both transitions can be encoded in certain dependence of $L_{1}$ and $C_{1}$
linear density with certain space dependence and with certain voltage
dependence. Transition from Schroedinger to Ginzbug-Landau formalims can mean
transition from semiconducor to superconductor. Good candidate for circuit
elements implementing given transition (Schrodinger to GL) are symmetrized
diodes (two diodes connected in antiparallel way) as way of inducing non-
linearity to the circuit. Furthermore one can introduce 3 non-linear elements
in vertical direction: resistance, capacitance and inductance with x
coordinate dependence and one can expect various forms of Ginzburg-Landau like
equation solution to emerge in dependence on nature of those elements.

##  III LTspice Simulation of Kron’s second circuit

It is quite straightforward to implement previously defined Kron’s model into
schematics as depicted in Fig.4, which was developed and tested in LTspice.

Figure 3: Schematic of twenty five (25) node circuit for simulating particle
in rectangular potential well [1].

The parameters of the one among many existing optimal simulations found by
various trials can be given as follows:

  * •

Inductance simulating well of potential:$1mH$

  * •

Inductance simulating the kinetic energy operator: $1mH$

  * •

Capacitance simulating the total energy:$1uF$

  * •

Frequency of the driving signal:$10Hz$

  * •

Amplitude of the driving signal:$2V$

  * •

Nodes 1 to 25 are points of measurement of voltage and thus level of
discretization of continuous position space

  * •

Maximal timestep of simulation:$0.00001seconds$

The data gathered from simulation took form of the voltage over time in each
of tested nodes alongside the current in the node containing the signal
generator.

It should be noted that some deformations of wave-function distribution is
observed in simulation results due to comparatively low sampling resolution of
the circuit.

Figure 4: Obtained numerical simulation results for the particle trapped in
rectangular potential well (upper picture), harmonic potential (lower left)
and V shape potential (lower right) with use of Kron’s second model
implemented in LTSpice environment [7],[5].

Observations about results of simulation experiments can be concluded by
following :

  * •

Circuit typically needs few cycles to stabilise. This time seems proportional
to capacitance’s and inductance’s of used components.

  * •

There is visible spike of voltage at the point where the signal generator is
located.

  * •

Achieving measurement at the point were current in node V2 equals exactly zero
was effectively impossible due to limits of simulation software. This factor
can be blamed on final plot being deformed via voltage spike in the node
containing the signal generator.

  * •

Resultant solution was acceptably close to the expected one quality wise to
make a decision about conducting the further experiments with use of stronger
hardware for simulation.

##  IV Derivation of Kron’s model expressed by long-line model and its
generalization towards Ginzburg-Landau equation

Non-dissipative Schroedinger equation having real value potential and
eigenenrgy states can be expressed by Kron’s model with inductance in
horizontal direction as well as by capacitance and non-uniform inductances in
vertical direction.

We can deform the long line by introduction of non-linear resistive elements
in series with inductance’s places horizontally. Let us refer to long-line
model as depicted in Fig.6 that can be characterized by equations

$\displaystyle dV(x)=dx\sqrt{-1}L\omega I(x),$ $\displaystyle
dI(x)=dx(\frac{1}{\sqrt{-1}L_{1}\omega}+\sqrt{-1}\omega C_{1})V(x)$ (6)

which implies

$\displaystyle\frac{d}{dx}V(x)=\sqrt{-1}L\omega I(x),$
$\displaystyle\frac{d}{dx}I(x)=(\frac{1}{\sqrt{-1}L_{1}\omega}+\sqrt{-1}\omega
C_{1})V(x)$ (7)

and that results in equation

$\displaystyle\frac{d^{2}}{dx^{2}}V(x)=\sqrt{-1}L\omega(\frac{1}{\sqrt{-1}L_{1}\omega}+\sqrt{-1}\omega
C_{1})V(x).$

Last equation can be rewritten to be of the form

$\displaystyle\frac{d^{2}}{dx^{2}}V(x)=(\frac{L}{L_{1}}-LC_{1}\omega^{2})V(x)=k_{1}(\omega)^{2}V(x),$
$\displaystyle k_{1}(\omega)=\sqrt{(\frac{L}{L_{1}}-LC_{1}\omega^{2})}.$ (9)

and has analytic solution as

$\displaystyle V(x)=a_{1}e^{+k_{1}x}+a_{2}e^{-k_{1}x}=$
$\displaystyle=a_{1}e^{+\sqrt{(\frac{L}{L_{1}}-LC_{1}\omega^{2})}x}+a_{2}e^{-\sqrt{(\frac{L}{L_{1}}-LC_{1}\omega^{2})}x}=b_{1}sinh(k_{1}x)+b_{2}cosh(k_{1}x)=$
$\displaystyle=b_{1}sinh(\sqrt{(\frac{L}{L_{1}}-LC_{1}\omega^{2})}x)+b_{2}cosh(\sqrt{(\frac{L}{L_{1}}-LC_{1}\omega^{2})}x),$
$\displaystyle I(x)=c_{1}e^{+k_{1}x}+c_{2}e^{-k_{1}x}=$
$\displaystyle=c_{1}e^{+\sqrt{(\frac{L}{L_{1}}-LC_{1}\omega^{2})}x}+c_{2}e^{-\sqrt{(\frac{L}{L_{1}}-LC_{1}\omega^{2})}x}=$
$\displaystyle=d_{1}sinh(+\sqrt{(\frac{L}{L_{1}}-LC_{1}\omega^{2})}x)+d_{2}cosh(-\sqrt{(\frac{L}{L_{1}}-LC_{1}\omega^{2})}x).$
(10)

Figure 5: Krons model in relation to non-dissisipative non-uniform long line
model based on passive linear elements that are space dependent: inductance
and capacitance.

Figure 6: Krons model in relation to long line model that is subjected to non-
linear deformation introduced by non-linear resistive elements.

It is not hard to generalize obtained result for the case of R, L and C
circuits components in series on upper cable branch and for the case of
circuits elements in parallel on inter-connecting branch with values of
$R_{1}$, $L_{1}$ and $C_{1}$. In such a case we arrive to equations

$\displaystyle dV(x)=dx(\sqrt{-1}L\omega+R+\frac{1}{i\omega C})I(x),$
$\displaystyle dI(x)=dx(\frac{1}{\sqrt{-1}L_{1}\omega}+\sqrt{-1}\omega
C_{1}+\frac{1}{R_{1}})V(x)$ (11)

that can be decoupled, so one deals with two independent equations

$\displaystyle\frac{d^{2}}{dx^{2}}V(x)=(\sqrt{-1}L\omega+R+\frac{1}{i\omega
C})(\frac{1}{\sqrt{-1}L_{1}\omega}+\sqrt{-1}\omega
C_{1}+\frac{1}{R_{1}})V(x)=k_{1q}(\omega)^{2}V(x),$
$\displaystyle\frac{d^{2}}{dx^{2}}I(x)=(\sqrt{-1}L\omega+R+\frac{1}{i\omega
C})(\frac{1}{\sqrt{-1}L_{1}\omega}+\sqrt{-1}\omega
C_{1}+\frac{1}{R_{1}})I(x)=k_{1q}(\omega)^{2}I(x).$ (12)

In the next step we can incorporate nonlinear elements in previously
considered long line models. We have situation as depicted in Fig.6. At first
we consider linear circuit with linear density (per unit length) of $L(x)$,
$L_{1}(x)$ and $C_{1}(x)$. We have equations

$\displaystyle dV(x)=dx\sqrt{-1}L(x)\omega I(x),$ $\displaystyle
dI(x)=dx(\frac{1}{\sqrt{-1}L_{1}(x)\omega}+\sqrt{-1}\omega C_{1}(x))V(x)$ (13)

what implies

$\displaystyle\frac{1}{\sqrt{-1}L(x)\omega}\frac{d}{dx}V(x)=I(x),$
$\displaystyle\frac{d}{dx}(\frac{1}{\sqrt{-1}L(x)\omega}\frac{d}{dx}V(x))=(\frac{1}{\sqrt{-1}L_{1}(x)\omega}+\sqrt{-1}\omega
C_{1}(x))V(x).$ (14)

Last equation implies

$\displaystyle(\frac{1}{\sqrt{-1}L(x)\omega})\frac{d^{2}}{dx^{2}}V(x)+(\frac{d}{dx}V(x))\frac{d}{dx}(\frac{1}{\sqrt{-1}L(x)\omega})=(\frac{1}{\sqrt{-1}L_{1}(x)\omega}+\sqrt{-1}\omega
C_{1}(x))V(x).$ (15)

and can be simplified to be

$\displaystyle(\frac{1}{L(x)})\frac{d^{2}}{dx^{2}}V(x)+(\frac{d}{dx}V(x))\frac{d}{dx}(\frac{1}{L(x)})=(\frac{1}{L_{1}(x)}-\omega^{2}C_{1}(x))V(x).$
(16)

and we arrive to

$\displaystyle-(\frac{1}{L(x)})\frac{d^{2}}{dx^{2}}V(x)+(\frac{d}{dx}V(x))(\frac{d}{dx}L(x))(\frac{1}{L(x)^{2}})+\frac{1}{L_{1}(x)}V(x)=(+\omega^{2}C_{1}(x))V(x).$
(17)

Under the assumption $L(x)=constants=L$ we have

$\displaystyle-(\frac{1}{L(x)})\frac{d^{2}}{dx^{2}}V(x)+\frac{1}{L_{1}(x)}V(x)=(+\omega^{2}C_{1}(x))V(x).$
(18)

that has similarity with Schroedinger equation of the form

$\displaystyle-\frac{\hbar^{2}}{2m}\frac{d^{2}}{dx^{2}}\psi(x)+V_{p}(x)\psi(x)=E\psi(x).$
(19)

indicating that energy E is related to $+\omega^{2}C_{1}(x)$ if we set
capacitance that is independent of position, so $C_{1}(x)=const_{2}$.
Furthermore with presumption that V(x) is equivalent to wave-function
$\psi(x)$ we can establish another analogies. We can spot that
$\frac{1}{L_{1}(x)}$ plays of potential from Schroedinger equation $V_{p}(x)$,
while $\frac{1}{L(x)}$ plays role of $\frac{\hbar^{2}}{2m}$. Placement of
linear resistance R in series with L inductance brings modification of
previous equation into form

$\displaystyle-(\frac{1}{L(x)+\frac{R}{i\omega}})\frac{d^{2}}{dx^{2}}V(x)+\frac{1}{L_{1}(x)}V(x)=(+\omega^{2}C_{1}(x))V(x).$
(20)

while placement of resistance $R_{1}$ in parallel to $L_{1}$ and $C_{1}$
brings the equation

$\displaystyle-(\frac{1}{L(x)})\frac{d^{2}}{dx^{2}}V(x)+\frac{1}{L_{1}(x)+\frac{R1}{i\omega}}V(x)=(+\omega^{2}C_{1}(x))V(x).$
(21)

and thus we have dissipative effective Schrodinger potential given as

$\displaystyle
V_{p}(x)=\frac{1}{L_{1}(x)+\frac{R1}{i\omega}}=\frac{i\omega}{i\omega
L_{1}(x)+R_{1}}=\frac{i\omega(-i\omega L_{1}(x)+R_{1})}{(\omega
L_{1}(x))^{2}+R_{1}^{2}}=\frac{(\omega^{2}L_{1}(x)+iR_{1}\omega)}{(\omega
L_{1}(x))^{2}+R_{1}^{2}}.$ (22)

###  IV-A Case of non-linear long line model generalization of Kron’s model

We consider long line with case of vertical inductance dependence on space
given by $L_{1}(x)$, while preassuming other passive elements to be constant
that leads to equation

$\displaystyle\frac{d}{dx}I(x)=(j\omega C_{1}+\frac{1}{j\omega
L_{1}(x)})V(x),$ $\displaystyle\frac{d}{dx}V(x)=(j\omega L+I(x)R(I(x)))I(x),$
(23)

and from previous form we obtain

$\displaystyle\frac{d}{dx}I(x)=\frac{d}{dx}(\frac{1}{(j\omega
L+I(x)R(I(x)))}\frac{d}{dx}V(x))=(j\omega C_{1}+\frac{1}{j\omega
L_{1}(x)})V(x).$ (24)

In general case we have

$\displaystyle\frac{d}{dx}I(x,t)=(\frac{d}{dt}V(x,t)C_{1}-\int_{t_{0}}^{t}dt^{\prime}\frac{1}{L_{1}(x)}V(x,t^{\prime})),$
$\displaystyle\frac{d}{dx}V(x)=(-L\frac{dI}{dt}+I(x)R(I(x))),$ (25)

In case of introduction of the non-linear resistance in series with upper
inductors and under assumption of the uniformity of the long line with
explicitly given dependence of $L_{1}(x)$ we arrive into non-linear integro-
differential equation. Consequently we obtain

$\displaystyle\frac{d^{2}}{dx^{2}}I(x,t)=(\frac{d}{dt}\frac{d}{dx}V(x,t)C_{1}-\int_{t_{0}}^{t}dt^{\prime}\frac{d}{dx}[\frac{1}{L_{1}(x)}V(x,t^{\prime})]),$
$\displaystyle\frac{d}{dx}V(x)=(-L\frac{dI}{dt}+I(x)R(I(x))),$ (26)

that results in

$\displaystyle\frac{d^{2}}{dx^{2}}I(x,t)=(\frac{d}{dt}C_{1}(-L\frac{dI}{dt}+I(x)R(I(x)))-\int_{t_{0}}^{t}dt^{\prime}[\frac{d}{dx}\frac{1}{L_{1}(x)}]V(x,t^{\prime}))-\int_{t_{0}}^{t}dt^{\prime}\frac{1}{L_{1}(x)}[\frac{d}{dx}V(x,t^{\prime})],$
$\displaystyle\frac{d}{dx}V(x)=(-L\frac{dI}{dt}+I(x)R(I(x))),$ (27)

and applying operator $\frac{d}{dt}$ results in

$\displaystyle\frac{d^{3}}{dx^{2}dt}I(x,t)=\frac{d}{dt}(\frac{d}{dt}C_{1}(-L\frac{dI}{dt}+I(x)R(I(x)))-[\frac{d}{dx}\frac{1}{L_{1}(x)}]V(x,t^{\prime}))-\frac{1}{L_{1}(x)}[\frac{d}{dx}V(x,t^{\prime})],$
$\displaystyle\frac{d}{dx}V(x)=(-L\frac{dI}{dt}+I(x)R(I(x))),$ (28)

that has brings

$\displaystyle\frac{d}{dx}\Bigg{[}\frac{1}{[\frac{d}{dx}\frac{1}{L_{1}(x)}]}[-\frac{d^{3}}{dx^{2}dt}I(x,t)+\frac{d}{dt}(\frac{d}{dt}C_{1}(-L\frac{dI}{dt}+I(x)R(I(x)))-\frac{1}{L_{1}(x)}[-L\frac{dI}{dt}+I(x)R(I(x))]]\Bigg{]}=,$
$\displaystyle=(-L\frac{dI}{dt}+I(x)R(I(x)))=\frac{d}{dx}V(x),$ (29)

and consequently we end up with non-linear partial differential equation for
$I(x,t)$ of form

$\displaystyle\frac{d}{dx}(\frac{1}{[\frac{d}{dx}\frac{1}{L_{1}(x)}]})][-\frac{d^{3}}{dx^{2}dt}I(x,t)+\frac{d}{dt}(\frac{d}{dt}C_{1}(-L\frac{dI(x,t)}{dt}+I(x,t)R(I(x,t)))-\frac{1}{L_{1}(x)}[-L\frac{dI(x,t)}{dt}+I(x,t)R(I(x,t))]]$
$\displaystyle+\Bigg{[}[(\frac{1}{[\frac{d}{dx}\frac{1}{L_{1}(x)}]})]\frac{d}{dx}[-\frac{d^{3}}{dx^{2}dt}I(x,t)+\frac{d}{dt}(\frac{d}{dt}C_{1}(-L\frac{dI(x,t)}{dt}+I(x,t)R(I(x,t)))-\frac{1}{L_{1}(x)}[-L\frac{dI}{dt}+I(x,t)R(I(x,t))]]\Bigg{]}$
$\displaystyle=(-L\frac{dI(x,t)}{dt}+I(x)R(I(x,t)))=\frac{d}{dx}V(x),$

where L, R, $C_{1}$ are considered to be non-dependent on x and they represent
upper inductance, linear or non-linear resistance and capacitance $C_{1}$,
while $L_{1}(x)$ is inductance linear density that is explicitly position
dependent.

In first approach it is instruction to consider non-linear resistance as
represented by two identical diodes connected in anti-parallel way and in such
a case the non-linear resistance is given by the formula:

$\displaystyle
R(I)=\frac{V}{I}=kT\frac{ln(\frac{I}{I_{0}})+1}{I}=\frac{V}{I_{0}(e^{\frac{V}{kT}}-1)}=R(V).$
(31)

Under the circumstance of low voltage $R(V\rightarrow 0)=\frac{kT}{I_{0}}$,
while for non-small values of voltage we can assume
$R(V)=\frac{V}{I_{0}}e^{-(\frac{V}{kT})}$.

##  V Conclusions

Conducted numerical simulations show potential of generalization of originally
proposed by Kron’s circuit towards representation of confined particle by
different shapes of effective potential, like the harmonic, V-shape or
polynomialy dependent.

We believe that the creation of the mathematical model of the generalised
solver based on Kron’s second circuit would enable us to create the dedicated
simulation hardware. Such hardware could potentially vastly improve the speed
and efficiency of the simulations of the quantum phenomena as one can conduct
simulation with omission of existence of computers on local chips.
Implementation of classical analog solvers in classical analog electronic
circuitry can express certain class of quantum phenomena. Later this concept
can be used in design of classical-quantum hardware. One of the open issues is
analog hardware simulation of two or more electrostatically interacting single
electron semiconductor devices as given by [4]. Another future targets can be
focused on representing superconducting single-photon detectors [9] and
Josephson junction [10] by classical analog electronics.

Based on already conducted simulations we can point out forthcoming research
goals:

  * •

Creation of the fully generalised model based on Kron’s second model of analog
solver.

  * •

Increasing the integration of the analog and digital electronics through use
of the more advanced signal generators, voltage probes and data storage.

  * •

Further automation of the simulation process.

##  VI Acknowledgment

This work was conducted equally in 50 percent by first Author and in 50
percent by second Author. The conceptual and mathematical scheme was drawn by
second Author, while all conducted simulation were implemented and carried out
by first Author.

## References

  * [1] P.A.M.Dirac, Principles of Quantum Mechanics, Oxford University Press, 1958
  * [2] R.Fitzpatrick, Quantum Mechanics, World Scientific Publishing, 2015
  * [3] J.Binney, D.Skinner, The Physics of Quantum Mechanics, Oxford University Press, 2013.
  * [4] K.Pomorski, Fundamental description of Wannier qubits of any topology in semiconductor by analytical and numerical computations, 2021 (https://arxiv.org/abs/2103.05869)
  * [5] Analog Devices: Analog Electronics Tutorials.
(https://www.analog.com/en/education/education-library/tutorials/analog-
electronics.html)

  * [6] K.Pomorski, From two types of electrostatic position-dependent semiconductor qubits to quantum universal gates and hybrid semiconductor-superconducting quantum computer, 2019
  * [7] LTSpice community documentation supplement (https://ltwiki.org)
  * [8] G.Kron, Electric Circuit Models of the Schroedinger Equation, General Electric Company, 1944
  * [9] D.Irwin, G.C.Hilton, "Transition-edge sensors", Cryogenic Particle Detection, ed. C. Enss, Springer, 2005
  * [10] K.Pomorski, A.Bednorz, Justification of the canonical quantization of the Josephson effect and its modifications due to high capacitance energy, Journal of Physics A: Math. Theor. Vol. 49


Sai Srujana Buddi1, Utkarsh Oggy Sarawgi1, Tashweena Heeramun1, Karan
Sawnhey1, Ed Yanosik1, Saravana Rathinam2∗††thanks: * Work performed at
Apple., Saurabh Adya1

# Efficient Multimodal Neural Networks for Trigger-less Voice Assistants

###### Abstract

The adoption of multimodal interactions by Voice Assistants (VAs) is growing
rapidly to enhance human-computer interactions. Smartwatches have now
incorporated trigger-less methods of invoking VAs, such as Raise To Speak
(RTS), where the user raises their watch and speaks to VAs without an explicit
trigger. Current state-of-the-art RTS systems rely on heuristics and
engineered Finite State Machines to fuse gesture and audio data for multimodal
decision-making. However, these methods have limitations, including limited
adaptability, scalability, and induced human biases. In this work, we propose
a neural network based audio-gesture multimodal fusion system that (1) Better
understands temporal correlation between audio and gesture data, leading to
precise invocations (2) Generalizes to a wide range of environments and
scenarios (3) Is lightweight and deployable on low-power devices, such as
smartwatches, with quick launch times (4) Improves productivity in asset
development processes.

Index Terms: speech detection, gesture detection, multimodal fusion, temporal
networks, neural networks, voice assitants, wearable smart devices, natural
human-computer interactions

##  1 Introduction

Multimodality is a fundamental aspect of human interactions [1, 2], as speech
is frequently combined with other modalities, such as gesture, and gaze [3].
Human Computer Interfaces (HCIs) have embraced this concept to enhance natural
modes of communication [4]. However, despite this trend, physical button press
(BP) and voice trigger (VT) using keyword detection remain the dominant ways
of invoking and interacting with Voice Assistants (VAs) on personal devices
[5]. These methods can be cumbersome, particularly in multi-turn conversations
[6]. Raise to Speak (RTS), developed by Zhao et al. [7], address these issues,
and provides a more natural and convenient way of interacting with VAs on
smartwatches using gesture and speech. The user can simply raise their watch
to their mouth and speak, without the need for any explicit trigger phrases.
The RTS task presents a complex challenge that requires the integration of
multiple modalities:

  * •

Recognizing and identifying the combined parallel and sequential pattern of
gesture (raise) and speech (audio).

  * •

Having the ability to adapt to a wide range of user scenarios, physical
movements, demographics, and languages.

  * •

Running efficiently on low-power devices such as smartwatches, with limited
resources.

  * •

Achieving a balance between fast response time and high precision.

RTS system described in [7] employs human-designed Finite State Machines
(FSMs) for audio-gesture integration, but FSMs have limitations in capturing
intricate human behavior accurately, lack adaptability, and are susceptible to
human biases. The difficulty in designing FSMs increases as the system
complexity grows [8]. In this work, we present a refined method for solving
the RTS problem by incorporating neural networks to seamlessly merge the audio
and gesture modalities for improved VA invocation accuracy on smartwatches. We
have devised Neural Policy, a lightweight, and low latency Gated Recurrent
Unit (GRU) [9] that integrates speech and gesture signals and makes a binary
RTS trigger/no-trigger decision. Due to the loose coupling between the speech
and gesture modalities [10, 11, 12] and resource constraints of a low-power
device like a smartwatch, we adopt a late fusion approach [13, 14]. Our
proposed Neural Policy delivers significant accuracy improvements, with a
$90\%$ relative reduction in False Reject Rate (FRR) and False Accept Rate
(FAR) while also eliminating the complexity in system design process.

(a) RTS detection model overview.

(b) An example timing diagram of an RTS trigger. $acl\\_x$, $acl\\_y$,
$acl\\_z$ corresponds to the 3-axis accelerometer values.

Figure 1: Raise to Speak (RTS) model and process overview.

##  2 Related work

Multimodal learning has a long-standing history that predates and continues
through the deep learning era [15]. The advancements in deep learning have
been instrumental in the considerable growth of multimodal learning [16],
leading to surge of interest in multimodal interfaces, with focus on improving
the flexibility, transparency, and expressiveness of HCIs [3, 17, 18].

Gesture signals have been used in various multimodal applications, such as
speech disambiguation [19] and object manipulation in augmented reality [20].
Recently, there has been an increasing trend of using gestures to enhance HCIs
[21, 22, 23, 24]. Zhao et al. introduced an innovative way of using gesture
and speech to make the VA invocation process quick, intuitive, and natural
[7]. However, most of the multimodal systems defined in HCI works [7, 23, 24,
25, 26] use rule-based or linear systems for multimodal fusion, which fail to
adapt to a diverse and sizable population. [21, 27] explored using deep
learning for gesture and speech fusion but these systems are not
computationally efficient for low-power devices.

##  3 Methods and Materials

###  3.1 RTS Detection: System and Components

The goal of the RTS detection systems is to detect RTS triggers by
understanding the correlation and underlying patterns of accelerometer and
audio signals as shown in Figure 1(b). To account for the asynchronous nature
of input signals and device limitations, we chose a late fusion approach for
this system. This approach involves processing each modality separately and
then fusing the processed modalities to arrive at a multimodal decision.
Figure 1(a) shows an overview of RTS detection system, with its components:

  * •

Audio featurizer, extracts $40$ dimensional mel filter banks,
$\hat{x}_{audio}\in\mathbb{R}^{40}$, from input audio
$X_{audio}\in\mathbb{R}$.

  * •

Gesture featurizer, extracts $31$ dimensional statistical features [7],
$\hat{x}_{accel}\in\mathbb{R}^{31}$, from $3-$axis accelerometer data
$X_{accel}$.

  * •

Speech Detector, a binary streaming classifier for detecting speech using two
classes - speech, no speech. It takes in the audio features $\hat{x}_{audio}$
and outputs $\hat{h}_{audio}\in\mathbb{R}^{2}$.

  * •

Gesture Detector, a $4$-way streaming classifier for recognizing the RTS
gesture, defined by a valid hand raise, using four classes - raising, raised,
dropping, dropped. It takes in gesture features $\hat{x}_{accel}$ and outputs
$\hat{h}_{accel}\in\mathbb{R}^{4}$.

  * •

Multimodal fusion, which aligns the audio and gesture detector models outputs,
$\hat{h}_{audio}$ and $\hat{h}_{accel}$, and makes a binary trigger decision,
$p_{intended}$, using two classes - Not Intended for VA and Intended for VA,
$\hat{y}_{RTS}\in\mathbb{R}^{2}$.

###  3.2 Training setup

We adopt a sequential training process for development of the RTS system and
components discussed in Section 3.1. Initially, we train the unimodal speech
and gesture detectors separately, and the resulting scores from the unimodal
detectors are then combined to train the multimodal fusion. This sequential
training setup has several benefits, including reducing the complexity of the
model and infrastructure, and enabling the utilization of all available data,
even when paired modalities or labels are missing. Designing this system under
real-world constraints, especially when dealing with a significant amount of
missing data and labels in the gesture modality, presented a significant
challenge. Nonetheless, the sequential training setup facilitated maximizing
the utilization of the available data.

###  3.3 Dataset

In order to gather data that is representative and high in quality, we
collected data from smartwatches, specifically, audio recordings through the
single-channel $16$ kHz microphone built into the smartwatch, as well as
$3-$axis accelerometer data through the smartwatch's accelerometer sensor. To
create a meaningful, and diverse dataset, we developed a detailed user study
program. Through this program, we collected a total of $8000$ hours of data
from $1500$ hired subjects with consent, with each subject recording $100-150$
sessions. We ensured chosen subjects are from a variety of demographics,
including gender, ethnicity, and location, as well as physical characteristics
like height, age, and preferred wrist for wearing the watch.

We chose an orchestrated approach to collect data, in which a moderator guides
subjects through a wide range of scenarios defined by a data collection
protocol. The objective of the protocol is to guarantee that the gathered data
encompasses the diverse range of dynamic environments and activities that a
smartwatch could potentially encounter in real-world settings. During the data
collection process, subjects perform activities such as sitting, standing,
walking, running, cycling, driving, kayaking, stretching, gesturing, and lying
down, in environments including meeting rooms, gyms, parks, crowded areas, and
quiet rooms. Half the scenarios involve the user interacting with the VA using
RTS. Each subject receives $75$ instructions, each with different activity and
environment combinations, and records one or two sessions per instruction.
Additionally, we collected video from the moderator's point of view to aid
annotation.

The data is split into train, validation, and test sets- with ratios of
$70\%$, $15\%$, and $15\%$, respectively. To maintain reliability in the model
development process, we ensure that each subject's recordings belong to only
one split. The data are annotated with audio, gesture, and intent to interact
with VA labels. Audio from the watch is used for audio labeling and video from
the recordings are used for gesture and intent labeling.

###  3.4 Evaluation Metrics

Owing to the user experience of the interaction, we perform evaluation of the
detection system on the following metrics:

  * •

False Reject/Miss Rate (FRR): ratio of triggers that are missed by the
detector to the total number of true triggers. A low FRR is desirable.

  * •

False Accept/Wake Rate (FAR): measures the frequency with which the detector
is falsely triggered. A low FAR is desirable.

  * •

Equal Error Rate (EER): point on Detection Error Tradeoff (DET) curve [28]
where FAR meets FRR. A low EER is preferred.

In addition to accuracy metrics, we evaluate on-device performance metrics
such as runtime latency, physical memory, and power consumption.

##  4 Multimodal Temporal Neural Fusion

We propose Neural Policy, that uses lightweight, temporal neural networks to
fuse audio and gesture signals for improved responsiveness, and enhanced RTS
performance. Such a neural fusion technique, is able to use vast amounts of
diverse training data to learn patterns, and correlations between the
underlying modalities. Our approach involves constructing streaming temporal
networks that integrate the outputs from the RTS gesture and speech detectors
on a per-frame basis and generate a probability of occurrence of RTS
invocation in a streaming fashion.

Firstly, the unimodal Gesture and Speech detectors as detailed in Section 3.1
are trained as $10Hz$ streaming classifiers with a $4$-way and $2$-way
classification setup, respectively. Both detectors are trained using Adam
optimizer [29] to minimize the frame-wise discriminative categorical cross-
entropy loss in accordance with their corresponding frame-wise labels as
defined in Equation (1) below.

$l_{CE}=-\sum_{i=1}^{n}\sum_{j=1}^{m}\sum_{k=1}^{K}y_{k}^{(i,j)}log(\hat{s}_{k}^{(i,j)})$
(1) $\text{where softmax scores,
}\hat{s}_{k}^{(i,j)}=\frac{e^{\hat{l}_{k}^{(i,j)}}}{\sum_{j=1}^{K}e^{\hat{l}_{k}^{(i,j)}}}$
(2)

$\hat{l}$ represents output of the detector's final layer (logits) , $n$
represents sample count, $m$ represents number of frames per session, and $K$
represents number of classes where $K=2$ for the speech detector, and $K=4$
for the gesture detector.

The outputs from the trained speech and gesture detectors at the frequency of
$10Hz$ are aligned, padded, and merged and are used as inputs to develop the
neural policy. As part of the neural policy development, we propose two
variations of the multimodal neural fusion technique, softmax neural fusion
and logit neural fusion, as also illustrated in Figure 2.

(a) Softmax neural fusion

(b) Logit neural fusion

Figure 2: Types of Neural Fusion

###  4.1 Softmax neural fusion

In softmax neural fusion approach, the frame-wise $4$ dimensional, and $2$
dimensional output probability score sequences from the gesture, and speech
detectors respectively are used as input to the neural policy as shown in
Figure 2(a). These output probability scores are computed by applying softmax
function to the unimodal detectors final output as defined by Equation (2).
The aligned audio and gesture scores are merged and used to train the neural
fusion model using the Adam optimizer [29] to minimize the frame-wise
discriminative categorical cross-entropy loss in accordance with their frame-
wise binary intended/unintended labels defined by Equation (1).

###  4.2 Logit neural fusion

Softmax non-linearity pushes the scores towards the extremes of 0 and 1,
causing perturbations in the fusion latent space by reducing its dynamic
range, thus making raw outputs of models like embeddings and logits more
suitable for downstream tasks and further computations [30]. Considering this,
we propose logit neural fusion, which incorporates the $4$ and $2$ dimensional
gesture and speech logits directly in the fusion process of neural policy. The
final layer output of the unimodal detectors, without applying the softmax
function, serve as inputs to the neural policy, as depicted in Figure 2(b).
With device limitations in mind, logits are preferred over embeddings. The
training procedure of logit neural fusion is similar to that of softmax neural
fusion, as discussed in Section 4.1. The merged $2$ and $4$ dimensional audio
and gesture logits are used to train the fusion model using the Adam optimizer
[29] to minimize the frame-wise discriminative categorical cross-entropy loss
in accordance with their frame-wise binary intended/unintended labels.

###  4.3 Model architecture

Due to their lightweightedness and effectiveness in speech detection and
gesture recognition tasks, we chose a $1-$dimensional convolutional neural
network (1DCNN) architecture [7, 31, 32], for the speech and gesture
detectors. Each 1DCNN detector consists of convolutional layers that operate
along the temporal dimension, followed by a flatten, and two dense layers with
dimensions $128$ and $32$.

Due to the sequential nature and temporal ordering of the data, we opted Gated
Recurrent Units (GRU) for the architecture of neural policy. Our chosen GRU
architecture consists of a $6$ dimensional input layer, followed by a single
hidden layer with $h_{dim}$ dimensions, followed by a fully connected layer,
that results in $2$ dimensional output. Softmax function performed on the
output results in probability scores for RTS invocation occurrence. Although
we have explored Long Short-Term Memory networks (LSTMs) as well for this
task, we found that GRUs are more efficient as they require fewer parameters
and computational resources to achieve high accuracy.

##  5 Experiments and Results

We experimented with the hyperparameters of the models described in Section
4.3, and produced $36$ distinct RTS detectors, by varying the size of the GRU
neural policy's hidden layer, $h_{dim}\in\\{32,64\\}$, the number of
convolutional layers in unimodal 1DCNN detectors, $n\in\\{1,3,5\\}$, and the
type of neural fusion, $f\in\text{\\{softmax, logit\\}}$. All the models
involved are trained over $200$ epochs with a learning rate of $1e^{-3}$. In
order to ensure an unbiased comparison, we also implemented an FSM policy as
outlined by Zhao et al. [7] using $15$ hyperparameters, and included this in
our experimentation, resulting in a total of $54$ distinct variations of the
RTS detector. All the 54 variations are trained, tuned and evaluated end-to-
end. We have selected a subset of models for further analysis based on their
accuracy and on-device performance. Out of the $54$ variations of the RTS
detector, $6$ candidates we have hand-picked for further evaluation are:

  * •

(a) : $n_{s}=1$, $n_{g}=1$, $f=FSM$ (baseline)

  * •

(b) : $n_{s}=1$, $n_{g}=1$, $f=Softmax$, $h_{dim}=32$

  * •

(c) : $n_{s}=1$, $n_{g}=1$, $f=Softmax$, $h_{dim}=64$

  * •

(d) : $n_{s}=1$, $n_{g}=1$, $f=Logit$, $h_{dim}=64$

  * •

(e) : $n_{s}=3$, $n_{g}=1$, $f=Logit$, $h_{dim}=64$

  * •

(f) : $n_{s}=5$, $n_{g}=1$, $f=Logit$, $h_{dim}=64$

$n_{s},n_{g}$ represents number of 1DCNN layers in speech and gesture
detectors respectively, $f$ represents neural fusion type, and $h_{dim}$
represents size of GRU neural policy's hidden layer.

Table 1: EER comparison of RTS detectors. #Speech, #Gesture, and #NP are number of parameters in Speech, Gesture and Neural Policy detectors respectively. | #Speech | #Gesture | #NP | EER(%)
---|---|---|---|---
(a) | 133466 | 133356 | 15 | 40.0
(b) | 133466 | 133356 | 3906 | 8.1
(c) | 133466 | 133356 | 13954 | 7.0
(d) | 133466 | 133356 | 13954 | 5.5
(e) | 58546 | 133356 | 13954 | 4.9
(f) | 77482 | 133356 | 13954 | 4.4

Table 1 compares accuracy and model parameters of the selected RTS detectors,
and several key observations can be drawn, (1) Replacing FSM with a neural
policy using softmax neural fusion results in an $80\%$ relative decrease in
EER, indicating that the neural policy approach is highly effective, and the
neural networks are more capable of learning from a large population than FSM.
It is also observed that increase in the size of the neural policy leads to
further accuracy improvements. (2) The accuracy of neural fusion is further
enhanced by using logit neural fusion, which addresses the limitations of
softmax non-linearity, as discussed in Section 4.2. (3) Increasing the depth
of the speech detector and adding more 1DCNN layers enhances the audio
representation, leading to neural policy accuracy improvements. Figure 3
illustrates that RTS detectors with deeper speech detectors have higher
accuracy in challenging negative scenarios, where RTS gestures are paired with
unintended human speech. However, increasing the depth of the gesture detector
did not lead to any performance improvements, likely due to the relatively
smaller dataset available for gesture. The RTS detector that employs a
5-layered speech detector, a 1-layered gesture detector, and a logit neural
policy with hidden size 64 (highlighted in Table 1) demonstrates the most
significant accuracy gains and achieves a relative reduction in EER of $90\%$.

Table 2: On-device performance metrics of RTS detectors. The values are recorded by running detectors on 32 bit ARM Cortex-A7 MPCore processor in low power mode, averaged over 30 iterations. Memory is physical memory consumed in megabytes. Power measures static and dynamic power drawn per inference along with static power of device when idle in milliWatts. Runtime is time taken per inference measured in milliseconds. | (a) | (b) | (c) | (d) | (e) | (f)
---|---|---|---|---|---|---
Memory(MB) | 1.04 | 1.06 | 1.06 | 1.06 | 0.77 | 0.85
Power(mW) | 100.4 | 122.2 | 122.4 | 122.4 | 124.1 | 128.9
Runtime(ms) | 0.30 | 0.54 | 0.57 | 0.57 | 0.76 | 1.4

Table 2 compares the on-device performance metrics for the selected RTS
detectors. The results are competitive, demonstrating that the chosen Neural
Policy, despite using neural networks, is lightweight and suitable for on-
device implementation. In this analysis, the models precision is set to FP32,
but we observed that quantizing them to FP16 yields even better competitive
outcomes with no accuracy reductions.

The neural policy not only improves accuracy but also significantly simplifies
the asset development process. The FSM policy development which involves no
training, but finetuning in a $15$ dimensional hyperparameter space to find a
single operating point, takes $3-5$ days and $1000$ compute nodes even with
the efficiency of Bayesian search and parallelism. In contrast, training and
tuning the neural policy using distributed data parallel on a single $4$ GPU
node takes $2-3$ hours reducing development time and compute resources
significantly.

Figure 3: False Positives of challenging scenarios at fixed 20% FRR and
variable FAR. (a) Check time on watch while talking (b) Check watch and read
what’s on it (c) Raise watch near mouth and cough (d) Hold phone in same hand
as watch and attend a call (e) Perform a right turn while on steering wheel
and speak loudly.

##  6 Conclusion and Future Work

In conclusion, this work introduces two versions of a GRU-based multimodal
temporal neural fusion approach, one using softmax and the other using logit,
for accurate identification of VA trigger-less invocations using audio and
gesture signals. The approach demonstrates its ability to learn the temporal
correlations between the input audio and gesture signals and achieve high
precision. It is lightweight and suitable for deployment on low-power devices,
such as smartwatches, while maintaining low launch time and latency. Compared
to the traditional FSM-based multimodal fusion, this approach results in a
significant improvement in the Raise To Speak experience, with a relative
decrease of nearly $90\%$ in false rejects and false accepts. Additionally,
this approach offers a more streamlined development process when compared to
the traditional FSM-based multimodal fusion.

Future research efforts can be focused on experimenting with techniques such
as modality dropout [33] to make the fusion more robust. Attention-based
fusion models like multimodal transformers [34] and attention bottlenecks [13]
can be explored, albeit it is crucial to ensure that performance metrics such
as memory, latency, and power are kept within acceptable limits for on-device
applications.

## References

  * [1] M. Kurosu, _Human-Computer Interaction. Multimodal and Natural Interaction: Thematic Area, HCI 2020, Held as Part of the 22nd International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, Proceedings, Part II_. Springer Nature, 2020, vol. 12182\.
  * [2] M. Rasenberg, W. Pouw, A. Özyürek, and M. Dingemanse, ``The multimodal nature of communicative efficiency in social interaction,'' _Scientific Reports_ , vol. 12, no. 1, p. 19111, 2022.
  * [3] M. Turk, ``Multimodal interaction: A review,'' _Pattern Recognition Letters_ , vol. 36, pp. 189–195, 2014. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0167865513002584
  * [4] S. Oviatt, ``Advances in robust multimodal interface design,'' _IEEE computer graphics and applications_ , vol. 23, no. 05, pp. 62–68, 2003.
  * [5] P. Nayak, T. Higuchi, A. Gupta, S. Ranjan, S. Shum, S. Sigtia, E. Marchi, V. Lakshminarasimhan, M. Cho, S. Adya _et al._ , ``Improving voice trigger detection with metric learning,'' _arXiv preprint arXiv:2204.02455_ , 2022.
  * [6] Y. Qin, C. Yu, Z. Li, M. Zhong, Y. Yan, and Y. Shi, ``Proximic: Convenient voice activation via close-to-mic speech detected by a single microphone,'' in _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_ , 2021, pp. 1–12.
  * [7] S. Zhao, B. Westing, S. Scully, H. Nieto, R. Holenstein, M. Jeong, K. Sridhar, B. Newendorp, M. Bastian, S. Raman _et al._ , ``Raise to speak: An accurate, low-power detector for activating voice assistants on smartwatches,'' in _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, 2019, pp. 2736–2744.
  * [8] D. Harel, ``Statecharts: A visual formalism for complex systems,'' _Science of computer programming_ , vol. 8, no. 3, pp. 231–274, 1987.
  * [9] J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, ``Empirical evaluation of gated recurrent neural networks on sequence modeling,'' _arXiv preprint arXiv:1412.3555_ , 2014.
  * [10] S. Kettebekov, M. Yeasin, and R. Sharma, ``Prosody based audiovisual coanalysis for coverbal gesture recognition,'' _IEEE Transactions on Multimedia_ , vol. 7, no. 2, pp. 234–242, 2005.
  * [11] ——, ``Improving continuous gesture recognition with spoken prosody,'' in _2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings._ , vol. 1. IEEE, 2003, pp. I–I.
  * [12] W. Pouw and J. A. Dixon, ``Quantifying gesture-speech synchrony,'' in _the 6th gesture and speech in interaction conference_. Universitaetsbibliothek Paderborn, 2019, pp. 75–80.
  * [13] A. Nagrani, S. Yang, A. Arnab, A. Jansen, C. Schmid, and C. Sun, ``Attention bottlenecks for multimodal fusion,'' _Advances in Neural Information Processing Systems_ , vol. 34, pp. 14 200–14 213, 2021.
  * [14] K. Gadzicki, R. Khamsehashari, and C. Zetzsche, ``Early vs late fusion in multimodal convolutional neural networks,'' in _2020 IEEE 23rd International Conference on Information Fusion (FUSION)_ , 2020, pp. 1–6.
  * [15] D. Ramachandram and G. W. Taylor, ``Deep multimodal learning: A survey on recent advances and trends,'' _IEEE Signal Processing Magazine_ , vol. 34, pp. 96–108, 2017.
  * [16] J. Summaira, X. Li, A. M. Shoib, S. Li, and J. Abdul, ``Recent advances and trends in multimodal deep learning: A review,'' _arXiv preprint arXiv:2105.11087_ , 2021.
  * [17] S. Ceraso, ``(re) educating the senses: Multimodal listening, bodily learning, and the composition of sonic experiences,'' _College English_ , vol. 77, no. 2, pp. 102–123, 2014.
  * [18] B. Dumas, D. Lalanne, and S. Oviatt, ``Multimodal interfaces: A survey of principles, models and frameworks,'' _Human machine interaction: Research results of the mmi program_ , pp. 3–26, 2009.
  * [19] H. Holzapfel, K. Nickel, and R. Stiefelhagen, ``Implementation and evaluation of a constraint-based multimodal fusion system for speech and 3d pointing gestures,'' in _Proceedings of the 6th international conference on Multimodal interfaces_ , 2004, pp. 175–182.
  * [20] A. S. Williams and F. R. Ortega, ``Understanding gesture and speech multimodal interactions for manipulation tasks in augmented reality using unconstrained elicitation,'' _Proceedings of the ACM on Human-Computer Interaction_ , vol. 4, no. ISS, pp. 1–21, 2020.
  * [21] X. Qiu, Z. Feng, X. Yang, and J. Tian, ``Multimodal fusion of speech and gesture recognition based on deep learning,'' _Journal of Physics: Conference Series_ , vol. 1453, no. 1, p. 012092, jan 2020. [Online]. Available: https://dx.doi.org/10.1088/1742-6596/1453/1/012092
  * [22] L. M. Brown and J. Williamson, ``Shake2talk: multimodal messaging for interpersonal communication,'' in _Haptic and Audio Interaction Design: Second International Workshop, HAID 2007 Seoul, South Korea, November 29-30, 2007 Proceedings 2_. Springer, 2007, pp. 44–55.
  * [23] R. Stiefelhagen, C. Fugen, R. Gieselmann, H. Holzapfel, K. Nickel, and A. Waibel, ``Natural human-robot interaction using speech, head pose and gestures,'' in _2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566)_ , vol. 3, 2004, pp. 2422–2427 vol.3.
  * [24] I. Rodomagoulakis, N. Kardaris, V. Pitsikalis, E. Mavroudi, A. Katsamanis, A. Tsiami, and P. Maragos, ``Multimodal human action recognition in assistive human-robot interaction,'' in _2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2016, pp. 2702–2706.
  * [25] M. Saleh and I. Jouny, ``Multimodal person identification through the fusion of face and voice biometrics,'' in _2022 17th Annual System of Systems Engineering Conference (SOSE)_ , 2022, pp. 164–169.
  * [26] G. Donkal and G. K. Verma, ``A multimodal fusion based framework to reinforce ids for securing big data environment using spark,'' _Journal of information security and applications_ , vol. 43, pp. 1–11, 2018.
  * [27] N. Neverova, C. Wolf, G. Taylor, and F. Nebout, ``Moddrop: Adaptive multi-modal gesture recognition,'' _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 38, no. 8, pp. 1692–1706, 2016.
  * [28] A. Martin, G. Doddington, T. Kamm, M. Ordowski, and M. Przybocki, ``The det curve in assessment of detection task performance,'' National Inst of Standards and Technology Gaithersburg MD, Tech. Rep., 1997.
  * [29] D. P. Kingma and J. Ba, ``Adam: A method for stochastic optimization,'' _arXiv preprint arXiv:1412.6980_ , 2014.
  * [30] S. Horiguchi, D. Ikami, and K. Aizawa, ``Significance of softmax-based features in comparison to distance metric learning-based features,'' _IEEE transactions on pattern analysis and machine intelligence_ , vol. 42, no. 5, pp. 1279–1285, 2019.
  * [31] F. Jia, S. Majumdar, and B. Ginsburg, ``Marblenet: Deep 1d time-channel separable convolutional neural network for voice activity detection,'' in _ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2021, pp. 6818–6822.
  * [32] Z. Yu, Y. Lu, Q. An, C. Chen, Y. Li, and Y. Wang, ``Real-time multiple gesture recognition: Application of a lightweight individualized 1d cnn model to an edge computing system,'' _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ , vol. 30, pp. 990–998, 2022.
  * [33] N. Neverova, C. Wolf, G. Taylor, and F. Nebout, ``Moddrop: adaptive multi-modal gesture recognition,'' _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 38, no. 8, pp. 1692–1706, 2015.
  * [34] P. Xu, X. Zhu, and D. A. Clifton, ``Multimodal learning with transformers: A survey,'' _arXiv preprint arXiv:2206.06488_ , 2022.