# On some Properties of Generalized Tribonacci Spinors

Gamaliel Cerda-Morales

###### Abstract.

Spinors are used in physics quite extensively. The goal of this study is also
the spinor structure lying in the basis of the quaternion algebra. In this
paper, first, we have introduced spinors mathematically. Then, we have defined
Tribonacci spinors using the generalized Tribonacci quaternions. Later, we
have established the structure of algebra for these spinors. Finally, we have
proved some important formulas such as Binet and Cassini-like formulas which
are given for some series of numbers in mathematics for Tribonacci spinors.

###### Key words and phrases:

Keywords: Binet’s formula, Spinors, Tribonacci numbers, Tribonacci
quaternions.

###### 1991 Mathematics Subject Classification:

Mathematical subject classification 2010: 11B39, 15A66, 11R52.

Instituto de Matemáticas, Pontificia Universidad Católica de Valparaíso,
Blanco Viel 596, Cerro Barón, Valparaíso, Chile.
E-mails<EMAIL_ADDRESS>
##  1\. Introduction

###  1.1. Spinors and quaternions

First, let us introduce the spinors mentioned by Cartan in [2]. Consider that
$\mathbf{a}=(a_{1},a_{2},a_{3})\in\mathbb{C}^{3}$ is the isotropic vector and
$\mathbb{C}^{3}$ is the three-dimensional complex vector space. So, we obtain
$a_{1}^{2}+a_{2}^{2}+a_{3}^{2}=0$. The set of isotropic vectors in the vector
space $\mathbb{C}^{3}$ forms a two-dimensional surface in the space
$\mathbb{C}^{2}$. If this two-dimensional surface is parameterized by
$\mu_{1}$ and $\mu_{2}$ coordinates, then

$a_{1}=\mu_{1}^{2}-\mu_{2}^{2},\ \ a_{2}=i(\mu_{1}^{2}+\mu_{2}^{2}),\ \
a_{3}=-2\mu_{1}\mu_{2}$

is obtained. It is seen from the solution of this equation that

$\mu_{1}=\pm\sqrt{\frac{a_{1}-ia_{2}}{2}},\
\mu_{2}=\pm\sqrt{\frac{-a_{1}-ia_{2}}{2}}.$

It is seen that in the complex vector space $\mathbb{C}^{3}$, each isotropic
vector corresponds to two vectors, $(\mu_{1},\mu_{2})$ and
$(-\mu_{1},-\mu_{2})$ in the space $\mathbb{C}^{2}$. Conversely, both vectors
so given in space $\mathbb{C}^{2}$ correspond to the same isotropic vector
$\mathbf{a}$. Cartan stated that the two-dimensional complex vectors

(1.1) $\mu=(\mu_{1},\mu_{2})\cong\left[\begin{array}[]{c}\mu_{1}\\\
\mu_{2}\end{array}\right]$

described in this way are called as spinor (see [2]). The set of spinors is
denoted by $\mathrm{S}_{p}$. In addition, Cartan emphasized that spinors are
not only two-dimensional complex vectors, but also represent three-dimensional
complex isotropic vectors (see also, for example, [7]).

Let us consider that the complex conjugate of spinor $\mu$ is
$\overline{\mu}$. So, Cartan in [2] wrote that the conjugate of spinor $\mu$
is

(1.2)
$\widetilde{\mu}=i\mathrm{C}\overline{\mu}=i\left[\begin{array}[]{cc}0&1\\\
-1&0\end{array}\right]\left[\begin{array}[]{c}\overline{\mu_{1}}\\\
\overline{\mu_{2}}\end{array}\right]=\left[\begin{array}[]{c}i\overline{\mu_{2}}\\\
-i\overline{\mu_{1}}\end{array}\right],$

where $\mathrm{C}=\left[\begin{array}[]{cc}0&1\\\ -1&0\end{array}\right]$.
Moreover, Torres Del Castillo and Sánchez Barrales [9] wrote that the mate of
spinor $\mu$ is

(1.3)
$\widehat{\mu}=-\mathrm{C}\overline{\mu}=-\left[\begin{array}[]{cc}0&1\\\
-1&0\end{array}\right]\left[\begin{array}[]{c}\overline{\mu_{1}}\\\
\overline{\mu_{2}}\end{array}\right]=\left[\begin{array}[]{c}-\overline{\mu_{2}}\\\
\overline{\mu_{1}}\end{array}\right].$

Furthermore, a real quaternion is defined with

$\mathrm{q}=q_{0}+\sum_{l=1}^{3}q_{l}\mathrm{e}_{l},\ \
q_{0}\mathrm{e}_{0}=q_{0},$

where $q_{l}\in\mathbb{R}$ ($l=0,1,2,3$) and the quaternion basis
$\\{\mathrm{e}_{l}:\ l=0,1,2,3\\}$ is given such that
$\mathrm{e}_{1}^{2}=\mathrm{e}_{2}^{2}=\mathrm{e}_{3}^{2}=-\mathrm{e}_{0}$,
$\mathrm{e}_{1}\mathrm{e}_{2}=\mathrm{e}_{3}=-\mathrm{e}_{2}\mathrm{e}_{1}$,
$\mathrm{e}_{2}\mathrm{e}_{3}=\mathrm{e}_{1}=-\mathrm{e}_{3}\mathrm{e}_{2}$
and
$\mathrm{e}_{3}\mathrm{e}_{1}=\mathrm{e}_{2}=-\mathrm{e}_{1}\mathrm{e}_{3}$.

Let $S_{\mathrm{q}}=q_{0}$ and
$V_{\mathrm{q}}=\sum_{l=1}^{3}q_{l}\mathrm{e}_{l}$ be scalar and vectorial
parts of the quaternion $\mathrm{q}$. So, we can write the quaternion
$\mathrm{q}$ as $\mathrm{q}=S_{\mathrm{\mathbf{q}}}+V_{\mathrm{\mathbf{q}}}$.
The set of these quaternions is $\mathbb{H}$. Let
$\mathrm{p}=S_{\mathrm{p}}+V_{\mathrm{p}},\mathrm{q}=S_{\mathrm{q}}+V_{\mathrm{q}}\in\mathbb{H}$
be two real quaternions. So, the quaternion product of these quaternions

$\mathrm{p}\times\mathrm{q}=S_{\mathrm{p}}S_{\mathrm{q}}-\langle
V_{\mathrm{p}},V_{\mathrm{q}}\rangle+S_{\mathrm{p}}V_{\mathrm{q}}+S_{\mathrm{q}}V_{\mathrm{p}}+V_{\mathrm{p}}\wedge
V_{\mathrm{q}},$

where $\langle,\rangle$ is usual inner product and $\wedge$ is cross product
in real vector space $\mathbb{R}^{3}$. It is clear that the product of two
real quaternions is non-commutative.In addition, if we consider that
$\mathrm{q}^{*}$ is the conjugate of the quaternion $\mathrm{q}$,
$\mathrm{q}^{*}$ is equal to $\mathrm{q}^{*}=S_{\mathrm{q}}-V_{\mathrm{q}}$,
the norm of a quaternion $\mathrm{q}$ is
$Nr(\mathrm{q})=\sum_{l=0}^{3}q_{l}^{2}$. If $Nr(\mathrm{q})=1$, then
$\mathrm{q}$ is called a unit quaternion.

Now, we give one relationship between spinors and quaternions expressed by
Vivarelli (see [10]). This correspondence between any quaternion
$\mathrm{q}=q_{0}+\sum_{l=1}^{3}q_{l}\mathrm{e}_{l}$ and a spinor
$\mu=\left[\begin{array}[]{c}\mu_{1}\\\ \mu_{2}\end{array}\right]$ is

(1.4) $\displaystyle\sigma:$
$\displaystyle\mathbb{H}\longrightarrow\mathrm{S}_{p}$ $\displaystyle\ \
\mathrm{q}\longmapsto\sigma\left(q_{0}+\sum_{l=1}^{3}q_{l}\mathrm{e}_{l}\right)=\left[\begin{array}[]{c}q_{3}+iq_{0}\\\
q_{1}+iq_{2}\end{array}\right].$

(see [10, Eq. (23)]). So, we obtain a spinor formulation of the kinematics of
rotation, which extends the quaternion formulation. Moreover, this function
$\sigma$ is clearly linear and injective. Then, we can write
$\sigma(\mathrm{p}+\mathrm{q})=\sigma(\mathrm{p})+\sigma(\mathrm{q})$,
$\sigma(k\mathrm{q})=k\sigma(\mathrm{q})$, $k\in\mathbb{R}$ and
$\ker(\sigma)=\\{0\\}$. If we take that the conjugate of the quaternion
$\mathrm{q}$ is $\mathrm{q}^{*}=q_{0}-\sum_{l=1}^{3}q_{l}\mathrm{e}_{l}$, the
spinor corresponding to conjugate quaternion can be written as

$\sigma(\mathrm{q}^{*})=\sigma\left(q_{0}-\sum_{l=1}^{3}q_{l}\mathrm{e}_{l}\right)=\left[\begin{array}[]{c}-q_{3}+iq_{0}\\\
-q_{1}-iq_{2}\end{array}\right].$

Vivarelli [10, Eq. (25)] associated to the product of two quaternions
$\mathrm{p}\times\mathrm{q}$, which is equal to a spinor matrix product as
follows:

(1.5) $\mathrm{p}\times\mathrm{q}\longrightarrow-i\breve{P}Q,$

where

$\breve{P}=\left[\begin{array}[]{cc}p_{3}+ip_{0}&p_{1}-ip_{2}\\\
p_{1}+ip_{2}&-p_{3}+ip_{0}\end{array}\right],\
Q=\left[\begin{array}[]{c}q_{3}+iq_{0}\\\ q_{1}+iq_{2}\end{array}\right].$

There are four known ways of approaching the theory of spinors in the
Euclidean three-space: by Cartan’s isotropic vectors, by Clifford algebra as
exemplified by Pauli matrices, by spinor ring algebra, by stereographic
projection (see [2, 7, 8]).

###  1.2. Generalized Tribonacci quaternions

The definitions of generalized Tribonacci quaternions were first given by
Cerda-Morales [4]. So, generalized Tribonacci can be written as

$Q_{v,n}=V_{n}+\sum_{l=1}^{3}V_{n+l}\mathrm{e}_{l},\ \
V_{n}\mathrm{e}_{0}=V_{n}$

where $V_{n}=V_{n}(V_{0},V_{1},V_{2};r,s,t)$ is the $n$-th generalized
Tribonacci number defined by

(1.6) $V_{n}=rV_{n-1}+sV_{n-2}+tV_{n-3},\ n\geq 3,$

and $V_{0}$, $V_{1}$, $V_{2}$ are arbitrary integers and $r$, $s$ and $t$ are
real numbers. Generalized Tribonacci quaternions are the third-order linear
recurrence sequence and for $n\geq 0$ one can write

(1.7) $Q_{v,n+3}=rQ_{v,n+2}+sQ_{v,n+1}+tQ_{v,n}.$

Let $Q_{v,n}$ be $n$-th generalized Tribonacci quaternion. Then, the Binet
formula for this quaternion is

(1.8)
$Q_{v,n}=\frac{\mathrm{P}\underline{\alpha}\alpha^{n}}{(\alpha-\omega_{1})(\alpha-\omega_{2})}-\frac{\mathrm{Q}\underline{\omega_{1}}\omega_{1}^{n}}{(\alpha-\omega_{1})(\omega_{1}-\omega_{2})}+\frac{\mathrm{R}\underline{\omega_{2}}\omega_{2}^{n}}{(\alpha-\omega_{2})(\omega_{1}-\omega_{2})},$

where
$\mathrm{P}=V_{2}-(\omega_{1}+\omega_{2})V_{1}+\omega_{1}\omega_{2}V_{0}$,
$\mathrm{Q}=V_{2}-(\alpha+\omega_{2})V_{1}+\alpha\omega_{2}V_{0}$,
$\mathrm{R}=V_{2}-(\alpha+\omega_{1})V_{1}+\alpha\omega_{1}V_{0}$,
$\underline{\alpha}=\mathrm{e}_{0}+\sum_{l=1}^{3}\alpha^{l}\mathrm{e}_{l}$,
$\underline{\omega_{1}}=\mathrm{e}_{0}+\sum_{l=1}^{3}\omega_{1}^{l}\mathrm{e}_{l}$,
$\underline{\omega_{2}}=\mathrm{e}_{0}+\sum_{l=1}^{3}\omega_{2}^{l}\mathrm{e}_{l}$
and $\alpha$, $\omega_{1}$ and $\omega_{2}$ are the roots of the cubic
equation $x^{3}-rx^{2}-sx-t=0$ (see [4]).

In addition, for the generalized Tribonacci quaternion $Q_{v,n}$, the
generating function is

$g_{Q_{v,n}}(x)=\frac{Q_{v,0}+(Q_{v,1}-rQ_{v,0})x+(Q_{v,2}-rQ_{v,1}-sQ_{v,0})x^{2}}{1-rx-
sx^{2}-tx^{3}}.$

Furthermore, the summation formula for generalized Tribonacci quaternions is
as follows:

$\sum_{l=0}^{n}Q_{v,l}=\frac{1}{\delta}\left(Q_{v,n+2}+(1-r)Q_{v,n+1}+tQ_{v,n}+\omega\right),$

where $\delta=r+s+t-1$, $\omega=\lambda+\mathrm{e}_{1}(\lambda-\delta
V_{0})+\mathrm{e}_{2}(\lambda-\delta(V_{0}+V_{1}))+\mathrm{e}_{3}(\lambda-\delta(V_{0}+V_{1}+V_{2}))$
and $\lambda=(r+s-1)V_{0}+(r-1)V_{1}-V_{2}$. Moreover, Cerda-Morales gave the
generalized Tribonacci quaternion matrix as follows:

(1.9) $\mathbf{Q}_{v}=\left[\begin{array}[]{ccc}Q_{v,4}&K_{v,2}&tQ_{v,3}\\\
Q_{v,3}&K_{v,1}&tQ_{v,2}\\\ Q_{v,2}&K_{v,0}&tQ_{v,1}\end{array}\right],$

where $K_{v,n}=sQ_{v,n+1}+tQ_{v,n}$. With the help of this matrix, the author
obtained the next formula for generalized Tribonacci quaternions

$Q_{v,n+2}=Q_{v,2}U_{n+2}+(sQ_{v,1}+tQ_{v,0})U_{n+1}+tQ_{v,1}U_{n},\ n\geq 0$

where $U_{n}=V_{n}(0,0,1;r,s,t)$.

##  2\. Main Theorems and Proofs

In this section, we consider that there is a spinor corresponding to every
real quaternion and we match each generalized Tribonacci quaternion with one
spinor which has two complex components. Then, we introduce generalized
Tribonacci spinors. Similarly, we express the particular case of third-order
Jacobsthal spinors. Then, using the recent work of Erişar and Güngör in [6],
we give some theorems and formulas for generalized Tribonacci and Tribonacci
spinors.

Let the quaternion $Q_{v,n}=V_{n}+\sum_{l=1}^{3}V_{n+l}\mathrm{e}_{l}$ be
$n$-th generalized Tribonacci quaternion where $V_{n}$ is $n$-th generalized
Tribonacci number. Let the set of generalized Tribonacci quaternions be
$\mathbb{T}_{r,s,t}$.

Now, we consider the correspondence between spinors and quaternions in Eq.
(1.4) and we write the following transformation:

(2.10) $\displaystyle\sigma:$
$\displaystyle\mathbb{T}_{r,s,t}\longrightarrow\mathrm{S}_{p}$ $\displaystyle\
\
\mathrm{q}\longmapsto\sigma\left(V_{n}+\sum_{l=1}^{3}V_{n+l}\mathrm{e}_{l}\right)=\left[\begin{array}[]{c}V_{n+3}+iV_{n}\\\
V_{n+1}+iV_{n+2}\end{array}\right]\equiv\mathrm{A}_{v,n},$

where $\\{\mathrm{e}_{l}:\ l=1,2,3\\}$ coincide with basis vectors given for
real quaternions. This transformation is linear and injective but it is not
surjective. So, with the help of this transformation, we obtain a new sequence
from generalized Tribonacci quaternions. We say that this new sequence is a
generalized Tribonacci spinor sequence. Note that

(2.11)
$\mathrm{A}_{v,n+3}=r\mathrm{A}_{v,n+2}+s\mathrm{A}_{v,n+1}+t\mathrm{A}_{v,n},\
n\geq 0,$

can be written. Then, the generalized Tribonacci spinor sequence is the linear
recurrence sequence. Now, we give some algebraic definitions for generalized
Tribonacci spinors.

######  Definition 2.1 (Conjugates).

Let the conjugate of the generalized Tribonacci quaternion $Q_{v,n}$ be
$Q_{v,n}^{*}=V_{n}-\sum_{l=1}^{3}V_{n+l}\mathrm{e}_{l}$. So, from the
correspondence in Eq. (2.11), the generalized Tribonacci spinor
$\mathrm{A}_{v,n}^{*}$ corresponding to conjugate of generalized Tribonacci
quaternion is written by

$\sigma\left(V_{n}-\sum_{l=1}^{3}V_{n+l}\mathrm{e}_{l}\right)=\left[\begin{array}[]{c}-V_{n+3}+iV_{n}\\\
-V_{n+1}-iV_{n+2}\end{array}\right]\equiv\mathrm{A}_{v,n}^{*}.$

On the other hand, we can write that the ordinary complex conjugate of
generalized Tribonacci spinor $\mathrm{A}_{v,n}$ is

$\mathrm{A}_{v,n}=\left[\begin{array}[]{c}V_{n+3}+iV_{n}\\\
V_{n+1}+iV_{n+2}\end{array}\right]\longrightarrow\overline{\mathrm{A}}_{v,n}=\left[\begin{array}[]{c}V_{n+3}-iV_{n}\\\
V_{n+1}-iV_{n+2}\end{array}\right].$

According to the spinor conjugate given in Eq. (1.2) by Cartan, generalized
Tribonacci spinor conjugate
$\widetilde{\mathrm{A}}_{v,n}=i\mathrm{C}\overline{\mathrm{A}}_{v,n}$ is as
follows:

$\mathrm{A}_{v,n}=\left[\begin{array}[]{c}V_{n+3}+iV_{n}\\\
V_{n+1}+iV_{n+2}\end{array}\right]\longrightarrow\widetilde{\mathrm{A}}_{v,n}=\left[\begin{array}[]{c}V_{n+2}+iV_{n+1}\\\
-V_{n}-iV_{n+3}\end{array}\right].$

In addition to that, Torres Del Castillo and Sánchez Barrales obtained the
mate of spinor in Eq. (1.3). According to this, the mate of generalized
Tribonacci spinor is
$\widehat{\mathrm{A}}_{v,n}=-\mathrm{C}\overline{\mathrm{A}}_{v,n}$ and

$\mathrm{A}_{v,n}=\left[\begin{array}[]{c}V_{n+3}+iV_{n}\\\
V_{n+1}+iV_{n+2}\end{array}\right]\longrightarrow\widehat{\mathrm{A}}_{v,n}=\left[\begin{array}[]{c}-V_{n+1}+iV_{n+2}\\\
V_{n+3}-iV_{n}\end{array}\right].$

The following corollary gives the relationship between the conjugates for
generalized Tribonacci spinors.

######  Corollary 2.2.

Let the $n$-th generalized Tribonacci spinor be $\mathrm{A}_{v,n}$. The
correspondences between conjugates of generalized Tribonacci spinors hold

(2.12) $\mathrm{C}\widehat{\mathrm{A}}_{v,n}=\left[\begin{array}[]{cc}0&1\\\
-1&0\end{array}\right]\left[\begin{array}[]{c}-V_{n+1}+iV_{n+2}\\\
V_{n+3}-iV_{n}\end{array}\right]=\overline{\mathrm{A}}_{v,n},$ (2.13)
$i\widetilde{\mathrm{A}}_{v,n}=i\left[\begin{array}[]{c}V_{n+2}+iV_{n+1}\\\
-V_{n}-iV_{n+3}\end{array}\right]=\widehat{\mathrm{A}}_{v,n},$ (2.14)
$i\mathrm{C}\widetilde{\mathrm{A}}_{v,n}=\left[\begin{array}[]{cc}0&i\\\
-i&0\end{array}\right]\left[\begin{array}[]{c}V_{n+2}+iV_{n+1}\\\
-V_{n}-iV_{n+3}\end{array}\right]=\overline{\mathrm{A}}_{v,n}.$

######  Definition 2.3 (Norm).

The norm of a generalized Tribonacci quaternion
$Nr(Q_{v,n})=Q_{v,n}Q_{v,n}^{*}$ is equal to the norm of associated
generalized Tribonacci spinor

$Nr(Q_{v,n})=\left[\begin{array}[]{cc}V_{n+3}-iV_{n}&V_{n+1}-iV_{n+2}\end{array}\right]\left[\begin{array}[]{c}V_{n+3}+iV_{n}\\\
V_{n+1}+iV_{n+2}\end{array}\right].$

In addition, if we use Corollary 2.12, then we give the norm of generalized
tribonacci spinor

$Nr(Q_{v,n})=\overline{\mathrm{A}}_{v,n}^{t}\mathrm{A}_{v,n}=-\widehat{\mathrm{A}}_{v,n}^{t}\mathrm{C}^{t}\mathrm{A}_{v,n}=-i\widetilde{\mathrm{A}}_{v,n}^{t}\mathrm{C}^{t}\mathrm{A}_{v,n}.$

Now, we give the Binet formula for generalized Tribonacci spinors solving
generalized Tribonacci spinor recurrence relation in Eq. (2.11).

######  Theorem 2.4.

For $n\geq 0$, the Binet formula for the $n$-th generalized Tribonacci spinor
$\mathrm{A}_{v,n}$ is that

(2.15) $\displaystyle\mathrm{A}_{v,n}$
$\displaystyle=\frac{\mathrm{P}\alpha^{n}}{(\alpha-\omega_{1})(\alpha-\omega_{2})}\left[\begin{array}[]{c}\alpha^{3}+i\\\
\alpha+i\alpha^{2}\end{array}\right]-\frac{\mathrm{Q}\omega_{1}^{n}}{(\alpha-\omega_{1})(\omega_{1}-\omega_{2})}\left[\begin{array}[]{c}\omega_{1}^{3}+i\\\
\omega_{1}+i\omega_{1}^{2}\end{array}\right]$ $\displaystyle\ \
+\frac{\mathrm{R}\omega_{2}^{n}}{(\alpha-\omega_{2})(\omega_{1}-\omega_{2})}\left[\begin{array}[]{c}\omega_{2}^{3}+i\\\
\omega_{2}+i\omega_{2}^{2}\end{array}\right],$

where $\mathrm{P}$, $\mathrm{Q}$ and $\mathrm{R}$ as in Eq. (1.8).

###### Proof.

We consider the generalized Tribonacci spinor sequence
$\\{\mathrm{A}_{v,n}\\}_{n\geq 0}$. The characteristic equation of recurrence
relation of generalized spinors is also $x^{3}-rx^{2}-sx-t=0$. Moreover, the
roots of this equation are $\alpha$, $\omega_{1}$ and $\omega_{2}$ as in Eq.
(1.8). We assume the initial values

$\mathrm{A}_{v,0}=\left[\begin{array}[]{c}V_{3}+iV_{0}\\\
V_{1}+iV_{2}\end{array}\right],\ \
\mathrm{A}_{v,1}=\left[\begin{array}[]{c}V_{4}+iV_{1}\\\
V_{2}+iV_{3}\end{array}\right],\ \
\mathrm{A}_{v,2}=\left[\begin{array}[]{c}V_{5}+iV_{2}\\\
V_{3}+iV_{4}\end{array}\right]$

for the equation
$\mathrm{A}_{v,n}=a_{1}\alpha^{n}+a_{2}\omega_{1}^{n}+a_{3}\omega_{2}^{n}$.
So, for generalized Tribonacci spinor $\mathrm{A}_{v,0}$, ($n=0$) we have
$\mathrm{A}_{v,0}=a_{1}+a_{2}+a_{3}=\left[\begin{array}[]{c}V_{3}+iV_{0}\\\
V_{1}+iV_{2}\end{array}\right]$, for generalized Tribonacci spinor
$\mathrm{A}_{v,1}$, ($n=1$) also we obtain
$\mathrm{A}_{v,1}=a_{1}\alpha+a_{2}\omega_{1}+a_{3}\omega_{2}=\left[\begin{array}[]{c}V_{4}+iV_{1}\\\
V_{2}+iV_{3}\end{array}\right]$ and finally for $n=2$, we have
$\mathrm{A}_{v,2}=a_{1}\alpha^{2}+a_{2}\omega_{1}^{2}+a_{3}\omega_{2}^{2}=\left[\begin{array}[]{c}V_{5}+iV_{2}\\\
V_{3}+iV_{4}\end{array}\right]$. If we make necessary adjustments we find that
the spinors

$a_{1}=\frac{\mathrm{P}}{(\alpha-\omega_{1})(\alpha-\omega_{2})}\left[\begin{array}[]{c}\alpha^{3}+i\\\
\alpha+i\alpha^{2}\end{array}\right],$
$a_{2}=-\frac{\mathrm{Q}}{(\alpha-\omega_{1})(\omega_{1}-\omega_{2})}\left[\begin{array}[]{c}\omega_{1}^{3}+i\\\
\omega_{1}+i\omega_{1}^{2}\end{array}\right]$

and

$a_{3}=\frac{\mathrm{R}}{(\alpha-\omega_{2})(\omega_{1}-\omega_{2})}\left[\begin{array}[]{c}\omega_{2}^{3}+i\\\
\omega_{2}+i\omega_{2}^{2}\end{array}\right].$

So, we obtain that Binet’s formula for $\mathrm{A}_{v,n}$ is obtained as
follows:

$\displaystyle\mathrm{A}_{v,n}$
$\displaystyle=a_{1}\alpha^{n}+a_{2}\omega_{1}^{n}+a_{3}\omega_{2}^{n}$
$\displaystyle=\frac{\mathrm{P}\alpha^{n}}{(\alpha-\omega_{1})(\alpha-\omega_{2})}\left[\begin{array}[]{c}\alpha^{3}+i\\\
\alpha+i\alpha^{2}\end{array}\right]-\frac{\mathrm{Q}\omega_{1}^{n}}{(\alpha-\omega_{1})(\omega_{1}-\omega_{2})}\left[\begin{array}[]{c}\omega_{1}^{3}+i\\\
\omega_{1}+i\omega_{1}^{2}\end{array}\right]$ $\displaystyle\ \
+\frac{\mathrm{R}\omega_{2}^{n}}{(\alpha-\omega_{2})(\omega_{1}-\omega_{2})}\left[\begin{array}[]{c}\omega_{2}^{3}+i\\\
\omega_{2}+i\omega_{2}^{2}\end{array}\right].$

The result is obtained. ∎

######  Corollary 2.5.

For $n\geq 0$, the Binet formula for the $n$-th Tribonacci spinor
$\mathrm{T}_{n}$ is that

(2.16) $\displaystyle\mathrm{T}_{n}$
$\displaystyle=\frac{\alpha^{n+1}}{(\alpha-\omega_{1})(\alpha-\omega_{2})}\left[\begin{array}[]{c}\alpha^{3}+i\\\
\alpha+i\alpha^{2}\end{array}\right]-\frac{\omega_{1}^{n+1}}{(\alpha-\omega_{1})(\omega_{1}-\omega_{2})}\left[\begin{array}[]{c}\omega_{1}^{3}+i\\\
\omega_{1}+i\omega_{1}^{2}\end{array}\right]$ $\displaystyle\ \
+\frac{\omega_{2}^{n+1}}{(\alpha-\omega_{2})(\omega_{1}-\omega_{2})}\left[\begin{array}[]{c}\omega_{2}^{3}+i\\\
\omega_{2}+i\omega_{2}^{2}\end{array}\right],$

where $\mathrm{T}_{n}=\mathrm{A}_{v,n}$ and $V_{n}=V_{n}(0,1,1;1,1,1)$ is the
$n$-th classic Tribonacci number.

Now, we obtain the generating functions for generalized Tribonacci spinors.

######  Theorem 2.6.

For $n\geq 0$, the generating function for the $n$-th generalized Tribonacci
spinor $\mathrm{A}_{v,n}$ is

(2.17)
$g_{\mathrm{A}_{v,n}}(x)=\frac{1}{\rho(x)}\left[\begin{array}[]{c}\rho_{1}V_{3}+\rho_{2}V_{4}+x^{2}V_{5}+i(\rho_{1}V_{0}+\rho_{2}V_{1}+x^{2}V_{2})\\\
\rho_{1}V_{1}+\rho_{2}V_{2}+x^{2}V_{3}+i(\rho_{1}V_{2}+\rho_{2}V_{3}+x^{2}V_{4})\end{array}\right],$

where $\rho(x)=1-rx-sx^{2}-tx^{3}$, $\rho_{1}=\rho_{1}(x)=1-rx-sx^{2}$ and
$\rho_{2}=\rho_{2}(x)=x-rx^{2}$. Furthermore, $V_{n}$ as in Eq. (1.6).

###### Proof.

If the generating function $g_{\mathrm{A}_{v,n}}(x)=\sum_{n\geq
0}\mathrm{A}_{v,n}x^{n}$ is considered, then using the equations
$rxg_{\mathrm{A}_{v,n}}(x)$, $sx^{2}g_{\mathrm{A}_{v,n}}(x)$ and
$tx^{3}g_{\mathrm{A}_{v,n}}(x)$, we obtain that

$\displaystyle(1-rx-$ $\displaystyle sx^{2}-tx^{3})g_{\mathrm{A}_{v,n}}(x)$
$\displaystyle=\mathrm{A}_{v,0}+(\mathrm{A}_{v,1}-r\mathrm{A}_{v,0})x+(\mathrm{A}_{v,2}-r\mathrm{A}_{v,1}-t\mathrm{A}_{v,0})x^{2}$
$\displaystyle=\left[\begin{array}[]{c}V_{3}+iV_{0}\\\
V_{1}+iV_{2}\end{array}\right]+\left[\begin{array}[]{c}V_{4}-rV_{3}+i(V_{1}-rV_{0})\\\
V_{2}-rV_{1}+i(V_{3}-rV_{2})\end{array}\right]x$ $\displaystyle\ \
+\left[\begin{array}[]{c}V_{5}-rV_{4}-sV_{3}+i(V_{2}-rV_{1}-sV_{0})\\\
V_{3}-rV_{2}-sV_{1}+i(V_{4}-rV_{3}-sV_{2})\end{array}\right]x^{2}.$

If we regulate the last equation, we have

$\displaystyle(1-$ $\displaystyle rx-sx^{2}-tx^{3})g_{\mathrm{A}_{v,n}}(x)$
$\displaystyle=\left[\begin{array}[]{c}\rho_{1}V_{3}+\rho_{2}V_{4}+x^{2}V_{5}+i(\rho_{1}V_{0}+\rho_{2}V_{1}+x^{2}V_{2})\\\
\rho_{1}V_{1}+\rho_{2}V_{2}+x^{2}V_{3}+i(\rho_{1}V_{2}+\rho_{2}V_{3}+x^{2}V_{4})\end{array}\right],$

where $\rho_{1}=\rho_{1}(x)=1-rx-sx^{2}$ and $\rho_{2}=\rho_{2}(x)=x-rx^{2}$.
Then, the result is obtained. ∎

######  Corollary 2.7.

For $n\geq 0$, the generating function for the $n$-th Tribonacci spinor
$\mathrm{T}_{n}$ is

(2.18)
$g_{\mathrm{T}_{n}}(x)=\frac{1}{1-x-x^{2}-x^{3}}\left[\begin{array}[]{c}2+2x+x^{2}+ix\\\
1+i(1+x+x^{2})\end{array}\right].$

Now, similar to $\mathbf{Q}_{v}$-matrix given for generalized Tribonacci
quaternions in Eq. (1.9) we can also give one matrix for generalized
Tribonacci spinors. So, with the aid of this matrix, we can obtain am special
formula for generalized Tribonacci spinors. Thus, we first express the
following theorem.

######  Theorem 2.8.

Generalized Tribonacci spinor matrix, which has the same behavior as the
$\mathbf{Q}_{v}$-matrix given for generalized Tribonacci quaternions, is given
by

$\mathcal{Q}_{v}=-\left[\begin{array}[]{ccc}\mathrm{A}_{v,4}&\breve{\mathrm{K}}_{v,2}&t\breve{\mathrm{A}}_{v,3}\\\
\mathrm{A}_{v,3}&\breve{\mathrm{K}}_{v,1}&t\breve{\mathrm{A}}_{v,2}\\\
\mathrm{A}_{v,2}&\breve{\mathrm{K}}_{v,0}&t\breve{\mathrm{A}}_{v,1}\end{array}\right],\
\breve{\mathrm{K}}_{v,n}=s\breve{\mathrm{A}}_{v,n+1}+t\breve{\mathrm{A}}_{v,n}$

where
$\breve{\mathrm{A}}_{v,0}=\left[\begin{array}[]{cc}V_{3}+iV_{0}&V_{1}-iV_{2}\\\
V_{1}+iV_{2}&-V_{3}+iV_{0}\end{array}\right]$,
$\breve{\mathrm{A}}_{v,1}=\left[\begin{array}[]{cc}V_{4}+iV_{1}&V_{2}-iV_{3}\\\
V_{2}+iV_{3}&-V_{4}+iV_{1}\end{array}\right]$ and
$\breve{\mathrm{A}}_{v,2}=\left[\begin{array}[]{cc}V_{5}+iV_{2}&V_{3}-iV_{4}\\\
V_{3}+iV_{4}&-V_{5}+iV_{2}\end{array}\right]$.

###### Proof.

Let $n$-th generalized Tribonacci spinor $\mathrm{A}_{v,n}$ correspond to
$n$-th generalized Tribonacci quaternion $Q_{v,n}$. Let us consider the
$\mathbf{Q}_{v}$-matrix in Eq. (1.9); then the product of two quaternions can
be written by spinor matrix in Eq. (1.5). For example we have the following
equations
$Q_{v,1}K_{v,3}Q_{v,4}=-\breve{\mathrm{A}}_{v,1}\breve{\mathrm{K}}_{v,1}\mathrm{A}_{v,4}$
($Q_{v,1}K_{v,1}Q_{v,4}$ is preferred over the multiplication
$Q_{v,4}K_{v,1}Q_{v,1}$), where
$\breve{\mathrm{K}}_{v,1}=s\breve{\mathrm{A}}_{v,2}+t\breve{\mathrm{A}}_{v,1}$.
Moreover, for $\mathbf{Q}_{v}$-matrix we obtain that

$\displaystyle\frac{1}{t}\det(\mathbf{Q}_{v})$
$\displaystyle=Q_{v,1}\begin{vmatrix}Q_{v,4}&K_{v,2}\\\
Q_{v,3}&K_{v,1}\end{vmatrix}-Q_{v,2}\begin{vmatrix}Q_{v,4}&K_{v,2}\\\
Q_{v,2}&K_{v,0}\end{vmatrix}+Q_{v,3}\begin{vmatrix}Q_{v,3}&K_{v,1}\\\
Q_{v,2}&K_{v,0}\end{vmatrix}$
$\displaystyle=Q_{v,1}K_{v,1}Q_{v,4}+Q_{v,2}K_{v,2}Q_{v,2}+Q_{v,3}K_{v,0}Q_{v,3}$
$\displaystyle\ \
-Q_{v,1}K_{v,2}Q_{v,3}-Q_{v,2}K_{v,0}Q_{v,4}-Q_{v,3}K_{v,1}Q_{v,2}$
$\displaystyle=-\breve{\mathrm{A}}_{v,1}\breve{\mathrm{K}}_{v,1}\mathrm{A}_{v,4}-\breve{\mathrm{A}}_{v,2}\breve{\mathrm{K}}_{v,2}\mathrm{A}_{v,2}-\breve{\mathrm{A}}_{v,3}\breve{\mathrm{K}}_{v,0}\mathrm{A}_{v,3}$
$\displaystyle\ \
+\breve{\mathrm{A}}_{v,1}\breve{\mathrm{K}}_{v,2}\mathrm{A}_{v,3}+\breve{\mathrm{A}}_{v,2}\breve{\mathrm{K}}_{v,0}\mathrm{A}_{v,4}+\breve{\mathrm{A}}_{v,3}\breve{\mathrm{K}}_{v,1}\mathrm{A}_{v,2}.$

Finally, we can choose

$\mathcal{Q}_{v}=-\left[\begin{array}[]{ccc}\mathrm{A}_{v,4}&\breve{\mathrm{K}}_{v,2}&t\breve{\mathrm{A}}_{v,3}\\\
\mathrm{A}_{v,3}&\breve{\mathrm{K}}_{v,1}&t\breve{\mathrm{A}}_{v,2}\\\
\mathrm{A}_{v,2}&\breve{\mathrm{K}}_{v,0}&t\breve{\mathrm{A}}_{v,1}\end{array}\right],$

where
$\breve{\mathrm{A}}_{v,0}=\left[\begin{array}[]{cc}V_{3}+iV_{0}&V_{1}-iV_{2}\\\
V_{1}+iV_{2}&-V_{3}+iV_{0}\end{array}\right]$,
$\breve{\mathrm{A}}_{v,1}=\left[\begin{array}[]{cc}V_{4}+iV_{1}&V_{2}-iV_{3}\\\
V_{2}+iV_{3}&-V_{4}+iV_{1}\end{array}\right]$ and
$\breve{\mathrm{A}}_{v,2}=\left[\begin{array}[]{cc}V_{5}+iV_{2}&V_{3}-iV_{4}\\\
V_{3}+iV_{4}&-V_{5}+iV_{2}\end{array}\right]$. ∎

######  Corollary 2.9.

Tribonacci spinor matrix, which has the same behavior as the
$\mathbf{T}_{v}$-matrix given for Tribonacci quaternions, is given by

$\mathcal{T}=-\left[\begin{array}[]{ccc}\mathrm{T}_{4}&\breve{\mathrm{T}}_{3}+\breve{\mathrm{T}}_{2}&\breve{\mathrm{T}}_{3}\\\
\mathrm{T}_{3}&\breve{\mathrm{T}}_{2}+\breve{\mathrm{T}}_{1}&\breve{\mathrm{T}}_{2}\\\
\mathrm{T}_{2}&\breve{\mathrm{T}}_{1}+\breve{\mathrm{T}}_{0}&\breve{\mathrm{T}}_{1}\end{array}\right],$

where

$\breve{\mathrm{T}}_{0}=\left[\begin{array}[]{cc}2&1-i\\\
1+i&-2\end{array}\right],\breve{\mathrm{T}}_{1}=\left[\begin{array}[]{cc}4+i&1-2i\\\
1+2i&-4+i\end{array}\right],\breve{\mathrm{T}}_{2}=\left[\begin{array}[]{cc}7+i&2-4i\\\
2+4i&-7+i\end{array}\right].$

Now, we give the determinant formula for Tribonacci spinors as a result of
Theorem 2.8 and [4, Theorem 3.1]:

$\mathbf{Q}_{v}\left[\begin{array}[]{ccc}r&s&t\\\ 1&0&0\\\
0&1&0\end{array}\right]^{n}=\left[\begin{array}[]{ccc}Q_{v,n+4}&K_{v,n+2}&tQ_{v,n+3}\\\
Q_{v,n+3}&K_{v,n+1}&tQ_{v,n+2}\\\
Q_{v,n+2}&K_{v,n}&tQ_{v,n+1}\end{array}\right],$

where $K_{v,n}=sQ_{v,n+1}+tQ_{v,n}$ and $r=s=t=1$. So, the following corollary
can be given without proof.

######  Corollary 2.10.

For $n\geq 0$, we have determinant formula for Tribonacci spinors

$\left\\{\begin{array}[]{cc}\breve{\mathrm{T}}_{n+1}\breve{\mathrm{L}}_{n+1}\mathrm{T}_{n+4}+\breve{\mathrm{T}}_{n+2}\breve{\mathrm{L}}_{n+2}\mathrm{T}_{n+2}+\breve{\mathrm{T}}_{n+3}\breve{\mathrm{L}}_{n}\mathrm{T}_{n+3}\\\
\ \
-\breve{\mathrm{T}}_{n+1}\breve{\mathrm{L}}_{n+2}\mathrm{T}_{n+3}-\breve{\mathrm{T}}_{n+2}\breve{\mathrm{L}}_{n}\mathrm{T}_{4}-\breve{\mathrm{T}}_{n+3}\breve{\mathrm{L}}_{n+1}\mathrm{T}_{n+2}\end{array}\right\\}=4\left[\begin{array}[]{c}-1+i\\\
1-i\end{array}\right],$

where
$\breve{\mathrm{L}}_{n}=\breve{\mathrm{T}}_{n+1}+\breve{\mathrm{T}}_{n}$.

Now, the following corollary can be given without proof. The proof is obtained
by mathematical induction.

######  Theorem 2.11.

For every integer $n\geq 0$. The summation formula for generalized Tribonacci
spinors is as follows:

$\displaystyle\delta\sum_{l=0}^{n}\mathrm{A}_{v,l}$
$\displaystyle=\mathrm{A}_{v,n+2}+(1-r)\mathrm{A}_{v,n+1}+t\mathrm{A}_{v,n}$
$\displaystyle\ \
+\left[\begin{array}[]{c}(r+s)V_{3}+(r-1)V_{4}-V_{5}+i((r+s)V_{0}+(r-1)V_{1}-V_{2})\\\
(r+s)V_{1}+(r-1)V_{2}-V_{3}+i((r+s)V_{2}+(r-1)V_{3}-V_{4})\end{array}\right],$

where $\delta=r+s+t-1$ and $V_{n}$ is the $n$-th generalized Tribonacci
number.

######  Corollary 2.12.

For every integer $n\geq 0$. The summation formula for Tribonacci spinors is
as follows:

$\sum_{l=0}^{n}\mathrm{T}_{l}=\frac{1}{2}\left\\{\mathrm{T}_{n+2}+\mathrm{T}_{n}+\left[\begin{array}[]{c}-3-i\\\
-2i\end{array}\right]\right\\}.$

##  3\. Conclusion

In this study, we introduced the Tribonacci spinor and generalized Tribonacci
spinor by using the relationships between spinors and quaternions.Today,
Tribonacci numbers are the most common generalization of Fibonacci numbers,
whose applications in science are infinite. For example, in sunflower, human
body, corn pyramids, Pascal triangle, representation groups. On the other
hand, quaternions are defined as obtained by enlarging the set of complex
numbers. Quaternions have applications in physics, mathematics, engineering
and robotics. Also, by means of spinors, a shorter and simpler representation
of quaternions can be obtained. By combining this information, we have
obtained a new series called generalized Tribonacci spinors
$\\{\mathrm{A}_{v,n}\\}_{n\in\mathbb{N}}$. And we think that this study, in
which we have obtained a new series, will be a basic study especially for
mathematicians working on geometry and algebra. In a later work, we obtain
curious properties of this type of numbers.

## References

  * [1] I. Akkus and G. Kızılaslan, On some Properties of Tribonacci Quaternion, An. Şt. Univ. Ovidius Constanţa 26(3) (2018), 5–20.
  * [2] E. Cartan, The Theory of Spinors, Dover Publications, New York, 1966.
  * [3] G. Cerda-Morales, Identities for third order Jacobsthal quaternions. Adv. Appl. Clifford Algebra 27(2) (2017), 1043–1053.
  * [4] G. Cerda-Morales, On a generalization of Tribonacci quaternions. Mediter. J. Math. 14:239 (2017), 1–12.
  * [5] G. Cerda-Morales, Comment On “On some Properties of Tribonacci Quaternion”, An. Şt. Univ. Ovidius Constanţa 28(1) (2020), 95–97.
  * [6] T. Erişar and M.A. Güngör, On Fibonacci spinors, Int. J. Geom. Methods Mod. Phys. 17(4) (2020), 1–13.
  * [7] D.H. Sattinger and O.L. Weaver, Lie Groups and Algebras with Applications to Physics, Geometry and Mechanics, Springer-Verlag, New York, 1986.
  * [8] G.F. Torres del Castillo, 3-D Spinors, Spin-Weighted Functions and their Applications, Birkhauser, Boston, 2003.
  * [9] G.F. Torres del Castillo and G. Sánchez Barrales, Spinor formulation of the differential geometry of curves, Rev. Colomb. de Mat. 38(1) (2004), 27–34.
  * [10] M.D. Vivarelli, Development of spinors descriptions of rotational mechanics from Euler’s rigid body displacement theorem, Celest. Mech. 32(3) (1984), 193–207.


# Mitigating Out-of-Distribution Data Density Overestimation in Energy-Based
Models

Beomsu Kim
Department of Mathematical Sciences
KAIST
<EMAIL_ADDRESS>
&Jong Chul Ye
Graduate School of AI
KAIST
<EMAIL_ADDRESS>

###### Abstract

Deep energy-based models (EBMs), which use deep neural networks (DNNs) as
energy functions, are receiving increasing attention due to their ability to
learn complex distributions. To train deep EBMs, the maximum likelihood
estimation (MLE) with short-run Langevin Monte Carlo (LMC) is often used.
While the MLE with short-run LMC is computationally efficient compared to an
MLE with full Markov Chain Monte Carlo (MCMC), it often assigns high density
to out-of-distribution (OOD) data. To address this issue, here we
systematically investigate why the MLE with short-run LMC can converge to EBMs
with wrong density estimates, and reveal that the heuristic modifications to
LMC introduced by previous works were the main problem. We then propose a
Uniform Support Partitioning (USP) scheme that optimizes a set of points to
evenly partition the support of the EBM and then uses the resulting points to
approximate the EBM-MLE loss gradient. We empirically demonstrate that USP
avoids the pitfalls of short-run LMC, leading to significantly improved OOD
data detection performance on Fashion-MNIST.

##  1 Introduction

For unsupervised learning, it is often of great interest to approximate a
given data distribution using a generative model. Applications of generative
models are abundant, ranging from data generation kingma2013 ; goodfellow2014
to out-of-distribution (OOD) data detection choi2018 ; nalisnick2019 ; ren2019
; hendrycks2019 ; serra2020 , improving calibration and robustness of
classifiers du2019 , etc. Among the wide variety of generative models, Energy-
Based Models (EBMs) lecun2006 parametrized by deep neural networks (DNNs) have
recently gained attention thanks to their flexibility in modeling complex
distributions.

There are multiple ways of training EBMs, and the two most studied methods are
maximum likelihood estimation (MLE) with Markov Chain Monte Carlo (MCMC) and
Score Matching (SM) song2021 . Both methods have undergone appropriate
modifications for training deep EBMs, i.e., EBMs parametrized by DNNs. For
instance, in the case of SM, Song et. al song2019 proposed the estimation of
gradients, not density, of the data distribution to bypass calculation of the
Hessian. For MLE with MCMC, Du and Mordatch du2019 replaced MCMC, which often
requires thousands of iterations until convergence, with short-run Langevin
Monte Carlo (LMC) and a replay buffer.

Despite such developments, EBMs suffer from the problem of density
overestimation on OOD data elflein2021 . Concretely, given an EBM trained by
an MLE with short-run LMC (SRLMC), OOD data often have density values similar
to or higher than that of training data. This does not make sense, since by
the definition of OOD data, the supports of training data distribution and OOD
data distribution do not intersect. Mahmood et. al mahmood2021 attempted to
use score functions to detect OOD data, but they provided little insights into
the density overestimation problem, since score functions model the gradient,
not the density.

In this paper, we approach this problem in two ways. First, we rigorously
investigate why and how the MLE with SRLMC can yield EBMs with wrong density
estimates. Based on the observations, we then propose a novel technique,
called Uniform Support Partitioning (USP), to solve the MLE for EBMs. In
contrast to LMC which uses a stochastic process to sample from the EBM, USP
solves a deterministic optimization problem to find points which uniformly
partition the support of the EBM. USP then uses those points to approximate
the MLE objective gradient through numerical integration.

We also introduce a practical version of USP, called Persistent Stochastic USP
(PS-USP) and demonstrate on the problem of learning a mixture of Gaussians
that PS-USP is capable of crossing low-density regions. On the Fashion-MNIST
dataset, we show deep EBM trained with PS-USP shows significantly better OOD
data detection performance than deep EBM trained with SRLMC.

Our contributions can be summarized as follows:

  * •

Through theoretical analysis and experiments, we rigorously investigate why
MLE with SRLMC could converge to an EBM with wrong density estimates, and
reveal that it is caused by a combination of two heuristic modifications to
LMC introduced by previous works: (a) early termination of LMC in short-run
LMC and (b) using incorrect learning rate and noise scale ratio in LMC.

  * •

To avoid the pitfalls of MLE with SRLMC, we propose a novel technique, USP, to
solve MLE for EBMs. USP solves an optimization problem to find a set of points
which uniformly partition the support of EBMs. Then, it uses the points to
approximate the MLE objective through numerical integration. We also introduce
a practical version of USP for training deep EBMs.

  * •

We demonstrate on a toy example that USP is capable of accurately learning a
distribution with multiple separated modes. We also show on the Fashion-MNIST
dataset fmnist that deep EBMs trained with USP attain significantly better OOD
data detection performance than deep EBMs trained with SRLMC.

##  2 Related Works

###  2.1 EBM Training via MLE with MCMC

One of the most popular ways of training deep EBMs is maximizing the expected
log-likelihood of the EBM via gradient ascent song2021 . However, calculating
the gradient of the log-likelihood of the EBM requires computing the
expectation of the energy gradient on the current EBM distribution. A
straightforward way to achieve this is to run MCMC on the EBM distribution and
use the samples to approximate the energy gradient expectation. A popular
choice of MCMC is Stochastic Gradient Langevin Dynamics (SGLD) welling2011 , a
stochastic variant of LMC.

Recent works have taken further steps to make training deep EBMs efficient.
Specifically, Du and Mordatch du2019 have proposed using non-convergent SRLMC
(or short-run SGLD) with a replay buffer instead of LMC, which usually
requires thousands of iterations until convergence, to sample from the EBM
distribution. Latter works use the same technique as well nijkamp2019 ;
grathwohl2020 . Furthermore, Yang et. al yang2021 combine short-run LMC with
Pontryagin’s Maximum Principle to reduce the number of forward and backward
propagations.

The works by Nijkamp et. al nijkamp2019 and Nijkamp et. al nijkamp2020 perform
an analysis of MLE with SRLMC and find that MLE with SRLMC trains EBMs to be
data generators rather than density estimators. However, they do not explain
how this leads to density overestimation for OOD data.

###  2.2 OOD Data Detection with EBMs

With the development of efficient deep EBM training methods, OOD data
detection with EBMs also gained interest. Du and Mordatch du2019 discovered
that an EBM trained by MLE with MCMC has slightly better OOD data detection
performance than other deep density models such as Glow kingma2018 and
PixelCNN++ salimans2017 . Grathwolh et. al grathwohl2020 incorporate label
information into training EBMs and find that the EBM also outperforms Glow at
OOD data detection. Finally, Elflein et. al elflein2021 discover that using
supervision such as labels improves OOD detection on natural data and
architectural modifications such as bottlenecks can also improve OOD
detection.

##  3 Preliminaries

Given an energy function $E_{\theta}:\mathbb{R}^{d}\rightarrow\mathbb{R}$
parametrized by $\theta$, an EBM is defined as

$\displaystyle q_{\theta}(x)=\frac{1}{Z(\theta)}\exp\\{-E_{\theta}(x)\\}$ (1)

where $Z(\theta)$ is the partition function, which ensures $q_{\theta}$
integrates to 1. Given a data distribution $p$, the EBM can be trained to
approximate $p$ by MLE

$\displaystyle\max_{\theta}\mathbb{E}_{p}[\log q_{\theta}(x)]$ (2)

with gradient ascent. The gradient of the MLE objective can be decomposed into
two terms:

$\displaystyle\nabla_{\theta}\mathbb{E}_{p}[\log
q_{\theta}(x)]=\mathbb{E}_{q_{\theta}}[\nabla
E_{\theta}(x)]-\mathbb{E}_{p}[\nabla E_{\theta}(x)].$ (3)

While we can easily calculate the second term since we have access to samples
from $p$ (the training data), it is not the case for the first term. Previous
works rely on LMC or its stochastic variant, SGLD, to sample from $q_{\theta}$
and calculate the first term. Given $x_{0}\sim q_{0}(x)$ for some proposal
distribution $q_{0}$, LMC iterates

$\displaystyle
x_{t+1}=x_{t}-\frac{\eta_{t}}{2}\nabla_{x}E_{\theta}(x)+\sqrt{\eta_{t}}\epsilon_{t},\qquad
t=0,1,2,\ldots,T$ (4)

where $\epsilon_{t}$ are i.i.d. standard normal Gaussian noises. For an
appropriate choice of the sequence $\\{\eta_{t}\\}$, the sequence
$\\{x_{t}\\}$ converges to a sample from $q_{\theta}$ as $T\rightarrow\infty$
welling2011 ; dalalyan2017 .

###  3.1 MLE with Short-Run LMC (SRLMC)

A problem with MCMC is that it usually requires large number of iterations,
i.e., large $T$ in (4), until convergence. This becomes problematic when we
train deep EBMs, as forward and backward propagations of DNNs are expensive.
In an attempt to alleviate this issue, Du and Mordatch du2019 propose three
heuristic modifications to the EBM training procedure. These modifications
have been adopted by latter works nijkamp2019 ; grathwohl2020 ; yang2021 for
training EBMs as well. We now describe the modifications.

Short-run LMC (SRLMC). The first modification is using extremely small $T$.
While conventional LMC can require thousands of iterations until convergence,
Du and Mordatch du2019 propose using $T\leq 100$. Then, SRLMC samples are used
to calculate the first term in (3).

Decoupling step size and noise scale. The second modification is decoupling
the gradient coefficient and noise coefficient in (4):

$\displaystyle
x_{t+1}=x_{t}-\frac{\alpha_{t}}{2}\nabla_{x}E_{\theta}(x)+\sqrt{\beta_{t}}\epsilon_{t},\qquad
t=0,1,2,\ldots,T$ (5)

where $\alpha_{t}$ is called the step size and $\beta_{t}$ is called the noise
scale. Since $T$ is set to be small, Du and Mordatch du2019 set
$\alpha_{t}\gg\beta_{t}$ to accelerate the convergence of SRLMC.

Replay buffer. The third modification is to maintain a replay buffer of SRLMC
samples. Specifically, instead of using random noise to initialize SRLMC at
each iteration of EBM update, Du and Mordatch du2019 maintain a replay buffer
which stores past SRLMC samples. A mixture of replay buffer samples and random
noise is used to initialize SRLMC at each iteration of EBM training, and the
outputs are used to update the replay buffer.

In the next section, we demonstrate that the first and second modifications
can lead to EBMs with incorrect density, and the third modification does not
alleviate the issue.

(a) Training via SRLMC with no replay buffer.

(b) Training via SRLMC with replay buffer.

(c) Training via self-normalized importance sampling / Riemann sum.

Figure 1: Comparison of methods for solving EBM MLE. The first column shows
the evolution of $\theta$ gradient norm throughout the training process. The
second column displays the trained EBM log density on the data distribution
support and OOD regions. The third column shows densities and histograms of
relevant distributions. The final column plots the negative energy for the
trained EBM. Note that the density plots for $q_{\theta}(x)$ in (a) and (b)
are unnormalized.

##  4 A Solution to MLE With SRLMC Can Overestimate OOD Data Density

From here on, we will refer to the EBM training procedure described in Section
3.1 as MLE with SRLMC. For the moment, let us assume we do not use a replay
buffer. Let $q_{0}$ be some proposal distribution, e.g., a uniform
distribution, and let $\tilde{q}_{\theta}$ be the distribution of $\tilde{x}$
produced by running SRLMC on $x\sim q_{0}$111We remark $\tilde{q}_{\theta}$
will generally not be equal to $q_{\theta}$ since SRLMC does not run LMC until
convergence.. Then, EBM gradient update with MLE with SRLMC becomes

$\displaystyle\theta\leftarrow\theta+\mathbb{E}_{\tilde{q}_{\theta}}[\nabla_{\theta}E_{\theta}(x)]-\mathbb{E}_{p}[\nabla_{\theta}E_{\theta}(x)]$
(6)

and thus stationarity is achieved when

$\displaystyle\tilde{q}_{\theta}=p$ (7)

for then the expectations in (6) will cancel out and no update to $\theta$ is
made. We now demonstrate that a stationary point of (6) can assign high
density to OOD regions. To this end, we consider the problem of training a
deep EBM to approximate the mixture of two Gaussians
$\mathcal{N}(-0.5,0.05^{2})$ and $\mathcal{N}(0.5,0.05^{2})$. The Gaussians
are given equal weights. Also, $\alpha_{t}=10\beta_{t}$ and $T=40$.

Figure 1(a) shows the result of training an EBM with MLE with SRLMC and no
replay buffer. The proposal distribution $q_{0}$ is the uniform distribution
on $(-1,1)$. The leftmost $\theta$ gradient norm plot indeed shows the EBM has
nearly converged to some stationary point222The gradient norm does not become
exactly zero due to the stochasticity in LMC and finite number of samples used
to approximate expectations. This causes oscillation of the gradient norm..
However, contrary to our hopes, the second figure shows that a significant
portion of the OOD region is assigned higher density than half of the data
distribution support. The third and fourth figures indicate this is because
the EBM has learned a density with wide modes at $x=\pm 0.5$, and the second
mode is much lower than the first mode. Yet, despite the discrepancy between
the EBM density and the data density, $\tilde{q}_{\theta}$ is identical to
$p$. In particular, $\tilde{q}_{\theta}$ has two modes of equal height
although the heights of modes of the EBM differ significantly.

So far, we have experimentally shown the existence of a stationary point of
(6) which exhibits density overestimation and that a deep EBM can converge to
this undesirable point. We now explain why the EBM of Figure 1(a) is a
stationary point of (6). According to (7), this amounts to explaining how
SRLMC can generate training data with this wrong EBM. Two factors play a role:
poor mixing of SRLMC and incorrect step size and noise scale ratio.

Poor mixing of SRLMC. In general, LMC itself mixes very slowly. This issue was
previously pointed out by Song and Ermon song2019 . As LMC uses the gradient
information, it will initially tend to follow the steepest path of ascent of
$-E_{\theta}(x)$. Theoretically, LMC (4) will converge in the limit
$T\rightarrow\infty$, but SRLMC terminates with a very small $T$. So, an SRLMC
sample will typically end up in the mode whose basin of attraction contained
its initialization point.

The EBM can exploit this pathology of SRLMC and learn a density with modes of
incorrect probability mass. Specifically, let us consider a mode of an EBM,
denoted $m_{q}$, and its basin of attraction $B_{q}$. We denote the
probability mass contained in the corresponding data mode $m_{p}$ as $M_{p}$.
Now, due to poor mixing of SRLMC, most points initialized in $B_{q}$ will end
up in $m_{q}$. So, if we denote the probability measure of $q_{0}$ as
$\mathbb{Q}_{0}$, the ratio of proposal samples that will be placed in $m_{q}$
by SRLMC will be approximately $\mathbb{Q}_{0}(B_{q})$. Since the EBM learns
to match the distribution of generated data with training data (c.f. Eq. (7)),
at convergence of the EBM, we must have $\mathbb{Q}_{0}(B_{q})\approx M_{p}$.
As $B_{q}$ is generally unrelated to the probability mass of $m_{q}$, the
modes of EBM can have incorrect probability mass. The left panel of Figure 2
confirms this: SRLMC samples initialized on $(-1,0)$ mostly converge to the
left mode, and samples initialized on $(1,0)$ mostly converge to the right
mode.

The above observations have serious implications in high dimensions. In high
dimensions, samples from $q_{0}$ can come from a very small subset of the
support of $q_{0}$. As the following proposition shows, this phenomenon holds
for a wide variety of $q_{0}$.

######  Proposition 1.

Suppose $X$ is a $d$-dimensional random vector whose components are i.i.d.
with mean $\mu$, variance $\sigma^{2}$, and finite fourth moment. Then, for
any $\epsilon>0$,

$\displaystyle\lim_{d\rightarrow\infty}\mathbb{P}\left\\{(1-\epsilon)\sqrt{d(\sigma^{2}+\mu^{2})}<\|X\|_{2}<(1+\epsilon)\sqrt{d(\sigma^{2}+\mu^{2})}\right\\}=1.$

Proposition 1 claims that if the components of a high-dimensional vector is
i.i.d. with finite variance, almost all samples come from a thin shell. This
result is applicable to high-dimensional Gaussian distributions333If the
Gaussian distribution has a non-identity covariance matrix, Proposition 1 can
be easily extended to show that samples will lie around the boundary of an
ellipsoid with high probability. and uniform distributions on $[-1,1]^{d}$,
which are common choices of $q_{0}$ nijkamp2019 ; grathwohl2020 ; yang2021 .

Let us call this thin shell $S$. Then, just like the case of Figure 1(a), the
EBM can learn a path of ascent from $S$ to the data support. Moreover, the EBM
is free to assign arbitrary density to OOD regions which do not intersect $S$,
the data support, and the path of ascent. Since $S$ is very small and natural
data lies on low-dimensional manifolds (if we adopt the manifold hypothesis),
the volume of such OOD region can be very large in high dimensions. The poor
OOD data detection performance of EBMs trained with SRLMC observed by Elflein
et. al elflein2021 provides evidence for this claim.

Figure 2: Effect of SRLMC. Left: histograms of SRLMC samples on the EBM of
Figure 1(a) with $q_{0}=$ uniform distribution on $(-1,0)$ and $q_{0}=$
uniform distribution on $(0,1)$. Middle: histograms of SRLMC samples with
$q_{0}=$ uniform distribution on $(-1,1)$ and various $\rho$. Right: EBMs
trained to approximate $p$ via MLE with SRLMC with various $\rho$. We remark
that for each EBM, $\tilde{q}_{\theta}=p$ when the same $\rho$ is used for
training and sample generation. Also, each EBM has a slight bump, i.e, a mode,
at $x=-0.5$, so SRLMC samples form the mode at $x=-0.5$.

Incorrect step size and noise scale ratio. The first factor, poor mixing of
SRLMC, explains how the wrong EBM in Fig 1(a) can generate two modes, each of
which has same probability mass as the corresponding data mode. But, it does
not explain how the EBM, which has wide modes, can generate data, which has
narrow modes. Incorrect step size and noise scale ratio in (5) is to blame.

######  Proposition 2.

Assume the sequences $\\{\alpha_{t}\\}$ and $\\{\beta_{t}\\}$ in (5) satisfy
$\alpha_{t}/\beta_{t}=\rho$ for some $\rho>0$ for all $t$. Also, assume the
sequence generated by (5) converges. Then $\\{x_{t}\\}$ converges in
distribution to

$\displaystyle
q_{\theta}^{\rho}(x)\coloneqq\frac{1}{Z(\theta,\rho)}\exp\left\\{-\rho
E_{\theta}(x)\right\\}.$ (8)

Proposition 2 tells us that $\rho$ controls the sharpness of the sampled
distribution. LMC with large $\rho$ will sample from a sharpened version of
$q_{\theta}$, and LMC with small $\rho$ will sample from a wide version of
$q_{\theta}$. The middle panel of Figure 2 illustrates this fact. Together
with the observation that SRLMC does not mix well, we can explain how the EBM
of Figure 1(a) can generate training data with SRLMC.

Since SRLMC does not mix well, without loss of generality, we can focus on one
mode of the EBM. As mentioned in Section 3.1, MLE with SRLMC uses large $\rho$
to accelerate convergence. So, SRLMC samples from a sharpened version of
$q_{\theta}$. Hence, though the EBM has wide modes, $\tilde{q}_{\theta}$ has
sharp modes which match the training data distribution. Furthermore,
Proposition 2 can be used to prove:

######  Proposition 3.

Assume the EBM $q_{\theta}$ is trained via MLE with convergent modified LMC
(5) with $\alpha_{t}/\beta_{t}=\rho>0$. Then, $\theta$ such that

$\displaystyle q_{\theta}(x)\propto p(x)^{1/\rho}.$ (9)

is a stationary point of MLE with gradient ascent.

Indeed, the third panel of Figure 2 shows smaller $\rho$ leads to sharper
modes and larger $\rho$ leads to even wider modes. So, combined with incorrect
probability mass within modes due to poor mixing of SRLMC, larger $\rho$
exacerbates OOD data density overestimation.

A replay buffer does not help. Let us now consider the scenario where we use a
replay buffer as well. Figure 1(b) shows that MLE with SRLMC and a buffer has
converged to essentially the same solution (up to reflection w.r.t. the
y-axis) as SRLMC without the buffer. This implies that the buffer does not
alleviate OOD data overestimation.

In fact, Figure 3(a) shows that as the EBM converges to the same solution as
that of Figure 1(b), the buffer sample distribution converges to the data
distribution. Next, as illustrated in Figure 3(b), even if SRLMC is
initialized from buffer samples (which are now equal to data samples), SRLMC
still does not mix well: SRLMC initialized from the left mode stays at the
left mode, and SRLMC initialized from the right mode stays at the right mode.
Thus, a replay buffer does not help mixing of SRLMC, so SRLMC with replay
buffer suffers from the same problems as MLE with SRLMC without buffer.

(a) Replay buffer evolution

(b) SRLMC initialized from buffer (data)

Figure 3: Analysis of MLE with SRLMC and replay buffer.

###  4.1 A Simple Method that Avoids the Pitfalls of SRLMC on Low Dimensions

So far, we have shown the heuristic modifications of SRLMC in Section 3.1
admits stationary points to the MLE problem which exhibit OOD data density
overestimation. Several factors, poor mixing of SRLMC and incorrect step size
and noise scale ratio, play a role in this. Furthermore, we have provided
experimental and theoretical evidence that these factors could cause an EBM to
converge to that problematic point. So, to train EBMs with correct density
estimates, we need a method to estimate integral w.r.t. $q_{\theta}$ which is
resilient to getting trapped in modes and accurate at estimating probability
mass within each mode.

In low dimensions, there is a method which meets these desiderata: self-
normalized importance sampling with the importance distribution as the uniform
distribution on $\Omega$, or equivalently, Riemann sum as an approximation to
the integral. Concretely, let us assume $q_{\theta}$ is supported on a subset
of a compact domain $\Omega$. Then, given a set of points
$\\{u_{i}\\}_{i=1}^{n}$ which are uniformly sampled from $\Omega$ or form a
partition of $\Omega$, we can approximate the expectation of a function $f$
with respect to $q_{\theta}$ as

$\displaystyle\mathbb{E}_{q_{\theta}}[f(x)]=\int_{\Omega}f(x)q_{\theta}(x)\,dx=\frac{\int_{\Omega}f(x)\exp\\{-E_{\theta}(x)\\}\,dx}{\int_{\Omega}\exp\\{-E_{\theta}(x)\\}\,dx}\approx\frac{\frac{1}{n}\sum_{i=1}^{n}f(u_{i})\exp\\{-E_{\theta}(u_{i})\\}}{\frac{1}{n}\sum_{j=1}^{n}\exp\\{-E_{\theta}(u_{j})\\}}$

which can be concisely written as

$\displaystyle\mathbb{E}_{q_{\theta}}[f(x)]\approx\sum_{i=1}^{n}w_{i}f(u_{i}),\qquad
w_{i}\coloneqq\frac{\exp\\{-E_{\theta}(u_{i})\\}}{\sum_{j=1}^{n}\exp\\{-E_{\theta}(u_{j})\\}}.$
(10)

In the perspective of self-normalized importance sampling, the approximation
(10) converges to the expectation as the size of uniform distribution samples
$n\rightarrow\infty$. In the perspective of Riemann sum, the approximation
converges to the true value as $n\rightarrow\infty$ and the norm of the
partition, i.e., the maximum distance between two points in
$\\{u_{i}\\}_{i=1}^{n}$, converges to zero.

According to Figure 1(c), using the MLE gradient (3) approximated by this
method shows excellent performance on the problem of learning the mixture of
two Gaussians. Contrary to SRLMC, this method is not affected by the fact that
the two Gaussians have approximately disjoint support. So, the EBM trained by
self-normalized importance sampling / Riemann sum places correct probability
mass within each mode.

Unfortunately, in general, this method is not applicable to learning high-
dimensional distributions. Suppose $p$ is the distribution of natural images,
where we set $\Omega=[0,1]^{d}$. Also, assume $\\{u_{n}\\}_{i=1}^{n}$ in (10)
is distributed uniformly on $\Omega$, and $d$ is the dimension of images,
where $d$ is generally very large. By Proposition 1, most samples from the
uniform distribution on $\Omega$ lie on a thin shell $S$. The volume of $S$
compared to the volume of $\Omega$ is vanishingly small, so
$\\{u_{i}\\}_{i=1}^{n}$ is unlikely to come from the high-density regions of
$q_{\theta}$. Thus, the approximation (10) becomes increasingly inaccurate
with larger $d$. Hence, in the next section, we propose a way to make self-
normalized importance sampling / Riemann sum work in high dimensions.

##  5 Uniform Support Partitioning (USP)

To overcome the curse of dimensionality described in Section 4.1, we propose
finding $\\{u_{i}\\}_{i=1}^{n}$ which lie uniformly on the support of
$q_{\theta}$. To this end, we solve the following optimization problem:

$\displaystyle\max_{u_{i}\in\Omega}\sum_{i=1}^{n}\log
q_{\theta}(u_{i})\quad\text{subject to}\quad\|u_{i}-u_{j}\|_{2}\geq\epsilon\
\text{for all}\ i\neq j.$ (11)

Here, $\epsilon$ is a parameter which controls the fineness of the
partition444If the Lebesgue measure of $\Omega$ is positive, the feasible set
of (11) will be nonempty for sufficiently small $\epsilon$.. Intuitively, the
above problem fills up the support of $q_{\theta}$ with $\epsilon$-balls
centered at $\\{u_{i}\\}_{i=1}^{n}$ with priority on high density regions. The
points $\\{u_{i}\\}_{i=1}^{n}$ are then used to approximate
$\mathbb{E}_{q_{\theta}}[\nabla_{\theta}E_{\theta}(x)]$ in the MLE gradient
(3) using the formula (10).

To solve (11), we take motivation from projected gradient ascent (PGA).
Specifically, we iterate between a maximization step and a projection step. In
the maximization step, we locally push each $u_{i}$ in the direction which
maximizes the density. In the projection step, we perturb each $u_{i}$ such
that the pairwise distance between points in $\\{u_{i}\\}_{i=1}^{n}$ is
$\geq\epsilon$. In the following paragraph, we give the full detail of our
algorithm, which we call Uniform Support Partitioning (USP).

USP proceeds by iterating two steps. Suppose $\\{u_{i}\\}_{i=1}^{n}$ is the
set of points produced by USP in the previous iteration. The first step,
called the maximization step, seeks new points $u_{i}^{\prime}$ in the
proximity of $u_{i}$ which maximize the log-density. Specifically, we solve

$\displaystyle\qquad\max_{u_{i}^{\prime}\in\Omega}\sum_{i=1}^{n}\log
q_{\theta}(u_{i}^{\prime})$ (12)

via projected gradient ascent (PGA). The second step, called the repulsion
step, repels the points $\\{u_{i}\\}_{i=1}^{n}$ apart so the constraint of
(11) is satisfied. Since there is no closed form formula for projecting an
arbitrary set of points on the constraint set of (11), we use the gradient
method to repel points from one another. Concretely, we solve

$\displaystyle\max_{u_{i}\in\Omega}\sum_{i\neq
j}\min\\{\|u_{i}-u_{j}\|_{2},\epsilon\\}$ (13)

again via PGA. In practice, there is no guarantee that particles
$\\{u_{i}\\}_{i=1}^{n}$ converge to the solution of (11), since we do not use
an exact projection step. Nonetheless, as we will show in Section 6, we find
this poses no problem in learning EBMs.

###  5.1 Persistent Stochastic USP (PS-USP) for Training Deep EBMs

If we are to train a deep EBM with USP, we face two problems: (a) if $n$ is
large, each evaluation of the objective of (12) and (13) can be expensive, and
(b) running USP until convergence at each iteration of EBM gradient update can
be computation costly. We introduce two modifications to USP which address
these problems.

Stochastic updates. Suppose we wish to run USP on $\\{u_{i}\\}_{i=1}^{n}$
where $n$ is very large. At each iteration of USP, we randomly choose
$\Lambda\subsetneq[n]$ and optimize $\\{u_{i}\\}_{i\in\Lambda}$. The
maximization step with $\\{u_{i}\\}_{i\in\Lambda}$ poses no problem, as the
objective of (12) is separable. However, naively applying the repulsion step
(13) to only $\\{u_{i}\\}_{i\in\Lambda}$ can cause $\\{u_{i}\\}_{i\in\Lambda}$
to collapse into the same configuration as $\\{u_{j}\\}_{j\in[n]-\Lambda}$.
So, to alleviate this issue, at each iteration of PGA of the repulsion step,
we solve

$\displaystyle\max_{\\{u_{i}\\}_{i\in\Lambda}\in\Omega^{|\Lambda|}}\sum_{i\in\Lambda}\sum_{j\in\Lambda\cup\Gamma}1_{i\neq
j}\cdot\min\\{\|u_{i}-u_{j}\|_{2},\epsilon\\}.$ (14)

where $\Gamma\subset[n]-\Lambda$ is sampled uniformly at random.

Persistent USP. In the spirit of persistent contrastive divergence
tieleman2008 , we only run a small number of USP iterations before each
gradient update of the EBM. While we have no theoretical justification for
this choice, experiments that follow show persistent USP performs sufficiently
well.

USP with the above modifications is called Persistent Stochastic USP (PS-USP).
One iteration of EBM parameter $\theta$ update with PS-USP proceeds as
follows: (a) sample $\Lambda\subsetneq[n]$, (b) apply the maximization step to
$\\{u_{i}\\}_{i\in\Lambda}$ via $n_{m}$ steps of PGA, (d) apply the repulsion
step with (14) with $n_{r}$ steps of PGA, (e) repeat steps (b) to (d) $N$
times, (e) choose $n_{s}$ points from $\\{u_{i}\\}_{i=1}^{n}$ and calculate
$\mathbb{E}_{q_{\theta}}[\nabla_{x}E_{\theta}(x)]$ via (10), (f) calculate the
MLE gradient (3) and update $\theta$ via gradient ascent.

(a) MoG density

(b) SRLMC result

(c) PS-USP results

Figure 4: Comparison of MLE with SRLMC and PS-USP on a 2D MoG.

##  6 Numerical Experiments

###  6.1 2 Dimensional Mixture of Gaussians (2D MoG)

We first consider the problem of learning a deep EBM on a 2D mixture of
Gaussians (MoG) with six modes (Figure 4(a)). To check the robustness SRLMC
and PS-USP to separated modes, for SRLMC, we set $q_{0}$ as the rightmost mode
of the MoG, and for PS-USP, we initialize $\\{u_{i}\\}_{i=1}^{n}$ as samples
from the rightmost mode of the MoG. Each method was run until the EBM
converged.

Figure 4(b) shows the result of using SRLMC to train a deep EBM to approximate
the MoG. We again observe that the EBM exhibits incorrect probability mass
ratio of the modes due to poor mixing of SRLMC. Also, the EBM density is
blurry due to the wrong step size and noise scale ratio. Consequently, some
OOD regions have higher density than the rightmost mode.

On the other hand, according to Figure 4(c), the EBM trained by PS-USP has
accurately learned the MoG. Moreover, the partition points are almost
uniformly distributed on the support of the EBM. Thus, PS-USP mitigates the
pitfalls of SRLMC .

###  6.2 Fashion-MNIST

We now turn to the more challenging task of training deep EBMs on Fashion-
MNIST fmnist and using the EBM for OOD data detection. We use a simple CNN-
based discriminative model as the energy function, without any special
structure such as bottlenecks. After training, we evaluate the OOD data
detection performances of EBMs using density values only. Evaluation metrics
are false positive rate at true positive rate $95\%$ (FPR95) and the area
under the precision-recall curve (AUPR). OOD data are MNIST mnist , KMNIST
kmnist , NotMNIST nmnist , Constant which consists of constant-valued images
whose values are sampled from the uniform distribution on $[0,1]$, and Noise
which consists of a mixture of uniform noise and standard Gaussian noise.

Table 1 compares the OOD data detection performances. We observe that except
in the case of Noise, which is an easy OOD data to detect, EBMs trained with
PS-USP beat EBMs trained with SRLMC by a nontrivial margin. In particular,
FPR95 scores show significant gaps. This provides concrete evidence that USP
can avoid the pitfalls of SRLMC in training deep EBMs. Also, Figure 5 displays
partition points found by PS-USP for the EBM trained by PS-USP. We observe
that the partition points resemble actual FMNIST data, which implies the EBM
has successfully learned to approximate the FMNIST distribution.

OOD Data | MNIST | KMNIST | NotMNIST | Constant | Noise
---|---|---|---|---|---
Statistic | FPR95 $\downarrow$ | AUPR $\uparrow$ | FPR95 $\downarrow$ | AUPR $\uparrow$ | FPR95 $\downarrow$ | AUPR $\uparrow$ | FPR95 $\downarrow$ | AUPR $\uparrow$ | FPR95 $\downarrow$ | AUPR $\uparrow$
SRLMC | $97.73_{\pm 0.76}$ | $86.01_{\pm 4.94}$ | $50.56_{\pm 6.02}$ | $90.57_{\pm 3.13}$ | $12.27_{\pm 3.06}$ | $99.05_{\pm 0.21}$ | $20.03_{\pm 12.47}$ | $99.89_{\pm 0.10}$ | $0.0_{\pm 0.0}$ | $100.0_{\pm 0.0}$
PS-USP | $2.18_{\pm 1.03}$ | $99.15_{\pm 0.29}$ | $1.84_{\pm 1.01}$ | $99.32_{\pm 0.12}$ | $7.04_{\pm 0.57}$ | $99.07_{\pm 0.27}$ | $0.97_{\pm 0.78}$ | $99.99_{\pm 0.00}$ | $0.0_{\pm 0.0}$ | $100.0_{\pm 0.0}$

Table 1: Comparison of OOD data detection performances on FMNIST. $\downarrow$
indicates lower is better, and $\uparrow$ means higher is better. We report
the mean and standard deviation over three trials. Figure 5: An example of EBM
support partition points found by PS-USP on FMNIST.

##  7 Limitations and Societal Impacts

Limitations. The main limitation of this work is that we do not have
convergence guarantees for USP and PS-USP. This is because we use approximate
projection instead of exact projection in the repulsion step. Yet, we observed
that USP performs well in practice. We believe a search for convergence proofs
of USP could lead to better ways of solving (11) and ultimately better
algorithms for training deep EBMs. Thus, we believe this is a promising
direction for future research.

Societal impacts. As EBMs are generative models, there is always a possibility
that it may be exploited for malicious purposes, e.g., for producing Deepfakes
or fake information. On the other hand, there are increasing efforts to apply
EBMs in safety critical domains such as medical imaging. We believe our
research is necessary for increasing the robustness of such machine learning
systems, as our research concerns making deep generative models “know what
they don’t know” nalisnick2019b .

##  8 Conclusions

In this work, we investigated why EBMs assign high density to OOD regions. We
found that poor mixing of SRLMC and incorrect step size and noise scale ratio
were the causes. Motivated by these observations, we proposed a novel
numerical integration method, USP which finds a uniform partition of the EBM
support and uses the partition points to calculate the MLE gradient. We
demonstrated on a MoG data that USP overcomes the pitfalls of SRLMC. Further,
we showed that EBMs trained by USP has significantly better OOD data detection
performance on FMNIST. We believe a theoretical analysis of USP could lead to
better EBM training algorithms, and leave this for future work.

## References

  * [1] Diederik P. Kingma and Max Welling.  Auto-encoding variational bayes.  In ICLR, 2013.
  * [2] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial nets.  In NeurIPS, 2014.
  * [3] Hyunsun Choi, Eric Jang, and Alexander A. Alemi.  Waic, but why? generative ensembles for robust anomaly detection.  arXiv:1810.01392, 2018.
  * [4] Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, and Balaji Lakshminarayanan.  Detecting out-of-distribution inputs to deep generative models using typicality.  arXiv:1906.02994, 2019.
  * [5] Jie Ren, Peter J. Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark A. DePristo, Joshua V. Dillon, and Balaji Lakshminarayanan.  Likelihood ratios for out-of-distribution detection.  In NeurIPS, 2019.
  * [6] Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich.  Deep anomaly detection with outlier exposure.  In ICLR, 2019.
  * [7] Joan Serrà, David Álvarez, Vicenç Gómez, Olga Slizovskaia, José F. Núñez, and Jordi Luque.  Input complexity and out-of-distribution detection with likelihood-based generative models.  In ICLR, 2020.
  * [8] Yilun Du and Igor Mordatch.  Implicit generation and generalization in energy-based models.  In NeurIPS, 2019.
  * [9] Yann LeCun, Sumit Chopra, Raia Hadsell, Marc’Aurelio Ranzato, and Fu Jie Huang.  A tutorial on energy-based learning.  Predicting Structured Data, 2006.
  * [10] Yang Song and Diederik P. Kingma.  How to train your energy-based models.  arxiv preprint arXiv:2101.03288, 2021.
  * [11] Yang Song and Stefano Ermon.  Generative modeling by estimating gradients of the data distribution.  In NeurIPS, 2019.
  * [12] Sven Elflein, Bertrand Charpentier, Daniel Zügner, and Stephan Günnemann.  On out-of-distibution detection with energy-based models.  In ICML Workshop on Uncertainty and Robustness in Deep Learning, 2021.
  * [13] Ahsan Mahmood, Junier Oliva, and Martin Styner.  Multiscale score matching for out-of-distribution detection.  In ICLR, 2021.
  * [14] Han Xiao, Kashif Rasul, and Roland Vollgraf.  Fashion-mnist: a novel dataset for benchmarking machine learning algorithms.  arxiv preprint arXiv:1708.07747, 2017.
  * [15] Max Welling and Yee Whye Teh.  Bayesian learning via stochastic gradient langevin dynamics.  In ICML, 2011.
  * [16] Erik Nijkamp, Mitch Hill, Tian Han, Song-Chun Zhu, and Ying Nian Wu.  Learning non-convergent short-run mcmc toward energy-based model.  In NeurIPS, 2019.
  * [17] Will Grathwohl, Kuan-Chieh Wang, Jörn-Henrik Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swerwky.  Your classifier is secretly and energy based model and you should treat it like one.  In ICLR, 2020.
  * [18] Xiulong Yang and Shihao Ji.  Jem++: Improved techniques for training jem.  In ICCV, 2021.
  * [19] Erik Nijkamp, Mitch Hill, Tian Han, Song-Chun Zhu, and Ying Nian Wu.  On the anatomy of mcmc-based maximum likelihood learning of energy-based models.  In AAAI, 2020.
  * [20] Diederik P. Kingma and Prafulla Dhariwal.  Glow: Generative flow with invertible 1x1 convolutions.  In NeurIPS, 2018.
  * [21] Tim Salimans, Andrej Karpathy, Xi Chen, and Diederik P. Kingma.  Pixelcnn++: Improving the pixelcnn with discretized logistic mixture likelihood and other modifications.  In ICLR, 2017.
  * [22] Arnak S. Dalalyan.  Theoretical guarantees for approximate sampling from smooth and log-concave densities.  Journal of the Royal Statistical Society, 79(3):651–676, 2017.
  * [23] Tijmen Tieleman.  Training restricted boltzmann machines using approximations to the likelihood gradient.  In Proceedings of the 25th International Conference on Machine Learning, 2008.
  * [24] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner.  Gradient-based learning applied to document recognition.  Proc. of the IEEE, 86(11):2278–2324, 1998.
  * [25] Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, and David Ha.  Deep learning for traditional japanese literature.  arXiv:1812.01718, 2018.
  * [26] Yaroslav Bulatov.  Machine learning, etc: notmnist dataset.  2011\.
  * [27] Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan.  Do deep generative models know what they don’t know?  In ICLR, 2019.

##  Appendix A Missing Proofs

Proposition 1. Suppose $X$ is a $d$-dimensional random vector whose components
are i.i.d. with mean $\mu$, variance $\sigma^{2}$, and finite fourth moment.
Then, for any $\epsilon>0$,

$\displaystyle\lim_{d\rightarrow\infty}\mathbb{P}\left\\{(1-\epsilon)\sqrt{d(\sigma^{2}+\mu^{2})}<\|X\|_{2}<(1+\epsilon)\sqrt{d(\sigma^{2}+\mu^{2})}\right\\}=1.$

###### Proof.

We observe that

$\displaystyle\operatorname{var}(X_{i}^{2})\leq\mathbb{E}[X_{i}^{4}]<\infty$
(15)

by assumption. So, by the $L^{2}$ weak law of large numbers,

$\displaystyle\frac{1}{d}\|X\|_{2}^{2}=\frac{1}{d}\sum_{i=1}^{d}X_{i}^{2}\rightarrow\mathbb{E}[X_{i}^{2}]=\sigma^{2}+\mu^{2}$
(16)

in probability as $d\rightarrow\infty$. This implies the claim of the
proposition. ∎

Proposition 2. Assume the sequences $\\{\alpha_{t}\\}$ and $\\{\beta_{t}\\}$
in (5) satisfy $\alpha_{t}/\beta_{t}=\rho$ for some $\rho>0$ for all $t$.
Also, assume the sequence generated by (5) converges. Then $\\{x_{t}\\}$
converges in distribution to

$\displaystyle
q_{\theta}^{\rho}(x)\coloneqq\frac{1}{Z(\theta,\rho)}\exp\left\\{-\rho
E_{\theta}(x)\right\\}.$

###### Proof.

It is known that with the LMC iteration

$\displaystyle
x_{t+1}=x_{t}-\frac{\eta_{t}}{2}\nabla_{x}E_{\theta}(x)+\sqrt{\eta_{t}}\epsilon_{t},\qquad
t=0,1,2,\ldots,T,$ (17)

the sequence $\\{x_{t}\\}$ converges to

$\displaystyle q_{\theta}(x)=\frac{1}{Z(\theta)}\exp\\{-E_{\theta}(x)\\}$ (18)

in distribution for an appropriate choice of $\\{\eta_{t}\\}$ welling2011 ;
dalalyan2017 . We now consider the modified iteration

$\displaystyle
x_{t+1}=x_{t}-\frac{\alpha_{t}}{2}\nabla_{x}E_{\theta}(x)+\sqrt{\beta_{t}}\epsilon_{t},\qquad
t=0,1,2,\ldots,T.$ (19)

By the assumption $\alpha_{t}=\rho\beta_{t}$, the modified iteration is
equivalent to

$\displaystyle x_{t+1}$
$\displaystyle=x_{t}-\frac{\rho\beta_{t}}{2}\nabla_{x}E_{\theta}(x)+\sqrt{\beta_{t}}\epsilon_{t},\qquad
t=0,1,2,\ldots,T$ (20) $\displaystyle=x_{t}-\frac{\beta_{t}}{2}\nabla_{x}(\rho
E_{\theta}(x))+\sqrt{\beta_{t}}\epsilon_{t},\qquad t=0,1,2,\ldots,T.$ (21)

Since we have assumed the sequence generated by (5) converges, by comparing
(21) with (17), we conclude that the sequence $\\{x_{t}\\}$ generated by (20)
must converge to

$\displaystyle
q_{\theta}^{\rho}(x)\coloneqq\frac{1}{Z(\theta,\rho)}\exp\left\\{-\rho
E_{\theta}(x)\right\\}$ (22)

in distribution. ∎

Proposition 3. Assume the EBM $q_{\theta}$ is trained via MLE with convergent
modified LMC (5) with $\alpha_{t}/\beta_{t}=\rho>0$. Then, $\theta$ such that

$\displaystyle q_{\theta}(x)\propto p(x)^{1/\rho}.$

is a stationary point of MLE with gradient ascent.

###### Proof.

Let $\hat{q}_{\theta}$ be the distribution of $\hat{x}$ produced by running
convergent modified LMC (5) with $\alpha_{t}/\beta_{t}=\rho>0$ on some
proposal sample $x\sim q_{0}$. Then, EBM gradient update with MLE with (5)
becomes

$\displaystyle\theta\leftarrow\theta+\mathbb{E}_{\hat{q}_{\theta}}[\nabla_{\theta}E_{\theta}(x)]-\mathbb{E}_{p}[\nabla_{\theta}E_{\theta}(x)]$
(23)

and thus stationarity is achieved when

$\displaystyle\hat{q}_{\theta}=p.$ (24)

By Proposition 2, we have

$\displaystyle q_{\theta}^{\rho}=\hat{q}_{\theta},$

so if $q_{\theta}(x)\propto p(x)^{1/\rho}$, (24) is satisfied. ∎

##  Appendix B Experiment Details

###  B.1 Experiments in Section 4

To learn the one-dimensional mixture of Gaussians, we use EBMs whose energy
function is the squared distance bewteen input and output of a multi-layer
perceptron (MLP) with four layers, each with 512 hidden units and leaky-ReLU
activations with negative slope $0.2$. For SRLMC, we set $T=40$,
$\alpha_{t}=0.001$, and $\beta_{t}=0.0001$. The replay buffer size is $50k$
and SRLMC chain reinitialization rate is $0.05$. For all methods, batch size
is $1k$ and the optimizer is SGD with no momentum and learning rate $0.01$.
Each EBM was trained for $5k$ iterations on a single GTX 1080 GPU.

###  B.2 Experiments in Section 6

2D MoG. MoG consists of six Gaussians, whose means are
$(\cos\theta,\sin\theta)$ for $\theta\in\\{n\pi/3:n=0,1,\ldots,5\\}$ and
covariance matrices are $\sigma^{2}\mathbf{I}$ for $\sigma=0.1$. We use EBMs
whose energy function is the output of a MLP with four layers, each with 512
hidden units and leaky-ReLU activations with negative slope $0.2$. For SRLMC,
we set $T=40$ and $\alpha_{t}=0.001$, and $\beta_{t}=0.0001$. The replay
buffer size is $50k$, SRLMC chain reinitialization rate is $0.05$, and batch
size is $1k$. For PS-USP, we set $n_{m}=1$ and $n_{r}=1$ so we can combine
maximization and repulsion into a single iteration (if $u_{i}$ does not
violate constraint, apply maximization, otherwise, apply repulsion), and
$N=50$. We set $\epsilon=0.05$, $n=5k$, $|\Lambda|=1k$, and $n_{s}=5k$. For
all methods, the optimizer is SGD with no momentum and learning rate $0.001$.
Each EBM was trained until convergence with a single GTX 1080 GPU.

FMNIST. The FMNIST dataset was scaled into the range $[-1,1]$. For SRLMC, we
also added Gaussian noise of standard deviation $0.1$ following the
recommendation of previous works nijkamp2019 ; grathwohl2020 ; yang2021 . We
use EBMs whose energy function is the output of a CNN with three convolution
layers followed by two fully-connected layers. We use the leaky-ReLU
activation with negative slope $0.4$. Each convolution layer has number of
filters $\in\\{32,64,128\\}$ with filter size $3$ and stride $1$. Each
convolution layer activation is followed by an average pooling layer with
kernel size $2$ and stride $2$. For SRLMC, we set $T=20$ and $\alpha_{t}=2.0$
and $\beta_{t}=0.01$. The replay buffer size is $50k$, SRLMC chain
reinitialization rate is $0.05$, and batch size is $125$. For PS-USP, we
proceed similar to the case of 2D MoG. We set $n_{m}=1$, $n_{r}=1$, $N=100$,
$\epsilon=10$, $n\in\\{10k,25k,50k\\}$, $|\Lambda|=125$, and $n_{s}=625$. For
all methods, the optimizer is Adam with learning rate $0.001$. Each EBM was
trained for at most $50$ epochs with a single GTX 1080 GPU. We choose models
based on FPR95 on the OOD validation datasets of MNIST and KMNIST following
Elflein et. al elflein2021 .