# Attached prime ideals over skew Ore polynomials

Sebastián Higuera  Universidad Nacional de Colombia - Sede Bogotá Campus
Universitario<EMAIL_ADDRESS>and  Armando Reyes  Universidad Nacional
de Colombia - Sede Bogotá Campus Universitario<EMAIL_ADDRESS>Dedicated
to Martha Rincón

###### Abstract.

In this paper, we investigate the attached prime ideals of inverse polynomial
modules over skew Ore polynomials.

###### Key words and phrases:

Attached prime ideal, inverse polynomial module, skew polynomial ring, skew
Ore polynomials, Bass module.

###### 2020 Mathematics Subject Classification:

16D10, 16D60, 16D80, 16E45, 16S36, 16S85, 16W50.

The authors were supported by the research fund of Department of Mathematics,
Faculty of Science, Universidad Nacional de Colombia - Sede Bogotá, Colombia,
HERMES CODE 53880.

##  1\. Introduction

Throughout the paper, every ring $R$ is associative (not necessarily
commutative) with identity. If $R$ is commutative, then it is denoted by $K$.
If $N_{R}$ is a right module, then the right annihilator of $N_{R}$ is defined
by ${\rm ann}_{R}(N)=\left\\{r\in R\mid Nr=0\right\\}$, and $N_{R}$ is said to
be prime if $N_{R}\neq 0$ and ${\rm ann}_{R}(N)={\rm ann}_{R}(N^{\prime})$ for
all submodule $N_{R}^{\prime}$ of $N_{R}$. If $M_{R}$ is a right module, then
a right prime ideal $P$ of $R$ is called associated of $M_{R}$ if there exists
a prime submodule $N_{R}$ of $M_{R}$ such that $P={\rm ann}_{R}(N)$. The set
of all associated prime ideals of $M_{R}$ is denoted by ${\rm Ass}(M_{R})$
[29, p. 86]. These ideals have been widely studied in the literature. For
instance, Brewer and Heinzer [10] showed that the associated prime ideals of
the commutative polynomial ring $K[x]$ are all extended, that is, every
$P\in{\rm Ass}(K[x]_{K[x]})$ may be expressed as $P=Q[x]$, where $Q\in{\rm
Ass}(K_{K})$ (see also Faith [18]). Annin [3, 4, 5] extended this result to
the setting of skew polynomial rings in the sense of Ore [40, 41], while
Nordstrom [38, 39] computed the associated prime ideals of simple torsion
modules over generalized Weyl algebras defined by Bavula [9]. Later, Ouyang
and Birkenmeier [42] defined the nilpotent associated primes as a
generalization of the associated prime ideals and described these ideals over
skew polynomial rings. In the setting of the skew PBW extensions introduced by
Gallego and Lezama [21], Niño et al. [35] characterized the associated primes
of modules over these rings, Later, Higuera et al. [25, 26] studied the
nilpotent associated prime ideals of a skew PBW extension and investigated the
associated prime ideals of induced modules over this kind of noncommutative
rings.

Given the importance of primary decomposition theory and its relationships
with associated prime ideals, Macdonald [32] considered a dual theory to the
primary decomposition which is commonly referred as secondary representation
where the main ideals of this theory are called attached primes. According to
Baig [8], $M_{K}$ is called a secondary module if $M_{K}\neq 0$ and the
endomorphism $\phi_{r}$ of $M_{K}$ defined by $\phi_{r}(m):=mr$ for all $m\in
M_{K}$ is either surjective or nilpotent (that is, there exists
$k\in\mathbb{N}$ such that $\phi_{r}^{k}=0$), for each $r\in K$ [8, Definition
3.1.1]. Secondary modules are also called $P$-secondary since their nilradical
is a prime ideal $P$ of $K$ [8, Claim 3.1.2]. If $M_{K}$ has a secondary
representation, that is, $M_{K}=\sum_{i=0}^{n}M_{i}$ where each $M_{i}$ is
secondary, then $M_{K}$ is called representable. If $M_{i}$ is
$P_{i}$-secondary for every $1\leq i\leq n$ with $P_{i}\neq P_{j}$ for $i\neq
j$, and the sums $\sum_{i\neq k}M_{i}$ are proper submodules of $M_{K}$, then
the representation is called minimal [8, Definition 3.1.9]. The prime ideals
$P_{1},\dotsc,P_{n}$ are called attached of $M_{K}$ and the set of all
attached prime ideals of $M_{K}$ is denoted by ${\rm Att}^{*}(M_{K})$ [8,
Definition 3.2.2].

Melkersson [34] studied the attached prime ideals over commutative polynomial
extensions. He investigated when multiplication by $f(x)\in K[x]$ defines a
surjective endomorphism over the module $M[x^{-1}]_{K[x]}$ which consists of
all the polynomials of the form $m(x)=m_{0}+m_{1}x^{-1}+\cdots+m_{k}x^{-k}$
with $m_{i}\in M_{K}$ for all $0\leq i\leq k$. He showed that for $g$ and $h$
endomorphisms of $M_{K}$ such that $gh=hg$, if $g$ is surjective and $h$ is
nilpotent then $f:=g+h$ is a surjective endomorphism [34, Lemma 2.1], and with
this result, he proved that if $M_{K}$ is Artinian or has a secondary
representation, then $M=c(f)M$ where $c(f)$ is the ideal generated by the
coefficients of $f(x)$. As a corollary, he obtained a characterization of the
attached prime ideals of the right module $M[x^{-1}]_{K[x]}$ by showing that
if $Q\in{\rm Att}^{*}(M[x^{-1}]_{K[x]})$ then we obtain that $Q=P[x]$, where
$P\in{\rm Att}^{*}(M_{K})$ [34, Corollary 2.3].

Annin [6] introduced the concept of attached prime ideal for arbitrary modules
(not necessarily representables), and provided an extension of Macdonald’s
theory of secondary representation to the noncommutative setting. Following
Annin [6], $N_{R}$ is called coprime if $N_{R}\neq 0$ and ${\rm
ann}_{R}(N)={\rm ann}_{R}(Q)$ for all non-zero quotient module $Q_{R}$ of
$N_{R}$ [6, Definition 2.1]. A right prime ideal $P$ of $R$ is called attached
of $M_{R}$ if there exists a coprime quotient module $Q_{R}$ of $M_{R}$ such
that $P={\rm ann}_{R}(Q)$. The set of attached prime ideals of $M_{R}$ is
denoted by ${\rm Att}(M_{R})$ [6, Definition 2.3]. Annin [7] defined the
completely $\sigma$-compatible modules and extended Melkersson’s result to the
noncommutative setting. He proved that if $Q\in{\rm Att}(M[x^{-1}]_{S})$ then
$Q=P[x]$ where $P\in{\rm Att}(M_{R})$ and $S$ is the skew polynomial ring
$R[x;\sigma]$ with $\sigma$ an automorphism of $R$ [7, Theorem 3.2].

Cohn [13] introduced the skew Ore polynomials of higher order as a
generalization of the skew polynomial rings considering the relation
$xr:=\Psi_{1}(r)x+\Psi_{2}(r)x^{2}+\dotsb$ for all $r\in R$, where the
$\Psi$’s are endomorphisms of $R$. Following Cohn’s ideas, Smits [44]
introduced the ring of skew Ore polynomials of higher order over a division
ring $D$ and commutation rule defined by

(1.1) $\displaystyle xr:=r_{1}x+\cdots+r_{k}x^{k},\ \text{for all}\ r\in R\
\text{and}\ k\geq 1.$

The relation (1.1) induces a family of endomorphisms
$\delta_{1},\ldots,\delta_{k}$ of the group $(D,+)$ with
$\delta_{i}(r):=r_{i}$ for every $1\leq i\leq k$ [44, p. 211]. Smits proved
that if $\\{\delta_{2},\ldots,\delta_{k}\\}$ is a set of left $D$-independient
endomorphisms (i.e., if $c_{2}\delta_{2}(r)+\cdots+c_{k}\delta_{k}(r)=0$ for
all $r\in D$ then $c_{i}=0$ for all $2\leq i\leq k$ [44, p. 212]), then
$\delta_{1}$ is a injective endomorphism [44, p. 213]. There exist some
algebras such as Clifford algebras, Weyl-Heisenberg algebras, and Sklyanin
algebras, in which this commutation relation is not sufficient to define the
noncommutative structure of the algebras since a free non-zero term $\Psi_{0}$
is required. Maksimov [33] considered the skew Ore polynomials of higher order
with free non-zero term $\Psi_{0}(r)$ where $\Psi_{0}$ satisfies the relation
$\Psi_{0}(rs)=\Psi_{0}(r)s+\Psi_{1}(r)\Psi_{0}(s)+\Psi_{2}(r)\Psi_{0}^{2}(s)+\dotsb$,
for every $r,s\in R$. Later, Golovashkin and Maksimov [22] introduced the
algebras $Q(a,b,c)$ over a field $\Bbbk$ of characteristic zero with two
generators $x$ and $y$, and generated by the quadratic relations
$yx=ax^{2}+bxy+cy^{2}$, where $a,b,c\in\Bbbk$. If $\\{x^{m}y^{n}\\}$ forms a
basis for $Q(a,b,c)$, then the ring generated by the quadratic relation is an
algebra of skew Ore polynomials and can be defined by a system of linear
mappings $\delta_{0},\ldots,\delta_{k}$ of $\Bbbk[x]$ into itself such that
for any $p(x)\in\Bbbk[x]$,
$yp(x)=\delta_{0}(p(x))+\delta_{1}(p(x))y+\cdots+\delta_{k}(p(x))y^{k}$, for
some $k\in\mathbb{N}$.

Motivated by Annin’s research [7] about the attached prime ideals of
$M[x^{-1}]_{S}$ and the importance of the algebras of skew Ore polynomials of
higher order, in this paper we introduce a family of noncommutative rings
called skew Ore polynomials and we study the attached prime ideals of the
inverse polynomial module over these rings. Since some of its ring-
theoretical, homological and combinatorial properties have been investigated
recently (e.g., [12, 36, 37] and references therein), this article can be
considered as a contribution to research on skew Ore polynomials of higher
order.

The paper is organized as follows. Section 2 establishes some preliminaries
and key results about skew Ore polynomials. In Section 3, we introduce the
completely $(\sigma,\delta)$-compatible modules and present original results
(Propositions 3.4, 3.3 and 3.5). Under compatibility conditions, we also
characterize the attached prime ideals of the right module $M[x^{-1}]_{A}$
where $A$ is a skew Ore polynomial ring (Theorems 3.8 and 3.12). As expected,
our results extend those above corresponding to skew polynomial rings of
automorphism type presented by Annin [6, 7]. Finally, we present some ideas
for a future work.

The symbols $\mathbb{N}$, $\mathbb{Z}$, $\mathbb{R}$, and $\mathbb{C}$ denote
the set of natural numbers including zero, the ring of integer numbers, the
fields of real numbers and the complex numbers, respectively. The term module
will always mean right module unless stated otherwise. The symbol $\Bbbk$
denotes a field and $\Bbbk^{*}:=\Bbbk\ \backslash\ \\{0\\}$.

##  2\. Preliminaries

If $\sigma$ is an endomorphism of $R$, then a map $\delta:R\rightarrow R$ is
called a $\sigma$-derivation of $R$ if it is additive and satisfies that
$\displaystyle\delta(rs)=\sigma(r)\delta(s)+\delta(r)s$, for every $r,s\in R$
[23, p. 26]. Following Ore [40, 41], the skew polynomial ring (also called Ore
extension of $R$) over $R$ is defined as the ring $R[x;\sigma,\delta]$
generated by $R$ and $x$ such that it is a free left $R$-module with basis
$\left\\{x^{k}\ |\ k\in\mathbb{N}\right\\}$ and $xr:=\sigma(r)x+\delta(r)$ for
every $r\in R$ [23, p. 34].

A derivation $\delta$ of $R$ is called locally nilpotent if for all $r\in R$
there exists $n(r)\geq 1$ such that $\delta^{n(r)}(r)=0$ [20, p. 11].
Following the ideas of Cohn [13] and Smits [44], we introduce the following
kind of skew Ore polynomials of higher order.

######  Definition 2.1.

If $R$ is a ring, $\sigma$ is an automorphism of $R$ and $\delta$ is a locally
nilpotent $\sigma$-derivation of $R$, then we define the skew Ore polynomial
ring $A:=R(x;\sigma,\delta)$ which consists of the uniquely representable
elements $r_{0}+r_{1}x+\cdots+r_{k}x^{k}$ where $r_{i}\in R$ and
$k\in\mathbb{N}$, with the commutation rule $xr:=\sigma(r)x+x\delta(r)x$ for
all $r\in R$.

According to Definition (2.1), if $r\in R$ and $\delta^{n(r)}(r)=0$ for some
$n(r)\geq 1$, then

(2.1) $\displaystyle
xr=\sigma(r)x+\sigma\delta(r)x^{2}+\cdots+\sigma\delta^{n(r)-1}(r)x^{n(r)}.$

If we define the endomorphisms $\Psi_{i}:=\sigma\delta^{i-1}$ for all $i\geq
1$ and $\Psi_{0}:=0$, then $A$ is a skew Ore polynomial of higher order in the
sense of Cohn [13].

######  Example 2.2.

We present some examples of skew Ore polynomial rings.

  1. (1)

If $\delta=0$ then $xr=\sigma(r)x$ and thus $R(x;\sigma)=R[x;\sigma]$ is the
skew polynomial ring where $\sigma$ is an automorphism of $R$.

  2. (2)

The quantum plane $\Bbbk_{q}[x,y]$ is the free algebra generated by $x,y$ over
$\Bbbk$, and subject to the commutation rule $xy=qyx$ with $q\in\Bbbk^{*}$ and
$q\neq 1$. We note that $\Bbbk_{q}[x,y]\cong\Bbbk[y](x;\sigma)$ where
$\sigma(y):=qy$ is an automorphism of $\Bbbk[y]$.

  3. (3)

The Jordan plane $\mathcal{J}(\Bbbk)$ defined by Jordan [27] is the free
algebra generated by the indeterminates $x,y$ over $\Bbbk$ and the relation
$yx=xy+y^{2}$. This algebra can be a written as the skew polynomial ring
$\Bbbk[y][x;\delta]$ with $\delta(y):=-y^{2}$. On the other hand, notice that
$\delta(x)=1$ is a locally nilpotent derivation of $\Bbbk[x]$, and thus the
Jordan plane also can be interpreted as $\Bbbk[x](y;\delta)$.

  4. (4)

Díaz and Pariguan [14] introduced the $q$-meromorphic Weyl algebra $MW_{q}$ as
the algebra generated by $x,y$ over $\mathbb{C}$, and defining relation
$yx=qxy+x^{2}$, for $0<q<1$. Lopes [31] showed that using the generator
$Y=y+(q-1)^{-1}x$ instead of $y$, it follows that $Yx=qxY$ and thus the
algebra $MW_{q}$ can be written as a quantum plane $\mathbb{C}_{q}[x,y]$ [31,
Example 3.1]. Following examples (2) and (4), we conclude the algebra $MW_{q}$
is a skew Ore polynomial ring.

  5. (5)

Consider the algebra $Q(0,b,c)$ defined by Golovashkin and Maksimov [22] with
$a=0$. It is straightforward to see that $\sigma(x)=bx$ is an automorphism of
$\Bbbk[x]$ with $b\neq 0$, $\delta(x)=c$ is a locally nilpotent
$\sigma$-derivation of $\Bbbk[x]$ and so $Q(0,b,c)$ can be interpreted as
$A=\Bbbk[x](y;\sigma,\delta)$.

  6. (6)

If $\delta_{1}$ is an automorphism of $D$ and
$\\{\delta_{2},\ldots,\delta_{k}\\}$ is a set of left $D$-independient
endomorphism, then $\delta:=\delta_{1}^{-1}\delta_{2}$ is a
$\delta_{1}$-derivation of $D$, $\delta_{i+1}(r)=\delta_{1}\delta^{i}(r)$, and
$\delta^{k}(r)=0$ for all $r\in D$ [44, p. 214], and thus (1.1) coincides with
(2.1). In this way, the skew Ore polynomial rings of higher order defined by
Smits can be seen as $D(x;\delta_{1},\delta)$.

We need to investigate the localization technique to define the module
$M[x^{-1}]_{A}$ (Section 3.2). In the localization of noncommutative rings the
Ore condition plays an important role. A multiplicative subset $X$ of $R$
satisfies the left Ore condition if $Xr\cap Rx\neq\emptyset$ for every $r\in
R$ and $x\in X$. If $X$ satisfies the left Ore condition then $X$ is called a
left Ore set. Proposition 2.3 shows that the set containing all powers of $x$
satisfies the left Ore condition.

######  Proposition 2.3.

$X=\\{x^{k}\ |\ k\geq 0\\}$ is a left Ore set of the algebra $A$.

###### Proof.

It is clear that $X$ is a multiplicative subset of $A$, so we have to show
that $X$ satisfies the left Ore condition. Let
$a=r_{0}+r_{1}x+\cdots+r_{k}x^{k}$ be an element of $A$ with $r_{k}\neq 0$.
Since $\delta$ is locally nilpotent, for each $r_{i}$ in the expression of $a$
there exists $m_{i}\geq 0$ such that

$xr_{i}:=\sum_{j=1}^{m_{i}}\sigma(\delta^{j-1}(r_{i}))x^{j}=a_{i}x,$

where
$a_{i}:=\sigma(r_{i})+\sigma(\delta(r_{i}))x+\cdots+\sigma(\delta^{m_{i}-1}(r_{i}))x^{m_{i}-1}\in
A$. In this way, for each $r_{i}$ there exists $a_{i}\in A$ such that
$xr_{i}=a_{i}x$ for some $a_{i}\in A$, and so $xa=a^{\prime}x$ for some
$a^{\prime}\in A$. By induction on $p$, assume that for any $a\in A$ and
$x^{p}\in X$ there exists $\overline{a}\in A$ such that
$x^{p}a=\overline{a}x$. Thus, $x^{p+1}a=x\overline{a}x$ and since
$xa=\overline{a}^{\prime}x$ for some $\overline{a}^{\prime}\in A$, we get
$x^{p+1}a=a^{\prime\prime}x$ with $a^{\prime\prime}=\overline{a}^{\prime}x\in
A$. Hence, $X$ is a left Ore set of $A$. ∎

By Proposition 2.3, we can localize $A$ by $X$, and so we denote this
localization by $X^{-1}A$. It is straightforward to see that the indeterminate
$x^{-1}$ satisfies the relation
$x^{-1}r:=\sigma^{\prime}(r)x^{-1}+\delta^{\prime}(r)$, for all $r\in R$ with
$\sigma^{\prime}(r):=\sigma^{-1}(r)$ and
$\delta^{\prime}(r):=-\delta\sigma^{-1}(r)$. Dumas [17] studied the field of
fractions of $D[x;\sigma,\delta]$ where $\sigma$ is an automorphism of $D$ and
stated that one technique for this purpose is to consider it as a subfield of
a certain field of series [17, p. 193]. According to Dumas, if $Q$ is the
field of fractions of $D[x;\sigma,\delta]$ then $Q$ is a subfield of the field
of series of Laurent $D((x^{-1};\sigma^{-1},-\delta\sigma^{-1}))$ whose
elements are of the form
$r_{-k}x^{-k}+\cdots+r_{-1}x^{-1}+r_{0}+r_{1}x+\cdots$ for some
$k\in\mathbb{N}$, and satisfies the commutation rule

$\displaystyle xr$
$\displaystyle:=\sigma(r)x+\sigma\delta(r)x^{2}+\cdots=\sigma(r)x+x\delta(r)x,\
\text{and}$ $\displaystyle x^{-1}r$
$\displaystyle:=\sigma^{\prime}(r)x^{-1}+\delta^{\prime}(r),\ \text{for all}\
r\in R.$

By Definition 2.1 and Proposition 2.3, if $\sigma$ is an automorphism of $D$
and $\delta$ is a locally nilpotent $\sigma$-derivation of $D$ then
$X^{-1}A\subseteq D((x^{-1};\sigma^{-1},-\delta\sigma^{-1}))$ (see [1, 15, 16]
for more details about fields of series of Laurent).

######  Remark 2.4.

Following Lam et al. [28, p. 2468], if $\sigma$ is an automorphism of $R$ and
$\delta$ is a $\sigma$-derivation of $R$, then we denote by $f_{j}^{i}$ the
endomorphism of $R$ which is the sum of all possible words in
$\sigma^{\prime},\delta^{\prime}$ built with $i$ letters $\sigma^{\prime}$ and
$j-i$ letters $\delta^{\prime}$, for $i\leq j$. In particular, $f_{0}^{0}=1$,
$f_{j}^{j}=\sigma^{\prime j}$, $f_{j}^{0}=\delta^{\prime j}$, and
$f_{j}^{j-1}=\sigma^{\prime j-1}\delta^{\prime}+\sigma^{\prime
j-2}\delta^{\prime}\sigma^{\prime}+\cdots+\delta^{\prime}\sigma^{\prime j-1}$;
if $\delta\sigma=\sigma\delta$, then $f_{j}^{i}=\binom{j}{i}\sigma^{\prime
i}\delta^{\prime j-i}$. If $r\in R$ and $k\in\mathbb{N}$, then the following
formula holds:

(2.2) $\displaystyle\displaystyle x^{-k}r=\sum_{i=0}^{k}f_{k}^{i}(r)x^{-i}.$

In addition, if $r,s\in R$ and $k,k^{\prime}\in\mathbb{N}$ then

(2.3)
$\displaystyle\displaystyle(rx^{-k})(sx^{-k^{\prime}})=\sum_{i=0}^{k}rf_{k}^{i}(s)x^{-(k+k^{\prime})}.$

Taking into account the usual addition of polynomials and the product induced
by (2.2) and (2.3), we define the ring of polynomials in the indeterminate
$x^{-1}$ with coefficients in $R$ and denote it by $R[x^{-1}]$. If $M_{R}$ is
a right module then the inverse polynomial module $M[x^{-1}]_{R}$ is defined
as the set of all polynomials of the form $f(x)=m_{0}+\cdots+m_{k}x^{-k}$ with
$m_{i}\in M_{R}$ for all $i$, the usual addition of polynomials and the action
of $R$ over any monomial $mx^{-k}$ is defined by (2.2) as follows:

(2.4) $\displaystyle\displaystyle
mx^{-k}r:=\sum_{i=0}^{k}mf_{k}^{i}(r)x^{-i},\ \text{for all}\ m\in M_{R}\
\text{and}\ r\in R.$

######  Remark 2.5.

We use expressions as $m(x)=m_{0}+m_{1}x^{-1}+\cdots+m_{k}x^{-k}\in
M[x^{-1}]_{R}$. With this notation, we define the leading monomial of $m(x)$
as ${\rm lm}(m(x)):=x^{-k}$, the leading coefficient of $m(x)$ by ${\rm
lc}(m(x)):=m_{k}$, and the leading term of $m(x)$ as ${\rm
lt}(m(x)):=m_{k}x^{-k}$. The negative degree of $x^{-k}$ is defined by
$\deg(x^{-k}):=-k$ for any $k\in\mathbb{N}$, and $\deg(m(x)):={\rm
max}\\{\deg(x^{-i})\\}_{i=0}^{k}$ for all $m(x)\in M[x^{-1}]_{R}$. For any
element $m(x)\in M[x^{-1}]_{R}$, we denote by $C_{m}$ the set of all
coefficients of $m(x)$.

##  3\. Completely compatible rings and attached prime ideals

###  3.1. Completely $(\sigma,\delta)$-compatible rings

Annin [5] (c.f. Hashemi and Moussavi [24]) introduced the notion of
compatibility with the aim of studying the associated prime ideals of modules
over skew polynomial rings. If $\sigma$ is an endomorphism of $R$ and $\delta$
is a $\sigma$-derivation of $R$, then $M_{R}$ is called $\sigma$-compatible if
for each $m\in M_{R}$ and $r\in R$, $mr=0$ if and only if $m\sigma(r)=0$;
$M_{R}$ is $\delta$-compatible if for each $m\in M_{R}$ and $r\in R$, $mr=0$
implies $m\delta(r)=0$; if $M_{R}$ is both $\sigma$-compatible and
$\delta$-compatible, then $M_{R}$ is a ($\sigma,\delta$)-compatible module [5,
Definition 2.1]. Continuing with the compatibility conditions on modules,
$M_{R}$ is called completely $\sigma$-compatible if $(M/N)_{R}$ is
$\sigma$-compatible, for every submodule $N_{R}$ of $M_{R}$ [7, Definition
1.4]. Since completely $\sigma$-compatible modules were used by Annin [7] in
their research about attached prime ideals of $M[x^{-1}]_{S}$, it is to be
expected that we have to consider the completely $(\sigma,\delta)$-compatible
modules to characterize this type of ideals in the setting of the skew Ore
polynomials. In this way, we consider the following definition.

######  Definition 3.1.

If $\sigma$ is an endomorphism of $R$ and $\delta$ is a $\sigma$-derivation of
$R$ then $M_{R}$ is called completely $\sigma$-compatible if for all submodule
$N_{R}$ of $M_{R}$, $(M/N)_{R}$ is a $\sigma$-compatible module; $M_{R}$ is
completely $\delta$-compatible if for every submodule $N_{R}$ of $M_{R}$,
$(M/N)_{R}$ is $\delta$-compatible; $M_{R}$ is said to be completely
$(\sigma,\delta)$-compatible if it is both completely $\sigma$-compatible and
$\delta$-compatible.

######  Example 3.2.

  * (1)

If $M_{R}$ is simple and $(\sigma,\delta)$-compatible, then it is not
difficult to see that $M_{R}$ is completely $(\sigma,\delta)$-compatible.

  * (2)

Let $K$ be a local ring with maximal ideal $\mathfrak{m}$ and $\sigma$ any
automorphism of $K$. Annin [3] proved that $M_{K}:=K/\mathfrak{m}$ is a
$\sigma$-compatible module, and since $M_{K}$ is simple it follows that
$M_{K}$ is completely $\sigma$-compatible [3, Example 3.35]. Additionally, if
$\delta$ is a $\sigma$-derivation of $K$ such that $\delta(r)\in\mathfrak{m}$
for every $r\in\mathfrak{m}$, then $M_{K}$ is completely $\delta$-compatible.
Indeed, if $\overline{0}\neq\overline{s}\in M_{K}$ and $r\in K$ satisfy that
$\overline{s}r=0$ then $sr\in\mathfrak{m}$, and since $s\notin\mathfrak{m}$ we
obtain $r\in\mathfrak{m}$. If $\delta(r)\in\mathfrak{m}$ for every
$r\in\mathfrak{m}$, it follows that $s\delta(r)\in\mathfrak{m}$ and so
$\overline{s}\delta(r)=0$. Therefore, $M_{K}$ is $\delta$-compatible and thus
$M_{K}$ is completely $\delta$-compatible.

The following proposition presents some properties of completely
$(\sigma,\delta)$-compatible modules. These properties are required to prove
some results of the paper.

######  Proposition 3.3.

If $M_{R}$ is completely $(\sigma,\delta)$-compatible and $N_{R}$ is a
submodule of $M_{R}$ then the following assertions hold:

  1. (1)

If $ma\in N_{R}$ then $m\sigma^{i}(a),m\delta^{j}(a)\in N_{R}$ for each
$i,j\in\mathbb{N}$.

  2. (2)

If $mab\in N_{R}$ then
$m\sigma(\delta^{j}(a))\delta(b),m\sigma^{i}(\delta(a))\delta^{j}(b)\in N_{R}$
for all $i,j\in\mathbb{N}$. In particular, $ma\delta^{j}(b),m\delta^{j}(a)b\in
N_{R}$ for all $j\in\mathbb{N}$.

  3. (3)

If $mab\in N_{R}$ or $m\sigma(a)b\in N_{R}$ then $m\delta(a)b\in N_{R}$.

###### Proof.

If $M_{R}$ is completely $(\sigma,\delta)$-compatible, then $(M/N)_{R}$ is
$(\sigma,\delta)$-compatible. Considering the elements
$\overline{ma}=\overline{mab}=\overline{0}\in(M/N)_{R}$, the assertions follow
from [2, Lemma 2.15]. ∎

Annin proved some properties of completely $\sigma$-compatible modules [7, p.
539]. Proposition 3.4 extends these statements for completely
$(\sigma,\delta)$-compatible modules.

######  Proposition 3.4.

Let $\sigma$ be an endomorphism of $R$ and $\delta$ a $\sigma$-derivation of
$R$.

  * (1)

If $M_{R}$ is completely $(\sigma,\delta)$-compatible then $M_{R}$ is
$(\sigma,\delta)$-compatible.

  * (2)

If $M_{R}$ is a completely $(\sigma,\delta)$-compatible module then
$(M/N)_{R}$ is completely $(\sigma,\delta)$-compatible, for every submodule
$N_{R}$ of $M_{R}$.

###### Proof.

  * (1)

If $M_{R}$ is a completely $(\sigma,\delta)$-compatible module then
$(M/N)_{R}$ is a $(\sigma,\delta)$-compatible module, for every submodule
$N_{R}$ of $M_{R}$. In particular, $(M/\\{0\\})_{R}\cong M_{R}$ is
$(\sigma,\delta)$-compatible for the submodule $\\{0\\}_{R}$ of $M_{R}$.

  * (2)

Suppose that $M_{R}$ is completely $(\sigma,\delta)$-compatible and consider a
submodule $N^{\prime}/N$ of $(M/N)_{R}$ where $N^{\prime}_{R}$ is a submodule
of $M_{R}$ with $N\subsetneq N^{\prime}$. By the third isomorphism theorem for
modules, $((M/N)/(N^{\prime}/N))_{R}\cong(M/N^{\prime})_{R}$, and since
$M_{R}$ is completely $(\sigma,\delta)$-compatible, we have that
$(M/N^{\prime})_{R}$ is $(\sigma,\delta)$-compatible and thus
$((M/N)/(N^{\prime}/N))_{R}$ is $(\sigma,\delta)$-compatible, whence
$(M/N)_{R}$ is a completely $(\sigma,\delta)$-compatible module.

∎

We present other important property of completely $(\sigma,\delta)$-compatible
modules.

######  Proposition 3.5.

If $\sigma$ is bijective and $M_{R}$ is a completely
$(\sigma,\delta)$-compatible module, then $M_{R}$ is a completely
$(\sigma^{\prime},\delta^{\prime})$-compatible module.

###### Proof.

Assume that $M_{R}$ is completely $(\sigma,\delta)$-compatible. If $N_{R}$ is
a submodule of $M_{R}$ then $mr\in N_{R}$ if and only if $m\sigma(r)\in
N_{R}$, for all $m\in M_{R}$ and $r\in R$. Since $\sigma$ is bijective we have
$m\sigma^{-1}(r)\in N_{R}$ if and only if $mr\in N_{R}$, and so $(M/N)_{R}$ is
a $\sigma^{\prime}$-compatible proving that $M_{R}$ is completely
$\sigma^{\prime}$-compatible. On the other hand, if $M_{R}$ is completely
$\sigma^{\prime}$-compatible then $mr\in N_{R}$ which implies that
$m\sigma^{-1}(r)\in N_{R}$. If $M_{R}$ is completely $\delta$-compatible and
$m\sigma^{-1}(r)\in N_{R}$, then $m\delta\sigma^{-1}(r)$ and thus $M_{R}$ is
completely $\delta^{\prime}$-compatible. Therefore $M_{R}$ is completely
$(\sigma^{\prime},\delta^{\prime})$-compatible. ∎

###  3.2. Attached prime ideals

In this section, we define an $A$-module structure to the inverse polynomial
module $M[x^{-1}]_{R}$ and study the attached prime ideals of the module
$M[x^{-1}]_{A}$. The action of $A$ over $M[x^{-1}]_{R}$ is given by

(3.1) $\displaystyle mx^{-1}r$
$\displaystyle:=m\sigma^{\prime}(r)x^{-1}+m\delta^{\prime}(r)\ \text{for all}\
r\in R\ \text{and}\ m\in M_{R},\ \text{and}$ (3.2) $\displaystyle x^{-i}x^{j}$
$\displaystyle:=x^{-i+j}\ \text{if}\ j\leq i\ \text{and}\ 0\
\text{otherwise}.$

######  Remark 3.6.

By (3.1) and (3.2), if $\delta:=0$ then $mx^{-i}rx^{j}:=m\sigma^{\prime
i}(r)x^{-i+j}$ for all $r\in R$ and $i,j\in\mathbb{N}$ with $j\leq i$, which
coincides with $M[x^{-1}]_{S}$ [7, p. 538].

If $N_{R}$ is a right module and $N[x^{-1}]_{R}$ (resp., $N[x^{-1}]_{A}$) is a
right module, then the right annihilator is denoted by ${\rm
ann}_{R}(N[x^{-1}])$ (resp., ${\rm ann}_{A}(N[x^{-1}])$). The following lemma
characterizes the ideals generated by right prime ideals of $A$ that
correspond to annihilators of quotient modules of $M[x^{-1}]_{A}$.

######  Lemma 3.7.

If $M_{R}$ is completely $(\sigma,\delta)$-compatible and $P$ is a right prime
ideal of $R$ such that $P={\rm ann}_{R}(M/N)$ for some submodule $N_{R}$ of
$M_{R}$, then

$PA={\rm ann}_{A}(M[x^{-1}]/N[x^{-1}]).$

###### Proof.

By Propositions 3.3 and 3.5, if $M_{R}$ is completely
$(\sigma,\delta)$-compatible then $C_{mf}\subseteq N$ for every $m(x)\in
M[x^{-1}]_{A}$, $f(x)\in PA$, and thus $m(x)f(x)\in N[x^{-1}]_{A}$ proving
that $PA\subseteq{\rm ann}_{A}(M[x^{-1}]/N[x^{-1}])$. For the other inclusion,
if $f(x)\notin PA$ then there exists a monomial $r_{l}x^{l}$ of $f(x)$ such
that $r_{l}\notin P$ for some $0\leq l\leq j$. So there exists $m\in M_{R}$
such that $mr_{l}\notin N_{R}$, which implies that $mf(x)\notin N[x^{-1}]_{A}$
and so $f(x)\notin{\rm ann}_{A}(M[x^{-1}]/N[x^{-1}])$ whence ${\rm
ann}_{A}(M[x^{-1}]/N[x^{-1}])\subseteq PA$. ∎

Theorem 3.8 shows that right ideals of $A$ generated by attached prime ideals
of $M_{R}$ are attached prime ideals of $M[x^{-1}]_{A}$, and extends [7,
Theorem 2.1].

######  Theorem 3.8.

If $M_{R}$ is completely $(\sigma,\delta)$-compatible then

$\displaystyle{\rm Att}(M[x^{-1}]_{A})\supseteq\left\\{PA\ |\ P\in{\rm
Att}(M_{R})\right\\}.$

###### Proof.

If $P$ is an attached prime ideal of $M_{R}$ and $(M/N)_{R}$ is the quotient
coprime of $M_{R}$ such that $P={\rm ann}_{R}(M/N)$ for some submodule $N_{R}$
of $M_{R}$, it follows that $PA={\rm ann}_{A}(M[x^{-1}]/N[x^{-1}])$ by Lemma
3.7. Let us prove that $(M[x^{-1}]/N[x^{-1}])_{A}$ is a quotient coprime. If
$M[x^{-1}]/N[x^{-1}]\neq 0$ then there exists a submodule $Q_{A}$ of
$M[x^{-1}]_{A}$ such that $Q\supsetneq N[x^{-1}]$. Let $C_{Q}$ be the subset
of $M$ that consists of all coefficients of the elements of $Q$ and consider
$Q_{R}^{\prime}$ the submodule of $M_{R}$ generated by $C_{Q}$. Notice that if
$Q\neq M[x^{-1}]$ then $Q^{\prime}\neq M$, and if $(M/N)_{R}$ is a coprime
module then $P={\rm ann}_{R}(M/N)={\rm ann}_{R}(M/Q^{\prime})$.

If $g(x)=r_{0}+\cdots+r_{j}x^{j}\in{\rm ann}_{A}(M[x^{-1}]/Q)$ then
$f(x)g(x)\in Q_{A}$ and hence $C_{fg}\subseteq Q_{R}^{\prime}$ for all
$f(x)=m_{0}+\cdots+m_{k}x^{-k}\in M[x^{-1}]_{A}$. By Propositions 3.3 and 3.5,
if $M_{R}$ is completely $(\sigma,\delta)$-compatible then $m_{i}r_{j}\in
Q_{R}^{\prime}$, whence $m_{i}r_{j}\in N_{R}$ for all $i,j$. Since
$m_{i}r_{j}\in N_{R}$, we have that $m(x)f(x)\in N[x^{-1}]_{A}$ by
Propositions 3.3 and 3.5, and thus $M[x^{-1}]f(x)\subseteq N[x^{-1}]$.
Therefore $f(x)\in{\rm ann}_{A}(M[x^{-1}]/N[x^{-1}])$ proving that ${\rm
ann}_{A}(M[x^{-1}]/Q)\subseteq{\rm ann}_{A}(M[x^{-1}]/N[x^{-1}])$.

If $f\in{\rm ann}_{A}(M[x^{-1}]/N[x^{-1}])$ then $M[x^{-1}]f\subseteq
N[x^{-1}]$, and since $N[x^{-1}]\subsetneq Q$ we have that
$M[x^{-1}]f\subsetneq Q$ showing that ${\rm
ann}_{A}(M[x^{-1}]/N[x^{-1}])\subseteq{\rm ann}_{A}(M[x^{-1}]/Q)$, and thus
$(M[x^{-1}]/N[x^{-1}])_{A}$ is a coprime module. ∎

######  Corollary 3.9 ([7, Theorem 2.1]).

If $M_{R}$ is completely $\sigma$-compatible then

$\displaystyle{\rm Att}(M[x^{-1}]_{S})\supseteq\left\\{P[x]\ |\ P\in{\rm
Att}(M_{R})\right\\}.$

Lemma 3.10 shows when a coprime module over the ring of skew Ore polynomials
$A$ is a coprime module over $R$ and generalizes [7, Lemma 2.4].

######  Lemma 3.10.

If $P_{A}$ is a coprime module and $P_{R}$ is a completely
$(\sigma,\delta)$-compatible module then $P_{R}$ is coprime.

###### Proof.

Consider a submodule $Q_{R}$ of $P_{R}$ and let us show that ${\rm
ann}_{R}(P)={\rm ann}_{R}(P/Q)$. If $r\in{\rm ann}_{R}(P)$ then $Pr=0\in Q$,
and so $r\in{\rm ann}_{R}(P/Q)$. For the other inclusion, assume that
$r\in{\rm ann}_{R}(P/Q)$ and $N_{A}:=\sum Q_{A}^{\prime}$ where
$Q_{A}^{\prime}$ is any submodule over $A$ with $Q^{\prime}\subseteq Q$. So
$N_{A}\subseteq Q_{R}\subsetneq P_{A}$, and since $P_{A}$ is a coprime module
then ${\rm ann}_{A}(P)={\rm ann}_{A}(P/N)$. Let $p\in P_{A}$ and denotes by
$prA_{A}$ the module generated by the element $pr$. Let us prove that
$prA\subseteq N$. Since
$x^{-1}r:=\sigma^{\prime}(r)x^{-1}+\delta^{\prime}(r)$, we have
$rx=x\sigma^{\prime}(r)+x\delta^{\prime}(r)x$ for every $r\in R$, and thus if
$f(x)=r_{0}+\cdots+r_{l}x^{l}\in A$ and $Prr_{j}\subseteq Q$ for every $0\leq
j\leq l$, then $prf(x)\in Q_{A}$ by Propositions 3.3 and 3.5. Hence
$prA\subseteq Q$ and so $prA\subseteq N$ by definition of $N_{A}$. In this
way, $Pr\subseteq N$ which implies that $r\in{\rm ann}_{A}(P/N)={\rm
ann}_{A}(P)$ proving that ${\rm ann}_{R}(P)={\rm ann}_{R}(P/Q)$ for every
proper submodule $Q_{R}$ of $P_{R}$, that is, $P_{R}$ is a coprime module. ∎

For any submodule $P_{R}$ of $M[x^{-1}]_{R}$, we set $P_{k}:=\\{m\in M\ |\
mx^{-k}\in P\\}$ for each $k\in\mathbb{N}$ and denote by $\langle
P_{k}\rangle$ the submodule of $M_{R}$ generated by $P_{k}$. Lemma 3.11
guarantees the existence of certain maximal submodules of $M_{R}$ and
generalizes [7, Lemma 2.5].

######  Lemma 3.11.

If $P_{R}$ is a maximal submodule of $M[x^{-1}]_{R}$, we either have $\langle
P_{k}\rangle=M$ or else $\langle P_{k}\rangle$ is a maximal submodule of
$M_{R}$ for each $k\in\mathbb{N}$. Additionally, there exists $k\in\mathbb{N}$
for which the latter holds.

###### Proof.

Assume that there exists a submodule $M_{R}^{\prime}$ such that $\langle
P_{k}\rangle\subsetneq M^{\prime}\subseteq M$ and let us see that
$M^{\prime}=M$. If $m^{\prime}\in M_{R}^{\prime}$ and $m\notin\langle
P_{k}\rangle$ then $m^{\prime}x^{-k}\notin P_{R}$, and since $P_{R}$ is a
maximal submodule of $M[x^{-1}]_{R}$ we obtain
$M[x^{-1}]_{R}=P_{R}+m^{\prime}x^{-k}R_{R}$, that is, for every $f(x)\in
M[x^{-1}]_{R}$ there exist $p\in P_{R}$ and $r\in R$ such that
$f(x)=p+m^{\prime}x^{-k}r$. If $m^{\prime}x^{-k}r=m\sigma^{\prime
k}(r)x^{-k}+mp_{k,r}$ where $p_{k,r}:=\sum_{i=0}^{k-1}f_{k}^{i}(r)x^{-i}$,
consider the element $f^{\prime}(x)=mx^{-k}+mp_{k,r}$. So
$f^{\prime}(x)=p+m^{\prime}x^{-k}r$ for some $p\in P_{R}$ which implies that
$p=(m-m^{\prime}\sigma^{\prime k}(r))x^{-k}\in P_{R}$, whence
$m-m^{\prime}\sigma^{\prime k}(r)\in\langle P_{k}\rangle\subseteq M^{\prime}$
and thus $m\in M_{R}^{\prime}$. Therefore $M\subseteq M^{\prime}$ and so
$M^{\prime}=M$ proving that $\langle P_{k}\rangle$ is a maximal submodule of
$M_{R}$. If $\langle P_{k}\rangle=M$ for all $k\in\mathbb{N}$, then
$P=M[x^{-1}]$ which is a contradiction. In this way, there is $k\in\mathbb{N}$
such that $\langle P_{k}\rangle$ is a maximal submodule of $M_{R}$. ∎

Annin [7] considered the Bass modules in his study of the attached prime
ideals of $M[x^{-1}]_{S}$ [7, p. 544]. We recall that $M_{R}$ is called a Bass
module if every proper submodule $N_{R}$ is contained in a maximal submodule
of $M_{R}$ [19, p. 205]. Theorem 3.12 characterizes the attached primes of
$M[x^{-1}]_{A}$ and extends [7, Theorem 3.2].

######  Theorem 3.12.

If $M[x^{-1}]_{R}$ is a completely $(\sigma,\delta)$-compatible Bass module
then

$\displaystyle{\rm Att}(M[x^{-1}]_{A})=\left\\{PA\ |\ P\in{\rm
Att}(M_{R})\right\\}.$

###### Proof.

In view of Theorem 3.8, we only need to prove

$\displaystyle{\rm Att}(M[x^{-1}]_{A})\subseteq\left\\{PA\ |\ P\in{\rm
Att}(M_{R})\right\\}.$

Let $I\in{\rm Att}(M[x^{-1}]_{A})$ and $Q_{A}$ be a submodule of
$M[x^{-1}]_{A}$ such that $(M[x^{-1}]/Q)_{A}$ is a coprime module with $I={\rm
ann}_{A}(M[x^{-1}]/Q)$. It is clear that $I\cap R$ is equal to ${\rm
ann}_{R}(M[x^{-1}]/Q)$. By Lemma 3.10, $(M[x^{-1}]/Q)_{R}$ is a coprime module
and since $M[x^{-1}]_{R}$ is a Bass module, $(M[x^{-1}]/Q)_{R}$ contains a
maximal submodule such that $P/Q\subseteq M[x^{-1}]/Q$. By the coprimality of
$(M[x^{-1}]/Q)_{R}$, $(M[x^{-1}]/P)_{R}$ is coprime and $I\cap R={\rm
ann}_{R}(M[x^{-1}]/P)$. Let us prove that $I\cap R\in{\rm Att}(M_{R})$ and
$I=(I\cap R)A$.

If $P_{R}$ is a maximal submodule of $M[x^{-1}]_{R}$, then there exists
$k\in\mathbb{N}$ such that $\langle P_{k}\rangle$ is a maximal submodule of
$M_{R}$ by Lemma 3.11, and we can set the smallest $k$ that satisfies this
hypothesis. If $\langle P_{k}\rangle$ is maximal then there exists $m_{k}\in
M$ such that $m_{k}\notin\langle P_{k}\rangle$, and so $mx^{-k}\notin P_{R}$
whence $mx^{-k}+P$ is a cyclic generator of the simple module
$(M[x^{-1}]/P)_{R}$. Let $\varphi$ be the map of $(M/\langle
P_{k}\rangle)_{R}$ over $(M[x^{-1}]/P)_{R}$ given by $\varphi(m_{k}+\langle
P_{k}\rangle):=m_{k}x^{-k}+P$. By the complete $(\sigma,\delta)$-compatibility
of $M_{R}$ and Propositions 3.3 and 3.5, if $m_{k}r\in\langle P_{k}\rangle$
for some $r\in R$ then $m_{k}\sigma^{\prime k}(r)\in\langle P_{k}\rangle$
whence $m_{k}\sigma^{\prime k}(r)x^{-k}\in P_{R}$. By minimality of $k$,
$m_{k}f_{k}^{i}(r)x^{-i}\in P_{R}$ for all $0\leq i\leq k-1$, which implies
that $m_{k}x^{-k}r\in P_{R}$ and hence $\varphi$ is well defined.

Let us see that $\varphi$ is surjective. If $m_{k}x^{-k}+P$ generates the
module $(M[x^{-1}]/P)_{R}$ and $\varphi(m_{k}+\langle
P_{k}\rangle):=m_{k}x^{-k}+P$ then $\varphi$ is surjective. Let $\psi$ be the
homomorphism of $M_{R}$ over $(M/\langle P_{k}\rangle)_{R}$ defined by
$\psi(m):=m+\langle P_{k}\rangle$ for every $m\in M_{R}$. If
$\varphi\circ\psi$ is a surjective homomorphism of $M_{R}$ over
$(M[x^{-1}]/P)_{R}$ and $I\cap{R}\in{\rm Att}((M[x^{-1}]/P)_{R})$, then $I\cap
R\in{\rm Att}(M_{R})$.

We need to prove that $I=(I\cap R)A$. Since $I$ is an ideal of $A$ we have
$(I\cap R)A\subseteq I$. For the other inclusion, take an element
$f(x)=r_{0}+\cdots+r_{j}x^{j}\in I$ and let us see by induction that $r_{i}\in
I\cap R$ for all $1\leq i\leq j$. Notice that
$(m_{k}x^{-k}+P)f(x)=m_{k}x^{-k}r_{0}+\text{lower terms}\in Q\subseteq P$, and
since every monomial of the “lower terms” belongs to the submodule $P_{R}$ by
minimality of $k$, we have $m_{k}x^{-k}r_{0}\in P_{R}$ which implies that
$r_{0}\in I\cap R$. Assume that $r_{0},\ldots,r_{i}\in I\cap R$ for some
$i\leq j$, and let us prove that $r_{i+1}\in I\cap R$. If
$r_{0},\ldots,r_{i}\in I\cap R$ then $r_{0}+\cdots+r_{i}x^{i}\in I$, whence
$r_{i+1}x^{i+1}+\cdots+r_{j}x^{j}\in I$.

$(m_{k}x^{-k-i-1})(r_{i+1}x^{i+1}+\cdots+r_{j}x^{j})=m_{k}\sigma^{\prime
k+i+1}(r_{i+1})x^{-k}+\text{lower terms}\in Q\subseteq P$.

By minimality of $k$, every monomial of the “lower terms” belongs to the
submodule $P_{R}$ and thus $m_{k}\sigma^{\prime k+i+1}(r_{i+1})x^{-k}\in
P_{R}$, and by relation $x^{-1}r:=\sigma^{\prime}(r)x^{-1}+\delta^{\prime}(r)$
it follows that

$m_{k}\sigma^{\prime k+i+1}(r_{i+1})x^{-k}=m_{k}x^{-k}\sigma^{\prime
i+1}(r_{i+1})+\text{lower terms}\in Q\subseteq P$,

where every monomial of the “lower terms” belongs to $P_{R}$ by minimality of
$k$. So, if $M[x^{-1}]_{R}$ is a completely $(\sigma,\delta)$-compatible
module and $m_{k}x^{-k}\sigma^{\prime i+1}(r_{i+1})\in P_{R}$, then
$m_{k}x^{-k}r_{i+1}\in P_{R}$ and thus $r_{i+1}\in I\cap R$ whence
$f(x)=r_{0}+\cdots+r_{j}x^{j}$ belongs to $(I\cap R)A$. Therefore $I=(I\cap
R)A$.

∎

######  Corollary 3.13 ([7, Theorem 3.2]).

If $M[x^{-1}]_{R}$ is a completely $\sigma$-compatible Bass module then

$\displaystyle{\rm Att}(M[x^{-1}]_{S})=\left\\{P[x]\ |\ P\in{\rm
Att}(M_{R})\right\\}.$

##  4\. Examples

The importance of our results is appreciated when we extend their application
to algebraic structures that are more general than those considered by Annin
[7], that is, noncommutative rings which cannot be expressed as skew
polynomial rings of endomorphism type. In this section, we consider families
of algebras that have been studied in the literature to exemplify the results
obtained in this paper.

######  Example 4.1.

Let $A$ be the Jordan plane $\mathcal{J}(\Bbbk)$ or the $q$-skew Jordan plane
$\mathcal{J}_{q}(\Bbbk)$ with $q\in\Bbbk^{*}$ and $q\neq 1$. Consider the
right module $M[y^{-1}]_{A}$ under the action defined by (3.1) and (3.2). If
$M_{\Bbbk[x]}$ is a right module such that $M[y^{-1}]_{\Bbbk[x]}$ is a
completely $(\sigma,\delta)$-compatible module, then the characterization of
the attached prime ideals of $M[y^{-1}]_{A}$ is obtained from Theorems 3.8 and
3.12.

######  Example 4.2.

Let $A$ be the $q$-meromorphic Weyl algebra $MW_{q}$ with $yx=qxy+x^{2}$, for
$0<q<1$. If $M[x^{-1}]_{\mathbb{C}[y]}$ is completely
$(\sigma,\delta)$-compatible where $\sigma(y):=q^{-1}y$ and
$\delta(y):=-q^{-1}$, then the characterization of the attached primes of
$M[x^{-1}]_{A}$ follows from Theorems 3.8 and 3.12. Thinking about the change
of variable presented by Lopes (Example 2.2 (4)), $MW_{q}$ can be interpreted
as the quantum plane $\mathbb{C}_{q}[x,y]$ with $yx=qxy$. In this way, if
$M[x^{-1}]_{\mathbb{C}[y]}$ is a completely $\sigma$-compatible module with
$\sigma(y):=q^{-1}y$, then the description of the attached prime ideals over
$M[x^{-1}]_{A}$ follows from Theorems 3.8 and 3.12 or Corollary 3.13.

######  Example 4.3.

If $A$ is the algebra of skew Ore polynomials of higher order $Q(0,b,c)$
subject to the relation $yx=bxy+cy^{2}$ where $b,c\in\Bbbk^{*}$ and
$M_{\Bbbk[x]}$ is a right module which satisfies that $M[y^{-1}]_{\Bbbk[x]}$
is completely $(\sigma,\delta)$-compatible, then Theorems 3.8 and 3.12
described the attached prime ideals of $M[y^{-1}]_{A}$. In a similar way, we
get the characterization of these ideals over the right module $M[x^{-1}]_{A}$
when $A$ is the algebra $Q(a,b,0)$.

######  Example 4.4.

With the aim of constructing new Artin-Schelter regular algebras, Zhang and
Zhang [45] defined the double Ore extensions (or double extensions, for short)
over a $\Bbbk$-algebra $R$ and presented $26$ new families of Artin-Schelter
regular algebras of global dimension four. It is possible to find some
similarities between the definition of double extensions and two-step iterated
skew polynomial rings. Nevertheless, there exist no inclusions between the
classes of all double extensions and of all length two iterated skew
polynomial rings (c.f. [11]). Several researchers have investigated different
relations of double extensions with Poisson, Hopf, Koszul and Calabi-Yau
algebra (see [43] and reference therein). We start by recalling the definition
of a double extension in the sense of Zhang and Zhang, and since some typos
occurred in their papers [45, p. 2674] and [46, p. 379] concerning the
relations that the data of a double extension must satisfy, we follow the
corrections presented by Carvalho et al. [11].

######  Definition 4.5 ([45, Definition 1.3]; [11, Definition 1.1]).

If $B$ is a $\Bbbk$-algebra and $R$ is a subalgebra of $B$, then

  * (a)

$B$ is called a right double extension of $R$ if the following conditions
hold:

    * (i)

$B$ is generated by $R$ and two new variables $y_{1}$ and $y_{2}$.

    * (ii)

$y_{1}$ and $y_{2}$ satisfy the relation

(4.1)
$y_{2}y_{1}=p_{12}y_{1}y_{2}+p_{11}y_{1}^{2}+\tau_{1}y_{1}+\tau_{2}y_{2}+\tau_{0},$

where $p_{12},p_{11}\in\Bbbk$ and $\tau_{1},\tau_{2},\tau_{0}\in R$.

    * (iii)

$B$ is a free left $R$-module with a basis $\left\\{y_{1}^{i}y_{2}^{j}\ |\
i,j\geq 0\right\\}$.

    * (iv)

$y_{1}R+y_{2}R+R\subseteq Ry_{1}+Ry_{2}+R$.

  * (b)

A right double extension $B$ of $R$ is called a double extension if

    * (i)

$p_{12}\neq 0$.

    * (iii)

$B$ is a free right $R$-module with a basis $\left\\{y_{2}^{i}y_{i}^{j}\ |\
i,j\geq 0\right\\}$.

    * (iv)

$y_{1}R+y_{2}R+R=Ry_{1}+Ry_{2}+R$.

Condition (a)(iv) from Definition 4.5 is equivalent to the existence of two
maps

$\sigma(r):=\begin{pmatrix}\sigma_{11}(r)&\sigma_{12}(r)\\\
\sigma_{21}(r)&\sigma_{22}(r)\end{pmatrix}\ \text{and}\
\delta(r):=\begin{pmatrix}\delta_{1}(r)\\\ \delta_{2}(r)\end{pmatrix}$ for all
$r\in R$,

such that

(4.2) $\begin{pmatrix}y_{1}\\\ y_{2}\end{pmatrix}r:=\begin{pmatrix}y_{1}r\\\
y_{2}r\end{pmatrix}=\begin{pmatrix}\sigma_{11}(r)&\sigma_{12}(r)\\\
\sigma_{21}(r)&\sigma_{22}(r)\end{pmatrix}\begin{pmatrix}y_{1}\\\
y_{2}\end{pmatrix}+\begin{pmatrix}\delta_{1}(r)\\\
\delta_{2}(r)\end{pmatrix}.$

If $B$ is a right double extension of $R$ then we write
$B:=R_{P}[y_{1},y_{2};\sigma,\delta,\tau]$ where
$P:=\\{p_{12},p_{11}\\}\subseteq\Bbbk$,
$\tau:=\\{\tau_{1},\tau_{2},\tau_{0}\\}\subseteq R$ and $\sigma,\delta$ are as
above. The set $P$ is called a parameter and $\tau$ a tail. If $\delta:=0$ and
$\tau$ consists of zero elements then the double extension is denoted by
$R_{P}[y_{1},y_{2};\sigma]$ and is called a trimmed double extension [45,
Convention 1.6 (c)]. It is straightforward to see that the relation (4.1) is
given by

(4.3) $y_{2}y_{1}=p_{12}y_{1}y_{2}+p_{11}y_{1}^{2}.$

Since $p_{12},p_{11}\in\Bbbk$ the expression (4.3) can be written as
$y_{1}y_{2}=p_{12}^{-1}y_{2}y_{1}-p_{12}^{-1}p_{11}y_{1}^{2}$. It is clear
that $\sigma(y_{2})=p_{12}^{-1}y_{2}$ is an automorphism of $\Bbbk[y_{2}]$ and
$\delta(y_{2})=-p_{12}^{-1}p_{11}$ is a locally nilpotent $\sigma$-derivation
of $\Bbbk[y_{2}]$. In this way, the trimmed double extension
$R_{P}[y_{1},y_{2};\sigma]$ can be seen as
$A=\Bbbk[y_{2}](y_{1};\sigma,\delta)$. If $M[y_{2}^{-1}]_{A}$ is a right
module under the action given by (3.1) and (3.2) and $M_{\Bbbk[y_{2}]}$ is a
module such that $M[y_{1}^{-1}]_{\Bbbk[y_{2}]}$ is completely
$(\sigma,\delta)$-compatible, then Theorems 3.8 and 3.12 describe the attached
prime ideals of $M[y_{2}^{-1}]_{A}$.

##  5\. Future work

As we mentioned in the introduction, Nordstrom [38, 39] studied the associated
primes of simple torsion modules over generalized Weyl algebras which are
$\mathbb{N}$-graded rings and contain the skew polynomial ring $R[x;\sigma]$
as a subring. As a possible future work, we have in mind to investigate the
attached prime ideals of inverse polynomial modules over generalized Weyl
algebras.

Lezama and Latorre [30] introduced the semi-graded rings with the aim of
generalizing $\mathbb{N}$-graded rings (such as the skew polynomial rings of
endomorphism type and the generalized Weyl algebras), skew PBW extensions and
many other algebras of interest in noncommutative algebraic geometry and
noncommutative differential geometry. We think as future work to investigate
the theory of the attached prime ideals and secondary representations over
semi-graded rings.

## References

  * [1] Alev, J., Dumas, F. (1995). Automorphismes de certains completés du corps de Weyl quantique. Collect. Math. 46:1–9.
  * [2] Alhevaz, A., Moussavi, A. (2012). On skew Armendariz and skew quasi-Armendariz modules. Bull. Iranian Math. Soc. 38(1):55–84.
  * [3] Annin, S. (2002). Associated and Attached Primes Over Noncommutative Rings. PhD Thesis. University of California, Berkeley.
  * [4] Annin, S. (2002). Associated primes over skew polynomial ring. Commun. Algebra 30(5):2511–2528.
  * [5] Annin, S. (2004). Associated primes over Ore extension rings. J. Algebra Appl. 3(2):193–205.
  * [6] Annin, S. (2008). Attached primes over noncommutative rings. J. Pure Appl. Algebra 212(3):510–521.
  * [7] Annin, S. (2011). Attached primes under skew polynomial extensions. J. Algebra Appl. 10(3):537–547.
  * [8] Baig, M. (2009). Primary decomposition and secondary representation of modules over a commutative ring. Master Thesis. Georgia State University.
  * [9] Bavula, V. V. (1992). Generalized Weyl algebras and their representations. St. Petersburg Math. J. 4(1):71–92.
  * [10] Brewer, J., Heinzer, W. (1974). Associated primes of principal ideals. Duke Math. J. 41(1):1–7.
  * [11] Carvalho, P. A. A. B., Lopes, S. A., Matczuk, J. (2011). Double Ore Extensions Versus Iterated Ore Extensions. Commun. Algebra 39(8):2838–2848.
  * [12] Chacón, A., Reyes, A. (2024). On the schematicness of some Ore polynomials of higher order generated by homogenous quadratic relations. J. Algebra Appl. 2550207\.
  * [13] Cohn, P. M. (1961). Quadratic extensions of skew fields. Proc. London Math. Soc. (3) 3(1):531–556.
  * [14] Díaz, R., Pariguan, E. (2009). On the $q$-meromorphic Weyl algebra. São Paulo J. Math. Sci. 3(2): 283–29
  * [15] Dumas, F., Martin, F. (2023). Invariants of formal pseudodifferential operator algebras and algebraic modular forms. Rev. Un. Mat. Argentina 65(1):1–31.
  * [16] Dumas, F. (1992). Skew power series rings with general commutation formula. Theoret. Comput. Sci. 98(1):99–114.
  * [17] Dumas, F. (1991). Sous-corps de fractions rationnelles des corps gauches de series de Laurent. Topics in Invariant Theory. Vol. 1478. Berlin, Heidelberg: Springer.
  * [18] Faith, C. (2000). Associated primes in commutative polynomial rings. Commun. Algebra 28(8):3983–3986.
  * [19] Faith, C. (1995). Rings whose modules have maximal submodules. Publ. Mat. 39(1):201–214.
  * [20] Freudenburg, G. (2006). Algebraic theory of locally nilpotent derivations. Vol. 136. Berlin: Springer.
  * [21] Gallego, C., Lezama, O. (2011). Gröbner Bases for Ideals of $\sigma$-PBW Extensions. Commun. Algebra 39(1):50–75.
  * [22] Golovashkin, A. V., Maksimov, V. M. (2005). On algebras of skew polynomials generated by quadratic homogeneous relations. J. Math. Sci. (N.Y.) 129(2):3757–3771.
  * [23] Goodearl, K. R., Warfield, R. B. (2004). An Introduction to Noncommutative Noetherian Rings. Vol. 61. Cambridge University Press.
  * [24] Hashemi, E., Moussavi, A. (2005). Polinomial extensions of quasi-Baer rings. Acta Math. Hungar. 107(3):207–224.
  * [25] Higuera, S., Ramírez, M. C., Reyes, A. (2024). On the uniform dimension and the associated primes of skew PBW extensions. https://arxiv.org/abs/2404.18698.
  * [26] Higuera, S., Reyes, A. (2023). On weak annihilators and nilpotent associated primes of skew PBW extensions. Commun. Algebra 51(11):4839–4861.
  * [27] Jordan, D. (2001). The Graded Algebra Generated by Two Eulerian Derivatives. Algebr. Represent. Theory 4(3):249–275.
  * [28] Lam, T. Y., Leroy, A., Matczuk, J. (1997) Primeness, semiprimeness and prime radical of Ore extensions, Commun. Algebra 25(8):2459–2506.
  * [29] Lam, T. Y. (1998). Lectures on Modules and Rings. Graduate Texts in Mathematics. Vol. 189. Berlin: Springer-Verlag.
  * [30] Lezama, O., Latorre, E. (2017). Non-commutative algebraic geometry of semi-graded rings. Int. J. Appl. Comput. Math. 27(4):361–389.
  * [31] Lopes, S. A. (2023). Noncommutative Algebra and Representation Theory: Symmetry, Structure & Invariants. Commun. Math. 32(3):63–117.
  * [32] Macdonald, I. G. (1973). Secondary representation of modules over a commutative ring. Sympos. Math. 11:23–43.
  * [33] Maksimov, V. M. (2000). On a generalization of the ring of skew Ore polynomials. Russian Math. Surveys 55(4):817–818.
  * [34] Melkersson, L. (1998). Content and inverse polynomials on Artinian modules. Commun. Algebra 26(4):1141–1145.
  * [35] Niño, A., Ramírez, M. C., Reyes, A. (2020). Associated prime ideals over skew PBW extensions. Commun. Algebra 48(12):5038–5055.
  * [36] Niño, A., Ramírez, M. C., Reyes, A. (2024). A first approach to the Burchnall-Chaundy theory for quadratic algebras having PBW bases. https://arxiv.org/abs/2401.10023.
  * [37] Niño, A., Reyes, A. (2023). On centralizers and pseudo-multidegree functions for non-commutative rings having PBW bases. J. Algebra Appl. 2550109\.
  * [38] Nordstrom, H. E. (2005). Associated primes over Ore extensions and generalized Weyl algebras. PhD Thesis. University of Oregon.
  * [39] Nordstrom, H. E. (2012). Simple Modules Over Generalized Weyl Algebras and Their Associated Primes. Commun. Algebra 40(9):3224–3235.
  * [40] Ore, O. (1931). Linear Equations in Non-commutative Fields. Ann. of Math. (2) 32(3):463–477.
  * [41] Ore, O. (1933). Theory of Non-Commutative Polynomials. Ann. of Math. (2) 34(3):480–508.
  * [42] Ouyang, L., Birkenmeier, G. F. (2012). Weak annihilator over extension rings. Bull. Malays. Math. Sci. Soc. 35(2):345–347.
  * [43] Ramírez, M. C., Reyes, A. (2024). A view toward homomorphisms and cv-polynomials between double Ore extensions. Algebra Colloq. To appear. https://arxiv.org/abs/2401.14162.
  * [44] Smits, T. H. M. (1968). Skew polynomial rings. Indag. Math. (N.S.) 30(1):209–224.
  * [45] Zhang, J. J., Zhang, J. (2008). Double Ore extensions. J. Pure Appl. Algebra 212(12):2668–2690.
  * [46] Zhang, J. J., Zhang J. (2009). Double extension regular algebras of type (14641). J. Algebra 322(2):373–409.


# Hessian-aware Quantized Node Embeddings for Recommendation

Huiyuan Chen<EMAIL_ADDRESS>Visa ResearchPalo AltoCAUSA ,  Kaixiong Zhou  ,
Kwei-Herng Lai<EMAIL_ADDRESS>Rice UniversityHoustonTXUSA ,  Chin-
Chia Michael Yeh<EMAIL_ADDRESS>Visa ResearchPalo AltoCAUSA ,  Yan Zheng
<EMAIL_ADDRESS>Visa ResearchPalo AltoCAUSA ,  Xia Hu<EMAIL_ADDRESS>Rice
UniversityHoustonTXUSA and  Hao Yang<EMAIL_ADDRESS>Visa ResearchPalo
AltoCAUSA

(2023)

###### Abstract.

Graph Neural Networks (GNNs) have achieved state-of-the-art performance in
recommender systems. Nevertheless, the process of searching and ranking from a
large item corpus usually requires high latency, which limits the widespread
deployment of GNNs in industry-scale applications. To address this issue, many
methods compress user/item representations into the binary embedding space to
reduce space requirements and accelerate inference. Also, they use the
Straight-through Estimator (STE) to prevent vanishing gradients during back-
propagation. However, the STE often causes the gradient mismatch problem,
leading to sub-optimal results.

In this work, we present the Hessian-aware Quantized GNN (HQ-GNN) as an
effective solution for discrete representations of users/items that enable
fast retrieval. HQ-GNN is composed of two components: a GNN encoder for
learning continuous node embeddings and a quantized module for compressing
full-precision embeddings into low-bit ones. Consequently, HQ-GNN benefits
from both lower memory requirements and faster inference speeds compared to
vanilla GNNs. To address the gradient mismatch problem in STE, we further
consider the quantized errors and its second-order derivatives for better
stability. The experimental results on several large-scale datasets show that
HQ-GNN achieves a good balance between latency and performance.

Collaborative Filtering, Graph Neural Networks, Low-bit Quantization,
Generalized Straight-Through Estimator

††journalyear: 2023††copyright: acmlicensed††conference: Seventeenth ACM
Conference on Recommender Systems; September 18–22, 2023; Singapore,
Singapore††booktitle: Seventeenth ACM Conference on Recommender Systems
(RecSys ’23), September 18–22, 2023, Singapore, Singapore††price: 15.00††doi:
10.1145/3604915.3608826††isbn: 979-8-4007-0241-9/23/09††ccs: Information
systems Recommender systems††ccs: Computing methodologies Neural networks

##  1\. Introduction

Recommender systems play an important role for e-commerce, such as display
advertising and ranking products (Huang et al., 2020; Chen et al., 2021).
Among different recommender models, Graph Neural Networks (GNNs) have achieved
cutting-edge performance on top-$k$ recommendations (Ying et al., 2018; Wang
et al., 2019; He et al., 2020; Huang et al., 2021). For instance, Pinterest
deploys a GNN model to train on a graph with $3$ billion nodes and 18 billion
edges, which has delivered state-of-the-art performance (Ying et al., 2018).
Despite the superior ability of GNNs, node representations are often stored in
continuous embedding space (e.g., 32-bit floating point (FP32)). This often
requires huge memory consumption (Lian et al., 2020). For example, the FP32
embeddings of 10 million items with a dimensional size of 256 will take up
over 9.5 GB of storage space, which is hard to be deployed into devices with
limited memory, especially under the federated learning settings (Reisizadeh
et al., 2020; Yuan et al., 2023). Therefore, searching and ranking from a
large item corpus to generate top-$k$ recommendations become intractable at
scale due to their high latency (Shi et al., 2020; Tan et al., 2020; Chen et
al., 2022b; Wang et al., 2023; Xu et al., 2023).

Low-bit quantization (Gong et al., 2019; Jacob et al., 2018; Lee et al., 2021;
Kim et al., 2021; Cao et al., 2017) is a promising method to save the memory
footprint and accelerate model inference for large-scale systems. By replacing
FP32 values with lower precision values, e.g., 8-bit integer (INT8),
quantization can shrink down the size of embeddings without modifying the
original network architectures. Also, quantized operators are widely supported
by modern hardwares, which allows to deploy very large networks to resource-
limited devices (Jacob et al., 2018; Chen et al., 2022b). For example, NVIDIA
Turing GPU architecture111https://www.nvidia.com/en-us/geforce/turing/
supports the INT8 arithmetic operations.

Recently, several studies have adopted quantization in large-scale recommender
systems (Cao et al., 2017; Tan et al., 2020; Wu et al., 2021; Kang and
McAuley, 2019). However, existing methods suffer from two drawbacks: 1) Most
of them employ binary hash techniques to compress user/item embeddings into
1-bit quantized representations. Nevertheless, recent studies show that ultra
low-bit quantizations (e.g., 1 or 2 bits) can be much more challenging due to
their significant degradation in the accuracy (Zhou et al., 2016; Gong et al.,
2019); 2) They often use the Straight-through Estimator (STE) (Bengio et al.,
2013) to avoid zero gradients during the back-propagation. Specifically, the
non-differentiable quantized function is replaced with a surrogate: the
identity function (Tan et al., 2020) or the scaled tanh function (Cao et al.,
2017; Kang and McAuley, 2019). However, the use of different forward and
backward functions results in a gradient mismatch problem, i.e., the modified
gradient is certainly not the gradient of loss function, which makes the
network training unstable (Yin et al., 2019; Chen et al., 2022a).

In this work, we propose the Hessian-aware Quantized GNN (HQ-GNN) for
effective discrete representations of users and items for fast retrieval.
Specifically, HQ-GNN consists of two components: a GNN encoder for learning
continuous user/item embeddings, and a quantized module for compressing the
full-precision embeddings into low-bit ones. Instead of 1-bit, HQ-GNN allows
arbitrary bit quantization for better trade-offs between latency and
performance. To address the gradient mismatch problem, we tailor the STE by
further considering the quantized errors and second-order derivatives (e.g.
Hessian) for better stability and accuracy. As such, HQ-GNN can benefit from
both lower memory footprint and faster inference speed comparing to vanilla
GNN. Experimental results on several large-scale datasets show the superiority
of our HQ-GNN.

##  2\. Related Work

#### GNN-based Recommenders

GNNs have received a lot of attention in graph domains. GNNs learn how to
aggregate messages from local neighbors using neural networks, which have been
successfully applied to user-item bipartite graphs (Ying et al., 2018; Wang et
al., 2019; He et al., 2020; Chen et al., 2022c, d; Wang et al., 2022). Some
representative models include PinSage (Ying et al., 2018), NGCF (Wang et al.,
2019), LightGCN (He et al., 2020), etc. Although GNNs have great ability of
capturing high-order collaborative signals between users and items, their node
embeddings are stored in continuous space (e.g., FP32), which is the major
bottleneck for searching and ranking (e.g., high computational cost of
similarity calculation between continuous embeddings). It is thus essential to
improve the efficiency of generating top-$k$ recommendations at scale (Shi et
al., 2020; Tan et al., 2020).

#### Network Quantizations

Quantization is a hardware-friendly approach by approximating real values with
low-bit ones (Gong et al., 2019; Jacob et al., 2018; Lee et al., 2021; Kim et
al., 2021; Cao et al., 2017; Jing et al., 2021; Jiang et al., 2021; Yeh et
al., 2022). Meanwhile, network inference can be performed using cheaper fixed-
point multiple-accumulation operations. As a result, quantization can reduce
the storage overhead and inference latency of networks (Zhou et al., 2016;
Gong et al., 2019; Zhu et al., 2020; Lee et al., 2021; Lian et al., 2020). In
recommender systems, HashNet (Cao et al., 2017) proposes to binarize the
embeddings by continuation method for multimedia retrieval. Similarly, CIGAR
(Kang and McAuley, 2019) learns binary codes to build a hash table for
retrieving top-$k$ item candidates. Recently, HashGNN (Tan et al., 2020)
learns hash functions and graph representations in an end-to-end fashion. Our
HQ-GNN builds on HashGNN. Specifically, we extend 1-bit quantization of
HashGNN to arbitrary-bit one, and address the gradient mismatch issue of STE,
resulting in better performance.

##  3\. methodology

###  3.1. Task Description

Generally, the input of recommender systems includes a set of users
$\mathcal{U}=\\{u\\}$, items $\mathcal{I}=\\{i\\}$, and users’ implicit
feedback $\mathcal{O}^{+}=\left\\{(u,i)\mid
u\in\mathcal{U},i\in\mathcal{I},y_{ui}=1\right\\}$, where $y_{ui}=1$ indicates
that user $u$ has adopted item $i$ before, $y_{ui}=0$ otherwise. One can
construct a corresponding bipartite graph
$\mathcal{G}=(\mathcal{V}=\mathcal{U}\cup\mathcal{I},\mathcal{E}=\mathcal{O}^{+})$.
The goal is to estimate the user preference towards unobserved items.

We next introduce our HQ-GNN that consists of two parts: a GNN encoder and a
quantized module.

###  3.2. GNN-based Recommenders

Most GNNs fit under the message-passing schema (Wang et al., 2019; He et al.,
2020), where the representation of each node is updated by collecting messages
from its neighbors via an aggregation operation $\text{Agg}(\cdot)$ followed
by an $\text{Update}(\cdot)$ operation as:

(1) $\displaystyle\mathbf{e}_{u}^{(l)}=$
$\displaystyle\text{Update}\left(\mathbf{e}_{u}^{(l-1)},\text{Agg
}(\\{\mathbf{e}_{i}^{(l-1)}\mid i\in\mathcal{N}_{u}\\})\right),$
$\displaystyle\mathbf{e}_{i}^{(l)}=$
$\displaystyle\text{Update}\left(\mathbf{e}_{i}^{(l-1)},\text{Agg
}(\\{\mathbf{e}_{u}^{(l-1)}\mid u\in\mathcal{N}_{i}\\})\right),$

where $\\{\mathbf{e}^{(l)}_{u},\mathbf{e}^{(l)}_{i}\\}\in\mathbb{R}^{d}$
denote the embeddings of user and item in the $l$-th layer; $\mathcal{N}_{u}$
and $\mathcal{N}_{i}$ denote neighbors of user $u$ and item $i$, respectively.
By propagating $L$ layer, a pooling operator is used to obtain the final
representations:

(2)
$\mathbf{e}_{u}=\text{Pool}(\mathbf{e}_{u}^{(0)},\ldots,\mathbf{e}_{u}^{(L)}),\quad\mathbf{e}_{i}=\text{Pool}(\mathbf{e}_{i}^{(0)},\ldots,\mathbf{e}_{i}^{(L)}),$

where the final representations $\mathbf{e}_{u}\in\mathbb{R}^{d}$ and
$\mathbf{e}_{i}\in\mathbb{R}^{d}$ can be used for downstream tasks. However,
the full-precision embeddings, e.g., FP$32$, usually require high memory cost
and power consumption to generate top-$k$ recommendations for the billion-
scale graphs.

###  3.3. Low-bit Quantization

Quantization is a hardware-friendly technique to reduce memory footprint and
energy consumption (Han et al., 2016; Sun et al., 2020; Zhu et al., 2020). For
a uniform $b$-bit quantization, one can clip and normalize a floating-point
number $x$ into a quantization interval, parameterized by an upper $u$ and a
lower $l$ bounds, as:

(3) $x_{n}=\frac{\text{clip}(x,l,u)-l}{\Delta},$

where $x_{n}$ is the normalized output,
$\text{clip}(x,l,u)=\min(\max(x,l),u)$, $\Delta=\frac{u-l}{2^{b}-1}$ is the
interval length, and $b$ denotes the number of quantization levels, e.g.,
$b=8$ for $8$-bit quantization. During training, the clipping interval $(l,u)$
is often unknown beforehand, two strategies are commonly used to determine the
upper/lower thresholds: exponential moving averages (Jacob et al., 2018) and
treating the thresholds as learnable parameters (Choi et al., 2018). The
normalized output $x_{n}$ can be then converted to a discrete value $x_{b}$
using a round function with post-scaling as (Zhou et al., 2016; Gong et al.,
2019; Zhu et al., 2020):

(4) $x_{b}=x_{q}\cdot\Delta,\quad x_{q}=\text{round}(x_{n}),$

where $\text{round}(\cdot)$ maps a full-precision value to its nearest
integer. The quantized tensor $x_{b}$ can be then used for efficient
computation by emergent accelerators (e.g., NVIDIA TensorRT) that are able to
handle $\Delta$ efficiently.

By combining Eq. (3) and Eq. (4), we can defined a quantization function
$Q_{b}(\cdot)$ as: $x_{b}=Q_{b}(x)$. If the input is a vector/matrix,
$Q_{b}(\cdot)$ would apply to each element of the vector/matrix. To this end,
we can quantize the GNN embeddings $\mathbf{e}_{u}$ and $\mathbf{e}_{i}$ in
Eq. (2) into:

(5)
$\mathbf{q}_{u}=Q_{b}(\mathbf{e}_{u}),\quad\mathbf{q}_{i}=Q_{b}(\mathbf{e}_{i}),$

where $\\{\mathbf{q}_{u},\mathbf{q}_{i}\\}\in\mathbb{R}^{d}$ are the $b$-bit
representations of user $u$ and item $i$, respectively. Our model follows the
mixed-precision quantization policy (Micikevicius et al., 2018), where we only
compress the activations of GNNs for faster inference, and leave the weights
of GNNs at full precision. Since GNNs often contain less than three layers and
have limited weights, the mixed-precision scheme could achieve good trade-offs
between performance and memory size (Dong et al., 2019). The mixed-precision
quantization has also become more and more common in deep learning
frameworks222https://www.tensorflow.org/guide/mixed_precision.

However, the non-differentiable quantized processes are undesirable for the
standard back-propagation, i.e., the quantization function is intrinsically a
discontinuous step function and nearly has zero gradients, which significantly
affects the training of HQ-GNN. We next present a Generalized Straight-Through
Estimator to address this problem.

###  3.4. Generalized Straight-Through Estimator

The main challenge of training our HQ-GNN arises from the discretized round
function in Eq. (4), where its derivative is either infinite or zero at almost
everywhere. One popular family of estimators are the so-called Straight-
Through Estimators (STE) (Bengio et al., 2013; Yin et al., 2019). In STE, the
forward computation of $\text{round}(\cdot)$ is unchanged, but back-
propagation is computed through a surrogate (Tan et al., 2020; Cao et al.,
2017; Zhou et al., 2016): replacing $\text{round}(\cdot)$ with an identity
function, i.e., $\mathcal{G}_{\mathbf{x_{n}}}=\mathcal{G}_{\mathbf{x_{q}}}$
where $\mathcal{G}$ denotes the gradient operator. However, STE runs the risk
of convergence to poor minima and unstable training (Yin et al., 2019). For
example, both values of $0.51$ and $1.49$ round to same integer $1$ with
different quantized errors. Moreover, STE forces to update both values equally
with the same gradient at integer $1$, which is likely to be biased with
cumulative quantized errors. Moreover, a small decrement (e.g., $-0.2$) for
value $0.51$ can largely change the quantized integer from $1$ to $0$, while a
same decrement to $1.49$ cannot.

To mitigate the impact of quantized errors, we generalize the STE as (Lee et
al., 2021):

(6)
$\mathcal{G}_{\mathbf{x_{n}}}=\mathcal{G}_{\mathbf{x_{q}}}\odot\left(1+\delta\cdot\text{sign}(\mathcal{G}_{\mathbf{x_{q}}})\odot(\mathbf{x_{n}}-\mathbf{x_{q}})\right),$

where $\odot$ denotes element-wise product; $\text{sign}(\cdot)$ is a sign
function such that $\text{sign}(x)=+1$ if $x\geq 0$, $-1$ otherwise; $\delta$
is the scaling factor. Eq. (6) is able to scale up/down the gradient of
$\mathcal{G}_{\mathbf{x_{q}}}$ when the $\mathbf{x_{n}}$ requires a
larger/smaller magnitude for an update. Moreover, Eq. (6) is equivalent to
vanilla STE when setting $\delta=0$. It is thus crucial to determine the
scaling factor $\delta$ during training.

Inspired by Hessian-aware quantized networks (Dong et al., 2019, 2020), we use
second-order information to guide the selection of $\delta$. Let
$\mathbf{\epsilon}=\mathbf{x_{n}}-\mathbf{x_{q}}$ denote the quantized error
for round function, where each element of $\mathbf{\epsilon}$ is well bound by
a small number, i.e., $|\epsilon_{i}|\leq\frac{0.5}{2^{b}-1}$, with element-
wise Taylor expansion, we have:

$\displaystyle\mathcal{G}_{\mathbf{x_{n}}}=$
$\displaystyle\mathcal{G}_{\mathbf{x_{q}}}+\frac{\mathcal{G}_{\mathbf{x_{n}}}-\mathcal{G}_{\mathbf{x_{q}}}}{\mathbf{x_{n}}-\mathbf{x_{q}}}\odot(\mathbf{x_{n}}-\mathbf{x_{q}})$
$\displaystyle=$
$\displaystyle\mathcal{G}_{\mathbf{x_{q}}}+\frac{\mathcal{G}_{\mathbf{x_{q}}+\mathbf{\epsilon}}-\mathcal{G}_{\mathbf{x_{q}}}}{\mathbf{\epsilon}}\odot(\mathbf{x_{n}}-\mathbf{x_{q}})$
$\displaystyle\approx$
$\displaystyle\mathcal{G}_{\mathbf{x_{q}}}+\mathcal{G}^{\prime}_{\mathbf{x_{q}}}\odot(\mathbf{x_{n}}-\mathbf{x_{q}}),$

where $\frac{[\cdot]}{[\cdot]}$ is the element-wise division,
$\mathcal{G}^{\prime}_{\mathbf{x_{q}}}=\frac{\partial\mathcal{G}_{\mathbf{x_{q}}}}{\partial\mathbf{x_{q}}}$
denotes the second-order derivative of a task loss with respect to
$\mathbf{x_{q}}$. The above equation can be represented as:

(7)
$\mathcal{G}_{\mathbf{x_{n}}}\approx\mathcal{G}_{\mathbf{x_{q}}}\odot\left(1+\frac{\mathcal{G}^{\prime}_{\mathbf{x_{q}}}}{|\mathcal{G}_{\mathbf{x_{q}}}|}\odot\text{sign}(\mathcal{G}_{\mathbf{x_{q}}})\odot(\mathbf{x_{n}}-\mathbf{x_{q}})\right),$

where $|\cdot|$ denotes the absolute value. Comparing Eq. (6) and Eq. (7)
suggests that we can connect $\delta$ with
$\frac{\mathcal{G}^{\prime}_{\mathbf{x_{q}}}}{|\mathcal{G}_{\mathbf{x_{q}}}|}$,
but explicitly forming the Hessian matrix $\mathbf{H}$ (containing all
$\mathcal{G}^{\prime}_{\mathbf{x_{q}}}$) is computationally infeasible in
practice. Instead, recent quantized networks approximate the second-order
information by the average Hessian Trace (Dong et al., 2020) or top Hessian
eigenvalues (Dong et al., 2019). In this work, we summarize the average trace
of Hessian and
$\frac{\mathcal{G}^{\prime}_{\mathbf{x_{q}}}}{|\mathcal{G}_{\mathbf{x_{q}}}|}$
as scaling factor:

(8) $\delta=\frac{\text{Tr}(\mathbf{H})/N}{G},$

where $N$ is the number of diagonal elements in $\mathbf{H}$ and $G$ is an
average over the absolute values of gradients, i.e.,
$\mathbb{E}[|\mathcal{G}_{\mathbf{x_{q}}}|]$.

Input: A GNN $f_{gnn}$, bipartite graph $\mathbf{A}$, bit-width $b$,
regularizer $\alpha$.

1

Output: Model parameters $\mathbf{\Theta}$ of $f_{gnn}$;

2

3Initialize $\mathbf{\Theta}$ ;

4

5for  _each mini-batch_ do

/* Forward pass */

6 Compute node embeddings $\mathbf{e}_{u}$ and $\mathbf{e}_{i}$ by Eq. (2);

7 Normalize outputs
$\hat{\mathbf{e}}_{u}=\frac{\text{clip}(\mathbf{e}_{u},l,u)-l}{\Delta}$ (same
for $\hat{\mathbf{e}}_{i}$);

8 Quantize values $\bar{\mathbf{e}}_{u}=\text{round}(\hat{\mathbf{e}}_{u})$
(same for $\bar{\mathbf{e}}_{i}$);

9 Post-scaling quantized values
$\mathbf{q}_{u}=\bar{\mathbf{e}}_{u}\odot\Delta$ (same for $\mathbf{q}_{i}$);

10 Compute the BPR loss by Eq. (9);

/* Backward propagation */

11 Compute the gradients $\mathcal{G}_{\bar{\mathbf{e}}_{u}}$ and
$\mathcal{G}_{\bar{\mathbf{e}}_{i}}$ via standard SGD;

12 Adjust the gradients $\mathcal{G}_{\hat{\mathbf{e}}_{u}}$ and
$\mathcal{G}_{\hat{\mathbf{e}}_{i}}$ by Eq. (6):

13
$\mathcal{G}_{\hat{\mathbf{e}}_{u}}=\mathcal{G}_{\bar{\mathbf{e}}_{u}}\odot\left(1+\delta\cdot\text{sign}(\mathcal{G}_{\bar{\mathbf{e}}_{u}})\odot({\hat{\mathbf{e}}_{u}}-{\bar{\mathbf{e}}_{u}})\right)$,

14
$\mathcal{G}_{\hat{\mathbf{e}}_{i}}=\mathcal{G}_{\bar{\mathbf{e}}_{i}}\odot\left(1+\delta\cdot\text{sign}(\mathcal{G}_{\bar{\mathbf{e}}_{i}})\odot({\hat{\mathbf{e}}_{i}}-{\bar{\mathbf{e}}_{i}})\right)$.

15 Compute the trace of Hessian by Hutchinson method (Avron and Toledo, 2011);

16 Update GNN parameters $\mathbf{\Theta}$ and the scaling factor $\delta$ by
Eq. (8);

17 end for

return _ $\mathbf{\Theta}$_

Algorithm 1 HQ-GNN

We compute the trace of Hessian via Hutchinson’s method (Avron and Toledo,
2011) Given a random vector $\mathbf{v}$, whose elements are i.i.d. sampled
from a Rademacher distribution such that
$\mathbb{E}[\mathbf{v}\mathbf{v}^{\top}]=\mathbf{I}$. Then, we have:

$\displaystyle\text{Tr}(\mathbf{H})$
$\displaystyle=\text{Tr}(\mathbf{H}\mathbb{E}[\mathbf{v}\mathbf{v}^{\top}])=\mathbb{E}[\text{Tr}(\mathbf{H}\mathbf{v}\mathbf{v}^{\top})]$
$\displaystyle=\mathbb{E}[\mathbf{v}^{\top}\mathbf{H}\mathbf{v}]\approx\frac{1}{m}\sum_{i=1}^{m}({\mathbf{v}^{(i)}}^{\top}\mathbf{H}\mathbf{v}^{(i)}),$

where $\mathbf{I}$ is the identity matrix. The trace of $\mathbf{H}$ can be
estimated by $\mathbb{E}[\mathbf{v}^{\top}\mathbf{H}\mathbf{v}]$, where the
expectation can be obtained by drawing $m$ random vectors. Note that we can
first compute $\mathbf{H}\mathbf{v}$, then
$\mathbf{v}^{\top}\mathbf{H}\mathbf{v}$ is a simple inner product between
$\mathbf{v}$ and $\mathbf{H}\mathbf{v}$. Also, we can obtain
$\mathbf{H}\mathbf{v}$ efficiently without computing an exact Hessian matrix
as follows:

$\frac{\partial(\mathcal{G}^{\top}_{\mathbf{x_{q}}}\mathbf{v})}{\partial\mathbf{x_{q}}}=\frac{\partial\mathcal{G}^{\top}_{\mathbf{x_{q}}}}{\partial\mathbf{x_{q}}}\mathbf{v}+\mathcal{G}^{\top}_{\mathbf{x_{q}}}\frac{\partial\mathbf{v}}{\partial\mathbf{x_{q}}}=\frac{\partial\mathcal{G}^{\top}_{\mathbf{x_{q}}}}{\partial\mathbf{x_{q}}}\mathbf{v}=\mathbf{H}\mathbf{v},$

where the first equality is the chain rule, while the second is due to the
independence of $\mathbf{v}$ and $\mathbf{x_{q}}$. As such, the cost of
Hessian matrix-vector multiply is the same as one gradient back-propagation.

###  3.5. Model Optimization

####  3.5.1. Loss function

Based on the $b$-bit representations $\mathbf{q}_{u}$ and $\mathbf{q}_{i}$
from Eq. (5), we can adopt the inner product to estimate the user’s preference
towards the target item as:
$\hat{y}_{ui}=\langle\mathbf{q}_{u},\mathbf{q}_{i}\rangle$. Also, we use
Bayesian Personalized Ranking loss to optimize the model (Kang and McAuley,
2019):

(9)
$\mathcal{L}_{BPR}(\mathbf{\Theta})=\sum_{\begin{subarray}{c}(u,i)\in\mathcal{O}^{+},(u,j)\notin\mathcal{O}^{+}\end{subarray}}-\ln\sigma\left(\hat{y}_{ui}-\hat{y}_{uj}\right)+\alpha\|\mathbf{\Theta}\|_{F}^{2},$

where $\sigma(\cdot)$ denotes the sigmoid function, $\mathbf{\Theta}$ denotes
the model parameters of GNNs, and $\alpha$ controls the $L_{2}$ regularization
strength. Finally, we briefly summarize our HQ-GNN in Algorithm 1.

####  3.5.2. Complexity

Compared to vanilla GNN, HQ-GNN has an extra time cost to perform gradient
adjustments in Eq. (6). The computation of Hessian Trace only requires one
gradient back-propagation, which is significantly faster than training the GNN
encoder itself (Dong et al., 2020). Thus, HQ-GNN has the same training
complexity as its GNN encoder. However, during the inference, we can use
integer-only node embeddings (without post-scaling) to generate the top-$k$
candidates, which has both lower memory footprint and faster inference speed
compared to the vanilla GNN.

Table 1. Dataset statistics. Dataset | Gowalla | Yelp2018 | Amazon-Book | Alibaba
---|---|---|---|---
—User— | 29,858 | 31,668 | 52,643 | 106,042
—Item— | 40,981 | 38,048 | 91,599 | 53,591
—Interaction— | 1,027,370 | 1,561,406 | 2,984,108 | 907,407

##  4\. Experiments

###  4.1. Experimental Settings

Table 2.  Performance comparison (bold and underline represent the best full-
precision and 1-bit quantized models).

| Gowalla | Yelp-2018 | Amazon-Book | Alibaba
---|---|---|---|---
Methods | Recall@50 | NDCG@50 | Recall@50 | NDCG@50 | Recall@50 | NDCG@50 | Recall@50 | NDCG@50
NGCF | 0.159 | 0.130 | 0.114 | 0.054 | 0.092 | 0.065 | 0.071 | 0.033
+HashNet | 0.104 | 0.082 | 0.071 | 0.030 | 0.057 | 0.038 | 0.047 | 0.021
+HashGNN | 0.122 | 0.098 | 0.091 | 0.042 | 0.073 | 0.043 | 0.054 | 0.023
+HQ-GNN | 0.145 | 0.112 | 0.101 | 0.048 | 0.081 | 0.054 | 0.065 | 0.029
LightGCN | 0.163 | 0.134 | 0.118 | 0.059 | 0.098 | 0.072 | 0.076 | 0.036
+HashNet | 0.113 | 0.088 | 0.074 | 0.036 | 0.064 | 0.041 | 0.052 | 0.024
+HashGNN | 0.128 | 0.112 | 0.094 | 0.047 | 0.075 | 0.053 | 0.062 | 0.029
+HQ-GNN | 0.152 | 0.122 | 0.108 | 0.051 | 0.089 | 0.062 | 0.070 | 0.032

####  4.1.1. Datasets.

We evaluate our method on four public datasets (Wang et al., 2019; He et al.,
2020; Huang et al., 2021): Gowalla333https://snap.stanford.edu/data/loc-
gowalla.html, Yelp-2018444https://www.yelp.com/dataset, Amazon-
book555https://jmcauley.ucsd.edu/data/amazon/, and
Alibaba666https://github.com/huangtinglin/MixGCF/tree/main/data/ali. Their
statistics are summarized in Table 1. For each dataset, we randomly select
$80\%$ of historical interactions of each user to construct the training set,
and treat the remaining as the test set. From the training set, we randomly
select $10\%$ of interactions as the validation set to tune the hyper-
parameters.

####  4.1.2. Baselines and Evaluations.

To verify the effectiveness of HQ-GNN, we mainly compare with graph-based
models: NGCF (Wang et al., 2019), LightGCN (He et al., 2020), HashNet (Cao et
al., 2017) and HashGNN (Tan et al., 2020). For HashNet, HashGNN and HQ-GNN, we
can choose any GNN encoder to compute the continuous node embeddings in Eq.
(2). The comparison against other methods (e.g., factorization machines) is
omitted, since most of them are outperformed by LightGCN. We choose the
widely-used Recall$@k$ and NDCG$@k$ as the evaluation metrics (Wang et al.,
2019; He et al., 2020; Huang et al., 2021). We simply set $k=50$ in all
experiments (Tan et al., 2020).

####  4.1.3. Implementation Details.

For all baselines, the embedding size of user/item is searched among
$\\{16,32,64,128\\}$. The hyper-parameters (e.g., batch size, learning rate)
of baselines are initialized as their original settings and are then carefully
tuned to achieve the optimal performance. For HQ-GNN, we search $L_{2}$
regularizer $\alpha$ within $\\{10^{-5},10^{-4},10^{-3},10^{-2},10^{-1}\\}$.
In addition, we determine the upper/lower thresholds (Eq. (3)) by exponential
moving averages (Jacob et al., 2018), and set the number of bits $b=1$ in Eq.
(5) for fair comparisons with binary hash methods: HashNet (Cao et al., 2017)
and HashGNN (Tan et al., 2020).

###  4.2. Experimental Results

####  4.2.1. Overall Performance.

We present a comprehensive performance comparison between full-precision GNNs
and quantization-aware GNNs. We summarize the results in terms of Recall$@50$
and NDCG$@50$ for different datasets in Table 2. From the table, we have two
major observations: 1) Among all 1-bit GNNs, our proposed HQ-GNN consistently
outperforms both HashNet and HashGNN by a large margin on all four datasets.
Clearly, this reveals that our HQ-GNNs provide a meaningful gradient
adjustments for non-differentiable quantized function. For example, for
LightGCN encoder, HQ-GNN has on average $15.80\%$ improvement with respect to
Recall$@50$ and over $15.63\%$ improvement with respect to NDCG$@50$,
comparing to the state-of-the-art HashGNN. 2) It is not surprised that full-
precision GNNs perform better than quantization-aware GNNs in all cases.
However, quantization-aware GNNs benefit from both lower memory footprint and
faster inference speed comparing to vanilla GNN.

In terms of memory and inference speed, we have observed similar results as
those reported in HashNet (Cao et al., 2017) and HashGNN (Tan et al., 2020).
This is because our HQ-GNN, with $b=1$, inherits all the benefits of HashGNN.
For instance, using binarized embeddings (1 bit) can significantly reduce
memory usage as compared to using FP32 embeddings. Moreover, the inference
speed of our HQ-GNNs is approximately 3.6 times faster than that of full-
precision GNNs because the Hamming distance between two binary embeddings can
be calculated efficiently (Tan et al., 2020). These features make our HQ-GNN
more desirable for large-scale retrieval applications in the industry.

####  4.2.2. Compared to GTE

The STE method propagates the same gradient from an output to an input of the
discretizer, assuming that the derivative of the discretizer is equal to 1. In
contrast, our GSTE method adopts the Hessian to refine the gradients. To
evaluate the effectiveness of our GSTE method, we chose LightGCN as the
backbone and quantized its embeddings into 1 bit. The performance on different
datasets is summarized in Table 3. From the table, it is clear that our GSTE
method performs better than STE for 1-bit quantization, with improvements
ranging from $14.7\%$ to $24.5\%$.

Regarding running time, during the training stage, our GSTE method requires
computing the trace of Hessian using Hutchinson’s method, which is however
fast. From Table 3, we can see that our GSTE method is slightly slower than
STE, which is negligible in practice. During inference, both our GSTE and STE
methods have the same speed as both use 1-bit quantized embeddings for
retrieval, and the trace of Hessian is not needed in the inference stage.

The left of Figure 1 also displays the training curves of GSTE and STE, and we
clearly observe that training quantized LightGCN with GSTE is better than STE
in terms of stability. This highlights the effectiveness of utilizing Hessian
information in the training process. The right of Figure 1 shows the impact of
quantization levels by varying $b$ within $\\{1,2,3,4\\}$ for both GSTE and
STE. As can be seen, aggressive quantization (less than 2-bit precision) can
lead to significant degradation in the accuracy. When $b=4$, HQ-GNN obtains
$98.5\%$ performance recovery of LightGCN. Comparing STE and GSTE, our GSTE
consistently performance better than STE in all cases. In summary, HQ-GNN
strikes a good balance between latency and performance.

Table 3.  The performance and the running time of 1-bit quantized LightGCN
with STE and GSTE.

| Gowalla | Yelp-2018 | Amazon-Book | Alibaba
---|---|---|---|---
LightGCN | Recall@50 | Time(sec) | Recall@50 | Time(sec) | Recall@50 | Time(sec) | Recall@50 | Time(sec)
+STE | 0.122 | 30.4 | 0.092 | 41.7 | 0.074 | 103.6 | 0.061 | 22.2
+GSTE | 0.152 | 32.9 | 0.108 | 45.1 | 0.089 | 110.7 | 0.070 | 23.9
Improv(%) | +$24.5\%$ | - | +$17.3\%$ | - | +$20.2\%$ | - | +$14.7\%$ | -

Figure 1. Left: GSTE vs. STE over training loss. Right: the impact of the
number of bits in the HQ-GNN.

##  5\. Conclusion

Training graph neural networks on large-scale user-item bipartite graphs has
been a challenging task due to the extensive memory requirement. To address
this problem, we propose HQ-GNN that explores the issue of low-bit
quantization of graph neural networks for large-scale recommendations.
Additionally, we introduce a Generalized Straight-Through Estimator to solve
the gradient mismatch problem that arises during the training of quantized
networks. HQ-GNN is flexible and can be applied to various graph neural
networks. The effectiveness of our proposed method is demonstrated through
extensive experiments on real-world datasets.

## References

  * (1)
  * Avron and Toledo (2011) Haim Avron and Sivan Toledo. 2011.  Randomized algorithms for estimating the trace of an implicit symmetric positive semi-definite matrix.  _J. ACM_ (2011), 1–34.
  * Bengio et al. (2013) Yoshua Bengio, Nicholas Léonard, and Aaron Courville. 2013\.  Estimating or propagating gradients through stochastic neurons for conditional computation.  _arXiv preprint arXiv:1308.3432_ (2013).
  * Cao et al. (2017) Zhangjie Cao, Mingsheng Long, Jianmin Wang, and Philip S Yu. 2017\.  Hashnet: Deep learning to hash by continuation. In _Proceedings of the IEEE international conference on computer vision_. 5608–5617.
  * Chen et al. (2022b) Huiyuan Chen, Xiaoting Li, Kaixiong Zhou, Xia Hu, Chin-Chia Michael Yeh, Yan Zheng, and Hao Yang. 2022b.  TinyKG: Memory-Efficient Training Framework for Knowledge Graph Neural Recommender Systems. In _Proceedings of the 16th ACM Conference on Recommender Systems_. 257–267.
  * Chen et al. (2021) Huiyuan Chen, Yusan Lin, Fei Wang, and Hao Yang. 2021\.  Tops, bottoms, and shoes: building capsule wardrobes via cross-attention tensor network. In _Proceedings of the 15th ACM Conference on Recommender Systems_. 453–462.
  * Chen et al. (2022c) Huiyuan Chen, Chin-Chia Michael Yeh, Fei Wang, and Hao Yang. 2022c.  Graph neural transport networks with non-local attentions for recommender systems. In _Proceedings of the ACM Web Conference 2022_. 1955–1964.
  * Chen et al. (2022d) Huiyuan Chen, Kaixiong Zhou, Kwei-Herng Lai, Xia Hu, Fei Wang, and Hao Yang. 2022d.  Adversarial graph perturbations for recommendations at scale. In _Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval_. 1854–1858.
  * Chen et al. (2022a) Yankai Chen, Huifeng Guo, Yingxue Zhang, Chen Ma, Ruiming Tang, Jingjie Li, and Irwin King. 2022a.  Learning binarized graph representations with multi-faceted quantization reinforcement for top-k recommendation. In _Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining_. 168–178.
  * Choi et al. (2018) Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, and Kailash Gopalakrishnan. 2018.  Pact: Parameterized clipping activation for quantized neural networks.  _arXiv preprint arXiv:1805.06085_ (2018).
  * Dong et al. (2020) Zhen Dong, Zhewei Yao, Daiyaan Arfeen, Amir Gholami, Michael W Mahoney, and Kurt Keutzer. 2020\.  Hawq-v2: Hessian aware trace-weighted quantization of neural networks.  _Advances in neural information processing systems_.
  * Dong et al. (2019) Zhen Dong, Zhewei Yao, Amir Gholami, Michael W Mahoney, and Kurt Keutzer. 2019.  Hawq: Hessian aware quantization of neural networks with mixed-precision. In _Proceedings of the IEEE/CVF International Conference on Computer Vision_. 293–302.
  * Gong et al. (2019) Ruihao Gong, Xianglong Liu, Shenghu Jiang, Tianxiang Li, Peng Hu, Jiazhen Lin, Fengwei Yu, and Junjie Yan. 2019\.  Differentiable soft quantization: Bridging full-precision and low-bit neural networks. In _Proceedings of the IEEE/CVF International Conference on Computer Vision_. 4852–4861.
  * Han et al. (2016) Song Han, Huizi Mao, and William J Dally. 2016.  Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding.  _International Conference on Learning Representations_.
  * He et al. (2020) Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020\.  Lightgcn: Simplifying and powering graph convolution network for recommendation. In _Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval_. 639–648.
  * Huang et al. (2020) Jui-Ting Huang, Ashish Sharma, Shuying Sun, Li Xia, David Zhang, Philip Pronin, Janani Padmanabhan, Giuseppe Ottaviano, and Linjun Yang. 2020.  Embedding-based retrieval in facebook search. In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 2553–2561.
  * Huang et al. (2021) Tinglin Huang, Yuxiao Dong, Ming Ding, Zhen Yang, Wenzheng Feng, Xinyu Wang, and Jie Tang. 2021.  MixGCF: An Improved Training Method for Graph Neural Network-Based Recommender systems. In _Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining_. 665–674.
  * Jacob et al. (2018) Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. 2018\.  Quantization and training of neural networks for efficient integer-arithmetic-only inference. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 2704–2713.
  * Jiang et al. (2021) Gangwei Jiang, Hao Wang, Jin Chen, Haoyu Wang, Defu Lian, and Enhong Chen. 2021\.  xLightFM: Extremely Memory-Efficient Factorization Machine. In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_. 337–346.
  * Jing et al. (2021) Yongcheng Jing, Yiding Yang, Xinchao Wang, Mingli Song, and Dacheng Tao. 2021.  Meta-Aggregator: Learning to Aggregate for 1-bit Graph Neural Networks. In _Proceedings of the IEEE/CVF International Conference on Computer Vision_. 5301–5310.
  * Kang and McAuley (2019) Wang-Cheng Kang and Julian McAuley. 2019.  Candidate generation with binary codes for large-scale top-n recommendation. In _Proceedings of the 28th ACM international conference on information and knowledge management_. 1523–1532.
  * Kim et al. (2021) Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer. 2021.  I-bert: Integer-only bert quantization. In _International conference on machine learning_. 5506–5518.
  * Lee et al. (2021) Junghyup Lee, Dohyung Kim, and Bumsub Ham. 2021.  Network Quantization with Element-wise Gradient Scaling. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 6448–6457.
  * Lian et al. (2020) Defu Lian, Haoyu Wang, Zheng Liu, Jianxun Lian, Enhong Chen, and Xing Xie. 2020\.  Lightrec: A memory and search-efficient recommender system. In _Proceedings of The Web Conference 2020_. 695–705.
  * Micikevicius et al. (2018) Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, and Hao Wu. 2018\.  Mixed Precision Training. In _International Conference on Learning Representations_.
  * Reisizadeh et al. (2020) Amirhossein Reisizadeh, Aryan Mokhtari, Hamed Hassani, Ali Jadbabaie, and Ramtin Pedarsani. 2020\.  Fedpaq: A communication-efficient federated learning method with periodic averaging and quantization. In _International Conference on Artificial Intelligence and Statistics_.
  * Shi et al. (2020) Hao-Jun Michael Shi, Dheevatsa Mudigere, Maxim Naumov, and Jiyan Yang. 2020.  Compositional embeddings using complementary partitions for memory-efficient recommendation systems. In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 165–175.
  * Sun et al. (2020) Xiao Sun, Naigang Wang, Chia-Yu Chen, Jiamin Ni, Ankur Agrawal, Xiaodong Cui, Swagath Venkataramani, Kaoutar El Maghraoui, Vijayalakshmi Viji Srinivasan, and Kailash Gopalakrishnan. 2020.  Ultra-low precision 4-bit training of deep neural networks.  _Advances in Neural Information Processing Systems_.
  * Tan et al. (2020) Qiaoyu Tan, Ninghao Liu, Xing Zhao, Hongxia Yang, Jingren Zhou, and Xia Hu. 2020\.  Learning to Hash with Graph Neural Networks for Recommender Systems. In _Proceedings of The Web Conference 2020_. 1988–1998.
  * Wang et al. (2023) Song Wang, Xingbo Fu, Kaize Ding, Chen Chen, Huiyuan Chen, and Jundong Li. 2023\.  Federated Few-shot Learning. In _Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining_.
  * Wang et al. (2019) Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019.  Neural graph collaborative filtering. In _Proceedings of the 42nd international ACM SIGIR conference on Research and development in Information Retrieval_. 165–174.
  * Wang et al. (2022) Yu Wang, Yuying Zhao, Yushun Dong, Huiyuan Chen, Jundong Li, and Tyler Derr. 2022\.  Improving fairness in graph neural networks via mitigating sensitive attribute leakage. In _Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining_. 1938–1948.
  * Wu et al. (2021) Wei Wu, Bin Li, Chuan Luo, and Wolfgang Nejdl. 2021\.  Hashing-accelerated graph neural networks for link prediction. In _Proceedings of the Web Conference 2021_. 2910–2920.
  * Xu et al. (2023) Zhe Xu, Yuzhong Chen, Menghai Pan, Huiyuan Chen, Mahashweta Das, and Hao Yang. 2023\.  Kernel Ridge Regression-Based Graph Dataset Distillation. In _Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining_.
  * Yeh et al. (2022) Chin-Chia Michael Yeh, Mengting Gu, Yan Zheng, Huiyuan Chen, Javid Ebrahimi, Zhongfang Zhuang, Junpeng Wang, Liang Wang, and Wei Zhang. 2022\.  Embedding Compression with Hashing for Efficient Representation Learning in Large-Scale Graph. In _Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining_. 4391–4401.
  * Yin et al. (2019) Penghang Yin, Jiancheng Lyu, Shuai Zhang, Stanley J. Osher, Yingyong Qi, and Jack Xin. 2019\.  Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets. In _International Conference on Learning Representations_.
  * Ying et al. (2018) Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018\.  Graph convolutional neural networks for web-scale recommender systems. In _Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 974–983.
  * Yuan et al. (2023) Wei Yuan, Hongzhi Yin, Fangzhao Wu, Shijie Zhang, Tieke He, and Hao Wang. 2023\.  Federated unlearning for on-device recommendation. In _Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining_. 393–401.
  * Zhou et al. (2016) Shuchang Zhou, Yuxin Wu, Zekun Ni, Xinyu Zhou, He Wen, and Yuheng Zou. 2016.  Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients.  _arXiv preprint arXiv:1606.06160_ (2016).
  * Zhu et al. (2020) Feng Zhu, Ruihao Gong, Fengwei Yu, Xianglong Liu, Yanfei Wang, Zhelong Li, Xiuqi Yang, and Junjie Yan. 2020\.  Towards unified int8 training for convolutional neural network. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 1969–1979.