# Twisted Rota-Baxter operators on $3$-Lie algebras and NS-$3$-Lie algebras

Shuai Hou  Department of Mathematics, Jilin University, Changchun 130012,
Jilin, China<EMAIL_ADDRESS>and  Yunhe Sheng  Department of Mathematics,
Jilin University, Changchun 130012, Jilin, China<EMAIL_ADDRESS>
###### Abstract.

In this paper, first we introduce the notion of a twisted Rota-Baxter operator
on a $3$-Lie algebra $\mathfrak{g}$ with a representation on $V$. We show that
a twisted Rota-Baxter operator induces a 3-Lie algebra structure on $V$, which
represents on $\mathfrak{g}$. By this fact, we define the cohomology of a
twisted Rota-Baxter operator and study infinitesimal deformations of a twisted
Rota-Baxter operator using the second cohomology group. Then we introduce the
notion of an NS-$3$-Lie algebra, which produces a 3-Lie algebra with a
representation on itself. We show that a twisted Rota-Baxter operator induces
an NS-$3$-Lie algebra naturally. Thus NS-$3$-Lie algebras can be viewed as the
underlying algebraic structures of twisted Rota-Baxter operators on $3$-Lie
algebras. Finally we show that a Nijenhuis operator on a 3-Lie algebra gives
rise to a representation of the deformed 3-Lie algebra and a 2-cocycle.
Consequently, the identity map will be a twisted Rota-Baxter operator on the
deformed 3-Lie algebra. We also introduce the notion of a Reynolds operator on
a $3$-Lie algebra, which can serve as a special case of twisted Rota-Baxter
operators on 3-Lie algebras.

###### Key words and phrases:

Twisted-Rota Baxter operator, NS-$3$-Lie algebra, Nijenhuis operator, Reynolds
operator
_2020 Mathematics Subject Classification._ 17A42, 17B38, 17B40, 17B56
Corresponding author: Yunhe Sheng

###### Contents

  1. 1 Introduction
    1. 1.1 Cohomology of Rota-Baxter operators on Lie algebras and 3-Lie algebras
    2. 1.2 Nijenhuis operators, Reynolds operators and NS-algebras
    3. 1.3 Main results and outline of the paper
  2. 2 Cohomologies of twisted Rota-Baxter operators on $3$-Lie algebras
    1. 2.1 Twisted Rota-Baxter operators on $3$-Lie algebras
    2. 2.2 Cohomology of $\Phi$-twisted Rota-Baxter operators on $3$-Lie algebras
    3. 2.3 Infinitesimal deformations of $\Phi$-twisted Rota-Baxter operators on $3$-Lie algebras
  3. 3 NS-$3$-Lie algebras
  4. 4 Nijenhuis operators and Reynolds operators on $3$-Lie algebras
    1. 4.1 Nijenhuis operators on $3$-Lie algebras
    2. 4.2 Reynolds operators on $3$-Lie algebras

##  1\. Introduction

###  1.1. Cohomology of Rota-Baxter operators on Lie algebras and 3-Lie
algebras

The notion of a Rota-Baxter operator on an associative algebra was introduced
in the probability study of G. Baxter and later found important applications
in the Connes-Kreimer’s algebraic approach to renormalization of quantum field
theory [7]. For further details, see [16, 17]. A Rota-Baxter operator on a Lie
algebra is naturally the operator form of a classical $r$-matrix [28] under
certain conditions. To better understand such connection in general,
Kupershmidt introduced the notion of an ${\mathcal{O}}$-operator (also called
a relative Rota-Baxter operator [25] or a generalized Rota-Baxter operator
[33]) on a Lie algebra in [20]. Recently the deformation theory and the
cohomology theory of relative Rota-Baxter operators on both Lie and
associative algebras are studied in [8, 31].

3-Lie algebras and more generally, $n$-Lie algebras (also called Filippov
algebras) [15], have attracted attention from both mathematics and physics.
See the review article [11, 24] for more details. In [2], The authors
introduced the notion of a Rota-Baxter operator on a 3-Lie algebra and studied
its relation with Rota-Baxter Lie algebras. To construct solutions of the
classical 3-Lie Yang-Baxter equation, the authors introduced a more general
notion, an ${\mathcal{O}}$-operator (also called a relative Rota-Baxter
operator) on a 3-Lie algebra with respect to a representation in [1]. In [32],
the authors constructed a Lie 3-algebra whose Maurer-Cartan elements are
relative Rota-Baxter operators on 3-Lie algebras, and studied cohomologies and
deformations of relative Rota-Baxter operators on 3-Lie algebras. See [14] for
more details about deformations of 3-Lie algebras.

###  1.2. Nijenhuis operators, Reynolds operators and NS-algebras

Nijenhuis operators play an important role in deformation theories due to
their relationship with trivial infinitesimal deformations. There are
interesting applications of Nijenhuis operators on associative algebras and
Lie algebras such as constructing biHamiltonian systems to study the
integrability of nonlinear evolution equations [4, 12]. The notion of a
Nijenhuis operator on an $n$-Lie algebra was introduced in [23] to study
deformations of $n$-Lie algebras. Then it was applied to study product
structures and complex structures on a 3-Lie algebra in [29]. See [35] for a
different notion of Nijenhuis operators on 3-Lie algebras.

Reynolds operators were introduced by Reynolds in [27] in the study of
fluctuation theory in fluid dynamics. In [18], the author coined the concept
of the Reynolds operator and regarded the operator as a mathematical subject
in general. In [34], the authors provided examples and properties of the
Reynolds operators and studied the free Reynolds algebras.

NS-algebras were introduced by Leroux in [22]. In [21], the authors studied
the relationship between the category of Nijenhuis algebras and the category
of NS-algebras. Uchino defined the notion of twisted Rota-Baxter operators on
associative algebras and observed that a twisted Rota-Baxter operator induces
an NS-algebra in [33]. In [10], Das studied the cohomology of twisted Rota-
Baxter operators on associative algebras and the cohomology of NS-algebras and
gave various application. Das also introduced the notions of twisted Rota-
Baxter operators, Reynolds operators on Lie algebras and NS-Lie algebras, and
showed that NS-Lie algebras are the underlying algebraic structures of twisted
Rota-Baxter operators on Lie algebras in [9].

###  1.3. Main results and outline of the paper

In this paper, we introduce the notion of a twisted Rota-Baxter operator (by a
2-cocycle) on a 3-Lie algebra, and study its cohomology theory and the
underlying algebraic structure. Note that a twisted Rota-Baxter operator
$T:V\,\rightarrow\,\mathfrak{g}$ on a 3-Lie algebra $\mathfrak{g}$ with
respect to a representation on $V$ naturally induces a 3-Lie algebra structure
on $V$ with a representation on $\mathfrak{g}$. The corresponding cohomology
is defined to be the cohomology of the relative Rota-Baxter operator. As
applications, we use the second cohomology group to study infinitesimal
deformations of twisted Rota-Baxter operators on 3-Lie algebras. We introduce
the notion of an NS-3-Lie algebra and show that a twisted Rota-Baxter operator
on a 3-Lie algebra naturally induces an NS-3-Lie algebra. Thus, NS-3-Lie
algebras can be viewed as the underlying algebraic structures of twisted Rota-
Baxter operators on 3-Lie algebras. We study two special classes of twisted
Rota-Baxter operators on 3-Lie algebras:

  * •

We show that the identity map is a twisted Rota-Baxter operator on the
deformed 3-Lie algebra by a Nijenhuis operator;

  * •

We introduce the notion of a Reynolds operator on a $3$-Lie algebra, which
turns out to be a special twisted Rota-Baxter operator.

We also give concrete examples of Reynolds operators on some infinite
dimensional 3-Lie algebras including the $\omega_{\infty}$ $3$-Lie algebra.

The paper is organized as follows. In Section 2, we introduce the notion of a
twisted Rota-Baxter operator on a 3-Lie algebra and establish the
corresponding cohomology theory. Applications are given to study infinitesimal
deformations. In Section 3, we introduce the notion of an NS-3-Lie algebra and
show that a twisted Rota-Baxter operator on a 3-Lie algebra naturally induces
an NS-3-Lie algebra. In Section 4, first we show that a Nijenhuis operator on
a 3-Lie algebra induces a representation of the deformed 3-Lie algebra and a
2-cocycle. Consequently, the identity map is a twisted Rota-Baxter operator on
the deformed 3-Lie algebra. Then we introduce the notion of a Reynolds
operator on a $3$-Lie algebra, which turns out to be a special twisted Rota-
Baxter operator. Finally we give examples of Reynolds operators on some
infinite dimensional 3-Lie algebras.

Acknowledgements.  This research is supported by NSFC (11922110).

##  2\. Cohomologies of twisted Rota-Baxter operators on $3$-Lie algebras

###  2.1. Twisted Rota-Baxter operators on $3$-Lie algebras

In this subsection, we introduce the notion of a $\Phi$-twisted Rota-Baxter
operator on a 3-Lie algebra and show that a linear map
$T:V\rightarrow\mathfrak{g}$ is a $\Phi$-twisted Rota-Baxter operator if and
only if the graph of $T$ is a subalgebra of the $\Phi$-twisted semidirect
product 3-Lie algebra $\mathfrak{g}\ltimes_{\Phi}V$, where $\Phi$ is a
2-cocycle of the 3-Lie algebra $\mathfrak{g}$ with coefficients in $V$.
Consequently, a $\Phi$-twisted Rota-Baxter operator $T$ induces a $3$-Lie
algebra structure on $V.$ We also use a $T$-admissible $1$-cocycle $f$ to
construct a new $\Phi$-twisted Rota-Baxter operator $T_{f}$, and the $3$-Lie
algebra structures on $V$ induced by $T$ and $T_{f}$ are isomorphic.

######  Definition 2.1.

([15]) A 3-Lie algebra is a vector space $\mathfrak{g}$ together with a skew-
symmetric linear map
$[\cdot,\cdot,\cdot]_{\mathfrak{g}}:\otimes^{3}\mathfrak{g}\rightarrow\mathfrak{g}$,
such that for $x_{i}\in\mathfrak{g},1\leq i\leq 5$, the following Fundamental
Identity holds:

(1)
$~{}[x_{1},x_{2},[x_{3},x_{4},x_{5}]_{\mathfrak{g}}]_{\mathfrak{g}}=[[x_{1},x_{2},x_{3}]_{\mathfrak{g}},x_{4},x_{5}]_{\mathfrak{g}}+[x_{3},[x_{1},x_{2},x_{4}]_{\mathfrak{g}},x_{5}]_{\mathfrak{g}}+[x_{3},x_{4},[x_{1},x_{2},x_{5}]_{\mathfrak{g}}]_{\mathfrak{g}}.$

A derivation on a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ is a linear map
$D:\mathfrak{g}\rightarrow\mathfrak{g}$ satisfying

(2) $\displaystyle
D[x_{1},x_{2},x_{2}]_{\mathfrak{g}}=[Dx_{1},x_{2},x_{3}]_{\mathfrak{g}}+[x_{1},Dx_{2},x_{3}]_{\mathfrak{g}}+[x_{1},x_{2},Dx_{3}]_{\mathfrak{g}},\quad\forall
x_{1},x_{2},x_{3}\in\mathfrak{g}.$

For $x_{1},x_{2}\in\mathfrak{g},$ define
$\mathrm{ad}:\wedge^{2}\mathfrak{g}\rightarrow\mathfrak{gl}(\mathfrak{g})$ by

$\mathrm{ad}_{x_{1},x_{2}}x_{3}=[x_{1},x_{2},x_{3}]_{\mathfrak{g}},\quad\forall
x_{3}\in\mathfrak{g}.$

Then (1) is equivalent to that $\mathrm{ad}_{x_{1},x_{2}}$ is a derivation,
i.e.

$\mathrm{ad}_{x_{1},x_{2}}[x_{3},x_{4},x_{5}]_{\mathfrak{g}}=[\mathrm{ad}_{x_{1},x_{2}}x_{3},x_{4},x_{5}]_{\mathfrak{g}}+[x_{3},\mathrm{ad}_{x_{1},x_{2}}x_{4},x_{5}]_{\mathfrak{g}}+[x_{3},x_{4},\mathrm{ad}_{x_{1},x_{2}}x_{5}]_{\mathfrak{g}}.$

######  Definition 2.2.

([19]) A representation of a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ is a pair $(V;\rho)$,
where $V$ is a vector space, and
$\rho:\wedge^{2}\mathfrak{g}\rightarrow\mathfrak{gl}(V)$ is a linear map
satisfying, for all $x_{1},x_{2},x_{3},x_{4}\in\mathfrak{g},$

(3) $\displaystyle[\rho(x_{1},x_{2}),\rho(x_{3},x_{4})]$ $\displaystyle=$
$\displaystyle\rho([x_{1},x_{2},x_{3}]_{\mathfrak{g}},x_{4})+\rho(x_{3},[x_{1},x_{2},x_{4}]_{\mathfrak{g}}),$
(4) $\displaystyle\rho([x_{1},x_{2},x_{3}]_{\mathfrak{g}},x_{4})$
$\displaystyle=$
$\displaystyle\rho(x_{1},x_{2})\rho(x_{3},x_{4})+\rho(x_{2},x_{3})\rho(x_{1},x_{4})+\rho(x_{3},x_{1})\rho(x_{2},x_{4}).$

See [13] for representations of vector product $n$-Lie algebras.

######  Example 2.3.

Let $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ be a $3$-Lie algebra.
The linear map
$\mathrm{ad}:\wedge^{2}\mathfrak{g}\rightarrow\mathfrak{gl}(\mathfrak{g})$
defines a representation of the $3$-Lie algebra $\mathfrak{g}$ on itself,
which is called the adjoint representation of $\mathfrak{g}.$

Let $(V;\rho)$ be a representation of a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$. Denote by

$C_{\mathsf{3Lie}}^{n}(\mathfrak{g};V):=\mathrm{Hom}(\underbrace{\wedge^{2}\mathfrak{g}\otimes\cdots\otimes\wedge^{2}\mathfrak{g}}_{(n-1)}\wedge\mathfrak{g},V),\quad(n\geq
1),$

which is the space of $n$-cochains. The coboundary operator ${\rm
d}:C_{\mathsf{3Lie}}^{n}(\mathfrak{g};V)\rightarrow
C_{\mathsf{3Lie}}^{n+1}(\mathfrak{g};V)$ is defined by

$\displaystyle({\rm d}f)(\mathfrak{X}_{1},\cdots,\mathfrak{X}_{n},x_{n+1})$
$\displaystyle=$ $\displaystyle\sum_{1\leq j<k\leq
n}(-1)^{j}f(\mathfrak{X}_{1},\cdots,\hat{\mathfrak{X}_{j}},\cdots,\mathfrak{X}_{k-1},[x_{j},y_{j},x_{k}]_{\mathfrak{g}}\wedge
y_{k}+x_{k}\wedge[x_{j},y_{j},y_{k}]_{\mathfrak{g}},\mathfrak{X}_{k+1},\cdots,\mathfrak{X}_{n},x_{n+1})$
$\displaystyle+\sum_{j=1}^{n}(-1)^{j}f(\mathfrak{X}_{1},\cdots,\hat{\mathfrak{X}_{j}},\cdots,\mathfrak{X}_{n},[x_{j},y_{j},x_{n+1}]_{\mathfrak{g}})$
$\displaystyle+\sum_{j=1}^{n}(-1)^{j+1}\rho(x_{j},y_{j})f(\mathfrak{X}_{1},\cdots,\hat{\mathfrak{X}_{j}},\cdots,\mathfrak{X}_{n},x_{n+1})$
$\displaystyle+(-1)^{n+1}\Big{(}\rho(y_{n},x_{n+1})f(\mathfrak{X}_{1},\cdots,\mathfrak{X}_{n-1},x_{n})+\rho(x_{n+1},x_{n})f(\mathfrak{X}_{1},\cdots,\mathfrak{X}_{n-1},y_{n})\Big{)},$

for all$~{}\mathfrak{X}_{i}=x_{i}\wedge
y_{i}\in\wedge^{2}\mathfrak{g},~{}i=1,2,\cdots,n~{}and~{}x_{n+1}\in\mathfrak{g}.$
It was proved in [5, 30] that ${\rm d}\circ{\rm d}=0.$ Thus,
$(\oplus_{n=1}^{+\infty}C_{\mathsf{3Lie}}^{n}(\mathfrak{g};V),{\rm d})$ is a
cochain complex.

######  Definition 2.4.

The cohomology of the $3$-Lie algebra $\mathfrak{g}$ with coefficients in $V$
is the cohomology of the cochain complex
$(\oplus_{n=1}^{+\infty}C_{\mathsf{3Lie}}^{n}(\mathfrak{g};V),{\rm d})$.
Denote by $\mathcal{Z}_{\mathsf{3Lie}}^{n}(\mathfrak{g};V)$ and
$\mathcal{B}_{\mathsf{3Lie}}^{n}(\mathfrak{g};V)$ the set of $n$-cocycles and
the set of $n$-coboundaries, respectively. The $n$-th cohomology group is
defined by

$\displaystyle\mathcal{H}_{\mathsf{3Lie}}^{n}(\mathfrak{g};V)=\mathcal{Z}_{\mathsf{3Lie}}^{n}(\mathfrak{g};V)/\mathcal{B}_{\mathsf{3Lie}}^{n}(\mathfrak{g};V).$

A $2$-cochain $\Phi\in
C_{\mathsf{3Lie}}^{2}(\mathfrak{g};V)=\mathrm{Hom}(\wedge^{3}\mathfrak{g},V)$
is a 2-cocycle on $\mathfrak{g}$ with coefficients in $(V;\rho)$ if $\Phi$
satisfies

$\displaystyle\Phi(x_{1},x_{2},[x_{3},x_{4},x_{5}]_{\mathfrak{g}})-\Phi([x_{1},x_{2},x_{3}]_{\mathfrak{g}},x_{4},x_{5})-\Phi(x_{3},[x_{1},x_{2},x_{4}]_{\mathfrak{g}},x_{5})-\Phi(x_{3},x_{4},[x_{1},x_{2},x_{5}]_{\mathfrak{g}})$
$\displaystyle+\rho(x_{1},x_{2})\Phi(x_{3},x_{4},x_{5})-\rho(x_{3},x_{4})\Phi(x_{1},x_{2},x_{5})-\rho(x_{4},x_{5})\Phi(x_{1},x_{2},x_{3})-\rho(x_{5},x_{3})\Phi(x_{1},x_{2},x_{4})$
$\displaystyle=$ $\displaystyle 0,$

for all $x_{i}\in\mathfrak{g},1\leq i\leq 5$.

Let $\mathfrak{g}$ be a $3$-Lie algebra and $(V;\rho)$ be a representation of
$\mathfrak{g}.$ For any $2$-cocycle $\Phi\in
C_{\mathsf{3Lie}}^{2}(\mathfrak{g};V),$ there is a $3$-Lie algebra structure
on the direct sum $\mathfrak{g}\oplus V$ of vector spaces, defined by

(6)
$\displaystyle[(x,u),(y,v),(z,w)]_{\Phi}=([x,y,z]_{\mathfrak{g}},\rho(x,y)w+\rho(y,z)u+\rho(z,x)v+\Phi(x,y,z)),$

for $x,y,z\in\mathfrak{g},u,v,w\in V.$ This $3$-Lie algebra is called the
$\Phi$-twisted semi-direct product $3$-Lie algebra and denoted by
$\mathfrak{g}\ltimes_{\Phi}V.$

In the sequel, $\Phi$ will always denote a $2$-cocycle.

######  Definition 2.5.

Let $\mathfrak{g}$ be a $3$-Lie algebra and $(V;\rho)$ be a representation of
$\mathfrak{g}.$ A linear operator $T:V\rightarrow\mathfrak{g}$ is called a
$\Phi$-twisted Rota-Baxter operator on $\mathfrak{g}$ with respect to
$(V;\rho)$ if $T$ satisfies

(7)
$[Tu,Tv,Tw]_{\mathfrak{g}}=T\Big{(}\rho(Tu,Tv)w+\rho(Tv,Tw)u+\rho(Tw,Tu)v+\Phi(Tu,Tv,Tw)\Big{)},\quad\forall~{}u,v,w\in
V.$

######  Remark 2.6.

In [2], the authors introduced the notion of a Rota-Baxter operator of weight
$\lambda$ on a $3$-Lie algebra and studied its relation with Rota-Baxter Lie
algebras. In [1], the authors introduced the notion of an
${\mathcal{O}}$-operator on a $3$-Lie algebra with respect to a representation
to study solutions of $3$-Lie Yang-Baxter equations. It is straightforward to
see that $\Phi$-twisted Rota-Baxter operators on $3$-Lie algebras are
generalizations of Rota-Baxter operators of weight $0$ and
${\mathcal{O}}$-operators on $3$-Lie algebras.

We give a construction of $\Phi$-twisted Rota-Baxter operators on $3$-Lie
algebras.

######  Proposition 2.7.

Let $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ be a $3$-Lie algebra
and $(V;\rho)$ be a representation. Suppose that $f:\mathfrak{g}\rightarrow V$
is an invertible linear map. Then $T=f^{-1}:V\rightarrow\mathfrak{g}$ is a
$\Phi$-twisted Rota-Baxter operator with $\Phi=-{\rm d}f.$

###### Proof.

By a direct computation, we have

$\displaystyle\Phi(Tu,Tv,Tw)$ $\displaystyle=$ $\displaystyle-({\rm
d}f)(Tu,Tv,Tw)$ $\displaystyle=$
$\displaystyle-\rho(Tu,Tv)w-\rho(Tv,Tw)u-\rho(Tw,Tu)v+f([Tu,Tv,Tw]_{\mathfrak{g}}).$

By applying $T$ to both sides of (2.1), we obtain the identity (7). ∎

The identity (7) can be characterized by the graph of $T$ being a subalgebra.

######  Theorem 2.8.

A linear map $T:V\rightarrow\mathfrak{g}$ is a $\Phi$-twisted Rota-Baxter
operator on a $3$-Lie algebra $\mathfrak{g}$ with respect to a representation
$(V;\rho)$ if and only if the graph $Gr(T)=\\{(Tu,u)|u\in V\\}$ is a
subalgebra of the $\Phi$-twisted semi-direct product $3$-Lie algebra
$\mathfrak{g}\ltimes_{\Phi}V.$

###### Proof.

Let $T:V\rightarrow\mathfrak{g}$ be a linear map. For all $u,v,w\in V,$ we
have

$\displaystyle[(Tu,u),(Tv,v),(Tw,w)]_{\Phi}$ $\displaystyle=$
$\displaystyle([Tu,Tv,Tw]_{\mathfrak{g}},\rho(Tu,Tv)w+\rho(Tv,Tw)u+\rho(Tw,Tu)v+\Phi(Tu,Tv,Tw)),$

which implies that the graph $Gr(T)=\\{(Tu,u)|u\in V\\}$ is a subalgebra of
the $\Phi$-twisted semi-direct product $3$-Lie algebra
$\mathfrak{g}\ltimes_{\Phi}V$ if and only if $T$ satisfies

$\displaystyle[Tu,Tv,Tw]_{\mathfrak{g}}=T\Big{(}\rho(Tu,Tv)w+\rho(Tv,Tw)u+\rho(Tw,Tu)v+\Phi(Tu,Tv,Tw)\Big{)},$

which means that $T$ is a $\Phi$-twisted Rota-Baxter operator. ∎

Since $V$ and $Gr(T)$ are isomorphic as vector spaces, we get the following
result immediately.

######  Corollary 2.9.

Let $T:V\rightarrow\mathfrak{g}$ be a $\Phi$-twisted Rota-Baxter operator on a
$3$-Lie algebra $\mathfrak{g}$ with respect to a representation $(V;\rho).$
Then there is a $3$-Lie algebra structure $[\cdot,\cdot,\cdot]_{T}$ on $V$
given by

(9)
$\displaystyle\qquad[u,v,w]_{T}=\rho(Tu,Tv)w+\rho(Tv,Tw)u+\rho(Tw,Tu)v+\Phi(Tu,Tv,Tw),\quad\forall
u,v,w\in V.$

Furthermore, $T$ is a homomorphism from the 3-Lie algebra
$(V,[\cdot,\cdot,\cdot]_{T})$ to
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$.

At the end of this subsection, we introduce the notion of a $T$-admissible
$1$-cocycle by which we construct a new $\Phi$-twisted Rota-Baxter operator on
a 3-Lie algebra.

Let $f:\mathfrak{g}\rightarrow V$ be a linear map. Define
$\Psi_{f}:\mathfrak{g}\oplus V\rightarrow\mathfrak{g}\oplus V$ by
$\Psi_{f}=\left(\begin{array}[]{cc}{\rm{Id}}&0\\\ f&{\rm{Id}}\\\
\end{array}\right).$

######  Proposition 2.10.

Let $\mathfrak{g}$ be a $3$-Lie algebra and $(V;\rho)$ be a representation.
Then $\Psi_{f}$ is an isomorphism from the $\Phi$-twisted semi-direct product
$3$-Lie algebra $\mathfrak{g}\ltimes_{\Phi}V$ to the $(\Phi-{\rm d}f)$-twisted
semi-direct product $3$-Lie algebra $\mathfrak{g}\ltimes_{(\Phi-{\rm d}f)}V$.

###### Proof.

For all $(x,u),(y,v),(z,w)\in\mathfrak{g}\oplus V,$ we have,

$\displaystyle\Psi_{f}[(x,u),(y,v),(z,w)]_{\Phi}$ $\displaystyle=$
$\displaystyle\Psi_{f}([x,y,z]_{\mathfrak{g}},\rho(x,y)w+\rho(z,x)v+\rho(y,z)u+\Phi(x,y,z))$
$\displaystyle=$
$\displaystyle([x,y,z]_{\mathfrak{g}},\rho(x,y)w+\rho(z,x)v+\rho(y,z)u+\Phi(x,y,z)+f[x,y,z]_{\mathfrak{g}})$
$\displaystyle=$
$\displaystyle([x,y,z]_{\mathfrak{g}},\rho(x,y)w+\rho(z,x)v+\rho(y,z)u+\Phi(x,y,z)$
$\displaystyle+\rho(x,y)f(z)+\rho(y,z)f(x)+\rho(z,x)f(y)-({\rm d}f)(x,y,z))$
$\displaystyle=$ $\displaystyle[(x,u+f(x)),(y,v+f(y)),(z,w+f(z))]_{(\Phi-{\rm
d}f)}$ $\displaystyle=$
$\displaystyle[\Psi_{f}(x,u),\Psi_{f}(y,v),\Psi_{f}(z,w)]_{(\Phi-{\rm d}f)},$

which implies that $\Psi_{f}$ is an isomorphism between $3$-Lie algebras. ∎

######  Definition 2.11.

Let $T$ be a $\Phi$-twisted Rota-Baxter operator on a $3$-Lie algebra
$\mathfrak{g}$ with respect to $(V;\rho)$. A $1$-cocycle $f$ of $\mathfrak{g}$
with coefficients in $V$ is called a $T$-admissible $1$-cocycle if the linear
map ${\rm{Id}}+f\circ T:V\rightarrow V$ is invertible.

######  Proposition 2.12.

With the above notations, if $f:\mathfrak{g}\rightarrow V$ is a $T$-admissible
$1$-cocycle, then $T({\rm{Id}}+f\circ T)^{-1}:V\rightarrow\mathfrak{g}$ is a
$\Phi$-twisted Rota-Baxter operator. We denote this $\Phi$-twisted Rota-Baxter
operator by $T_{f}.$

###### Proof.

Let $f$ be a $T$-admissible $1$-cocycle. Then $\Phi-{\rm d}f=\Phi$. By Theorem
2.8 and Proposition 2.10, $\Psi_{f}(Gr(T))=\\{(Tu,u+f(Tu))|u\in
V\\}\subset\mathfrak{g}\ltimes_{\Phi}V$ is a subalgebra of the $\Phi$-twisted
semidirect product $3$-Lie algebra $\mathfrak{g}\ltimes_{\Phi}V$. Since the
linear map $({\rm{Id}}+f\circ T):V\rightarrow V$ is invertible,
$\Psi_{f}(Gr(T))$ is the graph of $T({\rm{Id}}+f\circ
T)^{-1}:V\rightarrow\mathfrak{g},$ which implies that $T({\rm{Id}}+f\circ
T)^{-1}$ is a $\Phi$-twisted Rota-Baxter operator. This completes the proof. ∎

Recall from Corollary 2.9 that a $\Phi$-twisted Rota-Baxter operator induces a
$3$-Lie algebra on $V.$ We have the following conclusion.

######  Proposition 2.13.

Let $T$ be a $\Phi$-twisted Rota-Baxter operator and $f$ be a $T$-admissible
$1$-cocycle. Then the $3$-Lie algebra structures on $V$ induced by $T$ and
$T_{f}$ are isomorphic.

###### Proof.

Consider the linear map ${\rm{Id}}+f\circ T:V\rightarrow V.$ For all $u,v,w\in
V,$ we have

$\displaystyle[({\rm{Id}}+f\circ T)(u),({\rm{Id}}+f\circ
T)(v),({\rm{Id}}+f\circ T)(w)]_{T_{f}}$ $\displaystyle=$
$\displaystyle\rho(Tu,Tv)({\rm{Id}}+f\circ T)(w)+\rho(Tv,Tw)({\rm{Id}}+f\circ
T)(u)$ $\displaystyle+\rho(Tw,Tu)({\rm{Id}}+f\circ T)(v)+\Phi(Tu,Tv,Tw)$
$\displaystyle=$
$\displaystyle\rho(Tu,Tv)w+\rho(Tv,Tw)u+\rho(Tw,Tu)v+\Phi(Tu,Tv,Tw)$
$\displaystyle+\rho(Tu,Tv)f(Tw)+\rho(Tv,Tw)f(Tu)+\rho(Tw,Tu)f(Tv)$
$\displaystyle=$ $\displaystyle[u,v,w]_{T}+f([Tu,Tv,Tw]_{\mathfrak{g}})$
$\displaystyle=$ $\displaystyle[u,v,w]_{T}+fT([u,v,w]_{T})$ $\displaystyle=$
$\displaystyle({\rm{Id}}+f\circ T)([u,v,w]_{T}).$

Thus ${\rm{Id}}+f\circ T$ is an isomorphism of $3$-Lie algebras from
$(V,[\cdot,\cdot,\cdot]_{T})$ to $(V,[\cdot,\cdot,\cdot]_{T_{f}})$ . ∎

###  2.2. Cohomology of $\Phi$-twisted Rota-Baxter operators on $3$-Lie
algebras

In this subsection, we construct a representation of the $3$-Lie algebra
$(V,[\cdot,\cdot,\cdot]_{T})$ on the vector space $\mathfrak{g},$ and define
the cohomology of $\Phi$-twisted Rota-Baxter operators on 3-Lie algebras.

######  Lemma 2.14.

Let $T$ be a $\Phi$-twisted Rota-Baxter operator on a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ with respect to a
representation $(V;\rho)$. Define
$\varrho:\wedge^{2}V\rightarrow\mathfrak{gl}(\mathfrak{g})$ by

(10)
$\quad\varrho(u,v)(x)=[Tu,Tv,x]_{\mathfrak{g}}-T\Big{(}\rho(x,Tu)v+\rho(Tv,x)u+\Phi(Tu,Tv,x)\Big{)},\quad\forall
x\in\mathfrak{g},u,v\in V.$

Then $(\mathfrak{g};\varrho)$ is a representation of the $3$-Lie algebra
$(V,[\cdot,\cdot,\cdot]_{T})$.

###### Proof.

By a direct calculation using (1), (3)-(2.1), (7) and (9), for all $u_{i}\in
V,1\leq i\leq 4,x\in\mathfrak{g},$ we have

$\displaystyle\Big{(}\varrho(u_{1},u_{2})\varrho(u_{3},u_{4})-\varrho(u_{3},u_{4})\varrho(u_{1},u_{2})-\varrho([u_{1},u_{2},u_{3}]_{T},u_{4})+\varrho([u_{1},u_{2},u_{4}]_{T},u_{3})\Big{)}(x)$
$\displaystyle=$
$\displaystyle\varrho(u_{1},u_{2})\Big{(}[Tu_{3},Tu_{4},x]_{\mathfrak{g}}-T\rho(x,Tu_{3})u_{4}-T\rho(Tu_{4},x)u_{3}-T\Phi(Tu_{3},Tu_{4},x)\Big{)}$
$\displaystyle-\varrho(u_{3},u_{4})\Big{(}[Tu_{1},Tu_{2},x]_{\mathfrak{g}}-T\rho(x,Tu_{1})u_{2}-T\rho(Tu_{2},x)u_{1}-T\Phi(Tu_{1},Tu_{2},x)\Big{)}$
$\displaystyle+T\Big{(}\rho(x,T[u_{1},u_{2},u_{3}]_{T})u_{4}+\rho(Tu_{4},x)[u_{1},u_{2},u_{3}]_{T}+\Phi(T[u_{1},u_{2},u_{3}]_{T},Tu_{4},x)\Big{)}$
$\displaystyle-T\Big{(}\rho(x,T[u_{1},u_{2},u_{4}]_{T})u_{3}+\rho(Tu_{3},x)[u_{1},u_{2},u_{4}]_{T}+\Phi(T[u_{1},u_{2},u_{4}]_{T},Tu_{3},x)\Big{)}$
$\displaystyle-[T[u_{1},u_{2},u_{3}]_{T},Tu_{4},x]_{\mathfrak{g}}+[T[u_{1},u_{2},u_{4}]_{T},Tu_{3},x]_{\mathfrak{g}}$
$\displaystyle=$
$\displaystyle-[Tu_{1},Tu_{2},T\rho(x,Tu_{3})u_{4}]_{\mathfrak{g}}-[Tu_{1},Tu_{2},T\rho(Tu_{4},x)u_{3}]_{\mathfrak{g}}-[Tu_{1},Tu_{2},T\Phi(Tu_{3},Tu_{4},x)]_{\mathfrak{g}}$
$\displaystyle-T\Big{(}\rho([Tu_{3},Tu_{4},x]_{\mathfrak{g}},Tu_{1})u_{2}+\rho(Tu_{2},[Tu_{3},Tu_{4},x]_{\mathfrak{g}})u_{1}+\Phi(Tu_{1},Tu_{2},[Tu_{3},Tu_{4},x]_{\mathfrak{g}})\Big{)}$
$\displaystyle+T\Big{(}\rho(T\rho(x,Tu_{3})u_{4},Tu_{1})u_{2}+\rho(Tu_{2},T\rho(x,Tu_{3})u_{4})u_{1}+\Phi(Tu_{1},Tu_{2},T\rho(x,Tu_{3})u_{4})\Big{)}$
$\displaystyle+T\Big{(}\rho(T\rho(Tu_{4},x)u_{3},Tu_{1})u_{2}+\rho(Tu_{2},T\rho(Tu_{4},x)u_{3})u_{1}+\Phi(Tu_{1},Tu_{2},T\rho(Tu_{4},x)u_{3})\Big{)}$
$\displaystyle+T\Big{(}\rho(T\Phi(Tu_{3},Tu_{4},x),Tu_{1})u_{2}+\rho(Tu_{2},T\Phi(Tu_{3},Tu_{4},x))u_{1}+\Phi(Tu_{1},Tu_{2},T\Phi(Tu_{3},Tu_{4},x))\Big{)}$
$\displaystyle+[Tu_{3},Tu_{4},T\rho(x,Tu_{1})u_{2}]_{\mathfrak{g}}+[Tu_{3},Tu_{4},T\rho(Tu_{2},x)u_{1}]_{\mathfrak{g}}+[Tu_{3},Tu_{4},T\Phi(Tu_{1},Tu_{2},x)]_{\mathfrak{g}}$
$\displaystyle+T\Big{(}\rho([Tu_{1},Tu_{2},x]_{\mathfrak{g}},Tu_{3})u_{4}+\rho(Tu_{4},[Tu_{1},Tu_{2},x]_{\mathfrak{g}})u_{3}+\Phi(Tu_{3},Tu_{4},[Tu_{1},Tu_{2},x]_{\mathfrak{g}})\Big{)}$
$\displaystyle-T\Big{(}\rho(T\rho(x,Tu_{1})u_{2},Tu_{3})u_{4}+\rho(Tu_{4},T\rho(x,Tu_{1})u_{2})u_{3}+\Phi(Tu_{3},Tu_{4},T\rho(x,Tu_{1})u_{2})\Big{)}$
$\displaystyle-T\Big{(}\rho(T\rho(Tu_{2},x)u_{1},Tu_{3})u_{4}+\rho(Tu_{4},T\rho(Tu_{2},x)u_{1})u_{3}+\Phi(Tu_{3},Tu_{4},T\rho(Tu_{2},x)u_{1})\Big{)}$
$\displaystyle-T\Big{(}\rho(T\Phi(Tu_{1},Tu_{2},x),Tu_{3})u_{4}+\rho(Tu_{4},T\Phi(Tu_{1},Tu_{2},x))u_{1}+\Phi(Tu_{3},Tu_{4},T\Phi(Tu_{1},Tu_{2},x))\Big{)}$
$\displaystyle+T\Big{(}\rho(x,[Tu_{1},Tu_{2},Tu_{3}]_{\mathfrak{g}})u_{4}+\rho(Tu_{4},x)\rho(Tu_{1},Tu_{2})u_{3}+\rho(Tu_{4},x)\rho(Tu_{2},Tu_{3})u_{1}$
$\displaystyle+\rho(Tu_{4},x)\rho(Tu_{3},Tu_{1})u_{2}+\Phi([Tu_{1},Tu_{2},Tu_{3}]_{\mathfrak{g}},Tu_{4},x)\Big{)}$
$\displaystyle-T\Big{(}\rho(x,[Tu_{1},Tu_{2},Tu_{4}]_{\mathfrak{g}})u_{3}+\rho(Tu_{3},x)\rho(Tu_{1},Tu_{2})u_{4}+\rho(Tu_{3},x)\rho(Tu_{2},Tu_{4})u_{1}$
$\displaystyle+\rho(Tu_{3},x)\rho(Tu_{4},Tu_{1})u_{2}+\Phi([Tu_{1},Tu_{2},Tu_{4}]_{\mathfrak{g}},Tu_{3},x)\Big{)}$
$\displaystyle=$ $\displaystyle 0,$

and

$\displaystyle\Big{(}\varrho([u_{1},u_{2},u_{3}]_{T},u_{4})-\varrho(u_{1},u_{2})\varrho(u_{3},u_{4})-\varrho(u_{2},u_{3})\varrho(u_{1},u_{4})-\varrho(u_{3},u_{1})\varrho(u_{2},u_{4})\Big{)}(x)$
$\displaystyle=$
$\displaystyle-T\Big{(}\rho(x,T[u_{1},u_{2},u_{3}]_{T})u_{4}+\rho(Tu_{4},x)[u_{1},u_{2},u_{3}]_{T}+\Phi(T[u_{1},u_{2},u_{3}]_{T},Tu_{4},x)\Big{)}$
$\displaystyle-\varrho(u_{1},u_{2})\Big{(}[Tu_{3},Tu_{4},x]_{\mathfrak{g}}-T\rho(x,Tu_{3})u_{4}-T\rho(Tu_{4},x)u_{3}-T\Phi(Tu_{3},Tu_{4},x)\Big{)}$
$\displaystyle-\varrho(u_{2},u_{3})\Big{(}[Tu_{1},Tu_{4},x]_{\mathfrak{g}}-T\rho(x,Tu_{1})u_{4}-T\rho(Tu_{4},x)u_{1}-T\Phi(Tu_{1},Tu_{4},x)\Big{)}$
$\displaystyle-\varrho(u_{3},u_{1})\Big{(}[Tu_{2},Tu_{4},x]_{\mathfrak{g}}-T\rho(x,Tu_{2})u_{4}-T\rho(Tu_{4},x)u_{2}-T\Phi(Tu_{2},Tu_{4},x)\Big{)}$
$\displaystyle+[T[u_{1},u_{2},u_{3}]_{T},Tu_{4},x]_{\mathfrak{g}}$
$\displaystyle=$
$\displaystyle-T\Big{(}\rho(x,[Tu_{1},Tu_{2},Tu_{3}]_{\mathfrak{g}})u_{4}+\Phi([Tu_{1},Tu_{2},Tu_{3}]_{\mathfrak{g}},Tu_{4},x)\Big{)}$
$\displaystyle-T\rho(Tu_{4},x)\Big{(}\rho(Tu_{1},Tu_{2})u_{3}+\rho(Tu_{2},Tu_{3})u_{1}+\rho(Tu_{3},Tu_{1})u_{2}+\Phi(Tu_{1},Tu_{2},Tu_{3})\Big{)}$
$\displaystyle+[Tu_{1},Tu_{2},T\rho(x,Tu_{3})u_{4}+T\rho(Tu_{4},x)u_{3}+T\Phi(Tu_{3},Tu_{4},x)]_{\mathfrak{g}}$
$\displaystyle+T\Big{(}\rho([Tu_{3},Tu_{4},x]_{\mathfrak{g}},Tu_{1})u_{2}+\rho(Tu_{2},[Tu_{3},Tu_{4},x]_{\mathfrak{g}})u_{1}+\Phi(Tu_{1},Tu_{2},[Tu_{3},Tu_{4},x]_{\mathfrak{g}})\Big{)}$
$\displaystyle-T\Big{(}\rho(T\rho(x,Tu_{3})u_{4},Tu_{1})u_{2}+\rho(Tu_{2},T\rho(x,Tu_{3})u_{4})u_{1}+\Phi(Tu_{1},Tu_{2},T\rho(x,Tu_{3})u_{4})\Big{)}$
$\displaystyle-T\Big{(}\rho(T\rho(Tu_{4},x)u_{3},Tu_{1})u_{2}+\rho(Tu_{2},T\rho(Tu_{4},x)u_{3})u_{1}+\Phi(Tu_{1},Tu_{2},T\rho(Tu_{4},x)u_{3})\Big{)}$
$\displaystyle-T\Big{(}\rho(T\Phi(Tu_{3},Tu_{4},x),Tu_{1})u_{2}+\rho(Tu_{2},T\Phi(Tu_{3},Tu_{4},x))u_{1}+\Phi(Tu_{1},Tu_{2},T\Phi(Tu_{3},Tu_{4},x))\Big{)}$
$\displaystyle+[Tu_{2},Tu_{3},T\rho(x,Tu_{1})u_{4}+T\rho(Tu_{4},x)u_{1}+T\Phi(Tu_{1},Tu_{4},x)]_{\mathfrak{g}}$
$\displaystyle+T\Big{(}\rho([Tu_{1},Tu_{4},x]_{\mathfrak{g}},Tu_{2})u_{3}+\rho(Tu_{3},[Tu_{1},Tu_{4},x]_{\mathfrak{g}})u_{2}+\Phi(Tu_{2},Tu_{3},[Tu_{1},Tu_{4},x]_{\mathfrak{g}})\Big{)}$
$\displaystyle-T\Big{(}\rho(T\rho(x,Tu_{1})u_{4},Tu_{2})u_{3}+\rho(Tu_{3},T\rho(x,Tu_{1})u_{4})u_{2}+\Phi(Tu_{2},Tu_{3},T\rho(x,Tu_{1})u_{4})\Big{)}$
$\displaystyle-T\Big{(}\rho(T\rho(Tu_{4},x)u_{1},Tu_{2})u_{3}+\rho(Tu_{3},T\rho(Tu_{4},x)u_{1})u_{2}+\Phi(Tu_{2},Tu_{3},T\rho(Tu_{4},x)u_{1})\Big{)}$
$\displaystyle-T\Big{(}\rho(T\Phi(Tu_{1},Tu_{4},x),Tu_{2})u_{3}+\rho(Tu_{3},T\Phi(Tu_{1},Tu_{4},x))u_{2}+\Phi(Tu_{2},Tu_{3},T\Phi(Tu_{1},Tu_{4},x))\Big{)}$
$\displaystyle+[Tu_{3},Tu_{1},T\rho(x,Tu_{2})u_{4}+T\rho(Tu_{4},x)u_{2}+T\Phi(Tu_{2},Tu_{4},x)]_{\mathfrak{g}}$
$\displaystyle+T\Big{(}\rho([Tu_{2},Tu_{4},x]_{\mathfrak{g}},Tu_{3})u_{1}+\rho(Tu_{1},[Tu_{2},Tu_{4},x]_{\mathfrak{g}})u_{3}+\Phi(Tu_{3},Tu_{1},[Tu_{2},Tu_{4},x]_{\mathfrak{g}})\Big{)}$
$\displaystyle-T\Big{(}\rho(T\rho(x,Tu_{2})u_{4},Tu_{3})u_{1}+\rho(Tu_{1},T\rho(x,Tu_{2})u_{4})u_{3}+\Phi(Tu_{3},Tu_{1},T\rho(x,Tu_{2})u_{4})\Big{)}$
$\displaystyle-T\Big{(}\rho(T\rho(Tu_{4},x)u_{2},Tu_{3})u_{1}+\rho(Tu_{1},T\rho(Tu_{4},x)u_{2})u_{3}+\Phi(Tu_{3},Tu_{1},T\rho(Tu_{4},x)u_{2})\Big{)}$
$\displaystyle-T\Big{(}\rho(T\Phi(Tu_{2},Tu_{4},x),Tu_{3})u_{1}+\rho(Tu_{1},T\Phi(Tu_{2},Tu_{4},x))u_{3}+\Phi(Tu_{3},Tu_{1},T\Phi(Tu_{2},Tu_{4},x))\Big{)}$
$\displaystyle=$ $\displaystyle 0.$

Therefore, we deduce that $(\mathfrak{g};\varrho)$ is a representation of the
$3$-Lie algebra $(V,[\cdot,\cdot,\cdot]_{T})$. ∎

Let ${\rm d_{T}}:C_{\mathsf{3Lie}}^{n}(V;\mathfrak{g})\rightarrow
C_{\mathsf{3Lie}}^{n+1}(V;\mathfrak{g}),(n\geq 1)$ be the corresponding
coboundary operator of the $3$-Lie algebra $(V,[\cdot,\cdot,\cdot]_{T})$ with
coefficients in the representation $(\mathfrak{g};\varrho)$. More precisely,
${\rm d_{T}}:C_{\mathsf{3Lie}}^{n}(V;\mathfrak{g})\rightarrow
C_{\mathsf{3Lie}}^{n+1}(V;\mathfrak{g})$ is given by

$\displaystyle({\rm d_{T}}f)(\mathrm{U}_{1},\cdots,\mathrm{U}_{n},u_{n+1})$
$\displaystyle=$ $\displaystyle\sum_{1\leq j<k\leq
n}(-1)^{j}f(\mathrm{U}_{1},\cdots,\hat{\mathrm{U}_{j}},\cdots,\mathrm{U}_{k-1},[u_{j},v_{j},u_{k}]_{T}\wedge
v_{k}+u_{k}\wedge[u_{j},v_{j},v_{k}]_{T},\mathrm{U}_{k+1},\cdots,\mathrm{U}_{n},u_{n+1})$
$\displaystyle+\sum_{j=1}^{n}(-1)^{j}f(\mathrm{U}_{1},\cdots,\hat{\mathrm{U}_{j}},\cdots,\mathrm{U}_{n},[u_{j},v_{j},u_{n+1}]_{T})$
$\displaystyle+\sum_{j=1}^{n}(-1)^{j+1}\varrho(u_{j},v_{j})f(\mathrm{U}_{1},\cdots,\hat{\mathrm{U}_{j}},\cdots,\mathrm{U}_{n},u_{n+1})$
$\displaystyle+(-1)^{n+1}\Big{(}\varrho(v_{n},u_{n+1})f(\mathrm{U}_{1},\cdots,\mathrm{U}_{n-1},u_{n})+\varrho(u_{n+1},u_{n})f(\mathrm{U}_{1},\cdots,\mathrm{U}_{n+1},v_{n})\Big{)},$

for all $\mathrm{U}_{i}=u_{i}\wedge v_{i}\in\wedge^{2}V,~{}i=1,2,\cdots,n$ and
$u_{n+1}\in V.$

It is obvious that $f\in C_{\mathsf{3Lie}}^{1}(V;\mathfrak{g})$ is closed if
and only if

$\displaystyle[Tu_{1},Tu_{2},f(u_{3})]_{\mathfrak{g}}+[f(u_{1}),Tu_{2},Tu_{3}]_{\mathfrak{g}}+[Tu_{1},f(u_{2}),Tu_{3}]_{\mathfrak{g}}$
$\displaystyle=$ $\displaystyle
T\Big{(}\rho(f(u_{3}),Tu_{1})u_{2}+\rho(Tu_{2},f(u_{3}))u_{1}+\Phi(Tu_{1},Tu_{2},f(u_{3}))\Big{)}$
$\displaystyle+T\Big{(}\rho(f(u_{1}),Tu_{2})u_{3}+\rho(Tu_{3},f(u_{1}))u_{2}+\Phi(Tu_{2},Tu_{3},f(u_{1}))\Big{)}$
$\displaystyle+T\Big{(}\rho(f(u_{2}),Tu_{3})u_{1}+\rho(Tu_{1},f(u_{2}))u_{3}+\Phi(Tu_{3},Tu_{1},f(u_{2}))\Big{)}$
$\displaystyle+f\Big{(}\rho(Tu_{1},Tu_{2})u_{3}+\rho(Tu_{2},Tu_{3})u_{1}+\rho(Tu_{3},Tu_{1})u_{2}+\Phi(Tu_{1},Tu_{2},Tu_{3})\Big{)}.$

Define $\delta:\wedge^{2}\mathfrak{g}\rightarrow\mathrm{Hom}(V,\mathfrak{g})$
by

$\displaystyle\delta(\mathfrak{X})v=T\rho(\mathfrak{X})v-[\mathfrak{X},Tv]_{\mathfrak{g}}+T\Phi(\mathfrak{X},Tv),\quad\forall\mathfrak{X}\in\wedge^{2}\mathfrak{g},v\in
V.$

######  Proposition 2.15.

Let $T$ be a $\Phi$-twisted Rota-Baxter operator on a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ with respect to a
representation $(V;\rho).$ Then $\delta(\mathfrak{X})$ is a $1$-cocycle of the
$3$-Lie algebra $(V,[\cdot,\cdot,\cdot]_{T})$ with coefficients in
$(\mathfrak{g};\varrho).$

###### Proof.

For all $u_{1},u_{2},u_{3}\in V,$ we have

$\displaystyle({\rm d_{T}}\delta(\mathfrak{X}))(u_{1},u_{2},u_{3})$
$\displaystyle=$
$\displaystyle[Tu_{1},Tu_{2},T\rho(\mathfrak{X})u_{3}]_{\mathfrak{g}}-[Tu_{1},Tu_{2},[\mathfrak{X},Tu_{3}]_{\mathfrak{g}}]_{\mathfrak{g}}+[Tu_{1},Tu_{2},T\Phi(\mathfrak{X},Tu_{3})]_{\mathfrak{g}}$
$\displaystyle+[T\rho(\mathfrak{X})u_{1},Tu_{2},Tu_{3}]_{\mathfrak{g}}-[[\mathfrak{X},Tu_{1}]_{\mathfrak{g}},Tu_{2},Tu_{3},]_{\mathfrak{g}}+[T\Phi(\mathfrak{X},Tu_{1}),Tu_{2},Tu_{3}]_{\mathfrak{g}}$
$\displaystyle+[Tu_{1},T\rho(\mathfrak{X})u_{2},Tu_{3}]_{\mathfrak{g}}-[Tu_{1},[\mathfrak{X},Tu_{2}]_{\mathfrak{g}},Tu_{3}]_{\mathfrak{g}}+[Tu_{1},T\Phi(\mathfrak{X},Tu_{2}),Tu_{3}]_{\mathfrak{g}}$
$\displaystyle-T\rho(\mathfrak{X})\Big{(}\rho(Tu_{1},Tu_{2})u_{3}+\rho(Tu_{2},Tu_{3})u_{1}+\rho(Tu_{3},Tu_{1})u_{2}+\Phi(Tu_{1},Tu_{2},Tu_{3})\Big{)}$
$\displaystyle+[\mathfrak{X},[Tu_{1},Tu_{2},Tu_{3}]_{\mathfrak{g}}]_{\mathfrak{g}}-T\Phi(\mathfrak{X},[Tu_{1},Tu_{2},Tu_{3}]_{\mathfrak{g}})$
$\displaystyle-T\rho\Big{(}T\rho(\mathfrak{X})u_{3}-[\mathfrak{X},Tu_{3}]_{\mathfrak{g}}+T\Phi(\mathfrak{X},Tu_{3}),Tu_{1}\Big{)}u_{2}$
$\displaystyle-T\rho\Big{(}Tu_{2},T\rho(\mathfrak{X})u_{3}-[\mathfrak{X},Tu_{3}]_{\mathfrak{g}}+T\Phi(\mathfrak{X},Tu_{3})\Big{)}u_{1}$
$\displaystyle-T\Phi\Big{(}Tu_{1},Tu_{2},T\rho(\mathfrak{X})u_{3}-[\mathfrak{X},Tu_{3}]_{\mathfrak{g}}+T\Phi(\mathfrak{X},Tu_{3})\Big{)}$
$\displaystyle-T\rho\Big{(}T\rho(\mathfrak{X})u_{1}-[\mathfrak{X},Tu_{1}]_{\mathfrak{g}}+T\Phi(\mathfrak{X},Tu_{1}),Tu_{2}\Big{)}u_{3}$
$\displaystyle-T\rho\Big{(}Tu_{3},T\rho(\mathfrak{X})u_{1}-[\mathfrak{X},Tu_{1}]_{\mathfrak{g}}+T\Phi(\mathfrak{X},Tu_{1})\Big{)}u_{2}$
$\displaystyle-T\Phi\Big{(}Tu_{2},Tu_{3},T\rho(\mathfrak{X})u_{1}-[\mathfrak{X},Tu_{1}]_{\mathfrak{g}}+T\Phi(\mathfrak{X},Tu_{1})\Big{)}$
$\displaystyle-T\rho\Big{(}T\rho(\mathfrak{X})u_{2}-[\mathfrak{X},Tu_{2}]_{\mathfrak{g}}+T\Phi(\mathfrak{X},Tu_{2}),Tu_{3}\Big{)}u_{1}$
$\displaystyle-T\rho\Big{(}Tu_{1},T\rho(\mathfrak{X})u_{2}-[\mathfrak{X},Tu_{2}]_{\mathfrak{g}}+T\Phi(\mathfrak{X},Tu_{2})\Big{)}u_{3}$
$\displaystyle-T\Phi\Big{(}Tu_{3},Tu_{1},T\rho(\mathfrak{X})u_{2}-[\mathfrak{X},Tu_{2}]_{\mathfrak{g}}+T\Phi(\mathfrak{X},Tu_{2})\Big{)}$
$\displaystyle=$ $\displaystyle
T\Big{(}\rho(Tu_{1},Tu_{2})\rho(\mathfrak{X})u_{3}+\rho(Tu_{2},T\rho(\mathfrak{X})u_{3})u_{1}+\rho(T\rho(\mathfrak{X})u_{3},Tu_{1})u_{2}+\Phi(Tu_{1},Tu_{2},T\rho(\mathfrak{X})u_{3})\Big{)}$
$\displaystyle+T\Big{(}\rho(T\rho(\mathfrak{X})u_{1},Tu_{2})u_{3}+\rho(Tu_{2},Tu_{3})\rho(\mathfrak{X})u_{1}+\rho(Tu_{3},T\rho(\mathfrak{X})u_{1})u_{2}+\Phi(T\rho(\mathfrak{X})u_{1},Tu_{2},Tu_{3})\Big{)}$
$\displaystyle+T\Big{(}\rho(Tu_{1},T\rho(\mathfrak{X})u_{2})u_{3}+\rho(T\rho(\mathfrak{X})u_{2},Tu_{3})u_{1}+\rho(Tu_{3},Tu_{1})\rho(\mathfrak{X})u_{2}+\Phi(Tu_{1},T\rho(\mathfrak{X})u_{2},,Tu_{3})\Big{)}$
$\displaystyle+[Tu_{1},Tu_{2},T\Phi(\mathfrak{X},Tu_{3})]_{\mathfrak{g}}+[T\Phi(\mathfrak{X},Tu_{1}),Tu_{2},Tu_{3}]_{\mathfrak{g}}+[Tu_{1},T\Phi(\mathfrak{X},Tu_{2}),Tu_{3}]_{\mathfrak{g}}$
$\displaystyle-T\rho(\mathfrak{X})\Big{(}\rho(Tu_{1},Tu_{2})u_{3}+\rho(Tu_{2},Tu_{3})u_{1}+\rho(Tu_{3},Tu_{1})u_{2}+\Phi(Tu_{1},Tu_{2},Tu_{3})\Big{)}$
$\displaystyle-T\Big{(}\rho(T\rho(\mathfrak{X})u_{3},Tu_{1})u_{2}-\rho([\mathfrak{X},Tu_{3}]_{\mathfrak{g}},Tu_{1})u_{2}+\rho(T\Phi(\mathfrak{X},Tu_{3}),Tu_{1})u_{2}\Big{)}$
$\displaystyle-T\Big{(}\rho(Tu_{2},T\rho(\mathfrak{X})u_{3})u_{1}-\rho(Tu_{2},[\mathfrak{X},Tu_{3}]_{\mathfrak{g}})u_{1}+\rho(Tu_{2},T\Phi(\mathfrak{X},Tu_{3})u_{1})\Big{)}$
$\displaystyle-T\Big{(}\Phi(Tu_{1},Tu_{2},T\rho(\mathfrak{X})u_{3})-\Phi(Tu_{1},Tu_{2},[\mathfrak{X},Tu_{3}]_{\mathfrak{g}})+\Phi(Tu_{1},Tu_{2},T\Phi(\mathfrak{X},Tu_{3}))\Big{)}$
$\displaystyle-T\Big{(}\rho(T\rho(\mathfrak{X})u_{1},Tu_{2})u_{3}-\rho([\mathfrak{X},Tu_{1}]_{\mathfrak{g}},Tu_{2})u_{3}+\rho(T\Phi(\mathfrak{X},Tu_{1}),Tu_{2})u_{3}\Big{)}$
$\displaystyle-T\Big{(}\rho(Tu_{3},T\rho(\mathfrak{X})u_{1})u_{2}-\rho(Tu_{3},[\mathfrak{X},Tu_{1}]_{\mathfrak{g}})u_{2}+\rho(Tu_{3},T\Phi(\mathfrak{X},Tu_{1})u_{2})\Big{)}$
$\displaystyle-T\Big{(}\Phi(Tu_{2},Tu_{3},T\rho(\mathfrak{X})u_{1})-\Phi(Tu_{2},Tu_{3},[\mathfrak{X},Tu_{1}]_{\mathfrak{g}})+\Phi(Tu_{2},Tu_{3},T\Phi(\mathfrak{X},Tu_{1}))\Big{)}$
$\displaystyle-T\Big{(}\rho(T\rho(\mathfrak{X})u_{2},Tu_{3})u_{1}-\rho([\mathfrak{X},Tu_{2}]_{\mathfrak{g}},Tu_{3})u_{1}+\rho(T\Phi(\mathfrak{X},Tu_{2}),Tu_{3})u_{1}\Big{)}$
$\displaystyle-T\Big{(}\rho(Tu_{1},T\rho(\mathfrak{X})u_{2})u_{3}-\rho(Tu_{1},[\mathfrak{X},Tu_{2}]_{\mathfrak{g}})u_{3}+\rho(Tu_{1},T\Phi(\mathfrak{X},Tu_{2})u_{3})\Big{)}$
$\displaystyle-T\Big{(}\Phi(Tu_{3},Tu_{1},T\rho(\mathfrak{X})u_{2})-\Phi(Tu_{3},Tu_{1},[\mathfrak{X},Tu_{2}]_{\mathfrak{g}})+\Phi(Tu_{3},Tu_{1},T\Phi(\mathfrak{X},Tu_{2}))\Big{)}$
$\displaystyle-T\Phi(\mathfrak{X},[Tu_{1},Tu_{2},Tu_{3}]_{\mathfrak{g}})$
$\displaystyle=$ $\displaystyle
T\Big{(}\Phi(Tu_{1},Tu_{2},[\mathfrak{X},Tu_{3}]_{\mathfrak{g}})+\Phi(Tu_{2},Tu_{3},[\mathfrak{X},Tu_{1}]_{\mathfrak{g}})+\Phi(Tu_{3},Tu_{1},[\mathfrak{X},Tu_{2}]_{\mathfrak{g}})$
$\displaystyle+\rho(Tu_{1},Tu_{2})\Phi(\mathfrak{X},Tu_{3})+\rho(Tu_{2},Tu_{3})\Phi(\mathfrak{X},Tu_{1})+\rho(Tu_{3},Tu_{1})\Phi(\mathfrak{X},Tu_{2})$
$\displaystyle-\Phi(\mathfrak{X},[Tu_{1},Tu_{2},Tu_{3}]_{\mathfrak{g}})-\rho(\mathfrak{X})\Phi(Tu_{1},Tu_{2},Tu_{3})\Big{)}$
$\displaystyle=$ $\displaystyle 0.$

Thus, we deduce that ${\rm d_{T}}\delta(\mathfrak{X})=0.$ The proof is
finished. ∎

We now give the cohomology of $\Phi$-twisted Rota-Baxter operators on $3$-Lie
algebras.

Let $T$ be a $\Phi$-twisted Rota-Baxter operator on a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ with respect to a
representation $(V;\rho).$ Define the set of $n$-cochains by

(13) $\displaystyle
C_{T}^{n}(V;\mathfrak{g})=\left\\{\begin{array}[]{rcl}{}C_{\mathsf{3Lie}}^{n-1}(V;\mathfrak{g}),\quad
n\geq 2,\\\ {}\mathfrak{g}\wedge\mathfrak{g},\quad n=1.\end{array}\right.$

Define ${\partial}:C_{T}^{n}(V;\mathfrak{g})\rightarrow
C_{T}^{n+1}(V;\mathfrak{g})$ by

(16) $\displaystyle{\partial}=\left\\{\begin{array}[]{rcl}{}{\rm d_{T}},\quad
n\geq 2,\\\ {}\delta,\quad n=1.\end{array}\right.$

Then
$(\mathop{\oplus}\limits_{n=1}^{\infty}C_{T}^{n}(V;\mathfrak{g}),\partial)$ is
a cochain complex. Denote the set of $n$-cocycles by
$\mathcal{Z}^{n}_{T}(V;\mathfrak{g}),$ the set of $n$-coboundaries by
$\mathcal{B}^{n}_{T}(V;\mathfrak{g})$ and $n$-th cohomology group by

(17)
$\displaystyle\mathcal{H}^{n}_{T}(V;\mathfrak{g})=\mathcal{Z}^{n}(V;\mathfrak{g})/\mathcal{B}^{n}(V;\mathfrak{g}),\quad
n\geq 1.$

######  Definition 2.16.

The cohomology of the cochain complex
$(\mathop{\oplus}\limits_{n=1}^{\infty}C_{T}^{n}(V;\mathfrak{g}),\partial)$ is
taken to be the cohomology for the $\Phi$-twisted Rota-Baxter operator $T$.

###  2.3. Infinitesimal deformations of $\Phi$-twisted Rota-Baxter operators
on $3$-Lie algebras

Now we use the cohomology theory to characterize infinitesimal deformations of
$\Phi$-twisted Rota-Baxter operators.

######  Definition 2.17.

Let $T$ and $T^{\prime}$ be $\Phi$-twisted Rota-Baxter operators on a $3$-Lie
algebra $\mathfrak{g}$ with respect to a representation $(V;\rho).$ A
homomorphism of $\Phi$-twisted Rota-Baxter operators from $T$ to $T^{\prime}$
consists a $3$-Lie algebra homomorphism
$\phi:\mathfrak{g}\rightarrow\mathfrak{g}$ and a linear map $\psi:V\rightarrow
V$ such that

(18) $\displaystyle\phi\circ T$ $\displaystyle=$ $\displaystyle
T^{\prime}\circ\psi,$ (19) $\displaystyle\psi(\rho(x,y)u)$ $\displaystyle=$
$\displaystyle\rho(\phi(x),\phi(y))(\psi(u)),\quad\forall
x,y\in\mathfrak{g},u\in V,$ (20) $\displaystyle\psi\circ\Phi$ $\displaystyle=$
$\displaystyle\Phi\circ(\phi\otimes\phi\otimes\phi).$

In particular, if both $\phi$ and $\psi$ are invertible, $(\phi,\psi)$ is
called an isomorphism from $T$ to $T^{\prime}$.

######  Definition 2.18.

Let $T:V\rightarrow\mathfrak{g}$ be a $\Phi$-twisted Rota-Baxter operator on a
$3$-Lie algebra $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ with
respect to a representation $(V;\rho)$ and
$\mathfrak{T}:V\rightarrow\mathfrak{g}$ a linear map. If
$T_{t}=T+t\mathfrak{T}$ is still a $\Phi$-twisted Rota-Baxter operator on the
$3$-Lie algebra $\mathfrak{g}$ with respect to the representation $(V;\rho)$
for all $t$, we say that $\mathfrak{T}$ generates a one-parameter
infinitesimal deformation of $T$.

It is direct to deduce that $T_{t}=T+t\mathfrak{T}$ is a one-parameter
infinitesimal deformation of the $\Phi$-twisted Rota-Baxter operator $T$ if
and only if for any $u,v,w\in V,$

$\displaystyle[\mathfrak{T}u,Tv,Tw]_{\mathfrak{g}}+[Tu,\mathfrak{T}v,Tw]_{\mathfrak{g}}+[Tu,Tv,\mathfrak{T}w]_{\mathfrak{g}}$
$\displaystyle=$ $\displaystyle
T\Big{(}\rho(\mathfrak{T}w,Tu)v+\rho(Tv,\mathfrak{T}w)u+\rho(\mathfrak{T}u,Tv)w+\rho(Tw,\mathfrak{T}u)v+\rho(\mathfrak{T}v,Tw)u$
$\displaystyle+\rho(Tu,\mathfrak{T}v)w+\Phi(\mathfrak{T}u,Tv,Tw)+\Phi(Tu,\mathfrak{T}v,Tw)+\Phi(Tu,Tv,\mathfrak{T}w)\Big{)}$
$\displaystyle+\mathfrak{T}\Big{(}\rho(Tu,Tv)w+\rho(Tv,Tw)u+\rho(Tw,Tu)v+\Phi(Tu,Tv,Tw)\Big{)},$
$\displaystyle[\mathfrak{T}u,\mathfrak{T}v,T(w)]_{\mathfrak{g}}+[\mathfrak{T}v,\mathfrak{T}w,T(u)]_{\mathfrak{g}}+[\mathfrak{T}w,\mathfrak{T}u,T(v)]_{\mathfrak{g}}$
$\displaystyle=$
$\displaystyle\mathfrak{T}\Big{(}\rho(Tw,\mathfrak{T}u)v+\rho(\mathfrak{T}v,Tw)u+\rho(Tu,\mathfrak{T}v)w+\rho(\mathfrak{T}w,Tu)v+\rho(Tv,\mathfrak{T}w)u$
$\displaystyle+\rho(\mathfrak{T}u,Tv)w+\Phi(\mathfrak{T}u,Tv,Tw)+\Phi(Tu,\mathfrak{T}v,Tw)+\Phi(Tu,Tv,\mathfrak{T}w)\Big{)}$
$\displaystyle+T\Big{(}\rho(\mathfrak{T}u,\mathfrak{T}v)w+\rho(\mathfrak{T}v,\mathfrak{T}w)u+\rho(\mathfrak{T}w,\mathfrak{T}u)v$
$\displaystyle+\Phi(Tu,\mathfrak{T}v,\mathfrak{T}w)+\Phi(\mathfrak{T}u,Tv,\mathfrak{T}w)+\Phi(\mathfrak{T}u,\mathfrak{T}v,Tw)\Big{)},$
$\displaystyle[\mathfrak{T}u,\mathfrak{T}v,\mathfrak{T}w]_{\mathfrak{g}}$
$\displaystyle=$ $\displaystyle
T\Phi(\mathfrak{T}u,\mathfrak{T}v,\mathfrak{T}w)+\mathfrak{T}\Big{(}\rho(\mathfrak{T}u,\mathfrak{T}v)w+\rho(\mathfrak{T}v,\mathfrak{T}w)u+\rho(\mathfrak{T}w,\mathfrak{T}u)v\Big{)}$
$\displaystyle+\mathfrak{T}\Big{(}\Phi(Tu,\mathfrak{T}v,\mathfrak{T}w)+\Phi(\mathfrak{T}u,Tv,\mathfrak{T}w)+\Phi(\mathfrak{T}u,\mathfrak{T}v,Tw)\Big{)},$
(24)
$\displaystyle\mathfrak{T}\Phi(\mathfrak{T}u,\mathfrak{T}v,\mathfrak{T}w)=0.$

Note that (2.3) means that $\mathfrak{T}$ is a $1$-cocycle of the
$\Phi$-twisted Rota-Baxter operator $T$. Hence, $\mathfrak{T}$ defines a
cohomology class in $\mathcal{H}^{2}_{T}(V;\mathfrak{g})$.

######  Definition 2.19.

Let $T$ be a $\Phi$-twisted Rota-Baxter operator on a $3$-Lie algebra
$\mathfrak{g}$ with respect to a representation $(V;\rho).$ Two one-parameter
infinitesimal deformations $T^{1}_{t}=T+t\mathfrak{T}_{1}$ and
$T^{2}_{t}=T+t\mathfrak{T}_{2}$ are said to be equivalent if there exist
$\mathfrak{X}\in\mathfrak{g}\wedge\mathfrak{g}$ such that
$({\rm{Id}}_{\mathfrak{g}}+t\mathrm{ad}_{\mathfrak{X}},{\rm{Id}}_{V}+t\rho(\mathfrak{X})+t\Phi(\mathfrak{X},T))$
is a homomorphism from $T^{1}_{t}$ to $T^{2}_{t}$, where
$\Phi(\mathfrak{X},T):V\rightarrow V$ is defined by

$\Phi(\mathfrak{X},T)(u)=\Phi(\mathfrak{X},Tu),\quad\forall u\in V.$

In particular, a one-parameter infinitesimal deformation
$T_{t}=T+t\mathfrak{T}_{1}$ of a $\Phi$-twisted Rota-Baxter operator $T$ is
said to be trivial if there exist
$\mathfrak{X}\in\mathfrak{g}\wedge\mathfrak{g}$ such that
$({\rm{Id}}_{\mathfrak{g}}+t\mathrm{ad}_{\mathfrak{X}},{\rm{Id}}_{V}+t\rho(\mathfrak{X})+t\Phi(\mathfrak{X},T))$
is a homomorphism from $T_{t}$ to $T.$

Let
$({\rm{Id}}_{\mathfrak{g}}+t\mathrm{ad}_{\mathfrak{X}},{\rm{Id}}_{V}+t\rho(\mathfrak{X})+t\Phi(\mathfrak{X},T))$
be a homomorphism from $T^{1}_{t}$ to $T^{2}_{t}.$ By (18) we get,

$\quad({\rm{Id}}_{\mathfrak{g}}+t\mathrm{ad}_{\mathfrak{X}})(T+t\mathfrak{T}_{1})(u)=(T+t\mathfrak{T}_{2})({\rm{Id}}_{V}+t\rho(\mathfrak{X})+t\Phi(\mathfrak{X},T))(u),$

which implies

(27)
$\displaystyle\left\\{\begin{array}[]{rcl}{}\mathfrak{T}_{1}(u)-\mathfrak{T}_{2}(u)&=&T\rho(\mathfrak{X})u-[\mathfrak{X},Tu]_{\mathfrak{g}}+T\Phi(\mathfrak{X},Tu),\\\
{}[\mathfrak{X},\mathfrak{T}_{1}(u)]_{\mathfrak{g}}&=&\mathfrak{T}_{2}(\rho(\mathfrak{X})u+\Phi(\mathfrak{X},Tu)).\end{array}\right.$

Now we are ready to give the main result in this section.

######  Theorem 2.20.

Let $T$ be a $\Phi$-twisted Rota-Baxter operator on a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ with respect to a
representation $(V;\rho).$ If two one-parameter infinitesimal deformations
$T^{1}_{t}=T+t\mathfrak{T}_{1}$ and $T^{2}_{t}=T+t\mathfrak{T}_{2}$ are
equivalent, then $\mathfrak{T}_{1}$ and $\mathfrak{T}_{2}$ are in the same
cohomology class in $\mathcal{H}^{2}_{T}(V;\mathfrak{g})$.

###### Proof.

It is easy to see from the first condition of (27) that

$\displaystyle\mathfrak{T}_{1}(u)$ $\displaystyle=$
$\displaystyle\mathfrak{T}_{2}(u)+(\partial\mathfrak{X})(u),\quad\forall u\in
V,$

which implies that $\mathfrak{T}_{1}$ and $\mathfrak{T}_{2}$ are in the same
cohomology class. ∎

##  3\. NS-$3$-Lie algebras

In this section, we introduce the notion of an NS-$3$-Lie algebra. An
NS-$3$-Lie algebra gives rise to a 3-Lie algebra and a representation on
itself. We show that a twisted Rota-Baxter operator induces an NS-$3$-Lie
algebra. Thus NS-$3$-Lie algebras can be viewed as the underlying algebraic
structures of twisted Rota-Baxter operators on $3$-Lie algebras.

######  Definition 3.1.

Let $A$ be a vector space together with two linear maps
$\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot]:A\otimes A\otimes A\rightarrow A$
in which $[\cdot,\cdot,\cdot]$ is skew-symmetric. The triple
$(A,\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot])$ is called an NS-$3$-Lie
algebra if the following identities hold:

(28) $\displaystyle\\{x_{1},x_{2},x_{3}\\}$ $\displaystyle=$
$\displaystyle-\\{x_{2},x_{1},x_{3}\\},$
$\displaystyle\\{x_{1},x_{2},\\{x_{3},x_{4},x_{5}\\}\\}$ $\displaystyle=$
$\displaystyle\\{x_{3},x_{4},\\{x_{1},x_{2},x_{5}\\}\\}+\\{\left\llbracket
x_{1},x_{2},x_{3}\right\rrbracket,x_{4},x_{5}\\}$
$\displaystyle+\\{x_{3},\left\llbracket
x_{1},x_{2},x_{4}\right\rrbracket,x_{5}\\},$ (30)
$\displaystyle\\{\left\llbracket
x_{1},x_{2},x_{3}\right\rrbracket,x_{4},x_{5}\\}$ $\displaystyle=$
$\displaystyle\circlearrowleft_{x_{1},x_{2},x_{3}}\\{x_{1},x_{2},\\{x_{3},x_{4},x_{5}\\}\\},$
$\displaystyle[x_{1},x_{2},\left\llbracket x_{3},x_{4},x_{5}\right\rrbracket]$
$\displaystyle=$
$\displaystyle\circlearrowleft_{x_{3},x_{4},x_{5}}[x_{3},x_{4},\left\llbracket
x_{1},x_{2},x_{5}\right\rrbracket]-\\{x_{1},x_{2},[x_{3},x_{4},x_{5}]\\}$
$\displaystyle+\circlearrowleft_{x_{3},x_{4},x_{5}}\\{x_{3},x_{4},[x_{1},x_{2},x_{5}]\\},$

for all $x_{i}\in A,1\leq i\leq 5$ and
$\left\llbracket\cdot,\cdot,\cdot\right\rrbracket$ is defined by

(32) $\displaystyle\left\llbracket
x,y,z\right\rrbracket=\circlearrowleft_{x,y,z}\\{x,y,z\\}+[x,y,z],\quad\forall
x,y,z\in A,$

where
$\circlearrowleft_{x,y,z}\\{x,y,z\\}=\\{x,y,z\\}+\\{y,z,x\\}+\\{z,x,y\\}$.

######  Remark 3.2.

Let $(A,\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot])$ be an NS-$3$-Lie
algebra. On the one hand, if $\\{\cdot,\cdot,\cdot\\}=0,$ we get that
$(A,[\cdot,\cdot,\cdot])$ is a $3$-Lie algebra. On the other hand, if
$[\cdot,\cdot,\cdot]=0,$ then $(A,\\{\cdot,\cdot,\cdot\\})$ is a $3$-pre-Lie
algebra which was introduced in [1] in the study of $3$-Lie Yang-Baxter
equations. Thus, NS-$3$-Lie algebras are generalizations of both $3$-Lie
algebras and $3$-pre-Lie algebras.

######  Theorem 3.3.

Let $(A,\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot])$ be an NS-$3$-Lie
algebra. Then $(A,\left\llbracket\cdot,\cdot,\cdot\right\rrbracket)$ is a
$3$-Lie algebra which is called the sub-adjacent $3$-Lie algebra of
$(A,\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot])$ and denoted by $A^{c}$.

Moreover, $(A;L)$ is a representation of the $3$-Lie algebra $A^{c}$, where
the linear map $L:\otimes^{2}A\rightarrow\mathfrak{gl}(A)$ is defined by

$L(x,y)z=\\{x,y,z\\},\quad\forall~{}x,y,z\in A.$

###### Proof.

By (28), the $3$-commutator $\left\llbracket\cdot,\cdot,\cdot\right\rrbracket$
given by (32) is skew-symmetric.

For $x_{1},x_{2},x_{3},x_{4},x_{5}\in A,$ by (3.1), (30) and (3.1), we have

$\displaystyle\left\llbracket x_{1},x_{2},\left\llbracket
x_{3},x_{4},x_{5}\right\rrbracket\right\rrbracket-\left\llbracket\left\llbracket
x_{1},x_{2},x_{3}\right\rrbracket,x_{4},x_{5}\right\rrbracket-\left\llbracket
x_{3},\left\llbracket
x_{1},x_{2},x_{4}\right\rrbracket,x_{5}\right\rrbracket-\left\llbracket
x_{3},x_{4},\left\llbracket x_{1},x_{2},x_{5}\right\rrbracket\right\rrbracket$
$\displaystyle=$
$\displaystyle\\{x_{1},x_{2},\\{x_{3},x_{4},x_{5}\\}\\}+\\{x_{1},x_{2},\\{x_{4},x_{5},x_{3}\\}\\}+\\{x_{1},x_{2},\\{x_{5},x_{3},x_{4}\\}\\}+\\{x_{1},x_{2},[x_{3},x_{4},x_{5}]\\}$
$\displaystyle+\\{\left\llbracket
x_{3},x_{4},x_{5}\right\rrbracket,x_{1},x_{2}\\}+\\{x_{2},\left\llbracket
x_{3},x_{4},x_{5}\right\rrbracket,x_{1}\\}+[x_{1},x_{2},\left\llbracket
x_{3},x_{4},x_{5}\right\rrbracket]$
$\displaystyle-\\{x_{4},x_{5},\\{x_{1},x_{2},x_{3}\\}\\}-\\{x_{4},x_{5},\\{x_{2},x_{3},x_{1}\\}\\}-\\{x_{4},x_{5},\\{x_{3},x_{1},x_{2}\\}\\}-\\{x_{4},x_{5},[x_{1},x_{2},x_{3}]\\}$
$\displaystyle-\\{\left\llbracket
x_{1},x_{2},x_{3}\right\rrbracket,x_{4},x_{5}\\}-\\{x_{5},\left\llbracket
x_{1},x_{2},x_{3}\right\rrbracket,x_{4}\\}-[\left\llbracket
x_{1},x_{2},x_{3}\right\rrbracket,x_{4},x_{5}]$
$\displaystyle-\\{x_{5},x_{3},\\{x_{1},x_{2},x_{4}\\}\\}-\\{x_{5},x_{3},\\{x_{2},x_{4},x_{1}\\}\\}-\\{x_{5},x_{3},\\{x_{4},x_{1},x_{2}\\}\\}-\\{x_{5},x_{3},[x_{1},x_{2},x_{4}]\\}$
$\displaystyle-\\{x_{3},\left\llbracket
x_{1},x_{2},x_{4}\right\rrbracket,x_{5}\\}-\\{\left\llbracket
x_{1},x_{2},x_{4}\right\rrbracket,x_{3},x_{5}\\}-[x_{3},\left\llbracket
x_{1},x_{2},x_{4}\right\rrbracket,x_{5}]$
$\displaystyle-\\{x_{3},x_{4},\\{x_{1},x_{2},x_{5}\\}\\}-\\{x_{3},x_{4},\\{x_{5},x_{1},x_{2}\\}\\}-\\{x_{3},x_{4},\\{x_{2},x_{5},x_{1}\\}\\}-\\{x_{3},x_{4},[x_{1},x_{2},x_{5}]\\}$
$\displaystyle-\\{\left\llbracket
x_{1},x_{2},x_{5}\right\rrbracket,x_{3},x_{4}\\}-\\{x_{4},\left\llbracket
x_{1},x_{2},x_{5}\right\rrbracket,x_{3}\\}-[x_{3},x_{4},\left\llbracket
x_{1},x_{2},x_{5}\right\rrbracket]$ $\displaystyle=$ $\displaystyle 0,$

which implies that $(A,\left\llbracket\cdot,\cdot,\cdot\right\rrbracket)$ is a
$3$-Lie algebra.

By (3.1) and (30), we have

$\displaystyle[L(x_{1},x_{2}),L(x_{3},x_{4})](x_{5})$ $\displaystyle=$
$\displaystyle L(\left\llbracket
x_{1},x_{2},x_{3}\right\rrbracket,x_{4})(x_{5})+L(x_{3},\left\llbracket
x_{1},x_{2},x_{4}\right\rrbracket)(x_{5}),$ $\displaystyle L(\left\llbracket
x_{1},x_{2},x_{3}\right\rrbracket,x_{4})(x_{5})$ $\displaystyle=$
$\displaystyle\Big{(}L(x_{1},x_{2})L(x_{3},x_{4})+L(x_{2},x_{3})L(x_{1},x_{4})+L(x_{3},x_{1})L(x_{2},x_{4})\Big{)}(x_{5}).$

Therefore, $(A;L)$ is a representation the sub-adjacent $3$-Lie algebra
$A^{c}.$ ∎

The following results illustrate that NS-$3$-Lie algebras can be viewed as the
underlying algebraic structures of $\Phi$-twisted Rota-Baxter operators on
$3$-Lie algebras.

######  Theorem 3.4.

Let $T:V\rightarrow\mathfrak{g}$ be a $\Phi$-twisted Rota-Baxter operator on a
$3$-Lie algebra $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ with
respect to a representation $(V;\rho)$. Then

(33) $\displaystyle\qquad\\{u_{1},u_{2},u_{3}\\}=\rho(Tu_{1},Tu_{2})u_{3}\quad
and\quad[u_{1},u_{2},u_{3}]=\Phi(Tu_{1},Tu_{2},Tu_{3}),\quad\forall
u_{1},u_{2},u_{3}\in V,$

defines an NS-$3$-Lie algebra structure on $V$.

###### Proof.

For any $u_{1},u_{2},u_{3}\in V,$ it is obvious that

$\displaystyle\\{u_{1},u_{2},u_{3}\\}=\rho(Tu_{1},Tu_{2})u_{3}=-\rho(Tu_{2},Tu_{1})u_{3}=-\\{u_{2},u_{1},u_{3}\\}.$

Furthermore, for $u_{1},u_{2},u_{3},u_{4},u_{5}\in V$, by (3), (7) and (32),
we have

$\displaystyle\\{u_{1},u_{2},\\{u_{3},u_{4},u_{5}\\}\\}-\\{u_{3},u_{4},\\{u_{1},u_{2},u_{5}\\}\\}-\\{\left\llbracket
u_{1},u_{2},u_{3}\right\rrbracket,u_{4},u_{5}\\}-\\{u_{3},\left\llbracket
u_{1},u_{2},u_{4}\right\rrbracket,u_{5}\\}$ $\displaystyle=$
$\displaystyle\\{u_{1},u_{2},\rho(Tu_{3},Tu_{4})u_{5}\\}-\\{u_{3},u_{4},\rho(Tu_{1},Tu_{2})u_{5}\\}$
$\displaystyle-\\{\rho(Tu_{1},Tu_{2})u_{3}+\rho(Tu_{2},Tu_{3})u_{1}+\rho(Tu_{3},Tu_{1})u_{2}+\Phi(Tu_{1},Tu_{2},Tu_{3}),u_{4},u_{5}\\}$
$\displaystyle-\\{u_{3},\rho(Tu_{1},Tu_{2})u_{4}+\rho(Tu_{2},Tu_{4})u_{1}+\rho(Tu_{4},Tu_{1})u_{2}+\Phi(Tu_{1},Tu_{2},Tu_{4}),u_{5}\\}$
$\displaystyle=$
$\displaystyle\rho(Tu_{1},Tu_{2})\rho(Tu_{3},Tu_{4})u_{5}-\rho(Tu_{3},Tu_{4})\rho(Tu_{1},Tu_{2})u_{5}$
$\displaystyle-\rho\Big{(}T\rho(Tu_{1},Tu_{2})u_{3}+T\rho(Tu_{2},Tu_{3})u_{1}+T\rho(Tu_{3},Tu_{1})u_{2}+T\Phi(Tu_{1},Tu_{2},Tu_{3}),Tu_{4}\Big{)}u_{5}$
$\displaystyle-\rho\Big{(}Tu_{3},T\rho(Tu_{1},Tu_{2})u_{4}+T\rho(Tu_{2},Tu_{4})u_{1}+T\rho(Tu_{4},Tu_{1})u_{2}+T\Phi(Tu_{1},Tu_{2},Tu_{4})\Big{)}u_{5}$
$\displaystyle=$ $\displaystyle 0.$

This implies that (3.1) in Definition 3.1 holds. Similarly, for (30), by (4),
we can verify that

$\displaystyle\\{\left\llbracket
u_{1},u_{2},u_{3}\right\rrbracket,u_{4},u_{5}\\}-\circlearrowleft_{u_{1},u_{2},u_{3}}\\{u_{1},u_{2},\\{u_{3},u_{4},u_{5}\\}\\}$
$\displaystyle=$
$\displaystyle\rho([Tu_{1},Tu_{2},Tu_{3}]_{\mathfrak{g}},Tu_{4})u_{5}-\rho(Tu_{1},Tu_{2})\rho(Tu_{3},Tu_{4})u_{5}$
$\displaystyle-\rho(Tu_{2},Tu_{3})\rho(Tu_{1},Tu_{4})u_{5}-\rho(Tu_{3},Tu_{1})\rho(Tu_{2},Tu_{4})u_{5}$
$\displaystyle=$ $\displaystyle 0.$

Moreover, since $\Phi$ is a $2$-cocycle of $\mathfrak{g}$ with coefficients in
$V,$ we observe that

$\displaystyle[u_{1},u_{2},\left\llbracket
u_{3},u_{4},u_{5}\right\rrbracket]-\circlearrowleft_{u_{3},u_{4},u_{5}}[u_{3},u_{4},\left\llbracket
u_{1},u_{2},u_{5}\right\rrbracket]+\\{u_{1},u_{2},[u_{3},u_{4},u_{5}]\\}$
$\displaystyle-\circlearrowleft_{u_{3},u_{4},u_{5}}\\{u_{3},u_{4},[u_{1},u_{2},u_{5}]\\}$
$\displaystyle=$
$\displaystyle\Phi(Tu_{1},Tu_{2},[Tu_{3},Tu_{4},Tu_{5}]_{\mathfrak{g}})-\Phi([Tu_{1},Tu_{2},Tu_{3}]_{\mathfrak{g}},Tu_{4},Tu_{5})$
$\displaystyle-\Phi(Tu_{3},[Tu_{1},Tu_{2},Tu_{4}]_{\mathfrak{g}},Tu_{5})-\Phi(Tu_{3},Tu_{4},[Tu_{1},Tu_{2},Tu_{5}]_{\mathfrak{g}})$
$\displaystyle+[Tu_{1},Tu_{2},\Phi(Tu_{3},Tu_{4},Tu_{5})]_{\mathfrak{g}}-[Tu_{4},Tu_{5},\Phi(Tu_{1},Tu_{2},Tu_{3})]_{\mathfrak{g}}$
$\displaystyle-[Tu_{3},Tu_{4},\Phi(Tu_{1},Tu_{2},Tu_{5})]_{\mathfrak{g}}-[Tu_{5},Tu_{3},\Phi(Tu_{1},Tu_{2},Tu_{4})]_{\mathfrak{g}}$
$\displaystyle=$ $\displaystyle 0,$

which implies that (3.1) holds. Hence
$(V,\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot])$ is an NS-$3$-Lie algebra. ∎

Note that the subadjacent 3-Lie algebra of the above NS-3-Lie algebra
$(V,\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot])$ is exactly the 3-Lie algebra
given in Corollary 2.9.

######  Definition 3.5.

A homomorphism from an NS-$3$-Lie algebra
$(A,\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot])$ to
$(A^{\prime},\\{\cdot,\cdot,\cdot\\}^{\prime},[\cdot,\cdot,\cdot]^{\prime})$
is a linear map $\psi:A\,\rightarrow\,A^{\prime}$ satisfying

(34) $\displaystyle\psi(\\{x,y,z\\})=\\{\psi(x),\psi(y),\psi(z)\\}^{\prime},$
(35) $\displaystyle\psi([x,y,z])=[\psi(x),\psi(y),\psi(z)]^{\prime},$

for all $x,y,z\in A.$

######  Proposition 3.6.

Let $T$ and $T^{\prime}$ be $\Phi$-twisted Rota-Baxter operators on a $3$-Lie
algebra $\mathfrak{g}$ with respect to a representation $(V;\rho),$ and
$(V,\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot])$,
$(V,\\{\cdot,\cdot,\cdot\\}^{\prime},[\cdot,\cdot,\cdot]^{\prime})$ be the
induced NS-3-Lie algebras. Let $(\phi,\psi)$ be a homomorphism from $T$ to
$T^{\prime}$. Then $\psi$ is a homomorphism from the NS-3-Lie algebra
$(V,\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot])$ to
$(V,\\{\cdot,\cdot,\cdot\\}^{\prime},[\cdot,\cdot,\cdot]^{\prime})$.

###### Proof.

For all $u,v,w\in V,$ by (18)-(20) and (33), we have

$\displaystyle\psi(\\{u,v,w\\})$ $\displaystyle=$
$\displaystyle\psi(\rho(Tu,Tv)w)=\rho(\phi(Tu),\phi(Tv))\psi(w)=\rho(T^{\prime}\psi(u),T^{\prime}\psi(v))\psi(w)$
$\displaystyle=$ $\displaystyle\\{\psi(u),\psi(v),\psi(w)\\}^{\prime},$
$\displaystyle\psi([u,v,w])$ $\displaystyle=$
$\displaystyle\psi(\Phi(Tu,Tv,Tw))=\Phi(\phi(Tu),\phi(Tv),\phi(Tw))=\Phi(T^{\prime}\psi(u),T^{\prime}\psi(v),T^{\prime}\psi(w))$
$\displaystyle=$ $\displaystyle[\psi(u),\psi(v),\psi(w)]^{\prime},$

which implies that $\psi$ is a homomorphism between the induced NS-3-Lie
algebras. ∎

Thus, Theorem 3.4 can be enhanced to a functor from the category of
$\Phi$-twisted Rota-Baxter operators on a $3$-Lie algebra $\mathfrak{g}$ with
respect to a representation $(V;\rho)$ to the category of NS-3-Lie algebras.

##  4\. Nijenhuis operators and Reynolds operators on $3$-Lie algebras

In this section, we study two special classes of twisted Rota-Baxter operators
on 3-Lie algebras which are provided by Nijenhuis operators and Reynolds
operators on $3$-Lie algebras.

###  4.1. Nijenhuis operators on $3$-Lie algebras

In this subsection, we show that a Nijenhuis operator on a $3$-Lie algebra
gives rise to a twisted Rota-Baxter operator on a 3-Lie algebra.

Recall from [23] that a Nijenhuis operator on a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ is a linear map
$N:\mathfrak{g}\rightarrow\mathfrak{g}$ satisfying

$\displaystyle[Nx,Ny,Nz]_{\mathfrak{g}}$ $\displaystyle=$ $\displaystyle
N\Big{(}[Nx,Ny,z]_{\mathfrak{g}}+[x,Ny,Nz]_{\mathfrak{g}}+[Nx,y,Nz]_{\mathfrak{g}}$
$\displaystyle-N[Nx,y,z]_{\mathfrak{g}}-N[x,Ny,z]_{\mathfrak{g}}-N[x,y,Nz]_{\mathfrak{g}}$
$\displaystyle+N^{2}[x,y,z]_{\mathfrak{g}}\Big{)},\quad\forall
x,y,z\in\mathfrak{g}.$

A Nijenhuis operator endows the vector space $\mathfrak{g}$ a new $3$-Lie
bracket $[\cdot,\cdot,\cdot]_{N}$, which is defined by

$\displaystyle[x,y,z]_{N}$ $\displaystyle=$
$\displaystyle[Nx,Ny,z]_{\mathfrak{g}}+[x,Ny,Nz]_{\mathfrak{g}}+[Nx,y,Nz]_{\mathfrak{g}}$
$\displaystyle-N[Nx,y,z]_{\mathfrak{g}}-N[x,Ny,z]_{\mathfrak{g}}-N[x,y,Nz]_{\mathfrak{g}}+N^{2}[x,y,z]_{\mathfrak{g}},\quad\forall
x,y,z\in\mathfrak{g}.$

The 3-Lie algebra $(\mathfrak{g},[\cdot,\cdot,\cdot]_{N})$ will be called the
deformed $3$-Lie algebra, and denoted by $\mathfrak{g}_{N}.$ It is obvious
that $N$ is a homomorphism from the deformed $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{N})$ to
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$.

######  Lemma 4.1.

Let $N$ be a Nijenhuis operator on a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$. Define
$\rho_{N}:\wedge^{2}\mathfrak{g}_{N}\rightarrow\mathfrak{gl}(\mathfrak{g})$ by

(38)
$\displaystyle\rho_{N}(x,y)z=[Nx,Ny,z]_{\mathfrak{g}},\quad\forall~{}x,y\in\mathfrak{g}_{N},~{}z\in\mathfrak{g}.$

Then $(\mathfrak{g};\rho_{N})$ is a representation of the deformed $3$-Lie
algebra $\mathfrak{g}_{N}.$

###### Proof.

For all $x_{1},x_{2},x_{3},x_{4}\in\mathfrak{g}_{N},x\in\mathfrak{g},$ by (1),
(4.1) and (4.1), we have

$\displaystyle\Big{(}\rho_{N}(x_{1},x_{2})\rho_{N}(x_{3},x_{4})-\rho_{N}(x_{3},x_{4})\rho_{N}(x_{1},x_{2})-\rho_{N}([x_{1},x_{2},x_{3}]_{N},x_{4})-\rho_{N}(x_{3},[x_{1},x_{2},x_{4}]_{N})\Big{)}(x)$
$\displaystyle=$
$\displaystyle\rho_{N}(x_{1},x_{2})[Nx_{3},Nx_{4},x]_{\mathfrak{g}}-\rho_{N}(x_{3},x_{4})[Nx_{1},Nx_{2},x]_{\mathfrak{g}}$
$\displaystyle-[N[x_{1},x_{2},x_{3}]_{N},Nx_{4},x]_{\mathfrak{g}}-[Nx_{3},N[x_{1},x_{2},x_{4}]_{N},x]_{\mathfrak{g}}$
$\displaystyle=$
$\displaystyle[Nx_{1},Nx_{2},[Nx_{3},Nx_{4},x]_{\mathfrak{g}}]_{\mathfrak{g}}-[Nx_{3},Nx_{4},[Nx_{1},Nx_{2},x]_{\mathfrak{g}}]_{\mathfrak{g}}$
$\displaystyle-[[Nx_{1},Nx_{2},Nx_{3}]_{\mathfrak{g}},Nx_{4},x]_{\mathfrak{g}}-[Nx_{3},[Nx_{1},Nx_{2},Nx_{4}]_{\mathfrak{g}},x]_{\mathfrak{g}}$
$\displaystyle=$ $\displaystyle 0,$

which implies that (3) in Definition 2.2 holds.

Similarly, we can show that (4) also holds. Thus, $(\mathfrak{g};\rho_{N})$ is
a representation of the deformed $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{N})$. ∎

######  Theorem 4.2.

Let $N$ be a Nijenhuis operator on a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$. Define the map
$\Phi:\wedge^{3}\mathfrak{g}_{N}\rightarrow\mathfrak{g}$ by

(39)
$\displaystyle\Phi(x,y,z)=-N([Nx,y,z]_{\mathfrak{g}}+[x,Ny,z]_{\mathfrak{g}}+[x,y,Nz]_{\mathfrak{g}})+N^{2}[x,y,z]_{\mathfrak{g}},\quad\forall
x,y,z\in\mathfrak{g}_{N}.$

Then $\Phi$ is a $2$-cocycle of $\mathfrak{g}_{N}$ with coefficients in
$(\mathfrak{g};\rho_{N}),$ and the identity map
${\rm{Id}}:\mathfrak{g}\rightarrow\mathfrak{g}_{N}$ is a $\Phi$-twisted Rota-
Baxter operator on $\mathfrak{g}_{N}$ with respect to the representation
$(\mathfrak{g};\rho_{N}).$

###### Proof.

For all $x_{1},x_{2},x_{3},x_{4},x_{5}\in\mathfrak{g}_{N},$ by a direct
calculation, we have

$\displaystyle(\partial\Phi)(x_{1},x_{2},x_{3},x_{4},x_{5})$ $\displaystyle=$
$\displaystyle\rho_{N}(x_{1},x_{2})\Phi(x_{3},x_{4},x_{5})-\rho_{N}(x_{3},x_{4})\Phi(x_{1},x_{2},x_{5})-\rho_{N}(x_{4},x_{5})\Phi(x_{1},x_{2},x_{3})-\rho_{N}(x_{5},x_{3})\Phi(x_{1},x_{2},x_{4})$
$\displaystyle\Phi([x_{3},x_{4},x_{5}]_{N},x_{1},x_{2})-\Phi([x_{1},x_{2},x_{3}]_{N},x_{4},x_{5})-\Phi([x_{1},x_{2},x_{4}]_{N},x_{5},x_{3})-\Phi([x_{1},x_{2},x_{5}]_{N},x_{3},x_{4})$
$\displaystyle=$
$\displaystyle[Nx_{1},Nx_{2},N^{2}[x_{3},x_{4},x_{5}]_{\mathfrak{g}}-N[Nx_{3},x_{4},x_{5}]_{\mathfrak{g}}-N[x_{3},Nx_{4},x_{5}]_{\mathfrak{g}}-N[x_{3},x_{4},Nx_{5}]_{\mathfrak{g}}]_{\mathfrak{g}}$
$\displaystyle+[Nx_{3},Nx_{4},N[Nx_{1},x_{2},x_{5}]_{\mathfrak{g}}+N[x_{1},Nx_{2},x_{5}]_{\mathfrak{g}}+N[x_{1},x_{2},Nx_{5}]_{\mathfrak{g}}-N^{2}[x_{1},x_{2},x_{5}]_{\mathfrak{g}}]_{\mathfrak{g}}$
$\displaystyle+[Nx_{4},Nx_{5},N[Nx_{1},x_{2},x_{3}]_{\mathfrak{g}}+N[x_{1},Nx_{2},x_{3}]_{\mathfrak{g}}+N[x_{1},x_{2},Nx_{3}]_{\mathfrak{g}}-N^{2}[x_{1},x_{2},x_{3}]_{\mathfrak{g}}]_{\mathfrak{g}}$
$\displaystyle+[Nx_{5},Nx_{3},N[Nx_{1},x_{2},x_{4}]_{\mathfrak{g}}+N[x_{1},Nx_{2},x_{4}]_{\mathfrak{g}}+N[x_{1},x_{2},Nx_{4}]_{\mathfrak{g}}-N^{2}[x_{1},x_{2},x_{4}]_{\mathfrak{g}}]_{\mathfrak{g}}$
$\displaystyle+N\Big{(}N[[x_{3},x_{4},x_{5}]_{N},x_{1},x_{2}]_{\mathfrak{g}}-[N[x_{3},x_{4},x_{5}]_{N},x_{1},x_{2}]_{\mathfrak{g}}-[[x_{3},x_{4},x_{5}]_{N},Nx_{1},x_{2}]_{\mathfrak{g}}-[[x_{3},x_{4},x_{5}]_{N},x_{1},Nx_{2}]_{\mathfrak{g}}\Big{)}$
$\displaystyle+N\Big{(}[N[x_{1},x_{2},x_{3}]_{N},x_{4},x_{5}]_{\mathfrak{g}}+[[x_{1},x_{2},x_{3}]_{N},Nx_{4},x_{5}]_{\mathfrak{g}}+[[x_{1},x_{2},x_{3}]_{N},x_{4},Nx_{5}]_{\mathfrak{g}}-N[[x_{1},x_{2},x_{3}]_{N},x_{4},x_{5}]_{\mathfrak{g}}\Big{)}$
$\displaystyle+N\Big{(}[N[x_{1},x_{2},x_{4}]_{N},x_{5},x_{3}]_{\mathfrak{g}}+[[x_{1},x_{2},x_{4}]_{N},Nx_{5},x_{3}]_{\mathfrak{g}}+[[x_{1},x_{2},x_{4}]_{N},x_{5},Nx_{3}]_{\mathfrak{g}}-N[[x_{1},x_{2},x_{4}]_{N},x_{5},x_{3}]_{\mathfrak{g}}\Big{)}$
$\displaystyle+N\Big{(}[N[x_{1},x_{2},x_{5}]_{N},x_{3},x_{4}]_{\mathfrak{g}}+[[x_{1},x_{2},x_{5}]_{N},Nx_{3},x_{4}]_{\mathfrak{g}}+[[x_{1},x_{2},x_{5}]_{N},x_{3},Nx_{4}]_{\mathfrak{g}}-N[[x_{1},x_{2},x_{5}]_{N},x_{3},x_{4}]_{\mathfrak{g}}\Big{)}$
$\displaystyle=$ $\displaystyle 0.$

Thus, we deduce that $\Phi$ is a $2$-cocycle of $\mathfrak{g}_{N}$ with
coefficients in $(\mathfrak{g};\rho_{N}).$

Moreover, by (38) and (39), it is easy to prove that (7) is equivalent to
(4.1), which implies that the identity map
${\rm{Id}}:\mathfrak{g}\rightarrow\mathfrak{g}_{N}$ is a $\Phi$-twisted Rota-
Baxter operator. The proof is finished. ∎

######  Corollary 4.3.

Let $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ be a $3$-Lie algebra
and $N:\mathfrak{g}\rightarrow\mathfrak{g}$ be a Nijenhuis operator. Then
there exists an NS-$3$-Lie algebra on $\mathfrak{g}$ given by

(40) $\displaystyle\\{x,y,z\\}$ $\displaystyle=$
$\displaystyle[Nx,Ny,z]_{\mathfrak{g}},$ (41) $\displaystyle~{}[x,y,z]$
$\displaystyle=$
$\displaystyle-N([Nx,y,z]_{\mathfrak{g}}+[x,Ny,z]_{\mathfrak{g}}+[x,y,Nz]_{\mathfrak{g}})+N^{2}[x,y,z]_{\mathfrak{g}},\quad\forall
x,y,z\in\mathfrak{g}.$

###### Proof.

By Theorem 4.2, the identity map
${\rm{Id}}:\mathfrak{g}\rightarrow\mathfrak{g}_{N}$ is a $\Phi$-twisted Rota-
Baxter operator on $\mathfrak{g}_{N}$ with respect to the representation
$(\mathfrak{g};\rho_{N})$. By Theorem 3.4, a $\Phi$-twisted Rota-Baxter
operator on a $3$-Lie algebra can induce an NS-$3$-Lie algebra. Hence the
conclusion follows. ∎

######  Example 4.4.

Consider the $3$-dimensional $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ given with respect to a
basis $\\{e_{1},e_{2},e_{3}\\}$ by

$[e_{1},e_{2},e_{3}]_{\mathfrak{g}}=e_{1}.$

Thanks to Theorem 3.10 in [23], any linear transformation $N$ on
$\mathfrak{g}$ is a Nijenhuis operator. Suppose

$\displaystyle N=(a_{ij})_{3\times
3}=\left(\begin{array}[]{ccc}a_{11}&a_{12}&a_{13}\\\ a_{21}&a_{22}&a_{23}\\\
a_{31}&a_{32}&a_{33}\end{array}\right).$

By Corollary 4.3, $(\mathfrak{g},\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot])$
is a $3$-dimensional NS-$3$-Lie algebra, the brackets
$\\{\cdot,\cdot,\cdot\\}:\otimes^{3}\mathfrak{g}\rightarrow\mathfrak{g}$ and
$[\cdot,\cdot,\cdot]:\wedge^{3}\mathfrak{g}\rightarrow\mathfrak{g}$ are given
by

$\displaystyle~{}\\{e_{i},e_{j},e_{k}\\}$ $\displaystyle=$
$\displaystyle(-1)^{\tau(ijk)}M_{kk}e_{1},\quad i\neq j\neq k,$
$\displaystyle~{}\\{e_{1},e_{2},e_{1}\\}$ $\displaystyle=$
$\displaystyle-\\{e_{2},e_{1},e_{1}\\}=M_{13}e_{1},\qquad\\{e_{1},e_{3},e_{1}\\}=-\\{e_{3},e_{1},e_{1}\\}=M_{12}e_{1},$
$\displaystyle~{}\\{e_{2},e_{1},e_{2}\\}$ $\displaystyle=$
$\displaystyle-\\{e_{1},e_{2},e_{2}\\}=M_{23}e_{1},\qquad\\{e_{2},e_{3},e_{2}\\}=-\\{e_{3},e_{2},e_{2}\\}=-M_{21}e_{1},$
$\displaystyle~{}\\{e_{3},e_{1},e_{3}\\}$ $\displaystyle=$
$\displaystyle-\\{e_{1},e_{3},e_{3}\\}=-M_{32}e_{1},~{}~{}\quad\\{e_{3},e_{2},e_{3}\\}=-\\{e_{2},e_{3},e_{3}\\}=-M_{31}e_{1},$
$\displaystyle~{}[e_{i},e_{j},e_{k}]$ $\displaystyle=$
$\displaystyle(-1)^{\tau(ijk)}(-M_{22}-M_{33}e_{1}-M_{12}e_{2}+M_{13}e_{3}),\quad
i\neq j\neq k,$

and all the other brackets are zero, where the terms $\tau(ijk)$ is the
inverse table permutation of $i,j,k$ and $M_{ij}$ is the complementary minor
of element $a_{ij}.$

In fact, by (40), we have

$\displaystyle\\{e_{i},e_{j},e_{k}\\}$ $\displaystyle=$
$\displaystyle[Ne_{i},Ne_{i},e_{k}]_{\mathfrak{g}}$ $\displaystyle=$
$\displaystyle[a_{1i}e_{1}+a_{2i}e_{2}+a_{3i}e_{3},a_{1j}e_{1}+a_{2j}e_{2}+a_{3j}e_{3},e_{k}]_{\mathfrak{g}}$
$\displaystyle=$
$\displaystyle[a_{1i}e_{1},a_{2j}e_{2},e_{k}]_{\mathfrak{g}}+[a_{1i}e_{1},a_{3j}e_{3},e_{k}]_{\mathfrak{g}}+[a_{2i}e_{2},a_{1j}e_{1},e_{k}]_{\mathfrak{g}}$
$\displaystyle+[a_{2i}e_{2},a_{3j}e_{3},e_{k}]_{\mathfrak{g}}+[a_{3i}e_{3},a_{1j}e_{1},e_{k}]_{\mathfrak{g}}+[a_{3i}e_{3},a_{2j}e_{2},e_{k}]_{\mathfrak{g}}.$

When $k=1,$ we have

  * •

if $i=j,$ we get $\\{e_{i},e_{j},e_{1}\\}=0;$

  * •

if $i\neq j\neq k,$ we get
$\\{e_{i},e_{j},e_{1}\\}=\left|\begin{array}[]{cc}a_{2i}&a_{2j}\\\
a_{3i}&a_{3j}\end{array}\right|e_{1}=(-1)^{\tau(ij1)}M_{11}e_{1};$

  * •

if $i=k=1,j=2~{}\mbox{or}~{}3,$ we get

$\\{e_{1},e_{2},e_{1}\\}=\left|\begin{array}[]{cc}a_{21}&a_{22}\\\
a_{31}&a_{32}\end{array}\right|e_{1}=M_{13}e_{1},\qquad\\{e_{1},e_{3},e_{1}\\}=\left|\begin{array}[]{cc}a_{21}&a_{23}\\\
a_{31}&a_{33}\end{array}\right|e_{1}=M_{12}e_{1}.$

Similarly, we can obtain all the formulas listed above.

###  4.2. Reynolds operators on $3$-Lie algebras

In this subsection, we introduce the notion of a Reynolds operator on a
$3$-Lie algebra, which turns out to be a special twisted Rota-Baxter operator.
The relation between Reynolds operators and derivations on $3$-Lie algebras
are investigated.

######  Definition 4.5.

Let $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ be a $3$-Lie algebra.
A linear map $R:\mathfrak{g}\rightarrow\mathfrak{g}$ is called a Reynolds
operator if

(43)
$[Rx,Ry,Rz]_{\mathfrak{g}}=R\Big{(}[Rx,Ry,z]_{\mathfrak{g}}+[x,Ry,Rz]_{\mathfrak{g}}+[Rx,y,Rz]_{\mathfrak{g}}-[Rx,Ry,Rz]_{\mathfrak{g}}\Big{)},\quad\forall~{}x,y,z\in\mathfrak{g}.$

Moreover, a $3$-Lie algebra $\mathfrak{g}$ with a Reynolds operator $R$ is
called a Reynolds $3$-Lie algebra. We denote it by
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}},R).$

######  Definition 4.6.

Let $(\mathfrak{g}_{1},[\cdot,\cdot,\cdot]_{\mathfrak{g}_{1}},R_{1})$ and
$(\mathfrak{g}_{2},[\cdot,\cdot,\cdot]_{\mathfrak{g}_{2}},R_{2})$ be Reynolds
$3$-Lie algebras. A linear map
$\phi:\mathfrak{g}_{1}\rightarrow\mathfrak{g}_{2}$ is called a homomorphism of
Reynolds $3$-Lie algebras if $\phi$ is a $3$-Lie algebra homomorphism and
$\phi\circ R_{1}=R_{2}\circ\phi$.

######  Theorem 4.7.

Let $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}},R)$ be a Reynolds
$3$-Lie algebra. Define a multiplication $[\cdot,\cdot,\cdot]_{R}$ on
$\mathfrak{g}$ by

(44)
$[x,y,z]_{R}=[Rx,Ry,z]_{\mathfrak{g}}+[x,Ry,Rz]_{\mathfrak{g}}+[Rx,y,Rz]_{\mathfrak{g}}-[Rx,Ry,Rz]_{\mathfrak{g}},\quad\forall
x,y,z\in\mathfrak{g}.$

Then

  * (a)

$[Rx,Ry,Rz]_{\mathfrak{g}}=R([x,y,z]_{R})$;

  * (b)

$(\mathfrak{g},[\cdot,\cdot,\cdot]_{R})$ is a $3$-Lie algebra;

  * (c)

$(\mathfrak{g},[\cdot,\cdot,\cdot]_{R},R)$ is a Reynolds $3$-Lie algebra;

  * (d)

$R$ is a Reynolds $3$-Lie algebra homomorphism from
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{R},R)$ to
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}},R)$.

###### Proof.

It follows directly from (43).

Obviously, $[\cdot,\cdot,\cdot]_{R}$ is skew-symmetric. For
$x_{1},x_{2},x_{3},x_{4},x_{5}\in\mathfrak{g},$ by (1) and (44), we have

$\displaystyle[x_{1},x_{2},[x_{3},x_{4},x_{5}]_{R}]_{R}-[[x_{1},x_{2},x_{3}]_{R},x_{4},x_{5}]_{R}-[x_{3},[x_{1},x_{2},x_{4}]_{R},x_{5}]_{R}-[x_{3},x_{4},[x_{1},x_{2},x_{5}]_{R}]_{R}$
$\displaystyle=$
$\displaystyle[Rx_{1},Rx_{2},[Rx_{3},Rx_{4},x_{5}]_{\mathfrak{g}}]_{\mathfrak{g}}+[Rx_{1},Rx_{2},[x_{3},Rx_{4},Rx_{5}]_{\mathfrak{g}}]_{\mathfrak{g}}+[Rx_{1},Rx_{2},[Rx_{3},x_{4},x_{5}]_{\mathfrak{g}}]_{\mathfrak{g}}$
$\displaystyle+[x_{1},Rx_{2},[Rx_{3},Rx_{4},Rx_{5}]_{\mathfrak{g}}]_{\mathfrak{g}}+[Rx_{1},x_{2},[Rx_{3},Rx_{4},Rx_{5}]_{\mathfrak{g}}]_{\mathfrak{g}}$
$\displaystyle-[[Rx_{1},Rx_{2},x_{3}]_{\mathfrak{g}},Rx_{4},Rx_{5}]_{\mathfrak{g}}-[[Rx_{1},x_{2},Rx_{3}]_{\mathfrak{g}},Rx_{4},Rx_{5}]_{\mathfrak{g}}-[[x_{1},Rx_{2},Rx_{3}]_{\mathfrak{g}},Rx_{4},Rx_{5}]_{\mathfrak{g}}$
$\displaystyle-[[Rx_{1},Rx_{2},Rx_{3}]_{\mathfrak{g}},Rx_{4},x_{5}]_{\mathfrak{g}}-[[Rx_{1},Rx_{2},Rx_{3}]_{\mathfrak{g}},x_{4},Rx_{5}]_{\mathfrak{g}}$
$\displaystyle-[Rx_{3},[Rx_{1},Rx_{2},x_{4}]_{\mathfrak{g}},Rx_{5}]_{\mathfrak{g}}-[Rx_{3},[x_{1},Rx_{2},Rx_{4}]_{\mathfrak{g}},Rx_{5}]_{\mathfrak{g}}-[Rx_{3},[Rx_{1},x_{2},Rx_{4}]_{\mathfrak{g}},Rx_{5}]_{\mathfrak{g}}$
$\displaystyle-[Rx_{3},[Rx_{1},Rx_{2},Rx_{4}]_{\mathfrak{g}},x_{5}]_{\mathfrak{g}}-[x_{3},[Rx_{1},Rx_{2},Rx_{4}]_{\mathfrak{g}},Rx_{5}]_{\mathfrak{g}}$
$\displaystyle-[Rx_{3},Rx_{4},[Rx_{1},Rx_{2},x_{5}]_{\mathfrak{g}}]_{\mathfrak{g}}-[Rx_{3},Rx_{4},[Rx_{1},x_{2},Rx_{5}]_{\mathfrak{g}}]_{\mathfrak{g}}-[Rx_{3},Rx_{4},[x_{1},Rx_{2},Rx_{5}]_{\mathfrak{g}}]_{\mathfrak{g}}$
$\displaystyle-[Rx_{3},x_{4},[Rx_{1},Rx_{2},Rx_{5}]_{\mathfrak{g}}]_{\mathfrak{g}}-[x_{3},Rx_{4},[Rx_{1},Rx_{2},Rx_{5}]_{\mathfrak{g}}]_{\mathfrak{g}}$
$\displaystyle=$ $\displaystyle 0.$

Thus, $(\mathfrak{g},[\cdot,\cdot,\cdot]_{R})$ is a $3$-Lie algebra.

For $x,y,z\in\mathfrak{g},$ by (44) and Item (a), we have

$\displaystyle[Rx,Ry,Rz]_{R}$ $\displaystyle=$
$\displaystyle[R^{2}x,R^{2}y,Rz]_{\mathfrak{g}}+[Rx,R^{2}y,R^{2}z]_{\mathfrak{g}}+[R^{2}x,Ry,R^{2}z]_{\mathfrak{g}}-[R^{2}x,R^{2}y,R^{2}z]_{\mathfrak{g}}$
$\displaystyle=$ $\displaystyle
R([Rx,Ry,z]_{R}+[x,Ry,Rz]_{R}+[Rx,y,Rz]_{R}-[Rx,Ry,Rz]_{R}),$

which implies that $R$ is a Reynolds operator on the $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{R}).$

By Item (a), $R$ is a $3$-Lie algebra homomorphism. Moreover, $R$ commutes
with itself. Therefore, $R$ is a Reynolds $3$-Lie algebra homomorphism from
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{R},R)$ to
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}},R).$ ∎

The following results give the relation between Reynolds operators and twisted
Rota-Baxter operators on 3-Lie algebras.

######  Proposition 4.8.

Let $R$ be a Reynolds operator on a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$. Then $R$ is a
$\Phi$-twisted Rota-Baxter operator on $\mathfrak{g}$ with respect to the
adjoint representation $(\mathfrak{g};\mathrm{ad})$, where
$\Phi\in\mathrm{Hom}(\wedge^{3}\mathfrak{g},\mathfrak{g})$ is defined by

$\Phi(x,y,z)=-[x,y,z]_{\mathfrak{g}},\quad\forall x,y,z\in\mathfrak{g}.$

###### Proof.

Let $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ be a $3$-Lie algebra.
By (2.1), the $3$-Lie bracket $[\cdot,\cdot,\cdot]_{\mathfrak{g}}$ is a
$2$-cocycle with coefficients in the adjoint representation
$(\mathfrak{g};\mathrm{ad})$, which implies that $R$ is a $\Phi$-twisted Rota-
Baxter operator on $\mathfrak{g}$ with respect to the adjoint representation.
∎

As Reynolds operators on $3$-Lie algebras are particular $\Phi$-twisted Rota-
Baxter operators, the following conclusion is obvious.

######  Corollary 4.9.

Let $R:\mathfrak{g}\rightarrow\mathfrak{g}$ be a Reynolds operator on a
$3$-Lie algebra $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$. Then

$\displaystyle\\{x,y,z\\}=[Rx,Ry,z]_{\mathfrak{g}}\quad\mbox{and}\quad[x,y,z]=-[Rx,Ry,Rz]_{\mathfrak{g}},\quad\forall
x,y,z\in\mathfrak{g},$

defines an NS-$3$-Lie algebra structure on $\mathfrak{g}.$

Next, we illustrate the relationship between Reynolds operators and
derivations on a $3$-Lie algebra.

######  Proposition 4.10.

Let $R:\mathfrak{g}\rightarrow\mathfrak{g}$ be a Reynolds operator on a
$3$-Lie algebra $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$. If $R$ is
invertible, then
$(R^{-1}-\frac{1}{2}{\rm{Id}}):\mathfrak{g}\rightarrow\mathfrak{g}$ is a
derivation on the $3$-Lie algebra $\mathfrak{g}$, where ${\rm{Id}}$ is the
identity operator.

###### Proof.

Let $R:\mathfrak{g}\rightarrow\mathfrak{g}$ be an invertible Reynolds operator
on $\mathfrak{g}$. By (43), we have

$\displaystyle
R^{-1}[x,y,z]_{\mathfrak{g}}=[R^{-1}x,y,z]_{\mathfrak{g}}+[x,R^{-1}y,z]_{\mathfrak{g}}+[x,y,R^{-1}z]_{\mathfrak{g}}-[x,y,z]_{\mathfrak{g}},\quad\forall
x,y,z\in\mathfrak{g},$

which implies that, for all $x,y,z\in\mathfrak{g},$

$\displaystyle(R^{-1}-\frac{1}{2}{\rm{Id}})[x,y,z]_{\mathfrak{g}}=[(R^{-1}-\frac{1}{2}{\rm{Id}})x,y,z]_{\mathfrak{g}}+[x,(R^{-1}-\frac{1}{2}{\rm{Id}})y,z]_{\mathfrak{g}}+[x,y,(R^{-1}-\frac{1}{2}{\rm{Id}})z]_{\mathfrak{g}}.$

This shows that
$(R^{-1}-\frac{1}{2}{\rm{Id}}):\mathfrak{g}\rightarrow\mathfrak{g}$ is a
derivation on the $3$-Lie algebra $\mathfrak{g}$. ∎

Conversely, we can derive a Reynolds operator on a $3$-Lie algebra from a
derivation.

######  Proposition 4.11.

Let $D:\mathfrak{g}\rightarrow\mathfrak{g}$ be a derivation on a $3$-Lie
algebra $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$. If
$(D+\frac{1}{2}{\rm{Id}}):\mathfrak{g}\rightarrow\mathfrak{g}$ is invertible,
then $(D+\frac{1}{2}{\rm{Id}})^{-1}$ is a Reynolds operator.

###### Proof.

Let $D:\mathfrak{g}\rightarrow\mathfrak{g}$ be a derivation on a $3$-Lie
algebra $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}}).$ By (2), for all
$u,v,w\in\mathfrak{g}$, we have

$\displaystyle(D+\frac{1}{2}{\rm{Id}})[u,v,w]_{\mathfrak{g}}=[(D+\frac{1}{2}{\rm{Id}})u,v,w]_{\mathfrak{g}}+[u,(D+\frac{1}{2}{\rm{Id}})v,w]_{\mathfrak{g}}+[u,v,(D+\frac{1}{2}{\rm{Id}})w]_{\mathfrak{g}}-[u,v,w]_{\mathfrak{g}}.$

For convenience, we denote $P=D+\frac{1}{2}{\rm{Id}}.$ If $P$ is invertible,
we put $Pu=x,Pv=y~{}\mbox{and}~{}Pw=z,$ we get

$\displaystyle[P^{-1}x,P^{-1}y,P^{-1}z]_{\mathfrak{g}}$ $\displaystyle=$
$\displaystyle
P^{-1}\Big{(}[x,P^{-1}y,P^{-1}z]_{\mathfrak{g}}+[P^{-1}x,y,P^{-1}z]_{\mathfrak{g}}$
$\displaystyle+[P^{-1}x,P^{-1}y,z]_{\mathfrak{g}}-[P^{-1}x,P^{-1}y,P^{-1}z]_{\mathfrak{g}}\Big{)},$

which implies that $P^{-1}$ is a Reynolds operator. The proof is finished. ∎

At the end of this section, we provide some examples of Reynolds operators on
infinite dimensional $3$-Lie algebras.

In [3], the authors construct a class of infinite dimensional $3$-Lie algebras
by Laurent polynomials.

######  Example 4.12.

Let $A=F[t^{-1},t]$ be the Laurent polynomials over the field of complex
numbers. Then $A$ is an infinite dimensional $3$-Lie algebra with the
multiplication,

(45)
$[t^{l},t^{m},t^{n}]_{A}=\left|\begin{array}[]{ccc}(-1)^{l}&(-1)^{m}&(-1)^{n}\\\
1&1&1\\\ l&m&n\\\ \end{array}\right|t^{l+m+n-1},$

for all $l,m,n\in\mathbb{Z},~{}t^{l},t^{m},t^{n}\in A$.

Let $R:A\rightarrow A$ be a linear map defined by
$R(t^{l})=\frac{1}{l}t^{l},~{}l\in\mathbb{Z}.$ For any $l,m,n\in\mathbb{Z},$
we obtain

(46)
$\displaystyle[R(t^{l}),R(t^{m}),R(t^{n})]_{A}=\frac{1}{lmn}[t^{l},t^{m},t^{n}]_{A}.$

On the other hand,

$\displaystyle
R([R(t^{l}),R(t^{m}),t^{n}]_{A}+[t^{l},R(t^{m}),R(t^{n})]_{A}+[R(t^{l}),t^{m},R(t^{n})]_{A}-[R(t^{l}),R(t^{m}),R(t^{n})]_{A})$
$\displaystyle=$ $\displaystyle
R((\frac{1}{lm}+\frac{1}{mn}+\frac{1}{ln}-\frac{1}{lmn})[t^{l},t^{m},t^{n}]_{A})$
$\displaystyle=$
$\displaystyle(\frac{1}{lm}+\frac{1}{mn}+\frac{1}{ln}-\frac{1}{lmn})\frac{1}{l+m+n-1}[t^{l},t^{m},t^{n}]_{A}$
$\displaystyle=$ $\displaystyle\frac{1}{lmn}[t^{l},t^{m},t^{n}]_{A}.$

It follows from (46) and (4.12) that $R$ is a Reynolds operator on $A.$

By Item (b) and Item (c) in Theorem 4.7, $(A,[\cdot,\cdot,\cdot]_{R})$ is a
$3$-Lie algebra and $(A,[\cdot,\cdot,\cdot]_{R},R)$ is a Reynolds $3$-Lie
algebra, where the $3$-Lie bracket $[\cdot,\cdot,\cdot]_{R}$ is defined by

$\displaystyle[t^{l},t^{m},t^{n}]_{R}:$ $\displaystyle=$
$\displaystyle[R(t^{l}),R(t^{m}),t^{n}]_{A}+[t^{l},R(t^{m}),R(t^{n})]_{A}+[R(t^{l}),t^{m},R(t^{n})]_{A}-[R(t^{l}),R(t^{m}),R(t^{n})]_{A}$
$\displaystyle=$
$\displaystyle\frac{l+m+n-1}{lmn}\left|\begin{array}[]{ccc}(-1)^{l}&(-1)^{m}&(-1)^{n}\\\
1&1&1\\\ l&m&n\\\ \end{array}\right|t^{l+m+n-1}.$

According to Corollary 4.9, the Reynolds operator induces an NS-$3$-Lie
algebra $(A,\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot])$, where

$\displaystyle\\{t^{l},t^{m},t^{n}\\}$ $\displaystyle=$
$\displaystyle-\\{t^{l},t^{m},t^{n}\\}=\frac{1}{lm}[t^{l},t^{m},t^{n}]_{A},$
$\displaystyle~{}[t^{l},t^{m},t^{n}]$ $\displaystyle=$
$\displaystyle\frac{1}{lmn}[t^{l},t^{m},t^{n}]_{A}.$

In [6], the authors constructed an infinite-dimensional $3$-Lie algebra,
called the $\omega_{\infty}$ $3$-Lie algebra by applying a double scaling
limits on the generators of the $W_{\infty}$ algebra [26].

######  Example 4.13.

Let $\\{\omega^{a}_{m}|a,m\in\mathbb{Z}\\}$ generate an infinite-dimensional
vector space. Define

(49)
$[\omega^{a}_{m},\omega^{b}_{n},\omega^{c}_{p}]=\left|\begin{array}[]{ccc}1&1&1\\\
m&n&p\\\ a&b&c\\\
\end{array}\right|\omega^{a+b+c+1}_{m+n+p},\quad\forall~{}m,n,p,a,b,c\in\mathbb{Z}.$

Then $\\{\omega^{a}_{m}|a,m\in\mathbb{Z}\\}$ generate an infinite-dimensional
$3$-Lie algebra which is called the $\omega_{\infty}$ $3$-Lie algebra. We
denote the $3$-Lie algebra by $\langle\omega^{a}_{m}\rangle.$

Let $R:\langle\omega^{a}_{m}\rangle\rightarrow\langle\omega^{a}_{m}\rangle$ be
a linear map defined by

$R(\omega^{a}_{m})=\frac{1}{m+a+1}\omega^{a}_{m},\quad m,a\in\mathbb{Z}.$

We observe that, for any $m,n,p,a,b,c\in\mathbb{Z},$

(50)
$\displaystyle[R(\omega^{a}_{m}),R(\omega^{b}_{n}),R(\omega^{c}_{p})]=\frac{1}{(m+a+1)(n+b+1)(p+c+1)}[\omega^{a}_{m},\omega^{b}_{n},\omega^{c}_{p}].$

On the other hand,

$\displaystyle
R([R(\omega^{a}_{m}),R(\omega^{b}_{n}),\omega^{c}_{p}]+[\omega^{a}_{m},R(\omega^{b}_{n}),R(\omega^{c}_{p})]+[R(\omega^{a}_{m}),\omega^{b}_{n},R(\omega^{c}_{p})]-[R(\omega^{a}_{m}),R(\omega^{b}_{n}),R(\omega^{c}_{p})])$
$\displaystyle=$ $\displaystyle
R((\frac{1}{(m+a+1)(n+b+1)}+\frac{1}{(n+b+1)(p+c+1)}+\frac{1}{(m+a+1)(p+c+1)}$
$\displaystyle-\frac{1}{(m+a+1)(n+b+1)(p+c+1)})[\omega^{a}_{m},\omega^{b}_{n},\omega^{c}_{p}])$
$\displaystyle=$
$\displaystyle(\frac{1}{(m+a+1)(n+b+1)}+\frac{1}{(n+b+1)(p+c+1)}+\frac{1}{(m+a+1)(p+c+1)}$
$\displaystyle-\frac{1}{(m+a+1)(n+b+1)(p+c+1)})\frac{1}{m+n+p+a+b+c+2}[\omega^{a}_{m},\omega^{b}_{n},\omega^{c}_{p}]$
$\displaystyle=$
$\displaystyle\frac{1}{(m+a+1)(n+b+1)(p+c+1)}[\omega^{a}_{m},\omega^{b}_{n},\omega^{c}_{p}].$

By (50) and (4.13), $R$ is a Reynolds operator on
$\langle\omega^{a}_{m}\rangle.$

Similarly, $(\langle\omega^{a}_{m}\rangle,[\cdot,\cdot,\cdot]_{R})$ is a
$3$-Lie algebra and $(\langle\omega^{a}_{m}\rangle,[\cdot,\cdot,\cdot]_{R},R)$
is a Reynolds $3$-Lie algebra, where the $3$-Lie bracket
$[\cdot,\cdot,\cdot]_{R}$ is given by

$\displaystyle[\omega^{a}_{m},\omega^{b}_{n},\omega^{c}_{p}]_{R}:$
$\displaystyle=$
$\displaystyle[R(\omega^{a}_{m}),R(\omega^{b}_{n}),\omega^{c}_{p}]+[\omega^{a}_{m},R(\omega^{b}_{n}),R(\omega^{c}_{p})]$
$\displaystyle+[R(\omega^{a}_{m}),\omega^{b}_{n},R(\omega^{c}_{p})]-[R(\omega^{a}_{m}),R(\omega^{b}_{n}),R(\omega^{c}_{p})]$
$\displaystyle=$
$\displaystyle\frac{m+n+p+a+b+c+2}{(m+a+1)(n+b+1)(p+c+1)}\left|\begin{array}[]{ccc}1&1&1\\\
m&n&p\\\ a&b&c\\\ \end{array}\right|\omega^{a+b+c+1}_{m+n+p}.$

## References

  * [1] C. Bai, L. Guo and Y. Sheng, Bialgebras, the classical Yang-Baxter equation and Manin triples for $3$-Lie algebras, _Adv. Theor. Math. Phys._ 23 (2019), 27-74.
  * [2] R. Bai, L. Guo, J. Li and Y. Wu, Rota-Baxter $3$-Lie algebras, _J. Math. Phys._ 54 (2013), 064504, 14 pp.
  * [3] R. Bai, Y. Wu, Constructions of $3$-Lie algebras, _Linear Multilinear Algebra_ 63 (2015), no. 11, 2171-2186.
  * [4] J. Carinena, J. Grabowski and G. Marmo, Quantum bi-Hamiltonian systems, _Internat. J. Modern Phys. A._ 15 (2000), no. 30, 4797-4810.
  * [5] J. M. Casas, J. L. Loday and T. Pirashvili, Leibniz $n$-algebras, _Forum Math._ 14 (2002), no. 2, 189-207.
  * [6] S. Chakrabortty, A. Kumar and S. Jain, $\omega_{\infty}$ $3$-algebra, _J. High Energy Phys._ 091 (2008), no. 9, 8 pp.
  * [7] A. Connes and D. Kreimer, Renormalization in quantum field theory and the Riemann-Hilbert problem. I. The Hopf algebra structure of graphs and the main theorem, Comm. Math. Phys.  210 (2000), 249-273.
  * [8] A. Das, Deformations of associative Rota-Baxter operators, _J. Algebra_ 560 (2020), 144-180.
  * [9] A. Das, Twisted Rota-Baxter operators, Reynolds operators on Lie algebras and NS-Lie algebras, _arXiv:2009.09368_.
  * [10] A. Das, Cohomology and deformations of twisted Rota-Baxter operators and NS-algebras, _arXiv:2010.01156_.
  * [11] J. A. de Azcárraga and J. M. Izquierdo, $n$-ary algebras: a review with applications, _J. Phys. A_ 43 (2010), 293001.
  * [12] I. Dorfman, _Dirac structures and integrability of nonlinear evolution equations,_ Wiley, Chichester, 1993.
  * [13] A. S. Dzhumadil′daev, Representations of vector product $n$-Lie algebras, _Comm. Algebra_ 32 (2004), 3315-3326.
  * [14] J. Figueroa-O′Farrill, Deformations of 3-algebras, _J. Math. Phys._ 50 (2009), no. 11, 113514, 27 pp.
  * [15] V. T. Filippov, $n$-Lie algebras, _Sibirsk. Mat. Zh._ 26 (1985), 126-140, 191.
  * [16] L. Guo, What is a Rota-Baxter algebra? _Notice Amer. Math. Soc_ 56 (2009), 1436-1437.
  * [17] L. Guo, An introduction to Rota-Baxter algebra, _International Press_ (2012), xii+226 pp.
  * [18] J. Kampé de Fériet, Introduction to the statistical theory of turbulence, _Correlation and spectrum_. The Institute for Fluid Dynamics and Applied Mathematics, University of Maryland, College Park, Md., (1951), iv+162 pp.
  * [19] Sh. M. Kasymov, On a theory of $n$-Lie algebras, _Algebra i Logika_ 26 (1987), 277-297.
  * [20] B. A. Kupershmidt, What a classical $r$-matrix really is. _J. Nonlinear Math. Phys._ 6 (1999), 448-488.
  * [21] P. Lei and L. Guo, Nijenhuis algebras, NS algebras and N-dendriform algebras, _Front. Math._ 7 (2012), 827-846.
  * [22] P. Leroux, Construction of Nijenhuis operators and dendriform trialgebras, _Int. J. Math. Math. Sci._ (2004), no. 49-52, 2595-2615.
  * [23] J. Liu, Y. Sheng, Y. Zhou and C. Bai, Nijenhuis operators on $n$-Lie algebras, _Commun. Theor. Phys._ 65 (2016), no. 6, 659-670.
  * [24] A. Makhlouf, On deformations of $n$-Lie algebras. Non-associative and non-commutative algebra and operator theory, _Springer Proc. Math. Stat._ 160, Springer, Cham, (2016), 55-81.
  * [25] J. Pei, C. Bai and L. Guo, Splitting of operads and Rota-Baxter operators on operads, Appl. Cate. Stru. 25 (2017), 505-538.
  * [26] C. N. Pope, L. J. Romans and X. Shen, The complete structure of $W_{\infty}$, _Phys. Lett. B_ 236 (1990), no. 2, 173-178.
  * [27] O. Reynolds, On the dynamical theory of incompressible viscous fluids and the determination of the criterion, _Phil. Trans. Roy. Soc. A_ 136 (1895), 123-164; reprinted in _Proc. Roy. Soc. London Ser. A_ 451 (1995), no. 1941, 5-47.
  * [28] M. A. Semonov-Tian-Shansky, What is a classical R-matrix? _Funct. Anal. Appl._ 17 (1983), 259-272.
  * [29] Y. Sheng and R. Tang, Symplectic, product and complex structures on 3-Lie algebras, _J. Algebra_ 508 (2018), 256-300.
  * [30] L. A. Takhtajan, Higher order analog of Chevalley-Eilenberg complex and deformation theory of $n$-gebras, _St. Petersburg Math. J._ 6 (1995), 429-438.
  * [31] R. Tang, C. Bai, L. Guo and Y. Sheng, Deformations and their controlling cohomologies of $\mathcal{O}$-operators, _Comm. Math. Phys._ 368 (2019), no. 2, 665-700.
  * [32] R. Tang, S. Hou and Y. Sheng, Lie $3$-algebras and deformations of relative Rota-Baxter operators on $3$-Lie algebras, _J. Algebra_ 567 (2021), 37-62.
  * [33] K. Uchino, Twisting on associative algebras and Rota-Baxter type operators. J. Noncommut. Geom. 4 (2010), 349-379.
  * [34] T. Zhang, X. Gao and L. Guo, Reynolds algebras and their free objects from bracketed words and rooted trees, _arXiv:1911.08970_.
  * [35] T. Zhang, Deformations and Extensions of 3-Lie algebras, arXiv:1401.4656.