#  $3$-post-Lie algebras and relative Rota-Baxter operators of nonzero weight
on $3$-Lie algebras

Shuai Hou  Department of Mathematics, Jilin University, Changchun 130012,
Jilin, China<EMAIL_ADDRESS>,  Yunhe Sheng  Department of Mathematics,
Jilin University, Changchun 130012, Jilin, China<EMAIL_ADDRESS>and
Yanqiu Zhou  School of Science, Guangxi University of Science and Technology,
Liuzhou 545006, China<EMAIL_ADDRESS>
###### Abstract.

In this paper, first we introduce the notions of relative Rota-Baxter
operators of nonzero weight on $3$-Lie algebras and $3$-post-Lie algebras. A
3-post-Lie algebra consists of a 3-Lie algebra structure and a ternary
operation such that some compatibility conditions are satisfied. We show that
a relative Rota-Baxter operator of nonzero weight induces a $3$-post-Lie
algebra naturally. Conversely, a $3$-post-Lie algebra gives rise to a new
3-Lie algebra, which is called the subadjacent 3-Lie algebra, and an action on
the original 3-Lie algebra. Then we construct an $L_{\infty}$-algebra whose
Maurer-Cartan elements are relative Rota-Baxter operators of nonzero weight.
Consequently, we obtain the twisted $L_{\infty}$-algebra that controls
deformations of a given relative Rota-Baxter operator of nonzero weight on
3-Lie algebras. Finally, we introduce a cohomology theory for a relative Rota-
Baxter operator of nonzero weight on $3$-Lie algebras and use the second
cohomology group to classify infinitesimal deformations.

###### Key words and phrases:

$3$-Lie algebra, $3$-post-Lie algebra, relative Rota-Baxter operator,
cohomology, deformation.
2020 Mathematics Subject Classification. 17A42, 17B56, 17B38

###### Contents

  1. 1 Introduction
  2. 2 Relative Rota-Baxter operators of weight $\lambda$ on $3$-Lie algebras
  3. 3 $3$-post-Lie algebras
  4. 4 Maurer-Cartan characterization of relative Rota-Baxter operators of weight $\lambda$
  5. 5 Cohomologies of relative Rota-Baxter operators and infinitesimal deformations
    1. 5.1 Cohomologies of relative Rota-Baxter operators of weight $\lambda$ on 3-Lie algebras
    2. 5.2 Infinitesimal deformations of relative Rota-Baxter operators

##  1\. Introduction

Rota-Baxter associative algebras originated from the probability study of G.
Baxter [7]. In the Lie algebra context, Kupershmidt introduced the notion of
an ${\mathcal{O}}$-operator (also called a relative Rota-Baxter operator) in
[26] to better understand the classical Yang-Baxter equation. (Relative) Rota-
Baxter operators on Lie algebras and associative algebras have important
applications in various fields, such as the classical Yang-Baxter equation and
integrable systems [2, 26, 33], splitting of operads [3, 31], double Lie
algebras [21], Connes-Kreimer’s algebraic approach to renormalization of
quantum field theory [11], etc. See the book [22] for more details. Recently,
the deformation and cohomology theories of relative Rota-Baxter operators on
both Lie and associative algebras were studied in [14, 35, 39]. Post-Lie
algebras, as natural generalizations of pre-Lie algebras [8], were introduced
by Vallette in [37], and have important applications in geometric numerical
integration and mathematical physics [5, 9, 12, 13, 17, 29]. In particular, a
relative Rota-Baxter operator of nonzero weight on Lie algebras induces a
post-Lie algebra.

The notion of $3$-Lie algebras and more generally, $n$-Lie algebras was
introduced by Filippov in [19], which appeared naturally in various areas of
theoretical and mathematical physics [1, 16, 18]. See the review article [15,
28] for more details. Indeed, the $n$-Lie algebra is the algebraic structure
corresponding to the Nambu mechanics [30]. In [6], R. Bai et al. introduced
the notion of Rota-Baxter operators of weight $\lambda$ on $3$-Lie algebras
and give various constructions from Rota-Baxter operators on Lie algebras and
pre-Lie algebras. Then the notion of an ${\mathcal{O}}$-operator on a $3$-Lie
algebra with respect to a representation was introduced in [4] to study
solutions of the $3$-Lie classical Yang-Baxter equation. In particular, when
the representation is the adjoint representation, an ${\mathcal{O}}$-operator
is exactly a Rota-Baxter operator of weight $0.$ ${\mathcal{O}}$-operators on
3-Lie algebras were also used to study matched pairs of 3-Lie algebras and
3-Lie bialgebras in [24]. Cohomologies and deformations of
${\mathcal{O}}$-operators on 3-Lie algebras were studied in [36], where the
terminology of relative Rota-Baxter operators was used instead of
${\mathcal{O}}$-operators.

The first purpose of this paper is to study relative Rota-Baxter operators of
nonzero weight on 3-Lie algebras and associated structures. For this purpose,
first we study actions of a 3-Lie algebra $\mathfrak{g}$ on a 3-Lie algebra
$\mathfrak{h}$, which is totally different from the case of Lie algebra
actions. We introduce the notion of a relative Rota-Baxter operator of weight
$\lambda$ from a $3$-Lie algebra $\mathfrak{h}$ to a $3$-Lie algebra
$\mathfrak{g}$ with respect to an action $\rho$, and characterize it using
graphs of the semidirect product 3-Lie algebra. We further establish the
deformation and cohomology theories for relative Rota-Baxter operators of
weight $\lambda$ on $3$-Lie algebras. Note that relative Rota-Baxter operators
and Rota-Baxter operators of nonzero weight introduced in [6] are not
consistent, see Remark 2.7 for details of the explanation. The cohomology
theory for Rota-Baxter operators of weight $\lambda$ on $3$-Lie algebras is
considered in [23] separately.

The second purpose is to investigate the $3$-ary generalizations of post-Lie
algebras and analyze the relation with the aforementioned relative Rota-Baxter
operators of nonzero weight on 3-Lie algebras. We introduce a new algebraic
structure, which is called a $3$-post-Lie algebra. A $3$-post-Lie algebra
naturally gives rise to a new $3$-Lie algebra and an action on the original
3-Lie algebra such that the identity map is a relative Rota-Baxter operator of
weight 1. We show that a relative Rota-Baxter operator of weight $\lambda$
induces a $3$-post-Lie algebra structure naturally. We will explore further
applications of 3-post-Lie algebras along the line of applications of post-Lie
algebras in future works.

The paper is organized as follows. In Section 2, we introduce the notion of a
relative Rota-Baxter operator of weight $\lambda$ from a $3$-Lie algebra
$\mathfrak{h}$ to a $3$-Lie algebra $\mathfrak{g}$ with respect to an action
$\rho$. In Section 3, we introduce the notion of a $3$-post-Lie algebra and
show that a relative Rota-Baxter operator of weight $\lambda$ on $3$-Lie
algebras naturally induces a $3$-post-Lie algebra. Moreover, a $3$-post-Lie
algebra also gives rise to a new 3-Lie algebra together with an action. In
Section 4, we construct an $L_{\infty}$-algebra whose Maurer-Cartan elements
are precisely relative Rota-Baxter operators of weight $\lambda$ on $3$-Lie
algebras. In Section 5, we establish a cohomology theory for a relative Rota-
Baxter operator of weight $\lambda$ on $3$-Lie algebras, and classify
infinitesimal deformations using the second cohomology group.

In this paper, we work over an algebraically closed filed $\mathbb{K}$ of
characteristic $0$.

Acknowledgements. This research is supported by NSFC (11922110). We give
warmest thanks to the referee for helpful suggestions.

##  2\. Relative Rota-Baxter operators of weight $\lambda$ on $3$-Lie algebras

In this section, we first introduce the notion of action of 3-Lie algebras,
which give rise to the semidirect product 3-Lie algebras. Then we introduce
the notion of relative Rota-Baxter operators of weight $\lambda$ on $3$-Lie
algebras, which can be characterized by the graphs of the semidirect product
3-Lie algebras. Finally we establish the relation between relative Rota-Baxter
operators of weight $\lambda$ on $3$-Lie algebras and Nijenhuis operators on
3-Lie algebras. A class of examples are given via certain projections.

######  Definition 2.1.

([19]) A 3-Lie algebra is a vector space $\mathfrak{g}$ together with a skew-
symmetric linear map
$[\cdot,\cdot,\cdot]_{\mathfrak{g}}:\wedge^{3}\mathfrak{g}\rightarrow\mathfrak{g}$,
such that for $x_{i}\in\mathfrak{g},1\leq i\leq 5$, the following Fundamental
Identity holds:

(1)
$\displaystyle[x_{1},x_{2},[x_{3},x_{4},x_{5}]_{\mathfrak{g}}]_{\mathfrak{g}}$
$\displaystyle=$
$\displaystyle[[x_{1},x_{2},x_{3}]_{\mathfrak{g}},x_{4},x_{5}]_{\mathfrak{g}}+[x_{3},[x_{1},x_{2},x_{4}]_{\mathfrak{g}},x_{5}]_{\mathfrak{g}}+[x_{3},x_{4},[x_{1},x_{2},x_{5}]_{\mathfrak{g}}]_{\mathfrak{g}}.$

For $x_{1},x_{2}\in\mathfrak{g}$, define
$\mathrm{ad}_{x_{1},x_{2}}\in\mathfrak{gl}(\mathfrak{g})$ by

$\displaystyle\mathrm{ad}_{x_{1},x_{2}}x:=[x_{1},x_{2},x]_{\mathfrak{g}},\quad\forall
x\in\mathfrak{g}.$

Then $\mathrm{ad}_{x_{1},x_{2}}$ is a derivation, i.e.

$\mathrm{ad}_{x_{1},x_{2}}[x_{3},x_{4},x_{5}]_{\mathfrak{g}}=[\mathrm{ad}_{x_{1},x_{2}}x_{3},x_{4},x_{5}]_{\mathfrak{g}}+[x_{3},\mathrm{ad}_{x_{1},x_{2}}x_{4},x_{5}]_{\mathfrak{g}}+[x_{3},x_{4},\mathrm{ad}_{x_{1},x_{2}}x_{5}]_{\mathfrak{g}}.$

######  Definition 2.2.

([25]) A representation of a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ on a vector space $V$ is a
linear map: $\rho:\wedge^{2}\mathfrak{g}\rightarrow\mathfrak{gl}(V)$, such
that for all $x_{1},x_{2},x_{3},x_{4}\in\mathfrak{g},$ the following
equalities hold:

(2) $\displaystyle~{}\rho(x_{1},x_{2})\rho(x_{3},x_{4})$ $\displaystyle=$
$\displaystyle\rho([x_{1},x_{2},x_{3}]_{\mathfrak{g}},x_{4})+\rho(x_{3},[x_{1},x_{2},x_{4}]_{\mathfrak{g}})+\rho(x_{3},x_{4})\rho(x_{1},x_{2});$
(3) $\displaystyle~{}\rho(x_{1},[x_{2},x_{3},x_{4}]_{\mathfrak{g}})$
$\displaystyle=$
$\displaystyle\rho(x_{3},x_{4})\rho(x_{1},x_{2})-\rho(x_{2},x_{4})\rho(x_{1},x_{3})+\rho(x_{2},x_{3})\rho(x_{1},x_{4}).$

Let $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ be a $3$-Lie algebra.
The linear map
$\mathrm{ad}:\wedge^{2}\mathfrak{g}\rightarrow\mathfrak{gl}(\mathfrak{g})$
defines a representation of the $3$-Lie algebra $\mathfrak{g}$ on itself,
which is called the adjoint representation of $\mathfrak{g}.$

######  Definition 2.3.

([19]) Let $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ be a $3$-Lie
algebra. Then the subalgebra
$[\mathfrak{g},\mathfrak{g},\mathfrak{g}]_{\mathfrak{g}}$ is called the
derived algebra of $\mathfrak{g}$, and denoted by ${\mathfrak{g}}^{1}$. The
subspace

${\mathcal{C}}(\mathfrak{g})=\\{x\in\mathfrak{g}~{}|~{}[x,y,z]_{\mathfrak{g}}=0,~{}\forall
y,z\in\mathfrak{g}\\}$

is called the center of $\mathfrak{g}$.

######  Definition 2.4.

Let $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ and
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}})$ be two $3$-Lie algebras.
Let $\rho:\wedge^{2}\mathfrak{g}\rightarrow\mathfrak{gl}(\mathfrak{h})$ be a
representation of the $3$-Lie algebra $\mathfrak{g}$ on the vector space
$\mathfrak{h}$. If for all $x,y\in\mathfrak{g},u,v,w\in\mathfrak{h},$

(4) $\displaystyle{}\rho(x,y)u\in{\mathcal{C}}(\mathfrak{h}),$ (5)
$\displaystyle{}\rho(x,y)[u,v,w]_{\mathfrak{h}}=0,$

then $\rho$ is called an action of $\mathfrak{g}$ on $\mathfrak{h}.$

We denote an action by $(\mathfrak{h};\rho).$ Note that (4) and (5) imply that
$\rho(x,y)$ is a derivation.

######  Example 2.5.

Let $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ be a $3$-Lie algebra.
If $\mathfrak{g}$ satisfies
${\mathfrak{g}}^{1}\subset{\mathcal{C}}(\mathfrak{g}),$ then the adjoint
representation
$\mathrm{ad}:\wedge^{2}\mathfrak{g}\rightarrow\mathfrak{gl}(\mathfrak{g})$ is
an action of $\mathfrak{g}$ on itself.

######  Definition 2.6.

Let $\rho:\wedge^{2}\mathfrak{g}\rightarrow\mathfrak{gl}(\mathfrak{h})$ be an
action of a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ on a $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}}).$ A linear map
$T:\mathfrak{h}\rightarrow\mathfrak{g}$ is called a relative Rota-Baxter
operator of weight $\lambda\in\mathbb{K}$ from a $3$-Lie algebra
$\mathfrak{h}$ to a $3$-Lie algebra $\mathfrak{g}$ with respect to an action
$\rho$ if

(6)
$\displaystyle\qquad[Tu,Tv,Tw]_{\mathfrak{g}}=T\Big{(}\rho(Tu,Tv)w+\rho(Tv,Tw)u+\rho(Tw,Tu)v+\lambda[u,v,w]_{\mathfrak{h}}\Big{)},\quad\forall
u,v,w\in\mathfrak{h}.$

######  Remark 2.7.

The notion of a relative Rota-Baxter operator of weight $\lambda$ on a $3$-Lie
algebra given above is a natural generalization of the
${\mathcal{O}}$-operator introduced in [4]. While it is not consistent with
the Rota-Baxter operator of weight $\lambda$ introduced in [6]. Therefore,
there are two different theories for relative Rota-Baxter operators of weight
$\lambda$ introduced above and Rota-Baxter operators of weight $\lambda$ on
$3$-Lie algebras introduced in [6]. We will see that relative Rota-Baxter
operators of weight $\lambda$ on $3$-Lie algebras can be characterized by
Maurer-Cartan elements of the controlling $L_{\infty}$-algebras, and naturally
related to $3$-post-Lie algebras and Nijenhuis structures. These facts can be
viewed as justifications of relative Rota-Baxter operators of weight $\lambda$
on $3$-Lie algebras being interesting structures.

######  Definition 2.8.

Let $T$ and $T^{\prime}$ be two relative Rota-Baxter operators of weight
$\lambda$ from a $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}})$ to a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ with respect to an action
$\rho$. A homomorphism from $T$ to $T^{\prime}$ consists of $3$-Lie algebra
homomorphisms $\psi_{\mathfrak{g}}:\mathfrak{g}\,\rightarrow\,\mathfrak{g}$
and $\psi_{\mathfrak{h}}:\mathfrak{h}\,\rightarrow\,\mathfrak{h}$ such that

(7) $\displaystyle\psi_{\mathfrak{g}}\circ T$ $\displaystyle=$ $\displaystyle
T^{\prime}\circ\psi_{\mathfrak{h}},$ (8)
$\displaystyle\psi_{\mathfrak{h}}(\rho(x,y)u)$ $\displaystyle=$
$\displaystyle\rho(\psi_{\mathfrak{g}}(x),\psi_{\mathfrak{g}}(y))(\psi_{\mathfrak{h}}(u)),\quad\forall
x,y\in\mathfrak{g},u\in\mathfrak{h}.$

In particular, if both $\psi_{\mathfrak{g}}$ and $\psi_{\mathfrak{h}}$ are
invertible, $(\psi_{\mathfrak{g}},\psi_{\mathfrak{h}})$ is called an
isomorphism from $T$ to $T^{\prime}$.

Let $\rho:\wedge^{2}\mathfrak{g}\rightarrow\mathfrak{gl}(\mathfrak{h})$ be an
action of a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ on a $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}}).$ Define a skew-symmetric
bracket operation $[\cdot,\cdot,\cdot]_{\rho}$ on
$\mathfrak{g}\oplus\mathfrak{h}$ by

(9)
$\displaystyle{}[x+u,y+v,z+w]_{\rho}=[x,y,z]_{\mathfrak{g}}+\rho(x,y)w+\rho(y,z)u+\rho(z,x)v+\lambda[u,v,w]_{\mathfrak{h}},$

for all $x,y,z\in\mathfrak{g},~{}u,v,w\in\mathfrak{h}.$

######  Proposition 2.9.

With above notations,
$(\mathfrak{g}\oplus\mathfrak{h},[\cdot,\cdot,\cdot]_{\rho})$ is a $3$-Lie
algebra, which is called the semidirect product of the $3$-Lie algebra
$\mathfrak{g}$ and the $3$-Lie algebra $\mathfrak{h}$ with respect to the
action $\rho$, and denoted by $\mathfrak{g}\ltimes_{\rho}\mathfrak{h}.$

###### Proof.

It follows from straightforward computations, and we omit details. ∎

######  Theorem 2.10.

Let $\rho:\wedge^{2}\mathfrak{g}\rightarrow\mathfrak{gl}(\mathfrak{h})$ be an
action of a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ on a $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}}).$ Then a linear map
$T:\mathfrak{h}\rightarrow\mathfrak{g}$ is a relative Rota-Baxter operator of
weight $\lambda$ if and only if the graph

$Gr(T)=\\{Tu+u|u\in\mathfrak{h}\\}$

is a subalgebra of the $3$-Lie algebra
$\mathfrak{g}\ltimes_{\rho}\mathfrak{h}$.

###### Proof.

Let $T:\mathfrak{h}\rightarrow\mathfrak{g}$ be a linear map. For all
$u,v,w\in\mathfrak{h},$ we have

$\displaystyle[Tu+u,Tv+v,Tw+w]_{\rho}$ $\displaystyle=$
$\displaystyle[Tu,Tv,Tw]_{\mathfrak{g}}+\rho(Tu,Tv)w+\rho(Tv,Tw)u+\rho(Tw,Tu)v+\lambda[u,v,w]_{\mathfrak{h}},$

which implies that the graph $Gr(T)=\\{Tu+u|u\in\mathfrak{h}\\}$ is a
subalgebra of the $3$-Lie algebra $\mathfrak{g}\ltimes_{\rho}\mathfrak{h}$ if
and only if $T$ satisfies

$\displaystyle[Tu,Tv,Tw]_{\mathfrak{g}}=T\Big{(}\rho(Tu,Tv)w+\rho(Tv,Tw)u+\rho(Tw,Tu)v+\lambda[u,v,w]_{\mathfrak{h}}\Big{)},$

which means that $T$ is a relative Rota-Baxter operator of weight $\lambda$. ∎

Since the graph $Gr(T)$ is isomorphic to $\mathfrak{h}$ as a vector space,
there is an induced $3$-Lie algebra structure on $\mathfrak{h}.$

######  Corollary 2.11.

Let $T:\mathfrak{h}\rightarrow\mathfrak{g}$ be a relative Rota-Baxter operator
of weight $\lambda$ from a $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}})$ to a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ with respect to an action
$\rho$. Then $(\mathfrak{h},[\cdot,\cdot,\cdot]_{T})$ is a $3$-Lie algebra,
called the descendent $3$-Lie algebra of $T$, where

(10)
$[u,v,w]_{T}=\rho(Tu,Tv)w+\rho(Tv,Tw)u+\rho(Tw,Tu)v+\lambda[u,v,w]_{\mathfrak{h}},\quad\forall
u,v,w\in\mathfrak{h}.$

Moreover, $T$ is a $3$-Lie algebra homomorphism from
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{T})$ to
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$.

In the sequel, we give the relationship between relative Rota-Baxter operators
of weight $\lambda$ and Nijenhuis operators. Recall from [27] that a Nijenhuis
operator on a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ is a linear map
$N:\mathfrak{g}\rightarrow\mathfrak{g}$ satisfying

$\displaystyle[Nx,Ny,Nz]_{\mathfrak{g}}$ $\displaystyle=$ $\displaystyle
N\Big{(}[Nx,Ny,z]_{\mathfrak{g}}+[x,Ny,Nz]_{\mathfrak{g}}+[Nx,y,Nz]_{\mathfrak{g}}$
$\displaystyle-N[Nx,y,z]_{\mathfrak{g}}-N[x,Ny,z]_{\mathfrak{g}}-N[x,y,Nz]_{\mathfrak{g}}$
$\displaystyle+N^{2}[x,y,z]_{\mathfrak{g}}\Big{)},\quad\forall
x,y,z\in\mathfrak{g}.$

######  Proposition 2.12.

Let $\rho:\wedge^{2}\mathfrak{g}\rightarrow\mathfrak{gl}(\mathfrak{h})$ be an
action of a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ on a $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}}).$ Then a linear map
$T:\mathfrak{h}\rightarrow\mathfrak{g}$ is a relative Rota-Baxter operator of
weight $\lambda$ if and only if

$\overline{T}=\left(\begin{array}[]{cc}{{\rm{Id}}}&T\\\ 0&0\\\
\end{array}\right):\mathfrak{g}\oplus\mathfrak{h}\rightarrow\mathfrak{g}\oplus\mathfrak{h},$

is a Nijenhuis operator acting on the semidirect product $3$-Lie algebra
$\mathfrak{g}\ltimes_{\rho}\mathfrak{h}.$

###### Proof.

For all $x,y,z\in\mathfrak{g},u,v,w\in\mathfrak{h},$ on the one hand, we have

$\displaystyle[\overline{T}(x+u),\overline{T}(y+v),\overline{T}(z+w)]_{\rho}$
$\displaystyle=$
$\displaystyle[x,y,z]_{\mathfrak{g}}+[Tu,y,z]_{\mathfrak{g}}+[x,Tv,z]_{\mathfrak{g}}+[x,y,Tw]_{\mathfrak{g}}$
$\displaystyle+[x,Tv,Tw]_{\mathfrak{g}}+[Tu,y,Tw]_{\mathfrak{g}}+[Tu,Tv,z]_{\mathfrak{g}}+[Tu,Tv,Tw]_{\mathfrak{g}}.$

On the other hand, since $\overline{T}^{2}=\overline{T},$ we have

$\displaystyle\overline{T}\Big{(}[\overline{T}(x+u),\overline{T}(y+v),z+w]_{\rho}+[\overline{T}(x+u),y+v,\overline{T}(z+w)]_{\rho}+[x+u,\overline{T}(y+v),\overline{T}(z+w)]_{\rho}\Big{)}$
$\displaystyle-\overline{T}^{2}\Big{(}[\overline{T}(x+u),y+v,z+w]_{\rho}+[x+u,\overline{T}(y+v),z+w]_{\rho}+[x+u,y+v,\overline{T}(z+w)]_{\rho}\Big{)}$
$\displaystyle+\overline{T}^{3}\Big{(}[x+u,y+v,z+w]_{\rho}\Big{)}$
$\displaystyle=$
$\displaystyle[x,y,z]_{\mathfrak{g}}+[Tu,y,z]_{\mathfrak{g}}+[x,Tv,z]_{\mathfrak{g}}+[x,y,Tw]_{\mathfrak{g}}+[x,Tv,Tw]_{\mathfrak{g}}+[Tu,y,Tw]_{\mathfrak{g}}+[Tu,Tv,z]_{\mathfrak{g}}$
$\displaystyle+T\Big{(}\rho(Tu,Tv)w+\rho(Tv,Tw)u+\rho(Tw,Tu)v+\lambda[u,v,w]_{\mathfrak{h}}\Big{)},$

which implies that $\overline{T}$ is a Nijenhuis operator on the semidirect
product $3$-Lie algebra $\mathfrak{g}\ltimes_{\rho}\mathfrak{h}$ if and only
if (6) is satisfied. ∎

At the end of this section, we show that certain projections provide a class
of examples of relative Rota-Baxter operators on 3-Lie algebras.

######  Proposition 2.13.

Let $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ be a $3$-Lie algebra
such that the adjoint representation
$\mathrm{ad}:\wedge^{2}\mathfrak{g}\rightarrow\mathfrak{gl}(\mathfrak{g})$ is
an action of the $3$-Lie algebra $\mathfrak{g}$ on itself. Let $\mathfrak{h}$
be an abelian $3$-Lie subalgebra of $\mathfrak{g}$ satisfying
$\mathfrak{g}^{1}\cap\mathfrak{h}=0.$ Let $\mathfrak{k}$ be a complement of
$\mathfrak{h}$ such that $\mathfrak{g}=\mathfrak{k}\oplus\mathfrak{h}$ as a
vector space. Then the projection $P:\mathfrak{g}\rightarrow\mathfrak{g}$ onto
the subspace $\mathfrak{h}$, i.e. $P(k,u)=u$ for all
$k\in\mathfrak{k},~{}u\in\mathfrak{h}$, is a relative Rota-Baxter operator of
weight $\lambda$ from $\mathfrak{g}$ to $\mathfrak{g}$ with respect to the
adjoint action $\mathrm{ad}$.

###### Proof.

For all $x,y,z\in\mathfrak{g},$ denote by
$\overline{x},\overline{y},\overline{z}$ their images under the projection
$P$. Since $\mathfrak{h}$ is abelian and $\mathfrak{g}^{1}\cap\mathfrak{h}=0,$
we have

$\displaystyle[P(x),P(y),P(z)]_{\mathfrak{g}}-P\Big{(}[P(x),P(y),z]_{\mathfrak{g}}+[x,P(y),P(z)]_{\mathfrak{g}}+[P(x),y,P(z)]_{\mathfrak{g}}+\lambda[x,y,z]_{\mathfrak{g}}\Big{)}$
$\displaystyle=$
$\displaystyle[\overline{x},\overline{y},\overline{z}]_{\mathfrak{g}}-P\Big{(}[\overline{x},\overline{y},z]_{\mathfrak{g}}+[\overline{x},y,\overline{z}]_{\mathfrak{g}}+[x,\overline{y},\overline{z}]_{\mathfrak{g}}+\lambda[x,y,z]_{\mathfrak{g}}\Big{)}$
$\displaystyle=$ $\displaystyle 0,$

which implies that the projection $P$ is a relative Rota-Baxter operator of
weight $\lambda$ from $\mathfrak{g}$ to $\mathfrak{g}$ with respect to the
adjoint action $\mathrm{ad}$. ∎

######  Example 2.14.

Let $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ be a $4$-dimensional
$3$-Lie algebra with a basis $\\{e_{1},e_{2},e_{3},e_{4}\\}$ and the nonzero
multiplication is given by

$[e_{2},e_{3},e_{4}]=e_{1}.$

The center of $\mathfrak{g}$ is the subspace generated by $\\{e_{1}\\}.$ It is
obvious that the adjoint representation
$\mathrm{ad}:\wedge^{2}\mathfrak{g}\rightarrow\mathfrak{gl}(\mathfrak{g})$ is
an action of $\mathfrak{g}$ on itself. Let $\mathfrak{h}$ be a subalgebra of
$\mathfrak{g}$ generated by $\\{e_{3},e_{4}\\}$. By Proposition 2.13, the
projection $P:\mathfrak{g}\rightarrow\mathfrak{g}$ given by
$\begin{cases}P(e_{1})=0,\\\ P(e_{2})=0,\\\ P(e_{3})=e_{3},\\\
P(e_{4})=e_{4},\end{cases}$ is a relative Rota-Baxter operator of weight
$\lambda$ from $\mathfrak{g}$ to $\mathfrak{g}$ with respect to the adjoint
action $\mathrm{ad}$.

##  3\. $3$-post-Lie algebras

In this section, we introduce the notion of $3$-post-Lie algebras. A relative
Rota-Baxter operator of nonzero weight induces a $3$-post-Lie algebra
naturally. Therefore, $3$-post-Lie algebras can be viewed as the underlying
algebraic structures of relative Rota-Baxter operators of weight $\lambda$ on
$3$-Lie algebras. A $3$-post-Lie algebra also gives rise to a new 3-Lie
algebra and an action on the original 3-Lie algebra, which leads to the fact
that the identity map is a relative Rota-Baxter operator of weight 1.

######  Definition 3.1.

A $3$-post-Lie algebra $(A,\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot])$
consists of a $3$-Lie algebra $(A,[\cdot,\cdot,\cdot])$ and a ternary product
$\\{\cdot,\cdot,\cdot\\}:A\otimes A\otimes A\rightarrow A$ such that

(12) $\displaystyle\\{x_{1},x_{2},x_{3}\\}$ $\displaystyle=$
$\displaystyle-\\{x_{2},x_{1},x_{3}\\},$ (13)
$\displaystyle\\{x_{1},x_{2},\\{x_{3},x_{4},x_{5}\\}\\}$ $\displaystyle=$
$\displaystyle\\{x_{3},x_{4},\\{x_{1},x_{2},x_{5}\\}\\}+\\{\left\llbracket
x_{1},x_{2},x_{3}\right\rrbracket,x_{4},x_{5}\\}+\\{x_{3},\left\llbracket
x_{1},x_{2},x_{4}\right\rrbracket,x_{5}\\},$ (14)
$\displaystyle\qquad\\{\left\llbracket
x_{1},x_{2},x_{3}\right\rrbracket,x_{4},x_{5}\\}$ $\displaystyle=$
$\displaystyle\\{x_{1},x_{2},\\{x_{3},x_{4},x_{5}\\}\\}+\\{x_{2},x_{3},\\{x_{1},x_{4},x_{5}\\}\\}+\\{x_{3},x_{1},\\{x_{2},x_{4},x_{5}\\}\\},$
(15) $\displaystyle\\{x_{1},x_{2},[x_{3},x_{4},x_{5}]\\}$ $\displaystyle=$
$\displaystyle 0,$ (16) $\displaystyle[\\{x_{1},x_{2},x_{3}\\},x_{4},x_{5}]$
$\displaystyle=$ $\displaystyle 0,$

for all $x_{i}\in A,1\leq i\leq 5$ and
$\left\llbracket\cdot,\cdot,\cdot\right\rrbracket$ is defined by

(17) $\displaystyle\left\llbracket
x,y,z\right\rrbracket=\\{x,y,z\\}+\\{y,z,x\\}+\\{z,x,y\\}+[x,y,z],\quad\forall
x,y,z\in A.$

######  Remark 3.2.

Let $(A,\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot])$ be a $3$-post-Lie
algebra. If the $3$-Lie bracket $[\cdot,\cdot,\cdot]=0$, then
$(A,\\{\cdot,\cdot,\cdot\\})$ becomes a $3$-pre-Lie algebra which was
introduced in [4] in the study of the $3$-Lie Yang-Baxter equation.

######  Definition 3.3.

A homomorphism from a $3$-post-Lie algebra
$(A,\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot])$ to
$(A^{\prime},\\{\cdot,\cdot,\cdot\\}^{\prime},[\cdot,\cdot,\cdot]^{\prime})$
is a linear map $\psi:A\,\rightarrow\,A^{\prime}$ satisfying

(18) $\displaystyle\psi(\\{x,y,z\\})$ $\displaystyle=$
$\displaystyle\\{\psi(x),\psi(y),\psi(z)\\}^{\prime},$ (19)
$\displaystyle\psi([x,y,z])$ $\displaystyle=$
$\displaystyle[\psi(x),\psi(y),\psi(z)]^{\prime},$

for all $x,y,z\in A.$

######  Theorem 3.4.

Let $(A,\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot])$ be a $3$-post-Lie
algebra. Then

  * (i)

$(A,\left\llbracket\cdot,\cdot,\cdot\right\rrbracket)$ is a $3$-Lie algebra,
which is called the sub-adjacent $3$-Lie algebra of the $3$-post Lie algebra
$(A,\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot])$, where
$\left\llbracket\cdot,\cdot,\cdot\right\rrbracket$ is defined by (17).

  * (ii)

$(A;L)$ is an action of the $3$-Lie algebra
$(A,\left\llbracket\cdot,\cdot,\cdot\right\rrbracket)$ on the $3$-Lie algebra
$(A,[\cdot,\cdot,\cdot])$, where the action
$L:\otimes^{2}A\rightarrow\mathfrak{gl}(A)$ is defined by

$L(x,y)z=\\{x,y,z\\},\quad\forall~{}x,y,z\in A.$

  * (iii)

The identity map ${\rm{Id}}:A\to A$ is a relative Rota-Baxter operator of
weight $1$ from the $3$-Lie algebra $(A,[\cdot,\cdot,\cdot])$ to the $3$-Lie
algebra $(A,\left\llbracket\cdot,\cdot,\cdot\right\rrbracket)$ with respect to
the action $L.$

###### Proof.

(i) By (12), it is obvious that the operation
$\left\llbracket\cdot,\cdot,\cdot\right\rrbracket$ given by (17) is skew-
symmetric. For all $x_{1},x_{2},x_{3},x_{4},x_{5}\in A,$ by (13)-(16), we have

$\displaystyle\left\llbracket x_{1},x_{2},\left\llbracket
x_{3},x_{4},x_{5}\right\rrbracket\right\rrbracket-\left\llbracket\left\llbracket
x_{1},x_{2},x_{3}\right\rrbracket,x_{4},x_{5}\right\rrbracket$
$\displaystyle-\left\llbracket x_{3},\left\llbracket
x_{1},x_{2},x_{4}\right\rrbracket,x_{5}\right\rrbracket-\left\llbracket
x_{3},x_{4},\left\llbracket x_{1},x_{2},x_{5}\right\rrbracket\right\rrbracket$
$\displaystyle=$
$\displaystyle\\{x_{1},x_{2},\\{x_{3},x_{4},x_{5}\\}\\}+\\{x_{1},x_{2},\\{x_{4},x_{5},x_{3}\\}\\}+\\{x_{1},x_{2},\\{x_{5},x_{3},x_{4}\\}\\}$
$\displaystyle+\\{x_{1},x_{2},[x_{3},x_{4},x_{5}]\\}+\\{\left\llbracket
x_{3},x_{4},x_{5}\right\rrbracket,x_{1},x_{2}\\}+\\{x_{2},\left\llbracket
x_{3},x_{4},x_{5}\right\rrbracket,x_{1}\\}$
$\displaystyle+[x_{1},x_{2},\left\llbracket
x_{3},x_{4},x_{5}\right\rrbracket]-\\{x_{4},x_{5},\\{x_{1},x_{2},x_{3}\\}\\}-\\{x_{4},x_{5},\\{x_{2},x_{3},x_{1}\\}\\}$
$\displaystyle-\\{x_{4},x_{5},\\{x_{3},x_{1},x_{2}\\}\\}-\\{x_{4},x_{5},[x_{1},x_{2},x_{3}]\\}-\\{\left\llbracket
x_{1},x_{2},x_{3}\right\rrbracket,x_{4},x_{5}\\}$
$\displaystyle-\\{x_{5},\left\llbracket
x_{1},x_{2},x_{3}\right\rrbracket,x_{4}\\}-[\left\llbracket
x_{1},x_{2},x_{3}\right\rrbracket,x_{4},x_{5}]-\\{x_{5},x_{3},\\{x_{1},x_{2},x_{4}\\}\\}$
$\displaystyle-\\{x_{5},x_{3},\\{x_{2},x_{4},x_{1}\\}\\}-\\{x_{5},x_{3},\\{x_{4},x_{1},x_{2}\\}\\}-\\{x_{5},x_{3},[x_{1},x_{2},x_{4}]\\}$
$\displaystyle-\\{x_{3},\left\llbracket
x_{1},x_{2},x_{4}\right\rrbracket,x_{5}\\}-\\{\left\llbracket
x_{1},x_{2},x_{4}\right\rrbracket,x_{5},x_{3}\\}-[x_{3},\left\llbracket
x_{1},x_{2},x_{4}\right\rrbracket,x_{5}]$
$\displaystyle-\\{x_{3},x_{4},\\{x_{1},x_{2},x_{5}\\}\\}-\\{x_{3},x_{4},\\{x_{5},x_{1},x_{2}\\}\\}-\\{x_{3},x_{4},\\{x_{2},x_{5},x_{1}\\}\\}$
$\displaystyle-\\{x_{3},x_{4},[x_{1},x_{2},x_{5}]\\}-\\{\left\llbracket
x_{1},x_{2},x_{5}\right\rrbracket,x_{3},x_{4}\\}-\\{x_{4},\left\llbracket
x_{1},x_{2},x_{5}\right\rrbracket,x_{3}\\}$
$\displaystyle-[x_{3},x_{4},\left\llbracket
x_{1},x_{2},x_{5}\right\rrbracket]$ $\displaystyle=$ $\displaystyle 0,$

which implies that $(A,\left\llbracket\cdot,\cdot,\cdot\right\rrbracket)$ is a
$3$-Lie algebra.

(ii) By (13) and (14), we have

$\displaystyle[L(x_{1},x_{2}),L(x_{3},x_{4})](x_{5})$ $\displaystyle=$
$\displaystyle L(\left\llbracket
x_{1},x_{2},x_{3}\right\rrbracket,x_{4})(x_{5})+L(x_{3},\left\llbracket
x_{1},x_{2},x_{4}\right\rrbracket)(x_{5}),$ $\displaystyle L(\left\llbracket
x_{1},x_{2},x_{3}\right\rrbracket,x_{4})(x_{5})$ $\displaystyle=$
$\displaystyle\Big{(}L(x_{1},x_{2})L(x_{3},x_{4})+L(x_{2},x_{3})L(x_{1},x_{4})$
$\displaystyle+L(x_{3},x_{1})L(x_{2},x_{4})\Big{)}(x_{5}),$

which implies that $L$ is a representation of the 3-Lie algebra
$(A,\left\llbracket\cdot,\cdot,\cdot\right\rrbracket)$ on the vector space
$A$. By (16), $L(x_{1},x_{2})x_{3}$ is in the center of the $3$-Lie algebra
$(A,[\cdot,\cdot,\cdot])$. Then by (15), we have
$L(x_{1},x_{2})[x_{3},x_{4},x_{5}]=0.$ Therefore, $(A;L)$ is an action of the
$3$-Lie algebra $(A,\left\llbracket\cdot,\cdot,\cdot\right\rrbracket)$ on the
$3$-Lie algebra $(A,[\cdot,\cdot,\cdot])$.

(iii) It follows from the definition of the subadjacent 3-Lie bracket
$\left\llbracket\cdot,\cdot,\cdot\right\rrbracket$ directly. ∎

######  Corollary 3.5.

Let $\psi:A\,\rightarrow\,A^{\prime}$ be a homomorphism from a $3$-post-Lie
algebra $(A,\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot])$ to
$(A^{\prime},\\{\cdot,\cdot,\cdot\\}^{\prime},[\cdot,\cdot,\cdot]^{\prime})$.
Then $\psi$ is also a homomorphism from the sub-adjacent $3$-Lie algebra
$(A,\left\llbracket\cdot,\cdot,\cdot\right\rrbracket)$ to
$(A^{\prime},\left\llbracket\cdot,\cdot,\cdot\right\rrbracket^{\prime})$.

The following results illustrate that $3$-post-Lie algebras can be viewed as
the underlying algebraic structures of relative Rota-Baxter operators of
weight $\lambda$ on $3$-Lie algebras.

######  Theorem 3.6.

Let $T:\mathfrak{h}\rightarrow\mathfrak{g}$ be a relative Rota-Baxter operator
of weight $\lambda$ from a $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}})$ to a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ with respect to an action
$\rho$.

  * (i)

$(\mathfrak{h},\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot])$ is a $3$-post-Lie
algebra, where $\\{\cdot,\cdot,\cdot\\}$ and $[\cdot,\cdot,\cdot]$ are given
by

(20)
$\displaystyle\qquad\\{u_{1},u_{2},u_{3}\\}=\rho(Tu_{1},Tu_{2})u_{3},\quad[u_{1},u_{2},u_{3}]=\lambda[u_{1},u_{2},u_{3}]_{\mathfrak{h}},\quad\forall
u_{1},u_{2},u_{3}\in\mathfrak{h}.$

  * (ii)

$T$ is a $3$-Lie algebra homomorphism from the sub-adjacent $3$-Lie algebra
$(\mathfrak{h},\left\llbracket\cdot,\cdot,\cdot\right\rrbracket)$ to
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$, where the $3$-Lie bracket
$\left\llbracket\cdot,\cdot,\cdot\right\rrbracket$ is defined by

$\displaystyle\left\llbracket
u_{1},u_{2},u_{3}\right\rrbracket=\rho(Tu_{1},Tu_{2})u_{3}+\rho(Tu_{2},Tu_{3})u_{1}+\rho(Tu_{3},Tu_{1})u_{2}+\lambda[u_{1},u_{2},u_{3}]_{\mathfrak{h}}.$

###### Proof.

(i) For any $u_{1},u_{2},u_{3}\in\mathfrak{h},$ it is obvious that

$\displaystyle\\{u_{1},u_{2},u_{3}\\}=\rho(Tu_{1},Tu_{2})u_{3}=-\rho(Tu_{2},Tu_{1})u_{3}=-\\{u_{2},u_{1},u_{3}\\}.$

Furthermore, for $u_{1},u_{2},u_{3},u_{4},u_{5}\in\mathfrak{h}$, by (2), (6)
and (17), we have

$\displaystyle\\{u_{1},u_{2},\\{u_{3},u_{4},u_{5}\\}\\}-\\{u_{3},u_{4},\\{u_{1},u_{2},u_{5}\\}\\}$
$\displaystyle\quad-\\{\left\llbracket
u_{1},u_{2},u_{3}\right\rrbracket,u_{4},u_{5}\\}-\\{u_{3},\left\llbracket
u_{1},u_{2},u_{4}\right\rrbracket,u_{5}\\}$ $\displaystyle=$
$\displaystyle\\{u_{1},u_{2},\rho(Tu_{3},Tu_{4})u_{5}\\}-\\{u_{3},u_{4},\rho(Tu_{1},Tu_{2})u_{5}\\}$
$\displaystyle-\\{\rho(Tu_{1},Tu_{2})u_{3}+\rho(Tu_{2},Tu_{3})u_{1}+\rho(Tu_{3},Tu_{1})u_{2}+\lambda[u_{1},u_{2},u_{3}]_{\mathfrak{h}},u_{4},u_{5}\\}$
$\displaystyle-\\{u_{3},\rho(Tu_{1},Tu_{2})u_{4}+\rho(Tu_{2},Tu_{4})u_{1}+\rho(Tu_{4},Tu_{1})u_{2}+\lambda[u_{1},u_{2},u_{3}]_{\mathfrak{h}},u_{5}\\}$
$\displaystyle=$
$\displaystyle\rho(Tu_{1},Tu_{2})\rho(Tu_{3},Tu_{4})u_{5}-\rho(Tu_{3},Tu_{4})\rho(Tu_{1},Tu_{2})u_{5}$
$\displaystyle-\rho\Big{(}T\rho(Tu_{1},Tu_{2})u_{3}+T\rho(Tu_{2},Tu_{3})u_{1}+T\rho(Tu_{3},Tu_{1})u_{2}+T\lambda[u_{1},u_{2},u_{3}]_{\mathfrak{h}},Tu_{4}\Big{)}u_{5}$
$\displaystyle-\rho\Big{(}Tu_{3},T\rho(Tu_{1},Tu_{2})u_{4}+T\rho(Tu_{2},Tu_{4})u_{1}+T\rho(Tu_{4},Tu_{1})u_{2}+T\lambda[u_{1},u_{2},u_{4}]_{\mathfrak{h}}\Big{)}u_{5}$
$\displaystyle=$ $\displaystyle 0.$

This implies that (13) in Definition 3.1 holds. Similarly, by (3), we can
verify that (14) holds. Moreover, by (4) and (5), (15) and (16) hold, too.
Hence $(\mathfrak{h},\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot])$ is a
$3$-post-Lie algebra.

(ii) Note that the sub-adjacent $3$-Lie algebra of the above $3$-post-Lie
algebra $(\mathfrak{h},\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot])$ is
exactly the $3$-Lie algebra $(\mathfrak{h},[\cdot,\cdot,\cdot]_{T})$ given in
Corollary 2.11. Then the result follows. ∎

######  Proposition 3.7.

Let $T$ and $T^{\prime}$ be two relative Rota-Baxter operators of weight
$\lambda$ from a $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}})$ to a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ with respect to an action
$\rho$. Let $(\mathfrak{h},\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot])$ and
$(\mathfrak{h},\\{\cdot,\cdot,\cdot\\}^{\prime},[\cdot,\cdot,\cdot]^{\prime})$
be the induced $3$-post-Lie algebras and
$(\psi_{\mathfrak{g}},\psi_{\mathfrak{h}})$ be a homomorphism from
$T^{\prime}$ to $T$. Then $\psi_{\mathfrak{h}}$ is a homomorphism from the
$3$-post-Lie algebra
$(\mathfrak{h},\\{\cdot,\cdot,\cdot\\},[\cdot,\cdot,\cdot])$ to the $3$-post-
Lie algebra
$(\mathfrak{h},\\{\cdot,\cdot,\cdot\\}^{\prime},[\cdot,\cdot,\cdot]^{\prime})$.

###### Proof.

For all $u,v,w\in\mathfrak{h},$ by (7)-(8) and (20), we have

$\displaystyle\psi_{\mathfrak{h}}(\\{u,v,w\\})$ $\displaystyle=$
$\displaystyle\psi_{\mathfrak{h}}(\rho(Tu,Tv)w)$ $\displaystyle=$
$\displaystyle\rho(\psi_{\mathfrak{g}}(Tu),\psi_{\mathfrak{g}}(Tv))\psi_{\mathfrak{h}}(w)$
$\displaystyle=$
$\displaystyle\rho(T^{\prime}\psi_{\mathfrak{h}}(u),T^{\prime}\psi_{\mathfrak{h}}(v))\psi_{\mathfrak{h}}(w)$
$\displaystyle=$
$\displaystyle\\{\psi_{\mathfrak{h}}(u),\psi_{\mathfrak{h}}(v),\psi_{\mathfrak{h}}(w)\\}^{\prime},$
$\displaystyle\psi_{\mathfrak{h}}([u,v,w])$ $\displaystyle=$
$\displaystyle\psi_{\mathfrak{h}}\lambda[u,v,w]_{\mathfrak{h}}$
$\displaystyle=$
$\displaystyle\lambda[\psi_{\mathfrak{h}}(u),\psi_{\mathfrak{h}}(v),\psi_{\mathfrak{h}}(w)]_{\mathfrak{h}}$
$\displaystyle=$ $\displaystyle[\psi(u),\psi(v),\psi(w)]^{\prime},$

which implies that $\psi_{\mathfrak{h}}$ is a homomorphism between the induced
3-post-Lie algebras. ∎

Thus, Theorem 3.6 can be enhanced to a functor from the category of relative
Rota-Baxter operators of weight $\lambda$ on $3$-Lie algebras to the category
of $3$-post-Lie algebras.

##  4\. Maurer-Cartan characterization of relative Rota-Baxter operators of
weight $\lambda$

In this section, given an action of 3-Lie algebras, we construct an
$L_{\infty}$-algebra whose Maurer-Cartan elements are relative Rota-Baxter
operators of weight $\lambda$ on $3$-Lie algebras. Then we obtain the
$L_{\infty}$-algebra that controls deformations of relative Rota-Baxter
operators of weight $\lambda$ on $3$-Lie algebras. This fact can be viewed as
certain justification of relative Rota-Baxter operators of weight $\lambda$ on
$3$-Lie algebras being interesting structures.

######  Definition 4.1.

An $L_{\infty}$-algebra is a $\mathbb{Z}$-graded vector space
$\mathfrak{g}=\oplus_{k\in\mathbb{Z}}\mathfrak{g}^{k}$ equipped with a
collection $(k\geq 1)$ of linear maps
$l_{k}:\otimes^{k}\mathfrak{g}\,\rightarrow\,\mathfrak{g}$ of degree $1$ with
the property that, for any homogeneous elements
$x_{1},\cdots,x_{n}\in\mathfrak{g}$, we have

  * (i)

(graded symmetry) for every $\sigma\in\mathbb{S}_{n}$,

$\displaystyle
l_{n}(x_{\sigma(1)},\cdots,x_{\sigma(n-1)},x_{\sigma(n)})=\varepsilon(\sigma)l_{n}(x_{1},\cdots,x_{n-1},x_{n}),$

  * (ii)

(generalized Jacobi Identity) for all $n\geq 1$,

$\displaystyle\sum_{i=1}^{n}\sum_{\sigma\in\mathbb{S}_{(i,n-i)}}\varepsilon(\sigma)l_{n-i+1}(l_{i}(x_{\sigma(1)},\cdots,x_{\sigma(i)}),x_{\sigma(i+1)},\cdots,x_{\sigma(n)})=0.$

######  Definition 4.2.

A Maurer-Cartan element of an $L_{\infty}$-algebra
$(\mathfrak{g}=\oplus_{k\in\mathbb{Z}}\mathfrak{g}^{k},\\{l_{i}\\}_{i=1}^{+\infty})$
is an element $\alpha\in\mathfrak{g}^{0}$ satisfying the Maurer-Cartan
equation

(21)
$\displaystyle\sum_{n=1}^{+\infty}\frac{1}{n!}l_{n}(\alpha,\cdots,\alpha)=0.$

Let $\alpha$ be a Maurer-Cartan element of an $L_{\infty}$-algebra
$(\mathfrak{g},\\{l_{i}\\}_{i=1}^{+\infty})$. For all $k\geq 1$ and
$x_{1},\cdots,x_{n}\in\mathfrak{g},$ define a series of linear maps
$l_{k}^{\alpha}:\otimes^{k}\mathfrak{g}\,\rightarrow\,\mathfrak{g}$ of degree
$1$ by

(22) $\displaystyle
l^{\alpha}_{k}(x_{1},\cdots,x_{k})=\sum^{+\infty}_{n=0}\frac{1}{n!}l_{n+k}\\{\underbrace{\alpha,\cdots,\alpha}_{n},x_{1},\cdots,x_{k}\\}.$

######  Theorem 4.3.

([20]) With the above notations,
$(\mathfrak{g},\\{l^{\alpha}_{i}\\}_{i=1}^{+\infty})$ is an
$L_{\infty}$-algebra, obtained from the $L_{\infty}$-algebra
$(\mathfrak{g},\\{l_{i}\\}_{i=1}^{+\infty})$ by twisting with the Maurer-
Cartan element $\alpha$. Moreover, $\alpha+\alpha^{\prime}$ is a Maurer-Cartan
element of $(\mathfrak{g},\\{l_{i}\\}_{i=1}^{+\infty})$ if and only if
$\alpha^{\prime}$ is a Maurer-Cartan element of the twisted
$L_{\infty}$-algebra $(\mathfrak{g},\\{l^{\alpha}_{i}\\}_{i=1}^{+\infty})$.

In [38], Th. Voronov developed the theory of higher derived brackets, which is
a useful tool to construct explicit $L_{\infty}$-algebras.

######  Definition 4.4.

([38]) A $V$-data consists of a quadruple $(L,F,\mathcal{P},\Delta)$, where

  * $\bullet$

$(L,[\cdot,\cdot])$ is a graded Lie algebra,

  * $\bullet$

$F$ is an abelian graded Lie subalgebra of $(L,[\cdot,\cdot])$,

  * $\bullet$

$\mathcal{P}:L\,\rightarrow\,L$ is a projection, that is
$\mathcal{P}\circ\mathcal{P}=\mathcal{P}$, whose image is $F$ and kernel is a
graded Lie subalgebra of $(L,[\cdot,\cdot])$,

  * $\bullet$

$\Delta$ is an element in $\ker(\mathcal{P})^{1}$ such that
$[\Delta,\Delta]=0$.

######  Theorem 4.5.

([38]) Let $(L,F,\mathcal{P},\Delta)$ be a $V$-data. Then
$(F,\\{{l_{k}}\\}_{k=1}^{+\infty})$ is an $L_{\infty}$-algebra, where

(23) $\displaystyle
l_{k}(a_{1},\cdots,a_{k})=\mathcal{P}\underbrace{[\cdots[[}_{k}\Delta,a_{1}],a_{2}],\cdots,a_{k}],\quad\mbox{for
homogeneous}~{}a_{1},\cdots,a_{k}\in F.$

We call $\\{{l_{k}}\\}_{k=1}^{+\infty}$ the higher derived brackets of the
$V$-data $(L,F,\mathcal{P},\Delta)$.

Let $\mathfrak{g}$ be a vector space. We consider the graded vector space

$C^{*}(\mathfrak{g},\mathfrak{g})=\oplus_{n\geq
0}C^{n}(\mathfrak{g},\mathfrak{g})=\oplus_{n\geq
0}\mathrm{Hom}(\underbrace{\wedge^{2}\mathfrak{g}\otimes\cdots\otimes\wedge^{2}\mathfrak{g}}_{n}\wedge\mathfrak{g},\mathfrak{g}).$

######  Theorem 4.6.

([32]) The graded vector space $C^{*}(\mathfrak{g},\mathfrak{g})$ equipped
with the graded commutator bracket

(24)
$\displaystyle[P,Q]_{\mathsf{R}}=P{\circ}Q-(-1)^{pq}Q{\circ}P,\quad\forall~{}P\in
C^{p}(\mathfrak{g},\mathfrak{g}),Q\in C^{q}(\mathfrak{g},\mathfrak{g}),$

is a graded Lie algebra, where $P{\circ}Q\in
C^{p+q}(\mathfrak{g},\mathfrak{g})$ is defined by

$\displaystyle(P{\circ}Q)(\mathfrak{X}_{1},\cdots,\mathfrak{X}_{p+q},x)$
$\displaystyle=$
$\displaystyle\sum_{k=1}^{p}(-1)^{(k-1)q}\sum_{\sigma\in\mathbb{S}(k-1,q)}(-1)^{\sigma}P\Big{(}\mathfrak{X}_{\sigma(1)},\cdots,\mathfrak{X}_{\sigma(k-1)},Q\big{(}\mathfrak{X}_{\sigma(k)},\cdots,\mathfrak{X}_{\sigma(k+q-1)},x_{k+q}\big{)}\wedge
y_{k+q},\mathfrak{X}_{k+q+1},\cdots,\mathfrak{X}_{p+q},x\Big{)}$
$\displaystyle+\sum_{k=1}^{p}(-1)^{(k-1)q}\sum_{\sigma\in\mathbb{S}(k-1,q)}(-1)^{\sigma}P\Big{(}\mathfrak{X}_{\sigma(1)},\cdots,\mathfrak{X}_{\sigma(k-1)},x_{k+q}\wedge
Q\big{(}\mathfrak{X}_{\sigma(k)},\cdots,\mathfrak{X}_{\sigma(k+q-1)},y_{k+q}\big{)},\mathfrak{X}_{k+q+1},\cdots,\mathfrak{X}_{p+q},x\Big{)}$
$\displaystyle+\sum_{\sigma\in\mathbb{S}(p,q)}(-1)^{pq}(-1)^{\sigma}P\Big{(}\mathfrak{X}_{\sigma(1)},\cdots,\mathfrak{X}_{\sigma(p)},Q\big{(}\mathfrak{X}_{\sigma(p+1)},\cdots,\mathfrak{X}_{\sigma(p+q-1)},\mathfrak{X}_{\sigma(p+q)},x\big{)}\Big{)},$

for all $\mathfrak{X}_{i}=x_{i}\wedge y_{i}\in\wedge^{2}\mathfrak{g}$,
$~{}i=1,2,\cdots,p+q$ and $x\in\mathfrak{g}.$

Moreover, $\mu:\wedge^{3}\mathfrak{g}\longrightarrow\mathfrak{g}$ is a $3$-Lie
bracket if and only if $[\mu,\mu]_{\mathsf{R}}=0$, i.e. $\mu$ is a Maurer-
Cartan element of the graded Lie algebra
$(C^{*}(\mathfrak{g},\mathfrak{g}),[\cdot,\cdot]_{\mathsf{R}})$.

Let $\rho:\wedge^{2}\mathfrak{g}\rightarrow\mathfrak{gl}(\mathfrak{h})$ be an
action of a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ on a $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}})$. For convenience, we use
$\pi:\wedge^{3}\mathfrak{g}\rightarrow\mathfrak{g}$ to indicate the $3$-Lie
bracket $[\cdot,\cdot,\cdot]_{\mathfrak{g}}$ and
$\mu:\wedge^{3}\mathfrak{h}\rightarrow\mathfrak{h}$ to indicate the $3$-Lie
bracket $[\cdot,\cdot,\cdot]_{\mathfrak{h}}$. In the sequel, we use
$\pi+\rho+\lambda\mu$ to denote the element in
$\mathrm{Hom}(\wedge^{3}(\mathfrak{g}\oplus\mathfrak{h}),\mathfrak{g}\oplus\mathfrak{h})$
given by

(25)
$(\pi+\rho+\lambda\mu)(x+u,y+v,z+w)=[x,y,z]_{\mathfrak{g}}+\rho(x,y)w+\rho(y,z)u+\rho(z,x)v+\lambda[u,v,w]_{\mathfrak{h}},$

for all
$x,y,z\in\mathfrak{g},~{}u,v,w\in\mathfrak{h},~{}\lambda\in\mathbb{K}.$ Note
that the right hand side is exactly the semidirect product 3-Lie algebra
structure given in Proposition 2.9. Therefore by Theorem 4.6, we have

$[\pi+\rho+\lambda\mu,\pi+\rho+\lambda\mu]_{\mathsf{R}}=0.$

######  Proposition 4.7.

Let $\rho:\wedge^{2}\mathfrak{g}\rightarrow\mathfrak{gl}(\mathfrak{h})$ be an
action of a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ on a $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}})$. Then we have a $V$-data
$(L,F,\mathcal{P},\Delta)$ as follows:

  * $\bullet$

the graded Lie algebra $(L,[\cdot,\cdot])$ is given by
$(C^{*}(\mathfrak{g}\oplus\mathfrak{h},\mathfrak{g}\oplus\mathfrak{h}),[\cdot,\cdot]_{\mathsf{R}})$;

  * $\bullet$

the abelian graded Lie subalgebra $F$ is given by

$F=C^{*}(\mathfrak{h},\mathfrak{g})=\oplus_{n\geq
0}C^{n}(\mathfrak{h},\mathfrak{g})=\oplus_{n\geq
0}\mathrm{Hom}(\underbrace{\wedge^{2}\mathfrak{h}\otimes\cdots\otimes\wedge^{2}\mathfrak{h}}_{n}\wedge\mathfrak{h},\mathfrak{g});$

  * $\bullet$

$\mathcal{P}:L\,\rightarrow\,L$ is the projection onto the subspace $F$;

  * $\bullet$

$\Delta=\pi+\rho+\lambda\mu.$

Consequently, we obtain an $L_{\infty}$-algebra
$(C^{*}(\mathfrak{h},\mathfrak{g}),l_{1},l_{3})$, where

$\displaystyle l_{1}(P)$ $\displaystyle=$
$\displaystyle\mathcal{P}[\pi+\rho+\lambda\mu,P]_{\mathsf{R}},$ $\displaystyle
l_{3}(P,Q,R)$ $\displaystyle=$
$\displaystyle\mathcal{P}[[[\pi+\rho+\lambda\mu,P]_{\mathsf{R}},Q]_{\mathsf{R}},R]_{\mathsf{R}},$

for all $P\in C^{m}(\mathfrak{h},\mathfrak{g}),Q\in
C^{n}(\mathfrak{h},\mathfrak{g})$ and $R\in C^{k}(\mathfrak{h},\mathfrak{g}).$

###### Proof.

By Theorem 4.5, $(F,\\{l_{k}\\}^{\infty}_{k=1})$ is an $L_{\infty}$-algebra,
where $l_{k}$ is given by (23). It is obvious that
$\Delta=\pi+\rho+\lambda\mu\in\ker(\mathcal{P})^{1}$. For all $P\in
C^{m}(\mathfrak{h},\mathfrak{g}),Q\in C^{n}(\mathfrak{h},\mathfrak{g})$ and
$R\in C^{k}(\mathfrak{h},\mathfrak{g})$, we have

$\displaystyle[[\pi+\rho+\lambda\mu,P]_{\mathsf{R}},Q]_{\mathsf{R}}\in\ker(\mathcal{P}),$

which implies that $l_{2}=0$. Similarly, we have $l_{k}=0,$ when $k\geq 4$.
Therefore, the graded vector space $C^{*}(\mathfrak{h},\mathfrak{g})$ is an
$L_{\infty}$-algebra with nontrivial $l_{1}$, $l_{3},$ and other maps are
trivial. ∎

######  Theorem 4.8.

Let $\rho:\wedge^{2}\mathfrak{g}\rightarrow\mathfrak{gl}(\mathfrak{h})$ be an
action of a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ on a $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}})$. Then Maurer-Cartan
elements of the $L_{\infty}$-algebra
$(C^{*}(\mathfrak{h},\mathfrak{g}),l_{1},l_{3})$ are precisely relative Rota-
Baxter operators of weight $\lambda$ from the $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}})$ to the $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ with respect to the action
$\rho$.

###### Proof.

It is straightforward to deduce that

$\displaystyle\mathcal{P}[\pi+\rho+\lambda\mu,T]_{\mathsf{R}}(u,v,w)=-T\lambda\mu(u,v,w),$
$\displaystyle\mathcal{P}[[[\pi+\rho+\lambda\mu,T]_{\mathsf{R}},T]_{\mathsf{R}},T]_{\mathsf{R}}(u,v,w)$
$\displaystyle=$ $\displaystyle
6\Big{(}\pi(Tu,Tv,Tw)-T(\rho(Tu,Tv)w+\rho(Tv,Tw)u+\rho(Tw,Tu)v)\Big{)}.$

Let $T$ be a Maurer-Cartan element of the $L_{\infty}$-algebra
$(C^{*}(\mathfrak{h},\mathfrak{g}),l_{1},l_{3})$. We have

$\displaystyle\sum_{n=1}^{+\infty}\frac{1}{n!}l_{n}(T,\cdots,T)(u,v,w)$
$\displaystyle=$
$\displaystyle\mathcal{P}[\pi+\rho+\lambda\mu,T]_{\mathsf{R}}(u,v,w)+\frac{1}{3!}\mathcal{P}[[[\pi+\rho+\lambda\mu,T]_{\mathsf{R}},T]_{\mathsf{R}},T]_{\mathsf{R}}(u,v,w)$
$\displaystyle=$
$\displaystyle\pi(Tu,Tv,Tw)-T\Big{(}\rho(Tu,Tv)w+\rho(Tv,Tw)u+\rho(Tw,Tu)v+\lambda\mu(u,v,w)\Big{)}$
$\displaystyle=$ $\displaystyle 0,$

which implies that $T$ is a relative Rota-Baxter operator of weight $\lambda$
from the $3$-Lie algebra $(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}})$
to the $3$-Lie algebra $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$
with respect to the action $\rho$. ∎

######  Proposition 4.9.

Let $T$ be a relative Rota-Baxter operator of weight $\lambda$ from a $3$-Lie
algebra $\mathfrak{h}$ to a $3$-Lie algebra $\mathfrak{g}$ with respect to an
action $\rho$. Then $C^{*}(\mathfrak{h},\mathfrak{g})$ carries a twisted
$L_{\infty}$-algebra structure as following:

(26) $\displaystyle l_{1}^{T}(P)$ $\displaystyle=$ $\displaystyle
l_{1}(P)+\frac{1}{2}l_{3}(T,T,P),$ (27) $\displaystyle l_{2}^{T}(P,Q)$
$\displaystyle=$ $\displaystyle l_{3}(T,P,Q),$ (28) $\displaystyle
l_{3}^{T}(P,Q,R)$ $\displaystyle=$ $\displaystyle l_{3}(P,Q,R),$ (29)
$\displaystyle l^{T}_{k}$ $\displaystyle=$ $\displaystyle 0,\,\,\,\,k\geq 4,$

where $P\in C^{m}(\mathfrak{h},\mathfrak{g}),Q\in
C^{n}(\mathfrak{h},\mathfrak{g})$ and $R\in C^{k}(\mathfrak{h},\mathfrak{g})$.

###### Proof.

Since $T$ is a Maurer-Cartan element of the $L_{\infty}$-algebra
$(C^{*}(\mathfrak{h},\mathfrak{g}),l_{1},l_{3})$, by Theorem 4.3, we have the
conclusions. ∎

The above $L_{\infty}$-algebra controls deformations of relative Rota-Baxter
operators of weight $\lambda$ on $3$-Lie algebras.

######  Theorem 4.10.

Let $T:\mathfrak{h}\rightarrow\mathfrak{g}$ be a relative Rota-Baxter operator
of weight $\lambda$ from a $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}})$ to a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ with respect to an action
$\rho$. Then for a linear map
$T^{\prime}:\mathfrak{h}\rightarrow\mathfrak{g}$, $T+T^{\prime}$ is a relative
Rota-Baxter operator if and only if $T^{\prime}$ is a Maurer-Cartan element of
the twisted $L_{\infty}$-algebra
$(C^{*}(\mathfrak{h},\mathfrak{g}),l_{1}^{T},l_{2}^{T},l_{3}^{T})$, that is
$T^{\prime}$ satisfies the Maurer-Cartan equation:

$l_{1}^{T}(T^{\prime})+\frac{1}{2}l_{2}^{T}(T^{\prime},T^{\prime})+\frac{1}{3!}l_{3}^{T}(T^{\prime},T^{\prime},T^{\prime})=0.$

###### Proof.

By Theorem 4.8, $T+T^{\prime}$ is a relative Rota-Baxter operator if and only
if

$l_{1}(T+T^{\prime})+\frac{1}{3!}(T+T^{\prime},T+T^{\prime},T+T^{\prime})=0.$

Applying $l_{1}(T)+\frac{1}{3!}l_{3}(T,T,T)=0,$ the above condition is
equivalent to

$l_{1}(T^{\prime})+\frac{1}{2}l_{3}(T,T,T^{\prime})+\frac{1}{2}l_{3}(T,T^{\prime},T^{\prime})+\frac{1}{6}l_{3}(T^{\prime},T^{\prime},T^{\prime})=0.$

That is,
$l_{1}^{T}(T^{\prime})+\frac{1}{2}l_{2}^{T}(T^{\prime},T^{\prime})+\frac{1}{3!}l_{3}^{T}(T^{\prime},T^{\prime},T^{\prime})=0,$
which implies that $T^{\prime}$ is a Maurer-Cartan element of the twisted
$L_{\infty}$-algebra
$(C^{*}(\mathfrak{h},\mathfrak{g}),l_{1}^{T},l_{2}^{T},l_{3}^{T})$. ∎

##  5\. Cohomologies of relative Rota-Baxter operators and infinitesimal
deformations

Let $(V;\rho)$ be a representation of a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$. Denote by

$\mathfrak{C}_{\mathsf{3Lie}}^{n}(\mathfrak{g};V):=\mathrm{Hom}(\underbrace{\wedge^{2}\mathfrak{g}\otimes\cdots\otimes\wedge^{2}\mathfrak{g}}_{(n-1)}\wedge\mathfrak{g},V),\quad(n\geq
1),$

which is the space of $n$-cochains. The coboundary operator ${\rm
d}:\mathfrak{C}_{\mathsf{3Lie}}^{n}(\mathfrak{g};V)\rightarrow\mathfrak{C}_{\mathsf{3Lie}}^{n+1}(\mathfrak{g};V)$
is defined by

$\displaystyle({\rm d}f)(\mathfrak{X}_{1},\cdots,\mathfrak{X}_{n},x_{n+1})$
$\displaystyle=$ $\displaystyle\sum_{1\leq j<k\leq
n}(-1)^{j}f(\mathfrak{X}_{1},\cdots,\hat{\mathfrak{X}_{j}},\cdots,\mathfrak{X}_{k-1},[x_{j},y_{j},x_{k}]_{\mathfrak{g}}\wedge
y_{k}$
$\displaystyle+x_{k}\wedge[x_{j},y_{j},y_{k}]_{\mathfrak{g}},\mathfrak{X}_{k+1},\cdots,\mathfrak{X}_{n},x_{n+1})$
$\displaystyle+\sum_{j=1}^{n}(-1)^{j}f(\mathfrak{X}_{1},\cdots,\hat{\mathfrak{X}_{j}},\cdots,\mathfrak{X}_{n},[x_{j},y_{j},x_{n+1}]_{\mathfrak{g}})$
$\displaystyle+\sum_{j=1}^{n}(-1)^{j+1}\rho(x_{j},y_{j})f(\mathfrak{X}_{1},\cdots,\hat{\mathfrak{X}_{j}},\cdots,\mathfrak{X}_{n},x_{n+1})$
$\displaystyle+(-1)^{n+1}\Big{(}\rho(y_{n},x_{n+1})f(\mathfrak{X}_{1},\cdots,\mathfrak{X}_{n-1},x_{n})+\rho(x_{n+1},x_{n})f(\mathfrak{X}_{1},\cdots,\mathfrak{X}_{n-1},y_{n})\Big{)},$

for all$~{}\mathfrak{X}_{i}=x_{i}\wedge
y_{i}\in\wedge^{2}\mathfrak{g},~{}i=1,2,\cdots,n~{}and~{}x_{n+1}\in\mathfrak{g}.$
It was proved in [10, 34] that ${\rm d}\circ{\rm d}=0.$ Thus,
$(\oplus_{n=1}^{+\infty}\mathfrak{C}_{\mathsf{3Lie}}^{n}(\mathfrak{g};V),{\rm
d})$ is a cochain complex.

######  Definition 5.1.

The cohomology of the $3$-Lie algebra $\mathfrak{g}$ with coefficients in $V$
is the cohomology of the cochain complex
$(\oplus_{n=1}^{+\infty}\mathfrak{C}_{\mathsf{3Lie}}^{n}(\mathfrak{g};V),{\rm
d})$. Denote by $\mathcal{Z}_{\mathsf{3Lie}}^{n}(\mathfrak{g};V)$ and
$\mathcal{B}_{\mathsf{3Lie}}^{n}(\mathfrak{g};V)$ the set of $n$-cocycles and
the set of $n$-coboundaries, respectively. The $n$-th cohomology group is
defined by

$\displaystyle\mathcal{H}_{\mathsf{3Lie}}^{n}(\mathfrak{g};V)=\mathcal{Z}_{\mathsf{3Lie}}^{n}(\mathfrak{g};V)/\mathcal{B}_{\mathsf{3Lie}}^{n}(\mathfrak{g};V).$

###  5.1. Cohomologies of relative Rota-Baxter operators of weight $\lambda$
on 3-Lie algebras

In this subsection, we construct a representation of the $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{T})$ on the vector space $\mathfrak{g}$
from a relative Rota-Baxter operator $T:\mathfrak{h}\to\mathfrak{g}$ of weight
$\lambda$ and define the cohomologies of relative Rota-Baxter operators of
weight $\lambda$ on $3$-Lie algebras.

######  Lemma 5.2.

Let $T:\mathfrak{h}\rightarrow\mathfrak{g}$ be a relative Rota-Baxter operator
of weight $\lambda$ from a $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}})$ to a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ with respect to an action
$\rho$. Define
$\varrho:\wedge^{2}\mathfrak{h}\rightarrow\mathfrak{gl}(\mathfrak{g})$ by

(30)
$\quad\varrho(u,v)(x)=[Tu,Tv,x]_{\mathfrak{g}}-T\Big{(}\rho(x,Tu)v+\rho(Tv,x)u\Big{)},$

for all $x\in\mathfrak{g},u,v\in\mathfrak{h}.$ Then $(\mathfrak{g};\varrho)$
is a representation of the descendent $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{T})$.

###### Proof.

By a direct calculation using (1)-(6) and (10), for all
$u_{i}\in\mathfrak{h},1\leq i\leq 4,x\in\mathfrak{g},$ we have

$\displaystyle\Big{(}\varrho(u_{1},u_{2})\varrho(u_{3},u_{4})-\varrho(u_{3},u_{4})\varrho(u_{1},u_{2})-\varrho([u_{1},u_{2},u_{3}]_{T},u_{4})+\varrho([u_{1},u_{2},u_{4}]_{T},u_{3})\Big{)}(x)$
$\displaystyle=$
$\displaystyle\varrho(u_{1},u_{2})\Big{(}[Tu_{3},Tu_{4},x]_{\mathfrak{g}}-T\rho(x,Tu_{3})u_{4}-T\rho(Tu_{4},x)u_{3}\Big{)}$
$\displaystyle-\varrho(u_{3},u_{4})\Big{(}[Tu_{1},Tu_{2},x]_{\mathfrak{g}}-T\rho(x,Tu_{1})u_{2}-T\rho(Tu_{2},x)u_{1}\Big{)}$
$\displaystyle-[[Tu_{1},Tu_{2},Tu_{3}]_{\mathfrak{g}},Tu_{4},x]_{\mathfrak{g}}+T\Big{(}\rho(x,[Tu_{1},Tu_{2},Tu_{3}]_{\mathfrak{g}})u_{4}+\rho(Tu_{4},x)[u_{1},u_{2},u_{3}]_{T}\Big{)}$
$\displaystyle+[[Tu_{1},Tu_{2},Tu_{4}]_{\mathfrak{g}},Tu_{3},x]_{\mathfrak{g}}-T\Big{(}\rho(x,[Tu_{1},Tu_{2},Tu_{4}]_{\mathfrak{g}})u_{3}+\rho(Tu_{3},x)[u_{1},u_{2},u_{4}]_{T}\Big{)}$
$\displaystyle=$
$\displaystyle-[Tu_{1},Tu_{2},T\rho(x,Tu_{3})u_{4}]_{\mathfrak{g}}-[Tu_{1},Tu_{2},T\rho(Tu_{4},x)u_{3}]_{\mathfrak{g}}$
$\displaystyle-T\Big{(}\rho([Tu_{3},Tu_{4},x]_{\mathfrak{g}},Tu_{1})u_{2}+\rho(Tu_{2},[Tu_{3},Tu_{4},x]_{\mathfrak{g}})u_{1}\Big{)}$
$\displaystyle+T\Big{(}\rho(T\rho(x,Tu_{3})u_{4},Tu_{1})u_{2}+\rho(Tu_{2},T\rho(x,Tu_{3})u_{4})u_{1}\Big{)}$
$\displaystyle+T\Big{(}\rho(T\rho(Tu_{4},x)u_{3},Tu_{1})u_{2}+\rho(Tu_{2},T\rho(Tu_{4},x)u_{3})u_{1}\Big{)}$
$\displaystyle+[Tu_{3},Tu_{4},T\rho(Tu_{2},x)u_{1}]_{\mathfrak{g}}+[Tu_{3},Tu_{4},T\rho(x,Tu_{1})u_{2}]_{\mathfrak{g}}$
$\displaystyle+T\Big{(}\rho([Tu_{1},Tu_{2},x]_{\mathfrak{g}},Tu_{3})u_{4}+\rho(Tu_{4},[Tu_{1},Tu_{2},x]_{\mathfrak{g}})u_{3}\Big{)}$
$\displaystyle-T\Big{(}\rho(T\rho(x,Tu_{1})u_{2},Tu_{3})u_{4}+\rho(Tu_{4},T\rho(x,Tu_{1})u_{2})u_{3}\Big{)}$
$\displaystyle-T\Big{(}\rho(T\rho(Tu_{2},x)u_{1},Tu_{3})u_{4}+\rho(Tu_{4},T\rho(Tu_{2},x)u_{1})u_{3}\Big{)}$
$\displaystyle+T\Big{(}\rho(x,[Tu_{1},Tu_{2},Tu_{3}]_{\mathfrak{g}})u_{4}+\rho(Tu_{4},x)\rho(Tu_{1},Tu_{2})u_{3}$
$\displaystyle+\rho(Tu_{4},x)\rho(Tu_{2},Tu_{3})u_{1}+\rho(Tu_{4},x)\rho(Tu_{3},Tu_{1})u_{2}+\lambda\rho(Tu_{4},x)[u_{1},u_{2},u_{3}]_{\mathfrak{h}}\Big{)}$
$\displaystyle-T\Big{(}\rho(x,[Tu_{1},Tu_{2},Tu_{4}]_{\mathfrak{g}})u_{3}+\rho(Tu_{3},x)\rho(Tu_{1},Tu_{2})u_{4}$
$\displaystyle+\rho(Tu_{3},x)\rho(Tu_{2},Tu_{4})u_{1}+\rho(Tu_{3},x)\rho(Tu_{4},Tu_{1})u_{2}+\lambda\rho(Tu_{3},x)[u_{1},u_{2},u_{4}]_{\mathfrak{h}}\Big{)}$
$\displaystyle=$ $\displaystyle 0,$

and

$\displaystyle\Big{(}\varrho([u_{1},u_{2},u_{3}]_{T},u_{4})-\varrho(u_{1},u_{2})\varrho(u_{3},u_{4})-\varrho(u_{2},u_{3})\varrho(u_{1},u_{4})-\varrho(u_{3},u_{1})\varrho(u_{2},u_{4})\Big{)}(x)$
$\displaystyle=$
$\displaystyle[[Tu_{1},Tu_{2},Tu_{3}]_{\mathfrak{g}},Tu_{4},x]_{\mathfrak{g}}-T\Big{(}\rho(x,[Tu_{1},Tu_{2},Tu_{3}]_{\mathfrak{g}})u_{4}+\rho(Tu_{4},x)[u_{1},u_{2},u_{3}]_{T}\Big{)}$
$\displaystyle-\varrho(u_{1},u_{2})\Big{(}[Tu_{3},Tu_{4},x]_{\mathfrak{g}}-T\rho(x,Tu_{3})u_{4}-T\rho(Tu_{4},x)u_{3}\Big{)}$
$\displaystyle-\varrho(u_{2},u_{3})\Big{(}[Tu_{1},Tu_{4},x]_{\mathfrak{g}}-T\rho(x,Tu_{1})u_{4}-T\rho(Tu_{4},x)u_{1}\Big{)}$
$\displaystyle-\varrho(u_{3},u_{1})\Big{(}[Tu_{2},Tu_{4},x]_{\mathfrak{g}}-T\rho(x,Tu_{2})u_{4}-T\rho(Tu_{4},x)u_{2}\Big{)}$
$\displaystyle=$
$\displaystyle-T\Big{(}\rho(x,[Tu_{1},Tu_{2},Tu_{3}]_{\mathfrak{g}})u_{4}+\lambda\rho(Tu_{4},x)[u_{1},u_{2},u_{3}]_{\mathfrak{h}}\Big{)}$
$\displaystyle-T\rho(Tu_{4},x)\Big{(}\rho(Tu_{1},Tu_{2})u_{3}+\rho(Tu_{2},Tu_{3})u_{1}+\rho(Tu_{3},Tu_{1})u_{2}\Big{)}$
$\displaystyle+[Tu_{1},Tu_{2},T\rho(x,Tu_{3})u_{4}+T\rho(Tu_{4},x)u_{3}]$
$\displaystyle+T\Big{(}\rho([Tu_{3},Tu_{4},x]_{\mathfrak{g}},Tu_{1})u_{2}+\rho(Tu_{2},[Tu_{3},Tu_{4},x]_{\mathfrak{g}})u_{1}\Big{)}$
$\displaystyle-T\Big{(}\rho(T\rho(x,Tu_{3})u_{4},Tu_{1})u_{2}+\rho(Tu_{2},T\rho(x,Tu_{3})u_{4})u_{1}\Big{)}$
$\displaystyle-T\Big{(}\rho(T\rho(Tu_{4},x)u_{3},Tu_{1})u_{2}+\rho(Tu_{2},T\rho(Tu_{4},x)u_{3})u_{1}\Big{)}$
$\displaystyle+[Tu_{2},Tu_{3},T\rho(x,Tu_{1})u_{4}+T\rho(Tu_{4},x)u_{1}]$
$\displaystyle+T\Big{(}\rho([Tu_{1},Tu_{4},x]_{\mathfrak{g}},Tu_{2})u_{3}+\rho(Tu_{3},[Tu_{1},Tu_{4},x]_{\mathfrak{g}})u_{2}\Big{)}$
$\displaystyle-T\Big{(}\rho(T\rho(x,Tu_{1})u_{4},Tu_{2})u_{3}+\rho(Tu_{3},T\rho(x,Tu_{1})u_{4})u_{2}\Big{)}$
$\displaystyle-T\Big{(}\rho(T\rho(Tu_{4},x)u_{1},Tu_{2})u_{3}+\rho(Tu_{3},T\rho(Tu_{4},x)u_{1})u_{2}\Big{)}$
$\displaystyle+[Tu_{3},Tu_{1},T\rho(x,Tu_{2})u_{4}+T\rho(Tu_{4},x)u_{2}]$
$\displaystyle+T\Big{(}\rho([Tu_{2},Tu_{4},x]_{\mathfrak{g}},Tu_{3})u_{1}+\rho(Tu_{1},[Tu_{2},Tu_{4},x]_{\mathfrak{g}})u_{3}\Big{)}$
$\displaystyle-T\Big{(}\rho(T\rho(x,Tu_{2})u_{4},Tu_{3})u_{1}+\rho(Tu_{1},T\rho(x,Tu_{2})u_{4})u_{3}\Big{)}$
$\displaystyle-T\Big{(}\rho(T\rho(Tu_{4},x)u_{2},Tu_{3})u_{1}+\rho(Tu_{1},T\rho(Tu_{4},x)u_{2})u_{3}\Big{)}$
$\displaystyle=$ $\displaystyle 0.$

Thus, we deduce that $(\mathfrak{g};\varrho)$ is a representation of the
descendent $3$-Lie algebra $(\mathfrak{h},[\cdot,\cdot,\cdot]_{T})$. ∎

######  Proposition 5.3.

Let $T$ and $T^{\prime}$ be relative Rota-Baxter operators of weight $\lambda$
from a $3$-Lie algebra $(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}})$ to
a $3$-Lie algebra $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ with
respect to an action $\rho$. Let $(\psi_{\mathfrak{g}},\psi_{\mathfrak{h}})$
be a homomorphism from $T$ to $T^{\prime}$.

  * (i)

$\psi_{\mathfrak{h}}$ is also a $3$-Lie algebra homomorphism from the
descendent $3$-Lie algebra $(\mathfrak{h},[\cdot,\cdot,\cdot]_{T})$ of $T$ to
the descendent $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{T^{\prime}})$ of $T^{\prime}$;

  * (ii)

The induced representation $(\mathfrak{g};\varrho)$ of the $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{T})$ and the induced representation
$(\mathfrak{g};\varrho^{\prime})$ of the $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{T^{\prime}})$ satisfy the following
relation:

$\psi_{\mathfrak{g}}\circ\varrho(u,v)=\varrho^{\prime}(\psi_{\mathfrak{h}}(u),\psi_{\mathfrak{h}}(v))\circ\psi_{\mathfrak{g}},\quad\forall
u,v\in\mathfrak{h}.$

That is, the following diagram commutes:

$\textstyle{\mathfrak{g}\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\varrho(u,v)}$$\scriptstyle{\psi_{\mathfrak{g}}}$$\textstyle{\mathfrak{g}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\varrho^{\prime}(\psi_{\mathfrak{h}}(u),\psi_{\mathfrak{h}}(v))}$$\textstyle{\mathfrak{g}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\psi_{\mathfrak{g}}}$$\textstyle{\mathfrak{g}.}$

###### Proof.

(i) It follows from Corollary 3.5 and Proposition 3.7 directly.

(ii) By (30), (7)-(8) and the fact that $\psi_{\mathfrak{g}}$ is a $3$-Lie
algebra homomorphism, for all $x\in\mathfrak{g},u,v\in\mathfrak{h}$, we have

$\displaystyle\psi_{\mathfrak{g}}\Big{(}\varrho(u,v)x\Big{)}$ $\displaystyle=$
$\displaystyle\psi_{\mathfrak{g}}\Big{(}[Tu,Tv,x]_{\mathfrak{g}}-T(\rho(x,Tu)v)-T(\rho(Tv,x)u)\Big{)}$
$\displaystyle=$
$\displaystyle[\psi_{\mathfrak{g}}(Tu),\psi_{\mathfrak{g}}(Tv),\psi_{\mathfrak{g}}(x)]_{\mathfrak{g}}-T^{\prime}\psi_{\mathfrak{h}}\Big{(}\rho(x,Tu)v\Big{)}-T^{\prime}\psi_{\mathfrak{h}}\Big{(}\rho(Tv,x)u\Big{)}$
$\displaystyle=$
$\displaystyle[T^{\prime}\psi_{\mathfrak{h}}(u),T^{\prime}\psi_{\mathfrak{h}}(v),\psi_{\mathfrak{g}}(x)]_{\mathfrak{g}}-T^{\prime}\Big{(}\rho(\psi_{\mathfrak{g}}(x),T^{\prime}\psi_{\mathfrak{h}}(u))\psi_{\mathfrak{h}}(v)\Big{)}$
$\displaystyle-T^{\prime}\Big{(}\rho(T^{\prime}\psi_{\mathfrak{h}}(v),\psi_{\mathfrak{g}}(x))\psi_{\mathfrak{h}}(u)\Big{)}$
$\displaystyle=$
$\displaystyle\varrho^{\prime}(\psi_{\mathfrak{h}}(u),\psi_{\mathfrak{h}}(v))\psi_{\mathfrak{g}}(x).$

We finish the proof. ∎

Let ${\rm
d}_{T}:\mathfrak{C}_{\mathsf{3Lie}}^{n}(\mathfrak{h};\mathfrak{g})\rightarrow\mathfrak{C}_{\mathsf{3Lie}}^{n+1}(\mathfrak{h};\mathfrak{g}),(n\geq
1)$ be the corresponding coboundary operator of the descendent $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{T})$ with coefficients in the
representation $(\mathfrak{g};\varrho)$. More precisely, for all
$f\in\mathrm{Hom}(\underbrace{\wedge^{2}\mathfrak{h}\otimes\cdots\otimes\wedge^{2}\mathfrak{h}}_{(n-1)}\wedge\mathfrak{h},\mathfrak{g})$,
$\mathrm{U}_{i}=u_{i}\wedge v_{i}\in\wedge^{2}\mathfrak{h},~{}i=1,2,\cdots,n$
and $u_{n+1}\in\mathfrak{h},$ we have

$\displaystyle({\rm d}_{T}f)(\mathrm{U}_{1},\cdots,\mathrm{U}_{n},u_{n+1})$
$\displaystyle=$ $\displaystyle\sum_{1\leq j<k\leq
n}(-1)^{j}f(\mathrm{U}_{1},\cdots,\hat{\mathrm{U}_{j}},\cdots,\mathrm{U}_{k-1},[u_{j},v_{j},u_{k}]_{T}\wedge
v_{k}$
$\displaystyle+u_{k}\wedge[u_{j},v_{j},v_{k}]_{T},\mathrm{U}_{k+1},\cdots,\mathrm{U}_{n},u_{n+1})$
$\displaystyle+\sum_{j=1}^{n}(-1)^{j}f(\mathrm{U}_{1},\cdots,\hat{\mathrm{U}_{j}},\cdots,\mathrm{U}_{n},[u_{j},v_{j},u_{n+1}]_{T})$
$\displaystyle+\sum_{j=1}^{n}(-1)^{j+1}\varrho(u_{j},v_{j})f(\mathrm{U}_{1},\cdots,\hat{\mathrm{U}_{j}},\cdots,\mathrm{U}_{n},u_{n+1})$
$\displaystyle+(-1)^{n+1}\Big{(}\varrho(v_{n},u_{n+1})f(\mathrm{U}_{1},\cdots,\mathrm{U}_{n-1},u_{n})+\varrho(u_{n+1},u_{n})f(\mathrm{U}_{1},\cdots,\mathrm{U}_{n-1},v_{n})\Big{)}.$

It is obvious that
$f\in\mathfrak{C}_{\mathsf{3Lie}}^{1}(\mathfrak{h};\mathfrak{g})$ is closed if
and only if

$\displaystyle[f(u_{1}),Tu_{2},Tu_{3}]_{\mathfrak{g}}+[Tu_{1},f(u_{2}),Tu_{3}]_{\mathfrak{g}}+[Tu_{1},Tu_{2},f(u_{3})]_{\mathfrak{g}}$
$\displaystyle=$ $\displaystyle
T\Big{(}\rho(Tu_{2},f(u_{3}))u_{1}+\rho(f(u_{3}),Tu_{1})u_{2}+\rho(f(u_{1}),Tu_{2})u_{3}\Big{)}$
$\displaystyle+T\Big{(}\rho(f(u_{2}),Tu_{3})u_{1}+\rho(Tu_{3},f(u_{1}))u_{2}+\rho(Tu_{1},f(u_{2}))u_{3}\Big{)}$
$\displaystyle+f\Big{(}\rho(Tu_{2},Tu_{3})u_{1}+\rho(Tu_{3},Tu_{1})u_{2}+\rho(Tu_{1},Tu_{2})u_{3}+\lambda[u_{1},u_{2},u_{3}]_{\mathfrak{h}}\Big{)}.$

Define
$\delta:\wedge^{2}\mathfrak{g}\rightarrow\mathrm{Hom}(\mathfrak{h},\mathfrak{g})$
by

$\displaystyle\delta(\mathfrak{X})u=T\rho(\mathfrak{X})u-[\mathfrak{X},Tu]_{\mathfrak{g}},\quad\forall\mathfrak{X}\in\wedge^{2}\mathfrak{g},u\in\mathfrak{h}.$

######  Proposition 5.4.

Let $T:\mathfrak{h}\rightarrow\mathfrak{g}$ be a relative Rota-Baxter operator
of weight $\lambda$ from $\mathfrak{h}$ to $\mathfrak{g}$ with respect to an
action $\rho$. Then $\delta(\mathfrak{X})$ is a $1$-cocycle of the $3$-Lie
algebra $(\mathfrak{h},[\cdot,\cdot,\cdot]_{T})$ with coefficients in
$(\mathfrak{g};\varrho).$

###### Proof.

It follows from straightforward computations, and we omit details. ∎

We now introduce the cohomology theory of relative Rota-Baxter operators of
weight $\lambda$ on $3$-Lie algebras.

Let $T$ be a relative Rota-Baxter operator of weight $\lambda$ from a $3$-Lie
algebra $(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}})$ to a $3$-Lie
algebra $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ with respect to an
action $\rho$. Define the space of $n$-cochains by

(33)
$\displaystyle\mathfrak{C}_{T}^{n}(\mathfrak{h};\mathfrak{g})=\left\\{\begin{array}[]{rcl}{}\mathfrak{C}_{\mathsf{3Lie}}^{n-1}(\mathfrak{h};\mathfrak{g}),\quad
n\geq 2,\\\ {}\mathfrak{g}\wedge\mathfrak{g},\quad n=1.\end{array}\right.$

Define
${\partial}:\mathfrak{C}_{T}^{n}(\mathfrak{h};\mathfrak{g})\rightarrow\mathfrak{C}_{T}^{n+1}(\mathfrak{h};\mathfrak{g})$
by

(36) $\displaystyle{\partial}=\left\\{\begin{array}[]{rcl}{}{\rm d}_{T},\quad
n\geq 2,\\\ {}\delta,\quad n=1.\end{array}\right.$

######  Theorem 5.5.

$(\mathop{\oplus}\limits_{n=1}^{\infty}\mathfrak{C}_{T}^{n}(\mathfrak{h};\mathfrak{g}),\partial)$
is a cochain complex.

###### Proof.

It follows from Proposition 5.4 and the fact that ${\rm d}_{T}$ is the
corresponding coboundary operator of the descendent $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{T})$ with coefficients in the
representation $(\mathfrak{g};\varrho)$ directly. ∎

######  Definition 5.6.

The cohomology of the cochain complex
$(\mathop{\oplus}\limits_{n=1}^{\infty}\mathfrak{C}_{T}^{n}(\mathfrak{h};\mathfrak{g}),\partial)$
is taken to be the cohomology for the relative Rota-Baxter operator $T$ of
weight $\lambda$. Denote the set of $n$-cocycles by
$\mathcal{Z}^{n}_{T}(\mathfrak{h};\mathfrak{g}),$ the set of $n$-coboundaries
by $\mathcal{B}^{n}_{T}(\mathfrak{h};\mathfrak{g})$ and $n$-th cohomology
group by

(37)
$\displaystyle\mathcal{H}^{n}_{T}(\mathfrak{h};\mathfrak{g})=\mathcal{Z}_{T}^{n}(\mathfrak{h};\mathfrak{g})/\mathcal{B}_{T}^{n}(\mathfrak{h};\mathfrak{g}),\quad
n\geq 1.$

######  Remark 5.7.

The cohomology theory for relative Rota-Baxter operators of weight $\lambda$
on $3$-Lie algebras enjoys certain functorial properties. Let $T$ and
$T^{\prime}$ be relative Rota-Baxter operators of weight $\lambda$ from a
$3$-Lie algebra $(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}})$ to a
$3$-Lie algebra $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ with
respect to an action $\rho$. Let $(\psi_{\mathfrak{g}},\psi_{\mathfrak{h}})$
be a homomorphism from $T$ to $T^{\prime}$ in which $\psi_{\mathfrak{h}}$ is
invertible. Define a map
$p:\mathfrak{C}_{T}^{n}(\mathfrak{h};\mathfrak{g})\rightarrow\mathfrak{C}_{T^{\prime}}^{n}(\mathfrak{h};\mathfrak{g})$
by

$\displaystyle
p(\omega)(\mathrm{U}_{1},\cdots,\mathrm{U}_{n-2},u_{n-1})=\psi_{\mathfrak{g}}\Bigg{(}\omega\Big{(}\psi^{-1}_{\mathfrak{h}}(u_{1})\wedge\psi^{-1}_{\mathfrak{h}}(v_{1}),\cdots,\psi^{-1}_{\mathfrak{h}}(u_{n-2})\wedge\psi^{-1}_{\mathfrak{h}}(v_{n-2}),\psi^{-1}_{\mathfrak{h}}(u_{n-1})\Big{)}\Bigg{)},$

for all
$\omega\in\mathfrak{C}_{T}^{n}(\mathfrak{h};\mathfrak{g}),\mathrm{U}_{i}=u_{i}\wedge
v_{i}\in\wedge^{2}\mathfrak{h},~{}i=1,2,\cdots,n-2$ and
$u_{n-1}\in\mathfrak{h}.$ Then it is straightforward to deduce that $p$ is a
cochain map from the cochain complex
$(\mathop{\oplus}\limits_{n=2}^{\infty}\mathfrak{C}_{T}^{n}(\mathfrak{h};\mathfrak{g}),{\rm
d}_{T})$ to the cochain complex
$(\mathop{\oplus}\limits_{n=2}^{\infty}\mathfrak{C}_{T^{\prime}}^{n}(\mathfrak{h};\mathfrak{g}),{\rm
d}_{T^{\prime}})$. Consequently, it induces a homomorphism $p_{*}$ from the
cohomology group $\mathcal{H}^{n}_{T}(\mathfrak{h};\mathfrak{g})$ to
$\mathcal{H}^{n}_{T^{\prime}}(\mathfrak{h};\mathfrak{g})$.

At the end of this subsection, we give the relationship between the coboundary
operator ${\rm d}_{T}$ and the differential $l_{1}^{T}$ defined by (26) using
the Maurer-Cartan element $T$ of the $L_{\infty}$-algebra
$(C^{*}(\mathfrak{h},\mathfrak{g}),l_{1},l_{3})$.

######  Lemma 5.8.

Let $T:\mathfrak{h}\rightarrow\mathfrak{g}$ be a relative Rota-Baxter operator
of weight $\lambda$ from a $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}})$ to a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ with respect to an action
$\rho$. For all $x,y,z\in\mathfrak{g},u,v,w\in V,$ we have

$\displaystyle[[\pi+\rho+\lambda\mu,T]_{\mathsf{R}},T]_{\mathsf{R}}(x+u,y+v,z+w)$
$\displaystyle=$ $\displaystyle
2\Big{(}[Tu,Tv,z]_{\mathfrak{g}}+\rho(Tu,Tv)w+[Tv,Tw,x]_{\mathfrak{g}}+\rho(Tv,Tw)u+[Tw,Tu,y]_{\mathfrak{g}}+\rho(Tw,Tu)v\Big{)}$
$\displaystyle-2T\Big{(}\rho(Tu,y)w+\rho(z,Tu)v+\rho(x,Tv)w+\rho(Tv,z)u+\rho(y,Tw)u+\rho(Tw,x)v\Big{)}.$

###### Proof.

It follows from straightforward computations. ∎

######  Theorem 5.9.

Let $T$ be a relative Rota-Baxter operator of weight $\lambda$ from a $3$-Lie
algebra $(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}})$ to a $3$-Lie
algebra $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ with respect to an
action $\rho$. Then we have

${\rm d}_{T}f=(-1)^{n-1}l_{1}^{T}f,\quad\forall
f\in\mathrm{Hom}(\underbrace{\wedge^{2}\mathfrak{h}\otimes\cdots\otimes\wedge^{2}\mathfrak{h}}_{n-1}\wedge\mathfrak{h},\mathfrak{g}),~{}n=1,2,\cdots.$

###### Proof.

For all $\mathfrak{U}_{i}=u_{i}\wedge
v_{i}\in\wedge^{2}\mathfrak{h},~{}i=1,2,\cdots,n$ and
$u_{n+1}\in\mathfrak{h}$, we have

$\displaystyle l_{1}(f)(\mathfrak{U}_{1},\cdots,\mathfrak{U}_{n},u_{n+1})$
$\displaystyle=$
$\displaystyle[\pi+\rho+\lambda\mu,f]_{\mathsf{R}}(\mathfrak{U}_{1},\cdots,\mathfrak{U}_{n},u_{n+1})$
$\displaystyle=$ $\displaystyle\Big{(}(\pi+\rho+\lambda\mu)\circ
f-(-1)^{n-1}f\circ(\pi+\rho+\lambda\mu)\Big{)}(\mathfrak{U}_{1},\cdots,\mathfrak{U}_{n},u_{n+1})$
$\displaystyle=$
$\displaystyle(\pi+\rho+\lambda\mu)(f(\mathfrak{U}_{1},\cdots,\mathfrak{U}_{n-1},u_{n})\wedge
v_{n},u_{n+1})$ $\displaystyle+(\pi+\rho+\lambda\mu)(u_{n}\wedge
f(\mathfrak{U}_{1},\cdots,\mathfrak{U}_{n-1},v_{n}),u_{n+1})$
$\displaystyle+\sum_{i=1}^{n}(-1)^{n-1}(-1)^{i-1}(\pi+\rho+\lambda\mu)(\mathfrak{U}_{i},f(\mathfrak{U}_{1},\cdots,\hat{\mathfrak{U}_{i}},\cdots,\mathfrak{U}_{n},u_{n+1}))$
$\displaystyle-(-1)^{n-1}\sum_{k=1}^{n-1}\sum_{i=1}^{k}(-1)^{i+1}f\Big{(}\mathfrak{U}_{1},\cdots,\hat{\mathfrak{U}}_{i},\cdots,\mathfrak{U}_{k},(\pi+\rho+\lambda\mu)(\mathfrak{U}_{i},u_{k+1})\wedge
v_{k+1},\mathfrak{U}_{k+2},\cdots,\mathfrak{U}_{n},u_{n+1}\Big{)}$
$\displaystyle-(-1)^{n-1}\sum_{k=1}^{n-1}\sum_{i=1}^{k}(-1)^{i+1}f\Big{(}\mathfrak{U}_{1},\cdots,\hat{\mathfrak{U}}_{i},\cdots,\mathfrak{U}_{k},u_{k+1}\wedge(\pi+\rho+\lambda\mu)(\mathfrak{U}_{i},v_{k+1}),\mathfrak{U}_{k+2},\cdots,\mathfrak{U}_{n},u_{n+1}\Big{)}$
$\displaystyle-(-1)^{n-1}\sum_{i=1}^{n}(-1)^{i+1}f\Big{(}\mathfrak{U}_{1},\cdots,\hat{\mathfrak{U}}_{i},\cdots,\mathfrak{U}_{n},(\pi+\rho+\lambda\mu)(\mathfrak{U}_{i},u_{n+1})\Big{)}$
$\displaystyle=$
$\displaystyle-(-1)^{n-1}\sum_{k=1}^{n-1}\sum_{i=1}^{k}(-1)^{i+1}f\Big{(}\mathfrak{U}_{1},\cdots,\hat{\mathfrak{U}}_{i},\cdots,\mathfrak{U}_{k},\lambda\mu(u_{i},v_{i},u_{k+1})\wedge
v_{k+1},\mathfrak{U}_{k+2},\cdots,\mathfrak{U}_{n},u_{n+1}\Big{)}$
$\displaystyle-(-1)^{n-1}\sum_{k=1}^{n-1}\sum_{i=1}^{k}(-1)^{i+1}f\Big{(}\mathfrak{U}_{1},\cdots,\hat{\mathfrak{U}}_{i},\cdots,\mathfrak{U}_{k},u_{k+1}\wedge\lambda\mu(u_{i},v_{i},v_{k+1}),\mathfrak{U}_{k+2},\cdots,\mathfrak{U}_{n},u_{n+1}\Big{)}$
$\displaystyle-(-1)^{n-1}\sum_{i=1}^{n}(-1)^{i+1}f\Big{(}\mathfrak{U}_{1},\cdots,\hat{\mathfrak{U}}_{i},\cdots,\mathfrak{U}_{n},\lambda\mu(u_{i},v_{i},u_{n+1})\Big{)}.$

By Lemma 5.8, we have

$\displaystyle\frac{1}{2}l_{3}(T,T,f)(\mathfrak{U}_{1},\cdots,\mathfrak{U}_{n},u_{n+1})$
$\displaystyle=$
$\displaystyle[[[\pi+\rho+\lambda\mu,T]_{\mathsf{R}},T]_{\mathsf{R}},f]_{\mathsf{R}}(\mathfrak{U}_{1},\cdots,\mathfrak{U}_{n},u_{n+1})$
$\displaystyle=$
$\displaystyle[[\pi+\rho+\lambda\mu,T]_{\mathsf{R}},T]_{\mathsf{R}}\Big{(}f(\mathfrak{U}_{1},\cdots,\mathfrak{U}_{n-1},u_{n})\wedge
v_{n},u_{n+1}\Big{)}$
$\displaystyle+[[\pi+\rho+\lambda\mu,T]_{\mathsf{R}},T]_{\mathsf{R}}\Big{(}u_{n}\wedge
f(\mathfrak{U}_{1},\cdots,\mathfrak{U}_{n-1},v_{n}),u_{n+1}\Big{)}$
$\displaystyle+\sum_{i=1}^{n}(-1)^{n-1}(-1)^{i-1}[[\pi+\rho+\lambda\mu,T]_{\mathsf{R}},T]_{\mathsf{R}}\Big{(}\mathfrak{U}_{i},f(\mathfrak{U}_{1},\cdots,\hat{\mathfrak{U}}_{i},\cdots,\mathfrak{U}_{n},u_{n+1})\Big{)}$
$\displaystyle-(-1)^{n-1}\sum_{k=1}^{n-1}\sum_{i=1}^{k}(-1)^{i+1}f\Big{(}\mathfrak{U}_{1},\cdots,\hat{\mathfrak{U}}_{i},\cdots,\mathfrak{U}_{k},[[\pi+\rho+\lambda\mu,T]_{\mathsf{R}},T]_{\mathsf{R}}(\mathfrak{U}_{i},u_{k+1})\wedge
v_{k+1}$
$\displaystyle+u_{k+1}\wedge[[\pi+\rho+\lambda\mu,T]_{\mathsf{R}},T]_{\mathsf{R}}(\mathfrak{U}_{i},v_{k+1}),\mathfrak{U}_{k+2},\cdots,\mathfrak{U}_{n},u_{n+1}\Big{)}$
$\displaystyle-(-1)^{n-1}\sum_{i=1}^{n}(-1)^{i+1}f\Big{(}\mathfrak{U}_{1},\cdots,\hat{\mathfrak{U}}_{i},\cdots,\mathfrak{U}_{n},[[\pi+\rho+\lambda\mu,T]_{\mathsf{R}},T]_{\mathsf{R}}(\mathfrak{U}_{i},u_{n+1})\Big{)}$
$\displaystyle=$
$\displaystyle\varrho(v_{n},u_{n+1})f(\mathfrak{U}_{1},\cdots,\mathfrak{U}_{n-1},u_{n})+\varrho(u_{n+1},u_{n},)f(\mathfrak{U}_{1},\cdots,\mathfrak{U}_{n-1},v_{n})$
$\displaystyle+\sum_{i=1}^{n}(-1)^{n-1}(-1)^{i-1}\varrho(u_{i},v_{i})f(\mathfrak{U}_{1},\cdots,\mathfrak{U}_{i},\cdots,\mathfrak{U}_{n},u_{n+1})$
$\displaystyle-(-1)^{n-1}\sum_{k=1}^{n-1}\sum_{i=1}^{k}(-1)^{i+1}f\Big{(}\mathfrak{U}_{1},\cdots,\hat{\mathfrak{U}}_{i},\cdots,\mathfrak{U}_{k},\Big{(}\rho(Tu_{i},Tv_{i})u_{k+1}$
$\displaystyle+\rho(Tv_{i},Tu_{k+1})u_{i}+\rho(Tu_{k+1},Tu_{i})v_{i}\Big{)}\wedge
v_{k+1},\mathfrak{U}_{k+2},\cdots,\mathfrak{U}_{n},u_{n+1}\Big{)}$
$\displaystyle-(-1)^{n-1}\sum_{k=1}^{n-1}\sum_{i=1}^{k}(-1)^{i+1}f\Big{(}\mathfrak{U}_{1},\cdots,\hat{\mathfrak{U}}_{i},\cdots,\mathfrak{U}_{k},u_{k+1}\wedge\Big{(}\rho(Tu_{i},Tv_{i})v_{k+1}$
$\displaystyle+\rho(Tv_{i},Tv_{k+1})u_{i}+\rho(Tv_{k+1},Tu_{i})v_{i}\Big{)},\mathfrak{U}_{k+2},\cdots,\mathfrak{U}_{n},u_{n+1}\Big{)}$
$\displaystyle-(-1)^{n-1}\sum_{i=1}^{n}(-1)^{i+1}f\Big{(}\mathfrak{U}_{1},\cdots,\hat{\mathfrak{U}}_{i},\cdots,\mathfrak{U}_{n},\rho(Tu_{i},Tv_{i})u_{n+1}+\rho(Tv_{i},Tu_{n+1})u_{i}+\rho(Tu_{n+1},Tu_{i})v_{i}\Big{)}$

Thus, we deduce that ${\rm
d}_{T}f=(-1)^{n-1}\Big{(}l_{1}(f)+\frac{1}{2}l_{3}(T,T,f)\Big{)}$, that is
${\rm d}_{T}f=(-1)^{n-1}l_{1}^{T}f$. ∎

###  5.2. Infinitesimal deformations of relative Rota-Baxter operators

In this subsection, we use the established cohomology theory to characterize
infinitesimal deformations of relative Rota-Baxter operators of weight
$\lambda$ on 3-Lie algebras.

Let $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ be a $3$-Lie algebra
over $\mathbb{K}$ and $\mathbb{K}[t]$ be the polynomial ring in one variable
$t.$ Then $\mathbb{K}[t]/(t^{2})\otimes_{\mathbb{K}}\mathfrak{g}$ is an
$\mathbb{K}[t]/(t^{2})$-module. Moreover,
$\mathbb{K}[t]/(t^{2})\otimes_{\mathbb{K}}\mathfrak{g}$ is a $3$-Lie algebra
over $\mathbb{K}[t]/(t^{2})$, where the $3$-Lie algebra structure is defined
by

$\displaystyle[f_{1}(t)\otimes_{\mathbb{K}}x_{1},f_{2}(t)\otimes_{\mathbb{K}}x_{2},f_{3}(t)\otimes_{\mathbb{K}}x_{3}]=f_{1}(t)f_{2}(t)f_{3}(t)\otimes_{\mathbb{K}}[x_{1},x_{2},x_{3}]_{\mathfrak{g}},$

for $f_{i}(t)\in\mathbb{K}[t]/(t^{2}),1\leq i\leq
3,x_{1},x_{2},x_{3}\in\mathfrak{g}.$

In the sequel, all the vector spaces are finite dimensional vector spaces over
$\mathbb{K}$ and we denote $f(t)\otimes_{\mathbb{K}}x$ by $f(t)x,$ where
$f(t)\in\mathbb{K}[t]/(t^{2}).$

######  Definition 5.10.

Let $T:\mathfrak{h}\rightarrow\mathfrak{g}$ be a relative Rota-Baxter operator
of weight $\lambda$ from a $3$-Lie algebra
$(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}})$ to a $3$-Lie algebra
$(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ with respect to an action
$\rho$. Let $\mathfrak{T}:\mathfrak{h}\rightarrow\mathfrak{g}$ be a linear
map. If $T_{t}=T+t\mathfrak{T}$ is a relative Rota-Baxter operator of weight
$\lambda$ modulo $t^{2}$, we say that $\mathfrak{T}$ generates an
infinitesimal deformation of $T$.

Since $T_{t}=T+t\mathfrak{T}$ is a relative Rota-Baxter operator of weight
$\lambda$ modulo $t^{2}$, by consider the coefficients of $t$, for any
$u,v,w\in\mathfrak{h},$ we have

$\displaystyle[\mathfrak{T}u,Tv,Tw]_{\mathfrak{g}}+[Tu,\mathfrak{T}v,Tw]_{\mathfrak{g}}+[Tu,Tv,\mathfrak{T}w]_{\mathfrak{g}}$
$\displaystyle=$ $\displaystyle
T\Big{(}\rho(\mathfrak{T}w,Tu)v+\rho(Tv,\mathfrak{T}w)u+\rho(\mathfrak{T}u,Tv)w$
$\displaystyle+\rho(Tw,\mathfrak{T}u)v+\rho(\mathfrak{T}v,Tw)u+\rho(Tu,\mathfrak{T}v)w\Big{)}$
$\displaystyle+\mathfrak{T}\Big{(}\rho(Tu,Tv)w+\rho(Tv,Tw)u+\rho(Tw,Tu)v+\lambda[u,v,w]_{\mathfrak{h}}\Big{)}.$

Note that (5.2) means that $\mathfrak{T}$ is a $2$-cocycle of the relative
Rota-Baxter operator $T$. Hence, $\mathfrak{T}$ defines a cohomology class in
$\mathcal{H}^{2}_{T}(\mathfrak{h};\mathfrak{g})$.

######  Definition 5.11.

Let $T$ be a relative Rota-Baxter operator of weight $\lambda$ from a $3$-Lie
algebra $(\mathfrak{h},[\cdot,\cdot,\cdot]_{\mathfrak{h}})$ to a $3$-Lie
algebra $(\mathfrak{g},[\cdot,\cdot,\cdot]_{\mathfrak{g}})$ with respect to an
action $\rho$. Two one-parameter infinitesimal deformations
$T^{1}_{t}=T+t\mathfrak{T}_{1}$ and $T^{2}_{t}=T+t\mathfrak{T}_{2}$ are said
to be equivalent if there exists
$\mathfrak{X}\in\mathfrak{g}\wedge\mathfrak{g}$ such that
$({\rm{Id}}_{\mathfrak{g}}+t\mathrm{ad}_{\mathfrak{X}},{\rm{Id}}_{\mathfrak{h}}+t\rho(\mathfrak{X}))$
is a homomorphism from $T^{1}_{t}$ to $T^{2}_{t}$ modulo $t^{2}$. In
particular, an infinitesimal deformation $T_{t}=T+t\mathfrak{T}_{1}$ of a
relative Rota-Baxter operator $T$ is said to be trivial if there exists
$\mathfrak{X}\in\mathfrak{g}\wedge\mathfrak{g}$ such that
$({\rm{Id}}_{\mathfrak{g}}+t\mathrm{ad}_{\mathfrak{X}},{\rm{Id}}_{\mathfrak{h}}+t\rho(\mathfrak{X}))$
is a homomorphism from $T_{t}$ to $T$ modulo $t^{2}$.

Let
$({\rm{Id}}_{\mathfrak{g}}+t\mathrm{ad}_{\mathfrak{X}},{\rm{Id}}_{\mathfrak{h}}+t\rho(\mathfrak{X}))$
be a homomorphism from $T^{1}_{t}$ to $T^{2}_{t}$ modulo $t^{2}.$ By (7) we
get,

$\quad({\rm{Id}}_{\mathfrak{g}}+t\mathrm{ad}_{\mathfrak{X}})(T+t\mathfrak{T}_{1})(u)=(T+t\mathfrak{T}_{2})({\rm{Id}}_{\mathfrak{h}}+t\rho(\mathfrak{X}))(u),$

which implies

(39) $\displaystyle\mathfrak{T}_{1}(u)-\mathfrak{T}_{2}(u)$ $\displaystyle=$
$\displaystyle T\rho(\mathfrak{X})u-[\mathfrak{X},Tu]_{\mathfrak{g}}.$

Now we are ready to give the main result in this section.

######  Theorem 5.12.

Let $T$ be a relative Rota-Baxter operator of weight $\lambda$ from a $3$-Lie
algebra $\mathfrak{h}$ to a $3$-Lie algebra $\mathfrak{g}$ with respect to an
action $\rho$. If two one-parameter infinitesimal deformations
$T^{1}_{t}=T+t\mathfrak{T}_{1}$ and $T^{2}_{t}=T+t\mathfrak{T}_{2}$ are
equivalent, then $\mathfrak{T}_{1}$ and $\mathfrak{T}_{2}$ are in the same
cohomology class in $\mathcal{H}^{2}_{T}(\mathfrak{h};\mathfrak{g})$.

###### Proof.

It is easy to see from (39) that

$\displaystyle\mathfrak{T}_{1}(u)$ $\displaystyle=$
$\displaystyle\mathfrak{T}_{2}(u)+(\partial\mathfrak{X})(u),\quad\forall
u\in\mathfrak{h},$

which implies that $\mathfrak{T}_{1}$ and $\mathfrak{T}_{2}$ are in the same
cohomology class. ∎

## References

  * [1] J. Bagger and N. Lambert, Three-algebras and N=6 Chern-Simons gauge theories, _Phys. Rev. D_ 79 (2009), no. 2, 025002, 8 pp.
  * [2] C. Bai, A unified algebraic approach to the classical Yang-Baxter equation, _J. Phys. A: Math. Theor._ 40 (2007), 11073-11082.
  * [3] C. Bai, O. Bellier, L. Guo and X. Ni, Spliting of operations, Manin products and Rota-Baxter operators, Int. Math. Res. Not. (2013), 485-524.
  * [4] C. Bai, L. Guo and Y. Sheng, Bialgebras, the classical Yang-Baxter equation and Manin triples for $3$-Lie algebras, _Adv. Theor. Math. Phys._ 23 (2019), 27-74.
  * [5] C. Bai, L. Guo and X. Ni, Nonabelian generalized Lax pairs, the classical Yang-Baxter equation and PostLie algebras, _Comm. Math. Phys._ 297 (2010), no. 2, 553-596.
  * [6] R. Bai, L. Guo, J. Li and Y. Wu, Rota-Baxter 3-Lie algebras, _J. Math. Phys._ 54 (2013), 064504, 14 pp.
  * [7] G. Baxter, An analytic problem whose solution follows from a simple algebraic identity, _Pacific J. Math._ 10 (1960), 731-742.
  * [8] D. Burde, Left-symmetric algebras, or pre-Lie algebras in geometry and physics, _Cent. Eur. J. Math._ 4 (2006), no. 3, 323-357.
  * [9] D. Burde, W. Moens and K. Dekimpe, Commutative post-Lie algebra structures and linear equations for nilpotent Lie algebras, _J. Algebra_ 526 (2019), 12-29.
  * [10] J. M. Casas, J. L. Loday and T. Pirashvili, Leibniz $n$-algebras, _Forum Math._ 14 (2002), no. 2, 189-207.
  * [11] A. Connes and D. Kreimer, Renormalization in quantum field theory and the Riemann-Hilbert problem. I. The Hopf algebra structure of graphs and the main theorem, Comm. Math. Phys. 210 (2000), 249-273.
  * [12] C. Curry, K. Ebrahimi-Fard and H. Munthe-Kaas, What is a post-Lie algebra and why is it useful in geometric integration, Numerical mathematics and advanced applications-ENUMATH 2017, 429-437, Lect. Notes Comput. Sci. Eng., 126, Springer, Cham, 2019.
  * [13] C. Curry, K. Ebrahimi-Fard and B. Owren, The Magnus expansion and post-Lie algebras, Math. Comput. 89 (2020), 2785-2799.
  * [14] A. Das, Deformations of associative Rota-Baxter operators, _J. Algebra_ 560 (2020), 144-180.
  * [15] J. A. de Azcárraga and J. M. Izquierdo, $n$-ary algebras: a review with applications, _J. Phys. A_ 43 (2010), 293001.
  * [16] P. de Medeiros, J. Figueroa-O′Farrill, E. Méndez-Escobar and P. Ritter, On the Lie-algebraic origin of metric 3-algebras, _Comm. Math. Phys._ 290 (2009), 871-902.
  * [17] K. Ebrahimi-Fard, I. Mencattini and H. Munthe-Kaas, Post-Lie algebras and factorization theorems, _J. Geom. Phys._ 119 (2017), 19-33.
  * [18] J. Figueroa-O′Farrill, Deformations of 3-algebras, _J. Math. Phys._ 50 (2009), 113514.
  * [19] V. T. Filippov, $n$-Lie algebras, _Sibirsk. Mat. Zh._ 26 (1985), 126-140, 191.
  * [20] E. Getzler, Lie theory for nilpotent $L_{\infty}$-algebras, _Ann. of Math. (2)_ 170 (2009), 271-301.
  * [21] M. E. Goncharov and P. S. Kolesnikov, Simple finite-dimensional double algebras, _J. Algebra_ 500 (2018), 425-438.
  * [22] L. Guo, An introduction to Rota-Baxter algebra. Surveys of Modern Mathematics, 4. International Press, Somerville, MA; Higher Education Press, Beijing, 2012. xii+226 pp.
  * [23] S. Guo, Y. Qin, K. Wang and G. Zhou, Deformations and cohomology theory of weighted Rota-Baxter 3-Lie algebras, arXiv:2204.00160.
  * [24] S. Hou, Y. Sheng and R. Tang, Twilled 3-Lie algebras, generalized matched pairs of 3-Lie algebras and $\mathcal{O}$-operators, _J. Geom. Phys._ 163 (2021), 104148.
  * [25] Sh. M. Kasymov, On a theory of $n$-Lie algebras, _Algebra i Logika_ 26 (1987), 277-297.
  * [26] B. A. Kupershmidt, What a classical $r$-matrix really is, _J. Nonlinear Math. Phys._ 6 (1999), 448-488.
  * [27] J. Liu, Y. Sheng, Y. Zhou and C. Bai, Nijenhuis operators on $n$-Lie algebras, _Commun. Theor. Phys._ 65 (2016), no. 6, 659-670.
  * [28] A. Makhlouf, On deformations of $n$-Lie algebras. Non-associative and non-commutative algebra and operator theory, _Springer Proc. Math. Stat._ 160, Springer, Cham, (2016), 55-81.
  * [29] H. Munthe-Kaas and A. Lundervold, On post-Lie algebras, Lie-Butcher series and moving frames, _Found. Comput. Math._ 13 (2013), no. 4, 583-613.
  * [30] Y. Nambu, Generalized Hamiltonian dynamics, _Phys. Rev. D_ 7 (1973), 2405-2412.
  * [31] J. Pei, C. Bai and L. Guo, Splitting of operads and Rota-Baxter operators on operads, Appl. Cate. Stru. 25 (2017), 505-538.
  * [32] M. Rotkiewicz, Cohomology ring of $n$-Lie algebras, _Extracta Math._ 20 (2005), 219-232.
  * [33] M. A. Semonov-Tian-Shansky, What is a classical R-matrix? _Funct. Anal. Appl._ 17 (1983), 259-272.
  * [34] L. A. Takhtajan, Higher order analog of Chevalley-Eilenberg complex and deformation theory of $n$-gebras, _St. Petersburg Math. J._ 6 (1995), no. 2, 429-438.
  * [35] R. Tang, C. Bai, L. Guo and Y. Sheng, Deformations and their controlling cohomologies of $\mathcal{O}$-operators, _Comm. Math. Phys._ 368 (2019), 665-700.
  * [36] R. Tang, S. Hou and Y. Sheng, Lie $3$-algebras and deformations of relative Rota-Baxter operators on $3$-Lie algebras, _J. Algebra_ 567 (2021), 37-62.
  * [37] B. Vallette, Homology of generalized partition posets, _J. Pure Appl. Algebra_ 208 (2007), 699-725.
  * [38] Th. Voronov, Higher derived brackets and homotopy algebras, _J. Pure Appl. Algebra_ 202 (2005), 133-153.
  * [39] K. Wang and G. Zhou, Deformations and homotopy theory of Rota-Baxter algebras of any weight, arXiv:2108.06744.