# Statistical Deformation Reconstruction Using Multi-organ Shape Features for
Pancreatic Cancer Localization

Megumi Nakao, Mitsuhiro Nakamura, Takashi Mizowaki and Tetsuya Matsuda M.
Nakao, and T. Matsuda are with Graduate School of Informatics, Kyoto
University, Yoshida-Honmachi, Sakyo, Kyoto, 606-8501, JAPAN, e-mail:
<EMAIL_ADDRESS>Nakamura is with Graduate School of Medicine, Human
Health Sciences, Kyoto University.T. Mizowaki is with Dept. of Radiation
Oncology and Image-Applied Therapy, Kyoto University Hospital.M. Nakao and M.
Nakamura are contributed equally to this work.

###### Abstract

Respiratory motion and the associated deformations of abdominal organs and
tumors are essential information in clinical applications. However, inter- and
intra-patient multi-organ deformations are complex and have not been
statistically formulated, whereas single organ deformations have been widely
studied. In this paper, we introduce a multi-organ deformation library and its
application to deformation reconstruction based on the shape features of
multiple abdominal organs. Statistical multi-organ motion/deformation models
of the stomach, liver, left and right kidneys, and duodenum were generated by
shape matching their region labels defined on four-dimensional computed
tomography images. A total of 250 volumes were measured from 25 pancreatic
cancer patients. This paper also proposes a per-region-based deformation
learning using the reproducing kernel to predict the displacement of
pancreatic cancer for adaptive radiotherapy. The experimental results show
that the proposed concept estimates deformations better than general per-
patient-based learning models and achieves a clinically acceptable estimation
error with a mean distance of 1.2 $\pm$ 0.7 mm and a Hausdorff distance of 4.2
$\pm$ 2.3 mm throughout the respiratory motion.

###### Index Terms:

Statistical deformation library, multi-organ motion analysis, reproducing
kernel, adaptive radiotherapy

##  I Introduction

Statistical formulation of respiratory motion including the deformation of
multiple organs and tumors is of increasing interest in external-beam
radiotherapy. Specifically, in image-guided radiotherapy (IGRT), it is
important to reduce exposure in normal tissues while accurately targeting
lesions with respiratory motion. However, large anatomical variations in the
shape and motion of abdominal organs can occur when treatment lasts several
weeks, while time-series three-dimensional (3D) computed tomography (CT)
images can only be obtained for initial radiation planning [1][2]. Recent
technical advances enable treatment plans to be modified based on a daily
X-ray cone-beam CT (CBCT). This process is called adaptive radiotherapy (ART)
[3][4]. Abdominal organs such as the stomach, and duodenum or the pancreas
neighbor each other, but the pancreas cannot clearly be detected, even on CBCT
images. Because missing pixel values or artifacts often appear in CBCT images
(see Fig. 1), ART for abdominal regions is technically more difficult and
remains a challenging area of research [2][3]. To approach these issues, this
paper focuses on the potential of model-based tumor localization in pancreatic
cancer treatment using statistical modeling of patient-specific multi-organ
motion and deformation.

Statistical shape modeling (SSM) has been widely investigated for the modeling
of organ shapes based on prior knowledge [5], and interest has grown in its
application in machine learning with intra- or inter-patient datasets with
point-to-point correspondence [1][2][6][7]. For instance, interfractional
shape variations in the prostate and rectum have been statistically modeled
for radiation therapy planning [6][8][9]. A statistical deformation model
(SDM) [10][11] based on a four-dimensional (4D) CT images has also been
reported. Unlike physics-based modeling [12][13][14], statistical modeling is
a data-driven approach that does not explicitly describe the elasticity and
physical conditions of organs. Image-based [15][16], point-based [8][17], or
mesh-based [1][2][6][18] deformable registration techniques have been used to
obtain point-to-point correspondence between two datasets. Specifically,
respiratory motion [11][12][16][19] has mainly been investigated in the field
of image-based lung modeling.

Our aim in this paper is to statistically model inter- and intra-patient
deformation along with the motion of multiple abdominal organs. The
relationships among organs, especially those between pancreatic cancer and its
surrounding organs, are also interesting and worth investigating [20][21].
Image-based registration assumes that the 3D image or the regions of interest
of organs are a continuous space. It also assumes that the displacement is
spatially smooth for regularization [15][22][23][24]. However, the mechanism
for multi-organ deformation is complex and not fully understood. Respiratory
motion widely affects abdominal regions, in which several organs are adjacent
to or connected with each other. Therefore, to analyze individual motion and
deformation accurately, a model capable of capturing sliding motion or
rotation near the organ boundary is needed [11]. Hence, interest in deformable
mesh registration (DMR) has recently resurged. The inter- and intra-patient
variations in cervix–uterus anatomy were recently modeled for radiation
therapy planning [1][7]. The correlation of liver and pancreas tumor motion
[25] and daily changes in the stomach, duodenum, and pancreas [2] have been
analyzed through DMR. To the best of our knowledge, there has been no report
describing SDM-based deformation reconstruction or clarifying the relationship
between the inter- and intra-patient deformations of multiple organs and
tumors, even though the clinical needs for such modeling in radiotherapy and
motion recognition are high [26][27][28][29].

In this paper, we introduce a localized deformation reconstruction framework
based on the shape features of multiple organs. To model the relationships of
the spatial deformation fields between abdominal organs and the tumor, a
statistical multi-organ deformation model (SMDM) is constructed by shape
matching organ meshes generated from 4D-CT datasets (250 volumes). We
formulate the SMDM as a reproducing kernel, with the motion and deformation of
the pancreatic cancer region as the estimation target. Five abdominal organs
(the liver, stomach, duodenum, and right and left kidneys) located around the
pancreas are used as multidimensional shape features. In such a procedure, a
low number of cancer patients in the dataset can decrease the estimation
performance, which is a common problem in clinical machine learning. This
paper proposes the concept of localized deformation reconstruction to model
nonlinear motion for a local small region of the estimation target, rather
than on a per-patient basis. Here, to design the reproducing kernel framework
that maps motion with deformation of pancreatic cancer and its surrounding
organs, we address the following fundamental issues:

  * •

We investigate the level of complexity of the motion dynamics of multiple
abdominal organs, their deformation, and relationships.

  * •

We determine the design of DMR to build SMDM in abdominal region.

  * •

We investigate which organ sets are good estimators to predict the
motion/deformation of gross tumor volume (GTV) and determine an appropriate
number of dimensions of the feature space.

  * •

We evaluate the final estimation performance for the inter- and intra-patient
validation of GTV and address whether the proposed concept is clinically
acceptable.

Figure 1: Clinical needs for indirect deformation reconstruction in ART: (a)
CBCT images with missing pixels and (b) a conceptual image for pancreatic
cancer localization from multi-organ shape features.

We note that the purpose of shape/deformation reconstruction is different from
that of image segmentation, for which a variety of deep learning methods that
directly use image features in the regions of interest have been proposed
[30]. In the proposed kernel-based formulation of SMDM, as shown in Fig. 1(b),
even if pancreatic cancer is totally "invisible" because of missing pixels or
artifacts in CBCT images, its motion and deformation can be indirectly
reconstructed from multi-organ features. This estimation helps optimize
radiation treatment in that the radiation dose can be locally transported to
the moving tumor while keeping safe a margin around organs at risk.

Figure 2: Deformation reconstruction framework using the shape features of
multiple abdominal organs. (a) CT slice image with the 3D contours and vertex
distribution of the meshes. (b) The template meshes are registered to
individual organ regions using Laplacian-based diffeomorphic shape matching
(LDSM). The methods are mainly used to predict the motion and deformation of
pancreatic cancer using reproducing kernel regression.

##  II Methods

###  II-A Statistical Multi-organ Deformation Model

In this study, 4D-CT images ${I}_{t}^{(k)}$ ($k=1,2,...,25$: patient ID,
$t=1,2,...,10$: time phase of 4D-CT images) of 25 pancreatic cancer patients
who underwent intensity-modulated radiotherapy (IMRT) in Kyoto University
Hospital were used for statistical modeling and deformation learning. This
study was performed in accordance with the Declaration of Helsinki and was
approved by our institutional review board (approval number: R1446). 4D-CT
images for a patient consisted of 3D-CT images (image size: $512\times 512$
pixels and 88–152 slices, voxel resolution: $1.0mm\times 1.0mm\times 2.5mm$)
of 10 time phases for one respiratory cycle. All images were measured under
the condition of respiratory synchronization, where $t=1$ corresponds to the
end-inhalation phase and $t=6$ corresponds to the end-exhalation phase. The
region labels of the entire body, stomach, liver, duodenum, left and right
kidneys, and the GTV of the pancreatic cancer were predefined by board-
certified radiation oncologists in the radiotherapy planning, as shown in Fig.
2(a). We note that accurate automatic extraction is difficult for the stomach,
duodenum, and pancreas because of their unclear boundaries, contrasts, and air
contents. Their shapes can be substantially deformed during the radiotherapy
period. Most of anatomical segmentation techniques assume that there are no
image defects nor a wide range of artifacts appearing in the CBCT images. In
this study, the 3D contours manually defined as the organs at risk (OARs) for
dose calculation in the radiation planning were used as the multi-organ shape
database, and their tetrahedral meshes ${S}_{t}^{(k)}$ were generated. Figure
2(a) show an example of tetrahedral meshes and their volumetric point
distribution for five organs and the GTV of the pancreatic cancer.

Figure 2(b) shows the flow of the developed framework. The meshes
$S_{t}^{(k)}$ differ in the number of vertices and the structure of the mesh,
because they were independently generated from different CT images. As shown
in Fig. 3, the corresponding models $M_{t}^{(k)}$ (with the same vertex and
the same mesh structure) that precisely approximate the surfaces of
$S_{t}^{(k)}$, respectively, were computed by DMR using a template $T$ and the
target mesh. Because the registered models achieve point-to-point
correspondence, spatial deformation $D_{t}^{(k)}=M_{t}^{(k)}-M_{1}^{(k)}$ can
be represented by calculating the displacement vector of the corresponding
vertex.

To stably compute a 3D deformation field of multiple organs, accurate shape
matching is required. The shape variation and deformation of abdominal organs
such as the stomach and duodenum include considerable volume changes including
rotations and sliding boundaries. Figure 4 shows examples of complex inter-
patient shape changes and the interaction between multiple abdominal organs. A
part of the liver and stomach deform in the different directions near their
boundaries. The duodenum and right kidney do not contact each other in the
template model, but are in contact in the patient models. Image-based
registration generally deals with multiple organs as one continuous image
space, and an individual organ’s deformation or interactions among specific
organs cannot be discriminated. In addition, the registration error tends to
increase in areas with large curvature such as the tips of the organs and
boundaries of neighboring organs [11][18]. Specifically, our focus in this
paper is to model multi-organ motion/deformation and its interaction, and to
examine the efficacy of the proposed feature-preserving registration methods
for abdominal organs with large shape variations such as the stomach and
duodenum. To capture rotational components or possible sliding motions caused
by the interaction of multiple organs, the registration process is applied to
each organ’s mesh independently. In addition to intra-patient registration,
our approach enables the construction of a SDM, making inter-patient
deformation analysis possible. The details of the applied DMR method, LDSM,
are described in Section II-C.

Figure 3: Flow of the inter- and intra-patient shape matching. The spatial
deformation $D_{t}$ between the end-inhalation state and time phase $t$ is
obtained from the corresponding vertices of the two registered meshes.  Figure
4: Examples of complex inter-patient shape changes and the interactions among
multiple abdominal organs. (a) A part of the liver and stomach deform in
different directions near their boundaries (red and blue arrows). (b) The
duodenum and right kidney do not contact each other in the template model, but
are in contact in the patient models (green and yellow arrows).

For the template mesh generation, first, one case was randomly selected to be
the initial mesh for $T$, and its surface was resampled to 400 vertices and
796 triangles. Next, the corresponding mesh models ${M}_{I}^{(k)}$ were
obtained by registering $T$ to the individual surfaces $S_{I}^{(k)}$ in the
inflated state. Because the mesh models ${M}_{t}^{(k)}$ have point-to-point
correspondence, the average shape $\overline{M}$ can be obtained by
calculating the average of each coordinate. We used $\overline{M}$ as the
final template. This process was performed on the stomach, liver, left and
right kidneys, and duodenum. By keeping the template close to the data in
advance, our aim is to reduce the influence of the adopted data selection
while preventing an increase in matching error.

###  II-B Localized deformation learning model

In the prediction stage, the time-varying position of the pancreatic cancer is
estimated using the reproducing kernels of the multi-organ features. Here, as
the second technical challenge, we introduce the localized multi-organ
features for modeling inter- and intra-patient variation of deformation from a
dataset containing a limited number of patients. To model spatial deformation,
per-patient-based learning, in which one patient’s data are used as one set of
training data, has been generally employed as a straightforward approach. The
displacement of the region is computed by synthesizing that of the
corresponding regions of the training dataset with similar shape features. In
this case, the number of training data equals the number of patients in
dataset $m$.

Instead of reconstruction on a per-patient basis, the aim of this paper is to
formulate a method for local deformation reconstruction that models nonlinear
motion for a small region of the estimation target. In the proposed learning
model, a small local region of the target organ is used as one set of training
data. This approach is based on the hypothesis that local regions with similar
shape features show similar displacements, which is commonly assumed in each
continuous space of organs. In this approach, the displacement of the region
is calculated by combining the training datasets of all the regions including
other parts of the target organs. The number of training data is $m\times n$
when the number of patients in the data is $m$ and $n$ small regions of the
target organ are considered. The details of the deformation reconstruction
framework is described in Section II-D.

###  II-C Feature preserving deformable mesh registration

In this study, to achieve both globally stable and locally strict DMR, our aim
is to address the trade-off between feature-preserving shape matching and
spatially smooth deformation. To approach this in DMR, the concept of
progressive, feature-preserving shape update [18][31] is introduced into the
large deformation diffeomorphic metric mapping (LDDMM) scheme [22][32]. The
objective function in the proposed LDSM is described as follows.

$\displaystyle
E(\mbox{\boldmath$u$})=d(Y,\phi(X))+\int_{0}^{1}\|L(\mbox{\boldmath$u$}(s))\|^{2}ds,$
(1)

where $X$ is the source (template) mesh, $Y$ is the target surface, and $d$ is
the distance function between the two surfaces. In addition, $\phi(X)$ is a
continuous and differentiable transformation that maps $X$ to the deformed
mesh, $L(\cdot)$ is the Laplace–Beltrami operator and $L(\mbox{\boldmath$u$})$
is the discrete Laplacian of the displacement field.

The first term evaluates the difference between the deformed template and the
target surface. The second is a regularization term to make the deformation
field $u$ smooth. In the context of image-based LDDMM, the difference in voxel
intensities between the deformed and the target image is evaluated in the
first term. Minimizing the nearest point-to-point distance was originally
employed as a basic strategy in DMR, however, it does not consider 3D
geometry, and maintaining mesh topology is difficult with this approach. We
focus on the importance of feature preservation to avoid incorrectly matching
local structures [18] in mapping distant structures, 3D geometric information
of the template mesh is used to preserve the local features of organ shapes.
In the mapping function $\phi$, a Laplacian-based shape matching (LSM) scheme
[31] is introduced for progressive shape updates while preserving features as
much as possible. The mean value of the nearest bidirectional point-to-surface
distance (called the mean distance in this paper) is used for $d$. A discrete
Laplacian was first formulated for geometry modeling [33], and has recently
been applied to the non-rigid shape registration of anatomical structures
[18][31]. In [31], LSM registers curved surfaces with shape variations better
than LDDMM.

Let $X$ denote a tetrahedral mesh with $n$ vertices
$\mbox{\boldmath$v$}_{i}\in V(i=1,2,...,n)$ and edges, the deformation map
$\phi(X)$, that is, the deformation field $u$ in the LSM, is obtained by
iteratively updating $\mbox{\boldmath$v$}_{i}$ while minimizing the following
objective function:

$\displaystyle E(\mbox{\boldmath$u$})$ $\displaystyle=E_{shape}+\delta
E_{pos}$ (2)
$\displaystyle=\sum_{i=1}^{n}\|L(\mbox{\boldmath$v$}^{\prime}_{i})\
-L(\mbox{\boldmath$v$}_{i})\|^{2}+\delta\sum_{i=1}^{n}\|\mbox{\boldmath$p$}_{i}-\mbox{\boldmath$v$}_{i}\|^{2},$

where ${\mbox{\boldmath$v$}^{\prime}_{i}}$ is the vertex position to be
solved, ${\mbox{\boldmath$p$}_{i}}$ is a positional constraint set to
${\mbox{\boldmath$v$}_{i}}$, and $\delta$ is a weight parameter configured
according to the problem. For the definition of the positional constraints
between the template and target shape, refer to [18].
$L({\mbox{\boldmath$v$}_{i}})$ is the discrete Laplacian at vertex
${\mbox{\boldmath$v$}_{i}}$, defined by

$\displaystyle L(\mbox{\boldmath$v$}_{i})=\sum_{j\in
N(\mbox{\boldmath$v$}_{i})}\omega_{ij}(\mbox{\boldmath$v$}_{i}-\mbox{\boldmath$v$}_{j})$
(3)

Here, $\omega_{ij}$ are the edge weights and $N({\mbox{\boldmath$v$}_{i}})$ is
the number of adjacent vertices of one ring connected by vertex
${\mbox{\boldmath$v$}_{i}}$ and the edge. The discrete Laplacian is used as a
shape descriptor and approximates the mean curvature normal of the triangular
mesh. Although there are several variations of the weights, the general one is
cotangent discretization based on the per-edge Voronoi areas. The first term
in Eq. (2) is a penalty for shape changes to the mesh, and the second term
increases if the constrained vertex is distant from the nearest surface of the
target mesh. By computing ${\mbox{\boldmath$v$}^{\prime}_{i}}$, which
minimizes the objective function, the template model is updated while
preserving the shape as much as possible.

In the proposed LDSM, in addition to LSM, the regularization term in Eq. (1)
is used to achieve a spatially smooth and diffeomorphic deformation. This can
be simply implemented by adding the term to Eq. (2) as follows:

$\displaystyle E(\mbox{\boldmath$u$})$
$\displaystyle=E_{shape}+E_{deform}+\delta E_{pos}$ (4)

where $E_{deform}$ is the magnitude of the discrete Laplacian of the
deformation field $u$ defined by

$\displaystyle E_{deform}$
$\displaystyle=\sum_{i=1}^{n}\|L(\mbox{\boldmath$u$}_{i})\|^{2}$ (5)

Because Eq. (5) is a quadratic form at vertex positions
${\mbox{\boldmath$v$}_{i}}$, the minimization problem can be calculated
efficiently. The step-by-step update avoids local mismatches at the early
stage if there is a considerable distance between the two surfaces. Once the
template is updated, local displacement
$\mbox{\boldmath$u$}_{i}=\mbox{\boldmath$v$}^{\prime}_{i}-\mbox{\boldmath$v$}_{i}$
is obtained. We note that the original LDDMM does not preserve the shape of
the template, and some studies report results with unstable or irregular
matching especially for large deformations with rotation [18][31]. In the
proposed framework, in addition to the regularization term, Laplacian-based
shape preservation is introduced into the iterative deformation process.

###  II-D Reproducing kernels for deformation reconstruction

This section explains the per-region-based localized deformation learning
proposed in this paper, by comparing it with the per-patient-based deformation
learning generally used in population-based modeling [1][6]. Figure 5 briefly
illustrates the concept. Because the aim of this paper is to reconstruct
target deformation from multi-organ features, the displacement vector
$\mbox{\boldmath$y$}_{i}$ is mapped from multiple points sampled from the
surrounding organs, as shown in Fig. 5(a). The local displacement
${\mbox{\boldmath$y$}}_{i}$ of GTV can be calculated from the feature vectors
of the sampled points and is composed using the relative position
${\mbox{\boldmath$r$}}_{ij}$ and displacement ${\mbox{\boldmath$u$}}_{j}$.

Here, the problem is to learn the mapping from the registered multi-organ
models generated through DMR. Per-patient-based learning is a straightforward
approach, where the mesh model $X$ obtained from one patient’s data is used as
one set of training data. It is based on the idea that the displacement of
local regions of organs is similar to that of the corresponding regions of the
training dataset with similar shape features. Specifically, the displacement
$\mbox{\boldmath$u$}_{i}$ at vertex $i$ is learned from the displacement
$\mbox{\boldmath$u$}_{i}$ of the corresponding vertex
$\mbox{\boldmath$v$}_{i}$ in other cases. However, in per-region-based
localized deformation learning, a small region $\Omega$ of the mesh model $X$
is used as one set of training data. This approach is based on the idea that
local regions with similar shape features show similar displacement, which is
commonly assumed if a curved manifold is used to represent an organ surface.
For instance, in Fig. 5(b), the displacement vectors adjacent to
$\mbox{\boldmath$v$}_{i}$ should be similar to those of
$\mbox{\boldmath$v$}_{i}$. In other words, the displacement
$\mbox{\boldmath$u$}_{i}$ can be learned from the displacement
$\mbox{\boldmath$u$}_{j}$ of all vertices $\mbox{\boldmath$v$}_{j}$ in other
cases.

Figure 5: Per-region-based localized deformation learning: (a) a displacement
mapping model, and (b) deformation learning by considering the continuity of
spatial deformation.  Figure 6: Examples of displacement mapping models using
multi-organ shape features. (a) Shape features sampled from the liver and (b)
from the stomach and duodenum.

We formulate the per-region-based deformation learning model using reproducing
kernels. Based on the spatial mapping in Fig. 5(a), the local displacement
$\mbox{\boldmath$y$}_{i}$ of GTV is computed from multi-organ shape features
$\mbox{\boldmath$x$}_{i}$ using Eq. (6).

$\displaystyle\mbox{\boldmath$y$}_{i}=\sum_{j=1}^{N}\mbox{\boldmath$\alpha$}_{j}k(\mbox{\boldmath$x$}_{i},\mbox{\boldmath$x$}_{j}),\hskip
8.5359pt\mbox{\boldmath$x$}_{j}\in\it{X},\hskip
8.5359pt\mbox{\boldmath$\alpha$}_{j}\in\mathbb{R}^{N}$ (6)

where $k:\it{X}\times\it{X}\rightarrow\mathbb{R}$ is the reproducing kernel,
$N$ is the number of training datasets, and $\mbox{\boldmath$\alpha$}_{j}$ is
the weight vector. A Gaussian function is used for kernel function $k$, which
is
$k(\mbox{\boldmath$x$}_{i},\mbox{\boldmath$x$}_{j})=exp(-\beta||\mbox{\boldmath$x$}_{i}-\mbox{\boldmath$x$}_{j}||^{2}/N)$.
For a given
$\mbox{\boldmath$y$}=[\mbox{\boldmath$y$}_{1},\dots,\mbox{\boldmath$y$}_{N}]^{T}$,
$\mbox{\boldmath$\alpha$}=[\mbox{\boldmath$\alpha$}_{1},\dots,\mbox{\boldmath$\alpha$}_{N}]^{T}$
are calculated by minimizing the cost function E($\alpha$), which is expressed
as

$\displaystyle
E(\mbox{\boldmath$\alpha$})=\|\mbox{\boldmath$y$}-K\mbox{\boldmath$\alpha$}\|^{2}+\lambda\mbox{\boldmath$\alpha$}^{T}K\mbox{\boldmath$\alpha$}$
(7)

where $K\in\mathbb{R}^{N\times N}$ is the kernel matrix whose elements are
defined by $K_{ij}=k(\mbox{\boldmath$x$}_{i},\mbox{\boldmath$x$}_{j})$, and
$\lambda$ is the regularization parameter, which penalizes deviations of
$\alpha$. The optimized weights are given by
$\mbox{\boldmath$\alpha$}=(K+\lambda I)^{-1}\mbox{\boldmath$y$}$ ($I$:
indentity matrix).

The feature vector $\mbox{\boldmath$x$}_{i}$ of the local region for per-
region-based learning is constructed using the relative position
$\mbox{\boldmath$r$}_{ij}$ of the target vertex and the displacement vector
$\mbox{\boldmath$u$}_{j}$ of the surrounding organs as follows.

$\displaystyle\mbox{\boldmath$x$}_{i}=[\mbox{\boldmath$r$}_{ij},\mbox{\boldmath$u$}_{j}],\hskip
8.5359pt\mbox{\boldmath$r$}_{ij}=\mbox{\boldmath$v$}_{i}-\mbox{\boldmath$v$}_{j}$
(8)

In this study, the sampled vertices of the SMDM obtained from 24 patients were
used in leave-one-out cross-validation to construct the kernel matrix, and all
vertices of the five organs (liver, left and right kidneys, stomach, and
duodenum) were considered as candidates for shape features $x$.

$\displaystyle\mbox{\boldmath$x$}_{ALL}=[\mbox{\boldmath$x$}_{ST},\mbox{\boldmath$x$}_{DU},\mbox{\boldmath$x$}_{LI},\mbox{\boldmath$x$}_{LK},\mbox{\boldmath$x$}_{RK}]$
(9)

where ST, DU, LI, LK, and RK are the stomach, duodenum, liver, left kidney,
and right kidney, respectively. For instance, when 50 vertices are sampled
from each organ model, the dimension of the feature vector
$\mbox{\boldmath$x$}_{i}$ is 1,500 according to
$\mbox{\boldmath$r$}_{ij},\mbox{\boldmath$u$}_{j}\in\mathbb{R}^{750}$. Because
the shape features are evaluated in high-dimensional feature space, in per-
patient-based deformation reconstruction, the estimation error may increase if
there are no corresponding vertices with similar characteristics among the 24
cases.

Alternatively, in the proposed per-region-based deformation learning, the
displacement is locally learned per vertex; in other words, it can be
reconstructed from the deformation of different regions. Therefore, when 200
vertices are used to represent the target pancreatic cancer (for example), the
training dataset is substantially increased to $24\times 200=4,800$, whereas
the number of dimensions of the feature space is 1,500, which is the same as
that of per-patient-based approach. This means that the number of neighborhood
data points increase in the generated feature space, and the displacement of
the target vertex can be reconstructed using more datasets. Therefore, using a
per-region-based kernel formulation improves the estimation performance and
more stable results can be expected.

As described in the introduction, we are here interested in the following
questions: which organ sets are good estimators for deformation reconstruction
and what number of dimensions of the feature space is appropriate. Figure 6
shows two examples in which the three vertices are mapped to the target
vertex, which is a problem of obtaining the target displacement based on
features sampled from different organ sets, $[\mbox{\boldmath$x$}_{LI}]$ from
the liver or $[\mbox{\boldmath$x$}_{ST},\mbox{\boldmath$x$}_{DU}]$ from the
stomach and duodenum. The liver does not have clear anatomical connectivity to
the pancreas, and the shape feature is relatively stable with a small
deformation. The stomach and duodenum are connected to the pancreas, but their
deformation is large [2] and unstable. Because the shape features of all
organs are not always obtained from CBCT images, to explore the prediction
performance when using specific organ sets is worth investigating for clinical
application. In the experiments, we examine the efficacy of the proposed
concept and investigate the possible organ sets and the proper number of
dimensions of the feature space for localizing pancreatic cancer.

##  III Experiments

In the experiments, statistical multi-organ deformation models were first
generated by inter- and intra-patient shape matching. The registration
performance for each organ was confirmed while comparing it with the results
from three existing registration methods. Then, the efficacy of the multi-
organ shape features in predicting the motion and deformation of pancreatic
cancer was analyzed. The performance of the proposed per-region-based
deformation learning was evaluated by comparing it with conventional per-
patient-based learning. The value of weight parameter $\beta$ for kernel
function was determined from the results of numerical experiments.

###  III-A Shape matching performance

In this study, the mean distance [1][31], the Hausdorff distance [34], and
Laplacian of the displacement [18] were used as the shape similarity criteria.
The Hausdorff distance measures the longest distance between two surfaces,
whereas the mean distance is the mean value of the nearest bidirectional
point-to-surface distance. Unlike segmentation or recognition problems,
statistical modeling requires point-to-point local correspondence between two
shapes. For example, because the Dice coefficient only measures volume
overlap, it is not suitable for evaluating per-vertex correspondence, nor is
it suitable for measuring the quality of local matching. The Laplacian of the
displacement is the magnitude of the second derivatives of the displacement
field, and it evaluates the smoothness of deformation. If local shape matching
is achieved while preserving vertex density, this value decreases.

The proposed shape matching method (LDSM) was compared with three existing
shape matching approaches, that is,

  * •

Piecewise Affine Transformation (PWA) [35][36]

  * •

Large deformation diffeomorphic metric matching
(LDDMM) [32]

  * •

Laplacian-based shape matching (LSM) [31][37]

For all algorithms, the affine transformation was processed in advance to
match the posture and volume of the overall shape globally.

Figure 7: Statistical motion dynamics of five abdominal organs and the GTV of
pancreatic cancer. The means and standard deviations of the corresponding
vertices are plotted in the graphs. Figure 8: Statistical deformation
representation of the liver, stomach, duodenum, and right kidney. Deformation
variations corresponding to the first two eigenvalues from the registered
models are visualized. The color map shows the signed distance from the mean
deformation.

Table I shows the quantitative comparison results of the DMR algorithms for
each of the five organs. LDSM and LSM achieved a significantly smaller mean
distance and Hausdorff distance with an error of less than 1 mm, outperforming
PWA and LDDMM in terms of matching volumetric regions. Regarding the Laplacian
of the displacement field, LDSM obtained smaller values than LSM, indicating
that smooth deformation that reduces unstable surface matching can be
performed by LDSM. In DMR, the accuracy of shape representation and smooth
deformation from the template is a trade-off. Using the proposed LDSM method,
the Hausdorff distance is less than 1 mm, and the Laplacian of displacement is
better than that pf LSM. Based on these results, we selected the LDSM for
constructing SMDM of the five organs.

TABLE I: Quantitative comparison results: mean distance (MD), Hausdorff
distance (HD), mean and maximum Laplacian of the displacement (LD), and Dice
similarity coefficient (DSC) of the DMR algorithms for the five organs.

Liver | Methods
---|---
PWA | LDDMM | LSM | LDSM
MD [mm] | 1.6 (1.1 – 2.4) | 0.5 (0.3 – 0.7) | 0.2 (0.1 – 0.3) | 0.2 (0.1 – 0.3)
HD [mm] | 10.0 (4.7. – 28.7) | 3.6 (1.3 – 14.8) | 0.9 (0.4 – 1.9) | 1.1 (0.5 – 2.2)
LD (mean) [mm] | 2.3 (1.0 – 4.6) | 1.1 (0.8 – 1.6) | 1.5 (1.0 – 2.2) | 1.2 (0.8 – 1.7)
LD (max) [mm] | 12.0 (4.6 – 26.9) | 6.8 (3.6 – 13.3) | 8.8 (4.4 – 20.6) | 7.0 (3.7 – 15.2)
DSC [%] | 95.7 (92.4 – 97.2) | 98.0 (96.1 – 98.6) | 98.1 (96.1 – 98.7) | 98.1 (96.1 – 98.7)

Stomach | Methods
---|---
PWA | LDDMM | LSM | LDSM
MD [mm] | 1.4 (0.9 – 2.1) | 0.5 (0.2 – 1.1) | 0.2 (0.1 – 0.3) | 0.2 (0.1 – 0.4)
HD [mm] | 7.3 (3.7 – 14.9) | 3.1 (1.3 – 10.4) | 0.8 (0.3 – 1.9) | 0.9 (0.4 – 1.8)
LD (mean) [mm] | 2.1 (0.8 – 5.0) | 1.1 (0.7 – 2.0) | 1.3 (0.8 – 2.5) | 1.1 (0.6 – 2.0)
LD (max) [mm] | 8.7 (3.6 – 43.2) | 6.3 (3.0 – 10.8) | 7.7 (3.9 – 15.7) | 6.6 (2.9 – 12.2)
DSC [%] | 92.6 (89.4 – 95.6) | 96.5 (95.3 – 97.5) | 97.2 (96.0 – 98.1) | 97.0 (95.7 – 98.2)

Duodenum | Methods
---|---
PWA | LDDMM | LSM | LDSM
MD [mm] | 1.2 (0.7 – 2.9) | 0.5 (0.3 – 1.4) | 0.2 (0.1 – 0.5) | 0.2 (0.1 – 0.5)
HD [mm] | 6.7 (2.9 – 23.0) | 3.6 (0.8 – 13.1) | 0.8 (0.3 – 4.2) | 0.9 (0.4 – 5.0)
LD (mean) [mm] | 1.5 (0.6 – 2.6) | 1.0 (0.6 – 2.1) | 1.2 (0.6 – 2.9) | 1.0 (0.6 – 2.3)
LD (max) [mm] | 6.6 (2.2 – 16.3) | 6.9 (3.0 – 26.8) | 7.8 (2.9 – 28.6) | 7.2 (2.9 – 22.5)
DSC [%] | 86.9 (54.1 – 92.6) | 93.1 (70.6 – 96.4) | 94.2 (69.0 – 97.4) | 94.2 (72.2 – 97.2)

Left kidney | Methods
---|---
PWA | LDDMM | LSM | LDSM
MD [mm] | 0.8 (0.6 – 1.2) | 0.3 (0.2 – 0.9) | 0.1 (0.1 – 0.2) | 0.1 (0.1 – 0.3)
HD [mm] | 4.8 (2.7 – 9.4) | 2.8 (0.7 – 16.4) | 0.5 (0.2 – 1.8) | 0.7 (0.2 – 2.1)
LD (mean) [mm] | 1.7 (0.7 – 4.1) | 0.9 (0.5 – 2.2) | 1.2 (0.6 – 2.9) | 1.0 (0.5 – 2.3)
LD (max) [mm] | 7.5 (2.6 – 21.7) | 6.9 (3.1 – 15.4) | 7.8 (3.0 – 24.6) | 7.2 (2.6 – 21.7)
DSC [%] | 95.9 (90.0 – 96.9) | 97.2 (95.9 – 98.4) | 97.4 (96.0 – 98.4) | 97.4 (95.9 – 98.5)

Right kidney | Methods
---|---
PWA | LDDMM | LSM | LDSM
MD [mm] | 0.8 (0.6 – 1.0) | 0.3 (0.2 – 1.0) | 0.1 (0.1 – 0.2) | 0.1 (0.1 – 0.3)
HD [mm] | 4.3 (2.4 – 9.1) | 2.9 (0.5 – 13.1) | 0.4 (0.2 – 0.9) | 0.6 (0.3 – 2.0)
LD (mean) [mm] | 1.6 (0.7 – 3.1) | 0.8 (0.4 – 1.6) | 1.0 (0.5 – 2.0) | 0.8 (0.5 – 1.6)
LD (max) [mm] | 8.6 (2.7 – 20.9) | 6.9 (2.7 – 14.5) | 8.1 (2.9 – 23.1) | 7.0 (2.7 – 21.6)
DSC [%] | 96.2 (94.8 – 97.3) | 97.4 (96.3 – 97.9) | 97.7 (96.6 – 98.6) | 97.6 (96.4 – 98.4)

###  III-B Multi-organ deformation analysis

So far, no study has investigated the impact of inter- and intra-subject
variation on abdominal multi-organ deformation. Our DMR framework can directly
provide a statistical representation of the registered organ models
$M_{t}^{(k)}$, which can then generate the mean and variation of deaeration
deformation between subjects.

####  III-B1 Statistical motion dynamics

Figure 7 shows the statistical motion dynamics with deformation computed from
registered organ models for ten time phases in one respiratory cycle. The mean
displacement for all corresponding vertices are visualized as the centerline,
and the standard deviation is depicted as a colored band. The graph shows that
the mean and standard deviation of the displacement at the end-expiration
phase is $12.1\pm 7.2$ mm for the liver, $10.3\pm 5.5$ mm for the stomach,
$10.2\pm 5.9$ mm for the duodenum, $11.4\pm 6.2$ mm for the left kidney,
$13.8\pm 8.4$ mm for the right kidney, and $7.6\pm 4.2$ mm for the GTV of the
pancreatic cancer. The standard deviation is relatively large compared with
the magnitude of the displacement. This indicates that there are large
individual variations in respiratory motion and organ deformation. Hence, to
estimate the 3D tumor region with only a mean deformation model would be
difficult.

Figure 9: Comparison of prediction performance between per-patient- and per-
region-based learning. (a) mean distance, (b) Hausdorff distance, (c) Dice
similarity coefficient and (d) estimation error of tumor localization with
respect to the number of sampling points.

####  III-B2 Statistical deformation model

Figure 8 shows the deformation modes that correspond to the first two
eigenvalues of the registered organ models. The eigenvalues and eigenvectors
were computed from the set of displacement vectors of all vertices based on
singular value decomposition. The central figures show the mean shape and mean
displacement. The translucent shape is the end-inspiration phase ($t=1$), and
the opaque shape is the end-expiration phase ($t=6$). The left/right images
were generated by changing the weights to plus/minus $\sigma$, which is twice
the square root of the eigenvalues. The supplemental movies visualize the
sequential motion expression by interpolating the end-inspiration and end-
expiration phases. The color map shows the signed distance from the mean
deformation.

The types of variety of motion and deformation can be characterized according
to their morphological properties as follows:

  * •

The first eigenvector mainly encompasses variation in the scale of
deformation, which indicates that individual difference is large during the
respiratory cycle.

  * •

The second eigenvector is associated with the directions and rotations of
deformation. Interestingly, the rotation axis and direction of rotation differ
for each organ.

We also confirmed that the subspace representation using two eigenvectors
explains 96.1% of the total deformation variation.

###  III-C Deformation reconstruction performance

The aim of the next experiment was to investigate the prediction performance
and characteristics of per-patient- and per-region-based learning models. For
the experimental setup, 100 corresponding vertices were randomly sampled from
each organ model, and a total of 500 vertices were used as multi-organ shape
features, meaning that the dimensionality of the feature vector $x$ was 3,000.
The displacement vectors of the GTV $y$ were calculated using Eq. (1) through
leave-one-out cross-validation. The per-patient- and per-vertex-based learning
models were evaluated using the mean distance, Hausdorff distance, and Dice
similarity coefficient between the estimated and ground truth regions of the
pancreatic cancer. Figure 9 shows box plots of the three error metrics for
different weight parameter values. Significant differences were found for all
the error metrics when using $10^{-5},10^{-4},0.001,0.01$ for the weight
parameters by one-way analysis of variance (ANOVA; p < 0.05 significance
level). The minimal estimation error was $6.7\pm 2.6$ mm for per-patient-based
learning, and $4.8\pm 2.3$ mm for per-vertex-based learning, which shows that
per-vertex-based learning improved the estimation performance by 28.4%.

The characteristics of the multidimensional feature sets depend on the
complexity of the tumor localization problem and the number of data sets.
These are both important factors affecting the estimation performance and the
calculation cost. Therefore, we investigated the relationship between the
estimation error of tumor localization and the number of sampling points $N$.
A mean estimation error was calculated for ten trials of random sampling from
five organs while increasing $N$ from 1 to 400. The weight parameter of the
reproducing kernel was set to $3.0\times 10^{-5}$, which results in good
estimation performance.

Figure 9(d) shows the transition of the prediction performance of the two
models. In the graph, per-region-based learning model shows consistently
better prediction performance regardless of the number of sampling points. The
estimation error decreased as the number of sampling points increased, and the
error tends to converge over around 300 points. In the proposed per-vertex-
based learning model, an average of 4.8 mm estimation error was achieved in
the case of $N=300$, which is a performance that is similar to the previous
setting, which used 500 sampling points.

Figure 10: Multi-organ shape features and prediction performance. LI: liver,
ST: stomach, DU: duodenum, LK: left kidney, and RK: right kidney. Figure 11:
Inter-patient variation of the GTV’s motion with local deformation and
estimation errors. (a) Mean, maximum, and minimum paths over one respiratory
cycle, (b) The estimation error for ten time phases.

###  III-D Multi-organ shape features

The investigation of effective feature sets in respiratory motion analysis is
an important topic not only for reproducing kernels but for deep learning
applications. The next experiments confirm the relationship between 31 feature
sets (all combinations of the five organs) and estimation accuracy in the
prediction of pancreatic cancer deformation. This experiment also compares the
results of the performance of the proposed multidimensional features and that
of other regression approaches with a single organ or low-dimensional
features. The number of sampling points was fixed to 300 based on the results
of the previous experiments.

Figure 10 shows the median of the Hausdorff distance sorted in ascending order
for 31 feature sets. These results suggest the following:

  * •

Features from smaller organ sets show better estimation performance than ones
sampled from all five organs.

  * •

Estimations using only one organ tend to increase estimation error.
Specifically, the right kidney leads to poor estimation performance.

  * •

The stomach, duodenum, and left kidney are the best motion descriptors for
estimating the deformable region of pancreatic cancer.

These findings suggest the validity of using the features of multiple
neighboring organs rather than features sampled from the entire abdominal
area. Shape features from the liver perform worse despite the fact that this
organ is relatively close to the pancreas, but they are good candidates for
feature descriptors when the contours of the stomach and duodenum are not
available.

###  III-E Estimation performance on motion dynamics

The goal of the motion/deformation analysis in this paper was to investigate
the estimation performance of pancreatic cancer using multi-organ shape
features. We analyzed the estimation error of GTV for ten time phases of the
4D-CT dataset, as shown in Fig. 11. The analysis of the multi-organ feature
sets show that, the stomach, duodenum, and left kidney were used for this
prediction. The number of sampling points and parameters for the reproducing
kernels were the same as in the previous experiments. Figure 11(a) shows the
inter-patient variation of the GTV’s motion with local deformation. The mean,
maximum, and minimum path in one respiratory cycle are plotted as three lines,
and each colored bandwidth is the standard deviation for 25-patient data.
Because the displacement is calculated per vertex for the pancreas model, the
difference between the maximum and minimum paths represents a local
deformation exceeding 5 mm on average. The overall path also includes
considerable variations with over 10 mm differences in individual respiratory
motion. This can also be confirmed in the full bandwidth of Fig. 7.

The transition of the estimation error for ten time phases using the designed
reproducing kernel is plotted in Fig. 11(b). Despite the inter-patient
variation of motion and deformation observed in Fig. 11(a), the mean $\pm$
standard deviation of the distance was 1.2 $\pm$ 0.7 mm and that of the
Hausdorff distance was 4.2 $\pm$ 2.3 mm. Both errors remained small throughout
the respiration. These results suggest that the GTV can be adequately
localized by the shape features of the surrounding organs, specifically the
stomach, duodenum, and left kidney, even if the pancreas is not directly
detected.

##  IV Discussion

To our knowledge, this study is the first to build an SMDM, a statistical
multi-organ deformation library of five abdominal organs that includes inter-
and intra-patient shape variations. Image-based deformable registration
[15][23] is a popular approach for deformed bodies, however, matching multi-
organ regions tends to result in large registration errors, especially around
the organ boundaries. The potential of DMR has recently been rediscovered
[1][2][6], and the proposed LDSM for each organ address problems with matching
organs with rotational components and sliding boundaries. Moreover, it
achieved stable registration with a Hausdorff distance error of less than 1
mm. We note that this result outperforms the registration error of 3.1–3.3 mm
reported for the SSM of CTV of the cervix–uterus and bladder in recent work
[1].

To clarify the focus of this research, the spatial displacement of the
internal structures of the mesh was considered to be outside the scope of this
paper. This is because the assumed application of SMDM is radiotherapy
planning, in which the 3D contour (i.e., surface) of the GTV and OARs must be
estimated. Because the developed library uses a tetrahedral mesh in the DMR
and can express internal deformation, we consider that further modeling and
evaluation of internal structures would be possible in future work.

In this experiment, the kernel-based modeling approach was used as a nonlinear
deformation regression for a limited number of 4D-CT data. Despite this
study’s limited data size, the results showed that stable estimation could be
achieved throughout the respiration cycle. These findings and the developed
database will be useful for determining the planning target volume of
pancreatic cancer from the available features of the surrounding organs. Some
recent studies report pixel-to-shape techniques, i.e., 3D shape/deformation
reconstruction from a single 2D image [38][39][40]. Combining such 3D shape
reconstruction techniques, SMDM, and extracted multi-organ feature sets, would
be an interesting approach for real-time tumor localization from X-ray
images.Our future work includes application of the developed multi-organ
library to markerless radiation for tumor-tracking radiotherapy.

##  V Conclusion

In this paper, we introduced a multi-organ deformation library and its
application to pancreatic cancer localization based on the shape features of
multiple organs. The statistical multi-organ motion/deformation library of the
stomach, liver, left and right kidneys, and duodenum was generated by the DMRs
of their organ meshes generated from 4D-CT images (250 volumes). The proposed
LDSM method achieved stable registration with a Hausdorff distance error of
less than 1 mm. Per-region-based deformation learning using a reproducing
kernel was also proposed to predict the displacement of pancreatic cancer for
ART. The experiment results show that the proposed concept better estimates,
and achieves a clinically acceptable estimation error for mean distance (1.2
$\pm$ 0.7 mm) and the Hausdorff distance (4.2 $\pm$ 2.3 mm) throughout the
respiratory motion.

## References

  * [1] B. Rigaud, A. Simon, M. Gobeli, J. Leseur, L. Duverge, D. Williaume et al., “Statistical shape model to generate a planning library for cervical adaptive radiotherapy,” _IEEE Trans. Med. Imag._ , vol. 38, no. 2, pp. 406–416, 2019.
  * [2] A. Magallon-Baro, M. Loi, M. T. W. Milder, P. V. Granton, A. G. Zolnay, J. J. Nuyttens et al., “Modeling daily changes in organ-at-risk anatomy in a cohort of pancreatic cancer patients,” _Radiother. and Oncol._ , vol. 134, pp. 127–134, 2019.
  * [3] M. Posiewnik and T. Piotrowski, “A review of cone-beam CT applications for adaptive radiotherapy of prostate cancer,” _Phys. Med._ , vol. 59, pp. 13–21, 2019.
  * [4] C. A. Hvid, U. V. Elstrøm, K. Jensen, and C. Grau, “Cone-beam computed tomography (CBCT) for adaptive image guided head and neck radiation therapy,” _Acta Oncol._ , vol. 57, no. 4, pp. 552–556, 2018.
  * [5] T. Heimann and H. P. Meinzer, “Statistical shape models for 3D medical image segmentation: A review,” _Med. Image. Anal._ , vol. 13, no. 4, pp. 543–563, 2009.
  * [6] M. Nakamura, M. Nakao, H. Hirashima, H. Iramina, and T. Mizowaki, “Performance evaluation of a newly developed three-dimensional model-based global-to-local registration in prostate cancer,” _J. Radiat. Res._ , vol. 60, no. 5, pp. 595–602, 2019.
  * [7] D. Tilly, A. J. Schoot, E. Grusell, A. Bel, and A. Ahnesjö, “Dose coverage calculation using a statistical shape model applied to cervical cancer radiotherapy,” _Phys. Med. Biol._ , vol. 62, no. 10, pp. 4140–4159, 2017.
  * [8] M. Haekal, H. Arimura, T. A. Hirose, Y. Shibayama, S. Ohga, J. Fukunaga et al., “Computational analysis of interfractional anisotropic shape variations of the rectum in prostate cancer radiation therapy,” _Phys. Med. Biol._ , vol. 46, pp. 168–179, 2018.
  * [9] L. Bondar, M. Intven, J. P. Burbach, E. Budiarto, J. P. Kleijnen, M. Philippens et al., “Statistical modeling of CTV motion and deformation for IMRT of early-stage rectal cancer,” _Int. J. Radiat. Oncol. Biol. Phys._ , vol. 90, no. 3, pp. 664–672, 2014.
  * [10] J. Ehrhardt, R. Werner, A. Schmidt-Richberg, and H. Handels, “Statistical modeling of 4D respiratory lung motion using diffeomorphic image registration,” _IEEE Trans. Med. Imag._ , vol. 30, no. 2, pp. 251–265, 2011\.
  * [11] C. Jud, A. Giger, R. Sandkühler, and P. C. Cattin, “A localized statistical motion model as a reproducing kernel for non-rigid image registration,” _Medical Image Computing and Computer-Assisted Intervention (MICCAI)_ , pp. 261–269, 2017.
  * [12] B. Fuerst, T. Mansi, F. Carnis, M. Sälzle, J. Zhang, J. Declerck et al., “Patient-specific biomechanical model for the prediction of lung motion from 4D CT images,” _IEEE Trans. Med. Imag._ , vol. 34, no. 2, pp. 599–607, 2015.
  * [13] M. Nakao and K. Minato, “Physics-based interactive volume manipulation for sharing surgical process,” _IEEE Transactions on Information Technology in Biomedicine_ , vol. 14, no. 3, pp. 809–816, 2010.
  * [14] M. Nakao, A. Kawashima, M. Kokubo, and K. Minato, “Simulating lung tumor motion for dynamic tumor-tracking irradiation,” _Proc. IEEE NSS-MIC_ , vol. 6, pp. 4549–4551, 2007\.
  * [15] A. Sotiras, C. Davatzikos, and N. Paragios, “Deformable medical image registration: A survey,” _IEEE Trans. Med. Imag._ , vol. 32, no. 7, pp. 1153–1190, 2013.
  * [16] J. Rühaak, T. Polzin, S. Heldmann, I. J. A. Simpson, H. Handels, J. Modersitzki et al., “Estimation of large motion in lung CT by integrating regularized keypoint correspondences into dense deformable registration,” _IEEE Trans. Med. Imag._ , vol. 36, no. 8, pp. 1746–1757, 2017.
  * [17] Y. Shibayama, H. Arimura, T. A. Hirose, T. Nakamoto, T. Sasaki, S. Ohga et al., “Investigation of interfractional shape variations based on statistical point distribution model for prostate cancer radiation therapy,” _Med. Phys._ , vol. 44, no. 5, pp. 1837–1845, 2017.
  * [18] M. Nakao, J. Tokuno, T. Chen-Yoshikawa, H. Date, and T. Matsuda, “Surface deformation analysis of collapsed lungs using model-based shape matching,” _Int. J. Comput. Assist. Radiol. Surg._ , vol. 14, no. 10, pp. 1763–1774, 2019.
  * [19] M. Wilms, I. Y. Ha, H. Handels, and M. P. Heinrich, “Model-based regularisation for respiratory motion estimation with sparse features in image-guided interventions,” _Medical Image Computing and Computer-Assisted Intervention (MICCAI)_ , pp. 89–97, 2016.
  * [20] A. Yu, S. M. Woo, J. Joo, H. R. Yang, W. J. Lee, S. J. Park et al., “Development and validation of a prediction model to estimate individual risk of pancreatic cancer,” _PLoS ONE_ , vol. 11, no. 1, e0146473, 2016\.
  * [21] G. Fontana, M. Riboldi, C. Gianoli, C. I. Chirvase, G. Villa, C. Paganelli et al., “MRI quantification of pancreas motion as a function of patient setup for particle therapy - a preliminary study,” _J. Appl. Clin. Med. Phys._ , vol. 17, no. 5, pp. 60–75, 2016.
  * [22] D. Zhang and K. Chen, “A novel diffeomorphic model for image registration and its algorithm,” _J. Math. Imaging Vision_ , vol. 60, no. 8, pp. 1261–1283, 2018.
  * [23] S. Oh and S. Kim, “Deformable image registration in radiation therapy,” _Radiation oncology journal_ , vol. 35, no. 2, pp. 101–111, 2017.
  * [24] S. Klein, M. Staring, K. Murphy, M. A. Viergever, and J. P. Pluim, “Elastix: A toolbox for intensity-based medical image registration,” _IEEE Trans. Med. Imag._ , vol. 29, no. 1, pp. 196–205, 2010.
  * [25] K. Robert, J. P. Anthony, S. Reith, T. Josephine, E. F. Katherine, P. Neil et al., “Correlation of liver and pancreas tumor motion with normal anatomical structures determined with deformable image registration,” _Biomed. Phys. Eng. Exp._ , vol. 3, no. 1, e017001, 2017.
  * [26] R. Jadon, C. A. Pembroke, C. L. Hanna, N. Palaniappan, M. Evans, A. E. Cleves et al., “A systematic review of organ motion and image-guided strategies in external beam radiotherapy for cervical cancer,” _Clin. Oncol._ , vol. 26, no. 4, pp. 185–96, 2014.
  * [27] T. Iwai, M. Nakao, M. Nakamura, and T. Matsuda, “A tracking method of invisible tumors using surrounding features on time series X-ray images,” _Int. J. Comp. Assist. Radiol. Surg._ , vol. 12, no. 1, S206–207, 2017\.
  * [28] H. Teske, P. Mercea, M. Schwarz, N. H. Nicolay, F. Sterzing, and R. Bendl, “Real-time markerless lung tumor tracking in fluoroscopic video: Handling overlapping of projected structures,” _Med. Phys._ , vol. 42, no. 5, pp. 2540–2549, 2015.
  * [29] G. Whitfield, P. Jain, M. Green, G. Watkins, A. Henry, J. Stratford et al., “Quantifying motion for pancreatic radiotherapy margin calculation,” _Radiother. Oncol._ , vol. 103, no. 3, pp. 360–366, 2012.
  * [30] X. Xu, F. Zhou, B. Liu, D. Fu, and X. Bai, “Efficient multiple organ localization in CT image using 3D region proposal network,” _IEEE Trans. Med. Imag., published online_ , 2019.
  * [31] J. Kim, C. Valdes-Hernandez Mdel, N. A. Royle, and J. Park, “Hippocampal shape modeling based on a progressive template surface deformation and its verification,” _IEEE Trans. Med. Imag._ , vol. 34, no. 6, pp. 1242–1261, 2015\.
  * [32] M. F. Beg, M. I. Miller, A. Trouvé, and L. Younes, “Computing large deformation metric mappings via geodesic flows of diffeomorphisms,” _Int. J. Comput. Vision_ , vol. 61, no. 2, pp. 139–157, 2005.
  * [33] A. Nealen, T. Igarashi, O. Sorkine, and M. Alexa, “Laplacian mesh optimization,” _Proc. of 4th Int. Conf. on Computer Graphics and Interactive Techniques_ , pp. 381–389, 2006.
  * [34] D. P. Huttenlocher, G. A. Klanderman, and W. A. Rucklidge, “Comparing images using the Hausdorff distance,” _IEEE Trans. Pattern Anal. Mach. Intell._ , vol. 15, no. 9, pp. 850–863, 1993.
  * [35] A. Pitiot, E. Bardinet, P. M. Thompson, and G. Malandain, “Piecewise affine registration of biological images for volume reconstruction,” _Med. Image. Anal._ , vol. 10, no. 3, pp. 465–483, 2006.
  * [36] J. Zhou, S. Kim, S. Jabbour, S. Goyal, B. Haffty, T. Chen et al., “A 3D global-to-local deformable mesh model based registration and anatomy-constrained segmentation method for image guided prostate radiotherapy,” _Med. Phys._ , vol. 37, no. 3, pp. 1298–1308, 2010\.
  * [37] A. Saito, M. Nakao, Y. Uranishi, and T. Matsuda, “Deformation estimation of elastic bodies using multiple silhouette images for endoscopic image augmentation,” _IEEE Int. Symp. on Mixed and Augmented Reality_ , pp. 170–171, 2015.
  * [38] Y. Wang, Z. Zhong, and J. Hua, “DeepOrganNet: On-the-fly reconstruction and visualization of 3D/4D lung models from single-view projections by deep deformation network,” _arXiv preprint_ , 2019.
  * [39] S. Wu, M. Nakao, J. Tokuno, T. Chen-Yoshikawa, and T. Matsuda, “Reconstructing 3d lung shape from a single 2D image during the deaeration deformation process using model-based data augmentation,” _IEEE Int. Conf. on Biomed. Health Info.(BHI)_ , pp. 1–4, 2019.
  * [40] M. Nakao, A. Saito, and T. Matsuda, “A simulation study on deformation estimation of elastic materials using monocular images,” _Int. J. Comput. Assist. Radiol. Surg._ , vol. 12, no. 1, S257–258, 2017.


# CLASSIFICATION OF 5-DIMENSIONAL MD-ALGEBRAS HAVING NON-COMMUTATIVE DERIVED
IDEALS

Le Anh ${\bf Vu}^{*}$, Ha Van ${\bf Hieu}^{**}$ and Tran Thi Hieu ${\bf
Nghia}^{***}$
∗Department of Mathematics and Economic Statistics, University of Economics
and Law
Vietnam National University - Ho Chi Minh City, Viet nam
E-mail<EMAIL_ADDRESS>
∗∗ E-mail<EMAIL_ADDRESS>
∗∗∗ E-mail<EMAIL_ADDRESS>
###### Abstract

The paper presents a subclass of the class of MD5-algebras and MD5-groups,
i.e. five dimensional solvable Lie algebras and Lie groups such that their
orbits in the co-adjoint representation (K-orbits) are orbits of zero or
maximal dimension. The main result of the paper is the classification up to an
isomorphism of all MD5-algebras having non-commutative derived ideals.

AMS Mathematics Subject Classification: Primary 22E45, Secondary 46E25, 20C20.

Key words: Lie group, Lie algebra, MD5-group, MD5-algebra, K-orbits.

### INTRODUCTION

In 1962, studying theory of representations, A. A. Kirillov introduced the
Orbit Method (see [2]). This method quickly became the most important method
in the theory of representations of Lie groups. Using the Kirillov’s Orbit
Method, we can obtain all the unitary irreducible representations of solvable
and simply connected Lie Groups. The importance of Kirillov’s Orbit Method is
the co-adjoint representation (K-representation). Therefore, it is meaningful
to study the K-representation in the theory of representations of Lie groups.

After studying the Kirillov’s Orbit Method, Do Ngoc Diep in 1980 suggested to
consider the class of Lie groups and Lie Algebras MD such that the
$C^{*}-algebras$ of them can be described by using KK-functors (see [1]). Let
G be an n-dimensional real Lie group. G is called an MDn-group if and only if
its orbits in the K-representation (i.e. K-orbits) are orbits of dimension
zero or maximal dimension. The corresponding Lie algebra of G is called an
MDn-algebra. Thus, classification and studying of K-representation of the
class of MDn-groups and MDn-algebras is the problem of interest. Because all
Lie algebras of n dimension (with $n\leq 3$) were listed easily, we have to
consider MDn-groups and MDn-algebras with $n\geq 4$. In 1990, all MD4-algebras
were classified up to an isomorphism by Vu - the first author (see [5]).
Recently, Vu and some his colleagues have continued studying MD5-groups and
MD5-algebras having commutative derived ideals (see [6], [7], [8]). In 2008, a
classification of all MD5-algebras having commutative derived ideals was given
by Vu and Kar Ping Shum (see [9]).

In this paper, we shall give the classification up to an isomorphism of all
MD5-algebras $\mathcal{G}$ whose derived ideals
${\mathcal{G}}^{1}:=[\mathcal{G},\mathcal{G}]$ are non-commutative. This
classification is the main result of the paper.

The paper is organized as follows: The first section deals with some
preliminary notions, section 2 is devoted to the discussion of some results on
MDn-algebras, in particular, the main result of the paper is given in this
section.

##  1 PRELIMINARIES

We first recall in this section some preliminary results and notations which
will be used later. For details we refer the reader to the book [2] of A. A.
Kirillov and the book [1] of Do Ngoc Diep.

###  1.1 The K-representation and K-orbits

Let G be a Lie group, $\cal G$ =Lie(G) be the corresponding Lie algebra of G
and ${\cal G}^{*}$ be the dual space of $\cal G$. For every $g\in G$, we
denote the internal automorphism associated with g by $A_{(g)}$, and whence,
$A_{(g)}:G\to G$ can be defined as follows $A_{(g)}:=g.x.g^{-1},\forall x\in
G$. This automorphism induces the following map
${A_{\left(g\right)}}_{*}:{\cal G}\to{\cal G}$ which is defined as follows

${A_{\left(g\right)}}_{*}\left(X\right):=\frac{d}{{dt}}\left[{g.\exp\left({tX}\right).g^{-1}}\right]\left|{{}_{t=0}}\right.$,
$\forall X\in{\cal G}.$

This map is called the tangent map of $A_{(g)}$. We now formulate the
definitions of K-representation and K-orbit.

######  Definition 1.1.1.

The action

$K:G\longrightarrow Aut(\mathcal{G}^{*})$ $g\longmapsto K_{(g)}$

such that

$\left\langle{K_{\left(g\right)}(F),X}\right\rangle:=\left\langle{F,{A_{\left(g^{-1}\right)}}_{*}\left(X\right)}\right\rangle,\forall
F\in{\cal G}^{*},\,\forall X\in{\cal G}$

is called the co-adjoint representation or K-representation of G in
$\mathcal{{G}^{*}}$.

######  Definition 1.1.2.

Each orbit of the co-adjoint representation of G is called a K-orbit of G.

We denote the K-orbit containing F by $\Omega_{F}$. Thus, for every $F\in{\cal
G}^{*}$, we have $\Omega_{F}:=\left\\{{K\left(g\right)(F)|g\in G}\right\\}$.
The dimension of every K-orbit of an arbitrary Lie group G is always even. In
order to define the dimension of the K-orbits $\Omega_{F}$ for each F from the
dual space ${\cal G}^{*}$ of the Lie algebra $\cal G$ = Lie(G), it is useful
to consider the following skew-symmetric bilinear form $B_{F}$ on $\cal G$:
$B_{F}\left({X,Y}\right)=\left\langle{F,\left[{X,Y}\right]}\right\rangle,\forall
X,Y\in{\cal G}$. We denote the stabilizer of F under the co-adjoint
representation of G in ${\cal G}^{*}$ by $G_{F}$ and ${\cal G}_{F}:=$
Lie($G_{F}$).

We shall need in the sequel of the following result.

######  Proposition 1.1.3 (see [2, Section 15.1]).

$KerB_{F}={\cal{G}}_{F}$ and
$dim{\Omega}_{F}=dim{\mathcal{G}}-dim{\mathcal{G}}_{F}=rankB_{F}.$ $\square$

###  1.2 MD-groups and MD-algebras

######  Definition 1.2.1 (see [1, Chapter 2]).

An MD-group is a real solvable Lie group of finite dimension such that its
K-orbits are orbits of dimension zero or maximal dimension (i.e. dimension k,
where k is some even constant and no more than the dimension of the considered
group). When the dimension of considered group is n (n is a some positive
integer), the group is called an MDn-group. The Lie algebra of an MD-group
(MDn-group, respectively) is called an MD-algebra (MDn-algebra, respectively).

The following proposition gives a necessary condition for a Lie algebra
belonging to the class of MD-algebras.

######  Proposition 1.2.2 (see [3, Theorem 4]).

Let $\mathcal{G}$ be an MD-algebra. Then its second derived ideal
${\mathcal{G}}^{2}:=[[\mathcal{G},\mathcal{G}],[\mathcal{G},\mathcal{G}]]$ is
commutative. $\square$

We point out here that the converse of the above result is in general not
true. In other words, the above necessary condition is not a sufficient
condition. So, we now only consider the real solvable Lie algebras having
commutative second derived ideals. Thus, they could be MD-algebras.

######  Proposition 1.2.3 (see [1, Chapter 2, Proposition 2.1]).

Let $\cal{G}$ be an MD-algebra with F (in ${\cal{G}}^{*}$) is not vanishing
perfectly in ${\cal{G}}^{1}:=[\mathcal{G},\mathcal{G}]$, i.e. there exists
$U\in{\cal{G}}^{1}$ such that $\langle F,U\rangle\neq 0.$ Then the K-orbit
${\Omega}_{F}$ is one of the K-orbits having maximal dimension. $\square$

##  2 THE CLASS OF MD5-ALGEBRAS
HAVING NON-COMMUTATIVE
DERIVED IDEALS

###  2.1 Some Results on the Class of MD-algebras

In this subsection, we shall present some results on general MDn-algebras
($n\geq 4$).

Firstly, we consider a real solvable Lie algebra $\cal{G}$ of dimension $n$
such that $dim{\cal G}^{1}=n-k$ ($k$ is some integer constant, $1\leq k<n$),
${\cal G}^{2}$ is non - trivial commutative and $dim{\cal G}^{2}=dim{\cal
G}^{1}-1=n-k-1$. Without loss of generality, we may assume that

${\cal G}\,\,\,=gen\left(X_{1},X_{2},...,X_{n}\right),\,(n\geq 4)$,

${{\cal G}^{1}}=gen\left(X_{k+1},X_{k+2},...,X_{n}\right),\,(n>k\geq 1)$,

${{\cal G}^{2}}=gen\left(X_{k+2},...,X_{n}\right)$,

with the Lie brackets are given by

$\left[X_{i},X_{j}\right]=\sum\limits_{l=k+1}^{n}{C_{ij}^{l}{X_{l}}},\,1\leq
i<j\leq n,$

where $C_{ij}^{l}\left({1\leq i<j\leq n},k+1\leq l\leq n\right)$ are
constructional constants of $\cal{G}$.

######  Theorem 2.1.1.

There is no MD-algebra $\cal G$ such that its second derived ideal ${\cal
G}^{2}$ is not trivial and less than its first derived ideal ${\cal G}^{1}$ by
one dimension: $dim{\cal G}^{2}=dim{\cal G}^{1}-1.$

In order to prove this theorem, we need some lemmas.

######  Lemma 2.1.2.

The operator $ad_{X_{k+1}}$ restricted on ${\cal G}^{2}$ is an automorphism.

###### Proof.

Since ${\cal G}^{2}$ is commutative,
$\left[{{X_{i}},{X_{j}}}\right]=0,\,\,\forall i,j\geq k+2$. Hence,

$\begin{array}[]{l}\quad\,gen\left({X_{k+2}},{X_{k+3}},\cdots,{X_{n}}\right)={{\cal
G}^{2}}=\left[{{{\cal G}^{1}},{{\cal G}^{1}}}\right]\\\
=gen\left(\left[X_{k+1},X_{k+2}\right],\left[X_{k+1},X_{k+3}\right],\cdots,\left[X_{k+1},X_{n}\right]\right)\\\
=gen\left(ad_{X_{k+1}}\left(X_{k+2}\right),\cdots,ad_{X_{k+1}}\left(X_{n}\right)\right).\end{array}$

It follows that $ad_{X_{k+1}}$ restricted on ${\cal G}^{2}$ is automorphic. ∎

######  Lemma 2.1.3.

Without any restriction of generality, we can always suppose right from the
start that $\left[X_{i},X_{k+1}\right]=0$ for all indices $i$ such that $1\leq
i\leq k$.

###### Proof.

Firstly, we remark that $\left[X_{1},X_{k+1}\right]\in{\cal G}^{1}$, so there
exists $X\in{\cal G}^{2}$ such that
$\left[X_{1},X_{k+1}\right]=C_{1,k+1}^{k+1}X_{k+1}+X$. Since ${ad_{X_{k+1}}}$
restricted on ${\cal G}^{2}$ is automorphic, there exists $Y\in{\cal G}^{2}$
such that $ad_{X_{k+1}}\left(Y\right)=X$.

By changing $X^{\prime}_{1}=X_{1}+Y$, we get
$\left[X^{\prime}_{1},X_{k+1}\right]=C_{1,k+1}^{k+1}{X_{k+1}}$. Using the
Jacobi identity for $X^{\prime}_{1},X_{k+1}$, and an arbitrary element
$Z\in{\cal G}^{2}$ we obtain
$ad_{X_{1}}ad_{X_{k+1}}-ad_{X_{k+1}}ad_{X_{1}}=\alpha ad_{X_{k+1}}$, where
$\alpha$ is some real constant. Since $ad_{X_{k+1}}$ is automorphic on ${\cal
G}^{2}$, $\alpha$ must be zero. Therefore, $C_{1,k+1}^{k+1}=0$, i.e.
$\left[X^{\prime}_{1},X_{k+1}\right]=0$. So, we can suppose that
$\left[X_{1},X_{k+1}\right]=0$.

By the same way, we can suppose
$\left[X_{2},X_{k+1}\right]=\cdots=\left[X_{k},X_{k+1}\right]=0.$ ∎

######  Lemma 2.1.4.

$\left[X_{i},X_{j}\right]=C_{ij}^{k+1}X_{k+1}$ for all pairs of indices i, j
such that $1\leq i<j\leq k$.

###### Proof.

Consider an arbitrary pair of indices $i,j$ such that $1\leq i<j\leq k$. Note
that
$\left[X_{i},X_{j}\right]=\sum\limits_{l=k+1}^{n}{C_{ij}^{l}{X_{l}}}=C_{ij}^{k+1}X_{k+1}+\sum\limits_{l=k+2}^{n}{C_{ij}^{l}{X_{l}}}$.
By using the Jacobi identity, we have

$\left[X_{i},\left[X_{j},X_{k+1}\right]\right]+\left[X_{j},\left[X_{k+1},X_{i}\right]\right]+\left[X_{k+1},\left[X_{i},X_{j}\right]\right]=0$

$\Longrightarrow\left[X_{k+1},\left[X_{i},X_{j}\right]\right]=\left[X_{k+1},\,C_{ij}^{k+1}X_{k+1}+\sum\limits_{l=k+2}^{n}{C_{ij}^{l}{X_{l}}}\right]=0$

$\Longrightarrow
ad_{X_{k+1}}{\left(\sum\limits_{l=k+2}^{n}{C_{ij}^{l}{X_{l}}}\right)}=0$

$\Longrightarrow\sum\limits_{l=k+2}^{n}{C_{ij}^{l}{X_{l}}}=0$ (because
$ad_{X_{k+1}}$ is automorphic on ${\cal{G}}^{2}$)

$\Longrightarrow\left[{{X_{i}},{X_{j}}}\right]=C_{ij}^{k+1}{X_{k+1}};1\leq
i<j\leq k$.

∎

We now prove Theorem 2.1.1. Namely, we will prove that if $\cal G$ is a real
solvable Lie algebra such that ${\cal G}^{2}$ is non - trivial commutative and
$dim{\cal G}^{2}=dim{\cal G}^{1}-1=n-k-1,\,1\leq k<n$, then $\cal G$ is not an
MD-algebra.

######  Proof of Theorem 2.1.1.

According to above lemmas, we can choose a suitable basis
$\left(X_{1},X_{2},\cdots,X_{n}\right)$ of $\cal G$ which satisfies the
following conditions:

$\left[X_{i},X_{j}\right]=C_{ij}^{k+1}X_{k+1},\,\,1\leq i<j\leq k$;

$\left[X_{i},X_{k+1}\right]=0,\,\,1\leq i\leq k$;

$\left[X_{i},X_{j}\right]=\sum\limits_{l=k+2}^{n}{C_{ij}^{l}{X_{l}}},\,\,1\leq
i\leq k+1,\,k+2\leq j\leq n$.

Moreover, the constructional constants $C_{ij}^{k+1}$ can not concomitantly
vanish and the matrix $A={\left(C_{j,k+1}^{l}\right)}_{k+2\leq j,l\leq n}$ is
invertible because $ad_{X_{k+1}}$ restricted on ${\cal{G}}^{2}$ is
automorphic.

Since A is invertible, there exist
$\alpha_{k+2},\cdots,\alpha_{n}\in{\mathbb{R}}$, which are not concomitantly
vanished, such that

$A\left[\begin{array}[]{l}\alpha_{k+2}\\\ \,\,\,\,\vdots\\\
\alpha_{n}\end{array}\right]=\left[\begin{array}[]{l}1\\\ 0\\\ \vdots\\\
0\end{array}\right]\in{\mathbb{R}}^{n-k-1}.$

Let $\left(X_{1}^{*},X_{2}^{*},\cdots,X_{n}^{*}\right)$ is the dual basis in
${\cal G}^{*}$ of $\left(X_{1},X_{2},\cdots,X_{n}\right)$. We choose
${F_{1}}=X_{k+1}^{*}$ and
${F_{2}}=X_{k+1}^{*}+\alpha_{k+2}X_{k+2}^{*}+\cdots+{\alpha_{n}}X_{n}^{*}$ in
${\cal G}^{*}$ . It can easily be seen that $F_{1},F_{2}$ are not perfectly
vanishing in ${\mathcal{G}}^{1}$. In the view of Proposition 1.2.3, if $\cal
G$ is an MD-algebra then $\Omega_{F_{1}},\Omega_{F_{2}}$ are orbits of maximal
dimension, in particular we have

$rankB_{F_{2}}=dim\Omega_{F_{1}}=dim\Omega_{F_{2}}=rankB_{F_{2}}.$

But it is easy to verify that $rankB_{F_{2}}\geq rankB_{F_{1}}+2$. This
contradiction proves that $\cal G$ is not an MD-algebra and the proof of
Theorem 2.1.1 is therefore complete. $\square$

Now we consider an arbitrary real solvable Lie algebra $\cal{G}$ of dimension
$n$ ($n\geq 5$) such that $dim{\cal G}^{1}=n-1$. It is obvious that we can
choose one basis $\left(X_{1},X_{2},\cdots,X_{n}\right)$ of $\cal G$ such that
${\cal G}^{1}=gen\left(X_{2},X_{3},\cdots,X_{n}\right)$, $\,{\cal
G}^{2}\subset gen\left(X_{3},\cdots,X_{n}\right)$ and ${\cal G}^{2}$ is
commutative. Let $C_{ij}^{l}\left({1\leq i<j\leq n},2\leq l\leq n\right)$ are
constructional constants of $\cal{G}$. Then the Lie brackets are given by the
following formulas

$\left[{{X_{i}},{X_{j}}}\right]=\sum\limits_{l=2}^{n}{C_{ij}^{l}{X_{l}}}\left({1\leq
i<j\leq n}\right).$

######  Theorem 2.1.5.

Let ${\cal G}$ be a real solvable Lie algebra of dimension $n$ such that its
first derived ideal ${\cal G}^{1}$ is $(n-1)$-dimensional ($n\geq 5$) and its
second derived ideal ${\cal G}^{2}$ is commutative. Then ${\cal G}$ is MDn-
algebra if and only if $\,{\cal G}^{1}$ is commutative.

In order to prove this theorem, once again, we also need some lemmas.

######  Lemma 2.1.6.

If $\,\cal G$ is an MD-algebra of dimension $n\,$ ($n\geq 5$) such that
$\,\dim{\cal G}^{1}=n-1$ then $\dim{\Omega_{F}}\in\\{0,2\\}\,$ for every
$F\in{\cal G}^{*}$.

###### Proof.

Let $ad_{X_{1}}=\left(a_{ij}\right)_{n-1}\in End\left({\cal G}^{1}\right)$.
With $F_{0}=X_{2}^{*}\in{\cal G}^{*}$, the matrix of the bilinear form
$B_{F_{0}}$ in the chosen basis as follows

$B_{F_{0}}=\left[\begin{array}[]{l}0\,\,\,\,\,\,\,-{a_{12}}\,\,\,\,\,\,-{a_{13}}\,\,\,....\,\,-{a_{1n}}\\\
{a_{12}}\,\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,....\,\,\,\,\,\,\,\,0\\\
{a_{13}}\,\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,....\,\,\,\,\,\,\,\,0\\\
....\,\,\,\,\,\,\,\,\,\,\,....\,\,\,\,\,\,\,\,\,\,\,....\,\,\,\,\,\,\,\,....\,\,\,\,\,\,....\\\
{a_{1n}}\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,....\,\,\,\,\,\,\,\,0\end{array}\right].$

It is plain that $rank\,B_{F_{0}}=2$. Since ${\cal G}$ is an MD-algebra, we
get $\dim{\Omega_{F}}\in\\{0,2\\}\,$ for every $F\in{\cal G}^{*}$. ∎

######  Lemma 2.1.7.

Suppose that ${\cal G}=gen\left(X_{1},X_{2},\cdots,X_{n}\right)$ is a real
solvable Lie algebra of dimension n such that

${\cal G}^{1}=gen\left(X_{2},X_{3},\cdots,X_{n}\right)$ and $\,{\cal
G}^{2}=gen\left(X_{k+1},\cdots,X_{n}\right)$, $k>1$.

Let

$A=\left({\begin{array}[]{*{20}{c}}{C_{12}^{2}}&\ldots&{C_{1k}^{2}}\\\
\vdots&\ddots&\vdots\\\ {C_{12}^{k}}&\cdots&{C_{1k}^{k}}\end{array}}\right)$

be the matrix established by the constructional constants ${C_{1j}^{l}}$
$\left(2\leq j,l\leq k\right)$ of $\cal G$. Then A is invertible.

###### Proof.

Since ${\cal G}^{1}=\left[\cal G,\cal G\right]$, there exist real numbers
$\alpha_{ij},\,1\leq i<j\leq n$, such that

${X_{2}}=\sum\limits_{1\leq i<j\leq
n}{\alpha_{ij}}\left[{{X_{i}},{X_{j}}}\right]$

$=\sum\limits_{j=2}^{k}{{\alpha_{1j}}\left[{{X_{1}},{X_{j}}}\right]}+\sum\limits_{j=k+1}^{n}{{\alpha_{1j}}\left[{{X_{1}},{X_{j}}}\right]}+\sum\limits_{2\leq
i<j\leq n}{{\alpha_{{ij}}}\left[{{X_{i}},{X_{j}}}\right]}$

$=\sum\limits_{j=2}^{k}{{\alpha_{1j}}\left[{{X_{1}},{X_{j}}}\right]+LC_{1}\left({{{\cal
G}^{2}}}\right)}$

$=\sum\limits_{j=2}^{k}{{\alpha_{1j}}{\left(\sum\limits_{l=2}^{n}{C_{1j}^{l}{X_{l}}}\right)}+LC_{1}\left({\cal
G}^{2}\right)}$

$=\sum\limits_{j=2}^{k}{{\alpha_{1j}}{\left(\sum\limits_{l=2}^{k}{C_{1j}^{l}{X_{l}}}+\sum\limits_{l=k+1}^{n}{C_{1j}^{l}{X_{l}}}\right)}+LC_{1}\left({\cal
G}^{2}\right)}$

$=\sum\limits_{l=2}^{k}{\sum\limits_{j=2}^{k}{C_{1j}^{l}{\alpha_{1j}}X_{l}}+LC_{2}\left({\cal
G}^{2}\right)},$

where $LC_{1}\left({\cal G}^{2}\right),\,LC_{2}\left({\cal G}^{2}\right)$ are
linear combinations of some definite vectors from the chosen basis of ${\cal
G}^{2}.$ This implies that there exists columnar vector
${Y_{2}}\in{{\mathbb{R}}^{k-1}}$ such that

$AY_{2}=\left[\begin{array}[]{l}1\\\ 0\\\ \vdots\\\
0\end{array}\right]\in{\mathbb{R}}^{k-1}.$

Similarly, there exist columnar vectors
$Y_{3},\cdots,{Y_{k}}\in{{\mathbb{R}}^{k-1}}$ such that

$AY_{3}=\left[\begin{array}[]{l}0\\\ 1\\\ 0\\\ \vdots\\\
0\end{array}\right],\,\cdots,\,AY_{k}=\left[\begin{array}[]{l}0\\\ 0\\\
\vdots\\\ 0\\\ 1\end{array}\right]\in{\mathbb{R}}^{k-1}.$

Thus, there is a real matrix $P$ such that $A.P=I$, where $I$ is the identity
$(k-1)$-matrix. So $A$ is invertible and Lemma 2.1.7 is proved completely. ∎

### Proof of Theorem 2.1.5

Firstly, we shall prove that, if ${\cal
G}=gen\left(X_{1},X_{2},\cdots,X_{n}\right)\,\left(n\geq 5\right)$ such that
${\cal G}^{1}=gen\left(X_{2},X_{3},\cdots,X_{n}\right)$ is non-commutative,
then ${\cal G}$ is not an MD-algebra. Let ${{\cal
G}^{2}}=gen\left(X_{k+1},\cdots,X_{n}\right),\,2\leq k<n$.

We need consider some cases which contradict each other as follows.

  * 1.

$k=2$. Then, $\dim{\cal G}^{2}=\dim{\cal G}^{1}-1$. According to Theorem
2.1.1, G is not an MD-algebra.

  * 2.

$k=3$. That means that ${\cal G}^{2}=gen\left(X_{4},\cdots,X_{n}\right)$.
Assume that $\cal G$ is an MD-algebra. Remember that

$\left[X_{1},X_{2}\right]=\sum\limits_{l=2}^{n}{C_{12}^{l}{X_{l}}},\,\left[X_{1},X_{3}\right]=\sum\limits_{l=2}^{n}{C_{13}^{l}{X_{l}}},\,\left[X_{i},X_{j}\right]=\sum\limits_{l=4}^{n}{C_{ij}^{l}{X_{l}}}\scriptsize{\footnotesize\textbullet},$

for all $j>4$ when $i=1$, $j\geq 3$ when $i=2$ and $j>3$ when $i=3$. According
to Lemma 2.1.7, the matrix
$P=\left[{\begin{array}[]{*{20}{c}}{C_{12}^{2}}&{C_{12}^{3}}\\\
{C_{13}^{2}}&{C_{13}^{3}}\end{array}}\right]$ is invertible.
Let
$F={\alpha_{1}}X_{1}^{*}+{\alpha_{2}}X_{2}^{*}+\cdots+{\alpha_{n}}X_{n}^{*}$
be an arbitrary element of ${\cal G}^{*}$, where
$\alpha_{1},\alpha_{2},\cdots,\alpha_{n}\in\mathbb{R}$. The matrix of the
bilinear form ${B_{F}}$ is

${B_{F}}=\left[{\begin{array}[]{*{20}{c}}0&{-F\left({\left[{{X_{1}},{X_{2}}}\right]}\right)}&\cdots&{-F\left({\left[{{X_{1}},{X_{n}}}\right]}\right)}\\\
{F\left({\left[{{X_{1}},{X_{2}}}\right]}\right)}&0&\cdots&{-F\left({\left[{{X_{2}},{X_{n}}}\right]}\right)}\\\
\cdots&\cdots&\cdots&\cdots\\\
{F\left({\left[{{X_{1}},{X_{n}}}\right]}\right)}&{F\left({\left[{{X_{2}},{X_{n}}}\right]}\right)}&\cdots&0\end{array}}\right].$

Now we consider the $4$-submatrices of ${B_{F}}$ established by the elements
which are on the rows and the columns of the same numbers 1, 2, 3, i ($i>3$).
Because $\cal G$ is an MD-algebra, so according to Lemma 2.1.6, we get
$rank(B_{F})\in\\{0,2\\}$, this implies that the determinants of these
$4$-submatrices are zero for any $F\in{{\cal G}^{*}}$. By direct computations,
using the following obvious result of Linear Algebra: the determinant of any
skew-symmetric real $4$-matrix $\left(a_{ij}\right)_{4}$ is equal to zero if
and only if ${a_{12}}.{a_{34}}-{a_{13}}.{a_{24}}+{a_{14}}.{a_{23}}=0$, we get
$C_{2i}^{l}=C_{3i}^{l}=0,l\geq 4$. This implies
$\left[X_{2},X_{i}\right]=\left[X_{3},X_{i}\right]=0,i\geq 4$. Note that
${\cal G}^{2}$ is commutative. So we have

${\cal G}^{2}=\left[{\cal G}^{1},{\cal
G}^{1}\right]=gen\left(X_{4},\cdots,X_{n}\right)$

$=gen\left(\left[X_{i},X_{j}\right];i,j\geq 2\right)$

$=gen\left(\left[X_{2},X_{3}\right]\right)$.

Thus, $n-3=\dim{\cal G}^{2}\leq 1$, i.e. $n\leq 4$. This contradicts the
hypothesis $n\geq 5$. That means $\cal G$ is not an MD-algebra.

  * 3.

$k\geq 4$. By an argument analogous to that used above, we also prove that
$\cal G$ is not an MD-algebra.

Conversely, assume that ${\cal G}$ is a real solvable Lie algebra of dimension
$n$ such that its first derived ideal is $(n-1)$-dimensional and commutative,
i.e. ${{\cal
G}^{1}}\equiv{\mathbb{R}}.{X_{2}}\oplus{\mathbb{R}}.{X_{3}}\oplus...\oplus{\mathbb{R}}.{X_{n}}\equiv{{\mathbb{R}}^{n-1}}$.
We need show that $\cal G$ is an MD-algebra.

Let
$F={\alpha_{1}}X_{1}^{*}+{\alpha_{2}}X_{2}^{*}+\cdots+{\alpha_{n}}X_{n}^{*}\equiv\left(\alpha_{1},\alpha_{2},\cdots,\alpha_{n}\right)\in{\mathbb{R}}^{n}$
be an arbitrary element from ${\cal G}^{*}\equiv{\mathbb{R}}^{n}$, where
$\alpha_{1},\alpha_{2},\cdots,\alpha_{n}\in{\mathbb{R}}$. By simple
computation, we can see that the matrix of the bilinear form ${B_{F}}$ is

${B_{F}}=\left[{\begin{array}[]{*{20}{c}}0&{-F\left({\left[{{X_{1}},{X_{2}}}\right]}\right)}&\cdots&{-F\left({\left[{{X_{1}},{X_{n}}}\right]}\right)}\\\
{F\left({\left[{{X_{1}},{X_{2}}}\right]}\right)}&0&\cdots&0\\\
\cdots&\cdots&\cdots&\cdots\\\
{F\left({\left[{{X_{1}},{X_{n}}}\right]}\right)}&0&\cdots&0\end{array}}\right].$

It is clear that $rankB_{F}\in\left\\{{0,2}\right\\}$. Hence, ${\cal G}$ is an
MDn-algebra and Theorem 2.1.5 is proved completely. $\square$

###  2.2 Classification of MD5-algebras having
non-commutative derived ideals

The following theorem is the main result of the paper. It gives the
classification up to an isomorphism of MD5-algebras having non-commutative
derived ideals.

######  Theorem 2.2.1.

Let ${\cal G}$ be an MD5-algebra such that the first derived ideal ${\cal
G}^{1}=\left[{\cal G},{\cal G}\right]$ is non-commutative. Then the following
assertions hold.

  * (i)

If ${\cal G}$ is decomposable, then ${\cal G}\cong{\cal H}\oplus{\cal K}$,
where ${\cal H}$ and ${\cal K}$ are MD-algebras of dimensions which are no
more than 4.

  * (ii)

If ${\cal G}$ is indecomposable, then we can choose a suitable basis
$(X_{1},X_{2},X_{3},$
$X_{4},X_{5})$ of ${\cal G}$ such that ${\cal
G}^{1}=gen\left(X_{3},X_{4},X_{5}\right),\,\left[X_{3},X_{4}\right]=X_{5}$;
operators $ad_{X_{1}},ad_{X_{2}}$ act on ${\cal G}^{1}$ as the following
endomorphisms

$ad_{X_{1}}=\left({\begin{array}[]{*{20}{c}}1&0&0\\\ 0&1&0\\\
0&0&2\end{array}}\right),\,\,ad_{X_{2}}=\left({\begin{array}[]{*{20}{c}}0&-1&0\\\
1&0&0\\\ 0&0&0\end{array}}\right)$

and the other Lie brackets are trivial.

We need to prove some lemmas before we prove Theorem 2.2.1.

######  Lemma 2.2.2.

Let ${\cal G}$ be a real solvable Lie algebra. For any $Z\in{\cal G}$ we
consider $ad_{Z}$ as an operator acted on ${\cal G}^{1}$. Then we have
$Trace\left(ad_{Z}\right)=0$ for all $Z\in{\cal G}^{1}$.

###### Proof.

Using the Jacobi identity for $X,Y\in{\cal G}$ and an arbitrary element
$Z\in{{\cal G}^{1}}$, we have
$\left[{X,\left[{Y,Z}\right]}\right]+\left[{Y,\left[{Z,X}\right]}\right]+\left[{Z,\left[{X,Y}\right]}\right]=0$.
So, $a{d_{X}}\circ a{d_{Y}}-a{d_{Y}}\circ a{d_{X}}=a{d_{\left[{X,Y}\right]}}$.
This implies $Trace\left({a{d_{\left[{X,Y}\right]}}}\right)=0$. Note that
${{\cal G}^{1}}=\left[{{\cal G},{\cal G}}\right]$ and $a{d_{Z}}$ is a linear
map. So we get $Trace\left({a{d_{Z}}}\right)=0$ for all $Z\in{{\cal G}^{1}}$.
∎

######  Lemma 2.2.3.

If ${\cal G}$ is a real solvable Lie algebra with $\dim{\cal G}^{1}=2$ then
${{\cal G}^{1}}$ is commutative.

###### Proof.

We choose a basis $(X,Y)$ of ${{\cal G}^{1}}$. Assume that
$\left[X,Y\right]=aX+bY$. So we have
$ad_{X}=\left({\begin{array}[]{*{20}{c}}0&a\\\
0&b\end{array}}\right),ad_{Y}=\left({\begin{array}[]{*{20}{c}}{-a}&0\\\
{-b}&0\end{array}}\right)\in End({\cal G}^{1})$. According to Lemma 2.2.2, we
get $a=b=0$. Hence, ${{\cal G}^{1}}$ is commutative. ∎

Now we are ready to prove Theorem 2.2.1 - The main result of the paper.

######  Proof of Theorem 2.2.1.

It is clear that assertion (i) of Theorem 2.2.1 holds obviously. We only need
to prove assertion (ii). Let ${\cal G}$ be an indecomposable MD5-algebra with
the first derived ideal ${\cal G}^{1}=\left[{\cal G},{\cal G}\right]$ is non -
commutative and the second derived ideal ${\cal G}^{2}=\left[{\cal
G}^{1},{\cal G}^{1}\right]$ is commutative. According to Theorems 2.1.1, 2.1.5
and Lemma 2.2.3, the dimensions of ${\cal G}^{1}$ and ${\cal
G}^{2}=\left[{\cal G}^{1},{\cal G}^{1}\right]$ must be 3 and 1, respectively.
We choose a basis $\left(X_{1},X_{2},X_{3},X_{4},X_{5}\right)$ such that
${\cal G}^{1}=gen\left(X_{3},X_{4},X_{5}\right)$ and ${\cal
G}^{2}=gen\left(X_{5}\right)$ with the Lie brackets are given by

$\begin{array}[]{l}\left[X_{1},X_{2}\right]={a_{3}}{X_{3}}+{a_{4}}{X_{4}}+{a_{5}}{X_{5}},\\\
\left[X_{1},X_{3}\right]={b_{3}}{X_{3}}+{b_{4}}{X_{4}}+{b_{5}}{X_{5}},\\\
\left[X_{1},X_{4}\right]={c_{3}}{X_{3}}+{c_{4}}{X_{4}}+{c_{5}}{X_{5}},\\\
\left[X_{1},X_{5}\right]={d_{3}}{X_{3}}+{d_{4}}{X_{4}}+{d_{5}}{X_{5}},\\\
\left[X_{2},X_{3}\right]={e_{3}}{X_{3}}+{e_{4}}{X_{4}}+{e_{5}}{X_{5}},\\\
\left[X_{2},X_{4}\right]={f_{3}}{X_{3}}+{f_{4}}{X_{4}}+{f_{5}}{X_{5}},\\\
\left[X_{2},X_{5}\right]={k_{3}}{X_{3}}+{k_{4}}{X_{4}}+{k_{5}}{X_{5}},\\\
\left[X_{3},X_{4}\right]={g_{5}}{X_{5}},\left[X_{3},X_{5}\right]={h_{5}}{X_{5}},\left[X_{4},X_{5}\right]={l_{5}}{X_{5}},\end{array}$

where $a_{i},b_{i},c_{i},d_{i},e_{i},f_{i},k_{i}\,(i=3,4,5)$ and
$g_{5},h_{5},l_{5}$ are the definite real numbers.

Now we give some useful remarks as follows.

  * a.

According to Lemma 2.2.2,
$Trace\left(ad_{X_{3}}\right)=Trace\left(ad_{X_{4}}\right)=0$. That means
${h_{5}}={l_{5}}=0$.

  * b.

$g_{5}\neq 0$ because ${\cal G}^{2}=gen(X_{5})$. By changing ${X_{3}}$ with
${X^{\prime}_{3}}={\frac{1}{g_{5}}}X_{3}$, we get
$\left[X^{\prime}_{3},X_{4}\right]=X_{5}$. So, we can suppose right from the
start that $\left[X_{3},X_{4}\right]=X_{5}$, i.e. $g_{5}=1$.

  * c.

${d_{3}}={d_{4}}={k_{3}}={k_{4}}=0$ because ${\cal G}^{2}=\mathbb{R}.X_{5}$ is
an ideal of ${\cal G}$. So, we get

$\left[X_{1},X_{5}\right]={d_{5}}{X_{5}},\,\left[X_{2},X_{5}\right]={k_{5}}{X_{5}}.$

If $k_{5}\neq 0$, by changing
$X^{\prime}_{2}=X_{1}-{\frac{d_{5}}{k_{5}}}X_{2}$ we get
$\left[X^{\prime}_{2},X_{5}\right]=0$. So, we can always assume that
$k_{5}=0$.

  * d.

By changing ${X_{1}}$ with
${X^{\prime}_{1}}={X_{1}}-{c_{5}}{X_{3}}+{b_{5}}{X_{4}}$ and ${X_{2}}$ with
${X^{\prime}_{2}}={X_{2}}-{f_{5}}{X_{3}}+{e_{5}}{X_{4}}$, we get

$\left[X^{\prime}_{1},X_{3}\right]={b_{3}}{X_{3}}+{b_{4}}{X_{4}}$,
$\left[X^{\prime}_{1},X_{4}\right]={c_{3}}{X_{3}}+{c_{4}}{X_{4}}$,

$\left[X^{\prime}_{2},X_{3}\right]={e_{3}}{X_{3}}+{e_{4}}{X_{4}}$,
$\left[X^{\prime}_{2},X_{4}\right]={f_{3}}{X_{3}}+{f_{4}}{X_{4}}$.

Thus, we can suppose right from the start that
${b_{5}}={c_{5}}={e_{5}}={f_{5}}=0$.

Using the Jacobi identity for triads $X_{1},X_{2},X_{i}\,(i=3,4,5)$, we obtain

$(I)\left\\{\begin{array}[]{l}{a_{3}}={a_{4}}=0,\\\
{e_{4}}{c_{3}}={b_{4}}{f_{3}},\\\
{e_{3}}{b_{4}}+{e_{4}}{c_{4}}={b_{3}}{e_{4}}+{b_{4}}{f_{4}},\\\
{f_{3}}{b_{3}}+{f_{4}}{c_{3}}={c_{3}}{e_{3}}+{c_{4}}{f_{3}},\\\
{b_{3}}+{c_{4}}={d_{5}},\\\ {e_{3}}+{f_{4}}=0.\end{array}\right.$

So we can reduce the Lie brackets as follows

$\begin{array}[]{l}\left[{{X_{1}};{X_{2}}}\right]=\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,{a_{5}}{X_{5}},\\\
\left[{{X_{1}};{X_{3}}}\right]={b_{3}}{X_{3}}+{b_{4}}{X_{4}},\\\
\left[{{X_{1}};{X_{4}}}\right]={c_{3}}{X_{3}}+{c_{4}}{X_{4}},\\\
\left[{{X_{1}};{X_{5}}}\right]=\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\left({{b_{3}}+{c_{4}}}\right){X_{5}},\\\
\left[{{X_{2}};{X_{3}}}\right]={e_{3}}{X_{3}}+\,{e_{4}}{X_{4}},\\\
\left[{{X_{2}};{X_{4}}}\right]={f_{3}}{X_{3}}-{e_{3}}{X_{4}},\\\
\left[{{X_{3}};{X_{4}}}\right]=\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,{X_{5}}.\end{array}$

Thus, Relations $(I)$ can be rewritten as follows

$(II)\left\\{\begin{array}[]{l}{e_{4}}{c_{3}}={b_{4}}{f_{3}},\\\
2{e_{3}}{b_{4}}={e_{4}}\left({{b_{3}}-{c_{4}}}\right),\\\
2{c_{3}}{e_{3}}={f_{3}}\left({{b_{3}}-{c_{4}}}\right).\end{array}\right.$

Now we need consider the following cases which contradict each other.

  * Case 1:

${e_{3}}={e_{4}}=0.$

$(II)\Leftrightarrow\left\\{\begin{array}[]{l}{b_{4}}{f_{3}}=0,\\\
{f_{3}}\left({{b_{3}}-{c_{4}}}\right)=0.\end{array}\right.$

  * 1.1.

Assume that ${f_{3}}=0$. Then, Relations $(II)$ is automatically satisfied.

  * By choosing $F_{1}=X_{3}^{*}\in{\cal G}^{*}$, we get $rank{B_{{F_{1}}}}=2$.

  * Now we choose $F_{2}=X_{5}^{*}\in{\cal G}^{*}$. By simple computation, we obtain

${B_{{F_{2}}}}=\left[{\begin{array}[]{*{20}{c}}0&{-{a_{5}}}&0&0\\\
{{a_{5}}}&0&0&0\\\ 0&0&0&{-1}\\\ {\begin{array}[]{*{20}{c}}0\\\
{{b_{3}}+{c_{4}}}\end{array}}&{\begin{array}[]{*{20}{c}}0\\\
0\end{array}}&{\begin{array}[]{*{20}{c}}1\\\
0\end{array}}&{\begin{array}[]{*{20}{c}}0\\\
0\end{array}}\end{array}\,\,\,\,\,\,\,\begin{array}[]{*{20}{c}}{-\left({{b_{3}}+{c_{4}}}\right)}\\\
0\\\ 0\\\ {\begin{array}[]{*{20}{c}}0\\\ 0\end{array}}\end{array}}\right]$

Since ${\cal G}$ is an MD-algebra, this implies that
$rank{B_{{F_{2}}}}=rank{B_{{F_{1}}}}=2$. That fact implies that
${a_{5}}={b_{3}}+{c_{4}}=0$, so ${\cal G}$ is decomposable, which is a
contradiction. Thus, this case cannot happen.

  * 1.2.

Now we assume that ${f_{3}}\neq 0$. Then $b_{4}=0,b_{3}=c_{4}$.
By changing $X_{1}$ and $X_{2}$ with
${X^{\prime}_{1}}={X_{1}}-{c_{3}}{X^{\prime}_{2}}$ and
$X^{\prime}_{2}={\frac{1}{f_{3}}X_{2}}$ we can suppose right from the start
that $f_{3}=1,c_{3}=0$. Because of the dimension of ${\cal G}^{1}$ is 3,
$b_{3}\neq 0$. By changing $X_{1}$ with
$X^{\prime}_{1}={\frac{1}{b_{3}}}{X_{1}}$, we can always assume that
$b_{3}=1$. By changing $X_{2}$ with
$X^{\prime}_{2}=X_{2}-\frac{a_{5}}{2}{X_{5}}$, we can assume that $a_{5}=0$.
Now, we choose $F_{3}=X_{3}^{*}$ and $F_{4}=X_{4}^{*}$ from ${\cal G}^{*}$, we
get $rank{B_{{F_{3}}}}=2,\,rank{B_{{F_{4}}}}=4$. This cannot happen because
${\cal G}$ is an MD-algebra.

  * Case 2:

${e_{4}}=0,{e_{3}}\neq 0$

$\left({II}\right)\Leftrightarrow\left\\{\begin{array}[]{l}{b_{4}}{f_{3}}=0,\\\
{b_{4}}{e_{3}}=0,\\\
2{c_{3}}{e_{3}}={f_{3}}\left({{b_{3}}-{c_{4}}}\right).\end{array}\right.\Leftrightarrow\left\\{\begin{array}[]{l}{b_{4}}=0,\\\
2{c_{3}}{e_{3}}={f_{3}}\left({{b_{3}}-{c_{4}}}\right).\end{array}\right.$

By changing $X_{2}$ with $X^{\prime}_{2}={\frac{1}{e_{3}}}{X_{2}}$, we can
assume that $e_{3}=1$. Now Relations $(II)$ can be rewritten as follows

$(III)\left\\{\begin{array}[]{l}{b_{4}}=0,\\\
2{c_{3}}={f_{3}}\left({{b_{3}}-{c_{4}}}\right).\end{array}\right.$

By changing $X_{4}$ with $X^{\prime}_{4}=X_{4}-\frac{f_{3}}{2}{X_{3}}$ and
$X_{1}$ with $X^{\prime}_{1}=X_{1}-{b_{3}}{X_{2}}$, we can suppose that
$b_{3}=f_{3}=0$. From Relations (III) we get $b_{4}=c_{3}=0$. Let
$F=\alpha\,X_{1}^{*}+\beta X_{2}^{*}+\gamma X_{3}^{*}+\delta X_{4}^{*}+\sigma
X_{5}^{*}\equiv\left({\alpha,\beta,\gamma,\delta,\sigma}\right)$ be an
arbitrary element from ${{\cal G}^{*}}\equiv{\mathbb{R}^{5}}$;
$\alpha,\beta,\gamma,\delta,\sigma\in\mathbb{R}$. By simple computation, we
obtain the matrix of the bilinear form ${B_{F}}$ as follows

${B_{F}}=\left[{\begin{array}[]{*{20}{c}}0&{-{a_{5}}\sigma}&0&{-{c_{4}}\delta}\\\
{{a_{5}}\sigma}&0&{-\gamma}&\delta\\\ 0&\gamma&0&{-\sigma}\\\
{\begin{array}[]{*{20}{c}}{{c_{4}}\delta}\\\
{{c_{4}}\sigma}\end{array}}&{\begin{array}[]{*{20}{c}}\delta\\\
0\end{array}}&{\begin{array}[]{*{20}{c}}\sigma\\\
0\end{array}}&{\begin{array}[]{*{20}{c}}0\\\
0\end{array}}\end{array}\,\,\,\,\,\,\,\begin{array}[]{*{20}{c}}{-{c_{4}}\sigma}\\\
0\\\ 0\\\ {\begin{array}[]{*{20}{c}}0\\\ 0\end{array}}\end{array}}\right]$

Since ${\cal G}$ is an MD-algebra, $rank{B_{F}}$ only get two values zero or
two. Hence, it is easy to prove that $c_{4}=a_{5}=0$. But this implies that
${\cal G}$ is decomposable. This is a contradiction. Thus, this case cannot
happen.

  * Case 3:

${e_{4}}\neq 0$.

By changing $X_{4}$ with $X^{\prime}_{4}={e_{4}}{X_{4}}+{e_{3}}{X_{3}}$, we
can assume that $e_{3}=0,e_{4}=1$. Now, Relations $(II$) can be rewritten as
follows

$\left\\{\begin{array}[]{l}{c_{3}}={b_{4}}{f_{3}},\\\
{b_{3}}={c_{4}}.\end{array}\right.$

By changing $X_{1}$ with $X^{\prime}_{1}=X_{1}-{b_{4}}{X_{2}}$, we can assume
that $b_{4}=0$. Putting this into the relation above, we get $c_{3}=0$.

Then, the Lie brackets in $\cal G$ can be reduced as follows

$\begin{array}[]{l}\left[X_{1},X_{2}\right]=\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\lambda{X_{5}},\\\
\left[X_{1},X_{3}\right]=\mu{X_{3}},\\\
\left[X_{1},X_{4}\right]=\,\,\,\,\,\,\,\,\,\mu{X_{4}},\\\
\left[X_{1},X_{5}\right]=\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,2\mu{X_{5}},\\\
\left[X_{2},X_{3}\right]=\,\,\,\,\,\,\,\,\,\,\,\,\,{X_{4}},\\\
\left[X_{2},X_{4}\right]=\,\theta{X_{3}},\\\
\left[X_{3},X_{4}\right]=\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,{X_{5}}.\end{array}$

Let $F=\alpha\,X_{1}^{*}+\beta X_{2}^{*}+\gamma X_{3}^{*}+\delta
X_{4}^{*}+\sigma X_{5}^{*}\in{\cal G}^{*}$ be an arbitrary element from ${\cal
G}^{*}\equiv{\mathbb{R}^{5}}$. Then by simple computation, we see that

${B_{F}}=\left[\begin{array}[]{l}0\,\,\,\,\,-\lambda\sigma\,\,\,\,-\mu\gamma\,\,\,\,-\mu\delta\,\,\,\,-2\mu\sigma\\\
\lambda\sigma\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,-\delta\,\,\,\,\,\,-\theta\gamma\,\,\,\,\,\,\,\,\,\,\,\,0\\\
\mu\gamma\,\,\,\,\,\,\,\,\delta\,\,\,\,\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,-\sigma\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,0\\\
\mu\delta\,\,\,\,\,\,\,\,\theta\gamma\,\,\,\,\,\,\,\,\,\,\,\,\sigma\,\,\,\,\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,0\\\
2\mu\sigma\,\,\,\,0\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,0\end{array}\right].$

  * 3.1.

Assume $\mu\neq 0$.

  * When $\sigma\neq 0$, we have $rank{B_{F}}=4$. Because ${\cal G}$ is an MD5-algebra, we get $rank{B_{F}}\in\\{0,4\\}$ for all $\alpha,\beta,\delta,\gamma,\sigma\in\mathbb{R}$. But this only can happen if $\theta<0$. By changing $X_{1},X_{2},X_{4},X_{5}$ with $X^{\prime}_{1}=\mu{X_{1}},X^{\prime}_{2}=\sqrt{-\theta}{X_{2}},X^{\prime}_{4}=\sqrt{-\theta}{X_{4}},X^{\prime}_{5}=\sqrt{-\theta}{X_{5}}$, we can assume that $\theta=-1$. By changing $X_{2}$ with $X^{\prime}_{2}=X_{2}-\frac{\lambda}{2}{X_{5}}$, we can assume that $\lambda=0$. Hence, the Lie brackets of $\cal G$ can be reduced as follows

$\begin{array}[]{l}\left[{{X_{1}};{X_{3}}}\right]={X_{3}},\\\
\left[{{X_{1}};{X_{4}}}\right]=\,\,\,\,\,\,\,\,\,{X_{4}},\\\
\left[{{X_{1}};{X_{5}}}\right]=\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,2{X_{5}},\\\
\left[{{X_{2}};{X_{3}}}\right]=\,\,\,\,\,\,\,\,\,\,\,\,{X_{4}},\\\
\left[{{X_{2}};{X_{4}}}\right]=\,-{X_{3}},\\\
\left[{{X_{3}};{X_{4}}}\right]=\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,{X_{5}}.\end{array}$

  * 3.2

Assume $\mu=0$. By the same way, we consider the matrix of the bilinear form
${B_{F}}$ and obtain $\lambda=0$. But this shows that ${\cal G}$ is
decomposable. This construction show that this case can not happen.

The theorem 2.2.1 is proved completely. So there is only one MD5-algebra
having non-commutative derived ideal.$\hfill\square$

### CONCLUDING REMARK

Let us recall that each real Lie algebra $\cal G$ defines only one connected
and simply connected Lie group G such that Lie(G) = $\cal G$. Therefore we
obtain only one connected and simply connected MD5-group corresponding to the
MD5-algebra given in Theorem 2.2.1. In the next paper, we shall describe the
geometry of K-orbits of considered MD5-group, describe topological properties
of MD5-foliation formed by the generic K-orbits of this MD5-group and give the
characterization of the Connes’s $C^{*}$-algebra associated to this
MD5-foliation.

## References

  * [1] Do Ngoc Diep, Method of Noncommutative Geometry for Group $C^{*}$-algebras, Chapman and Hall / CRC Press Reseach Notes in Mathematics Series, # 416, 1999.
  * [2] A.A. Kirillov, Elements of the Theory of Representations, Springer-Verlag, Berlin – Heidenberg – New York, 1976.
  * [3] Vuong Manh Son et Ho Huu Viet, Sur La Structure Des $C^{*}-alg{\grave{e}}bres$ D’une Classe De Groupes De Lie, J. Operator Theory, 11 (1984), 77-90.
  * [4] Kristopher Tapp, Matrix Groups for Undergraduates, AMS, 2005.
  * [5] Le Anh Vu, On the foliations formed by the generic K-orbits of the MD4-groups, Acta Math. Vietnam, $N^{0}$ 2 (1990), 39 - 45.
  * [6] Le Anh Vu, On a Subclass of 5-dimensional Lie Algebras Which have 3-dimensional Commutative Derived Ideals, East-West J. of Mathematics, Vol.7, $N^{0}$ 1 (2005), 13 - 22.
  * [7] Le Anh Vu, Classification of 5-dimensional MD-algebras Which have 4-dimensional Commutative Derived Ideals, Scientific Journal of University of Pedagogy of Ho Chi Minh City, $N^{0}$ 12 (46) (2007), 3 - 15 (In Vietnamese).
  * [8] Le Anh Vu and Duong Quang Hoa, The Geometricaly Picture of K-orbits of Connected and Simply Connected MD5-groups such that their MD5-algebras have 4-dimensional Commutative Derived Ideals, Scientific Journal of University of Pedagogy of Ho Chi Minh City, $N^{0}$ 12 (46) (2007), 16 - 28 (In Vietnamese).
  * [9] Le Anh Vu and Kar Ping Shum, On a Subclass of 5-dimentional Solvable Lie Algebras Which Have Commutative Derived Ideal, Advances in Algebra and Combinatorics, World Scientific Publishing Co. (2008), pp 353 - 371.