# DREAM: Decentralized Reinforcement Learning for Exploration and Efficient
Energy Management in Multi-Robot Systems

Dipam Patel, Phu Pham, Kshitij Tiwari and Aniket Bera
Department of Computer Science, Purdue University, USA
<EMAIL_ADDRESS>Version:
ideas.cs.purdue.edu/research/dream

###### Abstract

Resource-constrained robots often suffer from energy inefficiencies,
underutilized computational abilities due to inadequate task allocation, and a
lack of robustness in dynamic environments, all of which strongly affect their
performance. This paper introduces DREAM \- Decentralized Reinforcement
Learning for Exploration and Efficient Energy Management in Multi-Robot
Systems, a comprehensive framework that optimizes the allocation of resources
for efficient exploration. It advances beyond conventional heuristic-based
task planning as observed conventionally. The framework incorporates
Operational Range Estimation using Reinforcement Learning to perform
exploration and obstacle avoidance in unfamiliar terrains. DREAM further
introduces an Energy Consumption Model for goal allocation, thereby ensuring
mission completion under constrained resources using a Graph Neural Network.
This approach also ensures that the entire Multi-Robot System can survive for
an extended period of time for further missions compared to the conventional
approach of randomly allocating goals, which compromises one or more agents.
Our approach adapts to prioritizing agents in real-time, showcasing remarkable
resilience against dynamic environments. This robust solution was evaluated in
various simulated environments, demonstrating adaptability and applicability
across diverse scenarios. We observed a substantial improvement of about 25%
over the baseline method, leading the way for future research in resource-
constrained robotics.

##  I Introduction

In the domain of robotics, addressing the unique challenges and opportunities
presented by resource-constrained robots is of paramount significance. Such
robots are expected to employ their computational ability and sensor
capabilities to execute tasks in the most energy-efficient manner. Integral to
these challenges are issues related to energy consumption, path planning, and
coordinated decision-making, especially in multi-robot scenarios. As robotic
systems are increasingly applied in diverse fields, ranging from environmental
monitoring, natural disasters [1], and search & rescue missions [2], solving
these resource constraints has become crucial.

Constructing an energy consumption model that comprehensively factors in the
variables impacting energy use within a robot poses significant challenges.
This complexity arises from the necessity to conduct precise system
identification procedures and accurately quantify each energy expense. The
operational range estimation of a robot refers to the distance within which it
can function given its resource constraints. Energy should be perceived as a
predetermined resource constraint that optimizes the mission execution within
these defined limitations. This model, therefore, needs to be adaptable, and
capable of responding to real-time changes.

Figure 1: Three Robots (in black) navigating to their respective Goal
positions using the Refined TD3 (RTD3) Model. Upon initiation, the GNN model
uses all the Robots’ states to do goal allocation, which accounts for
minimizing the cumulative energy used by the system at the end of the mission.

In the context of co-operative Multi-Robot Systems (MRS), the necessity for
efficient resource management becomes more complex and crucial. Compared to a
single robot, an MRS setup offers improved reliability and scalability. It can
cover large areas and complete tasks quickly and efficiently, making it
suitable for applications in diverse fields. However, this also leads to
challenges associated with goal allocation and trajectory coordination under
constrained resources.

Efficient task allocation in an MRS involves assigning suitable roles to
individual robots based on their specific capabilities and mission
requirements. Trajectory coordination, on the other hand, involves planning
the paths of individual robots in a way that they can collaborate effectively
without interfering with each other’s tasks. This requires careful planning
and real-time adaptability to avoid collisions and ensure smooth cooperation
among the robots with limited resources. This makes decentralized navigation
essential, where each robot is equipped to make independent decisions based on
local information. It can adjust its path dynamically, responding to real-time
changes in the environment.

We propose a Refined Twin Delayed Deep Deterministic Policy Gradient (RTD3)
based model for obstacle avoidance and exploration for Multi-Robot Systems. We
also propose a Graph Neural Network (GNN) based model that leverages real-time
operational data for instantaneous goal allocation. In summary, our model
prioritizes developing robust path-planning mechanisms under strict energy
constraints which drives the development of DREAM. The key contributions of
our work can be summarized as follows:

  * •

Introduced a Refined TD3 structure, leveraging the Reward Categorized Replay
Buffer, resulting in a 75% reduction in model parameters.

  * •

Developed a GNN-based Energy Management Model for adaptive mission planning
based on real-time energy availability, enhancing mission success and system
lifespan.

  * •

Expanded navigation capabilities for multi-agent, goal-driven exploration and
collaborative mapping, using a single agent-goal pair for training.

  * •

Amplified the model’s versatility across varying environments, regardless of
the robot-goal pair count, leveraging the benefits of Curriculum Learning.

##  II Related Works

The increasing complexity and scalability of problems for robotics have
necessitated the deployment of MRS, with agents discovering solutions using
learning mechanisms [3, 4, 5]. Such systems have gained tremendous attention
in recent years, with applications ranging from complex system modeling to
smart grids and computer networks. Nonetheless, MRS presents inherent
challenges, including agent coordination, security, and task allocation [4,
5].

Reinforcement learning techniques have been extensively used in multiagent
systems. For example, [3] provides a comprehensive survey of multi-agent
reinforcement learning. Likewise, [6] and [7] explore deep reinforcement
learning methods for multi-agent domains, with the latter introducing the Mean
Field Reinforcement Learning approach to address the scalability issue.
Interestingly, [7] also establishes a relationship between the learning of an
individual agent’s optimal policy and the dynamics of the population,
highlighting the mutual reinforcement between the two.

In the context of swarm robotics, inspired by natural swarms’ collaborative
intelligence, [8] investigates a fault-tolerant pattern formation algorithm.
The paper sheds light on how different agents can form a geometric pattern and
maintain it, even in the presence of faulty agents. In a related study, [9]
proposes a technique for the identification of biased-measurement agents in a
network of mobile robots, highlighting the impact of errors on system
performance.

Energy efficiency and optimization have also been a significant focus in
multi-agent systems and robotics. For instance, [10] offers an energy-aware
path planning algorithm for UAVs, while [11] examines energy optimization for
optimal motion planning for a dual-arm industrial robot. Further, [12]
describes an energy management system for mobile robots used for search and
rescue that integrates a battery and supercapacitor to manage power sharing.

Effective inter-agent communication is pivotal in decentralized multi-robot
coordination. Traditional methods have employed discrete communication
strategies, such as signal binarization [13]. The infusion of attention
mechanisms into GNNs offers a promising direction to process information-dense
graphs [14]. While attention on static graphs has shown potential, its
efficacy in dynamic multi-agent graphs remains to be fully explored [15].

Deep learning advancements have enabled robots to anticipate their
environments and conduct directed explorations. Notable works include
exploring navigation via deep Q-learning [16], utilizing RGB images to
anticipate maps and actions [17], [18], and using learned predictions to map
surroundings [19].

Our research extends the work of Cimurs et al. [20] for Multi-Robot Systems
under strict resource constraints. Our proposal integrates a streamlined
motion policy based on raw Lidar data with a holistic global navigation
strategy. Our approach not only aims for goal-driven exploration in uncharted
terrains along with obstacle avoidance but also does multi-robot mapping with
limited resources in order not to compromise any agent during a mission.

##  III Methodology

To optimize resource allocation in multi-agent systems, we use the RTD3 model
for multi-agent goal-based exploration and collaborative mapping. We also
implemented the GNN model, which does the goal allocation for each robot. This
section details our implementation and highlights the key improvements and
contributions.

###  III-A Refined TD3 Architecture

####  III-A1 Actor Network

The architecture of our Actor network is meticulously designed to cater to the
complex state-action relationship. The Actor network embodies the policy
function of the agent. Given a state $s$, the Actor predicts the optimal
action $a$ that the agent should execute.

The input to the network consists of 4 neurons along with the lidar input.
These four neurons make up the agent’s state space -
[$d_{goal},\theta_{goal},v,\omega$], which corresponds to the distance to the
goal, angle to goal, linear velocity, and angular velocity respectively at
each time step in the simulation. The first layer consists of 256 neurons,
followed by the subsequent 128-neuron layer (compared to Cimurs et al. [20]
where they have 800 and 600 neurons, respectively). Both the layers thereafter
have layer normalization, ReLU activation, and a dropout of 0.2. Lastly, the
output layer produces 2 action values -
$\left[v\left(\frac{(a+1)}{2}\right),\omega\right]$. The Actor function $\pi$
can be represented as:

$a=\pi(s;\theta_{\pi})$

where $\theta_{\pi}$ denotes the parameters of the Actor network and $s$
denotes the state space. Our model has one-fourth of the number of model
parameters, which was essentially due to incorporating Reward Categorized
Replay Buffer and Curriculum Learning. These concepts will be discussed in
detail in further sections.

####  III-A2 Critic Network

The Critic’s role is pivotal in value-based reinforcement learning. It
estimates the Q-value of a given state-action pair, guiding the Actor’s policy
optimization. Our architecture employs a twin network approach Q1 & Q2 to
alleviate overestimation bias, a common pitfall in Q-learning methods. This is
prevented by taking the $min(Q1,Q2)$.

The Critic network is similar to the Actor-network, except this has two
separate pathways for both the state and action. The action, on the other
hand, is transformed through a 256-neuron layer. The state and action
representations are then concatenated and processed to produce the Q-value:

$Q(s,a;\theta_{Q})=V_{a,s}$

where $Q(s,a;\theta_{Q})$ represents the Q-value function parameterized by
$\theta_{Q}$ and $V_{a,s}$ denotes the expected return when taking action $a$
in state $s$.

###  III-B Reward Categorized Replay Buffer

The quality of stored experiences can profoundly influence an agent’s learning
trajectory. Our approach incorporates a Reward Categorized Replay Buffer
(RCRB) that categorizes experiences based on their reward outcomes. This
improvement in using a replay buffer makes training the Actor-Critic network
faster than the original model. These categories include Positive Buffer
($B_{+}$), Neutral Buffer ($B_{0}$), and Negative Buffer ($B_{-}$). Depending
on the reward value, the experience tuple — comprising of $(s,a,r,s^{\prime})$
is sorted into one of these buffers.

When training the agent, a balanced batch of experiences is required. The
number of samples from each category is determined proportionally based on the
current size of each buffer. This ensures that even rare occasions have a fair
chance of being sampled, promoting a balanced learning experience for the
agent as described below:

$n_{+}=\Bigl{\lfloor}\frac{len(B_{+})}{N}\times
b\Bigl{\rfloor},n_{0}=\Bigl{\lfloor}\frac{len(B_{0})}{N}\times
b\Bigl{\rfloor},n_{-}=b-n_{+}-n_{0}$

where $N$ is the total number of experiences in the buffer, and $b$ is the
batch size for sampling. Figure 2 describes the above-discussed approach.

Figure 2: Representation of Reward Categorized Replay Buffer which accounts
for Positive, Neutral, and Negative buffers based on the input experience
replay

This categorization offers several distinct advantages:

  1. 1.

Balanced Sampling: Traditional buffers dilute sparse yet informative
experiences in a vast sea of frequent outcomes. RCRB ensures a balanced
representation, preventing bias towards commonly occurring rewards.

  2. 2.

Efficient Learning: By providing a representative sample of the environment’s
dynamics, RCRB can expedite the agent’s learning, leading to faster
convergence.

  3. 3.

Optimal Memory Utilization: RCRB’s balanced storage prevents the over-
representation of any specific reward category, ensuring efficient memory use.

###  III-C Training Pipeline - Refined TD3

Central to the DREAM framework is the deployment of the RTD3 algorithm, a
state-of-the-art approach for predicting continuous action spaces. The
intricate design of our model, combined with our custom architectural choices,
forms the backbone of our agent’s learning mechanism.

####  III-C1 Curriculum Learning

We commenced our training regimen with a basic environment setup involving one
robot-goal pair with some obstacles at a fixed position. This enabled the
model to grasp fundamental obstacle avoidance nuances. Eventually, the
complexity of the environment was increased along with different obstacle
poses during each episode to simulate real-world conditions. Figure 3
showcases this curriculum learning approach [21].

Figure 3: From left to right - The complexity of the environment and learning
process increases as the training progresses. Initially, the obstacles are set
at a fixed location. As time progresses, the number of obstacles grows, and
their placement becomes random. A similar trend is observed for the robot’s
odometry, goal, and home positions.

Another aspect involved the goal and home positions spawned in close proximity
to the robot’s initiation point. As the model showcased increased competence,
we expanded the distances to account for the challenge of prolonged journeys.
This extension demanded the model to account for increased energy consumption,
navigation challenges, and optimal path selection over elongated distances.

####  III-C2 Critic Update

The target Q-values for the Critic’s update are computed using a modified
Bellman equation:

$Q_{target}=r+\gamma(1-d)\min_{i=1,2}Q_{i,target}(s^{\prime},\pi(s^{\prime}))$

Here, $r$ represents the reward, $\gamma$ is the discount factor, $d$ is the
termination flag and $s^{\prime}$ is the subsequent state. The innovation lies
in the use of the minimum Q-value from the two target Critic networks, a
strategy that curbs overestimation bias. The primary Critic networks then
predict the Q-values for the original actions. The Mean Squared Error between
these predicted and target Q-values forms the loss, thereby guiding the
Critic’s optimization of parameters.

Algorithm 1 Overview of the Improvised TD3 Approach

Init: Actor, Critic networks; Target networks

Init: Reward Categorized Replay Buffer (RCRB)

Set: Curriculum Level $\leftarrow$ 1

for episode $\leftarrow$ 1 to N do

Adjust environment by Curriculum Level

Init environment; Obtain initial state $s$

while !done do

$a\leftarrow\pi(s;\theta_{\pi})+\text{noise}$

Execute $a$; Get $r$, $s^{\prime}$, done

Store ($s$, $a$, $r$, $s^{\prime}$, done) in RCRB

Sample from RCRB: states, actions, rewards, next_states, dones

$Q_{\text{target}}\leftarrow
r+(1-\text{done})\times\gamma\times\min(Q_{\text{targets}})$

Update Critic with MSE loss using $Q_{\text{target}}$

if iteration mod delay $==0$ then

Update Actor by maximizing Q-values

Soft update target networks

end if

$s\leftarrow s^{\prime}$

end while

Modify battery parameters using actions, state

Save networks at intervals or on improvement

if Performance meets Curriculum Level then

Increment Curriculum Level

end if

end for

####  III-C3 Actor Update

Unlike the Critic’s frequent updates, the Actor undergoes parameter refinement
every two iterations, which is a design choice to ensure stability. The
Actor’s objective is to maximize the expected Q-values from one of the Critic
networks. Given the Critic’s representation of the environment’s value
structure, this maximization ensures the Actor’s policy aligns with high-
reward trajectories. The pseudocode for training the RTD3 approach is provided
in 1.

####  III-C4 Reward Policy

The reward policy for this algorithm is designed according to the following
function:

$r(s_{t},a_{t})=\begin{cases}r_{goal}&\textit{if }D_{goal}<D_{thresh}\\\
r_{collision}&\textit{if }L_{min}<C_{thresh}\rightarrow\textit{collision}\\\
r_{nothing}&\textit{otherwise}\end{cases}$

Here, $r$ is the reward of the state-action pair $(s_{t},a_{t})$ at timestep
$t$. If the distance to the goal $D_{goal}$ was less than the threshold
$D_{thresh}$, $r_{goal}=200$ was awarded. If the distance from the closest
Lidar reading was less than the threshold $C_{thresh}$, it was considered as
collision, and $r_{collision}=-100$ was awarded. In case the robot never
encountered any of these conditions, $v-|\omega|$ was awarded to nudge the
system to move more linearly than in the angular direction and ensure the
agent learned to reach its goal in minimal steps.

###  III-D Obstacle Avoidance & Navigation

Since the robot used Lidar scan as input from the environment, it received
high-dimensional sensory data reflecting the distances to nearby obstacles in
a 180-degree view. This data can be utilized as the state space. However,
since 180 is a high dimension for the model to learn, with most of it being
redundant information, we represent the environment with only 30 values. This
is achieved by dividing the 180-degree view into 30 segments.

The Lidar data provides the state space describing the immediate environment.
Hence, the total state space for this algorithm consists of environment and
robot states, totaling to 34, which becomes the input to the network. The
agent trained in the RTD3 network takes action from a set of possible actions
at every timestep to maximize cumulative reward.

The reward function provides positive feedback for actions that move the robot
closer to its goal since it rewards more for moving linearly while avoiding
obstacles. As the agent iterates through numerous episodes, it learns an
optimal policy that maps states to actions, optimizing the trajectory to the
goal while ensuring obstacle-free navigation. The network, when trained for a
sufficient number of episodes, generalizes well to unseen environments,
allowing the robot to navigate and avoid obstacles in novel scenarios using
only its Lidar.

###  III-E Multi-Robot Decentralized Exploration & Mapping

We previously highlighted that our network was primarily designed to train a
single agent to reach its target goal. This choice to focus on one agent
rather than multiple agents simultaneously, was made to simplify the training
process. This allowed the agent to deeply understand the environment’s
intricacies, including obstacles and the goal’s location. Once the training
was complete, we deployed this model on three robots, each navigating to their
goals. With every episode, the environment, as well as the robot and goal
poses, were randomized. The state space comprised 30 Lidar readings and the
agent’s state space. This setup enabled the robot to navigate around both
static and dynamic obstacles (other robots).

Figure 4: RViz2 showing three robots navigating to their respective goals
along with mapping the environment collaboratively

We utilized slam toolbox [22] ROS2 package [23] for mapping. These tools
enabled collaborative mapping, using laser scans and odometry sensors to
produce an occupancy grid map of the environment. Each robot was set to
publish its parent TF transform as $/robot\\_x/odom$. To facilitate
collaborative mapping, we established additional TF layers, connecting these
individual robots under a single parent. Specifically, $robot\\_x/odom$ was
made a child to $robot\\_x/map$, and all such TFs were linked to the main
$map$ frame. This structure allowed for map visualization in RViz2, as shown
in Figure 4.

Initially, goals were randomly set before exploration began. Robots followed
waypoints, moving towards the goals while mapping their surroundings. Once the
goal was reached, that area was marked as explored, and a new goal was then
set, thereby continuing this cycle for all robots.

###  III-F Battery Modeling

In multi-agent systems, understanding and predicting battery behavior is
critical. With the integration of energy management in the DREAM framework,
simulating a battery’s behavior allows for more informed decisions regarding
energy allocation. In this work, we utilized the Doyle-Fuller-Newman (DFN)
model to simulate and visualize battery discharge over time.

This DFN model provides a comprehensive representation of a lithium-ion
battery by capturing the intricate electrochemical processes occurring within.
Leveraging the PyBaMM library [24], we readily instantiate this model to serve
as the foundation for our battery life simulation. Our target simulation
duration was set to 900 seconds, equivalent to approximately 15 minutes of
battery life for each robot.

The DFN model’s default parameters are updated to reflect our calculated
constant discharge current. This simulated battery was utilized in training
the GNN model since it was one of the important factors for goal allocation.
This module was not included in training the RL model as one of the rewards
addressed this perspective of reaching the target quickly.

###  III-G GNN for Energy Management & Goal Allocation

In the realm of decentralized multi-robot systems, understanding and
predicting the individual goals of robots are foremost. Inspired by the work
of Li et al. [25], our approach leverages Graph Neural Networks (GNN) to model
the relationship between different robots and predict their respective goals.
This facilitates mission completion under constraint resource conditions while
still being decentralized.

####  III-G1 Data Collection & Preprocessing

The framework is designed to simulate scenarios where three robots are
assigned three distinct goals. The overarching objective is to pair each robot
with a goal such that the cumulative energy consumption across all robots is
minimized while maintaining the overall lifespan of the entire MRS.

State space: the state space include robot odometry $(r_{x},r_{y})$, home
$(h_{x},h_{y})$, and goal $(g_{x},g_{y})$ locations.

Energy Management Model: The energy consumed by a robot is contingent on its
linear or angular movement and the distance to the goal. Given $R$ robots and
$G$ goals, we want to assign each robot to a goal. Each robot-goal pair has an
associated energy consumption, which depends on the distance and orientation
difference between the robot’s current state and the goal. For each robot $i$
and goal $j$ pair, the energy $E_{ij}$ is computed based on the distance and
orientation difference:

$E_{ij}=d_{target}\times battery_{straight}+\theta_{target}\times
battery_{turn}$

where, $d_{target}$ is the normalized distance between the robot and the goal,
$\theta_{target}$ accounts for the difference in orientation between the
robot’s current pose and its desired goal pose. $battery_{straight}$ = 10 and
$battery_{turn}=1.25\times battery_{turn}$. The goal here is to find an
assignment matrix $A$ of size $R\times G$ such that it minimizes the overall
energy consumption

$A=\sum_{i=1}^{R}\sum_{j=1}^{G}E_{ij}$

where $E_{ij}$ represents the energy consumed by robot $i$ to reach goal. This
methodology determines the permutation of robot-goal assignments that
optimally reduces energy consumption at both the collective and individual
levels. This is achieved by deploying a graph convolution neural network to
predict the goal and home allocations, one after another.

####  III-G2 Network Architecture

The architecture consists of multiple layers, beginning with a graph
convolutional layer (GCN) followed by a fully connected layer. The network
takes robot states, home locations, and goal positions as inputs. The inputs
are concatenated and processed through two GCN layers. The first GCN layer
takes the initial feature vector and transforms it into a higher-dimensional
representation (128 dimensions) using the GCNConv operation, followed by a
rectified linear unit (ReLU) activation function. Subsequently, the second GCN
layer further refines the representation to a lower-dimensional space (64
dimensions) using the same convolutional operation and activation function. To
prevent overfitting, dropout regularization is applied with a dropout rate of
0.5 during training. Finally, a fully connected layer maps the learned
features to the prediction space, with the output reshaped to size $3\times
2$, corresponding to the indices of goal and home locations for each robot.
This architecture is designed to learn the mapping between the world states
and the optimal goal allocation conditioned on robot battery levels.

####  III-G3 Training Pipeline - GCN

We use the Adam optimizer with a learning rate of 0.005 to update the network
weights. The model parameters are initialized using the Xavier uniform
initialization for the convolutional layers. This initialization method scales
the weights based on the number of input and output units, which can aid in
achieving a faster convergence. We design a special loss function to optimize
energy consumption while maintaining the lifespan of each robot in the system.
To this end, we compute the total energy of each robot on two journeys: from
the current location to the goal and from the goal to home. Specifically, the
network will optimize the total energy consumed by the whole system and also
guarantee that each robot has enough battery to reach home. Any assignment
that results in one or more stranded robots (low battery) during the mission,
will be heavily penalized by our design. Consequently, the network will favor
the optimal solutions for both global and individual levels.

####  III-G4 Model Insights

The utilization of a graph-based paradigm offers notable advantages in terms
of scalability and adaptability, particularly in the context of accommodating
a variable number of robots within a system. In contrast to conventional
methodologies which typically involve random goal assignment or demand
computational complexity, the proposed model demonstrates the ability to
efficiently predict individual goals for each robot based on their collective
states.

##  IV Experiments

###  IV-A System Setup

We used a computer equipped with an NVIDIA GeForce RTX 3060 graphics card,
with 32 GB of RAM, and 12th Gen Intel Core i7 12700K x 20 CPU running Ubuntu
22.04.3 and ROS2 Humble. We trained the Refined TD3 network using PyTorch [26]
in the Gazebo simulator [27] for 7000 iterations.

###  IV-B Training in Simulation

The training pipeline was set up using ROS2, while Gazebo and RViz2 were used
for visualization of laser scan, goal pose, home pose, robot odometry, and
camera. The Pioneer P3-DX robot was used in the simulation. During the testing
phase, three such robots were used with the same suite of sensors.

When the training starts, Gazebo and RViz2 were disabled to accelerate the
training process. One training episode is considered concluded when the robot
reaches a goal, a collision is detected, or 250 steps are taken. As shown in
figure 3, the training was carried out in a simulated 10x10m sized environment
as previously discussed.

We reward the robot for reaching the goal, penalize it for collisions, and
give a minor reward otherwise. This encourages the robot to move, initially
favoring forward motion over rotation. Despite early collisions, moving yields
higher rewards than being stationary. The robot soon learns that avoiding
obstacles, even if it means turning, is better than crashing. Hence, there’s
no need for a battery-saving penalty.

The state space doesn’t consist of a home location since that is just another
goal for the robot. This was confirmed during the evaluation phase as it is
independent of the type of the goal. The model also doesn’t account for
dynamic obstacles, only a randomized environment on every reset. This has made
the model robust enough to avoid dynamic obstacles like other robots. This was
the primary reason not to train the three robots simultaneously. This was also
confirmed in testing as the robots would swerve on encountering them in their
path.

###  IV-C Results

To evaluate the effectiveness of our GNN-based goal allocation strategy versus
a random allocation method, we introduce the Goal Allocation Evaluation Metric
(GAEM). This metric is designed to measure the efficiency of different methods
by comparing the overall energy consumption of robot teams in various
environments.

Metric Definition: Given a multi-robot system (MRS), let $E_{GNN}$ and
$E_{random}$ denote the energy consumption of robot $r$ to reach goal $g$ when
goals are assigned using the GNN and random methods, respectively. The
efficiency metric $\epsilon$ for a particular environment is defined as:

$\epsilon=\frac{\sum_{r=1}^{R}E^{random}_{rg}-\sum_{r=1}^{R}E^{GNN}_{rg}}{\sum_{r=1}^{R}E^{random}_{rg}}$

A higher $\epsilon$ indicates a more significant energy-saving using the GNN
method compared to the random allocation. $\epsilon$ offers a direct
understanding of the percentage reduction in energy consumption using GNN over
random goal allocation.

Env | Avg $E^{random}$ | Avg $E^{GNN}$ |  $\epsilon$ (% Reduction)
---|---|---|---
1 | 124.67 | 93.4 | 25.08
2 | 136.01 | 102.56 | 24.59
3 | 157.32 | 120.98 | 23.09
4 | 163.71 | 123.16 | 24.76
5 | 122.83 | 96.67 | 21.29
TABLE I: Comparison of energy consumption for five different environments run
five times each using random and GNN goal allocation methods.

From Table I, it is evident that this can be extended for various environments
and a number of agents to showcase the strength and consistency of the GNN
approach. The reduction in energy consumption not only improves the
sustainability of the robot team but also enhances the efficiency and
reliability of the operations on an individual level.

##  V Conclusion and Future Works

In this paper, we introduced DREAM \- a comprehensive framework that optimizes
the allocation of vital resources in multi-agent systems. We successfully
demonstrated the Refined TD3 model for obstacle avoidance, goal-based
exploration and collaborative mapping for a multi-agent setup. This network
had one-fourth number of model parameters due to the reward-categorized replay
buffer and curriculum learning which created a significant impact on the rapid
convergence of the model. We also presented a unique GNN model for goal
allocation with resource constraints, since it is more optimized and scalable
than a conventional approach.

Even though our model was trained in simulation, we believe this learned
policy should be easily transferrable with minimal parameter changes on any
type of robot with a Lidar sensor. Further, this model can be fine-tuned using
pretraining to account for more velocity inputs for certain robots.

In this work, we have not addressed certain scenarios where robot(s) become
incapable during a mission due to hardware or software issues. To alleviate
this, one or more of the remaining agents would take over the task of the
robot(s) which got compromised in order not to sacrifice the entire mission.
This will be addressed in future work.

## References

  * [1] C. Mouradian, J. Sahoo, R. H. Glitho, M. J. Morrow, and P. A. Polakos, “A coalition formation algorithm for multi-robot task allocation in large-scale natural disasters,” in _2017 13th International Wireless Communications and Mobile Computing Conference (IWCMC)_ , 2017, pp. 1909–1914.
  * [2] D. Drew, “Multi-agent systems for search and rescue applications,” _Current Robotics Reports_ , vol. 2, 03 2021.
  * [3] L. Busoniu, R. Babuska, and B. De Schutter, “A comprehensive survey of multiagent reinforcement learning,” _IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews)_ , vol. 38, no. 2, pp. 156–172, 2008.
  * [4] A. Dorri, S. S. Kanhere, and R. Jurdak, “Multi-agent systems: A survey,” _IEEE Access_ , vol. 6, pp. 28 573–28 593, 2018.
  * [5] S. S. Binyamin and S. Ben Slama, “Multi-agent systems for resource allocation and scheduling in a smart grid,” _Sensors_ , vol. 22, no. 21, 2022. [Online]. Available: https://www.mdpi.com/1424-8220/22/21/8099
  * [6] R. Lowe, Y. Wu, A. Tamar, J. Harb, P. Abbeel, and I. Mordatch, “Multi-agent actor-critic for mixed cooperative-competitive environments,” _CoRR_ , vol. abs/1706.02275, 2017. [Online]. Available: http://arxiv.org/abs/1706.02275
  * [7] Y. Yang, R. Luo, M. Li, M. Zhou, W. Zhang, and J. Wang, “Mean field multi-agent reinforcement learning,” _CoRR_ , vol. abs/1802.05438, 2018. [Online]. Available: http://arxiv.org/abs/1802.05438
  * [8] S. K. Rakesh and M. Shrivastava, “Performance analysis of fault tolerance algorithm for pattern formation of swarm agents,” _Knowledge-Based Systems_ , vol. 240, p. 108020, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0950705121011199
  * [9] P. Pierpaoli, D. Sauter, and M. Egerstedt, “Fault tolerant control for networked mobile robots,” in _2018 IEEE Conference on Control Technology and Applications (CCTA)_ , 2018, pp. 374–379.
  * [10] C. Di Franco and G. Buttazzo, “Energy-aware coverage path planning of uavs,” in _2015 IEEE International Conference on Autonomous Robot Systems and Competitions_ , 2015, pp. 111–117.
  * [11] K. Nonoyama, Z. Liu, T. Fujiwara, M. M. Alam, and T. Nishi, “Energy-efficient robot configuration and motion planning using genetic algorithm and particle swarm optimization,” _Energies_ , vol. 15, p. 2074, 03 2022.
  * [12] M.-F. R. Lee and A. Nugroho, “Intelligent energy management system for mobile robot,” _Sustainability_ , vol. 14, no. 16, 2022. [Online]. Available: https://www.mdpi.com/2071-1050/14/16/10056
  * [13] J. N. Foerster, Y. M. Assael, N. de Freitas, and S. Whiteson, “Learning to communicate with deep multi-agent reinforcement learning,” 2016.
  * [14] P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, “Graph attention networks,” 2018.
  * [15] Y.-C. Liu, J. Tian, N. Glaser, and Z. Kira, “When2com: Multi-agent perception via communication graph grouping,” 2020.
  * [16] L. Tai and M. Liu, “A robot exploration strategy based on q-learning network,” in _2016 IEEE International Conference on Real-time Computing and Robotics (RCAR)_ , 2016, pp. 57–62.
  * [17] D. S. Chaplot, D. Gandhi, S. Gupta, A. Gupta, and R. Salakhutdinov, “Learning to explore using active neural slam,” 2020.
  * [18] S. K. Ramakrishnan, Z. Al-Halah, and K. Grauman, “Occupancy anticipation for efficient exploration and navigation,” 2020.
  * [19] R. Shrestha, F.-P. Tian, W. Feng, P. Tan, and R. Vaughan, “Learned map prediction for enhanced mobile robot exploration,” in _2019 International Conference on Robotics and Automation (ICRA)_ , 2019, pp. 1197–1204.
  * [20] R. Cimurs, I. H. Suh, and J. H. Lee, “Goal-driven autonomous exploration through deep reinforcement learning,” _IEEE Robotics and Automation Letters_ , vol. 7, no. 2, pp. 730–737, 2022.
  * [21] Y. Bengio, J. Louradour, R. Collobert, and J. Weston, “Curriculum learning,” in _Proceedings of the 26th Annual International Conference on Machine Learning_ , ser. ICML ’09. New York, NY, USA: Association for Computing Machinery, 2009, p. 41–48. [Online]. Available: https://doi.org/10.1145/1553374.1553380
  * [22] S. Macenski and I. Jambrecic, “Slam toolbox: Slam for the dynamic world,” _Journal of Open Source Software_ , vol. 6, no. 61, p. 2783, 2021. [Online]. Available: https://doi.org/10.21105/joss.02783
  * [23] S. Macenski, T. Foote, B. Gerkey, C. Lalancette, and W. Woodall, “Robot operating system 2: Design, architecture, and uses in the wild,” _Science Robotics_ , vol. 7, no. 66, p. eabm6074, 2022. [Online]. Available: https://www.science.org/doi/abs/10.1126/scirobotics.abm6074
  * [24] V. Sulzer, S. G. Marquis, R. Timms, M. Robinson, and S. J. Chapman, “Python Battery Mathematical Modelling (PyBaMM),” p. 14, June 2021. [Online]. Available: https://github.com/pybamm-team/PyBaMM
  * [25] Q. Li, W. Lin, Z. Liu, and A. Prorok, “Message-aware graph attention networks for large-scale multi-robot path planning,” 2021.
  * [26] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Köpf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, high-performance deep learning library,” 2019.
  * [27] N. Koenig and A. Howard, “Design and use paradigms for gazebo, an open-source multi-robot simulator,” in _2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566)_ , vol. 3, 2004, pp. 2149–2154 vol.3.


Complete lists of low dimensional complex associative algebras

I.S. Rakhimov1, I.M. Rikhsiboev2 and W.Basri3

1,3Department of Mathematics, Faculty of Science,

1,2,3Laboratory of Theoretical studies, Institute for Mathematical Research
(INSPEM),

University Putra Malaysia

1<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

In this paper we present a complete classification (isomorphism classes with
some isomorphism invariants) of complex associative algebras up to dimension
five (including both cases: unitary and non-unitary). In some symbolic
computations we used Maple software.

Keywords: Associative algebras, isomorphism, invariants, nilpotent, semi-
simple algebra.

##  1 Introduction

In algebra, there are three strongly related classical algebras: associative,
Lie and Jordan algebras. The objects of our attention in this paper are
associative algebras, exactly finite dimensional associative algebras over
complex numbers. The major theorems on associative algebras include the most
splendid results of the great heroes of algebra: Wedderburn, Artin, Noether,
Hasse, Brauer, Albert, Jacobson, and many others.

It is known that the universal enveloping algebra of a Lie algebra has the
structure of an associative algebra. Introduced by Loday [5], the notion of
Leibniz algebra is a generalization of the Lie algebra, where the skew-
symmetry in the bracket is dropped. Loday [6] also showed that the
relationship between Lie and associative algebras can be translated into an
analogous relationship between Leibniz and the so-called dialgebras (or
diassociative algebras), which are a generalization of associative algebra
possessing two operations $\dashv$ and $\vdash.$ In particular, it was shown
that any dialgebra becomes a Leibniz algebra under the bracket $[x,y]=x\dashv
y-y\vdash x.$

The classification of low dimensional complex Lie and Leibniz algebras can be
found in [4], [5], [2].

Our motivation to classify associative algebras comes out from the intention
to classify the diassociative algebras. A diassociative algebra is a
generalization of associative algebra, it can be obtained as a combination of
two associative algebras. In order to make up this combination we need the
list of all associative algebras (both unitary and non-unitary). The latest
lists of all unitary associative algebras in dimension two, three, four, and
five are available in [8], [1], [3] and [7], respectively.

In 1993 Loday [5] introduced a non-antisymmetric version of Lie algebras,
whose bracket satisfies the Leibniz identity and therefore this generalization
has been called Leibniz algebras. The Leibniz identity, combined with
antisymmetricity, is a variation of the Jacobi identity. Hence a Lie algebra
is antisymmetric Leibniz algebra. Loday also introduced an “associative”
version of Leibniz algebras, called diassociative algebras, equipped with two
binary operations, which satisfy certain identities. These identities are all
variations of the associative law. An associative algebra is a diassociative
algebra when the two operations coincide. The main motivation of Loday to
introduce this class of algebras was the search of an “obstruction” to the
periodicity of algebraic K-theory. Besides this purely algebraic motivation,
some relationships with classical geometry, non-commutative geometry and
physics have been recently discovered. We are interested in classification of
diassociative algebras. One of the approaches to describe a finite dimensional
diassociative law is to consider it as a combination of two associative
algebras. Therefore, we need a complete list of associative algebras.
Obviously, the classification problem of algebras (even associative algebras),
in general, is nearly unreal problem. All existence classifications of
associative algebras concern unitary ones. In order to use the classification
of associative algebras to classify the diassociative algebras we need both
lists of unitary and non unitary associative algebras. In the present, paper
we give complete lists of low-dimensional complex associative algebras.

##  2 Lists of low-dimensional associative algebras with some isomorphism
invariants

Now and what it follows, all algebras are assumed to be over the field of
complex numbers $\mathbb{C}$.
In the next section we give lists of all complex associative algebras in
dimensions 2– 4. Some remarks on the tables. In the tables $As_{p}^{q}$ stands
for $q^{th}$ algebra in dimension $p$. In the second column only nonzero
products are given. The third column describes the automorphism groups of the
algebras. For all algebras we test to be nilpotent or not, in the last case we
indicate nilpotent $N$ and semi-simple $S$ parts of $As_{p}^{q}$. By
$C(As_{p}^{q}),$ $L(As_{p}^{q})$, and $R(As_{p}^{q})$ the maximal commutative
subalgebra, the left annihilator, and the right annihilator of $As_{p}^{q}$,
respectively are denoted, and their dimensions are given in the last three
columns of the tables.

A proof, that any associative algebra of dimensions 2– 4 is included in the
lists, is available from the authors. Because of its length, it is omitted
from the paper. We tabulate only indecomposable algebras.

###  2.1 Two-dimensional associative algebras

|  Table of multiplication |  Automorphisms |  Type of algebra |  dim |  dim |  dim
---|---|---|---|---|---|---
|  |  |  |  C$(As_{p}^{q})$ |  L$(As_{p}^{q})$ |  R$(As_{p}^{q})$
$As_{2}^{1}:$ |  ${e_{1}e_{1}=e_{2}}$ |  $\,\left(\begin{array}[]{l}{a\,\,\,\,\,0}\\\ {b\,\,\,\,a^{2}}\end{array}\right)$ |  commutative,
nilpotent |  2 |  1 |  1
$As_{2}^{2}:$ |  $e_{1}e_{1}=e_{1},$ $e_{1}e_{2}=e_{2}$ |  $\,\left(\begin{array}[]{l}{1\,\,\,\,0}\\\ {a\,\,\,\,b}\end{array}\right)$ |  $A=N\dotplus S,$
$N=<e_{2}>$,
$S=<e_{1}>$ |  1 |  1 |  0
$As_{2}^{3}:$ |  $e_{1}e_{1}=e_{1},$ $e_{2}e_{1}=e_{2}$ |  $\,\left(\begin{array}[]{l}{1\,\,\,\,0}\\\ {a\,\,\,\,b}\end{array}\right)$ |  $A=N\dotplus S,$
$N=<e_{2}>$,
$S=<e_{1}>$ |  1 |  0 |  1
$As_{2}^{4}:$ |  $e_{1}e_{1}=e_{1},$ $e_{1}e_{2}=e_{2},$
$e_{2}e_{1}=e_{2}$ |  $\left(\begin{array}[]{l}{1\,\,\,\,0}\\\ {0\,\,\,\,a}\end{array}\right)$ |  commutative,
unitary,
$A=N\dotplus S,$
$N=<e_{2}>$,
$S=<e_{1}>$ |  2 |  0 |  0

###  2.2 Three-dimensional associative algebras

|  Table of
multiplication |  Automorphisms |  Type of algebra |  dim C$(As_{p}^{q})$ |  dim L$(As_{p}^{q})$ |  dim R$(As_{p}^{q})$
---|---|---|---|---|---|---
$As_{3}^{1}:$ |  $e_{1}e_{3}=e_{2},$ $e_{3}e_{1}=e_{2}$ |  $\left(\begin{array}[]{l}{a\,\,\,\,\,0\,\,\,\,\,0}\\\ {b\,\,\,\,ac\,\,\,d}\\\ {0\,\,\,\,\,0\,\,\,\,\,c}\end{array}\right)$, $\left(\begin{array}[]{l}{0\,\,\,\,\,0\,\,\,\,\,a}\\\ {b\,\,\,\,ac\,\,\,d}\\\ {c\,\,\,\,\,0\,\,\,\,\,0}\end{array}\right)$ |  commutative,
nilpotent |  3 |  1 |  1
$As_{3}^{2}:$ |  $e_{1}e_{3}=e_{2},$
$e_{3}e_{1}=\alpha e_{2},$
$\alpha$ $\in\mathbb{C}\backslash\ \\{1\\}$ |  $\,\left(\begin{array}[]{l}{a\,\,\,\,\,0\,\,\,\,\,0}\\\ {b\,\,\,\,ac\,\,\,d}\\\ {0\,\,\,\,\,0\,\,\,\,\,c}\end{array}\right)$,$\,\left(\begin{array}[]{l}{0\,\,\,\,\,0\,\,\,\,\,a}\\\ {b\,\,\,\,ac\,\,\,d}\\\ {c\,\,\,\,\,0\,\,\,\,\,0}\end{array}\right)$,
$\left(\begin{array}[]{l}{a\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,b}\\\ {c\,\,\,\,af-be\,\,\,d}\\\ {e\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,f}\end{array}\right)$ |  nilpotent |  2 |  1 |  1
$As_{3}^{3}:$ |  $e_{1}e_{1}=e_{2},$ $e_{1}e_{2}=e_{3}$,
$e_{2}e_{1}=e_{3}$ |  $\left(\begin{array}[]{l}{a\,\,\,\,\,\,\,0\,\,\,\,\,\,0}\\\ {b\,\,\,\,\,\,a^{2}\,\,\,\,\,0}\\\ {c\,\,\,2ab\,\,\,a^{3}}\end{array}\right)$ |  commutative,
nilpotent |  3 |  1 |  1

$As_{3}^{4}:$ |  $e_{1}e_{3}=e_{2},$ $e_{2}e_{3}=e_{2},$
$e_{3}e_{3}=e_{3}$ |  $\,\left(\begin{array}[]{l}{a\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,0}\\\ {b\,\,\,\,a+b\,\,\,c}\\\ {0\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,1}\end{array}\right)$ |  $A=N\dotplus S,$
$N=<e_{1},e_{2}>,$
$S=<e_{3}>$ |  2 |  0 |  2
---|---|---|---|---|---|---
$As_{3}^{5}:$ |  $e_{2}e_{3}=e_{2},$ $e_{3}e_{1}=e_{1},$
$e_{3}e_{3}=e_{3}$ |  $\,\left(\begin{array}[]{l}{a\,\,\,\,0\,\,\,b}\\\ {0\,\,\,\,c\,\,\,\,d}\\\ {0\,\,\,\,0\,\,\,\,1}\end{array}\right)$ |  $A=N\dotplus S,$
$N=<e_{1},e_{2}>,$
$S=<e_{3}>$ |  2 |  1 |  1
$As_{3}^{6}:$ |  $e_{3}e_{1}=e_{2},$ $e_{3}e_{2}=e_{2},$
$e_{3}e_{3}=e_{3}$ |  $\,\left(\begin{array}[]{l}{a\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,0}\\\ {b\,\,\,\,a+b\,\,\,c}\\\ {0\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,1}\end{array}\right)$ |  $A=N\dotplus S,$
$N=<e_{1},e_{2}>,$
$S=<e_{3}>$ |  2 |  2 |  0
$As_{3}^{7}:$ |  $e_{1}e_{2}=e_{1},$ $e_{2}e_{2}=e_{2},$
$e_{3}e_{1}=e_{1},$ $e_{3}e_{3}=e_{3}$ |  $\,\left(\begin{array}[]{l}{a\,\,\,\,b\,\,\,-b}\\\ {0\,\,\,\,\,1\,\,\,\,\,\,\,0}\\\ {0\,\,\,\,0\,\,\,\,\,\,\,\,1}\end{array}\right)$ |  unitary,
$A=N\dotplus S,$
$N=<e_{1}>,$
$S=<e_{2},e_{3}>$ |  2 |  0 |  0
$As_{3}^{8}:$ |  $e_{1}e_{3}=e_{1},$ $e_{2}e_{3}=e_{2},$
$e_{3}e_{1}=e_{1},$ $e_{3}e_{3}=e_{3}$ |  $\,\left(\begin{array}[]{l}{a\,\,\,\,0\,\,\,\,0}\\\ {0\,\,\,\,b\,\,\,\,c}\\\ {0\,\,\,\,0\,\,\,\,1}\end{array}\right)$ |  $A=N\dotplus S,$
$N=<e_{1},e_{2}>,$
$S=<e_{3}>$ |  2 |  0 |  1
$\,As_{3}^{9}:$ |  $e_{2}e_{3}=e_{2},$ $e_{3}e_{1}=e_{1},$
$e_{3}e_{2}=e_{2},$ $e_{3}e_{3}=e_{3}$ |  $\,\left(\begin{array}[]{l}{a\,\,\,\,0\,\,\,\,b}\\\ {0\,\,\,\,c\,\,\,\,0}\\\ {0\,\,\,\,0\,\,\,\,1}\end{array}\right)$ |  $A=N\dotplus S,$
$N=<e_{1},e_{2}>,$
$S=<e_{3}>$ |  2 |  1 |  0
$\,As_{3}^{10}:\,$ |  $e_{1}e_{3}=e_{1},$ $e_{2}e_{3}=e_{2},$
$e_{3}e_{1}=e_{1},$ $e_{3}e_{2}=e_{2},$
$e_{3}e_{3}=e_{3}$ |  $\,\left(\begin{array}[]{l}{a\,\,\,\,b\,\,\,\,0}\\\ {c\,\,\,\,d\,\,\,\,0}\\\ {0\,\,\,\,0\,\,\,\,1}\end{array}\right)$ |  commutative, unitary,
$A=N\dotplus S,$
$N=<e_{1},e_{2}>,$
$S=<e_{3}>$ |  3 |  0 |  0
$\,As_{3}^{11}:$ |  $e_{1}e_{3}=e_{2},$ $e_{2}e_{3}=e_{2},$
$e_{3}e_{1}=e_{2},$ $e_{3}e_{2}=e_{2},$
$e_{3}e_{3}=e_{3}$ |  $\,\left(\begin{array}[]{l}{a\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,0}\\\ {b\,\,\,\,a+b\,\,\,0}\\\ {0\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,1}\end{array}\right)$ |  commutative,
$A=N\dotplus S,$
$N=<e_{1},e_{2}>,$
$S=<e_{3}>$ |  3 |  0 |  0
$\,As_{3}^{12}:\,$ |  $e_{1}e_{1}=e_{2},$ $e_{1}e_{3}=e_{1},$
$e_{2}e_{3}=e_{2},$ $e_{3}e_{1}=e_{1},$
$e_{3}e_{2}=e_{2},$ $e_{3}e_{3}=e_{3}$ |  $\,\left(\begin{array}[]{l}{a\,\,\,\,\,0\,\,\,\,\,0}\\\ {b\,\,\,\,\,a^{2}\,\,\,0}\\\ {0\,\,\,\,\,\,0\,\,\,\,\,\,1}\end{array}\right)$ |  commutative, unitary,
$A=N\dotplus S,$
$N=<e_{1},e_{2}>,$
$S=<e_{3}>$ |  3 |  0 |  0

###  2.3 Four-dimensional associative algebras

|  Table of
multiplication |  Automorphisms |  Type of algebra |  dim C$(As_{p}^{q})$ |  dim L$(As_{p}^{q})$ |  dim R$(As_{p}^{q})$
---|---|---|---|---|---|---
$As_{4}^{1}:$ |  $e_{1}e_{2}=e_{3},$ $e_{2}e_{1}=e_{4}$ |  $\left(\begin{array}[]{l}{a\,\,\,\,0\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,a\,\,\,\,0\,\,\,\,\,0}\\\ {b\,\,\,\,c\,\,\,\,a^{2}\,\,\,0}\\\ {d\,\,\,\,e\,\,\,\,0\,\,\,\,a^{2}}\end{array}\right)$, $\left(\begin{array}[]{l}{0\,\,\,\,a\,\,\,\,0\,\,\,\,\,0}\\\ {b\,\,\,\,0\,\,\,\,0\,\,\,\,\,0}\\\ {c\,\,\,\,d\,\,\,\,0\,\,\,ab}\\\ {e\,\,\,\,f\,\,\,\,ab\,\,\,\,0}\end{array}\right)$ |  nilpotent |  3 |  2 |  2
$As_{4}^{2}:$ |  $e_{1}e_{2}=e_{4},$ $e_{3}e_{1}=e_{4}$ |  $\left(\begin{array}[]{l}{\,\,a\,\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,0}\\\ {\,\,c\,\,\,\,\,b\,\,\,\,\,0\,\,\,\,\,0}\\\ {-c\,\,\,0\,\,\,\,b\,\,\,\,\,0}\\\ {\,\,d\,\,\,\,e\,\,\,\,f\,\,\,\,ab}\end{array}\right)$ |  nilpotent |  3 |  2 |  2
$As_{4}^{3}:$ |  $e_{1}e_{2}=e_{3}$, $e_{2}e_{1}=e_{4}$,
$e_{2}e_{2}=-e_{3}$ |  $\left(\begin{array}[]{l}{a\,\,\,\,0\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,a\,\,\,\,0\,\,\,\,\,0}\\\ {b\,\,\,\,c\,\,\,\,a^{2}\,\,\,0}\\\ {d\,\,\,\,e\,\,\,\,0\,\,\,\,a^{2}}\end{array}\right)$ |  nilpotent |  3 |  2 |  2
$As_{4}^{4}:$ |  $e_{1}e_{2}=e_{3},$ $e_{2}e_{2}=e_{4}$,
$e_{2}e_{1}=-e_{3}$ |  $\left(\begin{array}[]{l}{a\,\,\,\,c\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,b\,\,\,\,0\,\,\,\,\,0}\\\ {d\,\,\,\,e\,\,\,\,ab\,\,\,0}\\\ {f\,\,\,\,g\,\,\,\,0\,\,\,\,b^{2}}\end{array}\right)$ |  nilpotent |  3 |  2 |  2
$As_{4}^{5}:$ |  $e_{1}e_{2}=e_{4},$ $e_{3}e_{3}=e_{4}$
$e_{2}e_{1}=-e_{4},$ |  $\left(\begin{array}[]{l}{a\,\,-\frac{b^{2}}{c}\,\,\,\,0\,\,\,\,\,0}\\\ {c\,\,\,\,\,\,\,0\,\,\,\,\,\,\,0\,\,\,\,\,\,\,0}\\\ {0\,\,\,\,\,\,\,0\,\,\,\,\,\,\,b\,\,\,\,\,\,\,0}\\\ {d\,\,\,\,\,\,\,e\,\,\,\,\,\,\,f\,\,\,\,\,b^{2}}\end{array}\right)$, |  nilpotent |  3 |  1 |  1
|  |  $\left(\begin{array}[]{l}{\frac{d^{2}+ab}{c}\,\,\,a\,\,\,\,0\,\,\,\,\,0}\\\ {\,\,\,\,b\,\,\,\,\,\,\,\,c\,\,\,\,\,\,\,0\,\,\,\,\,\,\,0}\\\ {\,\,\,\,0\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,d\,\,\,\,\,\,\,0}\\\ {\,\,\,\,e\,\,\,\,\,\,\,\,f\,\,\,\,\,\,\,h\,\,\,\,\,d^{2}}\end{array}\right)$ |  |  |  |
$As_{4}^{6}(\alpha):$ |  $e_{1}e_{2}=e_{4}$, $e_{2}e_{2}=e_{3},$
$e_{2}e_{1}=\frac{1+\alpha}{1-\alpha}e_{4}$, $\alpha\neq 1$ |  $\left(\begin{array}[]{l}{a\,\,\,\,c\,\,\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,0}\\\ {0\,\,\,\,b\,\,\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,0}\\\ {d\,\,\,\,e\,\,\,\,\,\,\,\,\,\,\,\,b^{2}\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,0}\\\ {f\,\,\,\,g\,\,\,\,bc(1+\alpha)\,\,\,\,ab}\end{array}\right)$ |  nilpotent |  3 |  2 |  2
$As_{4}^{7}:$ |  $e_{1}e_{1}=e_{1},$ $e_{1}e_{4}=e_{4},$
$e_{2}e_{1}=e_{2},$ $e_{2}e_{4}=e_{3}$ |  $\left(\begin{array}[]{l}{1\,\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,0}\\\ {a\,\,\,\,\,b\,\,\,\,\,0\,\,\,\,\,0}\\\ {ac\,\,bc\,\,bd\,\,ad}\\\ {c\,\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,d}\end{array}\right)$ |  $A=N\dotplus S,$
$N=<e_{2},e_{3},e_{4}>$,
$S=<e_{1}>$ |  2 |  2 |  2
$As_{4}^{8}:$ |  $e_{1}e_{1}=e_{1},$ $e_{3}e_{1}=e_{3},$
$e_{4}e_{3}=e_{2},$ $e_{2}e_{1}=e_{2}$ |  $\left(\begin{array}[]{l}{\,\,1\,\,\,\,\,\,\,\,0\,\,\,\,\,0\,\,\,\,0}\\\ {\,\,a\,\,\,\,\,\,bc\,\,\,\,d\,\,\,\,e}\\\ {-\frac{e}{c}\,\,\,0\,\,\,\,\,b\,\,\,\,\,0}\\\ {\,\,0\,\,\,\,\,\,0\,\,\,\,\,0\,\,\,\,c}\end{array}\right)$ |  $A=N\dotplus S,$
$N=<e_{2},e_{3},e_{4}>$,
$S=<e_{1}>$ |  2 |  0 |  2
$As_{4}^{9}:$ |  $e_{1}e_{1}=e_{1},$ $e_{1}e_{3}=e_{3},$
$e_{1}e_{2}=e_{2},$ $e_{3}e_{4}=e_{2}$ |  $\left(\begin{array}[]{l}{1\,\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,\,0}\\\ {a\,\,\,\,bc\,\,\,d\,\,\,-ce}\\\ {e\,\,\,\,\,0\,\,\,\,\,b\,\,\,\,\,\,0}\\\ {0\,\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,\,c}\end{array}\right)$ |  $A=N\dotplus S,$
$N=<e_{2},e_{3},e_{4}>$,
$S=<e_{1}>$ |  2 |  2 |  0
$As_{4}^{10}:$ |  $e_{1}e_{1}=e_{3},$ $e_{1}e_{3}=e_{4},$
$e_{2}e_{2}=-e_{4},$
$e_{3}e_{1}=e_{4}$ |  $\left(\begin{array}[]{l}{\sqrt[3]{a^{2}}\,\,0\,\,\,\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,0}\\\ {\,b\,\,\,\,\,\,\,\,a\,\,\,\,\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,0}\\\ {\,c\,\,\,\,\,\,b\sqrt[3]{a}\,\,\,\,\,\,\,\,\,\sqrt[3]{a^{2}}\,\,\,\,\,\,\,\,\,0}\\\ {d\,\,\,\,\,\,e\,\,\,\,\,\,2c\sqrt[3]{a^{2}}-b^{2}\,\,\,a^{2}}\end{array}\right)$ |  commutative,
nilpotent |  4 |  1 |  1
$As_{4}^{11}:$ |  $e_{1}e_{1}=e_{4},$ $e_{2}e_{1}=e_{3},$
$e_{1}e_{4}=-e_{3},$
$e_{4}e_{1}=-e_{3}$ |  $\left(\begin{array}[]{l}{a\,\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,\,0}\\\ {b\,\,\,\,a^{2}\,\,\,0\,\,\,\,\,\,\,\,\,\,\,0}\\\ {c\,\,\,\,\,d\,\,\,\,\,a^{3}\,\,a(b-2e)}\\\ {e\,\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,\,a^{2}}\end{array}\right)$ |  nilpotent |  3 |  1 |  2
$As_{4}^{12}:$ |  $e_{1}e_{1}=e_{1},$ $e_{1}e_{2}=e_{2},$
$e_{3}e_{1}=e_{3},$ $e_{4}e_{1}=e_{4}$ |  $\left(\begin{array}[]{l}{1\,\,\,\,\,0\,\,\,\,0\,\,\,\,\,0}\\\ {a\,\,\,\,b\,\,\,\,0\,\,\,\,\,0}\\\ {c\,\,\,\,\,0\,\,\,\,d\,\,\,\,e}\\\ {f\,\,\,\,0\,\,\,\,g\,\,\,\,h}\end{array}\right)$ |  $A=N\dotplus S,$
$N=<e_{2},e_{3},e_{4}>$,
$S=<e_{1}>$ |  3 |  1 |  2
$As_{4}^{13}:$ |  $e_{2}e_{2}=e_{2},$ $e_{2}e_{3}=e_{3},$
$e_{2}e_{4}=e_{4},$ $e_{1}e_{2}=e_{1}$ |  $\left(\begin{array}[]{l}{a\,\,\,\,\,b\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,\,1\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,\,c\,\,\,\,d\,\,\,\,e}\\\ {0\,\,\,\,f\,\,\,\,g\,\,\,\,h}\end{array}\right)$ |  $A=N\dotplus S,$
$N=<e_{1},e_{3},e_{4}>$,
$S=<e_{2}>$ |  3 |  2 |  1

$As_{4}^{14}:$ |  $e_{1}e_{1}=e_{1},$ $e_{3}e_{1}=e_{3},$
$e_{4}e_{1}=e_{4},$ $e_{2}e_{1}=e_{2}$ |  $\left(\begin{array}[]{l}{1\,\,\,\,\,0\,\,\,\,0\,\,\,\,\,0}\\\ {a\,\,\,\,\,b\,\,\,\,c\,\,\,\,\,d}\\\ {e\,\,\,\,f\,\,\,g\,\,\,\,h}\\\ {i\,\,\,\,\,j\,\,\,\,k\,\,\,\,l}\end{array}\right)$ |  $A=N\dotplus S,$
$N=<e_{2},e_{3},e_{4}>,$
$S=<e_{1}>$ |  3 |  0 |  3
---|---|---|---|---|---|---
$As_{4}^{15}:$ |  $e_{2}e_{2}=e_{2},$ $e_{2}e_{3}=e_{3},$
$e_{2}e_{4}=e_{4},$ $e_{2}e_{1}=e_{1}$ |  $\left(\begin{array}[]{l}{a\,\,\,\,\,b\,\,\,\,c\,\,\,\,\,d}\\\ {0\,\,\,\,\,1\,\,\,\,0\,\,\,\,\,0}\\\ {e\,\,\,\,f\,\,\,g\,\,\,\,h}\\\ {i\,\,\,\,\,j\,\,\,\,k\,\,\,\,l}\end{array}\right)$ |  $A=N\dotplus S,$
$N=<e_{1},e_{3},e_{4}>$,
$S=<e_{2}>$ |  3 |  3 |  0
$As_{4}^{16}(\alpha):$ |  $e_{1}e_{1}=e_{4},$ $e_{1}e_{2}=e_{4},$
$e_{2}e_{1}=\alpha e_{4},$
$e_{3}e_{3}=e_{4}$ |  $\left(\begin{array}[]{l}{a\,\,\,0\,\,\,0\,\,\,\,0}\\\ {0\,\,\,a\,\,\,0\,\,\,\,0}\\\ {0\,\,\,0\,\,\,a\,\,\,0}\\\ {b\,\,\,c\,\,\,d\,\,\,a^{2}}\end{array}\right)$, $\left(\begin{array}[]{l}{a\,\,\,\,0\,\,\,\,0\,\,\,\,0}\\\ {b\,\,\,\frac{c^{2}}{a}\,\,\,0\,\,\,\,0}\\\ {0\,\,\,\,0\,\,\,c\,\,\,\,0}\\\ {d\,\,\,\,e\,\,\,f\,\,\,c^{2}}\end{array}\right)$ |  nilpotent |  3 |  1 |  1
$As_{4}^{17}:$ |  $e_{1}e_{1}=e_{4},$ $e_{1}e_{2}=e_{3},$
$e_{2}e_{1}=-e_{3},$
$e_{2}e_{2}=-2e_{3}+e_{4}$ |  $\left(\begin{array}[]{l}{a\,\,\,\,0\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,a\,\,\,\,0\,\,\,\,\,0}\\\ {b\,\,\,\,c\,\,\,\,a^{2}\,\,\,0}\\\ {d\,\,\,\,e\,\,\,\,0\,\,\,\,a^{2}}\end{array}\right)$, $\left(\begin{array}[]{l}{0\,\,\,\,a\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,0}\\\ {a\,\,\,\,0\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,0}\\\ {b\,\,\,\,c\,\,-a^{2}\,-2a^{2}}\\\ {d\,\,\,\,e\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,a^{2}}\end{array}\right)$ |  nilpotent |  3 |  2 |  2
$As_{4}^{18}(\alpha):$ |  $e_{1}e_{1}=e_{4},$ $e_{1}e_{2}=e_{3},$
$e_{2}e_{1}=-\alpha e_{4},$
$e_{2}e_{2}=-e_{3}$ |  $\left(\begin{array}[]{l}{a\,\,\,\,0\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,a\,\,\,\,0\,\,\,\,\,0}\\\ {b\,\,\,\,c\,\,\,\,a^{2}\,\,\,0}\\\ {d\,\,\,\,e\,\,\,\,0\,\,\,\,a^{2}}\end{array}\right)$, $\left(\begin{array}[]{l}{0\,\,\,\,a\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,0}\\\ {a\,\,\,\,0\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,0}\\\ {b\,\,\,\,c\,\,-a^{2}\,-2a^{2}}\\\ {d\,\,\,\,e\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,a^{2}}\end{array}\right)$ |  nilpotent |  3 |  2 |  2
$As_{4}^{19}:$ |  $e_{1}e_{1}=e_{1},$ $e_{2}e_{2}=e_{2},$
$e_{2}e_{3}=e_{3},$ $e_{3}e_{1}=e_{3},$
$e_{4}e_{1}=e_{4}$ |  $\,\left(\begin{array}[]{l}{1\,\,\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,\,\,1\,\,\,\,\,0\,\,\,\,\,0}\\\ {a\,\,\,-a\,\,\,\,b\,\,\,\,0}\\\ {c\,\,\,\,\,\,0\,\,\,\,\,0\,\,\,\,d}\end{array}\right)\,\,\,$ |  $A=N\dotplus S,$
$N=<e_{3},e_{4}>,$
$S=<e_{1},e_{2}>$ |  2 |  0 |  1
$As_{4}^{20}:$ |  $e_{1}e_{1}=e_{1},$ $e_{1}e_{3}=e_{3},$
$e_{1}e_{4}=e_{4},$ $e_{2}e_{2}=e_{2},$
$e_{3}e_{2}=e_{3}$ |  $\left(\begin{array}[]{l}{1\,\,\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,\,\,1\,\,\,\,\,0\,\,\,\,\,0}\\\ {a\,\,\,-a\,\,\,\,b\,\,\,\,0}\\\ {c\,\,\,\,\,\,0\,\,\,\,\,0\,\,\,\,d}\end{array}\right)\,\,\,$ |  $A=N\dotplus S,$
$N=<e_{3},e_{4}>,$
$S=<e_{1},e_{2}>$ |  2 |  1 |  0
$As_{4}^{21}:$ |  $e_{1}e_{1}=e_{1},$ $e_{2}e_{2}=e_{2},$
$e_{2}e_{4}=e_{4},$ $e_{4}e_{1}=e_{4},$
$e_{3}e_{2}=e_{3}$ |  $\left(\begin{array}[]{l}{1\,\,\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,\,\,1\,\,\,\,\,0\,\,\,\,\,\,0}\\\ {0\,\,\,\,\,a\,\,\,\,\,\,b\,\,\,\,\,0}\\\ {c\,\,-c\,\,\,\,0\,\,\,\,d}\end{array}\right)\,\,\,$ |  $A=N\dotplus S,$
$N=<e_{3},e_{4}>,$
$S=<e_{1},e_{2}>$ |  2 |  0 |  1
$As_{4}^{22}:$ |  $e_{2}e_{2}=e_{2},$ $e_{2}e_{4}=e_{4},$
$e_{3}e_{3}=e_{3},$ $e_{3}e_{1}=e_{1},$
$e_{1}e_{2}=e_{1}$ |  $\left(\begin{array}[]{l}{a\,\,\,\,\,\,b\,\,-b\,\,\,\,0}\\\ {0\,\,\,\,\,\,1\,\,\,\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,\,0\,\,\,\,\,\,\,\,1\,\,\,\,\,0}\\\ {0\,\,\,\,\,\,c\,\,\,\,\,\,0\,\,\,\,\,d}\end{array}\right)\,\,\,$ |  $A=N\dotplus S,$
$N=<e_{1},e_{4}>,$
$S=<e_{2},e_{3}>$ |  2 |  1 |  0
$As_{4}^{23}:$ |  $e_{1}e_{1}=e_{4},$ $e_{1}e_{4}=-e_{3},$
$e_{2}e_{1}=e_{3},$ $e_{2}e_{2}=e_{3},$
$e_{4}e_{1}=-e_{3}$ |  $\left(\begin{array}[]{l}{1\,\,\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,\,\,\,0}\\\ {a\,\,\,\,\,\,1\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,\,\,\,\,0}\\\ {b\,\,\,\,\,c\,\,\,\,\,1\,\,\,\,a(a+1)-2d}\\\ {d\,\,\,\,\,a\,\,\,\,\,o\,\,\,\,\,\,\,\,\,\,\,\,\,\,1}\end{array}\right)\,\,\,$ |  nilpotent |  3 |  1 |  1
$As_{4}^{24}:$ |  $e_{2}e_{2}=e_{2},$ $e_{2}e_{3}=e_{3},$
$e_{2}e_{1}=e_{1},$ $e_{4}e_{2}=e_{4},$
$e_{1}e_{2}=e_{1}$ |  $\left(\begin{array}[]{l}{a\,\,\,\,0\,\,\,\,0\,\,\,\,0}\\\ {0\,\,\,\,1\,\,\,\,0\,\,\,\,0}\\\ {0\,\,\,\,b\,\,\,\,c\,\,\,\,0}\\\ {0\,\,\,\,d\,\,\,\,o\,\,\,\,e}\end{array}\right)\,\,\,$ |  $A=N\dotplus S,$
$N=<e_{1},e_{3},e_{4}>,$
$S=<e_{2}>$ |  3 |  1 |  1
$As_{4}^{25}:$ |  $e_{1}e_{2}=e_{4},$ $e_{1}e_{3}=e_{4},$
$e_{2}e_{1}=-e_{4},$ $e_{2}e_{2}=e_{4},$
$e_{3}e_{1}=e_{4}$ |  $\left(\begin{array}[]{l}{\,\,\,a\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,0\,\,\,\,\,0}\\\ {\,\,\,b\,\,\,\,\,\,\,\,\,\,a\,\,\,\,\,\,0\,\,\,\,\,0}\\\ {-\frac{b^{2}}{2a}\,\,-b\,\,\,\,a\,\,\,\,0}\\\ {\,\,\,\,c\,\,\,\,\,\,\,\,\,d\,\,\,\,\,\,e\,\,\,\,a^{2}}\end{array}\right)\,\,\,$ |  nilpotent |  3 |  1 |  1
$As_{4}^{26}:$ |  $e_{1}e_{1}=e_{1},$ $e_{1}e_{2}=e_{2},$
$e_{2}e_{1}=e_{2},$ $e_{4}e_{1}=e_{4},$
$e_{3}e_{1}=e_{3}$ |  $\left(\begin{array}[]{l}{1\,\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,a\,\,\,\,\,0\,\,\,\,\,0}\\\ {b\,\,\,\,0\,\,\,\,\,c\,\,\,\,\,d}\\\ {e\,\,\,\,\,0\,\,\,\,\,f\,\,\,g}\end{array}\right)\,\,\,$ |  $A=N\dotplus S,$
$N=<e_{2},e_{3},e_{4}>,$
$S=<e_{1}>$ |  3 |  0 |  2

$As_{4}^{27}:$ |  $e_{1}e_{1}=e_{1},$ $e_{1}e_{2}=e_{2},$
$e_{1}e_{4}=e_{4},$ $e_{1}e_{3}=e_{3},$
$e_{2}e_{1}=e_{2}$ |  $\left(\begin{array}[]{l}{1\,\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,a\,\,\,\,\,0\,\,\,\,\,0}\\\ {b\,\,\,\,0\,\,\,\,\,c\,\,\,\,\,d}\\\ {e\,\,\,\,\,0\,\,\,\,\,f\,\,\,g}\end{array}\right)\,\,\,$ |  $A=N\dotplus S,$
$N=<e_{2},e_{3},e_{4}>,$
$S=<e_{1}>$ |  3 |  2 |  0
---|---|---|---|---|---|---
$As_{4}^{28}(\alpha):$ |  $e_{1}e_{1}=e_{4},$ $e_{1}e_{2}=\alpha e_{4},$
$e_{2}e_{1}=-\alpha e_{4},$
$e_{2}e_{2}=e_{4},$ $e_{3}e_{3}=e_{4},$ |  $\left(\begin{array}[]{l}{a\,\,\,b\,\,\,\,c\,\,\,\,\,\,\,\,\,d}\\\ {e\,\,\,f\,\,\,g\,\,\,\,\,\,\,\,\,h}\\\ {i\,\,\,j\,\,\,\,k\,\,\,\,\,\,\,\,\,\,l}\\\ {0\,\,\,0\,\,\,0\,\,af-be}\end{array}\right)$ |  nilpotent |  3 |  1 |  1
$As_{4}^{29}:$ |  $e_{1}e_{1}=e_{1},$ $e_{1}e_{3}=e_{3},$
$e_{2}e_{2}=e_{2},$ $e_{2}e_{4}=e_{4},$
$e_{3}e_{1}=e_{3},$ $e_{4}e_{1}=e_{4}$ |  $\left(\begin{array}[]{l}{1\,\,\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,\,1\,\,\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,\,0\,\,\,\,\,a\,\,\,\,\,0}\\\ {b\,\,-b\,\,\,0\,\,\,\,c}\end{array}\right)\,\,\,$ |  unitary,
$A=N\dotplus S,$
$N=<e_{3},e_{4}>,$
$S=<e_{1},e_{2}>$ |  3 |  0 |  0
$As_{4}^{30}:$ |  $e_{1}e_{1}=e_{1},$ $e_{1}e_{3}=e_{3},$
$e_{1}e_{4}=e_{4},$ $e_{2}e_{2}=e_{2},$
$e_{3}e_{1}=e_{3},$ $e_{4}e_{2}=e_{4}$ |  $\left(\begin{array}[]{l}{1\,\,\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,\,1\,\,\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,\,0\,\,\,\,\,a\,\,\,\,\,0}\\\ {b\,\,-b\,\,\,0\,\,\,\,c}\end{array}\right)\,\,\,$ |  unitary,
$A=N\dotplus S,$
$N=<e_{3},e_{4}>,$
$S=<e_{1},e_{2}>$ |  3 |  0 |  0
$As_{4}^{31}:$ |  $e_{1}e_{1}=e_{1},$ $e_{1}e_{4}=e_{4},$
$e_{2}e_{2}=e_{2},$ $e_{2}e_{3}=e_{3},$
$e_{3}e_{1}=e_{3},$ $e_{4}e_{2}=e_{4}$ |  $\left(\begin{array}[]{l}{0\,\,\,\,\,1\,\,\,\,\,\,0\,\,\,\,\,0}\\\ {1\,\,\,\,\,\,0\,\,\,\,\,\,0\,\,\,\,\,0}\\\ {a\,\,-a\,\,\,\,0\,\,\,\,b}\\\ {c\,\,-c\,\,\,\,d\,\,\,\,0}\end{array}\right)$, $\left(\begin{array}[]{l}{1\,\,\,\,\,0\,\,\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,\,\,1\,\,\,\,\,\,0\,\,\,\,\,0}\\\ {a\,\,-a\,\,\,\,b\,\,\,\,0}\\\ {c\,\,-c\,\,\,\,0\,\,\,\,d}\end{array}\right)$ |  unitary,
$A=N\dotplus S,$
$N=<e_{3},e_{4}>,$
$S=<e_{1},e_{2}>$ |  2 |  0 |  0
$As_{4}^{32}:$ |  $e_{2}e_{1}=e_{3},$ $e_{3}e_{4}=e_{3}$,
$e_{4}e_{2}=e_{2},$ $e_{4}e_{3}=e_{3},$
$e_{4}e_{4}=e_{4},$ $e_{1}e_{4}=e_{1}$ |  $\left(\begin{array}[]{l}{\,\,a\,\,\,\,\,\,0\,\,\,\,\,\,0\,\,\,-\frac{d}{b}}\\\ {\,\,0\,\,\,\,\,\,b\,\,\,\,\,0\,\,\,\,\,\,\,\,c}\\\ {-ac\,\,d\,\,\,\,ab\,\,\,\,\frac{cd}{b}}\\\ {\,\,0\,\,\,\,\,\,0\,\,\,\,\,\,0\,\,\,\,\,\,\,1}\end{array}\right)\,\,\,$ |  $A=N\dotplus S,$
$N=<e_{1},e_{2},e_{3}>,$
$S=<e_{4}>$ |  2 |  0 |  0
$As_{4}^{33}:$ |  $e_{1}e_{1}=e_{2},$ $e_{1}e_{2}=e_{3},$
$e_{1}e_{3}=e_{4},$ $e_{2}e_{1}=e_{3},$
$e_{2}e_{2}=e_{4},$ $e_{3}e_{1}=e_{4}$ |  $\left(\begin{array}[]{l}{a\,\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,0}\\\ {b\,\,\,\,\,\,\,\,\,\,a^{2}\,\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,0}\\\ {c\,\,\,\,\,\,\,\,\,2ab\,\,\,\,\,\,\,\,\,a^{3}\,\,\,\,\,\,\,\,0}\\\ {d\,\,\,2ac+b^{2}\,\,3a^{2}b\,\,\,\,a^{4}}\end{array}\right)\,\,$ |  commutative,
nilpotent |  4 |  1 |  1
$As_{4}^{34}:$ |  $e_{2}e_{2}=e_{2},$ $e_{2}e_{3}=e_{3},$
$e_{2}e_{4}=e_{4},$ $e_{2}e_{1}=e_{1},$
$e_{4}e_{2}=e_{4},$ $e_{4}e_{3}=e_{1}$ |  $\left(\begin{array}[]{l}{ab\,\,\,\,c\,\,\,\,d\,\,\,\,ae}\\\ {0\,\,\,\,\,1\,\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,\,e\,\,\,\,\,b\,\,\,\,\,0}\\\ {0\,\,\,\,\,0\,\,\,\,0\,\,\,\,\,a}\end{array}\right)\,\,\,$ |  $A=N\dotplus S,$
$N=<e_{1},e_{3},e_{4}>,$
$S=<e_{2}>$ |  2 |  2 |  0
$As_{4}^{35}:$ |  $e_{1}e_{2}=e_{1},$ $e_{3}e_{2}=e_{3},$
$e_{2}e_{2}=e_{2},$ $e_{2}e_{4}=e_{4},$
$e_{3}e_{4}=e_{1},$ $e_{4}e_{2}=e_{4}$ |  $\left(\begin{array}[]{l}{ab\,\,\,\,c\,\,\,\,d\,\,\,\,ae}\\\ {0\,\,\,\,\,1\,\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,\,e\,\,\,\,\,b\,\,\,\,\,0}\\\ {0\,\,\,\,\,0\,\,\,\,0\,\,\,\,\,a}\end{array}\right)\,\,\,$ |  $A=N\dotplus S,$
$N=<e_{1},e_{3},e_{4}>,$
$S=<e_{2}>$ |  2 |  0 |  2
$As_{4}^{36}:$ |  $e_{1}e_{1}=e_{1},$ $e_{2}e_{2}=e_{2},$
$e_{2}e_{3}=e_{3},$ $e_{2}e_{4}=e_{4},$
$e_{3}e_{1}=e_{3},$ $e_{4}e_{1}=e_{4}$ |  $\left(\begin{array}[]{l}{1\,\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,\,1\,\,\,\,\,0\,\,\,\,\,0}\\\ {a\,\,-a\,\,\,\,b\,\,\,\,\,c}\\\ {d\,\,-d\,\,\,\,e\,\,\,f}\end{array}\right)\,\,\,$ |  unitary,
$A=N\dotplus S,$
$N=<e_{3},e_{4}>,$
$S=<e_{1},e_{2}>$ |  2 |  0 |  0
$As_{4}^{37}:$ |  $e_{1}e_{1}=e_{1},$ $e_{1}e_{2}=e_{2},$
$e_{1}e_{3}=e_{3},$ $e_{2}e_{1}=e_{2},$
$e_{3}e_{1}=e_{3},$ $e_{4}e_{1}=e_{4}$ |  $\left(\begin{array}[]{l}{1\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,a\,\,\,\,b\,\,\,\,\,0}\\\ {0\,\,\,\,\,c\,\,\,\,d\,\,\,\,0}\\\ {e\,\,\,\,\,0\,\,\,\,0\,\,\,\,f}\end{array}\right)\,\,\,$ |  $A=N\dotplus S,$
$N=<e_{2},e_{3},e_{4}>,$
$S=<e_{1}>$ |  3 |  0 |  1
$As_{4}^{38}:$ |  $e_{1}e_{1}=e_{1},$ $e_{1}e_{2}=e_{2},$
$e_{1}e_{3}=e_{3},$ $e_{1}e_{4}=e_{4},$
$e_{2}e_{1}=e_{2},$ $e_{3}e_{1}=e_{3}$ |  $\left(\begin{array}[]{l}{1\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,a\,\,\,\,b\,\,\,\,\,0}\\\ {0\,\,\,\,\,c\,\,\,\,d\,\,\,\,0}\\\ {e\,\,\,\,\,0\,\,\,\,0\,\,\,\,f}\end{array}\right)\,\,\,$ |  $A=N\dotplus S,$
$N=<e_{2},e_{3},e_{4}>,$
$S=<e_{1}>$ |  3 |  1 |  0
$As_{4}^{39}:$ |  $e_{1}e_{1}=e_{1},$ $e_{1}e_{2}=e_{2},$
$e_{1}e_{3}=e_{3},$ $e_{2}e_{1}=e_{2},$
$e_{2}e_{2}=e_{3},$ $e_{4}e_{1}=e_{4},$
$e_{3}e_{1}=e_{3}$ |  $\left(\begin{array}[]{l}{1\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,a\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,b\,\,\,\,a^{2}\,\,0}\\\ {c\,\,\,\,\,0\,\,\,\,0\,\,\,\,d}\end{array}\right)\,\,\,$ |  $A=N\dotplus S,$
$N=<e_{2},e_{3},e_{4}>,$
$S=<e_{1}>$ |  3 |  0 |  1

$As_{4}^{40}:$ |  $e_{1}e_{1}=e_{1},$ $e_{1}e_{2}=e_{2},$
$e_{1}e_{3}=e_{3},$ $e_{1}e_{4}=e_{4},$
$e_{2}e_{1}=e_{2},$ $e_{3}e_{1}=e_{3},$
$e_{4}e_{1}=e_{4}$ |  $\left(\begin{array}[]{l}{1\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,a\,\,\,\,b\,\,\,\,\,c}\\\ {0\,\,\,\,\,d\,\,\,\,e\,\,\,\,f}\\\ {0\,\,\,\,\,i\,\,\,\,j\,\,\,\,k}\end{array}\right)\,\,\,$ |  commutative,
unitary,
$A=N\dotplus S,$
$N=<e_{2},e_{3},e_{4}>,$
$S=<e_{1}>$ |  4 |  0 |  0
---|---|---|---|---|---|---
$As_{4}^{41}:$ |  $e_{1}e_{1}=e_{1},$ $e_{1}e_{2}=e_{2},$
$e_{1}e_{4}=e_{4},$ $e_{1}e_{3}=e_{3},$
$e_{2}e_{1}=e_{2},$ $e_{2}e_{2}=e_{3},$
$e_{3}e_{1}=e_{3}$ |  $\left(\begin{array}[]{l}{1\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,a\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,b\,\,\,\,a^{2}\,\,0}\\\ {c\,\,\,\,\,0\,\,\,\,0\,\,\,\,d}\end{array}\right)\,\,$ |  $A=N\dotplus S,$
$N=<e_{2},e_{3},e_{4}>,$
$S=<e_{1}>$ |  3 |  1 |  0
$As_{4}^{42}:$ |  $e_{1}e_{1}=e_{1},$ $e_{1}e_{2}=e_{2},$
$e_{2}e_{3}=e_{1},$ $e_{2}e_{4}=e_{2},$
$e_{3}e_{1}=e_{3},$ $e_{3}e_{2}=e_{4},$
$e_{4}e_{3}=e_{3},$ $e_{4}e_{4}=e_{4}$ |  $\left(\begin{array}[]{l}{\,\,\,\,1\,\,\,\,\,\,\,\,\,\,\,a\,\,\,\,\,\,\,\,0\,\,\,\,\,0}\\\ {\,\,\,\,0\,\,\,\,\,\,\,\,\,\,\frac{\,1}{b}\,\,\,\,\,\,\,\,0\,\,\,\,\,0}\\\ {-ab\,\,-a^{2}b\,\,\,b\,\,\,\,ab}\\\ {\,\,\,\,0\,\,\,\,\,\,\,-a\,\,\,\,\,\,0\,\,\,\,\,1}\end{array}\right)\,\,\,$ |  unitary,
$A=N\dotplus S,$
$N=<e_{2},e_{3}>,$
$S=<e_{1},e_{4}>$ |  2 |  0 |  0
$As_{4}^{43}:$ |  $e_{1}e_{1}=e_{1},$ $e_{1}e_{2}=e_{2},$
$e_{1}e_{3}=e_{3},$ $e_{1}e_{4}=e_{4},$
$e_{2}e_{1}=e_{2},$ $e_{2}e_{2}=e_{4},$
$e_{3}e_{1}=e_{3},$ $e_{4}e_{1}=e_{4}$ |  $\left(\begin{array}[]{l}{1\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,a\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,b\,\,\,\,c\,\,\,\,\,0}\\\ {0\,\,\,\,d\,\,\,\,e\,\,\,\,a^{2}}\end{array}\right)\,\,\,$ |  commutative,
unitary,
$A=N\dotplus S,$
$N=<e_{2},e_{3},e_{4}>,$
$S=<e_{1}>$ |  4 |  0 |  0
$As_{4}^{44}(\alpha):$ |  $e_{1}e_{1}=e_{1},$ $e_{1}e_{2}=e_{2},$
$e_{1}e_{3}=e_{3},$ $e_{1}e_{4}=e_{4},$
$e_{2}e_{1}=e_{2},$ $e_{2}e_{3}=\alpha e_{4}$,
$e_{3}e_{1}=e_{3},$ $e_{3}e_{2}=e_{4},$
$e_{4}e_{1}=e_{4}$ |  $\left(\begin{array}[]{l}{1\,\,\,\,0\,\,\,\,\,0\,\,\,\,\,\,0}\\\ {0\,\,\,\,0\,\,\,\,a\,\,\,\,\,\,\,0}\\\ {0\,\,\,\,b\,\,\,\,0\,\,\,\,\,\,\,0}\\\ {0\,\,\,\,c\,\,\,\,d\,\,\,\,\,ab}\end{array}\right)$, $\left(\begin{array}[]{l}{1\,\,\,\,0\,\,\,\,\,0\,\,\,\,0}\\\ {0\,\,\,\,a\,\,\,\,0\,\,\,\,\,0}\\\ {0\,\,\,\,0\,\,\,\,b\,\,\,\,\,0}\\\ {0\,\,\,\,c\,\,\,\,d\,\,\,\,\,ab}\end{array}\right)$, $\left(\begin{array}[]{l}{1\,\,\,0\,\,\,0\,\,\,\,\,\,\,0}\\\ {0\,\,\,a\,\,\,b\,\,\,\,\,\,\,0}\\\ {0\,\,\,c\,\,\,d\,\,\,\,\,\,\,0}\\\ {0\,\,\,e\,\,\,f\,\,ad-bc}\end{array}\right)$ |  unitary,
$A=N\dotplus S,$
$N=<e_{2},e_{3},e_{4}>,$
$S=<e_{1}>$ |  3 |  0 |  0
$As_{4}^{45}:$ |  $e_{1}e_{1}=e_{1},$ $e_{1}e_{2}=e_{2},$
$e_{1}e_{3}=e_{3},$ $e_{1}e_{4}=e_{4},$
$e_{2}e_{1}=e_{2},$ $e_{2}e_{2}=e_{3},$
$e_{2}e_{3}=e_{4},$ $e_{3}e_{1}=e_{3},$
$e_{3}e_{2}=e_{4},$ $e_{4}e_{1}=e_{4}$ |  $\left(\begin{array}[]{l}{1\,\,\,\,0\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,0}\\\ {0\,\,\,\,a\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,0}\\\ {0\,\,\,\,b\,\,\,\,\,\,\,a^{2}\,\,\,\,\,\,0}\\\ {0\,\,\,\,c\,\,\,\,2a^{2}b\,\,\,a^{3}}\end{array}\right)\,\,\,$ |  commutative,
unitary,
$A=N\dotplus S,$
$N=<e_{2},e_{3},e_{4}>,$
$S=<e_{1}>$ |  4 |  0 |  0
$As_{4}^{46}:$ |  $e_{1}e_{1}=e_{1},$ $e_{1}e_{2}=e_{2},$
$e_{1}e_{3}=e_{3},$ $e_{1}e_{4}=e_{4},$
$e_{2}e_{1}=e_{2},$ $e_{2}e_{2}=-e_{4},$
$e_{2}e_{3}=-e_{4},$ $e_{3}e_{1}=e_{3},$
$e_{3}e_{2}=e_{4},$ $e_{4}e_{1}=e_{4}$ |  $\left(\begin{array}[]{l}{1\,\,\,\,0\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,0}\\\ {0\,\,\,\,a\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,0}\\\ {0\,\,\,\,b\,\,\,\,\,\,\,a^{2}\,\,\,\,\,\,0}\\\ {0\,\,\,\,c\,\,\,\,2a^{2}b\,\,\,a^{3}}\end{array}\right)\,\,\,$ |  unitary,
$A=N\dotplus S,$
$N=<e_{2},e_{3},e_{4}>,$
$S=<e_{1}>$ |  3 |  0 |  0

where $a,b,c,d,e,f,g,h,i,j,k,l,\alpha\in\mathbb{C}.$

## References

  * [1] Azarina P., Caldwell S., Davis J., Frederick B., Three-dimensional Associative Unital Algebras, Journal of PGSS, www.pgss.mcs.cmu.edu/home/Publications.html, 227 - 237.
  * [2] Ayupov Sh.A., Omirov B.A., On 3-dimensional Leibniz algebras, Uzbek Math. Jour., No.1,(1999), 9-14.
  * [3] Gabriel R. Finite representation type is open. Lecture Notes in Math., No. 488 (1974).
  * [4] Jacobson N., Lie algebras. Interscience Tracts in Pure and Applied Mathematics, No.10,(1962), New York.
  * [5] Loday J.-L., Une version non commutative des algebras de Lie: les algebras de Leibniz, Enseign. Math, No. 39, (1993), 269-293.
  * [6] Loday J.-L., Frabetti A., Chapoton F., Goichot F., Dialgebras and Related Operads, Lecture Notes in Math., No. 1763, (2001), Springer, Berlin.
  * [7] Mazolla G. The algebraic and geometric classification associative algebras of dimension five. _Manuscripta math._ Vol.27, (1979), 1-21.
  * [8] Peirce B. Linear associative algebra. Amer. J. Math., No. 4, (1881), 97-221.
  * [9] Rakhimov I.S., Rikhsiboev I.M. A simple classification of three-dimensional complex associative algebras. _Math Digest. Research Bulletin of INSPEM _(_ Institute for Mathematical Research, UPM _)__ , (to appear).


# Investigating Strategies for Clause Recommendation

Sagar Joshi0000-0001-6095-9713 Corresponding Author: Sagar Joshi; Email:
<EMAIL_ADDRESS>Sumanth Balaji0000-0002-5669-7519 Corresponding
Author: Sumanth Balaji; Email<EMAIL_ADDRESS>Both the
corresponding authors have contributed equally to this work. Jerrin Thomas
Aparna Garimella  Vasudeva Varma  International Institute of Information
technology, Hyderabad, India Adobe Research, India

###### Abstract

Clause recommendation is the problem of recommending a clause to a legal
contract, given the context of the contract in question and the clause type to
which the clause should belong. With not much prior work being done toward the
generation of legal contracts, this problem was proposed as a first step
toward the bigger problem of contract generation. As an open-ended text
generation problem, the distinguishing characteristics of this problem lie in
the nature of legal language as a sublanguage and the considerable similarity
of textual content within the clauses of a specific type. This similarity
aspect in legal clauses drives us to investigate the importance of similar
contracts’ representation for recommending clauses. In our work, we experiment
with generating clauses for 15 commonly occurring clause types in contracts
expanding upon the previous work on this problem and analyzing clause
recommendations in varying settings using information derived from similar
contracts.

###### keywords:

Clause recommendation Legal contracts Legal NLP

, , , and

##  1 Introduction

Figure 1: A pipeline for clause recommendation: We have an incomplete, in-
draft contract at the input and the name of a clause type for which we need to
recommend a new clause to the contract. We first compute the representation of
the current contract. Since we know the clause type, we also compute a
representation of the clause type based on a library of all the clauses under
that type. Based on the representation of the current contract, we fetch $k$
similar contracts and average them to obtain a similar contract
representation. Now, making use of these three representations, we aggregate
them before sending them to our transformer-based clause decoder to generate
or recommend a new clause of the specified clause type to be added to the in-
draft contract.

AI-driven assistance in drafting legal contracts as a tool can greatly benefit
small and medium-sized enterprises. These enterprises often have limited legal
support in contractual requirements compared to their large-scale counterparts
that can afford sophisticated support from large legal teams. Contracts, being
a type of legal document, can be characterized as being composed of a series
of individual clauses or provisions, each capturing the legally binding
rights, obligations, and agreements between the involved negotiating parties.
As pointed out in [1], these clauses serve as the fundamental discourse units
while drafting or reviewing contracts and encompass the legal essence of a
contract.

The problem of clause recommendation [2] (ClauseRec) was proposed to assist
contract drafting by recommending a clause to a contract. It was the first
attempt at clause generation itself with the use of transformer-based
techniques in NLP. For clause recommendation, consider an incomplete contract
in the draft to which the drafter of the contract wishes to add a new clause
of a specific clause type. Clause recommendation serves to add such a clause
based by taking into account the context of the current contract in the
process. The work also introduced the problem of clause type relevance
prediction to determine the relevance of a clause type to be added to the
contract before proceeding with clause generation. However, the problem was
modeled as a binary classification problem to determine whether or not a given
clause type is relevant to the current contract. The pipeline for clause
recommendation nevertheless makes use of the clause type information, hence
serving the problem of clause type relevance as a secondary problem - extra to
clause recommendation. However, it keeps opening the question of whether or
not the clause type information should be taken from the user (i.e., the
contract drafter).

In our work, we study the problem of clause recommendation by experimenting
with several strategies for representing the context for clause generation and
showing the significance of clause type information for clause generation. We
improve the pipeline proposed by [2] by adding similar contract
representations to the context. Figure 1 shows our best-performing pipeline
for clause recommendation, which incorporates information from similar
contracts in addition to the current contract and specified clause type. The
generation is analyzed over 15 commonly different clause types in legal
contracts. With research on legal clause generation currently in its infancy,
we conclude by discussing the potential areas for improvement in clause
generation. We open-source our code to ease future work on this
problem111https://github.com/sagarsj42/strategies_for_clause_recommendation.

##  2 Related Work

Previous work in legal contracts has focused on understanding tasks in legal
NLP directed towards easier review and analysis of contracts. Identification
of entities in contracts is looked upon in [3], [4], the latter introducing a
dataset of 179 contracts specific to lease agreements. The dataset introduced
by [4] also provides labeled data for identifying potentially unfair clauses,
a task attempted prior by [5]. [6] identify critical contract clauses using a
set of Context-Free Grammar (CFG) rules. The task of semantically retrieving
legal clauses from a library of contracts as a span identification problem was
introduced in [7]. An expert annotation dataset for contract review is
provided in [8]. LEDGAR - a dataset for large-scale multilabel classification
was introduced in [1]. The sheer size of this dataset and the variety of
clause types covered makes its applicability much beyond the task for which it
was introduced, an example being the current work for clause generation.

Since legal language is different from the general open domain language on
which most of the models are pretrained, special attention has been paid to
developing models to give representations specialized to the domain. LegalBERT
[9] introduces a family of BERT-based models for application on downstream
legal tasks. [2] train a further pretrained version of BERT for use in context
building for clause recommendation and show the superiority of the trained
embeddings by showing distinguishable representations based on the clause
type. ALeaseBERT trained in [4] is another task-specific pretrained model on
legal contracts. We make use of LegalBERT representations in our work.

##  3 Method

The problem of clause recommendation can be modeled as a controlled text
generation problem $P(y|context)$ in which we generate a clause $y$ given some
contextual representation $context$. The $context$ representation can be
modeled by making use of the inputs that can be made available from an
incomplete version of the contract being drafted. Since a contract is
essentially a collection of clauses, the representation of a contract can be
obtained by averaging the representation of each $clause$ in the contract. The
average representation serves as the contract-specific contextual input. Thus
for a contract having $n_{clause}$ clauses, we can use a sequence encoder to
get the representation of each clause before averaging to get the contract
representation.

$contract=\frac{\Sigma_{clause}Encoder(clause)}{n_{clause}}$ (1)

The type of a legal clause is highly indicative of the clause since clauses
under a single type are similar to each other. So, the clause type
representation $clause\textunderscore type$ can be considered indicative in
determining the characteristic text of the clause to be generated. For
calculating this representation, we consider the library of all the clauses,
$N_{clause\textunderscore type}$ occurring under the concerned clause type and
compute an average of the individual clause embeddings.

$clause\textunderscore
type=\frac{\Sigma_{clause}Encoder(clause)}{N_{clause\textunderscore type}}$
(2)

The $contract$ representation can be used to retrieve $k$ most similar
contracts ($sim\\_contracts$) from an index of all the contract
representations. Each of the retrieved $k$ contracts can be used to provide
additional context in two ways:

(1) $full\textunderscore sim\textunderscore contr$: Using the representations
calculated for an entire contract and averaging over all $k$ contracts

$\begin{split}full\\_sim\\_contr=(\Sigma_{sim\\_contract\in
sim\\_contracts}\\\ [\Sigma_{clause\in
sim\\_contract}\frac{\Sigma_{clause}Encoder(clause)}{n_{clause}}])/k\end{split}$
(3)

(2) $clause\textunderscore sim\textunderscore contr$: Using only the
representation of the clauses of clause type $t$ for which we need a clause
recommendation. Here, we average the per-contract representations calculated
for the clauses of the specified type before averaging over the $k$ contracts.

$\begin{split}clause\textunderscore sim\textunderscore
contr=(\Sigma_{sim\\_contract\in sim\\_contracts}\\\
\left[\frac{\Sigma_{clause\in
sim\\_contract}\mathds{1}\\{clause\>\epsilon\>t\\}\cdot
Encoder(clause)}{\Sigma_{clause\in
sim\\_contract}\mathds{1}\\{clause\>\epsilon\>t\\}}\right])/\>k\end{split}$
(4)

By making use of these representations, we compute the $context$ vector in the
following ways:

  1. 1.

ONLY_CONTR: Using only the $contract$ representation tries to predict the
clause in a clause type agnostic setting, i.e., the model has no information
of which particular clause type it has to recommend a clause, making the task
more difficult since the model has to predict the topic as well as the
underlying content, and thus results in much poorer performance as compared to
clause type aware outputs.

$context=contract$ (5)

  2. 2.

CONTR_TYPE: This was the methodology adopted in ClauseRec and that takes into
account both - $contract$ as well as $clause\textunderscore type$
representations.

$context=(contract+clause\textunderscore type)/2$ (6)

  3. 3.

CONTR_FULLSIM: In this experiment performed in a clause type agnostic setting,
we complement the $contract$ representation with the $full\textunderscore
sim\textunderscore contr$ to see the effect of similar contract representation
when the clause type is not known.

$context=\left[contract\;;\;full\textunderscore sim\textunderscore
contr\right]$ (7)

Here, $\left[.\>;\>.\right]$ indicates concatenation of the vectors.

  4. 4.

CONTR_TYPE_FULLSIM: The original ClauseRec representation is augmented with
full _sim _contr.

$\begin{split}context=\left[(contract+clause\textunderscore
type)/2\;;\;full\textunderscore sim\textunderscore contr\right]\end{split}$
(8)

  5. 5.

CONTR_TYPE_CLAUSESIM: Since we are aware of the clause type of clause to
generate, $clause\textunderscore sim\textunderscore contr$ is used of to see
if more specific information can help generate better clauses.

$\begin{split}context=\left[(contract+clause\textunderscore
type)/2\;;\;clause\textunderscore sim\textunderscore contr\right]\end{split}$
(9)

Once we have the strategy for computing the $context$ vector, a language model
with trainable parameters $\theta$ is trained to condition on this
representation by minimizing the negative log-likelihood loss between the
predicted and the expected output tokens.

$l_{gen}=-log[\,p(y\>|\>context,\theta)\,]$ (10)

##  4 Experimentation

This section explains our filtering and re-purposing of the LEDGAR dataset,
experiments with encoder models, different values of $k$ for similar contract
retrieval, and metrics chosen for evaluation.

Dataset. We re-purpose the LEDGAR [1] dataset for multilabel clause type
identification for our task. The dataset in its cleaned version consists of
60,540 contracts having 846,274 clauses from 12,608 different types. We filter
the dataset first to eliminate all contracts with less than five clauses
present, resulting in 34,442 contracts, following which we select the top 15
clause types for analyzing the generation. The selected clause types, along
with their clause counts and length information, can be seen in Table 1.

Clause type | # clauses | mean length | std length
---|---|---|---
governing laws | 15291 | 103.00 | 104.80
amendments | 12571 | 127.19 | 119.26
entire agreements | 11023 | 98.01 | 64.50
counterparts | 10415 | 80.47 | 55.49
notices | 9726 | 148.20 | 105.62
waivers | 8945 | 133.93 | 107.43
severability | 8776 | 107.72 | 61.93
expenses | 8365 | 138.66 | 119.72
successors | 8184 | 116.60 | 87.98
survival | 6102 | 89.14 | 84.58
assigns | 6099 | 106.94 | 82.61
assignments | 5976 | 127.23 | 95.94
representations | 5373 | 136.16 | 102.57
warranties | 5320 | 138.06 | 138.13
taxes | 5184 | 164.87 | 126.78
Table 1: Distribution of the selected clause types from LEDGAR.

Encoders. Considering the domain-specific nature of English in legal
contracts, directly using transformer encoder models pretrained on generic
corpora would not yield good results, as remarked in [2]. We experimented
across three models - LegalBERT [9] trained on diverse legal texts (LegalBERT-
all), LegalBERT trained only on US contracts from EDGAR (LegalBERT-contracts),
and a further pretrained version of BERT [10] on our task-specific data using
masked language modeling objective (BERT-mlm). The further pretrained BERT
model was pretrained for two epochs on the clauses in LEDGAR data. Unlike the
previous work, we did not pretrain a ContractBERT model due to computational
limitations and focused on using existing large pretrained models available.
base versions of all the BERT-based models were used in the experiments
performed. To determine the best encoder model for representation, we trained
using the representations from these three models on CONTR_TYPE,
CONTR_TYPE_FULLSIM and CONTR_TYPE _CLAUSESIM strategies, the results of which
are shown in Table 2.

Encoder | Strategy
---|---
CONTR_TYPE | CONTR_TYPE _FULLSIM | CONTR_CLAUSESIM
ROUGE-L | BLEU | ROUGE-L | BLEU | ROUGE-L | BLEU
BERT-mlm | 38.39 | 22.46 | 38.50 | 22.33 | 38.33 | 21.49
LegalBERT-contracts | 38.31 | 21.53 | 38.89 | 22.01 | 37.45 | 20.63
LegalBERT-all | 38.75 | 22.96 | 39.28 | 22.69 | 38.85 | 23.33
Table 2: Performance of encoder models on 3 strategies: CONTR_TYPE,
CONTR_TYPE_FULLSIM, CONTR_CLAUSESIM.

LegalBERT trained on diverse legal corpora was consistently observed to
outperform the other two models and was chosen as the base encoder for all
subsequent experiments.

Indexing. An HNSW [11] index of the precomputed contract representations was
built using FAISS [12] indexing library, and a similarity search was performed
using L2 distance. Care was taken to eliminate the contract for which the
recommended clause is not in retrieved contracts, and the similar contract
representation was averaged on the rest. Because of this consideration, the
minimum value of $k$ was kept as 2.

To understand the number of retrieved similar contracts needed to build a
suitable context, the experiments were conducted by varying values of $k$ from
2 to 12 in steps of 2 for CONTR_TYPE_FULLSIM strategy. Not much variance in
the text generations scores was observed, with the BLEU and ROUGE-L metrics
staying more or less around the same values. The value of $k$ was fixed to 6
for subsequent experiments.

Model. A transformer decoder [13] with three layers was trained from scratch
across all the experiments, following the setting in ClauseRec. A wordpiece
tokenizer was trained on output clauses with a vocabulary size of 8192 tokens.
For the final experiments, all the decoder models were trained for 50 epochs
using AdamW [14] optimization with a learning rate schedule having 25% warmup
up to 6e-5 followed by linear decay. A batch size of 24 with accumulated
gradients over three steps was used. Each of the experiments was performed
using 2 RTX 2080 Ti GPUs.

Computing the input representations to the decoder was computationally heavy
at training time, even without flowing gradients through the encoders for
backpropagation. One epoch took approximately 17 hours on a single GPU, most
of which was spent calculating the input representation. This prohibitively
high time severely restricted the number of experiments that could be
performed on our computational resources. Hence, to fasten the decoder
training process, we serialized all the representations to be used prior to
the experiment, which reduced the per-epoch per-GPU duration to less than 30
min.

Metrics. Commonly used text generation metrics ROUGE [15] and BLEU [16] were
used to evaluate the system’s performance. ROUGE, a recall-oriented metric,
can be used to indicate how many of the necessary legal phrases are generated
in clauses while BLEU is indicative of the precision of the phrases. Human
evaluation was not performed due to practical constraints in obtaining domain-
specific experts for clause comprehension.

##  5 Results

Strategy | Clause type / Overall | ROUGE | BLEU
---|---|---|---
ROUGE-1 | ROUGE-2 | ROUGE-L | BLEU-1 | BLEU-2 | BLEU
ONLY_CONTR | governing laws | 38.44 | 18.89 | 30.55 | 14.84 | 11.46 | 10.22
amendments | 33.25 | 13.78 | 24.35 | 15.78 | 12.45 | 12.18
entire agreements | 34.26 | 13.68 | 25.13 | 10.88 | 10.12 | 9.60
Overall | 34.94 | 15.08 | 25.52 | 13.49 | 12.23 | 11.32
CONTR_TYPE | governing laws | 52.92 | 37.78 | 46.38 | 47.90 | 33.04 | 31.32
amendments | 37.98 | 20.38 | 31.72 | 14.16 | 8.43 | 8.01
entire agreements | 48.48 | 28.49 | 40.78 | 26.28 | 15.02 | 13.53
Overall | 47.93 | 29.20 | 38.75 | 39.01 | 24.26 | 22.96
CONTR _FULLSIM | governing laws | 40.57 | 22.11 | 33.48 | 27.45 | 15.24 | 14.70
amendments | 28.62 | 7.85 | 19.23 | 20.13 | 6.58 | 5.61
entire agreements | 23.78 | 5.25 | 16.12 | 10.97 | 2.41 | 1.25
Overall | 31.45 | 12.43 | 23.30 | 21.06 | 8.60 | 8.14
CONTR_TYPE _FULLSIM | governing laws | 54.27 | 38.07 | 47.00 | 50.60 | 34.27 | 32.37
amendments | 41.35 | 23.86 | 34.74 | 20.72 | 13.19 | 12.78
entire agreements | 49.21 | 29.68 | 41.57 | 24.85 | 15.14 | 13.94
Overall | 48.38 | 29.72 | 39.26 | 39.04 | 24.39 | 23.05
CONTR_TYPE _CLAUSESIM | governing laws | 52.66 | 36.62 | 45.89 | 47.84 | 33.24 | 31.66
amendments | 40.54 | 21.42 | 32.12 | 24.37 | 14.63 | 14.16
entire agreements | 49.79 | 30.17 | 42.00 | 28.21 | 17.58 | 16.26
Overall | 47.75 | 29.30 | 38.85 | 39.12 | 24.60 | 23.33
Table 3: Clause recommendation results across all the strategies tried out
shown for top 3 clause types (based on no. of clauses) and overall metrics
calculated on all the types.

Table 3 shows the metrics across all the strategies for the top 3 clause types
(based on the no. of clauses in the dataset) and the overall scores computed
for clauses for all the 15 clause types.

From the results, it can be easily observed that similar contracts with clause
type and input contract provide the best contextual representation among the
strategies tried. However, the strategy excluding similar contract
representation performs competitively with the former. In a clause-type
agnostic setting, the problem becomes much more difficult with modest ROUGE
scores and dismal BLEU scores indicating a high amount of hallucinated,
irrelevant content. Unlike clause type aware setting, the augmentation of
similar contract information does not help, with an observed degradation in
model output.

Among the top 3 clause types shown, the highest values are achieved by
governing laws followed by entire agreements and amendment for most cases. The
relatively less value of BLEU scores indicates that models struggle to produce
content with a high precision, and thus may have a substantial amount of
hallucinated or irrelevant content.

##  6 Analysis

We conduct analysis based on the generation of clauses from the best
performing CONTR_TYPE_FULLSIM strategy.

The performance of clause generation across all the 15 clause types on BLEU
and ROUGE-L scores can be seen in Figure 2. The clause types successors and
counterparts have the highest BLEU and ROUGE-L scores, respectively. It can be
observed that the clause types with generally higher BLEU (governing laws,
counterparts, successors) scores follow suit in ROUGE-L scores, and those
performing poorly in terms of BLEU (amendments, representations, warranties,
taxes) have poor ROUGE-L scores.

Figure 2: Variation of BLEU and ROUGE score across all 15 clause types
considered using the CONTR_TYPE_FULLSIM

In Figure 3, we plot the TSNE representations of the actual and generated
clauses for a subset of 1000 randomly sampled clauses from our test set. We
use LegalBERT-all for encoding the clauses, followed by dimensionality
reduction to obtain the plot. The plot showcases the semantic closeness of the
actual and generated clauses based on the closeness of the two distributions.

Figure 3: TSNE plot comparing the representations obtained for clause
generated by the CONTR_TYPE_FULLSIM strategy against actual clauses

We show examples of a few generated clauses by the model in Figure 4. Based on
the comparison with their expected counterparts, we can appreciate the ability
of the model to generate characteristic clause content based on the type of
clause. The exactness of the clause is not guaranteed here, as the model may
generate semantically equivalent content deemed to be an equally valid clause,
such as “this release” v/s “this agreement.” Tailoring clauses to bear exact
phrasal content can entail incorporating keyword-level information as future
work.

Figure 4: Examples of generated clauses in comparison to their expected
(actual) counterparts using the CONTR_TYPE_FULLSIM strategy

One issue we would like to point out in the current generation is the tendency
of the model to typically generate verbose clauses, which might have resulted
from optimization on the training objective to include as many valid phrases
as possible in a clause. As seen in Table 4, the mean and median lengths of
generated clauses are far more than the actual clauses. The Pearson
correlation coefficient between the two length distributions turned out to be
0.36, indicating a weak correlation. Reducing such extraneous content in
generated clauses will work towards finer clause generation.

| mean | std | median
---|---|---|---
actual | 88.05 | 88.54 | 54.0
generated | 108.02 | 84.14 | 84.0
Table 4: Statistics of generated and actual clause lengths

##  7 Conclusion & Future scope

In this work, we explored the problem of clause recommendation by
experimenting with several strategies for modeling the contextual input for
recommendation and observed the effectiveness of similar contract
representation in a clause-type aware setting. While decent results are
obtained in this paradigm, the clause type agnostic setting remains a complex
problem to solve and is more relevant for scaling clause recommendation to a
broader set of clauses. With the considerably large size of contractual
documents, future work can explore tackling the cold start problem in clause
recommendation (i.e., recommending clause without prior contract state as
context) and allowing for disentangling and customization of named entities in
recommended clauses. Future work can also achieve robustness by handling a
more diverse range and a larger number of clause types compared to the 15
clause topics we focus on.

## References

  * [1] Tuggener, D, Däniken, P, Peetz, T, Cieliebak, M LEDGAR: A Large-Scale Multi-label Corpus for Text Classification of Legal Provisions in Contracts. In Proceedings of the 12th Language Resources and Evaluation Conference 2020 (pp. 1235–1241). European Language Resources Association.
  * [2] Aggarwal, V, Garimella, A, Srinivasan, B, N, A, Jain, R ClauseRec: A Clause Recommendation Framework for AI-aided Contract Authoring. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing 2021 (pp. 8770–8776). Association for Computational Linguistics.
  * [3] Chalkidis, I, Androutsopoulos, I, Michos, A Extracting Contract Elements. In Proceedings of the 16th Edition of the International Conference on Artificial Intelligence and Law 2017 (pp. 19–28). Association for Computing Machinery.
  * [4] Leivaditi, S, Rossi, J, Kanoulas, E. A Benchmark for Lease Contract Review.
  * [5] Marco Lippi, Przemyslaw Palka, Giuseppe Contissa, Francesca Lagioia, Hans-Wolfgang Micklitz, Giovanni Sartor, Paolo Torroni. ”CLAUDETTE: an automated detector of potentially unfair clauses in online terms of service”. Artificial Intelligence and Law 2019; 27(2):117–139.
  * [6] Kubeka, S, Ade-Ibijola, A. ”Automatic Comprehension and Summarisation of Legal Contracts”. Advances in Science, Technology and Engineering Systems Journal 2021; 6(2):19–28.
  * [7] Borchmann, L, Wisniewski, D, Gretkowski, A, Kosmala, I, Jurkiewicz, D, Szalkiewicz, , Palka, G, Kaczmarek, K, Kaliska, A, Gralinski F. ”Contract Discovery: Dataset and a Few-Shot Semantic Retrieval Challenge with Competitive Baselines”. In Findings of the Association for Computational Linguistics: EMNLP 2020 2020 (pp. 4254–4268). Association for Computational Linguistics.
  * [8] Dan Hendrycks, Collin Burns, Anya Chen, Spencer Ball. ”CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review”. NeurIPS 2021.
  * [9] Chalkidis, I, Fergadiotis, M, Malakasiotis, P, Aletras, N, Androutsopoulos, I. ”LEGAL-BERT: The Muppets straight out of Law School”. In Findings of the Association for Computational Linguistics: EMNLP 2020 2020 (pp. 2898–2904). Association for Computational Linguistics.
  * [10] Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova. ”BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”.
  * [11] Malkov, Y, Yashunin, D. ”Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs”. IEEE Transactions on Pattern Analysis and Machine Intelligence 2020; 42(4):824-836.
  * [12] Johnson, J, Douze, M, Jegou, H. ”Billion-scale similarity search with GPUs”. IEEE Transactions on Big Data 2019; 7(3):535–547.
  * [13] Vaswani, A, Shazeer, N, Parmar, N, Uszkoreit, J, Jones, L, Gomez, A, Kaiser, Polosukhin, I. ”Attention is All You Need”. In Proceedings of the 31st International Conference on Neural Information Processing Systems 2017 (pp. 6000–6010). Curran Associates Inc..
  * [14] Ilya Loshchilov, Frank Hutter. ”Decoupled Weight Decay Regularization”. In ICLR 2019 .
  * [15] Lin, CY ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out 2004 (pp. 74–81). Association for Computational Linguistics.
  * [16] Papineni, K, Roukos, S, Ward, T, Zhu, WJ BLEU: A Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics 2002 (pp. 311–318). Association for Computational Linguistics.