# Proximal Policy Optimization with Graph Neural Networks for Optimal Power
Flow

Ángela López-Cardona
Barcelona Neural Networking Center, Universitat Politècnica de Catalunya
<EMAIL_ADDRESS>Guillermo Bernárdez
Barcelona Neural Networking Center, Universitat Politècnica de Catalunya
<EMAIL_ADDRESS>Pere Barlet-Ros
Barcelona Neural Networking Center, Universitat Politècnica de Catalunya
<EMAIL_ADDRESS>Albert Cabellos-Aparicio
Barcelona Neural Networking Center, Universitat Politècnica de Catalunya
<EMAIL_ADDRESS>
###### Abstract

Optimal Power Flow (OPF)is a very traditional research area within the power
systems field that seeks for the optimal operation point of electric power
plants, and which needs to be solved every few minutes in real-world
scenarios. However, due to the nonconvexities that arise in power generation
systems, there is not yet a fast, robust solution technique for the full
Alternating Current Optimal Power Flow (ACOPF). In the last decades, power
grids have evolved into a typical dynamic, non-linear and large-scale control
system —known as the power system—, so searching for better and faster ACOPF
solutions is becoming crucial. Appearance of Graph Neural Networks (GNN)has
allowed the natural use of Machine Learning (ML) algorithms on graph data,
such as power networks. On the other hand, Deep Reinforcement Learning (DRL)is
known for its powerful capability to solve complex decision-making problems.
Although solutions that use these two methods separately are beginning to
appear in the literature, none has yet combined the advantages of both. We
propose a novel architecture based on the Proximal Policy Optimization
algorithm with Graph Neural Networks to solve the Optimal Power Flow. The
objective is to design an architecture that learns how to solve the
optimization problem and that is at the same time able to generalize to unseen
scenarios. We compare our solution with the DCOPF in terms of cost after
having trained our DRL agent on IEEE 30 bus system and then computing the OPF
on that base network with topology changes.

##  INTRODUCTION

After several decades of development, power grids have evolved into a typical
dynamic, non-linear and large-scale control system, known as the power system
[1]. Nowadays, this power system is changing because of different causes.
First of all, the high penetration of Renewable Energy Sources (RES), such as
photovoltaic plants and wind farms, which bring fluctuation and intermittence
to power systems. This generation is unstable and uncontrollable and can be
affected by various external factors, namely, solar irradiation and wind
velocity for solar and wind power, respectively [2]. This unstable nature of
RES leads to a consecutive change in the optimal operation point. At the same
time, the integration of flexible sources (e.g., electric vehicles) introduce
revolutionary changes to distribution networks, such as relay protection,
bidirectional power flow, and voltage regulation [1]. Finally, new concepts
like Demand Response —which can be defined as the changes in electricity usage
by end-use customers from their normal consumption patterns in response to
changes in the price of electricity over time— change the way the Transmission
System Operator (TSO)operates the grid [3]. All these changes make the
optimization of production in power networks increasingly complex. In this
context, Optimal Power Flow (OPF) accounts for the set of techniques that
attempt to find the best operating point by optimizing the power of generators
in power grids [4].

The conventional way to solve the OPF is with traditional numerical methods
(classical Newton model interpretation) [2] and the most commonly used is
Interior Point Optimizer (IPOPT)[5]. But as networks become more complex it is
more difficult for traditional methods to converge due to their non-linear and
non-convex nature [2]. Nonlinear Alternating Current Optimal Power Flow
(ACOPF)problems are commonly approximated by linearized Direct Current Optimal
Power Flow approximation (DCOPF)solutions to obtain real power solutions where
voltage angles and reactive power flows are eliminated by substitution
(eliminating Alternating Current (AC)electric behaviour). However, this
approximation is no longer valid when power grids are heavily loaded [6].
Moreover, the OPF problem is non-convex due to the sinusoidal nature of
electrical generation and thus, it is difficult to solve [4]. Different
techniques approximate the OPF solution by relaxing this non-convex constraint
with methods like Second Order Cone Programming (SOCP)[4].

In day-to-day operations that require solving OPF within a minute every five
minutes, TSO is forced to rely on linear approximations. Solutions produced by
these approximations are inefficient and therefore waste power and overproduce
hundreds of megatons of CO2-equivalent per year. Today, 50 years after the
problem was formulated, we still do not have a fast, robust solution technique
for the full Alternating Current Optimal Power Flow (ACOPF)[7]. Solving a
ACOPF with operational constraints remains an important and challenging
optimization problem to secure an economic operation of the power grid.
Current algorithms cannot compute quickly enough, and they are prone to
serious ill‐conditioning and convergence problems. For large and complex power
system networks with large numbers of variables and constraints, finding the
optimal solution for real-time OPF in a timely manner requires a massive
amount of computing power [8].

Recently, Machine Learning (ML) methods have been considered as solutions to
this problem. The latest approaches to solve this problem use Graph Neural
Networks (GNN), which naturally allows graph data to be processed [9]. There
is an increasing number of tasks in power systems that are starting to be
solved with these methods such as time series prediction of loads and RES,
fault diagnosis, scenario generation, operational control, etc [10]. The main
advantage of these methods is that they process power grids as graphs using
node’s information and their connection through electrical lines. These
architectures can be trained with different topologies and then be applied to
different ones, thus generalizing results.

On the other hand, Deep Reinforcement Learning (DRL)is known for its powerful
capability to solve complex decision-making problems in a computationally
efficient, scalable and flexible manner —which otherwise is numerically
intractable [2]—, and it is considered one of the state-of-the-art frameworks
in Artificial Intelligence (AI) for solving sequential decision making
problems [11]. DRL based approach aims to learn gradually how to optimize
power flow in Distributed Network s and dynamically find the optimal operating
point. Some approaches use different DRL algorithms to solve the OPF, but none
of them use GNN, so they lose the ability to generalize and do not take
advantage of the information about the connections between buses and the
properties of the electrical lines connecting them. Due to the lack of
proposals that mix these two methods, and also because combining DRL and GNN
have shown to increase generalizability and reduce computational complexity in
other domains [11], we use both in our proposal.

Contribution: This paper’s main contribution is the proposal of a novel
architecture based on the Proximal Policy Optimization algorithm with Graph
Neural Networks to solve the Optimal Power Flow. At the time of writing this
paper and to the best of our knowledge, a similar architecture has not been
used before to solve this problem. The objective is to test our architecture
design to see whether it is capable of solving the optimization problem by
learning the internal dynamics of the power network, as well as to analyze if
it is also able to generalize to new scenarios that were not seen during the
training pipeline. In a DRL framework with the PPO algorithm, we precisely
make use of GNN to achieve such an architecture that can compute OPF in
different power networks that those seen in training.

We compare our solution with the DCOPF in terms of cost after having trained
our DRL agent on IEEE 30 bus system. On that base network, with topology
changes regarding number of edges and loads, we obtain better cost results
than the DCOPF reducing the cost of generation up to 30%.

##  PROBLEM STATEMENT

The figure 1 shows schematically our DRL method to solve the OPF. First, an
agent is trained using DRL. During several episodes the agent modifies the
value of the generation values of a network trying to maximize the reward,
which represents the reduction of the generation cost with respect to the
previous step. Once this agent has been trained on a base case minimizing the
cost and optimizing the search for feasible solutions, this agent can be used
to compute OPF in a power grid with changes in the topology (like the fall of
an electrical line due to maintenance or the disconnection of a load)
obtaining better or similar results, in terms of the cost, than the DCOPF.

Figure 1: Problem statement

##  BACKGROUND

In this section, we provide the necessary background for GNN and the DRL
algorithm used. In addition, although it has already been introduced, we
expand the definition of OPF. Commonly, OPF minimizes the generation cost, so
the objective is to minimize the cost of power generation while satisfying
operating constraints and demands. Some of these constraints that the network
must respect are: restrictions of both maximum and minimum voltage in the
nodes or that the net power in each bus is equal to power consumed minus
generated [7]. At the same time, the Power Flow (PF)or load flow refers to the
generation, load, and transmission network equations. It is a quantitative
study to determine the flow of electric power in a distribution network under
given load conditions whose objective is to determine the steady-state
operating values of an electrical distribution network [7].

###  Graph Neural Networks

Graph Neural Networks are deep learning based methods that operate on graph
domain. Due to its convincing performance, GNN has recently become a widely
applied graph analysis method [1]. This class of artificial neural networks
was first introduced by [12]. The architecture can be seen as a generalization
of convolutional neural networks for graph structures by unfolding a finite
number of iterations. Among the numerous GNN variants developed to date, we
use Message Passing Neural Networks (MPNN)([13]), which is a type of GNN whose
operation is based on an iterative message-passing algorithm that propagates
information between elements in a graph $G=(N,E)$. First, the hidden states of
the nodes are initialized with graph’s node level features of the data, then,
the message-passing process [14]: Message (equation 1), Aggregation (equation
1), and Update (equation 2). After a certain number of message passing steps a
readout function $r(\textperiodcentered)$ takes as input the final node states
$h_{v}^{K}$ to produce the final output of the GNN model. Readout can predict
different things at different levels depending on the problem.

$\displaystyle M_{v}^{k}=a(\\{m(h_{v}^{k},h_{k}^{i})\\}_{i\in\beta(v)})$ (1)
$\displaystyle h_{v}^{k+1}=u(h_{v}^{k},M_{k}^{v})$ (2)

###  Deep Reinforcement Learning

The goal in Reinforcement Learning (RL)is learning a behaviour (policy). It is
different from other learning paradigms in the sense that neither does it use
a dataset nor does it intend to learn a mapping into classes. Instead, in RL,
we have an agent whose goal is to learn a behaviour while interacting with an
environment to achieve a goal [15]. It is based on the reward assumption: all
goals can be formalized as the outcome of maximizing a cumulative reward. Deep
Reinforcement Learning (DRL)is known for its powerful capability to solve
complex decision-making problems, so it can be adopted here to capture the
dynamic in power flow reallocation process [2].

Within the Deep Reinforcement Learning algorithms, we use Proximal Policy
Optimization, which is an online actor critic reinforcement learning
algorithm. It was formulated in 2017 ([16]), and it showed remarkable
performance when it was implemented by OpenAI ([17]). It has become the
default reinforcement learning algorithm at OpenAI [17] because of its ease of
use and its good performance. As an actor-critic algorithm, the critic
evaluates the current policy and the result is used in the policy training.
The actor implements the policy and it is trained using Policy Gradient with
estimations from the critic [15]. PPO strikes a balance between ease of
implementation, sample complexity, and ease of tuning, trying to compute an
update at each step that minimizes the cost function while ensuring that the
deviation from the previous policy is relatively small [17]. PPO uses Trust
Region and imposes policy ratio to stay within a small interval around 1.
Because of this restriction (policy ratio $r_{t}$ is clipped), $rt$ will only
grow to as much as $1+\varepsilon$ (equation 4) [17]. The total loss function
for the PPO consists of $L^{CLIP}$ (equation 4), the mean-square error loss of
the value estimator (critic loss), and a term to encourage higher entropy
(greater exploration) (equation 3). PPO uses Generalized Advantage Estimate
(GAE)to calculate the advantage ($\hat{A}_{t}$), equation 5. This advantage
method is presented in [18].

$\displaystyle L_{TOTAL}=L^{CLIP}+L^{VALUE}*k_{1}-L^{ENTROPY}*k2$ (3)
$\displaystyle
L^{CLIP}(\theta)=\hat{E_{t}}\left[min(r_{t}(\theta)\hat{A}_{t},clip(r_{t}(\theta),1-\epsilon,1+\epsilon)\hat{A}_{t})\right]$
(4) $\displaystyle A^{GAE}_{0}=\delta_{0}+(\lambda\gamma)A^{GAE}_{1}$ (5)

##  PROPOSED METHOD

In this section, our proposal is explained in detail. It is shown
schematically in Figure 2. Both actor and critic of the DRL agent are
represented as GNN and the state of the environment is the resulting graph of
the power grid. It also shows the internal dynamics of the DRL environment in
which the value of the generation in the generators is modified at each time
step and through a PF the power grid graph is updated.

Figure 2: PPO architecture implementation.

We treat our power grid as graph-structured data using the information the
power grid topology (electrical lines as edges and buses as nodes), the loads
and generations and the physical properties of the electrical lines:
resistance $R$ and reactance $X$ ($e_{n,n}^{ACLine}=[R_{n,m},X_{n,m}]$). For
the buses as features, we use the voltage information (it’s magnitude $V$ and
phase angle $\theta$) and exchange power in that bus between the loads and
generators connected to it, $X_{n}^{AC}=[V_{n},\theta{n},P_{n},Q_{n}]$.

The global architecture of the GNN can be seen in Figure 3. The message
passing part and the readout part. At every message-passing step $k$, each
node $v$ receives the current hidden state of all the nodes in its
neighbourhood, and processes them individually by applying a message function
$m(\textperiodcentered)$ (NN) together with its own internal state $h_{v}^{k}$
and the features of the edge which connects them. Then, these messages are
aggregated (concatenation of min, max an mean). Combining this message
aggregation together with node’s hidden state and, after updating the
combination with another NN, new hidden state representations are computed.
After a certain number of message passing steps a readout function
$r(\textperiodcentered)$ takes as input the final node states $h_{v}^{K}$ to
produce the final output of the GNN model.

Figure 3: GNN architecture

For the actor, whose output is the RL policy, the readout is an NN in which
the input is each of the $node\\_representations$ independently. It is a
3-layer MLP. We pass through this readout the representation (independently)
of all the nodes that have a generator, so we have $N$ output values. Each of
the output values represent the probability of choosing that generator to
increase its power. We manage the readout in this way to make the architecture
generalizable to any number of generators. These values are used to create a
probability distribution. Then, a value is sampled (which is the ID of the
generator from which the generation is raised in that horizon $t$). Critic
uses a centralized readout that takes as inputs all node hidden states, and
whose output is the value function estimation. When aggregating
representations of all nodes, critic concatenates the sum, minimum and
maximum. So, input dimension is $3*node\\_representation$ and only one output
for the whole graph. Critic is also a 3-layer MLP.

Regarding the environment with which the agent interacts (Figure 2), at each
time instant $t$, the state is defined by the graph updated by the Power Flow.
In each horizon step ($t$) the action that the agent can perform is to upload
the generation of one of the generator nodes. The agent will decide which one
in all the available generators set. The raise will be equal to a portion. For
each generator, the power range between its maximum and minimum power will be
divided into $N$ portions. When a generator has reached its maximum power, it
does not go up any more, so the power grid (and therefore the state of the
environment) remains the same.

When the PF does not converge, it implies that with that demand and that
generation it is not possible to meet the constraints. When this happens, we
say that the solution is not feasible. When initializing one episode (initial
state of the power grid), we want the generation to be as low as possible, so
that the agent can raise it until it reaches the optimum. Besides, we have to
take into account that if it is too low in the first time steps, regardless of
which generator goes up by the agent, the solution may not be feasible.
Therefore, we decide to raise the minimum generation by 20%.

Reward is computed as the improvement in solution’s cost with respect to
$t-1$. The reward is positive in a time step when the agent increases the
generation of a node, decreasing the generation cost. Cases where the agent
chooses a generator which is already in its maximum capacity or ends in a non
feasible solution are also taken into account with negative rewards
($cte_{1},cte_{2}$ in equation 6).

$r(t)=\left\\{\begin{array}[]{l}MinMaxScaler(cost)-Last(MinMaxScaler(cost))\\\
cte_{1}\text{ if selected generator already in }P_{max}\\\ cte_{2}\text{ if
solution no feasible}\end{array}\right.$ (6)

##  PERFORMANCE EVALUATION

This section details the experimentation used to validate the proposed
approach, the data used and discusses the obtained results.

Overview: We train the agent with a base case and later we evaluate the
performance in modified cases. On the one hand, we modify the number of loads
and their values, since, in real power grid operation the loads change
continuously and, on the other hand, we remove some electrical lines
simulating that they are not available due to any breakdown or maintenance.
This way we show that the agent, once it is trained, it is able to generalize
to unseen cases. We compare the difference in cost with respect to the OPF of
our method to the currently used method, the DCOPF. Our objective is to prove
that our method is capable of generating a solution with an equal or better
result (in cost measure) without the disadvantage of having to make an
approximation of all the alternating part of the network.

###  Experimental Setting

We train the agent with the IEEE 30 bus system as case base (Figure 4). It is
formed by thirty nodes, forty links, five generators and twenty loads. All
generators are modelled as thermal generators.

Figure 4: IEEE 30 bus system [19].

We use Pandapower, a Python-based BSD-licensed power system analysis tool
[5]). A Pandapower network is a Python dictionary that holds all information
about the network in tables. The tool allows calculations such as OPF (with
IPOPT optimizer) and PF, which we use to update our environment state and
compare our cost. The information tables are modified when changing the links,
the generation or the loads.

The objective at training is to optimize the parameters so that the actor
becomes a good estimator of the optimal global policy and the critic learns to
approximate the state value function of any global state. There are many
hyperparameters that can be modified and they are divided into different
groups. Grid search has been performed on many of them and the final selected
values of the most important ones are shown below. More implementation details
can be found in the public repository [20].

  * •

Related to learning loop: Minibatch (25), epochs (3) and optimizer (ADAM),
with its own parameters like learning rate $lr$ (0.003).

  * •

Related to power grid: Generator portions (50).

  * •

Related to RL: Episodes (500), horizon size $T$ (125), reward $cte_{1}$ (-1)
and $cte_{2}$ (-2) (equation 6).

  * •

Actor and Critic GNN: Message iterations $k$ (4), node representation size
(16). The NN to create the messages is a 2-layer MLP and the update one is
3-layer MLP

PPO is an online algorithm and as any reinforcement learning algorithm does,
it learns with experience. Training pipeline is as follows:

  * •

An episode of length $T$ is generated by following the current policy. While
at the same time the critic’s value function $V$ evaluates each visited global
state; this defines a trajectory
$\left\\{s_{t},a_{t},r_{t},p_{t},V_{t},s_{t+1}\right\\}_{t=0}^{T-1}$

  * •

This trajectory is used to update the model parameters –through several epochs
of minibatch Stochastic Gradient Descent– by maximizing the global PPO
objective.

The same process of generating episodes and updating the model is repeated for
a fixed number of iterations to guarantee convergence. MinMaxScaler has been
used for data preprocessing for node features, edge features and generation
output.

###  Experimental Results

Once the training has been done and the best combination of hyperparameters,
network design and reward modelling has been chosen, the best checkpoint of
the model is selected to infer OPF in different networks.

To validate the solution obtained with our method, we use the deviation of the
cost with respect to the minimum cost, the one obtained with the OPF. We
compare that cost deviation with the cost deviation obtained when solving the
same power grid casuistry with DCOPF. We also use the convergence ratio as the
proportion of solutions that do not respect the physical constraints with
respect to the number of solutions that are evaluated. Once the model is
loaded, only the actor part interests us in the evaluation. During $T$ steps
of an episode, the actions sampled from the probability distribution obtained
from the actor for each state of the network is executed. Finally, the mean
cost of the best ten evaluations is measured, as well as the convergence of
the problem. We evaluate $100$ times for each test case. We vary the network
loads. In Tables 1 \- 3, it is highlighted between the deviation in % of our
solution with respect to the OPF’s one and the deviation obtained by the
DCOPF, the lowest for each test.

First, all network loads are varied by multiplying their value by a random
number between a value less than 1 and a value more than 1 (Table 1). Each row
in the table is a test in which the name specifies the upper and lower
percentages by which all loads have been varied. In all tests, performance
with our method is better with ratios of up to 1.30.

% DRL+OPF perf. %DCOPF perf. ratio load_inf0.1_sup0.1 0,75 0,77 1,02
load_inf0.2_sup0.1 0,59 0,68 1,16 load_inf0.3_sup0.1 0,53 0,73 1,38
load_inf0.4_sup0.1 0,61 0,67 1,10

Table 1: Results on case IEEE 30 varying loads from base case.

After varying the loads value, we experiment removing $n$ loads from the grid.
We randomly choose a number of loads, remove them from the network and
evaluate the model (Table 2). Each row in the table is a test in which the
name specifies the number of charges that have been removed. It can be seen
that when we vary the loads our cost deviation is lower than the DCOPF and
even by eliminating 50% of the loads or less, our solution obtains costs
similar to the DCOPF.

% DRL+OPF perf. %DCOPF perf. ratio load_1 0,67 0,72 1,07 load_2 0,71 0,71 1,00
load_3 0,67 0,67 1,00 load_4 1,06 0,68 0,65 load_5 0,61 0,64 1,05 load_8 1,10
0,63 0,52

Table 2: Results on case IEEE 30 removing loads from base case.

Finally, we create networks from the original network by removing one or more
electrical lines (edges) (Table 3). Each row in the table is a test in which
the name specifies the number of power lines that have been removed. Also
taking into account experiments removing electrical lines and, although as
more power lines are removed, sometimes, the agent does not find a good
feasible solution, we can conclude that our model is capable of generalizing
to unseen typologies (based on the trained one).

% DRL+OPF perf. %DCOPF perf. ratio edge_1 0,73 0,77 1,05 edge_2 0,41 1,16 2,83
edge_3 0,62 0,61 0,99 edge_4 0,65 0,88 1,36 edge_5 0,90 0,96 1,07 edge_8 0,59
0,89 1,51

Table 3: Results on case IEEE 30 removing edges from base case.

##  RELATED WORK

Until this paper, for the OPF problem, there was still no solution taking
advantage of GNN to treat graph-type data and learn how to solve this problem
allowing generalization and learning internal dynamics of the power grid. Even
so, methods can be found in the literature using each of the approaches
independently. Data-driven methods based on deep learning have been introduced
to solve OPF in approaches like [6] [21] [22] [23] [24] among others. However,
these approaches require a large amount of historical data to train and need
to recollect a large amount of data once something in the grid has changed.

On the other hand, Deep Reinforcement Learning (DRL)is known for its powerful
capability to solve complex decision-making problems [2]. DRL based approach
aims to learn gradually how to optimize power flow in Distributed Network s
and dynamically find the optimal operating point. Approaches like [25] [2]
[26] [27] use different DRL algorithms to solve the OPF, but none of them use
GNN, so they lose the ability to generalize.

##  CONCLUSIONS

We successfully design a solution to solve the OPF, capable of generalizing
using DRL and GNN. The network configuration has been changed and it has shown
that the agent is capable of finding a good solution (performance very similar
to the currently most used approach DCOPF) and that this solution is feasible
and meets the constraints.

Due to the way how GNN has been designed, the problem by design can be trained
on different cases and then used to compute on different cases in turn. In
this paper, we have validated that the proposed architecture is capable of
successfully addressing OPF, and the generalization ability of our solution
has been verified by considering modifications over the network scenario seen
during training (different loads and reducing the number of edges). By joining
these two technologies for the first time, we conclude that it is possible to
combine them, obtaining the advantages of both. Our results suggest that the
designed architecture is a promising start to solve the OPF. As a future work,
it would be interesting to explore different ways of training the agent —e.g.
with episodes that vary the load between them, and even the configuration—,
and then analyze if the model generalizes better later. Moreover, in the line
of getting the algorithm to generalize better, the inclusion of more features
in the node representation such as the maximum and minimum allowed voltage
could be explored.

More implementation details can be found in the public repository [20].

##  Acknowledgment

This publication is part of the Spanish I+D+i project TRAINER-A
(ref.PID2020-118011GB-C21), funded by MCIN/ AEI/10.13039/501100011033. This
work is also partially funded by the Catalan Institution for Research and
Advanced Studies (ICREA) and the Secretariat for Universities and Research of
the Ministry of Business and Knowledge of the Government of Catalonia and the
European Social Fund.

## References

  * [1] Zhou J, Cui G, Hu S, Zhang Z, Yang C, Liu Z, et al.. Graph Neural Networks: A Review of Methods and Applications. arXiv; 2018.  Available from: https://arxiv.org/abs/1812.08434.
  * [2] Li J, Zhang R, Wang H, Liu Z, Lai H, Zhang Y. Deep Reinforcement Learning for Optimal Power Flow with Renewables Using Graph Information. arXiv; 2021.  Available from: https://arxiv.org/abs/2112.11461.
  * [3] Albadi MH, El-Saadany EF.  A summary of demand response in electricity markets.  Electric Power Systems Research. 2008;78(11):1989-96.  Available from: https://www.sciencedirect.com/science/article/pii/S0378779608001272.
  * [4] Wood A, Wollenberg B.  Power generation, operation, and control; 2012.
  * [5] Thurner L, Scheidler A, Schäfer F, Menke JH, Dollichon J, Meier F, et al.  Pandapower—An Open-Source Python Tool for Convenient Modeling, Analysis, and Optimization of Electric Power Systems.  IEEE Transactions on Power Systems. 2018;33(6):6510-21.
  * [6] Owerko D, Gama F, Ribeiro A.  Optimal Power Flow Using Graph Neural Networks.  In: ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); 2020. p. 5930-4.
  * [7] Cain MB, O’Neill RP.  History of Optimal Power Flow and Formulations; 2012. .
  * [8] Pham T, Li X. Reduced Optimal Power Flow Using Graph Neural Network. arXiv; 2022\.  Available from: https://arxiv.org/abs/2206.13591.
  * [9] Liao W, Bak-Jensen B, Pillai JR, Wang Y, Wang Y. A Review of Graph Neural Networks and Their Applications in Power Systems. arXiv; 2021.  Available from: https://arxiv.org/abs/2101.10025.
  * [10] Diehl F.  Warm-starting AC optimal power flow with graph neural networks.  In: 33rd Conference on Neural Information Processing Systems (NeurIPS 2019); 2019. p. 1-6.
  * [11] Munikoti S, Agarwal D, Das L, Halappanavar M, Natarajan B. Challenges and Opportunities in Deep Reinforcement Learning with Graph Neural Networks: A Comprehensive review of Algorithms and Applications. arXiv; 2022.  Available from: https://arxiv.org/abs/2206.07922.
  * [12] Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G.  The Graph Neural Network Model.  IEEE Transactions on Neural Networks. 2009;20:61-80.
  * [13] Gilmer J, Schoenholz SS, Riley PF, Vinyals O, Dahl GE.  Neural message passing for quantum chemistry.  In: International conference on machine learning. PMLR; 2017. p. 1263-72.
  * [14] Bernárdez G, Suárez-Varela J, López A, Wu B, Xiao S, Cheng X, et al.  Is machine learning ready for traffic engineering optimization?  In: 2021 IEEE 29th International Conference on Network Protocols (ICNP). IEEE; 2021. p. 1-11.
  * [15] Martin M. Reinforcement Learning; 2020.  University Lecture.
  * [16] Schulman J, Wolski F, Dhariwal P, Radford A, Klimov O. Proximal Policy Optimization Algorithms. arXiv; 2017.  Available from: https://arxiv.org/abs/1707.06347.
  * [17] OpenAI. Proximal Policy Optimization; 2017.  Accessed: 2022–09-01.  University Lecture https://openai.com/blog/openai-baselines-ppo/.
  * [18] Schulman J, Moritz P, Levine S, Jordan M, Abbeel P. High-Dimensional Continuous Control Using Generalized Advantage Estimation. arXiv; 2015.  Available from: https://arxiv.org/abs/1506.02438.
  * [19] Fraunhofer UoK IEE. pandapower documentation; 2022.  Available from: https://pandapower.readthedocs.io/en/v2.10.0/# [cited 14.08.2022].
  * [20] Angela L. opf_gnn_ppo. GitHub; 2022.  https://github.com/anlopez94/opf_gnn_ppo.
  * [21] Donon B, Donnot B, Guyon I, Marot A.  Graph Neural Solver for Power Systems.  In: IJCNN 2019 - International Joint Conference on Neural Networks. Budapest, Hungary; 2019. Available from: https://hal.archives-ouvertes.fr/hal-02175989.
  * [22] Donon B, Clément R, Donnot B, Marot A, Guyon I, Schoenauer M.  Neural networks for power flow: Graph neural solver.  Electric Power Systems Research. 2020;189:106547.  Available from: https://www.sciencedirect.com/science/article/pii/S0378779620303515.
  * [23] Pan X, Chen M, Zhao T, Low SH.  DeepOPF: A Feasibility-Optimized Deep Neural Network Approach for AC Optimal Power Flow Problems.  IEEE Systems Journal. 2022:1-11.
  * [24] Donnot B, Guyon I, Schoenauer M, Marot A, Panciatici P.  Fast Power system security analysis with Guided Dropout. 2018 01.
  * [25] Zhen H, Zhai H, Ma W, Zhao L, Weng Y, Xu Y, et al.  Design and tests of reinforcement-learning-based optimal power flow solution generator.  Energy Reports. 2022;8:43-50.  2021 The 8th International Conference on Power and Energy Systems Engineering.  Available from: https://www.sciencedirect.com/science/article/pii/S2352484721012737.
  * [26] Cao D, Hu W, Xu X, Wu Q, Huang Q, Chen Z, et al.  Deep Reinforcement Learning Based Approach for Optimal Power Flow of Distribution Networks Embedded with Renewable Energy and Storage Devices.  Journal of Modern Power Systems and Clean Energy. 2021;9(5):1101-10.
  * [27] Zhou Y, Zhang B, Xu C, Lan T, Diao R, Shi D, et al.  A Data-driven Method for Fast AC Optimal Power Flow Solutions via Deep Reinforcement Learning.  Journal of Modern Power Systems and Clean Energy. 2020;8(6):1128-39.


# Essential ideals represented by mod-annihilators of modules

Rameez Rajaa, Shariefuddin Pirzadab
aDepartment of Mathematics, National Institute of Technology,
Hazratbal, Srinagar, India
bDepartment of Mathematics, University of Kashmir,
Hazratbal, Srinagar, India
<EMAIL_ADDRESS><EMAIL_ADDRESS>

Abstract. Let $R$ be a commutative ring with unity, $M$ be a unitary
$R$-module and $G$ a finite abelian group (viewed as a $\mathbb{Z}$-module).
The main objective of this paper is to study properties of mod-annihilators of
$M$. For $x\in M$, we study the ideals $[x:M]=\\{r\in R\leavevmode\nobreak\
|\leavevmode\nobreak\ rM\subseteq Rx\\}$ of $R$ corresponding to mod-
annihilator of $M$. We investigate that when $[x:M]$ is an essential ideal of
$R$. We prove that arbitrary intersection of essential ideals represented by
mod-annihilators is an essential ideal. We observe that $[x:M]$ is injective
if and only if $R$ is non-singular and the radical of $R/[x:M]$ is zero.
Moreover, if essential socle of $M$ is non-zero, then we show that $[x:M]$ is
the intersection of maximal ideals and $[x:M]^{2}=[x:M]$. Finally, we discuss
the correspondence of essential ideals of $R$ and vertices of the annihilating
graphs realized by $M$ over $R$.

Keywords: Module, ring, essential ideal, annihilator, graph.

AMS subject classification: Primary: 13C70, 05C25.

##  1 Introduction

A nonzero ideal in a commutative ring is called essential if it intersects
with every other nonzero ideal nontrivially. The study of essential ideals in
a ring $R$ is a classical problem. For instance, Green and Van Wyk in [7]
characterized essential ideals in certain classes of commutative and non-
commutative rings. The authors in [4, 11] studied essential ideals in $C(X)$,
where $C(X)$ denotes the set of continuous functions on $X$. They
topologically characterized the scole and essential ideals. Moreover,
essential ideals have been investigated in rings of measurable functions [13]
and $C^{*}$\- algebras [10]. For more on essential ideals, see [3, 8, 9, 20].

Throughout, $R$ is a commutative ring (with $1\neq 0$) and all modules are
unitary unless otherwise stated. $[N:M]=\\{r\in R\leavevmode\nobreak\
|\leavevmode\nobreak\ rM\subseteq N\\}$ denotes an ideal of $R$. The symbols
$\subseteq$ and $\subset$ have usual set theoretic meaning as containment and
proper containment. We will denote the ring of integers by $\mathbb{Z}$,
positive integers by $\mathbb{N}$ and the ring of integers modulo $n$ by
$\mathbb{Z}_{n}$. For basic definitions from ring and module theory we refer
to [6, 23].

For a $R$-module $M$ and $x\in M$, set $[x:M]=\\{r\in R\leavevmode\nobreak\
|\leavevmode\nobreak\ rM\subseteq Rx\\}$, which clearly is an ideal of $R$ and
an annihilator of the factor module $M/Rx$, whereas the annihilator of $M$
denoted by $ann(M)$ is $[0:M]$.

Recently in [17], the elements of a module $M$ have been classified into full-
annihilators, semi-annihilators and star-annihilators. We recall a definition
concerning full-annihilators, semi-annihilators and star-annihilators of a
module $M$.

###### Definition 1.1

An element $x\in M$ is a:

(i) full-annihilator, if either $x=0$ or $[x:M][y:M]M=0$, for some nonzero
$y\in M$ with $[y:M]\neq R$,

(ii) semi-annihilator, if either $x=0$ or $[x:M]\neq 0$ and $[x:M][y:M]M=0$,
for some nonzero $y\in M$ with $0\neq[y:M]\neq R$,

(iii) star-annihilator, if either $x=0$ or $ann(M)\subset[x:M]$ and
$[x:M][y:M]M=0$, for some nonzero $y\in M$ with $ann(M)\subset[y:M]\neq R$.

We denote by $A_{f}(M)$, $A_{s}(M)$ and $A_{t}(M)$ respectively the sets of
full-annihilators, semi-annihilators and star-annihilators for any module $M$
over $R$ and call these annihilators as mod-annihilators. We set
$\widehat{A_{f}(M)}=A_{f}(M)\backslash\\{0\\}$,
$\widehat{A_{s}(M)}=A_{s}(M)\backslash\\{0\\}$ and
$\widehat{A_{t}(M)}=A_{t}(M)\backslash\\{0\\}$.

This paper is organized as follows. In Section 2, we study the correspondence
of essential ideals in $R$ and submodules of $M$ represented by mod-
annihilators. For some finite abelian group $G$ (viewed as a
$\mathbb{Z}$-module), we determine the value of $n$ such that
$[x:G]=n\mathbb{Z}$, where $x\in G$. We characterize all $\mathbb{Z}$-module
$M$ such that $[x:M]$ is an essential ideal of $R$. Furthermore, we discuss
that when $[x:M]$ as a $R$-module is injective and prove that if essential
socle of $M$ is non-zero, then $[x:M]$ is the intersection of maximal ideals
and $[x:M]^{2}=[x:M]$. In Section 3, we discuss the correspondence of
essential ideals of $R$ and vertices of the annihilating graphs realized by
modules over commutative rings. We conclude this paper with a discussion on
some problems in this area of research.

##  2 Essential ideals represented by mod-annihilators

In this section, we discuss the correspondence of essential ideals in $R$
represented by elements of $\widehat{A_{f}(M)}$, and submodules of $M$
generated by elements of $\widehat{A_{f}(M)}$. We characterize essential
ideals corresponding to $\mathbb{Z}$-modules. We discuss the cases of finite
abelian groups where essential ideals which are represented by elements of
$\widehat{A_{f}(M)}$ corresponding to submodules of $M$ are isomorphic. If $M$
is a non-simple $R$-module, then for $x\in\widehat{A_{f}(M)}$, we show that an
ideal $[x:M]$ considered as an $R$-module is injective. We also study
essential ideals represented by mod-annihilators over hereditary and regular
rings.

By Definition 1.1, we see that there is a correspondence of ideals in $R$
represented by elements of $\widehat{A_{f}(M)}$, $\widehat{A_{s}(M)}$, and
$\widehat{A_{t}(M)}$ and cyclic submodules of $M$ generated by elements of
sets $\widehat{A_{f}(M)}$, $\widehat{A_{s}(M)}$, and $\widehat{A_{t}(M)}$.
Furthermore, the containment $A_{t}(M)\subseteq A_{s}(M)\subseteq A_{f}(M)$ is
clear, so our main emphasis is on the set $\widehat{A_{f}(M)}$. However, one
can study these sets separately for any module $M$.

Let $\lambda=(\lambda_{1},\lambda_{2},\cdots,\lambda_{r})$ be a partition of
$n$ denoted by $\lambda\vdash n$. For any $\mu\vdash n$, we have an abelian
group of order $p^{n}$ and conversely every abelian group corresponds to some
partition of $n$. In fact, if
$H_{\mu,p}=\mathbb{Z}/p^{{\mu}_{1}}\mathbb{Z}\leavevmode\nobreak\
\oplus\leavevmode\nobreak\
\mathbb{Z}/p^{{\mu}_{2}}\mathbb{Z}\leavevmode\nobreak\
\oplus\leavevmode\nobreak\ \cdots\leavevmode\nobreak\
\oplus\leavevmode\nobreak\ \mathbb{Z}/p^{{\mu}_{r}}\mathbb{Z}$ is a subgroup
of
$G_{\lambda,p}=\mathbb{Z}/p^{\lambda_{1}}\mathbb{Z}\oplus\mathbb{Z}/p^{\lambda_{2}}\mathbb{Z}\oplus\cdots\oplus\mathbb{Z}/p^{\lambda_{r}}\mathbb{Z}$,
then
$\mu_{1}\leq\lambda_{1},\mu_{2}\leq\lambda_{2},\cdots,\mu_{r}\leq\lambda_{r}$.
If these inequalities holds we write $\mu\subset\lambda$, that is a
“containment order”on partitions. For example, a $p$-group
$\mathbb{Z}/p^{5}\mathbb{Z}\leavevmode\nobreak\ \oplus\leavevmode\nobreak\
\mathbb{Z}/p\mathbb{Z}\leavevmode\nobreak\ \oplus\leavevmode\nobreak\
\mathbb{Z}/p\mathbb{Z}$ is of type $\lambda=(5,1,1)$. The possible types for
its subgroup are:
$(5,1,1),(4,1,1),(3,1,1),(2,1,1),(1,1,1),2(5,1),2(4,1),2(3,1),2(2,1),2(1,1),(5),(4),(3),(2),2$

$(1)$. Note that the types $(5,1),(4,1),(3,1),(2,1),(1,1)$ are appearing twice
in the sequence of partitions for a subgroup.

Let $\lambda=(1,1,\cdots,1)=(1^{n})$. A group of type $\lambda$ is nothing but
the $\mathbb{Z}/p\mathbb{Z}$-vector space
$\mathbb{Z}/p\mathbb{Z}\oplus\mathbb{Z}/p\mathbb{Z}\oplus\cdots\oplus\mathbb{Z}/p\mathbb{Z}$.
Its subgroups are of type $(1^{r})$, where $0\leq r\leq n$. The essential
ideals corresponding to subspaces of vector space
$\mathbb{Z}/p\mathbb{Z}\oplus\mathbb{Z}/p\mathbb{Z}\oplus\cdots\oplus\mathbb{Z}/p\mathbb{Z}$
(represented by elements of the set
$A_{f}(\mathbb{Z}/p\mathbb{Z}\oplus\mathbb{Z}/p\mathbb{Z}\oplus\cdots\oplus\mathbb{Z}/p\mathbb{Z})$
are same. In fact,
$[x:\mathbb{Z}/p\mathbb{Z}\oplus\mathbb{Z}/p\mathbb{Z}\leavevmode\nobreak\
\oplus\leavevmode\nobreak\ \cdots\leavevmode\nobreak\
\oplus\leavevmode\nobreak\
\mathbb{Z}/p\mathbb{Z}]=ann(\mathbb{Z}/p\mathbb{Z}\oplus\mathbb{Z}/p\mathbb{Z}\oplus\cdots\oplus\mathbb{Z}/p\mathbb{Z})=p\mathbb{Z}$.

More generally, for a finite abelain $p$-group of the type
$\mathbb{Z}/p^{\alpha}\mathbb{Z}\oplus\mathbb{Z}/p^{\alpha}\mathbb{Z}\cdots\oplus\mathbb{Z}/p^{\alpha}\mathbb{Z}$,
where $\alpha\geq 2$. The essential ideals represented by elements of the set
$A_{f}(\mathbb{Z}/p^{\alpha}\mathbb{Z}\oplus\mathbb{Z}/p^{\alpha}\mathbb{Z}\oplus\cdots\oplus\mathbb{Z}/p^{\alpha}\mathbb{Z})=p^{\alpha}\mathbb{Z}$.

A finite abelian group is isomorphic to the group of the form
$\mathbb{Z}/p^{\alpha_{1}}\mathbb{Z}\oplus\mathbb{Z}/p^{\alpha_{2}}\mathbb{Z}\oplus\cdots\oplus\mathbb{Z}/p^{\alpha_{n}}\mathbb{Z}$
whereas a finitely generated abelian group with Betti number $n$ is of the
from
$\mathbb{Z}/p^{\alpha_{1}}\mathbb{Z}\oplus\mathbb{Z}/p^{\alpha_{2}}\mathbb{Z}\oplus\cdots\oplus\mathbb{Z}/p^{\alpha_{n}}\mathbb{Z}\oplus\mathbb{Z}\oplus\cdots\oplus\mathbb{Z}$.
It is very difficult to determine the exact ideals represented by mod-
annihilators of sets
$A_{f}(\mathbb{Z}/p^{\alpha_{1}}\mathbb{Z}\oplus\mathbb{Z}/p^{\alpha_{2}}\mathbb{Z}\oplus\cdots\oplus\mathbb{Z}/p^{\alpha_{n}}\mathbb{Z})$
and
$A_{f}(\mathbb{Z}/p^{\alpha_{1}}\mathbb{Z}\oplus\mathbb{Z}/p^{\alpha_{2}}\mathbb{Z}\oplus\cdots\oplus\mathbb{Z}/p^{\alpha_{n}}\mathbb{Z}\oplus\mathbb{Z}\oplus\cdots\oplus\mathbb{Z})$.
However, it is clear from the definition of mod-annihilators that for some
$x\in
A_{f}(\mathbb{Z}/p^{\alpha_{1}}\mathbb{Z}\oplus\mathbb{Z}/p^{\alpha_{2}}\mathbb{Z}\oplus\cdots\oplus\mathbb{Z}/p^{\alpha_{n}}\mathbb{Z}\oplus\mathbb{Z}\oplus\cdots\oplus\mathbb{Z})$,
$[x:\mathbb{Z}/p^{\alpha_{1}}\mathbb{Z}\oplus\mathbb{Z}/p^{\alpha_{2}}\mathbb{Z}\oplus\cdots\oplus\mathbb{Z}/p^{\alpha_{n}}\mathbb{Z}\oplus\mathbb{Z}\oplus\cdots\oplus\mathbb{Z}]$
is some ideal in $\mathbb{Z}$.

Using the description given above, we now characterize all essential ideals
represented by elements of $\widehat{A_{f}(M)}$ and corresponding to
$\mathbb{Z}$-modules.

###### Lemma 2.1

If $M$ is any $\mathbb{Z}$-module, then $[x:M]$ is an essential ideal if and
only if $[x:M]$ is non-zero for all $x\in\widehat{A_{f}(M)}$.

Proof. Let $M$ be a $\mathbb{Z}$-module. Clearly, $M$ is an abelian group in a
unique way. For all $x\in\widehat{A_{f}(M)}$, we have $[x:M]=n\mathbb{Z}$,
$n\in\mathbb{N}$. The ideal $n\mathbb{Z}$ intersects non-trivially with any
ideal $m\mathbb{Z}$, $m\in\mathbb{N}$ in $\mathbb{Z}$. So, if $M$ is a non-
simple $\mathbb{Z}$-module, then for every $x\in M$, it follows that $[x:M]$
is an essential ideal. Note that $M$ is simple if and only if
$\widehat{A_{f}(G)}=\emptyset$.

If possible, suppose $[x:M]=\\{0\\}$, then $[x:M]$ does not intersect non-
trivially with non-trivial ideals of $\mathbb{Z}$, a contradiction.

Since it is possible to have some finitely generated $\mathbb{Z}$-modules such
that the set of mod-annihilators is equal to zero only which of course by
definition is not an essential ideal. Consider a $\mathbb{Z}$-module
$M=\mathbb{Z}\oplus\mathbb{Z}\oplus\cdots\oplus\mathbb{Z}$, which is a direct
sum of $n$ copies of $\mathbb{Z}$. It is easy to verify that
$\widehat{A_{f}(M)}=\widehat{M}$ with $[x:M][y:M]M=0$ for all $x,y\in M$. The
cyclic submodules generated by elements of $\widehat{A_{f}(M)}$ are simply
lines with integral coordinates passing through the origin in the hyperplane
$\mathbb{R}\oplus\mathbb{R}\oplus\cdots\oplus\mathbb{R}$ and these lines
intersect at the origin only. Thus, for each $x\in M$, it follows that $[x:M]$
is not an essential ideal in $\mathbb{Z}$. In fact $[x:M]$ is a zero-ideal in
$\mathbb{Z}$.

For any $R$-module $M$ and $x\in\widehat{A_{f}(M)}$, it would be interesting
to characterize essential ideals $[x:M]$ represented by elements of
$\widehat{A_{f}(M)}$ such that the intersection of all essential ideals is
again an essential ideal. It is easy to see that a finite intersection of
essential ideals in any commutative ring is an essential ideal. But an
infinite intersection of essential ideals need not to be an essential ideal,
even a countable intersection of essential ideals in general is not an
essential ideal, as can be seen in [4]. If the cardinality of $M$ is finite
over $R$, then the submodules determined by elements of $\widehat{A_{f}(M)}$
are finite and therefore the ideals corresponding to submodules are finite in
number. Thus, we conclude that for every $x\in\widehat{A_{f}(M)}$, the
intersection of essential ideals $[x:M]$ in $R$ is an essential ideal. For the
other case, that is, if the cardinality of $M$ is infinite over $R$, we have
the following result. Note that, a nonzero submodule of a module $M$ is said
to be an essential submodule of $M$ if it intersects non-trivially with other
nonzero submodules of $M$.

###### Theorem 2.2

Let $M$ be a $R$-module such that every proper submodule of $M$ is cyclic over
$R$. For $x\in\widehat{A_{f}(M)}$, if the submodule generated by $x$
intersects non-trivially with every other nonzero submodule of $M$, then
$[x:M]$ is an essential ideal in $R$.

Proof. Assume $\bigcap\limits_{0\neq x\in M}Rx\neq 0$. If
$\widehat{A_{f}(M)}=\phi$, then $M$ is simple, a contradiction. Let
$x\in\widehat{A_{f}(M)}$ and let $Rx$ be the submodule generated by $x$. Since
$Rx$ intersects non-trivially with every other submodule, so there exist
$y\in\widehat{A_{f}(M)}$ such that $Rx\cap Ry\neq 0$. It suffices to prove the
result for $Rx\cap Ry$. Let $z\in Rx\cap Ry$ and let $[x:M]$, $[y:M]$, $[z:M]$
be ideals of $R$ corresponding to submodules $Rx$, $Ry$ and $Rz$. Then
$[z:M]\subseteq[x:M]\cap[y:M]\neq 0$, which implies $[x:M]$ intersects non-
trivially with every nonzero ideal corresponding to the submodule generated by
an element of $\widehat{A_{f}(M)}$. For any other ideal $I$ of $R$, it is
clear that $IM=\\{\sum\limits_{finite}am:a\in I,\leavevmode\nobreak\ m\in
M\\}=Ra$ for some $a\in M$. Thus $I$ corresponds to the cyclic submodule
generated by $a\in M$. It follows that $[x:M]\cap I\neq 0$, for every nonzero
ideal of $R$ and we conclude that $[x:M]$ is an essential ideal for each
$x\in\widehat{A_{f}(M)}$.

The converse of Theorem 2.2 is not true in general. We can easily construct
examples from $\mathbb{Z}$-modules such that an ideal corresponding to the
submodule generated by some element of $\widehat{A_{f}(M)}$ is an essential
ideal, but the intersection of all submodules determined by elements of
$\widehat{A_{f}(M)}$ is empty. However, if every ideal $[x:M]$, where
$x\in\widehat{A_{f}(M)}$ corresponds to an essential submodule of $M$, then we
have a non-zero intersection.

###### Corollary 2.3

Let $M$ be a $R$-module.

(i) For $x\in\widehat{A_{f}(M)}$, if the cyclic submodule $Rx$ intersects with
every other cyclic nonzero submodule of $M$ non-trivially, then $[x:M]$ is an
essential ideal in $R$.

(ii) The intersection $\bigcap\limits_{x\in\widehat{A_{f}(M)}}[x:M]$ is an
essential ideal in $R$ if and only if every submodule of $M$ is essentially
cyclic over $R$.

In the preceding results, we proved that “arbitrary intersection of essentials
ideals is an essential ideal”. We formulated this theory of essential ideals
using the concept of mod-annihilators and mainly the theory involves study of
cyclic submodules of $M$. It is interesting to develop a similar theory that
would employ the other finitely generated submodules of $M$. So, motivated by
[4], we have the following question regarding essential ideals represented by
elements of $\widehat{A_{f}(M_{N})}$, where $\widehat{A_{f}(M_{N})}=\\{r\in
R\leavevmode\nobreak\ |\leavevmode\nobreak\ rM\subseteq N\\}$, $N$ is a
finitely generated submodule of $M$.

###### Problem 2.4

Let $M$ be a $R$-module. For $x\in\widehat{A_{f}(M_{N})}$, characterize
essential ideals $[x:M]$ in $R$ such that their intersection is an essential
ideal.

For a $R$-module $M$, let $Z(M)$ denote the following.

$Z(M)=\\{m\in M:\leavevmode\nobreak\ ann(m)\leavevmode\nobreak\
is\leavevmode\nobreak\ an\leavevmode\nobreak\ essential\leavevmode\nobreak\
ideal\leavevmode\nobreak\ in\leavevmode\nobreak\ R\\}$.

If $Z(M)=M$, then $M$ is said to be singular and if $Z(M)=0$, then $M$ is said
to be non-singular. By $rad(M)$, we denote the intersection of all maximal
submodules of $M$. So, $rad(R)$ is the Jacobson radical $J(R)$ of a ring $R$.
The socle of an $R$-module $M$ denoted by $Soc(M)$ is the sum of simple
submodules or equivalently the intersection of all essential submodules. To
say that $Soc(M)$ is an essential socle is equivalent to saying that every
cyclic submodule of $M$ contains a simple submodule of $M$. An essential socle
of $M$ is denoted by $essoc(M)$.

###### Lemma 2.5

Let $M$ be a $R$-module with $essoc(M)\neq 0$, $\bigcap\limits_{0\neq x\in
M}Rx\neq 0$. Then for $x\in\widehat{A_{f}(M)}$, $R/[x:M]$ is a singular
module.

Proof. Since $\bigcap\limits_{0\neq x\in M}Rx\neq 0$ and $essoc(M)\neq 0$,
therefore, $\widehat{A_{f}(M)}\neq\emptyset$. Thus, $[x:M]$ is an essential
ideal. Moreover, $Z\big{(}R/[x:M]\big{)}=R/[x:M]$. Therefore, $R/[x:M]$ is a
singular module.

A ring $R$ is said to be a regular ring if for all $a\in R$, $a^{2}x=a$ for
some $x\in R$.

###### Lemma 2.6

[22] A commutative ring $R$ with unity is regular if and only if every simple
$R$-module is injective.

Now, we consider singular simple $R$-modules (ideals) which are injective, and
obtain some properties of essential ideals corresponding to submodules
generated by elements of $\widehat{A_{f}(M)}$.

###### Theorem 2.7

Let $M$ be a $R$-module with $essoc(M)\neq 0$ and $\bigcap\limits_{0\neq x\in
M}Rx\neq 0$. Then every singular simple $R$-module $[x:M]$,
$x\in\widehat{A_{f}(M)}$ is injective if and only if $Z(R)=0$ and
$rad(R/[x:M])=0$.

Proof. We have $essoc(M)\neq 0$ and $\bigcap\limits_{0\neq x\in M}Rx\neq 0$,
so that $\widehat{A_{f}(M)}\neq\emptyset$. Therefore corresponding to every
cyclic submodule generated by elements of $\widehat{A_{f}(M)}$, we have an
ideal in $R$. For $x\in\widehat{A_{f}(M)}$, suppose all singular simple
$R$-modules $[x:M]$ are injective. If for some $z\in\widehat{A_{f}(M)}$,
$I=[z:M]\subseteq Z(R)$ is a simple $R$-module, then $Z(I)=I$. This implies
that $I$ is injective and thus a direct summand of $R$. However, the set
$Z(R)$ is free from nonzero idempotent elements. Therefore, $I=0$ and so
$Z(R)=0$. For $x\in\widehat{A_{f}(M)}$, clearly $A=[x:M]$ is an essential
ideal of $R$. Thus, by Lemma 2.5, $R/A$ is a singular module and so is every
submodule of $R/A$. Therefore every simple submodule of $R/A$ is injective,
which implies that every simple submodule is excluded by some maximal
submodule. Thus we conclude that $rad(R/A)=0$.

For the converse, we again consider the correspondence of cyclic submodules of
$M$ and ideals of $R$. Let $\tilde{I}$ be a singular simple $R$-module
corresponding to the submodule of $M$. In order to show that $\tilde{I}$ is
injective, we must show that for every essential ideal $A$ in $R$
corresponding to the submodule determined by an element
$x\in\widehat{A_{f}(M)}$, every $\varphi\in Hom_{R}(A,\tilde{I})$ has a lift
$\psi\in Hom_{R}(R,\tilde{I})$ such that the following diagram commutes.

${A}$${R}$${\tilde{I}}$$\scriptstyle{i}$$\scriptstyle{\varphi}$$\scriptstyle{\psi}$

Let $K=ker(\varphi)$. We claim that $K$ is an essential ideal of $R$. For, if
$K\cap J=\\{0\\}$, for some nonzero ideal $J$ of $R$, then $I^{*}=J\cap A\neq
0$ and $I^{*}\cap K=\\{0\\}$. This implies that
$I^{*}\subseteq\varphi(I^{*})\subseteq\tilde{I}$, a contradiction, since
$\tilde{I}$ is a singular simple submodule and $Z(R)=0$. For $\mu\neq 0$, it
is clear that $\varphi$ induces an isomorphism
$\mu:A/K\longrightarrow\tilde{I}$. So, $A/K$ is a simple $R$-submodule of
$R/K$. By our assumption, $rad(R/K)=0$, so there is a maximal submodule $M/K$
such that $R/K=A/K\oplus M/K$. Let $g:R\longrightarrow R/K$ be a canonical map
and let $p:R/K\longrightarrow A/K$ be a projection map. Then, we have
$pg:R\longrightarrow A/K$. Therefore the composition $\psi=\mu
pg:R\longrightarrow\tilde{I}$ is the required lift such that the above diagram
commutes.

Now, we discuss some interesting consequences of the preceding theorem.

###### Theorem 2.8

Let $M$ be an $R$-module with $essoc(M)\neq 0$, $\bigcap\limits_{0\neq x\in
M}Rx\neq 0$ and for $x\in\widehat{A_{f}(M)}$, let every singular simple
$R$-module $[x:M]$ be injective. Then every ideal $[x:M]$ is an intersection
of maximal ideals, $J(R)^{2}=0$ and $[x:M]^{2}=[x:M]$.

Proof. For any $x\in\widehat{A_{f}(M)}$, clearly $[x:M]$ is an essential ideal
in $R$. Therefore, $J(R)\subseteq[x:M]$, since $J(R)$ is contained in every
essential ideal of $R$. On the other hand, intersection of all essential
ideals in $R$ is Socle of $R$, therefore $J(R)\subseteq Soc(R)$. This implies
that $J(R)^{2}=0$ and $[x:M]$ is the intersection of maximal ideals in $R$.
Suppose that $[x:M]^{2}\neq[x:M]$, for an essential ideal $[x:M]$ of $R$. By
Theorem 2.7, $Z(R)=0$ and therefore for every essential ideal $I$, we have
$I\subseteq I^{2}$. In particular, $[x:M]\subseteq[x:M]^{2}$ for each
$x\in\widehat{A_{f}(M)}$. It follows that $[x:M]^{2}$ is an essential ideal
and is the intersection of maximal ideals in $R$. Finally, if $y\in[x:M]^{2}$,
$y\notin[x:M]$, there is some maximal ideal $P$ of $R$ such that
$[x:M]\subseteq P$, $y\notin P$. Then $R=Ry+P$, that is, $1=ry+m$. This
implies that $y=yry+ym\in P$, a contradiction. Hence we conclude that
$[x:M]^{2}=[x:M]$.

###### Corollary 2.9

Let $M$ be an $R$-module, where $R$ is hereditary. For
$x\in\widehat{A_{f}(M)}$, if $[x:M]$ is an essential ideal of $R$ and
$J(R)^{2}=0$, then every singular simple $R$-module $[x:M]$ is injective.

Proof. Let $R$ be hereditary. From [6], the exact sequence

$0\longrightarrow ann(x)\longrightarrow R\longrightarrow Rx\longrightarrow 0$

splits for any $x\in R$. Since $J(R)^{2}=0$ and $R/J(R)$ is an artinian ring,
therefore $J(R)\subseteq Soc(R)$. But any essential ideal of $R$ contains
$Soc(R)$. So, $J(R)\subseteq[x:M]$. This implies that $R/[x:M]$ is a
completely reducible $R$-module and therefore $rad(R/[x:M])=0$. Thus, by
Theorem 2.7, every singular simple $R$-module $[x:M]$ is injective.

Next, we consider the modules over regular rings.

###### Theorem 2.10

Let $M$ be an $R$-module such that every submodule of $M$ is cyclic over $R$
and $\bigcap\limits_{0\neq x\in M}Rx\neq 0$. The following are equivalent.

(i) $R$ is regular

(ii) $A^{2}=A$ for each ideal $A$ of $R$

(iii) $[x:M]^{2}=[x:M]$ for each $x\in\widehat{A_{f}(M)}$

Proof. The equivalence of (i) and (ii) is clear and certainly (ii) implies
(iii). Thus, we just need to show that (iii) implies (ii). By Theorem 2.7,
$[x:M]$ is an essential ideal for each $x\in\widehat{A_{f}(M)}$. Suppose
$[x:M]^{2}=[x:M]$. Choose $J$ to be maximal ideal of $R$ such that $A\cap
J=0$, where $A$ is some non essential ideal of $R$. Then $A+J$ is an essential
ideal of $R$. Therefore again by Theorem 2.7, $A+J$ corresponds to some
submodule of $M$ and we have $A+J=[z:M]$ for some $z\in M$. So,
$(A+J)^{2}=A^{2}+J^{2}=A+J$. If $x\in A$, then
$x=\sum\limits_{finite}ab\leavevmode\nobreak\ +\sum\limits_{finite}mn$, where
$a,b\in A$ and $m,n\in J$. Therefore,

$x-\sum\limits_{finite}ab=\sum\limits_{finite}mn\in A\cap J=0.$

This implies that $x\in A^{2}$ and we conclude that $A=A^{2}$.

###### Corollary 2.11

Let $M$ be an $R$-module with $essoc(M)\neq 0$ and $\bigcap\limits_{x\in
M}Rx\neq 0$. Then every singular simple $R$-module $[x:M]$, where
$x\in\widehat{A_{f}(M)}$, is injective if and only if $R$ is regular.

Proof. By Theorem 2.8, if every singular simple $R$-module $[x:M]$ is
injective, then for $x\in\widehat{A_{f}(M)}$, we have $[x:M]^{2}=[x:M]$.
Therefore, by Theorem 2.10, $R$ is regular. If $R$ is regular, then by Lemma
2.6 every simple $R$-module is injective.

##  3 Representation of essential ideals by vertices of annihilating graphs

In this section, we give a brief discussion on representation of essential
ideals by vertices of graphs realized by modules over commutative rings.

A simple $graph$ $\Gamma$ consists of a $vertex\leavevmode\nobreak\ set$
$V(\Gamma)$ and an $edge\leavevmode\nobreak\ set$ $E(\Gamma)$, where an edge
is an unordered pair of distinct vertices of $\Gamma$. One of the areas in
algebraic combinatorics introduced by Beck [5] is to study the interplay
between graph theoretical and algebraic properties of an algebraic structure.
Continuing the concept of associating a graph to an algebraic structure,
another combinatorial approach of studying commutative rings was given by
Anderson and Livingston in [1]. They associated a simple graph to a
commutative ring $R$ with unity called the zero-divisor graph denoted by
$\Gamma(R)$ with vertex set $Z^{*}(R)=Z(R)\backslash\\{0\\}$, where two
distinct vertices $x,\leavevmode\nobreak\ y\in Z^{*}(R)$ are adjacent in
$\Gamma(R)$ if and only if $xy=0$. The study of graph theoretical parameters
and spectral properties in zero-divisor graphs of commutative rings are
explored in [1, 2, 14, 15, 16, 18]. In [1, 18], authors have discussed
chromatic number, clique number and metric dimensions of zero-divisor graphs
associated with finite commutative rings whereas [14, 16] are related to eigen
values and Laplacian eigen values of zero-divisor graphs associated to finite
commutative rings of type $\mathbb{Z}_{n}$ for $n=p^{N_{1}}q^{N_{2}}$, where
$p<q$ are primes and $N_{1},N_{2}$ are positive integers. The extension of
zero-divisor graphs to non-commutative rings and semigroups can be found in
[12, 21].

The combinatorial properties of zero-divisors discovered in [5] have also been
investigated in module theory. In [17], the authors introduced annihilating
graphs realized by modules over commutative rings known as full-annihilating,
semi-annihilating and star-annihilating graphs, denoted by
$ann_{f}(\Gamma(M))$, $ann_{s}(\Gamma(M))$ and $ann_{t}(\Gamma(M))$. The
vertices of annihilating graphs are elements of sets $\widehat{A_{f}(M)}$,
$\widehat{A_{s}(M)}$ and $\widehat{A_{t}(M)}$ respectively, where two vertices
$x$ and $y$ are adjacent if and only if $[x:M][y:M]M=0$. The three simple
graphs: full-annihilating, semi-annihilating and star-annihilating with vertex
sets: $\widehat{A_{f}(M)},\widehat{A_{s}(M)},\widehat{A_{t}(M)}$ are natural
generalizations of the zero-divisor graph introduced in [1]. This concept was
further studied in [19].

We call a vertex $x$, an essential vertex in $ann_{f}(\Gamma(M))$ if the ideal
represented by $x$ is essential in $R$. Recall that a graph $\Gamma$ is said
to be a complete if there is an edge between every pair of distinct vertices.

By Definition 1.1, we see the containment $ann_{t}(\Gamma(M))\subseteq
ann_{s}(\Gamma(M))\subseteq ann_{f}(\Gamma(M))$ as induced subgraphs of the
graph $ann_{f}(\Gamma(M))$, since $A_{t}(M)\subseteq A_{s}(M)\subseteq
A_{f}(M)$. If $ann_{f}(\Gamma(M))$ is a finite graph, then by [[17], Theorem
3.3 and Example 2.2], $|\widehat{A_{f}(M)}|=|\widehat{A_{s}(M)}|$ and
annihilating graphs $ann_{f}(\Gamma(M))$, $ann_{s}(\Gamma(M))$ coincide,
whereas the graph $ann_{t}(\Gamma(M))$ with vertex set $\widehat{A_{t}(M)}$
may be different. For a $\mathbb{Z}$-module
$M=\mathbb{Z}\oplus\cdots\oplus\mathbb{Z}$, we have by Definition 1.1
$[x:M][y:M]M=0$ for all $x,y\in\widehat{A_{f}(M)}$. Therefore
$ann_{f}(\Gamma(M))$ is a complete graph whereas the graph
$ann_{s}(\Gamma(M))$ is an empty graph. Thus for finitely generated infinite
modules, graphs $ann_{f}(\Gamma(M))$ and $ann_{s}(\Gamma(M))$ are different.

As discussed in Section 2, for a module
$M=\mathbb{Z}\oplus\cdots\oplus\mathbb{Z}$, the ideal $[x:M]$ represented by a
vertex $x\in\widehat{A_{f}(M)}$ of the graph $ann_{f}(\Gamma(M))$ is not an
essential ideal. So, $x$ is not an essential vertex of the graph
$ann_{f}(\Gamma(M))$. On the other hand, every vertex of a $\mathbb{Z}$-module
$\mathbb{Z}_{p}\oplus\mathbb{Z}_{q}$ is an essential vertex of the graph
$ann_{f}(\Gamma(\mathbb{Z}_{p}\oplus\mathbb{Z}_{q}))$, where $p$ and $q$ are
any two primes.

Finally, Problem 2.4 can be restated in the graph theoretical version as
follows.

###### Problem 3.1

Characterize all annihilating graphs realized by a module $M$ such that every
vertex $x\in\widehat{A_{f}(M_{N})}$ of an annihilating graph is an essential
vertex.

Conclusion: In this paper, we formulated a new approach of recognition of
essential ideals in a commutative ring $R$. This formulation of essential
ideals corresponds to mod-annihilators of a $R$-module $M$. It is interesting
to characterize essential ideals such that their arbitrary intersection is an
essential ideal, since it is specified in [4] that an arbitrary intersection
of essential ideals may not be an essential ideal. Furthermore, we obtained
the results related to ideals $[x:M]$ of $R$, where $x$ is a mod-annihilator
of $M$ and discussed the representation of vertices of annihilating graphs by
essential ideals of $R$. Apart from the research problems which we mentioned
in Sections 2 and 3, the following problems could be investigated for the
future work.

1. If $G$ is a finite abelian $p$-group (viewed as a finite $\mathbb{Z}$-module) of rank at least $3$. Determine value of $n$ for the essential ideal $[x:G]=n\mathbb{Z}$, where $x\in G$.

2. If $G$ is any finite abelian group (viewed as a finite $\mathbb{Z}$-module). Determine value of $n$ for the essential ideal $[x:G]=n\mathbb{Z}$, where $x\in G$.

## References

  * [1] D. F. Anderson and P. S. Livingston, The zero-divisor graph of a commutative ring, J. Algebra 217 (1999), 434 - 447.
  * [2] D. F. Anderson, R. Levy and J. Shapiro, Zero-divisor graphs, von Neumann regular rings, and Boolean algebras. J. Pure Appl. Algebra 180 (2003) 221 - 241.
  * [3] F. Azarpanah, Essential ideals in C(X), Period. Math. Hungar. 31 (1995) 105 - 112.
  * [4] F. Azarpanah, Intersection of essential ideals in C(X), Proc. Amer. Math. Soc. 125 (1997) 2149 - 2154.
  * [5] I. Beck, Coloring of commutative rings, J. Algebra 116 (1988) 208 - 226.
  * [6] H. Cartan and S. Eilenberg, _Homological algebra_ , Princeton, (1956).
  * [7] B. W. Green and L. Van Wyk, On the small and essential ideals in certain classes of rings, J. Austral. Math. Soc. Ser. A 46 (1989) 262 - 271.
  * [8] F. Hongin and P. Stewart, Graded rings and essential ideals, Acta Math. Sinica 9, 4 (1993) 344 - 351.
  * [9] S. K. Jain, S. R. Lopez-permouth and S. T. Rizvi, Continuous rings with acc on essentials is left artinian, Proc. Amer. Math. Soc. 108, 3 (1990) 192 - 195.
  * [10] M. Kaneda and V. I. Paulsen, Characterization of essential ideals as operator modules over $C^{*}-algebras$, J. Operator Theory 49 (2003) 245 - 262.
  * [11] O. A. S. Karamzadeh and M. Rostami, On the intrinsic topology and some related ideals of C(X), Proc. Amer. Math. Soc. 93, 1 (1985) 179 - 184.
  * [12] F. D. Meyer and L. D. Meyer, Zero-divisor graphs of semigroups, J. Algebra 283 (2005) 190 - 198.
  * [13] E. Momathan, Essential ideals in rings of measurable functions. Commun. Algebra 38 (2010) 4739 - 4746.
  * [14] Bilal A. Rather, S. Pirzada, T. A. Naikoo, Y. Shang, On Laplacian eigenvalues of the zero-divisor graph associated to the ring of integers modulo n. Mathematics, 9, 482 (2021) 1 - 17.
  * [15] S. Pirzada, Bilal A. Rather and T. A. Chishti, On distance Laplacian spectrum of zero divisor graphs of $\mathbb{Z}_{n}$. Carpathian Mathematical Publications, 13, 1 (2021) 48 - 57.
  * [16] S. Pirzada, Bilal A. Wani, A. Somasundaram, On the eigenvalues of zero divisor graph associated to finite commutative ring $Z_{p^{M}q^{N}}$. AKCE International J. Graphs Combinatorics, 18, 1 (2021) 1 - 16.
  * [17] S. Pirzada and Rameez Raja, On graphs associated with modules over commutative rings, J. Korean. Math. Soc. 53, 5 (2016) 1167 - 1182.
  * [18] S. Pirzada and Rameez Raja, On the metric dimension of a zero-divisor graph, Commun. Algebra 45, 4 (2017) 1399 - 1408.
  * [19] S. Pirzada and Rameez Raja, On annihilating graphs associated with modules over commutative rings, Algebra Colloquium, to appear.
  * [20] E. R. Puczylowski, On essential extensions of rings, Bull. Austral. Math. Soc. 35 (1987) 379 - 386.
  * [21] S. P. Redmond, The zero-divisor graph of a non-commutative ring, Internat. J. Commutative Rings 1, 4 (2002) 203 - 211.
  * [22] A. Resenberg and D. Zelinsky, Finiteness of the injective hull, Math. Z. 70 (1959) 372 - 380.
  * [23] R. Wisbauer, _Foundations of Modules and Ring Theory_ , Gordon and Breach Reading (1991).